add a robots.txt

Yahoo! has been pounding on ikiwiki.cgi again. While I'd prefer ikiwiki to
generate sites that avoided robots hitting it in other ways, I'm adding a
robots.txt, at least temporarily.

diff --git a/doc/robots.txt b/doc/robots.txt
new file mode 100644 (file)
index 0000000..7be87f9
--- /dev/null
+++ b/doc/robots.txt
@@ -0,0 +1,2 @@
+User-Agent: *
+Disallow: /ikiwiki.cgi