1:98d18b9ebcb4
Anton Shestakov <av6@dwimlabs.net>, Tue, 17 Jul 2018 13:17:59 +0800
playbook, hgweb role

roles/hgweb/files/robots.txt

Permissions: -rw-r--r--

Other formats: Feeds:
User-agent: *
# Expensive pages
Disallow: /*/annotate/
Disallow: /*/comparison/
Disallow: /*/diff/
Disallow: /*/archive/
# File log is too much
Disallow: /*/log/*/
# API is off-limits
Disallow: /*/json-*
Disallow: /*/raw-*
# Paginate instead
Disallow: /*?revcount=
# Graph page is a more expensive log
Disallow: /*/graph$
Disallow: /*/graph/
# It's for humans
Disallow: /*/help$
Disallow: /*/help/