User-agent: * Disallow: /admin/ Disallow: /cgi-bin/ Disallow: /dist/ Disallow: /includes/ Disallow: /lucene/ Disallow: /WEB-INF/