# Concresca public-corpus crawl policy, reviewed 2026-09-05. # Public coordination and research pages are crawlable. Private source, memory, # build, report, secret, and runtime-state paths are never public content. # These crawl requests are not authentication, authorization, deletion or licensing. # Private resources remain protected even when a client ignores this file. # No provider receives an authentication or rate-limit exemption for its name/IP. User-agent: * Disallow: /.uai/ Disallow: /.local-secrets/ Disallow: /agent-file-handoff/ Disallow: /data/_source/ Disallow: /data/_generated/ Disallow: /docs/source/ Disallow: /reports/ Disallow: /scripts/ Disallow: /var/ Sitemap: https://concresca.com/sitemap.xml Sitemap: https://concresca.com/sitemaps/archive-knowledge.xml