# Search engines, AI search tools and social preview crawlers are welcome:
# discovery is the point. What is not welcome is bulk collection for model
# training, and archive-scale scrapers that copy the corpus wholesale. The
# site's terms prohibit bulk copying; this is the machine-readable half of
# that. There is no rate limit behind it, by decision: see
# docs/ip-and-company-status.md.
#
# Most AI companies run TWO kinds of bot, and they are treated differently
# here on purpose (backlog L4.4, 2026-09-11). A TRAINING crawler copies pages
# into a model and never sends a reader back: blocked. A SEARCH or USER-FETCH
# bot reads a page to answer somebody's question and cites it: allowed, by
# name. The named Allow groups change nothing today, because the last group
# allows every other agent; they are there so a later edit to that wildcard
# cannot shut out the tools students now search with.
#
# Google-Extended controls training only. Google's AI Overviews are built by
# Googlebot, so blocking Google-Extended costs no Google visibility.

# Training crawlers: blocked.

User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: Applebot-Extended
Disallow: /

User-agent: meta-externalagent
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: Amazonbot
Disallow: /

# AI search and user-fetch bots: allowed.

User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: Claude-User
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Perplexity-User
Allow: /

# Everyone else, including Googlebot, Bingbot and Applebot.

User-agent: *
Allow: /

Sitemap: https://econlaunch.ca/sitemap.xml