llms.txt, explained
2026-05-12
llms.txt is a proposed convention: a markdown file at the root of a site that tells language-model systems what the site is and where its most useful content lives. Where robots.txt says what crawlers may not do, llms.txt says what they might want.
The idea is curation. A model assembling an answer benefits from a clean index of canonical pages more than from a raw crawl of navigation chrome, tag archives, and pagination. The file is an offer of the good stuff.
Adoption is real but uneven, and no major operator has committed to honoring it as policy. It costs a few minutes to publish, which is why sites that care about how they appear in AI answers tend to have one anyway.