The short answer: llms.txt is a proposed convention for a markdown file at the root of a website that gives large language models a concise, curated summary of what the site contains and which pages matter. It is a proposal rather than a standard, adoption by AI vendors is not confirmed, and it costs almost nothing to publish. Treat it as cheap insurance, not as a ranking lever.

What llms.txt is

The llms.txt proposal suggests that websites publish a markdown file at their root path — the same location convention as robots.txt — containing a human-written summary of the site and a curated list of its most important pages with short descriptions. The rationale is that language models operate within limited context windows and struggle to reconstruct a site’s purpose from crawling navigation, boilerplate and marketing copy. A short curated file removes that guesswork.

The format is deliberately simple: markdown, readable by a person, parseable by a machine, maintained by hand.

llms.txt versus robots.txt versus a sitemap

File	Purpose	Format	Status
robots.txt	Tells crawlers what they may and may not fetch	Plain text directives	Long-established convention, widely honoured
sitemap.xml	Lists every URL you want discovered	XML	Established standard, supported by major search engines
llms.txt	Explains what the site is and which pages matter most	Markdown	Proposal — adoption not confirmed by major AI vendors

The distinction that matters: robots.txt is about permission, sitemap.xml is about coverage, and llms.txt is about meaning. It is the only one of the three that tries to tell a machine what your site is for.

Publishing llms.txt does not replace either of the other two. If AI crawlers are blocked in robots.txt, an llms.txt file changes nothing at all.

The format

A conforming file follows a predictable structure:

1. An H1 with the name of the site or project.
2. A blockquote immediately beneath it containing a one or two sentence summary.
3. Optionally, a short paragraph or bulleted list of additional context — what the site does, who it is for, anything a model should know when describing it.
4. One or more H2 sections grouping links, each entry formatted as a markdown link followed by a colon and a short description.
5. Optionally, a final section for material that is lower priority, so a model reading under constraint knows what it can skip.

A related convention, llms-full.txt, proposes a single file containing the full text of your key documentation rather than links to it. It is more useful for documentation sites than for marketing sites, and it is considerably harder to keep current.

A worked example

For a small tools-and-guides site, a complete file might run to twenty lines: an H1 with the site name, a blockquote explaining that the site publishes free AI visibility tools and guides, then two H2 sections — one listing the tools with a sentence each, one listing the core guides. Each description should say what the page answers, not what it is called. “Scores a page for AI citation readiness and returns a prioritized fix list” is useful; “our GEO audit page” is not.

The free llms.txt generator builds this structure for you from a few fields and outputs a file you can download and upload to your root directory.

How to add llms.txt to your site

• Static sites. Drop the file in your public or root directory and redeploy. Confirm it serves as plain text rather than triggering a download.
• WordPress. Upload the file to your web root via SFTP or your host’s file manager. Avoid plugins that generate it dynamically from your sitemap — an auto-generated list of every URL defeats the entire purpose, which is curation.
• Managed platforms. Some hosts do not allow arbitrary root files. Where that is the case, a redirect or a rewrite rule from the root path to the file location is usually acceptable.

Verify by requesting the file directly and confirming a 200 response with a text content type. Then check that every URL it references also returns a 200, since a curated file full of redirects and 404s is worse than no file.

Does llms.txt actually work?

This is where most coverage of the topic becomes unreliable, so here is the honest position.

No major AI vendor has publicly confirmed that they read llms.txt files as part of retrieval or ranking. The proposal has real traction among documentation and developer-tool sites, and it is a sensible idea on its own terms — a concise, accurate, human-curated description of a site is genuinely useful context for any system trying to summarize it. But there is a large gap between “sensible idea with growing adoption” and “confirmed ranking factor”, and a lot of content in this space quietly closes that gap without evidence.

The practical conclusion: publish one, because it takes under an hour and costs nothing to maintain. Do not attribute changes in your AI visibility to it, and do not prioritize it above crawler access, content structure or third-party corroboration. Those three are where measurable movement comes from, and they are covered in the AI visibility optimization playbook.

Common mistakes

• Auto-generating it from your sitemap. The value is curation. A machine-generated list of every URL is just a worse sitemap.
• Writing marketing copy in the descriptions. Describe what each page answers. Adjectives are wasted context.
• Letting it go stale. A file describing pages you deleted six months ago actively misinforms.
• Blocking AI crawlers while publishing an llms.txt. A common and self-defeating combination — check your robots.txt and CDN rules.
• Treating it as a substitute for schema. They do different jobs. Structured data describes individual pages to parsers; llms.txt describes the site to a reader.
• Expecting it to move your numbers alone. It will not, and building a strategy on that assumption wastes a quarter.

Frequently asked questions

What is llms.txt?

llms.txt is a proposed convention for a markdown file placed at a website’s root that gives large language models a concise summary of the site and a curated list of its most important pages. It is intended to help models understand a site without crawling and interpreting its full navigation.

Is llms.txt an official standard?

No. It is a community proposal that has gained adoption, particularly among documentation and developer-focused sites. Major AI vendors have not publicly confirmed that they consume it, so it should not be described as a standard or a confirmed ranking factor.

Where does the llms.txt file go?

At the root of your domain, using the same location convention as robots.txt. It should be served as plain text and return a 200 status code.

What is the difference between llms.txt and robots.txt?

robots.txt controls which crawlers may access which paths — it is about permission. llms.txt explains what a site contains and which pages matter — it is about meaning. Blocking AI crawlers in robots.txt makes an llms.txt file irrelevant.

Will llms.txt improve my rankings in ChatGPT?

There is no evidence that it does on its own, and no vendor confirmation that the file is read during retrieval. It is worth publishing because it is cheap and sensible, not because it can be expected to change your citation rate.

What is llms-full.txt?

A related proposal for a single file containing the full text of a site’s key documentation rather than links to it. It suits documentation-heavy sites where the goal is to make complete reference material easy for a model to ingest, and it is significantly harder to keep current than the linked version.

Next steps

Generate your file with the free llms.txt generator, upload it, verify it returns plain text, then move on to the work that measurably matters: the optimization playbook and a baseline in the LLM Rank Tracker.