llms.txt Explained: Does It Help You Get Cited by AI?
Every week, another post promises that adding one text file to your site will get you cited by ChatGPT, Perplexity and Google AI Overviews. If you practice answer engine optimization, you have probably seen the advice and wondered whether you are falling behind. This article separates what is documented from what is hype, and points you to the work that does move the needle.
Quick Answer: llms.txt is a community proposal, not an official standard. We found no confirmation from Google, OpenAI or Perplexity that their systems read it, and no verified evidence that it increases citations. Treat it as unproven. Clear answer-first writing, crawlable pages, visible authorship and cited sources are the practices that matter.
What llms.txt is
Answer Capsule: llms.txt is a Markdown file at your domain root that lists your most useful pages for language models. It was proposed in 2024 and is not backed by any standards body.
The idea comes from Jeremy Howard of Answer.AI, who published the proposal at llmstxt.org in September 2024. The file lives at yoursite.com/llms.txt and is written in Markdown: a title, a short summary, and sections of links with one-line descriptions. The goal is to give an AI system a clean map of your best content so it does not have to untangle navigation, ads and scripts.
It is easy to confuse with robots.txt, but the two are very different:
| Aspect | robots.txt | llms.txt |
|---|---|---|
| Status | Formal standard (RFC 9309) | Community proposal |
| Purpose | Tell crawlers what they may fetch | Point language models to useful pages |
| Crawler support | Documented by Google and others | No confirmed support from major AI platforms |
| Can it block anything? | Yes, for crawlers that comply | No, it only suggests |
The practical consequence: llms.txt cannot restrict anyone, and nothing obliges an AI system to read it.
What the evidence says about AI citations
Answer Capsule: There is no verified evidence that llms.txt influences which sources ChatGPT, Perplexity or Google AI Overviews cite.
Here is what we can and cannot say:
- Documented: The proposal exists and describes a file format at llmstxt.org. That is the primary source for the idea itself.
- Not found: We checked the crawler documentation from Google, OpenAI and Perplexity. None describes llms.txt as something its crawlers or answer systems read from your site. Absence of documentation is not proof that it is ignored, but it means any claim of a citation boost is unsupported by the vendors.
- Easily misread: OpenAI and Perplexity both publish an llms.txt for their own documentation sites. That shows they find the format handy for their docs, not that their systems consult yours.
- Unproven: Any case study claiming llms.txt raised citations. AI answers vary from query to query and day to day, so a before-and-after on one site rarely isolates a single file as the cause.
Sober recommendation: if you already have a file like this or a plugin generates one for free, it carries no known downside. Do not spend an afternoon on it ahead of improving your content, and do not buy a service that promises citations from it.
Controlling AI crawler access is a different job
Answer Capsule: robots.txt is the documented way to allow or block AI crawlers, but it controls access, not whether you get cited.
Many posts blur two separate questions: "can I get cited?" and "can I control who uses my content?" For the second, robots.txt works through vendor-documented names:
- GPTBot is an OpenAI user agent for training generative AI models. OpenAI's crawler documentation lists separate agents for search (OAI-SearchBot) and user-initiated requests (ChatGPT-User), so check which one you are targeting before you block anything.
- Google-Extended is not a separate crawler. Google's common crawlers page describes it as a standalone product token in robots.txt that controls whether crawled content may be used to train Gemini models and ground Gemini apps. It does not affect Google Search inclusion or ranking.
- PerplexityBot is Perplexity's crawler, documented in the Perplexity bots guide along with the robots.txt tags it respects.
The trade-off is yours to weigh: blocking a crawler protects your content from that use, but it can also remove you from the answers that crawler powers. Decide deliberately rather than copying a blocklist.
What actually helps you get cited
Answer Capsule: The practices most consistent with how answer engines retrieve and quote content are answer-first structure, crawlable server-rendered pages, visible authorship and dates, and cited primary sources.
None of this is a guarantee. These are practices that make a page easy to retrieve, quote and trust, which is the logic behind AEO rather than a measured formula.
- Answer first. Put a direct two-to-three sentence answer under each heading, then add the detail. A retrieval system can lift that passage cleanly. Our guide to answer engine optimization covers the structure in depth, so we will not repeat it here.
- Keep pages crawlable. Make sure the text of your article is in the server-rendered HTML, not injected by JavaScript after load, and that nothing important sits behind a login or a blocked path. Check that your robots.txt is not accidentally excluding the bots you want.
- Show who wrote it and when. Put a real author name, a short bio and a published or updated date on the page. It gives both readers and systems a reason to trust the claim.
- Cite primary sources. Link the original study, specification or vendor documentation rather than a roundup of it. Specific, sourced statements are easier to quote accurately than vague ones. Structured data can support this; see our post on schema markup for blog posts for what still helps.
- Measure instead of guessing. Test the questions your readers ask and record whether you appear. Our walkthrough on tracking whether ChatGPT, Perplexity and Google AI Overviews cite your blog shows a simple, repeatable method, which is the only honest way to tell whether any change worked.
Frequently asked questions about llms.txt
Does llms.txt help you rank higher in Google Search?
There is no verified evidence that it does. We found no Google documentation treating it as a ranking signal or indexing directive, so treat any ranking claim as unproven.
Will llms.txt get your content cited by ChatGPT or Perplexity?
Unproven. Neither company documents reading it from your site, and we found no controlled evidence of a citation effect.
Is llms.txt the same as robots.txt?
No. robots.txt is a formal standard that compliant crawlers follow. llms.txt is a proposal that only suggests pages and cannot block or permit anything.
How do I block AI crawlers?
Use robots.txt rules for the user agents each vendor documents, such as User-agent: GPTBot followed by Disallow: /. Check each vendor's page first, because training and search agents can be separate.
Should WordPress bloggers create an llms.txt file?
It is optional. If a plugin makes one in a minute, there is no known downside, but there is also no proven upside. Spend your time on content structure and sourcing first.
Conclusion
llms.txt is a reasonable idea without evidence behind it yet. Watch the proposal and the vendors' documentation for changes, and put your effort into pages that are clear, crawlable, attributed and well sourced.
Key Takeaways:
- llms.txt is a 2024 community proposal, not a standard, and it cannot block or grant access.
- We found no vendor confirmation and no verified evidence that it affects citations. Treat it as unproven.
- Use robots.txt for access control, and check each vendor's documented user agents.
- Answer-first structure, crawlable text, visible authorship and primary sources are the dependable levers.
- Track your citations over time so you know what is working.