LLMs.txt File Explained: What It Is, Does It Help, and How to Create One

In 2024, a new file was proposed: llms.txt, meant to tell AI language models what to do with a website's content. It spread fast through the SEO and AEO community, and for a while it was pitched by some vendors as a must-have AI visibility lever. By mid-2026, Google had said enough about it - directly, in its own documentation - that the guessing is mostly over.

Here is the current, honest answer to every question businesses are actually asking about llms.txt in 2026: what it is, what Google's own documentation now says about it, which AI systems fetch it, what the largest adoption study found, what changed in the file's own specification this August, and whether creating one is worth your time.

What Is an LLMs.txt File and What Problem Does It Solve?

LLMs.txt is a plain text file placed at the root of a website (at yourdomain.com/llms.txt) that gives large language models a structured, Markdown summary of the site's content. It was originally proposed by Jeremy Howard of Answer.AI in September 2024, and the specification is still maintained at llmstxt.org.

The problem it was designed to solve is real: web pages built for humans are full of navigation, ads, and layout markup that waste an AI agent's limited context window when it just needs the substance of a page. Llms.txt was originally proposed to give agents "concise, expert-level information gathered in a single, accessible location" instead of forcing them to parse an entire page's HTML to find it.

It draws loosely on two older conventions: robots.txt, which tells search engine crawlers which pages to index, and humans.txt, an informal file for crediting the people behind a site. Llms.txt borrows the "file at the root" idea from both but is not a directive file like robots.txt - it does not block or allow anything. It is closer to a curated table of contents.

Does Google Search Use LLMs.txt for Ranking or AI Overviews?

No, and Google has now said so directly, not through a forum reply. Google's Search Central documentation was updated in June 2026 specifically to close this question, and its Guide to Optimizing for Generative AI Features on Google Search now states plainly:

"You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them."

The same guidance goes further, addressing llms.txt by name: creating and maintaining an llms.txt file "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them." Google logged this as a dedicated documentation change titled "Clarifying guidance on llms.txt files" in its Search Central changelog, dated June 15, 2026, stating the note was added "to address questions from the community."

This is not a policy reversal. Google Search has never used llms.txt as a ranking or AI Overview/AI Mode input - the June 2026 update simply put that answer in writing, in the primary source, instead of leaving it to a Google employee's forum comment. Any article, including our own earlier coverage of this topic, that implies llms.txt might carry indirect Google Search benefit is now out of step with Google's own documentation.

Which AI Systems Actually Fetch and Use LLMs.txt Today?

Google Search is one part of the AI landscape, not all of it. Here is the honest, system-by-system picture as of August 2026:

• Google Search / AI Overviews / AI Mode: Does not use it, per Google's own June-July 2026 documentation above.

• Perplexity: Publishes its own llms.txt for its developer documentation, and third-party crawl-log analyses report PerplexityBot fetching llms.txt files more often than other AI crawlers do. Perplexity has not made a public policy statement confirming that citation ranking is affected by it.

• Anthropic (Claude) and OpenAI: Both companies now publish llms.txt files for their own developer docs, and Google's Gemini team does the same for its documentation. That confirms the format is useful for developer-facing documentation sites - it does not confirm that ChatGPT or Claude's consumer products fetch and weight a third party's llms.txt when answering a user's question.

• Coding agents (Cursor, GitHub Copilot, and similar tools): These are the clearest real adopters. Agentic coding tools use llms.txt files to orient themselves inside a codebase's or library's documentation, which is closer to the original use case the specification was written for.

The pattern across all of this: llms.txt has real, demonstrated use as a documentation-navigation aid for agentic tools and retrieval systems that choose to fetch it. It has no demonstrated use as a lever that changes whether a general-purpose AI chatbot cites your marketing or content pages more often.

How Widely Adopted Is LLMs.txt Across the Web in 2026, Including Among High-Traffic Sites?

The largest independent study to date is worth citing directly. SE Ranking analyzed roughly 300,000 domains in 2026 and found llms.txt adoption at 10.13 percent - meaning close to nine in ten sites still had not implemented it. The analysis, reported by Search Engine Journal in November 2025, found no statistically significant relationship between having an llms.txt file and how often a domain was cited by major AI systems, after controlling for site authority and other confounders.

Does an LLMs.txt File Measurably Change AI Citation Rates, Based on Actual Studies?

That matches independent testing from individual publishers. Semrush's own analysis cites a three-month test by Search Engine Land, which logged zero visits to its llms.txt file from Google-Extended, GPTBot, PerplexityBot, or ClaudeBot over the test window.

Two studies and one primary-source statement from Google is not proof that llms.txt can never matter anywhere. It is enough evidence that nobody should implement llms.txt expecting a measurable citation or ranking lift, and any content claiming otherwise is not reflecting the current data.

How Is LLMs.txt Different From robots.txt? Format, Enforcement, and Consequences

The critical difference is enforcement and consequence. Ignore robots.txt disallow rules as a crawler operator and you risk being blocked outright. Ignore llms.txt as an AI system and nothing happens, because there was never an obligation to read it in the first place. Both files can coexist on the same site without conflict.

What Changed in the LLMs.txt v2 Specification Released in August 2026?

The original 2024 format got its first real revision on August 10, 2026, when Jeremy Howard published llms.txt v2. If your site's llms.txt file (or an article describing it) still only reflects the original 2024 structure, it is describing a format that is now a year and a half out of date. The main changes:

• A formal way for agents to discover a Markdown version of any page, either by adding .md to the URL (/docs/tutorial.html.md) or replacing the extension (/docs/tutorial.md)

• Standard link relations - rel="alternate" type="text/markdown" and rel="describedby" - so an agent can find the Markdown version or the relevant llms.txt file from an HTML tag or an HTTP response header, without a template rebuild

• The "context expansion" idea from v1, where the file's Optional section carried special meaning for bundling extra content, was dropped. Agents are now expected to view the llms.txt index, then follow the specific links they need.

Does Upgrading to v2 Affect Existing Files?

For a site that already has a v1 file, adopting v2 is a minor edit, not a rebuild: point your key pages at Markdown equivalents where you have them, and add the link relations if your CMS supports custom tags or headers.

Why Does Chrome Lighthouse Check for LLMs.txt If Google Search Doesn't Use It?

This is the one genuinely confusing part of Google's current position, and it deserves a direct answer instead of being glossed over: Google Search and Google Chrome are different products making different claims.

Chrome's Lighthouse tool added an Agentic Browsing category, and it includes an llms.txt audit. Per Chrome's own developer documentation, the audit flags a server error when Lighthouse tries to retrieve the file; if the file simply does not exist (a 404), the audit is marked "not applicable," because providing the file is explicitly optional. Lighthouse is not scoring your site's Google Search ranking here - Agentic Browsing does not even produce a weighted 0-100 score like Lighthouse's other categories, because Google says the standards for the agentic web are "still emerging" and the goal right now is gathering data, not issuing a verdict.

So the honest reconciliation is this: Google Search (the ranking system) confirms it ignores llms.txt. Google Chrome (the browser, via Lighthouse) checks whether the file exists as a readiness signal for AI browsing agents and coding tools - a separate, narrower concern about whether autonomous agents can navigate your site efficiently, not about whether your content ranks or gets cited. Both statements from Google are true at the same time; they are just about different products.

How Do You Create an LLMs.txt File, and Are There Tools That Generate One Automatically?

If you decide the agent-tooling case applies to you - documentation sites, developer tools, or products that expect to be used by coding agents - creating a basic v2-aligned file takes under an hour.

Step 1: Create the file

Create a plain text file named llms.txt and place it at your site's root, accessible at https://yourdomain.com/llms.txt. On WordPress, upload it via FTP, the file manager, or a plugin. On Webflow or Shopify, add it as a static or theme file. On Next.js or custom sites, place it in /public.

Step 2: Write the content in Markdown

# [Your Brand Name]

> [One to two sentence description of what your site/business is]

## [Section 1: e.g., Core Pages]

- [Page Title](URL): Brief description of what this page contains
- [Page Title](URL): Brief description

## [Section 2: e.g., Docs / Resources]

- [Page Title](URL): Brief description

## Optional

- [Markdown version](URL.md): Full page content in clean Markdown

Prioritize your most important, highest-quality pages - documentation, API references, and pages you would want an agent to treat as authoritative.

Step 3: Point to Markdown equivalents where you have them

If your CMS or static site generator can output a .md version of a page alongside the HTML, link to it using the v2 link-relation pattern. This is the single highest-leverage change from the August 2026 spec update.

Step 4: Keep it current

An out-of-date llms.txt pointing at moved or removed pages is worse than no file at all. Review it whenever you significantly restructure the site or ship major new documentation.

Is LLMs.txt Worth Creating in 2026? The Practical Verdict

Be honest about what llms.txt is and is not. It is not an SEO or AI-visibility lever - Google's own documentation confirms Search ignores it, and the largest adoption study to date found no measurable citation effect from it. It is a low-cost, optional convenience file that some agentic coding tools and documentation-retrieval systems can use to navigate a site faster, and Chrome now checks for it as a signal of "agent readiness," separate from search ranking entirely.

That means the honest recommendation depends on what kind of site you run. If you operate developer documentation, an API, or a product that coding agents are likely to interact with, an llms.txt file costs under an hour to create and has a plausible, if unproven, upside. If you run a marketing site, an ecommerce store, or a content site chasing AI Overview or chatbot citations, your time is better spent on the things Google's own guidance and the E-E-A-T literature actually tie to AI visibility: comprehensive topic coverage, real outbound citations, and content structured around the direct-answer sub-questions your audience is actually asking - not a file Google has explicitly said it ignores.

Do not take our word, Semrush's word, or SE Ranking's word for what AI crawlers do on your specific site - check it. AI Rank Lab's bot-tracking middleware detects real requests from GPTBot, ClaudeBot, Gemini, Googlebot/Google-Extended, CCBot, and PerplexityBot by user agent, plus AI-referrer-based citation events from ChatGPT, Claude, Gemini, Bing, and Perplexity, so you can see whether adding or changing an llms.txt file actually changes crawler behavior on your own domain rather than relying on someone else's study. If you want to create a v2-aligned file without hand-writing it, AI Rank Lab's llms.txt generator crawls your site and produces one automatically, and the llms.txt checker validates an existing file against the current spec.