A plain text file, sitting at the root of a website, listing out the pages a business thinks matter most. That’s the entire concept behind llms.txt, and the amount of debate it’s generated over the past two years is genuinely out of proportion to how simple the idea is.
It’s worth being straightforward about where this stands in 2026, because the discourse around llms.txt has split into two camps that don’t talk to each other much: people insisting it’s essential AI-search infrastructure, and people insisting it’s a waste of time. The honest answer sits in between, and which camp is right depends entirely on what kind of website you run.
What the File Actually Contains
An llms.txt file is a short, structured Markdown document — typically a page name, a one-line summary of what the site does, and a curated list of links to the pages considered most important, organised under simple headings. There’s also a variant, llms-full.txt, which goes further and includes the actual content of those pages inlined into the file, rather than just linking out to them.
The idea behind both versions is straightforward. When an AI system fetches a normal webpage, it has to work through navigation menus, cookie banners, footer links, and script tags to find the content that actually matters. llms.txt is meant to skip that step — a clean, curated shortcut straight to what the site owner considers essential.
This Is Not the Same Job as robots.txt or a Sitemap
It’s worth clearing up a common confusion early, because the three get lumped together constantly. Robots.txt is a set of rules telling crawlers which parts of a site they’re allowed to access — it’s permission, not curation. A sitemap is a comprehensive list of URLs meant to help a search engine discover everything that exists on a site. llms.txt does neither of those jobs. It’s a curated recommendation — “here’s what actually matters, out of everything you could crawl” — and reading it is entirely voluntary for whichever system encounters it. No standard requires any AI tool to look at it, use it, or respect it.
The Part Most Articles Get Wrong: Whether Anyone’s Actually Reading It
This is where the picture gets genuinely interesting, because the honest answer is different depending on which system you’re asking about.
Google has said, directly and more than once, that it doesn’t use llms.txt for Search rankings or for its AI Overviews and AI Mode features. This isn’t a guess based on absence of evidence — Google’s own Search Relations team has stated it on the record, and Google’s published guidance for site owners on optimising for AI search explicitly lists creating special AI text files as something that doesn’t help. If the entire reason a business is considering llms.txt is to rank better on Google or show up more in AI Overviews, that reason doesn’t hold up.
The picture for other AI systems is murkier and more mixed. Anthropic and Perplexity have both indicated, in various forms, that they make some use of llms.txt when it’s present, particularly Perplexity for brand-level lookups. OpenAI has never officially confirmed that ChatGPT reads or prioritises the file, and independent monitoring of AI crawler traffic has found that dedicated requests specifically for llms.txt are a tiny fraction of overall AI bot activity — most crawlers, including the big search-and-answer bots, are still fetching and parsing ordinary HTML rather than checking for a curated manifest first.
Adoption across the web sits somewhere around one in ten sites, based on large-scale domain studies, and that figure hasn’t moved dramatically despite nearly two years of industry conversation about it. That’s not a sign of an emerging standard about to tip into universal adoption. It’s a sign of a genuinely useful but niche convention that’s found its real audience elsewhere.
Where llms.txt Is Actually Earning Its Keep
The clearest, best-evidenced use case has nothing to do with search visibility at all. Coding assistants and AI development tools — the kind developers point at a documentation site while building something — do fetch and use llms.txt routinely. Tools in that category are explicitly built to look for it, because when an AI agent needs to understand a software library or API quickly, a clean, curated index is exactly the shortcut it’s designed to exploit.
This matters for a specific kind of business: software companies, API providers, and anyone running documentation that developers or AI coding agents are likely to reference directly. For that audience, llms.txt isn’t a marketing tactic — it’s closer to accessibility infrastructure, making a technical resource easier for a tool to use correctly.
For most ordinary businesses — a local service provider, a retailer, a professional services firm — that use case simply doesn’t apply. Nobody’s pointing a coding agent at a plumbing company’s website to build software against it.
The Honest Cost-Benefit
Building a basic llms.txt file is genuinely low effort — an hour or so for most small sites, sometimes automated entirely by a WordPress plugin or theme. At that cost, there’s a reasonable argument for creating one anyway, on the basis that it’s cheap insurance against a standard that might matter more in a year or two, particularly if Google’s position shifts, which as of now shows no sign of happening.
Where it becomes a genuine mistake is when a business treats it as a substitute for the work that actually moves visibility — clear, well-structured content that directly answers real questions, accurate and consistent business information across the web, valid schema markup, and a site that’s technically fast and easy to crawl. Several agencies have reported seeing llms.txt sold as the headline deliverable in AI-visibility retainers, sometimes at a price that implies far more work than an hour of file creation. That’s worth being sceptical of regardless of who’s selling it.
There’s also a specific technical trap worth flagging. Some guidance around llms-full.txt encourages publishing a full duplicate copy of every page’s content inside the file, in a simplified format, on the theory that it makes life easier for AI systems. If that duplicated content ends up visible to Google as well, it creates two versions of the same page competing with each other, which can actively confuse indexing rather than help it. Anyone implementing llms-full.txt should keep it genuinely separate from what search engines index, not accidentally create a second copy of the entire site.
Where This Leaves a Business Deciding Whether to Bother
For most website owners, the sensible position isn’t “essential” or “waste of time” — it’s “cheap enough that it doesn’t matter much either way, but not worth spending a real budget on.” A basic file costs almost nothing to produce and does no harm. Expecting it to move rankings or AI citation on its own is the mistake, not the file itself.
The work that reliably does move visibility — across Google, ChatGPT, Perplexity, and everywhere else — remains the same work it’s always been: content specific enough to be worth citing, technical foundations solid enough that crawlers and AI systems can actually read the page cleanly, and consistent, accurate information about the business wherever it appears online. llms.txt sits alongside that work as a minor, optional addition. It was never a replacement for it, whatever the more excitable corners of the SEO industry have claimed over the past two years.
Frequently Asked Questions
Does Google use llms.txt for ranking or AI Overviews? No. Google has stated this directly, and its published AI search guidance lists special AI text files among the tactics that don’t help visibility in its AI features.
Is llms.txt the same thing as robots.txt? No. Robots.txt controls crawler access — what a bot is allowed to visit. llms.txt is a curated content recommendation, and no system is required to read or follow it.
Should a small local business bother creating one? It does little harm and costs very little to set up, so there’s no real reason to avoid it, but it shouldn’t be expected to meaningfully improve visibility on its own. Time is generally better spent on content quality and technical SEO fundamentals.
Who genuinely benefits from llms-full.txt? Software companies, API providers, and documentation-heavy sites that developers or AI coding agents are likely to reference directly get the clearest benefit, since those tools are specifically built to look for and use the file.
Is llms.txt likely to become more important in the future? It’s possible, particularly if Google’s position changes or if AI agent-driven browsing becomes more common for ordinary consumer searches rather than mostly developer tools. As of now, that shift hasn’t happened, and the file remains a niche convention rather than a mainstream ranking input.
