EN
Book a call

Technology

Open Knowledge Format (OKF) vs llms.txt: Two Ways to Make Your Site Agent-Ready in 2026

Open Knowledge Format (OKF) vs llms.txt: Google's structured markdown-plus-YAML knowledge bundles versus the community llms.txt index for AI agents. Compare structure, maturity, setup, vendor independence, real-world crawler pickup and when each wins in 2026.

Reviewed by Michael Kerkhoff, as of

Definition
The web is quietly growing a second layer written for machines, and in 2026 two formats define how you publish it. llms.txt, introduced by Jeremy Howard of Answer.AI in September 2024, is a single markdown file at your domain root that hands AI systems a curated index of your most important pages with one-line descriptions. Open Knowledge Format (OKF), published by Google Cloud on 12 June 2026, goes one layer deeper: instead of one index file it standardizes a whole directory of markdown files, each carrying a small block of YAML front matter that types the content, titles it, describes it and links to its neighbours. Google calls OKF a vendor-neutral, agent- and human-friendly way to package curated context — no compression scheme, no runtime, no required SDK. So the question is not really 'which one wins.' It is which layer you need: a flat, zero-friction discovery surface that ships in minutes (llms.txt), or a structured, navigable knowledge bundle an agent can map before it reads a word (OKF). This comparison weighs them on structure, maturity, setup friction, scope, vendor independence, agent navigability, real-world crawler pickup and human readability — so you can decide what to ship, and whether you should ship both.
Category
Technology
Options
Open Knowledge Format (OKF)Llms.txt

Detailed Comparison

A side-by-side analysis of key factors to help you make the right choice.

Open Knowledge Format (OKF) vs Llms.txt
FactorOpen Knowledge Format (OKF)Llms.txt
Structure & metadata granularityA whole directory of markdown files, each with typed YAML front matter (type, title, description, tags, links) — granular, machine-typed context WinnerA single flat file: a curated list of links with one-line descriptions, with an optional llms-full.txt that embeds the content
Adoption & maturityPublished by Google Cloud on 12 June 2026 — days old at launch, with tooling and conventions still settlingLive since September 2024 with 20+ months of community adoption, generators and real-world examples Winner
Setup frictionAuthor a directory of typed markdown files plus an index.md so agents see the bundle's structure firstA single text file at your domain root — a non-engineer can ship it in minutes with nothing to install Winner
Scope & depthBuilt for curated knowledge bundles and enterprise agent context — a portable knowledge base, not just a pointer list WinnerA site discovery surface — an index of your key pages, not a structured body of knowledge
Vendor independenceA genuinely portable spec, but it originated from and is stewarded by Google Cloud (the open piece of its Knowledge Catalog launch)A community proposal from Jeremy Howard at Answer.AI, with no single vendor steering the standard Winner
Agent navigabilityTyped front matter and linked neighbours let an agent map a whole bundle before opening any single file — no scraping, no API WinnerAn agent gets a flat list of links, then still has to fetch and parse each target page to learn anything
Real-world crawler pickupToo new to measure — published in June 2026, with no field data on uptake yetEmpirically thin today: Limy's 500M+ bot-event study found GPTBot, ClaudeBot, PerplexityBot and others overwhelmingly skip it
Human readabilityPlain markdown in any editor, renderable on GitHub, indexable by any search tool — readable to people and machines alikePlain markdown too — a short, human-readable index anyone can open and edit in seconds
Total Score · 2 ties3 / 83 / 8

Key Statistics

Real data from verified industry sources to support your decision.

  • Open Knowledge Format launched on 12 June 2026 as a Google Cloud open spec: OKF v0.1 is a directory of markdown files with YAML front matter, with no compression scheme, no runtime and no required SDK — Google Cloud Blog (2026)
  • llms.txt was introduced in September 2024 by Jeremy Howard of Answer.AI as a curated markdown index at a site's root — roughly 21 months before OKF, giving it a long maturity head start — Limy (2026)
  • Limy analysed 500M+ LLM bot traffic events in May 2026 and found GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot and Google-Extended overwhelmingly skip /llms.txt and crawl HTML directly — Limy (2026)
  • OKF needs no registry and nothing to install; the spec fits on a single page, and an index.md lets an agent see a bundle's structure before opening every file — Suganthan Mohanadasan (2026)
  • OKF is the portable, open piece of Google's Dataplex-to-Knowledge-Catalog rebrand, repositioned as an 'always-on context engine' for AI agents — Suganthan Mohanadasan (2026)
  • llms.txt ships in two variants: llms.txt (a compact index with links) and llms-full.txt (which embeds full page content so an agent can ingest everything in a single fetch) — Fern (2026)

All statistics come from verified third-party sources. Source, year, and direct link are shown on each metric.

When to Choose Each Option

Clear guidance based on your specific situation and needs.

Our Recommendation

These are complementary layers, not rivals, and the honest 2026 answer is usually 'ship both.' Reach for llms.txt first if you run a public website: it is a single text file at your root, a non-engineer can publish it in minutes, it has 20+ months of community tooling behind it, and it is the established way to expose a curated index to agents. Reach for OKF when a flat index is not enough — when you are packaging curated internal or enterprise knowledge that agents must navigate by type and metadata before reading, when producers and consumers need to interoperate without translation, or when you want a portable bundle agents read as-is with no scraping and no API in between. The honesty caveat applies to both: neither is reliably fetched in the wild yet. Limy's analysis of 500M+ LLM bot events in May 2026 found the major crawlers overwhelmingly skip /llms.txt and read HTML directly, and OKF is only days old. So treat both as forward infrastructure bets, not traffic guarantees. The framing Context Studios uses with clients is a dual-format discoverability strategy: llms.txt as the public front door for any site, OKF for the deeper, typed knowledge bundles your agents will actually reason over — shipped now, so you are already there the day a major answer engine flips the switch.

Choose Open Knowledge Format (OKF) when...
  • You are packaging curated internal or enterprise knowledge for agents, not just indexing a public site
  • You need typed, per-document metadata an agent can navigate before it reads anything
  • You want a portable bundle agents consume as-is, with no scraping and no API in the way
  • Producers and consumers across teams must interoperate on the same context without translation
Choose Llms.txt when...
  • You want a zero-friction, single-file surface live at your domain root today
  • Your priority is public-website AI discoverability, not an internal knowledge base
  • You want the established community standard with 20+ months of tooling and examples
  • A non-engineer needs to ship it in minutes, with no structure to design and nothing to install

Common questions about this comparison answered.

Frequently Asked Questions

(01)Is OKF replacing llms.txt?
No. They operate at different layers. llms.txt is a flat index at your site root that points agents to your key pages; OKF packages a structured, typed knowledge bundle agents can navigate before reading. They coexist comfortably — many sites should publish both, llms.txt as the public index and OKF for deeper knowledge.
(02)Do AI crawlers actually fetch llms.txt yet?
Mostly not. Limy analysed 500M+ LLM bot events in May 2026 and found GPTBot, ClaudeBot, PerplexityBot and others overwhelmingly skip /llms.txt and crawl HTML directly. It is a forward Business-to-Agent infrastructure bet worth shipping for the day that changes — not a current traffic driver.
(03)Is OKF a Google lock-in?
The format itself is plain markdown plus YAML with no SDK, registry or runtime, and Google calls it vendor-neutral. But it originated from Google Cloud's Knowledge Catalog (formerly Dataplex) rebrand, so the spec is portable while the surrounding product is not. You can adopt OKF without touching Google Cloud.
(04)Which should I implement first?
If you run a public website, ship llms.txt now — it is a single trivial file. If you are handing curated knowledge to enterprise agents, adopt OKF for its structure and typed metadata. In 2026 the strongest answer is a dual-format strategy that ships both, since neither is reliably crawled yet and the cost of being early is low.

Need help deciding?

Book a free 30-minute consultation and we'll help you determine the best approach for your specific project.

Book Free ConsultationEmail Us

Free consultation · No obligation · Personal reply