llms.txt Guide: Create, Publish, and Test Your File

Want to create an llms.txt file? Start with a short description of your website, add links to your most useful pages, and publish the file where an assistant can fetch it. Then test the links and the answers it produces.

This guide walks through that process using a fictional product called ParcelKit. You can adapt the example for a documentation site, a service business, or a content library. You only need a text editor and a way to publish a file on your website.

Before you start: publishing this file does not guarantee AI citations or higher search rankings. Treat it as a small experiment in making your information easier to use.

1. What is llms.txt?

llms.txt is a proposed Markdown convention for giving AI agents a concise introduction to a website and links to useful content. The usual starting point is https://example.com/llms.txt. A file can also cover a narrower path, such as /docs/llms.txt.

The format uses a project heading, an optional summary, and sections of links. Markdown versions of linked pages are encouraged. See the llms.txt proposal for the current format and path rules.

Think about the questions someone asks about your business. Which page should they read first? What information would prevent a wrong answer? Use those questions to choose your links.

For ParcelKit, a shipping API, that might mean a quickstart, authentication instructions, and a reference for creating labels. A marketing site might start with its services, case studies, and contact information.

2. Does llms.txt help with SEO?

Do not make it the foundation of your SEO strategy. Google says that AI Overviews and AI Mode do not require special AI text files. Its guidance for AI features still emphasizes crawlability, useful content, internal links, and normal Search eligibility.

Our suggested order of work is simple:

1. Make your important pages accessible and useful.
2. Fix broken links, inaccurate information, and confusing navigation.
3. Publish a small llms.txt file if you have a clear use for it.
4. Evaluate the result with questions and evidence you can inspect.

For example, test whether an assistant can identify the correct integration guide. That is a more useful first question than asking whether a newly uploaded file has improved your rankings.

If you need help with the broader work, our AI SEO services cover the surrounding discovery and content problem.

3. llms.txt, robots.txt, and sitemap.xml

These files answer different questions. Use this table when deciding where a change belongs.

File	Main question	Example task
robots.txt	What crawling instructions should a compliant crawler follow?	Review access rules for a crawler.
sitemap.xml	Which page URLs should search engines discover?	Include a newly published article.
llms.txt	Where should an agent begin reading?	Point an assistant to a useful guide.
llms-full.txt	Can a reader consume a larger combined text export?	Offer a separate documentation bundle.

An index and a full export are separate publishing choices. Start with the short index. Consider a larger export only if you have a reader or tool that needs it, and you can keep it current.

Keep sensitive information behind authentication. A public text file is a poor place for private customer details, access tokens, or internal instructions.

4. Write your first llms.txt file

Here is an original example for the fictional ParcelKit API. Replace every example URL before publishing. The .md links assume you have actually created Markdown versions of those pages.

# ParcelKit

> ParcelKit helps merchants buy shipping labels through an API.
> This index covers the public integration documentation.

Examples use the test environment unless the page says otherwise.
Check the API reference for current request fields and limits.

## Start here

- [Quickstart](https://example.com/docs/quickstart.md): Create a test label.
- [Authentication](https://example.com/docs/authentication.md): Configure an API key.

## API reference

- [Labels](https://example.com/docs/labels.md): Request fields and responses.
- [Errors](https://example.com/docs/errors.md): Diagnose failed requests.

## Optional

- [Release notes](https://example.com/docs/releases.md): Changes to the API.

The heading identifies the project. The summary gives context. Each link description tells the reader what to expect. The Optional section groups secondary reading that can be skipped when context is limited. These conventions are described in the format proposal.

Keep your first version small. We suggest choosing five to ten pages you would send to a new customer or developer. That number is a starting point for your editorial review, not a requirement of the format.

Do not list every URL simply because it appears in your sitemap. Read each selected page and ask whether it adds something useful to the task.

5. Adapt the example for a service website

A service business needs a different set of links from an API company. Try this selection exercise before generating a file:

1. Write down three questions prospects ask before contacting you.
2. Pick the clearest published page answering each question.
3. Add a page explaining the business and one for contacting it.
4. Remove duplicate pages that repeat the same information.

For an agency, useful questions might include “Which services do you offer?”, “What does an engagement involve?”, and “Where can I see examples of your work?” Link to pages that actually answer those questions.

Write descriptions as factual labels. “Services and engagement process” helps a reader choose a page. “The best agency for every business” adds a claim that the linked page would need to substantiate.

If you have HTML pages but no Markdown exports, use the real page URLs and review how well your intended tool reads them. Do not invent .md addresses by appending an extension to every link.

6. Publish the file on your website

For a whole-site overview, publish at /llms.txt. For a documentation section, consider /docs/llms.txt. The proposal allows both locations; use the scope that matches the content you maintain.

For a Next.js site

Next.js serves files in the project-root public folder from the corresponding URL path. For example:

public/llms.txt       -> https://example.com/llms.txt
public/docs/llms.txt  -> https://example.com/docs/llms.txt

Add the file to your repository and use your normal deployment process. See the Next.js public-folder documentation for details. Check that middleware, redirects, or hosting rules do not intercept the URL.

For another website platform

Use the platform's supported file upload, static asset, or route mechanism. Uploading an attachment into a media library may produce a URL such as /uploads/llms.txt; check the final address instead of assuming it appears at the root.

If your platform already generates the file, inspect that output first. Update the existing configuration rather than maintaining two conflicting versions.

7. Check the published response

Open a private browser window and load the public URL. You should see the text you intended to publish without signing in.

You can also check it from a terminal. Substitute your own domain:

curl --fail --show-error --location --dump-header /tmp/llms-headers.txt \
  https://example.com/llms.txt --output /tmp/llms.txt
cat /tmp/llms-headers.txt
cat /tmp/llms.txt

Review both the headers and body. A successful response alone is insufficient: some websites return an HTML error page with a 200 status.

Use this checklist:

• The final response is successful and contains your Markdown text.
• The response uses a suitable text content type, such as text/plain.
• Every listed URL works without your signed-in browser session.
• The links resolve to the expected pages, including any Markdown exports.
• The text matches your current public product or service information.

If the body begins with a login form or an HTML application shell, fix the route before moving on.

8. Test whether an assistant can use it

Try a controlled reading task with a tool that can fetch URLs. Give it the file as the starting point and ask it to cite the page supporting its answer.

Start with https://example.com/llms.txt.
Find the instructions for authenticating a ParcelKit API request.
Follow the relevant link and cite the supporting page.
If the documentation does not answer the question, say what is missing.

Prepare a small set of questions before testing. Include one the documentation answers clearly and one it cannot answer. Review the sources, not just the wording of the response.

Record what happened:

• Did the tool fetch the file?
• Did it follow the relevant link?
• Did the answer agree with that page?
• Did it admit when the information was missing?

This checks whether the workflow works when you explicitly supply the file. It does not establish that an unrelated assistant will discover it automatically. If you have server logs, inspect actual requests separately. A request is evidence of access, not proof that the file caused a citation or a sale.

9. Common problems and fixes

Problem	What to check next
The file returns 404	Confirm its exact filename, deployed location, and route configuration.
The URL shows a login page	Check authentication middleware and hosting rules.
A linked .md page fails	Publish the export or replace the link with an existing page.
Answers describe an old product	Compare the index and linked pages with the current release.
The index is difficult to scan	Remove duplicates and make link descriptions more specific.
No measurable change appears	Review actual use before spending more time expanding the file.

Assign an owner to review the file whenever important pages move or change. Keep that review alongside the content update rather than relying on a separate reminder months later.

10. Frequently asked questions

Do I need llms.txt to appear in Google's AI features?

No. Google's published guidance does not require an AI text file. Focus first on useful, accessible pages and the normal requirements for Search.

Will uploading the file train a model on my website?

Uploading a public file is not a model-training operation. Whether another system fetches or uses your content depends on that system. Do not treat an upload as confirmation of ingestion.

Should I create llms-full.txt as well?

Start by identifying who will use it. If your intended tool needs a combined export, test one with that tool. Otherwise, maintain the smaller index and spend the remaining effort improving the linked pages.

Can I inspect a real example?

Yes. Open Unsloth's documentation index and compare it with the fine-tuning guide. The index lists documentation links; the guide contains the explanation a reader follows those links to find.

What should my team do next?

Choose a few useful pages, publish a small file, and run the reading test above. Keep a record of what worked and what failed. For help reviewing your site's content and discovery paths, contact NextGrowthLabs or read about our ChatGPT SEO services.