Markdown Page Generation
On this page
An Astro integration that generates .md companion files and an llms.txt index from your built HTML pages. The .md files are Markdown versions of your HTML content, suitable for consumption by LLMs or static site search tools. Instead of using the raw source Markdown, this integration reverse engineers Markdown based on the generated HTML pages to avoid losing content declared in components and included files.
Installation
$ npm install astro-better-gen-markdown-pages
Setup
In astro.config.ts:
import genMarkdownPages from 'astro-better-gen-markdown-pages';
export default defineConfig({
integrations: [
genMarkdownPages({
llmsTxtTitle: 'My Site Documentation',
llmsTxtDescription: 'Complete docs for My Site.',
trimTitleSuffix: ' | My Site',
}),
],
});
Options
| Option | Type | Default | Description |
|---|---|---|---|
llmsTxt | boolean | true | Write llms.txt, its .well-known mirror, and spoke files. Set to false to serve your own llms.txt from public/ (or omit the index files entirely, as you wish); always writes .md files |
llmsTxtTitle | string | 'Documentation' | Title in the generated llms.txt |
llmsTxtDescription | string | '' | Description in the generated llms.txt |
llmsTxtPath | string | 'llms.txt' | Output path for llms.txt |
trimTitleSuffix | string | '' | Strip this suffix from page titles in the index |
pageFilter | (path) => boolean | null | Filter which pages get .md files |
indexFilter | (path) => boolean | () => true | Filter which pages appear in llms.txt |
categorize | (path) => string | first path segment | Group pages in llms.txt by category |
docsIndexUrl | string | '' | URL to add as the docs index reference in llms.txt |
Output
After astro build:
- Each HTML page gets a sibling
.mdfile at the same path (e.g.,/docs/intro.md) - A top-level
llms.txtindex lists all pages grouped by category
llms.txt format
# My Site Documentation
> Complete docs for My Site.
## Getting Started
- [Introduction](/docs/intro.md): The basics.
- [Installation](/docs/install.md): How to install.
## Reference
- [API Reference](/docs/api.md): Full API docs.
MarkdownOnly component
MarkdownOnly renders content that is invisible in the browser but included in the generated .md output. Use it to add LLM-facing context without cluttering the visual page.
import MarkdownOnly from 'astro-better-gen-markdown-pages/MarkdownOnly.astro';
<MarkdownOnly>
This text appears only in the generated .md file, not in the browser.
</MarkdownOnly>
How it works
The integration hooks into astro:build:done. For each .html file in the output,
it runs a Turndown-based HTML-to-Markdown conversion, then writes the .md file.
The llms.txt is generated from the collected titles and paths.
Code blocks keep their language tag. When a code block contains its own backtick fences, such as a Markdown example with code blocks inside a list, the outer fence grows one backtick longer than the longest fence inside, indented or not, so the example stays in one block.
Conversion runs in a pool of worker threads, one per CPU, that hands out pages in small batches. When a site uses several instances of the integration (one per section, say), they share the pool, so its workers start once per build.