Web Pages for AI: Getting the Article Without the Page Around It
If you want an LLM to read a web page, convert its HTML to Markdown and keep only the article. The navigation, cookie banner, newsletter box, related headlines, ads and footer around the article are page clutter. Your eye filters all of that out without trying. An LLM reads the page clutter along with the article, and every word of it costs tokens.
Converted well, the LLM gets the article's headings, paragraphs, lists and links, and none of the page clutter.
HTML already has structure
Unlike a PDF, HTML tells you what everything is. An <h1> is a heading, a <ul> is a list, a <table> is a table and an <a> is a link. So the mapping to Markdown is direct: headings become #, lists become -, and links become [text](url). That is why HTML is one of the easiest formats to convert.
On a web page, the article's tags are mixed in with the page's own tags. The menu is a <ul> too, and the cookie banner is a <div> full of text. So a good HTML to Markdown conversion has two jobs: map the tags, and drop the ones that aren't content.
What a clean conversion removes
A clean conversion removes four kinds of page clutter:
- Navigation, headers and footers repeat on every page and mean nothing to a model.
- Scripts and styles are code.
- Cookie banners, newsletter prompts and share buttons are interface.
- Ads and "related content" widgets can mislead a model if they end up in its context.
How MarkPrep handles a file and a URL
MarkPrep has two routes, and they remove different amounts:
- An HTML file you already have. The HTML to Markdown converter drops scripts, styles,
<nav>and<footer>elements,<noscript>blocks and iframes. Other page clutter, such as a cookie banner inside an ordinary<div>, stays in the output, so delete it by hand if it appears. - A public web page by URL. The URL to Markdown converter fetches the page. Mozilla Readability, the engine behind Firefox Reader View, then picks out the main article. Menus, sidebars and cookie bars go. On a page with no clear article, such as a home page, it falls back to the whole page.
Both routes keep the article's structure. MarkPrep writes HTML tables as Markdown pipe tables and keeps links as Markdown links, so the LLM still sees where a phrase pointed.
A privacy note for URLs
Privacy matters less for a public web page than for your own contracts, but a file and a URL are handled differently. Converting an HTML file you already have happens in your browser, with nothing sent. Converting by URL means MarkPrep's server fetches that public page for you, because a browser can't read most other sites directly.
After the clutter is gone
Removing the page clutter also shrinks the token count. Once you have the article, MarkPrep shows its estimated token count and whether it fits your model's context window. To check another file, use the token counter.
Next step
Save the page as HTML and run it through HTML to Markdown. Or paste the address into URL to Markdown to keep only the main article. Skim the result for any banner text that slipped through before you give it to the LLM.
Try it on your own file
Convert a document and watch the token counter. It's free, needs no account and uploads nothing.