MarkPrep

PDF to Markdown for RAG pipelines

This PDF to Markdown converter prepares LLM-ready content for RAG. It converts each PDF in your browser, previews the chunks and exports them as JSONL.

Drop files, paste, or click to browse
PDF, Word, Excel, PowerPoint, HTML, CSV, JSON, EPUB, images and more. Nothing is uploaded.
Live count of requests to other websites during conversion: 0To see for yourself, press F12, open the Network tab and convert a file.

To use PDFs in a RAG pipeline, you convert them to text, split the text into chunks and embed each chunk. MarkPrep does the first two steps in your browser and exports the chunks as JSONL for your embedding step. This page starts the chunk preview at 800 tokens with 100 tokens of overlap, the same default as the chunk preview tool. You can change both in the Chunks tab.

  • Heading-aware chunks from 100 to 2,000 tokens, with 0 to 400 tokens of overlap
  • Metadata on every chunk: heading path, estimated tokens, and the page number for PDFs
  • JSONL export with one chunk per line

Why parse PDFs to Markdown before RAG?

A RAG index stores chunks of text and their embeddings, so every PDF needs a text step before anything is embedded. This holds even when your AI model can read PDFs itself, because the index stores only text. Converting to Markdown keeps the headings, lists and tables that plain text extraction loses.

Headings show where one section ends and the next begins. That gives the chunk preview clean places to split. Converting first also lets you read what will be embedded. You can catch a garbled scan or a broken table before it reaches your vector store.

How much does PDF parsing quality affect RAG?

A parsing error isn't fixed later in the pipeline. It moves into the chunks, then into the embeddings, then into the passages retrieval hands to the model. A table whose rows run together comes back as numbers with no column names. A header repeated on every page adds the same words to many chunks.

MarkPrep can't measure the effect on your own retrieval, but it shows you the problems it finds. It flags tables whose rows don't match the header's column count, and scans whose text looks garbled.

What to clean in PDFs before putting them into RAG

  • Running headers and footers: a first or last line that repeats on at least 60% of pages (3 pages or more) is removed.
  • Page numbers, including "Page 3 of 12" lines: removed.
  • Words split by a hyphen at the end of a line: joined back together.
  • Runs of blank lines: collapsed to one.
  • Tables: pipes padded and checked, with uneven rows flagged in the Verify tab.

Each clean-up step has its own switch in the options panel.

Chunk size and overlap

The chunk preview packs Markdown blocks, such as paragraphs, lists and tables, into chunks up to the size you set. It splits a block only when the block alone is bigger than that size. With heading-aware mode on, a chunk that is at least 80% full ends at the next # or ## heading, so that section starts a new chunk.

You can set the size from 100 to 2,000 tokens and the overlap from 0 to 400. The chunk preview uses one token estimate for every model, about 3.5 characters per token. Your embedding model may count more, so set the size a little below its input limit. Look at where the boundaries fall, then test retrieval with questions whose answers you know.

What the JSONL export looks like

Each line of the .jsonl file is one chunk: an id, the chunk text, and a metadata object. Here is one line, with the text shortened:

{"id":"q3-report-0","text":"# Q3 Field Report\n\nRainfall held steady...","metadata":{"source":"q3-report","title":"Q3 Field Report","chunk_index":0,"heading_path":["Q3 Field Report"],"page":1,"tokens":212}}

The page field holds the page number for PDFs. For files without pages, such as Word or HTML, it is null. In LangChain, LangChain.js or your own script, read each line, embed the text and store the metadata with the vector.

Large collections and local RAG

The browser converter accepts several files at once. For a large local collection, use the Pro command-line converter. It watches a folder and writes a Markdown file for each PDF, Word or Excel file you add. It runs on your machine, so the documents stay local.

The command-line converter writes Markdown only, and it has no OCR. Chunk preview, JSONL export and OCR for scanned PDFs are in the browser. MarkPrep doesn't embed, store or search chunks, so pair its output with the embedding model and vector store you already run, locally or hosted.

Related pages

Frequently asked questions

What chunk size should I use for RAG?+

Start with MarkPrep's default of 800 tokens and 100 tokens of overlap. The Chunks tab redraws the chunks as you move the size and overlap sliders, so you can see where each boundary falls before you export.

What metadata is in the JSONL?+

The metadata holds source, title, chunk_index, heading_path, page and tokens. The page field is the page number for PDFs and the slide number for PowerPoint files.

Do I need a PDF to Markdown parser, or can the LLM read my PDFs?+

A RAG index needs text to embed, so some parsing step happens either way. Converting to Markdown first lets you read the text, count the tokens and see the chunks before anything is embedded.

Can MarkPrep parse two-column research papers?+

It converts them, but it reads each page left to right, top to bottom, so lines from the two columns can interleave. Check the Markdown before you chunk it.

Is anything uploaded?+

No. Conversion, chunking and the JSONL export run in your browser, and the .jsonl file is saved straight to your computer.