Feeding Web Pages to a Local Model Without Wrecking the Context

The first time you hand a fetched page straight to a local model, one of two things happens. Either the runtime truncates it and the model answers from the navigation menu, or it fits and you wait forty seconds for a reply that misses the point. Both are the same problem: a modern web page is mostly not the thing you wanted, and on your own hardware you pay for every token of it twice — once in memory, once in time.

This is a format and budgeting problem, and it has a boring, effective fix.

What you’re actually pasting in

Open a source view on a typical article page. The prose you came for is a minority of the bytes. Around it: the document head, inline styles, script tags, SVG icon definitions, navigation, a cookie banner, a newsletter modal, social embeds, a comment widget, a footer with every category link on the site, and structured-data blobs. All of it tokenizes. Markup tokenizes badly — angle brackets, attribute names, base64 in a data URI, and long class-name strings fragment into far more tokens per character than English prose does.

So the token count of a raw page bears very little relationship to how much a human would say was on it. And the excess isn’t neutral filler. Boilerplate is text, the model reads text, and a small model asked to summarise a page will happily summarise the cookie notice if that’s the most coherent block it found.

Why this bites harder locally

On a hosted API a bloated page is a line item. Locally it hits three things at once.

Memory. Context is KV cache, and KV cache is VRAM you wanted for weights. The arithmetic in what a long context window costs is the whole story here: raising the context limit to accommodate raw pages can cost you the ability to keep the model on the GPU at all.

Time. Every token you insert gets processed before the first token comes out. Prompt processing is the phase people mistake for a slow model, and it scales with what you fed in — see prompt processing vs token generation.

Attention quality. Small models degrade on long inputs well before they hit their advertised limit. A 32K-context 7B model is not reliably using 32K. Material buried mid-document in a wall of markup is material the model may simply not act on, and it won’t tell you that happened.

A worked example with made-up but plausible numbers, purely to show the shape: say a page is 150 KB of HTML and 6 KB of actual prose. Feed the HTML and you’ve spent roughly twenty-five times the context for the same information, on a budget that was already the tightest resource you have.

Markdown as the interchange format

The fix is to convert before the model ever sees it. Markdown keeps exactly what a language model can use — headings, paragraphs, lists, tables, link text — and drops everything that only mattered to a browser. It’s also a format models have seen enormous amounts of in training, so structure reads as structure rather than as noise.

This is a solved commodity rather than something to build. Page-fetch services offer conversion as a response mode: Serply’s page-fetch endpoint documentation describes its markdown option as keeping headings, prose, lists and links while dropping navigation, scripts, styling and tracking pixels, with full HTML available separately for the cases where you need a specific attribute or embedded JSON-LD.

That last clause is the useful nuance. Keep raw HTML available for the narrow jobs that need it — pulling a price out of a known element, reading structured data, archiving a byte-exact copy. Those are jobs for your code, not for the model. The model gets markdown.

The order of operations

The sequence that works, in the loop you wrote around the model’s tool calls:

1. Fetch. Get the page.
2. Convert. HTML to markdown, outside the model.
3. Trim. Cut what conversion left behind — a nav bar rendered as a list of links is still a list of links. Strip everything above the first heading, drop link-dense lines with almost no prose, and cut the tail after the article body.
4. Cap. Enforce a hard character or token ceiling per document before it enters the context. A ceiling you chose beats a truncation the runtime chose, because you decide which end gets cut.
5. Label. Wrap each document with its source URL and a delimiter, so the model can tell documents apart and attribute claims to the right one.
6. Insert. Then run inference.

Steps 2 through 5 are all cheap CPU work that happens while the GPU is idle. There is no reason to make the model do any of it.

Multiple pages, one budget

Pulling three pages instead of one triples everything, and the ceiling is per conversation, not per document. Two habits that help:

Budget top-down. Decide the total you’ll allow for retrieved text, divide by the number of documents, and cap each accordingly. A fixed per-document limit silently becomes a per-conversation blowout as soon as the model fetches more than you expected.

Put the question last. Documents first, instruction after. Models follow a trailing instruction more reliably than one buried above a wall of text, and it costs nothing to order the prompt that way.

If the model is composing its own fetches — the situation described in giving a local model access to the live web — the budget needs enforcing in your dispatch code, because the model has no idea how much room is left and will keep asking.

The practical takeaway

Never hand raw HTML to a local model. Convert to markdown, trim the residue, cap each document against a budget you set, and label the sources. It’s unglamorous plumbing, and it does more for answer quality on a small local model than swapping the model does — because the constraint you were hitting was never the weights. It was the context you filled with a cookie banner.