Extract Text from Images in Your Browser: The Complete OCR Guide

The world hands you text as pixels. A screenshot of an error message, a photographed receipt, a signed contract scanned at the print shop, a slide shot from row fourteen, a Japanese product label. Turning those pixels back into characters is the job of OCR — and for most people it currently involves dragging a file onto someone else’s server and hoping their retention policy is as short as their marketing page implies.

You do not need the server. The same recognition engine that powers those services compiles to WebAssembly and runs in a tab, offline, on your machine. This guide covers what actually happens when you press extract, the three settings that decide whether you get a clean transcript or a page of noise, and how to audit the result instead of trusting it.

What runs when you press extract

The engine is Tesseract, a recognition project that started at HP in the 1980s and was rebuilt around an LSTM neural network in version 4. The browser port, Tesseract.js, wraps the compiled engine in a Web Worker so recognition never freezes the page. Two artefacts are downloaded on first use and then cached by the browser:

• The engine core — a WebAssembly build of Tesseract itself, a few megabytes, fetched once per site and cached like any other static asset.
• Language data (traineddata) — the trained model for each language you select, typically a few megabytes each. This is the part that grows: recognising English plus Simplified Chinese costs roughly double the download of English alone.

The tool runs the engine in LSTM-only mode (OEM 1), the modern neural recognition path. The older legacy engine is slower and less accurate and is deliberately not used. Because everything executes in a worker, the interface stays responsive while a large scan is being read, and you can keep typing elsewhere in the tab.

Step 1: choose languages deliberately

Language selection is not a formality. Tesseract loads one model per language and scores candidate characters against them, so a wrong or excessive selection measurably costs you accuracy and time.

• Select only what is on the page. A German invoice with an English product name needs deu and eng. Adding six more languages forces the engine to adjudicate between scripts that are not present, which is how you get umlauts silently“corrected” into English lookalikes.
• Simplified and Traditional Chinese are different models. chi_sim and chi_tra are separate downloads. Mixed-source documents sometimes need both; most do not.
• Japanese and Korean are their own models (jpn, kor), as are the major European languages. Mixed CJK-plus-Latin pages are the one case where selecting two or three languages is genuinely correct.

Practical shortcut: the tool re-reads the image automatically when you change the language set, so you can try eng, look at the confidence readout, then add a second language and compare. Swapping languages after the first download costs nothing beyond the initial fetch.

Step 2: page segmentation — the setting everyone skips

Before it recognises a single character, Tesseract has to decide where the text is. That is page segmentation, and it is the single highest-leverage control in the whole tool. Four modes are exposed, each mapping to a Tesseract PSM constant:

If a result comes back as garbage, change this before you do anything else. Recognising a single line of a terminal error with Auto mode is the most common self-inflicted OCR failure — the engine treats one line as a whole document layout and finds no structure at all.

Step 3: preprocessing — grayscale or Otsu binarisation

Both preprocessing options run client-side on a canvas before the image reaches the engine. They are cheap and they are not interchangeable.

• None — start here. Modern Tesseract handles colour and anti-aliasing well, and modern screenshots are already clean.
• Grayscale — removes colour information while keeping all luminance detail. The safe first intervention for photos of screens and colour scans.
• Binarisation (Otsu threshold) — pushes every pixel to pure black or pure white using a computed threshold. Excellent for scanned documents with faded ink or uneven lighting; destructive for gradient backgrounds, low-contrast UI text, and anything relying on anti-aliasing.

The rule of thumb: binarise scanned paper, leave screenshots and photographs alone. If text has a coloured background or a subtle shadow, binarisation will eat thin strokes and turn l into I.

Audit the result with confidence scores

OCR output is a draft, not a transcript. The tool exposes per-word confidence and lets you filter by a threshold — which turns proofreading from reading everything into reading the right 5%.

• Filter first, read second. Raise the confidence threshold until only the suspect words remain highlighted, then check those against the image. On a clean scan, very few words survive a high threshold.
• Look for patterns, not typos. If every instance of a product name is wrong, the fix is a find-and-replace, not a re-read. If every digit is wrong, you picked the wrong segmentation mode or preprocessing.
• Enable the box overlay. Seeing the recognised boxes drawn over the image instantly reveals mis-segmentation: boxes that span two columns, or a block of text the engine never entered.
• Export as JSON when it matters. Structured output carries the per-word confidence alongside the text, so a downstream script can flag anything under your threshold instead of silently shipping it.

Four recipes

Scanned contract or letter

Languages: the document language only. Mode: Single block. Preprocessing: binarise if the scan is grey or uneven, grayscale if it is already crisp. Expect near-perfect body text; check proper nouns, dates and currency amounts by hand — they are the only things that matter legally, and they are the things OCR is worst at.

Screenshot of an error message or code

Languages: eng. Mode: Single line for one line, Sparse text for a whole IDE window with scattered elements. Preprocessing: none — binarising a dark-theme screenshot destroys it. Paste straight from the clipboard instead of saving a file first.

Photographed receipt

Photograph flat, fill the frame, avoid shadow. Languages: the receipt language. Mode: Sparse text — receipts have no paragraph structure. Preprocessing: try grayscale first. Verify every number: on low-contrast thermal paper, a misread decimal point is the classic failure.

Mixed Japanese or Chinese page

Languages: jpn or chi_sim/chi_tra plus eng if Latin text appears. Mode: Auto for full pages, Single block for columns. Be aware that vertical Japanese text is a genuine weak spot for most browser OCR pipelines; horizontal body text is reliable, vertical typography is not.

Where browser OCR still fails

• Handwriting. Tesseract is trained on printed text. Neat printing sometimes works; cursive does not.
• Low resolution. Body text needs roughly 300 DPI at the original size. A photo of a screen from across the room does not have the pixels.
• Rotation and skew. More than two or three degrees degrades accuracy steeply. Straighten first.
• Tables and forms. The words are recognised; the structure usually is not. Expect to rebuild the table by hand.
• Stylised display type. Logos, posters, thin letter-spaced fonts.

What to expect, and how to measure it yourself

We deliberately publish ranges and mechanics rather than a single “0.4 seconds” figure, because recognition time depends on your CPU, your browser’s WebAssembly performance, the language count and how much of the image contains text. The tool prints the wall-clock time next to every result, so you can produce your own number in under a minute: run the same image once with one language and once with three, and once with Auto and once with Single line. The deltas you observe are the ones that matter for your workload, and they are typically far larger than any difference between browsers.

The structural costs are the ones worth planning around: the first run of a session pays for the engine plus language data; every later run is pure compute. That is the opposite of a cloud OCR service, where every page costs an API call and a retention event.

Why doing this locally matters

The documents people OCR are disproportionately the documents they should not upload: contracts, medical paperwork, identity documents, invoices with client names, internal financials, legal correspondence. Uploading them creates a copy on infrastructure you do not control, governed by terms you have not read, in a jurisdiction that may not be yours. Running recognition in the tab means the bytes never move: there is nothing to delete afterwards, nothing to breach, nothing to subpoena. For a great deal of real work, that is not a preference — it is the difference between a tool you are allowed to use and one you are not.

Open the OCR tool, drop in an image, and watch the boxes appear. Change one setting, re-run, and you will understand this engine better than most people who pay for it.