Appearance
Layout analysis and line segmentation determine manuscript OCR quality more than the recognition model itself: if the engine misidentifies regions, merges lines or gets the reading order wrong, even a near-perfect model produces unusable text. Good practice is to detect content regions first (main text, columns, marginalia), then detect baselines within them, then verify reading order before any recognition runs. On irregular historical pages, a brief human correction of segmentation is usually the highest-value step in the whole pipeline.
Why does segmentation matter more than recognition?
A recognition model only ever sees the cropped line images that segmentation hands it. Feed it a line that actually contains the tail of one row plus the heads of the next, and it will faithfully transcribe nonsense. Studies and practical experience converge on the same point: a large share of "bad OCR" on manuscripts is really bad segmentation. Fix the lines and the apparent recognition problem often disappears.
Regions first or lines first?
Regions first. Detecting content blocks before lines lets you keep marginalia, catchwords and decoration out of the main text stream.
bash
# Kraken: detect baselines and regions in one pass
kraken -i page.tif seg.json segment -blThe workflow is hierarchical:
- Region detection — main text, each column, margins, headers.
- Reading order — how regions and lines chain into running text.
- Baseline/line detection — the individual writing lines inside each region.
What are baselines and why use them?
A baseline is the imaginary line letters sit on. Older OCR drew rectangular bounding boxes around lines, which fails on slanted or interlinear handwriting where boxes overlap. Baseline detection traces the writing line itself, then a polygon (the "boundary") captures ascenders and descenders. This is why baseline-based engines like Kraken and Transkribus handle cramped, sloping manuscript hands far better than box-based approaches.
How do you handle columns, marginalia and irregular pages?
Each page type needs a slightly different setup.
| Page feature | Risk | Mitigation |
|---|---|---|
| Multiple columns | Interleaved reading order | Define column regions, set order |
| Heavy marginalia | Margin text mixed into body | Separate margin regions |
| Slanted writing | Overlapping bounding boxes | Use baseline polygons |
| Interlinear additions | Lines merged | Manual line correction |
| Tables / registers | Cells flattened to prose | Detect table structure separately |
For marginalia specifically, decide early whether margins are a separate transcription stream (common for scholarly editions) or excluded entirely.
How do you check and correct segmentation?
Never trust automatic segmentation blindly on irregular material. Open the segmentation overlay in eScriptorium or Transkribus and scan for three failure modes:
- Merged lines — two writing lines under one baseline.
- Split lines — one line broken into fragments.
- Stray regions — decoration or bleed-through detected as text.
python
# Quick sanity check: count detected lines per page
import json
seg = json.load(open("seg.json"))
print("lines detected:", len(seg["lines"]))A page with 24 visible lines but 40 detected lines is signalling a problem worth a human pass before recognition.
When should you train a custom segmentation model?
If automatic baselines fail consistently on your collection — dense glosses, unusual page geometry, two-column legal hands — train a segmentation model on a few dozen manually corrected pages. Segmentation training is separate from recognition training and often pays off faster, because one good segmentation model serves every recognition model you build afterwards.
Key Takeaways
- Segmentation quality, not the recognition model, usually sets the ceiling on accuracy.
- Detect regions first, then reading order, then baselines within regions.
- Baseline detection beats bounding boxes on slanted, crowded handwriting.
- Keep marginalia in separate regions to avoid contaminating the main text.
- Verify segmentation visually; watch for merged lines, split lines and stray regions.
- For persistently irregular pages, train a dedicated segmentation model.
Frequently Asked Questions
What is the difference between region detection and baseline detection?
Region detection finds blocks of content (main text, marginalia, columns); baseline detection finds the individual writing lines within those blocks. You normally do regions first, then baselines.
Why is segmentation more important than the recognition model?
If lines are merged, split or in the wrong reading order, the recognition model receives garbled input and produces garbled output no matter how accurate it is on clean lines.
How do I stop marginalia contaminating the main text?
Define separate regions for margins and main text during layout analysis, and either transcribe them as distinct streams or exclude margins from the main reading order.
What is a baseline in HTR segmentation?
A baseline is the line that the bottoms of letters sit on. Modern HTR engines detect baselines rather than bounding boxes because they cope better with slanted and crowded handwriting.
How do I fix wrong reading order in multi-column pages?
Set or correct the reading order during layout analysis so columns are read top-to-bottom then left-to-right, rather than letting lines interleave across columns.
Can layout analysis be automated reliably?
Automatic baseline models work well on regular pages, but irregular manuscripts usually need a quick human review and correction step before recognition.