improve receipt OCR detection via a real-receipt fixture corpus

## Problem OCR accuracy is the project's named risk (docs/00-overview.md, docs/adr/0001-self-hosted-ocr.md: ~70-85% line accuracy on real receipts). The parser corpus still holds only the 7 synthetic seed fixtures and no real-receipt coverage, and the Python sidecar does no image preprocessing. The docs mandate growing the corpus "from real receipts from day one" and turning every misparse into a fixture before fixing it (docs/11-testing-strategy.md) - this issue establishes that loop and drives both OCR layers toward a green corpus. ## Approach Two layers, one deterministic test surface: - **Image -> text lines**: Python PaddleOCR sidecar (ocr/app/recognizer.py) - improve preprocessing (deskew / denoise / binarize / upscale / angle classifier). - **Text lines -> ParsedReceipt**: pure C# ReceiptParser (backend/src/BillSplitter.Domain/ReceiptParser.cs) - improve heuristics; its fixture corpus is the real unit-test spec and can reach 100% green. Per real receipt image: 1. Add the image at backend/tests/BillSplitter.Tests/Fixtures/receipts/{name}/receipt.(jpg|png). 2. Record the live sidecar's response to {name}/ocr.json (dev helper POSTs raw bytes to POST /ocr). 3. Hand-author {name}/expected.json (ground-truth ParsedReceipt; money in integer minor units). 4. Run ReceiptParserCorpusTests; fix ReceiptParser for parser misses, fix the sidecar preprocessing (and re-record ocr.json) for raw-text misses. 5. Repeat until the corpus is green. ## Scope / tasks - [ ] Add a dev-only fixture-recording helper (POST image -> ocr.json). Not wired into app runtime. - [ ] Replace the brittle HaveCount(7) assertion in ReceiptParserCorpusTests with a "named seeds present" check so the corpus can grow freely. - [ ] Add the first batch of real-receipt fixtures (image + ocr.json + expected.json). - [ ] Fix ReceiptParser heuristics until the parser-side corpus is green. - [ ] Add sidecar image preprocessing in ocr/app/recognizer.py (+ config flags in ocr/app/config.py if needed); re-record affected ocr.json. - [ ] Update docs (docs/06-ocr-service.md, and docs/04-api-contract.md only if the /ocr response shape changes). ## Privacy Fixture images are permanent repo history. Use only receipts free of personal data (no cardholder names, full PANs, emails, phones) - synthetic, self-owned, or redacted. The runtime "never store image bytes" rule governs live sessions, not test fixtures. ## Acceptance criteria - [ ] Recording workflow is documented and repeatable (drop image -> record -> author expected -> test). - [ ] Real-receipt fixtures added and the full ReceiptParserCorpusTests corpus is green. - [ ] Sidecar preprocessing improvements land with the raw-text misreads they resolve captured as fixtures. - [ ] Any unrecoverable raw misreads are parked as warnings, not silent drops. - [ ] Docs updated for any contract/heuristic change.

TimCane commented

2026-07-06 22:50:56 +01:00

(Migrated from github.com)

Problem

OCR accuracy is the project's named risk (docs/00-overview.md,
docs/adr/0001-self-hosted-ocr.md: ~70-85% line accuracy on real receipts). The
parser corpus still holds only the 7 synthetic seed fixtures and no real-receipt
coverage, and the Python sidecar does no image preprocessing. The docs mandate
growing the corpus "from real receipts from day one" and turning every misparse
into a fixture before fixing it (docs/11-testing-strategy.md) - this issue
establishes that loop and drives both OCR layers toward a green corpus.

Approach

Two layers, one deterministic test surface:

Image -> text lines: Python PaddleOCR sidecar (ocr/app/recognizer.py) -
improve preprocessing (deskew / denoise / binarize / upscale / angle classifier).
Text lines -> ParsedReceipt: pure C# ReceiptParser
(backend/src/BillSplitter.Domain/ReceiptParser.cs) - improve heuristics; its
fixture corpus is the real unit-test spec and can reach 100% green.

Per real receipt image:

Add the image at
backend/tests/BillSplitter.Tests/Fixtures/receipts/{name}/receipt.(jpg|png).
Record the live sidecar's response to {name}/ocr.json (dev helper POSTs raw
bytes to POST /ocr).
Hand-author {name}/expected.json (ground-truth ParsedReceipt; money in integer
minor units).
Run ReceiptParserCorpusTests; fix ReceiptParser for parser misses, fix the
sidecar preprocessing (and re-record ocr.json) for raw-text misses.
Repeat until the corpus is green.

Scope / tasks

Add a dev-only fixture-recording helper (POST image -> ocr.json). Not wired
into app runtime.
Replace the brittle HaveCount(7) assertion in ReceiptParserCorpusTests with
a "named seeds present" check so the corpus can grow freely.
Add the first batch of real-receipt fixtures (image + ocr.json +
expected.json).
Fix ReceiptParser heuristics until the parser-side corpus is green.
Add sidecar image preprocessing in ocr/app/recognizer.py (+ config flags
in ocr/app/config.py if needed); re-record affected ocr.json.
Update docs (docs/06-ocr-service.md, and docs/04-api-contract.md only if
the /ocr response shape changes).

Privacy

Fixture images are permanent repo history. Use only receipts free of personal
data (no cardholder names, full PANs, emails, phones) - synthetic, self-owned, or
redacted. The runtime "never store image bytes" rule governs live sessions, not
test fixtures.

Acceptance criteria

Recording workflow is documented and repeatable (drop image -> record ->
author expected -> test).
Real-receipt fixtures added and the full ReceiptParserCorpusTests corpus is
green.
Sidecar preprocessing improvements land with the raw-text misreads they
resolve captured as fixtures.
Any unrecoverable raw misreads are parked as warnings, not silent drops.
Docs updated for any contract/heuristic change.

Rows
Columns

improve receipt OCR detection via a real-receipt fixture corpus #75

Problem

Approach

Scope / tasks

Privacy

Acceptance criteria