feat(frontend): crop and rotate receipt before upload #118

New issue

Open

opened 2026-07-08 08:41:53 +01:00 by TimCane · 0 comments

TimCane commented

2026-07-08 08:41:53 +01:00

(Migrated from github.com)

Context

OCR accuracy is the stated project risk and receipt re-upload is out of scope, so a bad photo costs the whole session. Nothing in the stack owns framing, background clutter, or fine skew: the client only downscales/re-encodes, and the sidecar's angle classifier fixes crop orientation, not skew. The sidecar assembles same-row detections into one line by horizontal band (#116), so a few degrees of skew splits an item name from its price and breaks row assembly.

Capture stays on the native OS camera (capture="environment") - no getUserMedia viewfinder; all correction is post-capture.

Scope

Preview screen becomes the crop screen via react-easy-crop (touch pinch/drag/rotate; first non-Radix UI dep)
Optional, never blocking: default crop is the full image, Use photo works immediately
Fine-angle rotation slider + 90° button
preprocess() order: crop -> rotate -> downscale -> JPEG, so cropping recovers resolution otherwise lost to the 2000px cap; MAX_EDGE/quality unchanged
No server or contract changes

Docs (same PR)

docs/09-ux-flows.md §1: crop handles + rotation control in the wireframe
docs/08-frontend-design.md #uploads: new preprocess shape, the dep

Acceptance

Crop/rotate on preview, skippable without interaction
Cropped region is what uploads; downscale applies after crop
Crop geometry pure functions covered by vitest
Fixture check: sample receipts shot raw vs cropped/rotated, sidecar output diffed, recordings added to the parser corpus

Rejected (with revival conditions)

Perspective/corner-drag warp + OpenCV.js auto-detect: only if fixture diffs prove keystone is a real failure mode (jscanify lazy-loaded is the path)
In-app getUserMedia camera: OS camera focus/HDR beats a video-stream frame

## Context OCR accuracy is the stated project risk and receipt re-upload is out of scope, so a bad photo costs the whole session. Nothing in the stack owns framing, background clutter, or fine skew: the client only downscales/re-encodes, and the sidecar's angle classifier fixes crop orientation, not skew. The sidecar assembles same-row detections into one line by horizontal band (#116), so a few degrees of skew splits an item name from its price and breaks row assembly. Capture stays on the native OS camera (`capture="environment"`) - no getUserMedia viewfinder; all correction is post-capture. ## Scope - Preview screen becomes the crop screen via **react-easy-crop** (touch pinch/drag/rotate; first non-Radix UI dep) - Optional, never blocking: default crop is the full image, `Use photo` works immediately - Fine-angle rotation slider + 90° button - `preprocess()` order: crop -> rotate -> downscale -> JPEG, so cropping recovers resolution otherwise lost to the 2000px cap; `MAX_EDGE`/quality unchanged - No server or contract changes ## Docs (same PR) - `docs/09-ux-flows.md` §1: crop handles + rotation control in the wireframe - `docs/08-frontend-design.md` #uploads: new preprocess shape, the dep ## Acceptance - [ ] Crop/rotate on preview, skippable without interaction - [ ] Cropped region is what uploads; downscale applies after crop - [ ] Crop geometry pure functions covered by vitest - [ ] Fixture check: sample receipts shot raw vs cropped/rotated, sidecar output diffed, recordings added to the parser corpus ## Rejected (with revival conditions) - Perspective/corner-drag warp + OpenCV.js auto-detect: only if fixture diffs prove keystone is a real failure mode (jscanify lazy-loaded is the path) - In-app getUserMedia camera: OS camera focus/HDR beats a video-stream frame