System design
Financial document intelligence
Design a pipeline that extracts structured data from 50,000 financial PDFs a day, including statements, invoices and filings, with high accuracy.
45:00
45 minutes in five phases. Talk out loud the whole time.
Ask first
Which document types and fields?
What accuracy target per field?
Batch or real time?
How many human reviewers?
Where does the data go afterwards?
Core design
Ingest to object storage and queue each document.
Classify the document type.
Layout parsing and OCR (Textract or Document AI), plus a vision model for charts and complex tables.
Schema-based extraction with structured outputs.
Validation rules: totals add up, and dates and IDs are valid.
Field-level confidence; low-confidence items go to a review queue, and corrections feed the eval set.
Store results with lineage: source page, model version and reviewer.
Trade-offs to raise
OCR plus a text model versus a vision model reading pages directly: cost versus accuracy on tables.
Per-page versus per-document processing.
Batch APIs for cost versus latency.
Curveballs
Field-level confidence, cross-field validation and human review of anything uncertain. Report accuracy after review, not just the model’s.
Detect them and route them to specialist OCR or to humans.
An idempotent pipeline and versioned outputs; compare on a labeled sample before switching.
Numbers to state
Pages per day × cost per page.
Reviewer capacity: documents per hour × reviewers.