99.2%
Structured-corpus reconciliation
124 of 125 purchase orders
Supplier pricing benchmark with independent validation
AI & agentic workflows Sanitised case study
Built a supplier-price comparison with 99.2% reconciliation against the structured reference data. Two independent data-extraction methods agreed on 98.2% to 98.3% of the results.
-
The problem
Pricing sat across structured purchase orders and unstructured catalogues, without a reliable normalised benchmark.
-
My role
Parsed structured purchase orders deterministically and reserved model vision for the harder catalogue.
-
Where it stands
The structured corpus reconciled to 99.2% (124 of 125 purchase orders, 722 lines, 333 tests); the two engines agreed to 98.2–98.3% across 1,847 price points; automated quality checks — unit, currency, and magnitude anomalies — raised zero flags across 8,508 extracted rows.
Results and evidence
98.2–98.3%
Cross-engine agreement
1,847
Price points cross-validated
722 lines · 333 tests
Validation depth
0
Quality flags raised
across 8,508 extracted rows
Business impact
Produced a defensible pricing benchmark with accuracy evidenced by cross-validation rather than assumed.
How I did it
- 01
Ran a second independent engine and cross-validated results price point by price point.
- 02
Selected the lowest-cost method that met the accuracy threshold for each document type.
Tools and working context
A supplier pricing benchmark across a structured purchase-order corpus and a harder catalogue set.
- Tools
- Python · regex · pdfplumber · pandas · Pydantic · Gemini vision
- Business area
- Procurement · Pricing
- Skill type
- AI · Data & Reporting
Why I made these choices
- Matched method to document structure instead of applying one extraction technique everywhere.
- Used independent-engine agreement as the accuracy evidence.
Skills used
- Regex-vs-LLM routing AI & Automation Strong
What I learned
- Independent agreement is stronger evidence than one engine validating itself.
What comes next
- Extend labelled evaluation across the full catalogue set and track accuracy by document type.