Elkwai SolutionsChoose a narrow document workflow, preserve what should have been extracted and measure the errors that make downstream work expensive. A well-formed payload is only the start.
Review supplier, currency, total and purchase-order reference against the original. Keep missing purchase orders as reviewed absences rather than invented values. Check totals, tax and approval policy separately before any accounting entry.
Extract the reference, category and agreed scalar details. Test missed identifiers, unexpected extra fields and incomplete runs. Arrays, repeating line items and matching across tables require a separate evaluation design.
Record stable case IDs and reviewer conclusions. Preserve where a field came from, reviewer disagreements and source hashes privately. This tool does not establish a document's authenticity or interpret legal obligations.
Classify the document first, then evaluate extraction for the right record type. Measure both stages independently: correct routing does not imply correct fields, and a perfect extraction can still reach the wrong system.
Give each case a stable ID, a cohort and every ruled field. Use null only when a reviewer determined that the field is absent. Keep source documents and judgments under your own access controls. Do not use the producing model's self-reported confidence as ground truth.
Start with exact matching for identifiers and currency. Enable text normalization only when those differences genuinely do not matter. Decimal fields require plain strings and a documented absolute tolerance; compare currency separately. Do not loosen rules merely because a release scored poorly.
Use reviewed cohorts such as layout family, scan quality, intake channel or document type. Include blurry scans, conflicting labels, empty optional fields, rotated pages and interrupted runs where relevant. The evaluator consumes recorded values; it does not create or validate these difficult source cases.
Keep an untouched holdout and record parser/model/prompt versions beside each run. Unknown case IDs, omitted values and unruled keys should remain visible, not disappear during preparation.
Inspect precision, recall and F1 for each field, then whole-record pass rate. Many correct empty fields can inflate slot accuracy while important values are still missed. Missing records never count as correct absences. A wrong expected value creates both FP and FN; a returned value at reviewed absence creates FP only.
Unknown IDs are excluded from reference metrics and retained separately. Extra unruled keys are excluded from field scores but prevent a whole-record pass. Undefined denominators are unavailable, not zero. The all-cohorts view remains unfiltered so other groups cannot disappear.
Primary reference for value precision/recall/F1: Google Document AI evaluation methodology. The absence, cohort and completion boundary above is an explicit Elkwai Solutions design choice.
Measure baseline handling time, extraction/run fees, reviewer minutes, rework and exception handling. Use observed volumes and accepted records rather than headline model accuracy. The tools do not infer labor rates, provider pricing, liabilities or business savings.
Baseline cost = documents × measured baseline minutes / 60 × hourly cost Workflow cost = run fees + review minutes / 60 × hourly cost + rework + fixed operation cost Estimated difference = baseline cost - workflow cost
Keep the same period and units, include failures and avoid counting the same review effort twice. An illustrative 100-record batch at four baseline minutes, one review minute each and $30/hour has $200 baseline labor versus $50 review labor. Adding $10 run fees and $20 operating cost leaves a $120 difference before unmeasured exception, setup or liability costs. This is arithmetic, not a client outcome.
Retain the complete source-hashed report, frozen rules, source documents, judgments and run configuration. Export includes field values, IDs and cohort labels: inspect confidentiality before sharing. Hashes identify bytes, not truth. Compare a candidate against the same holdout and investigate improved and regressed cases before changing the operating process.
No universal passing score establishes readiness. Acceptance depends on the consequences of each missed or wrong field, review coverage and downstream authorization. A parser passing these tests is not authorization to post payments or change records.
Document Desk extracts PDF text or English OCR and matches labels followed by a colon. Review conflicting candidates against the original. The free workspace handles one document, two pages; the $79 one-time offline edition adds batches. It is not a general-purpose AI document interpreter.
A separately quoted Elkwai Solutions pilot can connect intake, your chosen AI or deterministic parser, field validation, human review and downstream handoff. Agree on real reference documents, operating costs and acceptance criteria before expansion.
This evaluator and its offline CLI are free. They are separate from Document Desk and the Developer Assurance Kit; no extraction MCP tool is included in Kit 1.1.0. Using this page does not start a subscription, model call or paid engagement.