Elkwai SolutionsValid JSON is not necessarily a correct record. Compare reviewed invoice, order or form fields with the values your own AI, OCR or parser produced. Find exactly what was wrong or missing without uploading source documents.
Session-only inputs. No upload, autosave, provider connection or model calls.
Up to 500 records per input and 30 rules; 512 KiB and 20,000 parsed JSON nodes per input. These independent limits can reduce the usable case count for larger records. Values remain in this browser session. Reports include field values: review sensitive information before sharing.
Precision, recall and F1 measure non-null values. Slot accuracy also includes correct absences and can flatter absence-heavy suites. A missing run never passes. Field metrics exclude unruled keys; whole-record passes do not.
Contains every evaluated record, including rows outside the selected page or filter, actual and reference field values, rules, cohorts, source hashes and unknown records. Does not contain original source documents. Review before sharing.
A reference suite has rules and cases. Each case must include every rule field in values. A reviewed absence is null; an empty string, false or zero is a value. Recorded extractions are an array of unique IDs and scalar values. Missing keys and null are omissions when a value was expected.
{"rules":[{"field":"total","mode":"decimal","absolute_tolerance":"0.01"}],"cases":[{"id":"invoice-01","group":"Invoices","values":{"total":"120.00"}}]}[{"id":"invoice-01","values":{"total":"120.01"}}]| Mode | Comparison | Use deliberately |
|---|---|---|
| exact | Typed primitive equality; no string/number conversion. | Identifiers, currency codes, booleans and exact labels. |
| text | String comparison; trim, collapse_whitespace and ignore_case are optional booleans, off by default. | Enable only formatting differences you intend to ignore. Case uses JavaScript lowercase, not locale-aware or fuzzy matching. |
| decimal | Plain decimal strings; exact scaled-integer comparison within a non-negative absolute_tolerance, default "0". | 15 integer / 6 fractional digits. No currency symbols, grouping, exponent or locale interpretation. Check currency separately. |
IDs and field names: 1-128 letters, digits, dots, colons, underscores or hyphens. Values: strings up to 1,024 characters, booleans, supported JSON numbers or null. No nested fields, arrays or line-item alignment. Unknown reference keys, duplicate IDs/keys and malformed inputs are rejected.
The free, self-contained CLI uses the same engine and requires Node 22 or newer. It reads two bounded UTF-8 JSON files, makes no network requests and writes report JSON to standard output. No installation or account is required.
node extraction-audit.mjs --expected reference.json --observed extractions.json > report.json node extraction-audit.mjs --expected reference.json --observed extractions.json --fail-on-errors > report.json
Exit 0 means a valid report, not a quality pass. With --fail-on-errors, exit 2 means a reference record did not pass or an unknown record ID was returned. Input/I/O errors exit 1 and do not produce a passing report. Set this option only when all ruled fields and zero unexpected records are your intended check.
Correct non-null values are true positives. A wrong value contributes both a false positive and false negative; an omitted expected value contributes a false negative; a value at a reviewed absence contributes a false positive. Unknown record IDs are retained separately, excluded from metrics. Unruled keys prevent a record pass but are excluded from per-field value scores.
The illustrative four-record example has nine correct values, one wrong value, one omission, one correct absence, four missing-run slots and one unexpected field key. Value precision is 90%, recall approximately 64.286%, F1 0.75 and all-slot accuracy 62.5%. Only one of four whole records passes.
Reference values are judgments you supply, not independently verified truth. This evaluator does not run OCR, call AI, validate source documents or certify readiness. Keep held-out cases and inspect error consequences.
Primary technical reference: Google Document AI evaluation methodology. This implementation uses the explicitly documented flat-field, absence, missing-run and matching rules; it does not reproduce another vendor's evaluator.
Document Desk extracts PDF text or English OCR and matches labels followed by a colon. Review conflicting candidates against the original. The free workspace handles one document, two pages; the $79 one-time offline edition adds batches. It is not a general-purpose AI document interpreter.
A separately quoted Elkwai Solutions pilot can connect intake, your chosen AI or deterministic parser, field validation, human review and downstream handoff. Agree on real reference documents, operating costs and acceptance criteria before expansion.
This evaluator and its offline CLI are free. They are separate from Document Desk and the Developer Assurance Kit; no extraction MCP tool is included in Kit 1.1.0. Using this page does not start a subscription, model call or paid engagement.