Alpha / Synthetic and publicly available
What we tested.
What we changed.
What remains unproven.
These are development case studies using synthetic inputs. Public evidence below records specific tests, not customer outcomes or certification.
Case 1 / October 3, 2026Bill Check: manual CSV first
Source: buyer-test feedback, followed by our own live UI reproduction with synthetic inputs.
Before
Manual CSV fields followed a long unreliable-import explanation. The checkbox did not make its imported-draft scope clear. A synthetic sample ran without checking it.
After
Manual CSV is first. SAMPLE and IMPORTED draft states are explicit. Photo/PDF import is optional and still labelled not reliable yet. Edits invalidate confirmation and the download.
Retest
Live 390px and 1280px checks passed for manual-first layout, visible sample state, confirmation reset and no horizontal overflow. A synthetic text rebuild set IMPORTED state; an unchecked draft was blocked, a confirmed valid draft could be checked, and an edit disabled its download.
19 arithmetic core tests and 13 import regressions passed. The alternate CSV-header error wrapped fully at 390px; truncation was not reproduced in that case. Confirmation reset is a safety behavior, not a defect.
Not a photo-OCR validation or model-improvement result. No user conversion or receipt accuracy improvement is established.
Case 2 / October 3, 2026Assistant Audit: an empty check is not a clean result
Source: model-requested local tool use with fresh synthetic inputs. The first OpenAI request supplied an array instead of a conversation object. The tool returned "clean" after checking zero messages. The first Claude request omitted the log wrapper and failed. Neither was counted as completed use.
Before
Malformed or empty logs could produce a zero-message clean result, or fail without a useful input message. A previous downloadable report could remain available after invalid input.
After
The audit rejects wrong log types, empty messages, missing text, unsupported roles and user-only logs. The page says "Audit not run" with the required correction. Editing or failed input clears the previous report and disables download.
Retest
19 core checks passed, including all seven existing examples. At 390px and 1280px, wrong-array, empty-list and content-instead-of-text inputs were rejected. A valid one-message log flagged "Said done without showing proof". Edits invalidated the report; no horizontal overflow was found.
OpenAI, Anthropic and Groq model-requested schema retests checked one message and flagged the unsupported send claim. Their actual invocation model labels are recorded separately from synthetic log labels.
Input validation prevents these false empty results. It does not verify evidence, model identity, approval or actual execution. Synthetic model labels remain unverified data, not provenance proof.
October 3, 2026 / Local package 0.1.0Decision Audit: a real invocation route
Source: live downloadable package and model-mediated synthetic local runs. Four models requested fresh non-sample cases; a local runner downloaded and executed the Node wrapper.
Before
The browser tool had no documented local script/package route. Review opinions from product text were not completed use.
After
JSONL or an array can be piped into a dependency-free Node script. Audit JSON comes out. An explicit optional receipt stores counts and hashes locally, with no telemetry.
Completed operations
- OpenAI GPT-4.1 mini and Claude Haiku 4.5 requested fresh count-23 versus count-24 / execution-assertion cases. Each local run returned 1 failed row and 1 text-risk row.
- Gemma 3 1B and Qwen 2.5 3B authored fresh amount-47 versus amount-48 requests. Each local run returned 1 failed row and no text-risk row.
- All four wrappers exited 0 and created explicit local invocation receipts. Exit 0 means an audit was produced, not that the input passed.
The paid models put owner_approved below the row top level. The tool warned that this was not a restriction gate; expected.executed:false supplied the restriction. This points to clearer schema examples, not a model-learning claim.
Models requested calls through a local runner. This is not independent assistant installation or use of the ChatGPT app. Synthetic invocations do not establish demand, retention, correctness of answer keys, actual execution or runtime safety.
Keep the claims smaller than the evidence
No certificates or public quality scores are issued. We separate synthetic core tests, model-backed tests, actual tool invocation and independent use. A larger run count does not establish improvement.
Back to the tools