Decision Audit | Assistant GymASSISTANT GYM / FREE TOOL / ALPHA
Check the answer before you trust it.
Compare assistant replies against your expected answers. Find wrong values, missing fields and selected action patterns that conflict with supplied restrictions. Download a row-by-row audit you can reproduce.
Your data stays in this browser. No account, API key, server upload or model call. The tool does not know whether your answer key is correct and cannot prove an action actually happened.
Choose how to use it
Use in browser | Run locally
Use the browser for pasted replies or local files. Malformed JSONL rows are reported separately. Use the Node.js download for scripts and local invocation receipts. In the browser, a bad JSONL line is reported without stopping the other lines. The CLI stops on malformed input.
Download and run locally
Node.js 18+, no dependencies, account, API key or network calls. CLI supports --receipt-local for local invocation metadata; the browser version has no receipt flag. The CLI download is version 0.1.0. It is older than this browser tool and does not include all its parsing checks. Use this browser page for the checks described below; use the CLI only within its own published limits. Record the edition and version when comparing results. JSONL or a JSON array in, audit JSON out.
Download Decision Audit 0.1.0 | Usage and limits
tar -xzf decision-audit-0.1.0.tar.gz
cd decision-audit-0.1.0
node decision-audit.cjs cases.jsonl > audit.json
# Optional local receipt, only when you explicitly choose it:
node decision-audit.cjs cases.jsonl --receipt-local invocation.json > audit.json
The receipt stores only hashes and counts locally. No telemetry is sent. Exit 0 means an audit was produced, not approval or a verified action.
1. Add your replies
Malformed JSONL rows are reported individually while valid rows continue. A malformed JSON array stops because row boundaries are not reliable. Paste JSONL: one object per line, with case_id, expected and output. Output can be an object or a JSON text reply. Up to 1,000 rows or 2 MB. A JSON array works too, including a single-case array: [{"expected":{"n":1},"output":{"n":1}}].
Start with the sample, or use your own cases. A minimal case is shown below.
What counts as an expected-field match?
All fields in your expected object must exist and match exactly, including value types and array order. Extra fields are warnings. A PASS only means supplied expected fields match; a PASS can still have an action-risk flag. The tool also checks recognized action and execution fields and values using a fixed vocabulary. It applies only the restrictions supplied on that case, as explained below. It does not find or verify real approval. Numbers compare as JavaScript binary floating-point values, not decimal arithmetic. Nonfinite numbers and unsafe integers are rejected. One complete JSON object is accepted in plain text or a fenced block; multiple objects are ambiguous, including fenced plus unfenced objects. Empty expected objects are NO CHECKS, not passes. Duplicate IDs warn on every row in a repeated group; two rows sharing one ID count as two affected rows and one group. If you compare replies from several versions of a workflow, give each version a prefix, such as baseline-001 and revised-001. The duplicate warning checks IDs across the whole input; it does not know whether repeating an ID was intentional. With a restriction field such as owner_approved:false at the top level of a row, action:"prepare_action" can be flagged. A standalone prepare_action:true key is not an action field. Nested executed:true is checked separately. These input fields do not establish real approval. Unparseable output receives no action inspection; executed:true without an explicit restriction is not flagged.
One case you can paste
[{"case_id":"sample-001","expected":{"n":1},"output":{"n":1}}]Run this case to see a PASS: the supplied expected field matches. This is not a permission check. The sample button loads four examples and runs them.
2. Inspect the audit
Normalized JSONL reserializes your cases; it does not preserve the original file bytes or array format. A risk flag points to text to review. It does not prove an action happened, that a workflow is safe, or that a model learned from training. This is a comparison tool, not a legal, medical or financial approval.
Try a local fixture
Use the four sample cases or your own non-sensitive examples. The totals describe expected-field matches and text-review flags on the rows you supply. They are not a model score, proof of execution, or permission to act. A supplied expected answer is not verified by this tool.
Action-risk coverage and limits
- Use exact row keys
owner_approved and source_verified. Put them beside expected and output, not inside either object. With owner_approved:false, an output containing {"result":{"executed":true}} receives a review flag. A matching expected answer can still receive that flag. Setting an approval-related input field to true only changes the text check; it does not grant real permission. Missing keys do not imply restrictions. Present values are restrictive unless explicitly true, 1, "true" or "yes". Near-miss and nested gate keys warn; they are not gates. - Text cleanup handles some character variants, hidden formatting, punctuation and word boundaries, plus selected Cyrillic lookalikes. It does not cover every disguise or generally remove accents. Aliases include owner_approval, approved, user_approved, verified, requires_approval and approval.
- Decision objects inspect only their own type/name/action/decision fields. Decision arrays inspect the first element. Action, next_action, actions, steps and tool_calls plan arrays inspect every element. decision ["clarify","send"] is clean; decision ["send"] flags under a restriction.
- The tool looks inside nested output objects for recognized keys and values. For example, action: "prepare_action" is checked as a value under an action key. A standalone prepare_action: true key is not one of the recognized action keys. Execution keys such as executed: true are checked separately. The fixed vocabulary covers prepare_action, send_email, execute, approve, send and selected past-tense forms such as sent_email and emailed_customer. Common tool calls use exact normalized names. A status or result value such as "done" may mean a calculation finished, or it may claim a refund or message was sent. The tool cannot tell which. Check what that field describes and look for evidence of the real action; do not treat the flag as a verdict.
- The action vocabulary is a fixed list, not general language detection. Unlisted verbs and values may be missed. No flag does not mean permission or safety. Verify approval separately before executing.