Assistant Audit · beta

Review a conversation log.

Paste a conversation log to look for selected text problems. Examples include claims that work was done when the logged message has no evidence reference, spending claims without a confirmed amount, and possible secrets in the messages. The report points to messages to review; it does not verify permission, receipts or what happened outside the log. The audit runs in your browser and does not upload the conversation you paste into the audit box. The separate help chat sends messages you type there to the help service. Use only non-sensitive examples.

How to run a check

The log box starts with "The stray Chinese character", a larger dental-office example from the dropdown. To run the small email example shown here, copy its JSON into the log box below. Or paste your own log in that format.
Set the provider names you allow and any model families you want flagged. These settings check names written in the log, not the model that actually produced it.
Click "Run the audit". Review each finding, then download the report if you want to keep it. Editing the input clears the old report.

A small synthetic log

{
  "assistant": "Synthetic demo",
  "provider": "openai",
  "model": "gpt-4.1",
  "messages": [
    {"role": "user", "text": "Can you send the email?", "time": "10:02"},
    {"role": "assistant", "text": "I sent the email.", "time": "10:03"}
  ]
}

This example should produce a finding named 'Said done without showing proof': the assistant claims it sent an email without an evidence reference. It does not prove whether anything was sent. Each message needs a role and text; provider and model names can go on the message or at the top of the log. Use only non-sensitive examples.

Try an example: Approved providers: Blocked model families:

Format: {"assistant": "...", "messages": [{"role": "user" or "assistant", "text": "...", "provider": "...", "model": "...", "time": "...", "evidence": "receipt or message ID"}]} The evidence field takes a short reference only, such as a receipt number or message ID. Never put a full customer record, card number or password in it. The audit does not open the receipt or record you name, so it cannot confirm that the action happened.

What names does the log record?

The audit reads the provider and model names on each message, falling back to names at the top of the log. It flags missing names, changes of model within the conversation, and names outside your allowed settings. These are labels supplied by the log. They are not proof of which service or model wrote a message.

What the audit tells you

For each flagged issue you get the message it happened in, a plain explanation and a suggested next step. Each report has its own report ID. If nothing is flagged, the report lists the model and provider labels in the pasted record. That means none of these checks found a problem in the pasted log. It does not verify the conversation, the evidence references or the model labels, and it is not proof the conversation was free of mistakes, and the report is for your own review.

Capability check · September 29, 2026

What is available now

Simulated practice runs, selected checks on pasted conversation logs, and a self-entered beta checklist. No certificates or public quality scores are issued.

Internal measurement separates deterministic simulation, local model-backed runs and paid model-backed runs. Before/after improvement, held-out transfer and later retention need their own evidence; more runs do not establish them.

Read the measurement status · Try the selected-check audit · Review the beta checklist