Qualification FAQ
Qualification has to earn trust. It is not something you are expected to accept on faith. These are the questions a skeptical engineering leader asks before trusting a score - answered directly.
Private beta. This page explains how qualification works when it opens. During the beta, no certificates are issued and no scores are published.
It lets you inspect practice records and run selected checks on pasted logs. It does not provide a released qualification, universal safety claim or certificate.
No. Throughput and completed runs measure activity. Quality needs a defined rubric, exact version, test conditions, critical-failure handling and inspectable evidence.
The intended protocol freezes test conditions before the comparison and separates training from held-out items and graders. That design still needs validation; this page is not proof of resistance to gaming.
A useful evaluation fixes the required outcomes in advance. Adaptive practice must not lower the held-out standard. No qualification is issued in this beta.
Keep held-out scenarios and answer keys out of trainee materials. Suspected leakage invalidates an improvement claim until fresh unseen work is tested.
No public qualification pass is issued here. A future standard needs declared outcomes, permitted actions, thresholds and critical failures before results are calculated.
No. Verifiable outcomes should use direct checks. Semantic judgments need explicit rubrics and consistency testing. A plausible judge response is not evidence that an external action happened.
Treat disagreement as unresolved evidence. Do not turn it into a pass. Record the disagreement and seek direct checks or human review.
An average can hide a fabricated completion, unauthorized payment or unsupported policy claim. Those failures need separate reporting rather than being averaged away.
Model, prompt, tools, permissions and workflow changes need fresh testing. This beta has no active certificates, expiry policy or supersession claim. Internal signed receipts are not qualification certificates.
No. Keep your internal tests, production monitoring and domain checks. This beta is a practice and evidence-review tool, not an independent certified standard.
There is no released qualification or customer pass record to show. You can try practice runs and inspect example audit output. A signed or hash-chained record shows a kind of integrity, not that the assistant is safe or passed a validated standard.
Capability check · September 29, 2026
Simulated practice runs, selected checks on pasted conversation logs, and a self-entered beta checklist. No certificates or public quality scores are issued.
Internal measurement separates deterministic simulation, local model-backed runs and paid model-backed runs. Before/after improvement, held-out transfer and later retention need their own evidence; more runs do not establish them.
Read the measurement status · Try the selected-check audit · Review the beta checklist