AI Production Readiness Checklist: Pilot to Production
A good demo is weak evidence that an AI workflow is ready for production. The demo uses a clean input, a patient operator, and no concurrent failures. Real work brings incomplete records, changing APIs, impatient users, sensitive data, and actions that cannot be undone. A readiness checklist turns that gap into a launch decision.
The goal is one evidence packet per workflow: the business result, evaluation results, operating limits, security controls, an accountable owner, a staged rollout plan, and rollback authority. An unresolved red item pauses the launch. A passing packet earns a small canary, and the same evidence is then collected again with real traffic. This guide also covers two things most checklists leave out: how many clean cases a canary needs before it proves anything, and when the evidence stops being valid.
Why pilots pass while production workflows fail
A pilot answers a narrow question: can the model complete the happy path? Production asks several harder ones at once. Can the workflow finish the job across representative inputs, and notice when it lacks information? Can it fail without duplicating a payment, message, or record update? Can an operator explain what happened, can spend and latency be capped, and can a named person stop it?
Model quality covers only part of that. A better prompt cannot supply database isolation, idempotency, rollback, ownership, or an incident path. The Ask HN discussion about agents stuck at proof of concept reports inconsistent decisions, opaque reasoning, tool failures on edge cases, incomplete outputs, and weak handling of missing information. Treat that thread as practitioner evidence, not a failure rate. It still shows why a clean demonstration is weak acceptance evidence.
The failure often starts before engineering. A team says the pilot should be "accurate" or "save time" without defining the baseline, the acceptable error rate, or the unacceptable outcome. Every later review turns subjective: product likes the fluent output, engineering worries about edge cases, security asks for a data map, and operations has no staffing plan. Fix that by making readiness a recorded decision with named evidence. A workflow that saves ten hours a week but can expose another customer's data is not 80 percent ready, so do not hide a critical failure inside a blended score.
Define the production contract before adding features
Write a one-page contract that product, engineering, security, and the operating team can review together. Record these fields:
- Business job: the exact unit of work the workflow accepts and completes.
- Population: users, tenants, languages, document types, and systems included at launch.
- Baseline: current time, cost, quality, and error rate for the manual or existing process.
- Success target: one primary KPI and its measurement window.
- Unacceptable outcomes: events that block launch regardless of average performance.
- Decision authority: who approves, pauses, rolls back, and restarts the workflow.
- Operating owner: who receives alerts and reconciles failed or uncertain work.
- Exposure plan: shadow, internal canary, limited production group, then wider rollout.
Take an invoice-intake pilot. "Extract invoices accurately" is not a contract. A useful one says the workflow accepts English PDF invoices from three approved mailboxes, extracts six required fields, and stages a draft payable for human approval. It must never post a payment, merge vendors across tenants, or treat a low-confidence total as valid. The primary KPI is median handling time per invoice, compared over two weeks with the existing process.
The contract also stops scope from growing during review. If handwritten receipts are outside the launch population, they should not delay launch, and the workflow must reject or route them instead of pretending to support them.
Build a representative evaluation set
Evaluation data should look like the work the system will receive, including the cases that make operators slow down. OpenAI describes evaluations as tests of model outputs against criteria you specify and recommends a cycle of describing the task, running test inputs, and analyzing results. Use any suitable tooling, but keep that discipline.
Freeze a versioned set before the go-live review. Include ordinary cases, boundary cases, known historical failures, malformed inputs, missing fields, conflicting evidence, and attempts to induce unsafe behavior. Remove or transform sensitive data according to the project's data policy, but preserve the features that made each case difficult.
Each case needs an expected disposition, not necessarily one exact sentence. Useful dispositions include:
- complete automatically
- request missing information
- route to a human
- reject as unsupported
- stop before a side effect
- retry safely after a transient failure
Measure the complete workflow. A correct model response followed by a duplicate tool call is a failed case. A refusal that prevents legitimate work is also a failure, even if it looks safe in a model-only test.
Report task success, unsafe outcomes, human-review rate, latency, and cost separately, with counts and denominators. "92 percent passed" hides whether the failures were formatting slips or unauthorized actions. Every unacceptable outcome should have a count of zero in the frozen set and in the canary, and the next section explains what a zero is worth.
Test the workflow around the model
Write deterministic tests below and around model inference. The model can vary while the execution boundary stays strict.
Test authentication and authorization with the real identity path. Verify that one tenant cannot retrieve another tenant's context or invoke its tools. Confirm that credentials never enter prompts, traces, or generated output. Review every external destination and data store against the system's data map.
Test side effects with duplicate delivery, retries, concurrent work, stale approvals, timeouts, and uncertain responses. Every mutation needs a durable operation identity. If the provider times out after receiving a request, reconciliation should determine whether the action happened before another attempt runs.
Test limits. Set maximum model calls, tool calls, elapsed time, input size, output size, and spend per unit of work. OpenAI's production guidance covers access controls, key protection, spend limits, scaling, and operational planning. A workflow also needs its own per-run budgets so one loop cannot consume the team's monthly allowance.
Test degraded dependencies. The workflow should tell apart a rate limit, timeout, invalid response, unavailable database, rejected authorization, and policy denial, because each needs a different recovery action. "Something went wrong" cannot reconcile business work.
Tie the security review to the actual architecture. The OWASP GenAI security project maintains a current risk list. For each applicable risk, record the trust boundary, control, negative test, owner, and remaining exposure, and mark non-applicable items with a reason instead of omitting them.
What a clean canary proves
"Zero unacceptable outcomes in the canary" sounds like a pass. With a small sample it says very little. The statistical rule of three states that when n independent cases produce no failures, about 95 percent confidence puts the true failure rate below 3/n (see this explanation; the rule goes back to work reviewed by Tuyl, Gerlach and Mengersen). Thirty clean cases are consistent with a true rate near 10 percent. One hundred clean cases bring the bound to about 3 percent. To show with that confidence that an unacceptable outcome happens in under 1 percent of cases, you need about 300 clean cases.
Use that arithmetic when you set the canary size, and write the claim you are making next to it: "no more than 1 percent, with 95 percent confidence." If the workflow will not see 300 cases in the canary window, the honest conclusion is that you have not measured the rate. Keep a human approving the consequential step until volume catches up, rather than promoting on elapsed time. The rule assumes independent, representative cases, so a canary of the ten easiest documents counts for less than its size suggests.
Readiness expires when the system changes
The sign-off covers one model version, prompt, tool set, and policy. A change to any of them is a new system. Tie the re-check to the change instead of to the calendar.
The SEC's order on Knight Capital shows what a skipped re-check costs. In 2012 a new trading feature was deployed by hand to seven of eight servers. The eighth kept old, long-unused code that a repurposed flag then woke up, and in about 45 minutes the firm sent 4 million executions in 154 stocks and lost more than $460 million. The order records that no second technician reviewed the deployment and that Knight had no written procedure requiring one. It also records that 97 automated error emails, which Knight's staff generally did not review because the system was not designed as an alert, arrived before the market opened. Three checks transfer directly to an AI system: confirm that every running instance reports the prompt, model and tool versions you signed off, treat reused flags and dormant tools as new behavior, and route the warnings your system already emits to something a person must acknowledge.
Use a readiness scorecard that can block launch
Give each category green, amber, or red, and define the colors before the review.
| Category | Green evidence | Red blocker |
|---|---|---|
| Business value | KPI, baseline, target, and measurement owner are recorded | No baseline or no way to measure the result |
| Evaluation | Frozen representative set passes stated thresholds | Unacceptable outcome occurs or key population is missing |
| Reliability | Retries, idempotency, timeouts, and reconciliation tests pass | A retry can duplicate or lose a side effect |
| Security and privacy | Data map, authorization tests, secret handling, and risk review pass | Cross-tenant access, exposed secret, or unresolved critical risk |
| Operations | Dashboards, alerts, runbook, owner, and support hours are named | No operator or no path for uncertain work |
| Economics | Per-item cost and hard run budgets fit the business case | Cost cannot be bounded or erases expected value |
| Rollout | Canary group, thresholds, duration, and rollback action are defined | No safe exposure boundary or rollback authority |
Green means evidence exists and the approver accepts it. Amber means the issue has a bounded workaround, owner, and deadline. Red means no launch. Never average a red into a passing total.
The sign-off record should name the workflow version, model and prompt versions, evaluation-set version, enabled tools, launch population, approved limits, known amber items, approvers, and timestamp. The team can then identify which system was approved when a later prompt, model, tool, or policy changes.
The NIST AI Risk Management Framework is designed to incorporate trustworthiness into the design, development, use, and evaluation of AI systems. A small team does not need the whole framework in a launch meeting. Keep the practice: identify the context, measure the relevant risks, assign accountable controls, and review as the system changes.
Move through shadow mode and a canary
Do not go from the test set to all users. Use stages that produce new evidence without exposing every business process.
In shadow mode, copy eligible inputs to the new workflow without allowing it to create production side effects. Compare its proposed disposition with the existing process. Record disagreements and review a sample from both agreements and disagreements. Shadow mode suits decision and extraction workflows. It cannot prove how real mutations, permissions, or user interactions behave.
Next, launch a small canary on a coherent group such as one internal team, one low-risk tenant, or one document class. Avoid a group of expert users who can compensate for defects, unless experts are the real launch population.
Google SRE defines a canary as a partial, time-limited deployment evaluated to decide whether rollout should proceed. Name the canary duration, minimum case count (see the arithmetic above), baseline, promotion thresholds, and stop conditions before exposure starts.
A canary dashboard should show:
- work received, completed, rejected, and awaiting review
- unacceptable outcomes and authorization denials
- human overrides and their reasons
- latency and cost by completed unit
- retries, timeouts, and uncertain side effects
- user or operator complaints
- comparison with the baseline KPI
Pause automatically on a critical security event, an unacceptable business action, missing audit evidence, or loss of the reconciliation path. A human should also be able to pause immediately. Promotion should require a recorded review, not the absence of an alarm.
Plan rollback as a business operation
Rollback has to cover work the newer version already created, not only the deployed code.
Document how to stop new admission, disable side-effecting tools, preserve in-flight evidence, identify records produced by the canary, and reconcile uncertain outcomes. State whether queued items can run under the previous version or must remain quarantined. Keep the old execution path available only if its data and state contracts remain compatible.
Assign rollback authority to a role that is reachable during launch hours. A workflow sending duplicate messages cannot wait for a six-person meeting. Separate the authority to pause from the authority to resume. Restart should require evidence that the cause is understood, affected work is reconciled, and the repaired version passed the relevant regression cases.
Run one rollback exercise before launch. Trigger a synthetic stop condition, pause admission, inspect in-flight work, revert the canary, and prove that no duplicate side effects occur. Record the elapsed time and every manual dependency discovered.
Hold a seven-day production review
Schedule a review after seven days or after a defined case count, whichever comes later.
Compare production results with the frozen evaluation set and baseline. Review every human override, policy denial, uncertain outcome, and complaint. Check whether input mix differs from the pilot. Confirm actual cost and latency distributions rather than relying on pre-launch estimates. Add new failure cases to the evaluation set before changing prompts or models.
The review should produce one of four decisions: widen exposure, hold at the current canary, narrow the supported population, or roll back. Record the evidence and owner for each follow-up. A week with ten easy cases is not enough to promote when the contract expected hundreds of varied documents.
Run the go-live gate on one workflow
Take the workflow closest to production. Write its business job, launch population, KPI, unacceptable outcomes, owner, and rollback authority on one page. Freeze representative cases, run them through the complete system including tools and failure paths, and fill the scorecard with links to evidence.
If every category is green or has an accepted amber workaround, run shadow mode and then a bounded canary. If any is red, stop and assign the missing control. Keep the completed checklist with the launch record so the next model, prompt, tool, or policy change is tested against the same boundary.
References
- Ask HN: blockers beyond proof of concept is practitioner evidence about reliability, repeatability, tool-edge-case, and trust problems in production agents.
- OpenAI working with evals supports criteria-based model-output evaluation using repeatable test inputs and analysis.
- OpenAI production best practices supports access, secret, spend, scaling, and operational planning for production API use.
- OWASP GenAI LLM risks archive provides the maintained list of security risks used to scope system-specific checks.
- Statology, a concise guide to the rule of three states the 3/n bound after zero failures.
- Tuyl, Gerlach and Mengersen, The Rule of Three, its Variants and Extensions reviews the rule and its extensions.
- NIST AI Risk Management Framework supports accountable trustworthiness and risk management across the AI system lifecycle.
- Google SRE canarying releases supports partial, time-limited exposure evaluated against rollout criteria.
- SEC, In the Matter of Knight Capital Americas LLC supports the partial-deployment failure, the missing second review and the unreviewed warning emails.