Prototype code is optimized for learning. Production systems are optimized for sustained service. Hardening is the deliberate work of preserving the insight of the prototype while replacing its hidden assumptions.
Define the operating envelope
Document supported users, inputs, languages, volume, latency, and decisions. Explicitly exclude uses the system has not been designed to handle. A narrow promise that holds is more valuable than a broad promise nobody can verify.
Build layers of defense
- Validate inputs before they reach the model.
- Constrain tools, permissions, and accessible data.
- Evaluate outputs for critical quality and safety rules.
- Route uncertain or consequential cases to review.
- Preserve the ability to disable, degrade, or roll back.
Reliability is not the absence of failure. It is the ability to detect, limit, and recover from failure.
Instrument the product
Capture enough context to reconstruct behavior: model and prompt version, relevant inputs, tool calls, latency, cost, output disposition, user correction, and downstream outcome. Protect sensitive data while retaining the evidence needed to investigate.
Release progressively
Begin with internal users or a constrained cohort. Compare against the existing process. Expand autonomy only when evidence supports it. Feature flags, shadow mode, and staged rollouts turn launch into a controlled learning process rather than a single irreversible event.
Assign operational ownership
Someone must own monitoring, incident response, content or policy updates, model changes, vendor changes, and feedback review. Runbooks should define who acts and what they can do.
The production roadmap is complete when the team can answer not only “Does it work?” but also “How will we know when it stops working, and what will we do next?”