The artefact that determines whether a change is ready to ship — recognising that deployment readiness is a gate (proceed / defer / rollback), not a document, and that the checklist's items are calibrated by the specific production failure modes the team has actually experienced rather than by generic best practices that don't reflect the system's real risks.
A primitive deployment readiness check is "the tests are green and someone approved the PR." This catches some classes of regression but misses many: the change passes tests but introduces an unmonitored failure mode, the change is correct but its rollback path was never exercised, the change is fine in isolation but interacts badly with another change shipping the same day, the change relies on configuration that exists in staging but not in production. Deployment-readiness-as-test-pass is a coverage instrument that reflects what the test suite catches, not what production actually requires.
A production deployment readiness checklist treats readiness as a gate with three stages. Pre-deploy verification — what must be true before deploy initiates: rollback path exists and is exercised, monitoring exists for the new behaviour, runbooks exist for the new failure modes, dependent teams have been notified, capacity has been validated for the new load. Deploy-time monitoring expectations — what signals must be tracked during deploy: error rate, latency, key business metrics, downstream signals, with documented thresholds for pause-or-rollback. Post-deploy verification — what must be true after deploy completes for the change to be declared shipped: signals returned to baseline, soak time elapsed, no unaddressed user reports, the change actually does what it was supposed to do (not just that it deployed without error). Each stage has documented items demanding specific evidence; failed items produce documented decisions, not "we'll address it later."
The architectural shift is not "we have a deploy checklist." It is: deployment readiness is a three-stage gate (pre / during / post) calibrated by the team's actual production failures, with each gate's items demanding specific evidence and each failed gate producing a documented decision — and treating readiness as test-pass-equivalent or as nice-to-haves produces deploys that fail the way the team's last several deploys failed.
The most consequential property of a deployment readiness checklist is whether its outcome matters. A checklist that produces a list of items the team marks off and then ships regardless is a document — useful as a record, not as a gate. A checklist that produces a decision — proceed with deploy, defer until items are addressed, rollback if the deploy is in progress — is a gate. The architectural discipline is to make the gate explicit and binding: the checklist is the input to a documented decision; failed items mean the decision is not "proceed." The decision authority is documented (typically a deploy lead, on-call engineer, or service owner); the override authority for proceeding despite failed items is documented and used rarely; the override is itself documented when it happens, with reasoning, so calibration can examine whether the override was justified.
Pick the most recent deploy in your organisation that had a non-trivial change. What did the readiness checklist produce — a list of marks, or a documented decision? If the answer is "we marked items and shipped," the checklist is documentation, not a gate — and the items that don't quite pass are being rationalised through rather than triggering the decision the checklist exists to support.
Continuous Delivery — Humble & Farley treats deployment gates as a primary engineering discipline. DORA Metrics — change failure rate operationalises the failed-deploy outcome as a tracked metric, providing the calibration signal that the checklist's gate is or isn't doing its job.
A common pattern: the readiness checklist is one undifferentiated list applied at deploy time. Items that should have been verified weeks before the deploy ("does a runbook exist for the new failure modes?") are confirmed at the moment the deploy is starting, when the answer is "we'll write one later." Items that should be monitored during deploy ("is error rate within tolerance?") are not a checklist item but a watchful-eye-of-on-call-engineer concern. Items that should be verified after deploy ("did the change actually do what it was supposed to do?") are forgotten once the deploy completes without error. The architectural discipline is to recognise that readiness has three stages with different concerns: pre-deploy verifies preconditions; deploy-time verifies in-flight signals against documented thresholds; post-deploy verifies outcomes. Each stage's items have different timing, different decision criteria, and different remediation paths.
Pick the most recent deploy of a non-trivial change. What was verified before deploy initiated, during deploy, and after deploy completed? If the verification was concentrated at deploy time only, the pre-deploy work is being skipped (rollback paths untested, runbooks missing) and the post-deploy verification is happening implicitly through "no incident reported."
Release It! — Nygard covers the multi-stage readiness discipline at practitioner depth. Production-Ready Microservices — Fowler provides a multi-stage readiness model for microservice deployments.
The pre-deploy stage is where most readiness work actually happens. The team that's ready to deploy a change has — before the deploy initiates — exercised the rollback path (in non-production at minimum), confirmed that monitoring exists for the new behaviour and the new failure modes the change introduces, written or updated runbooks for incidents the change might cause, notified dependent teams whose systems read or write data the change affects, and validated that capacity exists for the new load profile the change implies. A team that arrives at deploy time without these preconditions verified can either deploy anyway (and discover the gaps post-deploy when an incident reveals them) or defer the deploy (and pay the schedule cost of skipped pre-deploy work). The architectural discipline is to make the pre-deploy items explicit and to verify them as part of deploy planning, not as checkbox-marking at deploy time.
Pick a recent deploy that caused a production incident. Walk through the pre-deploy items: was rollback exercised before deploy? Did monitoring exist for the failure mode that triggered the incident? Did a runbook exist? If any of these were "no," the pre-deploy gate would have caught the gap if the gate had been applied — and the incident is the cost of skipping that work.
Continuous Delivery — Humble & Farley covers pre-deploy preconditions as primary deployment discipline. Google SRE Workbook — Production-Readiness Reviews operationalises the pre-deploy checklist pattern at practitioner depth.
During the deploy itself, the team's job is to monitor in-flight signals and decide whether to continue, pause, or rollback. Without explicit thresholds, this decision runs on the operator's instinct: "the error rate looks elevated but maybe it's just noise; let's see what happens." The architectural discipline is to make the thresholds explicit before the deploy starts: "if error rate exceeds X for Y minutes during deploy, pause; if it exceeds X for 2Y minutes, rollback." "If P99 latency increases by more than 50% over baseline for Y minutes, pause." "If a key business metric (orders/sec, signups/sec) drops by more than X% from baseline, rollback regardless of error rate." With thresholds defined ahead of time, the deploy-time decision is mechanical — apply the threshold to the signal, take the documented action — not a judgment call under pressure with the deploy in progress.
Pick the most recent deploy of a non-trivial change. What were the pause-and-rollback thresholds during deploy, and where are they documented? If the answer is "we watch the dashboard and use judgment," the deploy-time gate is operating on instinct — and the operator who's tired or distracted will miss what an alerted threshold would catch.
Spinnaker — Continuous Delivery Platform operationalises threshold-based deploy-time decisions as automated canary-analysis. Argo CD provides similar threshold-driven deploy-time controls. The conceptual framing is treated in Continuous Delivery — Humble & Farley.
A deploy that completes without immediate error isn't necessarily a successful deploy. The change may have introduced a memory leak that takes hours to manifest. It may have a race condition that only surfaces under specific traffic patterns. It may not actually do what it was supposed to do — it deployed, but the new behaviour isn't working. The architectural discipline is to verify the deploy over time: a documented soak period during which the deploy is monitored against deploy-time thresholds at lower vigilance; outcome verification that the change does what it was supposed to do (a metric the team can check, a user-facing surface that should now behave differently, a signal that the new code path is being exercised); explicit declaration that the deploy is complete only when both the soak elapses without breach and the outcome is verified. Until both are true, the deploy is in flight and rollback remains an option.
Pick the most recent deploy in your organisation. When was it declared complete, and what verified that it was actually doing what it was supposed to do? If the answer is "after the deploy ran and nobody complained," the deploy was declared complete on the absence of bad news rather than the presence of good — and changes that silently fail are being shipped as successes.
Release It! — Nygard treats post-deploy verification as a primary engineering discipline. Google SRE Workbook — Canary Analysis covers the soak-and-verify pattern with concrete examples.
A deployment readiness checklist drawn from generic best practices catches the deploys that fail in generic ways. It doesn't catch the deploys that fail in the team's specific ways — the failure modes that recur in this system because of its architecture, dependencies, traffic profile, or history. The architectural discipline is calibration by actual production failure modes: every deploy-related production incident produces a candidate revision to the checklist. The post-incident review asks "would the existing checklist have caught this?" If yes, the deploy gate failed to apply it; if no, the checklist itself failed to cover this failure mode and an item gets added. Over time, the checklist accumulates the team's institutional learning about its specific risks — and the same failure mode that caused last quarter's incident is unlikely to cause next quarter's.
Pick the most consequential deploy-related incident in your organisation in the last year. Was a candidate checklist revision proposed as part of its post-mortem? If yes, was it incorporated? If the answer is "we discussed it but didn't update the checklist," the calibration loop is broken — and the next deploy in the same risk class will surface the same gap.
Etsy — How to Conduct a Postmortem (Allspaw) treats post-incident learning generally; the same discipline applied to deploy-checklist calibration produces continuously-improving readiness gates. Accelerate — Forsgren, Humble, Kim provides empirical research showing that deploy-readiness calibration correlates with engineering performance metrics.
The diagram below shows the canonical deployment-readiness-checklist architecture: the three-stage gate structure (pre-deploy preconditions, deploy-time thresholds, post-deploy verification), with explicit decisions at each stage; the calibration loop where deploy-related incidents produce candidate revisions; CI/CD pipeline integration where the checklist surfaces in deploy tickets, change requests, and the deploy tooling itself.
Items are marked, the deploy proceeds regardless. Failed items don't trigger decisions. The checklist is a record, not a binding instrument.
Outcome of the checklist is a documented decision (proceed / defer / rollback). Decision authority and override authority are explicit. Overrides are tracked and reviewed.
Items that should have been verified weeks ago are confirmed at the moment of deploy. Items that should monitor in-flight signals aren't on the checklist. Items that should verify outcomes after deploy are forgotten.
Three-stage structure: pre-deploy preconditions, deploy-time thresholds, post-deploy verification. Each stage has its own items, timing, and decision criteria.
The on-call operator watches the dashboard and decides what looks bad. Thresholds aren't documented. Decision quality varies with who's watching.
Documented thresholds before deploy: error rate, latency, key business metrics — with windows for sustained breach and documented actions per threshold. Automated alerts on breach. The decision is mechanical, not a judgment call.
The deploy ran. Nothing exploded in the next five minutes. The deploy is declared done. Memory leaks, race conditions, and silently-failing changes ship as successes.
Documented soak period with continued vigilance. Outcome verification that the change does what it was supposed to do. Deploy is "in flight" until both soak and outcome pass; then explicitly declared complete.
The checklist was drawn from a blog post about deploy best practices. It doesn't include items about the failure modes the team has actually experienced. The same deploy-related incidents recur.
Calibration by actual production failure modes. Every deploy-related incident's post-mortem produces a candidate revision. Items reflect the system's specific risks, not just generic best practices.
Decision authority and override authority are explicit. Failed items produce documented decisions. Overrides are tracked across deploys.
Each stage has documented items with appropriate timing, decision criteria, and remediation paths.
Rollback exercised (date, timing), monitoring exists for new behaviour and failure modes, runbooks exist, dependent teams notified, capacity validated. Items demand specifics, not affirmation.
Deploy-doc, deploy-ticket templates include pre-deploy items. Authors confront items before deploy execution.
Error rate above X, latency P99 above Y, key business metric below Z. Transient spikes don't trigger; sustained breach does.
The deploy-time decision is mechanical, not a judgment call. Automated alerts surface breach; operator applies documented decision.
Soak duration matches change risk (typically 30 min to several hours). Deploy-time thresholds continue to apply during soak.
Specific signals: new endpoint receives traffic, new field populated, new metric emitted. "Deployed without error" is not "deploy verified successful."
The transition from "in flight" to "complete" is explicit, not implicit. Triggers cleanup of rollback scaffolding.
Calibration loop. Findings feed candidate revisions. Versioned and reviewed updates. The checklist accumulates institutional learning over time.