Application recovery and deployment

What is being recovered?

This is not disaster recovery. Backups restore state after a loss. The situation here is a system that is running, serving customers, and effectively frozen, because the team that built it has gone, the build no longer runs on any machine anyone has, the deployment process lives in somebody’s memory, and nobody is confident enough to change anything.

Michael Feathers gave the useful definition: legacy code is code without tests, meaning code that cannot be changed safely because nothing protects its behaviour. Age is irrelevant. A system shipped last year with no tests and no reproducible build is in worse condition than a well-tested system from 2009.

Recovery restores the ability to change and deploy. That is the end state, and it is measurable.

The order is not negotiable

Most failed recovery projects begin with the interesting work. Someone starts refactoring, or upgrading dependencies, or writing the documentation, before the system can be built from a clean checkout. Everything after that point is unverifiable.

The build comes first. A build that runs from a clean checkout on a machine nobody configured by hand, producing the artefact that is currently deployed. Until this exists there is no way to know whether any change did what it was supposed to.

Then the inventory. What the running system actually depends on, discovered from the artefact rather than from documentation that may be years out of date. An analysed SBOM, produced by inspecting a deployed container image or binary, works precisely in the case where the build environment is long gone, which is the case at hand. Alongside it goes a discovery pass over the environment: services, scheduled jobs, queues, certificates, DNS records, storage.

Credentials come next, and the rule is simple. Every credential inherited with a system should be assumed to be held by people who no longer work on it, and rotated. Rotation also has a diagnostic effect, because the undocumented consumers announce themselves by breaking.

How do you test a system nobody understands?

Not by writing tests for correct behaviour, because nobody can say what correct is. Feathers’ answer is the characterisation test: a test that records what the code does today, without judging whether that is right.

The value is comparative. Once current behaviour is pinned, any change becomes visible as a difference rather than a hope. Where the code was never designed to be tested, seams provide the entry point — places where behaviour can be altered without editing that place, which is what allows a test to be wrapped around code that resists it.

Characterisation is applied selectively. Pinning every behaviour in a large system is neither possible nor useful. The paths that matter are the ones the business depends on and the ones the first planned change will touch.

The recovery sequence, and why each step blocks the next

1. Reproducible buildFrom a clean checkout. Until this works, nothing can be verified.

2. InventoryDependencies from the artefact; infrastructure from the environment.

3. Rotate credentialsAssume compromise. Breakage reveals the undocumented consumers.

4. CharacterisePin current behaviour on the paths that matter, before changing them.

5. Deploy and roll backExercise both on a trivial change before attempting a real one.

The way back matters as much as the way out

A deployment path is half a capability. The other half is rollback, rehearsed rather than documented, on a change small enough that failure costs nothing.

The measure of success is available in the client’s own numbers. How often the system deploys, how long a change takes to reach production, how quickly a failed deployment is recovered, and what proportion of deployments are unplanned responses to incidents. DORA’s measures were designed for delivery performance generally, and they work well as an objective end state for a recovery, because they describe capability rather than effort.

Recover, or replace?

The question cannot honestly be answered at the start, which is inconvenient for procurement.

A rewrite needs a specification of current behaviour, and producing that specification is most of the recovery work. Teams that skip recovery in favour of a rewrite usually discover this halfway through, at the point where the new system has to handle a case nobody knew existed. Doing the recovery first makes the replacement decision cheaper and better informed, and sometimes removes the need for it.

AI-assisted comprehension has genuinely changed the early phase. Tracing call graphs, summarising unfamiliar modules and drafting missing documentation are faster than they were, and Thoughtworks’ published work on legacy modernisation is the most visible example of the approach. What has not changed is verification. A generated explanation of an unfamiliar system is a hypothesis, and the characterisation test is what turns it into something safe to act on.

The standard we work to

An application recovery is complete when the software can be changed and deployed safely, not when it starts. A system that boots but cannot be modified has been restarted, not recovered, and the distinction is the whole engagement.