Your web browser is out of date. Update your browser for more security, speed and the best experience on this site.

Update your browser

Blog September 4, 2026

Your Legacy Systems Aren’t the Problem. Your Cost of Change Is.

Reese Edgington
Author
Reese Edgington

The metric that matters most in the age of AI, and how to measure it

Starting a new fiscal year as a State Agency Chief Information Officer (CIO) always came with the same quiet dread. Sometime in the next 12 months, a federal mandate, legislative change, or court decision was going to land on my desk, and my agency was going to have to respond. Fast. The question was never whether it would happen. It was how much it would hurt.

In my years as Chief of Technology Services at North Carolina Department of Health and Human Services (DHHS), I learned that the pain of government technology is rarely about the technology sitting still. It’s about how hard it is to move. And that is exactly what most of the current conversation about AI in government is missing.

We’re talking about the wrong AI

Walk into almost any agency today and the AI discussion sounds the same: chatbots for citizen interaction, assistants for benefits and eligibility, copilots for case management. These create real value, and I’m not dismissing them. But notice what they have in common? They are AI capabilities added to the finished system. They make the front door friendlier. They do nothing about the fact that changing what’s behind that door still takes 18 months, a fleet of SMEs, and a stack of vendor change requests.

There’s a clock on this that the chatbot conversation ignores entirely. The people who understand how our oldest systems actually work, both the code and the policy logic behind it, are retiring. When they walk out the door, decades of undocumented institutional knowledge goes with them. A friendlier citizen chatbot does nothing about that. What about using AI to read those systems, extract their rules, and capture that knowledge before it’s gone? That’s the opportunity we should be racing toward.

But there’s an even bigger opportunity, and no, it’s not merely adding AI to your systems. It’s using AI to change how you build and modify them. And to do that correctly, we need to stop measuring the wrong things and start measuring one right thing.

Cost of Change is the real metric

Engineers and the agile community have talked about the Cost of Change for decades, typically as it relates to software. The adage goes: the longer you wait to make a change, the more it costs you. I want to borrow the idea and move it up a floor, into the CIO’s chair, because it turns out to be the truest measure of something we rarely put a number on.

Most agencies can tell you their IT budget, their maintenance spend, and their vendor costs down to the dollar. Very few can tell you what it costs to implement a moderately complex policy change. That number, not your maintenance ratio, is the truest measure of your agency’s agility.

I define it simply: 

Cost of Change = the time, people, effort, and risk it takes to safely implement a policy or regulatory change.

It’s not just a financial figure, but a measure of time, people, complexity, and risk. And the reason it matters is that this is the number that determines whether you can keep up with the policy environment you actually live in. Policy changes annually. If your technology takes years to catch up, you’re always governing on a delay.

The hardest part isn’t the coding

Here’s the part I want every CIO to sit with. In my experience, the single hardest thing to get right on any project is not the coding or the testing. It’s eliciting the requirements. Getting an accurate, complete picture of what a system actually does, and what it now needs to do, is where projects quietly succeed or fail. And on a legacy system, that job is brutal, because the real requirements aren’t written in a document. They’re buried in decades of code that only a shrinking few can read, and reconstructed through interviews with people who half-remember why a rule was written 20 years ago.

The way I think about it is this: The truth of how that system behaves is already written down. It’s in the code. Our problem has never been one of missing knowledge. It’s that we can’t read it fast enough, so we rebuild it by hand, one interview at a time. This is precisely where AI changes the economics. Point it at the system and it extracts the business rules, maps the dependencies, and hands your analysts a factual starting point. Requirements elicitation ascends from archaeology to validation. You’re no longer guessing what the system does and asking people to confirm your guess. You’re confirming what the code already told you, and spending your SMEs’ soon-to-be-retired expertise on judgment instead of recall.

A framework you can use

You don’t need a consultant to start thinking this way. Here’s a simple scorecard you can run yourself, on four dimensions, in an afternoon.

Start by picking one system. Not your agency as a whole, and not your newest cloud platform. Pick the system that would hurt the most to change: the one carrying your heaviest caseload, running on your oldest technology, understood by the fewest people. For many of us, that’s a mainframe old enough to have its own pension, with business rules written in a language we’re no longer hiring for, understood by two people who are both eligible to retire.

Then score that system. Averaging your whole portfolio will only flatter you. Your exposure lives in the systems you’re most afraid to touch, and that’s exactly where AI-assisted delivery has the most to offer. With that system in mind, rate it honestly on each of the four dimensions below, from 1 to 5.

1. System Understanding. When a change lands, how quickly and independently can you determine what it affects?
  1. We’re reactive. We rely on a few people’s memory, and if they’re out, we stall. Impact analysis takes weeks.
  2. Some documentation exists, but it’s stale. We still lean hard on SMEs.
  3. Documentation is current for major systems. A trained analyst can trace impact in days.
  4. Dependencies are mapped and searchable, and impact analysis is largely tool-assisted.
  5. AI-assisted discovery extracts business rules and maps dependencies on demand. Understanding takes hours, not weeks.
2. Development Effort. Once you know what to change, how much effort does it take to make it safely?
  1. Every change is bespoke, with little reuse. Small changes routinely become large projects.
  2. Some reusable components, but most work is custom and slow.
  3. Modular enough that we can contain most changes to known areas.
  4. Developers are tool-assisted. Boilerplate and refactoring are accelerated.
  5. AI-assisted development drafts and refactors against known rules. Build effort is a fraction of what it was.
3. Testing and Validation. How much effort does it take to prove a change is safe to deploy?
  1. All manual. Regression testing is a scramble every release.
  2. A handful of automated tests. Most validation is still manual.
  3. Core paths are automated. Manual testing covers the edges.
  4. Broad automated coverage. Regression runs on demand.
  5. AI-generated test scenarios plus high automation. Validation keeps pace with change.
4. Knowledge Capture. When the work is done, does the knowledge stay with the agency or walk out the door?
  1. Knowledge lives in people’s heads. Documentation is an afterthought.
  2. Documentation happens sometimes, but usually too late to help.
  3. Documentation is required and reasonably current.
  4. Documentation is generated as part of delivery.
  5. AI captures rules and decisions as a byproduct of the work. Institutional memory no longer retires when your people do.

A note on what’s not scored here. Governance reviews, procurement cycles, and vendor timelines are real costs of change, sometimes the largest. I left them off on purpose, because they are driven by policy and process that sit largely outside your delivery organization’s direct control. Those costs are real. Hold them constant for now. This scorecard measures the four places where your own delivery capability, aided by AI, most directly lowers your Cost of Change today.

What your score means

Add the four dimensions for a total landing between 4 and 20.

  • 4 to 8, High Cost of Change. Every mandate is an emergency. Change is slow, expensive, and dependent on a handful of people. This is where most legacy-heavy agencies sit, and it’s your single biggest risk as SMEs retire.
  • 9 to 13, Moderate Cost of Change. You’ve documented and standardized, but the effort is still mostly manual. This is where AI has the most leverage, and the fastest payback.
  • 14 to 17, Low Cost of Change. You’re tool-assisted and moving at a reasonable pace. AI extends your lead.
  • 18 to 20, Adaptive. You change at the speed of policy. Rare in government today, and worth protecting.

Anything under 14 means you have real, measurable friction, and specific room for AI to lower your Cost of Change. Your lowest single dimension tells you where to start.

What this looks like in the real world

Imagine the legislature passing a bill that changes who qualifies for one of your agency’s core programs, and it takes effect at the start of the next fiscal year. In a traditional environment, the first weeks are spent just answering questions we should already know: Which systems are impacted? Which interfaces break? Which reports and notices are affected? Who still understands the original eligibility rules? The actual coding is often the smallest part. The months and dollars go to discovery, SME hunting, dependency tracing, and regression testing.

Now imagine AI doing the heavy lifting on that front end. Automated system discovery. Extracting business rules from the code itself. Mapping dependencies in hours instead of weeks. Generating documentation as a byproduct. Drafting test scenarios automatically. You haven’t replaced your people, you’ve just taken the friction off their plate so they can focus on judgment instead of archaeology.

That’s the shift. AI is not the product. Lower Cost of Change is the product.

Where to start this quarter

You can begin without a budget request or a procurement action. Start with your most feared system, the one nobody wants to touch, and run it through the four dimensions above. Then ground the score in your own history. Think back to your last significant change on that system and put real numbers to it. How many people did it take? How many months? How much of it depended on one or two individuals who could have retired mid-project? That is your baseline, and most CIOs find the honest version of it sobering.

Once you can see where your friction lives, you can start asking the harder question of your own organization: what would it take to move the lowest of those four scores up by even one level, and what is it costing us every year that we don’t?

Where CapTech fits

Scoring your own system with the framework above will tell you where your friction lives. For most agencies, the lowest scores cluster in the first dimension: understanding the system well enough to know what a change will touch. That’s the archaeology I described earlier, and it’s the single biggest driver of Cost of Change on a legacy platform.

That’s the problem we built Legacy Lift to solve. It uses AI to read the code of a live legacy system and recover what’s buried inside it: the business rules, the logic, the dependencies that today live only in aging documentation and a few people’s memories. Instead of reconstructing requirements through months of interviews, your team starts from what the system actually does. It’s a direct attack on your highest-cost dimension, and by extension, on your Cost of Change.

If that’s a challenge you’re wrestling with, I’d welcome a working conversation about where your friction really lives and whether an approach like this would help your agency.

Because in the end, this was always the job. Not chasing the newest technology, but building an agency that can innovate, solve problems, and create value at the speed the mission demands. The CIOs who win the next decade will not be the ones with the most AI. They will be the ones who made change cheap enough to keep their promises to the people they serve.

So I’ll leave you with the question I opened with. If a major mandate landed tomorrow, do you know your agency’s Cost of Change? Score your most-feared system this week. If the number surprises you, let’s talk.

Reese Edgington

Managing Director, Public Services

Reese is an IT Executive with 26 years of experience and a proven track record of leading large-scale IT transformations, leveraging innovative strategies, and delivering exceptional outcomes. He is an expert in aligning technology initiatives with organizational objectives, driving efficiencies, and spearheading multimillion dollar projects across diverse sectors.

Envelope

Related Insights

Articles

Rethinking AI in Government Technology Delivery

Articles

2026 Executive Research: Why AI Stalls

Articles

2026 Tech Trends: The Only Constants Are AI and Change