Legacy System Modernization With AI Agents: The Mainframe and COBOL Angle

AI agents are changing the economics of mainframe and COBOL modernization by automating the slowest, most expensive phases of the work: reading undocumented code, reconstructing lost business logic, and drafting equivalent modern services. They do not remove the need for human judgment, governance, or COBOL expertise. Used well, they compress timelines and cost; used as an unattended translation machine, they reproduce the same pilot failures that have plagued enterprise generative AI. The decision facing most leaders is not whether to apply agents to legacy estates, but where in the modernization lifecycle they create durable value versus new risk.

Key Takeaways

• The bottleneck was never syntax — it was knowledge. An estimated 250 billion lines of COBOL run in production, much of it undocumented, with the engineers who wrote it retiring. Agents help most by decoding and explaining that logic, not just translating it.
• Agentic modernization services are now shipping from major vendors. AWS Transform as an agentic service for mainframe workloads; Microsoft and GitHub have published production agent workflows for COBOL-to-Java migration.
• Vendor-reported results are real but scoped. Customers report large reductions in migration time and cost — but the headline gains cluster around discovery, code understanding, and boilerplate translation, not fully autonomous cutover.
• The failure pattern is well documented. MIT NANDA’s State of AI in Business 2025 found roughly 95% of enterprise GenAI pilots delivered no measurable P&L impact, largely because systems did not retain context or adapt. Legacy modernization amplifies this risk.
• Governance is the differentiator. Auditability, human validation of reconstructed logic, and preserved institutional knowledge separate a modernization program that lands from one that stalls.

Why mainframe and COBOL are the hardest legacy problem

COBOL is not a niche relic. IBM estimates roughly 250 billion lines of COBOL code in production across financial services, government, insurance, logistics, and retail — running the transactional systems that clear payments, adjudicate claims, and settle records. These systems are reliable and fast. The problem is not that they fail; it is that the organizations running them can no longer easily change them.

Three compounding factors make this the hardest category of legacy system modernization:

• Lost knowledge. Much of the code is decades old, sparsely documented, and encodes business rules that exist nowhere else. The people who wrote it are retiring, taking undocumented intent with them.
• Coupling and risk. Mainframe applications are deeply interdependent, and the workloads they run are the ones an enterprise least wants to break.
• Scale. Estates commonly run into the millions of lines of code, which makes manual analysis prohibitively slow and expensive.

For years, modernization stalled here because the expensive part was not writing new code — it was understanding the old code well enough to safely replace it. That is precisely where AI agents change the calculus.

What AI agents actually do in a modernization program

An agent, in this context, is an AI system that can plan a multi-step task, use tools (parsers, test harnesses, code repositories), and iterate toward a goal rather than answering a single prompt. Applied to a legacy estate, agentic workflows typically cover a lifecycle, and the value is uneven across it.

1. Discovery and dependency mapping. Agents ingest a codebase and reconstruct how programs, jobs, and data stores connect — work that historically consumed a large share of any modernization project’s timeline.

2. Code comprehension and documentation. This is the highest-leverage use. Agents generate natural-language explanations of what COBOL programs do, surfacing the buried business logic. IBM’s watsonx Code Assistant for Z and Microsoft’s published agent workflows both center on this explain-then-transform pattern.

3. Translation and refactoring. Agents draft equivalent code in a target language such as Java. AWS Transform as an agentic service for mainframe workloads, and Microsoft’s COBOL-to-Java Quarkus agents both automate this generation step.

4. Test generation and validation. Agents draft tests to check that modernized code behaves like the original — the step that determines whether a migration is trustworthy.

The pattern to internalize: agents are strongest at understanding and drafting, and weakest at guaranteeing correctness. The gain is concentrated in the earliest and most tedious phases. The final judgment — does this modernized system do exactly what the original did, including the edge cases no one remembers — remains human work.

The results are real, but read the scope

Vendors and their customers have published concrete outcomes, and they are meaningful. In AWS’s re:Invent 2025 materials, customers using AWS Transform reported results such as QAD citing 60–70% productivity gains and Thomson Reuters migrating 1.5 million lines of code per month with lower costs and reduced technical debt. These numbers should inform your business case — but read them precisely. The reductions concentrate in discovery, planning, code comprehension, and first-draft translation. None of them claim unattended, human-free cutover of mission-critical workloads. The productivity story is real; the “fire-and-forget” story is not. Anchor your own projections on the phases agents genuinely accelerate, and budget human validation as a first-class line item, not an afterthought.

Why most of these programs still fail — and how to avoid it

The uncomfortable context: modernization is being layered on top of an enterprise AI track record that is poor. MIT NANDA’s State of AI in Business 2025 found that roughly 95% of enterprise generative AI pilots delivered no measurable P&L impact. The report’s central diagnosis was not infrastructure, regulation, or talent — it was a learning gap: most systems did not retain feedback, adapt to context, or improve over time. Notably, the same research found that back-office and operational automation delivered the strongest returns, and that buying from specialized partners outperformed internal builds.

Legacy modernization inherits every one of these failure modes and adds its own. We have written before about why enterprise AI pilots fail; the mainframe-specific version comes down to four traps:

• Treating translation as modernization. Converting COBOL to Java line-for-line produces Java that thinks like COBOL — carrying forward the same undocumented assumptions, now in a language your team also doesn’t fully understand. Modernization means reconstructing intent, not just syntax.
• Skipping the validation gap. Without rigorous, human-reviewed equivalence testing, a plausible-looking migration is a latent production incident.
• Losing the humans in the loop. COBOL experts are not obstacles to automate around; they are the only source of ground truth for whether reconstructed logic is correct.
• No audit trail. In banking, insurance, and government, an unauditable modernization is a non-starter with risk and compliance.

The common thread is governance. The programs that land treat agents as accelerators inside a controlled, auditable, human-validated process — not as autonomous replacements for engineering judgment.

What enterprise leaders should govern before committing

If you own a legacy estate and a modernization mandate, these are the decisions that determine the outcome:

• Scope agents to comprehension and drafting first. The fastest, safest ROI is in discovery, documentation, and dependency mapping. Prove value there before automating translation of critical paths.
• Make validation non-negotiable. Require agent-generated equivalence tests plus human sign-off for every migrated component touching a critical workload.
• Preserve institutional knowledge as an asset. Use the comprehension phase to convert tribal knowledge into durable, living documentation that outlasts employee turnover — often the single most valuable byproduct of the effort.
• Demand a full audit trail. Every agent decision, transformation, and human override should be logged and reviewable.
• Keep COBOL expertise in the room. Plan for humans to define the transformation and adjudicate correctness, with agents doing the volume work between those bookends.
• Weigh partner-led delivery. Given the evidence that specialized partners outperform internal builds, staffing the effort with engineers who embed in your systems and own outcomes is a rational hedge — the model behind the forward-deployed engineer approach.

For a fuller view of how to build the underlying business case, see our enterprise AI agent ROI benchmarks for 2026.

The bottom line

AI agents make mainframe and COBOL modernization economically viable at a scale that manual approaches never could — but the gains are concentrated in understanding and drafting, and they evaporate without validation, auditability, and preserved human expertise. The enterprises that win will not be the ones that automate the most aggressively. They will be the ones that put agents to work on the tedious 80% while keeping disciplined human control over the 20% that determines whether the modernized system is actually correct.

If you are scoping a legacy modernization program and want to pressure-test where agents create value versus risk in your specific estate, start a conversation with our team.

Frequently asked questions

Can AI agents fully automate COBOL-to-Java migration?

No. Agents can automate a large share of the work — discovery, code comprehension, first-draft translation, and test generation — and vendors report substantial time and cost reductions. But mission-critical migrations still require human validation of reconstructed business logic and equivalence testing. The realistic model is human-supervised acceleration, not unattended replacement.

How much COBOL is still in production?

IBM estimates roughly 250 billion lines of COBOL code in production worldwide, across financial services, government, insurance, logistics, and retail. Other industry estimates run higher. Either way, COBOL remains foundational to critical transactional systems, which is why modernization is a strategic priority rather than a cleanup task.

Why do so many AI modernization pilots fail?

MIT NANDA’s State of AI in Business 2025 found roughly 95% of enterprise generative AI pilots delivered no measurable P&L impact, largely because the systems failed to retain context or adapt over time. In legacy modernization specifically, failures also come from treating translation as modernization, skipping rigorous validation, and removing the human experts who hold the ground truth about how the code should behave.

Do we still need COBOL developers if agents do the work?

Yes. COBOL experts are needed to define what a transformation should achieve and to judge whether reconstructed logic is correct — the parts agents cannot guarantee. The productive framing is that agents handle the high-volume, tedious work while experienced developers focus on intent, edge cases, and validation.

What is the safest place to start with agentic modernization?

Discovery, documentation, and dependency mapping. These phases deliver fast ROI, carry low risk because they don’t change production systems, and produce a durable asset: living documentation that captures institutional knowledge before it retires with your workforce. Prove value there before automating translation of critical paths.

Should we build this in-house or use a partner?

MIT’s research found that buying from specialized partners succeeded roughly twice as often as internal builds. For legacy modernization — where the stakes are high and the required blend of COBOL, modern architecture, and agent tooling is rare — an embedded, outcome-accountable delivery model is usually the lower-risk path.

Sources: MIT NANDA, State of AI in Business 2025; AWS, new agentic capabilities in AWS Transform; CIO Dive, AWS Transform aims AI agents at legacy workloads; IBM, What Is COBOL Modernization?; Microsoft, How We Use AI Agents for COBOL Migration and Mainframe Modernization; GitHub, How GitHub Copilot and AI agents are saving legacy systems.