Prefer listening over reading? Press play and enjoy
Recently, Codurance organised a code retreat in Manchester that explored a question facing many engineering leaders: how do AI coding tools perform when tackling the legacy codebases that dominate most organisations' technical estates? This article shares our initial findings and their implications for teams working on modernisation initiatives.
The Reality of AI on Legacy Code: How a controlled crossover experiment reveals the real impact of AI tooling on legacy codebases.
Testing Against Complexity: Why AI must be tested against technical debt, low test coverage, and complex dependencies, not just greenfield examples.
Accelerating Modernisation: Where AI genuinely accelerates delivery, from onboarding and codebase analysis to rapid test generation.
Strategic Risks: The risks of over-reliance on AI, including skill degradation, reduced architectural understanding, and new forms of technical debt.
The Human Role: When human-led engineering practices remain essential for safety, quality, and long-term ownership.
Leadership Balance: How engineering leaders can balance AI-driven acceleration with capability preservation and governance.
Future Insights: Early findings that will inform Codurance’s forthcoming report on AI-enabled modernisation strategies.
We designed a controlled experiment using a crossover methodology to eliminate bias. Participants were divided into two groups, both working on intentionally complex legacy codebases that mirrored real-world technical debt: high coupling, sparse test coverage (under 5%), and embedded defects. Working in pairs, each group had 2.5 hours per challenge to increase test coverage, resolve defects, and deliver new features.
Group A used AI tooling (Claude Code, Cursor, or GitHub Copilot) in the morning session, then worked without AI assistance in the afternoon. Group B reversed this approach. This ensured every participant experienced both conditions on comparable challenges, providing first-hand insight into the practical trade-offs.
Most AI coding demonstrations focus on greenfield development, yet this rarely reflects the reality your teams face. The majority of engineering effort goes into maintaining and evolving existing systems. For AI tools to deliver genuine ROI in enterprise settings, they must handle the messy reality of production codebases: understanding tangled dependencies, navigating unfamiliar patterns, and enabling safe refactoring.
This experiment directly tested whether AI tools can accelerate the modernisation work that consumes significant portions of many organisations’ engineering budget, or whether their value remains limited to prototype and greenfield scenarios.
The event surfaced several insights relevant to tooling decisions and team capability development. The crossover format proved particularly valuable, as participants could directly compare their productivity and confidence levels under both conditions. The retrospective discussions revealed nuanced observations about when AI assistance delivers measurable value versus when traditional engineering practices remain essential.
"Over-dependence on AI assistance could erode core engineering competencies"
One experienced developer noted that working on the afternoon challenge without AI support exposed how reliant he had become on the tooling. This observation has significant implications for capability planning. Over-dependence on AI assistance could erode core engineering competencies: reading complex code, navigating large systems, identifying architectural issues, and spotting defects. For organisations investing in AI tooling, this suggests a need for deliberate practices to maintain fundamental skills.
"AI-assisted analysis significantly reduces the time for codebase comprehension"
AI tools demonstrated clear value in reducing the time required to understand unfamiliar codebases. Using well-constructed prompts, the agentic capabilities of these tools systematically analysed the code, producing comprehensive summaries of architecture, technology choices, and potential defects.
This capability directly addresses one of the highest costs in modernisation projects: the ramp-up time for teams inheriting legacy systems. Codurance has observed similar results in software modernisation engagements, where AI-assisted analysis significantly reduces the time for codebase comprehension. For leaders evaluating AI adoption, this represents a clear use case with measurable time savings.
“The potential to compress months of test-writing into weeks could fundamentally alter modernisation economics.”
Both codebases featured minimal automated test coverage, deliberately reflecting the technical debt common in legacy systems. Without adequate tests, refactoring becomes high-risk and slow. Manually building comprehensive test suites represents a substantial investment of engineering time.
Participants using AI tools, particularly those with agentic features like Claude Code, rapidly generated missing tests that followed existing patterns. Whilst these required review and refinement, they provided the safety net necessary for confident refactoring. This transformed test coverage from a blocking constraint into an achievable milestone within the session timeframe.
For organisations facing modernisation backlogs, this capability warrants serious evaluation. The business case hinges on whether AI-generated tests are sufficiently reliable to enable faster, safer changes. The potential to compress months of test-writing into weeks could fundamentally alter modernisation economics.
"Over-delegation can weaken architectural ownership, reduce engineers’ ability to reason about the system, and create hidden dependencies on the tooling’s decisions."
One group chose to let the AI tooling “take over completely” to save time. While this initially felt productive, it quickly created a sense of losing control over the codebase: participants struggled to explain why certain design choices had been made, how components fitted together, or how they would safely adapt the system later.
For engineering leaders, this highlights a growing organisational risk: over-delegation can weaken architectural ownership, reduce engineers’ ability to reason about the system, and create hidden dependencies on the tooling’s decisions. Teams may move faster in the short term, but without structured review practices, shared understanding, and human-led decision-making, they risk accumulating a new form of AI-driven technical debt; subtle, difficult to unpick, and expensive to remediate.
Many organisations are now looking for experienced partners to help design AI-augmented delivery practices that accelerate modernisation without compromising long-term control, an area where Codurance is already supporting several engineering teams.
We are conducting detailed analysis of the final codebase states from the retreat, comparing outcomes between groups and sessions. These findings will be combined with insights from Codurance's client delivery teams who are actively using AI in production modernisation projects.
The forthcoming report will provide specific recommendations on integrating AI tooling into modernisation strategies, including guidance on measuring ROI, managing risks, and structuring teams for optimal results. This will build upon Codurance’s earlier publication, written by myself, "Beyond the Hype: Strategic AI Code Adoption for Technology Leaders”.
For technology leaders evaluating whether AI tools merit investment for legacy system work, this experiment suggests the answer is nuanced: significant value exists in specific domains, but success requires thoughtful implementation and ongoing attention to skill preservation.
Codurance helps leading organisations modernise legacy systems, build adaptable architectures, empower engineering teams, and enable safe, confident experimentation with AI. Our focus is simple: to help you build software that’s easy to change, easy to trust, and ready for the future.
Alongside this, our Data & AI Readiness Assessment (DARA), provides clear, independent recommendations to strengthen your data strategy and accelerate AI adoption in a structured, effective way.
Get in touch with us to speak to one of our specialists.
1. How does AI impact legacy software modernisation?
AI can help accelerate modernisation by automating codebase analysis and rapidly generating tests for systems with low coverage. However, Codurance's research indicates that without human oversight, it can introduce new forms of technical debt and weaken a team's architectural ownership.
2. What risks should engineering leaders watch for when adopting AI?
The primary risks are skill degradation and loss of control. Over-reliance on AI tools can erode core engineering competencies (such as debugging and reading complex code) and lead to "hidden dependencies," where engineers struggle to explain or modify AI-generated architectural decisions.
3. How can organisations evaluate where AI delivers real value?
Organisations should conduct a structured assessment, such as Codurance's Data and AI Readiness Assessment (DARA). This framework provides independent insights to help leaders identify high-value use cases - like test generation or prototyping - while avoiding low-value applications that carry high risk.
Matt Belcher is the Head of Emerging Technology at Codurance, where he leads the strategy for Data, AI, and innovation. With over 15 years of consultancy experience ranging from software developer to fractional CTO, Matt helps business leaders translate fast-moving technology trends into real-world solutions. He has worked extensively across the financial services, retail, and public sectors throughout the UK and Europe.
When a newly appointed CTO or Engineering Leader asks “What should I focus on in my first 90 days?” The answer is clear. Build trust, diagnose before..
From our recent Breakfast Briefings in London and Manchester to our latest webinar, one question dominated every discussion, “How is AI really..
As part of the tenth annual Leeds Digital Festival, Codurance partnered with techUK to host a dynamic panel discussion on ‘Modernising Retail:..
Join our newsletter for expert tips and inspirational case studies
Join our newsletter for expert tips and inspirational case studies
