Introduction
Modernizing legacy systems is a critical challenge for many organizations, especially with the rise of agentic programming and large language models (LLMs). Insights from a recent retreat of software professionals reveal practical strategies to harness AI for porting code, validating specifications, and streamlining change management. This guide transforms those insights into a structured, step-by-step approach to help your team modernize efficiently while avoiding common pitfalls.
What You Need
- Access to a capable LLM (e.g., GPT-4, Claude, or similar) with code generation abilities
- Source code of the legacy system (e.g., GNU Cobol, COBOL, or other legacy languages)
- Existing regression test suite or ability to create one from a working implementation
- Change-control board guidelines and historical records of past failures (the “scar tissue”)
- Domain experts familiar with the legacy system and current user needs
- A target platform (e.g., Rust, .NET, cloud-native) with supporting tools
- Regulatory compliance documentation (if in finance or similar sectors)
- Project management framework (Agile, water-scrum, etc.) to track progress
Step-by-Step Guide
Step 1: Analyze Your Change-Control History
The first action when consulting with any organization is to review the change-control board guidelines. These documents are the “scar tissue” of past failures—they reveal what went wrong, why workarounds exist, and what constraints shaped the current system. This step helps you understand the history behind the code, which is essential before making any changes. Gather all change logs, incident reports, and board meeting minutes. Interview long-tenured team members to fill gaps. Use this knowledge to identify which parts of the system are brittle or overly complex.
Step 2: Assess Unused Features and User Needs
Before porting, understand that legacy systems often bloat with features that no one uses. A 2014 Standish Group report found up to 50% of features may be unused. Work with product owners and users to map current workflows. Prioritize features based on actual usage data and business outcomes. This step prevents wasting effort on unnecessary functionality during migration.
Step 3: Use LLMs to Port Code to a New Platform
The recent ability of LLMs to produce accurate behavioral clones is a game-changer. For example, one team ported a GNU Cobol compiler to Rust in three days, producing 70,000 lines of functional code. Step-by-step:
- Feed the legacy source code into the LLM with instructions to rewrite in the target language (e.g., Rust, Java, Python).
- Run the output against your existing regression test suite to verify behavioral parity.
- If no test suite exists, leverage the original implementation to generate tests automatically—a process greatly aided by LLMs.
- Iterate on portions that fail tests, using the LLM to correct or refactor.
- Document all transformations for future maintenance.
The cost of porting is now far lower than traditionally feared, making “lift and shift” a viable first step—but don’t stop there (see Step 5).
Step 4: Validate Specifications with an Interrogatory LLM
Large specification documents are hard for humans to review thoroughly. An innovative technique from the retreat involves using an LLM to interview a human expert. The LLM asks targeted questions to verify correctness of the spec—a kind of “interrogatory LLM.”
- Upload the spec document to the LLM.
- Instruct it to generate questions that test edge cases, ambiguous statements, and assumptions.
- Have a domain expert answer each question, recording clarifications.
- Update the specification based on the expert’s responses.
- This process catches errors early, before they become costly bugs.
Step 5: Execute a Lift-and-Shift as a First Step – Then Evolve
Contrary to older beliefs that lift-and-shift is a missed opportunity, the new consensus (post-LLM) is to always start with a straight port to a modern platform. The reasoning: the cost is no longer prohibitive, and a better environment (e.g., cloud-native, Rust’s safety) makes further evolution much cheaper and safer. However, do not stop there. After the shift, use the new platform to refactor, remove unused features, and optimize for current business needs. Treat the port as a solid foundation, not a final product.
Step 6: Address Regulatory and Risk Constraints
If your domain involves money, medical data, or critical infrastructure, regulatory controls are paramount. For example, financial institutions face severe consequences for errors. During every step:
- Involve compliance and risk teams from the start.
- Maintain an immutable audit trail of all code transformations.
- Use the LLM-generated code as a draft, then have it reviewed and signed off by humans.
- Consider running both the old and new systems in parallel during a transition period.
- Document all testing and validation procedures for regulators.
Tips and Best Practices
- Don’t skip the history lesson. Understanding why a system is the way it is (change-control documents, incident logs) prevents repeating past mistakes.
- Let the LLM help with test generation. If good regression tests are missing, use the original system as an oracle and have the LLM generate test cases.
- Measure feature usage before porting. Prioritize only what users actually need—leave behind the dead weight.
- Interview your specs. The interrogatory LLM technique is cheap and incredibly effective for catching ambiguities.
- Lift and shift is a means, not an end. After porting, invest in continuous improvement—refactoring, removing bloat, and adopting modern practices like CI/CD.
- Stay agile with compliance. In regulated industries, involve legal and risk teams in every sprint review to avoid surprises.
- Use internal anchor links to cross-reference steps in your documentation. For example, from Step 5 link back to Step 3 to reinforce the porting process.
By following these steps, you can turn the chaos of legacy migration into a structured, AI-assisted process that saves time, reduces risk, and delivers a modern system aligned with today’s business needs.