Legacy Modernization meets GenAI

GenAI period: A timeline of key occasions

One main cause for the
present wave of hype and pleasure round GenAI is the
versatility and excessive efficiency of general-purpose LLMs. Every new era of those fashions has persistently
proven enhancements in pure language comprehension, inference, and response
high quality. We’re seeing a variety of organizations leveraging these highly effective
fashions to fulfill their particular wants. Moreover, the introduction of
multimodal AIs, equivalent to text-to-image generative fashions like DALL-E, alongside
with AI fashions able to video and audio comprehension and era,
has additional expanded the applicability of GenAIs. Furthermore, the
newest AI fashions can retrieve new data from real-time sources,
past what’s included of their coaching datasets, additional broadening
their scope and utility.

Since then, we’ve got noticed the emergence of latest software program merchandise designed
with GenAI at their core. In different circumstances, present merchandise have change into
GenAI-enabled by incorporating new options beforehand unavailable. These
merchandise sometimes make the most of normal goal LLMs, however these quickly hit limitations when their use case goes past
prompting the LLM to generate responses purely based mostly on the information it has been skilled with (text-to-text
transformations). For example, in case your use case requires an LLM to grasp and
entry your group’s knowledge, essentially the most economically viable answer typically
includes implementing a Retrieval-Augmented Technology (RAG) strategy.
Alternatively, or together with RAG, fine-tuning a general-purpose mannequin could be acceptable,
particularly in the event you want the mannequin to deal with complicated guidelines in a specialised
area, or if regulatory necessities necessitate exact management over the
mannequin’s outputs.

The widespread emergence of GenAI-powered merchandise might be partly
attributed to the supply of quite a few instruments and growth
frameworks. These instruments have democratized GenAI, offering abstractions
over the complexities of LLM-powered workflows and enabling groups to run
fast experiments in sandbox environments with out requiring AI technical
experience. Nonetheless, warning have to be exercised in these comparatively early
days to not fall into traps of comfort with frameworks to which
Thoughtworks’ latest know-how radar
attests.

Issues that make modernization costly

After we started exploring the usage of “GenAI for Modernization”, we
targeted on issues that we knew we might face many times – issues
we knew have been those inflicting modernization to be time or price
prohibitive.

• How can we perceive the present implementation particulars of a system?
• How can we perceive its design?
• How can we collect information about it with out having a human knowledgeable obtainable
  to information us?
• Can we assist with idiomatic translation of code at scale to our desired tech
  stack? How?
• How can we reduce dangers from modernization by enhancing and including
  automated checks as a security web?
• Can we extract from the codebase the domains, subdomains, and
  capabilities?
• How can we offer higher security nets in order that variations in conduct
  between previous methods and new methods are clear and intentional? How can we allow
  cut-overs to be as headache free as potential?

Not all of those questions could also be related in each modernization
effort. We’ve intentionally channeled our issues from essentially the most
difficult modernization situations: Mainframes. These are among the
most important legacy methods we encounter, each by way of dimension and
complexity. If we will clear up these questions on this situation, then there
will definitely be fruit born for different know-how stacks.

The Structure of CodeConcise

CodeConcise is impressed by the Code-as-data
idea, the place code is
handled and analyzed in methods historically reserved for knowledge. This implies
we aren’t treating code simply as textual content, however via using language
particular parsers, we will extract its intrinsic construction, and map the
relationships between entities within the code. That is executed by parsing the
code right into a forest of Summary Syntax Timber (ASTs), that are then
saved in a graph database.

Determine 2: An ingestion pipeline in CodeConcise.

Edges between nodes are then established, for instance an edge could be saying
“the code on this node transfers management to the code in that node”. This course of
doesn’t solely permit us to grasp how one file within the codebase would possibly relate
to a different, however we additionally extract at a a lot granular stage, for instance, which
conditional department of the code in a single file transfers management to code within the
different file. The flexibility to traverse the codebase at such a stage of granularity
is especially essential because it reduces noise (i.e. pointless code) from the
context supplied to LLMs, particularly related for information that don’t include
extremely cohesive code. Primarily, there are two advantages we observe from this
noise discount. First, the LLM is extra more likely to keep focussed on the immediate.
Second, we use the restricted house within the context window in an environment friendly approach so we
can match extra data into one single immediate. Successfully, this permits the
LLM to research code in a approach that’s not restricted by how the code is organized in
the primary place by builders. We check with this deterministic course of because the ingestion pipeline.

Determine 3: A simplified illustration of how a information graph would possibly appear to be for a Java codebase.

Subsequently, a comprehension pipeline traverses the graph utilizing a number of
algorithms, equivalent to Depth-first Search with
backtracking in post-order
traversal, to counterpoint the graph with LLM-generated explanations at numerous depths
(e.g. strategies, courses, packages). Whereas some approaches at this stage are
frequent throughout legacy tech stacks, we’ve got additionally engineered prompts in our
comprehension pipeline tailor-made to particular languages or frameworks. As we started
utilizing CodeConcise with actual, manufacturing consumer code, we recognised the necessity to
maintain the comprehension pipeline extensible. This ensures we will extract the
information most precious to our customers, contemplating their particular area context.
For instance, at one consumer, we found {that a} question to a particular database
desk carried out in code can be higher understood by Enterprise Analysts if
described utilizing our consumer’s enterprise terminology. That is significantly related
when there’s not a Ubiquitous
Language shared between
technical and enterprise groups. Whereas the (enriched) information graph is the primary
product of the comprehension pipeline, it’s not the one beneficial one. Some
enrichments produced in the course of the pipeline, equivalent to mechanically generated
documentation in regards to the system, are beneficial on their very own. When supplied
on to customers, these enrichments can complement or fill gaps in present
methods documentation, if one exists.

Determine 4: A comprehension pipeline in CodeConcise.

Neo4j, our graph database of alternative, holds the (enriched) Information Graph.
This DBMS options vector search capabilities, enabling us to combine the
Information Graph into the frontend software implementing RAG. This strategy
supplies the LLM with a a lot richer context by leveraging the graph’s construction,
permitting it to traverse neighboring nodes and entry LLM-generated explanations
at numerous ranges of abstraction. In different phrases, the retrieval element of RAG
pulls nodes related to the person’s immediate, whereas the LLM additional traverses the
graph to collect extra data from their neighboring nodes. For example,
when in search of data related to a question about “how does authorization
work when viewing card particulars?” the index could solely present again outcomes that
explicitly cope with validating person roles, and the direct code that does so.
Nonetheless, with each behavioral and structural edges within the graph, we will additionally
embrace related data in known as strategies, the encompassing bundle of code,
and within the knowledge buildings which have been handed into the code when offering
context to the LLM, thus upsetting a greater reply. The next is an instance
of an enriched information graph for AWS Card
Demo,
the place blue and inexperienced nodes are the outputs of the enrichments executed within the
comprehension pipeline.

Determine 5: An (enriched) information graph for AWS Card Demo.

The relevance of the context supplied by additional traversing the graph
finally relies on the factors used to assemble and enrich the graph within the
first place. There is no such thing as a one-size-fits-all answer for this; it can rely on
the particular context, the insights one goals to extract from their code, and,
finally, on the ideas and approaches that the event groups adopted
when developing the answer’s codebase. For example, heavy use of
inheritance buildings would possibly require extra emphasis on INHERITS_FROM edges vs
COMPOSED_OF edges in a codebase that favors composition.

For additional particulars on the CodeConcise answer mannequin, and insights into the
progressive studying we had via the three iterations of the accelerator, we
will quickly be publishing one other article: Code comprehension experiments with
LLMs.

Within the subsequent sections, we delve deeper into particular modernization
challenges that, if solved utilizing GenAI, might considerably affect the price,
worth, and time for modernization – components that usually discourage us from making
the choice to modernize now. In some circumstances, we’ve got begun exploring internally
how GenAI would possibly handle challenges we’ve got not but had the chance to
experiment with alongside our purchasers. The place that is the case, our writing is
extra speculative, and we’ve got highlighted these situations accordingly.

Reverse engineering: drawing out low-level necessities

When endeavor a legacy modernization journey and following a path
like Rewrite or Substitute, we’ve got discovered that, in an effort to draw a
complete checklist of necessities for our goal system, we have to
look at the supply code of the legacy system and carry out reverse
engineering. These will information your ahead engineering groups. Not all
these necessities will essentially be integrated into the goal
system, particularly for methods developed over a few years, a few of which
could now not be related in at this time’s enterprise and market context.
Nonetheless, it’s essential to grasp present conduct to make knowledgeable
selections about what to retain, discard, and introduce in your new
system.

The method of reverse engineering a legacy codebase might be time
consuming and requires experience from each technical and enterprise
individuals. Allow us to take into account beneath among the actions we carry out to achieve
a complete low-level understanding of the necessities, together with
how GenAI can assist improve the method.

Thoughtworks labored with the consumer talked about earlier to discover methods to
speed up the reverse engineering of a big legacy codebase written in COBOL/
IDMS. To attain this, we prolonged CodeConcise to assist the consumer’s tech
stack and developed a proof of idea (PoC) using the accelerator within the
method described above. Earlier than the PoC, reverse engineering 10,000 strains of code
sometimes took 6 weeks (2 FTEs working for 4 weeks, plus wait time and an SME
overview). On the finish of the PoC, we estimated that our answer might scale back this
by two-thirds, from 6 weeks to 2 weeks for a module. This interprets to a
potential saving of 240 FTE years for all the mainframe modernization
program.

Excessive-level, summary rationalization of a system

We’ve skilled that LLMs can assist us perceive low-level
necessities extra rapidly. The following query is whether or not they also can
assist us with high-level necessities. At this stage, there’s a lot
data to soak up and it’s powerful to digest all of it. To sort out this,
we create psychological fashions which function abstractions that present a
conceptual, manageable, and understandable view of the purposes we
are trying into. Normally, these fashions exist solely in individuals’s heads.
Our strategy includes working carefully with consultants, each technical and
enterprise focussed, early on within the venture. We maintain workshops, equivalent to
Occasion
Storming
from Area-driven Design, to extract SMEs’ psychological fashions and retailer them
on digital boards for visibility, steady evolution, and
collaboration. These fashions include a website language understood by each
enterprise and technical individuals, fostering a shared understanding of a
complicated area amongst all crew members. At the next stage of abstraction,
these fashions can also describe integrations with exterior methods, which
might be both inner or exterior to the group.

It’s changing into evident that entry to, and availability of SMEs is
important for understanding complicated legacy methods at an summary stage
in a cheap method. Lots of the constraints beforehand
highlighted are subsequently relevant to this modernization
problem.

Within the period of GenAI, particularly within the modernization house, we’re
seeing good outputs from LLMs when they’re prompted to clarify a small
subset of legacy code. Now, we need to discover whether or not LLMs might be as
helpful in explaining a system at the next stage of abstraction.

Our accelerator, CodeConcise, builds upon Code as Information methods by
using the graph illustration of a legacy system codebase to
generate LLM-generated explanations of code and ideas at completely different
ranges of abstraction:

• Graph traversal technique: We leverage all the codebase’s
  illustration as a graph and use traversal algorithms to counterpoint the graph with
  LLM-generated explanations at numerous depths.
• Contextual information: Past processing the code and storing it within the
  graph, we’re exploring methods to course of any obtainable system documentation, as
  it typically supplies beneficial insights into enterprise terminology, processes, and
  guidelines, assuming it’s of excellent high quality. By connecting this contextual
  documentation to code nodes on the graph, our speculation is we will improve
  additional the context obtainable to LLMs throughout each upfront code rationalization and
  when retrieving data in response to person queries.

Finally, the objective is to reinforce CodeConcise’s understanding of the
code with extra summary ideas, enabling its chatbot interface to
reply questions that sometimes require an SME, holding in thoughts that
such questions won’t be immediately answerable by analyzing the code
alone.

At Thoughtworks, we’re observing constructive outcomes in each
traversing the graph and producing LLM explanations at numerous ranges
of code abstraction. We’ve analyzed an open-source COBOL repository,
AWS Card
Demo,
and efficiently requested high-level questions equivalent to detailing the system
options and person interactions. On this event, the codebase included
documentation, which supplied extra contextual information for the
LLM. This enabled the LLM to generate higher-quality solutions to our
questions. Moreover, our GenAI-powered crew assistant, Haiven, has
demonstrated at a number of purchasers how contextual details about a
system can allow an LLM to supply solutions tailor-made to
the particular consumer context.

Discovering a functionality map of a system

One of many first issues we do when starting a modernization journey
is catalog present know-how, processes, and the individuals who assist
them. Inside this course of, we additionally outline the scope of what’s going to be
modernized. By assembling and agreeing on these components, we will construct a
sturdy enterprise case for the change, develop the know-how and enterprise
roadmaps, and take into account the organizational implications.
With out having this at hand, there isn’t a technique to decide what wants
to be included, what the plan to attain is, the incremental steps to
take, and once we are executed.

Earlier than GenAI, our groups have been utilizing a variety of
methods to construct this understanding, when it’s not already current.
These methods vary from Occasion Storming and Course of Mapping via
to “following the information” via the system, and even focused code
opinions for significantly complicated subdomains. By combining these
approaches, we will assemble a functionality map of our purchasers’
landscapes.
Whereas this may increasingly appear as if a considerable amount of guide effort, these can
be among the most precious actions because it not solely builds a plan for
the longer term supply, however the considering and collaboration that goes into
making it ensures alignment of the concerned stakeholders, particularly
round what’s going to be included or excluded from the modernization
scope. Additionally, we’ve got learnt that functionality maps are invaluable once we
take a capability-driven strategy to modernization. This helps modernize
the legacy system incrementally by progressively delivering capabilities in
the goal system, along with designing an structure the place
issues are cleanly separated.

GenAI adjustments this image so much.

One of the crucial highly effective capabilities that GenAI brings is
the flexibility to summarize giant volumes of textual content and different media. We are able to
use this functionality throughout present documentation which may be current
concerning know-how or processes to extract out, if not the top
information, then not less than a place to begin for additional conversations.
There are a selection of methods which might be being actively developed and
launched on this space. Specifically, we imagine that
GraphRAG which was just lately
launched by Microsoft may very well be used to extract a stage of data from
these paperwork via Graph Algorithm evaluation of the physique of
textual content.
We’ve additionally been trialing GenAI excessive of the information graph
that we construct out of the legacy code as talked about earlier by asking what
key capabilities modules have after which clustering and abstracting these
via hierarchical summarization. This then serves as a map of
capabilities, expressed succinctly at each a really excessive stage and a
detailed stage, the place every functionality is linked to the supply code
modules the place it’s carried out. That is then used to scope and plan for
the modernization in a sooner method. The next is an instance of a
functionality map for a system, together with the supply code modules (small
grey nodes) they’re carried out in.

However, we’ve got learnt to not view this solely LLM-generated
functionality map as mutually unique from the standard strategies of
creating functionality maps described earlier. These conventional approaches
are beneficial not just for aligning stakeholders on the scope of
modernization, but additionally as a result of, when a functionality already exists, it
can be utilized to cluster the supply code based mostly on the capabilities
carried out. This strategy produces functionality maps that resonate higher
with SMEs through the use of the group’s Ubiquitous language. Moreover,
evaluating each functionality maps could be a beneficial train, absolutely one
we stay up for experimenting with, as every would possibly supply insights the
different doesn’t.

Discovering unused / useless / duplicate code

One other a part of gathering data in your modernization efforts
is knowing inside your scope of labor, “what continues to be getting used at
all”, or “the place have we received a number of situations of the identical
functionality”.

At present this may be addressed fairly successfully by combining two
approaches: static and dynamic evaluation. Static evaluation can discover unused
methodology calls and statements inside sure scopes of interrogation, for
occasion, discovering unused strategies in a Java class, or discovering unreachable
paragraphs in COBOL. Nonetheless, it’s unable to find out whether or not entire
API endpoints or batch jobs are used or not.

That is the place we use dynamic evaluation which leverages system
observability and different runtime data to find out if these
features are nonetheless in use, or might be dropped from our modernization
backlog.

When trying to discover duplicate technical capabilities, static
evaluation is essentially the most generally used device as it could do chunk-by-chunk textual content
similarity checks. Nonetheless, there are main shortcomings when utilized to
even a modest know-how property: we will solely discover code similarities in
the similar language.
We speculate that by leveraging the results of {our capability}
extraction strategy, we will use these know-how agnostic descriptions
of what giant and small abstractions of the code are doing to carry out an
estate-wide evaluation of duplication, which can take our future
structure and roadmap planning to the following stage.

In terms of unused code nonetheless, we see little or no use in
making use of GenAI to the issue. Static evaluation instruments within the trade for
discovering useless code are very mature, leverage the structured nature of
code and are already at builders’ fingertips, like IntelliJ or Sonar.
Dynamic evaluation from APM instruments is so highly effective there’s little that instruments
like GenAI can add to the extraction of knowledge itself.

However, these two complicated approaches can yield an unlimited
quantity of information to grasp, interrogate and derive perception from. This
may very well be one space the place GenAI might present a minor acceleration
for discovery of little used code and know-how.
Much like having GenAI check with giant reams of product documentation
or specs, we will leverage its information of the static and
dynamic instruments to assist us use them in the precise approach as an example by
suggesting potential queries that may be run over observability stacks.
NewRelic, as an example, claims to have built-in LLMs in to its options to
speed up onboarding and error decision; this may very well be turned to a
modernization benefit too.

Idiomatic translation of tech paradigm

Translation from one programming language to a different isn’t one thing new. A lot of the instruments that do that have
utilized static evaluation methods – utilizing Summary Syntax Timber (ASTs) as intermediaries.

Though these methods and instruments have existed for a very long time, outcomes are sometimes poor when judged via
the lens of “would somebody have written it like this if they’d began authoring it at this time”.

Sometimes the produced code suffers from:

Assuming a company embarks on this journey, it can quickly face prolonged testing and verification
cycles to make sure the generated code behaves precisely the identical approach as earlier than. This turns into much more difficult
when little to no security web was in place initially.

Regardless of all of the drawbacks, code conversion approaches proceed to be an choice that pulls some organizations
due to their attract as probably the bottom price/ effort answer for leapfrogging from one tech paradigm
to the opposite.

We’ve additionally been fascinated with this and exploring how GenAI can assist enhance the code produced/ generated. It
can not help all of these points, however perhaps it could assist alleviate not less than the primary three or 4 of them.

From an strategy perspective, we try to use the ideas of
Refactoring
to this – basically
work out a approach we will safely and incrementally make the leap from one tech paradigm to a different. This strategy
has already seen some success – two examples that come to thoughts:

Conclusion

At the moment’s panorama has quite a few alternatives to leverage GenAI to
obtain outcomes that have been beforehand out of attain. Within the software program
trade, GenAI is already taking part in a major function in serving to individuals
throughout numerous roles full their duties extra effectively, and this
affect is predicted to develop. For example, GenAI has produced promising
ends in aiding technical engineers with writing code.

Over the previous a long time, our trade has developed considerably, growing patterns, finest practices, and
methodologies that information us in constructing fashionable software program. Nonetheless, one of many greatest challenges we now face is
updating the huge quantity of code that helps key operations every day. These methods are sometimes giant and sophisticated,
with a number of layers and patches constructed over time, making conduct tough to vary. Moreover, there are
typically only some consultants who totally perceive the intricate particulars of how these methods are carried out and
function. For these causes, we use an evolutionary strategy to legacy displacement, decreasing the dangers concerned
in modernizing these methods and producing worth early. Regardless of this, the price/time/worth equation for
modernizing giant methods is usually prohibitive. On this article, we mentioned methods GenAI might be harnessed to
flip this example round. We’ll proceed experimenting with making use of GenAI to those modernization challenges
and share our insights via this text, which we’ll maintain updated. This may embrace sharing what has
labored, what we imagine GenAI might probably clear up, and what, nonetheless, has not succeeded. Moreover, we
will lengthen our accelerator, CodeConcise, with the intention of additional innovating inside the modernization course of to
drive larger worth for our purchasers.

Hopefully, this text highlights the nice potential of harnessing
this new know-how, GenAI, to handle among the challenges posed by
legacy methods within the trade. Whereas there isn’t a one-size-fits-all
answer to those challenges – every context has its personal distinctive nuances –
there are sometimes similarities that may information our efforts. We additionally hope this
article conjures up others within the trade to additional develop experiments
with “GenAI for Modernization” and share their insights with the broader
group.