generate buyer journeys that seem easy and fascinating, however evaluating whether or not these journeys are structurally sound stays difficult for present strategies.
This text introduces Continuity, Deepening, and Development (CDP) — three deterministic, content-structure-based metrics for evaluating multi-step journeys utilizing a predefined taxonomy fairly than stylistic judgment.
Historically, optimizing customer-engagement programs has concerned fine-tuning supply mechanics similar to timing, channel, and frequency to realize engagement and enterprise outcomes.
In observe, this meant you skilled the mannequin to grasp guidelines and preferences, similar to “Don’t contact clients too usually”, “Consumer Alfa responds higher to cellphone calls”, and “Consumer Beta opens emails principally within the night.”
To handle this, you constructed a cool-off matrix to steadiness timing, channel constraints, and enterprise guidelines to manipulate buyer communication.
To date, so good. The mechanics of supply are optimized.
At this level, the core problem arises when the LLM generates the journey itself. The difficulty is not only about channel or timing, however whether or not the sequence of messages varieties a coherent, efficient narrative that meets enterprise aims.
And all of a sudden you notice:
There isn’t a normal metric to find out if an AI-generated journey is coherent, significant, or advances enterprise objectives.
What We Count on From a Profitable Buyer Journey
From a enterprise perspective, the sequence of contents per journey step can’t be random: it have to be a guided expertise that feels coherent, strikes the client ahead by means of significant phases, and deepens the connection over time.
Whereas this instinct is widespread, it is usually supported by customer-engagement analysis. Brodie et al. (2011) describe engagement as “a dynamic, iterative course of” that varies in depth and complexity as worth is co-created over time.
In observe, we consider journey high quality alongside three complementary dimensions:
Continuity — whether or not every message matches the context established by prior interactions.
Deepening — whether or not content material turns into extra particular, related, or personalised fairly than remaining generic.
Development — whether or not the journey advances by means of phases (e.g., from exploration to motion) with out pointless backtracking.
Why Present LLM Analysis Metrics Fall Quick
If we have a look at normal analysis strategies for LLMs, similar to accuracy metrics, similarity metrics, human-evaluation standards, and even LLM-as-a-judge, it turns into clear that none present a dependable, unambiguous solution to consider buyer journeys generated as multi-step sequences.
Let’s study what normal buyer journey metrics can and might’t present.
Accuracy Metrics (Perplexity, Cross-Entropy Loss)
These metrics measure confidence degree in predicting the following token given the coaching information. They don’t seize whether or not a generated sequence varieties a coherent or significant journey.
Similarity Metrics (BLEU, ROUGE, METEOR, BERTScore, MoveScore)
These metrics evaluate the generated consequence to a reference textual content. Nonetheless, buyer journeys hardly ever have a single appropriate reference, as they adapt to context, personalization, and prior interactions. Structurally legitimate journeys might differ considerably whereas remaining efficient.
Undoubtedly, semantic similarity has its benefits, and we’ll use cosine similarity, however extra on that later.
Human Analysis (Fluency, Relevance, Coherence)
Human judgment usually outperforms automated metrics in assessing language high quality, however it’s poorly fitted to steady journey analysis. It’s costly, suffers from cultural bias and ambiguity, and doesn’t perform as a everlasting a part of the workflow however fairly as a one-time effort to bootstrap a fine-tuning stage.
LLM-as-a-Decide (AI suggestions scoring)
Utilizing LLMs to judge outputs from different LLM programs is a formidable course of.
This method tends to focus extra on fashion, readability, and tone fairly than structural analysis.
LLM-as-a-Decide might be utilized in multi-stage use instances, however outcomes are sometimes much less exact because of the elevated threat of context overload. Moreover, fine-grained analysis scores from this technique are sometimes unreliable. Like human evaluators, LAAJ additionally carries biases and ambiguities.
A Structural Method to Evaluating Buyer Journeys
Finally, the first lacking ingredient in evaluating really helpful content material sequences throughout the buyer journey is construction.
Probably the most pure solution to characterize content material construction is as a taxonomic tree, a hierarchical mannequin consisting of phases, content material themes, and ranges of element.
As soon as buyer journeys are mapped onto this tree, CDP metrics might be outlined as structural variations:
- Continuity: easy motion throughout branches
- Deepening: shifting into extra particular nodes
- Development: shifting ahead by means of buyer journey phases
The answer is to characterize a journey as a path by means of a hierarchical taxonomy derived from the content material house. As soon as this illustration is established, CDP metrics might be computed deterministically from the trail. The diagram under summarizes the complete pipeline.
Setting up the Taxonomy Tree
To judge buyer journeys structurally, we first require a structured illustration of content material. We assemble this illustration as a multi-level taxonomy derived straight from customer-journey textual content utilizing semantic embeddings.
The taxonomy is anchored by a small set of high-level phases (e.g., motivation, buy, supply, possession, and loyalty). Each anchors and journey messages are embedded into the identical semantic vector house, permitting content material to be organized by semantic proximity.
Inside every anchor, messages are grouped into progressively extra particular themes, forming deeper ranges of the taxonomy. Every degree refines the earlier one, capturing rising topical specificity with out counting on handbook labeling.
The result’s a hierarchical construction that teams semantically associated journey messages and offers a secure basis for evaluating how journeys circulate, deepen, and progress over time.
Mapping Buyer Journeys onto the Taxonomy
As soon as the taxonomy is established, particular person buyer journeys are mapped onto it as ordered sequences of messages. Every step is embedded in the identical semantic house and matched to the closest taxonomy node utilizing cosine similarity.
This mapping converts a temporal sequence of messages right into a path by means of the taxonomy, enabling the structural evaluation of journey evolution fairly than treating the journey as a flat checklist of texts.
Defining the CDP Metrics
The CDP framework consists of three complementary metrics: Continuity, Deepening, and Development. Every captures a definite facet of journey high quality. We describe these metrics conceptually earlier than defining them formally based mostly on the taxonomy-mapped journey.
Setup and Computation
Earlier than analyzing actual journeys, we make clear two elements of the setup.
(1) how journey content material is structurally represented, and
(2) how CDP metrics are derived from that construction.
Buyer-journey content material is organized right into a hierarchical taxonomy consisting of anchors (L1 journey phases), thematic heads (L2 subjects), and deeper nodes that characterize rising specificity:
Anchor (L1)
└── Head (L2)
└── Youngster (L3)
└── Grandchild (L4+)As soon as a journey is mapped onto this hierarchy, Continuity, Deepening, and Development are computed deterministically from the journey’s path by means of the tree.
Let a buyer journey be an ordered sequence of steps:
J = (x₁, x₂, …, xₙ)
Every step xᵢ is assigned:
- aᵢ — anchor (L1 journey stage)
- tᵢ — thematic head (L2 subject), the place tᵢ = 0 means “unknown”
- ℓᵢ — taxonomy depth degree (L1 = 0, L2 = 1, L3 = 2, …)
Continuity (C)
Continuity evaluates whether or not consecutive messages stay contextually and thematically coherent.
For every transition (xᵢ →xᵢ₊₁), a step-level continuity rating cᵢ ∈ [0, 1] is assigned based mostly on taxonomy alignment, with greater weights given to transitions that keep throughout the similar subject or intently associated branches.
Transitions are ranked from strongest to weakest (e.g., similar subject, associated subject, ahead stage transfer, backward transfer), and
assigned reducing weights:
1 ≥ α₁ > α₂ > α₃ > α₄ > α₅ > α₆ ≥ 0
The general continuity rating is computed as:
C(J) = (1 / (n − 1)) · Σ cᵢ for i = 1 … n−1
Deepening (D)
Deepening measures whether or not a journey accumulates worth by shifting from basic content material towards extra particular or detailed
interactions. It’s computed utilizing two complementary parts.
Journey-based deepening captures how depth modifications alongside the noticed path:
Δᵢᵈᵉᵖᵗʰ = ℓᵢ₊₁ − ℓᵢ, dᵢ = max(Δᵢᵈᵉᵖᵗʰ, 0)
D_journey(J) = (1 / (n − 1)) · Σ dᵢ
Taxonomy-aware deepening measures how deeply a journey explores the precise taxonomy tree, based mostly on the heads it visits.
It evaluates how lots of the potential deeper content material gadgets (youngsters, sub-children, and so on.) below every visited head are later seen
in the course of the journey.
D_taxonomy(J) = |D_seen(J)| / |D_exist(J)|
The ultimate deepening rating is a weighted mixture:
D(J) = λ₁ · D_taxonomy(J) + λ₂ · D_journey(J), λ₁ + λ₂ = 1.
Deepening lies in [0, 1].
Development (P)
Development measures directional motion by means of journey phases. For every transition, we compute:
Δᵢ = aᵢ₊₁ − aᵢ.
Solely shifting steps (Δᵢ ≠ 0) are thought-about. Let wᵢ denote the relative significance of the present stage.
If Δᵢ > 0 (ahead motion):
cᵢ = wᵢ / Δᵢ
If Δᵢ < 0 (backward motion):
cᵢ = Δᵢ · wᵢ
The uncooked development rating is:
P_raw(J) = Σ cᵢ for all i the place Δᵢ ≠ 0
To sure the rating to[−1, +1], we apply a tanh normalization:
P(J) = (e^(P_raw) − e^(−P_raw)) / (e^(P_raw) + e^(−P_raw))
Making use of CDP Metrics to an Automotive Buyer Journey
To reveal how structured analysis works on reasonable journeys, we generated an artificial automotive customer-journey dataset overlaying the primary phases of the client lifecycle.
Enter Information: Anchors and Journey Content material
The CDP framework makes use of two essential inputs: anchors, which outline journey phases, and customer-journey content material, which offers the messages to judge.
Anchors characterize significant phases within the lifecycle, similar to motivation, buy, supply, possession, and loyalty. Every anchor is augmented with a small set of consultant key phrases to floor it semantically. Anchors serve each as reference factors for taxonomy building and because the anticipated directional circulate used later within the Development metric.
anchor Phrases:
motivation exploration analysis discovery curiosity take a look at drive wants evaluation expertise
buy financing comparability quotes mortgage negotiation credit score pre-approval deposit
supply paperwork signing deposit logistics handover activation
possession upkeep guarantee restore vendor help service inspections
loyalty suggestions satisfaction survey referral improve retention advocacyBuyer-journey content material consists of quick, action-oriented CRM-style messages (emails, calls, chats, in-person interactions) with various ranges of specificity and spanning a number of phases. Though this dataset is synthetically generated, anchor data shouldn’t be used throughout taxonomy building or CDP scoring.
CJ messages:
Discover fashions that match your way of life and private objectives.
Take a digital tour to find key options and trims.
Evaluate physique kinds to evaluate house, consolation, and utility.
E book a take a look at drive to expertise dealing with and visibility.
Use the wants evaluation to rank must-have options.
Filter fashions by vary, mpg, or towing to slender selections.Taxonomy Development Outcomes
Right here, we utilized the taxonomy building course of to the automotive customer-engagement dataset. The determine under reveals the ensuing customer-journey taxonomy, constructed from message content material and anchor semantics.
Every top-level department corresponds to a journey anchor (L1), which represents main journey phases similar to Motivation, Buy, Supply, Possession, and Loyalty.
Deeper ranges (L2, L3+) group messages by thematic similarity and rising specificity.
What the Taxonomy Reveals
Even on this compact dataset, the taxonomy highlights a number of practical patterns:
- Early-stage messages cluster round exploration and comparability, step by step narrowing towards concrete actions similar to reserving a take a look at drive.
- Buy-related content material separates naturally into monetary planning, doc dealing with, and finalization.
- Possession content material reveals a transparent development from upkeep scheduling to diagnostics, value estimation, and guarantee analysis.
- Loyalty content material shifts from transactional actions towards suggestions, upgrades, and advocacy.
Whereas these patterns align with how practitioners usually cause about journeys, they come up straight from the information fairly than from predefined guidelines.
Why This Issues for Analysis
This taxonomy now offers a shared structural reference:
- Any buyer journey might be mapped as a path by means of the tree.
- Motion throughout branches, depth ranges, and anchors turns into measurable.
- Continuity, Deepening, and Development are now not summary ideas; they now correspond to concrete structural modifications.
Within the subsequent part, we use this taxonomy to map actual journey examples and compute CDP scores in steps.
Mapping Buyer Journeys onto the Taxonomy
As soon as the taxonomy is constructed, evaluating a buyer journey turns into a structural drawback.
Every journey is represented as an ordered sequence of customer-facing messages.
As an alternative of judging these messages in isolation, we undertaking them onto the taxonomy and analyze the ensuing path.
Formally, a journey J = (x₁, x₂, …, xₙ) is mapped to a sequence of taxonomy nodes: (x₁→v₁),(x₂→v₂),…,(xₙ→vₙ) the place every vᵢ is the closest taxonomy node based mostly on embedding similarity.
A Step-by-Step Walkthrough: From Journey Textual content to CDP Scores
To make the CDP framework concrete, let’s stroll by means of a single buyer journey instance and present how it’s evaluated step-by-step.
Step 1 — The Buyer Journey Enter
We start with an ordered sequence of customer-facing messages generated by an LLM.
Every message represents a touchpoint in a sensible automotive buyer journey:
journey = ['Take a virtual tour to discover key features and trims.';
'We found a time slot for a test drive that fits your schedule.';
'Upload your income verification and ID to finalize the pre-approval decision.';
'Estimate costs for upcoming maintenance items.';
'Track retention offers as your lease end nears.';
'Add plates and registration info before handover.']Step 2 — Mapping the Journey into the Taxonomy
For structural analysis, every journey step is mapped into the customer-journey taxonomy. Utilizing textual content embeddings, every message is matched to its closest taxonomy node. This produces a journey map (jmap), a structured illustration of how the journey traverses the taxonomy.
Desk 2: Every message is assigned to an anchor (stage), a thematic head, and a depth degree within the taxonomy based mostly on semantic similarity within the shared embedding house. This desk acts as the muse for all future evaluations.
Step 3 — Making use of CDP Metrics to This Journey
As soon as the journey is mapped, we compute Continuity, Deepening, and Development deterministically from step-to-step transitions.
Desk 3: Every row represents a transition between consecutive journey steps, annotated with alerts for continuity, deepening, and development.
Closing CDP scores (this journey):
Taken collectively, the CDP alerts point out a journey that’s largely coherent and forward-moving, with one clear second of
deepening and one seen structural regression. Importantly, these insights are derived solely from construction, not from
stylistic judgments concerning the textual content.
Conclusion: From Scores to Profitable Journeys
Continuity, Deepening, and Development are decided by construction and might be utilized wherever LLMs generate multi-step
content material:
- to check various journeys generated by totally different prompts or fashions,
- to supply automated suggestions for bettering journey era over time.
On this approach, CDP scores provide structural suggestions for LLMs. They complement, fairly than substitute, stylistic or fluency-based analysis by offering alerts that mirror enterprise logic and buyer expertise.
Though this text focuses on automotive commerce, the idea is broadly relevant. Any system that generates ordered, goal-oriented content material requires robust structural foundations.
Giant language fashions are already able to producing fluent, persuasive textual content.
The better problem is guaranteeing that textual content sequences type coherent narratives that align with enterprise logic and person expertise.
CDP offers a solution to make construction express, measurable, and actionable.
Thanks for staying with me by means of this journey. Hopefully, this idea helps you suppose in another way about evaluating AI-generated sequences and evokes you to deal with construction as a major sign in your individual programs. All logic offered on this article is applied within the accompanying Python code on GitHub. When you have any questions or feedback, please go away them within the feedback part or attain out through LinkedIn.
References
Brodie, R. J., et al. (2011). Buyer engagement: Conceptual area, elementary propositions, and implications for analysis.
