Martech Partners Contact
MartechPartners

CJA Conversation Insights: Measuring LLM Chat Experiences

Customer Journey Analytics · Editorial Team · 19 November 2026

Adobe Customer Journey Analytics can now take unstructured chat data into Analysis Workspace through a feature called Conversation Insights. If you run a chatbot, an AI shopping assistant or Adobe Brand Concierge, you can start measuring what people ask, how the agent answers and how that connects to conversion. Rollout started on 29 October 2026, and general availability is still marked TBD.

The rest of the October and November 2026 notes are smaller. This post puts the chat measurement first because it asks the most of your team.

Key takeaways

  • Conversation Insights brings prompts, responses and agent metadata into Analysis Workspace, next to the journey data you already report on.
  • Rollout started on 29 October 2026, after an earlier planned date of 22 September 2026. General availability is listed as TBD, so don't promise a launch date to stakeholders.
  • Collection goes through Web SDK, which means engineering time for custom agents and schema decisions before any dashboard exists.
  • The Adobe Brand Visibility integration (general availability 8 October 2026) covers how people arrive from AI-driven discovery. Conversation Insights covers what happens once they talk to your agent.
  • Pilot one agent and one outcome, and involve privacy review before transcripts reach a shared data view.
  • The smaller visualization, permission and description features are useful, but none of them should delay the chat work.

What does Conversation Insights add to Analysis Workspace?

It adds a way to report on conversations as data. The release notes say Customer Journey Analytics now brings unstructured chat data into Analysis Workspace, so you can report on LLM-powered browsing and buying experiences across your properties.

The notes name four things you can do with it:

  1. Collect prompts, responses and agent metadata from conversational agents, either your organization's custom agents or Adobe Brand Concierge, through Web SDK.
  2. Analyze intent, tone and sentiment, so you can see what customers are asking, how the agent responds and how customers feel about the exchange.
  3. Analyze at scale using your existing schema, datasets and data views, then surface insights in Analysis Workspace.
  4. Connect conversations to outcomes by tying agent interactions to wider customer journeys, so you can measure effects on conversion, engagement and more.

The notes also say that LLM-powered experiences were previously hard to measure and nearly impossible to connect to existing journeys. That matches what most teams have lived with. The chat vendor's dashboard shows containment and satisfaction scores. The analytics platform shows page views and orders. Nobody can say whether the people who asked about delivery times bought more or less than the people who didn't.

This feature is aimed at that gap. It doesn't replace your chatbot vendor's tooling for tuning prompts or reviewing individual transcripts. It gives analysts a shared place to ask journey questions about chat.

Who should act now, and who can wait?

The teams that benefit first are the ones that already have an agent in production and a real question about it. A retailer with a product-advice assistant on its category pages is a good example. A bank with a support chat that sits beside its application flow is another.

If your agent is still a proof of concept, wait. You can't judge intent or sentiment patterns on a few hundred test chats, and you'll end up redoing the schema once the agent changes.

The notes mark general availability as TBD and describe a rollout, so availability in your environment may vary. Confirm with your Adobe contact or your admin console before planning around it.

Situation Who owns it Suggested move
Custom agent live, Web SDK already deployed Analytics engineering Design the schema fields for prompts, responses and metadata, then pilot
Adobe Brand Concierge in use Digital experience lead Check how its data reaches Web SDK and plan the data view
Agent in proof-of-concept stage Product owner Watch the rollout, delay the build
No agent, only a classic FAQ widget Nobody yet Ignore this for now
Heavy analyst team, no engineering support Analytics lead Line up a partner or internal developer before starting

How should you prepare your data before rollout reaches you?

Start with the part that isn't glamorous: the schema. The notes say the capability works with your existing schema, datasets and data views. That's convenient, but it means every decision you make about field names and identity now has to hold up for chat.

A sensible preparation list:

  1. Decide what a conversation is. One session, one chat thread, or one visit that contains several threads? Pick it before collecting anything, because it decides how you count.
  2. Agree the identity story. Chat only helps with outcomes if the person in the chat is the same person in the journey. Check how your identity namespaces already work on the pages where the agent sits.
  3. List the agent metadata you control. If it's a custom agent, decide which attributes you'll send, such as which agent or version answered. Keep names plain and stable.
  4. Settle consent and retention. Prompts are free text. People type names, order numbers and sometimes much more into chat boxes. Get your privacy team to say what may be collected and for how long before you switch collection on. This is a general data-handling point, not a statement about how the feature behaves, so read the product documentation for what controls exist.
  5. Plan the data view. Decide which components analysts will see, and keep the first version small.

A team that skips step 4 usually finds out about it from a legal email, not a dashboard.

How does Adobe Brand Visibility fit with it?

They sit on opposite sides of the same journey. The Adobe Brand Visibility integration reached general availability on 8 October 2026. Per the notes, it lets you connect Adobe Brand Visibility with your Customer Journey Analytics data so you can measure how AI-driven discovery translates into real website engagement and business outcomes.

In plain terms, one answers a question about arrival and the other about conversation. Brand Visibility is about how AI-driven discovery shows up in your engagement and outcomes. Conversation Insights is about what visitors say once they're talking to your agent.

Taken together, a team could in principle look at a journey that starts with AI-driven discovery and passes through an on-site conversation before an order. Whether you can build that view depends on your data, your identity setup and what each integration exposes in your environment. Don't assume the two join out of the box. Check the documentation Adobe links from the release notes, and test with a single journey before promising a combined report.

If you only have budget for one, pick the one that matches your actual problem. If leadership asks how AI assistants send people to the site, start with Brand Visibility. If leadership asks whether the on-site assistant helps or hurts conversion, start with Conversation Insights.

How do you test it and measure whether it's worth keeping?

Run a pilot narrow enough that you can finish it. One agent, one surface, one outcome, four weeks of data after collection is stable.

Week Task Owner Done when
1 Confirm availability in your environment and agree schema fields Analytics engineering Fields documented and approved by privacy
2 Send test conversations through Web SDK in a non-production setup Developer Prompts, responses and metadata appear in the dataset
3 Build the data view and a first Workspace project Analyst A freeform table shows intent, tone and sentiment by agent
4 Tie conversations to one outcome such as an order or a lead Analyst and product owner Journey view compares chatters and non-chatters

Measure the pilot with questions, not vanity counts. Which intents come up most often? Where does sentiment turn negative in the journey? Do conversations that end with a handoff to a person look different from those that don't? Does a segment of people who chatted behave differently on the outcome you picked?

Be careful with causation. People who chat are often more motivated or more confused than people who don't, so a conversion difference doesn't prove the agent caused it. If you can hold out a share of traffic from the agent, do that. If you can't, say so in the readout.

A useful pilot ends with a decision. Either the agent has a specific weakness you'll fix, or the data isn't clean enough yet and you know why.

Which smaller releases deserve a look?

None of the following are as big as chat measurement, but several will save time. Dates below are general availability dates from the notes unless stated otherwise.

Feature Date What it means for you
Read-only permission for the Customer Journey Analytics MCP server 6 October 2026 Admins can give users read-only access to the MCP server. The existing MCP Access item is renamed MCP Full Access. Review who holds which.
Journey canvas: Compare percentages 19 October 2026 Shows change in percentage points per node, arrow and fallout when comparing to a prior period, so comparisons hold up when volumes differ.
Concise legend labels in visualizations 29 October 2026 Off by default. Trims shared names from legend labels on area, bar, combo, donut, histogram, horizontal bar, line and Venn charts.
Automatically generate descriptions for components and projects 18 November 2026 (rollout began 29 October 2026) Generate descriptions for dimensions, metrics, calculated metrics, segments, date ranges and Workspace projects.
CX Enterprise Coworker: Ask questions about your content 3 November 2026 (rollout began 14 October 2026) Coworker Chat answers questions about your content from Content Analytics.

The notes also describe an in-product experience for Coworker in a chat panel in Analysis Workspace, which uses the open project as context. The text available here is cut off mid-description, so read the full entry on Experience League for its dates and details.

Two practical notes. If you grant MCP access, audit it now: anyone who previously had MCP Access now sits under the renamed MCP Full Access, and some of them may only need the read-only item. And if your component library is large and badly documented, generated descriptions are a cheap first pass. Read each one before you publish it, because a confident but wrong description is worse than a blank.

What shouldn't you do?

A few habits to avoid.

  • Don't switch on collection before privacy sign-off. Free-text chat is the riskiest data you'll put in an analytics platform.
  • Don't invent a date for stakeholders. General availability is TBD, and the rollout date has already moved once, from 22 September 2026 to 29 October 2026.
  • Don't build a giant data view. Start with a handful of components and add more when analysts ask for them.
  • Don't treat sentiment as a verdict. Use it to find conversations worth reading, then read some of them.
  • Don't skip the baseline. Without a pre-launch view of conversion on the pages where the agent sits, you can't say what changed.
  • Don't chase every small feature. Concise labels and Compare percentages are nice. They're not a project.

Where does an implementation partner help, and where can you do it yourself?

You can do a lot internally. If Web SDK is already running, your analysts know Workspace and your privacy team answers email, a four-week pilot is realistic without outside help. Renaming permission items, switching on concise labels and reviewing generated descriptions are admin tasks, not projects.

A partner earns their fee in three situations. The first is a custom-built agent where nobody on your side owns the front-end code that would send prompts and responses. The second is identity: if chat sessions can't be tied to known visitors, the outcome analysis falls apart, and untangling that across tools is specialist work. The third is governance, where transcripts need retention rules, access tiers and a documented review.

If you go that way, brief the partner on the question you want answered, not the feature you want switched on. Our Customer Journey Analytics partners page lists firms that work in this area, and the Customer Journey Analytics RFP template gives you a structure for scoping the pilot, the data model and the handover. If you want to compare shortlisted firms side by side, the compare tool is the quickest way to do it.

The short version: pick one agent, one outcome and a clear owner, then wait for the TBD to resolve before you build anything you can't change.

Sources

Frequently Asked Questions

What is Conversation Insights in Adobe Customer Journey Analytics?

It is a capability that brings unstructured chat data into Analysis Workspace. You collect prompts, responses and agent metadata from conversational agents, then analyze intent, tone and sentiment alongside your existing journey data. Adobe's release notes describe it as a way to report on LLM-powered browsing and buying experiences.

How does chat data get into Customer Journey Analytics?

According to the release notes, prompts, responses and agent metadata are collected via Web SDK from conversational agents. These can be your organization's own custom agents or Adobe Brand Concierge. The data then uses your existing schema, datasets and data views.

When does Conversation Insights become available?

The release notes list rollout starting on 29 October 2026, moved from an original plan of 22 September 2026. The general availability date is shown as TBD. Check the Experience League page for the current status before you commit a project date.

Do I need a partner to set up Conversation Insights?

Not always. A team that already runs Web SDK and keeps clean data views can pilot it internally. A partner is most useful when the agent is custom built, when identity stitching across chat and web is messy, or when privacy review of transcripts needs structured help.