# MCP analytics: your best users never load a page

> Product analytics was built for a person with a cursor. An MCP server's users are agents, so the event model, the identity model and the success metric all have to change.

Published 2026-11-18 by Nick Sawinyh.
Tags: ai, agents, metrics.
Canonical: https://sawinyh.com/blog/mcp-analytics-for-product-managers/

---

**TL;DR** Shipping an MCP server opens a channel your product analytics cannot see: no pageviews, no clicks, and a user who is an agent acting for a person. The unit of measurement becomes the tool call, tagged with three identities of very different reliability: the person behind the OAuth token, the client that names itself, and the model that reports itself unverified. The channel also offers something a UI never could: you can ask the caller why, by adding an intent argument to every tool and a way for agents to report the tool they needed and could not find. Call volume is the trap, since more calls often means an agent is flailing, so the metric that matters is completed jobs, measured from your own product's state and reported with how many outcomes you could not see.

The users who adopt your MCP server hardest are the ones your analytics stops seeing.

Every product analytics stack a PM inherits was built around a person with a cursor. Pageviews, clicks, sessions, funnels, the button that got pressed and the screen it was pressed on. Ship an [MCP server](https://modelcontextprotocol.io/) and you open a channel where none of that exists. A user asks Claude or Cursor to pull last quarter's churned accounts from your product, the agent calls three of your tools, the job gets done, and the dashboard records nothing. The user who now does most of their work through an agent looks, to your funnel, like a user who stopped showing up.

In May 2026, a customer filed [a feature request](https://jira.atlassian.com/browse/ECO-1498) against Atlassian's Rovo MCP Server that opens with "Currently there is no way to visualize or analyze usage metrics for the Atlassian Rovo MCP Server," and asks for tool-level breakdowns, a connected-clients view and error tracking. [Glean](https://docs.glean.com/administration/platform/mcp/analytics), which does ship MCP usage analytics, notes that MCP activity is tracked separately and does not roll up into its Search or Assistant engagement numbers. Even the vendors that measure it keep it in its own box.

Measuring it means taking three decisions again that your app analytics made for you years ago: what the event is, who the user is, and what success means.

## Your funnel was built for a person with a cursor

The easiest way to see what changes is to line up the concepts you already use against what the agent channel actually gives you.

| App analytics | MCP equivalent | What breaks |
|---|---|---|
| Pageview, screen view | `tools/list` response | Clients may cache the tool list, so a listing is not a visit |
| Click, form submit | `tools/call` | The caller is a model choosing from your tool descriptions, not a person reading a button |
| Logged-in user | The person behind the OAuth token | Shared service accounts and API keys stand in for many people |
| Browser, device | Client name and version | Self-reported by the client, best-effort |
| Session | Nothing at the protocol level | The current spec removed protocol sessions; you mint your own job ID |
| Funnel step completed | A state change in your product | The agent rarely tells you it finished |
| Survey, feedback widget | An intent argument on each tool | You can ask, but the answer is the model's paraphrase |

The session and pageview rows deserve a closer look, because they are where teams build on sand.

The session row is new. The [2026-07-28 revision of the MCP specification](https://modelcontextprotocol.io/specification/2026-07-28/changelog) removed protocol-level sessions and the `Mcp-Session-Id` header from the Streamable HTTP transport, along with the `initialize` handshake. Clients should now identify themselves on every request, and servers that need state across calls are told to mint their own handles and pass them as ordinary tool arguments. If your analytics grouped calls by that header, it will stop grouping them as clients upgrade. The replacement is a decision only you can make: what counts as one job in your product, and which ID ties its calls together.

The pageview row is a trap that looks like a funnel. A `tools/list` call feels like the top of one: the agent looked at what you offer. But the same revision lets servers attach a cache lifetime to list results, so a well-behaved client may fetch your tool list once and reuse it for a long time. Counting listings as discovery will undercount your best-integrated clients and overcount the sloppy ones. The step that matters is the first real call, and the gap before it can be wide. A server operator [described on X](https://x.com/RiskDataApiSol/status/2085646940507603034) seeing "hundreds of profile views on Glama but zero tool calls": most visitors were trying the server from a directory's built-in inspector without a key configured, so the first call never happened. They found it by digging manually, and fixed it with an anonymous demo path. That is an activation problem, and it only shows up if you measure the first call per person, not the listing.

## The event is the tool call

The unit of measurement in the agent channel is one `tools/call`: one tool, invoked once, with arguments, and a result or an error. Everything else is built from that.

A useful concrete reference is the event schema in [PostHog's MCP analytics SDK](https://posthog.com/docs/mcp-analytics), because it shows what a team that thought about this for a while decided to capture. Its `$mcp_tool_call` event carries, among other things, the tool name, the duration in milliseconds, an error flag and an error category, the client's self-reported name and version, the raw user agent on HTTP transports, the protocol version, the agent's stated intent, and sanitized copies of the arguments and response. You don't have to use PostHog to use the list. Most of it is what you would capture by hand.

Strip that down to what a PM needs on day one and you get a short list:

- Tool name, plus a category label you own, so that renaming a tool does not break your history
- Outcome: success, or an error with a category you can count (validation, permission, timeout, upstream)
- Duration
- Client name and version, as reported
- Protocol version, so you know when your clients have moved to a spec revision
- The authenticated person and their account, from your own auth layer
- A job ID your server mints, so related calls can be grouped
- Intent, if you can get it (more on this below)

Arguments and responses are the field to be careful with. They are where the product insight is, since the arguments tell you what people actually asked for, and they are also where customer data lives. PostHog's own reference says it stores sanitized parameters and never captures a resource body. Start from that posture and widen it deliberately, tool by tool, rather than logging everything and redacting later.

The obvious objection is that you already have this. Your server writes logs, your APM traces every request, and the [OpenTelemetry GenAI conventions](https://github.com/open-telemetry/semantic-conventions-genai) cover MCP spans. That is monitoring, and you need it. It answers whether the server is up, which tools are slow and which clients are failing. It doesn't answer who adopted the channel, which tools sit inside work that got finished, or what people tried to do that you could not serve. A [Flowlines post on MCP analytics](https://flowlines.ai/blog/mcp-analytics) puts the split well: monitoring protects reliability, analytics informs product decisions, and a healthy server does not prove a healthy product. Logs are a fine place to start, as long as somebody turns them into events with a person, an account and a job attached.

## Who is the user when an agent calls?

A tool call arrives with three identities attached, and they deserve very different levels of trust.

**The person** is the one you can rely on, because it comes from your own auth. The [MCP authorization spec](https://modelcontextprotocol.io/specification/2025-06-18/basic/authorization) is built on OAuth, so a remote server sees a token that resolves to a user in your system, and from there to an account, a plan and everything else your app analytics already knows. This is the join that makes MCP data useful to a PM: tie the call to the same person ID your product events use, and the agent channel becomes another surface of the same customer instead of a separate population. The exception is the shared credential. A service account or a team-wide API key looks like one very busy user, and Flowlines is right to insist those not be counted as individual users.

**The client** is the application the agent runs in: Claude, Cursor, ChatGPT, an internal agent someone built last week. The client names itself, and the spec now asks it to do so on every request. Treat the name as useful and soft. Glean's documentation calls host attribution best-effort, dependent on the metadata the client provides. Your client breakdown is a breakdown of what clients say about themselves.

**The model** is the least reliable of the three. Some clients pass it in metadata, and PostHog's SDK can also collect it as a self-reported argument. Its reference labels the property plainly: unverified. A model breakdown is interesting, but it shouldn't drive a roadmap decision.

The practical consequence is a rule about which questions each identity can answer. Questions about customers and accounts, adoption and retention, run on the person. Questions about where to invest integration effort run on the client, with a caveat on the chart. Questions about the model stay exploratory.

## You can ask the agent why

The agent channel gives you one thing a UI never could. In an app, intent is something you infer: the user clicked export, so presumably they wanted the data somewhere else. In the agent channel, the caller is a model that is already reasoning in language about what it is doing. You can ask it.

The mechanism is simple. Add an argument to each tool's input schema that asks the agent to state the goal, and record the answer as a property on the call. [PostHog's SDK does this by default](https://posthog.com/docs/mcp-analytics/intent): it injects a `context` argument with the description "Why are you calling this tool? Briefly describe the user's goal," records the answer as `$mcp_intent`, and strips the argument before your handler runs so your tool code never sees it.

A pile of these answers is closer to user research than anything a click stream gives you. Grouped into themes, they tell you which jobs people bring to your product through an agent, in roughly their own terms, and which of those jobs your tools fit badly.

Read that data with three caveats, all of which the documentation states itself:

- Coverage depends on the client. The argument is marked required in some SDKs, but the requirement isn't enforced, so an agent can skip it without the call failing. PostHog records whether intent came from the agent or from a fallback you supplied, and suggests watching that split. A high share of inferred intent means your clients aren't filling it in.
- The intent is a paraphrase: the model's summary of what it thinks the user wants, written in the model's words. It is a good signal for clustering and a poor one for quoting.
- It costs something. Every extra argument is more schema for the model to read and more tokens to produce. If you write your own fallback that calls an LLM to infer intent, it adds latency to every call that lacks it.

The second half of this idea is the more valuable one for a PM. PostHog's SDK adds a virtual tool, `get_more_tools`, that an agent can call when the tool it needs doesn't exist, and records each call as a missing-capability event with the agent's reasoning attached. The Go SDK also records calls to tool names your server never registered. Both are a feature-request feed generated by the work itself, from the moment a user's job ran into the edge of your product, which is a moment no survey reaches. Agents report gaps with the same fluency they use for everything else, so treat each request as a lead to investigate before you count it as demand.

All of this is early. PostHog describes the SDK as alpha, published as a 0.1.x release with event names and properties that may change before 1.0. Build your own reporting on the idea rather than on the exact property names.

## Call volume is a vanity metric

The first chart every MCP dashboard draws is tool calls over time, and it's the least informative chart you will own.

A rising call count fits two opposite stories. In one, more people are getting more work done through agents. In the other, the same people's agents are retrying a tool that returns an unhelpful error, paging through results one item at a time because the tool has no filter, or looping between two tools because their descriptions overlap. From the outside, a struggling agent and a busy one look the same: lots of calls. Volume cannot tell them apart, and the struggling case often produces more of it.

PostHog published the cleanest example I've seen, from its own MCP server, in [a thread on what its analytics turned up](https://x.com/posthog/status/2085415325118734738). A single agent session deleted 5,174 session replays one at a time. Bulk delete existed in the product. It just hadn't been exposed as an MCP tool. On a call-volume chart, that session is a spike of engagement. It was a missing tool, and not having it cost thousands of calls. It's a vendor reporting on its own product, but the shape is general: the heaviest users of your server are partly the agents working hardest around its gaps.

The same thread shows where the useful signal sits instead: in the errors, grouped by cause. PostHog's four insight tools each expected a different key for the same ID, so agents kept mixing them up. A required project ID that agents had no way of knowing produced a steady stream of validation failures until it became optional. Neither problem looks like anything in a volume chart. Both are obvious once errors are counted by category and tool.

This is the same trap as [measuring AI spend instead of AI value](https://sawinyh.com/blog/measuring-ai-spend-not-value/): the number you can count with precision stands in for the thing you care about. The fix is the same too. Change the unit from activity to outcome.

The outcome unit in the agent channel is the job: the sequence of calls an agent makes on behalf of one request, ending in done, failed, or unknown.

<div style="display:flex;justify-content:center;margin:1.5em 0;"><pre style="background:#24292e;color:#e1e4e8;padding:1rem 1.5rem;border-radius:6px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:0.9em;line-height:1.5;margin:0;width:max-content;max-width:100%;white-space:pre;word-wrap:normal;box-sizing:border-box;overflow-x:auto;">  user asks the agent for something
                 │
                 ▼
        job ID minted by your server
                 │
                 ▼
   search_accounts ──► error ──► retry ─┐
                 │                      │
                 ▼                      │
   search_accounts ◄────────────────────┘
                 │
                 ▼
   export_report
                 │
     ┌───────────┼──────────────┐
     ▼           ▼              ▼
   DONE        FAILED        UNKNOWN
 report row   error was     agent stopped,
 written in   the last      nothing in your
 your DB      thing seen    product changed</pre></div>

That picture gives you three numbers, and they belong on the same chart:

- **Completed jobs**, counted by people and accounts, not by calls
- **Calls per completed job**, which should fall as your tools get better, and retry incidence beside it
- **Outcome coverage**: the share of jobs whose result you could determine at all

The first decision is what "done" means, and the answer has to come from your product, not from the agent. Agents rarely announce that they finished, and when they do, it's the model's opinion. A state change in your own system is evidence: a record created, a report written, a status moved, an export downloaded. Pick that event per job type and you have a completion signal you can trust.

The third number is the one teams skip and shouldn't. Plenty of jobs end with the agent stopping and nothing in your product changing. Maybe the user got what they needed from a read-only answer. Maybe they gave up. You cannot tell, and folding those jobs into either success or failure manufactures a result. Report them as unknown, and report how many there are. It's the same lesson as [publishing how blind your instrument is](https://sawinyh.com/blog/proposed-a-metric-then-computed-it/): a completion rate is only as meaningful as the share of outcomes it could see. A completion rate that rises while coverage falls hasn't improved anything. Flowlines makes the same point in its own metric definitions, and it is the most important thing in them.

The fair objection is that defining jobs is work, and calls are free. That's true, and it's why the call chart gets drawn first. But you only have to define jobs for the handful of things people mostly use your server for, and the intent data from the previous section tells you what those are.

## Where to start on Monday

You don't need a vendor decision to begin, and you shouldn't make one before you know which questions you want answered.

Start with the join. Make sure every tool call is recorded against the same person and account IDs your product events already use. Without that, the agent channel will always be a separate dashboard nobody opens.

Next, mint a job ID, because the protocol no longer gives you one. A handle your server returns from the first call and accepts on the next ones is enough.

Add the intent argument and the missing-tool tool, and read the raw answers for a week before you cluster anything. That week will tell you more about what people bring to your product through agents than the call chart will in a quarter.

Once you know what people bring, pick the two or three jobs that matter most, define "done" for each as a state change in your product, and put completed jobs, calls per job and outcome coverage on one chart. Retire the call-volume chart, or at least move it to the monitoring dashboard where it belongs.

If you want the layer underneath this, [where an MCP server sits next to skills and CLAUDE.md](https://sawinyh.com/blog/claude-code-skills-vs-mcp-vs-claude-md/) covers what a server is for from the agent's side.

## Frequently asked questions

### What is MCP analytics?

MCP analytics is product analytics for the agent channel: measuring who uses your MCP server, which tools they call, why they call them, and whether the job behind the calls got done. It differs from MCP monitoring, which asks whether calls succeed and how fast they run. A server can be healthy by every monitoring measure while agents use it to do nothing of value, so a product team needs both.

### What should a product team capture on every MCP tool call?

At minimum: the tool name, whether it errored and how, the duration, the client's self-reported name and version, the protocol version, the authenticated user and account, and a job identifier your server mints so that related calls can be grouped. Add the agent's stated intent if you can get it. Capture arguments and responses only in sanitized form, and never assume the model name a client reports is accurate.

### How do you find out why an agent called a tool?

Add an optional argument to each tool's input schema that asks the agent to state the user's goal, then record the answer as a property on the event. PostHog's MCP SDK does this by default with a context argument and records it as $mcp_intent, noting whether the agent supplied it or a fallback inferred it. Treat the answer as the model's paraphrase of the user, not the user's words, and track how often clients leave it blank.

### Why is MCP tool call volume a bad success metric?

A rising call count fits two opposite stories: more people getting more done, or the same people's agents retrying, looping and failing to finish. Volume cannot tell them apart. Count completed jobs instead, measured from a state change in your own product, and report calls per completed job and retry incidence beside it. Publish the share of jobs whose outcome you could not determine, because unknown is not the same as success.


---

Source: Nick Sawinyh, https://sawinyh.com/blog/mcp-analytics-for-product-managers/
Nick Sawinyh writes on AI, engineering, product, and crypto business development at sawinyh.com.
Site index for agents: https://sawinyh.com/llms.txt
