Metric schema (core, low-cardinality)

All metrics prefixed mcpjungle_ and instrumented as OTel instruments; the Prometheus names will be converted appropriately.

counters

1. mcpjungle_tool_invocations_total

• Labels: org_id, server_id, tool_name, status (ok|error|timeout|rate_limited), client_type (cursor|claude|custom)

2. mcpjungle_client_requests_total

• Labels: org_id, rpc_method (list_tools|call_tool|describe_server|health), status

3. mcpjungle_server_registrations_total

• Labels: org_id, server_kind (http|grpc|process), result (created|updated|deleted|error)

4. mcpjungle_authn_failures_total

• Labels: org_id, reason (invalid_token|forbidden|expired)

Histograms (explicit buckets; seconds)

1. mcpjungle_tool_latency_seconds

• Labels: org_id, server_id, tool_name, status
• Buckets: [0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10]

2. mcpjungle_request_duration_seconds

• Labels: org_id, rpc_method, status

Gauges / UpDownCounters

1. mcpjungle_connected_sessions

• Labels: org_id, client_type

Just an overview for the metrics we could scrap, could change as per requirements.

@ankittk thanks, this is very well put!

When implementing some new functionality in mpcjungle, we always want to start by implementing the minimal thing that works well and doesn't break anything else. Every other extra thing or improvement comes only after user feedback

So keeping this in mind, here are a few comments from me:

1. We can simplify the scope and description by only covering the following:
   Instrument MCPJungle with OTel SDK (Go) → export via OTLP to an OTel Collector + expose a Prometheus-scrape endpoint
   IIUC, the rest is the user's responsibility (what they do with the metrics).

2. For now, we'll only focus on providing metrics (counters + histograms + gauge), as you suggested.
   But of course, we'll keep this dicsusion open because later, we also want to add logs & traces in our otel.

3. Regarding the labels:
   3.1 What does org_id mean? We currently don't have any concept of org on mcpjungle, so what value will this label contain?
   3.2 Does server_id refer to the mcp server? If yes, it is better to call it mcp_server_id (or mcp_server_name, since all mcp server names registered in mcpjungle are unique). server could also refer to mcpjungle server itself, so it might cause confusion. So just to create clear distinction - server = mcpjungle server, mcp_server = mcp servers.
   3.3 client_type - How do we get this info? I believe the client will supply their type in the request Headers? Another thing to keep in mind - in production mode, every client of mcpjungle is an authenticated client, ie, they use their API key to access mcpjungle. So maybe we'll need a way to let the user filter metrics based on the authenticated client. What do you think?
   3.4 rpc_method. - A more suitable name for this is mcp_method. Our users don't care that MCP internally uses (json) RPC. But they understand mcp methods.
   3.5 server_kind (http|grpc|process) - what is this for? Did you mean the mcp server's transport type? Their values would be streamable_http | stdio for now, since those are the things mcpjungle supports. And label name can be mcp_server_transport (or just transport)

4. I think for starters, the following metrics are good enough (but I'm still researching more on this and might update this list):
   4.1 mcpjungle_tool_invocations_total
   4.2 mcpjungle_client_requests_total
   4.2 tool latency
   4.2 request duration

5. Why measure the latencies in seconds and not milliseconds? I think ms is better, all mcp servers work in ms (or at least aim to be that performant) because they're in the critical path.

6. Behavior: Agree, otel should only kick in in production mode. This is mainly for enterprises. But we don't need ENABLE_OTEL env var. If the user supplies an otel endpoint in the env var, we treat it as otel enabled. If no endpoint is supplied, we keep it disabled (default behaviour).

7. As an MVP, we only want to provide the user with enough metrics so they could answer some basic questions like
   7.1 How many tool calls are we receiving right now?
   7.2 How many tool calls are success/failure/unknown?
   7.3 Latency of tool calls?
   7.4 (and some of the generic metrics like total request count to mcpjungle server, 5xx count, etc)

Auth failures and rate limits should be considered?

Not right now.

ps- I too haven't worked extensively on implementing otel myself, so I could be wrong about some of the things above. Feel free to correct me.

Cool thats the plan

• All names follow Prometheus conventions and voids unit mismatches across tooling, aggregation, and exemplars. durations in seconds with _seconds suffix (thats the industry best practice). Display as milliseconds in Grafana and pick ms-tuned buckets but expressed in seconds.

• You're right we don't need org_id, it was kind of an example. Will omit this.

• server_id: Yes, it refers to the MCP server or Use mcp_server_name (unique in registry). Disambiguates from the MCPJungle service itself. What do you think?

• client_type : We have to add a custom header such as X-MCP-Client: cursor|claude|custom if absent set unknown. that was the idea. We can add it later anytime though.

• mcp_server_transport (values today: streamable_http|stdio) is better.

• Prometheus vs VictoriaMetrics (what to follow)
  Industry practice: Prometheus for scraping, short-term Time Series Database, rules, and alerting.
  Optional remote_write to VictoriaMetrics (or Thanos/Mimir) for long-term, scalable retention. Would be a great TODO (can come back on this in future)

• FOR the MVP

1. How many tool calls right now? : sum(rate(mcpjungle_tool_invocations_total[1m]))
2. Success/failure/unknown split? - sum(rate(mcpjungle_tool_invocations_total[5m])) by (status)
3. Latency of tool calls? :

• P95: histogram_quantile(0.95, sum(rate(mcpjungle_tool_latency_seconds_bucket[5m])) by (le))
• P99 : histogram_quantile(0.99, sum(rate(mcpjungle_tool_latency_seconds_bucket[5m])) by (tool_name, le))

4. Generic Server metrics like

• Requests: sum(rate(mcpjungle_client_requests_total[5m])) by (mcp_method)
• Error Ratio: sum(rate(mcpjungle_client_requests_total{status=~"5.."}[5m])) / sum(rate(mcpjungle_client_requests_total[5m]))

I think we are clear on this now. What do you think?

@ankittk Thanks for clarifying.

• let's go ahead with mcp_server_name then.
• let's leave out client_type for now, this needs more clarity & feedback
• I don't want mcpjungle to be tied to any tool. That's why, it's best to only implement the standard industry practices of otel - to connect to an OTEL endpoint (if provided) to push metrics, and provide a metrics scraping endpoint so any prometheus-compatible tool could read metrics from mcpjungle server. So we'll think about the rest in future.

So yes, I believe we're in agreement about the first iteration of implementing OTEL 👍

Version 0.2.5 introduces OpenTelemetry in mcpjungle, all thanks to @ankittk 🚀

Currently, prometheus-compatible metrics are exposed at /metrics with some basic metrics available.
See docs
I've opened issue #93 & #92 for some follow-up tasks as well.

We're collecting feedback on what metrics will be useful to people, feel free to drop your suggestions in this discussion.

Feedback from a user

I'd want to see failed tool executions and retry patterns. How many attempts it took for something to actually work, since repeated failures usually mean either the prompts need work or the tool definitions are confusing the LLM.
Also tracking which tools fail most often and why. In my experience, a lot of retry loops come from the LLM generating malformed JSON, wrong parameter types, or missing required fields rather than actual server issues. Having visibility into these validation failures versus real runtime errors helps you know whether to fix your tool schemas or adjust your prompting approach.

We can use Go (OTel SDK) instrumentation [go.opentelemetry.io/otel/sdk]

Only initialize OTel when ENABLE_OTEL=1 AND DEPLOYMENT_ENVIRONMENT=production.
In non-prod, no exporter is created.

Prometheus configuration

• Scrapes the Collector’s /metrics
• Evaluates rules and alerts
• Remotely writes to VictoriaMetrics

Few questions:

1. What to measure first, MVP?
2. MCP metrics according to you?
3. Auth failures and rate limits should be considered?

Open to chat on that, thanks :)

PS: I haven't worked on these but research all that for this.