Skip to content
fernet.consultores

Reference scenarioSplunk Observability CloudMetrics and costExecutive and technical · 3 min read

Our metric volume keeps growing and nobody knows which metrics we actually use.

Written by Fernando Mateo Ruiz · Splunk Enterprise Certified Architect · Splunk O11y Cloud Certified Metrics User · Reviewed on 28 October 2026

Reference scenario: it describes a common situation and how we tackle it. It is not a client project and includes no measured results.

In 30 seconds

The problem: The number of metric time series grows with every deployment, the subscription limit gets closer, and most of those series never appear in a chart or a detector.

What solves it: Metrics Pipeline Management: aggregate on the dimensions you need, archive what you only need now and then, and drop what nobody uses, without touching the Collector.

What changes: Less volume and cost, while the charts and detectors that matter keep working.

Sound familiar?

  • One dimension like container_id or user_id multiplied the series.
  • Usage is close to the limit and the answer is “buy more”.
  • Nobody dares delete a metric in case someone uses it.
  • Every team sends everything the agent collects by default.

The question to answer

Which metrics and dimensions do we actually query, and how much are we paying to keep the ones we don't?

Why it matters

Cardinality grows silently. Controlling it centrally avoids surprises in cost and limits, and keeps charts and detectors fast.

Signals needed

  • EssentialMetric usage and cardinality
  • EssentialInventory of charts and detectors
  • RecommendedCollector configuration

Architecture

Reference scenario: the architecture is generic and does not belong to any client.

How it’s implemented

  1. Use the metric usage reports to find the metrics with the most time series and the dimensions that cause them.
  2. Check which charts and detectors use each metric and which dimensions they filter or group by.
  3. Create an aggregation rule that keeps only those dimensions (for example region instead of instance_id and host_name). It applies to new data, not to data already ingested.
  4. Move the charts and detectors to the aggregated metric before touching the original.
  5. Send the original to archived metrics if you might need it again, or drop it if nobody does. Archived metrics can't be used in charts or detectors until you route them back, and dropped data can't be recovered.
  6. For metrics nobody should be sending at all, filter them in the Collector.
Stop sending at source what nobody uses (Collector) · example
processors:
  filter/unused:
    error_mode: ignore
    metrics:
      metric:
        - 'name == "system.network.connections"'
# Add filter/unused to the processors of your metrics pipeline.

Step-by-step investigation

An illustrative example of how it would be investigated, from signal to action.

  1. 01 SignalMetric time series usage grows month after month and is getting close to the subscription limit.
  2. 02 ContextMost of the growth comes from one latency metric since a new Kubernetes cluster went live.
  3. 03 CorrelationThe usage report shows its pod and container dimensions multiply the series; the dashboards only group by service and region.
  4. 04 CauseEvery pod restart creates new series that nobody ever queries one by one.
  5. 05 DecisionAggregate by service and region, and keep the original archived for a month as a safety net.
  6. 06 ActionCreate the aggregation rule, move charts and detectors, route the original to archive and review it a month later.

Common mistakes

  • Dropping a metric before moving the charts and detectors that use it.
  • Using IDs or URLs with parameters as dimensions.
  • Expecting the rules to change data that's already been ingested.
  • Cutting at source a metric another team still uses, without asking.

What this case doesn’t solve

  • It doesn't apply to APM MetricSets or organisation metrics.
  • Metrics Pipeline Management is part of Enterprise Edition: check your subscription.

Expected result

Fewer time series and lower cost, with every dashboard and detector that matters still working.

Observed result

This is a reference scenario: we don’t publish measured results until it has been tested in our lab or in an authorised project.

Requirements

Platform

Splunk Observability Cloud Enterprise Edition

Permissions

Access to metrics pipeline management

Agreement

An owner for each metric family

Related services

Questions we’d ask you

  1. How close are you to your metric limit, and how fast has it grown?
  2. Do you know which metrics feed your dashboards and detectors?
  3. Who decides which dimensions each team can send?
Let’s talk about your case