Collector Configuration

The Collector chart deploys a fixed two-layer topology:

• two stateless OTLP routers;
• two stateful service-graph backends.

Both replica counts are fixed at two by the chart schema.

Trace-affine routing

Every router uses the same static hash ring containing the stable DNS names of backend-0 and backend-1. The load-balancing exporter hashes traceID, so all spans for one trace reach the same backend.

This is required because the service-graph connector pairs client and server spans in memory. Sending parts of a trace to different backends can create unpaired spans and missing edges.

Changing the backend count remaps the hash ring and can split in-flight traces. Treat it as a planned topology migration, not routine autoscaling.

OTLP endpoints

The router Service exposes:

Protocol	Port	Endpoint
OTLP gRPC	4317	<release>-router:4317
OTLP HTTP	4318	http://<release>-router:4318/v1/traces

Backends receive OTLP gRPC only through their headless Service.

When entity-event forwarding is enabled, producers send OTLP logs to the same router endpoint: gRPC on 4317 or HTTP at http://<release>-router:4318/v1/logs.

Optional entity-event forwarding

Entity-event forwarding is disabled by default. Enable it and select the externally created Kafka topic with:

streamContract:
  topics:
    servicegraphMetrics: otel.servicegraph.metrics
    entityEvents: otel.entity.events

entityEvents:
  enabled: true

The router adds a logs pipeline only when entityEvents.enabled=true. Its filter keeps log records with:

• event_name equal to entity.state or entity.delete; or
• the compatibility attribute otel.entity.event.type equal to entity_state, entity_delete, or entity_deleted.

All other logs are discarded from this dedicated pipeline. Matching records are batched and exported as OTLP JSON to streamContract.topics.entityEvents. They are not sent through the trace load-balancing exporter or the stateful service-graph backends.

The entity-event topic uses the same brokers, security protocol, SASL Secret, bounded sending queue, retries, compression, and disabled automatic topic creation as the metrics topic. Enabling Collector forwarding does not enable the Flink consumer automatically; enable the matching Flink setting and use the same topic name.

Generated dimensions

deploy/helm/servicegraph-collector/files/dimensions.yaml is generated from entities participating in service_graph relationships:

python -m tools.semconv_codegen

Do not edit this file manually. Add or change registry definitions, regenerate it, and review the resulting cardinality.

Metric temporality

Each service-graph backend owns an independent cumulative counter stream. Publishing both cumulative streams to one Kafka topic would make interleaved values resemble counter resets.

The backend pipeline therefore converts its connector-local cumulative metrics to delta before Kafka. Flink treats non-zero deltas as activity and ignores idle zero deltas.

Main values

image:
  repository: registry.internal.example/otelcol-contrib
  tag: "0.156.0"

backend:
  serviceGraph:
    storeTtl: 10s
    storeMaxItems: 10000
    metricsFlushInterval: 5s

streamContract:
  kafka:
    brokers:
      - kafka.internal.example:9093
    security:
      protocol: SASL_SSL
      saslMechanism: SCRAM-SHA-256
      existingSecret: servicegraph-kafka-auth
      usernameKey: username
      passwordKey: password
  topics:
    servicegraphMetrics: otel.servicegraph.metrics
    entityEvents: otel.entity.events

entityEvents:
  enabled: false

storeTtl is the span-pairing retention inside the connector. It is separate from Flink's contributor TTL.

Kafka security

Supported modes are:

• PLAINTEXT for a trusted local environment;
• SASL_PLAINTEXT with SCRAM-SHA-256 for authentication without encryption on a trusted internal network;
• SASL_SSL with SCRAM-SHA-256.

The named Secret must already exist in the release namespace. It contains the username and password keys selected in values. SASL_SSL validates broker certificates through the Collector image's default trust store. The chart does not mount a Kafka CA file, create credentials, or create Kafka topics.

Reliability and limits

Both pipelines put memory_limiter first. Exporters use bounded in-memory queues and unlimited retry duration. A prolonged Kafka outage can fill the queue, at which point new telemetry may be rejected; the queue is not persistent across pod replacement.

Size these together:

• backend memory limit;
• connector storeMaxItems;
• generated dimension cardinality;
• router and backend queue sizes;
• Kafka outage tolerance.

Traffic between routers and backends is plaintext inside the cluster. Apply platform network isolation when this crosses a trust boundary.

The optional entity-event logs pipeline also uses an in-memory queue. A prolonged Kafka outage can therefore reject matching entity events, and queue contents do not survive router replacement.

Validate

helm lint deploy/helm/servicegraph-collector
helm template collection deploy/helm/servicegraph-collector \
  --namespace servicegraph-system \
  --values internal-collector-values.yaml

After deployment:

kubectl get pods -n servicegraph-system
kubectl logs -n servicegraph-system statefulset/servicegraph-collector-backend
kubectl get endpoints -n servicegraph-system servicegraph-collector-backend-headless