Deploying Tempo

Single-Binary Configuration

# tempo-config.yaml — single-binary mode
server:
  http_listen_port: 3200

distributor:
  receivers:
    otlp:
      protocols:
        grpc:
          endpoint: 0.0.0.0:4317   # Receives traces from OTel Collector
        http:
          endpoint: 0.0.0.0:4318

ingester:
  max_block_duration: 5m          # Flush to storage after 5 minutes
  trace_idle_period: 10s

storage:
  trace:
    backend: local                  # Use 's3' for production
    local:
      path: /tmp/tempo/blocks
    wal:
      path: /tmp/tempo/wal          # Write-Ahead Log

querier:
  frontend_worker:
    frontend_address: 127.0.0.1:9095

compactor:
  compaction:
    block_retention: 168h           # 7 days

metrics_generator:
  registry:
    collection_interval: 15s
  storage:
    path: /tmp/tempo/generator/wal
  remote_write:
    - url: http://prometheus:9090/api/v1/write  # Write span metrics to Prometheus

overrides:
  metrics_generator_processors: [service-graphs, span-metrics]

Production Configuration with S3

# tempo-config.yaml — production with S3 backend
storage:
  trace:
    backend: s3
    s3:
      bucket: my-bizfirst-tempo-bucket
      endpoint: s3.us-east-1.amazonaws.com
      region: us-east-1
      access_key: "${AWS_ACCESS_KEY_ID}"
      secret_key: "${AWS_SECRET_ACCESS_KEY}"
    pool:
      max_workers: 100
      queue_depth: 10000
    wal:
      path: /var/tempo/wal          # Local WAL (SSD recommended)

compactor:
  compaction:
    block_retention: 168h           # 7 days hot; S3 lifecycle handles cold storage

Resource Requirements

Metrics Generator

Tempo's built-in metrics generator can compute service graph metrics and span metrics from ingested traces and write them to Prometheus via remote write. This enables span-derived RED metrics (Rate, Error, Duration) without any additional instrumentation:

# Service graph metrics generated by Tempo:
# traces_service_graph_request_total{client,server}
# traces_service_graph_request_failed_total{client,server}
# traces_service_graph_request_server_seconds_bucket{client,server,le}

# Span metrics:
# traces_spanmetrics_calls_total{service,operation,status_code}
# traces_spanmetrics_latency_bucket{service,operation,le}