Connecting Data Sources
Grafana connects to Loki, Prometheus, and Tempo via data source plugins. The connection is configured once — either via the Grafana UI or via provisioning YAML files — and then available for all dashboards, Explore sessions, and alert rules.
Provisioning via YAML (Recommended)
Configure data sources as YAML provisioning files so Grafana starts up fully configured without manual UI steps. Place these files in /etc/grafana/provisioning/datasources/:
# grafana-provisioning/datasources/bizfirst-observe.yaml
apiVersion: 1
datasources:
# ── Loki ────────────────────────────────────────────
- name: Loki
uid: loki-uid
type: loki
url: http://loki:3100
access: proxy
isDefault: false
jsonData:
maxLines: 5000
timeout: 60
derivedFields:
- name: TraceID
matcherRegex: '"traceId":"(\w+)"'
url: '${__value.raw}'
datasourceName: Tempo
datasourceUid: tempo-uid
# ── Prometheus ──────────────────────────────────────
- name: Prometheus
uid: prometheus-uid
type: prometheus
url: http://prometheus:9090
access: proxy
isDefault: true
jsonData:
timeInterval: 15s
queryTimeout: 60s
httpMethod: POST
exemplarTraceIdDestinations:
- name: traceID
datasourceUid: tempo-uid
# ── Tempo ────────────────────────────────────────────
- name: Tempo
uid: tempo-uid
type: tempo
url: http://tempo:3200
access: proxy
isDefault: false
jsonData:
tracesToLogs:
datasourceName: Loki
datasourceUid: loki-uid
tags: ['service.name', 'job']
spanStartTimeShift: -5m
spanEndTimeShift: 5m
filterByTraceID: true
tracesToMetrics:
datasourceName: Prometheus
datasourceUid: prometheus-uid
tags:
- key: service.name
value: job
serviceMap:
datasourceUid: prometheus-uid
search:
hide: false
nodeGraph:
enabled: true
Verifying Data Source Connections
After provisioning or manual configuration, verify each data source:
Loki — Check label values are returning
Go to Explore → select Loki → Label Browser → confirm that job, environment, tenant_id labels are populated with BizFirstGO service names.
Prometheus — Check targets are UP
Go to Explore → select Prometheus → enter up → run query → confirm all BizFirstGO service targets show value 1.
Tempo — Check trace search works
Go to Explore → select Tempo → Search tab → select Service Name: processengine → run search → confirm traces appear in results.
Common Data Source Issues
| Issue | Data Source | Likely Cause | Fix |
|---|---|---|---|
| Label browser empty | Loki | No logs ingested yet, or wrong URL | Check Collector is running and sending to Loki |
up shows 0 for services | Prometheus | Services not reachable on scrape port | Check network connectivity and /metrics endpoint |
| Trace search returns no results | Tempo | No traces ingested, or retention expired | Check OTel Collector Tempo exporter; trigger a workflow |
| TraceId links don't work | Loki → Tempo | Derived Fields regex doesn't match log format | Adjust matcherRegex to match your actual log format |