Default Stack Overview
The default BizFirst Observe deployment uses the Grafana Labs open-source stack: Loki for logs, Prometheus for metrics, Tempo for distributed traces, and Grafana as the unified UI. This combination provides complete observability for BizFirstGO with no licensing cost.
The Four-Component Stack
Grafana Loki
Log aggregation. Stores structured JSON logs from all BizFirstGO services. Queryable with LogQL. Cost-efficient: indexes only labels, not log content.
Prometheus
Metrics collection. Scrapes /metrics endpoints every 15 seconds. Time-series database (TSDB). Queryable with PromQL. Powers alerts via Alertmanager.
Grafana Tempo
Distributed tracing. Ingests traces via OTLP. Stores in object storage. Queryable by TraceId or attribute search. Links to logs and metrics.
Grafana
Unified visualization. Connects to all three as data sources. Pre-built dashboards for BizFirstGO. Alert management. Single login for all signals.
Why This Combination?
| Requirement | Alternative Considered | Why Loki Stack |
|---|---|---|
| Log aggregation | Elasticsearch / OpenSearch | Loki is 10x cheaper at scale; does not require full-text index; label-based query is sufficient for BizFirstGO's structured logs |
| Metrics | InfluxDB, Datadog | Prometheus is the industry standard; rich ecosystem; PromQL is widely understood; open-source |
| Distributed tracing | Jaeger, Zipkin, Datadog APM | Tempo is OpenTelemetry-native; object storage backend (cheapest); native Grafana integration; no index database required |
| Unified UI | Kibana, separate per-component UIs | Grafana connects to all three; single login; cross-signal correlation in one tool; pre-built dashboards for BizFirstGO |
Component Interoperability
The four components interoperate via defined integration points. Understanding these links is key to the cross-signal correlation that makes debugging efficient:
- Loki → Grafana: Grafana queries Loki via LogQL; Derived Fields in the Loki data source config render
traceIdin log lines as clickable links to Tempo - Prometheus → Alertmanager: Prometheus evaluates alert rules and sends firing alerts to Alertmanager, which routes them to Slack/email/PagerDuty
- Prometheus → Grafana: Grafana queries Prometheus via PromQL; histogram exemplars link data points to Tempo traces
- Tempo → Grafana: Grafana queries Tempo by TraceId or via TraceQL search; Tempo's trace detail view shows trace-to-log links back to Loki
- Grafana Unified Alerting → All three: Grafana can create alert rules against any of the three data sources
Suitable Deployment Scale
| Scale | Workflow Executions/Day | Deployment Mode | Notes |
|---|---|---|---|
| Development | < 1,000 | Docker Compose (single node) | SQLite for Grafana; local storage for all |
| Small production | 1,000 – 20,000 | Docker Compose or single-node Kubernetes | S3 for Loki + Tempo; Prometheus on SSD |
| Medium production | 20,000 – 200,000 | Kubernetes with Helm | Default architecture still sufficient; may need Loki distributed |
| Large production | 200,000+ | Enterprise (see Guide11) | Loki microservices; Thanos; Grafana Enterprise |
If you prefer not to self-host this stack, all four components are available as managed services via Grafana Cloud, Azure Monitor, or AWS-native equivalents. The BizFirstGO OTel SDK is configured identically regardless of whether you use self-hosted or managed backends — only the OTel Collector exporter configuration changes. See Guide12: PaaS Options.