BizFirst Observe
Trace Attribute Filtering
Trace spans carry attributes that describe what the span was doing — HTTP method, URL, database query, RPC method. Some of these attributes can contain PII (HTTP headers with tokens, database queries with user data, RPC payloads). The OTel Collector's attribute and redaction processors control what reaches Tempo.
Common Sensitive Span Attributes
| Attribute | Risk | Action |
|---|---|---|
http.request.header.authorization | Contains Bearer tokens or Basic auth credentials | Block entirely |
http.request.header.cookie | Session tokens, tracking cookies | Block entirely |
db.statement | SQL queries may contain WHERE clauses with user data | Block or truncate |
http.url / url.full | Query parameters may contain tokens or user data | Strip query string |
rpc.request.metadata | gRPC metadata may contain auth headers | Block entirely |
exception.message | Stack traces may include user data from exceptions | Truncate to 500 chars |
OTel Collector Attribute Filter for Traces
# otel-collector-config.yaml — trace attribute filtering
processors:
attributes/scrub-spans:
actions:
# Block sensitive HTTP headers:
- key: http.request.header.authorization
action: delete
- key: http.request.header.cookie
action: delete
- key: http.request.header.x-api-key
action: delete
- key: http.response.header.set-cookie
action: delete
# Remove SQL query content (keep db.system and db.name for context):
- key: db.statement
action: delete
# Remove full URL query params — keep path only:
# Replace "https://api/endpoint?token=abc&user=xyz" with "https://api/endpoint"
- key: url.full
action: extract
pattern: "^(?P<url_path>[^?]+)"
- key: url.full
from_attribute: url_path
action: insert
- key: url_path
action: delete
# Truncate exception messages to prevent data leakage in stack traces:
- key: exception.message
action: update
value: "exception.message truncated by policy" # Or use transform processorBizFirstGO Approved Span Attributes
These attributes are safe and provide useful debugging context without PII risk:
# Safe span attributes set by BizFirstGO's OTel instrumentation:
workflow.execute span:
- workflow.id (internal GUID — not PII)
- workflow.type (e.g., "approval-workflow")
- tenant.id (internal tenant identifier)
- execution.id (internal execution GUID)
- execution.status (success/failed/cancelled)
node.execute span:
- node.type (e.g., "HttpRequestNode", "ApprovalNode")
- node.id (internal GUID)
- node.name (node label from workflow definition)
- duration.ms (execution duration)
hil.suspend span:
- hil.task_id (internal GUID)
- hil.role_required (e.g., "Manager")
- hil.timeout_hours (numeric)
# NEVER: hil.assignee_email, hil.assignee_name
Auto-Instrumentation Attributes Require Review
The OTel SDK auto-instrumentation (ASP.NET Core, HttpClient, SqlClient) automatically captures many span attributes — including potentially sensitive ones like full HTTP URLs and SQL statements. Always review what auto-instrumentation is capturing and apply the attribute filter processor to block sensitive attributes before they reach Tempo.