Archiving to Cold Storage

S3 Lifecycle Rules for Loki Logs

{
  "Rules": [
    {
      "ID": "loki-log-archive",
      "Status": "Enabled",
      "Filter": {"Prefix": "loki/chunks/"},
      "Transitions": [
        {
          "Days": 30,
          "StorageClass": "STANDARD_IA"   // After 30 days: Infrequent Access (50% cheaper)
        },
        {
          "Days": 90,
          "StorageClass": "GLACIER"       // After 90 days: Glacier (80% cheaper than Standard)
        },
        {
          "Days": 365,
          "StorageClass": "DEEP_ARCHIVE"  // After 1 year: Deep Archive (cheapest)
        }
      ],
      "Expiration": {
        "Days": 2557                      // Delete after 7 years (2557 days)
      }
    }
  ]
}

S3 Lifecycle Rules for Tempo Traces

{
  "Rules": [
    {
      "ID": "tempo-trace-archive",
      "Status": "Enabled",
      "Filter": {"Prefix": "tempo/traces/"},
      "Transitions": [
        {
          "Days": 7,
          "StorageClass": "STANDARD_IA"   // After 7 days: cheaper storage
        }
      ],
      "Expiration": {
        "Days": 30                        // Delete traces after 30 days total
      }
    }
  ]
}

Prometheus Long-Term Archive via Thanos

# Thanos automatically uploads Prometheus TSDB blocks to S3.
# Apply lifecycle rules to the Thanos bucket for cost optimization:
{
  "Rules": [
    {
      "ID": "thanos-metrics-tiering",
      "Status": "Enabled",
      "Filter": {"Prefix": "thanos/"},
      "Transitions": [
        {
          "Days": 90,
          "StorageClass": "STANDARD_IA"
        },
        {
          "Days": 365,
          "StorageClass": "GLACIER_IR"    // Glacier Instant Retrieval for fast access
        }
      ]
    }
  ]
}

Cold Storage Retrieval

Data in S3 Glacier requires a restore step before it can be queried. For compliance retrieval requests:

# Restore an S3 Glacier object (e.g., for a compliance audit request):
aws s3api restore-object \
  --bucket bizfirst-loki-logs \
  --key "loki/chunks/tenant-financial-corp/2025/01/15/chunk-abc123.gz" \
  --restore-request '{"Days": 7, "GlacierJobParameters": {"Tier": "Standard"}}'

# "Standard" tier: restored within 3-5 hours
# "Bulk" tier: restored within 5-12 hours (cheaper)
# "Expedited" tier: restored within 1-5 minutes (more expensive)

# After restoration, the object is accessible for the specified Days count.
# Loki cannot directly query Glacier objects — you must restore to Standard first,
# then optionally re-import into a temporary Loki instance for querying.