Backup

Why restic + Backblaze B2?

The combination of restic and Backblaze B2 provides enterprise-grade data protection at a fraction of the cost of native S3 backup solutions. Both components are production-proven, open-standard, and interoperable with any S3-compatible toolchain.

Architecture

The backup pipeline runs as a separate Docker container. It mounts the MinIO data volume read-only, so it cannot modify or corrupt production data. Encrypted chunks are pushed directly to Backblaze B2 via the S3-compatible API. The production MinIO service is never paused or disrupted during backup.

The backup schedule is driven by the ofelia job scheduler (see Setting Up the Cron Schedule):

Docker Compose — Backup Service

Add the following service to your existing docker-compose.yml alongside the minio service. The backup container mounts the MinIO data volume as read-only and the backup scripts directory.

# docker-compose.yml — append to existing services block

  backup:
    image: restic/restic:latest
    container_name: bizfirst-backup
    environment:
      RESTIC_REPOSITORY: s3:https://s3.us-west-004.backblazeb2.com/bizfirst-backup
      RESTIC_PASSWORD: ${BACKUP_ENCRYPTION_KEY}
      AWS_ACCESS_KEY_ID: ${B2_APPLICATION_KEY_ID}
      AWS_SECRET_ACCESS_KEY: ${B2_APPLICATION_KEY}
    volumes:
      - minio-data:/data:ro
      - ./backup-scripts:/scripts
    entrypoint: /bin/sh
    command: /scripts/backup.sh
    depends_on:
      - minio

The container does not run continuously. It is invoked on demand by the cron scheduler (see Setting Up the Cron Schedule). When triggered, it runs backup.sh, completes, and exits. No long-running backup daemon is required.

Backup Script

Create the script at ./backup-scripts/backup.sh relative to your docker-compose.yml. Ensure it is executable (chmod +x backup.sh). The script handles first-run initialization, backup, retention enforcement, and integrity verification in a single execution.

#!/bin/sh
# backup.sh — restic backup script for InfraHub Storage
# Runs inside the bizfirst-backup container

# Initialize repository if it does not yet exist
# (idempotent — safe to run on every invocation)
restic snapshots > /dev/null 2>&1 || restic init

# Run incremental backup of the MinIO data volume
restic backup /data \
  --tag bizfirst-storage \
  --hostname bizfirst-minio

# Apply retention policy and prune orphaned pack files
restic forget \
  --keep-daily   7  \
  --keep-weekly  4  \
  --keep-monthly 12 \
  --keep-annual  2  \
  --prune

# Verify latest snapshot integrity (samples 5% of pack files)
# Full --read-data check can be run manually on a schedule (e.g., monthly)
restic check --read-data-subset=5%

echo "Backup completed: $(date)"

The --tag bizfirst-storage flag labels every snapshot, making it easy to filter snapshots in restic snapshots --tag bizfirst-storage. The --hostname flag identifies the source host in multi-host restic repositories.

Retention Policy

The retention policy balances recovery granularity against storage cost. Recent backups are kept daily for maximum recovery precision; older backups are consolidated to weekly, monthly, and annual checkpoints.

With this policy, approximately 25 snapshots are retained at any given time. Due to restic's block-level deduplication, the total B2 storage consumed by 25 snapshots of a 50 GB dataset is typically less than 3× the dataset size — restic only stores unique blocks across all snapshots. Snapshots outside the retention window are automatically pruned during each backup run.

Setting Up the Cron Schedule

The backup container is triggered by ofelia, a Docker-native cron scheduler. Ofelia reads scheduling configuration from Docker container labels and triggers the target container at the configured interval without requiring a separate system cron or SSH access.

# docker-compose.yml — add alongside the backup service

  backup-scheduler:
    image: mcuadros/ofelia:latest
    container_name: bizfirst-backup-scheduler
    command: daemon --docker
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    labels:
      ofelia.job-run.backup.schedule: "0 2 * * *"   # 02:00 UTC daily
      ofelia.job-run.backup.container: bizfirst-backup
    restart: unless-stopped

The cron expression 0 2 * * * means: at minute 0, hour 2, every day, every month, every weekday. Adjust the hour to match your lowest-traffic window if 02:00 UTC is not ideal for your deployment region.

Verifying a Backup

Always verify your backup configuration after initial setup, and perform periodic manual verifications (at minimum monthly). A backup that has never been verified is not a backup — it is an untested assumption.

List All Snapshots

# List all snapshots in the repository
docker exec bizfirst-backup restic snapshots

# Filter by tag
docker exec bizfirst-backup restic snapshots --tag bizfirst-storage

# Show detailed snapshot info
docker exec bizfirst-backup restic snapshots --verbose

Check Repository Integrity

# Fast structural integrity check (no data download)
docker exec bizfirst-backup restic check

# Check with data verification — samples 10% of pack files (recommended monthly)
docker exec bizfirst-backup restic check --read-data-subset=10%

# Full data verification — reads ALL pack files from B2 (run quarterly)
# This will incur B2 API call costs for large repositories
docker exec bizfirst-backup restic check --read-data

Inspect a Specific Snapshot

# List files in the latest snapshot
docker exec bizfirst-backup restic ls latest

# List files in a specific snapshot by ID
docker exec bizfirst-backup restic ls abc12345

# Show snapshot stats (unique data, total size)
docker exec bizfirst-backup restic stats latest

Environment Variables Required

All sensitive values are passed to the backup container as environment variables from your .env file or secrets manager. Never hard-code these values in docker-compose.yml or the backup script.

Generating a Strong Encryption Key

# Linux / macOS — generate a 64-character random key
openssl rand -base64 48

# Windows PowerShell
[Convert]::ToBase64String([System.Security.Cryptography.RandomNumberGenerator]::GetBytes(48))

# Or use your password manager's secure password generator (64+ chars, full character set)

Monitoring Backup Status

The backup container writes all output to stdout, which Docker captures as container logs. In a Grafana + Loki setup, these logs are automatically indexed and queryable.

Viewing Backup Logs

# View the last backup run logs
docker logs bizfirst-backup

# Follow logs of the next backup run (useful during testing)
docker logs bizfirst-backup --follow

# View logs with timestamps
docker logs bizfirst-backup --timestamps

Grafana + Loki Alert: Missing Daily Backup

In your Grafana instance, create a Loki-based alert rule that fires when the expected daily "Backup completed" log line has not appeared within 26 hours. The 26-hour window gives a 2-hour grace period beyond the 24-hour schedule interval, accommodating minor delays without false positives.

# Loki query — counts "Backup completed" lines in last 26 hours
# Alert fires when count = 0
count_over_time(
  {container="bizfirst-backup"} |= "Backup completed"
  [26h]
)

Configure the alert rule in Grafana as follows:

Prometheus Metric (Optional)

For teams running Prometheus, the backup script can emit a metric via the Pushgateway at the end of each run. Add the following to the end of backup.sh:

# Push backup timestamp metric to Prometheus Pushgateway
BACKUP_TIMESTAMP=$(date +%s)
echo "bizfirst_backup_last_success_timestamp $BACKUP_TIMESTAMP" | \
  curl --data-binary @- \
  http://pushgateway:9091/metrics/job/restic_backup/instance/bizfirst-minio

Then create a Prometheus alert on time() - bizfirst_backup_last_success_timestamp > 90000 (25 hours) to catch missed backups.

Backup Storage Cost Estimate

Backblaze B2 charges $6.00/TB/month for storage (as of 2026). With restic's block-level deduplication and incremental backups, the total backup repository size is typically 1.5× to 3× the current dataset size, depending on how frequently files change.

Egress fees are $0.00 when restoring to a Cloudflare-protected host (Cloudflare is a B2 bandwidth alliance partner). For non-Cloudflare egress, B2 charges $0.01/GB after the first 1 GB/day free.

For comparison, storing the same backup repository in AWS S3 Standard would cost approximately 3.8× more at $23/TB/month. For most BizFirst AI deployments (under 250 GB of production file data), the total annual backup cost on B2 is under $40.

Estimating Your Actual Deduplication Ratio

After your first week of backups, run restic stats to see the actual repository size vs. total data size. This gives you a precise deduplication ratio for your workload:

# Show repository statistics including dedup ratio
docker exec bizfirst-backup restic stats --mode raw-data

# Example output:
# Stats for all snapshots in restore-size mode:
# Total File Count:   142,847
# Total Size:         48.231 GiB   (your MinIO data size)
# Stats for all snapshots in raw-data mode:
# Total Blob Count:   89,234
# Total Size:         12.441 GiB   (actual B2 usage after dedup)

In the example above, 7 daily snapshots of 48 GB of data occupy only 12 GB in B2 — a 3.9× deduplication ratio. For datasets where files rarely change (archived invoices, completed reports), deduplication ratios above 10× are common.