Garage S3 Setup

Garage is an open-source, lightweight distributed S3-compatible storage system designed to be self-hosted on commodity hardware, bare-metal servers, or home labs with minimal resource usage.

---

Best Practices: Bucket & Cluster Setup

Follow these recommendations when configuring Garage for CI/CD caching:

1. Cluster Layout & Replication

• Single-Node CI Runner Cache: Set replication_factor = 1 in garage.toml if running on a single build server or local runner node to maximize available storage capacity.
• Multi-Node Cluster: Use replication_factor = 3 across multiple physical nodes or availability zones for high availability and zero-downtime runner caching.

2. Disk Space Protection & Bucket Quotas

Continuous integration workloads can quickly consume hundreds of gigabytes of storage if left unchecked. Garage provides built-in quota controls:

# Set a hard storage quota on the CI cache bucket (e.g. 50 GB)
garage bucket set --max-size 50G ci-cache

# Set a limit on the total number of cache objects (e.g. 10,000 objects)
garage bucket set --max-objects 10000 ci-cache

3. Periodic Garbage Collection

Run periodic garbage collection on your Garage cluster to prune stale blocks and reclaim disk space:

# Check block storage status and run GC
garage gc

4. Network Security

• The Garage S3 API port (3900) and Admin port (3902) should remain on internal networks, private VPCs, or encrypted VPN overlays (e.g. Tailscale / WireGuard).
• If exposing Garage across networks, place it behind an SSL/TLS reverse proxy (e.g. Traefik, Caddy, Nginx).

---

Credentials & Key Management

Garage provides a clean CLI for managing credentials and granting least-privilege bucket access:

1. Create a Dedicated API Key

# Create an access key named 'ci-runner-key'
garage key create ci-runner-key

Garage will output the Key ID (GK...) and Secret Key.

2. Create the Cache Bucket & Authorize Key

# Create the bucket
garage bucket create ci-cache

# Grant read and write permissions to the key
garage bucket allow ci-cache --read --write --key ci-runner-key

3. Verify Key Permissions

# Inspect key details and bound buckets at any time
garage key info ci-runner-key

4. Configure GitHub Secrets

Save the credentials in your repository or organization secrets:

Secret Name	Source
GARAGE_ACCESS_KEY	Output of garage key info ci-runner-key (Key ID)
GARAGE_SECRET_KEY	Output of garage key info ci-runner-key (Secret Key)

---

Workflow Configuration

- name: Cache dependencies using self-hosted Garage
  uses: xSAVIKx/cloud-cache-action@v1
  with:
    bucket: ci-cache
    endpoint: http://garage.internal:3900 # Replace with your Garage S3 endpoint
    provider: garage
    access-key: ${{ secrets.GARAGE_ACCESS_KEY }}
    secret-key: ${{ secrets.GARAGE_SECRET_KEY }}
    key: ${{ runner.os }}-build-${{ hashFiles('**/lock') }}
    path: build/

---

Running Garage Locally for Integration Testing

You can spin up Garage locally using the repository's docker-compose.test.yml:

docker compose -f docker-compose.test.yml up -d garage

docker-compose.test.yml (Click to view Compose definition)

name: cloud-cache-test

# Local S3 servers for the integration suite. Credentials match tests/support/s3Server.ts:
#   seaweedfs, minio, rustfs: cloudcacheci / cloudcachecisecret
#   garage: run `bash tests/ci/start-garage.sh` after `up`, then use the key in that script.
services:
  garage:
    image: dxflrs/garage:v2.4.1
    container_name: cloud-cache-garage
    ports:
      - '3900:3900' # S3 API
      - '3903:3903' # Admin API
    volumes:
      - ./tests/fixtures/garage.toml:/etc/garage.toml:ro
    environment:
      - RUST_LOG=info
    restart: unless-stopped

  seaweedfs:
    image: chrislusf/seaweedfs:4.46
    container_name: cloud-cache-seaweedfs
    ports:
      - '8333:8333' # S3 API
    volumes:
      - ./tests/fixtures/seaweedfs-s3.json:/etc/seaweedfs/s3.json:ro
    command: 'server -dir=/data -s3 -s3.port=8333 -s3.config=/etc/seaweedfs/s3.json -volume.max=0 -master.volumeSizeLimitMB=64'
    restart: unless-stopped

  minio:
    # Upstream MinIO no longer publishes community images; CleanStart rebuilds it from source.
    image: cleanstart/minio:0.20260330.001845@sha256:d5dc9098ea6239fc7215165c1496fa026abbb9560bdd7a67c4938e9226394839
    container_name: cloud-cache-minio
    ports:
      - '9000:9000'
    environment:
      - MINIO_ROOT_USER=cloudcacheci
      - MINIO_ROOT_PASSWORD=cloudcachecisecret
    tmpfs:
      - /data:uid=65532,gid=65532
    command: 'server /data'
    restart: unless-stopped

  rustfs:
    # RustFS serves S3 on 9000, the same port MinIO uses, so it is published on 9010 here to let
    # both run at once. The container runs as uid/gid 10001 and needs /data and /logs writable.
    image: rustfs/rustfs:1.0.0
    container_name: cloud-cache-rustfs
    ports:
      - '9010:9000' # S3 API
    environment:
      - RUSTFS_ACCESS_KEY=cloudcacheci
      - RUSTFS_SECRET_KEY=cloudcachecisecret
    tmpfs:
      - /data:uid=10001,gid=10001
      - /logs:uid=10001,gid=10001
    restart: unless-stopped

The Garage service requires a configuration file mounted at /etc/garage.toml. You can use the repository's tests/fixtures/garage.toml:

tests/fixtures/garage.toml (Click to view Garage configuration)

metadata_dir = "/tmp/garage/meta"
data_dir = "/tmp/garage/data"
db_engine = "sqlite"

replication_factor = 1

rpc_bind_addr = "0.0.0.0:3901"
rpc_public_addr = "127.0.0.1:3901"
rpc_secret = "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f"

[s3_api]
s3_region = "garage"
api_bind_addr = "0.0.0.0:3900"
root_domain = ".s3.garage"

[s3_web]
bind_addr = "0.0.0.0:3902"
root_domain = ".web.garage"

[admin]
api_bind_addr = "0.0.0.0:3903"
admin_token = "garageadmintoken123456"

Initializing Garage S3 Locally

After starting the container, initialize the layout and credentials:

# 1. Assign layout to the local node
NODE_ID=$(docker exec cloud-cache-garage /garage status | awk '/Node/ {print $2}')
docker exec -ti cloud-cache-garage /garage layout assign -z dc1 -c 10G $NODE_ID
docker exec -ti cloud-cache-garage /garage layout apply --version 1

# 2. Create access key and bucket
docker exec -ti cloud-cache-garage /garage key create ci-cache-key
docker exec -ti cloud-cache-garage /garage bucket create ci-cache
docker exec -ti cloud-cache-garage /garage bucket allow ci-cache --read --write --key ci-cache-key

Notes

• Checksums: for this provider, the action sends request checksums only when S3 requires them, because many S3-compatible services reject the CRC checksums recent AWS SDKs send by default. Nothing to configure.
• Expiry: the action never deletes caches. Add a lifecycle rule that expires objects after 30–60 days so old caches do not accumulate.