Dual Caching (S3 + GitHub Actions Cache)
Dual Caching allows your GitHub Actions workflows to save and restore cache bundles across both remote S3-compatible storage and GitHub's native Actions Cache service simultaneously.
Why Dual Caching?
In modern CI/CD setups, organizations often run a hybrid infrastructure:
- Standard linting, light tests, and PR checks run on GitHub-hosted runners (
ubuntu-latest). - Heavy compilation, Docker builds, GPU workloads, and integration tests run on self-hosted runners (Kubernetes, AWS EC2, or on-premise bare-metal).
This creates a common challenge:
- Runner Isolation: Self-hosted or remote runners may have high latency, bandwidth limits, or restricted access to GitHub's internal cache service, but have direct, high-speed access to an S3-compatible bucket (or local Garage / SeaweedFS / MinIO / Cloudflare R2).
- Quota Limits: GitHub enforces a strict 10 GB per repository cache limit. Once exceeded, older caches are evicted, causing unexpected slowdowns.
- Redundancy & Availability: If GitHub's cache service experiences transient 503 errors or an outage, having S3 as an active peer ensures your builds never suffer cold starts.
With Dual Caching enabled, the exact same cache keys are populated in both systems, and jobs can seamlessly read from whichever source is optimal.
Architecture & Flow
Restore Flow
mermaid
flowchart TD
Start[Restore Step Starts] --> CheckDual{dual-cache: true?}
CheckDual -- No --> S3Only[Standard S3 Restore]
CheckDual -- Yes --> Priority{restore-priority}
Priority -- s3-first --> S3Try[1. Query S3 Storage]
S3Try -- S3 Hit --> RestoreS3[Restore from S3\ncache-hit-source: s3]
S3Try -- S3 Miss/Fail --> GHTry[2. Fallback to GitHub Cache]
GHTry -- GH Hit --> RestoreGH[Restore from GitHub Cache\ncache-hit-source: github]
GHTry -- GH Miss --> Miss[Cache Miss across both tiers\ncache-hit-source: none]
Priority -- github-first --> GHTryFirst[1. Query GitHub Cache]
GHTryFirst -- GH Hit --> RestoreGHFirst[Restore from GitHub Cache\ncache-hit-source: github]
GHTryFirst -- GH Miss/Fail --> S3TrySecond[2. Fallback to S3 Storage]
S3TrySecond -- S3 Hit --> RestoreS3Second[Restore from S3\ncache-hit-source: s3]
S3TrySecond -- S3 Miss --> MissSave Flow (Post-step)
When dual-cache: true is enabled, the post-run step evaluates both tiers according to dual-cache-strategy:
| Strategy | Behavior |
|---|---|
backfill (default) | Before uploading to a tier that did not serve the exact key, checks whether that tier already has it (an S3 HEAD request or a GitHub lookup) and uploads only if it is missing. |
skip-on-hit | If an exact hit occurred on either tier during restore, saving is skipped on both. |
independent was removed in v1.1; it now behaves as backfill and logs a warning.
Real-World Use Cases & Examples
Use Case 1: Lightweight Runners for Dependencies $\to$ Remote Cloud VM for Heavy Builds
A common architectural pattern is separating concerns across runner tiers to minimize cloud compute costs:
- Lightweight GitHub-hosted runner (
ubuntu-latest): Runs quick tasks likenpm cito assemblenode_modulesor download package dependencies. - Heavyweight Remote Cloud runner (
[self-hosted, aws-heavy]): An EC2 or GCP machine with high CPU/GPU/RAM dedicated to compiling native artifacts, building Docker images, or executing end-to-end integration tests.
With Dual Caching:
- The GitHub-hosted runner uses
dual-cache: truewithrestore-priority: github-firstanddual-cache-strategy: backfill. It benefits from GitHub's internal runner cache, but automatically synchronizes the populatednode_modulesdirectly into your AWS S3 or GCP bucket. - The remote cloud machine then restores
node_modulesdirectly from S3 at line-rate internal VPC speeds (restore-priority: s3-first) without network bottlenecking or GitHub cache quota contention.
yaml
name: Full Pipeline (Lightweight Prep to Heavy Cloud Build)
on: [push, pull_request]
jobs:
# Job 1: Lightweight GitHub-hosted runner resolves dependencies
prepare-dependencies:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Cache node_modules (Dual Mode: GitHub + S3 Sync)
id: cache-deps
uses: xSAVIKx/cloud-cache-action@v1
with:
bucket: my-company-ci-cache
endpoint: https://s3.us-east-1.amazonaws.com # or GCS / Cloudflare R2
access-key: ${{ secrets.AWS_ACCESS_KEY_ID }}
secret-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
dual-cache: true
restore-priority: github-first # Quickest on GitHub-hosted runner
dual-cache-strategy: backfill # Ensures S3 gets populated even if GitHub Cache hit
key: ${{ runner.os }}-node-modules-${{ hashFiles('**/package-lock.json') }}
path: node_modules
- name: Install dependencies on miss
if: steps.cache-deps.outputs.cache-hit != 'true'
run: npm ci
# Job 2: Heavyweight remote AWS/GCP runner builds Docker / Native binaries
build-product:
needs: prepare-dependencies
runs-on: [self-hosted, aws-c6i-metal] # Heavy remote VM located inside AWS VPC
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
# Instantly pulls node_modules directly from local AWS S3 bucket over internal VPC
- name: Restore node_modules from S3
uses: xSAVIKx/cloud-cache-action@v1
with:
bucket: my-company-ci-cache
endpoint: https://s3.us-east-1.amazonaws.com
access-key: ${{ secrets.AWS_ACCESS_KEY_ID }}
secret-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
dual-cache: true
restore-priority: s3-first # Direct VPC speed, no GitHub egress lag
read-only: true # Dependencies were already saved by Job 1
key: Linux-node-modules-${{ hashFiles('**/package-lock.json') }}
path: node_modules
# Docker layer cache can also be persisted to S3
- name: Cache Docker Buildx layers
uses: xSAVIKx/cloud-cache-action@v1
with:
bucket: my-company-ci-cache
access-key: ${{ secrets.AWS_ACCESS_KEY_ID }}
secret-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
key: docker-layers-${{ github.sha }}
restore-keys: |
docker-layers-
path: /tmp/.buildx-cache
- name: Build Native / Docker Product
run: |
docker buildx build \
--cache-from=type=local,src=/tmp/.buildx-cache \
--cache-to=type=local,dest=/tmp/.buildx-cache-new,mode=max \
-t my-app:latest .Use Case 2: Quota Spillover & High-Availability Redundancy
Avoid cold builds when GitHub's 10GB per-repo cache limit evicts keys, or during GitHub service disruptions:
yaml
- name: Cache with S3 backup
id: cache-step
uses: xSAVIKx/cloud-cache-action@v1
with:
bucket: my-overflow-s3-bucket
access-key: ${{ secrets.AWS_ACCESS_KEY_ID }}
secret-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
dual-cache: true
restore-priority: github-first # Use GitHub cache while available
dual-cache-strategy: backfill # Keep S3 continuously primed as cold-cache safety net
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*') }}
restore-keys: |
${{ runner.os }}-gradle-
path: ~/.gradle/caches
- name: Inspect Cache Hit Source
run: |
echo "Cache Hit: ${{ steps.cache-step.outputs.cache-hit }}"
echo "Serviced by: ${{ steps.cache-step.outputs.cache-hit-source }}" # 'github', 's3', or 'none'Use Case 3: Zero-Downtime Migration from GitHub Cache to Cloud Storage
If your team is migrating from actions/cache to Cloudflare R2 or AWS S3:
- Enable
dual-cache: truewithrestore-priority: github-first. - Existing builds will continue restoring instantly from GitHub Actions Cache.
- In the background,
backfillsaves every cache bundle to your S3 bucket. - Once your S3 bucket is warmed up, switch
restore-priority: s3-firstor turn off dual-cache.
Dual-Cache Configuration Reference
Inputs
| Input | Type | Default | Description |
|---|---|---|---|
dual-cache | boolean | false | Enables simultaneous caching across S3-compatible storage and GitHub Actions Cache. |
restore-priority | string | s3-first | Order to check caches during restore: s3-first or github-first. |
dual-cache-strategy | string | backfill | Save synchronization mode: backfill or skip-on-hit. |
dual-cache-strict | boolean | false | When false (default), a tier error is a warning and the other tier continues. When true, an error from either tier fails the step, during restore and during save. |
What dual-cache-strict can see
Strict mode catches every S3 error. For the GitHub Actions Cache tier it only sees the failures @actions/cache reports: the library logs and swallows most cache service errors itself. When it does not save a key, because another job is creating the same entry, the cache mode forbids writes, or the service failed, the step logs the library's message, does not list github in cache-saved-sources, and does not fail. When none of path matches anything on disk, both tiers report the save as skipped instead of an error, so strict mode no longer fails the step for that case.
Outputs
| Output | Values | Description |
|---|---|---|
cache-hit-source | s3 | github | none | Identifies which storage tier provided the restored cache bundle. |
cache-saved-sources | s3,github | s3 | github | none | Comma-separated list of storage tiers that successfully stored the cache bundle. |
Live CI Dogfooding & Verification
The dual-cache backfill and skip-on-hit strategies are exercised continuously within the repository's test workflow: .github/workflows/test.yml.
.github/workflows/test.yml (Click to view full CI test workflow)
yaml
name: CI Tests
on:
push:
branches: [main]
pull_request:
branches: [main]
permissions:
contents: read
env:
REQUIRE_S3: '1'
jobs:
checks:
name: Format, lint, types, dist and docs
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: npm
- run: npm ci
- run: npm run format:check
- run: npm run lint
- name: Lint workflows with actionlint
run: docker run --rm -v "$PWD:/repo" -w /repo rhysd/actionlint:1.7.12 -color
- run: npm run typecheck
- run: npm run build
- name: Verify dist is clean and tracked
run: |
if [ -n "$(git status --porcelain dist)" ]; then
echo "::error::Uncommitted dist changes detected. Run npm run build and commit."
git status --porcelain dist
exit 1
fi
- run: npm run docs:build
unit:
name: Jest (${{ matrix.os }})
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: npm
- run: npm ci
- name: Start SeaweedFS
run: node tests/ci/startSeaweedfs.ts
- name: Run Jest (unit, contract and integration)
run: npm test
provider-compat:
name: S3 compatibility (${{ matrix.server }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- server: seaweedfs
endpoint: http://127.0.0.1:8333
region: us-east-1
access-key: cloudcacheci
secret-key: cloudcachecisecret
- server: garage
endpoint: http://127.0.0.1:3900
region: garage
access-key: GK0123456789abcdef01234567
secret-key: 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
- server: minio
endpoint: http://127.0.0.1:9000
region: us-east-1
access-key: cloudcacheci
secret-key: cloudcachecisecret
# RustFS serves S3 on 9000 like MinIO, so docker-compose.test.yml publishes it on 9010.
- server: rustfs
endpoint: http://127.0.0.1:9010
region: us-east-1
access-key: cloudcacheci
secret-key: cloudcachecisecret
env:
TEST_S3_ENDPOINT: ${{ matrix.endpoint }}
TEST_S3_REGION: ${{ matrix.region }}
TEST_S3_ACCESS_KEY: ${{ matrix.access-key }}
TEST_S3_SECRET_KEY: ${{ matrix.secret-key }}
TEST_S3_PROVIDER: ${{ matrix.server }}
KEY: compat-${{ matrix.server }}-${{ github.run_id }}-${{ github.run_attempt }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: npm
- run: npm ci
- name: Start ${{ matrix.server }}
run: docker compose -f docker-compose.test.yml up -d ${{ matrix.server }}
- name: Bootstrap Garage
if: matrix.server == 'garage'
run: bash tests/ci/start-garage.sh
- name: Wait for S3 and create the bucket
run: |
for _ in $(seq 1 60); do
if node tests/support/s3Server.ts; then exit 0; fi
sleep 2
done
docker compose -f docker-compose.test.yml logs ${{ matrix.server }}
exit 1
- name: Integration tests
run: npm run test:integration
- name: Build the fixture tree
id: fixture
run: node tests/support/fixtureTree.ts build "$RUNNER_TEMP/fixture.json" --large-bytes 33554432
- name: Save (save-only)
uses: ./save
with:
key: ${{ env.KEY }}
path: ${{ steps.fixture.outputs.paths }}
bucket: cloud-cache-test
endpoint: ${{ matrix.endpoint }}
region: ${{ matrix.region }}
provider: ${{ matrix.server }}
access-key: ${{ matrix.access-key }}
secret-key: ${{ matrix.secret-key }}
- name: Remove the fixture tree
run: node tests/support/fixtureTree.ts clear
- name: Restore (restore-only)
id: restore
uses: ./restore
with:
key: ${{ env.KEY }}
path: ${{ steps.fixture.outputs.paths }}
fail-on-cache-miss: true
bucket: cloud-cache-test
endpoint: ${{ matrix.endpoint }}
region: ${{ matrix.region }}
provider: ${{ matrix.server }}
access-key: ${{ matrix.access-key }}
secret-key: ${{ matrix.secret-key }}
- name: Verify the restored tree
run: |
test "${{ steps.restore.outputs.cache-hit }}" = "true"
node tests/support/fixtureTree.ts verify "$RUNNER_TEMP/fixture.json"
cli:
name: CLI bundle against RustFS
runs-on: ubuntu-latest
env:
TEST_S3_ENDPOINT: http://127.0.0.1:9010
TEST_S3_REGION: us-east-1
TEST_S3_ACCESS_KEY: cloudcacheci
TEST_S3_SECRET_KEY: cloudcachecisecret
TEST_S3_PROVIDER: rustfs
TEST_CLI: '1'
REQUIRE_S3: '1'
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: npm
- run: npm ci
- run: npm run build:cli
- name: Start RustFS
run: docker compose -f docker-compose.test.yml up -d rustfs
- name: Wait for S3 and create the bucket
run: |
for _ in $(seq 1 60); do
if node tests/support/s3Server.ts; then exit 0; fi
sleep 2
done
docker compose -f docker-compose.test.yml logs rustfs
exit 1
- name: Run the CLI integration test
run: npm test -- tests/integration/cli.test.ts --testTimeout=60000
roundtrip:
name: Round trip (${{ matrix.os }}, streaming ${{ matrix.streaming }})
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
streaming: ['false', 'true']
runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash
env:
AWS_ENDPOINT_URL: http://127.0.0.1:8333
AWS_S3_BUCKET: cloud-cache-test
AWS_ACCESS_KEY_ID: cloudcacheci
AWS_SECRET_ACCESS_KEY: cloudcachecisecret
KEY: rt-${{ matrix.os }}-${{ github.run_id }}-${{ github.run_attempt }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: npm
- run: npm ci --omit=dev --no-audit --no-fund
- run: node tests/ci/startSeaweedfs.ts
- name: Build the fixture tree
id: fixture
run: node tests/support/fixtureTree.ts build "$RUNNER_TEMP/fixture.json" --large-bytes 33554432
- name: Save (save-only)
uses: ./save
with:
bucket: cloud-cache-test
key: ${{ env.KEY }}-v1
path: ${{ steps.fixture.outputs.paths }}
streaming: ${{ matrix.streaming }}
metrics-file: metrics.jsonl
- name: Remove the fixture tree
run: node tests/support/fixtureTree.ts clear
- name: Restore the exact key
id: exact
uses: ./restore
with:
bucket: cloud-cache-test
key: ${{ env.KEY }}-v1
path: ${{ steps.fixture.outputs.paths }}
fail-on-cache-miss: true
streaming: ${{ matrix.streaming }}
metrics-file: metrics.jsonl
- name: Verify the restored tree
run: |
test "${{ steps.exact.outputs.cache-hit }}" = "true"
node tests/support/fixtureTree.ts verify "$RUNNER_TEMP/fixture.json"
- name: Verify the metrics file has one line per step
run: |
cat metrics.jsonl
test "$(wc -l < metrics.jsonl)" -eq 2
- name: Look up by primary-key prefix
id: prefix
uses: ./restore
with:
bucket: cloud-cache-test
key: ${{ env.KEY }}-v
path: ${{ steps.fixture.outputs.paths }}
lookup-only: true
- name: Look up by restore-keys
id: restore-keys
uses: ./restore
with:
bucket: cloud-cache-test
key: ${{ env.KEY }}-other
restore-keys: ${{ env.KEY }}-v
path: ${{ steps.fixture.outputs.paths }}
lookup-only: true
- name: Look up with a different path list
id: mismatch
uses: ./restore
with:
bucket: cloud-cache-test
key: ${{ env.KEY }}-v1
path: |
${{ steps.fixture.outputs.paths }}
not-in-the-original-list
lookup-only: true
- name: Verify the lookups
run: |
test "${{ steps.prefix.outputs.cache-hit }}" = "false"
test "${{ steps.prefix.outputs.cache-matched-key }}" = "$KEY-v1"
test "${{ steps.restore-keys.outputs.cache-hit }}" = "false"
test "${{ steps.restore-keys.outputs.cache-matched-key }}" = "$KEY-v1"
test "${{ steps.mismatch.outputs.cache-hit }}" = "false"
test -z "${{ steps.mismatch.outputs.cache-matched-key }}"
main-action:
name: Main action post step (${{ matrix.os }})
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash
env:
AWS_ENDPOINT_URL: http://127.0.0.1:8333
AWS_S3_BUCKET: cloud-cache-test
AWS_ACCESS_KEY_ID: cloudcacheci
AWS_SECRET_ACCESS_KEY: cloudcachecisecret
KEY: main-${{ matrix.os }}-${{ github.run_id }}-${{ github.run_attempt }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: npm
- run: npm ci --omit=dev --no-audit --no-fund
- run: node tests/ci/startSeaweedfs.ts
- name: Create a payload
run: mkdir -p payload && echo "${{ github.run_id }}" > payload/run.txt
- name: After every post step, check that the cache was saved
uses: ./.github/actions/assert-object
with:
key: ${{ env.KEY }}
- name: Cache the payload (restore now, save in the post step)
id: cache
uses: ./
with:
bucket: cloud-cache-test
key: ${{ env.KEY }}
path: payload
- name: Expect a miss on the first attempt
run: test "${{ steps.cache.outputs.cache-hit }}" = "false"
prune:
name: Prune sub-action
runs-on: ubuntu-latest
env:
AWS_ENDPOINT_URL: http://127.0.0.1:8333
AWS_S3_BUCKET: cloud-cache-test
AWS_ACCESS_KEY_ID: cloudcacheci
AWS_SECRET_ACCESS_KEY: cloudcachecisecret
KEY: prune-${{ github.run_id }}-${{ github.run_attempt }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: npm
- run: npm ci --omit=dev --no-audit --no-fund
- run: node tests/ci/startSeaweedfs.ts
- name: Create two payloads
run: |
mkdir -p payload-a payload-b
echo "a-${{ github.run_id }}" > payload-a/data.txt
echo "b-${{ github.run_id }}" > payload-b/data.txt
- name: Save the first key
uses: ./save
with:
bucket: cloud-cache-test
key: ${{ env.KEY }}-a
path: payload-a
- name: Save the second key
uses: ./save
with:
bucket: cloud-cache-test
key: ${{ env.KEY }}-b
path: payload-b
- name: Dry-run prune of caches older than 1 day
id: prune
uses: ./prune
with:
bucket: cloud-cache-test
older-than-days: 1
dry-run: true
- name: Expect nothing pruned, since the objects are brand new
run: test "${{ steps.prune.outputs.pruned-count }}" = "0"
- name: Confirm both objects still exist
run: |
count=$(node tests/ci/s3Objects.ts count "$GITHUB_REPOSITORY/" "/$KEY-")
test "$count" -eq 2
inspect:
name: Inspect sub-action
runs-on: ubuntu-latest
env:
AWS_ENDPOINT_URL: http://127.0.0.1:8333
AWS_S3_BUCKET: cloud-cache-test
AWS_ACCESS_KEY_ID: cloudcacheci
AWS_SECRET_ACCESS_KEY: cloudcachecisecret
KEY: inspect-${{ github.run_id }}-${{ github.run_attempt }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: npm
- run: npm ci --omit=dev --no-audit --no-fund
- run: node tests/ci/startSeaweedfs.ts
- name: Create two payloads
run: |
mkdir -p payload other-dir
echo "payload-${{ github.run_id }}" > payload/data.txt
echo "other-${{ github.run_id }}" > other-dir/data.txt
- name: Save the cache
uses: ./save
with:
bucket: cloud-cache-test
key: ${{ env.KEY }}
path: payload
- name: Inspect the same lookup
id: inspect
uses: ./inspect
with:
bucket: cloud-cache-test
key: ${{ env.KEY }}
path: payload
- name: Expect the lookup to hit the object that was just saved
run: |
test "${{ steps.inspect.outputs.would-hit }}" = "true"
test "${{ steps.inspect.outputs.would-match-key }}" = "$KEY"
test -n "${{ steps.inspect.outputs.would-match-object }}"
- name: Smoke-test the inspect bundle itself
env:
INPUT_BUCKET: cloud-cache-test
INPUT_KEY: ${{ env.KEY }}
INPUT_PATH: payload
GITHUB_OUTPUT: ${{ runner.temp }}/inspect-bundle-output.txt
run: |
: > "$GITHUB_OUTPUT"
node dist/inspect/index.js
cat "$GITHUB_OUTPUT"
# core.setOutput writes "<name><<<delimiter>\n<value>\n<delimiter>".
test "$(awk '/^would-hit<</ { getline; print; exit }' "$GITHUB_OUTPUT")" = "true"
- name: Inspect the same key with different paths
id: inspect-miss
uses: ./inspect
with:
bucket: cloud-cache-test
key: ${{ env.KEY }}
path: other-dir
- name: Expect a miss, since the paths hash to another version
run: |
test "${{ steps.inspect-miss.outputs.would-hit }}" = "false"
test -z "${{ steps.inspect-miss.outputs.would-match-object }}"
dual-cache:
name: Dual-cache with the live GitHub Actions Cache
runs-on: ubuntu-latest
env:
AWS_ENDPOINT_URL: http://127.0.0.1:8333
AWS_S3_BUCKET: cloud-cache-test
AWS_ACCESS_KEY_ID: cloudcacheci
AWS_SECRET_ACCESS_KEY: cloudcachecisecret
KEY: dual-${{ github.run_id }}-${{ github.run_attempt }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: npm
- run: npm ci --omit=dev --no-audit --no-fund
- run: node tests/ci/startSeaweedfs.ts
- name: Create a payload
run: mkdir -p payload && echo "${{ github.run_id }}" > payload/run.txt
- name: After the dual-cache post step, check that S3 received the object
uses: ./.github/actions/assert-object
with:
key: ${{ env.KEY }}
- name: Cache the payload in both tiers (restore now, post step saves to S3 and GitHub)
id: cache
uses: ./
with:
bucket: cloud-cache-test
key: ${{ env.KEY }}
path: payload
dual-cache: true
- name: Expect a miss in both tiers on the first attempt
run: test "${{ steps.cache.outputs.cache-hit }}" = "false"
strict-failure:
name: Strict dual-cache save failure
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: npm
- run: npm ci --omit=dev --no-audit --no-fund
- run: node tests/ci/startSeaweedfs.ts
- name: Create a payload
run: mkdir -p payload && echo "${{ github.run_id }}" > payload/run.txt
- name: Save with a wrong secret in strict dual-cache mode (must fail)
id: strict-save
continue-on-error: true
uses: ./save
with:
key: strict-${{ github.run_id }}-${{ github.run_attempt }}
path: payload
bucket: cloud-cache-test
endpoint: http://127.0.0.1:8333
access-key: cloudcacheci
secret-key: wrong-secret
dual-cache: true
dual-cache-strict: true
- name: Expect the strict save step to have failed
run: test "${{ steps.strict-save.outcome }}" = "failure"
cross-os-seed:
name: Cross-OS seed (Linux)
runs-on: ubuntu-latest
env:
AWS_ENDPOINT_URL: http://127.0.0.1:8333
AWS_S3_BUCKET: cloud-cache-test
AWS_ACCESS_KEY_ID: cloudcacheci
AWS_SECRET_ACCESS_KEY: cloudcachecisecret
KEY: xos-${{ github.run_id }}-${{ github.run_attempt }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: npm
- run: npm ci --omit=dev --no-audit --no-fund
- run: node tests/ci/startSeaweedfs.ts
- name: Build the portable fixture tree
id: fixture
run: node tests/support/fixtureTree.ts build "$RUNNER_TEMP/seed/fixture.json" --portable --large-bytes 8388608
- name: Save with enableCrossOsArchive
uses: ./save
with:
bucket: cloud-cache-test
key: ${{ env.KEY }}
path: ${{ steps.fixture.outputs.paths }}
enableCrossOsArchive: true
- name: Export the saved objects
run: node tests/ci/s3Objects.ts export "$RUNNER_TEMP/seed/objects" "$GITHUB_REPOSITORY/" "/$KEY/"
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: cross-os-seed
path: ${{ runner.temp }}/seed
if-no-files-found: error
cross-os-restore:
name: Cross-OS restore (${{ matrix.os }})
needs: cross-os-seed
strategy:
fail-fast: false
matrix:
include:
- os: macos-latest
hit-without-cross-os: 'true'
- os: windows-latest
hit-without-cross-os: 'false'
runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash
env:
AWS_ENDPOINT_URL: http://127.0.0.1:8333
AWS_S3_BUCKET: cloud-cache-test
AWS_ACCESS_KEY_ID: cloudcacheci
AWS_SECRET_ACCESS_KEY: cloudcachecisecret
KEY: xos-${{ github.run_id }}-${{ github.run_attempt }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
cache: npm
- run: npm ci --omit=dev --no-audit --no-fund
- name: Require zstd, which the Linux seed used
run: zstd --version
- run: node tests/ci/startSeaweedfs.ts
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: cross-os-seed
path: ${{ runner.temp }}/seed
- name: Import the seeded objects
run: node tests/ci/s3Objects.ts import "$RUNNER_TEMP/seed/objects"
- name: Read the seeded paths
id: seed
run: node tests/support/fixtureTree.ts paths "$RUNNER_TEMP/seed/fixture.json"
- name: Look up without enableCrossOsArchive
id: plain
uses: ./restore
with:
bucket: cloud-cache-test
key: ${{ env.KEY }}
path: ${{ steps.seed.outputs.paths }}
lookup-only: true
- name: Restore with enableCrossOsArchive
id: cross
uses: ./restore
with:
bucket: cloud-cache-test
key: ${{ env.KEY }}
path: ${{ steps.seed.outputs.paths }}
enableCrossOsArchive: true
fail-on-cache-miss: true
- name: Verify
run: |
test "${{ steps.plain.outputs.cache-hit }}" = "${{ matrix.hit-without-cross-os }}"
test "${{ steps.cross.outputs.cache-hit }}" = "true"
node tests/support/fixtureTree.ts verify "$RUNNER_TEMP/seed/fixture.json"