New releases took up to ~30 min to reach \`GET
/v1/craft_apps/release_info\` (ArtCraft Launcher 0.2.0 needed 17–29
min), pods disagreed with each other, and every pod polled GitHub on its
own, which uses up the unauthenticated quota (60/h per IP, and 304s
count) and risks backing off for up to 4 h.
The pods now share their results through Redis. There's no lock:
- **One Redis hash per app** (\`craftApps:releases:app:<key>\`, 7 d
TTL). It holds the latest release, the ETag and \`fetched_ms\`. Before
asking GitHub about an app, a pod reads its entry. If it's fresher than
\`interval − 30 s\` (a clock-skew margin), the pod takes it from Redis
and doesn't call GitHub.
- **Last write wins, guarded by timestamp.** A Lua script never lets an
entry go back in time. A 304 moves only \`fetched_ms\`, and only while
the stored ETag still matches.
- **A soft per-app claim** (60 s, no owner, it just lapses), so a fleet
that ticks every 30 s rarely fetches the same app twice.
- **Backoff shared by the whole fleet**
(\`craftApps:releases:backoff\`), so a rate limit slows everyone down.
Per-app failure retry is 60 s, doubling.
- **The endpoint is unchanged.** It still serves each pod's in-memory
copy. If Redis is down, a pod falls back to polling on its own and warns
at most every 10 minutes.
**The GitHub token is optional.** With
\`CRAFT_APPS_RELEASE_GITHUB_TOKEN\` set, the default interval is 120 s
(min 60). Without it (unset or blank), calls go out unauthenticated with
no Authorization header, at 1800 s (min 300), and startup and polling
are never blocked. \`CRAFT_APPS_RELEASE_COORDINATION_ENABLED=false\`
switches back to the old per-pod polling.
**Tests:** \`cargo test -p storyteller-web --bin storyteller-web craft\`
(49 pass) and \`cargo test -p redis_schema craft\` (4 pass). The Redis
tests passed against a local redis-server with
\`CRAFT_APPS_TEST_REDIS_URL=… -- --include-ignored --skip live_round\`
(51 pass).
**Follow-ups after review:**
- **The poller has its own Redis pool** (2 connections; connects within
3 s; every read and write, including the pool's `PING` check, times out
after 5 s). Before this, it shared the handlers' pool, which has no read
timeout, so a Redis that stopped answering could leave the poller
holding every connection while handlers waited 30 s for one. A test
against a fake Redis that goes silent after the handshake fails without
the timeouts and passes with them.
- **When Redis won't run scripts**, writes fall back to
`WATCH`/`MULTI`/`EXEC`: read and decode the guarding fields, decide, and
write in a transaction that Redis drops if another pod wrote in between.
This switch only happens when Redis refuses the script itself. Being
unreachable, busy, out of memory or read-only doesn't trigger it. It was
tested against a real Redis user denied `@scripting`.
- **New pods load from Redis at startup**, before the startup jitter, so
they serve the fleet's releases right away without asking GitHub.
- **`Cache-Control: no-store` until every app has been fetched**
(`last_fetched` set), so Cloudflare never caches a cold pod's "not
fetched yet" answer. The response body is unchanged.
**Tests:** `cargo test -p storyteller-web --bin storyteller-web craft --
--include-ignored --skip live_round` against a local Redis 8.6: 63 pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Brandon ThomasandClaude Opus 5.5
