Dev infra: fully-local generation and gallery for the dev stack

Closes the two gaps found smoke-testing the local stack: an empty gallery
(zero media_files rows, media URLs hardcoded to the remote r2.dev bucket)
and generation flows that could not complete (omni_gen calls providers
synchronously in the handler, so dummy keys meant a 500 after billing;
the webapp blocked submits earlier anyway because signup grants no wallet).

Backend (all development-only):
- CDN_BASE_URL env override in cdn_link.rs (host-only, validated loudly;
  precedence over the compiled-in hosts for every domain/env) and a
  dev-gated actix-files mount serving LOCAL_MEDIA_ROOT/media at /media,
  mirroring the public bucket layout - media URLs in API responses resolve
  against the backend itself
- DEV_FAKE_GENERATION flag on ServerState (easyenv bool, forced off outside
  Development): the omni_gen image/video pipelines skip the external
  provider call and return a synthetic Fal enqueue (external id fake_<uuid>),
  so billing, prompt insert, and the pending generic_inference_jobs row all
  stay real
- dev_fake_generation_resolver_thread: polls pending fake_ jobs (new
  runtime-query list_pending_fake_generation_jobs in mysql_queries) and
  after DEV_FAKE_GENERATION_RESOLVE_SECS (default 6s) marks them
  complete_success with a media_files row backed by a test_data asset
  copied under LOCAL_MEDIA_ROOT (mp4 results get -thumb.jpg/-thumb.gif
  siblings since the thumbnail job never runs locally), or complete_failure
  via the existing simulate_artcraft_failure prompt triggers (category word
  supported, e.g. "simulate_artcraft_failure face_not_detected")

Bootstrap/seeds (PS + bash kept in lockstep):
- seed_demo_user seeds an artcraft wallet (DEMO_CREDITS, default 100k
  banked + ledger rows; idempotent top-up) - signup creates no wallet and
  the webapp gates generate on balance, which is why submits went nowhere
- seed_demo_user also seeds 6 gallery images (test_data jpgs) with objects
  under .devstack/media in the exact bucket fan-out layout
- fixed the already-exists detection in seed_demo_user: create_account
  returns error_type, not error_code, so re-runs died on UsernameTaken
- generated secrets env gains DEV_FAKE_GENERATION/CDN_BASE_URL/
  LOCAL_MEDIA_ROOT (appended to existing files on bootstrap re-runs);
  stale "providers are only contacted by workers" comments corrected
- doctor checks wallet balance, media rows, and the new secrets keys

Docs: dev_setup_local_stack.md documents the new flags, corrects the
sync-provider-call and upload-500 claims, updates the seams table and
verification checklist; storyteller_web CLAUDE.md video model examples
fixed (seedance_2p0_fast_g -> seedance_2p0_fast).

Verified E2E on this machine: gallery lists 8 items on localhost URLs;
image and video submits go pending ~6s then complete with locally-served
results (mp4 + gif preview 200); failure trigger yields complete_failure
with category face_not_detected; credits genuinely deducted (100000 ->
99932); doctor 0 failures / 0 warnings.
This commit is contained in:
Tanish Manku committed 2026-07-29 20:48:31 -07:00
1 parent f23a6df80b
commit 335015d9e2
24 files changed
+840 -37

No files matched your search

@@ -323,13 +323,13 @@ function Seed-RolesAndBadges {
function Write-SecretsEnvIfMissing {
if (Test-Path $SecretsEnvFile) {
Write-Log "Already exists: $SecretsEnvFile (leaving it untouched)."
Add-DevStackKeysToExistingSecretsEnv
return
}
# Every var below is read with get_env_string_required at server startup but
# is NOT supplied by the checked-in config files. None is contacted at boot -
# placeholders are enough to run the server. Providers are only contacted by
# job worker binaries, which local dev does not run.
# placeholders are enough to run the server.
#
# NOTE: config loading is FIRST-VALUE-WINS (dotenv never overrides a set
# key), so values here cannot override storyteller-web.development.env.
@@ -358,8 +358,11 @@ W2L_PUBLIC_DOWNLOAD_BUCKET_NAME=dummy-local-dev-public
# Email (Resend)
RESEND_API_KEY=dummy-local-dev
# Generation providers (only job workers ever contact these; workers are not
# run in local dev, so enqueued jobs simply stay pending)
# Generation providers. NB: the omni_gen endpoints (webapp image/video
# generation) contact providers SYNCHRONOUSLY in the request handler, so with
# dummy keys a real submit would 500 - DEV_FAKE_GENERATION below avoids that.
# The legacy polling providers are only contacted by worker binaries, which
# local dev does not run.
FAL_API_KEY=dummy-local-dev
GMICLOUD_API_KEY=dummy-local-dev
GROK_API_KEY=dummy-local-dev
@@ -376,11 +379,42 @@ STRIPE_FAKEYOU_ACCOUNT_ID=acct_dummylocaldev
STRIPE_ARTCRAFT_ACCOUNT_ID=acct_dummylocaldev
STRIPE_ARTCRAFT_SECRET_KEY=sk_test_dummylocaldev
STRIPE_ARTCRAFT_SECRET_WEBHOOK_KEY=whsec_dummylocaldev
$(Get-DevStackKeysBlock)
"@
Set-Content -Path $SecretsEnvFile -Value $content -Encoding ascii
Write-Log "Wrote $SecretsEnvFile"
}
# The fully-local media + fake-generation settings (development-only; the
# server hard-ignores DEV_FAKE_GENERATION outside SERVER_ENVIRONMENT=Development).
function Get-DevStackKeysBlock {
$mediaRootForward = (Join-Path $DevStackDir "media") -replace '\\', '/'
return @"
# Fully-local dev stack (see _docs/dev_setup_local_stack.md):
# - DEV_FAKE_GENERATION: omni_gen image/video submits skip real providers and
# a resolver thread completes them a few seconds later with a placeholder
# result (or fails them when the prompt contains simulate_artcraft_failure).
# - CDN_BASE_URL: media URLs in API responses point at this server instead of
# the shared r2.dev bucket. Host only - no path suffix.
# - LOCAL_MEDIA_ROOT: directory served at /media (mirrors the bucket layout);
# seeded gallery media and fake generation results live here.
DEV_FAKE_GENERATION=true
CDN_BASE_URL=http://localhost:12345
LOCAL_MEDIA_ROOT=$mediaRootForward
"@
}
# Bootstrap re-runs on a stack whose secrets file predates the fully-local
# media/fake-generation support: append the new keys (first-value-wins never
# lets these override anything the user already set).
function Add-DevStackKeysToExistingSecretsEnv {
$existing = Get-Content $SecretsEnvFile -Raw
if ($existing -match '(?m)^\s*LOCAL_MEDIA_ROOT=') { return }
Add-Content -Path $SecretsEnvFile -Value "`r`n$(Get-DevStackKeysBlock)" -Encoding ascii
Write-Log "Appended fully-local dev stack keys (DEV_FAKE_GENERATION, CDN_BASE_URL, LOCAL_MEDIA_ROOT) to $SecretsEnvFile"
}
function Build-Backend {
if ($SkipRustBuild) {
Write-Log "Skipped (-SkipRustBuild)."