Commit Graph
4165 Commits
Author SHA1 Message Date
Tanish Manku f5b80ea982 Dev infra: local filesystem uploads + fully-local pagescene media
Closes the two remaining fully-local-stack seams found after the last round.

Uploads (previously every upload endpoint returned 500 with dummy S3 creds -
the bucket put happens before the DB insert and rust-s3 is fail-on-err):
- LegacyBucketClient gains an internal backend enum (S3 unchanged | LocalDir)
  and a create_local(root, bucket_name, root_prefix) constructor. All upload/
  download methods route LocalDir to the filesystem; bucket_name() reports the
  configured name. Objects map disk = root + rooted object path via the new
  pub local_object_disk_path (rejects ".." and backslash segments - PMX zips
  feed user-controlled names), unit-tested. The fake-generation resolver now
  uses the same function instead of its private copy so the layouts can't
  drift.
- build_dependencies: when SERVER_ENVIRONMENT=Development and
  LOCAL_MEDIA_ROOT is set, all three bucket clients are constructed in local
  mode (private/gc have zero callers in storyteller-web, so this is purely
  uniform); the server_environment read moved above client construction.
  Uploads land under .devstack/media and are immediately viewable through
  the existing /media mount + CDN_BASE_URL - no S3/R2/MinIO.

Frontend (pagescene was the only surface actually broken against a local
backend; moodboard/library/create-* already consume media_links correctly):
- The vite /__cdn dev proxy assumed https upstreams, so localhost media
  (http) died on a TLS handshake with a 502 - breaking every pagescene
  GLB/splat/animation fetch. fetchAsset no longer proxies localhost at all
  (the dev backend sends CORS for any localhost origin), and the proxy
  itself now picks http for localhost hosts as a backstop.
- GetCdnOrigin() returns the API host when it is localhost (the local
  backend serves media itself; CDN origin == API origin), production CDN
  otherwise. The pagescene adapter's getCdnUrl skips the Cloudflare
  /cdn-cgi/image resize segment on localhost - the local backend has no
  resizer, and dev serves originals anyway.
- vite /v1 fallback proxy now targets http://localhost:12345 instead of
  production, so nothing can silently fall back to prod traffic in dev;
  fixed the stale main.tsx comment claiming setDevelopment() is disabled.

Docs: seams table updated (uploads + vite proxy + GetCdnOrigin closed;
API_TARGETS/SegmentationApi noted as dead code; the earlier moodboard claim
was wrong - it never had a hardcoded host), LOCAL_MEDIA_ROOT now documents
the storage mode, verification checklist gains the upload check.

Verified E2E on this machine: POST /v1/media_files/upload/image (multipart,
real endpoint) returns success, the object lands under .devstack/media at
the canonical fan-out path, lists in the library with media_class=image,
and serves 200 image/jpeg from http://localhost:12345/media/...;
cloud_storage unit tests pass; storyteller-web + webapp dev server boot
clean with "Local filesystem storage mode is ON".
2026-07-29 21:10:42 -07:00
Tanish Manku 335015d9e2 Dev infra: fully-local generation and gallery for the dev stack
Closes the two gaps found smoke-testing the local stack: an empty gallery
(zero media_files rows, media URLs hardcoded to the remote r2.dev bucket)
and generation flows that could not complete (omni_gen calls providers
synchronously in the handler, so dummy keys meant a 500 after billing;
the webapp blocked submits earlier anyway because signup grants no wallet).

Backend (all development-only):
- CDN_BASE_URL env override in cdn_link.rs (host-only, validated loudly;
  precedence over the compiled-in hosts for every domain/env) and a
  dev-gated actix-files mount serving LOCAL_MEDIA_ROOT/media at /media,
  mirroring the public bucket layout - media URLs in API responses resolve
  against the backend itself
- DEV_FAKE_GENERATION flag on ServerState (easyenv bool, forced off outside
  Development): the omni_gen image/video pipelines skip the external
  provider call and return a synthetic Fal enqueue (external id fake_<uuid>),
  so billing, prompt insert, and the pending generic_inference_jobs row all
  stay real
- dev_fake_generation_resolver_thread: polls pending fake_ jobs (new
  runtime-query list_pending_fake_generation_jobs in mysql_queries) and
  after DEV_FAKE_GENERATION_RESOLVE_SECS (default 6s) marks them
  complete_success with a media_files row backed by a test_data asset
  copied under LOCAL_MEDIA_ROOT (mp4 results get -thumb.jpg/-thumb.gif
  siblings since the thumbnail job never runs locally), or complete_failure
  via the existing simulate_artcraft_failure prompt triggers (category word
  supported, e.g. "simulate_artcraft_failure face_not_detected")

Bootstrap/seeds (PS + bash kept in lockstep):
- seed_demo_user seeds an artcraft wallet (DEMO_CREDITS, default 100k
  banked + ledger rows; idempotent top-up) - signup creates no wallet and
  the webapp gates generate on balance, which is why submits went nowhere
- seed_demo_user also seeds 6 gallery images (test_data jpgs) with objects
  under .devstack/media in the exact bucket fan-out layout
- fixed the already-exists detection in seed_demo_user: create_account
  returns error_type, not error_code, so re-runs died on UsernameTaken
- generated secrets env gains DEV_FAKE_GENERATION/CDN_BASE_URL/
  LOCAL_MEDIA_ROOT (appended to existing files on bootstrap re-runs);
  stale "providers are only contacted by workers" comments corrected
- doctor checks wallet balance, media rows, and the new secrets keys

Docs: dev_setup_local_stack.md documents the new flags, corrects the
sync-provider-call and upload-500 claims, updates the seams table and
verification checklist; storyteller_web CLAUDE.md video model examples
fixed (seedance_2p0_fast_g -> seedance_2p0_fast).

Verified E2E on this machine: gallery lists 8 items on localhost URLs;
image and video submits go pending ~6s then complete with locally-served
results (mp4 + gif preview 200); failure trigger yields complete_failure
with category face_not_detected; credits genuinely deducted (100000 ->
99932); doctor 0 failures / 0 warnings.
2026-07-29 20:48:31 -07:00
Brandon Thomas 6ee5518c55 api keys for everyone (#1811) 2026-07-27 14:55:54 -04:00
Brandon Thomas f708402e8a grab billing data for reports (#1810) 2026-07-27 10:03:13 -04:00
Brandon Thomas 0003cad179 seedream5 (#1784) 2026-07-13 11:10:21 -04:00
Brandon Thomas eed865e57c cursor behavior fixes (#1780) 2026-07-12 15:28:34 -04:00
Brandon Thomas 348a603cae folderless endpoint (#1779) 2026-07-12 15:23:58 -04:00
Brandon Thomas a79ee6ad57 misc fixes for storyteller-web (#1777) 2026-07-12 03:21:23 -04:00
Brandon Thomas 2a9e71f5b8 fix pagination cursors (#1776) 2026-07-11 23:48:34 -04:00
Brandon Thomas 7dffc5fb5c list meshes and splats (#1774) 2026-07-12 09:32:35 +07:00
Brandon Thomas 9cc9d4b9cf Fix cover images (#1773) 2026-07-11 15:32:15 -04:00
Brandon Thomas dc5a4f8dd9 backup for cover images when absent 2026-07-10 17:14:13 -04:00
Brandon Thomas 11ddeb05b5 handle audio results (#1772) 2026-07-10 16:58:34 -04:00
Brandon Thomas 73b40d321e fal 3d fixes (#1771) 2026-07-10 16:41:59 -04:00
Brandon Thomas cb6d0fe27c new tags system (replacing the old one) (#1770) 2026-07-10 05:21:49 -04:00
Brandon Thomas 698c330734 remove old tags code (#1767) 2026-07-10 02:31:37 -04:00
Brandon Thomas 1dfb62454a 8 new persistence endpoints, 2 for each type (#1766) 2026-07-10 01:32:23 -04:00
Brandon Thomas 87f35f3e6f Remove endpoints for old Rust / Bevy game engine (#1765) 2026-07-09 23:52:16 -04:00
Brandon Thomas df0a82cbdb triposplat support (#1764) 2026-07-09 21:00:32 -04:00
Brandon Thomas c5071d4f2e list project media files endpoint (#1763) 2026-07-09 20:08:59 -04:00
Brandon Thomas 57c77cbbc2 stop writing Dimensional for Mesh/Splat types (#1762) 2026-07-09 17:59:10 -04:00
Brandon Thomas 98c4fad729 add metadata to project files; remove legacy 3d editor endpoint (#1761) 2026-07-09 17:23:09 -04:00
Brandon Thomas f50bfa9ee7 forgot to commit this omg 2026-07-09 16:25:38 -04:00
Brandon Thomas d90d446643 new field for media files, splat/mesh split (#1760) 2026-07-09 00:43:54 -04:00
Brandon Thomas 1d5d63fda1 media files: stop reading and writing deprecated fields and values (#1758) 2026-07-08 21:41:45 -04:00
Amit Kumar Ojha 6af5cc7d0b easyenv: add test coverage for typed env var getters (#1749)
## Summary

easyenv had no test coverage for boolean, num, string, or duration
getters, despite being used across every service and job in the
workspace. adds 34 tests covering the optional/required/or_default triad
for each type.tests use unique env var names per test instead of adding
`serial_test` dev-dep — `env::set_var` is process-global so cargo's
default parallel execution can race. cleanup via a `Drop` guard
(`EnvVarGuard`, in `tests_common.rs`) so a panicking test doesn't leak
state into its neighbors.

## follow-up

left `pathbuf.rs` for a separate PR — it depends on
`storyteller_root::get_substituted_path` which needs more setup than
belongs in this diff.

## behaviors these tests pin down but don't change

- boolean parsing only accepts `true` / `TRUE` / `false` / `FALSE` —
`"1"`, `"0"`, `"yes"`, `"no"` return `ParseError`
- duration parses whole seconds only, no `"5s"` style suffixes

worth flagging in case either is intentional and someone wants to loosen
these separately.

## verification

- `cargo test --package easyenv --test-threads=8` — 34/34 pass, no
flakes across 20 consecutive runs
- `cargo clippy --package easyenv -- -D warnings` — clean
2026-07-08 16:30:43 -04:00
Brandon Thomas a7cd9f3b2f fix validation errors (#1757) 2026-07-08 16:17:35 -04:00
Brandon Thomas 78a5c4ff87 handle many more mesh models (#1755) 2026-07-08 01:02:37 -04:00
Brandon Thomas d53fad0072 user input errors on seedance (#1754) 2026-07-07 23:58:24 -04:00
Brandon Thomas 2e5e9443a3 fix models (#1753) 2026-07-07 20:29:24 -04:00
Brandon Thomas 5bda2a6cb4 mesh and splat omni endpoints (#1752) 2026-07-07 19:25:49 -04:00
Brandon Thomas 9cf1b8e066 Add omni audio! router, web, etc. - seed audio, suno, etc. (#1747) 2026-07-07 03:32:16 -04:00
Brandon Thomas aebc6ab777 seed audio client / (fal) (#1745) 2026-07-07 02:27:50 -04:00
Brandon Thomas d3c89face5 suno client (#1744) 2026-07-07 02:02:58 -04:00
Brandon Thomas 692e0ecf75 4k veo (#1743) 2026-07-07 01:27:53 -04:00
Brandon Thomas 94e1aef78d artcraft_router: Vidu and Veo models (#1742) 2026-07-06 23:51:08 -04:00
Brandon Thomas 9a4bb3b490 reduce num queries from 3 -> 1 (#1741) 2026-07-06 20:07:57 -04:00
Brandon Thomas 299d9b8d23 fix redis tx over database tx pooling issues (#1740) 2026-07-06 19:23:59 -04:00
Brandon Thomas 89eff46c6f database perf (#1739) 2026-07-06 16:53:11 -04:00
Brandon Thomas cec3b50ae4 characters API support (#1738) 2026-07-06 16:12:00 -04:00
Brandon Thomas 4fbe9f54e0 more cleanup (#1734) 2026-07-04 11:53:00 -04:00
Brandon Thomas e850022fe8 further cleanup (#1733) 2026-07-04 11:43:49 -04:00
Brandon Thomas 1a528bba45 remove deprecated endpoints (#1732) 2026-07-04 10:15:59 -04:00
Brandon Thomas b625d5ff79 http header fields on alerts (#1731) 2026-07-03 20:59:52 -04:00
Brandon Thomas d8fd0bfe64 fix request logging (#1730) 2026-07-03 16:07:00 -04:00
Brandon Thomas 3cc02ae7bf admin dashboard spend (#1729) 2026-07-03 04:52:28 -04:00
Brandon Thomas 2c943b61ab improve media files user list query performance (#1728) 2026-07-03 04:10:13 -04:00
Brandon Thomas c0ca274168 cleanup dead handlers (#1726) 2026-07-03 03:08:59 -04:00
Brandon Thomas 27cc125089 trace id logging (#1725) 2026-07-03 02:28:22 -04:00
Brandon Thomas 4ce3f240c7 debug logs (#1724) 2026-07-02 22:09:38 -04:00