From 984e8d3845f852ff71e4f439d0eb486580c66963 Mon Sep 17 00:00:00 2001 From: Tanish Manku Date: Mon, 24 Aug 2026 19:49:59 -0700 Subject: [PATCH] mcp: project context, constraints, and one-time infra request Adds mcp/CLAUDE.md as the standing context for the Artcraft MCP server: hard constraints (no Rust changes, no webapp changes without approval, library OAuth server, Workers hosting, read-only M1), the verify-as-you-go working method, the environment model (bindings not branches, fake upstream), the architecture seams, verified upstream API facts, client requirements, security rules, and the API-shape source-of-truth rule. mcp/docs/infra-request.md batches the one-time Cloudflare, repo-secret, and Google console asks for the backend team. Root CLAUDE.md gains the mcp/ entry in the project tree. Claude-Session: https://claude.ai/code/session_01H3gfN2NCx3hBXSrZXYHUQ3 --- CLAUDE.md | 1 + mcp/CLAUDE.md | 329 ++++++++++++++++++++++++++++++++++++++ mcp/docs/infra-request.md | 53 ++++++ 3 files changed, 383 insertions(+) create mode 100644 mcp/CLAUDE.md create mode 100644 mcp/docs/infra-request.md diff --git a/CLAUDE.md b/CLAUDE.md index 4621dddf28..0595e284c8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -51,6 +51,7 @@ artcraft/ │ │ ├── artcraft/ # ArtCraft the Tauri app's frontend. Used in conjunction with `artcraft` the Rust crate. │ │ └── artcraft-website/ # The website for https://getartcraft.com │ └── libs/ # Support libraries, reusable React components, etc. +├── mcp/ # Artcraft MCP server (Cloudflare Workers, TypeScript). Standalone; see mcp/CLAUDE.md └── Cargo.toml # Rust monorepo workspace ``` diff --git a/mcp/CLAUDE.md b/mcp/CLAUDE.md new file mode 100644 index 0000000000..5a252deb12 --- /dev/null +++ b/mcp/CLAUDE.md @@ -0,0 +1,329 @@ +# Artcraft MCP Server (`mcp/`) + +A standalone TypeScript service at `mcp.getartcraft.com` that lets people connect Claude, +ChatGPT, Gemini CLI and other MCP clients to their own Artcraft account. Read-only in M1 +(credits, account, job status, model catalog, cost estimates). It talks to the production +Artcraft API (`storyteller_web`) exactly like a browser would, on the user's behalf. + +Design doc (decisions, auth flow, tool table, milestones): +https://claude.ai/code/artifact/ba27f019-b444-4d93-ba45-b6703edec913 + +This file is the standing context for anyone (human or Claude) working in `mcp/`. The rules +below are constraints, not suggestions. When a rule and convenience conflict, the rule wins; +raise it with the owner rather than working around it. + +## Hard constraints (decided 2026-08-24) + +- **No changes to the Rust backend.** Ever, from this project. Every gap is absorbed here. + Findings that need backend work go in `mcp/docs/backend-handoff.md`, not in a PR to `crates/`. +- **No changes to the webapp (`frontend/apps/artcraft-webapp`) without explicit owner approval.** + The only candidate changes are (a) a "Connected AI apps" settings link and (b) the M2 + `/connect` sign-in hand-off. Bring the exact diff and the justification *before* writing it. +- **Do not hand-roll the OAuth authorization server.** Use `@cloudflare/workers-oauth-provider`. + It is a solved problem; owning one is not a good use of this project's time. Wrap it behind + our interfaces; never fork it. +- **Minimize involvement of other teams and surface area outside `mcp/`.** DNS, Google Cloud + console, repo secrets are other people's; design so their absence delays nothing (ship on + `*.workers.dev`, fall back to password sign-in until the Google origin is added, etc.). +- **Hosting is Cloudflare Workers.** KV for OAuth state. `wrangler deploy` is the release. +- **Read-only in M1.** No tool may spend credits, upload, mutate, or delete. The upstream path + allowlist enforces this in code (see Architecture) — not tool descriptions, not model behaviour. + +## Environments — one code path, bindings not branches + +The MCP Worker has exactly one code path. Which upstream it talks to, which KV namespace it +uses, and which Google client id it presents are **Worker bindings** read once at +construction (`UPSTREAM_API_HOST`, KV id, `GOOGLE_CLIENT_ID`). There is no `if (env)` in +`mcp/src`, no debug mode, no test-only behaviour. Configuration lives in `wrangler.toml` +`[env.*]` blocks and is reviewed in git like code. + +| Env | MCP server | Upstream | Used for | +|--------------|-----------------------------------------|-------------------------------------------|-----------------------------------------------------------------| +| `local` | `wrangler dev` | `fake-upstream` on `localhost:12345` | the daily loop: unit / contract / OAuth tests, MCP Inspector, SDK client | +| `preview` | PR deploy on `*.workers.dev`, own KV | deployed `fake-upstream` Worker | real Claude.ai / ChatGPT / Gemini CLI end-to-end, fake data and credits | +| `production` | `mcp.getartcraft.com`, own KV | `https://api.storyteller.ai` | read-only smoke with the demo account, post-deploy and weekly | + +### `mcp/fake-upstream/` — a fake of the API, not of this server + +A separate package with its own `wrangler.toml` and Worker name (`artcraft-api-fake`): +in-memory, seeded fixture, stateful for a session, no database, no third-party calls, +sign-in as `localdev1` / `localdev1pass`. It implements only the allowlisted routes plus +sign-in/sign-out, and every response it emits is validated against the published +`api.json` in CI so it cannot drift silently. + +It is **ported from** the unmerged `infra/fake-storyteller-web` branch (same seeded-fixture +approach, same test account) — not copied wholesale, and not maintained as a second copy of +anything. If a route already exists there, port it; do not reinvent it. + +Boundary rules, all enforced mechanically: +- `mcp/src` never imports from `mcp/fake-upstream` (eslint `no-restricted-imports` plus a + test that the production bundle contains no fake-upstream module). +- `[env.production]` pins the literal `https://api.storyteller.ai`; a CI test parses + `wrangler.toml` and asserts production → real host, preview → not the real host. +- The Worker asserts at startup that the configured upstream host matches its environment's + expected host; on mismatch every request fails 500 naming the mismatch. An invariant, not + a switch. +- The production deploy job deploys `artcraft-mcp --env production` only; the fake deploys + only from the preview job. +- Credentials cannot cross: a fake session never validates against production (different + signing) and a real session never validates against the fake (seeded users only). + +### Keeping the fake honest + +- Contract tests validate the fake's responses and our fixtures against `api.json` (Ajv). +- A scheduled CI job runs the allowlisted read-only calls against production with the demo + account, redacts, and diffs response *shapes* against the fake's fixtures. Production is + touched by that job, not by people during development. + +### What still needs production + +Post-deploy read-only smoke, the weekly e2e run, the scheduled recorder, directory review. +Nothing else. The demo account's credentials live only in `mcp/.dev.vars` (gitignored) and +in repo secrets (`SMOKE_USERNAME`, `SMOKE_PASSWORD`) — never in code, fixtures, or test +names. Tests assert shapes, not values, and never assume specific data on that account. + +### Other-team dependencies + +Cloudflare account, GitHub repo secrets, and the Google Cloud console are held by the +backend team. Setup is one-time: batch every ask into `mcp/docs/infra-request.md` and hand it +over once. Until it lands nothing blocks development — `local` needs nothing external, +preview deploys can run from a developer's `wrangler login`, and sign-in is password-only +until the Google origin is added. All MCP work happens on `mcp/*` branches in the dedicated +worktree (a sibling checkout such as `../storyteller-mcp`), never on frontend or Rust branches. + +## Working method — verify as you go + +This is auth code fronting a live product. Errors here are account-security errors, and they +must never arrive in bulk. + +1. **Build in small units and verify each one before moving on.** A function, a route, a + schema. Write or extend its test, run the test, read the output, *then* start the next unit. + Never implement a whole feature and only then check whether it works — the scope is too big + to debug, and a wrong assumption in the first unit poisons everything built on it. +2. **Every PR ships with the tests that prove it.** No "tests in a follow-up". A PR without + tests for its new behaviour is not mergeable. Tests are part of the change, not a companion. +3. **Keep PRs small and single-purpose.** Scaffold; upstream client + types; OAuth provider + wiring; consent page; one tool group; connections page — each its own PR. If a PR needs + more than one paragraph to describe, split it. +4. **Do a broader debug pass after the feature is assembled** — run the full suite, the e2e + client test, and MCP Inspector against `wrangler dev`. This pass is *in addition to* + step 1, never a replacement for it. +5. **Verify claims about the upstream API against the spec or the Rust source, not memory.** + The published spec is https://storyteller-docs.netlify.app/api.json; the handlers are under + `crates/service/web/storyteller_web/src/http_server/`. Read-only reference — see constraints. +6. **When something is uncertain, say so in the PR description** and pin it with a test that + documents the assumed behaviour, so a wrong assumption fails loudly later instead of silently. + +## Architecture + +Layered so that the three things most likely to change are behind interfaces and can be +swapped without touching tools or transport. + +``` +MCP client ──► Worker entry (OAuthProvider wraps the Hono app) + │ + ├── /.well-known/*, /authorize, /token, /register, /revoke (library) + ├── /authorize UI ──► Authenticator ← swap point 1 + ├── /mcp, /sse ──► TokenResolver → Principal ← swap point 2 + │ │ + │ ▼ + │ UpstreamClient (allowlist + UpstreamCredential) ← swap point 3 + │ │ + │ ▼ + │ tools/* (pure functions of Principal + input) + ├── /connections (list / revoke grants; later: personal tokens) + └── /, /healthz +``` + +### Interfaces (the seams) + +- **`Authenticator`** — turns a consent interaction into an `UpstreamCredential`. + M1 implementation: `MpcHostFormAuthenticator` (Google Sign-In primary, username/password + fallback, both proxied server-side to `POST /v1/accounts/google_sso` / `POST /v1/login`). + Planned M2: `WebappRedirectAuthenticator` (`app.getartcraft.com/connect` hand-off). The + consent route must not know which one it has. +- **`UpstreamCredential`** — what we hold for a grant and how it is attached to a request. + M1: `{ kind: "session", signedSession }` sent as the `session` header. Designed to admit + `{ kind: "api_key" }` or `{ kind: "backend_token" }` later if the backend grows a proper + token system. Store `kind` in grant props from day one; never assume "session" outside + the credential module. `revoke()` is part of the interface (session → `POST /v1/logout`). +- **`TokenResolver`** — `resolveToken(bearer) → Principal | null`. M1: OAuth access tokens + via the library. M2: personal tokens (`artcraft_mcp_` prefix) via `resolveExternalToken`. + Tools and the upstream client receive a `Principal` and must not know which token kind + produced it. +- **`GrantStore`** — thin adapter over KV. Not because we plan to leave Workers, but so tests + run against an in-memory implementation and the encryption scheme lives in one place. +- **`UpstreamClient`** — `openapi-fetch` client built from generated types; refuses any path + not in `upstream/allowlist.ts`. Adding a path is a deliberate, reviewed change with a test. + +### Principal + +```ts +type Principal = { + grantId: string; + scopes: Scope[]; // "read:account" | "read:jobs" | "read:catalog" | (later) "generate" + credential: UpstreamCredential; + client: { label: string }; // "Claude", "ChatGPT", "personal token · Gemini API" +}; +``` + +### Lifetimes (why 90 days) + +An Artcraft session cannot be refreshed — there is no endpoint — only held or deleted. So: +access tokens 1 h; refresh tokens rotate, 30 days idle, **90 days absolute**, then re-consent. +Personal tokens (M2) inherit the same 90-day ceiling. On expiry or revoke, always call the +credential's `revoke()`. The 90-day window is also the runway for swapping `UpstreamCredential` +to a stronger backend token system if one appears: grants naturally roll over. + +## Verified facts about the upstream API (2026-08-24) + +Cite these; re-verify if the file references stop matching. + +- Sessions: signed JWT, accepted as the `session` **cookie or** a lowercase `session` **header** + (`crates/lib/actix_artcraft/src/sessions/user_sessions/http_user_session_manager.rs:143-163`). + `POST /v1/login` and `POST /v1/accounts/google_sso` return `signed_session` in the body. + Each login creates a new `user_sessions` row. `POST /v1/logout` with the header deletes + that row and purges its Redis cache (`endpoints/users/logout_handler.rs:40-48`). +- No captcha or bot gate on `/v1/login`. No upstream rate limit on it either — ours is required. +- Session-only endpoints (unreachable with an API key): `/v1/credits/namespace/{ns}`, + `/v1/session`, `/v1/jobs/session`, `/v1/subscriptions/namespace/{ns}`. This is why the + forwarded credential is a session, not an API key. +- `POST /v1/omni_gen/cost/*` is anonymous → estimates are plan-blind. Label them so. + `GET /v1/omni_gen/models/*` is plan-aware when a session is sent. +- Google Sign-In audience is one hardcoded client id checked server-side + (`endpoints/users/google_sso/check_claims.rs`). Using that client id from + `mcp.getartcraft.com` only needs the origin added in the Google Cloud console. +- Job tokens are `jinf_…`; batch status accepts mixed prefixes (`jinf_`, `batch_g_`). +- Error bodies: `{ success: false, error_code, error_code_str, message? }`. + +## Client requirements that bite + +- Unauthenticated `/mcp` → **401** with + `WWW-Authenticate: Bearer resource_metadata="https://mcp.getartcraft.com/.well-known/oauth-protected-resource"`. + Claude ignores this header on any status other than 401. +- Protected-resource `resource` must equal the MCP URL byte-for-byte as users paste it. + One canonical URL: `https://mcp.getartcraft.com/mcp`; 301 everything else. +- Authorization-server metadata must advertise `code_challenge_methods_supported: ["S256"]`, + `client_id_metadata_document_supported: true`, and `"none"` in + `token_endpoint_auth_methods_supported`, or Claude falls back to DCR per connection. +- `/token` accepts `application/x-www-form-urlencoded`; `/register` accepts JSON. Dead + refresh tokens return `invalid_grant`, not a custom code. +- Redirect allowlist: `https://claude.ai/api/mcp/auth_callback`, ChatGPT's callback, and + loopback `http://localhost/callback` + `http://127.0.0.1/callback` with **any port**. +- Claude waits 10 s for discovery/register/token, 30 s for refresh. Anthropic egress is + `160.79.104.0/21`; never WAF-block it on either host. +- Keep a legacy `GET /sse` alongside Streamable HTTP `/mcp`. + +## Security rules + +- Credentials (sessions, tokens, passwords) are never logged, never in URLs, never in error + messages. The fetch wrapper redacts the `session` header; logs carry `grantId` only. +- Grant props stay encrypted with the library's token-wrapped keys. Do not add a plaintext + copy anywhere "for convenience". +- Bearer tokens only in the `Authorization` header. Query-string tokens are rejected. +- Rate limits: per token on `/mcp`; per IP **and** per username on the consent POST; per IP + on `/register`, `/authorize`, `/token`. +- Scopes are enforced at the tool boundary and at the allowlist. Reserve `generate` now; no + M1 grant may carry it. + +## Tool conventions + +- One file per tool in `src/mcp/tools/`, exporting `{ name, description, inputSchema, + outputSchema, annotations, handler }`. Handlers are pure functions of `(principal, input)` + and call only `UpstreamClient`. +- Annotations in M1: `readOnlyHint: true`, `idempotentHint: true`, `openWorldHint: false`. +- Return `structuredContent` **and** a short text block (ChatGPT needs both). +- Descriptions are written for the model: what it returns, when to call it, what the + arguments mean, what tokens look like. Not marketing copy. +- Error mapping: upstream 401 → MCP 401 (metadata pointer); upstream 400 → tool error with + the server's `message`; upstream 5xx → one retry then tool error. Never swallow. +- Upstream types are **generated** from the published `api.json` into + `src/upstream/schema.d.ts` (`pnpm gen:api`). Never hand-write a DTO for an upstream shape. + +### Source of truth for API shape + +- **The published OpenAPI document is the source of truth**: + https://storyteller-docs.netlify.app/api.json (rendered at + https://storyteller-docs.netlify.app/). It is generated from the Rust handlers by + `cargo run --bin docs-cli`, so it is the backend's own statement of the contract. +- App usage (`frontend/libs/api` = `@storyteller/api`, the desktop app's `ApiManager`, the + webapp's call sites) is a **secondary reference** for how endpoints are used in practice — + which fields matter, what values are sent, sequencing — never for shape. +- **When the two disagree, surface it; don't silently pick one.** Note the discrepancy in + the PR description and in `docs/backend-handoff.md` (endpoint, what the spec says, what the + app sends/expects, which we followed). Known examples at time of writing: the TS + `OmniGen{Image,Video}CostResponse` omit `failures_are_refunded`; the TS image-generate + response omits `all_job_tokens`. Build against the spec; flag the drift. +- If the spec itself is wrong (a response observed from production doesn't validate against + it), that is a backend bug: record it in the handoff note with the captured, redacted + response, and pin our behaviour with a test that documents the observed shape. + +## Testing + +Four layers; all run offline in CI (`pnpm test`), inside the Workers runtime via +`@cloudflare/vitest-pool-workers`. + +1. **Unit** — tool handlers, credential module, allowlist, resolver — against an MSW-mocked + upstream with fixtures captured from real responses. +2. **Contract** — every fixture validated with Ajv against `api.json`. A regenerated spec that + changes a shape fails here first. +3. **OAuth conformance** — metadata shape, CIMD flags, DCR, PKCE S256, form-encoded `/token`, + refresh rotation and reuse detection, `invalid_grant`, the 401 header, loopback matching. +4. **End-to-end** — the real `@modelcontextprotocol/sdk` client with an `OAuthClientProvider` + that completes our consent page, against the Worker in miniflare. + +Manual: MCP Inspector against `wrangler dev`. Post-deploy: `pnpm smoke` with a dedicated test +account against production. Weekly scheduled e2e run against production. + +## Code style + +- TypeScript strict, ESM, two-space indent, Node 22 / Workers runtime, pnpm. +- Top-to-bottom reading order in every file, matching the repo convention: constants, then + the primary exported type/function, then supporting types, then helpers below their callers. +- No `any`; no `as` casts to silence the generated types — fix the fixture or the schema. +- `maybe_` is the upstream's optional-field prefix; keep upstream field names verbatim in + generated types, camelCase only in our own domain types. +- eslint 10 flat config (`typescript-eslint` strict + stylistic, type-checked) + prettier at + 100 columns. `pnpm check` runs typecheck → lint → format → test and must be green before a + PR is opened; CI runs the same four steps. +- Binding types come from `wrangler types` (`pnpm types`), not from `@cloudflare/workers-types`. + Rerun it after any `wrangler.toml` change; `pnpm typecheck` does so automatically. +- No feature flags or environment-dependent behaviour in production code paths (repo-wide + rule). Configuration is secrets and bindings, not branches. + +## Deferred, not removed + +Keep these possible; do not build them in M1. + +- Personal tokens for header-based clients (Gemini API/Vertex, OpenAI Responses API, + Anthropic Messages API connector): `resolveExternalToken` + `/connections` UI. Read-only + scopes only, 90-day max, shown once, hashed at rest. +- `WebappRedirectAuthenticator` (`app.getartcraft.com/connect`) replacing the password form. +- `generate` scope and generation tools with cost confirmation via elicitation. +- Swapping `UpstreamCredential` to a backend-issued token if the backend ever offers one. +- Directory submissions (Claude connector directory, ChatGPT apps) — keep annotations and + output schemas review-ready from day one so this is a form, not a refactor. + +## Layout + +``` +mcp/ +├── CLAUDE.md # this file +├── package.json # packageManager: pnpm@11 (via corepack), engines.node >=22, type: module +├── pnpm-workspace.yaml # workspace = root server package + fake-upstream; pnpm build-script allowlist lives here +├── wrangler.toml # local (top-level) / preview / production blocks; the security boundary for upstream hosts +├── worker-configuration.d.ts # GENERATED by `pnpm types` (wrangler types); gitignored; regenerated in typecheck and CI +├── docs/backend-handoff.md # findings for the storyteller-web owner; not our work +├── docs/infra-request.md # one-time batched asks for the backend team (Cloudflare, secrets, Google) +├── fake-upstream/ # separate Worker faking the allowlisted API routes; never imported by src/ +├── scripts/gen-api.ts # fetch api.json → openapi-typescript +├── src/ +│ ├── index.ts # Worker entry: OAuthProvider wrapping the Hono app +│ ├── app.ts # Hono routes +│ ├── auth/ # provider config, Authenticator impls, consent page (Hono JSX) +│ ├── tokens/ # TokenResolver, Principal, GrantStore adapter +│ ├── upstream/ # UpstreamClient, UpstreamCredential, allowlist, generated schema +│ ├── mcp/ # McpServer factory, tools/, resources, prompts +│ └── pages/ # landing, connections +└── test/ # unit/, contract/, oauth/, e2e/, fixtures/ +``` diff --git a/mcp/docs/infra-request.md b/mcp/docs/infra-request.md new file mode 100644 index 0000000000..83a38b8db9 --- /dev/null +++ b/mcp/docs/infra-request.md @@ -0,0 +1,53 @@ +# Artcraft MCP — one-time infrastructure request + +For the backend team, who hold the Cloudflare account, the GitHub repo secrets, and the +Google Cloud console. Everything below is one-time setup; nothing recurring. Items are +ordered so that partial completion still unblocks something. None of it changes +`storyteller-web` or any Rust code. + +Context: `mcp/CLAUDE.md` and the design doc linked there. + +## 1. Cloudflare (unblocks real deploys) + +| Item | Value / notes | +|------------------------------|-------------------------------------------------------------------------------| +| Workers project | name `artcraft-mcp` (matches `mcp/wrangler.toml`) | +| KV namespace | one, named `artcraft-mcp-oauth`; id goes into `wrangler.toml` `[[kv_namespaces]]` | +| API token | scoped to: Workers Scripts:Edit, Workers KV Storage:Edit, Account Settings:Read (for the account id). Not a global key. | +| Account id | needed alongside the token | +| Custom domain (can be later) | `mcp.getartcraft.com` → Workers custom domain if the zone is on Cloudflare; otherwise a CNAME to the `workers.dev` hostname we'll provide | +| Secrets on the Worker | we set these ourselves via `wrangler secret put` once we have the token; nothing for you to enter | + +## 2. GitHub repository secrets (unblocks CI deploys) + +| Secret name | Value | +|-------------------------|----------------------------------------------| +| `CLOUDFLARE_API_TOKEN` | the token from §1 | +| `CLOUDFLARE_ACCOUNT_ID` | the account id from §1 | +| `SMOKE_USERNAME` | the production demo account's username | +| `SMOKE_PASSWORD` | its password | + +The workflow that reads them is `.github/workflows/mcp.yml`, path-filtered on `mcp/**`. +It only deploys on `main`; pull requests run lint/typecheck/tests and nothing else. + +## 3. Google Cloud console (unblocks "Continue with Google" on the consent page) + +On the existing OAuth client `788843034237-uqcg8tbgofrcf1to37e1bqphd924jaf6.apps.googleusercontent.com` +(the one `storyteller-web` already verifies against — no backend change needed): + +- Add `https://mcp.getartcraft.com` to **Authorized JavaScript origins**. +- Add `https://artcraft-mcp..workers.dev` as well, for previews. + +No new client, no new secret. Until this is done, the consent page offers password sign-in +only. + +## 4. Demo account + +Confirm the production demo account may be used for automated read-only smoke tests +after each deploy and a weekly scheduled run. The tests never generate, upload, or delete, +so credits are not consumed. + +## What we'll send back once §1 is done + +- The `workers.dev` hostname (for the CNAME in §1 and the origin in §3). +- The KV namespace id confirmation in `wrangler.toml`.