diff --git a/README.md b/README.md index a55e9f862..e0bcebad5 100644 --- a/README.md +++ b/README.md @@ -1,82 +1,214 @@ -
-
+# ArtCraft Services -The IDE for artists.
- +This repository contains ArtCraft's backend, HTTP API, background workers, and web +frontends. It is a Rust and TypeScript monorepo with shared API definitions, +provider clients, database queries, and development tooling. The checkout also +includes the Tauri desktop application and its shared frontend libraries. ---- +For the product overview, feature demos, and desktop downloads, see +[storytold/artcraft](https://github.com/storytold/artcraft). -ArtCraft -======== -ArtCraft is the IDE for interactive AI image and video creation. -We turn prompting into *crafting*, so your ideas become a form of tangible expression and computing. -This is Adobe Photoshop for everyone, and we're giving away the source code! +## Backend architecture -## Show, Don't Tell: Advanced Crafting Features +[`storyteller-web`](./crates/service/web/storyteller_web) is the main HTTP service, +built with Rust, Actix Web, and Tokio. It handles authentication, accounts, +media uploads and libraries, generation requests, job status, credits, and Stripe +billing. The service retains the `storyteller-web` name, and its hosted API uses +`https://api.storyteller.ai`. -Text-to-image is great, but artists *need control*. It's important to know what your image will look like before you generate it, and it's vitally important to achieve consistency and repeatability. +Generation spans the HTTP service, provider integrations, and asynchronous workers: -| Feature | Demo + Explanation -|-----------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| **Image to Location** |  Placing virtual actors into physical environments establishes single-location consistency. You can film multiple shots within a room without having things disappear. | -| **3D Image Compositing** |  Use images (backdrops, foreground elements, props, etc.) in scenes with depth and blend them naturally together. Just a couple of images usually leads to great compositions. | -| **2D Image Compositing** |  Use images, background removal, layers, and simple drawing tools to precisely compose a scene. | -| **Image to 3D Mesh** |  It's almost impossible to lay out complicated objects or block complicated scenes; turning images into 3D helps position elements exactingly and intentionally. | -| **Character Posing** |  You can dynamically pose your characters to achieve the precise character, scene, and camera blocking before calling "action". | -| **Scene Blocking w/ Kit Bashing** |  Use 3D asset kits to precisely block out your scene: get the correct angles, object positions, and rich depth layering you can't with text prompting. | -| **Character Identity Transfer** |  Use mannequins as simple 3D ControlNets for posing any character. | -| **Background Removal** |  Remove backgrounds from images to make them useful in 2D or 3D compositing. They can be props, layers, or backdrops. | -| **Mixed Asset Crafting** |  You can use image cutouts, worlds, and simple 3D meshes all together to precisely and intentionally lay out your scenes. | -| **Scene Blocking** | (preview coming soon) | -| **Canvas Editing** | (preview coming soon) | -| **Scene Relighting** | (preview coming soon) | +```mermaid +flowchart LR + Clients[Web, desktop, and API clients] --> API[storyteller-web] + API --> Router[artcraft_router] + Router --> Providers[Generation providers] + Providers -->|Webhooks| API + Workers[Background workers] -->|Poll jobs| Providers + API --> DB[(MySQL)] + Workers --> DB + API --> Storage[(Object storage)] + Workers --> Storage +``` -Note: all of the above videos were generated for free with Grok Video; the cost to build this README was negligible. +A typical generation request follows this path: -## Quick and Easy Prompting -We haven't abandoned text-to-asset generation for quick prototyping and ideation. We support every popular workflow in a first class fashion. +1. The API authenticates the caller, validates the request and input media, and + checks the user's access and credits. +2. The generation pipeline calculates the cost, bills the wallet, and uses + [`artcraft_router`](./crates/api_clients/artcraft/artcraft_router) to build and + submit a provider-specific request. The API records inference jobs in MySQL + and returns job tokens to the client. +3. Completion is handled through provider webhooks or polling workers, depending + on the integration. Results are downloaded into object storage, registered as + media files, and associated with the completed jobs. Separate workers handle + follow-up processing such as video thumbnails. +4. Clients poll job-status endpoints and load the resulting media through CDN URLs. -| Feature | Demo + Explanation -|-----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------| -| **Text to Image** |  Text prompt over a dozen different image models. | -| **Image Editing** |  Edit with Nano Banana Pro and GPT Image 1.5. | -| **Image Editing** |  Use inpainting, drawing, masking, etc. to edit images. | -| **Image to Video** |  Image to video with lots of different options and controls. | -| **Image Inpainting** | (preview coming soon) | -| **Image Ingredients** | (preview coming soon) | +Storage responsibilities are split across these components: -## Models and Providers Supported within Artcraft +| Component | Role | +|----------------------------|--------------------------------------------------------------| +| MySQL + SQLx | Accounts, media metadata, inference jobs, wallets, and bills | +| Redis | Caching, rate limiting, and job progress | +| Elasticsearch | Search indexes and queries | +| S3-compatible storage / R2 | Uploaded media, generated assets, and derived files | +| SQLite + SQLx | Local desktop task persistence | -| Provider | Features | -|------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Artcraft | Nano Banana, Nano Banana Pro, GPT-Image-1 / 1.5, Seedance 1.0 Lite / 1.5 Pro / 2.0, Seedream 4 / 4.5, Flux 1.1 / Kontext, Veo 2 / 3 / 3.1, Kling 1.6 / 2.1 / 2.5 / 2.6, Sora 2 / Pro, Hunyuan 3d 2 / 3 | -| Grok | Grok Imagine, Grok Video | -| Midjourney | Image Gen (all versions) | -| Sora | Sora 1, Sora 2, GPT-Image-1 | -| WorldLabs | Marble (Gaussian Splat World Generation) | +HTTP routes and handlers live in +[`storyteller_web/src/http_server`](./crates/service/web/storyteller_web/src/http_server). +MySQL queries belong in the shared +[`mysql_queries`](./crates/schema/database/mysql_queries) crate so that handlers, +workers, and CLI tools use the same data access layer. Reusable billing components +live under [`crates/service/plugins`](./crates/service/plugins), and background +services live under [`crates/service/job`](./crates/service/job). -We're going to be adding the following providers soon: Kling (via Kling website accounts), Google (via API keys), -Runway (via website account), Luma (via website account). +## HTTP API -We're potentially interested in adding other aggregators for those who already have subscriptions and credits at -those providers, for example: OpenArt, FreePik, etc. +The API exposes two generation interfaces: -## Downloads +- **Application endpoints:** `/v1/omni_gen/generate/*` uses user sessions for image, + video, audio, mesh, and splat generation. `/v1/omni_gen/models/*` and + `/v1/omni_gen/cost/*` expose model discovery and cost estimation without requiring + a user session. Application job status is available under `/v1/jobs`. +- **Programmatic endpoints:** `/v1/omni_api` uses an API key in the `Authorization` + header. It provides image and video generation, image/video/audio uploads, and + job-status polling. Use `Authorization: Bearer