Compare commits

..
Author SHA1 Message Date
Brandon Thomas aacf7f5a2f app downloads 2 2026-10-05 06:38:17 -04:00
368 changed files with 29108 additions and 4783 deletions

No files matched your search

+9 -9
View File
@@ -34,6 +34,7 @@ artcraft-services/
├── frontend/
│ ├── apps/artcraft-webapp/ # Browser application at app.getartcraft.com
│ ├── apps/artcraft-website-next/ # Current public website at getartcraft.com (Next.js)
│ ├── apps/artcraft-website/ # Legacy product website (Vite)
│ ├── apps/artcraft-desktop-website/ # Static landing page at desktop.getartcraft.com (Vite)
│ └── libs/ # Shared React components and TypeScript libraries
└── Cargo.toml # Rust workspace
@@ -41,9 +42,9 @@ artcraft-services/
## Updating Desktop Download Links
Desktop release links are configured independently in **two places**. When
Desktop release links are configured independently in **three places**. When
updating the advertised ArtCraft release, update both `WINDOWS_VERSION` and
`MAC_VERSION` in both unless the request explicitly targets one platform or
`MAC_VERSION` in all three unless the request explicitly targets one platform or
app:
1. **Current public website (Next.js):**
@@ -51,7 +52,11 @@ app:
This controls the installer links and visible version at
`https://getartcraft.com/download`. Add the release to `DOWNLOAD_HISTORY`
(newest first), then update both version selectors.
2. **Browser webapp:**
2. **Legacy website (Vite):**
[frontend/apps/artcraft-website/src/config/github_download_links.ts](frontend/apps/artcraft-website/src/config/github_download_links.ts).
Add the release to `DOWNLOAD_HISTORY` and update both version selectors.
Updating only this file does **not** update the current public website.
3. **Browser webapp:**
[frontend/apps/artcraft-webapp/src/config/github_download_links.ts](frontend/apps/artcraft-webapp/src/config/github_download_links.ts).
Update both version constants; these links are used by the welcome and
checkout-success pages at `app.getartcraft.com`.
@@ -60,7 +65,7 @@ Keep previous releases for rollback. The Next.js download page reads its visible
version from `DOWNLOAD_VERSIONS`, which uses the same selectors as the URLs; do
not hardcode a separate version in the page component.
Check that both configs produce the requested release tag and installer
Check that all three configs produce the requested release tag and installer
filenames for Windows (`ArtCraft_<version>_x64-setup.exe`) and macOS
(`ArtCraft_<version>_universal.dmg`). Run `npm run typecheck` from
`frontend/apps/artcraft-website-next` after changing its config. Repository edits
@@ -84,11 +89,6 @@ report a local update as a deployed update.
Organize for top-to-bottom reading. Important things first, details later.
- **`mod.rs` holds module declarations only** (`pub mod foo;`, plus re-exports if needed). Never
put functions, structs, enums, constants, impls or tests in a `mod.rs`: give each its own file
in that directory (for example `threads/poll_foo_thread/poll_foo.rs`, declared from
`threads/poll_foo_thread/mod.rs`).
- **Constants** at the top (after imports)
- **Structs/enums** next; outer structs above inner sub-structs
- **API types** in order: Request, Response, Error
+8 -5
View File
@@ -104,7 +104,7 @@ and signals for state, Three.js for 3D scenes, and shared UI and generation tool
| Path | Purpose |
|----------------------------------|-----------------------------------------------|
| `frontend/apps/artcraft-webapp` | Browser application at `app.getartcraft.com` |
| `frontend/apps/artcraft-website-next` | Product website at `getartcraft.com` (Next.js) |
| `frontend/apps/artcraft-website` | Product website at `getartcraft.com` |
| `frontend/libs/api` | HTTP clients, API host selection, and models |
| `frontend/libs/omni-gen` | Shared generation logic |
| `frontend/libs/components` | Reusable UI, editors, and generation controls |
@@ -190,11 +190,14 @@ The browser app runs at `http://localhost:4201`. `VITE_USE_LOCAL_API=true` selec
the hosted API. API host selection lives in
[`StorytellerApiHostStore`](./frontend/libs/api/src/lib/config/StorytellerApiHostStore.ts).
From `frontend`, build the web app with `npx nx build artcraft-webapp`. The
product website is a standalone Next.js app (not part of the Nx workspace); run it
from the repository root with `./script/website/unix_frontend_website_dev.sh`.
From `frontend`, build the web app or run the product website with:
The website runs at `http://localhost:3000`. Repository-root launchers are in
```bash
npx nx build artcraft-webapp
npx nx dev artcraft-website
```
The website runs at `http://localhost:4200`. Repository-root launchers are in
[`script/website`](./script/website). For desktop development, use the
[ArtCraft desktop repository](https://github.com/storytold/artcraft).
+1
View File
@@ -255,6 +255,7 @@ From the services repository:
SQLX_OFFLINE=true cargo test --offline -p storyteller-web --bin storyteller-web login_challenges
cd frontend
npm exec vitest -- run --config apps/artcraft-webapp/vite.config.ts src/pages/login/desktop-login.spec.tsx src/pages/login/login-bridge-continuation.spec.tsx
npm exec vitest -- run --config apps/artcraft-website/vite.config.ts src/pages/login/desktop-login.spec.tsx
```
`SQLX_OFFLINE` controls compile-time query metadata only. These tests still execute
+2 -2
View File
@@ -35,6 +35,6 @@ VITE_USE_LOCAL_API=true npx nx dev artcraft-webapp
The web app runs at `http://localhost:4201`, using the local backend. To develop
against the hosted API, omit `VITE_USE_LOCAL_API=true`.
For the product website, run `./script/website/unix_frontend_website_dev.sh` from the
repository root; it is a standalone Next.js app on `http://localhost:3000`. Repository-root launchers live under [`script/website`](../script/website).
For the product website, run `npx nx dev artcraft-website` from `frontend`; it uses
`http://localhost:4200`. Repository-root launchers live under [`script/website`](../script/website).
See the [frontend README](../frontend/README.md) for builds and troubleshooting.
@@ -1 +0,0 @@
pub mod release_info;
@@ -1,203 +0,0 @@
use std::collections::BTreeMap;
use chrono::{DateTime, Utc};
use serde_derive::{Deserialize, Serialize};
use url::Url;
use utoipa::ToSchema;
// ── GET /v1/craft_apps/release_info ──
//
// The latest release of every Craft App (PhotoCraft, VectorCraft, …, and ArtCraft Launcher
// itself), as storyteller-web last fetched it from GitHub, plus renames and notices for the
// ArtCraft Launcher. The launcher polls this instead of GitHub; GitHub is its fallback.
//
// Compatibility: clients must ignore fields they don't know, and treat every field as optional.
// New fields may be added; existing ones keep their meaning.
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
pub struct GetCraftAppsReleaseInfoResponse {
pub success: bool,
pub release_info: CraftAppsReleaseInfo,
/// Apps that changed their name (for example PdfCraft → DocuCraft).
pub app_renames: Vec<CraftAppRename>,
/// Apps whose GitHub repository moved (for example `storytold/photocraft` → `art/photocraft`).
pub repo_renames: Vec<CraftAppRepoRename>,
pub important_notices: CraftAppsImportantNotices,
}
#[derive(Clone, Debug, Default, Serialize, Deserialize, ToSchema)]
pub struct CraftAppsReleaseInfo {
/// Keyed by app key: the app's lowercase machine name, as used in its binaries, asset file
/// names and `ai.storyteller.<key>` bundle id (`photocraft`, `cadcraft`, `artcraft-launcher`).
/// Not the display name and not the repo name. Clients should ignore keys they don't know.
pub apps: BTreeMap<String, CraftAppReleaseInfo>,
/// When the least recently fetched app was fetched: every app's data is at least this fresh.
/// Absent until every app has been fetched at least once.
pub last_fetched: Option<DateTime<Utc>>,
}
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
pub struct CraftAppReleaseInfo {
/// Display name (`PhotoCraft`).
pub name: String,
/// GitHub repository, `organization/project` (`storytold/photocraft`).
pub repo: String,
/// The latest published, non-draft, non-prerelease release. Absent when the repository has no
/// release yet, or when it hasn't been fetched yet (see `app_last_fetched`).
pub latest_release: Option<CraftAppRelease>,
/// When this app's release data was last fetched successfully. Absent if it never was; then
/// `latest_release` says nothing and clients should ask GitHub themselves.
pub app_last_fetched: Option<DateTime<Utc>>,
}
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
pub struct CraftAppRelease {
/// Git tag (`v0.5.0`).
pub tag: String,
/// The tag without its leading `v` (`0.5.0`, `0.6.0-rc.1`).
pub version: String,
/// Release title (`PhotoCraft v0.5.0`).
pub name: String,
pub published_at: Option<DateTime<Utc>>,
/// The release's page on GitHub.
pub html_url: String,
pub prerelease: bool,
/// Release notes (Markdown), truncated to 64 KiB.
pub notes_markdown: String,
pub assets: Vec<CraftAppReleaseAsset>,
}
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
pub struct CraftAppReleaseAsset {
/// File name (`photocraft-0.5.0-windows-x64.msi`).
pub name: String,
pub size_bytes: u64,
/// `https://github.com/<repo>/releases/download/<tag>/<name>`.
pub download_url: String,
/// Lowercase hex SHA-256, when GitHub published one for the file.
pub sha256: Option<String>,
/// What the file name says it is, when it follows the Craft App naming scheme
/// (`<app>-<version>-<os>-<arch>[-portable].<ext>`).
pub platform: Option<CraftAppAssetPlatform>,
}
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
pub struct CraftAppAssetPlatform {
pub os: CraftAppOs,
/// Absent for files that aren't for one CPU (the web build, checksums).
pub arch: Option<CraftAppArch>,
pub package_type: CraftAppPackageType,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, ToSchema)]
#[serde(rename_all = "snake_case")]
pub enum CraftAppOs {
Windows,
Macos,
Linux,
Freebsd,
/// The browser build.
Web,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, ToSchema)]
#[serde(rename_all = "snake_case")]
pub enum CraftAppArch {
X64,
X86,
Arm64,
/// macOS universal binary (Apple silicon and Intel).
Universal,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, ToSchema)]
#[serde(rename_all = "snake_case")]
pub enum CraftAppPackageType {
/// Windows Installer.
Msi,
/// Windows portable zip.
PortableZip,
/// macOS disk image.
Dmg,
/// macOS command-line tools zip.
CliZip,
AppImage,
/// AppImage delta-update file.
AppImageZsync,
Deb,
Rpm,
Flatpak,
/// Linux or FreeBSD tarball.
TarGz,
/// The web (WebAssembly) build.
WebZip,
}
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
pub struct CraftAppRename {
/// User-facing name before (`PdfCraft`).
pub old_name: String,
/// Lowercase name before (`pdfcraft`).
pub old_name_lower: String,
/// User-facing name after (`DocuCraft`).
pub new_name: String,
/// Lowercase name after (`docucraft`).
pub new_name_lower: String,
/// A sentence or two to show beside the rename.
pub description: String,
pub renamed_at: Option<DateTime<Utc>>,
}
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
pub struct CraftAppRepoRename {
/// `organization/project` before.
pub old_name: String,
/// `organization/project` after.
pub new_name: String,
pub renamed_at: Option<DateTime<Utc>>,
}
#[derive(Clone, Debug, Default, Serialize, Deserialize, ToSchema)]
pub struct CraftAppsImportantNotices {
/// News, shown inline; the user can dismiss each.
pub announcements: Vec<CraftAppsAnnouncement>,
/// Shown as a popup the user acknowledges.
pub alerts: Vec<CraftAppsAlert>,
}
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
pub struct CraftAppsAnnouncement {
pub title: String,
pub body: String,
pub created_at: Option<DateTime<Utc>>,
}
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
pub struct CraftAppsAlert {
pub title: String,
pub body: String,
/// A page the popup can offer to open.
pub maybe_web_link: Option<Url>,
pub created_at: Option<DateTime<Utc>>,
}
@@ -2,7 +2,6 @@ pub mod analytics;
pub mod api_keys;
pub mod characters;
pub mod common;
pub mod craft_apps;
pub mod credits;
pub mod folders;
pub mod generate;
@@ -1,41 +0,0 @@
//! The latest release of one Craft App, shared by every storyteller-web instance so the fleet
//! polls GitHub once per app instead of once per pod.
//!
//! A Redis hash, written by `poll_craft_app_releases_thread` (storyteller-web), which documents
//! the fields. Expires if nobody refreshes it for a week, which only happens if polling stops.
use chrono::Duration;
pub struct CraftAppReleaseRedisKey(pub String);
impl_string_key!(CraftAppReleaseRedisKey);
const REDIS_KEY_TTL_DURATION: Duration = Duration::milliseconds(1000 * 60 * 60 * 24 * 7);
impl CraftAppReleaseRedisKey {
/// `app_key` is the app's lowercase machine name (`photocraft`, `artcraft-launcher`).
pub fn new_for_app(app_key: &str) -> Self {
let key = format!("craftApps:releases:app:{}", app_key);
Self(key)
}
pub fn get_redis_ttl() -> Duration {
REDIS_KEY_TTL_DURATION
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_new_for_app() {
let key = CraftAppReleaseRedisKey::new_for_app("photocraft");
assert_eq!(key.as_str(), "craftApps:releases:app:photocraft");
}
#[test]
fn test_duration() {
assert_eq!(CraftAppReleaseRedisKey::get_redis_ttl().num_days(), 7);
}
}
@@ -1,39 +0,0 @@
//! The fleet-wide GitHub backoff for Craft App release polling: after GitHub rate-limits us (or
//! every request of a round fails), no storyteller-web instance asks GitHub again until it ends.
//!
//! A Redis hash, written by `poll_craft_app_releases_thread` (storyteller-web), which documents
//! the fields. It outlives the backoff itself so repeated trouble keeps growing the wait; it is
//! deleted after a good round.
use chrono::Duration;
pub struct CraftAppsReleaseBackoffRedisKey(pub String);
impl_string_key!(CraftAppsReleaseBackoffRedisKey);
const REDIS_KEY_TTL_DURATION: Duration = Duration::milliseconds(1000 * 60 * 60 * 24);
impl CraftAppsReleaseBackoffRedisKey {
pub fn new_for_fleet() -> Self {
Self("craftApps:releases:backoff".to_string())
}
pub fn get_redis_ttl() -> Duration {
REDIS_KEY_TTL_DURATION
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_new_for_fleet() {
assert_eq!(CraftAppsReleaseBackoffRedisKey::new_for_fleet().as_str(), "craftApps:releases:backoff");
}
#[test]
fn test_duration() {
assert_eq!(CraftAppsReleaseBackoffRedisKey::get_redis_ttl().num_hours(), 24);
}
}
@@ -1,2 +0,0 @@
pub mod craft_app_release_redis_key;
pub mod craft_apps_release_backoff_redis_key;
@@ -1,3 +1,2 @@
pub mod craft_apps;
pub mod premium;
pub mod inference_job;
pub mod inference_job;
@@ -1,11 +1,10 @@
use std::str::FromStr;
use std::time::Duration;
use log::info;
use errors::AnyhowResult;
use shared_env_var_config::mysql::env_get_mysql_connection_string_or_default;
use sqlx::mysql::{MySqlConnectOptions, MySqlPoolOptions};
use sqlx::mysql::MySqlPoolOptions;
use sqlx::MySqlPool;
pub async fn connect_to_database() -> AnyhowResult<MySqlPool> {
@@ -21,37 +20,20 @@ pub async fn connect_to_database() -> AnyhowResult<MySqlPool> {
// shorter timeout sheds load and surfaces the problem instead of hiding it behind long waits.
let acquire_timeout_seconds = easyenv::get_env_num::<u64>("MYSQL_ACQUIRE_TIMEOUT_SECONDS", 10)?;
// NB: Every cached prepared statement stays open on the server until its connection evicts it
// or closes, and the server caps the total across ALL clients (`max_prepared_stmt_count`,
// 16382 on the managed DB). sqlx's default of 100 per connection let `replicas *
// MYSQL_MAX_CONNECTIONS * 100` blow past that cap, failing new prepares with MySQL error 1461.
// Keep `replicas * MYSQL_MAX_CONNECTIONS * MYSQL_STATEMENT_CACHE_CAPACITY` well under it.
let statement_cache_capacity = easyenv::get_env_num::<usize>("MYSQL_STATEMENT_CACHE_CAPACITY", 25)?;
// NB: Recycling connections releases any server-side statements orphaned by cancelled queries
// (eg. a client disconnect between prepare and close). sqlx's default lifetime is 30 minutes.
let max_lifetime_seconds = easyenv::get_env_num::<u64>("MYSQL_MAX_LIFETIME_SECONDS", 600)?;
// NB: Log the settings before building/connecting so a connect failure has context. We do NOT
// log the connection string (it contains credentials).
info!(
"Building MySQL pool: max_connections={}, min_connections={}, acquire_timeout_seconds={}, \
statement_cache_capacity={}, max_lifetime_seconds={}",
"Building MySQL pool: max_connections={}, min_connections={}, acquire_timeout_seconds={}",
max_connections, min_connections, acquire_timeout_seconds,
statement_cache_capacity, max_lifetime_seconds,
);
let connect_options = MySqlConnectOptions::from_str(&db_connection_string)?
.statement_cache_capacity(statement_cache_capacity);
let pool_options = MySqlPoolOptions::new()
.max_connections(max_connections)
.min_connections(min_connections)
.acquire_timeout(Duration::from_secs(acquire_timeout_seconds))
.max_lifetime(Some(Duration::from_secs(max_lifetime_seconds)));
.acquire_timeout(Duration::from_secs(acquire_timeout_seconds));
let pool = pool_options
.connect_with(connect_options)
.connect(&db_connection_string)
.await?;
Ok(pool)
@@ -162,7 +162,6 @@ use artcraft_api_defs::moderation::user_daily_spends::user_daily_spends_list::*;
use artcraft_api_defs::moderation::top_spenders::list::*;
use artcraft_api_defs::moderation::user_spend_events::list::*;
use artcraft_api_defs::folders::subfolder::*;
use artcraft_api_defs::craft_apps::release_info::*;
use artcraft_api_defs::tags::add_media_file_tags::*;
use artcraft_api_defs::tags::bulk_add_tags::*;
use artcraft_api_defs::tags::bulk_list_media_file_tags::*;
@@ -735,9 +734,6 @@ use crate::http_server::endpoints::media_files::list::list_batch_generated_redux
crate::http_server::endpoints::folders::media_files::bulk_move_folder_media_files_handler::bulk_move_folder_media_files_handler,
crate::http_server::endpoints::folders::media_files::bulk_remove_folder_media_files_handler::bulk_remove_folder_media_files_handler,
// Craft Apps
crate::http_server::endpoints::craft_apps::get_craft_apps_release_info_handler::get_craft_apps_release_info_handler,
// Tags
crate::http_server::endpoints::tags::list_tags_handler::list_tags_handler,
crate::http_server::endpoints::tags::list_media_file_tags_handler::list_media_file_tags_handler,
@@ -1610,20 +1606,6 @@ use crate::http_server::endpoints::media_files::list::list_batch_generated_redux
ListTaggedMediaFilesSuccessResponse,
ListTagsQueryParams,
ListTagsSuccessResponse,
GetCraftAppsReleaseInfoResponse,
CraftAppsReleaseInfo,
CraftAppReleaseInfo,
CraftAppRelease,
CraftAppReleaseAsset,
CraftAppAssetPlatform,
CraftAppOs,
CraftAppArch,
CraftAppPackageType,
CraftAppRename,
CraftAppRepoRename,
CraftAppsImportantNotices,
CraftAppsAnnouncement,
CraftAppsAlert,
ListUntaggedMediaFilesQueryParams,
ListUntaggedMediaFilesSuccessResponse,
MediaFileTagsEntry,
@@ -1,73 +0,0 @@
use std::sync::Arc;
use actix_web::http::header::{CacheControl, CacheDirective};
use actix_web::{web, HttpResponse};
use artcraft_api_defs::craft_apps::release_info::{CraftAppsImportantNotices, CraftAppsReleaseInfo, GetCraftAppsReleaseInfoResponse};
use crate::state::server_state::ServerState;
/// Shared caches (Cloudflare) may serve the same answer for this long once every app has been
/// fetched: the data changes at most once per polling interval.
const CACHE_MAX_AGE_SECS: u32 = 60;
/// The latest release of every Craft App (and ArtCraft Launcher), as storyteller-web last fetched
/// it from GitHub, plus app/repo renames and notices for the launcher. Never fails: before the
/// first fetch, apps are listed without a release and without `app_last_fetched`.
#[utoipa::path(
get,
tag = "Craft Apps",
path = "/v1/craft_apps/release_info",
responses(
(status = 200, body = GetCraftAppsReleaseInfoResponse),
),
)]
pub async fn get_craft_apps_release_info_handler(
server_state: web::Data<Arc<ServerState>>,
) -> HttpResponse {
let release_info = server_state.craft_apps_release_cache.snapshot();
let cache_control = cache_control_for(&release_info);
let response = GetCraftAppsReleaseInfoResponse {
success: true,
release_info,
// None yet. When there are, they're listed here (newest first).
app_renames: Vec::new(),
repo_renames: Vec::new(),
important_notices: CraftAppsImportantNotices::default(),
};
HttpResponse::Ok()
.insert_header(cache_control)
.json(response)
}
/// Cacheable only once every app has been fetched (`last_fetched` is set). Until then (an
/// instance that just started, or one whose first fetch failed) the answer is `no-store`, so a
/// shared cache never keeps handing out "not fetched yet" after this instance has warmed up.
fn cache_control_for(release_info: &CraftAppsReleaseInfo) -> CacheControl {
if release_info.last_fetched.is_some() {
CacheControl(vec![CacheDirective::Public, CacheDirective::MaxAge(CACHE_MAX_AGE_SECS)])
} else {
CacheControl(vec![CacheDirective::NoStore])
}
}
#[cfg(test)]
mod tests {
use std::collections::BTreeMap;
use chrono::Utc;
use super::*;
#[test]
fn a_fully_fetched_answer_is_cacheable() {
let release_info = CraftAppsReleaseInfo { apps: BTreeMap::new(), last_fetched: Some(Utc::now()) };
assert_eq!(cache_control_for(&release_info).to_string(), "public, max-age=60");
}
#[test]
fn an_answer_with_unfetched_apps_is_never_stored() {
let release_info = CraftAppsReleaseInfo { apps: BTreeMap::new(), last_fetched: None };
assert_eq!(cache_control_for(&release_info).to_string(), "no-store");
}
}
@@ -1 +0,0 @@
pub mod get_craft_apps_release_info_handler;
@@ -5,7 +5,6 @@ pub mod beta_keys;
pub mod billing_fakeyou;
pub mod characters;
pub mod comments;
pub mod craft_apps;
pub mod credits;
pub mod dev;
pub mod download_job;
@@ -63,7 +63,6 @@ use crate::startup::setup_inference_providers::{
use crate::startup::setup_pager::build_pager;
use crate::startup::setup_static_feature_flags::setup_static_feature_flags;
use crate::state::certs::google_sign_in_cert::GoogleSignInCert;
use crate::state::craft_apps_release_cache::CraftAppsReleaseCache;
use crate::state::memory_cache::model_token_to_info_cache::ModelTokenToInfoCache;
use crate::state::server_state::{
Dashboards, DataboxDashboards, DurableInMemoryCaches, EnvConfig, EphemeralInMemoryCaches,
@@ -703,7 +702,6 @@ pub(crate) fn build_test_server_state(pool: MySqlPool, media_cdn_override_url: S
sort_key_crypto: WebSortKeyCrypto::new("test-sort-key"),
opaque_cursors: WebOpaqueCursorEncoderV2::new("test-sort-key"),
ip_ban_list: IpBanList::new(),
craft_apps_release_cache: CraftAppsReleaseCache::new(),
cidr_ban_set: BannedCidrSet::new(),
troll_bans: TrollBans {
user_tokens: TrollUserBanList::new(),
@@ -3,7 +3,6 @@ use crate::http_server::routes::application_routes::api_keys_routes::add_api_key
use crate::http_server::routes::application_routes::billing_fakeyou_routes::add_billing_fakeyou_routes;
use crate::http_server::routes::application_routes::character_routes::add_character_routes;
use crate::http_server::routes::application_routes::comments_routes::add_comments_routes;
use crate::http_server::routes::application_routes::craft_apps_routes::add_craft_apps_routes;
use crate::http_server::routes::application_routes::credits_routes::add_credits_routes;
use crate::http_server::routes::application_routes::dev_routes::add_dev_routes;
use crate::http_server::routes::application_routes::featured_item_routes::add_featured_item_routes;
@@ -77,7 +76,6 @@ where
app = add_video_info_routes(app); // /v1/video_info/... (video provenance detection)
app = add_analytics_routes(app); // /v1/analytics/...
app = add_web_referrals_routes(app); // /v1/web_referrals/...
app = add_craft_apps_routes(app); // /v1/craft_apps/... (ArtCraft Launcher release info)
app = add_dev_routes(app); // /v1/dev/... — dev-only error/response shape probes
// User and user-adjacent routes
@@ -1,28 +0,0 @@
use actix_http::body::MessageBody;
use actix_service::ServiceFactory;
use actix_web::dev::{ServiceRequest, ServiceResponse};
use actix_web::{web, App, Error, HttpResponse};
use crate::http_server::endpoints::craft_apps::get_craft_apps_release_info_handler::get_craft_apps_release_info_handler;
pub fn add_craft_apps_routes<T, B>(app: App<T>) -> App<T>
where
B: MessageBody,
T: ServiceFactory<
ServiceRequest,
Config = (),
Response = ServiceResponse<B>,
Error = Error,
InitError = (),
>,
{
app.service(
web::scope("/v1/craft_apps")
// Release info for ArtCraft Launcher (polled instead of GitHub).
.service(
web::resource("/release_info")
.route(web::get().to(get_craft_apps_release_info_handler))
.route(web::head().to(|| HttpResponse::Ok())),
),
)
}
@@ -3,7 +3,6 @@ mod api_keys_routes;
mod billing_fakeyou_routes;
mod character_routes;
mod comments_routes;
mod craft_apps_routes;
mod credits_routes;
mod dev_routes;
mod featured_item_routes;
@@ -37,7 +37,6 @@ use errors::AnyhowResult;
use log::{info, warn};
use server_environment::ServerEnvironment;
use shared_env_var_config::logging::DEFAULT_RUST_LOG;
use shared_env_var_config::redis::env_get_redis_0_connection_string_or_default;
use tokio::runtime::Runtime;
use user_traits_component::traits::internal_session_cache_purge::InternalSessionCachePurge;
@@ -57,9 +56,6 @@ use crate::startup::setup_metrics::build_metrics;
use crate::state::server_state::ServerState;
use crate::threads::db_health_checker_thread::db_health_checker_thread::db_health_checker_thread;
use crate::threads::expire_login_challenges::expire_login_challenges;
use crate::threads::poll_craft_app_releases_thread::config::CraftAppsPollConfig;
use crate::threads::poll_craft_app_releases_thread::poll_craft_app_releases::poll_craft_app_releases;
use crate::threads::poll_craft_app_releases_thread::shared_store::build_poller_redis_pool;
use crate::threads::poll_ip_banlist_thread::poll_ip_bans;
use crate::threads::poll_model_token_info_thread::poll_model_token_info_thread;
@@ -154,18 +150,6 @@ async fn main() -> AnyhowResult<()> {
poll_model_token_info_thread(model_token_info_cache_clone, mysql_pool_clone).await;
});
info!("Spawning craft app release polling thread.");
let craft_apps_release_cache_clone = server_state.craft_apps_release_cache.clone();
let craft_apps_poll_config = CraftAppsPollConfig::from_env();
// Its own small pool with I/O timeouts, so a hung Redis can't tie up the handlers' pool.
let craft_apps_redis_pool = build_poller_redis_pool(&env_get_redis_0_connection_string_or_default())?;
let hostname_clone = server_state.hostname.clone();
tokio_runtime.spawn(async move {
poll_craft_app_releases(craft_apps_release_cache_clone, craft_apps_poll_config, craft_apps_redis_pool, hostname_clone).await;
});
// ==================== Metrics worker ==================== //
let (metrics_collector, maybe_metrics_worker) =
@@ -41,7 +41,6 @@ use crate::startup::setup_static_feature_flags::setup_static_feature_flags;
use crate::startup::setup_stripe_artcraft::setup_stripe_artcraft;
use crate::startup::setup_stripe_fakeyou::setup_stripe_fakeyou;
use crate::state::certs::google_sign_in_cert::GoogleSignInCert;
use crate::state::craft_apps_release_cache::CraftAppsReleaseCache;
use crate::state::memory_cache::model_token_to_info_cache::ModelTokenToInfoCache;
use crate::state::server_state::{
DurableInMemoryCaches, EnvConfig, EphemeralInMemoryCaches,
@@ -260,7 +259,6 @@ pub async fn setup_dependencies(server_hostname: &str) -> AnyhowResult<SetupResu
}
},
ip_ban_list,
craft_apps_release_cache: CraftAppsReleaseCache::new(),
cidr_ban_set,
troll_bans: TrollBans {
user_tokens: user_token_troll_bans,
@@ -1,279 +0,0 @@
//! The latest release of every Craft App, as last fetched from GitHub by
//! `poll_craft_app_releases_thread` (directly, or from the fleet's shared copy in Redis), served
//! by `GET /v1/craft_apps/release_info`. The endpoint always answers from this memory, so trouble
//! with Redis or GitHub never breaks it.
//!
//! ## App keys
//!
//! Apps are keyed by their **app key**: the app's lowercase machine name, the same string the
//! Craft Apps use for their binaries, asset file names and `ai.storyteller.<key>` bundle ids.
//! For example `photocraft`, `pdfcraft`, `cadcraft`, and `artcraft-launcher` for ArtCraft Launcher
//! itself. The key is not the display name (`PhotoCraft`) and not the GitHub repo
//! (`storytold/photocraft`, whose project name can differ: the launcher's repo is
//! `storytold/craft-launcher`).
//!
//! The keys come from the polling config (`poll_craft_app_releases_thread::config`: the built-in
//! list, or `CRAFT_APPS_RELEASE_APPS`) and are served unchanged as the keys of
//! `release_info.apps`, which ArtCraft Launcher matches against its own app ids.
use std::collections::BTreeMap;
use std::sync::{Arc, PoisonError, RwLock};
use chrono::{DateTime, Utc};
use artcraft_api_defs::craft_apps::release_info::{CraftAppRelease, CraftAppReleaseInfo, CraftAppsReleaseInfo};
/// Shared between the polling thread and the HTTP handlers. Cheap to clone.
#[derive(Clone, Default)]
pub struct CraftAppsReleaseCache {
/// App key (lowercase machine name, e.g. `photocraft`; see the module docs) → what's known
/// about that app. A `BTreeMap` so the endpoint lists apps in a stable order.
inner: Arc<RwLock<BTreeMap<String, CachedCraftApp>>>,
}
/// One app: what's known about it, and the ETag to revalidate it with.
#[derive(Clone, Debug)]
pub struct CachedCraftApp {
/// Display name (`PhotoCraft`).
pub name: String,
/// GitHub repository, `organization/project` (`storytold/photocraft`).
pub repo: String,
/// The latest published release. `None` both before the first fetch and when the repository
/// has no release; `app_last_fetched` tells the two apart.
pub latest_release: Option<CraftAppRelease>,
/// When this app's release was last fetched (or confirmed unchanged) successfully. `None`
/// until the first success.
pub app_last_fetched: Option<DateTime<Utc>>,
/// GitHub's ETag for the last successful answer, sent back as `If-None-Match` so an unchanged
/// release costs a `304`.
pub etag: Option<String>,
/// When a request for this app last failed. Only meaningful when newer than
/// `app_last_fetched`; it delays the next attempt so a broken repo isn't retried every tick.
pub last_failed_at: Option<DateTime<Utc>>,
/// How many requests in a row have failed since the last success (when `last_failed_at` is
/// newer than `app_last_fetched`).
pub consecutive_failures: u32,
}
impl CraftAppsReleaseCache {
pub fn new() -> Self {
Self::default()
}
/// Makes sure every app in `apps` has an entry, so the endpoint reports apps that haven't been
/// fetched yet. Each tuple is `(app key, display name, repo)`, for example
/// `("photocraft", "PhotoCraft", "storytold/photocraft")`. Apps already present keep what
/// was fetched for them.
pub fn register_apps(&self, apps: &[(String, String, String)]) {
let mut map = self.inner.write().unwrap_or_else(PoisonError::into_inner);
for (key, name, repo) in apps {
map.entry(key.clone()).or_insert_with(|| CachedCraftApp {
name: name.clone(),
repo: repo.clone(),
latest_release: None,
app_last_fetched: None,
etag: None,
last_failed_at: None,
consecutive_failures: 0,
});
}
}
/// The ETag of the last successful answer for the app with app key `key` (`photocraft`), to
/// send as `If-None-Match`.
pub fn etag(&self, key: &str) -> Option<String> {
let map = self.inner.read().unwrap_or_else(PoisonError::into_inner);
map.get(key).and_then(|app| app.etag.clone())
}
/// Records a complete, successful fetch of the app with app key `key` (`photocraft`): its
/// latest release (`None` if it has none). Ignored for keys that were never registered.
pub fn store(&self, key: &str, latest_release: Option<CraftAppRelease>, etag: Option<String>, fetched_at: DateTime<Utc>) {
let mut map = self.inner.write().unwrap_or_else(PoisonError::into_inner);
if let Some(app) = map.get_mut(key) {
app.latest_release = latest_release;
app.etag = etag;
app.app_last_fetched = Some(fetched_at);
}
}
/// Records that GitHub confirmed the app with app key `key` (`photocraft`) is unchanged
/// (`304 Not Modified`). Ignored for keys that were never registered.
pub fn confirm_unchanged(&self, key: &str, fetched_at: DateTime<Utc>) {
let mut map = self.inner.write().unwrap_or_else(PoisonError::into_inner);
if let Some(app) = map.get_mut(key) {
app.app_last_fetched = Some(fetched_at);
}
}
/// A copy of what's known about the app with app key `key` (`photocraft`).
pub fn app(&self, key: &str) -> Option<CachedCraftApp> {
let map = self.inner.read().unwrap_or_else(PoisonError::into_inner);
map.get(key).cloned()
}
/// Takes a fetch made elsewhere (another instance, through Redis) for the app with app key
/// `key`, if it is newer than what this instance has. Returns whether it was taken. Ignored for
/// keys that were never registered, so the registered app list stays authoritative.
pub fn merge_fetched(&self, key: &str, latest_release: Option<CraftAppRelease>, etag: Option<String>, fetched_at: DateTime<Utc>) -> bool {
let mut map = self.inner.write().unwrap_or_else(PoisonError::into_inner);
let Some(app) = map.get_mut(key) else {
return false;
};
if app.app_last_fetched.is_some_and(|mine| mine >= fetched_at) {
return false;
}
app.latest_release = latest_release;
app.etag = etag;
app.app_last_fetched = Some(fetched_at);
true
}
/// Records a failed request for the app with app key `key` (`photocraft`) and returns how many
/// have failed in a row since its last success. Ignored (returns 0) for unregistered keys.
pub fn record_failure(&self, key: &str, failed_at: DateTime<Utc>) -> u32 {
let mut map = self.inner.write().unwrap_or_else(PoisonError::into_inner);
let Some(app) = map.get_mut(key) else {
return 0;
};
app.consecutive_failures = if app.is_failing() { app.consecutive_failures.saturating_add(1) } else { 1 };
app.last_failed_at = Some(failed_at);
app.consecutive_failures
}
/// Takes a failure recorded elsewhere (another instance, through Redis) for the app with app
/// key `key`, if it is newer than the one this instance knows.
pub fn merge_failure(&self, key: &str, failed_at: DateTime<Utc>, consecutive_failures: u32) {
let mut map = self.inner.write().unwrap_or_else(PoisonError::into_inner);
if let Some(app) = map.get_mut(key) {
if app.last_failed_at.is_none_or(|mine| mine < failed_at) {
app.last_failed_at = Some(failed_at);
app.consecutive_failures = consecutive_failures;
}
}
}
/// Everything, as the endpoint serves it: `apps` keyed by app key.
pub fn snapshot(&self) -> CraftAppsReleaseInfo {
let map = self.inner.read().unwrap_or_else(PoisonError::into_inner);
let apps: BTreeMap<String, CraftAppReleaseInfo> = map
.iter()
.map(|(key, app)| {
(key.clone(), CraftAppReleaseInfo {
name: app.name.clone(),
repo: app.repo.clone(),
latest_release: app.latest_release.clone(),
app_last_fetched: app.app_last_fetched,
})
})
.collect();
let last_fetched = least_recently_fetched(&apps);
CraftAppsReleaseInfo { apps, last_fetched }
}
}
impl CachedCraftApp {
/// Whether the last request for this app failed (no success since).
pub fn is_failing(&self) -> bool {
match (self.last_failed_at, self.app_last_fetched) {
(Some(failed), Some(fetched)) => failed > fetched,
(Some(_), None) => true,
(None, _) => false,
}
}
}
/// The oldest `app_last_fetched`, or `None` while any app has never been fetched.
fn least_recently_fetched(apps: &BTreeMap<String, CraftAppReleaseInfo>) -> Option<DateTime<Utc>> {
if apps.is_empty() {
return None;
}
let mut oldest: Option<DateTime<Utc>> = None;
for app in apps.values() {
let fetched = app.app_last_fetched?;
oldest = Some(oldest.map_or(fetched, |o| o.min(fetched)));
}
oldest
}
#[cfg(test)]
mod tests {
use chrono::TimeZone;
use super::*;
#[test]
fn last_fetched_is_the_oldest_app_and_absent_until_all_are_fetched() {
let cache = CraftAppsReleaseCache::new();
cache.register_apps(&[
("photocraft".into(), "PhotoCraft".into(), "storytold/photocraft".into()),
("filmcraft".into(), "FilmCraft".into(), "storytold/filmcraft".into()),
]);
assert_eq!(cache.snapshot().apps.len(), 2);
assert_eq!(cache.snapshot().last_fetched, None);
let early = Utc.with_ymd_and_hms(2026, 10, 10, 1, 0, 0).unwrap();
let late = Utc.with_ymd_and_hms(2026, 10, 10, 2, 0, 0).unwrap();
cache.store("photocraft", None, Some("\"abc\"".into()), late);
assert_eq!(cache.snapshot().last_fetched, None, "filmcraft hasn't been fetched yet");
cache.confirm_unchanged("filmcraft", early);
assert_eq!(cache.snapshot().last_fetched, Some(early));
assert_eq!(cache.etag("photocraft").as_deref(), Some("\"abc\""));
}
#[test]
fn registering_again_keeps_what_was_fetched() {
let cache = CraftAppsReleaseCache::new();
let apps = [("photocraft".to_string(), "PhotoCraft".to_string(), "storytold/photocraft".to_string())];
cache.register_apps(&apps);
let now = Utc::now();
cache.confirm_unchanged("photocraft", now);
cache.register_apps(&apps);
assert_eq!(cache.snapshot().apps["photocraft"].app_last_fetched, Some(now));
}
#[test]
fn merging_takes_only_newer_fetches_of_registered_apps() {
let cache = CraftAppsReleaseCache::new();
cache.register_apps(&[("photocraft".into(), "PhotoCraft".into(), "storytold/photocraft".into())]);
let early = Utc.with_ymd_and_hms(2026, 10, 10, 1, 0, 0).unwrap();
let late = Utc.with_ymd_and_hms(2026, 10, 10, 2, 0, 0).unwrap();
assert!(cache.merge_fetched("photocraft", None, Some("\"new\"".into()), late));
assert!(!cache.merge_fetched("photocraft", None, Some("\"old\"".into()), early), "older than ours");
assert!(!cache.merge_fetched("photocraft", None, Some("\"same\"".into()), late), "no newer than ours");
assert!(!cache.merge_fetched("unknowncraft", None, None, late), "not registered");
let snapshot = cache.snapshot();
assert_eq!(snapshot.apps.len(), 1);
assert_eq!(snapshot.apps["photocraft"].app_last_fetched, Some(late));
assert_eq!(cache.etag("photocraft").as_deref(), Some("\"new\""));
}
#[test]
fn failures_count_up_until_a_success() {
let cache = CraftAppsReleaseCache::new();
cache.register_apps(&[("photocraft".into(), "PhotoCraft".into(), "storytold/photocraft".into())]);
let t = |h| Utc.with_ymd_and_hms(2026, 10, 10, h, 0, 0).unwrap();
assert_eq!(cache.record_failure("photocraft", t(1)), 1);
assert_eq!(cache.record_failure("photocraft", t(2)), 2);
assert!(cache.app("photocraft").unwrap().is_failing());
cache.store("photocraft", None, None, t(3));
assert!(!cache.app("photocraft").unwrap().is_failing());
assert_eq!(cache.record_failure("photocraft", t(4)), 1, "counting restarts after a success");
cache.merge_failure("photocraft", t(3), 9);
assert_eq!(cache.app("photocraft").unwrap().consecutive_failures, 1, "older failure ignored");
cache.merge_failure("photocraft", t(5), 4);
assert_eq!(cache.app("photocraft").unwrap().consecutive_failures, 4);
}
}
@@ -1,6 +1,5 @@
pub mod cached_queries;
pub mod flags;
pub mod certs;
pub mod craft_apps_release_cache;
pub mod memory_cache;
pub mod server_state;
@@ -1,7 +1,6 @@
use std::collections::HashSet;
use crate::configs::app_startup::username_set::UsernameSet;
use crate::state::craft_apps_release_cache::CraftAppsReleaseCache;
use crate::configs::static_api_tokens::StaticApiTokenSet;
use crate::http_server::deprecated_endpoints::categories::tts::list_fully_computed_assigned_tts_categories::list_fully_computed_assigned_tts_categories::ModelTokensByCategoryToken;
use crate::http_server::endpoints::media_files::list::list_featured_media_files_handler::ListFeaturedMediaFilesQueryParams;
@@ -125,9 +124,6 @@ pub struct ServerState {
pub caches: InMemoryCaches,
/// The latest Craft App releases, kept fresh by `poll_craft_app_releases_thread`.
pub craft_apps_release_cache: CraftAppsReleaseCache,
pub google_sign_in_cert: GoogleSignInCert,
pub temp_dir_creator: ScopedTempDirCreator,
@@ -1,5 +1,4 @@
pub mod db_health_checker_thread;
pub mod expire_login_challenges;
pub mod poll_craft_app_releases_thread;
pub mod poll_ip_banlist_thread;
pub mod poll_model_token_info_thread;
@@ -1,164 +0,0 @@
//! How storyteller-web polls GitHub for Craft App releases, from the environment.
//!
//! | Variable | Default | Meaning |
//! |-------------------------------------------|-----------------------------|---------------------------------------------------------------|
//! | `CRAFT_APPS_RELEASE_POLLING_ENABLED` | `true` | Poll at all (the endpoint still answers) |
//! | `CRAFT_APPS_RELEASE_COORDINATION_ENABLED` | `true` | Share releases and backoff through Redis (see below) |
//! | `CRAFT_APPS_RELEASE_POLL_INTERVAL_SECS` | `120`; `1800` with no token | How old a release may get; at least `60`; `300` with no token |
//! | `CRAFT_APPS_RELEASE_MAX_BACKOFF_SECS` | `14400` (4 h) | Longest wait after repeated failures/rate limits |
//! | `CRAFT_APPS_RELEASE_GITHUB_TOKEN` | (none; optional) | Raises GitHub's limit from 60 to 5000 per hour |
//! | `CRAFT_APPS_RELEASE_APPS` | built-in list | `key=Name=org/repo,…` to replace the app list |
//!
//! The GitHub token is optional. Unset, empty or blank, polling still runs: requests go out
//! without an `Authorization` header, and the interval defaults to the slow no-token value.
//!
//! With coordination (the default), the fleet shares each app's release, ETag and fetch time in
//! Redis, so the whole fleet requests an app from GitHub about once per interval instead of every
//! pod doing so. That is what makes the 2 minute default safe with a token. Without a token
//! GitHub allows 60 requests an hour per IP address and also counts `304`s, so the interval stays
//! at 30 minutes (13 apps: about 26 requests an hour). While Redis is unreachable (or with
//! coordination off) every pod polls on its own at the same interval.
use std::time::Duration;
const DEFAULT_INTERVAL_SECS_WITH_TOKEN: u64 = 120;
const MIN_INTERVAL_SECS_WITH_TOKEN: u64 = 60;
const DEFAULT_INTERVAL_SECS_WITHOUT_TOKEN: u64 = 1800;
const MIN_INTERVAL_SECS_WITHOUT_TOKEN: u64 = 300;
const DEFAULT_MAX_BACKOFF_SECS: u64 = 14_400;
/// The Craft Apps the launcher knows: `(app key, display name, GitHub repo)`. The app key is
/// the app's lowercase machine name; it keys `CraftAppsReleaseCache` and the endpoint's
/// `release_info.apps`, and must match the app id ArtCraft Launcher uses. ArtCraft Launcher is
/// included (key `artcraft-launcher`, repo `storytold/craft-launcher`) so it can check for its
/// own updates.
const DEFAULT_APPS: &[(&str, &str, &str)] = &[
("photocraft", "PhotoCraft", "storytold/photocraft"),
("vectorcraft", "VectorCraft", "storytold/vectorcraft"),
("filmcraft", "FilmCraft", "storytold/filmcraft"),
("lightcraft", "LightCraft", "storytold/lightcraft"),
("pdfcraft", "PdfCraft", "storytold/pdfcraft"),
("effectcraft", "EffectCraft", "storytold/effectcraft"),
("designcraft", "DesignCraft", "storytold/designcraft"),
("wordcraft", "WordCraft", "storytold/wordcraft"),
("deckcraft", "DeckCraft", "storytold/deckcraft"),
("gridcraft", "GridCraft", "storytold/gridcraft"),
("cadcraft", "CADCraft", "storytold/cadcraft"),
("soundcraft", "SoundCraft", "storytold/soundcraft"),
("artcraft-launcher", "ArtCraft Launcher", "storytold/craft-launcher"),
];
#[derive(Clone, Debug)]
pub struct CraftAppsPollConfig {
pub enabled: bool,
/// Share releases and backoff with the fleet through Redis; when false (or while Redis is
/// unreachable) this pod polls on its own.
pub coordination_enabled: bool,
/// How old an app's release may get before it is requested from GitHub again.
pub interval: Duration,
pub max_backoff: Duration,
/// Never empty or blank: those are read as no token.
pub github_token: Option<String>,
/// `(app key, display name, repo)`; see [`DEFAULT_APPS`].
pub apps: Vec<(String, String, String)>,
}
impl CraftAppsPollConfig {
pub fn from_env() -> Self {
let github_token = non_blank_token(easyenv::get_env_string_optional("CRAFT_APPS_RELEASE_GITHUB_TOKEN"));
let has_token = github_token.is_some();
let default_interval = default_interval_secs(has_token);
let requested_interval = easyenv::get_env_num::<u64>("CRAFT_APPS_RELEASE_POLL_INTERVAL_SECS", default_interval)
.unwrap_or(default_interval);
let interval_secs = clamp_interval_secs(requested_interval, has_token);
let max_backoff_secs = easyenv::get_env_num::<u64>("CRAFT_APPS_RELEASE_MAX_BACKOFF_SECS", DEFAULT_MAX_BACKOFF_SECS)
.unwrap_or(DEFAULT_MAX_BACKOFF_SECS)
.max(interval_secs);
let apps = easyenv::get_env_string_optional("CRAFT_APPS_RELEASE_APPS")
.map(|v| parse_app_list(&v))
.filter(|apps| !apps.is_empty())
.unwrap_or_else(default_apps);
Self {
enabled: easyenv::get_env_bool_or_default("CRAFT_APPS_RELEASE_POLLING_ENABLED", true),
coordination_enabled: easyenv::get_env_bool_or_default("CRAFT_APPS_RELEASE_COORDINATION_ENABLED", true),
interval: Duration::from_secs(interval_secs),
max_backoff: Duration::from_secs(max_backoff_secs),
github_token,
apps,
}
}
}
pub fn default_apps() -> Vec<(String, String, String)> {
DEFAULT_APPS.iter().map(|(k, n, r)| (k.to_string(), n.to_string(), r.to_string())).collect()
}
/// The token, trimmed; `None` when unset, empty or blank.
fn non_blank_token(value: Option<String>) -> Option<String> {
value.map(|t| t.trim().to_string()).filter(|t| !t.is_empty())
}
fn default_interval_secs(has_token: bool) -> u64 {
if has_token { DEFAULT_INTERVAL_SECS_WITH_TOKEN } else { DEFAULT_INTERVAL_SECS_WITHOUT_TOKEN }
}
fn clamp_interval_secs(requested: u64, has_token: bool) -> u64 {
let min = if has_token { MIN_INTERVAL_SECS_WITH_TOKEN } else { MIN_INTERVAL_SECS_WITHOUT_TOKEN };
requested.max(min)
}
/// Reads `key=Name=org/repo,…` (app key, display name, repo). Keys are lowercased. Malformed
/// entries are skipped.
fn parse_app_list(value: &str) -> Vec<(String, String, String)> {
value
.split(',')
.filter_map(|entry| {
let mut parts = entry.trim().splitn(3, '=');
let key = parts.next()?.trim().to_lowercase();
let name = parts.next()?.trim().to_string();
let repo = parts.next()?.trim().to_string();
let valid_repo = repo.split('/').count() == 2 && !repo.starts_with('/') && !repo.ends_with('/');
(!key.is_empty() && !name.is_empty() && valid_repo).then_some((key, name, repo))
})
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn reads_an_app_list_and_skips_bad_entries() {
let apps = parse_app_list(" PhotoCraft=PhotoCraft=storytold/photocraft, junk, x=X=noslash, k=K=org/repo ");
assert_eq!(apps, vec![
("photocraft".to_string(), "PhotoCraft".to_string(), "storytold/photocraft".to_string()),
("k".to_string(), "K".to_string(), "org/repo".to_string()),
]);
}
#[test]
fn the_default_list_has_the_suite_and_the_launcher() {
let apps = default_apps();
assert_eq!(apps.len(), 13);
assert!(apps.iter().any(|(k, _, r)| k == "artcraft-launcher" && r == "storytold/craft-launcher"));
}
#[test]
fn a_missing_empty_or_blank_token_is_no_token() {
assert_eq!(non_blank_token(None), None);
assert_eq!(non_blank_token(Some(String::new())), None);
assert_eq!(non_blank_token(Some(" \n".to_string())), None);
assert_eq!(non_blank_token(Some(" ghp_x ".to_string())).as_deref(), Some("ghp_x"));
}
#[test]
fn the_interval_defaults_and_minimum_depend_on_the_token() {
assert_eq!(default_interval_secs(true), 120);
assert_eq!(default_interval_secs(false), 1800);
assert_eq!(clamp_interval_secs(10, true), 60);
assert_eq!(clamp_interval_secs(10, false), 300);
assert_eq!(clamp_interval_secs(90, true), 90);
assert_eq!(clamp_interval_secs(90, false), 300);
assert_eq!(clamp_interval_secs(3600, false), 3600);
}
}
@@ -1,97 +0,0 @@
//! Falling back to polling alone when Redis is unreachable, and saying so without flooding the
//! log every tick.
use std::time::{Duration, Instant};
use log::{info, warn};
/// While Redis keeps failing, repeat the warning at most this often.
const WARN_EVERY: Duration = Duration::from_secs(10 * 60);
/// How this tick works.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum PollMode {
/// With the fleet: shared releases, claims and backoff in Redis.
Coordinated,
/// Alone, from this instance's memory: coordination is off or Redis is unreachable. Same
/// interval and backoff rules, so a Redis outage never stops releases from refreshing.
Standalone,
}
/// Remembers Redis trouble so it is logged when it starts, every [`WARN_EVERY`] while it lasts,
/// and when it ends.
#[derive(Debug, Default)]
pub struct RedisTroubleLog {
failing_since: Option<Instant>,
last_warned: Option<Instant>,
suppressed: u32,
}
impl PollMode {
/// Coordinated only when coordination is on and the shared state could be read.
pub fn choose(coordination_enabled: bool, shared_state_readable: bool) -> Self {
if coordination_enabled && shared_state_readable { Self::Coordinated } else { Self::Standalone }
}
}
impl RedisTroubleLog {
/// Notes a Redis error; returns whether it was logged.
pub fn failed(&mut self, err: &anyhow::Error, now: Instant) -> bool {
self.failing_since.get_or_insert(now);
if self.last_warned.is_some_and(|last| now.duration_since(last) < WARN_EVERY) {
self.suppressed = self.suppressed.saturating_add(1);
return false;
}
warn!(
"Craft app releases: Redis unavailable ({:#}); polling GitHub on this instance alone until it is back ({} similar errors not logged).",
err, self.suppressed,
);
self.last_warned = Some(now);
self.suppressed = 0;
true
}
/// Notes that Redis works; logs the recovery once.
pub fn succeeded(&mut self, now: Instant) {
if let Some(since) = self.failing_since.take() {
info!("Craft app releases: Redis is back after {:?}; coordinating with the fleet again.", now.duration_since(since));
self.last_warned = None;
self.suppressed = 0;
}
}
pub fn is_failing(&self) -> bool {
self.failing_since.is_some()
}
}
#[cfg(test)]
mod tests {
use anyhow::anyhow;
use super::*;
#[test]
fn coordinates_only_when_enabled_and_redis_answers() {
assert_eq!(PollMode::choose(true, true), PollMode::Coordinated);
assert_eq!(PollMode::choose(true, false), PollMode::Standalone);
assert_eq!(PollMode::choose(false, true), PollMode::Standalone);
assert_eq!(PollMode::choose(false, false), PollMode::Standalone);
}
#[test]
fn warns_once_per_window_while_redis_keeps_failing() {
let mut log = RedisTroubleLog::default();
let start = Instant::now();
let err = anyhow!("connection refused");
assert!(log.failed(&err, start));
assert!(!log.failed(&err, start + Duration::from_secs(30)));
assert!(!log.failed(&err, start + Duration::from_secs(599)));
assert!(log.failed(&err, start + Duration::from_secs(600)));
assert!(log.is_failing());
log.succeeded(start + Duration::from_secs(700));
assert!(!log.is_failing());
assert!(log.failed(&err, start + Duration::from_secs(730)), "a new outage warns right away");
}
}
@@ -1,304 +0,0 @@
//! One request to GitHub for an app's latest release, and reading the answer.
use std::time::Duration;
use chrono::{DateTime, Utc};
use wreq::header::{ACCEPT, AUTHORIZATION, ETAG, IF_NONE_MATCH, RETRY_AFTER, USER_AGENT};
use wreq::StatusCode;
use serde_derive::Deserialize;
use artcraft_api_defs::craft_apps::release_info::{
CraftAppArch, CraftAppAssetPlatform, CraftAppOs, CraftAppPackageType, CraftAppRelease, CraftAppReleaseAsset,
};
const API_BASE: &str = "https://api.github.com";
const MAX_NOTES_BYTES: usize = 64 * 1024;
const MAX_ASSETS: usize = 200;
const MAX_BODY_BYTES: usize = 4 * 1024 * 1024;
/// What one request produced.
#[derive(Debug)]
pub enum FetchOutcome {
/// The latest release (`None`: the repository has no published release).
Fetched { release: Option<CraftAppRelease>, etag: Option<String> },
/// `If-None-Match` matched: what we have is current.
NotModified,
/// GitHub refused: out of quota until this time, if it said.
RateLimited { retry_at: Option<DateTime<Utc>> },
/// Network error, unexpected status or unreadable answer. Nothing is stored.
Failed(String),
}
pub fn build_client() -> wreq::Result<wreq::Client> {
wreq::Client::builder()
.connect_timeout(Duration::from_secs(10))
.timeout(Duration::from_secs(30))
.build()
}
/// `GET /repos/{repo}/releases/latest`, revalidated with `etag`. `token` is optional: without
/// one the request is unauthenticated.
pub async fn fetch_latest_release(client: &wreq::Client, repo: &str, etag: Option<&str>, token: Option<&str>) -> FetchOutcome {
let request = build_request(client, repo, etag, token);
let response = match request.send().await {
Ok(r) => r,
Err(err) => return FetchOutcome::Failed(format!("request failed: {err}")),
};
let status = response.status();
let header = |name: &str| response.headers().get(name).and_then(|v| v.to_str().ok()).map(str::to_string);
match status {
StatusCode::NOT_MODIFIED => FetchOutcome::NotModified,
StatusCode::NOT_FOUND => FetchOutcome::Fetched { release: None, etag: None },
StatusCode::FORBIDDEN | StatusCode::TOO_MANY_REQUESTS => {
let remaining = header("x-ratelimit-remaining");
let reset = header("x-ratelimit-reset").and_then(|r| r.parse::<i64>().ok()).and_then(|s| DateTime::from_timestamp(s, 0));
let retry_after = header(RETRY_AFTER.as_str())
.and_then(|r| r.parse::<i64>().ok())
.map(|s| Utc::now() + chrono::Duration::seconds(s.clamp(0, 86_400)));
if status == StatusCode::TOO_MANY_REQUESTS || remaining.as_deref() == Some("0") || retry_after.is_some() {
FetchOutcome::RateLimited { retry_at: retry_after.or(reset) }
} else {
FetchOutcome::Failed(format!("HTTP {status}"))
}
}
StatusCode::OK => {
let etag = header(ETAG.as_str());
let body = match response.bytes().await {
Ok(b) if b.len() <= MAX_BODY_BYTES => b,
Ok(b) => return FetchOutcome::Failed(format!("answer too large ({} bytes)", b.len())),
Err(err) => return FetchOutcome::Failed(format!("unreadable answer: {err}")),
};
match parse_release(&body, repo) {
Ok(release) => FetchOutcome::Fetched { release: Some(release), etag },
Err(err) => FetchOutcome::Failed(err),
}
}
other => FetchOutcome::Failed(format!("HTTP {other}")),
}
}
/// The request for `repo`'s latest release. Sends `Authorization: Bearer` only for a non-blank
/// token (an authenticated `304` doesn't count against the rate limit); never an empty header.
fn build_request(client: &wreq::Client, repo: &str, etag: Option<&str>, token: Option<&str>) -> wreq::RequestBuilder {
let url = format!("{API_BASE}/repos/{repo}/releases/latest");
let mut request = client
.get(&url)
.header(USER_AGENT, "storyteller-web (craft apps release poller)")
.header(ACCEPT, "application/vnd.github+json")
.header("X-GitHub-Api-Version", "2022-11-28");
if let Some(etag) = etag {
request = request.header(IF_NONE_MATCH, etag);
}
if let Some(token) = token.map(str::trim).filter(|t| !t.is_empty()) {
request = request.header(AUTHORIZATION, format!("Bearer {token}"));
}
request
}
/// The parts of GitHub's release object we use. Everything optional: GitHub's answer is input.
#[derive(Deserialize)]
struct GithubRelease {
#[serde(default)]
tag_name: Option<String>,
#[serde(default)]
name: Option<String>,
#[serde(default)]
html_url: Option<String>,
#[serde(default)]
published_at: Option<DateTime<Utc>>,
#[serde(default)]
prerelease: bool,
#[serde(default)]
draft: bool,
#[serde(default)]
body: Option<String>,
#[serde(default)]
assets: Vec<GithubAsset>,
}
#[derive(Deserialize)]
struct GithubAsset {
#[serde(default)]
name: Option<String>,
#[serde(default)]
size: u64,
#[serde(default)]
browser_download_url: Option<String>,
/// `sha256:<hex>` on assets uploaded since mid-2025.
#[serde(default)]
digest: Option<String>,
}
/// Reads a release, keeping only downloads that really belong to `repo`.
pub fn parse_release(body: &[u8], repo: &str) -> Result<CraftAppRelease, String> {
let raw: GithubRelease = serde_json::from_slice(body).map_err(|e| format!("unreadable release: {e}"))?;
if raw.draft {
return Err("the latest release is a draft".into());
}
let tag = raw.tag_name.filter(|t| !t.is_empty() && t.len() <= 64).ok_or("the release has no tag")?;
let releases_page = format!("https://github.com/{repo}/releases/");
let html_url = raw.html_url.filter(|u| u.starts_with(&releases_page)).unwrap_or_else(|| format!("{releases_page}tag/{tag}"));
let download_prefix = format!("https://github.com/{repo}/releases/download/");
let assets = raw
.assets
.into_iter()
.filter_map(|a| {
let name = a.name.filter(|n| valid_file_name(n))?;
let download_url = a.browser_download_url.filter(|u| u.starts_with(&download_prefix))?;
let sha256 = a.digest.as_deref().and_then(|d| d.strip_prefix("sha256:")).filter(|h| h.len() == 64 && h.bytes().all(|b| b.is_ascii_hexdigit())).map(str::to_ascii_lowercase);
let platform = parse_platform(&name);
Some(CraftAppReleaseAsset { name, size_bytes: a.size, download_url, sha256, platform })
})
.take(MAX_ASSETS)
.collect();
Ok(CraftAppRelease {
version: tag.strip_prefix(['v', 'V']).unwrap_or(&tag).to_string(),
name: raw.name.filter(|n| !n.is_empty()).map(|n| truncate(&n, 200)).unwrap_or_else(|| tag.clone()),
tag,
published_at: raw.published_at,
html_url,
prerelease: raw.prerelease,
notes_markdown: raw.body.map(|b| truncate(&b, MAX_NOTES_BYTES)).unwrap_or_default(),
assets,
})
}
/// Reads `<app>-<version>-<os>-<arch>[-portable].<ext>`, `<app>-cli-<version>-macos-universal.zip`
/// and `<app>-web-<version>.zip`. `None` for anything else (checksums, unknown files).
pub fn parse_platform(name: &str) -> Option<CraftAppAssetPlatform> {
if name.contains("-web-") && name.ends_with(".zip") {
return Some(CraftAppAssetPlatform { os: CraftAppOs::Web, arch: None, package_type: CraftAppPackageType::WebZip });
}
const KINDS: [(&str, CraftAppPackageType); 10] = [
("-portable.zip", CraftAppPackageType::PortableZip),
(".AppImage.zsync", CraftAppPackageType::AppImageZsync),
(".AppImage", CraftAppPackageType::AppImage),
(".tar.gz", CraftAppPackageType::TarGz),
(".msi", CraftAppPackageType::Msi),
(".dmg", CraftAppPackageType::Dmg),
(".deb", CraftAppPackageType::Deb),
(".rpm", CraftAppPackageType::Rpm),
(".flatpak", CraftAppPackageType::Flatpak),
(".zip", CraftAppPackageType::CliZip),
];
let (base, package_type) = KINDS.iter().find_map(|(suffix, kind)| name.strip_suffix(suffix).map(|b| (b, *kind)))?;
let mut parts = base.rsplitn(3, '-');
let (arch, os) = (parts.next()?, parts.next()?);
let os = match os {
"windows" => CraftAppOs::Windows,
"macos" => CraftAppOs::Macos,
"linux" => CraftAppOs::Linux,
"freebsd" => CraftAppOs::Freebsd,
_ => return None,
};
let arch = match arch {
"x64" | "x86_64" => CraftAppArch::X64,
"x86" => CraftAppArch::X86,
"arm64" | "aarch64" => CraftAppArch::Arm64,
"universal" => CraftAppArch::Universal,
_ => return None,
};
// A bare `.zip` is only the macOS command-line tools.
if package_type == CraftAppPackageType::CliZip && !(os == CraftAppOs::Macos && base.contains("-cli-")) {
return None;
}
Some(CraftAppAssetPlatform { os, arch: Some(arch), package_type })
}
fn valid_file_name(name: &str) -> bool {
!name.is_empty()
&& name.len() <= 200
&& !name.starts_with('.')
&& !name.chars().any(|c| c.is_control() || matches!(c, '/' | '\\' | ':' | '<' | '>' | '"' | '|' | '?' | '*'))
}
fn truncate(s: &str, max: usize) -> String {
if s.len() <= max {
return s.to_string();
}
let mut end = max;
while !s.is_char_boundary(end) {
end -= 1;
}
format!("{}…", &s[..end])
}
#[cfg(test)]
mod tests {
use super::*;
const REPO: &str = "storytold/photocraft";
const HASH: &str = "8a5edab282632443219e051e4ade2d1d5bbc671c781051bf1437897cbdfea0f1";
fn release_json(download_url: &str) -> String {
format!(
r#"{{"tag_name":"v0.5.0","name":"PhotoCraft v0.5.0","draft":false,"prerelease":false,
"html_url":"https://github.com/storytold/photocraft/releases/tag/v0.5.0",
"published_at":"2026-10-08T14:52:50Z","body":"Notes","unknown_field":[1,2,3],
"assets":[{{"name":"photocraft-0.5.0-windows-x64.msi","size":7,"browser_download_url":"{download_url}","digest":"sha256:{HASH}"}}]}}"#
)
}
#[test]
fn sends_the_token_as_a_bearer_header() {
let client = build_client().unwrap();
let request = build_request(&client, REPO, Some("\"etag\""), Some("ghp_x")).build().unwrap();
assert_eq!(request.headers().get(AUTHORIZATION).unwrap(), "Bearer ghp_x");
assert_eq!(request.headers().get(IF_NONE_MATCH).unwrap(), "\"etag\"");
}
#[test]
fn without_a_token_the_request_is_unauthenticated() {
let client = build_client().unwrap();
for token in [None, Some(""), Some(" ")] {
let request = build_request(&client, REPO, None, token).build().unwrap();
assert!(request.headers().get(AUTHORIZATION).is_none(), "{:?}", token);
assert!(request.headers().get(IF_NONE_MATCH).is_none());
assert_eq!(request.uri().to_string(), "https://api.github.com/repos/storytold/photocraft/releases/latest");
}
}
#[test]
fn reads_a_release() {
let r = parse_release(release_json("https://github.com/storytold/photocraft/releases/download/v0.5.0/photocraft-0.5.0-windows-x64.msi").as_bytes(), REPO).unwrap();
assert_eq!((r.tag.as_str(), r.version.as_str(), r.name.as_str()), ("v0.5.0", "0.5.0", "PhotoCraft v0.5.0"));
assert_eq!(r.assets.len(), 1);
assert_eq!(r.assets[0].sha256.as_deref(), Some(HASH));
let p = r.assets[0].platform.as_ref().unwrap();
assert_eq!((p.os, p.arch, p.package_type), (CraftAppOs::Windows, Some(CraftAppArch::X64), CraftAppPackageType::Msi));
}
#[test]
fn drops_downloads_that_point_elsewhere_and_rejects_garbage() {
for url in ["https://evil.example/x.msi", "https://github.com/storytold/other/releases/download/v1/x.msi"] {
assert!(parse_release(release_json(url).as_bytes(), REPO).unwrap().assets.is_empty(), "{url}");
}
assert!(parse_release(b"<html>", REPO).is_err());
assert!(parse_release(br#"{"assets":[]}"#, REPO).is_err());
assert!(parse_release(br#"{"tag_name":"v1.0.0","draft":true}"#, REPO).is_err());
}
#[test]
fn reads_every_asset_name_the_suite_publishes() {
let cases = [
("photocraft-0.5.0-macos-universal.dmg", Some((CraftAppOs::Macos, Some(CraftAppArch::Universal), CraftAppPackageType::Dmg))),
("photocraft-cli-0.5.0-macos-universal.zip", Some((CraftAppOs::Macos, Some(CraftAppArch::Universal), CraftAppPackageType::CliZip))),
("photocraft-0.5.0-windows-arm64-portable.zip", Some((CraftAppOs::Windows, Some(CraftAppArch::Arm64), CraftAppPackageType::PortableZip))),
("photocraft-0.5.0-windows-x86.msi", Some((CraftAppOs::Windows, Some(CraftAppArch::X86), CraftAppPackageType::Msi))),
("photocraft-0.5.0-linux-aarch64.AppImage", Some((CraftAppOs::Linux, Some(CraftAppArch::Arm64), CraftAppPackageType::AppImage))),
("photocraft-0.5.0-linux-x86_64.AppImage.zsync", Some((CraftAppOs::Linux, Some(CraftAppArch::X64), CraftAppPackageType::AppImageZsync))),
("photocraft-0.5.0-linux-x86_64.deb", Some((CraftAppOs::Linux, Some(CraftAppArch::X64), CraftAppPackageType::Deb))),
("photocraft-0.5.0-linux-x86_64.rpm", Some((CraftAppOs::Linux, Some(CraftAppArch::X64), CraftAppPackageType::Rpm))),
("photocraft-0.5.0-linux-x86_64.flatpak", Some((CraftAppOs::Linux, Some(CraftAppArch::X64), CraftAppPackageType::Flatpak))),
("photocraft-0.5.0-freebsd-x86_64.tar.gz", Some((CraftAppOs::Freebsd, Some(CraftAppArch::X64), CraftAppPackageType::TarGz))),
("photocraft-0.2.0-rc.1-linux-x86_64.tar.gz", Some((CraftAppOs::Linux, Some(CraftAppArch::X64), CraftAppPackageType::TarGz))),
("photocraft-web-0.5.0.zip", Some((CraftAppOs::Web, None, CraftAppPackageType::WebZip))),
("SHA256SUMS.txt", None),
("photocraft-0.5.0-plan9-x64.msi", None),
];
for (name, want) in cases {
let got = parse_platform(name).map(|p| (p.os, p.arch, p.package_type));
assert_eq!(got, want, "{name}");
}
}
}
@@ -1,7 +0,0 @@
pub mod config;
pub mod fallback;
pub mod github;
pub mod poll_craft_app_releases;
pub mod scheduling;
pub mod shared_entry;
pub mod shared_store;
@@ -1,469 +0,0 @@
//! Keeps [`CraftAppsReleaseCache`] holding the latest release of every Craft App, which
//! `GET /v1/craft_apps/release_info` serves to ArtCraft Launcher (so thousands of launchers don't
//! each poll GitHub).
//!
//! Every instance runs this loop, waking every [`TICK`] (30 s, with jitter). At startup it first
//! takes the fleet's shared releases from Redis, so a new instance serves them at once without
//! asking GitHub. With coordination (the default) the fleet shares its work through Redis
//! (`shared_entry` documents the keys):
//!
//! 1. Read every app's shared entry and the fleet's backoff; take anything newer into memory.
//! The endpoint always serves memory, so it keeps answering whatever Redis does.
//! 2. If the fleet is backing off (GitHub rate limited us, or a whole round failed), stop.
//! 3. For each app that is due (its shared fetch is `interval - CLOCK_SKEW_MARGIN` old, and any
//! failure retry has waited), read its entry again (another instance may just have fetched
//! it), claim it so instances ticking at the same moment don't all ask, then ask GitHub with
//! the shared ETag and share the answer. Shared writes never go back in time.
//! 4. After a round with requests, update the fleet's backoff (a good round clears it).
//!
//! Each app is all-or-nothing: either its latest release is stored with a fresh
//! `app_last_fetched`, or nothing about it changes.
//!
//! If Redis is unreachable (or coordination is off) the instance does the same from its own
//! memory, as every instance did before coordination, and returns to coordinating when Redis
//! answers again. Nothing here panics; trouble is logged and retried.
use std::time::{Duration, Instant};
use chrono::Utc;
use log::{error, info, warn};
use crate::state::craft_apps_release_cache::CraftAppsReleaseCache;
use crate::threads::poll_craft_app_releases_thread::config::CraftAppsPollConfig;
use crate::threads::poll_craft_app_releases_thread::fallback::{PollMode, RedisTroubleLog};
use crate::threads::poll_craft_app_releases_thread::github;
use crate::threads::poll_craft_app_releases_thread::github::FetchOutcome;
use crate::threads::poll_craft_app_releases_thread::scheduling::{backoff_after, is_due, jittered, Backoff, RoundResult, TICK};
use crate::threads::poll_craft_app_releases_thread::shared_entry::{SharedAppEntry, SharedFailure};
use crate::threads::poll_craft_app_releases_thread::shared_store::{PollerRedisPool, SharedStore};
/// Spreads a fleet's first tick over this window, so a deploy doesn't hit GitHub all at once.
const STARTUP_JITTER: Duration = Duration::from_secs(15);
/// Pause between the requests of one round.
const BETWEEN_REQUESTS: Duration = Duration::from_millis(1_000);
pub async fn poll_craft_app_releases(cache: CraftAppsReleaseCache, config: CraftAppsPollConfig, redis_pool: PollerRedisPool, hostname: String) {
cache.register_apps(&config.apps);
if !config.enabled {
info!("Craft app release polling is disabled (CRAFT_APPS_RELEASE_POLLING_ENABLED=false).");
return;
}
let client = match github::build_client() {
Ok(client) => client,
Err(err) => {
error!("Craft app release polling can't start: no HTTP client: {:?}", err);
return;
}
};
info!(
"Polling GitHub for {} craft app releases, each at most every {:?} (GitHub token: {}; coordinated through Redis: {}).",
config.apps.len(), config.interval, config.github_token.is_some(), config.coordination_enabled,
);
let store = config.coordination_enabled.then(|| SharedStore::new(redis_pool, hostname));
let mut poller = Poller::new(client, cache, config, store);
// Warm memory from the fleet's shared copy right away: it costs no GitHub quota, and the
// jitter below only needs to spread out requests to GitHub.
if poller.refresh_from_shared().await {
info!("Craft app releases: took the fleet's shared releases from Redis at startup.");
}
tokio::time::sleep(jittered(STARTUP_JITTER, 1.0)).await;
loop {
poller.tick().await;
tokio::time::sleep(jittered(TICK, 0.1)).await;
}
}
/// The polling loop's state.
struct Poller {
client: wreq::Client,
cache: CraftAppsReleaseCache,
config: CraftAppsPollConfig,
/// `None` when coordination is off.
store: Option<SharedStore>,
/// The backoff this instance knows: the fleet's while Redis answers, its own otherwise.
backoff: Backoff,
redis_log: RedisTroubleLog,
}
/// What one tick did.
#[derive(Debug)]
struct TickReport {
mode: PollMode,
round: RoundResult,
}
/// Whether to ask GitHub for an app, after looking at its shared entry once more.
enum SharedCheck {
Ask,
/// Fresh after all, or another instance is asking right now.
Skip,
/// The fleet started backing off.
StopRound,
}
impl Poller {
fn new(client: wreq::Client, cache: CraftAppsReleaseCache, config: CraftAppsPollConfig, store: Option<SharedStore>) -> Self {
Self { client, cache, config, store, backoff: Backoff::default(), redis_log: RedisTroubleLog::default() }
}
async fn tick(&mut self) -> TickReport {
let shared_state_readable = self.refresh_from_shared().await;
let mode = PollMode::choose(self.store.is_some(), shared_state_readable);
let mut report = TickReport { mode, round: RoundResult::default() };
let now = Utc::now();
if self.backoff.is_active(now) {
return report;
}
let due: Vec<(String, String)> = self
.config
.apps
.iter()
.filter(|(key, _, _)| self.cache.app(key).is_some_and(|app| is_due(&app, now, self.config.interval)))
.map(|(key, _, repo)| (key.clone(), repo.clone()))
.collect();
if due.is_empty() {
return report;
}
let (round, mode) = self.poll_round(&due, mode).await;
report.round = round;
report.mode = mode;
if report.round.requests() == 0 {
return report;
}
self.update_backoff(&report.round, mode).await;
info!(
"Craft app releases ({:?}): {} fetched, {} unchanged, {} failed{}{}.",
mode, report.round.fetched, report.round.unchanged, report.round.failed,
if report.round.rate_limited.is_some() { ", rate limited" } else { "" },
self.backoff.until.map(|until| format!("; backing off until {until}")).unwrap_or_default(),
);
report
}
/// Takes the fleet's shared state into memory. False when coordination is off or Redis
/// couldn't be read (this tick then works alone).
async fn refresh_from_shared(&mut self) -> bool {
let Some(store) = self.store.clone() else {
return false;
};
let keys: Vec<String> = self.config.apps.iter().map(|(key, _, _)| key.clone()).collect();
match store.read(&keys).await {
Ok((entries, backoff)) => {
self.redis_log.succeeded(Instant::now());
for (key, entry) in keys.iter().zip(entries) {
if let Some(entry) = entry {
merge_shared_entry(&self.cache, key, entry);
}
}
self.backoff = backoff;
true
}
Err(err) => {
self.redis_log.failed(&err, Instant::now());
false
}
}
}
/// Asks GitHub about each due app. Returns what happened and the mode it ended in (a Redis
/// error mid-round finishes the round alone).
async fn poll_round(&mut self, due: &[(String, String)], mut mode: PollMode) -> (RoundResult, PollMode) {
let mut round = RoundResult::default();
for (key, repo) in due {
if let Some(store) = self.coordinated_store(mode) {
match self.check_shared(&store, key).await {
Ok(SharedCheck::Ask) => {}
Ok(SharedCheck::Skip) => continue,
Ok(SharedCheck::StopRound) => break,
Err(err) => {
self.redis_log.failed(&err, Instant::now());
mode = PollMode::Standalone;
}
}
}
if round.requests() > 0 {
tokio::time::sleep(BETWEEN_REQUESTS).await;
}
let etag = self.cache.etag(key);
let outcome = github::fetch_latest_release(&self.client, repo, etag.as_deref(), self.config.github_token.as_deref()).await;
let now = Utc::now();
match outcome {
FetchOutcome::Fetched { release, etag } => {
round.fetched += 1;
if let Some(store) = self.coordinated_store(mode) {
let shared = store.write_fetched(key, release.as_ref(), etag.as_deref(), now).await;
mode = self.after_shared_write(shared, mode);
}
self.cache.store(key, release, etag, now);
}
FetchOutcome::NotModified => {
round.unchanged += 1;
self.cache.confirm_unchanged(key, now);
if let (Some(store), Some(etag)) = (self.coordinated_store(mode), etag.as_deref()) {
let shared = store.write_unchanged(key, etag, now).await;
mode = self.after_shared_write(shared, mode);
}
}
FetchOutcome::RateLimited { retry_at } => {
warn!("GitHub rate limit reached polling {repo}; retry after {:?}.", retry_at);
round.rate_limited = Some(retry_at);
// The rest of the round would be refused too.
break;
}
FetchOutcome::Failed(why) => {
warn!("Couldn't fetch the latest release of {repo}: {why}");
round.failed += 1;
let consecutive_failures = self.cache.record_failure(key, now);
if let Some(store) = self.coordinated_store(mode) {
let failure = SharedFailure { failed_at: now, consecutive_failures };
let shared = store.write_failure(key, failure).await;
mode = self.after_shared_write(shared, mode);
}
}
}
}
(round, mode)
}
/// Right before asking GitHub about `key`: takes its latest shared entry, and claims it if it
/// is still due and the fleet isn't backing off.
async fn check_shared(&mut self, store: &SharedStore, key: &str) -> anyhow::Result<SharedCheck> {
let (mut entries, backoff) = store.read(&[key.to_string()]).await?;
if let Some(entry) = entries.pop().flatten() {
merge_shared_entry(&self.cache, key, entry);
}
self.backoff = backoff;
let now = Utc::now();
if backoff.is_active(now) {
return Ok(SharedCheck::StopRound);
}
if !self.cache.app(key).is_some_and(|app| is_due(&app, now, self.config.interval)) {
return Ok(SharedCheck::Skip);
}
Ok(if store.claim(key, now).await? { SharedCheck::Ask } else { SharedCheck::Skip })
}
/// Sets the backoff after a round with requests, and shares it if it changed.
async fn update_backoff(&mut self, round: &RoundResult, mode: PollMode) {
let previous = self.backoff;
self.backoff = backoff_after(&self.config, round, previous, Utc::now());
if self.backoff == previous {
return;
}
if let Some(store) = self.coordinated_store(mode) {
if let Err(err) = store.write_backoff(self.backoff).await {
self.redis_log.failed(&err, Instant::now());
}
}
}
/// Logs a failed shared write; the rest of the round then works alone.
fn after_shared_write(&mut self, result: anyhow::Result<bool>, mode: PollMode) -> PollMode {
match result {
Ok(_) => mode,
Err(err) => {
self.redis_log.failed(&err, Instant::now());
PollMode::Standalone
}
}
}
/// The shared store, in [`PollMode::Coordinated`].
fn coordinated_store(&self, mode: PollMode) -> Option<SharedStore> {
self.store.clone().filter(|_| mode == PollMode::Coordinated)
}
}
/// Takes a shared entry into memory (only what is newer than what this instance has).
fn merge_shared_entry(cache: &CraftAppsReleaseCache, key: &str, entry: SharedAppEntry) {
if let Some(fetched) = entry.fetched {
cache.merge_fetched(key, fetched.latest_release, fetched.etag, fetched.fetched_at);
}
if let Some(failure) = entry.failure {
cache.merge_failure(key, failure.failed_at, failure.consecutive_failures);
}
}
#[cfg(test)]
mod tests {
use chrono::DateTime;
use redis_schema::keys::craft_apps::craft_app_release_redis_key::CraftAppReleaseRedisKey;
use crate::threads::poll_craft_app_releases_thread::config::default_apps;
use crate::threads::poll_craft_app_releases_thread::shared_entry::{decode_app_entry, encode_fetch};
use crate::threads::poll_craft_app_releases_thread::shared_store::build_poller_redis_pool;
use super::*;
/// Live: real rounds against GitHub (two apps), then the endpoint's response, written to
/// `CRAFT_APPS_SAMPLE_OUT` when set (ArtCraft Launcher's parser reads it back). Uses
/// `CRAFT_APPS_RELEASE_GITHUB_TOKEN` if set, and works without it.
/// `cargo test -p storyteller-web --bin storyteller-web live_round -- --ignored`
#[tokio::test]
#[ignore = "needs network access"]
async fn live_round_fetches_real_releases() {
use artcraft_api_defs::craft_apps::release_info::{CraftAppsImportantNotices, GetCraftAppsReleaseInfoResponse};
let mut config = config();
config.github_token = std::env::var("CRAFT_APPS_RELEASE_GITHUB_TOKEN").ok().filter(|t| !t.trim().is_empty());
config.apps.retain(|(key, _, _)| key == "photocraft" || key == "artcraft-launcher");
let cache = CraftAppsReleaseCache::new();
cache.register_apps(&config.apps);
let mut poller = Poller::new(github::build_client().unwrap(), cache.clone(), config.clone(), None);
let due: Vec<(String, String)> = config.apps.iter().map(|(k, _, r)| (k.clone(), r.clone())).collect();
let (round, _) = poller.poll_round(&due, PollMode::Standalone).await;
assert_eq!(round.fetched + round.unchanged, 2, "{round:?}");
// A second round revalidates with ETags.
let (again, _) = poller.poll_round(&due, PollMode::Standalone).await;
assert_eq!(again.fetched + again.unchanged, 2, "{again:?}");
let info = cache.snapshot();
assert!(info.last_fetched.is_some());
let photocraft = info.apps["photocraft"].latest_release.as_ref().unwrap();
assert!(photocraft.assets.iter().any(|a| a.platform.is_some() && a.sha256.is_some()));
let response = GetCraftAppsReleaseInfoResponse {
success: true,
release_info: info,
app_renames: Vec::new(),
repo_renames: Vec::new(),
important_notices: CraftAppsImportantNotices::default(),
};
let json = serde_json::to_string_pretty(&response).unwrap();
if let Ok(path) = std::env::var("CRAFT_APPS_SAMPLE_OUT") {
std::fs::write(path, &json).unwrap();
}
}
/// Against a real Redis (`CRAFT_APPS_TEST_REDIS_URL`, default `redis://127.0.0.1:6379/`): an
/// instance takes another's fresh fetches from Redis instead of asking GitHub.
/// `cargo test -p storyteller-web --bin storyteller-web takes_fresh -- --ignored`
#[tokio::test]
#[ignore = "needs a Redis server"]
async fn takes_fresh_releases_from_redis_instead_of_github() {
let url = std::env::var("CRAFT_APPS_TEST_REDIS_URL").unwrap_or_else(|_| "redis://127.0.0.1:6379/".to_string());
let pool = build_poller_redis_pool(&url).unwrap();
let mut config = config();
let prefix = format!("test-craft-{}", rand::random::<u32>());
config.apps = vec![(format!("{prefix}-a"), "A".into(), "storytold/a".into()), (format!("{prefix}-b"), "B".into(), "storytold/b".into())];
let pod_a = SharedStore::new(pool.clone(), "pod-a".into());
let now = Utc::now();
for (key, _, _) in &config.apps {
assert!(pod_a.write_fetched(key, None, Some("\"e\""), now).await.unwrap());
}
let cache = CraftAppsReleaseCache::new();
cache.register_apps(&config.apps);
let mut pod_b = Poller::new(github::build_client().unwrap(), cache.clone(), config.clone(), Some(SharedStore::new(pool.clone(), "pod-b".into())));
let report = pod_b.tick().await;
assert_eq!(report.mode, PollMode::Coordinated);
assert_eq!(report.round.requests(), 0, "fresh in Redis, so GitHub isn't asked");
for (key, _, _) in &config.apps {
assert_eq!(fetched_at(&cache, key).map(|t| t.timestamp_millis()), Some(now.timestamp_millis()));
assert_eq!(cache.etag(key).as_deref(), Some("\"e\""));
}
let keys: Vec<String> = config.apps.iter().map(|(k, _, _)| CraftAppReleaseRedisKey::new_for_app(k).to_string()).collect();
let mut conn = pool.get().unwrap();
redis::cmd("DEL").arg(&keys).query::<()>(&mut *conn).unwrap();
}
/// Against a real Redis, as above: a new instance serves the fleet's releases right after
/// startup, before its startup jitter ends, without asking GitHub.
/// `cargo test -p storyteller-web --bin storyteller-web warm -- --ignored`
#[tokio::test]
#[ignore = "needs a Redis server"]
async fn a_new_instance_is_warm_from_redis_at_startup() {
let url = std::env::var("CRAFT_APPS_TEST_REDIS_URL").unwrap_or_else(|_| "redis://127.0.0.1:6379/".to_string());
let pool = build_poller_redis_pool(&url).unwrap();
let mut config = config();
let prefix = format!("test-craft-{}", rand::random::<u32>());
config.apps = vec![(format!("{prefix}-a"), "A".into(), "storytold/a".into()), (format!("{prefix}-b"), "B".into(), "storytold/b".into())];
let fetched_by_another_pod = Utc::now();
let other_pod = SharedStore::new(pool.clone(), "other-pod".into());
for (key, _, _) in &config.apps {
assert!(other_pod.write_fetched(key, None, Some("\"e\""), fetched_by_another_pod).await.unwrap());
}
let cache = CraftAppsReleaseCache::new();
let poller = tokio::spawn(poll_craft_app_releases(cache.clone(), config.clone(), pool.clone(), "new-pod".into()));
let deadline = Instant::now() + Duration::from_secs(2);
while config.apps.iter().any(|(key, _, _)| fetched_at(&cache, key).is_none()) && Instant::now() < deadline {
tokio::time::sleep(Duration::from_millis(20)).await;
}
poller.abort();
for (key, _, _) in &config.apps {
assert_eq!(fetched_at(&cache, key).map(|t| t.timestamp_millis()), Some(fetched_by_another_pod.timestamp_millis()), "{}", key);
}
let keys: Vec<String> = config.apps.iter().map(|(k, _, _)| CraftAppReleaseRedisKey::new_for_app(k).to_string()).collect();
let mut conn = pool.get().unwrap();
redis::cmd("DEL").arg(&keys).query::<()>(&mut *conn).unwrap();
}
#[tokio::test]
async fn an_unreachable_redis_falls_back_to_polling_alone() {
let pool = build_poller_redis_pool("redis://127.0.0.1:1/").unwrap();
let mut poller = fresh_poller(Some(SharedStore::new(pool, "pod".into())));
let report = poller.tick().await;
assert_eq!(report.mode, PollMode::Standalone);
assert_eq!(report.round.requests(), 0, "everything was fresh");
assert!(poller.redis_log.is_failing());
}
#[tokio::test]
async fn with_coordination_off_redis_is_never_used() {
let mut poller = fresh_poller(None);
let report = poller.tick().await;
assert_eq!(report.mode, PollMode::Standalone);
assert!(!poller.redis_log.is_failing());
}
#[tokio::test]
async fn an_active_backoff_skips_github_even_when_apps_are_due() {
let mut poller = Poller::new(github::build_client().unwrap(), registered_cache(), config(), None);
poller.backoff = Backoff { until: Some(Utc::now() + chrono::Duration::minutes(5)), trouble: 1 };
let report = poller.tick().await;
assert_eq!(report.round.requests(), 0);
assert_eq!(fetched_at(&poller.cache, "photocraft"), None);
}
#[test]
fn shared_entries_merge_into_memory() {
let cache = registered_cache();
let at = Utc::now();
let fields = encode_fetch(None, Some("\"e\""), at, "pod").unwrap();
let map = fields.into_iter().map(|(k, v)| (k.to_string(), v)).collect();
merge_shared_entry(&cache, "photocraft", decode_app_entry(&map).unwrap());
assert_eq!(fetched_at(&cache, "photocraft").map(|t| t.timestamp_millis()), Some(at.timestamp_millis()));
assert_eq!(cache.etag("photocraft").as_deref(), Some("\"e\""));
}
fn config() -> CraftAppsPollConfig {
CraftAppsPollConfig {
enabled: true,
coordination_enabled: true,
interval: Duration::from_secs(1800),
max_backoff: Duration::from_secs(14_400),
github_token: None,
apps: default_apps(),
}
}
fn registered_cache() -> CraftAppsReleaseCache {
let cache = CraftAppsReleaseCache::new();
cache.register_apps(&config().apps);
cache
}
/// A poller whose apps were all just fetched, so a tick makes no GitHub requests.
fn fresh_poller(store: Option<SharedStore>) -> Poller {
let cache = registered_cache();
for (key, _, _) in &config().apps {
cache.store(key, None, None, Utc::now());
}
Poller::new(github::build_client().unwrap(), cache, config(), store)
}
fn fetched_at(cache: &CraftAppsReleaseCache, app_key: &str) -> Option<DateTime<Utc>> {
cache.app(app_key).and_then(|app| app.app_last_fetched)
}
}
@@ -1,286 +0,0 @@
//! When to ask GitHub: per-app freshness, retries after failures, and the backoff after rate
//! limits. Pure functions of what's known, so the same rules apply whether this instance works
//! from the fleet's shared state in Redis or (as a fallback) only from its own memory.
use std::time::Duration;
use chrono::{DateTime, Utc};
use crate::state::craft_apps_release_cache::CachedCraftApp;
use crate::threads::poll_craft_app_releases_thread::config::CraftAppsPollConfig;
/// How often every instance wakes up to look at the shared state (with jitter).
pub const TICK: Duration = Duration::from_secs(30);
/// Allowance for clocks that differ between instances: an app fetched by another instance
/// counts as fresh until `interval - CLOCK_SKEW_MARGIN` has passed.
pub const CLOCK_SKEW_MARGIN: Duration = Duration::from_secs(30);
/// First wait after a failed request (or a round where every request failed); doubles up to the
/// normal interval.
const FIRST_FAILURE_RETRY: Duration = Duration::from_secs(60);
/// Extra margin after GitHub's announced rate-limit reset.
const RESET_MARGIN: Duration = Duration::from_secs(60);
/// What one round did.
#[derive(Debug, Default)]
pub struct RoundResult {
pub fetched: usize,
pub unchanged: usize,
pub failed: usize,
/// GitHub refused; `Some(Some(t))` when it said when to come back.
pub rate_limited: Option<Option<DateTime<Utc>>>,
}
/// Don't ask GitHub before `until`. `trouble` counts the rounds in a row that were rate limited
/// or failed entirely; each one makes the next wait longer.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub struct Backoff {
pub until: Option<DateTime<Utc>>,
pub trouble: u32,
}
impl RoundResult {
/// How many requests reached a conclusion (a rate limit counts as one).
pub fn requests(&self) -> usize {
self.fetched + self.unchanged + self.failed + usize::from(self.rate_limited.is_some())
}
}
impl Backoff {
pub fn is_active(&self, now: DateTime<Utc>) -> bool {
self.until.is_some_and(|until| until > now)
}
}
/// Whether `app` should be requested from GitHub now. An app is due once its last fetch (by any
/// instance) is `interval - CLOCK_SKEW_MARGIN` old. After a failure it waits a minute, then two,
/// four, … up to the interval, so one broken repository is never retried every tick.
pub fn is_due(app: &CachedCraftApp, now: DateTime<Utc>, interval: Duration) -> bool {
if let Some(fetched) = app.app_last_fetched {
if age(fetched, now) < interval.saturating_sub(CLOCK_SKEW_MARGIN) {
return false;
}
}
if app.is_failing() {
if let Some(failed) = app.last_failed_at {
let retry = scaled(FIRST_FAILURE_RETRY, app.consecutive_failures).min(interval);
if age(failed, now) < retry.saturating_sub(CLOCK_SKEW_MARGIN) {
return false;
}
}
}
true
}
/// The backoff after a round that made at least one request, starting from `previous`. A good
/// round clears it.
pub fn backoff_after(config: &CraftAppsPollConfig, round: &RoundResult, previous: Backoff, now: DateTime<Utc>) -> Backoff {
let mut trouble = previous.trouble;
let wait = next_wait(config, round, &mut trouble, now);
if trouble == 0 {
return Backoff::default();
}
let wait = chrono::Duration::from_std(wait).unwrap_or(chrono::Duration::MAX);
Backoff { until: now.checked_add_signed(wait), trouble }
}
/// How long to wait before the next round. Rate limits wait for GitHub's reset and back off
/// further each time; rounds where everything failed retry sooner, but never faster than a
/// minute, doubling up to the normal interval.
fn next_wait(config: &CraftAppsPollConfig, round: &RoundResult, consecutive_trouble: &mut u32, now: DateTime<Utc>) -> Duration {
let base = if let Some(retry_at) = round.rate_limited {
*consecutive_trouble = consecutive_trouble.saturating_add(1);
let until_reset = retry_at
.and_then(|t| (t - now).to_std().ok())
.map(|d| d + RESET_MARGIN)
.unwrap_or(Duration::ZERO);
let backoff = scaled(config.interval, *consecutive_trouble);
until_reset.max(backoff).min(config.max_backoff)
} else if round.fetched == 0 && round.unchanged == 0 && round.failed > 0 {
*consecutive_trouble = consecutive_trouble.saturating_add(1);
scaled(FIRST_FAILURE_RETRY, *consecutive_trouble).min(config.interval)
} else {
*consecutive_trouble = 0;
config.interval
};
jittered(base, 0.1)
}
/// `base` ± `fraction`, so a fleet doesn't poll in lockstep. With `fraction == 1.0`, anywhere in
/// `0..=2×base`; used for the startup spread.
pub fn jittered(base: Duration, fraction: f64) -> Duration {
let factor = 1.0 + fraction * (rand::random::<f64>() * 2.0 - 1.0);
base.mul_f64(factor.max(0.0))
}
/// `base × 2^(n-1)`, without overflow.
fn scaled(base: Duration, n: u32) -> Duration {
base.saturating_mul(1u32 << n.saturating_sub(1).min(16))
}
/// How long ago `then` was; zero if it is in the future (another instance's clock runs ahead).
fn age(then: DateTime<Utc>, now: DateTime<Utc>) -> Duration {
(now - then).to_std().unwrap_or(Duration::ZERO)
}
#[cfg(test)]
mod tests {
use crate::threads::poll_craft_app_releases_thread::config::default_apps;
use super::*;
mod freshness {
use super::*;
#[test]
fn an_app_never_fetched_is_due() {
assert!(is_due(&app(None, None, 0), Utc::now(), Duration::from_secs(120)));
}
#[test]
fn a_fetch_counts_as_fresh_until_the_interval_minus_the_skew_margin() {
let now = Utc::now();
let interval = Duration::from_secs(120);
assert!(!is_due(&app(Some(now - secs(89)), None, 0), now, interval));
assert!(is_due(&app(Some(now - secs(90)), None, 0), now, interval));
assert!(is_due(&app(Some(now - secs(600)), None, 0), now, interval));
}
#[test]
fn a_fetch_from_a_clock_ahead_of_ours_is_fresh() {
let now = Utc::now();
assert!(!is_due(&app(Some(now + secs(20)), None, 0), now, Duration::from_secs(120)));
}
#[test]
fn a_failing_app_waits_longer_after_each_failure_up_to_the_interval() {
let now = Utc::now();
let interval = Duration::from_secs(1800);
// 1st failure: retry after 60 s (less the margin); 3rd: 240 s.
assert!(!is_due(&app(None, Some(now - secs(29)), 1), now, interval));
assert!(is_due(&app(None, Some(now - secs(30)), 1), now, interval));
assert!(!is_due(&app(None, Some(now - secs(200)), 3), now, interval));
assert!(is_due(&app(None, Some(now - secs(210)), 3), now, interval));
// Many failures: never longer than the interval.
assert!(is_due(&app(None, Some(now - secs(1770)), 30), now, interval));
}
#[test]
fn a_failure_older_than_the_last_fetch_is_ignored() {
let now = Utc::now();
let app = app(Some(now - secs(200)), Some(now - secs(300)), 5);
assert!(is_due(&app, now, Duration::from_secs(120)));
}
}
mod backoff {
use super::*;
#[test]
fn a_good_round_clears_the_backoff() {
let now = Utc::now();
let round = RoundResult { fetched: 1, unchanged: 3, failed: 1, ..RoundResult::default() };
let previous = Backoff { until: Some(now), trouble: 3 };
assert_eq!(backoff_after(&config(), &round, previous, now), Backoff::default());
}
#[test]
fn a_rate_limit_backs_off_the_fleet_until_after_the_reset() {
let now = Utc::now();
let round = RoundResult { rate_limited: Some(Some(now + chrono::Duration::minutes(50))), ..RoundResult::default() };
let b = backoff_after(&config(), &round, Backoff::default(), now);
assert_eq!(b.trouble, 1);
assert!(b.is_active(now + chrono::Duration::minutes(45)));
assert!(!b.is_active(now + chrono::Duration::minutes(57)));
}
#[test]
fn trouble_carries_over_between_rounds_and_instances() {
let now = Utc::now();
let round = RoundResult { failed: 2, ..RoundResult::default() };
let first = backoff_after(&config(), &round, Backoff::default(), now);
let second = backoff_after(&config(), &round, first, now);
assert_eq!((first.trouble, second.trouble), (1, 2));
assert!(second.until.unwrap() - now >= chrono::Duration::seconds(107), "{:?}", second);
}
#[test]
fn an_expired_backoff_is_not_active() {
let now = Utc::now();
assert!(!Backoff::default().is_active(now));
assert!(!Backoff { until: Some(now - secs(1)), trouble: 4 }.is_active(now));
assert!(Backoff { until: Some(now + secs(1)), trouble: 4 }.is_active(now));
}
}
mod waits {
use super::*;
#[test]
fn a_good_round_waits_the_interval_and_resets_trouble() {
let mut trouble = 3;
let round = RoundResult { fetched: 5, unchanged: 8, ..RoundResult::default() };
assert!(within(next_wait(&config(), &round, &mut trouble, Utc::now()), Duration::from_secs(1800)));
assert_eq!(trouble, 0);
}
#[test]
fn rate_limits_wait_for_the_reset_and_back_off_further_each_time() {
let now = Utc::now();
let mut trouble = 0;
// GitHub says come back in 50 minutes: wait that (plus margin), more than the interval.
let round = RoundResult { rate_limited: Some(Some(now + chrono::Duration::minutes(50))), ..RoundResult::default() };
assert!(within(next_wait(&config(), &round, &mut trouble, now), Duration::from_secs(51 * 60)));
// Again and again, with no reset time: 2×, 4×, … the interval, capped at the maximum.
let round = RoundResult { rate_limited: Some(None), ..RoundResult::default() };
assert!(within(next_wait(&config(), &round, &mut trouble, now), Duration::from_secs(3600)));
assert!(within(next_wait(&config(), &round, &mut trouble, now), Duration::from_secs(7200)));
for _ in 0..10 {
assert!(next_wait(&config(), &round, &mut trouble, now) <= Duration::from_secs(14_400).mul_f64(1.11));
}
}
#[test]
fn a_round_where_everything_failed_retries_sooner_but_not_in_a_tight_loop() {
let now = Utc::now();
let mut trouble = 0;
let round = RoundResult { failed: 13, ..RoundResult::default() };
assert!(within(next_wait(&config(), &round, &mut trouble, now), Duration::from_secs(60)));
assert!(within(next_wait(&config(), &round, &mut trouble, now), Duration::from_secs(120)));
for _ in 0..20 {
let w = next_wait(&config(), &round, &mut trouble, now);
assert!(w >= Duration::from_secs(53) && w <= Duration::from_secs(1800).mul_f64(1.11), "{:?}", w);
}
}
}
fn config() -> CraftAppsPollConfig {
CraftAppsPollConfig {
enabled: true,
coordination_enabled: true,
interval: Duration::from_secs(1800),
max_backoff: Duration::from_secs(14_400),
github_token: None,
apps: default_apps(),
}
}
fn app(fetched: Option<DateTime<Utc>>, failed: Option<DateTime<Utc>>, failures: u32) -> CachedCraftApp {
CachedCraftApp {
name: "PhotoCraft".into(),
repo: "storytold/photocraft".into(),
latest_release: None,
app_last_fetched: fetched,
etag: None,
last_failed_at: failed,
consecutive_failures: failures,
}
}
fn secs(n: i64) -> chrono::Duration {
chrono::Duration::seconds(n)
}
fn within(d: Duration, around: Duration) -> bool {
d >= around.mul_f64(0.89) && d <= around.mul_f64(1.11)
}
}
@@ -1,267 +0,0 @@
//! What the fleet shares in Redis about Craft App releases, and how it is encoded. Pure: the
//! Redis I/O is in `shared_store`.
//!
//! ## `craftApps:releases:app:<app key>` (hash, 7 day TTL, refreshed on every write)
//!
//! | Field | Meaning |
//! |--------------|--------------------------------------------------------------------------------|
//! | `v` | Format version, `1`. Entries with another version are ignored (and replaced) |
//! | `fetched_ms` | When the release was last fetched or confirmed by GitHub (Unix ms) |
//! | `etag` | GitHub's ETag for that answer; empty for none |
//! | `release` | The latest release as JSON (`CraftAppRelease`), `null` if the repo has none |
//! | `failed_ms` | When a request for the app last failed (Unix ms); stale once `fetched_ms` newer |
//! | `failures` | Failed requests in a row as of `failed_ms` |
//! | `claimed_ms` | When an instance last started a request for the app (Unix ms), to skip doubles |
//! | `by` | Hostname of the instance that wrote last, for debugging |
//!
//! ## `craftApps:releases:backoff` (hash, 1 day TTL; deleted after a good round)
//!
//! | Field | Meaning |
//! |------------|-----------------------------------------------------------------|
//! | `v` | Format version, `1` |
//! | `until_ms` | No instance asks GitHub before this time (Unix ms) |
//! | `trouble` | Rounds in a row that were rate limited or failed entirely |
//! | `by` | Hostname of the instance that wrote last, for debugging |
//!
//! Anything unreadable (wrong version, bad numbers, bad JSON) reads as absent, so it is
//! refetched and overwritten rather than trusted.
use std::collections::HashMap;
use chrono::{DateTime, Utc};
use artcraft_api_defs::craft_apps::release_info::CraftAppRelease;
use crate::threads::poll_craft_app_releases_thread::scheduling::Backoff;
pub const FORMAT_VERSION: &str = "1";
pub const FIELD_VERSION: &str = "v";
pub const FIELD_FETCHED_MS: &str = "fetched_ms";
pub const FIELD_ETAG: &str = "etag";
pub const FIELD_RELEASE: &str = "release";
pub const FIELD_FAILED_MS: &str = "failed_ms";
pub const FIELD_FAILURES: &str = "failures";
pub const FIELD_CLAIMED_MS: &str = "claimed_ms";
pub const FIELD_BY: &str = "by";
pub const FIELD_UNTIL_MS: &str = "until_ms";
pub const FIELD_TROUBLE: &str = "trouble";
/// One app's shared entry, as read from Redis.
#[derive(Clone, Debug, Default)]
pub struct SharedAppEntry {
pub fetched: Option<SharedFetch>,
pub failure: Option<SharedFailure>,
}
/// A successful fetch (or `304` confirmation) by some instance.
#[derive(Clone, Debug)]
pub struct SharedFetch {
pub latest_release: Option<CraftAppRelease>,
pub etag: Option<String>,
pub fetched_at: DateTime<Utc>,
}
/// The last failed request by some instance.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct SharedFailure {
pub failed_at: DateTime<Utc>,
pub consecutive_failures: u32,
}
/// Reads an app's hash. `None` for an empty hash or another format version.
pub fn decode_app_entry(fields: &HashMap<String, String>) -> Option<SharedAppEntry> {
if fields.get(FIELD_VERSION).map(String::as_str) != Some(FORMAT_VERSION) {
return None;
}
let fetched = decode_fetch(fields);
let failure = time_field(fields, FIELD_FAILED_MS).map(|failed_at| SharedFailure {
failed_at,
consecutive_failures: fields.get(FIELD_FAILURES).and_then(|f| f.parse().ok()).unwrap_or(1).max(1),
});
Some(SharedAppEntry { fetched, failure })
}
/// The fields written for a successful fetch.
pub fn encode_fetch(latest_release: Option<&CraftAppRelease>, etag: Option<&str>, fetched_at: DateTime<Utc>, by: &str) -> Result<Vec<(&'static str, String)>, serde_json::Error> {
Ok(vec![
(FIELD_VERSION, FORMAT_VERSION.to_string()),
(FIELD_FETCHED_MS, fetched_at.timestamp_millis().to_string()),
(FIELD_ETAG, etag.unwrap_or_default().to_string()),
(FIELD_RELEASE, serde_json::to_string(&latest_release)?),
(FIELD_BY, by.to_string()),
])
}
/// The fields written for a `304`: only the time moves.
pub fn encode_unchanged(fetched_at: DateTime<Utc>, by: &str) -> Vec<(&'static str, String)> {
vec![
(FIELD_VERSION, FORMAT_VERSION.to_string()),
(FIELD_FETCHED_MS, fetched_at.timestamp_millis().to_string()),
(FIELD_BY, by.to_string()),
]
}
/// The fields written for a failed request.
pub fn encode_failure(failure: SharedFailure, by: &str) -> Vec<(&'static str, String)> {
vec![
(FIELD_VERSION, FORMAT_VERSION.to_string()),
(FIELD_FAILED_MS, failure.failed_at.timestamp_millis().to_string()),
(FIELD_FAILURES, failure.consecutive_failures.to_string()),
(FIELD_BY, by.to_string()),
]
}
/// Reads the backoff hash; anything absent or unreadable is no backoff.
pub fn decode_backoff(fields: &HashMap<String, String>) -> Backoff {
if fields.get(FIELD_VERSION).map(String::as_str) != Some(FORMAT_VERSION) {
return Backoff::default();
}
Backoff {
until: time_field(fields, FIELD_UNTIL_MS),
trouble: fields.get(FIELD_TROUBLE).and_then(|t| t.parse().ok()).unwrap_or(0),
}
}
pub fn encode_backoff(backoff: Backoff, by: &str) -> Vec<(&'static str, String)> {
vec![
(FIELD_VERSION, FORMAT_VERSION.to_string()),
(FIELD_UNTIL_MS, backoff.until.map(|u| u.timestamp_millis()).unwrap_or(0).to_string()),
(FIELD_TROUBLE, backoff.trouble.to_string()),
(FIELD_BY, by.to_string()),
]
}
/// The fetch part of an app's hash; `None` unless the time, the ETag and the release all read.
fn decode_fetch(fields: &HashMap<String, String>) -> Option<SharedFetch> {
let fetched_at = time_field(fields, FIELD_FETCHED_MS)?;
let latest_release: Option<CraftAppRelease> = serde_json::from_str(fields.get(FIELD_RELEASE)?).ok()?;
let etag = fields.get(FIELD_ETAG)?;
Some(SharedFetch {
latest_release,
etag: (!etag.is_empty()).then(|| etag.clone()),
fetched_at,
})
}
/// A Unix-milliseconds field; `None` when absent, unreadable or zero.
fn time_field(fields: &HashMap<String, String>, name: &str) -> Option<DateTime<Utc>> {
let ms = fields.get(name)?.parse::<i64>().ok().filter(|ms| *ms > 0)?;
DateTime::from_timestamp_millis(ms)
}
#[cfg(test)]
mod tests {
use chrono::TimeZone;
use super::*;
const BY: &str = "storyteller-web-abc";
mod app_entries {
use super::*;
#[test]
fn a_fetch_round_trips() {
let release = release();
let at = time();
let fields = to_map(encode_fetch(Some(&release), Some("\"e1\""), at, BY).unwrap());
let entry = decode_app_entry(&fields).unwrap();
let fetched = entry.fetched.unwrap();
assert_eq!((fetched.etag.as_deref(), fetched.fetched_at), (Some("\"e1\""), at));
assert_eq!(json(&fetched.latest_release), json(&Some(release)));
assert_eq!(entry.failure, None);
}
#[test]
fn a_repo_without_release_or_etag_round_trips() {
let fields = to_map(encode_fetch(None, None, time(), BY).unwrap());
let fetched = decode_app_entry(&fields).unwrap().fetched.unwrap();
assert!(fetched.latest_release.is_none());
assert_eq!(fetched.etag, None);
}
#[test]
fn a_304_moves_only_the_time() {
let mut fields = to_map(encode_fetch(Some(&release()), Some("\"e1\""), time(), BY).unwrap());
let later = time() + chrono::Duration::minutes(2);
fields.extend(to_map(encode_unchanged(later, BY)));
let fetched = decode_app_entry(&fields).unwrap().fetched.unwrap();
assert_eq!((fetched.fetched_at, fetched.etag.as_deref()), (later, Some("\"e1\"")));
assert_eq!(json(&fetched.latest_release), json(&Some(release())));
}
#[test]
fn a_failure_round_trips_alongside_a_fetch() {
let mut fields = to_map(encode_fetch(None, None, time(), BY).unwrap());
let failure = SharedFailure { failed_at: time() + chrono::Duration::minutes(1), consecutive_failures: 3 };
fields.extend(to_map(encode_failure(failure, BY)));
let entry = decode_app_entry(&fields).unwrap();
assert_eq!(entry.failure, Some(failure));
assert!(entry.fetched.is_some());
}
#[test]
fn garbage_reads_as_absent() {
assert!(decode_app_entry(&HashMap::new()).is_none());
let good = to_map(encode_fetch(Some(&release()), Some("\"e1\""), time(), BY).unwrap());
let with = |field: &str, value: &str| {
let mut fields = good.clone();
fields.insert(field.to_string(), value.to_string());
fields
};
assert!(decode_app_entry(&with(FIELD_VERSION, "2")).is_none(), "unknown version");
assert!(decode_app_entry(&with(FIELD_RELEASE, "{not json")).unwrap().fetched.is_none());
assert!(decode_app_entry(&with(FIELD_RELEASE, "{\"tag\":1}")).unwrap().fetched.is_none());
assert!(decode_app_entry(&with(FIELD_FETCHED_MS, "yesterday")).unwrap().fetched.is_none());
assert!(decode_app_entry(&with(FIELD_FAILED_MS, "-5")).unwrap().failure.is_none());
}
}
mod backoffs {
use super::*;
#[test]
fn a_backoff_round_trips() {
let backoff = Backoff { until: Some(time()), trouble: 4 };
assert_eq!(decode_backoff(&to_map(encode_backoff(backoff, BY))), backoff);
}
#[test]
fn garbage_reads_as_no_backoff() {
assert_eq!(decode_backoff(&HashMap::new()), Backoff::default());
let mut fields = to_map(encode_backoff(Backoff { until: Some(time()), trouble: 4 }, BY));
fields.insert(FIELD_VERSION.into(), "zz".into());
assert_eq!(decode_backoff(&fields), Backoff::default());
fields.insert(FIELD_VERSION.into(), FORMAT_VERSION.into());
fields.insert(FIELD_UNTIL_MS.into(), "soon".into());
fields.insert(FIELD_TROUBLE.into(), "-1".into());
assert_eq!(decode_backoff(&fields), Backoff::default());
}
}
fn time() -> DateTime<Utc> {
Utc.with_ymd_and_hms(2026, 10, 10, 12, 0, 0).unwrap()
}
fn release() -> CraftAppRelease {
CraftAppRelease {
tag: "v0.5.0".into(),
version: "0.5.0".into(),
name: "PhotoCraft v0.5.0".into(),
published_at: Some(time()),
html_url: "https://github.com/storytold/photocraft/releases/tag/v0.5.0".into(),
prerelease: false,
notes_markdown: "Notes".into(),
assets: Vec::new(),
}
}
fn json(release: &Option<CraftAppRelease>) -> String {
serde_json::to_string(release).unwrap()
}
fn to_map(fields: Vec<(&'static str, String)>) -> HashMap<String, String> {
fields.into_iter().map(|(k, v)| (k.to_string(), v)).collect()
}
}
@@ -1,644 +0,0 @@
//! Reads and writes the fleet's shared Craft App release state in Redis (keys and fields are
//! documented in `shared_entry`).
//!
//! Writes are last-write-wins but guarded by time: a Lua script compares a timestamp field
//! before writing, so a slow instance never replaces a newer fetch with an older one. A `304`
//! moves only `fetched_ms`, and only if the stored ETag is still the one that was revalidated.
//!
//! Some Redis setups refuse scripts (scripting disabled, an ACL without `EVAL`, a proxy). The
//! first time Redis refuses the script, this store switches to applying the same rule itself:
//! it `WATCH`es the hash, reads and decodes the guarding fields, decides, and writes in a
//! `MULTI`/`EXEC` that Redis drops if another instance wrote in between (then it decides again).
//!
//! The pool is r2d2 (blocking), so every call runs on tokio's blocking pool with a timeout:
//! Redis trouble costs the polling loop a few seconds at most, never a stuck thread of the
//! async runtime. The poller gets its own small pool ([`build_poller_redis_pool`]) whose
//! connections time out reads and writes, so a hung Redis can neither strand blocking threads
//! nor take connections from the request handlers' pool.
use std::collections::HashMap;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, LazyLock};
use std::time::Duration;
use anyhow::anyhow;
use chrono::{DateTime, Utc};
use log::warn;
use r2d2::{ManageConnection, Pool};
use redis::{Client, Connection, ConnectionLike, ErrorKind, RedisError, RedisResult, Script, ServerErrorKind, Value};
use artcraft_api_defs::craft_apps::release_info::CraftAppRelease;
use redis_schema::keys::craft_apps::craft_app_release_redis_key::CraftAppReleaseRedisKey;
use redis_schema::keys::craft_apps::craft_apps_release_backoff_redis_key::CraftAppsReleaseBackoffRedisKey;
use crate::threads::poll_craft_app_releases_thread::scheduling::Backoff;
use crate::threads::poll_craft_app_releases_thread::shared_entry;
use crate::threads::poll_craft_app_releases_thread::shared_entry::{SharedAppEntry, SharedFailure};
/// Connections in the poller's pool: one per instance is all the loop uses at a time.
const POOL_MAX_SIZE: u32 = 2;
/// Longest wait for a pooled connection.
const CONNECTION_TIMEOUT: Duration = Duration::from_secs(3);
/// Longest wait for one read or write on a connection; after it, the connection gives up
/// instead of holding its blocking thread until the operating system notices a dead peer.
const IO_TIMEOUT: Duration = Duration::from_secs(5);
/// Longest wait for a whole call, connection included.
const CALL_TIMEOUT: Duration = Duration::from_secs(10);
/// How long a claim on an app keeps other instances from requesting it too: longer than one
/// request can take (10 s to connect, 30 s for the answer), plus a margin.
pub const CLAIM_WINDOW: Duration = Duration::from_secs(60);
/// Without scripts: how many times a guarded write decides again after another instance wrote
/// between its read and its write, before giving up (as if it lost the race).
const WATCH_ATTEMPTS: usize = 3;
/// Writes `ARGV[6..]` (field/value pairs) to the hash `KEYS[1]` and sets its TTL to `ARGV[4]` ms,
/// unless the hash's time field `ARGV[1]` already holds a value greater than
/// `ARGV[2] - ARGV[3]` (new time minus a minimum gap; a gap of 0 means "never go back in time"),
/// or `ARGV[5]` is non-empty and differs from the hash's `etag`. Returns 1 if written, else 0.
/// Unreadable stored times don't block the write, so garbage gets overwritten.
const GUARDED_HSET_LUA: &str = r#"
local current = tonumber(redis.call('HGET', KEYS[1], ARGV[1]))
if current and current + tonumber(ARGV[3]) > tonumber(ARGV[2]) then
return 0
end
if ARGV[5] ~= '' and redis.call('HGET', KEYS[1], 'etag') ~= ARGV[5] then
return 0
end
redis.call('HSET', KEYS[1], unpack(ARGV, 6))
redis.call('PEXPIRE', KEYS[1], ARGV[4])
return 1
"#;
static GUARDED_HSET: LazyLock<Script> = LazyLock::new(|| Script::new(GUARDED_HSET_LUA));
/// Handle on the shared state. Cheap to clone.
#[derive(Clone)]
pub struct SharedStore {
pool: PollerRedisPool,
/// Written as `by`, for debugging.
hostname: String,
/// Set once Redis refuses the guarded-write script; from then on guarded writes use `WATCH`.
scripts_refused: Arc<AtomicBool>,
}
/// The poller's Redis pool; see [`build_poller_redis_pool`].
pub type PollerRedisPool = Pool<PollerRedisConnections>;
/// Makes the poller's Redis connections, every one of them with I/O timeouts.
pub struct PollerRedisConnections {
client: Client,
}
/// A guarded write: which time field guards it, and what to write.
struct GuardedWrite {
key: String,
time_field: &'static str,
time: DateTime<Utc>,
min_gap: Duration,
required_etag: Option<String>,
fields: Vec<(&'static str, String)>,
}
impl SharedStore {
pub fn new(pool: PollerRedisPool, hostname: String) -> Self {
Self { pool, hostname, scripts_refused: Arc::new(AtomicBool::new(false)) }
}
/// The shared entries of `app_keys` (in order; `None` for absent or unreadable ones) and the
/// fleet's backoff, in one round trip.
pub async fn read(&self, app_keys: &[String]) -> anyhow::Result<(Vec<Option<SharedAppEntry>>, Backoff)> {
let keys: Vec<String> = app_keys.iter().map(|k| CraftAppReleaseRedisKey::new_for_app(k).to_string()).collect();
let mut hashes: Vec<HashMap<String, String>> = self
.run(move |conn| {
let mut pipe = redis::pipe();
for key in &keys {
pipe.hgetall(key);
}
pipe.hgetall(CraftAppsReleaseBackoffRedisKey::new_for_fleet().as_str());
pipe.query(conn)
})
.await?;
let backoff = hashes.pop().map(|fields| shared_entry::decode_backoff(&fields)).unwrap_or_default();
let entries = hashes.iter().map(shared_entry::decode_app_entry).collect();
Ok((entries, backoff))
}
/// Claims the next request for `app_key` at `now`. False when another instance claimed it
/// within [`CLAIM_WINDOW`]: it is asking GitHub already, so this one doesn't. Not a lock: it
/// is never released and simply lapses, so a crashed instance can't block anyone.
pub async fn claim(&self, app_key: &str, now: DateTime<Utc>) -> anyhow::Result<bool> {
self.guarded_write(GuardedWrite {
key: CraftAppReleaseRedisKey::new_for_app(app_key).to_string(),
time_field: shared_entry::FIELD_CLAIMED_MS,
time: now,
min_gap: CLAIM_WINDOW,
required_etag: None,
fields: vec![
(shared_entry::FIELD_VERSION, shared_entry::FORMAT_VERSION.to_string()),
(shared_entry::FIELD_CLAIMED_MS, now.timestamp_millis().to_string()),
(shared_entry::FIELD_BY, self.hostname.clone()),
],
}).await
}
/// Shares a successful fetch, unless a newer one is already shared. Returns whether written.
pub async fn write_fetched(&self, app_key: &str, latest_release: Option<&CraftAppRelease>, etag: Option<&str>, fetched_at: DateTime<Utc>) -> anyhow::Result<bool> {
self.guarded_write(GuardedWrite {
key: CraftAppReleaseRedisKey::new_for_app(app_key).to_string(),
time_field: shared_entry::FIELD_FETCHED_MS,
time: fetched_at,
min_gap: Duration::ZERO,
required_etag: None,
fields: shared_entry::encode_fetch(latest_release, etag, fetched_at, &self.hostname)?,
}).await
}
/// Shares a `304` for the release with ETag `etag`: moves only the fetch time, and only if
/// that release is still the shared one and nothing newer was shared. Returns whether written.
pub async fn write_unchanged(&self, app_key: &str, etag: &str, fetched_at: DateTime<Utc>) -> anyhow::Result<bool> {
if etag.is_empty() {
return Ok(false);
}
self.guarded_write(GuardedWrite {
key: CraftAppReleaseRedisKey::new_for_app(app_key).to_string(),
time_field: shared_entry::FIELD_FETCHED_MS,
time: fetched_at,
min_gap: Duration::ZERO,
required_etag: Some(etag.to_string()),
fields: shared_entry::encode_unchanged(fetched_at, &self.hostname),
}).await
}
/// Shares a failed request, unless a newer failure is already shared. Returns whether written.
pub async fn write_failure(&self, app_key: &str, failure: SharedFailure) -> anyhow::Result<bool> {
self.guarded_write(GuardedWrite {
key: CraftAppReleaseRedisKey::new_for_app(app_key).to_string(),
time_field: shared_entry::FIELD_FAILED_MS,
time: failure.failed_at,
min_gap: Duration::ZERO,
required_etag: None,
fields: shared_entry::encode_failure(failure, &self.hostname),
}).await
}
/// Shares the fleet's backoff; no trouble deletes it.
pub async fn write_backoff(&self, backoff: Backoff) -> anyhow::Result<()> {
let key = CraftAppsReleaseBackoffRedisKey::new_for_fleet().to_string();
if backoff.trouble == 0 {
return self.run(move |conn| redis::cmd("DEL").arg(&key).query::<()>(conn)).await;
}
let fields = shared_entry::encode_backoff(backoff, &self.hostname);
let ttl_ms = CraftAppsReleaseBackoffRedisKey::get_redis_ttl().num_milliseconds();
self
.run(move |conn| {
redis::pipe()
.hset_multiple(&key, &fields).ignore()
.pexpire(&key, ttl_ms).ignore()
.query::<()>(conn)
})
.await
}
/// Applies `write` with the script, or (once Redis has refused the script) with `WATCH`.
async fn guarded_write(&self, write: GuardedWrite) -> anyhow::Result<bool> {
let ttl_ms = CraftAppReleaseRedisKey::get_redis_ttl().num_milliseconds();
let scripts_refused = self.scripts_refused.clone();
self
.run(move |conn| {
if !scripts_refused.load(Ordering::Relaxed) {
match guarded_write_with_script(conn, &write, ttl_ms) {
Err(err) if is_script_refusal(&err) => {
if !scripts_refused.swap(true, Ordering::Relaxed) {
warn!("Craft app releases: Redis refused the guarded-write script ({err}); writing with WATCH/MULTI/EXEC instead.");
}
}
result => return result,
}
}
guarded_write_with_watch(conn, &write, ttl_ms)
})
.await
}
/// Runs `call` on a pooled connection, off the async runtime, within [`CALL_TIMEOUT`].
async fn run<T, F>(&self, call: F) -> anyhow::Result<T>
where
T: Send + 'static,
F: FnOnce(&mut Connection) -> RedisResult<T> + Send + 'static,
{
let pool = self.pool.clone();
let task = tokio::task::spawn_blocking(move || -> anyhow::Result<T> {
let mut conn = pool.get_timeout(CONNECTION_TIMEOUT)?;
Ok(call(&mut conn)?)
});
match tokio::time::timeout(CALL_TIMEOUT, task).await {
Ok(Ok(result)) => result,
Ok(Err(join_err)) => Err(anyhow!("redis task failed: {join_err}")),
Err(_) => Err(anyhow!("redis call timed out after {CALL_TIMEOUT:?}")),
}
}
}
/// The poller's own Redis pool, apart from the one the request handlers use. Built without
/// connecting, so an unreachable Redis at startup doesn't stop the server: the poller then
/// works alone until Redis answers.
pub fn build_poller_redis_pool(redis_url: &str) -> RedisResult<PollerRedisPool> {
let manager = PollerRedisConnections { client: Client::open(redis_url)? };
Ok(Pool::builder()
.max_size(POOL_MAX_SIZE)
.min_idle(Some(0))
.connection_timeout(CONNECTION_TIMEOUT)
.build_unchecked(manager))
}
impl ManageConnection for PollerRedisConnections {
type Connection = Connection;
type Error = RedisError;
/// Connects (handshake included) within [`CONNECTION_TIMEOUT`], then makes every later read
/// and write give up after [`IO_TIMEOUT`], before the connection is ever used.
fn connect(&self) -> RedisResult<Connection> {
let conn = self.client.get_connection_with_timeout(CONNECTION_TIMEOUT)?;
conn.set_read_timeout(Some(IO_TIMEOUT))?;
conn.set_write_timeout(Some(IO_TIMEOUT))?;
Ok(conn)
}
/// The pool's check before lending a connection; bounded by [`IO_TIMEOUT`] like any command.
fn is_valid(&self, conn: &mut Connection) -> RedisResult<()> {
redis::cmd("PING").query::<()>(conn)
}
fn has_broken(&self, conn: &mut Connection) -> bool {
!conn.is_open()
}
}
/// [`GUARDED_HSET_LUA`] decides and writes on the server, in one step.
fn guarded_write_with_script(conn: &mut Connection, write: &GuardedWrite, ttl_ms: i64) -> RedisResult<bool> {
let mut invocation = GUARDED_HSET.key(&write.key);
invocation
.arg(write.time_field)
.arg(write.time.timestamp_millis())
.arg(write.min_gap.as_millis() as u64)
.arg(ttl_ms)
.arg(write.required_etag.as_deref().unwrap_or_default());
for (field, value) in &write.fields {
invocation.arg(*field).arg(value);
}
let written: i64 = invocation.invoke(conn)?;
Ok(written == 1)
}
/// The same rule without scripts: `WATCH` the hash, read and decode its guarding fields, decide
/// with [`write_wins`], then write in a transaction. If another instance wrote the hash in
/// between, Redis drops the transaction and this decides again with what it wrote.
fn guarded_write_with_watch(conn: &mut Connection, write: &GuardedWrite, ttl_ms: i64) -> RedisResult<bool> {
for _ in 0..WATCH_ATTEMPTS {
redis::cmd("WATCH").arg(&write.key).query::<()>(conn)?;
let stored: RedisResult<(Option<String>, Option<String>)> = redis::cmd("HMGET")
.arg(&write.key)
.arg(write.time_field)
.arg(shared_entry::FIELD_ETAG)
.query(conn);
let (stored_time, stored_etag) = match stored {
Ok(stored) => stored,
Err(err) => {
// The connection goes back to the shared pool; don't leave it watching.
let _ = redis::cmd("UNWATCH").query::<()>(conn);
return Err(err);
}
};
if !write_wins(write, stored_time.as_deref(), stored_etag.as_deref()) {
redis::cmd("UNWATCH").query::<()>(conn)?;
return Ok(false);
}
// `EXEC` ends the watch either way; `None` means another write got in first.
let committed: Option<Value> = redis::pipe()
.atomic()
.hset_multiple(&write.key, &write.fields)
.pexpire(&write.key, ttl_ms)
.query(conn)?;
if committed.is_some() {
return Ok(true);
}
}
Ok(false)
}
/// Whether `write` may replace what the hash holds, by the script's rule: not if the stored time
/// plus the minimum gap is later than the new time, nor if an ETag is required and the stored
/// one differs. An unreadable stored time doesn't block the write, so garbage gets overwritten.
fn write_wins(write: &GuardedWrite, stored_time: Option<&str>, stored_etag: Option<&str>) -> bool {
let new_ms = write.time.timestamp_millis();
let gap_ms = write.min_gap.as_millis().min(i64::MAX as u128) as i64;
if let Some(current_ms) = stored_time.and_then(|t| t.parse::<i64>().ok()) {
if current_ms.saturating_add(gap_ms) > new_ms {
return false;
}
}
match write.required_etag.as_deref().filter(|etag| !etag.is_empty()) {
Some(required) => stored_etag == Some(required),
None => true,
}
}
/// Whether Redis answered that it won't run the script (as opposed to being unreachable, busy,
/// out of memory or read-only, which the `WATCH` path wouldn't get past either): an ACL
/// refusal, an unknown `EVALSHA`/`EVAL`/`SCRIPT` command, or scripting switched off.
fn is_script_refusal(err: &RedisError) -> bool {
if err.is_io_error() || err.is_timeout() || err.is_connection_dropped() || err.is_connection_refusal() {
return false;
}
match err.kind() {
ErrorKind::Server(ServerErrorKind::NoPerm) => true,
ErrorKind::Server(ServerErrorKind::ResponseError) | ErrorKind::Extension => {
if err.code() == Some("BUSY") {
return false;
}
let detail = err.detail().unwrap_or_default().to_ascii_lowercase();
detail.contains("unknown command") || detail.contains("eval") || detail.contains("script")
}
_ => false,
}
}
#[cfg(test)]
mod tests {
use std::io::{Read, Write};
use std::net::{TcpListener, TcpStream};
use chrono::TimeZone;
use url::Url;
use super::*;
const DEFAULT_TEST_REDIS_URL: &str = "redis://127.0.0.1:6379/";
/// Against a real Redis (`CRAFT_APPS_TEST_REDIS_URL`, default `redis://127.0.0.1:6379/`). Each
/// test uses app keys of its own, deletes them afterwards, and leaves the fleet backoff key
/// alone unless it was absent.
/// `cargo test -p storyteller-web --bin storyteller-web shared_store -- --ignored`
mod live_redis_tests {
use super::*;
#[tokio::test]
#[ignore = "needs a Redis server"]
async fn guarded_writes_with_the_script() {
let pool = test_pool(&test_redis_url());
let (pod_a, pod_b) = (SharedStore::new(pool.clone(), "pod-a".into()), SharedStore::new(pool, "pod-b".into()));
exercise_guarded_writes(&pod_a, &pod_b).await;
assert!(!pod_a.scripts_refused.load(Ordering::Relaxed));
}
#[tokio::test]
#[ignore = "needs a Redis server"]
async fn guarded_writes_with_watch() {
let pool = test_pool(&test_redis_url());
let (pod_a, pod_b) = (SharedStore::new(pool.clone(), "pod-a".into()), SharedStore::new(pool, "pod-b".into()));
pod_a.scripts_refused.store(true, Ordering::Relaxed);
pod_b.scripts_refused.store(true, Ordering::Relaxed);
exercise_guarded_writes(&pod_a, &pod_b).await;
}
/// A Redis user without scripting permissions: the first guarded write is refused, and the
/// store switches to `WATCH` and still applies every rule.
#[tokio::test]
#[ignore = "needs a Redis server that supports ACLs"]
async fn a_refused_script_falls_back_to_watch() {
let admin_url = test_redis_url();
let admin = test_pool(&admin_url);
let user = format!("craft-test-{}", rand::random::<u32>());
let password = format!("pw-{}", rand::random::<u64>());
let mut conn = admin.get().unwrap();
redis::cmd("ACL").arg("SETUSER").arg(&user).arg("reset").arg("on").arg(format!(">{password}")).arg("~*").arg("&*").arg("+@all").arg("-@scripting")
.query::<()>(&mut *conn).unwrap();
let mut url = Url::parse(&admin_url).unwrap();
url.set_username(&user).unwrap();
url.set_password(Some(&password)).unwrap();
let pool = test_pool(url.as_str());
let (pod_a, pod_b) = (SharedStore::new(pool.clone(), "pod-a".into()), SharedStore::new(pool, "pod-b".into()));
exercise_guarded_writes(&pod_a, &pod_b).await;
redis::cmd("ACL").arg("DELUSER").arg(&user).query::<()>(&mut *conn).unwrap();
assert!(pod_a.scripts_refused.load(Ordering::Relaxed), "the script should have been refused");
assert!(pod_b.scripts_refused.load(Ordering::Relaxed), "the script should have been refused");
}
}
/// An unreachable Redis fails fast, so the loop can fall back to polling on its own.
#[tokio::test]
async fn an_unreachable_redis_fails_fast() {
let pool = build_poller_redis_pool("redis://127.0.0.1:1/").unwrap();
let store = SharedStore::new(pool, "pod".into());
let started = std::time::Instant::now();
assert!(store.read(&["photocraft".to_string()]).await.is_err());
assert!(started.elapsed() < CALL_TIMEOUT + Duration::from_secs(1));
}
/// A Redis that completes the connection handshake and then stops answering (a wedged server,
/// a half-open connection): the call fails by itself, before [`CALL_TIMEOUT`] gives up on it,
/// so its blocking thread and connection are released rather than stranded until the
/// operating system notices. Without I/O timeouts the pool's `PING` would hang here forever.
#[tokio::test]
async fn a_redis_that_stops_answering_releases_the_blocking_thread() {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let url = format!("redis://{}/", listener.local_addr().unwrap());
std::thread::spawn(move || {
for stream in listener.incoming().flatten() {
std::thread::spawn(move || answer_handshake_then_go_silent(stream));
}
});
let store = SharedStore::new(build_poller_redis_pool(&url).unwrap(), "pod".into());
let started = std::time::Instant::now();
let err = store.read(&["photocraft".to_string()]).await.unwrap_err();
assert!(started.elapsed() < CALL_TIMEOUT, "{:?} after {:?}", err, started.elapsed());
}
/// [`write_wins`] must decide exactly as [`GUARDED_HSET_LUA`] does.
mod write_rule_tests {
use super::*;
#[test]
fn a_write_never_goes_back_in_time() {
let write = guarded_write_at(t(2), Duration::ZERO, None);
assert!(write_wins(&write, None, None), "nothing stored");
assert!(write_wins(&write, Some(&ms(t(1))), None), "newer than stored");
assert!(write_wins(&write, Some(&ms(t(2))), None), "same time, as the script allows");
assert!(!write_wins(&write, Some(&ms(t(3))), None), "older than stored");
}
#[test]
fn a_claim_waits_for_the_window() {
let claim = guarded_write_at(t(1), CLAIM_WINDOW, None);
assert!(!write_wins(&claim, Some(&ms(t(0) + chrono::Duration::seconds(30))), None));
assert!(write_wins(&claim, Some(&ms(t(0))), None), "exactly one window later");
}
#[test]
fn a_required_etag_must_match() {
let write = guarded_write_at(t(2), Duration::ZERO, Some("\"e2\""));
assert!(write_wins(&write, Some(&ms(t(1))), Some("\"e2\"")));
assert!(!write_wins(&write, Some(&ms(t(1))), Some("\"e1\"")), "another release is shared");
assert!(!write_wins(&write, Some(&ms(t(1))), None), "no release is shared");
assert!(write_wins(&guarded_write_at(t(2), Duration::ZERO, Some("")), None, None), "empty means none required");
}
#[test]
fn an_unreadable_stored_time_is_overwritten() {
let write = guarded_write_at(t(2), Duration::ZERO, None);
assert!(write_wins(&write, Some("soon"), None));
assert!(write_wins(&write, Some(""), None));
}
}
mod script_refusal_tests {
use redis::make_extension_error;
use super::*;
#[test]
fn redis_refusing_scripts_is_a_refusal() {
let refusals = [
server_error(ServerErrorKind::NoPerm, "this user has no permissions to run the 'evalsha' command"),
server_error(ServerErrorKind::ResponseError, "unknown command 'EVALSHA', with args beginning with: "),
server_error(ServerErrorKind::ResponseError, "This Redis instance has scripting disabled"),
make_extension_error("ERR".to_string(), Some("unknown command 'SCRIPT'".to_string())),
];
for err in refusals {
assert!(is_script_refusal(&err), "{:?}", err);
}
}
#[test]
fn redis_being_unavailable_is_not_a_refusal() {
let unavailable = [
RedisError::from(std::io::Error::new(std::io::ErrorKind::ConnectionRefused, "refused")),
RedisError::from(std::io::Error::new(std::io::ErrorKind::TimedOut, "timed out")),
make_extension_error("OOM".to_string(), Some("command not allowed when used memory > 'maxmemory'.".to_string())),
make_extension_error("BUSY".to_string(), Some("Redis is busy running a script.".to_string())),
server_error(ServerErrorKind::ReadOnly, "You can't write against a read only replica."),
server_error(ServerErrorKind::ResponseError, "wrong number of arguments for 'hset' command"),
];
for err in unavailable {
assert!(!is_script_refusal(&err), "{:?}", err);
}
}
fn server_error(kind: ServerErrorKind, detail: &str) -> RedisError {
RedisError::from((ErrorKind::Server(kind), "server error", detail.to_string()))
}
}
/// Every guarded-write rule, as two instances sharing one Redis see it.
async fn exercise_guarded_writes(pod_a: &SharedStore, pod_b: &SharedStore) {
let app = format!("test-craft-{}", rand::random::<u32>());
let other = format!("{app}-other");
let keys = vec![app.clone(), other.clone()];
let read_app = |store: SharedStore, keys: Vec<String>| async move { store.read(&keys).await.unwrap().0.remove(0) };
// Nothing yet.
let (entries, _) = pod_a.read(&keys).await.unwrap();
assert!(entries.iter().all(Option::is_none));
// Two instances tick together: only one gets to ask GitHub; the claim lapses after the window.
assert!(pod_a.claim(&app, t(0)).await.unwrap());
assert!(!pod_b.claim(&app, t(0) + chrono::Duration::seconds(5)).await.unwrap());
assert!(pod_b.claim(&app, t(2)).await.unwrap());
// A newer fetch is never replaced by an older one.
assert!(pod_a.write_fetched(&app, None, Some("\"e2\""), t(2)).await.unwrap());
assert!(!pod_b.write_fetched(&app, None, Some("\"e1\""), t(1)).await.unwrap());
let fetched = read_app(pod_a.clone(), keys.clone()).await.unwrap().fetched.unwrap();
assert_eq!((fetched.etag.as_deref(), fetched.fetched_at), (Some("\"e2\""), t(2)));
// A 304 moves the time only, and only for the release it revalidated.
assert!(!pod_b.write_unchanged(&app, "\"e1\"", t(3)).await.unwrap(), "stale ETag");
assert!(!pod_b.write_unchanged(&app, "\"e2\"", t(1)).await.unwrap(), "older than the shared fetch");
assert!(pod_b.write_unchanged(&app, "\"e2\"", t(3)).await.unwrap());
let fetched = read_app(pod_a.clone(), keys.clone()).await.unwrap().fetched.unwrap();
assert_eq!((fetched.etag.as_deref(), fetched.fetched_at), (Some("\"e2\""), t(3)));
// Failures: newest wins.
let failure = SharedFailure { failed_at: t(5), consecutive_failures: 2 };
assert!(pod_a.write_failure(&app, failure).await.unwrap());
assert!(!pod_b.write_failure(&app, SharedFailure { failed_at: t(4), consecutive_failures: 1 }).await.unwrap());
assert_eq!(read_app(pod_a.clone(), keys.clone()).await.unwrap().failure, Some(failure));
// Garbage in an entry is overwritten, not trusted.
let key = CraftAppReleaseRedisKey::new_for_app(&other).to_string();
let garbage_key = key.clone();
pod_a.run(move |conn| redis::cmd("HSET").arg(&garbage_key).arg("v").arg("1").arg("fetched_ms").arg("soon").arg("release").arg("{").query::<()>(conn)).await.unwrap();
assert!(pod_a.read(&keys).await.unwrap().0[1].as_ref().unwrap().fetched.is_none());
assert!(pod_a.write_fetched(&other, None, None, t(1)).await.unwrap());
assert!(pod_a.read(&keys).await.unwrap().0[1].as_ref().unwrap().fetched.is_some());
// Entries expire if nobody refreshes them.
let ttl_key = key.clone();
let ttl: i64 = pod_a.run(move |conn| redis::cmd("PTTL").arg(&ttl_key).query(conn)).await.unwrap();
assert!(ttl > 6 * 24 * 3600 * 1000, "{}", ttl);
// The fleet backoff round-trips (only touched when absent, to leave a real one alone).
let (_, existing) = pod_a.read(&keys).await.unwrap();
if existing == Backoff::default() {
let backoff = Backoff { until: Some(t(30)), trouble: 2 };
pod_a.write_backoff(backoff).await.unwrap();
assert_eq!(pod_b.read(&keys).await.unwrap().1, backoff);
pod_b.write_backoff(Backoff::default()).await.unwrap();
assert_eq!(pod_a.read(&keys).await.unwrap().1, Backoff::default());
}
let cleanup: Vec<String> = keys.iter().map(|k| CraftAppReleaseRedisKey::new_for_app(k).to_string()).collect();
pod_a.run(move |conn| redis::cmd("DEL").arg(&cleanup).query::<()>(conn)).await.unwrap();
}
fn guarded_write_at(time: DateTime<Utc>, min_gap: Duration, required_etag: Option<&str>) -> GuardedWrite {
GuardedWrite {
key: "craftApps:releases:app:test".to_string(),
time_field: shared_entry::FIELD_FETCHED_MS,
time,
min_gap,
required_etag: required_etag.map(str::to_string),
fields: Vec::new(),
}
}
/// Answers `+OK` to connection setup (`CLIENT SETINFO`, `SELECT`, `HELLO`), then keeps the
/// connection open without another word once a real command (`PING`, `HGETALL`, ...) arrives.
fn answer_handshake_then_go_silent(mut stream: TcpStream) {
let mut buf = [0u8; 4096];
loop {
let n = match stream.read(&mut buf) {
Ok(0) | Err(_) => return,
Ok(n) => n,
};
let chunk = String::from_utf8_lossy(&buf[..n]).to_ascii_uppercase();
if !(chunk.contains("CLIENT") || chunk.contains("SELECT") || chunk.contains("HELLO")) {
std::thread::sleep(Duration::from_secs(60));
return;
}
let commands = chunk.lines().filter(|line| line.starts_with('*')).count();
if stream.write_all("+OK\r\n".repeat(commands).as_bytes()).is_err() {
return;
}
}
}
fn test_redis_url() -> String {
std::env::var("CRAFT_APPS_TEST_REDIS_URL").unwrap_or_else(|_| DEFAULT_TEST_REDIS_URL.to_string())
}
fn test_pool(url: &str) -> PollerRedisPool {
build_poller_redis_pool(url).unwrap()
}
fn t(minute: u32) -> DateTime<Utc> {
Utc.with_ymd_and_hms(2026, 10, 10, 12, minute, 0).unwrap()
}
fn ms(time: DateTime<Utc>) -> String {
time.timestamp_millis().to_string()
}
}
+3 -1
View File
@@ -31,7 +31,7 @@ From the **repository root**, the launcher scripts (these preflight your
environment, install dependencies, and free the dev port first):
```bash
./script/website/unix_frontend_website_dev.sh # artcraft-website-next (marketing site), port 3000
./script/website/unix_frontend_website_dev.sh # artcraft-website (marketing site), port 4200
./script/website/unix_frontend_webapp_dev.sh # artcraft-webapp (user dashboard), port 4201
```
@@ -46,12 +46,14 @@ by its full or relative path:
Or directly from this directory:
```bash
nx dev artcraft-website
npx nx dev @frontend/artcraft-webapp
```
## Building
```bash
npx nx build artcraft-website
npx nx build artcraft-webapp
```
+18 -1
View File
@@ -4,6 +4,8 @@ import react from '@vitejs/plugin-react';
import wasm from 'vite-plugin-wasm';
import topLevelAwait from 'vite-plugin-top-level-await';
import { nxViteTsPaths } from '@nx/vite/plugins/nx-tsconfig-paths.plugin';
import { execSync } from 'child_process';
import path from 'path';
// Dev-only CDN proxy. Media assets (splats, GLBs, images) are fetched with
// fetch(), but the CDNs only send CORS headers for the production origin —
@@ -69,6 +71,21 @@ function cdnProxyPlugin() {
};
}
// Custom plugin to generate news.json on dev server start
function generateNewsPlugin() {
return {
name: 'generate-news',
buildStart() {
try {
const scriptPath = path.resolve(__dirname, '../../scripts/generate-news-json.mjs');
execSync(`node "${scriptPath}"`, { stdio: 'inherit' });
} catch (e) {
console.warn('Failed to generate news.json:', e);
}
},
};
}
export default defineConfig(() => ({
root: __dirname,
cacheDir: '../../node_modules/.vite/apps/artcraft-webapp',
@@ -93,7 +110,7 @@ export default defineConfig(() => ({
port: 4301,
host: 'localhost',
},
plugins: [cdnProxyPlugin(), nxViteTsPaths(), react(), wasm(), topLevelAwait()],
plugins: [cdnProxyPlugin(), generateNewsPlugin(), nxViteTsPaths(), react(), wasm(), topLevelAwait()],
// Uncomment this if you are using workers.
// worker: {
// plugins: [ nxViteTsPaths() ],
@@ -4,10 +4,6 @@
# Manage the build command and publish directory here; leave those UI fields empty.
# The Next.js runtime handles routing, the /cdn-media proxy rewrite, and the
# legacy-route redirects declared in next.config.ts. Other media is in public/.
#
# Google Analytics: set NEXT_PUBLIC_GA_MEASUREMENT_ID (G-XXXXXXXXXX) in the
# Netlify UI, scoped to the Production context so deploy previews don't
# report. It is inlined at build time, so redeploy after changing it.
[build]
command = "npm run build"
@@ -32,18 +32,6 @@ const nextConfig: NextConfig = {
destination: "/apps/vectorcraft",
permanent: true,
},
// PrintCraft was renamed PdfCraft. Old page links and share images
// (cached by social sites) follow it.
{
source: "/apps/printcraft",
destination: "/apps/pdfcraft",
permanent: true,
},
{
source: "/images/apps/printcraft/:file",
destination: "/images/apps/pdfcraft/:file",
permanent: true,
},
// Legacy share-link form (`/media?media=<token>`) from the Vite site;
// the canonical route is /media/<token>. Mirrored in netlify.toml so
// the edge answers it before the Next runtime.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 86 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 100 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 123 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 86 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 100 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 123 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 49 KiB

After

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 77 KiB

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 102 KiB

After

Width:  |  Height:  |  Size: 99 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 92 KiB

After

Width:  |  Height:  |  Size: 86 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

After

Width:  |  Height:  |  Size: 74 KiB

@@ -8,9 +8,7 @@ import {
import {
AppHeroDownloadButton,
AppHeroOtherDownloadsLink,
LauncherDownloadButton,
} from "@/components/apps/app-downloads";
import { craftLauncherRecommended } from "@/components/apps/launcher-callout";
import {
AppIcon,
AppTab,
@@ -81,10 +79,6 @@ export default async function CraftAppPage({ params }: { params: Params }) {
const name = craftAppName(app);
const release = craftAppRelease(app);
// The hero's download is ArtCraft Launcher, which installs this app and keeps
// it updated; the app's own installers are in #get-it, under the launcher.
const launcherDownloads = craftLauncherRecommended();
const hasDownload = launcherDownloads.length > 0 || release !== null;
const [hero] = app.shots;
const [before, accent, after] = app.headline;
const shareUrl = siteUrl(craftAppPath(app));
@@ -151,22 +145,18 @@ export default async function CraftAppPage({ params }: { params: Params }) {
mediaTag="Screenshot"
>
<div className="flex flex-col items-center gap-6">
<div className="flex flex-wrap items-start justify-center gap-3">
{launcherDownloads.length > 0 ? (
<LauncherDownloadButton recommended={launcherDownloads} fallbackHref="#get-it" />
) : (
release && (
<AppHeroDownloadButton
name={name}
recommended={craftReleaseDownloads(app, release).filter(
(download) => download.recommended,
)}
/>
)
<div className="flex flex-wrap justify-center gap-3">
{release && (
<AppHeroDownloadButton
name={name}
recommended={craftReleaseDownloads(app, release).filter(
(download) => download.recommended,
)}
/>
)}
<DiscordButton
size="lg"
variant={hasDownload ? "secondary" : "primary"}
variant={release ? "secondary" : "primary"}
>
Join the Discord
</DiscordButton>
@@ -181,7 +171,7 @@ export default async function CraftAppPage({ params }: { params: Params }) {
View on GitHub
</Button>
</div>
{hasDownload && <AppHeroOtherDownloadsLink />}
{release && <AppHeroOtherDownloadsLink />}
<p className="hud-label text-faint">
{app.platforms.join(" · ")} · Free and open source
</p>
@@ -9,8 +9,6 @@ import {
KeyboardIcon,
} from "lucide-react";
import AppCard from "@/components/apps/app-card";
import { LauncherDownloadButton } from "@/components/apps/app-downloads";
import { craftLauncherRecommended } from "@/components/apps/launcher-callout";
import ShareBar from "@/components/apps/share-bar";
import {
CampaignSection,
@@ -24,12 +22,6 @@ import { AppIcon, ColorAccent, appThemeClass } from "@/components/apps/app-wordm
import { PageHeader } from "@/components/page/page-header";
import RevealManager from "@/components/reveal-manager";
import { Button } from "@/components/ui";
import { trackAttrs } from "@/lib/analytics";
import {
CRAFT_LAUNCHER_NAME,
CRAFT_LAUNCHER_REPO,
craftLauncherReleasePageUrl,
} from "@/lib/craft-launcher";
import {
CRAFTING_APPS,
CRAFTING_APPS_COUNT_WORD,
@@ -72,7 +64,6 @@ const PRINCIPLE_ICONS = [
];
export default function CraftingAppsPage() {
const launcherDownloads = craftLauncherRecommended();
return (
<>
<RevealManager />
@@ -89,19 +80,8 @@ export default function CraftingAppsPage() {
}
lede={`${COVERAGE}. Native, open-source apps from the ArtCraft team, built in Rust and free to use.`}
>
<div data-reveal className="mt-8 flex flex-wrap items-start gap-3">
{launcherDownloads.length > 0 && (
<LauncherDownloadButton
recommended={launcherDownloads}
fallbackHref={craftLauncherReleasePageUrl()}
/>
)}
<DiscordButton
size="lg"
variant={launcherDownloads.length > 0 ? "secondary" : "primary"}
>
Join the Discord
</DiscordButton>
<div data-reveal className="mt-8 flex flex-wrap gap-3">
<DiscordButton size="lg">Join the Discord</DiscordButton>
<Button
href={CRAFTING_APPS_GITHUB_ORG}
variant="secondary"
@@ -113,21 +93,6 @@ export default function CraftingAppsPage() {
Browse on GitHub
</Button>
</div>
{launcherDownloads.length > 0 && (
<p data-reveal className="mt-4 max-w-xl leading-relaxed text-muted">
<span className="text-ink">Recommended:</span> {CRAFT_LAUNCHER_NAME}{" "}
installs every app below and keeps them all up to date. Each app
also has its own installers on its page.{" "}
<a
href={CRAFT_LAUNCHER_REPO}
target="_blank"
rel="noopener noreferrer"
className="text-ink underline decoration-line-strong underline-offset-4 hover:decoration-current"
>
Learn more
</a>
</p>
)}
<ShareBar
url={siteUrl("/apps")}
text={`Crafting Apps: ${CRAFTING_APPS_COUNT_WORD.toLowerCase()} open-source creative apps from ArtCraft`}
@@ -143,7 +108,6 @@ export default function CraftingAppsPage() {
key={app.slug}
href={craftAppPath(app)}
className={`group flex w-18 flex-col items-center gap-2 sm:w-20 ${appThemeClass(app)}`}
{...trackAttrs("app_select", { app_name: app.slug })}
>
<AppIcon
app={app}
@@ -321,14 +321,14 @@ html[data-ruler="on"]::-webkit-scrollbar {
.app-vectorcraft { --app: #e8573f; --app-ink: #b83a24; }
.app-filmcraft { --app: #8b5cf6; --app-ink: #6a3fd6; }
.app-lightcraft { --app: #f2a516; --app-ink: #8a5800; }
.app-pdfcraft { --app: #12a58a; --app-ink: #0a7563; }
.app-printcraft { --app: #12a58a; --app-ink: #0a7563; }
.app-effectcraft { --app: #e0368f; --app-ink: #b0206c; }
.app-designcraft { --app: #7bb51c; --app-ink: #457008; }
:root[data-theme="dark"] .app-photocraft { --app-ink: #7fb0ff; }
:root[data-theme="dark"] .app-vectorcraft { --app-ink: #ff8a70; }
:root[data-theme="dark"] .app-filmcraft { --app-ink: #b69cff; }
:root[data-theme="dark"] .app-lightcraft { --app-ink: #ffc14d; }
:root[data-theme="dark"] .app-pdfcraft { --app-ink: #3fd6b8; }
:root[data-theme="dark"] .app-printcraft { --app-ink: #3fd6b8; }
:root[data-theme="dark"] .app-effectcraft { --app-ink: #ff7ab8; }
:root[data-theme="dark"] .app-designcraft { --app-ink: #a6e04a; }
@media (prefers-color-scheme: dark) {
@@ -336,7 +336,7 @@ html[data-ruler="on"]::-webkit-scrollbar {
:root:not([data-theme="light"]) .app-vectorcraft { --app-ink: #ff8a70; }
:root:not([data-theme="light"]) .app-filmcraft { --app-ink: #b69cff; }
:root:not([data-theme="light"]) .app-lightcraft { --app-ink: #ffc14d; }
:root:not([data-theme="light"]) .app-pdfcraft { --app-ink: #3fd6b8; }
:root:not([data-theme="light"]) .app-printcraft { --app-ink: #3fd6b8; }
:root:not([data-theme="light"]) .app-effectcraft { --app-ink: #ff7ab8; }
:root:not([data-theme="light"]) .app-designcraft { --app-ink: #a6e04a; }
}
@@ -9,7 +9,6 @@ import SiteNav from "@/components/site-nav";
import SiteFooter from "@/components/site-footer";
import IntroConductor from "@/components/intro-conductor";
import LandingContext from "@/components/landing-context";
import Analytics from "@/components/analytics";
import MotionProvider from "@/components/motion-provider";
import ScrollRuler from "@/components/ruler/scroll-ruler";
import TunerPanel from "@/components/dev/tuner-panel";
@@ -139,7 +138,6 @@ export default function RootLayout({
</MotionProvider>
<TunerPanel />
<LandingContext />
<Analytics />
</body>
</html>
);
@@ -1,41 +0,0 @@
"use client";
import { useEffect } from "react";
import Script from "next/script";
import { GA_MEASUREMENT_ID, trackClick } from "@/lib/analytics";
// Loads gtag.js after hydration and installs the one delegated click
// listener behind the site's link and data-track events (lib/analytics.ts).
// Renders nothing when no measurement id is configured.
export default function Analytics() {
useEffect(() => {
if (!GA_MEASUREMENT_ID) return;
// Capture phase: runs before click-to-play facades swap themselves out,
// and before any handler can stop propagation. auxclick covers
// middle-click "open in new tab".
const onClick = (event: MouseEvent) => {
if (event.type === "auxclick" && event.button !== 1) return;
trackClick(event.target);
};
document.addEventListener("click", onClick, true);
document.addEventListener("auxclick", onClick, true);
return () => {
document.removeEventListener("click", onClick, true);
document.removeEventListener("auxclick", onClick, true);
};
}, []);
if (!GA_MEASUREMENT_ID) return null;
return (
<>
<Script
src={`https://www.googletagmanager.com/gtag/js?id=${GA_MEASUREMENT_ID}`}
strategy="afterInteractive"
/>
<Script id="ga-init" strategy="afterInteractive">
{`window.dataLayer=window.dataLayer||[];window.gtag=function(){dataLayer.push(arguments);};gtag("js",new Date());gtag("config",${JSON.stringify(GA_MEASUREMENT_ID)});`}
</Script>
</>
);
}
@@ -2,7 +2,6 @@ import Link from "next/link";
import { ArrowUpRightIcon } from "lucide-react";
import { twMerge } from "tailwind-merge";
import { Badge } from "@/components/ui";
import { trackAttrs } from "@/lib/analytics";
import {
craftAppName,
craftAppPath,
@@ -31,7 +30,6 @@ export default function AppCard({
<Link
href={craftAppPath(app)}
data-reveal
{...trackAttrs("app_select", { app_name: app.slug })}
className={twMerge(
"group relative flex h-full flex-col bg-bg",
appThemeClass(app),
@@ -9,7 +9,7 @@ import {
} from "lucide-react";
import { twMerge } from "tailwind-merge";
import { WindowsIcon } from "@/components/icons";
import { Button, type ButtonProps } from "@/components/ui";
import { Button } from "@/components/ui";
import {
CRAFT_DESKTOP_PLATFORMS,
type CraftDesktopPlatform,
@@ -35,17 +35,9 @@ const PLATFORM_ICONS: Record<CraftDesktopPlatform, ReactNode> = {
/**
* One card per desktop platform: its recommended build as the main button,
* then every other file for that platform. The visitor's platform is tagged
* "Your system" and gets the solid button. These are the app's standalone
* installers (the launcher above them installs the whole family), so each
* button says "Standalone for <platform>" and each file list names the app.
* "Your system" and gets the solid button.
*/
export function AppPlatformDownloads({
name,
downloads,
}: {
name: string;
downloads: CraftDownload[];
}) {
export function AppPlatformDownloads({ downloads }: { downloads: CraftDownload[] }) {
const detected = useDetectedDesktop();
return (
@@ -58,7 +50,7 @@ export function AppPlatformDownloads({
const others = files.filter((download) => download !== main);
return (
<article key={platform} data-reveal className="flex min-w-0 flex-col bg-bg">
<article key={platform} data-reveal className="flex flex-col bg-bg">
<div className="flex items-center justify-between gap-4 border-b border-line px-6 py-2.5 md:px-8">
<p className="hud-label text-muted">
{isDetected ? (
@@ -83,16 +75,14 @@ export function AppPlatformDownloads({
<Button
href={main.href}
variant={isDetected ? "primary" : "secondary"}
className="mt-6 w-full whitespace-normal text-center"
className="mt-6 w-full"
>
<ArrowDownToLineIcon aria-hidden className="h-4 w-4" />
Standalone for {platform}
Download for {platform}
</Button>
{others.length > 0 && (
<>
<p className="hud-label mt-8 text-faint">
Other {name} files for {platform}
</p>
<p className="hud-label mt-8 text-faint">Other {platform} downloads</p>
<ul className="mt-2 border-t border-line">
{others.map((download) => (
<li key={download.href} className="border-b border-line">
@@ -136,10 +126,7 @@ export function DownloadRow({ download, className }: { download: CraftDownload;
);
}
/**
* The app's own hero CTA, used only while ArtCraft Launcher has no release: a
* direct download once the OS is known, else a jump to #get-it.
*/
/** Hero CTA: a direct download once the OS is known, else a jump to #get-it. */
export function AppHeroDownloadButton({
recommended,
name,
@@ -157,49 +144,6 @@ export function AppHeroDownloadButton({
);
}
/**
* ArtCraft Launcher's CTA: a direct download for the visitor's system once
* it is known (with the file's platform and build underneath), else a jump
* to `fallbackHref`, where every platform is listed.
*/
export function LauncherDownloadButton({
recommended,
fallbackHref,
size = "lg",
variant = "primary",
fullWidth = false,
className,
}: {
recommended: CraftDownload[];
fallbackHref: string;
size?: ButtonProps["size"];
variant?: ButtonProps["variant"];
fullWidth?: boolean;
className?: string;
}) {
const detected = useDetectedDesktop();
const mine = detected && pickDownload(recommended, detected);
return (
<span className={twMerge("flex flex-col gap-2", className)}>
<Button
href={mine ? mine.href : fallbackHref}
size={size}
variant={variant}
className={fullWidth ? "w-full" : undefined}
>
<ArrowDownToLineIcon aria-hidden className="h-4 w-4" />
Download Launcher
</Button>
{/* Reserved line, so the layout doesn't shift when detection lands. It
takes the button's width rather than setting it (w-0 min-w-full), so
a long platform line can't push the buttons beside it away. */}
<span aria-hidden={!mine} className="hud-label min-h-4 w-0 min-w-full whitespace-nowrap text-faint">
{mine ? `For ${mine.group} · ${mine.label}` : "\u00a0"}
</span>
</span>
);
}
/**
* Shown under the hero once the direct download is live, so visitors on
* another machine, CPU or package format can still find their build.
@@ -6,8 +6,6 @@ import DiscordButton from "@/components/discord-button";
import { GitHubIcon } from "@/components/icons";
import { SectionShell, SectionEyebrow } from "@/components/landing/section-shell";
import { Badge, Button, CopyButton } from "@/components/ui";
import { trackAttrs } from "@/lib/analytics";
import { CRAFT_LAUNCHER_RELEASE } from "@/lib/crafting-app-releases";
import {
CRAFTING_APPS,
craftAppBuildCommand,
@@ -26,7 +24,6 @@ import {
import AppCard from "./app-card";
import { AppPlatformDownloads, DownloadRow } from "./app-downloads";
import { ColorAccent } from "./app-wordmark";
import { LauncherCallout } from "./launcher-callout";
// Body sections of an /apps/<slug> page, in page order. Server components;
// the client islands are the CopyButton and the OS-aware platform cards.
@@ -34,12 +31,11 @@ import { LauncherCallout } from "./launcher-callout";
const CELL_HEADING_CLASSES =
"mt-4 font-display text-3xl font-medium leading-[1.05] tracking-[-0.03em] text-ink-strong sm:text-4xl";
// The hero carries shot 01. Window captures pair up, and an odd one out
// leads at full width; portrait pages (layout spreads) sit four across.
// The hero carries shot 01. Window captures lead with a full-width shot,
// then pair; portrait pages (layout spreads) sit four across.
export function AppGallery({ app, index }: { app: CraftApp; index: string }) {
const [, ...shots] = app.shots;
const portrait = shots.every((shot) => shot.portrait);
const leadFullWidth = !portrait && shots.length % 2 === 1;
return (
<SectionShell id="gallery">
<SectionEyebrow
@@ -58,7 +54,7 @@ export function AppGallery({ app, index }: { app: CraftApp; index: string }) {
<figure
key={shot.file}
data-reveal
className={twMerge("bg-bg", leadFullWidth && i === 0 && "md:col-span-2")}
className={twMerge("bg-bg", !portrait && i === 0 && "md:col-span-2")}
>
<a
href={craftShotSourceUrl(app, shot)}
@@ -100,8 +96,7 @@ export function AppGallery({ app, index }: { app: CraftApp; index: string }) {
);
}
// ArtCraft Launcher first (installs and updates every app), then this app's
// own platform cards (your system first-class, every other build one click
// Platform cards (your system first-class, every other build one click
// away) above release details and build-from-source. Versions and files come
// from crafting-app-releases.ts; a null release there drops the cards and
// shows the Discord waitlist instead.
@@ -121,27 +116,9 @@ export function AppGetIt({ app, index }: { app: CraftApp; index: string }) {
annotation="Free · Open source"
/>
{release && (
<>
{CRAFT_LAUNCHER_RELEASE && (
<>
<LauncherCallout app={app} />
<div data-reveal className="border-y border-line bg-bg p-6 md:p-10">
<p className="hud-label text-faint">Option 2 · Standalone</p>
<h3 className={CELL_HEADING_CLASSES}>
Just {name}? Download it <ColorAccent>standalone</ColorAccent>.
</h3>
<p className="mt-4 max-w-xl leading-relaxed text-muted">
These installers add {name} on its own, without the
launcher. Come back here for new versions, or install the
launcher later and it will find {name} and keep it updated.
</p>
</div>
</>
)}
<div className="border-b border-line">
<AppPlatformDownloads name={name} downloads={downloads} />
</div>
</>
<div className="border-b border-line">
<AppPlatformDownloads downloads={downloads} />
</div>
)}
<div className="grid gap-px bg-line md:grid-cols-2">
<div data-reveal className="flex flex-col bg-bg p-6 md:p-10">
@@ -226,7 +203,6 @@ export function AppGetIt({ app, index }: { app: CraftApp; index: string }) {
value={command}
variant="ghost"
className="h-9 border-l border-line px-3"
{...trackAttrs("copy_command", { app_name: app.slug })}
/>
</div>
<pre className="overflow-x-auto p-4 font-mono text-[13px] leading-relaxed text-ink">
@@ -1,98 +0,0 @@
import { ArrowUpRightIcon, CheckIcon } from "lucide-react";
import {
CRAFT_LAUNCHER_NAME,
CRAFT_LAUNCHER_REPO,
craftLauncherDownloads,
craftLauncherReleasePageUrl,
} from "@/lib/craft-launcher";
import { CRAFT_LAUNCHER_RELEASE } from "@/lib/crafting-app-releases";
import { craftAppName, type CraftApp, type CraftDownload } from "@/lib/crafting-apps";
import { LauncherDownloadButton } from "./app-downloads";
import { ColorAccent } from "./app-wordmark";
// ArtCraft Launcher on the Crafting Apps pages: the "Recommended" callout
// that leads every app's Get-it section. The /apps hub header and every app
// page's hero use craftLauncherRecommended for their main download button. Server component;
// the download button is the client island.
const CALLOUT_POINTS = ["Every Crafting App", "One-click updates", "Verified downloads"];
/** The launcher's main download per platform, for the OS-aware buttons. */
export function craftLauncherRecommended(): CraftDownload[] {
return craftLauncherDownloads().filter((download) => download.recommended);
}
// Leads an app's Get-it section, above that app's own installers: the pitch
// beside its download panel, matching the release and build-from-source
// cells below it. Renders nothing while the launcher has no release.
export function LauncherCallout({ app }: { app: CraftApp }) {
const recommended = craftLauncherRecommended();
if (!CRAFT_LAUNCHER_RELEASE || recommended.length === 0) return null;
const name = craftAppName(app);
return (
<div
data-reveal
className="grid gap-px bg-line lg:grid-cols-[minmax(0,7fr)_minmax(0,4fr)]"
>
<div className="flex flex-col bg-bg p-6 md:p-10">
<div className="flex items-center gap-3">
<LauncherMark />
<p className="hud-label text-(--app-ink,var(--accent-ink))">
Option 1 · Recommended
</p>
</div>
<h3 className="mt-6 font-display text-3xl font-medium leading-[1.05] tracking-[-0.03em] text-ink-strong sm:text-4xl">
Get {name} with the <ColorAccent>Launcher</ColorAccent>.
</h3>
<p className="mt-4 max-w-xl leading-relaxed text-muted">
{CRAFT_LAUNCHER_NAME} installs {name} and every other Crafting App
from one place, picks the right build for your computer and keeps
them all up to date. Install it once and every update is a click
away.
</p>
<ul className="mt-6 flex flex-wrap gap-x-6 gap-y-2">
{CALLOUT_POINTS.map((point) => (
<li key={point} className="hud-label flex items-center gap-1.5 text-muted">
<CheckIcon aria-hidden className="h-3.5 w-3.5 text-(--app-ink,var(--accent-ink))" />
{point}
</li>
))}
</ul>
</div>
<div className="flex flex-col justify-center bg-bg p-6 md:p-10">
<p className="hud-label text-faint">
{CRAFT_LAUNCHER_NAME} · v{CRAFT_LAUNCHER_RELEASE.version}
</p>
<LauncherDownloadButton
recommended={recommended}
fallbackHref={craftLauncherReleasePageUrl()}
fullWidth
className="mt-4"
/>
<a
href={CRAFT_LAUNCHER_REPO}
target="_blank"
rel="noopener noreferrer"
className="hud-label mt-4 flex w-fit items-center gap-1.5 text-muted hover:text-ink"
>
About the Launcher
<ArrowUpRightIcon aria-hidden className="h-3.5 w-3.5" />
</a>
</div>
</div>
);
}
// The ArtCraft mark in a hairline square, like the platform icons in the
// download cards. Decorative — it always sits beside the launcher's name.
function LauncherMark() {
return (
<span
aria-hidden
className="flex h-10 w-10 shrink-0 items-center justify-center border border-line"
>
{/* eslint-disable-next-line @next/next/no-img-element */}
<img src="/artcraft-icon.svg" alt="" className="h-5 w-5" />
</span>
);
}
@@ -1,6 +1,5 @@
import { twMerge } from "tailwind-merge";
import { Button, CopyButton } from "@/components/ui";
import { trackAttrs } from "@/lib/analytics";
// Share strip: copy-link plus intent links for the networks our audience
// posts to. Hairline cells (gap-px over the line color) so it wraps cleanly
@@ -41,7 +40,6 @@ export default function ShareBar({
text: string;
className?: string;
}) {
const shareParams = { content_type: "page", item_id: new URL(url).pathname };
return (
<div
className={twMerge(
@@ -58,7 +56,6 @@ export default function ShareBar({
copiedLabel="Link copied"
variant="ghost"
className={CELL_CLASSES}
{...trackAttrs("share", { method: "copy_link", ...shareParams })}
/>
{TARGETS.map((target) => (
<Button
@@ -69,7 +66,6 @@ export default function ShareBar({
aria-label={`Share on ${target.name}`}
variant="ghost"
className={CELL_CLASSES}
{...trackAttrs("share", { method: target.name.toLowerCase(), ...shareParams })}
>
{target.name}
</Button>
@@ -4,7 +4,6 @@ import { useState, type FormEvent } from "react";
import { CheckIcon } from "lucide-react";
import { twMerge } from "tailwind-merge";
import { Button, Input, Label } from "@/components/ui";
import { trackEvent } from "@/lib/analytics";
import { webappUrl } from "@/lib/links";
// Submissions go to a Google Form whose responses land in a Google Sheet.
@@ -84,7 +83,6 @@ export default function BetaForm() {
// Google Forms sends no CORS headers, so this is fire-and-forget: the
// response is opaque and "request completed" counts as success.
await fetch(FORM_RESPONSE_URL, { method: "POST", mode: "no-cors", body });
trackEvent("generate_lead", { form_name: "beta", user_type: userType ?? "" });
setSubmitted(true);
} catch {
setSubmitError(
@@ -3,7 +3,6 @@
import { useState, type FormEvent } from "react";
import { CheckIcon, EyeIcon, EyeOffIcon } from "lucide-react";
import { Button, Input, Modal } from "@/components/ui";
import { trackEvent } from "@/lib/analytics";
import { login, signup } from "@/lib/api";
import {
getLandingUrl,
@@ -119,9 +118,8 @@ function SignupForm({
maybeReferralCode: getReferralCode(),
});
setLoading(false);
if (!result.success) return setError(result.errorMessage || "Failed to create account");
trackEvent("sign_up", { method: "email", signup_source: signupSource });
onSuccess();
if (result.success) onSuccess();
else setError(result.errorMessage || "Failed to create account");
};
return (
@@ -164,9 +162,8 @@ function LoginForm({ onSuccess }: { onSuccess: () => void }) {
setLoading(true);
const result = await login({ usernameOrEmail, password });
setLoading(false);
if (!result.success) return setError(result.errorMessage || "Invalid credentials");
trackEvent("login", { method: "email" });
onSuccess();
if (result.success) onSuccess();
else setError(result.errorMessage || "Invalid credentials");
};
return (
@@ -3,7 +3,6 @@
import { useState } from "react";
import { PlayIcon } from "lucide-react";
import { twMerge } from "tailwind-merge";
import { trackAttrs } from "@/lib/analytics";
import type { ExampleVideo } from "@/lib/campaign-data";
// Click-to-play facades: the mp4s (several MB each) are only requested
@@ -70,11 +69,6 @@ function ExampleCard({
onClick={() => setActive(true)}
aria-label={`Play example: ${example.label}`}
className="group absolute inset-0 h-full w-full"
{...trackAttrs("video_start", {
video_provider: "self_hosted",
video_title: example.label,
video_url: example.src,
})}
>
{/* eslint-disable-next-line @next/next/no-img-element */}
<img
@@ -9,7 +9,6 @@ import {
XIcon,
} from "lucide-react";
import { Button, Select, Switch } from "@/components/ui";
import { trackEvent } from "@/lib/analytics";
import { generateVideo, getVideoJob } from "@/lib/api";
import { H3, H3_GATE_PERKS, H3_SAMPLE_PROMPTS } from "@/lib/campaign-data";
import { webappUrl } from "@/lib/links";
@@ -102,7 +101,6 @@ export default function H3PromptBox() {
setSubmitting(false);
if (!result.success) {
patch(id, { status: "failed", error: result.errorMessage });
trackEvent("video_generation", { model: H3.modelId, status: "failed" });
return;
}
@@ -116,15 +114,12 @@ export default function H3PromptBox() {
if (job.status === "complete") {
patch(id, { status: "complete", videoUrl: job.videoUrl });
stops.current.delete(id);
trackEvent("video_generation", { model: H3.modelId, status: "complete" });
} else if (job.status === "failed") {
patch(id, { status: "failed", error: job.error });
stops.current.delete(id);
trackEvent("video_generation", { model: H3.modelId, status: "failed" });
} else if (attempts >= POLL_MAX_ATTEMPTS) {
patch(id, { status: "failed", error: "Generation timed out" });
stops.current.delete(id);
trackEvent("video_generation", { model: H3.modelId, status: "timeout" });
} else {
setTimeout(poll, POLL_INTERVAL_MS);
}
@@ -141,14 +136,6 @@ export default function H3PromptBox() {
setError(`Prompt exceeds the ${MAX_PROMPT_LENGTH} character limit for this model`);
return;
}
trackEvent("prompt_submit", {
model: H3.modelId,
logged_in: loggedIn,
aspect_ratio: aspectRatio,
resolution,
duration_seconds: Number(duration),
sound,
});
if (!loggedIn) {
setGateOpen(true);
return;
@@ -2,7 +2,6 @@
import { useState } from "react";
import { PlayIcon } from "lucide-react";
import { trackAttrs } from "@/lib/analytics";
// Click-to-load YouTube embed: a thumbnail until the visitor asks for the
// player, so the landing never pays the iframe cost up front.
@@ -33,11 +32,6 @@ export default function LiteYouTube({
onClick={() => setActivated(true)}
className="group absolute inset-0 h-full w-full"
aria-label={`Play video: ${title}`}
{...trackAttrs("video_start", {
video_provider: "youtube",
video_title: title,
video_url: `https://www.youtube.com/watch?v=${videoId}`,
})}
>
{/* eslint-disable-next-line @next/next/no-img-element */}
<img
@@ -4,7 +4,6 @@ import { useEffect, useRef, useState, type ReactNode } from "react";
import Link from "next/link";
import { ArrowDownToLineIcon, CheckIcon, CopyIcon, ExternalLinkIcon } from "lucide-react";
import { Button } from "@/components/ui";
import { trackEvent } from "@/lib/analytics";
import { MEDIA_LABELS, corsMediaUrl, mediaDetailLabel, mediaFormat, mediaKind, mediaThumbnail, safeMediaUrl, type MediaDimensions, type MediaPrompt, type SharedMedia } from "@/lib/media";
import { getModelDisplayName, getModelIcon, getProviderDisplayName, getProviderIcon } from "@/lib/model-names";
@@ -36,7 +35,6 @@ export default function MediaDetails({ media, prompt, promptLoading, dimensions
}
async function download() {
trackEvent("media_download", { media_type: kind, file_extension: mediaFormat(media) });
setDownloading(true);
setMessage("");
const controller = new AbortController();
@@ -104,10 +102,7 @@ export default function MediaDetails({ media, prompt, promptLoading, dimensions
<h2 className="hud-label text-faint">Prompt</h2>
{prompt?.maybe_positive_prompt && (
<button type="button" className="text-muted hover:text-ink" aria-label={copied === "prompt" ? "Prompt copied" : "Copy prompt"}
onClick={() => {
copy(prompt.maybe_positive_prompt!, "prompt");
trackEvent("copy_prompt", { media_type: kind });
}}>
onClick={() => copy(prompt.maybe_positive_prompt!, "prompt")}>
{copied === "prompt" ? <CheckIcon className="h-4 w-4" /> : <CopyIcon className="h-4 w-4" />}
</button>
)}
@@ -148,10 +143,7 @@ export default function MediaDetails({ media, prompt, promptLoading, dimensions
<Button onClick={download} loading={downloading} className="w-full">
<ArrowDownToLineIcon aria-hidden className="h-4 w-4" />{downloading ? "Downloading" : "Download file"}
</Button>
<Button variant="secondary" className="w-full" onClick={() => {
copy(`${SHARE_URL_BASE}${encodeURIComponent(media.token)}`, "link");
trackEvent("share", { method: "copy_link", content_type: kind, item_id: media.token });
}}>
<Button variant="secondary" className="w-full" onClick={() => copy(`${SHARE_URL_BASE}${encodeURIComponent(media.token)}`, "link")}>
{copied === "link" ? <CheckIcon aria-hidden className="h-4 w-4" /> : <CopyIcon aria-hidden className="h-4 w-4" />}
{copied === "link" ? "Link copied" : "Copy share link"}
</Button>
@@ -11,7 +11,6 @@ import {
import { twMerge } from "tailwind-merge";
import { SectionShell, SectionEyebrow } from "@/components/landing/section-shell";
import { Button, Modal } from "@/components/ui";
import { trackAttrs } from "@/lib/analytics";
import { mediaUrl } from "@/lib/links";
import {
PRESS_KIT_CATEGORIES,
@@ -149,10 +148,6 @@ function AssetCard({
onClick={onPlay}
aria-label={`Play ${asset.title}`}
className="group relative aspect-video w-full overflow-hidden bg-bg-sunken"
{...trackAttrs("video_start", {
video_provider: asset.type === "embed" ? "youtube" : "self_hosted",
video_title: asset.title,
})}
>
{thumbnail}
</button>
@@ -180,7 +175,6 @@ function AssetCard({
rel="noopener noreferrer"
variant="secondary"
className="w-full"
{...trackAttrs("press_kit_download", { asset_name: asset.title })}
>
<ArrowDownToLineIcon aria-hidden className="h-3.5 w-3.5" />
{asset.downloadLabel ?? "Download"}
@@ -3,7 +3,6 @@
import { useState } from "react";
import { CoinsIcon } from "lucide-react";
import { Button } from "@/components/ui";
import { trackEvent } from "@/lib/analytics";
import { webappUrl } from "@/lib/links";
import { useAccount } from "@/lib/use-account";
import CreditsModal from "./credits-modal";
@@ -29,10 +28,7 @@ export default function CreditsCta() {
<Button
type="button"
variant="secondary"
onClick={() => {
setOpen(true);
trackEvent("view_credit_packs", {});
}}
onClick={() => setOpen(true)}
className="mt-6"
>
<CoinsIcon aria-hidden className="h-3.5 w-3.5 text-accent-ink" />
@@ -4,7 +4,6 @@ import { useState } from "react";
import { CoinsIcon } from "lucide-react";
import { twMerge } from "tailwind-merge";
import { Button, Modal } from "@/components/ui";
import { trackEvent } from "@/lib/analytics";
import { creditsPackCheckout } from "@/lib/api";
type CreditPack = {
@@ -36,20 +35,6 @@ export default function CreditsModal({
const purchase = async (pack: CreditPack) => {
setPurchasingId(pack.id);
setError(null);
trackEvent("begin_checkout", {
currency: "USD",
value: pack.priceUsd,
checkout_type: "credits",
items: [
{
item_id: pack.id,
item_name: `${pack.total.toLocaleString("en-US")} credits`,
item_category: "credits",
price: pack.priceUsd,
quantity: 1,
},
],
});
const result = await creditsPackCheckout(pack.id);
if (result.success) {
window.location.href = result.data.checkoutUrl;
@@ -11,7 +11,6 @@ import {
} from "lucide-react";
import { twMerge } from "tailwind-merge";
import { Badge, Button, TabSelector } from "@/components/ui";
import { trackEvent } from "@/lib/analytics";
import {
getPortalUrl,
subscriptionCheckout,
@@ -86,26 +85,9 @@ export default function PricingTable({
if (plan.slug === activePlanSlug) return;
setProcessingPlan(plan.slug);
setError(null);
const checkoutType = !user ? "signup" : hasActivePlan ? "switch" : "subscribe";
const price = cadence === "yearly" ? plan.yearlyPrice : plan.monthlyPrice;
trackEvent("begin_checkout", {
currency: "USD",
value: price,
checkout_type: checkoutType,
items: [
{
item_id: plan.slug,
item_name: plan.name,
item_category: "subscription",
item_variant: cadence,
price,
quantity: 1,
},
],
});
const body = { plan: plan.slug, cadence };
const redirected = redirect(
checkoutType === "signup"
!user
? await userSignupSubscriptionCheckout({
...body,
maybeReferralUrl: getReferrer(),
@@ -113,7 +95,7 @@ export default function PricingTable({
maybeReferralUsername: getReferralUsername(),
maybeReferralCode: getReferralCode(),
})
: checkoutType === "switch"
: hasActivePlan
? await switchPlan(body)
: await subscriptionCheckout(body),
);
@@ -121,7 +103,6 @@ export default function PricingTable({
};
const managePlan = async () => {
trackEvent("manage_subscription", {});
setManaging(true);
setError(null);
if (!redirect(await getPortalUrl())) setManaging(false);
@@ -134,11 +115,7 @@ export default function PricingTable({
<TabSelector
tabs={BILLING_TABS}
activeTab={cadence}
onTabChange={(id) => {
if (id === cadence) return;
setCadence(id as BillingCadence);
trackEvent("billing_cadence_change", { cadence: id });
}}
onTabChange={(id) => setCadence(id as BillingCadence)}
tabClassName="w-24"
/>
<Badge
@@ -386,7 +363,6 @@ function ContactButtons({ className }: { className?: string }) {
try {
await navigator.clipboard.writeText(CONTACT_EMAIL);
setCopied(true);
trackEvent("contact_click", { method: "copy_email" });
setTimeout(() => setCopied(false), 1500);
} catch {
// Clipboard unavailable (insecure context) — the mailto still works.
@@ -86,7 +86,6 @@ export function Button({
</>
);
// Link buttons carry data-cta so analytics can report them as CTA clicks.
if (href) {
const anchorProps =
rest as unknown as AnchorHTMLAttributes<HTMLAnchorElement>;
@@ -97,7 +96,6 @@ export function Button({
target={target}
rel={rel}
className={classes}
data-cta=""
{...anchorProps}
>
{content}
@@ -105,7 +103,7 @@ export function Button({
);
}
return (
<Link href={href} className={classes} data-cta="" {...anchorProps}>
<Link href={href} className={classes} {...anchorProps}>
{content}
</Link>
);
@@ -1,232 +0,0 @@
// Google Analytics 4 (gtag.js). Off unless NEXT_PUBLIC_GA_MEASUREMENT_ID is
// set at build time, so local dev stays out of the reports.
//
// Page views come from GA4 Enhanced Measurement (it records App Router
// navigations via history changes), along with scrolls, outbound clicks and
// common file downloads. On top of that, the site sends the events below.
// Most link clicks need no per-component code: one delegated listener
// (see components/analytics.tsx) classifies them by href, and elements
// tagged with trackAttrs() report their event when clicked. Every delegated
// event also carries `link_location` (nav, footer, modal or the enclosing
// section id).
import { SOCIAL_LINKS, WEBAPP_URL } from "./links";
export const GA_MEASUREMENT_ID =
process.env.NEXT_PUBLIC_GA_MEASUREMENT_ID?.trim() || undefined;
export type AnalyticsItem = {
item_id: string;
item_name: string;
item_category: "subscription" | "credits";
item_variant?: string;
price: number;
quantity: number;
};
type AnalyticsEvents = {
// Link clicks, classified by href.
app_download: {
app_name: string;
app_version: string;
platform: "macos" | "windows" | "linux" | "other";
file_name: string;
};
webapp_click: { destination: string; link_text: string };
social_click: { network: string };
contact_click: { method: "mailto" | "copy_email" };
cta_click: { link_url: string; link_text: string };
// Clicks on elements tagged with trackAttrs().
video_start: { video_provider: string; video_title: string; video_url?: string };
share: { method: string; content_type: string; item_id: string };
copy_command: { app_name: string };
app_select: { app_name: string };
press_kit_download: { asset_name: string };
// Sent from component code.
generate_lead: { form_name: string; user_type: string };
sign_up: { method: "email"; signup_source: string };
login: { method: "email" };
prompt_submit: {
model: string;
logged_in: boolean;
aspect_ratio: string;
resolution: string;
duration_seconds: number;
sound: boolean;
};
video_generation: { model: string; status: "complete" | "failed" | "timeout" };
billing_cadence_change: { cadence: string };
view_credit_packs: Record<string, never>;
begin_checkout: {
currency: "USD";
value: number;
checkout_type: "signup" | "subscribe" | "switch" | "credits";
items: AnalyticsItem[];
};
manage_subscription: Record<string, never>;
copy_prompt: { media_type: string };
media_download: { media_type: string; file_extension: string };
};
export type AnalyticsEvent = keyof AnalyticsEvents;
type ParamValue = string | number | boolean | AnalyticsItem[] | undefined;
type EventParams = Record<string, ParamValue>;
declare global {
interface Window {
gtag?: (...args: unknown[]) => void;
}
}
export function trackEvent<E extends AnalyticsEvent>(
event: E,
params: AnalyticsEvents[E],
): void {
send(event, params);
}
/**
* Data attributes that make an element report `event` when clicked. Works in
* server components (no handler needed); spread onto the element or Button.
*/
export function trackAttrs<E extends AnalyticsEvent>(
event: E,
params: AnalyticsEvents[E],
): Record<`data-track${string}`, string> {
const attrs: Record<`data-track${string}`, string> = { "data-track": event };
for (const [key, value] of Object.entries(params)) {
if (value !== undefined) attrs[`data-track-${key}`] = String(value);
}
return attrs;
}
/** Delegated click handler: reports tagged elements and classified links. */
export function trackClick(target: EventTarget | null): void {
if (!(target instanceof Element)) return;
const tagged = target.closest<HTMLElement>("[data-track]");
if (tagged?.dataset.track) {
send(tagged.dataset.track, {
...taggedParams(tagged),
link_location: linkLocation(tagged),
});
return;
}
const anchor = target.closest<HTMLAnchorElement>("a[href]");
const link = anchor && classifyLink(anchor);
if (link) send(link.event, { ...link.params, link_location: linkLocation(anchor) });
}
function send(event: string, params: EventParams): void {
if (!GA_MEASUREMENT_ID) return;
window.gtag?.("event", event, params);
}
const WEBAPP_ORIGIN = new URL(WEBAPP_URL).origin;
// Hostname → network, derived from SOCIAL_LINKS.
const SOCIAL_NETWORKS = new Map(
Object.entries(SOCIAL_LINKS).map(([name, href]) => [
bareHostname(new URL(href).hostname),
name.toLowerCase(),
]),
);
// github.com/<org>/<repo>/releases/download/<tag>/<file>: the ArtCraft
// installers and every Crafting App download.
const RELEASE_ASSET_PATH = /^\/[^/]+\/([^/]+)\/releases\/download\/([^/]+)\/([^/]+)$/;
type ClassifiedLink = {
[E in AnalyticsEvent]: { event: E; params: AnalyticsEvents[E] };
}[AnalyticsEvent];
function classifyLink(anchor: HTMLAnchorElement): ClassifiedLink | null {
let url: URL;
try {
url = new URL(anchor.href);
} catch {
return null;
}
if (url.protocol === "mailto:") {
return { event: "contact_click", params: { method: "mailto" } };
}
const release = url.hostname === "github.com" && RELEASE_ASSET_PATH.exec(url.pathname);
if (release) {
const [, repo, tag, file] = release;
const fileName = decodeURIComponent(file);
return {
event: "app_download",
params: {
app_name: repo,
app_version: tag.replace(/^\D*/, ""),
platform: downloadPlatform(fileName),
file_name: fileName,
},
};
}
if (url.origin === WEBAPP_ORIGIN) {
return {
event: "webapp_click",
params: { destination: url.pathname, link_text: linkText(anchor) },
};
}
const network = SOCIAL_NETWORKS.get(bareHostname(url.hostname));
if (network) return { event: "social_click", params: { network } };
// Remaining Button links (internal CTAs, other outbound buttons). Plain
// text links are left to page views and Enhanced Measurement.
if (anchor.hasAttribute("data-cta")) {
return {
event: "cta_click",
params: {
link_url: url.origin === window.location.origin ? url.pathname + url.hash : url.href,
link_text: linkText(anchor),
},
};
}
return null;
}
function linkLocation(element: Element): string {
if (element.closest("#site-nav")) return "nav";
if (element.closest("footer")) return "footer";
if (element.closest('[role="dialog"]')) return "modal";
return element.closest("section[id]")?.id || "page";
}
// data-track-<param> attributes surface in the dataset as track<Param>.
function taggedParams(element: HTMLElement): EventParams {
const params: EventParams = {};
for (const [key, value] of Object.entries(element.dataset)) {
if (key.length > 5 && key.startsWith("track")) {
params[key.charAt(5).toLowerCase() + key.slice(6)] = value;
}
}
return params;
}
function downloadPlatform(fileName: string): AnalyticsEvents["app_download"]["platform"] {
if (/\.dmg$|macos/i.test(fileName)) return "macos";
if (/\.(exe|msi)$|windows/i.test(fileName)) return "windows";
if (/\.(appimage|deb|rpm)$|linux/i.test(fileName)) return "linux";
return "other";
}
// GA truncates parameter values at 100 characters.
function linkText(anchor: HTMLAnchorElement): string {
const text = anchor.getAttribute("aria-label") || anchor.textContent || "";
return text.replace(/\s+/g, " ").trim().slice(0, 100);
}
function bareHostname(hostname: string): string {
return hostname.replace(/^www\./, "");
}
@@ -1,31 +0,0 @@
import { CRAFT_LAUNCHER_RELEASE } from "./crafting-app-releases";
import {
CRAFTING_APPS_GITHUB_ORG,
craftReleaseTag,
releaseDownloads,
type CraftDownload,
} from "./crafting-apps";
// ArtCraft Launcher: one small native app that installs, updates and opens
// every Crafting App. The /apps hub's main button downloads it and every
// /apps/<slug> Get-it section recommends it above that app's own installers.
//
// Facts come from the craft-launcher README — keep them in sync. Version and
// files live in crafting-app-releases.ts (CRAFT_LAUNCHER_RELEASE); null there
// hides every launcher download.
export const CRAFT_LAUNCHER_NAME = "ArtCraft Launcher";
export const CRAFT_LAUNCHER_REPO = `${CRAFTING_APPS_GITHUB_ORG}/craft-launcher`;
/** Every launcher release file, or none while the release is null. */
export function craftLauncherDownloads(): CraftDownload[] {
return CRAFT_LAUNCHER_RELEASE
? releaseDownloads(CRAFT_LAUNCHER_REPO, "artcraft-launcher", CRAFT_LAUNCHER_RELEASE)
: [];
}
export function craftLauncherReleasePageUrl(): string {
return CRAFT_LAUNCHER_RELEASE
? `${CRAFT_LAUNCHER_REPO}/releases/tag/${craftReleaseTag(CRAFT_LAUNCHER_RELEASE)}`
: `${CRAFT_LAUNCHER_REPO}/releases`;
}
@@ -3,14 +3,17 @@ import type { CraftAppSlug } from "./crafting-apps";
// GitHub releases for the Crafting Apps, which drive every download button on
// the /apps pages.
//
// To bump an app: change its version in CRAFT_APP_RELEASES (each app has its
// own, e.g. `craftRelease("0.4.0")`).
// To bump every app: change CRAFTING_APPS_LATEST_VERSION.
// To pin one app to another version: give it its own release in
// CRAFT_APP_RELEASES, e.g. `craftRelease("0.1.1")`.
// To hide an app's downloads (Discord waitlist instead): set it to null.
// If a release renames, adds or drops files, edit CRAFT_RELEASE_ASSETS, or
// pass a custom list as craftRelease's second argument for a single app.
//
// Release tags are `v<version>` and live at github.com/storytold/<slug>.
export const CRAFTING_APPS_LATEST_VERSION = "0.2.0";
export type CraftDesktopPlatform = "macOS" | "Windows" | "Linux";
/** Order of the per-platform download buttons. */
@@ -19,7 +22,7 @@ export const CRAFT_DESKTOP_PLATFORMS: CraftDesktopPlatform[] = ["macOS", "Window
export type CraftReleaseAsset = {
/** Asset filename; `{slug}` and `{version}` are filled in per app. */
file: string;
/** Group in the full file list. "Other" holds the FreeBSD build, web build and checksums. */
/** Group in the full file list. "Other" holds the CLI, web build and checksums. */
group: CraftDesktopPlatform | "Other";
/** Short name, e.g. "Installer (x64)". */
label: string;
@@ -40,8 +43,6 @@ export type CraftRelease = {
version: string;
/** Git tag. Defaults to `v<version>`. */
tag?: string;
/** Fills `{slug}` in filenames. Defaults to the app's slug; a release from before a rename keeps the old one. */
fileSlug?: string;
assets: readonly CraftReleaseAsset[];
};
@@ -91,20 +92,6 @@ export const CRAFT_RELEASE_ASSETS: readonly CraftReleaseAsset[] = [
description: "Unzip and run on 32-bit Windows.",
arch: "x86",
},
{
file: "{slug}-{version}-windows-arm64.msi",
group: "Windows",
label: "Installer (ARM64)",
description: "Windows 11 on ARM laptops.",
arch: "aarch64",
},
{
file: "{slug}-{version}-windows-arm64-portable.zip",
group: "Windows",
label: "Portable (ARM64)",
description: "Unzip and run on Windows on ARM.",
arch: "aarch64",
},
{
file: "{slug}-{version}-linux-x86_64.AppImage",
group: "Linux",
@@ -113,13 +100,6 @@ export const CRAFT_RELEASE_ASSETS: readonly CraftReleaseAsset[] = [
arch: "x86_64",
recommended: true,
},
{
file: "{slug}-{version}-linux-x86_64.flatpak",
group: "Linux",
label: "Flatpak (x86_64)",
description: "Sandboxed. Install with flatpak install --user.",
arch: "x86_64",
},
{
file: "{slug}-{version}-linux-x86_64.deb",
group: "Linux",
@@ -149,13 +129,6 @@ export const CRAFT_RELEASE_ASSETS: readonly CraftReleaseAsset[] = [
arch: "aarch64",
recommended: true,
},
{
file: "{slug}-{version}-linux-aarch64.flatpak",
group: "Linux",
label: "Flatpak (ARM64)",
description: "Sandboxed, on 64-bit ARM.",
arch: "aarch64",
},
{
file: "{slug}-{version}-linux-aarch64.deb",
group: "Linux",
@@ -177,12 +150,6 @@ export const CRAFT_RELEASE_ASSETS: readonly CraftReleaseAsset[] = [
description: "Plain binaries for 64-bit ARM.",
arch: "aarch64",
},
{
file: "{slug}-{version}-freebsd-x86_64.tar.gz",
group: "Other",
label: "FreeBSD (x86_64)",
description: "Plain binaries for FreeBSD 14.",
},
{
file: "{slug}-web-{version}.zip",
group: "Other",
@@ -204,21 +171,15 @@ export function craftRelease(
return { version, assets };
}
// ArtCraft Launcher (github.com/storytold/craft-launcher) installs and updates
// every app above, and the /apps pages recommend it first. Its releases ship
// the same file set minus the macOS CLI, named artcraft-launcher-*.
export const CRAFT_LAUNCHER_RELEASE: CraftRelease | null = craftRelease(
"0.2.0",
CRAFT_RELEASE_ASSETS.filter((asset) => !asset.file.startsWith("{slug}-cli-")),
);
const LATEST = craftRelease(CRAFTING_APPS_LATEST_VERSION);
// Every app must be listed, so a new slug fails the type check until it is.
export const CRAFT_APP_RELEASES: Record<CraftAppSlug, CraftRelease | null> = {
photocraft: craftRelease("0.5.0"),
vectorcraft: craftRelease("0.7.0"),
filmcraft: craftRelease("0.4.0"),
lightcraft: craftRelease("0.4.0"),
pdfcraft: craftRelease("0.4.0"),
effectcraft: craftRelease("0.6.0"),
designcraft: craftRelease("0.4.0"),
photocraft: LATEST,
vectorcraft: LATEST,
filmcraft: LATEST,
lightcraft: LATEST,
printcraft: LATEST,
effectcraft: LATEST,
designcraft: LATEST,
};
@@ -25,7 +25,7 @@ export type CraftAppSlug =
| "vectorcraft"
| "filmcraft"
| "lightcraft"
| "pdfcraft"
| "printcraft"
| "effectcraft"
| "designcraft";
@@ -177,31 +177,25 @@ export const CRAFTING_APPS: CraftApp[] = [
file: "hero.webp",
source: "shot-1-neon.png",
caption: "Neon Drive — made in VectorCraft",
alt: "VectorCraft editing the Neon Drive poster, with the title's live outer glow settings in the Appearance panel and its character settings in the Properties panel",
alt: "VectorCraft editing a synthwave poster titled Neon Drive with an outer glow in the Appearance panel",
},
{
file: "ribbons.webp",
source: "shot-2-ribbons.png",
caption: "Live blends and layers",
alt: "VectorCraft showing three live blend ribbons clipped to the artboard, one selected with its key paths, and the Layers panel listing the clip group and blends",
},
{
file: "bezier.webp",
source: "shot-4-bezier.png",
caption: "Pen and direct selection",
alt: "VectorCraft direct-selecting the anchor points and bezier handles of a crescent moon, with the contextual task bar below it",
},
{
file: "perspective.webp",
source: "shot-5-perspective.png",
caption: "Perspective grid, still editable",
alt: "VectorCraft with two lit building facades drawn on the left and right planes of a two-point perspective grid at sunset",
caption: "Live blends between editable spines",
alt: "VectorCraft showing live blends: 70 smooth-color steps between two ribbon paths",
},
{
file: "sheet.webp",
source: "shot-3-sheet.png",
caption: "Booleans, mesh, repeat and envelope",
alt: "VectorCraft in its light theme with four artboards: shape booleans, a gradient mesh, a live radial repeat and an envelope distort",
alt: "VectorCraft feature sheet with exact booleans, a gradient mesh, a live radial repeat and an envelope distort",
},
{
file: "bezier.webp",
source: "shot-4-bezier.png",
caption: "Direct selection on bezier anchors",
alt: "VectorCraft direct-selecting the anchors and handles of a crescent moon path",
},
],
},
@@ -334,8 +328,8 @@ export const CRAFTING_APPS: CraftApp[] = [
],
},
{
slug: "pdfcraft",
prefix: "Pdf",
slug: "printcraft",
prefix: "Print",
category: "PDF workbench",
status: "Early alpha",
headline: ["The ", "open-source", " PDF workbench."],
@@ -373,27 +367,27 @@ export const CRAFTING_APPS: CraftApp[] = [
shots: [
{
file: "hero.webp",
source: "pdfcraft-viewer.png",
source: "printcraft-viewer.png",
caption: "Viewer with threaded comments",
alt: "PdfCraft showing a showcase PDF cover with the All tools panel and threaded comments",
alt: "PrintCraft showing a showcase PDF cover with the tools sidebar and threaded comments",
},
{
file: "organize.webp",
source: "pdfcraft-organize.png",
source: "printcraft-organize.png",
caption: "Organize pages like cards",
alt: "PdfCraft page organizer with three pages selected and the page tools in the toolbar above",
alt: "PrintCraft page organizer with three pages selected for rotate, delete and extract",
},
{
file: "palette.webp",
source: "pdfcraft-palette.png",
source: "printcraft-palette.png",
caption: "A command palette for every tool",
alt: "PdfCraft command palette searching for page and listing the page tools",
alt: "PrintCraft command palette filtering page tools by the word page",
},
{
file: "twoup.webp",
source: "pdfcraft-twoup.png",
source: "printcraft-twoup.png",
caption: "Two-up Read mode, dark theme",
alt: "PdfCraft two-up Read mode in the dark theme showing typeset specimen pages",
alt: "PrintCraft two-up Read mode in the dark theme showing typeset specimen pages",
},
],
},
@@ -652,19 +646,10 @@ export function craftReleaseDownloads(
app: CraftApp,
release: CraftRelease,
): CraftDownload[] {
return releaseDownloads(craftAppRepo(app), app.slug, release);
}
/** Every file in a release of `repo`, in catalog order; `{slug}` defaults to `fileSlug`. */
export function releaseDownloads(
repo: string,
fileSlug: string,
release: CraftRelease,
): CraftDownload[] {
const base = `${repo}/releases/download/${craftReleaseTag(release)}`;
const base = `${craftAppRepo(app)}/releases/download/${craftReleaseTag(release)}`;
return release.assets.map(({ file, ...asset }) => {
const fileName = file
.replaceAll("{slug}", release.fileSlug ?? fileSlug)
.replaceAll("{slug}", app.slug)
.replaceAll("{version}", release.version);
return { ...asset, fileName, href: `${base}/${fileName}` };
});
@@ -0,0 +1,4 @@
# Google OAuth Configuration
# Get the Client ID from the Google Cloud Console:
# https://console.cloud.google.com/apis/credentials
VITE_GOOGLE_CLIENT_ID=YOUR_CLIENT_ID.apps.googleusercontent.com
@@ -0,0 +1,143 @@
# Website → Webapp Migration
The marketing site (`artcraft-website`) is being split apart from the app
(`artcraft-webapp`). Anything that isn't pure marketing content belongs in
the webapp at `app.getartcraft.com`.
The migration is **gated by a single boolean** so it can be reverted while
we soak the new redirect behavior in production. When we're confident the
webapp is handling everything cleanly, do the **cleanup pass** below to
permanently delete everything that's only kept around for the rollback path.
---
## The switch
`src/config/links.ts`:
```ts
export const USE_WEBAPP_FOR_APP_FEATURES = true;
```
- `true` (default): legacy paths redirect to the webapp; navbar shows
"Launch App"; in-page links go cross-origin to `app.getartcraft.com`.
- `false`: everything runs locally on the marketing site (the original
pre-split behavior).
In dev, `WEBAPP_URL` auto-switches to `http://localhost:4201/` so cross-app
links work against the local webapp Vite server.
---
## What's behind the flag
### Pages that exist locally only as a fallback
When `USE_WEBAPP_FOR_APP_FEATURES = true`, these routes mount
`<WebappRedirect to="…" />` instead of the page component. The page files
are kept on disk so we can roll back by flipping the flag.
- `src/pages/login/`
- `src/pages/signup/`
- `src/pages/forgot-password/`
- `src/pages/create-image/`
- `src/pages/create-video/`
- `src/pages/create-vfx/` (the `/background-change` route)
- `src/pages/library/`
- `src/pages/referrals/`
- `src/pages/welcome/`
- `src/pages/onboarding/`
- `src/pages/checkout/` (success + cancel)
### Components only used by the above
- `src/components/auth/` (SignupForm, GoogleLoginButton)
- `src/components/credits-modal.tsx`
- `src/components/generation-gallery/`
- `src/components/lightbox/lightbox.tsx` (the modal `Lightbox`; the
`LightboxDetails` + `shared.ts` files in the same folder are still used
by the public `/media` page and must be kept)
- `src/components/navbar/task-queue.tsx`
- `src/components/prompt-box/`
- `src/components/settings-modal/`
### Lib files only used by the above
- `src/lib/cost-estimate-api.ts`
- `src/lib/enter-to-generate-store.ts`
- `src/lib/omni-gen-hooks.ts`
- `src/lib/prompts-cache.ts`
- `src/lib/recreate.ts`
### Files with conditional flag branches (need cleanup, not deletion)
- `src/app/app.tsx` — drop the `appOrWebapp()` helper, drop the local route
imports, leave only the `<WebappRedirect />` routes
- `src/components/navbar/navbar.tsx` — drop the `!USE_WEBAPP_FOR_APP_FEATURES`
branches (Login + Sign up buttons in desktop and mobile); keep only Launch
App for logged-in users and the Pricing + Launch App combo for logged-out
users
- `src/components/footer/footer.tsx` — drop the conditional Image/Video
links in the Pages column
- `src/components/download-modal.tsx` — drop the inline `SignupForm` flow
and the `view`/`navigate` state; keep only the webapp `/signup` and
`/login` link buttons
- `src/pages/landing3/landing3.tsx` — `appLink("/create-image")` /
`appLink("/create-video")` can become bare `webappUrl("…")` calls (or
inline `${WEBAPP_URL}create-image`)
- `src/pages/media/media.tsx` — drop the local
`applyRecreateFromMediaToken` branch and the `useNavigate` import; keep
only the `window.location.href = …` webapp redirect
- `src/pages/pricing/pricing.tsx` — the "Buy Credits" section + `CreditsModal`
is currently shown unconditionally; once `credits-modal.tsx` is deleted,
remove the section, the `creditsModalOpen` state, the `isLoggedIn` state,
and the session-check `useEffect`
### The flag itself
`src/config/links.ts` — drop `USE_WEBAPP_FOR_APP_FEATURES`, drop the
`appLink()` helper (use `webappUrl()` directly everywhere). Keep
`WEBAPP_URL`, `webappUrl()`, `SOCIAL_LINKS`, `SUPPORT_EMAIL`, and the dev
override.
---
## Cleanup pass — what to do when we're ready to commit to the webapp split
1. **Confirm the flag has been `true` in production for long enough** that
we're sure no real users hit the local routes.
2. **Delete** every directory and file listed under "Pages that exist
locally only as a fallback", "Components only used by the above", and
"Lib files only used by the above".
3. **Strip the flag branches** from each file under "Files with conditional
flag branches".
4. **Remove `USE_WEBAPP_FOR_APP_FEATURES`** and `appLink()` from
`src/config/links.ts`.
5. **Run** `npx tsc -p tsconfig.app.json --noEmit` from
`frontend/apps/artcraft-website` and fix any references the cleanup
exposed. (Pre-existing errors in `download-button.tsx` `LINUX` and
`pricing-table.tsx` `maybeReferralUsername` are unrelated and can be
ignored.)
6. **Verify** in dev with `nx serve @frontend/artcraft-website` that every
navbar/footer/landing/media flow still works and that legacy URLs still
redirect to the webapp via `WebappRedirect`.
---
## What stays on the marketing site no matter what
These are pure marketing content and are **not** behind the flag:
- `/` → `landing3`
- `/landing3`
- `/seedance-2` → `landing-sd2`
- `/download`
- `/media`, `/media/:id` (public viewer; the lightbox's "Recreate" button
does cross-origin to the webapp, but the viewer itself stays here)
- `/press-kit`
- `/tutorials`, `/tutorials/:slug`
- `/faq`, `/faq/:slug`
- `/news`, `/news/:slug`
- `/support`
- `/pricing` — Stripe checkout flow stays here (PricingTable → Stripe →
`/checkout/success` → WebappRedirect → webapp)
+157
View File
@@ -0,0 +1,157 @@
/background-change /index.html 200
/background-change/ /index.html 200
/background-change/* /index.html 200
/checkout_cancel /index.html 200
/checkout_cancel/ /index.html 200
/checkout_cancel/* /index.html 200
/onboarding /index.html 200
/onboarding/ /index.html 200
/onboarding/* /index.html 200
/portal_closed /index.html 200
/portal_closed/ /index.html 200
/portal_closed/* /index.html 200
/checkout_success /index.html 200
/checkout_success/ /index.html 200
/checkout_success/* /index.html 200
/checkout* /index.html 200
/checkout/* /index.html 200
/create-image /index.html 200
/create-image/ /index.html 200
/create-image/* /index.html 200
/create-video /index.html 200
/create-video/ /index.html 200
/create-video/* /index.html 200
/support /index.html 200
/support/ /index.html 200
/support/* /index.html 200
/library /index.html 200
/library/ /index.html 200
/library/* /index.html 200
/download /index.html 200
/download/ /index.html 200
/tutorials /index.html 200
/tutorials/ /index.html 200
/login /index.html 200
/login/ /index.html 200
/login/* /index.html 200
/media /index.html 200
/media/ /index.html 200
/media/* /index.html 200
# Prerendered: serve the per-route index.html so social crawlers get the
# correct OG/title tags (see scripts/prerender-meta.mjs). The SPA still boots
# from this file and React Router takes over for users.
/pricing /pricing/index.html 200
/pricing/ /pricing/index.html 200
/pricing/* /index.html 200
/beta /beta/index.html 200
/beta/ /beta/index.html 200
/beta/* /index.html 200
/faq /index.html 200
/faq/ /index.html 200
/faq/* /index.html 200
/news /index.html 200
/news/ /index.html 200
/news/* /index.html 200
/press-kit /index.html 200
/press-kit/ /index.html 200
/press-kit/* /index.html 200
/referrals /index.html 200
/referrals/ /index.html 200
/referrals/* /index.html 200
# Redirects for growth measurement
/reddit /?utm_source=reddit 301
/news.json /index.html 200
/signup /index.html 200
/signup/ /index.html 200
/signup/* /index.html 200
/forgot-password /index.html 200
/forgot-password/ /index.html 200
/forgot-password/* /index.html 200
/welcome /index.html 200
/welcome/ /index.html 200
/welcome/* /index.html 200
/password-reset /index.html 200
/password-reset/ /index.html 200
/password-reset/* /index.html 200
# Prerendered: exact paths serve their per-route index.html (correct OG tags
# for social crawlers); deep sub-paths fall back to the SPA shell. See
# scripts/prerender-meta.mjs. NOTE: these exact-match serving rules MUST come
# before the /seedance2* alias below, otherwise that wildcard would 301
# /seedance2-5 away to /seedance-2 (first match wins in _redirects).
/seedance2-5 /seedance2-5/index.html 200
/seedance2-5/ /seedance2-5/index.html 200
/seedance2-5/* /index.html 200
/seedance-2 /seedance-2/index.html 200
/seedance-2/ /seedance-2/index.html 200
/seedance-2/* /index.html 200
/minimax-h3 /minimax-h3/index.html 200
/minimax-h3/ /minimax-h3/index.html 200
/minimax-h3/* /index.html 200
/creators/jboogxcreative /creators/jboogxcreative/index.html 200
/creators/jboogxcreative/ /creators/jboogxcreative/index.html 200
/creators/* /index.html 200
# Similar URLs — creator spotlight aliases
/jboogx /creators/jboogxcreative 301
/jboogx/ /creators/jboogxcreative 301
/jboogxcreative /creators/jboogxcreative 301
/jboogxcreative/ /creators/jboogxcreative 301
# Similar URLs: MiniMax H3 aliases
/minimax /minimax-h3 301
/minimax/ /minimax-h3 301
/minimaxh3 /minimax-h3 301
/minimaxh3* /minimax-h3 301
/minimax-h-3 /minimax-h3 301
/minimax-h-3* /minimax-h3 301
/hailuo /minimax-h3 301
/hailuo/ /minimax-h3 301
# Google ads breakage
/seedance-2{ignore} /seedance-2 301
/seedance-2{ignore}* /seedance-2 301
/seedance-2%7Bignore%7D /seedance-2 301
/seedance-2%7Bignore%7D* /seedance-2 301
# Similar URLs — Seedance 2.5 aliases (kept above the broad /seedance2* rule)
/seedance-2-5 /seedance2-5 301
/seedance-2-5* /seedance2-5 301
/seedance-2.5 /seedance2-5 301
/seedance-2.5* /seedance2-5 301
/seedance25 /seedance2-5 301
/seedance25* /seedance2-5 301
# Similar URLs — Seedance 2.0 aliases
/seedance2 /seedance-2 301
/seedance2* /seedance-2 301
/seedance-2.0 /seedance-2 301
/seedance-2.0* /seedance-2 301
+52
View File
@@ -0,0 +1,52 @@
<!doctype html>
<html lang="en">
<head>
<!-- Google tag (gtag.js) -->
<script>
window.dataLayer = window.dataLayer || [];
function gtag() {
dataLayer.push(arguments);
}
if (window.location.pathname === "/login/desktop") {
const policy = document.createElement("meta");
policy.name = "referrer";
policy.content = "no-referrer";
document.head.appendChild(policy);
} else {
const analytics = document.createElement("script");
analytics.async = true;
analytics.src = "https://www.googletagmanager.com/gtag/js?id=G-VKW89949V0";
document.head.appendChild(analytics);
gtag("js", new Date());
gtag("config", "G-VKW89949V0");
}
</script>
<meta charset="utf-8" />
<title>ArtCraft. AI Video and Images. Fast and Open Desktop App.</title>
<meta
name="description"
content="ArtCraft is an Open Desktop app for generating AI Video and Images. You own ArtCraft!"
/>
<meta
name="keywords"
content="ArtCraft, AI, AI Video, AI Images, Open Source"
/>
<base href="/" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link rel="icon" type="image/x-icon" href="/favicon.ico" />
<link rel="stylesheet" href="/src/styles.css" />
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link
href="https://fonts.googleapis.com/css2?family=Bricolage+Grotesque:opsz,wght@12..96,200..800&family=Inter:ital,opsz,wght@0,14..32,100..900;1,14..32,100..900&family=Outfit:wght@100..900&family=Source+Sans+3:ital,wght@0,200..900;1,200..900&family=Instrument+Serif:ital@0;1&family=Geist+Mono:wght@300..800&display=swap"
rel="stylesheet"
/>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>
@@ -0,0 +1,29 @@
# Netlify settings used to discover this file:
# Base directory: frontend
# Package directory: apps/artcraft-website (relative to the base directory)
# Manage the build command and publish directory here; leave those UI fields empty.
# Paths below are relative to the frontend workspace.
[build]
command = "bash ./apps/artcraft-website/script/netlify_build.sh"
publish = "apps/artcraft-website/dist/"
[[headers]]
for = "/build/*"
[headers.values]
"Cache-Control" = "public, max-age=31560000, immutable"
# [[redirects]]
# from = "/ingest/static/*"
# to = "https://us-assets.i.posthog.com/static/:splat"
# host = "us-assets.i.posthog.com"
# status = 200
# force = true
#
# [[redirects]]
# from = "/ingest/*"
# to = "https://us.i.posthog.com/:splat"
# host = "us.i.posthog.com"
# status = 200
# force = true
@@ -0,0 +1,8 @@
{
"name": "@frontend/artcraft-website",
"version": "0.0.1",
"private": true,
"dependencies": {
"react-masonry-css": "^1.0.16"
}
}
@@ -0,0 +1,15 @@
const { join } = require('path');
// Note: If you use library-specific PostCSS/Tailwind configuration then you should remove the `postcssConfig` build
// option from your application's configuration (i.e. project.json).
//
// See: https://nx.dev/guides/using-tailwind-css-in-react#step-4:-applying-configuration-to-libraries
module.exports = {
plugins: {
tailwindcss: {
config: join(__dirname, 'tailwind.config.js'),
},
autoprefixer: {},
},
}
@@ -0,0 +1,13 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg id="Layer_2" data-name="Layer 2" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 116.34 97.5">
<defs>
<style>
.cls-1 {
fill: #4e7bfb;
}
</style>
</defs>
<g id="Layer_1-2" data-name="Layer 1">
<path class="cls-1" d="M104.28,49.49L81.55,0h-31.23l-3.17,4.76L14.75,53.63,0,75.85l21.55,21.55,63.79-36.94,16.99,37.04,14.01-21.74-12.06-26.27ZM32.89,65.66l32.42-48.87,10.91,23.77-43.32,25.09Z"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 483 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 662 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 199 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 463 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 181 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 MiB

Loaded 100 of 368 files, more files were not shown because too many files have changed in this diff. Show more