Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
aacf7f5a2f |
No files matched your search
@@ -34,6 +34,7 @@ artcraft-services/
|
||||
├── frontend/
|
||||
│ ├── apps/artcraft-webapp/ # Browser application at app.getartcraft.com
|
||||
│ ├── apps/artcraft-website-next/ # Current public website at getartcraft.com (Next.js)
|
||||
│ ├── apps/artcraft-website/ # Legacy product website (Vite)
|
||||
│ ├── apps/artcraft-desktop-website/ # Static landing page at desktop.getartcraft.com (Vite)
|
||||
│ └── libs/ # Shared React components and TypeScript libraries
|
||||
└── Cargo.toml # Rust workspace
|
||||
@@ -41,9 +42,9 @@ artcraft-services/
|
||||
|
||||
## Updating Desktop Download Links
|
||||
|
||||
Desktop release links are configured independently in **two places**. When
|
||||
Desktop release links are configured independently in **three places**. When
|
||||
updating the advertised ArtCraft release, update both `WINDOWS_VERSION` and
|
||||
`MAC_VERSION` in both unless the request explicitly targets one platform or
|
||||
`MAC_VERSION` in all three unless the request explicitly targets one platform or
|
||||
app:
|
||||
|
||||
1. **Current public website (Next.js):**
|
||||
@@ -51,7 +52,11 @@ app:
|
||||
This controls the installer links and visible version at
|
||||
`https://getartcraft.com/download`. Add the release to `DOWNLOAD_HISTORY`
|
||||
(newest first), then update both version selectors.
|
||||
2. **Browser webapp:**
|
||||
2. **Legacy website (Vite):**
|
||||
[frontend/apps/artcraft-website/src/config/github_download_links.ts](frontend/apps/artcraft-website/src/config/github_download_links.ts).
|
||||
Add the release to `DOWNLOAD_HISTORY` and update both version selectors.
|
||||
Updating only this file does **not** update the current public website.
|
||||
3. **Browser webapp:**
|
||||
[frontend/apps/artcraft-webapp/src/config/github_download_links.ts](frontend/apps/artcraft-webapp/src/config/github_download_links.ts).
|
||||
Update both version constants; these links are used by the welcome and
|
||||
checkout-success pages at `app.getartcraft.com`.
|
||||
@@ -60,7 +65,7 @@ Keep previous releases for rollback. The Next.js download page reads its visible
|
||||
version from `DOWNLOAD_VERSIONS`, which uses the same selectors as the URLs; do
|
||||
not hardcode a separate version in the page component.
|
||||
|
||||
Check that both configs produce the requested release tag and installer
|
||||
Check that all three configs produce the requested release tag and installer
|
||||
filenames for Windows (`ArtCraft_<version>_x64-setup.exe`) and macOS
|
||||
(`ArtCraft_<version>_universal.dmg`). Run `npm run typecheck` from
|
||||
`frontend/apps/artcraft-website-next` after changing its config. Repository edits
|
||||
@@ -84,11 +89,6 @@ report a local update as a deployed update.
|
||||
|
||||
Organize for top-to-bottom reading. Important things first, details later.
|
||||
|
||||
- **`mod.rs` holds module declarations only** (`pub mod foo;`, plus re-exports if needed). Never
|
||||
put functions, structs, enums, constants, impls or tests in a `mod.rs`: give each its own file
|
||||
in that directory (for example `threads/poll_foo_thread/poll_foo.rs`, declared from
|
||||
`threads/poll_foo_thread/mod.rs`).
|
||||
|
||||
- **Constants** at the top (after imports)
|
||||
- **Structs/enums** next; outer structs above inner sub-structs
|
||||
- **API types** in order: Request, Response, Error
|
||||
|
||||
@@ -104,7 +104,7 @@ and signals for state, Three.js for 3D scenes, and shared UI and generation tool
|
||||
| Path | Purpose |
|
||||
|----------------------------------|-----------------------------------------------|
|
||||
| `frontend/apps/artcraft-webapp` | Browser application at `app.getartcraft.com` |
|
||||
| `frontend/apps/artcraft-website-next` | Product website at `getartcraft.com` (Next.js) |
|
||||
| `frontend/apps/artcraft-website` | Product website at `getartcraft.com` |
|
||||
| `frontend/libs/api` | HTTP clients, API host selection, and models |
|
||||
| `frontend/libs/omni-gen` | Shared generation logic |
|
||||
| `frontend/libs/components` | Reusable UI, editors, and generation controls |
|
||||
@@ -190,11 +190,14 @@ The browser app runs at `http://localhost:4201`. `VITE_USE_LOCAL_API=true` selec
|
||||
the hosted API. API host selection lives in
|
||||
[`StorytellerApiHostStore`](./frontend/libs/api/src/lib/config/StorytellerApiHostStore.ts).
|
||||
|
||||
From `frontend`, build the web app with `npx nx build artcraft-webapp`. The
|
||||
product website is a standalone Next.js app (not part of the Nx workspace); run it
|
||||
from the repository root with `./script/website/unix_frontend_website_dev.sh`.
|
||||
From `frontend`, build the web app or run the product website with:
|
||||
|
||||
The website runs at `http://localhost:3000`. Repository-root launchers are in
|
||||
```bash
|
||||
npx nx build artcraft-webapp
|
||||
npx nx dev artcraft-website
|
||||
```
|
||||
|
||||
The website runs at `http://localhost:4200`. Repository-root launchers are in
|
||||
[`script/website`](./script/website). For desktop development, use the
|
||||
[ArtCraft desktop repository](https://github.com/storytold/artcraft).
|
||||
|
||||
|
||||
@@ -255,6 +255,7 @@ From the services repository:
|
||||
SQLX_OFFLINE=true cargo test --offline -p storyteller-web --bin storyteller-web login_challenges
|
||||
cd frontend
|
||||
npm exec vitest -- run --config apps/artcraft-webapp/vite.config.ts src/pages/login/desktop-login.spec.tsx src/pages/login/login-bridge-continuation.spec.tsx
|
||||
npm exec vitest -- run --config apps/artcraft-website/vite.config.ts src/pages/login/desktop-login.spec.tsx
|
||||
```
|
||||
|
||||
`SQLX_OFFLINE` controls compile-time query metadata only. These tests still execute
|
||||
|
||||
@@ -35,6 +35,6 @@ VITE_USE_LOCAL_API=true npx nx dev artcraft-webapp
|
||||
The web app runs at `http://localhost:4201`, using the local backend. To develop
|
||||
against the hosted API, omit `VITE_USE_LOCAL_API=true`.
|
||||
|
||||
For the product website, run `./script/website/unix_frontend_website_dev.sh` from the
|
||||
repository root; it is a standalone Next.js app on `http://localhost:3000`. Repository-root launchers live under [`script/website`](../script/website).
|
||||
For the product website, run `npx nx dev artcraft-website` from `frontend`; it uses
|
||||
`http://localhost:4200`. Repository-root launchers live under [`script/website`](../script/website).
|
||||
See the [frontend README](../frontend/README.md) for builds and troubleshooting.
|
||||
@@ -1 +0,0 @@
|
||||
pub mod release_info;
|
||||
@@ -1,203 +0,0 @@
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
use serde_derive::{Deserialize, Serialize};
|
||||
use url::Url;
|
||||
use utoipa::ToSchema;
|
||||
|
||||
// ── GET /v1/craft_apps/release_info ──
|
||||
//
|
||||
// The latest release of every Craft App (PhotoCraft, VectorCraft, …, and ArtCraft Launcher
|
||||
// itself), as storyteller-web last fetched it from GitHub, plus renames and notices for the
|
||||
// ArtCraft Launcher. The launcher polls this instead of GitHub; GitHub is its fallback.
|
||||
//
|
||||
// Compatibility: clients must ignore fields they don't know, and treat every field as optional.
|
||||
// New fields may be added; existing ones keep their meaning.
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
|
||||
pub struct GetCraftAppsReleaseInfoResponse {
|
||||
pub success: bool,
|
||||
|
||||
pub release_info: CraftAppsReleaseInfo,
|
||||
|
||||
/// Apps that changed their name (for example PdfCraft → DocuCraft).
|
||||
pub app_renames: Vec<CraftAppRename>,
|
||||
|
||||
/// Apps whose GitHub repository moved (for example `storytold/photocraft` → `art/photocraft`).
|
||||
pub repo_renames: Vec<CraftAppRepoRename>,
|
||||
|
||||
pub important_notices: CraftAppsImportantNotices,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Default, Serialize, Deserialize, ToSchema)]
|
||||
pub struct CraftAppsReleaseInfo {
|
||||
/// Keyed by app key: the app's lowercase machine name, as used in its binaries, asset file
|
||||
/// names and `ai.storyteller.<key>` bundle id (`photocraft`, `cadcraft`, `artcraft-launcher`).
|
||||
/// Not the display name and not the repo name. Clients should ignore keys they don't know.
|
||||
pub apps: BTreeMap<String, CraftAppReleaseInfo>,
|
||||
|
||||
/// When the least recently fetched app was fetched: every app's data is at least this fresh.
|
||||
/// Absent until every app has been fetched at least once.
|
||||
pub last_fetched: Option<DateTime<Utc>>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
|
||||
pub struct CraftAppReleaseInfo {
|
||||
/// Display name (`PhotoCraft`).
|
||||
pub name: String,
|
||||
|
||||
/// GitHub repository, `organization/project` (`storytold/photocraft`).
|
||||
pub repo: String,
|
||||
|
||||
/// The latest published, non-draft, non-prerelease release. Absent when the repository has no
|
||||
/// release yet, or when it hasn't been fetched yet (see `app_last_fetched`).
|
||||
pub latest_release: Option<CraftAppRelease>,
|
||||
|
||||
/// When this app's release data was last fetched successfully. Absent if it never was; then
|
||||
/// `latest_release` says nothing and clients should ask GitHub themselves.
|
||||
pub app_last_fetched: Option<DateTime<Utc>>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
|
||||
pub struct CraftAppRelease {
|
||||
/// Git tag (`v0.5.0`).
|
||||
pub tag: String,
|
||||
|
||||
/// The tag without its leading `v` (`0.5.0`, `0.6.0-rc.1`).
|
||||
pub version: String,
|
||||
|
||||
/// Release title (`PhotoCraft v0.5.0`).
|
||||
pub name: String,
|
||||
|
||||
pub published_at: Option<DateTime<Utc>>,
|
||||
|
||||
/// The release's page on GitHub.
|
||||
pub html_url: String,
|
||||
|
||||
pub prerelease: bool,
|
||||
|
||||
/// Release notes (Markdown), truncated to 64 KiB.
|
||||
pub notes_markdown: String,
|
||||
|
||||
pub assets: Vec<CraftAppReleaseAsset>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
|
||||
pub struct CraftAppReleaseAsset {
|
||||
/// File name (`photocraft-0.5.0-windows-x64.msi`).
|
||||
pub name: String,
|
||||
|
||||
pub size_bytes: u64,
|
||||
|
||||
/// `https://github.com/<repo>/releases/download/<tag>/<name>`.
|
||||
pub download_url: String,
|
||||
|
||||
/// Lowercase hex SHA-256, when GitHub published one for the file.
|
||||
pub sha256: Option<String>,
|
||||
|
||||
/// What the file name says it is, when it follows the Craft App naming scheme
|
||||
/// (`<app>-<version>-<os>-<arch>[-portable].<ext>`).
|
||||
pub platform: Option<CraftAppAssetPlatform>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
|
||||
pub struct CraftAppAssetPlatform {
|
||||
pub os: CraftAppOs,
|
||||
|
||||
/// Absent for files that aren't for one CPU (the web build, checksums).
|
||||
pub arch: Option<CraftAppArch>,
|
||||
|
||||
pub package_type: CraftAppPackageType,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, ToSchema)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum CraftAppOs {
|
||||
Windows,
|
||||
Macos,
|
||||
Linux,
|
||||
Freebsd,
|
||||
/// The browser build.
|
||||
Web,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, ToSchema)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum CraftAppArch {
|
||||
X64,
|
||||
X86,
|
||||
Arm64,
|
||||
/// macOS universal binary (Apple silicon and Intel).
|
||||
Universal,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, ToSchema)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum CraftAppPackageType {
|
||||
/// Windows Installer.
|
||||
Msi,
|
||||
/// Windows portable zip.
|
||||
PortableZip,
|
||||
/// macOS disk image.
|
||||
Dmg,
|
||||
/// macOS command-line tools zip.
|
||||
CliZip,
|
||||
AppImage,
|
||||
/// AppImage delta-update file.
|
||||
AppImageZsync,
|
||||
Deb,
|
||||
Rpm,
|
||||
Flatpak,
|
||||
/// Linux or FreeBSD tarball.
|
||||
TarGz,
|
||||
/// The web (WebAssembly) build.
|
||||
WebZip,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
|
||||
pub struct CraftAppRename {
|
||||
/// User-facing name before (`PdfCraft`).
|
||||
pub old_name: String,
|
||||
/// Lowercase name before (`pdfcraft`).
|
||||
pub old_name_lower: String,
|
||||
/// User-facing name after (`DocuCraft`).
|
||||
pub new_name: String,
|
||||
/// Lowercase name after (`docucraft`).
|
||||
pub new_name_lower: String,
|
||||
/// A sentence or two to show beside the rename.
|
||||
pub description: String,
|
||||
pub renamed_at: Option<DateTime<Utc>>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
|
||||
pub struct CraftAppRepoRename {
|
||||
/// `organization/project` before.
|
||||
pub old_name: String,
|
||||
/// `organization/project` after.
|
||||
pub new_name: String,
|
||||
pub renamed_at: Option<DateTime<Utc>>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Default, Serialize, Deserialize, ToSchema)]
|
||||
pub struct CraftAppsImportantNotices {
|
||||
/// News, shown inline; the user can dismiss each.
|
||||
pub announcements: Vec<CraftAppsAnnouncement>,
|
||||
/// Shown as a popup the user acknowledges.
|
||||
pub alerts: Vec<CraftAppsAlert>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
|
||||
pub struct CraftAppsAnnouncement {
|
||||
pub title: String,
|
||||
pub body: String,
|
||||
pub created_at: Option<DateTime<Utc>>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, ToSchema)]
|
||||
pub struct CraftAppsAlert {
|
||||
pub title: String,
|
||||
pub body: String,
|
||||
/// A page the popup can offer to open.
|
||||
pub maybe_web_link: Option<Url>,
|
||||
pub created_at: Option<DateTime<Utc>>,
|
||||
}
|
||||
@@ -2,7 +2,6 @@ pub mod analytics;
|
||||
pub mod api_keys;
|
||||
pub mod characters;
|
||||
pub mod common;
|
||||
pub mod craft_apps;
|
||||
pub mod credits;
|
||||
pub mod folders;
|
||||
pub mod generate;
|
||||
|
||||
@@ -1,41 +0,0 @@
|
||||
//! The latest release of one Craft App, shared by every storyteller-web instance so the fleet
|
||||
//! polls GitHub once per app instead of once per pod.
|
||||
//!
|
||||
//! A Redis hash, written by `poll_craft_app_releases_thread` (storyteller-web), which documents
|
||||
//! the fields. Expires if nobody refreshes it for a week, which only happens if polling stops.
|
||||
|
||||
use chrono::Duration;
|
||||
|
||||
pub struct CraftAppReleaseRedisKey(pub String);
|
||||
|
||||
impl_string_key!(CraftAppReleaseRedisKey);
|
||||
|
||||
const REDIS_KEY_TTL_DURATION: Duration = Duration::milliseconds(1000 * 60 * 60 * 24 * 7);
|
||||
|
||||
impl CraftAppReleaseRedisKey {
|
||||
/// `app_key` is the app's lowercase machine name (`photocraft`, `artcraft-launcher`).
|
||||
pub fn new_for_app(app_key: &str) -> Self {
|
||||
let key = format!("craftApps:releases:app:{}", app_key);
|
||||
Self(key)
|
||||
}
|
||||
|
||||
pub fn get_redis_ttl() -> Duration {
|
||||
REDIS_KEY_TTL_DURATION
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn test_new_for_app() {
|
||||
let key = CraftAppReleaseRedisKey::new_for_app("photocraft");
|
||||
assert_eq!(key.as_str(), "craftApps:releases:app:photocraft");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_duration() {
|
||||
assert_eq!(CraftAppReleaseRedisKey::get_redis_ttl().num_days(), 7);
|
||||
}
|
||||
}
|
||||
@@ -1,39 +0,0 @@
|
||||
//! The fleet-wide GitHub backoff for Craft App release polling: after GitHub rate-limits us (or
|
||||
//! every request of a round fails), no storyteller-web instance asks GitHub again until it ends.
|
||||
//!
|
||||
//! A Redis hash, written by `poll_craft_app_releases_thread` (storyteller-web), which documents
|
||||
//! the fields. It outlives the backoff itself so repeated trouble keeps growing the wait; it is
|
||||
//! deleted after a good round.
|
||||
|
||||
use chrono::Duration;
|
||||
|
||||
pub struct CraftAppsReleaseBackoffRedisKey(pub String);
|
||||
|
||||
impl_string_key!(CraftAppsReleaseBackoffRedisKey);
|
||||
|
||||
const REDIS_KEY_TTL_DURATION: Duration = Duration::milliseconds(1000 * 60 * 60 * 24);
|
||||
|
||||
impl CraftAppsReleaseBackoffRedisKey {
|
||||
pub fn new_for_fleet() -> Self {
|
||||
Self("craftApps:releases:backoff".to_string())
|
||||
}
|
||||
|
||||
pub fn get_redis_ttl() -> Duration {
|
||||
REDIS_KEY_TTL_DURATION
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn test_new_for_fleet() {
|
||||
assert_eq!(CraftAppsReleaseBackoffRedisKey::new_for_fleet().as_str(), "craftApps:releases:backoff");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_duration() {
|
||||
assert_eq!(CraftAppsReleaseBackoffRedisKey::get_redis_ttl().num_hours(), 24);
|
||||
}
|
||||
}
|
||||
@@ -1,2 +0,0 @@
|
||||
pub mod craft_app_release_redis_key;
|
||||
pub mod craft_apps_release_backoff_redis_key;
|
||||
@@ -1,3 +1,2 @@
|
||||
pub mod craft_apps;
|
||||
pub mod premium;
|
||||
pub mod inference_job;
|
||||
pub mod inference_job;
|
||||
@@ -1,11 +1,10 @@
|
||||
use std::str::FromStr;
|
||||
use std::time::Duration;
|
||||
|
||||
use log::info;
|
||||
|
||||
use errors::AnyhowResult;
|
||||
use shared_env_var_config::mysql::env_get_mysql_connection_string_or_default;
|
||||
use sqlx::mysql::{MySqlConnectOptions, MySqlPoolOptions};
|
||||
use sqlx::mysql::MySqlPoolOptions;
|
||||
use sqlx::MySqlPool;
|
||||
|
||||
pub async fn connect_to_database() -> AnyhowResult<MySqlPool> {
|
||||
@@ -21,37 +20,20 @@ pub async fn connect_to_database() -> AnyhowResult<MySqlPool> {
|
||||
// shorter timeout sheds load and surfaces the problem instead of hiding it behind long waits.
|
||||
let acquire_timeout_seconds = easyenv::get_env_num::<u64>("MYSQL_ACQUIRE_TIMEOUT_SECONDS", 10)?;
|
||||
|
||||
// NB: Every cached prepared statement stays open on the server until its connection evicts it
|
||||
// or closes, and the server caps the total across ALL clients (`max_prepared_stmt_count`,
|
||||
// 16382 on the managed DB). sqlx's default of 100 per connection let `replicas *
|
||||
// MYSQL_MAX_CONNECTIONS * 100` blow past that cap, failing new prepares with MySQL error 1461.
|
||||
// Keep `replicas * MYSQL_MAX_CONNECTIONS * MYSQL_STATEMENT_CACHE_CAPACITY` well under it.
|
||||
let statement_cache_capacity = easyenv::get_env_num::<usize>("MYSQL_STATEMENT_CACHE_CAPACITY", 25)?;
|
||||
|
||||
// NB: Recycling connections releases any server-side statements orphaned by cancelled queries
|
||||
// (eg. a client disconnect between prepare and close). sqlx's default lifetime is 30 minutes.
|
||||
let max_lifetime_seconds = easyenv::get_env_num::<u64>("MYSQL_MAX_LIFETIME_SECONDS", 600)?;
|
||||
|
||||
// NB: Log the settings before building/connecting so a connect failure has context. We do NOT
|
||||
// log the connection string (it contains credentials).
|
||||
info!(
|
||||
"Building MySQL pool: max_connections={}, min_connections={}, acquire_timeout_seconds={}, \
|
||||
statement_cache_capacity={}, max_lifetime_seconds={}",
|
||||
"Building MySQL pool: max_connections={}, min_connections={}, acquire_timeout_seconds={}",
|
||||
max_connections, min_connections, acquire_timeout_seconds,
|
||||
statement_cache_capacity, max_lifetime_seconds,
|
||||
);
|
||||
|
||||
let connect_options = MySqlConnectOptions::from_str(&db_connection_string)?
|
||||
.statement_cache_capacity(statement_cache_capacity);
|
||||
|
||||
let pool_options = MySqlPoolOptions::new()
|
||||
.max_connections(max_connections)
|
||||
.min_connections(min_connections)
|
||||
.acquire_timeout(Duration::from_secs(acquire_timeout_seconds))
|
||||
.max_lifetime(Some(Duration::from_secs(max_lifetime_seconds)));
|
||||
.acquire_timeout(Duration::from_secs(acquire_timeout_seconds));
|
||||
|
||||
let pool = pool_options
|
||||
.connect_with(connect_options)
|
||||
.connect(&db_connection_string)
|
||||
.await?;
|
||||
|
||||
Ok(pool)
|
||||
|
||||
@@ -162,7 +162,6 @@ use artcraft_api_defs::moderation::user_daily_spends::user_daily_spends_list::*;
|
||||
use artcraft_api_defs::moderation::top_spenders::list::*;
|
||||
use artcraft_api_defs::moderation::user_spend_events::list::*;
|
||||
use artcraft_api_defs::folders::subfolder::*;
|
||||
use artcraft_api_defs::craft_apps::release_info::*;
|
||||
use artcraft_api_defs::tags::add_media_file_tags::*;
|
||||
use artcraft_api_defs::tags::bulk_add_tags::*;
|
||||
use artcraft_api_defs::tags::bulk_list_media_file_tags::*;
|
||||
@@ -735,9 +734,6 @@ use crate::http_server::endpoints::media_files::list::list_batch_generated_redux
|
||||
crate::http_server::endpoints::folders::media_files::bulk_move_folder_media_files_handler::bulk_move_folder_media_files_handler,
|
||||
crate::http_server::endpoints::folders::media_files::bulk_remove_folder_media_files_handler::bulk_remove_folder_media_files_handler,
|
||||
|
||||
// Craft Apps
|
||||
crate::http_server::endpoints::craft_apps::get_craft_apps_release_info_handler::get_craft_apps_release_info_handler,
|
||||
|
||||
// Tags
|
||||
crate::http_server::endpoints::tags::list_tags_handler::list_tags_handler,
|
||||
crate::http_server::endpoints::tags::list_media_file_tags_handler::list_media_file_tags_handler,
|
||||
@@ -1610,20 +1606,6 @@ use crate::http_server::endpoints::media_files::list::list_batch_generated_redux
|
||||
ListTaggedMediaFilesSuccessResponse,
|
||||
ListTagsQueryParams,
|
||||
ListTagsSuccessResponse,
|
||||
GetCraftAppsReleaseInfoResponse,
|
||||
CraftAppsReleaseInfo,
|
||||
CraftAppReleaseInfo,
|
||||
CraftAppRelease,
|
||||
CraftAppReleaseAsset,
|
||||
CraftAppAssetPlatform,
|
||||
CraftAppOs,
|
||||
CraftAppArch,
|
||||
CraftAppPackageType,
|
||||
CraftAppRename,
|
||||
CraftAppRepoRename,
|
||||
CraftAppsImportantNotices,
|
||||
CraftAppsAnnouncement,
|
||||
CraftAppsAlert,
|
||||
ListUntaggedMediaFilesQueryParams,
|
||||
ListUntaggedMediaFilesSuccessResponse,
|
||||
MediaFileTagsEntry,
|
||||
|
||||
@@ -1,73 +0,0 @@
|
||||
use std::sync::Arc;
|
||||
|
||||
use actix_web::http::header::{CacheControl, CacheDirective};
|
||||
use actix_web::{web, HttpResponse};
|
||||
|
||||
use artcraft_api_defs::craft_apps::release_info::{CraftAppsImportantNotices, CraftAppsReleaseInfo, GetCraftAppsReleaseInfoResponse};
|
||||
|
||||
use crate::state::server_state::ServerState;
|
||||
|
||||
/// Shared caches (Cloudflare) may serve the same answer for this long once every app has been
|
||||
/// fetched: the data changes at most once per polling interval.
|
||||
const CACHE_MAX_AGE_SECS: u32 = 60;
|
||||
|
||||
/// The latest release of every Craft App (and ArtCraft Launcher), as storyteller-web last fetched
|
||||
/// it from GitHub, plus app/repo renames and notices for the launcher. Never fails: before the
|
||||
/// first fetch, apps are listed without a release and without `app_last_fetched`.
|
||||
#[utoipa::path(
|
||||
get,
|
||||
tag = "Craft Apps",
|
||||
path = "/v1/craft_apps/release_info",
|
||||
responses(
|
||||
(status = 200, body = GetCraftAppsReleaseInfoResponse),
|
||||
),
|
||||
)]
|
||||
pub async fn get_craft_apps_release_info_handler(
|
||||
server_state: web::Data<Arc<ServerState>>,
|
||||
) -> HttpResponse {
|
||||
let release_info = server_state.craft_apps_release_cache.snapshot();
|
||||
let cache_control = cache_control_for(&release_info);
|
||||
let response = GetCraftAppsReleaseInfoResponse {
|
||||
success: true,
|
||||
release_info,
|
||||
// None yet. When there are, they're listed here (newest first).
|
||||
app_renames: Vec::new(),
|
||||
repo_renames: Vec::new(),
|
||||
important_notices: CraftAppsImportantNotices::default(),
|
||||
};
|
||||
HttpResponse::Ok()
|
||||
.insert_header(cache_control)
|
||||
.json(response)
|
||||
}
|
||||
|
||||
/// Cacheable only once every app has been fetched (`last_fetched` is set). Until then (an
|
||||
/// instance that just started, or one whose first fetch failed) the answer is `no-store`, so a
|
||||
/// shared cache never keeps handing out "not fetched yet" after this instance has warmed up.
|
||||
fn cache_control_for(release_info: &CraftAppsReleaseInfo) -> CacheControl {
|
||||
if release_info.last_fetched.is_some() {
|
||||
CacheControl(vec![CacheDirective::Public, CacheDirective::MaxAge(CACHE_MAX_AGE_SECS)])
|
||||
} else {
|
||||
CacheControl(vec![CacheDirective::NoStore])
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use chrono::Utc;
|
||||
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn a_fully_fetched_answer_is_cacheable() {
|
||||
let release_info = CraftAppsReleaseInfo { apps: BTreeMap::new(), last_fetched: Some(Utc::now()) };
|
||||
assert_eq!(cache_control_for(&release_info).to_string(), "public, max-age=60");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_answer_with_unfetched_apps_is_never_stored() {
|
||||
let release_info = CraftAppsReleaseInfo { apps: BTreeMap::new(), last_fetched: None };
|
||||
assert_eq!(cache_control_for(&release_info).to_string(), "no-store");
|
||||
}
|
||||
}
|
||||
@@ -1 +0,0 @@
|
||||
pub mod get_craft_apps_release_info_handler;
|
||||
@@ -5,7 +5,6 @@ pub mod beta_keys;
|
||||
pub mod billing_fakeyou;
|
||||
pub mod characters;
|
||||
pub mod comments;
|
||||
pub mod craft_apps;
|
||||
pub mod credits;
|
||||
pub mod dev;
|
||||
pub mod download_job;
|
||||
|
||||
@@ -63,7 +63,6 @@ use crate::startup::setup_inference_providers::{
|
||||
use crate::startup::setup_pager::build_pager;
|
||||
use crate::startup::setup_static_feature_flags::setup_static_feature_flags;
|
||||
use crate::state::certs::google_sign_in_cert::GoogleSignInCert;
|
||||
use crate::state::craft_apps_release_cache::CraftAppsReleaseCache;
|
||||
use crate::state::memory_cache::model_token_to_info_cache::ModelTokenToInfoCache;
|
||||
use crate::state::server_state::{
|
||||
Dashboards, DataboxDashboards, DurableInMemoryCaches, EnvConfig, EphemeralInMemoryCaches,
|
||||
@@ -703,7 +702,6 @@ pub(crate) fn build_test_server_state(pool: MySqlPool, media_cdn_override_url: S
|
||||
sort_key_crypto: WebSortKeyCrypto::new("test-sort-key"),
|
||||
opaque_cursors: WebOpaqueCursorEncoderV2::new("test-sort-key"),
|
||||
ip_ban_list: IpBanList::new(),
|
||||
craft_apps_release_cache: CraftAppsReleaseCache::new(),
|
||||
cidr_ban_set: BannedCidrSet::new(),
|
||||
troll_bans: TrollBans {
|
||||
user_tokens: TrollUserBanList::new(),
|
||||
|
||||
@@ -3,7 +3,6 @@ use crate::http_server::routes::application_routes::api_keys_routes::add_api_key
|
||||
use crate::http_server::routes::application_routes::billing_fakeyou_routes::add_billing_fakeyou_routes;
|
||||
use crate::http_server::routes::application_routes::character_routes::add_character_routes;
|
||||
use crate::http_server::routes::application_routes::comments_routes::add_comments_routes;
|
||||
use crate::http_server::routes::application_routes::craft_apps_routes::add_craft_apps_routes;
|
||||
use crate::http_server::routes::application_routes::credits_routes::add_credits_routes;
|
||||
use crate::http_server::routes::application_routes::dev_routes::add_dev_routes;
|
||||
use crate::http_server::routes::application_routes::featured_item_routes::add_featured_item_routes;
|
||||
@@ -77,7 +76,6 @@ where
|
||||
app = add_video_info_routes(app); // /v1/video_info/... (video provenance detection)
|
||||
app = add_analytics_routes(app); // /v1/analytics/...
|
||||
app = add_web_referrals_routes(app); // /v1/web_referrals/...
|
||||
app = add_craft_apps_routes(app); // /v1/craft_apps/... (ArtCraft Launcher release info)
|
||||
app = add_dev_routes(app); // /v1/dev/... — dev-only error/response shape probes
|
||||
|
||||
// User and user-adjacent routes
|
||||
|
||||
@@ -1,28 +0,0 @@
|
||||
use actix_http::body::MessageBody;
|
||||
use actix_service::ServiceFactory;
|
||||
use actix_web::dev::{ServiceRequest, ServiceResponse};
|
||||
use actix_web::{web, App, Error, HttpResponse};
|
||||
|
||||
use crate::http_server::endpoints::craft_apps::get_craft_apps_release_info_handler::get_craft_apps_release_info_handler;
|
||||
|
||||
pub fn add_craft_apps_routes<T, B>(app: App<T>) -> App<T>
|
||||
where
|
||||
B: MessageBody,
|
||||
T: ServiceFactory<
|
||||
ServiceRequest,
|
||||
Config = (),
|
||||
Response = ServiceResponse<B>,
|
||||
Error = Error,
|
||||
InitError = (),
|
||||
>,
|
||||
{
|
||||
app.service(
|
||||
web::scope("/v1/craft_apps")
|
||||
// Release info for ArtCraft Launcher (polled instead of GitHub).
|
||||
.service(
|
||||
web::resource("/release_info")
|
||||
.route(web::get().to(get_craft_apps_release_info_handler))
|
||||
.route(web::head().to(|| HttpResponse::Ok())),
|
||||
),
|
||||
)
|
||||
}
|
||||
@@ -3,7 +3,6 @@ mod api_keys_routes;
|
||||
mod billing_fakeyou_routes;
|
||||
mod character_routes;
|
||||
mod comments_routes;
|
||||
mod craft_apps_routes;
|
||||
mod credits_routes;
|
||||
mod dev_routes;
|
||||
mod featured_item_routes;
|
||||
|
||||
@@ -37,7 +37,6 @@ use errors::AnyhowResult;
|
||||
use log::{info, warn};
|
||||
use server_environment::ServerEnvironment;
|
||||
use shared_env_var_config::logging::DEFAULT_RUST_LOG;
|
||||
use shared_env_var_config::redis::env_get_redis_0_connection_string_or_default;
|
||||
use tokio::runtime::Runtime;
|
||||
use user_traits_component::traits::internal_session_cache_purge::InternalSessionCachePurge;
|
||||
|
||||
@@ -57,9 +56,6 @@ use crate::startup::setup_metrics::build_metrics;
|
||||
use crate::state::server_state::ServerState;
|
||||
use crate::threads::db_health_checker_thread::db_health_checker_thread::db_health_checker_thread;
|
||||
use crate::threads::expire_login_challenges::expire_login_challenges;
|
||||
use crate::threads::poll_craft_app_releases_thread::config::CraftAppsPollConfig;
|
||||
use crate::threads::poll_craft_app_releases_thread::poll_craft_app_releases::poll_craft_app_releases;
|
||||
use crate::threads::poll_craft_app_releases_thread::shared_store::build_poller_redis_pool;
|
||||
use crate::threads::poll_ip_banlist_thread::poll_ip_bans;
|
||||
use crate::threads::poll_model_token_info_thread::poll_model_token_info_thread;
|
||||
|
||||
@@ -154,18 +150,6 @@ async fn main() -> AnyhowResult<()> {
|
||||
poll_model_token_info_thread(model_token_info_cache_clone, mysql_pool_clone).await;
|
||||
});
|
||||
|
||||
info!("Spawning craft app release polling thread.");
|
||||
|
||||
let craft_apps_release_cache_clone = server_state.craft_apps_release_cache.clone();
|
||||
let craft_apps_poll_config = CraftAppsPollConfig::from_env();
|
||||
// Its own small pool with I/O timeouts, so a hung Redis can't tie up the handlers' pool.
|
||||
let craft_apps_redis_pool = build_poller_redis_pool(&env_get_redis_0_connection_string_or_default())?;
|
||||
let hostname_clone = server_state.hostname.clone();
|
||||
|
||||
tokio_runtime.spawn(async move {
|
||||
poll_craft_app_releases(craft_apps_release_cache_clone, craft_apps_poll_config, craft_apps_redis_pool, hostname_clone).await;
|
||||
});
|
||||
|
||||
// ==================== Metrics worker ==================== //
|
||||
|
||||
let (metrics_collector, maybe_metrics_worker) =
|
||||
|
||||
@@ -41,7 +41,6 @@ use crate::startup::setup_static_feature_flags::setup_static_feature_flags;
|
||||
use crate::startup::setup_stripe_artcraft::setup_stripe_artcraft;
|
||||
use crate::startup::setup_stripe_fakeyou::setup_stripe_fakeyou;
|
||||
use crate::state::certs::google_sign_in_cert::GoogleSignInCert;
|
||||
use crate::state::craft_apps_release_cache::CraftAppsReleaseCache;
|
||||
use crate::state::memory_cache::model_token_to_info_cache::ModelTokenToInfoCache;
|
||||
use crate::state::server_state::{
|
||||
DurableInMemoryCaches, EnvConfig, EphemeralInMemoryCaches,
|
||||
@@ -260,7 +259,6 @@ pub async fn setup_dependencies(server_hostname: &str) -> AnyhowResult<SetupResu
|
||||
}
|
||||
},
|
||||
ip_ban_list,
|
||||
craft_apps_release_cache: CraftAppsReleaseCache::new(),
|
||||
cidr_ban_set,
|
||||
troll_bans: TrollBans {
|
||||
user_tokens: user_token_troll_bans,
|
||||
|
||||
@@ -1,279 +0,0 @@
|
||||
//! The latest release of every Craft App, as last fetched from GitHub by
|
||||
//! `poll_craft_app_releases_thread` (directly, or from the fleet's shared copy in Redis), served
|
||||
//! by `GET /v1/craft_apps/release_info`. The endpoint always answers from this memory, so trouble
|
||||
//! with Redis or GitHub never breaks it.
|
||||
//!
|
||||
//! ## App keys
|
||||
//!
|
||||
//! Apps are keyed by their **app key**: the app's lowercase machine name, the same string the
|
||||
//! Craft Apps use for their binaries, asset file names and `ai.storyteller.<key>` bundle ids.
|
||||
//! For example `photocraft`, `pdfcraft`, `cadcraft`, and `artcraft-launcher` for ArtCraft Launcher
|
||||
//! itself. The key is not the display name (`PhotoCraft`) and not the GitHub repo
|
||||
//! (`storytold/photocraft`, whose project name can differ: the launcher's repo is
|
||||
//! `storytold/craft-launcher`).
|
||||
//!
|
||||
//! The keys come from the polling config (`poll_craft_app_releases_thread::config`: the built-in
|
||||
//! list, or `CRAFT_APPS_RELEASE_APPS`) and are served unchanged as the keys of
|
||||
//! `release_info.apps`, which ArtCraft Launcher matches against its own app ids.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::sync::{Arc, PoisonError, RwLock};
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
|
||||
use artcraft_api_defs::craft_apps::release_info::{CraftAppRelease, CraftAppReleaseInfo, CraftAppsReleaseInfo};
|
||||
|
||||
/// Shared between the polling thread and the HTTP handlers. Cheap to clone.
|
||||
#[derive(Clone, Default)]
|
||||
pub struct CraftAppsReleaseCache {
|
||||
/// App key (lowercase machine name, e.g. `photocraft`; see the module docs) → what's known
|
||||
/// about that app. A `BTreeMap` so the endpoint lists apps in a stable order.
|
||||
inner: Arc<RwLock<BTreeMap<String, CachedCraftApp>>>,
|
||||
}
|
||||
|
||||
/// One app: what's known about it, and the ETag to revalidate it with.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct CachedCraftApp {
|
||||
/// Display name (`PhotoCraft`).
|
||||
pub name: String,
|
||||
|
||||
/// GitHub repository, `organization/project` (`storytold/photocraft`).
|
||||
pub repo: String,
|
||||
|
||||
/// The latest published release. `None` both before the first fetch and when the repository
|
||||
/// has no release; `app_last_fetched` tells the two apart.
|
||||
pub latest_release: Option<CraftAppRelease>,
|
||||
|
||||
/// When this app's release was last fetched (or confirmed unchanged) successfully. `None`
|
||||
/// until the first success.
|
||||
pub app_last_fetched: Option<DateTime<Utc>>,
|
||||
|
||||
/// GitHub's ETag for the last successful answer, sent back as `If-None-Match` so an unchanged
|
||||
/// release costs a `304`.
|
||||
pub etag: Option<String>,
|
||||
|
||||
/// When a request for this app last failed. Only meaningful when newer than
|
||||
/// `app_last_fetched`; it delays the next attempt so a broken repo isn't retried every tick.
|
||||
pub last_failed_at: Option<DateTime<Utc>>,
|
||||
|
||||
/// How many requests in a row have failed since the last success (when `last_failed_at` is
|
||||
/// newer than `app_last_fetched`).
|
||||
pub consecutive_failures: u32,
|
||||
}
|
||||
|
||||
impl CraftAppsReleaseCache {
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// Makes sure every app in `apps` has an entry, so the endpoint reports apps that haven't been
|
||||
/// fetched yet. Each tuple is `(app key, display name, repo)`, for example
|
||||
/// `("photocraft", "PhotoCraft", "storytold/photocraft")`. Apps already present keep what
|
||||
/// was fetched for them.
|
||||
pub fn register_apps(&self, apps: &[(String, String, String)]) {
|
||||
let mut map = self.inner.write().unwrap_or_else(PoisonError::into_inner);
|
||||
for (key, name, repo) in apps {
|
||||
map.entry(key.clone()).or_insert_with(|| CachedCraftApp {
|
||||
name: name.clone(),
|
||||
repo: repo.clone(),
|
||||
latest_release: None,
|
||||
app_last_fetched: None,
|
||||
etag: None,
|
||||
last_failed_at: None,
|
||||
consecutive_failures: 0,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/// The ETag of the last successful answer for the app with app key `key` (`photocraft`), to
|
||||
/// send as `If-None-Match`.
|
||||
pub fn etag(&self, key: &str) -> Option<String> {
|
||||
let map = self.inner.read().unwrap_or_else(PoisonError::into_inner);
|
||||
map.get(key).and_then(|app| app.etag.clone())
|
||||
}
|
||||
|
||||
/// Records a complete, successful fetch of the app with app key `key` (`photocraft`): its
|
||||
/// latest release (`None` if it has none). Ignored for keys that were never registered.
|
||||
pub fn store(&self, key: &str, latest_release: Option<CraftAppRelease>, etag: Option<String>, fetched_at: DateTime<Utc>) {
|
||||
let mut map = self.inner.write().unwrap_or_else(PoisonError::into_inner);
|
||||
if let Some(app) = map.get_mut(key) {
|
||||
app.latest_release = latest_release;
|
||||
app.etag = etag;
|
||||
app.app_last_fetched = Some(fetched_at);
|
||||
}
|
||||
}
|
||||
|
||||
/// Records that GitHub confirmed the app with app key `key` (`photocraft`) is unchanged
|
||||
/// (`304 Not Modified`). Ignored for keys that were never registered.
|
||||
pub fn confirm_unchanged(&self, key: &str, fetched_at: DateTime<Utc>) {
|
||||
let mut map = self.inner.write().unwrap_or_else(PoisonError::into_inner);
|
||||
if let Some(app) = map.get_mut(key) {
|
||||
app.app_last_fetched = Some(fetched_at);
|
||||
}
|
||||
}
|
||||
|
||||
/// A copy of what's known about the app with app key `key` (`photocraft`).
|
||||
pub fn app(&self, key: &str) -> Option<CachedCraftApp> {
|
||||
let map = self.inner.read().unwrap_or_else(PoisonError::into_inner);
|
||||
map.get(key).cloned()
|
||||
}
|
||||
|
||||
/// Takes a fetch made elsewhere (another instance, through Redis) for the app with app key
|
||||
/// `key`, if it is newer than what this instance has. Returns whether it was taken. Ignored for
|
||||
/// keys that were never registered, so the registered app list stays authoritative.
|
||||
pub fn merge_fetched(&self, key: &str, latest_release: Option<CraftAppRelease>, etag: Option<String>, fetched_at: DateTime<Utc>) -> bool {
|
||||
let mut map = self.inner.write().unwrap_or_else(PoisonError::into_inner);
|
||||
let Some(app) = map.get_mut(key) else {
|
||||
return false;
|
||||
};
|
||||
if app.app_last_fetched.is_some_and(|mine| mine >= fetched_at) {
|
||||
return false;
|
||||
}
|
||||
app.latest_release = latest_release;
|
||||
app.etag = etag;
|
||||
app.app_last_fetched = Some(fetched_at);
|
||||
true
|
||||
}
|
||||
|
||||
/// Records a failed request for the app with app key `key` (`photocraft`) and returns how many
|
||||
/// have failed in a row since its last success. Ignored (returns 0) for unregistered keys.
|
||||
pub fn record_failure(&self, key: &str, failed_at: DateTime<Utc>) -> u32 {
|
||||
let mut map = self.inner.write().unwrap_or_else(PoisonError::into_inner);
|
||||
let Some(app) = map.get_mut(key) else {
|
||||
return 0;
|
||||
};
|
||||
app.consecutive_failures = if app.is_failing() { app.consecutive_failures.saturating_add(1) } else { 1 };
|
||||
app.last_failed_at = Some(failed_at);
|
||||
app.consecutive_failures
|
||||
}
|
||||
|
||||
/// Takes a failure recorded elsewhere (another instance, through Redis) for the app with app
|
||||
/// key `key`, if it is newer than the one this instance knows.
|
||||
pub fn merge_failure(&self, key: &str, failed_at: DateTime<Utc>, consecutive_failures: u32) {
|
||||
let mut map = self.inner.write().unwrap_or_else(PoisonError::into_inner);
|
||||
if let Some(app) = map.get_mut(key) {
|
||||
if app.last_failed_at.is_none_or(|mine| mine < failed_at) {
|
||||
app.last_failed_at = Some(failed_at);
|
||||
app.consecutive_failures = consecutive_failures;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Everything, as the endpoint serves it: `apps` keyed by app key.
|
||||
pub fn snapshot(&self) -> CraftAppsReleaseInfo {
|
||||
let map = self.inner.read().unwrap_or_else(PoisonError::into_inner);
|
||||
let apps: BTreeMap<String, CraftAppReleaseInfo> = map
|
||||
.iter()
|
||||
.map(|(key, app)| {
|
||||
(key.clone(), CraftAppReleaseInfo {
|
||||
name: app.name.clone(),
|
||||
repo: app.repo.clone(),
|
||||
latest_release: app.latest_release.clone(),
|
||||
app_last_fetched: app.app_last_fetched,
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
let last_fetched = least_recently_fetched(&apps);
|
||||
CraftAppsReleaseInfo { apps, last_fetched }
|
||||
}
|
||||
}
|
||||
|
||||
impl CachedCraftApp {
|
||||
/// Whether the last request for this app failed (no success since).
|
||||
pub fn is_failing(&self) -> bool {
|
||||
match (self.last_failed_at, self.app_last_fetched) {
|
||||
(Some(failed), Some(fetched)) => failed > fetched,
|
||||
(Some(_), None) => true,
|
||||
(None, _) => false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The oldest `app_last_fetched`, or `None` while any app has never been fetched.
|
||||
fn least_recently_fetched(apps: &BTreeMap<String, CraftAppReleaseInfo>) -> Option<DateTime<Utc>> {
|
||||
if apps.is_empty() {
|
||||
return None;
|
||||
}
|
||||
let mut oldest: Option<DateTime<Utc>> = None;
|
||||
for app in apps.values() {
|
||||
let fetched = app.app_last_fetched?;
|
||||
oldest = Some(oldest.map_or(fetched, |o| o.min(fetched)));
|
||||
}
|
||||
oldest
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use chrono::TimeZone;
|
||||
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn last_fetched_is_the_oldest_app_and_absent_until_all_are_fetched() {
|
||||
let cache = CraftAppsReleaseCache::new();
|
||||
cache.register_apps(&[
|
||||
("photocraft".into(), "PhotoCraft".into(), "storytold/photocraft".into()),
|
||||
("filmcraft".into(), "FilmCraft".into(), "storytold/filmcraft".into()),
|
||||
]);
|
||||
assert_eq!(cache.snapshot().apps.len(), 2);
|
||||
assert_eq!(cache.snapshot().last_fetched, None);
|
||||
|
||||
let early = Utc.with_ymd_and_hms(2026, 10, 10, 1, 0, 0).unwrap();
|
||||
let late = Utc.with_ymd_and_hms(2026, 10, 10, 2, 0, 0).unwrap();
|
||||
cache.store("photocraft", None, Some("\"abc\"".into()), late);
|
||||
assert_eq!(cache.snapshot().last_fetched, None, "filmcraft hasn't been fetched yet");
|
||||
|
||||
cache.confirm_unchanged("filmcraft", early);
|
||||
assert_eq!(cache.snapshot().last_fetched, Some(early));
|
||||
assert_eq!(cache.etag("photocraft").as_deref(), Some("\"abc\""));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn registering_again_keeps_what_was_fetched() {
|
||||
let cache = CraftAppsReleaseCache::new();
|
||||
let apps = [("photocraft".to_string(), "PhotoCraft".to_string(), "storytold/photocraft".to_string())];
|
||||
cache.register_apps(&apps);
|
||||
let now = Utc::now();
|
||||
cache.confirm_unchanged("photocraft", now);
|
||||
cache.register_apps(&apps);
|
||||
assert_eq!(cache.snapshot().apps["photocraft"].app_last_fetched, Some(now));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn merging_takes_only_newer_fetches_of_registered_apps() {
|
||||
let cache = CraftAppsReleaseCache::new();
|
||||
cache.register_apps(&[("photocraft".into(), "PhotoCraft".into(), "storytold/photocraft".into())]);
|
||||
let early = Utc.with_ymd_and_hms(2026, 10, 10, 1, 0, 0).unwrap();
|
||||
let late = Utc.with_ymd_and_hms(2026, 10, 10, 2, 0, 0).unwrap();
|
||||
|
||||
assert!(cache.merge_fetched("photocraft", None, Some("\"new\"".into()), late));
|
||||
assert!(!cache.merge_fetched("photocraft", None, Some("\"old\"".into()), early), "older than ours");
|
||||
assert!(!cache.merge_fetched("photocraft", None, Some("\"same\"".into()), late), "no newer than ours");
|
||||
assert!(!cache.merge_fetched("unknowncraft", None, None, late), "not registered");
|
||||
|
||||
let snapshot = cache.snapshot();
|
||||
assert_eq!(snapshot.apps.len(), 1);
|
||||
assert_eq!(snapshot.apps["photocraft"].app_last_fetched, Some(late));
|
||||
assert_eq!(cache.etag("photocraft").as_deref(), Some("\"new\""));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn failures_count_up_until_a_success() {
|
||||
let cache = CraftAppsReleaseCache::new();
|
||||
cache.register_apps(&[("photocraft".into(), "PhotoCraft".into(), "storytold/photocraft".into())]);
|
||||
let t = |h| Utc.with_ymd_and_hms(2026, 10, 10, h, 0, 0).unwrap();
|
||||
|
||||
assert_eq!(cache.record_failure("photocraft", t(1)), 1);
|
||||
assert_eq!(cache.record_failure("photocraft", t(2)), 2);
|
||||
assert!(cache.app("photocraft").unwrap().is_failing());
|
||||
|
||||
cache.store("photocraft", None, None, t(3));
|
||||
assert!(!cache.app("photocraft").unwrap().is_failing());
|
||||
assert_eq!(cache.record_failure("photocraft", t(4)), 1, "counting restarts after a success");
|
||||
|
||||
cache.merge_failure("photocraft", t(3), 9);
|
||||
assert_eq!(cache.app("photocraft").unwrap().consecutive_failures, 1, "older failure ignored");
|
||||
cache.merge_failure("photocraft", t(5), 4);
|
||||
assert_eq!(cache.app("photocraft").unwrap().consecutive_failures, 4);
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,5 @@
|
||||
pub mod cached_queries;
|
||||
pub mod flags;
|
||||
pub mod certs;
|
||||
pub mod craft_apps_release_cache;
|
||||
pub mod memory_cache;
|
||||
pub mod server_state;
|
||||
@@ -1,7 +1,6 @@
|
||||
use std::collections::HashSet;
|
||||
|
||||
use crate::configs::app_startup::username_set::UsernameSet;
|
||||
use crate::state::craft_apps_release_cache::CraftAppsReleaseCache;
|
||||
use crate::configs::static_api_tokens::StaticApiTokenSet;
|
||||
use crate::http_server::deprecated_endpoints::categories::tts::list_fully_computed_assigned_tts_categories::list_fully_computed_assigned_tts_categories::ModelTokensByCategoryToken;
|
||||
use crate::http_server::endpoints::media_files::list::list_featured_media_files_handler::ListFeaturedMediaFilesQueryParams;
|
||||
@@ -125,9 +124,6 @@ pub struct ServerState {
|
||||
|
||||
pub caches: InMemoryCaches,
|
||||
|
||||
/// The latest Craft App releases, kept fresh by `poll_craft_app_releases_thread`.
|
||||
pub craft_apps_release_cache: CraftAppsReleaseCache,
|
||||
|
||||
pub google_sign_in_cert: GoogleSignInCert,
|
||||
|
||||
pub temp_dir_creator: ScopedTempDirCreator,
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
pub mod db_health_checker_thread;
|
||||
pub mod expire_login_challenges;
|
||||
pub mod poll_craft_app_releases_thread;
|
||||
pub mod poll_ip_banlist_thread;
|
||||
pub mod poll_model_token_info_thread;
|
||||
@@ -1,164 +0,0 @@
|
||||
//! How storyteller-web polls GitHub for Craft App releases, from the environment.
|
||||
//!
|
||||
//! | Variable | Default | Meaning |
|
||||
//! |-------------------------------------------|-----------------------------|---------------------------------------------------------------|
|
||||
//! | `CRAFT_APPS_RELEASE_POLLING_ENABLED` | `true` | Poll at all (the endpoint still answers) |
|
||||
//! | `CRAFT_APPS_RELEASE_COORDINATION_ENABLED` | `true` | Share releases and backoff through Redis (see below) |
|
||||
//! | `CRAFT_APPS_RELEASE_POLL_INTERVAL_SECS` | `120`; `1800` with no token | How old a release may get; at least `60`; `300` with no token |
|
||||
//! | `CRAFT_APPS_RELEASE_MAX_BACKOFF_SECS` | `14400` (4 h) | Longest wait after repeated failures/rate limits |
|
||||
//! | `CRAFT_APPS_RELEASE_GITHUB_TOKEN` | (none; optional) | Raises GitHub's limit from 60 to 5000 per hour |
|
||||
//! | `CRAFT_APPS_RELEASE_APPS` | built-in list | `key=Name=org/repo,…` to replace the app list |
|
||||
//!
|
||||
//! The GitHub token is optional. Unset, empty or blank, polling still runs: requests go out
|
||||
//! without an `Authorization` header, and the interval defaults to the slow no-token value.
|
||||
//!
|
||||
//! With coordination (the default), the fleet shares each app's release, ETag and fetch time in
|
||||
//! Redis, so the whole fleet requests an app from GitHub about once per interval instead of every
|
||||
//! pod doing so. That is what makes the 2 minute default safe with a token. Without a token
|
||||
//! GitHub allows 60 requests an hour per IP address and also counts `304`s, so the interval stays
|
||||
//! at 30 minutes (13 apps: about 26 requests an hour). While Redis is unreachable (or with
|
||||
//! coordination off) every pod polls on its own at the same interval.
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
const DEFAULT_INTERVAL_SECS_WITH_TOKEN: u64 = 120;
|
||||
const MIN_INTERVAL_SECS_WITH_TOKEN: u64 = 60;
|
||||
const DEFAULT_INTERVAL_SECS_WITHOUT_TOKEN: u64 = 1800;
|
||||
const MIN_INTERVAL_SECS_WITHOUT_TOKEN: u64 = 300;
|
||||
const DEFAULT_MAX_BACKOFF_SECS: u64 = 14_400;
|
||||
|
||||
/// The Craft Apps the launcher knows: `(app key, display name, GitHub repo)`. The app key is
|
||||
/// the app's lowercase machine name; it keys `CraftAppsReleaseCache` and the endpoint's
|
||||
/// `release_info.apps`, and must match the app id ArtCraft Launcher uses. ArtCraft Launcher is
|
||||
/// included (key `artcraft-launcher`, repo `storytold/craft-launcher`) so it can check for its
|
||||
/// own updates.
|
||||
const DEFAULT_APPS: &[(&str, &str, &str)] = &[
|
||||
("photocraft", "PhotoCraft", "storytold/photocraft"),
|
||||
("vectorcraft", "VectorCraft", "storytold/vectorcraft"),
|
||||
("filmcraft", "FilmCraft", "storytold/filmcraft"),
|
||||
("lightcraft", "LightCraft", "storytold/lightcraft"),
|
||||
("pdfcraft", "PdfCraft", "storytold/pdfcraft"),
|
||||
("effectcraft", "EffectCraft", "storytold/effectcraft"),
|
||||
("designcraft", "DesignCraft", "storytold/designcraft"),
|
||||
("wordcraft", "WordCraft", "storytold/wordcraft"),
|
||||
("deckcraft", "DeckCraft", "storytold/deckcraft"),
|
||||
("gridcraft", "GridCraft", "storytold/gridcraft"),
|
||||
("cadcraft", "CADCraft", "storytold/cadcraft"),
|
||||
("soundcraft", "SoundCraft", "storytold/soundcraft"),
|
||||
("artcraft-launcher", "ArtCraft Launcher", "storytold/craft-launcher"),
|
||||
];
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct CraftAppsPollConfig {
|
||||
pub enabled: bool,
|
||||
/// Share releases and backoff with the fleet through Redis; when false (or while Redis is
|
||||
/// unreachable) this pod polls on its own.
|
||||
pub coordination_enabled: bool,
|
||||
/// How old an app's release may get before it is requested from GitHub again.
|
||||
pub interval: Duration,
|
||||
pub max_backoff: Duration,
|
||||
/// Never empty or blank: those are read as no token.
|
||||
pub github_token: Option<String>,
|
||||
/// `(app key, display name, repo)`; see [`DEFAULT_APPS`].
|
||||
pub apps: Vec<(String, String, String)>,
|
||||
}
|
||||
|
||||
impl CraftAppsPollConfig {
|
||||
pub fn from_env() -> Self {
|
||||
let github_token = non_blank_token(easyenv::get_env_string_optional("CRAFT_APPS_RELEASE_GITHUB_TOKEN"));
|
||||
let has_token = github_token.is_some();
|
||||
let default_interval = default_interval_secs(has_token);
|
||||
let requested_interval = easyenv::get_env_num::<u64>("CRAFT_APPS_RELEASE_POLL_INTERVAL_SECS", default_interval)
|
||||
.unwrap_or(default_interval);
|
||||
let interval_secs = clamp_interval_secs(requested_interval, has_token);
|
||||
let max_backoff_secs = easyenv::get_env_num::<u64>("CRAFT_APPS_RELEASE_MAX_BACKOFF_SECS", DEFAULT_MAX_BACKOFF_SECS)
|
||||
.unwrap_or(DEFAULT_MAX_BACKOFF_SECS)
|
||||
.max(interval_secs);
|
||||
let apps = easyenv::get_env_string_optional("CRAFT_APPS_RELEASE_APPS")
|
||||
.map(|v| parse_app_list(&v))
|
||||
.filter(|apps| !apps.is_empty())
|
||||
.unwrap_or_else(default_apps);
|
||||
Self {
|
||||
enabled: easyenv::get_env_bool_or_default("CRAFT_APPS_RELEASE_POLLING_ENABLED", true),
|
||||
coordination_enabled: easyenv::get_env_bool_or_default("CRAFT_APPS_RELEASE_COORDINATION_ENABLED", true),
|
||||
interval: Duration::from_secs(interval_secs),
|
||||
max_backoff: Duration::from_secs(max_backoff_secs),
|
||||
github_token,
|
||||
apps,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn default_apps() -> Vec<(String, String, String)> {
|
||||
DEFAULT_APPS.iter().map(|(k, n, r)| (k.to_string(), n.to_string(), r.to_string())).collect()
|
||||
}
|
||||
|
||||
/// The token, trimmed; `None` when unset, empty or blank.
|
||||
fn non_blank_token(value: Option<String>) -> Option<String> {
|
||||
value.map(|t| t.trim().to_string()).filter(|t| !t.is_empty())
|
||||
}
|
||||
|
||||
fn default_interval_secs(has_token: bool) -> u64 {
|
||||
if has_token { DEFAULT_INTERVAL_SECS_WITH_TOKEN } else { DEFAULT_INTERVAL_SECS_WITHOUT_TOKEN }
|
||||
}
|
||||
|
||||
fn clamp_interval_secs(requested: u64, has_token: bool) -> u64 {
|
||||
let min = if has_token { MIN_INTERVAL_SECS_WITH_TOKEN } else { MIN_INTERVAL_SECS_WITHOUT_TOKEN };
|
||||
requested.max(min)
|
||||
}
|
||||
|
||||
/// Reads `key=Name=org/repo,…` (app key, display name, repo). Keys are lowercased. Malformed
|
||||
/// entries are skipped.
|
||||
fn parse_app_list(value: &str) -> Vec<(String, String, String)> {
|
||||
value
|
||||
.split(',')
|
||||
.filter_map(|entry| {
|
||||
let mut parts = entry.trim().splitn(3, '=');
|
||||
let key = parts.next()?.trim().to_lowercase();
|
||||
let name = parts.next()?.trim().to_string();
|
||||
let repo = parts.next()?.trim().to_string();
|
||||
let valid_repo = repo.split('/').count() == 2 && !repo.starts_with('/') && !repo.ends_with('/');
|
||||
(!key.is_empty() && !name.is_empty() && valid_repo).then_some((key, name, repo))
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn reads_an_app_list_and_skips_bad_entries() {
|
||||
let apps = parse_app_list(" PhotoCraft=PhotoCraft=storytold/photocraft, junk, x=X=noslash, k=K=org/repo ");
|
||||
assert_eq!(apps, vec![
|
||||
("photocraft".to_string(), "PhotoCraft".to_string(), "storytold/photocraft".to_string()),
|
||||
("k".to_string(), "K".to_string(), "org/repo".to_string()),
|
||||
]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_default_list_has_the_suite_and_the_launcher() {
|
||||
let apps = default_apps();
|
||||
assert_eq!(apps.len(), 13);
|
||||
assert!(apps.iter().any(|(k, _, r)| k == "artcraft-launcher" && r == "storytold/craft-launcher"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_missing_empty_or_blank_token_is_no_token() {
|
||||
assert_eq!(non_blank_token(None), None);
|
||||
assert_eq!(non_blank_token(Some(String::new())), None);
|
||||
assert_eq!(non_blank_token(Some(" \n".to_string())), None);
|
||||
assert_eq!(non_blank_token(Some(" ghp_x ".to_string())).as_deref(), Some("ghp_x"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_interval_defaults_and_minimum_depend_on_the_token() {
|
||||
assert_eq!(default_interval_secs(true), 120);
|
||||
assert_eq!(default_interval_secs(false), 1800);
|
||||
assert_eq!(clamp_interval_secs(10, true), 60);
|
||||
assert_eq!(clamp_interval_secs(10, false), 300);
|
||||
assert_eq!(clamp_interval_secs(90, true), 90);
|
||||
assert_eq!(clamp_interval_secs(90, false), 300);
|
||||
assert_eq!(clamp_interval_secs(3600, false), 3600);
|
||||
}
|
||||
}
|
||||
@@ -1,97 +0,0 @@
|
||||
//! Falling back to polling alone when Redis is unreachable, and saying so without flooding the
|
||||
//! log every tick.
|
||||
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use log::{info, warn};
|
||||
|
||||
/// While Redis keeps failing, repeat the warning at most this often.
|
||||
const WARN_EVERY: Duration = Duration::from_secs(10 * 60);
|
||||
|
||||
/// How this tick works.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum PollMode {
|
||||
/// With the fleet: shared releases, claims and backoff in Redis.
|
||||
Coordinated,
|
||||
/// Alone, from this instance's memory: coordination is off or Redis is unreachable. Same
|
||||
/// interval and backoff rules, so a Redis outage never stops releases from refreshing.
|
||||
Standalone,
|
||||
}
|
||||
|
||||
/// Remembers Redis trouble so it is logged when it starts, every [`WARN_EVERY`] while it lasts,
|
||||
/// and when it ends.
|
||||
#[derive(Debug, Default)]
|
||||
pub struct RedisTroubleLog {
|
||||
failing_since: Option<Instant>,
|
||||
last_warned: Option<Instant>,
|
||||
suppressed: u32,
|
||||
}
|
||||
|
||||
impl PollMode {
|
||||
/// Coordinated only when coordination is on and the shared state could be read.
|
||||
pub fn choose(coordination_enabled: bool, shared_state_readable: bool) -> Self {
|
||||
if coordination_enabled && shared_state_readable { Self::Coordinated } else { Self::Standalone }
|
||||
}
|
||||
}
|
||||
|
||||
impl RedisTroubleLog {
|
||||
/// Notes a Redis error; returns whether it was logged.
|
||||
pub fn failed(&mut self, err: &anyhow::Error, now: Instant) -> bool {
|
||||
self.failing_since.get_or_insert(now);
|
||||
if self.last_warned.is_some_and(|last| now.duration_since(last) < WARN_EVERY) {
|
||||
self.suppressed = self.suppressed.saturating_add(1);
|
||||
return false;
|
||||
}
|
||||
warn!(
|
||||
"Craft app releases: Redis unavailable ({:#}); polling GitHub on this instance alone until it is back ({} similar errors not logged).",
|
||||
err, self.suppressed,
|
||||
);
|
||||
self.last_warned = Some(now);
|
||||
self.suppressed = 0;
|
||||
true
|
||||
}
|
||||
|
||||
/// Notes that Redis works; logs the recovery once.
|
||||
pub fn succeeded(&mut self, now: Instant) {
|
||||
if let Some(since) = self.failing_since.take() {
|
||||
info!("Craft app releases: Redis is back after {:?}; coordinating with the fleet again.", now.duration_since(since));
|
||||
self.last_warned = None;
|
||||
self.suppressed = 0;
|
||||
}
|
||||
}
|
||||
|
||||
pub fn is_failing(&self) -> bool {
|
||||
self.failing_since.is_some()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use anyhow::anyhow;
|
||||
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn coordinates_only_when_enabled_and_redis_answers() {
|
||||
assert_eq!(PollMode::choose(true, true), PollMode::Coordinated);
|
||||
assert_eq!(PollMode::choose(true, false), PollMode::Standalone);
|
||||
assert_eq!(PollMode::choose(false, true), PollMode::Standalone);
|
||||
assert_eq!(PollMode::choose(false, false), PollMode::Standalone);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn warns_once_per_window_while_redis_keeps_failing() {
|
||||
let mut log = RedisTroubleLog::default();
|
||||
let start = Instant::now();
|
||||
let err = anyhow!("connection refused");
|
||||
assert!(log.failed(&err, start));
|
||||
assert!(!log.failed(&err, start + Duration::from_secs(30)));
|
||||
assert!(!log.failed(&err, start + Duration::from_secs(599)));
|
||||
assert!(log.failed(&err, start + Duration::from_secs(600)));
|
||||
assert!(log.is_failing());
|
||||
|
||||
log.succeeded(start + Duration::from_secs(700));
|
||||
assert!(!log.is_failing());
|
||||
assert!(log.failed(&err, start + Duration::from_secs(730)), "a new outage warns right away");
|
||||
}
|
||||
}
|
||||
@@ -1,304 +0,0 @@
|
||||
//! One request to GitHub for an app's latest release, and reading the answer.
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
use wreq::header::{ACCEPT, AUTHORIZATION, ETAG, IF_NONE_MATCH, RETRY_AFTER, USER_AGENT};
|
||||
use wreq::StatusCode;
|
||||
use serde_derive::Deserialize;
|
||||
|
||||
use artcraft_api_defs::craft_apps::release_info::{
|
||||
CraftAppArch, CraftAppAssetPlatform, CraftAppOs, CraftAppPackageType, CraftAppRelease, CraftAppReleaseAsset,
|
||||
};
|
||||
|
||||
const API_BASE: &str = "https://api.github.com";
|
||||
const MAX_NOTES_BYTES: usize = 64 * 1024;
|
||||
const MAX_ASSETS: usize = 200;
|
||||
const MAX_BODY_BYTES: usize = 4 * 1024 * 1024;
|
||||
|
||||
/// What one request produced.
|
||||
#[derive(Debug)]
|
||||
pub enum FetchOutcome {
|
||||
/// The latest release (`None`: the repository has no published release).
|
||||
Fetched { release: Option<CraftAppRelease>, etag: Option<String> },
|
||||
/// `If-None-Match` matched: what we have is current.
|
||||
NotModified,
|
||||
/// GitHub refused: out of quota until this time, if it said.
|
||||
RateLimited { retry_at: Option<DateTime<Utc>> },
|
||||
/// Network error, unexpected status or unreadable answer. Nothing is stored.
|
||||
Failed(String),
|
||||
}
|
||||
|
||||
pub fn build_client() -> wreq::Result<wreq::Client> {
|
||||
wreq::Client::builder()
|
||||
.connect_timeout(Duration::from_secs(10))
|
||||
.timeout(Duration::from_secs(30))
|
||||
.build()
|
||||
}
|
||||
|
||||
/// `GET /repos/{repo}/releases/latest`, revalidated with `etag`. `token` is optional: without
|
||||
/// one the request is unauthenticated.
|
||||
pub async fn fetch_latest_release(client: &wreq::Client, repo: &str, etag: Option<&str>, token: Option<&str>) -> FetchOutcome {
|
||||
let request = build_request(client, repo, etag, token);
|
||||
let response = match request.send().await {
|
||||
Ok(r) => r,
|
||||
Err(err) => return FetchOutcome::Failed(format!("request failed: {err}")),
|
||||
};
|
||||
let status = response.status();
|
||||
let header = |name: &str| response.headers().get(name).and_then(|v| v.to_str().ok()).map(str::to_string);
|
||||
match status {
|
||||
StatusCode::NOT_MODIFIED => FetchOutcome::NotModified,
|
||||
StatusCode::NOT_FOUND => FetchOutcome::Fetched { release: None, etag: None },
|
||||
StatusCode::FORBIDDEN | StatusCode::TOO_MANY_REQUESTS => {
|
||||
let remaining = header("x-ratelimit-remaining");
|
||||
let reset = header("x-ratelimit-reset").and_then(|r| r.parse::<i64>().ok()).and_then(|s| DateTime::from_timestamp(s, 0));
|
||||
let retry_after = header(RETRY_AFTER.as_str())
|
||||
.and_then(|r| r.parse::<i64>().ok())
|
||||
.map(|s| Utc::now() + chrono::Duration::seconds(s.clamp(0, 86_400)));
|
||||
if status == StatusCode::TOO_MANY_REQUESTS || remaining.as_deref() == Some("0") || retry_after.is_some() {
|
||||
FetchOutcome::RateLimited { retry_at: retry_after.or(reset) }
|
||||
} else {
|
||||
FetchOutcome::Failed(format!("HTTP {status}"))
|
||||
}
|
||||
}
|
||||
StatusCode::OK => {
|
||||
let etag = header(ETAG.as_str());
|
||||
let body = match response.bytes().await {
|
||||
Ok(b) if b.len() <= MAX_BODY_BYTES => b,
|
||||
Ok(b) => return FetchOutcome::Failed(format!("answer too large ({} bytes)", b.len())),
|
||||
Err(err) => return FetchOutcome::Failed(format!("unreadable answer: {err}")),
|
||||
};
|
||||
match parse_release(&body, repo) {
|
||||
Ok(release) => FetchOutcome::Fetched { release: Some(release), etag },
|
||||
Err(err) => FetchOutcome::Failed(err),
|
||||
}
|
||||
}
|
||||
other => FetchOutcome::Failed(format!("HTTP {other}")),
|
||||
}
|
||||
}
|
||||
|
||||
/// The request for `repo`'s latest release. Sends `Authorization: Bearer` only for a non-blank
|
||||
/// token (an authenticated `304` doesn't count against the rate limit); never an empty header.
|
||||
fn build_request(client: &wreq::Client, repo: &str, etag: Option<&str>, token: Option<&str>) -> wreq::RequestBuilder {
|
||||
let url = format!("{API_BASE}/repos/{repo}/releases/latest");
|
||||
let mut request = client
|
||||
.get(&url)
|
||||
.header(USER_AGENT, "storyteller-web (craft apps release poller)")
|
||||
.header(ACCEPT, "application/vnd.github+json")
|
||||
.header("X-GitHub-Api-Version", "2022-11-28");
|
||||
if let Some(etag) = etag {
|
||||
request = request.header(IF_NONE_MATCH, etag);
|
||||
}
|
||||
if let Some(token) = token.map(str::trim).filter(|t| !t.is_empty()) {
|
||||
request = request.header(AUTHORIZATION, format!("Bearer {token}"));
|
||||
}
|
||||
request
|
||||
}
|
||||
|
||||
/// The parts of GitHub's release object we use. Everything optional: GitHub's answer is input.
|
||||
#[derive(Deserialize)]
|
||||
struct GithubRelease {
|
||||
#[serde(default)]
|
||||
tag_name: Option<String>,
|
||||
#[serde(default)]
|
||||
name: Option<String>,
|
||||
#[serde(default)]
|
||||
html_url: Option<String>,
|
||||
#[serde(default)]
|
||||
published_at: Option<DateTime<Utc>>,
|
||||
#[serde(default)]
|
||||
prerelease: bool,
|
||||
#[serde(default)]
|
||||
draft: bool,
|
||||
#[serde(default)]
|
||||
body: Option<String>,
|
||||
#[serde(default)]
|
||||
assets: Vec<GithubAsset>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct GithubAsset {
|
||||
#[serde(default)]
|
||||
name: Option<String>,
|
||||
#[serde(default)]
|
||||
size: u64,
|
||||
#[serde(default)]
|
||||
browser_download_url: Option<String>,
|
||||
/// `sha256:<hex>` on assets uploaded since mid-2025.
|
||||
#[serde(default)]
|
||||
digest: Option<String>,
|
||||
}
|
||||
|
||||
/// Reads a release, keeping only downloads that really belong to `repo`.
|
||||
pub fn parse_release(body: &[u8], repo: &str) -> Result<CraftAppRelease, String> {
|
||||
let raw: GithubRelease = serde_json::from_slice(body).map_err(|e| format!("unreadable release: {e}"))?;
|
||||
if raw.draft {
|
||||
return Err("the latest release is a draft".into());
|
||||
}
|
||||
let tag = raw.tag_name.filter(|t| !t.is_empty() && t.len() <= 64).ok_or("the release has no tag")?;
|
||||
let releases_page = format!("https://github.com/{repo}/releases/");
|
||||
let html_url = raw.html_url.filter(|u| u.starts_with(&releases_page)).unwrap_or_else(|| format!("{releases_page}tag/{tag}"));
|
||||
let download_prefix = format!("https://github.com/{repo}/releases/download/");
|
||||
let assets = raw
|
||||
.assets
|
||||
.into_iter()
|
||||
.filter_map(|a| {
|
||||
let name = a.name.filter(|n| valid_file_name(n))?;
|
||||
let download_url = a.browser_download_url.filter(|u| u.starts_with(&download_prefix))?;
|
||||
let sha256 = a.digest.as_deref().and_then(|d| d.strip_prefix("sha256:")).filter(|h| h.len() == 64 && h.bytes().all(|b| b.is_ascii_hexdigit())).map(str::to_ascii_lowercase);
|
||||
let platform = parse_platform(&name);
|
||||
Some(CraftAppReleaseAsset { name, size_bytes: a.size, download_url, sha256, platform })
|
||||
})
|
||||
.take(MAX_ASSETS)
|
||||
.collect();
|
||||
Ok(CraftAppRelease {
|
||||
version: tag.strip_prefix(['v', 'V']).unwrap_or(&tag).to_string(),
|
||||
name: raw.name.filter(|n| !n.is_empty()).map(|n| truncate(&n, 200)).unwrap_or_else(|| tag.clone()),
|
||||
tag,
|
||||
published_at: raw.published_at,
|
||||
html_url,
|
||||
prerelease: raw.prerelease,
|
||||
notes_markdown: raw.body.map(|b| truncate(&b, MAX_NOTES_BYTES)).unwrap_or_default(),
|
||||
assets,
|
||||
})
|
||||
}
|
||||
|
||||
/// Reads `<app>-<version>-<os>-<arch>[-portable].<ext>`, `<app>-cli-<version>-macos-universal.zip`
|
||||
/// and `<app>-web-<version>.zip`. `None` for anything else (checksums, unknown files).
|
||||
pub fn parse_platform(name: &str) -> Option<CraftAppAssetPlatform> {
|
||||
if name.contains("-web-") && name.ends_with(".zip") {
|
||||
return Some(CraftAppAssetPlatform { os: CraftAppOs::Web, arch: None, package_type: CraftAppPackageType::WebZip });
|
||||
}
|
||||
const KINDS: [(&str, CraftAppPackageType); 10] = [
|
||||
("-portable.zip", CraftAppPackageType::PortableZip),
|
||||
(".AppImage.zsync", CraftAppPackageType::AppImageZsync),
|
||||
(".AppImage", CraftAppPackageType::AppImage),
|
||||
(".tar.gz", CraftAppPackageType::TarGz),
|
||||
(".msi", CraftAppPackageType::Msi),
|
||||
(".dmg", CraftAppPackageType::Dmg),
|
||||
(".deb", CraftAppPackageType::Deb),
|
||||
(".rpm", CraftAppPackageType::Rpm),
|
||||
(".flatpak", CraftAppPackageType::Flatpak),
|
||||
(".zip", CraftAppPackageType::CliZip),
|
||||
];
|
||||
let (base, package_type) = KINDS.iter().find_map(|(suffix, kind)| name.strip_suffix(suffix).map(|b| (b, *kind)))?;
|
||||
let mut parts = base.rsplitn(3, '-');
|
||||
let (arch, os) = (parts.next()?, parts.next()?);
|
||||
let os = match os {
|
||||
"windows" => CraftAppOs::Windows,
|
||||
"macos" => CraftAppOs::Macos,
|
||||
"linux" => CraftAppOs::Linux,
|
||||
"freebsd" => CraftAppOs::Freebsd,
|
||||
_ => return None,
|
||||
};
|
||||
let arch = match arch {
|
||||
"x64" | "x86_64" => CraftAppArch::X64,
|
||||
"x86" => CraftAppArch::X86,
|
||||
"arm64" | "aarch64" => CraftAppArch::Arm64,
|
||||
"universal" => CraftAppArch::Universal,
|
||||
_ => return None,
|
||||
};
|
||||
// A bare `.zip` is only the macOS command-line tools.
|
||||
if package_type == CraftAppPackageType::CliZip && !(os == CraftAppOs::Macos && base.contains("-cli-")) {
|
||||
return None;
|
||||
}
|
||||
Some(CraftAppAssetPlatform { os, arch: Some(arch), package_type })
|
||||
}
|
||||
|
||||
fn valid_file_name(name: &str) -> bool {
|
||||
!name.is_empty()
|
||||
&& name.len() <= 200
|
||||
&& !name.starts_with('.')
|
||||
&& !name.chars().any(|c| c.is_control() || matches!(c, '/' | '\\' | ':' | '<' | '>' | '"' | '|' | '?' | '*'))
|
||||
}
|
||||
|
||||
fn truncate(s: &str, max: usize) -> String {
|
||||
if s.len() <= max {
|
||||
return s.to_string();
|
||||
}
|
||||
let mut end = max;
|
||||
while !s.is_char_boundary(end) {
|
||||
end -= 1;
|
||||
}
|
||||
format!("{}…", &s[..end])
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
const REPO: &str = "storytold/photocraft";
|
||||
const HASH: &str = "8a5edab282632443219e051e4ade2d1d5bbc671c781051bf1437897cbdfea0f1";
|
||||
|
||||
fn release_json(download_url: &str) -> String {
|
||||
format!(
|
||||
r#"{{"tag_name":"v0.5.0","name":"PhotoCraft v0.5.0","draft":false,"prerelease":false,
|
||||
"html_url":"https://github.com/storytold/photocraft/releases/tag/v0.5.0",
|
||||
"published_at":"2026-10-08T14:52:50Z","body":"Notes","unknown_field":[1,2,3],
|
||||
"assets":[{{"name":"photocraft-0.5.0-windows-x64.msi","size":7,"browser_download_url":"{download_url}","digest":"sha256:{HASH}"}}]}}"#
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sends_the_token_as_a_bearer_header() {
|
||||
let client = build_client().unwrap();
|
||||
let request = build_request(&client, REPO, Some("\"etag\""), Some("ghp_x")).build().unwrap();
|
||||
assert_eq!(request.headers().get(AUTHORIZATION).unwrap(), "Bearer ghp_x");
|
||||
assert_eq!(request.headers().get(IF_NONE_MATCH).unwrap(), "\"etag\"");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn without_a_token_the_request_is_unauthenticated() {
|
||||
let client = build_client().unwrap();
|
||||
for token in [None, Some(""), Some(" ")] {
|
||||
let request = build_request(&client, REPO, None, token).build().unwrap();
|
||||
assert!(request.headers().get(AUTHORIZATION).is_none(), "{:?}", token);
|
||||
assert!(request.headers().get(IF_NONE_MATCH).is_none());
|
||||
assert_eq!(request.uri().to_string(), "https://api.github.com/repos/storytold/photocraft/releases/latest");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reads_a_release() {
|
||||
let r = parse_release(release_json("https://github.com/storytold/photocraft/releases/download/v0.5.0/photocraft-0.5.0-windows-x64.msi").as_bytes(), REPO).unwrap();
|
||||
assert_eq!((r.tag.as_str(), r.version.as_str(), r.name.as_str()), ("v0.5.0", "0.5.0", "PhotoCraft v0.5.0"));
|
||||
assert_eq!(r.assets.len(), 1);
|
||||
assert_eq!(r.assets[0].sha256.as_deref(), Some(HASH));
|
||||
let p = r.assets[0].platform.as_ref().unwrap();
|
||||
assert_eq!((p.os, p.arch, p.package_type), (CraftAppOs::Windows, Some(CraftAppArch::X64), CraftAppPackageType::Msi));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn drops_downloads_that_point_elsewhere_and_rejects_garbage() {
|
||||
for url in ["https://evil.example/x.msi", "https://github.com/storytold/other/releases/download/v1/x.msi"] {
|
||||
assert!(parse_release(release_json(url).as_bytes(), REPO).unwrap().assets.is_empty(), "{url}");
|
||||
}
|
||||
assert!(parse_release(b"<html>", REPO).is_err());
|
||||
assert!(parse_release(br#"{"assets":[]}"#, REPO).is_err());
|
||||
assert!(parse_release(br#"{"tag_name":"v1.0.0","draft":true}"#, REPO).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reads_every_asset_name_the_suite_publishes() {
|
||||
let cases = [
|
||||
("photocraft-0.5.0-macos-universal.dmg", Some((CraftAppOs::Macos, Some(CraftAppArch::Universal), CraftAppPackageType::Dmg))),
|
||||
("photocraft-cli-0.5.0-macos-universal.zip", Some((CraftAppOs::Macos, Some(CraftAppArch::Universal), CraftAppPackageType::CliZip))),
|
||||
("photocraft-0.5.0-windows-arm64-portable.zip", Some((CraftAppOs::Windows, Some(CraftAppArch::Arm64), CraftAppPackageType::PortableZip))),
|
||||
("photocraft-0.5.0-windows-x86.msi", Some((CraftAppOs::Windows, Some(CraftAppArch::X86), CraftAppPackageType::Msi))),
|
||||
("photocraft-0.5.0-linux-aarch64.AppImage", Some((CraftAppOs::Linux, Some(CraftAppArch::Arm64), CraftAppPackageType::AppImage))),
|
||||
("photocraft-0.5.0-linux-x86_64.AppImage.zsync", Some((CraftAppOs::Linux, Some(CraftAppArch::X64), CraftAppPackageType::AppImageZsync))),
|
||||
("photocraft-0.5.0-linux-x86_64.deb", Some((CraftAppOs::Linux, Some(CraftAppArch::X64), CraftAppPackageType::Deb))),
|
||||
("photocraft-0.5.0-linux-x86_64.rpm", Some((CraftAppOs::Linux, Some(CraftAppArch::X64), CraftAppPackageType::Rpm))),
|
||||
("photocraft-0.5.0-linux-x86_64.flatpak", Some((CraftAppOs::Linux, Some(CraftAppArch::X64), CraftAppPackageType::Flatpak))),
|
||||
("photocraft-0.5.0-freebsd-x86_64.tar.gz", Some((CraftAppOs::Freebsd, Some(CraftAppArch::X64), CraftAppPackageType::TarGz))),
|
||||
("photocraft-0.2.0-rc.1-linux-x86_64.tar.gz", Some((CraftAppOs::Linux, Some(CraftAppArch::X64), CraftAppPackageType::TarGz))),
|
||||
("photocraft-web-0.5.0.zip", Some((CraftAppOs::Web, None, CraftAppPackageType::WebZip))),
|
||||
("SHA256SUMS.txt", None),
|
||||
("photocraft-0.5.0-plan9-x64.msi", None),
|
||||
];
|
||||
for (name, want) in cases {
|
||||
let got = parse_platform(name).map(|p| (p.os, p.arch, p.package_type));
|
||||
assert_eq!(got, want, "{name}");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,7 +0,0 @@
|
||||
pub mod config;
|
||||
pub mod fallback;
|
||||
pub mod github;
|
||||
pub mod poll_craft_app_releases;
|
||||
pub mod scheduling;
|
||||
pub mod shared_entry;
|
||||
pub mod shared_store;
|
||||
@@ -1,469 +0,0 @@
|
||||
//! Keeps [`CraftAppsReleaseCache`] holding the latest release of every Craft App, which
|
||||
//! `GET /v1/craft_apps/release_info` serves to ArtCraft Launcher (so thousands of launchers don't
|
||||
//! each poll GitHub).
|
||||
//!
|
||||
//! Every instance runs this loop, waking every [`TICK`] (30 s, with jitter). At startup it first
|
||||
//! takes the fleet's shared releases from Redis, so a new instance serves them at once without
|
||||
//! asking GitHub. With coordination (the default) the fleet shares its work through Redis
|
||||
//! (`shared_entry` documents the keys):
|
||||
//!
|
||||
//! 1. Read every app's shared entry and the fleet's backoff; take anything newer into memory.
|
||||
//! The endpoint always serves memory, so it keeps answering whatever Redis does.
|
||||
//! 2. If the fleet is backing off (GitHub rate limited us, or a whole round failed), stop.
|
||||
//! 3. For each app that is due (its shared fetch is `interval - CLOCK_SKEW_MARGIN` old, and any
|
||||
//! failure retry has waited), read its entry again (another instance may just have fetched
|
||||
//! it), claim it so instances ticking at the same moment don't all ask, then ask GitHub with
|
||||
//! the shared ETag and share the answer. Shared writes never go back in time.
|
||||
//! 4. After a round with requests, update the fleet's backoff (a good round clears it).
|
||||
//!
|
||||
//! Each app is all-or-nothing: either its latest release is stored with a fresh
|
||||
//! `app_last_fetched`, or nothing about it changes.
|
||||
//!
|
||||
//! If Redis is unreachable (or coordination is off) the instance does the same from its own
|
||||
//! memory, as every instance did before coordination, and returns to coordinating when Redis
|
||||
//! answers again. Nothing here panics; trouble is logged and retried.
|
||||
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use chrono::Utc;
|
||||
use log::{error, info, warn};
|
||||
|
||||
use crate::state::craft_apps_release_cache::CraftAppsReleaseCache;
|
||||
use crate::threads::poll_craft_app_releases_thread::config::CraftAppsPollConfig;
|
||||
use crate::threads::poll_craft_app_releases_thread::fallback::{PollMode, RedisTroubleLog};
|
||||
use crate::threads::poll_craft_app_releases_thread::github;
|
||||
use crate::threads::poll_craft_app_releases_thread::github::FetchOutcome;
|
||||
use crate::threads::poll_craft_app_releases_thread::scheduling::{backoff_after, is_due, jittered, Backoff, RoundResult, TICK};
|
||||
use crate::threads::poll_craft_app_releases_thread::shared_entry::{SharedAppEntry, SharedFailure};
|
||||
use crate::threads::poll_craft_app_releases_thread::shared_store::{PollerRedisPool, SharedStore};
|
||||
|
||||
/// Spreads a fleet's first tick over this window, so a deploy doesn't hit GitHub all at once.
|
||||
const STARTUP_JITTER: Duration = Duration::from_secs(15);
|
||||
/// Pause between the requests of one round.
|
||||
const BETWEEN_REQUESTS: Duration = Duration::from_millis(1_000);
|
||||
|
||||
pub async fn poll_craft_app_releases(cache: CraftAppsReleaseCache, config: CraftAppsPollConfig, redis_pool: PollerRedisPool, hostname: String) {
|
||||
cache.register_apps(&config.apps);
|
||||
if !config.enabled {
|
||||
info!("Craft app release polling is disabled (CRAFT_APPS_RELEASE_POLLING_ENABLED=false).");
|
||||
return;
|
||||
}
|
||||
let client = match github::build_client() {
|
||||
Ok(client) => client,
|
||||
Err(err) => {
|
||||
error!("Craft app release polling can't start: no HTTP client: {:?}", err);
|
||||
return;
|
||||
}
|
||||
};
|
||||
info!(
|
||||
"Polling GitHub for {} craft app releases, each at most every {:?} (GitHub token: {}; coordinated through Redis: {}).",
|
||||
config.apps.len(), config.interval, config.github_token.is_some(), config.coordination_enabled,
|
||||
);
|
||||
let store = config.coordination_enabled.then(|| SharedStore::new(redis_pool, hostname));
|
||||
let mut poller = Poller::new(client, cache, config, store);
|
||||
// Warm memory from the fleet's shared copy right away: it costs no GitHub quota, and the
|
||||
// jitter below only needs to spread out requests to GitHub.
|
||||
if poller.refresh_from_shared().await {
|
||||
info!("Craft app releases: took the fleet's shared releases from Redis at startup.");
|
||||
}
|
||||
tokio::time::sleep(jittered(STARTUP_JITTER, 1.0)).await;
|
||||
loop {
|
||||
poller.tick().await;
|
||||
tokio::time::sleep(jittered(TICK, 0.1)).await;
|
||||
}
|
||||
}
|
||||
|
||||
/// The polling loop's state.
|
||||
struct Poller {
|
||||
client: wreq::Client,
|
||||
cache: CraftAppsReleaseCache,
|
||||
config: CraftAppsPollConfig,
|
||||
/// `None` when coordination is off.
|
||||
store: Option<SharedStore>,
|
||||
/// The backoff this instance knows: the fleet's while Redis answers, its own otherwise.
|
||||
backoff: Backoff,
|
||||
redis_log: RedisTroubleLog,
|
||||
}
|
||||
|
||||
/// What one tick did.
|
||||
#[derive(Debug)]
|
||||
struct TickReport {
|
||||
mode: PollMode,
|
||||
round: RoundResult,
|
||||
}
|
||||
|
||||
/// Whether to ask GitHub for an app, after looking at its shared entry once more.
|
||||
enum SharedCheck {
|
||||
Ask,
|
||||
/// Fresh after all, or another instance is asking right now.
|
||||
Skip,
|
||||
/// The fleet started backing off.
|
||||
StopRound,
|
||||
}
|
||||
|
||||
impl Poller {
|
||||
fn new(client: wreq::Client, cache: CraftAppsReleaseCache, config: CraftAppsPollConfig, store: Option<SharedStore>) -> Self {
|
||||
Self { client, cache, config, store, backoff: Backoff::default(), redis_log: RedisTroubleLog::default() }
|
||||
}
|
||||
|
||||
async fn tick(&mut self) -> TickReport {
|
||||
let shared_state_readable = self.refresh_from_shared().await;
|
||||
let mode = PollMode::choose(self.store.is_some(), shared_state_readable);
|
||||
let mut report = TickReport { mode, round: RoundResult::default() };
|
||||
let now = Utc::now();
|
||||
if self.backoff.is_active(now) {
|
||||
return report;
|
||||
}
|
||||
let due: Vec<(String, String)> = self
|
||||
.config
|
||||
.apps
|
||||
.iter()
|
||||
.filter(|(key, _, _)| self.cache.app(key).is_some_and(|app| is_due(&app, now, self.config.interval)))
|
||||
.map(|(key, _, repo)| (key.clone(), repo.clone()))
|
||||
.collect();
|
||||
if due.is_empty() {
|
||||
return report;
|
||||
}
|
||||
let (round, mode) = self.poll_round(&due, mode).await;
|
||||
report.round = round;
|
||||
report.mode = mode;
|
||||
if report.round.requests() == 0 {
|
||||
return report;
|
||||
}
|
||||
self.update_backoff(&report.round, mode).await;
|
||||
info!(
|
||||
"Craft app releases ({:?}): {} fetched, {} unchanged, {} failed{}{}.",
|
||||
mode, report.round.fetched, report.round.unchanged, report.round.failed,
|
||||
if report.round.rate_limited.is_some() { ", rate limited" } else { "" },
|
||||
self.backoff.until.map(|until| format!("; backing off until {until}")).unwrap_or_default(),
|
||||
);
|
||||
report
|
||||
}
|
||||
|
||||
/// Takes the fleet's shared state into memory. False when coordination is off or Redis
|
||||
/// couldn't be read (this tick then works alone).
|
||||
async fn refresh_from_shared(&mut self) -> bool {
|
||||
let Some(store) = self.store.clone() else {
|
||||
return false;
|
||||
};
|
||||
let keys: Vec<String> = self.config.apps.iter().map(|(key, _, _)| key.clone()).collect();
|
||||
match store.read(&keys).await {
|
||||
Ok((entries, backoff)) => {
|
||||
self.redis_log.succeeded(Instant::now());
|
||||
for (key, entry) in keys.iter().zip(entries) {
|
||||
if let Some(entry) = entry {
|
||||
merge_shared_entry(&self.cache, key, entry);
|
||||
}
|
||||
}
|
||||
self.backoff = backoff;
|
||||
true
|
||||
}
|
||||
Err(err) => {
|
||||
self.redis_log.failed(&err, Instant::now());
|
||||
false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Asks GitHub about each due app. Returns what happened and the mode it ended in (a Redis
|
||||
/// error mid-round finishes the round alone).
|
||||
async fn poll_round(&mut self, due: &[(String, String)], mut mode: PollMode) -> (RoundResult, PollMode) {
|
||||
let mut round = RoundResult::default();
|
||||
for (key, repo) in due {
|
||||
if let Some(store) = self.coordinated_store(mode) {
|
||||
match self.check_shared(&store, key).await {
|
||||
Ok(SharedCheck::Ask) => {}
|
||||
Ok(SharedCheck::Skip) => continue,
|
||||
Ok(SharedCheck::StopRound) => break,
|
||||
Err(err) => {
|
||||
self.redis_log.failed(&err, Instant::now());
|
||||
mode = PollMode::Standalone;
|
||||
}
|
||||
}
|
||||
}
|
||||
if round.requests() > 0 {
|
||||
tokio::time::sleep(BETWEEN_REQUESTS).await;
|
||||
}
|
||||
let etag = self.cache.etag(key);
|
||||
let outcome = github::fetch_latest_release(&self.client, repo, etag.as_deref(), self.config.github_token.as_deref()).await;
|
||||
let now = Utc::now();
|
||||
match outcome {
|
||||
FetchOutcome::Fetched { release, etag } => {
|
||||
round.fetched += 1;
|
||||
if let Some(store) = self.coordinated_store(mode) {
|
||||
let shared = store.write_fetched(key, release.as_ref(), etag.as_deref(), now).await;
|
||||
mode = self.after_shared_write(shared, mode);
|
||||
}
|
||||
self.cache.store(key, release, etag, now);
|
||||
}
|
||||
FetchOutcome::NotModified => {
|
||||
round.unchanged += 1;
|
||||
self.cache.confirm_unchanged(key, now);
|
||||
if let (Some(store), Some(etag)) = (self.coordinated_store(mode), etag.as_deref()) {
|
||||
let shared = store.write_unchanged(key, etag, now).await;
|
||||
mode = self.after_shared_write(shared, mode);
|
||||
}
|
||||
}
|
||||
FetchOutcome::RateLimited { retry_at } => {
|
||||
warn!("GitHub rate limit reached polling {repo}; retry after {:?}.", retry_at);
|
||||
round.rate_limited = Some(retry_at);
|
||||
// The rest of the round would be refused too.
|
||||
break;
|
||||
}
|
||||
FetchOutcome::Failed(why) => {
|
||||
warn!("Couldn't fetch the latest release of {repo}: {why}");
|
||||
round.failed += 1;
|
||||
let consecutive_failures = self.cache.record_failure(key, now);
|
||||
if let Some(store) = self.coordinated_store(mode) {
|
||||
let failure = SharedFailure { failed_at: now, consecutive_failures };
|
||||
let shared = store.write_failure(key, failure).await;
|
||||
mode = self.after_shared_write(shared, mode);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
(round, mode)
|
||||
}
|
||||
|
||||
/// Right before asking GitHub about `key`: takes its latest shared entry, and claims it if it
|
||||
/// is still due and the fleet isn't backing off.
|
||||
async fn check_shared(&mut self, store: &SharedStore, key: &str) -> anyhow::Result<SharedCheck> {
|
||||
let (mut entries, backoff) = store.read(&[key.to_string()]).await?;
|
||||
if let Some(entry) = entries.pop().flatten() {
|
||||
merge_shared_entry(&self.cache, key, entry);
|
||||
}
|
||||
self.backoff = backoff;
|
||||
let now = Utc::now();
|
||||
if backoff.is_active(now) {
|
||||
return Ok(SharedCheck::StopRound);
|
||||
}
|
||||
if !self.cache.app(key).is_some_and(|app| is_due(&app, now, self.config.interval)) {
|
||||
return Ok(SharedCheck::Skip);
|
||||
}
|
||||
Ok(if store.claim(key, now).await? { SharedCheck::Ask } else { SharedCheck::Skip })
|
||||
}
|
||||
|
||||
/// Sets the backoff after a round with requests, and shares it if it changed.
|
||||
async fn update_backoff(&mut self, round: &RoundResult, mode: PollMode) {
|
||||
let previous = self.backoff;
|
||||
self.backoff = backoff_after(&self.config, round, previous, Utc::now());
|
||||
if self.backoff == previous {
|
||||
return;
|
||||
}
|
||||
if let Some(store) = self.coordinated_store(mode) {
|
||||
if let Err(err) = store.write_backoff(self.backoff).await {
|
||||
self.redis_log.failed(&err, Instant::now());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Logs a failed shared write; the rest of the round then works alone.
|
||||
fn after_shared_write(&mut self, result: anyhow::Result<bool>, mode: PollMode) -> PollMode {
|
||||
match result {
|
||||
Ok(_) => mode,
|
||||
Err(err) => {
|
||||
self.redis_log.failed(&err, Instant::now());
|
||||
PollMode::Standalone
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The shared store, in [`PollMode::Coordinated`].
|
||||
fn coordinated_store(&self, mode: PollMode) -> Option<SharedStore> {
|
||||
self.store.clone().filter(|_| mode == PollMode::Coordinated)
|
||||
}
|
||||
}
|
||||
|
||||
/// Takes a shared entry into memory (only what is newer than what this instance has).
|
||||
fn merge_shared_entry(cache: &CraftAppsReleaseCache, key: &str, entry: SharedAppEntry) {
|
||||
if let Some(fetched) = entry.fetched {
|
||||
cache.merge_fetched(key, fetched.latest_release, fetched.etag, fetched.fetched_at);
|
||||
}
|
||||
if let Some(failure) = entry.failure {
|
||||
cache.merge_failure(key, failure.failed_at, failure.consecutive_failures);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use chrono::DateTime;
|
||||
|
||||
use redis_schema::keys::craft_apps::craft_app_release_redis_key::CraftAppReleaseRedisKey;
|
||||
|
||||
use crate::threads::poll_craft_app_releases_thread::config::default_apps;
|
||||
use crate::threads::poll_craft_app_releases_thread::shared_entry::{decode_app_entry, encode_fetch};
|
||||
use crate::threads::poll_craft_app_releases_thread::shared_store::build_poller_redis_pool;
|
||||
|
||||
use super::*;
|
||||
|
||||
/// Live: real rounds against GitHub (two apps), then the endpoint's response, written to
|
||||
/// `CRAFT_APPS_SAMPLE_OUT` when set (ArtCraft Launcher's parser reads it back). Uses
|
||||
/// `CRAFT_APPS_RELEASE_GITHUB_TOKEN` if set, and works without it.
|
||||
/// `cargo test -p storyteller-web --bin storyteller-web live_round -- --ignored`
|
||||
#[tokio::test]
|
||||
#[ignore = "needs network access"]
|
||||
async fn live_round_fetches_real_releases() {
|
||||
use artcraft_api_defs::craft_apps::release_info::{CraftAppsImportantNotices, GetCraftAppsReleaseInfoResponse};
|
||||
let mut config = config();
|
||||
config.github_token = std::env::var("CRAFT_APPS_RELEASE_GITHUB_TOKEN").ok().filter(|t| !t.trim().is_empty());
|
||||
config.apps.retain(|(key, _, _)| key == "photocraft" || key == "artcraft-launcher");
|
||||
let cache = CraftAppsReleaseCache::new();
|
||||
cache.register_apps(&config.apps);
|
||||
let mut poller = Poller::new(github::build_client().unwrap(), cache.clone(), config.clone(), None);
|
||||
let due: Vec<(String, String)> = config.apps.iter().map(|(k, _, r)| (k.clone(), r.clone())).collect();
|
||||
let (round, _) = poller.poll_round(&due, PollMode::Standalone).await;
|
||||
assert_eq!(round.fetched + round.unchanged, 2, "{round:?}");
|
||||
// A second round revalidates with ETags.
|
||||
let (again, _) = poller.poll_round(&due, PollMode::Standalone).await;
|
||||
assert_eq!(again.fetched + again.unchanged, 2, "{again:?}");
|
||||
let info = cache.snapshot();
|
||||
assert!(info.last_fetched.is_some());
|
||||
let photocraft = info.apps["photocraft"].latest_release.as_ref().unwrap();
|
||||
assert!(photocraft.assets.iter().any(|a| a.platform.is_some() && a.sha256.is_some()));
|
||||
let response = GetCraftAppsReleaseInfoResponse {
|
||||
success: true,
|
||||
release_info: info,
|
||||
app_renames: Vec::new(),
|
||||
repo_renames: Vec::new(),
|
||||
important_notices: CraftAppsImportantNotices::default(),
|
||||
};
|
||||
let json = serde_json::to_string_pretty(&response).unwrap();
|
||||
if let Ok(path) = std::env::var("CRAFT_APPS_SAMPLE_OUT") {
|
||||
std::fs::write(path, &json).unwrap();
|
||||
}
|
||||
}
|
||||
|
||||
/// Against a real Redis (`CRAFT_APPS_TEST_REDIS_URL`, default `redis://127.0.0.1:6379/`): an
|
||||
/// instance takes another's fresh fetches from Redis instead of asking GitHub.
|
||||
/// `cargo test -p storyteller-web --bin storyteller-web takes_fresh -- --ignored`
|
||||
#[tokio::test]
|
||||
#[ignore = "needs a Redis server"]
|
||||
async fn takes_fresh_releases_from_redis_instead_of_github() {
|
||||
let url = std::env::var("CRAFT_APPS_TEST_REDIS_URL").unwrap_or_else(|_| "redis://127.0.0.1:6379/".to_string());
|
||||
let pool = build_poller_redis_pool(&url).unwrap();
|
||||
let mut config = config();
|
||||
let prefix = format!("test-craft-{}", rand::random::<u32>());
|
||||
config.apps = vec![(format!("{prefix}-a"), "A".into(), "storytold/a".into()), (format!("{prefix}-b"), "B".into(), "storytold/b".into())];
|
||||
let pod_a = SharedStore::new(pool.clone(), "pod-a".into());
|
||||
let now = Utc::now();
|
||||
for (key, _, _) in &config.apps {
|
||||
assert!(pod_a.write_fetched(key, None, Some("\"e\""), now).await.unwrap());
|
||||
}
|
||||
|
||||
let cache = CraftAppsReleaseCache::new();
|
||||
cache.register_apps(&config.apps);
|
||||
let mut pod_b = Poller::new(github::build_client().unwrap(), cache.clone(), config.clone(), Some(SharedStore::new(pool.clone(), "pod-b".into())));
|
||||
let report = pod_b.tick().await;
|
||||
assert_eq!(report.mode, PollMode::Coordinated);
|
||||
assert_eq!(report.round.requests(), 0, "fresh in Redis, so GitHub isn't asked");
|
||||
for (key, _, _) in &config.apps {
|
||||
assert_eq!(fetched_at(&cache, key).map(|t| t.timestamp_millis()), Some(now.timestamp_millis()));
|
||||
assert_eq!(cache.etag(key).as_deref(), Some("\"e\""));
|
||||
}
|
||||
|
||||
let keys: Vec<String> = config.apps.iter().map(|(k, _, _)| CraftAppReleaseRedisKey::new_for_app(k).to_string()).collect();
|
||||
let mut conn = pool.get().unwrap();
|
||||
redis::cmd("DEL").arg(&keys).query::<()>(&mut *conn).unwrap();
|
||||
}
|
||||
|
||||
/// Against a real Redis, as above: a new instance serves the fleet's releases right after
|
||||
/// startup, before its startup jitter ends, without asking GitHub.
|
||||
/// `cargo test -p storyteller-web --bin storyteller-web warm -- --ignored`
|
||||
#[tokio::test]
|
||||
#[ignore = "needs a Redis server"]
|
||||
async fn a_new_instance_is_warm_from_redis_at_startup() {
|
||||
let url = std::env::var("CRAFT_APPS_TEST_REDIS_URL").unwrap_or_else(|_| "redis://127.0.0.1:6379/".to_string());
|
||||
let pool = build_poller_redis_pool(&url).unwrap();
|
||||
let mut config = config();
|
||||
let prefix = format!("test-craft-{}", rand::random::<u32>());
|
||||
config.apps = vec![(format!("{prefix}-a"), "A".into(), "storytold/a".into()), (format!("{prefix}-b"), "B".into(), "storytold/b".into())];
|
||||
let fetched_by_another_pod = Utc::now();
|
||||
let other_pod = SharedStore::new(pool.clone(), "other-pod".into());
|
||||
for (key, _, _) in &config.apps {
|
||||
assert!(other_pod.write_fetched(key, None, Some("\"e\""), fetched_by_another_pod).await.unwrap());
|
||||
}
|
||||
|
||||
let cache = CraftAppsReleaseCache::new();
|
||||
let poller = tokio::spawn(poll_craft_app_releases(cache.clone(), config.clone(), pool.clone(), "new-pod".into()));
|
||||
let deadline = Instant::now() + Duration::from_secs(2);
|
||||
while config.apps.iter().any(|(key, _, _)| fetched_at(&cache, key).is_none()) && Instant::now() < deadline {
|
||||
tokio::time::sleep(Duration::from_millis(20)).await;
|
||||
}
|
||||
poller.abort();
|
||||
|
||||
for (key, _, _) in &config.apps {
|
||||
assert_eq!(fetched_at(&cache, key).map(|t| t.timestamp_millis()), Some(fetched_by_another_pod.timestamp_millis()), "{}", key);
|
||||
}
|
||||
let keys: Vec<String> = config.apps.iter().map(|(k, _, _)| CraftAppReleaseRedisKey::new_for_app(k).to_string()).collect();
|
||||
let mut conn = pool.get().unwrap();
|
||||
redis::cmd("DEL").arg(&keys).query::<()>(&mut *conn).unwrap();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn an_unreachable_redis_falls_back_to_polling_alone() {
|
||||
let pool = build_poller_redis_pool("redis://127.0.0.1:1/").unwrap();
|
||||
let mut poller = fresh_poller(Some(SharedStore::new(pool, "pod".into())));
|
||||
let report = poller.tick().await;
|
||||
assert_eq!(report.mode, PollMode::Standalone);
|
||||
assert_eq!(report.round.requests(), 0, "everything was fresh");
|
||||
assert!(poller.redis_log.is_failing());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn with_coordination_off_redis_is_never_used() {
|
||||
let mut poller = fresh_poller(None);
|
||||
let report = poller.tick().await;
|
||||
assert_eq!(report.mode, PollMode::Standalone);
|
||||
assert!(!poller.redis_log.is_failing());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn an_active_backoff_skips_github_even_when_apps_are_due() {
|
||||
let mut poller = Poller::new(github::build_client().unwrap(), registered_cache(), config(), None);
|
||||
poller.backoff = Backoff { until: Some(Utc::now() + chrono::Duration::minutes(5)), trouble: 1 };
|
||||
let report = poller.tick().await;
|
||||
assert_eq!(report.round.requests(), 0);
|
||||
assert_eq!(fetched_at(&poller.cache, "photocraft"), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn shared_entries_merge_into_memory() {
|
||||
let cache = registered_cache();
|
||||
let at = Utc::now();
|
||||
let fields = encode_fetch(None, Some("\"e\""), at, "pod").unwrap();
|
||||
let map = fields.into_iter().map(|(k, v)| (k.to_string(), v)).collect();
|
||||
merge_shared_entry(&cache, "photocraft", decode_app_entry(&map).unwrap());
|
||||
assert_eq!(fetched_at(&cache, "photocraft").map(|t| t.timestamp_millis()), Some(at.timestamp_millis()));
|
||||
assert_eq!(cache.etag("photocraft").as_deref(), Some("\"e\""));
|
||||
}
|
||||
|
||||
fn config() -> CraftAppsPollConfig {
|
||||
CraftAppsPollConfig {
|
||||
enabled: true,
|
||||
coordination_enabled: true,
|
||||
interval: Duration::from_secs(1800),
|
||||
max_backoff: Duration::from_secs(14_400),
|
||||
github_token: None,
|
||||
apps: default_apps(),
|
||||
}
|
||||
}
|
||||
|
||||
fn registered_cache() -> CraftAppsReleaseCache {
|
||||
let cache = CraftAppsReleaseCache::new();
|
||||
cache.register_apps(&config().apps);
|
||||
cache
|
||||
}
|
||||
|
||||
/// A poller whose apps were all just fetched, so a tick makes no GitHub requests.
|
||||
fn fresh_poller(store: Option<SharedStore>) -> Poller {
|
||||
let cache = registered_cache();
|
||||
for (key, _, _) in &config().apps {
|
||||
cache.store(key, None, None, Utc::now());
|
||||
}
|
||||
Poller::new(github::build_client().unwrap(), cache, config(), store)
|
||||
}
|
||||
|
||||
fn fetched_at(cache: &CraftAppsReleaseCache, app_key: &str) -> Option<DateTime<Utc>> {
|
||||
cache.app(app_key).and_then(|app| app.app_last_fetched)
|
||||
}
|
||||
}
|
||||
@@ -1,286 +0,0 @@
|
||||
//! When to ask GitHub: per-app freshness, retries after failures, and the backoff after rate
|
||||
//! limits. Pure functions of what's known, so the same rules apply whether this instance works
|
||||
//! from the fleet's shared state in Redis or (as a fallback) only from its own memory.
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
|
||||
use crate::state::craft_apps_release_cache::CachedCraftApp;
|
||||
use crate::threads::poll_craft_app_releases_thread::config::CraftAppsPollConfig;
|
||||
|
||||
/// How often every instance wakes up to look at the shared state (with jitter).
|
||||
pub const TICK: Duration = Duration::from_secs(30);
|
||||
/// Allowance for clocks that differ between instances: an app fetched by another instance
|
||||
/// counts as fresh until `interval - CLOCK_SKEW_MARGIN` has passed.
|
||||
pub const CLOCK_SKEW_MARGIN: Duration = Duration::from_secs(30);
|
||||
/// First wait after a failed request (or a round where every request failed); doubles up to the
|
||||
/// normal interval.
|
||||
const FIRST_FAILURE_RETRY: Duration = Duration::from_secs(60);
|
||||
/// Extra margin after GitHub's announced rate-limit reset.
|
||||
const RESET_MARGIN: Duration = Duration::from_secs(60);
|
||||
|
||||
/// What one round did.
|
||||
#[derive(Debug, Default)]
|
||||
pub struct RoundResult {
|
||||
pub fetched: usize,
|
||||
pub unchanged: usize,
|
||||
pub failed: usize,
|
||||
/// GitHub refused; `Some(Some(t))` when it said when to come back.
|
||||
pub rate_limited: Option<Option<DateTime<Utc>>>,
|
||||
}
|
||||
|
||||
/// Don't ask GitHub before `until`. `trouble` counts the rounds in a row that were rate limited
|
||||
/// or failed entirely; each one makes the next wait longer.
|
||||
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
|
||||
pub struct Backoff {
|
||||
pub until: Option<DateTime<Utc>>,
|
||||
pub trouble: u32,
|
||||
}
|
||||
|
||||
impl RoundResult {
|
||||
/// How many requests reached a conclusion (a rate limit counts as one).
|
||||
pub fn requests(&self) -> usize {
|
||||
self.fetched + self.unchanged + self.failed + usize::from(self.rate_limited.is_some())
|
||||
}
|
||||
}
|
||||
|
||||
impl Backoff {
|
||||
pub fn is_active(&self, now: DateTime<Utc>) -> bool {
|
||||
self.until.is_some_and(|until| until > now)
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether `app` should be requested from GitHub now. An app is due once its last fetch (by any
|
||||
/// instance) is `interval - CLOCK_SKEW_MARGIN` old. After a failure it waits a minute, then two,
|
||||
/// four, … up to the interval, so one broken repository is never retried every tick.
|
||||
pub fn is_due(app: &CachedCraftApp, now: DateTime<Utc>, interval: Duration) -> bool {
|
||||
if let Some(fetched) = app.app_last_fetched {
|
||||
if age(fetched, now) < interval.saturating_sub(CLOCK_SKEW_MARGIN) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if app.is_failing() {
|
||||
if let Some(failed) = app.last_failed_at {
|
||||
let retry = scaled(FIRST_FAILURE_RETRY, app.consecutive_failures).min(interval);
|
||||
if age(failed, now) < retry.saturating_sub(CLOCK_SKEW_MARGIN) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
/// The backoff after a round that made at least one request, starting from `previous`. A good
|
||||
/// round clears it.
|
||||
pub fn backoff_after(config: &CraftAppsPollConfig, round: &RoundResult, previous: Backoff, now: DateTime<Utc>) -> Backoff {
|
||||
let mut trouble = previous.trouble;
|
||||
let wait = next_wait(config, round, &mut trouble, now);
|
||||
if trouble == 0 {
|
||||
return Backoff::default();
|
||||
}
|
||||
let wait = chrono::Duration::from_std(wait).unwrap_or(chrono::Duration::MAX);
|
||||
Backoff { until: now.checked_add_signed(wait), trouble }
|
||||
}
|
||||
|
||||
/// How long to wait before the next round. Rate limits wait for GitHub's reset and back off
|
||||
/// further each time; rounds where everything failed retry sooner, but never faster than a
|
||||
/// minute, doubling up to the normal interval.
|
||||
fn next_wait(config: &CraftAppsPollConfig, round: &RoundResult, consecutive_trouble: &mut u32, now: DateTime<Utc>) -> Duration {
|
||||
let base = if let Some(retry_at) = round.rate_limited {
|
||||
*consecutive_trouble = consecutive_trouble.saturating_add(1);
|
||||
let until_reset = retry_at
|
||||
.and_then(|t| (t - now).to_std().ok())
|
||||
.map(|d| d + RESET_MARGIN)
|
||||
.unwrap_or(Duration::ZERO);
|
||||
let backoff = scaled(config.interval, *consecutive_trouble);
|
||||
until_reset.max(backoff).min(config.max_backoff)
|
||||
} else if round.fetched == 0 && round.unchanged == 0 && round.failed > 0 {
|
||||
*consecutive_trouble = consecutive_trouble.saturating_add(1);
|
||||
scaled(FIRST_FAILURE_RETRY, *consecutive_trouble).min(config.interval)
|
||||
} else {
|
||||
*consecutive_trouble = 0;
|
||||
config.interval
|
||||
};
|
||||
jittered(base, 0.1)
|
||||
}
|
||||
|
||||
/// `base` ± `fraction`, so a fleet doesn't poll in lockstep. With `fraction == 1.0`, anywhere in
|
||||
/// `0..=2×base`; used for the startup spread.
|
||||
pub fn jittered(base: Duration, fraction: f64) -> Duration {
|
||||
let factor = 1.0 + fraction * (rand::random::<f64>() * 2.0 - 1.0);
|
||||
base.mul_f64(factor.max(0.0))
|
||||
}
|
||||
|
||||
/// `base × 2^(n-1)`, without overflow.
|
||||
fn scaled(base: Duration, n: u32) -> Duration {
|
||||
base.saturating_mul(1u32 << n.saturating_sub(1).min(16))
|
||||
}
|
||||
|
||||
/// How long ago `then` was; zero if it is in the future (another instance's clock runs ahead).
|
||||
fn age(then: DateTime<Utc>, now: DateTime<Utc>) -> Duration {
|
||||
(now - then).to_std().unwrap_or(Duration::ZERO)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use crate::threads::poll_craft_app_releases_thread::config::default_apps;
|
||||
|
||||
use super::*;
|
||||
|
||||
mod freshness {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn an_app_never_fetched_is_due() {
|
||||
assert!(is_due(&app(None, None, 0), Utc::now(), Duration::from_secs(120)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_fetch_counts_as_fresh_until_the_interval_minus_the_skew_margin() {
|
||||
let now = Utc::now();
|
||||
let interval = Duration::from_secs(120);
|
||||
assert!(!is_due(&app(Some(now - secs(89)), None, 0), now, interval));
|
||||
assert!(is_due(&app(Some(now - secs(90)), None, 0), now, interval));
|
||||
assert!(is_due(&app(Some(now - secs(600)), None, 0), now, interval));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_fetch_from_a_clock_ahead_of_ours_is_fresh() {
|
||||
let now = Utc::now();
|
||||
assert!(!is_due(&app(Some(now + secs(20)), None, 0), now, Duration::from_secs(120)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_failing_app_waits_longer_after_each_failure_up_to_the_interval() {
|
||||
let now = Utc::now();
|
||||
let interval = Duration::from_secs(1800);
|
||||
// 1st failure: retry after 60 s (less the margin); 3rd: 240 s.
|
||||
assert!(!is_due(&app(None, Some(now - secs(29)), 1), now, interval));
|
||||
assert!(is_due(&app(None, Some(now - secs(30)), 1), now, interval));
|
||||
assert!(!is_due(&app(None, Some(now - secs(200)), 3), now, interval));
|
||||
assert!(is_due(&app(None, Some(now - secs(210)), 3), now, interval));
|
||||
// Many failures: never longer than the interval.
|
||||
assert!(is_due(&app(None, Some(now - secs(1770)), 30), now, interval));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_failure_older_than_the_last_fetch_is_ignored() {
|
||||
let now = Utc::now();
|
||||
let app = app(Some(now - secs(200)), Some(now - secs(300)), 5);
|
||||
assert!(is_due(&app, now, Duration::from_secs(120)));
|
||||
}
|
||||
}
|
||||
|
||||
mod backoff {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn a_good_round_clears_the_backoff() {
|
||||
let now = Utc::now();
|
||||
let round = RoundResult { fetched: 1, unchanged: 3, failed: 1, ..RoundResult::default() };
|
||||
let previous = Backoff { until: Some(now), trouble: 3 };
|
||||
assert_eq!(backoff_after(&config(), &round, previous, now), Backoff::default());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_rate_limit_backs_off_the_fleet_until_after_the_reset() {
|
||||
let now = Utc::now();
|
||||
let round = RoundResult { rate_limited: Some(Some(now + chrono::Duration::minutes(50))), ..RoundResult::default() };
|
||||
let b = backoff_after(&config(), &round, Backoff::default(), now);
|
||||
assert_eq!(b.trouble, 1);
|
||||
assert!(b.is_active(now + chrono::Duration::minutes(45)));
|
||||
assert!(!b.is_active(now + chrono::Duration::minutes(57)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn trouble_carries_over_between_rounds_and_instances() {
|
||||
let now = Utc::now();
|
||||
let round = RoundResult { failed: 2, ..RoundResult::default() };
|
||||
let first = backoff_after(&config(), &round, Backoff::default(), now);
|
||||
let second = backoff_after(&config(), &round, first, now);
|
||||
assert_eq!((first.trouble, second.trouble), (1, 2));
|
||||
assert!(second.until.unwrap() - now >= chrono::Duration::seconds(107), "{:?}", second);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_expired_backoff_is_not_active() {
|
||||
let now = Utc::now();
|
||||
assert!(!Backoff::default().is_active(now));
|
||||
assert!(!Backoff { until: Some(now - secs(1)), trouble: 4 }.is_active(now));
|
||||
assert!(Backoff { until: Some(now + secs(1)), trouble: 4 }.is_active(now));
|
||||
}
|
||||
}
|
||||
|
||||
mod waits {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn a_good_round_waits_the_interval_and_resets_trouble() {
|
||||
let mut trouble = 3;
|
||||
let round = RoundResult { fetched: 5, unchanged: 8, ..RoundResult::default() };
|
||||
assert!(within(next_wait(&config(), &round, &mut trouble, Utc::now()), Duration::from_secs(1800)));
|
||||
assert_eq!(trouble, 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rate_limits_wait_for_the_reset_and_back_off_further_each_time() {
|
||||
let now = Utc::now();
|
||||
let mut trouble = 0;
|
||||
// GitHub says come back in 50 minutes: wait that (plus margin), more than the interval.
|
||||
let round = RoundResult { rate_limited: Some(Some(now + chrono::Duration::minutes(50))), ..RoundResult::default() };
|
||||
assert!(within(next_wait(&config(), &round, &mut trouble, now), Duration::from_secs(51 * 60)));
|
||||
// Again and again, with no reset time: 2×, 4×, … the interval, capped at the maximum.
|
||||
let round = RoundResult { rate_limited: Some(None), ..RoundResult::default() };
|
||||
assert!(within(next_wait(&config(), &round, &mut trouble, now), Duration::from_secs(3600)));
|
||||
assert!(within(next_wait(&config(), &round, &mut trouble, now), Duration::from_secs(7200)));
|
||||
for _ in 0..10 {
|
||||
assert!(next_wait(&config(), &round, &mut trouble, now) <= Duration::from_secs(14_400).mul_f64(1.11));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_round_where_everything_failed_retries_sooner_but_not_in_a_tight_loop() {
|
||||
let now = Utc::now();
|
||||
let mut trouble = 0;
|
||||
let round = RoundResult { failed: 13, ..RoundResult::default() };
|
||||
assert!(within(next_wait(&config(), &round, &mut trouble, now), Duration::from_secs(60)));
|
||||
assert!(within(next_wait(&config(), &round, &mut trouble, now), Duration::from_secs(120)));
|
||||
for _ in 0..20 {
|
||||
let w = next_wait(&config(), &round, &mut trouble, now);
|
||||
assert!(w >= Duration::from_secs(53) && w <= Duration::from_secs(1800).mul_f64(1.11), "{:?}", w);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn config() -> CraftAppsPollConfig {
|
||||
CraftAppsPollConfig {
|
||||
enabled: true,
|
||||
coordination_enabled: true,
|
||||
interval: Duration::from_secs(1800),
|
||||
max_backoff: Duration::from_secs(14_400),
|
||||
github_token: None,
|
||||
apps: default_apps(),
|
||||
}
|
||||
}
|
||||
|
||||
fn app(fetched: Option<DateTime<Utc>>, failed: Option<DateTime<Utc>>, failures: u32) -> CachedCraftApp {
|
||||
CachedCraftApp {
|
||||
name: "PhotoCraft".into(),
|
||||
repo: "storytold/photocraft".into(),
|
||||
latest_release: None,
|
||||
app_last_fetched: fetched,
|
||||
etag: None,
|
||||
last_failed_at: failed,
|
||||
consecutive_failures: failures,
|
||||
}
|
||||
}
|
||||
|
||||
fn secs(n: i64) -> chrono::Duration {
|
||||
chrono::Duration::seconds(n)
|
||||
}
|
||||
|
||||
fn within(d: Duration, around: Duration) -> bool {
|
||||
d >= around.mul_f64(0.89) && d <= around.mul_f64(1.11)
|
||||
}
|
||||
}
|
||||
@@ -1,267 +0,0 @@
|
||||
//! What the fleet shares in Redis about Craft App releases, and how it is encoded. Pure: the
|
||||
//! Redis I/O is in `shared_store`.
|
||||
//!
|
||||
//! ## `craftApps:releases:app:<app key>` (hash, 7 day TTL, refreshed on every write)
|
||||
//!
|
||||
//! | Field | Meaning |
|
||||
//! |--------------|--------------------------------------------------------------------------------|
|
||||
//! | `v` | Format version, `1`. Entries with another version are ignored (and replaced) |
|
||||
//! | `fetched_ms` | When the release was last fetched or confirmed by GitHub (Unix ms) |
|
||||
//! | `etag` | GitHub's ETag for that answer; empty for none |
|
||||
//! | `release` | The latest release as JSON (`CraftAppRelease`), `null` if the repo has none |
|
||||
//! | `failed_ms` | When a request for the app last failed (Unix ms); stale once `fetched_ms` newer |
|
||||
//! | `failures` | Failed requests in a row as of `failed_ms` |
|
||||
//! | `claimed_ms` | When an instance last started a request for the app (Unix ms), to skip doubles |
|
||||
//! | `by` | Hostname of the instance that wrote last, for debugging |
|
||||
//!
|
||||
//! ## `craftApps:releases:backoff` (hash, 1 day TTL; deleted after a good round)
|
||||
//!
|
||||
//! | Field | Meaning |
|
||||
//! |------------|-----------------------------------------------------------------|
|
||||
//! | `v` | Format version, `1` |
|
||||
//! | `until_ms` | No instance asks GitHub before this time (Unix ms) |
|
||||
//! | `trouble` | Rounds in a row that were rate limited or failed entirely |
|
||||
//! | `by` | Hostname of the instance that wrote last, for debugging |
|
||||
//!
|
||||
//! Anything unreadable (wrong version, bad numbers, bad JSON) reads as absent, so it is
|
||||
//! refetched and overwritten rather than trusted.
|
||||
|
||||
use std::collections::HashMap;
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
|
||||
use artcraft_api_defs::craft_apps::release_info::CraftAppRelease;
|
||||
|
||||
use crate::threads::poll_craft_app_releases_thread::scheduling::Backoff;
|
||||
|
||||
pub const FORMAT_VERSION: &str = "1";
|
||||
|
||||
pub const FIELD_VERSION: &str = "v";
|
||||
pub const FIELD_FETCHED_MS: &str = "fetched_ms";
|
||||
pub const FIELD_ETAG: &str = "etag";
|
||||
pub const FIELD_RELEASE: &str = "release";
|
||||
pub const FIELD_FAILED_MS: &str = "failed_ms";
|
||||
pub const FIELD_FAILURES: &str = "failures";
|
||||
pub const FIELD_CLAIMED_MS: &str = "claimed_ms";
|
||||
pub const FIELD_BY: &str = "by";
|
||||
pub const FIELD_UNTIL_MS: &str = "until_ms";
|
||||
pub const FIELD_TROUBLE: &str = "trouble";
|
||||
|
||||
/// One app's shared entry, as read from Redis.
|
||||
#[derive(Clone, Debug, Default)]
|
||||
pub struct SharedAppEntry {
|
||||
pub fetched: Option<SharedFetch>,
|
||||
pub failure: Option<SharedFailure>,
|
||||
}
|
||||
|
||||
/// A successful fetch (or `304` confirmation) by some instance.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct SharedFetch {
|
||||
pub latest_release: Option<CraftAppRelease>,
|
||||
pub etag: Option<String>,
|
||||
pub fetched_at: DateTime<Utc>,
|
||||
}
|
||||
|
||||
/// The last failed request by some instance.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub struct SharedFailure {
|
||||
pub failed_at: DateTime<Utc>,
|
||||
pub consecutive_failures: u32,
|
||||
}
|
||||
|
||||
/// Reads an app's hash. `None` for an empty hash or another format version.
|
||||
pub fn decode_app_entry(fields: &HashMap<String, String>) -> Option<SharedAppEntry> {
|
||||
if fields.get(FIELD_VERSION).map(String::as_str) != Some(FORMAT_VERSION) {
|
||||
return None;
|
||||
}
|
||||
let fetched = decode_fetch(fields);
|
||||
let failure = time_field(fields, FIELD_FAILED_MS).map(|failed_at| SharedFailure {
|
||||
failed_at,
|
||||
consecutive_failures: fields.get(FIELD_FAILURES).and_then(|f| f.parse().ok()).unwrap_or(1).max(1),
|
||||
});
|
||||
Some(SharedAppEntry { fetched, failure })
|
||||
}
|
||||
|
||||
/// The fields written for a successful fetch.
|
||||
pub fn encode_fetch(latest_release: Option<&CraftAppRelease>, etag: Option<&str>, fetched_at: DateTime<Utc>, by: &str) -> Result<Vec<(&'static str, String)>, serde_json::Error> {
|
||||
Ok(vec![
|
||||
(FIELD_VERSION, FORMAT_VERSION.to_string()),
|
||||
(FIELD_FETCHED_MS, fetched_at.timestamp_millis().to_string()),
|
||||
(FIELD_ETAG, etag.unwrap_or_default().to_string()),
|
||||
(FIELD_RELEASE, serde_json::to_string(&latest_release)?),
|
||||
(FIELD_BY, by.to_string()),
|
||||
])
|
||||
}
|
||||
|
||||
/// The fields written for a `304`: only the time moves.
|
||||
pub fn encode_unchanged(fetched_at: DateTime<Utc>, by: &str) -> Vec<(&'static str, String)> {
|
||||
vec![
|
||||
(FIELD_VERSION, FORMAT_VERSION.to_string()),
|
||||
(FIELD_FETCHED_MS, fetched_at.timestamp_millis().to_string()),
|
||||
(FIELD_BY, by.to_string()),
|
||||
]
|
||||
}
|
||||
|
||||
/// The fields written for a failed request.
|
||||
pub fn encode_failure(failure: SharedFailure, by: &str) -> Vec<(&'static str, String)> {
|
||||
vec![
|
||||
(FIELD_VERSION, FORMAT_VERSION.to_string()),
|
||||
(FIELD_FAILED_MS, failure.failed_at.timestamp_millis().to_string()),
|
||||
(FIELD_FAILURES, failure.consecutive_failures.to_string()),
|
||||
(FIELD_BY, by.to_string()),
|
||||
]
|
||||
}
|
||||
|
||||
/// Reads the backoff hash; anything absent or unreadable is no backoff.
|
||||
pub fn decode_backoff(fields: &HashMap<String, String>) -> Backoff {
|
||||
if fields.get(FIELD_VERSION).map(String::as_str) != Some(FORMAT_VERSION) {
|
||||
return Backoff::default();
|
||||
}
|
||||
Backoff {
|
||||
until: time_field(fields, FIELD_UNTIL_MS),
|
||||
trouble: fields.get(FIELD_TROUBLE).and_then(|t| t.parse().ok()).unwrap_or(0),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn encode_backoff(backoff: Backoff, by: &str) -> Vec<(&'static str, String)> {
|
||||
vec![
|
||||
(FIELD_VERSION, FORMAT_VERSION.to_string()),
|
||||
(FIELD_UNTIL_MS, backoff.until.map(|u| u.timestamp_millis()).unwrap_or(0).to_string()),
|
||||
(FIELD_TROUBLE, backoff.trouble.to_string()),
|
||||
(FIELD_BY, by.to_string()),
|
||||
]
|
||||
}
|
||||
|
||||
/// The fetch part of an app's hash; `None` unless the time, the ETag and the release all read.
|
||||
fn decode_fetch(fields: &HashMap<String, String>) -> Option<SharedFetch> {
|
||||
let fetched_at = time_field(fields, FIELD_FETCHED_MS)?;
|
||||
let latest_release: Option<CraftAppRelease> = serde_json::from_str(fields.get(FIELD_RELEASE)?).ok()?;
|
||||
let etag = fields.get(FIELD_ETAG)?;
|
||||
Some(SharedFetch {
|
||||
latest_release,
|
||||
etag: (!etag.is_empty()).then(|| etag.clone()),
|
||||
fetched_at,
|
||||
})
|
||||
}
|
||||
|
||||
/// A Unix-milliseconds field; `None` when absent, unreadable or zero.
|
||||
fn time_field(fields: &HashMap<String, String>, name: &str) -> Option<DateTime<Utc>> {
|
||||
let ms = fields.get(name)?.parse::<i64>().ok().filter(|ms| *ms > 0)?;
|
||||
DateTime::from_timestamp_millis(ms)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use chrono::TimeZone;
|
||||
|
||||
use super::*;
|
||||
|
||||
const BY: &str = "storyteller-web-abc";
|
||||
|
||||
mod app_entries {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn a_fetch_round_trips() {
|
||||
let release = release();
|
||||
let at = time();
|
||||
let fields = to_map(encode_fetch(Some(&release), Some("\"e1\""), at, BY).unwrap());
|
||||
let entry = decode_app_entry(&fields).unwrap();
|
||||
let fetched = entry.fetched.unwrap();
|
||||
assert_eq!((fetched.etag.as_deref(), fetched.fetched_at), (Some("\"e1\""), at));
|
||||
assert_eq!(json(&fetched.latest_release), json(&Some(release)));
|
||||
assert_eq!(entry.failure, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_repo_without_release_or_etag_round_trips() {
|
||||
let fields = to_map(encode_fetch(None, None, time(), BY).unwrap());
|
||||
let fetched = decode_app_entry(&fields).unwrap().fetched.unwrap();
|
||||
assert!(fetched.latest_release.is_none());
|
||||
assert_eq!(fetched.etag, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_304_moves_only_the_time() {
|
||||
let mut fields = to_map(encode_fetch(Some(&release()), Some("\"e1\""), time(), BY).unwrap());
|
||||
let later = time() + chrono::Duration::minutes(2);
|
||||
fields.extend(to_map(encode_unchanged(later, BY)));
|
||||
let fetched = decode_app_entry(&fields).unwrap().fetched.unwrap();
|
||||
assert_eq!((fetched.fetched_at, fetched.etag.as_deref()), (later, Some("\"e1\"")));
|
||||
assert_eq!(json(&fetched.latest_release), json(&Some(release())));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_failure_round_trips_alongside_a_fetch() {
|
||||
let mut fields = to_map(encode_fetch(None, None, time(), BY).unwrap());
|
||||
let failure = SharedFailure { failed_at: time() + chrono::Duration::minutes(1), consecutive_failures: 3 };
|
||||
fields.extend(to_map(encode_failure(failure, BY)));
|
||||
let entry = decode_app_entry(&fields).unwrap();
|
||||
assert_eq!(entry.failure, Some(failure));
|
||||
assert!(entry.fetched.is_some());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn garbage_reads_as_absent() {
|
||||
assert!(decode_app_entry(&HashMap::new()).is_none());
|
||||
let good = to_map(encode_fetch(Some(&release()), Some("\"e1\""), time(), BY).unwrap());
|
||||
let with = |field: &str, value: &str| {
|
||||
let mut fields = good.clone();
|
||||
fields.insert(field.to_string(), value.to_string());
|
||||
fields
|
||||
};
|
||||
assert!(decode_app_entry(&with(FIELD_VERSION, "2")).is_none(), "unknown version");
|
||||
assert!(decode_app_entry(&with(FIELD_RELEASE, "{not json")).unwrap().fetched.is_none());
|
||||
assert!(decode_app_entry(&with(FIELD_RELEASE, "{\"tag\":1}")).unwrap().fetched.is_none());
|
||||
assert!(decode_app_entry(&with(FIELD_FETCHED_MS, "yesterday")).unwrap().fetched.is_none());
|
||||
assert!(decode_app_entry(&with(FIELD_FAILED_MS, "-5")).unwrap().failure.is_none());
|
||||
}
|
||||
}
|
||||
|
||||
mod backoffs {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn a_backoff_round_trips() {
|
||||
let backoff = Backoff { until: Some(time()), trouble: 4 };
|
||||
assert_eq!(decode_backoff(&to_map(encode_backoff(backoff, BY))), backoff);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn garbage_reads_as_no_backoff() {
|
||||
assert_eq!(decode_backoff(&HashMap::new()), Backoff::default());
|
||||
let mut fields = to_map(encode_backoff(Backoff { until: Some(time()), trouble: 4 }, BY));
|
||||
fields.insert(FIELD_VERSION.into(), "zz".into());
|
||||
assert_eq!(decode_backoff(&fields), Backoff::default());
|
||||
fields.insert(FIELD_VERSION.into(), FORMAT_VERSION.into());
|
||||
fields.insert(FIELD_UNTIL_MS.into(), "soon".into());
|
||||
fields.insert(FIELD_TROUBLE.into(), "-1".into());
|
||||
assert_eq!(decode_backoff(&fields), Backoff::default());
|
||||
}
|
||||
}
|
||||
|
||||
fn time() -> DateTime<Utc> {
|
||||
Utc.with_ymd_and_hms(2026, 10, 10, 12, 0, 0).unwrap()
|
||||
}
|
||||
|
||||
fn release() -> CraftAppRelease {
|
||||
CraftAppRelease {
|
||||
tag: "v0.5.0".into(),
|
||||
version: "0.5.0".into(),
|
||||
name: "PhotoCraft v0.5.0".into(),
|
||||
published_at: Some(time()),
|
||||
html_url: "https://github.com/storytold/photocraft/releases/tag/v0.5.0".into(),
|
||||
prerelease: false,
|
||||
notes_markdown: "Notes".into(),
|
||||
assets: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
fn json(release: &Option<CraftAppRelease>) -> String {
|
||||
serde_json::to_string(release).unwrap()
|
||||
}
|
||||
|
||||
fn to_map(fields: Vec<(&'static str, String)>) -> HashMap<String, String> {
|
||||
fields.into_iter().map(|(k, v)| (k.to_string(), v)).collect()
|
||||
}
|
||||
}
|
||||
@@ -1,644 +0,0 @@
|
||||
//! Reads and writes the fleet's shared Craft App release state in Redis (keys and fields are
|
||||
//! documented in `shared_entry`).
|
||||
//!
|
||||
//! Writes are last-write-wins but guarded by time: a Lua script compares a timestamp field
|
||||
//! before writing, so a slow instance never replaces a newer fetch with an older one. A `304`
|
||||
//! moves only `fetched_ms`, and only if the stored ETag is still the one that was revalidated.
|
||||
//!
|
||||
//! Some Redis setups refuse scripts (scripting disabled, an ACL without `EVAL`, a proxy). The
|
||||
//! first time Redis refuses the script, this store switches to applying the same rule itself:
|
||||
//! it `WATCH`es the hash, reads and decodes the guarding fields, decides, and writes in a
|
||||
//! `MULTI`/`EXEC` that Redis drops if another instance wrote in between (then it decides again).
|
||||
//!
|
||||
//! The pool is r2d2 (blocking), so every call runs on tokio's blocking pool with a timeout:
|
||||
//! Redis trouble costs the polling loop a few seconds at most, never a stuck thread of the
|
||||
//! async runtime. The poller gets its own small pool ([`build_poller_redis_pool`]) whose
|
||||
//! connections time out reads and writes, so a hung Redis can neither strand blocking threads
|
||||
//! nor take connections from the request handlers' pool.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::{Arc, LazyLock};
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::anyhow;
|
||||
use chrono::{DateTime, Utc};
|
||||
use log::warn;
|
||||
use r2d2::{ManageConnection, Pool};
|
||||
use redis::{Client, Connection, ConnectionLike, ErrorKind, RedisError, RedisResult, Script, ServerErrorKind, Value};
|
||||
|
||||
use artcraft_api_defs::craft_apps::release_info::CraftAppRelease;
|
||||
use redis_schema::keys::craft_apps::craft_app_release_redis_key::CraftAppReleaseRedisKey;
|
||||
use redis_schema::keys::craft_apps::craft_apps_release_backoff_redis_key::CraftAppsReleaseBackoffRedisKey;
|
||||
|
||||
use crate::threads::poll_craft_app_releases_thread::scheduling::Backoff;
|
||||
use crate::threads::poll_craft_app_releases_thread::shared_entry;
|
||||
use crate::threads::poll_craft_app_releases_thread::shared_entry::{SharedAppEntry, SharedFailure};
|
||||
|
||||
/// Connections in the poller's pool: one per instance is all the loop uses at a time.
|
||||
const POOL_MAX_SIZE: u32 = 2;
|
||||
/// Longest wait for a pooled connection.
|
||||
const CONNECTION_TIMEOUT: Duration = Duration::from_secs(3);
|
||||
/// Longest wait for one read or write on a connection; after it, the connection gives up
|
||||
/// instead of holding its blocking thread until the operating system notices a dead peer.
|
||||
const IO_TIMEOUT: Duration = Duration::from_secs(5);
|
||||
/// Longest wait for a whole call, connection included.
|
||||
const CALL_TIMEOUT: Duration = Duration::from_secs(10);
|
||||
/// How long a claim on an app keeps other instances from requesting it too: longer than one
|
||||
/// request can take (10 s to connect, 30 s for the answer), plus a margin.
|
||||
pub const CLAIM_WINDOW: Duration = Duration::from_secs(60);
|
||||
/// Without scripts: how many times a guarded write decides again after another instance wrote
|
||||
/// between its read and its write, before giving up (as if it lost the race).
|
||||
const WATCH_ATTEMPTS: usize = 3;
|
||||
|
||||
/// Writes `ARGV[6..]` (field/value pairs) to the hash `KEYS[1]` and sets its TTL to `ARGV[4]` ms,
|
||||
/// unless the hash's time field `ARGV[1]` already holds a value greater than
|
||||
/// `ARGV[2] - ARGV[3]` (new time minus a minimum gap; a gap of 0 means "never go back in time"),
|
||||
/// or `ARGV[5]` is non-empty and differs from the hash's `etag`. Returns 1 if written, else 0.
|
||||
/// Unreadable stored times don't block the write, so garbage gets overwritten.
|
||||
const GUARDED_HSET_LUA: &str = r#"
|
||||
local current = tonumber(redis.call('HGET', KEYS[1], ARGV[1]))
|
||||
if current and current + tonumber(ARGV[3]) > tonumber(ARGV[2]) then
|
||||
return 0
|
||||
end
|
||||
if ARGV[5] ~= '' and redis.call('HGET', KEYS[1], 'etag') ~= ARGV[5] then
|
||||
return 0
|
||||
end
|
||||
redis.call('HSET', KEYS[1], unpack(ARGV, 6))
|
||||
redis.call('PEXPIRE', KEYS[1], ARGV[4])
|
||||
return 1
|
||||
"#;
|
||||
|
||||
static GUARDED_HSET: LazyLock<Script> = LazyLock::new(|| Script::new(GUARDED_HSET_LUA));
|
||||
|
||||
/// Handle on the shared state. Cheap to clone.
|
||||
#[derive(Clone)]
|
||||
pub struct SharedStore {
|
||||
pool: PollerRedisPool,
|
||||
/// Written as `by`, for debugging.
|
||||
hostname: String,
|
||||
/// Set once Redis refuses the guarded-write script; from then on guarded writes use `WATCH`.
|
||||
scripts_refused: Arc<AtomicBool>,
|
||||
}
|
||||
|
||||
/// The poller's Redis pool; see [`build_poller_redis_pool`].
|
||||
pub type PollerRedisPool = Pool<PollerRedisConnections>;
|
||||
|
||||
/// Makes the poller's Redis connections, every one of them with I/O timeouts.
|
||||
pub struct PollerRedisConnections {
|
||||
client: Client,
|
||||
}
|
||||
|
||||
/// A guarded write: which time field guards it, and what to write.
|
||||
struct GuardedWrite {
|
||||
key: String,
|
||||
time_field: &'static str,
|
||||
time: DateTime<Utc>,
|
||||
min_gap: Duration,
|
||||
required_etag: Option<String>,
|
||||
fields: Vec<(&'static str, String)>,
|
||||
}
|
||||
|
||||
impl SharedStore {
|
||||
pub fn new(pool: PollerRedisPool, hostname: String) -> Self {
|
||||
Self { pool, hostname, scripts_refused: Arc::new(AtomicBool::new(false)) }
|
||||
}
|
||||
|
||||
/// The shared entries of `app_keys` (in order; `None` for absent or unreadable ones) and the
|
||||
/// fleet's backoff, in one round trip.
|
||||
pub async fn read(&self, app_keys: &[String]) -> anyhow::Result<(Vec<Option<SharedAppEntry>>, Backoff)> {
|
||||
let keys: Vec<String> = app_keys.iter().map(|k| CraftAppReleaseRedisKey::new_for_app(k).to_string()).collect();
|
||||
let mut hashes: Vec<HashMap<String, String>> = self
|
||||
.run(move |conn| {
|
||||
let mut pipe = redis::pipe();
|
||||
for key in &keys {
|
||||
pipe.hgetall(key);
|
||||
}
|
||||
pipe.hgetall(CraftAppsReleaseBackoffRedisKey::new_for_fleet().as_str());
|
||||
pipe.query(conn)
|
||||
})
|
||||
.await?;
|
||||
let backoff = hashes.pop().map(|fields| shared_entry::decode_backoff(&fields)).unwrap_or_default();
|
||||
let entries = hashes.iter().map(shared_entry::decode_app_entry).collect();
|
||||
Ok((entries, backoff))
|
||||
}
|
||||
|
||||
/// Claims the next request for `app_key` at `now`. False when another instance claimed it
|
||||
/// within [`CLAIM_WINDOW`]: it is asking GitHub already, so this one doesn't. Not a lock: it
|
||||
/// is never released and simply lapses, so a crashed instance can't block anyone.
|
||||
pub async fn claim(&self, app_key: &str, now: DateTime<Utc>) -> anyhow::Result<bool> {
|
||||
self.guarded_write(GuardedWrite {
|
||||
key: CraftAppReleaseRedisKey::new_for_app(app_key).to_string(),
|
||||
time_field: shared_entry::FIELD_CLAIMED_MS,
|
||||
time: now,
|
||||
min_gap: CLAIM_WINDOW,
|
||||
required_etag: None,
|
||||
fields: vec![
|
||||
(shared_entry::FIELD_VERSION, shared_entry::FORMAT_VERSION.to_string()),
|
||||
(shared_entry::FIELD_CLAIMED_MS, now.timestamp_millis().to_string()),
|
||||
(shared_entry::FIELD_BY, self.hostname.clone()),
|
||||
],
|
||||
}).await
|
||||
}
|
||||
|
||||
/// Shares a successful fetch, unless a newer one is already shared. Returns whether written.
|
||||
pub async fn write_fetched(&self, app_key: &str, latest_release: Option<&CraftAppRelease>, etag: Option<&str>, fetched_at: DateTime<Utc>) -> anyhow::Result<bool> {
|
||||
self.guarded_write(GuardedWrite {
|
||||
key: CraftAppReleaseRedisKey::new_for_app(app_key).to_string(),
|
||||
time_field: shared_entry::FIELD_FETCHED_MS,
|
||||
time: fetched_at,
|
||||
min_gap: Duration::ZERO,
|
||||
required_etag: None,
|
||||
fields: shared_entry::encode_fetch(latest_release, etag, fetched_at, &self.hostname)?,
|
||||
}).await
|
||||
}
|
||||
|
||||
/// Shares a `304` for the release with ETag `etag`: moves only the fetch time, and only if
|
||||
/// that release is still the shared one and nothing newer was shared. Returns whether written.
|
||||
pub async fn write_unchanged(&self, app_key: &str, etag: &str, fetched_at: DateTime<Utc>) -> anyhow::Result<bool> {
|
||||
if etag.is_empty() {
|
||||
return Ok(false);
|
||||
}
|
||||
self.guarded_write(GuardedWrite {
|
||||
key: CraftAppReleaseRedisKey::new_for_app(app_key).to_string(),
|
||||
time_field: shared_entry::FIELD_FETCHED_MS,
|
||||
time: fetched_at,
|
||||
min_gap: Duration::ZERO,
|
||||
required_etag: Some(etag.to_string()),
|
||||
fields: shared_entry::encode_unchanged(fetched_at, &self.hostname),
|
||||
}).await
|
||||
}
|
||||
|
||||
/// Shares a failed request, unless a newer failure is already shared. Returns whether written.
|
||||
pub async fn write_failure(&self, app_key: &str, failure: SharedFailure) -> anyhow::Result<bool> {
|
||||
self.guarded_write(GuardedWrite {
|
||||
key: CraftAppReleaseRedisKey::new_for_app(app_key).to_string(),
|
||||
time_field: shared_entry::FIELD_FAILED_MS,
|
||||
time: failure.failed_at,
|
||||
min_gap: Duration::ZERO,
|
||||
required_etag: None,
|
||||
fields: shared_entry::encode_failure(failure, &self.hostname),
|
||||
}).await
|
||||
}
|
||||
|
||||
/// Shares the fleet's backoff; no trouble deletes it.
|
||||
pub async fn write_backoff(&self, backoff: Backoff) -> anyhow::Result<()> {
|
||||
let key = CraftAppsReleaseBackoffRedisKey::new_for_fleet().to_string();
|
||||
if backoff.trouble == 0 {
|
||||
return self.run(move |conn| redis::cmd("DEL").arg(&key).query::<()>(conn)).await;
|
||||
}
|
||||
let fields = shared_entry::encode_backoff(backoff, &self.hostname);
|
||||
let ttl_ms = CraftAppsReleaseBackoffRedisKey::get_redis_ttl().num_milliseconds();
|
||||
self
|
||||
.run(move |conn| {
|
||||
redis::pipe()
|
||||
.hset_multiple(&key, &fields).ignore()
|
||||
.pexpire(&key, ttl_ms).ignore()
|
||||
.query::<()>(conn)
|
||||
})
|
||||
.await
|
||||
}
|
||||
|
||||
/// Applies `write` with the script, or (once Redis has refused the script) with `WATCH`.
|
||||
async fn guarded_write(&self, write: GuardedWrite) -> anyhow::Result<bool> {
|
||||
let ttl_ms = CraftAppReleaseRedisKey::get_redis_ttl().num_milliseconds();
|
||||
let scripts_refused = self.scripts_refused.clone();
|
||||
self
|
||||
.run(move |conn| {
|
||||
if !scripts_refused.load(Ordering::Relaxed) {
|
||||
match guarded_write_with_script(conn, &write, ttl_ms) {
|
||||
Err(err) if is_script_refusal(&err) => {
|
||||
if !scripts_refused.swap(true, Ordering::Relaxed) {
|
||||
warn!("Craft app releases: Redis refused the guarded-write script ({err}); writing with WATCH/MULTI/EXEC instead.");
|
||||
}
|
||||
}
|
||||
result => return result,
|
||||
}
|
||||
}
|
||||
guarded_write_with_watch(conn, &write, ttl_ms)
|
||||
})
|
||||
.await
|
||||
}
|
||||
|
||||
/// Runs `call` on a pooled connection, off the async runtime, within [`CALL_TIMEOUT`].
|
||||
async fn run<T, F>(&self, call: F) -> anyhow::Result<T>
|
||||
where
|
||||
T: Send + 'static,
|
||||
F: FnOnce(&mut Connection) -> RedisResult<T> + Send + 'static,
|
||||
{
|
||||
let pool = self.pool.clone();
|
||||
let task = tokio::task::spawn_blocking(move || -> anyhow::Result<T> {
|
||||
let mut conn = pool.get_timeout(CONNECTION_TIMEOUT)?;
|
||||
Ok(call(&mut conn)?)
|
||||
});
|
||||
match tokio::time::timeout(CALL_TIMEOUT, task).await {
|
||||
Ok(Ok(result)) => result,
|
||||
Ok(Err(join_err)) => Err(anyhow!("redis task failed: {join_err}")),
|
||||
Err(_) => Err(anyhow!("redis call timed out after {CALL_TIMEOUT:?}")),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The poller's own Redis pool, apart from the one the request handlers use. Built without
|
||||
/// connecting, so an unreachable Redis at startup doesn't stop the server: the poller then
|
||||
/// works alone until Redis answers.
|
||||
pub fn build_poller_redis_pool(redis_url: &str) -> RedisResult<PollerRedisPool> {
|
||||
let manager = PollerRedisConnections { client: Client::open(redis_url)? };
|
||||
Ok(Pool::builder()
|
||||
.max_size(POOL_MAX_SIZE)
|
||||
.min_idle(Some(0))
|
||||
.connection_timeout(CONNECTION_TIMEOUT)
|
||||
.build_unchecked(manager))
|
||||
}
|
||||
|
||||
impl ManageConnection for PollerRedisConnections {
|
||||
type Connection = Connection;
|
||||
type Error = RedisError;
|
||||
|
||||
/// Connects (handshake included) within [`CONNECTION_TIMEOUT`], then makes every later read
|
||||
/// and write give up after [`IO_TIMEOUT`], before the connection is ever used.
|
||||
fn connect(&self) -> RedisResult<Connection> {
|
||||
let conn = self.client.get_connection_with_timeout(CONNECTION_TIMEOUT)?;
|
||||
conn.set_read_timeout(Some(IO_TIMEOUT))?;
|
||||
conn.set_write_timeout(Some(IO_TIMEOUT))?;
|
||||
Ok(conn)
|
||||
}
|
||||
|
||||
/// The pool's check before lending a connection; bounded by [`IO_TIMEOUT`] like any command.
|
||||
fn is_valid(&self, conn: &mut Connection) -> RedisResult<()> {
|
||||
redis::cmd("PING").query::<()>(conn)
|
||||
}
|
||||
|
||||
fn has_broken(&self, conn: &mut Connection) -> bool {
|
||||
!conn.is_open()
|
||||
}
|
||||
}
|
||||
|
||||
/// [`GUARDED_HSET_LUA`] decides and writes on the server, in one step.
|
||||
fn guarded_write_with_script(conn: &mut Connection, write: &GuardedWrite, ttl_ms: i64) -> RedisResult<bool> {
|
||||
let mut invocation = GUARDED_HSET.key(&write.key);
|
||||
invocation
|
||||
.arg(write.time_field)
|
||||
.arg(write.time.timestamp_millis())
|
||||
.arg(write.min_gap.as_millis() as u64)
|
||||
.arg(ttl_ms)
|
||||
.arg(write.required_etag.as_deref().unwrap_or_default());
|
||||
for (field, value) in &write.fields {
|
||||
invocation.arg(*field).arg(value);
|
||||
}
|
||||
let written: i64 = invocation.invoke(conn)?;
|
||||
Ok(written == 1)
|
||||
}
|
||||
|
||||
/// The same rule without scripts: `WATCH` the hash, read and decode its guarding fields, decide
|
||||
/// with [`write_wins`], then write in a transaction. If another instance wrote the hash in
|
||||
/// between, Redis drops the transaction and this decides again with what it wrote.
|
||||
fn guarded_write_with_watch(conn: &mut Connection, write: &GuardedWrite, ttl_ms: i64) -> RedisResult<bool> {
|
||||
for _ in 0..WATCH_ATTEMPTS {
|
||||
redis::cmd("WATCH").arg(&write.key).query::<()>(conn)?;
|
||||
let stored: RedisResult<(Option<String>, Option<String>)> = redis::cmd("HMGET")
|
||||
.arg(&write.key)
|
||||
.arg(write.time_field)
|
||||
.arg(shared_entry::FIELD_ETAG)
|
||||
.query(conn);
|
||||
let (stored_time, stored_etag) = match stored {
|
||||
Ok(stored) => stored,
|
||||
Err(err) => {
|
||||
// The connection goes back to the shared pool; don't leave it watching.
|
||||
let _ = redis::cmd("UNWATCH").query::<()>(conn);
|
||||
return Err(err);
|
||||
}
|
||||
};
|
||||
if !write_wins(write, stored_time.as_deref(), stored_etag.as_deref()) {
|
||||
redis::cmd("UNWATCH").query::<()>(conn)?;
|
||||
return Ok(false);
|
||||
}
|
||||
// `EXEC` ends the watch either way; `None` means another write got in first.
|
||||
let committed: Option<Value> = redis::pipe()
|
||||
.atomic()
|
||||
.hset_multiple(&write.key, &write.fields)
|
||||
.pexpire(&write.key, ttl_ms)
|
||||
.query(conn)?;
|
||||
if committed.is_some() {
|
||||
return Ok(true);
|
||||
}
|
||||
}
|
||||
Ok(false)
|
||||
}
|
||||
|
||||
/// Whether `write` may replace what the hash holds, by the script's rule: not if the stored time
|
||||
/// plus the minimum gap is later than the new time, nor if an ETag is required and the stored
|
||||
/// one differs. An unreadable stored time doesn't block the write, so garbage gets overwritten.
|
||||
fn write_wins(write: &GuardedWrite, stored_time: Option<&str>, stored_etag: Option<&str>) -> bool {
|
||||
let new_ms = write.time.timestamp_millis();
|
||||
let gap_ms = write.min_gap.as_millis().min(i64::MAX as u128) as i64;
|
||||
if let Some(current_ms) = stored_time.and_then(|t| t.parse::<i64>().ok()) {
|
||||
if current_ms.saturating_add(gap_ms) > new_ms {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
match write.required_etag.as_deref().filter(|etag| !etag.is_empty()) {
|
||||
Some(required) => stored_etag == Some(required),
|
||||
None => true,
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether Redis answered that it won't run the script (as opposed to being unreachable, busy,
|
||||
/// out of memory or read-only, which the `WATCH` path wouldn't get past either): an ACL
|
||||
/// refusal, an unknown `EVALSHA`/`EVAL`/`SCRIPT` command, or scripting switched off.
|
||||
fn is_script_refusal(err: &RedisError) -> bool {
|
||||
if err.is_io_error() || err.is_timeout() || err.is_connection_dropped() || err.is_connection_refusal() {
|
||||
return false;
|
||||
}
|
||||
match err.kind() {
|
||||
ErrorKind::Server(ServerErrorKind::NoPerm) => true,
|
||||
ErrorKind::Server(ServerErrorKind::ResponseError) | ErrorKind::Extension => {
|
||||
if err.code() == Some("BUSY") {
|
||||
return false;
|
||||
}
|
||||
let detail = err.detail().unwrap_or_default().to_ascii_lowercase();
|
||||
detail.contains("unknown command") || detail.contains("eval") || detail.contains("script")
|
||||
}
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use std::io::{Read, Write};
|
||||
use std::net::{TcpListener, TcpStream};
|
||||
|
||||
use chrono::TimeZone;
|
||||
use url::Url;
|
||||
|
||||
use super::*;
|
||||
|
||||
const DEFAULT_TEST_REDIS_URL: &str = "redis://127.0.0.1:6379/";
|
||||
|
||||
/// Against a real Redis (`CRAFT_APPS_TEST_REDIS_URL`, default `redis://127.0.0.1:6379/`). Each
|
||||
/// test uses app keys of its own, deletes them afterwards, and leaves the fleet backoff key
|
||||
/// alone unless it was absent.
|
||||
/// `cargo test -p storyteller-web --bin storyteller-web shared_store -- --ignored`
|
||||
mod live_redis_tests {
|
||||
use super::*;
|
||||
|
||||
#[tokio::test]
|
||||
#[ignore = "needs a Redis server"]
|
||||
async fn guarded_writes_with_the_script() {
|
||||
let pool = test_pool(&test_redis_url());
|
||||
let (pod_a, pod_b) = (SharedStore::new(pool.clone(), "pod-a".into()), SharedStore::new(pool, "pod-b".into()));
|
||||
exercise_guarded_writes(&pod_a, &pod_b).await;
|
||||
assert!(!pod_a.scripts_refused.load(Ordering::Relaxed));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[ignore = "needs a Redis server"]
|
||||
async fn guarded_writes_with_watch() {
|
||||
let pool = test_pool(&test_redis_url());
|
||||
let (pod_a, pod_b) = (SharedStore::new(pool.clone(), "pod-a".into()), SharedStore::new(pool, "pod-b".into()));
|
||||
pod_a.scripts_refused.store(true, Ordering::Relaxed);
|
||||
pod_b.scripts_refused.store(true, Ordering::Relaxed);
|
||||
exercise_guarded_writes(&pod_a, &pod_b).await;
|
||||
}
|
||||
|
||||
/// A Redis user without scripting permissions: the first guarded write is refused, and the
|
||||
/// store switches to `WATCH` and still applies every rule.
|
||||
#[tokio::test]
|
||||
#[ignore = "needs a Redis server that supports ACLs"]
|
||||
async fn a_refused_script_falls_back_to_watch() {
|
||||
let admin_url = test_redis_url();
|
||||
let admin = test_pool(&admin_url);
|
||||
let user = format!("craft-test-{}", rand::random::<u32>());
|
||||
let password = format!("pw-{}", rand::random::<u64>());
|
||||
let mut conn = admin.get().unwrap();
|
||||
redis::cmd("ACL").arg("SETUSER").arg(&user).arg("reset").arg("on").arg(format!(">{password}")).arg("~*").arg("&*").arg("+@all").arg("-@scripting")
|
||||
.query::<()>(&mut *conn).unwrap();
|
||||
|
||||
let mut url = Url::parse(&admin_url).unwrap();
|
||||
url.set_username(&user).unwrap();
|
||||
url.set_password(Some(&password)).unwrap();
|
||||
let pool = test_pool(url.as_str());
|
||||
let (pod_a, pod_b) = (SharedStore::new(pool.clone(), "pod-a".into()), SharedStore::new(pool, "pod-b".into()));
|
||||
exercise_guarded_writes(&pod_a, &pod_b).await;
|
||||
|
||||
redis::cmd("ACL").arg("DELUSER").arg(&user).query::<()>(&mut *conn).unwrap();
|
||||
assert!(pod_a.scripts_refused.load(Ordering::Relaxed), "the script should have been refused");
|
||||
assert!(pod_b.scripts_refused.load(Ordering::Relaxed), "the script should have been refused");
|
||||
}
|
||||
}
|
||||
|
||||
/// An unreachable Redis fails fast, so the loop can fall back to polling on its own.
|
||||
#[tokio::test]
|
||||
async fn an_unreachable_redis_fails_fast() {
|
||||
let pool = build_poller_redis_pool("redis://127.0.0.1:1/").unwrap();
|
||||
let store = SharedStore::new(pool, "pod".into());
|
||||
let started = std::time::Instant::now();
|
||||
assert!(store.read(&["photocraft".to_string()]).await.is_err());
|
||||
assert!(started.elapsed() < CALL_TIMEOUT + Duration::from_secs(1));
|
||||
}
|
||||
|
||||
/// A Redis that completes the connection handshake and then stops answering (a wedged server,
|
||||
/// a half-open connection): the call fails by itself, before [`CALL_TIMEOUT`] gives up on it,
|
||||
/// so its blocking thread and connection are released rather than stranded until the
|
||||
/// operating system notices. Without I/O timeouts the pool's `PING` would hang here forever.
|
||||
#[tokio::test]
|
||||
async fn a_redis_that_stops_answering_releases_the_blocking_thread() {
|
||||
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
|
||||
let url = format!("redis://{}/", listener.local_addr().unwrap());
|
||||
std::thread::spawn(move || {
|
||||
for stream in listener.incoming().flatten() {
|
||||
std::thread::spawn(move || answer_handshake_then_go_silent(stream));
|
||||
}
|
||||
});
|
||||
let store = SharedStore::new(build_poller_redis_pool(&url).unwrap(), "pod".into());
|
||||
let started = std::time::Instant::now();
|
||||
let err = store.read(&["photocraft".to_string()]).await.unwrap_err();
|
||||
assert!(started.elapsed() < CALL_TIMEOUT, "{:?} after {:?}", err, started.elapsed());
|
||||
}
|
||||
|
||||
/// [`write_wins`] must decide exactly as [`GUARDED_HSET_LUA`] does.
|
||||
mod write_rule_tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn a_write_never_goes_back_in_time() {
|
||||
let write = guarded_write_at(t(2), Duration::ZERO, None);
|
||||
assert!(write_wins(&write, None, None), "nothing stored");
|
||||
assert!(write_wins(&write, Some(&ms(t(1))), None), "newer than stored");
|
||||
assert!(write_wins(&write, Some(&ms(t(2))), None), "same time, as the script allows");
|
||||
assert!(!write_wins(&write, Some(&ms(t(3))), None), "older than stored");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_claim_waits_for_the_window() {
|
||||
let claim = guarded_write_at(t(1), CLAIM_WINDOW, None);
|
||||
assert!(!write_wins(&claim, Some(&ms(t(0) + chrono::Duration::seconds(30))), None));
|
||||
assert!(write_wins(&claim, Some(&ms(t(0))), None), "exactly one window later");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_required_etag_must_match() {
|
||||
let write = guarded_write_at(t(2), Duration::ZERO, Some("\"e2\""));
|
||||
assert!(write_wins(&write, Some(&ms(t(1))), Some("\"e2\"")));
|
||||
assert!(!write_wins(&write, Some(&ms(t(1))), Some("\"e1\"")), "another release is shared");
|
||||
assert!(!write_wins(&write, Some(&ms(t(1))), None), "no release is shared");
|
||||
assert!(write_wins(&guarded_write_at(t(2), Duration::ZERO, Some("")), None, None), "empty means none required");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_unreadable_stored_time_is_overwritten() {
|
||||
let write = guarded_write_at(t(2), Duration::ZERO, None);
|
||||
assert!(write_wins(&write, Some("soon"), None));
|
||||
assert!(write_wins(&write, Some(""), None));
|
||||
}
|
||||
}
|
||||
|
||||
mod script_refusal_tests {
|
||||
use redis::make_extension_error;
|
||||
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn redis_refusing_scripts_is_a_refusal() {
|
||||
let refusals = [
|
||||
server_error(ServerErrorKind::NoPerm, "this user has no permissions to run the 'evalsha' command"),
|
||||
server_error(ServerErrorKind::ResponseError, "unknown command 'EVALSHA', with args beginning with: "),
|
||||
server_error(ServerErrorKind::ResponseError, "This Redis instance has scripting disabled"),
|
||||
make_extension_error("ERR".to_string(), Some("unknown command 'SCRIPT'".to_string())),
|
||||
];
|
||||
for err in refusals {
|
||||
assert!(is_script_refusal(&err), "{:?}", err);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn redis_being_unavailable_is_not_a_refusal() {
|
||||
let unavailable = [
|
||||
RedisError::from(std::io::Error::new(std::io::ErrorKind::ConnectionRefused, "refused")),
|
||||
RedisError::from(std::io::Error::new(std::io::ErrorKind::TimedOut, "timed out")),
|
||||
make_extension_error("OOM".to_string(), Some("command not allowed when used memory > 'maxmemory'.".to_string())),
|
||||
make_extension_error("BUSY".to_string(), Some("Redis is busy running a script.".to_string())),
|
||||
server_error(ServerErrorKind::ReadOnly, "You can't write against a read only replica."),
|
||||
server_error(ServerErrorKind::ResponseError, "wrong number of arguments for 'hset' command"),
|
||||
];
|
||||
for err in unavailable {
|
||||
assert!(!is_script_refusal(&err), "{:?}", err);
|
||||
}
|
||||
}
|
||||
|
||||
fn server_error(kind: ServerErrorKind, detail: &str) -> RedisError {
|
||||
RedisError::from((ErrorKind::Server(kind), "server error", detail.to_string()))
|
||||
}
|
||||
}
|
||||
|
||||
/// Every guarded-write rule, as two instances sharing one Redis see it.
|
||||
async fn exercise_guarded_writes(pod_a: &SharedStore, pod_b: &SharedStore) {
|
||||
let app = format!("test-craft-{}", rand::random::<u32>());
|
||||
let other = format!("{app}-other");
|
||||
let keys = vec![app.clone(), other.clone()];
|
||||
let read_app = |store: SharedStore, keys: Vec<String>| async move { store.read(&keys).await.unwrap().0.remove(0) };
|
||||
|
||||
// Nothing yet.
|
||||
let (entries, _) = pod_a.read(&keys).await.unwrap();
|
||||
assert!(entries.iter().all(Option::is_none));
|
||||
|
||||
// Two instances tick together: only one gets to ask GitHub; the claim lapses after the window.
|
||||
assert!(pod_a.claim(&app, t(0)).await.unwrap());
|
||||
assert!(!pod_b.claim(&app, t(0) + chrono::Duration::seconds(5)).await.unwrap());
|
||||
assert!(pod_b.claim(&app, t(2)).await.unwrap());
|
||||
|
||||
// A newer fetch is never replaced by an older one.
|
||||
assert!(pod_a.write_fetched(&app, None, Some("\"e2\""), t(2)).await.unwrap());
|
||||
assert!(!pod_b.write_fetched(&app, None, Some("\"e1\""), t(1)).await.unwrap());
|
||||
let fetched = read_app(pod_a.clone(), keys.clone()).await.unwrap().fetched.unwrap();
|
||||
assert_eq!((fetched.etag.as_deref(), fetched.fetched_at), (Some("\"e2\""), t(2)));
|
||||
|
||||
// A 304 moves the time only, and only for the release it revalidated.
|
||||
assert!(!pod_b.write_unchanged(&app, "\"e1\"", t(3)).await.unwrap(), "stale ETag");
|
||||
assert!(!pod_b.write_unchanged(&app, "\"e2\"", t(1)).await.unwrap(), "older than the shared fetch");
|
||||
assert!(pod_b.write_unchanged(&app, "\"e2\"", t(3)).await.unwrap());
|
||||
let fetched = read_app(pod_a.clone(), keys.clone()).await.unwrap().fetched.unwrap();
|
||||
assert_eq!((fetched.etag.as_deref(), fetched.fetched_at), (Some("\"e2\""), t(3)));
|
||||
|
||||
// Failures: newest wins.
|
||||
let failure = SharedFailure { failed_at: t(5), consecutive_failures: 2 };
|
||||
assert!(pod_a.write_failure(&app, failure).await.unwrap());
|
||||
assert!(!pod_b.write_failure(&app, SharedFailure { failed_at: t(4), consecutive_failures: 1 }).await.unwrap());
|
||||
assert_eq!(read_app(pod_a.clone(), keys.clone()).await.unwrap().failure, Some(failure));
|
||||
|
||||
// Garbage in an entry is overwritten, not trusted.
|
||||
let key = CraftAppReleaseRedisKey::new_for_app(&other).to_string();
|
||||
let garbage_key = key.clone();
|
||||
pod_a.run(move |conn| redis::cmd("HSET").arg(&garbage_key).arg("v").arg("1").arg("fetched_ms").arg("soon").arg("release").arg("{").query::<()>(conn)).await.unwrap();
|
||||
assert!(pod_a.read(&keys).await.unwrap().0[1].as_ref().unwrap().fetched.is_none());
|
||||
assert!(pod_a.write_fetched(&other, None, None, t(1)).await.unwrap());
|
||||
assert!(pod_a.read(&keys).await.unwrap().0[1].as_ref().unwrap().fetched.is_some());
|
||||
|
||||
// Entries expire if nobody refreshes them.
|
||||
let ttl_key = key.clone();
|
||||
let ttl: i64 = pod_a.run(move |conn| redis::cmd("PTTL").arg(&ttl_key).query(conn)).await.unwrap();
|
||||
assert!(ttl > 6 * 24 * 3600 * 1000, "{}", ttl);
|
||||
|
||||
// The fleet backoff round-trips (only touched when absent, to leave a real one alone).
|
||||
let (_, existing) = pod_a.read(&keys).await.unwrap();
|
||||
if existing == Backoff::default() {
|
||||
let backoff = Backoff { until: Some(t(30)), trouble: 2 };
|
||||
pod_a.write_backoff(backoff).await.unwrap();
|
||||
assert_eq!(pod_b.read(&keys).await.unwrap().1, backoff);
|
||||
pod_b.write_backoff(Backoff::default()).await.unwrap();
|
||||
assert_eq!(pod_a.read(&keys).await.unwrap().1, Backoff::default());
|
||||
}
|
||||
|
||||
let cleanup: Vec<String> = keys.iter().map(|k| CraftAppReleaseRedisKey::new_for_app(k).to_string()).collect();
|
||||
pod_a.run(move |conn| redis::cmd("DEL").arg(&cleanup).query::<()>(conn)).await.unwrap();
|
||||
}
|
||||
|
||||
fn guarded_write_at(time: DateTime<Utc>, min_gap: Duration, required_etag: Option<&str>) -> GuardedWrite {
|
||||
GuardedWrite {
|
||||
key: "craftApps:releases:app:test".to_string(),
|
||||
time_field: shared_entry::FIELD_FETCHED_MS,
|
||||
time,
|
||||
min_gap,
|
||||
required_etag: required_etag.map(str::to_string),
|
||||
fields: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Answers `+OK` to connection setup (`CLIENT SETINFO`, `SELECT`, `HELLO`), then keeps the
|
||||
/// connection open without another word once a real command (`PING`, `HGETALL`, ...) arrives.
|
||||
fn answer_handshake_then_go_silent(mut stream: TcpStream) {
|
||||
let mut buf = [0u8; 4096];
|
||||
loop {
|
||||
let n = match stream.read(&mut buf) {
|
||||
Ok(0) | Err(_) => return,
|
||||
Ok(n) => n,
|
||||
};
|
||||
let chunk = String::from_utf8_lossy(&buf[..n]).to_ascii_uppercase();
|
||||
if !(chunk.contains("CLIENT") || chunk.contains("SELECT") || chunk.contains("HELLO")) {
|
||||
std::thread::sleep(Duration::from_secs(60));
|
||||
return;
|
||||
}
|
||||
let commands = chunk.lines().filter(|line| line.starts_with('*')).count();
|
||||
if stream.write_all("+OK\r\n".repeat(commands).as_bytes()).is_err() {
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn test_redis_url() -> String {
|
||||
std::env::var("CRAFT_APPS_TEST_REDIS_URL").unwrap_or_else(|_| DEFAULT_TEST_REDIS_URL.to_string())
|
||||
}
|
||||
|
||||
fn test_pool(url: &str) -> PollerRedisPool {
|
||||
build_poller_redis_pool(url).unwrap()
|
||||
}
|
||||
|
||||
fn t(minute: u32) -> DateTime<Utc> {
|
||||
Utc.with_ymd_and_hms(2026, 10, 10, 12, minute, 0).unwrap()
|
||||
}
|
||||
|
||||
fn ms(time: DateTime<Utc>) -> String {
|
||||
time.timestamp_millis().to_string()
|
||||
}
|
||||
}
|
||||
@@ -31,7 +31,7 @@ From the **repository root**, the launcher scripts (these preflight your
|
||||
environment, install dependencies, and free the dev port first):
|
||||
|
||||
```bash
|
||||
./script/website/unix_frontend_website_dev.sh # artcraft-website-next (marketing site), port 3000
|
||||
./script/website/unix_frontend_website_dev.sh # artcraft-website (marketing site), port 4200
|
||||
./script/website/unix_frontend_webapp_dev.sh # artcraft-webapp (user dashboard), port 4201
|
||||
```
|
||||
|
||||
@@ -46,12 +46,14 @@ by its full or relative path:
|
||||
Or directly from this directory:
|
||||
|
||||
```bash
|
||||
nx dev artcraft-website
|
||||
npx nx dev @frontend/artcraft-webapp
|
||||
```
|
||||
|
||||
## Building
|
||||
|
||||
```bash
|
||||
npx nx build artcraft-website
|
||||
npx nx build artcraft-webapp
|
||||
```
|
||||
|
||||
|
||||
@@ -4,6 +4,8 @@ import react from '@vitejs/plugin-react';
|
||||
import wasm from 'vite-plugin-wasm';
|
||||
import topLevelAwait from 'vite-plugin-top-level-await';
|
||||
import { nxViteTsPaths } from '@nx/vite/plugins/nx-tsconfig-paths.plugin';
|
||||
import { execSync } from 'child_process';
|
||||
import path from 'path';
|
||||
|
||||
// Dev-only CDN proxy. Media assets (splats, GLBs, images) are fetched with
|
||||
// fetch(), but the CDNs only send CORS headers for the production origin —
|
||||
@@ -69,6 +71,21 @@ function cdnProxyPlugin() {
|
||||
};
|
||||
}
|
||||
|
||||
// Custom plugin to generate news.json on dev server start
|
||||
function generateNewsPlugin() {
|
||||
return {
|
||||
name: 'generate-news',
|
||||
buildStart() {
|
||||
try {
|
||||
const scriptPath = path.resolve(__dirname, '../../scripts/generate-news-json.mjs');
|
||||
execSync(`node "${scriptPath}"`, { stdio: 'inherit' });
|
||||
} catch (e) {
|
||||
console.warn('Failed to generate news.json:', e);
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export default defineConfig(() => ({
|
||||
root: __dirname,
|
||||
cacheDir: '../../node_modules/.vite/apps/artcraft-webapp',
|
||||
@@ -93,7 +110,7 @@ export default defineConfig(() => ({
|
||||
port: 4301,
|
||||
host: 'localhost',
|
||||
},
|
||||
plugins: [cdnProxyPlugin(), nxViteTsPaths(), react(), wasm(), topLevelAwait()],
|
||||
plugins: [cdnProxyPlugin(), generateNewsPlugin(), nxViteTsPaths(), react(), wasm(), topLevelAwait()],
|
||||
// Uncomment this if you are using workers.
|
||||
// worker: {
|
||||
// plugins: [ nxViteTsPaths() ],
|
||||
|
||||
@@ -4,10 +4,6 @@
|
||||
# Manage the build command and publish directory here; leave those UI fields empty.
|
||||
# The Next.js runtime handles routing, the /cdn-media proxy rewrite, and the
|
||||
# legacy-route redirects declared in next.config.ts. Other media is in public/.
|
||||
#
|
||||
# Google Analytics: set NEXT_PUBLIC_GA_MEASUREMENT_ID (G-XXXXXXXXXX) in the
|
||||
# Netlify UI, scoped to the Production context so deploy previews don't
|
||||
# report. It is inlined at build time, so redeploy after changing it.
|
||||
|
||||
[build]
|
||||
command = "npm run build"
|
||||
|
||||
@@ -32,18 +32,6 @@ const nextConfig: NextConfig = {
|
||||
destination: "/apps/vectorcraft",
|
||||
permanent: true,
|
||||
},
|
||||
// PrintCraft was renamed PdfCraft. Old page links and share images
|
||||
// (cached by social sites) follow it.
|
||||
{
|
||||
source: "/apps/printcraft",
|
||||
destination: "/apps/pdfcraft",
|
||||
permanent: true,
|
||||
},
|
||||
{
|
||||
source: "/images/apps/printcraft/:file",
|
||||
destination: "/images/apps/pdfcraft/:file",
|
||||
permanent: true,
|
||||
},
|
||||
// Legacy share-link form (`/media?media=<token>`) from the Vite site;
|
||||
// the canonical route is /media/<token>. Mirrored in netlify.toml so
|
||||
// the edge answers it before the Next runtime.
|
||||
|
||||
|
Before Width: | Height: | Size: 86 KiB |
|
Before Width: | Height: | Size: 100 KiB |
|
Before Width: | Height: | Size: 75 KiB |
|
Before Width: | Height: | Size: 75 KiB |
|
Before Width: | Height: | Size: 123 KiB |
|
After Width: | Height: | Size: 86 KiB |
|
After Width: | Height: | Size: 100 KiB |
|
After Width: | Height: | Size: 75 KiB |
|
After Width: | Height: | Size: 75 KiB |
|
After Width: | Height: | Size: 123 KiB |
|
Before Width: | Height: | Size: 49 KiB After Width: | Height: | Size: 42 KiB |
|
Before Width: | Height: | Size: 77 KiB After Width: | Height: | Size: 71 KiB |
|
Before Width: | Height: | Size: 102 KiB After Width: | Height: | Size: 99 KiB |
|
Before Width: | Height: | Size: 78 KiB |
|
Before Width: | Height: | Size: 92 KiB After Width: | Height: | Size: 86 KiB |
|
Before Width: | Height: | Size: 78 KiB After Width: | Height: | Size: 74 KiB |
@@ -8,9 +8,7 @@ import {
|
||||
import {
|
||||
AppHeroDownloadButton,
|
||||
AppHeroOtherDownloadsLink,
|
||||
LauncherDownloadButton,
|
||||
} from "@/components/apps/app-downloads";
|
||||
import { craftLauncherRecommended } from "@/components/apps/launcher-callout";
|
||||
import {
|
||||
AppIcon,
|
||||
AppTab,
|
||||
@@ -81,10 +79,6 @@ export default async function CraftAppPage({ params }: { params: Params }) {
|
||||
|
||||
const name = craftAppName(app);
|
||||
const release = craftAppRelease(app);
|
||||
// The hero's download is ArtCraft Launcher, which installs this app and keeps
|
||||
// it updated; the app's own installers are in #get-it, under the launcher.
|
||||
const launcherDownloads = craftLauncherRecommended();
|
||||
const hasDownload = launcherDownloads.length > 0 || release !== null;
|
||||
const [hero] = app.shots;
|
||||
const [before, accent, after] = app.headline;
|
||||
const shareUrl = siteUrl(craftAppPath(app));
|
||||
@@ -151,22 +145,18 @@ export default async function CraftAppPage({ params }: { params: Params }) {
|
||||
mediaTag="Screenshot"
|
||||
>
|
||||
<div className="flex flex-col items-center gap-6">
|
||||
<div className="flex flex-wrap items-start justify-center gap-3">
|
||||
{launcherDownloads.length > 0 ? (
|
||||
<LauncherDownloadButton recommended={launcherDownloads} fallbackHref="#get-it" />
|
||||
) : (
|
||||
release && (
|
||||
<AppHeroDownloadButton
|
||||
name={name}
|
||||
recommended={craftReleaseDownloads(app, release).filter(
|
||||
(download) => download.recommended,
|
||||
)}
|
||||
/>
|
||||
)
|
||||
<div className="flex flex-wrap justify-center gap-3">
|
||||
{release && (
|
||||
<AppHeroDownloadButton
|
||||
name={name}
|
||||
recommended={craftReleaseDownloads(app, release).filter(
|
||||
(download) => download.recommended,
|
||||
)}
|
||||
/>
|
||||
)}
|
||||
<DiscordButton
|
||||
size="lg"
|
||||
variant={hasDownload ? "secondary" : "primary"}
|
||||
variant={release ? "secondary" : "primary"}
|
||||
>
|
||||
Join the Discord
|
||||
</DiscordButton>
|
||||
@@ -181,7 +171,7 @@ export default async function CraftAppPage({ params }: { params: Params }) {
|
||||
View on GitHub
|
||||
</Button>
|
||||
</div>
|
||||
{hasDownload && <AppHeroOtherDownloadsLink />}
|
||||
{release && <AppHeroOtherDownloadsLink />}
|
||||
<p className="hud-label text-faint">
|
||||
{app.platforms.join(" · ")} · Free and open source
|
||||
</p>
|
||||
|
||||
@@ -9,8 +9,6 @@ import {
|
||||
KeyboardIcon,
|
||||
} from "lucide-react";
|
||||
import AppCard from "@/components/apps/app-card";
|
||||
import { LauncherDownloadButton } from "@/components/apps/app-downloads";
|
||||
import { craftLauncherRecommended } from "@/components/apps/launcher-callout";
|
||||
import ShareBar from "@/components/apps/share-bar";
|
||||
import {
|
||||
CampaignSection,
|
||||
@@ -24,12 +22,6 @@ import { AppIcon, ColorAccent, appThemeClass } from "@/components/apps/app-wordm
|
||||
import { PageHeader } from "@/components/page/page-header";
|
||||
import RevealManager from "@/components/reveal-manager";
|
||||
import { Button } from "@/components/ui";
|
||||
import { trackAttrs } from "@/lib/analytics";
|
||||
import {
|
||||
CRAFT_LAUNCHER_NAME,
|
||||
CRAFT_LAUNCHER_REPO,
|
||||
craftLauncherReleasePageUrl,
|
||||
} from "@/lib/craft-launcher";
|
||||
import {
|
||||
CRAFTING_APPS,
|
||||
CRAFTING_APPS_COUNT_WORD,
|
||||
@@ -72,7 +64,6 @@ const PRINCIPLE_ICONS = [
|
||||
];
|
||||
|
||||
export default function CraftingAppsPage() {
|
||||
const launcherDownloads = craftLauncherRecommended();
|
||||
return (
|
||||
<>
|
||||
<RevealManager />
|
||||
@@ -89,19 +80,8 @@ export default function CraftingAppsPage() {
|
||||
}
|
||||
lede={`${COVERAGE}. Native, open-source apps from the ArtCraft team, built in Rust and free to use.`}
|
||||
>
|
||||
<div data-reveal className="mt-8 flex flex-wrap items-start gap-3">
|
||||
{launcherDownloads.length > 0 && (
|
||||
<LauncherDownloadButton
|
||||
recommended={launcherDownloads}
|
||||
fallbackHref={craftLauncherReleasePageUrl()}
|
||||
/>
|
||||
)}
|
||||
<DiscordButton
|
||||
size="lg"
|
||||
variant={launcherDownloads.length > 0 ? "secondary" : "primary"}
|
||||
>
|
||||
Join the Discord
|
||||
</DiscordButton>
|
||||
<div data-reveal className="mt-8 flex flex-wrap gap-3">
|
||||
<DiscordButton size="lg">Join the Discord</DiscordButton>
|
||||
<Button
|
||||
href={CRAFTING_APPS_GITHUB_ORG}
|
||||
variant="secondary"
|
||||
@@ -113,21 +93,6 @@ export default function CraftingAppsPage() {
|
||||
Browse on GitHub
|
||||
</Button>
|
||||
</div>
|
||||
{launcherDownloads.length > 0 && (
|
||||
<p data-reveal className="mt-4 max-w-xl leading-relaxed text-muted">
|
||||
<span className="text-ink">Recommended:</span> {CRAFT_LAUNCHER_NAME}{" "}
|
||||
installs every app below and keeps them all up to date. Each app
|
||||
also has its own installers on its page.{" "}
|
||||
<a
|
||||
href={CRAFT_LAUNCHER_REPO}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
className="text-ink underline decoration-line-strong underline-offset-4 hover:decoration-current"
|
||||
>
|
||||
Learn more
|
||||
</a>
|
||||
</p>
|
||||
)}
|
||||
<ShareBar
|
||||
url={siteUrl("/apps")}
|
||||
text={`Crafting Apps: ${CRAFTING_APPS_COUNT_WORD.toLowerCase()} open-source creative apps from ArtCraft`}
|
||||
@@ -143,7 +108,6 @@ export default function CraftingAppsPage() {
|
||||
key={app.slug}
|
||||
href={craftAppPath(app)}
|
||||
className={`group flex w-18 flex-col items-center gap-2 sm:w-20 ${appThemeClass(app)}`}
|
||||
{...trackAttrs("app_select", { app_name: app.slug })}
|
||||
>
|
||||
<AppIcon
|
||||
app={app}
|
||||
|
||||
@@ -321,14 +321,14 @@ html[data-ruler="on"]::-webkit-scrollbar {
|
||||
.app-vectorcraft { --app: #e8573f; --app-ink: #b83a24; }
|
||||
.app-filmcraft { --app: #8b5cf6; --app-ink: #6a3fd6; }
|
||||
.app-lightcraft { --app: #f2a516; --app-ink: #8a5800; }
|
||||
.app-pdfcraft { --app: #12a58a; --app-ink: #0a7563; }
|
||||
.app-printcraft { --app: #12a58a; --app-ink: #0a7563; }
|
||||
.app-effectcraft { --app: #e0368f; --app-ink: #b0206c; }
|
||||
.app-designcraft { --app: #7bb51c; --app-ink: #457008; }
|
||||
:root[data-theme="dark"] .app-photocraft { --app-ink: #7fb0ff; }
|
||||
:root[data-theme="dark"] .app-vectorcraft { --app-ink: #ff8a70; }
|
||||
:root[data-theme="dark"] .app-filmcraft { --app-ink: #b69cff; }
|
||||
:root[data-theme="dark"] .app-lightcraft { --app-ink: #ffc14d; }
|
||||
:root[data-theme="dark"] .app-pdfcraft { --app-ink: #3fd6b8; }
|
||||
:root[data-theme="dark"] .app-printcraft { --app-ink: #3fd6b8; }
|
||||
:root[data-theme="dark"] .app-effectcraft { --app-ink: #ff7ab8; }
|
||||
:root[data-theme="dark"] .app-designcraft { --app-ink: #a6e04a; }
|
||||
@media (prefers-color-scheme: dark) {
|
||||
@@ -336,7 +336,7 @@ html[data-ruler="on"]::-webkit-scrollbar {
|
||||
:root:not([data-theme="light"]) .app-vectorcraft { --app-ink: #ff8a70; }
|
||||
:root:not([data-theme="light"]) .app-filmcraft { --app-ink: #b69cff; }
|
||||
:root:not([data-theme="light"]) .app-lightcraft { --app-ink: #ffc14d; }
|
||||
:root:not([data-theme="light"]) .app-pdfcraft { --app-ink: #3fd6b8; }
|
||||
:root:not([data-theme="light"]) .app-printcraft { --app-ink: #3fd6b8; }
|
||||
:root:not([data-theme="light"]) .app-effectcraft { --app-ink: #ff7ab8; }
|
||||
:root:not([data-theme="light"]) .app-designcraft { --app-ink: #a6e04a; }
|
||||
}
|
||||
|
||||
@@ -9,7 +9,6 @@ import SiteNav from "@/components/site-nav";
|
||||
import SiteFooter from "@/components/site-footer";
|
||||
import IntroConductor from "@/components/intro-conductor";
|
||||
import LandingContext from "@/components/landing-context";
|
||||
import Analytics from "@/components/analytics";
|
||||
import MotionProvider from "@/components/motion-provider";
|
||||
import ScrollRuler from "@/components/ruler/scroll-ruler";
|
||||
import TunerPanel from "@/components/dev/tuner-panel";
|
||||
@@ -139,7 +138,6 @@ export default function RootLayout({
|
||||
</MotionProvider>
|
||||
<TunerPanel />
|
||||
<LandingContext />
|
||||
<Analytics />
|
||||
</body>
|
||||
</html>
|
||||
);
|
||||
|
||||
@@ -1,41 +0,0 @@
|
||||
"use client";
|
||||
|
||||
import { useEffect } from "react";
|
||||
import Script from "next/script";
|
||||
import { GA_MEASUREMENT_ID, trackClick } from "@/lib/analytics";
|
||||
|
||||
// Loads gtag.js after hydration and installs the one delegated click
|
||||
// listener behind the site's link and data-track events (lib/analytics.ts).
|
||||
// Renders nothing when no measurement id is configured.
|
||||
export default function Analytics() {
|
||||
useEffect(() => {
|
||||
if (!GA_MEASUREMENT_ID) return;
|
||||
// Capture phase: runs before click-to-play facades swap themselves out,
|
||||
// and before any handler can stop propagation. auxclick covers
|
||||
// middle-click "open in new tab".
|
||||
const onClick = (event: MouseEvent) => {
|
||||
if (event.type === "auxclick" && event.button !== 1) return;
|
||||
trackClick(event.target);
|
||||
};
|
||||
document.addEventListener("click", onClick, true);
|
||||
document.addEventListener("auxclick", onClick, true);
|
||||
return () => {
|
||||
document.removeEventListener("click", onClick, true);
|
||||
document.removeEventListener("auxclick", onClick, true);
|
||||
};
|
||||
}, []);
|
||||
|
||||
if (!GA_MEASUREMENT_ID) return null;
|
||||
|
||||
return (
|
||||
<>
|
||||
<Script
|
||||
src={`https://www.googletagmanager.com/gtag/js?id=${GA_MEASUREMENT_ID}`}
|
||||
strategy="afterInteractive"
|
||||
/>
|
||||
<Script id="ga-init" strategy="afterInteractive">
|
||||
{`window.dataLayer=window.dataLayer||[];window.gtag=function(){dataLayer.push(arguments);};gtag("js",new Date());gtag("config",${JSON.stringify(GA_MEASUREMENT_ID)});`}
|
||||
</Script>
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -2,7 +2,6 @@ import Link from "next/link";
|
||||
import { ArrowUpRightIcon } from "lucide-react";
|
||||
import { twMerge } from "tailwind-merge";
|
||||
import { Badge } from "@/components/ui";
|
||||
import { trackAttrs } from "@/lib/analytics";
|
||||
import {
|
||||
craftAppName,
|
||||
craftAppPath,
|
||||
@@ -31,7 +30,6 @@ export default function AppCard({
|
||||
<Link
|
||||
href={craftAppPath(app)}
|
||||
data-reveal
|
||||
{...trackAttrs("app_select", { app_name: app.slug })}
|
||||
className={twMerge(
|
||||
"group relative flex h-full flex-col bg-bg",
|
||||
appThemeClass(app),
|
||||
|
||||
@@ -9,7 +9,7 @@ import {
|
||||
} from "lucide-react";
|
||||
import { twMerge } from "tailwind-merge";
|
||||
import { WindowsIcon } from "@/components/icons";
|
||||
import { Button, type ButtonProps } from "@/components/ui";
|
||||
import { Button } from "@/components/ui";
|
||||
import {
|
||||
CRAFT_DESKTOP_PLATFORMS,
|
||||
type CraftDesktopPlatform,
|
||||
@@ -35,17 +35,9 @@ const PLATFORM_ICONS: Record<CraftDesktopPlatform, ReactNode> = {
|
||||
/**
|
||||
* One card per desktop platform: its recommended build as the main button,
|
||||
* then every other file for that platform. The visitor's platform is tagged
|
||||
* "Your system" and gets the solid button. These are the app's standalone
|
||||
* installers (the launcher above them installs the whole family), so each
|
||||
* button says "Standalone for <platform>" and each file list names the app.
|
||||
* "Your system" and gets the solid button.
|
||||
*/
|
||||
export function AppPlatformDownloads({
|
||||
name,
|
||||
downloads,
|
||||
}: {
|
||||
name: string;
|
||||
downloads: CraftDownload[];
|
||||
}) {
|
||||
export function AppPlatformDownloads({ downloads }: { downloads: CraftDownload[] }) {
|
||||
const detected = useDetectedDesktop();
|
||||
|
||||
return (
|
||||
@@ -58,7 +50,7 @@ export function AppPlatformDownloads({
|
||||
const others = files.filter((download) => download !== main);
|
||||
|
||||
return (
|
||||
<article key={platform} data-reveal className="flex min-w-0 flex-col bg-bg">
|
||||
<article key={platform} data-reveal className="flex flex-col bg-bg">
|
||||
<div className="flex items-center justify-between gap-4 border-b border-line px-6 py-2.5 md:px-8">
|
||||
<p className="hud-label text-muted">
|
||||
{isDetected ? (
|
||||
@@ -83,16 +75,14 @@ export function AppPlatformDownloads({
|
||||
<Button
|
||||
href={main.href}
|
||||
variant={isDetected ? "primary" : "secondary"}
|
||||
className="mt-6 w-full whitespace-normal text-center"
|
||||
className="mt-6 w-full"
|
||||
>
|
||||
<ArrowDownToLineIcon aria-hidden className="h-4 w-4" />
|
||||
Standalone for {platform}
|
||||
Download for {platform}
|
||||
</Button>
|
||||
{others.length > 0 && (
|
||||
<>
|
||||
<p className="hud-label mt-8 text-faint">
|
||||
Other {name} files for {platform}
|
||||
</p>
|
||||
<p className="hud-label mt-8 text-faint">Other {platform} downloads</p>
|
||||
<ul className="mt-2 border-t border-line">
|
||||
{others.map((download) => (
|
||||
<li key={download.href} className="border-b border-line">
|
||||
@@ -136,10 +126,7 @@ export function DownloadRow({ download, className }: { download: CraftDownload;
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The app's own hero CTA, used only while ArtCraft Launcher has no release: a
|
||||
* direct download once the OS is known, else a jump to #get-it.
|
||||
*/
|
||||
/** Hero CTA: a direct download once the OS is known, else a jump to #get-it. */
|
||||
export function AppHeroDownloadButton({
|
||||
recommended,
|
||||
name,
|
||||
@@ -157,49 +144,6 @@ export function AppHeroDownloadButton({
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* ArtCraft Launcher's CTA: a direct download for the visitor's system once
|
||||
* it is known (with the file's platform and build underneath), else a jump
|
||||
* to `fallbackHref`, where every platform is listed.
|
||||
*/
|
||||
export function LauncherDownloadButton({
|
||||
recommended,
|
||||
fallbackHref,
|
||||
size = "lg",
|
||||
variant = "primary",
|
||||
fullWidth = false,
|
||||
className,
|
||||
}: {
|
||||
recommended: CraftDownload[];
|
||||
fallbackHref: string;
|
||||
size?: ButtonProps["size"];
|
||||
variant?: ButtonProps["variant"];
|
||||
fullWidth?: boolean;
|
||||
className?: string;
|
||||
}) {
|
||||
const detected = useDetectedDesktop();
|
||||
const mine = detected && pickDownload(recommended, detected);
|
||||
return (
|
||||
<span className={twMerge("flex flex-col gap-2", className)}>
|
||||
<Button
|
||||
href={mine ? mine.href : fallbackHref}
|
||||
size={size}
|
||||
variant={variant}
|
||||
className={fullWidth ? "w-full" : undefined}
|
||||
>
|
||||
<ArrowDownToLineIcon aria-hidden className="h-4 w-4" />
|
||||
Download Launcher
|
||||
</Button>
|
||||
{/* Reserved line, so the layout doesn't shift when detection lands. It
|
||||
takes the button's width rather than setting it (w-0 min-w-full), so
|
||||
a long platform line can't push the buttons beside it away. */}
|
||||
<span aria-hidden={!mine} className="hud-label min-h-4 w-0 min-w-full whitespace-nowrap text-faint">
|
||||
{mine ? `For ${mine.group} · ${mine.label}` : "\u00a0"}
|
||||
</span>
|
||||
</span>
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Shown under the hero once the direct download is live, so visitors on
|
||||
* another machine, CPU or package format can still find their build.
|
||||
|
||||
@@ -6,8 +6,6 @@ import DiscordButton from "@/components/discord-button";
|
||||
import { GitHubIcon } from "@/components/icons";
|
||||
import { SectionShell, SectionEyebrow } from "@/components/landing/section-shell";
|
||||
import { Badge, Button, CopyButton } from "@/components/ui";
|
||||
import { trackAttrs } from "@/lib/analytics";
|
||||
import { CRAFT_LAUNCHER_RELEASE } from "@/lib/crafting-app-releases";
|
||||
import {
|
||||
CRAFTING_APPS,
|
||||
craftAppBuildCommand,
|
||||
@@ -26,7 +24,6 @@ import {
|
||||
import AppCard from "./app-card";
|
||||
import { AppPlatformDownloads, DownloadRow } from "./app-downloads";
|
||||
import { ColorAccent } from "./app-wordmark";
|
||||
import { LauncherCallout } from "./launcher-callout";
|
||||
|
||||
// Body sections of an /apps/<slug> page, in page order. Server components;
|
||||
// the client islands are the CopyButton and the OS-aware platform cards.
|
||||
@@ -34,12 +31,11 @@ import { LauncherCallout } from "./launcher-callout";
|
||||
const CELL_HEADING_CLASSES =
|
||||
"mt-4 font-display text-3xl font-medium leading-[1.05] tracking-[-0.03em] text-ink-strong sm:text-4xl";
|
||||
|
||||
// The hero carries shot 01. Window captures pair up, and an odd one out
|
||||
// leads at full width; portrait pages (layout spreads) sit four across.
|
||||
// The hero carries shot 01. Window captures lead with a full-width shot,
|
||||
// then pair; portrait pages (layout spreads) sit four across.
|
||||
export function AppGallery({ app, index }: { app: CraftApp; index: string }) {
|
||||
const [, ...shots] = app.shots;
|
||||
const portrait = shots.every((shot) => shot.portrait);
|
||||
const leadFullWidth = !portrait && shots.length % 2 === 1;
|
||||
return (
|
||||
<SectionShell id="gallery">
|
||||
<SectionEyebrow
|
||||
@@ -58,7 +54,7 @@ export function AppGallery({ app, index }: { app: CraftApp; index: string }) {
|
||||
<figure
|
||||
key={shot.file}
|
||||
data-reveal
|
||||
className={twMerge("bg-bg", leadFullWidth && i === 0 && "md:col-span-2")}
|
||||
className={twMerge("bg-bg", !portrait && i === 0 && "md:col-span-2")}
|
||||
>
|
||||
<a
|
||||
href={craftShotSourceUrl(app, shot)}
|
||||
@@ -100,8 +96,7 @@ export function AppGallery({ app, index }: { app: CraftApp; index: string }) {
|
||||
);
|
||||
}
|
||||
|
||||
// ArtCraft Launcher first (installs and updates every app), then this app's
|
||||
// own platform cards (your system first-class, every other build one click
|
||||
// Platform cards (your system first-class, every other build one click
|
||||
// away) above release details and build-from-source. Versions and files come
|
||||
// from crafting-app-releases.ts; a null release there drops the cards and
|
||||
// shows the Discord waitlist instead.
|
||||
@@ -121,27 +116,9 @@ export function AppGetIt({ app, index }: { app: CraftApp; index: string }) {
|
||||
annotation="Free · Open source"
|
||||
/>
|
||||
{release && (
|
||||
<>
|
||||
{CRAFT_LAUNCHER_RELEASE && (
|
||||
<>
|
||||
<LauncherCallout app={app} />
|
||||
<div data-reveal className="border-y border-line bg-bg p-6 md:p-10">
|
||||
<p className="hud-label text-faint">Option 2 · Standalone</p>
|
||||
<h3 className={CELL_HEADING_CLASSES}>
|
||||
Just {name}? Download it <ColorAccent>standalone</ColorAccent>.
|
||||
</h3>
|
||||
<p className="mt-4 max-w-xl leading-relaxed text-muted">
|
||||
These installers add {name} on its own, without the
|
||||
launcher. Come back here for new versions, or install the
|
||||
launcher later and it will find {name} and keep it updated.
|
||||
</p>
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
<div className="border-b border-line">
|
||||
<AppPlatformDownloads name={name} downloads={downloads} />
|
||||
</div>
|
||||
</>
|
||||
<div className="border-b border-line">
|
||||
<AppPlatformDownloads downloads={downloads} />
|
||||
</div>
|
||||
)}
|
||||
<div className="grid gap-px bg-line md:grid-cols-2">
|
||||
<div data-reveal className="flex flex-col bg-bg p-6 md:p-10">
|
||||
@@ -226,7 +203,6 @@ export function AppGetIt({ app, index }: { app: CraftApp; index: string }) {
|
||||
value={command}
|
||||
variant="ghost"
|
||||
className="h-9 border-l border-line px-3"
|
||||
{...trackAttrs("copy_command", { app_name: app.slug })}
|
||||
/>
|
||||
</div>
|
||||
<pre className="overflow-x-auto p-4 font-mono text-[13px] leading-relaxed text-ink">
|
||||
|
||||
@@ -1,98 +0,0 @@
|
||||
import { ArrowUpRightIcon, CheckIcon } from "lucide-react";
|
||||
import {
|
||||
CRAFT_LAUNCHER_NAME,
|
||||
CRAFT_LAUNCHER_REPO,
|
||||
craftLauncherDownloads,
|
||||
craftLauncherReleasePageUrl,
|
||||
} from "@/lib/craft-launcher";
|
||||
import { CRAFT_LAUNCHER_RELEASE } from "@/lib/crafting-app-releases";
|
||||
import { craftAppName, type CraftApp, type CraftDownload } from "@/lib/crafting-apps";
|
||||
import { LauncherDownloadButton } from "./app-downloads";
|
||||
import { ColorAccent } from "./app-wordmark";
|
||||
|
||||
// ArtCraft Launcher on the Crafting Apps pages: the "Recommended" callout
|
||||
// that leads every app's Get-it section. The /apps hub header and every app
|
||||
// page's hero use craftLauncherRecommended for their main download button. Server component;
|
||||
// the download button is the client island.
|
||||
|
||||
const CALLOUT_POINTS = ["Every Crafting App", "One-click updates", "Verified downloads"];
|
||||
|
||||
/** The launcher's main download per platform, for the OS-aware buttons. */
|
||||
export function craftLauncherRecommended(): CraftDownload[] {
|
||||
return craftLauncherDownloads().filter((download) => download.recommended);
|
||||
}
|
||||
|
||||
// Leads an app's Get-it section, above that app's own installers: the pitch
|
||||
// beside its download panel, matching the release and build-from-source
|
||||
// cells below it. Renders nothing while the launcher has no release.
|
||||
export function LauncherCallout({ app }: { app: CraftApp }) {
|
||||
const recommended = craftLauncherRecommended();
|
||||
if (!CRAFT_LAUNCHER_RELEASE || recommended.length === 0) return null;
|
||||
const name = craftAppName(app);
|
||||
return (
|
||||
<div
|
||||
data-reveal
|
||||
className="grid gap-px bg-line lg:grid-cols-[minmax(0,7fr)_minmax(0,4fr)]"
|
||||
>
|
||||
<div className="flex flex-col bg-bg p-6 md:p-10">
|
||||
<div className="flex items-center gap-3">
|
||||
<LauncherMark />
|
||||
<p className="hud-label text-(--app-ink,var(--accent-ink))">
|
||||
Option 1 · Recommended
|
||||
</p>
|
||||
</div>
|
||||
<h3 className="mt-6 font-display text-3xl font-medium leading-[1.05] tracking-[-0.03em] text-ink-strong sm:text-4xl">
|
||||
Get {name} with the <ColorAccent>Launcher</ColorAccent>.
|
||||
</h3>
|
||||
<p className="mt-4 max-w-xl leading-relaxed text-muted">
|
||||
{CRAFT_LAUNCHER_NAME} installs {name} and every other Crafting App
|
||||
from one place, picks the right build for your computer and keeps
|
||||
them all up to date. Install it once and every update is a click
|
||||
away.
|
||||
</p>
|
||||
<ul className="mt-6 flex flex-wrap gap-x-6 gap-y-2">
|
||||
{CALLOUT_POINTS.map((point) => (
|
||||
<li key={point} className="hud-label flex items-center gap-1.5 text-muted">
|
||||
<CheckIcon aria-hidden className="h-3.5 w-3.5 text-(--app-ink,var(--accent-ink))" />
|
||||
{point}
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
</div>
|
||||
<div className="flex flex-col justify-center bg-bg p-6 md:p-10">
|
||||
<p className="hud-label text-faint">
|
||||
{CRAFT_LAUNCHER_NAME} · v{CRAFT_LAUNCHER_RELEASE.version}
|
||||
</p>
|
||||
<LauncherDownloadButton
|
||||
recommended={recommended}
|
||||
fallbackHref={craftLauncherReleasePageUrl()}
|
||||
fullWidth
|
||||
className="mt-4"
|
||||
/>
|
||||
<a
|
||||
href={CRAFT_LAUNCHER_REPO}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
className="hud-label mt-4 flex w-fit items-center gap-1.5 text-muted hover:text-ink"
|
||||
>
|
||||
About the Launcher
|
||||
<ArrowUpRightIcon aria-hidden className="h-3.5 w-3.5" />
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
// The ArtCraft mark in a hairline square, like the platform icons in the
|
||||
// download cards. Decorative — it always sits beside the launcher's name.
|
||||
function LauncherMark() {
|
||||
return (
|
||||
<span
|
||||
aria-hidden
|
||||
className="flex h-10 w-10 shrink-0 items-center justify-center border border-line"
|
||||
>
|
||||
{/* eslint-disable-next-line @next/next/no-img-element */}
|
||||
<img src="/artcraft-icon.svg" alt="" className="h-5 w-5" />
|
||||
</span>
|
||||
);
|
||||
}
|
||||
@@ -1,6 +1,5 @@
|
||||
import { twMerge } from "tailwind-merge";
|
||||
import { Button, CopyButton } from "@/components/ui";
|
||||
import { trackAttrs } from "@/lib/analytics";
|
||||
|
||||
// Share strip: copy-link plus intent links for the networks our audience
|
||||
// posts to. Hairline cells (gap-px over the line color) so it wraps cleanly
|
||||
@@ -41,7 +40,6 @@ export default function ShareBar({
|
||||
text: string;
|
||||
className?: string;
|
||||
}) {
|
||||
const shareParams = { content_type: "page", item_id: new URL(url).pathname };
|
||||
return (
|
||||
<div
|
||||
className={twMerge(
|
||||
@@ -58,7 +56,6 @@ export default function ShareBar({
|
||||
copiedLabel="Link copied"
|
||||
variant="ghost"
|
||||
className={CELL_CLASSES}
|
||||
{...trackAttrs("share", { method: "copy_link", ...shareParams })}
|
||||
/>
|
||||
{TARGETS.map((target) => (
|
||||
<Button
|
||||
@@ -69,7 +66,6 @@ export default function ShareBar({
|
||||
aria-label={`Share on ${target.name}`}
|
||||
variant="ghost"
|
||||
className={CELL_CLASSES}
|
||||
{...trackAttrs("share", { method: target.name.toLowerCase(), ...shareParams })}
|
||||
>
|
||||
{target.name}
|
||||
</Button>
|
||||
|
||||
@@ -4,7 +4,6 @@ import { useState, type FormEvent } from "react";
|
||||
import { CheckIcon } from "lucide-react";
|
||||
import { twMerge } from "tailwind-merge";
|
||||
import { Button, Input, Label } from "@/components/ui";
|
||||
import { trackEvent } from "@/lib/analytics";
|
||||
import { webappUrl } from "@/lib/links";
|
||||
|
||||
// Submissions go to a Google Form whose responses land in a Google Sheet.
|
||||
@@ -84,7 +83,6 @@ export default function BetaForm() {
|
||||
// Google Forms sends no CORS headers, so this is fire-and-forget: the
|
||||
// response is opaque and "request completed" counts as success.
|
||||
await fetch(FORM_RESPONSE_URL, { method: "POST", mode: "no-cors", body });
|
||||
trackEvent("generate_lead", { form_name: "beta", user_type: userType ?? "" });
|
||||
setSubmitted(true);
|
||||
} catch {
|
||||
setSubmitError(
|
||||
|
||||
@@ -3,7 +3,6 @@
|
||||
import { useState, type FormEvent } from "react";
|
||||
import { CheckIcon, EyeIcon, EyeOffIcon } from "lucide-react";
|
||||
import { Button, Input, Modal } from "@/components/ui";
|
||||
import { trackEvent } from "@/lib/analytics";
|
||||
import { login, signup } from "@/lib/api";
|
||||
import {
|
||||
getLandingUrl,
|
||||
@@ -119,9 +118,8 @@ function SignupForm({
|
||||
maybeReferralCode: getReferralCode(),
|
||||
});
|
||||
setLoading(false);
|
||||
if (!result.success) return setError(result.errorMessage || "Failed to create account");
|
||||
trackEvent("sign_up", { method: "email", signup_source: signupSource });
|
||||
onSuccess();
|
||||
if (result.success) onSuccess();
|
||||
else setError(result.errorMessage || "Failed to create account");
|
||||
};
|
||||
|
||||
return (
|
||||
@@ -164,9 +162,8 @@ function LoginForm({ onSuccess }: { onSuccess: () => void }) {
|
||||
setLoading(true);
|
||||
const result = await login({ usernameOrEmail, password });
|
||||
setLoading(false);
|
||||
if (!result.success) return setError(result.errorMessage || "Invalid credentials");
|
||||
trackEvent("login", { method: "email" });
|
||||
onSuccess();
|
||||
if (result.success) onSuccess();
|
||||
else setError(result.errorMessage || "Invalid credentials");
|
||||
};
|
||||
|
||||
return (
|
||||
|
||||
@@ -3,7 +3,6 @@
|
||||
import { useState } from "react";
|
||||
import { PlayIcon } from "lucide-react";
|
||||
import { twMerge } from "tailwind-merge";
|
||||
import { trackAttrs } from "@/lib/analytics";
|
||||
import type { ExampleVideo } from "@/lib/campaign-data";
|
||||
|
||||
// Click-to-play facades: the mp4s (several MB each) are only requested
|
||||
@@ -70,11 +69,6 @@ function ExampleCard({
|
||||
onClick={() => setActive(true)}
|
||||
aria-label={`Play example: ${example.label}`}
|
||||
className="group absolute inset-0 h-full w-full"
|
||||
{...trackAttrs("video_start", {
|
||||
video_provider: "self_hosted",
|
||||
video_title: example.label,
|
||||
video_url: example.src,
|
||||
})}
|
||||
>
|
||||
{/* eslint-disable-next-line @next/next/no-img-element */}
|
||||
<img
|
||||
|
||||
@@ -9,7 +9,6 @@ import {
|
||||
XIcon,
|
||||
} from "lucide-react";
|
||||
import { Button, Select, Switch } from "@/components/ui";
|
||||
import { trackEvent } from "@/lib/analytics";
|
||||
import { generateVideo, getVideoJob } from "@/lib/api";
|
||||
import { H3, H3_GATE_PERKS, H3_SAMPLE_PROMPTS } from "@/lib/campaign-data";
|
||||
import { webappUrl } from "@/lib/links";
|
||||
@@ -102,7 +101,6 @@ export default function H3PromptBox() {
|
||||
setSubmitting(false);
|
||||
if (!result.success) {
|
||||
patch(id, { status: "failed", error: result.errorMessage });
|
||||
trackEvent("video_generation", { model: H3.modelId, status: "failed" });
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -116,15 +114,12 @@ export default function H3PromptBox() {
|
||||
if (job.status === "complete") {
|
||||
patch(id, { status: "complete", videoUrl: job.videoUrl });
|
||||
stops.current.delete(id);
|
||||
trackEvent("video_generation", { model: H3.modelId, status: "complete" });
|
||||
} else if (job.status === "failed") {
|
||||
patch(id, { status: "failed", error: job.error });
|
||||
stops.current.delete(id);
|
||||
trackEvent("video_generation", { model: H3.modelId, status: "failed" });
|
||||
} else if (attempts >= POLL_MAX_ATTEMPTS) {
|
||||
patch(id, { status: "failed", error: "Generation timed out" });
|
||||
stops.current.delete(id);
|
||||
trackEvent("video_generation", { model: H3.modelId, status: "timeout" });
|
||||
} else {
|
||||
setTimeout(poll, POLL_INTERVAL_MS);
|
||||
}
|
||||
@@ -141,14 +136,6 @@ export default function H3PromptBox() {
|
||||
setError(`Prompt exceeds the ${MAX_PROMPT_LENGTH} character limit for this model`);
|
||||
return;
|
||||
}
|
||||
trackEvent("prompt_submit", {
|
||||
model: H3.modelId,
|
||||
logged_in: loggedIn,
|
||||
aspect_ratio: aspectRatio,
|
||||
resolution,
|
||||
duration_seconds: Number(duration),
|
||||
sound,
|
||||
});
|
||||
if (!loggedIn) {
|
||||
setGateOpen(true);
|
||||
return;
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
|
||||
import { useState } from "react";
|
||||
import { PlayIcon } from "lucide-react";
|
||||
import { trackAttrs } from "@/lib/analytics";
|
||||
|
||||
// Click-to-load YouTube embed: a thumbnail until the visitor asks for the
|
||||
// player, so the landing never pays the iframe cost up front.
|
||||
@@ -33,11 +32,6 @@ export default function LiteYouTube({
|
||||
onClick={() => setActivated(true)}
|
||||
className="group absolute inset-0 h-full w-full"
|
||||
aria-label={`Play video: ${title}`}
|
||||
{...trackAttrs("video_start", {
|
||||
video_provider: "youtube",
|
||||
video_title: title,
|
||||
video_url: `https://www.youtube.com/watch?v=${videoId}`,
|
||||
})}
|
||||
>
|
||||
{/* eslint-disable-next-line @next/next/no-img-element */}
|
||||
<img
|
||||
|
||||
@@ -4,7 +4,6 @@ import { useEffect, useRef, useState, type ReactNode } from "react";
|
||||
import Link from "next/link";
|
||||
import { ArrowDownToLineIcon, CheckIcon, CopyIcon, ExternalLinkIcon } from "lucide-react";
|
||||
import { Button } from "@/components/ui";
|
||||
import { trackEvent } from "@/lib/analytics";
|
||||
import { MEDIA_LABELS, corsMediaUrl, mediaDetailLabel, mediaFormat, mediaKind, mediaThumbnail, safeMediaUrl, type MediaDimensions, type MediaPrompt, type SharedMedia } from "@/lib/media";
|
||||
import { getModelDisplayName, getModelIcon, getProviderDisplayName, getProviderIcon } from "@/lib/model-names";
|
||||
|
||||
@@ -36,7 +35,6 @@ export default function MediaDetails({ media, prompt, promptLoading, dimensions
|
||||
}
|
||||
|
||||
async function download() {
|
||||
trackEvent("media_download", { media_type: kind, file_extension: mediaFormat(media) });
|
||||
setDownloading(true);
|
||||
setMessage("");
|
||||
const controller = new AbortController();
|
||||
@@ -104,10 +102,7 @@ export default function MediaDetails({ media, prompt, promptLoading, dimensions
|
||||
<h2 className="hud-label text-faint">Prompt</h2>
|
||||
{prompt?.maybe_positive_prompt && (
|
||||
<button type="button" className="text-muted hover:text-ink" aria-label={copied === "prompt" ? "Prompt copied" : "Copy prompt"}
|
||||
onClick={() => {
|
||||
copy(prompt.maybe_positive_prompt!, "prompt");
|
||||
trackEvent("copy_prompt", { media_type: kind });
|
||||
}}>
|
||||
onClick={() => copy(prompt.maybe_positive_prompt!, "prompt")}>
|
||||
{copied === "prompt" ? <CheckIcon className="h-4 w-4" /> : <CopyIcon className="h-4 w-4" />}
|
||||
</button>
|
||||
)}
|
||||
@@ -148,10 +143,7 @@ export default function MediaDetails({ media, prompt, promptLoading, dimensions
|
||||
<Button onClick={download} loading={downloading} className="w-full">
|
||||
<ArrowDownToLineIcon aria-hidden className="h-4 w-4" />{downloading ? "Downloading" : "Download file"}
|
||||
</Button>
|
||||
<Button variant="secondary" className="w-full" onClick={() => {
|
||||
copy(`${SHARE_URL_BASE}${encodeURIComponent(media.token)}`, "link");
|
||||
trackEvent("share", { method: "copy_link", content_type: kind, item_id: media.token });
|
||||
}}>
|
||||
<Button variant="secondary" className="w-full" onClick={() => copy(`${SHARE_URL_BASE}${encodeURIComponent(media.token)}`, "link")}>
|
||||
{copied === "link" ? <CheckIcon aria-hidden className="h-4 w-4" /> : <CopyIcon aria-hidden className="h-4 w-4" />}
|
||||
{copied === "link" ? "Link copied" : "Copy share link"}
|
||||
</Button>
|
||||
|
||||
@@ -11,7 +11,6 @@ import {
|
||||
import { twMerge } from "tailwind-merge";
|
||||
import { SectionShell, SectionEyebrow } from "@/components/landing/section-shell";
|
||||
import { Button, Modal } from "@/components/ui";
|
||||
import { trackAttrs } from "@/lib/analytics";
|
||||
import { mediaUrl } from "@/lib/links";
|
||||
import {
|
||||
PRESS_KIT_CATEGORIES,
|
||||
@@ -149,10 +148,6 @@ function AssetCard({
|
||||
onClick={onPlay}
|
||||
aria-label={`Play ${asset.title}`}
|
||||
className="group relative aspect-video w-full overflow-hidden bg-bg-sunken"
|
||||
{...trackAttrs("video_start", {
|
||||
video_provider: asset.type === "embed" ? "youtube" : "self_hosted",
|
||||
video_title: asset.title,
|
||||
})}
|
||||
>
|
||||
{thumbnail}
|
||||
</button>
|
||||
@@ -180,7 +175,6 @@ function AssetCard({
|
||||
rel="noopener noreferrer"
|
||||
variant="secondary"
|
||||
className="w-full"
|
||||
{...trackAttrs("press_kit_download", { asset_name: asset.title })}
|
||||
>
|
||||
<ArrowDownToLineIcon aria-hidden className="h-3.5 w-3.5" />
|
||||
{asset.downloadLabel ?? "Download"}
|
||||
|
||||
@@ -3,7 +3,6 @@
|
||||
import { useState } from "react";
|
||||
import { CoinsIcon } from "lucide-react";
|
||||
import { Button } from "@/components/ui";
|
||||
import { trackEvent } from "@/lib/analytics";
|
||||
import { webappUrl } from "@/lib/links";
|
||||
import { useAccount } from "@/lib/use-account";
|
||||
import CreditsModal from "./credits-modal";
|
||||
@@ -29,10 +28,7 @@ export default function CreditsCta() {
|
||||
<Button
|
||||
type="button"
|
||||
variant="secondary"
|
||||
onClick={() => {
|
||||
setOpen(true);
|
||||
trackEvent("view_credit_packs", {});
|
||||
}}
|
||||
onClick={() => setOpen(true)}
|
||||
className="mt-6"
|
||||
>
|
||||
<CoinsIcon aria-hidden className="h-3.5 w-3.5 text-accent-ink" />
|
||||
|
||||
@@ -4,7 +4,6 @@ import { useState } from "react";
|
||||
import { CoinsIcon } from "lucide-react";
|
||||
import { twMerge } from "tailwind-merge";
|
||||
import { Button, Modal } from "@/components/ui";
|
||||
import { trackEvent } from "@/lib/analytics";
|
||||
import { creditsPackCheckout } from "@/lib/api";
|
||||
|
||||
type CreditPack = {
|
||||
@@ -36,20 +35,6 @@ export default function CreditsModal({
|
||||
const purchase = async (pack: CreditPack) => {
|
||||
setPurchasingId(pack.id);
|
||||
setError(null);
|
||||
trackEvent("begin_checkout", {
|
||||
currency: "USD",
|
||||
value: pack.priceUsd,
|
||||
checkout_type: "credits",
|
||||
items: [
|
||||
{
|
||||
item_id: pack.id,
|
||||
item_name: `${pack.total.toLocaleString("en-US")} credits`,
|
||||
item_category: "credits",
|
||||
price: pack.priceUsd,
|
||||
quantity: 1,
|
||||
},
|
||||
],
|
||||
});
|
||||
const result = await creditsPackCheckout(pack.id);
|
||||
if (result.success) {
|
||||
window.location.href = result.data.checkoutUrl;
|
||||
|
||||
@@ -11,7 +11,6 @@ import {
|
||||
} from "lucide-react";
|
||||
import { twMerge } from "tailwind-merge";
|
||||
import { Badge, Button, TabSelector } from "@/components/ui";
|
||||
import { trackEvent } from "@/lib/analytics";
|
||||
import {
|
||||
getPortalUrl,
|
||||
subscriptionCheckout,
|
||||
@@ -86,26 +85,9 @@ export default function PricingTable({
|
||||
if (plan.slug === activePlanSlug) return;
|
||||
setProcessingPlan(plan.slug);
|
||||
setError(null);
|
||||
const checkoutType = !user ? "signup" : hasActivePlan ? "switch" : "subscribe";
|
||||
const price = cadence === "yearly" ? plan.yearlyPrice : plan.monthlyPrice;
|
||||
trackEvent("begin_checkout", {
|
||||
currency: "USD",
|
||||
value: price,
|
||||
checkout_type: checkoutType,
|
||||
items: [
|
||||
{
|
||||
item_id: plan.slug,
|
||||
item_name: plan.name,
|
||||
item_category: "subscription",
|
||||
item_variant: cadence,
|
||||
price,
|
||||
quantity: 1,
|
||||
},
|
||||
],
|
||||
});
|
||||
const body = { plan: plan.slug, cadence };
|
||||
const redirected = redirect(
|
||||
checkoutType === "signup"
|
||||
!user
|
||||
? await userSignupSubscriptionCheckout({
|
||||
...body,
|
||||
maybeReferralUrl: getReferrer(),
|
||||
@@ -113,7 +95,7 @@ export default function PricingTable({
|
||||
maybeReferralUsername: getReferralUsername(),
|
||||
maybeReferralCode: getReferralCode(),
|
||||
})
|
||||
: checkoutType === "switch"
|
||||
: hasActivePlan
|
||||
? await switchPlan(body)
|
||||
: await subscriptionCheckout(body),
|
||||
);
|
||||
@@ -121,7 +103,6 @@ export default function PricingTable({
|
||||
};
|
||||
|
||||
const managePlan = async () => {
|
||||
trackEvent("manage_subscription", {});
|
||||
setManaging(true);
|
||||
setError(null);
|
||||
if (!redirect(await getPortalUrl())) setManaging(false);
|
||||
@@ -134,11 +115,7 @@ export default function PricingTable({
|
||||
<TabSelector
|
||||
tabs={BILLING_TABS}
|
||||
activeTab={cadence}
|
||||
onTabChange={(id) => {
|
||||
if (id === cadence) return;
|
||||
setCadence(id as BillingCadence);
|
||||
trackEvent("billing_cadence_change", { cadence: id });
|
||||
}}
|
||||
onTabChange={(id) => setCadence(id as BillingCadence)}
|
||||
tabClassName="w-24"
|
||||
/>
|
||||
<Badge
|
||||
@@ -386,7 +363,6 @@ function ContactButtons({ className }: { className?: string }) {
|
||||
try {
|
||||
await navigator.clipboard.writeText(CONTACT_EMAIL);
|
||||
setCopied(true);
|
||||
trackEvent("contact_click", { method: "copy_email" });
|
||||
setTimeout(() => setCopied(false), 1500);
|
||||
} catch {
|
||||
// Clipboard unavailable (insecure context) — the mailto still works.
|
||||
|
||||
@@ -86,7 +86,6 @@ export function Button({
|
||||
</>
|
||||
);
|
||||
|
||||
// Link buttons carry data-cta so analytics can report them as CTA clicks.
|
||||
if (href) {
|
||||
const anchorProps =
|
||||
rest as unknown as AnchorHTMLAttributes<HTMLAnchorElement>;
|
||||
@@ -97,7 +96,6 @@ export function Button({
|
||||
target={target}
|
||||
rel={rel}
|
||||
className={classes}
|
||||
data-cta=""
|
||||
{...anchorProps}
|
||||
>
|
||||
{content}
|
||||
@@ -105,7 +103,7 @@ export function Button({
|
||||
);
|
||||
}
|
||||
return (
|
||||
<Link href={href} className={classes} data-cta="" {...anchorProps}>
|
||||
<Link href={href} className={classes} {...anchorProps}>
|
||||
{content}
|
||||
</Link>
|
||||
);
|
||||
|
||||
@@ -1,232 +0,0 @@
|
||||
// Google Analytics 4 (gtag.js). Off unless NEXT_PUBLIC_GA_MEASUREMENT_ID is
|
||||
// set at build time, so local dev stays out of the reports.
|
||||
//
|
||||
// Page views come from GA4 Enhanced Measurement (it records App Router
|
||||
// navigations via history changes), along with scrolls, outbound clicks and
|
||||
// common file downloads. On top of that, the site sends the events below.
|
||||
// Most link clicks need no per-component code: one delegated listener
|
||||
// (see components/analytics.tsx) classifies them by href, and elements
|
||||
// tagged with trackAttrs() report their event when clicked. Every delegated
|
||||
// event also carries `link_location` (nav, footer, modal or the enclosing
|
||||
// section id).
|
||||
|
||||
import { SOCIAL_LINKS, WEBAPP_URL } from "./links";
|
||||
|
||||
export const GA_MEASUREMENT_ID =
|
||||
process.env.NEXT_PUBLIC_GA_MEASUREMENT_ID?.trim() || undefined;
|
||||
|
||||
export type AnalyticsItem = {
|
||||
item_id: string;
|
||||
item_name: string;
|
||||
item_category: "subscription" | "credits";
|
||||
item_variant?: string;
|
||||
price: number;
|
||||
quantity: number;
|
||||
};
|
||||
|
||||
type AnalyticsEvents = {
|
||||
// Link clicks, classified by href.
|
||||
app_download: {
|
||||
app_name: string;
|
||||
app_version: string;
|
||||
platform: "macos" | "windows" | "linux" | "other";
|
||||
file_name: string;
|
||||
};
|
||||
webapp_click: { destination: string; link_text: string };
|
||||
social_click: { network: string };
|
||||
contact_click: { method: "mailto" | "copy_email" };
|
||||
cta_click: { link_url: string; link_text: string };
|
||||
|
||||
// Clicks on elements tagged with trackAttrs().
|
||||
video_start: { video_provider: string; video_title: string; video_url?: string };
|
||||
share: { method: string; content_type: string; item_id: string };
|
||||
copy_command: { app_name: string };
|
||||
app_select: { app_name: string };
|
||||
press_kit_download: { asset_name: string };
|
||||
|
||||
// Sent from component code.
|
||||
generate_lead: { form_name: string; user_type: string };
|
||||
sign_up: { method: "email"; signup_source: string };
|
||||
login: { method: "email" };
|
||||
prompt_submit: {
|
||||
model: string;
|
||||
logged_in: boolean;
|
||||
aspect_ratio: string;
|
||||
resolution: string;
|
||||
duration_seconds: number;
|
||||
sound: boolean;
|
||||
};
|
||||
video_generation: { model: string; status: "complete" | "failed" | "timeout" };
|
||||
billing_cadence_change: { cadence: string };
|
||||
view_credit_packs: Record<string, never>;
|
||||
begin_checkout: {
|
||||
currency: "USD";
|
||||
value: number;
|
||||
checkout_type: "signup" | "subscribe" | "switch" | "credits";
|
||||
items: AnalyticsItem[];
|
||||
};
|
||||
manage_subscription: Record<string, never>;
|
||||
copy_prompt: { media_type: string };
|
||||
media_download: { media_type: string; file_extension: string };
|
||||
};
|
||||
|
||||
export type AnalyticsEvent = keyof AnalyticsEvents;
|
||||
|
||||
type ParamValue = string | number | boolean | AnalyticsItem[] | undefined;
|
||||
type EventParams = Record<string, ParamValue>;
|
||||
|
||||
declare global {
|
||||
interface Window {
|
||||
gtag?: (...args: unknown[]) => void;
|
||||
}
|
||||
}
|
||||
|
||||
export function trackEvent<E extends AnalyticsEvent>(
|
||||
event: E,
|
||||
params: AnalyticsEvents[E],
|
||||
): void {
|
||||
send(event, params);
|
||||
}
|
||||
|
||||
/**
|
||||
* Data attributes that make an element report `event` when clicked. Works in
|
||||
* server components (no handler needed); spread onto the element or Button.
|
||||
*/
|
||||
export function trackAttrs<E extends AnalyticsEvent>(
|
||||
event: E,
|
||||
params: AnalyticsEvents[E],
|
||||
): Record<`data-track${string}`, string> {
|
||||
const attrs: Record<`data-track${string}`, string> = { "data-track": event };
|
||||
for (const [key, value] of Object.entries(params)) {
|
||||
if (value !== undefined) attrs[`data-track-${key}`] = String(value);
|
||||
}
|
||||
return attrs;
|
||||
}
|
||||
|
||||
/** Delegated click handler: reports tagged elements and classified links. */
|
||||
export function trackClick(target: EventTarget | null): void {
|
||||
if (!(target instanceof Element)) return;
|
||||
|
||||
const tagged = target.closest<HTMLElement>("[data-track]");
|
||||
if (tagged?.dataset.track) {
|
||||
send(tagged.dataset.track, {
|
||||
...taggedParams(tagged),
|
||||
link_location: linkLocation(tagged),
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
const anchor = target.closest<HTMLAnchorElement>("a[href]");
|
||||
const link = anchor && classifyLink(anchor);
|
||||
if (link) send(link.event, { ...link.params, link_location: linkLocation(anchor) });
|
||||
}
|
||||
|
||||
function send(event: string, params: EventParams): void {
|
||||
if (!GA_MEASUREMENT_ID) return;
|
||||
window.gtag?.("event", event, params);
|
||||
}
|
||||
|
||||
const WEBAPP_ORIGIN = new URL(WEBAPP_URL).origin;
|
||||
|
||||
// Hostname → network, derived from SOCIAL_LINKS.
|
||||
const SOCIAL_NETWORKS = new Map(
|
||||
Object.entries(SOCIAL_LINKS).map(([name, href]) => [
|
||||
bareHostname(new URL(href).hostname),
|
||||
name.toLowerCase(),
|
||||
]),
|
||||
);
|
||||
|
||||
// github.com/<org>/<repo>/releases/download/<tag>/<file>: the ArtCraft
|
||||
// installers and every Crafting App download.
|
||||
const RELEASE_ASSET_PATH = /^\/[^/]+\/([^/]+)\/releases\/download\/([^/]+)\/([^/]+)$/;
|
||||
|
||||
type ClassifiedLink = {
|
||||
[E in AnalyticsEvent]: { event: E; params: AnalyticsEvents[E] };
|
||||
}[AnalyticsEvent];
|
||||
|
||||
function classifyLink(anchor: HTMLAnchorElement): ClassifiedLink | null {
|
||||
let url: URL;
|
||||
try {
|
||||
url = new URL(anchor.href);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
|
||||
if (url.protocol === "mailto:") {
|
||||
return { event: "contact_click", params: { method: "mailto" } };
|
||||
}
|
||||
|
||||
const release = url.hostname === "github.com" && RELEASE_ASSET_PATH.exec(url.pathname);
|
||||
if (release) {
|
||||
const [, repo, tag, file] = release;
|
||||
const fileName = decodeURIComponent(file);
|
||||
return {
|
||||
event: "app_download",
|
||||
params: {
|
||||
app_name: repo,
|
||||
app_version: tag.replace(/^\D*/, ""),
|
||||
platform: downloadPlatform(fileName),
|
||||
file_name: fileName,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
if (url.origin === WEBAPP_ORIGIN) {
|
||||
return {
|
||||
event: "webapp_click",
|
||||
params: { destination: url.pathname, link_text: linkText(anchor) },
|
||||
};
|
||||
}
|
||||
|
||||
const network = SOCIAL_NETWORKS.get(bareHostname(url.hostname));
|
||||
if (network) return { event: "social_click", params: { network } };
|
||||
|
||||
// Remaining Button links (internal CTAs, other outbound buttons). Plain
|
||||
// text links are left to page views and Enhanced Measurement.
|
||||
if (anchor.hasAttribute("data-cta")) {
|
||||
return {
|
||||
event: "cta_click",
|
||||
params: {
|
||||
link_url: url.origin === window.location.origin ? url.pathname + url.hash : url.href,
|
||||
link_text: linkText(anchor),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
function linkLocation(element: Element): string {
|
||||
if (element.closest("#site-nav")) return "nav";
|
||||
if (element.closest("footer")) return "footer";
|
||||
if (element.closest('[role="dialog"]')) return "modal";
|
||||
return element.closest("section[id]")?.id || "page";
|
||||
}
|
||||
|
||||
// data-track-<param> attributes surface in the dataset as track<Param>.
|
||||
function taggedParams(element: HTMLElement): EventParams {
|
||||
const params: EventParams = {};
|
||||
for (const [key, value] of Object.entries(element.dataset)) {
|
||||
if (key.length > 5 && key.startsWith("track")) {
|
||||
params[key.charAt(5).toLowerCase() + key.slice(6)] = value;
|
||||
}
|
||||
}
|
||||
return params;
|
||||
}
|
||||
|
||||
function downloadPlatform(fileName: string): AnalyticsEvents["app_download"]["platform"] {
|
||||
if (/\.dmg$|macos/i.test(fileName)) return "macos";
|
||||
if (/\.(exe|msi)$|windows/i.test(fileName)) return "windows";
|
||||
if (/\.(appimage|deb|rpm)$|linux/i.test(fileName)) return "linux";
|
||||
return "other";
|
||||
}
|
||||
|
||||
// GA truncates parameter values at 100 characters.
|
||||
function linkText(anchor: HTMLAnchorElement): string {
|
||||
const text = anchor.getAttribute("aria-label") || anchor.textContent || "";
|
||||
return text.replace(/\s+/g, " ").trim().slice(0, 100);
|
||||
}
|
||||
|
||||
function bareHostname(hostname: string): string {
|
||||
return hostname.replace(/^www\./, "");
|
||||
}
|
||||
@@ -1,31 +0,0 @@
|
||||
import { CRAFT_LAUNCHER_RELEASE } from "./crafting-app-releases";
|
||||
import {
|
||||
CRAFTING_APPS_GITHUB_ORG,
|
||||
craftReleaseTag,
|
||||
releaseDownloads,
|
||||
type CraftDownload,
|
||||
} from "./crafting-apps";
|
||||
|
||||
// ArtCraft Launcher: one small native app that installs, updates and opens
|
||||
// every Crafting App. The /apps hub's main button downloads it and every
|
||||
// /apps/<slug> Get-it section recommends it above that app's own installers.
|
||||
//
|
||||
// Facts come from the craft-launcher README — keep them in sync. Version and
|
||||
// files live in crafting-app-releases.ts (CRAFT_LAUNCHER_RELEASE); null there
|
||||
// hides every launcher download.
|
||||
|
||||
export const CRAFT_LAUNCHER_NAME = "ArtCraft Launcher";
|
||||
export const CRAFT_LAUNCHER_REPO = `${CRAFTING_APPS_GITHUB_ORG}/craft-launcher`;
|
||||
|
||||
/** Every launcher release file, or none while the release is null. */
|
||||
export function craftLauncherDownloads(): CraftDownload[] {
|
||||
return CRAFT_LAUNCHER_RELEASE
|
||||
? releaseDownloads(CRAFT_LAUNCHER_REPO, "artcraft-launcher", CRAFT_LAUNCHER_RELEASE)
|
||||
: [];
|
||||
}
|
||||
|
||||
export function craftLauncherReleasePageUrl(): string {
|
||||
return CRAFT_LAUNCHER_RELEASE
|
||||
? `${CRAFT_LAUNCHER_REPO}/releases/tag/${craftReleaseTag(CRAFT_LAUNCHER_RELEASE)}`
|
||||
: `${CRAFT_LAUNCHER_REPO}/releases`;
|
||||
}
|
||||
@@ -3,14 +3,17 @@ import type { CraftAppSlug } from "./crafting-apps";
|
||||
// GitHub releases for the Crafting Apps, which drive every download button on
|
||||
// the /apps pages.
|
||||
//
|
||||
// To bump an app: change its version in CRAFT_APP_RELEASES (each app has its
|
||||
// own, e.g. `craftRelease("0.4.0")`).
|
||||
// To bump every app: change CRAFTING_APPS_LATEST_VERSION.
|
||||
// To pin one app to another version: give it its own release in
|
||||
// CRAFT_APP_RELEASES, e.g. `craftRelease("0.1.1")`.
|
||||
// To hide an app's downloads (Discord waitlist instead): set it to null.
|
||||
// If a release renames, adds or drops files, edit CRAFT_RELEASE_ASSETS, or
|
||||
// pass a custom list as craftRelease's second argument for a single app.
|
||||
//
|
||||
// Release tags are `v<version>` and live at github.com/storytold/<slug>.
|
||||
|
||||
export const CRAFTING_APPS_LATEST_VERSION = "0.2.0";
|
||||
|
||||
export type CraftDesktopPlatform = "macOS" | "Windows" | "Linux";
|
||||
|
||||
/** Order of the per-platform download buttons. */
|
||||
@@ -19,7 +22,7 @@ export const CRAFT_DESKTOP_PLATFORMS: CraftDesktopPlatform[] = ["macOS", "Window
|
||||
export type CraftReleaseAsset = {
|
||||
/** Asset filename; `{slug}` and `{version}` are filled in per app. */
|
||||
file: string;
|
||||
/** Group in the full file list. "Other" holds the FreeBSD build, web build and checksums. */
|
||||
/** Group in the full file list. "Other" holds the CLI, web build and checksums. */
|
||||
group: CraftDesktopPlatform | "Other";
|
||||
/** Short name, e.g. "Installer (x64)". */
|
||||
label: string;
|
||||
@@ -40,8 +43,6 @@ export type CraftRelease = {
|
||||
version: string;
|
||||
/** Git tag. Defaults to `v<version>`. */
|
||||
tag?: string;
|
||||
/** Fills `{slug}` in filenames. Defaults to the app's slug; a release from before a rename keeps the old one. */
|
||||
fileSlug?: string;
|
||||
assets: readonly CraftReleaseAsset[];
|
||||
};
|
||||
|
||||
@@ -91,20 +92,6 @@ export const CRAFT_RELEASE_ASSETS: readonly CraftReleaseAsset[] = [
|
||||
description: "Unzip and run on 32-bit Windows.",
|
||||
arch: "x86",
|
||||
},
|
||||
{
|
||||
file: "{slug}-{version}-windows-arm64.msi",
|
||||
group: "Windows",
|
||||
label: "Installer (ARM64)",
|
||||
description: "Windows 11 on ARM laptops.",
|
||||
arch: "aarch64",
|
||||
},
|
||||
{
|
||||
file: "{slug}-{version}-windows-arm64-portable.zip",
|
||||
group: "Windows",
|
||||
label: "Portable (ARM64)",
|
||||
description: "Unzip and run on Windows on ARM.",
|
||||
arch: "aarch64",
|
||||
},
|
||||
{
|
||||
file: "{slug}-{version}-linux-x86_64.AppImage",
|
||||
group: "Linux",
|
||||
@@ -113,13 +100,6 @@ export const CRAFT_RELEASE_ASSETS: readonly CraftReleaseAsset[] = [
|
||||
arch: "x86_64",
|
||||
recommended: true,
|
||||
},
|
||||
{
|
||||
file: "{slug}-{version}-linux-x86_64.flatpak",
|
||||
group: "Linux",
|
||||
label: "Flatpak (x86_64)",
|
||||
description: "Sandboxed. Install with flatpak install --user.",
|
||||
arch: "x86_64",
|
||||
},
|
||||
{
|
||||
file: "{slug}-{version}-linux-x86_64.deb",
|
||||
group: "Linux",
|
||||
@@ -149,13 +129,6 @@ export const CRAFT_RELEASE_ASSETS: readonly CraftReleaseAsset[] = [
|
||||
arch: "aarch64",
|
||||
recommended: true,
|
||||
},
|
||||
{
|
||||
file: "{slug}-{version}-linux-aarch64.flatpak",
|
||||
group: "Linux",
|
||||
label: "Flatpak (ARM64)",
|
||||
description: "Sandboxed, on 64-bit ARM.",
|
||||
arch: "aarch64",
|
||||
},
|
||||
{
|
||||
file: "{slug}-{version}-linux-aarch64.deb",
|
||||
group: "Linux",
|
||||
@@ -177,12 +150,6 @@ export const CRAFT_RELEASE_ASSETS: readonly CraftReleaseAsset[] = [
|
||||
description: "Plain binaries for 64-bit ARM.",
|
||||
arch: "aarch64",
|
||||
},
|
||||
{
|
||||
file: "{slug}-{version}-freebsd-x86_64.tar.gz",
|
||||
group: "Other",
|
||||
label: "FreeBSD (x86_64)",
|
||||
description: "Plain binaries for FreeBSD 14.",
|
||||
},
|
||||
{
|
||||
file: "{slug}-web-{version}.zip",
|
||||
group: "Other",
|
||||
@@ -204,21 +171,15 @@ export function craftRelease(
|
||||
return { version, assets };
|
||||
}
|
||||
|
||||
// ArtCraft Launcher (github.com/storytold/craft-launcher) installs and updates
|
||||
// every app above, and the /apps pages recommend it first. Its releases ship
|
||||
// the same file set minus the macOS CLI, named artcraft-launcher-*.
|
||||
export const CRAFT_LAUNCHER_RELEASE: CraftRelease | null = craftRelease(
|
||||
"0.2.0",
|
||||
CRAFT_RELEASE_ASSETS.filter((asset) => !asset.file.startsWith("{slug}-cli-")),
|
||||
);
|
||||
const LATEST = craftRelease(CRAFTING_APPS_LATEST_VERSION);
|
||||
|
||||
// Every app must be listed, so a new slug fails the type check until it is.
|
||||
export const CRAFT_APP_RELEASES: Record<CraftAppSlug, CraftRelease | null> = {
|
||||
photocraft: craftRelease("0.5.0"),
|
||||
vectorcraft: craftRelease("0.7.0"),
|
||||
filmcraft: craftRelease("0.4.0"),
|
||||
lightcraft: craftRelease("0.4.0"),
|
||||
pdfcraft: craftRelease("0.4.0"),
|
||||
effectcraft: craftRelease("0.6.0"),
|
||||
designcraft: craftRelease("0.4.0"),
|
||||
photocraft: LATEST,
|
||||
vectorcraft: LATEST,
|
||||
filmcraft: LATEST,
|
||||
lightcraft: LATEST,
|
||||
printcraft: LATEST,
|
||||
effectcraft: LATEST,
|
||||
designcraft: LATEST,
|
||||
};
|
||||
@@ -25,7 +25,7 @@ export type CraftAppSlug =
|
||||
| "vectorcraft"
|
||||
| "filmcraft"
|
||||
| "lightcraft"
|
||||
| "pdfcraft"
|
||||
| "printcraft"
|
||||
| "effectcraft"
|
||||
| "designcraft";
|
||||
|
||||
@@ -177,31 +177,25 @@ export const CRAFTING_APPS: CraftApp[] = [
|
||||
file: "hero.webp",
|
||||
source: "shot-1-neon.png",
|
||||
caption: "Neon Drive — made in VectorCraft",
|
||||
alt: "VectorCraft editing the Neon Drive poster, with the title's live outer glow settings in the Appearance panel and its character settings in the Properties panel",
|
||||
alt: "VectorCraft editing a synthwave poster titled Neon Drive with an outer glow in the Appearance panel",
|
||||
},
|
||||
{
|
||||
file: "ribbons.webp",
|
||||
source: "shot-2-ribbons.png",
|
||||
caption: "Live blends and layers",
|
||||
alt: "VectorCraft showing three live blend ribbons clipped to the artboard, one selected with its key paths, and the Layers panel listing the clip group and blends",
|
||||
},
|
||||
{
|
||||
file: "bezier.webp",
|
||||
source: "shot-4-bezier.png",
|
||||
caption: "Pen and direct selection",
|
||||
alt: "VectorCraft direct-selecting the anchor points and bezier handles of a crescent moon, with the contextual task bar below it",
|
||||
},
|
||||
{
|
||||
file: "perspective.webp",
|
||||
source: "shot-5-perspective.png",
|
||||
caption: "Perspective grid, still editable",
|
||||
alt: "VectorCraft with two lit building facades drawn on the left and right planes of a two-point perspective grid at sunset",
|
||||
caption: "Live blends between editable spines",
|
||||
alt: "VectorCraft showing live blends: 70 smooth-color steps between two ribbon paths",
|
||||
},
|
||||
{
|
||||
file: "sheet.webp",
|
||||
source: "shot-3-sheet.png",
|
||||
caption: "Booleans, mesh, repeat and envelope",
|
||||
alt: "VectorCraft in its light theme with four artboards: shape booleans, a gradient mesh, a live radial repeat and an envelope distort",
|
||||
alt: "VectorCraft feature sheet with exact booleans, a gradient mesh, a live radial repeat and an envelope distort",
|
||||
},
|
||||
{
|
||||
file: "bezier.webp",
|
||||
source: "shot-4-bezier.png",
|
||||
caption: "Direct selection on bezier anchors",
|
||||
alt: "VectorCraft direct-selecting the anchors and handles of a crescent moon path",
|
||||
},
|
||||
],
|
||||
},
|
||||
@@ -334,8 +328,8 @@ export const CRAFTING_APPS: CraftApp[] = [
|
||||
],
|
||||
},
|
||||
{
|
||||
slug: "pdfcraft",
|
||||
prefix: "Pdf",
|
||||
slug: "printcraft",
|
||||
prefix: "Print",
|
||||
category: "PDF workbench",
|
||||
status: "Early alpha",
|
||||
headline: ["The ", "open-source", " PDF workbench."],
|
||||
@@ -373,27 +367,27 @@ export const CRAFTING_APPS: CraftApp[] = [
|
||||
shots: [
|
||||
{
|
||||
file: "hero.webp",
|
||||
source: "pdfcraft-viewer.png",
|
||||
source: "printcraft-viewer.png",
|
||||
caption: "Viewer with threaded comments",
|
||||
alt: "PdfCraft showing a showcase PDF cover with the All tools panel and threaded comments",
|
||||
alt: "PrintCraft showing a showcase PDF cover with the tools sidebar and threaded comments",
|
||||
},
|
||||
{
|
||||
file: "organize.webp",
|
||||
source: "pdfcraft-organize.png",
|
||||
source: "printcraft-organize.png",
|
||||
caption: "Organize pages like cards",
|
||||
alt: "PdfCraft page organizer with three pages selected and the page tools in the toolbar above",
|
||||
alt: "PrintCraft page organizer with three pages selected for rotate, delete and extract",
|
||||
},
|
||||
{
|
||||
file: "palette.webp",
|
||||
source: "pdfcraft-palette.png",
|
||||
source: "printcraft-palette.png",
|
||||
caption: "A command palette for every tool",
|
||||
alt: "PdfCraft command palette searching for page and listing the page tools",
|
||||
alt: "PrintCraft command palette filtering page tools by the word page",
|
||||
},
|
||||
{
|
||||
file: "twoup.webp",
|
||||
source: "pdfcraft-twoup.png",
|
||||
source: "printcraft-twoup.png",
|
||||
caption: "Two-up Read mode, dark theme",
|
||||
alt: "PdfCraft two-up Read mode in the dark theme showing typeset specimen pages",
|
||||
alt: "PrintCraft two-up Read mode in the dark theme showing typeset specimen pages",
|
||||
},
|
||||
],
|
||||
},
|
||||
@@ -652,19 +646,10 @@ export function craftReleaseDownloads(
|
||||
app: CraftApp,
|
||||
release: CraftRelease,
|
||||
): CraftDownload[] {
|
||||
return releaseDownloads(craftAppRepo(app), app.slug, release);
|
||||
}
|
||||
|
||||
/** Every file in a release of `repo`, in catalog order; `{slug}` defaults to `fileSlug`. */
|
||||
export function releaseDownloads(
|
||||
repo: string,
|
||||
fileSlug: string,
|
||||
release: CraftRelease,
|
||||
): CraftDownload[] {
|
||||
const base = `${repo}/releases/download/${craftReleaseTag(release)}`;
|
||||
const base = `${craftAppRepo(app)}/releases/download/${craftReleaseTag(release)}`;
|
||||
return release.assets.map(({ file, ...asset }) => {
|
||||
const fileName = file
|
||||
.replaceAll("{slug}", release.fileSlug ?? fileSlug)
|
||||
.replaceAll("{slug}", app.slug)
|
||||
.replaceAll("{version}", release.version);
|
||||
return { ...asset, fileName, href: `${base}/${fileName}` };
|
||||
});
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
# Google OAuth Configuration
|
||||
# Get the Client ID from the Google Cloud Console:
|
||||
# https://console.cloud.google.com/apis/credentials
|
||||
VITE_GOOGLE_CLIENT_ID=YOUR_CLIENT_ID.apps.googleusercontent.com
|
||||
@@ -0,0 +1,143 @@
|
||||
# Website → Webapp Migration
|
||||
|
||||
The marketing site (`artcraft-website`) is being split apart from the app
|
||||
(`artcraft-webapp`). Anything that isn't pure marketing content belongs in
|
||||
the webapp at `app.getartcraft.com`.
|
||||
|
||||
The migration is **gated by a single boolean** so it can be reverted while
|
||||
we soak the new redirect behavior in production. When we're confident the
|
||||
webapp is handling everything cleanly, do the **cleanup pass** below to
|
||||
permanently delete everything that's only kept around for the rollback path.
|
||||
|
||||
---
|
||||
|
||||
## The switch
|
||||
|
||||
`src/config/links.ts`:
|
||||
|
||||
```ts
|
||||
export const USE_WEBAPP_FOR_APP_FEATURES = true;
|
||||
```
|
||||
|
||||
- `true` (default): legacy paths redirect to the webapp; navbar shows
|
||||
"Launch App"; in-page links go cross-origin to `app.getartcraft.com`.
|
||||
- `false`: everything runs locally on the marketing site (the original
|
||||
pre-split behavior).
|
||||
|
||||
In dev, `WEBAPP_URL` auto-switches to `http://localhost:4201/` so cross-app
|
||||
links work against the local webapp Vite server.
|
||||
|
||||
---
|
||||
|
||||
## What's behind the flag
|
||||
|
||||
### Pages that exist locally only as a fallback
|
||||
|
||||
When `USE_WEBAPP_FOR_APP_FEATURES = true`, these routes mount
|
||||
`<WebappRedirect to="…" />` instead of the page component. The page files
|
||||
are kept on disk so we can roll back by flipping the flag.
|
||||
|
||||
- `src/pages/login/`
|
||||
- `src/pages/signup/`
|
||||
- `src/pages/forgot-password/`
|
||||
- `src/pages/create-image/`
|
||||
- `src/pages/create-video/`
|
||||
- `src/pages/create-vfx/` (the `/background-change` route)
|
||||
- `src/pages/library/`
|
||||
- `src/pages/referrals/`
|
||||
- `src/pages/welcome/`
|
||||
- `src/pages/onboarding/`
|
||||
- `src/pages/checkout/` (success + cancel)
|
||||
|
||||
### Components only used by the above
|
||||
|
||||
- `src/components/auth/` (SignupForm, GoogleLoginButton)
|
||||
- `src/components/credits-modal.tsx`
|
||||
- `src/components/generation-gallery/`
|
||||
- `src/components/lightbox/lightbox.tsx` (the modal `Lightbox`; the
|
||||
`LightboxDetails` + `shared.ts` files in the same folder are still used
|
||||
by the public `/media` page and must be kept)
|
||||
- `src/components/navbar/task-queue.tsx`
|
||||
- `src/components/prompt-box/`
|
||||
- `src/components/settings-modal/`
|
||||
|
||||
### Lib files only used by the above
|
||||
|
||||
- `src/lib/cost-estimate-api.ts`
|
||||
- `src/lib/enter-to-generate-store.ts`
|
||||
- `src/lib/omni-gen-hooks.ts`
|
||||
- `src/lib/prompts-cache.ts`
|
||||
- `src/lib/recreate.ts`
|
||||
|
||||
### Files with conditional flag branches (need cleanup, not deletion)
|
||||
|
||||
- `src/app/app.tsx` — drop the `appOrWebapp()` helper, drop the local route
|
||||
imports, leave only the `<WebappRedirect />` routes
|
||||
- `src/components/navbar/navbar.tsx` — drop the `!USE_WEBAPP_FOR_APP_FEATURES`
|
||||
branches (Login + Sign up buttons in desktop and mobile); keep only Launch
|
||||
App for logged-in users and the Pricing + Launch App combo for logged-out
|
||||
users
|
||||
- `src/components/footer/footer.tsx` — drop the conditional Image/Video
|
||||
links in the Pages column
|
||||
- `src/components/download-modal.tsx` — drop the inline `SignupForm` flow
|
||||
and the `view`/`navigate` state; keep only the webapp `/signup` and
|
||||
`/login` link buttons
|
||||
- `src/pages/landing3/landing3.tsx` — `appLink("/create-image")` /
|
||||
`appLink("/create-video")` can become bare `webappUrl("…")` calls (or
|
||||
inline `${WEBAPP_URL}create-image`)
|
||||
- `src/pages/media/media.tsx` — drop the local
|
||||
`applyRecreateFromMediaToken` branch and the `useNavigate` import; keep
|
||||
only the `window.location.href = …` webapp redirect
|
||||
- `src/pages/pricing/pricing.tsx` — the "Buy Credits" section + `CreditsModal`
|
||||
is currently shown unconditionally; once `credits-modal.tsx` is deleted,
|
||||
remove the section, the `creditsModalOpen` state, the `isLoggedIn` state,
|
||||
and the session-check `useEffect`
|
||||
|
||||
### The flag itself
|
||||
|
||||
`src/config/links.ts` — drop `USE_WEBAPP_FOR_APP_FEATURES`, drop the
|
||||
`appLink()` helper (use `webappUrl()` directly everywhere). Keep
|
||||
`WEBAPP_URL`, `webappUrl()`, `SOCIAL_LINKS`, `SUPPORT_EMAIL`, and the dev
|
||||
override.
|
||||
|
||||
---
|
||||
|
||||
## Cleanup pass — what to do when we're ready to commit to the webapp split
|
||||
|
||||
1. **Confirm the flag has been `true` in production for long enough** that
|
||||
we're sure no real users hit the local routes.
|
||||
2. **Delete** every directory and file listed under "Pages that exist
|
||||
locally only as a fallback", "Components only used by the above", and
|
||||
"Lib files only used by the above".
|
||||
3. **Strip the flag branches** from each file under "Files with conditional
|
||||
flag branches".
|
||||
4. **Remove `USE_WEBAPP_FOR_APP_FEATURES`** and `appLink()` from
|
||||
`src/config/links.ts`.
|
||||
5. **Run** `npx tsc -p tsconfig.app.json --noEmit` from
|
||||
`frontend/apps/artcraft-website` and fix any references the cleanup
|
||||
exposed. (Pre-existing errors in `download-button.tsx` `LINUX` and
|
||||
`pricing-table.tsx` `maybeReferralUsername` are unrelated and can be
|
||||
ignored.)
|
||||
6. **Verify** in dev with `nx serve @frontend/artcraft-website` that every
|
||||
navbar/footer/landing/media flow still works and that legacy URLs still
|
||||
redirect to the webapp via `WebappRedirect`.
|
||||
|
||||
---
|
||||
|
||||
## What stays on the marketing site no matter what
|
||||
|
||||
These are pure marketing content and are **not** behind the flag:
|
||||
|
||||
- `/` → `landing3`
|
||||
- `/landing3`
|
||||
- `/seedance-2` → `landing-sd2`
|
||||
- `/download`
|
||||
- `/media`, `/media/:id` (public viewer; the lightbox's "Recreate" button
|
||||
does cross-origin to the webapp, but the viewer itself stays here)
|
||||
- `/press-kit`
|
||||
- `/tutorials`, `/tutorials/:slug`
|
||||
- `/faq`, `/faq/:slug`
|
||||
- `/news`, `/news/:slug`
|
||||
- `/support`
|
||||
- `/pricing` — Stripe checkout flow stays here (PricingTable → Stripe →
|
||||
`/checkout/success` → WebappRedirect → webapp)
|
||||
@@ -0,0 +1,157 @@
|
||||
/background-change /index.html 200
|
||||
/background-change/ /index.html 200
|
||||
/background-change/* /index.html 200
|
||||
|
||||
/checkout_cancel /index.html 200
|
||||
/checkout_cancel/ /index.html 200
|
||||
/checkout_cancel/* /index.html 200
|
||||
|
||||
/onboarding /index.html 200
|
||||
/onboarding/ /index.html 200
|
||||
/onboarding/* /index.html 200
|
||||
|
||||
/portal_closed /index.html 200
|
||||
/portal_closed/ /index.html 200
|
||||
/portal_closed/* /index.html 200
|
||||
|
||||
/checkout_success /index.html 200
|
||||
/checkout_success/ /index.html 200
|
||||
/checkout_success/* /index.html 200
|
||||
|
||||
/checkout* /index.html 200
|
||||
/checkout/* /index.html 200
|
||||
|
||||
/create-image /index.html 200
|
||||
/create-image/ /index.html 200
|
||||
/create-image/* /index.html 200
|
||||
|
||||
/create-video /index.html 200
|
||||
/create-video/ /index.html 200
|
||||
/create-video/* /index.html 200
|
||||
|
||||
/support /index.html 200
|
||||
/support/ /index.html 200
|
||||
/support/* /index.html 200
|
||||
|
||||
/library /index.html 200
|
||||
/library/ /index.html 200
|
||||
/library/* /index.html 200
|
||||
|
||||
/download /index.html 200
|
||||
/download/ /index.html 200
|
||||
|
||||
/tutorials /index.html 200
|
||||
/tutorials/ /index.html 200
|
||||
|
||||
/login /index.html 200
|
||||
/login/ /index.html 200
|
||||
/login/* /index.html 200
|
||||
|
||||
/media /index.html 200
|
||||
/media/ /index.html 200
|
||||
/media/* /index.html 200
|
||||
|
||||
# Prerendered: serve the per-route index.html so social crawlers get the
|
||||
# correct OG/title tags (see scripts/prerender-meta.mjs). The SPA still boots
|
||||
# from this file and React Router takes over for users.
|
||||
/pricing /pricing/index.html 200
|
||||
/pricing/ /pricing/index.html 200
|
||||
/pricing/* /index.html 200
|
||||
|
||||
/beta /beta/index.html 200
|
||||
/beta/ /beta/index.html 200
|
||||
/beta/* /index.html 200
|
||||
|
||||
/faq /index.html 200
|
||||
/faq/ /index.html 200
|
||||
/faq/* /index.html 200
|
||||
|
||||
/news /index.html 200
|
||||
/news/ /index.html 200
|
||||
/news/* /index.html 200
|
||||
|
||||
/press-kit /index.html 200
|
||||
/press-kit/ /index.html 200
|
||||
/press-kit/* /index.html 200
|
||||
|
||||
/referrals /index.html 200
|
||||
/referrals/ /index.html 200
|
||||
/referrals/* /index.html 200
|
||||
|
||||
# Redirects for growth measurement
|
||||
/reddit /?utm_source=reddit 301
|
||||
|
||||
/news.json /index.html 200
|
||||
|
||||
/signup /index.html 200
|
||||
/signup/ /index.html 200
|
||||
/signup/* /index.html 200
|
||||
|
||||
/forgot-password /index.html 200
|
||||
/forgot-password/ /index.html 200
|
||||
/forgot-password/* /index.html 200
|
||||
|
||||
/welcome /index.html 200
|
||||
/welcome/ /index.html 200
|
||||
/welcome/* /index.html 200
|
||||
|
||||
/password-reset /index.html 200
|
||||
/password-reset/ /index.html 200
|
||||
/password-reset/* /index.html 200
|
||||
|
||||
# Prerendered: exact paths serve their per-route index.html (correct OG tags
|
||||
# for social crawlers); deep sub-paths fall back to the SPA shell. See
|
||||
# scripts/prerender-meta.mjs. NOTE: these exact-match serving rules MUST come
|
||||
# before the /seedance2* alias below, otherwise that wildcard would 301
|
||||
# /seedance2-5 away to /seedance-2 (first match wins in _redirects).
|
||||
/seedance2-5 /seedance2-5/index.html 200
|
||||
/seedance2-5/ /seedance2-5/index.html 200
|
||||
/seedance2-5/* /index.html 200
|
||||
|
||||
/seedance-2 /seedance-2/index.html 200
|
||||
/seedance-2/ /seedance-2/index.html 200
|
||||
/seedance-2/* /index.html 200
|
||||
|
||||
/minimax-h3 /minimax-h3/index.html 200
|
||||
/minimax-h3/ /minimax-h3/index.html 200
|
||||
/minimax-h3/* /index.html 200
|
||||
|
||||
/creators/jboogxcreative /creators/jboogxcreative/index.html 200
|
||||
/creators/jboogxcreative/ /creators/jboogxcreative/index.html 200
|
||||
/creators/* /index.html 200
|
||||
|
||||
# Similar URLs — creator spotlight aliases
|
||||
/jboogx /creators/jboogxcreative 301
|
||||
/jboogx/ /creators/jboogxcreative 301
|
||||
/jboogxcreative /creators/jboogxcreative 301
|
||||
/jboogxcreative/ /creators/jboogxcreative 301
|
||||
|
||||
# Similar URLs: MiniMax H3 aliases
|
||||
/minimax /minimax-h3 301
|
||||
/minimax/ /minimax-h3 301
|
||||
/minimaxh3 /minimax-h3 301
|
||||
/minimaxh3* /minimax-h3 301
|
||||
/minimax-h-3 /minimax-h3 301
|
||||
/minimax-h-3* /minimax-h3 301
|
||||
/hailuo /minimax-h3 301
|
||||
/hailuo/ /minimax-h3 301
|
||||
|
||||
# Google ads breakage
|
||||
/seedance-2{ignore} /seedance-2 301
|
||||
/seedance-2{ignore}* /seedance-2 301
|
||||
/seedance-2%7Bignore%7D /seedance-2 301
|
||||
/seedance-2%7Bignore%7D* /seedance-2 301
|
||||
|
||||
# Similar URLs — Seedance 2.5 aliases (kept above the broad /seedance2* rule)
|
||||
/seedance-2-5 /seedance2-5 301
|
||||
/seedance-2-5* /seedance2-5 301
|
||||
/seedance-2.5 /seedance2-5 301
|
||||
/seedance-2.5* /seedance2-5 301
|
||||
/seedance25 /seedance2-5 301
|
||||
/seedance25* /seedance2-5 301
|
||||
|
||||
# Similar URLs — Seedance 2.0 aliases
|
||||
/seedance2 /seedance-2 301
|
||||
/seedance2* /seedance-2 301
|
||||
/seedance-2.0 /seedance-2 301
|
||||
/seedance-2.0* /seedance-2 301
|
||||
@@ -0,0 +1,52 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<!-- Google tag (gtag.js) -->
|
||||
<script>
|
||||
window.dataLayer = window.dataLayer || [];
|
||||
function gtag() {
|
||||
dataLayer.push(arguments);
|
||||
}
|
||||
if (window.location.pathname === "/login/desktop") {
|
||||
const policy = document.createElement("meta");
|
||||
policy.name = "referrer";
|
||||
policy.content = "no-referrer";
|
||||
document.head.appendChild(policy);
|
||||
} else {
|
||||
const analytics = document.createElement("script");
|
||||
analytics.async = true;
|
||||
analytics.src = "https://www.googletagmanager.com/gtag/js?id=G-VKW89949V0";
|
||||
document.head.appendChild(analytics);
|
||||
gtag("js", new Date());
|
||||
gtag("config", "G-VKW89949V0");
|
||||
}
|
||||
</script>
|
||||
|
||||
<meta charset="utf-8" />
|
||||
<title>ArtCraft. AI Video and Images. Fast and Open Desktop App.</title>
|
||||
<meta
|
||||
name="description"
|
||||
content="ArtCraft is an Open Desktop app for generating AI Video and Images. You own ArtCraft!"
|
||||
/>
|
||||
<meta
|
||||
name="keywords"
|
||||
content="ArtCraft, AI, AI Video, AI Images, Open Source"
|
||||
/>
|
||||
|
||||
<base href="/" />
|
||||
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<link rel="icon" type="image/x-icon" href="/favicon.ico" />
|
||||
<link rel="stylesheet" href="/src/styles.css" />
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com" />
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
|
||||
<link
|
||||
href="https://fonts.googleapis.com/css2?family=Bricolage+Grotesque:opsz,wght@12..96,200..800&family=Inter:ital,opsz,wght@0,14..32,100..900;1,14..32,100..900&family=Outfit:wght@100..900&family=Source+Sans+3:ital,wght@0,200..900;1,200..900&family=Instrument+Serif:ital@0;1&family=Geist+Mono:wght@300..800&display=swap"
|
||||
rel="stylesheet"
|
||||
/>
|
||||
</head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
<script type="module" src="/src/main.tsx"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,29 @@
|
||||
# Netlify settings used to discover this file:
|
||||
# Base directory: frontend
|
||||
# Package directory: apps/artcraft-website (relative to the base directory)
|
||||
# Manage the build command and publish directory here; leave those UI fields empty.
|
||||
# Paths below are relative to the frontend workspace.
|
||||
|
||||
[build]
|
||||
command = "bash ./apps/artcraft-website/script/netlify_build.sh"
|
||||
publish = "apps/artcraft-website/dist/"
|
||||
|
||||
[[headers]]
|
||||
for = "/build/*"
|
||||
|
||||
[headers.values]
|
||||
"Cache-Control" = "public, max-age=31560000, immutable"
|
||||
|
||||
# [[redirects]]
|
||||
# from = "/ingest/static/*"
|
||||
# to = "https://us-assets.i.posthog.com/static/:splat"
|
||||
# host = "us-assets.i.posthog.com"
|
||||
# status = 200
|
||||
# force = true
|
||||
#
|
||||
# [[redirects]]
|
||||
# from = "/ingest/*"
|
||||
# to = "https://us.i.posthog.com/:splat"
|
||||
# host = "us.i.posthog.com"
|
||||
# status = 200
|
||||
# force = true
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"name": "@frontend/artcraft-website",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"dependencies": {
|
||||
"react-masonry-css": "^1.0.16"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
const { join } = require('path');
|
||||
|
||||
// Note: If you use library-specific PostCSS/Tailwind configuration then you should remove the `postcssConfig` build
|
||||
// option from your application's configuration (i.e. project.json).
|
||||
//
|
||||
// See: https://nx.dev/guides/using-tailwind-css-in-react#step-4:-applying-configuration-to-libraries
|
||||
|
||||
module.exports = {
|
||||
plugins: {
|
||||
tailwindcss: {
|
||||
config: join(__dirname, 'tailwind.config.js'),
|
||||
},
|
||||
autoprefixer: {},
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<svg id="Layer_2" data-name="Layer 2" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 116.34 97.5">
|
||||
<defs>
|
||||
<style>
|
||||
.cls-1 {
|
||||
fill: #4e7bfb;
|
||||
}
|
||||
</style>
|
||||
</defs>
|
||||
<g id="Layer_1-2" data-name="Layer 1">
|
||||
<path class="cls-1" d="M104.28,49.49L81.55,0h-31.23l-3.17,4.76L14.75,53.63,0,75.85l21.55,21.55,63.79-36.94,16.99,37.04,14.01-21.74-12.06-26.27ZM32.89,65.66l32.42-48.87,10.91,23.77-43.32,25.09Z"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 483 B |
|
After Width: | Height: | Size: 38 KiB |
|
After Width: | Height: | Size: 662 KiB |
|
After Width: | Height: | Size: 45 KiB |
|
After Width: | Height: | Size: 199 KiB |
|
After Width: | Height: | Size: 15 KiB |
|
After Width: | Height: | Size: 463 KiB |
|
After Width: | Height: | Size: 181 KiB |
|
After Width: | Height: | Size: 8.8 KiB |
|
After Width: | Height: | Size: 6.2 KiB |
|
After Width: | Height: | Size: 1.8 MiB |