craft-libs/crates/pcraft-format

craft-pcraft-format

The envelope of PhotoCraft's .pcraft bundle: the manifest around the document, tile references, the entry paths, the limits for reading a manifest, and the runner for manifest migrations. The document model stays in PhotoCraft and is a type parameter here. The crate does not read ZIP archives, open files or decode tiles. No unsafe, no panics on malformed input, and it builds for wasm32-unknown-unknown. The dependencies are serde, serde_json and thiserror.

craft-pcraft-format = { git = "https://github.com/storytold/craft-libs", rev = "<commit>" }

Bundle layout

A bundle is a ZIP archive or a directory with these entries:

Entry	Constant or function	Contents
manifest.json	MANIFEST	the versioned manifest
tiles/<blake3>.zst	tile_path(hash)	a 256×256 tile, zstd, little-endian samples
blobs/<blake3>.zst	blob_path(hash)	binary data, zstd
thumb.png	THUMB	optional thumbnail
composite/preview.png	COMPOSITE	optional flattened preview
composite/full.png	FULL_COMPOSITE	optional full-canvas flattened image
save.lock	SAVE_LOCK	directory bundles only: held while a save runs (empty)

<blake3> is the BLAKE3 hash of the uncompressed bytes as 64 lowercase hex digits. is_valid_hash checks that form. tile_path and blob_path return None for any other string, so a hash read from a file never becomes an entry name such as tiles/../x.zst. TILES_DIR (tiles), BLOBS_DIR (blobs) and COMPOSITE_DIR (composite) name the directories, and OBJECT_SUFFIX (.zst) ends every tile and blob entry. EXTENSION is pcraft and FORMAT_VERSION is 1.

Use

use craft_pcraft_format::{FORMAT_VERSION, Limits, Manifest, Step};

const STEPS: &[Step] = &[];

let manifest: Manifest<MyDocument> = craft_pcraft_format::read_manifest(&manifest_json, &Limits::default(), STEPS, FORMAT_VERSION)?;

• read_manifest(json, &Limits, steps, target) is the reader of a manifest. It runs parse_manifest, then migrate, which refuses a version newer than target, then reads the members, the document as D. read_manifest_with takes a function that reads the document from its serde_json::Value instead of D's Deserialize.
• Manifest<D> has format_version, generator, document: D, thumbnail and composite, and serializes them in that order. Absent previews serialize as null. On read, missing previews are None, members the type does not name are skipped, and a thumbnail other than thumb.png or a composite other than composite/preview.png gives Corrupt. Manifest has no Deserialize and is #[non_exhaustive]: Manifest::new(generator, document) builds one at FORMAT_VERSION without previews.
• Manifest::full_composite optionally names composite/full.png. A missing or null member reads as None; any other path gives Corrupt. When absent, serialization omits this member, preserving the bytes of manifests without a full-canvas image. It does not change the format version.
• TileRef has tx, ty and hash, and is #[non_exhaustive]: TileRef::new(tx, ty, hash) builds one. TileRef::path gives its entry, or None when the hash is not valid.
• check_manifest(json, &Limits) checks the size, the nesting and the item count in one pass without recursion. PhotoCraft's own check of a manifest it saves tests the nesting and the item count only. parse_manifest(json, &Limits) runs that check and then parses into a serde_json::Value with serde_json's 128-level recursion limit turned off.
• check_manifest_with_cancel(json, &Limits, &cancelled) applies those same checks while polling a callback between bounded portions of JSON. A true result stops with Error::Cancelled.
• migrate(&mut value, steps, target) upgrades the JSON in place, where steps[i] upgrades version i + 1 to i + 2, and returns the version the file had. Step<E> is fn(&mut Value) -> Result<(), E>; with E: From<Error>, the steps return the caller's own error type.
• Error is Corrupt, Unsupported, TooNew, LimitExceeded, Json or Cancelled. A manifest whose format_version is newer than target gives TooNew, and one with no step to the next version gives Unsupported. A missing or invalid format_version gives Corrupt, and any other missing or mistyped member gives Json. TooNew holds found, supported and the manifest's generator (None when it is missing or not a string), and is matched with ...

Limits

Limits (fields max_manifest_bytes, max_blob_bytes, max_total_bytes, max_depth, max_items) holds the limits PhotoCraft loads with:

• max_manifest_bytes (256 MiB): the size of manifest.json.
• max_blob_bytes (1 GiB): one decompressed blob.
• max_total_bytes (16 GiB): every decompressed tile and blob in one bundle.
• max_depth (367, manifest_depth(MAX_GROUP_DEPTH)): the nesting of JSON arrays and objects. PhotoCraft nests up to MAX_GROUP_DEPTH (100, a copy of PhotoCraft's own constant) layer groups, three levels each, and allows 64 more levels for the document and leaf layers. A value above MAX_DEPTH (400) is treated as 400, so parsing stays within the stack.
• max_items (MAX_MANIFEST_ITEMS, 4,194,304): arrays and objects opened, plus elements and members after a comma.

check_manifest, parse_manifest and read_manifest apply the manifest limits. The crate does not read tiles or blobs, so it does not apply max_blob_bytes or max_total_bytes: the caller that decompresses the entries enforces them. The struct is #[non_exhaustive]: start from the defaults and set the fields you change.

let mut limits = craft_pcraft_format::Limits::default();
limits.max_manifest_bytes = 16 << 20;

Testing

cargo test -p craft-pcraft-format runs:

• tests/manifest.rs: a manifest with and without previews serializes with serde_json::to_vec_pretty to JSON written by hand in the test, byte for byte, and reads back equal; missing previews and unknown members; entry names, hash checks, and no entry name for an invalid hash.
• tests/read.rs: read_manifest and read_manifest_with read the envelope and the document, refuse a newer version with the writer's generator, refuse a manifest past the limits, run the steps first, and report missing or mistyped members as a struct with a derived Deserialize reports them, in the same order; a missing or invalid format_version is Corrupt, and so is a preview that names another entry, such as ../x or an absolute path.
• Doc tests: Manifest and TileRef are not built by struct literal, Manifest has no Deserialize, and Error::TooNew is matched with ...
• tests/limits.rs: the default limits, nesting at and past max_depth, items at and past max_items, brackets inside strings, the manifest size, JSON errors and the messages.
• tests/migrate.rs: steps in order, partial chains, the current version, newer, missing, zero and out-of-range versions, a missing step, and steps that return the caller's error type.

.github/workflows/pcraft-format.yml runs fmt, clippy and the tests on Linux and Windows, and a wasm32-unknown-unknown check.