97c83dc9e8 craft-jpeg: decode sequential 8-bit JPEG a band of rows at a time (#743)
* craft-jpeg: decode sequential 8-bit JPEG a band of rows at a time

Bands reads the headers of a sequential Huffman-coded (SOF0, SOF1) 8-bit file whose first scan
holds every component, and a Reader decodes it one line of MCUs at a time: Huffman decoding,
dequantization, inverse transform, upsampling and colour conversion, holding at most three MCU
lines of samples. Nothing the size of the image is allocated, so a frame up to the format's
65,535 px sides can be decoded straight into the caller's storage.

The samples equal decode()'s for the same rows: both go through the same block transform
(idct_block, factored out of reconstruct), the same upsampling maps and the same colour
conversion (to_color, to_u8). A component subsampled down reads the MCU line above and below
its band, as the whole-image upsampling does.

A band that begins a restart interval is a seek point once find_seek_points has found every
restart marker, in sequence; readers on other threads can decode from there. Damaged data
leaves the rest of its restart interval at zero and decoding resumes at the next marker, so the
result is the same however the bands were read. Bits::restart now also clears the exhausted
flag: decode() never restarts after running out of data, the band decoder does.

parse_frame is split into parse_header (validation and geometry) and the whole-image
allocation; parse_scan takes the component identifiers. decode() is unchanged.

Tests: bands against decode() sample for sample over jpeg-encoder streams (gray, YCbCr, CMYK,
YCCK; 4:4:4, 4:2:2, 4:2:0, 4:4:0; four restart intervals) and a test-only encoder's interleaved
layouts with factors of 3 and 4, in order and by seeking; zune-jpeg within the existing
tolerance; damage with and without restarts; limits; unsupported files; the libjpeg test
images; the mutation run with a seek-equals-in-order check; and a `bands` fuzz target.

Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is>

* craft-jpeg: faster inverse transform, rounding, Huffman codes and band rows

Each change keeps every sample as it was: decode() and the bands give bit-identical output
before and after on the test images, 12-bit and arithmetic-coded files, 48 generated streams
and a 48,000 × 34,200 file decoded in bands.

- idct_block works on rows of eight: each sum keeps its order (over u, then over v), eight
  values of x at a time, and rows of zero coefficients are skipped (a zero term changes only the
  sign of a zero, which the level shift removes). The transposed basis is built once.
- to_u8 rounds half away from zero as truncation plus one from .5 up, exact on 0..=255, instead
  of the `round` library call targets without SSE4.1 make.
- Huffman::decode resolves codes of up to 9 bits from a table of the next 9 bits, built by
  running the bit-by-bit procedure on each, so both give the same symbol.
- The band rows are colour converted in loops written out for gray, YCbCr and RGB; YCbCr to
  RGB is one helper shared with the whole-image decode.

On the 48,000 × 34,200 4:4:4 file with a restart interval per line of MCUs (Ryzen 7 7800X3D,
idle), one thread decodes a band of 8 rows in about 3.0 ms (5.8 ms before), and 16 threads
decode the whole image in 2.05 s (3.8 s before).

Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is>

* craft-jpeg: bands report damage, never fail on it, and see data left before a restart

Two cases a mutation run through PhotoCraft's import turned up.

- Data that fails in the first MCU made a band read fail, as decode() refuses a stream with no
  image data. The baseline decoder PhotoCraft uses for other files opens such files (grey, with
  restarts the rest of the image), so a band read now marks it damaged and goes on: with
  restart intervals everything after the first decodes. Refusing a file with nothing
  decodable is the caller's, from the size of its scan data.
- An interval that decodes without an invalid code but leaves whole bytes before its restart
  marker was decoded from cut, shifted or added data. It now counts as damage (libjpeg warns
  "extraneous bytes before marker"). The check runs after an interval's last MCU, so readers
  that seek and readers that read in order agree.

Over 1,074 sequential 8-bit JPEGs on one machine the damage flag agrees with djpeg's warnings on
every file, reading from seek points equals reading in order on all 151 with restart intervals,
and the samples are within mean 0.57, max 8 of zune-jpeg's.

Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is>

* craft-jpeg: seek points follow the restart markers in the data, as reading in order does

find_seek_points gave up on a file whose markers were not all there in sequence, so a damaged
or cut-short file with a restart interval at every line of MCUs decoded in order on one core
(about 13 s for an undamaged 48,000 × 34,200 file on an idle 8-core machine, against 2 s on
16 threads). With these seek points the same file with one marker renumbered, cut at 60%, or
its first interval overwritten decodes in 2.8 to 4.4 s on 16 threads. A reader that reads in
order pairs the k-th restart marker in the data with interval k, whatever its number, and
resumes after each, so seek points built the same way give the same samples for any data:
missing, misnumbered or added markers only change which bands are seek points.

Tested: misnumbered, missing (file cut), added and removed markers read in order and by
seeking; the mutation run's seek-equals-in-order check (more of its cases now seek); 600,000
runs of the bands fuzz target, which asserts the same; 169 local files with restarts.

Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is>

* craft-jpeg: Bands::halo, the MCU lines a reader decodes around its bands

A reader that seeks decodes the MCU line above and below its bands when a component is
subsampled down. A caller splitting the work across threads needs to know it: reading one band
from each seek point of a 4:2:0 file decodes three lines per band (4.97 s for a 48,000 × 34,200
file on 16 threads, against 3.09 s reading four bands from each), while for 4:4:4 the run length
makes no difference. Tested against the sampling factors of every interleaved layout.

Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is>

* craft-jpeg: band decoder review fixes

- Data left over after the last MCU, before EOI, is damage, as it already
  was before a restart marker.
- A scan that names a component twice is Malformed (T.81 B.2.3), not
  Unsupported.
- The upsampling maps are sized from the header and checked against
  max_alloc before they are made.
- The bands fuzz target and the mutation test fail on a read or seek error
  once Bands::new has accepted a stream; damage never surfaces as an error.
- Known-sample CMYK and YCCK fixtures check the ink polarity directly; the
  bands-equal-decode tests cannot, since both share the colour conversion.
- Docs: a single-component frame's bands are 8 rows; Reader::damaged and
  the scan's size cannot tell a failure at the first MCU from later damage;
  with a halo a seek decodes from the previous seek point; each fuzz target
  writes to its own corpus; doc comments on the remaining new functions.

Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is>

* craft-jpeg: test YCCK chroma and trailing-data samples

- The ink fixture gains a YCCK case with Cb and Cr away from neutral, with
  the T.871 values worked out by hand: a conversion that drops or swaps
  the chroma now fails it.
- Bytes added before EOI leave the samples as the clean stream's; the test
  now checks that, not only that reading in order and seeking agree.

Co-Authored-By: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is>

---------

Co-authored-by: Νύξ (Nyx, AI) 🌑 <nyx@despotak.is>
Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
2026-11-11 07:37:06 -05:00

Craft App Libraries

Scanned by SonarCloud

This repo contains libraries used by the various Craft Apps. It's a Rust monorepo workspace, and the libraries can be included by any Craft Apps that need them.

In the future this may grow to include non-Rust library code.

If any libraries grow beyond Craft App usage, eg. RAW format support, they may be promoted out and into their own libraries for the broader ecosystem to take advantage of.

Layout

A Cargo workspace (edition 2024, resolver 3). Each library is a crate under crates/<name>, named craft-<name>, with version, edition, license and repository inherited from the workspace (license.workspace = true, and so on) and [lints] workspace = true. Register each crate in [workspace.dependencies] in the root Cargo.toml.

Apps depend on a crate by git:

craft-<name> = { git = "https://github.com/storytold/craft-libs", rev = "<commit>" }

Libraries

  • craft-platform: optional native dialog lifecycle, Linux/BSD desktop appearance observation and browser download ownership; application policy stays in the apps.

  • craft-ai-core: the data model and contract for AI in the Craft Apps (connections, models, tools, requests, jobs, consent, history, route spec). Pure, no I/O, wasm-safe. Contract in docs/ai-shared-layer.md.

  • craft-pdf: independently optional PDF parsing, interpretation, rasterization and guarded authoring. Apps retain layout, fonts, editing and export policy; writer-only consumers do not enable the shared reader.

  • craft-atomic: Crash-safe atomic file writing and replacement with same-directory staging, directory fsync, Apple F_FULLFSYNC fallback, transient lock backoff, and fault injection for the Craft suite.

  • craft-cms: pure-Rust ICC v2/v4 colour management: profile parsing and writing, transforms, rendering intents, black point compensation, soft proofing, gamut checks, 3D LUTs and built-in profiles (sRGB, Display P3, synthetic coated CMYK and others).

  • craft-crw: clean-room Canon CRW (CIFF) raw decoder, from LightCraft: the heap container, the entropy coding derived by black-box analysis (frozen code tables), sensor black and white levels and as-shot white balance. The complete derivation record (plan, sources log, scripts, verification) is in crates/crw/clean-room/. Colour stays in each app.

  • craft-dialogs: native file and message dialogs that never block the event loop: queued requests shown at the end of the frame and answered on a later frame, free-standing Open and Folder panels and Save sheets on macOS, case-insensitive filters. What to do with the answer stays in each app.

  • craft-doc: reads PhotoCraft .pcraft, VectorCraft .vectorcraft and EffectCraft .ecproj documents without the apps: the kind and version, the previews and PDF they carry, and their pages and layers. Rendering and editing stay in each app.

  • craft-docpath: decides whether a path or file: URL stored in a document may be opened. Paths stored in documents are followed only when they are local or on the document's own network share; other network paths, device paths and non-file URLs are refused.

  • craft-dxf: DXF tag reader and writer, from CADCraft: ASCII and binary DXF as (group code, value) tags, split into sections and entities, with pre-2007 code pages and version detection, and written back. Bounded on hostile input. Mapping tags to a drawing stays in each app.

  • craft-ecproj-format: the envelope of EffectCraft .ecproj project files: savedBy, the schema version and its check, and the file object with the project and its saved view as type parameters. The project model stays in EffectCraft.

  • craft-font-catalog: the installed fonts on every platform (font folders, fontconfig and Flatpak host folders, CoreText, DirectWrite, the font store), lookup by family, style, localized, legacy, full and PostScript names, family loads shared between threads, and rescans when fonts are installed or removed.

  • craft-font-manager: Craft Font Manager as a library: search and filters over the Google Fonts index, installed fonts by family, the font store interface, and (egui feature) FontsPanel, the whole interface as one panel for the launcher's Fonts tab and the standalone app.

  • craft-font-store: the suite's font store: the pinned Google Fonts and Noto indexes plus indexes the user adds (never curated), exact lookup by family and style or PostScript name, and installs from each source's URLs (jsDelivr, then raw GitHub) checked by size, sha256 and parsing, with the licence, a manifest, and removal. Off until the user turns downloading on; agents can't.

  • craft-fonts-build: embeds the optional craft-fonts faces from an app's build.rs (CRAFT_FONTS_DIR, CRAFT_FONTS_REQUIRED, script filters) as a table of EmbeddedFaces. No dependencies; the app picks the faces.

  • craft-gcode: Grbl-style G-code writer: polylines in millimetres to G0/G1 moves with laser power and feed, for cutting lasers and plotters.

  • craft-heif: optional HEIF/HEIC still-image decoder, from LightCraft: heic-rs reads the container and HEVC pictures, and this crate converts to 16-bit RGB to match libheif (VUI range and matrix, chroma upsampling, imir, ICC on grid tiles). Also orientation, alpha, EXIF/XMP and thumbnails. It never panics, and apps put it behind a non-default feature.

  • craft-i18n: translations over the embedded craft-i18n catalogs: lookup with fallback (pt-br → pt → English), plurals, language names, the system locale and a pseudo locale. The catalogs stay a fully optional build input; apps keep tl! and their language setting.

  • craft-tiff: TIFF and BigTIFF IFD reader and writer, from LightCraft: every field type, the IFD chain, SubIFDs and Exif/GPS/Interop pointer IFDs, maker-note IFDs (Canon, Nikon, Sony, Fujifilm, Olympus, Panasonic, Pentax), strip and tile layout. Bounded and panic-free on hostile input. Pixel decompression stays with the callers.

  • craft-jpeg: clean-room decoder for 12-bit and arithmetic-coded JPEG (T.81 SOF0/SOF1 at 12 bits, SOF9, SOF10), from PhotoCraft, and a frame probe for routing. Metadata stays in each app.

  • craft-keymap: keyboard bindings: chords and their text forms, scopes, conflicts, user overrides stored through craft-settings and import of other applications' shortcut sets. Command catalogs, file readers and the shortcut editor UI stay in each app.

  • craft-audio: audio file I/O and codecs (WAV, AIFF, FLAC, AAC, Opus), streaming and seeking, and packet decoding for any demuxer. See its README for formats and features.

  • craft-media: bit-level I/O (bitstream), clean-room MP4/MOV (isobmff) and Matroska/WebM (matroska) demuxers and muxers, an Ogg demuxer (ogg), from FilmCraft, and an ISO BMFF box reader. Containers only; codecs are in craft-audio, and video codecs will go in a separate crate.

  • craft-menu: an app's menus as a macOS menu bar: the Mac layout (app menu, Window, Help, system-key clashes), a platform backend trait, and key equivalents fed back to egui as the input egui-winit would have made. Commands and their enabled state stay in each app.

  • craft-pcraft-format: the envelope of PhotoCraft's .pcraft bundle: Manifest<D> with the document as a type parameter, TileRef, the entry paths, the manifest decode limits and the migration runner. The document model stays in PhotoCraft.

  • craft-print: PDF submission to CUPS and Windows, printer discovery, duplex/copies/colour and driver preferences. PDF generation stays in each app.

  • craft-proof: proofing per language (spelling, suggestions, grammar, hyphenation, a Hunspell reader, user words, Language packs). The word lists and patterns are craft-i18n's language-data/, embedded at build time from CRAFT_I18N_DIR: English and the carry-over locales by default, the set CRAFT_PROOF_EMBED names otherwise (en for English only); other languages come from installed Language packs.

  • craft-quit: native macOS quit, egui close handling and an optional app menu. Unsaved-document decisions and save prompts stay in each app.

  • craft-settings: config.toml settings storage for every app: platform folders, portable mode and overrides, atomic saves while running, backup, damaged-file recovery and schema migrations. Policy in docs/settings-storage.md.

  • craft-silhouette-studio: bounded, fail-closed reader for Silhouette Studio .studio and .studio3 documents: outlines, line colours and page size in millimetres. Protected designs are refused.

  • craft-svg-paths: SVG geometry reader: shapes, transforms, <use>/<symbol> instances and the CSS cascade as Bézier paths in millimetres, each tagged by the caller. Import only; not the shared SVG model.

  • craft-text: text processing below the composer. Its hyphen module finds hyphenation points, with a bundled US English dictionary and patterns and optional hypher patterns for 47 more languages (one cargo feature each).

  • craft-ui: SVG catalogs, icon buttons, tabs, pane controls, transactional docking, print settings and theme helpers for egui frontends; assets, themes, localization and command handling stay in each app. Requires Rust 1.95 or later.

  • craft-vectorcraft-format: the envelope of VectorCraft's .vectorcraft file: its members, versions, embedded images and the data after the JSON, with the document as a type parameter. The document and its migrations stay in VectorCraft.

  • craft-web-api: the page command API every Craft web build offers its page — one window.<app> shape (request, execute, commands, docInspect, uiInspect, screenshot) over the app's JSON control channel.

  • craft-web-host: the host messages between a Craft app's web build and the page that hosts it (?host=parent or data-host="page"): one contract and one bridge for every app.

  • Shared UI component audit: source inventory, extraction priorities and compatibility boundaries across the Craft suite.

  • Theme infrastructure: eight-app comparison and the small shared widget-state styling boundary.

  • Panel docking: validated panel moves, floating groups, workspace adapters and an optional egui docking surface.

License and credits

craft-libs is dual-licensed under MIT or Apache-2.0, at your option. Copyright (c) 2026 ArtCraft Team and the craft-libs contributors. Required notices are in NOTICE.

Bundled fonts, icons, images and other assets keep their own open licenses; each one is listed with its author, source and license in ATTRIBUTION.md.

S
Description
No description provided
Readme
130 MiB
0 Stars 1 Watchers 0 Forks
Languages
Rust 99.3%
Max 0.3%
Python 0.2%
RenderScript 0.2%