A non-Unicode document path holding '..' (/a/../\xff/d.doc) was refused,
because the native join kept the '..' while the lexical result folded it.
join_native now folds the folder's components as normalize does, so such a
document resolves as a Unicode one would, and a '..' that leaves a Unix name
holding a backslash is still caught by the comparison.
std::path sees no \??\ or \\.\ prefix and no //server/share boundary on Unix,
so the native root check let those document roots through. Resolve the link
against the lossy document path with the Unicode rules, and return the native
join only when it names the same components.
A relative link next to a document whose path is not Unicode was joined
to the document's folder natively, skipping the parsing that refuses
device and object-manager namespaces. On Windows a document path with an
unpaired surrogate under \\.\, \\?\GLOBALROOT or \??\ therefore let
the link resolve inside that namespace. The native join now refuses those
roots as resolve does for a Unicode path; drives and shares are kept.
The shared suggestion code already edits across the whole lowercased word, so no code change is needed; the test keeps it that way.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Add craft-doc: read PhotoCraft, VectorCraft and EffectCraft documents without the apps
A new crate, craft-doc, reads the native files of three apps from bytes:
PhotoCraft .pcraft (a ZIP bundle), VectorCraft .vectorcraft (JSON, plain
or gzip, with the version 4 data after the JSON) and EffectCraft .ecproj
(JSON).
- sniff tells which app wrote a file from its first bytes.
- info gives the format version, the writer the file names, and whether
the version is one the crate reads.
- renditions returns VectorCraft's preview and PDF and PhotoCraft's
thumbnail and composite, when the file has them.
- outline returns PhotoCraft's canvas and layer tree, VectorCraft's
artboards and top-level layers, and EffectCraft's compositions with
their layers.
A file whose version is newer than the crate reads is refused with
Error::TooNew. Limits bound the JSON, each entry, the total bytes
decompressed and the JSON nesting, and each is checked before the bytes
it covers are allocated. ZIP entries are read by a small STORE and
DEFLATE reader in the crate. The crate depends on serde, serde_json,
thiserror and flate2 and builds for wasm32-unknown-unknown.
The tests build every document they read: JSON written by hand, PNG,
ZIP and gzip bytes assembled in the test. doc.yml runs fmt, clippy, the
tests and a wasm32 check.
* ci: do not keep the checkout token in the doc workflow
The checkout step sets persist-credentials: false, as jpeg.yml and media.yml do. No later step in the job uses the token.
* craft-doc: read each format's envelope through its format crate
craft-doc no longer keeps its own copies of the envelope types and
constants. It reads them with craft-pcraft-format (`MANIFEST`,
`FORMAT_VERSION`, `THUMB`, `COMPOSITE`, `check_manifest`),
craft-vectorcraft-format (`Head`, `Image`, `sniff`, `check`, `VERSION`,
`BLOB_MAGIC`, base64) and craft-ecproj-format (`Head::read`,
`SCHEMA_VERSION`). Its own base64 module is removed. A `.pcraft`
manifest is still read in one pass without building a
`serde_json::Value`, with the checks `read_manifest` applies. The
outlines are still read with craft-doc's read-only views of each
document, and the public API (`sniff`, `info`, `renditions`, `outline`,
`Limits`) does not change.
These inputs are now read as the format crates read them:
- A `.pcraft` manifest without `generator`, or with a `thumbnail` other
than `thumb.png` or a `composite` other than `composite/preview.png`,
is malformed for `renditions` and `outline`; `info` still reports its
version. A manifest with more than 4,194,304 JSON items is refused
with `LimitExceeded`.
- A `.vectorcraft` preview or PDF member without `mime` gives no
rendition.
- `sniff` takes `format` only from a top-level member, so
`{"meta":{"format":"vectorcraft"}}` is not a VectorCraft file.
tests/envelopes.rs writes a file of each format with the crates' types
and reads it through every call, and checks the inputs above.
doc.yml also runs when one of the three crates changes.
* craft-doc: sniff reads only top-level members of an EffectCraft project
`sniff` took any JSON object whose first member is `savedBy` for an
EffectCraft project when `"schema"` appeared anywhere in its first
bytes, so `{"savedBy":"1.0","metadata":{"schema":1}}` was claimed. It
now checks the top-level member names only, one at a time, without
keeping them.
* craft-doc: bound list elements with max_items; no rendition without a MIME type
Limits gains max_items (default 1,048,576): EffectCraft project items and
composition layers, VectorCraft artboards and layers, and PhotoCraft layers
at every level are counted before each one is read, and a file that lists
more gives Error::LimitExceeded. A list of empty objects within max_json no
longer expands into gigabytes of layers.
A VectorCraft preview or pdf member with an empty mime gives no rendition, as
one without a mime does; a preview may be PNG or JPEG. An EffectCraft
composition without a duration or a frame rate gets no timing instead of a
duration of 0.
* craft-media: shared codec identifiers, one ByteSource, the testkit
The types a codec crate needs to describe a file and configure a
decoder, ahead of craft-audio:
- `Container`, `CodecId`, `PcmFormat` and `CodecParams` (`id.rs`),
always built, with optional serde.
- `codec_params()` on `isobmff::Track` and `matroska::Track`, both
returning `CodecParams` in the same form; `extradata` follows
Matroska's `CodecPrivate`.
- One `ByteSource` trait (`source.rs`), with `ReadSeekSource`;
`isobmff::ByteSource` and `matroska::ByteSource` are that trait.
- craft-media-testkit (`publish = false`): the ffmpeg oracle helpers
from craft-media's tests, so the codec crates share one copy.
The existing tests prove the refactor; the new types have their own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-media: APV extradata is the bare record; reseek after a poisoned lock
An MP4 apvC track gave CodecParams.extradata with the 4-byte FullBox header
in front, where Matroska's V_APV CodecPrivate is the bare
APVDecoderConfigurationRecord. ApvConfig::record_bytes serializes the record
alone and extradata uses it.
ReadSeekSource kept its tracked position after recovering a poisoned lock or
a failed seek, so a reader moved by a panic was read from the wrong offset
with no error. Both now force a seek on the next read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-media: MP4 codec params per sample description
Track::codec_params always described the first stsd entry, while each sample
selects its own through description_index. It now describes the first
sample's entry; codec_params_for gives the parameters of any entry, and
has_mixed_descriptions says whether one set covers every sample.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-media: test file-backed ByteSources; a File read past i64::MAX is an EOF
A File and a ReadSeekSource over a file now have the buffer's test: the same
bytes at the same offsets, and UnexpectedEof past the end. The test found
that a File read at an offset past i64::MAX failed with the OS's
InvalidInput instead of the UnexpectedEof the trait promises; it now
reports the end of the source like the other implementations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-media: Matroska codec params match MP4 for ALAC, DNx and Opus
Matroska's codec_params gave Unknown for A_ALAC and for DNxHD/DNxHR stored
as V_MS/VFW/FOURCC (AVdn, AVdh), where MP4 gives Alac and Dnxhd; ALAC's
extradata is the CodecPrivate, the ALACSpecificConfig MP4's alac box holds.
Opus now reports 48 kHz from Matroska as from MP4, whatever rate the muxer
declared.
ReadSeekSource clears the poison when it recovers the lock, so one
panicked read costs one recovery instead of a seek on every later read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* craft-media: MP4 reports the MPEG audio layer the first frame names
An MP4 track with MPEG audio reported CodecId::Mp3 whatever the layer, since
mp4a + 0x69/0x6B names none (ffmpeg stores MP2 that way), while Matroska
reports Mp1/Mp2/Mp3 from its CodecID. open() now reads the frame header of
each such entry's first sample and codec_params() reports the layer it
names; without a valid header, QuickTime's .mp2 gives Mp2 and anything else
Mp3 as before. CodecConfig::Mp3 is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A step now counts a rule, a condition class, a split's suffix lookup,
16 flags of a set looked through and a group visit, so hostile rules
whose steps were cheap to count but costly to run hit the budget in
time. Retuned: 2^13 steps per candidate, 2^25 per suggest call; the
81 packs' typo sets suggest exactly as at d0c341c8.
Condition character sets are sorted and looked up by binary search.
A stem's sets are worked out once per distinct entry list when its
entries hold 256+ flags (16 kept-apart homonyms of 8,190 flags on
17,576 stems: 55 s load, now 57 ms). A near-word candidate whose
steps ran out counts as a word, so suggest never offers a compound
check() would refuse.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
Every TRY/KEY/typo candidate got a fresh 65,536-rule check, so a hostile
.aff with a 128-letter TRY line took ~20 s per suggest call. Now a
suggest call shares 2^23 affix rules across its simple-word checks, each
capped at 2^11; prefix-rule collection counts against the budget and
mixed-case lookups try at most MAX_HOMONYMS stems. Suggestion lists are
unchanged on the 77 packs and the recall sets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
* Add craft-pcraft-format: the envelope of PhotoCraft's .pcraft bundle
The manifest around the document, with the document as a type parameter
(Manifest<D>), TileRef, the bundle's entry paths and hash check,
FORMAT_VERSION, the manifest decode limits with the nesting and item
scan, and the migration runner as a generic step list. Messages and
serialized field order are PhotoCraft's, so its saved manifests stay
byte-identical when it switches to these types.
Part of #59.
* ci: do not keep the checkout token in the pcraft-format workflow
The checkout step sets persist-credentials: false, as jpeg.yml and media.yml do. No later step in the job uses the token.
* pcraft-format: build no entry name from an invalid hash
tile_path and blob_path return None unless the hash is 64 lowercase hex
digits, so a hash read from a file such as "../x" never becomes an entry
name. TileRef::path gives a tile's entry the same way.
The docs note that MAX_GROUP_DEPTH is a copy of PhotoCraft's constant,
and that check_manifest also refuses a manifest larger than
max_manifest_bytes, which PhotoCraft's check before a save does not.
* ci: pin actions/checkout by commit and Rust to 1.95 in the pcraft-format workflow
* pcraft-format: read_manifest is the one reader of a manifest
read_manifest(json, limits, steps, target) checks the JSON against the
limits, refuses a format_version newer than target, runs the migration
steps, and reads the document as D and the other envelope members.
read_manifest_with reads the document with the caller's function
instead. Manifest no longer implements Deserialize.
Error::TooNew holds the manifest's generator, so a message can name the
writer; its message is unchanged. TooNew, Manifest and TileRef are
#[non_exhaustive], and Manifest::new and TileRef::new build them.
TILES_DIR, BLOBS_DIR, COMPOSITE_DIR and OBJECT_SUFFIX name the bundle's
directories and the object suffix. The docs of max_blob_bytes and
max_total_bytes say the caller enforces them.
* pcraft-format: read_manifest reports errors in serde's struct order
read_manifest reads the envelope members and the document in the order
the parsed object holds them, and reports a missing member after the
members that are present. A manifest with several faults gives the same
error as a derived Deserialize of the whole manifest, as PhotoCraft
reads it today.
* pcraft-format: document that a missing or invalid format_version is Corrupt
read_manifest's docs and the README say that a missing or invalid
format_version gives Error::Corrupt from migrate, and that a missing
generator or document, or any other mistyped member, gives Error::Json.
A test covers a string and a fractional format_version.
* pcraft-format: read_manifest refuses preview names other than its own
A manifest's `thumbnail` and `composite` named any entry the reader then
opened, while tile entries go through `is_valid_hash`. `read_manifest`
now gives `Error::Corrupt` for a `thumbnail` other than `thumb.png` or a
`composite` other than `composite/preview.png`, the only names PhotoCraft
writes. The check runs after the members are read, so JSON errors keep
their order.
The reader's doc now says a missing or invalid `format_version` is
`Error::Corrupt`, from `migrate`, not `Error::Json`.
- simple_word sorts the grouped prefix flags (non-cross then cross flags were unsorted, so
`redo`/`rework` from a no-cross prefix beside cross ones were missed).
- One budget of 65,536 affix rules per simple-word check, shared by every prefix group and
suffix walk; past it the word is taken for no word. Prefixes that each strip their own
text (h2.aff: 1,000 roots) no longer cost a full walk each: 1.7 s → 1.1 ms per two
words, suggest 14.9 s → 65 ms. The 77 fetched spelling packs answer as without a budget
on 6,000 generated words each.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
- simple_word groups the prefix rules that leave the same root (circumfix and NEEDAFFIX
alike) and answers them in one stem lookup, one chain walk and one suffix walk with all
their flags (FlagProbe): 1,000 or 16,000 prefixes adding nothing no longer cost one walk
each (h1.aff: 1.6 s → 17 ms per two words; 16k prefixes 22 s → 19 ms).
- Deferred suggestion candidates are not looked up again before their compound check.
- A word in capitals matches a mixed-case stem as a simple word only (`JQUERYKABEL`,
`MSCFOO` wrong, as in the CLI).
- to_bytes writes the flags only the kept-apart homonyms use (NEEDAFFIX beside a compound
flag survives a round trip).
- MAGIC WCAFF2: compact data from before homonyms were kept apart is refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
- A prefix with two chained suffixes (hu `meg`+`épít`+`és`+`e`, eo `enjugo`): every rule
cross, the prefix flag from the stem or a continuation class (CLI answers).
- Homonyms with different flags are kept apart (and written apart in the compact form):
a prefix and a suffix, or a compound flag and a suffix, must come from one homonym
(nl `afstopje`, `schuurpoederden` wrong in the CLI). nl now 100% in both comparisons.
- Hungarian syllable rule leaves out a last suffix with no continuation classes;
COMPOUNDROOT counts on the stem only.
- Near-word check: a word listed as an ONLYINCOMPOUND/NEEDAFFIX stem counts only with a
suffix, not a prefix (de `Fahrradverkehr` right).
- A word in capitals may be a mixed-case stem (`KB`/`KBS` from `kB/S`, hu `DNY`, nl `BSC`).
- Mixed-case retry keeps KEEPCASE for the first part; KEEPCASE kept apart from the other
exclusions (NOSUGGEST sharing its flag is no longer cleared); overflow of the homonym
groups counts as dropped.
- Suggestions: candidates are checked as simple words first; the rest get the compound
check cheapest first, those changed where the word stops being parts before others, in
one step budget per call, with piece analyses shared between candidates. The suggestions'
stem scan makes forms in one shared budget (mn).
- affix_check example: `suggest` command; AffixDict::suggest.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
