craft-doc: read envelopes as the format crates and apps do

- renditions checks where the last VectorCraft member after the JSON
  ends against max_total before it inflates anything past the marker,
  and keeps only the members' bytes, not the data before them.
- A ZIP archive whose manifest.json has no format_version gives
  Unrecognized instead of Malformed.
- A PhotoCraft manifest that is a JSON array is refused, as
  read_manifest refuses it.
- renditions and outline refuse a PhotoCraft manifest nested past
  PhotoCraft's limit (manifest_depth(MAX_GROUP_DEPTH), 367 levels).
- The TooNew check reads only the format's own version member, so a
  newer file that also has another format's member is TooNew, not
  Malformed.
- A generator that is not a string reads as None. A repeated VectorCraft
  generator, preview or pdf reads as its last value, as File::read
  reads it.
- The VectorCraft marker must follow the JSON's closing }, ] or ",
  through craft_vectorcraft_format::marker_at.
- sniff reads the names in a whole archive's central directory, which
  info reads too; a head too short to reach it still uses the local
  headers.

crates/doc/README.md

@@ -7,8 +7,8 @@ or composite. No `unsafe`, no panics on malformed input, and it builds for
 `wasm32-unknown-unknown`. Each format's envelope is read through its crate in this repository:
 [`craft-pcraft-format`](../pcraft-format/README.md) (the `Manifest` members and checks, and
 `check_manifest`),
-[`craft-vectorcraft-format`](../vectorcraft-format/README.md) (`Head`, `Image`, `sniff`, the
-version and the marker) and [`craft-ecproj-format`](../ecproj-format/README.md) (`Head`, the
+[`craft-vectorcraft-format`](../vectorcraft-format/README.md) (`Image`, `sniff`, the version
+and the marker) and [`craft-ecproj-format`](../ecproj-format/README.md) (`Head`, the
 schema check). The outlines are read with craft-doc's own read-only views of the documents. The
 other dependencies are `serde`, `serde_json`, `thiserror` and `flate2` (its pure-Rust backend).
 
@@ -34,7 +34,8 @@ let outline = craft_doc::outline(&bytes, &limits)?;
 
 - `sniff(head) -> Option<Kind>` reads the first bytes. The first 64 KiB are enough for every file
   the apps write. A re-zipped `.pcraft` whose `manifest.json` entry starts after the head is
-  recognized only when `head` holds the whole file.
+  recognized only when `head` holds the whole file. When `head` holds a whole ZIP archive, the
+  names in its central directory decide.
 - `info(bytes, &Limits) -> Result<Info, Error>`: the kind, the version the file declares, the
   writer it names, and `supported`, which is `false` when the version is newer than the crate
   reads.
@@ -42,7 +43,8 @@ let outline = craft_doc::outline(&bytes, &limits)?;
   (`Pdf`, `Thumbnail`, `Composite`), a `mime` and the `bytes`.
 - `outline(bytes, &Limits) -> Result<Outline, Error>`: `canvas`, `pages` and `layers`.
 - `Error` is `Unrecognized`, `Malformed`, `TooNew`, `Unsupported` or `LimitExceeded`. A file
-  whose version is newer than the crate reads gives `TooNew` from `renditions` and `outline`.
+  whose version is newer than the crate reads gives `TooNew` from `renditions` and `outline`. A
+  ZIP archive whose `manifest.json` has no `format_version` gives `Unrecognized`.
 
 ## What each format gives
 
@@ -75,6 +77,8 @@ A `.pcraft` bundle saved as a folder, ZIP64 archives and encrypted entries are n
   returns as a rendition, together.
 - `max_depth` (400, also the largest value it takes, `MAX_DEPTH`): the nesting of JSON arrays and
   objects, checked before parsing. PhotoCraft nests up to 100 groups of layers, three levels each.
+  `renditions` and `outline` refuse a PhotoCraft manifest nested deeper than 367 levels
+  (`craft_pcraft_format::manifest_depth(MAX_GROUP_DEPTH)`), as PhotoCraft does.
 
 A PhotoCraft manifest with more than 4,194,304 JSON items
 (`craft_pcraft_format::MAX_MANIFEST_ITEMS`) is refused with `Error::LimitExceeded`, as PhotoCraft
@@ -82,7 +86,8 @@ reads it.
 
 On a compressed `.vectorcraft` file, `info` and `outline` decompress only as far as the end of
 the JSON. `renditions` checks the size of each member after the JSON against `max_entry`
-before it decompresses any of that data, then decompresses only as far as the members end. The
+before it decompresses any of that data, and the offset where the last member ends against
+`max_total`. It then decompresses only as far as the members end, and keeps only their bytes. The
 struct is `#[non_exhaustive]`: start from the defaults and set the fields you
 change.
 
@@ -105,9 +110,11 @@ limits.max_entry = 32 << 20;
   size, members longer than `max_entry`, JSON nested past `max_depth`, ZIP64, encryption, other
   compression methods and CRC mismatches.
 - `tests/envelopes.rs`: files whose envelope is written with the format crates' types read
-  through each call, and envelopes those crates refuse (a manifest without `generator`, a preview
-  that names another entry, a `format` member inside another member) are refused. A preview or
-  PDF member without `mime` gives no rendition.
+  through each call, and envelopes those crates refuse (a manifest without `generator`, a
+  manifest that is a JSON array, a preview that names another entry, a `format` member inside
+  another member) are refused. A preview or PDF member without `mime` gives no rendition, and a
+  VectorCraft `generator`, `preview` or `pdf` reads as `File::read` reads it: as `None` when it
+  has the wrong shape, and as its last value when it is repeated.
 
 `.github/workflows/doc.yml` runs fmt, clippy and the tests on Linux and Windows, and a
 `wasm32-unknown-unknown` check.

crates/doc/src/lib.rs

@@ -91,7 +91,9 @@ pub struct Info {
 #[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
 #[non_exhaustive]
 pub enum Error {
-    /// [`sniff`] does not recognize the bytes as a Craft document.
+    /// [`sniff`] does not recognize the bytes as a Craft document, or a ZIP archive's
+    /// `manifest.json` is a JSON object without `format_version`, so the archive is not a
+    /// PhotoCraft bundle.
     #[error("not a PhotoCraft, VectorCraft or EffectCraft document")]
     Unrecognized,
     /// The file is damaged or does not follow its format.
@@ -129,7 +131,9 @@ pub struct Limits {
     /// renditions, together. Default 1 GiB.
     pub max_total: u64,
     /// Deepest nesting of JSON arrays and objects, at most [`MAX_DEPTH`] (a larger value counts
-    /// as [`MAX_DEPTH`]). Default [`MAX_DEPTH`].
+    /// as [`MAX_DEPTH`]). Default [`MAX_DEPTH`]. [`renditions`] and [`outline`] also refuse a
+    /// PhotoCraft manifest nested deeper than PhotoCraft reads, 367 levels
+    /// (`craft_pcraft_format::manifest_depth(MAX_GROUP_DEPTH)`).
     pub max_depth: u32,
     /// Most list elements one call reads, together: EffectCraft project items and composition
     /// layers, VectorCraft artboards and layers, and PhotoCraft layers at every level. Each is
@@ -487,27 +491,82 @@ fn parse_checked<T: DeserializeOwned>(json: &[u8]) -> Result<T, serde_json::Erro
     Ok(v)
 }
 
-/// A document's version number: VectorCraft's `version`, PhotoCraft's `format_version` or
-/// EffectCraft's `schema`.
-#[derive(serde::Deserialize)]
-struct Version {
-    #[serde(alias = "format_version", alias = "schema")]
-    version: u32,
+/// The top-level member that holds a document's version number: VectorCraft's `version`,
+/// PhotoCraft's `format_version` or EffectCraft's `schema`.
+fn version_member(kind: Kind) -> &'static str {
+    match kind {
+        Kind::PhotoCraft => "format_version",
+        Kind::VectorCraft => "version",
+        Kind::EffectCraft => "schema",
+    }
+}
+
+/// The version number in the member a [`Visitor`] is given, read from the top-level JSON object;
+/// the other members are skipped, and a repeated one reads as its last value.
+struct VersionIn(&'static str);
+
+impl<'de> Visitor<'de> for VersionIn {
+    type Value = Option<u32>;
+
+    fn expecting(&self, f: &mut fmt::Formatter) -> fmt::Result {
+        f.write_str("a JSON object")
+    }
+
+    fn visit_map<A: MapAccess<'de>>(self, mut map: A) -> Result<Option<u32>, A::Error> {
+        let mut version = None;
+        while let Some(key) = map.next_key::<String>()? {
+            if key == self.0 {
+                version = Some(map.next_value()?);
+            } else {
+                map.next_value::<de::IgnoredAny>()?;
+            }
+        }
+        Ok(version)
+    }
 }
 
 /// Parse `T` from `json`; when that fails, report [`Error::TooNew`] instead if the file's version
-/// (its top-level `version`, `format_version` or `schema`) is newer than `kind` supports, since a
+/// (its top-level member that [`version_member`] names) is newer than `kind` supports, since a
 /// newer format may change what `T` expects.
 pub(crate) fn parse_versioned<T: DeserializeOwned>(json: &[u8], limits: &Limits, kind: Kind) -> Result<T, Error> {
     check_depth(json, limits.max_depth)?;
     counting(limits.max_items, || {
-        parse_checked(json).map_err(|e| match parse_checked::<Version>(json) {
-            Ok(v) if v.version > kind.supported_version() => too_new(kind, v.version),
-            _ => malformed(e),
+        parse_checked(json).map_err(|e| {
+            let mut de = serde_json::Deserializer::from_slice(json);
+            de.disable_recursion_limit();
+            let version = (&mut de).deserialize_map(VersionIn(version_member(kind))).and_then(|v| de.end().map(|()| v));
+            match version {
+                Ok(Some(version)) if version > kind.supported_version() => too_new(kind, version),
+                _ => malformed(e),
+            }
         })
     })
 }
 
+/// A member read as `None` when its value has the wrong shape, as the format crates read a
+/// file's optional members.
+pub(crate) struct Lenient<T>(pub(crate) Option<T>);
+
+impl<'de, T: Deserialize<'de>> Deserialize<'de> for Lenient<T> {
+    fn deserialize<D: Deserializer<'de>>(d: D) -> Result<Self, D::Error> {
+        #[derive(Deserialize)]
+        #[serde(untagged)]
+        enum Shape<T> {
+            Value(T),
+            Other(de::IgnoredAny),
+        }
+        Ok(Lenient(match Shape::deserialize(d)? {
+            Shape::Value(v) => Some(v),
+            Shape::Other(_) => None,
+        }))
+    }
+}
+
+/// [`Lenient`] as a `deserialize_with` function.
+pub(crate) fn lenient<'de, D: Deserializer<'de>, T: Deserialize<'de>>(d: D) -> Result<Option<T>, D::Error> {
+    Lenient::deserialize(d).map(|l| l.0)
+}
+
 pub(crate) fn too_new(kind: Kind, version: u32) -> Error {
     Error::TooNew { kind, version, supported: kind.supported_version() }
 }

crates/doc/src/pcraft.rs

@@ -3,15 +3,26 @@
 //! (`thumbnail`, `composite`). Only the ZIP form is read; a bundle saved as a folder is not. The
 //! manifest's envelope is [`craft_pcraft_format::Manifest`]'s.
 
-use craft_pcraft_format::{self as format, COMPOSITE, FORMAT_VERSION, MANIFEST, THUMB};
+use std::fmt;
+use std::marker::PhantomData;
+
+use craft_pcraft_format::{self as format, COMPOSITE, FORMAT_VERSION, MANIFEST, MAX_GROUP_DEPTH, THUMB, manifest_depth};
 use serde::Deserialize;
-use serde::de::{DeserializeOwned, IgnoredAny};
+use serde::de::value::MapAccessDeserializer;
+use serde::de::{DeserializeOwned, Deserializer, IgnoredAny, MapAccess, Visitor};
 
 use crate::zip::{Zip, local_names_include};
-use crate::{Budget, Canvas, Error, Kind, Layer, MAX_DEPTH, Outline, Rendition, RenditionKind, counted, malformed, parse, parse_versioned, too_new};
+use crate::{Budget, Canvas, Error, Kind, Layer, Outline, Rendition, RenditionKind, counted, lenient, malformed, parse, too_new};
 
+/// A ZIP archive with a `manifest.json` entry. When `head` holds the whole archive, the names in
+/// its central directory decide, as [`Zip::find`] reads them; otherwise the names in the local
+/// headers that `head` holds.
 pub(crate) fn sniff(head: &[u8]) -> bool {
-    head.starts_with(b"PK\x03\x04") && (local_names_include(head, MANIFEST) || Zip::open(head).is_ok_and(|z| z.find(MANIFEST).is_some()))
+    head.starts_with(b"PK\x03\x04")
+        && match Zip::open(head) {
+            Ok(zip) => zip.find(MANIFEST).is_some(),
+            Err(_) => local_names_include(head, MANIFEST),
+        }
 }
 
 /// The archive and its `manifest.json`.
@@ -23,27 +34,60 @@ fn manifest<'a>(bytes: &'a [u8], budget: &mut Budget) -> Result<(Zip<'a>, Vec<u8
     Ok((zip, json))
 }
 
+/// A `T` read only from a JSON object, as [`craft_pcraft_format::read_manifest`] reads the
+/// manifest. serde's derived `Deserialize` also reads a struct from a JSON array.
+struct Object<T>(T);
+
+impl<'de, T: Deserialize<'de>> Deserialize<'de> for Object<T> {
+    fn deserialize<D: Deserializer<'de>>(d: D) -> Result<Self, D::Error> {
+        struct ObjectVisitor<T>(PhantomData<T>);
+        impl<'de, T: Deserialize<'de>> Visitor<'de> for ObjectVisitor<T> {
+            type Value = Object<T>;
+
+            fn expecting(&self, f: &mut fmt::Formatter) -> fmt::Result {
+                f.write_str("the manifest's JSON object")
+            }
+
+            fn visit_map<A: MapAccess<'de>>(self, map: A) -> Result<Object<T>, A::Error> {
+                T::deserialize(MapAccessDeserializer::new(map)).map(Object)
+            }
+        }
+        d.deserialize_map(ObjectVisitor(PhantomData))
+    }
+}
+
 /// The version and the writer, read without the rest of the manifest, so that a manifest of a
-/// newer format still reports them.
+/// newer format still reports them. A `generator` that is not a string reads as `None`, as
+/// [`craft_pcraft_format::migrate`] reads it.
 #[derive(Deserialize)]
 struct Stamp {
-    format_version: u32,
-    #[serde(default)]
+    /// `None` only when the member is missing; a `null` is not a version.
+    #[serde(default, deserialize_with = "present")]
+    format_version: Option<u32>,
+    #[serde(default, deserialize_with = "lenient")]
     generator: Option<String>,
 }
 
-fn checked(version: u32) -> Result<u32, Error> {
+/// A member that is present, read as `T`.
+fn present<'de, D: Deserializer<'de>, T: Deserialize<'de>>(d: D) -> Result<Option<T>, D::Error> {
+    T::deserialize(d).map(Some)
+}
+
+/// The manifest's version and writer. A manifest without `format_version` is not PhotoCraft's:
+/// [`Error::Unrecognized`]. A version of 0 is refused.
+fn stamp(json: &[u8], budget: &Budget) -> Result<(u32, Option<String>), Error> {
+    let Object(stamp) = parse::<Object<Stamp>>(json, budget.limits)?;
+    let version = stamp.format_version.ok_or(Error::Unrecognized)?;
     if version == 0 {
         return Err(malformed("format_version 0 is invalid"));
     }
-    Ok(version)
+    Ok((version, stamp.generator))
 }
 
-/// The members of [`craft_pcraft_format::Manifest`], with the document as `D`, read without
-/// building the manifest as a `serde_json::Value` first.
+/// The members of [`craft_pcraft_format::Manifest`] after `format_version`, with the document as
+/// `D`, read without building the manifest as a `serde_json::Value` first.
 #[derive(Deserialize)]
 struct Envelope<D> {
-    format_version: u32,
     /// Required, and a string, as in the manifest; [`info`] reports it.
     #[serde(rename = "generator")]
     _generator: String,
@@ -55,22 +99,23 @@ struct Envelope<D> {
 }
 
 /// The manifest's envelope, checked as [`craft_pcraft_format::read_manifest`] checks it: the
-/// size, nesting and item count with [`craft_pcraft_format::check_manifest`], a version of 0 or
-/// newer than [`FORMAT_VERSION`] refused, and a preview that names an entry other than [`THUMB`]
+/// size, nesting and item count with [`craft_pcraft_format::check_manifest`] and PhotoCraft's
+/// own nesting limit, a manifest that is not a JSON object refused, the version read first (0 or
+/// newer than [`FORMAT_VERSION`] refused), and a preview that names an entry other than [`THUMB`]
 /// or [`COMPOSITE`] refused. A member that appears twice is refused.
 fn read<D: DeserializeOwned>(json: &[u8], budget: &Budget) -> Result<Envelope<D>, Error> {
     let mut limits = format::Limits::default();
     limits.max_manifest_bytes = json.len();
-    limits.max_depth = budget.limits.max_depth.min(MAX_DEPTH) as usize;
+    limits.max_depth = (budget.limits.max_depth as usize).min(manifest_depth(MAX_GROUP_DEPTH));
     format::check_manifest(json, &limits).map_err(|e| match e {
         format::Error::LimitExceeded(m) => Error::LimitExceeded(m),
         e => malformed(e),
     })?;
-    let m: Envelope<D> = parse_versioned(json, budget.limits, Kind::PhotoCraft)?;
-    let version = checked(m.format_version)?;
+    let (version, _) = stamp(json, budget)?;
     if version > FORMAT_VERSION {
         return Err(too_new(Kind::PhotoCraft, version));
     }
+    let Object(m) = parse::<Object<Envelope<D>>>(json, budget.limits)?;
     for (member, entry, expected) in [("thumbnail", &m.thumbnail, THUMB), ("composite", &m.composite, COMPOSITE)] {
         if let Some(entry) = entry
             && entry != expected
@@ -83,8 +128,7 @@ fn read<D: DeserializeOwned>(json: &[u8], budget: &Budget) -> Result<Envelope<D>
 
 pub(crate) fn info(bytes: &[u8], budget: &mut Budget) -> Result<(u32, Option<String>), Error> {
     let (_, json) = manifest(bytes, budget)?;
-    let stamp: Stamp = parse(&json, budget.limits)?;
-    Ok((checked(stamp.format_version)?, stamp.generator))
+    stamp(&json, budget)
 }
 
 pub(crate) fn renditions(bytes: &[u8], budget: &mut Budget) -> Result<Vec<Rendition>, Error> {

crates/doc/src/vectorcraft.rs

@@ -11,17 +11,20 @@
 //! Since version 4, large members may follow the JSON after the marker `\n\0VCBLOBS\0`; such a
 //! member gives `"blob": [offset, length]` (from the end of the marker) instead of `data`. In a
 //! compressed file, the marker and the data after it are inside the gzip stream too. Files
-//! written before the rename say `"format": "drawcraft"`. The envelope's members are
-//! [`craft_vectorcraft_format::Head`] and [`craft_vectorcraft_format::Image`].
+//! written before the rename say `"format": "drawcraft"`. The envelope's members are those of
+//! [`craft_vectorcraft_format::File`], read as [`craft_vectorcraft_format::File::read`] reads
+//! them, and [`craft_vectorcraft_format::Image`].
 
 use std::borrow::Cow;
+use std::fmt;
 use std::io::Read;
 
-use craft_vectorcraft_format::{BLOB_MAGIC, Head, Image, is_compressed};
+use craft_vectorcraft_format::{BLOB_MAGIC, Image, entry, is_compressed, marker_at};
 use flate2::read::GzDecoder;
 use serde::Deserialize;
+use serde::de::{self, Deserializer, IgnoredAny, MapAccess, Visitor};
 
-use crate::{Budget, Error, Kind, Layer, Outline, Page, Rendition, RenditionKind, counted, malformed, parse, parse_versioned, too_new};
+use crate::{Budget, Error, Kind, Layer, Lenient, Outline, Page, Rendition, RenditionKind, counted, malformed, parse, parse_versioned, too_new};
 
 /// Bytes decompressed at a time.
 const CHUNK: u64 = 64 << 10;
@@ -31,13 +34,11 @@ pub(crate) fn sniff(head: &[u8]) -> bool {
 }
 
 /// Where the JSON ends and the data after the marker starts, when `text` holds a NUL. JSON holds
-/// no raw NUL and the marker's second byte is one, so the JSON ends right before the first NUL;
-/// a NUL that is not the marker's makes the file malformed.
-fn marker(text: &[u8], nul: usize) -> Result<(usize, usize), Error> {
-    match nul.checked_sub(1) {
-        Some(at) if text.get(at..).is_some_and(|r| r.starts_with(BLOB_MAGIC)) => Ok((at, at + BLOB_MAGIC.len())),
-        _ => Err(malformed("the file holds a NUL byte outside the marker after its JSON")),
-    }
+/// no raw NUL and the marker's second byte is one, so the JSON ends right before the first NUL.
+/// The marker must follow the JSON's closing `}`, `]` or `"`, as
+/// [`craft_vectorcraft_format::marker_at`] requires; any other NUL makes the file malformed.
+fn marker(text: &[u8]) -> Result<(usize, usize), Error> {
+    marker_at(text).map(|at| (at, at + BLOB_MAGIC.len())).ok_or_else(|| malformed("the file holds a NUL byte outside the marker after its JSON"))
 }
 
 fn damaged(e: std::io::Error) -> Error {
@@ -54,32 +55,61 @@ struct Unpacked<'a> {
 enum Tail<'a> {
     /// In an uncompressed file.
     Slice(&'a [u8]),
-    /// Still in the gzip stream: `read` holds what was decompressed past the marker already,
-    /// of which `counted` bytes are counted against the budget.
-    Stream { dec: Box<GzDecoder<&'a [u8]>>, read: Vec<u8>, counted: usize },
+    /// Still in the gzip stream: `read` holds what was decompressed past the marker already, not
+    /// yet counted against the budget.
+    Stream { dec: Box<GzDecoder<&'a [u8]>>, read: Vec<u8> },
 }
 
 impl Tail<'_> {
-    /// The data up to `end` (less where the file ends first), decompressing only that far.
-    fn up_to(&mut self, end: u64, budget: &mut Budget) -> Result<&[u8], Error> {
-        match self {
-            Tail::Slice(s) => Ok(s),
-            Tail::Stream { dec, read, counted } => {
-                let end = usize::try_from(end).map_err(|_| Error::LimitExceeded("the data after the JSON does not fit in memory".into()))?;
-                let had = read.len().min(end);
-                if had > *counted {
-                    budget.take((had - *counted) as u64, "the decompressed document")?;
-                    *counted = had;
-                }
-                if read.len() < end {
-                    let want = ((end - read.len()) as u64).min(budget.left().saturating_add(1));
-                    let n = (&mut **dec).take(want).read_to_end(read).map_err(damaged)?;
-                    budget.take(n as u64, "the decompressed document")?;
-                    *counted += n;
-                }
-                Ok(read.as_slice())
+    /// The bytes each `[offset, length]` in `ranges` names, or `None` for a range the file ends
+    /// before. A compressed file is decompressed only as far as the last range ends, and that
+    /// offset is checked against the budget before anything past the marker is decompressed.
+    /// Every byte decompressed is counted; only the bytes inside the ranges are kept.
+    fn ranges(self, ranges: &[[u64; 2]], budget: &mut Budget) -> Result<Vec<Option<Vec<u8>>>, Error> {
+        const WHAT: &str = "the decompressed document";
+        let (mut dec, read) = match self {
+            Tail::Slice(s) => return Ok(ranges.iter().map(|&r| entry(s, r).map(<[u8]>::to_vec)).collect()),
+            Tail::Stream { dec, read } => (dec, read),
+        };
+        let end = ranges.iter().map(|&[offset, len]| offset.saturating_add(len)).max().unwrap_or(0);
+        if end > budget.left() {
+            return Err(Error::LimitExceeded(format!(
+                "the file states data up to byte {end} after its JSON, more than max_total ({} bytes) in all",
+                budget.limits.max_total
+            )));
+        }
+        // The ranges in file order, overlapping ones merged.
+        let mut spans: Vec<[u64; 2]> = ranges.iter().map(|&[offset, len]| [offset, offset.saturating_add(len)]).collect();
+        spans.sort_unstable();
+        let mut merged: Vec<[u64; 2]> = vec![];
+        for [start, end] in spans {
+            match merged.last_mut() {
+                Some(last) if start <= last[1] => last[1] = last[1].max(end),
+                _ => merged.push([start, end]),
             }
         }
+        let mut src = read.as_slice().chain(&mut *dec);
+        let (mut at, mut kept) = (0u64, Vec::<(u64, Vec<u8>)>::new());
+        for [start, end] in merged {
+            let skipped = std::io::copy(&mut (&mut src).take(start.saturating_sub(at)), &mut std::io::sink()).map_err(damaged)?;
+            budget.take(skipped, WHAT)?;
+            at += skipped;
+            if at < start {
+                break;
+            }
+            let mut bytes = vec![];
+            let n = (&mut src).take(end - start).read_to_end(&mut bytes).map_err(damaged)?;
+            budget.take(n as u64, WHAT)?;
+            at += n as u64;
+            kept.push((start, bytes));
+            if at < end {
+                break;
+            }
+        }
+        Ok(ranges
+            .iter()
+            .map(|&[offset, len]| kept.iter().find_map(|(start, bytes)| entry(bytes, [offset.checked_sub(*start)?, len])).map(<[u8]>::to_vec))
+            .collect())
     }
 }
 
@@ -90,12 +120,11 @@ fn unpack<'a>(bytes: &'a [u8], budget: &mut Budget) -> Result<Unpacked<'a>, Erro
     const JSON: &str = "the document's JSON";
     let too_long = |budget: &Budget| Error::LimitExceeded(format!("{JSON} is more than max_json ({} bytes)", budget.limits.max_json));
     if !is_compressed(bytes) {
-        let (json, tail) = match bytes.iter().position(|&b| b == 0) {
-            Some(nul) => {
-                let (end, tail) = marker(bytes, nul)?;
-                (bytes.get(..end).unwrap_or_default(), bytes.get(tail..).unwrap_or_default())
-            }
-            None => (bytes, &[][..]),
+        let (json, tail) = if bytes.contains(&0) {
+            let (end, tail) = marker(bytes)?;
+            (bytes.get(..end).unwrap_or_default(), bytes.get(tail..).unwrap_or_default())
+        } else {
+            (bytes, &[][..])
         };
         budget.check_json(json.len() as u64, JSON)?;
         return Ok(Unpacked { json: Cow::Borrowed(json), tail: Tail::Slice(tail) });
@@ -120,7 +149,7 @@ fn unpack<'a>(bytes: &'a [u8], budget: &mut Budget) -> Result<Unpacked<'a>, Erro
         if let Some(nul) = nul
             && buf.len() >= nul + BLOB_MAGIC.len() - 1
         {
-            let s = marker(&buf, nul)?;
+            let s = marker(&buf)?;
             split = Some(s);
             // The data after the marker in this chunk is counted when it is read.
             used = s.1.saturating_sub(start).min(n);
@@ -133,29 +162,78 @@ fn unpack<'a>(bytes: &'a [u8], budget: &mut Budget) -> Result<Unpacked<'a>, Erro
     let (json_len, tail_at) = match (split, nul) {
         (Some(s), _) => s,
         // The stream ended inside the marker.
-        (None, Some(nul)) => marker(&buf, nul)?,
+        (None, Some(_)) => marker(&buf)?,
         (None, None) => (buf.len(), buf.len()),
     };
     budget.check_json(json_len as u64, JSON)?;
     let read = if tail_at <= buf.len() { buf.split_off(tail_at) } else { vec![] };
     buf.truncate(json_len);
-    Ok(Unpacked { json: Cow::Owned(buf), tail: Tail::Stream { dec, read, counted: 0 } })
+    Ok(Unpacked { json: Cow::Owned(buf), tail: Tail::Stream { dec, read } })
 }
 
-/// The format name, the version and the writer, read without the rest of the file, so that a file
-/// of a newer version still reports them.
-#[derive(Deserialize)]
-struct Stamp {
+/// The format name, the version, the writer and, when `IMAGES` is true, the preview and the PDF,
+/// read without the rest of the file, so that a file of a newer version still reports them. The
+/// members are read as [`craft_vectorcraft_format::File::read`] reads them: a missing or repeated
+/// `format` or `version` is refused, and a `generator`, `preview` or `pdf` of the wrong shape
+/// reads as `None` and a repeated one as its last value.
+struct Stamp<const IMAGES: bool> {
     format: String,
     version: u32,
-    #[serde(default)]
     generator: Option<String>,
+    preview: Option<Image>,
+    pdf: Option<Image>,
+}
+
+/// Reads the value of member `name` into `slot`, refusing a second one.
+fn once<'de, A: MapAccess<'de>, T: Deserialize<'de>>(map: &mut A, slot: &mut Option<T>, name: &'static str) -> Result<(), A::Error> {
+    if slot.is_some() {
+        return Err(de::Error::duplicate_field(name));
+    }
+    *slot = Some(map.next_value()?);
+    Ok(())
+}
+
+impl<'de, const IMAGES: bool> Deserialize<'de> for Stamp<IMAGES> {
+    fn deserialize<D: Deserializer<'de>>(d: D) -> Result<Self, D::Error> {
+        struct StampVisitor<const IMAGES: bool>;
+        impl<'de, const IMAGES: bool> Visitor<'de> for StampVisitor<IMAGES> {
+            type Value = Stamp<IMAGES>;
+
+            fn expecting(&self, f: &mut fmt::Formatter) -> fmt::Result {
+                f.write_str("a VectorCraft file's JSON object")
+            }
+
+            fn visit_map<A: MapAccess<'de>>(self, mut map: A) -> Result<Stamp<IMAGES>, A::Error> {
+                let (mut format, mut version, mut generator, mut preview, mut pdf) = (None, None, None, None, None);
+                while let Some(key) = map.next_key::<String>()? {
+                    match key.as_str() {
+                        "format" => once(&mut map, &mut format, "format")?,
+                        "version" => once(&mut map, &mut version, "version")?,
+                        "generator" => generator = map.next_value::<Lenient<String>>()?.0,
+                        "preview" if IMAGES => preview = map.next_value::<Lenient<Image>>()?.0,
+                        "pdf" if IMAGES => pdf = map.next_value::<Lenient<Image>>()?.0,
+                        _ => {
+                            map.next_value::<IgnoredAny>()?;
+                        }
+                    }
+                }
+                Ok(Stamp {
+                    format: format.ok_or_else(|| de::Error::missing_field("format"))?,
+                    version: version.ok_or_else(|| de::Error::missing_field("version"))?,
+                    generator,
+                    preview,
+                    pdf,
+                })
+            }
+        }
+        d.deserialize_map(StampVisitor::<IMAGES>)
+    }
 }
 
 /// The file's [`Stamp`], refused when the format is not VectorCraft's, and when the version is
 /// newer than this crate reads unless `newer` is true.
-fn stamp(json: &[u8], budget: &Budget, newer: bool) -> Result<Stamp, Error> {
-    let s: Stamp = if newer { parse(json, budget.limits)? } else { parse_versioned(json, budget.limits, Kind::VectorCraft)? };
+fn stamp<const IMAGES: bool>(json: &[u8], budget: &Budget, newer: bool) -> Result<Stamp<IMAGES>, Error> {
+    let s: Stamp<IMAGES> = if newer { parse(json, budget.limits)? } else { parse_versioned(json, budget.limits, Kind::VectorCraft)? };
     match craft_vectorcraft_format::check(&s.format, s.version) {
         Ok(()) => Ok(s),
         Err(craft_vectorcraft_format::Error::Format(format)) => Err(malformed(format!("format is `{format}`"))),
@@ -167,44 +245,31 @@ fn stamp(json: &[u8], budget: &Budget, newer: bool) -> Result<Stamp, Error> {
 
 pub(crate) fn info(bytes: &[u8], budget: &mut Budget) -> Result<(u32, Option<String>), Error> {
     let json = unpack(bytes, budget)?.json;
-    let s = stamp(&json, budget, true)?;
+    let s = stamp::<false>(&json, budget, true)?;
     Ok((s.version, s.generator))
 }
 
-/// Check the entry's size against `max_entry` and count it; for an entry after the JSON, the
-/// offset where it ends.
-fn take(image: &Image, budget: &mut Budget, what: &str) -> Result<Option<u64>, Error> {
-    budget.take_entry(image.len(), what)?;
-    Ok(image.blob.map(|[offset, len]| offset.saturating_add(len)))
-}
-
-/// The entry's bytes, after [`take`] has checked its size.
-fn entry_bytes(image: &Image, tail: &[u8], what: &str) -> Result<Vec<u8>, Error> {
-    image.bytes_at_most(tail, u64::MAX).ok_or_else(|| match image.blob {
-        Some(_) => malformed(format!("the file ends before its {what}")),
-        None => malformed(format!("the {what} is not valid base64")),
-    })
-}
-
 pub(crate) fn renditions(bytes: &[u8], budget: &mut Budget) -> Result<Vec<Rendition>, Error> {
-    let Unpacked { json, mut tail } = unpack(bytes, budget)?;
-    stamp(&json, budget, false)?;
-    let head: Head = parse(&json, budget.limits)?;
+    let Unpacked { json, tail } = unpack(bytes, budget)?;
+    let head = stamp::<true>(&json, budget, false)?;
     // A member without a MIME type gives no rendition: a preview may be PNG or JPEG.
     let members: Vec<_> = [(RenditionKind::Thumbnail, head.preview, "preview"), (RenditionKind::Pdf, head.pdf, "PDF")]
         .into_iter()
         .filter_map(|(kind, m, what)| Some((kind, m.filter(|m| !m.mime.is_empty())?, what)))
         .collect();
     // Every size is checked before any data after the JSON is decompressed.
-    let mut end = 0u64;
     for (_, m, what) in &members {
-        end = end.max(take(m, budget, what)?.unwrap_or(0));
+        budget.take_entry(m.len(), what)?;
     }
-    let tail = if end > 0 { tail.up_to(end, budget)? } else { &[] };
+    let ranges: Vec<[u64; 2]> = members.iter().filter_map(|(_, m, _)| m.blob).collect();
+    let mut blobs = if ranges.is_empty() { vec![] } else { tail.ranges(&ranges, budget)? }.into_iter();
     members
         .into_iter()
         .map(|(kind, m, what)| {
-            let bytes = entry_bytes(&m, tail, what)?;
+            let bytes = match m.blob {
+                Some(_) => blobs.next().flatten().ok_or_else(|| malformed(format!("the file ends before its {what}")))?,
+                None => m.bytes_at_most(&[], u64::MAX).ok_or_else(|| malformed(format!("the {what} is not valid base64")))?,
+            };
             Ok(Rendition { kind, mime: m.mime, bytes })
         })
         .collect()
@@ -269,7 +334,7 @@ struct KindM {
 
 pub(crate) fn outline(bytes: &[u8], budget: &mut Budget) -> Result<Outline, Error> {
     let json = unpack(bytes, budget)?.json;
-    stamp(&json, budget, false)?;
+    stamp::<false>(&json, budget, false)?;
     let f: WithDocument = parse(&json, budget.limits)?;
     let pages = f
         .document

crates/doc/tests/envelopes.rs

@@ -121,6 +121,65 @@ fn a_pcraft_manifest_with_a_repeated_member_or_a_late_newer_version_is_refused()
     assert_eq!(outline(&bundle, &limits()), Err(refused));
 }
 
+#[test]
+fn a_pcraft_manifest_that_is_a_json_array_is_malformed() {
+    // The members of a manifest in field order; read_manifest reads only a JSON object.
+    let array = r#"[1,"photocraft-format 0.1.0",{"size":{"width":1,"height":1},"resolution_dpi":72.0,"mode":"Rgb","depth":"U8","layers":[]},"thumb.png",null]"#;
+    let format_limits = craft_pcraft_format::Limits::default();
+    assert!(craft_pcraft_format::read_manifest::<serde_json::Value, craft_pcraft_format::Error>(array.as_bytes(), &format_limits, &[], 1).is_err());
+    let bundle = zip(&[store("manifest.json", array.as_bytes()), store("thumb.png", &png(1, 1, 0))]);
+    assert!(matches!(info(&bundle, &limits()), Err(Error::Malformed(_))));
+    assert!(matches!(renditions(&bundle, &limits()), Err(Error::Malformed(_))));
+    assert!(matches!(outline(&bundle, &limits()), Err(Error::Malformed(_))));
+}
+
+#[test]
+fn a_newer_file_with_another_format_s_version_member_is_too_new() {
+    // `version` and `schema` are not PhotoCraft's version member, nor `schema` VectorCraft's.
+    let manifest = r#"{"format_version":2,"version":1,"schema":1,"generator":"photocraft-format 9","document":"elsewhere"}"#;
+    let pc = zip(&[store("manifest.json", manifest.as_bytes())]);
+    // A repeated `format` makes the stamp unreadable; the version still decides.
+    let vc = vectorcraft_json(5, r#""schema":1,"format":"vectorcraft""#);
+    for (bytes, kind, version) in [(pc, Kind::PhotoCraft, 2), (vc.into_bytes(), Kind::VectorCraft, 5)] {
+        let refused = Error::TooNew { kind, version, supported: kind.supported_version() };
+        assert_eq!(renditions(&bytes, &limits()), Err(refused.clone()), "{kind:?}");
+        assert_eq!(outline(&bytes, &limits()), Err(refused), "{kind:?}");
+    }
+    // Bytes after the JSON make a newer file malformed, not too new.
+    let trailing = format!("{}x", vectorcraft_json(5, ""));
+    assert!(matches!(renditions(trailing.as_bytes(), &limits()), Err(Error::Malformed(_))));
+}
+
+#[test]
+fn a_generator_that_is_not_a_string_reads_as_none() {
+    let vc = vectorcraft_json(4, "").replacen(r#""generator":"VectorCraft 0.1.0""#, r#""generator":{"name":"VectorCraft"}"#, 1);
+    assert_eq!(info(vc.as_bytes(), &limits()).unwrap().generator, None);
+    assert_eq!(outline(vc.as_bytes(), &limits()).unwrap().pages.len(), 2);
+    let file = craft_vectorcraft_format::File::<serde_json::Value>::read(vc.as_bytes()).unwrap();
+    assert_eq!(file.generator, None);
+    // read_manifest needs a string, so the other calls refuse it; migrate reads it as none.
+    let manifest = pcraft_manifest(1, "[]", "").replacen(r#""generator":"photocraft-format 0.1.0""#, r#""generator":7"#, 1);
+    let bundle = zip(&[store("manifest.json", manifest.as_bytes())]);
+    assert_eq!(info(&bundle, &limits()).unwrap().generator, None);
+    assert!(matches!(renditions(&bundle, &limits()), Err(Error::Malformed(_))));
+}
+
+#[test]
+fn a_repeated_vectorcraft_member_reads_as_its_last_value() {
+    let (first, last) = (png(1, 1, 0), png(2, 2, 9));
+    let members = format!(r#""generator":"VectorCraft 0.2.0","preview":{},"preview":{}"#, inline("image/png", &first), inline("image/png", &last));
+    let vc = vectorcraft_json(4, &members);
+    let file = craft_vectorcraft_format::File::<serde_json::Value>::read(vc.as_bytes()).unwrap();
+    assert_eq!(file.preview.unwrap().bytes(&[]).unwrap(), last);
+    let r = renditions(vc.as_bytes(), &limits()).unwrap();
+    assert_eq!((r.len(), &r[0].bytes), (1, &last));
+    assert_eq!(info(vc.as_bytes(), &limits()).unwrap().generator.as_deref(), Some("VectorCraft 0.2.0"));
+    // A repeated version is refused, as File::read refuses it.
+    let twice = vectorcraft_json(4, r#""version":4"#);
+    assert!(craft_vectorcraft_format::File::<serde_json::Value>::read(twice.as_bytes()).is_err());
+    assert!(matches!(info(twice.as_bytes(), &limits()), Err(Error::Malformed(_))));
+}
+
 #[test]
 fn a_vectorcraft_member_without_a_mime_type_is_skipped() {
     let file = vectorcraft_json(4, &format!(r#""preview":{{"data":"{}"}}"#, base64(&png(1, 1, 0))));

crates/doc/tests/formats.rs

@@ -58,6 +58,34 @@ fn sniff_refuses_other_files() {
     }
 }
 
+#[test]
+fn sniff_reads_the_names_in_the_central_directory_of_a_whole_archive() {
+    let mut pc = zip(&[store("manifest.json", pcraft_manifest(1, "[]", "").as_bytes())]);
+    // The local header still names manifest.json; the central directory names Xanifest.json.
+    let c = pc.windows(4).rposition(|w| w == [0x50, 0x4b, 0x01, 0x02]).unwrap();
+    pc[c + 46] = b'X';
+    assert_eq!(sniff(&pc), None);
+    assert_eq!(info(&pc, &limits()), Err(Error::Unrecognized));
+    // A head too short to reach the central directory is read through its local headers.
+    assert_eq!(sniff(pc.get(..60).unwrap()), Some(Kind::PhotoCraft));
+}
+
+#[test]
+fn a_zip_whose_manifest_has_no_format_version_is_not_photocraft() {
+    // A browser extension: a ZIP archive with a manifest.json of its own.
+    let ext = zip(&[store("manifest.json", br#"{"manifest_version":3,"name":"Example","version":"1.0"}"#)]);
+    assert_eq!(sniff(&ext), Some(Kind::PhotoCraft));
+    assert_eq!(info(&ext, &limits()), Err(Error::Unrecognized));
+    assert_eq!(renditions(&ext, &limits()), Err(Error::Unrecognized));
+    assert_eq!(outline(&ext, &limits()), Err(Error::Unrecognized));
+    // A format_version that is present and not a number is a damaged manifest.
+    for version in ["null", r#""1""#] {
+        let damaged = zip(&[store("manifest.json", pcraft_manifest(1, "[]", "").replacen("1", version, 1).as_bytes())]);
+        assert!(matches!(info(&damaged, &limits()), Err(Error::Malformed(_))), "{version}");
+        assert!(matches!(outline(&damaged, &limits()), Err(Error::Malformed(_))), "{version}");
+    }
+}
+
 #[test]
 fn info_reads_version_and_writer() {
     let i = info(vectorcraft_json(3, "").as_bytes(), &limits()).unwrap();
@@ -133,6 +161,43 @@ fn vectorcraft_members_after_the_json() {
     }
 }
 
+#[test]
+fn compressed_members_after_the_json_read_their_own_ranges() {
+    // 100,000 bytes the document uses, then the preview and the PDF; then the same bytes named by
+    // a PDF member that overlaps the preview.
+    let image = vec![0xCD; 100_000];
+    let preview = png(5, 5, 2);
+    let mut tail = image.clone();
+    tail.extend(&preview);
+    tail.extend(PDF);
+    let at = image.len();
+    for (pdf_at, pdf_len, pdf) in [(at + preview.len(), PDF.len(), PDF.to_vec()), (at, preview.len() + PDF.len(), [&preview[..], PDF].concat())] {
+        let members = format!(r#""preview":{},"pdf":{}"#, blob("image/png", at, preview.len()), blob("application/pdf", pdf_at, pdf_len));
+        let mut file = vectorcraft_json(4, &members).into_bytes();
+        file.extend(BLOB_MAGIC);
+        file.extend(&tail);
+        for bytes in [file.clone(), gzip(&file)] {
+            let r = renditions(&bytes, &limits()).unwrap();
+            assert_eq!((&r[0].bytes, &r[1].bytes), (&preview, &pdf));
+        }
+    }
+}
+
+#[test]
+fn the_marker_follows_the_json_directly() {
+    // A space between the JSON and the marker: the format crate does not split the file there.
+    let mut file = vectorcraft_json(4, &format!(r#""pdf":{}"#, blob("application/pdf", 0, PDF.len()))).into_bytes();
+    file.push(b' ');
+    file.extend(BLOB_MAGIC);
+    file.extend(PDF);
+    assert!(craft_vectorcraft_format::split_tail(&file).1.is_empty());
+    for bytes in [file.clone(), gzip(&file)] {
+        assert!(matches!(info(&bytes, &limits()), Err(Error::Malformed(_))));
+        assert!(matches!(renditions(&bytes, &limits()), Err(Error::Malformed(_))));
+        assert!(matches!(outline(&bytes, &limits()), Err(Error::Malformed(_))));
+    }
+}
+
 #[test]
 fn a_compressed_marker_across_the_reader_s_64_kib_chunks() {
     // The marker starts at every offset around the first 64 KiB boundary of the unpacked stream.
@@ -213,21 +278,29 @@ fn pcraft_outline_has_the_canvas_and_layer_tree() {
 }
 
 #[test]
-fn pcraft_groups_nest_to_the_depth_cap() {
-    // 3 levels to the layer list, 3 per group and 4 for the leaf: 397 of the 400 levels.
-    let mut layer = pcraft_layer("Leaf", true, "Normal", 1.0, raster());
-    for i in 0..130 {
-        layer = pcraft_layer(&format!("G{i}"), true, "Normal", 1.0, &format!(r#"{{"kind":"group","expanded":false,"children":[{layer}]}}"#));
-    }
-    let pc = zip(&[deflated("manifest.json", pcraft_manifest(1, &format!("[{layer}]"), "").as_bytes())]);
-    let o = outline(&pc, &limits()).unwrap();
+fn pcraft_groups_nest_to_photocraft_s_depth_limit() {
+    // 3 levels to the layer list, 3 per group and 4 for the leaf: 367 levels for 120 groups,
+    // PhotoCraft's limit (`manifest_depth(MAX_GROUP_DEPTH)`).
+    let nested = |groups: usize| {
+        let mut layer = pcraft_layer("Leaf", true, "Normal", 1.0, raster());
+        for i in 0..groups {
+            layer = pcraft_layer(&format!("G{i}"), true, "Normal", 1.0, &format!(r#"{{"kind":"group","expanded":false,"children":[{layer}]}}"#));
+        }
+        zip(&[deflated("manifest.json", pcraft_manifest(1, &format!("[{layer}]"), "").as_bytes())])
+    };
+    let o = outline(&nested(120), &limits()).unwrap();
     let mut depth = 0;
     let mut l = &o.layers[0];
     while let Some(c) = l.children.first() {
         depth += 1;
         l = c;
     }
-    assert_eq!((depth, l.name.as_str()), (130, "Leaf"));
+    assert_eq!((depth, l.name.as_str()), (120, "Leaf"));
+    // One more group is past the limit, though within MAX_DEPTH.
+    let deeper = nested(121);
+    assert!(matches!(outline(&deeper, &limits()), Err(Error::LimitExceeded(_))));
+    assert!(matches!(renditions(&deeper, &limits()), Err(Error::LimitExceeded(_))));
+    assert_eq!(info(&deeper, &limits()).unwrap().version, 1);
 }
 
 #[test]

crates/doc/tests/hostile.rs

@@ -115,6 +115,25 @@ fn a_compressed_member_past_max_entry_is_refused_before_the_tail_is_inflated() {
     }
 }
 
+#[test]
+fn a_compressed_member_whose_offset_passes_max_total_is_refused_before_the_tail_is_inflated() {
+    // A 4-byte preview stated 200 MiB past the marker, in a file whose data after the JSON is
+    // 1 MiB.
+    let mut file = vectorcraft_json(4, &format!(r#""preview":{}"#, blob("image/png", 200 << 20, 4))).into_bytes();
+    file.extend(BLOB_MAGIC);
+    file.extend(vec![0u8; 1 << 20]);
+    let gz = gzip(&file);
+    let mut limits = small();
+    limits.max_total = 64 << 20;
+    match renditions(&gz, &limits) {
+        Err(Error::LimitExceeded(m)) => assert!(m.contains("max_total"), "{m}"),
+        other => panic!("{other:?}"),
+    }
+    // Within max_total, the stream is read as far as it goes, and it ends first.
+    limits.max_total = 256 << 20;
+    assert!(matches!(renditions(&gz, &limits), Err(Error::Malformed(_))));
+}
+
 #[test]
 fn a_nul_outside_the_marker_is_malformed_without_reading_on() {
     // A NUL where the marker is not, then 8 MiB that compress to almost nothing.