* M1: a crypt filter selected by reference is used A stream can name its own crypt filter, and every part of that can be indirect. cos read only the direct forms, and took anything else for no name, which is /Identity. So a stream selecting a filter by reference was saved in the clear in a file meant to be encrypted, and in a file from elsewhere its ciphertext was read as data. The /Encrypt dictionary's /StmF, /StrF, /EFF, /CFM, /Filter, /EncryptMetadata and a filter's /Length had the same fault, and /StmF by reference left every stream undecrypted. A stream's /Type and a signature dictionary's are read through references too. transform now reads these from the document, so a full save encrypts before it renumbers: the references in what it reads have to be the document's. Dropping a crypt filter that new protection doesn't define comes before the walk, so what it copies out of a reference is encrypted too. Loading can re-enter here, and LoadGuard ends a cycle, which reads as absent. A chain longer than any decoder takes names no filter: its entries would be followed for every stream sharing it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY * M2: a stream with a crypt filter of its own is drawn The renderer decrypted every stream with the document's filter and then failed on /Crypt as a filter it couldn't decode, so a stream naming its own filter read as nothing, however it was spelled, and a page whose content was such a stream stayed blank. It now uses the filter the stream names, /Identity without one, and /Crypt is nothing further to decode (hayro-syntax patch 30). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY * M1: tests for a crypt filter's place among the others A lone parameter dictionary is the first filter's, so /Crypt behind another filter has no name of its own, and an array too short for /Crypt's entry stays as it is when the filter is dropped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY * M1: encryption reads every legal form, and a full save keeps them From a review of the crypt filter fix. The renderer read a stream's own filter from the form /DecodeParms had to match the form of /Filter, so [/Crypt] with a lone dictionary, an everyday spelling, read as /Identity and the page stayed blank. Parameters are now taken in either form, which also gives [/FlateDecode] its predictor. It also refused a V4 file without /StmF or /StrF, which are optional, and used a /CF entry named /Identity, which is never one. A chain past what cos decodes names no filter there, so it names none here either (hayro-syntax patch 30). A full save lost what a direct /Encrypt refers to, and kept its references under the old numbers, so nothing in the file read afterwards. /O or /U given by reference were decrypted and encrypted again under a new number, so the file it wrote no longer opened. What /Encrypt and /ID refer to now stays as stored. /ID by reference, which has to be direct, is read, and written direct. A signature dictionary is known by a /ByteRange given by reference, and a null or dangling /Type or /Subtype is an absent one. References to references are followed in Document::resolve, so /Name by way of two no longer saves a stream in the clear. New protection drops every /Crypt in a chain. An object stream's /DecodeParms can be given by reference. Reading what decides how an object is treated can load more objects, each doing the same, which a file could make a tree that doubles with every level once the cache lets go of it. What is loaded for one of these no longer follows its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY * M1: what a read follows is bounded by a budget, and nothing shared is copied per stream From a review of the fixes before this, all in them. A flag that kept what is loaded for a selector from following its own stopped the fan-out, but a legal file lost object stream members to it: read first through a selector, an object stream with parameters by reference was kept undecoded, and a full save wrote nulls without saying so. A budget replaces it: one read or write of an object, with all it loads on the way, follows up to 32 references, which is more than any object needs when all of it is by reference. Past that they read as absent, so a file's ladder of selectors still stops, and nothing a real file holds is lost. Finding out whether a dictionary is a signature's loaded objects for every dictionary of an object, and a load that fails takes a scan of the file, so one read could hang. Only a dictionary with a direct string /Contents is asked now, and the budget bounds those. /ID written direct copied every entry, so a 1 MB file naming a large string thousands of times ran out of memory: it is at most two strings now. Dropping a crypt filter copied a shared /DecodeParms or /Filter into every stream, 1 GB from a 1 MB file: of what's shared only filter names and the parameters decoders read are copied now, through Document::with_direct_parms. The renderer skips /Crypt in a chain instead of copying the data for each entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY * M1: only an object itself follows references, and a failed scan is remembered From a review of the budget. Following a reference while walking deep inside an object added that depth again for each object loaded, which overflowed a 2 MiB stack on a file under 10 KB and took the process with it, past any catch_unwind. References are now followed for the object itself and never for what is nested in it. A stream can't be nested, and a signature dictionary nested in another object reads as it says directly, as it did before this PR. walk is back to following nothing. A reference that fails to load scans the file, and nothing remembered that, so a 1 MB file took minutes to save. What a scan didn't find is kept for the document: the file doesn't change. Filter names copied out of a shared chain are bounded, since a name can be any length. /ID keeps each entry in its place, or the key made from the first entry changes. direct_id's comment was on another function. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY * M1: tests with no deadline a busy machine can reach Wide margins on the two that time what they guard. A test of dictionaries nested in an object goes: nothing nested follows a reference now, which the small-stack test covers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY * M1: a crypt filter past what a read may follow is the document's own Past the budget a reference reads as absent, and a crypt filter with no /Name is /Identity, so a stream that selected one by reference had its ciphertext read as data. What can't be told takes the document's filter, as a /Filter that can't be read does. Found by CodeRabbit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY * M1: a read refused, not a budget full, is what leaves a crypt filter unknown My last commit took a read that used up all it may follow for one that was refused, so an /Identity stream read at exactly 32 follows was decrypted to garbage. Refused reads are counted now, and only one on the way to this stream's /Name says anything about it. Found by CodeRabbit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY * M1: a save keeps the handler, the key and the permissions Under every algorithm and by either kind of save, `/Encrypt`, `/ID`, the file key and the permissions come back as they were. No test held that. `/ID` by reference is held through an incremental save too, which leaves it as it was. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY * cos: encrypt an edited or added object whatever /Encrypt names stored_as_is now also requires the object to be unedited, so an edit of an object /Encrypt or /ID refers to, or a new object given a number they name, is encrypted on both saves. encryption_members skips numbers with no object behind them and streams, which those dictionaries never hold. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
hayro-syntax
A low-level library for reading PDF files.
This crate implements the Syntax chapter of the PDF reference, and therefore
serves as a very good basis for building various abstractions on top of it, without having to reimplement
the PDF parsing logic.
This crate does not provide more high-level functionality, such as parsing fonts or color spaces.
Such functionality is out-of-scope for hayro-syntax, since this crate is supposed to be
as light-weight and application-agnostic as possible.
Functionality-wise, this crate is therefore close to feature-complete. The main missing feature is support for password-protected documents. In addition to that, more low-level APIs might be added in the future.
The crate is no_std compatible but requires an allocator to be available.
Example
This short example shows you how to load a PDF file and iterate over the content streams of all pages.
use hayro_syntax::Pdf;
use std::path::PathBuf;
// First load the data that constitutes the PDF file.
let data = std::fs::read(
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../hayro-tests/pdfs/custom/text_with_rise.pdf"),
)
.unwrap();
// Then create a new PDF file from it.
//
// Here we are just unwrapping in case reading the file failed, but you
// might instead want to apply proper error handling.
let pdf = Pdf::new(data).unwrap();
// First access all pages, and then iterate over the operators of each page's
// content stream and print them.
let pages = pdf.pages();
for page in pages.iter() {
let mut ops = page.typed_operations();
while let Some(op) = ops.next() {
println!("{op:?}");
}
}
Safety
When the unsafe feature is disabled, there is only use of unsafe in hayro-syntax.
We also unconditionally use the smallvec crate which uses unsafe internally, but that's
it.
For better performance, it is strongly recommended to enable the unsafe feature,
which does result in slightly more unsafe code being, but does give better performance:
- We will use the
flate2crate for decoding flate streams. - SIMD will be enabled to accelerate decoding of images.
memchrwill be used to accelerate some of the fallback parsing code.
Features
The supported features include:
- Parsing xref tables in all its possible formats, including xref streams.
- Best-effort attempt at repairing PDF files with broken xref tables.
- Parsing of all objects types (also in object streams).
- Parsing and decoding PDF streams.
- Iterating over pages as well as their content streams in a typed or untyped fashion.
- The crate is very lightweight, especially in comparison to other PDF crates.
Limitations
- There are still a few features missing, for example, support for password-protected PDFs. In addition to that, many properties (like page annotations) are currently not exposed.
- This crate is for read-only processing, you cannot directly use it to manipulate PDF files. If you need to do that, there are other crates in the Rust ecosystem that are suitable for this.
License
Licensed under either of
- Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.