Compare commits
70
Commits
No files matched your search
@@ -47,6 +47,10 @@ jobs:
|
||||
- name: XML, desktop entry, AppStream, Flatpak manifest
|
||||
run: |
|
||||
xmllint --noout packaging/windows/filmcraft.wxs packaging/macos/*.plist* packaging/linux/*.xml*
|
||||
# Windows Installer needs Icon ids ending in .ico or .exe, or shortcuts show a blank icon.
|
||||
if grep -oE '<Icon Id="[^"]*"' packaging/windows/filmcraft.wxs | grep -vE '\.(ico|exe)"$'; then
|
||||
echo "::error::Icon Id must end in .ico or .exe"; exit 1
|
||||
fi
|
||||
desktop-file-validate packaging/linux/ai.storyteller.filmcraft.desktop
|
||||
sed -e 's/@VERSION@/0.0.0/' -e 's/@DATE@/2026-01-01/' packaging/linux/ai.storyteller.filmcraft.metainfo.xml.in >"$RUNNER_TEMP/ai.storyteller.filmcraft.metainfo.xml"
|
||||
appstreamcli validate --no-net --explain "$RUNNER_TEMP/ai.storyteller.filmcraft.metainfo.xml"
|
||||
|
||||
@@ -221,8 +221,9 @@ jobs:
|
||||
- name: Install wasm-bindgen CLI
|
||||
run: |
|
||||
# Must match the wasm-bindgen crate version exactly (WASM_BINDGEN in xtask/src/main.rs).
|
||||
command -v wasm-bindgen >/dev/null && wasm-bindgen --version | grep -q "$WASM_BINDGEN_VERSION" \
|
||||
|| cargo install wasm-bindgen-cli --version "$WASM_BINDGEN_VERSION" --locked
|
||||
if ! { command -v wasm-bindgen >/dev/null && wasm-bindgen --version | grep -q "$WASM_BINDGEN_VERSION"; }; then
|
||||
cargo install wasm-bindgen-cli --version "$WASM_BINDGEN_VERSION" --locked
|
||||
fi
|
||||
wasm-bindgen --version
|
||||
- name: Apply version override
|
||||
if: inputs.version != ''
|
||||
|
||||
@@ -19,7 +19,12 @@ panic path, and fix a crash before building on top of it. The cross-app standard
|
||||
fallback that is truly correct (never one that silently corrupts a project). An unfinished feature
|
||||
returns an "unsupported" error. The only exception is a call that is provably infallible from local
|
||||
code alone (a literal that always parses): a single-item `#[allow(clippy::expect_used)]` with the reason.
|
||||
3. **No `unsafe`** (`unsafe_code = "forbid"` for the workspace).
|
||||
3. **No `unsafe`** (`unsafe_code = "forbid"` for the workspace). The single exception is
|
||||
`crates/platform`, which needs `unsafe` to call the operating system's media APIs (hardware
|
||||
video decoding through VideoToolbox; later encoding and other OSes) and does nothing else. It
|
||||
uses `unsafe_code = "deny"` with `#[allow(unsafe_code)]` only on its FFI modules, a
|
||||
`// SAFETY:` comment on every `unsafe` block, a safe `Result`-returning public API, and the
|
||||
pure-Rust decoder as the tested fallback ([ADR 0001](docs/adr/0001-platform-ffi.md)).
|
||||
4. **Every input-derived number is hostile.** Media files, project and interchange files, presets, fonts,
|
||||
CLI / MCP / control-channel parameters and UI state are untrusted. Index and slice with `get()` (or
|
||||
validate bounds once, up front, for a hot loop); slice strings only at char boundaries; use
|
||||
@@ -115,6 +120,12 @@ See [§0](#0-never-crash). It outranks every other engineering rule.
|
||||
See `CLAUDE.md`: pure Rust, dependency layering (`cargo xtask layers`), exact `Tick` time, everything is
|
||||
a command, everything is agent-drivable, and the quality gates (`cargo xtask ci`) before every commit.
|
||||
|
||||
Shared real-file test corpora (Photoshop-authored PSDs, etc.) live in
|
||||
[`storytold/photocraft-corpus`](https://github.com/storytold/photocraft-corpus), explained in
|
||||
[craftrules `standards/test-corpora.md`](https://github.com/storytold/craftrules/blob/main/standards/test-corpora.md).
|
||||
Never commit large binary fixtures to this repo; fetch them pinned by commit and sha256-verified,
|
||||
as PhotoCraft does with `cargo xtask corpus`.
|
||||
|
||||
## See also
|
||||
|
||||
- [CONTRIBUTING.md](CONTRIBUTING.md) and [docs/contributing.md](docs/contributing.md): setup, gates, commits, how to add things
|
||||
|
||||
@@ -12,7 +12,7 @@ FilmCraft is a clean-room, open-source, pure-Rust non-linear video editor target
|
||||
**Read [`AGENTS.md`](AGENTS.md) first; its rules override everything here.**
|
||||
- **Never crash** ([`AGENTS.md`](AGENTS.md) §0, standard: [`craftrules/standards/never-crash.md`](https://github.com/storytold/craftrules/blob/main/standards/never-crash.md)). People depend on FilmCraft; a crash loses their work, so this outranks feature work. No `unwrap`/`expect`/`panic!`/`unreachable!`/`todo!`/`unimplemented!` or `unsafe` outside tests: return `Result<T, E>` and propagate with `?` (`ok_or`, `let … else`, `if let`). Treat every number from media, project files, presets or CLI/MCP/control params as hostile: `get()` instead of `[i]`, checked/saturating math, no division by possibly-zero values, capped allocations, bounded recursion. `lock().unwrap_or_else(PoisonError::into_inner)`; background jobs run under `catch_unwind`. The panic hook + UI `catch_unwind` (`crates/ui-egui/src/crash.rs`) are the last-resort guard, not a licence. Every crash fix gets a regression test that panicked before. In particular (§1): never use Adobe iconography, images or any other Adobe asset. Every asset must be openly licensed (OSS / public domain / Creative Commons, or original work by a contributor) and must have a `<file>.attribution` sidecar plus an entry in `ATTRIBUTION.md`. `cargo xtask assets` enforces this.
|
||||
- **Clean-room.** Never read/disassemble anything inside Adobe app bundles (names/listings only). Never copy Adobe icons, presets, LUTs, fonts. Never copy GPL/LGPL/AGPL code (FFmpeg, x264, x265, MLT, Kdenlive, Shotcut, Olive, LAME…). ffmpeg/ffprobe run only as external test oracles / fixture generators — never linked or shipped.
|
||||
- **Pure Rust** in the product. OS media APIs only via Rust bindings in `crates/platform` (planned; not created yet), optional, behind traits.
|
||||
- **Pure Rust** in the product. OS media APIs only via Rust bindings in `crates/platform` (hardware decoding; the one crate allowed `unsafe`, see `AGENTS.md` §0.3 and `docs/adr/0001-platform-ffi.md`), optional, behind traits, with our decoders as the fallback.
|
||||
- **Layering** ([`docs/architecture.md`](docs/architecture.md) §1, enforced by `cargo xtask layers`): nothing below L5 depends on egui/eframe/winit/rfd/cpal. L0 codec/container crates depend only on `filmcraft-bitstream`.
|
||||
- **Exact time:** all time is `filmcraft_time::Tick` (254 016 000 000/s). Never use f64 seconds for edit math.
|
||||
- **Everything is a command** (`crates/engine`): id, label, menu path, shortcut, params, enabled(), run(). UI, CLI, control channel and MCP all dispatch by id.
|
||||
|
||||
Generated
+116
-45
@@ -1655,13 +1655,14 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"chrono",
|
||||
"cpal",
|
||||
"eframe",
|
||||
"egui",
|
||||
"filmcraft-engine",
|
||||
"filmcraft-platform",
|
||||
"filmcraft-ui-egui",
|
||||
"log",
|
||||
"muda",
|
||||
@@ -1676,7 +1677,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-aac"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"filmcraft-testkit",
|
||||
@@ -1686,7 +1687,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-ac3"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
"thiserror 2.0.21",
|
||||
@@ -1694,14 +1695,14 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-audio-dsp"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-automation"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"base64 0.22.1",
|
||||
"filmcraft-engine",
|
||||
@@ -1717,7 +1718,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-av1"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"filmcraft-testkit",
|
||||
@@ -1728,14 +1729,14 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-bitstream"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"proptest",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-captions"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"filmcraft-project",
|
||||
@@ -1748,11 +1749,11 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-cfb"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-cli"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-automation",
|
||||
"filmcraft-codecs",
|
||||
@@ -1761,6 +1762,7 @@ dependencies = [
|
||||
"filmcraft-mpegts",
|
||||
"filmcraft-mxf",
|
||||
"filmcraft-ogg",
|
||||
"filmcraft-platform",
|
||||
"filmcraft-time",
|
||||
"serde_json",
|
||||
"tokio",
|
||||
@@ -1768,7 +1770,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-codecs"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-aac",
|
||||
"filmcraft-ac3",
|
||||
@@ -1800,7 +1802,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-color"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"proptest",
|
||||
"serde",
|
||||
@@ -1808,7 +1810,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-dnx"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"filmcraft-testkit",
|
||||
@@ -1818,7 +1820,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-edit"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-media",
|
||||
"filmcraft-project",
|
||||
@@ -1830,7 +1832,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-engine"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-audio-dsp",
|
||||
"filmcraft-captions",
|
||||
@@ -1861,7 +1863,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-export"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-aac",
|
||||
"filmcraft-audio-dsp",
|
||||
@@ -1890,7 +1892,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-format"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-media",
|
||||
"filmcraft-project",
|
||||
@@ -1902,7 +1904,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-frame"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"filmcraft-time",
|
||||
@@ -1912,7 +1914,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-geom"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"proptest",
|
||||
"serde",
|
||||
@@ -1920,7 +1922,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-golden"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"filmcraft-frame",
|
||||
@@ -1937,7 +1939,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-gpu"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"filmcraft-frame",
|
||||
@@ -1954,7 +1956,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-h264"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"filmcraft-testkit",
|
||||
@@ -1964,7 +1966,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-h264enc"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"filmcraft-testkit",
|
||||
@@ -1974,7 +1976,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-hevc"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"filmcraft-testkit",
|
||||
@@ -1984,7 +1986,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-interchange"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-cfb",
|
||||
"filmcraft-color",
|
||||
@@ -2001,7 +2003,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-isobmff"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"filmcraft-testkit",
|
||||
@@ -2012,7 +2014,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-matroska"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
"serde_json",
|
||||
@@ -2020,7 +2022,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-media"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"filmcraft-frame",
|
||||
@@ -2033,7 +2035,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-mpeg2v"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
"rayon",
|
||||
@@ -2042,7 +2044,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-mpegts"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
"thiserror 2.0.21",
|
||||
@@ -2050,28 +2052,46 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-mxf"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-ogg"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-opus"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-platform"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-codecs",
|
||||
"filmcraft-color",
|
||||
"filmcraft-frame",
|
||||
"filmcraft-isobmff",
|
||||
"filmcraft-media",
|
||||
"filmcraft-testkit",
|
||||
"filmcraft-time",
|
||||
"log",
|
||||
"objc2-core-foundation",
|
||||
"objc2-core-media",
|
||||
"objc2-core-video",
|
||||
"objc2-video-toolbox",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-project"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"filmcraft-geom",
|
||||
@@ -2085,7 +2105,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-prores"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"filmcraft-testkit",
|
||||
@@ -2095,7 +2115,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-render"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-audio-dsp",
|
||||
"filmcraft-captions",
|
||||
@@ -2115,7 +2135,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-scopes"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"serde",
|
||||
@@ -2124,7 +2144,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-speech"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"candle-core",
|
||||
"candle-nn",
|
||||
@@ -2142,14 +2162,14 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-testkit"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"png",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-text"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"harfrust",
|
||||
@@ -2160,7 +2180,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-time"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"proptest",
|
||||
"serde",
|
||||
@@ -2168,7 +2188,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-ui-egui"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"eframe",
|
||||
"egui",
|
||||
@@ -2183,6 +2203,7 @@ dependencies = [
|
||||
"filmcraft-gpu",
|
||||
"filmcraft-interchange",
|
||||
"filmcraft-media",
|
||||
"filmcraft-platform",
|
||||
"filmcraft-project",
|
||||
"filmcraft-render",
|
||||
"filmcraft-scopes",
|
||||
@@ -2200,7 +2221,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-vp9"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"rayon",
|
||||
@@ -2209,7 +2230,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-web"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"eframe",
|
||||
"egui",
|
||||
@@ -3779,7 +3800,9 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2a180dd8642fa45cdb7dd721cd4c11b1cadd4929ce112ebd8b9f5803cc79d536"
|
||||
dependencies = [
|
||||
"bitflags 2.13.2",
|
||||
"block2 0.6.2",
|
||||
"dispatch2",
|
||||
"libc",
|
||||
"objc2 0.6.4",
|
||||
]
|
||||
|
||||
@@ -3820,6 +3843,37 @@ dependencies = [
|
||||
"objc2-foundation 0.2.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2-core-media"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "05ec576860167a15dd9fce7fbee7512beb4e31f532159d3482d1f9c6caedf31d"
|
||||
dependencies = [
|
||||
"bitflags 2.13.2",
|
||||
"block2 0.6.2",
|
||||
"dispatch2",
|
||||
"objc2 0.6.4",
|
||||
"objc2-core-audio",
|
||||
"objc2-core-audio-types",
|
||||
"objc2-core-foundation",
|
||||
"objc2-core-video",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2-core-video"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d425caf1df73233f29fd8a5c3e5edbc30d2d4307870f802d18f00d83dc5141a6"
|
||||
dependencies = [
|
||||
"bitflags 2.13.2",
|
||||
"block2 0.6.2",
|
||||
"objc2 0.6.4",
|
||||
"objc2-core-foundation",
|
||||
"objc2-core-graphics",
|
||||
"objc2-io-surface",
|
||||
"objc2-metal 0.3.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2-encode"
|
||||
version = "4.1.0"
|
||||
@@ -3992,6 +4046,23 @@ dependencies = [
|
||||
"objc2-foundation 0.2.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2-video-toolbox"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "05bf9a3c14831a7d9641b0d81d87dd913ee238a012b2fde27db5a84b56f5df3e"
|
||||
dependencies = [
|
||||
"bitflags 2.13.2",
|
||||
"block2 0.6.2",
|
||||
"objc2 0.6.4",
|
||||
"objc2-core-foundation",
|
||||
"objc2-core-graphics",
|
||||
"objc2-core-media",
|
||||
"objc2-core-video",
|
||||
"objc2-foundation 0.3.2",
|
||||
"objc2-metal 0.3.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "once_cell"
|
||||
version = "1.21.4"
|
||||
@@ -7140,7 +7211,7 @@ checksum = "e450f9b2ed1dff33c94c12589a87338689467b9c4f5d8a5710bd09a847d2c8a7"
|
||||
|
||||
[[package]]
|
||||
name = "xtask"
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
"serde_json",
|
||||
|
||||
+2
-1
@@ -4,7 +4,7 @@ members = ["crates/*", "apps/*", "xtask"]
|
||||
default-members = ["crates/*", "apps/*", "xtask"]
|
||||
|
||||
[workspace.package]
|
||||
version = "0.1.1"
|
||||
version = "0.2.1"
|
||||
edition = "2024"
|
||||
license = "MIT OR Apache-2.0"
|
||||
rust-version = "1.95"
|
||||
@@ -51,6 +51,7 @@ filmcraft-format = { path = "crates/format" }
|
||||
filmcraft-engine = { path = "crates/engine" }
|
||||
filmcraft-ui-egui = { path = "crates/ui-egui" }
|
||||
filmcraft-automation = { path = "crates/automation" }
|
||||
filmcraft-platform = { path = "crates/platform" }
|
||||
filmcraft-testkit = { path = "crates/testkit" }
|
||||
|
||||
# foundation
|
||||
|
||||
@@ -184,7 +184,7 @@ No FFmpeg inside. The video codecs, AAC, Opus and the containers are our own Rus
|
||||
| **Matroska / WebM** | ✓ | | Lacing, Cues, header stripping, HDR colour metadata |
|
||||
| **Stills** | ✓ | ✓ | Import PNG, JPEG, GIF, WebP, TIFF and BMP; export PNG sequences and animated GIF |
|
||||
|
||||
MXF and hardware decode are next ([roadmap](ROADMAP.md)).
|
||||
Also imported: MPEG-2 / MPEG-1 video, AC-3, MP2, MXF (OP1a / OP-Atom), MPEG transport and program streams (AVCHD, broadcast, DVD), Ogg and image sequences. Not yet: HEVC and AV1 export, camera RAW, E-AC-3. Hardware decoding works on macOS (VideoToolbox), with Linux and Windows to follow ([#30](https://github.com/storytold/filmcraft/issues/30)).
|
||||
|
||||
<br>
|
||||
|
||||
@@ -266,11 +266,26 @@ The control protocol is documented in [docs/control-protocol.md](docs/control-pr
|
||||
| [docs/control-protocol.md](docs/control-protocol.md) | Control-channel and MCP reference |
|
||||
| [docs/project-files.md](docs/project-files.md) | `.fcproj` format, schema migrations, auto-save and crash recovery |
|
||||
| [docs/graphics.md](docs/graphics.md) · [docs/captions.md](docs/captions.md) | Text engine, graphic clips and tools; caption tracks and formats |
|
||||
| [ROADMAP.md](ROADMAP.md) | Milestones and estimates |
|
||||
| [ROADMAP.md](ROADMAP.md) | Honest assessment, what's missing, milestones and estimates |
|
||||
|
||||
## Status
|
||||
|
||||
FilmCraft is young and moving fast: roughly 45% of the way to Premiere Pro parity, with editing, trimming, colour, keyframes, titles, captions, mixing, codecs and export working today. [ROADMAP.md](ROADMAP.md) tracks every milestone with estimates.
|
||||
FilmCraft is young and moving fast. Editing, trimming, multicam, colour, keyframes, effects, titles, captions, mixing, codecs and export work today.
|
||||
|
||||
We track two numbers ([ROADMAP.md](ROADMAP.md#honest-assessment-2026-10-05)):
|
||||
|
||||
- **Feature checklist: ~87%.** Premiere Pro's menu items, effects, transitions, panels and formats that exist in FilmCraft.
|
||||
- **Ready for real work: ~50–60%.** Our honest estimate of how close FilmCraft is to replacing Premiere on real projects.
|
||||
|
||||
The biggest gaps today:
|
||||
|
||||
- **Speed on big footage.** Hardware decoding works on macOS; Windows and Linux have no hardware path yet. Blend modes and the most common effects run on the GPU, but Lumetri, keys, export and encoding still run on the CPU ([#30](https://github.com/storytold/filmcraft/issues/30)).
|
||||
- **No plugins.** No VST3 / Audio Units or OpenFX hosting.
|
||||
- **Delivery codecs.** H.264 is our only delivery-codec export; no HEVC or AV1 export yet.
|
||||
- **Real-world media and platforms.** Our decoders are bit-exact on conformance streams, but camera and phone files in the wild are less tested. Windows and Linux get far less testing than macOS.
|
||||
- **AI features.** Few so far; speech to text is optional and off by default.
|
||||
|
||||
Bug reports with real footage are the most useful thing you can send us: [open an issue](https://github.com/storytold/filmcraft/issues) or tell us in [Discord](https://discord.gg/artcraft).
|
||||
|
||||
## Architecture
|
||||
|
||||
|
||||
+90
-6
@@ -1,10 +1,88 @@
|
||||
# FilmCraft Roadmap
|
||||
|
||||
Progress toward feature parity with Adobe Premiere Pro, with estimates. Updated as milestones land.
|
||||
Progress toward parity with Adobe Premiere Pro, with estimates. Updated as milestones land.
|
||||
|
||||
**Last updated:** 2026-10-02 · **Overall parity:** ~87% (measured scorecard below) · **Code:** 34 crates, 1605 tests
|
||||
**Last updated:** 2026-10-05 · **Feature checklist:** ~87% · **Ready for real work:** ~50–60% · **Code:** 35 crates, 1605+ tests
|
||||
|
||||
## Parity scorecard
|
||||
Two numbers, because they answer different questions. The **feature checklist** counts whether
|
||||
Premiere's menu items, effects, panels and formats exist in FilmCraft. **Ready for real work** is
|
||||
our honest estimate of how close FilmCraft is to replacing Premiere for an editor on real
|
||||
projects: depth, correctness, robustness, speed and the pro ecosystem. The gap between the two
|
||||
is the work that matters most now. Agents: read [Where we are lacking](#where-we-are-lacking)
|
||||
before choosing work.
|
||||
|
||||
## Honest assessment (2026-10-05)
|
||||
|
||||
### How the checklist number is made, and what it doesn't tell you
|
||||
|
||||
- **Partly measured.** The menu figure comes from diffing a dump of Premiere 26.5's menus against
|
||||
`filmcraft-cli commands`. The effects, transitions and audio-effects counts are checked by unit
|
||||
tests against Premiere's Effects panel tree (`premiere_26_video_effects_catalogue`,
|
||||
`premiere_audio_set_is_complete_and_foldered`, `premiere_26_transition_tree`).
|
||||
- **Partly agent-reported.** The other areas' percentages are written by the agents doing the work.
|
||||
No automated parity check runs in CI (PhotoCraft has `cargo xtask parity`; we don't yet), so
|
||||
nothing stops a number drifting upward.
|
||||
- **Presence, not quality.** An item counts once it exists. An effect described as an
|
||||
"approximation" counts the same as an exact one. Nothing compares our rendering with
|
||||
Premiere's.
|
||||
- **Weights understate what users hit first.** Performance carries 5%. Plugins and AI features
|
||||
aren't in the scorecard at all.
|
||||
|
||||
### By dimension
|
||||
|
||||
| Dimension | Checklist says | Honest estimate | Evidence |
|
||||
|---|---|---|---|
|
||||
| **Feature breadth** (menus, commands, panels) | ~86–95% | **~85%** | Menu diff against Premiere 26.5; effect, transition and audio-effect catalogues complete by test. The long tail is mostly there. |
|
||||
| **Correctness in depth** | not tracked | **~60–70%** | A first-time contributor found four real bugs in core paths within hours (#5–#8: AIFF import failed, Slide left linked audio behind, transitions didn't follow ripple trims, export ignored start/end times). User reports: Color Matte always grey with no picker (#29); ALSA audio error on Linux (#23). Settings are "most wired", not all. |
|
||||
| **Codecs and media** | ~90% | **~75%** | Our decoders are bit-exact on conformance streams. Real camera and phone media (variable frame rate, damaged files, unusual containers) is far less tested. No camera RAW (RED, BRAW, ARRIRAW), no E-AC-3. |
|
||||
| **Export** | ~92% | **~70%** | Delivery: H.264 only (ProRes, DNxHR and image sequences cover mastering). No HEVC or AV1 export. AAF and OMF have never been validated in Avid Media Composer or Pro Tools. Export renders and encodes on the CPU only. |
|
||||
| **Performance and hardware** | ~62% | **~35–40%** | Premiere runs effects, decoding and encoding on the GPU and the hardware media engines. Here, hardware decode works on macOS only (VideoToolbox, #33). The GPU does compositing, blend modes (#32) and 31 common effects (GPU2): colour adjustments, crop/flip/transform, blurs, sharpen. Lumetri, keys, masks, the remaining effects, export and encoding are CPU. Linux and Windows have no hardware path. 8K and AV1 aren't real-time. The effect list shows Premiere's "GPU accelerated" badge, but only 31 effect ids run on the GPU here; the rest, Lumetri included, run on the CPU. |
|
||||
| **Stability** | not tracked | **improving, unproven** | The never-crash pass (no panics in product code, last-resort guards) landed on 2026-10-04. There's no field record from real users' projects yet. |
|
||||
| **Plugins and pro ecosystem** | not tracked | **~0–10%** | No audio plugin hosting (VST3 / Audio Units) and no third-party video effects (OpenFX). For many professional editors this alone rules FilmCraft out. Team Projects and Productions are out of scope by design. |
|
||||
| **AI features** | partial | **~25–35%** | Speech to text exists but is off in default builds (the `whisper` feature). Enhance Speech is a DSP chain, not a model. Auto Reframe is approximate. No Generative Extend, and no media-intelligence search or auto colour. |
|
||||
| **Platforms** | not tracked | **macOS solid; Windows and Linux thin** | Development and testing happen on macOS. Windows and Linux are built for releases but barely exercised at runtime. The web build is a compile target; the browser app shell isn't finished. |
|
||||
| **UI fidelity** | ~85% | **~80%** | Agents compare with Premiere reference screenshots; this hasn't been checked independently. |
|
||||
|
||||
### Where we are lacking
|
||||
|
||||
In priority order. Agents should prefer this work over adding more checklist items.
|
||||
|
||||
1. **Hardware acceleration** (#30), the most visible gap to users:
|
||||
- hardware decode on Linux (VA-API) and Windows (Media Foundation);
|
||||
- zero-copy decoded frames into wgpu;
|
||||
- the remaining effects on the GPU: Lumetri, keys, Vignette, Video Limiter, masks (31 common effects are done);
|
||||
- GPU export, then hardware encode (H.264 / HEVC).
|
||||
2. **A measured parity number.** Add `cargo xtask parity`, run in CI. It should cover menus,
|
||||
commands, effects, transitions, panels, preferences and formats against the Premiere snapshot in
|
||||
`plan/premiere/`. It should report presence and fidelity separately (exact, approximate, stub).
|
||||
3. **Real-world media corpus.** Phone, camera and screen recordings, variable frame rate, damaged
|
||||
files, long GOPs, odd containers. Each gets an import → edit → export round-trip test with an
|
||||
ffprobe / ffmpeg oracle.
|
||||
4. **Workflow acceptance tests.** Scripted MCP sessions that do real jobs end to end: assemble and
|
||||
trim a scene, colour it, mix and loudness-normalise it, caption it, deliver it, round-trip
|
||||
through XML / AAF.
|
||||
5. **Windows and Linux at runtime.** CI that runs the test suite and a headless playback/export
|
||||
smoke test on both, not just release builds. Fix the Linux audio path (#23).
|
||||
6. **Delivery codecs.** HEVC and AV1 export: hardware encoders first, and `rav1e` (BSD-2) for AV1.
|
||||
7. **Plugin hosting** (VST3 / Audio Units, then OpenFX). Needs FFI, so it falls under the isolated
|
||||
`unsafe` crate rule (AGENTS.md §0.3). **Owner decision needed** before starting.
|
||||
8. **AI features** with openly licensed local models: transcription on by default, text-based
|
||||
editing, speech enhancement, reframing.
|
||||
9. **The web app shell**: file access, WebCodecs, audio.
|
||||
|
||||
### Where we're going
|
||||
|
||||
**Next:** close items 1–5. Hardware acceleration fixes the gap users notice first (CPU-bound 4K playback and export). The
|
||||
parity tool, media corpus and workflow tests make the progress number trustworthy. Windows and
|
||||
Linux CI makes "runs everywhere" true.
|
||||
|
||||
**After that:** delivery codecs, plugin hosting (pending the owner's decision) and AI features,
|
||||
measured with the same tools.
|
||||
|
||||
**Goal:** "ready for real work" at 85%+, measured rather than estimated, on all three desktop
|
||||
platforms.
|
||||
|
||||
## Feature checklist scorecard
|
||||
|
||||
Measured against Premiere Pro 26.5 on this machine. Menu items: the native menu bar dump, minus
|
||||
Adobe-cloud-only items (Team Projects, Productions, Firefly, Stock, Dynamic Link, account/help pages),
|
||||
@@ -22,11 +100,14 @@ matched against `filmcraft-cli commands` and the UI command table. Effects: the
|
||||
| Graphics and captions | 8% | text engine, shapes, per-character styles, our own graphics templates (.fcgt, 8 built-ins, export/install/edit), rolls/crawls, responsive pins + time, captions SRT/VTT/SCC/MCC/STL/TTML/DFXP, transcripts | ~85% |
|
||||
| Export | 8% | own H.264/AAC, ProRes, DNxHR, MXF OP1a/OP-Atom, image sequences, GIF, WAV, 5.1; preset library + user presets, full Export-mode settings, queue, Quick Export; AAF (Edit Protocol) import/export, OMF 2.0 export; no AAF/OMF validation against Avid/Pro Tools yet | ~92% |
|
||||
| Preferences and project management | 5% | Settings dialog with 16 categories (most settings wired), project settings, scratch disks, search bins, templates | ~80% |
|
||||
| Performance | 5% | 1080p real-time, 3×1080p; 4K H.264 real-time at load ≤140 (−23% decoder cycles, opt-in draft decoding at 1/2–1/4); HEVC 2.5× cheaper, catch-up decoding, `cargo xtask bench` + `perf.stats`; 8K and AV1 not yet real-time on a loaded machine | ~62% |
|
||||
| Performance | 5% | 1080p real-time, 3×1080p; 4K H.264 real-time at load ≤140 (−23% decoder cycles, opt-in draft decoding at 1/2–1/4); HEVC 2.5× cheaper, catch-up decoding, `cargo xtask bench` + `perf.stats`; blend modes on the GPU (#32); 8K and AV1 not yet real-time on a loaded machine; no hardware path on Windows/Linux (see the honest assessment: ~35–40% against Premiere) | ~62% |
|
||||
| **Weighted total** | | | **~87%** |
|
||||
|
||||
## Estimate to parity
|
||||
|
||||
This estimate is for the **feature checklist**. Closing the honest-assessment gaps (above) is the
|
||||
"Robust on real-world material" row, plus plugins and AI features, which it doesn't include.
|
||||
|
||||
Measured throughput in the last work block (2026-10-01 night → 10-02): five Opus 5.5 agents in
|
||||
parallel for ~4.5 wall-clock hours (~18 agent-hours including integration) moved parity by ~4 points
|
||||
(≈4–5 agent-hours per point), on a machine that was heavily overloaded (load 150–300 on 14 cores) and
|
||||
@@ -62,7 +143,7 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining
|
||||
| M2 | H.264 decoder | ✅ | Own decoder, bit-exact on 37+ streams, 500–600 fps 1080p | — | — |
|
||||
| M3 | Editing core | 🟡 | Edit algebra (insert/overwrite/razor/lift/extract/ripple/roll/slip/slide/rate-stretch/nest/paste), tools, markers, trim mode + Trim Monitor + dynamic J/K/L trimming, Keyboard Shortcuts editor with FilmCraft/Premiere/FCP/Avid presets; Edit/Clip/File menu commands (Label colours and Select Label Group, Paste/Remove Attributes, Select All Matching, Remove Unused, Consolidate Duplicates, Sequence From Clip, Bin From Selection, Offline File, Close Project, Make/Edit Subclip, Modify Audio Channels/Timecode, Frame Hold Options/Add Frame Hold/Insert Frame Hold Segment, Time Interpolation with frame blending, Fit/Fill frame, Breakout to Mono, Extract Audio, Replace With Clip; M3.11: Scene Edit Detection (pure-Rust cut detection, background job), Normalize Mix Track, Simplify Sequence, Transcribe Sequence, Find/Find Next and search bins, Automate to Sequence, Edit Original, Edit Offline, Source Settings, Update Metadata (XMP), Generate Audio Waveform, Project Settings General/Scratch Disks, Get Media File Properties, Save as Template, Selection as FilmCraft Project, Avid Log Exchange export, Flash Cue markers, Dynamic Audio Waveforms, Reveal Log Files, System Compatibility Report); M3.12 keyboard parity: Premiere's default keyboard (~115 keyboard-only commands: edit-point navigation on targeted/any track, select clip at playhead/next/previous, extend edit to playhead, nudge/slip/slide selection, target and source-patch toggles, clip volume ±1 dB/many, frame maximize/full screen/panel cycling, monitor zoom, track heights, Project and Text panel keyboard navigation, text size/leading/alignment, Export Frame, poster frames; see docs/keyboard.md); multicam (Create Multi-Camera Source Sequence, Multi-Camera view with live switching on 1–9, angle switching, Enable/Flatten, Edit Cameras) and sync (Synchronize, Merge Clips; In/Out/timecode/marker/audio — GCC-PHAT, sample-accurate) | Multicam paging >16 angles and grid thumbnails, optical flow (renders as frame blending), 39 Premiere default shortcuts skipped with reasons in docs/keyboard.md (Productions, AI/cloud tools, work area bar, Production/Search panels) | 6–10 |
|
||||
| M4 | Playback | 🟡 | Audio-clock master, prefetch with cancellation, J/K/L, correct dropped-frame stats, playback resolution, render bar + content-hashed render previews, App Nap opt-out; 1080p H.264 and 3 stacked 1080p streams play with 0 dropped frames | 4K under load, 8K, frame-threaded AV1 decode, reduced-resolution decode for multicam grids | 6–10 |
|
||||
| M5 | Effects, keyframes, GPU | 🟡 | ~60 CPU effects, 30 transitions, keyframes + value/velocity graphs, wgpu compositor, effect + opacity masks (ellipse/polygon/pen, feather, expansion, modes; CPU/WGSL parity), mask tracking (Lucas–Kanade + RANSAC), adjustment layers, effect presets (built-in + user, JSON import/export) | Full ~150-effect catalogue, WGSL parity for all effects and masks in the live GPU path, Warp Stabilizer, Morph Cut | 20–30 |
|
||||
| M5 | Effects, keyframes, GPU | 🟡 | All 93 Premiere 26 video effects and 84 transitions (CPU), blend modes and 31 common effects on the GPU (#32, GPU2), keyframes + value/velocity graphs, wgpu compositor, effect + opacity masks (ellipse/polygon/pen, feather, expansion, modes; CPU/WGSL parity), mask tracking (Lucas–Kanade + RANSAC), adjustment layers, effect presets (built-in + user, JSON import/export) | Effects on the GPU (in progress, #30), GPU masks in the live path, GPU export, exact Warp Stabilizer (3-D) and Morph Cut | 20–30 |
|
||||
| M6 | Export | ✅ | Own H.264 encoder (High/Main/Baseline, B-frames, VBR/CBR/2-pass) → MP4 or QuickTime + own AAC; ProRes, DNxHR, MJPEG, PNG/TIFF/BMP sequences (Premiere-style numbering), GIF, WAV, AIFF; background jobs; M6.5: Export mode parity (frame size / rate / scaling / pixel aspect, profile / level, CBR / VBR 1- and 2-pass, keyframe distance, audio codec / rate / channels / bitrate / sample size, multiplexer, burn-in or sidecar captions, image / name / timecode overlays, video limiter, loudness normalization with true-peak limiter, metadata, ranges, estimated size, summary), 24 built-in presets + user presets with favourites and import/export (Preset Manager), export queue (reorder, cancel, retry, several sequences / ranges), Quick Export; `export.*` commands and `filmcraft-cli export --preset` | Interlaced encoding, 5.1 audio, smart render, publishing destinations | 6–8 |
|
||||
| M7 | Audio | 🟡 | Mixer graph (tracks → submixes → Mix, pre/post-fader inserts and sends, latency-compensated, sample-accurate, ~6× realtime for 24 tracks × 3 effects on one core), Audio Track Mixer + Audio Clip Mixer panels, track automation (Off/Read/Latch/Touch/Write, recorded live while playing, thinned to keyframes, timeline lanes with pen editing), solo/solo-safe, channel mapping basics, peak + BS.1770 loudness meters (match ffmpeg), DSP crate with 16 clip/track effects, Audio Gain (set/adjust/normalize), Constant Power / Constant Gain / Exponential Fade, Essential Sound (types, Loudness auto-match, Repair incl. DeEss/DeReverb, Clarity, Creative, Ducking keyframes, presets; full Dialogue chain 22× realtime) | Music duration remix, ML speech enhancement, 5.1 panner and multichannel buses, voice-over record, effect editor windows (EQ curve), remaining effects (multiband, convolution reverb), clip-mixer automation recording | 6–9 |
|
||||
| M8 | Colour | 🟡 | Lumetri: basic, creative + looks, RGB & hue curves, wheels, HSL secondary, vignette, section bypass; Input LUT / Look LUT (.cube 1D/3D/shaper, .3dl; tetrahedral CPU + WGSL; project LUT library; built-in camera conversions); Colour Match (Oklab tonal-range statistics, skin protection, solved in Lumetri wheels); colour management: Rec. 709 / Rec. 2100 PQ / HLG working spaces + wide gamut, Interpret Footage colour space (S-Log3, V-Log, Canon Log 2/3, LogC3/4, Apple Log, D-Log from published specs), metadata auto-detect, BT.2390 tone mapping, gamut mapping, HDR export signalling (VUI/colr/mdcv/clli/SEI, ffprobe-verified); Lumetri Scopes: Vectorscope YUV (75/100 % targets, skin-tone line) and HLS, Histogram, Parade (RGB/YUV/RGB-White), Waveform (RGB/Luma/YC/YC no Chroma), presets, Rec. 601/709/2100, 8-bit/float/HDR (cd/m²) scales, multi-scope grid, `scopes.read` | HDR-aware Lumetri maths, macOS EDR monitors, mastering metadata → tone-map peak, D-Log M (no published formula), HSL Secondary refine | 3–5 |
|
||||
@@ -73,10 +154,13 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining
|
||||
|
||||
## Running now
|
||||
|
||||
- Nothing; next: remaining Premiere menu items (Settings categories, Scene Edit Detection, Normalize Mix Track, OMF/AAF, Find), AV1 single-thread re-measure on an idle machine
|
||||
- **Hardware acceleration (#30):** landed: blend modes on the GPU (#32); VideoToolbox hardware decode + the `platform` FFI crate (#33); 31 common effects on the GPU (GPU2). Next: VA-API (Linux) and Media Foundation (Windows) decode, zero-copy upload, Lumetri on the GPU.
|
||||
- **Next:** [Where we are lacking](#where-we-are-lacking) items 1–5. GPU export waits for the export-crate work (#19 and the never-crash follow-up) to land.
|
||||
|
||||
## Log
|
||||
|
||||
- **2026-10-05:** honest assessment added (checklist ~87% vs ready for real work ~50–60%). Hardware acceleration started (#30): blend modes on the GPU (#32); VideoToolbox decode + `platform` crate, the one crate allowed `unsafe` (#33). Community fixes landed: AIFF import, export start/end ranges, transitions following ripple trims, Slide moving linked audio (#5–#8).
|
||||
|
||||
- **2026-10-04 (later):** interchange + MXF export (M11.4/M6.6): own compound-file container (`crates/cfb`), AAF Edit Protocol import/export (embedded/linked/consolidated audio, video mixdown, handles, breakout to mono), OMF 2.0 export (Bento), MXF OP1a/OP-Atom writer (DNxHR/ProRes/H.264/PCM) as export formats. 1605 tests.
|
||||
|
||||
- **2026-10-04:** MPEG-2 (M9.15): own MPEG-2/MPEG-1 video decoder (4:2:2, interlaced, field pictures; IEEE 1180 IDCT), MPEG TS/PS demux (AVCHD .mts, .m2ts, .ts, .mpg/.vob/.mod), own AC-3 decoder (ATSC A/52), MP2, LATM AAC, Blu-ray/DVD LPCM, MPEG-2 in MXF/MOV/MP4/MKV. 1562 tests.
|
||||
|
||||
@@ -21,6 +21,7 @@ whisper = ["filmcraft-engine/whisper", "filmcraft-engine/speech-download"]
|
||||
filmcraft-engine = { workspace = true }
|
||||
filmcraft-media = { workspace = true }
|
||||
filmcraft-codecs = { workspace = true }
|
||||
filmcraft-platform = { workspace = true }
|
||||
filmcraft-mxf = { workspace = true }
|
||||
filmcraft-mpegts = { workspace = true }
|
||||
filmcraft-ogg = { workspace = true }
|
||||
|
||||
@@ -88,7 +88,9 @@ fn insert_dotted(m: &mut Map<String, Value>, key: &str, v: Value) {
|
||||
if !e.is_object() {
|
||||
*e = Value::Object(Map::new());
|
||||
}
|
||||
insert_dotted(e.as_object_mut().expect("object"), tail, v);
|
||||
if let Value::Object(child) = e {
|
||||
insert_dotted(child, tail, v);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
//!
|
||||
//! Run `filmcraft-cli help` for the full reference (also in docs/agents.md).
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
mod args;
|
||||
mod probe;
|
||||
|
||||
@@ -153,6 +155,8 @@ async fn main() {
|
||||
println!("filmcraft-cli {}", env!("CARGO_PKG_VERSION"));
|
||||
return;
|
||||
}
|
||||
// OS hardware video decoders (VideoToolbox on macOS) in front of our own, as in the desktop app.
|
||||
let _ = filmcraft_platform::register();
|
||||
let a = Args::parse(std::env::args().skip(1));
|
||||
let Some(cmd) = a.pos(0) else { usage("missing subcommand") };
|
||||
match cmd {
|
||||
@@ -339,7 +343,7 @@ async fn main() {
|
||||
if a.opt("--bridge").is_some() {
|
||||
usage("render is headless; with --bridge use the MCP `render_frame` tool or `exec` ui commands");
|
||||
}
|
||||
let Backend::Local(mut s) = Backend::open(&a) else { unreachable!() };
|
||||
let Backend::Local(mut s) = Backend::open(&a) else { usage("render is headless") };
|
||||
let secs: f64 = a.opt("--seconds").and_then(|v| v.parse().ok()).unwrap_or(0.0);
|
||||
let scale: f32 = a.opt("--scale").and_then(|v| v.parse().ok()).unwrap_or(1.0);
|
||||
let out = a.opt("--out").unwrap_or("frame.png");
|
||||
@@ -356,7 +360,7 @@ async fn main() {
|
||||
Some(addr) => filmcraft_automation::FilmcraftMcp::bridge(addr).unwrap_or_else(|e| fail(e)),
|
||||
None => match Backend::open(&a) {
|
||||
Backend::Local(s) => filmcraft_automation::FilmcraftMcp::headless(*s),
|
||||
Backend::Bridge(_) => unreachable!(),
|
||||
Backend::Bridge(_) => usage("mcp: use --bridge ADDR to drive a running app"),
|
||||
},
|
||||
};
|
||||
if let Err(e) = server.serve_stdio().await {
|
||||
|
||||
@@ -289,20 +289,25 @@ impl ByteReader for BlobReader {
|
||||
}
|
||||
return Err(filmcraft_media::pending::would_block());
|
||||
}
|
||||
FS.with(|fs| {
|
||||
let complete = FS.with(|fs| {
|
||||
let mut fs = fs.borrow_mut();
|
||||
fs.clock += 1;
|
||||
let t = fs.clock;
|
||||
let mut pos = 0usize;
|
||||
for c in c0..=c1 {
|
||||
let (d, used) = fs.chunks.get_mut(&(self.id, c)).expect("checked above");
|
||||
// Present (checked above); evicted in between would be a bug, reported as an error.
|
||||
let Some((d, used)) = fs.chunks.get_mut(&(self.id, c)) else { return false };
|
||||
*used = t;
|
||||
let a = if c == c0 { (offset - c * CHUNK) as usize } else { 0 };
|
||||
let n = (d.len() - a).min(buf.len() - pos);
|
||||
buf[pos..pos + n].copy_from_slice(&d[a..a + n]);
|
||||
pos += n;
|
||||
}
|
||||
true
|
||||
});
|
||||
if !complete {
|
||||
return Err(filmcraft_media::pending::would_block());
|
||||
}
|
||||
// read-ahead for sequential access
|
||||
for c in c1 + 1..=(c1 + READ_AHEAD).min(last) {
|
||||
request(self.id, c);
|
||||
|
||||
@@ -16,6 +16,7 @@
|
||||
//!
|
||||
//! Everything here is `wasm32`-only; on other targets the crate is empty.
|
||||
#![cfg(target_arch = "wasm32")]
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
pub mod api;
|
||||
pub mod audio;
|
||||
|
||||
@@ -463,7 +463,7 @@ impl WcSource {
|
||||
c.frames.insert(pts, Arc::new(f));
|
||||
}
|
||||
while c.frames.len() > CACHE_FRAMES {
|
||||
let far = c.frames.keys().copied().max_by_key(|p| (p - want).abs()).expect("non-empty");
|
||||
let Some(far) = c.frames.keys().copied().max_by_key(|p| (p - want).abs()) else { break };
|
||||
c.frames.remove(&far);
|
||||
}
|
||||
// the session's own frame came out: other requests may restart it now
|
||||
|
||||
@@ -18,6 +18,7 @@ whisper = ["filmcraft-engine/whisper", "filmcraft-engine/speech-download"]
|
||||
[dependencies]
|
||||
filmcraft-engine = { workspace = true }
|
||||
filmcraft-ui-egui = { workspace = true }
|
||||
filmcraft-platform = { workspace = true }
|
||||
egui = { workspace = true }
|
||||
eframe = { workspace = true, features = ["default_fonts", "wgpu", "accesskit", "wayland", "x11"] }
|
||||
serde_json = { workspace = true }
|
||||
|
||||
@@ -14,6 +14,8 @@
|
||||
//! with unsaved changes the app asks to recover them; `--recover` recovers the newest without
|
||||
//! asking, `--no-recover` starts without asking (the changes stay available via File ▸ Recover
|
||||
//! Unsaved Changes…).
|
||||
#![cfg_attr(all(target_os = "windows", not(debug_assertions)), windows_subsystem = "windows")]
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
mod app_nap;
|
||||
mod audio;
|
||||
@@ -71,6 +73,9 @@ fn main() -> eframe::Result {
|
||||
// Panics anywhere go to <data dir>/Logs/crash-<day>.log with a backtrace; the UI pass and
|
||||
// frame workers catch them and keep running (see filmcraft_ui_egui::crash).
|
||||
filmcraft_ui_egui::crash::install(data_dir.clone().or_else(default_data_dir).map(|d| d.join("Logs")));
|
||||
// OS hardware video decoders (VideoToolbox on macOS) in front of our own; Settings ▸ Playback ▸
|
||||
// Hardware decoding switches them off. Unsupported streams and failures use our decoders.
|
||||
log::info!("hardware decoding: {:?}", filmcraft_platform::register());
|
||||
let options = eframe::NativeOptions {
|
||||
viewport: egui::ViewportBuilder::default()
|
||||
.with_title("FilmCraft")
|
||||
|
||||
@@ -31,6 +31,8 @@
|
||||
//! # Ok::<(), filmcraft_aac::Error>(())
|
||||
//! ```
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
mod config;
|
||||
mod decoder;
|
||||
mod encoder;
|
||||
|
||||
@@ -13,6 +13,8 @@
|
||||
//!
|
||||
//! E-AC-3 (bsid 11-16) is recognised and refused.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
mod tables;
|
||||
|
||||
use std::sync::OnceLock;
|
||||
@@ -359,8 +361,7 @@ fn bit_allocation(exp: &[u8; 256], p: &AllocParams, g: &Globals, bap: &mut [u8;
|
||||
let bndend = tables::MASKTAB[end - 1] as usize + 1;
|
||||
let mut excite = [0i32; 50];
|
||||
let (mut fastleak, mut slowleak) = p.leak.unwrap_or((0, 0));
|
||||
let begin;
|
||||
if bndstrt == 0 {
|
||||
let begin = if bndstrt == 0 {
|
||||
// the last band of the LFE channel (bndend 7) has no band above it
|
||||
let lfe_last = |bin: usize| !(bndend == 7 && bin == 6);
|
||||
let mut lowcomp = 0;
|
||||
@@ -391,10 +392,10 @@ fn bit_allocation(exp: &[u8; 256], p: &AllocParams, g: &Globals, bap: &mut [u8;
|
||||
slowleak = slowleak.max(bndpsd[bin] - sgain);
|
||||
excite[bin] = (fastleak - lowcomp).max(slowleak);
|
||||
}
|
||||
begin = 22;
|
||||
22
|
||||
} else {
|
||||
begin = bndstrt;
|
||||
}
|
||||
bndstrt
|
||||
};
|
||||
for bin in begin..bndend {
|
||||
fastleak -= fdecay;
|
||||
fastleak = fastleak.max(bndpsd[bin] - p.fgain);
|
||||
@@ -1030,7 +1031,7 @@ impl Decoder {
|
||||
/// Coded channel order (Table 5.8) → WAV / SMPTE order: fronts L R C, LFE, surrounds.
|
||||
fn wav_order(mut pcm: Vec<Vec<f32>>, acmod: usize, lfeon: bool) -> Vec<Vec<f32>> {
|
||||
let nf = NFCHANS[acmod];
|
||||
let lfe = lfeon.then(|| pcm.pop().expect("lfe"));
|
||||
let lfe = if lfeon { pcm.pop() } else { None };
|
||||
let mut fbw = pcm;
|
||||
// acmod 3, 5, 7 code L C R …: move C after R
|
||||
if acmod & 1 == 1 && acmod != 1 {
|
||||
|
||||
@@ -99,7 +99,7 @@ fn ffmpeg_encoded_streams_decode() {
|
||||
}
|
||||
}
|
||||
let o = Command::new(&ff).args(["-v", "error", "-i"]).arg(&f).args(["-f", "f32le", "-"]).output().unwrap();
|
||||
let want: Vec<f32> = o.stdout.chunks_exact(4).map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
|
||||
let want: Vec<f32> = o.stdout.as_chunks::<4>().0.iter().map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
|
||||
let ch = s.channels as usize;
|
||||
let n = want.len() / ch;
|
||||
assert_eq!(n, ours[0].len(), "{}: sample count", s.name);
|
||||
|
||||
@@ -123,7 +123,7 @@ pub fn matrix(from: Layout, to: Layout, mixdown: Mixdown) -> Vec<Vec<f32>> {
|
||||
let (ni, no) = (from.channels(), to.channels());
|
||||
let mut m = vec![vec![0.0f32; ni]; no];
|
||||
match (from, to) {
|
||||
(a, b) if a == b => {
|
||||
(Layout::Mono, Layout::Mono) | (Layout::Stereo, Layout::Stereo) | (Layout::Surround51, Layout::Surround51) => {
|
||||
for (i, row) in m.iter_mut().enumerate() {
|
||||
row[i] = 1.0;
|
||||
}
|
||||
@@ -159,7 +159,6 @@ pub fn matrix(from: Layout, to: Layout, mixdown: Mixdown) -> Vec<Vec<f32>> {
|
||||
m[1][LFE] = K;
|
||||
}
|
||||
}
|
||||
_ => unreachable!("all layout pairs are covered"),
|
||||
}
|
||||
m
|
||||
}
|
||||
|
||||
@@ -16,6 +16,8 @@
|
||||
//! denormals never appear (no FTZ/DAZ CPU flags needed, which keeps the crate `unsafe`-free and
|
||||
//! portable to wasm).
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
pub mod biquad;
|
||||
pub mod channels;
|
||||
pub mod design;
|
||||
@@ -169,12 +171,11 @@ impl ParamValues {
|
||||
pub(crate) fn get(&self, id: &str) -> Option<f32> {
|
||||
self.index_of(id).map(|i| self.values[i])
|
||||
}
|
||||
/// Value of a parameter that is known to exist (panics otherwise — programming error).
|
||||
/// Value of a parameter that is known to exist (0 otherwise — a programming error caught by
|
||||
/// debug builds).
|
||||
pub(crate) fn v(&self, id: &str) -> f32 {
|
||||
match self.get(id) {
|
||||
Some(v) => v,
|
||||
None => panic!("unknown parameter {id}"),
|
||||
}
|
||||
debug_assert!(self.get(id).is_some(), "unknown parameter {id}");
|
||||
self.get(id).unwrap_or(0.0)
|
||||
}
|
||||
pub(crate) fn on(&self, id: &str) -> bool {
|
||||
self.v(id) >= 0.5
|
||||
|
||||
@@ -42,7 +42,9 @@ impl BridgeClient {
|
||||
*guard = Some((BufReader::new(r), w));
|
||||
}
|
||||
let id = self.next_id.fetch_add(1, Ordering::Relaxed);
|
||||
let conn = guard.as_mut().expect("connected");
|
||||
let Some(conn) = guard.as_mut() else {
|
||||
return Err(AutomationError::Bridge(format!("not connected to {}", self.addr)));
|
||||
};
|
||||
let line = format!("{}\n", json!({"id": id, "method": method, "params": params}));
|
||||
let res: Result<Value, AutomationError> = async {
|
||||
conn.1.write_all(line.as_bytes()).await.map_err(|e| AutomationError::Bridge(e.to_string()))?;
|
||||
|
||||
@@ -8,7 +8,10 @@
|
||||
//! (`filmcraft --control <port>`), so agents can also inspect, screenshot, click, drag, scroll
|
||||
//! and type in the live UI — every menu, panel, timeline gesture and keystroke.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
pub mod bridge;
|
||||
pub mod long_job;
|
||||
pub mod server;
|
||||
|
||||
pub use bridge::BridgeClient;
|
||||
|
||||
@@ -0,0 +1,183 @@
|
||||
//! Long exports over MCP (docs/agents.md § Long exports): `file.exportMedia` with `wait: true`
|
||||
//! runs as a background engine job; the server reports its progress as MCP
|
||||
//! `notifications/progress` (when the request carried a `progressToken`) and stops it on
|
||||
//! `notifications/cancelled`, deleting the partial output. The session lock is only held for
|
||||
//! short polls, so other requests are answered while the export runs.
|
||||
|
||||
use std::path::Path;
|
||||
use std::time::{Duration, SystemTime};
|
||||
|
||||
use rmcp::RoleServer;
|
||||
use rmcp::model::{CallToolResult, ContentBlock as Content, ProgressNotificationParam, ProgressToken};
|
||||
use rmcp::service::{Peer, RequestContext};
|
||||
use serde_json::{Value, json};
|
||||
|
||||
use crate::server::FilmcraftMcp;
|
||||
|
||||
/// Engine commands that block until a render is written when called with `wait: true`.
|
||||
pub const LONG_COMMANDS: &[&str] = &["file.exportMedia"];
|
||||
|
||||
/// How often the job is polled (and at most how often progress is reported).
|
||||
const POLL: Duration = Duration::from_millis(100);
|
||||
|
||||
/// Whether `command_run {id, params}` is a long call this module runs.
|
||||
pub fn is_long(id: &str, params: &Value) -> bool {
|
||||
LONG_COMMANDS.contains(&id) && params.get("wait").and_then(Value::as_bool) == Some(true)
|
||||
}
|
||||
|
||||
/// Sends `notifications/progress` for one request; progress only ever increases.
|
||||
struct Reporter {
|
||||
peer: Peer<RoleServer>,
|
||||
token: Option<ProgressToken>,
|
||||
last: f64,
|
||||
}
|
||||
|
||||
impl Reporter {
|
||||
async fn report(&mut self, done: f64, total: f64, message: &str) {
|
||||
let Some(token) = self.token.clone() else { return };
|
||||
if total <= 0.0 || done <= self.last {
|
||||
return;
|
||||
}
|
||||
self.last = done;
|
||||
let mut p = ProgressNotificationParam::new(token, done).with_total(total);
|
||||
if !message.is_empty() {
|
||||
p = p.with_message(message);
|
||||
}
|
||||
let _ = self.peer.notify_progress(p).await;
|
||||
}
|
||||
}
|
||||
|
||||
impl FilmcraftMcp {
|
||||
/// Run `id` (a [`LONG_COMMANDS`] entry, `wait: true`) as a job, reporting progress and
|
||||
/// honouring cancellation.
|
||||
pub(crate) async fn run_long(&self, id: &str, mut params: Value, context: &RequestContext<RoleServer>) -> CallToolResult {
|
||||
let started = SystemTime::now();
|
||||
if let Some(p) = params.as_object_mut() {
|
||||
p.insert("wait".into(), json!(false));
|
||||
}
|
||||
let start = match self.run(id, params).await {
|
||||
Ok(v) => v,
|
||||
Err(e) => return CallToolResult::error(vec![Content::text(e.to_string())]),
|
||||
};
|
||||
let Some(job) = start.get("job").and_then(Value::as_u64) else {
|
||||
return CallToolResult::error(vec![Content::text(format!("{id} started no job: {start}"))]);
|
||||
};
|
||||
let path = start.get("path").and_then(Value::as_str).unwrap_or_default().to_string();
|
||||
let mut rep = Reporter { peer: context.peer.clone(), token: context.meta.get_progress_token(), last: 0.0 };
|
||||
loop {
|
||||
let state = self.job_state(job).await;
|
||||
let finished = state.as_ref().is_none_or(|j| j["finished"].as_bool() == Some(true));
|
||||
if let Some(j) = &state {
|
||||
let (done, total) = (j["done"].as_f64().unwrap_or(0.0), j["total"].as_f64().unwrap_or(0.0));
|
||||
rep.report(done, total, j["status"].as_str().unwrap_or_default()).await;
|
||||
}
|
||||
if finished {
|
||||
let result = state.map(|j| j["result"].clone()).unwrap_or(Value::Null);
|
||||
if let Some(e) = result.get("error").and_then(Value::as_str) {
|
||||
return CallToolResult::error(vec![Content::text(format!("export failed: {e}"))]);
|
||||
}
|
||||
let mut out = start;
|
||||
if let Some(o) = out.as_object_mut() {
|
||||
o.insert("result".into(), result);
|
||||
}
|
||||
return CallToolResult::success(vec![Content::text(serde_json::to_string_pretty(&out).unwrap_or_default())]);
|
||||
}
|
||||
tokio::select! {
|
||||
_ = context.ct.cancelled() => break,
|
||||
_ = tokio::time::sleep(POLL) => {}
|
||||
}
|
||||
}
|
||||
// Cancelled: stop the encode at the next batch, wait for the worker, delete the partial file.
|
||||
let _ = self.run("jobs.cancel", json!({"job": job})).await;
|
||||
for _ in 0..600 {
|
||||
if self.job_state(job).await.is_none_or(|j| j["finished"].as_bool() == Some(true)) {
|
||||
break;
|
||||
}
|
||||
tokio::time::sleep(POLL).await;
|
||||
}
|
||||
remove_partial(&path, started);
|
||||
CallToolResult::error(vec![Content::text("cancelled")])
|
||||
}
|
||||
|
||||
/// The job's `jobs.list` entry.
|
||||
async fn job_state(&self, job: u64) -> Option<Value> {
|
||||
let jobs = self.run("jobs.list", json!({})).await.ok()?;
|
||||
jobs.as_array()?.iter().find(|j| j["id"].as_u64() == Some(job)).cloned()
|
||||
}
|
||||
}
|
||||
|
||||
/// Delete what an interrupted export wrote, and nothing else: the output file itself, the numbered
|
||||
/// stills of an image sequence (`<stem><digits>.<same extension>`) and the caption sidecar
|
||||
/// (`<stem>.srt` / `<stem>.vtt`), each only if it was written since `since`. Other files that
|
||||
/// merely share the name's stem (`<stem>.aep`, `<stem>2.psd`, …) are never touched, even when they
|
||||
/// were saved during the export.
|
||||
pub fn remove_partial(path: &str, since: SystemTime) {
|
||||
let p = Path::new(path);
|
||||
let (Some(dir), Some(stem), Some(file)) = (p.parent(), p.file_stem().and_then(|s| s.to_str()), p.file_name().and_then(|s| s.to_str())) else {
|
||||
return;
|
||||
};
|
||||
let ext = p.extension().and_then(|s| s.to_str()).unwrap_or_default();
|
||||
let dir = if dir.as_os_str().is_empty() { Path::new(".") } else { dir };
|
||||
let Ok(entries) = std::fs::read_dir(dir) else { return };
|
||||
// file times are coarser than the clock
|
||||
let since = since.checked_sub(Duration::from_secs(1)).unwrap_or(since);
|
||||
for e in entries.flatten() {
|
||||
let name = e.file_name();
|
||||
let Some(name) = name.to_str() else { continue };
|
||||
if is_export_output(name, file, stem, ext) && e.metadata().is_ok_and(|m| m.is_file() && m.modified().is_ok_and(|t| t >= since)) {
|
||||
let _ = std::fs::remove_file(e.path());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether `name` is a file an export to `file` (`stem` + `.ext`) writes.
|
||||
fn is_export_output(name: &str, file: &str, stem: &str, ext: &str) -> bool {
|
||||
if name == file {
|
||||
return true;
|
||||
}
|
||||
let Some(rest) = name.strip_prefix(stem) else { return false };
|
||||
if let Some(sidecar) = rest.strip_prefix('.') {
|
||||
return sidecar.eq_ignore_ascii_case("srt") || sidecar.eq_ignore_ascii_case("vtt");
|
||||
}
|
||||
// image sequence: one or more digits, then the output's own extension
|
||||
let digits = rest.len() - rest.trim_start_matches(|c: char| c.is_ascii_digit()).len();
|
||||
digits > 0 && !ext.is_empty() && rest.get(digits..).is_some_and(|r| r.strip_prefix('.').is_some_and(|x| x == ext))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn only_the_exports_own_files_are_removed() {
|
||||
let dir = std::env::temp_dir().join(format!("fc-remove-partial-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
let started = SystemTime::now();
|
||||
let ours = ["trailer.png", "trailer000.png", "trailer123.png", "trailer.srt", "trailer.vtt"];
|
||||
// same stem, written during the export, but not the export's
|
||||
let theirs = ["trailer.aep", "trailer2.psd", "trailer.docx", "trailer000.jpg", "trailers.png", "trailer.png.bak", "other.png"];
|
||||
for f in ours.iter().chain(theirs.iter()) {
|
||||
std::fs::write(dir.join(f), b"x").unwrap();
|
||||
}
|
||||
remove_partial(&dir.join("trailer.png").to_string_lossy(), started);
|
||||
for f in ours {
|
||||
assert!(!dir.join(f).exists(), "{f} should be removed");
|
||||
}
|
||||
for f in theirs {
|
||||
assert!(dir.join(f).exists(), "{f} must not be removed");
|
||||
}
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn files_older_than_the_export_are_kept() {
|
||||
let dir = std::env::temp_dir().join(format!("fc-remove-partial-old-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
std::fs::write(dir.join("clip.mp4"), b"x").unwrap();
|
||||
remove_partial(&dir.join("clip.mp4").to_string_lossy(), SystemTime::now() + Duration::from_secs(60));
|
||||
assert!(dir.join("clip.mp4").exists());
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
}
|
||||
+546
-28
@@ -1,6 +1,6 @@
|
||||
//! The MCP server. Tool names are snake_case.
|
||||
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::sync::{Arc, Mutex, PoisonError};
|
||||
|
||||
use base64::Engine as _;
|
||||
use filmcraft_engine::Session;
|
||||
@@ -19,6 +19,19 @@ pub enum Backend {
|
||||
Bridge(Arc<BridgeClient>),
|
||||
}
|
||||
|
||||
/// A panic payload as text.
|
||||
fn panic_message(p: Box<dyn std::any::Any + Send>) -> String {
|
||||
p.downcast_ref::<&str>().map(|s| s.to_string()).or_else(|| p.downcast_ref::<String>().cloned()).unwrap_or_else(|| "panic".into())
|
||||
}
|
||||
|
||||
/// A failed blocking task: a panic is reported as an internal error and the session stays usable.
|
||||
fn join_error(e: tokio::task::JoinError) -> AutomationError {
|
||||
match e.try_into_panic() {
|
||||
Ok(p) => AutomationError::Other(format!("internal error: {}", panic_message(p))),
|
||||
Err(e) => AutomationError::Other(e.to_string()),
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct FilmcraftMcp {
|
||||
backend: Arc<Backend>,
|
||||
@@ -44,6 +57,24 @@ pub struct RunParams {
|
||||
pub params: Option<Value>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, JsonSchema)]
|
||||
pub struct BatchStep {
|
||||
/// Command id.
|
||||
pub id: String,
|
||||
/// Parameters object.
|
||||
#[serde(default)]
|
||||
pub params: Option<Value>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, JsonSchema)]
|
||||
pub struct BatchParams {
|
||||
/// Commands to run in order.
|
||||
pub steps: Vec<BatchStep>,
|
||||
/// Stop at the first failing step (default true).
|
||||
#[serde(default)]
|
||||
pub stop_on_error: Option<bool>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default, Deserialize, JsonSchema)]
|
||||
pub struct RenderParams {
|
||||
/// Timeline time in seconds (default: the playhead).
|
||||
@@ -148,9 +179,60 @@ impl FilmcraftMcp {
|
||||
}
|
||||
|
||||
pub async fn serve_stdio(self) -> Result<(), AutomationError> {
|
||||
let running = self.serve(rmcp::transport::stdio()).await.map_err(|e| AutomationError::Other(format!("MCP init: {e}")))?;
|
||||
running.waiting().await.map_err(|e| AutomationError::Other(e.to_string()))?;
|
||||
Ok(())
|
||||
self.serve_io(tokio::io::stdin(), tokio::io::stdout()).await
|
||||
}
|
||||
|
||||
/// Serve MCP over any line stream. A line that isn't JSON gets a JSON-RPC parse error
|
||||
/// (`-32700`, id null) instead of being dropped silently; the rest go to the rmcp service. One
|
||||
/// writer task owns `output`, so replies never interleave.
|
||||
pub async fn serve_io(
|
||||
self,
|
||||
input: impl tokio::io::AsyncRead + Unpin + Send + 'static,
|
||||
output: impl tokio::io::AsyncWrite + Unpin + Send + 'static,
|
||||
) -> Result<(), AutomationError> {
|
||||
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
|
||||
let (mut to_service, service_in) = tokio::io::duplex(1 << 20);
|
||||
let (service_out, from_service) = tokio::io::duplex(1 << 20);
|
||||
let (err_tx, mut err_rx) = tokio::sync::mpsc::unbounded_channel::<String>();
|
||||
let reader = tokio::spawn(async move {
|
||||
let mut lines = BufReader::new(input).lines();
|
||||
while let Ok(Some(line)) = lines.next_line().await {
|
||||
if line.trim().is_empty() {
|
||||
continue;
|
||||
}
|
||||
match serde_json::from_str::<Value>(&line) {
|
||||
Ok(_) => {
|
||||
if to_service.write_all(format!("{line}\n").as_bytes()).await.is_err() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
let reply = json!({"jsonrpc": "2.0", "id": null, "error": {"code": -32700, "message": format!("parse error: {e}")}});
|
||||
let _ = err_tx.send(reply.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
// EOF: closing the pipe ends the service
|
||||
let _ = to_service.shutdown().await;
|
||||
});
|
||||
let writer = tokio::spawn(async move {
|
||||
let mut output = output;
|
||||
let mut replies = BufReader::new(from_service).lines();
|
||||
loop {
|
||||
let line = tokio::select! {
|
||||
l = replies.next_line() => match l { Ok(Some(l)) => l, _ => break },
|
||||
Some(e) = err_rx.recv() => e,
|
||||
};
|
||||
if output.write_all(format!("{line}\n").as_bytes()).await.is_err() || output.flush().await.is_err() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
});
|
||||
let running = self.serve((service_in, service_out)).await.map_err(|e| AutomationError::Other(format!("MCP init: {e}")))?;
|
||||
let r = running.waiting().await.map_err(|e| AutomationError::Other(e.to_string()));
|
||||
reader.abort();
|
||||
let _ = writer.await;
|
||||
r.map(|_| ())
|
||||
}
|
||||
|
||||
/// Run an engine command on whichever backend.
|
||||
@@ -160,16 +242,24 @@ impl FilmcraftMcp {
|
||||
let s = s.clone();
|
||||
let id = id.to_string();
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let mut g = s.lock().map_err(|_| AutomationError::Other("session lock poisoned".into()))?;
|
||||
// A command that panicked earlier poisoned the lock; the session is still usable.
|
||||
let mut g = s.lock().unwrap_or_else(PoisonError::into_inner);
|
||||
g.execute(&id, params).map_err(AutomationError::from)
|
||||
})
|
||||
.await
|
||||
.map_err(|e| AutomationError::Other(e.to_string()))?
|
||||
.map_err(join_error)?
|
||||
}
|
||||
Backend::Bridge(b) => b.call("engine.execute", json!({"command": id, "params": params})).await,
|
||||
}
|
||||
}
|
||||
|
||||
/// The project tree plus the active sequence (null when there is none).
|
||||
async fn document(&self) -> Result<Value, AutomationError> {
|
||||
let project = self.run("project.inspect", json!({})).await?;
|
||||
let sequence = self.run("sequence.inspect", json!({})).await.ok();
|
||||
Ok(json!({"project": project, "sequence": sequence}))
|
||||
}
|
||||
|
||||
fn bridge_client(&self) -> Option<Arc<BridgeClient>> {
|
||||
match &*self.backend {
|
||||
Backend::Bridge(b) => Some(b.clone()),
|
||||
@@ -190,7 +280,9 @@ impl FilmcraftMcp {
|
||||
#[tool_router]
|
||||
impl FilmcraftMcp {
|
||||
#[tool(
|
||||
description = "List every command (id, label, menu path, shortcut, parameter doc, enabled now). Menus, shortcuts, panels and timeline gestures all map to these ids."
|
||||
title = "List commands",
|
||||
description = "List every command (id, label, menu path, shortcut, parameter doc, enabled now). Menus, shortcuts, panels and timeline gestures all map to these ids.",
|
||||
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
|
||||
)]
|
||||
async fn command_list(&self, Parameters(p): Parameters<ListParams>) -> Result<CallToolResult, McpError> {
|
||||
let r = self.run("command.list", json!({})).await.map(|v| {
|
||||
@@ -213,38 +305,99 @@ impl FilmcraftMcp {
|
||||
}
|
||||
|
||||
#[tool(
|
||||
description = "Run a command by id with JSON params, e.g. {\"id\":\"timeline.razor\",\"params\":{\"seconds\":3.5}} or {\"id\":\"effects.apply\",\"params\":{\"effect\":\"Gaussian Blur\"}}. Undoable edits land in History."
|
||||
title = "Run a command",
|
||||
description = "Run a command by id with JSON params, e.g. {\"id\":\"timeline.razor\",\"params\":{\"seconds\":3.5}} or {\"id\":\"effects.apply\",\"params\":{\"effect\":\"Gaussian Blur\"}}. Undoable edits land in History.",
|
||||
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
|
||||
)]
|
||||
async fn command_run(&self, Parameters(p): Parameters<RunParams>) -> Result<CallToolResult, McpError> {
|
||||
wrap(self.run(&p.id, p.params.unwrap_or(json!({}))).await)
|
||||
}
|
||||
|
||||
#[tool(description = "Project tree (bins and items with ids, types, durations), active sequence, source clip.")]
|
||||
#[tool(
|
||||
title = "Run several commands",
|
||||
description = "Run several commands in order: {steps: [{id, params?}], stop_on_error?: true}. Returns {completed, failed, results: [{ok, result | error}]}; each edit is its own undo step.",
|
||||
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
|
||||
)]
|
||||
async fn command_batch(&self, Parameters(p): Parameters<BatchParams>) -> Result<CallToolResult, McpError> {
|
||||
let stop = p.stop_on_error.unwrap_or(true);
|
||||
let (mut completed, mut failed, mut results) = (0u32, 0u32, Vec::new());
|
||||
for st in p.steps {
|
||||
match self.run(&st.id, st.params.unwrap_or(json!({}))).await {
|
||||
Ok(v) => {
|
||||
completed += 1;
|
||||
results.push(json!({"ok": true, "result": v}));
|
||||
}
|
||||
Err(e) => {
|
||||
failed += 1;
|
||||
results.push(json!({"ok": false, "error": e.to_string()}));
|
||||
if stop {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
let v = json!({"completed": completed, "failed": failed, "results": results});
|
||||
Ok(if failed > 0 { CallToolResult::error(vec![Content::text(v.to_string())]) } else { ok_json(&v) })
|
||||
}
|
||||
|
||||
#[tool(
|
||||
title = "Inspect the document",
|
||||
description = "The project as JSON: the tree (as project_inspect) plus the active sequence (as sequence_inspect; null when there is none).",
|
||||
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
|
||||
)]
|
||||
async fn doc_inspect(&self) -> Result<CallToolResult, McpError> {
|
||||
wrap(self.document().await)
|
||||
}
|
||||
|
||||
#[tool(
|
||||
title = "Render a preview",
|
||||
description = "The active sequence's frame at `seconds` (default: the playhead) as PNG, same as render_frame (bridge mode: Program monitor screenshot).",
|
||||
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
|
||||
)]
|
||||
async fn render_preview(&self, p: Parameters<RenderParams>) -> Result<CallToolResult, McpError> {
|
||||
self.render_frame(p).await
|
||||
}
|
||||
|
||||
#[tool(
|
||||
title = "Inspect the project tree",
|
||||
description = "Project tree (bins and items with ids, types, durations), active sequence, source clip.",
|
||||
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
|
||||
)]
|
||||
async fn project_inspect(&self) -> Result<CallToolResult, McpError> {
|
||||
wrap(self.run("project.inspect", json!({})).await)
|
||||
}
|
||||
|
||||
#[tool(
|
||||
description = "The active sequence as JSON: settings, tracks, clips (ids, start/duration in ticks and frames, effects), transitions, markers, in/out, playhead, selection. 254016000000 ticks = 1 second."
|
||||
title = "Inspect the active sequence",
|
||||
description = "The active sequence as JSON: settings, tracks, clips (ids, start/duration in ticks and frames, effects), transitions, markers, in/out, playhead, selection. 254016000000 ticks = 1 second.",
|
||||
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
|
||||
)]
|
||||
async fn sequence_inspect(&self) -> Result<CallToolResult, McpError> {
|
||||
wrap(self.run("sequence.inspect", json!({})).await)
|
||||
}
|
||||
|
||||
#[tool(description = "Import media files by absolute path (MP4/MOV, WAV/MP3/FLAC/AIFF/Ogg, PNG/JPEG/…).")]
|
||||
#[tool(
|
||||
title = "Import media",
|
||||
description = "Import media files by absolute path (MP4/MOV, WAV/MP3/FLAC/AIFF/Ogg, PNG/JPEG/…).",
|
||||
annotations(read_only_hint = false, destructive_hint = false, idempotent_hint = false, open_world_hint = false)
|
||||
)]
|
||||
async fn media_import(&self, Parameters(p): Parameters<TypeParams>) -> Result<CallToolResult, McpError> {
|
||||
let paths: Vec<&str> = p.text.split('\n').map(str::trim).filter(|s| !s.is_empty()).collect();
|
||||
wrap(self.run("file.import", json!({"paths": paths})).await)
|
||||
}
|
||||
|
||||
#[tool(description = "Render the active sequence's frame (at `seconds` or the playhead) and return it as PNG.")]
|
||||
#[tool(
|
||||
title = "Render a frame",
|
||||
description = "Render the active sequence's frame (at `seconds` or the playhead) and return it as PNG.",
|
||||
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
|
||||
)]
|
||||
async fn render_frame(&self, Parameters(p): Parameters<RenderParams>) -> Result<CallToolResult, McpError> {
|
||||
let max = p.max_side.unwrap_or(960);
|
||||
match &*self.backend {
|
||||
Backend::Headless(s) => {
|
||||
let s = s.clone();
|
||||
let r = tokio::task::spawn_blocking(move || -> Result<Vec<u8>, AutomationError> {
|
||||
let mut g = s.lock().map_err(|_| AutomationError::Other("lock".into()))?;
|
||||
let mut g = s.lock().unwrap_or_else(PoisonError::into_inner);
|
||||
if let Some(sec) = p.seconds {
|
||||
g.set_playhead(filmcraft_time::Tick::from_seconds_f64(sec));
|
||||
}
|
||||
@@ -255,7 +408,7 @@ impl FilmcraftMcp {
|
||||
png_rgba(img.w as u32, img.h as u32, img.over_black_rgba8(), max)
|
||||
})
|
||||
.await
|
||||
.map_err(|e| McpError::internal_error(e.to_string(), None))?;
|
||||
.unwrap_or_else(|e| Err(join_error(e)));
|
||||
Ok(match r {
|
||||
Ok(png) => png_result(&png, "rendered frame"),
|
||||
Err(e) => fail(e),
|
||||
@@ -272,17 +425,29 @@ impl FilmcraftMcp {
|
||||
}
|
||||
}
|
||||
|
||||
#[tool(description = "Live app: UI state (tool, workspace, panels, timeline zoom, playback, selection, fps).")]
|
||||
#[tool(
|
||||
title = "Live app: UI state",
|
||||
description = "Live app: UI state (tool, workspace, panels, timeline zoom, playback, selection, fps).",
|
||||
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
|
||||
)]
|
||||
async fn ui_inspect(&self) -> Result<CallToolResult, McpError> {
|
||||
self.ui("ui.inspect", json!({})).await
|
||||
}
|
||||
|
||||
#[tool(description = "Live app: every on-screen interactive element with id, label and rect (points). Use ids with ui_click/ui_drag.")]
|
||||
#[tool(
|
||||
title = "Live app: on-screen elements",
|
||||
description = "Live app: every on-screen interactive element with id, label and rect (points). Use ids with ui_click/ui_drag.",
|
||||
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
|
||||
)]
|
||||
async fn ui_elements(&self, Parameters(p): Parameters<ElementsParams>) -> Result<CallToolResult, McpError> {
|
||||
self.ui("ui.elements", json!({"prefix": p.prefix.unwrap_or_default()})).await
|
||||
}
|
||||
|
||||
#[tool(description = "Live app: click an element by id or at x,y (points). Supports right/middle button, double-click (count=2) and modifiers.")]
|
||||
#[tool(
|
||||
title = "Live app: click",
|
||||
description = "Live app: click an element by id or at x,y (points). Supports right/middle button, double-click (count=2) and modifiers.",
|
||||
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
|
||||
)]
|
||||
async fn ui_click(&self, Parameters(p): Parameters<ClickParams>) -> Result<CallToolResult, McpError> {
|
||||
let mut v = json!({"button": p.button, "count": p.count, "modifiers": p.modifiers});
|
||||
if let Some(id) = p.id {
|
||||
@@ -295,23 +460,37 @@ impl FilmcraftMcp {
|
||||
}
|
||||
|
||||
#[tool(
|
||||
description = "Live app: press-drag-release from one point/element to another (move clips, trim edges, scrub, resize panels, drop project items onto tracks)."
|
||||
title = "Live app: drag",
|
||||
description = "Live app: press-drag-release from one point/element to another (move clips, trim edges, scrub, resize panels, drop project items onto tracks).",
|
||||
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
|
||||
)]
|
||||
async fn ui_drag(&self, Parameters(p): Parameters<DragParams>) -> Result<CallToolResult, McpError> {
|
||||
self.ui("ui.drag", json!({"from": p.from, "to": p.to, "steps": p.steps, "modifiers": p.modifiers})).await
|
||||
}
|
||||
|
||||
#[tool(description = "Live app: press a key or shortcut (e.g. `Space`, `L`, `Cmd+K`, `Shift+Delete`, `Cmd+Z`).")]
|
||||
#[tool(
|
||||
title = "Live app: press a key",
|
||||
description = "Live app: press a key or shortcut (e.g. `Space`, `L`, `Cmd+K`, `Shift+Delete`, `Cmd+Z`).",
|
||||
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
|
||||
)]
|
||||
async fn ui_key(&self, Parameters(p): Parameters<KeyParams>) -> Result<CallToolResult, McpError> {
|
||||
self.ui("ui.key", json!({"key": p.key})).await
|
||||
}
|
||||
|
||||
#[tool(description = "Live app: type text into the focused field.")]
|
||||
#[tool(
|
||||
title = "Live app: type text",
|
||||
description = "Live app: type text into the focused field.",
|
||||
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
|
||||
)]
|
||||
async fn ui_type(&self, Parameters(p): Parameters<TypeParams>) -> Result<CallToolResult, McpError> {
|
||||
self.ui("ui.type", json!({"text": p.text})).await
|
||||
}
|
||||
|
||||
#[tool(description = "Live app: screenshot of the window or one panel (PNG).")]
|
||||
#[tool(
|
||||
title = "Live app: screenshot",
|
||||
description = "Live app: screenshot of the window or one panel (PNG).",
|
||||
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
|
||||
)]
|
||||
async fn ui_screenshot(&self, Parameters(p): Parameters<ShotParams>) -> Result<CallToolResult, McpError> {
|
||||
match self.bridge_client() {
|
||||
Some(b) => self.screenshot(&b, p.panel, p.max_side.unwrap_or(1600)).await,
|
||||
@@ -320,7 +499,9 @@ impl FilmcraftMcp {
|
||||
}
|
||||
|
||||
#[tool(
|
||||
description = "Live app: call any control-channel method directly (ui.set, ui.scroll, ui.timeline.hit, ui.timeline.locate, ui.playback, ui.panel.show, ui.menu.list, …)."
|
||||
title = "Live app: control-channel call",
|
||||
description = "Live app: call any control-channel method directly (ui.set, ui.scroll, ui.timeline.hit, ui.timeline.locate, ui.playback, ui.panel.show, ui.menu.list, …).",
|
||||
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
|
||||
)]
|
||||
async fn ui_control(&self, Parameters(p): Parameters<ControlParams>) -> Result<CallToolResult, McpError> {
|
||||
self.ui(&p.method, p.params.unwrap_or(json!({}))).await
|
||||
@@ -350,12 +531,104 @@ impl FilmcraftMcp {
|
||||
}
|
||||
}
|
||||
|
||||
#[tool_handler(
|
||||
router = self.tool_router,
|
||||
name = "filmcraft",
|
||||
instructions = "FilmCraft video editor (Premiere Pro-class). Every edit is an engine command: `command_list` to discover ids/params, `command_run` to execute (undoable). `project_inspect`/`sequence_inspect` return ids you can pass to commands; `render_frame` shows the result. In bridge mode the `ui_*` tools drive the live app: `ui_elements` lists clickable ids, `ui_click`/`ui_drag`/`ui_key` operate it, `ui_screenshot` shows it. Time is in ticks: 254016000000 per second (commands also accept `seconds`, `frame` or `timecode`)."
|
||||
)]
|
||||
impl ServerHandler for FilmcraftMcp {}
|
||||
impl FilmcraftMcp {
|
||||
/// The first argument `tool` doesn't take, as a message naming the ones it does.
|
||||
fn unknown_arg(&self, tool: &str, args: Option<&rmcp::model::JsonObject>) -> Option<String> {
|
||||
let def = self.tool_router.get(tool)?;
|
||||
let props = def.input_schema.get("properties").and_then(Value::as_object).cloned().unwrap_or_default();
|
||||
let bad = args?.keys().find(|k| !props.contains_key(*k))?;
|
||||
let accepted: Vec<&str> = props.keys().map(String::as_str).collect();
|
||||
let accepted = if accepted.is_empty() { "no arguments".to_string() } else { accepted.join(", ") };
|
||||
Some(format!("unknown argument \"{bad}\" for {tool}; expected: {accepted}"))
|
||||
}
|
||||
}
|
||||
|
||||
const INSTRUCTIONS: &str = "FilmCraft video editor (Premiere Pro-class). Every edit is an engine command: `command_list` to discover ids/params, `command_run` to execute (undoable; `command_batch` runs several). `doc_inspect` (or `project_inspect`/`sequence_inspect`) returns ids you can pass to commands; `render_preview` shows the result. In bridge mode the `ui_*` tools drive the live app: `ui_elements` lists clickable ids, `ui_click`/`ui_drag`/`ui_key` operate it, `ui_screenshot` shows it. Time is in ticks: 254016000000 per second (commands also accept `seconds`, `frame` or `timecode`).";
|
||||
|
||||
/// Resources: the project (as `doc_inspect`) and the command catalog (as `command_list`).
|
||||
const DOCUMENT_URI: &str = "filmcraft://document";
|
||||
const COMMANDS_URI: &str = "filmcraft://commands";
|
||||
|
||||
/// MCP 2026-07-28 clients (current Claude Code) require `ttlMs` and `cacheScope` on list and read
|
||||
/// results; rmcp fills `resultType` and leaves the cache hints to the server, as the generated
|
||||
/// `tools/list` does. The resource list never changes; the document does, so reads aren't cached.
|
||||
fn cache_hints(context: &rmcp::service::RequestContext<rmcp::RoleServer>, ttl_ms: u64) -> Option<(u64, rmcp::model::CacheScope)> {
|
||||
context.protocol_version().is_some_and(|v| v >= rmcp::model::ProtocolVersion::V_2026_07_28).then_some((ttl_ms, rmcp::model::CacheScope::Private))
|
||||
}
|
||||
|
||||
#[tool_handler(router = self.tool_router)]
|
||||
impl ServerHandler for FilmcraftMcp {
|
||||
fn get_info(&self) -> rmcp::model::ServerConfig {
|
||||
rmcp::model::ServerConfig::new(rmcp::model::ServerCapabilities::builder().enable_tools().enable_resources().build())
|
||||
.with_server_info(rmcp::model::Implementation::new("filmcraft", env!("CARGO_PKG_VERSION")))
|
||||
.with_instructions(INSTRUCTIONS.to_string())
|
||||
}
|
||||
|
||||
/// Strict arguments and long exports in front of the tool router.
|
||||
async fn call_tool(
|
||||
&self,
|
||||
request: rmcp::model::CallToolRequestParams,
|
||||
context: rmcp::service::RequestContext<rmcp::RoleServer>,
|
||||
) -> Result<rmcp::model::CallToolResponse, McpError> {
|
||||
if let Some(m) = self.unknown_arg(&request.name, request.arguments.as_ref()) {
|
||||
return Err(McpError::invalid_params(m, None));
|
||||
}
|
||||
// A blocking export reports progress and can be cancelled (docs/agents.md § Long exports).
|
||||
if request.name == "command_run"
|
||||
&& matches!(&*self.backend, Backend::Headless(_))
|
||||
&& let Some(a) = &request.arguments
|
||||
&& let Some(id) = a.get("id").and_then(Value::as_str)
|
||||
{
|
||||
let params = a.get("params").cloned().filter(Value::is_object).unwrap_or_else(|| json!({}));
|
||||
if crate::long_job::is_long(id, ¶ms) {
|
||||
return Ok(self.run_long(id, params, &context).await.into());
|
||||
}
|
||||
}
|
||||
let tcc = rmcp::handler::server::tool::ToolCallContext::new(self, request, context);
|
||||
self.tool_router.call(tcc).await
|
||||
}
|
||||
|
||||
async fn list_resources(
|
||||
&self,
|
||||
_request: Option<rmcp::model::PaginatedRequestParams>,
|
||||
context: rmcp::service::RequestContext<rmcp::RoleServer>,
|
||||
) -> Result<rmcp::model::ListResourcesResult, McpError> {
|
||||
use rmcp::model::Resource;
|
||||
let mut r = rmcp::model::ListResourcesResult::with_all_items(vec![
|
||||
Resource::new(DOCUMENT_URI, "document")
|
||||
.with_title("Project")
|
||||
.with_description("The project tree and the active sequence (same as doc_inspect).")
|
||||
.with_mime_type("application/json"),
|
||||
Resource::new(COMMANDS_URI, "commands")
|
||||
.with_title("Command catalog")
|
||||
.with_description("Every command with id, label, menu path, shortcut, params and enabled state (same as command_list).")
|
||||
.with_mime_type("application/json"),
|
||||
]);
|
||||
if let Some((ttl, scope)) = cache_hints(&context, 600_000) {
|
||||
r = r.with_ttl_ms(ttl).with_cache_scope(scope);
|
||||
}
|
||||
Ok(r)
|
||||
}
|
||||
|
||||
async fn read_resource(
|
||||
&self,
|
||||
request: rmcp::model::ReadResourceRequestParams,
|
||||
context: rmcp::service::RequestContext<rmcp::RoleServer>,
|
||||
) -> Result<rmcp::model::ReadResourceResponse, McpError> {
|
||||
let v = match request.uri.as_str() {
|
||||
DOCUMENT_URI => self.document().await,
|
||||
COMMANDS_URI => self.run("command.list", json!({})).await,
|
||||
other => return Err(McpError::resource_not_found(format!("resource not found: {other}"), None)),
|
||||
}
|
||||
.map_err(|e| McpError::internal_error(e.to_string(), None))?;
|
||||
let text = serde_json::to_string_pretty(&v).map_err(|e| McpError::internal_error(e.to_string(), None))?;
|
||||
let mut r = rmcp::model::ReadResourceResult::new(vec![rmcp::model::ResourceContents::text(text, request.uri).with_mime_type("application/json")]);
|
||||
if let Some((ttl, scope)) = cache_hints(&context, 0) {
|
||||
r = r.with_ttl_ms(ttl).with_cache_scope(scope);
|
||||
}
|
||||
Ok(r.into())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
@@ -371,4 +644,249 @@ mod tests {
|
||||
let r = m.render_frame(Parameters(RenderParams { seconds: Some(1.0), max_side: Some(320) })).await.unwrap();
|
||||
assert_ne!(r.is_error, Some(true));
|
||||
}
|
||||
|
||||
/// One request/response exchange over `serve_io`, as an MCP client sees it.
|
||||
struct Client {
|
||||
tx: tokio::io::DuplexStream,
|
||||
rx: tokio::io::Lines<tokio::io::BufReader<tokio::io::DuplexStream>>,
|
||||
}
|
||||
|
||||
impl Client {
|
||||
fn start(s: Session) -> Self {
|
||||
use tokio::io::AsyncBufReadExt;
|
||||
let (tx, server_in) = tokio::io::duplex(1 << 22);
|
||||
let (server_out, rx) = tokio::io::duplex(1 << 22);
|
||||
tokio::spawn(FilmcraftMcp::headless(s).serve_io(server_in, server_out));
|
||||
Self { tx, rx: tokio::io::BufReader::new(rx).lines() }
|
||||
}
|
||||
async fn send(&mut self, line: &str) {
|
||||
use tokio::io::AsyncWriteExt;
|
||||
self.tx.write_all(format!("{line}\n").as_bytes()).await.unwrap();
|
||||
}
|
||||
async fn next(&mut self) -> Value {
|
||||
serde_json::from_str(&self.rx.next_line().await.unwrap().unwrap()).unwrap()
|
||||
}
|
||||
async fn ask(&mut self, msg: Value) -> Value {
|
||||
self.send(&msg.to_string()).await;
|
||||
self.next().await
|
||||
}
|
||||
async fn init(&mut self) -> Value {
|
||||
let r = self.ask(json!({"jsonrpc":"2.0","id":0,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"t","version":"1"}}})).await;
|
||||
self.send(&json!({"jsonrpc":"2.0","method":"notifications/initialized"}).to_string()).await;
|
||||
r
|
||||
}
|
||||
async fn call(&mut self, id: u64, name: &str, arguments: Value) -> Value {
|
||||
self.ask(json!({"jsonrpc":"2.0","id":id,"method":"tools/call","params":{"name":name,"arguments":arguments}})).await
|
||||
}
|
||||
}
|
||||
|
||||
fn demo() -> Session {
|
||||
let mut s = Session::default();
|
||||
s.execute("file.openDemoProject", json!({})).unwrap();
|
||||
s
|
||||
}
|
||||
|
||||
/// Every listed tool has a title and the four hints, and the core tools are listed.
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn tools_are_titled_and_annotated() {
|
||||
let mut c = Client::start(demo());
|
||||
let init = c.init().await;
|
||||
assert_eq!(init["result"]["serverInfo"]["name"], "filmcraft");
|
||||
let tools = c.ask(json!({"jsonrpc":"2.0","id":1,"method":"tools/list"})).await["result"]["tools"].as_array().cloned().unwrap();
|
||||
for t in &tools {
|
||||
assert!(t["title"].is_string(), "{} has no title", t["name"]);
|
||||
for h in ["readOnlyHint", "destructiveHint", "idempotentHint", "openWorldHint"] {
|
||||
assert!(t["annotations"][h].is_boolean(), "{} lacks {h}", t["name"]);
|
||||
}
|
||||
}
|
||||
let names: Vec<&str> = tools.iter().filter_map(|t| t["name"].as_str()).collect();
|
||||
for want in ["command_list", "command_run", "command_batch", "doc_inspect", "render_preview", "project_inspect", "render_frame", "ui_click"] {
|
||||
assert!(names.contains(&want), "{want} not listed");
|
||||
}
|
||||
let ro = |n: &str| tools.iter().find(|t| t["name"] == n).map(|t| t["annotations"]["readOnlyHint"].clone());
|
||||
assert_eq!((ro("command_list"), ro("command_run")), (Some(json!(true)), Some(json!(false))));
|
||||
}
|
||||
|
||||
/// Strict arguments, malformed JSON, batch, doc_inspect and render_preview over the raw line
|
||||
/// protocol.
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn strict_arguments_and_parse_errors() {
|
||||
let mut c = Client::start(demo());
|
||||
c.init().await;
|
||||
let r = c.call(1, "command_list", json!({"filter": "razor", "enabled_only": true})).await;
|
||||
assert!(r["result"]["content"][0]["text"].as_str().unwrap().contains("timeline.razor"), "{r}");
|
||||
// an unknown argument names itself and the accepted ones
|
||||
let r = c.call(2, "command_list", json!({"filtr": "x"})).await;
|
||||
assert_eq!(r["error"]["code"], -32602, "{r}");
|
||||
let m = r["error"]["message"].as_str().unwrap();
|
||||
assert!(m.contains("\"filtr\"") && m.contains("filter") && m.contains("enabled_only"), "{m}");
|
||||
let r = c.call(3, "doc_inspect", json!({"verbose": true})).await;
|
||||
assert_eq!(r["error"]["code"], -32602, "{r}");
|
||||
// command params stay free-form
|
||||
let r = c.call(4, "command_run", json!({"id": "playhead.set", "params": {"seconds": 0.5}})).await;
|
||||
assert_eq!(r["result"]["isError"], false, "{r}");
|
||||
// malformed JSON: a parse error, and the server keeps serving
|
||||
c.send("{\"jsonrpc\": \"2.0\", \"id\": 5, BROKEN").await;
|
||||
let r = c.next().await;
|
||||
assert_eq!((r["error"]["code"].as_i64(), r["id"].is_null()), (Some(-32700), true), "{r}");
|
||||
let r = c.ask(json!({"jsonrpc":"2.0","id":6,"method":"ping"})).await;
|
||||
assert!(r["result"].is_object(), "{r}");
|
||||
// batch: stops at the first failure and reports both
|
||||
let r =
|
||||
c.call(7, "command_batch", json!({"steps": [{"id": "playhead.set", "params": {"seconds": 0.5}}, {"id": "no.such"}, {"id": "playhead.set"}]})).await;
|
||||
assert_eq!(r["result"]["isError"], true, "{r}");
|
||||
let v: Value = serde_json::from_str(r["result"]["content"][0]["text"].as_str().unwrap()).unwrap();
|
||||
assert_eq!((v["completed"].as_u64(), v["failed"].as_u64(), v["results"].as_array().map(Vec::len)), (Some(1), Some(1), Some(2)), "{v}");
|
||||
let r = c.call(8, "doc_inspect", json!({})).await;
|
||||
let v: Value = serde_json::from_str(r["result"]["content"][0]["text"].as_str().unwrap()).unwrap();
|
||||
assert!(v["project"].is_object() && v["sequence"].is_object(), "{v}");
|
||||
let r = c.call(9, "render_preview", json!({"max_side": 64})).await;
|
||||
assert_eq!(r["result"]["content"][0]["type"], "image", "{r}");
|
||||
}
|
||||
|
||||
/// `filmcraft://document` and `filmcraft://commands` are listed and readable as JSON.
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn resources_document_and_commands() {
|
||||
let mut c = Client::start(demo());
|
||||
let init = c.init().await;
|
||||
assert!(init["result"]["capabilities"]["resources"].is_object(), "{init}");
|
||||
assert!(init["result"]["instructions"].as_str().unwrap().contains("doc_inspect"), "{init}");
|
||||
let r = c.ask(json!({"jsonrpc":"2.0","id":1,"method":"resources/list"})).await;
|
||||
let uris: Vec<&str> = r["result"]["resources"].as_array().unwrap().iter().filter_map(|x| x["uri"].as_str()).collect();
|
||||
assert_eq!(uris, [DOCUMENT_URI, COMMANDS_URI], "{r}");
|
||||
let r = c.ask(json!({"jsonrpc":"2.0","id":2,"method":"resources/read","params":{"uri":DOCUMENT_URI}})).await;
|
||||
assert_eq!(r["result"]["contents"][0]["mimeType"], "application/json", "{r}");
|
||||
let v: Value = serde_json::from_str(r["result"]["contents"][0]["text"].as_str().unwrap()).unwrap();
|
||||
assert!(v["project"].is_object() && v["sequence"].is_object(), "{v}");
|
||||
let r = c.ask(json!({"jsonrpc":"2.0","id":3,"method":"resources/read","params":{"uri":COMMANDS_URI}})).await;
|
||||
assert!(r["result"]["contents"][0]["text"].as_str().unwrap().contains("timeline.razor"), "{r}");
|
||||
let r = c.ask(json!({"jsonrpc":"2.0","id":4,"method":"resources/read","params":{"uri":"filmcraft://nope"}})).await;
|
||||
assert!(r["error"]["code"].is_i64(), "{r}");
|
||||
}
|
||||
|
||||
/// MCP 2026-07-28 clients (current Claude Code) negotiate per request and reject list and read
|
||||
/// results without `resultType`, `ttlMs` and `cacheScope`; sessions that negotiated an older
|
||||
/// revision through `initialize` get the old shape.
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn modern_clients_get_result_type_and_cache_hints() {
|
||||
let mut c = Client::start(Session::default());
|
||||
let meta = json!({
|
||||
"io.modelcontextprotocol/protocolVersion": "2026-07-28",
|
||||
"io.modelcontextprotocol/clientInfo": {"name": "t", "version": "1"},
|
||||
"io.modelcontextprotocol/clientCapabilities": {},
|
||||
});
|
||||
let r = c.ask(json!({"jsonrpc":"2.0","id":1,"method":"server/discover","params":{"_meta":meta}})).await;
|
||||
assert!(r["result"]["supportedVersions"].as_array().unwrap().contains(&json!("2026-07-28")), "{r}");
|
||||
for (id, method, params) in [
|
||||
(2, "tools/list", json!({"_meta": meta})),
|
||||
(3, "resources/list", json!({"_meta": meta})),
|
||||
(4, "resources/read", json!({"_meta": meta, "uri": DOCUMENT_URI})),
|
||||
] {
|
||||
let r = c.ask(json!({"jsonrpc":"2.0","id":id,"method":method,"params":params})).await;
|
||||
assert_eq!(r["result"]["resultType"], "complete", "{method}: {r}");
|
||||
assert!(r["result"]["ttlMs"].is_u64(), "{method}: {r}");
|
||||
assert!(matches!(r["result"]["cacheScope"].as_str(), Some("public" | "private")), "{method}: {r}");
|
||||
}
|
||||
let mut c = Client::start(Session::default());
|
||||
c.init().await;
|
||||
for (id, method, params) in [(2, "tools/list", json!({})), (3, "resources/list", json!({})), (4, "resources/read", json!({"uri": DOCUMENT_URI}))] {
|
||||
let r = c.ask(json!({"jsonrpc":"2.0","id":id,"method":method,"params":params})).await;
|
||||
assert!(r["result"].is_object() && r["result"].get("resultType").is_none() && r["result"].get("ttlMs").is_none(), "{method}: {r}");
|
||||
}
|
||||
}
|
||||
|
||||
/// docs/agents.md § Long exports: a blocking export reports `notifications/progress` for its
|
||||
/// token (none without one), answers `ping` meanwhile, and `notifications/cancelled` stops
|
||||
/// it, deletes the partial file and sends no response.
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn export_progress_and_cancel() {
|
||||
let dir = std::env::temp_dir().join(format!("filmcraft-mcp-progress-{}", std::process::id()));
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
let mut c = Client::start(demo());
|
||||
c.init().await;
|
||||
let export = |id: u64, name: &str, token: Option<Value>, seconds: f64| {
|
||||
let path = dir.join(name).to_string_lossy().to_string();
|
||||
let params = json!({"path": path, "format": "h264", "width": 640, "height": 360, "audio": false, "range": "custom", "startSeconds": 0, "endSeconds": seconds, "wait": true});
|
||||
let mut call =
|
||||
json!({"jsonrpc":"2.0","id":id,"method":"tools/call","params":{"name":"command_run","arguments":{"id":"file.exportMedia","params":params}}});
|
||||
if let Some(t) = token {
|
||||
call["params"]["_meta"] = json!({"progressToken": t});
|
||||
}
|
||||
call.to_string()
|
||||
};
|
||||
// no token: no notifications, the same response as before
|
||||
c.send(&export(1, "quiet.mp4", None, 0.5)).await;
|
||||
let m = c.next().await;
|
||||
assert_eq!((m["id"].as_u64(), &m["result"]["isError"]), (Some(1), &json!(false)), "{m}");
|
||||
// progress, strictly increasing, with a total; ping answered while the export runs
|
||||
c.send(&export(2, "a.mp4", Some(json!("tok")), 1.0)).await;
|
||||
let (mut last, mut notes, mut pinged, mut pong) = (-1.0, 0, false, false);
|
||||
let done = loop {
|
||||
let m = c.next().await;
|
||||
if m["method"] == "notifications/progress" {
|
||||
assert_eq!(m["params"]["progressToken"], "tok", "{m}");
|
||||
let p = m["params"]["progress"].as_f64().unwrap();
|
||||
assert!(p > last && m["params"]["total"].as_f64().unwrap() >= p, "{m}");
|
||||
last = p;
|
||||
notes += 1;
|
||||
if !pinged {
|
||||
pinged = true;
|
||||
c.send(r#"{"jsonrpc":"2.0","id":3,"method":"ping"}"#).await;
|
||||
}
|
||||
} else if m["id"] == 3 {
|
||||
pong = true;
|
||||
} else if m["id"] == 2 {
|
||||
break m;
|
||||
}
|
||||
};
|
||||
assert_eq!(done["result"]["isError"], false, "{done}");
|
||||
assert!(notes >= 2 && pong, "{notes} notifications, ping answered during the export: {pong}");
|
||||
assert!(dir.join("a.mp4").exists());
|
||||
// cancel after the first notification: no response, no partial file
|
||||
c.send(&export(4, "b.mp4", Some(json!(7)), 8.0)).await;
|
||||
loop {
|
||||
let m = c.next().await;
|
||||
assert_ne!(m["id"], 4, "finished before the cancel: {m}");
|
||||
if m["method"] == "notifications/progress" {
|
||||
break;
|
||||
}
|
||||
}
|
||||
c.send(r#"{"jsonrpc":"2.0","method":"notifications/cancelled","params":{"requestId":4}}"#).await;
|
||||
let t0 = std::time::Instant::now();
|
||||
while dir.join("b.mp4").exists() || t0.elapsed() < std::time::Duration::from_millis(500) {
|
||||
assert!(t0.elapsed().as_secs() < 30, "partial file still there");
|
||||
tokio::time::sleep(std::time::Duration::from_millis(50)).await;
|
||||
}
|
||||
c.send(r#"{"jsonrpc":"2.0","id":5,"method":"tools/call","params":{"name":"command_run","arguments":{"id":"jobs.list"}}}"#).await;
|
||||
let m = loop {
|
||||
let m = c.next().await;
|
||||
assert_ne!(m["id"], 4, "no response for a cancelled request: {m}");
|
||||
if m["id"] == 5 {
|
||||
break m;
|
||||
}
|
||||
};
|
||||
let jobs: Value = serde_json::from_str(m["result"]["content"][0]["text"].as_str().unwrap()).unwrap();
|
||||
let last = jobs.as_array().and_then(|a| a.last()).cloned().unwrap_or_default();
|
||||
assert_eq!(last["result"]["error"], "cancelled", "{jobs}");
|
||||
assert!(!dir.join("b.mp4").exists());
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
/// A command that panicked used to poison the session lock, and every later call failed with
|
||||
/// "session lock poisoned" until the server was restarted.
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn a_poisoned_session_lock_keeps_serving() {
|
||||
let m = FilmcraftMcp::headless(demo());
|
||||
let Backend::Headless(s) = &*m.backend else { panic!("headless backend") };
|
||||
let s = s.clone();
|
||||
let _ = std::thread::spawn(move || {
|
||||
let _g = s.lock().unwrap();
|
||||
panic!("a command panicked");
|
||||
})
|
||||
.join();
|
||||
let v = m.run("sequence.inspect", json!({})).await.unwrap();
|
||||
assert!(v.is_object(), "{v}");
|
||||
let r = m.render_frame(Parameters(RenderParams { seconds: Some(0.5), max_side: Some(64) })).await.unwrap();
|
||||
assert_ne!(r.is_error, Some(true));
|
||||
}
|
||||
}
|
||||
@@ -237,12 +237,16 @@ fn filter_rows<const W: usize>(buf: &[i32; PAD_STRIDE * PAD_STRIDE], taps: &[(is
|
||||
}
|
||||
for i in 0..h {
|
||||
let ci = ((i + 2) * PAD_STRIDE + 2) as isize;
|
||||
let x: [i16; W] = b16[ci as usize..ci as usize + W].try_into().expect("centre row");
|
||||
let Some(&x) = b16.get(ci as usize..).and_then(|r| r.first_chunk::<W>()) else {
|
||||
return;
|
||||
};
|
||||
let (mut sum, mut max, mut min) = ([0i16; W], x, x);
|
||||
for &(off, weight, thr, adj) in taps {
|
||||
let (weight, thr) = (weight as i16, thr as i16);
|
||||
let start = (ci + off) as usize;
|
||||
let p: &[i16; W] = b16[start..start + W].try_into().expect("tap row");
|
||||
let Some(p) = b16.get(start..).and_then(|r| r.first_chunk::<W>()) else {
|
||||
return;
|
||||
};
|
||||
for j in 0..W {
|
||||
// constrain(): .max / .min instead of clamp (whose bound check panics), and
|
||||
// the sign applied branch-free, so the loop vectorises.
|
||||
|
||||
@@ -72,10 +72,12 @@ impl<T: Clone> Slot<T> {
|
||||
|
||||
fn wait(&self) -> Result<T> {
|
||||
let mut g = self.value.lock().unwrap_or_else(|e| e.into_inner());
|
||||
while g.is_none() {
|
||||
loop {
|
||||
if let Some(v) = g.as_ref() {
|
||||
return v.clone();
|
||||
}
|
||||
g = self.cv.wait(g).unwrap_or_else(|e| e.into_inner());
|
||||
}
|
||||
g.as_ref().expect("slot value").clone()
|
||||
}
|
||||
}
|
||||
|
||||
@@ -308,7 +310,7 @@ impl Decoder {
|
||||
if !o.is_ready() && self.out.len() <= max_queue {
|
||||
break;
|
||||
}
|
||||
let o = self.out.pop_front().expect("front");
|
||||
let Some(o) = self.out.pop_front() else { break };
|
||||
pics.push(o.take()?);
|
||||
}
|
||||
parsed?;
|
||||
@@ -358,7 +360,9 @@ impl Decoder {
|
||||
match last {
|
||||
Some((lt, lsid)) if lt == t => {
|
||||
if sid >= lsid {
|
||||
*sel.last_mut().expect("picture") = o;
|
||||
if let Some(l) = sel.last_mut() {
|
||||
*l = o;
|
||||
}
|
||||
last = Some((t, sid));
|
||||
}
|
||||
}
|
||||
@@ -612,7 +616,7 @@ impl Decoder {
|
||||
self.in_flight.push_back(slot);
|
||||
// Bound the frames in flight (memory): wait for the oldest.
|
||||
while self.in_flight.len() > self.max_in_flight {
|
||||
let s = self.in_flight.pop_front().expect("in flight");
|
||||
let Some(s) = self.in_flight.pop_front() else { break };
|
||||
let _ = s.wait();
|
||||
}
|
||||
self.in_flight.retain(|s| !s.is_set());
|
||||
|
||||
+17
-5
@@ -180,7 +180,11 @@ impl TileDecoder<'_, '_> {
|
||||
}
|
||||
|
||||
fn ref_plane(&self, ref_idx: i32, plane: usize) -> &Plane {
|
||||
if ref_idx < 0 { &self.t.cur.planes[plane] } else { &self.fs.refs[ref_idx as usize].as_ref().expect("reference frame").buf.planes[plane] }
|
||||
// A missing reference (rejected earlier for valid streams) predicts from the current frame.
|
||||
match usize::try_from(ref_idx).ok().and_then(|i| self.fs.refs.get(i)).and_then(|r| r.as_ref()) {
|
||||
Some(r) => &r.buf.planes[plane],
|
||||
None => &self.t.cur.planes[plane],
|
||||
}
|
||||
}
|
||||
|
||||
/// Block inter prediction process (7.11.3.4) into `pred` (stride PRED_STRIDE); `tmp` holds
|
||||
@@ -672,12 +676,16 @@ fn hpass<const W: usize>(src: &[u16], hf: &[i16; 8], out: &mut [i32], half: i32,
|
||||
let mut acc = [half; W];
|
||||
for t in 0..8 {
|
||||
let f = hf[t] as i32;
|
||||
let p: &[u16; W] = src[t..t + W].try_into().expect("row");
|
||||
let Some(p) = src.get(t..).and_then(|s| s.first_chunk::<W>()) else {
|
||||
return;
|
||||
};
|
||||
for j in 0..W {
|
||||
acc[j] += f * p[j] as i32;
|
||||
}
|
||||
}
|
||||
let o: &mut [i32; W] = (&mut out[..W]).try_into().expect("row");
|
||||
let Some(o) = out.first_chunk_mut::<W>() else {
|
||||
return;
|
||||
};
|
||||
for j in 0..W {
|
||||
o[j] = acc[j] >> sh;
|
||||
}
|
||||
@@ -689,12 +697,16 @@ fn vpass<const W: usize>(rows: &[i32], vf: &[i16; 8], out: &mut [i32], half: i32
|
||||
let mut acc = [half; W];
|
||||
for t in 0..8 {
|
||||
let f = vf[t] as i32;
|
||||
let p: &[i32; W] = rows[t * W..t * W + W].try_into().expect("row");
|
||||
let Some(p) = rows.get(t * W..).and_then(|s| s.first_chunk::<W>()) else {
|
||||
return;
|
||||
};
|
||||
for j in 0..W {
|
||||
acc[j] += f * p[j];
|
||||
}
|
||||
}
|
||||
let o: &mut [i32; W] = (&mut out[..W]).try_into().expect("row");
|
||||
let Some(o) = out.first_chunk_mut::<W>() else {
|
||||
return;
|
||||
};
|
||||
for j in 0..W {
|
||||
o[j] = acc[j] >> sh;
|
||||
}
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
|
||||
// Items used only by stages still being written (inter prediction, post filters).
|
||||
#![allow(dead_code)]
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
mod bits;
|
||||
mod cdef;
|
||||
|
||||
@@ -76,7 +76,7 @@ pub fn reference(ff: &Path, path: &Path, pix_fmt: &str) -> Vec<u16> {
|
||||
.expect("run ffmpeg");
|
||||
assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr));
|
||||
if pix_fmt.ends_with("le") {
|
||||
out.stdout.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
|
||||
out.stdout.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
|
||||
} else {
|
||||
out.stdout.iter().map(|&b| b as u16).collect()
|
||||
}
|
||||
|
||||
@@ -5,6 +5,8 @@
|
||||
//! - [`unescape_rbsp`] / [`escape_rbsp`]: remove/insert H.264/HEVC emulation-prevention bytes.
|
||||
//! - [`annexb_nals`] / [`length_prefixed_nals`]: split Annex-B byte streams and AVCC/HVCC samples into NAL units.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
use std::fmt;
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
|
||||
@@ -15,6 +15,8 @@
|
||||
//! - **Burn-in:** [`burn`] lays a caption out with the track style and draws it with the
|
||||
//! `filmcraft-text` engine (Inter SemiBold, shaped and kerned).
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
pub mod burn;
|
||||
pub mod cea608;
|
||||
pub mod mcc;
|
||||
@@ -212,7 +214,7 @@ pub fn parse(bytes: &[u8], format: Format) -> Result<Document> {
|
||||
Format::Scc => scc::parse(&text),
|
||||
Format::Mcc => mcc::parse(&text),
|
||||
Format::Ttml | Format::Dfxp => ttml::parse(&text),
|
||||
Format::Stl => unreachable!("handled above"),
|
||||
Format::Stl => stl::parse(bytes),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -253,7 +255,7 @@ pub fn decode_text(bytes: &[u8]) -> String {
|
||||
} else if bytes.starts_with(&[0xff, 0xfe]) || bytes.starts_with(&[0xfe, 0xff]) {
|
||||
let le = bytes[0] == 0xff;
|
||||
let units: Vec<u16> =
|
||||
bytes[2..].chunks_exact(2).map(|c| if le { u16::from_le_bytes([c[0], c[1]]) } else { u16::from_be_bytes([c[0], c[1]]) }).collect();
|
||||
bytes[2..].as_chunks::<2>().0.iter().map(|c| if le { u16::from_le_bytes([c[0], c[1]]) } else { u16::from_be_bytes([c[0], c[1]]) }).collect();
|
||||
String::from_utf16_lossy(&units)
|
||||
} else {
|
||||
match std::str::from_utf8(bytes) {
|
||||
|
||||
@@ -411,10 +411,10 @@ impl Dec708 {
|
||||
}
|
||||
fn put(&mut self, c: char) {
|
||||
let w = self.w();
|
||||
if w.rows.is_empty() {
|
||||
w.rows.push(String::new());
|
||||
match w.rows.last_mut() {
|
||||
Some(row) => row.push(c),
|
||||
None => w.rows.push(c.to_string()),
|
||||
}
|
||||
w.rows.last_mut().expect("row").push(c);
|
||||
}
|
||||
fn bitmap(&mut self, m: u8, f: impl Fn(&mut Win)) {
|
||||
for (i, w) in self.win.iter_mut().enumerate() {
|
||||
|
||||
@@ -251,7 +251,9 @@ pub fn parse(bytes: &[u8]) -> Result<Document> {
|
||||
if !sniff(bytes) {
|
||||
return Err(Error::NotFormat("EBU STL"));
|
||||
}
|
||||
let rate = rate_of(&bytes[3..11]).expect("sniffed");
|
||||
let Some(rate) = bytes.get(3..11).and_then(rate_of) else {
|
||||
return Err(Error::NotFormat("EBU STL"));
|
||||
};
|
||||
let gsi = &bytes[..GSI];
|
||||
// start of programme (HHMMSSFF, ASCII)
|
||||
let tcp = ascii_field(&gsi[256..264]);
|
||||
@@ -277,7 +279,7 @@ pub fn parse(bytes: &[u8]) -> Result<Document> {
|
||||
}
|
||||
}
|
||||
};
|
||||
for blk in bytes[GSI..].chunks_exact(TTI) {
|
||||
for blk in bytes[GSI..].as_chunks::<TTI>().0 {
|
||||
let sn = u16::from_le_bytes([blk[1], blk[2]]);
|
||||
let ebn = blk[3];
|
||||
let cf = blk[15];
|
||||
|
||||
+71
-16
@@ -103,7 +103,7 @@ fn inline(text: &str) -> String {
|
||||
rest = &after[end + 1..];
|
||||
continue;
|
||||
}
|
||||
let ch = rest.chars().next().expect("non-empty");
|
||||
let Some(ch) = rest.chars().next() else { break };
|
||||
if ch == '\n' {
|
||||
out.push_str("<br/>");
|
||||
} else {
|
||||
@@ -184,28 +184,45 @@ struct Timing {
|
||||
tick_rate: i64,
|
||||
}
|
||||
|
||||
fn positive_integer(s: &str) -> Option<i64> {
|
||||
let s = s.trim();
|
||||
if s.is_empty() || !s.bytes().all(|b| b.is_ascii_digit()) {
|
||||
return None;
|
||||
}
|
||||
s.parse::<i64>().ok().filter(|v| *v > 0)
|
||||
}
|
||||
|
||||
fn parse_ratio(s: &str) -> Option<(i64, i64)> {
|
||||
let mut it = s.split_whitespace().map(|x| x.parse::<i64>().ok());
|
||||
let mut it = s.split_whitespace().map(positive_integer);
|
||||
let a = it.next()??;
|
||||
let b = it.next().flatten().unwrap_or(1);
|
||||
(a > 0 && b > 0).then_some((a, b))
|
||||
let b = it.next()??;
|
||||
it.next().is_none().then_some((a, b))
|
||||
}
|
||||
|
||||
fn attr<'a>(n: roxmltree::Node<'a, '_>, local: &str) -> Option<&'a str> {
|
||||
n.attributes().find(|a| a.name() == local).map(|a| a.value())
|
||||
}
|
||||
|
||||
fn timing(root: roxmltree::Node) -> Timing {
|
||||
let base = attr(root, "frameRate").and_then(|v| v.trim().parse::<i64>().ok()).filter(|v| *v > 0).unwrap_or(30);
|
||||
let (mn, md) = attr(root, "frameRateMultiplier").and_then(parse_ratio).unwrap_or((1, 1));
|
||||
let rate = FrameRate::new(base * mn, md);
|
||||
let sub = attr(root, "subFrameRate").and_then(|v| v.trim().parse::<i64>().ok()).filter(|v| *v > 0).unwrap_or(1);
|
||||
let tick_rate = attr(root, "tickRate").and_then(|v| v.trim().parse::<i64>().ok()).filter(|v| *v > 0).unwrap_or(if attr(root, "frameRate").is_some() {
|
||||
base * sub
|
||||
} else {
|
||||
1
|
||||
});
|
||||
Timing { rate, base, sub_frame_rate: sub, tick_rate }
|
||||
fn timing(root: roxmltree::Node) -> Result<Timing> {
|
||||
let invalid =
|
||||
|name: &str| Error::Syntax { line: root.document().text_pos_at(root.range().start).row as usize, msg: format!("invalid TTML timing parameter {name}") };
|
||||
let integer = |name: &str, default| match attr(root, name) {
|
||||
Some(v) => positive_integer(v).ok_or_else(|| invalid(name)),
|
||||
None => Ok(default),
|
||||
};
|
||||
let base = integer("frameRate", 30)?;
|
||||
let (mn, md) = match attr(root, "frameRateMultiplier") {
|
||||
Some(v) => parse_ratio(v).ok_or_else(|| invalid("frameRateMultiplier"))?,
|
||||
None => (1, 1),
|
||||
};
|
||||
let rate = FrameRate::new(base.checked_mul(mn).ok_or_else(|| invalid("frameRateMultiplier"))?, md);
|
||||
let sub = integer("subFrameRate", 1)?;
|
||||
let tick_rate = match attr(root, "tickRate") {
|
||||
Some(v) => positive_integer(v).ok_or_else(|| invalid("tickRate"))?,
|
||||
None if attr(root, "frameRate").is_some() => base.checked_mul(sub).ok_or_else(|| invalid("subFrameRate"))?,
|
||||
None => 1,
|
||||
};
|
||||
Ok(Timing { rate, base, sub_frame_rate: sub, tick_rate })
|
||||
}
|
||||
|
||||
/// Seconds as an exact decimal string → ticks.
|
||||
@@ -383,7 +400,7 @@ pub fn parse(text: &str) -> Result<Document> {
|
||||
if root.tag_name().name() != "tt" {
|
||||
return Err(Error::NotFormat("TTML"));
|
||||
}
|
||||
let tm = timing(root);
|
||||
let tm = timing(root)?;
|
||||
let mut styles = std::collections::HashMap::new();
|
||||
for s in root.descendants().filter(|n| n.is_element() && n.tag_name().name() == "style") {
|
||||
if let Some(id) = s.attributes().find(|a| a.name() == "id").map(|a| a.value()) {
|
||||
@@ -478,6 +495,44 @@ mod tests {
|
||||
Timing { rate, base: rate.timecode_base(), sub_frame_rate: 1, tick_rate: 1 }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn invalid_timing_parameters_return_an_error() {
|
||||
for params in [
|
||||
r#"ttp:frameRate="0""#,
|
||||
r#"ttp:frameRate="-25""#,
|
||||
r#"ttp:frameRate="""#,
|
||||
r#"ttp:subFrameRate="0""#,
|
||||
r#"ttp:tickRate="-1""#,
|
||||
r#"ttp:frameRateMultiplier="1 0""#,
|
||||
r#"ttp:frameRateMultiplier="1""#,
|
||||
r#"ttp:frameRateMultiplier="1 2 3""#,
|
||||
] {
|
||||
let xml = format!(
|
||||
r#"<tt xmlns="http://www.w3.org/ns/ttml" xmlns:ttp="http://www.w3.org/ns/ttml#parameter" {params}><body><div><p begin="1s" end="2s">Hello</p></div></body></tt>"#
|
||||
);
|
||||
assert!(crate::parse(xml.as_bytes(), crate::Format::Ttml).is_err(), "{params}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_timing_parameters_keep_defaults() {
|
||||
for (params, begin, end) in [
|
||||
("", "1s", "2s"),
|
||||
(r#"ttp:frameRate="25""#, "25f", "50f"),
|
||||
(r#"ttp:frameRate="30" ttp:subFrameRate="2""#, "60t", "120t"),
|
||||
(r#"ttp:frameRate="30" ttp:subFrameRate="2" ttp:tickRate="10""#, "10t", "20t"),
|
||||
(r#"ttp:frameRate="25" ttp:frameRateMultiplier="2 1""#, "50f", "100f"),
|
||||
] {
|
||||
let xml = format!(
|
||||
r#"<tt xmlns="http://www.w3.org/ns/ttml" xmlns:ttp="http://www.w3.org/ns/ttml#parameter" {params}><body><div><p begin="{begin}" end="{end}">Hello</p></div></body></tt>"#
|
||||
);
|
||||
let doc = crate::parse(xml.as_bytes(), crate::Format::Ttml).unwrap();
|
||||
assert_eq!(doc.cues.len(), 1, "{params}");
|
||||
assert_eq!(doc.cues[0].start, Tick(TICKS_PER_SECOND), "{params}");
|
||||
assert_eq!(doc.cues[0].end, Tick(2 * TICKS_PER_SECOND), "{params}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn time_expressions() {
|
||||
let t = tm(FrameRate::FPS_25);
|
||||
|
||||
@@ -15,6 +15,8 @@
|
||||
//!
|
||||
//! Layer L0: no dependencies beyond `std`, no `unsafe`, builds for `wasm32-unknown-unknown`.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
mod read;
|
||||
mod write;
|
||||
|
||||
|
||||
@@ -143,7 +143,7 @@ impl<'a> CompoundFile<'a> {
|
||||
// Mini FAT and mini stream.
|
||||
if first_minifat <= MAXREGSECT {
|
||||
let mf = cf.read_chain(first_minifat, None)?;
|
||||
cf.minifat = mf.chunks_exact(4).map(|c| u32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect();
|
||||
cf.minifat = mf.as_chunks::<4>().0.iter().map(|c| u32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect();
|
||||
}
|
||||
let root = &cf.entries[0];
|
||||
if root.start <= MAXREGSECT && root.size > 0 {
|
||||
|
||||
@@ -36,7 +36,7 @@ log = { workspace = true }
|
||||
web-time = "1"
|
||||
image = { version = "0.25", default-features = false, features = ["jpeg"] }
|
||||
# Bootstrap audio decoding (MPL-2.0, used unmodified); replaced by own decoders over time.
|
||||
symphonia = { version = "0.5.5", default-features = false, features = ["aac", "mp1", "mp2", "mp3", "flac", "alac", "pcm", "wav", "ogg", "vorbis", "isomp4"] }
|
||||
symphonia = { version = "0.5.5", default-features = false, features = ["aac", "mp1", "mp2", "mp3", "flac", "alac", "pcm", "wav", "aiff", "ogg", "vorbis", "isomp4"] }
|
||||
|
||||
[dev-dependencies]
|
||||
filmcraft-testkit = { workspace = true }
|
||||
@@ -33,6 +33,9 @@ pub struct GopStats {
|
||||
pub skipped: u64,
|
||||
/// Frames decoded in draft mode (reduced-resolution playback, [`VideoDecoder::set_draft`]).
|
||||
pub draft: u64,
|
||||
/// Pictures decoders output (hardware and software; [`crate::hw::hw_stats`] counts the
|
||||
/// hardware ones).
|
||||
pub frames: u64,
|
||||
}
|
||||
|
||||
impl GopStats {
|
||||
@@ -56,6 +59,12 @@ static EVICTED: AtomicU64 = AtomicU64::new(0);
|
||||
static DECODE_NS: AtomicU64 = AtomicU64::new(0);
|
||||
static SKIPPED: AtomicU64 = AtomicU64::new(0);
|
||||
static DRAFT: AtomicU64 = AtomicU64::new(0);
|
||||
static FRAMES: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
/// Count pictures a decoder call output.
|
||||
fn count_frames<T>(out: &[T]) {
|
||||
FRAMES.fetch_add(out.len() as u64, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// Run a decoder call, adding its wall time to the process-wide counter.
|
||||
fn timed<R>(f: impl FnOnce() -> R) -> R {
|
||||
@@ -76,6 +85,7 @@ pub fn gop_stats() -> GopStats {
|
||||
decode_ns: DECODE_NS.load(Ordering::Relaxed),
|
||||
skipped: SKIPPED.load(Ordering::Relaxed),
|
||||
draft: DRAFT.load(Ordering::Relaxed),
|
||||
frames: FRAMES.load(Ordering::Relaxed),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -91,6 +101,7 @@ impl std::ops::Sub for GopStats {
|
||||
decode_ns: self.decode_ns - o.decode_ns,
|
||||
skipped: self.skipped - o.skipped,
|
||||
draft: self.draft - o.draft,
|
||||
frames: self.frames - o.frames,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -240,8 +251,7 @@ impl GopCache {
|
||||
}
|
||||
// evict frames far from the most recent (keep a window around the working position)
|
||||
while st.bytes > budget && st.frames.len() > 2 {
|
||||
let first = *st.frames.keys().next().expect("non-empty");
|
||||
let last = *st.frames.keys().next_back().expect("non-empty");
|
||||
let (Some(&first), Some(&last)) = (st.frames.keys().next(), st.frames.keys().next_back()) else { break };
|
||||
let victim = if pts - first > last - pts { first } else { last };
|
||||
if let Some(v) = st.frames.remove(&victim) {
|
||||
st.drafts.remove(&victim);
|
||||
@@ -253,6 +263,7 @@ impl GopCache {
|
||||
|
||||
/// Store decoder output (in presentation order) and advance `out_max`.
|
||||
fn store_output(&self, st: &mut State, out: Vec<crate::video::DecodedFrame>) {
|
||||
count_frames(&out);
|
||||
for d in out {
|
||||
st.out_max = st.out_max.max(d.pts);
|
||||
self.store(st, d.pts, d.frame, d.draft);
|
||||
@@ -323,7 +334,7 @@ impl GopCache {
|
||||
// every sample): step back to a sample the decoder can start from.
|
||||
while key > 0 {
|
||||
let data = s.read(key)?;
|
||||
if st.decoder.as_ref().expect("decoder").is_random_access(&data) != Some(false) {
|
||||
if st.decoder.as_ref().and_then(|d| d.is_random_access(&data)) != Some(false) {
|
||||
break;
|
||||
}
|
||||
key = s.sync_before(key - 1);
|
||||
@@ -337,7 +348,10 @@ impl GopCache {
|
||||
}
|
||||
let limit = (i.max(st.next) + 64).min(n);
|
||||
let late = late_before.unwrap_or(i64::MIN).min(want_pts);
|
||||
st.decoder.as_mut().expect("decoder").set_draft(draft);
|
||||
let Some(decoder) = st.decoder.as_mut() else {
|
||||
return Err(CodecError::Decode("no video decoder".into()));
|
||||
};
|
||||
decoder.set_draft(draft);
|
||||
while st.next < limit {
|
||||
if filmcraft_media::cancel::cancelled() {
|
||||
// The decoder state stays consistent (`next`, `out_max`): a later request continues.
|
||||
@@ -345,12 +359,15 @@ impl GopCache {
|
||||
}
|
||||
let k = st.next;
|
||||
let data = s.read(k)?;
|
||||
if k != i && s.pts(k) < late && st.decoder.as_ref().expect("decoder").is_disposable(&data) {
|
||||
if k != i && s.pts(k) < late && st.decoder.as_ref().is_some_and(|d| d.is_disposable(&data)) {
|
||||
st.next += 1;
|
||||
SKIPPED.fetch_add(1, Ordering::Relaxed);
|
||||
continue;
|
||||
}
|
||||
let out = timed(|| st.decoder.as_mut().expect("decoder").decode(&data, s.pts(k)))?;
|
||||
let Some(decoder) = st.decoder.as_mut() else {
|
||||
return Err(CodecError::Decode("no video decoder".into()));
|
||||
};
|
||||
let out = timed(|| decoder.decode(&data, s.pts(k)))?;
|
||||
st.next += 1;
|
||||
DECODED.fetch_add(1, Ordering::Relaxed);
|
||||
self.store_output(&mut st, out);
|
||||
@@ -359,7 +376,7 @@ impl GopCache {
|
||||
}
|
||||
}
|
||||
if st.cached(want_pts, draft).is_none() {
|
||||
let out = timed(|| st.decoder.as_mut().expect("decoder").flush());
|
||||
let out = st.decoder.as_mut().map(|d| timed(|| d.flush())).unwrap_or_default();
|
||||
self.store_output(&mut st, out);
|
||||
st.next = usize::MAX;
|
||||
}
|
||||
@@ -414,6 +431,7 @@ impl GopCache {
|
||||
let k = *next;
|
||||
let data = s.read(k)?;
|
||||
let pics = timed(|| d.decode(&data, s.pts(k)))?;
|
||||
count_frames(&pics);
|
||||
*next += 1;
|
||||
found |= pics.iter().any(|p| p.pts == want_pts);
|
||||
out.extend(pics);
|
||||
@@ -422,7 +440,9 @@ impl GopCache {
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
out.extend(timed(|| d.flush()));
|
||||
let pics = timed(|| d.flush());
|
||||
count_frames(&pics);
|
||||
out.extend(pics);
|
||||
*next = usize::MAX;
|
||||
}
|
||||
Ok(out)
|
||||
@@ -446,7 +466,9 @@ impl GopCache {
|
||||
if st.spare.len() < 16 {
|
||||
st.spare.push(dec);
|
||||
}
|
||||
for d in res? {
|
||||
let res = res?;
|
||||
count_frames(&res);
|
||||
for d in res {
|
||||
self.store(&mut st, d.pts, d.frame, d.draft);
|
||||
}
|
||||
st.at_or_before(want_pts, true).ok_or_else(|| CodecError::Decode("frame not produced".into()))
|
||||
|
||||
@@ -0,0 +1,350 @@
|
||||
//! Support for hardware (OS) video decoders registered by `filmcraft-platform`: the
|
||||
//! Settings ▸ Playback ▸ Hardware decoding switch, the hardware counters `perf.stats` reports, and
|
||||
//! what such a decoder needs from an `avcC` / `hvcC` sample entry to behave exactly like our
|
||||
//! software decoder for the same stream (parameter sets, cropping, colour, pixel aspect, reorder
|
||||
//! depth, random-access and disposable samples).
|
||||
//!
|
||||
//! The software decoders stay the reference: a hardware factory declines streams it cannot
|
||||
//! decode, and a hardware decoder that fails mid-stream switches to
|
||||
//! [`crate::software_video_decoder`], so registering one never makes a file undecodable.
|
||||
|
||||
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
|
||||
|
||||
use filmcraft_isobmff::{CodecConfig, SampleEntry};
|
||||
|
||||
use crate::video::{avcc_length_size, h264_disposable, hevc_disposable, hvcc_length_size_and_tid, sar_par, vui_color};
|
||||
use crate::{CodecError, Result};
|
||||
|
||||
static ENABLED: AtomicBool = AtomicBool::new(true);
|
||||
|
||||
/// Settings ▸ Playback ▸ Hardware decoding: `Auto` (true, the default) lets registered hardware
|
||||
/// decoders take streams they support; `Off` (false) makes every new decoder a software one.
|
||||
/// Decoders already created keep running: a source picks the change up when it is reopened.
|
||||
pub fn set_hardware_decoding(on: bool) {
|
||||
ENABLED.store(on, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// Whether hardware decoders may be used for new decoders ([`set_hardware_decoding`]).
|
||||
pub fn hardware_decoding() -> bool {
|
||||
ENABLED.load(Ordering::Relaxed)
|
||||
}
|
||||
|
||||
/// Process-wide hardware decoding counters (they only grow; subtract two snapshots to measure an
|
||||
/// interval). Frames decoded in software are `GopStats::frames - frames`.
|
||||
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
|
||||
pub struct HwStats {
|
||||
/// Pictures output by hardware decoders.
|
||||
pub frames: u64,
|
||||
/// Hardware decoders created (one per decoder instance; a seek reuses it).
|
||||
pub sessions: u64,
|
||||
/// Streams a hardware factory handed to the software decoder (unsupported profile, size,
|
||||
/// chroma format or bit depth, no hardware, or the switch is Off).
|
||||
pub declined: u64,
|
||||
/// Hardware decoders that failed mid-stream and continued in software.
|
||||
pub fallbacks: u64,
|
||||
}
|
||||
|
||||
static FRAMES: AtomicU64 = AtomicU64::new(0);
|
||||
static SESSIONS: AtomicU64 = AtomicU64::new(0);
|
||||
static DECLINED: AtomicU64 = AtomicU64::new(0);
|
||||
static FALLBACKS: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
/// The counters so far.
|
||||
pub fn hw_stats() -> HwStats {
|
||||
HwStats {
|
||||
frames: FRAMES.load(Ordering::Relaxed),
|
||||
sessions: SESSIONS.load(Ordering::Relaxed),
|
||||
declined: DECLINED.load(Ordering::Relaxed),
|
||||
fallbacks: FALLBACKS.load(Ordering::Relaxed),
|
||||
}
|
||||
}
|
||||
|
||||
/// A hardware decoder output `n` pictures.
|
||||
pub fn note_hw_frames(n: usize) {
|
||||
FRAMES.fetch_add(n as u64, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// A hardware decoder was created.
|
||||
pub fn note_hw_session() {
|
||||
SESSIONS.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// A hardware factory declined a stream it is responsible for.
|
||||
pub fn note_hw_declined() {
|
||||
DECLINED.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// A hardware decoder switched to software mid-stream.
|
||||
pub fn note_hw_fallback() {
|
||||
FALLBACKS.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// The two codecs whose `avcC` / `hvcC` streams hardware decoders take.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum NalCodec {
|
||||
H264,
|
||||
Hevc,
|
||||
}
|
||||
|
||||
/// What a decoder needs to know about a length-prefixed H.264 / HEVC stream, from its sample
|
||||
/// entry's configuration record and active sequence parameter set (parsed with the software
|
||||
/// decoders' own parsers, so both decoders derive the same values).
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct NalStreamInfo {
|
||||
pub codec: NalCodec,
|
||||
/// NAL unit length-prefix size of the samples (1, 2 or 4).
|
||||
pub length_size: usize,
|
||||
/// HEVC: highest TemporalId (numTemporalLayers - 1), when signalled.
|
||||
pub highest_tid: Option<u8>,
|
||||
/// The parameter-set NAL units of the record (header included, emulation prevention kept),
|
||||
/// in record order: H.264 SPS then PPS; HEVC VPS, SPS, PPS. SEI arrays are left out.
|
||||
pub parameter_sets: Vec<Vec<u8>>,
|
||||
/// Coded (decoded) picture size in luma samples.
|
||||
pub coded: (u32, u32),
|
||||
/// Output (cropped) rectangle in luma samples: x, y, width, height.
|
||||
pub crop: (u32, u32, u32, u32),
|
||||
/// 0 monochrome, 1 4:2:0, 2 4:2:2, 3 4:4:4.
|
||||
pub chroma_format_idc: u32,
|
||||
pub bit_depth_luma: u32,
|
||||
pub bit_depth_chroma: u32,
|
||||
/// Field (PAFF / MBAFF) coding (H.264 `frame_mbs_only_flag` 0).
|
||||
pub interlaced: bool,
|
||||
/// H.264 `profile_idc` / HEVC `general_profile_idc`.
|
||||
pub profile_idc: u8,
|
||||
/// Colour of every output picture, as the software decoder reports it.
|
||||
pub color: filmcraft_color::ColorInfo,
|
||||
/// Pixel aspect ratio, as the software decoder reports it.
|
||||
pub par: (u32, u32),
|
||||
/// Pictures that may precede a picture in decoding order and follow it in output order
|
||||
/// (H.264 `max_num_reorder_frames`, HEVC `sps_max_num_reorder_pics`).
|
||||
pub reorder: usize,
|
||||
}
|
||||
|
||||
/// Length-prefixed parameter-set entries (`u16` length + bytes) starting at `pos`.
|
||||
fn read_sets(rec: &[u8], pos: &mut usize, count: usize, out: &mut Vec<Vec<u8>>) -> Result<()> {
|
||||
for _ in 0..count {
|
||||
let len = rec
|
||||
.get(*pos..*pos + 2)
|
||||
.map(|b| u16::from_be_bytes([b[0], b[1]]) as usize)
|
||||
.ok_or_else(|| CodecError::Decode("configuration record truncated".into()))?;
|
||||
*pos += 2;
|
||||
let nal = rec.get(*pos..*pos + len).ok_or_else(|| CodecError::Decode("configuration record truncated".into()))?;
|
||||
*pos += len;
|
||||
if !nal.is_empty() {
|
||||
out.push(nal.to_vec());
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
impl NalStreamInfo {
|
||||
/// The stream info of an `avcC` / `hvcC` sample entry (`None` for other codecs).
|
||||
pub fn from_entry(e: &SampleEntry) -> Option<Result<Self>> {
|
||||
match &e.codec {
|
||||
CodecConfig::Avc(a) => Some(Self::from_avcc(&a.to_bytes())),
|
||||
CodecConfig::Hevc(c) => Some(Self::from_hvcc(&c.to_bytes())),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// From an `avcC` (AVCDecoderConfigurationRecord) payload.
|
||||
pub fn from_avcc(avcc: &[u8]) -> Result<Self> {
|
||||
if avcc.len() < 7 || avcc[0] != 1 {
|
||||
return Err(CodecError::Decode("bad avcC record".into()));
|
||||
}
|
||||
let mut sets = Vec::new();
|
||||
let mut pos = 6;
|
||||
read_sets(avcc, &mut pos, (avcc[5] & 0x1f) as usize, &mut sets)?;
|
||||
let nsps = sets.len();
|
||||
let npps = *avcc.get(pos).ok_or_else(|| CodecError::Decode("avcC truncated".into()))? as usize;
|
||||
pos += 1;
|
||||
read_sets(avcc, &mut pos, npps, &mut sets)?;
|
||||
let sps_nal = sets.iter().take(nsps).find(|n| n.first().is_some_and(|h| h & 0x1f == 7)).ok_or_else(|| CodecError::Decode("avcC has no SPS".into()))?;
|
||||
if sets.len() == nsps {
|
||||
return Err(CodecError::Decode("avcC has no PPS".into()));
|
||||
}
|
||||
let sps = filmcraft_h264::params::Sps::parse(&filmcraft_bitstream::unescape_rbsp(sps_nal.get(1..).unwrap_or_default()))
|
||||
.map_err(|e| CodecError::Decode(e.to_string()))?;
|
||||
let crop = sps.crop_rect();
|
||||
let vui = sps.vui.clone().unwrap_or_default();
|
||||
Ok(Self {
|
||||
codec: NalCodec::H264,
|
||||
length_size: avcc_length_size(avcc),
|
||||
highest_tid: None,
|
||||
parameter_sets: sets,
|
||||
coded: (sps.width(), sps.height()),
|
||||
crop,
|
||||
chroma_format_idc: sps.chroma_format_idc,
|
||||
bit_depth_luma: sps.bit_depth_luma,
|
||||
bit_depth_chroma: sps.bit_depth_chroma,
|
||||
interlaced: !sps.frame_mbs_only,
|
||||
profile_idc: sps.profile_idc,
|
||||
color: vui_color(crop.2, crop.3, vui.matrix_coefficients, vui.transfer_characteristics, vui.full_range),
|
||||
par: sar_par(vui.sar),
|
||||
// as the software decoder's DPB (`max_reorder`)
|
||||
reorder: sps.max_num_reorder_frames().min(sps.max_dpb_frames()),
|
||||
})
|
||||
}
|
||||
|
||||
/// From an `hvcC` (HEVCDecoderConfigurationRecord) payload.
|
||||
pub fn from_hvcc(hvcc: &[u8]) -> Result<Self> {
|
||||
if hvcc.len() < 23 || hvcc[0] != 1 {
|
||||
return Err(CodecError::Decode("bad hvcC record".into()));
|
||||
}
|
||||
let (length_size, highest_tid) = hvcc_length_size_and_tid(hvcc);
|
||||
let mut sets = Vec::new();
|
||||
let mut pos = 23;
|
||||
for _ in 0..hvcc[22] {
|
||||
let hdr = hvcc.get(pos..pos + 3).ok_or_else(|| CodecError::Decode("hvcC truncated".into()))?;
|
||||
let (kind, n) = (hdr[0] & 0x3f, u16::from_be_bytes([hdr[1], hdr[2]]) as usize);
|
||||
pos += 3;
|
||||
let mut arr = Vec::new();
|
||||
read_sets(hvcc, &mut pos, n, &mut arr)?;
|
||||
if (32..=34).contains(&kind) {
|
||||
sets.extend(arr);
|
||||
}
|
||||
}
|
||||
let nal_type = |n: &[u8]| n.first().map(|h| (h >> 1) & 0x3f);
|
||||
let sps_nal = sets.iter().find(|n| nal_type(n) == Some(33)).ok_or_else(|| CodecError::Decode("hvcC has no SPS".into()))?;
|
||||
if !sets.iter().any(|n| nal_type(n) == Some(32)) || !sets.iter().any(|n| nal_type(n) == Some(34)) {
|
||||
return Err(CodecError::Decode("hvcC lacks a VPS or PPS".into()));
|
||||
}
|
||||
let sps = filmcraft_hevc::params::Sps::parse(&filmcraft_bitstream::unescape_rbsp(sps_nal.get(2..).unwrap_or_default()))
|
||||
.map_err(|e| CodecError::Decode(e.to_string()))?;
|
||||
let crop = sps.crop_rect();
|
||||
let vui = sps.vui.clone().unwrap_or_default();
|
||||
Ok(Self {
|
||||
codec: NalCodec::Hevc,
|
||||
length_size,
|
||||
highest_tid,
|
||||
parameter_sets: sets,
|
||||
coded: (sps.width, sps.height),
|
||||
crop,
|
||||
chroma_format_idc: sps.chroma_format_idc,
|
||||
bit_depth_luma: sps.bit_depth_luma,
|
||||
bit_depth_chroma: sps.bit_depth_chroma,
|
||||
interlaced: false,
|
||||
profile_idc: sps.ptl.profile_idc,
|
||||
color: vui_color(crop.2, crop.3, vui.matrix_coefficients, vui.transfer_characteristics, vui.full_range),
|
||||
par: sar_par(vui.sar),
|
||||
reorder: sps.max_num_reorder as usize,
|
||||
})
|
||||
}
|
||||
|
||||
/// NAL unit types of a sample (stops at the first malformed length prefix).
|
||||
pub fn nal_types(&self, sample: &[u8]) -> Vec<u8> {
|
||||
self.nals(sample).into_iter().filter_map(|n| n.first().map(|&h| self.nal_type(h))).collect()
|
||||
}
|
||||
|
||||
/// The NAL units of a length-prefixed sample (stops at the first malformed length prefix).
|
||||
pub fn nals<'a>(&self, sample: &'a [u8]) -> Vec<&'a [u8]> {
|
||||
let ls = self.length_size;
|
||||
let mut out = Vec::new();
|
||||
let mut pos = 0usize;
|
||||
while let Some(pre) = sample.get(pos..pos.saturating_add(ls)) {
|
||||
if !(1..=4).contains(&ls) {
|
||||
break;
|
||||
}
|
||||
let len = pre.iter().fold(0usize, |a, &b| (a << 8) | b as usize);
|
||||
pos += ls;
|
||||
let Some(n) = sample.get(pos..pos.saturating_add(len)) else { break };
|
||||
pos += len;
|
||||
out.push(n);
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// The NAL unit type in a NAL header byte.
|
||||
pub fn nal_type(&self, header: u8) -> u8 {
|
||||
match self.codec {
|
||||
NalCodec::H264 => header & 0x1f,
|
||||
NalCodec::Hevc => (header >> 1) & 0x3f,
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether `t` is a parameter-set NAL unit type (H.264 SPS / PPS / SPS extension, HEVC VPS /
|
||||
/// SPS / PPS).
|
||||
pub fn is_parameter_set(&self, t: u8) -> bool {
|
||||
match self.codec {
|
||||
NalCodec::H264 => matches!(t, 7 | 8 | 13 | 15),
|
||||
NalCodec::Hevc => (32..=34).contains(&t),
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether decoding can restart at `sample` with no earlier state: an IDR access unit (H.264)
|
||||
/// or an IRAP one (HEVC IDR / CRA / BLA).
|
||||
pub fn is_irap(&self, sample: &[u8]) -> bool {
|
||||
self.nal_types(sample).iter().any(|&t| match self.codec {
|
||||
NalCodec::H264 => t == 5,
|
||||
NalCodec::Hevc => (16..=23).contains(&t),
|
||||
})
|
||||
}
|
||||
|
||||
/// As the software decoder's [`crate::VideoDecoder::is_random_access`] for this stream
|
||||
/// (length-prefixed samples: unknown, the container's sync flags are trusted).
|
||||
pub fn is_random_access(&self, _sample: &[u8]) -> Option<bool> {
|
||||
None
|
||||
}
|
||||
|
||||
/// As the software decoder's [`crate::VideoDecoder::is_disposable`] for this stream.
|
||||
pub fn is_disposable(&self, sample: &[u8]) -> bool {
|
||||
match self.codec {
|
||||
NalCodec::H264 => h264_disposable(sample, self.length_size),
|
||||
NalCodec::Hevc => hevc_disposable(sample, self.length_size, self.highest_tid),
|
||||
}
|
||||
}
|
||||
|
||||
/// Output picture size (cropped).
|
||||
pub fn size(&self) -> (u32, u32) {
|
||||
(self.crop.2, self.crop.3)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn switch_defaults_to_auto() {
|
||||
// other tests in this process may flip it; only check the round trip
|
||||
let was = hardware_decoding();
|
||||
set_hardware_decoding(false);
|
||||
assert!(!hardware_decoding());
|
||||
set_hardware_decoding(true);
|
||||
assert!(hardware_decoding());
|
||||
set_hardware_decoding(was);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hostile_records_are_errors() {
|
||||
for rec in [&[][..], &[1, 2, 3], &[1, 0x64, 0, 0x1f, 0xff, 0xe1, 0, 9, 0x67], &[0; 30]] {
|
||||
assert!(NalStreamInfo::from_avcc(rec).is_err());
|
||||
assert!(NalStreamInfo::from_hvcc(rec).is_err());
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn nal_splitting_is_bounded() {
|
||||
let info = NalStreamInfo {
|
||||
codec: NalCodec::Hevc,
|
||||
length_size: 4,
|
||||
highest_tid: None,
|
||||
parameter_sets: Vec::new(),
|
||||
coded: (16, 16),
|
||||
crop: (0, 0, 16, 16),
|
||||
chroma_format_idc: 1,
|
||||
bit_depth_luma: 8,
|
||||
bit_depth_chroma: 8,
|
||||
interlaced: false,
|
||||
profile_idc: 1,
|
||||
color: filmcraft_color::ColorInfo::REC709,
|
||||
par: (1, 1),
|
||||
reorder: 0,
|
||||
};
|
||||
// CRA (21) then a length that runs past the end
|
||||
let s = [0, 0, 0, 2, 21 << 1, 1, 0xff, 0xff, 0xff, 0xff, 1];
|
||||
assert_eq!(info.nal_types(&s), vec![21]);
|
||||
assert!(info.is_irap(&s));
|
||||
assert!(!info.is_irap(&[0, 0, 0, 2, 1 << 1, 1]));
|
||||
}
|
||||
}
|
||||
+27
-12
@@ -16,8 +16,11 @@
|
||||
//! - [`AudioFileSource`]: standalone compressed audio files (MP3, FLAC, AIFF, …).
|
||||
//! - [`openers`]: the openers to register with the engine's media pool.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
pub mod audio;
|
||||
pub mod gop;
|
||||
pub mod hw;
|
||||
pub mod mkv;
|
||||
pub mod mp4;
|
||||
pub mod mpeg;
|
||||
@@ -61,20 +64,21 @@ impl From<CodecError> for filmcraft_media::MediaError {
|
||||
}
|
||||
}
|
||||
|
||||
/// Our own (pure-Rust) decoders, in the order they are tried.
|
||||
const BUILTIN_FACTORIES: [VideoDecoderFactory; 8] = [
|
||||
video::h264_factory,
|
||||
video::hevc_factory,
|
||||
video::vp9_factory,
|
||||
video::av1_factory,
|
||||
video::prores_factory,
|
||||
video::dnx_factory,
|
||||
video::mjpeg_factory,
|
||||
video::mpeg2_factory,
|
||||
];
|
||||
|
||||
fn factories() -> &'static RwLock<Vec<VideoDecoderFactory>> {
|
||||
static F: std::sync::OnceLock<RwLock<Vec<VideoDecoderFactory>>> = std::sync::OnceLock::new();
|
||||
F.get_or_init(|| {
|
||||
RwLock::new(vec![
|
||||
video::h264_factory,
|
||||
video::hevc_factory,
|
||||
video::vp9_factory,
|
||||
video::av1_factory,
|
||||
video::prores_factory,
|
||||
video::dnx_factory,
|
||||
video::mjpeg_factory,
|
||||
video::mpeg2_factory,
|
||||
])
|
||||
})
|
||||
F.get_or_init(|| RwLock::new(BUILTIN_FACTORIES.to_vec()))
|
||||
}
|
||||
|
||||
/// Register a video decoder factory (tried before previously registered ones).
|
||||
@@ -96,6 +100,17 @@ pub fn make_video_decoder(entry: &filmcraft_isobmff::SampleEntry) -> Result<Box<
|
||||
Err(CodecError::Unsupported(format!("no decoder for {} video", entry.codec.name())))
|
||||
}
|
||||
|
||||
/// Create one of our own (software) decoders for a sample entry, skipping registered factories:
|
||||
/// what a hardware decoder falls back to when it fails mid-stream.
|
||||
pub fn software_video_decoder(entry: &filmcraft_isobmff::SampleEntry) -> Result<Box<dyn VideoDecoder>> {
|
||||
for f in BUILTIN_FACTORIES {
|
||||
if let Some(r) = f(entry) {
|
||||
return r;
|
||||
}
|
||||
}
|
||||
Err(CodecError::Unsupported(format!("no decoder for {} video", entry.codec.name())))
|
||||
}
|
||||
|
||||
/// Openers for the engine's media pool (MP4/MOV, Matroska/WebM, MXF, Ogg Opus/Vorbis, MPEG TS/PS and
|
||||
/// MPEG-1/2 video elementary streams, standalone audio).
|
||||
pub fn openers() -> Vec<filmcraft_media::Opener> {
|
||||
|
||||
@@ -369,9 +369,9 @@ impl MkvSource {
|
||||
if let Some(p) = st.packets.get(&i) {
|
||||
return Ok(p.clone());
|
||||
}
|
||||
let ti = self.atrack.expect("audio");
|
||||
let ti = self.atrack.ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
|
||||
let track = &self.file.tracks[ti];
|
||||
let ainfo = self.info.audio.as_ref().expect("audio info");
|
||||
let ainfo = self.info.audio.as_ref().ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
|
||||
let data = self.read(ti, i)?;
|
||||
let decoded = match &track.codec {
|
||||
Codec::Pcm { float, big_endian, bits } => {
|
||||
@@ -392,7 +392,7 @@ impl MkvSource {
|
||||
// Non-sequential access: reset and prime with the preceding packets (codec pre-roll:
|
||||
// one packet, or `SeekPreRoll` worth for Opus).
|
||||
if st.last_decoded.is_none_or(|l| l + 1 != i) {
|
||||
let d = st.decoder.as_mut().expect("decoder");
|
||||
let d = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?;
|
||||
d.reset();
|
||||
let from = if self.audio_preroll > 0 {
|
||||
self.audio_starts.partition_point(|&x| x <= self.audio_starts[i] - self.audio_preroll).saturating_sub(1)
|
||||
@@ -405,7 +405,7 @@ impl MkvSource {
|
||||
}
|
||||
}
|
||||
}
|
||||
let r = st.decoder.as_mut().expect("decoder").decode(&data, self.audio_starts[i].max(0) as u64);
|
||||
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, self.audio_starts[i].max(0) as u64);
|
||||
st.last_decoded = Some(i);
|
||||
r.unwrap_or_default()
|
||||
}
|
||||
|
||||
@@ -107,8 +107,11 @@ impl Mp4Source {
|
||||
pub fn open_reader(name: &str, reader: filmcraft_media::SharedReader) -> crate::Result<Self> {
|
||||
let bytes = crate::Src(reader);
|
||||
let file = filmcraft_isobmff::open(&bytes).map_err(|e| CodecError::Container(e.to_string()))?;
|
||||
let vtrack = file.tracks.iter().position(|t| t.kind == TrackKind::Video && !t.samples.is_empty());
|
||||
let atrack = file.tracks.iter().position(|t| t.kind == TrackKind::Audio && !t.samples.is_empty());
|
||||
// A track needs samples, a sample description and a timescale to be playable (damaged
|
||||
// files can lack them: indexing `entries[0]` or dividing by the timescale used to panic).
|
||||
let playable = |t: &&filmcraft_isobmff::Track, kind| t.kind == kind && !t.samples.is_empty() && !t.entries.is_empty() && t.timescale > 0;
|
||||
let vtrack = file.tracks.iter().position(|t| playable(&t, TrackKind::Video));
|
||||
let atrack = file.tracks.iter().position(|t| playable(&t, TrackKind::Audio));
|
||||
if vtrack.is_none() && atrack.is_none() {
|
||||
return Err(CodecError::Unsupported("no playable tracks".into()));
|
||||
}
|
||||
@@ -126,6 +129,8 @@ impl Mp4Source {
|
||||
durs.sort_unstable();
|
||||
let d = durs.get(durs.len() / 2).copied().unwrap_or(1).max(1);
|
||||
let rate = FrameRate::from_f64(t.timescale as f64 / d as f64);
|
||||
// A rate that rounds to zero (one sample per huge duration) can't be divided by.
|
||||
let rate = if rate.num > 0 && rate.den > 0 { rate } else { FrameRate::default() };
|
||||
color = color_from(entry, w, h);
|
||||
if entry
|
||||
.video
|
||||
@@ -264,9 +269,9 @@ impl Mp4Source {
|
||||
if let Some(p) = st.packets.get(&i) {
|
||||
return Ok(p.clone());
|
||||
}
|
||||
let ti = self.atrack.expect("audio");
|
||||
let ti = self.atrack.ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
|
||||
let track = &self.file.tracks[ti];
|
||||
let entry = &track.entries[0];
|
||||
let entry = track.entries.first().ok_or_else(|| CodecError::Container("audio track without a sample description".into()))?;
|
||||
let data = self.read(ti, i)?;
|
||||
let decoded = match &entry.codec {
|
||||
CodecConfig::Pcm(p) => decode_pcm(&data, p),
|
||||
@@ -277,7 +282,7 @@ impl Mp4Source {
|
||||
// Non-sequential access: reset and prime with the preceding packets (codec pre-roll:
|
||||
// one packet, or `OPUS_PRE_ROLL` worth for Opus).
|
||||
if st.last_decoded.is_none_or(|l| l + 1 != i) {
|
||||
let d = st.decoder.as_mut().expect("decoder");
|
||||
let d = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?;
|
||||
d.reset();
|
||||
let from = if self.audio_preroll > 0 {
|
||||
self.audio_starts.partition_point(|&x| x <= self.audio_starts[i] - self.audio_preroll).saturating_sub(1)
|
||||
@@ -290,7 +295,7 @@ impl Mp4Source {
|
||||
}
|
||||
}
|
||||
}
|
||||
let r = st.decoder.as_mut().expect("decoder").decode(&data, self.audio_starts[i].max(0) as u64);
|
||||
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, self.audio_starts[i].max(0) as u64);
|
||||
st.last_decoded = Some(i);
|
||||
match r {
|
||||
Ok(v) => v,
|
||||
@@ -513,6 +518,33 @@ mod tests {
|
||||
b.into()
|
||||
}
|
||||
|
||||
/// Overwrite the big-endian u32 `skip` bytes after the first `fourcc` box type.
|
||||
fn patch_u32(b: &mut [u8], fourcc: &[u8; 4], skip: usize, v: u32) {
|
||||
let k = b.windows(4).position(|x| x == fourcc).expect("box") + 4 + skip;
|
||||
b[k..k + 4].copy_from_slice(&v.to_be_bytes());
|
||||
}
|
||||
|
||||
/// Damaged MOVs found by mutation fuzzing panicked on import: a track with timescale 0
|
||||
/// ("attempt to divide by zero" in the duration) and one whose sample description list is
|
||||
/// empty ("index out of bounds" on `entries[0]`). Both must open as an error, or as media
|
||||
/// that answers requests without panicking.
|
||||
#[test]
|
||||
fn damaged_track_headers_never_panic() {
|
||||
let identity = [0x10000, 0, 0, 0, 0x10000, 0, 0, 0, 0x4000_0000];
|
||||
// mdhd v0: version/flags, creation, modification, then timescale
|
||||
let mut zero_timescale = rotated_mov(identity).to_vec();
|
||||
patch_u32(&mut zero_timescale, b"mdhd", 12, 0);
|
||||
// stsd: version/flags, then entry_count
|
||||
let mut no_entries = rotated_mov(identity).to_vec();
|
||||
patch_u32(&mut no_entries, b"stsd", 4, 0);
|
||||
for (name, b) in [("ts0.mov", zero_timescale), ("stsd0.mov", no_entries)] {
|
||||
if let Ok(s) = Mp4Source::open(name, b.into()) {
|
||||
let _ = s.video_frame(FrameRequest::full(Tick::ZERO));
|
||||
let _ = s.audio(0, 1024, 48_000);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Mean luma of the top and bottom halves of the decoded frame, and its size.
|
||||
fn halves(src: &Mp4Source) -> (u32, u32, u8, u8) {
|
||||
let f = src.video_frame(FrameRequest::full(Tick::ZERO)).expect("frame");
|
||||
|
||||
+41
-15
@@ -123,6 +123,19 @@ fn audio_supported(c: &Codec) -> bool {
|
||||
matches!(c, Codec::MpegAudio | Codec::AacAdts | Codec::AacLatm | Codec::LpcmBluray | Codec::LpcmDvd | Codec::Ac3)
|
||||
}
|
||||
|
||||
/// The coded rate unless the timestamps say otherwise (3:2 pulldown, field-rate coding). Never
|
||||
/// zero: damaged timestamps spread over a huge span give a rate that rounds to 0 fps, which the
|
||||
/// duration computation divided by.
|
||||
fn stream_rate(coded: Option<FrameRate>, from_pts: Option<f64>) -> FrameRate {
|
||||
let rate = match (coded, from_pts) {
|
||||
(Some(r), Some(p)) if (r.as_f64() - p).abs() / p > 0.01 => FrameRate::from_f64(p),
|
||||
(Some(r), _) => r,
|
||||
(None, Some(p)) => FrameRate::from_f64(p),
|
||||
(None, None) => FrameRate::FPS_25,
|
||||
};
|
||||
if rate.num > 0 && rate.den > 0 { rate } else { FrameRate::FPS_25 }
|
||||
}
|
||||
|
||||
/// Display-order keys of samples (decode order): the PTS when every unit has one, else (MPEG
|
||||
/// video) GOP start + temporal_reference, else the decode order.
|
||||
fn display_keys(units: &[&Unit], mpeg: bool) -> Vec<i64> {
|
||||
@@ -242,8 +255,9 @@ impl MpegSource {
|
||||
src.file = Some(file);
|
||||
src.video = Some(track);
|
||||
src.describe_video(rate_from_pts)?;
|
||||
let file = src.file.take().expect("file");
|
||||
src.setup_audio(file, astream, origin);
|
||||
if let Some(file) = src.file.take() {
|
||||
src.setup_audio(file, astream, origin);
|
||||
}
|
||||
}
|
||||
None => {
|
||||
src.unsupported_video = Some(format!("{} video in {} (FilmCraft has no decoder for it)", st.codec.name(), file.format.name()));
|
||||
@@ -369,14 +383,14 @@ impl MpegSource {
|
||||
if let Some((data, aus)) = &self.es {
|
||||
return Ok(data[aus[u].clone()].to_vec());
|
||||
}
|
||||
let file = self.file.as_ref().expect("file");
|
||||
let file = self.file.as_ref().ok_or_else(|| CodecError::Container("no stream file".into()))?;
|
||||
file.read_unit(&self.bytes, v.stream, u).map_err(|e| CodecError::Container(e.to_string()))
|
||||
}
|
||||
|
||||
/// Fill in the video stream info from the first picture.
|
||||
fn describe_video(&mut self, rate_from_pts: Option<f64>) -> crate::Result<()> {
|
||||
let mut seq_header = None;
|
||||
let v = self.video.as_ref().expect("video");
|
||||
let v = self.video.as_ref().ok_or_else(|| CodecError::Unsupported("no video stream".into()))?;
|
||||
let first = self.read_unit(v, 0)?;
|
||||
let n = v.order.len();
|
||||
let (width, height, par, codec, pixel_format, mut color);
|
||||
@@ -432,12 +446,7 @@ impl MpegSource {
|
||||
}
|
||||
}
|
||||
// the coded rate unless the timestamps say otherwise (3:2 pulldown, field-rate coding)
|
||||
let rate = match (rate, rate_from_pts) {
|
||||
(Some(r), Some(p)) if (r.as_f64() - p).abs() / p > 0.01 => FrameRate::from_f64(p),
|
||||
(Some(r), _) => r,
|
||||
(None, Some(p)) => FrameRate::from_f64(p),
|
||||
(None, None) => FrameRate::FPS_25,
|
||||
};
|
||||
let rate = stream_rate(rate, rate_from_pts);
|
||||
let duration = rate.tick_of(n as i64);
|
||||
let total_bytes: u64 = match (&self.es, &self.file) {
|
||||
(Some((_, aus)), _) => v.units.iter().map(|&u| aus[u].len() as u64).sum(),
|
||||
@@ -445,8 +454,10 @@ impl MpegSource {
|
||||
_ => 0,
|
||||
};
|
||||
let secs = duration.seconds();
|
||||
if let Some(h) = seq_header {
|
||||
self.video.as_mut().expect("video").header = h;
|
||||
if let Some(h) = seq_header
|
||||
&& let Some(v) = self.video.as_mut()
|
||||
{
|
||||
v.header = h;
|
||||
}
|
||||
self.info.duration = duration;
|
||||
self.info.video = Some(VideoStreamInfo {
|
||||
@@ -525,7 +536,7 @@ impl MpegSource {
|
||||
return self.audio_unsupported(&codec_name, "DVD LPCM: no format header".into());
|
||||
}
|
||||
},
|
||||
_ => unreachable!("audio_supported"),
|
||||
_ => return self.audio_unsupported(&codec_name, format!("{codec_name} audio")),
|
||||
};
|
||||
if codec_is_ac3_unsupported(codec) {
|
||||
return self.audio_unsupported(&codec_name, format!("{codec_name} audio (FilmCraft has no AC-3 decoder yet)"));
|
||||
@@ -608,12 +619,12 @@ impl MpegSource {
|
||||
if let Some(p) = st.cache.get(&i) {
|
||||
return Ok(p.clone());
|
||||
}
|
||||
let file = self.file.as_ref().expect("file");
|
||||
let file = self.file.as_ref().ok_or_else(|| CodecError::Container("no stream file".into()))?;
|
||||
let read = |k: usize| file.read_unit(&self.bytes, a.stream, k).map_err(|e| CodecError::Container(e.to_string()));
|
||||
if st.dec.is_none() {
|
||||
st.dec = Some(AudioDecoder::new(a.codec, a.rate, a.channels, a.latm.clone())?);
|
||||
}
|
||||
let dec = st.dec.as_mut().expect("decoder");
|
||||
let dec = st.dec.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?;
|
||||
if st.last.is_none_or(|l| l + 1 != i) {
|
||||
dec.reset();
|
||||
if i > 0
|
||||
@@ -840,6 +851,21 @@ pub fn reader_opener(name: &str, head: &[u8], reader: &filmcraft_media::SharedRe
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// A mutated transport stream whose PTS span gave ~0 fps panicked on open with "attempt to
|
||||
/// divide by zero" (`FrameRate::tick_of`).
|
||||
#[test]
|
||||
fn damaged_timestamps_never_give_a_zero_rate() {
|
||||
for (coded, p) in
|
||||
[(None, Some(1e-9)), (Some(FrameRate::FPS_25), Some(1e-9)), (Some(FrameRate::new(0, 0)), None), (None, Some(0.0)), (None, Some(f64::NAN))]
|
||||
{
|
||||
let r = stream_rate(coded, p);
|
||||
assert!(r.num > 0 && r.den > 0, "{coded:?} {p:?} -> {r:?}");
|
||||
let _ = r.tick_of(1000);
|
||||
}
|
||||
assert_eq!(stream_rate(Some(FrameRate::FPS_25), Some(25.0)), FrameRate::FPS_25);
|
||||
assert_eq!(stream_rate(None, Some(50.0)), FrameRate::from_f64(50.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn presentation_drops_leading_pictures_and_orders_by_key() {
|
||||
// decode order I(2) B(0) B(1) P(5) B(3) B(4): the leading B pictures are not presented
|
||||
|
||||
@@ -333,7 +333,7 @@ impl MxfSource {
|
||||
let mut channels = 0;
|
||||
for &ai in &atracks {
|
||||
let t = &file.tracks[ai];
|
||||
let si = t.sound.as_ref().expect("sound info");
|
||||
let Some(si) = t.sound.as_ref() else { continue };
|
||||
if si.sample_rate != rate {
|
||||
continue;
|
||||
}
|
||||
@@ -424,7 +424,7 @@ impl VideoSamples for MxfVideo<'_> {
|
||||
return Err(CodecError::Unsupported(why.clone()));
|
||||
}
|
||||
let t = &self.src.file.tracks[self.track];
|
||||
let v = self.src.info.video.as_ref().expect("video info");
|
||||
let v = self.src.info.video.as_ref().ok_or_else(|| CodecError::Unsupported("no video track".into()))?;
|
||||
let (w, h) = (v.width.min(u16::MAX as u32) as u16, v.height.min(u16::MAX as u32) as u16);
|
||||
match t.codec {
|
||||
Codec::Avc { .. } => Ok(Box::new(crate::video::H264Decoder::annexb())),
|
||||
|
||||
@@ -133,7 +133,7 @@ impl OggSource {
|
||||
}
|
||||
// Non-sequential access: reset and decode the pre-roll (RFC 7845 §4.6; see OPUS_PRE_ROLL).
|
||||
if st.last_decoded.is_none_or(|l| l + 1 != i) {
|
||||
let d = st.decoder.as_mut().expect("decoder");
|
||||
let d = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?;
|
||||
d.reset();
|
||||
let from = timing.starts.partition_point(|&x| x <= timing.starts[i] - OPUS_PRE_ROLL as i64).saturating_sub(1);
|
||||
for j in from..i {
|
||||
@@ -143,7 +143,7 @@ impl OggSource {
|
||||
}
|
||||
}
|
||||
let data = read(i)?;
|
||||
let r = st.decoder.as_mut().expect("decoder").decode(&data, 0);
|
||||
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, 0);
|
||||
st.last_decoded = Some(i);
|
||||
let p = Arc::new(r.unwrap_or_default());
|
||||
st.packets.insert(i, p.clone());
|
||||
|
||||
@@ -88,6 +88,33 @@ fn mp3_file_decodes() {
|
||||
assert!(a.peaks()[0] > 0.1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn aiff_file_decodes() {
|
||||
// 0.1 s of 48 kHz 16-bit stereo PCM: left a ramp, right silent (AIFF, big-endian samples)
|
||||
let frames = 4800u32;
|
||||
let mut b = b"FORM".to_vec();
|
||||
b.extend((4 + 26 + 16 + frames * 4).to_be_bytes());
|
||||
b.extend(b"AIFFCOMM");
|
||||
b.extend(18u32.to_be_bytes());
|
||||
b.extend(2u16.to_be_bytes());
|
||||
b.extend(frames.to_be_bytes());
|
||||
b.extend(16u16.to_be_bytes());
|
||||
b.extend([0x40, 0x0E, 0xBB, 0x80, 0, 0, 0, 0, 0, 0]); // 48000 as an 80-bit extended float
|
||||
b.extend(b"SSND");
|
||||
b.extend((8 + frames * 4).to_be_bytes());
|
||||
b.extend([0; 8]);
|
||||
for i in 0..frames {
|
||||
b.extend(((i as i16) * 4).to_be_bytes());
|
||||
b.extend(0i16.to_be_bytes());
|
||||
}
|
||||
let src = crate::open_bytes("tone.aif", b.into()).unwrap();
|
||||
assert!((src.info().duration.seconds() - 0.1).abs() < 1e-3, "{}", src.info().duration.seconds());
|
||||
let a = src.audio(0, 4800, 48_000).unwrap();
|
||||
assert_eq!(a.channels.len(), 2);
|
||||
assert!((a.channels[0][1000] - 4000.0 / 32768.0).abs() < 1e-4, "{}", a.channels[0][1000]);
|
||||
assert_eq!(a.channels[1][1000], 0.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn seek_backwards_and_forwards_mjpeg() {
|
||||
let Some(b) = fixture("counter_mjpeg.mov", &["-f", "lavfi", "-i", "testsrc2=s=160x120:r=25:d=3", "-c:v", "mjpeg"]) else { return };
|
||||
@@ -445,7 +472,7 @@ fn opus_reference(name: &str) -> Option<Vec<f32>> {
|
||||
if !st.success() {
|
||||
return None;
|
||||
}
|
||||
Some(std::fs::read(out).ok()?.chunks_exact(4).map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect())
|
||||
Some(std::fs::read(out).ok()?.as_chunks::<4>().0.iter().map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect())
|
||||
}
|
||||
|
||||
fn snr_db(reference: &[f32], test: &[f32]) -> f64 {
|
||||
|
||||
+49
-32
@@ -128,6 +128,42 @@ pub(crate) fn tight_plane<T: Copy>(src: Vec<T>, stride: usize, w: usize, h: usiz
|
||||
out
|
||||
}
|
||||
|
||||
/// Colour of an H.264 / HEVC picture from its VUI code points (ITU-T H.273), as the decoders
|
||||
/// report it: BT.709 by default with the matrix guessed from the size, the signalled matrix and
|
||||
/// transfer when known, and full range when flagged. Shared by the software and hardware
|
||||
/// decoders so the two are interchangeable.
|
||||
pub fn vui_color(width: u32, height: u32, matrix: u8, transfer: u8, full_range: bool) -> filmcraft_color::ColorInfo {
|
||||
let mut color = filmcraft_color::ColorInfo { matrix: filmcraft_frame::default_matrix(width, height), ..filmcraft_color::ColorInfo::REC709 };
|
||||
if let Some(m) = filmcraft_color::Matrix::from_code(matrix) {
|
||||
color.matrix = m;
|
||||
}
|
||||
if let Some(t) = filmcraft_color::Transfer::from_code(transfer) {
|
||||
color.transfer = t;
|
||||
}
|
||||
if full_range {
|
||||
color.range = filmcraft_color::Range::Full;
|
||||
}
|
||||
color
|
||||
}
|
||||
|
||||
/// Pixel aspect ratio from a VUI sample aspect ratio ((0, 0) = unspecified = square).
|
||||
pub fn sar_par(sar: (u16, u16)) -> (u32, u32) {
|
||||
if sar.0 > 0 && sar.1 > 0 { (sar.0 as u32, sar.1 as u32) } else { (1, 1) }
|
||||
}
|
||||
|
||||
/// NAL length-prefix size of an `avcC` record (byte 4: lengthSizeMinusOne).
|
||||
pub fn avcc_length_size(avcc: &[u8]) -> usize {
|
||||
avcc.get(4).map_or(4, |b| (b & 3) as usize + 1)
|
||||
}
|
||||
|
||||
/// NAL length-prefix size and highest TemporalId of an `hvcC` record.
|
||||
pub fn hvcc_length_size_and_tid(hvcc: &[u8]) -> (usize, Option<u8>) {
|
||||
// hvcC byte 21: constantFrameRate(2) numTemporalLayers(3) temporalIdNested(1) lengthSizeMinusOne(2)
|
||||
let length_size = hvcc.get(21).map_or(4, |b| (b & 3) as usize + 1);
|
||||
let highest_tid = hvcc.get(21).map(|b| (b >> 3) & 7).filter(|&n| n > 0).map(|n| n - 1);
|
||||
(length_size, highest_tid)
|
||||
}
|
||||
|
||||
/// A factory returns `None` when it does not handle the entry.
|
||||
pub type VideoDecoderFactory = fn(&SampleEntry) -> Option<Result<Box<dyn VideoDecoder>>>;
|
||||
|
||||
@@ -169,7 +205,7 @@ pub struct H264Decoder {
|
||||
impl H264Decoder {
|
||||
pub fn new(avcc: Vec<u8>) -> Result<Self> {
|
||||
let dec = filmcraft_h264::Decoder::from_avcc(&avcc).map_err(|e| CodecError::Decode(e.to_string()))?;
|
||||
let length_size = avcc.get(4).map_or(4, |b| (b & 3) as usize + 1);
|
||||
let length_size = avcc_length_size(&avcc);
|
||||
Ok(Self { avcc, dec, length_size, draft: false })
|
||||
}
|
||||
/// A decoder for Annex B byte-stream samples (start codes, in-band parameter sets: MXF, TS).
|
||||
@@ -184,17 +220,8 @@ impl H264Decoder {
|
||||
let y = tight_plane(p.y, p.y_stride, w, h);
|
||||
let u = tight_plane(p.u, p.uv_stride, cw, ch);
|
||||
let v = tight_plane(p.v, p.uv_stride, cw, ch);
|
||||
let mut color = filmcraft_color::ColorInfo { matrix: filmcraft_frame::default_matrix(p.width, p.height), ..filmcraft_color::ColorInfo::REC709 };
|
||||
if let Some(m) = filmcraft_color::Matrix::from_code(p.color.matrix) {
|
||||
color.matrix = m;
|
||||
}
|
||||
if let Some(t) = filmcraft_color::Transfer::from_code(p.color.transfer) {
|
||||
color.transfer = t;
|
||||
}
|
||||
if p.color.full_range {
|
||||
color.range = filmcraft_color::Range::Full;
|
||||
}
|
||||
let par = if p.sar.0 > 0 && p.sar.1 > 0 { (p.sar.0 as u32, p.sar.1 as u32) } else { (1, 1) };
|
||||
let color = vui_color(p.width, p.height, p.color.matrix, p.color.transfer, p.color.full_range);
|
||||
let par = sar_par(p.sar);
|
||||
let frame = VideoFrame {
|
||||
width: p.width,
|
||||
height: p.height,
|
||||
@@ -265,9 +292,7 @@ pub struct HevcDecoder {
|
||||
impl HevcDecoder {
|
||||
pub fn new(hvcc: Vec<u8>) -> Result<Self> {
|
||||
let dec = filmcraft_hevc::Decoder::from_hvcc(&hvcc).map_err(|e| CodecError::Decode(e.to_string()))?;
|
||||
// hvcC byte 21: constantFrameRate(2) numTemporalLayers(3) temporalIdNested(1) lengthSizeMinusOne(2)
|
||||
let length_size = hvcc.get(21).map_or(4, |b| (b & 3) as usize + 1);
|
||||
let highest_tid = hvcc.get(21).map(|b| (b >> 3) & 7).filter(|&n| n > 0).map(|n| n - 1);
|
||||
let (length_size, highest_tid) = hvcc_length_size_and_tid(&hvcc);
|
||||
Ok(Self { hvcc, dec, length_size, highest_tid })
|
||||
}
|
||||
/// A decoder for Annex B byte-stream samples (start codes, in-band parameter sets: TS).
|
||||
@@ -299,17 +324,8 @@ impl HevcDecoder {
|
||||
alpha: None,
|
||||
},
|
||||
};
|
||||
let mut color = filmcraft_color::ColorInfo { matrix: filmcraft_frame::default_matrix(p.width, p.height), ..filmcraft_color::ColorInfo::REC709 };
|
||||
if let Some(m) = filmcraft_color::Matrix::from_code(p.color.matrix) {
|
||||
color.matrix = m;
|
||||
}
|
||||
if let Some(t) = filmcraft_color::Transfer::from_code(p.color.transfer) {
|
||||
color.transfer = t;
|
||||
}
|
||||
if p.color.full_range {
|
||||
color.range = filmcraft_color::Range::Full;
|
||||
}
|
||||
let par = if p.sar.0 > 0 && p.sar.1 > 0 { (p.sar.0 as u32, p.sar.1 as u32) } else { (1, 1) };
|
||||
let color = vui_color(p.width, p.height, p.color.matrix, p.color.transfer, p.color.full_range);
|
||||
let par = sar_par(p.sar);
|
||||
let frame = VideoFrame { width: p.width, height: p.height, data, color, par, pts: filmcraft_time::Tick::ZERO };
|
||||
DecodedFrame { pts: p.pts, frame, draft: p.draft }
|
||||
}
|
||||
@@ -386,7 +402,8 @@ impl Vp9Decoder {
|
||||
}
|
||||
}
|
||||
|
||||
fn convert(&self, p: filmcraft_vp9::Picture) -> DecodedFrame {
|
||||
/// `None` for a picture whose planes mix sample types (never produced by the decoder).
|
||||
fn convert(&self, p: filmcraft_vp9::Picture) -> Option<DecodedFrame> {
|
||||
use filmcraft_color::{Matrix, Primaries, Range};
|
||||
use filmcraft_frame::{Chroma, PixelData};
|
||||
use filmcraft_vp9::Plane;
|
||||
@@ -423,7 +440,7 @@ impl Vp9Decoder {
|
||||
}
|
||||
let mut frame = VideoFrame::rgba8(p.width, p.height, rgba);
|
||||
frame.color = filmcraft_color::ColorInfo { range: Range::Full, transfer: filmcraft_color::Transfer::Srgb, ..color };
|
||||
return DecodedFrame { pts, frame, draft };
|
||||
return Some(DecodedFrame { pts, frame, draft });
|
||||
}
|
||||
// 4:4:0 has no frame format of its own: chroma rows are repeated to 4:4:4.
|
||||
let (chroma, rows_440) = match (p.subsampling_x, p.subsampling_y) {
|
||||
@@ -455,22 +472,22 @@ impl Vp9Decoder {
|
||||
bits: p.bit_depth,
|
||||
alpha: None,
|
||||
},
|
||||
_ => unreachable!("VP9 planes share one sample type"),
|
||||
_ => return None,
|
||||
};
|
||||
// render_size (the intended display size) is not applied: the container's display
|
||||
// dimensions / pixel aspect describe the presentation.
|
||||
let par = (1, 1);
|
||||
DecodedFrame { pts, frame: VideoFrame { width: p.width, height: p.height, data, color, par, pts: filmcraft_time::Tick::ZERO }, draft }
|
||||
Some(DecodedFrame { pts, frame: VideoFrame { width: p.width, height: p.height, data, color, par, pts: filmcraft_time::Tick::ZERO }, draft })
|
||||
}
|
||||
}
|
||||
|
||||
impl VideoDecoder for Vp9Decoder {
|
||||
fn decode(&mut self, sample: &[u8], pts: i64) -> Result<Vec<DecodedFrame>> {
|
||||
let pics = self.dec.decode(sample, pts).map_err(|e| CodecError::Decode(e.to_string()))?;
|
||||
Ok(pics.into_iter().map(|p| self.convert(p)).collect())
|
||||
Ok(pics.into_iter().filter_map(|p| self.convert(p)).collect())
|
||||
}
|
||||
fn flush(&mut self) -> Vec<DecodedFrame> {
|
||||
self.dec.flush().into_iter().map(|p| self.convert(p)).collect()
|
||||
self.dec.flush().into_iter().filter_map(|p| self.convert(p)).collect()
|
||||
}
|
||||
fn reset(&mut self) {
|
||||
self.dec.reset();
|
||||
|
||||
@@ -48,7 +48,7 @@ pub fn ffmpeg_audio_f32(ff: &Path, file: &Path, extra: &[&str]) -> Vec<f32> {
|
||||
let mut args = vec!["-i", file.to_str().unwrap()];
|
||||
args.extend_from_slice(extra);
|
||||
args.extend_from_slice(&["-map", "0:a:0", "-f", "f32le", "-"]);
|
||||
ffmpeg_out(ff, &args).chunks_exact(4).map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect()
|
||||
ffmpeg_out(ff, &args).as_chunks::<4>().0.iter().map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect()
|
||||
}
|
||||
|
||||
/// Our frame's three planes widened to u16 (and their sizes).
|
||||
@@ -63,7 +63,7 @@ pub fn planes(f: &VideoFrame) -> Vec<Vec<u16>> {
|
||||
/// Split one raw frame of `ffmpeg_frames` into planes (`bps` bytes per sample, chroma `cw`×`ch`).
|
||||
pub fn raw_planes(raw: &[u8], w: usize, h: usize, cw: usize, ch: usize, bps: usize) -> Vec<Vec<u16>> {
|
||||
let get = |b: &[u8]| -> Vec<u16> {
|
||||
if bps == 1 { b.iter().map(|&v| v as u16).collect() } else { b.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]])).collect() }
|
||||
if bps == 1 { b.iter().map(|&v| v as u16).collect() } else { b.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect() }
|
||||
};
|
||||
let y = w * h * bps;
|
||||
let c = cw * ch * bps;
|
||||
|
||||
@@ -92,7 +92,7 @@ fn decode_both(ff: &Path, file: &Path, decoder: &[&str]) -> (OggSource, Vec<f32>
|
||||
let ch = a.channels as usize;
|
||||
let mut args: Vec<&str> = decoder.to_vec();
|
||||
args.extend_from_slice(&["-i", file.to_str().unwrap(), "-map", "0:a:0", "-f", "f32le", "-"]);
|
||||
let want: Vec<f32> = ffmpeg_out(ff, &args).chunks_exact(4).map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
|
||||
let want: Vec<f32> = ffmpeg_out(ff, &args).as_chunks::<4>().0.iter().map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
|
||||
let n = want.len() / ch;
|
||||
let frames = src.info().duration.to_units_floor(a.sample_rate as i64) as usize;
|
||||
assert_eq!(frames, n, "{}: our length = ffmpeg's ({} samples)", file.display(), n);
|
||||
|
||||
@@ -5,6 +5,8 @@
|
||||
//! The compositor works in **linear-light, premultiplied RGBA f32** in the sequence working space
|
||||
//! (Rec.709 primaries by default). Decoded frames carry [`ColorInfo`] so conversions are explicit.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::sync::OnceLock;
|
||||
|
||||
|
||||
@@ -38,7 +38,7 @@ let f = decode_frame_with(&sample, &DecodeOptions { threads: false, ..Default::d
|
||||
let hdr: FrameHeader = probe(&sample)?; // cid, raster, depth, chroma…
|
||||
let name = cid_name(hdr.cid); // "DNxHR HQX", "DNxHD 1080i 8-bit (CID 1242)"
|
||||
|
||||
let mut enc = Encoder::new(Profile::Hq, 1920, 1080); // or Encoder::with_config(EncoderConfig { .. })
|
||||
let mut enc = Encoder::new(Profile::Hq, 1920, 1080)?; // or Encoder::with_config(EncoderConfig { .. })
|
||||
let sample: Vec<u8> = enc.encode(&frame)?; // exactly enc.frame_size() bytes (CBR)
|
||||
```
|
||||
|
||||
|
||||
@@ -19,8 +19,7 @@ pub(crate) fn idct8x8(block: &mut [f32; 64]) {
|
||||
let mut nz = [false; 8];
|
||||
let mut dc_only = true;
|
||||
// Horizontal pass: t[r][x] = Σ_u F[r][u] · B[u][x]
|
||||
for r in 0..8 {
|
||||
let c: &[f32; 8] = block[r * 8..r * 8 + 8].try_into().unwrap();
|
||||
for (r, c) in block.as_chunks::<8>().0.iter().enumerate() {
|
||||
if c[1..].iter().all(|&v| v == 0.0) {
|
||||
if c[0] != 0.0 {
|
||||
t[r] = [c[0] * BASIS[0][0]; 8];
|
||||
|
||||
@@ -61,8 +61,9 @@ struct Prepared {
|
||||
}
|
||||
|
||||
impl Encoder {
|
||||
pub fn new(profile: Profile, width: u32, height: u32) -> Encoder {
|
||||
Encoder::with_config(EncoderConfig::new(profile, width, height)).expect("valid default configuration")
|
||||
/// An encoder with the default configuration; fails on rasters the profile can't code.
|
||||
pub fn new(profile: Profile, width: u32, height: u32) -> Result<Encoder> {
|
||||
Encoder::with_config(EncoderConfig::new(profile, width, height))
|
||||
}
|
||||
|
||||
pub fn with_config(cfg: EncoderConfig) -> Result<Encoder> {
|
||||
@@ -79,7 +80,7 @@ impl Encoder {
|
||||
if !ok_depth {
|
||||
return Err(Error::Input(format!("{} does not support {}-bit", cfg.profile.name(), cfg.bit_depth)));
|
||||
}
|
||||
let info = cid_info(cfg.profile.cid()).expect("RI CID");
|
||||
let info = cid_info(cfg.profile.cid()).ok_or_else(|| Error::Input(format!("no compression table for {}", cfg.profile.name())))?;
|
||||
let frame_size = cfg.profile.frame_size(cfg.width, cfg.height);
|
||||
let header_size = header::header_size_for(cfg.height);
|
||||
Ok(Encoder { cfg, info, frame_size, header_size, last_q: 8 })
|
||||
|
||||
@@ -127,7 +127,7 @@ fn probe_weights() {
|
||||
let px: Vec<f64> = if depth == 8 {
|
||||
out.stdout.iter().map(|&b| b as f64).collect()
|
||||
} else {
|
||||
out.stdout.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]]) as f64).collect()
|
||||
out.stdout.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]]) as f64).collect()
|
||||
};
|
||||
let (off, stride) = if blk == 0 { (0, w as usize) } else { ((w * h) as usize, w as usize / 2) };
|
||||
let mid = (1 << (depth - 1)) as f64;
|
||||
@@ -224,7 +224,7 @@ fn probe_reconstruction() {
|
||||
let px: Vec<f64> = if depth == 8 {
|
||||
out.stdout.iter().map(|&b| b as f64).collect()
|
||||
} else {
|
||||
out.stdout.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]]) as f64).collect()
|
||||
out.stdout.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]]) as f64).collect()
|
||||
};
|
||||
let mid = (1 << (depth - 1)) as f64;
|
||||
let mut sum = 0f64;
|
||||
|
||||
@@ -152,7 +152,7 @@ pub fn parse(d: &[u8]) -> Result<FrameHeader> {
|
||||
2 => ColorVolume::Bt2020Cl,
|
||||
_ => ColorVolume::OutOfBand,
|
||||
};
|
||||
let timecode = (d[0x30] & 0x80 != 0).then(|| d[0x31..0x39].try_into().unwrap());
|
||||
let timecode = if d[0x30] & 0x80 != 0 { d.get(0x31..).and_then(|t| t.first_chunk::<8>()).copied() } else { None };
|
||||
let mb_rows = be16(d, 0x16C);
|
||||
if width == 0 || lines == 0 || width > 16384 || lines > 16384 {
|
||||
return Err(Error::Invalid("raster size"));
|
||||
|
||||
@@ -11,6 +11,8 @@
|
||||
//!
|
||||
//! See the crate README for accuracy, performance and limitations.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
mod bits;
|
||||
mod dct;
|
||||
mod decode;
|
||||
|
||||
@@ -91,7 +91,7 @@ pub fn reference(ff: &Path, mov: &Path, pix_fmt: &str) -> Vec<u16> {
|
||||
.expect("run ffmpeg");
|
||||
assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr));
|
||||
if pix_fmt.ends_with("le") {
|
||||
out.stdout.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
|
||||
out.stdout.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
|
||||
} else {
|
||||
out.stdout.iter().map(|&b| b as u16).collect()
|
||||
}
|
||||
@@ -247,7 +247,7 @@ pub fn ffmpeg_decode(ff: &Path, mov: &Path, pix_fmt: &str) -> Vec<u16> {
|
||||
let err = String::from_utf8_lossy(&out.stderr).to_string();
|
||||
assert!(out.status.success() && err.trim().is_empty(), "ffmpeg failed: {err}");
|
||||
if pix_fmt.ends_with("le") {
|
||||
out.stdout.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
|
||||
out.stdout.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
|
||||
} else {
|
||||
out.stdout.iter().map(|&b| b as u16).collect()
|
||||
}
|
||||
@@ -270,8 +270,11 @@ pub fn source_frames(ff: &Path, name: &str, lavfi: &str, frames: u32, w: u32, h:
|
||||
std::fs::rename(&tmp, &path).unwrap();
|
||||
}
|
||||
let bytes = std::fs::read(&path).unwrap();
|
||||
let s: Vec<u16> =
|
||||
if depth == 8 { bytes.iter().map(|&b| b as u16).collect() } else { bytes.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]])).collect() };
|
||||
let s: Vec<u16> = if depth == 8 {
|
||||
bytes.iter().map(|&b| b as u16).collect()
|
||||
} else {
|
||||
bytes.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
|
||||
};
|
||||
let proto = Frame::new(w, h, chroma, depth, false);
|
||||
let per = frame_samples(&proto);
|
||||
assert_eq!(s.len(), per * frames as usize);
|
||||
|
||||
@@ -68,17 +68,20 @@ fn roundtrip_all_profiles() {
|
||||
|
||||
#[test]
|
||||
fn rejects_bad_input() {
|
||||
let mut enc = Encoder::new(Profile::Hq, 64, 64);
|
||||
let mut enc = Encoder::new(Profile::Hq, 64, 64).unwrap();
|
||||
assert!(enc.encode(&Frame::new(32, 64, ChromaFormat::Yuv422, 8, false)).is_err());
|
||||
assert!(enc.encode(&Frame::new(64, 64, ChromaFormat::Yuv444, 8, false)).is_err());
|
||||
assert!(Encoder::with_config(EncoderConfig { bit_depth: 10, ..EncoderConfig::new(Profile::Hq, 64, 64) }).is_err());
|
||||
assert!(decode_frame(&[0u8; 100]).is_err());
|
||||
assert!(decode_frame(&[0u8; 1000]).is_err());
|
||||
// An unsupported raster is an error, not a panic.
|
||||
assert!(Encoder::new(Profile::Hq, 0, 0).is_err());
|
||||
assert!(Encoder::new(Profile::Hq, 20_000, 64).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn corrupt_streams_never_panic() {
|
||||
let mut enc = Encoder::new(Profile::Hq, 256, 128);
|
||||
let mut enc = Encoder::new(Profile::Hq, 256, 128).unwrap();
|
||||
let good = enc.encode(&picture(256, 128, ChromaFormat::Yuv422, 8, 1)).unwrap();
|
||||
let mut seed = 0x1234_5678u32;
|
||||
let mut rnd = || {
|
||||
|
||||
@@ -105,8 +105,9 @@ pub fn merge_captions(seq: &mut Sequence, ids: &[ClipId]) -> Result<ClipId> {
|
||||
if idx.windows(2).any(|w| w[1] != w[0] + 1) {
|
||||
return Err(EditError::Other("captions to merge must be next to each other".into()));
|
||||
}
|
||||
let first = idx[0];
|
||||
let last = *idx.last().expect("non-empty");
|
||||
let (Some(&first), Some(&last)) = (idx.first(), idx.last()) else {
|
||||
return Err(EditError::Nothing);
|
||||
};
|
||||
let end = tr.captions[last].end();
|
||||
let text: Vec<String> = idx.iter().map(|&i| tr.captions[i].text.trim().to_string()).filter(|t| !t.is_empty()).collect();
|
||||
let keep = tr.captions[first].id;
|
||||
@@ -205,7 +206,7 @@ pub fn move_captions(seq: &mut Sequence, ids: &[ClipId], delta: Tick) -> Result<
|
||||
return Err(EditError::Nothing);
|
||||
}
|
||||
for tid in tracks {
|
||||
let tr = seq.caption_track_mut(tid).expect("checked");
|
||||
let Some(tr) = seq.caption_track_mut(tid) else { continue };
|
||||
for c in tr.captions.iter_mut().filter(|c| ids.contains(&c.id)) {
|
||||
c.start += d;
|
||||
}
|
||||
|
||||
@@ -15,6 +15,8 @@
|
||||
//!
|
||||
//! Keyframes are stored in media time, so trims and splits never need to move them.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
pub mod captions;
|
||||
pub mod multicam;
|
||||
pub mod through;
|
||||
@@ -181,6 +183,34 @@ pub fn remove_orphan_transitions(track: &mut Track) {
|
||||
});
|
||||
}
|
||||
|
||||
/// Where a transition sits: the incoming clip's start, or the outgoing clip's end for a fade to nothing.
|
||||
fn transition_anchor(track: &Track, tr: &Transition) -> Option<Tick> {
|
||||
match (tr.from, tr.to) {
|
||||
(_, Some(to)) => track.item(to).map(|i| i.start),
|
||||
(Some(from), None) => track.item(from).map(|i| i.end()),
|
||||
(None, None) => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Keep transitions on their edit points after an edit moved cuts (ripple trim, roll, slide, ripple
|
||||
/// speed change): each transition shifts by as much as its anchor clip edge moved.
|
||||
fn transitions_follow_cuts(before: &Sequence, after: &mut Sequence) {
|
||||
for track in after.all_tracks_mut() {
|
||||
let Some(old) = before.track(track.id) else { continue };
|
||||
let shifts: Vec<Tick> = track
|
||||
.transitions
|
||||
.iter()
|
||||
.map(|tr| match (transition_anchor(old, tr), transition_anchor(track, tr)) {
|
||||
(Some(a), Some(b)) => b - a,
|
||||
_ => Tick::ZERO,
|
||||
})
|
||||
.collect();
|
||||
for (tr, d) in track.transitions.iter_mut().zip(shifts) {
|
||||
tr.start += d;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn place(track: &mut Track, item: TrackItem) {
|
||||
let idx = track.items.partition_point(|i| i.start <= item.start);
|
||||
track.items.insert(idx, item);
|
||||
@@ -520,6 +550,7 @@ pub fn trim(seq: &mut Sequence, clip: ClipId, edge: Edge, mode: TrimMode, delta:
|
||||
tr.sort();
|
||||
}
|
||||
}
|
||||
transitions_follow_cuts(seq, &mut work);
|
||||
}
|
||||
work.check().map_err(EditError::Other)?;
|
||||
*seq = work;
|
||||
@@ -597,6 +628,7 @@ pub fn ripple_trim_group(seq: &mut Sequence, clips: &[ClipId], edge: Edge, delta
|
||||
}
|
||||
tr.sort();
|
||||
}
|
||||
transitions_follow_cuts(seq, &mut work);
|
||||
work.check().map_err(EditError::Other)?;
|
||||
*seq = work;
|
||||
Ok(d)
|
||||
@@ -620,6 +652,7 @@ pub fn roll(seq: &mut Sequence, left: ClipId, right: ClipId, delta: Tick, ctx: &
|
||||
return Ok(d);
|
||||
}
|
||||
let (ls, rs) = (l.speed.abs(), r.speed.abs());
|
||||
let before = seq.clone();
|
||||
{
|
||||
let (_, l) = seq.find_item_mut(left).ok_or(EditError::NoItem(left))?;
|
||||
l.duration += d;
|
||||
@@ -631,6 +664,7 @@ pub fn roll(seq: &mut Sequence, left: ClipId, right: ClipId, delta: Tick, ctx: &
|
||||
r.source_in += src_of(d, rs);
|
||||
}
|
||||
let _ = ls;
|
||||
transitions_follow_cuts(&before, seq);
|
||||
Ok(d)
|
||||
}
|
||||
|
||||
@@ -673,6 +707,7 @@ pub fn slide(seq: &mut Sequence, clip: ClipId, delta: Tick, ctx: &mut EditCtx) -
|
||||
if d == Tick::ZERO {
|
||||
return Ok(d);
|
||||
}
|
||||
let before = seq.clone();
|
||||
let t = seq.track_mut(tid).ok_or(EditError::NoTrack(tid))?;
|
||||
if let Some(p) = prev {
|
||||
let pi = t.item_mut(p.id).ok_or(EditError::NoItem(p.id))?;
|
||||
@@ -686,6 +721,7 @@ pub fn slide(seq: &mut Sequence, clip: ClipId, delta: Tick, ctx: &mut EditCtx) -
|
||||
}
|
||||
let me = t.item_mut(clip).ok_or(EditError::NoItem(clip))?;
|
||||
me.start += d;
|
||||
transitions_follow_cuts(&before, seq);
|
||||
Ok(d)
|
||||
}
|
||||
|
||||
@@ -728,6 +764,7 @@ pub fn set_speed(seq: &mut Sequence, clip: ClipId, speed: f64, reverse: bool, ri
|
||||
new_dur = new_dur.min(next_start - it.start);
|
||||
}
|
||||
let delta = new_dur - it.duration;
|
||||
let before = ripple.then(|| seq.clone());
|
||||
{
|
||||
let (_, it) = seq.find_item_mut(clip).ok_or(EditError::NoItem(clip))?;
|
||||
it.speed = speed;
|
||||
@@ -746,6 +783,9 @@ pub fn set_speed(seq: &mut Sequence, clip: ClipId, speed: f64, reverse: bool, ri
|
||||
}
|
||||
}
|
||||
}
|
||||
if let Some(before) = before {
|
||||
transitions_follow_cuts(&before, seq);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
|
||||
@@ -210,6 +210,53 @@ fn roll_slip_slide() {
|
||||
fx.seq.check().unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn transitions_stay_on_their_cuts() {
|
||||
let mut fx = Fx::new();
|
||||
let v1 = fx.v(0);
|
||||
let a = fx.put(v1, 0, 10, 10);
|
||||
let b = fx.put(v1, 10, 10, 10);
|
||||
let c = fx.put(v1, 20, 10, 10);
|
||||
let cross = |id, start, from, to| Transition {
|
||||
id: TransitionId(id),
|
||||
effect: filmcraft_project::find_effect("cross_dissolve").unwrap().instance(),
|
||||
start: f(start),
|
||||
duration: f(4),
|
||||
from,
|
||||
to,
|
||||
align: Default::default(),
|
||||
reverse: false,
|
||||
};
|
||||
let t = fx.seq.track_mut(v1).unwrap();
|
||||
t.transitions = vec![cross(1, 8, Some(a), Some(b)), cross(2, 18, Some(b), Some(c)), cross(3, 26, Some(c), None)];
|
||||
// every transition is centred on its cut (or ends with the clip it fades out)
|
||||
let check = |fx: &Fx, what: &str| {
|
||||
let t = fx.seq.track(v1).unwrap();
|
||||
for tr in &t.transitions {
|
||||
let cut = match tr.to {
|
||||
Some(to) => t.item(to).unwrap().start,
|
||||
None => t.item(tr.from.unwrap()).unwrap().end() - f(2),
|
||||
};
|
||||
assert_eq!(tr.start + f(2), cut, "{what}: transition {:?} left its cut", tr.id);
|
||||
}
|
||||
assert_eq!(t.transitions.len(), 3, "{what}");
|
||||
};
|
||||
let mut n = fx.next;
|
||||
trim(&mut fx.seq, a, Edge::Out, TrimMode::Ripple, f(4), &mut Fx::ctx(&mut n)).unwrap();
|
||||
check(&fx, "ripple trim out");
|
||||
trim(&mut fx.seq, b, Edge::In, TrimMode::Ripple, f(3), &mut Fx::ctx(&mut n)).unwrap();
|
||||
check(&fx, "ripple trim in");
|
||||
ripple_trim_group(&mut fx.seq, &[b], Edge::Out, -f(2), &mut Fx::ctx(&mut n)).unwrap();
|
||||
check(&fx, "ripple trim group");
|
||||
roll(&mut fx.seq, b, c, f(2), &mut Fx::ctx(&mut n)).unwrap();
|
||||
check(&fx, "roll");
|
||||
slide(&mut fx.seq, b, -f(3), &mut Fx::ctx(&mut n)).unwrap();
|
||||
check(&fx, "slide");
|
||||
set_speed(&mut fx.seq, b, 0.5, false, true, &mut Fx::ctx(&mut n)).unwrap();
|
||||
check(&fx, "ripple speed change");
|
||||
fx.seq.check().unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rate_stretch_and_speed() {
|
||||
let mut fx = Fx::new();
|
||||
|
||||
@@ -344,7 +344,9 @@ pub fn caption_blocks(words: &[SeqWord], rules: &CaptionRules, rate: FrameRate)
|
||||
continue;
|
||||
}
|
||||
if fits_line {
|
||||
*lines.last_mut().expect("non-empty") += 1 + len;
|
||||
if let Some(l) = lines.last_mut() {
|
||||
*l += 1 + len;
|
||||
}
|
||||
} else {
|
||||
lines.push(len);
|
||||
}
|
||||
@@ -362,7 +364,7 @@ pub fn caption_blocks(words: &[SeqWord], rules: &CaptionRules, rate: FrameRate)
|
||||
for g in groups {
|
||||
let mut text_lines: Vec<String> = vec![String::new()];
|
||||
for w in &words[g.clone()] {
|
||||
let l = text_lines.last_mut().expect("non-empty");
|
||||
let Some(l) = text_lines.last_mut() else { break };
|
||||
if l.is_empty() {
|
||||
l.push_str(&w.text);
|
||||
} else if l.chars().count() + 1 + w.text.chars().count() <= max_chars {
|
||||
|
||||
@@ -82,6 +82,11 @@ pub struct PlaybackPrefs {
|
||||
/// skip deblocking (faster; those frames are approximate). Never used for exports, renders or
|
||||
/// a paused frame. Off by default.
|
||||
pub draft_decode: bool,
|
||||
/// "Hardware decoding": `auto` (default: the system's hardware decoder for streams it
|
||||
/// supports, e.g. VideoToolbox H.264 / HEVC on macOS, falling back to FilmCraft's own decoder)
|
||||
/// or `off` (always FilmCraft's own decoders). Media already open keeps its decoder until it
|
||||
/// is reopened.
|
||||
pub hardware_decoding: String,
|
||||
}
|
||||
|
||||
impl Default for PlaybackPrefs {
|
||||
@@ -94,6 +99,7 @@ impl Default for PlaybackPrefs {
|
||||
enable_transmit: false,
|
||||
disable_video_in_background: true,
|
||||
draft_decode: false,
|
||||
hardware_decoding: "auto".into(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1097,7 +1097,7 @@ fn subclip_plan(s: &Session, p: &Value) -> Result<(ItemId, TimeRange, String, La
|
||||
} else {
|
||||
let item = item_p(p, "item").or_else(|| subclip_source(s)).ok_or_else(|| bad("clip.makeSubclip", "open a clip in the Source monitor"))?;
|
||||
let (root, m, sub) = media_root(&s.project, item).ok_or_else(|| bad("clip.makeSubclip", "only media clips can be subclipped"))?;
|
||||
let pi = s.project.item(item).expect("resolved");
|
||||
let pi = s.project.item(item).ok_or_else(|| EngineError::Other("no such item".into()))?;
|
||||
let rate = m.frame_rate();
|
||||
let still = s.prefs.timeline.still_duration(rate);
|
||||
let is_still = matches!(m.info.kind, filmcraft_media::MediaKind::Still);
|
||||
@@ -1246,7 +1246,7 @@ fn audio_channels(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let mut out = Vec::new();
|
||||
let mut maps = Vec::new();
|
||||
for i in &items {
|
||||
let m = s.project.item(*i).and_then(|it| it.as_media()).expect("filtered");
|
||||
let Some(m) = s.project.item(*i).and_then(|it| it.as_media()) else { continue };
|
||||
let n = m.info.audio.as_ref().map_or(2, |a| a.channels) as u16;
|
||||
let fmt = format.unwrap_or_else(|| m.interpret.audio_channels.as_ref().map_or(AudioChannels::Stereo, |a| a.format));
|
||||
let map = match &custom {
|
||||
@@ -1603,7 +1603,7 @@ fn extract_audio(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
})
|
||||
.ok_or_else(|| EngineError::Other("save the project first: extracted audio is written next to it".into()))?,
|
||||
};
|
||||
let pi = s.project.item(item).expect("resolved");
|
||||
let pi = s.project.item(item).ok_or_else(|| EngineError::Other("no such item".into()))?;
|
||||
let name = pi.name.clone();
|
||||
let stem = std::path::Path::new(&name).file_stem().map(|x| x.to_string_lossy().into_owned()).unwrap_or_else(|| name.clone());
|
||||
let src = s.media.full_res_source(&s.project, root, &*s.services).ok_or_else(|| EngineError::Other(format!("{name}: media is offline")))?;
|
||||
|
||||
@@ -33,7 +33,7 @@ fn lumetri_text(s: &Session, clip: ClipId, p: &str) -> String {
|
||||
fn mean(img: &filmcraft_render::Image) -> [f32; 3] {
|
||||
assert!(img.px.iter().any(|v| *v > 0.01), "blank render");
|
||||
let mut m = [0f64; 3];
|
||||
for p in img.px.chunks_exact(4) {
|
||||
for p in img.px.as_chunks::<4>().0 {
|
||||
for k in 0..3 {
|
||||
m[k] += p[k] as f64;
|
||||
}
|
||||
@@ -213,7 +213,7 @@ fn lumetri_presets_list_apply_and_thumbnails() {
|
||||
assert!(it.effects[k + 1..].iter().all(|e| e.def().is_some_and(|d| d.intrinsic)) || k + 1 == it.effects.len());
|
||||
assert_eq!(it.effects[k].param("saturation").unwrap().value, ParamValue::Float(0.0));
|
||||
let img = s.render_program(0.25).unwrap();
|
||||
let grey = img.px.chunks_exact(4).filter(|p| p[3] > 0.5).all(|p| (p[0] - p[1]).abs() < 0.01 && (p[1] - p[2]).abs() < 0.01);
|
||||
let grey = img.px.as_chunks::<4>().0.iter().filter(|p| p[3] > 0.5).all(|p| (p[0] - p[1]).abs() < 0.01 && (p[1] - p[2]).abs() < 0.01);
|
||||
assert!(grey, "the program is black and white");
|
||||
s.execute("edit.undo", json!({})).unwrap();
|
||||
assert_eq!(s.active_sequence().unwrap().find_item(clip).unwrap().1.effects.len(), before);
|
||||
@@ -242,7 +242,7 @@ fn hdr_lumetri_and_mastering_metadata_through_commands() {
|
||||
for k in 0..3 {
|
||||
assert!((m1[k] / m0[k] - 2.0).abs() < 0.08, "channel {k}: {m0:?} → {m1:?}");
|
||||
}
|
||||
assert!(up.px.chunks_exact(4).any(|p| p[1] > 1.2), "values above reference white survive");
|
||||
assert!(up.px.as_chunks::<4>().0.iter().any(|p| p[1] > 1.2), "values above reference white survive");
|
||||
// HDR White / HDR Specular / HDR Range are ordinary Lumetri parameters
|
||||
s.execute("effects.setParam", json!({"clip": clip.0, "effect": idx, "param": "hdr_white", "value": 2000.0})).unwrap();
|
||||
s.execute("effects.setParam", json!({"clip": clip.0, "effect": idx, "param": "hdr_specular", "value": -50.0})).unwrap();
|
||||
|
||||
@@ -362,7 +362,7 @@ pub(crate) fn place_item(
|
||||
let frame = (seq.settings.width, seq.settings.height);
|
||||
let mut placements = Vec::new();
|
||||
let link = if has_v && has_a { Some(p.alloc_id()) } else { None };
|
||||
if has_v {
|
||||
if let Some(vdest) = vdest.filter(|_| has_v) {
|
||||
let mut v = p.make_track_item(item, TrackKind::Video, at, range, rate).ok_or_else(|| bad(label, "bad item"))?;
|
||||
v.link = link;
|
||||
for e in &mut v.effects {
|
||||
@@ -371,10 +371,9 @@ pub(crate) fn place_item(
|
||||
if is_media && src_size != frame {
|
||||
crate::settings::apply_media_scaling(&mut v, &scaling, frame, src_size);
|
||||
}
|
||||
placements.push((vdest.expect("checked"), v));
|
||||
placements.push((vdest, v));
|
||||
}
|
||||
if has_a {
|
||||
let adest = adest.expect("checked");
|
||||
if let Some(adest) = adest.filter(|_| has_a) {
|
||||
let (arange, aat) = audio_split.unwrap_or((range, at));
|
||||
let mut a = p.make_track_item(item, TrackKind::Audio, aat, arange, rate).ok_or_else(|| bad(label, "bad item"))?;
|
||||
// Modify ▸ Audio Channels with several audio clips: one per clip, on the tracks below
|
||||
@@ -675,27 +674,27 @@ fn build() -> Vec<CommandSpec> {
|
||||
Ok(id) => ids.push(id.0),
|
||||
Err(e) => errors.push(format!("{path}: {e}")),
|
||||
},
|
||||
Ok(b) if crate::captions::detect(&path, &b).is_some() => {
|
||||
let fmt = crate::captions::detect(&path, &b).expect("detected");
|
||||
match crate::captions::import(s, &path, &b, fmt, None) {
|
||||
Ok(r) => reports.push(r),
|
||||
Err(e) => errors.push(format!("{path}: {e}")),
|
||||
}
|
||||
}
|
||||
Ok(b) if crate::interchange::detect(&path, &b).is_some() => {
|
||||
let fmt = crate::interchange::detect(&path, &b).expect("detected");
|
||||
match crate::interchange::import(s, &path, &b, fmt) {
|
||||
Ok(r) => {
|
||||
sequences.extend(r["sequences"].as_array().cloned().unwrap_or_default());
|
||||
reports.push(r);
|
||||
Ok(b) => {
|
||||
if let Some(fmt) = crate::captions::detect(&path, &b) {
|
||||
match crate::captions::import(s, &path, &b, fmt, None) {
|
||||
Ok(r) => reports.push(r),
|
||||
Err(e) => errors.push(format!("{path}: {e}")),
|
||||
}
|
||||
} else if let Some(fmt) = crate::interchange::detect(&path, &b) {
|
||||
match crate::interchange::import(s, &path, &b, fmt) {
|
||||
Ok(r) => {
|
||||
sequences.extend(r["sequences"].as_array().cloned().unwrap_or_default());
|
||||
reports.push(r);
|
||||
}
|
||||
Err(e) => errors.push(format!("{path}: {e}")),
|
||||
}
|
||||
} else {
|
||||
match import_bytes(s, &path, b.into(), bin) {
|
||||
Ok(id) => ids.push(id.0),
|
||||
Err(e) => errors.push(format!("{path}: {e}")),
|
||||
}
|
||||
Err(e) => errors.push(format!("{path}: {e}")),
|
||||
}
|
||||
}
|
||||
Ok(b) => match import_bytes(s, &path, b.into(), bin) {
|
||||
Ok(id) => ids.push(id.0),
|
||||
Err(e) => errors.push(format!("{path}: {e}")),
|
||||
},
|
||||
Err(e) => errors.push(format!("{path}: {e}")),
|
||||
}
|
||||
}
|
||||
@@ -1280,7 +1279,7 @@ fn build() -> Vec<CommandSpec> {
|
||||
let p = p.clone();
|
||||
s.edit("Sequence Settings", |pr, _| {
|
||||
if let Some(n) = str_p(&p, "name") {
|
||||
pr.item_mut(id).expect("seq").name = n.to_string();
|
||||
pr.item_mut(id).ok_or(EngineError::NoSequence)?.name = n.to_string();
|
||||
}
|
||||
let q = pr.sequence_mut(id).ok_or(EngineError::NoSequence)?;
|
||||
if let Some(w) = u64_p(&p, "width") {
|
||||
@@ -1793,7 +1792,21 @@ fn build() -> Vec<CommandSpec> {
|
||||
.map(Tick)
|
||||
.or_else(|| p.get("deltaFrames").and_then(Value::as_i64).map(|f| s.sequence_rate().tick_of(f)))
|
||||
.unwrap_or_default();
|
||||
s.edit_sequence("Slide", |q, ctx, _| Ok(edit::slide(q, c, d, ctx)?))?;
|
||||
let clips = with_links(s, &[c]);
|
||||
s.edit_sequence("Slide", |q, ctx, _| {
|
||||
// clamp across all linked partners, then slide them by the common delta
|
||||
let mut dd = d;
|
||||
for c in &clips {
|
||||
let x = edit::slide(&mut q.clone(), *c, dd, ctx)?;
|
||||
if x.abs() < dd.abs() {
|
||||
dd = x;
|
||||
}
|
||||
}
|
||||
for c in &clips {
|
||||
edit::slide(q, *c, dd, ctx)?;
|
||||
}
|
||||
Ok(())
|
||||
})?;
|
||||
Ok(Value::Null)
|
||||
}),
|
||||
cmd!("timeline.rateStretch", "Rate Stretch", [], None, r#"{"clip":id,"edge":"in|out","delta":ticks}"#, has_seq, |s, p| {
|
||||
@@ -2440,7 +2453,7 @@ fn nest(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
s.edit("Nest", |pr, st| {
|
||||
let nid = pr.new_sequence(&name, q.settings.clone(), q.video_tracks.len(), q.audio_tracks.len(), None);
|
||||
{
|
||||
let nq = pr.sequence_mut(nid).expect("new");
|
||||
let nq = pr.sequence_mut(nid).ok_or(EngineError::NoSequence)?;
|
||||
for (k, ti, it) in &items {
|
||||
let mut it = it.clone();
|
||||
it.start -= start;
|
||||
@@ -2451,11 +2464,15 @@ fn nest(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
}
|
||||
}
|
||||
let rate = q.settings.frame_rate;
|
||||
let mut v = pr.make_track_item(nid, TrackKind::Video, start, TimeRange::new(Tick::ZERO, end - start), rate).expect("item");
|
||||
let mut v = pr
|
||||
.make_track_item(nid, TrackKind::Video, start, TimeRange::new(Tick::ZERO, end - start), rate)
|
||||
.ok_or_else(|| EngineError::Other("cannot place the nested sequence".into()))?;
|
||||
for e in &mut v.effects {
|
||||
resolve_auto_points(e, (q.settings.width, q.settings.height), (q.settings.width, q.settings.height));
|
||||
}
|
||||
let a = pr.make_track_item(nid, TrackKind::Audio, start, TimeRange::new(Tick::ZERO, end - start), rate).expect("item");
|
||||
let a = pr
|
||||
.make_track_item(nid, TrackKind::Audio, start, TimeRange::new(Tick::ZERO, end - start), rate)
|
||||
.ok_or_else(|| EngineError::Other("cannot place the nested sequence".into()))?;
|
||||
let link = pr.alloc_id();
|
||||
let seq = pr.sequence_mut(seq_id).ok_or(EngineError::NoSequence)?;
|
||||
edit::delete_items(seq, &sel);
|
||||
|
||||
+20
-11
@@ -44,6 +44,15 @@ pub fn add_generator(p: &mut Project, pool: &MediaPool, src: GeneratorSource, na
|
||||
|
||||
/// Build the demo project. Returns (project, main sequence).
|
||||
pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
|
||||
build_demo(pool).unwrap_or_else(|| {
|
||||
// Only reachable if the built-in items could not be placed: an empty demo, not a crash.
|
||||
let mut p = Project::new("FilmCraft Demo");
|
||||
let seq = p.new_sequence("Main Edit", SequenceSettings::default(), 3, 3, None);
|
||||
(p, seq)
|
||||
})
|
||||
}
|
||||
|
||||
fn build_demo(pool: &MediaPool) -> Option<(Project, ItemId)> {
|
||||
let mut p = Project::new("FilmCraft Demo");
|
||||
let footage = p.add_bin("Footage", None);
|
||||
let audio_bin = p.add_bin("Audio", None);
|
||||
@@ -108,8 +117,8 @@ pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
|
||||
for (ci, sin, dur) in cuts {
|
||||
let src_range = TimeRange::new(r.snap(Tick::from_seconds_f64(sin)), r.snap(Tick::from_seconds_f64(dur)));
|
||||
let link = p.alloc_id();
|
||||
let mut v = p.make_track_item(clips[ci], TrackKind::Video, t, src_range, r).expect("item");
|
||||
let mut a = p.make_track_item(clips[ci], TrackKind::Audio, t, src_range, r).expect("item");
|
||||
let mut v = p.make_track_item(clips[ci], TrackKind::Video, t, src_range, r)?;
|
||||
let mut a = p.make_track_item(clips[ci], TrackKind::Audio, t, src_range, r)?;
|
||||
v.link = Some(link);
|
||||
a.link = Some(link);
|
||||
for e in &mut v.effects {
|
||||
@@ -118,14 +127,14 @@ pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
|
||||
a.effect_mut("volume").map(|e| e.params.get_mut("level").map(|l| l.value = ParamValue::Float(-8.0)));
|
||||
let dur_t = v.duration;
|
||||
placed.push((v.id, a.id, t, dur_t));
|
||||
let s = p.sequence_mut(seq).expect("seq");
|
||||
let s = p.sequence_mut(seq)?;
|
||||
s.video_tracks[0].items.push(v);
|
||||
s.audio_tracks[0].items.push(a);
|
||||
t += dur_t;
|
||||
}
|
||||
// A lower-third-ish overlay on V2: the plasma clip scaled down in the corner with a drop shadow.
|
||||
let ov_range = TimeRange::new(Tick::ZERO, r.snap(Tick::from_seconds_f64(4.0)));
|
||||
let mut ov = p.make_track_item(clips[4], TrackKind::Video, r.snap(Tick::from_seconds_f64(6.0)), ov_range, r).expect("item");
|
||||
let mut ov = p.make_track_item(clips[4], TrackKind::Video, r.snap(Tick::from_seconds_f64(6.0)), ov_range, r)?;
|
||||
for e in &mut ov.effects {
|
||||
resolve_auto_points(e, (1920, 1080), (1920, 1080));
|
||||
}
|
||||
@@ -152,18 +161,18 @@ pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
|
||||
lum.params.get_mut("temperature").map(|v| v.value = ParamValue::Float(18.0));
|
||||
lum.params.get_mut("contrast").map(|v| v.value = ParamValue::Float(22.0));
|
||||
lum.params.get_mut("vignette_amount").map(|v| v.value = ParamValue::Float(-1.4));
|
||||
let s = p.sequence_mut(seq).expect("seq");
|
||||
let s = p.sequence_mut(seq)?;
|
||||
if let Some(it) = s.video_tracks[0].items.get_mut(3) {
|
||||
it.effects.insert(0, lum);
|
||||
}
|
||||
}
|
||||
let s = p.sequence_mut(seq).expect("seq");
|
||||
let s = p.sequence_mut(seq)?;
|
||||
s.video_tracks[1].items.push(ov);
|
||||
// Transitions: cross dissolve between shots 1-2, dip to black 3-4, push 5-6, fade in at start.
|
||||
let tr_len = r.tick_of(24);
|
||||
let mk = |id: u64, eff: &str, from: Option<filmcraft_project::ClipId>, to: Option<filmcraft_project::ClipId>, start: Tick| Transition {
|
||||
id: TransitionId(id),
|
||||
effect: find_effect(eff).expect("fx").instance(),
|
||||
effect: crate::presets::effect_instance(eff),
|
||||
start,
|
||||
duration: tr_len,
|
||||
from,
|
||||
@@ -181,7 +190,7 @@ pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
|
||||
s.video_tracks[0].transitions = trs;
|
||||
s.audio_tracks[0].transitions.push(Transition {
|
||||
id: TransitionId(900_010),
|
||||
effect: find_effect("constant_power").expect("fx").instance(),
|
||||
effect: crate::presets::effect_instance("constant_power"),
|
||||
start: cut(1) - tr_len.mul_ratio(1, 2),
|
||||
duration: tr_len,
|
||||
from: Some(placed[0].1),
|
||||
@@ -207,13 +216,13 @@ pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
|
||||
});
|
||||
}
|
||||
// Music on A2 (under the whole edit), ducked -14 dB.
|
||||
let mut m = p.make_track_item(music, TrackKind::Audio, Tick::ZERO, TimeRange::new(Tick::ZERO, end), r).expect("music");
|
||||
let mut m = p.make_track_item(music, TrackKind::Audio, Tick::ZERO, TimeRange::new(Tick::ZERO, end), r)?;
|
||||
m.effect_mut("volume").map(|e| e.params.get_mut("level").map(|l| l.value = ParamValue::Float(-14.0)));
|
||||
m.label = Label::Caribbean;
|
||||
let s = p.sequence_mut(seq).expect("seq");
|
||||
let s = p.sequence_mut(seq)?;
|
||||
s.audio_tracks[1].items.push(m);
|
||||
for tr in s.all_tracks_mut() {
|
||||
tr.sort();
|
||||
}
|
||||
(p, seq)
|
||||
Some((p, seq))
|
||||
}
|
||||
@@ -411,6 +411,9 @@ fn ranges_resolve_and_sidecar_captions_are_written() {
|
||||
assert_eq!(frames(&mut s, json!({"preset": "PNG Sequence"})), io, "In/Out is the default");
|
||||
assert!(s.execute("export.resolve", json!({"range": "workArea"})).is_err(), "no work area");
|
||||
assert_eq!(frames(&mut s, json!({"preset": "Animated GIF 640×360", "range": "custom", "startSeconds": 0, "endSeconds": 2})), 30, "15 fps");
|
||||
let bare = frames(&mut s, json!({"preset": "Animated GIF 640×360", "startSeconds": 0, "endSeconds": 2}));
|
||||
assert_eq!(bare, 30, "start/end times without `range` are a custom range, not the In/Out or whole sequence");
|
||||
assert!(s.execute("export.resolve", json!({"startSeconds": 1})).is_err(), "a start without an end is an error");
|
||||
let res = s.execute("export.resolve", json!({"preset": "YouTube 2160p 4K Ultra HD"})).unwrap();
|
||||
assert_eq!(res["output"]["width"], 3840);
|
||||
assert!(res["summary"]["video"].as_str().unwrap().contains("3840x2160"));
|
||||
|
||||
@@ -244,12 +244,14 @@ pub fn sequence_param(s: &Session, p: &Value, cmd: &str) -> Result<ItemId> {
|
||||
}
|
||||
|
||||
/// Export ▸ Range: `entire` | `inOut` | `workArea` | `custom` (`start…` / `end…` times).
|
||||
/// None = the default (In/Out when set, else the whole sequence).
|
||||
/// None = the default (In/Out when set, else the whole sequence); bare `start…` / `end…` times
|
||||
/// without a `range` mean `custom`.
|
||||
pub fn range_param(s: &Session, project: &Project, seq: ItemId, v: Option<&Value>, p: &Value, cmd: &str) -> Result<Option<TimeRange>> {
|
||||
let q = project.sequence(seq).ok_or(EngineError::NoSequence)?;
|
||||
let fd = q.settings.frame_rate.frame_duration();
|
||||
let whole = TimeRange::from_bounds(Tick::ZERO, q.duration().max(fd));
|
||||
let mode = match v {
|
||||
None | Some(Value::Null) if time_p(s, p, "start").is_some() || time_p(s, p, "end").is_some() => "custom".to_string(),
|
||||
None | Some(Value::Null) => return Ok(None),
|
||||
Some(Value::String(m)) => m.to_ascii_lowercase().replace(['-', '_', ' ', '/'], ""),
|
||||
Some(o @ Value::Object(_)) => return range_param(s, project, seq, o.get("mode").or(Some(&json!("custom"))), o, cmd),
|
||||
@@ -385,6 +387,7 @@ pub fn spawn_export(s: &mut Session, project: Arc<Project>, seq: ItemId, mut set
|
||||
}
|
||||
*res.lock().unwrap_or_else(|x| x.into_inner()) = Some(r);
|
||||
};
|
||||
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
|
||||
s.jobs.push(job);
|
||||
if wait || cfg!(target_arch = "wasm32") {
|
||||
run();
|
||||
@@ -394,6 +397,24 @@ pub fn spawn_export(s: &mut Session, project: Arc<Project>, seq: ItemId, mut set
|
||||
Ok(id)
|
||||
}
|
||||
|
||||
/// Last-resort guard for a background job body (export, previews, proxies, …): a panic marks the
|
||||
/// job failed with an error instead of killing its thread and leaving the job "running" forever.
|
||||
/// The panic hook has already logged where it happened.
|
||||
pub(crate) fn guard_job(
|
||||
progress: Arc<filmcraft_export::Progress>,
|
||||
result: Arc<std::sync::Mutex<Option<std::result::Result<filmcraft_export::Report, String>>>>,
|
||||
run: impl FnOnce() + Send + 'static,
|
||||
) -> impl FnOnce() + Send + 'static {
|
||||
move || {
|
||||
if std::panic::catch_unwind(std::panic::AssertUnwindSafe(run)).is_err() {
|
||||
let msg = "internal error (see the crash log); the job was stopped".to_string();
|
||||
*progress.error.lock().unwrap_or_else(|x| x.into_inner()) = Some(msg.clone());
|
||||
progress.finished.store(true, Ordering::Relaxed);
|
||||
*result.lock().unwrap_or_else(|x| x.into_inner()) = Some(Err(msg));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn file_name(path: &str) -> String {
|
||||
Path::new(path).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_else(|| path.to_string())
|
||||
}
|
||||
@@ -1008,3 +1029,20 @@ pub(crate) fn commands() -> Vec<CommandSpec> {
|
||||
spec("export.quick", "Quick Export", r#"{"preset":str?,"path":str?,"wait":bool?, …settings params}"#, has_seq, quick, true),
|
||||
]
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod guard_tests {
|
||||
use super::*;
|
||||
|
||||
/// A panicking job body must leave the job finished with an error, not "running" forever.
|
||||
#[test]
|
||||
fn panicking_job_is_reported_as_failed() {
|
||||
let progress: Arc<filmcraft_export::Progress> = Arc::default();
|
||||
let result: Arc<std::sync::Mutex<Option<std::result::Result<filmcraft_export::Report, String>>>> = Arc::default();
|
||||
let run = guard_job(progress.clone(), result.clone(), || panic!("job bug"));
|
||||
std::thread::spawn(run).join().expect("the guard catches the panic");
|
||||
assert!(progress.finished.load(Ordering::Relaxed));
|
||||
assert!(progress.error.lock().unwrap().is_some());
|
||||
assert!(matches!(&*result.lock().unwrap(), Some(Err(_))));
|
||||
}
|
||||
}
|
||||
@@ -157,15 +157,15 @@ pub fn thumbnail(t: &GraphicsTemplate, w: u32) -> (u32, u32, Vec<u8>) {
|
||||
let mut p = Project::new("thumb");
|
||||
let rate = filmcraft_time::FrameRate::FPS_30;
|
||||
let g = p.add_item("G", Label::Rose, ItemKind::Graphic { width: t.canvas[0], height: t.canvas[1], rate }, None);
|
||||
let mut it = p
|
||||
.make_track_item(
|
||||
g,
|
||||
filmcraft_project::TrackKind::Video,
|
||||
Tick::ZERO,
|
||||
filmcraft_time::TimeRange::new(Tick::ZERO, t.duration.max(rate.frame_duration())),
|
||||
rate,
|
||||
)
|
||||
.expect("item");
|
||||
let Some(mut it) = p.make_track_item(
|
||||
g,
|
||||
filmcraft_project::TrackKind::Video,
|
||||
Tick::ZERO,
|
||||
filmcraft_time::TimeRange::new(Tick::ZERO, t.duration.max(rate.frame_duration())),
|
||||
rate,
|
||||
) else {
|
||||
return (w, h, vec![0; w as usize * h as usize * 4]);
|
||||
};
|
||||
it.effects.extend(t.layers.iter().cloned());
|
||||
it.graphic = Some(Box::new(meta));
|
||||
// a moment into the clip, so intro animations have settled a little
|
||||
|
||||
@@ -551,11 +551,11 @@ macro_rules! arrange_cmd {
|
||||
fn new_shape_cmd(s: &mut Session, p: &Value, shape: &str) -> Result<Value> {
|
||||
let mut q = p.clone();
|
||||
q["shape"] = json!(shape);
|
||||
(find_spec("graphics.newShape").run)(s, &q)
|
||||
(find_spec("graphics.newShape")?.run)(s, &q)
|
||||
}
|
||||
|
||||
fn find_spec(id: &str) -> CommandSpec {
|
||||
commands().into_iter().find(|c| c.id == id).expect("graphics command")
|
||||
fn find_spec(id: &str) -> Result<CommandSpec> {
|
||||
commands().into_iter().find(|c| c.id == id).ok_or_else(|| EngineError::UnknownCommand(id.to_string()))
|
||||
}
|
||||
|
||||
/// Graphics menu commands (New Layer shapes, Align/Distribute/Arrange/Select submenus, resets).
|
||||
@@ -576,7 +576,7 @@ fn menu_commands() -> Vec<CommandSpec> {
|
||||
|s, p| {
|
||||
let mut p = if p.is_object() { p.clone() } else { json!({}) };
|
||||
p["vertical"] = json!(true);
|
||||
(find_spec("graphics.newText").run)(s, &p)
|
||||
(find_spec("graphics.newText")?.run)(s, &p)
|
||||
},
|
||||
),
|
||||
spec(
|
||||
@@ -731,7 +731,7 @@ fn menu_commands() -> Vec<CommandSpec> {
|
||||
let rate = q.settings.frame_rate;
|
||||
let want = rate.snap_nearest(Tick::from_seconds_f64(seconds)).max(rate.frame_duration());
|
||||
let tr = q.video_tracks.iter_mut().find(|tr| tr.items.iter().any(|i| i.id == clip)).ok_or(filmcraft_edit::EditError::NoItem(clip))?;
|
||||
let i = tr.items.iter().position(|i| i.id == clip).expect("found");
|
||||
let i = tr.items.iter().position(|i| i.id == clip).ok_or(filmcraft_edit::EditError::NoItem(clip))?;
|
||||
let start = tr.items[i].start;
|
||||
let limit = tr.items.get(i + 1).map_or(want, |n| n.start - start);
|
||||
let d = want.min(limit).max(rate.frame_duration());
|
||||
|
||||
@@ -8,6 +8,8 @@
|
||||
//! The project is an `Arc<Project>` edited copy-on-write; undo keeps whole-project snapshots (cheap
|
||||
//! thanks to structural sharing of untouched items).
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
pub mod aaf_omf;
|
||||
pub mod autosave;
|
||||
pub mod captions;
|
||||
@@ -475,6 +477,7 @@ impl Session {
|
||||
pub fn start_autosave(&mut self, cfg: autosave::AutosaveConfig) -> std::io::Result<()> {
|
||||
let prefs_path = cfg.data_dir.join("preferences.json");
|
||||
self.prefs = autosave::Preferences::load(&prefs_path);
|
||||
self.prefs.apply_hardware_decoding();
|
||||
self.media.set_use_proxies(self.prefs.media.enable_proxies);
|
||||
self.shortcuts.set_dir(&cfg.data_dir);
|
||||
self.presets.set_dir(&cfg.data_dir);
|
||||
@@ -546,6 +549,7 @@ impl Session {
|
||||
pub fn set_prefs(&mut self, p: autosave::Preferences) -> std::io::Result<()> {
|
||||
let cache_changed = self.prefs.media_cache != p.media_cache;
|
||||
self.prefs = p;
|
||||
self.prefs.apply_hardware_decoding();
|
||||
if cache_changed {
|
||||
self.apply_media_cache();
|
||||
}
|
||||
@@ -599,7 +603,11 @@ impl Session {
|
||||
return Err(e);
|
||||
}
|
||||
self.exec_depth += 1;
|
||||
let r = (spec.run)(self, ¶ms);
|
||||
// Last-resort guard: a panic in a command (a bug that escaped the never-crash rules)
|
||||
// becomes an error for the caller (UI, CLI, control channel, MCP) instead of taking the
|
||||
// session and its unsaved project down. The panic hook has logged where it happened.
|
||||
let r = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| (spec.run)(self, ¶ms)))
|
||||
.unwrap_or_else(|_| Err(EngineError::Other(format!("internal error in `{id}` (see the crash log); the command did not complete"))));
|
||||
self.exec_depth -= 1;
|
||||
// source graphics: an edited instance updates the shared layers and the other instances
|
||||
if r.is_ok() && spec.journal && id.starts_with("graphics.") && !self.project.source_graphics.is_empty() {
|
||||
|
||||
@@ -634,6 +634,7 @@ fn track(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
prog.finished.store(true, Ordering::Relaxed);
|
||||
*lock(&res) = Some(r);
|
||||
};
|
||||
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
|
||||
s.jobs.push(job);
|
||||
s.mask_jobs.push(PendingTrack { job: id, target: MaskSel { clip, effect: ei, mask: mi }, seq: seq_id, keys, applied: 0 });
|
||||
let wait = crate::commands::bool_p(p, "wait").unwrap_or(false);
|
||||
|
||||
@@ -101,7 +101,7 @@ pub fn psnr(a: &[u8], b: &[u8]) -> f64 {
|
||||
assert_eq!(a.len(), b.len());
|
||||
let mut se = 0.0;
|
||||
let mut n = 0.0;
|
||||
for (x, y) in a.chunks_exact(4).zip(b.chunks_exact(4)) {
|
||||
for (x, y) in a.as_chunks::<4>().0.iter().zip(b.as_chunks::<4>().0) {
|
||||
for c in 0..3 {
|
||||
let d = x[c] as f64 - y[c] as f64;
|
||||
se += d * d;
|
||||
|
||||
@@ -554,7 +554,7 @@ fn merge_clips(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
a.link = Some(link);
|
||||
a_items.push(a);
|
||||
}
|
||||
let q = pr.sequence_mut(sid).expect("new sequence");
|
||||
let q = pr.sequence_mut(sid).ok_or(EngineError::NoSequence)?;
|
||||
for (t, it) in q.video_tracks.iter_mut().zip(v_items) {
|
||||
t.items.push(it);
|
||||
}
|
||||
@@ -616,7 +616,7 @@ fn create_multicam(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let (mut vi, mut ai) = (0usize, 0usize);
|
||||
let mut placed: Vec<(TrackKind, usize, TrackItem)> = Vec::new();
|
||||
let (vtracks, atracks): (Vec<TrackId>, Vec<TrackId>) = {
|
||||
let q = pr.sequence(sid).expect("new sequence");
|
||||
let q = pr.sequence(sid).ok_or(EngineError::NoSequence)?;
|
||||
(q.video_tracks.iter().map(|t| t.id).collect(), q.audio_tracks.iter().map(|t| t.id).collect())
|
||||
};
|
||||
for (k, c) in clips.iter().enumerate() {
|
||||
@@ -640,7 +640,7 @@ fn create_multicam(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
cameras.push(cam);
|
||||
}
|
||||
{
|
||||
let q = pr.sequence_mut(sid).expect("new sequence");
|
||||
let q = pr.sequence_mut(sid).ok_or(EngineError::NoSequence)?;
|
||||
for (kind, idx, it) in placed {
|
||||
q.tracks_mut(kind)[idx].items.push(it);
|
||||
}
|
||||
@@ -892,10 +892,14 @@ fn record_start(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
fn apply_pass(s: &mut Session, end: Tick) -> Result<usize> {
|
||||
let Some(seq_id) = s.mcrec.seq else { return Ok(0) };
|
||||
let merging = s.history.merge_key.as_deref() == Some(s.mcrec.key.as_str()) && s.history.undo.last().is_some_and(|u| u.0 == RECORD_LABEL);
|
||||
if !merging || s.mcrec.base.is_none() {
|
||||
s.mcrec.base = Some(s.project.clone());
|
||||
}
|
||||
let base = s.mcrec.base.clone().expect("set above");
|
||||
let base = match s.mcrec.base.clone() {
|
||||
Some(b) if merging => b,
|
||||
_ => {
|
||||
let b = s.project.clone();
|
||||
s.mcrec.base = Some(b.clone());
|
||||
b
|
||||
}
|
||||
};
|
||||
let (tracks, cuts, key) = (s.mcrec.tracks.clone(), s.mcrec.cuts.clone(), s.mcrec.key.clone());
|
||||
let Some(source) = s.mcrec.source else { return Ok(0) };
|
||||
let media = s.media.clone();
|
||||
|
||||
@@ -9,6 +9,7 @@ use crate::Session;
|
||||
/// measure an interval).
|
||||
pub fn decode_json() -> Value {
|
||||
let g = filmcraft_codecs::gop_stats();
|
||||
let hw = filmcraft_codecs::hw::hw_stats();
|
||||
json!({
|
||||
"requests": g.hits + g.misses,
|
||||
"cacheHits": g.hits,
|
||||
@@ -22,6 +23,17 @@ pub fn decode_json() -> Value {
|
||||
"evicted": g.evicted,
|
||||
"decodeMs": g.decode_ns as f64 / 1e6,
|
||||
"decodeMsPerSample": g.decode_ms_per_sample(),
|
||||
"framesDecoded": g.frames,
|
||||
// Settings ▸ Playback ▸ Hardware decoding: pictures from OS hardware decoders vs ours,
|
||||
// hardware decoders created, streams handed to software up front, mid-stream fallbacks.
|
||||
"hardware": {
|
||||
"enabled": filmcraft_codecs::hw::hardware_decoding(),
|
||||
"frames": hw.frames,
|
||||
"softwareFrames": g.frames.saturating_sub(hw.frames),
|
||||
"sessions": hw.sessions,
|
||||
"declined": hw.declined,
|
||||
"fallbacks": hw.fallbacks,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
@@ -48,9 +60,24 @@ mod tests {
|
||||
s.execute("file.openDemoProject", json!({})).unwrap();
|
||||
let undo = s.history.undo.len();
|
||||
let v = s.execute("perf.stats", json!({})).unwrap();
|
||||
for k in ["requests", "cacheHitRate", "seeks", "samplesDecoded", "samplesSkipped", "draftFrames", "h264Threads", "decodeMs", "decodeMsPerSample"] {
|
||||
for k in [
|
||||
"requests",
|
||||
"cacheHitRate",
|
||||
"seeks",
|
||||
"samplesDecoded",
|
||||
"samplesSkipped",
|
||||
"draftFrames",
|
||||
"h264Threads",
|
||||
"decodeMs",
|
||||
"decodeMsPerSample",
|
||||
"framesDecoded",
|
||||
] {
|
||||
assert!(v["decode"][k].is_number(), "decode.{k} in {v}");
|
||||
}
|
||||
for k in ["frames", "softwareFrames", "sessions", "declined", "fallbacks"] {
|
||||
assert!(v["decode"]["hardware"][k].is_number(), "decode.hardware.{k} in {v}");
|
||||
}
|
||||
assert!(v["decode"]["hardware"]["enabled"].is_boolean());
|
||||
assert!(v["media"]["openSources"].is_number());
|
||||
assert_eq!(v["jobs"]["running"], json!(0));
|
||||
assert_eq!(s.history.undo.len(), undo, "a query adds no undo step");
|
||||
|
||||
@@ -151,8 +151,22 @@ fn kf(t_secs: f64, v: ParamValue) -> Keyframe {
|
||||
Keyframe::new(Tick((t_secs * TICKS_PER_SECOND as f64).round() as i64), v)
|
||||
}
|
||||
|
||||
/// A fresh instance of a built-in effect (a bare instance named `id` if the registry lacks it,
|
||||
/// which the preset tests rule out).
|
||||
pub(crate) fn effect_instance(id: &str) -> EffectInstance {
|
||||
filmcraft_project::find_effect(id).map(|d| d.instance()).unwrap_or_else(|| EffectInstance {
|
||||
effect: id.to_string(),
|
||||
enabled: true,
|
||||
params: Default::default(),
|
||||
masks: Vec::new(),
|
||||
post_fader: false,
|
||||
essential: false,
|
||||
layer: None,
|
||||
})
|
||||
}
|
||||
|
||||
fn effect(id: &str, params: &[(&str, ParamValue)]) -> EffectInstance {
|
||||
let mut e = filmcraft_project::find_effect(id).unwrap_or_else(|| panic!("built-in preset uses unknown effect {id}")).instance();
|
||||
let mut e = effect_instance(id);
|
||||
for (k, v) in params {
|
||||
if let Some(p) = e.params.get_mut(*k) {
|
||||
p.value = v.clone();
|
||||
|
||||
@@ -543,6 +543,7 @@ pub fn render(s: &mut Session, mode: RenderMode, p: &Value) -> Result<Value> {
|
||||
prog.finished.store(true, Ordering::Relaxed);
|
||||
*res.lock().unwrap_or_else(|x| x.into_inner()) = Some(r);
|
||||
};
|
||||
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
|
||||
s.jobs.push(job);
|
||||
let wait = p.get("wait").and_then(Value::as_bool).unwrap_or(false);
|
||||
if wait || cfg!(target_arch = "wasm32") {
|
||||
@@ -629,6 +630,7 @@ pub fn render_audio(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
prog.finished.store(true, Ordering::Relaxed);
|
||||
*res.lock().unwrap_or_else(|x| x.into_inner()) = Some(r);
|
||||
};
|
||||
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
|
||||
s.jobs.push(job);
|
||||
if p.get("wait").and_then(Value::as_bool).unwrap_or(false) || cfg!(target_arch = "wasm32") {
|
||||
run();
|
||||
@@ -680,7 +682,7 @@ pub fn read_wav_f32(b: &[u8]) -> Option<Vec<f32>> {
|
||||
if !float_stereo {
|
||||
return None;
|
||||
}
|
||||
return Some(body.chunks_exact(4).map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect());
|
||||
return Some(body.as_chunks::<4>().0.iter().map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect());
|
||||
}
|
||||
i += 8 + len + (len & 1);
|
||||
}
|
||||
|
||||
@@ -229,6 +229,7 @@ pub fn run(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
prog.finished.store(true, Ordering::Relaxed);
|
||||
*res.lock().unwrap_or_else(|x| x.into_inner()) = Some(r);
|
||||
};
|
||||
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
|
||||
s.jobs.push(job);
|
||||
if bool_p(p, "wait").unwrap_or(false) || cfg!(target_arch = "wasm32") {
|
||||
run();
|
||||
|
||||
@@ -241,6 +241,7 @@ pub fn start_job(s: &mut Session, label: String, work: Vec<(ItemId, String, &'st
|
||||
prog.finished.store(true, std::sync::atomic::Ordering::Relaxed);
|
||||
*res.lock().unwrap_or_else(|x| x.into_inner()) = Some(r);
|
||||
};
|
||||
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
|
||||
s.jobs.push(job);
|
||||
s.media_jobs.push(PendingJob { job: id, on_done, outputs });
|
||||
if wait || cfg!(target_arch = "wasm32") {
|
||||
|
||||
@@ -35,12 +35,11 @@ pub fn fingerprint(size: u64, head: &[u8], tail: &[u8]) -> u64 {
|
||||
h ^= h >> 29;
|
||||
};
|
||||
for part in [head, tail] {
|
||||
let mut chunks = part.chunks_exact(8);
|
||||
for c in &mut chunks {
|
||||
mix(u64::from_le_bytes([c[0], c[1], c[2], c[3], c[4], c[5], c[6], c[7]]));
|
||||
let (chunks, r) = part.as_chunks::<8>();
|
||||
for c in chunks {
|
||||
mix(u64::from_le_bytes(*c));
|
||||
}
|
||||
let mut last = [0u8; 8];
|
||||
let r = chunks.remainder();
|
||||
last[..r.len()].copy_from_slice(r);
|
||||
mix(u64::from_le_bytes(last) ^ (r.len() as u64) << 56);
|
||||
mix(part.len() as u64);
|
||||
|
||||
@@ -206,6 +206,7 @@ fn detect(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
*lock(&res) = Some(r);
|
||||
prog.finished.store(true, Ordering::Relaxed);
|
||||
};
|
||||
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
|
||||
s.jobs.push(job);
|
||||
s.scene_jobs.push(PendingScene { job: id, seq: seq_id, apply, results: results.clone() });
|
||||
let wait = bool_p(p, "wait").unwrap_or(false);
|
||||
|
||||
@@ -381,7 +381,10 @@ fn make_subsequence(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let offset = range.map(|r| r.start).unwrap_or_else(|| items.iter().map(|i| i.2.start).min().unwrap_or_default());
|
||||
let name = match str_p(p, "name") {
|
||||
Some(n) => n.to_string(),
|
||||
None => (1..).map(|k| format!("{seq_name}_Sub_{k:02}")).find(|n| !s.project.items.values().any(|i| &i.name == n)).expect("free name"),
|
||||
None => (1..)
|
||||
.map(|k| format!("{seq_name}_Sub_{k:02}"))
|
||||
.find(|n| !s.project.items.values().any(|i| &i.name == n))
|
||||
.unwrap_or_else(|| format!("{seq_name}_Sub")),
|
||||
};
|
||||
let id = s.edit("Make Subsequence", |pr, st| {
|
||||
let nid = pr.new_sequence(&name, q.settings.clone(), q.video_tracks.len(), q.audio_tracks.len(), None);
|
||||
|
||||
@@ -653,6 +653,7 @@ const CACHE_MGMT: &[(&str, &str)] = &[
|
||||
];
|
||||
const AUTO_SCROLL: &[(&str, &str)] = &[("noScroll", "No Scroll"), ("pageScroll", "Page Scroll"), ("smoothScroll", "Smooth Scroll")];
|
||||
const MOUSE_SCROLL: &[(&str, &str)] = &[("vertical", "Vertical"), ("horizontal", "Horizontal")];
|
||||
const HW_DECODE: &[(&str, &str)] = &[("auto", "Auto"), ("off", "Off")];
|
||||
const TRACKS: &[(&str, &str)] = &[("useFile", "Use File"), ("mono", "Mono"), ("stereo", "Stereo"), ("5.1", "5.1"), ("adaptive", "Adaptive")];
|
||||
pub const DURATION_UNITS: &[(&str, &str)] = &[("frames", "Frames"), ("seconds", "Seconds")];
|
||||
|
||||
@@ -952,6 +953,10 @@ static CATEGORIES: &[Category] = &[
|
||||
b("playback.enableTransmit", "Enable Mercury Transmit", false),
|
||||
b("playback.disableVideoInBackground", "Disable video output when in the background", false),
|
||||
b("playback.draftDecode", "Draft decoding at reduced playback resolution (H.264: faster, some frames less filtered)", true),
|
||||
f("playback.hardwareDecoding", "Hardware decoding", Kind::Choice(HW_DECODE), true),
|
||||
Row::Note(
|
||||
"Hardware decoding: Auto uses the system's video decoder (VideoToolbox on macOS) for the H.264 and HEVC streams it supports, and FilmCraft's own decoder for everything else or if the hardware fails. Media that is already open keeps its decoder until it is reopened.",
|
||||
),
|
||||
],
|
||||
},
|
||||
Category {
|
||||
@@ -1122,6 +1127,12 @@ pub fn migrate(v: &mut Value) {
|
||||
}
|
||||
|
||||
impl crate::autosave::Preferences {
|
||||
/// Settings ▸ Playback ▸ Hardware decoding as the process-wide switch the hardware decoder
|
||||
/// factories consult (new decoders only).
|
||||
pub fn apply_hardware_decoding(&self) {
|
||||
filmcraft_codecs::hw::set_hardware_decoding(self.playback.hardware_decoding != "off");
|
||||
}
|
||||
|
||||
/// Values of one category as a JSON object.
|
||||
pub fn category_value(&self, id: &str) -> Option<Value> {
|
||||
self.to_value().get(id).cloned()
|
||||
|
||||
@@ -480,3 +480,15 @@ fn auto_transcribe_on_import_and_transcription_defaults() {
|
||||
set(&mut s, "mediaAnalysis.whisperModel", json!("whisper-tiny"));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hardware_decoding_setting_drives_the_decoder_switch() {
|
||||
let mut s = Session::default();
|
||||
assert_eq!(s.prefs.playback.hardware_decoding, "auto", "Auto by default");
|
||||
set(&mut s, "playback.hardwareDecoding", json!("off"));
|
||||
assert!(!filmcraft_codecs::hw::hardware_decoding(), "Off reaches the decoder registry");
|
||||
assert!(s.execute("prefs.set", json!({"key": "playback.hardwareDecoding", "value": "gpu"})).is_err());
|
||||
set(&mut s, "playback.hardwareDecoding", json!("auto"));
|
||||
assert!(filmcraft_codecs::hw::hardware_decoding());
|
||||
assert!(settings::field("playback.hardwareDecoding").is_some_and(|f| f.wired && matches!(f.kind, Kind::Choice(_))));
|
||||
}
|
||||
@@ -441,7 +441,9 @@ fn bindable(c: &CommandSpec) -> bool {
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
fields.last_mut().expect("field").push(ch);
|
||||
if let Some(f) = fields.last_mut() {
|
||||
f.push(ch);
|
||||
}
|
||||
}
|
||||
fields.iter().all(|f| f.contains('?') || f.contains('='))
|
||||
}
|
||||
@@ -505,7 +507,7 @@ impl Shortcuts {
|
||||
dir: None,
|
||||
session_start: None,
|
||||
};
|
||||
s.bindings = s.builtin(DEFAULT_PRESET).expect("default preset");
|
||||
s.bindings = s.builtin(DEFAULT_PRESET).unwrap_or_default();
|
||||
s
|
||||
}
|
||||
|
||||
|
||||
@@ -83,6 +83,18 @@ fn trim_linked_and_ripple_delete() {
|
||||
assert_eq!(q.video_tracks[0].items[0].start, s.sequence_rate().tick_of(12), "the trim gap remains");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn slide_moves_linked_audio_with_the_video() {
|
||||
let mut s = demo();
|
||||
let v = s.active_sequence().unwrap().video_tracks[0].items[1].clone();
|
||||
s.execute("timeline.slide", json!({"clip": v.id.0, "deltaFrames": 5})).unwrap();
|
||||
let q = s.active_sequence().unwrap();
|
||||
let (_, nv) = q.find_item(v.id).unwrap();
|
||||
let a = q.audio_tracks[0].items.iter().find(|i| i.link == v.link).unwrap();
|
||||
assert_eq!(nv.start, v.start + s.sequence_rate().tick_of(5));
|
||||
assert_eq!((a.start, a.source_in), (nv.start, nv.source_in), "linked audio slides too and stays in sync");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn zero_fps_is_refused() {
|
||||
let mut s = demo();
|
||||
|
||||
@@ -433,7 +433,7 @@ pub fn shuttle(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
tick(s, clock)?;
|
||||
}
|
||||
let rate = s.sequence_rate();
|
||||
let d = s.trim_play.dynamic.as_mut().expect("dynamic trim");
|
||||
let d = s.trim_play.dynamic.as_mut().ok_or_else(|| EngineError::Other("dynamic trimming is not active".into()))?;
|
||||
let speed = if slow {
|
||||
0.25 * dir
|
||||
} else if d.speed * dir > 0.0 && !d.at_limit {
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
//! An export depends only on the project and the settings: the same file on every machine, however
|
||||
//! many cores render it and however the work is cut into steps.
|
||||
|
||||
use super::*;
|
||||
use crate::tests::project;
|
||||
|
||||
fn tmp(name: &str) -> String {
|
||||
let d = std::env::temp_dir().join(format!("fc-export-det-{}", std::process::id()));
|
||||
std::fs::create_dir_all(&d).unwrap();
|
||||
d.join(name).to_string_lossy().to_string()
|
||||
}
|
||||
|
||||
/// Export on a machine with `threads` cores: `export` renders one frame per core of the pool it
|
||||
/// runs in.
|
||||
fn export_with_cores(threads: usize, settings: &ExportSettings) -> Vec<u8> {
|
||||
let (p, seq, m) = project();
|
||||
let pool = rayon::ThreadPoolBuilder::new().num_threads(threads).build().unwrap();
|
||||
pool.install(|| export(&p, seq, settings, &m, &Progress::default())).unwrap();
|
||||
std::fs::read(&settings.path).unwrap()
|
||||
}
|
||||
|
||||
/// Export a step at a time with `batch` frames per step (the web app uses 1).
|
||||
fn export_stepped(batch: i64, settings: &ExportSettings) -> Vec<u8> {
|
||||
let (p, seq, m) = project();
|
||||
let prog = Progress::default();
|
||||
let mut ex = Exporter::new(p, seq, settings, &prog).unwrap();
|
||||
ex.set_batch(batch);
|
||||
while !matches!(ex.step(&m, &prog).unwrap(), Step::Done(_)) {}
|
||||
std::fs::read(&settings.path).unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn same_file_whatever_the_core_count() {
|
||||
// 24 frames: one full interleave group and a partial one
|
||||
for (format, ext) in [(Format::ProRes, "mov"), (Format::H264, "mp4"), (Format::Mjpeg, "mov")] {
|
||||
let settings = |name: &str| ExportSettings { format, path: tmp(&format!("{name}.{ext}")), ..Default::default() };
|
||||
let reference = export_with_cores(2, &settings(&format!("{}-2", format.id())));
|
||||
for threads in [3, 5, 16] {
|
||||
let other = export_with_cores(threads, &settings(&format!("{}-{threads}", format.id())));
|
||||
assert!(reference == other, "{}: {threads} cores give a different file than 2 cores", format.label());
|
||||
}
|
||||
for batch in [1, 7] {
|
||||
let other = export_stepped(batch, &settings(&format!("{}-b{batch}", format.id())));
|
||||
assert!(reference == other, "{}: {batch} frame(s) per step give a different file", format.label());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Slice NAL units (types 1 and 5) in a length-prefixed (4-byte) H.264 sample.
|
||||
fn slices_in(sample: &[u8]) -> usize {
|
||||
let (mut n, mut i) = (0, 0);
|
||||
while i + 4 <= sample.len() {
|
||||
let len = u32::from_be_bytes([sample[i], sample[i + 1], sample[i + 2], sample[i + 3]]) as usize;
|
||||
if let Some(&h) = sample.get(i + 4)
|
||||
&& matches!(h & 0x1f, 1 | 5)
|
||||
{
|
||||
n += 1;
|
||||
}
|
||||
i += 4 + len;
|
||||
}
|
||||
n
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn h264_slices_follow_the_frame_size_not_the_cores() {
|
||||
// 4096 lines = 256 macroblock rows: one slice per four rows on every machine, also on machines
|
||||
// with fewer than 64 cores (the encoder's default caps the slices at the core count)
|
||||
let path = tmp("tall.mp4");
|
||||
let s = ExportSettings { format: Format::H264, path: path.clone(), frame_size: Some((64, 4096)), include_audio: false, ..Default::default() };
|
||||
let (p, seq, m) = project();
|
||||
export(&p, seq, &s, &m, &Progress::default()).unwrap();
|
||||
let bytes = std::fs::read(&path).unwrap();
|
||||
let file = filmcraft_isobmff::open(bytes.as_slice()).unwrap();
|
||||
let vt = file.track_of_kind(filmcraft_isobmff::TrackKind::Video).unwrap();
|
||||
assert_eq!(file.tracks[vt].samples.len(), 24);
|
||||
for i in 0..file.tracks[vt].samples.len() {
|
||||
let sample = file.read_sample(bytes.as_slice(), vt, i).unwrap();
|
||||
assert_eq!(slices_in(&sample), 64, "picture {i}");
|
||||
}
|
||||
}
|
||||
+47
-18
@@ -11,6 +11,10 @@
|
||||
//! When a source's bytes are not available yet (asynchronous web reads mark
|
||||
//! [`filmcraft_media::pending`]), the batch is rendered again on the next call: nothing is encoded
|
||||
//! from frames or audio with missing media.
|
||||
//!
|
||||
//! The file does not depend on how many frames a step renders (one per core natively, one on the
|
||||
//! web): packets go to the muxer in fixed groups of [`INTERLEAVE`] output frames, each followed by
|
||||
//! the audio up to its end, so the same project and settings give the same bytes on any machine.
|
||||
|
||||
use std::sync::Arc;
|
||||
use std::sync::atomic::Ordering;
|
||||
@@ -30,6 +34,11 @@ use crate::{
|
||||
audio_factories, export_range, frame_span, video_factories,
|
||||
};
|
||||
|
||||
/// Output frames per interleaved group: the video packets of these frames, then the audio up to
|
||||
/// their end, form one chunk of each track. Fixed (not the batch size, which follows the core
|
||||
/// count) so the file layout is the same on every machine.
|
||||
pub const INTERLEAVE: i64 = 16;
|
||||
|
||||
/// What one [`Exporter::step`] did.
|
||||
#[derive(Debug)]
|
||||
pub enum Step {
|
||||
@@ -58,7 +67,9 @@ pub struct Exporter {
|
||||
measured: bool,
|
||||
/// The first pass of a two-pass encode is running.
|
||||
first_pass: bool,
|
||||
/// Created after the first batch (encoders may finalise their codec config then).
|
||||
/// Encoded video packets not written yet (the current [`INTERLEAVE`] group).
|
||||
queued: Vec<EncodedPacket>,
|
||||
/// Created after the first group was encoded (encoders may finalise their codec config then).
|
||||
mux: Option<Mp4Writer<Out>>,
|
||||
/// MXF exports write through this instead of `mux`.
|
||||
mxf: Option<MxfMux>,
|
||||
@@ -140,6 +151,7 @@ impl Exporter {
|
||||
audio,
|
||||
measured,
|
||||
first_pass: two_pass,
|
||||
queued: Vec::new(),
|
||||
mux: None,
|
||||
mxf: None,
|
||||
seq,
|
||||
@@ -177,7 +189,7 @@ impl Exporter {
|
||||
if n == 0 {
|
||||
return Ok(());
|
||||
}
|
||||
let mux = self.mux.as_mut().expect("mux created");
|
||||
let mux = self.mux.as_mut().ok_or_else(|| ExportError::Encode("internal: the container writer was not created".into()))?;
|
||||
match self.aenc.as_mut() {
|
||||
Some(a) => {
|
||||
for au in a.encode(&buf)? {
|
||||
@@ -209,7 +221,7 @@ impl Exporter {
|
||||
if let Some(m) = self.mxf.as_mut() {
|
||||
return m.write_video(packets);
|
||||
}
|
||||
let mux = self.mux.as_mut().expect("mux created");
|
||||
let mux = self.mux.as_mut().ok_or_else(|| ExportError::Encode("internal: the container writer was not created".into()))?;
|
||||
for p in packets {
|
||||
mux.write_sample(self.vt, WriteSample { data: &p.data, duration: p.duration, composition_offset: p.composition_offset, is_sync: p.key })
|
||||
.map_err(|e| ExportError::Io(e.to_string()))?;
|
||||
@@ -270,7 +282,7 @@ impl Exporter {
|
||||
if !self.settings.part_of_batch {
|
||||
progress.set_status("Measuring loudness");
|
||||
}
|
||||
let a = self.audio.as_mut().expect("audio");
|
||||
let a = self.audio.as_mut().ok_or_else(|| ExportError::Encode("internal: the audio pipeline was not created".into()))?;
|
||||
a.measure(&self.settings, sources, &|| progress.cancel.load(Ordering::Relaxed))?;
|
||||
if filmcraft_media::pending::take() {
|
||||
return Ok(Step::Pending);
|
||||
@@ -295,22 +307,33 @@ impl Exporter {
|
||||
self.next = end;
|
||||
return Ok(Step::Progress);
|
||||
}
|
||||
let mixed = match self.audio.as_ref().map(|a| a.sr) {
|
||||
Some(sr) if (self.mux.is_none() && self.mxf.is_none()) || self.at.is_some() || self.mxf.is_some() => {
|
||||
let until = self.sample_at_frame(end, sr);
|
||||
self.audio.as_mut().and_then(|a| a.pull(until, sources))
|
||||
// the audio of every interleave group that ends in this batch (before encoding, so a
|
||||
// pending source can still make the batch run again)
|
||||
let cuts: Vec<i64> = (f + 1..=end).filter(|&g| (g - self.f0) % INTERLEAVE == 0).collect();
|
||||
let mut mixed = Vec::with_capacity(cuts.len());
|
||||
if let Some(sr) = self.audio.as_ref().map(|a| a.sr) {
|
||||
for &g in &cuts {
|
||||
let until = self.sample_at_frame(g, sr);
|
||||
mixed.push(self.audio.as_mut().and_then(|a| a.pull(until, sources)));
|
||||
}
|
||||
_ => None,
|
||||
};
|
||||
}
|
||||
if filmcraft_media::pending::take() {
|
||||
return Ok(Step::Pending);
|
||||
}
|
||||
let packets = self.encode(&frames, f)?;
|
||||
if self.mux.is_none() && self.mxf.is_none() {
|
||||
self.open_mux()?;
|
||||
let mut mixed = mixed.into_iter();
|
||||
for (k, frame) in frames.iter().enumerate() {
|
||||
let fi = f + k as i64;
|
||||
let packets = self.encode(std::slice::from_ref(frame), fi)?;
|
||||
self.queued.extend(packets);
|
||||
if cuts.contains(&(fi + 1)) {
|
||||
if self.mux.is_none() && self.mxf.is_none() {
|
||||
self.open_mux()?;
|
||||
}
|
||||
let packets = std::mem::take(&mut self.queued);
|
||||
self.write_video(packets)?;
|
||||
self.write_audio(mixed.next().flatten())?;
|
||||
}
|
||||
}
|
||||
self.write_video(packets)?;
|
||||
self.write_audio(mixed)?;
|
||||
progress.done.fetch_add((end - f) as u64, Ordering::Relaxed);
|
||||
self.next = end;
|
||||
return Ok(Step::Progress);
|
||||
@@ -332,12 +355,13 @@ impl Exporter {
|
||||
if self.mux.is_none() && self.mxf.is_none() {
|
||||
self.open_mux()?;
|
||||
}
|
||||
let tail = self.venc.flush()?;
|
||||
let mut tail = std::mem::take(&mut self.queued);
|
||||
tail.extend(self.venc.flush()?);
|
||||
self.write_video(tail)?;
|
||||
self.write_audio(mixed)?;
|
||||
if let (Some(at), Some(a)) = (self.at, self.aenc.as_mut()) {
|
||||
let fs = a.frame_size();
|
||||
let mux = self.mux.as_mut().expect("mux created");
|
||||
let mux = self.mux.as_mut().ok_or_else(|| ExportError::Encode("internal: the container writer was not created".into()))?;
|
||||
for au in a.flush()? {
|
||||
mux.write_sample(at, WriteSample { data: &au, duration: fs, composition_offset: 0, is_sync: true })
|
||||
.map_err(|e| ExportError::Io(e.to_string()))?;
|
||||
@@ -346,7 +370,12 @@ impl Exporter {
|
||||
let (bytes, extra_files) = match self.mxf.take() {
|
||||
Some(m) => m.finish(&self.settings)?,
|
||||
None => {
|
||||
let w = self.mux.take().expect("mux created").finish().map_err(|e| ExportError::Io(e.to_string()))?;
|
||||
let w = self
|
||||
.mux
|
||||
.take()
|
||||
.ok_or_else(|| ExportError::Encode("internal: the container writer was not created".into()))?
|
||||
.finish()
|
||||
.map_err(|e| ExportError::Io(e.to_string()))?;
|
||||
(w.finish(&self.settings)?, Vec::new())
|
||||
}
|
||||
};
|
||||
|
||||
@@ -11,6 +11,7 @@
|
||||
//! [`ExportSettings`] carries every Export-mode setting (frame size, rate, bitrate encoding, audio
|
||||
//! format, multiplexer, captions, effects, metadata) as serde data; [`presets`] defines the
|
||||
//! built-in presets.
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
use std::io::Write;
|
||||
|
||||
@@ -568,7 +569,7 @@ impl VideoEncoder for MjpegEncoder {
|
||||
self.rate.num as u32
|
||||
}
|
||||
fn encode(&mut self, f: &EncoderFrame) -> Result<Vec<EncodedPacket>> {
|
||||
let rgb: Vec<u8> = f.rgba.chunks_exact(4).flat_map(|p| [p[0], p[1], p[2]]).collect();
|
||||
let rgb: Vec<u8> = f.rgba.as_chunks::<4>().0.iter().flat_map(|p| [p[0], p[1], p[2]]).collect();
|
||||
let mut out = Vec::new();
|
||||
let mut enc = image::codecs::jpeg::JpegEncoder::new_with_quality(&mut out, self.quality);
|
||||
enc.encode(&rgb, f.width, f.height, image::ExtendedColorType::Rgb8).map_err(|e| ExportError::Encode(e.to_string()))?;
|
||||
@@ -916,6 +917,13 @@ pub fn rgbf_to_yuv420_8(rgb: &[f32], w: usize, h: usize, kr: f32, kb: f32, y: &m
|
||||
});
|
||||
}
|
||||
|
||||
/// Slices per H.264 picture: one per four macroblock rows. The encoder's default follows the core
|
||||
/// count, which would make the stream (and every decoded picture) depend on the machine; this is
|
||||
/// what it picks on a machine with enough cores.
|
||||
pub(crate) fn h264_slices(height: u32) -> usize {
|
||||
(height.div_ceil(16) as usize).div_ceil(4).max(1)
|
||||
}
|
||||
|
||||
fn h264_factory(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettings) -> Option<Result<Box<dyn VideoEncoder>>> {
|
||||
if format != Format::H264 {
|
||||
return None;
|
||||
@@ -925,6 +933,7 @@ fn h264_factory(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSetti
|
||||
cfg.format = if s.format.is_mxf() { filmcraft_h264enc::PacketFormat::AnnexB } else { filmcraft_h264enc::PacketFormat::LengthPrefixed };
|
||||
cfg.aud = s.format.is_mxf();
|
||||
cfg.keyint = s.keyframe_distance.filter(|k| *k > 0).unwrap_or_else(|| (rate.num as f64 / rate.den as f64 * 2.0).round().max(1.0) as u32);
|
||||
cfg.slices = h264_slices(h);
|
||||
let kbps = s.bitrate_kbps.max(100);
|
||||
let max = s.max_bitrate_kbps.filter(|m| *m >= kbps).unwrap_or(kbps * 3 / 2);
|
||||
cfg.rate = match s.bitrate_mode {
|
||||
@@ -1108,7 +1117,10 @@ pub fn export(project: &Arc<Project>, seq: ItemId, settings: &ExportSettings, so
|
||||
}
|
||||
(total, count)
|
||||
}
|
||||
Format::H264 | Format::ProRes | Format::DnxHr | Format::Mjpeg | Format::MxfOp1a | Format::MxfOpAtom => unreachable!("stepped export"),
|
||||
Format::H264 | Format::ProRes | Format::DnxHr | Format::Mjpeg | Format::MxfOp1a | Format::MxfOpAtom => {
|
||||
// Handled by the stepped exporter above; reaching here would be a dispatch bug.
|
||||
return Err(ExportError::Unsupported(format!("{:?} must run as a stepped export", settings.format)));
|
||||
}
|
||||
};
|
||||
let secs = t0.elapsed().as_secs_f64();
|
||||
if !settings.part_of_batch {
|
||||
@@ -1129,3 +1141,6 @@ mod surround_tests;
|
||||
|
||||
#[cfg(test)]
|
||||
mod mxf_tests;
|
||||
|
||||
#[cfg(test)]
|
||||
mod determinism_tests;
|
||||
@@ -225,7 +225,7 @@ fn ffmpeg_oracle_op1a_and_op_atom() {
|
||||
let at = mx.track_of_kind(filmcraft_mxf::TrackKind::Sound).unwrap();
|
||||
let ours = mx.read_pcm(&bytes, at, 0, 48_000).unwrap();
|
||||
assert_eq!(pcm.len(), 48_000 * 2 * 4, "{codec:?}");
|
||||
for (i, c) in pcm.chunks_exact(4).enumerate() {
|
||||
for (i, c) in pcm.as_chunks::<4>().0.iter().enumerate() {
|
||||
let s = i32::from_le_bytes([c[0], c[1], c[2], c[3]]) >> 8;
|
||||
assert_eq!(s as f32 / 8_388_608.0, ours[i % 2][i / 2], "{codec:?}: sample {i}");
|
||||
}
|
||||
@@ -255,7 +255,7 @@ fn ffmpeg_oracle_op1a_and_op_atom() {
|
||||
assert_eq!((s["codec_name"].as_str(), s["channels"].as_u64(), s["sample_rate"].as_str()), (Some("pcm_s24le"), Some(1), Some("48000")), "{a}");
|
||||
let mono = ffmpeg_raw(&ffmpeg, a, "0:a:0", &["-f", "s32le", "-acodec", "pcm_s32le"]);
|
||||
assert_eq!(mono.len(), 48_000 * 4, "{a}");
|
||||
let want: Vec<u8> = stereo.chunks_exact(8).flat_map(|f| f[k * 4..k * 4 + 4].to_vec()).collect();
|
||||
let want: Vec<u8> = stereo.as_chunks::<8>().0.iter().flat_map(|f| f[k * 4..k * 4 + 4].to_vec()).collect();
|
||||
assert!(mono == want, "{a}: channel {k} differs from the WAV export");
|
||||
}
|
||||
}
|
||||
@@ -111,7 +111,7 @@ impl Pipeline {
|
||||
if let Some(tf) = self.out_tf.as_ref().filter(|_| self.hdr_out) {
|
||||
let mut out = vec![0f32; img.w * img.h * 3];
|
||||
out.par_chunks_mut(img.w * 3).zip(img.px.par_chunks(img.w * 4)).for_each(|(o, s)| {
|
||||
for (o, p) in o.chunks_exact_mut(3).zip(s.chunks_exact(4)) {
|
||||
for (o, p) in o.as_chunks_mut::<3>().0.iter_mut().zip(s.as_chunks::<4>().0) {
|
||||
o.copy_from_slice(&tf.encode([p[0], p[1], p[2]]));
|
||||
}
|
||||
});
|
||||
@@ -239,7 +239,7 @@ fn load_overlay(path: &str, width: f32) -> Result<Image> {
|
||||
let rgba = image::load_from_memory(&bytes).map_err(|e| ExportError::Unsupported(format!("image overlay {path}: {e}")))?.to_rgba8();
|
||||
let (iw, ih) = (rgba.width() as usize, rgba.height() as usize);
|
||||
let mut img = Image::new(iw, ih);
|
||||
for (o, p) in img.px.chunks_exact_mut(4).zip(rgba.as_raw().chunks_exact(4)) {
|
||||
for (o, p) in img.px.as_chunks_mut::<4>().0.iter_mut().zip(rgba.as_raw().as_chunks::<4>().0) {
|
||||
let a = p[3] as f32 / 255.0;
|
||||
for c in 0..3 {
|
||||
o[c] = filmcraft_color::srgb_to_linear(p[c] as f32 / 255.0) * a;
|
||||
@@ -310,7 +310,7 @@ fn resample_axis(src: &Image, n: usize, horizontal: bool) -> Image {
|
||||
pub fn limit_rgba8(rgba: &mut [u8], min_percent: f32, max_percent: f32) {
|
||||
let (lo, hi) = (min_percent.clamp(0.0, 100.0) / 100.0, max_percent.clamp(0.0, 100.0) / 100.0);
|
||||
rgba.par_chunks_mut(4 * 1024).for_each(|chunk| {
|
||||
for p in chunk.chunks_exact_mut(4) {
|
||||
for p in chunk.as_chunks_mut::<4>().0 {
|
||||
let mut c = [p[0] as f32 / 255.0, p[1] as f32 / 255.0, p[2] as f32 / 255.0];
|
||||
limit_px(&mut c, lo, hi);
|
||||
for i in 0..3 {
|
||||
@@ -323,7 +323,7 @@ pub fn limit_rgba8(rgba: &mut [u8], min_percent: f32, max_percent: f32) {
|
||||
fn limit_f32(rgb: &mut [f32], min_percent: f32, max_percent: f32) {
|
||||
let (lo, hi) = (min_percent.clamp(0.0, 100.0) / 100.0, max_percent.clamp(0.0, 100.0) / 100.0);
|
||||
rgb.par_chunks_mut(3 * 1024).for_each(|chunk| {
|
||||
for p in chunk.chunks_exact_mut(3) {
|
||||
for p in chunk.as_chunks_mut::<3>().0 {
|
||||
let mut c = [p[0], p[1], p[2]];
|
||||
limit_px(&mut c, lo, hi);
|
||||
p.copy_from_slice(&c);
|
||||
|
||||
@@ -204,7 +204,7 @@ fn read_wav16(path: &str) -> Vec<Vec<f32>> {
|
||||
let ch = u16::from_le_bytes([b[22], b[23]]) as usize;
|
||||
let data = &b[44..];
|
||||
let mut out = vec![Vec::new(); ch];
|
||||
for (i, s) in data.chunks_exact(2).enumerate() {
|
||||
for (i, s) in data.as_chunks::<2>().0.iter().enumerate() {
|
||||
out[i % ch].push(i16::from_le_bytes([s[0], s[1]]) as f32 / 32768.0);
|
||||
}
|
||||
out
|
||||
|
||||
@@ -110,7 +110,7 @@ fn decode(path: &str) -> Option<Vec<Vec<f64>>> {
|
||||
let out =
|
||||
std::process::Command::new(ffmpeg).args(["-v", "error", "-i", path, "-map", "0:a:0", "-f", "s32le", "-acodec", "pcm_s32le", "-"]).output().unwrap();
|
||||
assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr));
|
||||
let ints: Vec<i32> = out.stdout.chunks_exact(4).map(|b| i32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
|
||||
let ints: Vec<i32> = out.stdout.as_chunks::<4>().0.iter().map(|b| i32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
|
||||
let n = ints.len() / 6;
|
||||
Some((0..6).map(|c| (0..n).map(|i| ints[i * 6 + c] as f64).collect()).collect())
|
||||
}
|
||||
|
||||
@@ -34,6 +34,8 @@
|
||||
//! per-character text styles (`TrackItem::graphic`, `EffectInstance::layer`) and source graphics
|
||||
//! (`Project::source_graphics`). No-op step; older builds would drop these fields when saving.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
pub mod atomic;
|
||||
pub mod autosave;
|
||||
|
||||
|
||||
@@ -5,6 +5,8 @@
|
||||
//! compositor. Pixel data is `Arc`-shared so caches, monitors and export share frames for free.
|
||||
//! - [`AudioBuffer`]: planar f32 samples.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use filmcraft_color::{ColorInfo, DecodeTable, Matrix, Range, linear_to_srgb_u8, normalize_c, normalize_y, srgb_u8_to_linear_table, to_linear, ycbcr_to_rgb};
|
||||
@@ -383,14 +385,14 @@ impl VideoFrame {
|
||||
pub fn luma8(&self) -> Vec<u8> {
|
||||
match &self.data {
|
||||
PixelData::Yuv8 { planes, .. } => planes[0].as_ref().clone(),
|
||||
_ => self.to_rgba8().chunks_exact(4).map(|p| (0.2126 * p[0] as f32 + 0.7152 * p[1] as f32 + 0.0722 * p[2] as f32) as u8).collect(),
|
||||
_ => self.to_rgba8().as_chunks::<4>().0.iter().map(|p| (0.2126 * p[0] as f32 + 0.7152 * p[1] as f32 + 0.0722 * p[2] as f32) as u8).collect(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Convert a row of premultiplied linear f32 RGBA into straight sRGB RGBA8.
|
||||
pub fn linear_premul_to_srgb8(src: &[f32], dst: &mut [u8]) {
|
||||
for (s, o) in src.chunks_exact(4).zip(dst.chunks_exact_mut(4)) {
|
||||
for (s, o) in src.as_chunks::<4>().0.iter().zip(dst.as_chunks_mut::<4>().0) {
|
||||
let a = s[3].clamp(0.0, 1.0);
|
||||
if a <= 0.0 {
|
||||
o.copy_from_slice(&[0, 0, 0, 0]);
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
//! 2D geometry for FilmCraft: vectors, rectangles and affine transforms (Motion effect math).
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::ops::{Add, Mul, Sub};
|
||||
|
||||
|
||||
@@ -185,7 +185,9 @@ impl MediaSource for Encoded {
|
||||
let m = filmcraft_color::spaces::to_f32(&filmcraft_color::spaces::gamut_matrix(Gamut::Bt709, self.target.gamut()));
|
||||
let curve = self.target.curve();
|
||||
let out: Vec<u8> = rgba
|
||||
.chunks_exact(4)
|
||||
.as_chunks::<4>()
|
||||
.0
|
||||
.iter()
|
||||
.flat_map(|p| {
|
||||
let lin = [0, 1, 2].map(|k| filmcraft_color::srgb_to_linear(p[k] as f32 / 255.0) * self.gain);
|
||||
let c = filmcraft_color::spaces::apply3(&m, lin);
|
||||
@@ -723,7 +725,7 @@ fn lumetri_presets_grid() {
|
||||
fn scenes_are_distinct() {
|
||||
let imgs: Vec<(&str, Rgba8)> = scenes().into_iter().map(|(n, _, f)| (n, render_cpu(&f()))).collect();
|
||||
for (n, img) in &imgs {
|
||||
let lit = img.px.chunks_exact(4).filter(|p| p[0] as u32 + p[1] as u32 + p[2] as u32 > 30).count();
|
||||
let lit = img.px.as_chunks::<4>().0.iter().filter(|p| p[0] as u32 + p[1] as u32 + p[2] as u32 > 30).count();
|
||||
assert!(lit > img.px.len() / 4 / 4, "{n}: mostly black");
|
||||
}
|
||||
for i in 0..imgs.len() {
|
||||
@@ -758,7 +760,7 @@ fn gpu_matches_cpu_on_golden_scenes() {
|
||||
let kind = if matches!(plan, filmcraft_render::plan::FramePlan::Layers { .. }) { "layers" } else { "cpu image" };
|
||||
c.composite(&plan);
|
||||
let (gw, gh, mut px) = c.read_output().expect("GPU readback");
|
||||
px.chunks_exact_mut(4).for_each(|p| p[3] = 255);
|
||||
px.as_chunks_mut::<4>().0.iter_mut().for_each(|p| p[3] = 255);
|
||||
let gpu = Rgba8::new(gw, gh, px);
|
||||
let d = diff(&cpu, &gpu).unwrap();
|
||||
// mean over RGB only (alpha is equal by construction)
|
||||
@@ -773,3 +775,201 @@ fn gpu_matches_cpu_on_golden_scenes() {
|
||||
}
|
||||
assert!(failures.is_empty(), "GPU differs from the CPU reference:\n{}", failures.join("\n"));
|
||||
}
|
||||
|
||||
/// Per-pixel max RGB difference (99th percentile, mean) over the pixels at least 1.5 px from every
|
||||
/// plan layer's quad outline: the CPU resampler fades a layer's edge over one more pixel than the
|
||||
/// GPU rasterises, and with blend modes those edge pixels are a visible share of a 320×180 frame.
|
||||
fn interior_diff(plan: &filmcraft_render::plan::FramePlan, a: &Rgba8, b: &Rgba8) -> (u32, f64) {
|
||||
let filmcraft_render::plan::FramePlan::Layers { width, layers, .. } = plan else { return (u32::MAX, f64::MAX) };
|
||||
let inv: Vec<_> = layers.iter().filter_map(|l| Some((l.matrix.inverse()?, l.size().0 as f64, l.size().1 as f64))).collect();
|
||||
let mut d: Vec<u32> = Vec::new();
|
||||
for (i, (pa, pb)) in a.px.chunks(4).zip(b.px.chunks(4)).enumerate() {
|
||||
let (x, y) = ((i % width) as f64 + 0.5, (i / width) as f64 + 0.5);
|
||||
let edge = inv.iter().any(|(m, fw, fh)| {
|
||||
let c = [(-1.5, -1.5), (1.5, -1.5), (-1.5, 1.5), (1.5, 1.5)].map(|(dx, dy)| {
|
||||
let p = m.apply(Vec2::new(x + dx, y + dy));
|
||||
p.x >= 0.0 && p.y >= 0.0 && p.x <= *fw && p.y <= *fh
|
||||
});
|
||||
c.iter().any(|v| *v != c[0])
|
||||
});
|
||||
if !edge {
|
||||
d.push((0..3).map(|k| (pa[k] as i32 - pb[k] as i32).unsigned_abs()).max().unwrap_or(0));
|
||||
}
|
||||
}
|
||||
d.sort_unstable();
|
||||
(d[d.len() * 99 / 100], d.iter().sum::<u32>() as f64 / d.len() as f64)
|
||||
}
|
||||
|
||||
/// Every blend mode composited by the GPU from `plan_frame` (no CPU fallback): two overlapping
|
||||
/// transformed clips at partial opacity in the mode over footage, plus a clip with a standard
|
||||
/// effect (rendered on the CPU as a layer image) in the same mode, match the CPU render with the
|
||||
/// GPU parity tolerance (p99 ≤ 6, mean < 1.5) away from layer outlines; whole-frame numbers are
|
||||
/// printed too.
|
||||
#[test]
|
||||
fn gpu_blend_modes_match_cpu_render() {
|
||||
let Some((dev, q)) = device() else {
|
||||
eprintln!("SKIPPED (gpu blend parity): no GPU adapter");
|
||||
return;
|
||||
};
|
||||
let mut c = filmcraft_gpu::GpuCompositor::new(&dev, &q);
|
||||
let mut failures = Vec::new();
|
||||
for mode in filmcraft_project::effect::BLEND_MODES {
|
||||
let mut b = Builder::new(4);
|
||||
let bg = b.demo(DemoScene::OceanSunset);
|
||||
let (dunes, aurora, bars) = (b.demo(DemoScene::Dunes), b.demo(DemoScene::Aurora), b.media(Generator::BarsAndTone));
|
||||
b.place(0, bg, 0, 48);
|
||||
let v2 = b.place(1, dunes, 0, 48);
|
||||
b.fixed(v2, "motion", &[("scale", fl(60.0)), ("rotation", fl(12.0)), ("position", pt(130.0, 80.0))]);
|
||||
b.fixed(v2, "opacity", &[("opacity", fl(70.0)), ("blend", blend(mode))]);
|
||||
let v3 = b.place(2, aurora, 0, 48);
|
||||
b.fixed(v3, "motion", &[("scale", fl(45.0)), ("rotation", fl(-20.0)), ("position", pt(210.0, 110.0))]);
|
||||
b.fixed(v3, "opacity", &[("opacity", fl(60.0)), ("blend", blend(mode))]);
|
||||
let v4 = b.place(3, bars, 0, 48);
|
||||
b.fixed(v4, "motion", &[("scale", fl(30.0)), ("position", pt(250.0, 50.0))]);
|
||||
b.fixed(v4, "opacity", &[("opacity", fl(85.0)), ("blend", blend(mode))]);
|
||||
b.effect(v4, "crop", &[("left", fl(10.0)), ("bottom", fl(20.0))]);
|
||||
let s = b.at(12);
|
||||
let t = RATE.tick_of(s.frame);
|
||||
let cpu = render_cpu(&s);
|
||||
let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, t, RenderOptions::default(), &s.sources);
|
||||
let filmcraft_render::plan::FramePlan::Layers { layers, .. } = &plan else {
|
||||
failures.push(format!("{mode}: planned as a CPU image"));
|
||||
continue;
|
||||
};
|
||||
assert_eq!(layers.len(), 4, "{mode}");
|
||||
c.composite(&plan);
|
||||
let (gw, gh, mut px) = c.read_output().expect("GPU readback");
|
||||
px.as_chunks_mut::<4>().0.iter_mut().for_each(|p| p[3] = 255);
|
||||
let gpu = Rgba8::new(gw, gh, px);
|
||||
let d = diff(&cpu, &gpu).unwrap();
|
||||
let (p99, mean) = interior_diff(&plan, &cpu, &gpu);
|
||||
eprintln!("{mode}: GPU vs CPU interior p99 {p99}, mean {mean:.3}; whole frame {d}");
|
||||
if p99 > 6 || mean >= 1.5 {
|
||||
failures.push(format!("{mode}: interior p99 {p99}, mean {mean}; {d}"));
|
||||
}
|
||||
}
|
||||
assert!(failures.is_empty(), "GPU blend modes differ from the CPU render:\n{}", failures.join("\n"));
|
||||
}
|
||||
|
||||
/// Inside a transition the CPU mixes the clips and composites the result Normal, ignoring their
|
||||
/// blend modes: the plan does the same (Screen clips cross-dissolving on V2 over V1).
|
||||
#[test]
|
||||
fn gpu_transition_of_blended_clips_matches_cpu_render() {
|
||||
let Some((dev, q)) = device() else {
|
||||
eprintln!("SKIPPED (gpu blend parity): no GPU adapter");
|
||||
return;
|
||||
};
|
||||
let mut b = Builder::new(2);
|
||||
let (bg, x, y) = (b.demo(DemoScene::OceanSunset), b.demo(DemoScene::Aurora), b.demo(DemoScene::Dunes));
|
||||
b.place(0, bg, 0, 48);
|
||||
let ca = b.place(1, x, 0, 24);
|
||||
let cb = b.place(1, y, 24, 24);
|
||||
for id in [ca, cb] {
|
||||
b.fixed(id, "opacity", &[("blend", blend("Screen"))]);
|
||||
}
|
||||
b.transition(1, "cross_dissolve", ca, cb, 24, 12);
|
||||
let s = b.at(22);
|
||||
let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, RATE.tick_of(s.frame), RenderOptions::default(), &s.sources);
|
||||
let filmcraft_render::plan::FramePlan::Layers { layers, .. } = &plan else { panic!("planned as a CPU image") };
|
||||
assert!(layers.iter().all(|l| l.blend == filmcraft_render::Blend::Normal));
|
||||
let mut c = filmcraft_gpu::GpuCompositor::new(&dev, &q);
|
||||
c.composite(&plan);
|
||||
let (gw, gh, mut px) = c.read_output().expect("GPU readback");
|
||||
px.as_chunks_mut::<4>().0.iter_mut().for_each(|p| p[3] = 255);
|
||||
let d = diff(&render_cpu(&s), &Rgba8::new(gw, gh, px)).unwrap();
|
||||
eprintln!("Screen clips in a cross dissolve: GPU vs CPU {d}");
|
||||
assert!(d.p99 <= 6 && d.mean_abs * 4.0 / 3.0 < 1.5, "{d}");
|
||||
}
|
||||
|
||||
/// Standard effect chains run on the GPU: every clip below plans as a GPU layer with its effects
|
||||
/// (no CPU-rendered image), including keyframed parameters, transformed clips, partial opacity
|
||||
/// and blend modes, and matches the CPU render with the GPU parity tolerance (p99 ≤ 6, mean < 1.5
|
||||
/// away from layer outlines) at three times. A clip with an effect the GPU stage lacks (Emboss)
|
||||
/// still falls back to a CPU layer image.
|
||||
#[test]
|
||||
fn gpu_effect_chains_match_cpu_render() {
|
||||
let mut b = Builder::new(5);
|
||||
let (bg, dunes, aurora, bars, sunset) =
|
||||
(b.demo(DemoScene::OceanSunset), b.demo(DemoScene::Dunes), b.demo(DemoScene::Aurora), b.media(Generator::BarsAndTone), b.demo(DemoScene::OceanSunset));
|
||||
let v1 = b.place(0, bg, 0, 48);
|
||||
b.effect(v1, "brightness_contrast", &[("brightness", fl(-10.0)), ("contrast", fl(20.0))]);
|
||||
b.effect(v1, "gaussian_blur", &[("repeat_edge", ParamValue::Bool(true))]);
|
||||
// keyframed blur: evaluated at the frame's time
|
||||
let p = b.clip(v1).effects.last_mut().and_then(|e| e.params.get_mut("blurriness")).expect("blurriness");
|
||||
p.put_keyframe(Tick::ZERO, fl(2.0));
|
||||
p.put_keyframe(RATE.tick_of(24), fl(20.0));
|
||||
let v2 = b.place(1, dunes, 0, 48);
|
||||
b.fixed(v2, "motion", &[("scale", fl(60.0)), ("rotation", fl(12.0)), ("position", pt(130.0, 80.0))]);
|
||||
b.fixed(v2, "opacity", &[("opacity", fl(70.0)), ("blend", blend("Screen"))]);
|
||||
b.effect(v2, "tint", &[("amount", fl(60.0))]);
|
||||
b.effect(v2, "unsharp_mask", &[("amount", fl(120.0)), ("radius", fl(2.0))]);
|
||||
b.effect(v2, "crop", &[("left", fl(8.0)), ("bottom", fl(12.0)), ("feather", fl(6.0))]);
|
||||
let v3 = b.place(2, aurora, 0, 48);
|
||||
b.fixed(v3, "motion", &[("scale", fl(45.0)), ("rotation", fl(-20.0)), ("position", pt(210.0, 110.0))]);
|
||||
b.fixed(v3, "opacity", &[("opacity", fl(85.0))]);
|
||||
b.effect(v3, "levels", &[("in_black", fl(20.0)), ("gamma", fl(130.0))]);
|
||||
b.effect(v3, "mirror", &[("angle", fl(60.0))]);
|
||||
b.effect(v3, "directional_blur", &[("length", fl(4.0)), ("direction", fl(45.0))]);
|
||||
b.effect(v3, "transform", &[("rotation", fl(10.0)), ("scale_height", fl(110.0))]);
|
||||
let v4 = b.place(3, bars, 0, 48);
|
||||
b.fixed(v4, "motion", &[("scale", fl(30.0)), ("position", pt(250.0, 50.0))]);
|
||||
b.fixed(v4, "opacity", &[("blend", blend("Multiply"))]);
|
||||
b.effect(v4, "posterize", &[("levels", fl(5.0))]);
|
||||
b.effect(v4, "invert", &[("blend", fl(40.0))]);
|
||||
b.effect(v4, "offset", &[("shift", pt(400.0, 120.0))]);
|
||||
b.effect(v4, "horizontal_flip", &[]);
|
||||
b.effect(v4, "black_white", &[]);
|
||||
let v5 = b.place(4, sunset, 0, 48);
|
||||
b.fixed(v5, "motion", &[("scale", fl(25.0)), ("position", pt(60.0, 140.0))]);
|
||||
b.effect(v5, "emboss", &[]);
|
||||
let mut s = b.at(12);
|
||||
let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, RATE.tick_of(s.frame), RenderOptions::default(), &s.sources);
|
||||
let filmcraft_render::plan::FramePlan::Layers { layers, .. } = &plan else { panic!("planned as a CPU image") };
|
||||
assert_eq!(layers.len(), 5);
|
||||
for (i, l) in layers.iter().take(4).enumerate() {
|
||||
let fx = l.fx.as_ref().unwrap_or_else(|| panic!("V{}: not a GPU effect layer", i + 1));
|
||||
assert!(!fx.ops.is_empty());
|
||||
}
|
||||
assert!(layers[4].fx.is_none(), "Emboss is rendered on the CPU");
|
||||
let Some((dev, q)) = device() else {
|
||||
eprintln!("SKIPPED (gpu effect parity): no GPU adapter");
|
||||
return;
|
||||
};
|
||||
let mut c = filmcraft_gpu::GpuCompositor::new(&dev, &q);
|
||||
let mut failures = Vec::new();
|
||||
for frame in [0, 12, 30] {
|
||||
s.frame = frame;
|
||||
let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, RATE.tick_of(frame), RenderOptions::default(), &s.sources);
|
||||
c.composite(&plan);
|
||||
let (gw, gh, mut px) = c.read_output().expect("GPU readback");
|
||||
px.as_chunks_mut::<4>().0.iter_mut().for_each(|p| p[3] = 255);
|
||||
let gpu = Rgba8::new(gw, gh, px);
|
||||
let cpu = render_cpu(&s);
|
||||
let d = diff(&cpu, &gpu).unwrap();
|
||||
let (p99, mean) = interior_diff(&plan, &cpu, &gpu);
|
||||
eprintln!("effect chains @ {frame}: GPU vs CPU interior p99 {p99}, mean {mean:.3}; whole frame {d}");
|
||||
if p99 > 6 || mean >= 1.5 {
|
||||
let dir = filmcraft_testkit::golden::failures_dir();
|
||||
let _ = filmcraft_testkit::golden::write_png(&dir.join(format!("fx_chains_{frame}.gpu.png")), &gpu);
|
||||
let _ = filmcraft_testkit::golden::write_png(&dir.join(format!("fx_chains_{frame}.cpu.png")), &cpu);
|
||||
failures.push(format!("frame {frame}: interior p99 {p99}, mean {mean}; {d}; images in {}", dir.display()));
|
||||
}
|
||||
}
|
||||
// ½ playback resolution: the working images (and the effects' pixel radii) shrink with it
|
||||
let opts = RenderOptions { scale: 0.5, ..RenderOptions::default() };
|
||||
let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, RATE.tick_of(s.frame), opts, &s.sources);
|
||||
let filmcraft_render::plan::FramePlan::Layers { layers, .. } = &plan else { panic!("½: planned as a CPU image") };
|
||||
assert!(layers.iter().take(4).all(|l| l.fx.is_some()));
|
||||
c.composite(&plan);
|
||||
let (gw, gh, mut px) = c.read_output().expect("GPU readback");
|
||||
px.as_chunks_mut::<4>().0.iter_mut().for_each(|p| p[3] = 255);
|
||||
let gpu = Rgba8::new(gw, gh, px);
|
||||
let img = render_sequence(&s.project, s.seq, RATE.tick_of(s.frame), opts, &s.sources);
|
||||
let cpu = Rgba8::new(img.w as u32, img.h as u32, img.over_black_rgba8());
|
||||
let (p99, mean) = interior_diff(&plan, &cpu, &gpu);
|
||||
eprintln!("effect chains @ ½: GPU vs CPU interior p99 {p99}, mean {mean:.3}; whole frame {}", diff(&cpu, &gpu).unwrap());
|
||||
if p99 > 6 || mean >= 1.5 {
|
||||
failures.push(format!("½ resolution: interior p99 {p99}, mean {mean}"));
|
||||
}
|
||||
assert!(failures.is_empty(), "GPU effect chains differ from the CPU render:\n{}", failures.join("\n"));
|
||||
}
|
||||
Loaded 100 of 246 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user