Compare commits
84
Commits
No files matched your search
@@ -3,6 +3,49 @@
|
||||
These rules apply to every human and AI contributor. `CLAUDE.md` holds the working instructions; this
|
||||
file holds the rules that must never be broken. When the two disagree, this file wins.
|
||||
|
||||
## 0. Never crash
|
||||
|
||||
People trust FilmCraft with hours of work. A malformed file, a bad command or MCP parameter, a corrupt
|
||||
preset or project, an odd keystroke or a full disk must give an error the user (or agent) can act on,
|
||||
never a crash and never lost work. **This rule outranks feature work:** don't ship a feature by adding a
|
||||
panic path, and fix a crash before building on top of it. The cross-app standard is
|
||||
[`craftrules/standards/never-crash.md`](https://github.com/storytold/craftrules/blob/main/standards/never-crash.md).
|
||||
|
||||
1. **Fail with `Result<T, E>`.** Return the crate's error type and propagate with `?`; add context
|
||||
(`map_err`, an error variant) instead of discarding it. In the UI, report the error (status bar, error
|
||||
dialog) and carry on.
|
||||
2. **No panicking shortcuts outside tests:** no `unwrap()`, `expect()`, `panic!`, `unreachable!`, `todo!`
|
||||
or `unimplemented!`. Use `?`, `ok_or(..)?`, `let Some(x) = .. else { return Err(..) }`, `if let`, or a
|
||||
fallback that is truly correct (never one that silently corrupts a project). An unfinished feature
|
||||
returns an "unsupported" error. The only exception is a call that is provably infallible from local
|
||||
code alone (a literal that always parses): a single-item `#[allow(clippy::expect_used)]` with the reason.
|
||||
3. **No `unsafe`** (`unsafe_code = "forbid"` for the workspace). The single exception is
|
||||
`crates/platform`, which needs `unsafe` to call the operating system's media APIs (hardware
|
||||
video decoding through VideoToolbox; later encoding and other OSes) and does nothing else. It
|
||||
uses `unsafe_code = "deny"` with `#[allow(unsafe_code)]` only on its FFI modules, a
|
||||
`// SAFETY:` comment on every `unsafe` block, a safe `Result`-returning public API, and the
|
||||
pure-Rust decoder as the tested fallback ([ADR 0001](docs/adr/0001-platform-ffi.md)).
|
||||
4. **Every input-derived number is hostile.** Media files, project and interchange files, presets, fonts,
|
||||
CLI / MCP / control-channel parameters and UI state are untrusted. Index and slice with `get()` (or
|
||||
validate bounds once, up front, for a hot loop); slice strings only at char boundaries; use
|
||||
`checked_*` / `saturating_*` for lengths, offsets and counts; never divide by a value that can be zero
|
||||
(frame rates, timescales, sample rates, sizes); don't cast negative or NaN values to integers; don't
|
||||
call `clamp` with bounds that can cross; cap allocations sized by input.
|
||||
5. **Bound recursion and loops.** Projects can be cyclic (nested sequences) or deeply nested: walk them
|
||||
with seen-sets or depth limits.
|
||||
6. **Don't cascade.** Lock poisoning is not fatal: `lock().unwrap_or_else(PoisonError::into_inner)`.
|
||||
Every thread or job (export, previews, proxies, decoding workers…) runs under `catch_unwind` and
|
||||
reports a failure as an error: a dead worker leaves monitors blank or jobs "running" forever.
|
||||
7. **Last-resort guard.** The panic hook (`crates/ui-egui/src/crash.rs`) logs every panic, and the UI pass
|
||||
runs under `catch_unwind`, so an escaped panic becomes an error window and the session (and unsaved
|
||||
project) survives. It is a safety net, not a substitute for rules 1–6. Keep `panic = "unwind"`.
|
||||
8. **Prove it.** Every crash fix comes with a small synthetic regression test that panicked before the
|
||||
fix. Parsers and decoders get mutation-fuzz tests (truncation, bit flips, corrupt sizes) run under
|
||||
`catch_unwind`; new commands get hostile-parameter tests. Fuzzer findings become tests.
|
||||
9. **Enforced:** clean crates carry `#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic,
|
||||
clippy::unimplemented, clippy::todo, clippy::unreachable)]` (moving to `[workspace.lints.clippy]` once
|
||||
every crate is clean). `clippy.toml` allows them in tests: a failing test should fail loudly.
|
||||
|
||||
## 1. Assets: no Adobe artwork, every asset licensed and attributed
|
||||
|
||||
This rule is absolute. Breaking it is the most serious mistake a contributor can make on this project.
|
||||
@@ -44,7 +87,8 @@ reproduce an image (for example, point lists traced from someone else's icon).
|
||||
and are listed in `ATTRIBUTION.md`.
|
||||
7. **When in doubt, leave it out** and draw or generate it yourself.
|
||||
8. **The one exception: first-party ArtCraft brand marks.** The ArtCraft name and logos in `docs/brand/`
|
||||
(supplied by the ArtCraft team) are trademarks, all rights reserved, used with permission. They still
|
||||
are trademarks of the ArtCraft Team, not open source, usable only unmodified and only in the context of
|
||||
FilmCraft under `docs/brand/LICENSE-brand.txt`; forks and modified versions must remove them. They still
|
||||
need a sidecar and an `ATTRIBUTION.md` row (licence `LicenseRef-ArtCraft-Trademark`). No other
|
||||
non-open asset is allowed, and this exception never covers third-party marks (Adobe, Discord, GitHub
|
||||
and other logos stay out; draw a generic icon instead).
|
||||
@@ -67,6 +111,12 @@ reproduce an image (for example, point lists traced from someone else's icon).
|
||||
|
||||
## 3. Engineering rules
|
||||
|
||||
### 3.1 Never crash
|
||||
|
||||
See [§0](#0-never-crash). It outranks every other engineering rule.
|
||||
|
||||
### 3.2 The rest
|
||||
|
||||
See `CLAUDE.md`: pure Rust, dependency layering (`cargo xtask layers`), exact `Tick` time, everything is
|
||||
a command, everything is agent-drivable, and the quality gates (`cargo xtask ci`) before every commit.
|
||||
|
||||
|
||||
+15
-10
@@ -60,6 +60,8 @@ procedural demo footage (`crates/media/src/generators.rs`); no external media. R
|
||||
| `crates/golden/goldens/hdr_tone_map.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `hdr_tone_map`) from procedurally generated media re-encoded as Rec. 2100 PQ | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/graphic_title.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `graphic_title`) from procedurally generated media and bundled OFL fonts | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/lumetri_basic.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `lumetri_basic`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/lumetri_hdr_pq.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `lumetri_hdr_pq`) from procedurally generated media re-encoded as Rec. 2100 PQ | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/lumetri_presets_grid.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, `lumetri_presets_grid`): FilmCraft's own Lumetri presets on its procedural preview picture | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/text_burnin.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `text_burnin`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/transform_opacity.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `transform_opacity`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/transition_cross_dissolve.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `transition_cross_dissolve`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
@@ -101,23 +103,26 @@ is traced or derived from Adobe artwork.
|
||||
| Demo footage (ocean sunset, aurora, city night, dunes, forest, plasma), bars and tone, counting leader, colour matte | `crates/media/src/generators.rs` |
|
||||
| Built-in graphics templates (Lower Third – Slab / Rule, Ticker – Crawl, Title – Centered / Boxed, End Credits – Roll, Callout – Pointer / Tag) and their placeholder text | `crates/project/src/gtemplate.rs` (`builtin_templates`): original designs defined in code with the bundled Inter font; thumbnails are rendered at runtime, no template, image or font files |
|
||||
| Lumetri "Look" presets (Teal & Orange, Warm Film, …) | `crates/render/src/effects.rs` (`apply_look`): procedural colour transforms, not LUT files |
|
||||
| Lumetri Presets (Effects panel: Cinematic, Film Emulation, Monochrome, Technical) and their preview picture | `crates/render/src/lumetri_presets.rs`: Lumetri parameter values chosen by contributors, descriptive names, and a procedural preview picture (dusk sky, ridges, lake, grey ramp, colour chips) |
|
||||
| Colour palette and layout metrics | `crates/ui-egui/src/theme.rs`: colour values and sizes only; no artwork |
|
||||
|
||||
## First-party brand marks
|
||||
|
||||
The ArtCraft name and logo are trademarks of the ArtCraft team, used in FilmCraft with permission. They are not
|
||||
openly licensed and are not covered by the project licence; forks must remove or replace them (AGENTS.md §1.8).
|
||||
The ArtCraft name, wordmark and logos are trademarks of the ArtCraft Team. They are not open source and
|
||||
are not covered by the project licence; they may be used only unmodified and only in the context of
|
||||
FilmCraft, under [`docs/brand/LICENSE-brand.txt`](docs/brand/LICENSE-brand.txt). Forks and modified versions must
|
||||
remove them (AGENTS.md §1.8).
|
||||
|
||||
| File | Author | Source | Licence |
|
||||
|---|---|---|---|
|
||||
| `docs/brand/artcraft-logo-white.png` | ArtCraft team | Original work: first-party ArtCraft brand mark (ArtCraft wordmark, light ink for dark backgrounds) | ArtCraft trademark, all rights reserved; used with permission (not MIT/Apache) |
|
||||
| `docs/brand/artcraft-logo-white.svg` | ArtCraft team | Original work: first-party ArtCraft brand mark (ArtCraft wordmark, light ink for dark backgrounds) | ArtCraft trademark, all rights reserved; used with permission (not MIT/Apache) |
|
||||
| `docs/brand/artcraft-logo.png` | ArtCraft team | Original work: first-party ArtCraft brand mark (ArtCraft wordmark (symbol + "ARTCRAFT"), dark ink for light backgrounds) | ArtCraft trademark, all rights reserved; used with permission (not MIT/Apache) |
|
||||
| `docs/brand/artcraft-logo.svg` | ArtCraft team | Original work: first-party ArtCraft brand mark (ArtCraft wordmark (symbol + "ARTCRAFT"), dark ink for light backgrounds) | ArtCraft trademark, all rights reserved; used with permission (not MIT/Apache) |
|
||||
| `docs/brand/artcraft-mark-black.png` | ArtCraft team | Original work: first-party ArtCraft brand mark (ArtCraft symbol ("A" mark), black) | ArtCraft trademark, all rights reserved; used with permission (not MIT/Apache) |
|
||||
| `docs/brand/artcraft-mark-black.svg` | ArtCraft team | Original work: first-party ArtCraft brand mark (ArtCraft symbol ("A" mark), black) | ArtCraft trademark, all rights reserved; used with permission (not MIT/Apache) |
|
||||
| `docs/brand/artcraft-mark.png` | ArtCraft team | Original work: first-party ArtCraft brand mark (ArtCraft symbol ("A" mark), blue) | ArtCraft trademark, all rights reserved; used with permission (not MIT/Apache) |
|
||||
| `docs/brand/artcraft-mark.svg` | ArtCraft team | Original work: first-party ArtCraft brand mark (ArtCraft symbol ("A" mark), blue) | ArtCraft trademark, all rights reserved; used with permission (not MIT/Apache) |
|
||||
| `docs/brand/artcraft-logo-white.png` | ArtCraft Team | Original work: first-party ArtCraft brand mark (ArtCraft wordmark, light ink for dark backgrounds) | ArtCraft trademark, not open source; `docs/brand/LICENSE-brand.txt` (not MIT/Apache) |
|
||||
| `docs/brand/artcraft-logo-white.svg` | ArtCraft Team | Original work: first-party ArtCraft brand mark (ArtCraft wordmark, light ink for dark backgrounds) | ArtCraft trademark, not open source; `docs/brand/LICENSE-brand.txt` (not MIT/Apache) |
|
||||
| `docs/brand/artcraft-logo.png` | ArtCraft Team | Original work: first-party ArtCraft brand mark (ArtCraft wordmark (symbol + "ARTCRAFT"), dark ink for light backgrounds) | ArtCraft trademark, not open source; `docs/brand/LICENSE-brand.txt` (not MIT/Apache) |
|
||||
| `docs/brand/artcraft-logo.svg` | ArtCraft Team | Original work: first-party ArtCraft brand mark (ArtCraft wordmark (symbol + "ARTCRAFT"), dark ink for light backgrounds) | ArtCraft trademark, not open source; `docs/brand/LICENSE-brand.txt` (not MIT/Apache) |
|
||||
| `docs/brand/artcraft-mark-black.png` | ArtCraft Team | Original work: first-party ArtCraft brand mark (ArtCraft symbol ("A" mark), black) | ArtCraft trademark, not open source; `docs/brand/LICENSE-brand.txt` (not MIT/Apache) |
|
||||
| `docs/brand/artcraft-mark-black.svg` | ArtCraft Team | Original work: first-party ArtCraft brand mark (ArtCraft symbol ("A" mark), black) | ArtCraft trademark, not open source; `docs/brand/LICENSE-brand.txt` (not MIT/Apache) |
|
||||
| `docs/brand/artcraft-mark.png` | ArtCraft Team | Original work: first-party ArtCraft brand mark (ArtCraft symbol ("A" mark), blue) | ArtCraft trademark, not open source; `docs/brand/LICENSE-brand.txt` (not MIT/Apache) |
|
||||
| `docs/brand/artcraft-mark.svg` | ArtCraft Team | Original work: first-party ArtCraft brand mark (ArtCraft symbol ("A" mark), blue) | ArtCraft trademark, not open source; `docs/brand/LICENSE-brand.txt` (not MIT/Apache) |
|
||||
|
||||
## Downloaded at runtime
|
||||
|
||||
|
||||
@@ -9,9 +9,10 @@ FilmCraft is a clean-room, open-source, pure-Rust non-linear video editor target
|
||||
3. Follow the autonomous operation protocol (`plan/execution-plan.md` §7; public summary: [`docs/agents.md`](docs/agents.md) §5–7). Don't stop to ask unless it lists the decision as the user's.
|
||||
|
||||
## Non-negotiables
|
||||
**Read [`AGENTS.md`](AGENTS.md) first; its rules override everything here.** In particular (§1): never use Adobe iconography, images or any other Adobe asset. Every asset must be openly licensed (OSS / public domain / Creative Commons, or original work by a contributor) and must have a `<file>.attribution` sidecar plus an entry in `ATTRIBUTION.md`. `cargo xtask assets` enforces this.
|
||||
**Read [`AGENTS.md`](AGENTS.md) first; its rules override everything here.**
|
||||
- **Never crash** ([`AGENTS.md`](AGENTS.md) §0, standard: [`craftrules/standards/never-crash.md`](https://github.com/storytold/craftrules/blob/main/standards/never-crash.md)). People depend on FilmCraft; a crash loses their work, so this outranks feature work. No `unwrap`/`expect`/`panic!`/`unreachable!`/`todo!`/`unimplemented!` or `unsafe` outside tests: return `Result<T, E>` and propagate with `?` (`ok_or`, `let … else`, `if let`). Treat every number from media, project files, presets or CLI/MCP/control params as hostile: `get()` instead of `[i]`, checked/saturating math, no division by possibly-zero values, capped allocations, bounded recursion. `lock().unwrap_or_else(PoisonError::into_inner)`; background jobs run under `catch_unwind`. The panic hook + UI `catch_unwind` (`crates/ui-egui/src/crash.rs`) are the last-resort guard, not a licence. Every crash fix gets a regression test that panicked before. In particular (§1): never use Adobe iconography, images or any other Adobe asset. Every asset must be openly licensed (OSS / public domain / Creative Commons, or original work by a contributor) and must have a `<file>.attribution` sidecar plus an entry in `ATTRIBUTION.md`. `cargo xtask assets` enforces this.
|
||||
- **Clean-room.** Never read/disassemble anything inside Adobe app bundles (names/listings only). Never copy Adobe icons, presets, LUTs, fonts. Never copy GPL/LGPL/AGPL code (FFmpeg, x264, x265, MLT, Kdenlive, Shotcut, Olive, LAME…). ffmpeg/ffprobe run only as external test oracles / fixture generators — never linked or shipped.
|
||||
- **Pure Rust** in the product. OS media APIs only via Rust bindings in `crates/platform` (planned; not created yet), optional, behind traits.
|
||||
- **Pure Rust** in the product. OS media APIs only via Rust bindings in `crates/platform` (hardware decoding; the one crate allowed `unsafe`, see `AGENTS.md` §0.3 and `docs/adr/0001-platform-ffi.md`), optional, behind traits, with our decoders as the fallback.
|
||||
- **Layering** ([`docs/architecture.md`](docs/architecture.md) §1, enforced by `cargo xtask layers`): nothing below L5 depends on egui/eframe/winit/rfd/cpal. L0 codec/container crates depend only on `filmcraft-bitstream`.
|
||||
- **Exact time:** all time is `filmcraft_time::Tick` (254 016 000 000/s). Never use f64 seconds for edit math.
|
||||
- **Everything is a command** (`crates/engine`): id, label, menu path, shortcut, params, enabled(), run(). UI, CLI, control channel and MCP all dispatch by id.
|
||||
|
||||
+4
-1
@@ -5,7 +5,10 @@ Thanks for helping. Before you start:
|
||||
1. Read **[AGENTS.md](AGENTS.md)**. Its rules are absolute. In particular: no Adobe assets of any
|
||||
kind, every asset openly licensed with an attribution sidecar, no GPL/LGPL code, and ffmpeg only
|
||||
as an external test oracle.
|
||||
2. Follow **[docs/contributing.md](docs/contributing.md)**: setup per OS, building and running,
|
||||
2. **Never crash.** A crash loses someone's work, so this rule outranks feature work: no `unwrap()`,
|
||||
`expect()`, `panic!`, `unreachable!`, `todo!`, `unimplemented!` or `unsafe` in production code;
|
||||
return `Result` and treat every input as hostile. See [AGENTS.md](AGENTS.md) §0.
|
||||
3. Follow **[docs/contributing.md](docs/contributing.md)**: setup per OS, building and running,
|
||||
the quality gates (`cargo xtask ci`), commit conventions, and how to add commands, effects,
|
||||
transitions, codecs, panels and assets.
|
||||
|
||||
|
||||
Generated
+116
-45
@@ -1655,13 +1655,14 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"chrono",
|
||||
"cpal",
|
||||
"eframe",
|
||||
"egui",
|
||||
"filmcraft-engine",
|
||||
"filmcraft-platform",
|
||||
"filmcraft-ui-egui",
|
||||
"log",
|
||||
"muda",
|
||||
@@ -1676,7 +1677,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-aac"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"filmcraft-testkit",
|
||||
@@ -1686,7 +1687,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-ac3"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
"thiserror 2.0.21",
|
||||
@@ -1694,14 +1695,14 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-audio-dsp"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-automation"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"base64 0.22.1",
|
||||
"filmcraft-engine",
|
||||
@@ -1717,7 +1718,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-av1"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"filmcraft-testkit",
|
||||
@@ -1728,14 +1729,14 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-bitstream"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"proptest",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-captions"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"filmcraft-project",
|
||||
@@ -1748,11 +1749,11 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-cfb"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-cli"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-automation",
|
||||
"filmcraft-codecs",
|
||||
@@ -1761,6 +1762,7 @@ dependencies = [
|
||||
"filmcraft-mpegts",
|
||||
"filmcraft-mxf",
|
||||
"filmcraft-ogg",
|
||||
"filmcraft-platform",
|
||||
"filmcraft-time",
|
||||
"serde_json",
|
||||
"tokio",
|
||||
@@ -1768,7 +1770,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-codecs"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-aac",
|
||||
"filmcraft-ac3",
|
||||
@@ -1800,7 +1802,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-color"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"proptest",
|
||||
"serde",
|
||||
@@ -1808,7 +1810,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-dnx"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"filmcraft-testkit",
|
||||
@@ -1818,7 +1820,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-edit"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-media",
|
||||
"filmcraft-project",
|
||||
@@ -1830,7 +1832,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-engine"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-audio-dsp",
|
||||
"filmcraft-captions",
|
||||
@@ -1861,7 +1863,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-export"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-aac",
|
||||
"filmcraft-audio-dsp",
|
||||
@@ -1890,7 +1892,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-format"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-media",
|
||||
"filmcraft-project",
|
||||
@@ -1902,7 +1904,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-frame"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"filmcraft-time",
|
||||
@@ -1912,7 +1914,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-geom"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"proptest",
|
||||
"serde",
|
||||
@@ -1920,7 +1922,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-golden"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"filmcraft-frame",
|
||||
@@ -1937,7 +1939,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-gpu"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"filmcraft-frame",
|
||||
@@ -1954,7 +1956,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-h264"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"filmcraft-testkit",
|
||||
@@ -1964,7 +1966,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-h264enc"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"filmcraft-testkit",
|
||||
@@ -1974,7 +1976,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-hevc"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"filmcraft-testkit",
|
||||
@@ -1984,7 +1986,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-interchange"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-cfb",
|
||||
"filmcraft-color",
|
||||
@@ -2001,7 +2003,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-isobmff"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"filmcraft-testkit",
|
||||
@@ -2012,7 +2014,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-matroska"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
"serde_json",
|
||||
@@ -2020,7 +2022,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-media"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"filmcraft-frame",
|
||||
@@ -2033,7 +2035,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-mpeg2v"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
"rayon",
|
||||
@@ -2042,7 +2044,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-mpegts"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
"thiserror 2.0.21",
|
||||
@@ -2050,28 +2052,46 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-mxf"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-ogg"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-opus"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-platform"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-codecs",
|
||||
"filmcraft-color",
|
||||
"filmcraft-frame",
|
||||
"filmcraft-isobmff",
|
||||
"filmcraft-media",
|
||||
"filmcraft-testkit",
|
||||
"filmcraft-time",
|
||||
"log",
|
||||
"objc2-core-foundation",
|
||||
"objc2-core-media",
|
||||
"objc2-core-video",
|
||||
"objc2-video-toolbox",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-project"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"filmcraft-geom",
|
||||
@@ -2085,7 +2105,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-prores"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"filmcraft-testkit",
|
||||
@@ -2095,7 +2115,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-render"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-audio-dsp",
|
||||
"filmcraft-captions",
|
||||
@@ -2115,7 +2135,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-scopes"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"serde",
|
||||
@@ -2124,7 +2144,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-speech"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"candle-core",
|
||||
"candle-nn",
|
||||
@@ -2142,14 +2162,14 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-testkit"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"png",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-text"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"harfrust",
|
||||
@@ -2160,7 +2180,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-time"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"proptest",
|
||||
"serde",
|
||||
@@ -2168,7 +2188,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-ui-egui"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"eframe",
|
||||
"egui",
|
||||
@@ -2183,6 +2203,7 @@ dependencies = [
|
||||
"filmcraft-gpu",
|
||||
"filmcraft-interchange",
|
||||
"filmcraft-media",
|
||||
"filmcraft-platform",
|
||||
"filmcraft-project",
|
||||
"filmcraft-render",
|
||||
"filmcraft-scopes",
|
||||
@@ -2200,7 +2221,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-vp9"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-bitstream",
|
||||
"rayon",
|
||||
@@ -2209,7 +2230,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "filmcraft-web"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"eframe",
|
||||
"egui",
|
||||
@@ -3779,7 +3800,9 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2a180dd8642fa45cdb7dd721cd4c11b1cadd4929ce112ebd8b9f5803cc79d536"
|
||||
dependencies = [
|
||||
"bitflags 2.13.2",
|
||||
"block2 0.6.2",
|
||||
"dispatch2",
|
||||
"libc",
|
||||
"objc2 0.6.4",
|
||||
]
|
||||
|
||||
@@ -3820,6 +3843,37 @@ dependencies = [
|
||||
"objc2-foundation 0.2.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2-core-media"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "05ec576860167a15dd9fce7fbee7512beb4e31f532159d3482d1f9c6caedf31d"
|
||||
dependencies = [
|
||||
"bitflags 2.13.2",
|
||||
"block2 0.6.2",
|
||||
"dispatch2",
|
||||
"objc2 0.6.4",
|
||||
"objc2-core-audio",
|
||||
"objc2-core-audio-types",
|
||||
"objc2-core-foundation",
|
||||
"objc2-core-video",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2-core-video"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d425caf1df73233f29fd8a5c3e5edbc30d2d4307870f802d18f00d83dc5141a6"
|
||||
dependencies = [
|
||||
"bitflags 2.13.2",
|
||||
"block2 0.6.2",
|
||||
"objc2 0.6.4",
|
||||
"objc2-core-foundation",
|
||||
"objc2-core-graphics",
|
||||
"objc2-io-surface",
|
||||
"objc2-metal 0.3.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2-encode"
|
||||
version = "4.1.0"
|
||||
@@ -3992,6 +4046,23 @@ dependencies = [
|
||||
"objc2-foundation 0.2.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "objc2-video-toolbox"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "05bf9a3c14831a7d9641b0d81d87dd913ee238a012b2fde27db5a84b56f5df3e"
|
||||
dependencies = [
|
||||
"bitflags 2.13.2",
|
||||
"block2 0.6.2",
|
||||
"objc2 0.6.4",
|
||||
"objc2-core-foundation",
|
||||
"objc2-core-graphics",
|
||||
"objc2-core-media",
|
||||
"objc2-core-video",
|
||||
"objc2-foundation 0.3.2",
|
||||
"objc2-metal 0.3.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "once_cell"
|
||||
version = "1.21.4"
|
||||
@@ -7140,7 +7211,7 @@ checksum = "e450f9b2ed1dff33c94c12589a87338689467b9c4f5d8a5710bd09a847d2c8a7"
|
||||
|
||||
[[package]]
|
||||
name = "xtask"
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
dependencies = [
|
||||
"filmcraft-testkit",
|
||||
"serde_json",
|
||||
|
||||
+3
-2
@@ -4,7 +4,7 @@ members = ["crates/*", "apps/*", "xtask"]
|
||||
default-members = ["crates/*", "apps/*", "xtask"]
|
||||
|
||||
[workspace.package]
|
||||
version = "0.1.1"
|
||||
version = "0.2.0"
|
||||
edition = "2024"
|
||||
license = "MIT OR Apache-2.0"
|
||||
rust-version = "1.95"
|
||||
@@ -51,6 +51,7 @@ filmcraft-format = { path = "crates/format" }
|
||||
filmcraft-engine = { path = "crates/engine" }
|
||||
filmcraft-ui-egui = { path = "crates/ui-egui" }
|
||||
filmcraft-automation = { path = "crates/automation" }
|
||||
filmcraft-platform = { path = "crates/platform" }
|
||||
filmcraft-testkit = { path = "crates/testkit" }
|
||||
|
||||
# foundation
|
||||
@@ -69,7 +70,7 @@ egui_kittest = "0.36"
|
||||
rfd = "0.17"
|
||||
|
||||
[workspace.lints.rust]
|
||||
unsafe_code = "deny"
|
||||
unsafe_code = "forbid"
|
||||
|
||||
[workspace.lints.clippy]
|
||||
dbg_macro = "warn"
|
||||
|
||||
+151
-150
@@ -1,176 +1,177 @@
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
1. Definitions.
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
1. Definitions.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 The FilmCraft contributors
|
||||
Copyright (c) 2026 ArtCraft Team and the FilmCraft contributors
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
FilmCraft
|
||||
Copyright (c) 2026 ArtCraft Team and the FilmCraft contributors.
|
||||
Licensed under the MIT License or the Apache License, Version 2.0, at your option
|
||||
(LICENSE-MIT, LICENSE-APACHE).
|
||||
|
||||
The ArtCraft name, wordmark and logos in docs/brand/ are trademarks of the ArtCraft Team. They are
|
||||
not open source and are not covered by the license above; their terms are in
|
||||
docs/brand/LICENSE-brand.txt.
|
||||
|
||||
Third-party material
|
||||
|
||||
This product bundles third-party material under its own licenses, listed below. The per-file
|
||||
list (author, source, license) is in ATTRIBUTION.md.
|
||||
|
||||
Inter font (assets/fonts/Inter-*.ttf) — OFL-1.1. See assets/fonts/OFL-Inter.txt.
|
||||
Noto Serif font (assets/fonts/NotoSerif-Regular.ttf) — OFL-1.1. See assets/fonts/OFL-NotoSerif.txt.
|
||||
JetBrains Mono font (assets/fonts/JetBrainsMono-Regular.ttf) — OFL-1.1. See assets/fonts/OFL-JetBrainsMono.txt.
|
||||
Public-domain film and NASA frames and CC0 music shown in README screenshots (docs/images/) — public domain / CC0-1.0. See ATTRIBUTION.md.
|
||||
@@ -20,7 +20,7 @@
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="#license"><img alt="License: MIT or Apache-2.0" src="https://img.shields.io/badge/license-MIT%20%7C%20Apache--2.0-8b5cf6"></a>
|
||||
<a href="#license-and-credits"><img alt="License: MIT OR Apache-2.0" src="https://img.shields.io/badge/license-MIT%20%7C%20Apache--2.0-8b5cf6"></a>
|
||||
<img alt="Written in pure Rust" src="https://img.shields.io/badge/pure-Rust-6a3fd6?logo=rust&logoColor=white">
|
||||
<img alt="Runs on macOS, Windows and Linux" src="https://img.shields.io/badge/runs%20on-macOS%20%7C%20Windows%20%7C%20Linux-8b5cf6">
|
||||
<a href="#status"><img alt="Status: young and moving fast" src="https://img.shields.io/badge/status-young%20and%20moving%20fast-6a3fd6"></a>
|
||||
@@ -184,7 +184,7 @@ No FFmpeg inside. The video codecs, AAC, Opus and the containers are our own Rus
|
||||
| **Matroska / WebM** | ✓ | | Lacing, Cues, header stripping, HDR colour metadata |
|
||||
| **Stills** | ✓ | ✓ | Import PNG, JPEG, GIF, WebP, TIFF and BMP; export PNG sequences and animated GIF |
|
||||
|
||||
MXF and hardware decode are next ([roadmap](ROADMAP.md)).
|
||||
Also imported: MPEG-2 / MPEG-1 video, AC-3, MP2, MXF (OP1a / OP-Atom), MPEG transport and program streams (AVCHD, broadcast, DVD), Ogg and image sequences. Not yet: HEVC and AV1 export, camera RAW, E-AC-3. Hardware decoding works on macOS (VideoToolbox), with Linux and Windows to follow ([#30](https://github.com/storytold/filmcraft/issues/30)).
|
||||
|
||||
<br>
|
||||
|
||||
@@ -266,11 +266,26 @@ The control protocol is documented in [docs/control-protocol.md](docs/control-pr
|
||||
| [docs/control-protocol.md](docs/control-protocol.md) | Control-channel and MCP reference |
|
||||
| [docs/project-files.md](docs/project-files.md) | `.fcproj` format, schema migrations, auto-save and crash recovery |
|
||||
| [docs/graphics.md](docs/graphics.md) · [docs/captions.md](docs/captions.md) | Text engine, graphic clips and tools; caption tracks and formats |
|
||||
| [ROADMAP.md](ROADMAP.md) | Milestones and estimates |
|
||||
| [ROADMAP.md](ROADMAP.md) | Honest assessment, what's missing, milestones and estimates |
|
||||
|
||||
## Status
|
||||
|
||||
FilmCraft is young and moving fast: roughly 45% of the way to Premiere Pro parity, with editing, trimming, colour, keyframes, titles, captions, mixing, codecs and export working today. [ROADMAP.md](ROADMAP.md) tracks every milestone with estimates.
|
||||
FilmCraft is young and moving fast. Editing, trimming, multicam, colour, keyframes, effects, titles, captions, mixing, codecs and export work today.
|
||||
|
||||
We track two numbers ([ROADMAP.md](ROADMAP.md#honest-assessment-2026-10-05)):
|
||||
|
||||
- **Feature checklist: ~87%.** Premiere Pro's menu items, effects, transitions, panels and formats that exist in FilmCraft.
|
||||
- **Ready for real work: ~50–60%.** Our honest estimate of how close FilmCraft is to replacing Premiere on real projects.
|
||||
|
||||
The biggest gaps today:
|
||||
|
||||
- **Speed on big footage.** Hardware decoding works on macOS; Windows and Linux have no hardware path yet. Blend modes and the most common effects run on the GPU, but Lumetri, keys, export and encoding still run on the CPU ([#30](https://github.com/storytold/filmcraft/issues/30)).
|
||||
- **No plugins.** No VST3 / Audio Units or OpenFX hosting.
|
||||
- **Delivery codecs.** H.264 is our only delivery-codec export; no HEVC or AV1 export yet.
|
||||
- **Real-world media and platforms.** Our decoders are bit-exact on conformance streams, but camera and phone files in the wild are less tested. Windows and Linux get far less testing than macOS.
|
||||
- **AI features.** Few so far; speech to text is optional and off by default.
|
||||
|
||||
Bug reports with real footage are the most useful thing you can send us: [open an issue](https://github.com/storytold/filmcraft/issues) or tell us in [Discord](https://discord.gg/artcraft).
|
||||
|
||||
## Architecture
|
||||
|
||||
@@ -330,18 +345,25 @@ And [**ArtCraft**](https://getartcraft.com/) itself, our AI image and video stud
|
||||
|
||||
<br>
|
||||
|
||||
## Credits and clean room
|
||||
## License and credits
|
||||
|
||||
FilmCraft is dual-licensed under [MIT](LICENSE-MIT) or [Apache-2.0](LICENSE-APACHE), at your option.
|
||||
Copyright (c) 2026 ArtCraft Team and the FilmCraft contributors. Required notices are in [NOTICE](NOTICE).
|
||||
|
||||
Bundled fonts, icons, images and other assets keep their own open licenses; each one is listed
|
||||
with its author, source and license in [ATTRIBUTION.md](ATTRIBUTION.md).
|
||||
|
||||
**Footage and music in the screenshots:** NASA's Apollo 11 film and television footage from [images.nasa.gov](https://images.nasa.gov) (launch, Launch Control Center, lunar surface and recovery) and the Apollo 11 air-to-ground voice transcript, all US Government works in the public domain (NASA does not endorse this project); *Night of the Living Dead* (1968), *Carnival of Souls* (1962) and *Charade* (1963), all in the US public domain; *Earth Views from the ISS* by NASA; Chopin's Nocturne Op. 48 No. 1 and Ballade No. 1, performed for Musopen and released under CC0. The media itself is not in this repository. Sources and details for every asset are in [ATTRIBUTION.md](ATTRIBUTION.md).
|
||||
|
||||
FilmCraft is an independent implementation. It contains no Adobe code, icons, images, presets or LUTs, and no GPL or LGPL code; every icon is drawn in code and every asset is openly licensed and attributed ([AGENTS.md](AGENTS.md)), apart from the ArtCraft name and logo, which are trademarks of the ArtCraft team used with permission. ffmpeg is used only as an external test oracle.
|
||||
FilmCraft is an independent implementation. It contains no Adobe code, icons, images, presets or LUTs, and no GPL or LGPL code; every icon is drawn in code and every asset is openly licensed and attributed ([AGENTS.md](AGENTS.md)), apart from the ArtCraft name and logos, which are trademarks of the ArtCraft Team used under [`docs/brand/LICENSE-brand.txt`](docs/brand/LICENSE-brand.txt). ffmpeg is used only as an external test oracle.
|
||||
|
||||
The ArtCraft name, wordmark and logos in [`docs/brand/`](docs/brand/) are trademarks of the
|
||||
ArtCraft Team and are not covered by this license. They may be used only unmodified, and only as
|
||||
part of this repository and FilmCraft, under [`docs/brand/LICENSE-brand.txt`](docs/brand/LICENSE-brand.txt).
|
||||
Forks and modified versions must remove them.
|
||||
|
||||
<sub>Adobe, Photoshop, Illustrator, Premiere Pro, Lightroom, Acrobat, After Effects and InDesign are trademarks or registered trademarks of Adobe Inc. in the United States and/or other countries. FilmCraft is an independent, open-source project and is not affiliated with, sponsored by or endorsed by Adobe Inc.; these names are used only to describe the workflows it is compatible with.</sub>
|
||||
|
||||
## License
|
||||
|
||||
FilmCraft is dual-licensed under [MIT](LICENSE-MIT) or [Apache-2.0](LICENSE-APACHE), at your option. Bundled assets keep their own open licences, listed in [ATTRIBUTION.md](ATTRIBUTION.md). The ArtCraft name and logos in `docs/brand/` are trademarks of the ArtCraft team and are not covered by this licence.
|
||||
|
||||
<br>
|
||||
|
||||
<p align="center">
|
||||
|
||||
+91
-6
@@ -1,10 +1,88 @@
|
||||
# FilmCraft Roadmap
|
||||
|
||||
Progress toward feature parity with Adobe Premiere Pro, with estimates. Updated as milestones land.
|
||||
Progress toward parity with Adobe Premiere Pro, with estimates. Updated as milestones land.
|
||||
|
||||
**Last updated:** 2026-10-02 · **Overall parity:** ~87% (measured scorecard below) · **Code:** 34 crates, 1605 tests
|
||||
**Last updated:** 2026-10-05 · **Feature checklist:** ~87% · **Ready for real work:** ~50–60% · **Code:** 35 crates, 1605+ tests
|
||||
|
||||
## Parity scorecard
|
||||
Two numbers, because they answer different questions. The **feature checklist** counts whether
|
||||
Premiere's menu items, effects, panels and formats exist in FilmCraft. **Ready for real work** is
|
||||
our honest estimate of how close FilmCraft is to replacing Premiere for an editor on real
|
||||
projects: depth, correctness, robustness, speed and the pro ecosystem. The gap between the two
|
||||
is the work that matters most now. Agents: read [Where we are lacking](#where-we-are-lacking)
|
||||
before choosing work.
|
||||
|
||||
## Honest assessment (2026-10-05)
|
||||
|
||||
### How the checklist number is made, and what it doesn't tell you
|
||||
|
||||
- **Partly measured.** The menu figure comes from diffing a dump of Premiere 26.5's menus against
|
||||
`filmcraft-cli commands`. The effects, transitions and audio-effects counts are checked by unit
|
||||
tests against Premiere's Effects panel tree (`premiere_26_video_effects_catalogue`,
|
||||
`premiere_audio_set_is_complete_and_foldered`, `premiere_26_transition_tree`).
|
||||
- **Partly agent-reported.** The other areas' percentages are written by the agents doing the work.
|
||||
No automated parity check runs in CI (PhotoCraft has `cargo xtask parity`; we don't yet), so
|
||||
nothing stops a number drifting upward.
|
||||
- **Presence, not quality.** An item counts once it exists. An effect described as an
|
||||
"approximation" counts the same as an exact one. Nothing compares our rendering with
|
||||
Premiere's.
|
||||
- **Weights understate what users hit first.** Performance carries 5%. Plugins and AI features
|
||||
aren't in the scorecard at all.
|
||||
|
||||
### By dimension
|
||||
|
||||
| Dimension | Checklist says | Honest estimate | Evidence |
|
||||
|---|---|---|---|
|
||||
| **Feature breadth** (menus, commands, panels) | ~86–95% | **~85%** | Menu diff against Premiere 26.5; effect, transition and audio-effect catalogues complete by test. The long tail is mostly there. |
|
||||
| **Correctness in depth** | not tracked | **~60–70%** | A first-time contributor found four real bugs in core paths within hours (#5–#8: AIFF import failed, Slide left linked audio behind, transitions didn't follow ripple trims, export ignored start/end times). User reports: Color Matte always grey with no picker (#29); ALSA audio error on Linux (#23). Settings are "most wired", not all. |
|
||||
| **Codecs and media** | ~90% | **~75%** | Our decoders are bit-exact on conformance streams. Real camera and phone media (variable frame rate, damaged files, unusual containers) is far less tested. No camera RAW (RED, BRAW, ARRIRAW), no E-AC-3. |
|
||||
| **Export** | ~92% | **~70%** | Delivery: H.264 only (ProRes, DNxHR and image sequences cover mastering). No HEVC or AV1 export. AAF and OMF have never been validated in Avid Media Composer or Pro Tools. Export renders and encodes on the CPU only. |
|
||||
| **Performance and hardware** | ~62% | **~35–40%** | Premiere runs effects, decoding and encoding on the GPU and the hardware media engines. Here, hardware decode works on macOS only (VideoToolbox, #33). The GPU does compositing, blend modes (#32) and 31 common effects (GPU2): colour adjustments, crop/flip/transform, blurs, sharpen. Lumetri, keys, masks, the remaining effects, export and encoding are CPU. Linux and Windows have no hardware path. 8K and AV1 aren't real-time. The effect list shows Premiere's "GPU accelerated" badge, but only 31 effect ids run on the GPU here; the rest, Lumetri included, run on the CPU. |
|
||||
| **Stability** | not tracked | **improving, unproven** | The never-crash pass (no panics in product code, last-resort guards) landed on 2026-10-04. There's no field record from real users' projects yet. |
|
||||
| **Plugins and pro ecosystem** | not tracked | **~0–10%** | No audio plugin hosting (VST3 / Audio Units) and no third-party video effects (OpenFX). For many professional editors this alone rules FilmCraft out. Team Projects and Productions are out of scope by design. |
|
||||
| **AI features** | partial | **~25–35%** | Speech to text exists but is off in default builds (the `whisper` feature). Enhance Speech is a DSP chain, not a model. Auto Reframe is approximate. No Generative Extend, and no media-intelligence search or auto colour. |
|
||||
| **Platforms** | not tracked | **macOS solid; Windows and Linux thin** | Development and testing happen on macOS. Windows and Linux are built for releases but barely exercised at runtime. The web build is a compile target; the browser app shell isn't finished. |
|
||||
| **UI fidelity** | ~85% | **~80%** | Agents compare with Premiere reference screenshots; this hasn't been checked independently. |
|
||||
|
||||
### Where we are lacking
|
||||
|
||||
In priority order. Agents should prefer this work over adding more checklist items.
|
||||
|
||||
1. **Hardware acceleration** (#30), the most visible gap to users:
|
||||
- hardware decode on Linux (VA-API) and Windows (Media Foundation);
|
||||
- zero-copy decoded frames into wgpu;
|
||||
- the remaining effects on the GPU: Lumetri, keys, Vignette, Video Limiter, masks (31 common effects are done);
|
||||
- GPU export, then hardware encode (H.264 / HEVC).
|
||||
2. **A measured parity number.** Add `cargo xtask parity`, run in CI. It should cover menus,
|
||||
commands, effects, transitions, panels, preferences and formats against the Premiere snapshot in
|
||||
`plan/premiere/`. It should report presence and fidelity separately (exact, approximate, stub).
|
||||
3. **Real-world media corpus.** Phone, camera and screen recordings, variable frame rate, damaged
|
||||
files, long GOPs, odd containers. Each gets an import → edit → export round-trip test with an
|
||||
ffprobe / ffmpeg oracle.
|
||||
4. **Workflow acceptance tests.** Scripted MCP sessions that do real jobs end to end: assemble and
|
||||
trim a scene, colour it, mix and loudness-normalise it, caption it, deliver it, round-trip
|
||||
through XML / AAF.
|
||||
5. **Windows and Linux at runtime.** CI that runs the test suite and a headless playback/export
|
||||
smoke test on both, not just release builds. Fix the Linux audio path (#23).
|
||||
6. **Delivery codecs.** HEVC and AV1 export: hardware encoders first, and `rav1e` (BSD-2) for AV1.
|
||||
7. **Plugin hosting** (VST3 / Audio Units, then OpenFX). Needs FFI, so it falls under the isolated
|
||||
`unsafe` crate rule (AGENTS.md §0.3). **Owner decision needed** before starting.
|
||||
8. **AI features** with openly licensed local models: transcription on by default, text-based
|
||||
editing, speech enhancement, reframing.
|
||||
9. **The web app shell**: file access, WebCodecs, audio.
|
||||
|
||||
### Where we're going
|
||||
|
||||
**Next:** close items 1–5. Hardware acceleration fixes the gap users notice first (CPU-bound 4K playback and export). The
|
||||
parity tool, media corpus and workflow tests make the progress number trustworthy. Windows and
|
||||
Linux CI makes "runs everywhere" true.
|
||||
|
||||
**After that:** delivery codecs, plugin hosting (pending the owner's decision) and AI features,
|
||||
measured with the same tools.
|
||||
|
||||
**Goal:** "ready for real work" at 85%+, measured rather than estimated, on all three desktop
|
||||
platforms.
|
||||
|
||||
## Feature checklist scorecard
|
||||
|
||||
Measured against Premiere Pro 26.5 on this machine. Menu items: the native menu bar dump, minus
|
||||
Adobe-cloud-only items (Team Projects, Productions, Firefly, Stock, Dynamic Link, account/help pages),
|
||||
@@ -22,11 +100,14 @@ matched against `filmcraft-cli commands` and the UI command table. Effects: the
|
||||
| Graphics and captions | 8% | text engine, shapes, per-character styles, our own graphics templates (.fcgt, 8 built-ins, export/install/edit), rolls/crawls, responsive pins + time, captions SRT/VTT/SCC/MCC/STL/TTML/DFXP, transcripts | ~85% |
|
||||
| Export | 8% | own H.264/AAC, ProRes, DNxHR, MXF OP1a/OP-Atom, image sequences, GIF, WAV, 5.1; preset library + user presets, full Export-mode settings, queue, Quick Export; AAF (Edit Protocol) import/export, OMF 2.0 export; no AAF/OMF validation against Avid/Pro Tools yet | ~92% |
|
||||
| Preferences and project management | 5% | Settings dialog with 16 categories (most settings wired), project settings, scratch disks, search bins, templates | ~80% |
|
||||
| Performance | 5% | 1080p real-time, 3×1080p; 4K H.264 real-time at load ≤140 (−23% decoder cycles, opt-in draft decoding at 1/2–1/4); HEVC 2.5× cheaper, catch-up decoding, `cargo xtask bench` + `perf.stats`; 8K and AV1 not yet real-time on a loaded machine | ~62% |
|
||||
| Performance | 5% | 1080p real-time, 3×1080p; 4K H.264 real-time at load ≤140 (−23% decoder cycles, opt-in draft decoding at 1/2–1/4); HEVC 2.5× cheaper, catch-up decoding, `cargo xtask bench` + `perf.stats`; blend modes on the GPU (#32); 8K and AV1 not yet real-time on a loaded machine; no hardware path on Windows/Linux (see the honest assessment: ~35–40% against Premiere) | ~62% |
|
||||
| **Weighted total** | | | **~87%** |
|
||||
|
||||
## Estimate to parity
|
||||
|
||||
This estimate is for the **feature checklist**. Closing the honest-assessment gaps (above) is the
|
||||
"Robust on real-world material" row, plus plugins and AI features, which it doesn't include.
|
||||
|
||||
Measured throughput in the last work block (2026-10-01 night → 10-02): five Opus 5.5 agents in
|
||||
parallel for ~4.5 wall-clock hours (~18 agent-hours including integration) moved parity by ~4 points
|
||||
(≈4–5 agent-hours per point), on a machine that was heavily overloaded (load 150–300 on 14 cores) and
|
||||
@@ -62,7 +143,7 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining
|
||||
| M2 | H.264 decoder | ✅ | Own decoder, bit-exact on 37+ streams, 500–600 fps 1080p | — | — |
|
||||
| M3 | Editing core | 🟡 | Edit algebra (insert/overwrite/razor/lift/extract/ripple/roll/slip/slide/rate-stretch/nest/paste), tools, markers, trim mode + Trim Monitor + dynamic J/K/L trimming, Keyboard Shortcuts editor with FilmCraft/Premiere/FCP/Avid presets; Edit/Clip/File menu commands (Label colours and Select Label Group, Paste/Remove Attributes, Select All Matching, Remove Unused, Consolidate Duplicates, Sequence From Clip, Bin From Selection, Offline File, Close Project, Make/Edit Subclip, Modify Audio Channels/Timecode, Frame Hold Options/Add Frame Hold/Insert Frame Hold Segment, Time Interpolation with frame blending, Fit/Fill frame, Breakout to Mono, Extract Audio, Replace With Clip; M3.11: Scene Edit Detection (pure-Rust cut detection, background job), Normalize Mix Track, Simplify Sequence, Transcribe Sequence, Find/Find Next and search bins, Automate to Sequence, Edit Original, Edit Offline, Source Settings, Update Metadata (XMP), Generate Audio Waveform, Project Settings General/Scratch Disks, Get Media File Properties, Save as Template, Selection as FilmCraft Project, Avid Log Exchange export, Flash Cue markers, Dynamic Audio Waveforms, Reveal Log Files, System Compatibility Report); M3.12 keyboard parity: Premiere's default keyboard (~115 keyboard-only commands: edit-point navigation on targeted/any track, select clip at playhead/next/previous, extend edit to playhead, nudge/slip/slide selection, target and source-patch toggles, clip volume ±1 dB/many, frame maximize/full screen/panel cycling, monitor zoom, track heights, Project and Text panel keyboard navigation, text size/leading/alignment, Export Frame, poster frames; see docs/keyboard.md); multicam (Create Multi-Camera Source Sequence, Multi-Camera view with live switching on 1–9, angle switching, Enable/Flatten, Edit Cameras) and sync (Synchronize, Merge Clips; In/Out/timecode/marker/audio — GCC-PHAT, sample-accurate) | Multicam paging >16 angles and grid thumbnails, optical flow (renders as frame blending), 39 Premiere default shortcuts skipped with reasons in docs/keyboard.md (Productions, AI/cloud tools, work area bar, Production/Search panels) | 6–10 |
|
||||
| M4 | Playback | 🟡 | Audio-clock master, prefetch with cancellation, J/K/L, correct dropped-frame stats, playback resolution, render bar + content-hashed render previews, App Nap opt-out; 1080p H.264 and 3 stacked 1080p streams play with 0 dropped frames | 4K under load, 8K, frame-threaded AV1 decode, reduced-resolution decode for multicam grids | 6–10 |
|
||||
| M5 | Effects, keyframes, GPU | 🟡 | ~60 CPU effects, 30 transitions, keyframes + value/velocity graphs, wgpu compositor, effect + opacity masks (ellipse/polygon/pen, feather, expansion, modes; CPU/WGSL parity), mask tracking (Lucas–Kanade + RANSAC), adjustment layers, effect presets (built-in + user, JSON import/export) | Full ~150-effect catalogue, WGSL parity for all effects and masks in the live GPU path, Warp Stabilizer, Morph Cut | 20–30 |
|
||||
| M5 | Effects, keyframes, GPU | 🟡 | All 93 Premiere 26 video effects and 84 transitions (CPU), blend modes and 31 common effects on the GPU (#32, GPU2), keyframes + value/velocity graphs, wgpu compositor, effect + opacity masks (ellipse/polygon/pen, feather, expansion, modes; CPU/WGSL parity), mask tracking (Lucas–Kanade + RANSAC), adjustment layers, effect presets (built-in + user, JSON import/export) | Effects on the GPU (in progress, #30), GPU masks in the live path, GPU export, exact Warp Stabilizer (3-D) and Morph Cut | 20–30 |
|
||||
| M6 | Export | ✅ | Own H.264 encoder (High/Main/Baseline, B-frames, VBR/CBR/2-pass) → MP4 or QuickTime + own AAC; ProRes, DNxHR, MJPEG, PNG/TIFF/BMP sequences (Premiere-style numbering), GIF, WAV, AIFF; background jobs; M6.5: Export mode parity (frame size / rate / scaling / pixel aspect, profile / level, CBR / VBR 1- and 2-pass, keyframe distance, audio codec / rate / channels / bitrate / sample size, multiplexer, burn-in or sidecar captions, image / name / timecode overlays, video limiter, loudness normalization with true-peak limiter, metadata, ranges, estimated size, summary), 24 built-in presets + user presets with favourites and import/export (Preset Manager), export queue (reorder, cancel, retry, several sequences / ranges), Quick Export; `export.*` commands and `filmcraft-cli export --preset` | Interlaced encoding, 5.1 audio, smart render, publishing destinations | 6–8 |
|
||||
| M7 | Audio | 🟡 | Mixer graph (tracks → submixes → Mix, pre/post-fader inserts and sends, latency-compensated, sample-accurate, ~6× realtime for 24 tracks × 3 effects on one core), Audio Track Mixer + Audio Clip Mixer panels, track automation (Off/Read/Latch/Touch/Write, recorded live while playing, thinned to keyframes, timeline lanes with pen editing), solo/solo-safe, channel mapping basics, peak + BS.1770 loudness meters (match ffmpeg), DSP crate with 16 clip/track effects, Audio Gain (set/adjust/normalize), Constant Power / Constant Gain / Exponential Fade, Essential Sound (types, Loudness auto-match, Repair incl. DeEss/DeReverb, Clarity, Creative, Ducking keyframes, presets; full Dialogue chain 22× realtime) | Music duration remix, ML speech enhancement, 5.1 panner and multichannel buses, voice-over record, effect editor windows (EQ curve), remaining effects (multiband, convolution reverb), clip-mixer automation recording | 6–9 |
|
||||
| M8 | Colour | 🟡 | Lumetri: basic, creative + looks, RGB & hue curves, wheels, HSL secondary, vignette, section bypass; Input LUT / Look LUT (.cube 1D/3D/shaper, .3dl; tetrahedral CPU + WGSL; project LUT library; built-in camera conversions); Colour Match (Oklab tonal-range statistics, skin protection, solved in Lumetri wheels); colour management: Rec. 709 / Rec. 2100 PQ / HLG working spaces + wide gamut, Interpret Footage colour space (S-Log3, V-Log, Canon Log 2/3, LogC3/4, Apple Log, D-Log from published specs), metadata auto-detect, BT.2390 tone mapping, gamut mapping, HDR export signalling (VUI/colr/mdcv/clli/SEI, ffprobe-verified); Lumetri Scopes: Vectorscope YUV (75/100 % targets, skin-tone line) and HLS, Histogram, Parade (RGB/YUV/RGB-White), Waveform (RGB/Luma/YC/YC no Chroma), presets, Rec. 601/709/2100, 8-bit/float/HDR (cd/m²) scales, multi-scope grid, `scopes.read` | HDR-aware Lumetri maths, macOS EDR monitors, mastering metadata → tone-map peak, D-Log M (no published formula), HSL Secondary refine | 3–5 |
|
||||
@@ -73,10 +154,13 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining
|
||||
|
||||
## Running now
|
||||
|
||||
- Nothing; next: remaining Premiere menu items (Settings categories, Scene Edit Detection, Normalize Mix Track, OMF/AAF, Find), AV1 single-thread re-measure on an idle machine
|
||||
- **Hardware acceleration (#30):** landed: blend modes on the GPU (#32); VideoToolbox hardware decode + the `platform` FFI crate (#33); 31 common effects on the GPU (GPU2). Next: VA-API (Linux) and Media Foundation (Windows) decode, zero-copy upload, Lumetri on the GPU.
|
||||
- **Next:** [Where we are lacking](#where-we-are-lacking) items 1–5. GPU export waits for the export-crate work (#19 and the never-crash follow-up) to land.
|
||||
|
||||
## Log
|
||||
|
||||
- **2026-10-05:** honest assessment added (checklist ~87% vs ready for real work ~50–60%). Hardware acceleration started (#30): blend modes on the GPU (#32); VideoToolbox decode + `platform` crate, the one crate allowed `unsafe` (#33). Community fixes landed: AIFF import, export start/end ranges, transitions following ripple trims, Slide moving linked audio (#5–#8).
|
||||
|
||||
- **2026-10-04 (later):** interchange + MXF export (M11.4/M6.6): own compound-file container (`crates/cfb`), AAF Edit Protocol import/export (embedded/linked/consolidated audio, video mixdown, handles, breakout to mono), OMF 2.0 export (Bento), MXF OP1a/OP-Atom writer (DNxHR/ProRes/H.264/PCM) as export formats. 1605 tests.
|
||||
|
||||
- **2026-10-04:** MPEG-2 (M9.15): own MPEG-2/MPEG-1 video decoder (4:2:2, interlaced, field pictures; IEEE 1180 IDCT), MPEG TS/PS demux (AVCHD .mts, .m2ts, .ts, .mpg/.vob/.mod), own AC-3 decoder (ATSC A/52), MP2, LATM AAC, Blu-ray/DVD LPCM, MPEG-2 in MXF/MOV/MP4/MKV. 1562 tests.
|
||||
@@ -85,6 +169,7 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining
|
||||
|
||||
- **2026-10-03 (later):** graphics & captions (M10.7): `.fcgt` graphics templates (export/install/apply/edit, 8 original built-ins; Adobe .mogrt refused), Essential Graphics Browse/Edit, rolls/crawls, responsive design pins and time, per-character styles, Upgrade Caption to Graphic / to Source Graphic, MCC/EBU STL/TTML/DFXP, Replace Fonts; schema v12. 1479 tests.
|
||||
- **2026-10-03:** 4K H.264 playback (M4.9): vectorised deblocking, branch-free CABAC bins and column-parallel inverse transforms cut H.264 decoding by 23 % (cycles per frame, bit-exact; 4K cold seeks −27 % CPU); opt-in draft decoding (Settings ▸ Playback) skips deblocking of non-reference pictures and uploads decimated planes while playing at 1/2–1/4, never for paused frames or exports; conformance runs 1, 3 and all threads. 4K plays 192/0 at load 100–140 (before: only at load ~90).
|
||||
- **2026-10-03:** 4K VP9 / HEVC / AV1 playback (M4.10): VP9 frame threading (references published per superblock row, loop filter overlapping the next frames) plus a vectorised loop filter, column-parallel transforms and 16-bit MC cut VP9 work by 25 % single-threaded and 44 % with all cores (4K decode 8 → 27–34 fps at load ~200); HEVC −28 % (vectorised SAO, MC without per-block zeroing, batched bypass bins), AV1 −20 % (fixed-width MC, recycled planes); opt-in draft decoding for all three; bit-exact at 1, 3 and all threads. 4K VP9 and HEVC now play 192/0 at load ~200 (before: 0–58 of 192 shown).
|
||||
|
||||
- **2026-10-03:** keyboard parity (M3.12: 143 new commands on Premiere's default keys, table validated against the registry), audio (M7.8: 5.1 tracks/submixes/Mix, 5.1 panner, BS.775 downmix, 5.1 WAV/MOV/AAC export, voice-over record with punch-in, Remix, Clip Mixer Latch/Touch/Write). 1432 tests.
|
||||
|
||||
|
||||
@@ -21,6 +21,7 @@ whisper = ["filmcraft-engine/whisper", "filmcraft-engine/speech-download"]
|
||||
filmcraft-engine = { workspace = true }
|
||||
filmcraft-media = { workspace = true }
|
||||
filmcraft-codecs = { workspace = true }
|
||||
filmcraft-platform = { workspace = true }
|
||||
filmcraft-mxf = { workspace = true }
|
||||
filmcraft-mpegts = { workspace = true }
|
||||
filmcraft-ogg = { workspace = true }
|
||||
|
||||
@@ -88,7 +88,9 @@ fn insert_dotted(m: &mut Map<String, Value>, key: &str, v: Value) {
|
||||
if !e.is_object() {
|
||||
*e = Value::Object(Map::new());
|
||||
}
|
||||
insert_dotted(e.as_object_mut().expect("object"), tail, v);
|
||||
if let Value::Object(child) = e {
|
||||
insert_dotted(child, tail, v);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
//!
|
||||
//! Run `filmcraft-cli help` for the full reference (also in docs/agents.md).
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
mod args;
|
||||
mod probe;
|
||||
|
||||
@@ -153,6 +155,8 @@ async fn main() {
|
||||
println!("filmcraft-cli {}", env!("CARGO_PKG_VERSION"));
|
||||
return;
|
||||
}
|
||||
// OS hardware video decoders (VideoToolbox on macOS) in front of our own, as in the desktop app.
|
||||
let _ = filmcraft_platform::register();
|
||||
let a = Args::parse(std::env::args().skip(1));
|
||||
let Some(cmd) = a.pos(0) else { usage("missing subcommand") };
|
||||
match cmd {
|
||||
@@ -339,7 +343,7 @@ async fn main() {
|
||||
if a.opt("--bridge").is_some() {
|
||||
usage("render is headless; with --bridge use the MCP `render_frame` tool or `exec` ui commands");
|
||||
}
|
||||
let Backend::Local(mut s) = Backend::open(&a) else { unreachable!() };
|
||||
let Backend::Local(mut s) = Backend::open(&a) else { usage("render is headless") };
|
||||
let secs: f64 = a.opt("--seconds").and_then(|v| v.parse().ok()).unwrap_or(0.0);
|
||||
let scale: f32 = a.opt("--scale").and_then(|v| v.parse().ok()).unwrap_or(1.0);
|
||||
let out = a.opt("--out").unwrap_or("frame.png");
|
||||
@@ -356,7 +360,7 @@ async fn main() {
|
||||
Some(addr) => filmcraft_automation::FilmcraftMcp::bridge(addr).unwrap_or_else(|e| fail(e)),
|
||||
None => match Backend::open(&a) {
|
||||
Backend::Local(s) => filmcraft_automation::FilmcraftMcp::headless(*s),
|
||||
Backend::Bridge(_) => unreachable!(),
|
||||
Backend::Bridge(_) => usage("mcp: use --bridge ADDR to drive a running app"),
|
||||
},
|
||||
};
|
||||
if let Err(e) = server.serve_stdio().await {
|
||||
|
||||
@@ -289,20 +289,25 @@ impl ByteReader for BlobReader {
|
||||
}
|
||||
return Err(filmcraft_media::pending::would_block());
|
||||
}
|
||||
FS.with(|fs| {
|
||||
let complete = FS.with(|fs| {
|
||||
let mut fs = fs.borrow_mut();
|
||||
fs.clock += 1;
|
||||
let t = fs.clock;
|
||||
let mut pos = 0usize;
|
||||
for c in c0..=c1 {
|
||||
let (d, used) = fs.chunks.get_mut(&(self.id, c)).expect("checked above");
|
||||
// Present (checked above); evicted in between would be a bug, reported as an error.
|
||||
let Some((d, used)) = fs.chunks.get_mut(&(self.id, c)) else { return false };
|
||||
*used = t;
|
||||
let a = if c == c0 { (offset - c * CHUNK) as usize } else { 0 };
|
||||
let n = (d.len() - a).min(buf.len() - pos);
|
||||
buf[pos..pos + n].copy_from_slice(&d[a..a + n]);
|
||||
pos += n;
|
||||
}
|
||||
true
|
||||
});
|
||||
if !complete {
|
||||
return Err(filmcraft_media::pending::would_block());
|
||||
}
|
||||
// read-ahead for sequential access
|
||||
for c in c1 + 1..=(c1 + READ_AHEAD).min(last) {
|
||||
request(self.id, c);
|
||||
|
||||
@@ -16,6 +16,7 @@
|
||||
//!
|
||||
//! Everything here is `wasm32`-only; on other targets the crate is empty.
|
||||
#![cfg(target_arch = "wasm32")]
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
pub mod api;
|
||||
pub mod audio;
|
||||
@@ -135,7 +136,14 @@ pub async fn start(canvas_id: String) -> Result<(), JsValue> {
|
||||
import::install_drop_handlers()?;
|
||||
|
||||
let demo = !query_flag("empty");
|
||||
let web_options = eframe::WebOptions::default();
|
||||
let mut web_options = eframe::WebOptions::default();
|
||||
// `?webgl`: WebGL2 only. The page reloads with it when starting on WebGPU fails (an adapter
|
||||
// whose device can't be created, a driver the browser rejects…); see `index.html`.
|
||||
if query_flag("webgl")
|
||||
&& let eframe::egui_wgpu::WgpuSetup::CreateNew(c) = &mut web_options.wgpu_options.wgpu_setup
|
||||
{
|
||||
c.instance_descriptor.backends = eframe::wgpu::Backends::GL;
|
||||
}
|
||||
let runner = eframe::WebRunner::new();
|
||||
runner
|
||||
.start(
|
||||
|
||||
@@ -193,14 +193,16 @@ pub fn reader_opener(name: &str, head: &[u8], reader: &SharedReader) -> Option<R
|
||||
Err(_) => return Some(Ok(inner)),
|
||||
};
|
||||
let Some(track) = file.tracks.iter().position(|t| t.kind == TrackKind::Video && !t.samples.is_empty()) else { return Some(Ok(inner)) };
|
||||
let entry = &file.tracks[track].entries[0];
|
||||
let Some(entry) = file.tracks[track].entries.first() else { return Some(Ok(inner)) };
|
||||
let fourcc = entry.format.to_string();
|
||||
let Some((family, codec, description)) = codec_string(&entry.codec, &fourcc) else { return Some(Ok(inner)) };
|
||||
if !SUPPORTED.with(|s| s.borrow().contains(&family)) {
|
||||
return Some(Ok(inner));
|
||||
}
|
||||
let info = inner.info().clone();
|
||||
let (w, h) = info.video.as_ref().map(|v| (v.width, v.height)).unwrap_or((0, 0));
|
||||
// the decoder wants the coded size; `info` reports the display size after the track's rotation
|
||||
let rotation = file.tracks[track].display_rotation().unwrap_or(0);
|
||||
let (w, h) = info.video.as_ref().map(|v| if rotation % 2 == 1 { (v.height, v.width) } else { (v.width, v.height) }).unwrap_or((0, 0));
|
||||
let id = NEXT_ID.with(|n| {
|
||||
let id = n.get();
|
||||
n.set(id + 1);
|
||||
@@ -215,7 +217,7 @@ pub fn reader_opener(name: &str, head: &[u8], reader: &SharedReader) -> Option<R
|
||||
file,
|
||||
track,
|
||||
id,
|
||||
config: Config { codec, description, width: w, height: h },
|
||||
config: Config { codec, description, width: w, height: h, rotation },
|
||||
state: Mutex::new(Cache::default()),
|
||||
})))
|
||||
}
|
||||
@@ -225,6 +227,8 @@ struct Config {
|
||||
description: Option<Vec<u8>>,
|
||||
width: u32,
|
||||
height: u32,
|
||||
/// Clockwise quarter turns from the track matrix, applied to decoded frames.
|
||||
rotation: u8,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
@@ -455,10 +459,11 @@ impl WcSource {
|
||||
std::mem::take(&mut *s.out.borrow_mut())
|
||||
});
|
||||
for (pts, f) in out {
|
||||
let f = if self.config.rotation != 0 { f.rotated(self.config.rotation) } else { f };
|
||||
c.frames.insert(pts, Arc::new(f));
|
||||
}
|
||||
while c.frames.len() > CACHE_FRAMES {
|
||||
let far = c.frames.keys().copied().max_by_key(|p| (p - want).abs()).expect("non-empty");
|
||||
let Some(far) = c.frames.keys().copied().max_by_key(|p| (p - want).abs()) else { break };
|
||||
c.frames.remove(&far);
|
||||
}
|
||||
// the session's own frame came out: other requests may restart it now
|
||||
|
||||
@@ -6,8 +6,9 @@
|
||||
//
|
||||
// Steps: load (timing), demo project, play 2 s (frames shown/dropped), import the media through
|
||||
// `filmcraft.importUrl`, put it on a new sequence, export H.264 (download), screenshots of each
|
||||
// step. `--media` must be reachable from the page (copy it next to index.html). Prints a JSON
|
||||
// report; exits non-zero on failure.
|
||||
// step; then every mode and a tiny window. `--media` must be reachable from the page (copy it next
|
||||
// to index.html). Prints a JSON report; exits non-zero on failure, including any Rust panic or
|
||||
// uncaught exception in the console.
|
||||
import { spawn } from "node:child_process";
|
||||
import { mkdirSync, writeFileSync, readdirSync, statSync, mkdtempSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
@@ -148,6 +149,24 @@ try {
|
||||
await sleep(1000);
|
||||
report.steps.export.downloads = readdirSync(downloads).filter((f) => !f.endsWith(".crdownload")).map((f) => ({ f, bytes: statSync(join(downloads, f)).size }));
|
||||
report.steps.export.screenshot = await shot("04-exported");
|
||||
|
||||
// every top-level mode and a tiny window: these used to panic (temp_dir in Export mode, a dock
|
||||
// clamp below 40 points) and freeze the canvas
|
||||
for (const mode of ["import", "export", "edit"]) {
|
||||
await js(`filmcraft.request('ui.set', {mode: ${JSON.stringify(mode)}})`);
|
||||
await sleep(500);
|
||||
}
|
||||
await send("Emulation.setDeviceMetricsOverride", { width: 120, height: 80, deviceScaleFactor: 1, mobile: false });
|
||||
await sleep(500);
|
||||
await send("Emulation.setDeviceMetricsOverride", { width: 1600, height: 1000, deviceScaleFactor: 2, mobile: false });
|
||||
await sleep(500);
|
||||
await send("Emulation.setDeviceMetricsOverride", { width: 1600, height: 1000, deviceScaleFactor: 1, mobile: false });
|
||||
report.steps.modes = { screenshot: await shot("05-modes") };
|
||||
// still alive? (a panicked app never answers)
|
||||
await Promise.race([js("filmcraft.inspect().then(() => true)"), sleep(5000).then(() => { throw new Error("app stopped answering"); })]);
|
||||
const fatal = logs.filter((l) => /panicked at|RuntimeError|EXCEPTION/.test(l));
|
||||
if (fatal.length) throw new Error("panic / uncaught exception:\n" + fatal.join("\n").slice(0, 4000));
|
||||
if (await js("window.filmcraftLoad.fatal || null")) throw new Error("fatal overlay shown");
|
||||
report.ok = true;
|
||||
} catch (e) {
|
||||
report.ok = false;
|
||||
|
||||
@@ -16,24 +16,78 @@
|
||||
}
|
||||
#loading img { width: 56px; height: 56px; }
|
||||
#loading .err { color: #ff8a7a; white-space: pre-wrap; max-width: 70ch; pointer-events: auto; }
|
||||
#fatal {
|
||||
position: absolute; inset: 0; display: none; align-items: center; justify-content: center;
|
||||
background: rgba(20, 20, 22, 0.88); color: #e6e6e8; font: 14px/1.45 system-ui, sans-serif; z-index: 10;
|
||||
}
|
||||
#fatal.on { display: flex; }
|
||||
#fatal .box { max-width: min(78ch, calc(100vw - 32px)); padding: 20px 24px; background: #26272b; border-radius: 8px; }
|
||||
#fatal h1 { margin: 0 0 8px; font-size: 17px; color: #ff8a7a; }
|
||||
#fatal pre { max-height: 40vh; overflow: auto; white-space: pre-wrap; font-size: 12px; color: #c9c9cc; }
|
||||
#fatal button { font: inherit; padding: 6px 14px; border: 0; border-radius: 6px; background: #3b6fe0; color: #fff; cursor: pointer; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<canvas id="filmcraft_canvas"></canvas>
|
||||
<div id="loading"><img src="favicon.png" alt=""><div id="status">Loading FilmCraft…</div></div>
|
||||
<script type="module">
|
||||
import init, { start } from "./filmcraft_web.js";
|
||||
const t0 = performance.now();
|
||||
<div id="fatal" role="alertdialog" aria-labelledby="fatal-title">
|
||||
<div class="box">
|
||||
<h1 id="fatal-title">FilmCraft stopped working</h1>
|
||||
<div>Unsaved changes are kept in the browser every few seconds and come back when you reload.</div>
|
||||
<pre id="fatal-msg"></pre>
|
||||
<button id="fatal-reload" type="button">Reload</button>
|
||||
</div>
|
||||
</div>
|
||||
<script>
|
||||
// A Rust panic (or an out-of-memory abort) inside a frame stops the app for good: eframe logs
|
||||
// "panicked at …" with console.error, then the wasm traps ("RuntimeError: unreachable") and
|
||||
// the canvas freezes or goes blank. Show what happened instead of a dead page.
|
||||
window.filmcraftLoad = {};
|
||||
let lastPanic = null;
|
||||
const consoleError = console.error.bind(console);
|
||||
console.error = (...a) => {
|
||||
const m = a.map(String).join(" ");
|
||||
if (/panicked at|memory allocation of/.test(m)) lastPanic = m;
|
||||
consoleError(...a);
|
||||
};
|
||||
window.filmcraftFatal = (what) => {
|
||||
if (window.filmcraftLoad.fatal) return;
|
||||
const msg = lastPanic || String(what && (what.stack || what.message) || what);
|
||||
window.filmcraftLoad.fatal = msg;
|
||||
const el = document.getElementById("fatal");
|
||||
document.getElementById("fatal-msg").textContent = msg.split("\n\nStack:")[0];
|
||||
el.classList.add("on");
|
||||
};
|
||||
document.getElementById("fatal-reload").onclick = () => location.reload();
|
||||
const isFatal = (e) => e instanceof WebAssembly.RuntimeError || /panicked at|unreachable|memory access out of bounds/.test(String(e && e.message || e));
|
||||
// only after start-up: a failed start is shown in #loading below
|
||||
addEventListener("error", (ev) => { if (window.filmcraftLoad.readyMs && isFatal(ev.error || ev.message)) window.filmcraftFatal(ev.error || ev.message); });
|
||||
addEventListener("unhandledrejection", (ev) => { if (window.filmcraftLoad.readyMs && isFatal(ev.reason)) window.filmcraftFatal(ev.reason); });
|
||||
</script>
|
||||
<script type="module">
|
||||
// `__FC_BUILD__` is replaced by `cargo xtask web` with a hash of the build: the glue and the
|
||||
// wasm are cached as immutable, so their URLs must change with every build.
|
||||
const build = "__FC_BUILD__";
|
||||
const v = build.startsWith("__") ? "" : "?v=" + build;
|
||||
const t0 = performance.now();
|
||||
const params = new URLSearchParams(location.search);
|
||||
try {
|
||||
await init();
|
||||
const { default: init, start } = await import("./filmcraft_web.js" + v);
|
||||
await init({ module_or_path: new URL("filmcraft_web_bg.wasm" + v, location.href) });
|
||||
window.filmcraftLoad.wasmMs = performance.now() - t0;
|
||||
await start("filmcraft_canvas");
|
||||
window.filmcraftLoad.readyMs = performance.now() - t0;
|
||||
} catch (e) {
|
||||
const s = document.getElementById("status");
|
||||
if (s) { s.className = "err"; s.textContent = "FilmCraft failed to start:\n" + (e && e.stack || e); }
|
||||
const text = String(e && e.stack || e);
|
||||
window.filmcraftLoad.error = String(e);
|
||||
// WebGPU was found but could not be started: try again on WebGL2 (once)
|
||||
if (!params.has("webgl") && !params.has("cpu") && /wgpu|webgpu|requestDevice|requestAdapter|GPUDevice|surface/i.test(text)) {
|
||||
params.set("webgl", "");
|
||||
location.replace(location.pathname + "?" + params.toString().replace(/=(&|$)/g, "$1") + location.hash);
|
||||
} else {
|
||||
const s = document.getElementById("status");
|
||||
if (s) { s.className = "err"; s.textContent = "FilmCraft failed to start:\n" + text; }
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
</script>
|
||||
|
||||
@@ -18,6 +18,7 @@ whisper = ["filmcraft-engine/whisper", "filmcraft-engine/speech-download"]
|
||||
[dependencies]
|
||||
filmcraft-engine = { workspace = true }
|
||||
filmcraft-ui-egui = { workspace = true }
|
||||
filmcraft-platform = { workspace = true }
|
||||
egui = { workspace = true }
|
||||
eframe = { workspace = true, features = ["default_fonts", "wgpu", "accesskit", "wayland", "x11"] }
|
||||
serde_json = { workspace = true }
|
||||
|
||||
@@ -15,6 +15,8 @@
|
||||
//! asking, `--no-recover` starts without asking (the changes stay available via File ▸ Recover
|
||||
//! Unsaved Changes…).
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
mod app_nap;
|
||||
mod audio;
|
||||
mod audio_in;
|
||||
@@ -68,6 +70,12 @@ fn main() -> eframe::Result {
|
||||
_ => files.push(a),
|
||||
}
|
||||
}
|
||||
// Panics anywhere go to <data dir>/Logs/crash-<day>.log with a backtrace; the UI pass and
|
||||
// frame workers catch them and keep running (see filmcraft_ui_egui::crash).
|
||||
filmcraft_ui_egui::crash::install(data_dir.clone().or_else(default_data_dir).map(|d| d.join("Logs")));
|
||||
// OS hardware video decoders (VideoToolbox on macOS) in front of our own; Settings ▸ Playback ▸
|
||||
// Hardware decoding switches them off. Unsupported streams and failures use our decoders.
|
||||
log::info!("hardware decoding: {:?}", filmcraft_platform::register());
|
||||
let options = eframe::NativeOptions {
|
||||
viewport: egui::ViewportBuilder::default()
|
||||
.with_title("FilmCraft")
|
||||
@@ -85,7 +93,7 @@ fn main() -> eframe::Result {
|
||||
event_loop_builder: agent_event_loop(control_port.is_some()),
|
||||
..Default::default()
|
||||
};
|
||||
eframe::run_native(
|
||||
let started = eframe::run_native(
|
||||
"FilmCraft",
|
||||
options,
|
||||
Box::new(move |cc| {
|
||||
@@ -178,7 +186,16 @@ fn main() -> eframe::Result {
|
||||
}
|
||||
Ok(Box::new(app))
|
||||
}),
|
||||
)
|
||||
);
|
||||
// The window could not be created (no usable GPU / graphics driver, no display): say so
|
||||
// instead of exiting silently, and keep the reason in the crash log.
|
||||
if let Err(e) = &started {
|
||||
let msg = format!("FilmCraft could not start its window: {e}\n\nUpdating the graphics driver usually fixes this.");
|
||||
eprintln!("filmcraft: {msg}");
|
||||
filmcraft_ui_egui::crash::record(&msg);
|
||||
let _ = rfd::MessageDialog::new().set_title("FilmCraft").set_description(&msg).set_level(rfd::MessageLevel::Error).show();
|
||||
}
|
||||
started
|
||||
}
|
||||
|
||||
/// Open a file in its default application, or reveal it in the file manager (Edit Original,
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
allow-unwrap-in-tests = true
|
||||
allow-expect-in-tests = true
|
||||
allow-panic-in-tests = true
|
||||
allow-indexing-slicing-in-tests = true
|
||||
@@ -31,6 +31,8 @@
|
||||
//! # Ok::<(), filmcraft_aac::Error>(())
|
||||
//! ```
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
mod config;
|
||||
mod decoder;
|
||||
mod encoder;
|
||||
|
||||
@@ -13,6 +13,8 @@
|
||||
//!
|
||||
//! E-AC-3 (bsid 11-16) is recognised and refused.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
mod tables;
|
||||
|
||||
use std::sync::OnceLock;
|
||||
@@ -359,8 +361,7 @@ fn bit_allocation(exp: &[u8; 256], p: &AllocParams, g: &Globals, bap: &mut [u8;
|
||||
let bndend = tables::MASKTAB[end - 1] as usize + 1;
|
||||
let mut excite = [0i32; 50];
|
||||
let (mut fastleak, mut slowleak) = p.leak.unwrap_or((0, 0));
|
||||
let begin;
|
||||
if bndstrt == 0 {
|
||||
let begin = if bndstrt == 0 {
|
||||
// the last band of the LFE channel (bndend 7) has no band above it
|
||||
let lfe_last = |bin: usize| !(bndend == 7 && bin == 6);
|
||||
let mut lowcomp = 0;
|
||||
@@ -391,10 +392,10 @@ fn bit_allocation(exp: &[u8; 256], p: &AllocParams, g: &Globals, bap: &mut [u8;
|
||||
slowleak = slowleak.max(bndpsd[bin] - sgain);
|
||||
excite[bin] = (fastleak - lowcomp).max(slowleak);
|
||||
}
|
||||
begin = 22;
|
||||
22
|
||||
} else {
|
||||
begin = bndstrt;
|
||||
}
|
||||
bndstrt
|
||||
};
|
||||
for bin in begin..bndend {
|
||||
fastleak -= fdecay;
|
||||
fastleak = fastleak.max(bndpsd[bin] - p.fgain);
|
||||
@@ -1030,7 +1031,7 @@ impl Decoder {
|
||||
/// Coded channel order (Table 5.8) → WAV / SMPTE order: fronts L R C, LFE, surrounds.
|
||||
fn wav_order(mut pcm: Vec<Vec<f32>>, acmod: usize, lfeon: bool) -> Vec<Vec<f32>> {
|
||||
let nf = NFCHANS[acmod];
|
||||
let lfe = lfeon.then(|| pcm.pop().expect("lfe"));
|
||||
let lfe = if lfeon { pcm.pop() } else { None };
|
||||
let mut fbw = pcm;
|
||||
// acmod 3, 5, 7 code L C R …: move C after R
|
||||
if acmod & 1 == 1 && acmod != 1 {
|
||||
|
||||
@@ -99,7 +99,7 @@ fn ffmpeg_encoded_streams_decode() {
|
||||
}
|
||||
}
|
||||
let o = Command::new(&ff).args(["-v", "error", "-i"]).arg(&f).args(["-f", "f32le", "-"]).output().unwrap();
|
||||
let want: Vec<f32> = o.stdout.chunks_exact(4).map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
|
||||
let want: Vec<f32> = o.stdout.as_chunks::<4>().0.iter().map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
|
||||
let ch = s.channels as usize;
|
||||
let n = want.len() / ch;
|
||||
assert_eq!(n, ours[0].len(), "{}: sample count", s.name);
|
||||
|
||||
@@ -123,7 +123,7 @@ pub fn matrix(from: Layout, to: Layout, mixdown: Mixdown) -> Vec<Vec<f32>> {
|
||||
let (ni, no) = (from.channels(), to.channels());
|
||||
let mut m = vec![vec![0.0f32; ni]; no];
|
||||
match (from, to) {
|
||||
(a, b) if a == b => {
|
||||
(Layout::Mono, Layout::Mono) | (Layout::Stereo, Layout::Stereo) | (Layout::Surround51, Layout::Surround51) => {
|
||||
for (i, row) in m.iter_mut().enumerate() {
|
||||
row[i] = 1.0;
|
||||
}
|
||||
@@ -159,7 +159,6 @@ pub fn matrix(from: Layout, to: Layout, mixdown: Mixdown) -> Vec<Vec<f32>> {
|
||||
m[1][LFE] = K;
|
||||
}
|
||||
}
|
||||
_ => unreachable!("all layout pairs are covered"),
|
||||
}
|
||||
m
|
||||
}
|
||||
|
||||
@@ -16,6 +16,8 @@
|
||||
//! denormals never appear (no FTZ/DAZ CPU flags needed, which keeps the crate `unsafe`-free and
|
||||
//! portable to wasm).
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
pub mod biquad;
|
||||
pub mod channels;
|
||||
pub mod design;
|
||||
@@ -169,12 +171,11 @@ impl ParamValues {
|
||||
pub(crate) fn get(&self, id: &str) -> Option<f32> {
|
||||
self.index_of(id).map(|i| self.values[i])
|
||||
}
|
||||
/// Value of a parameter that is known to exist (panics otherwise — programming error).
|
||||
/// Value of a parameter that is known to exist (0 otherwise — a programming error caught by
|
||||
/// debug builds).
|
||||
pub(crate) fn v(&self, id: &str) -> f32 {
|
||||
match self.get(id) {
|
||||
Some(v) => v,
|
||||
None => panic!("unknown parameter {id}"),
|
||||
}
|
||||
debug_assert!(self.get(id).is_some(), "unknown parameter {id}");
|
||||
self.get(id).unwrap_or(0.0)
|
||||
}
|
||||
pub(crate) fn on(&self, id: &str) -> bool {
|
||||
self.v(id) >= 0.5
|
||||
|
||||
@@ -42,7 +42,9 @@ impl BridgeClient {
|
||||
*guard = Some((BufReader::new(r), w));
|
||||
}
|
||||
let id = self.next_id.fetch_add(1, Ordering::Relaxed);
|
||||
let conn = guard.as_mut().expect("connected");
|
||||
let Some(conn) = guard.as_mut() else {
|
||||
return Err(AutomationError::Bridge(format!("not connected to {}", self.addr)));
|
||||
};
|
||||
let line = format!("{}\n", json!({"id": id, "method": method, "params": params}));
|
||||
let res: Result<Value, AutomationError> = async {
|
||||
conn.1.write_all(line.as_bytes()).await.map_err(|e| AutomationError::Bridge(e.to_string()))?;
|
||||
|
||||
@@ -8,6 +8,8 @@
|
||||
//! (`filmcraft --control <port>`), so agents can also inspect, screenshot, click, drag, scroll
|
||||
//! and type in the live UI — every menu, panel, timeline gesture and keystroke.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
pub mod bridge;
|
||||
pub mod server;
|
||||
|
||||
|
||||
+32
-1
@@ -58,7 +58,8 @@ and `b10-23-film_grain-50` are **bit-exact** in the default run. The ignored
|
||||
with scaled inter-layer prediction), `24-monochrome` (8 and 10-bit) and
|
||||
`16-intra_only-intrabc-extreme-dv` (1080p intra block copy), `23-film_grain-50` (8 and 10-bit).
|
||||
|
||||
`tests/oracle_inter.rs` encodes SVT-AV1 GOPs (hierarchical references, compound prediction,
|
||||
`tests/oracle_inter.rs` also checks draft mode: every unflagged frame stays bit-exact with
|
||||
libdav1d and only flagged frames differ, with 1 and 4 threads. `tests/oracle_inter.rs` encodes SVT-AV1 GOPs (hierarchical references, compound prediction,
|
||||
OBMC / warped motion, motion-field projection, all loop filters; presets 3–8, 8- and 10-bit, odd
|
||||
sizes), plus super-resolution (denominator 12) and film-grain streams, and compares every frame
|
||||
with libdav1d: **bit-exact** on all 6 fixtures.
|
||||
@@ -91,6 +92,36 @@ noisy; single-threaded figures use thread CPU time and are comparable.
|
||||
| `perf_1080p_gop_hq` (crf 18) | 19.5 fps | 19.4 fps | 2.4 fps | 3.6 fps |
|
||||
| `perf_1080p_gop_tiles` (4x2 tiles) | 44.4 fps | 44.7 fps | 2.7 fps | 16.3 fps |
|
||||
|
||||
### M4.10 (2026-10-03)
|
||||
|
||||
CPU cycles per frame counted by the kernel (`/usr/bin/time -l`, all threads summed; mean of two
|
||||
interleaved base / after rounds at load average 190-320) with
|
||||
`AV1_THREADS=n cargo run --release -p filmcraft-av1 --example av1dec -- in.ivf` on the
|
||||
`cargo xtask bench` decode clips (SVT-AV1 preset 10, CRF 35, keyint 48, testsrc2 + grain),
|
||||
bit-exact with libdav1d:
|
||||
|
||||
| stream | threads | Mcycles / frame before | after | after, draft mode |
|
||||
|---|---|---|---|---|
|
||||
| 1080p (120 frames) | 1 | 60.5 | **48.3** (−20 %) | 46.2 |
|
||||
| 1080p | 14 | 63.6 | **51.9** (−18 %) | 50.3 |
|
||||
| 2160p (72 frames) | 1 | 212.5 | **172.4** (−19 %) | 167.9 |
|
||||
| 2160p | 14 | 220.0 | **177.1** (−20 %) | 171.1 |
|
||||
|
||||
What changed: the sub-pixel filter passes run over fixed block widths (taps outer, columns
|
||||
inner); frame and tile-region planes come from a bounded pool of recycled buffers (zeroed on
|
||||
reuse) instead of a fresh `calloc` and `munmap` per frame (allocation, page faults and zeroing
|
||||
were ~10 % of the time); the palette mode-info arrays (40 bytes per 4x4 unit) are only
|
||||
allocated when screen content tools are allowed; CDF adaptation is branch-free (a branch-free
|
||||
symbol search was tried and was slower: most symbols have 2-4 values). Single-threaded 4K now
|
||||
splits into coefficient parsing ~20 %, inter prediction ~18 %, other block-level parsing
|
||||
~12 %, deblocking ~11 % (still one edge at a time), inverse transforms ~12 %, CDEF ~5 %.
|
||||
|
||||
Draft mode (`Decoder::set_draft(true)`, `AV1_DRAFT=1`; reduced-resolution playback only): shown
|
||||
frames whose `refresh_frame_flags` is 0 — no reference slot keeps them, so neither their samples,
|
||||
motion vectors nor CDFs reach another frame — skip deblocking, CDEF and loop restoration
|
||||
(super-resolution still runs, it sets the output size) and are flagged `Picture::draft`. In
|
||||
SVT-AV1's hierarchical GOPs that is the top layer, half of the frames.
|
||||
|
||||
## API
|
||||
|
||||
```rust
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
//! Decode an IVF AV1 file: `cargo run --release -p filmcraft-av1 --example av1dec -- in.ivf [out.yuv]`.
|
||||
//! Writes raw planar 16-bit little-endian output. `AV1_THREADS=n` sets the thread count,
|
||||
//! `AV1_LOOPS=k` decodes the file k times (benchmarking), `AV1_STATS=1` prints stage statistics.
|
||||
|
||||
use std::io::Write;
|
||||
|
||||
fn main() {
|
||||
let args: Vec<String> = std::env::args().collect();
|
||||
if args.len() < 2 {
|
||||
eprintln!("usage: av1dec in.ivf [out.yuv]");
|
||||
std::process::exit(2);
|
||||
}
|
||||
let data = std::fs::read(&args[1]).expect("read input");
|
||||
let mut out = args.get(2).map(|p| std::io::BufWriter::new(std::fs::File::create(p).expect("create output")));
|
||||
let threads = std::env::var("AV1_THREADS").ok().and_then(|s| s.parse().ok()).unwrap_or(0usize);
|
||||
let loops = std::env::var("AV1_LOOPS").ok().and_then(|s| s.parse().ok()).unwrap_or(1usize);
|
||||
let hl = u16::from_le_bytes([data[6], data[7]]) as usize;
|
||||
let t0 = std::time::Instant::now();
|
||||
let mut n = 0;
|
||||
let mut dec = if threads > 0 { filmcraft_av1::Decoder::with_threads(threads) } else { filmcraft_av1::Decoder::new() };
|
||||
// AV1_DRAFT=1: draft mode (non-reference frames skip the in-loop filters).
|
||||
dec.set_draft(std::env::var_os("AV1_DRAFT").is_some());
|
||||
let mut write = |pics: Vec<filmcraft_av1::Picture>| {
|
||||
for pic in pics {
|
||||
n += 1;
|
||||
if let Some(o) = out.as_mut() {
|
||||
for pl in &pic.planes {
|
||||
let bytes: Vec<u8> = pl.iter().flat_map(|s| s.to_le_bytes()).collect();
|
||||
o.write_all(&bytes).unwrap();
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
for _ in 0..loops {
|
||||
let mut p = hl;
|
||||
let mut i = 0i64;
|
||||
while p + 12 <= data.len() {
|
||||
let sz = u32::from_le_bytes(data[p..p + 4].try_into().unwrap()) as usize;
|
||||
p += 12;
|
||||
let Some(tu) = data.get(p..p + sz) else { break };
|
||||
p += sz;
|
||||
match dec.decode_pts(tu, i) {
|
||||
Ok(pics) => write(pics),
|
||||
Err(e) => eprintln!("temporal unit {i}: {e}"),
|
||||
}
|
||||
i += 1;
|
||||
}
|
||||
write(dec.flush());
|
||||
}
|
||||
eprintln!("{n} pictures in {:.3}s", t0.elapsed().as_secs_f64());
|
||||
if std::env::var_os("AV1_STATS").is_some() {
|
||||
eprintln!("{:#?}", dec.stats());
|
||||
}
|
||||
}
|
||||
@@ -237,12 +237,16 @@ fn filter_rows<const W: usize>(buf: &[i32; PAD_STRIDE * PAD_STRIDE], taps: &[(is
|
||||
}
|
||||
for i in 0..h {
|
||||
let ci = ((i + 2) * PAD_STRIDE + 2) as isize;
|
||||
let x: [i16; W] = b16[ci as usize..ci as usize + W].try_into().expect("centre row");
|
||||
let Some(&x) = b16.get(ci as usize..).and_then(|r| r.first_chunk::<W>()) else {
|
||||
return;
|
||||
};
|
||||
let (mut sum, mut max, mut min) = ([0i16; W], x, x);
|
||||
for &(off, weight, thr, adj) in taps {
|
||||
let (weight, thr) = (weight as i16, thr as i16);
|
||||
let start = (ci + off) as usize;
|
||||
let p: &[i16; W] = b16[start..start + W].try_into().expect("tap row");
|
||||
let Some(p) = b16.get(start..).and_then(|r| r.first_chunk::<W>()) else {
|
||||
return;
|
||||
};
|
||||
for j in 0..W {
|
||||
// constrain(): .max / .min instead of clamp (whose bound check panics), and
|
||||
// the sign applied branch-free, so the loop vectorises.
|
||||
|
||||
@@ -72,10 +72,12 @@ impl<T: Clone> Slot<T> {
|
||||
|
||||
fn wait(&self) -> Result<T> {
|
||||
let mut g = self.value.lock().unwrap_or_else(|e| e.into_inner());
|
||||
while g.is_none() {
|
||||
loop {
|
||||
if let Some(v) = g.as_ref() {
|
||||
return v.clone();
|
||||
}
|
||||
g = self.cv.wait(g).unwrap_or_else(|e| e.into_inner());
|
||||
}
|
||||
g.as_ref().expect("slot value").clone()
|
||||
}
|
||||
}
|
||||
|
||||
@@ -95,6 +97,8 @@ struct FrameJob {
|
||||
picture: Option<Arc<Slot<Arc<Picture>>>>,
|
||||
pts: i64,
|
||||
apply_film_grain: bool,
|
||||
/// Draft mode: the frame refreshes no reference slot, so its in-loop filters are skipped.
|
||||
draft: bool,
|
||||
}
|
||||
|
||||
/// A shown picture in output order.
|
||||
@@ -193,6 +197,7 @@ pub struct Decoder {
|
||||
seen_frame_header: bool,
|
||||
/// Apply film grain synthesis to output frames (default true).
|
||||
pub apply_film_grain: bool,
|
||||
draft: bool,
|
||||
stats: Arc<Mutex<DecodeStats>>,
|
||||
threads: usize,
|
||||
pool: Pool,
|
||||
@@ -255,6 +260,7 @@ impl Decoder {
|
||||
pending: None,
|
||||
seen_frame_header: false,
|
||||
apply_film_grain: true,
|
||||
draft: false,
|
||||
stats,
|
||||
threads,
|
||||
pool,
|
||||
@@ -268,6 +274,14 @@ impl Decoder {
|
||||
}
|
||||
|
||||
/// Frame / tile counts and per-stage busy time of the frames finished so far.
|
||||
/// Draft mode for reduced-resolution playback (off by default): shown frames that refresh
|
||||
/// no reference slot (nothing can predict from them, read their motion vectors or CDFs)
|
||||
/// skip deblocking, CDEF and loop restoration and come out flagged [`Picture::draft`];
|
||||
/// every other picture is unchanged.
|
||||
pub fn set_draft(&mut self, on: bool) {
|
||||
self.draft = on;
|
||||
}
|
||||
|
||||
pub fn stats(&self) -> DecodeStats {
|
||||
self.stats.lock().map(|s| *s).unwrap_or_default()
|
||||
}
|
||||
@@ -296,7 +310,7 @@ impl Decoder {
|
||||
if !o.is_ready() && self.out.len() <= max_queue {
|
||||
break;
|
||||
}
|
||||
let o = self.out.pop_front().expect("front");
|
||||
let Some(o) = self.out.pop_front() else { break };
|
||||
pics.push(o.take()?);
|
||||
}
|
||||
parsed?;
|
||||
@@ -346,7 +360,9 @@ impl Decoder {
|
||||
match last {
|
||||
Some((lt, lsid)) if lt == t => {
|
||||
if sid >= lsid {
|
||||
*sel.last_mut().expect("picture") = o;
|
||||
if let Some(l) = sel.last_mut() {
|
||||
*l = o;
|
||||
}
|
||||
last = Some((t, sid));
|
||||
}
|
||||
}
|
||||
@@ -503,6 +519,7 @@ impl Decoder {
|
||||
return Err(Error::Invalid("primary reference frame missing"));
|
||||
}
|
||||
let picture = fh.show_frame.then(Slot::new);
|
||||
let draft = self.draft && fh.show_frame && fh.refresh_frame_flags == 0;
|
||||
self.pending = Some(Box::new(FrameJob {
|
||||
seq,
|
||||
fh,
|
||||
@@ -513,6 +530,7 @@ impl Decoder {
|
||||
picture,
|
||||
pts: self.pts,
|
||||
apply_film_grain: self.apply_film_grain,
|
||||
draft,
|
||||
}));
|
||||
Ok(None)
|
||||
}
|
||||
@@ -598,7 +616,7 @@ impl Decoder {
|
||||
self.in_flight.push_back(slot);
|
||||
// Bound the frames in flight (memory): wait for the oldest.
|
||||
while self.in_flight.len() > self.max_in_flight {
|
||||
let s = self.in_flight.pop_front().expect("in flight");
|
||||
let Some(s) = self.in_flight.pop_front() else { break };
|
||||
let _ = s.wait();
|
||||
}
|
||||
self.in_flight.retain(|s| !s.is_set());
|
||||
@@ -692,7 +710,8 @@ fn decode_frame(job: &FrameJob, pool: &Pool, stats: &mut DecodeStats) -> Result<
|
||||
{
|
||||
cdfs = s;
|
||||
}
|
||||
crate::postfilter::apply(&mut fs, stats, pool);
|
||||
crate::postfilter::apply(&mut fs, stats, pool, job.draft);
|
||||
stats.draft_frames += job.draft as u64;
|
||||
let t = Timer::start();
|
||||
stats.frames += 1;
|
||||
if fh.seg.enabled && !fh.seg.update_map {
|
||||
@@ -714,8 +733,11 @@ fn decode_frame(job: &FrameJob, pool: &Pool, stats: &mut DecodeStats) -> Result<
|
||||
saved_ref_frames: mf_refs,
|
||||
saved_mvs: mf_mvs,
|
||||
});
|
||||
let pic =
|
||||
fh.show_frame.then(|| output_picture(seq, &rf, fh.upscaled_width as usize, fh.frame_height as usize, &fh.film_grain, job.apply_film_grain, job.pts));
|
||||
let pic = fh.show_frame.then(|| {
|
||||
let mut p = output_picture(seq, &rf, fh.upscaled_width as usize, fh.frame_height as usize, &fh.film_grain, job.apply_film_grain, job.pts);
|
||||
p.draft = job.draft;
|
||||
p
|
||||
});
|
||||
stats.add(Stage::Output, t.secs());
|
||||
Ok((rf, pic))
|
||||
}
|
||||
@@ -800,6 +822,7 @@ fn output_picture(seq: &SequenceHeader, rf: &RefFrame, w: usize, h: usize, grain
|
||||
matrix_coefficients: rf.color.2,
|
||||
full_range: rf.color.3,
|
||||
pts,
|
||||
draft: false,
|
||||
};
|
||||
if apply_film_grain && seq.film_grain_params_present && grain.apply_grain {
|
||||
crate::grain::apply(&mut pic, grain, seq);
|
||||
|
||||
+57
-9
@@ -4,8 +4,32 @@
|
||||
//! or [`MiInfo`] may hold a region of the frame (origin `ox`, `oy`), addressed with frame
|
||||
//! coordinates throughout.
|
||||
|
||||
/// Recycled plane allocations: every frame allocates (and every evicted reference frees)
|
||||
/// megabytes of planes, and fresh large allocations cost page faults and kernel calls. Buffers
|
||||
/// are zeroed again on reuse, so contents never leak between frames.
|
||||
static PLANE_POOL: std::sync::Mutex<Vec<Vec<u16>>> = std::sync::Mutex::new(Vec::new());
|
||||
const POOL_MIN_LEN: usize = 1 << 16;
|
||||
/// Samples kept at most (96 M samples = 192 MB, a few 4K frames).
|
||||
const POOL_MAX_SAMPLES: usize = 96 << 20;
|
||||
|
||||
/// A zeroed buffer of `len` samples (recycled memory for large ones).
|
||||
fn zeroed(len: usize) -> Vec<u16> {
|
||||
if len >= POOL_MIN_LEN {
|
||||
let found = {
|
||||
let mut pool = PLANE_POOL.lock().unwrap_or_else(|e| e.into_inner());
|
||||
pool.iter().position(|v| v.capacity() >= len && v.capacity() <= 2 * len).map(|i| pool.swap_remove(i))
|
||||
};
|
||||
if let Some(mut v) = found {
|
||||
v.clear();
|
||||
v.resize(len, 0);
|
||||
return v;
|
||||
}
|
||||
}
|
||||
vec![0; len]
|
||||
}
|
||||
|
||||
/// One plane of samples (u16 at every bit depth), or a rectangular region of one.
|
||||
#[derive(Clone, Default)]
|
||||
#[derive(Default)]
|
||||
pub struct Plane {
|
||||
pub data: Vec<u16>,
|
||||
pub stride: usize,
|
||||
@@ -16,13 +40,34 @@ pub struct Plane {
|
||||
pub oy: usize,
|
||||
}
|
||||
|
||||
impl Clone for Plane {
|
||||
fn clone(&self) -> Plane {
|
||||
let mut data = zeroed(self.data.len());
|
||||
data.copy_from_slice(&self.data);
|
||||
Plane { data, stride: self.stride, rows: self.rows, ox: self.ox, oy: self.oy }
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for Plane {
|
||||
fn drop(&mut self) {
|
||||
if self.data.capacity() >= POOL_MIN_LEN {
|
||||
let v = std::mem::take(&mut self.data);
|
||||
let mut pool = PLANE_POOL.lock().unwrap_or_else(|e| e.into_inner());
|
||||
let kept: usize = pool.iter().map(|b| b.capacity()).sum();
|
||||
if kept + v.capacity() <= POOL_MAX_SAMPLES {
|
||||
pool.push(v);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Plane {
|
||||
pub fn new(width: usize, height: usize) -> Plane {
|
||||
Plane { data: vec![0; width * height], stride: width, rows: height, ox: 0, oy: 0 }
|
||||
Plane { data: zeroed(width * height), stride: width, rows: height, ox: 0, oy: 0 }
|
||||
}
|
||||
/// A `width` x `height` region with its top-left sample at frame position (ox, oy).
|
||||
pub fn region(ox: usize, oy: usize, width: usize, height: usize) -> Plane {
|
||||
Plane { data: vec![0; width * height], stride: width, rows: height, ox, oy }
|
||||
Plane { data: zeroed(width * height), stride: width, rows: height, ox, oy }
|
||||
}
|
||||
#[inline(always)]
|
||||
pub fn at(&self, x: usize, y: usize) -> u16 {
|
||||
@@ -173,13 +218,16 @@ pub struct MiInfo {
|
||||
}
|
||||
|
||||
impl MiInfo {
|
||||
pub fn new(cols: usize, rows: usize) -> MiInfo {
|
||||
MiInfo::region(cols, rows, 0, 0, cols, rows)
|
||||
/// `palette`: whether palette mode can occur (allow_screen_content_tools); without it the
|
||||
/// palette arrays (40 bytes per unit) stay empty.
|
||||
pub fn new(cols: usize, rows: usize, palette: bool) -> MiInfo {
|
||||
MiInfo::region(cols, rows, 0, 0, cols, rows, palette)
|
||||
}
|
||||
|
||||
/// Storage for the w x h mode info units at (ox, oy) of a cols x rows frame.
|
||||
pub fn region(cols: usize, rows: usize, ox: usize, oy: usize, w: usize, h: usize) -> MiInfo {
|
||||
pub fn region(cols: usize, rows: usize, ox: usize, oy: usize, w: usize, h: usize, palette: bool) -> MiInfo {
|
||||
let n = w * h;
|
||||
let np = if palette { n } else { 0 };
|
||||
MiInfo {
|
||||
cols,
|
||||
rows,
|
||||
@@ -198,8 +246,8 @@ impl MiInfo {
|
||||
inter_tx_size: vec![0; n],
|
||||
mi_size: vec![0; n],
|
||||
segment_id: vec![0; n],
|
||||
palette_size: [vec![0; n], vec![0; n]],
|
||||
palette_colors: [vec![[0; 8]; n], vec![[0; 8]; n]],
|
||||
palette_size: [vec![0; np], vec![0; np]],
|
||||
palette_colors: [vec![[0; 8]; np], vec![[0; 8]; np]],
|
||||
delta_lf: vec![[0; 4]; n],
|
||||
comp_group_idx: vec![0; n],
|
||||
compound_idx: vec![0; n],
|
||||
@@ -228,7 +276,7 @@ impl MiInfo {
|
||||
)*};
|
||||
}
|
||||
copy!(y_mode uv_mode ref_frame mv is_inter skip_mode skip tx_size inter_tx_size mi_size segment_id delta_lf comp_group_idx compound_idx interp_filter motion_mode tx_type written);
|
||||
for k in 0..2 {
|
||||
for k in (0..2).filter(|&k| !self.palette_size[k].is_empty()) {
|
||||
for y in 0..h {
|
||||
let d = dst.idx(self.oy + y, self.ox);
|
||||
dst.palette_size[k][d..d + w].copy_from_slice(&self.palette_size[k][y * w..(y + 1) * w]);
|
||||
|
||||
+87
-19
@@ -180,7 +180,11 @@ impl TileDecoder<'_, '_> {
|
||||
}
|
||||
|
||||
fn ref_plane(&self, ref_idx: i32, plane: usize) -> &Plane {
|
||||
if ref_idx < 0 { &self.t.cur.planes[plane] } else { &self.fs.refs[ref_idx as usize].as_ref().expect("reference frame").buf.planes[plane] }
|
||||
// A missing reference (rejected earlier for valid streams) predicts from the current frame.
|
||||
match usize::try_from(ref_idx).ok().and_then(|i| self.fs.refs.get(i)).and_then(|r| r.as_ref()) {
|
||||
Some(r) => &r.buf.planes[plane],
|
||||
None => &self.t.cur.planes[plane],
|
||||
}
|
||||
}
|
||||
|
||||
/// Block inter prediction process (7.11.3.4) into `pred` (stride PRED_STRIDE); `tmp` holds
|
||||
@@ -665,6 +669,49 @@ impl TileDecoder<'_, '_> {
|
||||
|
||||
const IDENTITY_TAPS: [i16; 8] = [0, 0, 0, 128, 0, 0, 0, 0];
|
||||
|
||||
/// Horizontal pass of one row for a block `W` wide: taps outer, columns inner, fixed width so
|
||||
/// the row stays in registers.
|
||||
#[inline(always)]
|
||||
fn hpass<const W: usize>(src: &[u16], hf: &[i16; 8], out: &mut [i32], half: i32, sh: u32) {
|
||||
let mut acc = [half; W];
|
||||
for t in 0..8 {
|
||||
let f = hf[t] as i32;
|
||||
let Some(p) = src.get(t..).and_then(|s| s.first_chunk::<W>()) else {
|
||||
return;
|
||||
};
|
||||
for j in 0..W {
|
||||
acc[j] += f * p[j] as i32;
|
||||
}
|
||||
}
|
||||
let Some(o) = out.first_chunk_mut::<W>() else {
|
||||
return;
|
||||
};
|
||||
for j in 0..W {
|
||||
o[j] = acc[j] >> sh;
|
||||
}
|
||||
}
|
||||
|
||||
/// Vertical pass of one output row from 8 intermediate rows (`rows`, stride `W`).
|
||||
#[inline(always)]
|
||||
fn vpass<const W: usize>(rows: &[i32], vf: &[i16; 8], out: &mut [i32], half: i32, sh: u32) {
|
||||
let mut acc = [half; W];
|
||||
for t in 0..8 {
|
||||
let f = vf[t] as i32;
|
||||
let Some(p) = rows.get(t * W..).and_then(|s| s.first_chunk::<W>()) else {
|
||||
return;
|
||||
};
|
||||
for j in 0..W {
|
||||
acc[j] += f * p[j];
|
||||
}
|
||||
}
|
||||
let Some(o) = out.first_chunk_mut::<W>() else {
|
||||
return;
|
||||
};
|
||||
for j in 0..W {
|
||||
o[j] = acc[j] >> sh;
|
||||
}
|
||||
}
|
||||
|
||||
/// Block inter prediction without reference scaling: every column / row uses the same filter
|
||||
/// phase, so the filters run over whole rows (bit-identical to the per-sample process).
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
@@ -711,17 +758,27 @@ fn mc_unscaled(
|
||||
*o = (v as i32) << (7 - r0);
|
||||
}
|
||||
} else {
|
||||
let acc = &mut acc[..w];
|
||||
acc.fill(0);
|
||||
for t in 0..8 {
|
||||
let f = hf[t] as i32;
|
||||
for (a, &v) in acc.iter_mut().zip(&src[t..t + w]) {
|
||||
*a += f * v as i32;
|
||||
match w {
|
||||
2 => hpass::<2>(src, hf, out, half0, r0),
|
||||
4 => hpass::<4>(src, hf, out, half0, r0),
|
||||
8 => hpass::<8>(src, hf, out, half0, r0),
|
||||
16 => hpass::<16>(src, hf, out, half0, r0),
|
||||
32 => hpass::<32>(src, hf, out, half0, r0),
|
||||
64 => hpass::<64>(src, hf, out, half0, r0),
|
||||
_ => {
|
||||
let acc = &mut acc[..w];
|
||||
acc.fill(0);
|
||||
for t in 0..8 {
|
||||
let f = hf[t] as i32;
|
||||
for (a, &v) in acc.iter_mut().zip(&src[t..t + w]) {
|
||||
*a += f * v as i32;
|
||||
}
|
||||
}
|
||||
for (o, &a) in out.iter_mut().zip(acc.iter()) {
|
||||
*o = (a + half0) >> r0;
|
||||
}
|
||||
}
|
||||
}
|
||||
for (o, &a) in out.iter_mut().zip(acc.iter()) {
|
||||
*o = (a + half0) >> r0;
|
||||
}
|
||||
}
|
||||
}
|
||||
for r in 0..h {
|
||||
@@ -731,17 +788,28 @@ fn mc_unscaled(
|
||||
*o = (128 * v + half1) >> r1;
|
||||
}
|
||||
} else {
|
||||
let acc = &mut acc[..w];
|
||||
acc.fill(0);
|
||||
for t in 0..8 {
|
||||
let f = vf[t] as i32;
|
||||
for (a, &v) in acc.iter_mut().zip(&tmp[(r + t) * w..(r + t + 1) * w]) {
|
||||
*a += f * v;
|
||||
let rows = &tmp[r * w..(r + 8) * w];
|
||||
match w {
|
||||
2 => vpass::<2>(rows, vf, out, half1, r1),
|
||||
4 => vpass::<4>(rows, vf, out, half1, r1),
|
||||
8 => vpass::<8>(rows, vf, out, half1, r1),
|
||||
16 => vpass::<16>(rows, vf, out, half1, r1),
|
||||
32 => vpass::<32>(rows, vf, out, half1, r1),
|
||||
64 => vpass::<64>(rows, vf, out, half1, r1),
|
||||
_ => {
|
||||
let acc = &mut acc[..w];
|
||||
acc.fill(0);
|
||||
for t in 0..8 {
|
||||
let f = vf[t] as i32;
|
||||
for (a, &v) in acc.iter_mut().zip(&tmp[(r + t) * w..(r + t + 1) * w]) {
|
||||
*a += f * v;
|
||||
}
|
||||
}
|
||||
for (o, &a) in out.iter_mut().zip(acc.iter()) {
|
||||
*o = (a + half1) >> r1;
|
||||
}
|
||||
}
|
||||
}
|
||||
for (o, &a) in out.iter_mut().zip(acc.iter()) {
|
||||
*o = (a + half1) >> r1;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -5,6 +5,7 @@
|
||||
|
||||
// Items used only by stages still being written (inter prediction, post filters).
|
||||
#![allow(dead_code)]
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
mod bits;
|
||||
mod cdef;
|
||||
@@ -62,6 +63,9 @@ pub struct Picture {
|
||||
pub full_range: bool,
|
||||
/// Timestamp of the temporal unit that showed the picture ([`Decoder::decode_pts`]).
|
||||
pub pts: i64,
|
||||
/// Decoded in draft mode without deblocking / CDEF / loop restoration
|
||||
/// ([`Decoder::set_draft`]): approximate, for reduced-resolution playback only.
|
||||
pub draft: bool,
|
||||
}
|
||||
|
||||
impl Picture {
|
||||
|
||||
@@ -6,14 +6,15 @@ use crate::state::FrameState;
|
||||
use crate::stats::{DecodeStats, Stage, Timer};
|
||||
|
||||
/// The post-filters run on the frame worker; `_pool` is for row-parallel filtering (not yet used).
|
||||
pub(crate) fn apply(fs: &mut FrameState, stats: &mut DecodeStats, _pool: &crate::par::Pool) {
|
||||
/// `draft`: only super-resolution (which sets the output size) runs.
|
||||
pub(crate) fn apply(fs: &mut FrameState, stats: &mut DecodeStats, _pool: &crate::par::Pool, draft: bool) {
|
||||
let mut t = Timer::start();
|
||||
let lvl = fs.fh.lf.level;
|
||||
if lvl[0] != 0 || lvl[1] != 0 {
|
||||
if (lvl[0] != 0 || lvl[1] != 0) && !draft {
|
||||
loop_filter(fs);
|
||||
}
|
||||
stats.add(Stage::LoopFilter, t.lap());
|
||||
let cdef = crate::cdef::apply(fs);
|
||||
let cdef = if draft { None } else { crate::cdef::apply(fs) };
|
||||
stats.add(Stage::Cdef, t.lap());
|
||||
let (up_cur, up_cdef) = if fs.fh.use_superres {
|
||||
let c = crate::restoration::upscale(fs, &fs.cur);
|
||||
@@ -23,7 +24,11 @@ pub(crate) fn apply(fs: &mut FrameState, stats: &mut DecodeStats, _pool: &crate:
|
||||
(std::mem::take(&mut fs.cur), cdef)
|
||||
};
|
||||
stats.add(Stage::Superres, t.lap());
|
||||
fs.cur = if fs.fh.lr.uses_lr { crate::restoration::loop_restoration(fs, &up_cur, up_cdef.as_ref().unwrap_or(&up_cur)) } else { up_cdef.unwrap_or(up_cur) };
|
||||
fs.cur = if fs.fh.lr.uses_lr && !draft {
|
||||
crate::restoration::loop_restoration(fs, &up_cur, up_cdef.as_ref().unwrap_or(&up_cur))
|
||||
} else {
|
||||
up_cdef.unwrap_or(up_cur)
|
||||
};
|
||||
stats.add(Stage::Restoration, t.lap());
|
||||
}
|
||||
|
||||
|
||||
@@ -215,7 +215,7 @@ impl FrameState {
|
||||
prev_segment_ids: vec![0; mi_cols * mi_rows],
|
||||
},
|
||||
cur,
|
||||
mi: MiInfo::new(mi_cols, mi_rows),
|
||||
mi: MiInfo::new(mi_cols, mi_rows, fh.allow_screen_content_tools),
|
||||
cdef: vec![-1; cdef_cols * cdef_rows],
|
||||
lf_tx_size: std::array::from_fn(|_| Grid8::new(0, 0, mi_cols + 32, mi_rows + 32)),
|
||||
lr,
|
||||
@@ -260,7 +260,7 @@ impl FrameState {
|
||||
let x1 = if c1 >= mi_cols { usize::MAX } else { c1 * 4 };
|
||||
let y1 = if r1 >= mi_rows { usize::MAX } else { r1 * 4 };
|
||||
let cur = self.cur.region_like(c0 * 4, r0 * 4, x1, y1);
|
||||
let mi = MiInfo::region(mi_cols, mi_rows, c0, r0, c1 - c0, r1 - r0);
|
||||
let mi = MiInfo::region(mi_cols, mi_rows, c0, r0, c1 - c0, r1 - r0, !self.mi.palette_size[0].is_empty());
|
||||
let lf = std::array::from_fn(|p| {
|
||||
let (sx, sy) = if p == 0 { (0, 0) } else { (self.ssx, self.ssy) };
|
||||
let full = &self.lf_tx_size[p];
|
||||
|
||||
@@ -43,6 +43,8 @@ impl Stage {
|
||||
pub struct DecodeStats {
|
||||
/// Frames decoded (excluding show_existing_frame repeats).
|
||||
pub frames: u64,
|
||||
/// Frames decoded in draft mode without in-loop filters.
|
||||
pub draft_frames: u64,
|
||||
/// Tiles decoded.
|
||||
pub tiles: u64,
|
||||
/// Busy seconds per [`Stage`] (index with `stage as usize`).
|
||||
@@ -60,6 +62,7 @@ impl DecodeStats {
|
||||
|
||||
pub(crate) fn merge(&mut self, o: &DecodeStats) {
|
||||
self.frames += o.frames;
|
||||
self.draft_frames += o.draft_frames;
|
||||
self.tiles += o.tiles;
|
||||
for (a, b) in self.stage_secs.iter_mut().zip(o.stage_secs.iter()) {
|
||||
*a += b;
|
||||
|
||||
@@ -127,17 +127,12 @@ impl<'a> SymbolDecoder<'a> {
|
||||
pub(crate) fn update_cdf(cdf: &mut [u16], n: usize, symbol: usize) {
|
||||
let count = cdf[n];
|
||||
let rate = 3 + (count > 15) as u32 + (count > 31) as u32 + (31 - (n as u32).leading_zeros()).min(2);
|
||||
let mut tmp = 0u32;
|
||||
for i in 0..n - 1 {
|
||||
if i == symbol {
|
||||
tmp = 1 << 15;
|
||||
}
|
||||
let c = cdf[i] as u32;
|
||||
if tmp < c {
|
||||
cdf[i] = (c - ((c - tmp) >> rate)) as u16;
|
||||
} else {
|
||||
cdf[i] = (c + ((tmp - c) >> rate)) as u16;
|
||||
}
|
||||
// Entries below `symbol` move towards 0, the others towards 1 << 15 (the specification's
|
||||
// comparison with tmp, resolved per entry: c >= 0 and c <= 1 << 15), without branching on
|
||||
// the decoded symbol.
|
||||
for (i, e) in cdf[..n - 1].iter_mut().enumerate() {
|
||||
let c = *e as u32;
|
||||
*e = if i < symbol { c - (c >> rate) } else { c + (((1 << 15) - c) >> rate) } as u16;
|
||||
}
|
||||
cdf[n] += (count < 32) as u16;
|
||||
}
|
||||
@@ -474,10 +474,12 @@ impl<'a, 'f> TileDecoder<'a, 'f> {
|
||||
mi.tx_size[span.clone()].fill(b.tx_size as u8);
|
||||
mi.mi_size[span.clone()].fill(b.mi_size as u8);
|
||||
mi.segment_id[span.clone()].fill(b.segment_id as u8);
|
||||
mi.palette_size[0][span.clone()].fill(b.palette_size_y as u8);
|
||||
mi.palette_size[1][span.clone()].fill(b.palette_size_uv as u8);
|
||||
mi.palette_colors[0][span.clone()].fill(b.palette_colors_y);
|
||||
mi.palette_colors[1][span.clone()].fill(b.palette_colors_u);
|
||||
if !mi.palette_size[0].is_empty() {
|
||||
mi.palette_size[0][span.clone()].fill(b.palette_size_y as u8);
|
||||
mi.palette_size[1][span.clone()].fill(b.palette_size_uv as u8);
|
||||
mi.palette_colors[0][span.clone()].fill(b.palette_colors_y);
|
||||
mi.palette_colors[1][span.clone()].fill(b.palette_colors_u);
|
||||
}
|
||||
mi.delta_lf[span.clone()].fill(dlf);
|
||||
mi.motion_mode[span].fill(b.motion_mode);
|
||||
}
|
||||
|
||||
@@ -76,7 +76,7 @@ pub fn reference(ff: &Path, path: &Path, pix_fmt: &str) -> Vec<u16> {
|
||||
.expect("run ffmpeg");
|
||||
assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr));
|
||||
if pix_fmt.ends_with("le") {
|
||||
out.stdout.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
|
||||
out.stdout.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
|
||||
} else {
|
||||
out.stdout.iter().map(|&b| b as u16).collect()
|
||||
}
|
||||
@@ -89,7 +89,13 @@ pub fn decode_all(path: &Path) -> Result<Vec<Picture>, String> {
|
||||
|
||||
/// Decode every frame of an IVF file with a decoder using `threads` threads.
|
||||
pub fn decode_all_threads(path: &Path, threads: usize) -> Result<Vec<Picture>, String> {
|
||||
decode_all_opts(path, threads, false)
|
||||
}
|
||||
|
||||
/// [`decode_all_threads`] with draft mode on or off.
|
||||
pub fn decode_all_opts(path: &Path, threads: usize, draft: bool) -> Result<Vec<Picture>, String> {
|
||||
let mut dec = Decoder::with_threads(threads);
|
||||
dec.set_draft(draft);
|
||||
let mut pics = Vec::new();
|
||||
for (i, f) in ivf_frames(path).iter().enumerate() {
|
||||
let p = dec.decode_pts(f, i as i64).map_err(|e| format!("frame {i}: {e}"))?;
|
||||
@@ -174,6 +180,7 @@ pub fn check_file(ff: &Path, name: &str, path: &Path) {
|
||||
assert!(a == b, "{name}: frame {i} differs with {threads} threads");
|
||||
}
|
||||
assert!(!pics.is_empty(), "{name}: no frames");
|
||||
assert!(pics.iter().all(|p| !p.draft), "{name}: picture flagged draft without draft mode");
|
||||
let raw = reference(ff, path, pix_fmt_for(&pics[0]));
|
||||
let per = picture_samples(&pics[0]);
|
||||
assert_eq!(raw.len(), per * pics.len(), "{name}: frame count/size ({} pictures)", pics.len());
|
||||
|
||||
@@ -40,6 +40,34 @@ fn inter_gops() {
|
||||
}
|
||||
}
|
||||
|
||||
/// Draft mode: frames that refresh no reference slot skip the in-loop filters and are flagged;
|
||||
/// every other picture stays bit-exact, with 1 and several threads.
|
||||
#[test]
|
||||
fn draft_mode_changes_only_flagged_non_reference_frames() {
|
||||
let Some(ff) = ffmpeg() else { return };
|
||||
let s = &specs()[0];
|
||||
let path = make(&ff, s);
|
||||
let first = decode_all_threads(&path, 1).expect("decode");
|
||||
let raw = reference(&ff, &path, pix_fmt_for(&first[0]));
|
||||
let per = picture_samples(&first[0]);
|
||||
for threads in [1, 4] {
|
||||
let pics = decode_all_opts(&path, threads, true).expect("decode");
|
||||
assert_eq!(pics.len(), first.len(), "threads {threads}");
|
||||
let (mut draft, mut changed) = (0, 0);
|
||||
for (i, p) in pics.iter().enumerate() {
|
||||
let (_, count) = compare(p, &raw[i * per..(i + 1) * per]);
|
||||
if p.draft {
|
||||
draft += 1;
|
||||
changed += (count > 0) as usize;
|
||||
} else {
|
||||
assert_eq!(count, 0, "threads {threads}: unflagged frame {i} differs");
|
||||
}
|
||||
}
|
||||
assert!(draft > 0, "threads {threads}: no non-reference frames");
|
||||
assert!(changed > 0, "threads {threads}: draft frames identical to the filtered ones");
|
||||
}
|
||||
}
|
||||
|
||||
/// Several tiles (decoded in parallel into private buffers, then merged).
|
||||
fn specs_tiles() -> Vec<Spec> {
|
||||
let p = |preset: &'static str, params: &'static str| -> Vec<&'static str> { vec!["-preset", preset, "-crf", "32", "-svtav1-params", params] };
|
||||
|
||||
@@ -5,6 +5,8 @@
|
||||
//! - [`unescape_rbsp`] / [`escape_rbsp`]: remove/insert H.264/HEVC emulation-prevention bytes.
|
||||
//! - [`annexb_nals`] / [`length_prefixed_nals`]: split Annex-B byte streams and AVCC/HVCC samples into NAL units.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
use std::fmt;
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
|
||||
@@ -15,6 +15,8 @@
|
||||
//! - **Burn-in:** [`burn`] lays a caption out with the track style and draws it with the
|
||||
//! `filmcraft-text` engine (Inter SemiBold, shaped and kerned).
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
pub mod burn;
|
||||
pub mod cea608;
|
||||
pub mod mcc;
|
||||
@@ -212,7 +214,7 @@ pub fn parse(bytes: &[u8], format: Format) -> Result<Document> {
|
||||
Format::Scc => scc::parse(&text),
|
||||
Format::Mcc => mcc::parse(&text),
|
||||
Format::Ttml | Format::Dfxp => ttml::parse(&text),
|
||||
Format::Stl => unreachable!("handled above"),
|
||||
Format::Stl => stl::parse(bytes),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -253,7 +255,7 @@ pub fn decode_text(bytes: &[u8]) -> String {
|
||||
} else if bytes.starts_with(&[0xff, 0xfe]) || bytes.starts_with(&[0xfe, 0xff]) {
|
||||
let le = bytes[0] == 0xff;
|
||||
let units: Vec<u16> =
|
||||
bytes[2..].chunks_exact(2).map(|c| if le { u16::from_le_bytes([c[0], c[1]]) } else { u16::from_be_bytes([c[0], c[1]]) }).collect();
|
||||
bytes[2..].as_chunks::<2>().0.iter().map(|c| if le { u16::from_le_bytes([c[0], c[1]]) } else { u16::from_be_bytes([c[0], c[1]]) }).collect();
|
||||
String::from_utf16_lossy(&units)
|
||||
} else {
|
||||
match std::str::from_utf8(bytes) {
|
||||
|
||||
@@ -411,10 +411,10 @@ impl Dec708 {
|
||||
}
|
||||
fn put(&mut self, c: char) {
|
||||
let w = self.w();
|
||||
if w.rows.is_empty() {
|
||||
w.rows.push(String::new());
|
||||
match w.rows.last_mut() {
|
||||
Some(row) => row.push(c),
|
||||
None => w.rows.push(c.to_string()),
|
||||
}
|
||||
w.rows.last_mut().expect("row").push(c);
|
||||
}
|
||||
fn bitmap(&mut self, m: u8, f: impl Fn(&mut Win)) {
|
||||
for (i, w) in self.win.iter_mut().enumerate() {
|
||||
|
||||
@@ -251,7 +251,9 @@ pub fn parse(bytes: &[u8]) -> Result<Document> {
|
||||
if !sniff(bytes) {
|
||||
return Err(Error::NotFormat("EBU STL"));
|
||||
}
|
||||
let rate = rate_of(&bytes[3..11]).expect("sniffed");
|
||||
let Some(rate) = bytes.get(3..11).and_then(rate_of) else {
|
||||
return Err(Error::NotFormat("EBU STL"));
|
||||
};
|
||||
let gsi = &bytes[..GSI];
|
||||
// start of programme (HHMMSSFF, ASCII)
|
||||
let tcp = ascii_field(&gsi[256..264]);
|
||||
@@ -277,7 +279,7 @@ pub fn parse(bytes: &[u8]) -> Result<Document> {
|
||||
}
|
||||
}
|
||||
};
|
||||
for blk in bytes[GSI..].chunks_exact(TTI) {
|
||||
for blk in bytes[GSI..].as_chunks::<TTI>().0 {
|
||||
let sn = u16::from_le_bytes([blk[1], blk[2]]);
|
||||
let ebn = blk[3];
|
||||
let cf = blk[15];
|
||||
|
||||
@@ -103,7 +103,7 @@ fn inline(text: &str) -> String {
|
||||
rest = &after[end + 1..];
|
||||
continue;
|
||||
}
|
||||
let ch = rest.chars().next().expect("non-empty");
|
||||
let Some(ch) = rest.chars().next() else { break };
|
||||
if ch == '\n' {
|
||||
out.push_str("<br/>");
|
||||
} else {
|
||||
|
||||
@@ -15,6 +15,8 @@
|
||||
//!
|
||||
//! Layer L0: no dependencies beyond `std`, no `unsafe`, builds for `wasm32-unknown-unknown`.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
mod read;
|
||||
mod write;
|
||||
|
||||
|
||||
@@ -143,7 +143,7 @@ impl<'a> CompoundFile<'a> {
|
||||
// Mini FAT and mini stream.
|
||||
if first_minifat <= MAXREGSECT {
|
||||
let mf = cf.read_chain(first_minifat, None)?;
|
||||
cf.minifat = mf.chunks_exact(4).map(|c| u32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect();
|
||||
cf.minifat = mf.as_chunks::<4>().0.iter().map(|c| u32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect();
|
||||
}
|
||||
let root = &cf.entries[0];
|
||||
if root.start <= MAXREGSECT && root.size > 0 {
|
||||
|
||||
@@ -36,7 +36,7 @@ log = { workspace = true }
|
||||
web-time = "1"
|
||||
image = { version = "0.25", default-features = false, features = ["jpeg"] }
|
||||
# Bootstrap audio decoding (MPL-2.0, used unmodified); replaced by own decoders over time.
|
||||
symphonia = { version = "0.5.5", default-features = false, features = ["aac", "mp1", "mp2", "mp3", "flac", "alac", "pcm", "wav", "ogg", "vorbis", "isomp4"] }
|
||||
symphonia = { version = "0.5.5", default-features = false, features = ["aac", "mp1", "mp2", "mp3", "flac", "alac", "pcm", "wav", "aiff", "ogg", "vorbis", "isomp4"] }
|
||||
|
||||
[dev-dependencies]
|
||||
filmcraft-testkit = { workspace = true }
|
||||
+49
-16
@@ -33,6 +33,9 @@ pub struct GopStats {
|
||||
pub skipped: u64,
|
||||
/// Frames decoded in draft mode (reduced-resolution playback, [`VideoDecoder::set_draft`]).
|
||||
pub draft: u64,
|
||||
/// Pictures decoders output (hardware and software; [`crate::hw::hw_stats`] counts the
|
||||
/// hardware ones).
|
||||
pub frames: u64,
|
||||
}
|
||||
|
||||
impl GopStats {
|
||||
@@ -56,6 +59,12 @@ static EVICTED: AtomicU64 = AtomicU64::new(0);
|
||||
static DECODE_NS: AtomicU64 = AtomicU64::new(0);
|
||||
static SKIPPED: AtomicU64 = AtomicU64::new(0);
|
||||
static DRAFT: AtomicU64 = AtomicU64::new(0);
|
||||
static FRAMES: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
/// Count pictures a decoder call output.
|
||||
fn count_frames<T>(out: &[T]) {
|
||||
FRAMES.fetch_add(out.len() as u64, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// Run a decoder call, adding its wall time to the process-wide counter.
|
||||
fn timed<R>(f: impl FnOnce() -> R) -> R {
|
||||
@@ -76,6 +85,7 @@ pub fn gop_stats() -> GopStats {
|
||||
decode_ns: DECODE_NS.load(Ordering::Relaxed),
|
||||
skipped: SKIPPED.load(Ordering::Relaxed),
|
||||
draft: DRAFT.load(Ordering::Relaxed),
|
||||
frames: FRAMES.load(Ordering::Relaxed),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -91,6 +101,7 @@ impl std::ops::Sub for GopStats {
|
||||
decode_ns: self.decode_ns - o.decode_ns,
|
||||
skipped: self.skipped - o.skipped,
|
||||
draft: self.draft - o.draft,
|
||||
frames: self.frames - o.frames,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -172,6 +183,8 @@ pub struct GopCache {
|
||||
state: Mutex<State>,
|
||||
/// Colour signalled by the container, which wins over the bitstream's.
|
||||
explicit_color: Option<ColorInfo>,
|
||||
/// Clockwise quarter turns from the container's display matrix, applied to every frame.
|
||||
rotation: u8,
|
||||
budget: usize,
|
||||
}
|
||||
|
||||
@@ -200,21 +213,31 @@ impl GopCache {
|
||||
bytes: 0,
|
||||
}),
|
||||
explicit_color,
|
||||
rotation: 0,
|
||||
budget: 384 << 20,
|
||||
}
|
||||
}
|
||||
|
||||
/// Colour signalled by the container wins over the bitstream's (YUV frames only).
|
||||
fn apply_color(&self, f: &mut VideoFrame) {
|
||||
/// Turn every decoded frame clockwise by `quarter_turns` × 90° (the container's display
|
||||
/// rotation, e.g. portrait phone video stored landscape).
|
||||
pub fn with_rotation(mut self, quarter_turns: u8) -> Self {
|
||||
self.rotation = quarter_turns % 4;
|
||||
self
|
||||
}
|
||||
|
||||
/// Colour signalled by the container wins over the bitstream's (YUV frames only); the
|
||||
/// container's display rotation is applied.
|
||||
fn finish(&self, mut f: VideoFrame) -> VideoFrame {
|
||||
if let Some(c) = self.explicit_color
|
||||
&& !matches!(f.data, filmcraft_frame::PixelData::Rgba8(_) | filmcraft_frame::PixelData::RgbaF32(_))
|
||||
{
|
||||
f.color = c;
|
||||
}
|
||||
if self.rotation != 0 { f.rotated(self.rotation) } else { f }
|
||||
}
|
||||
|
||||
fn store(&self, st: &mut State, pts: i64, mut f: VideoFrame, draft: bool) {
|
||||
self.apply_color(&mut f);
|
||||
fn store(&self, st: &mut State, pts: i64, f: VideoFrame, draft: bool) {
|
||||
let f = self.finish(f);
|
||||
if draft {
|
||||
st.drafts.insert(pts);
|
||||
DRAFT.fetch_add(1, Ordering::Relaxed);
|
||||
@@ -228,8 +251,7 @@ impl GopCache {
|
||||
}
|
||||
// evict frames far from the most recent (keep a window around the working position)
|
||||
while st.bytes > budget && st.frames.len() > 2 {
|
||||
let first = *st.frames.keys().next().expect("non-empty");
|
||||
let last = *st.frames.keys().next_back().expect("non-empty");
|
||||
let (Some(&first), Some(&last)) = (st.frames.keys().next(), st.frames.keys().next_back()) else { break };
|
||||
let victim = if pts - first > last - pts { first } else { last };
|
||||
if let Some(v) = st.frames.remove(&victim) {
|
||||
st.drafts.remove(&victim);
|
||||
@@ -241,6 +263,7 @@ impl GopCache {
|
||||
|
||||
/// Store decoder output (in presentation order) and advance `out_max`.
|
||||
fn store_output(&self, st: &mut State, out: Vec<crate::video::DecodedFrame>) {
|
||||
count_frames(&out);
|
||||
for d in out {
|
||||
st.out_max = st.out_max.max(d.pts);
|
||||
self.store(st, d.pts, d.frame, d.draft);
|
||||
@@ -311,7 +334,7 @@ impl GopCache {
|
||||
// every sample): step back to a sample the decoder can start from.
|
||||
while key > 0 {
|
||||
let data = s.read(key)?;
|
||||
if st.decoder.as_ref().expect("decoder").is_random_access(&data) != Some(false) {
|
||||
if st.decoder.as_ref().and_then(|d| d.is_random_access(&data)) != Some(false) {
|
||||
break;
|
||||
}
|
||||
key = s.sync_before(key - 1);
|
||||
@@ -325,7 +348,10 @@ impl GopCache {
|
||||
}
|
||||
let limit = (i.max(st.next) + 64).min(n);
|
||||
let late = late_before.unwrap_or(i64::MIN).min(want_pts);
|
||||
st.decoder.as_mut().expect("decoder").set_draft(draft);
|
||||
let Some(decoder) = st.decoder.as_mut() else {
|
||||
return Err(CodecError::Decode("no video decoder".into()));
|
||||
};
|
||||
decoder.set_draft(draft);
|
||||
while st.next < limit {
|
||||
if filmcraft_media::cancel::cancelled() {
|
||||
// The decoder state stays consistent (`next`, `out_max`): a later request continues.
|
||||
@@ -333,12 +359,15 @@ impl GopCache {
|
||||
}
|
||||
let k = st.next;
|
||||
let data = s.read(k)?;
|
||||
if k != i && s.pts(k) < late && st.decoder.as_ref().expect("decoder").is_disposable(&data) {
|
||||
if k != i && s.pts(k) < late && st.decoder.as_ref().is_some_and(|d| d.is_disposable(&data)) {
|
||||
st.next += 1;
|
||||
SKIPPED.fetch_add(1, Ordering::Relaxed);
|
||||
continue;
|
||||
}
|
||||
let out = timed(|| st.decoder.as_mut().expect("decoder").decode(&data, s.pts(k)))?;
|
||||
let Some(decoder) = st.decoder.as_mut() else {
|
||||
return Err(CodecError::Decode("no video decoder".into()));
|
||||
};
|
||||
let out = timed(|| decoder.decode(&data, s.pts(k)))?;
|
||||
st.next += 1;
|
||||
DECODED.fetch_add(1, Ordering::Relaxed);
|
||||
self.store_output(&mut st, out);
|
||||
@@ -347,7 +376,7 @@ impl GopCache {
|
||||
}
|
||||
}
|
||||
if st.cached(want_pts, draft).is_none() {
|
||||
let out = timed(|| st.decoder.as_mut().expect("decoder").flush());
|
||||
let out = st.decoder.as_mut().map(|d| timed(|| d.flush())).unwrap_or_default();
|
||||
self.store_output(&mut st, out);
|
||||
st.next = usize::MAX;
|
||||
}
|
||||
@@ -361,14 +390,13 @@ impl GopCache {
|
||||
let mut d = s.make_decoder()?;
|
||||
let mut next = usize::MAX;
|
||||
let out = Self::decode_to(s, d.as_mut(), &mut next, i, want_pts, n)?;
|
||||
let mut f = out
|
||||
let f = out
|
||||
.into_iter()
|
||||
.filter(|p| p.pts <= want_pts)
|
||||
.max_by_key(|p| p.pts)
|
||||
.map(|p| p.frame)
|
||||
.ok_or_else(|| CodecError::Decode("frame not produced".into()))?;
|
||||
self.apply_color(&mut f);
|
||||
Ok(Arc::new(f))
|
||||
Ok(Arc::new(self.finish(f)))
|
||||
}
|
||||
|
||||
/// Decode with `d` (positioned at `next`) until the picture with `want_pts` (sample `i`) comes out.
|
||||
@@ -403,6 +431,7 @@ impl GopCache {
|
||||
let k = *next;
|
||||
let data = s.read(k)?;
|
||||
let pics = timed(|| d.decode(&data, s.pts(k)))?;
|
||||
count_frames(&pics);
|
||||
*next += 1;
|
||||
found |= pics.iter().any(|p| p.pts == want_pts);
|
||||
out.extend(pics);
|
||||
@@ -411,7 +440,9 @@ impl GopCache {
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
out.extend(timed(|| d.flush()));
|
||||
let pics = timed(|| d.flush());
|
||||
count_frames(&pics);
|
||||
out.extend(pics);
|
||||
*next = usize::MAX;
|
||||
}
|
||||
Ok(out)
|
||||
@@ -435,7 +466,9 @@ impl GopCache {
|
||||
if st.spare.len() < 16 {
|
||||
st.spare.push(dec);
|
||||
}
|
||||
for d in res? {
|
||||
let res = res?;
|
||||
count_frames(&res);
|
||||
for d in res {
|
||||
self.store(&mut st, d.pts, d.frame, d.draft);
|
||||
}
|
||||
st.at_or_before(want_pts, true).ok_or_else(|| CodecError::Decode("frame not produced".into()))
|
||||
|
||||
@@ -0,0 +1,350 @@
|
||||
//! Support for hardware (OS) video decoders registered by `filmcraft-platform`: the
|
||||
//! Settings ▸ Playback ▸ Hardware decoding switch, the hardware counters `perf.stats` reports, and
|
||||
//! what such a decoder needs from an `avcC` / `hvcC` sample entry to behave exactly like our
|
||||
//! software decoder for the same stream (parameter sets, cropping, colour, pixel aspect, reorder
|
||||
//! depth, random-access and disposable samples).
|
||||
//!
|
||||
//! The software decoders stay the reference: a hardware factory declines streams it cannot
|
||||
//! decode, and a hardware decoder that fails mid-stream switches to
|
||||
//! [`crate::software_video_decoder`], so registering one never makes a file undecodable.
|
||||
|
||||
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
|
||||
|
||||
use filmcraft_isobmff::{CodecConfig, SampleEntry};
|
||||
|
||||
use crate::video::{avcc_length_size, h264_disposable, hevc_disposable, hvcc_length_size_and_tid, sar_par, vui_color};
|
||||
use crate::{CodecError, Result};
|
||||
|
||||
static ENABLED: AtomicBool = AtomicBool::new(true);
|
||||
|
||||
/// Settings ▸ Playback ▸ Hardware decoding: `Auto` (true, the default) lets registered hardware
|
||||
/// decoders take streams they support; `Off` (false) makes every new decoder a software one.
|
||||
/// Decoders already created keep running: a source picks the change up when it is reopened.
|
||||
pub fn set_hardware_decoding(on: bool) {
|
||||
ENABLED.store(on, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// Whether hardware decoders may be used for new decoders ([`set_hardware_decoding`]).
|
||||
pub fn hardware_decoding() -> bool {
|
||||
ENABLED.load(Ordering::Relaxed)
|
||||
}
|
||||
|
||||
/// Process-wide hardware decoding counters (they only grow; subtract two snapshots to measure an
|
||||
/// interval). Frames decoded in software are `GopStats::frames - frames`.
|
||||
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
|
||||
pub struct HwStats {
|
||||
/// Pictures output by hardware decoders.
|
||||
pub frames: u64,
|
||||
/// Hardware decoders created (one per decoder instance; a seek reuses it).
|
||||
pub sessions: u64,
|
||||
/// Streams a hardware factory handed to the software decoder (unsupported profile, size,
|
||||
/// chroma format or bit depth, no hardware, or the switch is Off).
|
||||
pub declined: u64,
|
||||
/// Hardware decoders that failed mid-stream and continued in software.
|
||||
pub fallbacks: u64,
|
||||
}
|
||||
|
||||
static FRAMES: AtomicU64 = AtomicU64::new(0);
|
||||
static SESSIONS: AtomicU64 = AtomicU64::new(0);
|
||||
static DECLINED: AtomicU64 = AtomicU64::new(0);
|
||||
static FALLBACKS: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
/// The counters so far.
|
||||
pub fn hw_stats() -> HwStats {
|
||||
HwStats {
|
||||
frames: FRAMES.load(Ordering::Relaxed),
|
||||
sessions: SESSIONS.load(Ordering::Relaxed),
|
||||
declined: DECLINED.load(Ordering::Relaxed),
|
||||
fallbacks: FALLBACKS.load(Ordering::Relaxed),
|
||||
}
|
||||
}
|
||||
|
||||
/// A hardware decoder output `n` pictures.
|
||||
pub fn note_hw_frames(n: usize) {
|
||||
FRAMES.fetch_add(n as u64, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// A hardware decoder was created.
|
||||
pub fn note_hw_session() {
|
||||
SESSIONS.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// A hardware factory declined a stream it is responsible for.
|
||||
pub fn note_hw_declined() {
|
||||
DECLINED.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// A hardware decoder switched to software mid-stream.
|
||||
pub fn note_hw_fallback() {
|
||||
FALLBACKS.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// The two codecs whose `avcC` / `hvcC` streams hardware decoders take.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum NalCodec {
|
||||
H264,
|
||||
Hevc,
|
||||
}
|
||||
|
||||
/// What a decoder needs to know about a length-prefixed H.264 / HEVC stream, from its sample
|
||||
/// entry's configuration record and active sequence parameter set (parsed with the software
|
||||
/// decoders' own parsers, so both decoders derive the same values).
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct NalStreamInfo {
|
||||
pub codec: NalCodec,
|
||||
/// NAL unit length-prefix size of the samples (1, 2 or 4).
|
||||
pub length_size: usize,
|
||||
/// HEVC: highest TemporalId (numTemporalLayers - 1), when signalled.
|
||||
pub highest_tid: Option<u8>,
|
||||
/// The parameter-set NAL units of the record (header included, emulation prevention kept),
|
||||
/// in record order: H.264 SPS then PPS; HEVC VPS, SPS, PPS. SEI arrays are left out.
|
||||
pub parameter_sets: Vec<Vec<u8>>,
|
||||
/// Coded (decoded) picture size in luma samples.
|
||||
pub coded: (u32, u32),
|
||||
/// Output (cropped) rectangle in luma samples: x, y, width, height.
|
||||
pub crop: (u32, u32, u32, u32),
|
||||
/// 0 monochrome, 1 4:2:0, 2 4:2:2, 3 4:4:4.
|
||||
pub chroma_format_idc: u32,
|
||||
pub bit_depth_luma: u32,
|
||||
pub bit_depth_chroma: u32,
|
||||
/// Field (PAFF / MBAFF) coding (H.264 `frame_mbs_only_flag` 0).
|
||||
pub interlaced: bool,
|
||||
/// H.264 `profile_idc` / HEVC `general_profile_idc`.
|
||||
pub profile_idc: u8,
|
||||
/// Colour of every output picture, as the software decoder reports it.
|
||||
pub color: filmcraft_color::ColorInfo,
|
||||
/// Pixel aspect ratio, as the software decoder reports it.
|
||||
pub par: (u32, u32),
|
||||
/// Pictures that may precede a picture in decoding order and follow it in output order
|
||||
/// (H.264 `max_num_reorder_frames`, HEVC `sps_max_num_reorder_pics`).
|
||||
pub reorder: usize,
|
||||
}
|
||||
|
||||
/// Length-prefixed parameter-set entries (`u16` length + bytes) starting at `pos`.
|
||||
fn read_sets(rec: &[u8], pos: &mut usize, count: usize, out: &mut Vec<Vec<u8>>) -> Result<()> {
|
||||
for _ in 0..count {
|
||||
let len = rec
|
||||
.get(*pos..*pos + 2)
|
||||
.map(|b| u16::from_be_bytes([b[0], b[1]]) as usize)
|
||||
.ok_or_else(|| CodecError::Decode("configuration record truncated".into()))?;
|
||||
*pos += 2;
|
||||
let nal = rec.get(*pos..*pos + len).ok_or_else(|| CodecError::Decode("configuration record truncated".into()))?;
|
||||
*pos += len;
|
||||
if !nal.is_empty() {
|
||||
out.push(nal.to_vec());
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
impl NalStreamInfo {
|
||||
/// The stream info of an `avcC` / `hvcC` sample entry (`None` for other codecs).
|
||||
pub fn from_entry(e: &SampleEntry) -> Option<Result<Self>> {
|
||||
match &e.codec {
|
||||
CodecConfig::Avc(a) => Some(Self::from_avcc(&a.to_bytes())),
|
||||
CodecConfig::Hevc(c) => Some(Self::from_hvcc(&c.to_bytes())),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// From an `avcC` (AVCDecoderConfigurationRecord) payload.
|
||||
pub fn from_avcc(avcc: &[u8]) -> Result<Self> {
|
||||
if avcc.len() < 7 || avcc[0] != 1 {
|
||||
return Err(CodecError::Decode("bad avcC record".into()));
|
||||
}
|
||||
let mut sets = Vec::new();
|
||||
let mut pos = 6;
|
||||
read_sets(avcc, &mut pos, (avcc[5] & 0x1f) as usize, &mut sets)?;
|
||||
let nsps = sets.len();
|
||||
let npps = *avcc.get(pos).ok_or_else(|| CodecError::Decode("avcC truncated".into()))? as usize;
|
||||
pos += 1;
|
||||
read_sets(avcc, &mut pos, npps, &mut sets)?;
|
||||
let sps_nal = sets.iter().take(nsps).find(|n| n.first().is_some_and(|h| h & 0x1f == 7)).ok_or_else(|| CodecError::Decode("avcC has no SPS".into()))?;
|
||||
if sets.len() == nsps {
|
||||
return Err(CodecError::Decode("avcC has no PPS".into()));
|
||||
}
|
||||
let sps = filmcraft_h264::params::Sps::parse(&filmcraft_bitstream::unescape_rbsp(sps_nal.get(1..).unwrap_or_default()))
|
||||
.map_err(|e| CodecError::Decode(e.to_string()))?;
|
||||
let crop = sps.crop_rect();
|
||||
let vui = sps.vui.clone().unwrap_or_default();
|
||||
Ok(Self {
|
||||
codec: NalCodec::H264,
|
||||
length_size: avcc_length_size(avcc),
|
||||
highest_tid: None,
|
||||
parameter_sets: sets,
|
||||
coded: (sps.width(), sps.height()),
|
||||
crop,
|
||||
chroma_format_idc: sps.chroma_format_idc,
|
||||
bit_depth_luma: sps.bit_depth_luma,
|
||||
bit_depth_chroma: sps.bit_depth_chroma,
|
||||
interlaced: !sps.frame_mbs_only,
|
||||
profile_idc: sps.profile_idc,
|
||||
color: vui_color(crop.2, crop.3, vui.matrix_coefficients, vui.transfer_characteristics, vui.full_range),
|
||||
par: sar_par(vui.sar),
|
||||
// as the software decoder's DPB (`max_reorder`)
|
||||
reorder: sps.max_num_reorder_frames().min(sps.max_dpb_frames()),
|
||||
})
|
||||
}
|
||||
|
||||
/// From an `hvcC` (HEVCDecoderConfigurationRecord) payload.
|
||||
pub fn from_hvcc(hvcc: &[u8]) -> Result<Self> {
|
||||
if hvcc.len() < 23 || hvcc[0] != 1 {
|
||||
return Err(CodecError::Decode("bad hvcC record".into()));
|
||||
}
|
||||
let (length_size, highest_tid) = hvcc_length_size_and_tid(hvcc);
|
||||
let mut sets = Vec::new();
|
||||
let mut pos = 23;
|
||||
for _ in 0..hvcc[22] {
|
||||
let hdr = hvcc.get(pos..pos + 3).ok_or_else(|| CodecError::Decode("hvcC truncated".into()))?;
|
||||
let (kind, n) = (hdr[0] & 0x3f, u16::from_be_bytes([hdr[1], hdr[2]]) as usize);
|
||||
pos += 3;
|
||||
let mut arr = Vec::new();
|
||||
read_sets(hvcc, &mut pos, n, &mut arr)?;
|
||||
if (32..=34).contains(&kind) {
|
||||
sets.extend(arr);
|
||||
}
|
||||
}
|
||||
let nal_type = |n: &[u8]| n.first().map(|h| (h >> 1) & 0x3f);
|
||||
let sps_nal = sets.iter().find(|n| nal_type(n) == Some(33)).ok_or_else(|| CodecError::Decode("hvcC has no SPS".into()))?;
|
||||
if !sets.iter().any(|n| nal_type(n) == Some(32)) || !sets.iter().any(|n| nal_type(n) == Some(34)) {
|
||||
return Err(CodecError::Decode("hvcC lacks a VPS or PPS".into()));
|
||||
}
|
||||
let sps = filmcraft_hevc::params::Sps::parse(&filmcraft_bitstream::unescape_rbsp(sps_nal.get(2..).unwrap_or_default()))
|
||||
.map_err(|e| CodecError::Decode(e.to_string()))?;
|
||||
let crop = sps.crop_rect();
|
||||
let vui = sps.vui.clone().unwrap_or_default();
|
||||
Ok(Self {
|
||||
codec: NalCodec::Hevc,
|
||||
length_size,
|
||||
highest_tid,
|
||||
parameter_sets: sets,
|
||||
coded: (sps.width, sps.height),
|
||||
crop,
|
||||
chroma_format_idc: sps.chroma_format_idc,
|
||||
bit_depth_luma: sps.bit_depth_luma,
|
||||
bit_depth_chroma: sps.bit_depth_chroma,
|
||||
interlaced: false,
|
||||
profile_idc: sps.ptl.profile_idc,
|
||||
color: vui_color(crop.2, crop.3, vui.matrix_coefficients, vui.transfer_characteristics, vui.full_range),
|
||||
par: sar_par(vui.sar),
|
||||
reorder: sps.max_num_reorder as usize,
|
||||
})
|
||||
}
|
||||
|
||||
/// NAL unit types of a sample (stops at the first malformed length prefix).
|
||||
pub fn nal_types(&self, sample: &[u8]) -> Vec<u8> {
|
||||
self.nals(sample).into_iter().filter_map(|n| n.first().map(|&h| self.nal_type(h))).collect()
|
||||
}
|
||||
|
||||
/// The NAL units of a length-prefixed sample (stops at the first malformed length prefix).
|
||||
pub fn nals<'a>(&self, sample: &'a [u8]) -> Vec<&'a [u8]> {
|
||||
let ls = self.length_size;
|
||||
let mut out = Vec::new();
|
||||
let mut pos = 0usize;
|
||||
while let Some(pre) = sample.get(pos..pos.saturating_add(ls)) {
|
||||
if !(1..=4).contains(&ls) {
|
||||
break;
|
||||
}
|
||||
let len = pre.iter().fold(0usize, |a, &b| (a << 8) | b as usize);
|
||||
pos += ls;
|
||||
let Some(n) = sample.get(pos..pos.saturating_add(len)) else { break };
|
||||
pos += len;
|
||||
out.push(n);
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// The NAL unit type in a NAL header byte.
|
||||
pub fn nal_type(&self, header: u8) -> u8 {
|
||||
match self.codec {
|
||||
NalCodec::H264 => header & 0x1f,
|
||||
NalCodec::Hevc => (header >> 1) & 0x3f,
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether `t` is a parameter-set NAL unit type (H.264 SPS / PPS / SPS extension, HEVC VPS /
|
||||
/// SPS / PPS).
|
||||
pub fn is_parameter_set(&self, t: u8) -> bool {
|
||||
match self.codec {
|
||||
NalCodec::H264 => matches!(t, 7 | 8 | 13 | 15),
|
||||
NalCodec::Hevc => (32..=34).contains(&t),
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether decoding can restart at `sample` with no earlier state: an IDR access unit (H.264)
|
||||
/// or an IRAP one (HEVC IDR / CRA / BLA).
|
||||
pub fn is_irap(&self, sample: &[u8]) -> bool {
|
||||
self.nal_types(sample).iter().any(|&t| match self.codec {
|
||||
NalCodec::H264 => t == 5,
|
||||
NalCodec::Hevc => (16..=23).contains(&t),
|
||||
})
|
||||
}
|
||||
|
||||
/// As the software decoder's [`crate::VideoDecoder::is_random_access`] for this stream
|
||||
/// (length-prefixed samples: unknown, the container's sync flags are trusted).
|
||||
pub fn is_random_access(&self, _sample: &[u8]) -> Option<bool> {
|
||||
None
|
||||
}
|
||||
|
||||
/// As the software decoder's [`crate::VideoDecoder::is_disposable`] for this stream.
|
||||
pub fn is_disposable(&self, sample: &[u8]) -> bool {
|
||||
match self.codec {
|
||||
NalCodec::H264 => h264_disposable(sample, self.length_size),
|
||||
NalCodec::Hevc => hevc_disposable(sample, self.length_size, self.highest_tid),
|
||||
}
|
||||
}
|
||||
|
||||
/// Output picture size (cropped).
|
||||
pub fn size(&self) -> (u32, u32) {
|
||||
(self.crop.2, self.crop.3)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn switch_defaults_to_auto() {
|
||||
// other tests in this process may flip it; only check the round trip
|
||||
let was = hardware_decoding();
|
||||
set_hardware_decoding(false);
|
||||
assert!(!hardware_decoding());
|
||||
set_hardware_decoding(true);
|
||||
assert!(hardware_decoding());
|
||||
set_hardware_decoding(was);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hostile_records_are_errors() {
|
||||
for rec in [&[][..], &[1, 2, 3], &[1, 0x64, 0, 0x1f, 0xff, 0xe1, 0, 9, 0x67], &[0; 30]] {
|
||||
assert!(NalStreamInfo::from_avcc(rec).is_err());
|
||||
assert!(NalStreamInfo::from_hvcc(rec).is_err());
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn nal_splitting_is_bounded() {
|
||||
let info = NalStreamInfo {
|
||||
codec: NalCodec::Hevc,
|
||||
length_size: 4,
|
||||
highest_tid: None,
|
||||
parameter_sets: Vec::new(),
|
||||
coded: (16, 16),
|
||||
crop: (0, 0, 16, 16),
|
||||
chroma_format_idc: 1,
|
||||
bit_depth_luma: 8,
|
||||
bit_depth_chroma: 8,
|
||||
interlaced: false,
|
||||
profile_idc: 1,
|
||||
color: filmcraft_color::ColorInfo::REC709,
|
||||
par: (1, 1),
|
||||
reorder: 0,
|
||||
};
|
||||
// CRA (21) then a length that runs past the end
|
||||
let s = [0, 0, 0, 2, 21 << 1, 1, 0xff, 0xff, 0xff, 0xff, 1];
|
||||
assert_eq!(info.nal_types(&s), vec![21]);
|
||||
assert!(info.is_irap(&s));
|
||||
assert!(!info.is_irap(&[0, 0, 0, 2, 1 << 1, 1]));
|
||||
}
|
||||
}
|
||||
+27
-12
@@ -16,8 +16,11 @@
|
||||
//! - [`AudioFileSource`]: standalone compressed audio files (MP3, FLAC, AIFF, …).
|
||||
//! - [`openers`]: the openers to register with the engine's media pool.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
pub mod audio;
|
||||
pub mod gop;
|
||||
pub mod hw;
|
||||
pub mod mkv;
|
||||
pub mod mp4;
|
||||
pub mod mpeg;
|
||||
@@ -61,20 +64,21 @@ impl From<CodecError> for filmcraft_media::MediaError {
|
||||
}
|
||||
}
|
||||
|
||||
/// Our own (pure-Rust) decoders, in the order they are tried.
|
||||
const BUILTIN_FACTORIES: [VideoDecoderFactory; 8] = [
|
||||
video::h264_factory,
|
||||
video::hevc_factory,
|
||||
video::vp9_factory,
|
||||
video::av1_factory,
|
||||
video::prores_factory,
|
||||
video::dnx_factory,
|
||||
video::mjpeg_factory,
|
||||
video::mpeg2_factory,
|
||||
];
|
||||
|
||||
fn factories() -> &'static RwLock<Vec<VideoDecoderFactory>> {
|
||||
static F: std::sync::OnceLock<RwLock<Vec<VideoDecoderFactory>>> = std::sync::OnceLock::new();
|
||||
F.get_or_init(|| {
|
||||
RwLock::new(vec![
|
||||
video::h264_factory,
|
||||
video::hevc_factory,
|
||||
video::vp9_factory,
|
||||
video::av1_factory,
|
||||
video::prores_factory,
|
||||
video::dnx_factory,
|
||||
video::mjpeg_factory,
|
||||
video::mpeg2_factory,
|
||||
])
|
||||
})
|
||||
F.get_or_init(|| RwLock::new(BUILTIN_FACTORIES.to_vec()))
|
||||
}
|
||||
|
||||
/// Register a video decoder factory (tried before previously registered ones).
|
||||
@@ -96,6 +100,17 @@ pub fn make_video_decoder(entry: &filmcraft_isobmff::SampleEntry) -> Result<Box<
|
||||
Err(CodecError::Unsupported(format!("no decoder for {} video", entry.codec.name())))
|
||||
}
|
||||
|
||||
/// Create one of our own (software) decoders for a sample entry, skipping registered factories:
|
||||
/// what a hardware decoder falls back to when it fails mid-stream.
|
||||
pub fn software_video_decoder(entry: &filmcraft_isobmff::SampleEntry) -> Result<Box<dyn VideoDecoder>> {
|
||||
for f in BUILTIN_FACTORIES {
|
||||
if let Some(r) = f(entry) {
|
||||
return r;
|
||||
}
|
||||
}
|
||||
Err(CodecError::Unsupported(format!("no decoder for {} video", entry.codec.name())))
|
||||
}
|
||||
|
||||
/// Openers for the engine's media pool (MP4/MOV, Matroska/WebM, MXF, Ogg Opus/Vorbis, MPEG TS/PS and
|
||||
/// MPEG-1/2 video elementary streams, standalone audio).
|
||||
pub fn openers() -> Vec<filmcraft_media::Opener> {
|
||||
|
||||
+25
-186
@@ -4,10 +4,6 @@
|
||||
//! containers (H.264, HEVC, VP9, AV1, ProRes, MJPEG). Audio: AAC and Opus
|
||||
//! via our decoders, PCM directly, MP3/FLAC/Vorbis via the bootstrap decoders.
|
||||
//!
|
||||
//! VP9 alpha (WebM `AlphaMode` 1): each block's `BlockAdditional` (`BlockAddID` 1) is a second VP9
|
||||
//! bitstream whose luma plane is the straight alpha channel. A second decoder instance decodes it
|
||||
//! alongside the colour stream ([`AlphaDecoder`]) and its luma becomes the frame's alpha plane.
|
||||
//!
|
||||
//! Opus (`A_OPUS`): `CodecPrivate` is the `OpusHead`; output is always 48 kHz. The demuxer
|
||||
//! subtracts `CodecDelay` from timestamps, so the pre-skip samples land before zero and are never
|
||||
//! read; random access decodes `SeekPreRoll` (at least [`crate::audio::OPUS_PRE_ROLL`]) of preceding
|
||||
@@ -47,8 +43,6 @@ pub struct MkvSource {
|
||||
atrack: Option<usize>,
|
||||
/// Video codec as an ISO-BMFF sample entry (for the decoder factories).
|
||||
ventry: Option<SampleEntry>,
|
||||
/// VP9 with `AlphaMode` 1: decode the BlockAdditions alpha stream too.
|
||||
valpha: bool,
|
||||
video: GopCache,
|
||||
audio: Mutex<AudioState>,
|
||||
/// Audio packet start positions in source sample frames.
|
||||
@@ -160,6 +154,20 @@ fn sample_entry(c: &Codec, private: &[u8], v: Option<&filmcraft_matroska::VideoI
|
||||
})
|
||||
}
|
||||
|
||||
/// HDR static metadata from the Matroska `Colour` element.
|
||||
fn hdr_metadata(c: &filmcraft_matroska::Colour) -> Option<filmcraft_color::HdrMetadata> {
|
||||
if c.mastering.is_none() && c.max_cll.is_none() && c.max_fall.is_none() {
|
||||
return None;
|
||||
}
|
||||
let nz = |v: f64| (v > 0.0).then_some(v as f32);
|
||||
Some(filmcraft_color::HdrMetadata {
|
||||
mastering_max_nits: c.mastering.as_ref().and_then(|m| nz(m.luminance_max)),
|
||||
mastering_min_nits: c.mastering.as_ref().map(|m| m.luminance_min as f32),
|
||||
max_cll: c.max_cll.and_then(|v| nz(v as f64)),
|
||||
max_fall: c.max_fall.and_then(|v| nz(v as f64)),
|
||||
})
|
||||
}
|
||||
|
||||
fn color_of(v: &filmcraft_matroska::VideoInfo, w: u32, h: u32) -> (ColorInfo, bool) {
|
||||
let mut c = ColorInfo { matrix: filmcraft_frame::default_matrix(w, h), transfer: Transfer::Bt709, primaries: Primaries::Bt709, range: Range::Limited };
|
||||
let Some(col) = &v.colour else { return (c, false) };
|
||||
@@ -226,7 +234,8 @@ impl MkvSource {
|
||||
}
|
||||
let mut explicit_color = None;
|
||||
let mut ventry = None;
|
||||
let mut valpha = false;
|
||||
// display rotation from the track's Projection (portrait phone video is stored landscape)
|
||||
let rotation = vtrack.and_then(|i| file.tracks[i].video.as_ref()).and_then(|v| v.display_rotation()).unwrap_or(0);
|
||||
let video = vtrack.map(|i| {
|
||||
let t = &file.tracks[i];
|
||||
let v = t.video.clone().unwrap_or_default();
|
||||
@@ -246,19 +255,20 @@ impl MkvSource {
|
||||
explicit_color = Some(color);
|
||||
}
|
||||
ventry = sample_entry(&t.codec, &t.codec_private, t.video.as_ref(), w as u16, h as u16);
|
||||
valpha = v.alpha_mode == 1 && matches!(t.codec, Codec::Vp9 { .. });
|
||||
let secs = file.duration_ns().unwrap_or(0) as f64 / 1e9;
|
||||
let bitrate = (secs > 0.0).then(|| (t.samples.iter().map(|s| s.size as u64).sum::<u64>() as f64 * 8.0 / secs) as u64);
|
||||
let ((w, h), par) = if rotation % 2 == 1 { ((h, w), (v.pixel_aspect().1, v.pixel_aspect().0)) } else { ((w, h), v.pixel_aspect()) };
|
||||
VideoStreamInfo {
|
||||
width: w,
|
||||
height: h,
|
||||
frame_rate: rate,
|
||||
par: v.pixel_aspect(),
|
||||
par,
|
||||
codec: codec_label(&t.codec),
|
||||
pixel_format: String::new(),
|
||||
color,
|
||||
has_alpha: v.alpha_mode != 0,
|
||||
bitrate,
|
||||
hdr: v.colour.as_ref().and_then(hdr_metadata),
|
||||
}
|
||||
});
|
||||
let audio = atrack.map(|i| {
|
||||
@@ -325,8 +335,7 @@ impl MkvSource {
|
||||
vtrack,
|
||||
atrack,
|
||||
ventry,
|
||||
valpha,
|
||||
video: GopCache::new(explicit_color),
|
||||
video: GopCache::new(explicit_color).with_rotation(rotation),
|
||||
audio: Mutex::new(AudioState { decoder: None, packets: HashMap::new(), order: Vec::new(), last_decoded: None }),
|
||||
audio_starts,
|
||||
audio_preroll,
|
||||
@@ -360,9 +369,9 @@ impl MkvSource {
|
||||
if let Some(p) = st.packets.get(&i) {
|
||||
return Ok(p.clone());
|
||||
}
|
||||
let ti = self.atrack.expect("audio");
|
||||
let ti = self.atrack.ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
|
||||
let track = &self.file.tracks[ti];
|
||||
let ainfo = self.info.audio.as_ref().expect("audio info");
|
||||
let ainfo = self.info.audio.as_ref().ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
|
||||
let data = self.read(ti, i)?;
|
||||
let decoded = match &track.codec {
|
||||
Codec::Pcm { float, big_endian, bits } => {
|
||||
@@ -383,7 +392,7 @@ impl MkvSource {
|
||||
// Non-sequential access: reset and prime with the preceding packets (codec pre-roll:
|
||||
// one packet, or `SeekPreRoll` worth for Opus).
|
||||
if st.last_decoded.is_none_or(|l| l + 1 != i) {
|
||||
let d = st.decoder.as_mut().expect("decoder");
|
||||
let d = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?;
|
||||
d.reset();
|
||||
let from = if self.audio_preroll > 0 {
|
||||
self.audio_starts.partition_point(|&x| x <= self.audio_starts[i] - self.audio_preroll).saturating_sub(1)
|
||||
@@ -396,7 +405,7 @@ impl MkvSource {
|
||||
}
|
||||
}
|
||||
}
|
||||
let r = st.decoder.as_mut().expect("decoder").decode(&data, self.audio_starts[i].max(0) as u64);
|
||||
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, self.audio_starts[i].max(0) as u64);
|
||||
st.last_decoded = Some(i);
|
||||
r.unwrap_or_default()
|
||||
}
|
||||
@@ -432,186 +441,16 @@ impl VideoSamples for MkvVideo<'_> {
|
||||
self.src.file.tracks[self.track].sample_at_pts(t)
|
||||
}
|
||||
fn read(&self, i: usize) -> crate::Result<Vec<u8>> {
|
||||
let main = self.src.read(self.track, i)?;
|
||||
if !self.src.valpha {
|
||||
return Ok(main);
|
||||
}
|
||||
let alpha = self.src.file.read_block_additional(&self.src.bytes, self.track, i).map_err(|e| CodecError::Container(e.to_string()))?;
|
||||
Ok(pack_alpha(&main, alpha.as_deref()))
|
||||
self.src.read(self.track, i)
|
||||
}
|
||||
fn make_decoder(&self) -> crate::Result<Box<dyn VideoDecoder>> {
|
||||
match &self.src.ventry {
|
||||
Some(e) if self.src.valpha => Ok(Box::new(AlphaDecoder::new(make_video_decoder(e)?, make_video_decoder(e)?))),
|
||||
Some(e) => make_video_decoder(e),
|
||||
None => Err(CodecError::Unsupported(format!("no decoder for {} video", codec_label(&self.src.file.tracks[self.track].codec)))),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A colour sample and its alpha sample as one [`AlphaDecoder`] sample: the colour sample's length
|
||||
/// (u32 LE), the colour sample, then the alpha sample (empty: the block has no alpha).
|
||||
fn pack_alpha(main: &[u8], alpha: Option<&[u8]>) -> Vec<u8> {
|
||||
let alpha = alpha.unwrap_or_default();
|
||||
let mut v = Vec::with_capacity(4 + main.len() + alpha.len());
|
||||
v.extend_from_slice(&(main.len() as u32).to_le_bytes());
|
||||
v.extend_from_slice(main);
|
||||
v.extend_from_slice(alpha);
|
||||
v
|
||||
}
|
||||
|
||||
/// The (colour, alpha) halves of a [`pack_alpha`] sample (alpha `None` when absent).
|
||||
fn unpack_alpha(sample: &[u8]) -> (&[u8], Option<&[u8]>) {
|
||||
let Some(len) = sample.get(..4).map(|b| u32::from_le_bytes([b[0], b[1], b[2], b[3]]) as usize) else { return (sample, None) };
|
||||
let body = &sample[4..];
|
||||
if len > body.len() {
|
||||
return (body, None);
|
||||
}
|
||||
let (main, alpha) = body.split_at(len);
|
||||
(main, (!alpha.is_empty()).then_some(alpha))
|
||||
}
|
||||
|
||||
/// A decoded alpha stream's luma plane, at its own bit depth.
|
||||
enum AlphaPlane {
|
||||
U8(Arc<Vec<u8>>),
|
||||
U16(Arc<Vec<u16>>, u32),
|
||||
}
|
||||
|
||||
impl AlphaPlane {
|
||||
fn of(f: &VideoFrame) -> Option<(u32, u32, AlphaPlane)> {
|
||||
use filmcraft_frame::PixelData;
|
||||
let p = match &f.data {
|
||||
PixelData::Yuv8 { planes, .. } => AlphaPlane::U8(planes[0].clone()),
|
||||
PixelData::Yuv16 { planes, bits, .. } => AlphaPlane::U16(planes[0].clone(), *bits),
|
||||
_ => return None,
|
||||
};
|
||||
Some((f.width, f.height, p))
|
||||
}
|
||||
|
||||
/// As 8-bit values.
|
||||
fn to_u8(&self) -> Arc<Vec<u8>> {
|
||||
match self {
|
||||
AlphaPlane::U8(a) => a.clone(),
|
||||
AlphaPlane::U16(a, bits) => {
|
||||
let max = ((1u32 << bits) - 1).max(1);
|
||||
Arc::new(a.iter().map(|&v| ((v as u32 * 255 + max / 2) / max).min(255) as u8).collect())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// As `bits`-bit values.
|
||||
fn to_u16(&self, bits: u32) -> Arc<Vec<u16>> {
|
||||
let to = (1u32 << bits) - 1;
|
||||
match self {
|
||||
AlphaPlane::U16(a, b) if *b == bits => a.clone(),
|
||||
AlphaPlane::U16(a, b) => {
|
||||
let from = ((1u32 << b) - 1).max(1);
|
||||
Arc::new(a.iter().map(|&v| ((v as u32 * to + from / 2) / from).min(to) as u16).collect())
|
||||
}
|
||||
AlphaPlane::U8(a) => Arc::new(a.iter().map(|&v| ((v as u32 * to + 127) / 255) as u16).collect()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// VP9 with a BlockAdditions alpha stream: one decoder for the colour samples, a second for the
|
||||
/// alpha samples (same codec), fed [`pack_alpha`] samples. Each output picture gets the alpha
|
||||
/// picture of the same pts as its (straight) alpha plane; a picture without one stays opaque.
|
||||
struct AlphaDecoder {
|
||||
main: Box<dyn VideoDecoder>,
|
||||
alpha: Box<dyn VideoDecoder>,
|
||||
/// Decoded alpha planes by pts, waiting for their colour picture.
|
||||
planes: std::collections::BTreeMap<i64, (u32, u32, AlphaPlane)>,
|
||||
}
|
||||
|
||||
impl AlphaDecoder {
|
||||
fn new(main: Box<dyn VideoDecoder>, alpha: Box<dyn VideoDecoder>) -> Self {
|
||||
Self { main, alpha, planes: Default::default() }
|
||||
}
|
||||
|
||||
fn take_alpha(&mut self, out: Vec<crate::video::DecodedFrame>) {
|
||||
for d in out {
|
||||
if let Some(p) = AlphaPlane::of(&d.frame) {
|
||||
self.planes.insert(d.pts, p);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Attach the alpha plane decoded for each picture's pts (older unmatched planes are dropped).
|
||||
fn attach(&mut self, out: Vec<crate::video::DecodedFrame>) -> Vec<crate::video::DecodedFrame> {
|
||||
use filmcraft_frame::PixelData;
|
||||
out.into_iter()
|
||||
.map(|mut d| {
|
||||
let plane = self.planes.remove(&d.pts);
|
||||
self.planes = self.planes.split_off(&d.pts);
|
||||
let Some((w, h, a)) = plane.filter(|(w, h, _)| (*w, *h) == (d.frame.width, d.frame.height)) else { return d };
|
||||
let n = w as usize * h as usize;
|
||||
match &mut d.frame.data {
|
||||
PixelData::Yuv8 { alpha, .. } => *alpha = Some(a.to_u8()),
|
||||
PixelData::Yuv16 { alpha, bits, .. } => *alpha = Some(a.to_u16(*bits)),
|
||||
PixelData::Rgba8(px) => {
|
||||
let a = a.to_u8();
|
||||
let px = Arc::make_mut(px);
|
||||
for (i, v) in a.iter().take(n).enumerate() {
|
||||
px[i * 4 + 3] = *v;
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
d
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
|
||||
impl VideoDecoder for AlphaDecoder {
|
||||
fn decode(&mut self, sample: &[u8], pts: i64) -> crate::Result<Vec<crate::video::DecodedFrame>> {
|
||||
let (main, alpha) = unpack_alpha(sample);
|
||||
if let Some(a) = alpha {
|
||||
// a damaged alpha stream leaves the pictures opaque rather than failing them
|
||||
if let Ok(out) = self.alpha.decode(a, pts) {
|
||||
self.take_alpha(out);
|
||||
}
|
||||
}
|
||||
let out = self.main.decode(main, pts)?;
|
||||
Ok(self.attach(out))
|
||||
}
|
||||
fn flush(&mut self) -> Vec<crate::video::DecodedFrame> {
|
||||
let a = self.alpha.flush();
|
||||
self.take_alpha(a);
|
||||
let out = self.main.flush();
|
||||
let out = self.attach(out);
|
||||
self.planes.clear();
|
||||
out
|
||||
}
|
||||
fn reset(&mut self) {
|
||||
self.main.reset();
|
||||
self.alpha.reset();
|
||||
self.planes.clear();
|
||||
}
|
||||
fn name(&self) -> &str {
|
||||
"FilmCraft VP9 + alpha"
|
||||
}
|
||||
fn intra_only(&self) -> bool {
|
||||
self.main.intra_only() && self.alpha.intra_only()
|
||||
}
|
||||
/// Decoding can start only where both streams can.
|
||||
fn is_random_access(&self, sample: &[u8]) -> Option<bool> {
|
||||
let (main, alpha) = unpack_alpha(sample);
|
||||
let m = self.main.is_random_access(main);
|
||||
match alpha.map(|a| self.alpha.is_random_access(a)) {
|
||||
Some(Some(false)) => Some(false),
|
||||
_ => m,
|
||||
}
|
||||
}
|
||||
fn is_disposable(&self, sample: &[u8]) -> bool {
|
||||
let (main, alpha) = unpack_alpha(sample);
|
||||
self.main.is_disposable(main) && alpha.is_none_or(|a| self.alpha.is_disposable(a))
|
||||
}
|
||||
fn set_draft(&mut self, on: bool) {
|
||||
self.main.set_draft(on);
|
||||
self.alpha.set_draft(on);
|
||||
}
|
||||
}
|
||||
|
||||
impl MediaSource for MkvSource {
|
||||
fn info(&self) -> &MediaInfo {
|
||||
&self.info
|
||||
|
||||
+147
-7
@@ -46,6 +46,20 @@ pub fn sniff(b: &[u8]) -> bool {
|
||||
b.len() >= 12 && matches!(&b[4..8], b"ftyp" | b"moov" | b"mdat" | b"wide" | b"free" | b"skip")
|
||||
}
|
||||
|
||||
/// HDR static metadata from the sample entry's `mdcv` / `clli` boxes (0 = unknown).
|
||||
fn hdr_metadata(md: Option<&filmcraft_isobmff::MasteringDisplay>, cll: Option<(u16, u16)>) -> Option<filmcraft_color::HdrMetadata> {
|
||||
if md.is_none() && cll.is_none() {
|
||||
return None;
|
||||
}
|
||||
let nz = |v: f32| (v > 0.0).then_some(v);
|
||||
Some(filmcraft_color::HdrMetadata {
|
||||
mastering_max_nits: md.and_then(|m| nz(m.max_nits() as f32)),
|
||||
mastering_min_nits: md.map(|m| m.min_luminance as f32 / 10_000.0),
|
||||
max_cll: cll.and_then(|c| nz(c.0 as f32)),
|
||||
max_fall: cll.and_then(|c| nz(c.1 as f32)),
|
||||
})
|
||||
}
|
||||
|
||||
fn color_from(entry: &filmcraft_isobmff::SampleEntry, w: u32, h: u32) -> ColorInfo {
|
||||
let mut c = ColorInfo { matrix: filmcraft_frame::default_matrix(w, h), transfer: Transfer::Bt709, primaries: Primaries::Bt709, range: Range::Limited };
|
||||
// VP9 carries its colour description in vpcC (used when there is no colr box).
|
||||
@@ -93,13 +107,18 @@ impl Mp4Source {
|
||||
pub fn open_reader(name: &str, reader: filmcraft_media::SharedReader) -> crate::Result<Self> {
|
||||
let bytes = crate::Src(reader);
|
||||
let file = filmcraft_isobmff::open(&bytes).map_err(|e| CodecError::Container(e.to_string()))?;
|
||||
let vtrack = file.tracks.iter().position(|t| t.kind == TrackKind::Video && !t.samples.is_empty());
|
||||
let atrack = file.tracks.iter().position(|t| t.kind == TrackKind::Audio && !t.samples.is_empty());
|
||||
// A track needs samples, a sample description and a timescale to be playable (damaged
|
||||
// files can lack them: indexing `entries[0]` or dividing by the timescale used to panic).
|
||||
let playable = |t: &&filmcraft_isobmff::Track, kind| t.kind == kind && !t.samples.is_empty() && !t.entries.is_empty() && t.timescale > 0;
|
||||
let vtrack = file.tracks.iter().position(|t| playable(&t, TrackKind::Video));
|
||||
let atrack = file.tracks.iter().position(|t| playable(&t, TrackKind::Audio));
|
||||
if vtrack.is_none() && atrack.is_none() {
|
||||
return Err(CodecError::Unsupported("no playable tracks".into()));
|
||||
}
|
||||
let mut color = ColorInfo::REC709;
|
||||
let mut explicit_color = None;
|
||||
// display rotation from the track matrix (portrait phone video is stored landscape)
|
||||
let rotation = vtrack.and_then(|i| file.tracks[i].display_rotation()).unwrap_or(0);
|
||||
let video = vtrack.map(|i| {
|
||||
let t = &file.tracks[i];
|
||||
let entry = &t.entries[0];
|
||||
@@ -110,6 +129,8 @@ impl Mp4Source {
|
||||
durs.sort_unstable();
|
||||
let d = durs.get(durs.len() / 2).copied().unwrap_or(1).max(1);
|
||||
let rate = FrameRate::from_f64(t.timescale as f64 / d as f64);
|
||||
// A rate that rounds to zero (one sample per huge duration) can't be divided by.
|
||||
let rate = if rate.num > 0 && rate.den > 0 { rate } else { FrameRate::default() };
|
||||
color = color_from(entry, w, h);
|
||||
if entry
|
||||
.video
|
||||
@@ -129,7 +150,12 @@ impl Mp4Source {
|
||||
color,
|
||||
has_alpha: matches!(&entry.codec, CodecConfig::ProRes { fourcc } if fourcc.0 == *b"ap4h" || fourcc.0 == *b"ap4x"),
|
||||
bitrate,
|
||||
hdr: entry.video.as_ref().and_then(|v| hdr_metadata(v.mastering_display.as_ref(), v.content_light)),
|
||||
};
|
||||
if rotation % 2 == 1 {
|
||||
(info.width, info.height) = (info.height, info.width);
|
||||
info.par = (info.par.1, info.par.0);
|
||||
}
|
||||
if matches!(entry.codec, CodecConfig::Dnx { .. }) {
|
||||
// the sample entry doesn't say which VC-3 compression ID it is: read the first frame header
|
||||
let s0 = &t.samples[0];
|
||||
@@ -220,7 +246,7 @@ impl Mp4Source {
|
||||
file,
|
||||
vtrack,
|
||||
atrack,
|
||||
video: GopCache::new(explicit_color),
|
||||
video: GopCache::new(explicit_color).with_rotation(rotation),
|
||||
audio: Mutex::new(AudioState { decoder: None, packets: HashMap::new(), order: Vec::new(), last_decoded: None }),
|
||||
audio_starts,
|
||||
audio_offset,
|
||||
@@ -243,9 +269,9 @@ impl Mp4Source {
|
||||
if let Some(p) = st.packets.get(&i) {
|
||||
return Ok(p.clone());
|
||||
}
|
||||
let ti = self.atrack.expect("audio");
|
||||
let ti = self.atrack.ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
|
||||
let track = &self.file.tracks[ti];
|
||||
let entry = &track.entries[0];
|
||||
let entry = track.entries.first().ok_or_else(|| CodecError::Container("audio track without a sample description".into()))?;
|
||||
let data = self.read(ti, i)?;
|
||||
let decoded = match &entry.codec {
|
||||
CodecConfig::Pcm(p) => decode_pcm(&data, p),
|
||||
@@ -256,7 +282,7 @@ impl Mp4Source {
|
||||
// Non-sequential access: reset and prime with the preceding packets (codec pre-roll:
|
||||
// one packet, or `OPUS_PRE_ROLL` worth for Opus).
|
||||
if st.last_decoded.is_none_or(|l| l + 1 != i) {
|
||||
let d = st.decoder.as_mut().expect("decoder");
|
||||
let d = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?;
|
||||
d.reset();
|
||||
let from = if self.audio_preroll > 0 {
|
||||
self.audio_starts.partition_point(|&x| x <= self.audio_starts[i] - self.audio_preroll).saturating_sub(1)
|
||||
@@ -269,7 +295,7 @@ impl Mp4Source {
|
||||
}
|
||||
}
|
||||
}
|
||||
let r = st.decoder.as_mut().expect("decoder").decode(&data, self.audio_starts[i].max(0) as u64);
|
||||
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, self.audio_starts[i].max(0) as u64);
|
||||
st.last_decoded = Some(i);
|
||||
match r {
|
||||
Ok(v) => v,
|
||||
@@ -462,3 +488,117 @@ pub fn reader_opener(name: &str, head: &[u8], reader: &filmcraft_media::SharedRe
|
||||
}
|
||||
Some(Mp4Source::open_reader(name, reader.clone()).map(|s| Arc::new(s) as SharedSource).map_err(Into::into))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use filmcraft_isobmff::{Brand, FourCc, Mp4Writer, SampleEntry, TrackConfig, WriteSample, WriterOptions};
|
||||
use filmcraft_media::FrameRequest;
|
||||
|
||||
/// A one-frame 64×32 ProRes MOV (bright left half, dark right half) whose `tkhd` matrix is
|
||||
/// replaced by `matrix` — built here, no media files.
|
||||
fn rotated_mov(matrix: [i32; 9]) -> Arc<[u8]> {
|
||||
let (w, h) = (64u32, 32u32);
|
||||
let mut fr = filmcraft_prores::Frame::new(w, h, filmcraft_prores::ChromaFormat::Yuv422, 10, false);
|
||||
for (i, y) in fr.y.iter_mut().enumerate() {
|
||||
*y = if (i as u32 % w) < w / 2 { 800 } else { 100 };
|
||||
}
|
||||
let data = filmcraft_prores::Encoder::new(filmcraft_prores::Profile::Hq, w, h).encode(&fr).expect("encode");
|
||||
let mut mux = Mp4Writer::new(std::io::Cursor::new(Vec::new()), WriterOptions::new(Brand::Mov)).expect("writer");
|
||||
let t = mux.add_track(TrackConfig::new(SampleEntry::prores(FourCc(*b"apch"), w as u16, h as u16), 25)).expect("track");
|
||||
mux.write_sample(t, WriteSample { data: &data, duration: 1, composition_offset: 0, is_sync: true }).expect("sample");
|
||||
let mut b = mux.finish().expect("finish").into_inner();
|
||||
// tkhd (ISO/IEC 14496-12 §8.3.2): version/flags, then 20 (v0) or 32 (v1) bytes of
|
||||
// times/id/duration, 8 reserved, layer, alternate_group, volume, reserved, matrix
|
||||
let k = b.windows(4).position(|x| x == b"tkhd").expect("tkhd") + 4;
|
||||
let m = k + 4 + if b[k] == 1 { 32 } else { 20 } + 16;
|
||||
for (j, v) in matrix.iter().enumerate() {
|
||||
b[m + j * 4..m + j * 4 + 4].copy_from_slice(&v.to_be_bytes());
|
||||
}
|
||||
b.into()
|
||||
}
|
||||
|
||||
/// Overwrite the big-endian u32 `skip` bytes after the first `fourcc` box type.
|
||||
fn patch_u32(b: &mut [u8], fourcc: &[u8; 4], skip: usize, v: u32) {
|
||||
let k = b.windows(4).position(|x| x == fourcc).expect("box") + 4 + skip;
|
||||
b[k..k + 4].copy_from_slice(&v.to_be_bytes());
|
||||
}
|
||||
|
||||
/// Damaged MOVs found by mutation fuzzing panicked on import: a track with timescale 0
|
||||
/// ("attempt to divide by zero" in the duration) and one whose sample description list is
|
||||
/// empty ("index out of bounds" on `entries[0]`). Both must open as an error, or as media
|
||||
/// that answers requests without panicking.
|
||||
#[test]
|
||||
fn damaged_track_headers_never_panic() {
|
||||
let identity = [0x10000, 0, 0, 0, 0x10000, 0, 0, 0, 0x4000_0000];
|
||||
// mdhd v0: version/flags, creation, modification, then timescale
|
||||
let mut zero_timescale = rotated_mov(identity).to_vec();
|
||||
patch_u32(&mut zero_timescale, b"mdhd", 12, 0);
|
||||
// stsd: version/flags, then entry_count
|
||||
let mut no_entries = rotated_mov(identity).to_vec();
|
||||
patch_u32(&mut no_entries, b"stsd", 4, 0);
|
||||
for (name, b) in [("ts0.mov", zero_timescale), ("stsd0.mov", no_entries)] {
|
||||
if let Ok(s) = Mp4Source::open(name, b.into()) {
|
||||
let _ = s.video_frame(FrameRequest::full(Tick::ZERO));
|
||||
let _ = s.audio(0, 1024, 48_000);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Mean luma of the top and bottom halves of the decoded frame, and its size.
|
||||
fn halves(src: &Mp4Source) -> (u32, u32, u8, u8) {
|
||||
let f = src.video_frame(FrameRequest::full(Tick::ZERO)).expect("frame");
|
||||
let l = f.luma8();
|
||||
let half = l.len() / 2;
|
||||
let mean = |s: &[u8]| (s.iter().map(|&v| v as u32).sum::<u32>() / s.len() as u32) as u8;
|
||||
(f.width, f.height, mean(&l[..half]), mean(&l[half..]))
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn display_matrix_rotates_frames_and_reported_size() {
|
||||
const ONE: i32 = 0x10000;
|
||||
const W: i32 = 0x4000_0000;
|
||||
// identity: landscape 64x32, top and bottom alike
|
||||
let s = Mp4Source::open("id.mov", rotated_mov([ONE, 0, 0, 0, ONE, 0, 0, 0, W])).expect("open");
|
||||
let v = s.info().video.as_ref().expect("video");
|
||||
assert_eq!((v.width, v.height), (64, 32));
|
||||
let (w, h, top, bottom) = halves(&s);
|
||||
assert_eq!((w, h), (64, 32));
|
||||
assert!(top.abs_diff(bottom) < 4);
|
||||
|
||||
// 90° clockwise (what an iPhone writes for portrait): the left half ends up on top
|
||||
let s = Mp4Source::open("cw.mov", rotated_mov([0, ONE, 0, -ONE, 0, 0, 32 * ONE, 0, W])).expect("open");
|
||||
let v = s.info().video.as_ref().expect("video");
|
||||
assert_eq!((v.width, v.height), (32, 64));
|
||||
let (w, h, top, bottom) = halves(&s);
|
||||
assert_eq!((w, h), (32, 64));
|
||||
assert!(top > bottom + 100, "top {top} bottom {bottom}");
|
||||
|
||||
// 270° clockwise: the left half ends up at the bottom
|
||||
let s = Mp4Source::open("ccw.mov", rotated_mov([0, -ONE, 0, ONE, 0, 0, 0, 64 * ONE, W])).expect("open");
|
||||
let (w, h, top, bottom) = halves(&s);
|
||||
assert_eq!((w, h), (32, 64));
|
||||
assert!(bottom > top + 100, "top {top} bottom {bottom}");
|
||||
|
||||
// 180°: still landscape
|
||||
let s = Mp4Source::open("180.mov", rotated_mov([-ONE, 0, 0, 0, -ONE, 0, 64 * ONE, 32 * ONE, W])).expect("open");
|
||||
assert_eq!(s.info().video.as_ref().map(|v| (v.width, v.height)), Some((64, 32)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mirrored_or_scaled_matrices_are_not_mistaken_for_rotations() {
|
||||
const ONE: i32 = 0x10000;
|
||||
const W: i32 = 0x4000_0000;
|
||||
// horizontal mirror: not a pure rotation, so the frame is left as stored
|
||||
let s = Mp4Source::open("mirror.mov", rotated_mov([-ONE, 0, 0, 0, ONE, 0, 64 * ONE, 0, W])).expect("open");
|
||||
let (w, h, top, bottom) = halves(&s);
|
||||
assert_eq!((w, h), (64, 32));
|
||||
assert!(top.abs_diff(bottom) < 4);
|
||||
// a 2× scaled identity is still upright (only the signs decide the rotation)
|
||||
let s = Mp4Source::open("scaled.mov", rotated_mov([2 * ONE, 0, 0, 0, 2 * ONE, 0, 0, 0, W])).expect("open");
|
||||
assert_eq!(s.info().video.as_ref().map(|v| (v.width, v.height)), Some((64, 32)));
|
||||
// a scaled quarter turn is still a quarter turn
|
||||
let s = Mp4Source::open("cw2.mov", rotated_mov([0, 2 * ONE, 0, -2 * ONE, 0, 0, 64 * ONE, 0, W])).expect("open");
|
||||
assert_eq!(s.info().video.as_ref().map(|v| (v.width, v.height)), Some((32, 64)));
|
||||
}
|
||||
}
|
||||
+43
-15
@@ -123,6 +123,19 @@ fn audio_supported(c: &Codec) -> bool {
|
||||
matches!(c, Codec::MpegAudio | Codec::AacAdts | Codec::AacLatm | Codec::LpcmBluray | Codec::LpcmDvd | Codec::Ac3)
|
||||
}
|
||||
|
||||
/// The coded rate unless the timestamps say otherwise (3:2 pulldown, field-rate coding). Never
|
||||
/// zero: damaged timestamps spread over a huge span give a rate that rounds to 0 fps, which the
|
||||
/// duration computation divided by.
|
||||
fn stream_rate(coded: Option<FrameRate>, from_pts: Option<f64>) -> FrameRate {
|
||||
let rate = match (coded, from_pts) {
|
||||
(Some(r), Some(p)) if (r.as_f64() - p).abs() / p > 0.01 => FrameRate::from_f64(p),
|
||||
(Some(r), _) => r,
|
||||
(None, Some(p)) => FrameRate::from_f64(p),
|
||||
(None, None) => FrameRate::FPS_25,
|
||||
};
|
||||
if rate.num > 0 && rate.den > 0 { rate } else { FrameRate::FPS_25 }
|
||||
}
|
||||
|
||||
/// Display-order keys of samples (decode order): the PTS when every unit has one, else (MPEG
|
||||
/// video) GOP start + temporal_reference, else the decode order.
|
||||
fn display_keys(units: &[&Unit], mpeg: bool) -> Vec<i64> {
|
||||
@@ -242,8 +255,9 @@ impl MpegSource {
|
||||
src.file = Some(file);
|
||||
src.video = Some(track);
|
||||
src.describe_video(rate_from_pts)?;
|
||||
let file = src.file.take().expect("file");
|
||||
src.setup_audio(file, astream, origin);
|
||||
if let Some(file) = src.file.take() {
|
||||
src.setup_audio(file, astream, origin);
|
||||
}
|
||||
}
|
||||
None => {
|
||||
src.unsupported_video = Some(format!("{} video in {} (FilmCraft has no decoder for it)", st.codec.name(), file.format.name()));
|
||||
@@ -257,6 +271,7 @@ impl MpegSource {
|
||||
color: filmcraft_color::ColorInfo::REC709,
|
||||
has_alpha: false,
|
||||
bitrate: None,
|
||||
hdr: None,
|
||||
});
|
||||
src.setup_audio(file, astream, None);
|
||||
}
|
||||
@@ -368,14 +383,14 @@ impl MpegSource {
|
||||
if let Some((data, aus)) = &self.es {
|
||||
return Ok(data[aus[u].clone()].to_vec());
|
||||
}
|
||||
let file = self.file.as_ref().expect("file");
|
||||
let file = self.file.as_ref().ok_or_else(|| CodecError::Container("no stream file".into()))?;
|
||||
file.read_unit(&self.bytes, v.stream, u).map_err(|e| CodecError::Container(e.to_string()))
|
||||
}
|
||||
|
||||
/// Fill in the video stream info from the first picture.
|
||||
fn describe_video(&mut self, rate_from_pts: Option<f64>) -> crate::Result<()> {
|
||||
let mut seq_header = None;
|
||||
let v = self.video.as_ref().expect("video");
|
||||
let v = self.video.as_ref().ok_or_else(|| CodecError::Unsupported("no video stream".into()))?;
|
||||
let first = self.read_unit(v, 0)?;
|
||||
let n = v.order.len();
|
||||
let (width, height, par, codec, pixel_format, mut color);
|
||||
@@ -431,12 +446,7 @@ impl MpegSource {
|
||||
}
|
||||
}
|
||||
// the coded rate unless the timestamps say otherwise (3:2 pulldown, field-rate coding)
|
||||
let rate = match (rate, rate_from_pts) {
|
||||
(Some(r), Some(p)) if (r.as_f64() - p).abs() / p > 0.01 => FrameRate::from_f64(p),
|
||||
(Some(r), _) => r,
|
||||
(None, Some(p)) => FrameRate::from_f64(p),
|
||||
(None, None) => FrameRate::FPS_25,
|
||||
};
|
||||
let rate = stream_rate(rate, rate_from_pts);
|
||||
let duration = rate.tick_of(n as i64);
|
||||
let total_bytes: u64 = match (&self.es, &self.file) {
|
||||
(Some((_, aus)), _) => v.units.iter().map(|&u| aus[u].len() as u64).sum(),
|
||||
@@ -444,8 +454,10 @@ impl MpegSource {
|
||||
_ => 0,
|
||||
};
|
||||
let secs = duration.seconds();
|
||||
if let Some(h) = seq_header {
|
||||
self.video.as_mut().expect("video").header = h;
|
||||
if let Some(h) = seq_header
|
||||
&& let Some(v) = self.video.as_mut()
|
||||
{
|
||||
v.header = h;
|
||||
}
|
||||
self.info.duration = duration;
|
||||
self.info.video = Some(VideoStreamInfo {
|
||||
@@ -458,6 +470,7 @@ impl MpegSource {
|
||||
color,
|
||||
has_alpha: false,
|
||||
bitrate: (secs > 0.0).then(|| (total_bytes as f64 * 8.0 / secs) as u64),
|
||||
hdr: None,
|
||||
});
|
||||
Ok(())
|
||||
}
|
||||
@@ -523,7 +536,7 @@ impl MpegSource {
|
||||
return self.audio_unsupported(&codec_name, "DVD LPCM: no format header".into());
|
||||
}
|
||||
},
|
||||
_ => unreachable!("audio_supported"),
|
||||
_ => return self.audio_unsupported(&codec_name, format!("{codec_name} audio")),
|
||||
};
|
||||
if codec_is_ac3_unsupported(codec) {
|
||||
return self.audio_unsupported(&codec_name, format!("{codec_name} audio (FilmCraft has no AC-3 decoder yet)"));
|
||||
@@ -606,12 +619,12 @@ impl MpegSource {
|
||||
if let Some(p) = st.cache.get(&i) {
|
||||
return Ok(p.clone());
|
||||
}
|
||||
let file = self.file.as_ref().expect("file");
|
||||
let file = self.file.as_ref().ok_or_else(|| CodecError::Container("no stream file".into()))?;
|
||||
let read = |k: usize| file.read_unit(&self.bytes, a.stream, k).map_err(|e| CodecError::Container(e.to_string()));
|
||||
if st.dec.is_none() {
|
||||
st.dec = Some(AudioDecoder::new(a.codec, a.rate, a.channels, a.latm.clone())?);
|
||||
}
|
||||
let dec = st.dec.as_mut().expect("decoder");
|
||||
let dec = st.dec.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?;
|
||||
if st.last.is_none_or(|l| l + 1 != i) {
|
||||
dec.reset();
|
||||
if i > 0
|
||||
@@ -838,6 +851,21 @@ pub fn reader_opener(name: &str, head: &[u8], reader: &filmcraft_media::SharedRe
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// A mutated transport stream whose PTS span gave ~0 fps panicked on open with "attempt to
|
||||
/// divide by zero" (`FrameRate::tick_of`).
|
||||
#[test]
|
||||
fn damaged_timestamps_never_give_a_zero_rate() {
|
||||
for (coded, p) in
|
||||
[(None, Some(1e-9)), (Some(FrameRate::FPS_25), Some(1e-9)), (Some(FrameRate::new(0, 0)), None), (None, Some(0.0)), (None, Some(f64::NAN))]
|
||||
{
|
||||
let r = stream_rate(coded, p);
|
||||
assert!(r.num > 0 && r.den > 0, "{coded:?} {p:?} -> {r:?}");
|
||||
let _ = r.tick_of(1000);
|
||||
}
|
||||
assert_eq!(stream_rate(Some(FrameRate::FPS_25), Some(25.0)), FrameRate::FPS_25);
|
||||
assert_eq!(stream_rate(None, Some(50.0)), FrameRate::from_f64(50.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn presentation_drops_leading_pictures_and_orders_by_key() {
|
||||
// decode order I(2) B(0) B(1) P(5) B(3) B(4): the leading B pictures are not presented
|
||||
|
||||
@@ -320,6 +320,7 @@ impl MxfSource {
|
||||
color,
|
||||
has_alpha: p.alpha_depth > 0,
|
||||
bitrate: (secs > 0.0).then(|| (bytes_total as f64 * 8.0 / (t.samples.len().max(1) as f64 / rate.as_f64().max(1e-9))) as u64),
|
||||
hdr: None,
|
||||
});
|
||||
}
|
||||
// Audio: all PCM sound tracks of the first track's rate, channels in track order
|
||||
@@ -332,7 +333,7 @@ impl MxfSource {
|
||||
let mut channels = 0;
|
||||
for &ai in &atracks {
|
||||
let t = &file.tracks[ai];
|
||||
let si = t.sound.as_ref().expect("sound info");
|
||||
let Some(si) = t.sound.as_ref() else { continue };
|
||||
if si.sample_rate != rate {
|
||||
continue;
|
||||
}
|
||||
@@ -423,7 +424,7 @@ impl VideoSamples for MxfVideo<'_> {
|
||||
return Err(CodecError::Unsupported(why.clone()));
|
||||
}
|
||||
let t = &self.src.file.tracks[self.track];
|
||||
let v = self.src.info.video.as_ref().expect("video info");
|
||||
let v = self.src.info.video.as_ref().ok_or_else(|| CodecError::Unsupported("no video track".into()))?;
|
||||
let (w, h) = (v.width.min(u16::MAX as u32) as u16, v.height.min(u16::MAX as u32) as u16);
|
||||
match t.codec {
|
||||
Codec::Avc { .. } => Ok(Box::new(crate::video::H264Decoder::annexb())),
|
||||
|
||||
@@ -133,7 +133,7 @@ impl OggSource {
|
||||
}
|
||||
// Non-sequential access: reset and decode the pre-roll (RFC 7845 §4.6; see OPUS_PRE_ROLL).
|
||||
if st.last_decoded.is_none_or(|l| l + 1 != i) {
|
||||
let d = st.decoder.as_mut().expect("decoder");
|
||||
let d = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?;
|
||||
d.reset();
|
||||
let from = timing.starts.partition_point(|&x| x <= timing.starts[i] - OPUS_PRE_ROLL as i64).saturating_sub(1);
|
||||
for j in from..i {
|
||||
@@ -143,7 +143,7 @@ impl OggSource {
|
||||
}
|
||||
}
|
||||
let data = read(i)?;
|
||||
let r = st.decoder.as_mut().expect("decoder").decode(&data, 0);
|
||||
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, 0);
|
||||
st.last_decoded = Some(i);
|
||||
let p = Arc::new(r.unwrap_or_default());
|
||||
st.packets.insert(i, p.clone());
|
||||
|
||||
+28
-64
@@ -88,6 +88,33 @@ fn mp3_file_decodes() {
|
||||
assert!(a.peaks()[0] > 0.1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn aiff_file_decodes() {
|
||||
// 0.1 s of 48 kHz 16-bit stereo PCM: left a ramp, right silent (AIFF, big-endian samples)
|
||||
let frames = 4800u32;
|
||||
let mut b = b"FORM".to_vec();
|
||||
b.extend((4 + 26 + 16 + frames * 4).to_be_bytes());
|
||||
b.extend(b"AIFFCOMM");
|
||||
b.extend(18u32.to_be_bytes());
|
||||
b.extend(2u16.to_be_bytes());
|
||||
b.extend(frames.to_be_bytes());
|
||||
b.extend(16u16.to_be_bytes());
|
||||
b.extend([0x40, 0x0E, 0xBB, 0x80, 0, 0, 0, 0, 0, 0]); // 48000 as an 80-bit extended float
|
||||
b.extend(b"SSND");
|
||||
b.extend((8 + frames * 4).to_be_bytes());
|
||||
b.extend([0; 8]);
|
||||
for i in 0..frames {
|
||||
b.extend(((i as i16) * 4).to_be_bytes());
|
||||
b.extend(0i16.to_be_bytes());
|
||||
}
|
||||
let src = crate::open_bytes("tone.aif", b.into()).unwrap();
|
||||
assert!((src.info().duration.seconds() - 0.1).abs() < 1e-3, "{}", src.info().duration.seconds());
|
||||
let a = src.audio(0, 4800, 48_000).unwrap();
|
||||
assert_eq!(a.channels.len(), 2);
|
||||
assert!((a.channels[0][1000] - 4000.0 / 32768.0).abs() < 1e-4, "{}", a.channels[0][1000]);
|
||||
assert_eq!(a.channels[1][1000], 0.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn seek_backwards_and_forwards_mjpeg() {
|
||||
let Some(b) = fixture("counter_mjpeg.mov", &["-f", "lavfi", "-i", "testsrc2=s=160x120:r=25:d=3", "-c:v", "mjpeg"]) else { return };
|
||||
@@ -429,69 +456,6 @@ fn mkv_vp9_rgb() {
|
||||
}
|
||||
}
|
||||
|
||||
/// WebM VP9 with alpha (`AlphaMode` 1, alpha stream in BlockAdditions): colour and alpha planes
|
||||
/// are sample-exact against libvpx (which decodes the alpha stream; ffmpeg's native VP9 decoder
|
||||
/// ignores it), also after seeks; frames convert to straight-alpha RGBA.
|
||||
#[test]
|
||||
fn webm_vp9_alpha_matches_libvpx() {
|
||||
let name = "alpha_vp9.webm";
|
||||
let frames = 40usize;
|
||||
let n = frames.to_string();
|
||||
let args = [
|
||||
"-f",
|
||||
"lavfi",
|
||||
"-i",
|
||||
"testsrc2=s=176x144:r=25,format=yuva420p,geq=lum='p(X,Y)':cb='cb(X,Y)':cr='cr(X,Y)':a='mod(X*3+Y+N*7,256)'",
|
||||
"-frames:v",
|
||||
&n,
|
||||
"-c:v",
|
||||
"libvpx-vp9",
|
||||
"-pix_fmt",
|
||||
"yuva420p",
|
||||
"-g",
|
||||
"16",
|
||||
"-b:v",
|
||||
"400k",
|
||||
"-deadline",
|
||||
"realtime",
|
||||
"-speed",
|
||||
"8",
|
||||
];
|
||||
let Some(path) = fixture_path(name, &args) else { return };
|
||||
let p = path.to_str().unwrap();
|
||||
let Some(reference) =
|
||||
fixture("alpha_vp9.webm.libvpx.yuva420p", &["-c:v", "libvpx-vp9", "-i", p, "-fps_mode", "passthrough", "-f", "rawvideo", "-pix_fmt", "yuva420p"])
|
||||
else {
|
||||
return;
|
||||
};
|
||||
let bytes: Arc<[u8]> = std::fs::read(&path).unwrap().into();
|
||||
let src = crate::open_bytes(name, bytes).unwrap();
|
||||
let info = src.info().clone();
|
||||
assert_eq!(info.container, "WebM");
|
||||
let v = info.video.as_ref().unwrap();
|
||||
assert!(v.has_alpha);
|
||||
let (w, h) = (176usize, 144usize);
|
||||
let frame_len = w * h * 5 / 2;
|
||||
assert_eq!(reference.len(), frame_len * frames);
|
||||
let rate = info.frame_rate();
|
||||
let order: Vec<usize> = [frames - 1, 3, 20, 17, 16, 0, 8].into_iter().chain(25..32).collect();
|
||||
let mut translucent = 0;
|
||||
for k in order {
|
||||
let f = src.video_frame(FrameRequest::full(rate.tick_of(k as i64))).unwrap();
|
||||
let filmcraft_frame::PixelData::Yuv8 { planes, alpha, .. } = &f.data else { panic!("not 8-bit YUV: {}", f.format_label()) };
|
||||
let r = &reference[k * frame_len..(k + 1) * frame_len];
|
||||
let colour: Vec<u8> = planes.iter().flat_map(|p| p.iter().copied()).collect();
|
||||
assert!(colour == r[..w * h * 3 / 2], "frame {k}: colour differs from libvpx");
|
||||
let a = alpha.as_ref().unwrap_or_else(|| panic!("frame {k}: no alpha plane"));
|
||||
assert!(a[..] == r[w * h * 3 / 2..], "frame {k}: alpha differs from libvpx");
|
||||
// straight alpha survives conversion to RGBA
|
||||
let px = f.to_rgba8();
|
||||
assert!(px.chunks_exact(4).zip(a.iter()).all(|(p, a)| p[3] == *a), "frame {k}: RGBA alpha");
|
||||
translucent += a.iter().filter(|&&v| v > 8 && v < 247).count();
|
||||
}
|
||||
assert!(translucent > 0, "the fixture's alpha should not be binary");
|
||||
}
|
||||
|
||||
/// ffmpeg + libopus decode of a container fixture (pre-skip / codec delay / edit list applied) as
|
||||
/// interleaved f32 at 48 kHz: the reference for our Opus path.
|
||||
fn opus_reference(name: &str) -> Option<Vec<f32>> {
|
||||
@@ -508,7 +472,7 @@ fn opus_reference(name: &str) -> Option<Vec<f32>> {
|
||||
if !st.success() {
|
||||
return None;
|
||||
}
|
||||
Some(std::fs::read(out).ok()?.chunks_exact(4).map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect())
|
||||
Some(std::fs::read(out).ok()?.as_chunks::<4>().0.iter().map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect())
|
||||
}
|
||||
|
||||
fn snr_db(reference: &[f32], test: &[f32]) -> f64 {
|
||||
|
||||
+62
-35
@@ -128,6 +128,42 @@ pub(crate) fn tight_plane<T: Copy>(src: Vec<T>, stride: usize, w: usize, h: usiz
|
||||
out
|
||||
}
|
||||
|
||||
/// Colour of an H.264 / HEVC picture from its VUI code points (ITU-T H.273), as the decoders
|
||||
/// report it: BT.709 by default with the matrix guessed from the size, the signalled matrix and
|
||||
/// transfer when known, and full range when flagged. Shared by the software and hardware
|
||||
/// decoders so the two are interchangeable.
|
||||
pub fn vui_color(width: u32, height: u32, matrix: u8, transfer: u8, full_range: bool) -> filmcraft_color::ColorInfo {
|
||||
let mut color = filmcraft_color::ColorInfo { matrix: filmcraft_frame::default_matrix(width, height), ..filmcraft_color::ColorInfo::REC709 };
|
||||
if let Some(m) = filmcraft_color::Matrix::from_code(matrix) {
|
||||
color.matrix = m;
|
||||
}
|
||||
if let Some(t) = filmcraft_color::Transfer::from_code(transfer) {
|
||||
color.transfer = t;
|
||||
}
|
||||
if full_range {
|
||||
color.range = filmcraft_color::Range::Full;
|
||||
}
|
||||
color
|
||||
}
|
||||
|
||||
/// Pixel aspect ratio from a VUI sample aspect ratio ((0, 0) = unspecified = square).
|
||||
pub fn sar_par(sar: (u16, u16)) -> (u32, u32) {
|
||||
if sar.0 > 0 && sar.1 > 0 { (sar.0 as u32, sar.1 as u32) } else { (1, 1) }
|
||||
}
|
||||
|
||||
/// NAL length-prefix size of an `avcC` record (byte 4: lengthSizeMinusOne).
|
||||
pub fn avcc_length_size(avcc: &[u8]) -> usize {
|
||||
avcc.get(4).map_or(4, |b| (b & 3) as usize + 1)
|
||||
}
|
||||
|
||||
/// NAL length-prefix size and highest TemporalId of an `hvcC` record.
|
||||
pub fn hvcc_length_size_and_tid(hvcc: &[u8]) -> (usize, Option<u8>) {
|
||||
// hvcC byte 21: constantFrameRate(2) numTemporalLayers(3) temporalIdNested(1) lengthSizeMinusOne(2)
|
||||
let length_size = hvcc.get(21).map_or(4, |b| (b & 3) as usize + 1);
|
||||
let highest_tid = hvcc.get(21).map(|b| (b >> 3) & 7).filter(|&n| n > 0).map(|n| n - 1);
|
||||
(length_size, highest_tid)
|
||||
}
|
||||
|
||||
/// A factory returns `None` when it does not handle the entry.
|
||||
pub type VideoDecoderFactory = fn(&SampleEntry) -> Option<Result<Box<dyn VideoDecoder>>>;
|
||||
|
||||
@@ -169,7 +205,7 @@ pub struct H264Decoder {
|
||||
impl H264Decoder {
|
||||
pub fn new(avcc: Vec<u8>) -> Result<Self> {
|
||||
let dec = filmcraft_h264::Decoder::from_avcc(&avcc).map_err(|e| CodecError::Decode(e.to_string()))?;
|
||||
let length_size = avcc.get(4).map_or(4, |b| (b & 3) as usize + 1);
|
||||
let length_size = avcc_length_size(&avcc);
|
||||
Ok(Self { avcc, dec, length_size, draft: false })
|
||||
}
|
||||
/// A decoder for Annex B byte-stream samples (start codes, in-band parameter sets: MXF, TS).
|
||||
@@ -184,17 +220,8 @@ impl H264Decoder {
|
||||
let y = tight_plane(p.y, p.y_stride, w, h);
|
||||
let u = tight_plane(p.u, p.uv_stride, cw, ch);
|
||||
let v = tight_plane(p.v, p.uv_stride, cw, ch);
|
||||
let mut color = filmcraft_color::ColorInfo { matrix: filmcraft_frame::default_matrix(p.width, p.height), ..filmcraft_color::ColorInfo::REC709 };
|
||||
if let Some(m) = filmcraft_color::Matrix::from_code(p.color.matrix) {
|
||||
color.matrix = m;
|
||||
}
|
||||
if let Some(t) = filmcraft_color::Transfer::from_code(p.color.transfer) {
|
||||
color.transfer = t;
|
||||
}
|
||||
if p.color.full_range {
|
||||
color.range = filmcraft_color::Range::Full;
|
||||
}
|
||||
let par = if p.sar.0 > 0 && p.sar.1 > 0 { (p.sar.0 as u32, p.sar.1 as u32) } else { (1, 1) };
|
||||
let color = vui_color(p.width, p.height, p.color.matrix, p.color.transfer, p.color.full_range);
|
||||
let par = sar_par(p.sar);
|
||||
let frame = VideoFrame {
|
||||
width: p.width,
|
||||
height: p.height,
|
||||
@@ -265,9 +292,7 @@ pub struct HevcDecoder {
|
||||
impl HevcDecoder {
|
||||
pub fn new(hvcc: Vec<u8>) -> Result<Self> {
|
||||
let dec = filmcraft_hevc::Decoder::from_hvcc(&hvcc).map_err(|e| CodecError::Decode(e.to_string()))?;
|
||||
// hvcC byte 21: constantFrameRate(2) numTemporalLayers(3) temporalIdNested(1) lengthSizeMinusOne(2)
|
||||
let length_size = hvcc.get(21).map_or(4, |b| (b & 3) as usize + 1);
|
||||
let highest_tid = hvcc.get(21).map(|b| (b >> 3) & 7).filter(|&n| n > 0).map(|n| n - 1);
|
||||
let (length_size, highest_tid) = hvcc_length_size_and_tid(&hvcc);
|
||||
Ok(Self { hvcc, dec, length_size, highest_tid })
|
||||
}
|
||||
/// A decoder for Annex B byte-stream samples (start codes, in-band parameter sets: TS).
|
||||
@@ -299,19 +324,10 @@ impl HevcDecoder {
|
||||
alpha: None,
|
||||
},
|
||||
};
|
||||
let mut color = filmcraft_color::ColorInfo { matrix: filmcraft_frame::default_matrix(p.width, p.height), ..filmcraft_color::ColorInfo::REC709 };
|
||||
if let Some(m) = filmcraft_color::Matrix::from_code(p.color.matrix) {
|
||||
color.matrix = m;
|
||||
}
|
||||
if let Some(t) = filmcraft_color::Transfer::from_code(p.color.transfer) {
|
||||
color.transfer = t;
|
||||
}
|
||||
if p.color.full_range {
|
||||
color.range = filmcraft_color::Range::Full;
|
||||
}
|
||||
let par = if p.sar.0 > 0 && p.sar.1 > 0 { (p.sar.0 as u32, p.sar.1 as u32) } else { (1, 1) };
|
||||
let color = vui_color(p.width, p.height, p.color.matrix, p.color.transfer, p.color.full_range);
|
||||
let par = sar_par(p.sar);
|
||||
let frame = VideoFrame { width: p.width, height: p.height, data, color, par, pts: filmcraft_time::Tick::ZERO };
|
||||
DecodedFrame { pts: p.pts, frame, draft: false }
|
||||
DecodedFrame { pts: p.pts, frame, draft: p.draft }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -336,6 +352,9 @@ impl VideoDecoder for HevcDecoder {
|
||||
fn is_disposable(&self, sample: &[u8]) -> bool {
|
||||
hevc_disposable(sample, self.length_size, self.highest_tid)
|
||||
}
|
||||
fn set_draft(&mut self, on: bool) {
|
||||
self.dec.set_draft(on);
|
||||
}
|
||||
fn is_random_access(&self, sample: &[u8]) -> Option<bool> {
|
||||
// Annex B samples carry their parameter sets: an IDR / CRA / BLA access unit is a
|
||||
// starting point.
|
||||
@@ -383,7 +402,8 @@ impl Vp9Decoder {
|
||||
}
|
||||
}
|
||||
|
||||
fn convert(&self, p: filmcraft_vp9::Picture) -> DecodedFrame {
|
||||
/// `None` for a picture whose planes mix sample types (never produced by the decoder).
|
||||
fn convert(&self, p: filmcraft_vp9::Picture) -> Option<DecodedFrame> {
|
||||
use filmcraft_color::{Matrix, Primaries, Range};
|
||||
use filmcraft_frame::{Chroma, PixelData};
|
||||
use filmcraft_vp9::Plane;
|
||||
@@ -410,7 +430,7 @@ impl Vp9Decoder {
|
||||
if p.color.full_range {
|
||||
color.range = Range::Full;
|
||||
}
|
||||
let pts = p.pts;
|
||||
let (pts, draft) = (p.pts, p.draft);
|
||||
if p.color.color_space == 7 {
|
||||
// RGB (profiles 1 / 3, 4:4:4): the planes carry G, B, R.
|
||||
let shift = p.bit_depth - 8;
|
||||
@@ -420,7 +440,7 @@ impl Vp9Decoder {
|
||||
}
|
||||
let mut frame = VideoFrame::rgba8(p.width, p.height, rgba);
|
||||
frame.color = filmcraft_color::ColorInfo { range: Range::Full, transfer: filmcraft_color::Transfer::Srgb, ..color };
|
||||
return DecodedFrame { pts, frame, draft: false };
|
||||
return Some(DecodedFrame { pts, frame, draft });
|
||||
}
|
||||
// 4:4:0 has no frame format of its own: chroma rows are repeated to 4:4:4.
|
||||
let (chroma, rows_440) = match (p.subsampling_x, p.subsampling_y) {
|
||||
@@ -452,22 +472,22 @@ impl Vp9Decoder {
|
||||
bits: p.bit_depth,
|
||||
alpha: None,
|
||||
},
|
||||
_ => unreachable!("VP9 planes share one sample type"),
|
||||
_ => return None,
|
||||
};
|
||||
// render_size (the intended display size) is not applied: the container's display
|
||||
// dimensions / pixel aspect describe the presentation.
|
||||
let par = (1, 1);
|
||||
DecodedFrame { pts, frame: VideoFrame { width: p.width, height: p.height, data, color, par, pts: filmcraft_time::Tick::ZERO }, draft: false }
|
||||
Some(DecodedFrame { pts, frame: VideoFrame { width: p.width, height: p.height, data, color, par, pts: filmcraft_time::Tick::ZERO }, draft })
|
||||
}
|
||||
}
|
||||
|
||||
impl VideoDecoder for Vp9Decoder {
|
||||
fn decode(&mut self, sample: &[u8], pts: i64) -> Result<Vec<DecodedFrame>> {
|
||||
let pics = self.dec.decode(sample, pts).map_err(|e| CodecError::Decode(e.to_string()))?;
|
||||
Ok(pics.into_iter().map(|p| self.convert(p)).collect())
|
||||
Ok(pics.into_iter().filter_map(|p| self.convert(p)).collect())
|
||||
}
|
||||
fn flush(&mut self) -> Vec<DecodedFrame> {
|
||||
self.dec.flush().into_iter().map(|p| self.convert(p)).collect()
|
||||
self.dec.flush().into_iter().filter_map(|p| self.convert(p)).collect()
|
||||
}
|
||||
fn reset(&mut self) {
|
||||
self.dec.reset();
|
||||
@@ -475,6 +495,9 @@ impl VideoDecoder for Vp9Decoder {
|
||||
fn name(&self) -> &str {
|
||||
"FilmCraft VP9"
|
||||
}
|
||||
fn set_draft(&mut self, on: bool) {
|
||||
self.dec.set_draft(on);
|
||||
}
|
||||
fn is_random_access(&self, sample: &[u8]) -> Option<bool> {
|
||||
Some(filmcraft_vp9::is_keyframe(sample))
|
||||
}
|
||||
@@ -502,6 +525,7 @@ impl Av1Decoder {
|
||||
|
||||
fn convert(p: filmcraft_av1::Picture, pts: i64) -> DecodedFrame {
|
||||
use std::sync::Arc;
|
||||
let draft = p.draft;
|
||||
let w = p.width as usize;
|
||||
let h = p.height as usize;
|
||||
let mut color = filmcraft_color::ColorInfo::REC709;
|
||||
@@ -541,7 +565,7 @@ impl Av1Decoder {
|
||||
} else {
|
||||
filmcraft_frame::PixelData::Yuv16 { planes: [Arc::new(y), Arc::new(u), Arc::new(v)], chroma, bits: p.bit_depth as u32, alpha: None }
|
||||
};
|
||||
DecodedFrame { pts, frame: VideoFrame { width: p.width, height: p.height, data, color, par: (1, 1), pts: filmcraft_time::Tick::ZERO }, draft: false }
|
||||
DecodedFrame { pts, frame: VideoFrame { width: p.width, height: p.height, data, color, par: (1, 1), pts: filmcraft_time::Tick::ZERO }, draft }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -580,6 +604,9 @@ impl VideoDecoder for Av1Decoder {
|
||||
fn name(&self) -> &str {
|
||||
"FilmCraft AV1"
|
||||
}
|
||||
fn set_draft(&mut self, on: bool) {
|
||||
self.dec.set_draft(on);
|
||||
}
|
||||
fn is_random_access(&self, sample: &[u8]) -> Option<bool> {
|
||||
Some(filmcraft_av1::is_key_frame_unit(sample))
|
||||
}
|
||||
|
||||
@@ -48,7 +48,7 @@ pub fn ffmpeg_audio_f32(ff: &Path, file: &Path, extra: &[&str]) -> Vec<f32> {
|
||||
let mut args = vec!["-i", file.to_str().unwrap()];
|
||||
args.extend_from_slice(extra);
|
||||
args.extend_from_slice(&["-map", "0:a:0", "-f", "f32le", "-"]);
|
||||
ffmpeg_out(ff, &args).chunks_exact(4).map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect()
|
||||
ffmpeg_out(ff, &args).as_chunks::<4>().0.iter().map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect()
|
||||
}
|
||||
|
||||
/// Our frame's three planes widened to u16 (and their sizes).
|
||||
@@ -63,7 +63,7 @@ pub fn planes(f: &VideoFrame) -> Vec<Vec<u16>> {
|
||||
/// Split one raw frame of `ffmpeg_frames` into planes (`bps` bytes per sample, chroma `cw`×`ch`).
|
||||
pub fn raw_planes(raw: &[u8], w: usize, h: usize, cw: usize, ch: usize, bps: usize) -> Vec<Vec<u16>> {
|
||||
let get = |b: &[u8]| -> Vec<u16> {
|
||||
if bps == 1 { b.iter().map(|&v| v as u16).collect() } else { b.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]])).collect() }
|
||||
if bps == 1 { b.iter().map(|&v| v as u16).collect() } else { b.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect() }
|
||||
};
|
||||
let y = w * h * bps;
|
||||
let c = cw * ch * bps;
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
//! Draft decoding of VP9, HEVC and AV1 through the media stack (as `h264_draft.rs` for H.264):
|
||||
//! off by default, approximate frames only reach draft requests, and decoding for an export after
|
||||
//! draft playback (same source, same GOP cache) still matches ffmpeg bit for bit.
|
||||
|
||||
mod common;
|
||||
|
||||
use common::*;
|
||||
use filmcraft_media::FrameRequest;
|
||||
use filmcraft_media::cancel::with_draft;
|
||||
|
||||
/// Play `name` without and with draft decoding, then "export" it: every non-draft request must
|
||||
/// be bit-exact with ffmpeg, and draft playback must have produced some approximate frames.
|
||||
fn check(name: &str, args: &[&str], w: usize, h: usize) {
|
||||
let ff = filmcraft_testkit::require_ffmpeg!();
|
||||
let Some(file) = fixture(&ff, name, args) else { return };
|
||||
let raw = ffmpeg_frames(&ff, &file, "yuv420p");
|
||||
let fsize = w * h * 3 / 2;
|
||||
let n = raw.len() / fsize;
|
||||
assert!(n >= 16, "{name}: {n} frames");
|
||||
let reference = |i: usize| raw_planes(&raw[i * fsize..(i + 1) * fsize], w, h, w / 2, h / 2, 1);
|
||||
|
||||
// Default: nothing is a draft, every frame exact.
|
||||
let src = filmcraft_codecs::open_bytes(name, bytes(&file)).expect("open");
|
||||
let rate = src.info().frame_rate();
|
||||
let g0 = filmcraft_codecs::gop_stats();
|
||||
for i in 0..n {
|
||||
let f = src.video_frame(FrameRequest { time: rate.tick_of(i as i64), scale: 0.5 }).expect("frame");
|
||||
assert_eq!(max_diff(&planes(&f), &reference(i)), 0, "{name}: frame {i} without draft mode");
|
||||
}
|
||||
assert_eq!(filmcraft_codecs::gop_stats().draft, g0.draft, "{name}: draft decoding is off by default");
|
||||
|
||||
// Draft playback at 1/2 resolution on a fresh source: non-reference frames are drafts.
|
||||
let src = filmcraft_codecs::open_bytes(name, bytes(&file)).expect("open");
|
||||
let g1 = filmcraft_codecs::gop_stats();
|
||||
let mut approximate = 0;
|
||||
for i in 0..n {
|
||||
let f = with_draft(true, || src.video_frame(FrameRequest { time: rate.tick_of(i as i64), scale: 0.5 })).expect("frame");
|
||||
approximate += (max_diff(&planes(&f), &reference(i)) > 0) as usize;
|
||||
}
|
||||
let drafts = filmcraft_codecs::gop_stats().draft - g1.draft;
|
||||
assert!(drafts > 0 && approximate > 0 && approximate as u64 <= drafts, "{name}: {drafts} draft frames, {approximate} differ");
|
||||
|
||||
// Then an export from the same source (no draft hint): every frame is exact again.
|
||||
for i in 0..n {
|
||||
let f = src.video_frame(FrameRequest::full(rate.tick_of(i as i64))).expect("frame");
|
||||
assert_eq!(max_diff(&planes(&f), &reference(i)), 0, "{name}: export frame {i} after draft playback");
|
||||
}
|
||||
}
|
||||
|
||||
/// One test (the GOP cache statistics are process-wide, so the codecs run one after another).
|
||||
#[test]
|
||||
fn draft_playback_never_reaches_exports() {
|
||||
vp9();
|
||||
hevc();
|
||||
av1();
|
||||
}
|
||||
|
||||
fn vp9() {
|
||||
// Two temporal layers: every other frame refreshes no reference slot. (25 fps: WebM stores
|
||||
// millisecond timestamps.)
|
||||
check(
|
||||
"draft_vp9.webm",
|
||||
&[
|
||||
"-f",
|
||||
"lavfi",
|
||||
"-i",
|
||||
"testsrc2=s=320x240:r=25:d=1,noise=alls=10:allf=t",
|
||||
"-c:v",
|
||||
"libvpx-vp9",
|
||||
"-deadline",
|
||||
"realtime",
|
||||
"-speed",
|
||||
"8",
|
||||
"-b:v",
|
||||
"600k",
|
||||
"-ts-parameters",
|
||||
"ts_number_layers=2:ts_target_bitrate=300,600:ts_rate_decimator=2,1:ts_periodicity=2:ts_layer_id=0,1:ts_layering_mode=2",
|
||||
"-pix_fmt",
|
||||
"yuv420p",
|
||||
],
|
||||
320,
|
||||
240,
|
||||
);
|
||||
}
|
||||
|
||||
fn hevc() {
|
||||
// B-pyramid: non-reference B pictures (sub-layer non-reference NAL types).
|
||||
check(
|
||||
"draft_hevc.mp4",
|
||||
&[
|
||||
"-f",
|
||||
"lavfi",
|
||||
"-i",
|
||||
"testsrc2=s=320x240:r=25:d=1,noise=alls=10:allf=t",
|
||||
"-c:v",
|
||||
"libx265",
|
||||
"-preset",
|
||||
"fast",
|
||||
"-x265-params",
|
||||
"bframes=4:b-pyramid=1:keyint=24:log-level=error",
|
||||
"-pix_fmt",
|
||||
"yuv420p",
|
||||
"-tag:v",
|
||||
"hvc1",
|
||||
],
|
||||
320,
|
||||
240,
|
||||
);
|
||||
}
|
||||
|
||||
fn av1() {
|
||||
// SVT-AV1 hierarchical GOP: the top layer refreshes no reference slot.
|
||||
check(
|
||||
"draft_av1.mp4",
|
||||
&["-f", "lavfi", "-i", "testsrc2=s=320x240:r=25:d=1,noise=alls=10:allf=t", "-c:v", "libsvtav1", "-preset", "8", "-crf", "35", "-pix_fmt", "yuv420p"],
|
||||
320,
|
||||
240,
|
||||
);
|
||||
}
|
||||
@@ -92,7 +92,7 @@ fn decode_both(ff: &Path, file: &Path, decoder: &[&str]) -> (OggSource, Vec<f32>
|
||||
let ch = a.channels as usize;
|
||||
let mut args: Vec<&str> = decoder.to_vec();
|
||||
args.extend_from_slice(&["-i", file.to_str().unwrap(), "-map", "0:a:0", "-f", "f32le", "-"]);
|
||||
let want: Vec<f32> = ffmpeg_out(ff, &args).chunks_exact(4).map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
|
||||
let want: Vec<f32> = ffmpeg_out(ff, &args).as_chunks::<4>().0.iter().map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
|
||||
let n = want.len() / ch;
|
||||
let frames = src.info().duration.to_units_floor(a.sample_rate as i64) as usize;
|
||||
assert_eq!(frames, n, "{}: our length = ffmpeg's ({} samples)", file.display(), n);
|
||||
|
||||
+23
-1
@@ -41,7 +41,8 @@ for limited- and full-range files.
|
||||
|
||||
The encoders are the exact inverses given in the same documents. DJI **D-Log M** is not
|
||||
implemented: DJI has not published its formula (only conversion LUTs), and we do not derive curves
|
||||
from third-party LUTs.
|
||||
from third-party LUTs. (Checked again in October 2026: DJI's published white papers cover D-Log /
|
||||
D-Gamut only; D-Log M remains unsupported, so D-Log M footage should be interpreted with a LUT.)
|
||||
|
||||
Gamut primaries (all D65 white, x/y):
|
||||
|
||||
@@ -84,6 +85,27 @@ HLG, and for log media the curve's own peak capped at 4000 cd/m² (so the knee s
|
||||
grey). Target peak: 203 cd/m² = SDR white. HDR reference white then lands at ≈ 0.79 linear (≈ 90 %
|
||||
on the sRGB curve).
|
||||
|
||||
## Grading signal (`grade`)
|
||||
|
||||
Lumetri's controls work on a 0…1 signal. `GradeSpace` picks it from the working space:
|
||||
|
||||
| Working space | Signal of working-linear `c` (1.0 = 203 cd/m² in HDR) | 1.0 means |
|
||||
|---|---|---|
|
||||
| Rec. 709 | sRGB curve, clamped back to 0…1 | SDR white |
|
||||
| Rec. 2100 PQ | `PQ⁻¹(c·203/10000) / PQ⁻¹(W/10000)` (SMPTE ST 2084) | HDR White `W` cd/m² |
|
||||
| Rec. 2100 HLG | `F = c·203/W`, `Y = BT.2020 luma(F)`, `E = F·Y^((1−γ)/γ)`, signal = HLG OETF(E), γ = 1.2 + 0.42·log10(W/1000) (BT.2100 inverse OOTF, note 5f) | display peak `W` |
|
||||
|
||||
`W` is Lumetri's HDR White (Basic Correction) or HDR Range (Curves), default 1000 cd/m². HDR
|
||||
values above `W` are signals above 1 and are kept (PQ to 10 000 cd/m², HLG to signal 1.5). Checked
|
||||
values: PQ 100 / 203 / 1000 cd/m² = 0.508 / 0.581 / 0.752 of the plain PQ signal (BT.2408); HLG on
|
||||
a 1000 cd/m² display: 203 cd/m² = 0.75, 26 cd/m² (18 % grey) ≈ 0.38 (BT.2408).
|
||||
|
||||
## HDR static metadata
|
||||
|
||||
`HdrMetadata` (mastering display max / min luminance, MaxCLL, MaxFALL; 0 = unknown) gives the
|
||||
tone-mapping source peak of PQ media: MaxCLL when known (the brightest pixel of the content),
|
||||
else the mastering peak, clamped to 203…10 000 cd/m²; without metadata 1000 cd/m² is assumed.
|
||||
|
||||
## Gamut mapping
|
||||
|
||||
* **Input** (camera gamuts wider than the working gamut): per-channel distance compression towards
|
||||
|
||||
@@ -0,0 +1,176 @@
|
||||
//! The grading signal colour correctors (Lumetri) work on.
|
||||
//!
|
||||
//! Grading controls (whites / blacks, highlights / shadows, contrast, curves, wheels, keys) are
|
||||
//! defined on a perceptual signal from 0 (black) to 1 (white), not on linear light. In an SDR
|
||||
//! working space that signal is the sRGB-encoded display value, and 1 is SDR white.
|
||||
//!
|
||||
//! In an HDR working space (Rec. 2100 PQ or HLG) the signal follows the working space's own
|
||||
//! transfer function and is **normalised to HDR White** (cd/m², Lumetri ▸ Basic Correction ▸ HDR
|
||||
//! White; the curves have their own HDR Range): signal 1 is HDR White, so the sliders and curves
|
||||
//! span 0 … HDR White nits the way they span black … white in SDR, and highlights above HDR
|
||||
//! White (speculars) are values above 1 that pass through instead of clipping:
|
||||
//!
|
||||
//! - **PQ** (SMPTE ST 2084): `signal = PQ⁻¹(nits / 10000) / PQ⁻¹(white / 10000)`.
|
||||
//! - **HLG** (ITU-R BT.2100): the inverse OOTF of a display with peak `white` (system gamma
|
||||
//! γ = 1.2 + 0.42·log10(white / 1000), BT.2100 note 5f), then the HLG OETF; so 203 cd/m²
|
||||
//! (BT.2408 HDR reference white) is signal 0.75 with HDR White = 1000 cd/m².
|
||||
//!
|
||||
//! Working units are linear light with 1.0 = 203 cd/m² ([`crate::REFERENCE_WHITE_NITS`]) in HDR
|
||||
//! working spaces, as in [`crate::transform`].
|
||||
|
||||
use crate::spaces::{Gamut, WorkingSpace};
|
||||
use crate::{REFERENCE_WHITE_NITS, hlg_inverse_oetf, hlg_oetf, linear_to_srgb, pq_eotf, pq_inverse_eotf, srgb_to_linear};
|
||||
|
||||
/// Default HDR White / curve HDR Range (cd/m²): the nominal HLG display peak and the usual PQ
|
||||
/// mastering peak.
|
||||
pub const DEFAULT_HDR_WHITE_NITS: f32 = 1000.0;
|
||||
|
||||
/// The signal a grading operation works on.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub enum GradeSpace {
|
||||
/// sRGB-encoded display values, clamped to 0…1 on the way back (the classic SDR behaviour).
|
||||
Sdr,
|
||||
/// PQ signal normalised to `white` cd/m².
|
||||
Pq { white: f32, norm: f32 },
|
||||
/// HLG signal of a display with peak `white` cd/m² (`gamma` = its system gamma).
|
||||
Hlg { white: f32, gamma: f32 },
|
||||
}
|
||||
|
||||
impl GradeSpace {
|
||||
/// The grading space of a working space, with HDR White `white_nits` (ignored for SDR).
|
||||
pub fn new(working: WorkingSpace, white_nits: f32) -> Self {
|
||||
let white = white_nits.clamp(100.0, 10_000.0);
|
||||
match working {
|
||||
WorkingSpace::Rec709 => GradeSpace::Sdr,
|
||||
WorkingSpace::Rec2100Pq => GradeSpace::Pq { white, norm: pq_inverse_eotf(white / 10_000.0) },
|
||||
WorkingSpace::Rec2100Hlg => GradeSpace::Hlg { white, gamma: 1.2 + 0.42 * (white / 1000.0).log10() },
|
||||
}
|
||||
}
|
||||
|
||||
pub fn is_hdr(&self) -> bool {
|
||||
!matches!(self, GradeSpace::Sdr)
|
||||
}
|
||||
|
||||
/// HDR White in cd/m² (None for SDR).
|
||||
pub fn white_nits(&self) -> Option<f32> {
|
||||
match *self {
|
||||
GradeSpace::Sdr => None,
|
||||
GradeSpace::Pq { white, .. } | GradeSpace::Hlg { white, .. } => Some(white),
|
||||
}
|
||||
}
|
||||
|
||||
/// Working linear → grading signal (1 = white; HDR values above HDR White are > 1).
|
||||
#[inline]
|
||||
pub fn encode(&self, c: [f32; 3]) -> [f32; 3] {
|
||||
match *self {
|
||||
GradeSpace::Sdr => c.map(|v| linear_to_srgb(v.max(0.0))),
|
||||
GradeSpace::Pq { norm, .. } => c.map(|v| pq_inverse_eotf((v.max(0.0) * REFERENCE_WHITE_NITS as f32 / 10_000.0).min(1.0)) / norm),
|
||||
GradeSpace::Hlg { white, gamma } => {
|
||||
// display light normalised to the peak, inverse OOTF (scene light), OETF
|
||||
let fd = c.map(|v| v.max(0.0) * REFERENCE_WHITE_NITS as f32 / white);
|
||||
let l = luma2020();
|
||||
let yd = l[0] * fd[0] + l[1] * fd[1] + l[2] * fd[2];
|
||||
if yd <= 0.0 {
|
||||
return [0.0; 3];
|
||||
}
|
||||
let k = yd.powf((1.0 - gamma) / gamma);
|
||||
fd.map(|v| hlg_oetf(v * k))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Grading signal → working linear. SDR clamps to 0…1 (white); HDR keeps values above HDR
|
||||
/// White up to 10 000 cd/m² (PQ) or the HLG signal's range.
|
||||
#[inline]
|
||||
pub fn decode(&self, v: [f32; 3]) -> [f32; 3] {
|
||||
match *self {
|
||||
GradeSpace::Sdr => v.map(|q| srgb_to_linear(q.clamp(0.0, 1.0))),
|
||||
GradeSpace::Pq { norm, .. } => v.map(|q| pq_eotf((q * norm).clamp(0.0, 1.0)) * 10_000.0 / REFERENCE_WHITE_NITS as f32),
|
||||
GradeSpace::Hlg { white, gamma } => {
|
||||
// graded highlights may go past signal 1.0 (the display peak); keep them up to
|
||||
// 1.5 (≈ 28× the peak) instead of clipping
|
||||
let es = v.map(|q| hlg_inverse_oetf(q.clamp(0.0, 1.5)));
|
||||
let l = luma2020();
|
||||
let ys = l[0] * es[0] + l[1] * es[1] + l[2] * es[2];
|
||||
if ys <= 0.0 {
|
||||
return [0.0; 3];
|
||||
}
|
||||
let k = ys.powf(gamma - 1.0);
|
||||
es.map(|e| e * k * white / REFERENCE_WHITE_NITS as f32)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The signal of a grey of `nits` cd/m² (HDR) — what a nit-scaled slider position means.
|
||||
pub fn signal_of_nits(&self, nits: f32) -> f32 {
|
||||
let w = nits / REFERENCE_WHITE_NITS as f32;
|
||||
self.encode([w; 3])[1]
|
||||
}
|
||||
|
||||
/// The cd/m² of a grey of signal `s` (HDR).
|
||||
pub fn nits_of_signal(&self, s: f32) -> f32 {
|
||||
self.decode([s; 3])[1] * REFERENCE_WHITE_NITS as f32
|
||||
}
|
||||
}
|
||||
|
||||
fn luma2020() -> [f32; 3] {
|
||||
let l = Gamut::Bt2020.luma();
|
||||
[l[0] as f32, l[1] as f32, l[2] as f32]
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn grey(nits: f32) -> [f32; 3] {
|
||||
[nits / REFERENCE_WHITE_NITS as f32; 3]
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sdr_is_the_srgb_signal() {
|
||||
let g = GradeSpace::new(WorkingSpace::Rec709, 1000.0);
|
||||
assert!(!g.is_hdr());
|
||||
let e = g.encode([0.18, 0.5, 1.0]);
|
||||
assert!((e[0] - linear_to_srgb(0.18)).abs() < 1e-6 && (e[2] - 1.0).abs() < 1e-6);
|
||||
// values above white clip on the way back (SDR)
|
||||
assert_eq!(g.decode([1.4, 1.4, 1.4]), [1.0; 3]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pq_known_nit_values() {
|
||||
// HDR White 10 000: the plain PQ signal; 203 cd/m² ≈ 58 % (BT.2408), 1000 cd/m² ≈ 75.2 %
|
||||
let g = GradeSpace::new(WorkingSpace::Rec2100Pq, 10_000.0);
|
||||
assert!((g.signal_of_nits(203.0) - 0.5807).abs() < 1e-3);
|
||||
assert!((g.signal_of_nits(1000.0) - 0.7518).abs() < 1e-3);
|
||||
assert!((g.signal_of_nits(100.0) - 0.5081).abs() < 1e-3);
|
||||
// HDR White 1000: 1000 cd/m² is signal 1, reference white 0.772
|
||||
let g = GradeSpace::new(WorkingSpace::Rec2100Pq, 1000.0);
|
||||
assert!((g.signal_of_nits(1000.0) - 1.0).abs() < 1e-4);
|
||||
assert!((g.signal_of_nits(203.0) - 0.5807 / 0.7518).abs() < 1e-3);
|
||||
// above HDR White: > 1, not clipped, round trip to 4000 cd/m²
|
||||
let s = g.signal_of_nits(4000.0);
|
||||
assert!(s > 1.1);
|
||||
assert!((g.nits_of_signal(s) - 4000.0).abs() < 4.0);
|
||||
for n in [0.05f32, 1.0, 26.0, 100.0, 203.0, 600.0, 1000.0, 2500.0, 10_000.0] {
|
||||
let back = g.decode(g.encode(grey(n)))[0] * REFERENCE_WHITE_NITS as f32;
|
||||
assert!((back - n).abs() <= n * 2e-3 + 1e-3, "{n} → {back}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hlg_known_nit_values() {
|
||||
// a 1000 cd/m² HLG display: reference white 203 cd/m² = 75 % (BT.2408), the peak = 100 %
|
||||
let g = GradeSpace::new(WorkingSpace::Rec2100Hlg, 1000.0);
|
||||
assert!((g.signal_of_nits(203.0) - 0.75).abs() < 2e-3, "{}", g.signal_of_nits(203.0));
|
||||
assert!((g.signal_of_nits(1000.0) - 1.0).abs() < 1e-3);
|
||||
// 18 % grey card (≈ 26 cd/m² on that display, BT.2408 table) ≈ 38 %
|
||||
assert!((g.signal_of_nits(26.0) - 0.38).abs() < 0.01, "{}", g.signal_of_nits(26.0));
|
||||
// colour round trip (the inverse OOTF depends on luminance)
|
||||
let c = [2.0f32, 0.5, 0.25];
|
||||
let back = g.decode(g.encode(c));
|
||||
assert!(c.iter().zip(back).all(|(a, b)| (a - b).abs() < 1e-3 * a.max(1.0)), "{back:?}");
|
||||
// a brighter display has a higher system gamma (BT.2100: 1.2 + 0.42 log10(Lw / 1000))
|
||||
let GradeSpace::Hlg { gamma, .. } = GradeSpace::new(WorkingSpace::Rec2100Hlg, 2000.0) else { panic!() };
|
||||
assert!((gamma - 1.2 - 0.42 * 2f32.log10()).abs() < 1e-5);
|
||||
}
|
||||
}
|
||||
@@ -5,14 +5,18 @@
|
||||
//! The compositor works in **linear-light, premultiplied RGBA f32** in the sequence working space
|
||||
//! (Rec.709 primaries by default). Decoded frames carry [`ColorInfo`] so conversions are explicit.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::sync::OnceLock;
|
||||
|
||||
pub mod grade;
|
||||
pub mod log;
|
||||
pub mod lut;
|
||||
pub mod spaces;
|
||||
pub mod transform;
|
||||
|
||||
pub use grade::GradeSpace;
|
||||
pub use log::LogCurve;
|
||||
pub use lut::{Lut, Lut1d, Lut3d, LutFormat};
|
||||
pub use spaces::{ColorPipeline, ColorSpace, Curve, Gamut, WorkingSpace};
|
||||
@@ -101,6 +105,33 @@ pub struct ColorInfo {
|
||||
pub range: Range,
|
||||
}
|
||||
|
||||
/// HDR static metadata of a stream: SMPTE ST 2086 mastering display luminance and the CTA-861.3
|
||||
/// content light levels (MaxCLL / MaxFALL), in cd/m². Read from `mdcv` / `clli` (MP4 / MOV) and
|
||||
/// Matroska `MasteringMetadata` / `MaxCLL` / `MaxFALL`.
|
||||
#[derive(Clone, Copy, Debug, Default, PartialEq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct HdrMetadata {
|
||||
pub mastering_max_nits: Option<f32>,
|
||||
pub mastering_min_nits: Option<f32>,
|
||||
pub max_cll: Option<f32>,
|
||||
pub max_fall: Option<f32>,
|
||||
}
|
||||
|
||||
impl HdrMetadata {
|
||||
/// The content's peak for tone mapping: MaxCLL when signalled (the brightest pixel actually
|
||||
/// in the content), else the mastering display's peak; `None` when neither is (0 means
|
||||
/// unknown in both). Never below HDR reference white.
|
||||
pub fn peak_nits(&self) -> Option<f32> {
|
||||
let known = |v: Option<f32>| v.filter(|n| *n > 0.0);
|
||||
let p = match (known(self.max_cll), known(self.mastering_max_nits)) {
|
||||
(Some(c), Some(m)) => c.min(m),
|
||||
(Some(c), None) => c,
|
||||
(None, m) => m?,
|
||||
};
|
||||
Some(p.clamp(transform::REFERENCE_WHITE_NITS as f32, 10_000.0))
|
||||
}
|
||||
}
|
||||
|
||||
impl ColorInfo {
|
||||
pub const SRGB_FULL: ColorInfo = ColorInfo { matrix: Matrix::Bt709, transfer: Transfer::Srgb, primaries: Primaries::Bt709, range: Range::Full };
|
||||
pub const REC709: ColorInfo = ColorInfo { matrix: Matrix::Bt709, transfer: Transfer::Bt709, primaries: Primaries::Bt709, range: Range::Limited };
|
||||
|
||||
@@ -38,7 +38,7 @@ let f = decode_frame_with(&sample, &DecodeOptions { threads: false, ..Default::d
|
||||
let hdr: FrameHeader = probe(&sample)?; // cid, raster, depth, chroma…
|
||||
let name = cid_name(hdr.cid); // "DNxHR HQX", "DNxHD 1080i 8-bit (CID 1242)"
|
||||
|
||||
let mut enc = Encoder::new(Profile::Hq, 1920, 1080); // or Encoder::with_config(EncoderConfig { .. })
|
||||
let mut enc = Encoder::new(Profile::Hq, 1920, 1080)?; // or Encoder::with_config(EncoderConfig { .. })
|
||||
let sample: Vec<u8> = enc.encode(&frame)?; // exactly enc.frame_size() bytes (CBR)
|
||||
```
|
||||
|
||||
|
||||
@@ -19,8 +19,7 @@ pub(crate) fn idct8x8(block: &mut [f32; 64]) {
|
||||
let mut nz = [false; 8];
|
||||
let mut dc_only = true;
|
||||
// Horizontal pass: t[r][x] = Σ_u F[r][u] · B[u][x]
|
||||
for r in 0..8 {
|
||||
let c: &[f32; 8] = block[r * 8..r * 8 + 8].try_into().unwrap();
|
||||
for (r, c) in block.as_chunks::<8>().0.iter().enumerate() {
|
||||
if c[1..].iter().all(|&v| v == 0.0) {
|
||||
if c[0] != 0.0 {
|
||||
t[r] = [c[0] * BASIS[0][0]; 8];
|
||||
|
||||
@@ -61,8 +61,9 @@ struct Prepared {
|
||||
}
|
||||
|
||||
impl Encoder {
|
||||
pub fn new(profile: Profile, width: u32, height: u32) -> Encoder {
|
||||
Encoder::with_config(EncoderConfig::new(profile, width, height)).expect("valid default configuration")
|
||||
/// An encoder with the default configuration; fails on rasters the profile can't code.
|
||||
pub fn new(profile: Profile, width: u32, height: u32) -> Result<Encoder> {
|
||||
Encoder::with_config(EncoderConfig::new(profile, width, height))
|
||||
}
|
||||
|
||||
pub fn with_config(cfg: EncoderConfig) -> Result<Encoder> {
|
||||
@@ -79,7 +80,7 @@ impl Encoder {
|
||||
if !ok_depth {
|
||||
return Err(Error::Input(format!("{} does not support {}-bit", cfg.profile.name(), cfg.bit_depth)));
|
||||
}
|
||||
let info = cid_info(cfg.profile.cid()).expect("RI CID");
|
||||
let info = cid_info(cfg.profile.cid()).ok_or_else(|| Error::Input(format!("no compression table for {}", cfg.profile.name())))?;
|
||||
let frame_size = cfg.profile.frame_size(cfg.width, cfg.height);
|
||||
let header_size = header::header_size_for(cfg.height);
|
||||
Ok(Encoder { cfg, info, frame_size, header_size, last_q: 8 })
|
||||
|
||||
@@ -127,7 +127,7 @@ fn probe_weights() {
|
||||
let px: Vec<f64> = if depth == 8 {
|
||||
out.stdout.iter().map(|&b| b as f64).collect()
|
||||
} else {
|
||||
out.stdout.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]]) as f64).collect()
|
||||
out.stdout.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]]) as f64).collect()
|
||||
};
|
||||
let (off, stride) = if blk == 0 { (0, w as usize) } else { ((w * h) as usize, w as usize / 2) };
|
||||
let mid = (1 << (depth - 1)) as f64;
|
||||
@@ -224,7 +224,7 @@ fn probe_reconstruction() {
|
||||
let px: Vec<f64> = if depth == 8 {
|
||||
out.stdout.iter().map(|&b| b as f64).collect()
|
||||
} else {
|
||||
out.stdout.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]]) as f64).collect()
|
||||
out.stdout.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]]) as f64).collect()
|
||||
};
|
||||
let mid = (1 << (depth - 1)) as f64;
|
||||
let mut sum = 0f64;
|
||||
|
||||
@@ -152,7 +152,7 @@ pub fn parse(d: &[u8]) -> Result<FrameHeader> {
|
||||
2 => ColorVolume::Bt2020Cl,
|
||||
_ => ColorVolume::OutOfBand,
|
||||
};
|
||||
let timecode = (d[0x30] & 0x80 != 0).then(|| d[0x31..0x39].try_into().unwrap());
|
||||
let timecode = if d[0x30] & 0x80 != 0 { d.get(0x31..).and_then(|t| t.first_chunk::<8>()).copied() } else { None };
|
||||
let mb_rows = be16(d, 0x16C);
|
||||
if width == 0 || lines == 0 || width > 16384 || lines > 16384 {
|
||||
return Err(Error::Invalid("raster size"));
|
||||
|
||||
@@ -11,6 +11,8 @@
|
||||
//!
|
||||
//! See the crate README for accuracy, performance and limitations.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
mod bits;
|
||||
mod dct;
|
||||
mod decode;
|
||||
|
||||
@@ -91,7 +91,7 @@ pub fn reference(ff: &Path, mov: &Path, pix_fmt: &str) -> Vec<u16> {
|
||||
.expect("run ffmpeg");
|
||||
assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr));
|
||||
if pix_fmt.ends_with("le") {
|
||||
out.stdout.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
|
||||
out.stdout.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
|
||||
} else {
|
||||
out.stdout.iter().map(|&b| b as u16).collect()
|
||||
}
|
||||
@@ -247,7 +247,7 @@ pub fn ffmpeg_decode(ff: &Path, mov: &Path, pix_fmt: &str) -> Vec<u16> {
|
||||
let err = String::from_utf8_lossy(&out.stderr).to_string();
|
||||
assert!(out.status.success() && err.trim().is_empty(), "ffmpeg failed: {err}");
|
||||
if pix_fmt.ends_with("le") {
|
||||
out.stdout.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
|
||||
out.stdout.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
|
||||
} else {
|
||||
out.stdout.iter().map(|&b| b as u16).collect()
|
||||
}
|
||||
@@ -270,8 +270,11 @@ pub fn source_frames(ff: &Path, name: &str, lavfi: &str, frames: u32, w: u32, h:
|
||||
std::fs::rename(&tmp, &path).unwrap();
|
||||
}
|
||||
let bytes = std::fs::read(&path).unwrap();
|
||||
let s: Vec<u16> =
|
||||
if depth == 8 { bytes.iter().map(|&b| b as u16).collect() } else { bytes.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]])).collect() };
|
||||
let s: Vec<u16> = if depth == 8 {
|
||||
bytes.iter().map(|&b| b as u16).collect()
|
||||
} else {
|
||||
bytes.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
|
||||
};
|
||||
let proto = Frame::new(w, h, chroma, depth, false);
|
||||
let per = frame_samples(&proto);
|
||||
assert_eq!(s.len(), per * frames as usize);
|
||||
|
||||
@@ -68,17 +68,20 @@ fn roundtrip_all_profiles() {
|
||||
|
||||
#[test]
|
||||
fn rejects_bad_input() {
|
||||
let mut enc = Encoder::new(Profile::Hq, 64, 64);
|
||||
let mut enc = Encoder::new(Profile::Hq, 64, 64).unwrap();
|
||||
assert!(enc.encode(&Frame::new(32, 64, ChromaFormat::Yuv422, 8, false)).is_err());
|
||||
assert!(enc.encode(&Frame::new(64, 64, ChromaFormat::Yuv444, 8, false)).is_err());
|
||||
assert!(Encoder::with_config(EncoderConfig { bit_depth: 10, ..EncoderConfig::new(Profile::Hq, 64, 64) }).is_err());
|
||||
assert!(decode_frame(&[0u8; 100]).is_err());
|
||||
assert!(decode_frame(&[0u8; 1000]).is_err());
|
||||
// An unsupported raster is an error, not a panic.
|
||||
assert!(Encoder::new(Profile::Hq, 0, 0).is_err());
|
||||
assert!(Encoder::new(Profile::Hq, 20_000, 64).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn corrupt_streams_never_panic() {
|
||||
let mut enc = Encoder::new(Profile::Hq, 256, 128);
|
||||
let mut enc = Encoder::new(Profile::Hq, 256, 128).unwrap();
|
||||
let good = enc.encode(&picture(256, 128, ChromaFormat::Yuv422, 8, 1)).unwrap();
|
||||
let mut seed = 0x1234_5678u32;
|
||||
let mut rnd = || {
|
||||
|
||||
@@ -105,8 +105,9 @@ pub fn merge_captions(seq: &mut Sequence, ids: &[ClipId]) -> Result<ClipId> {
|
||||
if idx.windows(2).any(|w| w[1] != w[0] + 1) {
|
||||
return Err(EditError::Other("captions to merge must be next to each other".into()));
|
||||
}
|
||||
let first = idx[0];
|
||||
let last = *idx.last().expect("non-empty");
|
||||
let (Some(&first), Some(&last)) = (idx.first(), idx.last()) else {
|
||||
return Err(EditError::Nothing);
|
||||
};
|
||||
let end = tr.captions[last].end();
|
||||
let text: Vec<String> = idx.iter().map(|&i| tr.captions[i].text.trim().to_string()).filter(|t| !t.is_empty()).collect();
|
||||
let keep = tr.captions[first].id;
|
||||
@@ -205,7 +206,7 @@ pub fn move_captions(seq: &mut Sequence, ids: &[ClipId], delta: Tick) -> Result<
|
||||
return Err(EditError::Nothing);
|
||||
}
|
||||
for tid in tracks {
|
||||
let tr = seq.caption_track_mut(tid).expect("checked");
|
||||
let Some(tr) = seq.caption_track_mut(tid) else { continue };
|
||||
for c in tr.captions.iter_mut().filter(|c| ids.contains(&c.id)) {
|
||||
c.start += d;
|
||||
}
|
||||
@@ -319,7 +320,7 @@ mod tests {
|
||||
|
||||
fn ctx(next: &mut u64) -> EditCtx<'_> {
|
||||
static NONE: fn(filmcraft_project::ItemId) -> Option<Tick> = |_| None;
|
||||
EditCtx { next_id: next, media_duration: &NONE, min_duration: Tick(10) }
|
||||
EditCtx { next_id: next, media_duration: &NONE, media_start: &|_| Tick::ZERO, min_duration: Tick(10) }
|
||||
}
|
||||
|
||||
fn spans(seq: &Sequence, tid: TrackId) -> Vec<(i64, i64, String)> {
|
||||
|
||||
+58
-5
@@ -15,6 +15,8 @@
|
||||
//!
|
||||
//! Keyframes are stored in media time, so trims and splits never need to move them.
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
pub mod captions;
|
||||
pub mod multicam;
|
||||
pub mod through;
|
||||
@@ -50,8 +52,12 @@ pub type Result<T> = std::result::Result<T, EditError>;
|
||||
/// Supplies fresh ids and media durations to edits.
|
||||
pub struct EditCtx<'a> {
|
||||
pub next_id: &'a mut u64,
|
||||
/// Media duration of a project item (None = unlimited, e.g. stills / adjustment layers).
|
||||
/// Media duration of a project item (None = unlimited, e.g. stills / adjustment layers): the
|
||||
/// latest media time its clips may show.
|
||||
pub media_duration: &'a dyn Fn(ItemId) -> Option<Tick>,
|
||||
/// The earliest media time clips of a project item may show (a subclip that restricts trims
|
||||
/// starts at its In point; everything else at 0).
|
||||
pub media_start: &'a dyn Fn(ItemId) -> Tick,
|
||||
/// Minimum item duration (one sequence frame).
|
||||
pub min_duration: Tick,
|
||||
}
|
||||
@@ -177,6 +183,34 @@ pub fn remove_orphan_transitions(track: &mut Track) {
|
||||
});
|
||||
}
|
||||
|
||||
/// Where a transition sits: the incoming clip's start, or the outgoing clip's end for a fade to nothing.
|
||||
fn transition_anchor(track: &Track, tr: &Transition) -> Option<Tick> {
|
||||
match (tr.from, tr.to) {
|
||||
(_, Some(to)) => track.item(to).map(|i| i.start),
|
||||
(Some(from), None) => track.item(from).map(|i| i.end()),
|
||||
(None, None) => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Keep transitions on their edit points after an edit moved cuts (ripple trim, roll, slide, ripple
|
||||
/// speed change): each transition shifts by as much as its anchor clip edge moved.
|
||||
fn transitions_follow_cuts(before: &Sequence, after: &mut Sequence) {
|
||||
for track in after.all_tracks_mut() {
|
||||
let Some(old) = before.track(track.id) else { continue };
|
||||
let shifts: Vec<Tick> = track
|
||||
.transitions
|
||||
.iter()
|
||||
.map(|tr| match (transition_anchor(old, tr), transition_anchor(track, tr)) {
|
||||
(Some(a), Some(b)) => b - a,
|
||||
_ => Tick::ZERO,
|
||||
})
|
||||
.collect();
|
||||
for (tr, d) in track.transitions.iter_mut().zip(shifts) {
|
||||
tr.start += d;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn place(track: &mut Track, item: TrackItem) {
|
||||
let idx = track.items.partition_point(|i| i.start <= item.start);
|
||||
track.items.insert(idx, item);
|
||||
@@ -412,6 +446,14 @@ fn media_len(ctx: &EditCtx, item: &TrackItem) -> Option<Tick> {
|
||||
(ctx.media_duration)(item.item)
|
||||
}
|
||||
|
||||
/// Media available before a clip's source In (its head handle, media ticks).
|
||||
fn head(ctx: &EditCtx, item: &TrackItem) -> Tick {
|
||||
if item.frame_hold.is_some() {
|
||||
return item.source_in;
|
||||
}
|
||||
(item.source_in - (ctx.media_start)(item.item)).max(Tick::ZERO)
|
||||
}
|
||||
|
||||
fn src_of(dur: Tick, speed: f64) -> Tick {
|
||||
Tick((dur.0 as f64 * speed.abs()).round() as i64)
|
||||
}
|
||||
@@ -435,7 +477,7 @@ pub fn clamp_trim(seq: &Sequence, clip: ClipId, edge: Edge, mode: TrimMode, delt
|
||||
match edge {
|
||||
Edge::In => {
|
||||
// extending left (d<0) needs media before source_in; shortening needs min duration
|
||||
let max_ext = Tick((it.source_in.0 as f64 / speed).floor() as i64);
|
||||
let max_ext = Tick((head(ctx, it).0 as f64 / speed).floor() as i64);
|
||||
let lo = if mode == TrimMode::Regular { (-(it.start - prev_end)).max(-max_ext) } else { -max_ext };
|
||||
let hi = it.duration - ctx.min_duration;
|
||||
d = d.clamp(if it.frame_hold.is_some() { Tick::MIN } else { lo }, hi);
|
||||
@@ -508,6 +550,7 @@ pub fn trim(seq: &mut Sequence, clip: ClipId, edge: Edge, mode: TrimMode, delta:
|
||||
tr.sort();
|
||||
}
|
||||
}
|
||||
transitions_follow_cuts(seq, &mut work);
|
||||
}
|
||||
work.check().map_err(EditError::Other)?;
|
||||
*seq = work;
|
||||
@@ -585,6 +628,7 @@ pub fn ripple_trim_group(seq: &mut Sequence, clips: &[ClipId], edge: Edge, delta
|
||||
}
|
||||
tr.sort();
|
||||
}
|
||||
transitions_follow_cuts(seq, &mut work);
|
||||
work.check().map_err(EditError::Other)?;
|
||||
*seq = work;
|
||||
Ok(d)
|
||||
@@ -603,11 +647,12 @@ pub fn roll(seq: &mut Sequence, left: ClipId, right: ClipId, delta: Tick, ctx: &
|
||||
if let Some(m) = media_len(ctx, l) {
|
||||
d = d.min(Tick(((m - l.source_out()).0 as f64 / l.speed.abs()).floor() as i64));
|
||||
}
|
||||
d = d.max(-Tick((r.source_in.0 as f64 / r.speed.abs()).floor() as i64));
|
||||
d = d.max(-Tick((head(ctx, r).0 as f64 / r.speed.abs()).floor() as i64));
|
||||
if d == Tick::ZERO {
|
||||
return Ok(d);
|
||||
}
|
||||
let (ls, rs) = (l.speed.abs(), r.speed.abs());
|
||||
let before = seq.clone();
|
||||
{
|
||||
let (_, l) = seq.find_item_mut(left).ok_or(EditError::NoItem(left))?;
|
||||
l.duration += d;
|
||||
@@ -619,6 +664,7 @@ pub fn roll(seq: &mut Sequence, left: ClipId, right: ClipId, delta: Tick, ctx: &
|
||||
r.source_in += src_of(d, rs);
|
||||
}
|
||||
let _ = ls;
|
||||
transitions_follow_cuts(&before, seq);
|
||||
Ok(d)
|
||||
}
|
||||
|
||||
@@ -627,7 +673,8 @@ pub fn slip(seq: &mut Sequence, clip: ClipId, delta_media: Tick, ctx: &mut EditC
|
||||
let (_, it) = seq.find_item(clip).ok_or(EditError::NoItem(clip))?;
|
||||
let used = src_of(it.duration, it.speed);
|
||||
let max_in = media_len(ctx, it).map(|m| m - used).unwrap_or(Tick::MAX);
|
||||
let new_in = (it.source_in + delta_media).clamp(Tick::ZERO, max_in.max(Tick::ZERO));
|
||||
let lo = if it.frame_hold.is_some() { Tick::ZERO } else { (ctx.media_start)(it.item) };
|
||||
let new_in = (it.source_in + delta_media).clamp(lo, max_in.max(lo));
|
||||
let d = new_in - it.source_in;
|
||||
let (_, it) = seq.find_item_mut(clip).ok_or(EditError::NoItem(clip))?;
|
||||
it.source_in = new_in;
|
||||
@@ -653,13 +700,14 @@ pub fn slide(seq: &mut Sequence, clip: ClipId, delta: Tick, ctx: &mut EditCtx) -
|
||||
}
|
||||
if let Some(n) = &next {
|
||||
d = d.min(n.duration - ctx.min_duration);
|
||||
d = d.max(-Tick((n.source_in.0 as f64 / n.speed.abs()).floor() as i64));
|
||||
d = d.max(-Tick((head(ctx, n).0 as f64 / n.speed.abs()).floor() as i64));
|
||||
} else {
|
||||
d = d.min(neighbours(tr, clip).1 - it.end());
|
||||
}
|
||||
if d == Tick::ZERO {
|
||||
return Ok(d);
|
||||
}
|
||||
let before = seq.clone();
|
||||
let t = seq.track_mut(tid).ok_or(EditError::NoTrack(tid))?;
|
||||
if let Some(p) = prev {
|
||||
let pi = t.item_mut(p.id).ok_or(EditError::NoItem(p.id))?;
|
||||
@@ -673,6 +721,7 @@ pub fn slide(seq: &mut Sequence, clip: ClipId, delta: Tick, ctx: &mut EditCtx) -
|
||||
}
|
||||
let me = t.item_mut(clip).ok_or(EditError::NoItem(clip))?;
|
||||
me.start += d;
|
||||
transitions_follow_cuts(&before, seq);
|
||||
Ok(d)
|
||||
}
|
||||
|
||||
@@ -715,6 +764,7 @@ pub fn set_speed(seq: &mut Sequence, clip: ClipId, speed: f64, reverse: bool, ri
|
||||
new_dur = new_dur.min(next_start - it.start);
|
||||
}
|
||||
let delta = new_dur - it.duration;
|
||||
let before = ripple.then(|| seq.clone());
|
||||
{
|
||||
let (_, it) = seq.find_item_mut(clip).ok_or(EditError::NoItem(clip))?;
|
||||
it.speed = speed;
|
||||
@@ -733,6 +783,9 @@ pub fn set_speed(seq: &mut Sequence, clip: ClipId, speed: f64, reverse: bool, ri
|
||||
}
|
||||
}
|
||||
}
|
||||
if let Some(before) = before {
|
||||
transitions_follow_cuts(&before, seq);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ impl Fx {
|
||||
Self { seq, next: 10_000 }
|
||||
}
|
||||
fn ctx<'a>(next: &'a mut u64) -> EditCtx<'a> {
|
||||
EditCtx { next_id: next, media_duration: &media, min_duration: f(1) }
|
||||
EditCtx { next_id: next, media_duration: &media, media_start: &|_| Tick::ZERO, min_duration: f(1) }
|
||||
}
|
||||
fn v(&self, i: usize) -> TrackId {
|
||||
self.seq.video_tracks[i].id
|
||||
@@ -210,6 +210,53 @@ fn roll_slip_slide() {
|
||||
fx.seq.check().unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn transitions_stay_on_their_cuts() {
|
||||
let mut fx = Fx::new();
|
||||
let v1 = fx.v(0);
|
||||
let a = fx.put(v1, 0, 10, 10);
|
||||
let b = fx.put(v1, 10, 10, 10);
|
||||
let c = fx.put(v1, 20, 10, 10);
|
||||
let cross = |id, start, from, to| Transition {
|
||||
id: TransitionId(id),
|
||||
effect: filmcraft_project::find_effect("cross_dissolve").unwrap().instance(),
|
||||
start: f(start),
|
||||
duration: f(4),
|
||||
from,
|
||||
to,
|
||||
align: Default::default(),
|
||||
reverse: false,
|
||||
};
|
||||
let t = fx.seq.track_mut(v1).unwrap();
|
||||
t.transitions = vec![cross(1, 8, Some(a), Some(b)), cross(2, 18, Some(b), Some(c)), cross(3, 26, Some(c), None)];
|
||||
// every transition is centred on its cut (or ends with the clip it fades out)
|
||||
let check = |fx: &Fx, what: &str| {
|
||||
let t = fx.seq.track(v1).unwrap();
|
||||
for tr in &t.transitions {
|
||||
let cut = match tr.to {
|
||||
Some(to) => t.item(to).unwrap().start,
|
||||
None => t.item(tr.from.unwrap()).unwrap().end() - f(2),
|
||||
};
|
||||
assert_eq!(tr.start + f(2), cut, "{what}: transition {:?} left its cut", tr.id);
|
||||
}
|
||||
assert_eq!(t.transitions.len(), 3, "{what}");
|
||||
};
|
||||
let mut n = fx.next;
|
||||
trim(&mut fx.seq, a, Edge::Out, TrimMode::Ripple, f(4), &mut Fx::ctx(&mut n)).unwrap();
|
||||
check(&fx, "ripple trim out");
|
||||
trim(&mut fx.seq, b, Edge::In, TrimMode::Ripple, f(3), &mut Fx::ctx(&mut n)).unwrap();
|
||||
check(&fx, "ripple trim in");
|
||||
ripple_trim_group(&mut fx.seq, &[b], Edge::Out, -f(2), &mut Fx::ctx(&mut n)).unwrap();
|
||||
check(&fx, "ripple trim group");
|
||||
roll(&mut fx.seq, b, c, f(2), &mut Fx::ctx(&mut n)).unwrap();
|
||||
check(&fx, "roll");
|
||||
slide(&mut fx.seq, b, -f(3), &mut Fx::ctx(&mut n)).unwrap();
|
||||
check(&fx, "slide");
|
||||
set_speed(&mut fx.seq, b, 0.5, false, true, &mut Fx::ctx(&mut n)).unwrap();
|
||||
check(&fx, "ripple speed change");
|
||||
fx.seq.check().unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rate_stretch_and_speed() {
|
||||
let mut fx = Fx::new();
|
||||
|
||||
@@ -193,7 +193,7 @@ mod tests {
|
||||
}
|
||||
|
||||
fn ctx(next: &mut u64) -> EditCtx<'_> {
|
||||
EditCtx { next_id: next, media_duration: &|_| None, min_duration: Tick(1) }
|
||||
EditCtx { next_id: next, media_duration: &|_| None, media_start: &|_| Tick::ZERO, min_duration: Tick(1) }
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -344,7 +344,9 @@ pub fn caption_blocks(words: &[SeqWord], rules: &CaptionRules, rate: FrameRate)
|
||||
continue;
|
||||
}
|
||||
if fits_line {
|
||||
*lines.last_mut().expect("non-empty") += 1 + len;
|
||||
if let Some(l) = lines.last_mut() {
|
||||
*l += 1 + len;
|
||||
}
|
||||
} else {
|
||||
lines.push(len);
|
||||
}
|
||||
@@ -362,7 +364,7 @@ pub fn caption_blocks(words: &[SeqWord], rules: &CaptionRules, rate: FrameRate)
|
||||
for g in groups {
|
||||
let mut text_lines: Vec<String> = vec![String::new()];
|
||||
for w in &words[g.clone()] {
|
||||
let l = text_lines.last_mut().expect("non-empty");
|
||||
let Some(l) = text_lines.last_mut() else { break };
|
||||
if l.is_empty() {
|
||||
l.push_str(&w.text);
|
||||
} else if l.chars().count() + 1 + w.text.chars().count() <= max_chars {
|
||||
|
||||
@@ -170,7 +170,7 @@ fn extract_text_closes_the_gap() {
|
||||
let w = sequence_words(&q, &transcripts());
|
||||
let r = word_range(&w, 0, 3, R).unwrap();
|
||||
let mut next = 1000;
|
||||
let mut ctx = EditCtx { next_id: &mut next, media_duration: &media_dur, min_duration: R.frame_duration() };
|
||||
let mut ctx = EditCtx { next_id: &mut next, media_duration: &media_dur, media_start: &|_| Tick::ZERO, min_duration: R.frame_duration() };
|
||||
let tracks: Vec<TrackId> = q.all_tracks().map(|t| t.id).collect();
|
||||
crate::extract(&mut q, &tracks, r, &mut ctx);
|
||||
q.check().unwrap();
|
||||
@@ -192,7 +192,7 @@ fn pauses_are_found_and_removed_in_one_pass() {
|
||||
let p2 = find_pauses(&w, s(0.4), s(0.1), R);
|
||||
assert_eq!(p2.len(), 3);
|
||||
let mut next = 1000;
|
||||
let mut ctx = EditCtx { next_id: &mut next, media_duration: &media_dur, min_duration: R.frame_duration() };
|
||||
let mut ctx = EditCtx { next_id: &mut next, media_duration: &media_dur, media_start: &|_| Tick::ZERO, min_duration: R.frame_duration() };
|
||||
let removed = ripple_delete_ranges(&mut q, p2.clone(), &mut ctx);
|
||||
q.check().unwrap();
|
||||
assert_eq!(removed, p2.iter().map(|r| r.duration).fold(Tick::ZERO, |a, b| a + b));
|
||||
@@ -219,7 +219,7 @@ fn fillers_are_found_with_phrases_and_removed() {
|
||||
// "Um," 3.70–4.00 → nearest frames 3.68–4.00; "uh" 8.20–8.50 → 8.20–8.48
|
||||
assert_eq!(ranges.iter().map(|r| (r.start, r.end())).collect::<Vec<_>>(), vec![(s(3.68), s(4.00)), (s(8.20), s(8.48))]);
|
||||
let mut next = 1000;
|
||||
let mut ctx = EditCtx { next_id: &mut next, media_duration: &media_dur, min_duration: R.frame_duration() };
|
||||
let mut ctx = EditCtx { next_id: &mut next, media_duration: &media_dur, media_start: &|_| Tick::ZERO, min_duration: R.frame_duration() };
|
||||
ripple_delete_ranges(&mut q, ranges, &mut ctx);
|
||||
q.check().unwrap();
|
||||
let w2 = sequence_words(&q, &transcripts());
|
||||
|
||||
@@ -82,6 +82,11 @@ pub struct PlaybackPrefs {
|
||||
/// skip deblocking (faster; those frames are approximate). Never used for exports, renders or
|
||||
/// a paused frame. Off by default.
|
||||
pub draft_decode: bool,
|
||||
/// "Hardware decoding": `auto` (default: the system's hardware decoder for streams it
|
||||
/// supports, e.g. VideoToolbox H.264 / HEVC on macOS, falling back to FilmCraft's own decoder)
|
||||
/// or `off` (always FilmCraft's own decoders). Media already open keeps its decoder until it
|
||||
/// is reopened.
|
||||
pub hardware_decoding: String,
|
||||
}
|
||||
|
||||
impl Default for PlaybackPrefs {
|
||||
@@ -94,6 +99,7 @@ impl Default for PlaybackPrefs {
|
||||
enable_transmit: false,
|
||||
disable_video_in_background: true,
|
||||
draft_decode: false,
|
||||
hardware_decoding: "auto".into(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -424,6 +424,84 @@ fn media_root(p: &Project, id: ItemId) -> Option<(ItemId, &MediaClip, Option<Tim
|
||||
}
|
||||
}
|
||||
|
||||
/// What the Source monitor shows for an item: the media-time span of its time ruler, its frame
|
||||
/// rate, In / Out marks and markers.
|
||||
///
|
||||
/// A subclip that restricts trims shows only its range (the playhead can't leave it); one that
|
||||
/// doesn't shows its parent's whole media with the subclip's range as In / Out. Either way a
|
||||
/// subclip shows the markers of its parent media that fall inside its range (inherited: they
|
||||
/// belong to the master clip, so adding one there shows on every subclip that covers it).
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct SourceView {
|
||||
pub item: ItemId,
|
||||
/// The media item whose frames are shown (the parent of a subclip).
|
||||
pub media: ItemId,
|
||||
pub start: Tick,
|
||||
pub end: Tick,
|
||||
pub rate: FrameRate,
|
||||
pub mark_in: Option<Tick>,
|
||||
pub mark_out: Option<Tick>,
|
||||
pub markers: Vec<filmcraft_project::Marker>,
|
||||
/// For a subclip: its range and whether trims are restricted to it.
|
||||
pub subclip: Option<(TimeRange, bool)>,
|
||||
}
|
||||
|
||||
impl SourceView {
|
||||
pub fn to_json(&self, playhead: Tick) -> Value {
|
||||
json!({
|
||||
"item": self.item.0, "media": self.media.0, "start": self.start.0, "end": self.end.0, "fps": self.rate.as_f64(),
|
||||
"playhead": playhead.0, "markIn": self.mark_in.map(|t| t.0), "markOut": self.mark_out.map(|t| t.0),
|
||||
"markers": self.markers.iter().map(|m| json!({"id": m.id.0, "start": m.start.0, "name": m.name, "duration": m.duration.0})).collect::<Vec<_>>(),
|
||||
"subclip": self.subclip.map(|(r, restrict)| json!({"start": r.start.0, "end": r.end().0, "restrictTrims": restrict})),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// The Source monitor view of `item` (see [`SourceView`]).
|
||||
pub fn source_view(s: &Session, item: ItemId) -> Option<SourceView> {
|
||||
let pi = s.project.item(item)?;
|
||||
let still = |m: &MediaClip| matches!(m.info.kind, filmcraft_media::MediaKind::Still) || m.info.duration.0 <= 0;
|
||||
Some(match &pi.kind {
|
||||
ItemKind::Media(m) => {
|
||||
let rate = m.frame_rate();
|
||||
let end = if still(m) { s.prefs.timeline.still_duration(rate) } else { m.duration() };
|
||||
SourceView { item, media: item, start: Tick::ZERO, end, rate, mark_in: m.mark_in, mark_out: m.mark_out, markers: m.markers.clone(), subclip: None }
|
||||
}
|
||||
ItemKind::Sequence(q) => SourceView {
|
||||
item,
|
||||
media: item,
|
||||
start: Tick::ZERO,
|
||||
end: q.duration(),
|
||||
rate: q.settings.frame_rate,
|
||||
mark_in: q.mark_in,
|
||||
mark_out: q.mark_out,
|
||||
markers: q.markers.clone(),
|
||||
subclip: None,
|
||||
},
|
||||
ItemKind::Subclip { range, restrict_trims, .. } => {
|
||||
let (root, m, _) = media_root(&s.project, item)?;
|
||||
let rate = m.frame_rate();
|
||||
let fd = rate.frame_duration();
|
||||
let markers = m.markers.iter().filter(|k| k.start >= range.start && k.start < range.end()).cloned().collect();
|
||||
let full = if still(m) { range.end() } else { m.duration().max(range.end()) };
|
||||
let (start, end, mi, mo) =
|
||||
if *restrict_trims { (range.start, range.end(), None, None) } else { (Tick::ZERO, full, Some(range.start), Some(range.end() - fd)) };
|
||||
SourceView { item, media: root, start, end, rate, mark_in: mi, mark_out: mo, markers, subclip: Some((*range, *restrict_trims)) }
|
||||
}
|
||||
_ => SourceView {
|
||||
item,
|
||||
media: item,
|
||||
start: Tick::ZERO,
|
||||
end: pi.duration(),
|
||||
rate: pi.frame_rate(),
|
||||
mark_in: None,
|
||||
mark_out: None,
|
||||
markers: Vec::new(),
|
||||
subclip: None,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
/// Frame rate of an item, resolving subclips to their media.
|
||||
fn item_rate(p: &Project, id: ItemId) -> FrameRate {
|
||||
match media_root(p, id) {
|
||||
@@ -679,6 +757,7 @@ fn offline_file(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
color: Default::default(),
|
||||
has_alpha: false,
|
||||
bitrate: None,
|
||||
hdr: None,
|
||||
}),
|
||||
audio: has_a.then(|| filmcraft_media::AudioStreamInfo {
|
||||
sample_rate: u64_p(p, "sampleRate").unwrap_or(48_000) as u32,
|
||||
@@ -1018,7 +1097,7 @@ fn subclip_plan(s: &Session, p: &Value) -> Result<(ItemId, TimeRange, String, La
|
||||
} else {
|
||||
let item = item_p(p, "item").or_else(|| subclip_source(s)).ok_or_else(|| bad("clip.makeSubclip", "open a clip in the Source monitor"))?;
|
||||
let (root, m, sub) = media_root(&s.project, item).ok_or_else(|| bad("clip.makeSubclip", "only media clips can be subclipped"))?;
|
||||
let pi = s.project.item(item).expect("resolved");
|
||||
let pi = s.project.item(item).ok_or_else(|| EngineError::Other("no such item".into()))?;
|
||||
let rate = m.frame_rate();
|
||||
let still = s.prefs.timeline.still_duration(rate);
|
||||
let is_still = matches!(m.info.kind, filmcraft_media::MediaKind::Still);
|
||||
@@ -1103,10 +1182,13 @@ fn edit_subclip(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
s.edit("Edit Subclip", |pr, _| {
|
||||
let it = pr.item_mut(item).ok_or_else(|| bad("clip.editSubclip", "no such item"))?;
|
||||
if convert {
|
||||
// a master clip of the whole media, marked with the subclip's range; the parent's
|
||||
// markers inside that range come along (they were the subclip's inherited markers)
|
||||
let mut m = parent_media.ok_or_else(|| bad("clip.editSubclip", "the subclip's media is gone"))?;
|
||||
m.mark_in = None;
|
||||
m.mark_out = None;
|
||||
m.markers.clear();
|
||||
let fd = m.frame_rate().frame_duration();
|
||||
m.mark_in = Some(a);
|
||||
m.mark_out = Some((b - fd).max(a));
|
||||
m.markers.retain(|k| k.start >= a && k.start < b);
|
||||
it.kind = ItemKind::Media(m);
|
||||
} else {
|
||||
it.kind = ItemKind::Subclip { parent, range: TimeRange::from_bounds(a, b), restrict_trims: restrict };
|
||||
@@ -1164,7 +1246,7 @@ fn audio_channels(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let mut out = Vec::new();
|
||||
let mut maps = Vec::new();
|
||||
for i in &items {
|
||||
let m = s.project.item(*i).and_then(|it| it.as_media()).expect("filtered");
|
||||
let Some(m) = s.project.item(*i).and_then(|it| it.as_media()) else { continue };
|
||||
let n = m.info.audio.as_ref().map_or(2, |a| a.channels) as u16;
|
||||
let fmt = format.unwrap_or_else(|| m.interpret.audio_channels.as_ref().map_or(AudioChannels::Stereo, |a| a.format));
|
||||
let map = match &custom {
|
||||
@@ -1521,7 +1603,7 @@ fn extract_audio(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
})
|
||||
.ok_or_else(|| EngineError::Other("save the project first: extracted audio is written next to it".into()))?,
|
||||
};
|
||||
let pi = s.project.item(item).expect("resolved");
|
||||
let pi = s.project.item(item).ok_or_else(|| EngineError::Other("no such item".into()))?;
|
||||
let name = pi.name.clone();
|
||||
let stem = std::path::Path::new(&name).file_stem().map(|x| x.to_string_lossy().into_owned()).unwrap_or_else(|| name.clone());
|
||||
let src = s.media.full_res_source(&s.project, root, &*s.services).ok_or_else(|| EngineError::Other(format!("{name}: media is offline")))?;
|
||||
|
||||
@@ -614,3 +614,78 @@ fn audio_source_channels_pick_what_a_clip_plays() {
|
||||
it.source_channels = vec![5];
|
||||
assert_eq!(filmcraft_render::audio::source_pair(&it, 2), (0, 0), "missing channels fall back to the first");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn subclip_source_monitor_trims_and_markers() {
|
||||
let mut s = demo();
|
||||
let ocean = item_named(&s, "Ocean_Sunset.mp4");
|
||||
let r24 = filmcraft_time::FrameRate::FPS_23_976;
|
||||
let f = |n: i64| r24.tick_of(n);
|
||||
// clip markers on the master clip at frames 10, 30, 60 and 90
|
||||
let mut p = (*s.project).clone();
|
||||
if let Some(ItemKind::Media(m)) = p.item_mut(ocean).map(|i| &mut i.kind) {
|
||||
for (k, n) in [10, 30, 60, 90].into_iter().enumerate() {
|
||||
m.markers.push(filmcraft_project::Marker {
|
||||
id: filmcraft_project::MarkerId(9000 + k as u64),
|
||||
start: f(n),
|
||||
duration: Tick::ZERO,
|
||||
name: format!("m{n}"),
|
||||
comment: String::new(),
|
||||
kind: filmcraft_project::MarkerKind::Comment,
|
||||
color: Label::Green,
|
||||
});
|
||||
}
|
||||
}
|
||||
s.project = std::sync::Arc::new(p);
|
||||
s.execute("project.setMarks", json!({"item": ocean.0, "in": f(24).0, "out": f(71).0})).unwrap();
|
||||
s.execute("project.select", json!({"items": [ocean.0]})).unwrap();
|
||||
s.state.source_item = None;
|
||||
let sub = ItemId(s.execute("clip.makeSubclip", json!({})).unwrap()["item"].as_u64().unwrap());
|
||||
// the Source monitor shows the subclip's range, opens at its In and keeps the playhead inside
|
||||
s.execute("source.open", json!({"item": sub.0})).unwrap();
|
||||
let v = s.execute("source.inspect", json!({})).unwrap();
|
||||
assert_eq!((v["start"].as_i64(), v["end"].as_i64(), v["playhead"].as_i64()), (Some(f(24).0), Some(f(72).0), Some(f(24).0)));
|
||||
assert_eq!(v["media"], ocean.0);
|
||||
// inherited markers: the master clip's markers inside the range
|
||||
let names: Vec<&str> = v["markers"].as_array().unwrap().iter().map(|m| m["name"].as_str().unwrap()).collect();
|
||||
assert_eq!(names, ["m30", "m60"]);
|
||||
assert_eq!(s.execute("source.setPlayhead", json!({"frame": 0})).unwrap()["time"], f(24).0);
|
||||
assert_eq!(s.execute("source.setPlayhead", json!({"frame": 500})).unwrap()["time"], f(71).0);
|
||||
// edited in on V3 at 100 s: In 24, 48 frames
|
||||
let r = s.execute("timeline.place", json!({"item": sub.0, "track": "V3", "seconds": 100.0})).unwrap();
|
||||
let c = r["clips"][0].as_u64().unwrap();
|
||||
let start = clip(&s, c).start;
|
||||
assert_eq!((clip(&s, c).source_in, clip(&s, c).duration), (f(24), f(48)));
|
||||
// restricted trims: no handles beyond the subclip on either side
|
||||
let trim = |s: &mut Session, edge: &str, d: i64| {
|
||||
s.execute("timeline.trim", json!({"clip": c, "edge": edge, "deltaFrames": d})).unwrap();
|
||||
let it = clip(s, c);
|
||||
(r24.frame_at(it.start - start), r24.frame_at(it.source_in), r24.frame_at(it.duration))
|
||||
};
|
||||
assert_eq!(trim(&mut s, "in", -10), (0, 24, 48), "head stops at the subclip's In");
|
||||
assert_eq!(trim(&mut s, "out", 10), (0, 24, 48), "tail stops at the subclip's Out");
|
||||
assert_eq!(trim(&mut s, "in", 6), (6, 30, 42));
|
||||
assert_eq!(trim(&mut s, "in", -10), (0, 24, 48), "only back to the subclip's In");
|
||||
// slip stays inside the subclip too
|
||||
trim(&mut s, "out", -8);
|
||||
s.execute("timeline.slip", json!({"clip": c, "deltaFrames": -20})).unwrap();
|
||||
assert_eq!(r24.frame_at(clip(&s, c).source_in), 24);
|
||||
s.execute("timeline.slip", json!({"clip": c, "deltaFrames": 20})).unwrap();
|
||||
assert_eq!(r24.frame_at(clip(&s, c).source_in), 32, "In + 40 frames used = the subclip's Out");
|
||||
// without the restriction the master clip's media is the limit (and the monitor shows it all,
|
||||
// with the subclip's range as In / Out)
|
||||
s.execute("project.select", json!({"items": [sub.0]})).unwrap();
|
||||
s.execute("clip.editSubclip", json!({"restrictTrims": false})).unwrap();
|
||||
s.execute("timeline.slip", json!({"clip": c, "deltaFrames": -20})).unwrap();
|
||||
assert_eq!(r24.frame_at(clip(&s, c).source_in), 12);
|
||||
let v = s.execute("source.inspect", json!({})).unwrap();
|
||||
assert_eq!((v["start"].as_i64(), v["markIn"].as_i64(), v["markOut"].as_i64()), (Some(0), Some(f(24).0), Some(f(71).0)));
|
||||
assert_eq!(v["subclip"]["restrictTrims"], false);
|
||||
// Convert to Master Clip: the whole media, marked with the range, keeping the inherited markers
|
||||
s.execute("clip.editSubclip", json!({"convertToMaster": true})).unwrap();
|
||||
let m = s.project.item(sub).unwrap().as_media().unwrap().clone();
|
||||
assert_eq!((m.mark_in, m.mark_out), (Some(f(24)), Some(f(71))));
|
||||
assert_eq!(m.markers.iter().map(|k| k.name.as_str()).collect::<Vec<_>>(), ["m30", "m60"]);
|
||||
s.execute("edit.undo", json!({})).unwrap();
|
||||
assert!(matches!(s.project.item(sub).unwrap().kind, ItemKind::Subclip { restrict_trims: false, .. }));
|
||||
}
|
||||
@@ -4,6 +4,10 @@
|
||||
//!
|
||||
//! Lumetri commands address a clip by `clip` (default: the first selected video clip with
|
||||
//! Lumetri Color, else the first selected video clip — Lumetri is added if missing).
|
||||
//!
|
||||
//! Lumetri Presets (Effects panel): `lumetri.presets` lists them, `lumetri.applyPreset` adds a
|
||||
//! Lumetri Color configured as the preset to clips (one undo step), `lumetri.presetThumbnails`
|
||||
//! renders a folder's thumbnail grid (our Lumetri on the procedural preview picture) to a PNG.
|
||||
|
||||
use filmcraft_color::{ColorSpace, Lut, LutFormat, WorkingSpace};
|
||||
use filmcraft_project::{ClipId, ItemId, ItemKind, ParamValue, ProjectLut, TrackKind};
|
||||
@@ -41,6 +45,14 @@ pub(crate) fn commands() -> Vec<CommandSpec> {
|
||||
interpret,
|
||||
),
|
||||
query("color.spaces", "List Colour Spaces", "{}", spaces),
|
||||
query("lumetri.presets", "List Lumetri Presets", r#"{"folder":str?}"#, list_presets),
|
||||
spec("lumetri.applyPreset", "Apply Lumetri Preset", &[], r#"{"name":str,"clips":[id]?}"#, has_seq, apply_preset),
|
||||
query(
|
||||
"lumetri.presetThumbnails",
|
||||
"Lumetri Preset Thumbnails",
|
||||
r#"{"folder":str?,"names":[str]?,"width":n=160,"columns":n=4,"path":str?}"#,
|
||||
preset_thumbnails,
|
||||
),
|
||||
query("media.colorInfo", "Media Colour Info", r#"{"item":id}"#, color_info),
|
||||
spec("lut.import", "Import LUT…", &[], r#"{"path":str,"name":str?}"#, always, import),
|
||||
query("lut.list", "List LUTs", "{}", list),
|
||||
@@ -323,9 +335,74 @@ fn color_info(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
"effective": effective.map(|c| c.id()),
|
||||
"effectiveLabel": effective.map(|c| c.label()),
|
||||
"hdr": effective.is_some_and(|c| c.is_hdr()),
|
||||
// mastering display / content light level and the peak tone mapping uses
|
||||
"hdrMetadata": m.info.video.as_ref().and_then(|v| v.hdr),
|
||||
"toneMapPeakNits": m.info.video.as_ref().and_then(|v| v.hdr.and_then(|h| h.peak_nits())),
|
||||
}))
|
||||
}
|
||||
|
||||
fn presets_of(p: &Value) -> Vec<filmcraft_render::lumetri_presets::LumetriPreset> {
|
||||
let all = filmcraft_render::lumetri_presets::presets();
|
||||
let names: Option<Vec<String>> = p.get("names").and_then(Value::as_array).map(|a| a.iter().filter_map(|v| v.as_str().map(str::to_lowercase)).collect());
|
||||
let folder = str_p(p, "folder");
|
||||
all.into_iter()
|
||||
.filter(|x| folder.is_none_or(|f| x.folder.eq_ignore_ascii_case(f)))
|
||||
.filter(|x| names.as_ref().is_none_or(|n| n.contains(&x.name.to_lowercase())))
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn list_presets(_: &mut Session, p: &Value) -> Result<Value> {
|
||||
let v: Vec<Value> = presets_of(p).iter().map(|x| json!({"folder": x.folder, "name": x.name, "description": x.description})).collect();
|
||||
Ok(json!({"folders": filmcraft_render::lumetri_presets::FOLDERS, "presets": v}))
|
||||
}
|
||||
|
||||
/// Lumetri Presets ▸ apply: a new Lumetri Color configured as the preset on each video clip
|
||||
/// (standard effects go before the intrinsic ones, like `effects.apply`).
|
||||
fn apply_preset(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let cmd = "lumetri.applyPreset";
|
||||
let name = str_p(p, "name").ok_or_else(|| bad(cmd, "need `name`"))?;
|
||||
let preset = filmcraft_render::lumetri_presets::find(name).ok_or_else(|| bad(cmd, format!("no Lumetri preset `{name}`")))?;
|
||||
let clips = crate::commands::clips_p(s, p);
|
||||
if clips.is_empty() {
|
||||
return Err(bad(cmd, "select clips first"));
|
||||
}
|
||||
let inst = preset.instance();
|
||||
let n = s.edit_sequence(&format!("Apply {}", preset.name), |q, _, _| {
|
||||
let mut n = 0;
|
||||
for t in q.video_tracks.iter_mut() {
|
||||
for it in t.items.iter_mut().filter(|i| clips.contains(&i.id)) {
|
||||
let pos = it.effects.iter().position(|e| e.def().is_some_and(|d| d.intrinsic)).unwrap_or(it.effects.len());
|
||||
it.effects.insert(pos, inst.clone());
|
||||
n += 1;
|
||||
}
|
||||
}
|
||||
Ok(n)
|
||||
})?;
|
||||
if n == 0 {
|
||||
return Err(bad(cmd, "no video clips among the clips"));
|
||||
}
|
||||
Ok(json!({"preset": preset.name, "clips": n}))
|
||||
}
|
||||
|
||||
fn preset_thumbnails(_: &mut Session, p: &Value) -> Result<Value> {
|
||||
let cmd = "lumetri.presetThumbnails";
|
||||
let presets = presets_of(p);
|
||||
if presets.is_empty() {
|
||||
return Err(bad(cmd, "no presets match"));
|
||||
}
|
||||
let w = p.get("width").and_then(Value::as_u64).unwrap_or(160).clamp(16, 1920) as usize;
|
||||
let h = (w * 9 / 16).max(9);
|
||||
let cols = p.get("columns").and_then(Value::as_u64).unwrap_or(4).clamp(1, 32) as usize;
|
||||
let img = filmcraft_render::lumetri_presets::grid(&presets, cols, w, h);
|
||||
let mut out = json!({"presets": presets.iter().map(|x| x.name).collect::<Vec<_>>(), "width": img.w, "height": img.h, "cell": [w, h], "columns": cols});
|
||||
if let Some(path) = str_p(p, "path") {
|
||||
let png = filmcraft_export::encode_png(img.over_black_rgba8(), img.w as u32, img.h as u32).map_err(|e| bad(cmd, e.to_string()))?;
|
||||
std::fs::write(path, png).map_err(|e| bad(cmd, format!("{path}: {e}")))?;
|
||||
out["path"] = json!(path);
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Lumetri ▸ Color Wheels & Match ▸ Apply Match (see `filmcraft_render::color_match`): match the
|
||||
/// clip at the playhead to the sequence frame at the reference time. Sets the three wheels, their
|
||||
/// lightness and Basic saturation in one undo step.
|
||||
|
||||
@@ -33,7 +33,7 @@ fn lumetri_text(s: &Session, clip: ClipId, p: &str) -> String {
|
||||
fn mean(img: &filmcraft_render::Image) -> [f32; 3] {
|
||||
assert!(img.px.iter().any(|v| *v > 0.01), "blank render");
|
||||
let mut m = [0f64; 3];
|
||||
for p in img.px.chunks_exact(4) {
|
||||
for p in img.px.as_chunks::<4>().0 {
|
||||
for k in 0..3 {
|
||||
m[k] += p[k] as f64;
|
||||
}
|
||||
@@ -194,3 +194,63 @@ fn interpret_footage_and_sequence_colour_settings() {
|
||||
assert_eq!(spaces["colorSpaces"].as_array().unwrap().len(), filmcraft_color::ColorSpace::ALL.len());
|
||||
assert!(s.execute("sequence.colorSettings", json!({"workingSpace": "xyz"})).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lumetri_presets_list_apply_and_thumbnails() {
|
||||
let mut s = demo();
|
||||
let clip = pick_clip(&mut s);
|
||||
let r = s.execute("lumetri.presets", json!({})).unwrap();
|
||||
assert_eq!(r["folders"], json!(["Cinematic", "Film Emulation", "Monochrome", "Technical"]));
|
||||
let mono = s.execute("lumetri.presets", json!({"folder": "monochrome"})).unwrap();
|
||||
assert!(mono["presets"].as_array().unwrap().iter().all(|p| p["folder"] == "Monochrome"));
|
||||
// apply: one undo step adding a configured Lumetri Color before the intrinsic effects
|
||||
let before = s.active_sequence().unwrap().find_item(clip).unwrap().1.effects.len();
|
||||
let r = s.execute("lumetri.applyPreset", json!({"name": "Neutral Mono"})).unwrap();
|
||||
assert_eq!(r["clips"], 1);
|
||||
let it = s.active_sequence().unwrap().find_item(clip).unwrap().1.clone();
|
||||
assert_eq!(it.effects.len(), before + 1);
|
||||
let k = it.effects.iter().position(|e| e.effect == "lumetri").unwrap();
|
||||
assert!(it.effects[k + 1..].iter().all(|e| e.def().is_some_and(|d| d.intrinsic)) || k + 1 == it.effects.len());
|
||||
assert_eq!(it.effects[k].param("saturation").unwrap().value, ParamValue::Float(0.0));
|
||||
let img = s.render_program(0.25).unwrap();
|
||||
let grey = img.px.as_chunks::<4>().0.iter().filter(|p| p[3] > 0.5).all(|p| (p[0] - p[1]).abs() < 0.01 && (p[1] - p[2]).abs() < 0.01);
|
||||
assert!(grey, "the program is black and white");
|
||||
s.execute("edit.undo", json!({})).unwrap();
|
||||
assert_eq!(s.active_sequence().unwrap().find_item(clip).unwrap().1.effects.len(), before);
|
||||
assert!(s.execute("lumetri.applyPreset", json!({"name": "No Such Look"})).is_err());
|
||||
// thumbnails of a folder, written as a PNG
|
||||
let path = tmp("lumetri-presets.png");
|
||||
let r = s.execute("lumetri.presetThumbnails", json!({"folder": "Technical", "width": 64, "columns": 3, "path": path})).unwrap();
|
||||
let n = r["presets"].as_array().unwrap().len();
|
||||
assert_eq!((r["width"].as_u64(), r["height"].as_u64()), (Some(3 * 68 + 4), Some((n.div_ceil(3) * 40 + 4) as u64)));
|
||||
assert!(std::fs::read(&path).unwrap().starts_with(b"\x89PNG"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hdr_lumetri_and_mastering_metadata_through_commands() {
|
||||
let mut s = demo();
|
||||
let clip = pick_clip(&mut s);
|
||||
s.execute("sequence.colorSettings", json!({"workingSpace": "rec2100-pq"})).unwrap();
|
||||
s.execute("effects.apply", json!({"clips": [clip.0], "effect": "lumetri"})).unwrap();
|
||||
let idx = s.active_sequence().unwrap().find_item(clip).unwrap().1.effects.iter().position(|e| e.effect == "lumetri").unwrap();
|
||||
// +1 stop doubles the light of the (SDR) clip in the PQ working space: no clipping at
|
||||
// 203 cd/m², the HDR working output keeps the values above reference white
|
||||
let base = s.render_program_working(0.25).unwrap();
|
||||
s.execute("effects.setParam", json!({"clip": clip.0, "effect": idx, "param": "exposure", "value": 1.0})).unwrap();
|
||||
let up = s.render_program_working(0.25).unwrap();
|
||||
let (m0, m1) = (mean(&base), mean(&up));
|
||||
for k in 0..3 {
|
||||
assert!((m1[k] / m0[k] - 2.0).abs() < 0.08, "channel {k}: {m0:?} → {m1:?}");
|
||||
}
|
||||
assert!(up.px.as_chunks::<4>().0.iter().any(|p| p[1] > 1.2), "values above reference white survive");
|
||||
// HDR White / HDR Specular / HDR Range are ordinary Lumetri parameters
|
||||
s.execute("effects.setParam", json!({"clip": clip.0, "effect": idx, "param": "hdr_white", "value": 2000.0})).unwrap();
|
||||
s.execute("effects.setParam", json!({"clip": clip.0, "effect": idx, "param": "hdr_specular", "value": -50.0})).unwrap();
|
||||
s.execute("effects.setParam", json!({"clip": clip.0, "effect": idx, "param": "curves_hdr_range", "value": 4000.0})).unwrap();
|
||||
let it = s.active_sequence().unwrap().find_item(clip).unwrap().1.clone();
|
||||
assert_eq!(it.effects[idx].param("hdr_white").unwrap().value, ParamValue::Float(2000.0));
|
||||
// media.colorInfo reports the HDR metadata (none for the demo's generated media)
|
||||
let item = it.item;
|
||||
let r = s.execute("media.colorInfo", json!({"item": item.0})).unwrap();
|
||||
assert!(r["hdrMetadata"].is_null() && r["toneMapPeakNits"].is_null(), "{r}");
|
||||
}
|
||||
@@ -121,6 +121,15 @@ pub(crate) fn str_p<'a>(p: &'a Value, k: &str) -> Option<&'a str> {
|
||||
pub(crate) fn f64_p(p: &Value, k: &str) -> Option<f64> {
|
||||
p.get(k).and_then(Value::as_f64)
|
||||
}
|
||||
/// `fps` as a sequence frame rate; zero or negative rates are refused (all frame maths divides by them).
|
||||
fn fps_p(p: &Value, cmd: &str) -> Result<Option<FrameRate>> {
|
||||
let Some(fps) = f64_p(p, "fps") else { return Ok(None) };
|
||||
let r = FrameRate::from_f64(fps);
|
||||
if r.num <= 0 || r.frame_duration() <= Tick::ZERO {
|
||||
return Err(bad(cmd, format!("fps must be a positive frame rate, got {fps}")));
|
||||
}
|
||||
Ok(Some(r))
|
||||
}
|
||||
pub(crate) fn bool_p(p: &Value, k: &str) -> Option<bool> {
|
||||
p.get(k).and_then(Value::as_bool)
|
||||
}
|
||||
@@ -353,7 +362,7 @@ pub(crate) fn place_item(
|
||||
let frame = (seq.settings.width, seq.settings.height);
|
||||
let mut placements = Vec::new();
|
||||
let link = if has_v && has_a { Some(p.alloc_id()) } else { None };
|
||||
if has_v {
|
||||
if let Some(vdest) = vdest.filter(|_| has_v) {
|
||||
let mut v = p.make_track_item(item, TrackKind::Video, at, range, rate).ok_or_else(|| bad(label, "bad item"))?;
|
||||
v.link = link;
|
||||
for e in &mut v.effects {
|
||||
@@ -362,10 +371,9 @@ pub(crate) fn place_item(
|
||||
if is_media && src_size != frame {
|
||||
crate::settings::apply_media_scaling(&mut v, &scaling, frame, src_size);
|
||||
}
|
||||
placements.push((vdest.expect("checked"), v));
|
||||
placements.push((vdest, v));
|
||||
}
|
||||
if has_a {
|
||||
let adest = adest.expect("checked");
|
||||
if let Some(adest) = adest.filter(|_| has_a) {
|
||||
let (arange, aat) = audio_split.unwrap_or((range, at));
|
||||
let mut a = p.make_track_item(item, TrackKind::Audio, aat, arange, rate).ok_or_else(|| bad(label, "bad item"))?;
|
||||
// Modify ▸ Audio Channels with several audio clips: one per clip, on the tracks below
|
||||
@@ -387,12 +395,14 @@ pub(crate) fn place_item(
|
||||
placements.push((*tid, b));
|
||||
}
|
||||
}
|
||||
let snapshot = p.clone();
|
||||
let snapshot = std::sync::Arc::new(p.clone());
|
||||
let snap = snapshot.clone();
|
||||
let durations = move |id: ItemId| crate::media_duration(&snapshot, &media, id);
|
||||
let starts = move |id: ItemId| crate::media_start(&snap, id);
|
||||
let min = rate.frame_duration();
|
||||
let mut next = p.next_id;
|
||||
let seq = p.sequence_mut(seq_id).ok_or(EngineError::NoSequence)?;
|
||||
let mut ctx = edit::EditCtx { next_id: &mut next, media_duration: &durations, min_duration: min };
|
||||
let mut ctx = edit::EditCtx { next_id: &mut next, media_duration: &durations, media_start: &starts, min_duration: min };
|
||||
let span = placements.iter().map(|x| x.1.start).min().zip(placements.iter().map(|x| x.1.end()).max());
|
||||
let ids = if insert { edit::insert(seq, placements, &mut ctx)? } else { edit::overwrite(seq, placements, &mut ctx)? };
|
||||
if insert
|
||||
@@ -506,8 +516,8 @@ fn build() -> Vec<CommandSpec> {
|
||||
if let Some(h) = u64_p(p, "height") {
|
||||
st.height = h as u32;
|
||||
}
|
||||
if let Some(fps) = f64_p(p, "fps") {
|
||||
st.frame_rate = FrameRate::from_f64(fps);
|
||||
if let Some(fps) = fps_p(p, "file.newSequence")? {
|
||||
st.frame_rate = fps;
|
||||
}
|
||||
if let Some(sr) = u64_p(p, "sampleRate") {
|
||||
st.sample_rate = sr as u32;
|
||||
@@ -664,27 +674,27 @@ fn build() -> Vec<CommandSpec> {
|
||||
Ok(id) => ids.push(id.0),
|
||||
Err(e) => errors.push(format!("{path}: {e}")),
|
||||
},
|
||||
Ok(b) if crate::captions::detect(&path, &b).is_some() => {
|
||||
let fmt = crate::captions::detect(&path, &b).expect("detected");
|
||||
match crate::captions::import(s, &path, &b, fmt, None) {
|
||||
Ok(r) => reports.push(r),
|
||||
Err(e) => errors.push(format!("{path}: {e}")),
|
||||
}
|
||||
}
|
||||
Ok(b) if crate::interchange::detect(&path, &b).is_some() => {
|
||||
let fmt = crate::interchange::detect(&path, &b).expect("detected");
|
||||
match crate::interchange::import(s, &path, &b, fmt) {
|
||||
Ok(r) => {
|
||||
sequences.extend(r["sequences"].as_array().cloned().unwrap_or_default());
|
||||
reports.push(r);
|
||||
Ok(b) => {
|
||||
if let Some(fmt) = crate::captions::detect(&path, &b) {
|
||||
match crate::captions::import(s, &path, &b, fmt, None) {
|
||||
Ok(r) => reports.push(r),
|
||||
Err(e) => errors.push(format!("{path}: {e}")),
|
||||
}
|
||||
} else if let Some(fmt) = crate::interchange::detect(&path, &b) {
|
||||
match crate::interchange::import(s, &path, &b, fmt) {
|
||||
Ok(r) => {
|
||||
sequences.extend(r["sequences"].as_array().cloned().unwrap_or_default());
|
||||
reports.push(r);
|
||||
}
|
||||
Err(e) => errors.push(format!("{path}: {e}")),
|
||||
}
|
||||
} else {
|
||||
match import_bytes(s, &path, b.into(), bin) {
|
||||
Ok(id) => ids.push(id.0),
|
||||
Err(e) => errors.push(format!("{path}: {e}")),
|
||||
}
|
||||
Err(e) => errors.push(format!("{path}: {e}")),
|
||||
}
|
||||
}
|
||||
Ok(b) => match import_bytes(s, &path, b.into(), bin) {
|
||||
Ok(id) => ids.push(id.0),
|
||||
Err(e) => errors.push(format!("{path}: {e}")),
|
||||
},
|
||||
Err(e) => errors.push(format!("{path}: {e}")),
|
||||
}
|
||||
}
|
||||
@@ -1259,6 +1269,7 @@ fn build() -> Vec<CommandSpec> {
|
||||
has_seq,
|
||||
|s, p| {
|
||||
let id = s.state.active_sequence.ok_or(EngineError::NoSequence)?;
|
||||
let fps = fps_p(p, "sequence.settings")?;
|
||||
let mix = match str_p(p, "mix") {
|
||||
Some(m) => Some(
|
||||
crate::mixer::channels_from(m).ok_or_else(|| bad("sequence.settings", format!("unknown mix `{m}` (Stereo, Mono, 5.1, Adaptive)")))?,
|
||||
@@ -1268,7 +1279,7 @@ fn build() -> Vec<CommandSpec> {
|
||||
let p = p.clone();
|
||||
s.edit("Sequence Settings", |pr, _| {
|
||||
if let Some(n) = str_p(&p, "name") {
|
||||
pr.item_mut(id).expect("seq").name = n.to_string();
|
||||
pr.item_mut(id).ok_or(EngineError::NoSequence)?.name = n.to_string();
|
||||
}
|
||||
let q = pr.sequence_mut(id).ok_or(EngineError::NoSequence)?;
|
||||
if let Some(w) = u64_p(&p, "width") {
|
||||
@@ -1277,8 +1288,8 @@ fn build() -> Vec<CommandSpec> {
|
||||
if let Some(h) = u64_p(&p, "height") {
|
||||
q.settings.height = h as u32;
|
||||
}
|
||||
if let Some(f) = f64_p(&p, "fps") {
|
||||
q.settings.frame_rate = FrameRate::from_f64(f);
|
||||
if let Some(f) = fps {
|
||||
q.settings.frame_rate = f;
|
||||
}
|
||||
if let Some(sr) = u64_p(&p, "sampleRate") {
|
||||
q.settings.sample_rate = sr as u32;
|
||||
@@ -1514,13 +1525,15 @@ fn build() -> Vec<CommandSpec> {
|
||||
.ok_or_else(|| bad("source.open", "need `item` (or a Project panel selection)"))?;
|
||||
s.project.item(id).ok_or_else(|| bad("source.open", "no such item"))?;
|
||||
s.state.source_item = Some(id);
|
||||
s.state.source_playhead = Tick::ZERO;
|
||||
// a subclip opens at its In point
|
||||
s.state.source_playhead = crate::clip_ops::source_view(s, id).map(|v| v.start).unwrap_or_default();
|
||||
s.events.push(crate::Event::OpenSource(id));
|
||||
Ok(Value::Null)
|
||||
}),
|
||||
cmd!("source.setPlayhead", "Set Source Playhead", [], None, r#"{"time":ticks|"frame":i64|"seconds":f64}"#, always, |s, p| {
|
||||
let item = s.state.source_item.ok_or_else(|| EngineError::Other("no source clip".into()))?;
|
||||
let rate = s.project.item(item).map(|i| i.frame_rate()).unwrap_or_default();
|
||||
let view = crate::clip_ops::source_view(s, item);
|
||||
let rate = view.as_ref().map(|v| v.rate).unwrap_or_default();
|
||||
let t = p
|
||||
.get("time")
|
||||
.and_then(Value::as_i64)
|
||||
@@ -1528,10 +1541,16 @@ fn build() -> Vec<CommandSpec> {
|
||||
.or_else(|| p.get("frame").and_then(Value::as_i64).map(|f| rate.tick_of(f)))
|
||||
.or_else(|| f64_p(p, "seconds").map(Tick::from_seconds_f64))
|
||||
.unwrap_or_default();
|
||||
let dur = s.project.item(item).map(|i| i.duration()).unwrap_or(Tick::MAX);
|
||||
s.state.source_playhead = rate.snap(t.clamp(Tick::ZERO, (dur - rate.frame_duration()).max(Tick::ZERO)));
|
||||
// the Source monitor's span: a subclip that restricts trims keeps the playhead inside it
|
||||
let (lo, hi) = view.map(|v| (v.start, v.end)).unwrap_or((Tick::ZERO, Tick::MAX));
|
||||
s.state.source_playhead = rate.snap(t.clamp(lo, (hi - rate.frame_duration()).max(lo))).max(lo);
|
||||
Ok(json!({"time": s.state.source_playhead.0}))
|
||||
}),
|
||||
query!("source.inspect", "Inspect Source Monitor", "{}", |s, _| {
|
||||
let Some(item) = s.state.source_item else { return Ok(json!({"item": null})) };
|
||||
let v = crate::clip_ops::source_view(s, item).ok_or_else(|| bad("source.inspect", "the source item is gone"))?;
|
||||
Ok(v.to_json(s.state.source_playhead))
|
||||
}),
|
||||
cmd!("source.insert", "Insert", ["Clip"], Some(","), "{}", has_source, |s, _| edit_at_source(s, true)),
|
||||
cmd!("source.overwrite", "Overwrite", ["Clip"], Some("."), "{}", has_source, |s, _| edit_at_source(s, false)),
|
||||
// ================= Timeline gestures =================
|
||||
@@ -1773,7 +1792,21 @@ fn build() -> Vec<CommandSpec> {
|
||||
.map(Tick)
|
||||
.or_else(|| p.get("deltaFrames").and_then(Value::as_i64).map(|f| s.sequence_rate().tick_of(f)))
|
||||
.unwrap_or_default();
|
||||
s.edit_sequence("Slide", |q, ctx, _| Ok(edit::slide(q, c, d, ctx)?))?;
|
||||
let clips = with_links(s, &[c]);
|
||||
s.edit_sequence("Slide", |q, ctx, _| {
|
||||
// clamp across all linked partners, then slide them by the common delta
|
||||
let mut dd = d;
|
||||
for c in &clips {
|
||||
let x = edit::slide(&mut q.clone(), *c, dd, ctx)?;
|
||||
if x.abs() < dd.abs() {
|
||||
dd = x;
|
||||
}
|
||||
}
|
||||
for c in &clips {
|
||||
edit::slide(q, *c, dd, ctx)?;
|
||||
}
|
||||
Ok(())
|
||||
})?;
|
||||
Ok(Value::Null)
|
||||
}),
|
||||
cmd!("timeline.rateStretch", "Rate Stretch", [], None, r#"{"clip":id,"edge":"in|out","delta":ticks}"#, has_seq, |s, p| {
|
||||
@@ -2420,7 +2453,7 @@ fn nest(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
s.edit("Nest", |pr, st| {
|
||||
let nid = pr.new_sequence(&name, q.settings.clone(), q.video_tracks.len(), q.audio_tracks.len(), None);
|
||||
{
|
||||
let nq = pr.sequence_mut(nid).expect("new");
|
||||
let nq = pr.sequence_mut(nid).ok_or(EngineError::NoSequence)?;
|
||||
for (k, ti, it) in &items {
|
||||
let mut it = it.clone();
|
||||
it.start -= start;
|
||||
@@ -2431,11 +2464,15 @@ fn nest(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
}
|
||||
}
|
||||
let rate = q.settings.frame_rate;
|
||||
let mut v = pr.make_track_item(nid, TrackKind::Video, start, TimeRange::new(Tick::ZERO, end - start), rate).expect("item");
|
||||
let mut v = pr
|
||||
.make_track_item(nid, TrackKind::Video, start, TimeRange::new(Tick::ZERO, end - start), rate)
|
||||
.ok_or_else(|| EngineError::Other("cannot place the nested sequence".into()))?;
|
||||
for e in &mut v.effects {
|
||||
resolve_auto_points(e, (q.settings.width, q.settings.height), (q.settings.width, q.settings.height));
|
||||
}
|
||||
let a = pr.make_track_item(nid, TrackKind::Audio, start, TimeRange::new(Tick::ZERO, end - start), rate).expect("item");
|
||||
let a = pr
|
||||
.make_track_item(nid, TrackKind::Audio, start, TimeRange::new(Tick::ZERO, end - start), rate)
|
||||
.ok_or_else(|| EngineError::Other("cannot place the nested sequence".into()))?;
|
||||
let link = pr.alloc_id();
|
||||
let seq = pr.sequence_mut(seq_id).ok_or(EngineError::NoSequence)?;
|
||||
edit::delete_items(seq, &sel);
|
||||
|
||||
+20
-11
@@ -44,6 +44,15 @@ pub fn add_generator(p: &mut Project, pool: &MediaPool, src: GeneratorSource, na
|
||||
|
||||
/// Build the demo project. Returns (project, main sequence).
|
||||
pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
|
||||
build_demo(pool).unwrap_or_else(|| {
|
||||
// Only reachable if the built-in items could not be placed: an empty demo, not a crash.
|
||||
let mut p = Project::new("FilmCraft Demo");
|
||||
let seq = p.new_sequence("Main Edit", SequenceSettings::default(), 3, 3, None);
|
||||
(p, seq)
|
||||
})
|
||||
}
|
||||
|
||||
fn build_demo(pool: &MediaPool) -> Option<(Project, ItemId)> {
|
||||
let mut p = Project::new("FilmCraft Demo");
|
||||
let footage = p.add_bin("Footage", None);
|
||||
let audio_bin = p.add_bin("Audio", None);
|
||||
@@ -108,8 +117,8 @@ pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
|
||||
for (ci, sin, dur) in cuts {
|
||||
let src_range = TimeRange::new(r.snap(Tick::from_seconds_f64(sin)), r.snap(Tick::from_seconds_f64(dur)));
|
||||
let link = p.alloc_id();
|
||||
let mut v = p.make_track_item(clips[ci], TrackKind::Video, t, src_range, r).expect("item");
|
||||
let mut a = p.make_track_item(clips[ci], TrackKind::Audio, t, src_range, r).expect("item");
|
||||
let mut v = p.make_track_item(clips[ci], TrackKind::Video, t, src_range, r)?;
|
||||
let mut a = p.make_track_item(clips[ci], TrackKind::Audio, t, src_range, r)?;
|
||||
v.link = Some(link);
|
||||
a.link = Some(link);
|
||||
for e in &mut v.effects {
|
||||
@@ -118,14 +127,14 @@ pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
|
||||
a.effect_mut("volume").map(|e| e.params.get_mut("level").map(|l| l.value = ParamValue::Float(-8.0)));
|
||||
let dur_t = v.duration;
|
||||
placed.push((v.id, a.id, t, dur_t));
|
||||
let s = p.sequence_mut(seq).expect("seq");
|
||||
let s = p.sequence_mut(seq)?;
|
||||
s.video_tracks[0].items.push(v);
|
||||
s.audio_tracks[0].items.push(a);
|
||||
t += dur_t;
|
||||
}
|
||||
// A lower-third-ish overlay on V2: the plasma clip scaled down in the corner with a drop shadow.
|
||||
let ov_range = TimeRange::new(Tick::ZERO, r.snap(Tick::from_seconds_f64(4.0)));
|
||||
let mut ov = p.make_track_item(clips[4], TrackKind::Video, r.snap(Tick::from_seconds_f64(6.0)), ov_range, r).expect("item");
|
||||
let mut ov = p.make_track_item(clips[4], TrackKind::Video, r.snap(Tick::from_seconds_f64(6.0)), ov_range, r)?;
|
||||
for e in &mut ov.effects {
|
||||
resolve_auto_points(e, (1920, 1080), (1920, 1080));
|
||||
}
|
||||
@@ -152,18 +161,18 @@ pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
|
||||
lum.params.get_mut("temperature").map(|v| v.value = ParamValue::Float(18.0));
|
||||
lum.params.get_mut("contrast").map(|v| v.value = ParamValue::Float(22.0));
|
||||
lum.params.get_mut("vignette_amount").map(|v| v.value = ParamValue::Float(-1.4));
|
||||
let s = p.sequence_mut(seq).expect("seq");
|
||||
let s = p.sequence_mut(seq)?;
|
||||
if let Some(it) = s.video_tracks[0].items.get_mut(3) {
|
||||
it.effects.insert(0, lum);
|
||||
}
|
||||
}
|
||||
let s = p.sequence_mut(seq).expect("seq");
|
||||
let s = p.sequence_mut(seq)?;
|
||||
s.video_tracks[1].items.push(ov);
|
||||
// Transitions: cross dissolve between shots 1-2, dip to black 3-4, push 5-6, fade in at start.
|
||||
let tr_len = r.tick_of(24);
|
||||
let mk = |id: u64, eff: &str, from: Option<filmcraft_project::ClipId>, to: Option<filmcraft_project::ClipId>, start: Tick| Transition {
|
||||
id: TransitionId(id),
|
||||
effect: find_effect(eff).expect("fx").instance(),
|
||||
effect: crate::presets::effect_instance(eff),
|
||||
start,
|
||||
duration: tr_len,
|
||||
from,
|
||||
@@ -181,7 +190,7 @@ pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
|
||||
s.video_tracks[0].transitions = trs;
|
||||
s.audio_tracks[0].transitions.push(Transition {
|
||||
id: TransitionId(900_010),
|
||||
effect: find_effect("constant_power").expect("fx").instance(),
|
||||
effect: crate::presets::effect_instance("constant_power"),
|
||||
start: cut(1) - tr_len.mul_ratio(1, 2),
|
||||
duration: tr_len,
|
||||
from: Some(placed[0].1),
|
||||
@@ -207,13 +216,13 @@ pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
|
||||
});
|
||||
}
|
||||
// Music on A2 (under the whole edit), ducked -14 dB.
|
||||
let mut m = p.make_track_item(music, TrackKind::Audio, Tick::ZERO, TimeRange::new(Tick::ZERO, end), r).expect("music");
|
||||
let mut m = p.make_track_item(music, TrackKind::Audio, Tick::ZERO, TimeRange::new(Tick::ZERO, end), r)?;
|
||||
m.effect_mut("volume").map(|e| e.params.get_mut("level").map(|l| l.value = ParamValue::Float(-14.0)));
|
||||
m.label = Label::Caribbean;
|
||||
let s = p.sequence_mut(seq).expect("seq");
|
||||
let s = p.sequence_mut(seq)?;
|
||||
s.audio_tracks[1].items.push(m);
|
||||
for tr in s.all_tracks_mut() {
|
||||
tr.sort();
|
||||
}
|
||||
(p, seq)
|
||||
Some((p, seq))
|
||||
}
|
||||
@@ -411,6 +411,9 @@ fn ranges_resolve_and_sidecar_captions_are_written() {
|
||||
assert_eq!(frames(&mut s, json!({"preset": "PNG Sequence"})), io, "In/Out is the default");
|
||||
assert!(s.execute("export.resolve", json!({"range": "workArea"})).is_err(), "no work area");
|
||||
assert_eq!(frames(&mut s, json!({"preset": "Animated GIF 640×360", "range": "custom", "startSeconds": 0, "endSeconds": 2})), 30, "15 fps");
|
||||
let bare = frames(&mut s, json!({"preset": "Animated GIF 640×360", "startSeconds": 0, "endSeconds": 2}));
|
||||
assert_eq!(bare, 30, "start/end times without `range` are a custom range, not the In/Out or whole sequence");
|
||||
assert!(s.execute("export.resolve", json!({"startSeconds": 1})).is_err(), "a start without an end is an error");
|
||||
let res = s.execute("export.resolve", json!({"preset": "YouTube 2160p 4K Ultra HD"})).unwrap();
|
||||
assert_eq!(res["output"]["width"], 3840);
|
||||
assert!(res["summary"]["video"].as_str().unwrap().contains("3840x2160"));
|
||||
@@ -427,3 +430,18 @@ fn ranges_resolve_and_sidecar_captions_are_written() {
|
||||
// a bad preset name is a clean error
|
||||
assert!(s.execute("file.exportMedia", json!({"preset": "Nope", "path": out})).is_err());
|
||||
}
|
||||
|
||||
/// Export mode asks for a default folder as soon as it opens. With no saved project and no
|
||||
/// HOME (the web build) this used to reach `std::env::temp_dir()`, which panics on wasm32 and
|
||||
/// killed the web app whenever the Export tab was clicked.
|
||||
#[test]
|
||||
fn default_export_dir_never_needs_a_real_filesystem() {
|
||||
let s = Session::default();
|
||||
let d = crate::export_tools::default_export_dir(&s);
|
||||
assert!(!d.as_os_str().is_empty());
|
||||
// the shared temp-dir helper is what every former `std::env::temp_dir()` call goes through
|
||||
assert!(!crate::temp_dir().as_os_str().is_empty());
|
||||
// a saved project exports next to itself
|
||||
let s = Session { path: Some("/projects/show/edit.fcproj".into()), ..Default::default() };
|
||||
assert_eq!(crate::export_tools::default_export_dir(&s), std::path::PathBuf::from("/projects/show"));
|
||||
}
|
||||
@@ -244,12 +244,14 @@ pub fn sequence_param(s: &Session, p: &Value, cmd: &str) -> Result<ItemId> {
|
||||
}
|
||||
|
||||
/// Export ▸ Range: `entire` | `inOut` | `workArea` | `custom` (`start…` / `end…` times).
|
||||
/// None = the default (In/Out when set, else the whole sequence).
|
||||
/// None = the default (In/Out when set, else the whole sequence); bare `start…` / `end…` times
|
||||
/// without a `range` mean `custom`.
|
||||
pub fn range_param(s: &Session, project: &Project, seq: ItemId, v: Option<&Value>, p: &Value, cmd: &str) -> Result<Option<TimeRange>> {
|
||||
let q = project.sequence(seq).ok_or(EngineError::NoSequence)?;
|
||||
let fd = q.settings.frame_rate.frame_duration();
|
||||
let whole = TimeRange::from_bounds(Tick::ZERO, q.duration().max(fd));
|
||||
let mode = match v {
|
||||
None | Some(Value::Null) if time_p(s, p, "start").is_some() || time_p(s, p, "end").is_some() => "custom".to_string(),
|
||||
None | Some(Value::Null) => return Ok(None),
|
||||
Some(Value::String(m)) => m.to_ascii_lowercase().replace(['-', '_', ' ', '/'], ""),
|
||||
Some(o @ Value::Object(_)) => return range_param(s, project, seq, o.get("mode").or(Some(&json!("custom"))), o, cmd),
|
||||
@@ -281,11 +283,15 @@ pub fn default_export_dir(s: &Session) -> PathBuf {
|
||||
if let Some(d) = s.path.as_deref().and_then(|p| Path::new(p).parent()).filter(|d| !d.as_os_str().is_empty()) {
|
||||
return d.to_path_buf();
|
||||
}
|
||||
if cfg!(target_arch = "wasm32") {
|
||||
// the web build's virtual file table: written files are offered as downloads
|
||||
return PathBuf::from("/exports");
|
||||
}
|
||||
if let Ok(h) = std::env::var("HOME").or_else(|_| std::env::var("USERPROFILE")) {
|
||||
let movies = Path::new(&h).join("Movies");
|
||||
return if movies.is_dir() { movies } else { PathBuf::from(h) };
|
||||
}
|
||||
std::env::temp_dir()
|
||||
crate::temp_dir()
|
||||
}
|
||||
|
||||
fn file_safe(n: &str) -> String {
|
||||
@@ -381,6 +387,7 @@ pub fn spawn_export(s: &mut Session, project: Arc<Project>, seq: ItemId, mut set
|
||||
}
|
||||
*res.lock().unwrap_or_else(|x| x.into_inner()) = Some(r);
|
||||
};
|
||||
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
|
||||
s.jobs.push(job);
|
||||
if wait || cfg!(target_arch = "wasm32") {
|
||||
run();
|
||||
@@ -390,6 +397,24 @@ pub fn spawn_export(s: &mut Session, project: Arc<Project>, seq: ItemId, mut set
|
||||
Ok(id)
|
||||
}
|
||||
|
||||
/// Last-resort guard for a background job body (export, previews, proxies, …): a panic marks the
|
||||
/// job failed with an error instead of killing its thread and leaving the job "running" forever.
|
||||
/// The panic hook has already logged where it happened.
|
||||
pub(crate) fn guard_job(
|
||||
progress: Arc<filmcraft_export::Progress>,
|
||||
result: Arc<std::sync::Mutex<Option<std::result::Result<filmcraft_export::Report, String>>>>,
|
||||
run: impl FnOnce() + Send + 'static,
|
||||
) -> impl FnOnce() + Send + 'static {
|
||||
move || {
|
||||
if std::panic::catch_unwind(std::panic::AssertUnwindSafe(run)).is_err() {
|
||||
let msg = "internal error (see the crash log); the job was stopped".to_string();
|
||||
*progress.error.lock().unwrap_or_else(|x| x.into_inner()) = Some(msg.clone());
|
||||
progress.finished.store(true, Ordering::Relaxed);
|
||||
*result.lock().unwrap_or_else(|x| x.into_inner()) = Some(Err(msg));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn file_name(path: &str) -> String {
|
||||
Path::new(path).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_else(|| path.to_string())
|
||||
}
|
||||
@@ -672,9 +697,9 @@ fn queue_cancel(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
cancel_item(s, i);
|
||||
}
|
||||
}
|
||||
if bool_p(p, "wait").unwrap_or(false) {
|
||||
// wait for a cancelled encode to stop
|
||||
let t0 = std::time::Instant::now();
|
||||
// wait for a cancelled encode to stop (there is no encode thread, clock or sleep on wasm32)
|
||||
if bool_p(p, "wait").unwrap_or(false) && !cfg!(target_arch = "wasm32") {
|
||||
let t0 = web_time::Instant::now();
|
||||
while s.export_queue.items.iter().any(|i| i.status == QueueStatus::Encoding) && t0.elapsed() < std::time::Duration::from_secs(60) {
|
||||
let running = s.export_queue.running;
|
||||
s.export_queue.running = false;
|
||||
@@ -1004,3 +1029,20 @@ pub(crate) fn commands() -> Vec<CommandSpec> {
|
||||
spec("export.quick", "Quick Export", r#"{"preset":str?,"path":str?,"wait":bool?, …settings params}"#, has_seq, quick, true),
|
||||
]
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod guard_tests {
|
||||
use super::*;
|
||||
|
||||
/// A panicking job body must leave the job finished with an error, not "running" forever.
|
||||
#[test]
|
||||
fn panicking_job_is_reported_as_failed() {
|
||||
let progress: Arc<filmcraft_export::Progress> = Arc::default();
|
||||
let result: Arc<std::sync::Mutex<Option<std::result::Result<filmcraft_export::Report, String>>>> = Arc::default();
|
||||
let run = guard_job(progress.clone(), result.clone(), || panic!("job bug"));
|
||||
std::thread::spawn(run).join().expect("the guard catches the panic");
|
||||
assert!(progress.finished.load(Ordering::Relaxed));
|
||||
assert!(progress.error.lock().unwrap().is_some());
|
||||
assert!(matches!(&*result.lock().unwrap(), Some(Err(_))));
|
||||
}
|
||||
}
|
||||
@@ -157,15 +157,15 @@ pub fn thumbnail(t: &GraphicsTemplate, w: u32) -> (u32, u32, Vec<u8>) {
|
||||
let mut p = Project::new("thumb");
|
||||
let rate = filmcraft_time::FrameRate::FPS_30;
|
||||
let g = p.add_item("G", Label::Rose, ItemKind::Graphic { width: t.canvas[0], height: t.canvas[1], rate }, None);
|
||||
let mut it = p
|
||||
.make_track_item(
|
||||
g,
|
||||
filmcraft_project::TrackKind::Video,
|
||||
Tick::ZERO,
|
||||
filmcraft_time::TimeRange::new(Tick::ZERO, t.duration.max(rate.frame_duration())),
|
||||
rate,
|
||||
)
|
||||
.expect("item");
|
||||
let Some(mut it) = p.make_track_item(
|
||||
g,
|
||||
filmcraft_project::TrackKind::Video,
|
||||
Tick::ZERO,
|
||||
filmcraft_time::TimeRange::new(Tick::ZERO, t.duration.max(rate.frame_duration())),
|
||||
rate,
|
||||
) else {
|
||||
return (w, h, vec![0; w as usize * h as usize * 4]);
|
||||
};
|
||||
it.effects.extend(t.layers.iter().cloned());
|
||||
it.graphic = Some(Box::new(meta));
|
||||
// a moment into the clip, so intro animations have settled a little
|
||||
|
||||
@@ -551,11 +551,11 @@ macro_rules! arrange_cmd {
|
||||
fn new_shape_cmd(s: &mut Session, p: &Value, shape: &str) -> Result<Value> {
|
||||
let mut q = p.clone();
|
||||
q["shape"] = json!(shape);
|
||||
(find_spec("graphics.newShape").run)(s, &q)
|
||||
(find_spec("graphics.newShape")?.run)(s, &q)
|
||||
}
|
||||
|
||||
fn find_spec(id: &str) -> CommandSpec {
|
||||
commands().into_iter().find(|c| c.id == id).expect("graphics command")
|
||||
fn find_spec(id: &str) -> Result<CommandSpec> {
|
||||
commands().into_iter().find(|c| c.id == id).ok_or_else(|| EngineError::UnknownCommand(id.to_string()))
|
||||
}
|
||||
|
||||
/// Graphics menu commands (New Layer shapes, Align/Distribute/Arrange/Select submenus, resets).
|
||||
@@ -576,7 +576,7 @@ fn menu_commands() -> Vec<CommandSpec> {
|
||||
|s, p| {
|
||||
let mut p = if p.is_object() { p.clone() } else { json!({}) };
|
||||
p["vertical"] = json!(true);
|
||||
(find_spec("graphics.newText").run)(s, &p)
|
||||
(find_spec("graphics.newText")?.run)(s, &p)
|
||||
},
|
||||
),
|
||||
spec(
|
||||
@@ -731,7 +731,7 @@ fn menu_commands() -> Vec<CommandSpec> {
|
||||
let rate = q.settings.frame_rate;
|
||||
let want = rate.snap_nearest(Tick::from_seconds_f64(seconds)).max(rate.frame_duration());
|
||||
let tr = q.video_tracks.iter_mut().find(|tr| tr.items.iter().any(|i| i.id == clip)).ok_or(filmcraft_edit::EditError::NoItem(clip))?;
|
||||
let i = tr.items.iter().position(|i| i.id == clip).expect("found");
|
||||
let i = tr.items.iter().position(|i| i.id == clip).ok_or(filmcraft_edit::EditError::NoItem(clip))?;
|
||||
let start = tr.items[i].start;
|
||||
let limit = tr.items.get(i + 1).map_or(want, |n| n.start - start);
|
||||
let d = want.min(limit).max(rate.frame_duration());
|
||||
|
||||
@@ -8,6 +8,8 @@
|
||||
//! The project is an `Arc<Project>` edited copy-on-write; undo keeps whole-project snapshots (cheap
|
||||
//! thanks to structural sharing of untouched items).
|
||||
|
||||
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
|
||||
|
||||
pub mod aaf_omf;
|
||||
pub mod autosave;
|
||||
pub mod captions;
|
||||
@@ -63,6 +65,12 @@ pub use filmcraft_render as render;
|
||||
pub use filmcraft_time as time;
|
||||
pub use media_pool::MediaPool;
|
||||
|
||||
/// The OS temp folder; `/tmp` on wasm32, where `std::env::temp_dir()` panics ("no filesystem on
|
||||
/// this platform"). Use this instead of `std::env::temp_dir()` in any code the web build runs.
|
||||
pub fn temp_dir() -> std::path::PathBuf {
|
||||
if cfg!(target_arch = "wasm32") { std::path::PathBuf::from("/tmp") } else { std::env::temp_dir() }
|
||||
}
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum EngineError {
|
||||
#[error("unknown command `{0}`")]
|
||||
@@ -247,6 +255,10 @@ pub struct EditorState {
|
||||
/// audio clips too.
|
||||
#[serde(default)]
|
||||
pub multicam_audio_follows_video: bool,
|
||||
/// Multi-Camera view settings: grid layout and page, selection order, preview monitor,
|
||||
/// playback quality, transmit.
|
||||
#[serde(default)]
|
||||
pub multicam_view: multicam::MulticamView,
|
||||
/// Audio Clip Mixer automation mode per audio track (track id → mode; missing = Read):
|
||||
/// Latch / Touch / Write record the clip Volume and Panner moves as clip keyframes.
|
||||
#[serde(default)]
|
||||
@@ -465,6 +477,7 @@ impl Session {
|
||||
pub fn start_autosave(&mut self, cfg: autosave::AutosaveConfig) -> std::io::Result<()> {
|
||||
let prefs_path = cfg.data_dir.join("preferences.json");
|
||||
self.prefs = autosave::Preferences::load(&prefs_path);
|
||||
self.prefs.apply_hardware_decoding();
|
||||
self.media.set_use_proxies(self.prefs.media.enable_proxies);
|
||||
self.shortcuts.set_dir(&cfg.data_dir);
|
||||
self.presets.set_dir(&cfg.data_dir);
|
||||
@@ -536,6 +549,7 @@ impl Session {
|
||||
pub fn set_prefs(&mut self, p: autosave::Preferences) -> std::io::Result<()> {
|
||||
let cache_changed = self.prefs.media_cache != p.media_cache;
|
||||
self.prefs = p;
|
||||
self.prefs.apply_hardware_decoding();
|
||||
if cache_changed {
|
||||
self.apply_media_cache();
|
||||
}
|
||||
@@ -589,7 +603,11 @@ impl Session {
|
||||
return Err(e);
|
||||
}
|
||||
self.exec_depth += 1;
|
||||
let r = (spec.run)(self, ¶ms);
|
||||
// Last-resort guard: a panic in a command (a bug that escaped the never-crash rules)
|
||||
// becomes an error for the caller (UI, CLI, control channel, MCP) instead of taking the
|
||||
// session and its unsaved project down. The panic hook has logged where it happened.
|
||||
let r = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| (spec.run)(self, ¶ms)))
|
||||
.unwrap_or_else(|_| Err(EngineError::Other(format!("internal error in `{id}` (see the crash log); the command did not complete"))));
|
||||
self.exec_depth -= 1;
|
||||
// source graphics: an edited instance updates the shared layers and the other instances
|
||||
if r.is_ok() && spec.journal && id.starts_with("graphics.") && !self.project.source_graphics.is_empty() {
|
||||
@@ -671,13 +689,15 @@ impl Session {
|
||||
let seq_id = self.state.active_sequence.ok_or(EngineError::NoSequence)?;
|
||||
let media = self.media.clone();
|
||||
self.edit(label, move |p, st| {
|
||||
let project_snapshot = p.clone();
|
||||
let project_snapshot = std::sync::Arc::new(p.clone());
|
||||
let snap = project_snapshot.clone();
|
||||
let durations = move |id: ItemId| -> Option<Tick> { media_duration(&project_snapshot, &media, id) };
|
||||
let starts = move |id: ItemId| media_start(&snap, id);
|
||||
let min = p.sequence(seq_id).map(|s| s.settings.frame_rate.frame_duration()).unwrap_or(Tick(1));
|
||||
let mut next = p.next_id;
|
||||
let r = {
|
||||
let seq = p.sequence_mut(seq_id).ok_or(EngineError::NoSequence)?;
|
||||
let mut ctx = EditCtx { next_id: &mut next, media_duration: &durations, min_duration: min };
|
||||
let mut ctx = EditCtx { next_id: &mut next, media_duration: &durations, media_start: &starts, min_duration: min };
|
||||
f(seq, &mut ctx, st)?
|
||||
};
|
||||
p.next_id = next;
|
||||
@@ -822,6 +842,15 @@ impl Session {
|
||||
}
|
||||
}
|
||||
|
||||
/// The earliest media time clips of an item may show: a subclip that restricts trims starts at its
|
||||
/// In point, everything else at 0.
|
||||
pub fn media_start(p: &Project, id: ItemId) -> Tick {
|
||||
match p.item(id).map(|i| &i.kind) {
|
||||
Some(filmcraft_project::ItemKind::Subclip { range, restrict_trims: true, .. }) => range.start,
|
||||
_ => Tick::ZERO,
|
||||
}
|
||||
}
|
||||
|
||||
/// Media duration of an item (None for stills/adjustment layers = unlimited handles).
|
||||
pub fn media_duration(p: &Project, _pool: &MediaPool, id: ItemId) -> Option<Tick> {
|
||||
let it = p.item(id)?;
|
||||
|
||||
@@ -634,6 +634,7 @@ fn track(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
prog.finished.store(true, Ordering::Relaxed);
|
||||
*lock(&res) = Some(r);
|
||||
};
|
||||
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
|
||||
s.jobs.push(job);
|
||||
s.mask_jobs.push(PendingTrack { job: id, target: MaskSel { clip, effect: ei, mask: mi }, seq: seq_id, keys, applied: 0 });
|
||||
let wait = crate::commands::bool_p(p, "wait").unwrap_or(false);
|
||||
|
||||
@@ -101,7 +101,7 @@ pub fn psnr(a: &[u8], b: &[u8]) -> f64 {
|
||||
assert_eq!(a.len(), b.len());
|
||||
let mut se = 0.0;
|
||||
let mut n = 0.0;
|
||||
for (x, y) in a.chunks_exact(4).zip(b.chunks_exact(4)) {
|
||||
for (x, y) in a.as_chunks::<4>().0.iter().zip(b.as_chunks::<4>().0) {
|
||||
for c in 0..3 {
|
||||
let d = x[c] as f64 - y[c] as f64;
|
||||
se += d * d;
|
||||
|
||||
+210
-14
@@ -14,6 +14,13 @@
|
||||
//! | `multicam.audioFollowsVideo` | | switching video switches the linked audio clips too |
|
||||
//! | `multicam.editCameras` | | rename / show / hide cameras, change the audio mode |
|
||||
//! | `multicam.inspect` | | the multi-camera clip at the playhead, its cameras and the recorder state |
|
||||
//! | `multicam.gridLayout` | Program ▸ Multi-Camera ▸ Layout | angle grid: automatic, 2×2, 3×3 or 4×4 (paged when the angles don't fit) |
|
||||
//! | `multicam.page` / `multicam.nextPage` / `multicam.prevPage` | page arrows | page of the angle grid (keys 1–9 pick cameras on the shown page) |
|
||||
//! | `multicam.selectionTopDown` | Multi-Camera Selection Top Down | stacked multi-camera clips: switch the topmost instead of the lowest |
|
||||
//! | `multicam.showPreviewMonitor` | Show Multi-Camera Preview Monitor | the program next to the grid (off: the grid fills the monitor) |
|
||||
//! | `multicam.autoAdjustQuality` | Auto-Adjust Multi-Camera Playback Quality | lower-resolution grid decode while playing |
|
||||
//! | `multicam.transmitView` | Transmit Multi-Camera View | send the grid to the transmit device (no transmit output yet: stored only) |
|
||||
//! | `multicam.grid` | | the grid page at the playhead: layout, pages, cells (camera, angle, name, active) |
|
||||
//!
|
||||
//! Sync methods are described in [`crate::sync`].
|
||||
|
||||
@@ -28,6 +35,7 @@ use filmcraft_project::{
|
||||
TrackId, TrackItem, TrackKind, resolve_auto_points,
|
||||
};
|
||||
use filmcraft_time::{FrameRate, Tick, TimeRange};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::{Value, json};
|
||||
|
||||
use crate::commands::{CommandSpec, always, bad, bool_p, clips_p, has_selection, has_seq, item_p, str_p, time_p, track_p, u64_p, with_links};
|
||||
@@ -52,7 +60,7 @@ macro_rules! camera_keys {
|
||||
shortcut: Some(concat!($n)),
|
||||
params: r#"{"videoOnly":bool?}"#,
|
||||
enabled: has_seq,
|
||||
run: |s, p| cut(s, &with_camera(p, $n)),
|
||||
run: |s, p| cut(s, &on_page(with_camera(p, $n))),
|
||||
journal: true,
|
||||
},
|
||||
CommandSpec {
|
||||
@@ -62,7 +70,7 @@ macro_rules! camera_keys {
|
||||
shortcut: Some(concat!("Ctrl+", $n)),
|
||||
params: r#"{"videoOnly":bool?}"#,
|
||||
enabled: has_seq,
|
||||
run: |s, p| cut_to_camera(s, &with_camera(p, $n)),
|
||||
run: |s, p| cut_to_camera(s, &on_page(with_camera(p, $n))),
|
||||
journal: true,
|
||||
},
|
||||
)*]
|
||||
@@ -75,6 +83,51 @@ fn with_camera(p: &Value, n: u64) -> Value {
|
||||
v
|
||||
}
|
||||
|
||||
/// Keys 1–9 pick the camera on the grid page shown.
|
||||
fn on_page(mut v: Value) -> Value {
|
||||
v["pageRelative"] = json!(true);
|
||||
v
|
||||
}
|
||||
|
||||
/// Multi-Camera view settings (Program monitor ▸ wrench menu); editor state, not project data.
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
#[serde(default, rename_all = "camelCase")]
|
||||
pub struct MulticamView {
|
||||
/// Grid side (2 = 2×2, 3 = 3×3, 4 = 4×4); None = automatic.
|
||||
pub layout: Option<usize>,
|
||||
/// Grid page (0-based).
|
||||
pub page: usize,
|
||||
/// Multi-Camera Selection Top Down: of stacked multi-camera clips, the topmost one is switched.
|
||||
pub top_down: bool,
|
||||
/// Show Multi-Camera Preview Monitor: the program next to the grid.
|
||||
pub show_preview: bool,
|
||||
/// Auto-Adjust Multi-Camera Playback Quality: lower-resolution grid decode while playing.
|
||||
pub auto_quality: bool,
|
||||
/// Transmit Multi-Camera View: send the grid (not the program) to the transmit device.
|
||||
pub transmit: bool,
|
||||
}
|
||||
|
||||
impl Default for MulticamView {
|
||||
fn default() -> Self {
|
||||
MulticamView { layout: None, page: 0, top_down: false, show_preview: true, auto_quality: false, transmit: false }
|
||||
}
|
||||
}
|
||||
|
||||
impl MulticamView {
|
||||
pub fn layout_name(&self) -> &'static str {
|
||||
match self.layout {
|
||||
Some(2) => "2x2",
|
||||
Some(3) => "3x3",
|
||||
Some(4) => "4x4",
|
||||
_ => "auto",
|
||||
}
|
||||
}
|
||||
/// The grid page for `shown` angles.
|
||||
pub fn page_layout(&self, shown: usize) -> filmcraft_render::multicam::PageLayout {
|
||||
filmcraft_render::multicam::page_layout(shown, self.layout, self.page)
|
||||
}
|
||||
}
|
||||
|
||||
pub fn commands() -> Vec<CommandSpec> {
|
||||
let mut v = base_commands();
|
||||
v.extend(camera_keys!(1, 2, 3, 4, 5, 6, 7, 8, 9));
|
||||
@@ -135,6 +188,40 @@ fn base_commands() -> Vec<CommandSpec> {
|
||||
true,
|
||||
),
|
||||
spec("multicam.cutToCamera", "Cut to Camera", &[], r#"{"camera":1..16|"angle":0-based,"time":ticks?,"videoOnly":bool?}"#, has_seq, cut_to_camera, true),
|
||||
spec("multicam.gridLayout", "Multi-Camera Layout", &[], r#"{"layout":"auto|2x2|3x3|4x4"}"#, always, grid_layout, true),
|
||||
spec("multicam.page", "Multi-Camera Page", &[], r#"{"page":0-based|"next"|"prev"}"#, always, set_page, true),
|
||||
spec("multicam.nextPage", "Next Multi-Camera Page", &[], "{}", always, |s, _| set_page(s, &json!({"page": "next"})), true),
|
||||
spec("multicam.prevPage", "Previous Multi-Camera Page", &[], "{}", always, |s, _| set_page(s, &json!({"page": "prev"})), true),
|
||||
spec("multicam.selectionTopDown", "Multi-Camera Selection Top Down", &[], r#"{"enabled":bool?}"#, always, |s, p| view_flag(s, p, Flag::TopDown), true),
|
||||
spec(
|
||||
"multicam.showPreviewMonitor",
|
||||
"Show Multi-Camera Preview Monitor",
|
||||
&[],
|
||||
r#"{"enabled":bool?}"#,
|
||||
always,
|
||||
|s, p| view_flag(s, p, Flag::Preview),
|
||||
true,
|
||||
),
|
||||
spec(
|
||||
"multicam.autoAdjustQuality",
|
||||
"Auto-Adjust Multi-Camera Playback Quality",
|
||||
&[],
|
||||
r#"{"enabled":bool?}"#,
|
||||
always,
|
||||
|s, p| view_flag(s, p, Flag::AutoQuality),
|
||||
true,
|
||||
),
|
||||
spec("multicam.transmitView", "Transmit Multi-Camera View", &[], r#"{"enabled":bool?}"#, always, |s, p| view_flag(s, p, Flag::Transmit), true),
|
||||
CommandSpec {
|
||||
id: "multicam.grid",
|
||||
label: "Multi-Camera Grid",
|
||||
menu: &[],
|
||||
shortcut: None,
|
||||
params: r#"{"time":ticks?,"playing":bool?,"cellPixels":f32?,"playbackScale":f32?}"#,
|
||||
enabled: always,
|
||||
run: grid_info,
|
||||
journal: false,
|
||||
},
|
||||
CommandSpec {
|
||||
id: "multicam.inspect",
|
||||
label: "Inspect Multi-Camera",
|
||||
@@ -189,14 +276,16 @@ fn offset_p(p: &Value, rate: FrameRate) -> Tick {
|
||||
|
||||
/// Run `f` on sequence `seq_id` of `p` with an edit context (ids, media durations), then check it.
|
||||
fn seq_edit<R>(p: &mut Project, seq_id: ItemId, media: &Arc<MediaPool>, f: impl FnOnce(&mut Sequence, &mut EditCtx) -> Result<R>) -> Result<R> {
|
||||
let snapshot = p.clone();
|
||||
let snapshot = Arc::new(p.clone());
|
||||
let snap = snapshot.clone();
|
||||
let media = media.clone();
|
||||
let durations = move |id: ItemId| crate::media_duration(&snapshot, &media, id);
|
||||
let starts = move |id: ItemId| crate::media_start(&snap, id);
|
||||
let min = p.sequence(seq_id).map(|s| s.settings.frame_rate.frame_duration()).unwrap_or(Tick(1));
|
||||
let mut next = p.next_id;
|
||||
let r = {
|
||||
let seq = p.sequence_mut(seq_id).ok_or(EngineError::NoSequence)?;
|
||||
let mut ctx = EditCtx { next_id: &mut next, media_duration: &durations, min_duration: min };
|
||||
let mut ctx = EditCtx { next_id: &mut next, media_duration: &durations, media_start: &starts, min_duration: min };
|
||||
f(seq, &mut ctx)?
|
||||
};
|
||||
p.next_id = next;
|
||||
@@ -465,7 +554,7 @@ fn merge_clips(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
a.link = Some(link);
|
||||
a_items.push(a);
|
||||
}
|
||||
let q = pr.sequence_mut(sid).expect("new sequence");
|
||||
let q = pr.sequence_mut(sid).ok_or(EngineError::NoSequence)?;
|
||||
for (t, it) in q.video_tracks.iter_mut().zip(v_items) {
|
||||
t.items.push(it);
|
||||
}
|
||||
@@ -527,7 +616,7 @@ fn create_multicam(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let (mut vi, mut ai) = (0usize, 0usize);
|
||||
let mut placed: Vec<(TrackKind, usize, TrackItem)> = Vec::new();
|
||||
let (vtracks, atracks): (Vec<TrackId>, Vec<TrackId>) = {
|
||||
let q = pr.sequence(sid).expect("new sequence");
|
||||
let q = pr.sequence(sid).ok_or(EngineError::NoSequence)?;
|
||||
(q.video_tracks.iter().map(|t| t.id).collect(), q.audio_tracks.iter().map(|t| t.id).collect())
|
||||
};
|
||||
for (k, c) in clips.iter().enumerate() {
|
||||
@@ -551,7 +640,7 @@ fn create_multicam(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
cameras.push(cam);
|
||||
}
|
||||
{
|
||||
let q = pr.sequence_mut(sid).expect("new sequence");
|
||||
let q = pr.sequence_mut(sid).ok_or(EngineError::NoSequence)?;
|
||||
for (kind, idx, it) in placed {
|
||||
q.tracks_mut(kind)[idx].items.push(it);
|
||||
}
|
||||
@@ -590,12 +679,16 @@ fn create_multicam(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
|
||||
/// The topmost enabled multi-camera video clip at `t` (targeted tracks first) in the active
|
||||
/// sequence: (track, clip).
|
||||
///
|
||||
/// Targeted tracks are searched first, then every video track; from the lowest track up, or from
|
||||
/// the topmost down with Multi-Camera Selection Top Down.
|
||||
pub fn multicam_clip_at(s: &Session, t: Tick) -> Option<(TrackId, TrackItem)> {
|
||||
let q = s.active_sequence()?;
|
||||
let tg = s.targeting().targeted;
|
||||
let at = |tr: &Track| tr.item_at(t).filter(|i| i.multicam.is_some_and(|m| m.enabled) && s.project.sequence(i.item).is_some()).cloned();
|
||||
let targeted = q.video_tracks.iter().rev().filter(|tr| tg.contains(&tr.id)).find_map(|tr| at(tr).map(|i| (tr.id, i)));
|
||||
targeted.or_else(|| q.video_tracks.iter().rev().find_map(|tr| at(tr).map(|i| (tr.id, i))))
|
||||
let order: Vec<&Track> = if s.state.multicam_view.top_down { q.video_tracks.iter().rev().collect() } else { q.video_tracks.iter().collect() };
|
||||
let targeted = order.iter().filter(|tr| tg.contains(&tr.id)).find_map(|tr| at(tr).map(|i| (tr.id, i)));
|
||||
targeted.or_else(|| order.iter().find_map(|tr| at(tr).map(|i| (tr.id, i))))
|
||||
}
|
||||
|
||||
/// Resolve `camera` (1-based, shown order) or `angle` (0-based) for the source of `clip`.
|
||||
@@ -608,8 +701,11 @@ fn angle_p(s: &Session, p: &Value, source: ItemId, cmd: &str) -> Result<u32> {
|
||||
}
|
||||
return Err(bad(cmd, format!("angle {a} out of range (0–{})", cams.cameras.len().saturating_sub(1))));
|
||||
}
|
||||
let n = u64_p(p, "camera").ok_or_else(|| bad(cmd, "need `camera` (1-based) or `angle`"))? as usize;
|
||||
let mut n = u64_p(p, "camera").ok_or_else(|| bad(cmd, "need `camera` (1-based) or `angle`"))? as usize;
|
||||
let shown = cams.shown_angles();
|
||||
if bool_p(p, "pageRelative").unwrap_or(false) {
|
||||
n += s.state.multicam_view.page_layout(shown.len()).first;
|
||||
}
|
||||
shown.get(n.wrapping_sub(1)).map(|a| *a as u32).ok_or_else(|| bad(cmd, format!("there is no camera {n} ({} shown)", shown.len())))
|
||||
}
|
||||
|
||||
@@ -796,10 +892,14 @@ fn record_start(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
fn apply_pass(s: &mut Session, end: Tick) -> Result<usize> {
|
||||
let Some(seq_id) = s.mcrec.seq else { return Ok(0) };
|
||||
let merging = s.history.merge_key.as_deref() == Some(s.mcrec.key.as_str()) && s.history.undo.last().is_some_and(|u| u.0 == RECORD_LABEL);
|
||||
if !merging || s.mcrec.base.is_none() {
|
||||
s.mcrec.base = Some(s.project.clone());
|
||||
}
|
||||
let base = s.mcrec.base.clone().expect("set above");
|
||||
let base = match s.mcrec.base.clone() {
|
||||
Some(b) if merging => b,
|
||||
_ => {
|
||||
let b = s.project.clone();
|
||||
s.mcrec.base = Some(b.clone());
|
||||
b
|
||||
}
|
||||
};
|
||||
let (tracks, cuts, key) = (s.mcrec.tracks.clone(), s.mcrec.cuts.clone(), s.mcrec.key.clone());
|
||||
let Some(source) = s.mcrec.source else { return Ok(0) };
|
||||
let media = s.media.clone();
|
||||
@@ -959,9 +1059,105 @@ pub fn inspect_at(s: &Session, t: Tick) -> Value {
|
||||
"cameras": cams.cameras.iter().enumerate().map(|(i, c)| json!({"angle": i, "name": c.name, "enabled": c.enabled, "video": c.video_track.is_some()})).collect::<Vec<_>>(),
|
||||
"recorder": rec,
|
||||
"audioFollowsVideo": s.state.multicam_audio_follows_video,
|
||||
"view": s.state.multicam_view,
|
||||
})
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- Multi-Camera view settings
|
||||
|
||||
fn grid_layout(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let cmd = "multicam.gridLayout";
|
||||
let l = str_p(p, "layout").ok_or_else(|| bad(cmd, "need `layout` (auto, 2x2, 3x3, 4x4)"))?;
|
||||
s.state.multicam_view.layout = match l.to_ascii_lowercase().replace(['×', ' '], "x").as_str() {
|
||||
"auto" | "automatic" => None,
|
||||
"2x2" | "2" => Some(2),
|
||||
"3x3" | "3" => Some(3),
|
||||
"4x4" | "4" => Some(4),
|
||||
_ => return Err(bad(cmd, format!("unknown layout `{l}` (auto, 2x2, 3x3, 4x4)"))),
|
||||
};
|
||||
s.state.multicam_view.page = 0;
|
||||
Ok(json!({"layout": s.state.multicam_view.layout_name()}))
|
||||
}
|
||||
|
||||
/// Shown angles of the multi-camera clip at the playhead (0 without one).
|
||||
fn shown_count(s: &Session) -> usize {
|
||||
multicam_clip_at(s, s.playhead()).and_then(|(_, c)| s.project.sequence(c.item).map(|q| q.cameras().shown_angles().len())).unwrap_or(0)
|
||||
}
|
||||
|
||||
fn set_page(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let n = shown_count(s);
|
||||
let l = s.state.multicam_view.page_layout(n);
|
||||
let page = match p.get("page") {
|
||||
Some(Value::String(d)) if d == "next" => (l.page + 1).min(l.pages - 1),
|
||||
Some(Value::String(d)) if d == "prev" || d == "previous" => l.page.saturating_sub(1),
|
||||
Some(v) => v.as_u64().ok_or_else(|| bad("multicam.page", "`page` is a 0-based number, \"next\" or \"prev\""))? as usize,
|
||||
None => return Err(bad("multicam.page", "need `page`")),
|
||||
};
|
||||
let l = filmcraft_render::multicam::page_layout(n, s.state.multicam_view.layout, page);
|
||||
s.state.multicam_view.page = l.page;
|
||||
Ok(json!({"page": l.page, "pages": l.pages}))
|
||||
}
|
||||
|
||||
enum Flag {
|
||||
TopDown,
|
||||
Preview,
|
||||
AutoQuality,
|
||||
Transmit,
|
||||
}
|
||||
|
||||
fn view_flag(s: &mut Session, p: &Value, f: Flag) -> Result<Value> {
|
||||
let v = &mut s.state.multicam_view;
|
||||
let slot = match f {
|
||||
Flag::TopDown => &mut v.top_down,
|
||||
Flag::Preview => &mut v.show_preview,
|
||||
Flag::AutoQuality => &mut v.auto_quality,
|
||||
Flag::Transmit => &mut v.transmit,
|
||||
};
|
||||
*slot = bool_p(p, "enabled").unwrap_or(!*slot);
|
||||
let on = *slot;
|
||||
let mut out = json!({"enabled": on});
|
||||
if matches!(f, Flag::Transmit) {
|
||||
// no transmit (video output) devices yet: the setting is kept for when there are
|
||||
out["device"] = Value::Null;
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// The grid page at the playhead (or `time`): layout, pages and cells. With `cellPixels` (on-screen
|
||||
/// cell width in pixels), `playing` and `playbackScale` it also returns the decode scale the
|
||||
/// view uses (Auto-Adjust Multi-Camera Playback Quality).
|
||||
fn grid_info(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let t = time_p(s, p, "").unwrap_or(s.playhead());
|
||||
let v = s.state.multicam_view.clone();
|
||||
let Some((_, clip)) = multicam_clip_at(s, t) else {
|
||||
return Ok(json!({"clip": null, "view": v}));
|
||||
};
|
||||
let q = s.project.sequence(clip.item).ok_or(EngineError::NoSequence)?;
|
||||
let cams = q.cameras();
|
||||
let shown = cams.shown_angles();
|
||||
let l = v.page_layout(shown.len());
|
||||
let active = clip.multicam.map(|m| m.angle as usize);
|
||||
let cells: Vec<Value> = shown
|
||||
.iter()
|
||||
.enumerate()
|
||||
.skip(l.first)
|
||||
.take(l.count)
|
||||
.map(|(k, &a)| {
|
||||
json!({"cell": k - l.first, "camera": k + 1, "angle": a, "name": cams.cameras.get(a).map(|c| c.name.clone()), "active": active == Some(a)})
|
||||
})
|
||||
.collect();
|
||||
let mut out = json!({
|
||||
"clip": clip.id.0, "source": clip.item.0, "layout": v.layout_name(), "cols": l.cols, "rows": l.rows,
|
||||
"page": l.page, "pages": l.pages, "angles": shown.len(), "cells": cells, "view": v,
|
||||
});
|
||||
if let Some(px) = p.get("cellPixels").and_then(Value::as_f64) {
|
||||
let playing = bool_p(p, "playing").unwrap_or(false);
|
||||
let pb = p.get("playbackScale").and_then(Value::as_f64).unwrap_or(1.0) as f32;
|
||||
out["cellScale"] = json!(filmcraft_render::multicam::grid_cell_scale(q.settings.width, px as f32, pb, playing, v.auto_quality, l.per_page));
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn inspect(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let t = time_p(s, p, "").unwrap_or(s.playhead());
|
||||
Ok(inspect_at(s, t))
|
||||
|
||||
@@ -612,6 +612,168 @@ fn camera_keys_cut_and_select() {
|
||||
assert!(keys.contains(&("multicam.cutToCamera9".into(), "Ctrl+9".into())), "{keys:?}");
|
||||
}
|
||||
|
||||
/// Colour of camera `k` (0-based) in the many-angle tests: distinct reds and greens.
|
||||
fn cam_rgb(k: usize) -> [u8; 3] {
|
||||
[(k * 12) as u8, (250 - k * 12) as u8, 128]
|
||||
}
|
||||
|
||||
/// `n` colour-matte cameras (64×36, 4 s) in a multi-camera source sequence, edited into a sequence
|
||||
/// on V1/A1. Returns (session, source sequence).
|
||||
fn many_angles(n: usize) -> (Session, ItemId) {
|
||||
let mut s = Session::default();
|
||||
let mut items = Vec::new();
|
||||
for k in 0..n {
|
||||
let [r, g, b] = cam_rgb(k);
|
||||
let hex = format!("#{r:02x}{g:02x}{b:02x}");
|
||||
let r = s.execute("file.newColorMatte", json!({"color": hex, "seconds": 4.0, "width": 64, "height": 36, "name": format!("Cam {}", k + 1)})).unwrap();
|
||||
items.push(ItemId(r["item"].as_u64().unwrap()));
|
||||
}
|
||||
let r = on_items(&mut s, "clip.createMulticam", &items, json!({"method": "in", "cameraNames": "clip"})).unwrap();
|
||||
let src = ItemId(r["sequence"].as_u64().unwrap());
|
||||
let mut p = (*s.project).clone();
|
||||
let edit =
|
||||
p.new_sequence("Cut", filmcraft_project::SequenceSettings { width: 64, height: 36, frame_rate: FrameRate::FPS_24, ..Default::default() }, 2, 2, None);
|
||||
s.project = std::sync::Arc::new(p);
|
||||
s.state.active_sequence = Some(edit);
|
||||
s.execute("source.open", json!({"item": src.0})).unwrap();
|
||||
s.execute("source.overwrite", json!({})).unwrap();
|
||||
s.set_playhead(FrameRate::FPS_24.tick_of(10));
|
||||
(s, src)
|
||||
}
|
||||
|
||||
/// A media source that records the scale of every frame request.
|
||||
struct Recording {
|
||||
inner: filmcraft_media::SharedSource,
|
||||
scales: std::sync::Arc<std::sync::Mutex<Vec<f32>>>,
|
||||
}
|
||||
|
||||
impl filmcraft_media::MediaSource for Recording {
|
||||
fn info(&self) -> &filmcraft_media::MediaInfo {
|
||||
self.inner.info()
|
||||
}
|
||||
fn video_frame(&self, req: filmcraft_media::FrameRequest) -> filmcraft_media::Result<std::sync::Arc<filmcraft_frame::VideoFrame>> {
|
||||
self.scales.lock().unwrap().push(req.scale);
|
||||
self.inner.video_frame(req)
|
||||
}
|
||||
fn audio(&self, start: i64, frames: usize, sample_rate: u32) -> filmcraft_media::Result<filmcraft_frame::AudioBuffer> {
|
||||
self.inner.audio(start, frames, sample_rate)
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn more_than_sixteen_angles_page_and_switch() {
|
||||
let (mut s, src) = many_angles(20);
|
||||
assert_eq!(s.project.sequence(src).unwrap().multicam.as_ref().unwrap().cameras.len(), 20);
|
||||
// automatic layout: 4×4 pages, two of them
|
||||
let g = s.execute("multicam.grid", json!({})).unwrap();
|
||||
assert_eq!((g["cols"].as_u64(), g["rows"].as_u64(), g["pages"].as_u64(), g["page"].as_u64()), (Some(4), Some(4), Some(2), Some(0)));
|
||||
assert_eq!(g["cells"].as_array().unwrap().len(), 16);
|
||||
assert_eq!(g["cells"][0]["active"], true, "angle 0 is on");
|
||||
// page 2 holds cameras 17–20
|
||||
s.execute("multicam.nextPage", json!({})).unwrap();
|
||||
let g = s.execute("multicam.grid", json!({})).unwrap();
|
||||
let cams: Vec<u64> = g["cells"].as_array().unwrap().iter().map(|c| c["camera"].as_u64().unwrap()).collect();
|
||||
assert_eq!((g["page"].as_u64(), cams), (Some(1), vec![17, 18, 19, 20]));
|
||||
assert_eq!(g["cells"][3]["name"], "Cam 20");
|
||||
// past the last page clamps
|
||||
assert_eq!(s.execute("multicam.nextPage", json!({})).unwrap()["page"], 1);
|
||||
// keys 1–9 pick cameras on the shown page: 2 → camera 18 (angle 17)
|
||||
s.execute("multicam.selectCamera2", json!({})).unwrap();
|
||||
assert_eq!(v1(&s), [(0, 17)]);
|
||||
assert_eq!(a1(&s), [(0, 0)], "audio follows video is off");
|
||||
// absolute cameras beyond 16 switch from any page; angle is 0-based
|
||||
s.execute("multicam.page", json!({"page": 0})).unwrap();
|
||||
s.execute("multicam.cut", json!({"camera": 20})).unwrap();
|
||||
assert_eq!(v1(&s), [(0, 19)]);
|
||||
// Ctrl+3 on page 1 at frame 30: an edit, camera 3 after it
|
||||
s.set_playhead(FrameRate::FPS_24.tick_of(30));
|
||||
s.execute("multicam.cutToCamera3", json!({})).unwrap();
|
||||
assert_eq!(v1(&s), [(0, 19), (30, 2)]);
|
||||
// the program shows the switched angle's colour
|
||||
let px = frame(&mut s, 40);
|
||||
let c = &px[(18 * 64 + 32) * 4..][..3];
|
||||
assert!(c.iter().zip(cam_rgb(2)).all(|(a, b)| (*a as i32 - b as i32).abs() <= 3), "{c:?}");
|
||||
s.execute("edit.undo", json!({})).unwrap();
|
||||
assert_eq!(v1(&s), [(0, 19)]);
|
||||
// fixed layouts re-page: 3×3 → three pages; 2×2 → five; switching layout resets the page
|
||||
s.execute("multicam.gridLayout", json!({"layout": "3x3"})).unwrap();
|
||||
assert_eq!(s.execute("multicam.page", json!({"page": 7})).unwrap(), json!({"page": 2, "pages": 3}));
|
||||
let g = s.execute("multicam.grid", json!({})).unwrap();
|
||||
assert_eq!(g["cells"].as_array().unwrap().len(), 2, "cameras 19 and 20");
|
||||
assert_eq!(g["cells"][1]["active"], true, "camera 20 is on");
|
||||
s.execute("multicam.gridLayout", json!({"layout": "2x2"})).unwrap();
|
||||
let g = s.execute("multicam.grid", json!({})).unwrap();
|
||||
assert_eq!((g["page"].as_u64(), g["pages"].as_u64(), g["layout"].as_str()), (Some(0), Some(5), Some("2x2")));
|
||||
assert!(s.execute("multicam.gridLayout", json!({"layout": "5x5"})).is_err());
|
||||
// the view settings are editor state: they survive a state round trip
|
||||
let st: crate::EditorState = serde_json::from_value(serde_json::to_value(&s.state).unwrap()).unwrap();
|
||||
assert_eq!(st.multicam_view.layout, Some(2));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn grid_pages_decode_only_their_angles_at_reduced_resolution() {
|
||||
let (mut s, src) = many_angles(20);
|
||||
let scales = std::sync::Arc::new(std::sync::Mutex::new(Vec::new()));
|
||||
let provider = s.media.provider(s.project.clone(), s.services.clone());
|
||||
let recording = {
|
||||
let scales = scales.clone();
|
||||
move |item: ItemId| {
|
||||
use filmcraft_render::SourceProvider;
|
||||
provider.source(item).map(|inner| std::sync::Arc::new(Recording { inner, scales: scales.clone() }) as filmcraft_media::SharedSource)
|
||||
}
|
||||
};
|
||||
let t = FrameRate::FPS_24.tick_of(10);
|
||||
let (img, angles) = filmcraft_render::multicam::render_grid_page(&s.project, src, t, 0.25, None, 1, &recording).unwrap();
|
||||
assert_eq!(angles, [16, 17, 18, 19]);
|
||||
// 4×4 cells of 16×9 (¼ of 64×36): only the page's four angles decoded, each at ¼ scale
|
||||
assert_eq!((img.w, img.h), (64, 36));
|
||||
let asked = scales.lock().unwrap().clone();
|
||||
assert_eq!(asked.len(), 4, "{asked:?}");
|
||||
assert!(asked.iter().all(|s| (*s - 0.25).abs() < 1e-6), "{asked:?}");
|
||||
// cell 0 is camera 17, cell 3 camera 20; empty cells are black
|
||||
let px = img.over_black_rgba8();
|
||||
let at = |x: usize, y: usize| px[(y * 64 + x) * 4..][..3].to_vec();
|
||||
for (cell, cam) in [(0usize, 16usize), (3, 19)] {
|
||||
let c = at(cell % 4 * 16 + 8, cell / 4 * 9 + 4);
|
||||
assert!(c.iter().zip(cam_rgb(cam)).all(|(a, b)| (*a as i32 - b as i32).abs() <= 3), "cell {cell}: {c:?}");
|
||||
}
|
||||
assert_eq!(at(8, 20), [0, 0, 0]);
|
||||
// Auto-Adjust Multi-Camera Playback Quality: the view asks for a lower cell scale while playing
|
||||
let q =
|
||||
|s: &mut Session, playing: bool| s.execute("multicam.grid", json!({"cellPixels": 16.0, "playing": playing})).unwrap()["cellScale"].as_f64().unwrap();
|
||||
assert_eq!((q(&mut s, false), q(&mut s, true)), (0.25, 0.25));
|
||||
s.execute("multicam.autoAdjustQuality", json!({})).unwrap();
|
||||
assert_eq!((q(&mut s, false), q(&mut s, true)), (0.25, 0.0625));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn selection_top_down_and_view_toggles() {
|
||||
let (mut s, src) = many_angles(3);
|
||||
// a second multi-camera clip of the same source stacked on V2
|
||||
let edit = s.state.active_sequence.unwrap();
|
||||
s.execute("timeline.place", json!({"item": src.0, "track": "V2", "frame": 0})).unwrap();
|
||||
let ids = |s: &Session| {
|
||||
let q = s.project.sequence(edit).unwrap();
|
||||
(q.video_tracks[0].items[0].id.0, q.video_tracks[1].items[0].id.0)
|
||||
};
|
||||
let (low, high) = ids(&s);
|
||||
// untargeted tracks: lowest first by default, topmost with Selection Top Down
|
||||
let clip = |s: &mut Session| s.execute("multicam.inspect", json!({})).unwrap()["clip"].as_u64().unwrap();
|
||||
let expect_default = clip(&mut s);
|
||||
assert!(expect_default == low || expect_default == high);
|
||||
s.execute("multicam.selectionTopDown", json!({"enabled": true})).unwrap();
|
||||
let top = clip(&mut s);
|
||||
s.execute("multicam.selectionTopDown", json!({"enabled": false})).unwrap();
|
||||
let bottom = clip(&mut s);
|
||||
assert_eq!((top, bottom), (high, low));
|
||||
// preview monitor on by default; transmit has no device yet
|
||||
assert!(s.state.multicam_view.show_preview);
|
||||
assert_eq!(s.execute("multicam.showPreviewMonitor", json!({})).unwrap()["enabled"], false);
|
||||
let t = s.execute("multicam.transmitView", json!({})).unwrap();
|
||||
assert_eq!((t["enabled"].as_bool(), t["device"].is_null()), (Some(true), true));
|
||||
assert_eq!(s.execute("multicam.audioFollowsVideo", json!({})).unwrap()["enabled"], true);
|
||||
}
|
||||
|
||||
/// Process CPU time in seconds (user + system), for the perf test.
|
||||
fn process_cpu() -> f64 {
|
||||
let out = Command::new("ps").args(["-o", "cputime=", "-p", &std::process::id().to_string()]).output().ok();
|
||||
|
||||
@@ -9,6 +9,7 @@ use crate::Session;
|
||||
/// measure an interval).
|
||||
pub fn decode_json() -> Value {
|
||||
let g = filmcraft_codecs::gop_stats();
|
||||
let hw = filmcraft_codecs::hw::hw_stats();
|
||||
json!({
|
||||
"requests": g.hits + g.misses,
|
||||
"cacheHits": g.hits,
|
||||
@@ -22,6 +23,17 @@ pub fn decode_json() -> Value {
|
||||
"evicted": g.evicted,
|
||||
"decodeMs": g.decode_ns as f64 / 1e6,
|
||||
"decodeMsPerSample": g.decode_ms_per_sample(),
|
||||
"framesDecoded": g.frames,
|
||||
// Settings ▸ Playback ▸ Hardware decoding: pictures from OS hardware decoders vs ours,
|
||||
// hardware decoders created, streams handed to software up front, mid-stream fallbacks.
|
||||
"hardware": {
|
||||
"enabled": filmcraft_codecs::hw::hardware_decoding(),
|
||||
"frames": hw.frames,
|
||||
"softwareFrames": g.frames.saturating_sub(hw.frames),
|
||||
"sessions": hw.sessions,
|
||||
"declined": hw.declined,
|
||||
"fallbacks": hw.fallbacks,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
@@ -48,9 +60,24 @@ mod tests {
|
||||
s.execute("file.openDemoProject", json!({})).unwrap();
|
||||
let undo = s.history.undo.len();
|
||||
let v = s.execute("perf.stats", json!({})).unwrap();
|
||||
for k in ["requests", "cacheHitRate", "seeks", "samplesDecoded", "samplesSkipped", "draftFrames", "h264Threads", "decodeMs", "decodeMsPerSample"] {
|
||||
for k in [
|
||||
"requests",
|
||||
"cacheHitRate",
|
||||
"seeks",
|
||||
"samplesDecoded",
|
||||
"samplesSkipped",
|
||||
"draftFrames",
|
||||
"h264Threads",
|
||||
"decodeMs",
|
||||
"decodeMsPerSample",
|
||||
"framesDecoded",
|
||||
] {
|
||||
assert!(v["decode"][k].is_number(), "decode.{k} in {v}");
|
||||
}
|
||||
for k in ["frames", "softwareFrames", "sessions", "declined", "fallbacks"] {
|
||||
assert!(v["decode"]["hardware"][k].is_number(), "decode.hardware.{k} in {v}");
|
||||
}
|
||||
assert!(v["decode"]["hardware"]["enabled"].is_boolean());
|
||||
assert!(v["media"]["openSources"].is_number());
|
||||
assert_eq!(v["jobs"]["running"], json!(0));
|
||||
assert_eq!(s.history.undo.len(), undo, "a query adds no undo step");
|
||||
|
||||
@@ -151,8 +151,22 @@ fn kf(t_secs: f64, v: ParamValue) -> Keyframe {
|
||||
Keyframe::new(Tick((t_secs * TICKS_PER_SECOND as f64).round() as i64), v)
|
||||
}
|
||||
|
||||
/// A fresh instance of a built-in effect (a bare instance named `id` if the registry lacks it,
|
||||
/// which the preset tests rule out).
|
||||
pub(crate) fn effect_instance(id: &str) -> EffectInstance {
|
||||
filmcraft_project::find_effect(id).map(|d| d.instance()).unwrap_or_else(|| EffectInstance {
|
||||
effect: id.to_string(),
|
||||
enabled: true,
|
||||
params: Default::default(),
|
||||
masks: Vec::new(),
|
||||
post_fader: false,
|
||||
essential: false,
|
||||
layer: None,
|
||||
})
|
||||
}
|
||||
|
||||
fn effect(id: &str, params: &[(&str, ParamValue)]) -> EffectInstance {
|
||||
let mut e = filmcraft_project::find_effect(id).unwrap_or_else(|| panic!("built-in preset uses unknown effect {id}")).instance();
|
||||
let mut e = effect_instance(id);
|
||||
for (k, v) in params {
|
||||
if let Some(p) = e.params.get_mut(*k) {
|
||||
p.value = v.clone();
|
||||
|
||||
@@ -153,7 +153,7 @@ impl PreviewStore {
|
||||
return;
|
||||
}
|
||||
if let Some(old) = &old
|
||||
&& (old.starts_with(std::env::temp_dir()) || self.temp_root().is_some_and(|r| old.starts_with(r)))
|
||||
&& (old.starts_with(crate::temp_dir()) || self.temp_root().is_some_and(|r| old.starts_with(r)))
|
||||
{
|
||||
let _ = std::fs::create_dir_all(&dir);
|
||||
for name in self.files.read().unwrap_or_else(|e| e.into_inner()).iter() {
|
||||
@@ -410,7 +410,7 @@ fn default_temp_dir() -> Option<PathBuf> {
|
||||
if cfg!(target_arch = "wasm32") {
|
||||
return None;
|
||||
}
|
||||
Some(untitled_dir(&std::env::temp_dir().join("FilmCraft Previews")))
|
||||
Some(untitled_dir(&crate::temp_dir().join("FilmCraft Previews")))
|
||||
}
|
||||
|
||||
/// A fresh per-process folder for an unsaved project's previews under `root`.
|
||||
@@ -543,6 +543,7 @@ pub fn render(s: &mut Session, mode: RenderMode, p: &Value) -> Result<Value> {
|
||||
prog.finished.store(true, Ordering::Relaxed);
|
||||
*res.lock().unwrap_or_else(|x| x.into_inner()) = Some(r);
|
||||
};
|
||||
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
|
||||
s.jobs.push(job);
|
||||
let wait = p.get("wait").and_then(Value::as_bool).unwrap_or(false);
|
||||
if wait || cfg!(target_arch = "wasm32") {
|
||||
@@ -629,6 +630,7 @@ pub fn render_audio(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
prog.finished.store(true, Ordering::Relaxed);
|
||||
*res.lock().unwrap_or_else(|x| x.into_inner()) = Some(r);
|
||||
};
|
||||
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
|
||||
s.jobs.push(job);
|
||||
if p.get("wait").and_then(Value::as_bool).unwrap_or(false) || cfg!(target_arch = "wasm32") {
|
||||
run();
|
||||
@@ -680,7 +682,7 @@ pub fn read_wav_f32(b: &[u8]) -> Option<Vec<f32>> {
|
||||
if !float_stereo {
|
||||
return None;
|
||||
}
|
||||
return Some(body.chunks_exact(4).map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect());
|
||||
return Some(body.as_chunks::<4>().0.iter().map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect());
|
||||
}
|
||||
i += 8 + len + (len & 1);
|
||||
}
|
||||
|
||||
Loaded 100 of 332 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user