Compare commits

..
455 Commits
Author SHA1 Message Date
Brandon ThomasandClaude Opus 5.5 7bd7621262 Bump version to 0.5.0 (from 0.4.0) for release; refresh contributors
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 21:02:11 -04:00
Brandon Thomas 7b6c134472 Merge pull request #363 from jerrykimo/fix/chinese-ui-glyphs
Fix Chinese glyph rendering in native timeline and UI labels
2026-10-09 11:02:46 -04:00
Brandon ThomasandClaude Opus 5.5 12f2ea17d6 CJK fonts: Chinese faces after the Japanese ones, so Japanese keeps Japanese glyph forms
The two share code points; putting the Chinese faces first drew Japanese UI text's kanji
with Chinese shapes. Also clears the test's texture deltas (epaint 0.36 panics on
dropping them unapplied).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:02:38 -04:00
Brandon Thomas e9c09d243a Merge remote-tracking branch 'origin/main' into fix-363 2026-10-09 10:56:40 -04:00
Brandon Thomas cb03d34c15 Merge pull request #361 from wisegamesyt-droid/fix-multitrack-audio
feat(media, engine): implement multi-stream container audio support and schema v13
2026-10-09 10:56:24 -04:00
Brandon Thomas e06018e07b Merge remote-tracking branch 'origin/main' into fix-361 2026-10-09 10:53:43 -04:00
Brandon ThomasandClaude Opus 5.5 fc32a19bd1 Proxies with fewer audio streams keep the proxy picture, audio from the original
Rejecting the whole proxy lost proxy playback for every multi-track recording
(and video-only proxies of media with audio); only the audio needs the original.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:53:42 -04:00
Brandon Thomas ef6391df90 Merge pull request #428 from storytold/fix-clippy-as-chunks-2
CI: gpu Push test uses as_chunks (clippy on Rust 1.99)
2026-10-09 10:38:46 -04:00
Brandon ThomasandClaude Opus 5.5 1f5ee123b2 CI: gpu Push test uses as_chunks (clippy chunks_exact_to_as_chunks on Rust 1.99)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:38:39 -04:00
Brandon Thomas d19e1b2783 Merge remote-tracking branch 'origin/main' into fix-361 2026-10-09 10:34:54 -04:00
Brandon Thomas ade68437d7 Merge pull request #269 from carlitosteso3/feat/new-sequence-dialog
New Sequence… opens the Sequence Settings dialog as New Sequence (stacked on #422)
2026-10-09 10:33:02 -04:00
Brandon ThomasandClaude Opus 5.5 3cc03f1447 Source playback: has_audio() for multi-stream MediaInfo
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:22:48 -04:00
carlitosteso3andClaude Opus 5.5 f0e1cfcec5 New Sequence… opens the Sequence Settings dialog as New Sequence
File ▸ New ▸ Sequence… (Cmd+N) and the Project panel's New Item ▸ Sequence…
made a sequence with the default settings at once. They now open the
Sequence Settings dialog (#355) as New Sequence (`open_new`): the General
tab from the default settings, named like file.newSequence would name it,
and a Tracks tab with the number of video and audio tracks. OK runs
file.newSequence with them; drop-frame timecode and Maximum Render Quality,
which the command has no parameter for, follow with sequence.settings when
they aren't the defaults. Making a sequence from a clip, or by dropping one
on an empty timeline, still passes params and stays direct.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:22:47 -05:00
carlitosteso3andClaude Opus 5.5 632420a01e Sequence Settings: Sequence Name on the General tab
Follow-up to #355, as suggested in #225: the dialog's General tab gets a
Sequence Name field, filled with the sequence's name when it opens. OK
sends `name` to `sequence.settings` (which already accepts it) only when it
changed, so renaming is part of the dialog's single undo step; a blank name
keeps the old one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:21:45 -05:00
Brandon Thomas b32d1cd5ed Merge remote-tracking branch 'origin/main' into fix-361 2026-10-09 10:17:58 -04:00
Brandon ThomasandClaude Opus 5.5 850ac4342f CLI test: probe reports audio as audio_streams (schema v13 MediaInfo)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:17:40 -04:00
Brandon Thomas 9c43a0d02f Merge pull request #425 from storytold/pr/353-monitor-export-frame
M3.12: Export frames from Source and Program monitors (#353 by @cshapiro)
2026-10-09 10:14:15 -04:00
Brandon Thomas 81eb842e2e Merge remote-tracking branch 'origin/main' into fix-353 2026-10-09 10:13:59 -04:00
Brandon Thomas a880fa9154 Merge pull request #386 from storybored379/feat/nvenc-export
platform: enable existing NVENC H.264 export on Linux
2026-10-09 10:13:43 -04:00
Brandon Thomas 38f4255735 Merge remote-tracking branch 'origin/main' into fix-386 2026-10-09 10:13:01 -04:00
Brandon ThomasandClaude Opus 5.5 f267a941fb Merge origin/main into feat/nvenc-export
main gained NVENC H.265 Main / Main 10 (#328, #360), export stage timings
(#370) and VA-API decoding (#358). Resolution: main's NVENC session, FFI and
export factory, with this PR's changes ported onto them:
- ffi: the portable Guid (now also for the HEVC GUIDs), NV_ENC_DEVICE_TYPE_CUDA;
- session: device.rs (Direct3D 11 / retained CUDA primary context), the
  Linux libnvidia-encode.so.1 loader, Device::KIND;
- export: factory_with and its tests, the even-size check, the saturating
  max bitrate; H.265 through NVENC stays Windows-only (the factory declines
  it on Linux, where nothing probed or tested it);
- Cargo.toml: main's Linux libloading dependency covers both users;
- docs: README / ROADMAP / architecture now say NVENC H.264 runs on Windows
  and Linux; ADR and platform README keep both addenda / test rows;
- abi_tests: note why the MSVC-generated expectations hold on LP64 Linux.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:12:46 -04:00
Brandon Thomas 5f4e2d7c99 Merge remote-tracking branch 'origin/main' into fix-353
# Conflicts:
#	crates/engine/src/keyboard.rs
#	crates/engine/src/lib.rs
#	crates/ui-egui/src/panels/monitor.rs
2026-10-09 10:11:15 -04:00
Brandon Thomas ab06c110d1 Merge pull request #423 from storytold/pr/338-source-playback
M3.12: Source playback, clip dragging and monitor range editing (#338 by @cshapiro)
2026-10-09 10:10:34 -04:00
Brandon Thomas 560355f3d2 Merge remote-tracking branch 'origin/main' into fix-338 2026-10-09 10:09:57 -04:00
Brandon Thomas 666e891538 Merge pull request #370 from SaymonMaster14/m8/export-stage-timings
Export: stage timings in perf.stats, and render the next batch while encoding
2026-10-09 10:08:49 -04:00
Brandon ThomasandClaude Opus 5.5 9526ba50e3 M3.12: Spanish strings for Source playback controls
#238's i18n landed on main; its catalog-coverage tests need Spanish for
the new Loop marked range checkbox and the Source play/stop command names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:08:42 -04:00
Brandon ThomasandClaude Opus 5.5 285e81b5da Merge origin/main into m8/export-stage-timings
NVENC export: keep main's Main 10 / Main dispatch and time both colour
conversions (rgbf_to_yuv420_10, rgba_to_yuv420_8) as Stage::Convert.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:08:35 -04:00
Brandon ThomasandClaude Opus 5.5 bb4c6d0bb2 Format: repair out-of-range audio stream data on load instead of refusing the project
A clip with audio_stream >= MAX_AUDIO_STREAMS is reset to stream 0 and media
with more than MAX_AUDIO_STREAMS streams keeps the first ones, each with a
warning: one bad index must never lose a whole project.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:08:10 -04:00
Brandon ThomasandClaude Opus 5.5 701d7c7111 Merge origin/main into multi-stream audio
Keeps #359's contiguous_starts audio timing, now per stream in Mp4Audio /
MkvAudio (starts and preroll in source sample frames). MP4 audio streams and
their decoders are built from the same (track, info) pairs so they stay
index-aligned. Adapts new main code to audio_streams / TrackItem::audio_stream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:08:09 -04:00
Brandon Thomas e196de8f64 Merge remote-tracking branch 'origin/main' into fix-338
# Conflicts:
#	ROADMAP.md
#	crates/engine/src/commands.rs
#	crates/engine/src/lib.rs
#	crates/ui-egui/README.md
#	crates/ui-egui/src/panels/monitor.rs
2026-10-09 10:06:01 -04:00
Brandon Thomas 95230b9072 Merge pull request #360 from SaymonMaster14/m3/nvenc-hevc-main10
Windows: NVENC H.265 Main 10 HDR (PQ / HLG) export
2026-10-09 10:05:10 -04:00
Brandon ThomasandClaude Opus 5.5 c9e2bfcac0 Merge origin/main into m3/nvenc-hevc-main10
Export panel: the HEVC profile labels go through tl! with Spanish entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:04:58 -04:00
Brandon Thomas 95e585d989 Merge pull request #418 from storytold/pr/337-windows-validation
WINCI: Fix Windows paths and portable validation (#337 by @cshapiro)
2026-10-09 10:04:57 -04:00
Brandon Thomas d84fdc1268 Merge remote-tracking branch 'origin/main' into fix-337 2026-10-09 10:04:46 -04:00
Brandon Thomas 9a61c7610b Merge pull request #308 from WavJaby/perf/shared-transitions
perf(gpu): composite Push transitions on the shared GPU path
2026-10-09 10:04:24 -04:00
Brandon ThomasandClaude Opus 5.5 faa4bf953d Render plan: restore the detailed module doc and add the Push / PlanStep paragraph
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:03:38 -04:00
Brandon Thomas a6ac2d7eff Merge remote-tracking branch 'origin/main' into fix-308
# Conflicts:
#	crates/ui-egui/src/perf.rs
2026-10-09 10:01:25 -04:00
Brandon ThomasandClaude Opus 5.5 afd5a6b347 M3: NVENC Main 10 review fixes: saturating plane sizes, as_chunks for clippy
encode / encode_10 compute the plane sizes they check with saturating_mul;
fill_p010, rgbf_to_yuv420_10 and the HDR export test use as_chunks instead of
chunks_exact with a constant size (clippy::chunks_exact_to_as_chunks).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:01:08 -04:00
Brandon Thomas 3931562101 Merge pull request #238 from txiki739/i18n-spanish-panels
i18n: Spanish for the whole interface, catalog files and a remembered language
2026-10-09 10:00:38 -04:00
Brandon ThomasandClaude Opus 5.5 a66be150e3 WINCI: ignore the mixer throughput benchmark only where it times wall clock
On Unix the benchmark measures the test thread's CPU time, so suite load
does not move it and it keeps running. Without a per-thread CPU clock
(Windows) it falls back to wall time, which is load-dependent and flaky on
shared CI runners; ignore it only there and document how to run it.
The GPU effect-source hunks were dropped in the merge in favour of #376's
fs_fx_source, now on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:59:45 -04:00
Brandon Thomas 156535a6e4 Merge origin/main into i18n-spanish-panels
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# Conflicts:
#	crates/codecs/tests/mpeg_oracle.rs
2026-10-09 09:59:28 -04:00
Brandon Thomas efa89a7d2d Merge remote-tracking branch 'origin/main' into fix-337
# Conflicts:
#	crates/gpu/src/fx_tests.rs
2026-10-09 09:59:26 -04:00
Brandon Thomas 4d3a814968 Merge pull request #376 from sanyamsingh001/codex/gpu-effect-source
GPU.EFFECT_SOURCE: preserve exact texel centers for effect inputs
2026-10-09 09:59:15 -04:00
Brandon Thomas ef46f64a28 Merge pull request #375 from WavJaby/perf/web-yuv-frames
perf(web): feed WebCodecs YUV planes to the shared compositor
2026-10-09 09:58:35 -04:00
Brandon Thomas 1dea07a85a Merge remote-tracking branch 'origin/main' into fix-376 2026-10-09 09:58:16 -04:00
Brandon Thomas 9c24f4419c Merge pull request #368 from sanyamsingh001/codex/linux-validation-baseline
LINUX.VALIDATION: honor preset shortcuts and retain every MPEG oracle timestamp
2026-10-09 09:58:10 -04:00
Brandon ThomasandClaude Opus 5.5 183884fd34 web: WebCodecs busy check honours STALE_MS; checked row offsets in plane()
A copyTo that never settles kept copying > 0 and bypassed the stale timeout,
wedging the session; now busy requires recent output. plane() computes each
row's start/end with checked arithmetic and bounds it by the validated end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:58:06 -04:00
Brandon Thomas 6893b19314 Merge remote-tracking branch 'origin/main' into fix-368
# Conflicts:
#	contributors/people.toml
2026-10-09 09:57:52 -04:00
Brandon ThomasandClaude Opus 5.5 de1320dc88 Merge origin/main into i18n-spanish-panels
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:57:17 -04:00
Brandon ThomasandClaude Opus 5.5 82cb6fbc74 i18n: Spanish for strings added on main (Sequence Settings, keyframe navigator, export)
Translate the new Sequence Settings dialog (labels, sections, tabs,
values, hints), the shared keyframe navigator's tooltips, the
non-macOS tool and mask hints, Include Alpha Channel and the export
status. Update the hardware decoding description to main's wording.
Automation ids and labels stay English.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:57:13 -04:00
Brandon Thomas 67634241c8 Merge remote-tracking branch 'origin/main' into fix-375 2026-10-09 09:56:15 -04:00
Brandon Thomas 0d2c21163c Merge pull request #299 from dcarnelutti/export/trim-pool-when-done
export: free the frame pool when an export ends
2026-10-09 09:56:07 -04:00
Brandon ThomasandClaude Opus 5.5 79a3b623fe Pool trim: free only the float shelf, and only after a standalone export
pool::trim() from Pipeline::drop flushed the whole process-wide pool after every
render-preview segment and proxy, and threw away playback's U8/U16 planes too.
Now Pipeline::drop calls pool::trim_f32() only when !settings.part_of_batch.
PoolStats gains f32_idle_bytes so the tests can check the float shelf alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:55:47 -04:00
Brandon Thomas 584f742bf4 Merge pull request #247 from switch87/readme-gentoo
README: install on Gentoo from the ::snakebyte overlay
2026-10-09 09:55:23 -04:00
Brandon ThomasandClaude Opus 5.5 7050055a58 README: move the Gentoo overlay into Downloads > Linux
The Gentoo paragraph sat after the build-from-source and control-protocol
notes; it belongs with the other Linux install options. Mark it clearly as
community-maintained.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:54:46 -04:00
Brandon Thomas c21c86089f Merge remote-tracking branch 'origin/main' into fix-247
# Conflicts:
#	contributors/people.toml
2026-10-09 09:52:32 -04:00
Brandon Thomas 6ebc21f093 Merge remote-tracking branch 'origin/main' into fix-360
# Conflicts:
#	README.md
#	ROADMAP.md
#	crates/platform/README.md
2026-10-09 09:46:48 -04:00
Brandon Thomas 18f5bad7a8 Merge pull request #328 from SaymonMaster14/m2/nvenc-hevc
Windows: NVENC H.265 (HEVC) Main hardware encoding for Export
2026-10-09 09:46:13 -04:00
Brandon Thomas 97a8eaffec Merge remote-tracking branch 'origin/main' into fix-299
# Conflicts:
#	docs/control-protocol.md
2026-10-09 09:45:27 -04:00
Brandon Thomas 66a74afc38 Merge pull request #365 from DWAA1660/fix/windows-dx12-startup
Fix #179: leave OpenGL out of the Windows wgpu instance (Vulkan | DX12 by default)
2026-10-09 09:45:03 -04:00
Brandon ThomasandClaude Opus 5.5 b3f2a89944 Windows: default to Vulkan | DX12 | Metal (drop only GL) instead of forcing DX12
Forcing DX12 compiles shaders through FXC, which rejects fx.wgsl (silent CPU
fallback), and leaves machines without DX12 with no backend. Leaving GL out is
what avoids the AMD atio6axx.dll crash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:44:20 -04:00
Brandon Thomas aff136ff67 Merge remote-tracking branch 'origin/main' into fix-365
# Conflicts:
#	apps/filmcraft/src/main.rs
2026-10-09 09:44:20 -04:00
Brandon ThomasandClaude Opus 5.5 cc5e3c1065 Revert shortcut-preset changes in favour of #337's approach
The maintainers chose #337's handling of Ctrl/Cmd shortcut collisions off
macOS (drop_control_collisions / audit_for / builtin_for), which audits the
collisions and keeps documented winners rather than keeping the first listed
binding. This reverts crates/engine/src/shortcuts.rs and its tests to main
so the two approaches don't overlap; the MPEG-oracle ffprobe-JSON change and
the frame-pool test rewrite from this PR are kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:40:15 -04:00
Brandon Thomas 1b4b6fbffa Merge remote-tracking branch 'origin/main' into fix-368
# Conflicts:
#	contributors/people.toml
2026-10-09 09:40:15 -04:00
Brandon Thomas 1879d472e1 Merge pull request #417 from storytold/fix-clippy-as-chunks
CI: gpu test uses as_chunks (clippy on Rust 1.99)
2026-10-09 09:38:52 -04:00
Brandon ThomasandClaude Opus 5.5 7a29eb4dff CI: gpu test uses as_chunks (clippy chunks_exact_to_as_chunks on Rust 1.99)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:38:45 -04:00
Brandon Thomas 570d06d5bd Merge remote-tracking branch 'origin/main' into fix-328
# Conflicts:
#	Cargo.lock
#	README.md
#	ROADMAP.md
#	crates/platform/Cargo.toml
#	crates/platform/README.md
#	docs/architecture.md
2026-10-09 09:34:12 -04:00
Brandon ThomasandClaude Opus 5.5 c497b648f5 Merge origin/main into i18n-spanish-panels
Resolve conflicts: keep main's macOS/other hint text and mask vertex
tooltip variants (each wrapped in tl!), take main's shared keyframe_nav
helper and translate its tooltips.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:34:01 -04:00
Brandon Thomas d63df6882b Merge pull request #256 from wstock/fix/245-ctrl-chords-control
control: Ctrl+ chords through ui.key fire shortcuts off macOS (#245)
2026-10-09 09:31:39 -04:00
Brandon Thomas a739ffea84 Merge remote-tracking branch 'origin/main' into fix-256
# Conflicts:
#	crates/ui-egui/tests/keyboard_ui.rs
#	docs/control-protocol.md
2026-10-09 09:29:27 -04:00
Brandon Thomas e5214ab7eb Merge pull request #355 from swillunderscore/sequence-settings-dialog
Sequence ▸ Sequence Settings… opens Premiere's dialog (#172, #287)
2026-10-09 09:28:59 -04:00
Brandon Thomas afe35e4c41 Merge pull request #240 from WavJaby/fix/web-import-collision
fix(web): preserve distinct same-name media imports
2026-10-09 09:28:53 -04:00
Brandon Thomas de0299bd1b Merge remote-tracking branch 'origin/main' into fix-240
# Conflicts:
#	apps/filmcraft-web/tests/smoke.mjs
2026-10-09 09:27:59 -04:00
Brandon Thomas 96f76bac2a Merge remote-tracking branch 'origin/main' into fix-355
# Conflicts:
#	contributors/people.toml
2026-10-09 09:27:39 -04:00
Brandon Thomas c2f2018707 Merge pull request #359 from IppSec/fix/236-audio-crackle
#236: audio in MKV and OBS-remuxed MP4 plays without crackling
2026-10-09 09:27:34 -04:00
Brandon Thomas 6e80bab6b8 Merge remote-tracking branch 'origin/main' into fix-359
# Conflicts:
#	contributors/people.toml
2026-10-09 09:26:08 -04:00
Brandon Thomas 0bae537ad0 Merge pull request #300 from dcarnelutti/platform/warm-hevc-probe
platform: ask for the hardware HEVC encoder in the background at startup
2026-10-09 09:25:32 -04:00
Brandon Thomas 22d2046c0a Merge pull request #270 from Tom1tk/windows-modifier-hints
Windows: platform modifier names in status-bar and mask hints
2026-10-09 09:25:26 -04:00
Brandon Thomas 9736b2a6ba Merge pull request #264 from carlitosteso3/fix/recovery-retry-197
#197, #193: web recovery retries failed snapshot writes and keeps skipped snapshots dirty
2026-10-09 09:25:20 -04:00
Brandon Thomas 48e0b65110 Merge pull request #237 from wstock/fix/230-wasm-clock
#230: jobs time themselves with web_time, so Create Proxies and mask tracking don't panic on the web
2026-10-09 09:25:14 -04:00
Brandon Thomas 202d5f061c Merge pull request #242 from wstock/fix/160-png-alpha
export: keep alpha in PNG/TIFF sequences (Include Alpha Channel) (#160)
2026-10-09 09:25:08 -04:00
Brandon Thomas 5cca829c39 Merge pull request #369 from storybored379/fix/mcp-cancel-numbered-movies
MCP: cancelling a movie export keeps numbered files beside it
2026-10-09 09:25:03 -04:00
Brandon Thomas 0409fdccf4 Merge pull request #372 from storybored379/fix/graphic-duration-placement
graphics: keep duration separate from timeline placement
2026-10-09 09:24:58 -04:00
Brandon Thomas d1f491823c Merge pull request #373 from storybored379/fix/audio-gain-result
mixer: report only the audio gains actually applied
2026-10-09 09:24:51 -04:00
Brandon Thomas bf0f2bd8ff Merge pull request #305 from Pathos0925/fix/296-mono-loudness
#296: Auto-Match measures a mono clip as one channel
2026-10-09 09:24:45 -04:00
Brandon Thomas bf21ee86c2 Merge pull request #282 from Tom1tk/windows-export-paths
Windows: export paths expand ~, accept trailing \ and verbatim paths
2026-10-09 09:24:38 -04:00
Brandon Thomas 996ccf72fd Merge pull request #349 from rossaai/rossa/embedding-hooks
Embedding: set_context on the web app, header and status bar flags, status bar colour token
2026-10-09 09:24:31 -04:00
Brandon Thomas 7cf322f19d Merge pull request #319 from WavJaby/fix/yuv-code-range
fix(gpu): normalize YUV code ranges independently of texture scaling
2026-10-09 09:24:25 -04:00
Brandon Thomas 275abb3193 Merge pull request #385 from SaymonMaster14/m4c/background-decoders
Thumbnails decode with small decoders that are not kept (#55)
2026-10-09 09:24:19 -04:00
Brandon Thomas cae8eb9f4f Merge pull request #381 from SaymonMaster14/m4/memory-stress
Decode caches: navigation bursts no longer keep every clip's decoder and frames (#55)
2026-10-09 09:24:13 -04:00
Brandon Thomas ae79acad8a Merge pull request #241 from WavJaby/fix/import-bin
fix(import): use the open Project bin as destination
2026-10-09 09:24:07 -04:00
Brandon Thomas 16ba523625 Merge pull request #253 from wstock/fix/244-project-delete-named-bins
engine: project.delete takes explicit bin ids without a Project-panel selection (#244)
2026-10-09 09:24:02 -04:00
Brandon Thomas 20ece5313e Merge pull request #261 from wstock/fix/243-delete-key
shortcuts: Delete clears the selected timeline clips (#243)
2026-10-09 09:23:56 -04:00
Brandon Thomas e7fe8f46b7 Merge pull request #250 from MrCloudy2/fix/unlink-keeps-clicked-clip-220
#220: Unlink leaves only the right-clicked clip selected
2026-10-09 09:23:49 -04:00
Brandon Thomas 5d69487925 Merge pull request #307 from justjohn12345/pr/bugfix
Keyframes: Properties gets Premiere's navigator, Effect Controls a time ruler
2026-10-09 09:23:44 -04:00
Brandon Thomas 354bb86332 Merge pull request #304 from justjohn12345/pr/monitor-follows-drags
Program monitor: the picture follows a value that is dragged
2026-10-09 09:23:37 -04:00
Brandon Thomas 55b6c0a916 Merge pull request #221 from carlitosteso3/fix/playhead-controls-208
Go to In / Go to Out move to the sequence start / end without marks (#208)
2026-10-09 09:23:31 -04:00
Brandon Thomas 929e05d996 Merge pull request #303 from MarekSpodBiedry/issue-139
Issue #139: portable.txt next to the exe keeps the data directory beside it
2026-10-09 09:23:25 -04:00
Brandon Thomas 0f3343cc4a Merge pull request #277 from midasdf/fix/vertical-japanese-punctuation
M10: use Japanese vertical forms and origins for titles
2026-10-09 09:23:20 -04:00
Brandon Thomas 298391574e Merge pull request #289 from midasdf/fix/vertical-grapheme-cells
M10: keep combining marks in one vertical text cell
2026-10-09 09:23:14 -04:00
Brandon Thomas e919d3f80b Merge pull request #249 from switch87/desktop-log-file
Desktop: write log records to standard error and a rotating log file
2026-10-09 09:23:08 -04:00
Brandon Thomas d16f9adb0a Merge pull request #387 from txiki739/fix/linux-tar-installer
#214: install script for the Linux tarball
2026-10-09 09:23:02 -04:00
Brandon Thomas bfa819a448 Merge pull request #358 from JordyUSA/vaapi-h264-decode
HW5: Linux VA-API hardware decoding for H.264 and HEVC (Main / Main 10)
2026-10-09 09:22:56 -04:00
Brandon Thomas 3e6043c46a Merge pull request #263 from XusBadia/dmg-background
macOS DMG: branded window background and icon layout
2026-10-09 09:20:31 -04:00
Brandon Thomas 9c24778039 Merge pull request #399 from ItzBloxy/fix/259-trim-hit-testing
#259: the edge you press is the edge you trim
2026-10-09 08:17:17 -04:00
Brandon Thomas 6a240ca701 Merge pull request #391 from txiki739/fix/export-menu-paths
#382: File > Export entries work from the menus
2026-10-09 08:17:12 -04:00
Brandon Thomas 6a1145eb42 Merge pull request #324 from mjohnson7cd/fix/301-nonstandard-fps-stepping
fix: frame stepping broken for non-standard frame rates (#301)
2026-10-09 08:17:06 -04:00
Brandon Thomas 28235e25b8 Merge pull request #394 from txiki739/fix/reverse-source-frames
#318: a reversed clip shows exactly its source range
2026-10-09 08:17:01 -04:00
Brandon Thomas 7605a7a7ce Merge pull request #392 from txiki739/fix/fcp7-subframe-spans
#340: FCP7 XML clip source span always matches its record span
2026-10-09 08:16:56 -04:00
Brandon Thomas 475b30d793 Merge pull request #393 from MarekSpodBiedry/issue-379
#379: preset export and import go through the host, so the web build no longer traps
2026-10-09 08:16:51 -04:00
Brandon Thomas fee418e2b6 Merge pull request #252 from WavJaby/fix/workspace-lints
chore(lints): satisfy strict workspace Clippy on Rust 1.95
2026-10-09 08:16:46 -04:00
Brandon Thomas d51f6b1b7d Merge pull request #362 from JordyUSA/fix/341-export-message
#341: CLI export message names the file written, not the requested path
2026-10-09 08:16:41 -04:00
Brandon Thomas 0f409f3fb1 Merge pull request #297 from mjohnson7cd/fix/285-cli-broken-pipe
fix: filmcraft-cli exits quietly on broken pipe (#285)
2026-10-09 08:16:36 -04:00
BloxyandClaude Opus 5.5 7401f5038b contributors: credit @ItzBloxy
My own entry for the About window: display name "Bloxy".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 12:17:08 +02:00
BloxyandClaude Opus 5.5 af2b4601c8 #259: caption edges trim from where they were pressed
Caption blocks chose move / trim in / trim out where egui recognised the drag, about 9 px in
from a press 2 px inside the edge and past the 5 px edge zone, so they moved instead of
trimming. They now decide at the press origin and measure the drag from it. UI test that
failed before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:46:35 +02:00
BloxyandClaude Opus 5.5 42cc47e7db #259: the timeline shows the edge a press will grab
Hovering a clip edge draws the edit-point bracket at half strength on the edge a press would
grab: red for a ripple; yellow for a trim, roll, rate stretch or remix; on both sides of the
cut for a roll. It follows the tool and the modifiers, from the same grab_at as the press.
The bracket drawing is shared with the selected edit points (trim_monitor::paint_bracket).

An ignored test renders the four cases at a demo cut to PNGs for review:
cargo test -p filmcraft-ui-egui --test timeline_trim_ui -- --ignored hover_bracket_screenshots

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:40:07 +02:00
txiki739andClaude Opus 5.5 dfd67b2774 #318: a reversed clip shows exactly its source range
A reversed clip's source time is the instant just before its mirrored
position (source_out - rel - 1 tick). The MP4 reader rounds a request to
the nearest unit of the track timescale before picking the frame at or
before it, so one tick below a frame boundary landed on the next frame:
a clip trimmed to source frames 10-29 played 30 down to 11.

video_source_time snaps a reversed clip's video request down to the
start of the media frame that contains it (clamped to the source In and
the unsnapped time). Both places that ask a source for a clip's picture
use it: media_frame and the plan path. source_time_at is unchanged for
audio and nested sequences, and forward clips and frame holds keep
their request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:28:24 +02:00
BloxyandClaude Opus 5.5 4eb34ae6c7 #259: the edge you press is the edge you trim
Pressing a clip edge and dragging inward selected and moved the clip: the press was hit-tested
only once egui recognised a drag, 6 pt later, by when the pointer had left the 7 px edge zone.
And at a cut the left clip's Out edge claimed both sides, so the right clip's In edge could not
be grabbed.

What a press grabs is now worked out on the frame the button goes down (timeline_hit::grab_at,
shared by the cursor, the press and ui.timeline.hit) and acted on by both the click and the
drag. Inside a clip beats outside it, so the side of a cut picks the clip; zero-width clips
have no edges. Trims, rolls, rate stretches and remixes move by how far the pointer moved since
the press, and a moved clip stays under the point it was grabbed by. ui.timeline.hit takes
`modifiers` and reports `kind`, the trim a press would start.

Unit tests for the zones and tools; headless UI tests that drag a pixel or a few per frame at
60 fps (before: the clip moved, the wrong clip was trimmed, a wobbly click selected the clip, a
moved clip lagged).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:27:48 +02:00
txiki739andClaude Opus 5.5 43e3d26823 #218: System Language follows the operating system's language
Settings > General > Interface Language gains System Language
(`system`), now the default: a new installation shows the first of the
user's preferred languages that the interface has (es-419, pt_BR.UTF-8,
ja-JP...; any Portuguese gets pt-br), otherwise English. A language
already chosen, or saved in preferences.json, is kept.

The host supplies the preferred languages through
HostHooks::system_languages: sys-locale on the desktop (LANGUAGE,
LC_ALL, LC_MESSAGES, LANG on Linux; the OS lists on macOS/Windows) and
navigator.languages on the web. Without the hook (tests, headless),
System Language is English, so tests do not depend on the machine's
locale.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:06:43 +02:00
MarekSpodBiedry 55e43e6147 #379: preset export and import go through the host, so the web build no longer traps 2026-10-09 10:45:28 +02:00
txiki739andClaude Opus 5.5 d39692af63 #340: FCP7 XML clip source span always matches its record span
write_clip rounded start, end, in and out to frames independently, so
a clip placed by seconds at 23.976 (start 0, in 20 s, 3 s long) was
written as start 0 / end 72 / in 480 / out 551: 71 source frames for
72 record frames.

At normal speed and on a freeze frame the source span is the record
span, so out is now in + (end - start). Speed-changed clips keep their
independently rounded out. The master duration is raised to out when
rounding pushes out past it; pproTicksIn/Out still carry exact ticks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:37:12 +02:00
txiki739andClaude Opus 5.5 a076e50553 #382: File > Export entries work from the menus
A menu entry runs its command with no parameters, so every File >
Export entry whose command needs a `path` failed with "need `path`",
from the egui menu bar and the native macOS menu alike.

- Media… opens the Export mode, like Cmd+M (Premiere's File > Export >
  Media); with a `path` it still exports directly.
- EDL, Final Cut Pro XML, FCPXML, OpenTimelineIO, Avid Log Exchange and
  Selection as FilmCraft Project ask the host's save panel for the
  destination, suggest the sequence's (or project's) name with the
  right extension, then export. "Selection" suggests "<project>
  selection.fcproj" so it does not offer to overwrite the open project.
  Cancelling writes nothing and reports no error.

Disabled entries (no sequence, no selection) still fail before any
dialog opens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:27:36 +02:00
txiki739andClaude Opus 5.5 ae09ba904c #214: install script for the Linux tarball
The tarball now ships install.sh. It installs filmcraft, filmcraft-cli,
the desktop entry, icons, MIME type, metainfo and docs under ~/.local
without sudo; --prefix takes any absolute path (sudo ./install.sh
--prefix /usr/local for all users). Running it again updates.

The desktop entry's Exec/TryExec point at the installed binary's
absolute path, escaped per the Desktop Entry spec, so the menu entry
works without ~/.local/bin on PATH. The script checks the bundle is
complete before writing, refuses /, prefixes inside the extracted
bundle and characters launchers cannot express (=, %, control
characters), and treats the cache refresh tools as optional.

packaging/linux/test_install.py covers install and update, launching
through gio, bad arguments, an incomplete bundle, --help and a real
package.sh tarball build; Packaging lint runs it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:54:20 +02:00
Davi AndreiandClaude Sonnet 5.5 b485e2c180 Thumbnails decode with small decoders that are not kept
Right after importing many 4K clips the Project panel and the timeline ask
every clip for a thumbnail. Up to six frame workers decoded them at once,
each with a frame-threaded software H.264 decoder (16 threads and 18
pictures in flight here, about 0.5 GB at 4K), and the decoders stayed: 42
4K clips with hardware decoding Off took the process past 6 GB during import.

Thumbnail jobs (item frames at the Media Browser / thumbnail priorities) now
run under filmcraft_media::cancel::with_background: H.264 decoders made for
them use 2 threads (thread_limited), parallelism comes from the workers
decoding different files, and the decoders of caches only thumbnails used
are released after each thumbnail (the clip served last keeps its own). A
monitor or playback request on such a cache replaces the small decoder with
a full one. Sequence frames are never background (the scopes ask for the
program frame at 40 while playing).

42 4K clips (Windows), peak private bytes: hardware Off 6.1-6.8 GB (killed)
-> 3.1-3.2 GB; hardware Auto 2.7-2.8 GB (#381) -> 2.3-2.4 GB. Thumbnails of
12 / 42 clips ready in 0.6 / 2.2 s instead of 2.2 s / not finishing.
Example crates/ui-egui/examples/thumb_burst.rs reproduces the burst.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 04:32:43 -03:00
Davi AndreiandClaude Sonnet 5.5 0c64e41b53 Decode caches: navigation bursts no longer keep every clip's decoder and frames (#55)
A cache asked for a frame within IDLE (3 s) counted as in use, and the caps
of #41 (4 live decoders, 1 GiB of frames) only bound caches idle for longer,
so moving the playhead across 42 4K clips kept 42 hardware decoders (6.5 GB
of VRAM) and up to 3.9 GB of frames. Now only caches touched within RECENT
(500 ms) are protected: caches idle 0.5-3 s give up decoders beyond
HARD_DECODERS (8) and frames beyond the shared budget, least recently used
first; the 3 s rule down to 4 decoders is unchanged. store() refreshes
last_used.

Measured with a 42-clip 4K reproduction (Windows, Media Foundation): peak
private bytes 6.6 -> 4.0 GB, frame cache 3.9 -> 1.1 GB, live decoders
42 -> 8-14, GPU memory 6.5 -> 4.3-4.6 GB, 0 dropped frames before and after.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 03:13:42 -03:00
WavJaby 6893cb4f03 perf(web): feed native YUV frames to the shared compositor 2026-10-09 13:47:06 +08:00
sanyamsingh001andCodex 892003f3d4 GPU.EFFECT_SOURCE: preserve exact texel centers for effect inputs
Draw effect sources from working pixel positions and integer decimation.
Cover identity, alpha inversion and box minification without changing tolerances.

Co-authored-by: Codex <codex@openai.com>
2026-10-09 15:44:39 +10:00
Xus BadiaandClaude Opus 5.5 10b71eec85 macOS DMG: branded window background and icon layout
The DMG opened as a plain Finder window. It now shows the app icon's figure on
the FilmCraft colour, with FilmCraft.app and the Applications link side by side.
Same approach as PhotoCraft's and VectorCraft's (storytold/photocraft#1201,
storytold/vectorcraft#493).

- packaging/macos/dmg/: background.svg (source; text outlined, no paper grain,
  the figure an <image> of assets/app-icon), background.tiff (1x + 2x HiDPI,
  16-colour palette, Deflate, sRGB; ~106 KB)
  and dmg-layout.DS_Store (Finder view settings: window, icon size 128,
  positions, background).
- generate.py writes the TIFF and the layout on any OS, byte-for-byte
  reproducible, with nothing from the machine that ran it (no paths, users,
  disk names or volume UUIDs).
- package.sh copies them into the staging folder; the image is still built with
  hdiutil makehybrid. Nothing extra runs in the signing job.
- The volume name drops the version: the layout finds the background through
  an alias that includes the volume name. The DMG file name keeps the version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 07:43:43 +02:00
storybored379andOpenAI Codex 7264594494 mixer: report only the audio gains actually applied
Fixes #343.

Co-Authored-By: OpenAI Codex <noreply@openai.com>
2026-10-08 21:35:58 -07:00
Davi AndreiandClaude Sonnet 5.5 9510e26584 Export: render the next batch while the current one is encoded
A step now renders batch k+1 on the rayon pool while the calling thread
encodes, muxes and mixes the audio of batch k. The file is byte-identical
with or without overlap. A prefetch that hits pending media is dropped and
rendered again, cancel frees it, a render panic becomes an error, and wasm
never overlaps. perf.stats export.stages gains waitMs.

Memory: up to 2 x batch x frame bytes of finished frames are in flight, so
the batch is capped to a 512 MiB budget (1080p keeps 16, 4K SDR 8, 4K HDR 2
frames). Measured peak RSS vs before: 1080p +67..+158 MB; 4K SDR -535..-927
MB; 4K HDR about -2.6 GB (5.2 -> 2.6 GB). Wall time 1.06-1.16x faster.

Known pre-existing web issue, untouched: AudioOut::pull advances before the
pending check, so a retried audio group is lost.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 01:26:41 -03:00
Davi AndreiandClaude Opus 5.5 1996f319c3 Export: stage wall times in perf.stats (export.stages)
filmcraft_export::Stage, stage_times(), note_stage() and timed(): the wall
time of setup, loudness, render, encode (and its RGB->YUV convert part),
audio, mux and finish, summed per process like export.hardware. Converters
in the H.264 / ProRes / NVENC / VideoToolbox encoders are timed as Convert.
Two clock reads per stage per batch; the exported file does not change
(determinism tests). Test: an export grows every stage counter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 01:26:09 -03:00
storybored379andOpenAI Codex 3971b2666f graphics: keep duration separate from timeline placement
Fixes #326.

Co-Authored-By: OpenAI Codex <noreply@openai.com>
2026-10-08 21:08:47 -07:00
sanyamsingh001andCodex 0e0f0485da LINUX.VALIDATION: honor preset shortcuts and retain MPEG oracle frames
Resolve inherited physical key collisions when loading built-in presets,
while preserving established explicit winners and deliberate custom conflicts.
Read every ffprobe display-frame timestamp and register the requested credit.
Verify exact frame-plane reuse without depending on concurrent global metrics.

Co-authored-by: Codex <codex@openai.com>
2026-10-09 14:06:37 +10:00
storybored379andClaude Opus 5.5 78053191ce MCP: cancelling a movie export keeps numbered files beside it
remove_partial treated any `<stem><digits>.<ext>` file as a still of an image sequence, so
cancelling an export to `clip.mp4` also deleted a `clip1.mp4` written in the same second (or
during the export). Only PNG/TIFF/BMP sequence exports write numbered stills, and their frame
numbers have at least three digits (`image_sequence_path`); match only those.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 21:05:31 -07:00
storybored379andOpenAI Codex ea79751a5c platform: enable existing NVENC H.264 export on Linux
Runtime-load the NVIDIA encode and CUDA driver libraries, retaining the primary context through session destruction. Preserve the Windows Direct3D path and default-off selection, and test driver rejection and deterministic software fallback.

Co-Authored-By: OpenAI Codex <noreply@openai.com>
2026-10-08 20:40:01 -07:00
Cathy Shapiro ac145dbaa8 M3.12: Dismiss Export Frame when clicking outside
Cancel the export draft when clicking outside the dialog, including top menus and mode tabs, while allowing the background control to receive the click. Keep the dialog's own format and depth dropdowns interactive.

Cover dismissal for both monitors, mode switching, dropdown selection, and background monitor clicks in a UI regression. No frame is exported when clicking away.

Assisted-by: Codex
2026-10-08 21:57:30 -05:00
JordyUSAandClaude Opus 5.5 9e8ad1d6a2 #341: CLI export message names the file written, not the requested path
When the format changes the extension (APV into `out.mp4` writes
`out.mp4.mov`), `filmcraft-cli export` said "exported out.mp4" on stderr
while stdout JSON and the filesystem had `out.mp4.mov`. The message now
takes the path(s) from the export result: `path` of a direct export, each
item's `path` with --queue; the requested path only when the result names
none. Unit test for the helper; end-to-end test (direct and queued APV
export) that failed before the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:53:28 +07:00
Davi AndreiandClaude Sonnet 5.5 dbae11b843 Windows: NVENC H.265 Main 10 HDR (PQ / HLG) export
HDR sequences export as real 10-bit HEVC Main 10 through NVENC where the GPU
reports 10-bit HEVC: float BT.2020 R'G'B' -> rgbf_to_yuv420_10 -> P010 input;
VUI BT.2020 + PQ/HLG; HDR10 mastering-display / content-light as SEI on every
IDR (PQ); colr/mdcv/clli in the sample entry. New register_hdr_probe /
hdr_available in the export crate; macOS and GPUs without 10-bit keep
tone-mapping HDR sequences to 8-bit SDR HEVC. ABI asserts extended and
re-verified against nvEncodeAPI.h 12.1 (0 drift). 8-bit SDR HEVC unchanged;
the two #328 tests that asserted HDR is declined now expect Main 10.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 23:38:14 -03:00
Idiotic f146d7ba9f feat(media,engine): implement multi-stream container audio support and schema v13
- Update MediaInfo to store all discovered audio streams in audio_streams

- Retain independent stream metadata and decoder state for MP4/MOV and Matroska/WebM

- Update audio_placement_specs to allocate successive tracks via ensure_audio_tracks

- Isolate clip rendering to read assigned audio stream rather than replaying stream 0

- Bump project schema to v13 with backwards compatibility for legacy v12 projects

Resolves #312

Fixes #200

Fixes #315
2026-10-08 22:36:45 -04:00
ippsecandClaude Opus 5.5 c43d1090c1 contributors: credit IppSec by GitHub profile name
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 22:30:49 -04:00
ippsecandClaude Opus 5.5 d51945c807 #236: audio in MKV and OBS-remuxed MP4 plays without crackling
Matroska timestamps are whole milliseconds and a 1024-sample AAC frame at 48 kHz lasts
21.33 ms, so frames are stamped at 0, 1008, 2064, 3072… samples; OBS's remux to MP4 carries
that rounding into the sample durations (1008, 1008, 1056…). The MKV and MP4 readers put each
decoded packet at its own timestamp, overwriting the end of one packet and leaving a hole of
silence after another: a click every few packets, crackling throughout playback and export.

Packets of codecs with a fixed frame length (AAC, MPEG audio, AC-3) now run on from one
another and resynchronise to the timestamp only across gaps of more than half a packet, the
rule the MPEG-TS/PS reader already uses. Laced Matroska frames without a duration (which
repeat the block's timestamp) follow on too, and MKV Opus uses the same helper. MP4 audio
packet starts are kept in samples instead of track timescale units.

Regression tests mux our own AAC into MKV (millisecond stamps, plain and laced, a real gap)
and into an MP4 with OBS-remux durations: block-by-block playback and seeks must equal the
packets decoded back to back (all fail before the fix). An ffmpeg oracle test plays
ffmpeg-made MKVs with AAC, MP3, MP2 and AC-3 and requires them as smooth as ffmpeg's decode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 22:29:23 -04:00
swillunderscoreandClaude Opus 5.5 57d8990765 contributors: display name for @swillunderscore
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 21:19:21 -05:00
Cathy Shapiro 6c0e3ecc39 M3.12: Match Export Frame timecode to its monitor
Display the captured frame with the same frame rate and drop-frame timecode format as Source or Program, instead of decimal seconds. Keep the captured timecode fixed while the export dialog is open.

Add a UI regression comparing both monitor labels at standard and NTSC rates, with Program drop-frame formatting and playhead movement after capture.

Assisted-by: Codex
2026-10-08 21:11:44 -05:00
JordyUSAandClaude Opus 5.5 e741267125 HW5: Linux VA-API HEVC hardware decoding, Main and Main 10 (#30)
HEVC through VA-API the same way as H.264: the software HEVC decoder's
parsers and DPB (now generic over the picture type in filmcraft-hevc:
POC, reference picture sets, reference lists, output order, RASL
handling) drive the stateless hardware. Main, Main 10 and Main Still
Picture, 4:2:0 8/10-bit (P010 read-back); scaling lists, tiles, WPP,
dependent slices, weighted prediction. Missing references and range
extensions go to the software decoder.

Tests: hardware parity on HEVC Main / Main 10 incl. open GOP, scaling
lists, slices+WPP, weighted prediction, tskip+AMP, 1080p/2160p (streamed
comparison to stay within memory); HEVC front end against a recording
stand-in on every OS; ABI checks extended to va_dec_hevc.h. Benchmark on
Iris Xe: 4K HEVC 203 -> 10 ms CPU per frame, 4K Main 10 212 -> 18 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:00:30 +07:00
swillunderscoreandClaude Opus 5.5 87bb39f5f5 #172: Sequence Settings… opens Premiere's dialog (General, Color Management, VR Properties)
The menu item ran sequence.settings with no parameters and changed nothing. It now opens the
dialog: Timebase, Frame Size with "Scale motion effects proportionally" (on by default), the
timecode Display Format (drop-frame / non-drop-frame for 29.97, 59.94, 119.88), Channel Format,
Sample Rate, Maximum Render Quality, and the Color Management tab, applied in one undo step; OK
with nothing changed adds no undo step. Settings FilmCraft has no equivalent for yet (Editing
Mode, Pixel Aspect Ratio, Fields, Frames / Feet + Frames, Number of Channels, audio Display
Format, Video Previews, Maximum Bit Depth, VR Properties) are shown greyed out with their value
and a "Not supported in FilmCraft yet" hint; Composite in Linear Color is shown on (the
compositor is always linear). Every control has an automation id.

sequence_ui.rs: two tests driving the dialog over the control channel, and
FILMCRAFT_UI_SNAPSHOT_DIR support like the other UI tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 20:58:37 -05:00
swillunderscoreandClaude Opus 5.5 1f6f7d4c7d #172: sequence.settings scales Motion with the frame size and takes timecode, render quality and colour
- filmcraft_edit::frame_size::scale_motion: Sequence Settings' "Scale motion effects
  proportionally when changing frame size". Every video clip's Motion Position maps through
  p' = c1 + k(p - c0) and Scale / Scale Width multiply by k = min(W1/W0, H1/H0), static values
  and keyframes: a plain proportional scale for the same aspect, and on an aspect change the old
  frame fits inside the new one, centred. Automatic positions stay automatic, Scale to Frame
  Size clips keep their Scale (the renderer refits them), Anchor Point and Rotation stay.
- sequence.settings: new dropFrame, maxRenderQuality, workingSpace, wideGamut, autoToneMap and
  scaleMotion (default false, so existing callers are unchanged) params, in the same undo step;
  returns {"scaledClips": n}. Drop-frame on a rate without it is refused atomically.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 20:58:37 -05:00
jerrykimo 34cf6005d0 ui: Render Chinese document names with font fallbacks 2026-10-08 20:50:09 -05:00
Cathy Shapiro 9f0ed51b49 M3.12: Export frames from Source and Program monitors
Wire both camera buttons to captured-monitor frame export with original media at full resolution. Add a movable settings dialog with format/depth selection, a visible destination folder, Browse, overwrite confirmation, and optional project import.

Reuse existing 8-bit still encoding; add JPEG and direct float-to-16-bit PNG/TIFF encoding. Cover monitor routing, captured frame time, proxy bypass, hostile parameters, depth, import, cancellation, overwrite, and fractional header movement with synthetic regressions.

Assisted-by: Codex
2026-10-08 20:46:12 -05:00
JordyUSAandClaude Opus 5.5 3496e553e7 HW5: Linux VA-API H.264 hardware decoding (#30)
Stateless VA-API decoding with FilmCraft's own H.264 host side: the
software decoder's SPS/PPS/slice-header parsers and its DPB (now generic
over the picture type in filmcraft-h264) drive the GPU, so reference
marking, reference lists and output order are shared and the pictures are
bit-exact with the software decoder. libva is loaded at run time; no libva
or driver means software decoding as before. frame_num gaps (open-GOP
seeks) get the software decoder's stand-in frames. HybridDecoder keeps the
transparent software fallback.

Tests: hardware parity on 7 H.264 streams incl. 1080p/2160p, seeks,
flushes, forced failures, fuzzing, concurrency; front end against a
recording stand-in on every OS; ABI checks against va.h. Docs: platform
README, ADR 0001 addendum, architecture, performance (H.264 2160p CPU per
frame 417 -> 11 ms on Iris Xe), roadmap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 08:35:57 +07:00
Davi AndreiandClaude Opus 5.5 a8f91e3ffe M2: H.265 stays in the Main tier (pick the level)
Left to choose, NVENC answered a peak bitrate above its level's Main tier
limit with the High tier: 1080p30 at 12/18 Mbit/s was written as level 4 High
tier, 2160p30/60 at 40/60 Mbit/s as 5.0/5.1 High tier, which many hardware
decoders refuse. The level is now the lowest whose Main tier limits hold the
picture size, sample rate and peak bitrate (H.265 Table A.8), e.g. 1080p30 at
18 Mbit/s is level 4.1 Main tier. Unit test of the table; hardware test that
the hvcC says Main tier at the expected level.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 22:23:25 -03:00
Idiotic 78ed3edd84 feat(engine): dynamically expand audio tracks on multi-clip placement 2026-10-08 21:11:12 -04:00
rossaaiandClaude Opus 5.5 0506a01f29 UI: the status bar fills with the header_bg token
The status bar was a fixed #1c1c1c, dark on the Light and Medium themes
and on a theme an embedding app sets through FilmcraftApp::tokens. It now
takes the header bar's colour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 19:57:43 -05:00
rossaaiandClaude Opus 5.5 1e9f37864b UI: UiState::show_header and show_status_bar, on by default
An app that embeds FilmCraft can hide the header bar and the status bar
and give the panels the whole window. Saved UI state without the fields
keeps both shown. With the status bar hidden, a finished preview render
still plays: the job watching moved out of job_status into watch_jobs,
which runs every frame.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 19:57:43 -05:00
rossaaiandClaude Opus 5.5 83501964f6 Web: public set_context for a host that runs its own WebApp
post and repaint wake the egui context that start stores in a private
thread-local. An app that embeds FilmCraft and builds WebApp in its own
eframe runner had no way to set it, so posted actions and window.filmcraft
requests waited for the next input event. start now calls the same
set_context.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 19:57:43 -05:00
Cathy Shapiro 8a7275c821 M3.12: Source playback and monitor range editing
Play source video and audio, drag marked ranges as video, audio, or a linked pair, and route Source commands to clip marks and markers. Add shared draggable In/Out handles and range movement, preserving scrubber priority and explicit-only Program marks.

Include original code-drawn controls, documentation, attribution, and regression coverage for playback, placement, routing, undo, cancellation, and hostile parameters.

Assisted-by: Codex
2026-10-08 18:58:46 -05:00
Cathy Shapiro 26d972ce01 WINCI: Merge current upstream 0.4.0 release and packaging
Preserve the Windows validation fixes and original #282 commit while bringing the contribution up to date with upstream main.

Assisted-by: Codex
2026-10-08 18:58:11 -05:00
DWAA1660andClaude Opus 5.5 8d6461705e Fix #179: default Windows startup to DirectX 12
eframe's default wgpu instance enables PRIMARY | GL on Windows. Creating
the OpenGL backend can crash inside AMD's atio6axx.dll before the window
appears, so the app flashes in Task Manager and exits. When WGPU_BACKEND
is not set, the Windows instance now enables only DirectX 12; an explicit
WGPU_BACKEND is still honoured. macOS, Linux and web are unchanged.

Same fix as storytold/designcraft#168.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LyVbT6Vv4BwWZ4hY8973qD
2026-10-08 17:52:30 -06:00
Cathy Shapiro 0b34c076ce WINCI: Windows path handling and portable validation
Normalize explicit relink prefixes, compare native paths in portable tests, close fixture sessions before directory moves, and preserve exact GPU effect-source pixel centres. Keep the hardware-dependent mixer benchmark opt-in with its 4x target unchanged.

Shortcut portability changes reused from storytold/filmcraft#37 (bd944ee and f2b36fe). The existing export fix from #282 remains in its original parent commit.

Assisted-by: Codex
2026-10-08 18:28:32 -05:00
mjohnson7cd 7048a7905a fix: frame stepping broken for non-standard frame rates (#301)
tick_of() used floor division (div_euclid) which truncated the frame
duration for non-standard rates like 37.516 fps (9379/250) where the
frame duration is 6770871094.999 ticks, not an integer. This caused
frame_at(tick_of(f)) to return f-1 instead of f, making step forward
stall at frame 0 and step back jump backward by 2+ frames.

Changed tick_of() to use ceiling division so the round-trip is exact
for all integer frame indices. Ceiling is correct because the frame
duration is always > 1 tick, so floor(ceil(f*D) / D) == f for every
integer f. frame_duration() remains floor division (unchanged) since
many callers do end - frame_duration() to get the previous frame.

All 11 standard broadcast rates are unaffected since their frame
durations are already exact integers (ceil == floor there).

Added regression tests:
- exotic_frame_rate_roundtrip: round-trip over -20k..20k for 5
  non-standard rates, snap invariant, step-back via frame_duration,
  forward/backward stepping simulation
- exotic_frame_rate_durations: frame_duration is floor for exotic,
  exact for standard
- exotic_frame_tick_roundtrip proptest: arbitrary FrameRate::new(num,
  den) round-trip

Thanks to @justjohn12345 for catching that round (not ceil) still
fails for ~50% of frames starting at f=938.
2026-10-08 18:37:41 -04:00
Oldrich333andClaude Opus 5.5 44248c060f CLI: a closed stdout no longer panics filmcraft-cli, and the work still finishes (#285)
`filmcraft-cli commands | head -c 10` panicked with "failed printing to stdout: Broken pipe
(os error 32)". Output now goes through `outln!` / `out!`. When stdout fails, later output is
dropped but the program does not stop: the rest of `exec`, `run` and `import` (later script
lines, `--save`, `--save-as`) still runs and the exit status still reports a failing command.
Exiting at the first failed print would report success while skipping the save, e.g.
`filmcraft-cli run edits.jsonl --project p.fcproj --save | head -1`. A closed pipe is not an
error; any other stdout write error is reported and makes the exit status 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 17:09:53 -04:00
WavJaby 4d74065aa6 fix(gpu): separate YUV code ranges from texture scaling
- Normalize full-range 8-bit and limited-range high-depth YUV
- Gate code ranges and CPU/GPU parity for inline/prepared uploads
- Existing main GPU effects/frame-pool failures reproduced
- Full workspace and app lifecycle checks remain unrun
2026-10-09 03:57:01 +08:00
Davi AndreiandClaude Opus 5.5 29b454ac17 M2: review fixes for NVENC H.265
- register() asks the HEVC probe on a thread of its own (the first probe is
  about half a second: D3D11 device + NVENC session), so the first draw of
  Export's format list does not wait for it; tests/nvenc_hevc_warm.rs.
- A machine with no HEVC encoder is not a declined hardware attempt: the
  factory returns None without counting `declined`.
- HEVC tests skip only when the probe says there is no HEVC encoder; on a
  machine that has one, a configuration NVENC refuses fails the test.
- Wording: NVENC is FilmCraft's only HEVC encoder on Windows (Windows itself
  ships others).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 16:48:43 -03:00
Davi AndreiandClaude Sonnet 5.5 deebe2e5b2 Windows: NVENC H.265 (HEVC) Main hardware encoding for Export
One NVENC session, ring and NV12 path now serve H.264 and HEVC (Codec enum,
Profile::HevcMain). HEVC Main 8-bit 4:2:0 SDR BT.709, MP4/MOV hvc1, parameter
sets only in an hvcC built from the SPS the encoder wrote. The H.265 format is
available through a cached NVENC probe; choosing it is the opt-in, declines are
counted errors. ABI asserts regenerated from nvEncodeAPI.h 12.1 (no drift) and
extended for HEVC; tests, docs and roadmap updated.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 16:28:50 -03:00
WavJaby 944d581cde perf(gpu): composite Push transitions on the shared GPU path 2026-10-09 01:41:43 +08:00
Pathos0925andClaude Fable 5.1 24cde5e9fe #296: Auto-Match measures a mono clip as one channel
Loudness Auto-Match metered every clip as the stereo pair clip_signal
plays, so a mono source (or a mono pick from Modify > Audio Channels /
Breakout to Mono), copied onto both sides, read 3.01 LU louder than the
file and got 3 dB too little gain. BS.1770 and ffmpeg's ebur128 measure
a mono programme as one channel; now so does clip_loudness, through a new
render::audio::clip_is_mono. Stereo and wider sources are unchanged, as
are the export loudness pass and the Meters panel (they measure the real
output mix).

Regression test: the same speech as a 1-channel WAV and as dual-mono
stereo measure 3.01 LU apart, the mono clip agrees with a one-channel
meter over the file within 0.1 LU, and a mono pick counts as mono.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 12:09:15 -05:00
justjohn12345andClaude Opus 5.5 a6b21fe7cb Keyframes: Properties gets Premiere's navigator, Effect Controls a time ruler
Properties panel: the diamond on a row was the stopwatch (effects.toggleAnimation),
so clicking it on a keyframe dropped every keyframe and kept the value at the
playhead. It is now Add/Remove Keyframe (effects.addKeyframe), as in Premiere:
it turns animation on with a keyframe at the playhead, adds one there, or removes
the one there, and the value falls back to what the remaining keyframes give in
Properties, Effect Controls and the Program monitor. An animated row shows the
navigator (previous / diamond / next); the diamond is filled only while the
playhead is on a keyframe. Crop's diamonds work too (the effect is applied first).

Effect Controls: the keyframe lane has a time ruler (ticks and sequence timecode
over the clip's stretch of the sequence) with the playhead's handle on it; click
or drag the ruler to move the playhead. The playhead is drawn only while it is on
the clip. Both panels share one navigator widget, with automation ids
<panel>.<effect>.<param>.addKeyframe / .prevKeyframe / .nextKeyframe, plus
effectControls.ruler and effectControls.playhead.

Tests: engine add/remove at the playhead with undo, redo and bad parameters;
headless UI tests of the navigator, Crop and the ruler; ruler labels and limits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:06:25 -04:00
justjohn12345andClaude Opus 5.5 d09e95663c Program monitor: the picture follows a value that is dragged
While a value was dragged in Effect Controls or Properties, the Program monitor
kept the picture from before the drag and only caught up when the mouse rested.

Two causes:

- Every step of a drag is a new project revision, so the frame the monitor asks
  for is never ready on the refresh that asks for it, and the stand-in lookup
  only knew frames of the same revision. The monitor now remembers the frames it
  asked for on its latest paused refreshes and, when neither the frame due nor a
  cached frame just before it is ready, shows the newest of those that is. A
  backward or fast scrub had the same gap and gets the same stand-in. Playback
  is unchanged.
- A frame still rendering was cancelled when the same frame was asked for at a
  newer revision. With frames slower than a refresh (CPU effects) each one was
  thrown away for the next. Such a job now finishes; a request for a different
  frame of the view (scrubbing) still cancels it.

perf.stats gains `monitor`: the frame and revision of the picture the Program
monitor shows, against the playhead's frame and the project's revision.

Tests (headless, perf_ui): a parameter changed on every UI pass and a backward
scrub leave the picture at most one step behind; a frame being rendered survives
a request for the same frame at a newer revision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:06:13 -04:00
MarekSpodBiedry 502c9a33a7 Issue #139: portable.txt next to the exe keeps the data directory beside it 2026-10-08 18:58:41 +02:00
Dario CarneluttiandClaude Sonnet 5.5 389cfefe92 platform: ask for the hardware HEVC encoder in the background at startup
`Format::Hevc` is available only where a hardware HEVC session can be created, and
`hardware_encode::hevc_available` finds out by creating a small one on the first question. That
question comes from the first draw of Export mode's format list, on the UI thread, and the session
takes about 0.1 s (88-100 ms in seven of eight fresh processes on an Apple M1, 222 ms in the first):
the app stalls for that long the first time Export is opened.

`register()` now starts the probe on a thread of its own (`warm_hevc_probe`, once however many times
`register` is called). Whoever asks while it runs waits for the same `OnceLock` answer, no longer than
asking first; a thread that cannot start, or a probe that panics (caught), leaves the question to the
first caller as before. `hevc_probed()` says whether the answer exists, for diagnostics and tests.

Test: `tests/hevc_probe.rs` registers, never calls `hevc_available`, and waits for `hevc_probed()` (it
fails after 60 s without the call in `register`).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 10:17:58 -06:00
Dario CarneluttiandClaude Sonnet 5.5 10879f385e export: free the frame pool when an export ends
The export pipeline hands the float image of every frame it converts back to
`filmcraft_frame::pool` (#184), which keeps up to 320 MiB of them, and the decoders' plane shelves
hold up to 192 MiB of each type. An export's images are the size of its frames, which nothing else
asks for, so a finished export left them idle until the next one.

`pool::trim()` frees every idle buffer. `Pipeline` calls it when it is dropped, one place for every
way an export ends (done, failed, cancelled, dropped half way by a host that steps it, as the web
build does) and for every caller of `export` / `Exporter` (previews, proxies and AAF/OMF mixdowns
share the pipeline). A job still rendering at that moment allocates a few images again: the pool is
only a cache.

Measured with a synthetic 3 s 1080p24 matte exported as PNG frames three times in one process
(Apple M1, `footprint`, 10 s after the last export): 575 MB without the trim, the pool holding
332 MB idle, and 258 MB with it. macOS returns the freed pages over a few seconds, not at once.

Tests: `crates/frame/tests/pool_trim.rs` (every shelf emptied, the pool works afterwards) and
`crates/export/tests/pool_trim.rs` (an image sequence, a finished and a cancelled video, an export
that could not start and a stepped export dropped half way each leave nothing idle; without the
trim they fail).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 10:13:07 -06:00
midasdfandCodex d77728008f Keep grapheme clusters together in vertical text
Co-Authored-By: Codex <noreply@openai.com>
2026-10-09 00:40:01 +09:00
Tom 2b843d3d81 Windows: export paths expand ~, accept trailing \ and verbatim paths
- expand_home uses std_home_dir (USERPROFILE on Windows) instead of
  HOME only; the Export panel reuses the engine function.
- std_home_dir prefers USERPROFILE on Windows, where Git Bash sets
  HOME to a Unix-style path.
- On Windows, / in an export path becomes \: after a verbatim
  prefix (what canonicalize returns) / is not a separator, giving
  os error 123.
- One is_folder check for output_path and export.queue.add, so a
  trailing \ no longer numbers several queued sequences.
2026-10-08 15:57:33 +01:00
midasdfandCodex d67d225da5 M10: use Japanese vertical forms and origins for titles
Co-Authored-By: Codex <noreply@openai.com>
2026-10-08 23:20:38 +09:00
Brandon ThomasandClaude Opus 5.5 5231852443 Bump version to 0.4.0 (from 0.3.0) for release
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:01:39 -04:00
207b179428 README Downloads: where to get it first, then builds with Windows before macOS (#271)
Co-authored-by: Brandon Thomas <shared2@storyteller.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 09:50:42 -04:00
Tom 0bf83a27c0 Windows: platform modifier names in status-bar and mask hints
The Selection and Zoom status-bar hints and the mask-vertex tooltip
named Mac keys (Opt, Cmd) on every platform. Show Alt/Ctrl on
Windows and Linux, matching the keys the code actually reads
(egui `alt` / `command`). macOS text is unchanged.
2026-10-08 14:47:40 +01:00
86a531ad22 Release: FreeBSD, Flatpak (x86_64 + aarch64) and AppImage zsync; README Downloads (#267)
- release.yml: new `flatpak` jobs (x86_64 on ubuntu-24.04, aarch64 on ubuntu-24.04-arm) that
  repackage the Linux tarball as filmcraft-<v>-linux-<arch>.flatpak, and a `freebsd` job that
  builds filmcraft-<v>-freebsd-x86_64.tar.gz in a FreeBSD 14.3 VM; both feed the draft release.
- AppImages embed gh-releases-zsync update information and ship a .zsync beside them; the
  Linux job checks both.
- Jobs that sign nothing (version, Linux, Flatpak, FreeBSD, web) no longer use the `release`
  environment, so a dispatch on a branch dry-runs them; the signing jobs and the draft-release
  job stay in the environment, so nothing is published from a branch.
- packaging/linux/flatpak-bundle.sh + bundle manifest (same runtime and finish-args as the
  Flathub manifest; packaging-lint checks they agree), packaging/freebsd/package.sh (with a
  --dry-run that packaging-lint runs), and a nightly FreeBSD build workflow.
- README: Downloads section listing every build; docs/releasing.md updated.

Co-authored-by: Brandon Thomas <shared2@storyteller.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 09:41:42 -04:00
wstockandClaude Opus 5.5 06abca1bb9 shortcuts: Delete clears the selected timeline clips (#243)
Windows and Linux keyboards' Delete key is the forward-delete key. In
FilmCraft Default it was bound only to Project ▸ Clear (application-
wide), so with the Timeline focused it ran project.delete, which does
nothing without a Project-panel selection: the selected clips stayed.
Backspace (the Mac "delete" key) already cleared them.

Add a Timeline-panel binding of Delete to edit.clear, as Premiere has
it; in the Project panel and application-wide Delete still runs
Project ▸ Clear.

Tests: shortcuts_tests::delete_clears_timeline_clips_and_project_items
(Delete resolves to edit.clear with the Timeline focused and to
project.delete otherwise, macOS and Windows) and
keyboard_ui::delete_key_clears_the_selected_timeline_clip (headless app:
D selects a clip, Delete removes it, undo restores it). Both failed
before the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:14:52 +00:00
Brandon ThomasandClaude Opus 5.5 08735f3103 Bump version to 0.3.0 (from 0.2.1) for release
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 08:12:49 -04:00
wstockandClaude Fable 5.1 32764392f1 control: Ctrl+ chords through ui.key fire shortcuts off macOS (#245)
Bindings are written Cmd+… and parse to `command`; egui's matching then
requires `command`. A physical Ctrl press off macOS carries it
(egui-winit sets command = ctrl there), but ui.key built its event from
parse_shortcut, which mapped "Ctrl" to `ctrl` alone, so an injected
Ctrl+Z matched nothing while Cmd+Z worked. ui.click's `ctrl` modifier
had the same gap.

Off macOS "Ctrl" and "Cmd" are one key (the engine's Chord::effective
already says so), so parse_shortcut maps both to `command` there, and
the control channel synthesises presses the way the OS reports them:
a press of either carries both flags. macOS is unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 11:08:19 +00:00
wstockandClaude Fable 5.1 acf56bcd27 engine: project.delete takes explicit bin ids without a Project-panel selection (#244)
A caller that names its targets does not need a selection: enabled_for
re-runs the enablement check with the named ids standing in for it. But
named_items only recognised project items, and project.delete's `items`
take bin ids too (the Project panel's selection holds both), so
{"items":[<bin ids>]} with nothing selected was still "not available
right now: select an item in the Project panel".

Count a named bin as a target, except the project's own top bin, which
no command acts on.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 10:54:42 +00:00
WavJaby 92c6aa2b63 chore(lints): preserve behavior in workspace clippy cleanup
- Move conditional no-op match arms into guards

- Preserve renderer predicate order with De Morgan's law

- Strict clippy, fmt, layers, assets and WASM pass on Rust 1.95

- Clean engine runs match main: 379 pass, 10 fail, 3 ignored

- Full CI is not green; upstream baseline failures remain
2026-10-08 18:22:27 +08:00
Gert Pellin 2d6c4d3a3d Desktop: write log records to standard error and a rotating log file
The desktop app installed no logger, so every log::warn!/log::error! in
the workspace was discarded: GPU compositor fallbacks, audio output
problems, a panicked frame job or audio mixer, container warnings. A
launch from a desktop menu has no terminal either, so there was nothing
to attach to a bug report.

apps/filmcraft/src/logging.rs (no new dependency):
- standard error plus <data dir>/Logs/filmcraft.log, next to the crash
  logs (--data-dir / FILMCRAFT_DATA_DIR respected, Help > Reveal Log Files
  opens the folder); each launch rotates the previous log to
  filmcraft.1.log and .2.log, so the log of a run that crashed survives
  the next start; the file is capped at 16 MiB
- records logged before the data directory is known are buffered
  (bounded) and written once the file is attached
- default levels: info for filmcraft*, warn for everything else;
  RUST_LOG replaces them with env_logger-style directives
- never panics: an unusable directory or a failed write leaves standard
  error as the only sink; a poisoned lock is tolerated
- the realtime audio callbacks do not log; a record holds the lock for
  one write of an already formatted line

The logger is installed first in main (before the panic hook) and the
file is opened after argument parsing (--help, --version and usage
errors leave no file). The start-up eprintln!s in main (auto-save
unavailable, open/recovery errors, no window) now go through the logger.
Documented in docs/contributing.md (environment variables, Logs).
2026-10-08 11:37:02 +02:00
Gert Pellin d018fb9abb Contributors: credit switch87 as Gert Pellin 2026-10-08 11:27:34 +02:00
Gert Pellin 7a042cd72f README: install on Gentoo from the ::snakebyte overlay 2026-10-08 11:25:54 +02:00
MrCloudy2andClaude Opus 5.5 bee690f050 #220: Unlink leaves only the right-clicked clip selected
Unlinking from the timeline clip menu cleared the link, but the picture and
its sound stayed selected together, so dragging the picture still moved the
sound and Unlink looked like it did nothing. The clip that was right-clicked
is now the only one selected after Unlink, and can be dragged away straight
away. Link leaves the selection as it was.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:53:54 +02:00
wstockandClaude Fable 5.1 6068bfc589 export: keep alpha in PNG/TIFF sequences with Include Alpha Channel (#160)
The export pipeline flattened every frame over black
(`over_black_rgba8`), so a PNG sequence of a sequence with transparency
came out on a black background, while Export Frame already wrote straight
RGBA. `ExportSettings.alpha` (serde default false, Premiere's default)
makes the pipeline emit straight sRGB RGBA for PNG and TIFF sequences and
`encode_still` keep the channel in TIFF; BMP and video formats ignore it.
Exposed as the `alpha` flat parameter of file.exportMedia and an
"Include Alpha Channel" checkbox in Export mode's Video section, shown
for PNG and TIFF only.

Fixes #160

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 08:31:37 +00:00
WavJaby aa66dd9c8d fix(import): use the open Project bin as destination
Resolve the shown Project bin for file-dialog and dropped imports. Capture the browser destination before asynchronous media reads so navigation cannot redirect an in-flight batch.
2026-10-08 16:27:48 +08:00
WavJaby 3e39952e0b fix(web): preserve distinct same-name media imports
Allocate an unused virtual path for every browser import so equal-size media cannot replace existing sources. Add an OPFS recovery regression for issue #194.
2026-10-08 16:18:07 +08:00
wstockandClaude Fable 5.1 0e129e2811 #230: jobs time themselves with web_time, so Create Proxies and mask tracking don't panic on the web
The web build runs every job synchronously, and four job bodies (proxies,
mask tracking, video and audio previews, Project Manager) still called
std::time::Instant::now() for their "Done in N s" report — which panics
with "time not implemented on this platform" on wasm32 and took the whole
app down. They now use web_time::Instant like the rest of the engine
(std::time::Instant on native targets). autosave imports its clock from
web_time too, and a regression test scans the crate's non-test sources
for std::time::Instant so the next copy-pasted job fails cargo test
instead of the web app.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 07:45:37 +00:00
txiki739andClaude Opus 5.5 4af4765278 Merge storytold/main into i18n-spanish-panels
Brazilian Portuguese (#176) moves to the catalog system as i18n/pt-br.tsv (the
menu entries unchanged) and joins the general.interfaceLanguage preference. The
new UI text from main is wrapped and translated into Spanish: the credits
window, Text Properties, H.265 export rows, Clip Speed / Duration, Color Matte
dialogs, the voice-over discard button, the time-left suffix and the audio
output status messages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 08:59:32 +02:00
Claude 7fbb223707 #197, #193: web recovery retries failed snapshot writes and keeps skipped snapshots dirty
The OPFS recovery snapshot (apps/filmcraft-web) had two problems:

- #197: opfs::write never reported failures, and Autosave recorded the
  revision as snapshotted before the write ran. A failed write was only
  retried after the next edit.
- #193: the first tick of a clean session wrote {"dirty": false} to the
  meta. Starting with ?norecover or ?fresh therefore discarded a
  snapshot that was never reopened.

The decision of when to snapshot or mark clean moves to a pure module,
recovery_policy, tested natively in tests/recovery_policy.rs. Failed
writes (snapshot or clean meta) are retried after the 5 s interval
without another edit, and the meta is only marked clean to retire a
snapshot this session wrote. opfs::write_then reports whether a queued
write landed (coalesced writes report the newest one's result).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012HrAqZmrrDLRpxBJatxvaJ
2026-10-08 04:37:53 +00:00
carlitosteso3andClaude Opus 5.5 7392a69d0f #208: Go to In / Go to Out move to the sequence start / end without marks
The Program monitor's Go to In and Go to Out buttons (Shift+I / Shift+O) did
nothing when the sequence had no In / Out mark. Like Premiere (and like the
Source monitor already did), they now go to the start / end of the sequence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:43:49 -05:00
Brandon ThomasandClaude Opus 5.5 61238bf8fc Release CI: macOS job on macos-15-xlarge
Custom macOS larger runners in a runner group do not pick up jobs; the
built-in M2 Pro label does. Only release.yml uses it, and the
actions_macos_xl budget caps the monthly spend.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 22:07:57 -04:00
Brandon ThomasandClaude Opus 5.5 08b769a92a Release CI: macOS job on the org macos-release runner
The standard macOS runners are shared with all CI and queue for an hour or
more. The macos-release runner (GitHub-hosted M2 Pro) is in a runner group
usable only by release.yml on the release branch, with a monthly spend cap
on the actions_macos_xl SKU.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:54:54 -04:00
Brandon Thomas 5812509d3b Merge pull request #213 from carlitosteso3/fix/speed-dialog-undo-201
Speed/Duration opens its dialog; dragging an effect parameter is one undo step (#201)
2026-10-07 21:25:58 -04:00
Brandon ThomasandClaude Opus 5.5 ee8f886332 #201: each press of a drag begins its own undo step
A drag value changes only once the mouse moves, after the press frame, so sending `begin`
only on the press frame merged two drags of the same parameter into one undo step.
Remember the press that began a step by its start time instead; unit test drives two drags
through `run` with a real egui context (fails on the press-frame version).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:25:44 -04:00
carlitosteso3andClaude Opus 5.5 70aa661586 #201: Speed/Duration opens its dialog; dragging an effect parameter is one undo step
- The clip menu's Speed/Duration… applied 50% at once, and Cmd+R / Clip ▸
  Speed/Duration… ran it at 100% with no dialog. It now opens Clip Speed /
  Duration (speed, the resulting duration, Reverse Speed, Ripple Edit, Time
  Interpolation), seeded from the selected clip.
- effects.setParam took one undo step per frame of a drag, so Undo after
  dragging Time Remapping's speed (or any Effect Controls value) crept back in
  small steps. It takes `merge` / `begin` like the mixer and masks, and Effect
  Controls sends them while the mouse button is down: a drag is one step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:25:44 -04:00
Brandon ThomasandClaude Opus 5.5 87c4d1516e About: contributor and model credits compiled in (grab bag + table, self-submitted names) (#215)
* About: contributors compiled in from contributors.json (grab bag + table, sortable, name toggle)

Help ▸ About FilmCraft gets tabs About · Contributors · Models, following the craftrules standard
(standards/contributors.md): contributors/contributors.json is generated by
craftrules/scripts/contributors.py and baked into the binary by crates/ui-egui/build.rs as static
tables (nothing read at run time; the web build has them too). Each contributor line: @username,
opt-in display/real name, PRs, commits, lines +/−/Δ, binary assets +/−, first/last commit.
Sortable by every column; alphabetical ignores case and the leading @. Tabs and credits controls
register automation ids (about.tab.*, about.credits.*).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Contributors docs: craftrules is two levels up (../../craftrules)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Contributors: self-submitted names in contributors/people.toml; AGENTS.md credit-yourself section

build.rs reads contributors/people.toml next to the generated contributors.json and shows a name only
for entries the generator verified as written by that person (craftrules standards/contributors.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Contributors: regenerate (verified people.toml entries)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:16:52 -04:00
Brandon Thomas c337dcce01 Merge pull request #210 from cirocueto2013-wq/fix/hostile-input-bounds
fix: validate hostile project, export and MXF inputs
2026-10-07 20:53:11 -04:00
Brandon ThomasandClaude Opus 5.5 de70468ab6 #210 review: bound MXF reads by the file, keep legacy projects opening, allow 16K
- mxf: read_pcm bounds a request by the sample frames the file actually stores (plus 16M
  samples of zero padding) instead of a fixed 16,777,216-sample cap, so whole-clip reads of
  long stereo MXF (transcription) work again; PCM/AES3 chunk buffers are clamped to the source
  length instead of a 256 MiB cap; decode_pcm's output is bounded by its input, so its
  fixed cap goes (the AES3 >8-channel refusal stays; a request is also bounded by the source
  length divided by the channel count). read_sample keeps only the source-length bound.
- mxf: index_entries no longer fails open() past 1,048,576 entries (the sparse BTreeMap
  already bounds memory by real entries); long recordings open.
- project/format: new Sequence/Track/CaptionTrack::check_bounds() hold only the hostile-value
  bounds (settings, marks, overflowing time ranges); project load and export ranges use it, so
  projects that break only structural rules (overlap, zero duration) still open and export.
  check() = check_bounds() + the structural rules, as before.
- project: frame-size cap raised to 268,435,456 pixels, 32768 per side (16K, 16384x8192).
- engine: checked_u32_p accepts integer-valued floats (1920.0); renderable_sequence returns
  an EngineError (new Session::try_render_program_at, used by automation render) instead of
  eprintln.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:52:53 -04:00
cirocueto2013-wqandCodex 07d85b0f02 fix: validate hostile project, export and MXF inputs
Validate staged speed edits before adopting their timeline state.

Co-authored-by: Codex <noreply@openai.com>
2026-10-07 20:38:43 -04:00
Brandon Thomas a847671dbb Merge pull request #212 from storytold/test-188
#188: regression test — a copied Graphic clip pastes at the playhead
2026-10-07 20:37:34 -04:00
c57541674d #188: regression test — a copied Graphic clip pastes at the playhead
Engine test from #205 (the shortcut half of that PR is superseded by #204).

Co-Authored-By: carlitosteso3 <carlitosteso3@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:36:21 -04:00
5ee3ff64e9 Web: Link Media Locate… links the chosen file to the selected missing clip (#111) (#174)
* Web: Link Media Locate… links the chosen file to the selected missing clip (#111)

Link Media ▸ Locate…, `media.attachProxies` and `media.reconnectFullRes` all
need the chosen file's path back synchronously so they can call the engine
with `item` + `path`. The existing `pick_files` hook imports every chosen
file as a new project item, so on the web build (where the browser picker is
asynchronous and the hook returned `Vec::new()`) the command ran without a
path and the file silently landed as a new project item instead of being
relinked. On native the same hook worked because rfd returns paths
synchronously.

Add a separate `pick_file_for_relink` hook that returns the chosen file's
virtual path synchronously. Native hosts open an rfd dialog; the web host
registers the file in `/files/<name>` via a pending slot and runs
`media.relink` itself when the picker resolves. The three call sites prefer
the new hook and fall back to `pick_files` when the host has not installed
it (so the import flow used by every other dialog stays unchanged).

Fixes #111

Co-Authored-By: ninso112 <ninso112@pm.me>

* Co-ship: Luna161 ack'd and reviewed fix/111-link-media-locate

Reviewed the pick_file_for_relink hook routing, the take_files relink path,
and the regression test. Looks correct end to end; ready for upstream review.

Co-Authored-By: ninso112 <ninso112@pm.me>

* #111: relink hint names its command; no pending state after a cancelled picker

- RelinkHint carries the command (media.relink / attachProxies / reconnectFullRes), so the
  web host no longer runs media.relink for Attach Proxies and Reconnect Full Resolution.
- The web picker hands the hint to its own callback instead of a thread-local slot, so a
  cancelled picker can't turn the next ordinary import into a relink.
- Replace the hook-only unit tests with a UI test that drives Link Media ▸ Locate….

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: ninso112 <ninso112@pm.me>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:35:58 -04:00
Brandon Thomas 4f98721cab Merge pull request #209 from cirocueto2013-wq/fix/cpal-voiceover-recovery
fix: support integer CPAL devices and recover voice-over/audio failures
2026-10-07 20:35:54 -04:00
Brandon Thomas a4d65d3263 Merge pull request #204 from DWAA1660/fix/199-clipboard-shortcuts
#199: make Ctrl+C / Ctrl+X / Ctrl+V work on Windows and Linux
2026-10-07 20:35:50 -04:00
Brandon Thomas 6d904c1b37 Merge pull request #192 from cirocueto2013-wq/fix/editing-workflow-stability
Stabilize project lifecycle, timeline, audio and export workflows
2026-10-07 20:35:46 -04:00
Brandon Thomas d4ca3594ec Merge pull request #203 from carlitosteso3/fix/ripple-razor-snap-164
Ripple delete names the sync-locked track in the way; playhead snaps to cuts (#164, items 2, 3 and snap)
2026-10-07 20:35:42 -04:00
Brandon Thomas cd7395ec7c Merge pull request #207 from carlitosteso3/fix/ripple-trim-targets-164
Q/W ripple trim only the selected clips' tracks (#164)
2026-10-07 20:35:37 -04:00
carlitosteso3andClaude Opus 5.5 41659ae769 #164: Q/W ripple trim only the selected clips' tracks
With clips selected under the playhead, Ripple Trim Previous/Next Edit to
Playhead (and the plain trims) now trim only their tracks and linked partners,
like Add Edit; with none selected they use the targeted tracks as before. All
tracks are targeted by default, so W on a V1 clip also cut the music on A2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:50:39 -05:00
carlitosteso3andClaude Opus 5.5 e7e96b247f #164: ripple delete says which sync-locked track blocks it; playhead snaps to cuts
- A refused ripple delete / ripple trim now names the sync-locked track whose
  clip is in the way and how to proceed, instead of a generic "would break sync"
  (the edit is refused the same way Premiere refuses it).
- Dragging the playhead no longer snaps onto its own position (it stuck and moved
  in SNAP_PX jumps), snaps on the initial press too, and "Snap playhead in
  Timeline when Snap is enabled" is on by default, as in Premiere.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:47:21 -05:00
txiki739andClaude Opus 5.5 c7ddd4f314 tests: shortcut conflict test compares against the platform's own baseline
On Windows and Linux the macOS Control key folds into the primary modifier, so
the default set already reports conflicts there (documented in shortcuts.rs).
The test now checks that assigning adds exactly one conflict and that undo
restores the original set, on every platform.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 01:41:48 +02:00
txiki739andClaude Opus 5.5 3d67c4b73d tests: the MPEG oracle reads ffprobe's best-effort timestamps
Recent ffprobe versions report PTS=N/A for a program-stream picture whose
timestamp is inferred, so the oracle dropped that frame and compared 29 of 30.
best_effort_timestamp keeps the inferred time and all 30 frames are compared.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 01:41:48 +02:00
txiki739andClaude Opus 5.5 b6cdc9dd7a i18n: Spanish for the whole interface, catalogs and a remembered language
Translations move from Rust tables to one catalog per language
(crates/ui-egui/src/i18n/<code>.tsv), looked up through HashMaps. UI code
wraps its strings in tl!("…") and tlf!("…{name}", name); names from
registries (effects and parameters, settings, commands, panels, workspaces,
export and effect presets) are translated where they are drawn, so command
ids, automation ids and preference values stay English.

Spanish now covers menus, panels, dialogs, the Settings pages, the Effects and
Effect Controls registry, the Keyboard Shortcuts dialog and status messages
(about 3,350 entries). Tests fail when a tl!/tlf! literal, a menu label or a
registry name has no Spanish entry, and check catalog syntax, duplicates,
placeholders and ellipses. Searches match the Spanish label as well as the
English source. Window titles keep their English egui ids.

The language is stored in the general.interfaceLanguage preference (also in
Settings > General) and restored on startup; a failed preferences write still
switches the interface and says it could not be saved. Template values are
inserted literally, so a file name containing braces is not re-substituted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 01:41:48 +02:00
cirocueto2013-wqandCodex 7151d329d6 fix: support integer CPAL devices and recover voice-over/audio failures
Co-authored-by: Codex <noreply@openai.com>
2026-10-07 20:33:33 -03:00
DWAA1660andClaude Opus 5.5 f4221cd1b5 #199: make Ctrl+C / Ctrl+X / Ctrl+V work on Windows and Linux
egui-winit reports the clipboard shortcuts as Event::Copy / Cut / Paste
instead of key presses, so the shortcut bindings never matched them.
Outside text fields, add the key press each clipboard event stands for
before matching bindings, and put copied clip names on the system
clipboard so Ctrl+V is delivered right after a copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QQNunPf8nMeAQi6M92RT68
2026-10-07 16:57:38 -06:00
cirocueto2013-wqandCodex 1fd30aa29d engine: bound subclip media resolution and reject cycles
Co-authored-by: Codex <noreply@openai.com>
2026-10-07 19:56:58 -03:00
Brandon Thomas a4536bcfed Merge pull request #180 from SaymonMaster14/feat/windows-nvenc-h264
Windows: NVENC H.264 hardware encoding for Export (opt-in)
2026-10-07 18:17:05 -04:00
Brandon ThomasandClaude Opus 5.5 853f03b1aa M-HW: NVENC on top of the shared hardware-encoding setting (#183); never-crash fixes
Merge origin/main into the NVENC branch. #183 already provides the
HardwareEncoding enum, ExportSettings.hardware_encoding, the
hardwareEncoding export param and the Export > Video > Hardware Encoding
checkbox, so the duplicates from this branch are dropped and NVENC uses
them. filmcraft-export is now an all-platform dependency of
filmcraft-platform (VideoToolbox and NVENC both register encoders).

Never-crash fixes in the NVENC backend:
- submit() rejects a driver input pitch narrower than a row (and unlocks
  the buffer); fill_nv12 uses get_mut instead of slicing rows.
- composition offsets use saturating math and i32::try_from; frame-rate,
  size and timestamp casts are checked; sizes above u16::MAX decline.
- Annex B scanning and NAL type checks use get()/first().
- A failed nvEncOpenEncodeSessionEx that returns a handle destroys it.

ffi.rs carries NVIDIA's MIT notice for nvEncodeAPI.h, listed in
ATTRIBUTION.md under a new "Third-party source code" section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:16:12 -04:00
Brandon Thomas 71483312b6 Merge pull request #162 from SaymonMaster14/feat/windows-vp9-av1-hwdecode
Windows: VP9 + AV1 hardware decode (Media Foundation / DXVA)

# Conflicts:
#	ROADMAP.md
#	crates/platform/README.md
2026-10-07 17:58:12 -04:00
Brandon Thomas 13f71e3111 Merge pull request #175 from Luna161/fix/110-import-refreshes-offline
Engine: file.import refreshes the offline-missing list (#110)
2026-10-07 17:56:50 -04:00
Brandon ThomasandClaude Opus 5.5 4e4efcef12 M-relink: file.import also drops the cached slate of media it brought back (#110)
refresh() cleared the Missing badge, but the media pool kept the
MEDIA NOT FOUND slate cached under the item, so the monitors kept
showing it. The regression test now checks the rendered frame too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 17:56:33 -04:00
Brandon Thomas a4b92f1bec Merge remote-tracking branch 'origin/main' into pr-175 2026-10-07 17:54:19 -04:00
Brandon Thomas 1fe80344c7 Merge pull request #195 from midasdf/fix/vertical-type-tool-flyout
M10: choose vertical titles from the Type tool flyout
2026-10-07 17:53:53 -04:00
Brandon ThomasandClaude Opus 5.5 bd5387dff6 Merge main into #195: vertical type on top of point/paragraph text (#166)
The Type-tool click now goes through new_text_action; Vertical Type swaps
in graphics.newVerticalText and never draws a paragraph box (vertical text
is point text). Drops the dated ROADMAP log line (the maintainers keep the log).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 17:53:45 -04:00
Brandon ThomasandClaude Opus 5.5 6ce2038a00 M-HW: keep the Windows-only annexb/biplanar helpers warning-free on other systems (#141)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 17:50:20 -04:00
Brandon Thomas fcf1c45583 Merge pull request #141 from SaymonMaster14/feat/windows-media-foundation-hwdecode
Windows: Media Foundation/D3D11 hardware H.264 + HEVC decode

# Conflicts:
#	README.md
#	ROADMAP.md
#	crates/platform/Cargo.toml
#	crates/platform/src/lib.rs
#	docs/architecture.md
#	docs/performance.md
2026-10-07 17:48:06 -04:00
Brandon Thomas b5f197567b Merge pull request #186 from dcarnelutti/platform/videotoolbox-hevc-encode
platform: H.265 (HEVC) export with VideoToolbox

# Conflicts:
#	crates/export/src/settings_tests.rs
#	crates/ui-egui/tests/export_ui.rs
2026-10-07 17:47:10 -04:00
Brandon Thomas 84fcbeb1b9 Merge pull request #183 from dcarnelutti/platform/videotoolbox-h264-encode
platform: opt-in hardware H.264 encoding with VideoToolbox

# Conflicts:
#	crates/engine/src/export_tests.rs
2026-10-07 17:46:57 -04:00
Brandon Thomas d18bc04023 Merge pull request #184 from dcarnelutti/render/overlay-bounds-and-buffer-reuse
render: composite only what overlays cover, and reuse the float images
2026-10-07 17:46:38 -04:00
Brandon Thomas fed7201fcf Merge pull request #185 from dcarnelutti/export/progress-eta
export: show the estimated time left of a render next to its percentage
2026-10-07 17:46:37 -04:00
Brandon Thomas cbf181bd1f Merge pull request #176 from RicSchonfelder/i18n-pt-br
i18n: add Brazilian Portuguese (pt-br) UI translation
2026-10-07 17:46:33 -04:00
Brandon Thomas ce5d365c78 Merge pull request #165 from carlitosteso3/fix/media-browser-probe-157
Media Browser: probing a file doesn't read it whole (#157)

# Conflicts:
#	crates/engine/src/media_pool.rs
2026-10-07 17:46:32 -04:00
Brandon Thomas d01e1b893d Merge pull request #145 from carlitosteso3/fix/color-matte-29
Color Matte: pick the color on creation and change it later (#29)
2026-10-07 17:46:23 -04:00
Brandon Thomas c86847ff72 Merge pull request #155 from carlitosteso3/fix/playback-stalled-audio-136
Playback: a stalled audio device no longer freezes the playhead (#136)
2026-10-07 17:46:22 -04:00
Brandon Thomas 0810b403ce Merge pull request #148 from EthanSuquet/fix/h264-rc-slow-start
h264enc: 1-pass bitrate modes no longer starve the opening second
2026-10-07 17:46:21 -04:00
Brandon Thomas 8b3f118878 Merge pull request #173 from carlitosteso3/fix/shortcut-modifiers-164
Shortcut recorder accepts modifier chords; Add Edit cuts only selected clips (#164, items 1 and 5)
2026-10-07 17:46:20 -04:00
Brandon Thomas b4c65bdf8a Merge pull request #189 from Luna161/fix/168-project-select-all
Engine: project.selectAll and project.deselectAll for the Project panel (#168)
2026-10-07 17:46:19 -04:00
Brandon Thomas 47f75df15e Merge pull request #182 from justjohn12345/pr/layer-box-visibility
Graphics: only selected layers have a box, and a hidden layer has none
2026-10-07 17:44:08 -04:00
Brandon Thomas 91a36a4a3f Merge pull request #166 from justjohn12345/pr/text-box-modes
Graphics: point text and paragraph text, as Premiere has them
2026-10-07 17:44:07 -04:00
Brandon Thomas a3667ad26e Merge pull request #190 from ajsongpilot/fix/new-from-file-anchor
Graphics: New Layer from File anchors the picture at its own centre
2026-10-07 17:44:06 -04:00
midasdf b7beca9bb9 M10: choose vertical titles from the Type tool flyout 2026-10-08 06:13:41 +09:00
AJ DepewandClaude Opus 5.5 57069293fb Graphics: New Layer from File anchors the picture at its own centre
graphics.newFromFile resolved the Motion anchor with the sequence frame size
instead of the imported picture's size, so a picture smaller than the frame
got its anchor at the frame centre (e.g. 960,540 for a 4x2 image). Setting
Motion ▸ Position then moved the wrong point and the picture landed far from
where it was asked to go. place_video_clip now resolves auto points with the
source's own size (Project::source_size), falling back to the frame size.

The new-layer-from-file test checks the anchor is the picture's centre; it
fails before this change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:35:41 -06:00
06bc7ef9ec Engine: project.selectAll and project.deselectAll for the Project panel (#168)
Root cause: the only Select All was `edit.selectAll`, which fills the
timeline clip selection (`state.selection`). Nothing filled the Project
panel's selection (`state.project_selection`), so with 161 imported
items the user had to click each one.

Fix:
- `project_panel::commands()` gets `project.selectAll` (selects every
  item in the project) and `project.deselectAll` (clears the Project
  panel selection). They are regular journalled commands with no menu
  entry and no shortcut of their own, so they add no shortcut conflicts.
- `handle_shortcuts` (ui-egui) remaps a fired `edit.selectAll` /
  `edit.deselectAll` to the `project.*` command when the Project panel
  has focus. Cmd+A / Cmd+Shift+A keep acting on timeline clips
  everywhere else, and `edit.selectAll` behaves exactly as before.

Out of scope: no menu entry. The Edit ▸ Select All menu item still
calls `edit.selectAll` directly, because menu clicks don't go through
the focus remap. The regression test checks the engine commands but not
the UI keyboard remap.

Regression test: project_panel_tests::select_all_selects_every_project_item
(fails without the fix: unknown command).

Co-Authored-By: ninso112 <ninso112@pm.me>
Co-Authored-By: Luna161 <Rosenkranzrene@gmail.com>
2026-10-07 20:21:49 +00:00
Dario CarneluttiandClaude Sonnet 5.5 790b3a0697 platform: H.265 (HEVC) export with VideoToolbox
Adds `Format::Hevc` (Export > Format > H.265 (HEVC); `"format": "hevc"` in
`file.exportMedia`): HEVC Main, 8-bit 4:2:0, in MP4 (`hvc1`) or QuickTime, with AAC audio,
encoded by the macOS hardware encoder. There is no built-in H.265 encoder (one in pure Rust
is a project of its own, and x265 is GPL), so the format exists only where the OS has one.

- `videotoolbox_encode`: the session wrapper creates HEVC sessions too (`VtProfile::HevcMain`;
  the profile also says which codec). Keyframes are the HEVC random access points (NAL types
  16 to 21); the parameter sets are VPS / SPS / PPS; the `hvcC` record in the MP4 sample
  entry is the one VideoToolbox writes (sample description extension atoms), so profile,
  level and flags are the encoder's own. `VtEncoder::uses_hardware()` asks VideoToolbox
  whether the session runs on a hardware encoder (a test checks H.264 and HEVC), so a
  requirement dropped by accident could never become a silent software encode.
- `hardware_encode`: one adapter for both codecs. Choosing the format is the opt-in (the
  `hardware_encoding` setting is for H.264). `hevc_available()` (one small hardware session,
  created once) is registered with the new `filmcraft_export::register_format_probe`, so
  `available(Format::Hevc)`, the format list of Export mode and `exportFormats` follow the
  machine. `Format::has_builtin_encoder` says which formats need such a probe.
- `export`: `Format::is_h26x` where H.264 and H.265 behave alike (MP4 or QuickTime, AAC, size
  estimate, summary line). Two-pass VBR is refused up front ("H.265 export has no two-pass
  mode") and HDR sequences export as SDR, because there is no encoder to fall back to.
- Export mode: H.265 shows H.264's multiplexer, audio, metadata and bitrate controls, without
  profile, level, two-pass or the hardware checkbox, and says "Encoder: Hardware".

Apple M1 (8 cores), 1080p25, a 9:29 timeline (camera clip, ProRes 4444 overlays, AAC,
loudness): hardware H.265 at 10 Mb/s 195 s, hardware H.264 at 20 Mb/s 204 s (CPU 1029 s and
1042 s; the same H.264 export took 193 s and 200 s on other runs). It is not faster: the codec
that compresses further is the more expensive one (raw encoders: 166 and 189 frames/s), and
the export is bound by the CPU compositor, not by the media engine.

Quality per bit, 30 s of the same timeline against a ProRes 422 HQ export (average PSNR over
the three planes, SSIM): 20 Mb/s 46.88 dB / 0.9902 (H.264) and 46.91 dB / 0.9903 (H.265);
10 Mb/s 44.96 / 0.9863 and 45.09 / 0.9863; 6 Mb/s 43.72 / 0.9833 and 44.31 / 0.9843; 4 Mb/s
42.91 / 0.9807 and 43.64 / 0.9825; 3 Mb/s 42.37 / 0.9786 and 43.05 / 0.9807. The same picture
at 10 Mb/s and above; at 3 to 6 Mb/s H.265 needs 22 to 31 % less bitrate for the same PSNR.
Both formats therefore keep the same default bitrate. Details in docs/performance.md (HW3).

Tests: tests/hardware_encode.rs gets the HEVC counterparts of the H.264 ones (what is taken
and declined, a round trip through our HEVC decoder with luma PSNR above 30 dB, keyframes, MP4
and QuickTime exports with AAC, ffmpeg / ffprobe reading `hevc` / `Main` / `hvc1` /
`yuv420p` / BT.709, exact sizes, hostile configurations, encoders dropped at any point, the
format list agreeing with the probe, sessions on the hardware encoder); settings_tests.rs
checks names, ids, defaults, the summary, two-pass refusal and the probe; export_ui.rs the
controls H.265 shows and hides.

Stacked on the hardware H.264 encoder (the first commit of this branch).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 14:00:08 -06:00
Dario CarneluttiandClaude Sonnet 5.5 283ea80a62 export: show the estimated time left of a render next to its percentage
Export mode, the status bar and the Progress panel showed "Encoding 37%" and nothing about how
long the rest would take. They now say "37% · 2:05 left".

- `Progress::eta()` (the new `pace` module) is the time left at the job's speed over the last
  15 seconds. Each call is also a reading, so the UI polling on every frame it draws the job
  feeds it, and none of the places that advance `done` had to change. A trailing window follows
  the real speed changes of an export (an intro, a banner, plain camera footage) where an
  average since the start lags; a stall makes the estimate grow instead of freezing it; a long
  idle start (the loudness pass) is not counted as the job's speed; nothing is shown until a
  second of readings has progress, and nothing once the job is done or finished.
- `format_eta`: `45 s`, `2:05`, `1:02:05`, rounded up.
- Shown in the status bar ("Exporting... 37% · 2:05 left", and the `status.job.progress`
  automation label), in the Export queue (on its own line: the column is narrow) and in the
  Progress panel. The engine adds `etaSeconds` (null while unknown) to every job of `jobs.list`
  and to the encoding items of `export.queue.list`; docs/agents.md says so.

Measured on a whole 9:29 export (193 s, Apple M1), asking every 3 s: the first estimate comes
after 6 s; from 30 s on, the predicted end of the export is on average 1.1 s early and never
more than 14.5 s off (7.5 % of the run). The status bar read "9% · 3:00 left" at 21 s and
"97% · 5 s left" at 184 s.

Tests: `pace` (steady job, speed change, stall, idle start, count going back, bounded history,
extreme values, a whole `Progress`), `format_eta`, the engine (`etaSeconds` of jobs and queue
items), and export_ui (status bar label and queue item).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 13:48:52 -06:00
Dario CarneluttiandClaude Sonnet 5.5 d2ab33ba76 render: composite only what overlays cover, and reuse the float images
The CPU compositor, which is what export runs, works on premultiplied linear f32 images,
33 MB each at 1080p, and every layer was one of them. A profile of a 9:29 export with a
hardware H.264 encoder (so the encoder is no longer the bottleneck) put 61-64 % of the busy
CPU in compositing, 12 % in zeroing and copying memory, 8-11 % each in the float -> 8-bit
and RGBA -> Y'CbCr conversions and 4-7 % in decoding the ProRes overlays: a banner that is
transparent except for its bottom 100 rows still cost a whole conversion and a whole mix,
and the bottom layer paid for a zero-filled canvas and a pass that mixes it over that canvas.

Three changes, each of which gives the bits the general path gives:

- The canvas is allocated when a layer needs it, and an opaque first layer (a Y'CbCr frame
  without an alpha plane, opacity 100 %, Normal blend, the size of the output) becomes the
  canvas instead of being mixed over zeros.
- A clip of plain media whose frame has an alpha plane (a ProRes 4444 banner) is converted
  and mixed only inside the rectangle where its alpha is not zero: `VideoFrame::alpha_region`,
  `to_linear_f32_region` and `blend::composite_at` (`composite` is `composite_at` at the
  origin; Dissolve's pattern follows the pixel's place in the output). A clip with effects,
  opacity masks, frame blending, colour management or a picture that is not placed one to
  one on the output takes the general path, as before.
- `filmcraft_frame::pool` keeps idle f32 images (320 MiB at most). `take_f32_overwritten`
  hands one back with its old contents, which the conversions overwrite whole (so no
  zero-fill), and the export pipeline gives each frame's image back after the 8-bit
  conversion.

Apple M1 (8 cores), a machine that was not idle, so the two builds alternate and the
median is reported:

- Synthetic 1080p frame (`cargo run --release -p filmcraft-render --example frame_bench`,
  5 rounds): camera only 8.9 -> 6.5 ms, camera + banner 16.2 -> 7.8 ms, same picture checksum.
- A real project (1080p25 camera clip, ProRes 4444 banner and subscribe bar), 30 s with the
  banner exported with the built-in encoder, which is deterministic: 46.4 -> 36.7 s, and
  both exports are the same file (same MD5), so every frame of the real pipeline matches.
- The same project with a hardware encoder, 3 rounds: 30 s with the banner 17.4 -> 11.2 s
  (CPU 99 -> 56 s), 30 s of the camera alone 11.4 -> 8.8 s, 20 s with the subscribe bar
  11.1 -> 7.0 s.

Tests: `render/src/region_tests.rs` compares each shortcut with the general path bit for bit
(an opaque bottom layer, banners over and under a camera, an 8-bit sticker, every blend mode,
several opacities, moved, scaled and effect clips, a bigger source, reduced-resolution
rendering), each render twice so the second starts from recycled buffers, and checks which
layers take a shortcut; the frame tests check that a region is the crop of the whole
conversion, that `alpha_region` is the box around what is not transparent, and that a buffer
full of NaN never leaks into a conversion, for every pixel format. Breaking a shortcut on
purpose (a region one pixel off, an opaque layer taken as the canvas at half opacity, a
conversion that skips the alpha channel) makes them fail.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 13:35:20 -06:00
Dario CarneluttiandClaude Sonnet 5.5 4cab6bc5da platform: opt-in hardware H.264 encoding with VideoToolbox
Export encodes H.264 on the CPU with the built-in encoder, which is most of the time
of an export (about two thirds of the busy CPU in a profile of a 20 s export). On
macOS the media engine can do it: a VideoToolbox compression session behind
`filmcraft_export::VideoEncoder`, registered by `filmcraft_platform::register()`.

- `ExportSettings::hardware_encoding` (`HardwareEncoding::Off` | `Auto`; Export > Video >
  Hardware Encoding, `"hardwareEncoding": "off|auto"` in `file.exportMedia`, a boolean is
  understood too), off by default: the built-in encoder's output is byte-identical on
  every machine, a hardware encoder's is not.
- `videotoolbox_encode` (FFI, same containment as the decoder: SAFETY comments, no
  panic across the callback, safe API) and `hardware_encode` (safe adapter and factory).
- The factory declines, so the built-in encoder is used, for other formats, MXF, two-pass
  VBR, HDR, odd sizes, non-square pixels and any configuration VideoToolbox cannot create a
  hardware session for. An encoder that fails mid-export stops it with an error: unlike a
  decoder, it cannot hand a half-written stream to another one.
- No B-frames: on real 1080p footage the quality is the same with and without them
  (+-0.3 dB at equal bitrate) and the bitrate lands closer to the target, and without
  them every frame is presented at its decode time (no composition offsets, no edit list).
- `export::register_encoder` ignores a factory registered twice; `encoder_registered`.

Apple M1, 8 cores, single runs: a 9:29 timeline (camera clip, ProRes 4444 overlays with
alpha, AAC, loudness) exports in 311 s instead of 793 s (2.5x faster); its densest 134 s
in 84 s instead of 395 s (4.7x). The outputs compare at SSIM 0.990 / PSNR 46.5 dB. The encoder
alone runs 200 fps at 1080p (docs/performance.md).

Tests (tests/hardware_encode.rs): what is taken and declined, a round trip through our
software decoder, an export that decodes in our decoder and in ffmpeg, the fallback for
everything hardware declines, exact output sizes, an empty range, hostile configurations
and encoders dropped at any point; the engine's `hardwareEncoding` parameter.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 13:34:51 -06:00
Brandon ThomasandClaude Opus 5.5 86f3375775 Rename PrintCraft to PdfCraft (storytold/pdfcraft) (#177)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:45:32 -04:00
Davi Andrei a7b51aa313 nvenc: fix a doc path 2026-10-07 15:42:38 -03:00
Davi AndreiandClaude Sonnet 5.5 11b199dbcd Docs: NVENC H.264 hardware encoding
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMorBHwct72udS2LiC3ZHp
2026-10-07 15:42:28 -03:00
Davi AndreiandClaude Sonnet 5.5 6db932a9ba Windows: NVENC H.264 hardware encoding for Export (work in progress, #30)
NVENC through nvEncodeAPI64.dll (API 12.1, FFI layout checked against the C header by generated
tests), registered as an Export encoder in front of the software one; Export > Hardware encoding
(off by default). Declines two-pass VBR, HDR, MXF, unsupported sizes; B-frames, IDR/GOP, VUI, avcC.
Tests: decode with our decoder (PSNR), pipeline export vs software, decline paths.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMorBHwct72udS2LiC3ZHp
2026-10-07 15:41:02 -03:00
RicSchonfelder 9a4a85d8e1 i18n: Brazilian Portuguese (pt-br) joins Edit > Language
Language::PtBr is persisted as pt-br and translated through a new
PORTUGUESE table with the same 347 menu labels as SPANISH, so every
translated key still resolves to a real menu item. The new
app.language.portuguese command checks, switches and restores the
language like the other three, and the native menu maps it back.
Cargo.lock is untouched.
2026-10-07 15:26:26 -03:00
Luna161andninso112 16cc1c808c Engine: file.import refreshes the offline-missing list (#110)
file.import created new items but never rebuilt Session::offline.missing.
The UI reads that list to drive the Program slate and the Project
panel 'Media missing' badge, so any re-imported item the project
already listed as missing stayed listed as missing until on_open ran
again (project reopen) or the caller manually relinked.

Add a single crate::relink::refresh(s) call at the end of file.import,
matching the call on_open already makes after a project opens. Closes
the slate/badge symptom without requiring reopen or per-item relink.

The deeper SharedSource cache eviction issue the reporter observed
with media.autoRelink is out of scope here; a follow-up should call
MediaPool::remove for items whose path now resolves before re-insert.

Co-Authored-By: ninso112 <ninso112@pm.me>
Co-Authored-By: Luna161 <Rosenkranzrene@gmail.com>
2026-10-07 18:20:23 +00:00
carlitosteso3andClaude Opus 5.5 d47e794437 sequence.addEdit: selected clips under the playhead are cut, and only they (#164)
Add Edit always cut every targeted track, so with one clip selected it cut
all tracks. As in Premiere, selected clips under the playhead are now cut
and nothing else; with none there, the targeted tracks are, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 12:37:50 -05:00
carlitosteso3andClaude Opus 5.5 98c022e0b0 shortcuts: recording a chord skips the modifier key presses (#164)
egui 0.36 reports pressing Shift, Ctrl, Alt or the Windows / Cmd key as a
key event of its own, before the key they modify. The Keyboard Shortcuts
recorder took that first press and refused it ("AltLeft can't be used as a
shortcut"), so no chord with a modifier could be recorded. Modifier keys on
their own are now skipped; the chord is recorded with the key they modify.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 12:37:49 -05:00
justjohn12345andClaude Opus 5.5 b6281f968f Graphics: only selected layers have a box, and a hidden layer has none
Every layer of the selected graphic was outlined on the Program monitor,
so a box stayed after its layer was deselected, and a layer whose
visibility was off kept its box and could still be clicked and dragged.

As in Premiere Pro 26: a layer has a box only while it is selected (or
typed into), and a layer whose visibility is off is not on the monitor
at all: no box, no handles, no click target.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 13:24:43 -04:00
carlitosteso3andClaude Opus 5.5 a54bf13ebc Merge upstream main into fix/color-matte-29 (keep both tests at the end of clip_ops_tests.rs)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 12:02:28 -05:00
carlitosteso3andClaude Opus 5.5 dd404568ae media browser: probing a file doesn't read it whole (#157)
Looking at a folder probes each visible file for its properties. A format
no streaming reader takes (AVI, which isn't supported at all, and WAV,
MP3 and the other byte-opened formats) fell through open_reader's
whole-file path: a 290 MB AVI took 518 MB of memory and a full read just
to report that it isn't supported, a 247 MB WAV 500 MB.

- filmcraft-media: open_reader_within(.., max_whole) refuses, after the
  sniffing head, a file no reader opener takes when it is larger than
  max_whole. open_reader is open_reader_within with no limit.
- engine: MediaPool::probe_file uses it with PROBE_WHOLE_FILE_MAX (64 MiB);
  media_browser::probe calls probe_file. Import and playback still use
  open_file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 11:21:53 -05:00
Davi AndreiandClaude Sonnet 5.5 79cb1d0f8d Windows: VP9 + AV1 hardware decode through Media Foundation / DXVA (#30)
Extends the Windows backend (#141) to VP9 (profiles 0 and 2) and AV1 (main, 8/10-bit), 4:2:0.

- codecs::hw: FrameStreamInfo (vpcC / av1C): profile, bit depth, chroma, picture size and colour as
  the software decoders report them (vp9_color / av1_color are now shared with them), random-access
  samples, in-band format changes; StreamInfo lets HybridDecoder handle NAL and frame-coded streams.
- platform: media_foundation::stream (what each codec needs: DXVA profile, MFT request, input
  format), MfDecoder generalised; VP9 frames / AV1 temporal units go in as they are; the MFT's
  declared picture size (display aperture) is checked against the bitstream's.
- Declined up front: VP9 profiles 1/3, AV1 profiles 1/2, 4:2:2/4:4:4, mono, 12-bit, no codec
  extension, no DXVA profile. A key frame of another size or a different sequence header falls back
  to the software decoder.
- Tests: bit-exact parity (360p / 1080p / 2160p, seeks, resets, flush, hidden frames), forced
  failure, damaged samples and av1C, format change fallback, WebM / Matroska through the media stack.
- Bench: AV1 fixtures with libaom when ffmpeg lacks SVT-AV1.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMorBHwct72udS2LiC3ZHp
2026-10-07 12:39:45 -03:00
Brandon Thomas 8fcad73c92 Merge pull request #159 from storytold/fix/90-web-api-rejects-errors
Web: window.filmcraft rejects with Error objects everywhere (#90)
2026-10-07 10:57:54 -04:00
Brandon ThomasandClaude Opus 5.5 b82825acb8 Web: window.filmcraft rejects with Error objects everywhere (#90)
readFile (missing path), importUrl and openProject (HTTP errors) and openProject (neither a File
nor a URL) rejected with plain strings, so e.message was undefined and e instanceof Error false,
unlike execute / request. Their promises now go through as_error, which wraps a string in an Error
and passes an Error through unchanged.

Checked in Chrome against the web build: all four cases reject with an Error carrying the message
("/files/nope.mp4: not found", "…: HTTP 404", "need a File or URL").

Fixes #90

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 10:57:48 -04:00
Brandon Thomas 3561e856a6 Merge pull request #158 from storytold/fix/107-flatpak-media-access
Packaging: the Flatpak can read a project's media wherever it lives (#107)
2026-10-07 10:53:01 -04:00
Brandon ThomasandClaude Opus 5.5 20ddf9c49e Packaging: the Flatpak can read a project's media wherever it lives (#107)
The manifest granted only xdg-pictures, on the premise that the portals make filesystem access
unnecessary. But a project refers to its media by absolute path and the document portal exposes
only the project file, so a project in ~/Videos or ~/Music opened with all its media missing.
Grant the home directory and the usual external-drive mount points (/media, /run/media, /mnt), as
other video editors on Flathub do; nothing system-wide.

Fixes #107

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 10:50:24 -04:00
Brandon Thomas 5695626216 Merge pull request #156 from storytold/fix/docs-85-86-87-88-99
Docs: releasing guide, CLI export options, README and CLAUDE.md drift (#85, #86, #87, #88, #99)
2026-10-07 10:49:14 -04:00
Brandon ThomasandClaude Opus 5.5 fa67dfa372 Docs: releasing guide, CLI export options, README and CLAUDE.md drift (#85, #86, #87, #88, #99)
- docs/releasing.md (#99): release.yml points readers to it twice and it didn't exist. Written from
  the workflow and packaging scripts as they are (macOS, Windows x64/x86/arm64, Linux x86_64/aarch64
  with the glibc 2.35 floor, web; draft release, secrets).
- filmcraft-cli help (#86): export lists --scale, --quality, --no-audio and --queue, and apv among
  the formats.
- README (#87): more than 650 engine commands (671 today), not about 135.
- README (#88): the web line says the browser app ships with every release and links docs/web.md.
- CLAUDE.md (#85): the Illustrator-class sibling is ../vectorcraft (formerly DrawCraft).

Fixes #85
Fixes #86
Fixes #87
Fixes #88
Fixes #99

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 10:49:08 -04:00
Brandon Thomas 3186a754f2 Merge pull request #151 from carlitosteso3/fix/transcripts-89-97-98
Transcripts: disabled, not failing, in builds without speech-to-text (#89, #97, #98)
2026-10-07 10:44:59 -04:00
Brandon ThomasandClaude Opus 5.5 a066748354 Merge main: the transcript checks landed in #146; keep Transcribe Sequence and the docs
#146 landed the same transcript.generate / downloadModel / import changes, so this branch keeps
main's versions of those files and adds what #146 missed: Transcribe Sequence (sequence.transcribe)
is disabled like transcript.generate in a build without speech-to-text, and docs/transcripts.md
describes the disabled state. A test asserts both directions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 10:44:47 -04:00
Brandon Thomas 2647a2d9f8 Merge remote-tracking branch 'origin/main' into pr151
# Conflicts:
#	crates/engine/src/settings_tests.rs
#	crates/engine/src/transcript_tests.rs
2026-10-07 10:44:47 -04:00
carlitosteso3andClaude Opus 5.5 0da0a3430a playback: a stalled audio device no longer freezes the playhead (#136)
The playhead follows the audio clock (frames the output device consumed).
An output stream can open without error and then never call back (ALSA
with a busy or misconfigured device): the clock stayed at 0 and Play did
nothing, with no message.

advance_playback now notes when the audio clock last moved; if it stands
still for AUDIO_STALL_S (1 s) while playing, playback continues from where
the audio got to on the wall clock, the stream is stopped, a warning is
logged and the status bar says the audio output is not responding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 09:42:42 -05:00
Brandon Thomas 3628bbe744 Merge pull request #154 from storytold/fix/91-92-bridge-blocking-export
Bridge: a blocking export runs as an app job, so it neither times out nor freezes the app (#91, #92)
2026-10-07 10:42:35 -04:00
Brandon ThomasandClaude Opus 5.5 bd12dcae81 Bridge: a blocking export runs as an app job, so it neither times out nor freezes the app (#91, #92)
file.exportMedia with wait: true was forwarded to the app unchanged. The app then encoded on its UI
thread: it showed no progress and answered no other control request until the encode ended, and the
control server's 60 s reply limit reported "timeout" for any longer export while it went on and
succeeded.

BridgeClient::execute now starts the export without wait and polls jobs.list until the job
finishes, returning the same {job, path, result} (or the export's error). filmcraft-cli --bridge
exec and MCP bridge mode both use it, and MCP command_run in bridge mode now gets the same progress
notifications and cancellation as headless mode.

Tested against a fake app (wait: true never reaches it, one export started, result and error
returned) and end to end against the desktop app: a blocking demo export returned its 642-frame
result with exit 0 while ui.inspect was answered in ~0.05 s throughout.

Fixes #91
Fixes #92

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 10:42:28 -04:00
carlitosteso3andClaude Opus 5.5 b8fa786cad transcripts: disabled, not failing, in builds without speech-to-text (#89, #97, #98)
- transcript.generate is enabled only with a recogniser: one installed in
  Session::transcriber, or the build's own (feature `whisper`). Otherwise
  describe, command_list enabled_only and the menu report it disabled with
  "speech-to-text is not available in this build" as the reason (#97).
  Transcribe Sequence, which runs it, follows.
- transcript.downloadModel is enabled only with `speech-download` (#98).
- file.import with Automatically transcribe clips on no longer skips
  transcription silently when it can't run: the disabled command's reason
  lands in the import's errors as "transcription: ..." (#89).
- docs/transcripts.md describes the disabled state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 09:14:05 -05:00
Brandon Thomas a273a95ad2 Merge pull request #147 from storytold/fix/93-rpm-glibc-floor
Packaging: the rpm requires glibc >= 2.35, as the deb does (#93)
2026-10-07 10:07:56 -04:00
Brandon ThomasandClaude Opus 5.5 a50f6162ba Packaging: the rpm requires glibc >= 2.35, as the deb does (#93)
The binaries are built on Ubuntu 22.04 (glibc 2.35). The deb says libc6 (>= 2.35), but the rpm
named no glibc floor, so dnf installed it on the RHEL 9 family (glibc 2.34), where both binaries
then failed with "GLIBC_2.35 not found". Now dnf refuses the install there.

Fixes #93

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 10:06:42 -04:00
Brandon Thomas be5b34b383 Merge pull request #146 from storytold/fix/97-98-89-transcripts-without-speech
Transcripts: builds without speech-to-text say so (#97, #98, #89)
2026-10-07 10:05:02 -04:00
Brandon ThomasandClaude Opus 5.5 621240f761 Transcripts: builds without speech-to-text say so (#97, #98, #89)
- transcript.generate is enabled only with an installed transcriber or the whisper feature; it
  reported enabled and then always failed with "speech-to-text is not available in this build".
- transcript.downloadModel is enabled only with the speech-download feature.
- An import with Automatically transcribe clips (all imported) on reports a
  "transcription: speech-to-text is not available in this build …" error instead of importing
  silently untranscribed.

Fixes #97
Fixes #98
Fixes #89

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 10:04:55 -04:00
carlitosteso3andClaude Opus 5.5 56eaea2fa3 Color Matte: pick the color on creation and change it later (#29)
- File ▸ New ▸ Color Matte… (and Project ▸ New Item) opens a New Color Matte
  dialog: color picker, name, width, height and duration. Called with params
  (CLI, control, MCP) it still creates the matte directly.
- project.matteColor {item?, color}: change a Color Matte's color (the
  selected one by default); every clip of it follows and it undoes.
  Double-clicking a Color Matte in the Project panel opens its color picker,
  as in Premiere.
- media pool: a generator's cache key now holds its parameters, so a matte
  whose color changes (or comes back with undo) is generated again instead of
  the pool serving the first one forever.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 08:58:01 -05:00
Brandon Thomas e975621bbd Merge pull request #144 from storytold/fix/114-render-tools-read-only
MCP: render_frame and render_preview leave the playhead where it was (#114)
2026-10-07 09:55:32 -04:00
Brandon ThomasandClaude Opus 5.5 65c6b7365d MCP: render_frame and render_preview leave the playhead where it was (#114)
Both tools are annotated readOnlyHint, but a call with `seconds` moved the active sequence's
playhead (and, with Selection Follows Playhead on, the selection) to that time, in headless and
bridge mode alike.

- headless: Session::render_program_at renders the frame at `seconds` (frame-snapped, as the
  playhead would be) without touching the session.
- bridge: the Program monitor screenshot needs the live playhead there, so the tool records the
  playhead and selection, takes the screenshot and puts both back (the selection only if it changed,
  since selecting also leaves trim mode).

Regression test: rendering at 5 s and 8 s with Selection Follows Playhead on leaves playhead and
selection unchanged and still renders the requested frame (it moved the playhead to 8 s before).

Fixes #114

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 09:55:24 -04:00
justjohn12345andClaude Opus 5.5 8407f233c1 Graphics: point text and paragraph text, as Premiere has them
A text layer is now one of two types, each with Premiere Pro 26's handles:

- Point text (Type tool click): every handle scales the layer about its
  anchor point, both axes alike; the font size is untouched.
- Paragraph text (Type tool drag): the dragged rectangle is a box the text
  wraps in. Handles move that side or corner of the box, the opposite side
  stays, the text re-wraps at the same size. Lines below the box are not
  shown and the bottom-right handle turns into a red plus.
- The anchor point can be dragged with the Selection tool; the layer stays.
  It is picked before anything under it, so a shape dragged by its exact
  centre moves its anchor.
- Properties > Text > wrench opens Text Properties, where Text Layer Type
  switches a layer between the two (graphics.setTextType); the text stays
  where it is.

Shapes keep stretching away from their opposite side.

New: text layer parameter box_height, graphics.newText `box`, and
textType / box / overflow / anchor / scale in graphics.list. A layer saved
before a parameter existed can now be given it with graphics.set.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 09:54:26 -04:00
Brandon Thomas 50f8d12dd9 Merge pull request #143 from storytold/fix/125-oracle-sequence-fixture-race
Tests: image-sequence oracle fixtures are written atomically (#125)
2026-10-07 09:49:41 -04:00
Brandon ThomasandClaude Opus 5.5 ba411138d4 Tests: image-sequence oracle fixtures are written atomically (#125)
make_sequence had ffmpeg write numbered stills straight into the shared seq_<ext> directory and
treated it as ready once frames 1 and 12 existed. Two tests generating it at once (png_sequence_exact
and missing_frames_hold_the_previous_frame) read each other's half-written PNGs; one that fails to
decode is held as missing, so frame 6 equalled frame 3. Six parallel processes on a fresh fixture
dir failed 18 of 120 runs before, 0 of 120 after.

ffmpeg now writes into a testkit temp_path directory that is renamed into place (with a .complete
marker) only when all twelve frames are there; the gap copy is private to its test.

Fixes #125

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 09:49:35 -04:00
Brandon Thomas 0dfad0b321 Merge pull request #142 from storytold/fix/66-hostile-edit-list-subclips
Hardening: hostile MP4 edit lists and cyclic subclip chains (#66)
2026-10-07 09:46:03 -04:00
Brandon ThomasandClaude Opus 5.5 0dd66d8aa3 Hardening: hostile MP4 edit lists and cyclic subclip chains (#66)
- codecs: a resampled MP4 audio read scales the edit-list offset to the source rate in i128 and
  saturates the window arithmetic; a hostile media_time overflowed i64.
- engine: media_duration follows a subclip's parent chain in a loop bounded at 16 hops instead of
  recursing; a cyclic or very deep chain in a damaged project overflowed the stack.

Each has a regression test that failed before (overflow panic / stack overflow).

Fixes #66

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 09:44:45 -04:00
EthanSuquetandClaude Opus 5.5 68173aa82a h264enc: 1-pass bitrate modes no longer starve the opening second (#74)
The 1-pass ABR rate factor counts one second of virtual inter frames so the
first I frame gets a fair share of the budget. That history was priced once,
at the first frame, with the model's initial coefficient guess, which is
several times too high on ordinary content, and then only faded out. Every
VBR/CBR export therefore coded its first second 6-9 dB below the rest.

A virtual frame now costs what the P and B frames coded so far cost on
average (the old guess only until the first of them), re-estimated per frame.
On the synthetic 640x360p60 test at 2 Mb/s, 0.1-0.5 s goes from 25.0 dB to
32.1 dB against 33 dB after the first second, at the same bitrate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 09:41:14 -04:00
Davi AndreiandClaude Sonnet 5.5 932cdb77f5 Docs: Windows hardware decoding (platform README, ADR 0001, architecture, performance, roadmap)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMorBHwct72udS2LiC3ZHp
2026-10-07 10:07:40 -03:00
Davi AndreiandClaude Sonnet 5.5 5e440a6607 Windows: Media Foundation / Direct3D 11 hardware H.264 + HEVC decode (#30)
New backend in crates/platform (media_foundation/): a Direct3D-aware decoder MFT is driven at the
level of single access units (Annex B), given the process's D3D11 video device through an
IMFDXGIDeviceManager so it decodes with DXVA, and its NV12 / P010 textures are read back through a
staging texture into planar Yuv8 / Yuv16. H.264 Baseline/Main/High and HEVC Main / Main 10, 4:2:0.

- Hardware is verified: non-D3D-aware or async MFTs, streams the GPU's DXVA decoder does not list,
  and system-memory output (Microsoft's MFT software fallback) are declined or fail the stream so
  HybridDecoder continues with our decoder.
- mfplat.dll is loaded at run time so Windows N editions still start.
- register() / registered() / hardware_decoder_for() cover Windows; perf.stats reports the backend.
- Tests (Windows): bit-exact parity with the software decoders at 640x360, 1080p and 2160p
  (resets, seeks, flush), forced mid-stream failure, damaged input, concurrency, repeated init.
- Bench: runs on Windows (ffmpeg on PATH, process/thread CPU clocks), HEVC Main 10 2160p row,
  hardware counters per row.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMorBHwct72udS2LiC3ZHp
2026-10-07 10:07:40 -03:00
Brandon Thomas 94a22d7a83 Merge pull request #130 from punkhop/fix/premiere-ui-type
Theme: interface type closer to Premiere's
2026-10-07 08:33:06 -04:00
Brandon Thomas 6a02e92004 Merge remote-tracking branch 'origin/main' into pr-130 2026-10-07 08:27:41 -04:00
Brandon ThomasandClaude Opus 5.5 736760752b Theme: keep the interface in Inter; Source Sans 3 is an Adobe font (AGENTS.md §1.1)
The timecode, colour and menu changes stay; the interface face from craft-fonts is left out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 08:27:41 -04:00
Brandon Thomas 6262126056 Merge pull request #105 from mweinbach/feat/apv-codec
Codecs: add clean-room APV (RFC 9924) decoder, encoder, container support, and export presets
2026-10-07 07:10:17 -04:00
Brandon Thomas cd3fe20ef9 Merge remote-tracking branch 'origin/main' into apv-105 2026-10-07 07:09:54 -04:00
Brandon Thomas 94cda9dca4 Merge pull request #116 from justjohn12345/pr/nested-interchange
Nested sequences: interchange (AAF compositions, OMF rendered sound, EDL reel AX)
2026-10-07 07:09:39 -04:00
Brandon Thomas 16475596d3 Merge remote-tracking branch 'origin/main' into nest-116
# Conflicts:
#	crates/engine/src/commands.rs
2026-10-07 07:09:20 -04:00
Brandon Thomas d4e438e932 Merge remote-tracking branch 'origin/main' into apv-105 2026-10-07 07:08:58 -04:00
Brandon Thomas 9c026067a9 Merge pull request #132 from punkhop/fix/playback-rate-conversion
Audio: play a sequence on a device at another rate without phase jumps
2026-10-07 07:08:32 -04:00
Brandon Thomas 7916cfb80e Merge pull request #131 from punkhop/fix/audio-plays-ahead
Audio: mix playback 200 ms ahead so a busy Mac doesn't drop sound
2026-10-07 07:08:29 -04:00
Brandon Thomas 566164cf7d Merge pull request #129 from carlitosteso3/fix/he-aac-108
HE-AAC plays at its output rate; AAC export caps the bitrate (#108, parts 1–2)
2026-10-07 07:08:25 -04:00
Brandon Thomas b4caf39f5f Merge pull request #117 from justjohn12345/pr/add-tracks-dialog
Sequence: Add Tracks… opens Premiere's dialog (amount, placement, track type)
2026-10-07 07:08:21 -04:00
Brandon ThomasandClaude Opus 5.5 3f3d6ab45b Audio: the channel test lets the mixer get ahead before playing, so a busy runner doesn't flake it
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 06:34:20 -04:00
Brandon ThomasandClaude Opus 5.5 b94c81b5d1 HE-AAC: probing reads at most one access unit's worth of bytes per sample
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 06:34:19 -04:00
Brandon ThomasandClaude Opus 5.5 a1d245e517 Sequence: the Add Tracks dialog's OK check cannot overflow on huge counts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 06:34:16 -04:00
Brandon Thomas 005745d9ec Merge remote-tracking branch 'origin/main' into apv-105
# Conflicts:
#	crates/codecs/src/mp4.rs
2026-10-07 06:01:14 -04:00
Brandon ThomasandClaude Opus 5.5 d3c3b0f6e8 NEST: hostile project data cannot panic a nest's length, oversize its mix or multiply AAF needs
A negative clip duration no longer panics the clamp when a nest follows its contents; a nested
sequence's claimed sample rate is clamped before it sizes the mix buffer; audio_needs lists each
nested sequence once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 06:00:36 -04:00
Brandon Thomas 19d2f15148 Merge remote-tracking branch 'origin/main' into nest-116
# Conflicts:
#	crates/ui-egui/src/lib.rs
#	crates/ui-egui/src/panels/timeline.rs
2026-10-07 06:00:36 -04:00
Brandon Thomas a4c4d38550 Merge pull request #123 from Luskish/fix/control-close-on-malformed-input
Control: close the connection on input that isn't a request (#119)
2026-10-07 05:57:55 -04:00
Brandon Thomas 0232c53404 Merge pull request #124 from punkhop/fix/ripple-trim-l-cut-head
Ripple trim: a shorter head after an L cut keeps the earlier sound in place
2026-10-07 05:57:51 -04:00
Brandon Thomas a481f05e26 Merge pull request #104 from punkhop/fix/ripple-trim-j-cut-head
Ripple trim: a shorter head on a J-cut take is no longer refused
2026-10-07 05:57:46 -04:00
Brandon Thomas 13e5591bb2 Merge pull request #121 from punkhop/fix/user-workspaces
Window ▸ Workspaces: save your own workspaces, like Premiere
2026-10-07 05:57:42 -04:00
Brandon Thomas 7811da4ec0 Merge pull request #122 from coygeek/docs/add-star-history
docs: add star history chart to README
2026-10-07 05:57:38 -04:00
Brandon Thomas ce9d43a014 Merge pull request #128 from punkhop/fix/crisp-line-icons
Icons: draw straight lines on whole device pixels
2026-10-07 05:57:34 -04:00
Brandon Thomas 98979fdf74 Merge pull request #127 from carlitosteso3/i18n-spanish
i18n: Spanish interface language (Español)
2026-10-07 05:57:31 -04:00
Brandon Thomas dc16c0e219 Merge pull request #126 from carlitosteso3/fix/packaging-metadata
Packaging: describe FilmCraft, not PhotoCraft, in package metadata (#94, #100)
2026-10-07 05:57:27 -04:00
Brandon Thomas 631235e8b2 Merge pull request #120 from justjohn12345/pr/timeline-scrolling
Timeline: the wheel scrolls as in Premiere, and the view cannot be lost
2026-10-07 05:57:23 -04:00
Brandon Thomas 2bfcd512c2 Merge pull request #79 from justjohn12345/pr/layer-corner-scaling
graphics: all eight layer handles scale towards the pointer
2026-10-07 05:57:19 -04:00
Brandon Thomas aa1143ea74 Merge pull request #109 from dcarnelutti/gpu/alpha-plane-upload
gpu: upload and composite the alpha plane of Y'CbCr frames
2026-10-07 05:57:15 -04:00
Brandon Thomas 36d8114d2f Merge pull request #118 from HEOJUNFO/fix/probe-pixfmt-from-config
codecs: MP4/MOV pixel format and alpha come from hvcC, avcC and the ProRes frame header (#103)
2026-10-07 05:57:10 -04:00
Brandon Thomas 79057f9c36 Merge pull request #102 from EthanSuquet/fix/mp4-duration-edit-list
codecs: an MP4/MOV clip lasts as long as its edit list plays it
2026-10-07 05:57:06 -04:00
Brandon Thomas cb69b55e56 Merge pull request #101 from EthanSuquet/fix/h264enc-corrupt-frames
h264enc: strict ffmpeg oracle that sees damaged slices (#74)
2026-10-07 05:57:01 -04:00
Brandon Thomas 7fcbaeab5e Merge pull request #83 from EthanSuquet/fix/webm-frame-lookup-rounding
codecs: look video frames up by the nearest timestamp unit (WebM at 30 fps drops every third frame)
2026-10-07 05:56:56 -04:00
Brandon Thomas 205e4bf68e Merge pull request #96 from punkhop/fix/move-onto-occupied-spot
timeline.move: report the clips an overwrite move covers
2026-10-07 05:56:52 -04:00
Brandon Thomas b8cc3925c3 Merge pull request #68 from punkhop/fix/clear-deletes-a-bin
project: Clear deletes a bin it is given, with everything in it
2026-10-07 05:56:48 -04:00
Brandon Thomas 6a0fca1cd1 Merge pull request #64 from punkhop/fix/xml-master-clip-keeps-its-bin
interchange: an FCP7 master clip lands in its bin when a sequence used it first
2026-10-07 05:56:44 -04:00
Brandon Thomas 05a731b309 Merge pull request #77 from justjohn12345/pr/test-fixes
Tests: fix seed_sweep overflow and the speed-dependent export progress test
2026-10-07 05:56:39 -04:00
Brandon ThomasandClaude Opus 5.5 df3b4ecbde Control: a connection's slot is freed even if its thread panics or can't start
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 05:56:23 -04:00
Peet GuercioandClaude Opus 5.5 ea68e14a39 Merge fix/audio-plays-ahead: the resampler runs inside playback_mix
Problem: both branches change start_audio, so the nightly updater stops on the conflict.
Solution: keep play-ahead's structure; the rate converter lives in playback_mix (now FnMut).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 00:52:42 -07:00
Peet GuercioandClaude Opus 5.5 a4ff06f4ad Audio: play a sequence on a device at another rate without phase jumps
Problem: with the device at another rate than the sequence (44.1 kHz
device, 48 kHz sequence), each buffer truncated its source start (phase
lost every block: a buzz), took the nearest sample (aliasing), and asked
the mixer for overlapping spans, so the mixer's graph cache missed every
buffer and rebuilt effect state.

Solution: audio-dsp StreamResampler interpolates each output sample from
its absolute position (exact rational), bit-identical however the stream
is cut, and pulls the source as one contiguous stream. start_audio uses
it when rates differ and restarts it when the rate or layout changes.

Context: tests cover block-size invariance at four rate pairs, a 1 kHz
tone within 3e-3, contiguous pulls, seeks and hostile input. Same lines
as #131; whichever lands second needs a small rebase.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 00:35:29 -07:00
Peet Guercio bf00c29b14 Merge remote-tracking branch 'origin/fix/ripple-trim-j-cut-head' into fix/ripple-trim-l-cut-head
# Conflicts:
#	crates/edit/src/lib.rs
#	crates/edit/src/tests.rs
#	crates/engine/src/split_edit_ripple_trim_tests.rs
2026-10-06 23:56:32 -07:00
Peet GuercioandClaude Opus 5.5 bb5d7ca0a6 Audio: mix playback 200 ms ahead so a busy Mac doesn't drop sound
Problem: playback decoded and mixed every ~10 ms device buffer inside the
cpal callback (AAC decode, effects, opening a clip's file at a cut, rayon,
locks). On a loaded Mac any stall missed the device deadline: static and
dropouts, different on every play.

Solution: play_ahead mixes on its own thread into a lock-free ring kept
200 ms ahead; the callback only copies. An underrun plays silence, is
counted, and the mixer skips the same span so sound stays locked to the
picture. Any device width plays the channels it shares with the mix.
perf.stats reports playback.audio (callbacks, underruns, missingFrames,
minLeadMs). The program mix moved to playback_mix so bench_audio can play a
real project through a simulated 48 kHz device and count dropouts.

Context: on a real 22-sequence project, load average 38-58, 15 s per cut:
mixing in the callback overran 8-170 times, mixing ahead 0 (ring never
below 149 ms). The web build keeps mixing in its callback (no threads).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 23:47:47 -07:00
carlitosteso3andClaude Opus 5.5 463a59c1b4 codecs: HE-AAC plays at its output rate; AAC export caps the bitrate (#108)
- aac: the decoder notes SBR data in fill elements (EXT_SBR_DATA /
  EXT_SBR_DATA_CRC), so implicitly signalled HE-AAC is recognised;
  Decoder::sbr() covers explicit signalling too.
- codecs: aac_output_rate() gives an AAC stream's output rate (the
  extension rate, or twice a <= 24 kHz core when SBR data is present); the
  MP4 reader reports it instead of the core rate, so sequences made from an
  HE-AAC clip get 44.1 kHz instead of 22.05 kHz. Our decoder now also takes
  explicitly signalled HE-AAC (it decodes the AAC-LC core) and, when the
  stream rate is twice the core rate, interpolates the core to it with a
  streaming half-band upsampler. The SBR high band is not reconstructed yet.
- export: the AAC bitrate is capped at 6 bits per sample and channel
  (ISO/IEC 14496-3 frame limit) instead of failing with "bitrate out of
  range" (320 kbps stereo at 22.05 kHz).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 22:28:30 -05:00
carlitosteso3andClaude Opus 5.5 96e9a00997 packaging: describe FilmCraft, not PhotoCraft, in package metadata (#94, #100)
The Linux, macOS, web and Windows packaging still carried PhotoCraft's
image-editor metadata.

- nfpm.yaml (#94): deb/rpm summary and description describe the video
  editor; filmcraft-cli is a command runner, MCP server and exporter;
  Debian section video instead of graphics.
- AppStream (#100): AudioVideo/Video/AudioVideoEditing categories, matching
  the desktop entry; video keywords; the screenshot is the README hero image
  (the old filmcraft-demo.jpg URL was a 404).
- macOS Info.plist.in (#100): export the .fcproj project type (JSON) instead
  of PhotoCraft's .pcraft; app category video; drop the PSB and QOI imported
  types, which FilmCraft does not read.
- filmcraft.wxs header comment: the MSI declares no file associations.
- web hosting guide: iframe title "FilmCraft video editor".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 22:06:08 -05:00
carlitosteso3andClaude Opus 5.5 ed535180cb i18n: Spanish interface language
Add Español next to English and 日本語 (Edit ▸ Language ▸ Español, also in the
native menu bar). SPANISH covers every menu, submenu and menu item label
(347 entries, Latin American Spanish); names, formats and percentages stay
as they are. A new test checks that every entry is a current menu label, so
a renamed command cannot leave a dead entry behind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 22:06:02 -05:00
Peet GuercioandClaude Opus 5.5 066dcf5b98 Theme: menu items every 24 px, as in Premiere
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 20:02:13 -07:00
Peet GuercioandClaude Opus 5.5 f9ca62f67e Theme: interface type closer to Premiere's
Measured from Premiere screenshots at 1x: panel tabs and tooltips are a
humanist sans at 12 px; the big timecode is regular weight, 76 x 10 px
for 00:00:16:05, blue #4096f3; menus are the system's, 13 pt, #dedede.

- The interface face is Source Sans 3 (OFL), the closest open face to
  Premiere's, read from craft-fonts (CRAFT_FONTS_DIR) like the Japanese
  faces. Inter stays right behind it for symbols it lacks, and is the
  face when the app is built without craft-fonts. The web build embeds
  it too.
- The timecode over the timeline, under the Source and Program monitors
  and in the Reference Monitor is regular weight at 16 (Source Sans 3's
  width for Premiere's), in the timecode blue #4096f3 (also the scrubby
  value blue).
- The menu bar's menus and submenus are 13 pt in #dedede.

Tests: with craft-fonts, every weight of the interface text is Source
Sans 3 with Inter behind it; without, the interface keeps Inter; Japanese
still renders from craft-fonts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:48:51 -07:00
Peet GuercioandClaude Opus 5.5 3a48a99c94 Icons: draw straight lines on whole device pixels
The panel menu mark (≡) on the active tab was three 1.5 pt lines placed
at fractional positions, so at 1x they straddled pixels and anti-aliasing
made the gaps look uneven (3 px then 4 px). Every line icon had the same
half-pixel blur.

The icon pen now draws strokes a whole number of device pixels wide and
snaps every horizontal or vertical segment (and any that runs less than
a pixel sideways) onto the pixel grid, rounding from the icon's centre
the same way on both sides, so equal steps on the 16 grid stay equal on
screen at 1x and 2x. The tab's menu mark is now the Hamburger icon drawn
by the pen instead of hand-drawn lines.

Tests: every icon at 1x and 2x at four sizes off the pixel grid has only
whole-pixel straight lines; the menu mark's two gaps are equal at 1x,
1.5x and 2x.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:40:55 -07:00
Coy Geek 8d5acae231 docs: add star history chart to README
Show FilmCraft GitHub star growth with a linked Star History chart at the bottom of the README. Preserve the date view and top-left legend.
2026-10-06 19:33:01 -07:00
LuskishandClaude Opus 5.5 023127e146 Control: close the connection on input that isn't a request (#119)
A line that is not a JSON object with a string `method` (an HTTP request
line, other non-JSON text, a JSON array, invalid UTF-8, a line over 4 MiB)
now gets one error reply and the connection is closed, so a browser's
cross-origin fetch can't smuggle a command in its body. Connections are
capped at 16. Same transport as LightCraft (storytold/lightcraft#129).

Fixes #119.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 22:32:13 -04:00
Peet GuercioandClaude Opus 5.5 480b9e7655 Ripple trim: an L cut's sound stays only when its clip ends at the cut
A linked clip that spans the cut while its partner reaches past it too is
no L cut, and still refuses, like the same clip unlinked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:30:22 -07:00
Peet GuercioandClaude Opus 5.5 77d3b41a5c Ripple trim: a shorter head after an L cut keeps the earlier sound in place
A ripple trim that shortens a clip's head was refused when the previous
clip's linked sound ran on past the cut on a sync-locked track (an L cut).
That clip never moves, so its sound stays put and the L cut still ends
where it did. Unlinked material across the cut (a music bed) still refuses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:21:38 -07:00
Peet GuercioandClaude Opus 5.5 fc5fcd3322 feat(ui): save your own workspaces, like Premiere
Problem: Window > Workspaces only had the nine built-in layouts; a layout
you arranged could not be kept, renamed or brought back.

Solution: one alphabetical list of built-in and user workspaces, plus
Save Changes to this Workspace, Save as New Workspace..., Reset to Saved
Layout and Edit Workspaces... (rename, delete; built-ins restore instead
of delete). Saved in workspaces.json beside preferences.json, so an older
build rewriting preferences never erases them. The workspace in use
reopens at launch; ui.set and window.workspace.<name> switch to user
workspaces like built-ins.

Context: tests in crates/ui-egui/tests/workspaces_ui.rs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:18:02 -07:00
justjohn12345andClaude Opus 5.5 0bcfcfff16 Timeline: the wheel scrolls as in Premiere, and the view cannot be lost
Three problems with scrolling the Timeline:

1. Cmd + wheel zoomed, like Option + wheel. egui turns Cmd + wheel into a
   zoom before the panel sees it, and the panel only knew "zoom" and "scroll
   sideways".
2. The plain wheel scrolled sideways; the tracks could only be scrolled up
   and down with Shift held.
3. The view could be scrolled without limit past the end of the sequence (39
   minutes past a 26 second one). There the zoom scroll bar's thumb sat at
   the right end and stayed there when dragged: the bar's range grew with the
   scroll position, so the view moved but the thumb did not.

Now, as Premiere Pro 26.5.2 does on macOS (checked in the app):
- wheel: the tracks under the pointer, up and down (video tracks above the
  divider, audio tracks below);
- Cmd + wheel: the Timeline, sideways;
- Option + wheel: zoom about the pointer.
The existing setting Timeline Mouse Scrolling ("Vertical" by default) is
wired up: "Horizontal" swaps the first two. A sideways gesture (trackpad
swipe, tilt wheel, Shift + wheel) always scrolls sideways and a pinch zooms.
The wheel is read from its events, with their own modifiers.

The Timeline ends ten minutes past the end of the sequence, where Premiere's
view stops too, and the view is kept inside it however it is moved (wheel,
scroll bar, zoom, a script; not-a-number positions are reset).

The zoom scroll bar spans that extent. Its thumb is as wide as the share in
view, never narrower than 34 points, and its travel covers the whole scroll
range: it reaches both ends of the bar and stays under the pointer while
dragged (the drag follows the pointer's position, not a sum of steps). A
click or press beside the thumb brings the thumb there. The handles at its
ends zoom as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 22:14:00 -04:00
justjohn12345andClaude Opus 5.5 f6d78f7e0c Sequence: the Add Tracks dialog's fields span its width
The placement and track type lists had a fixed width, narrower than the
dialog, which left an empty band down its right side. Each row is now a label
in a column as wide as the widest label and a list that takes the rest of the
row, up to the group's right edge, as in Premiere's dialog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 21:45:38 -04:00
heojunfoandClaude Opus 5.5 f761a299a7 codecs: MP4/MOV pixel format and alpha come from hvcC, avcC and the ProRes frame header (#103)
The probe guessed the pixel format from the codec profile alone: HEVC
reported nothing, H.264 High 4:2:2 had no bit depth, and ProRes 4444
claimed alpha because of its fourcc.

- HEVC: chroma format and bit depth from hvcC.
- H.264: from the avcC High-profile extension (ISO/IEC 14496-15 §5.3.3),
  else from the first SPS; unknown (empty) if neither can be read.
- ProRes: the first frame header gives the chroma format and alpha
  (alpha_channel_type), as the first DNx frame header already does.

Checked against ffprobe on ffmpeg-made HEVC 4:2:0/4:2:2 8/10-bit,
H.264 4:2:0/4:2:2 8/10-bit and ProRes 422 HQ / 4444 with and without
alpha. Unit tests cover hvcC, avcC (extension and SPS fallback) and
ProRes 4444 with and without alpha.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 10:44:38 +09:00
justjohn12345andClaude Opus 5.5 13a0da9c2b Sequence: Add Tracks… opens Premiere's dialog (amount, placement, track type)
Sequence ▸ Add Tracks… added one video track at once: no dialog, and no way
in the interface to add an audio track.

It now opens the dialog Premiere Pro 26.5.2 shows, checked in the app:
- "Add video tracks": Amount, Placement.
- "Add audio tracks": Amount, Placement, Track type (Standard, 5.1, Adaptive,
  Mono).
- "Add audio submix tracks": Amount, Placement, Track type (Stereo, 5.1,
  Adaptive, Mono); its placement is off while the sequence has no submix
  track.
Placement is "Before First Track" or after one of the tracks, after the last
by default. The defaults are Premiere's: one video and one audio track, no
submix track. OK is one undo step.

`sequence.addTracks` takes the same choices as params (`videoAfter`,
`audioAfter`, `submixAfter`, `audioType`, `submix`, `submixType`) and returns
the new tracks' ids. Called without a place it still appends, and without
amounts it still adds one video track, so scripts keep working; only the menu
item opens the dialog. Tracks are numbered by their place, as in Premiere: a
track added after Video 1 is the new Video 2 and default names above it move
up, while a name the user gave is kept.

Amounts were not checked before (any number of tracks was created). At most
99 tracks are added at a time and a sequence has at most 999 of a kind; a
wrong amount, place or type is an error that changes nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 21:39:03 -04:00
justjohn12345andClaude Opus 5.5 1ebc1440e7 NEST11: the architecture doc says a plain nest's layers go into the frame plan
docs/architecture.md still listed every nested sequence among the things
rendered on the CPU and handed to the GPU as one image. Since NEST11 that
holds only for nests that are not plain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 20:53:53 -04:00
justjohn12345andClaude Opus 5.5 e13151d87b NEST10: the captions doc says nested captions are part of the nest's picture
docs/captions.md still said nested sequences do not burn in their own
captions. Since NEST10 the captions of a nested sequence are drawn wherever
the nest is shown, in the Program monitor and in exports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 20:53:53 -04:00
justjohn12345andClaude Opus 5.5 9b31f4eb84 NEST15: nested sequences in AAF and OMF, the way Premiere Pro writes them
A nested sequence was left out of both formats as a gap, with a report entry
telling the user to "render and replace" it first (a command FilmCraft does
not have). An AAF with nested compositions was read with gaps in their place.

What Premiere Pro 26.5.2 does, read off files it exported for a sequence with
a nest:
- AAF: the nested sequence is a composition mob of its own, and its clips are
  source clips that point at it.
- OMF: the nested sequence's sound is rendered into the document as one clip
  named after the sequence.

Export
- AAF: a nested sequence is written as a composition mob (no usage code; the
  exported sequence stays the top-level one). A clip of it is a source clip
  at that mob's first slot of the clip's kind, its start in that slot's edit
  units. Nests in nests are followed; a nest used twice is written once. With
  embedded or consolidated audio, the media of the clips inside a nest is
  prepared with the rest (`NestNeeds::Inside`).
- OMF: `audio_needs` lists the range of the nested sequence each nest clip
  plays (`NestNeeds::Render`); the engine mixes it through the export audio
  pipeline with what is on the clip, and it is written as media. Without that
  essence the nest is a gap and the report names the sequence.

Import (AAF)
- A source clip that points at a composition becomes a clip of a nested
  sequence; the composition is read once.
- Premiere Pro's layout is read: all video tracks in one slot (a nested scope
  with a segment per track; before, only the last segment was read, so a
  Premiere AAF came in with one video track), and "top-level" tagged on nested
  compositions too (top-level are those no other composition uses).
- A composition without a frame size takes that of the first picture it
  shows, not of the first picture in the file.

A sequence that claims to be inside itself is a gap with a report entry in
both directions; the walks are bounded (16 levels).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 20:53:34 -04:00
justjohn12345andClaude Opus 5.5 12585410b3 NEST15: a nested sequence is reel AX in an EDL, in every reel mode
The export report already said a nested sequence is written "as a single
source (reel AX)", but the reel came from the reel mode: with the default
(file name) a nest called "Nested Sequence 02" got reel NESTED_S. A nested
sequence has no tape and no file, so its reel is now AX whatever the mode.

Premiere Pro 26.5.2 writes the same for a nest (seen in an EDL it exported):
one event from reel AX, the nested sequence's name in "FROM CLIP NAME", and
the nested sequence's own time as source timecode. A test pins that event.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 20:53:19 -04:00
justjohn12345andClaude Opus 5.5 cdd9836705 NEST13: the panel menu closes on a click elsewhere and on Escape
Found while looking at the tab menu in the running app: once open, a panel
menu stayed open until one of its entries was chosen. It was meant to close on
a click elsewhere, but the test for that needed the press and the release of
the click in the same frame, which a real mouse never delivers. A right-click
on a tab now opens the menu, so the stuck menu was easy to run into.

The menu now remembers the frame it opened in (the click that opens it is not
a click elsewhere) and closes on any later click outside it, or on Escape. A
right-click on another tab moves it to that tab.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 20:29:41 -04:00
justjohn12345andClaude Opus 5.5 eeec772671 NEST14: a project reopens with its sequence tabs and their views
Premiere Pro 26.5.2 reopens a project with the same sequence tabs, in the
same order, the same one shown, each at the zoom it was left at. FilmCraft
opened every project on its first sequence; the "Restore open sequences when
opening projects" preference existed and nothing read it.

A saved project file now has a `view` beside `project`: the open sequences in
tab order, the active one, and each sequence's view from NEST12. It is not
part of the project: no undo step, no "edited" mark, and no schema bump (older
builds ignore it). `file.open` restores it when the preference is on.

Nothing in the view is trusted. Ids that are not sequences of the project and
repeats are dropped, numbers are clamped, and a view that cannot be read is
ignored instead of failing the open. A view with no open sequence leaves the
project on its first sequence, so a project saved by a script or the CLI does
not open on an empty Timeline.

Opening or recovering a project now tells the frontend which sequence is
shown, so the Timeline takes that sequence's view.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 20:23:19 -04:00
justjohn12345andClaude Opus 5.5 6cff0f0b5d NEST13: sequence tabs get Premiere's menu, drag to reorder and a list for tabs that do not fit
Checked in Premiere Pro 26.5.2: a right-click on a sequence tab that is not
shown shows it and opens the panel menu, which has "Close Panel", "Close Other
Timeline Panels" and "Reveal Sequence in Project"; a dragged tab follows the
pointer and its neighbours make room.

- A right-click on any tab shows it and opens its panel menu.
- The Timeline's menu gains "Close Other Timeline Panels" (new command
  `sequence.closeOthers`); its entries have automation ids
  (`panel.menu.Timeline.close`, `.closeOthers`, `.revealSequence`).
- A sequence tab dragged sideways changes places with a neighbour when the
  pointer passes that neighbour's middle (new command `sequence.moveTab`).
- Tabs that do not fit: the shown tab is always in the strip (tabs before it
  are left out until it fits), and the chevrons at the right end open a list
  of every tab instead of stepping to the next one. Ids `timeline.tabs.more`
  and `timeline.tabs.list.<sequence>`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 20:23:07 -04:00
justjohn12345andClaude Opus 5.5 6aefdef661 NEST12: every sequence keeps its own Timeline zoom, scroll and track heights
The Timeline had one view for all sequence tabs, and showing another sequence
fitted it to the panel every time. In Premiere Pro 26.5.2 each sequence tab
has its own zoom and its own track heights (expanding the tracks of one
sequence leaves the others as they were).

Each sequence now has a `SequenceView` (zoom, horizontal scroll, the vertical
scroll of the video and audio halves, video and audio track height) in
`EditorState::timeline_views`. The panel's view is the active sequence's: it
is put away when another sequence is shown and taken back when the sequence is
shown again, also when a tab is closed. A sequence shown for the first time is
fitted, at the default track heights. A view changed in the session (a
command, the control channel) is taken over by the panel. The view of a
deleted sequence is dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 20:21:19 -04:00
justjohn12345andClaude Opus 5.5 5e730ab020 NEST11: the frame plan draws a plain nest's layers itself; a nest's sound clip gets a waveform
Frame plan. A nest was always handed to the compositor as one image rendered
on the CPU. A plain nest (same frame size and colour settings as the sequence
it is in, no effects, masks, opacity, blend mode or motion on the clip, every
layer inside it blended normally, not a multicam angle) now contributes its own
layers to the plan, with its captions, so the GPU composites them directly.
Nests in nests are followed up to the nesting depth limit. Anything that is
not plain is planned as one image as before.

Waveforms. Premiere draws a waveform on the sound clip of a nest. The Timeline
now mixes the nested sequence in the background (under `catch_unwind`), keeps
one peak per 256 samples, and mixes again when the sequence's sound changes,
showing the old waveform until the new one is ready.

Tests: a plain nest is planned as its own layers and looks the same as the
reference render, also nested twice and over another layer; five kinds of nest
that are not plain are still drawn right; a nest's sound clip gets a waveform
that follows its sequence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:52:28 -04:00
justjohn12345andClaude Opus 5.5 a8b803c7a9 NEST10: a nest's sound follows its speed, direction and effects; its captions are part of its picture
Sound. A nest's audio went around the path that clips of media take, so speed,
reverse and the clip's audio effects did nothing to it, and a nested sequence
at another sample rate was mixed at the outer rate without converting. The
nested mix is now read through the same path as a media source (`effected`
takes a reader instead of a `MediaSource`): it is mixed at the nested
sequence's own rate, resampled to the outer one, and then goes through speed,
reverse and the clip's effects like any clip. The nesting depth limit still
applies.

Captions. Premiere Pro 26.5.2 draws the captions of a nested sequence in the
sequence it is nested in, as part of the nest's picture. A nested sequence is
now rendered with its captions whether or not the outer sequence shows its
own, and the preview cache key of a nest covers its caption tracks.

Tests: a nest sounds as its sequence does, keeps pitch and timing at another
sample rate, changes with speed and reverse like a clip, and starts later when
trimmed in; speed, reverse and frame hold pick the right frame of the nested
sequence; opacity and a Crop effect apply to the nest's whole picture; nested
captions are drawn and outer ones only when asked for.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:52:20 -04:00
justjohn12345andClaude Opus 5.5 642d3b2800 NEST9: a nest of another size or frame rate, and Fit / Fill frame in the clip menu
A nested sequence whose frame size differs from the sequence it is in comes in
centred at its own size with Scale 100, as in Premiere Pro 26.5.2 (a 1280x720
nest in a 1920x1080 sequence). The clip context menu gains Premiere's "Fit to
frame" and "Fill frame" next to "Scale to Frame Size".

Tests pin the behaviour: a smaller nest sits in the middle, a larger one is
cropped by the frame, a nest at another frame rate shows its sequence at the
same moment, Fit and Fill frame set Scale to 150 for that pair of sizes, and
Scale to Frame Size fills the frame and leaves Scale alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:50:27 -04:00
justjohn12345andClaude Opus 5.5 5ce9fbb48a NEST8: Multi-Camera submenu in the clip context menu
Enable (ticked when on), Flatten, and the cameras of the selected nested
sequence clip to switch to; greyed out when no selected clip is a nested
sequence. The commands already existed in the Clip menu.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:25:01 -04:00
justjohn12345andClaude Opus 5.5 4164fa2e7c NEST6: Reveal in Project and Reveal Sequence in Project
clip.revealInProject (clip context menu) selects the clip's source in the
Project panel and shows it there: the panel comes forward with the search
cleared, the list opens the bins on the way, and the icon and freeform views go
into the bin that holds the item. sequence.revealInProject (Timeline tab menu)
does the same for the sequence being edited.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:24:41 -04:00
justjohn12345andClaude Opus 5.5 598268721a NEST5: a sequence in the Source Monitor is marked and edited in like a clip
Mark In / Mark Out on a sequence loaded in the Source Monitor did nothing: the
marks were only stored for media. They are now the sequence's own In and Out, so
Insert and Overwrite edit in the marked part (as a nest, or as its clips with
the nest toggle off).

Tests also pin Match Frame on a nest: it loads the nested sequence at the frame
the nest shows, through a trim and a speed change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:23:45 -04:00
justjohn12345andClaude Opus 5.5 559a8b7543 NEST4: the nest toggle edits a sequence in as a nest or as its clips
The Timeline's "Insert and overwrite sequences as nests or individual clips"
button was drawn always on and did nothing. It now toggles
(sequence.nestSequences). Off, a sequence is edited in as its clips, as observed
in Premiere Pro 26.5.2: the clips come with their transitions and links, the
source's tracks that hold clips go to consecutive tracks from the destination
track up, and missing tracks are added. A sequence can then go into itself as
plain clips. Multi-camera sources stay one clip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:23:17 -04:00
justjohn12345 642cfb6bdf Merge branch 'pr/nested-safety' into pr/nested-editing 2026-10-06 19:10:58 -04:00
justjohn12345 29fd297045 Merge branch 'pr/sequence-tabs' into pr/nested-editing 2026-10-06 19:10:58 -04:00
justjohn12345andClaude Opus 5.5 ca489f9fce NEST2: Nest and Make Subsequence as Premiere Pro 26.5.2 does them
Observed in the app on 2026-10-06 (about 35 test nests):

- The nest is a linked picture + sound pair only when the selected sound is on
  one track and that track is free for the nest's length. With sound on several
  tracks, or another clip in the way on its track, the nest is picture only: the
  selected sound stays in the sequence (unlinked) and a copy with its own ids
  goes into the nest. How many video tracks the selection uses does not matter.
- Where the picture goes. With L the lowest selected video track and `a` the
  number of the highest audio track holding selected sound, it takes the lowest
  track T from L up such that T is free for the nest's whole length, and so is
  every video track numbered from T up to a - 1 that the selection had clips on.
  If none will do, a new track is added. Before, the nest could land on top of
  an unselected clip.
- In the new sequence, picture tracks keep their positions up to the highest one
  used. Sound tracks are A1 followed by the other tracks used, in order
  (A1 + A3 become A1, A2; A3 alone becomes an empty A1 and A2).
- After Nest nothing is selected in the timeline and the new sequence is
  selected in the Project panel; clip.nest returns the nest's clips.
- Make Subsequence loads its result in the Source Monitor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 18:23:01 -04:00
mweinbach 15787f7fe8 Codecs: add clean-room APV (RFC 9924) decoder, encoder, container support, and export presets 2026-10-06 17:24:34 -04:00
justjohn12345andClaude Opus 5.5 fc557a1c10 NEST3: a nest follows its contents
A nest keeps its length when its sequence changes. Tests now pin what already
held: it trims out to reveal added material and no further, shows and plays
nothing past shortened contents, and the outer sequence shows edits made inside.

New: the Timeline hatches the part of a nest past the end of its contents, as
Premiere does (Project::nest_overhang, automation id timeline.clip.<id>.empty).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 17:23:38 -04:00
justjohn12345andClaude Opus 5.5 094ea74bde NEST2: Nest keeps transitions, track names and channel layouts
Nest… copied only the clips: transitions between them were lost and the new
sequence's tracks had default names and stereo layouts. Transitions whose clips
are all nested now go into the nest with them, and the tracks are laid out like
the parent's (shared with Make Subsequence). Clips on locked tracks are left out
instead of being duplicated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 17:23:38 -04:00
justjohn12345andClaude Opus 5.5 29d9d0ac5a NEST1: a sequence can never contain itself
A sequence placed inside itself (directly or through another nest) crashed the
app: nested audio mixing and the frame plan's multi-camera shortcut recursed
until the stack overflowed.

- render: the audio mixer and the multi-camera shortcut follow nests to the same
  depth as the picture (MAX_NEST_DEPTH), then give silence / fall back.
- project: Project::nest_cycle finds a sequence that contains itself.
- engine: every edit that would create one is refused, whichever command asks
  (place, Source insert/overwrite, paste, ...). A project opened with a cycle
  already in it can still be edited and repaired.

Regression tests: both crashes (stack overflow before), the refusals, repair.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 17:23:38 -04:00
Peet GuercioandClaude Opus 5.5 7f77bdfd3a Ripple trim: a head trim grabbed by its early sound uses the picture's cut
Sync-locked tracks rippled from the grabbed member's start, so grabbing a
J cut's early sound checked and shifted from before the picture's cut and
was refused. For a head trim the cut is the latest member's start. Adds the
test only the new stretch catches: a clip starting at the cut on a
sync-locked track, which the old stretch let fall out of sync.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 14:22:13 -07:00
Peet GuercioandClaude Opus 5.5 27124080bf Ripple trim: a shorter head closes the stretch after the cut
A ripple trim that shortens a clip's head was refused as a sync-lock
conflict whenever a sync-locked track held material just BEFORE the cut,
because the check looked at the stretch before it. With a J cut that is
the outgoing clip's sound on the other audio track, so the J-cut take
could never lose its head. The stretch that closes is the one just after
the cut; the clip's own linked partners stay the caller's to trim.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 14:16:39 -07:00
Dario CarneluttiandClaude Sonnet 5.5 1b92ea210b gpu: upload and composite the alpha plane of Y'CbCr frames
The GPU compositor uploaded only the Y, Cb and Cr planes of a Yuv8/Yuv16 frame and
the shader returned alpha 1.0 for them, so a ProRes 4444 overlay (banner, lower third,
outro) was drawn opaque in the Program monitor and covered the footage under it.
Exports were correct because they use the CPU compositor.

Frames with an alpha plane now upload it as a fourth texture (R8Unorm, or R16Float
converted like the other planes), bound at binding 5 of the layer and blend layouts.
A uniform slot (p3.y) carries the alpha scale (0 = no alpha plane); the shader reads
the alpha at full resolution and premultiplies the linear colour by it.

Adds gpu_draws_yuv_alpha_plane_like_the_cpu: an alpha ramp over an opaque layer at
8, 10 and 12 bits, GPU against the CPU compositor. It fails without the change
(p99 diff 190/255) and passes with it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-06 15:04:48 -06:00
justjohn12345andClaude Opus 5.5 3f7fd8252f Timeline: tab menu says Close Panel, as Premiere's does; test finds the nest by its sequence
Checked in Premiere Pro 26.5.2: the tab's menu item is "Close Panel" and it
closes that sequence's tab. Premiere also leaves nothing selected after Nest,
so the tabs test no longer relies on the selection to find the nest clip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:46:18 -04:00
EthanSuquetandClaude Opus 5.5 1f2f20a700 codecs: an MP4/MOV clip lasts as long as its edit list plays it
The clip duration came from the media duration (`mdhd`), which runs on
the decode timeline. ffmpeg / x264 MP4s with B-frames skip the reorder
delay with an edit, so every one imported a frame too long (a 120-frame
file became 121 frames, the last a held repeat). A video track delayed
by a leading empty edit was a frame short, and its last frame could not
be reached.

Use the sum of the track's edits (movie timescale, what `tkhd` records)
when there is an edit list. Fragmented files, and edit lists that add up
to nothing or overflow a Tick, keep the media duration.

Tests: a unit test with crafted edit lists (skip, empty edit, hostile
sizes) and an ffmpeg oracle test (libx264 with B-frames: 60 frames,
was 61).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:37:06 -04:00
Peet GuercioandClaude Opus 5.5 dfaf31bfec Timeline: timeline.move reports the clips an overwrite move covers
An overwrite move shortens, splits or removes whatever already sits where a
clip lands, as Premiere's drag does, but the reply only named the clips it
moved: a script moving clips one at a time never learned that an earlier move
had cut one it had not reached yet. The reply now lists each covered clip
under `overwritten` with its length before and after and the ids what is
left of it lives on as. The edit itself is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 13:19:04 -07:00
EthanSuquetandClaude Opus 5.5 3eae70b11e h264enc: strict ffmpeg oracle that sees damaged slices (#74)
ffmpeg prints "corrupt decoded frame" as a warning, so the oracle's
`-v error -ec 0` decode let a stream with a whole slice missing pass
silently. Switching that decode to `-v warning` is not enough: with
`-ec 0`, ffmpeg 7.1/8.1 flags every picture that has more than one
slice, libx264's multi-slice streams included, while the pixels are
unchanged. That is the 119/120 in #74; the encoder's bitstream is not
at fault (VideoToolbox decodes the same streams bit-identically, and
ffmpeg with strict error detection reports nothing).

The oracle now decodes with ffmpeg's default concealment, `-v warning`
and `-err_detect +crccheck+bitstream+buffer+explode`, fails on any
output or a non-zero exit, and still compares every pixel with the
encoder's reconstruction. A negative control drops and truncates
slices (CAVLC and CABAC, 3 slices) and checks the oracle reports each
one; it fails with the old flags.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 16:02:53 -04:00
EthanSuquetandClaude Opus 5.5 2b49201e84 codecs: video frames are looked up by the nearest timestamp unit, so a 1 ms WebM shows every frame
Problem (#73): a 30 fps WebM played and exported with every third frame
missing and the frame before it shown twice (20 unique frames a second).
The same happens at 29.97, 60, 59.94, 24 and 23.976 fps, and with MP4
files whose video track has a 1 ms timescale. 25 and 50 fps were fine.

Cause: Matroska stores block timestamps in TimestampScale units (usually
1 ms), rounded to the nearest unit, so at 30 fps frame 2 (66.67 ms) is
stored as 67. MkvSource::video_frame converted the requested time to track
units with a floor (66), and the lookup (last sample with pts <= target)
returned frame 1. Every frame whose timestamp was rounded up was skipped.
Mp4Source::video_at, and the web build's WebCodecs MP4 path, floored the
same way against the track timescale; normal MP4 timescales hold frame
times exactly, so only coarse (1000) timescales were hit.

Fix: new Tick::to_rational_round (nearest unit, halves up, total for zero
and negative timebases), used for the video lookup target and the
catch-up threshold in mkv.rs, mp4.rs and webcodecs.rs. A requested time
now matches a frame whose stored timestamp is within half a unit of it,
the same precision the timestamps have. Truncating muxers (stamps rounded
down) still work. Times on exact timebases (90 kHz, 15360, 30000/1001…)
convert exactly as before.

Test: crates/codecs/src/rounded_pts_tests.rs builds synthetic Motion-JPEG
streams in the test (each picture shows its index as black/white blocks),
muxed as WebM with 1 ms timestamps and as MOV with a 1000 timescale, at
30, 29.97, 60, 59.94, 24, 23.976 and 25 fps with nearest and truncated
stamps. Requesting every frame's start over 3 s must show every frame
exactly once, and every frame's midpoint must show that frame. Before the
fix both tests failed ("30 fps, Nearest ms stamps: 30 of 90 frames never
shown [2, 5, 8, 11]…, 30 shown twice [1, 4, 7, 10]…"). The time crate
tests cover the rounding, ties and hostile timebases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 15:16:37 -04:00
Brandon Thomas 771b614ad4 Merge pull request #81 from storytold/pr/craft-fonts
Fonts: use craft-fonts as an optional build input (Japanese UI and title fallback)
2026-10-06 15:08:06 -04:00
Brandon ThomasandClaude Opus 5.5 7f8e102b13 Fonts: the Japanese interface uses craft-fonts first; absolute CRAFT_FONTS_DIR
The merged Japanese interface (#38) looked only for a system font. When
FilmCraft is built with craft-fonts, theme::install already appends the
Japanese faces to every font family, so switching to Japanese uses them
(no system scan) and works on the web too; otherwise it falls back to an
installed system font as before. Docs use an absolute CRAFT_FONTS_DIR
(build.rs runs in crates/text, so a relative path resolves from there).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 14:26:14 -04:00
justjohn12345andClaude Opus 5.5 b0aeabdb8d graphics: all eight layer handles scale towards the pointer
A corner handle scales both axes alike with the opposite corner held in place;
the edge handles (top, right, bottom, left) are now grabbable and stretch only
their own axis with the opposite edge held in place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 14:17:29 -04:00
justjohn12345andClaude Opus 5.5 c80d23fd2e graphics: a corner handle scales the layer away from the opposite corner
Dragging a corner handle of a graphic layer in the Program monitor scaled about
the layer's anchor, so a text layer (anchor at its bottom-left) grew up and to
the right whichever corner was dragged. The opposite corner now stays in place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 13:52:20 -04:00
justjohn12345andClaude Opus 5.5 ea491ef467 automation: the export progress test no longer depends on machine speed
A short export can finish inside one 100 ms poll, which left a single progress
notification and no time to answer the ping. The test now tries longer exports
until one outlasts a few polls, and cancels a longer one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 13:51:43 -04:00
justjohn12345andClaude Opus 5.5 10b007cea9 Timeline: sequence tabs, Premiere-style clip menu, Nest name dialog
- The Timeline panel shows one tab per open sequence (x closes the active one);
  closing a sequence keeps the panel, and opening one restores a closed panel.
- Double-clicking a nested sequence clip opens it in its own tab.
- The clip context menu follows Premiere's grouping and adds Paste/Remove
  Attributes, Ungroup, Synchronize, Merge Clips, Make Subsequence, Reveal Nested
  Sequence and the frame hold / field options; right-click selects the clip.
- Nest... asks for the name (Nested Sequence Name) and numbers the default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 13:51:43 -04:00
justjohn12345andClaude Opus 5.5 fb824c6fb0 mpeg2v: seed_sweep derives its seeds without overflowing
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 13:51:43 -04:00
Brandon ThomasandClaude Opus 5.5 9953befe28 Fonts: use craft-fonts as an optional build input
crates/text/build.rs embeds the fonts of a storytold/craft-fonts checkout
named by CRAFT_FONTS_DIR as filmcraft_text::fonts::CRAFT_FONTS (native:
all of them; wasm32: only BIZ UDPGothic Regular). Unset, CRAFT_FONTS is
empty and nothing changes. A bad CRAFT_FONTS_DIR is a warning, or an
error with CRAFT_FONTS_REQUIRED=1. Never a Cargo dependency, no network.

- UI: the Japanese faces are the last fallback of every egui font family
  (BIZ UDPGothic first, Bold first in the semibold/medium families).
- Text engine: registered after the bundled faces; fallback_for picks
  them after the bundled faces and before system faces (Mincho for serif
  text, Gothic otherwise, nearest weight).
- Tests: Japanese glyphs (no tofu) in the UI and title paths, skipped
  without craft-fonts; the theme works with CRAFT_FONTS empty.
- Release: every build job checks out craft-fonts at a pinned commit and
  builds with CRAFT_FONTS_DIR + CRAFT_FONTS_REQUIRED; packages carry
  OFL-<family>.txt. NOTICE, ATTRIBUTION.md (optional build input), AGENTS.md,
  CLAUDE.md, README and docs/contributing.md describe it.

FilmCraft main bundles no Japanese font, so no font files move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 11:40:29 -04:00
Brandon Thomas a2f6ba8b44 Merge pull request #72 from storytold/pr/docs-hw-decode-correction
Docs: hardware decode correction; log the 2026-10-06 community batch
2026-10-06 11:39:47 -04:00
Brandon ThomasandClaude Opus 5.5 83481c146f Docs: hardware decode was only active in the desktop app since #41; log the 2026-10-06 community batch
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 11:39:41 -04:00
Brandon Thomas 5c8471c9f4 Merge pull request #41 from justjohn12345/pr/decode-memory
MEM1–MEM4 + HW1: clip memory on a timeline from 12 GB to 1.9 GB; hardware decode actually on
2026-10-06 11:38:03 -04:00
Brandon Thomas 37361900a9 Merge pull request #38 from midasdf/feat/japanese-interface-language
i18n: Japanese interface language (uses an installed system font)
2026-10-06 11:37:57 -04:00
Brandon Thomas e89feeac4d Merge pull request #58 from punkhop/fix/speed-ripples-linked-clips-once
edit: a speed change with ripple moves later clips once
2026-10-06 11:37:52 -04:00
Brandon Thomas 555bfe7c18 Merge pull request #57 from punkhop/fix/ripple-delete-split-edit
edit: ripple delete closes one gap for a clip with a split edit
2026-10-06 11:37:47 -04:00
Brandon Thomas 5dd09386f6 Merge pull request #56 from punkhop/fix/ripple-trim-keeps-split-edit
edit: a ripple trim takes a later split edit along
2026-10-06 11:37:42 -04:00
Brandon Thomas 475c294678 Merge pull request #45 from punkhop/fix/move-linked-partners
engine: timeline.move takes linked partners along
2026-10-06 11:37:38 -04:00
Brandon Thomas 51026bd2b2 Merge pull request #60 from punkhop/fix/place-on-a-sound-track
engine: timeline.place on an audio track places the sound there
2026-10-06 11:37:32 -04:00
Brandon Thomas d886d2f322 Merge pull request #44 from punkhop/fix/unknown-track-is-an-error
engine: naming a track that does not exist is an error
2026-10-06 11:37:28 -04:00
Brandon Thomas 39ef62607d Merge pull request #47 from punkhop/fix/app-help-and-unknown-flags
filmcraft: --help prints usage and an unknown option is an error
2026-10-06 11:37:22 -04:00
Brandon Thomas 8dfa8fff4c Merge pull request #52 from punkhop/fix/explicit-targets-enable
engine: a command whose targets are named in the parameters does not need a selection
2026-10-06 11:37:16 -04:00
Brandon Thomas 8a795324f4 Merge pull request #50 from punkhop/fix/replace-reports-short-media
engine: Replace With Clip reports clips that run past the new media
2026-10-06 11:37:11 -04:00
Brandon Thomas eb3ca3e811 Merge pull request #49 from punkhop/fix/replace-from-bin-keep-source-in
engine: clip.replaceFromBin can keep the clip's source In
2026-10-06 11:37:05 -04:00
Brandon Thomas 950879ef10 Merge pull request #59 from punkhop/fix/inspect-reports-reverse
engine: sequence.inspect reports a reversed clip
2026-10-06 11:36:58 -04:00
Brandon Thomas 7377def476 Merge pull request #48 from punkhop/fix/sequence-inspect-readback
engine: sequence.inspect reports clip ends, gain and marker comments
2026-10-06 11:36:53 -04:00
Brandon Thomas 5bc07026c7 Merge pull request #43 from punkhop/fix/import-auto-points
engine: a project saved after an interchange import opens again
2026-10-06 11:36:46 -04:00
Brandon Thomas a3b077877f Merge pull request #51 from punkhop/fix/interchange-export-reports-dropped-graphics
interchange: an export names every title it cannot carry
2026-10-06 11:36:41 -04:00
Brandon Thomas a407099019 Merge pull request #46 from punkhop/fix/export-interchange-unknown-format
engine: file.exportInterchange refuses an unknown format
2026-10-06 11:36:36 -04:00
Brandon Thomas bccc3dd031 Merge pull request #42 from punkhop/fix/mp4-audio-thumps
codecs: resampled MP4 audio reads apply the edit list once
2026-10-06 11:36:31 -04:00
Brandon Thomas bd40764756 Merge pull request #69 from storytold/pr/windows-arm64
Windows on ARM64: release builds, PE header check, install test on ARM64 hardware
2026-10-06 10:10:16 -04:00
Brandon ThomasandClaude Opus 5.5 e6d693cc24 Windows ARM64 workflow: pin third-party actions by commit
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:51:06 -04:00
Brandon ThomasandClaude Opus 5.5 ac906666d9 Windows on ARM64: release builds, PE header check, install test on ARM64 hardware
Release builds add an arm64 MSI and portable zip, cross-compiled on the x64
runner (signing and WiX as for x64). package.ps1 checks each binary's PE
header before packaging: Machine must match -Arch, the app must be a GUI
program and the CLI a console program. It skips the CLI smoke run when an
ARM64 build can't run on the host. A new Windows ARM64 workflow packages the
ARM64 MSI, installs it on a Windows 11 ARM64 runner, checks both installed
exes are ARM64, runs the CLI, and uninstalls. Ported from printcraft#66.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:38:38 -04:00
Brandon ThomasandClaude Opus 5.5 6a9fdce2ef codecs: released decoders are dropped off the cache lock; dirty-pool bit-exact test
trim_decoders dropped the decoders it took from idle caches while the requesting cache's lock was held. Dropping a decoder can block (a VideoToolbox session waits for its frames in flight), so the render path paid that latency under the lock. trim_decoders now returns the released decoders and frame_late keeps them in a local declared before its lock guard, so they drop after the guard on every return path; the cap is enforced exactly as before.

New tests/pool_bitexact.rs: primes the frame pool with 0xAA-filled plane buffers, then decodes an H.264 fixture into them; every frame must match ffmpeg bit for bit, and the pool must report the reuse.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:16:09 -04:00
punkhop 64d41b266a project: Clear counts each item and bin once
A bin inside another named bin goes with it and is not counted again;
an item named and also inside a named bin is counted once. The reply's
counts now match what went. The params line stays {"items":[id]?} so
a caller naming targets keeps the same shape as before.
2026-10-06 06:10:56 -07:00
punkhop 8e867cb8c9 project: Clear deletes a bin it is given, with everything in it
Problem: no command could delete a bin. `project.delete` takes item
ids, and a bin's id passed to it answered ok and changed nothing, so a
bin emptied after an XML import (every clip moved to new bins) stayed
in the project for good.

Solution: `project.delete` also accepts bin ids in `items`. A bin
named there is taken out of the tree with its sub-bins, and every item
inside goes the way a named item goes (out of the project, off every
timeline). The project's own top bin is never removed. The reply now
says how many items and bins went. One undo step, as before.

Test: clear_deletes_a_bin_with_everything_in_it (fails before: the
bins stay).
2026-10-06 06:06:34 -07:00
Brandon ThomasandClaude Opus 5.5 47c6b03681 filmcraft: --help, --version and argument errors show a message box on Windows
Windows release builds have no console (windows_subsystem = "windows", added on main after this branch), so --help printed nothing and an unknown option or a bad FILMCRAFT_CONTROL_PORT quit with exit code 2 and no visible message. The text is still printed, and on Windows release builds also shown in a message box.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:05:59 -04:00
Brandon Thomas 39d81aee14 Merge remote-tracking branch 'origin/main' into pr-47-fix 2026-10-06 08:46:53 -04:00
Brandon ThomasandClaude Opus 5.5 2f49131f64 edit: speed changes refuse NaN and never overflow the timeline
set_speed_group accepted NaN (speed <= 0.0 is false for NaN), and a tiny speed (1e-7) saturated the new duration to i64::MAX, after which the ripple shift overflowed (panic in debug, wraparound in release). Non-finite speeds are refused, the new duration is capped at what the timeline can hold, and the ripple shift uses checked arithmetic; a refused edit changes nothing (it works on a copy). Regression test with hostile speeds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:46:30 -04:00
Peet GuercioandClaude Opus 5.5 56cc93f4ea interchange: an FCP7 master clip lands in its bin when a sequence used it first
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 05:42:03 -07:00
midasdfandClaude Opus 5.5 bab130c358 i18n: Japanese interface language using an installed system font
Edit > Language switches English/Japanese and persists in UI state
(`app.language.english` / `app.language.japanese`, reachable through the
control channel). Core native and in-window menu labels are translated;
untranslated labels stay English. The in-window header fits the document
title beside the localized menus.

No font is bundled. Japanese text uses a Japanese font already installed on
the system (Hiragino, Yu Gothic, Meiryo, Noto Sans CJK JP, Source Han Sans,
IPA, … or any installed face covering the menus' characters), added to every
theme font family as the last fallback. Without one, switching to Japanese is
refused with a message and a saved Japanese setting falls back to English
(the web build cannot read system fonts). Titles already fall back to
installed fonts through the text engine.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:41:17 -04:00
Brandon Thomas 61ab6f5c7d Merge pull request #63 from storytold/pr/release-0.2.1
Release: FilmCraft v0.2.1
2026-10-06 08:01:17 -04:00
Brandon ThomasandClaude Opus 5.5 e659671bdf Release: FilmCraft v0.2.1
Fixes the Windows installer: the Start Menu shortcut and Add/Remove Programs icon were blank, and the shortcut description was wrong. Release builds no longer open a console window behind the app. Export can no longer panic (filmcraft-export never-crash).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:01:04 -04:00
Brandon Thomas bc1523a33f Merge pull request #62 from storytold/pr/never-crash-export
Never crash: filmcraft-export without expect()/unreachable!()
2026-10-06 07:58:18 -04:00
Brandon ThomasandClaude Opus 5.5 136424dc93 Never crash: filmcraft-export without expect()/unreachable!()
The five 'mux created' / 'audio' expects in the stepped exporter now return
an ExportError::Encode instead of panicking if the writer or audio pipeline
is missing, and the non-stepped dispatch returns Unsupported instead of
unreachable!(). The crate root gets the never-crash deny attribute, so every
FilmCraft production crate now carries it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 07:58:07 -04:00
Brandon Thomas 6bf2756cde Merge pull request #61 from storytold/pr/windows-packaging
Windows packaging: Start Menu/Add-Remove icon, no console window, correct description
2026-10-06 07:54:51 -04:00
Brandon ThomasandClaude Opus 5.5 c60d3f9c4c release.yml: explicit if for the wasm-bindgen install (shellcheck SC2015)
packaging-lint has failed on main since this step was added; same behaviour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 07:53:46 -04:00
Brandon ThomasandClaude Opus 5.5 7fee797ee7 Windows packaging: Start Menu/Add-Remove icon, no console window, correct description
The MSI Icon id had no .ico/.exe extension, which Windows Installer needs for
advertised shortcuts and ARPPRODUCTICON, so both showed a blank icon. Release
builds of the desktop app are now GUI-subsystem (no console window; the CLI stays
a console program), the shortcut/summary description names what FilmCraft does, and
packaging-lint rejects an Icon id without an extension. Found in printcraft#57.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 07:51:02 -04:00
Brandon Thomas 508a5a933f Merge pull request #28 from storybored379/pr/mcp-conventions
MCP: tool annotations, core tools, resources, export progress and cancellation
2026-10-06 07:27:58 -04:00
Brandon Thomas 5012fbd598 Merge pull request #19 from storybored379/pr/deterministic-export
Export: same file on every machine (fixed mux interleave, H.264 slices by frame size)
2026-10-06 07:27:44 -04:00
Brandon Thomas 1f7e4f802e Merge pull request #39 from storybored379/fix/ttml-timing
Captions: report invalid TTML timing parameters on import
2026-10-06 07:27:28 -04:00
punkhopandClaude Opus 5.5 a8e4c815eb edit: ripple delete closes one gap for a clip with a split edit
Problem: `edit.rippleDelete` on a clip whose linked sound has a
different edge than its picture (a J or an L cut) treated the two
ranges as two gaps and closed both on every affected track. With two or
more clips after it the later ones were moved up twice and the edit was
refused ("items overlap"). With one clip after it the delete went
through, and sequence markers after it were moved by both gaps (one
that landed inside the second gap was removed).

Solution: one set of stretches closes on every track: the time the
deleted clips covered, less whatever stays on the tracks that lose a
clip. Those tracks move up by the same amount, into time that is free
on all of them, so later clips stay in sync and nothing is overwritten.
For a J or an L cut that is the picture's range. Straight cuts close as
before, also with the sounds on alternating tracks. A track that is
locked loses nothing and does not count. `ripple_delete_items` returns
the stretches it closed and the command ripples the markers by those,
so the two cannot disagree.

Things to know. A gap can stay on one track when a clip that stays on
another of the tracks covers part of the deleted time: a sound shorter
than its picture with other sound right up against it, or unlinked
clips on different tracks deleted in one call. When nothing can close
at all the edit is refused and nothing changes; some such calls went
through before, moving clips by a gap another track did not have. A
selection wholly on locked tracks is refused as locked; before, it
deleted nothing and still shifted the sync-locked tracks.

Tests: an edit-crate test (a split on both sides, two neighbours,
clips apart in time, sounds on alternating tracks, a short sound with
and without neighbours, a locked track, the refusal); engine tests
through `edit.rippleDelete` for a J cut with a marker, undo and redo,
an L cut, two neighbours with a split between them, two neighbours with
sounds on alternating tracks, and a straight cut.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 23:23:27 -07:00
punkhopandClaude Opus 5.5 4a0ab32480 edit: a ripple trim takes a later split edit along
Problem: `timeline.trim` with `mode: "ripple"` on the clip before a
split edit moved the later picture and left its linked sound behind.
With a J cut (the next clip's sound starts before its picture, on a
sync-locked track where the trimmed clip has no partner) a sound that
led its picture by 12 frames led by 20 after lengthening by 8, with no
error. Shortening was refused as a sync-lock conflict, because the
early sound sits in the stretch that closes.

Solution: in `ripple_trim_group`, a clip that starts before the cut,
reaches past it and is linked to a clip that shifts now shifts with it,
on the tracks that ripple (the members' tracks and sync-locked tracks).
It no longer counts as material in the way of a shortening. When it has
no room to follow (another clip right before it, or the sequence start)
the trim is refused with the sync-lock error and nothing changes (an
overlap on any track that ripples now reports that error instead of
the generic "items overlap"). A
linked clip that ends before the cut is left alone, as before, and so
are the trimmed clips' own partners. A transition into a follower from
a clip that stays behind would be left over a gap; it is removed.

Tests: an edit-crate test for longer, shorter, both refusals, the
clip that ends before the cut, the In edge, the trimmed clip's own
partners and the transition; engine tests through `timeline.trim` for
longer and shorter with undo and redo, and for both refusals.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 23:23:27 -07:00
punkhopandClaude Opus 5.5 4df7e75085 engine: timeline.place on an audio track places the sound there
Problem: `timeline.place {item, track: "A3"}` (an audio track given as
`track`). For an item with picture and sound it put a video item on the
audio track A3 and a second, sound clip on the default audio track. For
a sound-only item it ignored `track` and placed on the default audio
track.

Solution: `track` is where the picture goes and `audioTrack` where the
sound goes. An audio track given as `track` now places the sound alone,
on that track. What cannot be meant is a parameter error that says why,
and changes nothing: an audio track as `track` together with
`audioTrack`; a video track as `audioTrack`; an audio track for an item
with no sound; a video track alone for an item with no picture. Calls
that name a video track as `track` and an audio track as `audioTrack`
(the timeline panel's drop does) are unchanged. The parameter hint and
`docs/agents.md` say so.

Tests: track_params_tests.rs places a clip with picture and sound and a
sound-only item on audio tracks by name and by id, checks each refusal
for its error kind, its text, an untouched project and undo history,
and checks the usual forms still work.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 23:14:24 -07:00
punkhopandClaude Opus 5.5 f06bfc4648 edit: a speed change with ripple moves later clips once
Problem: `clip.speedDuration` with `ripple: true` on a clip that has
linked sound set the speed on the picture and then on its sound, and
each one rippled the later material on its own. Slower (50%) moved the
later clips by twice the change and left a double gap. Faster (200%)
was refused ("items overlap") when more than one clip followed: the
second ripple pulled the farther clips onto the nearer ones.

Solution: `set_speed_group` changes a clip and its linked partners on
other tracks as one edit and ripples once, on the members' tracks and
on sync-locked tracks. The command groups the selected clips by link
(one clip per track in a group); clips that are not linked, or linked
clips of one track, still change one after the other. When the members'
lengths change by different amounts (a split edit), later material
moves by the largest change, so no member is overlapped and the tracks
stay in sync; the shorter member is followed by a gap. The group edit
works on a copy, so a failure changes nothing. `set_speed` keeps its
signature and behaviour for one clip.

Tests: an edit-crate test (slower, faster, a split edit both ways, two
clips of one track and a missing clip refused with nothing changed);
engine tests through `clip.speedDuration` for slower with undo and
redo, faster, two clips in one call, no ripple, picture and sound both
named with Linked Selection off, linked clips of one track, a split
edit, and a clip that is not in the sequence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 23:14:24 -07:00
punkhopandClaude Opus 5.5 7eb4f1fc7b engine: sequence.inspect reports a reversed clip
Problem: a clip set to play backward (`clip.speedDuration` with
`reverse: true`) read back from `sequence.inspect` exactly like a
forward clip: same `speed`, same source range, nothing that said it was
reversed. An agent could set it and not confirm it.

Solution: clips gain `reverse` (bool) in `sequence.inspect`, named as
in the project file and in the command. Additive; existing keys are
unchanged. `docs/agents.md` lists it and says what it means for the
source range.

Tests: sequence_inspect_tests.rs reverses a demo clip through the
command and checks that only that clip reports `reverse: true`, and
none after undo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 22:45:21 -07:00
punkhopandClaude Opus 5.5 ff8aa8749d engine: a project saved after an interchange import opens again
Problem: a Final Cut 7 XML whose Basic Motion filter sets only `scale`
makes the importer add the Motion effect with its defaults, where
`position` and `anchor` are the NaN "auto" sentinel. Placing a clip
resolves those with `resolve_auto_points`; the interchange import did
not (any format: the clips are built with the effect defaults).
serde_json writes NaN as `null`, and the reader then refused the file:
"project file is damaged: invalid type: null, expected f64". The project
could be saved but never opened again.

Solution:
- project: a point parameter's value (`ParamValue::Vec2`) reads a `null`
  coordinate back as NaN, so "auto" survives a save. Only there: `Vec2`
  itself is unchanged, and a `null` in a mask vertex or any other point
  is still a damaged file.
- project: `Project::resolve_placed_auto_points` resolves the points of
  the clips in chosen sequences. Points measured in the frame use the
  sequence size. `anchor` is measured in the clip's source
  (`Project::source_size`, a subclip through its parent, as the renderer
  resolves it) and is resolved only for sources the caller says have
  their final size.
- engine: the import resolves in the edits it already makes, so the undo
  steps are what they were. In "Import ...": everything but the anchors
  of file media, whose size is only what the document says so far (an
  EDL says nothing and the importer assumes the sequence size). In each
  "Link Media": the anchors of that file's clips, from the probed size.
  A clip whose file is not found keeps an "auto" anchor, which the
  renderer centres in whatever size the media has, as before this
  change.
- engine: a later Link Media (relink.rs `apply`, behind `media.relink`
  and `media.autoRelink`) resolves those waiting anchors in its own undo
  step, when it has read the file. Nothing else resolves them: a file
  that merely comes back at its old path, or a forced relink that does
  not read the file, leaves the anchor "auto" (still rendered centred).

Why no schema step and no repair on load: the file's shape is not
changed. Files with `null` points were written by every build before
this one, at the current schema version, so a migration (keyed on the
version) cannot select them, and a bump would only make every new
project unreadable to those builds. They open now and their points are
"auto", exactly as in the session that wrote them; resolving them on
load would mean guessing the size of media that was never linked.
docs/project-files.md describes `null` points.

Tests: project
`point_params_keep_auto_through_json_and_no_other_point_reads_null` and
`placed_auto_points_resolve_once_the_source_size_is_final`; format
`auto_points_written_as_null_open_again`; engine
(interchange_auto_points_tests.rs): import with the media missing, save,
reopen, undo, redo; a linked file whose real size differs from the
declared and the sequence size (undo of Link Media and of the import,
redo); an EDL clip smaller than its sequence; import with the media
missing, save, reopen, relink (anchor becomes a number, undo, redo, a
second relink leaves it). All fail without the change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 20:28:54 -07:00
punkhopandClaude Opus 5.5 8f74d18829 engine: timeline.move takes linked partners along
Behaviour change to the default: with Linked Selection on (the default),
`timeline.move` now also moves the linked partners of the listed clips.
A partner lands on its own track in the same mode as the listed clip, so
a caller that lists only a picture clip can now overwrite audio at the
destination time. `"linked": false` gives the old behaviour (only the
listed clips move); the timeline panel passes it, so dragging is
unchanged.

Problem: `timeline.move {"moves": [{clip, track, time}]}` moved only the
listed clip. Its linked sound stayed where it was, out of sync, although
Linked Selection was on and `timeline.trim`, slide and select all act on
linked partners. Separately, a `time` near i64::MAX overflowed the tick
addition in the edit (a panic in debug builds, a wrapped position in
release).

Solution: a linked partner that is not itself in `moves` moves by the
same time offset and stays on its own track. A partner that is listed is
moved once, to where the list says. Two kinds of partner are left
exactly as they are: one on a locked track (the listed clip still moves,
as it did before), and one whose position would not change, which is the
case when the listed clip only changes track. Re-placing such a partner
would drop the transitions at its edges and report a move that did not
happen. A new optional `linked` parameter overrides the toggle for one
call. The command returns `{"moved": [ids]}`, the clips that moved.

No clip is moved before the sequence start. When one would be (a split
edit whose sound starts earlier, or several listed clips asked too far
left), all clips of the call land later by the same amount, so the
spacing asked for is kept. This replaces the per-clip clamp for a call
that lists several clips with a time below zero; a single clip still
lands at zero. The offsets are summed in i128 and checked on the way
back: a start past Tick::MAX is a parameter error and nothing moves.

Tests (timeline_move_tests.rs): partners follow, undo / redo, both
listed, `linked: false`, toggle off, `linked: true`; a locked partner; a
picture moved to another track at the same time leaves its sound and the
sound's crossfade untouched; the common stop at the sequence start with
a split edit and with two listed clips; hostile times (i64::MAX,
i64::MIN, junk). All fail without the change, the hostile one with the
overflow panic. docs/control-protocol.md and docs/agents.md describe the
default, the `linked` parameter and the result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 20:24:23 -07:00
punkhopandClaude Opus 5.5 6de389c75a engine: a command whose targets are named in the parameters does not need a selection
Problem: a command gated on the selection refused to run although the
caller named what to act on. `clip.replaceFromBin {clips, item}` answered
"select a clip in the Project panel"; `edit.clear`, `edit.rippleDelete`,
`edit.pasteAttributes`, `clip.enable`, `clip.link`, `clip.speedDuration`
and `clip.audioGain` with `clips` answered "no clips selected". A script
or agent had to change the user's selection first. The cause is the order
in Session::execute: `enabled()` runs before the parameters are read, and
it only knows the selection.

Fix: Session::execute asks enablement per call. When `enabled()` passes,
nothing changes. When it fails and the parameters name targets under a
key the command documents as ids (`clips` / `clip`: clips of the active
sequence, `items` / `item`: project items), the same `enabled()` runs
once more with those targets standing in for the selection, and the
selections are put back before the command runs. So only the selection
condition is answered by the named targets: no sequence, an empty
clipboard or the wrong kind of clip still disable the command, ids of
nothing do not count, and a command that does not document the key
(`edit.cut`) still needs a selection. This works for every command
because the predicates already resolve their targets the way `run` does
with empty parameters. `is_enabled` (menus, `engine.commands`) is
unchanged.

`clip.audioGain` already read `clips`; its parameter doc now says so.

Tests (explicit_targets_tests.rs): the commands above with and without
parameters, undo/redo, the selections left untouched, menu enablement
unchanged, and the cases that must still be refused. Four of the five
fail without the change in execute.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 20:05:12 -07:00
punkhopandClaude Opus 5.5 4aa3f79953 interchange: FCP7 XML import reports a clip item whose file is not defined
Problem: a clip item whose `<file id="...">` reference has no definition
anywhere in the document was dropped without a word, so a sequence could
come back one clip short with an empty import report. FilmCraft's own
export writes titles and adjustment layers this way.

Solution: the importer adds a warning that names the clip item: `clip
"Title" refers to a file the document does not define; skipped`. Nothing
else about the import changes.

Test: `fcp7_import_reports_a_clip_item_whose_file_is_not_defined`
(FilmCraft's own export, a hand-written document, and a document whose
files are all defined).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 20:02:27 -07:00
punkhopandClaude Opus 5.5 6899f15e51 interchange: an export names every title it cannot carry
Problem: exporting a sequence with a graphic (title) clip to Final Cut
Pro 7 XML returned a report that listed other losses and said nothing
about the title. The XML holds a clip item for it that points at a file
the document never defines, and importing the XML gives the sequence
back without the title. OTIO (a clip with a missing media reference,
read back as offline media) and EDL (an ordinary event) were silent in
the same way; FCPXML said "clip without media skipped" and AAF "graphics,
titles and adjustment layers are not exported" without saying which clip.
Adjustment layers take the same paths.

Solution: every exporter names each graphic clip and adjustment layer
it cannot carry, one warning per clip, with what was written instead:
`graphic clip "Title" at frame 48 has no Final Cut Pro 7 XML equivalent:
it is written as a clip item without media and is not read back on
import`. One helper (`common::uncarried_clip`) builds the name for all
formats. What is written is not changed, byte for byte.

Why only a warning: FCP7 XML has a Text generator, but this crate's
importer does not read one (unknown generators become Black Video), and
a title here is a stack of text and shape layers with per-character
styles, pins and roll settings that the generator's few parameters
cannot hold. Writing one would need a new import path and would still
lose most of a title, so the honest result for now is to say what
happens. crates/interchange/README.md has the table per format.

Tests: crates/interchange/tests/uncarried_clips.rs (each format names
the title and the layer once; two titles are two lines; nothing is said
without such clips; a re-import gives back what the report says), and
the engine's `interchange_export_reports_the_title_it_cannot_carry`
through `file.exportInterchange`. The two naming tests fail without the
change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 20:01:34 -07:00
punkhopandClaude Opus 5.5 5bf2203bb8 codecs: resampled MP4 audio reads apply the edit list once
Problem: a clip with AAC audio in MP4 whose sample rate differs from the
sequence's (48 kHz clip in a 44.1 kHz sequence) exported with loud
low-frequency thumps, several times full scale, recurring through the clip.
Every exported container was affected (AAC clips them into thumps, PCM
clips at full scale); a WAV export had a single one at the clip's start.

Cause: Mp4Source::audio decodes a window starting at s0, the requested
presentation sample minus the edit-list offset, then resamples it. The
resampler derived its sub-sample phase from s0, so the edit-list offset
(-1024 samples of AAC priming) was applied a second time: the read
position started about 1024 samples before the window, was clamped to
index 0 with a large negative fraction, and linear interpolation
extrapolated a ramp over the first ~940 samples of every request. Export
asks for audio one batch of video frames at a time, so the ramp repeated
at each request. Sources at the sequence rate take the copy path and
were unaffected, as were tracks without an edit list.

Fix: take the phase from the presentation sample the window starts at
(before the edit-list offset is subtracted).

Test: aac_mp4_resampled_reads_follow_the_edit_list
(crates/codecs/tests/mp4_oracle.rs, with `require_ffmpeg!` and the
shared fixture helper as docs/testing.md asks of new oracle tests) reads
an ffmpeg AAC fixture at 44.1 kHz in frame-sized pieces and compares it
with the 48 kHz decode interpolated at the same times (peak was 7.35
against a source level of 0.15 before the fix). docs/testing.md lists it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:29:26 -07:00
punkhopandClaude Opus 5.5 d3ee1dfd22 engine: Replace With Clip reports clips that run past the new media
Problem: when the replacement is shorter than the clip's slot, the clip
silently runs past the end of its media and the renderer holds the last
frame. Nothing in the command's result says so, so a script or agent
that swaps a take cannot tell.

Solution: the result of the three Replace With Clip commands
(`clip.replaceFromSource`, `clip.replaceFromSourceMatchFrame`,
`clip.replaceFromBin`) gains `short`: one `{clip, shortByFrames}` per
replaced clip that now plays past the end of its media, in sequence
frames and never more than the clip's own length. It is empty when none
does; stills and other sources without an end, and held frames, are
never short. The edit itself is not changed, and the parameter docs are
untouched (results are described in docs/control-protocol.md).

Test: `replace_with_clip_reports_clips_that_run_past_the_new_media`
(From Bin with a 12-frame subclip, undo / redo, From Source Monitor,
Match Frame, the cases that are not short, and an In past the end of the
new media capped at the clip's length).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:28:20 -07:00
punkhopandClaude Opus 5.5 0e3aebee5d engine: clip.replaceFromBin can keep the clip's source In
Problem: `clip.replaceFromBin` always restarts the clip at the new
item's In point (or its start). That matches Premiere's Replace With
Clip > From Bin, but a script or agent swapping one take for another has
no way to keep the clip's place in the media.

Solution: the default is unchanged, and so is its result. An optional
`keepSourceIn` (default false) keeps each clip's current source In. A
subclip that restricts trims has no media outside its range, so there a
kept In outside the range is moved to the subclip's first or last frame;
with `keepSourceIn` the result gains `sourceInClamped`, the clips this
was done for. The parameter doc lists the new parameter only (a result
description in it would take the command out of the Keyboard Shortcuts
list, which reads the doc to see whether a command needs parameters).

Tests: `replace_from_bin_keeps_the_source_in_on_request` (default and
its result, keepSourceIn, undo / redo, the disabled case, subclips before,
after and around the In, an unrestricted subclip, junk values), and
`list_searches_labels_and_keys` now asserts the three Replace With Clip
commands are bindable. docs/control-protocol.md describes the commands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:25:50 -07:00
punkhopandClaude Opus 5.5 22229df105 filmcraft: --help prints usage and an unknown option is an error
Problem: `filmcraft --help` opened the app. Every argument the parser
did not know was taken as a file to open, so `--help`, `-h` or a
mistyped flag started a window (and tried to import "--help" as media).
`--control` with a missing or invalid port, and an invalid
`FILMCRAFT_CONTROL_PORT`, started the app without the control server and
said nothing, which leaves a script waiting for a server that never
comes.

Solution: argument parsing moves into its own module (`args.rs`,
`args::parse`), which runs before anything else starts. `--help` / `-h`
print the usage (the options the app parses: --control, --demo, --empty,
--recover, --no-recover, --data-dir, --version) and exit 0. An unknown
option, or `--control` / `--data-dir` without a usable value, prints the
error and the usage on stderr and exits 2. The whole line is checked, so
`--version --bogus` is an error too. A `FILMCRAFT_CONTROL_PORT` that is
not a port is the same error as a bad `--control` (exit 2), for the same
reason; an empty value counts as unset, and the value is not looked at
when `--control` is given or for `--help` / `--version`. `--` ends the
options, so a file whose name starts with a dash can still be opened; a
lone `-` and the `-psn_...` argument macOS adds to app launches are not
treated as options.

Tests (in args.rs): `help_and_version_do_not_start_the_app`,
`unknown_options_are_errors`,
`a_bad_port_in_the_environment_is_an_error_like_a_bad_control_flag`,
`known_options_and_files_parse_as_before`. docs/contributing.md
mentions `--help` and the errors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:21:56 -07:00
punkhopandClaude Opus 5.5 285da5c66b engine: file.exportInterchange refuses an unknown format
Problem: `file.exportInterchange {"format": "<anything unknown>"}` wrote
FCP7 XML and returned ok, so a typo or an unsupported format produced a
file of the wrong kind under the name that was asked for.

Solution: only a missing `format` means FCP7 XML. A given format must be
one of the six documented names (edl, xml, fcpxml, otio, aaf, omf: each
format's extension, in either case). Anything else, including a value
that is not a string, is a parameter error that lists exactly those
names, and nothing is written. No other spelling is accepted (".xml",
"fcpxmld", "omfi"): none was accepted before either, they silently
became XML. The parameter doc now lists aaf and omf, which the command
already took, and the default; docs/architecture.md says the same.

Test: `export_interchange_refuses_an_unknown_format`, next to the
interchange round-trip test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:20:02 -07:00
punkhopandClaude Opus 5.5 51ecd70763 engine: sequence.inspect reports clip ends, gain and marker comments
Problem: `sequence.inspect` is how an agent verifies an edit, but it
left out where a clip ends in the sequence and in its media, a clip's
audio gain, and the comment of a marker, so those edits could not be
checked from the readback.

Solution: each clip also reports `end`, `endFrame`, `sourceOut` (which
depends on speed, so it cannot be derived from `sourceIn` + `duration`)
and `gainDb`; each sequence marker also reports `comment`. Every
existing key stays. docs/agents.md lists what the readback holds.

Test: `sequence_inspect_reports_clip_ends_gain_and_marker_comments`
(sequence_inspect_tests.rs), on a clip at 200 %, a clip with gain and a
marker with a comment made through the commands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:17:52 -07:00
punkhopandClaude Opus 5.5 9d3975a697 engine: naming a track that does not exist is an error
Problem: `timeline.place {"audioTrack": "A2"}` on a sequence that has
only A1 placed the clip on A1, overwrote what was there and returned ok.
`track_p` returned `None` both for "no track asked for" and for "that
track does not exist", so callers fell back to the source-patched track.
It also took any first letter other than V as an audio track, accepted
track ids that are not in the sequence, and `split_at(1)` panicked on an
empty or non-ASCII name.

Solution: `track_p` returns `Result<Option<TrackId>>` and takes the
command id, like the other fallible parameter helpers. An absent (or
null) parameter is `Ok(None)` and keeps every default as it was. A name
or id the active sequence does not have is a parameter error
(`EngineError::BadParams`): "invalid parameters for `timeline.place`: no
track A2 in this sequence (`audioTrack`)". All callers propagate it:
timeline.place (`track`, `audioTrack`), timeline.move, timeline.razor,
timeline.setTrack, timeline.setTargeting, sequence.closeGap,
sequence.deleteTrack, sequence.deleteTracks, the go-to-gap commands,
sequence.transcribe and the multicam sync reference track.

Not changed: a `timeline.move` entry without a `clip`, `track` or `time`
is still skipped, as before; only a track that was named and is missing
is an error.

Test: `naming_a_missing_track_is_an_error_not_a_default` in
track_params_tests.rs (missing and hostile names, the error kind, nothing
edited, no undo step, existing tracks by name and id still resolve).
docs/agents.md notes the rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:10:19 -07:00
justjohn12345andClaude Opus 5.5 4f3e119ac2 MEM4: decode into the planes of evicted frames
With MEM1-MEM3 twelve clips on a timeline still took 5.1 GB, only 2.5 GB of
it live: each decoded picture allocated three planes, each eviction freed
them, and the system allocator kept the freed pages.

filmcraft_frame::pool keeps the planes of frames the GOP caches evict (up to
192 MiB per sample type; a frame something else still holds is left alone).
The H.264 decoder, through a plane allocator hook that keeps the crate free
of dependencies, and the VideoToolbox path take their output planes from it.

perf.stats reports decode.planePoolMB and decode.planesReused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:57:16 -04:00
Brandon ThomasandClaude Opus 5.5 5d1165d06b MCP: cancelled exports delete only their own files
remove_partial removed every '<stem>[digits].<any ext>' file in the output folder modified since the export started, so cancelling an export to trailer.mp4 also deleted trailer.aep or trailer2.psd if they were saved during the export. It now removes only the output file, numbered stills with the output's extension, and the .srt/.vtt caption sidecar. Regression test with decoy files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:54:24 -04:00
justjohn12345andClaude Opus 5.5 b999b69f07 HW1: register the hardware decoders in every desktop run
The desktop app called filmcraft_platform::register() as an argument of
log::info!. A log macro does not evaluate its arguments while no logger takes
its level, and the app installs none, so VideoToolbox was never registered:
perf.stats showed hardware decoding enabled with 0 sessions, 0 declined and
every frame decoded in software. The CLI and the bench register directly and
were not affected.

Registration is now a statement of its own, with a start-up test that runs
without a logger.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:42:28 -04:00
justjohn12345andClaude Opus 5.5 4486652f07 MEM3: one frame budget shared by all sources
Each GOP cache keeps up to 384 MB of decoded frames (at least 64), and
nothing bounded the total: twelve clips on a timeline held 3.8 GB of frames
nobody was looking at, and no perf.stats counter showed it.

The pool now counts the frames of every cache against a 1 GiB budget. Over
it, caches idle for 3 s give up frames, least recently used cache first.
Caches in use keep their own budget, since a frame evicted before it is
shown costs a re-decode from the keyframe.

perf.stats reports decode.cacheMB and decode.cacheBudgetMB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:39:48 -04:00
justjohn12345andClaude Opus 5.5 517b6b041b MEM2: idle sources give up their decoders beyond a cap
Every source kept its decoder for good once it had decoded a frame, with its
reference pictures and the pictures its frame threads hold in flight: about
290 MB per H.264 clip, 3.5 GB for twelve clips on a timeline.

The GOP caches now share a pool. Beyond four live decoders, the least
recently used caches that have been idle for 3 s drop theirs and make a new
one when asked again (a restart at the sync sample, as after a seek). Caches
in use keep their decoder whatever the count, so a composite of many layers
never restarts decoders per frame. Other caches are only try-locked, so two
caches trimming each other cannot deadlock.

perf.stats reports decode.liveDecoders.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:39:48 -04:00
justjohn12345andClaude Opus 5.5 ac8279d293 MEM1: desktop reads media files in place
The desktop host had no Services::reader, so MediaPool::open_file read every
clip whole and kept its bytes for as long as the source stayed open: 2.9 GB
for twelve clips on a timeline (malloc_history, 15 allocations in open_file).

FsServices::reader now returns a FileReader that keeps the file open and
serves positional reads, so MP4/MOV, Matroska and the other reader openers
load the index and then single samples, as the web host already does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:39:48 -04:00
Brandon Thomas ada55eb625 Merge pull request #35 from storytold/pr/gpu-effects
GPU2: standard effects on the GPU compositor (#30)
2026-10-05 16:27:05 -04:00
Brandon ThomasandClaude Opus 5.5 adfd9a66de docs: point agents to the shared test corpora (#36)
Real-file test oracles live in storytold/photocraft-corpus, explained in
craftrules standards/test-corpora.md. Never commit large binary fixtures
here; fetch them pinned and sha256-verified.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 08:24:17 -04:00
storybored379andOpenAI Codex 9c06722119 Captions: report invalid TTML timing parameters on import
Co-Authored-By: OpenAI Codex <noreply@openai.com>
2026-10-04 21:48:25 -07:00
storybored379andClaude Opus 5.5 cad16825e3 MCP: progress notifications and cancellation for blocking exports
command_run file.exportMedia with wait: true (headless) runs as the
background export job and uses the MCP progress and cancellation utilities:

- with _meta.progressToken, notifications/progress reports frames done /
  total with the job status, at most every 100 ms (none without a token);
- notifications/cancelled stops the job at the next batch (jobs.cancel),
  deletes the partial output and sends no response;
- the session lock is only held for short polls, so ping, doc_inspect,
  jobs.list etc. are answered while it encodes.

Without wait, and in bridge mode, nothing changes. Test
export_progress_and_cancel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 13:51:38 -07:00
storybored379andClaude Opus 5.5 8ee7658303 MCP: filmcraft://document and filmcraft://commands resources, with MCP 2026-07-28 cache hints
- resources/list and resources/read serve the project (tree + active
  sequence, as doc_inspect) and the command catalog (as command_list) as
  JSON, for clients that attach context from resources.
- Results are built with rmcp's types, so resultType follows the negotiated
  revision; for 2026-07-28 clients (current Claude Code) they also carry
  ttlMs and cacheScope, like the generated tools/list. The resource list is
  fixed (10 min), document reads are never cached (0).
- get_info now declares the resources capability; the instructions text
  moves to a const, unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 13:51:28 -07:00
storybored379andClaude Opus 5.5 0f70015b93 MCP: tool titles and annotations, command_batch / doc_inspect / render_preview, strict arguments
- Every tool has a title and readOnly / destructive / idempotent / openWorld
  hints, set in its #[tool] attribute, so clients can run the read-only
  tools without asking and ask before edits.
- New tools: command_batch (several commands, stop at the first error by
  default), doc_inspect (project tree + active sequence in one call) and
  render_preview (= render_frame).
- An unknown tool argument is a -32602 error naming it and the accepted
  ones, instead of being ignored.
- A line that is not JSON gets a -32700 reply (id null) instead of being
  dropped without an answer: serve_io filters lines in front of rmcp.
- A panicking command no longer poisons the session lock for the rest of the
  session ("session lock poisoned"); it is reported as "internal error: …"
  (AGENTS.md §0.6).
- docs/agents.md § Conventions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 13:51:21 -07:00
storybored379andClaude Opus 5.5 a55d20c454 export: same file on every machine (fixed mux interleave, H.264 slices by frame size)
Exporting the same project twice could give different files, because two
export parameters followed the number of cores the process sees:

- The muxer got one chunk per render batch, and a batch is one frame per
  rayon thread. The chunk layout (stsc/stco, PCM packet sizes) changed with
  the core count; the samples themselves did not.
- The H.264 encoder used one slice per core (capped at one per four MB rows),
  so the bitstream and every decoded picture changed with the core count.

On a host whose CPU quota changes (containers, power limits), two exports in a
row differed. Now packets are muxed in fixed groups of 16 output frames
(INTERLEAVE), each followed by its audio, whatever the batch size, and H.264
exports use one slice per four macroblock rows. Hosts with 17 or more threads
already used these values, so their 1080p output does not change.

Tests: determinism_tests (fail on main): same bytes for ProRes, H.264 and
Motion-JPEG at 2/3/5/16 cores and 1/7 frames per step; 64 slices for a
4096-line H.264 export.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 08:32:49 -07:00
454 changed files with 60647 additions and 5066 deletions

No files matched your search

+58
View File
@@ -0,0 +1,58 @@
# Builds FilmCraft on FreeBSD, inside a FreeBSD VM on a Linux runner (GitHub has no FreeBSD
# runners), then runs packaging/freebsd/package.sh, the script the release job runs, and checks
# the tarball. A broken FreeBSD build or package fails here before it can fail a release. Nothing
# is uploaded and no secrets are used. (The test suite isn't run here yet: the ffmpeg-oracle and
# golden-image tests need tools and a GPU the VM doesn't have.)
name: FreeBSD
# The VM build takes an hour or more, so it runs nightly on main rather than on every push or
# dependency change: a drifted dependency still shows up within a day. PRs that change the
# FreeBSD packaging itself still run it.
on:
schedule:
- cron: "37 2 * * *"
pull_request:
paths:
- .github/workflows/freebsd.yml
- packaging/freebsd/**
- packaging/env.sh
workflow_dispatch:
concurrency:
group: freebsd-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
freebsd:
name: FreeBSD 14 (build + package)
runs-on: ubuntu-latest
timeout-minutes: 240
steps:
- uses: actions/checkout@v4
- name: Build and package in a FreeBSD VM
# vmactions/freebsd-vm v1.5.9, pinned by commit (it runs our build).
uses: vmactions/freebsd-vm@c46abacb49f09938ca4e1702d15d836285d694cc
with:
release: "14.3"
usesh: true
cpu: 4
mem: 12288
# Nothing to bring back; copying the target dir out of the VM fails on large builds.
copyback: false
# The same packages as release.yml's freebsd job: winit/wgpu (X11, Wayland, Vulkan,
# Mesa), the file dialogs (GTK 3), fonts, and audio (cpal over ALSA: alsa-lib).
prepare: |
pkg install -y bash git curl pkgconf cmake python3 libxkbcommon wayland libX11 libXcursor libXrandr libXi libxcb mesa-libs vulkan-loader gtk3 fontconfig freetype2 alsa-lib
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain stable
run: |
set -e
. "$HOME/.cargo/env"
git config --global --add safe.directory "*"
rustc --version
export CARGO_TARGET_DIR=/var/tmp/filmcraft-target CARGO_BUILD_JOBS=4
bash packaging/freebsd/package.sh
tar -tzf dist/release/filmcraft-*-freebsd-x86_64.tar.gz \
| grep -E '/bin/filmcraft$|/bin/filmcraft-cli$|/share/applications/ai\.storyteller\.filmcraft\.desktop$|/share/doc/filmcraft/LICENSE-MIT$'
+18 -1
View File
@@ -35,6 +35,8 @@ jobs:
"$RUNNER_TEMP/actionlint" -color
- name: shellcheck
run: shellcheck packaging/*.sh packaging/*/*.sh
- name: Linux tarball installer
run: python3 packaging/linux/test_install.py
- name: PowerShell
shell: pwsh
run: |
@@ -47,7 +49,22 @@ jobs:
- name: XML, desktop entry, AppStream, Flatpak manifest
run: |
xmllint --noout packaging/windows/filmcraft.wxs packaging/macos/*.plist* packaging/linux/*.xml*
# Windows Installer needs Icon ids ending in .ico or .exe, or shortcuts show a blank icon.
if grep -oE '<Icon Id="[^"]*"' packaging/windows/filmcraft.wxs | grep -vE '\.(ico|exe)"$'; then
echo "::error::Icon Id must end in .ico or .exe"; exit 1
fi
desktop-file-validate packaging/linux/ai.storyteller.filmcraft.desktop
sed -e 's/@VERSION@/0.0.0/' -e 's/@DATE@/2026-01-01/' packaging/linux/ai.storyteller.filmcraft.metainfo.xml.in >"$RUNNER_TEMP/ai.storyteller.filmcraft.metainfo.xml"
appstreamcli validate --no-net --explain "$RUNNER_TEMP/ai.storyteller.filmcraft.metainfo.xml"
python3 -c "import yaml; m = yaml.safe_load(open('packaging/linux/flatpak/ai.storyteller.filmcraft.yml')); assert m['id'] == 'ai.storyteller.filmcraft' and m['command'] == 'filmcraft'"
python3 - <<'EOF'
import yaml
src = yaml.safe_load(open("packaging/linux/flatpak/ai.storyteller.filmcraft.yml"))
bundle = yaml.safe_load(open("packaging/linux/flatpak/ai.storyteller.filmcraft.bundle.yml"))
for m in (src, bundle):
assert m["id"] == "ai.storyteller.filmcraft" and m["command"] == "filmcraft", m["id"]
for key in ("runtime", "runtime-version", "sdk", "finish-args"):
assert src[key] == bundle[key], f"Flatpak manifests disagree on {key}: {src[key]} vs {bundle[key]}"
print("Flatpak manifests ok")
EOF
- name: FreeBSD package layout (stub binaries)
run: bash packaging/freebsd/package.sh --dry-run
+144 -11
View File
@@ -1,9 +1,12 @@
# Release pipeline: every push to `release` builds signed installers for every platform and
# creates or updates the draft GitHub Release "FilmCraft v<version>". See docs/releasing.md.
#
# All jobs run in the `release` environment (restricted to the `release` branch), which holds
# the signing secrets. Every secret is optional: missing ones produce unsigned artifacts and a
# warning, never a failed build.
# The jobs that sign (macOS, Windows) and the one that publishes the draft run in the `release`
# environment (restricted to the `release` branch), which holds the signing secrets. Every secret
# is optional: missing ones produce unsigned artifacts and a warning, never a failed build. The
# jobs that sign nothing (version, Linux, Flatpak, FreeBSD, web) get no environment and no
# secrets, so a dispatch on any other branch dry-runs them; there the signing jobs are refused,
# so the draft-release job (which needs them) is skipped and nothing is published.
name: Release
on:
@@ -35,7 +38,6 @@ jobs:
version:
name: version
runs-on: ubuntu-latest
environment: release
outputs:
version: ${{ steps.v.outputs.version }}
date: ${{ steps.v.outputs.date }}
@@ -63,13 +65,19 @@ jobs:
macos:
name: macOS universal
needs: version
runs-on: macos-15
runs-on: macos-15-xlarge # M2 Pro larger runner; use only in release.yml (actions_macos_xl budget caps spend)
environment: release
env:
FILMCRAFT_VERSION: ${{ needs.version.outputs.version }}
FILMCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@v4
# Fonts (optional build input, required for releases): docs/contributing.md#building-with-craft-fonts
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
with:
targets: aarch64-apple-darwin, x86_64-apple-darwin
@@ -90,6 +98,8 @@ jobs:
APPLE_ID: ${{ secrets.APPLE_ID }}
APPLE_PASSWORD: ${{ secrets.APPLE_PASSWORD }}
APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }}
CRAFT_FONTS_DIR: ${{ github.workspace }}/craft-fonts
CRAFT_FONTS_REQUIRED: "1"
run: packaging/macos/package.sh --arch universal
- uses: actions/upload-artifact@v4
with:
@@ -112,11 +122,20 @@ jobs:
include:
- { arch: x64, target: x86_64-pc-windows-msvc }
- { arch: x86, target: i686-pc-windows-msvc }
# Windows on ARM: cross-compiled on the x64 runner, so signing and WiX run as above.
# .github/workflows/windows-arm64.yml installs and runs it on ARM64 hardware.
- { arch: arm64, target: aarch64-pc-windows-msvc }
env:
FILMCRAFT_VERSION: ${{ needs.version.outputs.version }}
FILMCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@v4
# Fonts (optional build input, required for releases): docs/contributing.md#building-with-craft-fonts
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
with:
targets: ${{ matrix.target }}
@@ -141,6 +160,8 @@ jobs:
AZURE_SIGNING_ENDPOINT: ${{ secrets.AZURE_SIGNING_ENDPOINT }}
AZURE_SIGNING_ACCOUNT: ${{ secrets.AZURE_SIGNING_ACCOUNT }}
AZURE_CERT_PROFILE: ${{ secrets.AZURE_CERT_PROFILE }}
CRAFT_FONTS_DIR: ${{ github.workspace }}/craft-fonts
CRAFT_FONTS_REQUIRED: "1"
run: ./packaging/windows/package.ps1 -Arch ${{ matrix.arch }}
- uses: actions/upload-artifact@v4
with:
@@ -154,7 +175,7 @@ jobs:
# The oldest GitHub-hosted image, so the binaries only need glibc >= 2.35 (Ubuntu 22.04+,
# Debian 12+, Fedora 36+, RHEL 10). See docs/releasing.md.
runs-on: ${{ matrix.runner }}
environment: release
# No `environment: release`: this job signs nothing, so it gets no release secrets.
strategy:
fail-fast: false
matrix:
@@ -167,12 +188,18 @@ jobs:
NFPM_VERSION: 2.47.0
steps:
- uses: actions/checkout@v4
# Fonts (optional build input, required for releases): docs/contributing.md#building-with-craft-fonts
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
path: craft-fonts
- name: Install build and packaging tools
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
libxkbcommon-dev libwayland-dev libx11-dev libxrandr-dev libxi-dev libgl1-mesa-dev libasound2-dev \
desktop-file-utils appstream python3-yaml
desktop-file-utils appstream python3-yaml zsync
curl -fsSL -o "$RUNNER_TEMP/nfpm.deb" \
"https://github.com/goreleaser/nfpm/releases/download/v${NFPM_VERSION}/nfpm_${NFPM_VERSION}_${{ matrix.deb }}.deb"
sudo dpkg -i "$RUNNER_TEMP/nfpm.deb"
@@ -184,6 +211,9 @@ jobs:
if: inputs.version != ''
run: cargo xtask version set "$FILMCRAFT_VERSION"
- name: Build and package (AppImage, deb, rpm, tar.gz)
env:
CRAFT_FONTS_DIR: ${{ github.workspace }}/craft-fonts
CRAFT_FONTS_REQUIRED: "1"
run: packaging/linux/package.sh
- name: Check the packages
run: |
@@ -194,6 +224,9 @@ jobs:
for appimage in dist/release/*.AppImage; do
chmod +x "$appimage"
APPIMAGE_EXTRACT_AND_RUN=1 "$appimage" --version
# Update information is embedded, and the .zsync for delta updates sits beside it.
"$appimage" --appimage-updateinformation | grep -F 'gh-releases-zsync|'
test -s "$appimage.zsync"
done
- uses: actions/upload-artifact@v4
with:
@@ -201,17 +234,113 @@ jobs:
path: dist/release/*
if-no-files-found: error
flatpak:
name: Flatpak ${{ matrix.arch }}
needs: [version, linux]
# Repackages the Linux job's tarball (the same binaries as the AppImage/deb/rpm); no Rust
# build here. 24.04 for a newer flatpak-builder (1.4) than 22.04 ships.
runs-on: ${{ matrix.runner }}
# No `environment: release`: this job signs nothing, so it gets no release secrets.
strategy:
fail-fast: false
matrix:
include:
- { arch: x86_64, runner: ubuntu-24.04 }
- { arch: aarch64, runner: ubuntu-24.04-arm }
env:
FILMCRAFT_VERSION: ${{ needs.version.outputs.version }}
FILMCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
with:
name: release-linux-${{ matrix.arch }}
path: dist/linux
- name: Install flatpak-builder
run: |
sudo apt-get update
# librsvg2-common: the SVG pixbuf loader. flatpak-builder runs the host's
# `appstreamcli compose`, which renders the scalable icon and fails without it.
sudo apt-get install -y --no-install-recommends flatpak flatpak-builder librsvg2-common
# Ubuntu 24.04's AppArmor blocks the unprivileged user namespaces bubblewrap needs.
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
- name: Build the Flatpak bundle
run: packaging/linux/flatpak-bundle.sh "dist/linux/filmcraft-$FILMCRAFT_VERSION-linux-${{ matrix.arch }}.tar.gz"
- uses: actions/upload-artifact@v4
with:
name: release-flatpak-${{ matrix.arch }}
path: dist/release/*.flatpak
if-no-files-found: error
freebsd:
name: FreeBSD x86_64
needs: version
# GitHub has no FreeBSD runners: build in a FreeBSD 14.3 VM, the same image and packages as
# .github/workflows/freebsd.yml.
runs-on: ubuntu-latest
timeout-minutes: 180
# No `environment: release`: this job signs nothing, so it gets no release secrets.
env:
FILMCRAFT_VERSION: ${{ needs.version.outputs.version }}
FILMCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@v4
# Fonts (optional build input, required for releases): docs/contributing.md#building-with-craft-fonts
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
path: craft-fonts
- name: Build and package in a FreeBSD VM (tar.gz)
# vmactions/freebsd-vm v1.5.9, pinned by commit (it runs our build).
uses: vmactions/freebsd-vm@c46abacb49f09938ca4e1702d15d836285d694cc
with:
release: "14.3"
usesh: true
cpu: 4
mem: 12288
# Passed into the VM by name; never templated into the script below.
envs: "FILMCRAFT_VERSION FILMCRAFT_BUILD_DATE FILMCRAFT_BUILD_SHA INPUT_VERSION"
# The FreeBSD CI packages (.github/workflows/freebsd.yml), plus bash for the script.
prepare: |
pkg install -y bash git curl pkgconf cmake python3 libxkbcommon wayland libX11 libXcursor libXrandr libXi libxcb mesa-libs vulkan-loader gtk3 fontconfig freetype2 alsa-lib
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain stable
run: |
set -e
. "$HOME/.cargo/env"
git config --global --add safe.directory "$PWD"
# Build outside the checkout so only dist/ is copied back out of the VM (copying the
# target dir back fails on large builds). 4 jobs: more runs the VM out of memory.
export CARGO_TARGET_DIR=/var/tmp/filmcraft-target CARGO_BUILD_JOBS=4
if [ -n "$INPUT_VERSION" ]; then cargo xtask version set "$FILMCRAFT_VERSION"; fi
# craft-fonts was checked out next to the sources and synced into the VM with them.
export CRAFT_FONTS_DIR="$PWD/craft-fonts" CRAFT_FONTS_REQUIRED=1
bash packaging/freebsd/package.sh
tar -tzf dist/release/filmcraft-*-freebsd-x86_64.tar.gz \
| grep -E '/bin/filmcraft$|/bin/filmcraft-cli$|/share/applications/ai\.storyteller\.filmcraft\.desktop$|/share/doc/filmcraft/LICENSE-MIT$'
- uses: actions/upload-artifact@v4
with:
name: release-freebsd-x86_64
path: dist/release/*
if-no-files-found: error
web:
name: Web (wasm)
needs: version
runs-on: ubuntu-latest
environment: release
# No `environment: release`: this job signs nothing, so it gets no release secrets.
env:
FILMCRAFT_VERSION: ${{ needs.version.outputs.version }}
FILMCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
WASM_BINDGEN_VERSION: 0.2.129
steps:
- uses: actions/checkout@v4
# Fonts (optional build input, required for releases): docs/contributing.md#building-with-craft-fonts
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-unknown-unknown
@@ -221,13 +350,17 @@ jobs:
- name: Install wasm-bindgen CLI
run: |
# Must match the wasm-bindgen crate version exactly (WASM_BINDGEN in xtask/src/main.rs).
command -v wasm-bindgen >/dev/null && wasm-bindgen --version | grep -q "$WASM_BINDGEN_VERSION" \
|| cargo install wasm-bindgen-cli --version "$WASM_BINDGEN_VERSION" --locked
if ! { command -v wasm-bindgen >/dev/null && wasm-bindgen --version | grep -q "$WASM_BINDGEN_VERSION"; }; then
cargo install wasm-bindgen-cli --version "$WASM_BINDGEN_VERSION" --locked
fi
wasm-bindgen --version
- name: Apply version override
if: inputs.version != ''
run: cargo xtask version set "$FILMCRAFT_VERSION"
- name: Build and zip
env:
CRAFT_FONTS_DIR: ${{ github.workspace }}/craft-fonts
CRAFT_FONTS_REQUIRED: "1"
run: packaging/web/package.sh
- uses: actions/upload-artifact@v4
with:
@@ -237,7 +370,7 @@ jobs:
release:
name: Draft GitHub Release
needs: [version, macos, windows, linux, web]
needs: [version, macos, windows, linux, flatpak, freebsd, web]
runs-on: ubuntu-latest
environment: release
permissions:
+95
View File
@@ -0,0 +1,95 @@
# Windows on ARM: package the ARM64 release the way release.yml does (cross-compiled on an x64
# runner, unsigned here), then install that MSI on a Windows 11 ARM64 runner, check the installed
# binaries are ARM64, run the CLI natively, and uninstall. (A native ARM64 `cargo test` job is left
# out for now: PdfCraft saw an intermittent access violation in egui UI tests there.)
name: Windows ARM64
on:
push:
branches: [main]
pull_request:
paths:
- Cargo.toml
- Cargo.lock
- "**/Cargo.toml"
- "**/build.rs"
- .github/workflows/windows-arm64.yml
- packaging/windows/**
schedule:
- cron: "41 4 * * 1" # weekly, so drift in the runner images shows up
workflow_dispatch:
concurrency:
group: windows-arm64-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
jobs:
package:
name: package (cross-compiled on x64)
runs-on: windows-latest
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@89b12181fb390509a0842a86cc55eeb8eb928c1d # stable
with:
targets: aarch64-pc-windows-msvc
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
key: windows-arm64-package
- name: Install WiX
shell: pwsh
run: dotnet tool install --global wix --version 5.0.2
- name: Build and package (unsigned)
shell: pwsh
env:
DIST: ${{ github.workspace }}\dist\arm64
run: ./packaging/windows/package.ps1 -Arch arm64
- uses: actions/upload-artifact@v4
with:
name: filmcraft-windows-arm64
path: dist/arm64/*
if-no-files-found: error
retention-days: 7
install:
name: install and run on ARM64
needs: package
runs-on: windows-11-arm
timeout-minutes: 30
steps:
- uses: actions/download-artifact@v4
with:
name: filmcraft-windows-arm64
path: dist
- name: Install the MSI, check it, run it, uninstall it
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'
$msi = (Get-ChildItem dist/*.msi | Select-Object -First 1).FullName
$p = Start-Process msiexec.exe -ArgumentList '/i', "`"$msi`"", '/qn', '/l*v', 'install.log' -Wait -PassThru
if ($p.ExitCode -ne 0) { Get-Content install.log -Tail 60; throw "msiexec /i exited $($p.ExitCode)" }
$dir = Join-Path $env:ProgramFiles 'FilmCraft'
foreach ($exe in 'filmcraft.exe', 'filmcraft-cli.exe') {
$path = Join-Path $dir $exe
if (-not (Test-Path $path)) { throw "$exe not installed in $dir" }
$bytes = [System.IO.File]::ReadAllBytes($path)
$machine = [BitConverter]::ToUInt16($bytes, [BitConverter]::ToInt32($bytes, 0x3C) + 4)
if ($machine -ne 0xAA64) { throw "$exe is for machine 0x$('{0:X}' -f $machine), not ARM64" }
Write-Output "ok $path (ARM64)"
}
# The installed CLI runs natively on ARM64.
$cli = Join-Path $dir 'filmcraft-cli.exe'
$out = & $cli --version 2>&1 | Out-String
if ($LASTEXITCODE -ne 0) { Write-Output $out; throw "filmcraft-cli --version exited $LASTEXITCODE" }
Write-Output "ok filmcraft-cli --version: $($out.Trim())"
$p = Start-Process msiexec.exe -ArgumentList '/x', "`"$msi`"", '/qn' -Wait -PassThru
if ($p.ExitCode -ne 0) { throw "msiexec /x exited $($p.ExitCode)" }
if (Test-Path (Join-Path $dir 'filmcraft.exe')) { throw 'uninstall left filmcraft.exe behind' }
Write-Output 'ok uninstalled'
+33 -1
View File
@@ -21,7 +21,7 @@ panic path, and fix a crash before building on top of it. The cross-app standard
code alone (a literal that always parses): a single-item `#[allow(clippy::expect_used)]` with the reason.
3. **No `unsafe`** (`unsafe_code = "forbid"` for the workspace). The single exception is
`crates/platform`, which needs `unsafe` to call the operating system's media APIs (hardware
video decoding through VideoToolbox; later encoding and other OSes) and does nothing else. It
video decoding through VideoToolbox, Media Foundation / Direct3D 11 and VA-API; hardware encoding) and does nothing else. It
uses `unsafe_code = "deny"` with `#[allow(unsafe_code)]` only on its FFI modules, a
`// SAFETY:` comment on every `unsafe` block, a safe `Result`-returning public API, and the
pure-Rust decoder as the tested fallback ([ADR 0001](docs/adr/0001-platform-ffi.md)).
@@ -92,6 +92,13 @@ reproduce an image (for example, point lists traced from someone else's icon).
need a sidecar and an `ATTRIBUTION.md` row (licence `LicenseRef-ArtCraft-Trademark`). No other
non-open asset is allowed, and this exception never covers third-party marks (Adobe, Discord, GitHub
and other logos stay out; draw a generic icon instead).
9. **Fonts live in [storytold/craft-fonts](https://github.com/storytold/craft-fonts), never in this
repo.** Don't commit new font files (the small Latin UI fonts already in `assets/fonts/` stay). A
font FilmCraft needs is added to craft-fonts, which the app reads through the optional build input
`CRAFT_FONTS_DIR=<absolute path of a craft-fonts checkout>` (`crates/text/build.rs` embeds the fonts in its
`fonts/manifest.txt` as `filmcraft_text::fonts::CRAFT_FONTS`; unset, it is empty and the app uses its
bundled and system fonts). Never add craft-fonts to a `Cargo.toml`. Code using `CRAFT_FONTS` must work
when it is empty. Standard: [`craftrules/standards/fonts.md`](https://github.com/storytold/craftrules/blob/main/standards/fonts.md).
## 2. Clean-room code
@@ -120,6 +127,13 @@ See [§0](#0-never-crash). It outranks every other engineering rule.
See `CLAUDE.md`: pure Rust, dependency layering (`cargo xtask layers`), exact `Tick` time, everything is
a command, everything is agent-drivable, and the quality gates (`cargo xtask ci`) before every commit.
Shared real-file test corpora (Photoshop-authored PSDs, etc.) live in
[`storytold/photocraft-corpus`](https://github.com/storytold/photocraft-corpus), explained in
[craftrules `standards/test-corpora.md`](https://github.com/storytold/craftrules/blob/main/standards/test-corpora.md).
Never commit large binary fixtures to this repo; fetch them pinned by commit and sha256-verified,
as PhotoCraft does with `cargo xtask corpus`.
## See also
- [CONTRIBUTING.md](CONTRIBUTING.md) and [docs/contributing.md](docs/contributing.md): setup, gates, commits, how to add things
@@ -128,3 +142,21 @@ a command, everything is agent-drivable, and the quality gates (`cargo xtask ci`
- [docs/agents.md](docs/agents.md): driving FilmCraft over MCP / the control channel, and the agent work loop
- [docs/control-protocol.md](docs/control-protocol.md): control-channel method reference
- [ATTRIBUTION.md](ATTRIBUTION.md): asset index
## Contributor credits (About window)
- About ▸ Contributors/Models are compiled into the binary from `contributors/contributors.json`
(commit stats; generated, never hand-edit) and `contributors/people.toml` (names people chose for
themselves). See `docs/contributors.md`.
- **Agents working for a contributor:** when you prepare a PR, check whether your human's GitHub
username has a `[people.<username>]` entry in `contributors/people.toml`. If not, ask them once
whether they want to be credited by more than their username: a real name, a display name, and/or
their public GitHub profile name (`sync_github_name = true`). If yes, add **only their own** entry
(copy the template at the top of the file, or run
`python3 ../../craftrules/scripts/contributors.py --add-me . --real-name "…" --sync-github-name`)
and include it in their PR, committed as them. If no, change nothing: they are credited as
`@username` anyway.
- Never add, edit, guess or copy anyone else's entry or name (not from git config, commit authors or
GitHub profiles). Never hand-edit `contributors.json`.
- Maintainers refresh the stats with `python3 ../../craftrules/scripts/contributors.py .` (it also
re-verifies who wrote each `people.toml` entry; `--check` only verifies).
+27
View File
@@ -26,6 +26,8 @@ iconography, images or other Adobe assets; open licences only; every asset attri
| `docs/images/filmcraft-effects.png` | FilmCraft contributors | Original work: FilmCraft screenshot crop; NASA Earth Views (US Government work, public domain) | MIT OR Apache-2.0; footage public domain |
| `docs/images/filmcraft-timeline.png` | FilmCraft contributors | Original work: FilmCraft UI crop; thumbnails from Night of the Living Dead (1968) and Carnival of Souls (1962), CC0 Chopin waveform | MIT OR Apache-2.0; film frames public domain |
| `assets/app-icon/filmcraft.svg` | Brandon Thomas (FilmCraft owner) | Original work: FilmCraft app icon (owl with ear tufts), drawn by the owner in ArtCraft and vectorised; canonical vector | MIT OR Apache-2.0 (`assets/app-icon/LICENSE.txt`) |
| `packaging/macos/dmg/background.svg` | @XusBadia | Original work: FilmCraft macOS DMG window background, derived from the FilmCraft app icon (`assets/app-icon/filmcraft.svg`); rendered to `background.tiff` by `packaging/macos/dmg/generate.py` | MIT OR Apache-2.0 (`LICENSE-MIT`, `LICENSE-APACHE`) |
| `packaging/macos/dmg/background.tiff` | @XusBadia | Rendered from `packaging/macos/dmg/background.svg` by `packaging/macos/dmg/generate.py` | MIT OR Apache-2.0 (`LICENSE-MIT`, `LICENSE-APACHE`) |
| `assets/app-icon/filmcraft-small.svg` | Brandon Thomas (FilmCraft owner) | Original work: FilmCraft app icon (owl with ear tufts), drawn by the owner in ArtCraft and vectorised; lighter vector | MIT OR Apache-2.0 (`assets/app-icon/LICENSE.txt`) |
| `assets/app-icon/filmcraft-1024.png` | Brandon Thomas (FilmCraft owner) | Original work: FilmCraft app icon (owl with ear tufts), drawn by the owner in ArtCraft and vectorised; 1024 px render | MIT OR Apache-2.0 (`assets/app-icon/LICENSE.txt`) |
| `assets/app-icon/filmcraft-macos-512.png` | Brandon Thomas (FilmCraft owner) | Original work: FilmCraft app icon (owl with ear tufts), drawn by the owner in ArtCraft and vectorised; macOS runtime Dock icon (512 px) | MIT OR Apache-2.0 (`assets/app-icon/LICENSE.txt`) |
@@ -99,6 +101,7 @@ is traced or derived from Adobe artwork.
| Asset | Where |
|---|---|
| UI icons (tools, transport, panels, header) | `crates/ui-egui/src/icons.rs`: vector paths on a 16×16 grid |
| Source and Program range braces and trim hover cues | `crates/ui-egui/src/panels/monitor.rs` and `source_range.rs`: original generic vector shapes by FilmCraft contributors, MIT OR Apache-2.0; no third-party artwork |
| Align / distribute / paragraph-alignment glyphs in the graphics panel | `crates/ui-egui/src/panels/graphics.rs` (`align_glyph`): bars and lines drawn with the egui painter |
| Demo footage (ocean sunset, aurora, city night, dunes, forest, plasma), bars and tone, counting leader, colour matte | `crates/media/src/generators.rs` |
| Built-in graphics templates (Lower Third – Slab / Rule, Ticker – Crawl, Title – Centered / Boxed, End Credits – Roll, Callout – Pointer / Tag) and their placeholder text | `crates/project/src/gtemplate.rs` (`builtin_templates`): original designs defined in code with the bundled Inter font; thumbnails are rendered at runtime, no template, image or font files |
@@ -106,6 +109,21 @@ is traced or derived from Adobe artwork.
| Lumetri Presets (Effects panel: Cinematic, Film Emulation, Monochrome, Technical) and their preview picture | `crates/render/src/lumetri_presets.rs`: Lumetri parameter values chosen by contributors, descriptive names, and a procedural preview picture (dusk sky, ridges, lake, grey ramp, colour chips) |
| Colour palette and layout metrics | `crates/ui-egui/src/theme.rs`: colour values and sizes only; no artwork |
## Optional build input: craft-fonts
These fonts are **not files in this repository**. Builds made with the optional build input
`CRAFT_FONTS_DIR` (all official releases; see [docs/contributing.md](docs/contributing.md#building-with-craft-fonts))
embed them from [storytold/craft-fonts](https://github.com/storytold/craft-fonts), where each file,
its author, pinned upstream source and licence text are listed in its
[ATTRIBUTION.md](https://github.com/storytold/craft-fonts/blob/main/ATTRIBUTION.md). Release packages
carry each font's licence as `OFL-<family>.txt`.
| Font | Use in FilmCraft | Licence |
|---|---|---|
| BIZ UDPGothic Regular, Bold (Morisawa) | Japanese UI text; Japanese fallback for sans titles (the web build embeds only Regular) | OFL-1.1 |
| Shippori Mincho Regular (FONTDASU) | Japanese fallback for serif titles (native builds) | OFL-1.1 |
| BIZ UDMincho Regular (Morisawa) | Japanese fallback for serif titles (native builds) | OFL-1.1 |
## First-party brand marks
The ArtCraft name, wordmark and logos are trademarks of the ArtCraft Team. They are not open source and
@@ -124,6 +142,15 @@ remove them (AGENTS.md §1.8).
| `docs/brand/artcraft-mark.png` | ArtCraft Team | Original work: first-party ArtCraft brand mark (ArtCraft symbol ("A" mark), blue) | ArtCraft trademark, not open source; `docs/brand/LICENSE-brand.txt` (not MIT/Apache) |
| `docs/brand/artcraft-mark.svg` | ArtCraft Team | Original work: first-party ArtCraft brand mark (ArtCraft symbol ("A" mark), blue) | ArtCraft trademark, not open source; `docs/brand/LICENSE-brand.txt` (not MIT/Apache) |
## Third-party source code
Source files that are transcribed from third-party code rather than written from scratch. Each keeps
the original copyright and permission notice in its header.
| File | Author | Source | Licence |
|---|---|---|---|
| `crates/platform/src/nvenc/ffi.rs` | NVIDIA Corporation (`nvEncodeAPI.h`); FilmCraft contributors (Rust transcription) | Structures (including `NV_ENC_CONFIG_HEVC` and its bit-field masks, `NV_ENC_PIC_PARAMS_HEVC`, `NV_ENC_SEI_PAYLOAD`, `NV_ENC_TIME_CODE`), function table, codec / profile GUIDs (H.264, HEVC Main, HEVC Main 10) and constants (`NV_ENC_LEVEL_HEVC_*`, tier, CU size, `NV_ENC_BUFFER_FORMAT_YUV420_10BIT`, `NV_ENC_CAPS_SUPPORT_10BIT_ENCODE`, BT.2020 / PQ / HLG VUI codes) transcribed from `nvEncodeAPI.h`, NVIDIA Video Codec SDK, API 12.1 (https://developer.nvidia.com/video-codec-sdk) | MIT (Copyright (c) 2010-2023 NVIDIA Corporation; notice kept in the file header) |
## Downloaded at runtime
These files are never in the repository or the app bundle. FilmCraft downloads them into the
+2 -1
View File
@@ -1,6 +1,6 @@
# FilmCraft — instructions for agents
FilmCraft is a clean-room, open-source, pure-Rust non-linear video editor targeting Adobe Premiere Pro parity (and beyond). Native on macOS, Windows, Linux; web via WASM. Sibling of `../photocraft` (Photoshop), `../printcraft` (Acrobat) and `../drawcraft` (Illustrator), with the same conventions.
FilmCraft is a clean-room, open-source, pure-Rust non-linear video editor targeting Adobe Premiere Pro parity (and beyond). Native on macOS, Windows, Linux; web via WASM. Sibling of `../photocraft` (Photoshop), `../pdfcraft` (Acrobat) and `../vectorcraft` (Illustrator; formerly DrawCraft), with the same conventions.
## Start every session here
`plan/` is maintainer-local (gitignored, not in the repo). Each step names the public equivalent to use if you don't have it.
@@ -11,6 +11,7 @@ FilmCraft is a clean-room, open-source, pure-Rust non-linear video editor target
## Non-negotiables
**Read [`AGENTS.md`](AGENTS.md) first; its rules override everything here.**
- **Never crash** ([`AGENTS.md`](AGENTS.md) §0, standard: [`craftrules/standards/never-crash.md`](https://github.com/storytold/craftrules/blob/main/standards/never-crash.md)). People depend on FilmCraft; a crash loses their work, so this outranks feature work. No `unwrap`/`expect`/`panic!`/`unreachable!`/`todo!`/`unimplemented!` or `unsafe` outside tests: return `Result<T, E>` and propagate with `?` (`ok_or`, `let … else`, `if let`). Treat every number from media, project files, presets or CLI/MCP/control params as hostile: `get()` instead of `[i]`, checked/saturating math, no division by possibly-zero values, capped allocations, bounded recursion. `lock().unwrap_or_else(PoisonError::into_inner)`; background jobs run under `catch_unwind`. The panic hook + UI `catch_unwind` (`crates/ui-egui/src/crash.rs`) are the last-resort guard, not a licence. Every crash fix gets a regression test that panicked before. In particular (§1): never use Adobe iconography, images or any other Adobe asset. Every asset must be openly licensed (OSS / public domain / Creative Commons, or original work by a contributor) and must have a `<file>.attribution` sidecar plus an entry in `ATTRIBUTION.md`. `cargo xtask assets` enforces this.
- **Fonts** live in [storytold/craft-fonts](https://github.com/storytold/craft-fonts), never in this repo (no new font files here; [`AGENTS.md`](AGENTS.md) §1.9, standard: [`craftrules/standards/fonts.md`](https://github.com/storytold/craftrules/blob/main/standards/fonts.md)). Build with them: `git clone https://github.com/storytold/craft-fonts ../craft-fonts && CRAFT_FONTS_DIR="$PWD/../craft-fonts" cargo run -p filmcraft` (`crates/text/build.rs` embeds them as `filmcraft_text::fonts::CRAFT_FONTS`, used for Japanese in the UI and titles). Unset, `CRAFT_FONTS` is empty and everything must still build, test and run; never add craft-fonts to a `Cargo.toml`. Tests on its glyphs skip when it is empty, so run the gates with and without `CRAFT_FONTS_DIR` when you touch fonts.
- **Clean-room.** Never read/disassemble anything inside Adobe app bundles (names/listings only). Never copy Adobe icons, presets, LUTs, fonts. Never copy GPL/LGPL/AGPL code (FFmpeg, x264, x265, MLT, Kdenlive, Shotcut, Olive, LAME…). ffmpeg/ffprobe run only as external test oracles / fixture generators — never linked or shipped.
- **Pure Rust** in the product. OS media APIs only via Rust bindings in `crates/platform` (hardware decoding; the one crate allowed `unsafe`, see `AGENTS.md` §0.3 and `docs/adr/0001-platform-ffi.md`), optional, behind traits, with our decoders as the fallback.
- **Layering** ([`docs/architecture.md`](docs/architecture.md) §1, enforced by `cargo xtask layers`): nothing below L5 depends on egui/eframe/winit/rfd/cpal. L0 codec/container crates depend only on `filmcraft-bitstream`.
Generated
+92 -46
View File
@@ -992,6 +992,16 @@ dependencies = [
"windows 0.54.0",
]
[[package]]
name = "cpu-time"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e9e393a7668fe1fad3075085b86c781883000b4ede868f43627b34a87c8b7ded"
dependencies = [
"libc",
"winapi",
]
[[package]]
name = "cpufeatures"
version = "0.2.17"
@@ -1655,7 +1665,7 @@ dependencies = [
[[package]]
name = "filmcraft"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"chrono",
"cpal",
@@ -1671,13 +1681,14 @@ dependencies = [
"objc2-foundation 0.3.2",
"rfd",
"serde_json",
"sys-locale",
"winit",
"winresource",
]
[[package]]
name = "filmcraft-aac"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1687,22 +1698,32 @@ dependencies = [
[[package]]
name = "filmcraft-ac3"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-testkit",
"thiserror 2.0.21",
]
[[package]]
name = "filmcraft-apv"
version = "0.5.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
"rayon",
"thiserror 2.0.21",
]
[[package]]
name = "filmcraft-audio-dsp"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-testkit",
]
[[package]]
name = "filmcraft-automation"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"base64 0.22.1",
"filmcraft-engine",
@@ -1718,7 +1739,7 @@ dependencies = [
[[package]]
name = "filmcraft-av1"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1729,14 +1750,14 @@ dependencies = [
[[package]]
name = "filmcraft-bitstream"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"proptest",
]
[[package]]
name = "filmcraft-captions"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-color",
"filmcraft-project",
@@ -1749,11 +1770,11 @@ dependencies = [
[[package]]
name = "filmcraft-cfb"
version = "0.2.0"
version = "0.5.0"
[[package]]
name = "filmcraft-cli"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-automation",
"filmcraft-codecs",
@@ -1770,10 +1791,11 @@ dependencies = [
[[package]]
name = "filmcraft-codecs"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-aac",
"filmcraft-ac3",
"filmcraft-apv",
"filmcraft-av1",
"filmcraft-bitstream",
"filmcraft-color",
@@ -1795,6 +1817,7 @@ dependencies = [
"filmcraft-vp9",
"image",
"log",
"serde_json",
"symphonia",
"thiserror 2.0.21",
"web-time",
@@ -1802,7 +1825,7 @@ dependencies = [
[[package]]
name = "filmcraft-color"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"proptest",
"serde",
@@ -1810,7 +1833,7 @@ dependencies = [
[[package]]
name = "filmcraft-dnx"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1820,8 +1843,9 @@ dependencies = [
[[package]]
name = "filmcraft-edit"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-geom",
"filmcraft-media",
"filmcraft-project",
"filmcraft-time",
@@ -1832,7 +1856,7 @@ dependencies = [
[[package]]
name = "filmcraft-engine"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-audio-dsp",
"filmcraft-captions",
@@ -1863,9 +1887,10 @@ dependencies = [
[[package]]
name = "filmcraft-export"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-aac",
"filmcraft-apv",
"filmcraft-audio-dsp",
"filmcraft-codecs",
"filmcraft-color",
@@ -1892,11 +1917,12 @@ dependencies = [
[[package]]
name = "filmcraft-format"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-media",
"filmcraft-project",
"filmcraft-time",
"log",
"serde",
"serde_json",
"thiserror 2.0.21",
@@ -1904,7 +1930,7 @@ dependencies = [
[[package]]
name = "filmcraft-frame"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-color",
"filmcraft-time",
@@ -1914,7 +1940,7 @@ dependencies = [
[[package]]
name = "filmcraft-geom"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"proptest",
"serde",
@@ -1922,7 +1948,7 @@ dependencies = [
[[package]]
name = "filmcraft-golden"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-color",
"filmcraft-frame",
@@ -1939,7 +1965,7 @@ dependencies = [
[[package]]
name = "filmcraft-gpu"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-color",
"filmcraft-frame",
@@ -1956,7 +1982,7 @@ dependencies = [
[[package]]
name = "filmcraft-h264"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1966,7 +1992,7 @@ dependencies = [
[[package]]
name = "filmcraft-h264enc"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1976,7 +2002,7 @@ dependencies = [
[[package]]
name = "filmcraft-hevc"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1986,7 +2012,7 @@ dependencies = [
[[package]]
name = "filmcraft-interchange"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-cfb",
"filmcraft-color",
@@ -2003,7 +2029,7 @@ dependencies = [
[[package]]
name = "filmcraft-isobmff"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -2014,7 +2040,7 @@ dependencies = [
[[package]]
name = "filmcraft-matroska"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-testkit",
"serde_json",
@@ -2022,7 +2048,7 @@ dependencies = [
[[package]]
name = "filmcraft-media"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-color",
"filmcraft-frame",
@@ -2035,7 +2061,7 @@ dependencies = [
[[package]]
name = "filmcraft-mpeg2v"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-testkit",
"rayon",
@@ -2044,7 +2070,7 @@ dependencies = [
[[package]]
name = "filmcraft-mpegts"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-testkit",
"thiserror 2.0.21",
@@ -2052,46 +2078,54 @@ dependencies = [
[[package]]
name = "filmcraft-mxf"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-testkit",
]
[[package]]
name = "filmcraft-ogg"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-testkit",
]
[[package]]
name = "filmcraft-opus"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-testkit",
]
[[package]]
name = "filmcraft-platform"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-codecs",
"filmcraft-color",
"filmcraft-export",
"filmcraft-frame",
"filmcraft-h264",
"filmcraft-hevc",
"filmcraft-isobmff",
"filmcraft-media",
"filmcraft-project",
"filmcraft-render",
"filmcraft-testkit",
"filmcraft-time",
"libloading",
"log",
"objc2-core-foundation",
"objc2-core-media",
"objc2-core-video",
"objc2-video-toolbox",
"windows 0.62.2",
]
[[package]]
name = "filmcraft-project"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-color",
"filmcraft-geom",
@@ -2105,7 +2139,7 @@ dependencies = [
[[package]]
name = "filmcraft-prores"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -2115,7 +2149,7 @@ dependencies = [
[[package]]
name = "filmcraft-render"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-audio-dsp",
"filmcraft-captions",
@@ -2135,7 +2169,7 @@ dependencies = [
[[package]]
name = "filmcraft-scopes"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-color",
"serde",
@@ -2144,7 +2178,7 @@ dependencies = [
[[package]]
name = "filmcraft-speech"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"candle-core",
"candle-nn",
@@ -2162,25 +2196,26 @@ dependencies = [
[[package]]
name = "filmcraft-testkit"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"png",
]
[[package]]
name = "filmcraft-text"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-color",
"harfrust",
"skrifa",
"unicode-bidi",
"unicode-linebreak",
"unicode-segmentation",
]
[[package]]
name = "filmcraft-time"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"proptest",
"serde",
@@ -2188,8 +2223,9 @@ dependencies = [
[[package]]
name = "filmcraft-ui-egui"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"cpu-time",
"eframe",
"egui",
"egui_kittest",
@@ -2216,12 +2252,13 @@ dependencies = [
"rustix 1.1.5",
"serde",
"serde_json",
"toml 1.1.6+spec-1.1.0",
"web-time",
]
[[package]]
name = "filmcraft-vp9"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-bitstream",
"rayon",
@@ -2230,7 +2267,7 @@ dependencies = [
[[package]]
name = "filmcraft-web"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"eframe",
"egui",
@@ -5754,6 +5791,15 @@ dependencies = [
"syn 3.0.6",
]
[[package]]
name = "sys-locale"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8eab9a99a024a169fe8a903cf9d4a3b3601109bcc13bd9e3c6fff259138626c4"
dependencies = [
"libc",
]
[[package]]
name = "sysctl"
version = "0.6.0"
@@ -7211,7 +7257,7 @@ checksum = "e450f9b2ed1dff33c94c12589a87338689467b9c4f5d8a5710bd09a847d2c8a7"
[[package]]
name = "xtask"
version = "0.2.0"
version = "0.5.0"
dependencies = [
"filmcraft-testkit",
"serde_json",
+5 -1
View File
@@ -4,7 +4,7 @@ members = ["crates/*", "apps/*", "xtask"]
default-members = ["crates/*", "apps/*", "xtask"]
[workspace.package]
version = "0.2.0"
version = "0.5.0"
edition = "2024"
license = "MIT OR Apache-2.0"
rust-version = "1.95"
@@ -29,6 +29,7 @@ filmcraft-hevc = { path = "crates/hevc" }
filmcraft-vp9 = { path = "crates/vp9" }
filmcraft-prores = { path = "crates/prores" }
filmcraft-dnx = { path = "crates/dnx" }
filmcraft-apv = { path = "crates/apv" }
filmcraft-av1 = { path = "crates/av1" }
filmcraft-aac = { path = "crates/aac" }
filmcraft-opus = { path = "crates/opus" }
@@ -101,6 +102,9 @@ opt-level = 3
[profile.dev.package.filmcraft-dnx]
opt-level = 3
[profile.dev.package.filmcraft-apv]
opt-level = 3
[profile.dev.package.filmcraft-av1]
opt-level = 3
+3
View File
@@ -15,4 +15,7 @@ list (author, source, license) is in ATTRIBUTION.md.
Inter font (assets/fonts/Inter-*.ttf) — OFL-1.1. See assets/fonts/OFL-Inter.txt.
Noto Serif font (assets/fonts/NotoSerif-Regular.ttf) — OFL-1.1. See assets/fonts/OFL-NotoSerif.txt.
JetBrains Mono font (assets/fonts/JetBrainsMono-Regular.ttf) — OFL-1.1. See assets/fonts/OFL-JetBrainsMono.txt.
Builds made with craft-fonts (CRAFT_FONTS_DIR; all official releases) also embed the fonts listed in
https://github.com/storytold/craft-fonts/blob/main/ATTRIBUTION.md (BIZ UDPGothic, Shippori Mincho,
BIZ UDMincho) — OFL-1.1. Their licence texts ship with the packages as OFL-<family>.txt.
Public-domain film and NASA frames and CC0 music shown in README screenshots (docs/images/) — public domain / CC0-1.0. See ATTRIBUTION.md.
+98 -9
View File
@@ -59,6 +59,7 @@
<a href="#interchange">Interchange</a> ·
<a href="#built-for-agents">Agents</a> ·
<a href="#get-started">Get started</a> ·
<a href="#downloads">Downloads</a> ·
<a href="#documentation">Docs</a> ·
<a href="#status">Status</a>
</p>
@@ -136,7 +137,7 @@ FilmCraft is a non-linear editor for people who know Premiere: the same panels,
- **Premiere 26's full Video Effects bin (93 effects in 16 folders), the Legacy bin and the obsolete effects old projects use, plus around 30 transitions.** Blurs (Bokeh, Focus, Compound), keys (Ultra Key, Track Matte), distortions (Corner Pin, Turbulent Displace, Warp Stabilizer), Lights & Glows, Immersive Video (VR) effects on equirectangular footage, Posterize Time, Echo and more. Cross dissolve, dip to black or white, film dissolve, wipes, irises, pushes, slides, zooms, page peel, cube spin and more.
- **Motion and opacity on every clip:** position, scale, rotation, anchor point and anti-flicker, plus 26 blend modes.
- **Keyframes like Premiere's:** linear, Bezier, auto and continuous Bezier, hold, ease in and ease out. Effect Controls shows a keyframe lane for every parameter, and each animated parameter opens into **value and velocity graphs** with draggable influence handles.
- **Keyframes like Premiere's:** linear, Bezier, auto and continuous Bezier, hold, ease in and ease out. Effect Controls shows a keyframe lane for every parameter under a time ruler with the playhead's handle, and each animated parameter opens into **value and velocity graphs** with draggable influence handles. Effect Controls and the Properties panel share the keyframe navigator (◀ ◆ ▶): add or remove the keyframe at the playhead, step to the previous or next one.
- **A GPU compositor** built on wgpu (Metal, Vulkan, DirectX 12, WebGPU). It samples YUV straight from the decoder with footprint supersampling and blends in linear light. A CPU path renders the same frames, and the two are tested against each other.
<p align="center">
@@ -184,7 +185,7 @@ No FFmpeg inside. The video codecs, AAC, Opus and the containers are our own Rus
| **Matroska / WebM** | ✓ | | Lacing, Cues, header stripping, HDR colour metadata |
| **Stills** | ✓ | ✓ | Import PNG, JPEG, GIF, WebP, TIFF and BMP; export PNG sequences and animated GIF |
Also imported: MPEG-2 / MPEG-1 video, AC-3, MP2, MXF (OP1a / OP-Atom), MPEG transport and program streams (AVCHD, broadcast, DVD), Ogg and image sequences. Not yet: HEVC and AV1 export, camera RAW, E-AC-3. Hardware decoding works on macOS (VideoToolbox), with Linux and Windows to follow ([#30](https://github.com/storytold/filmcraft/issues/30)).
Also imported: MPEG-2 / MPEG-1 video, AC-3, MP2, MXF (OP1a / OP-Atom), MPEG transport and program streams (AVCHD, broadcast, DVD), Ogg and image sequences. Not yet: AV1 export, camera RAW, E-AC-3. Hardware decoding works on macOS (VideoToolbox), Windows (Media Foundation) and, for H.264 and HEVC, Linux (VA-API); H.264 export can use NVIDIA's encoder on Windows and Linux (opt-in), and H.265 (HEVC Main, 8-bit) export works through it (NVENC on Windows, VideoToolbox on macOS) ([#30](https://github.com/storytold/filmcraft/issues/30)).
<br>
@@ -196,7 +197,7 @@ Also imported: MPEG-2 / MPEG-1 video, AC-3, MP2, MXF (OP1a / OP-Atom), MPEG tran
<p align="center"><sub>Export mode, set to write the trailer as H.264 MP4.</sub></p>
- **H.264 MP4 with AAC, using our own encoders.** A 6-second 960×540 render takes 1.3 seconds and decodes cleanly in ffmpeg with error concealment switched off.
- **H.264 MP4 with AAC, using our own encoders.** A 6-second 960×540 render takes 1.3 seconds and decodes in ffmpeg without a single warning, with strict error detection on.
- **Also:** Apple ProRes 422 HQ and Motion JPEG in QuickTime, MXF OP1a and Avid-style OP-Atom (DNxHR, ProRes or H.264 with PCM and start timecode), PNG sequences, animated GIF and WAV.
- **Background jobs** with progress and cancel, so you keep editing while it renders.
- **Render previews:** the render bar marks segments green, yellow or red; rendered previews are cached by content, so an edit only invalidates what it touches and undo brings the green back.
@@ -220,7 +221,7 @@ Import merges the document's bins, media and sequences into your project as one
## Built for agents
Every menu item, button, slider and drag in FilmCraft is a **command** with an id, typed parameters and an enabled state. There are about 135 engine commands so far, with the rest of Premiere's catalogue on the way. The UI, the CLI, a JSON control channel and an **MCP server** all dispatch the same commands, so Claude or any agent can cut, trim, grade, mix and export exactly the way a person does. The UI can also be driven at the level of mouse and keyboard: every widget has an automation id, and agents can click, drag, type and take screenshots.
Every menu item, button, slider and drag in FilmCraft is a **command** with an id, typed parameters and an enabled state. There are more than 650 engine commands (`filmcraft-cli commands` lists them), with the rest of Premiere's catalogue on the way. The UI, the CLI, a JSON control channel and an **MCP server** all dispatch the same commands, so Claude or any agent can cut, trim, grade, mix and export exactly the way a person does. The UI can also be driven at the level of mouse and keyboard: every widget has an automation id, and agents can click, drag, type and take screenshots.
```jsonc
// over the control channel (JSON lines on TCP) or as MCP tool calls
@@ -238,7 +239,7 @@ The trailer and the grades in these screenshots were built exactly this way, by
## Everywhere
- **Native** on macOS, Windows and Linux, with a native macOS menu bar.
- **The web:** every crate up to the engine compiles to `wasm32`; the browser front end is next.
- **The web:** the same engine and UI run in the browser via WebAssembly (`apps/filmcraft-web`, see [docs/web.md](docs/web.md)); every release ships it as `filmcraft-web-<version>.zip`.
- **Swappable UI.** The interface is one crate (`ui-egui`) over the engine, so a different front end can replace it without touching editing logic.
<br>
@@ -252,6 +253,13 @@ cargo run --release -p filmcraft-cli -- commands # list every engine c
cargo run --release -p filmcraft-cli -- mcp # MCP server (headless)
```
Japanese text in the interface and in titles comes from [craft-fonts](https://github.com/storytold/craft-fonts), an optional build input (release builds always include it; without it FilmCraft uses its own and the system's fonts):
```sh
git clone https://github.com/storytold/craft-fonts ../craft-fonts
CRAFT_FONTS_DIR="$PWD/../craft-fonts" cargo run --release -p filmcraft
```
The control protocol is documented in [docs/control-protocol.md](docs/control-protocol.md). Stuck, or want to show what you made? Ask in [Discord](https://discord.gg/artcraft).
## Documentation
@@ -279,9 +287,9 @@ We track two numbers ([ROADMAP.md](ROADMAP.md#honest-assessment-2026-10-05)):
The biggest gaps today:
- **Speed on big footage.** Hardware decoding works on macOS; Windows and Linux have no hardware path yet. Blend modes and the most common effects run on the GPU, but Lumetri, keys, export and encoding still run on the CPU ([#30](https://github.com/storytold/filmcraft/issues/30)).
- **Speed on big footage.** Hardware decoding works on macOS and Windows, and for H.264 and HEVC on Linux (VA-API; VP9 and AV1 decode in software there). Blend modes and the most common effects run on the GPU, but Lumetri, keys and export rendering still run on the CPU; H.264 encoding can use the hardware encoder on macOS and on Windows and Linux with an NVIDIA GPU (opt-in, Export ▸ Hardware encoding), and H.265 export is hardware-only on both (HDR sequences export Main 10 PQ / HLG with NVENC; VideoToolbox writes 8-bit SDR) ([#30](https://github.com/storytold/filmcraft/issues/30)).
- **No plugins.** No VST3 / Audio Units or OpenFX hosting.
- **Delivery codecs.** H.264 is our only delivery-codec export; no HEVC or AV1 export yet.
- **Delivery codecs.** H.264 is our only software delivery-codec export; H.265 exports only through a hardware encoder (NVIDIA on Windows: Main 8-bit SDR and Main 10 HDR PQ / HLG; VideoToolbox on macOS: 8-bit Main, SDR); no AV1 export yet.
- **Real-world media and platforms.** Our decoders are bit-exact on conformance streams, but camera and phone files in the wild are less tested. Windows and Linux get far less testing than macOS.
- **AI features.** Few so far; speech to text is optional and off by default.
@@ -303,6 +311,81 @@ A layered Cargo workspace:
Nothing below the front ends depends on a UI toolkit or OS API. `cargo xtask ci` checks formatting, lints, tests, the layering rules, asset attribution and the wasm build.
## Downloads
**New to FilmCraft?** Download it from the [FilmCraft page on getartcraft.com](https://getartcraft.com/apps/filmcraft). That's the easiest way to install it.
**Want a specific build or format?** On GitHub, the [latest release](https://github.com/storytold/filmcraft/releases/latest) has every build listed below, and [all releases](https://github.com/storytold/filmcraft/releases) has earlier versions and their notes. `<ver>` in the file names is the version number, and `SHA256SUMS.txt` lists a checksum for every file.
### Windows
| Build | Installer | Portable |
|---|---|---|
| x64 (64-bit Intel/AMD) | `filmcraft-<ver>-windows-x64.msi` | `filmcraft-<ver>-windows-x64-portable.zip` |
| arm64 (Snapdragon and other ARM PCs) | `filmcraft-<ver>-windows-arm64.msi` | `filmcraft-<ver>-windows-arm64-portable.zip` |
| x86 (32-bit) | `filmcraft-<ver>-windows-x86.msi` | `filmcraft-<ver>-windows-x86-portable.zip` |
Installers and executables are code-signed.
**If the app doesn't open on Windows:** the desktop app initializes only Vulkan and DirectX 12 by
default, not OpenGL. Letting wgpu also create an OpenGL instance can crash some graphics drivers (AMD's
`atio6axx.dll`) before the window appears, so the app would flash in Task Manager and quit.
`WGPU_BACKEND` overrides the default for troubleshooting (for example `dx12` or `vulkan`). In
PowerShell, from the folder containing the executable:
```powershell
$env:WGPU_BACKEND = "vulkan"
& .\filmcraft.exe
Remove-Item Env:WGPU_BACKEND # restore the default for later launches
```
An explicit `gl` override can bring the driver crash back on affected systems. The macOS, Linux
and web backend defaults are unchanged.
### macOS
| Build | File | Notes |
|---|---|---|
| App, universal (Apple silicon + Intel) | `filmcraft-<ver>-macos-universal.dmg` | Signed and notarized |
| Command-line tool, universal | `filmcraft-cli-<ver>-macos-universal.zip` | Signed and notarized |
### Linux
| Format | x86_64 | aarch64 (ARM64) | Notes |
|---|---|---|---|
| AppImage | `filmcraft-<ver>-linux-x86_64.AppImage` | `filmcraft-<ver>-linux-aarch64.AppImage` | Runs anywhere; updates itself with [AppImageUpdate](https://github.com/AppImageCommunity/AppImageUpdate) (`.zsync` files) |
| Flatpak | `filmcraft-<ver>-linux-x86_64.flatpak` | `filmcraft-<ver>-linux-aarch64.flatpak` | Sandboxed; `flatpak install --user <file>` |
| Debian/Ubuntu | `filmcraft-<ver>-linux-x86_64.deb` | `filmcraft-<ver>-linux-aarch64.deb` | |
| Fedora/RHEL/openSUSE | `filmcraft-<ver>-linux-x86_64.rpm` | `filmcraft-<ver>-linux-aarch64.rpm` | |
| Tarball | `filmcraft-<ver>-linux-x86_64.tar.gz` | `filmcraft-<ver>-linux-aarch64.tar.gz` | Unpack and run `./install.sh` |
The tarball installer places FilmCraft, its command-line tool and desktop integration in
`~/.local`, without administrator permissions. For all users, run
`sudo ./install.sh --prefix /usr/local` instead. Running the installer again updates the installation.
**Gentoo (community-maintained):** the [::snakebyte overlay](https://github.com/switch87/snakebyte-overlay)
packages the Linux release as `media-video/filmcraft-bin`. It is maintained by the community, not by the
FilmCraft team, so report packaging problems to the overlay:
```sh
eselect repository add snakebyte git https://github.com/switch87/snakebyte-overlay.git
emaint sync -r snakebyte
echo 'media-video/filmcraft-bin ~amd64' >> /etc/portage/package.accept_keywords/filmcraft
emerge --ask media-video/filmcraft-bin
```
### FreeBSD
| Build | File |
|---|---|
| x86_64 | `filmcraft-<ver>-freebsd-x86_64.tar.gz` |
### Web (WebAssembly)
| Build | File | Notes |
|---|---|---|
| Static site | `filmcraft-web-<ver>.zip` | Runs in a modern browser; host it on any static server |
## The Crafting Apps
FilmCraft is one of the **Crafting Apps**: free, open-source creative tools from the
@@ -315,7 +398,7 @@ stand on its own.
| <img src="https://raw.githubusercontent.com/storytold/vectorcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.vectorcraft.png" alt="" width="32" height="32"> | **VectorCraft** | Vector illustration | [GitHub](https://github.com/storytold/vectorcraft) | [Website](https://getartcraft.com/apps/vectorcraft) |
| <img src="https://raw.githubusercontent.com/storytold/filmcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.filmcraft.png" alt="" width="32" height="32"> | **FilmCraft** | **Video editing, color and sound · you are here** | [GitHub](https://github.com/storytold/filmcraft) | [Website](https://getartcraft.com/apps/filmcraft) |
| <img src="https://raw.githubusercontent.com/storytold/lightcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.lightcraft.png" alt="" width="32" height="32"> | **LightCraft** | Photo library and raw development | [GitHub](https://github.com/storytold/lightcraft) | [Website](https://getartcraft.com/apps/lightcraft) |
| <img src="https://raw.githubusercontent.com/storytold/printcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.printcraft.png" alt="" width="32" height="32"> | **PrintCraft** | Reading, organizing and protecting PDFs | [GitHub](https://github.com/storytold/printcraft) | [Website](https://getartcraft.com/apps/printcraft) |
| <img src="https://raw.githubusercontent.com/storytold/pdfcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.pdfcraft.png" alt="" width="32" height="32"> | **PdfCraft** | Reading, organizing and protecting PDFs | [GitHub](https://github.com/storytold/pdfcraft) | [Website](https://getartcraft.com/apps/pdfcraft) |
| <img src="https://raw.githubusercontent.com/storytold/effectcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.effectcraft.png" alt="" width="32" height="32"> | **EffectCraft** | Motion graphics and visual effects | [GitHub](https://github.com/storytold/effectcraft) | [Website](https://getartcraft.com/apps/effectcraft) |
| <img src="https://raw.githubusercontent.com/storytold/designcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.designcraft.png" alt="" width="32" height="32"> | **DesignCraft** | Page layout and publishing | [GitHub](https://github.com/storytold/designcraft) | [Website](https://getartcraft.com/apps/designcraft) |
@@ -351,7 +434,9 @@ FilmCraft is dual-licensed under [MIT](LICENSE-MIT) or [Apache-2.0](LICENSE-APAC
Copyright (c) 2026 ArtCraft Team and the FilmCraft contributors. Required notices are in [NOTICE](NOTICE).
Bundled fonts, icons, images and other assets keep their own open licenses; each one is listed
with its author, source and license in [ATTRIBUTION.md](ATTRIBUTION.md).
with its author, source and license in [ATTRIBUTION.md](ATTRIBUTION.md). Builds made with
[craft-fonts](https://github.com/storytold/craft-fonts) (all official releases) also embed its fonts
(OFL-1.1), listed in its [ATTRIBUTION.md](https://github.com/storytold/craft-fonts/blob/main/ATTRIBUTION.md).
**Footage and music in the screenshots:** NASA's Apollo 11 film and television footage from [images.nasa.gov](https://images.nasa.gov) (launch, Launch Control Center, lunar surface and recovery) and the Apollo 11 air-to-ground voice transcript, all US Government works in the public domain (NASA does not endorse this project); *Night of the Living Dead* (1968), *Carnival of Souls* (1962) and *Charade* (1963), all in the US public domain; *Earth Views from the ISS* by NASA; Chopin's Nocturne Op. 48 No. 1 and Ballade No. 1, performed for Musopen and released under CC0. The media itself is not in this repository. Sources and details for every asset are in [ATTRIBUTION.md](ATTRIBUTION.md).
@@ -370,3 +455,7 @@ Forks and modified versions must remove them.
<a href="https://getartcraft.com/"><img alt="ArtCraft" src="docs/brand/artcraft-mark.svg" width="28"></a><br>
<sub>Made by the <a href="https://getartcraft.com/">ArtCraft</a> team and community.</sub>
</p>
## Star history
[![Star History Chart](https://api.star-history.com/svg?repos=storytold/filmcraft&type=Date&legend=top-left)](https://www.star-history.com/?repos=storytold%2Ffilmcraft&type=date&legend=top-left)
+19 -7
View File
@@ -35,8 +35,8 @@ before choosing work.
| **Feature breadth** (menus, commands, panels) | ~86–95% | **~85%** | Menu diff against Premiere 26.5; effect, transition and audio-effect catalogues complete by test. The long tail is mostly there. |
| **Correctness in depth** | not tracked | **~60–70%** | A first-time contributor found four real bugs in core paths within hours (#5–#8: AIFF import failed, Slide left linked audio behind, transitions didn't follow ripple trims, export ignored start/end times). User reports: Color Matte always grey with no picker (#29); ALSA audio error on Linux (#23). Settings are "most wired", not all. |
| **Codecs and media** | ~90% | **~75%** | Our decoders are bit-exact on conformance streams. Real camera and phone media (variable frame rate, damaged files, unusual containers) is far less tested. No camera RAW (RED, BRAW, ARRIRAW), no E-AC-3. |
| **Export** | ~92% | **~70%** | Delivery: H.264 only (ProRes, DNxHR and image sequences cover mastering). No HEVC or AV1 export. AAF and OMF have never been validated in Avid Media Composer or Pro Tools. Export renders and encodes on the CPU only. |
| **Performance and hardware** | ~62% | **~35–40%** | Premiere runs effects, decoding and encoding on the GPU and the hardware media engines. Here, hardware decode works on macOS only (VideoToolbox, #33). The GPU does compositing, blend modes (#32) and 31 common effects (GPU2): colour adjustments, crop/flip/transform, blurs, sharpen. Lumetri, keys, masks, the remaining effects, export and encoding are CPU. Linux and Windows have no hardware path. 8K and AV1 aren't real-time. The effect list shows Premiere's "GPU accelerated" badge, but only 31 effect ids run on the GPU here; the rest, Lumetri included, run on the CPU. |
| **Export** | ~92% | **~70%** | Delivery: H.264, plus H.265 through a hardware encoder only (NVENC on Windows: Main 8-bit SDR and Main 10 HDR PQ / HLG; VideoToolbox on macOS: Main 8-bit SDR); ProRes, DNxHR and image sequences cover mastering. No AV1 export, no 10-bit / HDR HEVC on macOS. AAF and OMF have never been validated in Avid Media Composer or Pro Tools. Export renders and encodes on the CPU only. |
| **Performance and hardware** | ~62% | **~35–40%** | Premiere runs effects, decoding and encoding on the GPU and the hardware media engines. Here, hardware decode works on macOS (VideoToolbox, #33; active in the desktop app only since #41, so releases up to 0.2.1 decoded in software) and Windows (Media Foundation / Direct3D 11 DXVA, H.264, HEVC Main / Main 10, VP9 and AV1 4:2:0 8/10-bit, #30), and for H.264 and HEVC on Linux (VA-API: 4K H.264 at 11 ms of CPU per frame instead of 417 ms, 4K HEVC at 10 ms instead of 203 ms). The GPU does compositing, blend modes (#32) and 31 common effects (GPU2): colour adjustments, crop/flip/transform, blurs, sharpen. Lumetri, keys, masks, the remaining effects and export rendering are CPU; H.264 encoding can use VideoToolbox on macOS (opt-in per export, no B-frames) and NVENC on Windows and Linux with an NVIDIA GPU (opt-in per export), and H.265 (HEVC) export exists on macOS and Windows only, through the same hardware encoders. Linux decodes H.264 and HEVC in hardware (VP9 and AV1 there are software); its only hardware encoder is NVENC H.264. 8K isn't real-time (AV1 is on Windows with a hardware decoder). The effect list shows Premiere's "GPU accelerated" badge, but only 31 effect ids run on the GPU here; the rest, Lumetri included, run on the CPU. |
| **Stability** | not tracked | **improving, unproven** | The never-crash pass (no panics in product code, last-resort guards) landed on 2026-10-04. There's no field record from real users' projects yet. |
| **Plugins and pro ecosystem** | not tracked | **~0–10%** | No audio plugin hosting (VST3 / Audio Units) and no third-party video effects (OpenFX). For many professional editors this alone rules FilmCraft out. Team Projects and Productions are out of scope by design. |
| **AI features** | partial | **~25–35%** | Speech to text exists but is off in default builds (the `whisper` feature). Enhance Speech is a DSP chain, not a model. Auto Reframe is approximate. No Generative Extend, and no media-intelligence search or auto colour. |
@@ -48,10 +48,10 @@ before choosing work.
In priority order. Agents should prefer this work over adding more checklist items.
1. **Hardware acceleration** (#30), the most visible gap to users:
- hardware decode on Linux (VA-API) and Windows (Media Foundation);
- hardware decode on Linux (VA-API): H.264 and HEVC are in; VP9 and AV1 remain;
- zero-copy decoded frames into wgpu;
- the remaining effects on the GPU: Lumetri, keys, Vignette, Video Limiter, masks (31 common effects are done);
- GPU export, then hardware encode (H.264 / HEVC).
- GPU export, then hardware encode: H.264 and H.265 on macOS are in (H.264 opt-in; H.265 has no other encoder), H.264 and H.265 (Main 8-bit, and Main 10 HDR PQ / HLG) on NVENC (Windows); Main 10 / HDR HEVC on VideoToolbox, Linux and other Windows vendors remain.
2. **A measured parity number.** Add `cargo xtask parity`, run in CI. It should cover menus,
commands, effects, transitions, panels, preferences and formats against the Premiere snapshot in
`plan/premiere/`. It should report presence and fidelity separately (exact, approximate, stub).
@@ -63,7 +63,7 @@ In priority order. Agents should prefer this work over adding more checklist ite
through XML / AAF.
5. **Windows and Linux at runtime.** CI that runs the test suite and a headless playback/export
smoke test on both, not just release builds. Fix the Linux audio path (#23).
6. **Delivery codecs.** HEVC and AV1 export: hardware encoders first, and `rav1e` (BSD-2) for AV1.
6. **Delivery codecs.** HEVC export is in on macOS (VideoToolbox, 8-bit, no software encoder) and on Windows (NVENC, 8-bit and Main 10 HDR); still to do: HEVC elsewhere, 10-bit / HDR HEVC on macOS, and AV1 (`rav1e`, BSD-2).
7. **Plugin hosting** (VST3 / Audio Units, then OpenFX). Needs FFI, so it falls under the isolated
`unsafe` crate rule (AGENTS.md §0.3). **Owner decision needed** before starting.
8. **AI features** with openly licensed local models: transcription on by default, text-based
@@ -100,7 +100,7 @@ matched against `filmcraft-cli commands` and the UI command table. Effects: the
| Graphics and captions | 8% | text engine, shapes, per-character styles, our own graphics templates (.fcgt, 8 built-ins, export/install/edit), rolls/crawls, responsive pins + time, captions SRT/VTT/SCC/MCC/STL/TTML/DFXP, transcripts | ~85% |
| Export | 8% | own H.264/AAC, ProRes, DNxHR, MXF OP1a/OP-Atom, image sequences, GIF, WAV, 5.1; preset library + user presets, full Export-mode settings, queue, Quick Export; AAF (Edit Protocol) import/export, OMF 2.0 export; no AAF/OMF validation against Avid/Pro Tools yet | ~92% |
| Preferences and project management | 5% | Settings dialog with 16 categories (most settings wired), project settings, scratch disks, search bins, templates | ~80% |
| Performance | 5% | 1080p real-time, 3×1080p; 4K H.264 real-time at load ≤140 (−23% decoder cycles, opt-in draft decoding at 1/2–1/4); HEVC 2.5× cheaper, catch-up decoding, `cargo xtask bench` + `perf.stats`; blend modes on the GPU (#32); 8K and AV1 not yet real-time on a loaded machine; no hardware path on Windows/Linux (see the honest assessment: ~35–40% against Premiere) | ~62% |
| Performance | 5% | 1080p real-time, 3×1080p; 4K H.264 real-time at load ≤140 (−23% decoder cycles, opt-in draft decoding at 1/2–1/4); HEVC 2.5× cheaper, catch-up decoding, `cargo xtask bench` + `perf.stats`; blend modes on the GPU (#32); 8K and AV1 not yet real-time on a loaded machine; H.264 and HEVC decode in hardware on Linux (VA-API), and on Windows only decode and opt-in NVENC H.264 export (see the honest assessment: ~35–40% against Premiere) | ~62% |
| **Weighted total** | | | **~87%** |
## Estimate to parity
@@ -154,11 +154,19 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining
## Running now
- **Hardware acceleration (#30):** landed: blend modes on the GPU (#32); VideoToolbox hardware decode + the `platform` FFI crate (#33); 31 common effects on the GPU (GPU2). Next: VA-API (Linux) and Media Foundation (Windows) decode, zero-copy upload, Lumetri on the GPU.
- **Hardware acceleration (#30):** landed: blend modes on the GPU (#32); VideoToolbox hardware decode + the `platform` FFI crate (#33); 31 common effects on the GPU (GPU2). Windows: Media Foundation / Direct3D 11 H.264 + HEVC decode (HW2) and VP9 + AV1 (HW3), NVENC H.264 export (HW4, opt-in) and H.265 export. Linux: VA-API H.264 + HEVC decode (HW5). Next: VA-API VP9 / AV1, zero-copy upload, Lumetri on the GPU.
- **Next:** [Where we are lacking](#where-we-are-lacking) items 1–5. GPU export waits for the export-crate work (#19 and the never-crash follow-up) to land.
## Log
- **2026-10-08 (Source range dragging):** implicit full-clip In/Out, video-only/audio-only/linked both controls, picture/waveform drag gestures and one-command undoable timeline placement. Shared Source/Program draggable frame-bounded handles and range translation, cancellation, Source marker/navigation routing and marked-range playback/looping.
- **2026-10-08 (Source playback):** normal forward playback for video/audio media and subclips in the Source monitor, an independent Source clock, Play/Pause and focused Space, frame stepping, device-clock audio with wall-clock fallback, seeking, and end-of-clip stopping. Windows desktop review verified a real video file and standalone WAV, including In/Out marking. Reverse/shuttle Source playback and source sequences remain unsupported; no parity percentage changed.
- **2026-10-08 (night):** HDR H.265 (HEVC Main 10, PQ and HLG, BT.2020, limited range) export on Windows through NVENC (#30): an HDR sequence now exports as real 10-bit HDR (`rgbf_to_yuv420_10` into P010 input buffers; VUI BT.2020 + PQ / HLG; the HDR10 mastering-display and content-light messages as SEI on every IDR for PQ, through the generic SEI payload array because NVENC 12.1 has no mastering-display field; `colr` / `mdcv` / `clli` in the sample entry), where the GPU reports 10-bit HEVC (new `register_hdr_probe` / `hdr_available` in the export crate; elsewhere, and on macOS, HDR sequences still tone-map to 8-bit SDR HEVC, and `settings.sdr` forces that). ffprobe reads `hevc` / `Main 10` / `hvc1` / `yuv420p10le` / `bt2020` / `smpte2084` or `arib-std-b67` / `bt2020nc` / `tv` with the mastering-display and content-light side data for PQ; ffmpeg's decode equals ours code for code; the in-process round trip is at 71.1 dB luma PSNR on the 10-bit scale with 803 distinct levels on a ramp and the peak code 940 intact (RTX 5060). 8-bit SDR H.265 is unchanged.
- **2026-10-08:** H.265 (HEVC Main, 8-bit 4:2:0, SDR) export on Windows through NVENC (#30): the NVENC session, ring and NV12 path of the H.264 encoder now serve both codecs (the H.264 export is byte-for-byte unchanged); the format is available where the GPU has an HEVC encoder, choosing it is the opt-in, and `hvcC` is built from the SPS the encoder wrote. ffprobe reads `hevc` / `Main` / `hvc1` / `yuv420p` / BT.709; our own HEVC decoder decodes it at 48.9 dB luma PSNR (RTX 5060). Main 10 and HDR remain.
- **2026-10-08 (#285):** `filmcraft-cli` no longer panics with "Broken pipe" when its stdout is closed early (`filmcraft-cli commands | head`): later output is dropped, the rest of the work (script lines, `--save`, `--save-as`) still runs, and the exit status still reports failures. Other stdout write errors are reported and make the exit status 1. No change to the checklist or estimates.
- **2026-10-06:** 21 community PRs landed (#19, #28, #38, #39, #41–#60). Highlights: hardware decoding is now actually on in the desktop app (#41: `register()` sat inside a `log::info!` that never ran, so #33's VideoToolbox path only worked in the CLI and benchmarks); decode memory on long timelines cut sharply (frame pool, shared GOP-cache budget, positional file reads); exports are byte-identical across machines (#19); crash fix for odd track names over MCP (#44); ripple trim, ripple delete and speed changes move split-edit clips once (#56–#58); Japanese interface using an installed system font (#38; no fonts bundled); MCP annotations, resources, export progress and cancellation (#28).
- **2026-10-05:** honest assessment added (checklist ~87% vs ready for real work ~50–60%). Hardware acceleration started (#30): blend modes on the GPU (#32); VideoToolbox decode + `platform` crate, the one crate allowed `unsafe` (#33). Community fixes landed: AIFF import, export start/end ranges, transitions following ripple trims, Slide moving linked audio (#5–#8).
- **2026-10-04 (later):** interchange + MXF export (M11.4/M6.6): own compound-file container (`crates/cfb`), AAF Edit Protocol import/export (embedded/linked/consolidated audio, video mixdown, handles, breakout to mono), OMF 2.0 export (Bento), MXF OP1a/OP-Atom writer (DNxHR/ProRes/H.264/PCM) as export formats. 1605 tests.
@@ -200,3 +208,7 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining
- **2026-09-30 (midday):** HEVC decoder, Matroska/WebM demuxer and audio DSP merged; HEVC import wired; asset rules (AGENTS.md, ATTRIBUTION.md, `cargo xtask assets`); README with hero screenshot and the Craft family.
- **2026-09-30 (late morning):** keyframe value/velocity graphs; xtask gates; H.264 MP4 export; Lumetri curves, wheels, looks, HSL secondary.
- **2026-09-30 (early morning):** H.264 decoder, ProRes, AAC; GPU compositor; MCP; Premiere 26 visual fidelity pass.
### Interface localization
Edit > Language switches English/Japanese/Spanish/Portuguese (Brazilian) and persists across restarts through `general.interfaceLanguage` (also available in Settings > General). Until a language is chosen, System Language (the default) follows the operating system's or browser's preferred languages and falls back to English. Commands `app.language.english`, `app.language.japanese`, `app.language.spanish` and `app.language.portuguese` are reachable through the control channel. Translations live in one catalog per language (`crates/ui-egui/src/i18n/<code>.tsv`); UI code wraps its strings in `tl!("…")` / `tlf!("…{name}", name)`, and names that come from registries (effects and their parameters, settings, commands, panels, workspaces, presets) are translated where they are drawn. **Spanish covers the whole interface**: menus, panels, dialogs, the Settings pages, the Effects/Effect Controls registry, the Keyboard Shortcuts dialog and status messages; tests fail when a `tl!` literal, a menu label or a registry name has no Spanish entry. Searches match the displayed labels as well as English sources. Japanese and Brazilian Portuguese cover the core menus; untranslated labels use English. Engine error messages, the control channel, the CLI and MCP stay in English. Japanese text uses the Japanese fonts from [craft-fonts](https://github.com/storytold/craft-fonts) when FilmCraft is built with them (`CRAFT_FONTS_DIR`; all official releases, including the web build), otherwise a Japanese font already installed on the system; with neither, switching to Japanese is refused with a message. Existing vertical text support is preserved.
+120 -15
View File
@@ -4,6 +4,32 @@
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
/// `println!` that doesn't panic when stdout is closed (`filmcraft-cli commands | head`) with
/// "failed printing to stdout: Broken pipe (os error 32)". Once stdout has failed, later output is
/// dropped and the work goes on: see `stdout_failed`.
macro_rules! outln {
($($arg:tt)*) => {{
use std::io::Write as _;
if !$crate::stdout_gone() {
if let Err(e) = writeln!(std::io::stdout(), $($arg)*) {
$crate::stdout_failed(e);
}
}
}};
}
/// `print!` counterpart of `outln!`.
macro_rules! out {
($($arg:tt)*) => {{
use std::io::Write as _;
if !$crate::stdout_gone() {
if let Err(e) = write!(std::io::stdout(), $($arg)*) {
$crate::stdout_failed(e);
}
}
}};
}
mod args;
mod probe;
@@ -11,6 +37,28 @@ use args::{Args, parse_value};
use filmcraft_automation::BridgeClient;
use filmcraft_engine::Session;
use serde_json::{Value, json};
use std::sync::atomic::{AtomicBool, Ordering};
/// Set once a write to stdout has failed; later output is dropped.
static STDOUT_GONE: AtomicBool = AtomicBool::new(false);
/// Set when stdout failed for a reason other than a closed reader: the exit status becomes 1.
static STDOUT_ERROR: AtomicBool = AtomicBool::new(false);
fn stdout_gone() -> bool {
STDOUT_GONE.load(Ordering::Relaxed)
}
/// stdout went away. The program doesn't stop there: the rest of `exec`/`run`/`import` (later
/// script lines, `--save`, `--save-as`) still runs and the exit status still tells how it went, so
/// `exec … --save | head -1` keeps the edit. A reader that stopped early (a closed pipe) is not an
/// error, as in ripgrep; any other write error is reported and makes the exit status 1.
fn stdout_failed(e: std::io::Error) {
if STDOUT_GONE.swap(true, Ordering::Relaxed) || e.kind() == std::io::ErrorKind::BrokenPipe {
return;
}
eprintln!("filmcraft-cli: can't write to stdout: {e}");
STDOUT_ERROR.store(true, Ordering::Relaxed);
}
const HELP: &str = "\
filmcraft-cli: drive FilmCraft from the shell (headless engine, or the live app with --bridge)
@@ -27,12 +75,16 @@ SUBCOMMANDS
inspect [project|sequence] project tree or active sequence as JSON (default: both)
import <file>... import media into the project
export <out> [--preset name] [--format f] [--range r] [--start s --end s] [--settings json]
[--scale f] [--quality 0-100] [--no-audio] [--queue]
export the active sequence and wait for it to finish: with an
export preset (`export --list-presets`; built-in or the user's), or
a format (h264|prores|dnxhr|mjpeg|mxf-op1a|mxf-opatom|png|tiff|bmp|gif|wav|aiff, guessed
a format (h264|hevc|prores|dnxhr|apv|mjpeg|mxf-op1a|mxf-opatom|png|tiff|bmp|gif|wav|aiff, guessed
from the extension); --range entire|inOut|workArea, or a custom
range in seconds; --settings is ExportSettings JSON merged over the
preset; --queue adds to the export queue and runs it instead
preset; --scale renders at a fraction of the frame size (0.5 =
half); --quality 0-100 for the formats that take one; --no-audio
leaves the sound out; --queue adds to the export queue and runs it
instead
export --list-presets [query] list export presets (name, category, format) as JSON
render --seconds S --out f.png [--scale 0.5] render one Program frame to PNG
probe <media> [--image-sequence]
@@ -105,7 +157,8 @@ impl Backend {
async fn exec(&mut self, id: &str, params: Value) -> Result<Value, String> {
match self {
Backend::Local(s) => s.execute(id, params).map_err(|e| e.to_string()),
Backend::Bridge(b) => b.call("engine.execute", json!({"command": id, "params": params})).await.map_err(|e| e.to_string()),
// a blocking export runs as an app job polled by the client (#91, #92)
Backend::Bridge(b) => b.execute(id, params).await.map_err(|e| e.to_string()),
}
}
@@ -129,7 +182,7 @@ impl Backend {
fn print(a: &Args, v: &Value) {
let s = if a.flag("--compact") { serde_json::to_string(v) } else { serde_json::to_string_pretty(v) };
println!("{}", s.unwrap_or_default());
outln!("{}", s.unwrap_or_default());
}
/// Export format from the output file's extension.
@@ -149,19 +202,31 @@ fn format_for(path: &str) -> Option<&'static str> {
})
}
/// OS hardware video decoders (VideoToolbox on macOS, Media Foundation on Windows) in front of our
/// own, as in the desktop app. Also what `mcp` and the headless commands decode with.
fn register_hardware_decoders() -> filmcraft_platform::Availability {
filmcraft_platform::register()
}
#[tokio::main]
async fn main() {
cli().await;
if STDOUT_ERROR.load(Ordering::Relaxed) {
std::process::exit(1);
}
}
async fn cli() {
if matches!(std::env::args().nth(1).as_deref(), Some("--version" | "-V")) {
println!("filmcraft-cli {}", env!("CARGO_PKG_VERSION"));
outln!("filmcraft-cli {}", env!("CARGO_PKG_VERSION"));
return;
}
// OS hardware video decoders (VideoToolbox on macOS) in front of our own, as in the desktop app.
let _ = filmcraft_platform::register();
register_hardware_decoders();
let a = Args::parse(std::env::args().skip(1));
let Some(cmd) = a.pos(0) else { usage("missing subcommand") };
match cmd {
"help" | "--help" | "-h" => print!("{HELP}"),
"version" => println!("filmcraft-cli {}", env!("CARGO_PKG_VERSION")),
"help" | "--help" | "-h" => out!("{HELP}"),
"version" => outln!("filmcraft-cli {}", env!("CARGO_PKG_VERSION")),
"probe" => {
let path = a.pos(1).unwrap_or_else(|| usage("probe <media> [--image-sequence]"));
match probe::probe(path, a.flag("--image-sequence")) {
@@ -179,11 +244,11 @@ async fn main() {
src.video_frame(filmcraft_media::FrameRequest::full(rate.tick_of(f))).unwrap_or_else(|e| fail(e));
}
let dt = t0.elapsed().as_secs_f64();
println!("{n} frames in {dt:.2}s → {:.1} fps (sequential, via Mp4Source)", n as f64 / dt);
outln!("{n} frames in {dt:.2}s → {:.1} fps (sequential, via Mp4Source)", n as f64 / dt);
let t1 = std::time::Instant::now();
let f = src.video_frame(filmcraft_media::FrameRequest::full(rate.tick_of(n / 2))).unwrap_or_else(|e| fail(e));
let (w, h, _) = f.to_linear_f32_decimated(2);
println!("½-res linear conversion {w}x{h}: {:.1} ms", t1.elapsed().as_secs_f64() * 1000.0);
outln!("½-res linear conversion {w}x{h}: {:.1} ms", t1.elapsed().as_secs_f64() * 1000.0);
}
"commands" | "describe" => {
let mut b = Backend::open(&a);
@@ -209,7 +274,7 @@ async fn main() {
} else {
for c in &list {
let s = |k: &str| c[k].as_str().unwrap_or("").to_string();
println!("{:<32} {:<34} {:<14} {}", s("id"), s("label"), s("shortcut"), s("params"));
outln!("{:<32} {:<34} {:<14} {}", s("id"), s("label"), s("shortcut"), s("params"));
}
}
}
@@ -236,9 +301,9 @@ async fn main() {
let v: Value = serde_json::from_str(line).unwrap_or_else(|e| usage(format!("line {}: {e}", n + 1)));
let id = v["id"].as_str().unwrap_or_else(|| usage(format!("line {}: missing \"id\"", n + 1)));
match b.exec(id, v.get("params").cloned().unwrap_or(json!({}))).await {
Ok(r) => println!("{}", json!({"line": n + 1, "id": id, "ok": true, "result": r})),
Ok(r) => outln!("{}", json!({"line": n + 1, "id": id, "ok": true, "result": r})),
Err(e) => {
println!("{}", json!({"line": n + 1, "id": id, "ok": false, "error": e}));
outln!("{}", json!({"line": n + 1, "id": id, "ok": false, "error": e}));
failed += 1;
if !a.flag("--keep-going") {
std::process::exit(1);
@@ -334,7 +399,7 @@ async fn main() {
match r {
Ok(v) => {
print(&a, &v);
eprintln!("exported {out} in {:.1}s", t0.elapsed().as_secs_f64());
eprintln!("exported {} in {:.1}s", written_paths(&v, out).join(", "), t0.elapsed().as_secs_f64());
}
Err(e) => fail(format!("export: {e}")),
}
@@ -371,8 +436,48 @@ async fn main() {
}
}
/// The files an export wrote, from its result: `path` of a direct export, or each queued item's
/// `path` (the format may have changed the extension: `out.mp4` exported as APV is `out.mp4.mov`).
/// `requested` when the result names none.
fn written_paths(result: &Value, requested: &str) -> Vec<String> {
let paths: Vec<String> = match result.get("path").and_then(Value::as_str) {
Some(p) => vec![p.to_string()],
None => result
.get("items")
.and_then(Value::as_array)
.into_iter()
.flatten()
.filter_map(|i| i.get("path").and_then(Value::as_str))
.map(str::to_string)
.collect(),
};
if paths.is_empty() { vec![requested.to_string()] } else { paths }
}
#[cfg(test)]
mod format_tests {
/// The CLI (and MCP / headless runs, which share its entry point) registers the hardware
/// decoders at start-up.
#[test]
fn startup_registers_the_hardware_decoders() {
let hardware = super::register_hardware_decoders();
// always on macOS and Windows; on Linux when a VA-API driver is there
let expected = cfg!(any(target_os = "macos", target_os = "windows"))
|| (cfg!(target_os = "linux") && matches!(hardware, filmcraft_platform::Availability::Available(_)));
assert_eq!(filmcraft_platform::registered(), expected, "{hardware:?}");
}
#[test]
fn export_message_names_the_written_files() {
use serde_json::json;
let direct = json!({"job": 1, "path": "out-apv.mp4.mov", "result": {"path": "out-apv.mp4.mov"}});
assert_eq!(super::written_paths(&direct, "out-apv.mp4"), ["out-apv.mp4.mov"]);
let queued = json!({"added": [1, 2], "items": [{"path": "a.mp4.mov"}, {"path": "b.wav"}]});
assert_eq!(super::written_paths(&queued, "a.mp4"), ["a.mp4.mov", "b.wav"]);
assert_eq!(super::written_paths(&json!({}), "x.mov"), ["x.mov"]);
assert_eq!(super::written_paths(&json!({"items": []}), "x.mov"), ["x.mov"]);
}
#[test]
fn export_format_from_extension() {
assert_eq!(super::format_for("out/clip.MXF"), Some("mxf-op1a"));
+81 -1
View File
@@ -44,7 +44,7 @@ fn probe_reports_mpeg_transport_and_program_streams() {
assert!(st.success());
let v = json_out(&cli(&["probe", out.to_str().unwrap()]));
assert!(v["video"]["codec"].as_str().unwrap().starts_with("MPEG-2 Video"), "{v}");
assert_eq!(v["audio"]["codec"], "MPEG Audio");
assert_eq!(v["audio_streams"][0]["codec"], "MPEG Audio");
let streams = v["mpeg"]["streams"].as_array().unwrap();
assert_eq!(streams.len(), 2, "{v}");
assert_eq!(streams[0]["codec"], "MPEG-2 Video");
@@ -179,9 +179,89 @@ fn export_with_presets() {
let v = json_out(&cli(&["--demo", "--data-dir", data_s, "export", q.to_str().unwrap(), "--preset", "Tiny WAV", "--queue", "--range", "entire"]));
assert_eq!(v["items"][0]["status"], "done", "{v}");
assert!(q.exists());
// a format that changes the extension: the completion message names the file written (#341)
let apv = dir.join("apv.mp4");
let o =
cli(&["--demo", "--data-dir", data_s, "export", apv.to_str().unwrap(), "--preset", "APV 422-10", "--start", "0", "--end", "0.1", "--scale", "0.125"]);
let v = json_out(&o);
let written = v["path"].as_str().unwrap().to_string();
assert!(written.ends_with("apv.mp4.mov") && std::path::Path::new(&written).exists(), "{v}");
let msg = String::from_utf8_lossy(&o.stderr);
assert!(msg.contains(&format!("exported {written} in ")), "{msg}");
let qa = dir.join("queued-apv.mp4");
let o = cli(&[
"--demo",
"--data-dir",
data_s,
"export",
qa.to_str().unwrap(),
"--preset",
"APV 422-10",
"--queue",
"--start",
"0",
"--end",
"0.1",
"--scale",
"0.125",
]);
let v = json_out(&o);
let written = v["items"][0]["path"].as_str().unwrap().to_string();
assert!(written.ends_with("queued-apv.mp4.mov"), "{v}");
assert!(String::from_utf8_lossy(&o.stderr).contains(&format!("exported {written} in ")), "{}", String::from_utf8_lossy(&o.stderr));
// unknown preset: exit status 1 and a message naming it
let bad = cli(&["--demo", "--data-dir", data_s, "export", dir.join("x.mp4").to_str().unwrap(), "--preset", "No Such Preset"]);
assert_eq!(bad.status.code(), Some(1));
assert!(String::from_utf8_lossy(&bad.stderr).contains("No Such Preset"));
let _ = std::fs::remove_dir_all(&dir);
}
/// `filmcraft-cli commands | head -c 10` panicked with "failed printing to stdout: Broken pipe
/// (os error 32)" and exit status 101 (#285).
#[test]
fn closed_stdout_ends_quietly() {
use std::io::Read;
for args in [&["commands"][..], &["help"], &["--demo", "commands", "--json"]] {
let mut child = Command::new(env!("CARGO_BIN_EXE_filmcraft-cli")).args(args).stdout(Stdio::piped()).stderr(Stdio::piped()).spawn().unwrap();
let mut first = [0u8; 10];
child.stdout.as_mut().unwrap().read_exact(&mut first).unwrap();
drop(child.stdout.take());
let out = child.wait_with_output().unwrap();
let err = String::from_utf8_lossy(&out.stderr);
assert!(out.status.success() && err.is_empty(), "{args:?}: {:?}: {err}", out.status);
}
}
/// `run -` with stdout already closed: `run` reads its script to the end before it prints, so its
/// first write fails for sure. The rest of the script and the save still happen, and the exit status
/// still reports a failing command.
fn run_with_closed_stdout(args: &[&str], script: &[u8]) -> Output {
let mut child =
Command::new(env!("CARGO_BIN_EXE_filmcraft-cli")).args(args).stdin(Stdio::piped()).stdout(Stdio::piped()).stderr(Stdio::piped()).spawn().unwrap();
drop(child.stdout.take());
child.stdin.take().unwrap().write_all(script).unwrap();
child.wait_with_output().unwrap()
}
#[test]
fn closed_stdout_still_runs_the_script_and_saves() {
let dir = std::env::temp_dir().join(format!("fc-cli-pipe-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let proj = dir.join("p.fcproj");
let p = proj.to_str().unwrap();
let script = b"{\"id\":\"file.newBin\",\"params\":{\"name\":\"Piped One\"}}\n{\"id\":\"file.newBin\",\"params\":{\"name\":\"Piped Two\"}}\n";
let out = run_with_closed_stdout(&["--demo", "--save-as", p, "run", "-"], script);
assert!(out.status.success(), "{:?}: {}", out.status, String::from_utf8_lossy(&out.stderr));
let tree = json_out(&cli(&["--project", p, "inspect", "project"])).to_string();
assert!(tree.contains("Piped One") && tree.contains("Piped Two"), "{tree}");
// a failing command is still a failure, whether or not anyone reads stdout
let out = run_with_closed_stdout(&["--demo", "run", "-"], b"{\"id\":\"no.such.command\"}\n");
assert_eq!(out.status.code(), Some(1));
let out = run_with_closed_stdout(
&["--demo", "--keep-going", "run", "-"],
b"{\"id\":\"no.such.command\"}\n{\"id\":\"file.newBin\",\"params\":{\"name\":\"x\"}}\n",
);
assert_eq!(out.status.code(), Some(1));
let _ = std::fs::remove_dir_all(&dir);
}
+22 -6
View File
@@ -170,11 +170,11 @@ pub fn install() -> Result<(), JsValue> {
&obj,
"readFile",
Closure::<dyn Fn(String) -> js_sys::Promise>::new(|path: String| {
future_to_promise(async move {
future_to_promise(caught(async move {
let blob = crate::fs::blob(&path).ok_or_else(|| JsValue::from_str(&format!("{path}: not found")))?;
let buf = JsFuture::from(blob.array_buffer()).await?;
Ok(js_sys::Uint8Array::new(&buf).into())
})
}))
})
.into_js_value(),
)?;
@@ -191,10 +191,10 @@ pub fn install() -> Result<(), JsValue> {
&obj,
"importUrl",
Closure::<dyn Fn(String, JsValue) -> js_sys::Promise>::new(|url: String, name: JsValue| {
future_to_promise(async move {
future_to_promise(caught(async move {
let f = fetch_file(&url, name.as_string()).await?;
Ok(to_js(&crate::import::import_files(vec![f]).await))
})
}))
})
.into_js_value(),
)?;
@@ -202,13 +202,13 @@ pub fn install() -> Result<(), JsValue> {
&obj,
"openProject",
Closure::<dyn Fn(JsValue) -> js_sys::Promise>::new(|src: JsValue| {
future_to_promise(async move {
future_to_promise(caught(async move {
let file = match src.as_string() {
Some(url) => fetch_file(&url, None).await?,
None => files_of(&src).into_iter().next().ok_or("need a File or URL")?,
};
crate::import::open_project_file(file).await.map(|v| to_js(&v)).map_err(|e| JsValue::from(js_sys::Error::new(&e)))
})
}))
})
.into_js_value(),
)?;
@@ -216,3 +216,19 @@ pub fn install() -> Result<(), JsValue> {
js_sys::Reflect::set(&w, &"filmcraft".into(), &obj)?;
Ok(())
}
/// `f`, rejecting with an [`as_error`] `Error`.
async fn caught(f: impl std::future::Future<Output = Result<JsValue, JsValue>>) -> Result<JsValue, JsValue> {
f.await.map_err(as_error)
}
/// Every member rejects with an `Error` (#90), as `execute` and `request` do, so `e.message` and
/// `e instanceof Error` work: a plain string (a fetch status, "not found") becomes one, and an
/// `Error` passes through unchanged.
fn as_error(e: JsValue) -> JsValue {
if e.is_instance_of::<js_sys::Error>() {
return e;
}
let msg = e.as_string().unwrap_or_else(|| format!("{e:?}"));
js_sys::Error::new(&msg).into()
}
+5 -13
View File
@@ -70,9 +70,8 @@ fn file_name(path: &str) -> &str {
path.rsplit('/').next().unwrap_or(path)
}
/// Register a browser `File` (or `Blob` with a name) and return its virtual path. A file with
/// the same name and size as an existing entry replaces it (the same file picked again keeps
/// its path, so projects referencing it come back online); otherwise a numbered name is used.
/// Register a browser `File` (or named `Blob`) at an unused virtual path; collisions get numbered names.
/// Name + size do not identify content: replacing an occupied path changes every project item referencing it.
pub fn register_blob(name: &str, blob: web_sys::Blob) -> String {
let size = blob.size() as u64;
let clean: String = name.chars().map(|c| if c == '/' || c == '\\' { '_' } else { c }).collect();
@@ -85,18 +84,11 @@ pub fn register_blob(name: &str, blob: web_sys::Blob) -> String {
let mut n = 1;
let path = loop {
let p = if n == 1 { format!("/files/{clean}") } else { format!("/files/{stem} ({n}){ext}") };
match fs.entries.get(&p) {
None => break p,
Some(Entry::Blob { size: s, .. }) if *s == size => break p,
Some(Entry::Mem(b)) if b.len() as u64 == size => break p,
_ => n += 1,
if !fs.entries.contains_key(&p) {
break p;
}
n += 1;
};
if let Some(Entry::Blob { id, .. }) = fs.entries.get(&path) {
let id = *id;
fs.blobs.remove(&id);
fs.chunks.retain(|k, _| k.0 != id);
}
let id = fs.next_id;
fs.next_id += 1;
fs.blobs.insert(id, blob);
+47 -6
View File
@@ -10,7 +10,7 @@ use std::cell::RefCell;
use std::rc::Rc;
use filmcraft_engine::Services;
use filmcraft_ui_egui::FilmcraftApp;
use filmcraft_ui_egui::{FilmcraftApp, RelinkHint};
use serde_json::{Value, json};
use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
@@ -81,12 +81,12 @@ async fn prewarm(path: &str, size: u64) {
}
/// `file.import` one path, retried while its bytes are loading.
async fn import_path(path: String) -> Result<Value, String> {
async fn import_path(path: String, bin: u64) -> Result<Value, String> {
for _ in 0..2000 {
let p = path.clone();
let (r, pending) = on_ui(move |w, _| {
let _ = filmcraft_media::pending::take();
let r = w.app.session.execute("file.import", json!({"paths": [p]})).map_err(|e| e.to_string());
let r = w.app.session.execute("file.import", json!({"paths": [p], "bin": bin})).map_err(|e| e.to_string());
(r, filmcraft_media::pending::take())
})
.await;
@@ -105,6 +105,8 @@ async fn import_path(path: String) -> Result<Value, String> {
/// Import browser files (no copies): returns `{items, errors, paths}`.
pub async fn import_files(files: Vec<web_sys::File>) -> Value {
// Capture once per batch: navigation while prewarming must not redirect pending imports.
let bin = on_ui(|w, _| w.app.import_bin().0).await;
let mut items = Vec::new();
let mut errors = Vec::new();
let mut paths = Vec::new();
@@ -113,7 +115,7 @@ pub async fn import_files(files: Vec<web_sys::File>) -> Value {
let size = f.size() as u64;
let path = fs::register_blob(&name, f.clone().into());
prewarm(&path, size).await;
match import_path(path.clone()).await {
match import_path(path.clone(), bin).await {
Ok(v) => {
items.extend(v.get("items").and_then(Value::as_array).cloned().unwrap_or_default());
errors.extend(v.get("errors").and_then(Value::as_array).cloned().unwrap_or_default());
@@ -153,12 +155,43 @@ pub fn take_files(files: Vec<web_sys::File>) {
});
}
/// Files a picker returned: imported, or with a relink hint, the first one is registered under
/// `/files/<name>` and the hint's command runs on it (#111).
fn picked(files: Vec<web_sys::File>, relink: Option<RelinkHint>) {
let Some(hint) = relink else { return take_files(files) };
let Some(f) = files.into_iter().next() else { return };
wasm_bindgen_futures::spawn_local(async move {
let path = fs::register_blob(&f.name(), f.clone().into());
crate::recovery::keep_media(&path, &f);
let mut params = hint.params;
if !params.is_object() {
params = json!({});
}
params["path"] = json!(path);
let command = hint.command;
let outcome = on_ui(move |w, _| w.app.session.execute(&command, params).map_err(|e| e.to_string())).await;
crate::post(move |w, _| match outcome {
Ok(v) => {
if let Some(n) = v.get("relinked").and_then(Value::as_array).map(Vec::len) {
w.app.status(format!("Linked {n} clip(s)"));
}
}
Err(e) => w.app.status(e),
});
});
}
fn has_fsa_picker() -> bool {
web_sys::window().is_some_and(|w| js_sys::Reflect::has(&w, &"showOpenFilePicker".into()).unwrap_or(false))
}
/// Let the user pick files: `showOpenFilePicker` when available, else a file input.
pub fn pick(exts: &[&str], multiple: bool) {
pick_then(exts, multiple, None);
}
/// [`pick`], handing the files to [`picked`]: a cancelled picker leaves nothing pending.
fn pick_then(exts: &[&str], multiple: bool, relink: Option<RelinkHint>) {
let accept: Vec<String> = exts.iter().map(|e| format!(".{e}")).collect();
if has_fsa_picker() {
let opts = js_sys::Object::new();
@@ -178,7 +211,7 @@ pub fn pick(exts: &[&str], multiple: bool) {
files.push(f);
}
}
take_files(files);
picked(files, relink);
});
return;
}
@@ -190,7 +223,7 @@ pub fn pick(exts: &[&str], multiple: bool) {
let inp = input.clone();
let on_change = Closure::<dyn FnMut()>::new(move || {
if let Some(list) = inp.files() {
take_files((0..list.length()).filter_map(|i| list.get(i)).collect());
picked((0..list.length()).filter_map(|i| list.get(i)).collect(), relink.clone());
}
});
input.set_onchange(Some(on_change.as_ref().unchecked_ref()));
@@ -231,4 +264,12 @@ pub fn install_hooks(app: &mut FilmcraftApp) {
// Saving writes the file in memory and offers it as a download (`fs::WebServices`).
app.hooks.pick_save = Some(Box::new(|name: &str| Some(format!("/projects/{name}"))));
app.hooks.pick_save_as = Some(Box::new(|_filter: &str, _exts: &[&str], name: &str| Some(format!("/exports/{name}"))));
// Link Media ▸ Locate…, Attach Proxies, Reconnect Full Resolution: the picker is async, so the
// hint's command runs once the user has chosen (#111).
app.hooks.pick_file_for_relink = Some(Box::new(|exts: &[&str], hint: Option<RelinkHint>| {
if hint.is_some() {
pick_then(exts, false, hint);
}
None
}));
}
+12 -1
View File
@@ -24,6 +24,7 @@ pub mod fs;
pub mod import;
pub mod opfs;
pub mod recovery;
pub mod recovery_policy;
pub mod webcodecs;
use std::cell::RefCell;
@@ -58,6 +59,13 @@ pub fn repaint() {
});
}
/// Give a host that builds its own [`WebApp`] the egui context that [`post`] and [`repaint`] wake.
/// [`start`] calls it; an embedding app that runs `WebApp` in its own eframe runner calls it once
/// from its app creator, or posted actions and JS API requests wait for the next input event.
pub fn set_context(ctx: egui::Context) {
CTX.with(|c| *c.borrow_mut() = Some(ctx));
}
/// Environment facts reported by `filmcraft.info()` (backend, isolation, codecs…).
pub fn set_info(key: &str, v: serde_json::Value) {
INFO.with(|i| {
@@ -150,7 +158,7 @@ pub async fn start(canvas_id: String) -> Result<(), JsValue> {
canvas,
web_options,
Box::new(move |cc| {
CTX.with(|c| *c.borrow_mut() = Some(cc.egui_ctx.clone()));
set_context(cc.egui_ctx.clone());
let ctx = cc.egui_ctx.clone();
fs::set_on_data(move || ctx.request_repaint());
fs::set_on_write(|path, data| {
@@ -177,6 +185,9 @@ pub async fn start(canvas_id: String) -> Result<(), JsValue> {
}
let mut app = FilmcraftApp::new(session);
import::install_hooks(&mut app);
// Settings ▸ General ▸ Interface Language ▸ System Language (#218): the browser's languages.
app.hooks.system_languages =
Some(Box::new(|| web_sys::window().map(|w| w.navigator().languages().iter().filter_map(|v| v.as_string()).collect()).unwrap_or_default()));
let (tx, rx) = std::sync::mpsc::channel();
api::set_sender(tx);
app = app.with_control(rx);
+21 -4
View File
@@ -10,8 +10,13 @@ use wasm_bindgen::{JsCast, JsValue};
use wasm_bindgen_futures::JsFuture;
use web_sys::{FileSystemDirectoryHandle, FileSystemFileHandle};
/// Called with whether a queued write reached OPFS.
pub type Done = Box<dyn FnOnce(bool)>;
thread_local! {
static QUEUE: RefCell<(BTreeMap<String, Vec<u8>>, bool)> = const { RefCell::new((BTreeMap::new(), false)) };
/// Pending writes per path (bytes and the callbacks of every write coalesced into them), and
/// whether the writer task is running.
static QUEUE: RefCell<(BTreeMap<String, (Vec<u8>, Vec<Done>)>, bool)> = const { RefCell::new((BTreeMap::new(), false)) };
}
/// Whether this browser has OPFS (`navigator.storage.getDirectory`).
@@ -60,9 +65,17 @@ async fn write_now(rel: &str, data: &[u8]) -> Result<(), JsValue> {
/// Queue a write of `data` to `rel` (relative to the OPFS root).
pub fn write(rel: String, data: Vec<u8>) {
write_then(rel, data, None);
}
/// [`write`], then call `done` with whether it succeeded. A write replaced by a newer one to the
/// same path reports the newer one's result.
pub fn write_then(rel: String, data: Vec<u8>, done: Option<Done>) {
let start = QUEUE.with(|q| {
let mut q = q.borrow_mut();
q.0.insert(rel, data);
let e = q.0.entry(rel).or_default();
e.0 = data;
e.1.extend(done);
!std::mem::replace(&mut q.1, true)
});
if !start {
@@ -81,10 +94,14 @@ pub fn write(rel: String, data: Vec<u8>) {
}
}
});
let Some((rel, data)) = next else { break };
if let Err(e) = write_now(&rel, &data).await {
let Some((rel, (data, done))) = next else { break };
let r = write_now(&rel, &data).await;
if let Err(e) = &r {
log::warn!("OPFS write {rel}: {e:?}");
}
for d in done {
d(r.is_ok());
}
}
});
}
+48 -18
View File
@@ -6,24 +6,32 @@
//! - Imported media are copied to `media/<name>` (the browser streams the copy; nothing goes
//! through wasm memory), so the snapshot's `/files/<name>` references resolve after a reload.
//! - At start-up a dirty snapshot is reopened (`?norecover` skips it, `?fresh` also skips the media).
//! A skipped snapshot stays dirty until this session writes its own ([`crate::recovery_policy`]).
use std::cell::{Cell, RefCell};
use std::rc::Rc;
use filmcraft_engine::Session;
use serde_json::{Value, json};
use crate::opfs;
use crate::recovery_policy::{Action, Policy};
const SNAPSHOT: &str = "recovery/snapshot.fcproj";
const META: &str = "recovery/meta.json";
/// Seconds between snapshots of a changing project.
const INTERVAL_S: f64 = 5.0;
/// Media larger than this are not copied to OPFS.
const KEEP_MAX: f64 = 4.0 * 1024.0 * 1024.0 * 1024.0;
/// A finished OPFS write, reported back to the next [`Autosave::tick`].
enum Outcome {
Snapshot { revision: u64, ok: bool },
Clean { ok: bool },
}
#[derive(Default)]
pub struct Autosave {
last_revision: u64,
last_time: f64,
last_dirty: Option<bool>,
policy: Policy,
outcomes: Rc<RefCell<Vec<Outcome>>>,
}
impl Autosave {
@@ -33,22 +41,44 @@ impl Autosave {
return;
}
let now = ctx.input(|i| i.time);
let dirty = s.revision != s.saved_revision;
if dirty {
if s.revision != self.last_revision && now - self.last_time >= INTERVAL_S {
opfs::write(SNAPSHOT.into(), filmcraft_format::encode(&s.project, false));
for o in self.outcomes.borrow_mut().drain(..) {
match o {
Outcome::Snapshot { revision, ok } => self.policy.written(revision, ok, now),
Outcome::Clean { ok } => self.policy.cleaned(ok, now),
}
}
match self.policy.tick(now, s.revision, s.saved_revision) {
Action::Nothing => {}
Action::WakeAfter(secs) => ctx.request_repaint_after(std::time::Duration::from_secs_f64(secs)),
Action::WriteSnapshot(revision) => {
// Snapshot and meta both have to land; either failing retries the pair.
let failed = Rc::new(Cell::new(false));
let left = Rc::new(Cell::new(2u8));
let done = |out: Rc<RefCell<Vec<Outcome>>>, failed: Rc<Cell<bool>>, left: Rc<Cell<u8>>, ctx: egui::Context| -> opfs::Done {
Box::new(move |ok| {
failed.set(failed.get() || !ok);
left.set(left.get().saturating_sub(1));
if left.get() == 0 {
out.borrow_mut().push(Outcome::Snapshot { revision, ok: !failed.get() });
ctx.request_repaint();
}
})
};
let meta = json!({"dirty": true, "name": s.project.name, "revision": s.revision, "time": js_sys::Date::now()});
opfs::write(META.into(), meta.to_string().into_bytes());
self.last_revision = s.revision;
self.last_time = now;
} else if s.revision != self.last_revision {
// the UI may go idle: wake up when the snapshot is due
ctx.request_repaint_after(std::time::Duration::from_secs_f64((INTERVAL_S - (now - self.last_time)).max(0.05)));
let d1 = done(self.outcomes.clone(), failed.clone(), left.clone(), ctx.clone());
let d2 = done(self.outcomes.clone(), failed, left, ctx.clone());
opfs::write_then(SNAPSHOT.into(), filmcraft_format::encode(&s.project, false), Some(d1));
opfs::write_then(META.into(), meta.to_string().into_bytes(), Some(d2));
}
Action::WriteClean => {
let (out, ctx) = (self.outcomes.clone(), ctx.clone());
let done: opfs::Done = Box::new(move |ok| {
out.borrow_mut().push(Outcome::Clean { ok });
ctx.request_repaint();
});
opfs::write_then(META.into(), json!({"dirty": false}).to_string().into_bytes(), Some(done));
}
} else if self.last_dirty != Some(false) {
opfs::write(META.into(), json!({"dirty": false}).to_string().into_bytes());
}
self.last_dirty = Some(dirty);
}
}
+88
View File
@@ -0,0 +1,88 @@
//! When the web recovery snapshot is written or marked clean (pure, so it is tested natively in
//! `tests/recovery_policy.rs`; the OPFS side lives in [`crate::recovery`]).
//!
//! - A dirty project is snapshotted at most every [`INTERVAL_S`] seconds, once per revision. A write
//! that fails is retried after the interval, without waiting for another edit (#197).
//! - The meta is marked clean only to retire a snapshot this session wrote. A snapshot left by an
//! earlier session that was not reopened (`?norecover`, `?fresh`) stays dirty (#193).
/// Seconds between snapshots of a changing project.
pub const INTERVAL_S: f64 = 5.0;
/// What the caller should do this frame.
#[derive(Clone, Copy, Debug, PartialEq)]
pub enum Action {
Nothing,
/// Write the snapshot and a dirty meta for this revision, then report with [`Policy::written`].
WriteSnapshot(u64),
/// Write a clean meta, then report with [`Policy::cleaned`].
WriteClean,
/// Nothing now; wake up after this many seconds.
WakeAfter(f64),
}
#[derive(Debug, Default)]
pub struct Policy {
/// Revision whose snapshot was last written or is being written.
last_revision: Option<u64>,
last_time: Option<f64>,
/// This session wrote a dirty meta that is not yet retired.
owns_dirty_meta: bool,
/// A clean-meta write is in flight.
cleaning: bool,
/// Snapshot writes started so far, and how many had started when the clean write did (a
/// snapshot started after it makes the meta dirty again).
snapshots: u64,
snapshots_at_clean: u64,
}
impl Policy {
/// Called every frame with the session's revisions and the time in seconds.
pub fn tick(&mut self, now: f64, revision: u64, saved_revision: u64) -> Action {
if revision != saved_revision {
if self.last_revision == Some(revision) {
return Action::Nothing;
}
let since = self.last_time.map_or(f64::INFINITY, |t| now - t);
if since >= INTERVAL_S {
self.last_revision = Some(revision);
self.last_time = Some(now);
self.owns_dirty_meta = true;
self.snapshots += 1;
return Action::WriteSnapshot(revision);
}
return Action::WakeAfter((INTERVAL_S - since).max(0.05));
}
if self.owns_dirty_meta && !self.cleaning {
let since = self.last_time.map_or(f64::INFINITY, |t| now - t);
if self.last_revision.is_some() || since >= INTERVAL_S {
self.cleaning = true;
self.snapshots_at_clean = self.snapshots;
return Action::WriteClean;
}
return Action::WakeAfter((INTERVAL_S - since).max(0.05));
}
Action::Nothing
}
/// The snapshot write for `revision` finished.
pub fn written(&mut self, revision: u64, ok: bool, now: f64) {
if !ok && self.last_revision == Some(revision) {
// retry after the interval, even if nothing else changes
self.last_revision = None;
self.last_time = Some(now);
}
}
/// The clean-meta write finished.
pub fn cleaned(&mut self, ok: bool, now: f64) {
self.cleaning = false;
if ok {
self.owns_dirty_meta = self.snapshots != self.snapshots_at_clean;
} else {
// retry after the interval
self.last_revision = None;
self.last_time = Some(now);
}
}
}
+111 -21
View File
@@ -12,8 +12,7 @@
//! the job on a later frame, when the decoder's output callback has delivered the picture.
//! - A session that has not delivered the frame it was started for is not restarted by other
//! requests (thumbnails and the monitor would otherwise keep resetting each other).
//! - Outputs are converted to RGBA by drawing the `VideoFrame` on an `OffscreenCanvas` (the
//! browser applies the stream's colour description).
//! - Native YUV planes feed the desktop frame/compositor contract. Unsupported browser formats or geometry use Canvas RGBA.
//! - Any decoder error or unsupported configuration switches the source to our decoder.
//!
//! `?nowebcodecs` disables it. Support is probed at start-up with `VideoDecoder.isConfigSupported`.
@@ -34,6 +33,20 @@ use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
use wasm_bindgen_futures::JsFuture;
mod pixels;
#[derive(Default)]
struct Transfers {
native: u64,
canvas: u64,
closed: u64,
stale: u64,
bytes: u64,
native_ms: f64,
canvas_ms: f64,
reasons: BTreeMap<&'static str, u64>,
}
/// Samples fed past the wanted one (decoders hold pictures for reordering).
const LOOKAHEAD: usize = 8;
/// Decoded frames kept per source.
@@ -52,6 +65,7 @@ thread_local! {
static DECODED: Cell<u64> = const { Cell::new(0) };
static FALLBACKS: Cell<u64> = const { Cell::new(0) };
static SOURCES: Cell<u64> = const { Cell::new(0) };
static TRANSFERS: RefCell<Transfers> = RefCell::new(Transfers::default());
}
fn now_ms() -> f64 {
@@ -160,11 +174,15 @@ pub fn stats() -> serde_json::Value {
.iter()
.map(|(id, s)| {
json!({"id": id, "start": s.start, "next": s.next, "target": s.target, "needsKey": s.needs_key, "flushing": s.flushing.get(),
"errored": s.errored.get(), "queue": s.queue(), "idleMs": now_ms() - s.last_output.get()})
"errored": s.errored.get(), "queue": s.queue(), "copies": s.copying.get(), "idleMs": now_ms() - s.last_output.get()})
})
.collect()
});
json!({"sources": SOURCES.with(Cell::get), "framesDecoded": DECODED.with(Cell::get), "fallbacks": FALLBACKS.with(Cell::get), "sessions": sessions})
let transfers = TRANSFERS.with(|s| {
let s = s.borrow();
json!({"nativeFrames":s.native,"canvasFrames":s.canvas,"closedFrames":s.closed,"staleFrames":s.stale,"nativeBytes":s.bytes,"nativeMs":s.native_ms,"canvasMs":s.canvas_ms,"canvasReasons":s.reasons})
});
json!({"sources": SOURCES.with(Cell::get), "framesDecoded": DECODED.with(Cell::get), "fallbacks": FALLBACKS.with(Cell::get), "sessions": sessions,"transfers":transfers})
}
/// A [`SharedReader`] as the demuxer's byte source.
@@ -210,6 +228,8 @@ pub fn reader_opener(name: &str, head: &[u8], reader: &SharedReader) -> Option<R
});
SOURCES.with(|c| c.set(c.get() + 1));
log::info!("{name}: video decoded with WebCodecs ({codec})");
let color = info.video.as_ref().map(|v| v.color).unwrap_or_default();
let par = info.video.as_ref().map(|v| v.par).unwrap_or((1, 1));
Some(Ok(Arc::new(WcSource {
inner,
info,
@@ -217,11 +237,12 @@ pub fn reader_opener(name: &str, head: &[u8], reader: &SharedReader) -> Option<R
file,
track,
id,
config: Config { codec, description, width: w, height: h, rotation },
config: Config { codec, description, width: w, height: h, rotation, color, par },
state: Mutex::new(Cache::default()),
})))
}
#[derive(Clone)]
struct Config {
codec: String,
description: Option<Vec<u8>>,
@@ -229,6 +250,8 @@ struct Config {
height: u32,
/// Clockwise quarter turns from the track matrix, applied to decoded frames.
rotation: u8,
color: filmcraft_color::ColorInfo,
par: (u32, u32),
}
#[derive(Default)]
@@ -253,6 +276,9 @@ struct Session {
/// Highest presentation time output since the session (re)started.
out_max: Rc<Cell<i64>>,
out: Rc<RefCell<Vec<(i64, VideoFrame)>>>,
copying: Rc<Cell<u32>>,
generation: Rc<Cell<u64>>,
alive: Rc<Cell<bool>>,
_closures: (Closure<dyn FnMut(JsValue)>, Closure<dyn FnMut(JsValue)>),
}
@@ -286,17 +312,65 @@ impl Session {
let errored = Rc::new(Cell::new(false));
let last_output = Rc::new(Cell::new(now_ms()));
let out_max = Rc::new(Cell::new(i64::MIN));
let copying = Rc::new(Cell::new(0u32));
let generation = Rc::new(Cell::new(0u64));
let alive = Rc::new(Cell::new(true));
let (o, lo, om) = (out.clone(), last_output.clone(), out_max.clone());
let (copies, epoch, active, config, errors) = (copying.clone(), generation.clone(), alive.clone(), cfg.clone(), errored.clone());
let on_output = Closure::<dyn FnMut(JsValue)>::new(move |frame: JsValue| {
let ts = get(&frame, "timestamp").as_f64().unwrap_or(0.0) as i64;
om.set(om.get().max(ts));
if let Some(f) = to_rgba(&frame) {
o.borrow_mut().push((ts, f));
DECODED.with(|c| c.set(c.get() + 1));
}
let _ = call(&frame, "close", &[]);
lo.set(now_ms());
crate::repaint();
let expected = epoch.get();
let (o, lo, om, copies, epoch, active, config, errors) =
(o.clone(), lo.clone(), om.clone(), copies.clone(), epoch.clone(), active.clone(), config.clone(), errors.clone());
copies.set(copies.get().saturating_add(1));
wasm_bindgen_futures::spawn_local(async move {
let begin = now_ms();
let converted = match pixels::read(&frame, &config).await {
Ok((f, bytes)) => {
TRANSFERS.with(|s| {
let mut s = s.borrow_mut();
s.native = s.native.saturating_add(1);
s.bytes = s.bytes.saturating_add(bytes as u64);
s.native_ms += now_ms() - begin;
});
Some(f)
}
Err(reason) => {
let begin = now_ms();
let f = to_rgba(&frame);
TRANSFERS.with(|s| {
let mut s = s.borrow_mut();
s.canvas = s.canvas.saturating_add(1);
s.canvas_ms += now_ms() - begin;
let count = s.reasons.entry(reason).or_default();
*count = count.saturating_add(1);
});
f
}
};
let _ = call(&frame, "close", &[]);
copies.set(copies.get().saturating_sub(1));
TRANSFERS.with(|s| {
let mut s = s.borrow_mut();
s.closed = s.closed.saturating_add(1);
});
if !active.get() || epoch.get() != expected {
TRANSFERS.with(|s| {
let mut s = s.borrow_mut();
s.stale = s.stale.saturating_add(1);
});
return;
}
if let Some(f) = converted {
o.borrow_mut().push((ts, f));
om.set(om.get().max(ts));
DECODED.with(|c| c.set(c.get().saturating_add(1)));
} else {
errors.set(true);
}
lo.set(now_ms());
crate::repaint();
});
});
let e = errored.clone();
let on_error = Closure::<dyn FnMut(JsValue)>::new(move |err: JsValue| {
@@ -317,6 +391,9 @@ impl Session {
last_output,
out_max,
out,
copying,
generation,
alive,
_closures: (on_output, on_error),
};
s.configure(cfg)?;
@@ -341,6 +418,14 @@ impl Session {
}
}
impl Drop for Session {
fn drop(&mut self) {
self.alive.set(false);
self.generation.set(self.generation.get().wrapping_add(1));
let _ = call(&self.decoder, "close", &[]);
}
}
/// An MP4/MOV source whose video is decoded by the browser.
pub struct WcSource {
inner: SharedSource,
@@ -388,12 +473,15 @@ impl WcSource {
// already output but the cache dropped needs a restart).
let covers = !s.needs_key && s.start <= i && key <= s.next && (i >= s.next || want > s.out_max.get());
if !covers {
// Don't interrupt a session that is still working towards its own frame.
let busy = !s.needs_key && s.target != i64::MIN && s.out_max.get() < s.target && now_ms() - s.last_output.get() < STALE_MS;
// Don't interrupt a session that is still working towards its own frame, unless it
// has gone quiet for STALE_MS (a copyTo that never settles must not wedge it).
let fresh = now_ms() - s.last_output.get() < STALE_MS;
let busy = fresh && (s.copying.get() > 0 || (!s.needs_key && s.target != i64::MIN && s.out_max.get() < s.target));
if busy {
return Drive::Waiting;
}
if !s.needs_key || s.next > 0 {
s.generation.set(s.generation.get().wrapping_add(1));
let _ = call(&s.decoder, "reset", &[]);
if s.configure(&self.config).is_err() {
return Drive::Failed;
@@ -410,7 +498,7 @@ impl WcSource {
s.target = want;
}
let upto = (i + LOOKAHEAD).min(n - 1);
while s.next <= upto && s.queue() < MAX_QUEUE {
while s.next <= upto && s.queue().saturating_add(s.copying.get()) < MAX_QUEUE {
let smp = &t.samples[s.next];
let mut data = vec![0u8; smp.size as usize];
match self.reader.read_at(smp.offset, &mut data) {
@@ -480,10 +568,8 @@ impl WcSource {
impl Drop for WcSource {
fn drop(&mut self) {
SESSIONS.with(|ss| {
if let Ok(mut ss) = ss.try_borrow_mut()
&& let Some(s) = ss.remove(&self.id)
{
let _ = call(&s.decoder, "close", &[]);
if let Ok(mut ss) = ss.try_borrow_mut() {
ss.remove(&self.id);
}
});
}
@@ -502,7 +588,8 @@ impl MediaSource for WcSource {
}
let t = &self.file.tracks[self.track];
let n = t.samples.len();
let target = req.time.max(Tick::ZERO).to_rational_floor(1, t.timescale.max(1) as i64);
// nearest, as `Mp4Source` looks frames up (sample times are rounded to the timescale)
let target = req.time.max(Tick::ZERO).to_rational_round(1, t.timescale.max(1) as i64);
let i = t.sample_at_presentation_time(target).unwrap_or(n - 1).min(n - 1);
let want = t.samples[i].pts;
self.collect(&mut c, want);
@@ -531,4 +618,7 @@ impl MediaSource for WcSource {
fn audio(&self, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
self.inner.audio(start, frames, sample_rate)
}
fn audio_stream(&self, stream: usize, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
self.inner.audio_stream(stream, start, frames, sample_rate)
}
}
+148
View File
@@ -0,0 +1,148 @@
//! Browser planes -> core frames; no color conversion or chroma reconstruction here.
use std::sync::Arc;
use filmcraft_color::{ColorInfo, Matrix, Primaries, Range, Transfer};
use filmcraft_frame::{Chroma, PixelData, VideoFrame};
use filmcraft_time::Tick;
use wasm_bindgen::{JsCast, JsValue};
use wasm_bindgen_futures::JsFuture;
use super::{Config, call, get, obj};
// One 8K 4:4:4 16-bit picture + alpha fits within 256 MiB; cap browser-driven allocations.
const MAX_COPY_BYTES: u32 = 256 * 1024 * 1024;
fn integer(value: JsValue) -> Option<u32> {
let n = value.as_f64()?;
if n.is_finite() && n >= 0.0 && n <= u32::MAX as f64 && n.fract() == 0.0 { Some(n as u32) } else { None }
}
fn color(frame: &JsValue, fallback: ColorInfo) -> Option<ColorInfo> {
let c = get(frame, "colorSpace");
let matrix = match get(&c, "matrix").as_string().as_deref() {
Some("bt709") => Matrix::Bt709,
Some("bt470bg" | "smpte170m") => Matrix::Bt601,
Some("bt2020-ncl") => Matrix::Bt2020Ncl,
None => fallback.matrix,
_ => return None,
};
let transfer = match get(&c, "transfer").as_string().as_deref() {
Some("bt709") => Transfer::Bt709,
Some("iec61966-2-1") => Transfer::Srgb,
Some("smpte2084") => Transfer::Pq,
Some("arib-std-b67") => Transfer::Hlg,
None => fallback.transfer,
_ => return None,
};
let primaries = match get(&c, "primaries").as_string().as_deref() {
Some("bt709") => Primaries::Bt709,
Some("bt470bg") => Primaries::Bt601_625,
Some("smpte170m") => Primaries::Bt601_525,
Some("bt2020") => Primaries::Bt2020,
None => fallback.primaries,
_ => return None,
};
let range = match get(&c, "fullRange").as_bool() {
Some(true) => Range::Full,
Some(false) => Range::Limited,
None => fallback.range,
};
Some(ColorInfo { matrix, transfer, primaries, range })
}
fn plane(raw: &js_sys::Uint8Array, layout: &JsValue, width: u32, height: u32) -> Result<Vec<u8>, &'static str> {
let offset = integer(get(layout, "offset")).ok_or("layout")?;
let stride = integer(get(layout, "stride")).filter(|s| *s >= width).ok_or("layout")?;
let end =
height.checked_sub(1).and_then(|h| h.checked_mul(stride)).and_then(|n| offset.checked_add(n)).and_then(|n| n.checked_add(width)).ok_or("layout")?;
if end > raw.length() {
return Err("layout");
}
let len = width.checked_mul(height).filter(|n| *n <= MAX_COPY_BYTES).ok_or("allocation")? as usize;
let mut out = Vec::new();
out.try_reserve_exact(len).map_err(|_| "memory")?;
out.resize(len, 0);
if stride == width {
raw.subarray(offset, end).copy_to(&mut out);
} else {
for (y, row) in out.chunks_exact_mut(width as usize).enumerate() {
// Checked for the last row above; earlier rows are smaller, but stay checked anyway.
let start = u32::try_from(y).ok().and_then(|y| y.checked_mul(stride)).and_then(|n| offset.checked_add(n)).ok_or("layout")?;
let row_end = start.checked_add(width).filter(|e| *e <= end).ok_or("layout")?;
raw.subarray(start, row_end).copy_to(row);
}
}
Ok(out)
}
/// Copy native 8-bit YUV planes; unsupported formats/geometry return a counted Canvas fallback reason.
pub(super) async fn read(frame: &JsValue, cfg: &Config) -> Result<(VideoFrame, u32), &'static str> {
let format = get(frame, "format").as_string().ok_or("format")?;
let (chroma, alpha, interleaved) = match format.as_str() {
"NV12" => (Chroma::C420, false, true),
"I420" => (Chroma::C420, false, false),
"I420A" => (Chroma::C420, true, false),
"I422" => (Chroma::C422, false, false),
"I422A" => (Chroma::C422, true, false),
"I444" => (Chroma::C444, false, false),
"I444A" => (Chroma::C444, true, false),
_ => return Err("format"),
};
let rect = get(frame, "visibleRect");
let w = integer(get(&rect, "width")).filter(|n| *n > 0).ok_or("geometry")?;
let h = integer(get(&rect, "height")).filter(|n| *n > 0).ok_or("geometry")?;
if integer(get(frame, "displayWidth")) != Some(w)
|| integer(get(frame, "displayHeight")) != Some(h)
|| get(frame, "rotation").as_f64().is_some_and(|r| r != 0.0)
|| get(frame, "flip").as_bool() == Some(true)
{
return Err("geometry");
}
let color = color(frame, cfg.color).ok_or("color")?;
let options = obj(&[("rect", rect)]);
let bytes =
integer(call(frame, "allocationSize", &[&options]).map_err(|_| "allocation")?).filter(|n| *n > 0 && *n <= MAX_COPY_BYTES).ok_or("allocation")?;
let array: js_sys::Function = get(&js_sys::global(), "Uint8Array").dyn_into().map_err(|_| "allocation")?;
let raw: js_sys::Uint8Array =
js_sys::Reflect::construct(&array, &js_sys::Array::of1(&bytes.into())).and_then(|v| v.dyn_into()).map_err(|_| "allocation")?;
let promise = call(frame, "copyTo", &[raw.as_ref(), &options]).map_err(|_| "copy")?.dyn_into::<js_sys::Promise>().map_err(|_| "copy")?;
let layout = js_sys::Array::from(&JsFuture::from(promise).await.map_err(|_| "copy")?);
let count = if interleaved {
2
} else if alpha {
4
} else {
3
};
if layout.length() != count {
return Err("layout");
}
let (sx, sy) = chroma.shifts();
let (cw, ch) = (w.div_ceil(1 << sx), h.div_ceil(1 << sy));
let y = plane(&raw, &layout.get(0), w, h)?;
let (u, v) = if interleaved {
let uv = plane(&raw, &layout.get(1), cw.checked_mul(2).ok_or("allocation")?, ch)?;
let pairs = uv.as_chunks::<2>().0;
let (mut u, mut v) = (Vec::new(), Vec::new());
u.try_reserve_exact(pairs.len()).map_err(|_| "memory")?;
v.try_reserve_exact(pairs.len()).map_err(|_| "memory")?;
u.extend(pairs.iter().map(|p| p[0]));
v.extend(pairs.iter().map(|p| p[1]));
(u, v)
} else {
(plane(&raw, &layout.get(1), cw, ch)?, plane(&raw, &layout.get(2), cw, ch)?)
};
let alpha = if alpha { Some(Arc::new(plane(&raw, &layout.get(3), w, h)?)) } else { None };
Ok((
VideoFrame {
width: w,
height: h,
data: PixelData::Yuv8 { planes: [Arc::new(y), Arc::new(u), Arc::new(v)], chroma, alpha },
color,
par: cfg.par,
pts: Tick::ZERO,
},
bytes,
))
}
+67
View File
@@ -0,0 +1,67 @@
// Web import destinations: active bin tab, browser drop + navigation during pending reads.
// Synthetic browser drop event exercises the production handler; physical OS drag/picker UI is outside this test.
export async function checkImportBins({ evaluate, media, timeoutMs = 120000 }) {
const report = {gates: {}, cases: []};
await evaluate('filmcraft.execute("file.newProject", {name: "Import bins"})');
const parent = (await evaluate('filmcraft.execute("file.newBin", {name: "Footage"})')).bin;
const child = (await evaluate(`filmcraft.execute("file.newBin", {name: "Selects", parent: ${parent}})`)).bin;
await evaluate(`(async () => {window.__importBinFixture = new Uint8Array(await (await fetch(${JSON.stringify(media)})).arrayBuffer());})()`);
const setBin = bin => evaluate(`filmcraft.request("ui.set", {projectPanel: {bin: ${JSON.stringify(bin)}, activeTab: null}})`);
const importExpression = name => `filmcraft.importFiles([new File([window.__importBinFixture], ${JSON.stringify(name)}, {type: "video/mp4"})])`;
const locate = async name => {
const deadline = Date.now() + timeoutMs;
const pollMs = 100;
for (;;) {
const project = await evaluate('filmcraft.execute("project.inspect")');
const find = bin => {
for (const entry of bin.children) {
if (entry.item !== undefined && entry.name === name) return bin.id;
if (entry.children) {const found = find(entry); if (found !== undefined) return found;}
}
};
const found = find(project.root);
if (found !== undefined) return found;
if (Date.now() >= deadline) throw new Error(`Import never appeared: ${name}`);
await new Promise(resolve => setTimeout(resolve, pollMs));
}
};
const check = async (key, expected, name) => {
const actual = await locate(name);
report.cases.push({key, expected, actual, name}); report.gates[key] = actual === expected;
};
await evaluate(`filmcraft.request("ui.set", {projectPanel: {bin: ${child}, activeTab: 0, tabs: [{bin: ${parent}}]}})`);
await evaluate(importExpression("tab.mp4")); await check("activeTab", parent, "tab.mp4");
await setBin(child);
await evaluate(`(() => {
const transfer = new DataTransfer();
transfer.items.add(new File([window.__importBinFixture], "drop.mp4", {type: "video/mp4"}));
window.dispatchEvent(new DragEvent("drop", {dataTransfer: transfer, bubbles: true, cancelable: true}));
})()`);
await check("browserDrop", child, "drop.mp4");
await setBin(child);
await evaluate(`(() => {
const original = Blob.prototype.arrayBuffer;
let release;
const pending = new Promise(resolve => {release = resolve;});
const gate = window.__importBinGate = {hits: 0, release, original};
Blob.prototype.arrayBuffer = function () {gate.hits++; return pending.then(() => original.call(this));};
window.__pendingBinImport = ${importExpression("pending.mp4")};
})()`);
try {
const deadline = Date.now() + timeoutMs;
const pollMs = 100;
for (;;) {
if (await evaluate("window.__importBinGate.hits > 0")) break;
if (Date.now() >= deadline) throw new Error("Read-delay probe did not intercept any Blob reads");
await new Promise(resolve => setTimeout(resolve, pollMs));
}
report.delayedReadHits = await evaluate("window.__importBinGate.hits");
await setBin(parent);
await evaluate("window.__importBinGate.release(); window.__pendingBinImport");
await check("destinationCapturedBeforeRead", child, "pending.mp4");
} finally {
await evaluate("Blob.prototype.arrayBuffer = window.__importBinGate.original; window.__importBinGate.release(); delete window.__importBinGate; delete window.__pendingBinImport; delete window.__importBinFixture");
}
report.ok = Object.values(report.gates).every(Boolean);
return report;
}
@@ -0,0 +1,78 @@
// #194 regression: same-name/same-size imports must retain independent sources across OPFS recovery.
// evaluate(expression) awaits a browser expression; navigate(url) waits for FilmCraft readiness. Generated PCM16 mono WAVs only; no binary fixtures.
import { createHash } from "node:crypto";
function tone(frequency, seconds) {
// RIFF/WAVE PCM16 mono: 44-byte header + 2 bytes/sample; fixture matches #194 (48 kHz, amplitude 12000).
const rate = 48000, bytesPerSample = 2, samples = rate * seconds;
const b = Buffer.alloc(44 + samples * bytesPerSample);
b.write("RIFF", 0); b.writeUInt32LE(b.length - 8, 4); b.write("WAVEfmt ", 8);
b.writeUInt32LE(16, 16); b.writeUInt16LE(1, 20); b.writeUInt16LE(1, 22);
b.writeUInt32LE(rate, 24); b.writeUInt32LE(rate * bytesPerSample, 28);
b.writeUInt16LE(bytesPerSample, 32); b.writeUInt16LE(16, 34); b.write("data", 36); b.writeUInt32LE(samples * bytesPerSample, 40);
for (let i = 0; i < samples; i++) b.writeInt16LE(Math.round(12000 * Math.sin(2 * Math.PI * frequency * i / rate)), 44 + bytesPerSample * i);
return b;
}
export async function checkImportCollision({ evaluate, navigate, url, timeoutMs = 120000 }) {
const fixtures = [tone(440, 2), tone(880, 2)];
const hashes = fixtures.map(b => createHash("sha256").update(b).digest("hex"));
const report = { hashes, fixtureBytes: fixtures.map(b => b.length), gates: {} };
const hash = expression => evaluate(`(async () => Array.from(new Uint8Array(await crypto.subtle.digest("SHA-256", ${expression})), b => b.toString(16).padStart(2, "0")).join(""))()`);
const readHash = path => hash(`await filmcraft.readFile(${JSON.stringify(path)})`);
const importFile = index => evaluate(`filmcraft.importFiles([new File([Uint8Array.from(atob(${JSON.stringify(fixtures[index].toString("base64"))}), c => c.charCodeAt(0))], "collision.wav", {type: "audio/wav"})])`);
await evaluate('filmcraft.execute("file.newProject", {name: "Import collision"})');
const a = await importFile(0); report.a = a;
if (a.errors?.length || !a.items?.length) throw new Error("baseline import failed: " + JSON.stringify(a));
report.initialHash = await readHash(a.paths[0]);
report.gates.healthyInitial = report.initialHash === hashes[0];
await evaluate(`filmcraft.execute("file.newSequence", {name: "Original A", fromItem: ${a.items[0]}})`);
const b = await importFile(1); report.b = b;
if (b.errors?.length || !b.items?.length) throw new Error("second import failed: " + JSON.stringify(b));
report.afterBHash = await readHash(a.paths[0]);
report.gates.distinctPaths = a.paths[0] !== b.paths[0];
report.gates.originalPreserved = report.afterBHash === hashes[0];
report.beforeProject = await evaluate('filmcraft.execute("project.inspect")');
const revision = report.beforeProject.revision;
const deadline = Date.now() + timeoutMs;
const persistencePollMs = 100;
for (;;) {
const snapshot = await evaluate(`(async () => {
try {
const dir = await (await navigator.storage.getDirectory()).getDirectoryHandle("recovery");
const read = async name => JSON.parse(await (await (await dir.getFileHandle(name)).getFile()).text());
return {meta: await read("meta.json"), project: await read("snapshot.fcproj")};
} catch (e) {return {error: String(e)};}
})()`);
report.recoveryMeta = snapshot.meta;
if (snapshot.meta?.dirty && snapshot.meta.revision === revision && JSON.stringify(snapshot.project).includes("Original A")) break;
if (Date.now() >= deadline) throw new Error("Recovery persistence gate failed: " + JSON.stringify(snapshot));
await new Promise(resolve => setTimeout(resolve, persistencePollMs));
}
// Media writes run separately from the snapshot queue; gate their actual bytes before navigation.
for (;;) {
report.opfs = await evaluate(`(async () => {
const dir = await (await navigator.storage.getDirectory()).getDirectoryHandle("media");
const files = [];
for await (const [name, handle] of dir.entries()) {
const bytes = await (await handle.getFile()).arrayBuffer();
files.push({name, hash: Array.from(new Uint8Array(await crypto.subtle.digest("SHA-256", bytes)), b => b.toString(16).padStart(2, "0")).join("")});
}
return files;
})()`);
// Before the fix, A and B intentionally share the same path; qualify the last write there.
const expected = new Map([[a.paths[0], hashes[0]], [b.paths[0], hashes[1]]]);
if ([...expected].every(([path, digest]) => report.opfs.some(f => f.name === path.slice("/files/".length) && f.hash === digest))) break;
if (Date.now() >= deadline) throw new Error("Imported media did not reach OPFS: " + JSON.stringify(report.opfs));
await new Promise(resolve => setTimeout(resolve, persistencePollMs));
}
const recoveryUrl = new URL(url);
recoveryUrl.searchParams.delete("fresh"); recoveryUrl.searchParams.delete("norecover");
await navigate(recoveryUrl.href);
report.recovered = await evaluate('filmcraft.execute("project.inspect")');
report.afterReload = {a: await readHash(a.paths[0]), b: await readHash(b.paths[0])};
report.gates.recoveredProject = JSON.stringify(report.recovered).includes("Original A");
report.gates.recoveredSources = report.afterReload.a === hashes[0] && report.afterReload.b === hashes[1];
report.ok = Object.values(report.gates).every(Boolean);
return report;
}
@@ -0,0 +1,77 @@
//! The web recovery snapshot policy. It is pure, so it runs natively even though the rest of the
//! crate is wasm32-only.
#[path = "../src/recovery_policy.rs"]
mod recovery_policy;
use recovery_policy::{Action, INTERVAL_S, Policy};
/// #197: a snapshot write that fails is retried after the interval without another edit.
#[test]
fn failed_snapshot_write_is_retried_without_another_edit() {
let mut p = Policy::default();
assert_eq!(p.tick(0.0, 3, 0), Action::WriteSnapshot(3));
p.written(3, false, 0.1);
// not immediately (no spinning on a failing disk)…
assert!(matches!(p.tick(1.0, 3, 0), Action::WakeAfter(_)));
// …but once the interval has passed, with the revision unchanged
assert_eq!(p.tick(0.1 + INTERVAL_S, 3, 0), Action::WriteSnapshot(3));
p.written(3, true, 5.2);
assert_eq!(p.tick(20.0, 3, 0), Action::Nothing);
}
/// A failure reported for an older revision doesn't make the newer one be written again.
#[test]
fn stale_failure_is_ignored() {
let mut p = Policy::default();
assert_eq!(p.tick(0.0, 1, 0), Action::WriteSnapshot(1));
assert_eq!(p.tick(INTERVAL_S, 2, 0), Action::WriteSnapshot(2));
p.written(1, false, INTERVAL_S);
assert_eq!(p.tick(3.0 * INTERVAL_S, 2, 0), Action::Nothing);
}
/// #193: a clean session (started with `?norecover` / `?fresh`, so the snapshot left by an
/// earlier session was not reopened) never marks that snapshot clean.
#[test]
fn clean_start_leaves_an_unconsumed_snapshot_dirty() {
let mut p = Policy::default();
for t in 0..100 {
assert_eq!(p.tick(t as f64, 0, 0), Action::Nothing);
}
}
/// The snapshot this session wrote is retired once the project is saved.
#[test]
fn saving_marks_own_snapshot_clean() {
let mut p = Policy::default();
assert_eq!(p.tick(0.0, 1, 0), Action::WriteSnapshot(1));
p.written(1, true, 0.1);
assert_eq!(p.tick(1.0, 1, 1), Action::WriteClean);
assert_eq!(p.tick(1.1, 1, 1), Action::Nothing);
p.cleaned(true, 1.2);
assert_eq!(p.tick(30.0, 1, 1), Action::Nothing);
}
/// A failed clean write is retried after the interval.
#[test]
fn failed_clean_write_is_retried() {
let mut p = Policy::default();
assert_eq!(p.tick(0.0, 1, 0), Action::WriteSnapshot(1));
p.written(1, true, 0.1);
assert_eq!(p.tick(1.0, 1, 1), Action::WriteClean);
p.cleaned(false, 1.1);
assert!(matches!(p.tick(2.0, 1, 1), Action::WakeAfter(_)));
assert_eq!(p.tick(1.1 + INTERVAL_S, 1, 1), Action::WriteClean);
}
/// An edit made while the clean write is in flight keeps the meta dirty after it lands.
#[test]
fn edit_during_clean_write_keeps_meta_owned() {
let mut p = Policy::default();
assert_eq!(p.tick(0.0, 1, 0), Action::WriteSnapshot(1));
assert_eq!(p.tick(10.0, 1, 1), Action::WriteClean);
assert_eq!(p.tick(20.0, 2, 1), Action::WriteSnapshot(2));
p.cleaned(true, 20.1);
// saved again: the newer snapshot must be retired too
assert_eq!(p.tick(30.0, 2, 2), Action::WriteClean);
}
+15
View File
@@ -13,6 +13,8 @@ import { spawn } from "node:child_process";
import { mkdirSync, writeFileSync, readdirSync, statSync, mkdtempSync } from "node:fs";
import { join } from "node:path";
import { tmpdir } from "node:os";
import { checkImportBins } from "./import-bins.mjs";
import { checkImportCollision } from "./import-collision.mjs";
const arg = (k, d) => {
const i = process.argv.indexOf(`--${k}`);
@@ -162,6 +164,19 @@ try {
await sleep(500);
await send("Emulation.setDeviceMetricsOverride", { width: 1600, height: 1000, deviceScaleFactor: 1, mobile: false });
report.steps.modes = { screenshot: await shot("05-modes") };
report.steps.importBins = await checkImportBins({evaluate: js, media});
if (!report.steps.importBins.ok) throw new Error("import bin regression: " + JSON.stringify(report.steps.importBins.gates));
report.steps.importCollision = await checkImportCollision({
evaluate: js,
url,
navigate: async (nextUrl) => {
await send("Page.navigate", { url: nextUrl });
await until("!!(window.filmcraftLoad && (window.filmcraftLoad.readyMs || window.filmcraftLoad.error))", 120000);
const load = await js("window.filmcraftLoad");
if (load.error) throw new Error(load.error);
},
});
if (!report.steps.importCollision.ok) throw new Error("import collision regression: " + JSON.stringify(report.steps.importCollision.gates));
// still alive? (a panicked app never answers)
await Promise.race([js("filmcraft.inspect().then(() => true)"), sleep(5000).then(() => { throw new Error("app stopped answering"); })]);
const fatal = logs.filter((l) => /panicked at|RuntimeError|EXCEPTION/.test(l));
+1
View File
@@ -26,6 +26,7 @@ log = { workspace = true }
rfd = { workspace = true }
cpal = "0.16"
chrono = { version = "0.4", default-features = false, features = ["clock"] }
sys-locale = "0.3"
[target.'cfg(target_os = "macos")'.dependencies]
muda = "0.21"
+171
View File
@@ -0,0 +1,171 @@
//! The desktop app's command line (`filmcraft --help`).
/// `filmcraft --help`.
pub const USAGE: &str = "\
Usage: filmcraft [options] [project.fcproj | media files...]
Options:
--control <port> start the localhost JSON-lines control server
(also: FILMCRAFT_CONTROL_PORT)
--demo open the demo project
--empty start with an empty project
--recover recover the newest unsaved changes without asking
--no-recover start without asking about unsaved changes
--data-dir <dir> folder for auto-save, crash recovery and logs
(also: FILMCRAFT_DATA_DIR)
--version print the version and exit
-h, --help print this help and exit
";
/// What the command line asks for.
#[derive(Debug, PartialEq)]
pub enum Cli {
Run(Launch),
Help,
Version,
}
/// Options of a normal start.
#[derive(Debug, PartialEq)]
pub struct Launch {
pub control_port: Option<u16>,
/// A project and / or media files to open.
pub files: Vec<String>,
pub demo: bool,
/// `--demo` / `--empty` given: skip Settings ▸ General ▸ At Startup.
pub startup_flag: bool,
pub recover: Option<bool>,
pub data_dir: Option<std::path::PathBuf>,
}
/// Parse the arguments after the program name. `env_control_port` is the value of
/// `FILMCRAFT_CONTROL_PORT`, used when `--control` is not given (an empty value counts as unset).
///
/// An option we don't know, or one without a usable value, is an error: opening a window for a
/// mistyped flag helps nobody, and starting without the control server that was asked for leaves
/// a script waiting for it. That holds for the whole line, so `--version --bogus` is an error
/// too, and for a port from the environment exactly as for one from `--control`.
pub fn parse(env_control_port: Option<String>, args: impl IntoIterator<Item = String>) -> Result<Cli, String> {
let mut l = Launch { control_port: None, files: Vec::new(), demo: true, startup_flag: false, recover: None, data_dir: None };
let (mut help, mut version) = (false, false);
let mut args = args.into_iter();
while let Some(a) = args.next() {
match a.as_str() {
"-h" | "--help" => help = true,
"--version" => version = true,
"--control" => {
let port = args.next().ok_or("--control needs a port number")?;
l.control_port = Some(port.parse().map_err(|_| format!("--control needs a port number, got `{port}`"))?);
}
"--demo" => (l.demo, l.startup_flag) = (true, true),
"--empty" => (l.demo, l.startup_flag) = (false, true),
"--recover" => l.recover = Some(true),
"--no-recover" => l.recover = Some(false),
"--data-dir" => l.data_dir = Some(args.next().map(std::path::PathBuf::from).ok_or("--data-dir needs a folder")?),
// everything after `--` is a file, whatever it looks like
"--" => l.files.extend(args.by_ref()),
// macOS adds a process serial number when it launches an app bundle
_ if a.starts_with("-psn_") => {}
_ if a.len() > 1 && a.starts_with('-') => return Err(format!("unknown option `{a}`")),
_ => l.files.push(a),
}
}
if help {
return Ok(Cli::Help);
}
if version {
return Ok(Cli::Version);
}
if l.control_port.is_none()
&& let Some(port) = env_control_port.as_deref().map(str::trim).filter(|p| !p.is_empty())
{
l.control_port = Some(port.parse().map_err(|_| format!("FILMCRAFT_CONTROL_PORT needs a port number, got `{port}`"))?);
}
Ok(Cli::Run(l))
}
#[cfg(test)]
mod tests {
use super::*;
fn parse_env(env: Option<&str>, args: &[&str]) -> Result<Cli, String> {
parse(env.map(str::to_string), args.iter().map(|a| a.to_string()))
}
fn parse_args(args: &[&str]) -> Result<Cli, String> {
parse_env(None, args)
}
fn launch(args: &[&str]) -> Launch {
match parse_args(args) {
Ok(Cli::Run(l)) => l,
other => panic!("{args:?} should start the app, got {other:?}"),
}
}
/// `filmcraft --help` used to open the app with "--help" as a media file to import.
#[test]
fn help_and_version_do_not_start_the_app() {
assert_eq!(parse_args(&["--help"]), Ok(Cli::Help));
assert_eq!(parse_args(&["-h"]), Ok(Cli::Help));
assert_eq!(parse_args(&["--empty", "--help"]), Ok(Cli::Help));
assert_eq!(parse_args(&["--version"]), Ok(Cli::Version));
assert_eq!(parse_args(&["--version", "--help"]), Ok(Cli::Help));
assert_eq!(parse_args(&["a.mp4", "--version"]), Ok(Cli::Version));
for flag in ["--control", "--demo", "--empty", "--recover", "--no-recover", "--data-dir", "--version", "--help"] {
assert!(USAGE.contains(flag), "usage lists {flag}");
}
}
#[test]
fn unknown_options_are_errors() {
assert_eq!(parse_args(&["--frobnicate"]), Err("unknown option `--frobnicate`".to_string()));
assert!(parse_args(&["a.mp4", "-x"]).is_err());
assert!(parse_args(&["--Help"]).is_err());
assert!(parse_args(&["--control"]).is_err(), "missing port");
assert!(parse_args(&["--control", "abc"]).unwrap_err().contains("abc"));
assert!(parse_args(&["--control", "70000"]).is_err(), "not a port");
assert!(parse_args(&["--data-dir"]).is_err(), "missing folder");
// the whole line is checked, whatever comes first
assert_eq!(parse_args(&["--version", "--bogus"]), Err("unknown option `--bogus`".to_string()));
assert_eq!(parse_args(&["--bogus", "--version"]), Err("unknown option `--bogus`".to_string()));
assert_eq!(parse_args(&["--help", "--bogus"]), Err("unknown option `--bogus`".to_string()));
assert!(parse_args(&["--help", "--control"]).is_err());
}
/// A bad `FILMCRAFT_CONTROL_PORT` used to start the app without the control server, silently.
#[test]
fn a_bad_port_in_the_environment_is_an_error_like_a_bad_control_flag() {
for bad in ["abc", "70000", "-1", "98 76", "9876x", "\u{fffd}"] {
let e = parse_env(Some(bad), &[]).unwrap_err();
assert!(e.contains("FILMCRAFT_CONTROL_PORT") && e.contains(bad), "{bad}: {e}");
assert!(parse_env(Some(bad), &["--empty", "a.mp4"]).is_err(), "{bad}");
// the flag wins, so the environment's value is not looked at; nor for --help / --version
assert_eq!(parse_env(Some(bad), &["--control", "9"]).map(|c| matches!(c, Cli::Run(l) if l.control_port == Some(9))), Ok(true), "{bad}");
assert_eq!(parse_env(Some(bad), &["--help"]), Ok(Cli::Help));
assert_eq!(parse_env(Some(bad), &["--version"]), Ok(Cli::Version));
}
// unset or empty: no control server
assert_eq!(launch(&[]).control_port, None);
assert_eq!(parse_env(Some(""), &[]), Ok(Cli::Run(launch(&[]))));
assert_eq!(parse_env(Some(" "), &[]), Ok(Cli::Run(launch(&[]))));
// a good one is the default
assert_eq!(parse_env(Some("1234"), &[]), Ok(Cli::Run(Launch { control_port: Some(1234), ..launch(&[]) })));
assert_eq!(parse_env(Some("1234"), &["--control", "9"]).map(|c| matches!(c, Cli::Run(l) if l.control_port == Some(9))), Ok(true));
}
#[test]
fn known_options_and_files_parse_as_before() {
let none = launch(&[]);
assert_eq!(none, Launch { control_port: None, files: vec![], demo: true, startup_flag: false, recover: None, data_dir: None });
let l = launch(&["--control", "9876", "--empty", "--no-recover", "--data-dir", "/tmp/fc", "cut.fcproj", "a.mp4"]);
assert_eq!(l.control_port, Some(9876));
assert_eq!((l.demo, l.startup_flag, l.recover), (false, true, Some(false)));
assert_eq!(l.data_dir, Some(std::path::PathBuf::from("/tmp/fc")));
assert_eq!(l.files, ["cut.fcproj", "a.mp4"]);
assert_eq!((launch(&["--demo"]).demo, launch(&["--demo"]).startup_flag, launch(&["--recover"]).recover), (true, true, Some(true)));
// `--` ends the options; a lone `-` and macOS' process serial number are not options
assert_eq!(launch(&["--", "--odd name.mov", "-h"]).files, ["--odd name.mov", "-h"]);
assert_eq!(launch(&["-", "-psn_0_12345"]).files, ["-"]);
}
}
+123 -15
View File
@@ -2,7 +2,7 @@
//! ("Device Class"), the output device, the I/O buffer size and the sample rate.
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use cpal::traits::{DeviceTrait, HostTrait, StreamTrait};
use filmcraft_engine::settings::AudioHardwarePrefs;
@@ -11,7 +11,9 @@ use filmcraft_ui_egui::{AudioDevices, AudioOut};
pub struct CpalOut {
stream: Option<cpal::Stream>,
played: Arc<AtomicU64>,
failed: Arc<AtomicBool>,
rate: u32,
channels: u16,
hw: AudioHardwarePrefs,
/// Sequence sample rate for "Attempt to force hardware to document sample rate".
document_rate: Option<u32>,
@@ -40,27 +42,82 @@ fn output_device(h: &cpal::Host, name: &str) -> Option<cpal::Device> {
}
impl CpalOut {
pub fn new() -> Option<Self> {
pub fn new() -> Self {
let host = cpal::default_host();
let dev = host.default_output_device()?;
let cfg = dev.default_output_config().ok()?;
Some(Self { stream: None, played: Arc::new(AtomicU64::new(0)), rate: cfg.sample_rate().0, hw: AudioHardwarePrefs::default(), document_rate: None })
let cfg = host.default_output_device().and_then(|dev| dev.default_output_config().ok());
Self {
stream: None,
played: Arc::new(AtomicU64::new(0)),
failed: Arc::new(AtomicBool::new(false)),
rate: cfg.as_ref().map_or(48_000, |c| c.sample_rate().0),
channels: cfg.as_ref().map_or(2, |c| c.channels()),
hw: AudioHardwarePrefs::default(),
document_rate: None,
}
}
/// The stream configuration the settings ask for, falling back to the device default.
fn config(&self, dev: &cpal::Device) -> Result<cpal::SupportedStreamConfig, String> {
let default = dev.default_output_config().map_err(|e| e.to_string())?;
let want = if self.hw.force_document_rate { self.document_rate.unwrap_or(self.hw.sample_rate) } else { self.hw.sample_rate };
let exact = dev.supported_output_configs().ok().and_then(|mut it| {
it.find(|c| {
c.sample_format() == cpal::SampleFormat::F32
&& c.channels() == default.channels()
&& (c.min_sample_rate().0..=c.max_sample_rate().0).contains(&want)
})
.and_then(|c| c.try_with_sample_rate(cpal::SampleRate(want)))
let exact = dev.supported_output_configs().ok().and_then(|it| {
it.filter(|c| c.channels() > 0 && (c.min_sample_rate().0..=c.max_sample_rate().0).contains(&want))
.max_by_key(|c| {
(c.channels() == default.channels(), c.sample_format() == cpal::SampleFormat::F32, c.sample_format() == default.sample_format())
})
.and_then(|c| c.try_with_sample_rate(cpal::SampleRate(want)))
});
Ok(exact.unwrap_or(default))
}
fn converted_stream<T>(
&self,
dev: &cpal::Device,
config: &cpal::StreamConfig,
mut fill: Box<dyn FnMut(&mut [f32], usize) + Send>,
) -> Result<cpal::Stream, cpal::BuildStreamError>
where
T: cpal::SizedSample + cpal::FromSample<f32>,
{
let channels = usize::from(config.channels);
let played = self.played.clone();
let failed = self.failed.clone();
let error_flag = failed.clone();
let mut scratch = Vec::new();
dev.build_output_stream(
config,
move |buf: &mut [T], _| {
// Reuse the conversion buffer. A pathological driver buffer must not panic the audio thread.
if buf.len() > 1_048_576 || scratch.try_reserve(buf.len().saturating_sub(scratch.len())).is_err() {
buf.fill(T::EQUILIBRIUM);
if !failed.swap(true, Ordering::SeqCst) {
eprintln!("filmcraft: unable to allocate the audio conversion buffer");
}
return;
}
scratch.resize(buf.len(), 0.0);
fill(&mut scratch, channels);
convert_samples(&scratch, buf);
played.fetch_add((buf.len() / channels) as u64, Ordering::SeqCst);
},
move |e| stream_error(&error_flag, e),
None,
)
}
}
fn stream_error(failed: &AtomicBool, e: cpal::StreamError) {
if !failed.swap(true, Ordering::SeqCst) {
eprintln!("filmcraft: audio stream error: {e}");
}
}
fn convert_samples<T: cpal::Sample + cpal::FromSample<f32>>(input: &[f32], output: &mut [T]) {
// Integer conversion represents [-1, 1). In particular I24 cannot represent +1 exactly.
let upper = f32::from_bits(1.0_f32.to_bits() - 1);
for (sample, converted) in input.iter().zip(output) {
*converted = T::from_sample(if sample.is_finite() { sample.clamp(-1.0, upper) } else { 0.0 });
}
}
impl AudioOut for CpalOut {
@@ -70,6 +127,9 @@ impl AudioOut for CpalOut {
let dev = output_device(&host, &self.hw.default_output).ok_or("no output device")?;
let cfg = self.config(&dev)?;
let channels = cfg.channels() as usize;
if channels == 0 || cfg.sample_rate().0 == 0 {
return Err("the output device returned an invalid channel count or sample rate".into());
}
let mut config: cpal::StreamConfig = cfg.clone().into();
if let cpal::SupportedBufferSize::Range { min, max } = cfg.buffer_size()
&& (*min..=*max).contains(&self.hw.buffer_size)
@@ -77,9 +137,11 @@ impl AudioOut for CpalOut {
config.buffer_size = cpal::BufferSize::Fixed(self.hw.buffer_size);
}
self.rate = cfg.sample_rate().0;
self.channels = cfg.channels();
self.played.store(0, Ordering::SeqCst);
self.failed.store(false, Ordering::SeqCst);
let played = self.played.clone();
let err = |e| eprintln!("filmcraft: audio stream error: {e}");
let failed = self.failed.clone();
let stream = match cfg.sample_format() {
cpal::SampleFormat::F32 => dev.build_output_stream(
&config,
@@ -87,9 +149,19 @@ impl AudioOut for CpalOut {
fill(buf, channels);
played.fetch_add((buf.len() / channels) as u64, Ordering::SeqCst);
},
err,
move |e| stream_error(&failed, e),
None,
),
cpal::SampleFormat::I8 => self.converted_stream::<i8>(&dev, &config, fill),
cpal::SampleFormat::I16 => self.converted_stream::<i16>(&dev, &config, fill),
cpal::SampleFormat::I24 => self.converted_stream::<cpal::I24>(&dev, &config, fill),
cpal::SampleFormat::I32 => self.converted_stream::<i32>(&dev, &config, fill),
cpal::SampleFormat::I64 => self.converted_stream::<i64>(&dev, &config, fill),
cpal::SampleFormat::U8 => self.converted_stream::<u8>(&dev, &config, fill),
cpal::SampleFormat::U16 => self.converted_stream::<u16>(&dev, &config, fill),
cpal::SampleFormat::U32 => self.converted_stream::<u32>(&dev, &config, fill),
cpal::SampleFormat::U64 => self.converted_stream::<u64>(&dev, &config, fill),
cpal::SampleFormat::F64 => self.converted_stream::<f64>(&dev, &config, fill),
other => return Err(format!("unsupported sample format {other:?}")),
}
.map_err(|e| e.to_string())?;
@@ -103,8 +175,11 @@ impl AudioOut for CpalOut {
fn sample_rate(&self) -> u32 {
self.rate
}
fn channels(&self) -> usize {
self.channels as usize
}
fn played_frames(&self) -> Option<u64> {
self.stream.as_ref().map(|_| self.played.load(Ordering::SeqCst))
self.stream.as_ref().filter(|_| !self.failed.load(Ordering::SeqCst)).map(|_| self.played.load(Ordering::SeqCst))
}
fn devices(&self) -> AudioDevices {
let h = host(&self.hw.device_class);
@@ -122,6 +197,39 @@ impl AudioOut for CpalOut {
&& let Ok(cfg) = self.config(&dev)
{
self.rate = cfg.sample_rate().0;
self.channels = cfg.channels();
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use cpal::Sample;
#[test]
fn integer_conversion_clips_and_preserves_interleaved_channels() {
let mut signed = [0_i16; 8];
convert_samples(&[-2.0, 2.0, -0.5, 0.5, 0.0, f32::NAN, f32::INFINITY, f32::NEG_INFINITY], &mut signed);
assert_eq!(signed, [i16::MIN, i16::MAX, -16_384, 16_384, 0, 0, 0, 0]);
let mut unsigned = [0_u16; 3];
convert_samples(&[-1.0, 0.0, 1.0], &mut unsigned);
assert_eq!(unsigned, [0, 32_768, u16::MAX]);
}
#[test]
fn full_scale_24_bit_samples_remain_representable() {
let mut samples = [cpal::I24::EQUILIBRIUM; 3];
convert_samples(&[-1.0, 0.0, 1.0], &mut samples);
assert_eq!(samples.map(|s| s.inner()), [-8_388_608, 0, 8_388_607]);
}
#[test]
fn failed_stream_relinquishes_the_playback_clock() {
let failed = AtomicBool::new(false);
stream_error(&failed, cpal::StreamError::DeviceNotAvailable);
assert!(failed.load(Ordering::SeqCst));
let output = CpalOut::new();
assert_eq!(output.played_frames(), None);
}
}
+170 -56
View File
@@ -14,11 +14,13 @@ pub struct CpalIn {
host: String,
buf: Arc<Mutex<Vec<Vec<f32>>>>,
stop: Option<mpsc::Sender<()>>,
worker: Option<std::thread::JoinHandle<()>>,
error: Arc<Mutex<Option<String>>>,
}
impl CpalIn {
pub fn new(host: &str) -> Self {
Self { host: host.to_string(), buf: Arc::default(), stop: None }
Self { host: host.to_string(), buf: Arc::default(), stop: None, worker: None, error: Arc::default() }
}
}
@@ -46,7 +48,63 @@ fn lock<T>(m: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
m.lock().unwrap_or_else(|e| e.into_inner())
}
fn report_error(error: &Mutex<Option<String>>, message: String) {
let mut error = lock(error);
if error.is_none() {
eprintln!("filmcraft: audio input error: {message}");
*error = Some(message);
}
}
fn append<T: cpal::Sample>(data: &[T], channels: usize, buf: &Mutex<Vec<Vec<f32>>>, error: &Mutex<Option<String>>)
where
f32: cpal::FromSample<T>,
{
if lock(error).is_some() {
return;
}
let mut b = lock(buf);
if channels == 0 || channels > 256 || data.len() > 1_048_576 {
report_error(error, "invalid input device buffer".into());
return;
}
if b.len() != channels {
*b = vec![Vec::new(); channels];
}
// The recorder drains at stop. Bound accumulated samples and report exhaustion instead of
// allowing a long take or a broken/unpaced driver to exhaust the application heap.
let samples = b.first().map_or(0, Vec::len).saturating_mul(channels).saturating_add(data.len());
let frames = data.len() / channels;
if samples > 268_435_456 || b.iter_mut().any(|c| c.try_reserve(frames).is_err()) {
report_error(error, "voice-over input buffer is full; recording failed".into());
return;
}
for frame in data.chunks_exact(channels) {
for (channel, sample) in b.iter_mut().zip(frame) {
let sample = sample.to_sample::<f32>();
channel.push(if sample.is_finite() { sample.clamp(-1.0, 1.0) } else { 0.0 });
}
}
}
fn input_stream<T: cpal::SizedSample>(
dev: &cpal::Device,
config: &cpal::StreamConfig,
buf: Arc<Mutex<Vec<Vec<f32>>>>,
error: Arc<Mutex<Option<String>>>,
) -> Result<cpal::Stream, cpal::BuildStreamError>
where
f32: cpal::FromSample<T>,
{
let channels = usize::from(config.channels);
let stream_error = error.clone();
dev.build_input_stream(config, move |data: &[T], _| append(data, channels, &buf, &error), move |e| report_error(&stream_error, e.to_string()), None)
}
impl AudioInput for CpalIn {
fn configure_host(&mut self, name: &str) {
self.host = name.to_string();
}
fn devices(&self) -> Vec<String> {
host(&self.host).input_devices().map(|d| d.filter_map(|x| x.name().ok()).collect()).unwrap_or_default()
}
@@ -56,68 +114,77 @@ impl AudioInput for CpalIn {
fn start(&mut self, name: &str, sample_rate: u32) -> Result<InputFormat, String> {
self.stop();
lock(&self.buf).clear();
*lock(&self.error) = None;
let (host_name, name) = (self.host.clone(), name.to_string());
let buf = self.buf.clone();
let error = self.error.clone();
let (fmt_tx, fmt_rx) = mpsc::channel::<Result<InputFormat, String>>();
let (stop_tx, stop_rx) = mpsc::channel::<()>();
std::thread::Builder::new()
.name("voice-over input".into())
.spawn(move || {
let open = || -> Result<(cpal::Stream, InputFormat), String> {
let h = host(&host_name);
let dev = device(&h, &name).ok_or("no input device")?;
let default = dev.default_input_config().map_err(|e| e.to_string())?;
// the sequence rate when the device offers it in f32, else the device default
let cfg = dev
.supported_input_configs()
.ok()
.and_then(|mut it| {
it.find(|c| c.sample_format() == cpal::SampleFormat::F32 && (c.min_sample_rate().0..=c.max_sample_rate().0).contains(&sample_rate))
.and_then(|c| c.try_with_sample_rate(cpal::SampleRate(sample_rate)))
})
.unwrap_or(default);
if cfg.sample_format() != cpal::SampleFormat::F32 {
return Err(format!("unsupported input sample format {:?}", cfg.sample_format()));
}
let ch = cfg.channels() as usize;
let fmt = InputFormat { sample_rate: cfg.sample_rate().0, channels: cfg.channels() };
let stream = dev
.build_input_stream(
&cfg.into(),
move |data: &[f32], _| {
let mut b = lock(&buf);
if b.len() != ch {
*b = vec![Vec::new(); ch];
}
for frame in data.chunks_exact(ch.max(1)) {
for (c, x) in frame.iter().enumerate() {
b[c].push(*x);
}
}
},
|e| eprintln!("filmcraft: audio input error: {e}"),
None,
)
self.worker = Some(
std::thread::Builder::new()
.name("voice-over input".into())
.spawn(move || {
let open = || -> Result<(cpal::Stream, InputFormat), String> {
let h = host(&host_name);
let dev = device(&h, &name).ok_or("no input device")?;
let default = dev.default_input_config().map_err(|e| e.to_string())?;
// Prefer f32 at the sequence rate, retaining integer-only input devices.
let cfg = dev
.supported_input_configs()
.ok()
.and_then(|it| {
it.filter(|c| c.channels() > 0 && (c.min_sample_rate().0..=c.max_sample_rate().0).contains(&sample_rate))
.max_by_key(|c| (c.channels() == default.channels(), c.sample_format() == cpal::SampleFormat::F32))
.and_then(|c| c.try_with_sample_rate(cpal::SampleRate(sample_rate)))
})
.unwrap_or(default);
if cfg.channels() == 0 || cfg.channels() > 256 || cfg.sample_rate().0 == 0 {
return Err("the input device returned an invalid format".into());
}
let fmt = InputFormat { sample_rate: cfg.sample_rate().0, channels: cfg.channels() };
let format = cfg.sample_format();
let config = cfg.into();
let stream = match format {
cpal::SampleFormat::I8 => input_stream::<i8>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::I16 => input_stream::<i16>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::I24 => input_stream::<cpal::I24>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::I32 => input_stream::<i32>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::I64 => input_stream::<i64>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::U8 => input_stream::<u8>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::U16 => input_stream::<u16>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::U32 => input_stream::<u32>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::U64 => input_stream::<u64>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::F32 => input_stream::<f32>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::F64 => input_stream::<f64>(&dev, &config, buf.clone(), error.clone()),
other => return Err(format!("unsupported input sample format {other:?}")),
}
.map_err(|e| e.to_string())?;
stream.play().map_err(|e| e.to_string())?;
Ok((stream, fmt))
};
match open() {
Ok((stream, fmt)) => {
let _ = fmt_tx.send(Ok(fmt));
// keep the stream alive until stop (or the sender is dropped)
let _ = stop_rx.recv();
drop(stream);
stream.play().map_err(|e| e.to_string())?;
Ok((stream, fmt))
};
let opened = std::panic::catch_unwind(std::panic::AssertUnwindSafe(open)).unwrap_or_else(|_| Err("audio input worker panicked".into()));
match opened {
Ok((stream, fmt)) => {
let _ = fmt_tx.send(Ok(fmt));
// keep the stream alive until stop (or the sender is dropped)
let _ = stop_rx.recv();
drop(stream);
}
Err(e) => {
let _ = fmt_tx.send(Err(e));
}
}
Err(e) => {
let _ = fmt_tx.send(Err(e));
}
}
})
.map_err(|e| e.to_string())?;
let fmt = fmt_rx.recv().map_err(|e| e.to_string())??;
})
.map_err(|e| e.to_string())?,
);
self.stop = Some(stop_tx);
Ok(fmt)
match fmt_rx.recv().map_err(|e| e.to_string()).and_then(|f| f) {
Ok(fmt) => Ok(fmt),
Err(e) => {
self.stop();
Err(e)
}
}
}
fn read(&mut self, frames: usize) -> Vec<Vec<f32>> {
let mut b = lock(&self.buf);
@@ -131,9 +198,56 @@ impl AudioInput for CpalIn {
fn discard(&mut self) {
lock(&self.buf).iter_mut().for_each(Vec::clear);
}
fn error(&self) -> Option<String> {
lock(&self.error).clone()
}
fn stop(&mut self) {
if let Some(tx) = self.stop.take() {
let _ = tx.send(());
}
if let Some(worker) = self.worker.take()
&& worker.join().is_err()
{
report_error(&self.error, "audio input worker panicked".into());
}
}
}
impl Drop for CpalIn {
fn drop(&mut self) {
self.stop();
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn signed_and_unsigned_input_is_deinterleaved_to_float() {
let buf = Mutex::new(Vec::new());
let error = Mutex::new(None);
append(&[i16::MIN, i16::MAX, 0, 16384], 2, &buf, &error);
let data = lock(&buf);
assert_eq!(data[0], [-1.0, 0.0]);
assert!((data[1][0] - 1.0).abs() < 0.0001);
assert_eq!(data[1][1], 0.5);
drop(data);
lock(&buf).clear();
append(&[0_u16, 32768, 65535, 32768], 2, &buf, &error);
let data = lock(&buf);
assert_eq!(data[1], [0.0, 0.0]);
assert_eq!(data[0][0], -1.0);
assert!(lock(&error).is_none());
}
#[test]
fn invalid_input_is_reported_without_panicking_or_appending() {
let buf = Mutex::new(Vec::new());
let error = Mutex::new(None);
append(&[0.0_f32], 0, &buf, &error);
assert!(lock(&error).is_some());
append(&[1.0_f32], 1, &buf, &error);
assert!(lock(&buf).is_empty());
}
}
+261 -34
View File
@@ -1,13 +1,27 @@
//! Localhost JSON-lines control server (one request per line, one reply per line). Loopback only.
//!
//! The server only ever reads requests: a line that is not a JSON object with a string `method`
//! (an HTTP request line from a browser's cross-origin `fetch`, a stray `nc`, binary junk) gets
//! one error reply and the connection is closed, so nothing after it is executed. Lines are
//! bounded ([`MAX_LINE`]) and so is the number of open connections ([`MAX_CONNECTIONS`]); junk
//! never reaches the UI thread. See `docs/control-protocol.md`.
use std::io::{BufRead, BufReader, Write};
use std::io::{BufRead, BufReader, Read, Write};
use std::net::{TcpListener, TcpStream};
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::mpsc::{Receiver, Sender, channel};
use std::time::Duration;
use filmcraft_ui_egui::ControlRequest;
use serde_json::{Value, json};
/// Longest accepted request line (bytes, without the newline). Requests are small JSON objects;
/// anything longer is refused and the connection closed.
pub const MAX_LINE: usize = 4 * 1024 * 1024;
/// Connections served at once; further ones get an error line and are closed.
pub const MAX_CONNECTIONS: usize = 16;
pub fn start(port: u16, ctx: egui::Context) -> Receiver<ControlRequest> {
let (tx, rx) = channel::<ControlRequest>();
let listener = match TcpListener::bind(("127.0.0.1", port)) {
@@ -18,44 +32,257 @@ pub fn start(port: u16, ctx: egui::Context) -> Receiver<ControlRequest> {
}
};
eprintln!("filmcraft: control server listening on 127.0.0.1:{port}");
std::thread::spawn(move || {
for stream in listener.incoming().flatten() {
let tx = tx.clone();
let ctx = ctx.clone();
std::thread::spawn(move || serve(stream, tx, ctx));
}
});
std::thread::spawn(move || accept_loop(listener, tx, ctx));
rx
}
fn serve(stream: TcpStream, tx: Sender<ControlRequest>, ctx: egui::Context) {
let Ok(read) = stream.try_clone() else { return };
let mut out = stream;
for line in BufReader::new(read).lines() {
let Ok(line) = line else { break };
if line.trim().is_empty() {
fn accept_loop(listener: TcpListener, tx: Sender<ControlRequest>, ctx: egui::Context) {
let open = Arc::new(AtomicUsize::new(0));
for mut stream in listener.incoming().flatten() {
if open.fetch_add(1, Ordering::SeqCst) >= MAX_CONNECTIONS {
open.fetch_sub(1, Ordering::SeqCst);
let _ = writeln!(stream, "{}", json!({"ok": false, "error": "too many control connections"}));
continue;
}
let reply = match serde_json::from_str::<Value>(&line) {
Ok(msg) => {
let id = msg.get("id").cloned().unwrap_or(Value::Null);
let method = msg.get("method").and_then(Value::as_str).unwrap_or("").to_string();
let params = msg.get("params").cloned().unwrap_or(json!({}));
let (req, rrx) = ControlRequest::new(method, params);
if tx.send(req).is_err() {
break;
}
ctx.request_repaint();
let mut r = rrx.recv_timeout(Duration::from_secs(60)).unwrap_or_else(|_| json!({"ok": false, "error": "timeout"}));
if let Some(o) = r.as_object_mut() {
o.insert("id".into(), id);
}
r
}
Err(e) => json!({"ok": false, "error": format!("bad JSON: {e}")}),
};
if writeln!(out, "{reply}").is_err() {
break;
let tx = tx.clone();
let ctx = ctx.clone();
// The slot is given back when the guard drops: after `serve` returns, if it panics, or if
// the thread can't be started (the closure, and with it the guard, is dropped).
let slot = Slot(Arc::clone(&open));
let spawned = std::thread::Builder::new().name("control-conn".into()).spawn(move || {
let _slot = slot;
serve(stream, &tx, &ctx);
});
if let Err(e) = spawned {
eprintln!("filmcraft: control server could not start a connection thread: {e}");
}
}
}
/// One counted connection; decrements the open-connection count when dropped.
struct Slot(Arc<AtomicUsize>);
impl Drop for Slot {
fn drop(&mut self) {
self.0.fetch_sub(1, Ordering::SeqCst);
}
}
fn serve(stream: TcpStream, tx: &Sender<ControlRequest>, ctx: &egui::Context) {
let Ok(read) = stream.try_clone() else { return };
serve_lines(read, stream, |method, params| {
let (req, rrx) = ControlRequest::new(method, params);
tx.send(req).ok()?;
ctx.request_repaint();
Some(rrx.recv_timeout(Duration::from_secs(60)).unwrap_or_else(|_| json!({"ok": false, "error": "timeout"})))
});
}
/// One parsed line.
enum Line {
Request {
id: Value,
method: String,
params: Value,
},
Blank,
/// Not a request: reply with this error and close the connection.
Reject(String),
}
fn parse_line(bytes: &[u8]) -> Line {
let Ok(text) = std::str::from_utf8(bytes) else {
return Line::Reject("not a control request (invalid UTF-8); closing the connection".into());
};
let text = text.trim();
if text.is_empty() {
return Line::Blank;
}
let msg = match serde_json::from_str::<Value>(text) {
Ok(v) => v,
Err(e) => return Line::Reject(format!("bad JSON ({e}); expected one JSON request object per line; closing the connection")),
};
let Some(obj) = msg.as_object() else {
return Line::Reject("not a control request (expected a JSON object); closing the connection".into());
};
let Some(method) = obj.get("method").and_then(Value::as_str) else {
return Line::Reject("not a control request (missing string `method`); closing the connection".into());
};
Line::Request { id: obj.get("id").cloned().unwrap_or(Value::Null), method: method.to_string(), params: obj.get("params").cloned().unwrap_or(json!({})) }
}
/// Read one `\n`-terminated line of at most `max` bytes into `buf` (newline stripped).
/// `Ok(false)` at end of stream; `Err` when the line is too long or the read fails.
fn read_bounded_line(reader: &mut impl BufRead, buf: &mut Vec<u8>, max: usize) -> Result<bool, String> {
buf.clear();
// `take` bounds the bytes buffered for one line: a peer can't make us allocate more than `max`.
let limit = u64::try_from(max).unwrap_or(u64::MAX).saturating_add(1);
let n = reader.take(limit).read_until(b'\n', buf).map_err(|e| e.to_string())?;
if n == 0 {
return Ok(false);
}
if buf.last() == Some(&b'\n') {
buf.pop();
if buf.last() == Some(&b'\r') {
buf.pop();
}
return Ok(true);
}
if buf.len() > max {
return Err(format!("request line longer than {max} bytes; closing the connection"));
}
// Last line without a newline, then end of stream.
Ok(true)
}
/// Serve requests from `read`, answering on `out`, until the peer closes, sends something that is
/// not a request, or `handle` returns `None` (app gone). `handle` gets `(method, params)` and
/// returns the reply object (the request's `id` is added here).
fn serve_lines<R: Read, W: Write>(read: R, mut out: W, mut handle: impl FnMut(String, Value) -> Option<Value>) {
let mut reader = BufReader::new(read);
let mut buf = Vec::new();
loop {
let reject = match read_bounded_line(&mut reader, &mut buf, MAX_LINE) {
Ok(false) => return,
Err(e) => e,
Ok(true) => match parse_line(&buf) {
Line::Blank => continue,
Line::Reject(e) => e,
Line::Request { id, method, params } => {
let Some(mut r) = handle(method, params) else { return };
if let Some(o) = r.as_object_mut() {
o.insert("id".into(), id);
}
if writeln!(out, "{r}").and_then(|()| out.flush()).is_err() {
return;
}
continue;
}
},
};
let _ = writeln!(out, "{}", json!({"ok": false, "error": reject})).and_then(|()| out.flush());
return;
}
}
#[cfg(test)]
mod tests {
use super::*;
fn run(input: &[u8]) -> (Vec<Value>, Vec<String>) {
let mut out = Vec::new();
let mut calls = Vec::new();
serve_lines(input, &mut out, |m, _p| {
calls.push(m);
Some(json!({"ok": true, "result": null}))
});
let replies = String::from_utf8(out).unwrap().lines().map(|l| serde_json::from_str::<Value>(l).unwrap()).collect();
(replies, calls)
}
#[test]
fn requests_are_answered_in_order_with_ids() {
let (r, calls) = run(b"{\"id\":1,\"method\":\"a\"}\n\n{\"id\":2,\"method\":\"b\",\"params\":{}}\r\n{\"method\":\"c\"}");
assert_eq!(calls, ["a", "b", "c"]);
assert_eq!(r.len(), 3);
assert_eq!(r[0]["id"], 1);
assert_eq!(r[1]["id"], 2);
assert_eq!(r[2]["id"], Value::Null);
}
#[test]
fn http_request_closes_before_the_body_runs() {
// A browser's cross-origin `fetch` POST (issue #119): the request line is rejected and the
// connection closed, so the JSON body is never executed.
let http = b"POST / HTTP/1.1\r\nHost: 127.0.0.1:9876\r\nOrigin: https://attacker.example\r\nContent-Type: text/plain\r\n\r\n\
{\"id\":7,\"method\":\"engine.execute\",\"params\":{\"command\":\"file.openDemoProject\",\"params\":{}}}\n";
let (r, calls) = run(http);
assert!(calls.is_empty(), "nothing may run: {calls:?}");
assert_eq!(r.len(), 1);
assert_eq!(r[0]["ok"], false);
assert!(r[0]["error"].as_str().unwrap().contains("closing the connection"));
}
#[test]
fn non_request_json_closes_the_connection() {
for junk in [
&b"[1,2]\n{\"method\":\"a\"}\n"[..],
b"{\"id\":1}\n{\"method\":\"a\"}\n",
b"{\"method\":5}\n{\"method\":\"a\"}\n",
b"\xff\xfe\n{\"method\":\"a\"}\n",
] {
let (r, calls) = run(junk);
assert!(calls.is_empty(), "{calls:?}");
assert_eq!(r.len(), 1);
assert_eq!(r[0]["ok"], false);
}
}
#[test]
fn a_request_before_junk_still_runs() {
let (r, calls) = run(b"{\"method\":\"a\"}\nGET / HTTP/1.1\n{\"method\":\"b\"}\n");
assert_eq!(calls, ["a"]);
assert_eq!(r.len(), 2);
assert_eq!(r[1]["ok"], false);
}
#[test]
fn overlong_line_is_refused() {
// One byte over the limit, then a valid request: neither runs.
let mut input = vec![b'x'; MAX_LINE + 1];
input.extend_from_slice(b"\n{\"method\":\"a\"}\n");
let (r, calls) = run(&input);
assert!(calls.is_empty(), "{calls:?}");
assert_eq!(r.len(), 1);
assert!(r[0]["error"].as_str().unwrap().contains("longer than"));
}
#[test]
fn line_at_the_limit_is_accepted() {
let mut buf = Vec::new();
let line = vec![b'a'; 10];
let mut input = line.clone();
input.push(b'\n');
assert_eq!(read_bounded_line(&mut &input[..], &mut buf, 10), Ok(true));
assert_eq!(buf, line);
assert!(read_bounded_line(&mut &[b'a'; 11][..], &mut buf, 10).is_err());
}
#[test]
fn tcp_connection_is_closed_after_junk() {
// End to end over a real loopback socket (the transport the app uses).
let listener = TcpListener::bind(("127.0.0.1", 0)).unwrap();
let addr = listener.local_addr().unwrap();
let server = std::thread::spawn(move || {
let (s, _) = listener.accept().unwrap();
let read = s.try_clone().unwrap();
serve_lines(read, s, |_, _| Some(json!({"ok": true})));
});
let mut c = TcpStream::connect(addr).unwrap();
c.set_read_timeout(Some(Duration::from_secs(5))).unwrap();
c.write_all(b"{\"id\":7,\"method\":\"x\"}\nhello\n{\"method\":\"y\"}\n").unwrap();
let mut text = String::new();
c.read_to_string(&mut text).unwrap(); // returns at EOF: the server closed the connection
server.join().unwrap();
let lines: Vec<&str> = text.lines().collect();
assert_eq!(lines.len(), 2, "{text}");
assert!(lines[0].contains("\"id\":7"));
assert!(lines[1].contains("closing the connection"));
}
#[test]
fn extra_connections_are_refused() {
let listener = TcpListener::bind(("127.0.0.1", 0)).unwrap();
let addr = listener.local_addr().unwrap();
let (tx, _rx) = channel::<ControlRequest>();
std::thread::spawn(move || accept_loop(listener, tx, egui::Context::default()));
// Idle connections are accepted in order and held open.
let held: Vec<TcpStream> = (0..16).map(|_| TcpStream::connect(addr).unwrap()).collect();
let mut extra = TcpStream::connect(addr).unwrap();
extra.set_read_timeout(Some(Duration::from_secs(5))).unwrap();
let mut line = String::new();
BufReader::new(&mut extra).read_line(&mut line).unwrap();
assert!(line.contains("too many control connections"), "{line}");
drop(held);
}
}
+52
View File
@@ -0,0 +1,52 @@
//! Windows graphics instance configuration, applied before wgpu can initialize any driver.
use eframe::{NativeOptions, egui_wgpu::WgpuSetup, wgpu::Backends};
/// Restrict the wgpu instance to the primary backends (Vulkan, DirectX 12, Metal), i.e. leave
/// OpenGL out, unless `backend_override` (normally `Backends::from_env()`, i.e. `WGPU_BACKEND`)
/// names other backends.
pub fn configure(options: &mut NativeOptions, backend_override: Option<Backends>) {
if let WgpuSetup::CreateNew(create) = &mut options.wgpu_options.wgpu_setup {
// eframe's default includes GL. Creating that backend can crash inside AMD's
// atio6axx.dll before adapter selection or any Rust error handling runs, so the
// window never appears. Choosing an adapter afterwards is too late: only leaving GL
// out of the instance avoids it. Keep Vulkan and DX12 both: forcing DX12 alone sends
// shaders through FXC (which rejects fx.wgsl, silently falling back to the CPU) and
// leaves machines without DX12 with no backend at all.
create.instance_descriptor.backends = backend_override.unwrap_or(Backends::PRIMARY);
}
}
#[cfg(test)]
mod tests {
use super::*;
fn configured_backends(backend_override: Option<Backends>) -> Backends {
let mut options = NativeOptions::default();
configure(&mut options, backend_override);
let WgpuSetup::CreateNew(create) = options.wgpu_options.wgpu_setup else {
panic!("default native options must create a wgpu instance");
};
create.instance_descriptor.backends
}
#[test]
fn windows_default_leaves_out_gl() {
// eframe's own default (PRIMARY | GL) would initialize OpenGL as well.
let WgpuSetup::CreateNew(eframe_default) = NativeOptions::default().wgpu_options.wgpu_setup else {
panic!("default native options must create a wgpu instance");
};
assert!(eframe_default.instance_descriptor.backends.contains(Backends::GL));
let configured = configured_backends(None);
assert_eq!(configured, Backends::PRIMARY);
assert!(!configured.contains(Backends::GL));
assert!(configured.contains(Backends::DX12) && configured.contains(Backends::VULKAN));
}
#[test]
fn explicit_backend_override_is_preserved() {
for backends in [Backends::VULKAN, Backends::GL, Backends::DX12, Backends::VULKAN | Backends::DX12, Backends::empty()] {
assert_eq!(configured_backends(Some(backends)), backends);
}
}
}
+562
View File
@@ -0,0 +1,562 @@
//! The desktop app's logger: `log::` records go to standard error and to a log file.
//!
//! Without a logger every `log::warn!`/`log::error!` in the workspace (GPU compositor fallbacks,
//! audio output problems, a panicked frame job or audio mixer, container warnings) vanished. A
//! launch from a desktop menu has no terminal, so the file is what a bug report can attach:
//! `<data dir>/Logs/filmcraft.log`, next to the crash logs (`--data-dir`, `FILMCRAFT_DATA_DIR` or
//! `filmcraft_engine::autosave::default_data_dir`; Help ▸ Reveal Log Files opens the folder). Each
//! start moves the previous log to `filmcraft.1.log` (and that one to `.2`), so the log of a run
//! that crashed survives the next launch.
//!
//! Levels: `info` for FilmCraft's own crates, `warn` for everything else (wgpu and naga are
//! chatty). `RUST_LOG` replaces that with env_logger-style directives: `debug`,
//! `warn,filmcraft_codecs=trace`, `wgpu_core=info`. A directive ending in `*` matches every target
//! that starts with it (`filmcraft*=debug`).
//!
//! Records logged before the data directory is known are kept (up to [`MAX_PENDING`]) and written
//! once the file is attached. Writing never panics: a file that can't be created or written leaves
//! standard error as the only sink.
//!
//! Realtime audio: the cpal callbacks (`audio.rs`, `audio_in.rs`, the play-ahead reader) do not
//! log; they report through flags and a one-time `eprintln!`. A record takes the sink lock only for
//! one `write_all` of an already formatted line (standard error is written outside the lock), so a
//! logging thread such as the audio mixer never waits on more than one other line being written.
use std::fs::File;
use std::io::Write;
use std::path::{Path, PathBuf};
use std::sync::{Mutex, OnceLock, PoisonError};
use std::time::SystemTime;
use log::LevelFilter;
/// The current log file's name inside the log directory.
pub const LOG_FILE: &str = "filmcraft.log";
/// The log directory inside the data directory (shared with the crash logs and Reveal Log Files).
pub const LOG_DIR: &str = "Logs";
/// How many previous logs are kept (`filmcraft.1.log` … `filmcraft.<KEEP>.log`).
pub const KEEP: usize = 2;
/// The log file stops growing past this size (a runaway warning can't fill the disk).
pub const MAX_FILE_BYTES: u64 = 16 * 1024 * 1024;
/// Records kept in memory until the log file is attached.
pub const MAX_PENDING: usize = 512;
/// The built-in filter when `RUST_LOG` is unset or empty.
pub const DEFAULT_FILTER: &str = "warn,filmcraft*=info";
/// Per-target level filter parsed from env_logger-style directives.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Filter {
default: LevelFilter,
directives: Vec<(String, LevelFilter)>,
}
impl Filter {
/// Parse `spec`; unknown levels and empty parts are skipped. Without a bare level, targets no
/// directive names log errors only.
pub fn parse(spec: &str) -> Filter {
let mut f = Filter { default: LevelFilter::Error, directives: Vec::new() };
for part in spec.split(',').map(str::trim).filter(|p| !p.is_empty()) {
match part.split_once('=') {
Some((name, level)) => {
let name = name.trim();
if let (false, Some(level)) = (name.is_empty(), parse_level(level)) {
f.directives.push((name.to_owned(), level));
}
}
None => match parse_level(part) {
Some(level) => f.default = level,
// A bare target name: everything from it (env_logger does the same).
None => f.directives.push((part.to_owned(), LevelFilter::Trace)),
},
}
}
f
}
/// The level that applies to `target` (the most specific matching directive wins).
pub fn level_for(&self, target: &str) -> LevelFilter {
self.directives.iter().filter(|(name, _)| matches(name, target)).max_by_key(|(name, _)| name.len()).map_or(self.default, |(_, level)| *level)
}
/// The most verbose level any target can reach (for `log::set_max_level`).
pub fn max(&self) -> LevelFilter {
self.directives.iter().map(|(_, level)| *level).fold(self.default, Ord::max)
}
}
fn parse_level(s: &str) -> Option<LevelFilter> {
s.trim().parse().ok()
}
/// `name` matches `target` itself and its submodules (`a` matches `a` and `a::b`, not `ab`);
/// `name*` matches every target starting with `name`.
fn matches(name: &str, target: &str) -> bool {
match name.strip_suffix('*') {
Some(prefix) => target.starts_with(prefix),
None => target.strip_prefix(name).is_some_and(|rest| rest.is_empty() || rest.starts_with("::")),
}
}
/// `2026-10-08T07:59:17.728Z` (UTC, milliseconds). A clock before 1970 reads as the epoch.
pub fn timestamp(t: SystemTime) -> String {
let since = t.duration_since(std::time::UNIX_EPOCH).unwrap_or_default();
let secs = since.as_secs();
let (days, rem) = (secs / 86_400, secs % 86_400);
let (y, m, d) = civil_from_days(days);
format!("{y:04}-{m:02}-{d:02}T{:02}:{:02}:{:02}.{:03}Z", rem / 3600, rem % 3600 / 60, rem % 60, since.subsec_millis())
}
/// Days since 1970-01-01 to a proleptic Gregorian (year, month, day) (Howard Hinnant's
/// `civil_from_days`, unsigned because `timestamp` clamps at the epoch; saturating so an absurd
/// clock can't overflow).
fn civil_from_days(days: u64) -> (u64, u64, u64) {
let z = days.saturating_add(719_468);
let era = z / 146_097;
let doe = z % 146_097;
let yoe = (doe - doe / 1460 + doe / 36_524 - doe / 146_096) / 365;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let d = doy - (153 * mp + 2) / 5 + 1;
let m = if mp < 10 { mp + 3 } else { mp - 9 };
let y = era.saturating_mul(400).saturating_add(yoe).saturating_add(u64::from(m <= 2));
(y, m, d)
}
fn numbered(dir: &Path, n: usize) -> PathBuf {
if n == 0 { dir.join(LOG_FILE) } else { dir.join(format!("filmcraft.{n}.log")) }
}
/// Shift the previous logs up one (`.log` → `.1.log` → … → `.<KEEP>.log`, the oldest dropped)
/// and create a fresh, empty log file in `dir` (created if missing).
pub fn rotate(dir: &Path) -> std::io::Result<(PathBuf, File)> {
std::fs::create_dir_all(dir)?;
for n in (1..=KEEP).rev() {
match std::fs::rename(numbered(dir, n - 1), numbered(dir, n)) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => return Err(e),
_ => {}
}
}
let path = numbered(dir, 0);
let file = File::create(&path)?;
Ok((path, file))
}
/// One formatted log line: `<timestamp> <LEVEL> [<thread>] <target>: <message>\n`.
pub fn format_line(ts: &str, level: log::Level, thread: &str, target: &str, message: &str) -> String {
let message = message.strip_suffix('\n').unwrap_or(message);
format!("{ts} {level:<5} [{thread}] {target}: {message}\n")
}
/// Where formatted lines go besides standard error: kept in memory until a file is attached,
/// then appended to it until it reaches its size cap.
pub struct Sink {
file: Option<File>,
pending: Vec<String>,
dropped: usize,
written: u64,
max_bytes: u64,
capped: bool,
}
impl Sink {
pub fn new(max_bytes: u64) -> Sink {
Sink { file: None, pending: Vec::new(), dropped: 0, written: 0, max_bytes, capped: false }
}
/// Append `line` to the file, or keep it until [`Sink::attach`].
pub fn write(&mut self, line: &str) {
if self.file.is_none() {
if self.capped {
return;
}
if self.pending.len() < MAX_PENDING {
self.pending.push(line.to_owned());
} else {
self.dropped = self.dropped.saturating_add(1);
}
return;
}
self.append(line);
}
/// Start writing to `file`, first the lines kept so far.
pub fn attach(&mut self, file: File) {
self.file = Some(file);
for line in std::mem::take(&mut self.pending) {
self.append(&line);
}
if self.dropped > 0 {
let note = format!("{} earlier log lines were dropped before the log file was opened\n", self.dropped);
self.dropped = 0;
self.append(&note);
}
}
/// No file will come: drop the kept lines and keep none from now on.
pub fn close(&mut self) {
self.file = None;
self.pending = Vec::new();
self.capped = true;
}
fn append(&mut self, line: &str) {
if self.capped {
return;
}
let Some(file) = self.file.as_mut() else { return };
let len = u64::try_from(line.len()).unwrap_or(u64::MAX);
let text = if self.written.saturating_add(len) > self.max_bytes {
self.capped = true;
format!("log file reached {} bytes; later records go to standard error only\n", self.max_bytes)
} else {
self.written = self.written.saturating_add(len);
line.to_owned()
};
// A full disk or a vanished file must not take the app down: drop the file sink (and
// stop buffering, since no file will come).
if file.write_all(text.as_bytes()).is_err() {
self.file = None;
self.capped = true;
}
}
}
/// The installed logger: a [`Filter`], standard error and a [`Sink`].
pub struct AppLogger {
filter: Filter,
stderr: bool,
sink: Mutex<Sink>,
}
impl AppLogger {
pub fn new(filter: Filter, stderr: bool) -> AppLogger {
AppLogger { filter, stderr, sink: Mutex::new(Sink::new(MAX_FILE_BYTES)) }
}
/// Rotate the logs in `dir` and send records to the fresh file; returns its path.
/// An unusable directory leaves standard error as the only sink ([`AppLogger::stderr_only`]).
pub fn attach_dir(&self, dir: &Path) -> Result<PathBuf, String> {
match rotate(dir) {
Ok((path, file)) => {
self.sink.lock().unwrap_or_else(PoisonError::into_inner).attach(file);
Ok(path)
}
Err(e) => {
self.stderr_only();
Err(format!("{}: {e}", dir.display()))
}
}
}
/// No log file this run (no data directory, or it can't be used): stop keeping records for one.
pub fn stderr_only(&self) {
self.sink.lock().unwrap_or_else(PoisonError::into_inner).close();
}
}
impl log::Log for AppLogger {
fn enabled(&self, metadata: &log::Metadata<'_>) -> bool {
metadata.level() <= self.filter.level_for(metadata.target())
}
fn log(&self, record: &log::Record<'_>) {
if !self.enabled(record.metadata()) {
return;
}
// Format before taking the lock, so the lock covers one write only.
let thread = std::thread::current();
let line = format_line(&timestamp(SystemTime::now()), record.level(), thread.name().unwrap_or("?"), record.target(), &record.args().to_string());
if self.stderr {
let _ = std::io::stderr().write_all(line.as_bytes());
}
self.sink.lock().unwrap_or_else(PoisonError::into_inner).write(&line);
}
fn flush(&self) {}
}
/// Install the logger (filter from `RUST_LOG`, else [`DEFAULT_FILTER`]); `None` when another
/// logger was installed first. Call [`AppLogger::attach_dir`] once the data directory is known.
pub fn install() -> Option<&'static AppLogger> {
static LOGGER: OnceLock<AppLogger> = OnceLock::new();
let spec = std::env::var("RUST_LOG").ok().filter(|s| !s.trim().is_empty());
let filter = Filter::parse(spec.as_deref().unwrap_or(DEFAULT_FILTER));
let max = filter.max();
let logger = LOGGER.get_or_init(|| AppLogger::new(filter, true));
log::set_logger(logger).ok()?;
log::set_max_level(max);
Some(logger)
}
#[cfg(test)]
mod tests {
use super::*;
use log::Log;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::{Duration, UNIX_EPOCH};
fn temp_dir(tag: &str) -> PathBuf {
static N: AtomicUsize = AtomicUsize::new(0);
let d = std::env::temp_dir().join(format!("filmcraft-logging-{tag}-{}-{}", std::process::id(), N.fetch_add(1, Ordering::Relaxed)));
let _ = std::fs::remove_dir_all(&d);
d
}
fn read(path: &Path) -> String {
std::fs::read_to_string(path).expect("read log")
}
fn record(logger: &AppLogger, level: log::Level, target: &'static str, msg: &'static str) {
logger.log(&log::Record::builder().level(level).target(target).args(format_args!("{msg}")).build());
}
#[test]
fn the_default_filter_shows_filmcraft_info_and_other_crates_warnings() {
let f = Filter::parse(DEFAULT_FILTER);
assert_eq!(f.level_for("filmcraft"), LevelFilter::Info);
assert_eq!(f.level_for("filmcraft::audio"), LevelFilter::Info);
assert_eq!(f.level_for("filmcraft_ui_egui::play_ahead"), LevelFilter::Info);
assert_eq!(f.level_for("filmcraft_codecs::mxf"), LevelFilter::Info);
assert_eq!(f.level_for("wgpu_core::device"), LevelFilter::Warn);
assert_eq!(f.level_for("naga"), LevelFilter::Warn);
assert_eq!(f.max(), LevelFilter::Info);
}
#[test]
fn directives_follow_env_logger_and_the_most_specific_one_wins() {
let f = Filter::parse("info,wgpu_core=error,filmcraft_codecs=trace,filmcraft_codecs::mxf=off");
assert_eq!(f.level_for("eframe"), LevelFilter::Info);
assert_eq!(f.level_for("wgpu_core::instance"), LevelFilter::Error);
assert_eq!(f.level_for("filmcraft_codecs::ogg"), LevelFilter::Trace);
assert_eq!(f.level_for("filmcraft_codecs::mxf"), LevelFilter::Off);
assert_eq!(f.max(), LevelFilter::Trace);
// A module name is matched at `::` boundaries, not as a bare prefix.
assert_eq!(f.level_for("wgpu_core_extra"), LevelFilter::Info);
// A trailing `*` is a prefix.
assert_eq!(Filter::parse("warn,filmcraft_ui*=debug").level_for("filmcraft_ui_egui::frames"), LevelFilter::Debug);
assert_eq!(Filter::parse("warn,filmcraft_ui*=debug").level_for("filmcraft_engine"), LevelFilter::Warn);
// A bare target name sets that target to the most verbose level, as env_logger does.
assert_eq!(Filter::parse("naga").level_for("naga::front"), LevelFilter::Trace);
assert_eq!(Filter::parse("naga").level_for("eframe"), LevelFilter::Error);
// Levels are case-insensitive; the last bare level wins.
assert_eq!(Filter::parse("debug,WARN").level_for("x"), LevelFilter::Warn);
}
#[test]
fn hostile_specs_never_panic_and_fall_back_sensibly() {
let long = "a".repeat(100_000);
let specs = [
"",
",,,",
"=",
"==",
"=debug",
"filmcraft=",
"nonsense=loud",
"🦀=info",
"*",
"*=debug",
"a=b=c",
"warn,,filmcraft=DEBUG",
" \t ",
"\u{0}",
"é*=trace",
&long,
];
for spec in specs {
let f = Filter::parse(spec);
for target in ["filmcraft", "", "🦀", "é", "a::b"] {
let _ = f.level_for(target);
}
let _ = f.max();
}
assert_eq!(Filter::parse("warn,,filmcraft=DEBUG").level_for("filmcraft"), LevelFilter::Debug);
assert_eq!(Filter::parse("nonsense=loud").level_for("nonsense"), LevelFilter::Error);
assert_eq!(Filter::parse("=debug").max(), LevelFilter::Error, "a directive without a name is skipped");
assert_eq!(Filter::parse("*=debug").level_for("anything"), LevelFilter::Debug, "a lone `*` matches every target");
assert_eq!(Filter::parse("é*=trace").level_for("éa"), LevelFilter::Trace);
}
#[test]
fn timestamps_are_utc_with_milliseconds() {
assert_eq!(timestamp(UNIX_EPOCH), "1970-01-01T00:00:00.000Z");
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_millis(1_791_446_357_728)), "2026-10-08T07:59:17.728Z");
// Leap day, and the day after it.
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(1_709_164_800)), "2024-02-29T00:00:00.000Z");
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(1_709_251_199)), "2024-02-29T23:59:59.000Z");
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(1_709_251_200)), "2024-03-01T00:00:00.000Z");
// 2000 is a leap year (divisible by 400), 2100 is not.
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(951_782_400)), "2000-02-29T00:00:00.000Z");
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(4_107_542_400)), "2100-03-01T00:00:00.000Z");
// Before the epoch (a clock set wrong) clamps instead of panicking.
assert_eq!(timestamp(UNIX_EPOCH - Duration::from_secs(5)), "1970-01-01T00:00:00.000Z");
}
#[test]
fn rotation_keeps_the_previous_logs_and_starts_an_empty_file() {
let dir = temp_dir("rotate");
for run in 1..=4 {
let (path, _file) = rotate(&dir).expect("rotate");
assert_eq!(path, dir.join(LOG_FILE));
assert_eq!(read(&path), "");
std::fs::write(&path, format!("run {run}")).expect("write");
}
assert_eq!(read(&dir.join(LOG_FILE)), "run 4");
assert_eq!(read(&dir.join("filmcraft.1.log")), "run 3");
assert_eq!(read(&dir.join("filmcraft.2.log")), "run 2");
assert!(!dir.join("filmcraft.3.log").exists(), "only {KEEP} old logs are kept");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn rotation_into_an_unusable_directory_is_an_error_not_a_panic() {
let dir = temp_dir("blocked");
std::fs::create_dir_all(dir.parent().expect("parent")).expect("tmp");
std::fs::write(&dir, "a file where the directory should be").expect("block");
assert!(rotate(&dir).is_err());
assert!(rotate(&dir.join("below")).is_err());
let _ = std::fs::remove_file(&dir);
}
#[test]
fn a_line_carries_time_level_thread_target_and_message() {
assert_eq!(
format_line("2026-10-08T07:59:17.728Z", log::Level::Warn, "main", "filmcraft_ui_egui", "audio output unavailable: no output device"),
"2026-10-08T07:59:17.728Z WARN [main] filmcraft_ui_egui: audio output unavailable: no output device\n"
);
// A multi-line message keeps its lines; the record still ends in exactly one newline.
assert!(format_line("t", log::Level::Error, "w", "x", "a\nb\n").ends_with("x: a\nb\n"));
}
#[test]
fn lines_logged_before_the_file_exists_are_written_when_it_is_attached() {
let dir = temp_dir("pending");
let (path, file) = rotate(&dir).expect("rotate");
let mut sink = Sink::new(MAX_FILE_BYTES);
sink.write("early 1\n");
sink.write("early 2\n");
sink.attach(file);
sink.write("late\n");
assert_eq!(read(&path), "early 1\nearly 2\nlate\n");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn the_pending_buffer_is_bounded_and_says_how_much_it_dropped() {
let dir = temp_dir("bounded");
let (path, file) = rotate(&dir).expect("rotate");
let mut sink = Sink::new(MAX_FILE_BYTES);
for i in 0..MAX_PENDING + 5 {
sink.write(&format!("line {i}\n"));
}
sink.attach(file);
let text = read(&path);
assert_eq!(text.lines().filter(|l| l.starts_with("line ")).count(), MAX_PENDING);
assert_eq!(text.matches("5 earlier log lines were dropped").count(), 1, "{text}");
assert!(text.contains("line 0\n") && !text.contains(&format!("line {MAX_PENDING}\n")));
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn exactly_max_pending_lines_drop_nothing() {
let dir = temp_dir("exact");
let (path, file) = rotate(&dir).expect("rotate");
let mut sink = Sink::new(MAX_FILE_BYTES);
for i in 0..MAX_PENDING {
sink.write(&format!("line {i}\n"));
}
sink.attach(file);
let text = read(&path);
assert_eq!(text.lines().count(), MAX_PENDING);
assert!(!text.contains("dropped"), "{text}");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn the_file_stops_at_its_size_cap_with_one_note() {
let dir = temp_dir("cap");
let (path, file) = rotate(&dir).expect("rotate");
let mut sink = Sink::new(100);
sink.attach(file);
for i in 0..50 {
sink.write(&format!("line {i:02} with some padding\n"));
}
let text = read(&path);
assert_eq!(text.lines().filter(|l| l.starts_with("line ")).count(), 3, "{text}");
assert_eq!(text.matches("log file reached 100 bytes").count(), 1, "{text}");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn the_logger_filters_records_and_writes_them_to_the_attached_file() {
let dir = temp_dir("logger");
let logger = AppLogger::new(Filter::parse(DEFAULT_FILTER), false);
record(&logger, log::Level::Info, "filmcraft", "FilmCraft starting");
record(&logger, log::Level::Info, "wgpu_core::device", "too chatty");
let path = logger.attach_dir(&dir).expect("attach");
assert_eq!(path, dir.join(LOG_FILE));
record(&logger, log::Level::Warn, "wgpu_hal::vulkan", "a real warning");
record(&logger, log::Level::Debug, "filmcraft", "below info");
let text = read(&path);
let lines: Vec<&str> = text.lines().collect();
assert_eq!(lines.len(), 2, "{text}");
assert!(lines.first().is_some_and(|l| l.contains(" INFO [") && l.ends_with("filmcraft: FilmCraft starting")), "{text}");
assert!(lines.get(1).is_some_and(|l| l.ends_with("wgpu_hal::vulkan: a real warning")), "{text}");
assert!(logger.enabled(&log::Metadata::builder().level(log::Level::Info).target("filmcraft_engine::autosave").build()));
assert!(!logger.enabled(&log::Metadata::builder().level(log::Level::Info).target("naga").build()));
// A second launch rotates: the first log becomes `.1`.
let second = AppLogger::new(Filter::parse(DEFAULT_FILTER), false);
assert_eq!(second.attach_dir(&dir).expect("attach again"), dir.join(LOG_FILE));
assert_eq!(read(&dir.join("filmcraft.1.log")), text);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn records_from_many_threads_are_whole_lines() {
let dir = temp_dir("threads");
let logger = AppLogger::new(Filter::parse("info"), false);
let path = logger.attach_dir(&dir).expect("attach");
std::thread::scope(|s| {
for _ in 0..4 {
s.spawn(|| {
for _ in 0..100 {
record(&logger, log::Level::Info, "filmcraft", "from a worker");
}
});
}
});
let text = read(&path);
assert_eq!(text.lines().count(), 400);
assert!(text.lines().all(|l| l.ends_with("filmcraft: from a worker")), "{text}");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn attaching_to_an_unusable_directory_reports_an_error_and_keeps_logging() {
let dir = temp_dir("unusable");
std::fs::write(&dir, "not a directory").expect("block");
let logger = AppLogger::new(Filter::parse("info"), false);
let e = logger.attach_dir(&dir).expect_err("a file is not a log directory");
assert!(e.contains(&dir.display().to_string()), "{e}");
record(&logger, log::Level::Error, "x", "still fine");
let sink = logger.sink.lock().expect("sink");
assert!(sink.file.is_none() && sink.pending.is_empty(), "nothing is kept for a file that will not come");
drop(sink);
let _ = std::fs::remove_file(&dir);
}
#[test]
fn a_poisoned_lock_does_not_stop_logging() {
let dir = temp_dir("poison");
let logger = AppLogger::new(Filter::parse("info"), false);
let path = logger.attach_dir(&dir).expect("attach");
let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
let _guard = logger.sink.lock();
panic!("poison the sink");
}));
assert!(logger.sink.is_poisoned());
record(&logger, log::Level::Info, "filmcraft", "after the panic");
assert!(read(&path).contains("after the panic"));
let _ = std::fs::remove_dir_all(&dir);
}
}
+134 -42
View File
@@ -1,30 +1,40 @@
//! FilmCraft desktop app.
//!
//! Usage: `filmcraft [--control <port>] [--demo|--empty] [--recover|--no-recover] [--data-dir <dir>]
//! [project.fcproj | media files…]`
//! [project.fcproj | media files…]`; `--help` prints the options, `--version` the version. An
//! unknown option is an error (exit code 2), not a file to open.
//!
//! Without a project, `--demo` or `--empty`, Settings ▸ General ▸ At Startup decides: Show Home
//! (the demo project), Open Most Recent, or an empty project.
//!
//! `--control <port>` (or `FILMCRAFT_CONTROL_PORT`) starts a localhost JSON-lines control server;
//! see `filmcraft_ui_egui::control` for the methods.
//! see `filmcraft_ui_egui::control` for the methods. A port that is not a number is an error from
//! either source.
//!
//! Auto-save and the crash-recovery journal run in every session (data in `--data-dir`, else
//! `FILMCRAFT_DATA_DIR`, else the per-user application data folder). If a previous session died
//! with unsaved changes the app asks to recover them; `--recover` recovers the newest without
//! asking, `--no-recover` starts without asking (the changes stay available via File ▸ Recover
//! Unsaved Changes…).
//!
//! `log` records go to standard error and `<data dir>/Logs/filmcraft.log` (`RUST_LOG` sets the
//! levels); see [`logging`].
#![cfg_attr(all(target_os = "windows", not(debug_assertions)), windows_subsystem = "windows")]
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
mod app_nap;
mod args;
mod audio;
mod audio_in;
mod control_server;
#[cfg(any(target_os = "windows", test))]
mod graphics;
mod logging;
#[cfg(target_os = "macos")]
mod native_menu;
mod window_raise;
use args::{Cli, Launch};
use chrono::TimeZone;
use filmcraft_engine::Session;
use filmcraft_engine::autosave::{AutosaveConfig, default_data_dir};
@@ -45,37 +55,63 @@ fn app_icon() -> egui::IconData {
eframe::icon_data::from_png_bytes(png).unwrap_or_default()
}
/// Print `text` (stdout, or stderr for an error). A Windows release build has no console
/// (`windows_subsystem = "windows"`), so there it is also shown in a message box: otherwise `--help`
/// would print nothing and a bad option would quit without a word.
fn tell(text: &str, error: bool) {
if error {
eprint!("{text}");
} else {
print!("{text}");
}
#[cfg(all(target_os = "windows", not(debug_assertions)))]
{
let level = if error { rfd::MessageLevel::Error } else { rfd::MessageLevel::Info };
let _ = rfd::MessageDialog::new().set_level(level).set_title("FilmCraft").set_description(text).show();
}
}
fn main() -> eframe::Result {
app_nap::disable();
let mut control_port: Option<u16> = std::env::var("FILMCRAFT_CONTROL_PORT").ok().and_then(|p| p.parse().ok());
let mut files = Vec::new();
let mut demo = true;
// --demo / --empty given: skip Settings ▸ General ▸ At Startup
let mut startup_flag = false;
let mut recover: Option<bool> = None;
let mut data_dir = None;
let mut args = std::env::args().skip(1);
while let Some(a) = args.next() {
match a.as_str() {
"--control" => control_port = args.next().and_then(|p| p.parse().ok()),
"--demo" => (demo, startup_flag) = (true, true),
"--empty" => (demo, startup_flag) = (false, true),
"--recover" => recover = Some(true),
"--no-recover" => recover = Some(false),
"--data-dir" => data_dir = args.next().map(std::path::PathBuf::from),
"--version" => {
println!("filmcraft {}", env!("CARGO_PKG_VERSION"));
return Ok(());
// First, so the panic hook and every start-up record are logged (`logging`).
let logger = logging::install();
// lossy, so a value that is not Unicode is reported like any other bad port instead of ignored
let env_port = std::env::var_os("FILMCRAFT_CONTROL_PORT").map(|p| p.to_string_lossy().into_owned());
let Launch { control_port, files, demo, startup_flag, recover, data_dir } = match args::parse(env_port, std::env::args().skip(1)) {
Ok(Cli::Run(l)) => l,
Ok(Cli::Help) => {
tell(args::USAGE, false);
return Ok(());
}
Ok(Cli::Version) => {
tell(&format!("filmcraft {}\n", env!("CARGO_PKG_VERSION")), false);
return Ok(());
}
Err(e) => {
tell(&format!("filmcraft: {e}\n\n{}", args::USAGE), true);
std::process::exit(2);
}
};
let log_dir = data_dir.clone().or_else(default_data_dir).map(|d| d.join(logging::LOG_DIR));
log::info!("FilmCraft {} ({} {})", env!("CARGO_PKG_VERSION"), std::env::consts::OS, std::env::consts::ARCH);
// The log file lives next to the crash logs; opened after the arguments, so `--help`,
// `--version` and usage errors leave no file behind. Records logged until now go in first.
if let Some(logger) = logger {
match log_dir.as_deref().map(|d| logger.attach_dir(d)) {
Some(Ok(path)) => log::info!("log file {}", path.display()),
Some(Err(e)) => log::warn!("no log file: {e}"),
None => {
logger.stderr_only();
log::warn!("no log file: no data directory (set FILMCRAFT_DATA_DIR or --data-dir)");
}
_ => files.push(a),
}
}
app_nap::disable();
// Panics anywhere go to <data dir>/Logs/crash-<day>.log with a backtrace; the UI pass and
// frame workers catch them and keep running (see filmcraft_ui_egui::crash).
filmcraft_ui_egui::crash::install(data_dir.clone().or_else(default_data_dir).map(|d| d.join("Logs")));
filmcraft_ui_egui::crash::install(log_dir);
// OS hardware video decoders (VideoToolbox on macOS) in front of our own; Settings ▸ Playback ▸
// Hardware decoding switches them off. Unsupported streams and failures use our decoders.
log::info!("hardware decoding: {:?}", filmcraft_platform::register());
register_hardware_decoders();
let options = eframe::NativeOptions {
viewport: egui::ViewportBuilder::default()
.with_title("FilmCraft")
@@ -93,6 +129,13 @@ fn main() -> eframe::Result {
event_loop_builder: agent_event_loop(control_port.is_some()),
..Default::default()
};
// Before eframe creates the wgpu instance: leave OpenGL out on Windows (see graphics.rs).
#[cfg(target_os = "windows")]
let options = {
let mut options = options;
graphics::configure(&mut options, eframe::wgpu::Backends::from_env());
options
};
let started = eframe::run_native(
"FilmCraft",
options,
@@ -102,7 +145,7 @@ fn main() -> eframe::Result {
let mut cfg = AutosaveConfig::new(dir);
cfg.local_offset = local_offset;
if let Err(e) = session.start_autosave(cfg) {
eprintln!("filmcraft: auto-save and crash recovery unavailable: {e}");
log::warn!("auto-save and crash recovery unavailable: {e}");
}
}
// voice-over recording reads the microphone through cpal
@@ -110,7 +153,7 @@ fn main() -> eframe::Result {
let project = files.iter().find(|f| f.ends_with(".fcproj")).cloned();
if let Some(p) = project {
if let Err(e) = session.execute("file.open", json!({"path": p})) {
eprintln!("filmcraft: {e}");
log::error!("{e}");
}
} else if !startup_flag && session.prefs.general.at_startup == "openMostRecent" {
// Settings ▸ General ▸ At Startup ▸ Open Most Recent
@@ -118,7 +161,7 @@ fn main() -> eframe::Result {
match recent {
Some(p) => {
if let Err(e) = session.execute("file.open", json!({"path": p})) {
eprintln!("filmcraft: {e}");
log::error!("{e}");
}
}
None => {
@@ -136,8 +179,8 @@ fn main() -> eframe::Result {
if recover == Some(true) && !session.recovery_candidates().is_empty() {
let id = session.recovery_candidates()[0].id.clone();
match session.execute("file.recover", json!({"id": id})) {
Ok(r) => eprintln!("filmcraft: recovered {r}"),
Err(e) => eprintln!("filmcraft: recovery failed: {e}"),
Ok(r) => log::info!("recovered {r}"),
Err(e) => log::error!("recovery failed: {e}"),
}
}
let mut app = FilmcraftApp::new(session);
@@ -150,29 +193,54 @@ fn main() -> eframe::Result {
{
app.set_wgpu(rs);
}
if let Some(out) = audio::CpalOut::new() {
// Settings ▸ Audio Hardware is applied on the first frame (`apply_prefs`)
app.audio = Some(Box::new(out));
}
// Keep device selection available even when the default device is unavailable.
// Settings ▸ Audio Hardware is applied on the first frame (`apply_prefs`).
app.audio = Some(Box::new(audio::CpalOut::new()));
app.hooks.pick_files = Some(Box::new(|exts: &[&str]| {
rfd::FileDialog::new().add_filter("Media", exts).pick_files().unwrap_or_default().into_iter().map(|p| p.to_string_lossy().to_string()).collect()
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t("Media"), exts)
.pick_files()
.unwrap_or_default()
.into_iter()
.map(|p| p.to_string_lossy().to_string())
.collect()
}));
// Link Media ▸ Locate…, Attach Proxies, Reconnect Full Resolution: one path, not imported.
app.hooks.pick_file_for_relink = Some(Box::new(|exts: &[&str], _hint| {
rfd::FileDialog::new().add_filter(filmcraft_ui_egui::i18n::t("Media"), exts).pick_file().map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_save = Some(Box::new(|name: &str| {
rfd::FileDialog::new().add_filter("FilmCraft Project", &["fcproj"]).set_file_name(name).save_file().map(|p| p.to_string_lossy().to_string())
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t("FilmCraft Project"), &["fcproj"])
.set_file_name(name)
.save_file()
.map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_save_as = Some(Box::new(|filter: &str, exts: &[&str], name: &str| {
rfd::FileDialog::new().add_filter(filter, exts).set_file_name(name).save_file().map(|p| p.to_string_lossy().to_string())
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t(filter), exts)
.set_file_name(name)
.save_file()
.map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_folder = Some(Box::new(|| rfd::FileDialog::new().pick_folder().map(|p| p.to_string_lossy().to_string())));
app.hooks.pick_folder_at =
Some(Box::new(|dir: &str| rfd::FileDialog::new().set_directory(dir).pick_folder().map(|p| p.to_string_lossy().into_owned())));
app.hooks.open_path = Some(Box::new(open_path));
// Settings ▸ General ▸ Interface Language ▸ System Language (#218).
app.hooks.system_languages = Some(Box::new(|| sys_locale::get_locales().collect()));
app.hooks.raise_without_focus = Some(Box::new(|| {
window_raise::raise_without_focus();
}));
app.hooks.pick_open_file = Some(Box::new(|filter: &str, exts: &[&str]| {
rfd::FileDialog::new().add_filter(filter, exts).pick_file().map(|p| p.to_string_lossy().to_string())
rfd::FileDialog::new().add_filter(filmcraft_ui_egui::i18n::t(filter), exts).pick_file().map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_open_project = Some(Box::new(|| {
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t("FilmCraft Project"), &["fcproj"])
.pick_file()
.map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_open_project =
Some(Box::new(|| rfd::FileDialog::new().add_filter("FilmCraft Project", &["fcproj"]).pick_file().map(|p| p.to_string_lossy().to_string())));
#[cfg(target_os = "macos")]
{
let (rx, update) = native_menu::install(&app, cc.egui_ctx.clone());
@@ -191,7 +259,7 @@ fn main() -> eframe::Result {
// instead of exiting silently, and keep the reason in the crash log.
if let Err(e) = &started {
let msg = format!("FilmCraft could not start its window: {e}\n\nUpdating the graphics driver usually fixes this.");
eprintln!("filmcraft: {msg}");
log::error!("{msg}");
filmcraft_ui_egui::crash::record(&msg);
let _ = rfd::MessageDialog::new().set_title("FilmCraft").set_description(&msg).set_level(rfd::MessageLevel::Error).show();
}
@@ -225,6 +293,15 @@ fn open_path(path: &str, reveal: bool) -> Result<(), String> {
cmd.spawn().map(|_| ()).map_err(|e| format!("can't open {path}: {e}"))
}
/// Put the OS hardware video decoders in front of our own. Registered in a statement of its own:
/// a log macro does not evaluate its arguments while no logger takes its level, which left the
/// hardware decoders out of every run.
fn register_hardware_decoders() -> filmcraft_platform::Availability {
let hardware = filmcraft_platform::register();
log::info!("hardware decoding: {hardware:?}");
hardware
}
/// Local UTC offset at a unix time (auto-save file names and recovery times use local time).
fn local_offset(unix: i64) -> i32 {
chrono::Local.timestamp_opt(unix, 0).single().map(|d| d.offset().local_minus_utc()).unwrap_or(0)
@@ -243,3 +320,18 @@ fn agent_event_loop(agent: bool) -> Option<eframe::EventLoopBuilderHook> {
let _ = agent;
None
}
#[cfg(test)]
mod tests {
/// Start-up registers the hardware decoders whether or not anything is logged (no logger is
/// installed here, as in a normal run).
#[test]
fn startup_registers_the_hardware_decoders_without_a_logger() {
assert!(!log::log_enabled!(log::Level::Info));
let hardware = super::register_hardware_decoders();
// always on macOS and Windows; on Linux when a VA-API driver is there
let expected = cfg!(any(target_os = "macos", target_os = "windows"))
|| (cfg!(target_os = "linux") && matches!(hardware, filmcraft_platform::Availability::Available(_)));
assert_eq!(filmcraft_platform::registered(), expected, "{hardware:?}");
}
}
+21 -3
View File
@@ -25,10 +25,12 @@ pub type ShortcutUpdater = Box<dyn FnMut(&[Item])>;
pub fn install(app: &FilmcraftApp, ctx: egui::Context) -> (Receiver<String>, ShortcutUpdater) {
let items = menu_items(app);
let mut native: Vec<(String, MenuItem)> = Vec::new();
let mut titles: Vec<(String, Submenu)> = Vec::new();
let bar = Menu::new();
let app_menu = Submenu::new("FilmCraft", true);
// FilmCraft ▸ Settings ▸ <category> (Premiere's app-menu layout; General is Cmd+,)
let settings = Submenu::new("Settings", true);
let settings = Submenu::new(app.ui.language.tr("Settings"), true);
titles.push(("Settings".into(), settings.clone()));
for it in items.iter().filter(|i| i.id.starts_with("app.settings.")) {
let label = it.label.trim_end_matches('…').to_string();
let mi = MenuItem::with_id(it.id.clone(), label, true, it.shortcut.as_deref().and_then(accel));
@@ -48,7 +50,8 @@ pub fn install(app: &FilmcraftApp, ctx: egui::Context) -> (Receiver<String>, Sho
]);
let _ = bar.append(&app_menu);
for top in MENUS {
let sub = Submenu::new(top, true);
let sub = Submenu::new(app.ui.language.tr(top), true);
titles.push((top.into(), sub.clone()));
let mine: Vec<&Item> = items.iter().filter(|i| i.path.first().map(String::as_str) == Some(top) && !i.id.starts_with("app.settings.")).collect();
// submenus by path prefix (e.g. Clip ▸ Video Options ▸ Time Interpolation), created where
// their first item appears
@@ -62,7 +65,8 @@ pub fn install(app: &FilmcraftApp, ctx: egui::Context) -> (Receiver<String>, Sho
parent = match subs.iter().find(|(k, _)| *k == key) {
Some((_, s)) => s.clone(),
None => {
let s = Submenu::new(&it.path[depth], true);
let s = Submenu::new(app.ui.language.tr(&it.path[depth]), true);
titles.push((it.path[depth].clone(), s.clone()));
let _ = parent.append(&s);
subs.push((key, s.clone()));
s
@@ -88,8 +92,22 @@ pub fn install(app: &FilmcraftApp, ctx: egui::Context) -> (Receiver<String>, Sho
}
});
let update: ShortcutUpdater = Box::new(move |items: &[Item]| {
let checked = |id: &str| items.iter().any(|it| it.id == id && it.checked == Some(true));
let language = if checked("app.language.japanese") {
filmcraft_ui_egui::i18n::Language::Ja
} else if checked("app.language.spanish") {
filmcraft_ui_egui::i18n::Language::Es
} else if checked("app.language.portuguese") {
filmcraft_ui_egui::i18n::Language::PtBr
} else {
filmcraft_ui_egui::i18n::Language::En
};
for (title, submenu) in &titles {
submenu.set_text(language.tr(title));
}
for (id, mi) in &native {
if let Some(it) = items.iter().find(|i| &i.id == id) {
mi.set_text(&it.label);
let _ = mi.set_accelerator(it.shortcut.as_deref().and_then(accel));
}
}
+574
View File
@@ -0,0 +1,574 @@
{
"schema": 1,
"repo": "storytold/filmcraft",
"branch": "main",
"head": "7b6c134472287db4367fca80bcb498240ba36567",
"head_date": "2026-10-09T15:02:38Z",
"commits": 539,
"unattributed_commits": 0,
"contributors": [
{
"login": "echelon",
"people_entry": true,
"github_name": null,
"prs": 48,
"commits": 337,
"lines_added": 311167,
"lines_deleted": 13200,
"binary_added": 74,
"binary_deleted": 0,
"first_commit": "2026-09-30T09:22:00Z",
"last_commit": "2026-10-09T15:02:38Z"
},
{
"login": "storybored379",
"people_entry": false,
"github_name": null,
"prs": 15,
"commits": 17,
"lines_added": 1960,
"lines_deleted": 179,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-03T16:31:42Z",
"last_commit": "2026-10-09T04:35:58Z"
},
{
"login": "justjohn12345",
"people_entry": false,
"github_name": null,
"prs": 15,
"commits": 37,
"lines_added": 9087,
"lines_deleted": 938,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-05T19:57:28Z",
"last_commit": "2026-10-08T17:06:13Z"
},
{
"login": "punkhop",
"people_entry": false,
"github_name": null,
"prs": 26,
"commits": 31,
"lines_added": 4977,
"lines_deleted": 304,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-06T00:15:40Z",
"last_commit": "2026-10-07T07:35:29Z"
},
{
"login": "midasdf",
"people_entry": false,
"github_name": null,
"prs": 4,
"commits": 4,
"lines_added": 618,
"lines_deleted": 32,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-06T12:41:17Z",
"last_commit": "2026-10-08T15:40:01Z"
},
{
"login": "EthanSuquet",
"people_entry": false,
"github_name": null,
"prs": 4,
"commits": 4,
"lines_added": 369,
"lines_deleted": 35,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-06T19:16:37Z",
"last_commit": "2026-10-07T13:41:14Z"
},
{
"login": "dcarnelutti",
"people_entry": false,
"github_name": null,
"prs": 7,
"commits": 7,
"lines_added": 4069,
"lines_deleted": 240,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-06T21:04:48Z",
"last_commit": "2026-10-08T16:17:58Z"
},
{
"login": "mweinbach",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 3230,
"lines_deleted": 21,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-06T21:24:34Z",
"last_commit": "2026-10-06T21:24:34Z"
},
{
"login": "HEOJUNFO",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 122,
"lines_deleted": 8,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T01:44:38Z",
"last_commit": "2026-10-07T01:44:38Z"
},
{
"login": "Luskish",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 255,
"lines_deleted": 32,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T02:32:13Z",
"last_commit": "2026-10-07T02:32:13Z"
},
{
"login": "coygeek",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 4,
"lines_deleted": 0,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T02:33:01Z",
"last_commit": "2026-10-07T02:33:01Z"
},
{
"login": "carlitosteso3",
"people_entry": false,
"github_name": null,
"prs": 15,
"commits": 15,
"lines_added": 1740,
"lines_deleted": 207,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T03:06:02Z",
"last_commit": "2026-10-09T14:21:09Z"
},
{
"login": "SaymonMaster14",
"people_entry": false,
"github_name": null,
"prs": 8,
"commits": 14,
"lines_added": 10039,
"lines_deleted": 651,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T06:59:57Z",
"last_commit": "2026-10-09T07:32:43Z"
},
{
"login": "Luna161",
"people_entry": false,
"github_name": null,
"prs": 3,
"commits": 3,
"lines_added": 195,
"lines_deleted": 9,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T18:20:23Z",
"last_commit": "2026-10-08T00:35:58Z"
},
{
"login": "RicSchonfelder",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 402,
"lines_deleted": 4,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T18:26:26Z",
"last_commit": "2026-10-07T18:26:26Z"
},
{
"login": "ajsongpilot",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 6,
"lines_deleted": 1,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T20:35:41Z",
"last_commit": "2026-10-07T20:35:41Z"
},
{
"login": "cirocueto2013-wq",
"people_entry": false,
"github_name": null,
"prs": 3,
"commits": 3,
"lines_added": 1127,
"lines_deleted": 226,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T22:56:58Z",
"last_commit": "2026-10-07T23:50:33Z"
},
{
"login": "DWAA1660",
"people_entry": false,
"github_name": null,
"prs": 2,
"commits": 2,
"lines_added": 209,
"lines_deleted": 0,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T22:57:38Z",
"last_commit": "2026-10-08T23:52:30Z"
},
{
"login": "txiki739",
"people_entry": false,
"github_name": null,
"prs": 5,
"commits": 8,
"lines_added": 7976,
"lines_deleted": 3333,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T23:41:48Z",
"last_commit": "2026-10-09T09:28:24Z"
},
{
"login": "wstock",
"people_entry": false,
"github_name": null,
"prs": 5,
"commits": 5,
"lines_added": 288,
"lines_deleted": 24,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-08T07:45:37Z",
"last_commit": "2026-10-08T12:14:46Z"
},
{
"login": "WavJaby",
"people_entry": false,
"github_name": null,
"prs": 6,
"commits": 6,
"lines_added": 1090,
"lines_deleted": 110,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-08T08:18:07Z",
"last_commit": "2026-10-09T05:47:06Z"
},
{
"login": "MrCloudy2",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 132,
"lines_deleted": 4,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-08T08:53:54Z",
"last_commit": "2026-10-08T08:53:54Z"
},
{
"login": "switch87",
"people_entry": true,
"github_name": null,
"prs": 2,
"commits": 3,
"lines_added": 629,
"lines_deleted": 9,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-08T09:25:54Z",
"last_commit": "2026-10-08T09:37:02Z"
},
{
"login": "h-a-n-a-s-h-i",
"people_entry": false,
"github_name": null,
"prs": 0,
"commits": 2,
"lines_added": 96,
"lines_deleted": 96,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-08T12:12:49Z",
"last_commit": "2026-10-08T14:01:39Z"
},
{
"login": "XusBadia",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 260,
"lines_deleted": 1,
"binary_added": 2,
"binary_deleted": 0,
"first_commit": "2026-10-08T13:12:32Z",
"last_commit": "2026-10-08T13:12:32Z"
},
{
"login": "MarekSpodBiedry",
"people_entry": false,
"github_name": null,
"prs": 2,
"commits": 2,
"lines_added": 74,
"lines_deleted": 6,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-08T13:44:56Z",
"last_commit": "2026-10-09T08:45:28Z"
},
{
"login": "Tom1tk",
"people_entry": false,
"github_name": null,
"prs": 2,
"commits": 2,
"lines_added": 47,
"lines_deleted": 26,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-08T13:47:40Z",
"last_commit": "2026-10-08T14:56:39Z"
},
{
"login": "Pathos0925",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 68,
"lines_deleted": 8,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-08T17:09:15Z",
"last_commit": "2026-10-08T17:09:15Z"
},
{
"login": "mjohnson7cd",
"people_entry": false,
"github_name": null,
"prs": 2,
"commits": 1,
"lines_added": 100,
"lines_deleted": 2,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-08T20:36:28Z",
"last_commit": "2026-10-08T20:36:28Z"
},
{
"login": "Oldrich333",
"people_entry": false,
"github_name": null,
"prs": 0,
"commits": 1,
"lines_added": 119,
"lines_deleted": 9,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-08T20:58:16Z",
"last_commit": "2026-10-08T20:58:16Z"
},
{
"login": "cshapiro",
"people_entry": false,
"github_name": null,
"prs": 3,
"commits": 5,
"lines_added": 3128,
"lines_deleted": 111,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-08T23:28:32Z",
"last_commit": "2026-10-09T02:57:30Z"
},
{
"login": "rossaai",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 3,
"lines_added": 112,
"lines_deleted": 10,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-09T00:57:43Z",
"last_commit": "2026-10-09T00:57:43Z"
},
{
"login": "wisegamesyt-droid",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 2,
"lines_added": 758,
"lines_deleted": 275,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-09T01:11:12Z",
"last_commit": "2026-10-09T02:33:35Z"
},
{
"login": "JordyUSA",
"people_entry": false,
"github_name": null,
"prs": 2,
"commits": 3,
"lines_added": 4158,
"lines_deleted": 283,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-09T01:35:57Z",
"last_commit": "2026-10-09T02:53:28Z"
},
{
"login": "jerrykimo",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 117,
"lines_deleted": 6,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-09T01:50:09Z",
"last_commit": "2026-10-09T01:50:09Z"
},
{
"login": "swillunderscore",
"people_entry": true,
"github_name": null,
"prs": 1,
"commits": 3,
"lines_added": 1008,
"lines_deleted": 13,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-09T01:58:37Z",
"last_commit": "2026-10-09T02:19:21Z"
},
{
"login": "IppSec",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 2,
"lines_added": 486,
"lines_deleted": 54,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-09T02:29:23Z",
"last_commit": "2026-10-09T02:30:49Z"
},
{
"login": "sanyamsingh001",
"people_entry": true,
"github_name": null,
"prs": 2,
"commits": 2,
"lines_added": 323,
"lines_deleted": 25,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-09T04:06:37Z",
"last_commit": "2026-10-09T05:44:39Z"
},
{
"login": "ItzBloxy",
"people_entry": true,
"github_name": null,
"prs": 1,
"commits": 4,
"lines_added": 887,
"lines_deleted": 167,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-09T09:27:48Z",
"last_commit": "2026-10-09T10:17:08Z"
}
],
"models": [
{
"id": "anthropic/claude-opus/5.5",
"company": "Anthropic",
"model": "Claude Opus",
"version": "5.5",
"commits": 474,
"lines_added": 346706,
"lines_deleted": 19057
},
{
"id": "anthropic/claude-sonnet/5.5",
"company": "Anthropic",
"model": "Claude Sonnet",
"version": "5.5",
"commits": 18,
"lines_added": 13935,
"lines_deleted": 854
},
{
"id": "openai/codex",
"company": "OpenAI",
"model": "Codex",
"version": "",
"commits": 11,
"lines_added": 2057,
"lines_deleted": 361
},
{
"id": "anthropic/claude-fable/5.1",
"company": "Anthropic",
"model": "Claude Fable",
"version": "5.1",
"commits": 5,
"lines_added": 322,
"lines_deleted": 32
},
{
"id": "anthropic/claude",
"company": "Anthropic",
"model": "Claude",
"version": "",
"commits": 1,
"lines_added": 235,
"lines_deleted": 22
},
{
"id": "anthropic/claude-opus/4.8",
"company": "Anthropic",
"model": "Claude Opus",
"version": "4.8",
"commits": 1,
"lines_added": 1632,
"lines_deleted": 3
}
]
}
+42
View File
@@ -0,0 +1,42 @@
# Contributor names for the About window (craftrules standards/contributors.md).
#
# Everyone who contributes is credited by GitHub username automatically. Anything more is up to
# you: add YOUR OWN entry below, in your own PR (commit it yourself). Never add someone else; a
# maintainer may add a person only when that person asked, recording where in consent_source.
#
# Easiest: python3 ../../craftrules/scripts/contributors.py --add-me . \
# [--real-name "Ada Lovelace"] [--display-name "Ada"] [--sync-github-name]
#
# Or copy this block, replacing your-github-username (every field is optional):
#
# [people.your-github-username]
# real_name = "Your Name" # shown when the viewer picks "Real name"
# display_name = "Nickname" # shown when the viewer picks "Display name"
# sync_github_name = true # or: use the name on your public GitHub profile as display name
#
# To remove your names, delete your entry (or set fields empty) in a PR. The next
# contributors.py run, and the next build, drop them.
[people.echelon]
real_name = "Brandon Thomas"
display_name = "Brandon Thomas"
sync_github_name = false
[people.sanyamsingh001]
display_name = "sanyamsingh001"
sync_github_name = false
[people.switch87]
real_name = "Gert Pellin"
display_name = "Gert Pellin"
sync_github_name = false
[people.swillunderscore]
display_name = "Swill Software"
[people.IppSec]
sync_github_name = true
[people.ItzBloxy]
display_name = "Bloxy"
+42 -2
View File
@@ -19,6 +19,10 @@ const ID_PCE: u32 = 5;
const ID_FIL: u32 = 6;
const ID_END: u32 = 7;
/// `extension_type` of a fill element carrying SBR data (ISO/IEC 14496-3 §4.5.2.8.2, Table 4.121).
const EXT_SBR_DATA: u32 = 13;
const EXT_SBR_DATA_CRC: u32 = 14;
/// One decoded `individual_channel_stream` before stereo processing and synthesis.
struct Ics {
info: IcsInfo,
@@ -45,6 +49,8 @@ pub struct Decoder {
offsets: Vec<usize>,
channels: Vec<ChannelState>,
noise_state: u32,
/// A fill element carried SBR data (implicitly signalled HE-AAC).
sbr_seen: bool,
}
impl Decoder {
@@ -67,7 +73,7 @@ impl Decoder {
if layout.is_empty() && asc.channel_config != 0 {
return Err(Error::InvalidConfig("unsupported channel configuration"));
}
let mut d = Decoder { asc, layout: Vec::new(), offsets: Vec::new(), channels: Vec::new(), noise_state: 0x2545_F491 };
let mut d = Decoder { asc, layout: Vec::new(), offsets: Vec::new(), channels: Vec::new(), noise_state: 0x2545_F491, sbr_seen: false };
d.set_layout(layout);
Ok(d)
}
@@ -90,6 +96,12 @@ impl Decoder {
pub fn sample_rate(&self) -> u32 {
self.asc.sample_rate
}
/// Whether the stream is HE-AAC: SBR signalled in the `AudioSpecificConfig` (explicit) or SBR
/// data seen in a fill element of a decoded access unit (implicit). Only the AAC-LC core is
/// decoded, at [`Self::sample_rate`]; the SBR high band is not reconstructed.
pub fn sbr(&self) -> bool {
self.asc.sbr || self.sbr_seen
}
pub fn channels(&self) -> usize {
self.channels.len()
}
@@ -174,7 +186,11 @@ impl Decoder {
count += br.read_bits(8)? as usize;
count -= 1;
}
br.skip(count * 8)?;
if count > 0 {
let ext = br.read_bits(4)?;
self.sbr_seen |= matches!(ext, EXT_SBR_DATA | EXT_SBR_DATA_CRC);
br.skip(count * 8 - 4)?;
}
}
_ => break, // ID_END
}
@@ -542,3 +558,27 @@ fn synth(state: &mut ChannelState, info: &IcsInfo, spec: &[f32]) -> Vec<f32> {
state.prev_shape = cur;
out
}
#[cfg(test)]
mod tests {
use super::*;
/// Implicitly signalled HE-AAC (an AAC-LC config) is recognised from the SBR fill element.
#[test]
fn sbr_fill_element_marks_implicit_he_aac() {
// AAC-LC, 22.05 kHz, stereo
let mut dec = Decoder::new(&[0x13, 0x90]).unwrap();
assert!(!dec.sbr());
// ID_FIL, count 1, extension_type EXT_SBR_DATA (+ 4 bits of payload), ID_END
let _ = dec.decode(&[0b1100_0011, 0b1010_0001, 0b1100_0000]);
assert!(dec.sbr());
// other fill payloads (EXT_FILL) don't
let mut dec = Decoder::new(&[0x13, 0x90]).unwrap();
let _ = dec.decode(&[0b1100_0010, 0b0000_0001, 0b1100_0000]);
assert!(!dec.sbr());
// explicit signalling: AOT 5, core 22.05 kHz, stereo, extension 44.1 kHz, core AOT 2
let dec = Decoder::new(&[0x2B, 0x92, 0x08, 0x00]).unwrap();
assert!(dec.sbr());
assert_eq!(dec.sample_rate(), 22_050);
}
}
+21
View File
@@ -0,0 +1,21 @@
[package]
name = "filmcraft-apv"
description = "Clean-room Advanced Professional Video (APV, IETF RFC 9924) decoder and encoder"
version.workspace = true
edition.workspace = true
license.workspace = true
[features]
default = ["threads"]
threads = ["dep:rayon"]
[dependencies]
filmcraft-bitstream.workspace = true
rayon = { workspace = true, optional = true }
thiserror.workspace = true
[dev-dependencies]
filmcraft-testkit.workspace = true
[lints]
workspace = true
+74
View File
@@ -0,0 +1,74 @@
# filmcraft-apv
Clean-room Advanced Professional Video (APV) decoder and encoder in pure Rust, implemented strictly
from the public **IETF RFC 9924** specification (*Advanced Professional Video*, February 2026,
ISSN 2070-1721).
Layer L0: depends only on `filmcraft-bitstream`, `thiserror`, and (optional, feature `threads`,
default on) `rayon`. Builds for `wasm32-unknown-unknown`; `unsafe_code = "forbid"`.
## API
```rust
use filmcraft_apv::*;
// Decode one APV Access Unit (or raw bitstream AU prefixed with a 32-bit au_size).
let frame: Frame = decode_frame(&au)?;
let f8 = decode_frame_with(&au, &DecodeOptions { bit_depth: Some(8), threads: true })?;
decode_frame_into(&au, &DecodeOptions::default(), &mut frame_buf)?;
let hdr: FrameHeader = probe(&au)?;
// Split a multi-AU raw .apv bitstream (RFC 9924 Appendix A).
let aus: Vec<&[u8]> = split_raw_bitstream(&raw_apv_bytes)?;
// Encode progressive intra frames.
let mut enc = Encoder::new(Profile::P422_10, 1920, 1080)?;
let au: Vec<u8> = enc.encode(&frame)?; // Access Unit starting with 'aPv1'
let raw_au: Vec<u8> = enc.encode_raw_au(&frame)?; // Prefixed with 32-bit au_size (ISOBMFF apv1 / .apv)
let apvc: Vec<u8> = enc.decoder_config_record(); // APVDecoderConfigurationRecord payload
```
## Specification Coverage (IETF RFC 9924)
### Decoder
- **Access Units & PBUs (§5.3.1–5.3.3, Appendix A)**: Accepts both unprefixed Access Units
(`'aPv1'` signature) and length-prefixed `raw_bitstream_access_unit()` packets. Parses primary
(`pbu_type == 1`), non-primary (`2`), preview (`25`), auxiliary alpha (`27`), AU info (`65`),
metadata (`66`), and filler (`67`) PBUs. PBUs or frame headers with non-zero `reserved_zero_*`
fields are ignored per §5.3.3 and §5.3.5.
- **Profiles & Sampling (§9.3)**: All 7 standard profiles (`422-10` idc 33, `422-12` idc 44,
`444-10` idc 55, `444-12` idc 66, `4444-10` idc 77, `4444-12` idc 88, `400-10` idc 99) at
coded bit depths 10–16, with optional output depth rescaling (`8..=16`).
- **Quantization Matrices & Tiles (§5.3.7–5.3.8, §5.3.12–5.3.14)**: Default flat (`16`) and
custom per-component 8×8 quantization matrices (`use_q_matrix`), optional `tile_size_in_fh`
verification, and independent per-tile, per-component byte-aligned entropy partitions. Tiles
decode in parallel via `rayon` when `threads` is enabled.
- **Entropy Decoding (§5.3.15–5.3.16, §7.1)**: Adaptive Golomb-Rice / Exp-Golomb VLC decoding for
`abs_dc_coeff_diff`, `coeff_zero_run`, and `abs_ac_coeff_minus1` with state tracking (`PrevDC`,
`PrevDcDiff`, `Prev1stAcLevel`, `PrevLevel`, `PrevRun`) across the 8×8 blocks of each macroblock.
- **Scaling & Inverse Transform (§6.3)**: Exact integer dequantization (`levelScale = [40, 45, 51,
57, 64, 71]`, `bdShift = BitDepth - 2`) and 2-D separable 8×8 integer inverse transform using the
RFC 9924 Figure 25 `transMatrix`. Note: in RFC 9924 §6.3.2.2, `transMatrix[m][n]` has row `m` as
the frequency index (row 0 is `[64; 8]`) and column `n` as the spatial sample index, so the 1-D
inverse transform evaluates `sum_j transMatrix[j][i] * x[j]`.
- **Metadata (§8)**: Parses `metadata_mdcv` (`payloadType == 5`, Mastering Display Color Volume)
and `metadata_cll` (`payloadType == 6`, Content Light-Level Information).
- **Robustness (§10)**: All size fields, tile coordinates, VLC codeword lengths, and coefficient
ranges (`-32768..=32767`) are checked before allocation or shift. Verified by `tests/fuzz.rs`
across thousands of mutated and random inputs.
### Encoder
- Progressive intra encoding for all 7 profiles (`422-10`, `422-12`, `444-10`, `444-12`,
`4444-10`, `4444-12`, `400-10`).
- Supports fixed `qp` or binary-search frame-budget rate control over forward-transformed tiles.
- Emits conforming tile grids (`tile_width_in_mbs >= 16`, `tile_height_in_mbs >= 8`,
`TileCols, TileRows <= 20`), optional custom quantization matrices, optional `MDCV`/`CLL`
metadata PBUs, and `APVDecoderConfigurationRecord` (`apvC`) metadata.
## Accuracy (vs FFmpeg as an External Oracle)
`tests/oracle.rs` encodes synthetic test patterns across all 7 profiles, odd dimensions, multiple
tiles, custom asymmetric quantization matrices, and HDR metadata PBUs, then decodes the resulting
`.apv` streams with `ffmpeg -v error -xerror -f apv` and compares against `filmcraft_apv::decode_frame`.
Because APV specifies an exact integer inverse transform, **`filmcraft-apv` is 100% bit-exact
(`max_diff = 0`) with FFmpeg's APV decoder across every profile and test case.**
+217
View File
@@ -0,0 +1,217 @@
//! RFC 9924 §6.3 scaling and 8x8 integer inverse transform, plus the matching forward transform.
use crate::tables::LEVEL_SCALE;
/// 1D 8-point inverse transform using the RFC 9924 §6.3.2.3 even/odd butterfly decomposition,
/// shifting each output by `(sum + round) >> shift`.
#[inline(always)]
fn idct_1d(x0: i32, x1: i32, x2: i32, x3: i32, x4: i32, x5: i32, x6: i32, x7: i32, round: i32, shift: u32) -> [i32; 8] {
if (x1 | x2 | x3 | x4 | x5 | x6 | x7) == 0 {
let v = (x0 * 64 + round) >> shift;
return [v; 8];
}
let e0 = 64 * (x0 + x4);
let e1 = 64 * (x0 - x4);
let e2 = 35 * x2 - 84 * x6;
let e3 = 84 * x2 + 35 * x6;
let even0 = e0 + e3;
let even1 = e1 + e2;
let even2 = e1 - e2;
let even3 = e0 - e3;
let odd0 = 89 * x1 + 75 * x3 + 50 * x5 + 18 * x7;
let odd1 = 75 * x1 - 18 * x3 - 89 * x5 - 50 * x7;
let odd2 = 50 * x1 - 89 * x3 + 18 * x5 + 75 * x7;
let odd3 = 18 * x1 - 50 * x3 + 75 * x5 - 89 * x7;
[
(even0 + odd0 + round) >> shift,
(even1 + odd1 + round) >> shift,
(even2 + odd2 + round) >> shift,
(even3 + odd3 + round) >> shift,
(even3 - odd3 + round) >> shift,
(even2 - odd2 + round) >> shift,
(even1 - odd1 + round) >> shift,
(even0 - odd0 + round) >> shift,
]
}
/// Fast path when an 8x8 block has only a DC coefficient (`coeff[1..64]` are all zero).
#[inline]
pub fn scale_and_idct8x8_dc_only(dc_coeff: i16, q0: u8, qp: u8, bit_depth: u8, out: &mut [u16; 64]) {
let bd_shift_scale = (bit_depth as u32).saturating_sub(2);
let scale_round = if bd_shift_scale > 0 { 1i64 << (bd_shift_scale - 1) } else { 0 };
let level_scale = LEVEL_SCALE[(qp % 6) as usize] as i64;
let qp_shift = (qp / 6) as u32;
let scaled = ((dc_coeff as i64 * q0 as i64 * level_scale) << qp_shift) + scale_round;
let d0 = (scaled >> bd_shift_scale).clamp(-32768, 32767) as i32;
let g0 = (d0 * 64 + 64) >> 7;
let bd_shift = 20u32.saturating_sub(bit_depth as u32);
let round = 1i32 << (bd_shift - 1);
let mid = 1i32 << (bit_depth - 1);
let max_val = (1i32 << bit_depth) - 1;
let sample = (((g0 * 64 + round) >> bd_shift) + mid).clamp(0, max_val) as u16;
out.fill(sample);
}
/// Scale and inverse-transform one 8x8 block of transform coefficients (indexed by `y * 8 + x`)
/// into reconstructed samples at `bit_depth` (`10..=16`), per RFC 9924 §6.3.
#[inline]
pub fn scale_and_idct8x8(coeff: &[i16; 64], q_matrix: &[u8; 64], qp: u8, bit_depth: u8, out: &mut [u16; 64]) {
let bd_shift_scale = (bit_depth as u32).saturating_sub(2);
let scale_round = if bd_shift_scale > 0 { 1i64 << (bd_shift_scale - 1) } else { 0 };
let level_scale = LEVEL_SCALE[(qp % 6) as usize] as i64;
let qp_shift = (qp / 6) as u32;
let mut d = [0i32; 64];
for idx in 0..64 {
let c = coeff[idx] as i64;
if c != 0 {
let qm = q_matrix[idx] as i64;
let scaled = ((c * qm * level_scale) << qp_shift) + scale_round;
d[idx] = (scaled >> bd_shift_scale).clamp(-32768, 32767) as i32;
}
}
// 1D vertical transform on each column x = 0..8, followed by (e + 64) >> 7
let mut g = [0i32; 64];
for x in 0..8 {
let col = idct_1d(d[x], d[8 + x], d[16 + x], d[24 + x], d[32 + x], d[40 + x], d[48 + x], d[56 + x], 64, 7);
for i in 0..8 {
g[i * 8 + x] = col[i];
}
}
// 1D horizontal transform on each row y = 0..8, followed by final bit-depth shift and offset
let bd_shift = 20u32.saturating_sub(bit_depth as u32);
let round = 1i32 << (bd_shift - 1);
let mid = 1i32 << (bit_depth - 1);
let max_val = (1i32 << bit_depth) - 1;
for y in 0..8 {
let r = &g[y * 8..y * 8 + 8];
let row = idct_1d(r[0], r[1], r[2], r[3], r[4], r[5], r[6], r[7], round, bd_shift);
for i in 0..8 {
out[y * 8 + i] = (row[i] + mid).clamp(0, max_val) as u16;
}
}
}
/// 1D 8-point forward butterfly transform matching `TRANS_MATRIX`.
#[inline(always)]
fn fdct_1d(x0: i32, x1: i32, x2: i32, x3: i32, x4: i32, x5: i32, x6: i32, x7: i32, round: i32, shift: u32) -> [i32; 8] {
let s0 = x0 + x7;
let d0 = x0 - x7;
let s1 = x1 + x6;
let d1 = x1 - x6;
let s2 = x2 + x5;
let d2 = x2 - x5;
let s3 = x3 + x4;
let d3 = x3 - x4;
let es0 = s0 + s3;
let ed0 = s0 - s3;
let es1 = s1 + s2;
let ed1 = s1 - s2;
[
(64 * (es0 + es1) + round) >> shift,
(89 * d0 + 75 * d1 + 50 * d2 + 18 * d3 + round) >> shift,
(84 * ed0 + 35 * ed1 + round) >> shift,
(75 * d0 - 18 * d1 - 89 * d2 - 50 * d3 + round) >> shift,
(64 * (es0 - es1) + round) >> shift,
(50 * d0 - 89 * d1 + 18 * d2 + 75 * d3 + round) >> shift,
(35 * ed0 - 84 * ed1 + round) >> shift,
(18 * d0 - 50 * d1 + 75 * d2 - 89 * d3 + round) >> shift,
]
}
/// Forward 8x8 transform of level-shifted samples (`sample - (1 << (bit_depth - 1))`) into the
/// scaled coefficient domain `d[y * 8 + x]` matching [`scale_and_idct8x8`].
#[inline]
pub fn fdct8x8(samples: &[u16; 64], bit_depth: u8) -> [i32; 64] {
let mid = 1i32 << (bit_depth - 1);
let shift1 = (bit_depth as u32).saturating_sub(5);
let add1 = if shift1 > 0 { 1i32 << (shift1 - 1) } else { 0 };
// Horizontal forward pass
let mut t = [0i32; 64];
for y in 0..8 {
let s = &samples[y * 8..y * 8 + 8];
let row = fdct_1d(
s[0] as i32 - mid,
s[1] as i32 - mid,
s[2] as i32 - mid,
s[3] as i32 - mid,
s[4] as i32 - mid,
s[5] as i32 - mid,
s[6] as i32 - mid,
s[7] as i32 - mid,
add1,
shift1,
);
t[y * 8..y * 8 + 8].copy_from_slice(&row);
}
// Vertical forward pass
let mut f = [0i32; 64];
for u in 0..8 {
let col = fdct_1d(t[u], t[8 + u], t[16 + u], t[24 + u], t[32 + u], t[40 + u], t[48 + u], t[56 + u], 128, 8);
for v in 0..8 {
f[v * 8 + u] = col[v];
}
}
f
}
/// Quantize forward-transformed coefficients `f` (in the `d` domain) at `qp` with `q_matrix`.
#[inline]
pub fn quantize8x8(f: &[i32; 64], q_matrix: &[u8; 64], qp: u8, bit_depth: u8) -> [i16; 64] {
let bd_shift_scale = (bit_depth as u32).saturating_sub(2);
let level_scale = LEVEL_SCALE[(qp % 6) as usize] as i64;
let qp_shift = (qp / 6) as u32;
let mut out = [0i16; 64];
for idx in 0..64 {
let val = f[idx] as i64;
if val == 0 {
continue;
}
let sign = if val < 0 { -1i64 } else { 1i64 };
let abs_val = val.unsigned_abs() as i64;
let qm = q_matrix[idx].max(1) as i64;
let den = (qm * level_scale) << qp_shift;
let num = abs_val << bd_shift_scale;
// Deadzone/rounding offset: 1/2 for DC, 3/8 for AC
let offset = if idx == 0 { den / 2 } else { (den * 3) / 8 };
let level = ((num + offset) / den).min(32767);
out[idx] = (level * sign) as i16;
}
out
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn fdct_idct_roundtrip_10bit() {
let mut src = [0u16; 64];
for (i, s) in src.iter_mut().enumerate() {
*s = (64 + (i * 13) % 850) as u16;
}
let qm = [16u8; 64];
// At qp = 0 (very fine quantization), roundtrip is within 1 LSB
let f = fdct8x8(&src, 10);
let q = quantize8x8(&f, &qm, 0, 10);
let mut rec = [0u16; 64];
scale_and_idct8x8(&q, &qm, 0, 10, &mut rec);
for i in 0..64 {
let err = (src[i] as i32 - rec[i] as i32).abs();
assert!(err <= 1, "idx {i}: src={} rec={} err={err}", src[i], rec[i]);
}
}
}
+484
View File
@@ -0,0 +1,484 @@
//! RFC 9924 APV frame and tile decoder.
use filmcraft_bitstream::BitReader;
use crate::dct::{scale_and_idct8x8, scale_and_idct8x8_dc_only};
use crate::header::{
FrameHeader, PBU_ALPHA_FRAME, PBU_METADATA, PBU_NON_PRIMARY_FRAME, PBU_PREVIEW_FRAME, PBU_PRIMARY_FRAME, parse_au_pbus, parse_frame_header,
parse_metadata_pbu, read_byte_alignment,
};
use crate::tables::ZIGZAG_8X8;
use crate::{ChromaFormat, ContentLightLevel, Error, Frame, MasteringDisplay, Result};
/// Options for [`decode_frame_with`] and [`decode_frame_into`].
#[derive(Debug, Clone)]
pub struct DecodeOptions {
/// Output sample bit depth (`8..=16`). `None` keeps the stream's coded bit depth (`10..=16`).
pub bit_depth: Option<u8>,
/// Decode tiles in parallel when the `threads` feature is enabled.
pub threads: bool,
}
impl Default for DecodeOptions {
fn default() -> Self {
Self { bit_depth: None, threads: true }
}
}
/// Parse RFC 9924 §7.1.4 variable-length codeword (`h(v)`).
#[inline]
pub(crate) fn read_vlc(r: &mut BitReader, k_param: u32) -> Result<u32> {
let mut symbol_value: u64 = 0;
let mut k = k_param;
let parse_exp_golomb = if r.read_bit().map_err(|_| Error::Truncated)? {
false
} else if !r.read_bit().map_err(|_| Error::Truncated)? {
symbol_value += 1u64 << k;
false
} else {
symbol_value += 2u64 << k;
true
};
if parse_exp_golomb {
loop {
if r.read_bit().map_err(|_| Error::Truncated)? {
break;
}
symbol_value += 1u64 << k;
k += 1;
if k > 28 {
return Err(Error::Invalid("VLC codeword exceeds maximum length"));
}
}
}
if k > 0 {
symbol_value += r.read_bits(k).map_err(|_| Error::Truncated)? as u64;
}
u32::try_from(symbol_value).map_err(|_| Error::Invalid("VLC symbol overflow"))
}
/// Probe the primary frame header of an Access Unit without decoding its tiles.
pub fn probe(data: &[u8]) -> Result<FrameHeader> {
let pbus = parse_au_pbus(data)?;
for pbu in &pbus {
if matches!(pbu.pbu_type, PBU_PRIMARY_FRAME | PBU_NON_PRIMARY_FRAME | PBU_PREVIEW_FRAME)
&& let Some(hdr) = parse_frame_header(pbu.payload)?
{
return Ok(hdr);
}
}
Err(Error::Invalid("no decodable frame PBU in access unit"))
}
/// Decode one APV Access Unit with default options.
pub fn decode_frame(data: &[u8]) -> Result<Frame> {
decode_frame_with(data, &DecodeOptions::default())
}
/// Decode one APV Access Unit with custom [`DecodeOptions`].
pub fn decode_frame_with(data: &[u8], opts: &DecodeOptions) -> Result<Frame> {
let mut out = Frame::new(2, 2, ChromaFormat::Yuv422, 10, false);
decode_frame_into(data, opts, &mut out)?;
Ok(out)
}
/// Decode one APV Access Unit into `out`, reusing its plane allocations when possible.
pub fn decode_frame_into(data: &[u8], opts: &DecodeOptions, out: &mut Frame) -> Result<FrameHeader> {
if let Some(d) = opts.bit_depth
&& !(8..=16).contains(&d)
{
return Err(Error::Invalid("requested output bit_depth must be in 8..=16"));
}
let pbus = parse_au_pbus(data)?;
let mut primary: Option<(u16, FrameHeader, &[u8])> = None;
let mut alpha_pbu: Option<(FrameHeader, &[u8])> = None;
let mut mdcv: Option<MasteringDisplay> = None;
let mut cll: Option<ContentLightLevel> = None;
for pbu in &pbus {
match pbu.pbu_type {
PBU_PRIMARY_FRAME => {
if primary.is_none()
&& let Some(hdr) = parse_frame_header(pbu.payload)?
{
primary = Some((pbu.group_id, hdr, pbu.payload));
}
}
PBU_NON_PRIMARY_FRAME | PBU_PREVIEW_FRAME => {
if primary.is_none()
&& let Some(hdr) = parse_frame_header(pbu.payload)?
{
primary = Some((pbu.group_id, hdr, pbu.payload));
}
}
PBU_ALPHA_FRAME => {
if alpha_pbu.is_none()
&& let Some(hdr) = parse_frame_header(pbu.payload)?
{
alpha_pbu = Some((hdr, pbu.payload));
}
}
PBU_METADATA => {
let _ = parse_metadata_pbu(pbu.payload, &mut mdcv, &mut cll);
}
_ => {}
}
}
let (_group_id, hdr, payload) = primary.ok_or(Error::Invalid("no primary frame PBU in access unit"))?;
let out_depth = opts.bit_depth.unwrap_or(hdr.bit_depth);
decode_pbu_planes(&hdr, payload, out_depth, opts.threads, out)?;
out.mastering_display = mdcv;
out.content_light = cll;
// If the primary frame has no 4th component and an auxiliary alpha frame PBU (type 27) is present
// with matching dimensions, decode its luma plane as the alpha plane.
if out.alpha.is_none()
&& let Some((ahdr, apayload)) = alpha_pbu
&& ahdr.width == hdr.width
&& ahdr.height == hdr.height
{
let mut aframe = Frame::new(ahdr.width, ahdr.height, ahdr.chroma, out_depth, false);
if decode_pbu_planes(&ahdr, apayload, out_depth, opts.threads, &mut aframe).is_ok() {
out.alpha = Some(aframe.y);
}
}
Ok(hdr)
}
#[derive(Clone)]
struct DecodedTile {
x0: u32,
y0: u32,
w_luma: u32,
h_luma: u32,
/// Decoded pixels for each component `c` in `0..num_comps`, packed with stride `w_luma / sub_w`.
comps: [Vec<u16>; 4],
}
fn decode_pbu_planes(hdr: &FrameHeader, payload: &[u8], out_depth: u8, threads: bool, out: &mut Frame) -> Result<()> {
let num_tiles = hdr.num_tiles();
let mut pos = hdr.header_bytes;
let mut tile_slices: Vec<(usize, &[u8])> = Vec::with_capacity(num_tiles);
for tile_idx in 0..num_tiles {
let sz_bytes = payload.get(pos..pos + 4).ok_or(Error::Truncated)?;
let tile_size = u32::from_be_bytes([sz_bytes[0], sz_bytes[1], sz_bytes[2], sz_bytes[3]]) as usize;
if tile_size == 0 {
return Err(Error::Invalid("tile_size cannot be 0"));
}
if let Some(fh_sizes) = &hdr.tile_sizes_in_fh
&& fh_sizes.get(tile_idx).copied() != Some(tile_size as u32)
{
return Err(Error::Invalid("tile_size does not match tile_size_in_fh"));
}
pos += 4;
let tile_end = pos.checked_add(tile_size).ok_or(Error::Truncated)?;
let tile_bytes = payload.get(pos..tile_end).ok_or(Error::Truncated)?;
pos = tile_end;
tile_slices.push((tile_idx, tile_bytes));
}
// Verify any trailing filler bytes in the frame PBU are 0xFF (RFC 9924 §5.3.4 & §5.3.11)
if payload.get(pos..).is_some_and(|rest| rest.iter().any(|&b| b != 0xFF)) {
return Err(Error::Invalid("non-0xFF filler byte at end of frame PBU"));
}
#[cfg(feature = "threads")]
let decoded_tiles: Result<Vec<DecodedTile>> = if threads && tile_slices.len() > 1 {
use rayon::prelude::*;
tile_slices.into_par_iter().map(|(idx, bytes)| decode_single_tile(hdr, idx, bytes, out_depth, false)).collect()
} else {
tile_slices.into_iter().map(|(idx, bytes)| decode_single_tile(hdr, idx, bytes, out_depth, threads)).collect()
};
#[cfg(not(feature = "threads"))]
let decoded_tiles: Result<Vec<DecodedTile>> = {
let _ = threads;
tile_slices.into_iter().map(|(idx, bytes)| decode_single_tile(hdr, idx, bytes, out_depth, false)).collect()
};
let mut decoded_tiles = decoded_tiles?;
// Prepare output frame metadata
let w = hdr.width as usize;
let h = hdr.height as usize;
let cw = hdr.chroma.chroma_width(hdr.width) as usize;
let ch = hdr.chroma.chroma_height(hdr.height) as usize;
let n_luma = w.checked_mul(h).ok_or(Error::Invalid("frame luma size overflow"))?;
let n_chroma = cw.checked_mul(ch).ok_or(Error::Invalid("frame chroma size overflow"))?;
out.width = hdr.width;
out.height = hdr.height;
out.chroma = hdr.chroma;
out.bit_depth = out_depth;
out.color = hdr.color;
out.profile_idc = hdr.profile_idc;
out.level_idc = hdr.level_idc;
out.band_idc = hdr.band_idc;
// Zero-copy fast path when a single tile covers the exact frame dimensions without macroblock padding.
if decoded_tiles.len() == 1 && decoded_tiles[0].w_luma == hdr.width && decoded_tiles[0].h_luma == hdr.height {
let [y, cb, cr, a] = std::mem::take(&mut decoded_tiles[0].comps);
out.y = y;
if hdr.chroma.num_comps() >= 3 {
out.cb = cb;
out.cr = cr;
} else {
out.cb.clear();
out.cr.clear();
}
out.alpha = if hdr.chroma == ChromaFormat::Yuv4444 { Some(a) } else { None };
return Ok(());
}
out.y.resize(n_luma, 0);
out.cb.resize(n_chroma, 0);
out.cr.resize(n_chroma, 0);
if hdr.chroma == ChromaFormat::Yuv4444 {
let a = out.alpha.get_or_insert_with(Vec::new);
a.resize(n_luma, 0);
} else {
out.alpha = None;
}
// Copy each tile's reconstructed samples into the cropped output planes (RFC 9924 §6)
let sub_w = hdr.chroma.sub_width_c() as usize;
let sub_h = hdr.chroma.sub_height_c() as usize;
for tile in &decoded_tiles {
copy_tile_comp(&tile.comps[0], tile.w_luma as usize, tile.h_luma as usize, tile.x0 as usize, tile.y0 as usize, &mut out.y, w, h);
if hdr.chroma.num_comps() >= 3 {
let tw_c = (tile.w_luma as usize) / sub_w;
let th_c = (tile.h_luma as usize) / sub_h;
let tx_c = (tile.x0 as usize) / sub_w;
let ty_c = (tile.y0 as usize) / sub_h;
copy_tile_comp(&tile.comps[1], tw_c, th_c, tx_c, ty_c, &mut out.cb, cw, ch);
copy_tile_comp(&tile.comps[2], tw_c, th_c, tx_c, ty_c, &mut out.cr, cw, ch);
}
if hdr.chroma == ChromaFormat::Yuv4444
&& let Some(alpha) = out.alpha.as_mut()
{
copy_tile_comp(&tile.comps[3], tile.w_luma as usize, tile.h_luma as usize, tile.x0 as usize, tile.y0 as usize, alpha, w, h);
}
}
Ok(())
}
fn copy_tile_comp(src: &[u16], tw: usize, th: usize, tx: usize, ty: usize, dst: &mut [u16], fw: usize, fh: usize) {
if tx >= fw || ty >= fh {
return;
}
let copy_w = tw.min(fw - tx);
let copy_h = th.min(fh - ty);
for r in 0..copy_h {
let s_off = r * tw;
let d_off = (ty + r) * fw + tx;
dst[d_off..d_off + copy_w].copy_from_slice(&src[s_off..s_off + copy_w]);
}
}
fn decode_single_tile(hdr: &FrameHeader, tile_idx: usize, tile_bytes: &[u8], out_depth: u8, par_comps: bool) -> Result<DecodedTile> {
let num_comps = hdr.chroma.num_comps();
let min_hdr_size = 5 + 5 * num_comps;
if tile_bytes.len() < min_hdr_size {
return Err(Error::Truncated);
}
// tile_header(tileIdx) (RFC 9924 §5.3.13)
let tile_header_size = u16::from_be_bytes([tile_bytes[0], tile_bytes[1]]) as usize;
let tile_index = u16::from_be_bytes([tile_bytes[2], tile_bytes[3]]) as usize;
if tile_index != tile_idx {
return Err(Error::CorruptTile(tile_idx));
}
if tile_header_size < min_hdr_size || tile_header_size > tile_bytes.len() {
return Err(Error::CorruptTile(tile_idx));
}
let mut tile_data_size = [0usize; 4];
for c in 0..num_comps {
let off = 4 + c * 4;
let sz = u32::from_be_bytes([tile_bytes[off], tile_bytes[off + 1], tile_bytes[off + 2], tile_bytes[off + 3]]) as usize;
if sz == 0 {
return Err(Error::CorruptTile(tile_idx));
}
tile_data_size[c] = sz;
}
let max_tile_qp = (51 + hdr.qp_bd_offset()) as u8;
let mut tile_qp = [0u8; 4];
for c in 0..num_comps {
let qp = tile_bytes[4 + num_comps * 4 + c];
if qp > max_tile_qp {
return Err(Error::CorruptTile(tile_idx));
}
tile_qp[c] = qp;
}
let reserved_zero_8bits = tile_bytes[4 + num_comps * 5];
if reserved_zero_8bits != 0 {
return Err(Error::CorruptTile(tile_idx));
}
let tx = tile_idx % (hdr.tile_cols as usize);
let ty = tile_idx / (hdr.tile_cols as usize);
let x0 = hdr.col_starts[tx];
let x1 = hdr.col_starts[tx + 1];
let y0 = hdr.row_starts[ty];
let y1 = hdr.row_starts[ty + 1];
let w_luma = x1.saturating_sub(x0);
let h_luma = y1.saturating_sub(y0);
let mut comp_off = tile_header_size;
let mut comp_slices: Vec<(usize, &[u8])> = Vec::with_capacity(num_comps);
for c in 0..num_comps {
let comp_end = comp_off.checked_add(tile_data_size[c]).ok_or(Error::CorruptTile(tile_idx))?;
let comp_bytes = tile_bytes.get(comp_off..comp_end).ok_or(Error::CorruptTile(tile_idx))?;
comp_off = comp_end;
comp_slices.push((c, comp_bytes));
}
#[cfg(feature = "threads")]
let decoded_comps: Result<Vec<Vec<u16>>> = if par_comps && comp_slices.len() > 1 {
use rayon::prelude::*;
comp_slices.into_par_iter().map(|(c, comp_bytes)| decode_tile_comp(hdr, tile_idx, c, comp_bytes, tile_qp[c], w_luma, h_luma, out_depth)).collect()
} else {
comp_slices.into_iter().map(|(c, comp_bytes)| decode_tile_comp(hdr, tile_idx, c, comp_bytes, tile_qp[c], w_luma, h_luma, out_depth)).collect()
};
#[cfg(not(feature = "threads"))]
let decoded_comps: Result<Vec<Vec<u16>>> = {
let _ = par_comps;
comp_slices.into_iter().map(|(c, comp_bytes)| decode_tile_comp(hdr, tile_idx, c, comp_bytes, tile_qp[c], w_luma, h_luma, out_depth)).collect()
};
let mut comps: [Vec<u16>; 4] = Default::default();
for (c, buf) in decoded_comps?.into_iter().enumerate() {
comps[c] = buf;
}
Ok(DecodedTile { x0, y0, w_luma, h_luma, comps })
}
#[allow(clippy::too_many_arguments)]
fn decode_tile_comp(hdr: &FrameHeader, tile_idx: usize, c: usize, comp_bytes: &[u8], qp: u8, w_luma: u32, h_luma: u32, out_depth: u8) -> Result<Vec<u16>> {
let num_mb_cols = w_luma / 16;
let num_mb_rows = h_luma / 16;
let sub_w = if c == 0 { 1 } else { hdr.chroma.sub_width_c() };
let sub_h = if c == 0 { 1 } else { hdr.chroma.sub_height_c() };
let tw_c = (w_luma / sub_w) as usize;
let th_c = (h_luma / sub_h) as usize;
let blk_w = (16 / sub_w) as usize;
let blk_h = (16 / sub_h) as usize;
let blocks_x = blk_w / 8;
let blocks_y = blk_h / 8;
let mut comp_buf = vec![0u16; tw_c * th_c];
let mut r = BitReader::new(comp_bytes);
let mut prev_dc: i32 = 0;
let mut prev_dc_diff: u32 = 20;
let mut prev_1st_ac_level: u32 = 0;
let q_mat = &hdr.q_matrix[c];
let coded_depth = hdr.bit_depth;
for mb_y in 0..(num_mb_rows as usize) {
for mb_x in 0..(num_mb_cols as usize) {
for by in 0..blocks_y {
for bx in 0..blocks_x {
// abs_dc_coeff_diff (RFC 9924 §5.3.15 & §7.1.1)
let k_dc = (prev_dc_diff >> 1).min(5);
let abs_dc_diff = read_vlc(&mut r, k_dc).map_err(|_| Error::CorruptTile(tile_idx))?;
let sign_dc = if abs_dc_diff > 0 { r.read_bit().map_err(|_| Error::CorruptTile(tile_idx))? } else { false };
let diff = if sign_dc { -(abs_dc_diff as i64) } else { abs_dc_diff as i64 };
let dc = prev_dc as i64 + diff;
if !(-32768..=32767).contains(&dc) {
return Err(Error::CorruptTile(tile_idx));
}
prev_dc = dc as i32;
prev_dc_diff = abs_dc_diff;
let mut coeff = [0i16; 64];
coeff[0] = dc as i16;
// ac_coeff_coding (RFC 9924 §5.3.16 & §7.1.2–7.1.3)
let mut scan_pos = 1usize;
let mut first_ac = true;
let mut prev_level = prev_1st_ac_level;
let mut prev_run = 0u32;
while scan_pos < 64 {
let k_run = (prev_run >> 2).min(2);
let run = read_vlc(&mut r, k_run).map_err(|_| Error::CorruptTile(tile_idx))? as usize;
scan_pos = scan_pos.checked_add(run).ok_or(Error::CorruptTile(tile_idx))?;
if scan_pos > 64 {
return Err(Error::CorruptTile(tile_idx));
}
prev_run = run as u32;
if scan_pos < 64 {
let k_lvl = (prev_level >> 2).min(4);
let abs_ac_minus1 = read_vlc(&mut r, k_lvl).map_err(|_| Error::CorruptTile(tile_idx))?;
let sign_ac = r.read_bit().map_err(|_| Error::CorruptTile(tile_idx))?;
let abs_level = (abs_ac_minus1 as u64) + 1;
let level_i64 = if sign_ac { -(abs_level as i64) } else { abs_level as i64 };
if !(-32768..=32767).contains(&level_i64) {
return Err(Error::CorruptTile(tile_idx));
}
coeff[ZIGZAG_8X8[scan_pos] as usize] = level_i64 as i16;
scan_pos += 1;
prev_level = abs_level as u32;
if first_ac {
first_ac = false;
prev_1st_ac_level = prev_level;
}
}
}
let mut rec = [0u16; 64];
if first_ac {
scale_and_idct8x8_dc_only(coeff[0], q_mat[0], qp, coded_depth, &mut rec);
} else {
scale_and_idct8x8(&coeff, q_mat, qp, coded_depth, &mut rec);
}
if out_depth != coded_depth {
rescale_block(&mut rec, coded_depth, out_depth);
}
let px0 = mb_x * blk_w + bx * 8;
let py0 = mb_y * blk_h + by * 8;
for ry in 0..8 {
let dst_row = (py0 + ry) * tw_c + px0;
comp_buf[dst_row..dst_row + 8].copy_from_slice(&rec[ry * 8..ry * 8 + 8]);
}
}
}
}
}
read_byte_alignment(&mut r).map_err(|_| Error::CorruptTile(tile_idx))?;
Ok(comp_buf)
}
#[inline]
fn rescale_block(block: &mut [u16; 64], from_depth: u8, to_depth: u8) {
if to_depth > from_depth {
let shift = (to_depth - from_depth) as u32;
let fill_shift = from_depth as u32;
for s in block.iter_mut() {
let v = *s as u32;
*s = ((v << shift) | (v >> (fill_shift - shift))) as u16;
}
} else if to_depth < from_depth {
let shift = (from_depth - to_depth) as u32;
let round = 1u32 << (shift - 1);
let max_v = (1u32 << to_depth) - 1;
for s in block.iter_mut() {
*s = (((*s as u32) + round) >> shift).min(max_v) as u16;
}
}
}
+598
View File
@@ -0,0 +1,598 @@
//! RFC 9924 APV intra frame encoder.
use filmcraft_bitstream::BitWriter;
use crate::dct::{fdct8x8, quantize8x8};
use crate::header::{FrameHeader, MAX_DIMENSION, PBU_PRIMARY_FRAME, write_frame_header, write_metadata_pbu};
use crate::tables::{DEFAULT_Q_MATRIX_VAL, ZIGZAG_8X8, select_level_and_band};
use crate::{ChromaFormat, ColorInfo, ContentLightLevel, Error, Frame, MasteringDisplay, Profile, Result};
/// Generate RFC 9924 §7.2.4 variable-length codeword (`h(v)`).
#[inline]
pub(crate) fn write_vlc(w: &mut BitWriter, symbol_val: u32, k_param: u32) {
const PREFIX_VLC: [[u32; 2]; 3] = [[1, 0], [0, 0], [0, 1]];
let mut symbol_value = symbol_val;
let val_prefix = ((symbol_val >> k_param) as usize).min(2);
let mut bit_count = 0usize;
let mut k = k_param;
while symbol_value >= (1u32 << k) {
symbol_value -= 1u32 << k;
if bit_count < 2 {
w.write_bits(PREFIX_VLC[val_prefix][bit_count], 1);
} else {
w.write_bits(0, 1);
}
if bit_count >= 2 {
k += 1;
}
bit_count += 1;
}
if bit_count < 2 {
w.write_bits(PREFIX_VLC[val_prefix][bit_count], 1);
} else {
w.write_bits(1, 1);
}
if k > 0 {
w.write_bits(symbol_value, k);
}
}
/// Exact bit length of a VLC codeword generated by [`write_vlc`].
#[inline]
pub(crate) fn vlc_bit_len(symbol_val: u32, k_param: u32) -> u32 {
if symbol_val < (1u32 << k_param) {
1 + k_param
} else if symbol_val < (2u32 << k_param) {
2 + k_param
} else {
let rem = (symbol_val >> k_param) - 1;
let zeros = 31 - rem.leading_zeros();
3 + k_param + 2 * zeros
}
}
/// APV encoder configuration.
#[derive(Debug, Clone)]
pub struct EncoderConfig {
pub profile: Profile,
pub width: u32,
pub height: u32,
pub chroma: ChromaFormat,
pub bit_depth: u8,
pub color: ColorInfo,
pub color_description_present: bool,
/// Fixed tile quantization parameter (`0..=51 + QpBdOffset`). When `None`, rate control
/// chooses a uniform QP per frame to fit [`Self::target_frame_bytes`].
pub qp: Option<u8>,
/// Target encoded frame size in bytes when `qp` is `None`.
pub target_frame_bytes: u32,
/// Tile width in 16x16 macroblocks (RFC 9924 §9.4.1 requires `>= 16`).
pub tile_width_in_mbs: u32,
/// Tile height in 16x16 macroblocks (RFC 9924 §9.4.1 requires `>= 8`).
pub tile_height_in_mbs: u32,
pub level_idc: u8,
pub band_idc: u8,
/// Optional custom per-component 8x8 quantization matrices (`y * 8 + x`).
pub q_matrix: Option<[[u8; 64]; 4]>,
/// Write `tile_size_in_fh` array in `frame_header()` (`tile_size_present_in_fh_flag = 1`).
pub write_tile_sizes_in_fh: bool,
pub mastering_display: Option<MasteringDisplay>,
pub content_light: Option<ContentLightLevel>,
}
impl EncoderConfig {
pub fn new(profile: Profile, width: u32, height: u32) -> Self {
let chroma = profile.default_chroma();
let bit_depth = profile.default_bit_depth();
let w_mbs = width.div_ceil(16).max(1);
let h_mbs = height.div_ceil(16).max(1);
// Choose conforming tile dimensions (>= 16x8 MBs, <= 20x20 tiles) with ~4-8 tiles on HD/4K.
let tile_width_in_mbs = w_mbs.div_ceil(4).max(16).max(w_mbs.div_ceil(20));
let tile_height_in_mbs = h_mbs.div_ceil(4).max(8).max(h_mbs.div_ceil(20));
let target_frame_bytes = profile.nominal_frame_bytes(width, height);
let kbps = ((target_frame_bytes as u64) * 8 * 30 / 1000).max(1000) as u32;
let (level_idc, band_idc) = select_level_and_band((width as u64) * (height as u64) * 30, kbps);
Self {
profile,
width,
height,
chroma,
bit_depth,
color: ColorInfo::BT709,
color_description_present: true,
qp: None,
target_frame_bytes,
tile_width_in_mbs,
tile_height_in_mbs,
level_idc,
band_idc,
q_matrix: None,
write_tile_sizes_in_fh: false,
mastering_display: None,
content_light: None,
}
}
}
/// APV intra encoder.
pub struct Encoder {
cfg: EncoderConfig,
hdr: FrameHeader,
}
struct PreparedTileComp {
/// Forward-transformed 8x8 blocks in MB-then-subblock scan order.
blocks: Vec<[i32; 64]>,
}
struct PreparedTile {
tile_idx: usize,
comps: Vec<PreparedTileComp>,
}
impl Encoder {
pub fn new(profile: Profile, width: u32, height: u32) -> Result<Self> {
Self::with_config(EncoderConfig::new(profile, width, height))
}
pub fn with_config(cfg: EncoderConfig) -> Result<Self> {
if cfg.width == 0 || cfg.height == 0 || cfg.width > MAX_DIMENSION || cfg.height > MAX_DIMENSION {
return Err(Error::Input(format!("unsupported dimensions {}x{}", cfg.width, cfg.height)));
}
if cfg.chroma == ChromaFormat::Yuv422 && !cfg.width.is_multiple_of(2) {
return Err(Error::Input("4:2:2 requires even frame width".into()));
}
if !(10..=16).contains(&cfg.bit_depth) {
return Err(Error::Input(format!("unsupported coded bit depth {}", cfg.bit_depth)));
}
if !cfg.profile.supports(cfg.chroma, cfg.bit_depth) {
return Err(Error::Input(format!("profile {} does not support {:?} {}-bit", cfg.profile.name(), cfg.chroma, cfg.bit_depth)));
}
if cfg.tile_width_in_mbs == 0 || cfg.tile_height_in_mbs == 0 {
return Err(Error::Input("tile dimensions in MBs must be non-zero".into()));
}
let w_mbs = cfg.width.div_ceil(16);
let h_mbs = cfg.height.div_ceil(16);
let tile_cols = w_mbs.div_ceil(cfg.tile_width_in_mbs);
let tile_rows = h_mbs.div_ceil(cfg.tile_height_in_mbs);
if tile_cols > 20 || tile_rows > 20 {
return Err(Error::Input(format!("tile grid {tile_cols}x{tile_rows} exceeds RFC 9924 limit of 20x20")));
}
let mut col_starts = Vec::with_capacity(tile_cols as usize + 1);
let mut s = 0u32;
while s < w_mbs {
col_starts.push(s * 16);
s += cfg.tile_width_in_mbs;
}
col_starts.push(w_mbs * 16);
let mut row_starts = Vec::with_capacity(tile_rows as usize + 1);
let mut s = 0u32;
while s < h_mbs {
row_starts.push(s * 16);
s += cfg.tile_height_in_mbs;
}
row_starts.push(h_mbs * 16);
if let Some(qm) = &cfg.q_matrix {
for c in 0..cfg.chroma.num_comps() {
if qm[c].contains(&0) {
return Err(Error::Input("q_matrix entries must be non-zero (1..=255)".into()));
}
}
}
let hdr = FrameHeader {
profile_idc: cfg.profile.idc(),
level_idc: cfg.level_idc,
band_idc: cfg.band_idc.min(3),
width: cfg.width,
height: cfg.height,
chroma: cfg.chroma,
bit_depth: cfg.bit_depth,
capture_time_distance: 0,
color_description_present: cfg.color_description_present,
color: cfg.color,
use_q_matrix: cfg.q_matrix.is_some(),
q_matrix: cfg.q_matrix.unwrap_or([[DEFAULT_Q_MATRIX_VAL; 64]; 4]),
tile_width_in_mbs: cfg.tile_width_in_mbs,
tile_height_in_mbs: cfg.tile_height_in_mbs,
tile_cols,
tile_rows,
col_starts,
row_starts,
tile_sizes_in_fh: None,
header_bytes: 0,
};
Ok(Self { cfg, hdr })
}
pub fn config(&self) -> &EncoderConfig {
&self.cfg
}
pub fn target_frame_bytes(&self) -> u32 {
self.cfg.target_frame_bytes
}
/// Encode one frame into an APV Access Unit (`b"aPv1"...`), suitable for MP4/MOV (`apv1`) or MKV (`V_APV`).
pub fn encode(&mut self, frame: &Frame) -> Result<Vec<u8>> {
if frame.width != self.cfg.width || frame.height != self.cfg.height {
return Err(Error::Input(format!("frame is {}x{}, encoder is {}x{}", frame.width, frame.height, self.cfg.width, self.cfg.height)));
}
if frame.chroma != self.cfg.chroma {
return Err(Error::Input(format!("frame chroma {:?} does not match encoder {:?}", frame.chroma, self.cfg.chroma)));
}
if !(8..=16).contains(&frame.bit_depth) {
return Err(Error::Input(format!("unsupported input bit depth {}", frame.bit_depth)));
}
let n_luma = (frame.width as usize) * (frame.height as usize);
let n_chroma = (frame.chroma_width() as usize) * (frame.chroma_height() as usize);
if frame.y.len() != n_luma || frame.cb.len() != n_chroma || frame.cr.len() != n_chroma {
return Err(Error::Input("plane buffer lengths do not match frame geometry".into()));
}
if self.cfg.chroma == ChromaFormat::Yuv4444 && frame.alpha.as_ref().is_none_or(|a| a.len() != n_luma) {
return Err(Error::Input("4:4:4:4 profile requires an alpha plane of width * height samples".into()));
}
let prepared = self.prepare_tiles(frame);
let max_qp = (51 + self.hdr.qp_bd_offset()) as u8;
let qp = match self.cfg.qp {
Some(q) => q.min(max_qp),
None => self.select_qp(&prepared, max_qp),
};
let encoded_tiles: Vec<Vec<u8>> = {
#[cfg(feature = "threads")]
{
use rayon::prelude::*;
prepared.par_iter().map(|t| self.encode_tile(t, qp)).collect()
}
#[cfg(not(feature = "threads"))]
{
prepared.iter().map(|t| self.encode_tile(t, qp)).collect()
}
};
let mut hdr = self.hdr.clone();
hdr.color = if !self.cfg.color_description_present {
ColorInfo::default()
} else if frame.color != ColorInfo::BT709 {
frame.color
} else {
self.cfg.color
};
if self.cfg.write_tile_sizes_in_fh {
hdr.tile_sizes_in_fh = Some(encoded_tiles.iter().map(|t| t.len() as u32).collect());
}
let fh_bytes = write_frame_header(&hdr);
let tiles_total_bytes: usize = encoded_tiles.iter().map(|t| 4 + t.len()).sum();
let pbu_size = 4 + fh_bytes.len() + tiles_total_bytes;
let mdcv = frame.mastering_display.as_ref().or(self.cfg.mastering_display.as_ref());
let cll = frame.content_light.as_ref().or(self.cfg.content_light.as_ref());
let meta_pbu = write_metadata_pbu(1, mdcv, cll);
let meta_total = meta_pbu.as_ref().map_or(0, |m| 4 + m.len());
let mut au = Vec::with_capacity(4 + 4 + pbu_size + meta_total);
au.extend_from_slice(b"aPv1");
// Primary frame PBU (pbu_type = 1, group_id = 1)
au.extend_from_slice(&(pbu_size as u32).to_be_bytes());
au.push(PBU_PRIMARY_FRAME);
au.extend_from_slice(&1u16.to_be_bytes());
au.push(0); // reserved_zero_8bits
au.extend_from_slice(&fh_bytes);
for t in &encoded_tiles {
au.extend_from_slice(&(t.len() as u32).to_be_bytes());
au.extend_from_slice(t);
}
if let Some(m) = meta_pbu {
au.extend_from_slice(&(m.len() as u32).to_be_bytes());
au.extend_from_slice(&m);
}
Ok(au)
}
/// Encode one frame as a `raw_bitstream_access_unit()` (prefixed by `au_size: u32 BE`, RFC 9924 Appendix A).
pub fn encode_raw_au(&mut self, frame: &Frame) -> Result<Vec<u8>> {
let au = self.encode(frame)?;
let mut raw = Vec::with_capacity(4 + au.len());
raw.extend_from_slice(&(au.len() as u32).to_be_bytes());
raw.extend_from_slice(&au);
Ok(raw)
}
/// Build the `APVDecoderConfigurationRecord` payload (for ISOBMFF `apvC` or Matroska CodecPrivate).
pub fn decoder_config_record(&self) -> Vec<u8> {
let mut v = Vec::with_capacity(24);
v.push(1); // configurationVersion = 1
v.push(1); // number_of_configuration_entry = 1
v.push(PBU_PRIMARY_FRAME); // pbu_type[0] = 1
v.push(1); // number_of_frame_info[0] = 1
let flags = if self.cfg.color_description_present { 0x02 } else { 0x00 } | 0x01; // capture_time_distance_ignored = 1
v.push(flags);
v.push(self.hdr.profile_idc);
v.push(self.hdr.level_idc);
v.push(self.hdr.band_idc);
v.extend_from_slice(&self.hdr.width.to_be_bytes());
v.extend_from_slice(&self.hdr.height.to_be_bytes());
v.push((self.hdr.chroma.idc() << 4) | (self.hdr.bit_depth.saturating_sub(8) & 0x0F));
v.push(0); // capture_time_distance = 0
if self.cfg.color_description_present {
v.push(self.cfg.color.primaries);
v.push(self.cfg.color.transfer);
v.push(self.cfg.color.matrix);
v.push(if self.cfg.color.full_range { 0x80 } else { 0x00 });
}
v
}
fn prepare_tiles(&self, frame: &Frame) -> Vec<PreparedTile> {
let num_tiles = self.hdr.num_tiles();
let prepare_one = |tile_idx: usize| -> PreparedTile {
let tx = tile_idx % (self.hdr.tile_cols as usize);
let ty = tile_idx / (self.hdr.tile_cols as usize);
let x0 = self.hdr.col_starts[tx];
let x1 = self.hdr.col_starts[tx + 1];
let y0 = self.hdr.row_starts[ty];
let y1 = self.hdr.row_starts[ty + 1];
let num_mb_cols = (x1 - x0) / 16;
let num_mb_rows = (y1 - y0) / 16;
let num_comps = self.hdr.chroma.num_comps();
let mut comps = Vec::with_capacity(num_comps);
for c in 0..num_comps {
let (plane, pw, ph, sub_w, sub_h) = match c {
0 => (frame.y.as_slice(), frame.width as usize, frame.height as usize, 1u32, 1u32),
1 => (
frame.cb.as_slice(),
frame.chroma_width() as usize,
frame.chroma_height() as usize,
self.hdr.chroma.sub_width_c(),
self.hdr.chroma.sub_height_c(),
),
2 => (
frame.cr.as_slice(),
frame.chroma_width() as usize,
frame.chroma_height() as usize,
self.hdr.chroma.sub_width_c(),
self.hdr.chroma.sub_height_c(),
),
_ => (frame.alpha.as_deref().unwrap_or(&[]), frame.width as usize, frame.height as usize, 1u32, 1u32),
};
let blk_w = (16 / sub_w) as usize;
let blk_h = (16 / sub_h) as usize;
let blocks_x = blk_w / 8;
let blocks_y = blk_h / 8;
let total_blocks = (num_mb_cols as usize) * (num_mb_rows as usize) * blocks_x * blocks_y;
let mut blocks = Vec::with_capacity(total_blocks);
let coded_depth = self.hdr.bit_depth;
let in_depth = frame.bit_depth;
for mb_y in 0..num_mb_rows {
for mb_x in 0..num_mb_cols {
let x_mb_c = ((x0 + mb_x * 16) / sub_w) as usize;
let y_mb_c = ((y0 + mb_y * 16) / sub_h) as usize;
for by in 0..blocks_y {
for bx in 0..blocks_x {
let bx0 = x_mb_c + bx * 8;
let by0 = y_mb_c + by * 8;
let mut samples = [0u16; 64];
for ry in 0..8 {
let sy = (by0 + ry).min(ph.saturating_sub(1));
for rx in 0..8 {
let sx = (bx0 + rx).min(pw.saturating_sub(1));
let v = plane[sy * pw + sx];
samples[ry * 8 + rx] = scale_sample(v, in_depth, coded_depth);
}
}
blocks.push(fdct8x8(&samples, coded_depth));
}
}
}
}
comps.push(PreparedTileComp { blocks });
}
PreparedTile { tile_idx, comps }
};
#[cfg(feature = "threads")]
{
use rayon::prelude::*;
(0..num_tiles).into_par_iter().map(prepare_one).collect()
}
#[cfg(not(feature = "threads"))]
{
(0..num_tiles).map(prepare_one).collect()
}
}
fn select_qp(&self, tiles: &[PreparedTile], max_qp: u8) -> u8 {
let target_bytes = (self.cfg.target_frame_bytes as usize).max(64);
let frame_bytes_at = |qp: u8| -> usize {
let tile_bytes = |t: &PreparedTile| -> usize {
let num_comps = t.comps.len();
let mut sz = 4 + (5 + 5 * num_comps);
for (c, comp) in t.comps.iter().enumerate() {
let bits = self.estimate_comp_bits(comp, c, qp);
sz += (bits.div_ceil(8) as usize).max(1);
}
sz
};
#[cfg(feature = "threads")]
let sum: usize = {
use rayon::prelude::*;
tiles.par_iter().map(tile_bytes).sum()
};
#[cfg(not(feature = "threads"))]
let sum: usize = tiles.iter().map(tile_bytes).sum();
24 + sum
};
let mut lo = 0u8;
let mut hi = max_qp;
if frame_bytes_at(0) <= target_bytes {
return 0;
}
while lo < hi {
let mid = lo + (hi - lo) / 2;
if frame_bytes_at(mid) <= target_bytes {
hi = mid;
} else {
lo = mid + 1;
}
}
lo
}
fn estimate_comp_bits(&self, comp: &PreparedTileComp, c: usize, qp: u8) -> u64 {
let qm = &self.hdr.q_matrix[c];
let depth = self.hdr.bit_depth;
let mut prev_dc: i32 = 0;
let mut prev_dc_diff: u32 = 20;
let mut prev_1st_ac_level: u32 = 0;
let mut bits: u64 = 0;
for f_blk in &comp.blocks {
let coeff = quantize8x8(f_blk, qm, qp, depth);
let dc = coeff[0] as i32;
let diff = dc - prev_dc;
let abs_dc_diff = diff.unsigned_abs();
let k_dc = (prev_dc_diff >> 1).min(5);
bits += vlc_bit_len(abs_dc_diff, k_dc) as u64;
if abs_dc_diff > 0 {
bits += 1;
}
prev_dc = dc;
prev_dc_diff = abs_dc_diff;
let mut scan_pos = 1usize;
let mut first_ac = true;
let mut prev_level = prev_1st_ac_level;
let mut prev_run = 0u32;
while scan_pos < 64 {
let mut run = 0u32;
while scan_pos + (run as usize) < 64 && coeff[ZIGZAG_8X8[scan_pos + (run as usize)] as usize] == 0 {
run += 1;
}
let k_run = (prev_run >> 2).min(2);
bits += vlc_bit_len(run, k_run) as u64;
scan_pos += run as usize;
prev_run = run;
if scan_pos < 64 {
let level = coeff[ZIGZAG_8X8[scan_pos] as usize];
let abs_level = level.unsigned_abs() as u32;
let k_lvl = (prev_level >> 2).min(4);
bits += vlc_bit_len(abs_level - 1, k_lvl) as u64 + 1;
scan_pos += 1;
prev_level = abs_level;
if first_ac {
first_ac = false;
prev_1st_ac_level = prev_level;
}
}
}
}
bits
}
fn encode_tile(&self, tile: &PreparedTile, qp: u8) -> Vec<u8> {
let num_comps = tile.comps.len();
let mut comp_payloads: Vec<Vec<u8>> = Vec::with_capacity(num_comps);
let depth = self.hdr.bit_depth;
for (c, comp) in tile.comps.iter().enumerate() {
let qm = &self.hdr.q_matrix[c];
let mut w = BitWriter::new();
let mut prev_dc: i32 = 0;
let mut prev_dc_diff: u32 = 20;
let mut prev_1st_ac_level: u32 = 0;
for f_blk in &comp.blocks {
let coeff = quantize8x8(f_blk, qm, qp, depth);
let dc = coeff[0] as i32;
let diff = dc - prev_dc;
let abs_dc_diff = diff.unsigned_abs();
let k_dc = (prev_dc_diff >> 1).min(5);
write_vlc(&mut w, abs_dc_diff, k_dc);
if abs_dc_diff > 0 {
w.write_bit(diff < 0);
}
prev_dc = dc;
prev_dc_diff = abs_dc_diff;
let mut scan_pos = 1usize;
let mut first_ac = true;
let mut prev_level = prev_1st_ac_level;
let mut prev_run = 0u32;
while scan_pos < 64 {
let mut run = 0u32;
while scan_pos + (run as usize) < 64 && coeff[ZIGZAG_8X8[scan_pos + (run as usize)] as usize] == 0 {
run += 1;
}
let k_run = (prev_run >> 2).min(2);
write_vlc(&mut w, run, k_run);
scan_pos += run as usize;
prev_run = run;
if scan_pos < 64 {
let level = coeff[ZIGZAG_8X8[scan_pos] as usize];
let abs_level = level.unsigned_abs() as u32;
let k_lvl = (prev_level >> 2).min(4);
write_vlc(&mut w, abs_level - 1, k_lvl);
w.write_bit(level < 0);
scan_pos += 1;
prev_level = abs_level;
if first_ac {
first_ac = false;
prev_1st_ac_level = prev_level;
}
}
}
}
comp_payloads.push(w.finish());
}
// Write tile_header(tileIdx) (RFC 9924 §5.3.13)
let tile_header_size = (5 + 5 * num_comps) as u16;
let total_size = tile_header_size as usize + comp_payloads.iter().map(Vec::len).sum::<usize>();
let mut out = Vec::with_capacity(total_size);
out.extend_from_slice(&tile_header_size.to_be_bytes());
out.extend_from_slice(&(tile.tile_idx as u16).to_be_bytes());
for cp in &comp_payloads {
out.extend_from_slice(&(cp.len() as u32).to_be_bytes());
}
for _ in 0..num_comps {
out.push(qp);
}
out.push(0); // reserved_zero_8bits
for cp in comp_payloads {
out.extend_from_slice(&cp);
}
out
}
}
#[inline]
fn scale_sample(v: u16, from_depth: u8, to_depth: u8) -> u16 {
let max_in = (1u32 << from_depth) - 1;
let v = (v as u32).min(max_in);
if to_depth == from_depth {
v as u16
} else if to_depth > from_depth {
let shift = (to_depth - from_depth) as u32;
((v << shift) | (v >> ((from_depth as u32) - shift))) as u16
} else {
let shift = (from_depth - to_depth) as u32;
let round = 1u32 << (shift - 1);
let max_out = (1u32 << to_depth) - 1;
(((v + round) >> shift).min(max_out)) as u16
}
}
+460
View File
@@ -0,0 +1,460 @@
//! RFC 9924 §5.3 and §8 syntax structures: Access Unit, PBU, Frame Header, Tile Info, and Metadata.
use filmcraft_bitstream::{BitReader, BitWriter};
use crate::tables::DEFAULT_Q_MATRIX_VAL;
use crate::{ChromaFormat, ColorInfo, ContentLightLevel, Error, MasteringDisplay, Result};
/// Four-character signature `'aPv1'` (`0x61507631`) at the start of every APV Access Unit (RFC 9924 §5.3.1).
pub const AU_SIGNATURE: [u8; 4] = *b"aPv1";
/// Primitive Bitstream Unit (PBU) types (RFC 9924 §5.3.3 Table 3).
pub const PBU_PRIMARY_FRAME: u8 = 1;
pub const PBU_NON_PRIMARY_FRAME: u8 = 2;
pub const PBU_PREVIEW_FRAME: u8 = 25;
pub const PBU_DEPTH_FRAME: u8 = 26;
pub const PBU_ALPHA_FRAME: u8 = 27;
pub const PBU_AU_INFO: u8 = 65;
pub const PBU_METADATA: u8 = 66;
pub const PBU_FILLER: u8 = 67;
/// Maximum supported frame dimension in luma samples (guards against hostile allocations).
pub const MAX_DIMENSION: u32 = 16384;
/// Maximum tiles per axis (RFC 9924 §9.4.1 specifies 20; we allow up to 64 for robustness).
pub const MAX_TILE_DIM: u32 = 64;
/// Parsed `frame_header()` and derived tile geometry (RFC 9924 §5.3.5–5.3.8).
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct FrameHeader {
pub profile_idc: u8,
pub level_idc: u8,
pub band_idc: u8,
pub width: u32,
pub height: u32,
pub chroma: ChromaFormat,
pub bit_depth: u8,
pub capture_time_distance: u8,
pub color_description_present: bool,
pub color: ColorInfo,
pub use_q_matrix: bool,
/// Per-component 8x8 quantization matrix in raster order (`y * 8 + x`).
pub q_matrix: [[u8; 64]; 4],
pub tile_width_in_mbs: u32,
pub tile_height_in_mbs: u32,
pub tile_cols: u32,
pub tile_rows: u32,
/// Tile column start positions in luma samples (`len == tile_cols + 1`).
pub col_starts: Vec<u32>,
/// Tile row start positions in luma samples (`len == tile_rows + 1`).
pub row_starts: Vec<u32>,
pub tile_sizes_in_fh: Option<Vec<u32>>,
/// Byte offset within the PBU payload where `frame_header()` ends (start of the tile loop).
pub header_bytes: usize,
}
impl FrameHeader {
pub fn frame_width_in_mbs(&self) -> u32 {
self.width.div_ceil(16)
}
pub fn frame_height_in_mbs(&self) -> u32 {
self.height.div_ceil(16)
}
pub fn num_tiles(&self) -> usize {
(self.tile_cols as usize) * (self.tile_rows as usize)
}
pub fn qp_bd_offset(&self) -> i32 {
(self.bit_depth.saturating_sub(8) as i32) * 6
}
}
/// A parsed PBU slice inside an Access Unit.
#[derive(Debug, Clone)]
pub struct PbuSlice<'a> {
pub pbu_type: u8,
pub group_id: u16,
/// Payload after the 4-byte `pbu_header()`.
pub payload: &'a [u8],
}
/// Unwrap an Access Unit byte slice: accepts either a direct `access_unit` starting with `'aPv1'`
/// or a `raw_bitstream_access_unit` prefixed with a 32-bit big-endian `au_size` (RFC 9924 Appendix A).
pub fn unwrap_au(data: &[u8]) -> Result<&[u8]> {
if data.len() < 4 {
return Err(Error::Truncated);
}
if data[..4] == AU_SIGNATURE {
return Ok(data);
}
if data.len() >= 8 && data[4..8] == AU_SIGNATURE {
let au_size = u32::from_be_bytes([data[0], data[1], data[2], data[3]]) as usize;
if au_size < 4 || au_size == 0xFFFF_FFFF {
return Err(Error::Invalid("invalid au_size in raw bitstream"));
}
let end = 4usize.checked_add(au_size).ok_or(Error::Truncated)?;
return data.get(4..end).ok_or(Error::Truncated);
}
Err(Error::Invalid("missing aPv1 access unit signature"))
}
/// Split a raw `.apv` bitstream (`raw_bitstream_access_unit()` sequence per RFC 9924 Appendix A)
/// into individual Access Unit slices (each starting with `'aPv1'`).
pub fn split_raw_bitstream(mut data: &[u8]) -> Result<Vec<&[u8]>> {
if data.is_empty() {
return Err(Error::Truncated);
}
// If the buffer starts directly with 'aPv1', treat the entire buffer as a single AU.
if data.len() >= 4 && data[..4] == AU_SIGNATURE {
return Ok(vec![data]);
}
let mut aus = Vec::new();
while !data.is_empty() {
if data.len() < 8 {
return Err(Error::Truncated);
}
let au_size = u32::from_be_bytes([data[0], data[1], data[2], data[3]]) as usize;
if au_size < 4 || au_size == 0xFFFF_FFFF {
return Err(Error::Invalid("invalid au_size in raw bitstream"));
}
let end = 4usize.checked_add(au_size).ok_or(Error::Truncated)?;
let au = data.get(4..end).ok_or(Error::Truncated)?;
if au[..4] != AU_SIGNATURE {
return Err(Error::Invalid("missing aPv1 signature in raw bitstream AU"));
}
aus.push(au);
data = &data[end..];
}
if aus.is_empty() {
return Err(Error::Truncated);
}
Ok(aus)
}
/// Parse the PBU list of a single Access Unit (RFC 9924 §5.3.1–5.3.3).
/// PBUs with `reserved_zero_8bits > 0` are ignored as required by RFC 9924 §5.3.3.
pub fn parse_au_pbus(data: &[u8]) -> Result<Vec<PbuSlice<'_>>> {
let au = unwrap_au(data)?;
if au.len() < 8 {
return Err(Error::Truncated);
}
let mut pos = 4usize;
let mut pbus = Vec::new();
while pos < au.len() {
let sz_bytes = au.get(pos..pos + 4).ok_or(Error::Truncated)?;
let pbu_size = u32::from_be_bytes([sz_bytes[0], sz_bytes[1], sz_bytes[2], sz_bytes[3]]) as usize;
if pbu_size == 0 || pbu_size == 0xFFFF_FFFF {
return Err(Error::Invalid("invalid pbu_size"));
}
if pbu_size < 4 {
return Err(Error::Truncated);
}
pos += 4;
let pbu_end = pos.checked_add(pbu_size).ok_or(Error::Truncated)?;
let pbu = au.get(pos..pbu_end).ok_or(Error::Truncated)?;
pos = pbu_end;
let pbu_type = pbu[0];
let group_id = u16::from_be_bytes([pbu[1], pbu[2]]);
let reserved_zero_8bits = pbu[3];
if reserved_zero_8bits > 0 {
// RFC 9924 §5.3.3: Decoders MUST ignore PBU with reserved_zero_8bits > 0.
continue;
}
if group_id == 0xFFFF || (group_id == 0 && pbu_type <= 64) {
return Err(Error::Invalid("invalid PBU group_id"));
}
pbus.push(PbuSlice { pbu_type, group_id, payload: &pbu[4..] });
}
Ok(pbus)
}
/// Read `byte_alignment()` and verify all alignment bits are zero (RFC 9924 §5.3.17).
pub fn read_byte_alignment(r: &mut BitReader) -> Result<()> {
while !r.is_byte_aligned() {
let bit = r.read_bit().map_err(|_| Error::Truncated)?;
if bit {
return Err(Error::Invalid("non-zero alignment bit"));
}
}
Ok(())
}
/// Parse `frame_header()` from the payload of a frame PBU (after `pbu_header()`), per RFC 9924 §5.3.5–5.3.8.
/// Returns `Ok(None)` if any `reserved_zero_*` field is non-zero (RFC 9924 §5.3.5 requires ignoring the PBU).
pub fn parse_frame_header(payload: &[u8]) -> Result<Option<FrameHeader>> {
let mut r = BitReader::new(payload);
// frame_info() (RFC 9924 §5.3.6)
let profile_idc = r.read_bits(8).map_err(|_| Error::Truncated)? as u8;
let level_idc = r.read_bits(8).map_err(|_| Error::Truncated)? as u8;
let band_idc = r.read_bits(3).map_err(|_| Error::Truncated)? as u8;
let reserved_zero_5bits = r.read_bits(5).map_err(|_| Error::Truncated)? as u8;
let width = r.read_bits(24).map_err(|_| Error::Truncated)?;
let height = r.read_bits(24).map_err(|_| Error::Truncated)?;
let chroma_format_idc = r.read_bits(4).map_err(|_| Error::Truncated)? as u8;
let bit_depth_minus8 = r.read_bits(4).map_err(|_| Error::Truncated)? as u8;
let capture_time_distance = r.read_bits(8).map_err(|_| Error::Truncated)? as u8;
let fi_reserved_8 = r.read_bits(8).map_err(|_| Error::Truncated)? as u8;
// frame_header() (RFC 9924 §5.3.5)
let fh_reserved_8_a = r.read_bits(8).map_err(|_| Error::Truncated)? as u8;
if reserved_zero_5bits > 0 || fi_reserved_8 > 0 || fh_reserved_8_a > 0 {
return Ok(None);
}
if width == 0 || height == 0 || width > MAX_DIMENSION || height > MAX_DIMENSION {
return Err(Error::Invalid("unsupported frame dimensions"));
}
let chroma = ChromaFormat::from_idc(chroma_format_idc).ok_or(Error::Invalid("invalid chroma_format_idc"))?;
if chroma == ChromaFormat::Yuv422 && !width.is_multiple_of(2) {
return Err(Error::Invalid("4:2:2 frame_width must be even"));
}
if !(2..=8).contains(&bit_depth_minus8) {
return Err(Error::Invalid("bit_depth_minus8 out of range 2..=8"));
}
let bit_depth = bit_depth_minus8 + 8;
let color_description_present = r.read_bit().map_err(|_| Error::Truncated)?;
let color = if color_description_present {
let primaries = r.read_bits(8).map_err(|_| Error::Truncated)? as u8;
let transfer = r.read_bits(8).map_err(|_| Error::Truncated)? as u8;
let matrix = r.read_bits(8).map_err(|_| Error::Truncated)? as u8;
let full_range = r.read_bit().map_err(|_| Error::Truncated)?;
ColorInfo { primaries, transfer, matrix, full_range }
} else {
ColorInfo::default()
};
let use_q_matrix = r.read_bit().map_err(|_| Error::Truncated)?;
let mut q_matrix = [[DEFAULT_Q_MATRIX_VAL; 64]; 4];
if use_q_matrix {
for c in 0..chroma.num_comps() {
for y in 0..8 {
for x in 0..8 {
let q = r.read_bits(8).map_err(|_| Error::Truncated)? as u8;
if q == 0 {
return Err(Error::Invalid("q_matrix entry 0 is reserved"));
}
q_matrix[c][y * 8 + x] = q;
}
}
}
}
// tile_info() (RFC 9924 §5.3.8)
let tile_width_in_mbs = r.read_bits(20).map_err(|_| Error::Truncated)?;
let tile_height_in_mbs = r.read_bits(20).map_err(|_| Error::Truncated)?;
if tile_width_in_mbs == 0 || tile_height_in_mbs == 0 {
return Err(Error::Invalid("tile dimensions in MBs must be non-zero"));
}
let w_mbs = width.div_ceil(16);
let h_mbs = height.div_ceil(16);
let tile_cols = w_mbs.div_ceil(tile_width_in_mbs);
let tile_rows = h_mbs.div_ceil(tile_height_in_mbs);
if tile_cols == 0 || tile_rows == 0 || tile_cols > MAX_TILE_DIM || tile_rows > MAX_TILE_DIM {
return Err(Error::Invalid("too many tiles in frame"));
}
let mut col_starts = Vec::with_capacity(tile_cols as usize + 1);
let mut start_mb = 0u32;
while start_mb < w_mbs {
col_starts.push(start_mb * 16);
start_mb = start_mb.saturating_add(tile_width_in_mbs);
}
col_starts.push(w_mbs * 16);
let mut row_starts = Vec::with_capacity(tile_rows as usize + 1);
let mut start_mb = 0u32;
while start_mb < h_mbs {
row_starts.push(start_mb * 16);
start_mb = start_mb.saturating_add(tile_height_in_mbs);
}
row_starts.push(h_mbs * 16);
let num_tiles = (tile_cols as usize) * (tile_rows as usize);
let tile_size_present_in_fh = r.read_bit().map_err(|_| Error::Truncated)?;
let tile_sizes_in_fh = if tile_size_present_in_fh {
let mut sizes = Vec::with_capacity(num_tiles);
for _ in 0..num_tiles {
let sz = r.read_bits(32).map_err(|_| Error::Truncated)?;
if sz == 0 {
return Err(Error::Invalid("tile_size_in_fh cannot be 0"));
}
sizes.push(sz);
}
Some(sizes)
} else {
None
};
let fh_reserved_8_b = r.read_bits(8).map_err(|_| Error::Truncated)? as u8;
if fh_reserved_8_b > 0 {
return Ok(None);
}
read_byte_alignment(&mut r)?;
Ok(Some(FrameHeader {
profile_idc,
level_idc,
band_idc,
width,
height,
chroma,
bit_depth,
capture_time_distance,
color_description_present,
color,
use_q_matrix,
q_matrix,
tile_width_in_mbs,
tile_height_in_mbs,
tile_cols,
tile_rows,
col_starts,
row_starts,
tile_sizes_in_fh,
header_bytes: r.byte_pos(),
}))
}
/// Write `frame_header()` to a byte vector (RFC 9924 §5.3.5–5.3.8).
pub fn write_frame_header(h: &FrameHeader) -> Vec<u8> {
let mut w = BitWriter::new();
// frame_info()
w.write_bits(h.profile_idc as u32, 8);
w.write_bits(h.level_idc as u32, 8);
w.write_bits((h.band_idc & 7) as u32, 3);
w.write_bits(0, 5); // reserved_zero_5bits
w.write_bits(h.width, 24);
w.write_bits(h.height, 24);
w.write_bits(h.chroma.idc() as u32, 4);
w.write_bits(h.bit_depth.saturating_sub(8) as u32, 4);
w.write_bits(h.capture_time_distance as u32, 8);
w.write_bits(0, 8); // reserved_zero_8bits
// frame_header()
w.write_bits(0, 8); // reserved_zero_8bits
w.write_bit(h.color_description_present);
if h.color_description_present {
w.write_bits(h.color.primaries as u32, 8);
w.write_bits(h.color.transfer as u32, 8);
w.write_bits(h.color.matrix as u32, 8);
w.write_bit(h.color.full_range);
}
w.write_bit(h.use_q_matrix);
if h.use_q_matrix {
for c in 0..h.chroma.num_comps() {
for y in 0..8 {
for x in 0..8 {
w.write_bits(h.q_matrix[c][y * 8 + x].max(1) as u32, 8);
}
}
}
}
// tile_info()
w.write_bits(h.tile_width_in_mbs, 20);
w.write_bits(h.tile_height_in_mbs, 20);
if let Some(sizes) = &h.tile_sizes_in_fh {
w.write_bit(true);
for &sz in sizes {
w.write_bits(sz, 32);
}
} else {
w.write_bit(false);
}
w.write_bits(0, 8); // reserved_zero_8bits
w.finish()
}
/// Parse a `metadata()` PBU payload (after `pbu_header()`), extracting MDCV (`payloadType == 5`)
/// and CLL (`payloadType == 6`) if present (RFC 9924 §5.3.10 & §8).
pub fn parse_metadata_pbu(payload: &[u8], mdcv: &mut Option<MasteringDisplay>, cll: &mut Option<ContentLightLevel>) -> Result<()> {
if payload.len() < 4 {
return Err(Error::Truncated);
}
let metadata_size = u32::from_be_bytes([payload[0], payload[1], payload[2], payload[3]]) as usize;
let body = payload.get(4..4usize.checked_add(metadata_size).ok_or(Error::Truncated)?).ok_or(Error::Truncated)?;
let mut pos = 0usize;
while pos < body.len() {
let mut payload_type = 0u32;
while *body.get(pos).ok_or(Error::Truncated)? == 0xFF {
payload_type = payload_type.checked_add(255).ok_or(Error::Invalid("metadata payloadType overflow"))?;
pos += 1;
}
payload_type = payload_type.checked_add(*body.get(pos).ok_or(Error::Truncated)? as u32).ok_or(Error::Invalid("metadata payloadType overflow"))?;
pos += 1;
let mut payload_size = 0usize;
while *body.get(pos).ok_or(Error::Truncated)? == 0xFF {
payload_size = payload_size.checked_add(255).ok_or(Error::Invalid("metadata payloadSize overflow"))?;
pos += 1;
}
payload_size = payload_size.checked_add(*body.get(pos).ok_or(Error::Truncated)? as usize).ok_or(Error::Invalid("metadata payloadSize overflow"))?;
pos += 1;
let end = pos.checked_add(payload_size).ok_or(Error::Truncated)?;
let item = body.get(pos..end).ok_or(Error::Truncated)?;
pos = end;
match payload_type {
5 if item.len() >= 24 => {
let u16_at = |i: usize| u16::from_be_bytes([item[i], item[i + 1]]);
let u32_at = |i: usize| u32::from_be_bytes([item[i], item[i + 1], item[i + 2], item[i + 3]]);
*mdcv = Some(MasteringDisplay {
primaries: [(u16_at(0), u16_at(2)), (u16_at(4), u16_at(6)), (u16_at(8), u16_at(10))],
white_point: (u16_at(12), u16_at(14)),
max_luminance: u32_at(16),
min_luminance: u32_at(20),
});
}
6 if item.len() >= 4 => {
*cll = Some(ContentLightLevel { max_cll: u16::from_be_bytes([item[0], item[1]]), max_fall: u16::from_be_bytes([item[2], item[3]]) });
}
// Filler metadata (§8.2.1): must be 0xFF bytes
10 if item.iter().any(|&b| b != 0xFF) => {
return Err(Error::Invalid("non-0xFF byte in filler metadata"));
}
_ => {
// RFC 9924 §10: A decoder MUST NOT try to process metadata whose type is not recognized.
}
}
}
Ok(())
}
/// Build a `metadata()` PBU (including its 4-byte `pbu_header`) carrying MDCV and/or CLL metadata.
pub fn write_metadata_pbu(group_id: u16, mdcv: Option<&MasteringDisplay>, cll: Option<&ContentLightLevel>) -> Option<Vec<u8>> {
if mdcv.is_none() && cll.is_none() {
return None;
}
let mut body = Vec::new();
if let Some(m) = mdcv {
body.push(5); // payloadType = 5 (MDCV)
body.push(24); // payloadSize = 24
for &(x, y) in &m.primaries {
body.extend_from_slice(&x.to_be_bytes());
body.extend_from_slice(&y.to_be_bytes());
}
body.extend_from_slice(&m.white_point.0.to_be_bytes());
body.extend_from_slice(&m.white_point.1.to_be_bytes());
body.extend_from_slice(&m.max_luminance.to_be_bytes());
body.extend_from_slice(&m.min_luminance.to_be_bytes());
}
if let Some(c) = cll {
body.push(6); // payloadType = 6 (CLL)
body.push(4); // payloadSize = 4
body.extend_from_slice(&c.max_cll.to_be_bytes());
body.extend_from_slice(&c.max_fall.to_be_bytes());
}
let mut pbu = Vec::with_capacity(8 + body.len());
pbu.push(PBU_METADATA);
pbu.extend_from_slice(&group_id.to_be_bytes());
pbu.push(0); // reserved_zero_8bits
pbu.extend_from_slice(&(body.len() as u32).to_be_bytes());
pbu.extend_from_slice(&body);
Some(pbu)
}
+329
View File
@@ -0,0 +1,329 @@
//! Clean-room Advanced Professional Video (APV) codec, implemented strictly from IETF RFC 9924
//! (February 2026).
//!
//! - **Decoder** ([`decode_frame`], [`decode_frame_with`], [`decode_frame_into`], [`probe`]):
//! every profile in RFC 9924 §9 (`422-10`, `422-12`, `444-10`, `444-12`, `4444-10`, `4444-12`,
//! `400-10`) at 10–16 bits per sample, custom quantization matrices, multi-tile parallel decoding
//! (behind the `threads` feature), 4:4:4:4 and auxiliary alpha PBU (`pbu_type == 27`) support,
//! and HDR static metadata (`MDCV` and `CLL`, RFC 9924 §8).
//! - **Encoder** ([`Encoder`]): progressive intra frame encoding across all 7 profiles with fixed-QP
//! or frame-budget rate control, plus `APVDecoderConfigurationRecord` (`apvC`) generation.
//!
//! See the crate README for specification edition, accuracy against external oracles, and design notes.
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
pub mod dct;
mod decode;
mod encode;
pub mod header;
pub mod tables;
pub use decode::{DecodeOptions, decode_frame, decode_frame_into, decode_frame_with, probe};
pub use encode::{Encoder, EncoderConfig};
pub use header::{AU_SIGNATURE, FrameHeader, split_raw_bitstream, unwrap_au};
/// APV codec errors.
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
pub enum Error {
#[error("truncated APV bitstream")]
Truncated,
#[error("invalid APV bitstream: {0}")]
Invalid(&'static str),
#[error("corrupt APV tile {0}")]
CorruptTile(usize),
#[error("invalid APV encoder input: {0}")]
Input(String),
}
pub type Result<T> = std::result::Result<T, Error>;
/// Chroma format (`chroma_format_idc` in RFC 9924 §4.1 Table 2).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum ChromaFormat {
/// `chroma_format_idc == 0`: 4:0:0 monochrome (1 component).
Monochrome,
/// `chroma_format_idc == 2`: 4:2:2 YCbCr (3 components, `SubWidthC = 2, SubHeightC = 1`).
Yuv422,
/// `chroma_format_idc == 3`: 4:4:4 YCbCr (3 components, `SubWidthC = 1, SubHeightC = 1`).
Yuv444,
/// `chroma_format_idc == 4`: 4:4:4:4 YCbCrA (4 components, `SubWidthC = 1, SubHeightC = 1`).
Yuv4444,
}
impl ChromaFormat {
pub fn from_idc(idc: u8) -> Option<Self> {
match idc {
0 => Some(Self::Monochrome),
2 => Some(Self::Yuv422),
3 => Some(Self::Yuv444),
4 => Some(Self::Yuv4444),
_ => None,
}
}
pub fn idc(self) -> u8 {
match self {
Self::Monochrome => 0,
Self::Yuv422 => 2,
Self::Yuv444 => 3,
Self::Yuv4444 => 4,
}
}
pub fn num_comps(self) -> usize {
match self {
Self::Monochrome => 1,
Self::Yuv422 | Self::Yuv444 => 3,
Self::Yuv4444 => 4,
}
}
pub fn sub_width_c(self) -> u32 {
match self {
Self::Yuv422 => 2,
_ => 1,
}
}
pub fn sub_height_c(self) -> u32 {
1
}
pub fn chroma_width(self, width: u32) -> u32 {
match self {
Self::Monochrome => 0,
Self::Yuv422 => width.div_ceil(2),
Self::Yuv444 | Self::Yuv4444 => width,
}
}
pub fn chroma_height(self, height: u32) -> u32 {
match self {
Self::Monochrome => 0,
_ => height,
}
}
pub fn has_alpha(self) -> bool {
self == Self::Yuv4444
}
}
/// APV profiles defined in RFC 9924 §9.3.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Default)]
pub enum Profile {
/// `profile_idc == 33`: 422-10 profile (4:2:2, 10-bit).
#[default]
P422_10,
/// `profile_idc == 44`: 422-12 profile (4:2:2, 10–12-bit).
P422_12,
/// `profile_idc == 55`: 444-10 profile (4:2:2..4:4:4, 10-bit).
P444_10,
/// `profile_idc == 66`: 444-12 profile (4:2:2..4:4:4, 10–12-bit).
P444_12,
/// `profile_idc == 77`: 4444-10 profile (4:2:2..4:4:4:4, 10-bit).
P4444_10,
/// `profile_idc == 88`: 4444-12 profile (4:2:2..4:4:4:4, 10–12-bit).
P4444_12,
/// `profile_idc == 99`: 400-10 profile (4:0:0, 10-bit).
P400_10,
}
impl Profile {
pub const ALL: [Profile; 7] =
[Profile::P422_10, Profile::P422_12, Profile::P444_10, Profile::P444_12, Profile::P4444_10, Profile::P4444_12, Profile::P400_10];
pub fn idc(self) -> u8 {
match self {
Self::P422_10 => 33,
Self::P422_12 => 44,
Self::P444_10 => 55,
Self::P444_12 => 66,
Self::P4444_10 => 77,
Self::P4444_12 => 88,
Self::P400_10 => 99,
}
}
pub fn from_idc(idc: u8) -> Option<Self> {
Self::ALL.into_iter().find(|p| p.idc() == idc)
}
pub fn name(self) -> &'static str {
match self {
Self::P422_10 => "APV 422-10",
Self::P422_12 => "APV 422-12",
Self::P444_10 => "APV 444-10",
Self::P444_12 => "APV 444-12",
Self::P4444_10 => "APV 4444-10",
Self::P4444_12 => "APV 4444-12",
Self::P400_10 => "APV 400-10",
}
}
pub fn short_name(self) -> &'static str {
match self {
Self::P422_10 => "422-10",
Self::P422_12 => "422-12",
Self::P444_10 => "444-10",
Self::P444_12 => "444-12",
Self::P4444_10 => "4444-10",
Self::P4444_12 => "4444-12",
Self::P400_10 => "400-10",
}
}
pub fn default_chroma(self) -> ChromaFormat {
match self {
Self::P422_10 | Self::P422_12 => ChromaFormat::Yuv422,
Self::P444_10 | Self::P444_12 => ChromaFormat::Yuv444,
Self::P4444_10 | Self::P4444_12 => ChromaFormat::Yuv4444,
Self::P400_10 => ChromaFormat::Monochrome,
}
}
pub fn default_bit_depth(self) -> u8 {
match self {
Self::P422_12 | Self::P444_12 | Self::P4444_12 => 12,
_ => 10,
}
}
/// Check whether `(chroma, bit_depth)` conforms to this profile per RFC 9924 §9.3.
pub fn supports(self, chroma: ChromaFormat, bit_depth: u8) -> bool {
let c = chroma.idc();
let b = bit_depth.saturating_sub(8);
match self {
Self::P422_10 => c == 2 && b == 2,
Self::P422_12 => c == 2 && (2..=4).contains(&b),
Self::P444_10 => (2..=3).contains(&c) && b == 2,
Self::P444_12 => (2..=3).contains(&c) && (2..=4).contains(&b),
Self::P4444_10 => (2..=4).contains(&c) && b == 2,
Self::P4444_12 => (2..=4).contains(&c) && (2..=4).contains(&b),
Self::P400_10 => c == 0 && b == 2,
}
}
/// Nominal target bytes per frame for `(width, height)`.
pub fn nominal_frame_bytes(self, width: u32, height: u32) -> u32 {
let bpp = match self {
Self::P400_10 => 1.5,
Self::P422_10 => 3.0,
Self::P422_12 => 3.8,
Self::P444_10 => 4.2,
Self::P444_12 => 5.0,
Self::P4444_10 => 5.2,
Self::P4444_12 => 6.2,
};
let bytes = ((width as f64) * (height as f64) * bpp / 8.0).round() as u32;
bytes.max(512)
}
/// Nominal data rate in Mbit/s at 1920x1080 29.97 fps.
pub fn nominal_mbps_1080p30(self) -> f64 {
(self.nominal_frame_bytes(1920, 1080) as f64) * 8.0 * 29.97 / 1e6
}
}
/// Human-readable profile name from `profile_idc` (e.g. `"APV 422-10"`).
pub fn profile_name(profile_idc: u8) -> String {
Profile::from_idc(profile_idc).map(|p| p.name().to_string()).unwrap_or_else(|| format!("APV (profile {profile_idc})"))
}
/// Colour description from `frame_header()` (ITU-T H.273 code points, RFC 9924 §5.3.5).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub struct ColorInfo {
pub primaries: u8,
pub transfer: u8,
pub matrix: u8,
pub full_range: bool,
}
impl Default for ColorInfo {
/// Inferred values when `color_description_present_flag == 0` (RFC 9924 §5.3.5: 2, 2, 2, 0).
fn default() -> Self {
Self { primaries: 2, transfer: 2, matrix: 2, full_range: false }
}
}
impl ColorInfo {
pub const BT709: Self = Self { primaries: 1, transfer: 1, matrix: 1, full_range: false };
pub const BT2020_PQ: Self = Self { primaries: 9, transfer: 16, matrix: 9, full_range: false };
pub const BT2020_HLG: Self = Self { primaries: 9, transfer: 18, matrix: 9, full_range: false };
}
/// Mastering Display Color Volume metadata (`payloadType == 5`, RFC 9924 §8.2.3).
///
/// Chromaticities are in 0.16 fixed-point (1/65536), order `[R, G, B]`;
/// `max_luminance` is 24.8 fixed-point cd/m²; `min_luminance` is 18.14 fixed-point cd/m².
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub struct MasteringDisplay {
pub primaries: [(u16, u16); 3],
pub white_point: (u16, u16),
pub max_luminance: u32,
pub min_luminance: u32,
}
/// Content Light-Level Information metadata (`payloadType == 6`, RFC 9924 §8.2.4).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub struct ContentLightLevel {
pub max_cll: u16,
pub max_fall: u16,
}
/// A decoded (or to-be-encoded) APV frame: planar `u16` samples, rows packed without padding.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Frame {
pub width: u32,
pub height: u32,
pub chroma: ChromaFormat,
/// Significant bits per sample (`10..=16`, or `8..=16` when rescaled via [`DecodeOptions`]).
pub bit_depth: u8,
pub y: Vec<u16>,
pub cb: Vec<u16>,
pub cr: Vec<u16>,
pub alpha: Option<Vec<u16>>,
pub color: ColorInfo,
pub profile_idc: u8,
pub level_idc: u8,
pub band_idc: u8,
pub mastering_display: Option<MasteringDisplay>,
pub content_light: Option<ContentLightLevel>,
}
impl Frame {
/// Allocate a black video-range frame.
pub fn new(width: u32, height: u32, chroma: ChromaFormat, bit_depth: u8, with_alpha: bool) -> Self {
let n = (width as usize) * (height as usize);
let cw = chroma.chroma_width(width) as usize;
let ch = chroma.chroma_height(height) as usize;
let cn = cw * ch;
let shift = bit_depth.saturating_sub(8) as u32;
let has_alpha = with_alpha || chroma == ChromaFormat::Yuv4444;
Self {
width,
height,
chroma,
bit_depth,
y: vec![(16u32 << shift) as u16; n],
cb: vec![(128u32 << shift) as u16; cn],
cr: vec![(128u32 << shift) as u16; cn],
alpha: has_alpha.then(|| vec![((1u32 << bit_depth) - 1) as u16; n]),
color: ColorInfo::BT709,
profile_idc: Profile::P422_10.idc(),
level_idc: 90,
band_idc: 2,
mastering_display: None,
content_light: None,
}
}
pub fn chroma_width(&self) -> u32 {
self.chroma.chroma_width(self.width)
}
pub fn chroma_height(&self) -> u32 {
self.chroma.chroma_height(self.height)
}
}
+59
View File
@@ -0,0 +1,59 @@
//! RFC 9924 tables: zig-zag scan, 8x8 transform matrix, quantization level scales, and level limits.
/// 8x8 zig-zag scan order (`ScanOrder[sPos] = yC * 8 + xC`), derived from RFC 9924 §4.4.1 Figure 5.
pub const ZIGZAG_8X8: [u8; 64] = [
0, 1, 8, 16, 9, 2, 3, 10, 17, 24, 32, 25, 18, 11, 4, 5, 12, 19, 26, 33, 40, 48, 41, 34, 27, 20, 13, 6, 7, 14, 21, 28, 35, 42, 49, 56, 57, 50, 43, 36, 29,
22, 15, 23, 30, 37, 44, 51, 58, 59, 52, 45, 38, 31, 39, 46, 53, 60, 61, 54, 47, 55, 62, 63,
];
/// 8x8 transform matrix (`transMatrix` in RFC 9924 §6.3.2.3 Figure 25).
pub const TRANS_MATRIX: [[i32; 8]; 8] = [
[64, 64, 64, 64, 64, 64, 64, 64],
[89, 75, 50, 18, -18, -50, -75, -89],
[84, 35, -35, -84, -84, -35, 35, 84],
[75, -18, -89, -50, 50, 89, 18, -75],
[64, -64, -64, 64, 64, -64, -64, 64],
[50, -89, 18, 75, -75, -18, 89, -50],
[35, -84, 84, -35, -35, 84, -84, 35],
[18, -50, 75, -89, 89, -75, 50, -18],
];
/// Quantization scaling list `levelScale[k]` for `k = qP % 6` (RFC 9924 §6.3.1).
pub const LEVEL_SCALE: [i32; 6] = [40, 45, 51, 57, 64, 71];
/// Default flat quantization matrix value when `use_q_matrix == 0` (RFC 9924 §5.3.7).
pub const DEFAULT_Q_MATRIX_VAL: u8 = 16;
/// General level limits (`level_idc`, max luma sample rate, and max coded data rate in Mbit/s per band)
/// from RFC 9924 §9.4.2 Table 4.
pub const LEVEL_LIMITS: [(u8, u64, [u32; 4]); 14] = [
(30, 3_041_280, [8, 11, 15, 23]),
(33, 6_082_560, [16, 21, 30, 45]),
(60, 15_667_200, [39, 54, 76, 114]),
(63, 31_334_400, [78, 108, 152, 227]),
(90, 66_846_720, [114, 159, 222, 333]),
(93, 133_693_440, [227, 317, 444, 666]),
(120, 265_420_800, [455, 637, 892, 1_338]),
(123, 530_841_600, [910, 1_274, 1_784, 2_675]),
(150, 1_061_683_200, [1_820, 2_548, 3_567, 5_350]),
(153, 2_123_366_400, [3_639, 5_095, 7_133, 10_699]),
(180, 4_777_574_400, [7_278, 10_189, 14_265, 21_397]),
(183, 8_493_465_600, [14_556, 20_378, 28_529, 42_793]),
(210, 16_986_931_200, [29_111, 40_756, 57_058, 85_586]),
(213, 33_973_862_400, [58_222, 81_511, 114_115, 171_172]),
];
/// Choose the lowest conforming `(level_idc, band_idc)` for a luma sample rate and bitrate (in kbps).
pub fn select_level_and_band(luma_samples_per_sec: u64, kbps: u32) -> (u8, u8) {
let mbps = kbps.div_ceil(1000);
for &(level_idc, max_samples, bands) in &LEVEL_LIMITS {
if luma_samples_per_sec <= max_samples {
for (band_idc, &max_mbps) in bands.iter().enumerate() {
if mbps <= max_mbps {
return (level_idc, band_idc as u8);
}
}
}
}
(213, 3)
}
+136
View File
@@ -0,0 +1,136 @@
//! Robustness: mutated, truncated, spliced, and random bitstreams must never panic (errors are fine).
use std::panic::{AssertUnwindSafe, catch_unwind};
use filmcraft_apv::*;
struct Rng(u64);
impl Rng {
fn next(&mut self) -> u64 {
self.0 ^= self.0 << 13;
self.0 ^= self.0 >> 7;
self.0 ^= self.0 << 17;
self.0
}
fn below(&mut self, n: usize) -> usize {
(self.next() % n.max(1) as u64) as usize
}
}
fn seeds() -> Vec<Vec<u8>> {
let mut v = Vec::new();
for (p, w, h) in
[(Profile::P422_10, 64u32, 48u32), (Profile::P422_12, 70, 38), (Profile::P444_10, 48, 32), (Profile::P4444_10, 64, 32), (Profile::P400_10, 32, 32)]
{
let chroma = p.default_chroma();
let depth = p.default_bit_depth();
let mut f = Frame::new(w, h, chroma, depth, chroma == ChromaFormat::Yuv4444);
for (i, s) in f.y.iter_mut().enumerate() {
*s = (64 + (i * 37) % 800) as u16;
}
if let Some(a) = f.alpha.as_mut() {
for (i, s) in a.iter_mut().enumerate() {
*s = ((i * 19) % 1024) as u16;
}
}
let mut cfg = EncoderConfig::new(p, w, h);
cfg.qp = Some(16);
let mut enc = Encoder::with_config(cfg).expect("seed encoder");
v.push(enc.encode(&f).expect("seed encode"));
v.push(enc.encode_raw_au(&f).expect("seed encode_raw_au"));
}
v
}
#[test]
fn mutated_streams_never_panic() {
let seeds = seeds();
for s in &seeds {
decode_frame(s).expect("seed decodes");
}
let mut rng = Rng(0x9e3779b97f4a7c15);
let mut decoded_ok = 0usize;
let iters = 3000usize;
for i in 0..iters {
let mut d = seeds[i % seeds.len()].clone();
match rng.below(6) {
0 => {
// Bit flips
for _ in 0..1 + rng.below(8) {
let p = rng.below(d.len());
d[p] ^= 1 << rng.below(8);
}
}
1 => {
// Random byte overwrites
for _ in 0..1 + rng.below(4) {
let p = rng.below(d.len());
d[p] = rng.next() as u8;
}
}
2 => {
// Truncation
let n = rng.below(d.len());
d.truncate(n);
}
3 => {
// Corrupt header fields
if d.len() > 8 {
let p = 4 + rng.below(24.min(d.len() - 4));
d[p] = rng.next() as u8;
}
}
4 => {
// Corrupt tile payload region
let start = 24.min(d.len().saturating_sub(1));
let off = start + rng.below(d.len().saturating_sub(start).max(1));
for b in d.iter_mut().skip(off).take(1 + rng.below(48)) {
*b = rng.next() as u8;
}
}
_ => {
// Splice random bytes
let p = rng.below(d.len());
let extra: Vec<u8> = (0..1 + rng.below(16)).map(|_| rng.next() as u8).collect();
d.splice(p..p, extra);
}
}
let res = catch_unwind(AssertUnwindSafe(|| {
for depth in [None, Some(8), Some(16)] {
let opts = DecodeOptions { bit_depth: depth, threads: i % 2 == 0 };
if let Ok(f) = decode_frame_with(&d, &opts) {
decoded_ok += 1;
assert_eq!(f.y.len(), (f.width as usize) * (f.height as usize));
}
}
let _ = probe(&d);
let _ = split_raw_bitstream(&d);
}));
assert!(res.is_ok(), "panic on mutated stream iteration {i}");
}
println!("{decoded_ok} of {} mutated decodes succeeded", iters * 3);
}
#[test]
fn random_bytes_never_panic() {
let mut rng = Rng(123456789);
for i in 0..2000 {
let n = rng.below(400);
let mut d: Vec<u8> = (0..n).map(|_| rng.next() as u8).collect();
if d.len() >= 8 && rng.below(2) == 0 {
d[..4].copy_from_slice(b"aPv1");
} else if d.len() >= 12 && rng.below(2) == 0 {
let au_sz = (n - 4) as u32;
d[..4].copy_from_slice(&au_sz.to_be_bytes());
d[4..8].copy_from_slice(b"aPv1");
}
let res = catch_unwind(AssertUnwindSafe(|| {
let _ = decode_frame(&d);
let _ = probe(&d);
let _ = split_raw_bitstream(&d);
}));
assert!(res.is_ok(), "panic on random bytes iteration {i}");
}
}
+230
View File
@@ -0,0 +1,230 @@
//! Conformance, bit-exactness, and quality tests against ffmpeg's APV decoder (external oracle).
use std::path::{Path, PathBuf};
use std::process::Command;
use filmcraft_apv::{
ChromaFormat, ContentLightLevel, DecodeOptions, Encoder, EncoderConfig, Frame, MasteringDisplay, Profile, decode_frame, decode_frame_with, probe,
};
fn ffmpeg() -> Option<PathBuf> {
filmcraft_testkit::ffmpeg_or_skip("apv oracle")
}
fn fixture_dir() -> PathBuf {
filmcraft_testkit::fixtures_dir("apv")
}
fn native_pix_fmt(profile: Profile) -> &'static str {
match profile {
Profile::P422_10 => "yuv422p10le",
Profile::P422_12 => "yuv422p12le",
Profile::P444_10 => "yuv444p10le",
Profile::P444_12 => "yuv444p12le",
Profile::P4444_10 => "yuva444p10le",
Profile::P4444_12 => "yuva444p12le",
Profile::P400_10 => "gray10le",
}
}
/// Decode a raw `.apv` file with ffmpeg (`-v error -xerror`) to planar `u16` samples.
fn ffmpeg_decode_raw(ff: &Path, apv_path: &Path, pix_fmt: &str) -> (Vec<u16>, String) {
let out = Command::new(ff)
.args(["-hide_banner", "-v", "error", "-xerror", "-f", "apv", "-i"])
.arg(apv_path)
.args(["-f", "rawvideo", "-pix_fmt", pix_fmt, "-"])
.output()
.expect("run ffmpeg");
let stderr = String::from_utf8_lossy(&out.stderr).into_owned();
assert!(out.status.success(), "ffmpeg failed on {:?}: {}", apv_path, stderr);
let samples = out.stdout.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect();
(samples, stderr)
}
fn synthetic_frame(profile: Profile, w: u32, h: u32) -> Frame {
let chroma = profile.default_chroma();
let depth = profile.default_bit_depth();
let mut f = Frame::new(w, h, chroma, depth, chroma == ChromaFormat::Yuv4444);
let max_v = (1u32 << depth) - 1;
let lo = 16u32 << (depth - 8);
let hi = 235u32 << (depth - 8);
let span = hi - lo;
for y in 0..h {
for x in 0..w {
// Mix smooth gradients, diagonal edges, and high-frequency patterns
let grad = (x * 7 + y * 13) % span;
let checker = if ((x / 3) ^ (y / 5)) & 1 == 0 { 40 << (depth - 10) } else { 0 };
f.y[(y * w + x) as usize] = (lo + (grad + checker) % span).min(max_v) as u16;
}
}
let cw = f.chroma_width();
let ch = f.chroma_height();
for y in 0..ch {
for x in 0..cw {
let u = lo + ((x * 11 + y * 5 + 100) % span);
let v = lo + ((x * 3 + y * 17 + 300) % span);
f.cb[(y * cw + x) as usize] = u.min(max_v) as u16;
f.cr[(y * cw + x) as usize] = v.min(max_v) as u16;
}
}
if let Some(a) = f.alpha.as_mut() {
for y in 0..h {
for x in 0..w {
let dx = x as i32 - (w as i32) / 2;
let dy = y as i32 - (h as i32) / 2;
a[(y * w + x) as usize] =
if dx * dx + dy * dy < ((w.min(h) as i32) / 3).pow(2) { max_v as u16 } else { ((x * 19 + y * 23) % (max_v + 1)) as u16 };
}
}
}
f
}
fn flatten_frame(f: &Frame) -> Vec<u16> {
let mut v = Vec::new();
v.extend_from_slice(&f.y);
v.extend_from_slice(&f.cb);
v.extend_from_slice(&f.cr);
if let Some(a) = &f.alpha {
v.extend_from_slice(a);
}
v
}
fn psnr(a: &[u16], b: &[u16], peak: f64) -> f64 {
if a.is_empty() {
return 99.0;
}
let mse: f64 = a.iter().zip(b).map(|(&x, &y)| (x as f64 - y as f64).powi(2)).sum::<f64>() / a.len() as f64;
if mse == 0.0 { 99.0 } else { 10.0 * (peak * peak / mse).log10() }
}
#[test]
fn all_profiles_bit_exact_with_ffmpeg() {
let Some(ff) = ffmpeg() else { return };
let dir = fixture_dir();
for profile in Profile::ALL {
let (w, h) = (320u32, 192u32);
let src = synthetic_frame(profile, w, h);
let mut cfg = EncoderConfig::new(profile, w, h);
cfg.qp = Some(12);
let mut enc = Encoder::with_config(cfg).expect("encoder init");
let raw_au = enc.encode_raw_au(&src).expect("encode_raw_au");
let apv_path = dir.join(format!("oracle_{}.apv", profile.short_name()));
std::fs::write(&apv_path, &raw_au).expect("write .apv");
let (ff_samples, stderr) = ffmpeg_decode_raw(&ff, &apv_path, native_pix_fmt(profile));
assert!(stderr.trim().is_empty(), "{profile:?}: ffmpeg stderr: {stderr}");
let ours = decode_frame(&raw_au).expect("our decode");
let our_samples = flatten_frame(&ours);
assert_eq!(our_samples.len(), ff_samples.len(), "{profile:?}: sample count mismatch");
let max_diff = our_samples.iter().zip(&ff_samples).map(|(&a, &b)| (a as i32 - b as i32).unsigned_abs()).max().unwrap_or(0);
assert_eq!(max_diff, 0, "{profile:?}: our decoder must be 100% bit-exact with ffmpeg (max_diff={max_diff})");
let peak = ((1u32 << profile.default_bit_depth()) - 1) as f64;
let y_psnr = psnr(&ours.y, &src.y, peak);
assert!(y_psnr > 45.0, "{profile:?}: expected high PSNR at qp=12, got {y_psnr:.2} dB");
}
}
#[test]
fn odd_dimensions_and_multi_tile_bit_exact_with_ffmpeg() {
let Some(ff) = ffmpeg() else { return };
let dir = fixture_dir();
for (profile, w, h) in [
(Profile::P422_10, 640u32, 360u32),
(Profile::P422_10, 334, 202),
(Profile::P444_10, 333, 201),
(Profile::P4444_12, 512, 288),
(Profile::P400_10, 70, 38),
] {
let mut src = synthetic_frame(profile, w, h);
src.mastering_display = Some(MasteringDisplay {
primaries: [(46400, 19136), (11141, 52232), (8585, 3015)],
white_point: (20493, 21561),
max_luminance: 1000 << 8,
min_luminance: 1,
});
src.content_light = Some(ContentLightLevel { max_cll: 1000, max_fall: 400 });
let mut enc = Encoder::new(profile, w, h).expect("encoder init");
let raw_au = enc.encode_raw_au(&src).expect("encode");
let apv_path = dir.join(format!("oracle_geom_{}_{w}x{h}.apv", profile.short_name()));
std::fs::write(&apv_path, &raw_au).expect("write .apv");
let (ff_samples, stderr) = ffmpeg_decode_raw(&ff, &apv_path, native_pix_fmt(profile));
assert!(stderr.trim().is_empty(), "{profile:?} {w}x{h}: {stderr}");
let hdr = probe(&raw_au).expect("probe");
assert_eq!((hdr.width, hdr.height), (w, h));
let ours = decode_frame(&raw_au).expect("decode");
assert_eq!(ours.mastering_display, src.mastering_display);
assert_eq!(ours.content_light, src.content_light);
let our_samples = flatten_frame(&ours);
assert_eq!(our_samples, ff_samples, "{profile:?} {w}x{h}: bit-exact match with ffmpeg failed");
// Also verify single-threaded decode produces identical output
let ours_st = decode_frame_with(&raw_au, &DecodeOptions { bit_depth: None, threads: false }).expect("decode single-threaded");
assert_eq!(ours, ours_st);
}
}
#[test]
fn custom_q_matrix_and_fh_tile_sizes_bit_exact_with_ffmpeg() {
let Some(ff) = ffmpeg() else { return };
let dir = fixture_dir();
let profile = Profile::P444_12;
let (w, h) = (512u32, 256u32);
let src = synthetic_frame(profile, w, h);
let mut q_matrix = [[16u8; 64]; 4];
for (c, qm) in q_matrix.iter_mut().enumerate() {
for y in 0..8 {
for x in 0..8 {
// Intentionally asymmetric (x != y weight) to verify exact (x, y) matrix orientation
qm[y * 8 + x] = (10 + c * 3 + y * 5 + x * 2) as u8;
}
}
}
let mut cfg = EncoderConfig::new(profile, w, h);
cfg.qp = Some(18);
cfg.q_matrix = Some(q_matrix);
cfg.write_tile_sizes_in_fh = true;
cfg.tile_width_in_mbs = 16;
cfg.tile_height_in_mbs = 8;
let mut enc = Encoder::with_config(cfg).expect("encoder init");
let raw_au = enc.encode_raw_au(&src).expect("encode");
let apv_path = dir.join("oracle_custom_qmat_444-12.apv");
std::fs::write(&apv_path, &raw_au).expect("write .apv");
let (ff_samples, stderr) = ffmpeg_decode_raw(&ff, &apv_path, native_pix_fmt(profile));
assert!(stderr.trim().is_empty(), "ffmpeg stderr: {stderr}");
let hdr = probe(&raw_au).expect("probe");
assert!(hdr.use_q_matrix);
assert!(hdr.tile_sizes_in_fh.is_some());
assert_eq!(hdr.num_tiles(), 4);
let ours = decode_frame(&raw_au).expect("decode");
let our_samples = flatten_frame(&ours);
assert_eq!(our_samples, ff_samples, "asymmetric q_matrix decode must be bit-exact with ffmpeg");
// Verify 8-bit and 16-bit rescaled output
let d8 = decode_frame_with(&raw_au, &DecodeOptions { bit_depth: Some(8), threads: true }).expect("decode 8-bit");
assert_eq!(d8.bit_depth, 8);
assert!(d8.y.iter().all(|&s| s <= 255));
let d16 = decode_frame_with(&raw_au, &DecodeOptions { bit_depth: Some(16), threads: true }).expect("decode 16-bit");
assert_eq!(d16.bit_depth, 16);
}
+2
View File
@@ -7,6 +7,7 @@
//! engine and UI can build controls generically.
//! * [`biquad`] — RBJ-cookbook biquads (TDF-II) with analytic magnitude response.
//! * [`sync`] — offset between two recordings of one event (GCC-PHAT, sample-accurate).
//! * [`resample`] — streaming, block-invariant rate conversion for playback on a device at another rate.
//! * [`channels`] — channel layouts, ITU-R BS.775 up/downmix, 5.1 mixdown types, the 5.1 panner.
//!
//! Conventions: audio is **planar f32** (`&mut [&mut [f32]]`, one slice per channel, all the same
@@ -27,6 +28,7 @@ pub mod fft;
pub mod loudness;
pub mod oversample;
pub mod remix;
pub mod resample;
mod smooth;
pub mod sync;
+170
View File
@@ -0,0 +1,170 @@
//! Streaming sample-rate conversion for playback, when the output device runs at another rate
//! than the sequence (a 44.1 kHz device playing a 48 kHz sequence).
//!
//! Every output sample's source position comes from its absolute output frame, as an exact
//! rational (`frame × from / to`), and is linearly interpolated, so the output is identical however
//! the stream is cut into blocks: no phase jump at block boundaries. Source frames are pulled in
//! contiguous, non-overlapping spans, so whatever produces them (the mixer, with its effect state)
//! sees one continuous stream. Output lags the source by one source sample (≈21 µs at 48 kHz),
//! which is what lets each block interpolate without reading ahead.
/// Converts a planar stream from `from` Hz to `to` Hz, block by block.
#[derive(Debug, Clone)]
pub struct StreamResampler {
from: u32,
to: u32,
/// Source frames `[held_start, held_start + held[c].len())` kept from the last pull.
held_start: i64,
held: Vec<Vec<f32>>,
/// The output frame the next block is expected to start at (contiguous playback).
next_out: Option<i64>,
}
impl StreamResampler {
/// Rates of zero are treated as 1 Hz rather than dividing by zero.
pub fn new(from: u32, to: u32) -> Self {
Self { from: from.max(1), to: to.max(1), held_start: 0, held: Vec::new(), next_out: None }
}
pub fn rates(&self) -> (u32, u32) {
(self.from, self.to)
}
/// The source frame whose sample lands at (or just before) output frame `out`.
fn source_index(&self, out: i64) -> (i64, f32) {
let p = out as i128 * self.from as i128;
let to = self.to as i128;
(p.div_euclid(to) as i64, (p.rem_euclid(to) as f64 / to as f64) as f32)
}
/// Output frames `[out, out + n)`, planar. `pull(start, frames)` returns source frames
/// `[start, start + frames)`, planar (missing samples read as silence). A block that does not
/// continue the previous one (a seek) starts the stream afresh.
pub fn process(&mut self, out: i64, n: usize, mut pull: impl FnMut(i64, usize) -> Vec<Vec<f32>>) -> Vec<Vec<f32>> {
if n == 0 {
return vec![Vec::new(); self.held.len().max(1)];
}
let (first, _) = self.source_index(out);
let (last, _) = self.source_index(out.saturating_add(n as i64 - 1));
// interpolation reads source frames first - 1 ..= last
if self.next_out != Some(out) || self.held.is_empty() {
self.held_start = first - 1;
self.held.clear();
}
let held_end = self.held_start + self.held.first().map_or(0, Vec::len) as i64;
if last + 1 > held_end {
let want = (last + 1 - held_end) as usize;
let got = pull(held_end, want);
if self.held.is_empty() {
self.held = vec![Vec::new(); got.len().max(1)];
}
for (c, h) in self.held.iter_mut().enumerate() {
let src = got.get(c).map(Vec::as_slice).unwrap_or_default();
h.extend((0..want).map(|i| src.get(i).copied().unwrap_or(0.0)));
}
}
let mut dst = vec![vec![0.0f32; n]; self.held.len()];
for (i, frame) in (out..).take(n).enumerate() {
let (s, frac) = self.source_index(frame);
// held starts at first - 1, so the earlier frame is always held
let k = usize::try_from(s - self.held_start).unwrap_or(0);
for (d, h) in dst.iter_mut().zip(&self.held) {
let a = k.checked_sub(1).and_then(|j| h.get(j)).copied().unwrap_or(0.0);
let b = h.get(k).copied().unwrap_or(0.0);
if let Some(x) = d.get_mut(i) {
*x = a + (b - a) * frac;
}
}
}
// keep what the next block can still read: from its first frame - 1 on
let (next_first, _) = self.source_index(out.saturating_add(n as i64));
let drop = (next_first - 1 - self.held_start).clamp(0, self.held.first().map_or(0, Vec::len) as i64) as usize;
for h in &mut self.held {
h.drain(..drop);
}
self.held_start += drop as i64;
self.next_out = Some(out.saturating_add(n as i64));
dst
}
}
#[cfg(test)]
mod tests {
use super::*;
/// A sine at `hz`, sampled at `rate`, from absolute frame `start`; logs every pull.
fn sine(hz: f64, rate: u32, pulls: &mut Vec<(i64, usize)>) -> impl FnMut(i64, usize) -> Vec<Vec<f32>> + '_ {
move |start, frames| {
pulls.push((start, frames));
let ch: Vec<f32> = (0..frames).map(|i| (2.0 * std::f64::consts::PI * hz * (start + i as i64) as f64 / rate as f64).sin() as f32).collect();
vec![ch.clone(), ch]
}
}
fn convert(from: u32, to: u32, blocks: &[usize], total: usize) -> (Vec<f32>, Vec<(i64, usize)>) {
let mut r = StreamResampler::new(from, to);
let mut pulls = Vec::new();
let mut out = Vec::new();
let mut pos = 0i64;
let mut k = 0;
while (pos as usize) < total {
let n = blocks[k % blocks.len()].min(total - pos as usize);
let b = r.process(pos, n, sine(1000.0, from, &mut pulls));
assert_eq!(b.len(), 2);
assert!(b.iter().all(|c| c.len() == n));
out.extend_from_slice(&b[0]);
pos += n as i64;
k += 1;
}
(out, pulls)
}
#[test]
fn output_is_the_same_however_the_stream_is_cut() {
for (from, to) in [(48_000, 44_100), (44_100, 48_000), (48_000, 96_000), (96_000, 48_000)] {
let (whole, _) = convert(from, to, &[20_000], 20_000);
for blocks in [&[512][..], &[333], &[1, 7, 512, 64]] {
let (cut, _) = convert(from, to, blocks, 20_000);
assert_eq!(cut, whole, "{from}→{to} in blocks {blocks:?}");
}
}
}
#[test]
fn a_tone_converts_to_the_same_tone() {
// 1 kHz at 48 kHz played on a 44.1 kHz device: within linear interpolation's error of the
// ideal tone, one source sample late
let (from, to) = (48_000u32, 44_100u32);
let (out, _) = convert(from, to, &[512], 20_000);
for (o, x) in out.iter().enumerate().skip(2) {
let t = o as f64 / to as f64 - 1.0 / from as f64;
let want = (2.0 * std::f64::consts::PI * 1000.0 * t).sin() as f32;
assert!((x - want).abs() < 3e-3, "frame {o}: {x} vs {want}");
}
}
#[test]
fn the_source_is_pulled_as_one_continuous_stream() {
for (from, to) in [(48_000, 44_100), (44_100, 48_000)] {
let (_, pulls) = convert(from, to, &[512, 333], 20_000);
for w in pulls.windows(2) {
assert_eq!(w[0].0 + w[0].1 as i64, w[1].0, "{from}→{to}: {:?} then {:?}", w[0], w[1]);
}
}
}
#[test]
fn a_seek_starts_afresh_and_hostile_input_stays_finite() {
let mut r = StreamResampler::new(48_000, 44_100);
let mut pulls = Vec::new();
let _ = r.process(0, 512, sine(1000.0, 48_000, &mut pulls));
let _ = r.process(1_000_000, 512, sine(1000.0, 48_000, &mut pulls));
assert_eq!(pulls.last().map(|p| p.0), Some(1_000_000 * 48_000 / 44_100 - 1));
// a mixer returning too few channels and samples, and zero rates
let mut r = StreamResampler::new(0, 0);
let b = r.process(5, 64, |_, _| vec![vec![1.0; 3]]);
assert_eq!(b.len(), 1);
assert!(b[0].iter().all(|x| x.is_finite()));
assert!(r.process(0, 0, |_, _| Vec::new()).iter().all(Vec::is_empty));
}
}
+111
View File
@@ -10,6 +10,9 @@ use tokio::sync::Mutex;
use crate::AutomationError;
/// How often a blocking export's app job is polled.
const JOB_POLL: Duration = Duration::from_millis(250);
type Conn = (BufReader<tokio::net::tcp::OwnedReadHalf>, tokio::net::tcp::OwnedWriteHalf);
pub struct BridgeClient {
@@ -29,6 +32,40 @@ impl BridgeClient {
Ok(Self { addr, conn: Mutex::new(None), next_id: AtomicU64::new(1) })
}
/// Run engine command `id` in the app. A blocking export (`file.exportMedia` with
/// `wait: true`, see [`crate::long_job::LONG_COMMANDS`]) is started as an app job and polled
/// here until it finishes, so the app keeps repainting (its status bar shows the progress) and
/// answering other requests, and no request waits on the control server's 60 s reply limit
/// (#91, #92). The reply is the same as a blocking export's: `{job, path, result}`, or the
/// export's error.
pub async fn execute(&self, id: &str, mut params: Value) -> Result<Value, AutomationError> {
if !crate::long_job::is_long(id, &params) {
return self.call("engine.execute", json!({"command": id, "params": params})).await;
}
if let Some(p) = params.as_object_mut() {
p.insert("wait".into(), json!(false));
}
let mut start = self.call("engine.execute", json!({"command": id, "params": params})).await?;
let Some(job) = start.get("job").and_then(Value::as_u64) else { return Ok(start) };
let result = loop {
tokio::time::sleep(JOB_POLL).await;
let jobs = self.call("engine.execute", json!({"command": "jobs.list", "params": {}})).await?;
let Some(j) = jobs.as_array().and_then(|a| a.iter().find(|j| j["id"].as_u64() == Some(job))).cloned() else {
break Value::Null;
};
if j["finished"].as_bool() == Some(true) {
break j["result"].clone();
}
};
if let Some(e) = result.get("error").and_then(Value::as_str) {
return Err(AutomationError::App(format!("export failed: {e}")));
}
if let Some(o) = start.as_object_mut() {
o.insert("result".into(), result);
}
Ok(start)
}
/// Call a control method; returns `result` or the app's error.
pub async fn call(&self, method: &str, params: Value) -> Result<Value, AutomationError> {
let mut guard = self.conn.lock().await;
@@ -74,3 +111,77 @@ impl BridgeClient {
Err(AutomationError::Bridge("unreachable".into()))
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Arc;
use tokio::net::TcpListener;
/// A stand-in for the app's control server: the export starts job 7, which `jobs.list` reports
/// running twice and then finished with `result`. Every request is recorded.
async fn fake_app(result: Value) -> (String, Arc<Mutex<Vec<Value>>>) {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap().to_string();
let seen = Arc::new(Mutex::new(Vec::new()));
let log = seen.clone();
tokio::spawn(async move {
let (s, _) = listener.accept().await.unwrap();
let (r, mut w) = s.into_split();
let mut lines = BufReader::new(r).lines();
let mut polls = 0;
while let Ok(Some(line)) = lines.next_line().await {
let req: Value = serde_json::from_str(&line).unwrap();
log.lock().await.push(req.clone());
let reply = match req["params"]["command"].as_str() {
Some("file.exportMedia") => json!({"job": 7, "path": "/tmp/out.mov"}),
Some("jobs.list") => {
polls += 1;
let finished = polls >= 3;
json!([{"id": 7, "finished": finished, "result": if finished { result.clone() } else { Value::Null }}])
}
_ => json!(null),
};
let line = format!("{}\n", json!({"id": req["id"], "ok": true, "result": reply}));
w.write_all(line.as_bytes()).await.unwrap();
}
});
(addr, seen)
}
/// A blocking bridge export runs as an app job (#91, #92): the app never sees `wait: true` (it
/// would encode on its UI thread and the reply would time out after 60 s), and the caller still
/// gets the finished result.
#[tokio::test]
async fn a_blocking_export_is_polled_as_a_job() {
let (addr, seen) = fake_app(json!({"frames": 642})).await;
let b = BridgeClient::new(addr).unwrap();
let r = b.execute("file.exportMedia", json!({"path": "/tmp/out.mov", "wait": true})).await.unwrap();
assert_eq!(r, json!({"job": 7, "path": "/tmp/out.mov", "result": {"frames": 642}}));
let seen = seen.lock().await;
assert_eq!(seen[0]["params"]["params"]["wait"], json!(false));
assert!(seen.iter().all(|q| q["params"]["params"]["wait"] != json!(true)));
assert_eq!(seen.iter().filter(|q| q["params"]["command"] == "file.exportMedia").count(), 1, "started once");
assert_eq!(seen.iter().filter(|q| q["params"]["command"] == "jobs.list").count(), 3);
}
#[tokio::test]
async fn a_failed_blocking_export_is_an_error() {
let (addr, _) = fake_app(json!({"error": "encode: disk full"})).await;
let b = BridgeClient::new(addr).unwrap();
let e = b.execute("file.exportMedia", json!({"path": "/tmp/out.mov", "wait": true})).await.unwrap_err();
assert!(e.to_string().contains("export failed: encode: disk full"), "{e}");
}
/// Everything else is forwarded unchanged.
#[tokio::test]
async fn other_commands_are_forwarded() {
let (addr, seen) = fake_app(Value::Null).await;
let b = BridgeClient::new(addr).unwrap();
b.execute("file.exportMedia", json!({"path": "/tmp/out.mov"})).await.unwrap();
b.execute("sequence.inspect", json!({})).await.unwrap();
let seen = seen.lock().await;
assert_eq!(seen.len(), 2);
assert_eq!(seen[1]["params"]["command"], "sequence.inspect");
}
}
+1
View File
@@ -11,6 +11,7 @@
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
pub mod bridge;
pub mod long_job;
pub mod server;
pub use bridge::BridgeClient;
+238
View File
@@ -0,0 +1,238 @@
//! Long exports over MCP (docs/agents.md § Long exports): `file.exportMedia` with `wait: true`
//! runs as a background engine job; the server reports its progress as MCP
//! `notifications/progress` (when the request carried a `progressToken`) and stops it on
//! `notifications/cancelled`, deleting the partial output. The session lock is only held for
//! short polls, so other requests are answered while the export runs.
use std::path::Path;
use std::time::{Duration, SystemTime};
use rmcp::RoleServer;
use rmcp::model::{CallToolResult, ContentBlock as Content, ProgressNotificationParam, ProgressToken};
use rmcp::service::{Peer, RequestContext};
use serde_json::{Value, json};
use crate::server::FilmcraftMcp;
/// Engine commands that block until a render is written when called with `wait: true`.
pub const LONG_COMMANDS: &[&str] = &["file.exportMedia"];
/// How often the job is polled (and at most how often progress is reported).
const POLL: Duration = Duration::from_millis(100);
/// Whether `command_run {id, params}` is a long call this module runs.
pub fn is_long(id: &str, params: &Value) -> bool {
LONG_COMMANDS.contains(&id) && params.get("wait").and_then(Value::as_bool) == Some(true)
}
/// Sends `notifications/progress` for one request; progress only ever increases.
struct Reporter {
peer: Peer<RoleServer>,
token: Option<ProgressToken>,
last: f64,
}
impl Reporter {
async fn report(&mut self, done: f64, total: f64, message: &str) {
let Some(token) = self.token.clone() else { return };
if total <= 0.0 || done <= self.last {
return;
}
self.last = done;
let mut p = ProgressNotificationParam::new(token, done).with_total(total);
if !message.is_empty() {
p = p.with_message(message);
}
let _ = self.peer.notify_progress(p).await;
}
}
impl FilmcraftMcp {
/// Run `id` (a [`LONG_COMMANDS`] entry, `wait: true`) as a job, reporting progress and
/// honouring cancellation.
pub(crate) async fn run_long(&self, id: &str, mut params: Value, context: &RequestContext<RoleServer>) -> CallToolResult {
let started = SystemTime::now();
if let Some(p) = params.as_object_mut() {
p.insert("wait".into(), json!(false));
}
let start = match self.run(id, params).await {
Ok(v) => v,
Err(e) => return CallToolResult::error(vec![Content::text(e.to_string())]),
};
let Some(job) = start.get("job").and_then(Value::as_u64) else {
return CallToolResult::error(vec![Content::text(format!("{id} started no job: {start}"))]);
};
let path = start.get("path").and_then(Value::as_str).unwrap_or_default().to_string();
let mut rep = Reporter { peer: context.peer.clone(), token: context.meta.get_progress_token(), last: 0.0 };
loop {
let state = self.job_state(job).await;
let finished = state.as_ref().is_none_or(|j| j["finished"].as_bool() == Some(true));
if let Some(j) = &state {
let (done, total) = (j["done"].as_f64().unwrap_or(0.0), j["total"].as_f64().unwrap_or(0.0));
rep.report(done, total, j["status"].as_str().unwrap_or_default()).await;
}
if finished {
let result = state.map(|j| j["result"].clone()).unwrap_or(Value::Null);
if let Some(e) = result.get("error").and_then(Value::as_str) {
return CallToolResult::error(vec![Content::text(format!("export failed: {e}"))]);
}
let mut out = start;
if let Some(o) = out.as_object_mut() {
o.insert("result".into(), result);
}
return CallToolResult::success(vec![Content::text(serde_json::to_string_pretty(&out).unwrap_or_default())]);
}
tokio::select! {
_ = context.ct.cancelled() => break,
_ = tokio::time::sleep(POLL) => {}
}
}
// Cancelled: stop the encode at the next batch, wait for the worker, delete the partial file.
let _ = self.run("jobs.cancel", json!({"job": job})).await;
for _ in 0..600 {
if self.job_state(job).await.is_none_or(|j| j["finished"].as_bool() == Some(true)) {
break;
}
tokio::time::sleep(POLL).await;
}
remove_partial(&path, started);
CallToolResult::error(vec![Content::text("cancelled")])
}
/// The job's `jobs.list` entry.
async fn job_state(&self, job: u64) -> Option<Value> {
let jobs = self.run("jobs.list", json!({})).await.ok()?;
jobs.as_array()?.iter().find(|j| j["id"].as_u64() == Some(job)).cloned()
}
}
/// Delete what an interrupted export wrote, and nothing else: the output file itself, the numbered
/// stills of an image sequence (`<stem><3+ digits>.<same extension>`, only when the output is a
/// PNG/TIFF/BMP sequence) and the caption sidecar
/// (`<stem>.srt` / `<stem>.vtt`), each only if it was written since `since`. Other files that
/// merely share the name's stem (`<stem>.aep`, `<stem>2.psd`, `<stem>1.mp4` beside a movie, …) are
/// never touched, even when they were saved during the export.
pub fn remove_partial(path: &str, since: SystemTime) {
let p = Path::new(path);
let (Some(dir), Some(stem), Some(file)) = (p.parent(), p.file_stem().and_then(|s| s.to_str()), p.file_name().and_then(|s| s.to_str())) else {
return;
};
let ext = p.extension().and_then(|s| s.to_str()).unwrap_or_default();
let dir = if dir.as_os_str().is_empty() { Path::new(".") } else { dir };
let Ok(entries) = std::fs::read_dir(dir) else { return };
// file times are coarser than the clock
let since = since.checked_sub(Duration::from_secs(1)).unwrap_or(since);
for e in entries.flatten() {
let name = e.file_name();
let Some(name) = name.to_str() else { continue };
if is_export_output(name, file, stem, ext) && e.metadata().is_ok_and(|m| m.is_file() && m.modified().is_ok_and(|t| t >= since)) {
let _ = std::fs::remove_file(e.path());
}
}
}
/// Whether `name` is a file an export to `file` (`stem` + `.ext`) writes.
fn is_export_output(name: &str, file: &str, stem: &str, ext: &str) -> bool {
if name == file {
return true;
}
let Some(rest) = name.strip_prefix(stem) else { return false };
if let Some(sidecar) = rest.strip_prefix('.') {
return sidecar.eq_ignore_ascii_case("srt") || sidecar.eq_ignore_ascii_case("vtt");
}
// image sequence: the frame number (at least three digits, like `image_sequence_path`), then the
// output's own extension; movies and audio are a single file, so numbered neighbours are not theirs
if !is_sequence_extension(ext) {
return false;
}
let digits = rest.len() - rest.trim_start_matches(|c: char| c.is_ascii_digit()).len();
digits >= 3 && rest.get(digits..).is_some_and(|r| r.strip_prefix('.').is_some_and(|x| x == ext))
}
/// Whether an export to a file with extension `ext` writes an image sequence (one still per frame).
fn is_sequence_extension(ext: &str) -> bool {
use filmcraft_engine::export::Format;
Format::from_name(ext).is_some_and(|f| matches!(f, Format::PngSequence | Format::TiffSequence | Format::BmpSequence))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn only_the_exports_own_files_are_removed() {
let dir = std::env::temp_dir().join(format!("fc-remove-partial-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let started = SystemTime::now();
let ours = ["trailer.png", "trailer000.png", "trailer123.png", "trailer.srt", "trailer.vtt"];
// same stem, written during the export, but not the export's
let theirs = ["trailer.aep", "trailer2.psd", "trailer.docx", "trailer000.jpg", "trailers.png", "trailer.png.bak", "other.png"];
for f in ours.iter().chain(theirs.iter()) {
std::fs::write(dir.join(f), b"x").unwrap();
}
remove_partial(&dir.join("trailer.png").to_string_lossy(), started);
for f in ours {
assert!(!dir.join(f).exists(), "{f} should be removed");
}
for f in theirs {
assert!(dir.join(f).exists(), "{f} must not be removed");
}
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn numbered_files_beside_a_movie_are_kept() {
let dir = std::env::temp_dir().join(format!("fc-remove-partial-movie-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let started = SystemTime::now();
// a movie export writes one file: `clip1.mp4` and `clip002.mp4` are other exports
let ours = ["clip.mp4", "clip.srt"];
let theirs = ["clip1.mp4", "clip002.mp4", "clip2.mov"];
for f in ours.iter().chain(theirs.iter()) {
std::fs::write(dir.join(f), b"x").unwrap();
}
remove_partial(&dir.join("clip.mp4").to_string_lossy(), started);
for f in ours {
assert!(!dir.join(f).exists(), "{f} should be removed");
}
for f in theirs {
assert!(dir.join(f).exists(), "{f} must not be removed");
}
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn sequence_frames_have_at_least_three_digits() {
let dir = std::env::temp_dir().join(format!("fc-remove-partial-digits-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let started = SystemTime::now();
// frames are numbered with at least three digits (`shot000.tiff`); `shot2.tiff` is not one
let ours = ["shot.tiff", "shot000.tiff", "shot1234.tiff"];
let theirs = ["shot2.tiff", "shot12.tiff"];
for f in ours.iter().chain(theirs.iter()) {
std::fs::write(dir.join(f), b"x").unwrap();
}
remove_partial(&dir.join("shot.tiff").to_string_lossy(), started);
for f in ours {
assert!(!dir.join(f).exists(), "{f} should be removed");
}
for f in theirs {
assert!(dir.join(f).exists(), "{f} must not be removed");
}
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn files_older_than_the_export_are_kept() {
let dir = std::env::temp_dir().join(format!("fc-remove-partial-old-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
std::fs::write(dir.join("clip.mp4"), b"x").unwrap();
remove_partial(&dir.join("clip.mp4").to_string_lossy(), SystemTime::now() + Duration::from_secs(60));
assert!(dir.join("clip.mp4").exists());
let _ = std::fs::remove_dir_all(&dir);
}
}
+610 -37
View File
@@ -1,6 +1,6 @@
//! The MCP server. Tool names are snake_case.
use std::sync::{Arc, Mutex};
use std::sync::{Arc, Mutex, PoisonError};
use base64::Engine as _;
use filmcraft_engine::Session;
@@ -19,6 +19,19 @@ pub enum Backend {
Bridge(Arc<BridgeClient>),
}
/// A panic payload as text.
fn panic_message(p: Box<dyn std::any::Any + Send>) -> String {
p.downcast_ref::<&str>().map(|s| s.to_string()).or_else(|| p.downcast_ref::<String>().cloned()).unwrap_or_else(|| "panic".into())
}
/// A failed blocking task: a panic is reported as an internal error and the session stays usable.
fn join_error(e: tokio::task::JoinError) -> AutomationError {
match e.try_into_panic() {
Ok(p) => AutomationError::Other(format!("internal error: {}", panic_message(p))),
Err(e) => AutomationError::Other(e.to_string()),
}
}
#[derive(Clone)]
pub struct FilmcraftMcp {
backend: Arc<Backend>,
@@ -44,6 +57,24 @@ pub struct RunParams {
pub params: Option<Value>,
}
#[derive(Debug, Deserialize, JsonSchema)]
pub struct BatchStep {
/// Command id.
pub id: String,
/// Parameters object.
#[serde(default)]
pub params: Option<Value>,
}
#[derive(Debug, Deserialize, JsonSchema)]
pub struct BatchParams {
/// Commands to run in order.
pub steps: Vec<BatchStep>,
/// Stop at the first failing step (default true).
#[serde(default)]
pub stop_on_error: Option<bool>,
}
#[derive(Debug, Default, Deserialize, JsonSchema)]
pub struct RenderParams {
/// Timeline time in seconds (default: the playhead).
@@ -148,9 +179,60 @@ impl FilmcraftMcp {
}
pub async fn serve_stdio(self) -> Result<(), AutomationError> {
let running = self.serve(rmcp::transport::stdio()).await.map_err(|e| AutomationError::Other(format!("MCP init: {e}")))?;
running.waiting().await.map_err(|e| AutomationError::Other(e.to_string()))?;
Ok(())
self.serve_io(tokio::io::stdin(), tokio::io::stdout()).await
}
/// Serve MCP over any line stream. A line that isn't JSON gets a JSON-RPC parse error
/// (`-32700`, id null) instead of being dropped silently; the rest go to the rmcp service. One
/// writer task owns `output`, so replies never interleave.
pub async fn serve_io(
self,
input: impl tokio::io::AsyncRead + Unpin + Send + 'static,
output: impl tokio::io::AsyncWrite + Unpin + Send + 'static,
) -> Result<(), AutomationError> {
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
let (mut to_service, service_in) = tokio::io::duplex(1 << 20);
let (service_out, from_service) = tokio::io::duplex(1 << 20);
let (err_tx, mut err_rx) = tokio::sync::mpsc::unbounded_channel::<String>();
let reader = tokio::spawn(async move {
let mut lines = BufReader::new(input).lines();
while let Ok(Some(line)) = lines.next_line().await {
if line.trim().is_empty() {
continue;
}
match serde_json::from_str::<Value>(&line) {
Ok(_) => {
if to_service.write_all(format!("{line}\n").as_bytes()).await.is_err() {
break;
}
}
Err(e) => {
let reply = json!({"jsonrpc": "2.0", "id": null, "error": {"code": -32700, "message": format!("parse error: {e}")}});
let _ = err_tx.send(reply.to_string());
}
}
}
// EOF: closing the pipe ends the service
let _ = to_service.shutdown().await;
});
let writer = tokio::spawn(async move {
let mut output = output;
let mut replies = BufReader::new(from_service).lines();
loop {
let line = tokio::select! {
l = replies.next_line() => match l { Ok(Some(l)) => l, _ => break },
Some(e) = err_rx.recv() => e,
};
if output.write_all(format!("{line}\n").as_bytes()).await.is_err() || output.flush().await.is_err() {
break;
}
}
});
let running = self.serve((service_in, service_out)).await.map_err(|e| AutomationError::Other(format!("MCP init: {e}")))?;
let r = running.waiting().await.map_err(|e| AutomationError::Other(e.to_string()));
reader.abort();
let _ = writer.await;
r.map(|_| ())
}
/// Run an engine command on whichever backend.
@@ -160,16 +242,24 @@ impl FilmcraftMcp {
let s = s.clone();
let id = id.to_string();
tokio::task::spawn_blocking(move || {
let mut g = s.lock().map_err(|_| AutomationError::Other("session lock poisoned".into()))?;
// A command that panicked earlier poisoned the lock; the session is still usable.
let mut g = s.lock().unwrap_or_else(PoisonError::into_inner);
g.execute(&id, params).map_err(AutomationError::from)
})
.await
.map_err(|e| AutomationError::Other(e.to_string()))?
.map_err(join_error)?
}
Backend::Bridge(b) => b.call("engine.execute", json!({"command": id, "params": params})).await,
Backend::Bridge(b) => b.execute(id, params).await,
}
}
/// The project tree plus the active sequence (null when there is none).
async fn document(&self) -> Result<Value, AutomationError> {
let project = self.run("project.inspect", json!({})).await?;
let sequence = self.run("sequence.inspect", json!({})).await.ok();
Ok(json!({"project": project, "sequence": sequence}))
}
fn bridge_client(&self) -> Option<Arc<BridgeClient>> {
match &*self.backend {
Backend::Bridge(b) => Some(b.clone()),
@@ -190,7 +280,9 @@ impl FilmcraftMcp {
#[tool_router]
impl FilmcraftMcp {
#[tool(
description = "List every command (id, label, menu path, shortcut, parameter doc, enabled now). Menus, shortcuts, panels and timeline gestures all map to these ids."
title = "List commands",
description = "List every command (id, label, menu path, shortcut, parameter doc, enabled now). Menus, shortcuts, panels and timeline gestures all map to these ids.",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn command_list(&self, Parameters(p): Parameters<ListParams>) -> Result<CallToolResult, McpError> {
let r = self.run("command.list", json!({})).await.map(|v| {
@@ -213,76 +305,167 @@ impl FilmcraftMcp {
}
#[tool(
description = "Run a command by id with JSON params, e.g. {\"id\":\"timeline.razor\",\"params\":{\"seconds\":3.5}} or {\"id\":\"effects.apply\",\"params\":{\"effect\":\"Gaussian Blur\"}}. Undoable edits land in History."
title = "Run a command",
description = "Run a command by id with JSON params, e.g. {\"id\":\"timeline.razor\",\"params\":{\"seconds\":3.5}} or {\"id\":\"effects.apply\",\"params\":{\"effect\":\"Gaussian Blur\"}}. Undoable edits land in History.",
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
)]
async fn command_run(&self, Parameters(p): Parameters<RunParams>) -> Result<CallToolResult, McpError> {
wrap(self.run(&p.id, p.params.unwrap_or(json!({}))).await)
}
#[tool(description = "Project tree (bins and items with ids, types, durations), active sequence, source clip.")]
#[tool(
title = "Run several commands",
description = "Run several commands in order: {steps: [{id, params?}], stop_on_error?: true}. Returns {completed, failed, results: [{ok, result | error}]}; each edit is its own undo step.",
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
)]
async fn command_batch(&self, Parameters(p): Parameters<BatchParams>) -> Result<CallToolResult, McpError> {
let stop = p.stop_on_error.unwrap_or(true);
let (mut completed, mut failed, mut results) = (0u32, 0u32, Vec::new());
for st in p.steps {
match self.run(&st.id, st.params.unwrap_or(json!({}))).await {
Ok(v) => {
completed += 1;
results.push(json!({"ok": true, "result": v}));
}
Err(e) => {
failed += 1;
results.push(json!({"ok": false, "error": e.to_string()}));
if stop {
break;
}
}
}
}
let v = json!({"completed": completed, "failed": failed, "results": results});
Ok(if failed > 0 { CallToolResult::error(vec![Content::text(v.to_string())]) } else { ok_json(&v) })
}
#[tool(
title = "Inspect the document",
description = "The project as JSON: the tree (as project_inspect) plus the active sequence (as sequence_inspect; null when there is none).",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn doc_inspect(&self) -> Result<CallToolResult, McpError> {
wrap(self.document().await)
}
#[tool(
title = "Render a preview",
description = "The active sequence's frame at `seconds` (default: the playhead) as PNG, same as render_frame (bridge mode: Program monitor screenshot).",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn render_preview(&self, p: Parameters<RenderParams>) -> Result<CallToolResult, McpError> {
self.render_frame(p).await
}
#[tool(
title = "Inspect the project tree",
description = "Project tree (bins and items with ids, types, durations), active sequence, source clip.",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn project_inspect(&self) -> Result<CallToolResult, McpError> {
wrap(self.run("project.inspect", json!({})).await)
}
#[tool(
description = "The active sequence as JSON: settings, tracks, clips (ids, start/duration in ticks and frames, effects), transitions, markers, in/out, playhead, selection. 254016000000 ticks = 1 second."
title = "Inspect the active sequence",
description = "The active sequence as JSON: settings, tracks, clips (ids, start/duration in ticks and frames, effects), transitions, markers, in/out, playhead, selection. 254016000000 ticks = 1 second.",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn sequence_inspect(&self) -> Result<CallToolResult, McpError> {
wrap(self.run("sequence.inspect", json!({})).await)
}
#[tool(description = "Import media files by absolute path (MP4/MOV, WAV/MP3/FLAC/AIFF/Ogg, PNG/JPEG/…).")]
#[tool(
title = "Import media",
description = "Import media files by absolute path (MP4/MOV, WAV/MP3/FLAC/AIFF/Ogg, PNG/JPEG/…).",
annotations(read_only_hint = false, destructive_hint = false, idempotent_hint = false, open_world_hint = false)
)]
async fn media_import(&self, Parameters(p): Parameters<TypeParams>) -> Result<CallToolResult, McpError> {
let paths: Vec<&str> = p.text.split('\n').map(str::trim).filter(|s| !s.is_empty()).collect();
wrap(self.run("file.import", json!({"paths": paths})).await)
}
#[tool(description = "Render the active sequence's frame (at `seconds` or the playhead) and return it as PNG.")]
#[tool(
title = "Render a frame",
description = "Render the active sequence's frame (at `seconds` or the playhead) and return it as PNG.",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn render_frame(&self, Parameters(p): Parameters<RenderParams>) -> Result<CallToolResult, McpError> {
let max = p.max_side.unwrap_or(960);
match &*self.backend {
Backend::Headless(s) => {
let s = s.clone();
let r = tokio::task::spawn_blocking(move || -> Result<Vec<u8>, AutomationError> {
let mut g = s.lock().map_err(|_| AutomationError::Other("lock".into()))?;
if let Some(sec) = p.seconds {
g.set_playhead(filmcraft_time::Tick::from_seconds_f64(sec));
}
let g = s.lock().unwrap_or_else(PoisonError::into_inner);
// read-only: render at `seconds` without moving the playhead (#114)
let t = p.seconds.map_or_else(|| g.playhead(), filmcraft_time::Tick::from_seconds_f64);
let (w, h) =
g.active_sequence().map(|q| (q.settings.width, q.settings.height)).ok_or_else(|| AutomationError::Other("no sequence".into()))?;
let scale = (max as f32 / w.max(h) as f32).min(1.0);
let img = g.render_program(scale).ok_or_else(|| AutomationError::Other("no sequence".into()))?;
let img = g.try_render_program_at(scale, t).map_err(|e| AutomationError::Other(e.to_string()))?;
png_rgba(img.w as u32, img.h as u32, img.over_black_rgba8(), max)
})
.await
.map_err(|e| McpError::internal_error(e.to_string(), None))?;
.unwrap_or_else(|e| Err(join_error(e)));
Ok(match r {
Ok(png) => png_result(&png, "rendered frame"),
Err(e) => fail(e),
})
}
Backend::Bridge(b) => {
if let Some(sec) = p.seconds
&& let Err(e) = b.call("engine.execute", json!({"command": "playhead.set", "params": {"seconds": sec}})).await
{
let Some(sec) = p.seconds else {
return self.screenshot(&b.clone(), Some("Program".into()), max).await;
};
// The Program monitor shows the playhead's frame, so the live playhead has to move
// for the screenshot; put it (and the selection, which may follow it) back after,
// so the tool stays read-only (#114).
let before = match self.run("sequence.inspect", json!({})).await {
Ok(v) => v,
Err(e) => return Ok(fail(e)),
};
if let Err(e) = self.run("playhead.set", json!({"seconds": sec})).await {
return Ok(fail(e));
}
self.screenshot(&b.clone(), Some("Program".into()), max).await
let shot = self.screenshot(&b.clone(), Some("Program".into()), max).await;
if let Some(t) = before.get("playhead").and_then(Value::as_i64) {
let _ = self.run("playhead.set", json!({"time": t})).await;
}
// only when it changed: selecting also leaves trim mode
let now = self.run("sequence.inspect", json!({})).await.ok();
if let Some(sel) = before.get("selection").filter(|v| v.is_array())
&& now.as_ref().and_then(|n| n.get("selection")) != Some(sel)
{
let _ = self.run("timeline.select", json!({"clips": sel})).await;
}
shot
}
}
}
#[tool(description = "Live app: UI state (tool, workspace, panels, timeline zoom, playback, selection, fps).")]
#[tool(
title = "Live app: UI state",
description = "Live app: UI state (tool, workspace, panels, timeline zoom, playback, selection, fps).",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn ui_inspect(&self) -> Result<CallToolResult, McpError> {
self.ui("ui.inspect", json!({})).await
}
#[tool(description = "Live app: every on-screen interactive element with id, label and rect (points). Use ids with ui_click/ui_drag.")]
#[tool(
title = "Live app: on-screen elements",
description = "Live app: every on-screen interactive element with id, label and rect (points). Use ids with ui_click/ui_drag.",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn ui_elements(&self, Parameters(p): Parameters<ElementsParams>) -> Result<CallToolResult, McpError> {
self.ui("ui.elements", json!({"prefix": p.prefix.unwrap_or_default()})).await
}
#[tool(description = "Live app: click an element by id or at x,y (points). Supports right/middle button, double-click (count=2) and modifiers.")]
#[tool(
title = "Live app: click",
description = "Live app: click an element by id or at x,y (points). Supports right/middle button, double-click (count=2) and modifiers.",
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
)]
async fn ui_click(&self, Parameters(p): Parameters<ClickParams>) -> Result<CallToolResult, McpError> {
let mut v = json!({"button": p.button, "count": p.count, "modifiers": p.modifiers});
if let Some(id) = p.id {
@@ -295,23 +478,37 @@ impl FilmcraftMcp {
}
#[tool(
description = "Live app: press-drag-release from one point/element to another (move clips, trim edges, scrub, resize panels, drop project items onto tracks)."
title = "Live app: drag",
description = "Live app: press-drag-release from one point/element to another (move clips, trim edges, scrub, resize panels, drop project items onto tracks).",
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
)]
async fn ui_drag(&self, Parameters(p): Parameters<DragParams>) -> Result<CallToolResult, McpError> {
self.ui("ui.drag", json!({"from": p.from, "to": p.to, "steps": p.steps, "modifiers": p.modifiers})).await
}
#[tool(description = "Live app: press a key or shortcut (e.g. `Space`, `L`, `Cmd+K`, `Shift+Delete`, `Cmd+Z`).")]
#[tool(
title = "Live app: press a key",
description = "Live app: press a key or shortcut (e.g. `Space`, `L`, `Cmd+K`, `Shift+Delete`, `Cmd+Z`).",
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
)]
async fn ui_key(&self, Parameters(p): Parameters<KeyParams>) -> Result<CallToolResult, McpError> {
self.ui("ui.key", json!({"key": p.key})).await
}
#[tool(description = "Live app: type text into the focused field.")]
#[tool(
title = "Live app: type text",
description = "Live app: type text into the focused field.",
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
)]
async fn ui_type(&self, Parameters(p): Parameters<TypeParams>) -> Result<CallToolResult, McpError> {
self.ui("ui.type", json!({"text": p.text})).await
}
#[tool(description = "Live app: screenshot of the window or one panel (PNG).")]
#[tool(
title = "Live app: screenshot",
description = "Live app: screenshot of the window or one panel (PNG).",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn ui_screenshot(&self, Parameters(p): Parameters<ShotParams>) -> Result<CallToolResult, McpError> {
match self.bridge_client() {
Some(b) => self.screenshot(&b, p.panel, p.max_side.unwrap_or(1600)).await,
@@ -320,7 +517,9 @@ impl FilmcraftMcp {
}
#[tool(
description = "Live app: call any control-channel method directly (ui.set, ui.scroll, ui.timeline.hit, ui.timeline.locate, ui.playback, ui.panel.show, ui.menu.list, …)."
title = "Live app: control-channel call",
description = "Live app: call any control-channel method directly (ui.set, ui.scroll, ui.timeline.hit, ui.timeline.locate, ui.playback, ui.panel.show, ui.menu.list, …).",
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
)]
async fn ui_control(&self, Parameters(p): Parameters<ControlParams>) -> Result<CallToolResult, McpError> {
self.ui(&p.method, p.params.unwrap_or(json!({}))).await
@@ -350,12 +549,104 @@ impl FilmcraftMcp {
}
}
#[tool_handler(
router = self.tool_router,
name = "filmcraft",
instructions = "FilmCraft video editor (Premiere Pro-class). Every edit is an engine command: `command_list` to discover ids/params, `command_run` to execute (undoable). `project_inspect`/`sequence_inspect` return ids you can pass to commands; `render_frame` shows the result. In bridge mode the `ui_*` tools drive the live app: `ui_elements` lists clickable ids, `ui_click`/`ui_drag`/`ui_key` operate it, `ui_screenshot` shows it. Time is in ticks: 254016000000 per second (commands also accept `seconds`, `frame` or `timecode`)."
)]
impl ServerHandler for FilmcraftMcp {}
impl FilmcraftMcp {
/// The first argument `tool` doesn't take, as a message naming the ones it does.
fn unknown_arg(&self, tool: &str, args: Option<&rmcp::model::JsonObject>) -> Option<String> {
let def = self.tool_router.get(tool)?;
let props = def.input_schema.get("properties").and_then(Value::as_object).cloned().unwrap_or_default();
let bad = args?.keys().find(|k| !props.contains_key(*k))?;
let accepted: Vec<&str> = props.keys().map(String::as_str).collect();
let accepted = if accepted.is_empty() { "no arguments".to_string() } else { accepted.join(", ") };
Some(format!("unknown argument \"{bad}\" for {tool}; expected: {accepted}"))
}
}
const INSTRUCTIONS: &str = "FilmCraft video editor (Premiere Pro-class). Every edit is an engine command: `command_list` to discover ids/params, `command_run` to execute (undoable; `command_batch` runs several). `doc_inspect` (or `project_inspect`/`sequence_inspect`) returns ids you can pass to commands; `render_preview` shows the result. In bridge mode the `ui_*` tools drive the live app: `ui_elements` lists clickable ids, `ui_click`/`ui_drag`/`ui_key` operate it, `ui_screenshot` shows it. Time is in ticks: 254016000000 per second (commands also accept `seconds`, `frame` or `timecode`).";
/// Resources: the project (as `doc_inspect`) and the command catalog (as `command_list`).
const DOCUMENT_URI: &str = "filmcraft://document";
const COMMANDS_URI: &str = "filmcraft://commands";
/// MCP 2026-07-28 clients (current Claude Code) require `ttlMs` and `cacheScope` on list and read
/// results; rmcp fills `resultType` and leaves the cache hints to the server, as the generated
/// `tools/list` does. The resource list never changes; the document does, so reads aren't cached.
fn cache_hints(context: &rmcp::service::RequestContext<rmcp::RoleServer>, ttl_ms: u64) -> Option<(u64, rmcp::model::CacheScope)> {
context.protocol_version().is_some_and(|v| v >= rmcp::model::ProtocolVersion::V_2026_07_28).then_some((ttl_ms, rmcp::model::CacheScope::Private))
}
#[tool_handler(router = self.tool_router)]
impl ServerHandler for FilmcraftMcp {
fn get_info(&self) -> rmcp::model::ServerConfig {
rmcp::model::ServerConfig::new(rmcp::model::ServerCapabilities::builder().enable_tools().enable_resources().build())
.with_server_info(rmcp::model::Implementation::new("filmcraft", env!("CARGO_PKG_VERSION")))
.with_instructions(INSTRUCTIONS.to_string())
}
/// Strict arguments and long exports in front of the tool router.
async fn call_tool(
&self,
request: rmcp::model::CallToolRequestParams,
context: rmcp::service::RequestContext<rmcp::RoleServer>,
) -> Result<rmcp::model::CallToolResponse, McpError> {
if let Some(m) = self.unknown_arg(&request.name, request.arguments.as_ref()) {
return Err(McpError::invalid_params(m, None));
}
// A blocking export reports progress and can be cancelled (docs/agents.md § Long exports).
// Bridge mode too: the app runs it as a job, so it stays responsive (#91, #92).
if request.name == "command_run"
&& let Some(a) = &request.arguments
&& let Some(id) = a.get("id").and_then(Value::as_str)
{
let params = a.get("params").cloned().filter(Value::is_object).unwrap_or_else(|| json!({}));
if crate::long_job::is_long(id, &params) {
return Ok(self.run_long(id, params, &context).await.into());
}
}
let tcc = rmcp::handler::server::tool::ToolCallContext::new(self, request, context);
self.tool_router.call(tcc).await
}
async fn list_resources(
&self,
_request: Option<rmcp::model::PaginatedRequestParams>,
context: rmcp::service::RequestContext<rmcp::RoleServer>,
) -> Result<rmcp::model::ListResourcesResult, McpError> {
use rmcp::model::Resource;
let mut r = rmcp::model::ListResourcesResult::with_all_items(vec![
Resource::new(DOCUMENT_URI, "document")
.with_title("Project")
.with_description("The project tree and the active sequence (same as doc_inspect).")
.with_mime_type("application/json"),
Resource::new(COMMANDS_URI, "commands")
.with_title("Command catalog")
.with_description("Every command with id, label, menu path, shortcut, params and enabled state (same as command_list).")
.with_mime_type("application/json"),
]);
if let Some((ttl, scope)) = cache_hints(&context, 600_000) {
r = r.with_ttl_ms(ttl).with_cache_scope(scope);
}
Ok(r)
}
async fn read_resource(
&self,
request: rmcp::model::ReadResourceRequestParams,
context: rmcp::service::RequestContext<rmcp::RoleServer>,
) -> Result<rmcp::model::ReadResourceResponse, McpError> {
let v = match request.uri.as_str() {
DOCUMENT_URI => self.document().await,
COMMANDS_URI => self.run("command.list", json!({})).await,
other => return Err(McpError::resource_not_found(format!("resource not found: {other}"), None)),
}
.map_err(|e| McpError::internal_error(e.to_string(), None))?;
let text = serde_json::to_string_pretty(&v).map_err(|e| McpError::internal_error(e.to_string(), None))?;
let mut r = rmcp::model::ReadResourceResult::new(vec![rmcp::model::ResourceContents::text(text, request.uri).with_mime_type("application/json")]);
if let Some((ttl, scope)) = cache_hints(&context, 0) {
r = r.with_ttl_ms(ttl).with_cache_scope(scope);
}
Ok(r.into())
}
}
#[cfg(test)]
mod tests {
@@ -371,4 +662,286 @@ mod tests {
let r = m.render_frame(Parameters(RenderParams { seconds: Some(1.0), max_side: Some(320) })).await.unwrap();
assert_ne!(r.is_error, Some(true));
}
/// `render_frame` / `render_preview` are annotated read-only (#114): rendering at `seconds`
/// leaves the playhead, and the selection that follows it, where they were, and still renders
/// the frame at `seconds`.
#[tokio::test]
async fn rendering_at_a_time_does_not_move_the_playhead() {
let m = FilmcraftMcp::headless(demo());
m.run("sequence.selectionFollowsPlayhead", json!({"on": true})).await.unwrap();
m.run("playhead.set", json!({"seconds": 1.0})).await.unwrap();
let before = m.run("sequence.inspect", json!({})).await.unwrap();
let png = |r: CallToolResult| serde_json::to_string(&r.content).unwrap();
let at_playhead = png(m.render_frame(Parameters(RenderParams { seconds: None, max_side: Some(64) })).await.unwrap());
for seconds in [5.0, 8.0] {
let r = m.render_frame(Parameters(RenderParams { seconds: Some(seconds), max_side: Some(64) })).await.unwrap();
assert_ne!(png(r), at_playhead, "renders the frame at {seconds} s");
m.render_preview(Parameters(RenderParams { seconds: Some(seconds), max_side: Some(64) })).await.unwrap();
}
let after = m.run("sequence.inspect", json!({})).await.unwrap();
assert_eq!(after["playhead"], before["playhead"]);
assert_eq!(after["selection"], before["selection"]);
}
/// One request/response exchange over `serve_io`, as an MCP client sees it.
struct Client {
tx: tokio::io::DuplexStream,
rx: tokio::io::Lines<tokio::io::BufReader<tokio::io::DuplexStream>>,
}
impl Client {
fn start(s: Session) -> Self {
use tokio::io::AsyncBufReadExt;
let (tx, server_in) = tokio::io::duplex(1 << 22);
let (server_out, rx) = tokio::io::duplex(1 << 22);
tokio::spawn(FilmcraftMcp::headless(s).serve_io(server_in, server_out));
Self { tx, rx: tokio::io::BufReader::new(rx).lines() }
}
async fn send(&mut self, line: &str) {
use tokio::io::AsyncWriteExt;
self.tx.write_all(format!("{line}\n").as_bytes()).await.unwrap();
}
async fn next(&mut self) -> Value {
serde_json::from_str(&self.rx.next_line().await.unwrap().unwrap()).unwrap()
}
async fn ask(&mut self, msg: Value) -> Value {
self.send(&msg.to_string()).await;
self.next().await
}
async fn init(&mut self) -> Value {
let r = self.ask(json!({"jsonrpc":"2.0","id":0,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"t","version":"1"}}})).await;
self.send(&json!({"jsonrpc":"2.0","method":"notifications/initialized"}).to_string()).await;
r
}
async fn call(&mut self, id: u64, name: &str, arguments: Value) -> Value {
self.ask(json!({"jsonrpc":"2.0","id":id,"method":"tools/call","params":{"name":name,"arguments":arguments}})).await
}
}
fn demo() -> Session {
let mut s = Session::default();
s.execute("file.openDemoProject", json!({})).unwrap();
s
}
/// Every listed tool has a title and the four hints, and the core tools are listed.
#[tokio::test(flavor = "multi_thread")]
async fn tools_are_titled_and_annotated() {
let mut c = Client::start(demo());
let init = c.init().await;
assert_eq!(init["result"]["serverInfo"]["name"], "filmcraft");
let tools = c.ask(json!({"jsonrpc":"2.0","id":1,"method":"tools/list"})).await["result"]["tools"].as_array().cloned().unwrap();
for t in &tools {
assert!(t["title"].is_string(), "{} has no title", t["name"]);
for h in ["readOnlyHint", "destructiveHint", "idempotentHint", "openWorldHint"] {
assert!(t["annotations"][h].is_boolean(), "{} lacks {h}", t["name"]);
}
}
let names: Vec<&str> = tools.iter().filter_map(|t| t["name"].as_str()).collect();
for want in ["command_list", "command_run", "command_batch", "doc_inspect", "render_preview", "project_inspect", "render_frame", "ui_click"] {
assert!(names.contains(&want), "{want} not listed");
}
let ro = |n: &str| tools.iter().find(|t| t["name"] == n).map(|t| t["annotations"]["readOnlyHint"].clone());
assert_eq!((ro("command_list"), ro("command_run")), (Some(json!(true)), Some(json!(false))));
}
/// Strict arguments, malformed JSON, batch, doc_inspect and render_preview over the raw line
/// protocol.
#[tokio::test(flavor = "multi_thread")]
async fn strict_arguments_and_parse_errors() {
let mut c = Client::start(demo());
c.init().await;
let r = c.call(1, "command_list", json!({"filter": "razor", "enabled_only": true})).await;
assert!(r["result"]["content"][0]["text"].as_str().unwrap().contains("timeline.razor"), "{r}");
// an unknown argument names itself and the accepted ones
let r = c.call(2, "command_list", json!({"filtr": "x"})).await;
assert_eq!(r["error"]["code"], -32602, "{r}");
let m = r["error"]["message"].as_str().unwrap();
assert!(m.contains("\"filtr\"") && m.contains("filter") && m.contains("enabled_only"), "{m}");
let r = c.call(3, "doc_inspect", json!({"verbose": true})).await;
assert_eq!(r["error"]["code"], -32602, "{r}");
// command params stay free-form
let r = c.call(4, "command_run", json!({"id": "playhead.set", "params": {"seconds": 0.5}})).await;
assert_eq!(r["result"]["isError"], false, "{r}");
// malformed JSON: a parse error, and the server keeps serving
c.send("{\"jsonrpc\": \"2.0\", \"id\": 5, BROKEN").await;
let r = c.next().await;
assert_eq!((r["error"]["code"].as_i64(), r["id"].is_null()), (Some(-32700), true), "{r}");
let r = c.ask(json!({"jsonrpc":"2.0","id":6,"method":"ping"})).await;
assert!(r["result"].is_object(), "{r}");
// batch: stops at the first failure and reports both
let r =
c.call(7, "command_batch", json!({"steps": [{"id": "playhead.set", "params": {"seconds": 0.5}}, {"id": "no.such"}, {"id": "playhead.set"}]})).await;
assert_eq!(r["result"]["isError"], true, "{r}");
let v: Value = serde_json::from_str(r["result"]["content"][0]["text"].as_str().unwrap()).unwrap();
assert_eq!((v["completed"].as_u64(), v["failed"].as_u64(), v["results"].as_array().map(Vec::len)), (Some(1), Some(1), Some(2)), "{v}");
let r = c.call(8, "doc_inspect", json!({})).await;
let v: Value = serde_json::from_str(r["result"]["content"][0]["text"].as_str().unwrap()).unwrap();
assert!(v["project"].is_object() && v["sequence"].is_object(), "{v}");
let r = c.call(9, "render_preview", json!({"max_side": 64})).await;
assert_eq!(r["result"]["content"][0]["type"], "image", "{r}");
}
/// `filmcraft://document` and `filmcraft://commands` are listed and readable as JSON.
#[tokio::test(flavor = "multi_thread")]
async fn resources_document_and_commands() {
let mut c = Client::start(demo());
let init = c.init().await;
assert!(init["result"]["capabilities"]["resources"].is_object(), "{init}");
assert!(init["result"]["instructions"].as_str().unwrap().contains("doc_inspect"), "{init}");
let r = c.ask(json!({"jsonrpc":"2.0","id":1,"method":"resources/list"})).await;
let uris: Vec<&str> = r["result"]["resources"].as_array().unwrap().iter().filter_map(|x| x["uri"].as_str()).collect();
assert_eq!(uris, [DOCUMENT_URI, COMMANDS_URI], "{r}");
let r = c.ask(json!({"jsonrpc":"2.0","id":2,"method":"resources/read","params":{"uri":DOCUMENT_URI}})).await;
assert_eq!(r["result"]["contents"][0]["mimeType"], "application/json", "{r}");
let v: Value = serde_json::from_str(r["result"]["contents"][0]["text"].as_str().unwrap()).unwrap();
assert!(v["project"].is_object() && v["sequence"].is_object(), "{v}");
let r = c.ask(json!({"jsonrpc":"2.0","id":3,"method":"resources/read","params":{"uri":COMMANDS_URI}})).await;
assert!(r["result"]["contents"][0]["text"].as_str().unwrap().contains("timeline.razor"), "{r}");
let r = c.ask(json!({"jsonrpc":"2.0","id":4,"method":"resources/read","params":{"uri":"filmcraft://nope"}})).await;
assert!(r["error"]["code"].is_i64(), "{r}");
}
/// MCP 2026-07-28 clients (current Claude Code) negotiate per request and reject list and read
/// results without `resultType`, `ttlMs` and `cacheScope`; sessions that negotiated an older
/// revision through `initialize` get the old shape.
#[tokio::test(flavor = "multi_thread")]
async fn modern_clients_get_result_type_and_cache_hints() {
let mut c = Client::start(Session::default());
let meta = json!({
"io.modelcontextprotocol/protocolVersion": "2026-07-28",
"io.modelcontextprotocol/clientInfo": {"name": "t", "version": "1"},
"io.modelcontextprotocol/clientCapabilities": {},
});
let r = c.ask(json!({"jsonrpc":"2.0","id":1,"method":"server/discover","params":{"_meta":meta}})).await;
assert!(r["result"]["supportedVersions"].as_array().unwrap().contains(&json!("2026-07-28")), "{r}");
for (id, method, params) in [
(2, "tools/list", json!({"_meta": meta})),
(3, "resources/list", json!({"_meta": meta})),
(4, "resources/read", json!({"_meta": meta, "uri": DOCUMENT_URI})),
] {
let r = c.ask(json!({"jsonrpc":"2.0","id":id,"method":method,"params":params})).await;
assert_eq!(r["result"]["resultType"], "complete", "{method}: {r}");
assert!(r["result"]["ttlMs"].is_u64(), "{method}: {r}");
assert!(matches!(r["result"]["cacheScope"].as_str(), Some("public" | "private")), "{method}: {r}");
}
let mut c = Client::start(Session::default());
c.init().await;
for (id, method, params) in [(2, "tools/list", json!({})), (3, "resources/list", json!({})), (4, "resources/read", json!({"uri": DOCUMENT_URI}))] {
let r = c.ask(json!({"jsonrpc":"2.0","id":id,"method":method,"params":params})).await;
assert!(r["result"].is_object() && r["result"].get("resultType").is_none() && r["result"].get("ttlMs").is_none(), "{method}: {r}");
}
}
/// docs/agents.md § Long exports: a blocking export reports `notifications/progress` for its
/// token (none without one), answers `ping` meanwhile, and `notifications/cancelled` stops
/// it, deletes the partial file and sends no response.
#[tokio::test(flavor = "multi_thread")]
async fn export_progress_and_cancel() {
let dir = std::env::temp_dir().join(format!("filmcraft-mcp-progress-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let mut c = Client::start(demo());
c.init().await;
let export = |id: u64, name: &str, token: Option<Value>, seconds: f64| {
let path = dir.join(name).to_string_lossy().to_string();
let params = json!({"path": path, "format": "h264", "width": 640, "height": 360, "audio": false, "range": "custom", "startSeconds": 0, "endSeconds": seconds, "wait": true});
let mut call =
json!({"jsonrpc":"2.0","id":id,"method":"tools/call","params":{"name":"command_run","arguments":{"id":"file.exportMedia","params":params}}});
if let Some(t) = token {
call["params"]["_meta"] = json!({"progressToken": t});
}
call.to_string()
};
// no token: no notifications, the same response as before
c.send(&export(1, "quiet.mp4", None, 0.5)).await;
let m = c.next().await;
assert_eq!((m["id"].as_u64(), &m["result"]["isError"]), (Some(1), &json!(false)), "{m}");
// progress, strictly increasing, with a total; ping answered while the export runs. The
// job is polled every 100 ms, so a fast machine finishes a short export within one poll:
// a longer one is tried until the export outlasts a few polls.
let (mut notes, mut pong) = (0, false);
for (attempt, seconds) in [1.0, 4.0, 16.0].into_iter().enumerate() {
let (id, ping, name) = (20 + attempt as u64, 30 + attempt as u64, format!("a{attempt}.mp4"));
c.send(&export(id, &name, Some(json!("tok")), seconds)).await;
let (mut last, mut pinged) = (-1.0, false);
(notes, pong) = (0, false);
let done = loop {
let m = c.next().await;
if m["method"] == "notifications/progress" {
assert_eq!(m["params"]["progressToken"], "tok", "{m}");
let p = m["params"]["progress"].as_f64().unwrap();
assert!(p > last && m["params"]["total"].as_f64().unwrap() >= p, "{m}");
last = p;
notes += 1;
if !pinged {
pinged = true;
c.send(&json!({"jsonrpc": "2.0", "id": ping, "method": "ping"}).to_string()).await;
}
} else if m["id"] == ping {
pong = true;
} else if m["id"] == id {
break m;
}
};
assert_eq!(done["result"]["isError"], false, "{done}");
assert!(notes >= 1, "a finished export reports its progress at least once");
assert!(dir.join(&name).exists());
if notes >= 2 && pong {
break;
}
// the ping's answer is still on its way: read it so it cannot be taken for a later reply
while pinged && !pong {
pong = c.next().await["id"] == ping;
}
pong = false;
}
assert!(notes >= 2 && pong, "{notes} notifications, ping answered during the export: {pong}");
// cancel after the first notification: no response, no partial file
c.send(&export(4, "b.mp4", Some(json!(7)), 20.0)).await;
loop {
let m = c.next().await;
assert_ne!(m["id"], 4, "finished before the cancel: {m}");
if m["method"] == "notifications/progress" {
break;
}
}
c.send(r#"{"jsonrpc":"2.0","method":"notifications/cancelled","params":{"requestId":4}}"#).await;
let t0 = std::time::Instant::now();
while dir.join("b.mp4").exists() || t0.elapsed() < std::time::Duration::from_millis(500) {
assert!(t0.elapsed().as_secs() < 30, "partial file still there");
tokio::time::sleep(std::time::Duration::from_millis(50)).await;
}
c.send(r#"{"jsonrpc":"2.0","id":5,"method":"tools/call","params":{"name":"command_run","arguments":{"id":"jobs.list"}}}"#).await;
let m = loop {
let m = c.next().await;
assert_ne!(m["id"], 4, "no response for a cancelled request: {m}");
if m["id"] == 5 {
break m;
}
};
let jobs: Value = serde_json::from_str(m["result"]["content"][0]["text"].as_str().unwrap()).unwrap();
let last = jobs.as_array().and_then(|a| a.last()).cloned().unwrap_or_default();
assert_eq!(last["result"]["error"], "cancelled", "{jobs}");
assert!(!dir.join("b.mp4").exists());
let _ = std::fs::remove_dir_all(&dir);
}
/// A command that panicked used to poison the session lock, and every later call failed with
/// "session lock poisoned" until the server was restarted.
#[tokio::test(flavor = "multi_thread")]
async fn a_poisoned_session_lock_keeps_serving() {
let m = FilmcraftMcp::headless(demo());
let Backend::Headless(s) = &*m.backend else { panic!("headless backend") };
let s = s.clone();
let _ = std::thread::spawn(move || {
let _g = s.lock().unwrap();
panic!("a command panicked");
})
.join();
let v = m.run("sequence.inspect", json!({})).await.unwrap();
assert!(v.is_object(), "{v}");
let r = m.render_frame(Parameters(RenderParams { seconds: Some(0.5), max_side: Some(64) })).await.unwrap();
assert_ne!(r.is_error, Some(true));
}
}
+70 -15
View File
@@ -184,28 +184,45 @@ struct Timing {
tick_rate: i64,
}
fn positive_integer(s: &str) -> Option<i64> {
let s = s.trim();
if s.is_empty() || !s.bytes().all(|b| b.is_ascii_digit()) {
return None;
}
s.parse::<i64>().ok().filter(|v| *v > 0)
}
fn parse_ratio(s: &str) -> Option<(i64, i64)> {
let mut it = s.split_whitespace().map(|x| x.parse::<i64>().ok());
let mut it = s.split_whitespace().map(positive_integer);
let a = it.next()??;
let b = it.next().flatten().unwrap_or(1);
(a > 0 && b > 0).then_some((a, b))
let b = it.next()??;
it.next().is_none().then_some((a, b))
}
fn attr<'a>(n: roxmltree::Node<'a, '_>, local: &str) -> Option<&'a str> {
n.attributes().find(|a| a.name() == local).map(|a| a.value())
}
fn timing(root: roxmltree::Node) -> Timing {
let base = attr(root, "frameRate").and_then(|v| v.trim().parse::<i64>().ok()).filter(|v| *v > 0).unwrap_or(30);
let (mn, md) = attr(root, "frameRateMultiplier").and_then(parse_ratio).unwrap_or((1, 1));
let rate = FrameRate::new(base * mn, md);
let sub = attr(root, "subFrameRate").and_then(|v| v.trim().parse::<i64>().ok()).filter(|v| *v > 0).unwrap_or(1);
let tick_rate = attr(root, "tickRate").and_then(|v| v.trim().parse::<i64>().ok()).filter(|v| *v > 0).unwrap_or(if attr(root, "frameRate").is_some() {
base * sub
} else {
1
});
Timing { rate, base, sub_frame_rate: sub, tick_rate }
fn timing(root: roxmltree::Node) -> Result<Timing> {
let invalid =
|name: &str| Error::Syntax { line: root.document().text_pos_at(root.range().start).row as usize, msg: format!("invalid TTML timing parameter {name}") };
let integer = |name: &str, default| match attr(root, name) {
Some(v) => positive_integer(v).ok_or_else(|| invalid(name)),
None => Ok(default),
};
let base = integer("frameRate", 30)?;
let (mn, md) = match attr(root, "frameRateMultiplier") {
Some(v) => parse_ratio(v).ok_or_else(|| invalid("frameRateMultiplier"))?,
None => (1, 1),
};
let rate = FrameRate::new(base.checked_mul(mn).ok_or_else(|| invalid("frameRateMultiplier"))?, md);
let sub = integer("subFrameRate", 1)?;
let tick_rate = match attr(root, "tickRate") {
Some(v) => positive_integer(v).ok_or_else(|| invalid("tickRate"))?,
None if attr(root, "frameRate").is_some() => base.checked_mul(sub).ok_or_else(|| invalid("subFrameRate"))?,
None => 1,
};
Ok(Timing { rate, base, sub_frame_rate: sub, tick_rate })
}
/// Seconds as an exact decimal string → ticks.
@@ -383,7 +400,7 @@ pub fn parse(text: &str) -> Result<Document> {
if root.tag_name().name() != "tt" {
return Err(Error::NotFormat("TTML"));
}
let tm = timing(root);
let tm = timing(root)?;
let mut styles = std::collections::HashMap::new();
for s in root.descendants().filter(|n| n.is_element() && n.tag_name().name() == "style") {
if let Some(id) = s.attributes().find(|a| a.name() == "id").map(|a| a.value()) {
@@ -478,6 +495,44 @@ mod tests {
Timing { rate, base: rate.timecode_base(), sub_frame_rate: 1, tick_rate: 1 }
}
#[test]
fn invalid_timing_parameters_return_an_error() {
for params in [
r#"ttp:frameRate="0""#,
r#"ttp:frameRate="-25""#,
r#"ttp:frameRate="""#,
r#"ttp:subFrameRate="0""#,
r#"ttp:tickRate="-1""#,
r#"ttp:frameRateMultiplier="1 0""#,
r#"ttp:frameRateMultiplier="1""#,
r#"ttp:frameRateMultiplier="1 2 3""#,
] {
let xml = format!(
r#"<tt xmlns="http://www.w3.org/ns/ttml" xmlns:ttp="http://www.w3.org/ns/ttml#parameter" {params}><body><div><p begin="1s" end="2s">Hello</p></div></body></tt>"#
);
assert!(crate::parse(xml.as_bytes(), crate::Format::Ttml).is_err(), "{params}");
}
}
#[test]
fn missing_timing_parameters_keep_defaults() {
for (params, begin, end) in [
("", "1s", "2s"),
(r#"ttp:frameRate="25""#, "25f", "50f"),
(r#"ttp:frameRate="30" ttp:subFrameRate="2""#, "60t", "120t"),
(r#"ttp:frameRate="30" ttp:subFrameRate="2" ttp:tickRate="10""#, "10t", "20t"),
(r#"ttp:frameRate="25" ttp:frameRateMultiplier="2 1""#, "50f", "100f"),
] {
let xml = format!(
r#"<tt xmlns="http://www.w3.org/ns/ttml" xmlns:ttp="http://www.w3.org/ns/ttml#parameter" {params}><body><div><p begin="{begin}" end="{end}">Hello</p></div></body></tt>"#
);
let doc = crate::parse(xml.as_bytes(), crate::Format::Ttml).unwrap();
assert_eq!(doc.cues.len(), 1, "{params}");
assert_eq!(doc.cues[0].start, Tick(TICKS_PER_SECOND), "{params}");
assert_eq!(doc.cues[0].end, Tick(2 * TICKS_PER_SECOND), "{params}");
}
}
#[test]
fn time_expressions() {
let t = tm(FrameRate::FPS_25);
+2
View File
@@ -28,6 +28,7 @@ filmcraft-vp9 = { workspace = true }
filmcraft-prores = { workspace = true }
filmcraft-dnx = { workspace = true }
filmcraft-av1 = { workspace = true }
filmcraft-apv = { workspace = true }
filmcraft-aac = { workspace = true }
filmcraft-opus = { workspace = true }
thiserror = { workspace = true }
@@ -40,3 +41,4 @@ symphonia = { version = "0.5.5", default-features = false, features = ["aac", "m
[dev-dependencies]
filmcraft-testkit = { workspace = true }
serde_json = { workspace = true }
+177
View File
@@ -0,0 +1,177 @@
//! Standalone APV raw bitstream source (RFC 9924 Appendix A: `.apv` elementary streams).
//!
//! Each access unit in a raw bitstream is prefixed by a 4-byte big-endian `au_size` followed by
//! the `'aPv1'` signature (unprefixed single-AU streams starting directly with `'aPv1'` are also
//! accepted). Every frame is intra-coded, so random access is direct.
use std::sync::Arc;
use filmcraft_color::{ColorInfo, Matrix, Range, Transfer};
use filmcraft_frame::{AudioBuffer, VideoFrame};
use filmcraft_media::{FrameRequest, MediaError, MediaInfo, MediaKind, MediaSource, SharedSource, VideoStreamInfo};
use filmcraft_time::{FrameRate, Tick};
use crate::CodecError;
use crate::gop::{GopCache, VideoSamples};
use crate::video::{ApvDecoder, VideoDecoder, primaries_from_code};
/// Sniff a raw APV bitstream: either a `raw_bitstream_access_unit()` (`[au_size: u32]['aPv1']`) or
/// an unprefixed `access_unit()` starting with `'aPv1'`.
pub fn sniff(head: &[u8]) -> bool {
(head.len() >= 8 && &head[4..8] == b"aPv1" && u32::from_be_bytes([head[0], head[1], head[2], head[3]]) >= 8) || head.starts_with(b"aPv1")
}
pub struct ApvSource {
info: MediaInfo,
bytes: crate::Src,
samples: Vec<(u64, u32)>,
rate: FrameRate,
video: GopCache,
}
pub(crate) fn apv_pixfmt_label(chroma: filmcraft_apv::ChromaFormat, bit_depth: u8) -> String {
let sub = match chroma {
filmcraft_apv::ChromaFormat::Monochrome => "Monochrome",
filmcraft_apv::ChromaFormat::Yuv422 => "YUV 4:2:2",
filmcraft_apv::ChromaFormat::Yuv444 => "YUV 4:4:4",
filmcraft_apv::ChromaFormat::Yuv4444 => "YUVA 4:4:4:4",
};
format!("{sub} {bit_depth}-bit")
}
impl ApvSource {
pub fn open(name: &str, bytes: Arc<[u8]>) -> crate::Result<Self> {
Self::open_reader(name, Arc::new(filmcraft_media::reader::MemReader(bytes)))
}
pub fn open_reader(name: &str, reader: filmcraft_media::SharedReader) -> crate::Result<Self> {
let len = reader.len();
let mut samples = Vec::new();
let mut pos = 0u64;
let mut hdr = [0u8; 8];
while pos + 8 <= len {
if reader.read_at(pos, &mut hdr).is_err() {
break;
}
if &hdr[4..8] == b"aPv1" {
let sz = u32::from_be_bytes([hdr[0], hdr[1], hdr[2], hdr[3]]);
if sz < 8 || pos + 4 + sz as u64 > len {
break;
}
samples.push((pos + 4, sz));
pos += 4 + sz as u64;
} else if pos == 0 && &hdr[0..4] == b"aPv1" && len <= u32::MAX as u64 {
samples.push((0, len as u32));
break;
} else {
break;
}
}
let Some(&(off0, sz0)) = samples.first() else {
return Err(CodecError::Container("APV: no valid access units".into()));
};
let first = filmcraft_media::reader::read_range(&*reader, off0, sz0 as usize).map_err(|e| CodecError::Container(e.to_string()))?;
let fh = filmcraft_apv::probe(&first).map_err(|e| CodecError::Decode(e.to_string()))?;
let mut color = ColorInfo { matrix: filmcraft_frame::default_matrix(fh.width, fh.height), ..ColorInfo::REC709 };
let mut explicit_color = None;
if fh.color_description_present {
if let Some(m) = Matrix::from_code(fh.color.matrix) {
color.matrix = m;
}
if let Some(t) = Transfer::from_code(fh.color.transfer) {
color.transfer = t;
}
if let Some(p) = primaries_from_code(fh.color.primaries) {
color.primaries = p;
}
if fh.color.full_range {
color.range = Range::Full;
}
explicit_color = Some(color);
}
let rate = FrameRate::FPS_24;
let duration = Tick::from_rational(samples.len() as i64 * rate.den, 1, rate.num);
let total_bits: u64 = samples.iter().map(|s| s.1 as u64 * 8).sum();
let secs = duration.seconds();
let bitrate = (secs > 0.0).then(|| (total_bits as f64 / secs) as u64);
let codec = match filmcraft_apv::Profile::from_idc(fh.profile_idc) {
Some(p) => p.name().into(),
None => "APV".into(),
};
let info = MediaInfo {
name: name.to_string(),
kind: MediaKind::Movie,
duration,
video: Some(VideoStreamInfo {
width: fh.width,
height: fh.height,
frame_rate: rate,
par: (1, 1),
codec,
pixel_format: apv_pixfmt_label(fh.chroma, fh.bit_depth),
color,
has_alpha: fh.chroma == filmcraft_apv::ChromaFormat::Yuv4444,
bitrate,
hdr: None,
}),
audio_streams: Vec::new(),
container: "APV".into(),
start_timecode: None,
file_size: Some(len),
};
Ok(Self { info, bytes: crate::Src(reader), samples, rate, video: GopCache::new(explicit_color) })
}
}
struct ApvVideo<'a>(&'a ApvSource);
impl VideoSamples for ApvVideo<'_> {
fn count(&self) -> usize {
self.0.samples.len()
}
fn pts(&self, i: usize) -> i64 {
i as i64
}
fn sync_before(&self, i: usize) -> usize {
i
}
fn sample_at(&self, t: i64) -> Option<usize> {
(!self.0.samples.is_empty()).then(|| t.clamp(0, self.0.samples.len() as i64 - 1) as usize)
}
fn read(&self, i: usize) -> crate::Result<Vec<u8>> {
let (off, sz) = self.0.samples.get(i).copied().ok_or_else(|| CodecError::Container("sample index out of range".into()))?;
filmcraft_media::reader::read_range(&*self.0.bytes.0, off, sz as usize).map_err(|e| CodecError::Container(e.to_string()))
}
fn make_decoder(&self) -> crate::Result<Box<dyn VideoDecoder>> {
Ok(Box::new(ApvDecoder))
}
}
impl MediaSource for ApvSource {
fn info(&self) -> &MediaInfo {
&self.info
}
fn video_frame(&self, req: FrameRequest) -> Result<Arc<VideoFrame>, MediaError> {
let frame_idx = self.rate.frame_at(req.time.max(Tick::ZERO));
Ok(self.video.frame(&ApvVideo(self), frame_idx)?)
}
fn audio(&self, _start: i64, _frames: usize, _sample_rate: u32) -> Result<AudioBuffer, MediaError> {
Err(MediaError::NoStream("audio"))
}
}
pub fn opener(name: &str, bytes: Arc<[u8]>) -> Option<Result<SharedSource, MediaError>> {
if !sniff(&bytes) {
return None;
}
Some(ApvSource::open(name, bytes).map(|s| Arc::new(s) as SharedSource).map_err(Into::into))
}
pub fn reader_opener(name: &str, head: &[u8], reader: &filmcraft_media::SharedReader) -> Option<Result<SharedSource, MediaError>> {
if !sniff(head) {
return None;
}
Some(ApvSource::open_reader(name, reader.clone()).map(|s| Arc::new(s) as SharedSource).map_err(Into::into))
}
+299 -13
View File
@@ -15,8 +15,8 @@ use symphonia::core::probe::Hint;
use crate::{CodecError, Result};
enum Inner {
/// Our own AAC-LC decoder.
Aac { dec: Box<filmcraft_aac::Decoder>, asc: Vec<u8> },
/// Our own AAC-LC decoder. `up` interpolates HE-AAC's core to the output rate (see [`Upsample2x`]).
Aac { dec: Box<filmcraft_aac::Decoder>, asc: Vec<u8>, up: Option<Box<Upsample2x>> },
/// AAC whose configuration arrives with the first frame (ADTS / LATM).
AacPending,
/// Our own Opus decoder (always 48 kHz output; pre-skip is left to container timestamps).
@@ -82,11 +82,13 @@ impl PacketDecoder {
let dec = symphonia::default::get_codecs().make(&p, &DecoderOptions::default()).map_err(|e| CodecError::Unsupported(e.to_string()))?;
Ok(Self { inner: Inner::Symphonia(dec), channels: 0 })
}
/// AAC: our decoder for AAC-LC; symphonia for other object types (HE-AAC…).
/// AAC: our decoder for AAC-LC and the AAC-LC core of HE-AAC (v1/v2); symphonia for other
/// object types. `sample_rate` is the stream's output rate: when it is twice the core rate
/// (HE-AAC, see [`aac_output_rate`]) the decoded core is upsampled to it.
pub fn aac(asc: &[u8], sample_rate: u32) -> Result<Self> {
let lc = asc.first().map(|b| b >> 3) == Some(2);
if lc && let Ok(dec) = filmcraft_aac::Decoder::new(asc) {
return Ok(Self { inner: Inner::Aac { dec: Box::new(dec), asc: asc.to_vec() }, channels: 0 });
if let Ok(dec) = filmcraft_aac::Decoder::new(asc) {
let up = upsamples_core(dec.sample_rate(), sample_rate).then(|| Box::new(Upsample2x::new(dec.channels())));
return Ok(Self { inner: Inner::Aac { dec: Box::new(dec), asc: asc.to_vec(), up }, channels: 0 });
}
Self::new(CODEC_TYPE_AAC, sample_rate, Some(asc.to_vec()))
}
@@ -119,7 +121,7 @@ impl PacketDecoder {
return Ok(());
}
let dec = filmcraft_aac::Decoder::new(asc).map_err(|e| CodecError::Unsupported(format!("AAC: {e}")))?;
self.inner = Inner::Aac { dec: Box::new(dec), asc: asc.to_vec() };
self.inner = Inner::Aac { dec: Box::new(dec), asc: asc.to_vec(), up: None };
Ok(())
}
/// AC-3 (ATSC A/52).
@@ -133,7 +135,7 @@ impl PacketDecoder {
pub fn for_isobmff(c: &filmcraft_isobmff::CodecConfig, rate: u32) -> Result<Self> {
use filmcraft_isobmff::CodecConfig as C;
match c {
C::Aac(a) => Self::aac(&a.asc, if a.sample_rate > 0 { a.sample_rate } else { rate }),
C::Aac(a) => Self::aac(&a.asc, if rate > 0 { rate } else { a.sample_rate }),
C::Mp3 => Self::new(CODEC_TYPE_MP3, rate, None),
C::Ac3 { .. } => Self::ac3(),
C::Alac { cookie } => Self::new(CODEC_TYPE_ALAC, rate, Some(cookie.clone())),
@@ -145,8 +147,11 @@ impl PacketDecoder {
/// Decode one packet into planar channels.
pub fn decode(&mut self, data: &[u8], ts: u64) -> Result<Vec<Vec<f32>>> {
let dec = match &mut self.inner {
Inner::Aac { dec, .. } => {
let out = dec.decode(data).map_err(|e| CodecError::Decode(e.to_string()))?;
Inner::Aac { dec, up, .. } => {
let mut out = dec.decode(data).map_err(|e| CodecError::Decode(e.to_string()))?;
if let Some(up) = up {
out = up.process(out);
}
self.channels = out.len();
return Ok(out);
}
@@ -202,10 +207,13 @@ impl PacketDecoder {
}
pub fn reset(&mut self) {
match &mut self.inner {
Inner::Aac { dec, asc } => {
Inner::Aac { dec, asc, up } => {
if let Ok(d) = filmcraft_aac::Decoder::new(asc) {
**dec = d;
}
if let Some(up) = up {
up.reset();
}
}
Inner::Opus { dec, .. } => dec.reset(),
Inner::Symphonia(d) => d.reset(),
@@ -215,6 +223,165 @@ impl PacketDecoder {
}
}
/// The output rate of an AAC stream: twice the core rate for HE-AAC (ISO/IEC 14496-3 §1.6.5).
/// Explicit signalling names the rate in the `AudioSpecificConfig`; with implicit signalling (an
/// AAC-LC config at 24 kHz or less) SBR is only visible as fill-element data in the access units,
/// so the first ones are decoded to look for it. `None` if `asc` is not one our decoder reads.
pub fn aac_output_rate<'a>(asc: &[u8], first_units: impl IntoIterator<Item = &'a [u8]>) -> Option<u32> {
let mut dec = filmcraft_aac::Decoder::new(asc).ok()?;
let core = dec.sample_rate();
if let Some(ext) = dec.config().extension_sample_rate {
return Some(ext);
}
if core <= 24_000 {
for au in first_units {
let _ = dec.decode(au);
if dec.sbr() {
return Some(core * 2);
}
}
}
Some(core)
}
/// Whether [`PacketDecoder::aac`] upsamples a decoded AAC core at `core` Hz to `rate` (HE-AAC).
fn upsamples_core(core: u32, rate: u32) -> bool {
core.checked_mul(2) == Some(rate)
}
/// A codec whose packets all decode to the same number of samples (see [`fixed_packet_samples`]).
#[derive(Clone, Copy, Debug)]
pub enum FixedFrames<'a> {
/// AAC with this AudioSpecificConfig.
Aac(&'a [u8]),
/// MPEG-1/2 audio, layer I, II or III.
MpegAudio,
/// AC-3 (and E-AC-3).
Ac3,
}
/// Samples each packet decodes to at the stream's output `rate`, for codecs where that is the same
/// for every packet: AAC 1024 (2048 when [`PacketDecoder::aac`] upsamples an HE-AAC core); MPEG
/// audio and AC-3 from the frame header of `first`, the stream's first packet. `None` when it is
/// not known: an AAC configuration our decoder does not read, a header that does not parse, or a
/// header whose rate is not `rate`.
pub fn fixed_packet_samples(codec: FixedFrames, first: &[u8], rate: u32) -> Option<i64> {
let mpegts = match codec {
FixedFrames::Aac(asc) => {
let core = filmcraft_aac::Decoder::new(asc).ok()?.sample_rate();
return Some(if upsamples_core(core, rate) { 2048 } else { 1024 });
}
FixedFrames::MpegAudio => filmcraft_mpegts::Codec::MpegAudio,
FixedFrames::Ac3 => filmcraft_mpegts::Codec::Ac3,
};
let f = filmcraft_mpegts::frame_info(&mpegts, first)?;
(f.sample_rate == rate && f.samples > 0).then_some(f.samples as i64)
}
/// Where the container puts one audio packet, for [`contiguous_starts`].
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct PacketTime {
/// Its timestamp, in output samples.
pub stamp: i64,
/// Whether `stamp` is the packet's own. A Matroska frame after the first in a laced block
/// without a duration repeats the block's timestamp.
pub stamped: bool,
/// Samples it decodes to, when known.
pub samples: Option<i64>,
}
/// Packet start positions in output samples: each packet starts where the previous one ends,
/// unless its own timestamp is more than `tolerance` samples from there (a real gap or overlap in
/// the recording) or the previous packet's length is unknown; then it starts at its timestamp.
///
/// Container timestamps are coarser than a sample. Matroska stores milliseconds, and a remux to MP4
/// (OBS's) turns that rounding into uneven sample durations: a 1024-sample AAC frame at 48 kHz lasts
/// 21.33 ms, so frames are stamped at 0, 1008, 2064, 3072… Placing each decoded packet at its
/// stamp overwrote the end of one packet and left a hole of silence after another, a click every
/// few packets: crackling throughout playback and export (#236).
pub fn contiguous_starts(packets: impl IntoIterator<Item = PacketTime>, tolerance: i64) -> Vec<i64> {
let tolerance = tolerance.max(0).unsigned_abs();
let mut next: Option<i64> = None;
packets
.into_iter()
.map(|p| {
let start = match next {
Some(n) if !p.stamped || n.abs_diff(p.stamp) <= tolerance => n,
_ => p.stamp,
};
next = p.samples.and_then(|k| start.checked_add(k));
start
})
.collect()
}
/// Taps on each side of the interpolation point in [`Upsample2x`].
const UPSAMPLE_HALF: usize = 12;
/// 2× upsampler for HE-AAC. Our decoder reconstructs the AAC-LC core at half the output rate and
/// not yet the SBR high band, so the core is interpolated to the output rate: timing and pitch are
/// right and the band above the core's Nyquist stays empty. Even outputs are the input samples,
/// odd ones a Blackman-windowed sinc half-band interpolation over 2 × [`UPSAMPLE_HALF`] inputs.
/// The look-ahead delays the output by `UPSAMPLE_HALF` input samples (0.5 ms at 24 kHz); the
/// state carries across packets, so a seek primed with the preceding packet gives the same
/// samples as continuous decoding.
#[derive(Clone)]
pub struct Upsample2x {
taps: [f32; 2 * UPSAMPLE_HALF],
/// The last `2 × UPSAMPLE_HALF` input samples per channel.
hist: Vec<Vec<f32>>,
}
impl Upsample2x {
pub fn new(channels: usize) -> Self {
let m = UPSAMPLE_HALF as f64;
let mut taps = [0f32; 2 * UPSAMPLE_HALF];
let mut sum = 0.0;
for (k, t) in taps.iter_mut().enumerate() {
// distance from the interpolation point (halfway between two inputs)
let x = k as f64 - m + 0.5;
let sinc = (std::f64::consts::PI * x).sin() / (std::f64::consts::PI * x);
let w = 0.42 + 0.5 * (std::f64::consts::PI * x / m).cos() + 0.08 * (2.0 * std::f64::consts::PI * x / m).cos();
*t = (sinc * w) as f32;
sum += sinc * w;
}
for t in &mut taps {
*t = (*t as f64 / sum) as f32;
}
Self { taps, hist: vec![vec![0.0; 2 * UPSAMPLE_HALF]; channels] }
}
pub fn reset(&mut self) {
for h in &mut self.hist {
h.fill(0.0);
}
}
pub fn process(&mut self, input: Vec<Vec<f32>>) -> Vec<Vec<f32>> {
if self.hist.len() < input.len() {
self.hist.resize(input.len(), vec![0.0; 2 * UPSAMPLE_HALF]);
}
let m = UPSAMPLE_HALF;
input
.into_iter()
.zip(&mut self.hist)
.map(|(x, hist)| {
let mut buf = std::mem::take(hist);
buf.extend_from_slice(&x);
let mut out = Vec::with_capacity(x.len() * 2);
// output pair i is centred on buf[i + m]: the sample itself, then the point halfway
// to the next one from buf[i + 1 ..= i + 2m]
for w in buf.windows(2 * m + 1).take(x.len()) {
out.push(w[m]);
out.push(w[1..].iter().zip(&self.taps).map(|(a, b)| a * b).sum());
}
*hist = buf.split_off(buf.len() - 2 * m);
out
})
.collect()
}
}
/// Whether AC-3 audio decodes (our ATSC A/52 decoder).
pub const AC3_DECODER: bool = true;
@@ -570,12 +737,12 @@ impl AudioFileSource {
kind: MediaKind::AudioOnly,
duration: Tick::from_units(frames as i64, rate as i64),
video: None,
audio: Some(AudioStreamInfo {
audio_streams: vec![AudioStreamInfo {
sample_rate: rate,
channels: samples.len() as u32,
codec: codec_name,
bits_per_sample: track.codec_params.bits_per_sample,
}),
}],
container: ext.to_uppercase(),
start_timecode: None,
file_size: Some(bytes.len() as u64),
@@ -603,3 +770,122 @@ pub fn opener(name: &str, bytes: Arc<[u8]>) -> Option<std::result::Result<Shared
}
Some(AudioFileSource::decode(name, bytes).map(|s| Arc::new(s) as SharedSource).map_err(Into::into))
}
#[cfg(test)]
mod he_aac_tests {
use super::*;
/// AAC-LC config, 22.05 kHz stereo.
const LC_22K: [u8; 2] = [0x13, 0x90];
/// An access unit holding only a fill element with SBR data (ID_FIL, EXT_SBR_DATA, ID_END).
const SBR_FILL: [u8; 3] = [0b1100_0011, 0b1010_0001, 0b1100_0000];
#[test]
fn output_rate_doubles_for_he_aac() {
// explicit: the extension rate from the config
assert_eq!(aac_output_rate(&[0x2B, 0x92, 0x08, 0x00], []), Some(44_100));
// implicit: SBR data in the first access units of a ≤ 24 kHz AAC-LC stream
assert_eq!(aac_output_rate(&LC_22K, [&SBR_FILL[..]]), Some(44_100));
assert_eq!(aac_output_rate(&LC_22K, []), Some(22_050));
// AAC-LC at 44.1 kHz stays as it is, even with SBR-looking fill data
assert_eq!(aac_output_rate(&[0x12, 0x10], [&SBR_FILL[..]]), Some(44_100));
}
#[test]
fn upsampler_interpolates_continuously_across_packets() {
let f = 1000.0 / 22_050.0;
let x: Vec<f32> = (0..4096).map(|i| (2.0 * std::f32::consts::PI * f * i as f32).sin()).collect();
let mut up = Upsample2x::new(1);
let mut y = Vec::new();
for chunk in x.chunks(1024) {
y.extend(up.process(vec![chunk.to_vec()]).remove(0));
}
assert_eq!(y.len(), 8192);
// output 2k + 1 is input k − UPSAMPLE_HALF + ½ (the look-ahead delay), within 0.1 %
let d = UPSAMPLE_HALF as f32;
for k in 100..4000 {
let want = (2.0 * std::f32::consts::PI * f * (k as f32 - d + 0.5)).sin();
assert!((y[2 * k + 1] - want).abs() < 1e-3, "{k}: {} vs {want}", y[2 * k + 1]);
assert_eq!(y[2 * k], x[k - UPSAMPLE_HALF]);
}
}
}
#[cfg(test)]
mod packet_time_tests {
use super::*;
/// 1024-sample frames at 48 kHz stamped to the nearest millisecond, then back in samples.
fn ms_stamps(count: i64) -> Vec<i64> {
(0..count).map(|k| (k * 1024 * 1000 + 24_000) / 48_000 * 48).collect()
}
fn timed(stamps: &[i64], samples: Option<i64>) -> Vec<PacketTime> {
stamps.iter().map(|&stamp| PacketTime { stamp, stamped: true, samples }).collect()
}
#[test]
fn rounded_stamps_give_back_to_back_packets() {
let stamps = ms_stamps(200);
assert_eq!(&stamps[..4], [0, 1008, 2064, 3072]);
let starts = contiguous_starts(timed(&stamps, Some(1024)), 512);
assert_eq!(starts, (0..200).map(|k| k * 1024).collect::<Vec<_>>());
}
#[test]
fn real_gaps_and_overlaps_resynchronise() {
// a 100 ms gap after packet 3, then a packet stamped 600 samples early
let mut stamps: Vec<i64> = (0..8).map(|k| k * 1024).collect();
for s in &mut stamps[4..] {
*s += 4800;
}
stamps[7] -= 600;
let starts = contiguous_starts(timed(&stamps, Some(1024)), 512);
assert_eq!(starts, [0, 1024, 2048, 3072, 8896, 9920, 10944, 11368]);
// within the tolerance: runs on
stamps[7] += 200;
assert_eq!(contiguous_starts(timed(&stamps, Some(1024)), 512)[7], 11968);
}
#[test]
fn unstamped_packets_follow_on_and_unknown_lengths_resynchronise() {
// a laced block of three frames that repeat its timestamp
let packets = [0, 1024, 1024, 1024, 4096].map(|stamp| PacketTime { stamp, stamped: true, samples: Some(1024) });
let mut laced = packets;
laced[2].stamped = false;
laced[3].stamped = false;
assert_eq!(contiguous_starts(laced, 512), [0, 1024, 2048, 3072, 4096]);
// a packet whose length is unknown: the next one starts at its own stamp
let mut unknown = timed(&[0, 1024, 2100, 3124], Some(1024));
unknown[1].samples = None;
assert_eq!(contiguous_starts(unknown, 512), [0, 1024, 2100, 3124]);
}
#[test]
fn hostile_values_stay_finite() {
let packets = [
PacketTime { stamp: i64::MAX, stamped: true, samples: Some(i64::MAX) },
PacketTime { stamp: i64::MIN, stamped: false, samples: Some(-5) },
PacketTime { stamp: 0, stamped: true, samples: Some(i64::MIN) },
PacketTime { stamp: 7, stamped: true, samples: None },
];
assert_eq!(contiguous_starts(packets, i64::MIN).len(), 4);
assert_eq!(contiguous_starts(packets, i64::MAX).len(), 4);
assert!(contiguous_starts([], 512).is_empty());
}
#[test]
fn fixed_frame_lengths() {
// AAC-LC 48 kHz stereo; HE-AAC (24 kHz core) played at 48 kHz is upsampled
assert_eq!(fixed_packet_samples(FixedFrames::Aac(&[0x11, 0x90]), &[], 48_000), Some(1024));
assert_eq!(fixed_packet_samples(FixedFrames::Aac(&[0x13, 0x10]), &[], 48_000), Some(2048));
assert_eq!(fixed_packet_samples(FixedFrames::Aac(&[]), &[], 48_000), None);
// MPEG-1 layer III 44.1 kHz, 128 kbps; MPEG-2 layer III 22.05 kHz
assert_eq!(fixed_packet_samples(FixedFrames::MpegAudio, &[0xFF, 0xFB, 0x90, 0x44], 44_100), Some(1152));
assert_eq!(fixed_packet_samples(FixedFrames::MpegAudio, &[0xFF, 0xF3, 0x90, 0x44], 22_050), Some(576));
// the header's rate must be the stream's
assert_eq!(fixed_packet_samples(FixedFrames::MpegAudio, &[0xFF, 0xFB, 0x90, 0x44], 48_000), None);
assert_eq!(fixed_packet_samples(FixedFrames::MpegAudio, &[0xFF], 44_100), None);
assert_eq!(fixed_packet_samples(FixedFrames::Ac3, &[0x0B, 0x77], 48_000), None);
}
}
+178
View File
@@ -0,0 +1,178 @@
//! Audio packets placed by container timestamps (#236).
//!
//! An AAC access unit decodes to 1024 samples, 21.33 ms at 48 kHz. Matroska stores timestamps in
//! whole milliseconds, and OBS's remux to MP4 carries that rounding into sample durations of 1008,
//! 1008, 1056… Placing each decoded packet at its own timestamp overwrote the end of one packet and
//! left a hole of silence after another: playback and export crackled throughout. The streams here
//! are AAC from our own encoder muxed with such timestamps; read back block by block, as playback
//! reads them, they must be exactly the access units decoded back to back.
use std::io::Cursor;
use std::sync::Arc;
use filmcraft_isobmff::{Brand, Mp4Writer, SampleEntry, TrackConfig, WriteSample, WriterOptions};
use filmcraft_matroska::{LacingMode, MkvWriter, MuxOptions, TrackKind, TrackSpec};
use filmcraft_media::MediaSource;
use crate::audio::PacketDecoder;
use crate::{MkvSource, Mp4Source};
/// A second of a stereo tone (440 Hz left, 660 Hz right) encoded with our AAC encoder: the
/// AudioSpecificConfig and the access units.
fn aac(rate: u32) -> (Vec<u8>, Vec<Vec<u8>>) {
let tone = |hz: f32| (0..rate as usize).map(|i| 0.5 * (std::f32::consts::TAU * hz * i as f32 / rate as f32).sin()).collect::<Vec<f32>>();
let (l, r) = (tone(440.0), tone(660.0));
let mut enc = filmcraft_aac::Encoder::new(filmcraft_aac::EncoderConfig::cbr(rate, 2, 160_000)).unwrap();
let mut units = enc.encode(&[&l, &r]);
units.extend(enc.flush());
(enc.audio_specific_config(), units)
}
/// The access units decoded back to back.
fn continuous(asc: &[u8], rate: u32, units: &[Vec<u8>]) -> Vec<Vec<f32>> {
let mut dec = PacketDecoder::aac(asc, rate).unwrap();
let mut out = vec![Vec::new(), Vec::new()];
for u in units {
for (o, c) in out.iter_mut().zip(dec.decode(u, 0).unwrap()) {
o.extend(c);
}
}
out
}
/// Access unit `k`'s start to the nearest millisecond, as muxers stamp it.
fn stamp_ms(rate: u32, k: usize) -> i64 {
(k as i64 * 1024 * 2000 + rate as i64) / (2 * rate as i64)
}
/// Matroska (1 ms timestamps), with the units after `gap` delayed by `gap_ms` (a real gap).
fn mkv(asc: &[u8], rate: u32, units: &[Vec<u8>], lacing: LacingMode, gap: usize, gap_ms: i64) -> Arc<[u8]> {
let mut spec = TrackSpec::new(TrackKind::Audio, "A_AAC");
spec.codec_private = asc.to_vec();
spec.audio = Some((rate as f64, 2, None));
let opts = MuxOptions { lacing, ..MuxOptions::default() };
assert_eq!(opts.timestamp_scale, 1_000_000, "1 ms timestamps");
let mut w = MkvWriter::new(Cursor::new(Vec::new()), vec![spec], opts).unwrap();
for (k, u) in units.iter().enumerate() {
let ms = stamp_ms(rate, k) + if k >= gap { gap_ms } else { 0 };
w.write_frame(0, ms * 1_000_000, true, u, None).unwrap();
}
w.finish().unwrap().into_inner().into()
}
/// MP4 as OBS remuxes Matroska: each millisecond timestamp rescaled to the sample rate, the
/// differences written as sample durations.
fn mp4_remuxed(asc: &[u8], rate: u32, units: &[Vec<u8>]) -> Arc<[u8]> {
let mut w = Mp4Writer::new(Cursor::new(Vec::new()), WriterOptions::new(Brand::Mp4)).unwrap();
let t = w.add_track(TrackConfig::new(SampleEntry::aac(asc.to_vec(), 2, rate), rate)).unwrap();
let at = |k: usize| (stamp_ms(rate, k) * rate as i64 + 500) / 1000;
for (k, u) in units.iter().enumerate() {
let duration = (at(k + 1) - at(k)) as u32;
w.write_sample(t, WriteSample { data: u, duration, composition_offset: 0, is_sync: true }).unwrap();
}
w.finish().unwrap().into_inner().into()
}
/// `frames` samples per channel read in 512-sample blocks at `rate`, as playback reads them.
fn played(src: &dyn MediaSource, rate: u32, frames: usize) -> Vec<Vec<f32>> {
let mut out = vec![Vec::new(), Vec::new()];
let mut pos = 0;
while pos < frames {
let n = 512.min(frames - pos);
let b = src.audio(pos as i64, n, rate).unwrap();
for (o, c) in out.iter_mut().zip(&b.channels) {
o.extend_from_slice(c);
}
pos += n;
}
out
}
/// Samples of `got` that differ from `want[k - shift]` (silence where that is out of range), and
/// the first few.
fn mismatches(got: &[Vec<f32>], want: &[Vec<f32>], shift: usize) -> (usize, Vec<(usize, usize)>) {
let mut bad = Vec::new();
for (c, (g, w)) in got.iter().zip(want).enumerate() {
for (k, x) in g.iter().enumerate() {
let expect = k.checked_sub(shift).and_then(|j| w.get(j)).copied().unwrap_or(0.0);
if x.to_bits() != expect.to_bits() {
bad.push((c, k));
}
}
}
(bad.len(), bad.into_iter().take(4).collect())
}
/// Runs of two or more exact zeros inside the signal: holes left between packets.
fn holes(got: &[Vec<f32>]) -> usize {
got.iter().map(|c| c.windows(2).filter(|w| w[0] == 0.0 && w[1] == 0.0).count()).sum()
}
#[test]
fn matroska_millisecond_stamps_play_back_to_back() {
for rate in [48_000, 44_100] {
let (asc, units) = aac(rate);
let want = continuous(&asc, rate, &units);
let frames = units.len().saturating_sub(1) * 1024;
for lacing in [LacingMode::None, LacingMode::Xiph] {
let bytes = mkv(&asc, rate, &units, lacing, usize::MAX, 0);
let src = MkvSource::open("tone.mkv", bytes.clone()).unwrap();
let got = played(&src, rate, frames);
let (n, first) = mismatches(&got, &want, 0);
assert_eq!(n, 0, "{rate} Hz, {lacing:?}: {n} samples differ from the continuous decode, first (channel, sample) {first:?}");
// random access (a fresh source, so a cold decoder primed with the packet before) gives
// the same samples
let src = MkvSource::open("tone.mkv", bytes).unwrap();
for start in [30_011usize, 7_000] {
let got = src.audio(start as i64, 4_800, rate).unwrap().channels;
let at: Vec<Vec<f32>> = want.iter().map(|c| c[start..start + 4_800].to_vec()).collect();
assert_eq!(mismatches(&got, &at, 0).0, 0, "{rate} Hz, {lacing:?}: seek to {start}");
}
}
}
}
#[test]
fn obs_remuxed_mp4_plays_back_to_back() {
for rate in [48_000, 44_100] {
let (asc, units) = aac(rate);
let want = continuous(&asc, rate, &units);
let bytes = mp4_remuxed(&asc, rate, &units);
let src = Mp4Source::open("tone.mp4", bytes).unwrap();
let got = played(&src, rate, units.len().saturating_sub(1) * 1024);
let (n, first) = mismatches(&got, &want, 0);
assert_eq!(n, 0, "{rate} Hz: {n} samples differ from the continuous decode, first (channel, sample) {first:?}");
}
}
#[test]
fn a_real_gap_stays_where_the_timestamps_put_it() {
let rate = 48_000;
let (asc, units) = aac(rate);
let want = continuous(&asc, rate, &units);
// 100 ms of the recording missing before unit 20: it plays as silence, and the units after it
// keep their timestamps (to the millisecond)
let src = MkvSource::open("gap.mkv", mkv(&asc, rate, &units, LacingMode::None, 20, 100)).unwrap();
let shift = ((stamp_ms(rate, 20) + 100) * 48 - 20 * 1024) as usize;
assert!((4_700..4_900).contains(&shift), "{shift}");
let got = played(&src, rate, units.len().saturating_sub(1) * 1024 + shift);
let before: Vec<Vec<f32>> = got.iter().map(|c| c[..20 * 1024].to_vec()).collect();
assert_eq!(mismatches(&before, &want, 0).0, 0);
assert!(got.iter().all(|c| c[20 * 1024..20 * 1024 + shift].iter().all(|x| *x == 0.0)), "the gap is silent");
let after: Vec<Vec<f32>> = got.iter().map(|c| c[20 * 1024 + shift..].to_vec()).collect();
let rest: Vec<Vec<f32>> = want.iter().map(|c| c[20 * 1024..].to_vec()).collect();
let (n, first) = mismatches(&after, &rest, 0);
assert_eq!(n, 0, "{n} samples after the gap differ, first {first:?}");
}
#[test]
fn playing_at_another_rate_leaves_no_holes() {
// a 44.1 kHz recording in a 48 kHz sequence (export's default rate)
let (asc, units) = aac(44_100);
let src = MkvSource::open("tone.mkv", mkv(&asc, 44_100, &units, LacingMode::None, usize::MAX, 0)).unwrap();
let frames = (units.len().saturating_sub(2) * 1024) * 48_000 / 44_100;
let got = played(&src, 48_000, frames);
// skip the encoder's priming (the first frame decodes to near-silence)
let tail: Vec<Vec<f32>> = got.iter().map(|c| c[2048..].to_vec()).collect();
assert_eq!(holes(&tail), 0);
}
+547 -14
View File
@@ -5,8 +5,11 @@
//! for the next frames therefore hit the cache or continue the running decoder without re-seeking.
use std::collections::BTreeMap;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex};
use std::sync::atomic::{AtomicU64, AtomicUsize, Ordering};
use std::sync::{Arc, Mutex, MutexGuard, PoisonError, TryLockError, Weak};
use std::time::Duration;
use web_time::Instant;
use filmcraft_color::ColorInfo;
use filmcraft_frame::VideoFrame;
@@ -68,7 +71,7 @@ fn count_frames<T>(out: &[T]) {
/// Run a decoder call, adding its wall time to the process-wide counter.
fn timed<R>(f: impl FnOnce() -> R) -> R {
let t0 = web_time::Instant::now();
let t0 = Instant::now();
let r = f();
DECODE_NS.fetch_add(t0.elapsed().as_nanos() as u64, Ordering::Relaxed);
r
@@ -135,9 +138,39 @@ struct State {
/// The cached frames that were decoded in draft mode: served to draft requests only.
drafts: std::collections::BTreeSet<i64>,
bytes: usize,
/// When a frame was last asked of this cache.
last_used: Instant,
/// Only background requests (thumbnails, [`filmcraft_media::cancel::with_background`]) have
/// used the decoder since it was made: it is given up as soon as another background request
/// is done, whatever the caches' recency.
background: bool,
}
impl State {
/// Give up the decoder (and the spare intra decoders): the next request that has to decode
/// makes a new one and restarts at a sync sample. Returned so they are dropped off the lock.
fn release_decoder(&mut self, pool: &Pool) -> Vec<Box<dyn VideoDecoder>> {
let mut out = std::mem::take(&mut self.spare);
if let Some(d) = self.decoder.take() {
pool.decoders.fetch_sub(1, Ordering::Relaxed);
out.push(d);
}
self.next = usize::MAX;
out
}
/// Evict frames, earliest first, while `more` says so.
fn release_frames(&mut self, pool: &Pool, more: impl Fn() -> bool) {
while more() {
let Some((pts, f)) = self.frames.pop_first() else { break };
self.drafts.remove(&pts);
self.bytes = self.bytes.saturating_sub(f.byte_size());
pool.bytes.fetch_sub(f.byte_size(), Ordering::Relaxed);
EVICTED.fetch_add(1, Ordering::Relaxed);
filmcraft_frame::pool::recycle(f);
}
}
/// The cached frame at `pts`, unless it is a draft frame and the request wants the exact one.
fn cached(&self, pts: i64, draft_ok: bool) -> Option<&Arc<VideoFrame>> {
self.frames.get(&pts).filter(|_| draft_ok || !self.drafts.contains(&pts))
@@ -178,9 +211,201 @@ impl Drop for DecodingGuard {
}
}
/// A decoder keeps its reference and in-flight pictures (hundreds of MB for a frame-threaded
/// 4K decoder), so only this many caches keep theirs once idle: a timeline's other clips make a
/// new one when the playhead reaches them, which costs what a seek costs.
const MAX_LIVE_DECODERS: usize = 4;
/// A cache asked for a frame this recently may be needed again soon (the next clip being
/// prefetched, a clip the user scrubs back to): it keeps its decoder up to [`HARD_DECODERS`], and
/// beyond [`MAX_LIVE_DECODERS`] only until it has been idle this long. Its frames are only kept
/// within [`FRAME_BUDGET`] once it has been idle for [`RECENT`]: going back to a clip left (or
/// paused) more than that long ago may cost a re-decode from its keyframe when other clips have
/// filled the budget meanwhile. Frames go least recently used cache first, earliest frames first.
const IDLE: Duration = Duration::from_secs(3);
/// A cache asked for a frame this recently is in use (a layer of the frame being composited): it
/// keeps its decoder and frames whatever the count. Caches idle for less than [`IDLE`] but longer
/// than this are a burst of navigation (thumbnails of every clip, the playhead jumping from clip
/// to clip) that the playhead has already left: they give up decoders beyond [`HARD_DECODERS`]
/// and frames beyond [`FRAME_BUDGET`]. Without this, everything touched within [`IDLE`] counted
/// as in use, so a burst held every clip's decoder (42 live hardware sessions, 6.5 GB of VRAM)
/// and hundreds of MB of frames per clip.
const RECENT: Duration = Duration::from_millis(500);
/// Most decoders caches idle for [`RECENT`] to [`IDLE`] keep together (a composite of a few
/// layers plus the clips just left behind).
const HARD_DECODERS: usize = 8;
/// Decoded frames all caches may hold together before idle ones give theirs up. Each cache also
/// has its own budget, which alone let every clip on a timeline keep hundreds of MB of frames
/// nobody was looking at.
pub const FRAME_BUDGET: usize = 1 << 30;
/// What the caches of a process share: the cap on live decoders and the frame budget.
struct Pool {
caches: Mutex<Vec<Weak<Shared>>>,
/// Caches that hold a decoder.
decoders: AtomicUsize,
max_decoders: usize,
idle: Duration,
/// Caches idle at least this long are no longer in use (see [`RECENT`]).
recent: Duration,
/// Decoders the caches idle for [`Self::recent`] but not [`Self::idle`] may keep.
hard_decoders: usize,
/// Bytes of the frames the caches hold.
bytes: AtomicUsize,
budget: usize,
}
/// The part of a [`GopCache`] its pool reaches.
struct Shared {
state: Mutex<State>,
pool: Arc<Pool>,
}
impl Drop for Shared {
fn drop(&mut self) {
let st = self.state.get_mut().unwrap_or_else(PoisonError::into_inner);
drop(st.release_decoder(&self.pool));
self.pool.bytes.fetch_sub(st.bytes, Ordering::Relaxed);
}
}
/// A cache's state unless another thread is using it (it is then decoding: not idle).
fn try_state(c: &Shared) -> Option<MutexGuard<'_, State>> {
match c.state.try_lock() {
Ok(g) => Some(g),
Err(TryLockError::Poisoned(e)) => Some(e.into_inner()),
Err(TryLockError::WouldBlock) => None,
}
}
impl Pool {
fn new(max_decoders: usize, idle: Duration, budget: usize) -> Self {
Self {
caches: Mutex::new(Vec::new()),
decoders: AtomicUsize::new(0),
max_decoders,
idle,
recent: idle,
hard_decoders: max_decoders,
bytes: AtomicUsize::new(0),
budget,
}
}
/// Caches idle for `recent` (< the idle time) are still trimmed, down to `hard_decoders`.
fn tiered(mut self, recent: Duration, hard_decoders: usize) -> Self {
self.recent = recent.min(self.idle);
self.hard_decoders = hard_decoders.max(self.max_decoders);
self
}
/// The pool of every cache in the process.
fn global() -> Arc<Pool> {
static POOL: std::sync::OnceLock<Arc<Pool>> = std::sync::OnceLock::new();
POOL.get_or_init(|| Arc::new(Pool::new(MAX_LIVE_DECODERS, IDLE, FRAME_BUDGET).tiered(RECENT, HARD_DECODERS))).clone()
}
fn register(&self, c: &Arc<Shared>) {
let mut g = self.caches.lock().unwrap_or_else(PoisonError::into_inner);
g.retain(|w| w.strong_count() > 0);
g.push(Arc::downgrade(c));
}
/// The caches other than `me` idle for at least `idle` that `keep` selects, least recently used first. Other
/// caches are only ever try-locked, so two caches trimming each other cannot deadlock.
fn idle_others(&self, me: &Shared, idle: Duration, keep: impl Fn(&State) -> bool) -> Vec<Arc<Shared>> {
let all: Vec<Arc<Shared>> = self.caches.lock().unwrap_or_else(PoisonError::into_inner).iter().filter_map(Weak::upgrade).collect();
let mut idle: Vec<(Instant, Arc<Shared>)> = all
.into_iter()
.filter(|c| !std::ptr::eq(Arc::as_ptr(c), me))
.filter_map(|c| {
let used = try_state(&c).filter(|st| keep(st) && st.last_used.elapsed() >= idle).map(|st| st.last_used)?;
Some((used, c))
})
.collect();
idle.sort_by_key(|(used, _)| *used);
idle.into_iter().map(|(_, c)| c).collect()
}
/// Take the decoders of idle caches, least recently used first, down to the cap. The caller
/// drops them with no cache lock held: dropping a decoder can block (a hardware session waits
/// for its frames in flight).
#[must_use = "drop the released decoders with no cache lock held"]
fn trim_decoders(&self, me: &Shared) -> Vec<Box<dyn VideoDecoder>> {
let mut released = Vec::new();
// first the caches idle for long, down to the cap; then those a burst has left behind
for (idle, cap) in [(self.idle, self.max_decoders), (self.recent, self.hard_decoders)] {
if self.decoders.load(Ordering::Relaxed) <= cap {
continue;
}
for c in self.idle_others(me, idle, |st| st.decoder.is_some()) {
if self.decoders.load(Ordering::Relaxed) <= cap {
break;
}
if let Some(mut st) = try_state(&c) {
released.extend(st.release_decoder(self));
}
}
}
released
}
/// Take the decoders of the caches other than `me` that only background requests have used
/// (see [`State::background`]) and that are not decoding now. A burst of thumbnails (an import
/// of many clips) touches every clip within moments, so the recency rules treat them all as
/// in use; each software 4K decoder holds ~0.5 GB. The cache `me` just served keeps its
/// decoder (hovering a clip asks for more of its frames). Drop the result with no cache lock
/// held.
#[must_use = "drop the released decoders with no cache lock held"]
fn release_background(&self, me: &Shared) -> Vec<Box<dyn VideoDecoder>> {
let all: Vec<Arc<Shared>> = self.caches.lock().unwrap_or_else(PoisonError::into_inner).iter().filter_map(Weak::upgrade).collect();
let mut released = Vec::new();
for c in all.iter().filter(|c| !std::ptr::eq(Arc::as_ptr(c), me)) {
if let Some(mut st) = try_state(c)
&& st.background
&& st.decoder.is_some()
{
released.extend(st.release_decoder(self));
}
}
released
}
/// Evict the frames of caches idle for [`RECENT`], least recently used first, down to the
/// budget. Caches in use keep theirs (each within its own budget): a frame evicted before it is shown costs a
/// re-decode from the keyframe.
fn trim_frames(&self, me: &Shared) {
let over = || self.bytes.load(Ordering::Relaxed) > self.budget;
if !over() {
return;
}
for c in self.idle_others(me, self.recent, |st| !st.frames.is_empty()) {
if !over() {
break;
}
if let Some(mut st) = try_state(&c) {
st.release_frames(self, over);
}
}
}
}
/// Caches that hold a decoder right now, process-wide (`perf.stats`).
pub fn live_decoders() -> usize {
Pool::global().decoders.load(Ordering::Relaxed)
}
/// Bytes of decoded frames the caches hold right now, process-wide (`perf.stats`).
pub fn cached_bytes() -> usize {
Pool::global().bytes.load(Ordering::Relaxed)
}
/// Decoder + decoded-frame cache for one video track.
pub struct GopCache {
state: Mutex<State>,
shared: Arc<Shared>,
/// Colour signalled by the container, which wins over the bitstream's.
explicit_color: Option<ColorInfo>,
/// Clockwise quarter turns from the container's display matrix, applied to every frame.
@@ -200,7 +425,13 @@ const CONTINUE_THROUGH: usize = 48;
impl GopCache {
pub fn new(explicit_color: Option<ColorInfo>) -> Self {
Self {
// unit tests count decoder restarts: each of their caches gets a pool of its own
let pool = if cfg!(test) { Arc::new(Pool::new(MAX_LIVE_DECODERS, IDLE, FRAME_BUDGET).tiered(RECENT, HARD_DECODERS)) } else { Pool::global() };
Self::in_pool(explicit_color, pool)
}
fn in_pool(explicit_color: Option<ColorInfo>, pool: Arc<Pool>) -> Self {
let shared = Arc::new(Shared {
state: Mutex::new(State {
decoder: None,
intra: false,
@@ -211,11 +442,13 @@ impl GopCache {
frames: BTreeMap::new(),
drafts: Default::default(),
bytes: 0,
last_used: Instant::now(),
background: false,
}),
explicit_color,
rotation: 0,
budget: 384 << 20,
}
pool: pool.clone(),
});
pool.register(&shared);
Self { shared, explicit_color, rotation: 0, budget: 384 << 20 }
}
/// Turn every decoded frame clockwise by `quarter_turns` × 90° (the container's display
@@ -238,6 +471,7 @@ impl GopCache {
fn store(&self, st: &mut State, pts: i64, f: VideoFrame, draft: bool) {
let f = self.finish(f);
st.last_used = Instant::now();
if draft {
st.drafts.insert(pts);
DRAFT.fetch_add(1, Ordering::Relaxed);
@@ -245,9 +479,11 @@ impl GopCache {
st.drafts.remove(&pts);
}
let budget = self.budget.max(MIN_FRAMES * f.byte_size());
let before = st.bytes;
st.bytes += f.byte_size();
if let Some(old) = st.frames.insert(pts, Arc::new(f)) {
st.bytes -= old.byte_size();
filmcraft_frame::pool::recycle(old);
}
// evict frames far from the most recent (keep a window around the working position)
while st.bytes > budget && st.frames.len() > 2 {
@@ -257,8 +493,17 @@ impl GopCache {
st.drafts.remove(&victim);
st.bytes -= v.byte_size();
EVICTED.fetch_add(1, Ordering::Relaxed);
// its planes serve the next decoded pictures instead of going back to the allocator
filmcraft_frame::pool::recycle(v);
}
}
let pool = &self.shared.pool;
if st.bytes >= before {
pool.bytes.fetch_add(st.bytes - before, Ordering::Relaxed);
} else {
pool.bytes.fetch_sub(before - st.bytes, Ordering::Relaxed);
}
pool.trim_frames(&self.shared);
}
/// Store decoder output (in presentation order) and advance `out_max`.
@@ -289,6 +534,16 @@ impl GopCache {
/// ([`VideoDecoder::set_draft`]); its draft frames are cached for draft requests only, and an
/// exact request for one re-decodes it.
pub fn frame_late(&self, s: &dyn VideoSamples, target: i64, late_before: Option<i64>) -> crate::Result<Arc<VideoFrame>> {
let r = self.frame_late_locked(s, target, late_before);
if filmcraft_media::cancel::background() {
// Outside the cache's lock (see `release_background`). A thumbnail leaves the decoders
// of earlier thumbnails no reason to stay: each costs hundreds of MB at 4K.
drop(self.shared.pool.release_background(&self.shared));
}
r
}
fn frame_late_locked(&self, s: &dyn VideoSamples, target: i64, late_before: Option<i64>) -> crate::Result<Arc<VideoFrame>> {
let n = s.count();
let i = s.sample_at(target.max(0)).or_else(|| (n > 0).then(|| n - 1)).ok_or_else(|| CodecError::Decode("empty track".into()))?;
let want_pts = s.pts(i);
@@ -297,19 +552,38 @@ impl GopCache {
return self.private_frame(s, i, want_pts, n);
}
let draft = filmcraft_media::cancel::draft();
let mut st = self.state.lock().unwrap_or_else(|e| e.into_inner());
// Idle caches' decoders released below. Declared before the lock guard, so it is dropped
// after the guard on every return path: dropping a decoder can block (a hardware session
// waits for its frames in flight) and must not hold this cache's lock.
let mut released: Vec<Box<dyn VideoDecoder>> = Vec::new();
let mut st = self.shared.state.lock().unwrap_or_else(|e| e.into_inner());
let _decoding = DecodingGuard::enter(me);
if let Some(f) = st.cached(want_pts, draft) {
st.last_used = Instant::now();
let background = filmcraft_media::cancel::background();
if let Some(f) = st.cached(want_pts, draft).cloned() {
HITS.fetch_add(1, Ordering::Relaxed);
return Ok(f.clone());
// a monitor asking for a frame of a thumbnail's clip makes its decoder one in use
st.background &= background;
return Ok(f);
}
MISSES.fetch_add(1, Ordering::Relaxed);
if !background {
st.background = false;
} else if st.decoder.is_none() {
st.background = true;
}
if !background && st.decoder.as_ref().is_some_and(|d| d.thread_limited()) {
// made for a thumbnail (few threads): a monitor gets a full one
released.extend(st.release_decoder(&self.shared.pool));
}
if st.decoder.is_none() {
let d = s.make_decoder()?;
st.intra = d.intra_only();
st.decoder = Some(d);
st.next = usize::MAX;
self.shared.pool.decoders.fetch_add(1, Ordering::Relaxed);
}
released.extend(self.shared.pool.trim_decoders(&self.shared));
// Nothing cached: the work ahead (possibly a seek and a GOP of decoding) is only worth it
// while someone still wants the frame.
if filmcraft_media::cancel::cancelled() {
@@ -462,7 +736,7 @@ impl GopCache {
DECODED.fetch_add(1, Ordering::Relaxed);
timed(|| dec.decode(&data, want_pts))
});
let mut st = self.state.lock().unwrap_or_else(|e| e.into_inner());
let mut st = self.shared.state.lock().unwrap_or_else(|e| e.into_inner());
if st.spare.len() < 16 {
st.spare.push(dec);
}
@@ -480,7 +754,6 @@ mod tests {
use super::*;
use crate::video::DecodedFrame;
use std::collections::VecDeque;
use std::sync::atomic::AtomicUsize;
/// `n` samples, a sync sample every `gop`, pts = 1000 · index (no reordering).
struct Samples {
@@ -499,6 +772,8 @@ mod tests {
/// (pts, decoded as a draft picture)
held: VecDeque<(i64, bool)>,
draft: bool,
/// made for background work (few threads, like the H.264 decoder's)
limited: bool,
resets: Arc<AtomicUsize>,
decodes: Arc<AtomicUsize>,
}
@@ -533,6 +808,9 @@ mod tests {
fn name(&self) -> &str {
"test"
}
fn thread_limited(&self) -> bool {
self.limited
}
fn intra_only(&self) -> bool {
self.intra
}
@@ -565,6 +843,7 @@ mod tests {
intra: self.intra,
held: VecDeque::new(),
draft: false,
limited: filmcraft_media::cancel::background(),
resets: self.resets.clone(),
decodes: self.decodes.clone(),
}))
@@ -716,4 +995,258 @@ mod tests {
assert_eq!(index_of(&c.frame(&s, 7_000).expect("frame")), 7);
assert_eq!(s.resets.load(Ordering::Relaxed), 1);
}
#[test]
fn idle_caches_give_up_their_decoders_beyond_the_cap() {
let pool = Arc::new(Pool::new(2, Duration::ZERO, FRAME_BUDGET));
let live = || pool.decoders.load(Ordering::Relaxed);
let s: Vec<Samples> = (0..5).map(|_| samples(300, 250, 3, false)).collect();
let c: Vec<GopCache> = (0..5).map(|_| GopCache::in_pool(None, pool.clone())).collect();
// a clip after another, as when clips are dropped on a timeline
for k in 0..5 {
assert_eq!(index_of(&c[k].frame(&s[k], 10_000).expect("frame")), 10);
assert!(live() <= 2, "{} live decoders after clip {k}", live());
}
assert_eq!(live(), 2);
// the most recently used keep theirs and carry on without a seek
assert_eq!(index_of(&c[4].frame(&s[4], 20_000).expect("frame")), 20);
assert_eq!(s[4].resets.load(Ordering::Relaxed), 1);
// a cache that lost its decoder still serves what it cached…
let decodes = s[0].decodes.load(Ordering::Relaxed);
assert_eq!(index_of(&c[0].frame(&s[0], 10_000).expect("cached")), 10);
assert_eq!(s[0].decodes.load(Ordering::Relaxed), decodes);
// …and decodes again with a new decoder, from the sync sample
assert_eq!(index_of(&c[0].frame(&s[0], 20_000).expect("frame")), 20);
assert_eq!(s[0].resets.load(Ordering::Relaxed), 2);
assert_eq!(live(), 2);
// a dropped cache no longer counts
drop(c);
assert_eq!(live(), 0);
}
#[test]
fn caches_in_use_keep_their_decoders_whatever_the_cap() {
// every layer of one composited frame was asked moments ago: none is idle
let pool = Arc::new(Pool::new(2, Duration::from_secs(3600), FRAME_BUDGET));
let s: Vec<Samples> = (0..5).map(|_| samples(300, 250, 3, false)).collect();
let c: Vec<GopCache> = (0..5).map(|_| GopCache::in_pool(None, pool.clone())).collect();
for t in [10_000, 11_000, 12_000] {
for k in 0..5 {
assert_eq!(index_of(&c[k].frame(&s[k], t).expect("frame")) as i64, t / 1000);
}
}
assert_eq!(pool.decoders.load(Ordering::Relaxed), 5);
assert!(s.iter().all(|s| s.resets.load(Ordering::Relaxed) == 1), "no decoder restarted");
}
/// A burst of navigation (every clip of a timeline asked once within moments, as when
/// thumbnails are made or the playhead jumps clip to clip) must not keep every clip's decoder
/// and frames: only the caches of the frame being composited are in use.
#[test]
fn a_burst_of_single_requests_does_not_keep_every_decoder_and_frame() {
// the real decoder caps and idle time; RECENT scaled down from 500 ms to the test's pace
// (3 ms per clip). With the old rule (recent = idle) all 24 decoders stay live.
let pool = Arc::new(Pool::new(MAX_LIVE_DECODERS, IDLE, FRAME_BUDGET).tiered(Duration::from_millis(2), HARD_DECODERS));
let n = 24;
let s: Vec<Samples> = (0..n).map(|_| samples(300, 250, 3, false)).collect();
let c: Vec<GopCache> = (0..n).map(|_| GopCache::in_pool(None, pool.clone())).collect();
for k in 0..n {
assert_eq!(index_of(&c[k].frame(&s[k], 10_000).expect("frame")), 10);
std::thread::sleep(Duration::from_millis(3));
}
assert!(
pool.decoders.load(Ordering::Relaxed) <= HARD_DECODERS + 1,
"{} live decoders after a burst over {n} clips",
pool.decoders.load(Ordering::Relaxed)
);
// the clip asked last, and a clip in use together with it, keep theirs and carry on
let resets = s[n - 1].resets.load(Ordering::Relaxed);
assert_eq!(index_of(&c[n - 1].frame(&s[n - 1], 11_000).expect("frame")), 11);
assert_eq!(s[n - 1].resets.load(Ordering::Relaxed), resets);
}
/// The thumbnails an import queues: every clip asked once, quickly (the real caps and recency,
/// no sleeping: all of them are "recent"). Each 4K software decoder costs ~0.5 GB, and 42
/// clips used to hold 6+ GB at once.
#[test]
fn background_requests_leave_no_decoders_behind() {
use filmcraft_media::cancel::with_background;
let pool = Arc::new(Pool::new(MAX_LIVE_DECODERS, IDLE, FRAME_BUDGET).tiered(RECENT, HARD_DECODERS));
let n = 24;
let s: Vec<Samples> = (0..n).map(|_| samples(300, 250, 3, false)).collect();
let c: Vec<GopCache> = (0..n).map(|_| GopCache::in_pool(None, pool.clone())).collect();
// a clip in use by a monitor
assert_eq!(index_of(&c[0].frame(&s[0], 10_000).expect("frame")), 10);
for k in 1..n {
assert_eq!(index_of(&with_background(true, || c[k].frame(&s[k], 10_000)).expect("frame")), 10);
assert!(pool.decoders.load(Ordering::Relaxed) <= 2, "{} live decoders after thumbnail {k}", pool.decoders.load(Ordering::Relaxed));
}
// the monitor's clip and the last thumbnail keep theirs and carry on without a seek
assert_eq!(pool.decoders.load(Ordering::Relaxed), 2);
assert_eq!(index_of(&c[0].frame(&s[0], 11_000).expect("frame")), 11);
assert_eq!(s[0].resets.load(Ordering::Relaxed), 1);
assert_eq!(index_of(&with_background(true, || c[n - 1].frame(&s[n - 1], 12_000)).expect("frame")), 12);
assert_eq!(s[n - 1].resets.load(Ordering::Relaxed), 1);
// a monitor asking for a frame of a thumbnail's clip makes its decoder one in use
assert_eq!(index_of(&c[n - 1].frame(&s[n - 1], 13_000).expect("frame")), 13);
for k in 1..4 {
with_background(true, || c[k].frame(&s[k], 20_000)).expect("frame");
}
assert_eq!(index_of(&c[n - 1].frame(&s[n - 1], 14_000).expect("frame")), 14);
assert!(c[n - 1].shared.state.lock().unwrap().decoder.is_some(), "the thumbnails left its decoder");
// a cache that lost its decoder still decodes (from the sync sample)
assert_eq!(index_of(&with_background(true, || c[1].frame(&s[1], 30_000)).expect("frame")), 30);
}
#[test]
fn a_monitor_replaces_a_decoder_made_for_a_thumbnail() {
use filmcraft_media::cancel::with_background;
let pool = Arc::new(Pool::new(MAX_LIVE_DECODERS, IDLE, FRAME_BUDGET).tiered(RECENT, HARD_DECODERS));
let (s, c) = (samples(300, 250, 3, false), GopCache::in_pool(None, pool.clone()));
assert_eq!(index_of(&with_background(true, || c.frame(&s, 10_000)).expect("frame")), 10);
// another thumbnail of the same clip carries on with the thumbnail decoder
assert_eq!(index_of(&with_background(true, || c.frame(&s, 12_000)).expect("frame")), 12);
assert_eq!(s.resets.load(Ordering::Relaxed), 1);
// a frame a monitor waits for (not cached: far ahead) is decoded by a full-thread decoder
assert_eq!(index_of(&c.frame(&s, 100_000).expect("frame")), 100);
assert!(!c.shared.state.lock().unwrap().decoder.as_ref().is_some_and(|d| d.thread_limited()));
assert_eq!(pool.decoders.load(Ordering::Relaxed), 1);
// and keeps it for the next frames
let resets = s.resets.load(Ordering::Relaxed);
assert_eq!(index_of(&c.frame(&s, 101_000).expect("frame")), 101);
assert_eq!(s.resets.load(Ordering::Relaxed), resets);
}
#[test]
fn a_burst_of_single_requests_does_not_keep_every_clips_frames() {
// 1×1 RGBA pictures (4 bytes): a shared budget of ten of them
let pool = Arc::new(Pool::new(usize::MAX, Duration::from_secs(3600), 40).tiered(Duration::from_millis(2), usize::MAX));
let n = 12;
let s: Vec<Samples> = (0..n).map(|_| samples(300, 250, 0, false)).collect();
let c: Vec<GopCache> = (0..n).map(|_| GopCache::in_pool(None, pool.clone())).collect();
for k in 0..n {
assert_eq!(index_of(&c[k].frame(&s[k], 10_000).expect("frame")), 10);
std::thread::sleep(Duration::from_millis(5));
}
let held = pool.bytes.load(Ordering::Relaxed);
assert!(held <= 40 + 11 * 4, "{held} bytes held after a burst over {n} clips");
}
#[test]
fn intra_caches_give_up_their_spare_decoders_too() {
let pool = Arc::new(Pool::new(1, Duration::ZERO, FRAME_BUDGET));
let (a, b) = (samples(50, 1, 0, true), samples(50, 1, 0, true));
let (ca, cb) = (GopCache::in_pool(None, pool.clone()), GopCache::in_pool(None, pool.clone()));
assert_eq!(index_of(&ca.frame(&a, 3_000).expect("frame")), 3);
assert_eq!(index_of(&cb.frame(&b, 4_000).expect("frame")), 4);
let st = ca.shared.state.lock().unwrap();
assert!(st.decoder.is_none() && st.spare.is_empty());
drop(st);
assert_eq!(pool.decoders.load(Ordering::Relaxed), 1);
assert_eq!(index_of(&ca.frame(&a, 5_000).expect("frame")), 5);
}
#[test]
fn idle_caches_give_up_their_frames_beyond_the_shared_budget() {
// 1×1 RGBA pictures: 4 bytes each, a shared budget of ten of them
let pool = Arc::new(Pool::new(usize::MAX, Duration::ZERO, 40));
let held = || pool.bytes.load(Ordering::Relaxed);
let (a, b) = (samples(300, 250, 0, false), samples(300, 250, 0, false));
let (ca, cb) = (GopCache::in_pool(None, pool.clone()), GopCache::in_pool(None, pool.clone()));
// the only cache keeps what its own budget allows
assert_eq!(index_of(&ca.frame(&a, 10_000).expect("frame")), 10);
assert_eq!(held(), 44);
// a second clip takes the room from the idle first one, earliest frames first
assert_eq!(index_of(&cb.frame(&b, 5_000).expect("frame")), 5);
assert_eq!(held(), 40);
let decodes = b.decodes.load(Ordering::Relaxed);
assert!((0..=5).all(|i| index_of(&cb.frame(&b, i * 1000).expect("cached")) as i64 == i));
assert_eq!(b.decodes.load(Ordering::Relaxed), decodes, "the clip in use lost nothing");
let st = ca.shared.state.lock().unwrap();
assert_eq!((st.frames.len(), st.bytes), (4, 16));
assert_eq!(st.frames.keys().next(), Some(&7_000));
drop(st);
// an evicted frame decodes again
assert_eq!(index_of(&ca.frame(&a, 2_000).expect("frame")), 2);
assert!(held() <= 40 + 4 * 3, "{} bytes held", held());
drop((ca, cb));
assert_eq!(held(), 0);
}
#[test]
fn caches_in_use_keep_their_frames_whatever_the_shared_budget() {
let pool = Arc::new(Pool::new(usize::MAX, Duration::from_secs(3600), 40));
let (a, b) = (samples(300, 250, 0, false), samples(300, 250, 0, false));
let (ca, cb) = (GopCache::in_pool(None, pool.clone()), GopCache::in_pool(None, pool.clone()));
assert_eq!(index_of(&ca.frame(&a, 10_000).expect("frame")), 10);
assert_eq!(index_of(&cb.frame(&b, 10_000).expect("frame")), 10);
assert_eq!(pool.bytes.load(Ordering::Relaxed), 88);
let decodes = a.decodes.load(Ordering::Relaxed);
assert_eq!(index_of(&ca.frame(&a, 0).expect("cached")), 0);
assert_eq!(a.decodes.load(Ordering::Relaxed), decodes);
}
/// Pictures big enough for the plane pool (its own size, so that parallel tests cannot take
/// the buffers), no reordering, one long GOP.
struct Big(usize);
const BIG: (u32, u32) = (200, 101);
impl VideoDecoder for Big {
fn decode(&mut self, _sample: &[u8], pts: i64) -> crate::Result<Vec<DecodedFrame>> {
let mut px = filmcraft_frame::pool::take_u8((BIG.0 * BIG.1 * 4) as usize);
px.resize((BIG.0 * BIG.1 * 4) as usize, (pts / 1000) as u8);
Ok(vec![DecodedFrame { pts, frame: VideoFrame::rgba8(BIG.0, BIG.1, px), draft: false }])
}
fn flush(&mut self) -> Vec<DecodedFrame> {
Vec::new()
}
fn reset(&mut self) {}
fn name(&self) -> &str {
"big"
}
}
impl VideoSamples for Big {
fn count(&self) -> usize {
self.0
}
fn pts(&self, i: usize) -> i64 {
i as i64 * 1000
}
fn sync_before(&self, _i: usize) -> usize {
0
}
fn sample_at(&self, t: i64) -> Option<usize> {
let i = (t / 1000) as usize;
(i < self.0).then_some(i)
}
fn read(&self, i: usize) -> crate::Result<Vec<u8>> {
Ok(vec![i as u8])
}
fn make_decoder(&self) -> crate::Result<Box<dyn VideoDecoder>> {
Ok(Box::new(Big(self.0)))
}
}
#[test]
fn evicted_frames_are_decoded_into_again() {
let s = Big(200);
// room for MIN_FRAMES pictures only
let mut c = GopCache::new(None);
c.budget = 0;
let reused = || filmcraft_frame::pool::stats().reused;
let before = reused();
// the cache holds MIN_FRAMES pictures: none is evicted yet, every plane is new
for i in 0..MIN_FRAMES as i64 {
assert_eq!(c.frame(&s, i * 1000).expect("frame").width, BIG.0);
}
assert_eq!(reused(), before);
// from here on each picture evicts one, whose buffer the next picture is decoded into
let more = 40;
for i in MIN_FRAMES as i64..MIN_FRAMES as i64 + more {
let f = c.frame(&s, i * 1000).expect("frame");
assert!(matches!(&f.data, filmcraft_frame::PixelData::Rgba8(d) if d.iter().all(|&b| b == i as u8)), "frame {i} has its own pixels");
}
assert!(reused() - before >= more as u64 - 2, "{} of {more} planes recycled", reused() - before);
}
}
+79
View File
@@ -12,6 +12,7 @@ use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use filmcraft_isobmff::{CodecConfig, SampleEntry};
pub use crate::hw_frame::{FrameCodec, FrameStreamInfo, PictureParams};
use crate::video::{avcc_length_size, h264_disposable, hevc_disposable, hvcc_length_size_and_tid, sar_par, vui_color};
use crate::{CodecError, Result};
@@ -29,6 +30,19 @@ pub fn hardware_decoding() -> bool {
ENABLED.load(Ordering::Relaxed)
}
static BACKEND: std::sync::Mutex<Option<&'static str>> = std::sync::Mutex::new(None);
/// Record the OS decoder backend `filmcraft_platform::register` put in front of our decoders
/// ("VideoToolbox", "Media Foundation"), for `perf.stats`.
pub fn set_hw_backend(name: &'static str) {
*BACKEND.lock().unwrap_or_else(std::sync::PoisonError::into_inner) = Some(name);
}
/// The registered hardware decoder backend, if any ([`set_hw_backend`]).
pub fn hw_backend() -> Option<&'static str> {
*BACKEND.lock().unwrap_or_else(std::sync::PoisonError::into_inner)
}
/// Process-wide hardware decoding counters (they only grow; subtract two snapshots to measure an
/// interval). Frames decoded in software are `GopStats::frames - frames`.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
@@ -120,6 +134,71 @@ pub struct NalStreamInfo {
pub reorder: usize,
}
/// Any stream a hardware decoder can take: what [`crate::hw`]'s hybrid decoder needs to know about
/// it to find restart points and in-band parameter changes.
#[derive(Clone, Debug)]
pub enum StreamInfo {
Nal(NalStreamInfo),
Frame(FrameStreamInfo),
}
impl From<NalStreamInfo> for StreamInfo {
fn from(i: NalStreamInfo) -> Self {
Self::Nal(i)
}
}
impl From<FrameStreamInfo> for StreamInfo {
fn from(i: FrameStreamInfo) -> Self {
Self::Frame(i)
}
}
impl StreamInfo {
/// Whether decoding can restart at `sample` with no earlier state (IDR / IRAP, a key frame).
pub fn is_irap(&self, sample: &[u8]) -> bool {
match self {
Self::Nal(i) => i.is_irap(sample),
Self::Frame(i) => i.is_random_access(sample),
}
}
/// An HEVC CRA restarts nothing by itself (its RASL pictures need the previous restart point).
pub fn keeps_previous_restart(&self, sample: &[u8]) -> bool {
match self {
Self::Nal(i) => i.codec == NalCodec::Hevc && i.nal_types(sample).contains(&21),
Self::Frame(_) => false,
}
}
/// Whether `sample` carries parameters different from the ones the hardware decoder was set up
/// with (parameter sets, a sequence header, a key frame of another format).
pub fn parameters_changed(&self, sample: &[u8]) -> bool {
match self {
Self::Nal(i) => {
i.nals(sample).into_iter().any(|n| n.first().is_some_and(|&h| i.is_parameter_set(i.nal_type(h))) && !i.parameter_sets.iter().any(|p| p == n))
}
Self::Frame(i) => i.parameters_changed(sample),
}
}
/// As the software decoder's [`crate::VideoDecoder::is_random_access`].
pub fn is_random_access(&self, sample: &[u8]) -> Option<bool> {
match self {
Self::Nal(i) => i.is_random_access(sample),
Self::Frame(i) => Some(i.is_random_access(sample)),
}
}
/// As the software decoder's [`crate::VideoDecoder::is_disposable`].
pub fn is_disposable(&self, sample: &[u8]) -> bool {
match self {
Self::Nal(i) => i.is_disposable(sample),
Self::Frame(_) => false,
}
}
}
/// Length-prefixed parameter-set entries (`u16` length + bytes) starting at `pos`.
fn read_sets(rec: &[u8], pos: &mut usize, count: usize, out: &mut Vec<Vec<u8>>) -> Result<()> {
for _ in 0..count {
+256
View File
@@ -0,0 +1,256 @@
//! What a hardware decoder needs to know about a VP9 or AV1 stream so that it behaves exactly like
//! our software decoder for it (the frame-coded counterpart of [`crate::hw::NalStreamInfo`]):
//! codec profile, bit depth and chroma format from the container's `vpcC` / `av1C`, the picture
//! size and colour as the software decoder reports them (read from the bitstream the same way),
//! random-access samples, and in-band changes of the stream's parameters.
use filmcraft_color::{ColorInfo, Primaries, Transfer};
use filmcraft_isobmff::{CodecConfig, SampleEntry};
use crate::video::primaries_from_code;
use crate::{CodecError, Result};
/// The two frame-coded codecs hardware decoders take besides H.264 / HEVC.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum FrameCodec {
Vp9,
Av1,
}
/// Picture parameters read from a random-access sample.
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct PictureParams {
/// Picture size in luma samples (the decoded size).
pub size: (u32, u32),
pub bit_depth: u32,
/// `subsampling_x`, `subsampling_y`.
pub subsampling: (u8, u8),
/// Colour of every output picture, as the software decoder reports it.
pub color: ColorInfo,
}
/// A VP9 or AV1 stream, from its sample entry.
#[derive(Clone, Debug)]
pub struct FrameStreamInfo {
pub codec: FrameCodec,
/// VP9 `profile` (0..=3) / AV1 `seq_profile` (0..=2).
pub profile: u8,
pub bit_depth: u32,
pub subsampling: (u8, u8),
pub mono: bool,
/// Picture size: the sample entry's, checked against the bitstream's.
pub size: (u32, u32),
/// AV1: the `av1C` configuration OBUs (the sequence header the decoder is primed with).
pub config_obus: Vec<u8>,
/// AV1: the sequence header's payload; VP9: none.
sequence_header: Option<Vec<u8>>,
/// Colour of the stream when it is known before the first picture (AV1: from the sequence
/// header).
pub color: Option<ColorInfo>,
/// VP9: transfer and primaries from the container (not in the bitstream).
transfer: Option<Transfer>,
primaries: Option<Primaries>,
}
/// The OBUs of a low-overhead AV1 sample: `(type, payload)`, stopping at the first malformed one.
fn obus(data: &[u8]) -> Vec<(u8, &[u8])> {
let mut out = Vec::new();
let mut pos = 0usize;
while let Some(&h) = data.get(pos) {
let (obu_type, ext, has_size) = ((h >> 3) & 0xf, h & 4 != 0, h & 2 != 0);
let mut p = pos + 1 + usize::from(ext);
let size = if has_size {
let (mut v, mut shift, mut n) = (0usize, 0u32, 0usize);
loop {
let Some(&b) = data.get(p + n) else { return out };
v |= usize::from(b & 0x7f).checked_shl(shift).unwrap_or(0);
n += 1;
if b & 0x80 == 0 || n >= 8 {
break;
}
shift += 7;
}
p += n;
v
} else {
data.len().saturating_sub(p)
};
let Some(payload) = data.get(p..p.saturating_add(size)) else { return out };
out.push((obu_type, payload));
pos = p + size;
}
out
}
/// VP9 colour as our decoder reports it: `color_space` and range from the bitstream, transfer and
/// primaries from the container when it says. (Shared with `Vp9Decoder`.)
pub(crate) fn vp9_color(width: u32, height: u32, color_space: u8, full_range: bool, transfer: Option<Transfer>, primaries: Option<Primaries>) -> ColorInfo {
use filmcraft_color::{Matrix, Range};
let mut color = ColorInfo { matrix: filmcraft_frame::default_matrix(width, height), ..ColorInfo::REC709 };
// color_space (7.2.2): 1 BT.601, 2 BT.709, 3 SMPTE-170, 4 SMPTE-240, 5 BT.2020, 7 sRGB.
match color_space {
1 | 3 => color.matrix = Matrix::Bt601,
2 | 4 => color.matrix = Matrix::Bt709,
5 => {
color.matrix = Matrix::Bt2020Ncl;
color.primaries = Primaries::Bt2020;
}
_ => {}
}
if let Some(t) = transfer {
color.transfer = t;
}
if let Some(p) = primaries {
color.primaries = p;
}
if full_range {
color.range = Range::Full;
}
color
}
/// AV1 colour as our decoder reports it. (Shared with `Av1Decoder`.)
pub(crate) fn av1_color(width: u32, height: u32, matrix_coefficients: u8, transfer: u8, primaries: u8, full_range: bool) -> ColorInfo {
let mut color = ColorInfo::REC709;
color.matrix = filmcraft_color::Matrix::from_code(matrix_coefficients).unwrap_or_else(|| filmcraft_frame::default_matrix(width, height));
if let Some(t) = Transfer::from_code(transfer) {
color.transfer = t;
}
color.primaries = match primaries {
9 => Primaries::Bt2020,
12 => Primaries::P3D65,
5 => Primaries::Bt601_625,
6 => Primaries::Bt601_525,
_ => Primaries::Bt709,
};
if full_range {
color.range = filmcraft_color::Range::Full;
}
color
}
impl FrameStreamInfo {
/// The stream info of a `vpcC` / `av1C` sample entry (`None` for other codecs).
pub fn from_entry(e: &SampleEntry) -> Option<Result<Self>> {
let size = e.video.as_ref().map_or((0, 0), |v| (u32::from(v.width), u32::from(v.height)));
match &e.codec {
CodecConfig::Vp9(c) => {
// vpcC chroma_subsampling: 0 / 1 4:2:0, 2 4:2:2, 3 4:4:4
let subsampling = match c.chroma_subsampling {
0 | 1 => (1, 1),
2 => (1, 0),
_ => (0, 0),
};
Some(Ok(Self {
codec: FrameCodec::Vp9,
profile: c.profile,
bit_depth: u32::from(c.bit_depth),
subsampling,
mono: false,
size,
config_obus: Vec::new(),
sequence_header: None,
color: None,
transfer: Transfer::from_code(c.transfer_characteristics),
primaries: primaries_from_code(c.colour_primaries),
}))
}
CodecConfig::Av1(c) => Some(Self::from_av1(c, size)),
_ => None,
}
}
fn from_av1(c: &filmcraft_isobmff::Av1Config, entry_size: (u32, u32)) -> Result<Self> {
let seq =
obus(&c.config_obus).into_iter().find(|(t, _)| *t == 1).map(|(_, p)| p).ok_or_else(|| CodecError::Decode("av1C has no sequence header".into()))?;
let h = filmcraft_av1::SequenceHeader::parse(seq).map_err(|e| CodecError::Decode(e.to_string()))?;
let size = (h.max_frame_width, h.max_frame_height);
if entry_size != (0, 0) && entry_size != size {
// the sample entry and the sequence header disagree: leave the stream to our decoder
return Err(CodecError::Decode(format!("sample entry says {}x{}, the sequence header {}x{}", entry_size.0, entry_size.1, size.0, size.1)));
}
let cc = &h.color;
Ok(Self {
codec: FrameCodec::Av1,
profile: h.profile,
bit_depth: u32::from(cc.bit_depth),
subsampling: (cc.subsampling_x, cc.subsampling_y),
mono: cc.mono_chrome,
size,
config_obus: c.config_obus.clone(),
sequence_header: Some(seq.to_vec()),
color: Some(av1_color(size.0, size.1, cc.matrix_coefficients, cc.transfer_characteristics, cc.color_primaries, cc.color_range)),
transfer: None,
primaries: None,
})
}
/// Whether `sample` has a sequence header OBU of its own (AV1).
pub fn carries_sequence_header(&self, sample: &[u8]) -> bool {
self.codec == FrameCodec::Av1 && obus(sample).iter().any(|(t, _)| *t == 1)
}
/// A description made by hand (tests of what a backend declines).
#[doc(hidden)]
pub fn for_tests(codec: FrameCodec, profile: u8, bit_depth: u32, subsampling: (u8, u8), size: (u32, u32)) -> Self {
Self {
codec,
profile,
bit_depth,
subsampling,
mono: false,
size,
config_obus: Vec::new(),
sequence_header: None,
color: None,
transfer: None,
primaries: None,
}
}
/// Whether decoding can start at `sample` (a VP9 key frame / an AV1 temporal unit whose first
/// frame is a shown key frame): the software decoders' own test.
pub fn is_random_access(&self, sample: &[u8]) -> bool {
match self.codec {
FrameCodec::Vp9 => filmcraft_vp9::is_keyframe(sample),
FrameCodec::Av1 => filmcraft_av1::is_key_frame_unit(sample),
}
}
/// The picture parameters a random-access `sample` declares (a VP9 key frame header; an AV1
/// sample carrying a sequence header), or `None`.
pub fn picture_params(&self, sample: &[u8]) -> Option<PictureParams> {
match self.codec {
FrameCodec::Vp9 => {
let k = filmcraft_vp9::keyframe_info(sample)?;
let color = vp9_color(k.width, k.height, k.color.color_space, k.color.full_range, self.transfer, self.primaries);
Some(PictureParams {
size: (k.width, k.height),
bit_depth: k.bit_depth,
subsampling: (u8::from(k.subsampling_x), u8::from(k.subsampling_y)),
color,
})
}
FrameCodec::Av1 => {
let seq = obus(sample).into_iter().find(|(t, _)| *t == 1).map(|(_, p)| p)?;
let h = filmcraft_av1::SequenceHeader::parse(seq).ok()?;
let cc = &h.color;
let size = (h.max_frame_width, h.max_frame_height);
let color = av1_color(size.0, size.1, cc.matrix_coefficients, cc.transfer_characteristics, cc.color_primaries, cc.color_range);
Some(PictureParams { size, bit_depth: u32::from(cc.bit_depth), subsampling: (cc.subsampling_x, cc.subsampling_y), color })
}
}
}
/// Whether `sample` declares parameters the hardware session was not set up for: an AV1
/// sequence header that differs from `av1C`'s, a VP9 key frame of another size, bit depth or
/// chroma format. The decoder then hands the stream to our software decoder.
pub fn parameters_changed(&self, sample: &[u8]) -> bool {
match self.codec {
FrameCodec::Av1 => obus(sample).into_iter().any(|(t, p)| t == 1 && self.sequence_header.as_deref() != Some(p)),
FrameCodec::Vp9 => {
self.picture_params(sample).is_some_and(|p| p.size != self.size || p.bit_depth != self.bit_depth || p.subsampling != self.subsampling)
}
}
}
}
+18 -5
View File
@@ -18,9 +18,11 @@
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
pub mod apv;
pub mod audio;
pub mod gop;
pub mod hw;
mod hw_frame;
pub mod mkv;
pub mod mp4;
pub mod mpeg;
@@ -30,8 +32,9 @@ pub mod video;
use std::sync::{Arc, RwLock};
pub use apv::ApvSource;
pub use audio::AudioFileSource;
pub use gop::{GopStats, gop_stats};
pub use gop::{FRAME_BUDGET, GopStats, cached_bytes, gop_stats, live_decoders};
pub use mkv::MkvSource;
pub use mp4::Mp4Source;
pub use mpeg::MpegSource;
@@ -65,11 +68,12 @@ impl From<CodecError> for filmcraft_media::MediaError {
}
/// Our own (pure-Rust) decoders, in the order they are tried.
const BUILTIN_FACTORIES: [VideoDecoderFactory; 8] = [
const BUILTIN_FACTORIES: [VideoDecoderFactory; 9] = [
video::h264_factory,
video::hevc_factory,
video::vp9_factory,
video::av1_factory,
video::apv_factory,
video::prores_factory,
video::dnx_factory,
video::mjpeg_factory,
@@ -89,6 +93,11 @@ pub fn register_video_decoder(f: VideoDecoderFactory) {
}
}
/// Whether `f` is among the registered video decoder factories (startup diagnostics, tests).
pub fn video_decoder_registered(f: VideoDecoderFactory) -> bool {
factories().read().unwrap_or_else(|e| e.into_inner()).iter().any(|x| std::ptr::fn_addr_eq(*x, f))
}
/// Create a decoder for a sample entry.
pub fn make_video_decoder(entry: &filmcraft_isobmff::SampleEntry) -> Result<Box<dyn VideoDecoder>> {
let g = factories().read().unwrap_or_else(|e| e.into_inner());
@@ -112,14 +121,14 @@ pub fn software_video_decoder(entry: &filmcraft_isobmff::SampleEntry) -> Result<
}
/// Openers for the engine's media pool (MP4/MOV, Matroska/WebM, MXF, Ogg Opus/Vorbis, MPEG TS/PS and
/// MPEG-1/2 video elementary streams, standalone audio).
/// MPEG-1/2 video elementary streams, APV raw bitstreams, standalone audio).
pub fn openers() -> Vec<filmcraft_media::Opener> {
vec![mp4::opener, mkv::opener, mxf::opener, ogg::opener, mpeg::opener, audio::opener]
vec![mp4::opener, mkv::opener, mxf::opener, ogg::opener, mpeg::opener, apv::opener, audio::opener]
}
fn reader_registry() -> &'static RwLock<Vec<filmcraft_media::ReaderOpener>> {
static R: std::sync::OnceLock<RwLock<Vec<filmcraft_media::ReaderOpener>>> = std::sync::OnceLock::new();
R.get_or_init(|| RwLock::new(vec![mp4::reader_opener, mkv::reader_opener, mxf::reader_opener, ogg::reader_opener, mpeg::reader_opener]))
R.get_or_init(|| RwLock::new(vec![mp4::reader_opener, mkv::reader_opener, mxf::reader_opener, ogg::reader_opener, mpeg::reader_opener, apv::reader_opener]))
}
/// Openers that read containers through a [`filmcraft_media::ByteReader`] (index now, samples on
@@ -164,5 +173,9 @@ pub fn open_bytes(name: &str, bytes: Arc<[u8]>) -> std::result::Result<filmcraft
filmcraft_media::open_bytes(name, bytes, &openers())
}
#[cfg(test)]
mod audio_timing_tests;
#[cfg(test)]
mod rounded_pts_tests;
#[cfg(test)]
mod tests;
+171 -87
View File
@@ -8,6 +8,9 @@
//! subtracts `CodecDelay` from timestamps, so the pre-skip samples land before zero and are never
//! read; random access decodes `SeekPreRoll` (at least [`crate::audio::OPUS_PRE_ROLL`]) of preceding
//! packets before the target. Packet starts are accumulated from TOC durations (see [`opus_starts`]).
//!
//! AAC, MPEG audio and AC-3 packets each decode to a fixed number of samples, which millisecond
//! timestamps can't express: their packets run on from one another (see [`audio_starts`]).
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
@@ -35,51 +38,94 @@ struct AudioState {
last_decoded: Option<usize>,
}
/// One playable audio track with its own decoder state and timing tables.
struct MkvAudio {
/// Index into `MkvFile::tracks`.
track: usize,
state: Mutex<AudioState>,
/// Packet start positions in source sample frames.
starts: Vec<i64>,
/// Decoder pre-roll after a seek, in source sample frames (0: prime with one packet).
preroll: i64,
}
pub struct MkvSource {
info: MediaInfo,
bytes: crate::Src,
file: MkvFile,
vtrack: Option<usize>,
atrack: Option<usize>,
/// Video codec as an ISO-BMFF sample entry (for the decoder factories).
ventry: Option<SampleEntry>,
video: GopCache,
audio: Mutex<AudioState>,
/// Audio packet start positions in source sample frames.
audio_starts: Vec<i64>,
/// Decoder pre-roll after a seek, in source sample frames (0: prime with one packet).
audio_preroll: i64,
/// The playable audio tracks in file order: `info.audio_streams[k]` describes `audios[k]`.
audios: Vec<MkvAudio>,
}
/// Audio packet start positions in source sample frames.
///
/// Block timestamps are quantised to `TimestampScale` (usually 1 ms), so they are only accurate to a
/// tick: a 1024-sample AAC frame at 48 kHz lasts 21.33 ms. Packets that all decode to the same
/// length (AAC, MPEG audio, AC-3) therefore run on from one another, resynchronising to the
/// timestamp only across gaps of more than half a packet (and more than two ticks); see
/// [`crate::audio::contiguous_starts`]. Opus packets: [`opus_starts`]. Other codecs start at their
/// timestamps.
fn audio_starts(file: &MkvFile, bytes: &crate::Src, ti: usize, rate: i64) -> Vec<i64> {
use crate::audio::{FixedFrames, PacketTime, contiguous_starts, fixed_packet_samples};
let Some(t) = file.tracks.get(ti) else { return Vec::new() };
if let Some(h) = opus_head(&t.codec) {
return opus_starts(file, bytes, ti, &h, rate);
}
let (n, d) = tb(t);
let at = |pts: i64| (pts as i128 * n as i128 * rate as i128 / d as i128) as i64;
let first = || file.read_sample(bytes, ti, 0).unwrap_or_default();
let out_rate = u32::try_from(rate).unwrap_or(0);
let frame = match &t.codec {
Codec::Aac { asc } => fixed_packet_samples(FixedFrames::Aac(asc), &[], out_rate),
Codec::Mp3 | Codec::Mp2 => fixed_packet_samples(FixedFrames::MpegAudio, &first(), out_rate),
Codec::Ac3 => fixed_packet_samples(FixedFrames::Ac3, &first(), out_rate),
_ => None,
};
let Some(frame) = frame else {
return t.samples.iter().map(|s| at(s.pts)).collect();
};
let packets = t.samples.iter().map(|s| PacketTime { stamp: at(s.pts), stamped: own_stamp(s), samples: Some(frame) });
contiguous_starts(packets, (frame / 2).max(stamp_tolerance(t, rate)))
}
/// Sample-exact Opus packet start positions (48 kHz frames, pre-skip removed).
///
/// Block timestamps are quantised to `TimestampScale` (usually 1 ms) and the demuxer subtracts a
/// rounded `CodecDelay`, so they are only accurate to a tick. Starts are therefore accumulated from
/// each packet's TOC duration, resynchronising to the timestamp only across real gaps (more than
/// two ticks off). Pre-skip is the exact `CodecDelay` (or the header's pre-skip when it is absent).
/// The demuxer also subtracts a rounded `CodecDelay` from the timestamps. Starts are accumulated
/// from each packet's TOC duration, resynchronising to the timestamp only across real gaps (more
/// than two ticks off). Pre-skip is the exact `CodecDelay` (or the header's pre-skip when it is
/// absent).
fn opus_starts(file: &MkvFile, bytes: &crate::Src, ti: usize, head: &filmcraft_opus::OpusHead, rate: i64) -> Vec<i64> {
let t = &file.tracks[ti];
let Some(t) = file.tracks.get(ti) else { return Vec::new() };
let (n, d) = tb(t);
let scale_ns = (n as i128 * 1_000_000_000 / d as i128).max(1);
let delay_ns = t.codec_delay_ns as i128;
// The demuxer's rounding of CodecDelay to ticks (half away from zero), undone here.
let delay_ticks = ((delay_ns + scale_ns / 2) / scale_ns) as i64;
let skip = if t.codec_delay_ns > 0 { (delay_ns * rate as i128 / 1_000_000_000) as i64 } else { head.pre_skip as i64 };
let at = |pts: i64| ((pts + delay_ticks) as i128 * n as i128 * rate as i128 / d as i128) as i64 - skip;
let tolerance = (2 * scale_ns * rate as i128 / 1_000_000_000) as i64 + 1;
let mut starts = Vec::with_capacity(t.samples.len());
let mut next: Option<i64> = None;
for (i, s) in t.samples.iter().enumerate() {
let stamped = at(s.pts);
let start = match next {
Some(p) if (p - stamped).abs() <= tolerance => p,
_ => stamped,
};
starts.push(start);
let dur = file.read_sample(bytes, ti, i).ok().and_then(|p| crate::audio::opus_packet_samples(&p));
next = dur.map(|k| start + k as i64);
}
starts
let at = |pts: i64| ((pts.saturating_add(delay_ticks) as i128 * n as i128 * rate as i128 / d as i128) as i64).saturating_sub(skip);
let packets = t.samples.iter().enumerate().map(|(i, s)| crate::audio::PacketTime {
stamp: at(s.pts),
stamped: own_stamp(s),
samples: file.read_sample(bytes, ti, i).ok().and_then(|p| crate::audio::opus_packet_samples(&p)).map(|k| k as i64),
});
crate::audio::contiguous_starts(packets, stamp_tolerance(t, rate))
}
/// Two timestamp ticks in sample frames at `rate`, plus one: how far a packet's Matroska timestamp
/// can be from its exact start.
fn stamp_tolerance(t: &filmcraft_matroska::Track, rate: i64) -> i64 {
let (n, d) = tb(t);
i64::try_from(2 * n as i128 * rate as i128 / d as i128).unwrap_or(i64::MAX).saturating_add(1)
}
/// Whether a sample's timestamp is its own: frames after the first in a laced block without a
/// duration repeat the block's timestamp.
fn own_stamp(s: &filmcraft_matroska::Sample) -> bool {
s.lace == 0 || s.duration > 0
}
/// The parsed `OpusHead` of an `A_OPUS` track.
@@ -100,9 +146,12 @@ fn to_tick(t: &filmcraft_matroska::Track, pts: i64) -> Tick {
Tick::from_rational(pts * n, 1, d)
}
/// A time as the track timestamp to look a frame up by: rounded to the nearest tick, as muxers
/// round frame times to `TimestampScale` (usually 1 ms). Flooring skipped every frame whose
/// timestamp was rounded up (every third frame at 30 fps).
fn from_tick(t: &filmcraft_matroska::Track, time: Tick) -> i64 {
let (n, d) = tb(t);
time.to_rational_floor(n, d)
time.to_rational_round(n, d)
}
/// VP9 configuration from the Matroska `CodecPrivate` feature list (ID / length / value triples:
@@ -142,6 +191,7 @@ fn sample_entry(c: &Codec, private: &[u8], v: Option<&filmcraft_matroska::VideoI
Some(match c {
Codec::Vp9 { private } => SampleEntry::video(FourCc(*b"vp09"), CodecConfig::Vp9(vp9_config(private, v)), w, h),
Codec::Av1 { av1c } => SampleEntry::video(FourCc(*b"av01"), CodecConfig::Av1(filmcraft_isobmff::Av1Config::parse(av1c).unwrap_or_default()), w, h),
Codec::Apv { apvc } => SampleEntry::apv(filmcraft_isobmff::ApvConfig::parse(apvc).unwrap_or_default(), w, h),
Codec::Avc { avcc } => SampleEntry::avc(AvcConfig::parse(avcc).ok()?, w, h),
Codec::Hevc { hvcc } => SampleEntry::hevc(HevcConfig::parse(hvcc).ok()?, w, h),
Codec::ProRes { fourcc } => SampleEntry::prores(FourCc(fourcc.unwrap_or(*b"apcn")), w, h),
@@ -203,6 +253,10 @@ fn codec_label(c: &Codec) -> String {
Codec::Vp8 => "VP8".into(),
Codec::Vp9 { .. } => "VP9".into(),
Codec::Av1 { .. } => "AV1".into(),
Codec::Apv { apvc } => match filmcraft_isobmff::ApvConfig::parse(apvc).ok().and_then(|a| filmcraft_apv::Profile::from_idc(a.profile_idc)) {
Some(p) => p.name().into(),
None => "APV".into(),
},
Codec::ProRes { .. } => "Apple ProRes".into(),
Codec::Mjpeg => "Motion JPEG".into(),
Codec::Aac { .. } => "AAC".into(),
@@ -228,8 +282,16 @@ impl MkvSource {
let bytes = crate::Src(reader);
let file = filmcraft_matroska::open(&bytes).map_err(|e| CodecError::Container(e.to_string()))?;
let vtrack = file.tracks.iter().position(|t| t.kind == TrackKind::Video && !t.samples.is_empty());
let atrack = file.tracks.iter().position(|t| t.kind == TrackKind::Audio && !t.samples.is_empty());
if vtrack.is_none() && atrack.is_none() {
// every audio track is a stream (the track count comes from the file: capped)
let atracks: Vec<usize> = file
.tracks
.iter()
.enumerate()
.filter(|(_, t)| t.kind == TrackKind::Audio && !t.samples.is_empty())
.map(|(i, _)| i)
.take(filmcraft_media::MAX_AUDIO_STREAMS)
.collect();
if vtrack.is_none() && atracks.is_empty() {
return Err(CodecError::Unsupported("no playable tracks".into()));
}
let mut explicit_color = None;
@@ -271,21 +333,24 @@ impl MkvSource {
hdr: v.colour.as_ref().and_then(hdr_metadata),
}
});
let audio = atrack.map(|i| {
let t = &file.tracks[i];
let a = t.audio.clone().unwrap_or_default();
let mut rate = a.output_sampling_frequency.unwrap_or(a.sampling_frequency).round().max(1.0) as u32;
let mut channels = (a.channels as u32).max(1);
if let Some(h) = opus_head(&t.codec) {
rate = crate::audio::OPUS_RATE;
channels = h.channels as u32;
}
AudioStreamInfo { sample_rate: rate, channels, codec: codec_label(&t.codec), bits_per_sample: a.bit_depth.map(|b| b as u32) }
});
let audio_streams: Vec<AudioStreamInfo> = atracks
.iter()
.filter_map(|&i| file.tracks.get(i))
.map(|t| {
let a = t.audio.clone().unwrap_or_default();
let mut rate = a.output_sampling_frequency.unwrap_or(a.sampling_frequency).round().max(1.0) as u32;
let mut channels = (a.channels as u32).max(1);
if let Some(h) = opus_head(&t.codec) {
rate = crate::audio::OPUS_RATE;
channels = h.channels as u32;
}
AudioStreamInfo { sample_rate: rate, channels, codec: codec_label(&t.codec), bits_per_sample: a.bit_depth.map(|b| b as u32) }
})
.collect();
let duration = match file.duration_ns() {
Some(ns) => Tick::from_rational(ns as i64, 1, 1_000_000_000),
None => vtrack
.or(atrack)
.or(atracks.first().copied())
.map(|i| {
let t = &file.tracks[i];
let end = t.samples.iter().map(|s| s.pts + s.duration as i64).max().unwrap_or(0);
@@ -298,48 +363,31 @@ impl MkvSource {
kind: if video.is_some() { MediaKind::Movie } else { MediaKind::AudioOnly },
duration,
video,
audio,
audio_streams,
container: if file.is_webm() { "WebM".into() } else { "Matroska".into() },
start_timecode: None,
file_size: Some(bytes.0.len()),
};
let audio_starts = atrack
.map(|i| {
let t = &file.tracks[i];
let rate = info.audio.as_ref().map_or(48_000, |a| a.sample_rate) as i64;
let (n, d) = tb(t);
let at = |pts: i64| (pts as i128 * n as i128 * rate as i128 / d as i128) as i64;
match opus_head(&t.codec) {
Some(h) => opus_starts(&file, &bytes, i, &h, rate),
None => t.samples.iter().map(|s| at(s.pts)).collect(),
}
})
.unwrap_or_default();
let audio_preroll = atrack
.map(|i| {
let t = &file.tracks[i];
let rate = info.audio.as_ref().map_or(48_000, |a| a.sample_rate) as i64;
match t.codec {
// `info.audio_streams[k]` was made from `atracks[k]`, so the two stay index-aligned
let audios: Vec<MkvAudio> = atracks
.iter()
.zip(&info.audio_streams)
.filter_map(|(&i, ainfo)| {
let t = file.tracks.get(i)?;
let rate = ainfo.sample_rate as i64;
let starts = audio_starts(&file, &bytes, i, rate);
let preroll = match t.codec {
Codec::Opus { .. } => {
let container = (t.seek_pre_roll_ns as u128 * rate as u128).div_ceil(1_000_000_000) as i64;
container.max(crate::audio::OPUS_PRE_ROLL as i64 * rate / 48_000)
}
_ => 0,
}
};
let state = Mutex::new(AudioState { decoder: None, packets: HashMap::new(), order: Vec::new(), last_decoded: None });
Some(MkvAudio { track: i, state, starts, preroll })
})
.unwrap_or(0);
Ok(Self {
info,
bytes,
file,
vtrack,
atrack,
ventry,
video: GopCache::new(explicit_color).with_rotation(rotation),
audio: Mutex::new(AudioState { decoder: None, packets: HashMap::new(), order: Vec::new(), last_decoded: None }),
audio_starts,
audio_preroll,
})
.collect();
Ok(Self { info, bytes, file, vtrack, ventry, video: GopCache::new(explicit_color).with_rotation(rotation), audios })
}
fn read(&self, track: usize, i: usize) -> crate::Result<Vec<u8>> {
@@ -365,13 +413,14 @@ impl MkvSource {
}
}
fn audio_packet(&self, st: &mut AudioState, i: usize) -> crate::Result<Arc<Vec<Vec<f32>>>> {
/// Decoded packet `i` of audio stream `a` (`ainfo` is its `AudioStreamInfo`; `st` its decoder state).
fn audio_packet(&self, a: &MkvAudio, ainfo: &AudioStreamInfo, st: &mut AudioState, i: usize) -> crate::Result<Arc<Vec<Vec<f32>>>> {
if let Some(p) = st.packets.get(&i) {
return Ok(p.clone());
}
let ti = self.atrack.ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
let track = &self.file.tracks[ti];
let ainfo = self.info.audio.as_ref().ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
let ti = a.track;
let track = self.file.tracks.get(ti).ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
let pk_start = a.starts.get(i).copied().ok_or_else(|| CodecError::Container("audio packet out of range".into()))?;
let data = self.read(ti, i)?;
let decoded = match &track.codec {
Codec::Pcm { float, big_endian, bits } => {
@@ -394,8 +443,8 @@ impl MkvSource {
if st.last_decoded.is_none_or(|l| l + 1 != i) {
let d = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?;
d.reset();
let from = if self.audio_preroll > 0 {
self.audio_starts.partition_point(|&x| x <= self.audio_starts[i] - self.audio_preroll).saturating_sub(1)
let from = if a.preroll > 0 {
a.starts.partition_point(|&x| x <= pk_start.saturating_sub(a.preroll)).saturating_sub(1)
} else {
i.saturating_sub(1)
};
@@ -405,7 +454,7 @@ impl MkvSource {
}
}
}
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, self.audio_starts[i].max(0) as u64);
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, pk_start.max(0) as u64);
st.last_decoded = Some(i);
r.unwrap_or_default()
}
@@ -465,23 +514,27 @@ impl MediaSource for MkvSource {
}
fn audio(&self, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
let ti = self.atrack.ok_or(MediaError::NoStream("audio"))?;
let n_packets = self.file.tracks[ti].samples.len();
let ainfo = self.info.audio.as_ref().ok_or(MediaError::NoStream("audio"))?;
self.audio_stream(0, start, frames, sample_rate)
}
fn audio_stream(&self, stream: usize, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
let a = self.audios.get(stream).ok_or(MediaError::NoStream("audio"))?;
let n_packets = self.file.tracks.get(a.track).ok_or(MediaError::NoStream("audio"))?.samples.len();
let ainfo = self.info.audio_streams.get(stream).ok_or(MediaError::NoStream("audio"))?;
let src_rate = ainfo.sample_rate;
let ch = ainfo.channels.max(1) as usize;
let ratio = src_rate as f64 / sample_rate as f64;
let s0 = (start as f64 * ratio).floor() as i64;
let need = (frames as f64 * ratio).ceil() as i64 + 2;
let mut src: Vec<Vec<f32>> = vec![vec![0.0; need.max(0) as usize]; ch];
let mut st = self.audio.lock().unwrap_or_else(|e| e.into_inner());
let mut i = self.audio_starts.partition_point(|&x| x <= s0).saturating_sub(1);
let mut st = a.state.lock().unwrap_or_else(|e| e.into_inner());
let mut i = a.starts.partition_point(|&x| x <= s0).saturating_sub(1);
while i < n_packets {
let pk_start = self.audio_starts[i];
let Some(&pk_start) = a.starts.get(i) else { break };
if pk_start >= s0 + need {
break;
}
let pk = self.audio_packet(&mut st, i)?;
let pk = self.audio_packet(a, ainfo, &mut st, i)?;
for (c, dst) in src.iter_mut().enumerate() {
let Some(chan) = pk.get(c.min(pk.len().saturating_sub(1))) else { continue };
for (k, v) in chan.iter().enumerate() {
@@ -524,3 +577,34 @@ pub fn reader_opener(name: &str, head: &[u8], reader: &filmcraft_media::SharedRe
}
Some(MkvSource::open_reader(name, reader.clone()).map(|s| Arc::new(s) as SharedSource).map_err(Into::into))
}
#[cfg(test)]
mod multi_audio_tests {
use filmcraft_matroska::{MkvWriter, MuxOptions, TrackKind, TrackSpec};
use filmcraft_media::MediaSource;
#[test]
fn every_audio_track_is_inspected_and_decoded_independently() {
let tracks: Vec<TrackSpec> = [(48_000.0, "first"), (24_000.0, "second")]
.into_iter()
.map(|(rate, name)| {
let mut t = TrackSpec::new(TrackKind::Audio, "A_PCM/INT/LIT");
t.audio = Some((rate, 1, Some(16)));
t.name = Some(name.into());
t
})
.collect();
let mut writer = MkvWriter::new(std::io::Cursor::new(Vec::new()), tracks, MuxOptions::default()).unwrap();
for (stream, value) in [(0, 8192i16), (1, -16384)] {
let data: Vec<u8> = (0..100).flat_map(|_| value.to_le_bytes()).collect();
writer.write_frame(stream, 0, true, &data, Some(4_000_000)).unwrap();
}
let source = super::MkvSource::open("two.mkv", writer.finish().unwrap().into_inner().into()).unwrap();
assert_eq!(source.info().audio_streams.iter().map(|a| a.sample_rate).collect::<Vec<_>>(), vec![48_000, 24_000]);
let primary = source.audio(0, 50, 48_000).unwrap();
let secondary = source.audio_stream(1, 0, 50, 48_000).unwrap();
assert!((primary.channels[0][20] - 0.25).abs() < 0.001);
assert!((secondary.channels[0][20] + 0.5).abs() < 0.001);
assert!(source.audio_stream(2, 0, 50, 48_000).is_err());
}
}
+430 -105
View File
@@ -18,6 +18,9 @@ use crate::gop::{GopCache, VideoSamples};
use crate::video::VideoDecoder;
use crate::{CodecError, make_video_decoder};
/// Most bytes read from one AAC access unit when probing for HE-AAC at open (8 channels × 6144 bits).
const MAX_AAC_PROBE_UNIT: usize = 8 * 6144 / 8;
struct AudioState {
decoder: Option<PacketDecoder>,
/// Decoded packets by sample index.
@@ -26,26 +29,67 @@ struct AudioState {
last_decoded: Option<usize>,
}
/// One playable audio track with its own decoder state and timing tables.
struct Mp4Audio {
/// Index into `Mp4File::tracks`.
track: usize,
state: Mutex<AudioState>,
/// Packet start positions in source sample frames (see [`audio_starts`]).
starts: Vec<i64>,
/// Presentation offset of the track in its timescale (edit list, or Opus pre-skip).
offset: i64,
/// Decoder pre-roll after a seek, in source sample frames (0: prime with one packet).
preroll: i64,
}
pub struct Mp4Source {
info: MediaInfo,
bytes: crate::Src,
file: Mp4File,
vtrack: Option<usize>,
atrack: Option<usize>,
video: GopCache,
audio: Mutex<AudioState>,
/// Cumulative sample start frames for the audio track (for packet lookup).
audio_starts: Vec<i64>,
/// Presentation offset of the audio track in its timescale (edit list, or Opus pre-skip).
audio_offset: i64,
/// Decoder pre-roll after a seek, in audio track timescale units (0: prime with one packet).
audio_preroll: i64,
/// The playable audio tracks in file order: `info.audio_streams[k]` describes `audios[k]`.
audios: Vec<Mp4Audio>,
}
pub fn sniff(b: &[u8]) -> bool {
b.len() >= 12 && matches!(&b[4..8], b"ftyp" | b"moov" | b"mdat" | b"wide" | b"free" | b"skip")
}
/// Audio packet start positions in source sample frames at `rate`: the running total of the sample
/// durations, except that packets which all decode to the same length (AAC, MPEG audio, AC-3) run
/// on from one another, resynchronising to the total only across gaps of more than half a packet.
/// A remux from Matroska (OBS's) carries the millisecond rounding of its timestamps into the
/// durations: 1008, 1008, 1056… for 1024-sample AAC frames at 48 kHz (see
/// [`crate::audio::contiguous_starts`]).
fn audio_starts(file: &Mp4File, bytes: &crate::Src, ti: usize, rate: u32) -> Vec<i64> {
use crate::audio::{FixedFrames, PacketTime, contiguous_starts, fixed_packet_samples};
let Some(t) = file.tracks.get(ti) else { return Vec::new() };
let ts = i128::from(t.timescale.max(1));
let mut total = 0i128;
let stamps: Vec<i64> = t
.samples
.iter()
.map(|s| {
let at = total * i128::from(rate) / ts;
total += i128::from(s.duration);
i64::try_from(at).unwrap_or(i64::MAX)
})
.collect();
let first = || file.read_sample(bytes, ti, 0).unwrap_or_default();
let frame = match t.entries.first().map(|e| &e.codec) {
Some(CodecConfig::Aac(a)) => fixed_packet_samples(FixedFrames::Aac(&a.asc), &[], rate),
Some(CodecConfig::Mp3) => fixed_packet_samples(FixedFrames::MpegAudio, &first(), rate),
Some(CodecConfig::Ac3 { .. }) => fixed_packet_samples(FixedFrames::Ac3, &first(), rate),
_ => None,
};
let Some(frame) = frame else { return stamps };
// samples per track timescale unit, rounded up: how coarse the running total is
let unit = i64::try_from((i128::from(rate) + ts - 1) / ts).unwrap_or(i64::MAX);
let packets = stamps.iter().map(|&stamp| PacketTime { stamp, stamped: true, samples: Some(frame) });
contiguous_starts(packets, (frame / 2).max(unit.saturating_mul(2).saturating_add(1)))
}
/// HDR static metadata from the sample entry's `mdcv` / `clli` boxes (0 = unknown).
fn hdr_metadata(md: Option<&filmcraft_isobmff::MasteringDisplay>, cll: Option<(u16, u16)>) -> Option<filmcraft_color::HdrMetadata> {
if md.is_none() && cll.is_none() {
@@ -62,7 +106,7 @@ fn hdr_metadata(md: Option<&filmcraft_isobmff::MasteringDisplay>, cll: Option<(u
fn color_from(entry: &filmcraft_isobmff::SampleEntry, w: u32, h: u32) -> ColorInfo {
let mut c = ColorInfo { matrix: filmcraft_frame::default_matrix(w, h), transfer: Transfer::Bt709, primaries: Primaries::Bt709, range: Range::Limited };
// VP9 carries its colour description in vpcC (used when there is no colr box).
// VP9 and APV carry their colour description in vpcC / apvC (used when there is no colr box).
let vpc = match &entry.codec {
CodecConfig::Vp9(v) => Some(filmcraft_isobmff::ColorInfo::Nclx {
primaries: v.colour_primaries as u16,
@@ -70,6 +114,12 @@ fn color_from(entry: &filmcraft_isobmff::SampleEntry, w: u32, h: u32) -> ColorIn
matrix: v.matrix_coefficients as u16,
full_range: v.full_range,
}),
CodecConfig::Apv(a) if a.color_description_present => Some(filmcraft_isobmff::ColorInfo::Nclx {
primaries: a.color_primaries as u16,
transfer: a.transfer_characteristics as u16,
matrix: a.matrix_coefficients as u16,
full_range: a.full_range,
}),
_ => None,
};
if let Some(col) = entry.video.as_ref().and_then(|v| v.color.as_ref()).or(vpc.as_ref()) {
@@ -111,8 +161,10 @@ impl Mp4Source {
// files can lack them: indexing `entries[0]` or dividing by the timescale used to panic).
let playable = |t: &&filmcraft_isobmff::Track, kind| t.kind == kind && !t.samples.is_empty() && !t.entries.is_empty() && t.timescale > 0;
let vtrack = file.tracks.iter().position(|t| playable(&t, TrackKind::Video));
let atrack = file.tracks.iter().position(|t| playable(&t, TrackKind::Audio));
if vtrack.is_none() && atrack.is_none() {
// every audio track is a stream (the track count comes from the file: capped)
let atracks: Vec<usize> =
file.tracks.iter().enumerate().filter(|(_, t)| playable(t, TrackKind::Audio)).map(|(i, _)| i).take(filmcraft_media::MAX_AUDIO_STREAMS).collect();
if vtrack.is_none() && atracks.is_empty() {
return Err(CodecError::Unsupported("no playable tracks".into()));
}
let mut color = ColorInfo::REC709;
@@ -148,7 +200,8 @@ impl Mp4Source {
codec: codec_label(&entry.codec),
pixel_format: pixfmt_label(&entry.codec),
color,
has_alpha: matches!(&entry.codec, CodecConfig::ProRes { fourcc } if fourcc.0 == *b"ap4h" || fourcc.0 == *b"ap4x"),
// ProRes 4444 alpha comes from its first frame header (`open_reader`)
has_alpha: matches!(&entry.codec, CodecConfig::Apv(a) if a.chroma_format_idc == 4),
bitrate,
hdr: entry.video.as_ref().and_then(|v| hdr_metadata(v.mastering_display.as_ref(), v.content_light)),
};
@@ -170,35 +223,26 @@ impl Mp4Source {
info.has_alpha = h.alpha;
}
}
if matches!(entry.codec, CodecConfig::ProRes { .. }) {
// a 4444 fourcc doesn't mean the frames carry alpha (alpha_channel_type does): read the first frame header
let s0 = &t.samples[0];
if let Some(h) = filmcraft_media::reader::read_range(&*bytes.0, s0.offset, s0.size as usize).ok().and_then(|d| filmcraft_prores::probe(&d).ok())
{
info.has_alpha = h.alpha != filmcraft_prores::AlphaType::None;
let sub = match h.chroma {
filmcraft_prores::ChromaFormat::Yuv422 => "4:2:2 10-bit",
filmcraft_prores::ChromaFormat::Yuv444 => "4:4:4 12-bit",
};
info.pixel_format = format!("{} {sub}", if info.has_alpha { "YUVA" } else { "YUV" });
}
}
info
});
let audio = atrack.map(|i| {
let t = &file.tracks[i];
let entry = &t.entries[0];
let ap = entry.audio.clone().unwrap_or_default();
let (rate, ch, bits) = match &entry.codec {
CodecConfig::Aac(a) => (
if a.sample_rate > 0 { a.sample_rate } else { ap.sample_rate as u32 },
if a.channel_config > 0 { a.channel_config as u32 } else { ap.channels },
None,
),
CodecConfig::Pcm(p) => (p.sample_rate as u32, p.channels, Some(p.bits as u32)),
CodecConfig::Opus(o) => (crate::audio::OPUS_RATE, (o.output_channels as u32).max(1), None),
_ => (if ap.sample_rate > 0.0 { ap.sample_rate as u32 } else { t.timescale }, ap.channels.max(1), None),
};
AudioStreamInfo { sample_rate: rate.max(1), channels: ch.max(1), codec: codec_label(&entry.codec), bits_per_sample: bits }
});
let duration = {
let v = vtrack.map(|i| {
let t = &file.tracks[i];
Tick::from_rational(t.duration as i64, 1, t.timescale as i64)
});
let a = atrack.map(|i| {
let t = &file.tracks[i];
Tick::from_rational(t.duration as i64, 1, t.timescale as i64)
});
v.or(a).unwrap_or_default()
};
// one entry per playable audio track (`playable` guarantees a sample description); a track
// that somehow has none is dropped so `audio_streams` and `audios` stay index-aligned
let (atracks, audio_streams): (Vec<usize>, Vec<AudioStreamInfo>) =
atracks.into_iter().filter_map(|i| Some((i, audio_stream_info(file.tracks.get(i)?, &bytes)?))).unzip();
let duration = vtrack.or(atracks.first().copied()).and_then(|i| file.tracks.get(i)).map(|t| presentation_duration(&file, t)).unwrap_or_default();
let start_timecode = file.tracks.iter().find_map(|t| match t.codec() {
Some(CodecConfig::Timecode(tc)) => tc.start_frame.map(|f| f as i64),
_ => None,
@@ -208,50 +252,31 @@ impl Mp4Source {
kind: if video.is_some() { MediaKind::Movie } else { MediaKind::AudioOnly },
duration,
video,
audio,
audio_streams,
container: if file.is_quicktime { "QuickTime".into() } else { "MPEG-4".into() },
start_timecode,
file_size: Some(bytes.0.len()),
};
let audio_starts = atrack
.map(|i| {
let t = &file.tracks[i];
let mut acc = 0i64;
t.samples
.iter()
.map(|s| {
let st = acc;
acc += s.duration as i64;
st
})
.collect()
})
.unwrap_or_default();
let (audio_offset, audio_preroll) = atrack
.map(|i| {
let t = &file.tracks[i];
let audios: Vec<Mp4Audio> = atracks
.iter()
.zip(&info.audio_streams)
.filter_map(|(&i, ainfo)| {
let t = file.tracks.get(i)?;
let starts = audio_starts(&file, &bytes, i, ainfo.sample_rate);
let ts = t.timescale.max(1) as i64;
match &t.entries[0].codec {
CodecConfig::Opus(o) => {
let (offset, preroll) = match t.entries.first().map(|e| &e.codec) {
Some(CodecConfig::Opus(o)) => {
let off = if t.edit_offset != 0 { t.edit_offset } else { -(o.pre_skip as i64) * ts / 48_000 };
(off, (crate::audio::OPUS_PRE_ROLL as u64 * ts as u64).div_ceil(48_000) as i64)
// (Opus plays at 48 kHz: the pre-roll is in its sample frames)
(off, crate::audio::OPUS_PRE_ROLL as i64)
}
_ => (t.edit_offset, 0),
}
};
let state = Mutex::new(AudioState { decoder: None, packets: HashMap::new(), order: Vec::new(), last_decoded: None });
Some(Mp4Audio { track: i, state, starts, offset, preroll })
})
.unwrap_or((0, 0));
Ok(Self {
info,
bytes,
file,
vtrack,
atrack,
video: GopCache::new(explicit_color).with_rotation(rotation),
audio: Mutex::new(AudioState { decoder: None, packets: HashMap::new(), order: Vec::new(), last_decoded: None }),
audio_starts,
audio_offset,
audio_preroll,
})
.collect();
Ok(Self { info, bytes, file, vtrack, video: GopCache::new(explicit_color).with_rotation(rotation), audios })
}
fn read(&self, track: usize, i: usize) -> crate::Result<Vec<u8>> {
@@ -261,31 +286,36 @@ impl Mp4Source {
fn video_at(&self, t: Tick) -> crate::Result<Arc<VideoFrame>> {
let ti = self.vtrack.ok_or_else(|| CodecError::Unsupported("no video".into()))?;
let ts = self.file.tracks[ti].timescale as i64;
let late = filmcraft_media::cancel::catch_up().map(|m| (t - m).to_rational_floor(1, ts));
self.video.frame_late(&Mp4Video { src: self, track: ti }, t.to_rational_floor(1, ts), late)
// Nearest, not floor: with a coarse timescale (1000) sample times are rounded, and a
// floored lookup lands one unit short of every frame whose time was rounded up.
let late = filmcraft_media::cancel::catch_up().map(|m| (t - m).to_rational_round(1, ts));
self.video.frame_late(&Mp4Video { src: self, track: ti }, t.to_rational_round(1, ts), late)
}
fn audio_packet(&self, st: &mut AudioState, i: usize) -> crate::Result<Arc<Vec<Vec<f32>>>> {
/// Decoded packet `i` of audio stream `a` (`ainfo` is its `AudioStreamInfo`; `st` its decoder state).
fn audio_packet(&self, a: &Mp4Audio, ainfo: &AudioStreamInfo, st: &mut AudioState, i: usize) -> crate::Result<Arc<Vec<Vec<f32>>>> {
if let Some(p) = st.packets.get(&i) {
return Ok(p.clone());
}
let ti = self.atrack.ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
let track = &self.file.tracks[ti];
let ti = a.track;
let track = self.file.tracks.get(ti).ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
let entry = track.entries.first().ok_or_else(|| CodecError::Container("audio track without a sample description".into()))?;
let sample = track.samples.get(i).ok_or_else(|| CodecError::Container("audio packet out of range".into()))?;
let pk_start = a.starts.get(i).copied().unwrap_or(0);
let data = self.read(ti, i)?;
let decoded = match &entry.codec {
CodecConfig::Pcm(p) => decode_pcm(&data, p),
c => {
if st.decoder.is_none() {
st.decoder = Some(PacketDecoder::for_isobmff(c, self.info.audio.as_ref().map_or(48_000, |a| a.sample_rate))?);
st.decoder = Some(PacketDecoder::for_isobmff(c, ainfo.sample_rate)?);
}
// Non-sequential access: reset and prime with the preceding packets (codec pre-roll:
// one packet, or `OPUS_PRE_ROLL` worth for Opus).
if st.last_decoded.is_none_or(|l| l + 1 != i) {
let d = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?;
d.reset();
let from = if self.audio_preroll > 0 {
self.audio_starts.partition_point(|&x| x <= self.audio_starts[i] - self.audio_preroll).saturating_sub(1)
let from = if a.preroll > 0 {
a.starts.partition_point(|&x| x <= pk_start.saturating_sub(a.preroll)).saturating_sub(1)
} else {
i.saturating_sub(1)
};
@@ -295,11 +325,11 @@ impl Mp4Source {
}
}
}
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, self.audio_starts[i].max(0) as u64);
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, pk_start.max(0) as u64);
st.last_decoded = Some(i);
match r {
Ok(v) => v,
Err(_) => vec![vec![0.0; track.samples[i].duration as usize]; self.info.audio.as_ref().map_or(2, |a| a.channels as usize)],
Err(_) => vec![vec![0.0; sample.duration as usize]; ainfo.channels.max(1) as usize],
}
}
};
@@ -352,31 +382,37 @@ impl MediaSource for Mp4Source {
}
fn audio(&self, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
let ti = self.atrack.ok_or(MediaError::NoStream("audio"))?;
let track = &self.file.tracks[ti];
let ainfo = self.info.audio.as_ref().ok_or(MediaError::NoStream("audio"))?;
self.audio_stream(0, start, frames, sample_rate)
}
fn audio_stream(&self, stream: usize, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
let a = self.audios.get(stream).ok_or(MediaError::NoStream("audio"))?;
let ti = a.track;
let track = self.file.tracks.get(ti).ok_or(MediaError::NoStream("audio"))?;
let ainfo = self.info.audio_streams.get(stream).ok_or(MediaError::NoStream("audio"))?;
let src_rate = ainfo.sample_rate;
let ch = ainfo.channels.max(1) as usize;
// Map the requested window to source samples (edit list offset applied: presentation = pts + edit_offset).
let ratio = src_rate as f64 / sample_rate as f64;
let s0 = (start as f64 * ratio).floor() as i64 - self.audio_offset * src_rate as i64 / track.timescale.max(1) as i64;
let p0 = (start as f64 * ratio).floor() as i64;
// (in i128: a hostile edit list can make the offset large enough to overflow i64)
let offset = i128::from(a.offset) * i128::from(src_rate) / i128::from(track.timescale.max(1));
let s0 = i64::try_from(i128::from(p0) - offset).unwrap_or(if offset > 0 { i64::MIN } else { i64::MAX });
let need = (frames as f64 * ratio).ceil() as i64 + 2;
// Samples-per-unit: the track timescale is usually the sample rate for audio.
let unit = src_rate as f64 / track.timescale.max(1) as f64;
let mut src: Vec<Vec<f32>> = vec![vec![0.0; need.max(0) as usize]; ch];
let mut st = self.audio.lock().unwrap_or_else(|e| e.into_inner());
let first = self.audio_starts.partition_point(|&x| (x as f64 * unit) as i64 <= s0.max(0)).saturating_sub(1);
let mut st = a.state.lock().unwrap_or_else(|e| e.into_inner());
let first = a.starts.partition_point(|&x| x <= s0.max(0)).saturating_sub(1);
let mut i = first;
while i < track.samples.len() {
let pk_start = (self.audio_starts[i] as f64 * unit) as i64;
if pk_start >= s0 + need {
let Some(&pk_start) = a.starts.get(i) else { break };
if pk_start >= s0.saturating_add(need) {
break;
}
let pk = self.audio_packet(&mut st, i)?;
let pk = self.audio_packet(a, ainfo, &mut st, i)?;
for (c, dst) in src.iter_mut().enumerate() {
let Some(chan) = pk.get(c.min(pk.len().saturating_sub(1))) else { continue };
for (k, v) in chan.iter().enumerate() {
let pos = pk_start + k as i64 - s0;
let pos = pk_start.saturating_add(k as i64).saturating_sub(s0);
if pos >= 0 && (pos as usize) < dst.len() {
dst[pos as usize] = *v;
}
@@ -393,8 +429,9 @@ impl MediaSource for Mp4Source {
}
return Ok(out);
}
// resample the local window
let frac = s0 as f64 - start as f64 * ratio;
// Resample the local window. Its first sample is presentation sample `p0`; the edit list
// offset is already in `s0` and must not shift the read position a second time.
let frac = p0 as f64 - start as f64 * ratio;
let mut out = AudioBuffer::silence(sample_rate, ch, frames);
for k in 0..frames {
let pos = k as f64 * ratio - frac;
@@ -410,6 +447,37 @@ impl MediaSource for Mp4Source {
}
}
/// What the file says about one audio track (`None`: the track has no sample description).
fn audio_stream_info(t: &filmcraft_isobmff::Track, bytes: &crate::Src) -> Option<AudioStreamInfo> {
let entry = t.entries.first()?;
let ap = entry.audio.clone().unwrap_or_default();
let (rate, ch, bits) = match &entry.codec {
CodecConfig::Aac(a) => (
{
// HE-AAC plays at twice the core rate the AudioSpecificConfig starts with
// an AAC access unit is at most 6144 bits per channel: a hostile `stsz` can't make us read more
let units = t
.samples
.iter()
.take(8)
.filter_map(|x| filmcraft_media::reader::read_range(&*bytes.0, x.offset, (x.size as usize).min(MAX_AAC_PROBE_UNIT)).ok());
let units: Vec<Vec<u8>> = units.collect();
match crate::audio::aac_output_rate(&a.asc, units.iter().map(Vec::as_slice)) {
Some(r) => r,
None if a.sample_rate > 0 => a.sample_rate,
None => ap.sample_rate as u32,
}
},
if a.channel_config > 0 { a.channel_config as u32 } else { ap.channels },
None,
),
CodecConfig::Pcm(p) => (p.sample_rate as u32, p.channels, Some(p.bits as u32)),
CodecConfig::Opus(o) => (crate::audio::OPUS_RATE, (o.output_channels as u32).max(1), None),
_ => (if ap.sample_rate > 0.0 { ap.sample_rate as u32 } else { t.timescale }, ap.channels.max(1), None),
};
Some(AudioStreamInfo { sample_rate: rate.max(1), channels: ch.max(1), codec: codec_label(&entry.codec), bits_per_sample: bits })
}
fn codec_label(c: &CodecConfig) -> String {
match c {
CodecConfig::Avc(a) => format!(
@@ -427,6 +495,10 @@ fn codec_label(c: &CodecConfig) -> String {
CodecConfig::Hevc(_) => "HEVC".into(),
CodecConfig::Vp9(c) => format!("VP9 (Profile {})", c.profile),
CodecConfig::Av1(c) => format!("AV1 ({} Profile)", ["Main", "High", "Professional"].get(c.seq_profile as usize).unwrap_or(&"Main")),
CodecConfig::Apv(a) => match filmcraft_apv::Profile::from_idc(a.profile_idc) {
Some(p) => p.name().into(),
None => "APV".into(),
},
CodecConfig::ProRes { fourcc } => match &fourcc.0 {
b"apco" => "Apple ProRes 422 Proxy".into(),
b"apcs" => "Apple ProRes 422 LT".into(),
@@ -451,14 +523,37 @@ fn codec_label(c: &CodecConfig) -> String {
}
}
/// "YUV 4:2:2 10-bit" from an H.264 / HEVC `chroma_format_idc` and bit depth.
fn yuv_label(chroma_format_idc: u32, bits: u32) -> String {
let sub = match chroma_format_idc {
0 => "4:0:0",
1 => "4:2:0",
2 => "4:2:2",
_ => "4:4:4",
};
format!("YUV {sub} {bits}-bit")
}
/// `chroma_format_idc` and luma bit depth of an H.264 track: from the `avcC` High-profile extension
/// (ISO/IEC 14496-15 §5.3.3), else from its first SPS. The profile alone can't say (High 4:2:2 is
/// 8 or 10-bit, High 10 can be 8-bit).
fn avc_format(a: &filmcraft_isobmff::AvcConfig) -> Option<(u32, u32)> {
// reserved bits set: '111111' chroma_format, '11111' bit_depth_luma_minus8 (else not an extension)
if !matches!(a.profile, 66 | 77 | 88)
&& let &[chroma, depth, ..] = a.ext.as_slice()
&& chroma & 0xFC == 0xFC
&& depth & 0xF8 == 0xF8
{
return Some(((chroma & 3) as u32, (depth & 7) as u32 + 8));
}
let sps = filmcraft_h264::params::Sps::parse(&filmcraft_bitstream::unescape_rbsp(a.sps.first()?.get(1..)?)).ok()?;
Some((sps.chroma_format_idc, sps.bit_depth_luma))
}
fn pixfmt_label(c: &CodecConfig) -> String {
match c {
CodecConfig::Avc(a) => match a.profile {
110 => "YUV 4:2:0 10-bit".into(),
122 => "YUV 4:2:2".into(),
244 => "YUV 4:4:4".into(),
_ => "YUV 4:2:0 8-bit".into(),
},
CodecConfig::Avc(a) => avc_format(a).map(|(chroma, bits)| yuv_label(chroma, bits)).unwrap_or_default(),
CodecConfig::Hevc(h) => yuv_label(h.chroma_format_idc as u32, h.bit_depth_luma as u32),
CodecConfig::Vp9(c) => {
let sub = match c.chroma_subsampling {
2 => "4:2:2",
@@ -467,7 +562,12 @@ fn pixfmt_label(c: &CodecConfig) -> String {
};
format!("YUV {sub} {}-bit", c.bit_depth.max(8))
}
CodecConfig::ProRes { fourcc } if fourcc.0[2] == b'4' => "YUVA 4:4:4 12-bit".into(),
CodecConfig::Apv(a) => {
let chroma = filmcraft_apv::ChromaFormat::from_idc(a.chroma_format_idc).unwrap_or(filmcraft_apv::ChromaFormat::Yuv422);
crate::apv::apv_pixfmt_label(chroma, a.bit_depth_minus8.saturating_add(8))
}
// 4444 until the first frame header says whether it codes alpha (`open_reader`)
CodecConfig::ProRes { fourcc } if fourcc.0[2] == b'4' => "YUV 4:4:4 12-bit".into(),
CodecConfig::ProRes { .. } => "YUV 4:2:2 10-bit".into(),
CodecConfig::Jpeg { .. } => "YUV 4:2:x 8-bit".into(),
_ => String::new(),
@@ -481,6 +581,25 @@ pub fn opener(name: &str, bytes: Arc<[u8]>) -> Option<Result<SharedSource, Media
Some(Mp4Source::open(name, bytes).map(|s| Arc::new(s) as SharedSource).map_err(Into::into))
}
/// How long a track plays. With an edit list that is the sum of its edits (ISO/IEC 14496-12
/// §8.6.6, movie timescale), as `tkhd` records it. The media duration (`mdhd`) runs on the decode
/// timeline: it is longer than the presentation by the B-frame delay an edit skips (`media_time`, one
/// extra frame on every ffmpeg / x264 MP4 with B-frames) and shorter by a leading empty edit (the last
/// frame of a delayed video track was never shown). Fragmented files, and edit lists that add up to
/// nothing or to more than a [`Tick`] holds, keep the media duration.
fn presentation_duration(file: &Mp4File, t: &filmcraft_isobmff::Track) -> Tick {
let media = Tick::from_rational(t.duration as i64, 1, t.timescale as i64);
if file.fragmented || t.edits.is_empty() || file.timescale == 0 {
return media;
}
let edits = t.edits.iter().fold(0u128, |a, e| a.saturating_add(e.segment_duration as u128));
let ticks = edits.saturating_mul(filmcraft_time::TICKS_PER_SECOND as u128) / file.timescale as u128;
match i64::try_from(ticks) {
Ok(t) if t > 0 => Tick(t),
_ => media,
}
}
/// [`filmcraft_media::ReaderOpener`] for MP4/MOV.
pub fn reader_opener(name: &str, head: &[u8], reader: &filmcraft_media::SharedReader) -> Option<Result<SharedSource, MediaError>> {
if !sniff(head) {
@@ -518,6 +637,45 @@ mod tests {
b.into()
}
/// A four-frame 25 fps ProRes MOV (64×32) whose video track carries `edits`.
fn mov_with_edits(edits: Vec<filmcraft_isobmff::Edit>) -> Arc<[u8]> {
let fr = filmcraft_prores::Frame::new(64, 32, filmcraft_prores::ChromaFormat::Yuv422, 10, false);
let data = filmcraft_prores::Encoder::new(filmcraft_prores::Profile::Hq, 64, 32).encode(&fr).expect("encode");
let mut mux = Mp4Writer::new(std::io::Cursor::new(Vec::new()), WriterOptions::new(Brand::Mov)).expect("writer");
let mut cfg = TrackConfig::new(SampleEntry::prores(FourCc(*b"apch"), 64, 32), 25);
cfg.edits = edits;
let t = mux.add_track(cfg).expect("track");
for _ in 0..4 {
mux.write_sample(t, WriteSample { data: &data, duration: 1, composition_offset: 0, is_sync: true }).expect("sample");
}
mux.finish().expect("finish").into_inner().into()
}
/// A clip lasts as long as its edit list presents it, not as long as its media (`mdhd`) runs.
/// Every ffmpeg / x264 MP4 with B-frames skips its reorder delay with an edit and imported one
/// frame too long; a video track delayed by an empty edit lost its last frame.
#[test]
fn duration_follows_the_edit_list() {
use filmcraft_isobmff::Edit;
let frames = |s: &Mp4Source| s.info().duration.to_rational_floor(1, 25);
// no edit list: the media duration
assert_eq!(frames(&Mp4Source::open("plain.mov", mov_with_edits(Vec::new())).expect("open")), 4);
// skip the first frame (B-frame delay style): three frames play (movie timescale 1000 ms)
let skip = vec![Edit { segment_duration: 120, media_time: 1, media_rate: 0x10000 }];
assert_eq!(frames(&Mp4Source::open("skip.mov", mov_with_edits(skip)).expect("open")), 3);
// a one-frame empty edit, then all four frames: five frames, and the last one is there
let delayed =
vec![Edit { segment_duration: 40, media_time: -1, media_rate: 0x10000 }, Edit { segment_duration: 160, media_time: 0, media_rate: 0x10000 }];
let s = Mp4Source::open("delayed.mov", mov_with_edits(delayed)).expect("open");
assert_eq!(frames(&s), 5);
assert!(s.video_frame(FrameRequest::full(Tick::from_rational(4, 1, 25))).is_ok());
// hostile segment durations fall back to the media duration instead of wrapping
let huge = vec![Edit { segment_duration: u64::MAX, media_time: 0, media_rate: 0x10000 }];
assert_eq!(frames(&Mp4Source::open("huge.mov", mov_with_edits(huge)).expect("open")), 4);
let zero = vec![Edit { segment_duration: 0, media_time: 0, media_rate: 0x10000 }];
assert_eq!(frames(&Mp4Source::open("zero.mov", mov_with_edits(zero)).expect("open")), 4);
}
/// Overwrite the big-endian u32 `skip` bytes after the first `fourcc` box type.
fn patch_u32(b: &mut [u8], fourcc: &[u8; 4], skip: usize, v: u32) {
let k = b.windows(4).position(|x| x == fourcc).expect("box") + 4 + skip;
@@ -601,4 +759,171 @@ mod tests {
let s = Mp4Source::open("cw2.mov", rotated_mov([0, 2 * ONE, 0, -2 * ONE, 0, 0, 64 * ONE, 0, W])).expect("open");
assert_eq!(s.info().video.as_ref().map(|v| (v.width, v.height)), Some((32, 64)));
}
#[test]
fn hevc_pixel_format_comes_from_hvcc() {
for (chroma, bits, want) in [(1, 8, "YUV 4:2:0 8-bit"), (1, 10, "YUV 4:2:0 10-bit"), (2, 8, "YUV 4:2:2 8-bit"), (2, 10, "YUV 4:2:2 10-bit")] {
let c = filmcraft_isobmff::HevcConfig { general_profile_idc: 1, chroma_format_idc: chroma, bit_depth_luma: bits, ..Default::default() };
assert_eq!(pixfmt_label(&CodecConfig::Hevc(c)), want);
}
}
/// A High-profile SPS NAL unit (ITU-T H.264 §7.3.2.1.1) with the given chroma format and bit depth, 16×16.
fn high_sps(profile: u8, chroma: u32, bits: u32) -> Vec<u8> {
let mut w = filmcraft_bitstream::BitWriter::new();
w.write_bits(0x67, 8);
w.write_bits(profile as u32, 8);
w.write_bits(0, 8); // constraint flags
w.write_bits(30, 8); // level
w.write_ue(0); // seq_parameter_set_id
w.write_ue(chroma);
if chroma == 3 {
w.write_bit(false); // separate_colour_plane_flag
}
w.write_ue(bits - 8); // luma
w.write_ue(bits - 8); // chroma
w.write_bits(0, 2); // no transform bypass, no scaling matrices
w.write_ue(0); // log2_max_frame_num_minus4
w.write_ue(2); // pic_order_cnt_type
w.write_ue(1); // max_num_ref_frames
w.write_bit(false); // gaps_in_frame_num_allowed
w.write_ue(0); // pic_width_in_mbs_minus1
w.write_ue(0); // pic_height_in_map_units_minus1
w.write_bits(0b110, 3); // frame_mbs_only, direct_8x8_inference, no cropping
w.write_bit(false); // no VUI
w.rbsp_trailing();
w.finish()
}
#[test]
fn avc_pixel_format_comes_from_avcc_not_the_profile() {
let avc =
|profile: u8, sps: Vec<u8>, ext: Vec<u8>| CodecConfig::Avc(filmcraft_isobmff::AvcConfig { profile, sps: vec![sps], ext, ..Default::default() });
// High 4:2:2 at 10 and 8 bits, from the avcC extension (chroma_format 2, bit_depth_luma_minus8)
assert_eq!(pixfmt_label(&avc(122, high_sps(122, 2, 10), vec![0xFE, 0xFA, 0xFA, 0])), "YUV 4:2:2 10-bit");
assert_eq!(pixfmt_label(&avc(122, high_sps(122, 2, 8), vec![0xFE, 0xF8, 0xF8, 0])), "YUV 4:2:2 8-bit");
// no extension (or not a valid one): the SPS says
assert_eq!(pixfmt_label(&avc(122, high_sps(122, 2, 10), Vec::new())), "YUV 4:2:2 10-bit");
assert_eq!(pixfmt_label(&avc(110, high_sps(110, 1, 10), vec![0, 0, 0, 0])), "YUV 4:2:0 10-bit");
assert_eq!(pixfmt_label(&avc(244, high_sps(244, 3, 10), Vec::new())), "YUV 4:4:4 10-bit");
// an unreadable SPS and no extension: unknown, not a guess
assert_eq!(pixfmt_label(&avc(122, vec![0x67, 122], Vec::new())), "");
}
/// A one-frame 32×16 ProRes MOV with the given profile, chroma format and fourcc.
fn prores_mov(profile: filmcraft_prores::Profile, fourcc: &[u8; 4], chroma: filmcraft_prores::ChromaFormat, alpha: bool) -> Arc<[u8]> {
let (w, h) = (32u32, 16u32);
let fr = filmcraft_prores::Frame::new(w, h, chroma, if chroma == filmcraft_prores::ChromaFormat::Yuv444 { 12 } else { 10 }, alpha);
let data = filmcraft_prores::Encoder::new(profile, w, h).encode(&fr).expect("encode");
let mut mux = Mp4Writer::new(std::io::Cursor::new(Vec::new()), WriterOptions::new(Brand::Mov)).expect("writer");
let t = mux.add_track(TrackConfig::new(SampleEntry::prores(FourCc(*fourcc), w as u16, h as u16), 25)).expect("track");
mux.write_sample(t, WriteSample { data: &data, duration: 1, composition_offset: 0, is_sync: true }).expect("sample");
mux.finish().expect("finish").into_inner().into()
}
/// `ap4h` / `ap4x` don't imply alpha: the frame header's alpha_channel_type does.
#[test]
fn prores_alpha_comes_from_the_frame_header() {
use filmcraft_prores::{ChromaFormat, Profile};
for (profile, fourcc, chroma, alpha, want) in [
(Profile::P4444, b"ap4h", ChromaFormat::Yuv444, false, "YUV 4:4:4 12-bit"),
(Profile::P4444, b"ap4h", ChromaFormat::Yuv444, true, "YUVA 4:4:4 12-bit"),
(Profile::P4444Xq, b"ap4x", ChromaFormat::Yuv444, false, "YUV 4:4:4 12-bit"),
(Profile::Hq, b"apch", ChromaFormat::Yuv422, false, "YUV 4:2:2 10-bit"),
] {
let s = Mp4Source::open("p.mov", prores_mov(profile, fourcc, chroma, alpha)).expect("open");
let v = s.info().video.as_ref().expect("video");
assert_eq!((v.pixel_format.as_str(), v.has_alpha), (want, alpha), "{}", String::from_utf8_lossy(fourcc));
}
}
}
#[cfg(test)]
mod multi_audio_tests {
use filmcraft_isobmff::{Brand, Mp4Writer, PcmConfig, SampleEntry, TrackConfig, WriteSample, WriterOptions};
use filmcraft_media::MediaSource;
/// Two mono PCM tracks with different rates and samples. Stream indices follow file order;
/// decoder caches, sample timing and resampling must not cross streams.
#[test]
fn every_audio_track_is_inspected_and_decoded_independently() {
let mut mux = Mp4Writer::new(std::io::Cursor::new(Vec::new()), WriterOptions::new(Brand::Mov)).unwrap();
for (rate, value) in [(48_000u32, 8192i16), (24_000, -16384)] {
let entry = SampleEntry::pcm(PcmConfig { bits: 16, float: false, big_endian: false, signed: true, channels: 1, sample_rate: rate as f64 });
let track = mux.add_track(TrackConfig::new(entry, rate)).unwrap();
let data: Vec<u8> = (0..rate / 10).flat_map(|_| value.to_le_bytes()).collect();
mux.write_sample(track, WriteSample { data: &data, duration: rate / 10, composition_offset: 0, is_sync: true }).unwrap();
}
let bytes = mux.finish().unwrap().into_inner();
let source = super::Mp4Source::open("two.mov", bytes.into()).unwrap();
assert_eq!(source.info().audio_streams.iter().map(|a| a.sample_rate).collect::<Vec<_>>(), vec![48_000, 24_000]);
let primary = source.audio(0, 100, 48_000).unwrap();
let secondary = source.audio_stream(1, 0, 100, 48_000).unwrap();
assert!((primary.channels[0][50] - 0.25).abs() < 0.001);
assert!((secondary.channels[0][50] + 0.5).abs() < 0.001);
assert!(source.audio_stream(2, 0, 100, 48_000).is_err());
}
}
#[cfg(test)]
mod he_aac_tests {
use filmcraft_isobmff::{Brand, Mp4Writer, SampleEntry, TrackConfig, WriteSample, WriterOptions};
use filmcraft_media::MediaSource;
/// #108: an HE-AAC MP4 (explicit SBR signalling, core 22.05 kHz, output 44.1 kHz, timescale
/// 44.1 kHz) built here from our AAC-LC encoder: the stream has no SBR data, which only the
/// high band would need. It was reported at the 22.05 kHz core rate, so sequences made from
/// such a clip were created at 22.05 kHz.
#[test]
fn he_aac_plays_at_the_output_rate() {
let core = 22_050u32;
let n = core as usize * 2;
let tone: Vec<f32> = (0..n).map(|i| 0.25 * (2.0 * std::f32::consts::PI * 1000.0 * i as f32 / core as f32).sin()).collect();
let mut enc = filmcraft_aac::Encoder::new(filmcraft_aac::EncoderConfig::cbr(core, 2, 64_000)).unwrap();
let mut aus = enc.encode(&[&tone, &tone]);
aus.extend(enc.flush());
// AOT 5, core 22.05 kHz, stereo, extension 44.1 kHz, core AOT 2
let asc = vec![0x2B, 0x92, 0x08, 0x00];
let mut mux = Mp4Writer::new(std::io::Cursor::new(Vec::new()), WriterOptions::new(Brand::Mp4)).unwrap();
let t = mux.add_track(TrackConfig::new(SampleEntry::aac(asc, 2, 44_100), 44_100)).unwrap();
for au in &aus {
mux.write_sample(t, WriteSample { data: au, duration: 2048, composition_offset: 0, is_sync: true }).unwrap();
}
let b = mux.finish().unwrap().into_inner();
let src = super::Mp4Source::open("he.mp4", b.into()).unwrap();
assert_eq!(src.info().audio().unwrap().sample_rate, 44_100);
// one second from 0.5 s at 44.1 kHz: the 1 kHz tone at its level (not silence, not shifted)
let buf = src.audio(22_050, 44_100, 44_100).unwrap();
let x = &buf.channels[0];
let peak = x.iter().fold(0f32, |m, v| m.max(v.abs()));
assert!((0.2..0.3).contains(&peak), "peak {peak}");
let crossings = x.windows(2).filter(|w| w[0] < 0.0 && w[1] >= 0.0).count();
assert!((995..=1005).contains(&crossings), "{crossings} Hz");
}
/// A hostile edit list (#66): an AAC track whose edit puts it absurdly far off, read at another
/// rate, scaled the offset to the source rate in i64 and overflowed. It reads as silence now.
#[test]
fn hostile_audio_edit_offset_never_overflows() {
use filmcraft_isobmff::Edit;
let rate = 32_000u32;
let tone: Vec<f32> = (0..rate as usize / 2).map(|i| 0.25 * (i as f32 * 0.05).sin()).collect();
let mut enc = filmcraft_aac::Encoder::new(filmcraft_aac::EncoderConfig::cbr(rate, 1, 64_000)).unwrap();
let mut aus = enc.encode(&[&tone]);
aus.extend(enc.flush());
// AOT 2 (AAC-LC), 32 kHz (index 5), mono
let asc = vec![0x12, 0x88];
for media_time in [i64::MAX, i64::MAX / 3, i64::MIN / 3] {
let mut mux = Mp4Writer::new(std::io::Cursor::new(Vec::new()), WriterOptions::new(Brand::Mp4)).unwrap();
let mut cfg = TrackConfig::new(SampleEntry::aac(asc.clone(), 1, rate), rate);
cfg.edits = vec![Edit { segment_duration: 500, media_time, media_rate: 0x10000 }];
let t = mux.add_track(cfg).unwrap();
for au in &aus {
mux.write_sample(t, WriteSample { data: au, duration: 1024, composition_offset: 0, is_sync: true }).unwrap();
}
let src = super::Mp4Source::open("hostile.mp4", mux.finish().unwrap().into_inner().into()).unwrap();
let buf = src.audio(0, 4800, 48_000).unwrap();
assert!(buf.channels.iter().flatten().all(|v| v.is_finite() && v.abs() <= 1.0), "media_time {media_time}");
}
}
}
+6 -4
View File
@@ -211,7 +211,7 @@ impl MpegSource {
kind: MediaKind::Movie,
duration: Tick::ZERO,
video: None,
audio: None,
audio_streams: Vec::new(),
container: file.format.name().to_string(),
start_timecode: None,
file_size: Some(bytes.0.len()),
@@ -316,7 +316,7 @@ impl MpegSource {
kind: MediaKind::Movie,
duration: Tick::ZERO,
video: None,
audio: None,
audio_streams: Vec::new(),
container: "MPEG video elementary stream".into(),
start_timecode: None,
file_size: Some(bytes.0.len()),
@@ -485,7 +485,8 @@ impl MpegSource {
/// Mark the audio stream as not decodable (it still shows in the media info).
fn audio_unsupported(&mut self, codec_name: &str, why: String) -> Option<AudioTrack> {
self.unsupported_audio = Some(why);
self.info.audio = Some(AudioStreamInfo { sample_rate: 48_000, channels: 2, codec: format!("{codec_name} (unsupported)"), bits_per_sample: None });
self.info.audio_streams =
vec![AudioStreamInfo { sample_rate: 48_000, channels: 2, codec: format!("{codec_name} (unsupported)"), bits_per_sample: None }];
None
}
@@ -599,7 +600,8 @@ impl MpegSource {
self.info.kind = MediaKind::AudioOnly;
}
}
self.info.audio = Some(AudioStreamInfo { sample_rate: rate.max(1), channels: channels.max(1) as u32, codec: codec_name, bits_per_sample: bits });
self.info.audio_streams =
vec![AudioStreamInfo { sample_rate: rate.max(1), channels: channels.max(1) as u32, codec: codec_name, bits_per_sample: bits }];
Some(AudioTrack {
stream: ai,
codec,
+2 -2
View File
@@ -361,7 +361,7 @@ impl MxfSource {
kind: if video.is_some() { MediaKind::Movie } else { MediaKind::AudioOnly },
duration,
video,
audio,
audio_streams: audio.into_iter().collect(),
container: format!("MXF {}", file.operational_pattern.name()),
start_timecode: None,
file_size: Some(bytes.0.len()),
@@ -457,7 +457,7 @@ impl MediaSource for MxfSource {
}
fn audio(&self, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
let ainfo = self.info.audio.as_ref().ok_or(MediaError::NoStream("audio"))?;
let ainfo = self.info.audio().ok_or(MediaError::NoStream("audio"))?;
let src_rate = ainfo.sample_rate;
let ch = ainfo.channels.max(1) as usize;
let mut out = AudioBuffer::silence(sample_rate, ch, frames);
+1 -1
View File
@@ -110,7 +110,7 @@ impl OggSource {
kind: MediaKind::AudioOnly,
duration: Tick::from_units(frames, rate as i64),
video: None,
audio: Some(AudioStreamInfo { sample_rate: rate, channels: channels.max(1), codec: codec.into(), bits_per_sample: None }),
audio_streams: vec![AudioStreamInfo { sample_rate: rate, channels: channels.max(1), codec: codec.into(), bits_per_sample: None }],
container: "Ogg".into(),
start_timecode: None,
file_size: Some(bytes.0.len()),
+136
View File
@@ -0,0 +1,136 @@
//! Frame lookup against timestamps rounded to the container's timebase (issue #73).
//!
//! WebM stores timestamps in 1 ms units, so at 30 fps frame 2 (66.67 ms) is stored as 67. Flooring
//! the requested time to 66 picked frame 1 instead: every third frame was skipped and the one
//! before it shown twice. MP4 has the same problem when a muxer picks a 1 ms track timescale.
//!
//! The streams are synthetic Motion-JPEG: each picture shows its index as a row of black and
//! white blocks, so a decoded frame says which sample it came from.
use std::io::Cursor;
use std::sync::Arc;
use filmcraft_frame::{PixelData, VideoFrame};
use filmcraft_isobmff::{Brand, Mp4Writer, SampleEntry, TrackConfig, WriteSample, WriterOptions};
use filmcraft_matroska::{MkvWriter, MuxOptions, TrackKind, TrackSpec};
use filmcraft_media::{FrameRequest, MediaSource};
use filmcraft_time::{FrameRate, TICKS_PER_SECOND, Tick};
/// Bits of the frame index drawn per picture, one `BLOCK`-pixel square each.
const BITS: u32 = 8;
const BLOCK: u32 = 16;
const SECONDS: i64 = 3;
/// The rates that don't divide 1 ms evenly (affected), and 25 fps (never was).
const RATES: [FrameRate; 7] =
[FrameRate::FPS_30, FrameRate::FPS_29_97, FrameRate::FPS_60, FrameRate::FPS_59_94, FrameRate::FPS_24, FrameRate::FPS_23_976, FrameRate::FPS_25];
/// How the muxer quantised the exact frame times to milliseconds.
#[derive(Clone, Copy, Debug)]
enum Stamp {
/// To the nearest millisecond, halves up (mkvmerge, ffmpeg).
Nearest,
/// Truncated.
Floor,
}
fn picture(index: usize) -> Vec<u8> {
let img = image::GrayImage::from_fn(BITS * BLOCK, BLOCK, |x, _| image::Luma([if (index >> (x / BLOCK)) & 1 == 1 { 255 } else { 0 }]));
let mut out = Vec::new();
image::codecs::jpeg::JpegEncoder::new_with_quality(&mut out, 90).encode_image(&img).unwrap();
out
}
fn index_of(f: &VideoFrame) -> usize {
let PixelData::Rgba8(px) = &f.data else { panic!("MJPEG decodes to RGBA8") };
(0..BITS).filter(|b| px[((BLOCK / 2 * f.width + b * BLOCK + BLOCK / 2) * 4) as usize] > 127).map(|b| 1 << b).sum()
}
fn frame_count(rate: FrameRate) -> usize {
rate.frame_at(Tick(SECONDS * TICKS_PER_SECOND)) as usize
}
/// Frame `k`'s start in milliseconds, as the muxer stores it.
fn stamp_ms(rate: FrameRate, k: usize, stamp: Stamp) -> i64 {
let (num, den) = (rate.num as i128, rate.den as i128);
let exact2 = 2 * k as i128 * 1000 * den; // twice the time in ms, times num
(match stamp {
Stamp::Nearest => (exact2 + num) / (2 * num),
Stamp::Floor => exact2 / (2 * num),
}) as i64
}
fn webm(rate: FrameRate, stamp: Stamp) -> Arc<[u8]> {
let mut spec = TrackSpec::new(TrackKind::Video, "V_MJPEG");
spec.video_size = Some((BITS * BLOCK, BLOCK));
spec.default_duration_ns = Some((1_000_000_000 * rate.den / rate.num) as u64);
let opts = MuxOptions { doc_type: "webm".into(), ..MuxOptions::default() };
assert_eq!(opts.timestamp_scale, 1_000_000, "1 ms timestamps");
let mut w = MkvWriter::new(Cursor::new(Vec::new()), vec![spec], opts).unwrap();
for k in 0..frame_count(rate) {
w.write_frame(0, stamp_ms(rate, k, stamp) * 1_000_000, true, &picture(k), None).unwrap();
}
w.finish().unwrap().into_inner().into()
}
fn mp4_1ms(rate: FrameRate, stamp: Stamp) -> Arc<[u8]> {
let mut w = Mp4Writer::new(Cursor::new(Vec::new()), WriterOptions::new(Brand::Mov)).unwrap();
let t = w.add_track(TrackConfig::new(SampleEntry::jpeg((BITS * BLOCK) as u16, BLOCK as u16), 1000)).unwrap();
let n = frame_count(rate);
for k in 0..n {
let duration = (stamp_ms(rate, k + 1, stamp) - stamp_ms(rate, k, stamp)) as u32;
w.write_sample(t, WriteSample { data: &picture(k), duration, composition_offset: 0, is_sync: true }).unwrap();
}
w.finish().unwrap().into_inner().into()
}
/// Plays the stream back at its own rate (every frame's start), and asks for every frame's
/// midpoint too. Returns what went wrong: frames never shown, frames shown twice, wrong midpoints.
fn problems(src: &dyn MediaSource, rate: FrameRate) -> Vec<String> {
let n = frame_count(rate);
let at = |t: Tick| index_of(&src.video_frame(FrameRequest::full(t)).unwrap());
let mut shown = vec![0usize; n];
for k in 0..n {
shown[at(rate.tick_of(k as i64)).min(n - 1)] += 1;
}
let mut out = Vec::new();
let dropped: Vec<usize> = (0..n).filter(|&k| shown[k] == 0).collect();
let repeated: Vec<usize> = (0..n).filter(|&k| shown[k] > 1).collect();
if !dropped.is_empty() || !repeated.is_empty() {
out.push(format!(
"{} of {n} frames never shown {:?}…, {} shown twice {:?}…",
dropped.len(),
&dropped[..dropped.len().min(4)],
repeated.len(),
&repeated[..repeated.len().min(4)]
));
}
let mid: Vec<(usize, usize)> = (0..n).map(|k| (k, at(rate.tick_of(k as i64) + Tick(rate.frame_duration().0 / 2)))).filter(|(k, got)| k != got).collect();
if !mid.is_empty() {
out.push(format!("{} mid-frame requests wrong, e.g. (wanted, got) {:?}", mid.len(), &mid[..mid.len().min(4)]));
}
out
}
fn check(container: &str, open: fn(FrameRate, Stamp) -> Arc<[u8]>) {
let mut failures = Vec::new();
for rate in RATES {
for stamp in [Stamp::Nearest, Stamp::Floor] {
let src = crate::open_bytes(&format!("synthetic.{container}"), open(rate, stamp)).unwrap();
for p in problems(src.as_ref(), rate) {
failures.push(format!("{rate}, {stamp:?} ms stamps: {p}"));
}
}
}
assert!(failures.is_empty(), "{container}:\n{}", failures.join("\n"));
}
#[test]
fn webm_with_1ms_timestamps_shows_every_frame_once() {
check("webm", webm);
}
#[test]
fn mp4_with_a_1ms_timescale_shows_every_frame_once() {
check("mov", mp4_1ms);
}
+39 -2
View File
@@ -70,7 +70,7 @@ fn mjpeg_mov_with_pcm() {
fn aac_mp4_decodes() {
let Some(b) = fixture("tone_aac.mp4", &["-f", "lavfi", "-i", "sine=frequency=440:sample_rate=48000:d=3", "-c:a", "aac", "-b:a", "128k"]) else { return };
let src = crate::open_bytes("tone_aac.mp4", b).unwrap();
assert!(src.info().audio.as_ref().unwrap().codec.contains("AAC"));
assert!(src.info().audio().unwrap().codec.contains("AAC"));
// read in the middle (random access) and sequentially
let a = src.audio(48_000, 4800, 48_000).unwrap();
let p = a.peaks()[0];
@@ -510,7 +510,7 @@ fn check_opus(name: &str, channels: usize, expr: &str, extra: &[&str], container
let src = crate::open_bytes(name, b).expect("open");
let info = src.info().clone();
assert_eq!(info.container, container);
let a = info.audio.as_ref().expect("audio");
let a = info.audio().expect("audio");
assert_eq!((a.codec.as_str(), a.sample_rate, a.channels as usize), ("Opus", 48_000, channels));
let d = info.duration.seconds();
assert!((d - 3.0).abs() < 0.03, "{name}: duration {d}");
@@ -578,3 +578,40 @@ fn opus_mp4_stereo() {
"MPEG-4",
);
}
#[test]
fn apv_mp4_and_raw_bitstream_decode() {
use filmcraft_isobmff::{ApvConfig, Brand, Mp4Writer, SampleEntry, TrackConfig, WriteSample, WriterOptions};
let (w, h) = (64u32, 32u32);
let mut fr = filmcraft_apv::Frame::new(w, h, filmcraft_apv::ChromaFormat::Yuv422, 10, false);
for (i, y) in fr.y.iter_mut().enumerate() {
*y = if (i as u32 % w) < w / 2 { 850 } else { 150 };
}
let mut enc = filmcraft_apv::Encoder::new(filmcraft_apv::Profile::P422_10, w, h).expect("encoder");
let au = enc.encode_raw_au(&fr).expect("encode_raw_au");
let apvc = ApvConfig::parse(&enc.decoder_config_record()).expect("parse apvC");
// 1. Raw .apv elementary stream
let raw_src = crate::open_bytes("clip.apv", au.clone().into()).expect("open raw .apv");
let raw_v = raw_src.info().video.as_ref().expect("video info");
assert_eq!((raw_v.width, raw_v.height), (w, h));
assert_eq!(raw_v.codec, "APV 422-10");
assert_eq!(raw_v.pixel_format, "YUV 4:2:2 10-bit");
let decoded_raw = raw_src.video_frame(FrameRequest::full(Tick::ZERO)).expect("decode raw frame");
assert_eq!((decoded_raw.width, decoded_raw.height), (w, h));
// 2. MP4 with apv1 + apvC
let mut mux = Mp4Writer::new(std::io::Cursor::new(Vec::new()), WriterOptions::new(Brand::Mp4)).expect("writer");
let vt = mux.add_track(TrackConfig::new(SampleEntry::apv(apvc, w as u16, h as u16), 24)).expect("track");
mux.write_sample(vt, WriteSample { data: &au, duration: 1, composition_offset: 0, is_sync: true }).expect("sample");
let mp4_bytes: Arc<[u8]> = mux.finish().expect("finish").into_inner().into();
let mp4_src = crate::open_bytes("clip.mp4", mp4_bytes).expect("open mp4");
let mp4_v = mp4_src.info().video.as_ref().expect("mp4 video info");
assert_eq!((mp4_v.width, mp4_v.height), (w, h));
assert_eq!(mp4_v.codec, "APV 422-10");
assert_eq!(mp4_v.pixel_format, "YUV 4:2:2 10-bit");
let decoded_mp4 = mp4_src.video_frame(FrameRequest::full(Tick::ZERO)).expect("decode mp4 frame");
assert_eq!(decoded_mp4.to_rgba8(), decoded_raw.to_rgba8());
}
+114 -46
View File
@@ -45,8 +45,19 @@ pub trait VideoDecoder: Send {
/// shortcuts that only change pictures nothing references, which come out flagged
/// [`DecodedFrame::draft`]. Decoders without such a mode ignore it.
fn set_draft(&mut self, _on: bool) {}
/// Made with fewer worker threads than a decoder for playback gets, because it was made for
/// background work ([`filmcraft_media::cancel::with_background`]). A request for a monitor
/// replaces such a decoder.
fn thread_limited(&self) -> bool {
false
}
}
/// Worker threads of the H.264 decoders made for background work (thumbnails): a frame-threaded
/// 4K decoder holds `threads + 2` pictures in flight (~0.5 GB with 16 threads), and the frame
/// workers already decode different files in parallel.
const BACKGROUND_H264_THREADS: usize = 2;
/// NAL unit headers of a length-prefixed (avcC / hvcC) sample: the first two bytes of each unit.
fn nal_headers(sample: &[u8], length_size: usize) -> impl Iterator<Item = (u8, u8)> + '_ {
let mut pos = 0usize;
@@ -200,17 +211,36 @@ pub struct H264Decoder {
dec: filmcraft_h264::Decoder,
length_size: usize,
draft: bool,
/// Worker threads (a background decoder gets fewer).
threads: usize,
}
impl H264Decoder {
/// The worker threads for a decoder made now.
fn threads_now() -> usize {
if filmcraft_media::cancel::background() { BACKGROUND_H264_THREADS.min(h264_threads()) } else { h264_threads() }
}
fn make_inner(avcc: &[u8], threads: usize) -> Result<filmcraft_h264::Decoder> {
let mut dec = filmcraft_h264::Decoder::with_threads(threads);
if !avcc.is_empty() {
dec.configure_avcc(avcc).map_err(|e| CodecError::Decode(e.to_string()))?;
}
Ok(dec)
}
pub fn new(avcc: Vec<u8>) -> Result<Self> {
let dec = filmcraft_h264::Decoder::from_avcc(&avcc).map_err(|e| CodecError::Decode(e.to_string()))?;
recycle_h264_planes();
let threads = Self::threads_now();
let dec = Self::make_inner(&avcc, threads)?;
let length_size = avcc_length_size(&avcc);
Ok(Self { avcc, dec, length_size, draft: false })
Ok(Self { avcc, dec, length_size, draft: false, threads })
}
/// A decoder for Annex B byte-stream samples (start codes, in-band parameter sets: MXF, TS).
pub fn annexb() -> Self {
Self { avcc: Vec::new(), dec: filmcraft_h264::Decoder::new(), length_size: 0, draft: false }
recycle_h264_planes();
let threads = Self::threads_now();
Self { avcc: Vec::new(), dec: filmcraft_h264::Decoder::with_threads(threads), length_size: 0, draft: false, threads }
}
fn convert(p: filmcraft_h264::Picture) -> DecodedFrame {
use std::sync::Arc;
@@ -243,9 +273,7 @@ impl VideoDecoder for H264Decoder {
self.dec.flush().into_iter().map(Self::convert).collect()
}
fn reset(&mut self) {
if self.avcc.is_empty() {
self.dec = filmcraft_h264::Decoder::new();
} else if let Ok(d) = filmcraft_h264::Decoder::from_avcc(&self.avcc) {
if let Ok(d) = Self::make_inner(&self.avcc, self.threads) {
self.dec = d;
}
self.dec.set_draft(self.draft);
@@ -253,6 +281,9 @@ impl VideoDecoder for H264Decoder {
fn name(&self) -> &str {
"FilmCraft H.264"
}
fn thread_limited(&self) -> bool {
self.threads < h264_threads()
}
fn set_draft(&mut self, on: bool) {
self.draft = on;
self.dec.set_draft(on);
@@ -269,6 +300,12 @@ impl VideoDecoder for H264Decoder {
}
}
/// The H.264 decoder's output planes come from the frames the caches evicted
/// (`filmcraft_frame::pool`) instead of the allocator.
fn recycle_h264_planes() {
filmcraft_h264::set_plane_allocator(filmcraft_frame::pool::take_u8);
}
/// Worker threads each H.264 decoder uses (frame threading; 1 on wasm).
pub fn h264_threads() -> usize {
filmcraft_h264::default_threads()
@@ -404,32 +441,14 @@ impl Vp9Decoder {
/// `None` for a picture whose planes mix sample types (never produced by the decoder).
fn convert(&self, p: filmcraft_vp9::Picture) -> Option<DecodedFrame> {
use filmcraft_color::{Matrix, Primaries, Range};
use filmcraft_color::Range;
use filmcraft_frame::{Chroma, PixelData};
use filmcraft_vp9::Plane;
use std::sync::Arc;
let (w, h) = (p.width as usize, p.height as usize);
let (cw, ch) = (p.chroma_width as usize, p.chroma_height as usize);
let mut color = filmcraft_color::ColorInfo { matrix: filmcraft_frame::default_matrix(p.width, p.height), ..filmcraft_color::ColorInfo::REC709 };
// color_space (7.2.2): 1 BT.601, 2 BT.709, 3 SMPTE-170, 4 SMPTE-240, 5 BT.2020, 7 sRGB.
match p.color.color_space {
1 | 3 => color.matrix = Matrix::Bt601,
2 | 4 => color.matrix = Matrix::Bt709,
5 => {
color.matrix = Matrix::Bt2020Ncl;
color.primaries = Primaries::Bt2020;
}
_ => {}
}
if let Some(t) = self.transfer {
color.transfer = t;
}
if let Some(pr) = self.primaries {
color.primaries = pr;
}
if p.color.full_range {
color.range = Range::Full;
}
// (shared with the hardware decoders: `hw_frame::vp9_color`)
let color = crate::hw_frame::vp9_color(p.width, p.height, p.color.color_space, p.color.full_range, self.transfer, self.primaries);
let (pts, draft) = (p.pts, p.draft);
if p.color.color_space == 7 {
// RGB (profiles 1 / 3, 4:4:4): the planes carry G, B, R.
@@ -528,25 +547,8 @@ impl Av1Decoder {
let draft = p.draft;
let w = p.width as usize;
let h = p.height as usize;
let mut color = filmcraft_color::ColorInfo::REC709;
if let Some(m) = filmcraft_color::Matrix::from_code(p.matrix_coefficients) {
color.matrix = m;
} else {
color.matrix = filmcraft_frame::default_matrix(p.width, p.height);
}
if let Some(t) = filmcraft_color::Transfer::from_code(p.transfer_characteristics) {
color.transfer = t;
}
color.primaries = match p.color_primaries {
9 => filmcraft_color::Primaries::Bt2020,
12 => filmcraft_color::Primaries::P3D65,
5 => filmcraft_color::Primaries::Bt601_625,
6 => filmcraft_color::Primaries::Bt601_525,
_ => filmcraft_color::Primaries::Bt709,
};
if p.full_range {
color.range = filmcraft_color::Range::Full;
}
// (shared with the hardware decoders: `hw_frame::av1_color`)
let color = crate::hw_frame::av1_color(p.width, p.height, p.matrix_coefficients, p.transfer_characteristics, p.color_primaries, p.full_range);
let chroma = match (p.subsampling_x, p.subsampling_y) {
(1, 1) => filmcraft_frame::Chroma::C420,
(1, 0) => filmcraft_frame::Chroma::C422,
@@ -668,6 +670,72 @@ pub fn prores_factory(e: &SampleEntry) -> Option<Result<Box<dyn VideoDecoder>>>
matches!(e.codec, CodecConfig::ProRes { .. }).then(|| Ok(Box::new(ProResDecoder) as Box<dyn VideoDecoder>))
}
/// Our APV (Advanced Professional Video, RFC 9924) decoder (every frame is intra; tiles decode in
/// parallel).
pub struct ApvDecoder;
/// Convert a decoded APV frame to a [`VideoFrame`].
pub fn apv_to_video_frame(f: filmcraft_apv::Frame) -> VideoFrame {
use std::sync::Arc;
let chroma = match f.chroma {
filmcraft_apv::ChromaFormat::Yuv422 => filmcraft_frame::Chroma::C422,
filmcraft_apv::ChromaFormat::Monochrome | filmcraft_apv::ChromaFormat::Yuv444 | filmcraft_apv::ChromaFormat::Yuv4444 => filmcraft_frame::Chroma::C444,
};
let mut color = filmcraft_color::ColorInfo { matrix: filmcraft_frame::default_matrix(f.width, f.height), ..filmcraft_color::ColorInfo::REC709 };
if let Some(m) = filmcraft_color::Matrix::from_code(f.color.matrix) {
color.matrix = m;
}
if let Some(t) = filmcraft_color::Transfer::from_code(f.color.transfer) {
color.transfer = t;
}
if let Some(p) = primaries_from_code(f.color.primaries) {
color.primaries = p;
}
if f.color.full_range {
color.range = filmcraft_color::Range::Full;
}
let neutral = 1u16 << f.bit_depth.saturating_sub(1);
let (cb, cr) = if f.chroma == filmcraft_apv::ChromaFormat::Monochrome {
let u = vec![neutral; (f.width as usize) * (f.height as usize)];
(u.clone(), u)
} else {
(f.cb, f.cr)
};
let data = if f.bit_depth == 8 {
let to8 = |p: Vec<u16>| Arc::new(p.into_iter().map(|v| v as u8).collect::<Vec<u8>>());
filmcraft_frame::PixelData::Yuv8 { planes: [to8(f.y), to8(cb), to8(cr)], chroma, alpha: f.alpha.map(to8) }
} else {
filmcraft_frame::PixelData::Yuv16 {
planes: [Arc::new(f.y), Arc::new(cb), Arc::new(cr)],
chroma,
bits: f.bit_depth as u32,
alpha: f.alpha.map(Arc::new),
}
};
VideoFrame { width: f.width, height: f.height, data, color, par: (1, 1), pts: filmcraft_time::Tick::ZERO }
}
impl VideoDecoder for ApvDecoder {
fn decode(&mut self, sample: &[u8], pts: i64) -> Result<Vec<DecodedFrame>> {
let f = filmcraft_apv::decode_frame(sample).map_err(|e| CodecError::Decode(e.to_string()))?;
Ok(vec![DecodedFrame { pts, frame: apv_to_video_frame(f), draft: false }])
}
fn flush(&mut self) -> Vec<DecodedFrame> {
Vec::new()
}
fn reset(&mut self) {}
fn name(&self) -> &str {
"FilmCraft APV"
}
fn intra_only(&self) -> bool {
true
}
}
pub fn apv_factory(e: &SampleEntry) -> Option<Result<Box<dyn VideoDecoder>>> {
matches!(e.codec, CodecConfig::Apv(_)).then(|| Ok(Box::new(ApvDecoder) as Box<dyn VideoDecoder>))
}
/// Our DNxHD / DNxHR (VC-3) decoder (every frame is intra; macroblock rows decode in parallel).
pub struct DnxDecoder;
@@ -0,0 +1,50 @@
//! Matroska audio made by ffmpeg, against ffmpeg's decode (#236). ffmpeg stamps blocks to the
//! millisecond, so at 44.1 kHz AAC (1024-sample), MP3 (1152) and AC-3 (1536) packets are each
//! stamped up to half a millisecond off their exact start. Read back block by block, as playback
//! reads it, a tone must stay as smooth as ffmpeg's decode of it: no holes or overlaps between
//! packets.
mod common;
use common::*;
/// Largest |second difference| of `x` from `from` on: a click shows up as a spike.
fn roughness(x: &[f32], from: usize) -> f32 {
x.windows(3).skip(from).map(|w| (w[0] - 2.0 * w[1] + w[2]).abs()).fold(0.0, f32::max)
}
#[test]
fn matroska_audio_plays_without_holes() {
let ff = filmcraft_testkit::require_ffmpeg!();
let cases: [(&str, u32, &[&str]); 5] = [
("tone_aac_48k.mkv", 48_000, &["-c:a", "aac", "-b:a", "160k"]),
("tone_aac_44k.mkv", 44_100, &["-c:a", "aac", "-b:a", "160k"]),
("tone_mp3_44k.mkv", 44_100, &["-c:a", "libmp3lame", "-b:a", "192k"]),
("tone_mp2_44k.mkv", 44_100, &["-c:a", "mp2", "-b:a", "192k"]),
("tone_ac3_44k.mkv", 44_100, &["-c:a", "ac3", "-b:a", "192k"]),
];
for (name, rate, codec) in cases {
let input = format!("sine=frequency=440:sample_rate={rate}:duration=3");
let args: Vec<&str> = ["-f", "lavfi", "-i", input.as_str(), "-af", "volume=4"].iter().copied().chain(codec.iter().copied()).collect();
let Some(path) = fixture(&ff, name, &args) else {
eprintln!("SKIPPED ({name}): this ffmpeg can't encode it");
continue;
};
let src = filmcraft_codecs::open_bytes(name, bytes(&path)).unwrap();
assert_eq!(src.info().audio().map(|a| a.sample_rate), Some(rate), "{name}");
let frames = rate as usize * 29 / 10;
let mut ours = Vec::with_capacity(frames);
while ours.len() < frames {
let n = 512.min(frames - ours.len());
ours.extend_from_slice(&src.audio(ours.len() as i64, n, rate).unwrap().channels[0]);
}
let theirs = ffmpeg_audio_f32(&ff, &path, &[]);
// past the codecs' start-up (priming, MP3's first frame)
let skip = rate as usize / 10;
let (r_ours, r_theirs) = (roughness(&ours, skip), roughness(&theirs[..theirs.len().min(frames)], skip));
let peak = ours.iter().skip(skip).fold(0f32, |m, x| m.max(x.abs()));
println!("{name}: peak {peak:.3}, max |second difference| ours {r_ours:.5}, ffmpeg {r_theirs:.5}");
assert!(peak > 0.4, "{name}: the tone is missing (peak {peak})");
assert!(r_ours < 2.0 * r_theirs + 0.005, "{name}: clicks between packets (|Δ²| {r_ours} vs ffmpeg's {r_theirs})");
}
}
+25 -10
View File
@@ -13,19 +13,33 @@ use filmcraft_media::sequence::{FrameLoader, ImageSequenceSource, Numbered, sequ
use filmcraft_media::{FrameRequest, MediaKind, MediaSource};
use filmcraft_time::FrameRate;
/// Twelve numbered stills written by ffmpeg, shared by every test that uses `ext` (#125).
///
/// ffmpeg writes into a private temporary directory, which is renamed into place only when all
/// twelve frames are there, so a test never reads a sequence another test is still writing (a
/// half-written PNG decodes as missing, and the sequence then holds the previous frame). A test that
/// loses the race keeps the winner's copy. The `.complete` marker tells a finished cache from a
/// directory left by an older, non-atomic generator (hence the `_v2` name).
fn make_sequence(ff: &Path, ext: &str, codec_args: &[&str]) -> Option<std::path::PathBuf> {
let d = dir().join(format!("seq_{ext}"));
let first = d.join(format!("img_0001.{ext}"));
if first.exists() && d.join(format!("img_0012.{ext}")).exists() {
let d = dir().join(format!("seq_{ext}_v2"));
let complete = |d: &Path| d.join(".complete").exists();
if complete(&d) {
return Some(d);
}
let _ = std::fs::create_dir_all(&d);
let pattern = d.join(format!("img_%04d.{ext}"));
let tmp = filmcraft_testkit::fixtures::temp_path(&d);
std::fs::create_dir_all(&tmp).ok()?;
let pattern = tmp.join(format!("img_%04d.{ext}"));
let mut args = vec!["-y", "-v", "error", "-f", "lavfi", "-i", "testsrc2=size=96x64:rate=25", "-frames:v", "12"];
args.extend_from_slice(codec_args);
args.extend_from_slice(&["-f", "image2"]);
let st = std::process::Command::new(ff).args(&args).arg(&pattern).status().ok()?;
st.success().then_some(d)
let made = std::process::Command::new(ff).args(&args).arg(&pattern).status().is_ok_and(|st| st.success())
&& (1..=12).all(|n| tmp.join(format!("img_{n:04}.{ext}")).exists())
&& std::fs::write(tmp.join(".complete"), b"").is_ok();
if made && std::fs::rename(&tmp, &d).is_ok() {
return Some(d);
}
let _ = std::fs::remove_dir_all(&tmp);
complete(&d).then_some(d)
}
fn loader() -> FrameLoader {
@@ -104,9 +118,9 @@ fn webp_sequence_lossless() {
fn missing_frames_hold_the_previous_frame() {
let ff = filmcraft_testkit::require_ffmpeg!();
let Some(d) = make_sequence(&ff, "png", &[]) else { return };
// a copy without frames 5 and 6
let gap = dir().join("seq_png_gap");
let _ = std::fs::create_dir_all(&gap);
// a private copy without frames 5 and 6
let gap = filmcraft_testkit::fixtures::temp_path(&dir().join("seq_png_gap"));
std::fs::create_dir_all(&gap).unwrap();
for n in (1..=12).filter(|n| *n != 5 && *n != 6) {
let name = format!("img_{n:04}.png");
std::fs::copy(d.join(&name), gap.join(&name)).unwrap();
@@ -119,6 +133,7 @@ fn missing_frames_hold_the_previous_frame() {
assert_eq!(at(4), at(3));
assert_eq!(at(5), at(3));
assert_ne!(at(6), at(3));
let _ = std::fs::remove_dir_all(&gap);
}
#[test]
+55
View File
@@ -0,0 +1,55 @@
//! MP4 audio against ffmpeg: ffmpeg writes an AAC track (with the edit list that skips the encoder
//! priming), and reads at another sample rate must land on the same source samples as the native
//! decode, however the requests are cut.
mod common;
use common::*;
/// A source whose rate differs from the sequence's is resampled per request. An AAC track in MP4
/// carries an edit list that skips the encoder priming (ffmpeg: 1024 samples), and every request
/// must still land on the same source samples: reading at 44.1 kHz in video-frame-sized pieces (as
/// an export does) equals the 48 kHz decode interpolated at the same times.
#[test]
fn aac_mp4_resampled_reads_follow_the_edit_list() {
let ff = filmcraft_testkit::require_ffmpeg!();
let Some(f) = fixture(&ff, "tone_aac_48k.mp4", &["-f", "lavfi", "-i", "sine=frequency=440:sample_rate=48000:d=3", "-c:a", "aac", "-b:a", "128k"]) else {
panic!("could not write the AAC fixture")
};
let src = filmcraft_codecs::open_bytes("tone_aac_48k.mp4", bytes(&f)).unwrap();
let native = src.audio(0, 2 * 48_000, 48_000).unwrap().channels[0].clone();
let ratio = 48_000.0 / 44_100.0;
// 30 fps at 44.1 kHz, starting mid-file and off the 48 kHz grid
let (mut start, piece) = (22_051i64, 1470usize);
let (mut worst, mut peak) = (0f32, 0f32);
for _ in 0..20 {
let got = src.audio(start, piece, 44_100).unwrap();
for (k, &v) in got.channels[0].iter().enumerate() {
let pos = (start + k as i64) as f64 * ratio;
let i0 = pos.floor() as usize;
let want = native[i0] + (native[i0 + 1] - native[i0]) * (pos - i0 as f64) as f32;
worst = worst.max((v - want).abs());
peak = peak.max(v.abs());
}
start += piece as i64;
}
let level = native.iter().fold(0f32, |m, v| m.max(v.abs()));
assert!(peak <= level * 1.01, "resampled peak {peak} above the source's {level}");
assert!(worst < 1e-4, "resampled read differs from the source by {worst}");
}
/// ffmpeg's MP4 muxer skips the B-frame reorder delay with an edit list, so the media (`mdhd`) runs
/// one frame past what plays. The clip must be as long as ffmpeg decodes it: 60 frames, not 61.
#[test]
fn bframe_mp4_duration_is_the_decoded_frame_count() {
let ff = filmcraft_testkit::require_ffmpeg!();
let args = ["-f", "lavfi", "-i", "testsrc2=size=160x96:rate=30:duration=2", "-c:v", "libx264", "-bf", "2", "-pix_fmt", "yuv420p"];
let Some(f) = fixture(&ff, "bframes_x264.mp4", &args) else {
eprintln!("SKIPPED: ffmpeg without libx264");
return;
};
let n = ffmpeg_frames(&ff, &f, "yuv420p").len() / (160 * 96 * 3 / 2);
assert_eq!(n, 60);
let src = filmcraft_codecs::open_bytes("bframes_x264.mp4", bytes(&f)).unwrap();
assert_eq!(src.info().duration, filmcraft_time::FrameRate::FPS_30.tick_of(n as i64));
}
+65 -16
View File
@@ -177,15 +177,64 @@ fn open(p: &Path) -> MpegSource {
MpegSource::open(p.file_name().unwrap().to_str().unwrap(), bytes(p)).unwrap()
}
/// ffprobe's video frames in display order: pts (90 kHz).
/// ffprobe's video frames in display order: PTS (90 kHz). A program-stream picture
/// need not carry its own PTS; use the decoder's presentation timestamp for that picture.
fn ffprobe_frame_pts(file: &Path) -> Option<Vec<i64>> {
let fp = filmcraft_testkit::ffprobe()?;
let fp = filmcraft_testkit::ffprobe_or_skip("MPEG video frame presentation timestamps")?;
let o = std::process::Command::new(fp)
.args(["-v", "error", "-select_streams", "v:0", "-show_entries", "frame=pts", "-of", "csv=p=0"])
.args(["-v", "error", "-select_streams", "v:0", "-show_entries", "frame=pts,best_effort_timestamp", "-of", "json"])
.arg(file)
.output()
.ok()?;
Some(String::from_utf8_lossy(&o.stdout).lines().filter_map(|l| l.trim().trim_end_matches(',').parse().ok()).collect())
.unwrap_or_else(|error| panic!("{}: cannot run ffprobe: {error}", file.display()));
assert!(o.status.success(), "{}: ffprobe failed: {}", file.display(), String::from_utf8_lossy(&o.stderr));
Some(parse_ffprobe_frame_pts(&o.stdout).unwrap_or_else(|error| panic!("{}: {error}", file.display())))
}
fn parse_ffprobe_frame_pts(bytes: &[u8]) -> Result<Vec<i64>, String> {
let data: serde_json::Value = serde_json::from_slice(bytes).map_err(|error| format!("invalid ffprobe JSON: {error}"))?;
let frames = data.get("frames").and_then(serde_json::Value::as_array).ok_or("ffprobe output is missing its frames array")?;
if frames.is_empty() {
return Err("ffprobe returned no video frames".into());
}
frames
.iter()
.enumerate()
.map(|(index, frame)| {
let field = if frame.get("pts").is_some() { "pts" } else { "best_effort_timestamp" };
frame.get(field).and_then(serde_json::Value::as_i64).ok_or_else(|| format!("ffprobe video frame {index} is missing an integer {field}"))
})
.collect()
}
#[test]
fn ffprobe_pts_preserve_missing_raw_timestamps_and_side_data() {
let output = br#"{"frames":[
{"pts":131400,"best_effort_timestamp":131401,"side_data_list":[{},{}]},
{"best_effort_timestamp":135000,"side_data_list":[{}]},
{"pts":138600,"best_effort_timestamp":138600}
]}"#;
assert_eq!(parse_ffprobe_frame_pts(output).unwrap(), [131400, 135000, 138600], "keep every display frame, with raw PTS preferred when present");
assert_eq!(parse_ffprobe_frame_pts(br#"{"frames":[{"pts":-3600},{"pts":0},{"pts":9223372036854775807}]}"#).unwrap(), [-3600, 0, i64::MAX]);
}
#[test]
fn malformed_ffprobe_pts_fail_instead_of_dropping_frames() {
for output in [
"not JSON",
"{}",
r#"{"frames":{}}"#,
r#"{"frames":[]}"#,
r#"{"frames":[{}]}"#,
r#"{"frames":[null]}"#,
r#"{"frames":[{"pts":"N/A","best_effort_timestamp":135000}]}"#,
r#"{"frames":[{"pts":null,"best_effort_timestamp":135000}]}"#,
r#"{"frames":[{"pts":1.5}]}"#,
r#"{"frames":[{"pts":9223372036854775808}]}"#,
r#"{"frames":[{"best_effort_timestamp":"135000"}]}"#,
r#"{"frames":[{"pts":131400},{"best_effort_timestamp":null},{"pts":138600}]}"#,
] {
assert!(parse_ffprobe_frame_pts(output.as_bytes()).is_err(), "malformed oracle output must fail: {output}");
}
}
/// Decode every frame in order and `seeks` random frames; compare with ffmpeg within `tol`.
@@ -245,7 +294,7 @@ fn check_video(ff: &Path, file: &Path, tol: u16, seeks: usize) -> (u16, f64) {
/// Our decoded audio from the first audio frame against ffmpeg's decode. Returns the max error.
fn check_audio(ff: &Path, file: &Path, tol: f32) -> f32 {
let src = open(file);
let a = src.info().audio.clone().unwrap();
let a = src.info().audio().cloned().unwrap();
let (start, len) = src.audio_extent().unwrap();
let want = ffmpeg_audio_f32(ff, file, &[]);
let ch = a.channels as usize;
@@ -314,7 +363,7 @@ fn transport_stream_mpeg2_and_mp2() {
let (worst, psnr) = check_video(&ff, &f, 4, 12);
assert!(psnr >= 58.0, "PSNR {psnr}");
println!("ts_mpeg2_mp2: video max diff {worst}, PSNR {psnr:.2} dB");
let a = src.info().audio.clone().unwrap();
let a = src.info().audio().cloned().unwrap();
assert_eq!((a.codec.as_str(), a.sample_rate, a.channels), ("MPEG Audio", 48_000, 2));
let e = check_audio(&ff, &f, 2e-4);
println!("ts_mpeg2_mp2: MP2 audio max error {e:.2e}");
@@ -337,7 +386,7 @@ fn bdav_h264_with_lpcm_sample_exact() {
let f = make(&ff, "m2ts_h264_lpcm.m2ts");
let src = open(&f);
assert_eq!(src.info().container, "MPEG-2 TS (BDAV/AVCHD)");
let a = src.info().audio.clone().unwrap();
let a = src.info().audio().cloned().unwrap();
assert_eq!((a.codec.as_str(), a.channels, a.bits_per_sample), ("LPCM (Blu-ray)", 2, Some(24)));
check_video(&ff, &f, 0, 10);
assert_eq!(check_audio(&ff, &f, 0.0), 0.0);
@@ -349,7 +398,7 @@ fn avchd_ac3_audio() {
let f = make(&ff, "m2ts_h264_ac3.mts");
let src = open(&f);
check_video(&ff, &f, 0, 6);
let a = src.info().audio.clone().unwrap();
let a = src.info().audio().cloned().unwrap();
if filmcraft_codecs::audio::AC3_DECODER {
assert_eq!((a.codec.as_str(), a.channels), ("AC-3", 2));
// AC-3 zero-bit mantissas carry decoder-specific dither (A/52 §7.3.4)
@@ -370,7 +419,7 @@ fn transport_stream_hevc_and_latm() {
let f = make(&ff, "ts_hevc_latm.ts");
let src = open(&f);
assert_eq!(src.info().video.as_ref().unwrap().codec, "HEVC");
assert_eq!(src.info().audio.as_ref().unwrap().codec, "AAC (LATM)");
assert_eq!(src.info().audio().unwrap().codec, "AAC (LATM)");
check_video(&ff, &f, 0, 8);
let e = check_audio(&ff, &f, 1e-4);
println!("ts_hevc_latm: AAC (LATM) max error {e:.2e}");
@@ -389,17 +438,17 @@ fn program_streams() {
// DVD LPCM: sample-exact
let f = make(&ff, "ps_mpeg2_lpcm.vob");
let src = open(&f);
assert_eq!(src.info().audio.as_ref().unwrap().codec, "LPCM (DVD)");
assert_eq!(src.info().audio.as_ref().unwrap().bits_per_sample, Some(24));
assert_eq!(src.info().audio().unwrap().codec, "LPCM (DVD)");
assert_eq!(src.info().audio().unwrap().bits_per_sample, Some(24));
assert_eq!(check_audio(&ff, &f, 0.0), 0.0);
check_video(&ff, &f, 4, 4);
let f = make(&ff, "ps_mpeg2_lpcm16.vob");
assert_eq!(open(&f).info().audio.as_ref().unwrap().bits_per_sample, Some(16));
assert_eq!(open(&f).info().audio().unwrap().bits_per_sample, Some(16));
assert_eq!(check_audio(&ff, &f, 0.0), 0.0);
// AC-3 in private stream 1
let f = make(&ff, "ps_mpeg2_ac3.vob");
let src = open(&f);
assert_eq!(src.info().audio.as_ref().unwrap().codec, "AC-3");
assert_eq!(src.info().audio().unwrap().codec, "AC-3");
let e = check_audio(&ff, &f, 5e-3);
println!("ps_mpeg2_ac3: AC-3 max error {e:.2e}");
// MPEG-1 system stream
@@ -457,7 +506,7 @@ fn standalone_mp2_file() {
let ff = filmcraft_testkit::require_ffmpeg!();
let f = make(&ff, "audio.mp2");
let src = filmcraft_codecs::open_bytes("audio.mp2", bytes(&f)).unwrap();
let a = src.info().audio.clone().unwrap();
let a = src.info().audio().cloned().unwrap();
assert_eq!((a.sample_rate, a.channels), (48_000, 2));
let want = ffmpeg_audio_f32(&ff, &f, &[]);
let n = want.len() / 2;
@@ -483,7 +532,7 @@ fn truncated_and_corrupt_files_never_panic() {
let _ = s.video_frame(FrameRequest::full(v.frame_rate.tick_of(i)));
}
}
if let Some(a) = &info.audio {
if let Some(a) = info.audio() {
let _ = s.audio(rng.below(48_000) as i64, 2048, a.sample_rate);
}
};
+7 -7
View File
@@ -179,7 +179,7 @@ fn check_video(ff: &Path, file: &Path, pix_fmt: &str, chroma: (usize, usize), bp
/// Our audio, as interleaved f32, equals ffmpeg's decode exactly.
fn check_audio_exact(ff: &Path, file: &Path, extra: &[&str]) {
let src = open(file);
let a = src.info().audio.clone().unwrap();
let a = src.info().audio().cloned().unwrap();
let ch = a.channels as usize;
let want = ffmpeg_audio_f32(ff, file, extra);
let n = want.len() / ch;
@@ -236,7 +236,7 @@ fn h264_2997_drop_frame_timecode_and_24_bit_audio() {
assert!(tc.drop_frame);
assert_eq!(tc.format(), "01:00:00;00");
assert_eq!(info.start_timecode, Some(107_892));
assert_eq!(info.audio.as_ref().unwrap().bits_per_sample, Some(24));
assert_eq!(info.audio().unwrap().bits_per_sample, Some(24));
check_video(&ff, &f, "yuv420p", (2, 2), 1, 0, 10);
check_audio_exact(&ff, &f, &[]);
}
@@ -299,14 +299,14 @@ fn op_atom_video_and_audio_files() {
let v = make(&ff, "mxf_atom_dnxhr.mxf");
let src = open(&v);
assert_eq!(src.info().container, "MXF OP-Atom");
assert!(src.info().audio.is_none());
assert!(!src.info().has_audio());
assert_eq!(src.file().tracks[0].wrapping, filmcraft_mxf::Wrapping::Clip);
check_video(&ff, &v, "yuv422p", (2, 1), 1, 2, 10);
let a = make(&ff, "mxf_atom_pcm.mxf");
let src = open(&a);
assert_eq!(src.info().kind, MediaKind::AudioOnly);
assert_eq!(src.info().container, "MXF OP-Atom");
let ai = src.info().audio.clone().unwrap();
let ai = src.info().audio().cloned().unwrap();
assert_eq!((ai.sample_rate, ai.channels, ai.bits_per_sample), (48_000, 1, Some(24)));
assert_eq!(src.info().duration, filmcraft_time::Tick::from_units(48_000, 48_000));
check_audio_exact(&ff, &a, &[]);
@@ -317,7 +317,7 @@ fn d10_aes3_elements_sample_exact() {
let ff = filmcraft_testkit::require_ffmpeg!();
let f = make(&ff, "mxf_d10.mxf");
let src = open(&f);
let a = src.info().audio.clone().unwrap();
let a = src.info().audio().cloned().unwrap();
assert_eq!(a.codec, "AES3 PCM");
assert!(a.channels >= 4, "{}", a.channels);
let v = src.info().video.as_ref().unwrap();
@@ -347,7 +347,7 @@ fn truncated_and_corrupt_files_never_panic() {
let _ = s.video_frame(FrameRequest::full(t));
}
}
if let Some(a) = &info.audio {
if let Some(a) = info.audio() {
let _ = s.audio(rng.below(96_000) as i64, 4000, a.sample_rate);
}
}
@@ -364,7 +364,7 @@ fn truncated_and_corrupt_files_never_panic() {
let _ = s.video_frame(FrameRequest::full(s.info().frame_rate().tick_of(i)));
}
}
if s.info().audio.is_some() {
if s.info().has_audio() {
let _ = s.audio(0, 9600, 48_000);
}
}
+3 -3
View File
@@ -88,7 +88,7 @@ fn snr_db(reference: &[f32], ours: &[f32]) -> f64 {
/// (our continuous decode interleaved, ffmpeg's decode interleaved, channels, rate)
fn decode_both(ff: &Path, file: &Path, decoder: &[&str]) -> (OggSource, Vec<f32>, Vec<f32>, usize, u32) {
let src = OggSource::open(file.file_name().unwrap().to_str().unwrap(), bytes(file)).unwrap();
let a = src.info().audio.clone().unwrap();
let a = src.info().audio().cloned().unwrap();
let ch = a.channels as usize;
let mut args: Vec<&str> = decoder.to_vec();
args.extend_from_slice(&["-i", file.to_str().unwrap(), "-map", "0:a:0", "-f", "f32le", "-"]);
@@ -134,7 +134,7 @@ fn opus_celt_music_matches_libopus() {
assert_eq!(src.info().container, "Ogg");
assert_eq!(src.info().kind, MediaKind::AudioOnly);
assert_eq!((rate, ch), (48_000, 2));
assert_eq!(src.info().audio.as_ref().unwrap().codec, "Opus");
assert_eq!(src.info().audio().unwrap().codec, "Opus");
let snr = snr_db(&want, &ours);
eprintln!("{}: {} samples, SNR {snr:.1} dB vs libopus", f.display(), want.len() / ch);
assert!(snr >= 50.0, "SNR {snr:.1} dB");
@@ -173,7 +173,7 @@ fn vorbis_matches_ffmpeg() {
let ff = filmcraft_testkit::require_ffmpeg!();
let f = make(&ff, "ogg_vorbis.ogg");
let (src, ours, want, ch, rate) = decode_both(&ff, &f, &[]);
assert_eq!(src.info().audio.as_ref().unwrap().codec, "Vorbis");
assert_eq!(src.info().audio().unwrap().codec, "Vorbis");
assert_eq!((rate, ch), (44_100, 2));
let snr = snr_db(&want, &ours);
eprintln!("{}: SNR {snr:.1} dB vs ffmpeg's Vorbis decoder", f.display());
+63
View File
@@ -0,0 +1,63 @@
//! Decoded pictures are written into recycled plane buffers (`filmcraft_frame::pool`). A recycled
//! buffer still holds an old picture, so the decoder must overwrite every byte: here the pool is
//! primed with buffers full of 0xAA before decoding, and every frame must still match ffmpeg bit
//! for bit. (Its own process: the pool is global.)
mod common;
use std::sync::Arc;
use common::*;
use filmcraft_frame::{Chroma, PixelData, VideoFrame};
use filmcraft_media::FrameRequest;
#[test]
fn dirty_recycled_planes_decode_bit_exact() {
let ff = filmcraft_testkit::require_ffmpeg!();
// 640×480: luma and chroma planes are both large enough to be pooled
let Some(file) = fixture(
&ff,
"pool_h264.mp4",
&[
"-f",
"lavfi",
"-i",
"testsrc2=s=640x480:r=24:d=1,noise=alls=10:allf=t",
"-c:v",
"libx264",
"-preset",
"fast",
"-x264-params",
"bframes=2:keyint=12",
"-pix_fmt",
"yuv420p",
],
) else {
return;
};
let (w, h) = (640usize, 480usize);
let raw = ffmpeg_frames(&ff, &file, "yuv420p");
let fsize = w * h * 3 / 2;
let n = raw.len() / fsize;
assert!(n >= 20, "{n} frames");
let reference = |i: usize| raw_planes(&raw[i * fsize..(i + 1) * fsize], w, h, w / 2, h / 2, 1);
// prime the pool with dirty buffers of exactly the plane sizes the decoder asks for
for _ in 0..8 {
let dirty = |len: usize| Arc::new(vec![0xAAu8; len]);
let f = VideoFrame {
data: PixelData::Yuv8 { planes: [dirty(w * h), dirty(w * h / 4), dirty(w * h / 4)], chroma: Chroma::C420, alpha: None },
..VideoFrame::rgba8(w as u32, h as u32, vec![0; w * h * 4])
};
filmcraft_frame::pool::recycle(Arc::new(f));
}
let reused_before = filmcraft_frame::pool::stats().reused;
let src = filmcraft_codecs::open_bytes("pool_h264.mp4", bytes(&file)).expect("open");
let rate = src.info().frame_rate();
for i in 0..n {
let f = src.video_frame(FrameRequest::full(rate.tick_of(i as i64))).expect("frame");
assert_eq!(max_diff(&planes(&f), &reference(i)), 0, "frame {i} decoded into a recycled buffer");
}
assert!(filmcraft_frame::pool::stats().reused > reused_before, "the decoder took the dirty pooled buffers");
}
+1
View File
@@ -16,5 +16,6 @@ thiserror = { workspace = true }
[dev-dependencies]
proptest = { workspace = true }
filmcraft-geom = { workspace = true }
filmcraft-media = { workspace = true }
serde_json = { workspace = true }
+213
View File
@@ -0,0 +1,213 @@
//! Sequence Settings ▸ "Scale motion effects proportionally when changing frame size".
//!
//! Motion's Position is in sequence pixels and its Scale is relative to the source, so a new frame
//! size leaves every clip where it was in pixels: smaller and off-centre in a larger frame, cropped
//! in a smaller one. With the option on, the whole picture scales instead, by one factor about the
//! frame centre. On an aspect change (16:9 → 9:16) that factor is the one that fits the old frame
//! inside the new one (`min(W1/W0, H1/H0)`), so nothing is cropped and the old composition sits
//! centred, letterboxed or pillarboxed, rather than filling the new frame.
use filmcraft_project::{ParamValue, Sequence, TrackKind};
/// Motion Scale's upper bound (the parameter's range, in %).
const MAX_SCALE: f64 = 10_000.0;
/// The uniform factor `k` and the old and new frame centres for `from` → `to`, or `None` for an
/// empty frame on either side.
fn fit(from: (u32, u32), to: (u32, u32)) -> Option<(f64, (f64, f64), (f64, f64))> {
if from.0 == 0 || from.1 == 0 || to.0 == 0 || to.1 == 0 {
return None;
}
let (w0, h0, w1, h1) = (f64::from(from.0), f64::from(from.1), f64::from(to.0), f64::from(to.1));
let k = (w1 / w0).min(h1 / h0);
(k.is_finite() && k > 0.0).then_some((k, (w0 / 2.0, h0 / 2.0), (w1 / 2.0, h1 / 2.0)))
}
/// Rescale the Motion of every video clip in `seq` for a frame-size change from `from` to `to`:
/// static values and every keyframe. Returns how many clips were rescaled (0 when the size does not
/// change).
///
/// - Position maps through `p' = c1 + k·(p − c0)`. An automatic (NaN) position stays automatic: it
/// follows the frame centre on its own.
/// - Scale and Scale Width are multiplied by `k`, up to the parameter's 10 000 %. A clip with Scale to
/// Frame Size keeps its Scale, because the renderer refits it to the new frame anyway.
/// - Anchor Point (source pixels) and Rotation do not depend on the frame size and stay as they are.
/// - Only the intrinsic Motion effect changes, as the option's name says: the points of standard
/// effects (a Mirror centre, the Transform effect's Position) keep their pixel values.
pub fn scale_motion(seq: &mut Sequence, from: (u32, u32), to: (u32, u32)) -> usize {
if from == to {
return 0;
}
let Some((k, c0, c1)) = fit(from, to) else { return 0 };
let mut n = 0;
for track in seq.all_tracks_mut().filter(|t| t.kind == TrackKind::Video) {
for item in &mut track.items {
let refits = item.scale_to_frame;
let Some(motion) = item.effect_mut("motion") else { continue };
if let Some(p) = motion.param_mut("position") {
p.map_values(|v| match v {
ParamValue::Vec2(mut v) => {
v.x = c1.0 + k * (v.x - c0.0);
v.y = c1.1 + k * (v.y - c0.1);
ParamValue::Vec2(v)
}
other => other,
});
}
if !refits {
for id in ["scale", "scale_width"] {
if let Some(p) = motion.param_mut(id) {
p.map_values(|v| match v {
ParamValue::Float(s) => ParamValue::Float((s * k).clamp(0.0, MAX_SCALE)),
other => other,
});
}
}
}
n += 1;
}
}
n
}
#[cfg(test)]
mod tests {
use super::*;
use filmcraft_project::{ClipId, ItemId, Keyframe, Label, Project, SequenceSettings, TrackItem, find_effect};
use filmcraft_time::Tick;
fn seq_with(items: Vec<TrackItem>) -> Sequence {
let mut p = Project::new("t");
let s = p.new_sequence("s", SequenceSettings::default(), 2, 1, None);
let mut seq = p.sequence(s).unwrap().clone();
seq.video_tracks[0].items = items;
seq
}
fn clip(id: u64, pos: (f64, f64), scale: f64) -> TrackItem {
let mut motion = find_effect("motion").unwrap().instance();
motion.param_mut("position").unwrap().value = ParamValue::Vec2(filmcraft_geom::Vec2::new(pos.0, pos.1));
motion.param_mut("scale").unwrap().value = ParamValue::Float(scale);
TrackItem {
id: ClipId(id),
item: ItemId(1),
name: format!("c{id}"),
label: Label::Iris,
start: Tick(id as i64 * 1_000_000_000_000),
duration: Tick(1_000_000_000_000),
source_in: Tick::ZERO,
speed: 1.0,
reverse: false,
enabled: true,
link: None,
group: None,
effects: vec![motion],
markers: vec![],
gain_db: 0.0,
frame_hold: None,
scale_to_frame: false,
essential: None,
multicam: None,
time_interpolation: Default::default(),
hold_filters: false,
field_options: None,
source_channels: Vec::new(),
audio_stream: 0,
graphic: None,
}
}
fn pos(seq: &Sequence, i: usize) -> (f64, f64) {
match seq.video_tracks[0].items[i].effect("motion").unwrap().param("position").unwrap().value {
ParamValue::Vec2(v) => (v.x, v.y),
_ => panic!("position is a point"),
}
}
fn scale(seq: &Sequence, i: usize) -> f64 {
match seq.video_tracks[0].items[i].effect("motion").unwrap().param("scale").unwrap().value {
ParamValue::Float(s) => s,
_ => panic!("scale is a number"),
}
}
fn close(a: (f64, f64), b: (f64, f64)) -> bool {
(a.0 - b.0).abs() < 1e-9 && (a.1 - b.1).abs() < 1e-9
}
#[test]
fn same_aspect_scales_position_and_scale_together() {
let mut seq = seq_with(vec![clip(1, (960.0, 540.0), 100.0), clip(2, (300.0, 90.0), 50.0)]);
assert_eq!(scale_motion(&mut seq, (1920, 1080), (1280, 720)), 2);
assert!(close(pos(&seq, 0), (640.0, 360.0)), "{:?}", pos(&seq, 0));
assert!(close(pos(&seq, 1), (200.0, 60.0)), "{:?}", pos(&seq, 1));
assert!((scale(&seq, 0) - 100.0 * 2.0 / 3.0).abs() < 1e-9);
assert!((scale(&seq, 1) - 50.0 * 2.0 / 3.0).abs() < 1e-9);
}
#[test]
fn aspect_change_fits_the_old_frame_inside_the_new_one() {
// 1920x1080 -> 1080x1920: k = min(1080/1920, 1920/1080) = 0.5625, centred
let mut seq = seq_with(vec![clip(1, (960.0, 540.0), 100.0), clip(2, (0.0, 0.0), 100.0)]);
scale_motion(&mut seq, (1920, 1080), (1080, 1920));
assert!(close(pos(&seq, 0), (540.0, 960.0)), "{:?}", pos(&seq, 0));
// the old top-left corner lands on the new frame's left edge, letterboxed vertically
assert!(close(pos(&seq, 1), (0.0, 960.0 - 0.5625 * 540.0)), "{:?}", pos(&seq, 1));
assert!((scale(&seq, 0) - 56.25).abs() < 1e-9);
}
#[test]
fn keyframes_scale_too() {
let mut c = clip(1, (960.0, 540.0), 100.0);
let m = c.effect_mut("motion").unwrap();
m.param_mut("position").unwrap().keyframes = vec![
Keyframe::new(Tick::ZERO, ParamValue::Vec2(filmcraft_geom::Vec2::new(0.0, 0.0))),
Keyframe::new(Tick(10), ParamValue::Vec2(filmcraft_geom::Vec2::new(1920.0, 1080.0))),
];
m.param_mut("scale").unwrap().keyframes = vec![Keyframe::new(Tick::ZERO, ParamValue::Float(100.0)), Keyframe::new(Tick(10), ParamValue::Float(200.0))];
let mut seq = seq_with(vec![c]);
scale_motion(&mut seq, (1920, 1080), (3840, 2160));
let m = seq.video_tracks[0].items[0].effect("motion").unwrap();
let kp: Vec<_> = m.param("position").unwrap().keyframes.iter().map(|k| k.value.clone()).collect();
assert_eq!(kp, vec![ParamValue::Vec2(filmcraft_geom::Vec2::new(0.0, 0.0)), ParamValue::Vec2(filmcraft_geom::Vec2::new(3840.0, 2160.0))]);
let ks: Vec<_> = m.param("scale").unwrap().keyframes.iter().map(|k| k.value.clone()).collect();
assert_eq!(ks, vec![ParamValue::Float(200.0), ParamValue::Float(400.0)]);
}
#[test]
fn automatic_position_stays_automatic() {
let mut seq = seq_with(vec![clip(1, (f64::NAN, f64::NAN), 100.0)]);
scale_motion(&mut seq, (1920, 1080), (1280, 720));
let (x, y) = pos(&seq, 0);
assert!(x.is_nan() && y.is_nan());
assert!((scale(&seq, 0) - 100.0 * 2.0 / 3.0).abs() < 1e-9);
}
#[test]
fn scale_to_frame_clips_keep_their_scale_but_move() {
let mut c = clip(1, (480.0, 270.0), 100.0);
c.scale_to_frame = true;
let mut seq = seq_with(vec![c]);
scale_motion(&mut seq, (1920, 1080), (3840, 2160));
assert!(close(pos(&seq, 0), (960.0, 540.0)), "{:?}", pos(&seq, 0));
assert_eq!(scale(&seq, 0), 100.0);
}
#[test]
fn scale_is_capped_at_the_parameter_range() {
let mut seq = seq_with(vec![clip(1, (1.0, 1.0), 5000.0)]);
scale_motion(&mut seq, (2, 2), (8, 8));
assert_eq!(scale(&seq, 0), MAX_SCALE);
}
#[test]
fn same_size_or_empty_frame_changes_nothing() {
let mut seq = seq_with(vec![clip(1, (100.0, 100.0), 100.0)]);
let before = seq.clone();
assert_eq!(scale_motion(&mut seq, (1920, 1080), (1920, 1080)), 0);
assert_eq!(scale_motion(&mut seq, (0, 1080), (1280, 720)), 0);
assert_eq!(scale_motion(&mut seq, (1920, 1080), (1280, 0)), 0);
// Debug, not ==: the automatic anchor is NaN, which never equals itself
assert_eq!(format!("{:?}", seq.video_tracks[0].items[0].effects), format!("{:?}", before.video_tracks[0].items[0].effects));
}
}
+198 -61
View File
@@ -18,6 +18,7 @@
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
pub mod captions;
pub mod frame_size;
pub mod multicam;
pub mod through;
pub mod transcript;
@@ -35,8 +36,9 @@ pub enum EditError {
NoTrack(TrackId),
#[error("track is locked")]
Locked,
#[error("this edit would break sync on a sync-locked track")]
SyncLockConflict,
/// Names the sync-locked track (`A2`) whose clip is in the way.
#[error("this edit would break sync: {0} is sync-locked and has a clip in the way (turn off its sync lock, or lock the track, to make this edit)")]
SyncLockConflict(String),
#[error("not enough media (handles) for this trim")]
NoHandles,
#[error("edit would make a clip shorter than one frame")]
@@ -351,21 +353,77 @@ pub fn delete_items(seq: &mut Sequence, items: &[ClipId]) -> usize {
n
}
/// Ripple delete track items: remove them and close the resulting gaps.
pub fn ripple_delete_items(seq: &mut Sequence, items: &[ClipId]) -> Result<()> {
// Collect per-track ranges to close (merged), processed right-to-left.
let mut ranges: Vec<(TrackId, TimeRange)> = Vec::new();
/// The stretches a ripple delete of `items` closes, in time order: the time the deleted items
/// covered, less whatever stays on the tracks that lose an item. Every such track then moves up
/// by the same amount, into time that is free on all of them, so later clips stay in sync and
/// nothing is overwritten. A clip and its linked sound with a split edit have different edges;
/// closing each one's own range would move the tracks twice. Locked tracks lose nothing and do
/// not count.
fn ripple_delete_spans(seq: &Sequence, items: &[ClipId]) -> Vec<TimeRange> {
let (mut deleted, mut kept) = (Vec::new(), Vec::new());
for tr in seq.all_tracks().filter(|t| !t.locked && t.items.iter().any(|i| items.contains(&i.id))) {
for i in &tr.items {
if items.contains(&i.id) { deleted.push(i.range()) } else { kept.push(i.range()) }
}
}
let kept = merged(kept);
let mut spans = Vec::new();
for r in merged(deleted) {
let mut at = r.start;
for k in kept.iter().filter(|k| k.overlaps(&r)) {
if k.start > at {
spans.push(TimeRange::from_bounds(at, k.start));
}
at = at.max(k.end());
}
if at < r.end() {
spans.push(TimeRange::from_bounds(at, r.end()));
}
}
spans
}
/// How the UI names a track: `V1`, `A2` (by position), else its name.
pub fn track_label(seq: &Sequence, id: TrackId) -> String {
if let Some(i) = seq.video_tracks.iter().position(|t| t.id == id) {
return format!("V{}", i + 1);
}
if let Some(i) = seq.audio_tracks.iter().position(|t| t.id == id) {
return format!("A{}", i + 1);
}
seq.track(id).map(|t| t.name.clone()).unwrap_or_else(|| format!("track {}", id.0))
}
/// `ranges` in time order, with the ones that touch or overlap joined.
fn merged(mut ranges: Vec<TimeRange>) -> Vec<TimeRange> {
ranges.sort_by_key(|r| r.start);
let mut out: Vec<TimeRange> = Vec::new();
for r in ranges {
match out.last_mut() {
Some(last) if r.start <= last.end() => *last = TimeRange::from_bounds(last.start, last.end().max(r.end())),
_ => out.push(r),
}
}
out
}
/// Ripple delete track items: remove them and close the resulting gaps. Returns the stretches
/// that were closed, in time order. Refused when no gap can close (what stays on the items'
/// tracks covers all of their time): that would be a plain delete.
pub fn ripple_delete_items(seq: &mut Sequence, items: &[ClipId]) -> Result<Vec<TimeRange>> {
// Collect the tracks that lose an item; the gaps close right-to-left.
let mut affected: Vec<TrackId> = Vec::new();
for c in items {
let (tid, it) = seq.find_item(*c).ok_or(EditError::NoItem(*c))?;
ranges.push((tid, it.range()));
let (tid, _) = seq.find_item(*c).ok_or(EditError::NoItem(*c))?;
affected.push(tid);
}
let spans = ripple_delete_spans(seq, items);
if spans.is_empty() && !items.is_empty() {
let locked = affected.iter().all(|t| seq.track(*t).is_some_and(|tr| tr.locked));
return Err(if locked { EditError::Locked } else { EditError::Other("no gap can close: other clips on these tracks cover the deleted time".into()) });
}
// Group identical ranges across tracks (linked A/V) into one ripple.
let mut spans: Vec<TimeRange> = ranges.iter().map(|r| r.1).collect();
spans.sort_by_key(|r| (r.start, r.duration));
spans.dedup();
let mut work = seq.clone();
delete_items(&mut work, items);
let affected: Vec<TrackId> = ranges.iter().map(|r| r.0).collect();
for span in spans.iter().rev() {
// the gap actually closable: from span.start to the next material on affected tracks
for tr in work.all_tracks_mut() {
@@ -377,13 +435,13 @@ pub fn ripple_delete_items(seq: &mut Sequence, items: &[ClipId]) -> Result<()> {
continue;
}
if !on_affected && !track_range_empty(tr, *span) {
return Err(EditError::SyncLockConflict);
return Err(EditError::SyncLockConflict(track_label(seq, tr.id)));
}
shift_track_from(tr, span.end(), -span.duration);
}
}
*seq = work;
Ok(())
Ok(spans)
}
/// Close the gap containing `t` on a track (Ripple Delete on a gap).
@@ -405,7 +463,7 @@ pub fn close_gap(seq: &mut Sequence, track: TrackId, t: Tick) -> Result<()> {
continue;
}
if !track_range_empty(tr, gap) {
return Err(EditError::SyncLockConflict);
return Err(EditError::SyncLockConflict(track_label(seq, tr.id)));
}
}
shift_track_from(tr, gap.end(), -gap.duration);
@@ -505,6 +563,7 @@ pub fn trim(seq: &mut Sequence, clip: ClipId, edge: Edge, mode: TrimMode, delta:
let (tid, it) = seq.find_item(clip).ok_or(EditError::NoItem(clip))?;
let old_end = it.end();
let old_start = it.start;
let own_link = it.link;
let speed = it.speed.abs();
let mut work = seq.clone();
{
@@ -537,9 +596,13 @@ pub fn trim(seq: &mut Sequence, clip: ClipId, edge: Edge, mode: TrimMode, delta:
continue;
}
if tr.id == tid || tr.sync_lock {
if tr.id != tid && shift < Tick::ZERO && !track_range_empty(tr, TimeRange::new(at + shift - Tick(if edge == Edge::In { 1 } else { 0 }), -shift))
{
return Err(EditError::SyncLockConflict);
// the stretch that closes: a shorter head takes it from just after the cut, a
// shorter tail from just before it
let closing = if edge == Edge::In { TimeRange::new(old_start, -shift) } else { TimeRange::new(at + shift, -shift) };
// (the clip's own linked partners are the caller's to trim or leave)
let in_the_way = tr.items.iter().any(|i| i.range().overlaps(&closing) && (own_link.is_none() || i.link != own_link));
if tr.id != tid && shift < Tick::ZERO && in_the_way {
return Err(EditError::SyncLockConflict(track_label(seq, tr.id)));
}
let from = if edge == Edge::In { old_start + Tick(1) } else { old_end };
for i in &mut tr.items {
@@ -559,8 +622,9 @@ pub fn trim(seq: &mut Sequence, clip: ClipId, edge: Edge, mode: TrimMode, delta:
/// Ripple-trim one edge of a group of linked items (e.g. a clip and its audio partners) as a single
/// edit: every member's edge moves by the same delta, later material on the members' tracks and on
/// sync-locked tracks shifts by the change. Only tracks without a group member can block the edit
/// (a sync-locked track whose material would be overwritten).
/// sync-locked tracks shifts by the change, and so does a clip that starts before the cut, reaches
/// past it and is linked to a clip that shifts (a split edit). The edit is refused when it would
/// overwrite material on a sync-locked track, or when such a linked clip has no room to follow.
pub fn ripple_trim_group(seq: &mut Sequence, clips: &[ClipId], edge: Edge, delta: Tick, ctx: &mut EditCtx) -> Result<Tick> {
let Some(&first) = clips.first() else { return Ok(Tick::ZERO) };
let mut d = delta;
@@ -601,32 +665,75 @@ pub fn ripple_trim_group(seq: &mut Sequence, clips: &[ClipId], edge: Edge, delta
origins.push((tid, from));
}
}
let main_from = {
// a shorter or longer head: the cut is the latest member's start (a sound that leads its picture
// starts earlier), whichever member was grabbed
let main_from = if edge == Edge::In {
origins.iter().map(|o| o.1).max().unwrap_or_default()
} else {
let (tid, _) = seq.find_item(first).ok_or(EditError::NoItem(first))?;
origins.iter().find(|(t, _)| *t == tid).map(|o| o.1).unwrap_or_default()
};
let shift = if edge == Edge::In { -d } else { d };
for tr in work.all_tracks_mut() {
if tr.locked {
continue;
}
let member = origins.iter().find(|(t, _)| *t == tr.id).map(|o| o.1);
if member.is_none() && !tr.sync_lock {
continue;
}
let from = member.unwrap_or(main_from);
if member.is_none() && shift < Tick::ZERO {
let at = if edge == Edge::In { from - Tick(1) } else { from };
if !track_range_empty(tr, TimeRange::new(at + shift, -shift)) {
return Err(EditError::SyncLockConflict);
// where the shift starts on each track that ripples: the members' tracks and the sync-locked ones
let rippling: Vec<(TrackId, Tick, bool)> = work
.all_tracks()
.filter(|tr| !tr.locked)
.filter_map(|tr| match origins.iter().find(|(t, _)| *t == tr.id) {
Some(o) => Some((tr.id, o.1, true)),
None => tr.sync_lock.then_some((tr.id, main_from, false)),
})
.collect();
// A split edit: the linked partner of a later clip can start before the shift does and reach
// past it (its sound leads the cut). It follows the clip it is linked to, or the two would
// drift apart.
// (the trimmed clips' own partners are trimmed, or left alone, by the caller: they never follow)
let own_links: Vec<u64> = clips.iter().filter_map(|c| work.find_item(*c).and_then(|(_, i)| i.link)).collect();
let mut shifting_links: Vec<u64> = Vec::new();
for (tid, from, _) in &rippling {
let Some(tr) = work.track(*tid) else { continue };
let later = tr.items.iter().filter(|i| !clips.contains(&i.id) && i.start >= *from);
shifting_links.extend(later.filter_map(|i| i.link).filter(|l| !own_links.contains(l)));
}
// every linked clip: its link, and where it is and ends
let linked: Vec<(u64, ClipId, Tick)> = work.all_tracks().flat_map(|tr| tr.items.iter().filter_map(|i| i.link.map(|l| (l, i.id, i.end())))).collect();
for (tid, from, member) in rippling {
let Some(tr) = work.track_mut(tid) else { continue };
let follows = |i: &TrackItem| i.start < from && i.end() > from && i.link.is_some_and(|l| shifting_links.contains(&l));
// An L cut: the sound of a clip that ends at the cut runs on into the shortened head. That clip
// stays put, so its sound stays with it and the L cut still ends where it did. (Unlinked material
// across the cut, a music bed, still refuses, and so does a linked clip whose partner reaches past
// the cut.)
let stays = |i: &TrackItem| {
let cut = from - Tick(1);
let mut partners = linked.iter().filter(|p| Some(p.0) == i.link && p.1 != i.id).peekable();
edge == Edge::In && i.start < cut && i.link.is_some_and(|l| !own_links.contains(&l)) && partners.peek().is_some() && partners.all(|p| p.2 <= cut)
};
if !member && shift < Tick::ZERO {
// a shorter head closes the stretch just after the cut, a shorter tail the one before it
let closing = if edge == Edge::In { TimeRange::new(from - Tick(1), -shift) } else { TimeRange::new(from + shift, -shift) };
let own = |i: &TrackItem| i.link.is_some_and(|l| own_links.contains(&l));
if tr.items.iter().any(|i| i.range().overlaps(&closing) && !follows(i) && !own(i) && !stays(i)) {
return Err(EditError::SyncLockConflict(track_label(seq, tid)));
}
}
let followers: Vec<ClipId> = tr.items.iter().filter(|i| !clips.contains(&i.id) && follows(i)).map(|i| i.id).collect();
for i in &mut tr.items {
if !clips.contains(&i.id) && i.start >= from {
if !clips.contains(&i.id) && (i.start >= from || followers.contains(&i.id)) {
i.start += shift;
}
}
tr.sort();
// a transition into a follower from a clip that stayed behind no longer sits on a cut
let ends: Vec<(ClipId, Tick)> = tr.items.iter().map(|i| (i.id, i.end())).collect();
let starts: Vec<(ClipId, Tick)> = tr.items.iter().map(|i| (i.id, i.start)).collect();
tr.transitions.retain(|t| match (t.from, t.to) {
(Some(a), Some(b)) if followers.contains(&b) => ends.iter().find(|e| e.0 == a).map(|e| e.1) == starts.iter().find(|x| x.0 == b).map(|x| x.1),
_ => true,
});
// a follower with no room (a clip in its way, or the sequence start) blocks the edit
if tr.items.first().is_some_and(|i| i.start < Tick::ZERO) || tr.items.windows(2).any(|w| w[0].end() > w[1].start) {
return Err(EditError::SyncLockConflict(track_label(seq, tr.id)));
}
}
transitions_follow_cuts(seq, &mut work);
work.check().map_err(EditError::Other)?;
@@ -751,41 +858,71 @@ pub fn rate_stretch(seq: &mut Sequence, clip: ClipId, edge: Edge, delta: Tick, c
/// Set speed/duration (Clip ▸ Speed/Duration…). `ripple` shifts following material.
pub fn set_speed(seq: &mut Sequence, clip: ClipId, speed: f64, reverse: bool, ripple: bool, ctx: &mut EditCtx) -> Result<()> {
if speed <= 0.0 {
set_speed_group(seq, &[clip], speed, reverse, ripple, ctx)
}
/// Speed/Duration on a clip and its linked partners on other tracks (its sound) as a single edit.
/// `clips` holds at most one clip per track. Every member takes the speed, and with `ripple` the
/// later material moves once, on the members' tracks and on sync-locked tracks. Rippling each
/// member on its own would move the later clips once per member. When the members' lengths change
/// by different amounts (a split edit), the later material moves by the largest change, so no
/// member is overlapped and the tracks stay in sync. Nothing changes when the edit fails.
pub fn set_speed_group(seq: &mut Sequence, clips: &[ClipId], speed: f64, reverse: bool, ripple: bool, ctx: &mut EditCtx) -> Result<()> {
if !(speed.is_finite() && speed > 0.0) {
return Err(EditError::Other("speed must be positive".into()));
}
let (tid, it) = seq.find_item(clip).ok_or(EditError::NoItem(clip))?;
let src_len = it.source_out() - it.source_in;
let mut new_dur = Tick((src_len.0 as f64 / speed).round() as i64).max(ctx.min_duration);
let old_end = it.end();
let tr = seq.track(tid).ok_or(EditError::NoTrack(tid))?;
let (_, next_start) = neighbours(tr, clip);
if !ripple {
new_dur = new_dur.min(next_start - it.start);
}
let delta = new_dur - it.duration;
let before = ripple.then(|| seq.clone());
{
let (_, it) = seq.find_item_mut(clip).ok_or(EditError::NoItem(clip))?;
let mut work = seq.clone();
// per member track: where its later material starts, and by how much the member's length changed
let mut origins: Vec<(TrackId, Tick, Tick)> = Vec::new();
for clip in clips {
let (tid, it) = work.find_item(*clip).ok_or(EditError::NoItem(*clip))?;
if origins.iter().any(|o| o.0 == tid) {
return Err(EditError::Other("clips on one track change speed one at a time".into()));
}
let src_len = it.source_out() - it.source_in;
// a tiny speed saturates the cast: keep the clip's end representable
let room = Tick(Tick::MAX.0.saturating_sub(it.start.0.max(0)));
let mut new_dur = Tick((src_len.0 as f64 / speed).round() as i64).min(room).max(ctx.min_duration);
let old_end = it.end();
let tr = work.track(tid).ok_or(EditError::NoTrack(tid))?;
let (_, next_start) = neighbours(tr, *clip);
if !ripple {
new_dur = new_dur.min(next_start - it.start);
}
let delta = new_dur - it.duration;
let (_, it) = work.find_item_mut(*clip).ok_or(EditError::NoItem(*clip))?;
it.speed = speed;
it.reverse = reverse;
it.duration = new_dur;
origins.push((tid, old_end, delta));
}
if ripple && delta != Tick::ZERO {
for tr in seq.all_tracks_mut() {
if !tr.locked && (tr.id == tid || tr.sync_lock) {
for i in &mut tr.items {
if i.id != clip && i.start >= old_end {
i.start += delta;
}
}
tr.sort();
let main_from = origins.first().map(|o| o.1).unwrap_or_default();
let shift = origins.iter().map(|o| o.2).max().unwrap_or_default();
if ripple && shift != Tick::ZERO {
for tr in work.all_tracks_mut() {
if tr.locked {
continue;
}
let member = origins.iter().find(|o| o.0 == tr.id).map(|o| o.1);
if member.is_none() && !tr.sync_lock {
continue;
}
let from = member.unwrap_or(main_from);
for i in &mut tr.items {
if !clips.contains(&i.id) && i.start >= from {
// later clips must still end on the representable timeline
let start = i.start.0.checked_add(shift.0).filter(|s| s.checked_add(i.duration.0).is_some());
i.start = Tick(start.ok_or_else(|| EditError::Other("the speed change would move later clips past the end of the timeline".into()))?);
}
}
tr.sort();
}
}
if let Some(before) = before {
transitions_follow_cuts(&before, seq);
if ripple {
transitions_follow_cuts(seq, &mut work);
}
work.check().map_err(EditError::Other)?;
*seq = work;
Ok(())
}
+307 -1
View File
@@ -60,6 +60,7 @@ impl Fx {
hold_filters: false,
field_options: None,
source_channels: Vec::new(),
audio_stream: 0,
graphic: None,
}
}
@@ -144,10 +145,85 @@ fn ripple_delete_and_conflict() {
// audio under the gap blocks the ripple
let b = fx.put(v1, 20, 10, 0);
fx.put(a1, 22, 3, 0);
assert_eq!(ripple_delete_items(&mut fx.seq, &[b]), Err(EditError::SyncLockConflict));
assert_eq!(ripple_delete_items(&mut fx.seq, &[b]), Err(EditError::SyncLockConflict("A1".into())), "names the track in the way");
assert_eq!(fx.spans(v1), vec![(0, 10), (20, 10)], "unchanged on failure");
}
#[test]
fn ripple_delete_closes_the_time_free_on_every_track() {
// four clips on V1 with their sounds on A1; the second sound starts 2 early and ends 2 late
let build = || {
let mut fx = Fx::new();
let (v1, a1) = (fx.v(0), fx.a(0));
let v: Vec<ClipId> = [(0, 10), (10, 10), (20, 10), (30, 10)].iter().map(|(s, d)| fx.put(v1, *s, *d, 0)).collect();
let a: Vec<ClipId> = [(0, 8), (8, 14), (22, 8), (30, 10)].iter().map(|(s, d)| fx.put(a1, *s, *d, 0)).collect();
(fx, v, a)
};
let span = |start, len| TimeRange::new(f(start), f(len));
// one clip with a split edit on both sides: the picture's stretch closes, everything later stays in sync
let (mut fx, v, a) = build();
assert_eq!(ripple_delete_items(&mut fx.seq, &[v[1], a[1]]), Ok(vec![span(10, 10)]));
assert_eq!(fx.spans(fx.v(0)), vec![(0, 10), (10, 10), (20, 10)]);
assert_eq!(fx.spans(fx.a(0)), vec![(0, 8), (12, 8), (20, 10)]);
fx.seq.check().unwrap();
// two neighbours with a split edit between them close as one stretch
let (mut fx, v, a) = build();
assert_eq!(ripple_delete_items(&mut fx.seq, &[v[1], a[1], v[2], a[2]]), Ok(vec![span(10, 20)]));
assert_eq!(fx.spans(fx.v(0)), vec![(0, 10), (10, 10)]);
assert_eq!(fx.spans(fx.a(0)), vec![(0, 8), (10, 10)]);
// clips apart in time each close their own gap
let (mut fx, v, a) = build();
assert_eq!(ripple_delete_items(&mut fx.seq, &[v[0], a[0], v[3], a[3]]), Ok(vec![span(0, 8), span(30, 10)]));
fx.seq.check().unwrap();
// straight cuts with the sounds on alternating tracks: V1 [0,10) [10,20) [20,30), A1 [0,10) [20,30), A2 [10,20)
let mut fx = Fx::new();
let (v1, a1, a2) = (fx.v(0), fx.a(0), fx.a(1));
let v: Vec<ClipId> = [0, 10, 20].iter().map(|s| fx.put(v1, *s, 10, 0)).collect();
let (s0, s1) = (fx.put(a1, 0, 10, 0), fx.put(a2, 10, 10, 0));
fx.put(a1, 20, 10, 0);
assert_eq!(ripple_delete_items(&mut fx.seq, &[v[0], s0, v[1], s1]), Ok(vec![span(0, 20)]));
assert_eq!((fx.spans(v1), fx.spans(a1), fx.spans(a2)), (vec![(0, 10)], vec![(0, 10)], vec![]));
// a sound shorter than its picture with nothing round it: the whole picture closes
let mut fx = Fx::new();
let (v1, a1) = (fx.v(0), fx.a(0));
let (p, s) = (fx.put(v1, 10, 10, 0), fx.put(a1, 12, 6, 0));
fx.put(v1, 20, 10, 0);
fx.put(a1, 20, 10, 0);
assert_eq!(ripple_delete_items(&mut fx.seq, &[p, s]), Ok(vec![span(10, 10)]));
// with the neighbours' sound right up against it, only the sound's stretch can close
let mut fx = Fx::new();
let (v1, a1) = (fx.v(0), fx.a(0));
fx.put(a1, 0, 12, 0);
let (p, s) = (fx.put(v1, 10, 10, 0), fx.put(a1, 12, 6, 0));
fx.put(v1, 20, 10, 0);
fx.put(a1, 18, 12, 0);
assert_eq!(ripple_delete_items(&mut fx.seq, &[p, s]), Ok(vec![span(12, 6)]));
assert_eq!((fx.spans(v1), fx.spans(a1)), (vec![(14, 10)], vec![(0, 12), (12, 12)]));
// a linked sound on a locked track stays, and does not shrink the gap that closes
let mut fx = Fx::new();
let (v1, a1) = (fx.v(0), fx.a(0));
let (p, s) = (fx.put(v1, 10, 10, 0), fx.put(a1, 12, 6, 0));
fx.put(v1, 20, 10, 0);
fx.seq.track_mut(a1).unwrap().locked = true;
fx.seq.track_mut(a1).unwrap().sync_lock = false;
assert_eq!(ripple_delete_items(&mut fx.seq, &[p, s]), Ok(vec![span(10, 10)]));
assert_eq!((fx.spans(v1), fx.spans(a1)), (vec![(10, 10)], vec![(12, 6)]));
// nothing can close: what stays on each track covers the other track's deleted time. Refused, unchanged.
let mut fx = Fx::new();
let (v1, a1) = (fx.v(0), fx.a(0));
let x = fx.put(v1, 10, 10, 0);
fx.put(v1, 30, 10, 0);
fx.put(a1, 10, 10, 0);
let y = fx.put(a1, 30, 10, 0);
let before = fx.seq.clone();
assert!(matches!(ripple_delete_items(&mut fx.seq, &[x, y]), Err(EditError::Other(_))));
assert_eq!(fx.seq, before);
}
#[test]
fn close_gap_works() {
let mut fx = Fx::new();
@@ -192,6 +268,79 @@ fn ripple_trim_shifts_following() {
assert_eq!(fx.spans(v1), vec![(0, 12), (12, 10)]);
}
#[test]
fn ripple_trim_group_takes_a_split_edit_along() {
// V1: a [0,10) then b [10,20). b's linked sound leads it by 4 on sync-locked A1: [6,20).
let mut fx = Fx::new();
let (v1, a1) = (fx.v(0), fx.a(0));
let a = fx.put(v1, 0, 10, 0);
let b = fx.put(v1, 10, 10, 0);
let sound = fx.put(a1, 6, 14, 0);
for c in [b, sound] {
fx.seq.find_item_mut(c).unwrap().1.link = Some(7);
}
let mut n = fx.next;
ripple_trim_group(&mut fx.seq, &[a], Edge::Out, f(3), &mut Fx::ctx(&mut n)).unwrap();
assert_eq!((fx.spans(v1), fx.spans(a1)), (vec![(0, 13), (13, 10)], vec![(9, 14)]), "longer: the early sound follows its picture");
ripple_trim_group(&mut fx.seq, &[a], Edge::Out, -f(5), &mut Fx::ctx(&mut n)).unwrap();
assert_eq!((fx.spans(v1), fx.spans(a1)), (vec![(0, 8), (8, 10)], vec![(4, 14)]), "shorter: it follows too");
// the sound would start before the sequence does: refused, nothing changes
assert!(matches!(ripple_trim_group(&mut fx.seq, &[a], Edge::Out, -f(5), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict(_))));
// another clip right before the sound leaves it no room either
fx.put(a1, 0, 4, 0);
assert!(matches!(ripple_trim_group(&mut fx.seq, &[a], Edge::Out, -f(2), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict(_))));
assert_eq!((fx.spans(v1), fx.spans(a1)), (vec![(0, 8), (8, 10)], vec![(0, 4), (4, 14)]), "unchanged on failure");
// a linked clip that ends before the cut is not a split edit and stays where it is
let early = fx.put(fx.a(1), 0, 3, 0);
fx.seq.find_item_mut(early).unwrap().1.link = Some(7);
ripple_trim_group(&mut fx.seq, &[a], Edge::Out, f(2), &mut Fx::ctx(&mut n)).unwrap();
assert_eq!(fx.spans(fx.a(1)), vec![(0, 3)]);
fx.seq.check().unwrap();
// In edge: V1 b [10,20), c [20,30); c's sound starts 12 early on A1: [8,30). b loses 3 at its head.
let mut fx = Fx::new();
let (v1, v2, a1, a2) = (fx.v(0), fx.v(1), fx.a(0), fx.a(1));
fx.put(v1, 0, 10, 0);
let b = fx.put(v1, 10, 10, 0);
let c = fx.put(v1, 20, 10, 0);
let c_sound = fx.put(a1, 8, 22, 0);
// two more clips linked to b that the caller did not put in the group
let b_sound = fx.put(a2, 10, 10, 0);
let b_top = fx.put(v2, 12, 8, 0);
for (x, link) in [(c, 7), (c_sound, 7), (b, 8), (b_sound, 8), (b_top, 8)] {
fx.seq.find_item_mut(x).unwrap().1.link = Some(link);
}
let mut n = fx.next;
ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(3), &mut Fx::ctx(&mut n)).unwrap();
assert_eq!((fx.spans(v1), fx.spans(a1)), (vec![(0, 10), (10, 7), (17, 10)], vec![(5, 22)]), "c and its early sound move up together");
assert_eq!((fx.spans(v2), fx.spans(a2)), (vec![(9, 8)], vec![(10, 10)]), "the trimmed clip's own partners never follow");
// a crossfade into the early sound from a clip that stays behind no longer sits on a cut: it goes
let mut fx = Fx::new();
let (v1, a1) = (fx.v(0), fx.a(0));
let a = fx.put(v1, 0, 10, 0);
let b = fx.put(v1, 10, 10, 0);
let stays = fx.put(a1, 0, 6, 0);
let sound = fx.put(a1, 6, 14, 0);
for x in [b, sound] {
fx.seq.find_item_mut(x).unwrap().1.link = Some(7);
}
fx.seq.track_mut(a1).unwrap().transitions = vec![Transition {
id: TransitionId(1),
effect: filmcraft_project::find_effect("cross_dissolve").unwrap().instance(),
start: f(4),
duration: f(4),
from: Some(stays),
to: Some(sound),
align: Default::default(),
reverse: false,
}];
let mut n = fx.next;
ripple_trim_group(&mut fx.seq, &[a], Edge::Out, f(3), &mut Fx::ctx(&mut n)).unwrap();
assert_eq!(fx.spans(a1), vec![(0, 6), (9, 14)]);
assert!(fx.seq.track(a1).unwrap().transitions.is_empty());
}
#[test]
fn roll_slip_slide() {
let mut fx = Fx::new();
@@ -269,6 +418,44 @@ fn rate_stretch_and_speed() {
assert_eq!(fx.spans(v1), vec![(0, 5)]);
}
#[test]
fn speed_on_a_linked_group_ripples_once() {
// V1: a [0,10), b [10,20). A1: a's sound [0,10), b's sound [10,20).
let build = |sound_len: i64| {
let mut fx = Fx::new();
let (v1, a1) = (fx.v(0), fx.a(0));
let a = fx.put(v1, 0, 10, 0);
fx.put(v1, 10, 10, 0);
let sound = fx.put(a1, 0, sound_len, 0);
fx.put(a1, sound_len, 10, 0);
(fx, a, sound)
};
for (speed, picture, later) in [(0.5, (0, 20), (20, 10)), (2.0, (0, 5), (5, 10))] {
let (mut fx, a, sound) = build(10);
let mut n = fx.next;
set_speed_group(&mut fx.seq, &[a, sound], speed, false, true, &mut Fx::ctx(&mut n)).unwrap();
assert_eq!((fx.spans(fx.v(0)), fx.spans(fx.a(0))), (vec![picture, later], vec![picture, later]), "{speed}");
}
// a split edit: the sound is 4 longer. Later clips move by the larger change, on both tracks.
let (mut fx, a, sound) = build(14);
let mut n = fx.next;
set_speed_group(&mut fx.seq, &[a, sound], 0.5, false, true, &mut Fx::ctx(&mut n)).unwrap();
assert_eq!((fx.spans(fx.v(0)), fx.spans(fx.a(0))), (vec![(0, 20), (24, 10)], vec![(0, 28), (28, 10)]));
let (mut fx, a, sound) = build(14);
let mut n = fx.next;
set_speed_group(&mut fx.seq, &[a, sound], 2.0, false, true, &mut Fx::ctx(&mut n)).unwrap();
assert_eq!((fx.spans(fx.v(0)), fx.spans(fx.a(0))), (vec![(0, 5), (5, 10)], vec![(0, 7), (9, 10)]));
fx.seq.check().unwrap();
// two clips of one track are not one group; a missing clip changes nothing
let (mut fx, a, _) = build(10);
let b = fx.seq.video_tracks[0].items[1].id;
let before = fx.seq.clone();
let mut n = fx.next;
assert!(set_speed_group(&mut fx.seq, &[a, b], 2.0, false, true, &mut Fx::ctx(&mut n)).is_err());
assert!(set_speed_group(&mut fx.seq, &[a, ClipId(1)], 2.0, false, true, &mut Fx::ctx(&mut n)).is_err());
assert_eq!(fx.seq, before);
}
#[test]
fn move_with_insert_and_overwrite() {
let mut fx = Fx::new();
@@ -367,3 +554,122 @@ fn merge(v: Vec<(i64, i64)>) -> Vec<(i64, i64)> {
fn unused_kind_import() {
let _ = TrackKind::Video;
}
#[test]
fn hostile_speeds_are_refused_or_clamped_never_overflow() {
// a tiny speed used to saturate the new duration to i64::MAX, then the ripple shift overflowed
for speed in [1e-7, 1e-300, f64::MIN_POSITIVE, f64::NAN, f64::INFINITY, -1.0, 0.0] {
let mut fx = Fx::new();
let v1 = fx.v(0);
let a = fx.put(v1, 0, 10, 10);
let _b = fx.put(v1, 10, 10, 10);
let before = fx.seq.clone();
let mut n = fx.next;
let r = set_speed_group(&mut fx.seq, &[a], speed, false, true, &mut Fx::ctx(&mut n));
match r {
Ok(()) => fx.seq.check().unwrap(),
Err(_) => assert_eq!(format!("{:?}", fx.seq), format!("{before:?}"), "{speed}: a refused edit changes nothing"),
}
}
}
#[test]
fn shorter_head_into_an_l_cut_keeps_the_earlier_sound_where_it_is() {
// V1: a [0,10), b [10,20), c [20,30). a's sound on sync-locked A1 runs 3 into b: [0,13) (an L
// cut). b loses 3 at its head: a never moves, so its sound stays and the L cut still ends at 13.
let mut fx = Fx::new();
let (v1, a1) = (fx.v(0), fx.a(0));
let a = fx.put(v1, 0, 10, 0);
let b = fx.put(v1, 10, 10, 0);
fx.put(v1, 20, 10, 0);
let a_sound = fx.put(a1, 0, 13, 0);
for x in [a, a_sound] {
fx.seq.find_item_mut(x).unwrap().1.link = Some(7);
}
let mut n = fx.next;
ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(3), &mut Fx::ctx(&mut n)).unwrap();
assert_eq!((fx.spans(v1), fx.spans(a1)), (vec![(0, 10), (10, 7), (17, 10)], vec![(0, 13)]));
fx.seq.check().unwrap();
// later material on A1 moves up with the pictures; landing on the L cut's sound refuses
let later = fx.put(a1, 15, 5, 0);
assert!(matches!(ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(3), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict(_))));
assert_eq!(fx.spans(a1), vec![(0, 13), (15, 5)], "unchanged on failure");
fx.seq.find_item_mut(later).unwrap().1.start = f(20);
ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(2), &mut Fx::ctx(&mut n)).unwrap();
assert_eq!((fx.spans(v1), fx.spans(a1)), (vec![(0, 10), (10, 5), (15, 10)], vec![(0, 13), (18, 5)]));
// unlinked material across the cut (a music bed) still refuses
let mut fx = Fx::new();
let (v1, a1) = (fx.v(0), fx.a(0));
fx.put(v1, 0, 10, 0);
let b = fx.put(v1, 10, 10, 0);
fx.put(a1, 0, 13, 0);
let mut n = fx.next;
assert!(matches!(ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(3), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict(_))));
// so does a link with no partner left
fx.seq.track_mut(a1).unwrap().items[0].link = Some(9);
assert!(matches!(ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(3), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict(_))));
// a linked cutaway across the cut whose own partner reaches past it too is no L cut: it refuses
let mut fx = Fx::new();
let (v1, v2, a1) = (fx.v(0), fx.v(1), fx.a(0));
fx.put(v1, 0, 10, 0);
let b = fx.put(v1, 10, 10, 0);
fx.put(v1, 20, 10, 0);
let x = fx.put(v2, 5, 20, 0);
let xs = fx.put(a1, 5, 20, 0);
for c in [x, xs] {
fx.seq.find_item_mut(c).unwrap().1.link = Some(7);
}
let mut n = fx.next;
assert!(matches!(ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(3), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict(_))));
}
#[test]
fn shorter_head_closes_the_stretch_after_the_cut_on_sync_locked_tracks() {
// V1: a [0,10), b [10,20). a's sound on sync-locked A1 runs up to the cut: [0,10). b loses 3 at
// its head; the stretch that closes is [10,13), where A1 is empty, so nothing refuses.
for group in [false, true] {
let mut fx = Fx::new();
let (v1, a1) = (fx.v(0), fx.a(0));
fx.put(v1, 0, 10, 0);
let b = fx.put(v1, 10, 10, 0);
fx.put(v1, 20, 10, 0);
fx.put(a1, 0, 10, 0);
let later = fx.put(a1, 20, 5, 0);
let mut n = fx.next;
if group {
ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(3), &mut Fx::ctx(&mut n)).unwrap();
} else {
trim(&mut fx.seq, b, Edge::In, TrimMode::Ripple, f(3), &mut Fx::ctx(&mut n)).unwrap();
}
assert_eq!((fx.spans(v1), fx.spans(a1)), (vec![(0, 10), (10, 7), (17, 10)], vec![(0, 10), (17, 5)]), "group {group}");
// material inside the stretch that closes still refuses
fx.seq.find_item_mut(later).unwrap().1.start = f(11);
let r = if group {
ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(2), &mut Fx::ctx(&mut n))
} else {
trim(&mut fx.seq, b, Edge::In, TrimMode::Ripple, f(2), &mut Fx::ctx(&mut n))
};
assert!(matches!(r, Err(EditError::SyncLockConflict(_))), "group {group}");
}
// a clip on a sync-locked track that starts at the cut stays put while the pictures after it move
// up: only the stretch after the cut sees it (the old stretch, before the cut, let it through)
for group in [false, true] {
let mut fx = Fx::new();
let (v1, a1) = (fx.v(0), fx.a(0));
fx.put(v1, 0, 10, 0);
let b = fx.put(v1, 10, 10, 0);
fx.put(v1, 20, 10, 0);
fx.put(a1, 10, 5, 0);
let mut n = fx.next;
let r = if group {
ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(3), &mut Fx::ctx(&mut n))
} else {
trim(&mut fx.seq, b, Edge::In, TrimMode::Ripple, f(3), &mut Fx::ctx(&mut n))
};
assert!(matches!(r, Err(EditError::SyncLockConflict(_))), "group {group}");
assert_eq!(fx.spans(v1), vec![(0, 10), (10, 10), (20, 10)], "unchanged on failure");
}
}
+1
View File
@@ -180,6 +180,7 @@ mod tests {
hold_filters: false,
field_options: None,
source_channels: Vec::new(),
audio_stream: 0,
graphic: None,
}
}
+1
View File
@@ -57,6 +57,7 @@ fn clip(id: u64, start: f64, dur: f64, src_in: f64) -> TrackItem {
hold_filters: false,
field_options: None,
source_channels: Vec::new(),
audio_stream: 0,
graphic: None,
}
}
+32 -13
View File
@@ -19,8 +19,8 @@ use std::sync::{Arc, Mutex};
use serde_json::{Value, json};
use filmcraft_interchange::essence::{AudioEssence, AudioNeed, EssenceData, EssenceKey, MediaOptions, MixdownVideo, NeedOptions, audio_needs};
use filmcraft_project::{ItemId, Project, SequenceSettings, TrackKind};
use filmcraft_interchange::essence::{AudioEssence, AudioNeed, EssenceData, EssenceKey, MediaOptions, MixdownVideo, NeedOptions, NestNeeds, audio_needs};
use filmcraft_project::{ClipId, ItemId, Project, Sequence, SequenceSettings, Track, TrackItem, TrackKind};
use filmcraft_time::{Tick, TimeRange};
use crate::commands::{bad, bool_p, str_p, u64_p};
@@ -50,6 +50,10 @@ pub struct AudioPlan {
pub handles: Tick,
pub render_effects: bool,
pub breakout: bool,
/// Nested sequences on the audio tracks. OMF: their sound is rendered into the export, as
/// Premiere Pro does. AAF: they are written as compositions, so the media of the clips inside
/// them is prepared like the rest.
pub nests: NestNeeds,
}
fn plan_from(s: &Session, p: &Value, seq: ItemId, cmd: &str, omf: bool) -> Result<AudioPlan> {
@@ -85,6 +89,7 @@ fn plan_from(s: &Session, p: &Value, seq: ItemId, cmd: &str, omf: bool) -> Resul
handles: q.settings.frame_rate.tick_of(frames),
render_effects: bool_p(p, "renderAudioEffects").unwrap_or(false),
breakout: bool_p(p, "breakoutToMono").unwrap_or(false),
nests: if omf { NestNeeds::Render } else { NestNeeds::Inside },
})
}
@@ -154,12 +159,8 @@ fn decode(s: &Session, need: &AudioNeed, sample_rate: u32) -> Result<Vec<Vec<f32
/// pipeline's audio mixer.
fn render_clip(s: &Session, seq: ItemId, need: &AudioNeed, sample_rate: u32, bits: u16) -> Result<Vec<Vec<f32>>> {
let EssenceKey::Clip(clip_id) = need.key else { return decode(s, need, sample_rate) };
let q = s.project.sequence(seq).ok_or(EngineError::NoSequence)?;
let (track, c) = q
.tracks(TrackKind::Audio)
.iter()
.find_map(|t| t.items.iter().find(|c| c.id == clip_id).map(|c| (t, c.clone())))
.ok_or_else(|| EngineError::Other("clip not found".into()))?;
let (q, track, c) = audio_clip(&s.project, seq, clip_id).ok_or_else(|| EngineError::Other("clip not found".into()))?;
let c = c.clone();
let mut p: Project = (*s.project).clone();
let settings = SequenceSettings { sample_rate, ..q.settings.clone() };
let rs = p.new_sequence("AAF audio render", settings, 0, 1, None);
@@ -204,6 +205,19 @@ fn render_clip(s: &Session, seq: ItemId, need: &AudioNeed, sample_rate: u32, bit
Ok(dequantize(&pcm, ch as usize, b))
}
/// The audio clip `clip` with its track and sequence: in `seq`, or in a sequence nested in it
/// (clip ids are unique in a project).
fn audio_clip(p: &Project, seq: ItemId, clip: ClipId) -> Option<(&Sequence, &Track, &TrackItem)> {
let first = p.sequence(seq).into_iter();
let others = p.sequences().filter(|i| i.id != seq).filter_map(|i| p.sequence(i.id));
for q in first.chain(others) {
if let Some((t, c)) = q.tracks(TrackKind::Audio).iter().find_map(|t| t.items.iter().find(|c| c.id == clip).map(|c| (t, c))) {
return Some((q, t, c));
}
}
None
}
fn safe_name(s: &str) -> String {
let n: String = s.chars().map(|c| if c.is_alphanumeric() || " -_.".contains(c) { c } else { '_' }).collect();
let n = n.trim().trim_matches('.').to_string();
@@ -221,19 +235,24 @@ pub fn prepare_audio(s: &Session, seq: ItemId, plan: &AudioPlan, media_dir: &str
if plan.mode == AudioMode::Linked && !plan.trim && !plan.render_effects {
return Ok((Vec::new(), Vec::new()));
}
let needs =
audio_needs(&s.project, seq, &NeedOptions { handles: plan.handles, per_clip: plan.render_effects, whole_media: !plan.trim && !plan.render_effects });
let needs = audio_needs(
&s.project,
seq,
&NeedOptions { handles: plan.handles, per_clip: plan.render_effects, whole_media: !plan.trim && !plan.render_effects, nests: plan.nests },
);
let mut essence = Vec::new();
let mut files = Vec::new();
let mut used = std::collections::HashSet::new();
for need in &needs {
let planar = if plan.render_effects { render_clip(s, seq, need, plan.sample_rate, plan.bits)? } else { decode(s, need, plan.sample_rate)? };
// a nested sequence has no file to read: its sound is mixed, with what is on the clip
let nest = s.project.sequence(need.item).is_some();
let planar = if plan.render_effects || nest { render_clip(s, seq, need, plan.sample_rate, plan.bits)? } else { decode(s, need, plan.sample_rate)? };
let frames = planar.first().map_or(0, Vec::len) as u64;
let groups: Vec<(Option<u32>, Vec<&Vec<f32>>)> =
if plan.breakout { planar.iter().enumerate().map(|(c, x)| (Some(c as u32), vec![x])).collect() } else { vec![(None, planar.iter().collect())] };
let base = match need.key {
EssenceKey::Clip(c) => {
let name = s.project.sequence(seq).and_then(|q| q.find_item(c)).map(|(_, ti)| ti.name.clone()).unwrap_or_default();
let name = audio_clip(&s.project, seq, c).map(|(_, _, ti)| ti.name.clone()).unwrap_or_default();
format!("{} {}", safe_name(&name), c.0)
}
EssenceKey::Media(_) => safe_name(
@@ -302,7 +321,7 @@ pub fn prepare_audio(s: &Session, seq: ItemId, plan: &AudioPlan, media_dir: &str
bits: plan.bits,
channels: n,
data,
effects_rendered: plan.render_effects,
effects_rendered: plan.render_effects || nest,
});
}
}
+92 -3
View File
@@ -69,7 +69,7 @@ fn aaf_embedded_trimmed_audio_round_trips_sample_exact() {
assert_eq!(a, b, "{kind:?}");
}
// video links to the original movies, audio to WAV files extracted next to the AAF
assert_eq!(path_of(&s, q.video_tracks[0].items[0].item), path_of(&s, items[0]));
assert_eq!(std::path::Path::new(&path_of(&s, q.video_tracks[0].items[0].item)), std::path::Path::new(&path_of(&s, items[0])));
for (c, oc) in q.audio_tracks[0].items.iter().zip(&orig.audio_tracks[0].items) {
let p = path_of(&s, c.item);
assert!(p.ends_with(".wav") && std::path::Path::new(&p).exists(), "{p}");
@@ -138,7 +138,7 @@ fn aaf_separate_aiff_breakout_and_linked() {
let n = imported_seq(&r);
let q = s.project.sequence(n).unwrap();
let a = &q.audio_tracks[0].items[0];
assert_eq!(path_of(&s, a.item), path_of(&s, items[0]));
assert_eq!(std::path::Path::new(&path_of(&s, a.item)), std::path::Path::new(&path_of(&s, items[0])));
assert_eq!(a.source_in, FrameRate::FPS_24.tick_of(12));
assert!(!s.project.item(a.item).unwrap().as_media().unwrap().offline);
let _ = std::fs::remove_dir_all(dir);
@@ -159,7 +159,7 @@ fn aaf_video_mixdown() {
assert_eq!(q.video_tracks.iter().map(|t| t.items.len()).sum::<usize>(), 1);
let v = &q.video_tracks[0].items[0];
assert_eq!(v.duration, FrameRate::FPS_24.tick_of(48));
assert_eq!(path_of(&s, v.item), mov);
assert_eq!(std::path::Path::new(&path_of(&s, v.item)), std::path::Path::new(mov));
let m = s.project.item(v.item).unwrap().as_media().unwrap();
assert!(!m.offline);
assert_eq!(m.info.video.as_ref().map(|v| (v.width, v.height)), Some((64, 36)));
@@ -208,6 +208,95 @@ fn omf_renders_clip_effects_into_embedded_audio() {
let _ = std::fs::remove_dir_all(dir);
}
/// `setup`, with both clips nested into "Inner" (one linked picture + sound nest on V1 / A1) and
/// −6.0206 dB (half amplitude) on the nest's sound clip.
fn setup_nested(name: &str) -> (Session, Vec<ItemId>, ItemId, ItemId, std::path::PathBuf) {
let (mut s, items, seq, dir) = setup(name);
s.execute("sequence.open", json!({"item": seq.0})).unwrap();
let clips: Vec<u64> = s.project.sequence(seq).unwrap().all_tracks().flat_map(|t| t.items.iter().map(|c| c.id.0)).collect();
s.execute("timeline.select", json!({"clips": clips})).unwrap();
let inner = ItemId(s.execute("clip.nest", json!({"name": "Inner"})).unwrap()["sequence"].as_u64().unwrap());
let mut p = (*s.project).clone();
let q = p.sequence_mut(seq).unwrap();
assert_eq!((q.video_tracks[0].items.len(), q.audio_tracks[0].items.len()), (1, 1), "one linked nest");
q.audio_tracks[0].items[0].effect_mut("volume").unwrap().params.insert("level".into(), Param::new(ParamValue::Float(-6.020_599_913)));
s.project = Arc::new(p);
(s, items, seq, inner, dir)
}
/// Premiere Pro renders the sound of a nested sequence into an OMF as one clip named after the
/// sequence (seen in an OMF it exported). So do we: the nest is mixed, with what is on its clip.
#[test]
fn omf_renders_the_sound_of_a_nested_sequence() {
let (mut s, items, seq, inner, dir) = setup_nested("omf-nest");
let nest = s.project.sequence(seq).unwrap().audio_tracks[0].items[0].clone();
let path = dir.join("nest.omf").to_string_lossy().into_owned();
let r = s.execute("file.exportOmf", json!({"path": path, "handles": 0, "sequence": seq.0})).unwrap();
assert!(!r["report"].to_string().contains("nested"), "{r}");
let r = s.execute("file.importAaf", json!({"path": path})).unwrap();
let q = s.project.sequence(imported_seq(&r)).unwrap().clone();
let a = &q.audio_tracks[0].items;
assert_eq!(a.len(), 1, "the nest is one clip");
assert_eq!((a[0].name.as_str(), a[0].start, a[0].duration), ("Inner", nest.start, nest.duration));
assert!(s.project.sequence(a[0].item).is_none(), "media in the document, not a sequence");
// what it plays: the nested sequence's two clips one after the other, at half amplitude
let inside = s.project.sequence(inner).unwrap().audio_tracks[0].items.clone();
assert_eq!(inside.len(), 2);
let got = audio_of(&s, a[0].item, a[0].source_in.to_units_floor(48_000), 96_000);
for (k, oc) in inside.iter().enumerate() {
let want = audio_of(&s, oc.item, oc.source_in.to_units_floor(48_000), 48_000);
assert_eq!(oc.item, items[k]);
let peak = want[0].iter().fold(0f32, |m, x| m.max(x.abs()));
assert!(peak > 0.01, "the demo scene has audio");
// (away from the cut between the two clips)
for i in 480..47_520 {
let g = got[0][k * 48_000 + i];
assert!((g - want[0][i] * 0.5).abs() < 2e-3, "clip {k} sample {i}: {g} vs {}", want[0][i] * 0.5);
}
}
let _ = std::fs::remove_dir_all(dir);
}
/// An AAF holds a nested sequence as a composition of its own (as Premiere Pro's does): it comes
/// back as a sequence that the nest clips use, with its own clips, and with embedded audio the
/// media inside it is in the document too.
#[test]
fn aaf_keeps_a_nested_sequence_as_a_sequence() {
let (mut s, items, seq, inner, dir) = setup_nested("aaf-nest");
let path = dir.join("nest.aaf").to_string_lossy().into_owned();
let r = s.execute("file.exportAaf", json!({"path": path, "audio": "embedded", "trimAudio": true, "handles": 6, "sequence": seq.0})).unwrap();
assert!(!r["report"].to_string().contains("gap"), "{r}");
let r = s.execute("file.import", json!({"paths": [path]})).unwrap();
let q = s.project.sequence(imported_seq(&r)).unwrap().clone();
let (v, a) = (&q.video_tracks[0].items, &q.audio_tracks[0].items);
assert_eq!((v.len(), a.len()), (1, 1));
let nested = v[0].item;
assert_ne!(nested, inner, "a sequence of the imported document");
assert_eq!((a[0].item, s.project.item(nested).unwrap().name.as_str()), (nested, "Inner"));
assert!(v[0].link.is_some() && v[0].link == a[0].link, "picture and sound of the nest are linked");
let level = a[0].effect("volume").and_then(|e| e.param("level")).and_then(|p| p.value.as_f64()).unwrap();
assert!((level + 6.0206).abs() < 1e-2, "the level stays on the nest's clip: {level}");
// inside: the two clips, their picture linked to the movies, their sound in the document
let n = s.project.sequence(nested).unwrap().clone();
let orig = s.project.sequence(inner).unwrap().clone();
for kind in [TrackKind::Video, TrackKind::Audio] {
let got: Vec<_> = n.tracks(kind)[0].items.iter().map(|c| (c.start, c.duration)).collect();
let want: Vec<_> = orig.tracks(kind)[0].items.iter().map(|c| (c.start, c.duration)).collect();
assert_eq!(got, want, "{kind:?}");
}
assert_eq!(std::path::Path::new(&path_of(&s, n.video_tracks[0].items[0].item)), std::path::Path::new(&path_of(&s, items[0])));
for (c, oc) in n.audio_tracks[0].items.iter().zip(&orig.audio_tracks[0].items) {
let p = path_of(&s, c.item);
assert!(p.ends_with(".wav") && std::path::Path::new(&p).exists(), "{p}");
let got = audio_of(&s, c.item, c.source_in.to_units_floor(48_000), 4800);
let want = audio_of(&s, oc.item, oc.source_in.to_units_floor(48_000), 4800);
for i in 0..4800 {
assert!((got[0][i] - want[0][i]).abs() <= 1.0 / 32_768.0 + 1e-6, "sample {i}");
}
}
let _ = std::fs::remove_dir_all(dir);
}
#[test]
fn commands_are_registered_with_menus_and_validate_params() {
for (id, menu) in [("file.exportAaf", &["File", "Export"][..]), ("file.exportOmf", &["File", "Export"][..]), ("file.importAaf", &[][..])] {
+135
View File
@@ -0,0 +1,135 @@
//! Multi-clip audio placement: `audio_placement_specs` and dynamic audio track allocation.
use filmcraft_media::MediaSource;
use filmcraft_project::{AudioChannels, ItemId, TrackKind};
use filmcraft_time::{FrameRate, Tick, TimeRange};
use serde_json::json;
use crate::Session;
use crate::commands::{AudioPlacementSpec, audio_placement_specs, ensure_audio_tracks};
fn demo() -> Session {
let mut s = Session::default();
s.execute("file.openDemoProject", json!({})).unwrap();
s
}
fn active(s: &Session) -> ItemId {
s.state.active_sequence.expect("demo session has an active sequence")
}
#[test]
fn specs_always_place_one_clip_on_the_destination() {
assert_eq!(audio_placement_specs(&[], 1), vec![AudioPlacementSpec { track_offset: 0, audio_stream: 0, source_channels: None }]);
assert_eq!(audio_placement_specs(&[vec![0, 1]], 1).len(), 1);
}
#[test]
fn specs_put_further_clips_on_the_tracks_below() {
let specs = audio_placement_specs(&[vec![0], vec![1], vec![2, 3]], 1);
let got: Vec<(usize, Option<Vec<u16>>)> = specs.into_iter().map(|s| (s.track_offset, s.source_channels)).collect();
assert_eq!(got, vec![(0, None), (1, Some(vec![1])), (2, Some(vec![2, 3]))]);
}
#[test]
fn specs_are_capped_for_hostile_channel_maps() {
let clips = vec![vec![0u16]; 100_000];
assert!(audio_placement_specs(&clips, usize::MAX).len() <= crate::sequence_tools::MAX_TRACKS);
}
#[test]
fn specs_append_container_streams_after_channel_map_clips() {
let specs = audio_placement_specs(&[vec![1], vec![0]], 3);
assert_eq!(
specs.iter().map(|s| (s.track_offset, s.audio_stream, s.source_channels.clone())).collect::<Vec<_>>(),
vec![(0, 0, None), (1, 0, Some(vec![0])), (2, 1, None), (3, 2, None)]
);
assert_eq!(audio_placement_specs(&[], 0).len(), 1);
}
#[test]
fn placing_media_with_two_streams_expands_tracks_and_keeps_stream_indices() {
let mut s = Session::default();
let mut p = (*s.project).clone();
let rate = FrameRate::FPS_24;
let seq = p.new_sequence("Two streams", Default::default(), 1, 1, None);
let source = filmcraft_media::generators::GeneratorSource::new(filmcraft_media::Generator::BarsAndTone, 32, 32, rate, rate.tick_of(24));
let mut info = source.info().clone();
info.audio_streams.push(info.audio_streams[0].clone());
let item = p.add_item(
"two streams",
filmcraft_project::Label::Iris,
filmcraft_project::ItemKind::Media(filmcraft_project::MediaClip {
media: filmcraft_project::MediaRef::Generator(filmcraft_media::Generator::BarsAndTone),
info,
interpret: Default::default(),
mark_in: None,
mark_out: None,
markers: vec![],
offline: false,
proxy: None,
identity: None,
}),
None,
);
let (vdest, adest) = {
let q = p.sequence(seq).unwrap();
(q.video_tracks[0].id, q.audio_tracks[0].id)
};
s.project = std::sync::Arc::new(p);
s.state.active_sequence = Some(seq);
let ids =
crate::commands::place_item(&mut s, item, TimeRange::new(Tick::ZERO, rate.tick_of(24)), Tick::ZERO, Some(vdest), Some(adest), false, "test", None)
.unwrap();
assert_eq!(ids.len(), 3);
let q = s.project.sequence(seq).unwrap();
assert_eq!(q.audio_tracks.len(), 2);
let (a, b) = (&q.audio_tracks[0].items[0], &q.audio_tracks[1].items[0]);
assert_eq!((a.audio_stream, b.audio_stream), (0, 1));
assert_eq!(a.link, b.link);
assert!(a.link.is_some());
assert_ne!(a.id, b.id);
assert_eq!(a.source_in, b.source_in);
assert_eq!(a.start, b.start);
}
#[test]
fn missing_audio_tracks_are_cloned_from_the_destination() {
let s = demo();
let seq = active(&s);
let mut p = (*s.project).clone();
let have = p.sequence(seq).map(|q| q.audio_tracks.len()).unwrap_or(0);
assert!(have >= 1);
if let Some(t) = p.sequence_mut(seq).and_then(|q| q.audio_tracks.get_mut(0)) {
t.channels = AudioChannels::Mono;
t.volume_db = -3.0;
t.locked = true;
}
let ids = ensure_audio_tracks(&mut p, seq, 0, have + 3, "test").unwrap();
assert_eq!(ids.len(), have + 3);
let q = p.sequence(seq).unwrap();
for t in &q.audio_tracks[have..] {
assert_eq!(t.kind, TrackKind::Audio);
assert_eq!(t.channels, AudioChannels::Mono);
assert_eq!(t.volume_db, -3.0);
assert!(t.items.is_empty() && !t.locked);
}
let mut unique = ids.clone();
unique.sort_by_key(|t| t.0);
unique.dedup();
assert_eq!(unique.len(), ids.len(), "track ids are unique");
// enough tracks already: nothing is added
let again = ensure_audio_tracks(&mut p, seq, 0, have + 3, "test").unwrap();
assert_eq!(again, ids);
}
#[test]
fn hostile_track_counts_are_an_error_not_a_crash() {
let s = demo();
let seq = active(&s);
let mut p = (*s.project).clone();
assert!(ensure_audio_tracks(&mut p, seq, usize::MAX, usize::MAX, "test").is_err());
assert!(ensure_audio_tracks(&mut p, seq, 0, 1_000_000, "test").is_err());
assert!(ensure_audio_tracks(&mut p, ItemId(u64::MAX), 0, 1, "test").is_err());
}
+16 -2
View File
@@ -22,12 +22,13 @@ use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::sync::mpsc::{Receiver, RecvTimeoutError, Sender, channel};
use std::thread::JoinHandle;
use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};
use std::time::Duration;
use filmcraft_format::autosave as names;
use filmcraft_project::Project;
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
use web_time::{Instant, SystemTime, UNIX_EPOCH};
use crate::settings;
@@ -470,7 +471,17 @@ impl Preferences {
}
}
/// The per-user data directory (`FILMCRAFT_DATA_DIR` overrides):
/// A file with this name next to the executable makes the install portable: its data directory is
/// `<exe dir>/data` instead of the per-user one (the Windows portable zip ships it).
pub const PORTABLE_MARKER: &str = "portable.txt";
/// `<exe_dir>/data` when `exe_dir` holds [`PORTABLE_MARKER`], else `None`.
pub fn portable_data_dir(exe_dir: &Path) -> Option<PathBuf> {
exe_dir.join(PORTABLE_MARKER).is_file().then(|| exe_dir.join("data"))
}
/// The data directory. `FILMCRAFT_DATA_DIR` overrides; a portable install ([`PORTABLE_MARKER`] next
/// to the executable) uses `<exe dir>/data`; otherwise the per-user directory:
/// macOS `~/Library/Application Support/FilmCraft`, Windows `%APPDATA%\FilmCraft`,
/// elsewhere `$XDG_DATA_HOME/filmcraft` or `~/.local/share/filmcraft`.
pub fn default_data_dir() -> Option<PathBuf> {
@@ -478,6 +489,9 @@ pub fn default_data_dir() -> Option<PathBuf> {
if let Some(d) = env("FILMCRAFT_DATA_DIR") {
return Some(d);
}
if let Some(d) = std::env::current_exe().ok().and_then(|exe| exe.parent().and_then(portable_data_dir)) {
return Some(d);
}
if cfg!(target_os = "macos") {
return env("HOME").map(|h| h.join("Library/Application Support/FilmCraft"));
}
+13
View File
@@ -176,6 +176,19 @@ fn auto_save_ring_rotates_and_can_save_the_project() {
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn portable_marker_keeps_data_next_to_the_executable() {
// #139: a portable install must not write to %APPDATA%.
let d = temp_dir("portable");
assert_eq!(autosave::portable_data_dir(&d), None);
std::fs::create_dir_all(d.join(autosave::PORTABLE_MARKER)).unwrap();
assert_eq!(autosave::portable_data_dir(&d), None, "a directory is not the marker");
std::fs::remove_dir_all(d.join(autosave::PORTABLE_MARKER)).unwrap();
std::fs::write(d.join(autosave::PORTABLE_MARKER), "").unwrap();
assert_eq!(autosave::portable_data_dir(&d), Some(d.join("data")));
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn journal_costs_the_ui_thread_only_a_handoff() {
// Per change the UI thread pays an Arc clone and a channel send; encoding happens on the worker.
+90 -24
View File
@@ -237,9 +237,15 @@ pub(crate) fn commands() -> Vec<CommandSpec> {
can_replace_from_source,
|s, p| replace_with(s, p, Replace::MatchFrame),
),
spec("clip.replaceFromBin", "From Bin", &["Clip", "Replace With Clip"], None, r#"{"clips":[id]?,"item":id?}"#, can_replace_from_bin, |s, p| {
replace_with(s, p, Replace::Bin)
}),
spec(
"clip.replaceFromBin",
"From Bin",
&["Clip", "Replace With Clip"],
None,
r#"{"clips":[id]?,"item":id?,"keepSourceIn":bool=false}"#,
can_replace_from_bin,
|s, p| replace_with(s, p, Replace::Bin),
),
]
}
@@ -417,11 +423,7 @@ fn can_replace_from_bin(s: &Session) -> std::result::Result<(), String> {
/// The media item behind `id` (subclips resolve to their parent) and the subclip range.
fn media_root(p: &Project, id: ItemId) -> Option<(ItemId, &MediaClip, Option<TimeRange>)> {
match &p.item(id)?.kind {
ItemKind::Media(m) => Some((id, m, None)),
ItemKind::Subclip { parent, range, .. } => media_root(p, *parent).map(|(i, m, _)| (i, m, Some(*range))),
_ => None,
}
p.resolve_media(id)
}
/// What the Source monitor shows for an item: the media-time span of its time ruler, its frame
@@ -447,6 +449,16 @@ pub struct SourceView {
}
impl SourceView {
/// The marked Source span. Unset marks mean the beginning/end of the source; Out is inclusive.
pub fn selected_range(&self) -> TimeRange {
if self.start.0 < 0 || self.end <= self.start {
return TimeRange::new(self.start, Tick::ZERO);
}
let start = self.mark_in.unwrap_or(self.start).clamp(self.start, self.end);
let end = self.mark_out.map(|o| Tick(o.0.saturating_add(self.rate.frame_duration().0))).unwrap_or(self.end).clamp(start, self.end);
TimeRange::from_bounds(start, end)
}
pub fn to_json(&self, playhead: Tick) -> Value {
json!({
"item": self.item.0, "media": self.media.0, "start": self.start.0, "end": self.end.0, "fps": self.rate.as_f64(),
@@ -634,8 +646,12 @@ fn sequence_from_clip(s: &mut Session, p: &Value) -> Result<Value> {
settings.height = *height;
settings.frame_rate = *rate;
}
let channels = media_root(&s.project, first).and_then(|(_, m, _)| m.info.audio.as_ref().map(|a| a.channels)).unwrap_or(2);
let n_audio = media_root(&s.project, first).and_then(|(_, m, _)| m.interpret.audio_channels.as_ref().map(|a| a.clips.len())).unwrap_or(1).max(1);
let channels = media_root(&s.project, first).and_then(|(_, m, _)| m.info.audio().map(|a| a.channels)).unwrap_or(2);
let n_audio = media_root(&s.project, first)
.map(|(_, m, _)| {
crate::commands::audio_placement_specs(m.interpret.audio_channels.as_ref().map_or(&[], |a| a.clips.as_slice()), m.info.audio_streams.len()).len()
})
.unwrap_or(1);
let n0 = s.history.undo.len();
let name = pi.name.clone();
let bin = s.project.root.parent_of(first).filter(|b| *b != s.project.root.id);
@@ -759,12 +775,15 @@ fn offline_file(s: &mut Session, p: &Value) -> Result<Value> {
bitrate: None,
hdr: None,
}),
audio: has_a.then(|| filmcraft_media::AudioStreamInfo {
sample_rate: u64_p(p, "sampleRate").unwrap_or(48_000) as u32,
channels: u64_p(p, "channels").unwrap_or(2).max(1) as u32,
codec: String::new(),
bits_per_sample: None,
}),
audio_streams: has_a
.then(|| filmcraft_media::AudioStreamInfo {
sample_rate: u64_p(p, "sampleRate").unwrap_or(48_000) as u32,
channels: u64_p(p, "channels").unwrap_or(2).max(1) as u32,
codec: String::new(),
bits_per_sample: None,
})
.into_iter()
.collect(),
container: String::new(),
start_timecode: start_tc,
file_size: None,
@@ -911,7 +930,7 @@ fn consolidate_duplicates(s: &mut Session, _: &Value) -> Result<Value> {
for it in s.project.items.values() {
if let ItemKind::Media(m) = &it.kind {
// same file (or generator), interpretation and streams
let shape = (m.info.video.is_some(), m.info.audio.is_some(), m.info.duration);
let shape = (m.info.video.is_some(), m.info.has_audio(), m.info.duration);
let key = serde_json::to_string(&(&m.media, &m.interpret, m.offline, shape)).unwrap_or_default();
groups.entry(key).or_default().push(it.id);
}
@@ -1247,7 +1266,7 @@ fn audio_channels(s: &mut Session, p: &Value) -> Result<Value> {
let mut maps = Vec::new();
for i in &items {
let Some(m) = s.project.item(*i).and_then(|it| it.as_media()) else { continue };
let n = m.info.audio.as_ref().map_or(2, |a| a.channels) as u16;
let n = m.info.audio().map_or(2, |a| a.channels) as u16;
let fmt = format.unwrap_or_else(|| m.interpret.audio_channels.as_ref().map_or(AudioChannels::Stereo, |a| a.format));
let map = match &custom {
Some(c) => {
@@ -1552,7 +1571,7 @@ fn breakout_to_mono(s: &mut Session, p: &Value) -> Result<Value> {
for i in &items {
let Some(it) = pr.item(*i).cloned() else { continue };
let Some(m) = it.as_media() else { continue };
let n = m.info.audio.as_ref().map_or(1, |a| a.channels).max(1) as u16;
let n = m.info.audio().map_or(1, |a| a.channels).max(1) as u16;
let bin = pr.root.parent_of(*i).filter(|b| *b != pr.root.id);
for c in 0..n {
let suffix = match (n, c) {
@@ -1588,7 +1607,7 @@ fn extract_audio(s: &mut Session, p: &Value) -> Result<Value> {
let mut out = Vec::new();
for item in items {
let (root, m, sub) = media_root(&s.project, item).ok_or_else(|| bad("clip.extractAudio", "not a media clip"))?;
let a = m.info.audio.clone().ok_or_else(|| bad("clip.extractAudio", "the clip has no audio"))?;
let a = m.info.audio().cloned().ok_or_else(|| bad("clip.extractAudio", "the clip has no audio"))?;
let range = sub.unwrap_or(TimeRange::new(Tick::ZERO, m.info.duration));
let dir = match str_p(p, "dir") {
Some(d) => d.to_string(),
@@ -1664,12 +1683,22 @@ fn replace_with(s: &mut Session, p: &Value, how: Replace) -> Result<Value> {
Replace::Bin => item_range(&s.project, new_item, s.prefs.timeline.still_duration(s.sequence_rate())).map(|r| r.start).unwrap_or_default(),
_ => source_range(s).map(|(_, r)| r.start).unwrap_or_default(),
};
// From Bin starts at the item's In point, as Premiere does; `keepSourceIn` keeps each clip's own
let keep_in = how == Replace::Bin && bool_p(p, "keepSourceIn").unwrap_or(false);
// ...but a subclip that restricts trims has no media outside its range: a kept In stays on
// one of its frames (first frame, last frame)
let keep_within = match &pi.kind {
ItemKind::Subclip { range, restrict_trims: true, .. } => {
Some((range.start, (range.end() - item_rate(&s.project, new_item).frame_duration()).max(range.start)))
}
_ => None,
};
let src_ph = s.state.source_playhead;
let ph = s.playhead();
let targets = with_links(s, &clips_p(s, p));
let src_channels = pi.as_media().and_then(|m| m.interpret.audio_channels.as_ref()).and_then(|a| a.clips.first().cloned()).unwrap_or_default();
let replaced = s.edit_sequence("Replace With Clip", |q, _, _| {
let mut out = Vec::new();
let (replaced, clamped) = s.edit_sequence("Replace With Clip", |q, _, _| {
let (mut out, mut clamped) = (Vec::new(), Vec::new());
for c in &targets {
let Some((tid, _)) = q.find_item(*c) else { continue };
let kind = q.track(tid).map(|t| t.kind).unwrap_or_default();
@@ -1680,6 +1709,13 @@ fn replace_with(s: &mut Session, p: &Value, how: Replace) -> Result<Value> {
let src_in = match how {
// the Source monitor frame lines up with the sequence playhead
Replace::MatchFrame => src_ph - Tick(((ph - it.start).0 as f64 * it.speed.abs()).round() as i64),
_ if keep_in => {
let kept = keep_within.map(|(first, last)| it.source_in.clamp(first, last)).unwrap_or(it.source_in);
if kept != it.source_in {
clamped.push(c.0);
}
kept
}
_ => in_point,
};
if src_in < Tick::ZERO {
@@ -1699,7 +1735,37 @@ fn replace_with(s: &mut Session, p: &Value, how: Replace) -> Result<Value> {
if out.is_empty() {
return Err(EngineError::Other("the replacement has no video or audio for the selected clips".into()));
}
Ok(out)
Ok((out, clamped))
})?;
Ok(json!({"clips": replaced, "item": new_item.0}))
let mut result = json!({"clips": replaced, "item": new_item.0, "short": short_clips(s, &replaced)});
if keep_in {
// the clips whose In was outside a trims-restricted subclip and now sits on its nearest frame
result["sourceInClamped"] = json!(clamped);
}
Ok(result)
}
/// The clips among `clips` that play past the end of their media, with how many sequence frames
/// of the clip have no media (at most the clip's own length). The edit is left as it is (the
/// renderer holds the last frame there); this is so whoever replaced the clip knows.
fn short_clips(s: &Session, clips: &[u64]) -> Vec<Value> {
let Some(q) = s.active_sequence() else { return Vec::new() };
let frame = q.settings.frame_rate.frame_duration().0.max(1);
clips
.iter()
.filter_map(|c| {
let (_, it) = q.find_item(ClipId(*c))?;
// stills, mattes and other sources without an end are never short; nor is a held frame
let media_end = crate::media_duration(&s.project, &s.media, it.item)?;
let over = it.source_out().0.saturating_sub(media_end.0);
if over <= 0 || it.frame_hold.is_some() {
return None;
}
// media ticks past the end, as sequence ticks of this clip
let length = it.duration.0.max(0);
let ticks = ((over as f64 / it.speed.abs().max(1e-9)).round().min(length as f64) as i64).min(length);
let frames = ticks.saturating_add(frame - 1) / frame;
(frames > 0).then(|| json!({"clip": c, "shortByFrames": frames}))
})
.collect()
}
+215 -3
View File
@@ -261,7 +261,7 @@ fn new_sequence_from_clip_bin_from_selection_and_offline_file() {
.unwrap();
let it = s.project.item(ItemId(r["item"].as_u64().unwrap())).unwrap();
let m = it.as_media().unwrap();
assert!(m.offline && m.info.audio.is_none() && m.info.video.is_some());
assert!(m.offline && !m.info.has_audio() && m.info.video.is_some());
assert_eq!(m.info.duration, s.sequence_rate().snap_nearest(Tick(4 * TICKS_PER_SECOND)));
assert_eq!(m.info.start_timecode, Some(3600 * 24));
assert_eq!(it.metadata["Tape Name"], "A001");
@@ -372,7 +372,7 @@ fn modify_audio_channels_and_breakout_to_mono() {
s.project = std::sync::Arc::new({
let mut p = (*s.project).clone();
if let Some(m) = p.item_mut(item_named(&s, "Bars and Tone")).and_then(|i| i.as_media_mut()) {
m.info.audio = None;
m.info.audio_streams.clear();
}
p
});
@@ -557,7 +557,7 @@ fn extract_audio_writes_a_wav_and_imports_it() {
let item = s.project.item(ItemId(e["item"].as_u64().unwrap())).unwrap();
let m = item.as_media().unwrap();
assert!(m.info.video.is_none());
assert_eq!(m.info.audio.as_ref().unwrap().channels, 2);
assert_eq!(m.info.audio().unwrap().channels, 2);
let src_dur = s.project.item(forest).unwrap().duration();
assert!((m.info.duration - src_dur).0.abs() < TICKS_PER_SECOND / 100, "same length");
assert_eq!(s.project.root.parent_of(item.id), s.project.root.parent_of(forest));
@@ -602,6 +602,154 @@ fn replace_with_clip_from_source_match_frame_and_bin() {
assert!(s.execute("clip.replaceFromBin", json!({"item": music.0})).is_err());
}
/// Replace With Clip ▸ From Bin restarts the clip at the new item's In point (as Premiere does);
/// `keepSourceIn` keeps the clip's place in the media instead.
#[test]
fn replace_from_bin_keeps_the_source_in_on_request() {
let mut s = demo();
let rate = s.sequence_rate();
s.execute("sequence.linkedSelection", json!({"on": false})).unwrap();
let c = v1(&s)[3].clone();
let dunes = item_named(&s, "Desert_Dunes.mp4");
let forest = item_named(&s, "Misty_Forest.mp4");
// disabled without a clip and a replacement, with or without the new parameter
s.execute("timeline.select", json!({"clips": []})).unwrap();
assert!(matches!(s.execute("clip.replaceFromBin", json!({"keepSourceIn": true})), Err(crate::EngineError::Disabled(..))));
// give the clip a source In that is not the start of its media
s.execute("timeline.trim", json!({"clip": c.id.0, "edge": "in", "mode": "regular", "deltaFrames": 7})).unwrap();
let trimmed = clip(&s, c.id.0);
assert_eq!(trimmed.source_in, c.source_in + rate.tick_of(7));
select(&mut s, &[c.id.0]);
s.execute("project.select", json!({"items": [dunes.0]})).unwrap();
let before = s.project.clone();
// default: the replacement starts at the item's In point, and the result is what it was
let r = s.execute("clip.replaceFromBin", json!({})).unwrap();
assert_eq!((clip(&s, c.id.0).item, clip(&s, c.id.0).source_in), (dunes, Tick::ZERO));
assert_eq!(r, json!({"clips": [c.id.0], "item": dunes.0, "short": []}));
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(*s.project, *before);
// keepSourceIn: the same place in the new media, everything else as before
let r = s.execute("clip.replaceFromBin", json!({"keepSourceIn": true})).unwrap();
let got = clip(&s, c.id.0);
assert_eq!((got.item, got.source_in, got.start, got.duration), (dunes, trimmed.source_in, trimmed.start, trimmed.duration));
assert_eq!(r, json!({"clips": [c.id.0], "item": dunes.0, "short": [], "sourceInClamped": []}));
let after = s.project.clone();
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(*s.project, *before);
s.execute("edit.redo", json!({})).unwrap();
assert_eq!(*s.project, *after);
s.execute("edit.undo", json!({})).unwrap();
s.execute("clip.replaceFromBin", json!({"keepSourceIn": false})).unwrap();
assert_eq!(clip(&s, c.id.0).source_in, Tick::ZERO);
s.execute("edit.undo", json!({})).unwrap();
// a subclip that restricts trims has no media outside its range: a kept In before or after it
// moves to its first or last frame, and the result names the clip
let sub = |s: &mut Session, from: Tick, to: Tick, restrict: bool| {
let r = s.execute("clip.makeSubclip", json!({"item": forest.0, "start": from.0, "end": to.0, "restrictTrims": restrict})).unwrap();
ItemId(r["item"].as_u64().unwrap())
};
let kept = trimmed.source_in;
let media_frame = s.project.item(forest).unwrap().as_media().unwrap().frame_rate().frame_duration();
assert!(kept > rate.tick_of(10) && media_frame > Tick::ZERO);
let later = sub(&mut s, kept + rate.tick_of(3), kept + rate.tick_of(15), true);
let earlier = sub(&mut s, Tick::ZERO, kept - rate.tick_of(2), true);
let loose = sub(&mut s, kept + rate.tick_of(3), kept + rate.tick_of(15), false);
for (item, want, clamped) in [(later, kept + rate.tick_of(3), true), (earlier, kept - rate.tick_of(2) - media_frame, true), (loose, kept, false)] {
let r = s.execute("clip.replaceFromBin", json!({"item": item.0, "keepSourceIn": true})).unwrap();
let got = clip(&s, c.id.0);
assert_eq!((got.item, got.source_in, got.start, got.duration), (item, want, trimmed.start, trimmed.duration));
assert_eq!(r["sourceInClamped"], if clamped { json!([c.id.0]) } else { json!([]) });
s.execute("edit.undo", json!({})).unwrap();
}
// a kept In that is already inside the subclip stays where it is
let around = sub(&mut s, kept - rate.tick_of(5), kept + rate.tick_of(23), true);
let r = s.execute("clip.replaceFromBin", json!({"item": around.0, "keepSourceIn": true})).unwrap();
assert_eq!((clip(&s, c.id.0).source_in, &r["sourceInClamped"]), (kept, &json!([])));
// hostile values of the flag fall back to the default
s.execute("edit.undo", json!({})).unwrap();
for junk in [json!("yes"), json!(1), json!(null), json!([true])] {
let r = s.execute("clip.replaceFromBin", json!({"item": dunes.0, "keepSourceIn": junk})).unwrap();
assert_eq!((clip(&s, c.id.0).source_in, r.get("sourceInClamped")), (Tick::ZERO, None));
s.execute("edit.undo", json!({})).unwrap();
}
}
/// A replacement shorter than the clip's slot left the clip running past the end of its media
/// without a word. The three Replace With Clip commands now say which clips are short, by how
/// many sequence frames.
#[test]
fn replace_with_clip_reports_clips_that_run_past_the_new_media() {
let mut s = demo();
let rate = s.sequence_rate();
let demo_seq = s.state.active_sequence.unwrap();
s.execute("sequence.linkedSelection", json!({"on": false})).unwrap();
let c = v1(&s)[3].clone();
let slot = rate.frame_at(c.duration);
let forest = item_named(&s, "Misty_Forest.mp4");
let forest_len = s.project.item(forest).unwrap().as_media().unwrap().info.duration;
assert!(slot > 12 && forest_len > c.duration + rate.tick_of(40), "the clip is longer than 12 frames and shorter than the media");
let state = |s: &Session| {
let it = clip(s, c.id.0);
(it.item, it.source_in, it.start, it.duration)
};
select(&mut s, &[c.id.0]);
// From Bin, with 12 frames of media (a subclip that restricts trims): the edit is what it
// always was, and the result says the clip is short
let r = s.execute("clip.makeSubclip", json!({"item": forest.0, "start": rate.tick_of(10).0, "end": rate.tick_of(22).0})).unwrap();
let short = ItemId(r["item"].as_u64().unwrap());
let before = s.project.clone();
let r = s.execute("clip.replaceFromBin", json!({"item": short.0})).unwrap();
assert_eq!(state(&s), (short, rate.tick_of(10), c.start, c.duration), "the edit itself is not changed");
assert_eq!(r["short"], json!([{"clip": c.id.0, "shortByFrames": slot - 12}]));
let after = s.project.clone();
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(*s.project, *before);
s.execute("edit.redo", json!({})).unwrap();
assert_eq!(*s.project, *after);
s.execute("edit.undo", json!({})).unwrap();
// long enough media: nothing is short
assert_eq!(s.execute("clip.replaceFromBin", json!({"item": forest.0})).unwrap()["short"], json!([]));
s.execute("edit.undo", json!({})).unwrap();
// From Source Monitor, with the Source In 20 frames before the end of the media
let frames = |t: Tick| (t.0 + rate.frame_duration().0 - 1) / rate.frame_duration().0;
s.execute("source.open", json!({"item": forest.0})).unwrap();
let mark = forest_len - rate.tick_of(20);
s.execute("project.setMarks", json!({"item": forest.0, "in": mark.0})).unwrap();
let r = s.execute("clip.replaceFromSource", json!({})).unwrap();
let got = clip(&s, c.id.0);
assert_eq!(r["short"], json!([{"clip": c.id.0, "shortByFrames": frames(got.source_out() - forest_len)}]));
assert_eq!(frames(got.source_out() - forest_len), slot - 20);
s.execute("edit.undo", json!({})).unwrap();
// Match Frame: the last frame of the media on the clip's first frame
s.execute("playhead.set", json!({"time": c.start.0})).unwrap();
s.execute("source.setPlayhead", json!({"time": (forest_len - rate.tick_of(1)).0})).unwrap();
let r = s.execute("clip.replaceFromSourceMatchFrame", json!({})).unwrap();
assert_eq!(r["short"], json!([{"clip": c.id.0, "shortByFrames": slot - 1}]));
s.execute("edit.undo", json!({})).unwrap();
// with enough media after the frame, nothing is short
s.execute("source.setPlayhead", json!({"frame": 5})).unwrap();
assert_eq!(s.execute("clip.replaceFromSourceMatchFrame", json!({})).unwrap()["short"], json!([]));
s.execute("edit.undo", json!({})).unwrap();
// never more than the clip itself: an In that lies past the end of the new media altogether
s.execute("timeline.trim", json!({"clip": c.id.0, "edge": "in", "mode": "regular", "deltaFrames": 7})).unwrap();
let trimmed = clip(&s, c.id.0);
s.execute("file.newSequence", json!({"name": "Brief"})).unwrap();
let brief = s.state.active_sequence.unwrap();
s.execute("timeline.place", json!({"item": forest.0, "time": 0})).unwrap();
let placed = s.active_sequence().unwrap().video_tracks[0].items[0].clone();
s.execute("timeline.trim", json!({"clip": placed.id.0, "edge": "out", "mode": "regular", "delta": (rate.tick_of(3) - placed.duration).0})).unwrap();
s.execute("sequence.open", json!({"item": demo_seq.0})).unwrap();
select(&mut s, &[c.id.0]);
assert!(trimmed.source_in > s.project.sequence(brief).unwrap().duration(), "the kept In is past the end of the nested sequence");
let r = s.execute("clip.replaceFromBin", json!({"clips": [c.id.0], "item": brief.0, "keepSourceIn": true})).unwrap();
assert_eq!(clip(&s, c.id.0).source_in, trimmed.source_in);
assert_eq!(r["short"], json!([{"clip": c.id.0, "shortByFrames": rate.frame_at(trimmed.duration)}]), "all of the clip, not more");
}
#[test]
fn audio_source_channels_pick_what_a_clip_plays() {
let mut it = v1(&demo())[0].clone();
@@ -689,3 +837,67 @@ fn subclip_source_monitor_trims_and_markers() {
s.execute("edit.undo", json!({})).unwrap();
assert!(matches!(s.project.item(sub).unwrap().kind, ItemKind::Subclip { restrict_trims: false, .. }));
}
/// #29: a Color Matte's color can be changed after it is created; the generated frames follow,
/// and undo brings the old color back (the media pool regenerates the matte, it doesn't keep
/// serving the first one).
#[test]
fn color_matte_color_changes_and_undoes() {
let mut s = demo();
let matte = s.execute("file.newColorMatte", json!({"width": 64, "height": 36, "color": "#ff0000"})).unwrap()["item"].as_u64().unwrap();
let px = |s: &Session| {
let f = s.source(ItemId(matte)).unwrap().video_frame(filmcraft_media::FrameRequest::full(Tick::ZERO)).unwrap();
f.to_rgba8()[..3].to_vec()
};
assert_eq!(px(&s), [255, 0, 0]);
let r = s.execute("project.matteColor", json!({"item": matte, "color": "#0000ff"})).unwrap();
assert_eq!(r["color"], "#0000ff");
assert_eq!(px(&s), [0, 0, 255]);
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(px(&s), [255, 0, 0]);
// the selected matte when no item is named; anything else is refused
s.state.project_selection = vec![ItemId(matte)];
s.execute("project.matteColor", json!({"color": "#00ff00"})).unwrap();
assert_eq!(px(&s), [0, 255, 0]);
let bars = s.execute("file.newBarsAndTone", json!({})).unwrap()["item"].as_u64().unwrap();
assert!(s.execute("project.matteColor", json!({"item": bars, "color": "#00ff00"})).is_err());
assert!(s.execute("project.matteColor", json!({"item": matte, "color": "green"})).is_err());
}
/// A damaged project whose subclip is its own parent (#66): `media_duration` followed the chain
/// without a bound and overflowed the stack. It gives up after a few hops now.
#[test]
fn a_cyclic_subclip_chain_has_no_media_duration() {
let mut s = demo();
let ocean = item_named(&s, "Ocean_Sunset.mp4");
s.execute("source.open", json!({"item": ocean.0})).unwrap();
let sub = ItemId(s.execute("clip.makeSubclip", json!({})).unwrap()["item"].as_u64().unwrap());
s.execute("clip.editSubclip", json!({"item": sub.0, "restrictTrims": false})).unwrap();
let Some(ItemKind::Subclip { parent, .. }) = std::sync::Arc::make_mut(&mut s.project).item_mut(sub).map(|i| &mut i.kind) else { panic!() };
*parent = sub;
assert_eq!(crate::media_duration(&s.project, &s.media, sub), None);
}
/// #164: Add Edit with a clip selected cut every targeted track. Like Premiere it now cuts only
/// the selected clips under the playhead; with nothing selected there, the targeted tracks.
#[test]
fn add_edit_cuts_only_the_selected_clips() {
let mut s = demo();
let count = |s: &Session| {
let q = s.active_sequence().unwrap();
(q.video_tracks.iter().map(|t| t.items.len()).sum::<usize>(), q.audio_tracks.iter().map(|t| t.items.len()).sum::<usize>())
};
// a V1 clip and a time inside it
let first = v1(&s)[1].clone();
let t = first.start + filmcraft_time::Tick(first.duration.0 / 2);
let (v0, a0) = count(&s);
s.state.selection = vec![first.id];
s.execute("sequence.addEdit", json!({"time": t.0})).unwrap();
let (v1n, a1n) = count(&s);
assert_eq!((v1n, a1n), (v0 + 1, a0), "only the selected V1 clip is cut");
// nothing selected under the playhead: every targeted track is cut
s.execute("edit.undo", json!({})).unwrap();
s.state.selection.clear();
let r = s.execute("sequence.addEdit", json!({"time": t.0})).unwrap();
assert!(r["cuts"].as_u64().unwrap() > 1, "{r}");
}
File diff suppressed because it is too large. Load diff
+14 -18
View File
@@ -21,24 +21,20 @@ pub fn add_generator(p: &mut Project, pool: &MediaPool, src: GeneratorSource, na
i.name = name.to_string();
i
};
let g = src.generator.clone();
let id = p.add_item(
name,
label,
ItemKind::Media(MediaClip {
media: MediaRef::Generator(g),
info,
interpret: Default::default(),
mark_in: None,
mark_out: None,
markers: vec![],
offline: false,
proxy: None,
identity: None,
}),
bin,
);
pool.insert(id, Arc::new(src.with_name(name)));
let clip = MediaClip {
media: MediaRef::Generator(src.generator.clone()),
info,
interpret: Default::default(),
mark_in: None,
mark_out: None,
markers: vec![],
offline: false,
proxy: None,
identity: None,
};
let key = crate::media_pool::media_key(&clip);
let id = p.add_item(name, label, ItemKind::Media(clip), bin);
pool.insert_keyed(id, key, Arc::new(src.with_name(name)));
id
}
+14 -3
View File
@@ -12,7 +12,7 @@
//! | `essentialSound.setType` / `clearType` | assign / clear the audio type (clearing removes its effects and gain) |
//! | `essentialSound.set` | one setting by dotted key (`repair.noise.amount`), or several (`values`) |
//! | `essentialSound.applyPreset` / `savePreset` / `deletePreset` | built-in and user presets |
//! | `essentialSound.autoMatch` | measure integrated loudness (BS.1770) and set the match gain to the target |
//! | `essentialSound.autoMatch` | measure integrated loudness (BS.1770, mono clips as one channel) and set the match gain to the target |
//! | `essentialSound.generateDucking` | Volume keyframes that duck Music / Ambience under the trigger types |
use serde_json::{Value, json};
@@ -262,15 +262,26 @@ fn delete_preset(s: &mut Session, p: &Value) -> Result<Value> {
/// Integrated loudness (LUFS) of a clip's own signal (clip gain + clip effects) over its whole
/// timeline range, or `None` when it is silent / has no audio.
///
/// The clip is measured with the channels it has: a stereo (or wider) source as the two channels
/// it plays, a mono source or a mono pick as one channel (its two identical sides folded to one),
/// the way BS.1770 and ffmpeg's `ebur128` measure a mono programme. Metering the dual-mono pair the
/// clip plays would read 3 LU louder than the file and match it 3 dB too quietly (#296).
pub fn clip_loudness(item: &TrackItem, sr: u32, sources: &dyn SourceProvider) -> Option<f64> {
let a0 = item.start.to_units_floor(sr as i64);
let a1 = item.end().to_units_floor(sr as i64);
let mut meter = LoudnessMeter::new(sr as f64, 2);
let mono = filmcraft_render::audio::clip_is_mono(item, sources)?;
let mut meter = LoudnessMeter::new(sr as f64, if mono { 1 } else { 2 });
let mut pos = a0;
while pos < a1 {
let n = (a1 - pos).min(sr as i64) as usize;
let [l, r] = filmcraft_render::audio::clip_signal(item, pos, n, sr, sources)?;
meter.process(&[&l, &r]);
if mono {
let m: Vec<f32> = l.iter().zip(&r).map(|(a, b)| 0.5 * (a + b)).collect();
meter.process(&[&m]);
} else {
meter.process(&[&l, &r]);
}
pos += n as i64;
}
Some(meter.integrated()).filter(|v| v.is_finite())
@@ -265,6 +265,41 @@ fn auto_match_hits_the_target_loudness() {
}
}
/// #296: a mono file measured ~3 LU too loud. The same speech imported as a 1-channel WAV and as
/// a dual-mono stereo WAV must measure 3.01 LU apart (the stereo pair carries twice the energy),
/// the mono clip must agree with a one-channel BS.1770 meter over the file (what ffmpeg's
/// `ebur128` reports), and a mono pick (Modify ▸ Audio Channels) of the stereo file counts as mono.
#[test]
fn auto_match_measures_a_mono_clip_as_one_channel() {
let sig = speech(12.0, 0.3, |t| (t % 3.0) < 2.2);
let (mut s, c) = session_with(&[(sig.clone(), 0.0)]);
let stereo = c[0];
let mono_wav: Arc<[u8]> = filmcraft_media::wav::write_wav16(&sig, 1, SR).into();
let mono_item = crate::commands::import_bytes(&mut s, "/mono.wav", mono_wav, None).unwrap();
let r = s.execute("timeline.place", json!({"item": mono_item.0, "audioTrack": "A2", "seconds": 0.0})).unwrap();
let mono = ClipId(r["clips"][0].as_u64().unwrap());
for c in [stereo, mono] {
s.execute("essentialSound.setType", json!({"clips": [c.0], "type": "dialogue"})).unwrap();
}
let measured = |s: &mut Session, c: ClipId| {
let r = s.execute("essentialSound.autoMatch", json!({"clips": [c.0], "target": -23.0})).unwrap();
r["clips"][0]["measuredLufs"].as_f64().unwrap()
};
let (two, one) = (measured(&mut s, stereo), measured(&mut s, mono));
let mut file = LoudnessMeter::new(SR as f64, 1);
file.process(&[&sig]);
let want = file.integrated();
println!("stereo (dual mono) {two:.3} LUFS, mono {one:.3} LUFS, mono file {want:.3} LUFS");
assert!((one - want).abs() <= 0.1, "mono clip {one} LUFS, the file reads {want} LUFS");
assert!((two - one - 3.0103).abs() <= 0.1, "dual mono {two} LUFS should read 3.01 LU above mono {one} LUFS");
let st = item(&s, mono).essential.unwrap();
assert!((st.loudness.gain_db - (-23.0 - want)).abs() <= 0.1, "match gain {} dB for {want} LUFS", st.loudness.gain_db);
// one picked channel of the stereo file plays on both sides: measured as mono too
s.execute("clip.audioChannels", json!({"clips": [stereo.0], "channels": [0]})).unwrap();
let picked = measured(&mut s, stereo);
assert!((picked - want).abs() <= 0.1, "mono pick {picked} LUFS, the file reads {want} LUFS");
}
#[test]
fn ducking_generates_volume_keyframes_under_dialogue() {
// dialogue in [2, 4] and [6, 7.5] s; music for 10 s
+213
View File
@@ -0,0 +1,213 @@
//! A caller that names its targets in the parameters (`clips`, `clip`, `item`, `items`) does not
//! need a selection: enablement is checked against the named targets. Without such parameters
//! (the menus, shortcuts and `engine.commands`) enablement is the selection's, as before.
use filmcraft_project::{ClipId, ItemId, TrackItem};
use serde_json::json;
use crate::{EngineError, Session};
fn demo() -> Session {
let mut s = Session::default();
s.execute("file.openDemoProject", json!({})).unwrap();
s.execute("sequence.linkedSelection", json!({"on": false})).unwrap();
s.state.selection.clear();
s.state.project_selection.clear();
s
}
fn item_named(s: &Session, name: &str) -> ItemId {
s.project.items.values().find(|i| i.name == name).map(|i| i.id).unwrap_or_else(|| panic!("no item {name}"))
}
fn v1(s: &Session) -> Vec<TrackItem> {
s.active_sequence().unwrap().video_tracks[0].items.clone()
}
fn a1(s: &Session) -> Vec<TrackItem> {
s.active_sequence().unwrap().audio_tracks[0].items.clone()
}
fn clip(s: &Session, id: ClipId) -> Option<TrackItem> {
s.active_sequence().unwrap().find_item(id).map(|(_, i)| i.clone())
}
fn disabled(r: crate::Result<serde_json::Value>) -> String {
match r {
Err(EngineError::Disabled(_, why)) => why,
other => panic!("expected a disabled command, got {other:?}"),
}
}
#[test]
fn replace_from_bin_takes_an_explicit_item_and_clips_without_any_selection() {
let mut s = demo();
let c = v1(&s)[0].clone();
let dunes = item_named(&s, "Desert_Dunes.mp4");
assert_ne!(c.item, dunes);
// nothing named, nothing selected: disabled, for the menus and for a caller alike
assert!(!s.is_enabled("clip.replaceFromBin"));
assert_eq!(disabled(s.execute("clip.replaceFromBin", json!({}))), "no clips selected");
// the clip is named but the replacement is not: still the Project panel's call
assert_eq!(disabled(s.execute("clip.replaceFromBin", json!({"clips": [c.id.0]}))), "select a clip in the Project panel");
// both named: runs with no timeline and no Project-panel selection
s.execute("clip.replaceFromBin", json!({"clips": [c.id.0], "item": dunes.0})).unwrap();
assert_eq!(clip(&s, c.id).unwrap().item, dunes);
assert!(s.state.selection.is_empty() && s.state.project_selection.is_empty(), "the selections are left as they were");
assert!(!s.is_enabled("clip.replaceFromBin"), "menu enablement still follows the selection");
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(clip(&s, c.id).unwrap().item, c.item);
// a timeline selection plus an explicit item (no Project-panel selection)
s.execute("timeline.select", json!({"clips": [c.id.0]})).unwrap();
s.execute("clip.replaceFromBin", json!({"item": dunes.0})).unwrap();
assert_eq!(clip(&s, c.id).unwrap().item, dunes);
}
#[test]
fn clip_commands_take_explicit_clips_without_a_selection() {
let mut s = demo();
let (a, b) = (v1(&s)[0].clone(), v1(&s)[1].clone());
for id in ["edit.clear", "edit.rippleDelete", "clip.audioGain", "edit.pasteAttributes", "clip.enable"] {
assert!(!s.is_enabled(id), "{id} is disabled without a selection");
assert!(matches!(s.execute(id, json!({})), Err(EngineError::Disabled(..))), "{id} with no parameters stays disabled");
}
// Clear
s.execute("edit.clear", json!({"clips": [a.id.0]})).unwrap();
assert!(clip(&s, a.id).is_none() && clip(&s, b.id).is_some());
s.execute("edit.undo", json!({})).unwrap();
assert!(clip(&s, a.id).is_some());
// Ripple Delete
let a_and_its_audio: Vec<u64> = s
.active_sequence()
.unwrap()
.all_tracks()
.flat_map(|t| &t.items)
.filter(|i| i.id == a.id || (i.link.is_some() && i.link == a.link))
.map(|i| i.id.0)
.collect();
s.execute("timeline.setTrack", json!({"track": "A2", "syncLock": false})).unwrap(); // the music on A2 would block the ripple
s.execute("edit.rippleDelete", json!({"clips": a_and_its_audio})).unwrap();
assert!(clip(&s, a.id).is_none());
assert_eq!(clip(&s, b.id).unwrap().start, a.start, "the next clip closes the gap");
s.execute("edit.undo", json!({})).unwrap();
// Audio Gain
let au = a1(&s)[0].clone();
s.execute("clip.audioGain", json!({"clips": [au.id.0], "mode": "set", "db": -6.0})).unwrap();
assert_eq!(clip(&s, au.id).unwrap().gain_db, -6.0);
// Enable
s.execute("clip.enable", json!({"clips": [a.id.0]})).unwrap();
assert!(!clip(&s, a.id).unwrap().enabled);
s.execute("edit.undo", json!({})).unwrap();
assert!(clip(&s, a.id).unwrap().enabled);
s.execute("edit.redo", json!({})).unwrap();
assert!(!clip(&s, a.id).unwrap().enabled);
// `clip.enable` documents `clips` only: a key it does not document is not a named target
assert_eq!(disabled(s.execute("clip.enable", json!({"clip": a.id.0}))), "no clips selected");
// Paste Attributes: naming the clips lifts only the selection condition, not the clipboard one
assert_eq!(disabled(s.execute("edit.pasteAttributes", json!({"clips": [b.id.0]}))), "copy a clip first");
s.execute("timeline.select", json!({"clips": [a.id.0]})).unwrap();
s.execute("edit.copy", json!({})).unwrap();
s.state.selection.clear();
s.execute("edit.pasteAttributes", json!({"clips": [b.id.0]})).unwrap();
assert!(s.state.selection.is_empty(), "the selection is left as it was");
for id in ["edit.clear", "edit.rippleDelete", "clip.audioGain", "edit.pasteAttributes"] {
assert!(!s.is_enabled(id), "menu enablement of {id} still follows the selection");
}
}
#[test]
fn naming_nothing_usable_does_not_enable_a_command() {
let mut s = demo();
let before = s.project.clone();
// no ids, ids of no clip, the wrong type, and a key the command does not take
for p in [json!({"clips": []}), json!({"clips": [987_654_321u64]}), json!({"clips": "all"}), json!({"clips": [-1, 1.5, null]}), json!({"items": [1]})] {
assert_eq!(disabled(s.execute("edit.rippleDelete", p.clone())), "nothing selected", "{p}");
}
assert_eq!(disabled(s.execute("clip.replaceFromBin", json!({"clips": [v1(&s)[0].id.0], "item": 987_654_321u64}))), "select a clip in the Project panel");
// `edit.cut` takes no `clips`: it works on the selection only
assert_eq!(disabled(s.execute("edit.cut", json!({"clips": [v1(&s)[0].id.0]}))), "no clips selected");
assert!(std::sync::Arc::ptr_eq(&before, &s.project), "nothing was edited");
}
#[test]
fn link_and_speed_take_explicit_clips_without_a_selection() {
let mut s = demo();
let (v, au) = (v1(&s)[0].clone(), a1(&s)[0].clone());
for id in ["clip.link", "clip.speedDuration"] {
assert!(!s.is_enabled(id));
assert_eq!(disabled(s.execute(id, json!({}))), "no clips selected");
}
// Link toggles the named pair
let was = v.link.is_some() && v.link == au.link;
let r = s.execute("clip.link", json!({"clips": [v.id.0, au.id.0]})).unwrap();
assert_eq!(r["linked"], !was);
assert_eq!(clip(&s, v.id).unwrap().link.is_some(), !was);
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(clip(&s, v.id).unwrap().link, v.link);
s.execute("edit.redo", json!({})).unwrap();
assert_eq!(clip(&s, v.id).unwrap().link.is_some(), !was);
s.execute("edit.undo", json!({})).unwrap();
// Speed/Duration on the last clip of V1 (nothing after it to run into)
let last = v1(&s).last().unwrap().clone();
s.execute("clip.speedDuration", json!({"clips": [last.id.0], "speed": 200})).unwrap();
let fast = clip(&s, last.id).unwrap();
assert!(fast.duration < last.duration, "{:?} -> {:?}", last.duration, fast.duration);
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(clip(&s, last.id).unwrap().duration, last.duration);
s.execute("edit.redo", json!({})).unwrap();
assert_eq!(clip(&s, last.id).unwrap().duration, fast.duration);
assert!(s.state.selection.is_empty() && !s.is_enabled("clip.speedDuration"), "menu enablement still follows the selection");
}
#[test]
fn naming_targets_lifts_only_the_selection_condition() {
// no sequence: named clips cannot stand in for one
let mut empty = Session::default();
assert_eq!(disabled(empty.execute("edit.clear", json!({"clips": [1]}))), "no sequence is open");
assert_eq!(disabled(empty.execute("clip.speedDuration", json!({"clips": [1], "speed": 50}))), "no sequence is open");
let mut s = demo();
let a = v1(&s)[0].clone();
// the wrong kind of target: a video clip is not a graphic
assert_eq!(disabled(s.execute("graphics.setText", json!({"clip": a.id.0, "text": "x"}))), "select a graphic clip");
// a locked track is still the edit's own business: named or selected, the clip stays
s.execute("timeline.setTrack", json!({"track": "V1", "locked": true})).unwrap();
let _ = s.execute("edit.clear", json!({"clips": [a.id.0]}));
assert!(clip(&s, a.id).is_some(), "a clip on a locked track is not removed");
}
/// `project.delete` with explicit ids needs no Project-panel selection, for bins as much as for
/// items: the selection can hold both, so a named bin stands in for it (#244).
#[test]
fn project_delete_takes_explicit_bins_and_items_without_a_selection() {
let mut s = demo();
let before = s.project.items.len();
let empty = s.execute("file.newBin", json!({"name": "Empty"})).unwrap()["bin"].as_u64().unwrap();
let outer = s.execute("file.newBin", json!({"name": "Old footage"})).unwrap()["bin"].as_u64().unwrap();
let ocean = item_named(&s, "Ocean_Sunset.mp4");
s.execute("project.moveToBin", json!({"items": [ocean.0], "bin": outer})).unwrap();
s.state.project_selection.clear();
assert!(!s.is_enabled("project.delete"));
assert_eq!(disabled(s.execute("project.delete", json!({}))), "select an item in the Project panel");
// an empty bin, by its id alone
let r = s.execute("project.delete", json!({"items": [empty]})).unwrap();
assert_eq!(r, json!({"items": 0, "bins": 1}));
assert!(s.project.root.find_bin(filmcraft_project::BinId(empty)).is_none());
// a bin with a clip in it, and an item, each by id alone
let r = s.execute("project.delete", json!({"items": [outer]})).unwrap();
assert_eq!(r, json!({"items": 1, "bins": 1}));
assert!(s.project.item(ocean).is_none());
let dunes = item_named(&s, "Desert_Dunes.mp4");
let r = s.execute("project.delete", json!({"items": [dunes.0]})).unwrap();
assert_eq!(r, json!({"items": 1, "bins": 0}));
assert_eq!(s.project.items.len(), before - 2);
assert!(s.state.project_selection.is_empty(), "the selection is left as it was");
assert!(!s.is_enabled("project.delete"), "menu enablement still follows the selection");
// ids of nothing, or of the project's own top bin, do not stand in for a selection
let root = s.project.root.id.0;
assert_eq!(disabled(s.execute("project.delete", json!({"items": [root]}))), "select an item in the Project panel");
assert_eq!(disabled(s.execute("project.delete", json!({"items": [999_999]}))), "select an item in the Project panel");
s.execute("edit.undo", json!({})).unwrap();
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(s.project.items.len(), before);
}
+124
View File
@@ -9,6 +9,20 @@ use serde_json::{Value, json};
use crate::Session;
#[test]
fn hostile_export_ranges_are_rejected_without_panicking() {
let mut session = demo();
for params in [
json!({"range":"custom", "startTime":i64::MIN, "endTime":i64::MAX}),
json!({"range":"custom", "startTime":-1, "endTime":100}),
json!({"range":"custom", "startTime":100, "endTime":100}),
] {
let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| session.execute("export.resolve", params)));
assert!(result.is_ok(), "a hostile export range must not panic");
assert!(result.unwrap().is_err());
}
}
/// A scratch directory under the workspace `target/`, removed when dropped.
struct Scratch(PathBuf);
@@ -36,6 +50,20 @@ fn demo() -> Session {
s
}
#[test]
fn export_integer_parameters_cannot_wrap_or_overflow() {
let mut s = demo();
for name in ["bitrateKbps", "maxBitrateKbps", "keyframeDistance"] {
for value in [json!(-1), json!(1.5), json!(u64::from(u32::MAX) + 1)] {
let mut params = json!({});
params[name] = value;
assert!(s.execute("export.resolve", params).is_err(), "{name}");
}
}
assert!(s.execute("export.resolve", json!({"bitrateKbps":u32::MAX})).is_ok());
assert!(s.execute("export.resolve", json!({"bitrateKbps":8000.0, "keyframeDistance":48.0})).is_ok(), "integer-valued floats are integers");
}
fn probe(path: &str) -> Option<Value> {
let ffprobe = filmcraft_testkit::ffprobe_or_skip("export presets")?;
let out = std::process::Command::new(ffprobe).args(["-v", "error", "-of", "json", "-show_format", "-show_streams", path]).output().unwrap();
@@ -179,6 +207,24 @@ fn every_builtin_preset_exports_and_ffprobe_confirms() {
assert!((vbr / (mbps * px) - 1.0).abs() < 0.25, "{name}: {vbr:.1} Mb/s vs nominal {:.1}", mbps * px);
assert_eq!(a["codec_name"], "pcm_s24le", "{name}");
}
"apv" => {
assert!(fmt_name.contains("mov"), "{name}: {fmt_name}");
assert_eq!(v["codec_name"], "apv", "{name}");
let p = settings["apvProfile"].as_str().unwrap();
let idc = match p {
"422-10" => "33",
"422-12" => "44",
"444-10" => "55",
"444-12" => "66",
"4444-10" => "77",
"4444-12" => "88",
"400-10" => "99",
other => other,
};
let got_profile = v["profile"].as_str().unwrap_or("");
assert!(got_profile == p || got_profile == idc, "{name}: profile {got_profile} != {p} ({idc})");
assert_eq!(a["codec_name"], "pcm_s24le", "{name}");
}
"mxf-op1a" => {
assert_eq!(fmt_name, "mxf", "{name}");
let codec = match settings["mxfVideoCodec"].as_str().unwrap() {
@@ -345,6 +391,67 @@ fn queue_cancels_a_running_export_and_retries_a_failed_one() {
assert!(queue(&mut s).iter().all(|i| i["id"] != bad));
}
#[test]
fn jobs_and_queue_items_report_the_time_they_have_left() {
use std::sync::atomic::Ordering;
use std::time::Duration;
use crate::export_tools::QueueStatus;
let mut s = demo();
let dir = Scratch::new("eta");
let job = crate::Job { id: 77, label: "Exporting".into(), progress: Default::default(), result: Default::default() };
job.progress.total.store(1000, Ordering::Relaxed);
assert!(job.to_json()["etaSeconds"].is_null(), "no speed measured yet");
// 50 units a second for 5 s, read from the near future (the JSON reads the real clock, which is then behind them)
let t0 = web_time::Instant::now() + Duration::from_secs(60);
for i in 0..=50u64 {
job.progress.done.store(i * 5, Ordering::Relaxed);
job.progress.eta_at(t0 + Duration::from_millis(i * 100));
}
let eta = job.to_json()["etaSeconds"].as_f64().expect("a number once the job has a speed");
assert!((eta - 15.0).abs() < 0.5, "750 units left at 50 a second: {eta}");
s.jobs.push(job);
// a queue item encoding that job shows it; one that waits shows nothing
for name in ["a.wav", "b.wav"] {
s.execute(
"export.queue.add",
json!({"preset": "Waveform Audio 48 kHz 16-bit", "path": dir.path(name), "range": "custom", "startSeconds": 0, "endSeconds": 0.25}),
)
.unwrap();
}
s.export_queue.items[0].status = QueueStatus::Encoding;
s.export_queue.items[0].job = Some(77);
let items = queue(&mut s);
assert!((items[0]["etaSeconds"].as_f64().expect("encoding") - 15.0).abs() < 0.5, "{}", items[0]);
assert_eq!(items[0]["status"], "encoding");
assert!(items[1]["etaSeconds"].is_null(), "{}", items[1]);
// and `jobs.list` carries it too
let jobs = s.execute("jobs.list", json!({})).unwrap();
let listed = jobs["jobs"].as_array().or(jobs.as_array()).expect("a list of jobs").iter().find(|j| j["id"] == 77).cloned().expect("the job is listed");
assert!(listed["etaSeconds"].as_f64().is_some(), "{listed}");
}
#[test]
fn hardware_encoding_is_off_unless_asked_for() {
use filmcraft_export::HardwareEncoding::{Auto, Off};
let s = demo();
let setting = |p: Value| crate::export_tools::settings_from_params(&s, &p, "file.exportMedia").map(|(_, st)| st.hardware_encoding);
assert_eq!(setting(json!({"path": "x.mp4"})).unwrap(), Off);
assert_eq!(setting(json!({"path": "x.mp4", "hardwareEncoding": "auto"})).unwrap(), Auto);
assert_eq!(setting(json!({"path": "x.mp4", "hardwareEncoding": "off"})).unwrap(), Off);
// a boolean is understood too: true is auto
assert_eq!(setting(json!({"path": "x.mp4", "hardwareEncoding": true})).unwrap(), Auto);
assert_eq!(setting(json!({"path": "x.mp4", "hardwareEncoding": false})).unwrap(), Off);
// anything else names the choices
let e = setting(json!({"path": "x.mp4", "hardwareEncoding": "gpu"})).unwrap_err().to_string();
assert!(e.contains("off | auto"), "{e}");
// and it travels in the settings object, with the default filling in when it is missing
assert_eq!(setting(json!({"path": "x.mp4", "settings": {"hardwareEncoding": "auto"}})).unwrap(), Auto);
let old: filmcraft_export::ExportSettings = serde_json::from_value(json!({"format": "h264"})).unwrap();
assert_eq!(old.hardware_encoding, Off);
}
#[test]
fn queue_exports_several_sequences_and_ranges() {
let mut s = demo();
@@ -377,6 +484,23 @@ fn queue_exports_several_sequences_and_ranges() {
assert_eq!(len("parts_2.wav"), 44 + 12_000 * 4, "0.25 s of 16-bit stereo");
}
#[test]
fn queue_folder_paths_expand_home_and_take_either_separator() {
let mut s = demo();
let first = s.state.active_sequence.unwrap();
let second = s.execute("file.newSequence", json!({"name": "Second Cut", "width": 640, "height": 360, "fps": 25})).unwrap()["sequence"].as_u64().unwrap();
let preset = "Waveform Audio 48 kHz 16-bit";
// a folder that does not exist yet, named by a trailing `\`: each sequence keeps its own name
s.execute("export.queue.add", json!({"preset": preset, "path": "not-yet-made\\", "sequences": [first.0, second]})).unwrap();
let paths: Vec<String> = queue(&mut s).iter().map(|i| i["path"].as_str().unwrap().to_string()).collect();
assert!(paths.iter().any(|p| p.ends_with("Second Cut.wav")), "{paths:?}");
if let Some(home) = crate::media_browser::std_home_dir() {
s.execute("export.queue.add", json!({"preset": preset, "path": "~/Exports/", "sequence": first.0})).unwrap();
let last = queue(&mut s).last().unwrap()["path"].as_str().unwrap().to_string();
assert!(last.starts_with(&home), "{last}");
}
}
#[test]
fn quick_export_uses_and_remembers_a_preset() {
let mut s = demo();
+53 -28
View File
@@ -26,7 +26,7 @@ use std::sync::Arc;
use std::sync::atomic::Ordering;
use filmcraft_export::presets::{DEFAULT_PRESET, preset_key};
use filmcraft_export::{ExportPreset, ExportSettings, Format};
use filmcraft_export::{ExportPreset, ExportSettings, Format, HardwareEncoding};
use filmcraft_project::{ItemId, Project};
use filmcraft_time::{FrameRate, Tick, TimeRange};
use serde::{Deserialize, Serialize};
@@ -187,18 +187,30 @@ pub fn settings_from_params(s: &Session, p: &Value, cmd: &str) -> Result<(Option
if let Some(v) = u64_p(p, "quality") {
settings.quality = v.min(100) as u8;
}
if let Some(v) = u64_p(p, "bitrateKbps") {
settings.bitrate_kbps = v as u32;
if let Some(v) = crate::commands::checked_u32_p(p, "bitrateKbps", cmd)? {
settings.bitrate_kbps = v;
settings.adaptive_bitrate = None;
}
if let Some(v) = u64_p(p, "maxBitrateKbps") {
settings.max_bitrate_kbps = Some(v as u32);
if let Some(v) = crate::commands::checked_u32_p(p, "maxBitrateKbps", cmd)? {
settings.max_bitrate_kbps = Some(v);
}
if let Some(v) = str_p(p, "bitrateMode") {
settings.bitrate_mode = serde_json::from_value(json!(v)).map_err(|_| bad(cmd, "bitrateMode: cbr | vbr1Pass | vbr2Pass"))?;
}
if let Some(v) = u64_p(p, "keyframeDistance") {
settings.keyframe_distance = Some(v as u32);
if let Some(v) = crate::commands::checked_u32_p(p, "keyframeDistance", cmd)? {
settings.keyframe_distance = Some(v);
}
if let Some(v) = p.get("hardwareEncoding") {
settings.hardware_encoding = match v {
Value::Bool(on) => {
if *on {
HardwareEncoding::Auto
} else {
HardwareEncoding::Off
}
}
other => serde_json::from_value(other.clone()).map_err(|_| bad(cmd, "hardwareEncoding: off | auto"))?,
};
}
if let Some(v) = bool_p(p, "burnCaptions") {
settings.burn_captions = v;
@@ -209,14 +221,22 @@ pub fn settings_from_params(s: &Session, p: &Value, cmd: &str) -> Result<(Option
if let Some(v) = str_p(p, "dnxProfile") {
settings.dnx_profile = v.to_string();
}
if let Some(v) = str_p(p, "apvProfile") {
settings.apv_profile = v.to_string();
}
if let Some(v) = str_p(p, "mxfVideoCodec") {
settings.mxf_video_codec = serde_json::from_value(json!(v)).map_err(|_| bad(cmd, "mxfVideoCodec: dnxhr | proRes | h264"))?;
}
if let Some(v) = bool_p(p, "sdr") {
settings.sdr = v;
}
if let (Some(w), Some(h)) = (u64_p(p, "width"), u64_p(p, "height")) {
settings.frame_size = Some((w as u32, h as u32));
if let Some(v) = bool_p(p, "alpha") {
settings.alpha = v;
}
match (crate::commands::checked_u32_p(p, "width", cmd)?, crate::commands::checked_u32_p(p, "height", cmd)?) {
(Some(w), Some(h)) => settings.frame_size = Some((w, h)),
(None, None) => {}
_ => return Err(bad(cmd, "provide both width and height")),
}
if let Some(v) = p.get("fps") {
settings.frame_rate = Some(parse_rate(v).ok_or_else(|| bad(cmd, "fps: a number (29.97) or \"num/den\""))?);
@@ -248,6 +268,7 @@ pub fn sequence_param(s: &Session, p: &Value, cmd: &str) -> Result<ItemId> {
/// without a `range` mean `custom`.
pub fn range_param(s: &Session, project: &Project, seq: ItemId, v: Option<&Value>, p: &Value, cmd: &str) -> Result<Option<TimeRange>> {
let q = project.sequence(seq).ok_or(EngineError::NoSequence)?;
q.check_bounds().map_err(|e| bad(cmd, e))?;
let fd = q.settings.frame_rate.frame_duration();
let whole = TimeRange::from_bounds(Tick::ZERO, q.duration().max(fd));
let mode = match v {
@@ -257,13 +278,9 @@ pub fn range_param(s: &Session, project: &Project, seq: ItemId, v: Option<&Value
Some(o @ Value::Object(_)) => return range_param(s, project, seq, o.get("mode").or(Some(&json!("custom"))), o, cmd),
Some(other) => return Err(bad(cmd, format!("range: unexpected {other}"))),
};
Ok(Some(match mode.as_str() {
let range = match mode.as_str() {
"entire" | "entiresequence" | "sequence" | "all" => whole,
"inout" | "sequenceinout" => {
let a = q.mark_in.unwrap_or(Tick::ZERO);
let b = q.mark_out.map(|o| o + fd).unwrap_or(q.duration());
TimeRange::from_bounds(a, b.max(a + fd))
}
"inout" | "sequenceinout" => filmcraft_export::export_range(project, seq, &ExportSettings::default()).map_err(|e| bad(cmd, e.to_string()))?,
"workarea" => q.work_area.ok_or_else(|| bad(cmd, "the sequence has no work area"))?,
"custom" => {
let a = time_p(s, p, "start").ok_or_else(|| bad(cmd, "custom range: need start (startTime / startSeconds / startFrame / startTimecode)"))?;
@@ -271,10 +288,13 @@ pub fn range_param(s: &Session, project: &Project, seq: ItemId, v: Option<&Value
if b <= a {
return Err(bad(cmd, "custom range: end must be after start"));
}
TimeRange::from_bounds(a, b)
let duration = b.0.checked_sub(a.0).ok_or_else(|| bad(cmd, "custom range: duration overflows"))?;
TimeRange::new(a, Tick(duration))
}
other => return Err(bad(cmd, format!("range `{other}`: entire | inOut | workArea | custom"))),
}))
};
filmcraft_export::validate_range(range).map_err(|e| bad(cmd, e.to_string()))?;
Ok(Some(range))
}
/// Where exports go when no path is given: next to the saved project, else ~/Movies, else the
@@ -287,7 +307,7 @@ pub fn default_export_dir(s: &Session) -> PathBuf {
// the web build's virtual file table: written files are offered as downloads
return PathBuf::from("/exports");
}
if let Ok(h) = std::env::var("HOME").or_else(|_| std::env::var("USERPROFILE")) {
if let Some(h) = crate::media_browser::std_home_dir() {
let movies = Path::new(&h).join("Movies");
return if movies.is_dir() { movies } else { PathBuf::from(h) };
}
@@ -310,11 +330,19 @@ fn with_extension(path: &str, settings: &ExportSettings) -> String {
}
}
fn expand_home(p: &str) -> String {
match (p.strip_prefix("~/"), std::env::var("HOME")) {
(Some(rest), Ok(h)) => format!("{h}/{rest}"),
/// `~/` expanded to the home directory (`USERPROFILE` on Windows). On Windows `/` becomes `\`:
/// after a `\\?\` prefix (what `canonicalize` returns) `/` is not a separator.
pub fn expand_home(p: &str) -> String {
let p = match (p.strip_prefix("~/"), crate::media_browser::std_home_dir()) {
(Some(rest), Some(h)) => format!("{h}/{rest}"),
_ => p.to_string(),
}
};
if cfg!(windows) { p.replace('/', "\\") } else { p }
}
/// `x` names a folder: it ends in a separator or already exists as one.
fn is_folder(x: &str) -> bool {
x.ends_with(['/', '\\']) || Path::new(x).is_dir()
}
/// Output path for `seq`: the `path` param (a file, or a directory ending in `/` or existing), else
@@ -322,7 +350,7 @@ fn expand_home(p: &str) -> String {
fn output_path(s: &Session, p: &Value, seq: ItemId, settings: &ExportSettings, suffix: Option<usize>) -> String {
let name = s.project.item(seq).map(|i| i.name.clone()).unwrap_or_else(|| "Sequence".into());
let base = match str_p(p, "path").map(expand_home) {
Some(x) if x.ends_with('/') || x.ends_with('\\') || Path::new(&x).is_dir() => Path::new(&x).join(file_safe(&name)).to_string_lossy().to_string(),
Some(x) if is_folder(&x) => Path::new(&x).join(file_safe(&name)).to_string_lossy().to_string(),
Some(x) => x,
None => default_export_dir(s).join(file_safe(&name)).to_string_lossy().to_string(),
};
@@ -513,6 +541,7 @@ fn item_json(s: &Session, it: &QueueItem) -> Value {
"range": it.settings.range,
"status": it.status,
"progress": progress,
"etaSeconds": job.filter(|_| it.status == QueueStatus::Encoding).and_then(|j| j.progress.eta()).map(|d| d.as_secs_f64()),
"statusText": status_text,
"job": it.job,
"error": it.error,
@@ -615,11 +644,7 @@ fn queue_add(s: &mut Session, p: &Value) -> Result<Value> {
};
st.range = range_param(s, &project, seq, *r, times, cmd)?;
// several ranges of one sequence get numbered names; several sequences their own names
let suffix = if many && (ranges.len() > 1 || str_p(p, "path").is_some_and(|x| !x.ends_with('/') && !Path::new(&expand_home(x)).is_dir())) {
Some(n)
} else {
None
};
let suffix = if many && (ranges.len() > 1 || str_p(p, "path").is_some_and(|x| !is_folder(&expand_home(x)))) { Some(n) } else { None };
st.path = output_path(s, p, seq, &st, suffix);
n += 1;
items.push((seq, st));
+103
View File
@@ -0,0 +1,103 @@
use crate::{Session, keyboard};
use filmcraft_media::{DemoScene, Generator, generators::GeneratorSource};
use filmcraft_project::{ItemId, Label};
use filmcraft_time::{FrameRate, TICKS_PER_SECOND, Tick};
use serde_json::json;
use std::sync::Arc;
fn session() -> (Session, ItemId) {
let mut s = Session::default();
s.execute("file.newSequence", json!({"width":128,"height":72,"fps":24})).unwrap();
let item = crate::demo::add_generator(
Arc::make_mut(&mut s.project),
&s.media,
GeneratorSource::new(Generator::Demo(DemoScene::Aurora), 128, 72, FrameRate::FPS_24, Tick(5 * TICKS_PER_SECOND)),
"Source clip",
Label::Iris,
None,
);
s.execute("timeline.place", json!({"item":item.0,"track":"V1"})).unwrap();
s.execute("source.open", json!({"item":item.0})).unwrap();
(s, item)
}
#[test]
fn frame_export_selects_the_correct_monitor_time_and_original_media() {
let (mut s, item) = session();
let dir = std::env::temp_dir().join(format!("fc-frame-original-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let proxy = dir.join("proxy.png");
image::RgbaImage::from_pixel(16, 9, image::Rgba([255, 0, 0, 255])).save(&proxy).unwrap();
Arc::make_mut(&mut s.project).item_mut(item).unwrap().as_media_mut().unwrap().proxy =
Some(filmcraft_project::MediaRef::File { path: proxy.to_string_lossy().into_owned() });
s.media.set_use_proxies(true);
s.state.source_playhead = FrameRate::FPS_24.tick_of(20);
s.set_playhead(FrameRate::FPS_24.tick_of(35));
let before = s.project.to_json();
let revision = s.revision;
let provider = s.media.full_res_provider(s.project.clone(), s.services.clone());
let expected_source = filmcraft_render::render_item(&s.project, item, s.state.source_playhead, 1.0, &provider).unwrap().to_rgba8();
let preview_provider = s.media.provider(s.project.clone(), s.services.clone());
let preview = filmcraft_render::render_item(&s.project, item, s.state.source_playhead, 1.0, &preview_provider).unwrap().to_rgba8();
assert_ne!(preview, expected_source, "the preview really uses the red proxy");
let seq = s.state.active_sequence.unwrap();
let expected_program =
filmcraft_render::render_sequence(&s.project, seq, s.playhead(), filmcraft_render::RenderOptions { captions: true, ..Default::default() }, &provider)
.to_rgba8();
assert_ne!(expected_source, expected_program);
for (target, time, expected) in [("source", s.state.source_playhead, expected_source), ("program", s.playhead(), expected_program)] {
let path = dir.join(format!("{target}.png"));
let result = s.execute("file.exportFrame", json!({"target":target,"path":path.to_string_lossy()})).unwrap();
assert_eq!(result["time"], json!(time.0));
let image = image::load_from_memory(&std::fs::read(path).unwrap()).unwrap().to_rgba8();
assert_eq!(image.dimensions(), (128, 72));
assert_eq!(image.into_raw(), expected);
}
assert_eq!(s.project.to_json(), before);
assert_eq!(s.revision, revision);
assert!(s.media.use_proxies());
drop(s);
std::fs::remove_dir_all(dir).unwrap();
}
#[test]
fn source_frame_export_works_without_an_active_sequence_and_rejects_bad_params() {
let (mut s, item) = session();
s.state.active_sequence = None;
let path = std::env::temp_dir().join(format!("fc-source-frame-{}.png", std::process::id()));
let r = s.execute("file.exportFrame", json!({"target":"source","path":path.to_string_lossy()})).unwrap();
assert_eq!(r["width"], 128);
assert!(s.execute("file.exportFrame", json!({"path":path.to_string_lossy()})).is_err());
for params in [
json!({"target":"wrong"}),
json!({"target":3}),
json!({"target":"source","time":-1}),
json!({"target":"source","time":"bad"}),
json!({"target":"source","item":0}),
json!({"target":"source","format":false}),
json!({"target":"source","import":"yes"}),
json!({"target":"source","path":""}),
] {
assert!(s.execute("file.exportFrame", params.clone()).is_err(), "{params}");
}
Arc::make_mut(&mut s.project).item_mut(item).unwrap().as_media_mut().unwrap().info.video = None;
assert!(s.execute("file.exportFrame", json!({"target":"source","path":path.to_string_lossy()})).is_err());
std::fs::remove_file(path).unwrap();
}
#[test]
fn frame_export_snapshots_are_independent_of_later_monitor_selection() {
let (mut s, item) = session();
s.state.source_playhead = FrameRate::FPS_24.tick_of(19);
let target = keyboard::frame_export_target(&s, &json!({"target":"source"})).unwrap();
s.state.source_item = None;
s.state.source_playhead = Tick::ZERO;
let path = std::env::temp_dir().join(format!("fc-captured-frame-{}.bmp", std::process::id()));
let r = s.execute("file.exportFrame", json!({"target":"source","item":item.0,"time":target.time.0,"format":"bmp","path":path.to_string_lossy()})).unwrap();
assert_eq!(r["time"], json!(target.time.0));
assert_eq!(&std::fs::read(&path).unwrap()[..2], b"BM");
std::fs::remove_file(path).unwrap();
let seq = s.state.active_sequence.unwrap();
Arc::make_mut(&mut s.project).sequence_mut(seq).unwrap().settings.frame_rate = FrameRate { num: 1, den: i64::MAX };
assert!(keyboard::frame_export_target(&s, &json!({})).is_err());
}
Loaded 100 of 454 files, more files were not shown because too many files have changed in this diff. Show more