Compare commits

...
Author SHA1 Message Date
Brandon Thomas 61ab6f5c7d Merge pull request #63 from storytold/pr/release-0.2.1
Release: FilmCraft v0.2.1
2026-10-06 08:01:17 -04:00
Brandon ThomasandClaude Opus 5.5 e659671bdf Release: FilmCraft v0.2.1
Fixes the Windows installer: the Start Menu shortcut and Add/Remove Programs icon were blank, and the shortcut description was wrong. Release builds no longer open a console window behind the app. Export can no longer panic (filmcraft-export never-crash).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:01:04 -04:00
Brandon Thomas bc1523a33f Merge pull request #62 from storytold/pr/never-crash-export
Never crash: filmcraft-export without expect()/unreachable!()
2026-10-06 07:58:18 -04:00
Brandon ThomasandClaude Opus 5.5 136424dc93 Never crash: filmcraft-export without expect()/unreachable!()
The five 'mux created' / 'audio' expects in the stepped exporter now return
an ExportError::Encode instead of panicking if the writer or audio pipeline
is missing, and the non-stepped dispatch returns Unsupported instead of
unreachable!(). The crate root gets the never-crash deny attribute, so every
FilmCraft production crate now carries it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 07:58:07 -04:00
Brandon Thomas 6bf2756cde Merge pull request #61 from storytold/pr/windows-packaging
Windows packaging: Start Menu/Add-Remove icon, no console window, correct description
2026-10-06 07:54:51 -04:00
Brandon ThomasandClaude Opus 5.5 c60d3f9c4c release.yml: explicit if for the wasm-bindgen install (shellcheck SC2015)
packaging-lint has failed on main since this step was added; same behaviour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 07:53:46 -04:00
Brandon ThomasandClaude Opus 5.5 7fee797ee7 Windows packaging: Start Menu/Add-Remove icon, no console window, correct description
The MSI Icon id had no .ico/.exe extension, which Windows Installer needs for
advertised shortcuts and ARPPRODUCTICON, so both showed a blank icon. Release
builds of the desktop app are now GUI-subsystem (no console window; the CLI stays
a console program), the shortcut/summary description names what FilmCraft does, and
packaging-lint rejects an Icon id without an extension. Found in printcraft#57.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 07:51:02 -04:00
Brandon Thomas 508a5a933f Merge pull request #28 from storybored379/pr/mcp-conventions
MCP: tool annotations, core tools, resources, export progress and cancellation
2026-10-06 07:27:58 -04:00
Brandon Thomas 5012fbd598 Merge pull request #19 from storybored379/pr/deterministic-export
Export: same file on every machine (fixed mux interleave, H.264 slices by frame size)
2026-10-06 07:27:44 -04:00
Brandon Thomas 1f7e4f802e Merge pull request #39 from storybored379/fix/ttml-timing
Captions: report invalid TTML timing parameters on import
2026-10-06 07:27:28 -04:00
Brandon ThomasandClaude Opus 5.5 5d1165d06b MCP: cancelled exports delete only their own files
remove_partial removed every '<stem>[digits].<any ext>' file in the output folder modified since the export started, so cancelling an export to trailer.mp4 also deleted trailer.aep or trailer2.psd if they were saved during the export. It now removes only the output file, numbered stills with the output's extension, and the .srt/.vtt caption sidecar. Regression test with decoy files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:54:24 -04:00
Brandon Thomas ada55eb625 Merge pull request #35 from storytold/pr/gpu-effects
GPU2: standard effects on the GPU compositor (#30)
2026-10-05 16:27:05 -04:00
Brandon ThomasandClaude Opus 5.5 adfd9a66de docs: point agents to the shared test corpora (#36)
Real-file test oracles live in storytold/photocraft-corpus, explained in
craftrules standards/test-corpora.md. Never commit large binary fixtures
here; fetch them pinned and sha256-verified.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 08:24:17 -04:00
Brandon ThomasandClaude Opus 5.5 e9a0c49d15 Docs: hardware decode on macOS and 31 GPU effects landed (#30)
ROADMAP honest assessment, Where we are lacking and Running now; README status and codec note. 35 crates (platform).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 07:50:45 -04:00
Brandon ThomasandClaude Opus 5.5 4aea06e2ae GPU2: standard effects on the GPU compositor (#30)
A media clip whose enabled standard effects all have a GPU implementation
plans as a GPU layer carrying them (render::plan::LayerFx): parameters are
evaluated on the CPU at the frame's time (render::gpufx::FxOp::eval, shared
with the CPU path so both clamp hostile values identically), the source is
drawn into an Rgba32Float working image at the size the CPU decodes it, the
effects run as WGSL compute passes (gpu::fx, fx.wgsl) with the CPU
reference's math, and the result is placed with Motion, opacity and blend
mode as before. Layers without effects keep the single draw.

GPU effects: Brightness & Contrast, ProcAmp, Tint, Black & White, Color
Balance, Leave Color, Change to Color, Color Pass, Color Replace, Channel
Mix, ASC CDL, Gamma Correction, Levels, Extract, Invert, Posterize, Alpha
Adjust, Gaussian / Camera / Directional Blur (+ legacy aliases), Sharpen,
Unsharp Mask, Crop, Edge Feather, Transform, H/V Flip, Mirror, Offset.
Masked effects, opacity masks, non-finite parameters and minifying
Transform/Crop zooms stay on the CPU. Devices without compute shaders run
the effects on the CPU inside the compositor.

Also fixes a debug-build overflow in Image::transformed for non-finite maps.
Parity tests (exact f32 checks and composited plans), a golden test that
plans effect chains as GPU layers, and the stack3-fx playback benchmark.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 07:50:20 -04:00
Brandon Thomas e9a01dbf38 Merge pull request #33 from storytold/pr/hw-decode-videotoolbox
HW1: platform crate + VideoToolbox H.264/HEVC hardware decode (#30)
2026-10-05 07:26:47 -04:00
Brandon Thomas 2e8df0541a Merge pull request #34 from storytold/pr/honest-assessment
Docs: honest parity assessment (ROADMAP, README, agent docs)
2026-10-05 06:32:58 -04:00
Brandon ThomasandClaude Opus 5.5 3d4e6d77bb Docs: honest parity assessment in ROADMAP, README and agent docs
ROADMAP now reports two numbers: the feature checklist (~87%, what exists)
and ready for real work (~50-60%, an honest estimate). Adds a by-dimension
assessment with evidence, a prioritised "Where we are lacking" list (hardware
acceleration, a measured parity tool, real-world media corpus, workflow
acceptance tests, Windows/Linux runtime CI, delivery codecs, plugins, AI, web),
and where we're going. Fixes stale rows (M5 effect count, Running now) and logs
2026-10-05.

README: the stale "45%" status replaced with both numbers and the biggest
gaps; the codec note now lists what imports today and what's missing.

docs/agents.md: agents read the honest assessment first, prefer its gaps,
and report progress as measured vs estimated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 06:04:09 -04:00
Brandon Thomas 98792afd1c Merge pull request #32 from storytold/pr/gpu-blend-modes
GPU1: blend modes on the GPU compositor (#30)
2026-10-05 05:28:35 -04:00
Brandon ThomasandClaude Opus 5.5 03f5439d9a HW1: plane views borrow the pixel-buffer lock guard (#30)
PlaneView's slices were tied to the pixel buffer's lifetime, not the lock's, so nothing stopped a later edit from reading plane memory after CVPixelBufferUnlockBaseAddress. They now borrow the Locked guard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 04:53:28 -04:00
Brandon ThomasandClaude Opus 5.5 c3ec0e85d3 HW1: platform crate + VideoToolbox H.264/HEVC hardware decode (#30)
- New crates/platform (filmcraft-platform, L5): OS media FFI only. The one
  crate allowed `unsafe` (own [lints] with unsafe_code = "deny",
  #[allow(unsafe_code)] only on the videotoolbox module, SAFETY comment on
  every block, safe Result-returning public API). AGENTS.md §0.3 names the
  exception; docs/adr/0001-platform-ffi.md records why and the rules.
- VideoToolbox decoder for avcC / hvcC, 8/10-bit 4:2:0 and 4:2:2: session from
  the sample entry's parameter sets with hardware required, asynchronous
  decode (2 in flight), biplanar CVPixelBuffer copied to planar Yuv8/Yuv16,
  reorder buffer of the stream's own depth, RASL skipped after a starting CRA.
  Output callback under catch_unwind.
- HybridDecoder: transparent mid-stream fallback to the software decoder
  (replays samples since the last restart point, drops already-returned
  pictures). New codecs::software_video_decoder (built-in factories only) and
  codecs::hw (switch, counters, NalStreamInfo shared with software decoders;
  vui_color / sar_par helpers now used by both).
- Settings ▸ Playback ▸ Hardware decoding: Auto / Off (engine applies it to
  the codecs switch); desktop app, CLI/MCP and bench call register().
- perf.stats decode.framesDecoded and decode.hardware counters.
- Tests: bit-exact parity with the software decoders (H.264 High, HEVC Main
  open GOP, Main 10; resets, reseeks, flushes), forced-failure fallback,
  mutation fuzz of samples and parameter sets, cross-platform fallback with a
  stand-in decoder, setting Off, HEVC 4:2:2 vs ffmpeg.
- Bench --hw auto|off; results in docs/performance.md (4K CPU/frame
  ~86-119 ms -> ~4 ms).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 04:53:28 -04:00
Brandon ThomasandClaude Opus 5.5 6102a3794a GPU1: blend modes on the GPU compositor (#30)
All 27 blend modes now composite on the GPU instead of sending the
whole frame to the CPU.

- render::plan: PlanLayer carries its Blend; plan_frame no longer falls
  back to a CPU frame for non-Normal modes. Media, graphic clips,
  multicam angles and CPU-rendered layer images (standard effects) keep
  their clip's mode; inside a transition the mode is Normal, as in the
  CPU reference. execute_cpu composites with each layer's mode.
- gpu: Normal keeps the single fixed-function pass. Dissolve also uses
  fixed-function "over": the CPU's 64-bit pixel hash is reproduced in
  WGSL on u32 pairs (pattern bit-exact). The other 25 modes copy the
  accumulator region under the layer's quad into a backdrop texture and
  composite in fs_blend with the CPU math (sRGB-encoded straight colour,
  W3C formulas, back to linear, same premultiplied mix and clamping).
- Tests: per-mode GPU-vs-CPU parity (gpu crate: overlapping transformed
  layers, partial opacity/alpha, empty/partial backdrop; golden: every
  mode through plan_frame incl. an effect layer, blended clips in a
  cross dissolve), exact-value edge cases (black/white, Divide by 0,
  Dodge/Burn at 0 and 1, zero alpha) within 4.9e-4, Dissolve pattern
  exact, off-output layers, quad bounds.
- Bench: new stack3-blend playback scenario (V2 Screen, V3 Overlay);
  CPU ms/frame 151-214 -> 86-89, Full 19/173 and 138/54 -> 191/1, 192/0.
- Docs: architecture §5, gpu crate docs, testing, performance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 03:25:16 -04:00
Brandon Thomas 4f06306e45 Merge pull request #31 from storytold/pr/release-0.2.0
Release: FilmCraft v0.2.0
2026-10-05 02:45:48 -04:00
Brandon ThomasandClaude Opus 5.5 798b1591ba Release: FilmCraft v0.2.0
Bump the workspace version 0.1.1 -> 0.2.0 (cargo xtask version set 0.2.0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 02:45:36 -04:00
storybored379andOpenAI Codex 9c06722119 Captions: report invalid TTML timing parameters on import
Co-Authored-By: OpenAI Codex <noreply@openai.com>
2026-10-04 21:48:25 -07:00
storybored379andClaude Opus 5.5 cad16825e3 MCP: progress notifications and cancellation for blocking exports
command_run file.exportMedia with wait: true (headless) runs as the
background export job and uses the MCP progress and cancellation utilities:

- with _meta.progressToken, notifications/progress reports frames done /
  total with the job status, at most every 100 ms (none without a token);
- notifications/cancelled stops the job at the next batch (jobs.cancel),
  deletes the partial output and sends no response;
- the session lock is only held for short polls, so ping, doc_inspect,
  jobs.list etc. are answered while it encodes.

Without wait, and in bridge mode, nothing changes. Test
export_progress_and_cancel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 13:51:38 -07:00
storybored379andClaude Opus 5.5 8ee7658303 MCP: filmcraft://document and filmcraft://commands resources, with MCP 2026-07-28 cache hints
- resources/list and resources/read serve the project (tree + active
  sequence, as doc_inspect) and the command catalog (as command_list) as
  JSON, for clients that attach context from resources.
- Results are built with rmcp's types, so resultType follows the negotiated
  revision; for 2026-07-28 clients (current Claude Code) they also carry
  ttlMs and cacheScope, like the generated tools/list. The resource list is
  fixed (10 min), document reads are never cached (0).
- get_info now declares the resources capability; the instructions text
  moves to a const, unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 13:51:28 -07:00
storybored379andClaude Opus 5.5 0f70015b93 MCP: tool titles and annotations, command_batch / doc_inspect / render_preview, strict arguments
- Every tool has a title and readOnly / destructive / idempotent / openWorld
  hints, set in its #[tool] attribute, so clients can run the read-only
  tools without asking and ask before edits.
- New tools: command_batch (several commands, stop at the first error by
  default), doc_inspect (project tree + active sequence in one call) and
  render_preview (= render_frame).
- An unknown tool argument is a -32602 error naming it and the accepted
  ones, instead of being ignored.
- A line that is not JSON gets a -32700 reply (id null) instead of being
  dropped without an answer: serve_io filters lines in front of rmcp.
- A panicking command no longer poisons the session lock for the rest of the
  session ("session lock poisoned"); it is reported as "internal error: …"
  (AGENTS.md §0.6).
- docs/agents.md § Conventions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 13:51:21 -07:00
Brandon Thomas 694e40546c Merge pull request #27 from storytold/pr/never-crash-clean-crates
Never crash: deny the panicking lints in the already-clean crates and apps
2026-10-04 16:19:27 -04:00
Brandon Thomas 72edec524d Merge pull request #26 from storytold/pr/never-crash-ui
ui-egui: never crash (panels without expect/unreachable)
2026-10-04 16:19:08 -04:00
Brandon Thomas bc57ebee66 Merge pull request #25 from storytold/pr/never-crash-guard
engine: last-resort guard around commands and background jobs
2026-10-04 16:18:49 -04:00
Brandon Thomas 0171afa89d Merge pull request #24 from storytold/pr/never-crash-engine
engine, edit: never crash (commands without expect/unwrap/panic)
2026-10-04 16:18:29 -04:00
Brandon Thomas b081fd20df Merge pull request #21 from storytold/pr/never-crash-export-render
render, gpu, cli: never crash (no expect/unreachable, deny the panicking lints)
2026-10-04 16:18:11 -04:00
Brandon Thomas 1fb70e6841 Merge pull request #22 from storytold/pr/never-crash-h264enc
h264enc: never crash (encode returns Result; no unwrap/expect/unreachable)
2026-10-04 16:17:52 -04:00
Brandon Thomas 9cf892a7b0 Merge pull request #20 from storytold/pr/never-crash-importers
Importers: never crash (interchange, captions, project, text, automation)
2026-10-04 16:17:33 -04:00
Brandon Thomas de57e91c80 Merge pull request #18 from storytold/pr/never-crash-codecs
codecs: never crash (damaged MP4/TS headers; no expect/unreachable)
2026-10-04 16:17:14 -04:00
Brandon Thomas 28563e8030 Merge pull request #16 from storytold/pr/never-crash-codecs-small
Codecs: never crash (ac3, audio-dsp, opus, mpeg2v, prores, dnx)
2026-10-04 16:16:55 -04:00
Brandon Thomas e62fabb1ea Merge pull request #15 from storytold/pr/never-crash-containers
Containers: never crash (mpegts, ogg, mxf, matroska)
2026-10-04 16:16:36 -04:00
Brandon Thomas 08ebb8fd46 Merge pull request #14 from storytold/pr/never-crash-av1
av1: never crash (no expect, deny the panicking lints)
2026-10-04 16:16:18 -04:00
Brandon Thomas a366c41d7d Merge pull request #13 from storytold/pr/never-crash-hevc
hevc: never crash (no unwrap/expect, deny the panicking lints)
2026-10-04 16:15:58 -04:00
Brandon Thomas 4436d4d887 Merge pull request #12 from storytold/pr/never-crash-h264
h264: never crash (no unwrap/expect, deny the panicking lints)
2026-10-04 16:15:39 -04:00
Brandon Thomas 51e5adc627 Merge pull request #11 from storytold/pr/never-crash-vp9
vp9: never crash (no unwrap/expect/panic, deny them)
2026-10-04 16:15:19 -04:00
Brandon Thomas 9fdb8598a3 Merge pull request #17 from storytold/pr/never-crash-time
time: never crash (zero rates and timebases, huge timecodes)
2026-10-04 16:15:00 -04:00
Brandon ThomasandClaude Opus 5.5 b16e0889f3 Never crash: deny the panicking lints in the already-clean crates and apps
aac, bitstream, cfb, color, format, frame, geom, isobmff, media, scopes, speech, the desktop
app and the web app have no production unwrap/expect/panic/unreachable (the web app's two
`expect`s — frame-cache eviction and chunked file reads — become `let … else`); each now
carries `#![deny(clippy::unwrap_used, …)]` outside tests so it stays that way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 16:08:23 -04:00
Brandon ThomasandClaude Opus 5.5 d40a371724 ui-egui: never crash — panels without expect/unreachable
- Timeline, monitors, Reference, Effect Controls and Export mode: a missing active sequence,
  track or clip (stale ids from a damaged project, or a clip deleted mid-drag) shows the empty
  state or skips the row instead of `expect`.
- Frame queue pops without `expect`; Graphics text editing, clip dialogs, Lumetri panel and
  Project rename without `expect`/`unreachable!`.
- The deliberate fault-injection panics (UI pass and frame jobs, used to test the guard) go
  through one documented `crash::injected_fault`.
- `#![deny(clippy::unwrap_used, …)]` outside tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 16:07:12 -04:00
Brandon ThomasandClaude Opus 5.5 3a392ae8da Clippy 1.99: fix new lints so cargo xtask ci is green again
Rust/clippy 1.99 added `chunks_exact_to_as_chunks` and tightened
`needless_late_init` and `drain_collect`, failing `clippy -D warnings`
on main.

- `chunks_exact(N)` / `chunks_exact_mut(N)` with a constant N become
  `as_chunks::<N>()` / `as_chunks_mut::<N>()` (fixed-size arrays, so
  several byte reads drop their per-element indexing). relink's
  fingerprint uses the `(chunks, remainder)` split; its hash output is
  unchanged.
- Late-initialised `let`s in ac3 bit allocation and opus CELT
  bands/rate become `let x = if/match …`.
- `dpb.drain(..).collect()` becomes `std::mem::take(&mut dpb)`.

No behaviour change; `cargo xtask ci` passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 15:55:23 -04:00
Brandon ThomasandClaude Opus 5.5 75ed688e09 engine: last-resort guard around commands and background jobs
The UI already turns a panic in the UI pass into an error window (ui-egui `crash` hook +
`catch_unwind` around the frame), and frame workers survive panics. Two gaps remained:
- commands run headless (CLI, MCP over stdio) or over the control channel had no guard: a panic
  in one command ended the process and the unsaved session;
- background jobs (export, render previews/audio, proxies, Project Manager, mask tracking,
  scene detection) ran their body unguarded: a panic killed the thread and left the job
  "running" forever.

`Session::execute` now runs the command under `catch_unwind` and returns
`EngineError::Other("internal error in `<id>` …")`; every job body is wrapped by
`export_tools::guard_job`, which marks the job finished with an error. Regression test
`export_tools::guard_tests::panicking_job_is_reported_as_failed`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 15:23:18 -04:00
Brandon ThomasandClaude Opus 5.5 31ae72f015 engine, edit: never crash — commands without expect/unwrap/panic
- Commands: place/nest/sequence settings/import/multicam/subclip/audio channels/extract audio,
  graphics reset duration, dynamic trim and Make Subsequence return errors instead of `expect`
  when an item, sequence or track is missing (stale ids from a damaged project or a racing
  agent); File ▸ Import detects caption/interchange documents without a second `expect`.
- `find_spec` (graphics) returns `UnknownCommand` instead of `expect`.
- Built-in effect lookups (presets, demo project) fall back to a bare instance instead of
  `panic!`/`expect`; the demo project falls back to an empty one.
- Multicam record base, shortcut presets and field parsing without `expect`.
- edit: caption merge/move and transcript caption blocks without `expect`.
- Both crates deny unwrap/expect/panic/unimplemented/todo/unreachable outside tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 15:02:01 -04:00
Brandon ThomasandClaude Opus 5.5 b05833ef67 h264enc: never crash — no unwrap/expect/unreachable, encode returns Result
- `Encoder::encode` returns `Result` (it was `try_encode(..).expect(..)`, panicking on short
  planes); tests and the example unwrap explicitly.
- P/B macroblock analysis without a reference codes intra instead of `expect`/`unwrap`;
  spatial direct without a list-1 picture predicts zero motion.
- Rate control prior, DPB sliding window, DTS queue, reconstruction buffers, CAVLC writer,
  SAD rows and forward 8x8 transforms without `unwrap`/`expect`; skip-macroblock arms in
  `write_mb` debug-assert instead of `unreachable!`.
- `#![deny(clippy::unwrap_used, …)]` outside tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 14:14:06 -04:00
Brandon ThomasandClaude Opus 5.5 5628fc55bc render, gpu, cli: never crash — no expect/unreachable, deny the panicking lints
- gpu: `composite` returns its output view by value (a cheap handle clone) taken from the
  render-target setup, so the accumulation/output views no longer `expect` their slots.
- render: lightning bolt subdivision, tracker pyramid and the Lumetri preset instance without
  `expect`.
- cli: dotted parameter insertion without `expect`; the render/mcp backend arms report a usage
  error instead of `unreachable!`.
- Each crate denies unwrap/expect/panic/unimplemented/todo/unreachable outside tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 14:04:42 -04:00
Brandon ThomasandClaude Opus 5.5 f0e6b12819 Importers: never crash — interchange, captions, project, text, automation
- interchange: exporters (FCP7 XML, FCPXML, OTIO) skip a missing nested sequence or media with a
  report warning instead of `expect`/`unwrap`; EDL event grouping and M2 lines, the XML writer
  and the AAF/OMF dispatch arms have no panics. The WAVE header saturates instead of overflowing
  for >4 GiB descriptors (found by fuzzing: a mutated EDL exported to OMF), regression test
  `wav::tests::huge_wav_header_saturates`.
- captions: EBU STL frame rate read with `get()`; MCC rows, TTML text and the STL dispatch arm
  without `expect`/`unreachable!`.
- project: bin moves, 5.1 pan lanes and text/shape layer constructors without `expect`.
- text: font resolution and line layout without `expect` (the bundled-face lookup keeps one
  documented `#[allow(clippy::expect_used)]`: the faces are compiled in).
- automation: the MCP bridge returns an error instead of `expect` when not connected.
- Each crate denies unwrap/expect/panic/unimplemented/todo/unreachable outside tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 12:38:13 -04:00
Brandon ThomasandClaude Opus 5.5 6aa3c37342 codecs: never crash — damaged MP4/TS headers, no expect/unreachable
Mutation fuzzing (mutated ffmpeg-made files through `open_bytes`) found three import crashes:
- an MP4/MOV track with an empty sample-description list: `entries[0]` index out of bounds
  (video and audio paths);
- an MP4/MOV track with timescale 0: division by zero in the duration;
- an MPEG-TS stream whose PTS span gave a ~0 fps rate: division by zero in `tick_of`.

Tracks now need samples, a sample description and a timescale to count as playable; frame rates
that round to zero fall back to the default; the TS/PS rate choice is a small function that never
returns a zero rate. Regression tests: `mp4::tests::damaged_track_headers_never_panic`,
`mpeg::tests::damaged_timestamps_never_give_a_zero_rate`.

Also: GOP cache, MKV/MP4/Ogg/MPEG audio paths, MXF and VP9 conversion without
`expect`/`unreachable!`; the crate denies the panicking lints outside tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 12:20:30 -04:00
Brandon ThomasandClaude Opus 5.5 0f678aa105 h264: dequantisation wraps instead of overflowing on huge levels
Mutation fuzzing found a corrupt H.264 transport stream whose chroma DC levels overflowed
`chroma_dc_dequant_420` ("attempt to multiply with overflow" in debug builds, a decoder worker
panic). The 4x4 / 8x8 / DC dequantisers now use wrapping arithmetic, which is what release
builds always computed. Regression test `huge_levels_dequantise_without_overflow_panics`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:56:22 -04:00
Brandon ThomasandClaude Opus 5.5 953f0aca8b time: never crash — total conversions for zero rates, saturating timecodes
Mutation fuzzing found two crashes on import: an MP4 track with timescale 0 panicked in
`Tick::from_rational` (division by zero) and an MPEG-TS stream whose timestamps gave a zero
frame rate panicked in `FrameRate::tick_of`. A FrameRate with a zero num/den can also come from a
project file (it's serde).

- `Tick::{from_units, from_rational, to_rational_floor, mul_ratio}` return 0 for a zero
  denominator instead of dividing by zero.
- `FrameRate::sane()`: an invalid (zero / negative) rate counts as the default rate in
  `frame_duration`, `frame_at`, `tick_of` and `timecode_base`.
- `Tick::clamp` no longer panics when the bounds cross.
- Timecode parsing and drop-frame conversion use saturating math (huge typed or agent-sent
  timecodes overflowed).
- Regression test `hostile_rates_and_timecodes_never_panic`.
- `#![deny(clippy::unwrap_used, …)]` outside tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:51:31 -04:00
Brandon ThomasandClaude Opus 5.5 46b286edaf Codecs: never crash — ac3, audio-dsp, opus, mpeg2v, prores, dnx
- ac3: LFE channel taken without `expect`.
- audio-dsp: the channel matrix match is exhaustive (no `unreachable!`); an unknown internal
  parameter reads as 0 (debug-asserted) instead of `panic!`.
- opus: dual-stereo bands and SILK frames take the optional buffer / range decoder with `if let`.
- mpeg2v: 64-bit peek through `get()` + `first_chunk`; the literal VLC table parser no longer
  panics (debug-asserted).
- prores, dnx: IDCT rows through `as_chunks`; ProRes block store and alpha slices without
  unwrap/expect; DNx header timecode through `get()`.
- dnx: `Encoder::new` returns `Result` instead of panicking on a raster the profile can't code
  (e.g. 0x0), with a regression test; missing CID table is an error.
- Each crate denies unwrap/expect/panic/unimplemented/todo/unreachable outside tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:43:19 -04:00
Brandon ThomasandClaude Opus 5.5 fa40a3c030 Containers: never crash — mpegts, ogg, mxf, matroska without expect/unreachable
- mpegts: a PES unit is built before it is stored (no `as_mut().expect`); stream maps are
  drained with `let … else` instead of `remove().expect`.
- ogg: completing a packet and reading the page granule no longer expect.
- mxf writer: the picture descriptor uses `let … else` on the picture configuration.
- matroska: the `Lacing::None` arm returns the single frame instead of `unreachable!`.
- Each crate denies unwrap/expect/panic/unimplemented/todo/unreachable outside tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:37:58 -04:00
Brandon ThomasandClaude Opus 5.5 7190fb9009 av1: never crash — no expect in the decoder, deny the panicking lints
- CDEF and inter-prediction filter rows: fixed-size views through `get()` + `first_chunk(_mut)`
  instead of slice + `try_into().expect()`.
- A missing reference frame predicts from the current frame instead of `expect`.
- Frame slots, output and in-flight queues, scalable-layer selection: no `expect`.
- `#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented,
  clippy::todo, clippy::unreachable)]` outside tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:35:48 -04:00
storybored379andClaude Opus 5.5 a55d20c454 export: same file on every machine (fixed mux interleave, H.264 slices by frame size)
Exporting the same project twice could give different files, because two
export parameters followed the number of cores the process sees:

- The muxer got one chunk per render batch, and a batch is one frame per
  rayon thread. The chunk layout (stsc/stco, PCM packet sizes) changed with
  the core count; the samples themselves did not.
- The H.264 encoder used one slice per core (capped at one per four MB rows),
  so the bitstream and every decoded picture changed with the core count.

On a host whose CPU quota changes (containers, power limits), two exports in a
row differed. Now packets are muxed in fixed groups of 16 output frames
(INTERLEAVE), each followed by its audio, whatever the batch size, and H.264
exports use one slice per four macroblock rows. Hosts with 17 or more threads
already used these values, so their 1080p output does not change.

Tests: determinism_tests (fail on main): same bytes for ProRes, H.264 and
Motion-JPEG at 2/3/5/16 cores and 1/7 frames per step; 64 slices for a
4096-line H.264 export.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 08:32:49 -07:00
Brandon ThomasandClaude Opus 5.5 970653628f hevc: never crash — no expect/unwrap in the decoder, deny them
- SAO runs and 8-tap filter rows: fixed-size lanes through `get()` + `first_chunk(_mut)`
  instead of slice + `try_into().expect()`.
- Decoder: a slice segment with no started picture is an error; the output queue pop no longer
  expects.
- Weighted prediction without a parsed table falls back to the default (unweighted) prediction
  instead of `expect`.
- Tile boundaries accumulate without `last().unwrap()`.
- `#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented,
  clippy::todo, clippy::unreachable)]` outside tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:31:02 -04:00
Brandon ThomasandClaude Opus 5.5 a87ba26844 h264: never crash — no unwrap/expect in the decoder, deny them
- I_PCM samples, deblocking lines, transform rows, block copies: fixed-size views through
  `get()` + `first_chunk(_mut)` instead of slice + `try_into().unwrap()`. A truncated I_PCM
  macroblock is `Error::Invalid` from the read itself.
- Decoder: a missing parameter set or a slice without a started picture is an error, not an
  `expect`; the output queue pop no longer expects.
- CAVLC table builder parses its literal codes without `from_str_radix().unwrap()`.
- `#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented,
  clippy::todo, clippy::unreachable)]` outside tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:30:55 -04:00
Brandon ThomasandClaude Opus 5.5 d499b0c20e vp9: never crash — no unwrap/expect/panic in the decoder, deny them
- Transforms, 8-tap filters, probability tables: fixed-size views through `first_chunk(_mut)`
  instead of slice + `try_into().unwrap()/expect()` (same code on the happy path).
- Tile sizes read with `get()`; a short tile header is `Error::Invalid`.
- Buffer pool tolerates a poisoned lock.
- A failed post-processing thread spawn no longer panics: the job reaches the thread over a
  channel, so it is still in hand and runs inline.
- Without the `threads` feature an unpublished reference band reads as empty instead of panicking.
- `#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented,
  clippy::todo, clippy::unreachable)]` outside tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:15:05 -04:00
Brandon Thomas 7b96f22684 Merge pull request #10 from storytold/pr/never-crash-steering
Rules: never crash (AGENTS.md §0, clippy.toml, forbid unsafe)
2026-10-04 11:14:08 -04:00
Brandon ThomasandClaude Opus 5.5 d0048b0256 Merge PR #8 from storybored379
Timeline: Slide moves linked audio with the video

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:50:15 -04:00
Brandon ThomasandClaude Opus 5.5 ff62f8af27 Merge PR #7 from storybored379
Timeline: transitions stay on their cuts after ripple trim, roll, slide and ripple speed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:50:14 -04:00
Brandon ThomasandClaude Opus 5.5 3c5e913879 Merge PR #6 from storybored379
Export: start/end times without a range mean a custom range

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:50:13 -04:00
Brandon ThomasandClaude Opus 5.5 7dd3ee4d77 Merge PR #5 from storybored379
Media: AIFF files import (enable symphonia's aiff reader)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:50:12 -04:00
storybored379andClaude Opus 5.5 a80769c096 Timeline: Slide moves linked audio with the video
timeline.slide (the Slide tool drag and the MCP/control-channel command)
only moved the clip it was given, so sliding a linked video clip left its
audio partner behind: the audio kept its old start and source In, a
visible A/V desync of the slide distance. Slip, Trim, Rate Stretch and the
keyboard slide (timeline.slideLeft/Right) already include linked partners.

Slide every linked partner (with_links) by a common delta, clamped across
the partners like timeline.trim does, so a shorter handle on one track
cannot slide the pair apart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 09:47:34 -04:00
storybored379andClaude Opus 5.5 d9a35e7130 Timeline: transitions stay on their cuts after ripple trim, roll, slide and ripple speed
Ripple trims (trim in ripple mode and ripple_trim_group), rolling edits,
slides and Speed/Duration with ripple moved clips but left every
transition at its old timeline position. After e.g. a 1 s ripple trim the
dip to black between the next two clips played 1 s after their cut,
dissolving into the outgoing clip's handle mid-shot; a rolled cut kept its
dissolve where the cut used to be and then hard-cut later.

After these edits each transition now shifts by as much as its anchor moved
(the incoming clip's start, or the outgoing clip's end for a fade to
nothing), keeping its alignment on the edit point. Insert, extract, ripple
delete and close gap already shift transitions and are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 01:17:34 -07:00
storybored379andClaude Opus 5.5 8776168a63 Export: start/end times without a range mean a custom range
file.exportMedia (and export.resolve / export.quick / export.queue.add)
document startSeconds/endSeconds next to range, but bare start/end times
without "range": "custom" were silently ignored and the whole sequence
(or In/Out) was exported. Start or end times now imply a custom range; a
start without an end is a clean error from the custom-range check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 01:17:34 -07:00
storybored379andClaude Opus 5.5 320b6eede8 Media: AIFF files import (enable symphonia's aiff reader)
AIFF is listed as an importable audio format and FilmCraft exports AIFF,
but the symphonia build had only the wav reader enabled, so every .aif /
.aiff import (including FilmCraft's own AIFF exports) failed with
"unsupported media: end of stream". Enable symphonia's aiff reader (the
same MPL-2.0 symphonia-format-riff crate already used for WAV; Cargo.lock
is unchanged). The new test decodes a synthetic 16-bit stereo AIFF.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 01:17:34 -07:00
246 changed files with 8227 additions and 1216 deletions

No files matched your search

+4
View File
@@ -47,6 +47,10 @@ jobs:
- name: XML, desktop entry, AppStream, Flatpak manifest
run: |
xmllint --noout packaging/windows/filmcraft.wxs packaging/macos/*.plist* packaging/linux/*.xml*
# Windows Installer needs Icon ids ending in .ico or .exe, or shortcuts show a blank icon.
if grep -oE '<Icon Id="[^"]*"' packaging/windows/filmcraft.wxs | grep -vE '\.(ico|exe)"$'; then
echo "::error::Icon Id must end in .ico or .exe"; exit 1
fi
desktop-file-validate packaging/linux/ai.storyteller.filmcraft.desktop
sed -e 's/@VERSION@/0.0.0/' -e 's/@DATE@/2026-01-01/' packaging/linux/ai.storyteller.filmcraft.metainfo.xml.in >"$RUNNER_TEMP/ai.storyteller.filmcraft.metainfo.xml"
appstreamcli validate --no-net --explain "$RUNNER_TEMP/ai.storyteller.filmcraft.metainfo.xml"
+3 -2
View File
@@ -221,8 +221,9 @@ jobs:
- name: Install wasm-bindgen CLI
run: |
# Must match the wasm-bindgen crate version exactly (WASM_BINDGEN in xtask/src/main.rs).
command -v wasm-bindgen >/dev/null && wasm-bindgen --version | grep -q "$WASM_BINDGEN_VERSION" \
|| cargo install wasm-bindgen-cli --version "$WASM_BINDGEN_VERSION" --locked
if ! { command -v wasm-bindgen >/dev/null && wasm-bindgen --version | grep -q "$WASM_BINDGEN_VERSION"; }; then
cargo install wasm-bindgen-cli --version "$WASM_BINDGEN_VERSION" --locked
fi
wasm-bindgen --version
- name: Apply version override
if: inputs.version != ''
+12 -1
View File
@@ -19,7 +19,12 @@ panic path, and fix a crash before building on top of it. The cross-app standard
fallback that is truly correct (never one that silently corrupts a project). An unfinished feature
returns an "unsupported" error. The only exception is a call that is provably infallible from local
code alone (a literal that always parses): a single-item `#[allow(clippy::expect_used)]` with the reason.
3. **No `unsafe`** (`unsafe_code = "forbid"` for the workspace).
3. **No `unsafe`** (`unsafe_code = "forbid"` for the workspace). The single exception is
`crates/platform`, which needs `unsafe` to call the operating system's media APIs (hardware
video decoding through VideoToolbox; later encoding and other OSes) and does nothing else. It
uses `unsafe_code = "deny"` with `#[allow(unsafe_code)]` only on its FFI modules, a
`// SAFETY:` comment on every `unsafe` block, a safe `Result`-returning public API, and the
pure-Rust decoder as the tested fallback ([ADR 0001](docs/adr/0001-platform-ffi.md)).
4. **Every input-derived number is hostile.** Media files, project and interchange files, presets, fonts,
CLI / MCP / control-channel parameters and UI state are untrusted. Index and slice with `get()` (or
validate bounds once, up front, for a hot loop); slice strings only at char boundaries; use
@@ -115,6 +120,12 @@ See [§0](#0-never-crash). It outranks every other engineering rule.
See `CLAUDE.md`: pure Rust, dependency layering (`cargo xtask layers`), exact `Tick` time, everything is
a command, everything is agent-drivable, and the quality gates (`cargo xtask ci`) before every commit.
Shared real-file test corpora (Photoshop-authored PSDs, etc.) live in
[`storytold/photocraft-corpus`](https://github.com/storytold/photocraft-corpus), explained in
[craftrules `standards/test-corpora.md`](https://github.com/storytold/craftrules/blob/main/standards/test-corpora.md).
Never commit large binary fixtures to this repo; fetch them pinned by commit and sha256-verified,
as PhotoCraft does with `cargo xtask corpus`.
## See also
- [CONTRIBUTING.md](CONTRIBUTING.md) and [docs/contributing.md](docs/contributing.md): setup, gates, commits, how to add things
+1 -1
View File
@@ -12,7 +12,7 @@ FilmCraft is a clean-room, open-source, pure-Rust non-linear video editor target
**Read [`AGENTS.md`](AGENTS.md) first; its rules override everything here.**
- **Never crash** ([`AGENTS.md`](AGENTS.md) §0, standard: [`craftrules/standards/never-crash.md`](https://github.com/storytold/craftrules/blob/main/standards/never-crash.md)). People depend on FilmCraft; a crash loses their work, so this outranks feature work. No `unwrap`/`expect`/`panic!`/`unreachable!`/`todo!`/`unimplemented!` or `unsafe` outside tests: return `Result<T, E>` and propagate with `?` (`ok_or`, `let … else`, `if let`). Treat every number from media, project files, presets or CLI/MCP/control params as hostile: `get()` instead of `[i]`, checked/saturating math, no division by possibly-zero values, capped allocations, bounded recursion. `lock().unwrap_or_else(PoisonError::into_inner)`; background jobs run under `catch_unwind`. The panic hook + UI `catch_unwind` (`crates/ui-egui/src/crash.rs`) are the last-resort guard, not a licence. Every crash fix gets a regression test that panicked before. In particular (§1): never use Adobe iconography, images or any other Adobe asset. Every asset must be openly licensed (OSS / public domain / Creative Commons, or original work by a contributor) and must have a `<file>.attribution` sidecar plus an entry in `ATTRIBUTION.md`. `cargo xtask assets` enforces this.
- **Clean-room.** Never read/disassemble anything inside Adobe app bundles (names/listings only). Never copy Adobe icons, presets, LUTs, fonts. Never copy GPL/LGPL/AGPL code (FFmpeg, x264, x265, MLT, Kdenlive, Shotcut, Olive, LAME…). ffmpeg/ffprobe run only as external test oracles / fixture generators — never linked or shipped.
- **Pure Rust** in the product. OS media APIs only via Rust bindings in `crates/platform` (planned; not created yet), optional, behind traits.
- **Pure Rust** in the product. OS media APIs only via Rust bindings in `crates/platform` (hardware decoding; the one crate allowed `unsafe`, see `AGENTS.md` §0.3 and `docs/adr/0001-platform-ffi.md`), optional, behind traits, with our decoders as the fallback.
- **Layering** ([`docs/architecture.md`](docs/architecture.md) §1, enforced by `cargo xtask layers`): nothing below L5 depends on egui/eframe/winit/rfd/cpal. L0 codec/container crates depend only on `filmcraft-bitstream`.
- **Exact time:** all time is `filmcraft_time::Tick` (254 016 000 000/s). Never use f64 seconds for edit math.
- **Everything is a command** (`crates/engine`): id, label, menu path, shortcut, params, enabled(), run(). UI, CLI, control channel and MCP all dispatch by id.
Generated
+116 -45
View File
@@ -1655,13 +1655,14 @@ dependencies = [
[[package]]
name = "filmcraft"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"chrono",
"cpal",
"eframe",
"egui",
"filmcraft-engine",
"filmcraft-platform",
"filmcraft-ui-egui",
"log",
"muda",
@@ -1676,7 +1677,7 @@ dependencies = [
[[package]]
name = "filmcraft-aac"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1686,7 +1687,7 @@ dependencies = [
[[package]]
name = "filmcraft-ac3"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-testkit",
"thiserror 2.0.21",
@@ -1694,14 +1695,14 @@ dependencies = [
[[package]]
name = "filmcraft-audio-dsp"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-testkit",
]
[[package]]
name = "filmcraft-automation"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"base64 0.22.1",
"filmcraft-engine",
@@ -1717,7 +1718,7 @@ dependencies = [
[[package]]
name = "filmcraft-av1"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1728,14 +1729,14 @@ dependencies = [
[[package]]
name = "filmcraft-bitstream"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"proptest",
]
[[package]]
name = "filmcraft-captions"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-color",
"filmcraft-project",
@@ -1748,11 +1749,11 @@ dependencies = [
[[package]]
name = "filmcraft-cfb"
version = "0.1.1"
version = "0.2.1"
[[package]]
name = "filmcraft-cli"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-automation",
"filmcraft-codecs",
@@ -1761,6 +1762,7 @@ dependencies = [
"filmcraft-mpegts",
"filmcraft-mxf",
"filmcraft-ogg",
"filmcraft-platform",
"filmcraft-time",
"serde_json",
"tokio",
@@ -1768,7 +1770,7 @@ dependencies = [
[[package]]
name = "filmcraft-codecs"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-aac",
"filmcraft-ac3",
@@ -1800,7 +1802,7 @@ dependencies = [
[[package]]
name = "filmcraft-color"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"proptest",
"serde",
@@ -1808,7 +1810,7 @@ dependencies = [
[[package]]
name = "filmcraft-dnx"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1818,7 +1820,7 @@ dependencies = [
[[package]]
name = "filmcraft-edit"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-media",
"filmcraft-project",
@@ -1830,7 +1832,7 @@ dependencies = [
[[package]]
name = "filmcraft-engine"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-audio-dsp",
"filmcraft-captions",
@@ -1861,7 +1863,7 @@ dependencies = [
[[package]]
name = "filmcraft-export"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-aac",
"filmcraft-audio-dsp",
@@ -1890,7 +1892,7 @@ dependencies = [
[[package]]
name = "filmcraft-format"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-media",
"filmcraft-project",
@@ -1902,7 +1904,7 @@ dependencies = [
[[package]]
name = "filmcraft-frame"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-color",
"filmcraft-time",
@@ -1912,7 +1914,7 @@ dependencies = [
[[package]]
name = "filmcraft-geom"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"proptest",
"serde",
@@ -1920,7 +1922,7 @@ dependencies = [
[[package]]
name = "filmcraft-golden"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-color",
"filmcraft-frame",
@@ -1937,7 +1939,7 @@ dependencies = [
[[package]]
name = "filmcraft-gpu"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-color",
"filmcraft-frame",
@@ -1954,7 +1956,7 @@ dependencies = [
[[package]]
name = "filmcraft-h264"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1964,7 +1966,7 @@ dependencies = [
[[package]]
name = "filmcraft-h264enc"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1974,7 +1976,7 @@ dependencies = [
[[package]]
name = "filmcraft-hevc"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1984,7 +1986,7 @@ dependencies = [
[[package]]
name = "filmcraft-interchange"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-cfb",
"filmcraft-color",
@@ -2001,7 +2003,7 @@ dependencies = [
[[package]]
name = "filmcraft-isobmff"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -2012,7 +2014,7 @@ dependencies = [
[[package]]
name = "filmcraft-matroska"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-testkit",
"serde_json",
@@ -2020,7 +2022,7 @@ dependencies = [
[[package]]
name = "filmcraft-media"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-color",
"filmcraft-frame",
@@ -2033,7 +2035,7 @@ dependencies = [
[[package]]
name = "filmcraft-mpeg2v"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-testkit",
"rayon",
@@ -2042,7 +2044,7 @@ dependencies = [
[[package]]
name = "filmcraft-mpegts"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-testkit",
"thiserror 2.0.21",
@@ -2050,28 +2052,46 @@ dependencies = [
[[package]]
name = "filmcraft-mxf"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-testkit",
]
[[package]]
name = "filmcraft-ogg"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-testkit",
]
[[package]]
name = "filmcraft-opus"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-testkit",
]
[[package]]
name = "filmcraft-platform"
version = "0.2.1"
dependencies = [
"filmcraft-codecs",
"filmcraft-color",
"filmcraft-frame",
"filmcraft-isobmff",
"filmcraft-media",
"filmcraft-testkit",
"filmcraft-time",
"log",
"objc2-core-foundation",
"objc2-core-media",
"objc2-core-video",
"objc2-video-toolbox",
]
[[package]]
name = "filmcraft-project"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-color",
"filmcraft-geom",
@@ -2085,7 +2105,7 @@ dependencies = [
[[package]]
name = "filmcraft-prores"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -2095,7 +2115,7 @@ dependencies = [
[[package]]
name = "filmcraft-render"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-audio-dsp",
"filmcraft-captions",
@@ -2115,7 +2135,7 @@ dependencies = [
[[package]]
name = "filmcraft-scopes"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-color",
"serde",
@@ -2124,7 +2144,7 @@ dependencies = [
[[package]]
name = "filmcraft-speech"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"candle-core",
"candle-nn",
@@ -2142,14 +2162,14 @@ dependencies = [
[[package]]
name = "filmcraft-testkit"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"png",
]
[[package]]
name = "filmcraft-text"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-color",
"harfrust",
@@ -2160,7 +2180,7 @@ dependencies = [
[[package]]
name = "filmcraft-time"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"proptest",
"serde",
@@ -2168,7 +2188,7 @@ dependencies = [
[[package]]
name = "filmcraft-ui-egui"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"eframe",
"egui",
@@ -2183,6 +2203,7 @@ dependencies = [
"filmcraft-gpu",
"filmcraft-interchange",
"filmcraft-media",
"filmcraft-platform",
"filmcraft-project",
"filmcraft-render",
"filmcraft-scopes",
@@ -2200,7 +2221,7 @@ dependencies = [
[[package]]
name = "filmcraft-vp9"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-bitstream",
"rayon",
@@ -2209,7 +2230,7 @@ dependencies = [
[[package]]
name = "filmcraft-web"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"eframe",
"egui",
@@ -3779,7 +3800,9 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2a180dd8642fa45cdb7dd721cd4c11b1cadd4929ce112ebd8b9f5803cc79d536"
dependencies = [
"bitflags 2.13.2",
"block2 0.6.2",
"dispatch2",
"libc",
"objc2 0.6.4",
]
@@ -3820,6 +3843,37 @@ dependencies = [
"objc2-foundation 0.2.2",
]
[[package]]
name = "objc2-core-media"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05ec576860167a15dd9fce7fbee7512beb4e31f532159d3482d1f9c6caedf31d"
dependencies = [
"bitflags 2.13.2",
"block2 0.6.2",
"dispatch2",
"objc2 0.6.4",
"objc2-core-audio",
"objc2-core-audio-types",
"objc2-core-foundation",
"objc2-core-video",
]
[[package]]
name = "objc2-core-video"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d425caf1df73233f29fd8a5c3e5edbc30d2d4307870f802d18f00d83dc5141a6"
dependencies = [
"bitflags 2.13.2",
"block2 0.6.2",
"objc2 0.6.4",
"objc2-core-foundation",
"objc2-core-graphics",
"objc2-io-surface",
"objc2-metal 0.3.2",
]
[[package]]
name = "objc2-encode"
version = "4.1.0"
@@ -3992,6 +4046,23 @@ dependencies = [
"objc2-foundation 0.2.2",
]
[[package]]
name = "objc2-video-toolbox"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05bf9a3c14831a7d9641b0d81d87dd913ee238a012b2fde27db5a84b56f5df3e"
dependencies = [
"bitflags 2.13.2",
"block2 0.6.2",
"objc2 0.6.4",
"objc2-core-foundation",
"objc2-core-graphics",
"objc2-core-media",
"objc2-core-video",
"objc2-foundation 0.3.2",
"objc2-metal 0.3.2",
]
[[package]]
name = "once_cell"
version = "1.21.4"
@@ -7140,7 +7211,7 @@ checksum = "e450f9b2ed1dff33c94c12589a87338689467b9c4f5d8a5710bd09a847d2c8a7"
[[package]]
name = "xtask"
version = "0.1.1"
version = "0.2.1"
dependencies = [
"filmcraft-testkit",
"serde_json",
+2 -1
View File
@@ -4,7 +4,7 @@ members = ["crates/*", "apps/*", "xtask"]
default-members = ["crates/*", "apps/*", "xtask"]
[workspace.package]
version = "0.1.1"
version = "0.2.1"
edition = "2024"
license = "MIT OR Apache-2.0"
rust-version = "1.95"
@@ -51,6 +51,7 @@ filmcraft-format = { path = "crates/format" }
filmcraft-engine = { path = "crates/engine" }
filmcraft-ui-egui = { path = "crates/ui-egui" }
filmcraft-automation = { path = "crates/automation" }
filmcraft-platform = { path = "crates/platform" }
filmcraft-testkit = { path = "crates/testkit" }
# foundation
+18 -3
View File
@@ -184,7 +184,7 @@ No FFmpeg inside. The video codecs, AAC, Opus and the containers are our own Rus
| **Matroska / WebM** | ✓ | | Lacing, Cues, header stripping, HDR colour metadata |
| **Stills** | ✓ | ✓ | Import PNG, JPEG, GIF, WebP, TIFF and BMP; export PNG sequences and animated GIF |
MXF and hardware decode are next ([roadmap](ROADMAP.md)).
Also imported: MPEG-2 / MPEG-1 video, AC-3, MP2, MXF (OP1a / OP-Atom), MPEG transport and program streams (AVCHD, broadcast, DVD), Ogg and image sequences. Not yet: HEVC and AV1 export, camera RAW, E-AC-3. Hardware decoding works on macOS (VideoToolbox), with Linux and Windows to follow ([#30](https://github.com/storytold/filmcraft/issues/30)).
<br>
@@ -266,11 +266,26 @@ The control protocol is documented in [docs/control-protocol.md](docs/control-pr
| [docs/control-protocol.md](docs/control-protocol.md) | Control-channel and MCP reference |
| [docs/project-files.md](docs/project-files.md) | `.fcproj` format, schema migrations, auto-save and crash recovery |
| [docs/graphics.md](docs/graphics.md) · [docs/captions.md](docs/captions.md) | Text engine, graphic clips and tools; caption tracks and formats |
| [ROADMAP.md](ROADMAP.md) | Milestones and estimates |
| [ROADMAP.md](ROADMAP.md) | Honest assessment, what's missing, milestones and estimates |
## Status
FilmCraft is young and moving fast: roughly 45% of the way to Premiere Pro parity, with editing, trimming, colour, keyframes, titles, captions, mixing, codecs and export working today. [ROADMAP.md](ROADMAP.md) tracks every milestone with estimates.
FilmCraft is young and moving fast. Editing, trimming, multicam, colour, keyframes, effects, titles, captions, mixing, codecs and export work today.
We track two numbers ([ROADMAP.md](ROADMAP.md#honest-assessment-2026-10-05)):
- **Feature checklist: ~87%.** Premiere Pro's menu items, effects, transitions, panels and formats that exist in FilmCraft.
- **Ready for real work: ~50–60%.** Our honest estimate of how close FilmCraft is to replacing Premiere on real projects.
The biggest gaps today:
- **Speed on big footage.** Hardware decoding works on macOS; Windows and Linux have no hardware path yet. Blend modes and the most common effects run on the GPU, but Lumetri, keys, export and encoding still run on the CPU ([#30](https://github.com/storytold/filmcraft/issues/30)).
- **No plugins.** No VST3 / Audio Units or OpenFX hosting.
- **Delivery codecs.** H.264 is our only delivery-codec export; no HEVC or AV1 export yet.
- **Real-world media and platforms.** Our decoders are bit-exact on conformance streams, but camera and phone files in the wild are less tested. Windows and Linux get far less testing than macOS.
- **AI features.** Few so far; speech to text is optional and off by default.
Bug reports with real footage are the most useful thing you can send us: [open an issue](https://github.com/storytold/filmcraft/issues) or tell us in [Discord](https://discord.gg/artcraft).
## Architecture
+90 -6
View File
@@ -1,10 +1,88 @@
# FilmCraft Roadmap
Progress toward feature parity with Adobe Premiere Pro, with estimates. Updated as milestones land.
Progress toward parity with Adobe Premiere Pro, with estimates. Updated as milestones land.
**Last updated:** 2026-10-02 · **Overall parity:** ~87% (measured scorecard below) · **Code:** 34 crates, 1605 tests
**Last updated:** 2026-10-05 · **Feature checklist:** ~87% · **Ready for real work:** ~50–60% · **Code:** 35 crates, 1605+ tests
## Parity scorecard
Two numbers, because they answer different questions. The **feature checklist** counts whether
Premiere's menu items, effects, panels and formats exist in FilmCraft. **Ready for real work** is
our honest estimate of how close FilmCraft is to replacing Premiere for an editor on real
projects: depth, correctness, robustness, speed and the pro ecosystem. The gap between the two
is the work that matters most now. Agents: read [Where we are lacking](#where-we-are-lacking)
before choosing work.
## Honest assessment (2026-10-05)
### How the checklist number is made, and what it doesn't tell you
- **Partly measured.** The menu figure comes from diffing a dump of Premiere 26.5's menus against
`filmcraft-cli commands`. The effects, transitions and audio-effects counts are checked by unit
tests against Premiere's Effects panel tree (`premiere_26_video_effects_catalogue`,
`premiere_audio_set_is_complete_and_foldered`, `premiere_26_transition_tree`).
- **Partly agent-reported.** The other areas' percentages are written by the agents doing the work.
No automated parity check runs in CI (PhotoCraft has `cargo xtask parity`; we don't yet), so
nothing stops a number drifting upward.
- **Presence, not quality.** An item counts once it exists. An effect described as an
"approximation" counts the same as an exact one. Nothing compares our rendering with
Premiere's.
- **Weights understate what users hit first.** Performance carries 5%. Plugins and AI features
aren't in the scorecard at all.
### By dimension
| Dimension | Checklist says | Honest estimate | Evidence |
|---|---|---|---|
| **Feature breadth** (menus, commands, panels) | ~86–95% | **~85%** | Menu diff against Premiere 26.5; effect, transition and audio-effect catalogues complete by test. The long tail is mostly there. |
| **Correctness in depth** | not tracked | **~60–70%** | A first-time contributor found four real bugs in core paths within hours (#5–#8: AIFF import failed, Slide left linked audio behind, transitions didn't follow ripple trims, export ignored start/end times). User reports: Color Matte always grey with no picker (#29); ALSA audio error on Linux (#23). Settings are "most wired", not all. |
| **Codecs and media** | ~90% | **~75%** | Our decoders are bit-exact on conformance streams. Real camera and phone media (variable frame rate, damaged files, unusual containers) is far less tested. No camera RAW (RED, BRAW, ARRIRAW), no E-AC-3. |
| **Export** | ~92% | **~70%** | Delivery: H.264 only (ProRes, DNxHR and image sequences cover mastering). No HEVC or AV1 export. AAF and OMF have never been validated in Avid Media Composer or Pro Tools. Export renders and encodes on the CPU only. |
| **Performance and hardware** | ~62% | **~35–40%** | Premiere runs effects, decoding and encoding on the GPU and the hardware media engines. Here, hardware decode works on macOS only (VideoToolbox, #33). The GPU does compositing, blend modes (#32) and 31 common effects (GPU2): colour adjustments, crop/flip/transform, blurs, sharpen. Lumetri, keys, masks, the remaining effects, export and encoding are CPU. Linux and Windows have no hardware path. 8K and AV1 aren't real-time. The effect list shows Premiere's "GPU accelerated" badge, but only 31 effect ids run on the GPU here; the rest, Lumetri included, run on the CPU. |
| **Stability** | not tracked | **improving, unproven** | The never-crash pass (no panics in product code, last-resort guards) landed on 2026-10-04. There's no field record from real users' projects yet. |
| **Plugins and pro ecosystem** | not tracked | **~0–10%** | No audio plugin hosting (VST3 / Audio Units) and no third-party video effects (OpenFX). For many professional editors this alone rules FilmCraft out. Team Projects and Productions are out of scope by design. |
| **AI features** | partial | **~25–35%** | Speech to text exists but is off in default builds (the `whisper` feature). Enhance Speech is a DSP chain, not a model. Auto Reframe is approximate. No Generative Extend, and no media-intelligence search or auto colour. |
| **Platforms** | not tracked | **macOS solid; Windows and Linux thin** | Development and testing happen on macOS. Windows and Linux are built for releases but barely exercised at runtime. The web build is a compile target; the browser app shell isn't finished. |
| **UI fidelity** | ~85% | **~80%** | Agents compare with Premiere reference screenshots; this hasn't been checked independently. |
### Where we are lacking
In priority order. Agents should prefer this work over adding more checklist items.
1. **Hardware acceleration** (#30), the most visible gap to users:
- hardware decode on Linux (VA-API) and Windows (Media Foundation);
- zero-copy decoded frames into wgpu;
- the remaining effects on the GPU: Lumetri, keys, Vignette, Video Limiter, masks (31 common effects are done);
- GPU export, then hardware encode (H.264 / HEVC).
2. **A measured parity number.** Add `cargo xtask parity`, run in CI. It should cover menus,
commands, effects, transitions, panels, preferences and formats against the Premiere snapshot in
`plan/premiere/`. It should report presence and fidelity separately (exact, approximate, stub).
3. **Real-world media corpus.** Phone, camera and screen recordings, variable frame rate, damaged
files, long GOPs, odd containers. Each gets an import → edit → export round-trip test with an
ffprobe / ffmpeg oracle.
4. **Workflow acceptance tests.** Scripted MCP sessions that do real jobs end to end: assemble and
trim a scene, colour it, mix and loudness-normalise it, caption it, deliver it, round-trip
through XML / AAF.
5. **Windows and Linux at runtime.** CI that runs the test suite and a headless playback/export
smoke test on both, not just release builds. Fix the Linux audio path (#23).
6. **Delivery codecs.** HEVC and AV1 export: hardware encoders first, and `rav1e` (BSD-2) for AV1.
7. **Plugin hosting** (VST3 / Audio Units, then OpenFX). Needs FFI, so it falls under the isolated
`unsafe` crate rule (AGENTS.md §0.3). **Owner decision needed** before starting.
8. **AI features** with openly licensed local models: transcription on by default, text-based
editing, speech enhancement, reframing.
9. **The web app shell**: file access, WebCodecs, audio.
### Where we're going
**Next:** close items 1–5. Hardware acceleration fixes the gap users notice first (CPU-bound 4K playback and export). The
parity tool, media corpus and workflow tests make the progress number trustworthy. Windows and
Linux CI makes "runs everywhere" true.
**After that:** delivery codecs, plugin hosting (pending the owner's decision) and AI features,
measured with the same tools.
**Goal:** "ready for real work" at 85%+, measured rather than estimated, on all three desktop
platforms.
## Feature checklist scorecard
Measured against Premiere Pro 26.5 on this machine. Menu items: the native menu bar dump, minus
Adobe-cloud-only items (Team Projects, Productions, Firefly, Stock, Dynamic Link, account/help pages),
@@ -22,11 +100,14 @@ matched against `filmcraft-cli commands` and the UI command table. Effects: the
| Graphics and captions | 8% | text engine, shapes, per-character styles, our own graphics templates (.fcgt, 8 built-ins, export/install/edit), rolls/crawls, responsive pins + time, captions SRT/VTT/SCC/MCC/STL/TTML/DFXP, transcripts | ~85% |
| Export | 8% | own H.264/AAC, ProRes, DNxHR, MXF OP1a/OP-Atom, image sequences, GIF, WAV, 5.1; preset library + user presets, full Export-mode settings, queue, Quick Export; AAF (Edit Protocol) import/export, OMF 2.0 export; no AAF/OMF validation against Avid/Pro Tools yet | ~92% |
| Preferences and project management | 5% | Settings dialog with 16 categories (most settings wired), project settings, scratch disks, search bins, templates | ~80% |
| Performance | 5% | 1080p real-time, 3×1080p; 4K H.264 real-time at load ≤140 (−23% decoder cycles, opt-in draft decoding at 1/2–1/4); HEVC 2.5× cheaper, catch-up decoding, `cargo xtask bench` + `perf.stats`; 8K and AV1 not yet real-time on a loaded machine | ~62% |
| Performance | 5% | 1080p real-time, 3×1080p; 4K H.264 real-time at load ≤140 (−23% decoder cycles, opt-in draft decoding at 1/2–1/4); HEVC 2.5× cheaper, catch-up decoding, `cargo xtask bench` + `perf.stats`; blend modes on the GPU (#32); 8K and AV1 not yet real-time on a loaded machine; no hardware path on Windows/Linux (see the honest assessment: ~35–40% against Premiere) | ~62% |
| **Weighted total** | | | **~87%** |
## Estimate to parity
This estimate is for the **feature checklist**. Closing the honest-assessment gaps (above) is the
"Robust on real-world material" row, plus plugins and AI features, which it doesn't include.
Measured throughput in the last work block (2026-10-01 night → 10-02): five Opus 5.5 agents in
parallel for ~4.5 wall-clock hours (~18 agent-hours including integration) moved parity by ~4 points
(≈4–5 agent-hours per point), on a machine that was heavily overloaded (load 150–300 on 14 cores) and
@@ -62,7 +143,7 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining
| M2 | H.264 decoder | ✅ | Own decoder, bit-exact on 37+ streams, 500–600 fps 1080p | — | — |
| M3 | Editing core | 🟡 | Edit algebra (insert/overwrite/razor/lift/extract/ripple/roll/slip/slide/rate-stretch/nest/paste), tools, markers, trim mode + Trim Monitor + dynamic J/K/L trimming, Keyboard Shortcuts editor with FilmCraft/Premiere/FCP/Avid presets; Edit/Clip/File menu commands (Label colours and Select Label Group, Paste/Remove Attributes, Select All Matching, Remove Unused, Consolidate Duplicates, Sequence From Clip, Bin From Selection, Offline File, Close Project, Make/Edit Subclip, Modify Audio Channels/Timecode, Frame Hold Options/Add Frame Hold/Insert Frame Hold Segment, Time Interpolation with frame blending, Fit/Fill frame, Breakout to Mono, Extract Audio, Replace With Clip; M3.11: Scene Edit Detection (pure-Rust cut detection, background job), Normalize Mix Track, Simplify Sequence, Transcribe Sequence, Find/Find Next and search bins, Automate to Sequence, Edit Original, Edit Offline, Source Settings, Update Metadata (XMP), Generate Audio Waveform, Project Settings General/Scratch Disks, Get Media File Properties, Save as Template, Selection as FilmCraft Project, Avid Log Exchange export, Flash Cue markers, Dynamic Audio Waveforms, Reveal Log Files, System Compatibility Report); M3.12 keyboard parity: Premiere's default keyboard (~115 keyboard-only commands: edit-point navigation on targeted/any track, select clip at playhead/next/previous, extend edit to playhead, nudge/slip/slide selection, target and source-patch toggles, clip volume ±1 dB/many, frame maximize/full screen/panel cycling, monitor zoom, track heights, Project and Text panel keyboard navigation, text size/leading/alignment, Export Frame, poster frames; see docs/keyboard.md); multicam (Create Multi-Camera Source Sequence, Multi-Camera view with live switching on 1–9, angle switching, Enable/Flatten, Edit Cameras) and sync (Synchronize, Merge Clips; In/Out/timecode/marker/audio — GCC-PHAT, sample-accurate) | Multicam paging >16 angles and grid thumbnails, optical flow (renders as frame blending), 39 Premiere default shortcuts skipped with reasons in docs/keyboard.md (Productions, AI/cloud tools, work area bar, Production/Search panels) | 6–10 |
| M4 | Playback | 🟡 | Audio-clock master, prefetch with cancellation, J/K/L, correct dropped-frame stats, playback resolution, render bar + content-hashed render previews, App Nap opt-out; 1080p H.264 and 3 stacked 1080p streams play with 0 dropped frames | 4K under load, 8K, frame-threaded AV1 decode, reduced-resolution decode for multicam grids | 6–10 |
| M5 | Effects, keyframes, GPU | 🟡 | ~60 CPU effects, 30 transitions, keyframes + value/velocity graphs, wgpu compositor, effect + opacity masks (ellipse/polygon/pen, feather, expansion, modes; CPU/WGSL parity), mask tracking (Lucas–Kanade + RANSAC), adjustment layers, effect presets (built-in + user, JSON import/export) | Full ~150-effect catalogue, WGSL parity for all effects and masks in the live GPU path, Warp Stabilizer, Morph Cut | 20–30 |
| M5 | Effects, keyframes, GPU | 🟡 | All 93 Premiere 26 video effects and 84 transitions (CPU), blend modes and 31 common effects on the GPU (#32, GPU2), keyframes + value/velocity graphs, wgpu compositor, effect + opacity masks (ellipse/polygon/pen, feather, expansion, modes; CPU/WGSL parity), mask tracking (Lucas–Kanade + RANSAC), adjustment layers, effect presets (built-in + user, JSON import/export) | Effects on the GPU (in progress, #30), GPU masks in the live path, GPU export, exact Warp Stabilizer (3-D) and Morph Cut | 20–30 |
| M6 | Export | ✅ | Own H.264 encoder (High/Main/Baseline, B-frames, VBR/CBR/2-pass) → MP4 or QuickTime + own AAC; ProRes, DNxHR, MJPEG, PNG/TIFF/BMP sequences (Premiere-style numbering), GIF, WAV, AIFF; background jobs; M6.5: Export mode parity (frame size / rate / scaling / pixel aspect, profile / level, CBR / VBR 1- and 2-pass, keyframe distance, audio codec / rate / channels / bitrate / sample size, multiplexer, burn-in or sidecar captions, image / name / timecode overlays, video limiter, loudness normalization with true-peak limiter, metadata, ranges, estimated size, summary), 24 built-in presets + user presets with favourites and import/export (Preset Manager), export queue (reorder, cancel, retry, several sequences / ranges), Quick Export; `export.*` commands and `filmcraft-cli export --preset` | Interlaced encoding, 5.1 audio, smart render, publishing destinations | 6–8 |
| M7 | Audio | 🟡 | Mixer graph (tracks → submixes → Mix, pre/post-fader inserts and sends, latency-compensated, sample-accurate, ~6× realtime for 24 tracks × 3 effects on one core), Audio Track Mixer + Audio Clip Mixer panels, track automation (Off/Read/Latch/Touch/Write, recorded live while playing, thinned to keyframes, timeline lanes with pen editing), solo/solo-safe, channel mapping basics, peak + BS.1770 loudness meters (match ffmpeg), DSP crate with 16 clip/track effects, Audio Gain (set/adjust/normalize), Constant Power / Constant Gain / Exponential Fade, Essential Sound (types, Loudness auto-match, Repair incl. DeEss/DeReverb, Clarity, Creative, Ducking keyframes, presets; full Dialogue chain 22× realtime) | Music duration remix, ML speech enhancement, 5.1 panner and multichannel buses, voice-over record, effect editor windows (EQ curve), remaining effects (multiband, convolution reverb), clip-mixer automation recording | 6–9 |
| M8 | Colour | 🟡 | Lumetri: basic, creative + looks, RGB & hue curves, wheels, HSL secondary, vignette, section bypass; Input LUT / Look LUT (.cube 1D/3D/shaper, .3dl; tetrahedral CPU + WGSL; project LUT library; built-in camera conversions); Colour Match (Oklab tonal-range statistics, skin protection, solved in Lumetri wheels); colour management: Rec. 709 / Rec. 2100 PQ / HLG working spaces + wide gamut, Interpret Footage colour space (S-Log3, V-Log, Canon Log 2/3, LogC3/4, Apple Log, D-Log from published specs), metadata auto-detect, BT.2390 tone mapping, gamut mapping, HDR export signalling (VUI/colr/mdcv/clli/SEI, ffprobe-verified); Lumetri Scopes: Vectorscope YUV (75/100 % targets, skin-tone line) and HLS, Histogram, Parade (RGB/YUV/RGB-White), Waveform (RGB/Luma/YC/YC no Chroma), presets, Rec. 601/709/2100, 8-bit/float/HDR (cd/m²) scales, multi-scope grid, `scopes.read` | HDR-aware Lumetri maths, macOS EDR monitors, mastering metadata → tone-map peak, D-Log M (no published formula), HSL Secondary refine | 3–5 |
@@ -73,10 +154,13 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining
## Running now
- Nothing; next: remaining Premiere menu items (Settings categories, Scene Edit Detection, Normalize Mix Track, OMF/AAF, Find), AV1 single-thread re-measure on an idle machine
- **Hardware acceleration (#30):** landed: blend modes on the GPU (#32); VideoToolbox hardware decode + the `platform` FFI crate (#33); 31 common effects on the GPU (GPU2). Next: VA-API (Linux) and Media Foundation (Windows) decode, zero-copy upload, Lumetri on the GPU.
- **Next:** [Where we are lacking](#where-we-are-lacking) items 1–5. GPU export waits for the export-crate work (#19 and the never-crash follow-up) to land.
## Log
- **2026-10-05:** honest assessment added (checklist ~87% vs ready for real work ~50–60%). Hardware acceleration started (#30): blend modes on the GPU (#32); VideoToolbox decode + `platform` crate, the one crate allowed `unsafe` (#33). Community fixes landed: AIFF import, export start/end ranges, transitions following ripple trims, Slide moving linked audio (#5–#8).
- **2026-10-04 (later):** interchange + MXF export (M11.4/M6.6): own compound-file container (`crates/cfb`), AAF Edit Protocol import/export (embedded/linked/consolidated audio, video mixdown, handles, breakout to mono), OMF 2.0 export (Bento), MXF OP1a/OP-Atom writer (DNxHR/ProRes/H.264/PCM) as export formats. 1605 tests.
- **2026-10-04:** MPEG-2 (M9.15): own MPEG-2/MPEG-1 video decoder (4:2:2, interlaced, field pictures; IEEE 1180 IDCT), MPEG TS/PS demux (AVCHD .mts, .m2ts, .ts, .mpg/.vob/.mod), own AC-3 decoder (ATSC A/52), MP2, LATM AAC, Blu-ray/DVD LPCM, MPEG-2 in MXF/MOV/MP4/MKV. 1562 tests.
+1
View File
@@ -21,6 +21,7 @@ whisper = ["filmcraft-engine/whisper", "filmcraft-engine/speech-download"]
filmcraft-engine = { workspace = true }
filmcraft-media = { workspace = true }
filmcraft-codecs = { workspace = true }
filmcraft-platform = { workspace = true }
filmcraft-mxf = { workspace = true }
filmcraft-mpegts = { workspace = true }
filmcraft-ogg = { workspace = true }
+3 -1
View File
@@ -88,7 +88,9 @@ fn insert_dotted(m: &mut Map<String, Value>, key: &str, v: Value) {
if !e.is_object() {
*e = Value::Object(Map::new());
}
insert_dotted(e.as_object_mut().expect("object"), tail, v);
if let Value::Object(child) = e {
insert_dotted(child, tail, v);
}
}
}
}
+6 -2
View File
@@ -2,6 +2,8 @@
//!
//! Run `filmcraft-cli help` for the full reference (also in docs/agents.md).
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
mod args;
mod probe;
@@ -153,6 +155,8 @@ async fn main() {
println!("filmcraft-cli {}", env!("CARGO_PKG_VERSION"));
return;
}
// OS hardware video decoders (VideoToolbox on macOS) in front of our own, as in the desktop app.
let _ = filmcraft_platform::register();
let a = Args::parse(std::env::args().skip(1));
let Some(cmd) = a.pos(0) else { usage("missing subcommand") };
match cmd {
@@ -339,7 +343,7 @@ async fn main() {
if a.opt("--bridge").is_some() {
usage("render is headless; with --bridge use the MCP `render_frame` tool or `exec` ui commands");
}
let Backend::Local(mut s) = Backend::open(&a) else { unreachable!() };
let Backend::Local(mut s) = Backend::open(&a) else { usage("render is headless") };
let secs: f64 = a.opt("--seconds").and_then(|v| v.parse().ok()).unwrap_or(0.0);
let scale: f32 = a.opt("--scale").and_then(|v| v.parse().ok()).unwrap_or(1.0);
let out = a.opt("--out").unwrap_or("frame.png");
@@ -356,7 +360,7 @@ async fn main() {
Some(addr) => filmcraft_automation::FilmcraftMcp::bridge(addr).unwrap_or_else(|e| fail(e)),
None => match Backend::open(&a) {
Backend::Local(s) => filmcraft_automation::FilmcraftMcp::headless(*s),
Backend::Bridge(_) => unreachable!(),
Backend::Bridge(_) => usage("mcp: use --bridge ADDR to drive a running app"),
},
};
if let Err(e) = server.serve_stdio().await {
+7 -2
View File
@@ -289,20 +289,25 @@ impl ByteReader for BlobReader {
}
return Err(filmcraft_media::pending::would_block());
}
FS.with(|fs| {
let complete = FS.with(|fs| {
let mut fs = fs.borrow_mut();
fs.clock += 1;
let t = fs.clock;
let mut pos = 0usize;
for c in c0..=c1 {
let (d, used) = fs.chunks.get_mut(&(self.id, c)).expect("checked above");
// Present (checked above); evicted in between would be a bug, reported as an error.
let Some((d, used)) = fs.chunks.get_mut(&(self.id, c)) else { return false };
*used = t;
let a = if c == c0 { (offset - c * CHUNK) as usize } else { 0 };
let n = (d.len() - a).min(buf.len() - pos);
buf[pos..pos + n].copy_from_slice(&d[a..a + n]);
pos += n;
}
true
});
if !complete {
return Err(filmcraft_media::pending::would_block());
}
// read-ahead for sequential access
for c in c1 + 1..=(c1 + READ_AHEAD).min(last) {
request(self.id, c);
+1
View File
@@ -16,6 +16,7 @@
//!
//! Everything here is `wasm32`-only; on other targets the crate is empty.
#![cfg(target_arch = "wasm32")]
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
pub mod api;
pub mod audio;
+1 -1
View File
@@ -463,7 +463,7 @@ impl WcSource {
c.frames.insert(pts, Arc::new(f));
}
while c.frames.len() > CACHE_FRAMES {
let far = c.frames.keys().copied().max_by_key(|p| (p - want).abs()).expect("non-empty");
let Some(far) = c.frames.keys().copied().max_by_key(|p| (p - want).abs()) else { break };
c.frames.remove(&far);
}
// the session's own frame came out: other requests may restart it now
+1
View File
@@ -18,6 +18,7 @@ whisper = ["filmcraft-engine/whisper", "filmcraft-engine/speech-download"]
[dependencies]
filmcraft-engine = { workspace = true }
filmcraft-ui-egui = { workspace = true }
filmcraft-platform = { workspace = true }
egui = { workspace = true }
eframe = { workspace = true, features = ["default_fonts", "wgpu", "accesskit", "wayland", "x11"] }
serde_json = { workspace = true }
+5
View File
@@ -14,6 +14,8 @@
//! with unsaved changes the app asks to recover them; `--recover` recovers the newest without
//! asking, `--no-recover` starts without asking (the changes stay available via File ▸ Recover
//! Unsaved Changes…).
#![cfg_attr(all(target_os = "windows", not(debug_assertions)), windows_subsystem = "windows")]
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
mod app_nap;
mod audio;
@@ -71,6 +73,9 @@ fn main() -> eframe::Result {
// Panics anywhere go to <data dir>/Logs/crash-<day>.log with a backtrace; the UI pass and
// frame workers catch them and keep running (see filmcraft_ui_egui::crash).
filmcraft_ui_egui::crash::install(data_dir.clone().or_else(default_data_dir).map(|d| d.join("Logs")));
// OS hardware video decoders (VideoToolbox on macOS) in front of our own; Settings ▸ Playback ▸
// Hardware decoding switches them off. Unsupported streams and failures use our decoders.
log::info!("hardware decoding: {:?}", filmcraft_platform::register());
let options = eframe::NativeOptions {
viewport: egui::ViewportBuilder::default()
.with_title("FilmCraft")
+2
View File
@@ -31,6 +31,8 @@
//! # Ok::<(), filmcraft_aac::Error>(())
//! ```
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
mod config;
mod decoder;
mod encoder;
+7 -6
View File
@@ -13,6 +13,8 @@
//!
//! E-AC-3 (bsid 11-16) is recognised and refused.
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
mod tables;
use std::sync::OnceLock;
@@ -359,8 +361,7 @@ fn bit_allocation(exp: &[u8; 256], p: &AllocParams, g: &Globals, bap: &mut [u8;
let bndend = tables::MASKTAB[end - 1] as usize + 1;
let mut excite = [0i32; 50];
let (mut fastleak, mut slowleak) = p.leak.unwrap_or((0, 0));
let begin;
if bndstrt == 0 {
let begin = if bndstrt == 0 {
// the last band of the LFE channel (bndend 7) has no band above it
let lfe_last = |bin: usize| !(bndend == 7 && bin == 6);
let mut lowcomp = 0;
@@ -391,10 +392,10 @@ fn bit_allocation(exp: &[u8; 256], p: &AllocParams, g: &Globals, bap: &mut [u8;
slowleak = slowleak.max(bndpsd[bin] - sgain);
excite[bin] = (fastleak - lowcomp).max(slowleak);
}
begin = 22;
22
} else {
begin = bndstrt;
}
bndstrt
};
for bin in begin..bndend {
fastleak -= fdecay;
fastleak = fastleak.max(bndpsd[bin] - p.fgain);
@@ -1030,7 +1031,7 @@ impl Decoder {
/// Coded channel order (Table 5.8) → WAV / SMPTE order: fronts L R C, LFE, surrounds.
fn wav_order(mut pcm: Vec<Vec<f32>>, acmod: usize, lfeon: bool) -> Vec<Vec<f32>> {
let nf = NFCHANS[acmod];
let lfe = lfeon.then(|| pcm.pop().expect("lfe"));
let lfe = if lfeon { pcm.pop() } else { None };
let mut fbw = pcm;
// acmod 3, 5, 7 code L C R …: move C after R
if acmod & 1 == 1 && acmod != 1 {
+1 -1
View File
@@ -99,7 +99,7 @@ fn ffmpeg_encoded_streams_decode() {
}
}
let o = Command::new(&ff).args(["-v", "error", "-i"]).arg(&f).args(["-f", "f32le", "-"]).output().unwrap();
let want: Vec<f32> = o.stdout.chunks_exact(4).map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
let want: Vec<f32> = o.stdout.as_chunks::<4>().0.iter().map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
let ch = s.channels as usize;
let n = want.len() / ch;
assert_eq!(n, ours[0].len(), "{}: sample count", s.name);
+1 -2
View File
@@ -123,7 +123,7 @@ pub fn matrix(from: Layout, to: Layout, mixdown: Mixdown) -> Vec<Vec<f32>> {
let (ni, no) = (from.channels(), to.channels());
let mut m = vec![vec![0.0f32; ni]; no];
match (from, to) {
(a, b) if a == b => {
(Layout::Mono, Layout::Mono) | (Layout::Stereo, Layout::Stereo) | (Layout::Surround51, Layout::Surround51) => {
for (i, row) in m.iter_mut().enumerate() {
row[i] = 1.0;
}
@@ -159,7 +159,6 @@ pub fn matrix(from: Layout, to: Layout, mixdown: Mixdown) -> Vec<Vec<f32>> {
m[1][LFE] = K;
}
}
_ => unreachable!("all layout pairs are covered"),
}
m
}
+6 -5
View File
@@ -16,6 +16,8 @@
//! denormals never appear (no FTZ/DAZ CPU flags needed, which keeps the crate `unsafe`-free and
//! portable to wasm).
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
pub mod biquad;
pub mod channels;
pub mod design;
@@ -169,12 +171,11 @@ impl ParamValues {
pub(crate) fn get(&self, id: &str) -> Option<f32> {
self.index_of(id).map(|i| self.values[i])
}
/// Value of a parameter that is known to exist (panics otherwise — programming error).
/// Value of a parameter that is known to exist (0 otherwise — a programming error caught by
/// debug builds).
pub(crate) fn v(&self, id: &str) -> f32 {
match self.get(id) {
Some(v) => v,
None => panic!("unknown parameter {id}"),
}
debug_assert!(self.get(id).is_some(), "unknown parameter {id}");
self.get(id).unwrap_or(0.0)
}
pub(crate) fn on(&self, id: &str) -> bool {
self.v(id) >= 0.5
+3 -1
View File
@@ -42,7 +42,9 @@ impl BridgeClient {
*guard = Some((BufReader::new(r), w));
}
let id = self.next_id.fetch_add(1, Ordering::Relaxed);
let conn = guard.as_mut().expect("connected");
let Some(conn) = guard.as_mut() else {
return Err(AutomationError::Bridge(format!("not connected to {}", self.addr)));
};
let line = format!("{}\n", json!({"id": id, "method": method, "params": params}));
let res: Result<Value, AutomationError> = async {
conn.1.write_all(line.as_bytes()).await.map_err(|e| AutomationError::Bridge(e.to_string()))?;
+3
View File
@@ -8,7 +8,10 @@
//! (`filmcraft --control <port>`), so agents can also inspect, screenshot, click, drag, scroll
//! and type in the live UI — every menu, panel, timeline gesture and keystroke.
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
pub mod bridge;
pub mod long_job;
pub mod server;
pub use bridge::BridgeClient;
+183
View File
@@ -0,0 +1,183 @@
//! Long exports over MCP (docs/agents.md § Long exports): `file.exportMedia` with `wait: true`
//! runs as a background engine job; the server reports its progress as MCP
//! `notifications/progress` (when the request carried a `progressToken`) and stops it on
//! `notifications/cancelled`, deleting the partial output. The session lock is only held for
//! short polls, so other requests are answered while the export runs.
use std::path::Path;
use std::time::{Duration, SystemTime};
use rmcp::RoleServer;
use rmcp::model::{CallToolResult, ContentBlock as Content, ProgressNotificationParam, ProgressToken};
use rmcp::service::{Peer, RequestContext};
use serde_json::{Value, json};
use crate::server::FilmcraftMcp;
/// Engine commands that block until a render is written when called with `wait: true`.
pub const LONG_COMMANDS: &[&str] = &["file.exportMedia"];
/// How often the job is polled (and at most how often progress is reported).
const POLL: Duration = Duration::from_millis(100);
/// Whether `command_run {id, params}` is a long call this module runs.
pub fn is_long(id: &str, params: &Value) -> bool {
LONG_COMMANDS.contains(&id) && params.get("wait").and_then(Value::as_bool) == Some(true)
}
/// Sends `notifications/progress` for one request; progress only ever increases.
struct Reporter {
peer: Peer<RoleServer>,
token: Option<ProgressToken>,
last: f64,
}
impl Reporter {
async fn report(&mut self, done: f64, total: f64, message: &str) {
let Some(token) = self.token.clone() else { return };
if total <= 0.0 || done <= self.last {
return;
}
self.last = done;
let mut p = ProgressNotificationParam::new(token, done).with_total(total);
if !message.is_empty() {
p = p.with_message(message);
}
let _ = self.peer.notify_progress(p).await;
}
}
impl FilmcraftMcp {
/// Run `id` (a [`LONG_COMMANDS`] entry, `wait: true`) as a job, reporting progress and
/// honouring cancellation.
pub(crate) async fn run_long(&self, id: &str, mut params: Value, context: &RequestContext<RoleServer>) -> CallToolResult {
let started = SystemTime::now();
if let Some(p) = params.as_object_mut() {
p.insert("wait".into(), json!(false));
}
let start = match self.run(id, params).await {
Ok(v) => v,
Err(e) => return CallToolResult::error(vec![Content::text(e.to_string())]),
};
let Some(job) = start.get("job").and_then(Value::as_u64) else {
return CallToolResult::error(vec![Content::text(format!("{id} started no job: {start}"))]);
};
let path = start.get("path").and_then(Value::as_str).unwrap_or_default().to_string();
let mut rep = Reporter { peer: context.peer.clone(), token: context.meta.get_progress_token(), last: 0.0 };
loop {
let state = self.job_state(job).await;
let finished = state.as_ref().is_none_or(|j| j["finished"].as_bool() == Some(true));
if let Some(j) = &state {
let (done, total) = (j["done"].as_f64().unwrap_or(0.0), j["total"].as_f64().unwrap_or(0.0));
rep.report(done, total, j["status"].as_str().unwrap_or_default()).await;
}
if finished {
let result = state.map(|j| j["result"].clone()).unwrap_or(Value::Null);
if let Some(e) = result.get("error").and_then(Value::as_str) {
return CallToolResult::error(vec![Content::text(format!("export failed: {e}"))]);
}
let mut out = start;
if let Some(o) = out.as_object_mut() {
o.insert("result".into(), result);
}
return CallToolResult::success(vec![Content::text(serde_json::to_string_pretty(&out).unwrap_or_default())]);
}
tokio::select! {
_ = context.ct.cancelled() => break,
_ = tokio::time::sleep(POLL) => {}
}
}
// Cancelled: stop the encode at the next batch, wait for the worker, delete the partial file.
let _ = self.run("jobs.cancel", json!({"job": job})).await;
for _ in 0..600 {
if self.job_state(job).await.is_none_or(|j| j["finished"].as_bool() == Some(true)) {
break;
}
tokio::time::sleep(POLL).await;
}
remove_partial(&path, started);
CallToolResult::error(vec![Content::text("cancelled")])
}
/// The job's `jobs.list` entry.
async fn job_state(&self, job: u64) -> Option<Value> {
let jobs = self.run("jobs.list", json!({})).await.ok()?;
jobs.as_array()?.iter().find(|j| j["id"].as_u64() == Some(job)).cloned()
}
}
/// Delete what an interrupted export wrote, and nothing else: the output file itself, the numbered
/// stills of an image sequence (`<stem><digits>.<same extension>`) and the caption sidecar
/// (`<stem>.srt` / `<stem>.vtt`), each only if it was written since `since`. Other files that
/// merely share the name's stem (`<stem>.aep`, `<stem>2.psd`, …) are never touched, even when they
/// were saved during the export.
pub fn remove_partial(path: &str, since: SystemTime) {
let p = Path::new(path);
let (Some(dir), Some(stem), Some(file)) = (p.parent(), p.file_stem().and_then(|s| s.to_str()), p.file_name().and_then(|s| s.to_str())) else {
return;
};
let ext = p.extension().and_then(|s| s.to_str()).unwrap_or_default();
let dir = if dir.as_os_str().is_empty() { Path::new(".") } else { dir };
let Ok(entries) = std::fs::read_dir(dir) else { return };
// file times are coarser than the clock
let since = since.checked_sub(Duration::from_secs(1)).unwrap_or(since);
for e in entries.flatten() {
let name = e.file_name();
let Some(name) = name.to_str() else { continue };
if is_export_output(name, file, stem, ext) && e.metadata().is_ok_and(|m| m.is_file() && m.modified().is_ok_and(|t| t >= since)) {
let _ = std::fs::remove_file(e.path());
}
}
}
/// Whether `name` is a file an export to `file` (`stem` + `.ext`) writes.
fn is_export_output(name: &str, file: &str, stem: &str, ext: &str) -> bool {
if name == file {
return true;
}
let Some(rest) = name.strip_prefix(stem) else { return false };
if let Some(sidecar) = rest.strip_prefix('.') {
return sidecar.eq_ignore_ascii_case("srt") || sidecar.eq_ignore_ascii_case("vtt");
}
// image sequence: one or more digits, then the output's own extension
let digits = rest.len() - rest.trim_start_matches(|c: char| c.is_ascii_digit()).len();
digits > 0 && !ext.is_empty() && rest.get(digits..).is_some_and(|r| r.strip_prefix('.').is_some_and(|x| x == ext))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn only_the_exports_own_files_are_removed() {
let dir = std::env::temp_dir().join(format!("fc-remove-partial-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let started = SystemTime::now();
let ours = ["trailer.png", "trailer000.png", "trailer123.png", "trailer.srt", "trailer.vtt"];
// same stem, written during the export, but not the export's
let theirs = ["trailer.aep", "trailer2.psd", "trailer.docx", "trailer000.jpg", "trailers.png", "trailer.png.bak", "other.png"];
for f in ours.iter().chain(theirs.iter()) {
std::fs::write(dir.join(f), b"x").unwrap();
}
remove_partial(&dir.join("trailer.png").to_string_lossy(), started);
for f in ours {
assert!(!dir.join(f).exists(), "{f} should be removed");
}
for f in theirs {
assert!(dir.join(f).exists(), "{f} must not be removed");
}
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn files_older_than_the_export_are_kept() {
let dir = std::env::temp_dir().join(format!("fc-remove-partial-old-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
std::fs::write(dir.join("clip.mp4"), b"x").unwrap();
remove_partial(&dir.join("clip.mp4").to_string_lossy(), SystemTime::now() + Duration::from_secs(60));
assert!(dir.join("clip.mp4").exists());
let _ = std::fs::remove_dir_all(&dir);
}
}
+546 -28
View File
@@ -1,6 +1,6 @@
//! The MCP server. Tool names are snake_case.
use std::sync::{Arc, Mutex};
use std::sync::{Arc, Mutex, PoisonError};
use base64::Engine as _;
use filmcraft_engine::Session;
@@ -19,6 +19,19 @@ pub enum Backend {
Bridge(Arc<BridgeClient>),
}
/// A panic payload as text.
fn panic_message(p: Box<dyn std::any::Any + Send>) -> String {
p.downcast_ref::<&str>().map(|s| s.to_string()).or_else(|| p.downcast_ref::<String>().cloned()).unwrap_or_else(|| "panic".into())
}
/// A failed blocking task: a panic is reported as an internal error and the session stays usable.
fn join_error(e: tokio::task::JoinError) -> AutomationError {
match e.try_into_panic() {
Ok(p) => AutomationError::Other(format!("internal error: {}", panic_message(p))),
Err(e) => AutomationError::Other(e.to_string()),
}
}
#[derive(Clone)]
pub struct FilmcraftMcp {
backend: Arc<Backend>,
@@ -44,6 +57,24 @@ pub struct RunParams {
pub params: Option<Value>,
}
#[derive(Debug, Deserialize, JsonSchema)]
pub struct BatchStep {
/// Command id.
pub id: String,
/// Parameters object.
#[serde(default)]
pub params: Option<Value>,
}
#[derive(Debug, Deserialize, JsonSchema)]
pub struct BatchParams {
/// Commands to run in order.
pub steps: Vec<BatchStep>,
/// Stop at the first failing step (default true).
#[serde(default)]
pub stop_on_error: Option<bool>,
}
#[derive(Debug, Default, Deserialize, JsonSchema)]
pub struct RenderParams {
/// Timeline time in seconds (default: the playhead).
@@ -148,9 +179,60 @@ impl FilmcraftMcp {
}
pub async fn serve_stdio(self) -> Result<(), AutomationError> {
let running = self.serve(rmcp::transport::stdio()).await.map_err(|e| AutomationError::Other(format!("MCP init: {e}")))?;
running.waiting().await.map_err(|e| AutomationError::Other(e.to_string()))?;
Ok(())
self.serve_io(tokio::io::stdin(), tokio::io::stdout()).await
}
/// Serve MCP over any line stream. A line that isn't JSON gets a JSON-RPC parse error
/// (`-32700`, id null) instead of being dropped silently; the rest go to the rmcp service. One
/// writer task owns `output`, so replies never interleave.
pub async fn serve_io(
self,
input: impl tokio::io::AsyncRead + Unpin + Send + 'static,
output: impl tokio::io::AsyncWrite + Unpin + Send + 'static,
) -> Result<(), AutomationError> {
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
let (mut to_service, service_in) = tokio::io::duplex(1 << 20);
let (service_out, from_service) = tokio::io::duplex(1 << 20);
let (err_tx, mut err_rx) = tokio::sync::mpsc::unbounded_channel::<String>();
let reader = tokio::spawn(async move {
let mut lines = BufReader::new(input).lines();
while let Ok(Some(line)) = lines.next_line().await {
if line.trim().is_empty() {
continue;
}
match serde_json::from_str::<Value>(&line) {
Ok(_) => {
if to_service.write_all(format!("{line}\n").as_bytes()).await.is_err() {
break;
}
}
Err(e) => {
let reply = json!({"jsonrpc": "2.0", "id": null, "error": {"code": -32700, "message": format!("parse error: {e}")}});
let _ = err_tx.send(reply.to_string());
}
}
}
// EOF: closing the pipe ends the service
let _ = to_service.shutdown().await;
});
let writer = tokio::spawn(async move {
let mut output = output;
let mut replies = BufReader::new(from_service).lines();
loop {
let line = tokio::select! {
l = replies.next_line() => match l { Ok(Some(l)) => l, _ => break },
Some(e) = err_rx.recv() => e,
};
if output.write_all(format!("{line}\n").as_bytes()).await.is_err() || output.flush().await.is_err() {
break;
}
}
});
let running = self.serve((service_in, service_out)).await.map_err(|e| AutomationError::Other(format!("MCP init: {e}")))?;
let r = running.waiting().await.map_err(|e| AutomationError::Other(e.to_string()));
reader.abort();
let _ = writer.await;
r.map(|_| ())
}
/// Run an engine command on whichever backend.
@@ -160,16 +242,24 @@ impl FilmcraftMcp {
let s = s.clone();
let id = id.to_string();
tokio::task::spawn_blocking(move || {
let mut g = s.lock().map_err(|_| AutomationError::Other("session lock poisoned".into()))?;
// A command that panicked earlier poisoned the lock; the session is still usable.
let mut g = s.lock().unwrap_or_else(PoisonError::into_inner);
g.execute(&id, params).map_err(AutomationError::from)
})
.await
.map_err(|e| AutomationError::Other(e.to_string()))?
.map_err(join_error)?
}
Backend::Bridge(b) => b.call("engine.execute", json!({"command": id, "params": params})).await,
}
}
/// The project tree plus the active sequence (null when there is none).
async fn document(&self) -> Result<Value, AutomationError> {
let project = self.run("project.inspect", json!({})).await?;
let sequence = self.run("sequence.inspect", json!({})).await.ok();
Ok(json!({"project": project, "sequence": sequence}))
}
fn bridge_client(&self) -> Option<Arc<BridgeClient>> {
match &*self.backend {
Backend::Bridge(b) => Some(b.clone()),
@@ -190,7 +280,9 @@ impl FilmcraftMcp {
#[tool_router]
impl FilmcraftMcp {
#[tool(
description = "List every command (id, label, menu path, shortcut, parameter doc, enabled now). Menus, shortcuts, panels and timeline gestures all map to these ids."
title = "List commands",
description = "List every command (id, label, menu path, shortcut, parameter doc, enabled now). Menus, shortcuts, panels and timeline gestures all map to these ids.",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn command_list(&self, Parameters(p): Parameters<ListParams>) -> Result<CallToolResult, McpError> {
let r = self.run("command.list", json!({})).await.map(|v| {
@@ -213,38 +305,99 @@ impl FilmcraftMcp {
}
#[tool(
description = "Run a command by id with JSON params, e.g. {\"id\":\"timeline.razor\",\"params\":{\"seconds\":3.5}} or {\"id\":\"effects.apply\",\"params\":{\"effect\":\"Gaussian Blur\"}}. Undoable edits land in History."
title = "Run a command",
description = "Run a command by id with JSON params, e.g. {\"id\":\"timeline.razor\",\"params\":{\"seconds\":3.5}} or {\"id\":\"effects.apply\",\"params\":{\"effect\":\"Gaussian Blur\"}}. Undoable edits land in History.",
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
)]
async fn command_run(&self, Parameters(p): Parameters<RunParams>) -> Result<CallToolResult, McpError> {
wrap(self.run(&p.id, p.params.unwrap_or(json!({}))).await)
}
#[tool(description = "Project tree (bins and items with ids, types, durations), active sequence, source clip.")]
#[tool(
title = "Run several commands",
description = "Run several commands in order: {steps: [{id, params?}], stop_on_error?: true}. Returns {completed, failed, results: [{ok, result | error}]}; each edit is its own undo step.",
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
)]
async fn command_batch(&self, Parameters(p): Parameters<BatchParams>) -> Result<CallToolResult, McpError> {
let stop = p.stop_on_error.unwrap_or(true);
let (mut completed, mut failed, mut results) = (0u32, 0u32, Vec::new());
for st in p.steps {
match self.run(&st.id, st.params.unwrap_or(json!({}))).await {
Ok(v) => {
completed += 1;
results.push(json!({"ok": true, "result": v}));
}
Err(e) => {
failed += 1;
results.push(json!({"ok": false, "error": e.to_string()}));
if stop {
break;
}
}
}
}
let v = json!({"completed": completed, "failed": failed, "results": results});
Ok(if failed > 0 { CallToolResult::error(vec![Content::text(v.to_string())]) } else { ok_json(&v) })
}
#[tool(
title = "Inspect the document",
description = "The project as JSON: the tree (as project_inspect) plus the active sequence (as sequence_inspect; null when there is none).",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn doc_inspect(&self) -> Result<CallToolResult, McpError> {
wrap(self.document().await)
}
#[tool(
title = "Render a preview",
description = "The active sequence's frame at `seconds` (default: the playhead) as PNG, same as render_frame (bridge mode: Program monitor screenshot).",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn render_preview(&self, p: Parameters<RenderParams>) -> Result<CallToolResult, McpError> {
self.render_frame(p).await
}
#[tool(
title = "Inspect the project tree",
description = "Project tree (bins and items with ids, types, durations), active sequence, source clip.",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn project_inspect(&self) -> Result<CallToolResult, McpError> {
wrap(self.run("project.inspect", json!({})).await)
}
#[tool(
description = "The active sequence as JSON: settings, tracks, clips (ids, start/duration in ticks and frames, effects), transitions, markers, in/out, playhead, selection. 254016000000 ticks = 1 second."
title = "Inspect the active sequence",
description = "The active sequence as JSON: settings, tracks, clips (ids, start/duration in ticks and frames, effects), transitions, markers, in/out, playhead, selection. 254016000000 ticks = 1 second.",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn sequence_inspect(&self) -> Result<CallToolResult, McpError> {
wrap(self.run("sequence.inspect", json!({})).await)
}
#[tool(description = "Import media files by absolute path (MP4/MOV, WAV/MP3/FLAC/AIFF/Ogg, PNG/JPEG/…).")]
#[tool(
title = "Import media",
description = "Import media files by absolute path (MP4/MOV, WAV/MP3/FLAC/AIFF/Ogg, PNG/JPEG/…).",
annotations(read_only_hint = false, destructive_hint = false, idempotent_hint = false, open_world_hint = false)
)]
async fn media_import(&self, Parameters(p): Parameters<TypeParams>) -> Result<CallToolResult, McpError> {
let paths: Vec<&str> = p.text.split('\n').map(str::trim).filter(|s| !s.is_empty()).collect();
wrap(self.run("file.import", json!({"paths": paths})).await)
}
#[tool(description = "Render the active sequence's frame (at `seconds` or the playhead) and return it as PNG.")]
#[tool(
title = "Render a frame",
description = "Render the active sequence's frame (at `seconds` or the playhead) and return it as PNG.",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn render_frame(&self, Parameters(p): Parameters<RenderParams>) -> Result<CallToolResult, McpError> {
let max = p.max_side.unwrap_or(960);
match &*self.backend {
Backend::Headless(s) => {
let s = s.clone();
let r = tokio::task::spawn_blocking(move || -> Result<Vec<u8>, AutomationError> {
let mut g = s.lock().map_err(|_| AutomationError::Other("lock".into()))?;
let mut g = s.lock().unwrap_or_else(PoisonError::into_inner);
if let Some(sec) = p.seconds {
g.set_playhead(filmcraft_time::Tick::from_seconds_f64(sec));
}
@@ -255,7 +408,7 @@ impl FilmcraftMcp {
png_rgba(img.w as u32, img.h as u32, img.over_black_rgba8(), max)
})
.await
.map_err(|e| McpError::internal_error(e.to_string(), None))?;
.unwrap_or_else(|e| Err(join_error(e)));
Ok(match r {
Ok(png) => png_result(&png, "rendered frame"),
Err(e) => fail(e),
@@ -272,17 +425,29 @@ impl FilmcraftMcp {
}
}
#[tool(description = "Live app: UI state (tool, workspace, panels, timeline zoom, playback, selection, fps).")]
#[tool(
title = "Live app: UI state",
description = "Live app: UI state (tool, workspace, panels, timeline zoom, playback, selection, fps).",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn ui_inspect(&self) -> Result<CallToolResult, McpError> {
self.ui("ui.inspect", json!({})).await
}
#[tool(description = "Live app: every on-screen interactive element with id, label and rect (points). Use ids with ui_click/ui_drag.")]
#[tool(
title = "Live app: on-screen elements",
description = "Live app: every on-screen interactive element with id, label and rect (points). Use ids with ui_click/ui_drag.",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn ui_elements(&self, Parameters(p): Parameters<ElementsParams>) -> Result<CallToolResult, McpError> {
self.ui("ui.elements", json!({"prefix": p.prefix.unwrap_or_default()})).await
}
#[tool(description = "Live app: click an element by id or at x,y (points). Supports right/middle button, double-click (count=2) and modifiers.")]
#[tool(
title = "Live app: click",
description = "Live app: click an element by id or at x,y (points). Supports right/middle button, double-click (count=2) and modifiers.",
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
)]
async fn ui_click(&self, Parameters(p): Parameters<ClickParams>) -> Result<CallToolResult, McpError> {
let mut v = json!({"button": p.button, "count": p.count, "modifiers": p.modifiers});
if let Some(id) = p.id {
@@ -295,23 +460,37 @@ impl FilmcraftMcp {
}
#[tool(
description = "Live app: press-drag-release from one point/element to another (move clips, trim edges, scrub, resize panels, drop project items onto tracks)."
title = "Live app: drag",
description = "Live app: press-drag-release from one point/element to another (move clips, trim edges, scrub, resize panels, drop project items onto tracks).",
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
)]
async fn ui_drag(&self, Parameters(p): Parameters<DragParams>) -> Result<CallToolResult, McpError> {
self.ui("ui.drag", json!({"from": p.from, "to": p.to, "steps": p.steps, "modifiers": p.modifiers})).await
}
#[tool(description = "Live app: press a key or shortcut (e.g. `Space`, `L`, `Cmd+K`, `Shift+Delete`, `Cmd+Z`).")]
#[tool(
title = "Live app: press a key",
description = "Live app: press a key or shortcut (e.g. `Space`, `L`, `Cmd+K`, `Shift+Delete`, `Cmd+Z`).",
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
)]
async fn ui_key(&self, Parameters(p): Parameters<KeyParams>) -> Result<CallToolResult, McpError> {
self.ui("ui.key", json!({"key": p.key})).await
}
#[tool(description = "Live app: type text into the focused field.")]
#[tool(
title = "Live app: type text",
description = "Live app: type text into the focused field.",
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
)]
async fn ui_type(&self, Parameters(p): Parameters<TypeParams>) -> Result<CallToolResult, McpError> {
self.ui("ui.type", json!({"text": p.text})).await
}
#[tool(description = "Live app: screenshot of the window or one panel (PNG).")]
#[tool(
title = "Live app: screenshot",
description = "Live app: screenshot of the window or one panel (PNG).",
annotations(read_only_hint = true, destructive_hint = false, idempotent_hint = true, open_world_hint = false)
)]
async fn ui_screenshot(&self, Parameters(p): Parameters<ShotParams>) -> Result<CallToolResult, McpError> {
match self.bridge_client() {
Some(b) => self.screenshot(&b, p.panel, p.max_side.unwrap_or(1600)).await,
@@ -320,7 +499,9 @@ impl FilmcraftMcp {
}
#[tool(
description = "Live app: call any control-channel method directly (ui.set, ui.scroll, ui.timeline.hit, ui.timeline.locate, ui.playback, ui.panel.show, ui.menu.list, …)."
title = "Live app: control-channel call",
description = "Live app: call any control-channel method directly (ui.set, ui.scroll, ui.timeline.hit, ui.timeline.locate, ui.playback, ui.panel.show, ui.menu.list, …).",
annotations(read_only_hint = false, destructive_hint = true, idempotent_hint = false, open_world_hint = false)
)]
async fn ui_control(&self, Parameters(p): Parameters<ControlParams>) -> Result<CallToolResult, McpError> {
self.ui(&p.method, p.params.unwrap_or(json!({}))).await
@@ -350,12 +531,104 @@ impl FilmcraftMcp {
}
}
#[tool_handler(
router = self.tool_router,
name = "filmcraft",
instructions = "FilmCraft video editor (Premiere Pro-class). Every edit is an engine command: `command_list` to discover ids/params, `command_run` to execute (undoable). `project_inspect`/`sequence_inspect` return ids you can pass to commands; `render_frame` shows the result. In bridge mode the `ui_*` tools drive the live app: `ui_elements` lists clickable ids, `ui_click`/`ui_drag`/`ui_key` operate it, `ui_screenshot` shows it. Time is in ticks: 254016000000 per second (commands also accept `seconds`, `frame` or `timecode`)."
)]
impl ServerHandler for FilmcraftMcp {}
impl FilmcraftMcp {
/// The first argument `tool` doesn't take, as a message naming the ones it does.
fn unknown_arg(&self, tool: &str, args: Option<&rmcp::model::JsonObject>) -> Option<String> {
let def = self.tool_router.get(tool)?;
let props = def.input_schema.get("properties").and_then(Value::as_object).cloned().unwrap_or_default();
let bad = args?.keys().find(|k| !props.contains_key(*k))?;
let accepted: Vec<&str> = props.keys().map(String::as_str).collect();
let accepted = if accepted.is_empty() { "no arguments".to_string() } else { accepted.join(", ") };
Some(format!("unknown argument \"{bad}\" for {tool}; expected: {accepted}"))
}
}
const INSTRUCTIONS: &str = "FilmCraft video editor (Premiere Pro-class). Every edit is an engine command: `command_list` to discover ids/params, `command_run` to execute (undoable; `command_batch` runs several). `doc_inspect` (or `project_inspect`/`sequence_inspect`) returns ids you can pass to commands; `render_preview` shows the result. In bridge mode the `ui_*` tools drive the live app: `ui_elements` lists clickable ids, `ui_click`/`ui_drag`/`ui_key` operate it, `ui_screenshot` shows it. Time is in ticks: 254016000000 per second (commands also accept `seconds`, `frame` or `timecode`).";
/// Resources: the project (as `doc_inspect`) and the command catalog (as `command_list`).
const DOCUMENT_URI: &str = "filmcraft://document";
const COMMANDS_URI: &str = "filmcraft://commands";
/// MCP 2026-07-28 clients (current Claude Code) require `ttlMs` and `cacheScope` on list and read
/// results; rmcp fills `resultType` and leaves the cache hints to the server, as the generated
/// `tools/list` does. The resource list never changes; the document does, so reads aren't cached.
fn cache_hints(context: &rmcp::service::RequestContext<rmcp::RoleServer>, ttl_ms: u64) -> Option<(u64, rmcp::model::CacheScope)> {
context.protocol_version().is_some_and(|v| v >= rmcp::model::ProtocolVersion::V_2026_07_28).then_some((ttl_ms, rmcp::model::CacheScope::Private))
}
#[tool_handler(router = self.tool_router)]
impl ServerHandler for FilmcraftMcp {
fn get_info(&self) -> rmcp::model::ServerConfig {
rmcp::model::ServerConfig::new(rmcp::model::ServerCapabilities::builder().enable_tools().enable_resources().build())
.with_server_info(rmcp::model::Implementation::new("filmcraft", env!("CARGO_PKG_VERSION")))
.with_instructions(INSTRUCTIONS.to_string())
}
/// Strict arguments and long exports in front of the tool router.
async fn call_tool(
&self,
request: rmcp::model::CallToolRequestParams,
context: rmcp::service::RequestContext<rmcp::RoleServer>,
) -> Result<rmcp::model::CallToolResponse, McpError> {
if let Some(m) = self.unknown_arg(&request.name, request.arguments.as_ref()) {
return Err(McpError::invalid_params(m, None));
}
// A blocking export reports progress and can be cancelled (docs/agents.md § Long exports).
if request.name == "command_run"
&& matches!(&*self.backend, Backend::Headless(_))
&& let Some(a) = &request.arguments
&& let Some(id) = a.get("id").and_then(Value::as_str)
{
let params = a.get("params").cloned().filter(Value::is_object).unwrap_or_else(|| json!({}));
if crate::long_job::is_long(id, &params) {
return Ok(self.run_long(id, params, &context).await.into());
}
}
let tcc = rmcp::handler::server::tool::ToolCallContext::new(self, request, context);
self.tool_router.call(tcc).await
}
async fn list_resources(
&self,
_request: Option<rmcp::model::PaginatedRequestParams>,
context: rmcp::service::RequestContext<rmcp::RoleServer>,
) -> Result<rmcp::model::ListResourcesResult, McpError> {
use rmcp::model::Resource;
let mut r = rmcp::model::ListResourcesResult::with_all_items(vec![
Resource::new(DOCUMENT_URI, "document")
.with_title("Project")
.with_description("The project tree and the active sequence (same as doc_inspect).")
.with_mime_type("application/json"),
Resource::new(COMMANDS_URI, "commands")
.with_title("Command catalog")
.with_description("Every command with id, label, menu path, shortcut, params and enabled state (same as command_list).")
.with_mime_type("application/json"),
]);
if let Some((ttl, scope)) = cache_hints(&context, 600_000) {
r = r.with_ttl_ms(ttl).with_cache_scope(scope);
}
Ok(r)
}
async fn read_resource(
&self,
request: rmcp::model::ReadResourceRequestParams,
context: rmcp::service::RequestContext<rmcp::RoleServer>,
) -> Result<rmcp::model::ReadResourceResponse, McpError> {
let v = match request.uri.as_str() {
DOCUMENT_URI => self.document().await,
COMMANDS_URI => self.run("command.list", json!({})).await,
other => return Err(McpError::resource_not_found(format!("resource not found: {other}"), None)),
}
.map_err(|e| McpError::internal_error(e.to_string(), None))?;
let text = serde_json::to_string_pretty(&v).map_err(|e| McpError::internal_error(e.to_string(), None))?;
let mut r = rmcp::model::ReadResourceResult::new(vec![rmcp::model::ResourceContents::text(text, request.uri).with_mime_type("application/json")]);
if let Some((ttl, scope)) = cache_hints(&context, 0) {
r = r.with_ttl_ms(ttl).with_cache_scope(scope);
}
Ok(r.into())
}
}
#[cfg(test)]
mod tests {
@@ -371,4 +644,249 @@ mod tests {
let r = m.render_frame(Parameters(RenderParams { seconds: Some(1.0), max_side: Some(320) })).await.unwrap();
assert_ne!(r.is_error, Some(true));
}
/// One request/response exchange over `serve_io`, as an MCP client sees it.
struct Client {
tx: tokio::io::DuplexStream,
rx: tokio::io::Lines<tokio::io::BufReader<tokio::io::DuplexStream>>,
}
impl Client {
fn start(s: Session) -> Self {
use tokio::io::AsyncBufReadExt;
let (tx, server_in) = tokio::io::duplex(1 << 22);
let (server_out, rx) = tokio::io::duplex(1 << 22);
tokio::spawn(FilmcraftMcp::headless(s).serve_io(server_in, server_out));
Self { tx, rx: tokio::io::BufReader::new(rx).lines() }
}
async fn send(&mut self, line: &str) {
use tokio::io::AsyncWriteExt;
self.tx.write_all(format!("{line}\n").as_bytes()).await.unwrap();
}
async fn next(&mut self) -> Value {
serde_json::from_str(&self.rx.next_line().await.unwrap().unwrap()).unwrap()
}
async fn ask(&mut self, msg: Value) -> Value {
self.send(&msg.to_string()).await;
self.next().await
}
async fn init(&mut self) -> Value {
let r = self.ask(json!({"jsonrpc":"2.0","id":0,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"t","version":"1"}}})).await;
self.send(&json!({"jsonrpc":"2.0","method":"notifications/initialized"}).to_string()).await;
r
}
async fn call(&mut self, id: u64, name: &str, arguments: Value) -> Value {
self.ask(json!({"jsonrpc":"2.0","id":id,"method":"tools/call","params":{"name":name,"arguments":arguments}})).await
}
}
fn demo() -> Session {
let mut s = Session::default();
s.execute("file.openDemoProject", json!({})).unwrap();
s
}
/// Every listed tool has a title and the four hints, and the core tools are listed.
#[tokio::test(flavor = "multi_thread")]
async fn tools_are_titled_and_annotated() {
let mut c = Client::start(demo());
let init = c.init().await;
assert_eq!(init["result"]["serverInfo"]["name"], "filmcraft");
let tools = c.ask(json!({"jsonrpc":"2.0","id":1,"method":"tools/list"})).await["result"]["tools"].as_array().cloned().unwrap();
for t in &tools {
assert!(t["title"].is_string(), "{} has no title", t["name"]);
for h in ["readOnlyHint", "destructiveHint", "idempotentHint", "openWorldHint"] {
assert!(t["annotations"][h].is_boolean(), "{} lacks {h}", t["name"]);
}
}
let names: Vec<&str> = tools.iter().filter_map(|t| t["name"].as_str()).collect();
for want in ["command_list", "command_run", "command_batch", "doc_inspect", "render_preview", "project_inspect", "render_frame", "ui_click"] {
assert!(names.contains(&want), "{want} not listed");
}
let ro = |n: &str| tools.iter().find(|t| t["name"] == n).map(|t| t["annotations"]["readOnlyHint"].clone());
assert_eq!((ro("command_list"), ro("command_run")), (Some(json!(true)), Some(json!(false))));
}
/// Strict arguments, malformed JSON, batch, doc_inspect and render_preview over the raw line
/// protocol.
#[tokio::test(flavor = "multi_thread")]
async fn strict_arguments_and_parse_errors() {
let mut c = Client::start(demo());
c.init().await;
let r = c.call(1, "command_list", json!({"filter": "razor", "enabled_only": true})).await;
assert!(r["result"]["content"][0]["text"].as_str().unwrap().contains("timeline.razor"), "{r}");
// an unknown argument names itself and the accepted ones
let r = c.call(2, "command_list", json!({"filtr": "x"})).await;
assert_eq!(r["error"]["code"], -32602, "{r}");
let m = r["error"]["message"].as_str().unwrap();
assert!(m.contains("\"filtr\"") && m.contains("filter") && m.contains("enabled_only"), "{m}");
let r = c.call(3, "doc_inspect", json!({"verbose": true})).await;
assert_eq!(r["error"]["code"], -32602, "{r}");
// command params stay free-form
let r = c.call(4, "command_run", json!({"id": "playhead.set", "params": {"seconds": 0.5}})).await;
assert_eq!(r["result"]["isError"], false, "{r}");
// malformed JSON: a parse error, and the server keeps serving
c.send("{\"jsonrpc\": \"2.0\", \"id\": 5, BROKEN").await;
let r = c.next().await;
assert_eq!((r["error"]["code"].as_i64(), r["id"].is_null()), (Some(-32700), true), "{r}");
let r = c.ask(json!({"jsonrpc":"2.0","id":6,"method":"ping"})).await;
assert!(r["result"].is_object(), "{r}");
// batch: stops at the first failure and reports both
let r =
c.call(7, "command_batch", json!({"steps": [{"id": "playhead.set", "params": {"seconds": 0.5}}, {"id": "no.such"}, {"id": "playhead.set"}]})).await;
assert_eq!(r["result"]["isError"], true, "{r}");
let v: Value = serde_json::from_str(r["result"]["content"][0]["text"].as_str().unwrap()).unwrap();
assert_eq!((v["completed"].as_u64(), v["failed"].as_u64(), v["results"].as_array().map(Vec::len)), (Some(1), Some(1), Some(2)), "{v}");
let r = c.call(8, "doc_inspect", json!({})).await;
let v: Value = serde_json::from_str(r["result"]["content"][0]["text"].as_str().unwrap()).unwrap();
assert!(v["project"].is_object() && v["sequence"].is_object(), "{v}");
let r = c.call(9, "render_preview", json!({"max_side": 64})).await;
assert_eq!(r["result"]["content"][0]["type"], "image", "{r}");
}
/// `filmcraft://document` and `filmcraft://commands` are listed and readable as JSON.
#[tokio::test(flavor = "multi_thread")]
async fn resources_document_and_commands() {
let mut c = Client::start(demo());
let init = c.init().await;
assert!(init["result"]["capabilities"]["resources"].is_object(), "{init}");
assert!(init["result"]["instructions"].as_str().unwrap().contains("doc_inspect"), "{init}");
let r = c.ask(json!({"jsonrpc":"2.0","id":1,"method":"resources/list"})).await;
let uris: Vec<&str> = r["result"]["resources"].as_array().unwrap().iter().filter_map(|x| x["uri"].as_str()).collect();
assert_eq!(uris, [DOCUMENT_URI, COMMANDS_URI], "{r}");
let r = c.ask(json!({"jsonrpc":"2.0","id":2,"method":"resources/read","params":{"uri":DOCUMENT_URI}})).await;
assert_eq!(r["result"]["contents"][0]["mimeType"], "application/json", "{r}");
let v: Value = serde_json::from_str(r["result"]["contents"][0]["text"].as_str().unwrap()).unwrap();
assert!(v["project"].is_object() && v["sequence"].is_object(), "{v}");
let r = c.ask(json!({"jsonrpc":"2.0","id":3,"method":"resources/read","params":{"uri":COMMANDS_URI}})).await;
assert!(r["result"]["contents"][0]["text"].as_str().unwrap().contains("timeline.razor"), "{r}");
let r = c.ask(json!({"jsonrpc":"2.0","id":4,"method":"resources/read","params":{"uri":"filmcraft://nope"}})).await;
assert!(r["error"]["code"].is_i64(), "{r}");
}
/// MCP 2026-07-28 clients (current Claude Code) negotiate per request and reject list and read
/// results without `resultType`, `ttlMs` and `cacheScope`; sessions that negotiated an older
/// revision through `initialize` get the old shape.
#[tokio::test(flavor = "multi_thread")]
async fn modern_clients_get_result_type_and_cache_hints() {
let mut c = Client::start(Session::default());
let meta = json!({
"io.modelcontextprotocol/protocolVersion": "2026-07-28",
"io.modelcontextprotocol/clientInfo": {"name": "t", "version": "1"},
"io.modelcontextprotocol/clientCapabilities": {},
});
let r = c.ask(json!({"jsonrpc":"2.0","id":1,"method":"server/discover","params":{"_meta":meta}})).await;
assert!(r["result"]["supportedVersions"].as_array().unwrap().contains(&json!("2026-07-28")), "{r}");
for (id, method, params) in [
(2, "tools/list", json!({"_meta": meta})),
(3, "resources/list", json!({"_meta": meta})),
(4, "resources/read", json!({"_meta": meta, "uri": DOCUMENT_URI})),
] {
let r = c.ask(json!({"jsonrpc":"2.0","id":id,"method":method,"params":params})).await;
assert_eq!(r["result"]["resultType"], "complete", "{method}: {r}");
assert!(r["result"]["ttlMs"].is_u64(), "{method}: {r}");
assert!(matches!(r["result"]["cacheScope"].as_str(), Some("public" | "private")), "{method}: {r}");
}
let mut c = Client::start(Session::default());
c.init().await;
for (id, method, params) in [(2, "tools/list", json!({})), (3, "resources/list", json!({})), (4, "resources/read", json!({"uri": DOCUMENT_URI}))] {
let r = c.ask(json!({"jsonrpc":"2.0","id":id,"method":method,"params":params})).await;
assert!(r["result"].is_object() && r["result"].get("resultType").is_none() && r["result"].get("ttlMs").is_none(), "{method}: {r}");
}
}
/// docs/agents.md § Long exports: a blocking export reports `notifications/progress` for its
/// token (none without one), answers `ping` meanwhile, and `notifications/cancelled` stops
/// it, deletes the partial file and sends no response.
#[tokio::test(flavor = "multi_thread")]
async fn export_progress_and_cancel() {
let dir = std::env::temp_dir().join(format!("filmcraft-mcp-progress-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let mut c = Client::start(demo());
c.init().await;
let export = |id: u64, name: &str, token: Option<Value>, seconds: f64| {
let path = dir.join(name).to_string_lossy().to_string();
let params = json!({"path": path, "format": "h264", "width": 640, "height": 360, "audio": false, "range": "custom", "startSeconds": 0, "endSeconds": seconds, "wait": true});
let mut call =
json!({"jsonrpc":"2.0","id":id,"method":"tools/call","params":{"name":"command_run","arguments":{"id":"file.exportMedia","params":params}}});
if let Some(t) = token {
call["params"]["_meta"] = json!({"progressToken": t});
}
call.to_string()
};
// no token: no notifications, the same response as before
c.send(&export(1, "quiet.mp4", None, 0.5)).await;
let m = c.next().await;
assert_eq!((m["id"].as_u64(), &m["result"]["isError"]), (Some(1), &json!(false)), "{m}");
// progress, strictly increasing, with a total; ping answered while the export runs
c.send(&export(2, "a.mp4", Some(json!("tok")), 1.0)).await;
let (mut last, mut notes, mut pinged, mut pong) = (-1.0, 0, false, false);
let done = loop {
let m = c.next().await;
if m["method"] == "notifications/progress" {
assert_eq!(m["params"]["progressToken"], "tok", "{m}");
let p = m["params"]["progress"].as_f64().unwrap();
assert!(p > last && m["params"]["total"].as_f64().unwrap() >= p, "{m}");
last = p;
notes += 1;
if !pinged {
pinged = true;
c.send(r#"{"jsonrpc":"2.0","id":3,"method":"ping"}"#).await;
}
} else if m["id"] == 3 {
pong = true;
} else if m["id"] == 2 {
break m;
}
};
assert_eq!(done["result"]["isError"], false, "{done}");
assert!(notes >= 2 && pong, "{notes} notifications, ping answered during the export: {pong}");
assert!(dir.join("a.mp4").exists());
// cancel after the first notification: no response, no partial file
c.send(&export(4, "b.mp4", Some(json!(7)), 8.0)).await;
loop {
let m = c.next().await;
assert_ne!(m["id"], 4, "finished before the cancel: {m}");
if m["method"] == "notifications/progress" {
break;
}
}
c.send(r#"{"jsonrpc":"2.0","method":"notifications/cancelled","params":{"requestId":4}}"#).await;
let t0 = std::time::Instant::now();
while dir.join("b.mp4").exists() || t0.elapsed() < std::time::Duration::from_millis(500) {
assert!(t0.elapsed().as_secs() < 30, "partial file still there");
tokio::time::sleep(std::time::Duration::from_millis(50)).await;
}
c.send(r#"{"jsonrpc":"2.0","id":5,"method":"tools/call","params":{"name":"command_run","arguments":{"id":"jobs.list"}}}"#).await;
let m = loop {
let m = c.next().await;
assert_ne!(m["id"], 4, "no response for a cancelled request: {m}");
if m["id"] == 5 {
break m;
}
};
let jobs: Value = serde_json::from_str(m["result"]["content"][0]["text"].as_str().unwrap()).unwrap();
let last = jobs.as_array().and_then(|a| a.last()).cloned().unwrap_or_default();
assert_eq!(last["result"]["error"], "cancelled", "{jobs}");
assert!(!dir.join("b.mp4").exists());
let _ = std::fs::remove_dir_all(&dir);
}
/// A command that panicked used to poison the session lock, and every later call failed with
/// "session lock poisoned" until the server was restarted.
#[tokio::test(flavor = "multi_thread")]
async fn a_poisoned_session_lock_keeps_serving() {
let m = FilmcraftMcp::headless(demo());
let Backend::Headless(s) = &*m.backend else { panic!("headless backend") };
let s = s.clone();
let _ = std::thread::spawn(move || {
let _g = s.lock().unwrap();
panic!("a command panicked");
})
.join();
let v = m.run("sequence.inspect", json!({})).await.unwrap();
assert!(v.is_object(), "{v}");
let r = m.render_frame(Parameters(RenderParams { seconds: Some(0.5), max_side: Some(64) })).await.unwrap();
assert_ne!(r.is_error, Some(true));
}
}
+6 -2
View File
@@ -237,12 +237,16 @@ fn filter_rows<const W: usize>(buf: &[i32; PAD_STRIDE * PAD_STRIDE], taps: &[(is
}
for i in 0..h {
let ci = ((i + 2) * PAD_STRIDE + 2) as isize;
let x: [i16; W] = b16[ci as usize..ci as usize + W].try_into().expect("centre row");
let Some(&x) = b16.get(ci as usize..).and_then(|r| r.first_chunk::<W>()) else {
return;
};
let (mut sum, mut max, mut min) = ([0i16; W], x, x);
for &(off, weight, thr, adj) in taps {
let (weight, thr) = (weight as i16, thr as i16);
let start = (ci + off) as usize;
let p: &[i16; W] = b16[start..start + W].try_into().expect("tap row");
let Some(p) = b16.get(start..).and_then(|r| r.first_chunk::<W>()) else {
return;
};
for j in 0..W {
// constrain(): .max / .min instead of clamp (whose bound check panics), and
// the sign applied branch-free, so the loop vectorises.
+9 -5
View File
@@ -72,10 +72,12 @@ impl<T: Clone> Slot<T> {
fn wait(&self) -> Result<T> {
let mut g = self.value.lock().unwrap_or_else(|e| e.into_inner());
while g.is_none() {
loop {
if let Some(v) = g.as_ref() {
return v.clone();
}
g = self.cv.wait(g).unwrap_or_else(|e| e.into_inner());
}
g.as_ref().expect("slot value").clone()
}
}
@@ -308,7 +310,7 @@ impl Decoder {
if !o.is_ready() && self.out.len() <= max_queue {
break;
}
let o = self.out.pop_front().expect("front");
let Some(o) = self.out.pop_front() else { break };
pics.push(o.take()?);
}
parsed?;
@@ -358,7 +360,9 @@ impl Decoder {
match last {
Some((lt, lsid)) if lt == t => {
if sid >= lsid {
*sel.last_mut().expect("picture") = o;
if let Some(l) = sel.last_mut() {
*l = o;
}
last = Some((t, sid));
}
}
@@ -612,7 +616,7 @@ impl Decoder {
self.in_flight.push_back(slot);
// Bound the frames in flight (memory): wait for the oldest.
while self.in_flight.len() > self.max_in_flight {
let s = self.in_flight.pop_front().expect("in flight");
let Some(s) = self.in_flight.pop_front() else { break };
let _ = s.wait();
}
self.in_flight.retain(|s| !s.is_set());
+17 -5
View File
@@ -180,7 +180,11 @@ impl TileDecoder<'_, '_> {
}
fn ref_plane(&self, ref_idx: i32, plane: usize) -> &Plane {
if ref_idx < 0 { &self.t.cur.planes[plane] } else { &self.fs.refs[ref_idx as usize].as_ref().expect("reference frame").buf.planes[plane] }
// A missing reference (rejected earlier for valid streams) predicts from the current frame.
match usize::try_from(ref_idx).ok().and_then(|i| self.fs.refs.get(i)).and_then(|r| r.as_ref()) {
Some(r) => &r.buf.planes[plane],
None => &self.t.cur.planes[plane],
}
}
/// Block inter prediction process (7.11.3.4) into `pred` (stride PRED_STRIDE); `tmp` holds
@@ -672,12 +676,16 @@ fn hpass<const W: usize>(src: &[u16], hf: &[i16; 8], out: &mut [i32], half: i32,
let mut acc = [half; W];
for t in 0..8 {
let f = hf[t] as i32;
let p: &[u16; W] = src[t..t + W].try_into().expect("row");
let Some(p) = src.get(t..).and_then(|s| s.first_chunk::<W>()) else {
return;
};
for j in 0..W {
acc[j] += f * p[j] as i32;
}
}
let o: &mut [i32; W] = (&mut out[..W]).try_into().expect("row");
let Some(o) = out.first_chunk_mut::<W>() else {
return;
};
for j in 0..W {
o[j] = acc[j] >> sh;
}
@@ -689,12 +697,16 @@ fn vpass<const W: usize>(rows: &[i32], vf: &[i16; 8], out: &mut [i32], half: i32
let mut acc = [half; W];
for t in 0..8 {
let f = vf[t] as i32;
let p: &[i32; W] = rows[t * W..t * W + W].try_into().expect("row");
let Some(p) = rows.get(t * W..).and_then(|s| s.first_chunk::<W>()) else {
return;
};
for j in 0..W {
acc[j] += f * p[j];
}
}
let o: &mut [i32; W] = (&mut out[..W]).try_into().expect("row");
let Some(o) = out.first_chunk_mut::<W>() else {
return;
};
for j in 0..W {
o[j] = acc[j] >> sh;
}
+1
View File
@@ -5,6 +5,7 @@
// Items used only by stages still being written (inter prediction, post filters).
#![allow(dead_code)]
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
mod bits;
mod cdef;
+1 -1
View File
@@ -76,7 +76,7 @@ pub fn reference(ff: &Path, path: &Path, pix_fmt: &str) -> Vec<u16> {
.expect("run ffmpeg");
assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr));
if pix_fmt.ends_with("le") {
out.stdout.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
out.stdout.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
} else {
out.stdout.iter().map(|&b| b as u16).collect()
}
+2
View File
@@ -5,6 +5,8 @@
//! - [`unescape_rbsp`] / [`escape_rbsp`]: remove/insert H.264/HEVC emulation-prevention bytes.
//! - [`annexb_nals`] / [`length_prefixed_nals`]: split Annex-B byte streams and AVCC/HVCC samples into NAL units.
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
use std::fmt;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+4 -2
View File
@@ -15,6 +15,8 @@
//! - **Burn-in:** [`burn`] lays a caption out with the track style and draws it with the
//! `filmcraft-text` engine (Inter SemiBold, shaped and kerned).
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
pub mod burn;
pub mod cea608;
pub mod mcc;
@@ -212,7 +214,7 @@ pub fn parse(bytes: &[u8], format: Format) -> Result<Document> {
Format::Scc => scc::parse(&text),
Format::Mcc => mcc::parse(&text),
Format::Ttml | Format::Dfxp => ttml::parse(&text),
Format::Stl => unreachable!("handled above"),
Format::Stl => stl::parse(bytes),
}
}
@@ -253,7 +255,7 @@ pub fn decode_text(bytes: &[u8]) -> String {
} else if bytes.starts_with(&[0xff, 0xfe]) || bytes.starts_with(&[0xfe, 0xff]) {
let le = bytes[0] == 0xff;
let units: Vec<u16> =
bytes[2..].chunks_exact(2).map(|c| if le { u16::from_le_bytes([c[0], c[1]]) } else { u16::from_be_bytes([c[0], c[1]]) }).collect();
bytes[2..].as_chunks::<2>().0.iter().map(|c| if le { u16::from_le_bytes([c[0], c[1]]) } else { u16::from_be_bytes([c[0], c[1]]) }).collect();
String::from_utf16_lossy(&units)
} else {
match std::str::from_utf8(bytes) {
+3 -3
View File
@@ -411,10 +411,10 @@ impl Dec708 {
}
fn put(&mut self, c: char) {
let w = self.w();
if w.rows.is_empty() {
w.rows.push(String::new());
match w.rows.last_mut() {
Some(row) => row.push(c),
None => w.rows.push(c.to_string()),
}
w.rows.last_mut().expect("row").push(c);
}
fn bitmap(&mut self, m: u8, f: impl Fn(&mut Win)) {
for (i, w) in self.win.iter_mut().enumerate() {
+4 -2
View File
@@ -251,7 +251,9 @@ pub fn parse(bytes: &[u8]) -> Result<Document> {
if !sniff(bytes) {
return Err(Error::NotFormat("EBU STL"));
}
let rate = rate_of(&bytes[3..11]).expect("sniffed");
let Some(rate) = bytes.get(3..11).and_then(rate_of) else {
return Err(Error::NotFormat("EBU STL"));
};
let gsi = &bytes[..GSI];
// start of programme (HHMMSSFF, ASCII)
let tcp = ascii_field(&gsi[256..264]);
@@ -277,7 +279,7 @@ pub fn parse(bytes: &[u8]) -> Result<Document> {
}
}
};
for blk in bytes[GSI..].chunks_exact(TTI) {
for blk in bytes[GSI..].as_chunks::<TTI>().0 {
let sn = u16::from_le_bytes([blk[1], blk[2]]);
let ebn = blk[3];
let cf = blk[15];
+71 -16
View File
@@ -103,7 +103,7 @@ fn inline(text: &str) -> String {
rest = &after[end + 1..];
continue;
}
let ch = rest.chars().next().expect("non-empty");
let Some(ch) = rest.chars().next() else { break };
if ch == '\n' {
out.push_str("<br/>");
} else {
@@ -184,28 +184,45 @@ struct Timing {
tick_rate: i64,
}
fn positive_integer(s: &str) -> Option<i64> {
let s = s.trim();
if s.is_empty() || !s.bytes().all(|b| b.is_ascii_digit()) {
return None;
}
s.parse::<i64>().ok().filter(|v| *v > 0)
}
fn parse_ratio(s: &str) -> Option<(i64, i64)> {
let mut it = s.split_whitespace().map(|x| x.parse::<i64>().ok());
let mut it = s.split_whitespace().map(positive_integer);
let a = it.next()??;
let b = it.next().flatten().unwrap_or(1);
(a > 0 && b > 0).then_some((a, b))
let b = it.next()??;
it.next().is_none().then_some((a, b))
}
fn attr<'a>(n: roxmltree::Node<'a, '_>, local: &str) -> Option<&'a str> {
n.attributes().find(|a| a.name() == local).map(|a| a.value())
}
fn timing(root: roxmltree::Node) -> Timing {
let base = attr(root, "frameRate").and_then(|v| v.trim().parse::<i64>().ok()).filter(|v| *v > 0).unwrap_or(30);
let (mn, md) = attr(root, "frameRateMultiplier").and_then(parse_ratio).unwrap_or((1, 1));
let rate = FrameRate::new(base * mn, md);
let sub = attr(root, "subFrameRate").and_then(|v| v.trim().parse::<i64>().ok()).filter(|v| *v > 0).unwrap_or(1);
let tick_rate = attr(root, "tickRate").and_then(|v| v.trim().parse::<i64>().ok()).filter(|v| *v > 0).unwrap_or(if attr(root, "frameRate").is_some() {
base * sub
} else {
1
});
Timing { rate, base, sub_frame_rate: sub, tick_rate }
fn timing(root: roxmltree::Node) -> Result<Timing> {
let invalid =
|name: &str| Error::Syntax { line: root.document().text_pos_at(root.range().start).row as usize, msg: format!("invalid TTML timing parameter {name}") };
let integer = |name: &str, default| match attr(root, name) {
Some(v) => positive_integer(v).ok_or_else(|| invalid(name)),
None => Ok(default),
};
let base = integer("frameRate", 30)?;
let (mn, md) = match attr(root, "frameRateMultiplier") {
Some(v) => parse_ratio(v).ok_or_else(|| invalid("frameRateMultiplier"))?,
None => (1, 1),
};
let rate = FrameRate::new(base.checked_mul(mn).ok_or_else(|| invalid("frameRateMultiplier"))?, md);
let sub = integer("subFrameRate", 1)?;
let tick_rate = match attr(root, "tickRate") {
Some(v) => positive_integer(v).ok_or_else(|| invalid("tickRate"))?,
None if attr(root, "frameRate").is_some() => base.checked_mul(sub).ok_or_else(|| invalid("subFrameRate"))?,
None => 1,
};
Ok(Timing { rate, base, sub_frame_rate: sub, tick_rate })
}
/// Seconds as an exact decimal string → ticks.
@@ -383,7 +400,7 @@ pub fn parse(text: &str) -> Result<Document> {
if root.tag_name().name() != "tt" {
return Err(Error::NotFormat("TTML"));
}
let tm = timing(root);
let tm = timing(root)?;
let mut styles = std::collections::HashMap::new();
for s in root.descendants().filter(|n| n.is_element() && n.tag_name().name() == "style") {
if let Some(id) = s.attributes().find(|a| a.name() == "id").map(|a| a.value()) {
@@ -478,6 +495,44 @@ mod tests {
Timing { rate, base: rate.timecode_base(), sub_frame_rate: 1, tick_rate: 1 }
}
#[test]
fn invalid_timing_parameters_return_an_error() {
for params in [
r#"ttp:frameRate="0""#,
r#"ttp:frameRate="-25""#,
r#"ttp:frameRate="""#,
r#"ttp:subFrameRate="0""#,
r#"ttp:tickRate="-1""#,
r#"ttp:frameRateMultiplier="1 0""#,
r#"ttp:frameRateMultiplier="1""#,
r#"ttp:frameRateMultiplier="1 2 3""#,
] {
let xml = format!(
r#"<tt xmlns="http://www.w3.org/ns/ttml" xmlns:ttp="http://www.w3.org/ns/ttml#parameter" {params}><body><div><p begin="1s" end="2s">Hello</p></div></body></tt>"#
);
assert!(crate::parse(xml.as_bytes(), crate::Format::Ttml).is_err(), "{params}");
}
}
#[test]
fn missing_timing_parameters_keep_defaults() {
for (params, begin, end) in [
("", "1s", "2s"),
(r#"ttp:frameRate="25""#, "25f", "50f"),
(r#"ttp:frameRate="30" ttp:subFrameRate="2""#, "60t", "120t"),
(r#"ttp:frameRate="30" ttp:subFrameRate="2" ttp:tickRate="10""#, "10t", "20t"),
(r#"ttp:frameRate="25" ttp:frameRateMultiplier="2 1""#, "50f", "100f"),
] {
let xml = format!(
r#"<tt xmlns="http://www.w3.org/ns/ttml" xmlns:ttp="http://www.w3.org/ns/ttml#parameter" {params}><body><div><p begin="{begin}" end="{end}">Hello</p></div></body></tt>"#
);
let doc = crate::parse(xml.as_bytes(), crate::Format::Ttml).unwrap();
assert_eq!(doc.cues.len(), 1, "{params}");
assert_eq!(doc.cues[0].start, Tick(TICKS_PER_SECOND), "{params}");
assert_eq!(doc.cues[0].end, Tick(2 * TICKS_PER_SECOND), "{params}");
}
}
#[test]
fn time_expressions() {
let t = tm(FrameRate::FPS_25);
+2
View File
@@ -15,6 +15,8 @@
//!
//! Layer L0: no dependencies beyond `std`, no `unsafe`, builds for `wasm32-unknown-unknown`.
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
mod read;
mod write;
+1 -1
View File
@@ -143,7 +143,7 @@ impl<'a> CompoundFile<'a> {
// Mini FAT and mini stream.
if first_minifat <= MAXREGSECT {
let mf = cf.read_chain(first_minifat, None)?;
cf.minifat = mf.chunks_exact(4).map(|c| u32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect();
cf.minifat = mf.as_chunks::<4>().0.iter().map(|c| u32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect();
}
let root = &cf.entries[0];
if root.start <= MAXREGSECT && root.size > 0 {
+1 -1
View File
@@ -36,7 +36,7 @@ log = { workspace = true }
web-time = "1"
image = { version = "0.25", default-features = false, features = ["jpeg"] }
# Bootstrap audio decoding (MPL-2.0, used unmodified); replaced by own decoders over time.
symphonia = { version = "0.5.5", default-features = false, features = ["aac", "mp1", "mp2", "mp3", "flac", "alac", "pcm", "wav", "ogg", "vorbis", "isomp4"] }
symphonia = { version = "0.5.5", default-features = false, features = ["aac", "mp1", "mp2", "mp3", "flac", "alac", "pcm", "wav", "aiff", "ogg", "vorbis", "isomp4"] }
[dev-dependencies]
filmcraft-testkit = { workspace = true }
+31 -9
View File
@@ -33,6 +33,9 @@ pub struct GopStats {
pub skipped: u64,
/// Frames decoded in draft mode (reduced-resolution playback, [`VideoDecoder::set_draft`]).
pub draft: u64,
/// Pictures decoders output (hardware and software; [`crate::hw::hw_stats`] counts the
/// hardware ones).
pub frames: u64,
}
impl GopStats {
@@ -56,6 +59,12 @@ static EVICTED: AtomicU64 = AtomicU64::new(0);
static DECODE_NS: AtomicU64 = AtomicU64::new(0);
static SKIPPED: AtomicU64 = AtomicU64::new(0);
static DRAFT: AtomicU64 = AtomicU64::new(0);
static FRAMES: AtomicU64 = AtomicU64::new(0);
/// Count pictures a decoder call output.
fn count_frames<T>(out: &[T]) {
FRAMES.fetch_add(out.len() as u64, Ordering::Relaxed);
}
/// Run a decoder call, adding its wall time to the process-wide counter.
fn timed<R>(f: impl FnOnce() -> R) -> R {
@@ -76,6 +85,7 @@ pub fn gop_stats() -> GopStats {
decode_ns: DECODE_NS.load(Ordering::Relaxed),
skipped: SKIPPED.load(Ordering::Relaxed),
draft: DRAFT.load(Ordering::Relaxed),
frames: FRAMES.load(Ordering::Relaxed),
}
}
@@ -91,6 +101,7 @@ impl std::ops::Sub for GopStats {
decode_ns: self.decode_ns - o.decode_ns,
skipped: self.skipped - o.skipped,
draft: self.draft - o.draft,
frames: self.frames - o.frames,
}
}
}
@@ -240,8 +251,7 @@ impl GopCache {
}
// evict frames far from the most recent (keep a window around the working position)
while st.bytes > budget && st.frames.len() > 2 {
let first = *st.frames.keys().next().expect("non-empty");
let last = *st.frames.keys().next_back().expect("non-empty");
let (Some(&first), Some(&last)) = (st.frames.keys().next(), st.frames.keys().next_back()) else { break };
let victim = if pts - first > last - pts { first } else { last };
if let Some(v) = st.frames.remove(&victim) {
st.drafts.remove(&victim);
@@ -253,6 +263,7 @@ impl GopCache {
/// Store decoder output (in presentation order) and advance `out_max`.
fn store_output(&self, st: &mut State, out: Vec<crate::video::DecodedFrame>) {
count_frames(&out);
for d in out {
st.out_max = st.out_max.max(d.pts);
self.store(st, d.pts, d.frame, d.draft);
@@ -323,7 +334,7 @@ impl GopCache {
// every sample): step back to a sample the decoder can start from.
while key > 0 {
let data = s.read(key)?;
if st.decoder.as_ref().expect("decoder").is_random_access(&data) != Some(false) {
if st.decoder.as_ref().and_then(|d| d.is_random_access(&data)) != Some(false) {
break;
}
key = s.sync_before(key - 1);
@@ -337,7 +348,10 @@ impl GopCache {
}
let limit = (i.max(st.next) + 64).min(n);
let late = late_before.unwrap_or(i64::MIN).min(want_pts);
st.decoder.as_mut().expect("decoder").set_draft(draft);
let Some(decoder) = st.decoder.as_mut() else {
return Err(CodecError::Decode("no video decoder".into()));
};
decoder.set_draft(draft);
while st.next < limit {
if filmcraft_media::cancel::cancelled() {
// The decoder state stays consistent (`next`, `out_max`): a later request continues.
@@ -345,12 +359,15 @@ impl GopCache {
}
let k = st.next;
let data = s.read(k)?;
if k != i && s.pts(k) < late && st.decoder.as_ref().expect("decoder").is_disposable(&data) {
if k != i && s.pts(k) < late && st.decoder.as_ref().is_some_and(|d| d.is_disposable(&data)) {
st.next += 1;
SKIPPED.fetch_add(1, Ordering::Relaxed);
continue;
}
let out = timed(|| st.decoder.as_mut().expect("decoder").decode(&data, s.pts(k)))?;
let Some(decoder) = st.decoder.as_mut() else {
return Err(CodecError::Decode("no video decoder".into()));
};
let out = timed(|| decoder.decode(&data, s.pts(k)))?;
st.next += 1;
DECODED.fetch_add(1, Ordering::Relaxed);
self.store_output(&mut st, out);
@@ -359,7 +376,7 @@ impl GopCache {
}
}
if st.cached(want_pts, draft).is_none() {
let out = timed(|| st.decoder.as_mut().expect("decoder").flush());
let out = st.decoder.as_mut().map(|d| timed(|| d.flush())).unwrap_or_default();
self.store_output(&mut st, out);
st.next = usize::MAX;
}
@@ -414,6 +431,7 @@ impl GopCache {
let k = *next;
let data = s.read(k)?;
let pics = timed(|| d.decode(&data, s.pts(k)))?;
count_frames(&pics);
*next += 1;
found |= pics.iter().any(|p| p.pts == want_pts);
out.extend(pics);
@@ -422,7 +440,9 @@ impl GopCache {
}
}
if !found {
out.extend(timed(|| d.flush()));
let pics = timed(|| d.flush());
count_frames(&pics);
out.extend(pics);
*next = usize::MAX;
}
Ok(out)
@@ -446,7 +466,9 @@ impl GopCache {
if st.spare.len() < 16 {
st.spare.push(dec);
}
for d in res? {
let res = res?;
count_frames(&res);
for d in res {
self.store(&mut st, d.pts, d.frame, d.draft);
}
st.at_or_before(want_pts, true).ok_or_else(|| CodecError::Decode("frame not produced".into()))
+350
View File
@@ -0,0 +1,350 @@
//! Support for hardware (OS) video decoders registered by `filmcraft-platform`: the
//! Settings ▸ Playback ▸ Hardware decoding switch, the hardware counters `perf.stats` reports, and
//! what such a decoder needs from an `avcC` / `hvcC` sample entry to behave exactly like our
//! software decoder for the same stream (parameter sets, cropping, colour, pixel aspect, reorder
//! depth, random-access and disposable samples).
//!
//! The software decoders stay the reference: a hardware factory declines streams it cannot
//! decode, and a hardware decoder that fails mid-stream switches to
//! [`crate::software_video_decoder`], so registering one never makes a file undecodable.
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use filmcraft_isobmff::{CodecConfig, SampleEntry};
use crate::video::{avcc_length_size, h264_disposable, hevc_disposable, hvcc_length_size_and_tid, sar_par, vui_color};
use crate::{CodecError, Result};
static ENABLED: AtomicBool = AtomicBool::new(true);
/// Settings ▸ Playback ▸ Hardware decoding: `Auto` (true, the default) lets registered hardware
/// decoders take streams they support; `Off` (false) makes every new decoder a software one.
/// Decoders already created keep running: a source picks the change up when it is reopened.
pub fn set_hardware_decoding(on: bool) {
ENABLED.store(on, Ordering::Relaxed);
}
/// Whether hardware decoders may be used for new decoders ([`set_hardware_decoding`]).
pub fn hardware_decoding() -> bool {
ENABLED.load(Ordering::Relaxed)
}
/// Process-wide hardware decoding counters (they only grow; subtract two snapshots to measure an
/// interval). Frames decoded in software are `GopStats::frames - frames`.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub struct HwStats {
/// Pictures output by hardware decoders.
pub frames: u64,
/// Hardware decoders created (one per decoder instance; a seek reuses it).
pub sessions: u64,
/// Streams a hardware factory handed to the software decoder (unsupported profile, size,
/// chroma format or bit depth, no hardware, or the switch is Off).
pub declined: u64,
/// Hardware decoders that failed mid-stream and continued in software.
pub fallbacks: u64,
}
static FRAMES: AtomicU64 = AtomicU64::new(0);
static SESSIONS: AtomicU64 = AtomicU64::new(0);
static DECLINED: AtomicU64 = AtomicU64::new(0);
static FALLBACKS: AtomicU64 = AtomicU64::new(0);
/// The counters so far.
pub fn hw_stats() -> HwStats {
HwStats {
frames: FRAMES.load(Ordering::Relaxed),
sessions: SESSIONS.load(Ordering::Relaxed),
declined: DECLINED.load(Ordering::Relaxed),
fallbacks: FALLBACKS.load(Ordering::Relaxed),
}
}
/// A hardware decoder output `n` pictures.
pub fn note_hw_frames(n: usize) {
FRAMES.fetch_add(n as u64, Ordering::Relaxed);
}
/// A hardware decoder was created.
pub fn note_hw_session() {
SESSIONS.fetch_add(1, Ordering::Relaxed);
}
/// A hardware factory declined a stream it is responsible for.
pub fn note_hw_declined() {
DECLINED.fetch_add(1, Ordering::Relaxed);
}
/// A hardware decoder switched to software mid-stream.
pub fn note_hw_fallback() {
FALLBACKS.fetch_add(1, Ordering::Relaxed);
}
/// The two codecs whose `avcC` / `hvcC` streams hardware decoders take.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum NalCodec {
H264,
Hevc,
}
/// What a decoder needs to know about a length-prefixed H.264 / HEVC stream, from its sample
/// entry's configuration record and active sequence parameter set (parsed with the software
/// decoders' own parsers, so both decoders derive the same values).
#[derive(Clone, Debug)]
pub struct NalStreamInfo {
pub codec: NalCodec,
/// NAL unit length-prefix size of the samples (1, 2 or 4).
pub length_size: usize,
/// HEVC: highest TemporalId (numTemporalLayers - 1), when signalled.
pub highest_tid: Option<u8>,
/// The parameter-set NAL units of the record (header included, emulation prevention kept),
/// in record order: H.264 SPS then PPS; HEVC VPS, SPS, PPS. SEI arrays are left out.
pub parameter_sets: Vec<Vec<u8>>,
/// Coded (decoded) picture size in luma samples.
pub coded: (u32, u32),
/// Output (cropped) rectangle in luma samples: x, y, width, height.
pub crop: (u32, u32, u32, u32),
/// 0 monochrome, 1 4:2:0, 2 4:2:2, 3 4:4:4.
pub chroma_format_idc: u32,
pub bit_depth_luma: u32,
pub bit_depth_chroma: u32,
/// Field (PAFF / MBAFF) coding (H.264 `frame_mbs_only_flag` 0).
pub interlaced: bool,
/// H.264 `profile_idc` / HEVC `general_profile_idc`.
pub profile_idc: u8,
/// Colour of every output picture, as the software decoder reports it.
pub color: filmcraft_color::ColorInfo,
/// Pixel aspect ratio, as the software decoder reports it.
pub par: (u32, u32),
/// Pictures that may precede a picture in decoding order and follow it in output order
/// (H.264 `max_num_reorder_frames`, HEVC `sps_max_num_reorder_pics`).
pub reorder: usize,
}
/// Length-prefixed parameter-set entries (`u16` length + bytes) starting at `pos`.
fn read_sets(rec: &[u8], pos: &mut usize, count: usize, out: &mut Vec<Vec<u8>>) -> Result<()> {
for _ in 0..count {
let len = rec
.get(*pos..*pos + 2)
.map(|b| u16::from_be_bytes([b[0], b[1]]) as usize)
.ok_or_else(|| CodecError::Decode("configuration record truncated".into()))?;
*pos += 2;
let nal = rec.get(*pos..*pos + len).ok_or_else(|| CodecError::Decode("configuration record truncated".into()))?;
*pos += len;
if !nal.is_empty() {
out.push(nal.to_vec());
}
}
Ok(())
}
impl NalStreamInfo {
/// The stream info of an `avcC` / `hvcC` sample entry (`None` for other codecs).
pub fn from_entry(e: &SampleEntry) -> Option<Result<Self>> {
match &e.codec {
CodecConfig::Avc(a) => Some(Self::from_avcc(&a.to_bytes())),
CodecConfig::Hevc(c) => Some(Self::from_hvcc(&c.to_bytes())),
_ => None,
}
}
/// From an `avcC` (AVCDecoderConfigurationRecord) payload.
pub fn from_avcc(avcc: &[u8]) -> Result<Self> {
if avcc.len() < 7 || avcc[0] != 1 {
return Err(CodecError::Decode("bad avcC record".into()));
}
let mut sets = Vec::new();
let mut pos = 6;
read_sets(avcc, &mut pos, (avcc[5] & 0x1f) as usize, &mut sets)?;
let nsps = sets.len();
let npps = *avcc.get(pos).ok_or_else(|| CodecError::Decode("avcC truncated".into()))? as usize;
pos += 1;
read_sets(avcc, &mut pos, npps, &mut sets)?;
let sps_nal = sets.iter().take(nsps).find(|n| n.first().is_some_and(|h| h & 0x1f == 7)).ok_or_else(|| CodecError::Decode("avcC has no SPS".into()))?;
if sets.len() == nsps {
return Err(CodecError::Decode("avcC has no PPS".into()));
}
let sps = filmcraft_h264::params::Sps::parse(&filmcraft_bitstream::unescape_rbsp(sps_nal.get(1..).unwrap_or_default()))
.map_err(|e| CodecError::Decode(e.to_string()))?;
let crop = sps.crop_rect();
let vui = sps.vui.clone().unwrap_or_default();
Ok(Self {
codec: NalCodec::H264,
length_size: avcc_length_size(avcc),
highest_tid: None,
parameter_sets: sets,
coded: (sps.width(), sps.height()),
crop,
chroma_format_idc: sps.chroma_format_idc,
bit_depth_luma: sps.bit_depth_luma,
bit_depth_chroma: sps.bit_depth_chroma,
interlaced: !sps.frame_mbs_only,
profile_idc: sps.profile_idc,
color: vui_color(crop.2, crop.3, vui.matrix_coefficients, vui.transfer_characteristics, vui.full_range),
par: sar_par(vui.sar),
// as the software decoder's DPB (`max_reorder`)
reorder: sps.max_num_reorder_frames().min(sps.max_dpb_frames()),
})
}
/// From an `hvcC` (HEVCDecoderConfigurationRecord) payload.
pub fn from_hvcc(hvcc: &[u8]) -> Result<Self> {
if hvcc.len() < 23 || hvcc[0] != 1 {
return Err(CodecError::Decode("bad hvcC record".into()));
}
let (length_size, highest_tid) = hvcc_length_size_and_tid(hvcc);
let mut sets = Vec::new();
let mut pos = 23;
for _ in 0..hvcc[22] {
let hdr = hvcc.get(pos..pos + 3).ok_or_else(|| CodecError::Decode("hvcC truncated".into()))?;
let (kind, n) = (hdr[0] & 0x3f, u16::from_be_bytes([hdr[1], hdr[2]]) as usize);
pos += 3;
let mut arr = Vec::new();
read_sets(hvcc, &mut pos, n, &mut arr)?;
if (32..=34).contains(&kind) {
sets.extend(arr);
}
}
let nal_type = |n: &[u8]| n.first().map(|h| (h >> 1) & 0x3f);
let sps_nal = sets.iter().find(|n| nal_type(n) == Some(33)).ok_or_else(|| CodecError::Decode("hvcC has no SPS".into()))?;
if !sets.iter().any(|n| nal_type(n) == Some(32)) || !sets.iter().any(|n| nal_type(n) == Some(34)) {
return Err(CodecError::Decode("hvcC lacks a VPS or PPS".into()));
}
let sps = filmcraft_hevc::params::Sps::parse(&filmcraft_bitstream::unescape_rbsp(sps_nal.get(2..).unwrap_or_default()))
.map_err(|e| CodecError::Decode(e.to_string()))?;
let crop = sps.crop_rect();
let vui = sps.vui.clone().unwrap_or_default();
Ok(Self {
codec: NalCodec::Hevc,
length_size,
highest_tid,
parameter_sets: sets,
coded: (sps.width, sps.height),
crop,
chroma_format_idc: sps.chroma_format_idc,
bit_depth_luma: sps.bit_depth_luma,
bit_depth_chroma: sps.bit_depth_chroma,
interlaced: false,
profile_idc: sps.ptl.profile_idc,
color: vui_color(crop.2, crop.3, vui.matrix_coefficients, vui.transfer_characteristics, vui.full_range),
par: sar_par(vui.sar),
reorder: sps.max_num_reorder as usize,
})
}
/// NAL unit types of a sample (stops at the first malformed length prefix).
pub fn nal_types(&self, sample: &[u8]) -> Vec<u8> {
self.nals(sample).into_iter().filter_map(|n| n.first().map(|&h| self.nal_type(h))).collect()
}
/// The NAL units of a length-prefixed sample (stops at the first malformed length prefix).
pub fn nals<'a>(&self, sample: &'a [u8]) -> Vec<&'a [u8]> {
let ls = self.length_size;
let mut out = Vec::new();
let mut pos = 0usize;
while let Some(pre) = sample.get(pos..pos.saturating_add(ls)) {
if !(1..=4).contains(&ls) {
break;
}
let len = pre.iter().fold(0usize, |a, &b| (a << 8) | b as usize);
pos += ls;
let Some(n) = sample.get(pos..pos.saturating_add(len)) else { break };
pos += len;
out.push(n);
}
out
}
/// The NAL unit type in a NAL header byte.
pub fn nal_type(&self, header: u8) -> u8 {
match self.codec {
NalCodec::H264 => header & 0x1f,
NalCodec::Hevc => (header >> 1) & 0x3f,
}
}
/// Whether `t` is a parameter-set NAL unit type (H.264 SPS / PPS / SPS extension, HEVC VPS /
/// SPS / PPS).
pub fn is_parameter_set(&self, t: u8) -> bool {
match self.codec {
NalCodec::H264 => matches!(t, 7 | 8 | 13 | 15),
NalCodec::Hevc => (32..=34).contains(&t),
}
}
/// Whether decoding can restart at `sample` with no earlier state: an IDR access unit (H.264)
/// or an IRAP one (HEVC IDR / CRA / BLA).
pub fn is_irap(&self, sample: &[u8]) -> bool {
self.nal_types(sample).iter().any(|&t| match self.codec {
NalCodec::H264 => t == 5,
NalCodec::Hevc => (16..=23).contains(&t),
})
}
/// As the software decoder's [`crate::VideoDecoder::is_random_access`] for this stream
/// (length-prefixed samples: unknown, the container's sync flags are trusted).
pub fn is_random_access(&self, _sample: &[u8]) -> Option<bool> {
None
}
/// As the software decoder's [`crate::VideoDecoder::is_disposable`] for this stream.
pub fn is_disposable(&self, sample: &[u8]) -> bool {
match self.codec {
NalCodec::H264 => h264_disposable(sample, self.length_size),
NalCodec::Hevc => hevc_disposable(sample, self.length_size, self.highest_tid),
}
}
/// Output picture size (cropped).
pub fn size(&self) -> (u32, u32) {
(self.crop.2, self.crop.3)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn switch_defaults_to_auto() {
// other tests in this process may flip it; only check the round trip
let was = hardware_decoding();
set_hardware_decoding(false);
assert!(!hardware_decoding());
set_hardware_decoding(true);
assert!(hardware_decoding());
set_hardware_decoding(was);
}
#[test]
fn hostile_records_are_errors() {
for rec in [&[][..], &[1, 2, 3], &[1, 0x64, 0, 0x1f, 0xff, 0xe1, 0, 9, 0x67], &[0; 30]] {
assert!(NalStreamInfo::from_avcc(rec).is_err());
assert!(NalStreamInfo::from_hvcc(rec).is_err());
}
}
#[test]
fn nal_splitting_is_bounded() {
let info = NalStreamInfo {
codec: NalCodec::Hevc,
length_size: 4,
highest_tid: None,
parameter_sets: Vec::new(),
coded: (16, 16),
crop: (0, 0, 16, 16),
chroma_format_idc: 1,
bit_depth_luma: 8,
bit_depth_chroma: 8,
interlaced: false,
profile_idc: 1,
color: filmcraft_color::ColorInfo::REC709,
par: (1, 1),
reorder: 0,
};
// CRA (21) then a length that runs past the end
let s = [0, 0, 0, 2, 21 << 1, 1, 0xff, 0xff, 0xff, 0xff, 1];
assert_eq!(info.nal_types(&s), vec![21]);
assert!(info.is_irap(&s));
assert!(!info.is_irap(&[0, 0, 0, 2, 1 << 1, 1]));
}
}
+27 -12
View File
@@ -16,8 +16,11 @@
//! - [`AudioFileSource`]: standalone compressed audio files (MP3, FLAC, AIFF, …).
//! - [`openers`]: the openers to register with the engine's media pool.
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
pub mod audio;
pub mod gop;
pub mod hw;
pub mod mkv;
pub mod mp4;
pub mod mpeg;
@@ -61,20 +64,21 @@ impl From<CodecError> for filmcraft_media::MediaError {
}
}
/// Our own (pure-Rust) decoders, in the order they are tried.
const BUILTIN_FACTORIES: [VideoDecoderFactory; 8] = [
video::h264_factory,
video::hevc_factory,
video::vp9_factory,
video::av1_factory,
video::prores_factory,
video::dnx_factory,
video::mjpeg_factory,
video::mpeg2_factory,
];
fn factories() -> &'static RwLock<Vec<VideoDecoderFactory>> {
static F: std::sync::OnceLock<RwLock<Vec<VideoDecoderFactory>>> = std::sync::OnceLock::new();
F.get_or_init(|| {
RwLock::new(vec![
video::h264_factory,
video::hevc_factory,
video::vp9_factory,
video::av1_factory,
video::prores_factory,
video::dnx_factory,
video::mjpeg_factory,
video::mpeg2_factory,
])
})
F.get_or_init(|| RwLock::new(BUILTIN_FACTORIES.to_vec()))
}
/// Register a video decoder factory (tried before previously registered ones).
@@ -96,6 +100,17 @@ pub fn make_video_decoder(entry: &filmcraft_isobmff::SampleEntry) -> Result<Box<
Err(CodecError::Unsupported(format!("no decoder for {} video", entry.codec.name())))
}
/// Create one of our own (software) decoders for a sample entry, skipping registered factories:
/// what a hardware decoder falls back to when it fails mid-stream.
pub fn software_video_decoder(entry: &filmcraft_isobmff::SampleEntry) -> Result<Box<dyn VideoDecoder>> {
for f in BUILTIN_FACTORIES {
if let Some(r) = f(entry) {
return r;
}
}
Err(CodecError::Unsupported(format!("no decoder for {} video", entry.codec.name())))
}
/// Openers for the engine's media pool (MP4/MOV, Matroska/WebM, MXF, Ogg Opus/Vorbis, MPEG TS/PS and
/// MPEG-1/2 video elementary streams, standalone audio).
pub fn openers() -> Vec<filmcraft_media::Opener> {
+4 -4
View File
@@ -369,9 +369,9 @@ impl MkvSource {
if let Some(p) = st.packets.get(&i) {
return Ok(p.clone());
}
let ti = self.atrack.expect("audio");
let ti = self.atrack.ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
let track = &self.file.tracks[ti];
let ainfo = self.info.audio.as_ref().expect("audio info");
let ainfo = self.info.audio.as_ref().ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
let data = self.read(ti, i)?;
let decoded = match &track.codec {
Codec::Pcm { float, big_endian, bits } => {
@@ -392,7 +392,7 @@ impl MkvSource {
// Non-sequential access: reset and prime with the preceding packets (codec pre-roll:
// one packet, or `SeekPreRoll` worth for Opus).
if st.last_decoded.is_none_or(|l| l + 1 != i) {
let d = st.decoder.as_mut().expect("decoder");
let d = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?;
d.reset();
let from = if self.audio_preroll > 0 {
self.audio_starts.partition_point(|&x| x <= self.audio_starts[i] - self.audio_preroll).saturating_sub(1)
@@ -405,7 +405,7 @@ impl MkvSource {
}
}
}
let r = st.decoder.as_mut().expect("decoder").decode(&data, self.audio_starts[i].max(0) as u64);
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, self.audio_starts[i].max(0) as u64);
st.last_decoded = Some(i);
r.unwrap_or_default()
}
+38 -6
View File
@@ -107,8 +107,11 @@ impl Mp4Source {
pub fn open_reader(name: &str, reader: filmcraft_media::SharedReader) -> crate::Result<Self> {
let bytes = crate::Src(reader);
let file = filmcraft_isobmff::open(&bytes).map_err(|e| CodecError::Container(e.to_string()))?;
let vtrack = file.tracks.iter().position(|t| t.kind == TrackKind::Video && !t.samples.is_empty());
let atrack = file.tracks.iter().position(|t| t.kind == TrackKind::Audio && !t.samples.is_empty());
// A track needs samples, a sample description and a timescale to be playable (damaged
// files can lack them: indexing `entries[0]` or dividing by the timescale used to panic).
let playable = |t: &&filmcraft_isobmff::Track, kind| t.kind == kind && !t.samples.is_empty() && !t.entries.is_empty() && t.timescale > 0;
let vtrack = file.tracks.iter().position(|t| playable(&t, TrackKind::Video));
let atrack = file.tracks.iter().position(|t| playable(&t, TrackKind::Audio));
if vtrack.is_none() && atrack.is_none() {
return Err(CodecError::Unsupported("no playable tracks".into()));
}
@@ -126,6 +129,8 @@ impl Mp4Source {
durs.sort_unstable();
let d = durs.get(durs.len() / 2).copied().unwrap_or(1).max(1);
let rate = FrameRate::from_f64(t.timescale as f64 / d as f64);
// A rate that rounds to zero (one sample per huge duration) can't be divided by.
let rate = if rate.num > 0 && rate.den > 0 { rate } else { FrameRate::default() };
color = color_from(entry, w, h);
if entry
.video
@@ -264,9 +269,9 @@ impl Mp4Source {
if let Some(p) = st.packets.get(&i) {
return Ok(p.clone());
}
let ti = self.atrack.expect("audio");
let ti = self.atrack.ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
let track = &self.file.tracks[ti];
let entry = &track.entries[0];
let entry = track.entries.first().ok_or_else(|| CodecError::Container("audio track without a sample description".into()))?;
let data = self.read(ti, i)?;
let decoded = match &entry.codec {
CodecConfig::Pcm(p) => decode_pcm(&data, p),
@@ -277,7 +282,7 @@ impl Mp4Source {
// Non-sequential access: reset and prime with the preceding packets (codec pre-roll:
// one packet, or `OPUS_PRE_ROLL` worth for Opus).
if st.last_decoded.is_none_or(|l| l + 1 != i) {
let d = st.decoder.as_mut().expect("decoder");
let d = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?;
d.reset();
let from = if self.audio_preroll > 0 {
self.audio_starts.partition_point(|&x| x <= self.audio_starts[i] - self.audio_preroll).saturating_sub(1)
@@ -290,7 +295,7 @@ impl Mp4Source {
}
}
}
let r = st.decoder.as_mut().expect("decoder").decode(&data, self.audio_starts[i].max(0) as u64);
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, self.audio_starts[i].max(0) as u64);
st.last_decoded = Some(i);
match r {
Ok(v) => v,
@@ -513,6 +518,33 @@ mod tests {
b.into()
}
/// Overwrite the big-endian u32 `skip` bytes after the first `fourcc` box type.
fn patch_u32(b: &mut [u8], fourcc: &[u8; 4], skip: usize, v: u32) {
let k = b.windows(4).position(|x| x == fourcc).expect("box") + 4 + skip;
b[k..k + 4].copy_from_slice(&v.to_be_bytes());
}
/// Damaged MOVs found by mutation fuzzing panicked on import: a track with timescale 0
/// ("attempt to divide by zero" in the duration) and one whose sample description list is
/// empty ("index out of bounds" on `entries[0]`). Both must open as an error, or as media
/// that answers requests without panicking.
#[test]
fn damaged_track_headers_never_panic() {
let identity = [0x10000, 0, 0, 0, 0x10000, 0, 0, 0, 0x4000_0000];
// mdhd v0: version/flags, creation, modification, then timescale
let mut zero_timescale = rotated_mov(identity).to_vec();
patch_u32(&mut zero_timescale, b"mdhd", 12, 0);
// stsd: version/flags, then entry_count
let mut no_entries = rotated_mov(identity).to_vec();
patch_u32(&mut no_entries, b"stsd", 4, 0);
for (name, b) in [("ts0.mov", zero_timescale), ("stsd0.mov", no_entries)] {
if let Ok(s) = Mp4Source::open(name, b.into()) {
let _ = s.video_frame(FrameRequest::full(Tick::ZERO));
let _ = s.audio(0, 1024, 48_000);
}
}
}
/// Mean luma of the top and bottom halves of the decoded frame, and its size.
fn halves(src: &Mp4Source) -> (u32, u32, u8, u8) {
let f = src.video_frame(FrameRequest::full(Tick::ZERO)).expect("frame");
+41 -15
View File
@@ -123,6 +123,19 @@ fn audio_supported(c: &Codec) -> bool {
matches!(c, Codec::MpegAudio | Codec::AacAdts | Codec::AacLatm | Codec::LpcmBluray | Codec::LpcmDvd | Codec::Ac3)
}
/// The coded rate unless the timestamps say otherwise (3:2 pulldown, field-rate coding). Never
/// zero: damaged timestamps spread over a huge span give a rate that rounds to 0 fps, which the
/// duration computation divided by.
fn stream_rate(coded: Option<FrameRate>, from_pts: Option<f64>) -> FrameRate {
let rate = match (coded, from_pts) {
(Some(r), Some(p)) if (r.as_f64() - p).abs() / p > 0.01 => FrameRate::from_f64(p),
(Some(r), _) => r,
(None, Some(p)) => FrameRate::from_f64(p),
(None, None) => FrameRate::FPS_25,
};
if rate.num > 0 && rate.den > 0 { rate } else { FrameRate::FPS_25 }
}
/// Display-order keys of samples (decode order): the PTS when every unit has one, else (MPEG
/// video) GOP start + temporal_reference, else the decode order.
fn display_keys(units: &[&Unit], mpeg: bool) -> Vec<i64> {
@@ -242,8 +255,9 @@ impl MpegSource {
src.file = Some(file);
src.video = Some(track);
src.describe_video(rate_from_pts)?;
let file = src.file.take().expect("file");
src.setup_audio(file, astream, origin);
if let Some(file) = src.file.take() {
src.setup_audio(file, astream, origin);
}
}
None => {
src.unsupported_video = Some(format!("{} video in {} (FilmCraft has no decoder for it)", st.codec.name(), file.format.name()));
@@ -369,14 +383,14 @@ impl MpegSource {
if let Some((data, aus)) = &self.es {
return Ok(data[aus[u].clone()].to_vec());
}
let file = self.file.as_ref().expect("file");
let file = self.file.as_ref().ok_or_else(|| CodecError::Container("no stream file".into()))?;
file.read_unit(&self.bytes, v.stream, u).map_err(|e| CodecError::Container(e.to_string()))
}
/// Fill in the video stream info from the first picture.
fn describe_video(&mut self, rate_from_pts: Option<f64>) -> crate::Result<()> {
let mut seq_header = None;
let v = self.video.as_ref().expect("video");
let v = self.video.as_ref().ok_or_else(|| CodecError::Unsupported("no video stream".into()))?;
let first = self.read_unit(v, 0)?;
let n = v.order.len();
let (width, height, par, codec, pixel_format, mut color);
@@ -432,12 +446,7 @@ impl MpegSource {
}
}
// the coded rate unless the timestamps say otherwise (3:2 pulldown, field-rate coding)
let rate = match (rate, rate_from_pts) {
(Some(r), Some(p)) if (r.as_f64() - p).abs() / p > 0.01 => FrameRate::from_f64(p),
(Some(r), _) => r,
(None, Some(p)) => FrameRate::from_f64(p),
(None, None) => FrameRate::FPS_25,
};
let rate = stream_rate(rate, rate_from_pts);
let duration = rate.tick_of(n as i64);
let total_bytes: u64 = match (&self.es, &self.file) {
(Some((_, aus)), _) => v.units.iter().map(|&u| aus[u].len() as u64).sum(),
@@ -445,8 +454,10 @@ impl MpegSource {
_ => 0,
};
let secs = duration.seconds();
if let Some(h) = seq_header {
self.video.as_mut().expect("video").header = h;
if let Some(h) = seq_header
&& let Some(v) = self.video.as_mut()
{
v.header = h;
}
self.info.duration = duration;
self.info.video = Some(VideoStreamInfo {
@@ -525,7 +536,7 @@ impl MpegSource {
return self.audio_unsupported(&codec_name, "DVD LPCM: no format header".into());
}
},
_ => unreachable!("audio_supported"),
_ => return self.audio_unsupported(&codec_name, format!("{codec_name} audio")),
};
if codec_is_ac3_unsupported(codec) {
return self.audio_unsupported(&codec_name, format!("{codec_name} audio (FilmCraft has no AC-3 decoder yet)"));
@@ -608,12 +619,12 @@ impl MpegSource {
if let Some(p) = st.cache.get(&i) {
return Ok(p.clone());
}
let file = self.file.as_ref().expect("file");
let file = self.file.as_ref().ok_or_else(|| CodecError::Container("no stream file".into()))?;
let read = |k: usize| file.read_unit(&self.bytes, a.stream, k).map_err(|e| CodecError::Container(e.to_string()));
if st.dec.is_none() {
st.dec = Some(AudioDecoder::new(a.codec, a.rate, a.channels, a.latm.clone())?);
}
let dec = st.dec.as_mut().expect("decoder");
let dec = st.dec.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?;
if st.last.is_none_or(|l| l + 1 != i) {
dec.reset();
if i > 0
@@ -840,6 +851,21 @@ pub fn reader_opener(name: &str, head: &[u8], reader: &filmcraft_media::SharedRe
mod tests {
use super::*;
/// A mutated transport stream whose PTS span gave ~0 fps panicked on open with "attempt to
/// divide by zero" (`FrameRate::tick_of`).
#[test]
fn damaged_timestamps_never_give_a_zero_rate() {
for (coded, p) in
[(None, Some(1e-9)), (Some(FrameRate::FPS_25), Some(1e-9)), (Some(FrameRate::new(0, 0)), None), (None, Some(0.0)), (None, Some(f64::NAN))]
{
let r = stream_rate(coded, p);
assert!(r.num > 0 && r.den > 0, "{coded:?} {p:?} -> {r:?}");
let _ = r.tick_of(1000);
}
assert_eq!(stream_rate(Some(FrameRate::FPS_25), Some(25.0)), FrameRate::FPS_25);
assert_eq!(stream_rate(None, Some(50.0)), FrameRate::from_f64(50.0));
}
#[test]
fn presentation_drops_leading_pictures_and_orders_by_key() {
// decode order I(2) B(0) B(1) P(5) B(3) B(4): the leading B pictures are not presented
+2 -2
View File
@@ -333,7 +333,7 @@ impl MxfSource {
let mut channels = 0;
for &ai in &atracks {
let t = &file.tracks[ai];
let si = t.sound.as_ref().expect("sound info");
let Some(si) = t.sound.as_ref() else { continue };
if si.sample_rate != rate {
continue;
}
@@ -424,7 +424,7 @@ impl VideoSamples for MxfVideo<'_> {
return Err(CodecError::Unsupported(why.clone()));
}
let t = &self.src.file.tracks[self.track];
let v = self.src.info.video.as_ref().expect("video info");
let v = self.src.info.video.as_ref().ok_or_else(|| CodecError::Unsupported("no video track".into()))?;
let (w, h) = (v.width.min(u16::MAX as u32) as u16, v.height.min(u16::MAX as u32) as u16);
match t.codec {
Codec::Avc { .. } => Ok(Box::new(crate::video::H264Decoder::annexb())),
+2 -2
View File
@@ -133,7 +133,7 @@ impl OggSource {
}
// Non-sequential access: reset and decode the pre-roll (RFC 7845 §4.6; see OPUS_PRE_ROLL).
if st.last_decoded.is_none_or(|l| l + 1 != i) {
let d = st.decoder.as_mut().expect("decoder");
let d = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?;
d.reset();
let from = timing.starts.partition_point(|&x| x <= timing.starts[i] - OPUS_PRE_ROLL as i64).saturating_sub(1);
for j in from..i {
@@ -143,7 +143,7 @@ impl OggSource {
}
}
let data = read(i)?;
let r = st.decoder.as_mut().expect("decoder").decode(&data, 0);
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, 0);
st.last_decoded = Some(i);
let p = Arc::new(r.unwrap_or_default());
st.packets.insert(i, p.clone());
+28 -1
View File
@@ -88,6 +88,33 @@ fn mp3_file_decodes() {
assert!(a.peaks()[0] > 0.1);
}
#[test]
fn aiff_file_decodes() {
// 0.1 s of 48 kHz 16-bit stereo PCM: left a ramp, right silent (AIFF, big-endian samples)
let frames = 4800u32;
let mut b = b"FORM".to_vec();
b.extend((4 + 26 + 16 + frames * 4).to_be_bytes());
b.extend(b"AIFFCOMM");
b.extend(18u32.to_be_bytes());
b.extend(2u16.to_be_bytes());
b.extend(frames.to_be_bytes());
b.extend(16u16.to_be_bytes());
b.extend([0x40, 0x0E, 0xBB, 0x80, 0, 0, 0, 0, 0, 0]); // 48000 as an 80-bit extended float
b.extend(b"SSND");
b.extend((8 + frames * 4).to_be_bytes());
b.extend([0; 8]);
for i in 0..frames {
b.extend(((i as i16) * 4).to_be_bytes());
b.extend(0i16.to_be_bytes());
}
let src = crate::open_bytes("tone.aif", b.into()).unwrap();
assert!((src.info().duration.seconds() - 0.1).abs() < 1e-3, "{}", src.info().duration.seconds());
let a = src.audio(0, 4800, 48_000).unwrap();
assert_eq!(a.channels.len(), 2);
assert!((a.channels[0][1000] - 4000.0 / 32768.0).abs() < 1e-4, "{}", a.channels[0][1000]);
assert_eq!(a.channels[1][1000], 0.0);
}
#[test]
fn seek_backwards_and_forwards_mjpeg() {
let Some(b) = fixture("counter_mjpeg.mov", &["-f", "lavfi", "-i", "testsrc2=s=160x120:r=25:d=3", "-c:v", "mjpeg"]) else { return };
@@ -445,7 +472,7 @@ fn opus_reference(name: &str) -> Option<Vec<f32>> {
if !st.success() {
return None;
}
Some(std::fs::read(out).ok()?.chunks_exact(4).map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect())
Some(std::fs::read(out).ok()?.as_chunks::<4>().0.iter().map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect())
}
fn snr_db(reference: &[f32], test: &[f32]) -> f64 {
+49 -32
View File
@@ -128,6 +128,42 @@ pub(crate) fn tight_plane<T: Copy>(src: Vec<T>, stride: usize, w: usize, h: usiz
out
}
/// Colour of an H.264 / HEVC picture from its VUI code points (ITU-T H.273), as the decoders
/// report it: BT.709 by default with the matrix guessed from the size, the signalled matrix and
/// transfer when known, and full range when flagged. Shared by the software and hardware
/// decoders so the two are interchangeable.
pub fn vui_color(width: u32, height: u32, matrix: u8, transfer: u8, full_range: bool) -> filmcraft_color::ColorInfo {
let mut color = filmcraft_color::ColorInfo { matrix: filmcraft_frame::default_matrix(width, height), ..filmcraft_color::ColorInfo::REC709 };
if let Some(m) = filmcraft_color::Matrix::from_code(matrix) {
color.matrix = m;
}
if let Some(t) = filmcraft_color::Transfer::from_code(transfer) {
color.transfer = t;
}
if full_range {
color.range = filmcraft_color::Range::Full;
}
color
}
/// Pixel aspect ratio from a VUI sample aspect ratio ((0, 0) = unspecified = square).
pub fn sar_par(sar: (u16, u16)) -> (u32, u32) {
if sar.0 > 0 && sar.1 > 0 { (sar.0 as u32, sar.1 as u32) } else { (1, 1) }
}
/// NAL length-prefix size of an `avcC` record (byte 4: lengthSizeMinusOne).
pub fn avcc_length_size(avcc: &[u8]) -> usize {
avcc.get(4).map_or(4, |b| (b & 3) as usize + 1)
}
/// NAL length-prefix size and highest TemporalId of an `hvcC` record.
pub fn hvcc_length_size_and_tid(hvcc: &[u8]) -> (usize, Option<u8>) {
// hvcC byte 21: constantFrameRate(2) numTemporalLayers(3) temporalIdNested(1) lengthSizeMinusOne(2)
let length_size = hvcc.get(21).map_or(4, |b| (b & 3) as usize + 1);
let highest_tid = hvcc.get(21).map(|b| (b >> 3) & 7).filter(|&n| n > 0).map(|n| n - 1);
(length_size, highest_tid)
}
/// A factory returns `None` when it does not handle the entry.
pub type VideoDecoderFactory = fn(&SampleEntry) -> Option<Result<Box<dyn VideoDecoder>>>;
@@ -169,7 +205,7 @@ pub struct H264Decoder {
impl H264Decoder {
pub fn new(avcc: Vec<u8>) -> Result<Self> {
let dec = filmcraft_h264::Decoder::from_avcc(&avcc).map_err(|e| CodecError::Decode(e.to_string()))?;
let length_size = avcc.get(4).map_or(4, |b| (b & 3) as usize + 1);
let length_size = avcc_length_size(&avcc);
Ok(Self { avcc, dec, length_size, draft: false })
}
/// A decoder for Annex B byte-stream samples (start codes, in-band parameter sets: MXF, TS).
@@ -184,17 +220,8 @@ impl H264Decoder {
let y = tight_plane(p.y, p.y_stride, w, h);
let u = tight_plane(p.u, p.uv_stride, cw, ch);
let v = tight_plane(p.v, p.uv_stride, cw, ch);
let mut color = filmcraft_color::ColorInfo { matrix: filmcraft_frame::default_matrix(p.width, p.height), ..filmcraft_color::ColorInfo::REC709 };
if let Some(m) = filmcraft_color::Matrix::from_code(p.color.matrix) {
color.matrix = m;
}
if let Some(t) = filmcraft_color::Transfer::from_code(p.color.transfer) {
color.transfer = t;
}
if p.color.full_range {
color.range = filmcraft_color::Range::Full;
}
let par = if p.sar.0 > 0 && p.sar.1 > 0 { (p.sar.0 as u32, p.sar.1 as u32) } else { (1, 1) };
let color = vui_color(p.width, p.height, p.color.matrix, p.color.transfer, p.color.full_range);
let par = sar_par(p.sar);
let frame = VideoFrame {
width: p.width,
height: p.height,
@@ -265,9 +292,7 @@ pub struct HevcDecoder {
impl HevcDecoder {
pub fn new(hvcc: Vec<u8>) -> Result<Self> {
let dec = filmcraft_hevc::Decoder::from_hvcc(&hvcc).map_err(|e| CodecError::Decode(e.to_string()))?;
// hvcC byte 21: constantFrameRate(2) numTemporalLayers(3) temporalIdNested(1) lengthSizeMinusOne(2)
let length_size = hvcc.get(21).map_or(4, |b| (b & 3) as usize + 1);
let highest_tid = hvcc.get(21).map(|b| (b >> 3) & 7).filter(|&n| n > 0).map(|n| n - 1);
let (length_size, highest_tid) = hvcc_length_size_and_tid(&hvcc);
Ok(Self { hvcc, dec, length_size, highest_tid })
}
/// A decoder for Annex B byte-stream samples (start codes, in-band parameter sets: TS).
@@ -299,17 +324,8 @@ impl HevcDecoder {
alpha: None,
},
};
let mut color = filmcraft_color::ColorInfo { matrix: filmcraft_frame::default_matrix(p.width, p.height), ..filmcraft_color::ColorInfo::REC709 };
if let Some(m) = filmcraft_color::Matrix::from_code(p.color.matrix) {
color.matrix = m;
}
if let Some(t) = filmcraft_color::Transfer::from_code(p.color.transfer) {
color.transfer = t;
}
if p.color.full_range {
color.range = filmcraft_color::Range::Full;
}
let par = if p.sar.0 > 0 && p.sar.1 > 0 { (p.sar.0 as u32, p.sar.1 as u32) } else { (1, 1) };
let color = vui_color(p.width, p.height, p.color.matrix, p.color.transfer, p.color.full_range);
let par = sar_par(p.sar);
let frame = VideoFrame { width: p.width, height: p.height, data, color, par, pts: filmcraft_time::Tick::ZERO };
DecodedFrame { pts: p.pts, frame, draft: p.draft }
}
@@ -386,7 +402,8 @@ impl Vp9Decoder {
}
}
fn convert(&self, p: filmcraft_vp9::Picture) -> DecodedFrame {
/// `None` for a picture whose planes mix sample types (never produced by the decoder).
fn convert(&self, p: filmcraft_vp9::Picture) -> Option<DecodedFrame> {
use filmcraft_color::{Matrix, Primaries, Range};
use filmcraft_frame::{Chroma, PixelData};
use filmcraft_vp9::Plane;
@@ -423,7 +440,7 @@ impl Vp9Decoder {
}
let mut frame = VideoFrame::rgba8(p.width, p.height, rgba);
frame.color = filmcraft_color::ColorInfo { range: Range::Full, transfer: filmcraft_color::Transfer::Srgb, ..color };
return DecodedFrame { pts, frame, draft };
return Some(DecodedFrame { pts, frame, draft });
}
// 4:4:0 has no frame format of its own: chroma rows are repeated to 4:4:4.
let (chroma, rows_440) = match (p.subsampling_x, p.subsampling_y) {
@@ -455,22 +472,22 @@ impl Vp9Decoder {
bits: p.bit_depth,
alpha: None,
},
_ => unreachable!("VP9 planes share one sample type"),
_ => return None,
};
// render_size (the intended display size) is not applied: the container's display
// dimensions / pixel aspect describe the presentation.
let par = (1, 1);
DecodedFrame { pts, frame: VideoFrame { width: p.width, height: p.height, data, color, par, pts: filmcraft_time::Tick::ZERO }, draft }
Some(DecodedFrame { pts, frame: VideoFrame { width: p.width, height: p.height, data, color, par, pts: filmcraft_time::Tick::ZERO }, draft })
}
}
impl VideoDecoder for Vp9Decoder {
fn decode(&mut self, sample: &[u8], pts: i64) -> Result<Vec<DecodedFrame>> {
let pics = self.dec.decode(sample, pts).map_err(|e| CodecError::Decode(e.to_string()))?;
Ok(pics.into_iter().map(|p| self.convert(p)).collect())
Ok(pics.into_iter().filter_map(|p| self.convert(p)).collect())
}
fn flush(&mut self) -> Vec<DecodedFrame> {
self.dec.flush().into_iter().map(|p| self.convert(p)).collect()
self.dec.flush().into_iter().filter_map(|p| self.convert(p)).collect()
}
fn reset(&mut self) {
self.dec.reset();
+2 -2
View File
@@ -48,7 +48,7 @@ pub fn ffmpeg_audio_f32(ff: &Path, file: &Path, extra: &[&str]) -> Vec<f32> {
let mut args = vec!["-i", file.to_str().unwrap()];
args.extend_from_slice(extra);
args.extend_from_slice(&["-map", "0:a:0", "-f", "f32le", "-"]);
ffmpeg_out(ff, &args).chunks_exact(4).map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect()
ffmpeg_out(ff, &args).as_chunks::<4>().0.iter().map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect()
}
/// Our frame's three planes widened to u16 (and their sizes).
@@ -63,7 +63,7 @@ pub fn planes(f: &VideoFrame) -> Vec<Vec<u16>> {
/// Split one raw frame of `ffmpeg_frames` into planes (`bps` bytes per sample, chroma `cw`×`ch`).
pub fn raw_planes(raw: &[u8], w: usize, h: usize, cw: usize, ch: usize, bps: usize) -> Vec<Vec<u16>> {
let get = |b: &[u8]| -> Vec<u16> {
if bps == 1 { b.iter().map(|&v| v as u16).collect() } else { b.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]])).collect() }
if bps == 1 { b.iter().map(|&v| v as u16).collect() } else { b.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect() }
};
let y = w * h * bps;
let c = cw * ch * bps;
+1 -1
View File
@@ -92,7 +92,7 @@ fn decode_both(ff: &Path, file: &Path, decoder: &[&str]) -> (OggSource, Vec<f32>
let ch = a.channels as usize;
let mut args: Vec<&str> = decoder.to_vec();
args.extend_from_slice(&["-i", file.to_str().unwrap(), "-map", "0:a:0", "-f", "f32le", "-"]);
let want: Vec<f32> = ffmpeg_out(ff, &args).chunks_exact(4).map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
let want: Vec<f32> = ffmpeg_out(ff, &args).as_chunks::<4>().0.iter().map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
let n = want.len() / ch;
let frames = src.info().duration.to_units_floor(a.sample_rate as i64) as usize;
assert_eq!(frames, n, "{}: our length = ffmpeg's ({} samples)", file.display(), n);
+2
View File
@@ -5,6 +5,8 @@
//! The compositor works in **linear-light, premultiplied RGBA f32** in the sequence working space
//! (Rec.709 primaries by default). Decoded frames carry [`ColorInfo`] so conversions are explicit.
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
use serde::{Deserialize, Serialize};
use std::sync::OnceLock;
+1 -1
View File
@@ -38,7 +38,7 @@ let f = decode_frame_with(&sample, &DecodeOptions { threads: false, ..Default::d
let hdr: FrameHeader = probe(&sample)?; // cid, raster, depth, chroma…
let name = cid_name(hdr.cid); // "DNxHR HQX", "DNxHD 1080i 8-bit (CID 1242)"
let mut enc = Encoder::new(Profile::Hq, 1920, 1080); // or Encoder::with_config(EncoderConfig { .. })
let mut enc = Encoder::new(Profile::Hq, 1920, 1080)?; // or Encoder::with_config(EncoderConfig { .. })
let sample: Vec<u8> = enc.encode(&frame)?; // exactly enc.frame_size() bytes (CBR)
```
+1 -2
View File
@@ -19,8 +19,7 @@ pub(crate) fn idct8x8(block: &mut [f32; 64]) {
let mut nz = [false; 8];
let mut dc_only = true;
// Horizontal pass: t[r][x] = Σ_u F[r][u] · B[u][x]
for r in 0..8 {
let c: &[f32; 8] = block[r * 8..r * 8 + 8].try_into().unwrap();
for (r, c) in block.as_chunks::<8>().0.iter().enumerate() {
if c[1..].iter().all(|&v| v == 0.0) {
if c[0] != 0.0 {
t[r] = [c[0] * BASIS[0][0]; 8];
+4 -3
View File
@@ -61,8 +61,9 @@ struct Prepared {
}
impl Encoder {
pub fn new(profile: Profile, width: u32, height: u32) -> Encoder {
Encoder::with_config(EncoderConfig::new(profile, width, height)).expect("valid default configuration")
/// An encoder with the default configuration; fails on rasters the profile can't code.
pub fn new(profile: Profile, width: u32, height: u32) -> Result<Encoder> {
Encoder::with_config(EncoderConfig::new(profile, width, height))
}
pub fn with_config(cfg: EncoderConfig) -> Result<Encoder> {
@@ -79,7 +80,7 @@ impl Encoder {
if !ok_depth {
return Err(Error::Input(format!("{} does not support {}-bit", cfg.profile.name(), cfg.bit_depth)));
}
let info = cid_info(cfg.profile.cid()).expect("RI CID");
let info = cid_info(cfg.profile.cid()).ok_or_else(|| Error::Input(format!("no compression table for {}", cfg.profile.name())))?;
let frame_size = cfg.profile.frame_size(cfg.width, cfg.height);
let header_size = header::header_size_for(cfg.height);
Ok(Encoder { cfg, info, frame_size, header_size, last_q: 8 })
+2 -2
View File
@@ -127,7 +127,7 @@ fn probe_weights() {
let px: Vec<f64> = if depth == 8 {
out.stdout.iter().map(|&b| b as f64).collect()
} else {
out.stdout.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]]) as f64).collect()
out.stdout.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]]) as f64).collect()
};
let (off, stride) = if blk == 0 { (0, w as usize) } else { ((w * h) as usize, w as usize / 2) };
let mid = (1 << (depth - 1)) as f64;
@@ -224,7 +224,7 @@ fn probe_reconstruction() {
let px: Vec<f64> = if depth == 8 {
out.stdout.iter().map(|&b| b as f64).collect()
} else {
out.stdout.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]]) as f64).collect()
out.stdout.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]]) as f64).collect()
};
let mid = (1 << (depth - 1)) as f64;
let mut sum = 0f64;
+1 -1
View File
@@ -152,7 +152,7 @@ pub fn parse(d: &[u8]) -> Result<FrameHeader> {
2 => ColorVolume::Bt2020Cl,
_ => ColorVolume::OutOfBand,
};
let timecode = (d[0x30] & 0x80 != 0).then(|| d[0x31..0x39].try_into().unwrap());
let timecode = if d[0x30] & 0x80 != 0 { d.get(0x31..).and_then(|t| t.first_chunk::<8>()).copied() } else { None };
let mb_rows = be16(d, 0x16C);
if width == 0 || lines == 0 || width > 16384 || lines > 16384 {
return Err(Error::Invalid("raster size"));
+2
View File
@@ -11,6 +11,8 @@
//!
//! See the crate README for accuracy, performance and limitations.
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
mod bits;
mod dct;
mod decode;
+7 -4
View File
@@ -91,7 +91,7 @@ pub fn reference(ff: &Path, mov: &Path, pix_fmt: &str) -> Vec<u16> {
.expect("run ffmpeg");
assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr));
if pix_fmt.ends_with("le") {
out.stdout.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
out.stdout.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
} else {
out.stdout.iter().map(|&b| b as u16).collect()
}
@@ -247,7 +247,7 @@ pub fn ffmpeg_decode(ff: &Path, mov: &Path, pix_fmt: &str) -> Vec<u16> {
let err = String::from_utf8_lossy(&out.stderr).to_string();
assert!(out.status.success() && err.trim().is_empty(), "ffmpeg failed: {err}");
if pix_fmt.ends_with("le") {
out.stdout.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
out.stdout.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
} else {
out.stdout.iter().map(|&b| b as u16).collect()
}
@@ -270,8 +270,11 @@ pub fn source_frames(ff: &Path, name: &str, lavfi: &str, frames: u32, w: u32, h:
std::fs::rename(&tmp, &path).unwrap();
}
let bytes = std::fs::read(&path).unwrap();
let s: Vec<u16> =
if depth == 8 { bytes.iter().map(|&b| b as u16).collect() } else { bytes.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]])).collect() };
let s: Vec<u16> = if depth == 8 {
bytes.iter().map(|&b| b as u16).collect()
} else {
bytes.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect()
};
let proto = Frame::new(w, h, chroma, depth, false);
let per = frame_samples(&proto);
assert_eq!(s.len(), per * frames as usize);
+5 -2
View File
@@ -68,17 +68,20 @@ fn roundtrip_all_profiles() {
#[test]
fn rejects_bad_input() {
let mut enc = Encoder::new(Profile::Hq, 64, 64);
let mut enc = Encoder::new(Profile::Hq, 64, 64).unwrap();
assert!(enc.encode(&Frame::new(32, 64, ChromaFormat::Yuv422, 8, false)).is_err());
assert!(enc.encode(&Frame::new(64, 64, ChromaFormat::Yuv444, 8, false)).is_err());
assert!(Encoder::with_config(EncoderConfig { bit_depth: 10, ..EncoderConfig::new(Profile::Hq, 64, 64) }).is_err());
assert!(decode_frame(&[0u8; 100]).is_err());
assert!(decode_frame(&[0u8; 1000]).is_err());
// An unsupported raster is an error, not a panic.
assert!(Encoder::new(Profile::Hq, 0, 0).is_err());
assert!(Encoder::new(Profile::Hq, 20_000, 64).is_err());
}
#[test]
fn corrupt_streams_never_panic() {
let mut enc = Encoder::new(Profile::Hq, 256, 128);
let mut enc = Encoder::new(Profile::Hq, 256, 128).unwrap();
let good = enc.encode(&picture(256, 128, ChromaFormat::Yuv422, 8, 1)).unwrap();
let mut seed = 0x1234_5678u32;
let mut rnd = || {
+4 -3
View File
@@ -105,8 +105,9 @@ pub fn merge_captions(seq: &mut Sequence, ids: &[ClipId]) -> Result<ClipId> {
if idx.windows(2).any(|w| w[1] != w[0] + 1) {
return Err(EditError::Other("captions to merge must be next to each other".into()));
}
let first = idx[0];
let last = *idx.last().expect("non-empty");
let (Some(&first), Some(&last)) = (idx.first(), idx.last()) else {
return Err(EditError::Nothing);
};
let end = tr.captions[last].end();
let text: Vec<String> = idx.iter().map(|&i| tr.captions[i].text.trim().to_string()).filter(|t| !t.is_empty()).collect();
let keep = tr.captions[first].id;
@@ -205,7 +206,7 @@ pub fn move_captions(seq: &mut Sequence, ids: &[ClipId], delta: Tick) -> Result<
return Err(EditError::Nothing);
}
for tid in tracks {
let tr = seq.caption_track_mut(tid).expect("checked");
let Some(tr) = seq.caption_track_mut(tid) else { continue };
for c in tr.captions.iter_mut().filter(|c| ids.contains(&c.id)) {
c.start += d;
}
+40
View File
@@ -15,6 +15,8 @@
//!
//! Keyframes are stored in media time, so trims and splits never need to move them.
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
pub mod captions;
pub mod multicam;
pub mod through;
@@ -181,6 +183,34 @@ pub fn remove_orphan_transitions(track: &mut Track) {
});
}
/// Where a transition sits: the incoming clip's start, or the outgoing clip's end for a fade to nothing.
fn transition_anchor(track: &Track, tr: &Transition) -> Option<Tick> {
match (tr.from, tr.to) {
(_, Some(to)) => track.item(to).map(|i| i.start),
(Some(from), None) => track.item(from).map(|i| i.end()),
(None, None) => None,
}
}
/// Keep transitions on their edit points after an edit moved cuts (ripple trim, roll, slide, ripple
/// speed change): each transition shifts by as much as its anchor clip edge moved.
fn transitions_follow_cuts(before: &Sequence, after: &mut Sequence) {
for track in after.all_tracks_mut() {
let Some(old) = before.track(track.id) else { continue };
let shifts: Vec<Tick> = track
.transitions
.iter()
.map(|tr| match (transition_anchor(old, tr), transition_anchor(track, tr)) {
(Some(a), Some(b)) => b - a,
_ => Tick::ZERO,
})
.collect();
for (tr, d) in track.transitions.iter_mut().zip(shifts) {
tr.start += d;
}
}
}
fn place(track: &mut Track, item: TrackItem) {
let idx = track.items.partition_point(|i| i.start <= item.start);
track.items.insert(idx, item);
@@ -520,6 +550,7 @@ pub fn trim(seq: &mut Sequence, clip: ClipId, edge: Edge, mode: TrimMode, delta:
tr.sort();
}
}
transitions_follow_cuts(seq, &mut work);
}
work.check().map_err(EditError::Other)?;
*seq = work;
@@ -597,6 +628,7 @@ pub fn ripple_trim_group(seq: &mut Sequence, clips: &[ClipId], edge: Edge, delta
}
tr.sort();
}
transitions_follow_cuts(seq, &mut work);
work.check().map_err(EditError::Other)?;
*seq = work;
Ok(d)
@@ -620,6 +652,7 @@ pub fn roll(seq: &mut Sequence, left: ClipId, right: ClipId, delta: Tick, ctx: &
return Ok(d);
}
let (ls, rs) = (l.speed.abs(), r.speed.abs());
let before = seq.clone();
{
let (_, l) = seq.find_item_mut(left).ok_or(EditError::NoItem(left))?;
l.duration += d;
@@ -631,6 +664,7 @@ pub fn roll(seq: &mut Sequence, left: ClipId, right: ClipId, delta: Tick, ctx: &
r.source_in += src_of(d, rs);
}
let _ = ls;
transitions_follow_cuts(&before, seq);
Ok(d)
}
@@ -673,6 +707,7 @@ pub fn slide(seq: &mut Sequence, clip: ClipId, delta: Tick, ctx: &mut EditCtx) -
if d == Tick::ZERO {
return Ok(d);
}
let before = seq.clone();
let t = seq.track_mut(tid).ok_or(EditError::NoTrack(tid))?;
if let Some(p) = prev {
let pi = t.item_mut(p.id).ok_or(EditError::NoItem(p.id))?;
@@ -686,6 +721,7 @@ pub fn slide(seq: &mut Sequence, clip: ClipId, delta: Tick, ctx: &mut EditCtx) -
}
let me = t.item_mut(clip).ok_or(EditError::NoItem(clip))?;
me.start += d;
transitions_follow_cuts(&before, seq);
Ok(d)
}
@@ -728,6 +764,7 @@ pub fn set_speed(seq: &mut Sequence, clip: ClipId, speed: f64, reverse: bool, ri
new_dur = new_dur.min(next_start - it.start);
}
let delta = new_dur - it.duration;
let before = ripple.then(|| seq.clone());
{
let (_, it) = seq.find_item_mut(clip).ok_or(EditError::NoItem(clip))?;
it.speed = speed;
@@ -746,6 +783,9 @@ pub fn set_speed(seq: &mut Sequence, clip: ClipId, speed: f64, reverse: bool, ri
}
}
}
if let Some(before) = before {
transitions_follow_cuts(&before, seq);
}
Ok(())
}
+47
View File
@@ -210,6 +210,53 @@ fn roll_slip_slide() {
fx.seq.check().unwrap();
}
#[test]
fn transitions_stay_on_their_cuts() {
let mut fx = Fx::new();
let v1 = fx.v(0);
let a = fx.put(v1, 0, 10, 10);
let b = fx.put(v1, 10, 10, 10);
let c = fx.put(v1, 20, 10, 10);
let cross = |id, start, from, to| Transition {
id: TransitionId(id),
effect: filmcraft_project::find_effect("cross_dissolve").unwrap().instance(),
start: f(start),
duration: f(4),
from,
to,
align: Default::default(),
reverse: false,
};
let t = fx.seq.track_mut(v1).unwrap();
t.transitions = vec![cross(1, 8, Some(a), Some(b)), cross(2, 18, Some(b), Some(c)), cross(3, 26, Some(c), None)];
// every transition is centred on its cut (or ends with the clip it fades out)
let check = |fx: &Fx, what: &str| {
let t = fx.seq.track(v1).unwrap();
for tr in &t.transitions {
let cut = match tr.to {
Some(to) => t.item(to).unwrap().start,
None => t.item(tr.from.unwrap()).unwrap().end() - f(2),
};
assert_eq!(tr.start + f(2), cut, "{what}: transition {:?} left its cut", tr.id);
}
assert_eq!(t.transitions.len(), 3, "{what}");
};
let mut n = fx.next;
trim(&mut fx.seq, a, Edge::Out, TrimMode::Ripple, f(4), &mut Fx::ctx(&mut n)).unwrap();
check(&fx, "ripple trim out");
trim(&mut fx.seq, b, Edge::In, TrimMode::Ripple, f(3), &mut Fx::ctx(&mut n)).unwrap();
check(&fx, "ripple trim in");
ripple_trim_group(&mut fx.seq, &[b], Edge::Out, -f(2), &mut Fx::ctx(&mut n)).unwrap();
check(&fx, "ripple trim group");
roll(&mut fx.seq, b, c, f(2), &mut Fx::ctx(&mut n)).unwrap();
check(&fx, "roll");
slide(&mut fx.seq, b, -f(3), &mut Fx::ctx(&mut n)).unwrap();
check(&fx, "slide");
set_speed(&mut fx.seq, b, 0.5, false, true, &mut Fx::ctx(&mut n)).unwrap();
check(&fx, "ripple speed change");
fx.seq.check().unwrap();
}
#[test]
fn rate_stretch_and_speed() {
let mut fx = Fx::new();
+4 -2
View File
@@ -344,7 +344,9 @@ pub fn caption_blocks(words: &[SeqWord], rules: &CaptionRules, rate: FrameRate)
continue;
}
if fits_line {
*lines.last_mut().expect("non-empty") += 1 + len;
if let Some(l) = lines.last_mut() {
*l += 1 + len;
}
} else {
lines.push(len);
}
@@ -362,7 +364,7 @@ pub fn caption_blocks(words: &[SeqWord], rules: &CaptionRules, rate: FrameRate)
for g in groups {
let mut text_lines: Vec<String> = vec![String::new()];
for w in &words[g.clone()] {
let l = text_lines.last_mut().expect("non-empty");
let Some(l) = text_lines.last_mut() else { break };
if l.is_empty() {
l.push_str(&w.text);
} else if l.chars().count() + 1 + w.text.chars().count() <= max_chars {
+6
View File
@@ -82,6 +82,11 @@ pub struct PlaybackPrefs {
/// skip deblocking (faster; those frames are approximate). Never used for exports, renders or
/// a paused frame. Off by default.
pub draft_decode: bool,
/// "Hardware decoding": `auto` (default: the system's hardware decoder for streams it
/// supports, e.g. VideoToolbox H.264 / HEVC on macOS, falling back to FilmCraft's own decoder)
/// or `off` (always FilmCraft's own decoders). Media already open keeps its decoder until it
/// is reopened.
pub hardware_decoding: String,
}
impl Default for PlaybackPrefs {
@@ -94,6 +99,7 @@ impl Default for PlaybackPrefs {
enable_transmit: false,
disable_video_in_background: true,
draft_decode: false,
hardware_decoding: "auto".into(),
}
}
}
+3 -3
View File
@@ -1097,7 +1097,7 @@ fn subclip_plan(s: &Session, p: &Value) -> Result<(ItemId, TimeRange, String, La
} else {
let item = item_p(p, "item").or_else(|| subclip_source(s)).ok_or_else(|| bad("clip.makeSubclip", "open a clip in the Source monitor"))?;
let (root, m, sub) = media_root(&s.project, item).ok_or_else(|| bad("clip.makeSubclip", "only media clips can be subclipped"))?;
let pi = s.project.item(item).expect("resolved");
let pi = s.project.item(item).ok_or_else(|| EngineError::Other("no such item".into()))?;
let rate = m.frame_rate();
let still = s.prefs.timeline.still_duration(rate);
let is_still = matches!(m.info.kind, filmcraft_media::MediaKind::Still);
@@ -1246,7 +1246,7 @@ fn audio_channels(s: &mut Session, p: &Value) -> Result<Value> {
let mut out = Vec::new();
let mut maps = Vec::new();
for i in &items {
let m = s.project.item(*i).and_then(|it| it.as_media()).expect("filtered");
let Some(m) = s.project.item(*i).and_then(|it| it.as_media()) else { continue };
let n = m.info.audio.as_ref().map_or(2, |a| a.channels) as u16;
let fmt = format.unwrap_or_else(|| m.interpret.audio_channels.as_ref().map_or(AudioChannels::Stereo, |a| a.format));
let map = match &custom {
@@ -1603,7 +1603,7 @@ fn extract_audio(s: &mut Session, p: &Value) -> Result<Value> {
})
.ok_or_else(|| EngineError::Other("save the project first: extracted audio is written next to it".into()))?,
};
let pi = s.project.item(item).expect("resolved");
let pi = s.project.item(item).ok_or_else(|| EngineError::Other("no such item".into()))?;
let name = pi.name.clone();
let stem = std::path::Path::new(&name).file_stem().map(|x| x.to_string_lossy().into_owned()).unwrap_or_else(|| name.clone());
let src = s.media.full_res_source(&s.project, root, &*s.services).ok_or_else(|| EngineError::Other(format!("{name}: media is offline")))?;
+3 -3
View File
@@ -33,7 +33,7 @@ fn lumetri_text(s: &Session, clip: ClipId, p: &str) -> String {
fn mean(img: &filmcraft_render::Image) -> [f32; 3] {
assert!(img.px.iter().any(|v| *v > 0.01), "blank render");
let mut m = [0f64; 3];
for p in img.px.chunks_exact(4) {
for p in img.px.as_chunks::<4>().0 {
for k in 0..3 {
m[k] += p[k] as f64;
}
@@ -213,7 +213,7 @@ fn lumetri_presets_list_apply_and_thumbnails() {
assert!(it.effects[k + 1..].iter().all(|e| e.def().is_some_and(|d| d.intrinsic)) || k + 1 == it.effects.len());
assert_eq!(it.effects[k].param("saturation").unwrap().value, ParamValue::Float(0.0));
let img = s.render_program(0.25).unwrap();
let grey = img.px.chunks_exact(4).filter(|p| p[3] > 0.5).all(|p| (p[0] - p[1]).abs() < 0.01 && (p[1] - p[2]).abs() < 0.01);
let grey = img.px.as_chunks::<4>().0.iter().filter(|p| p[3] > 0.5).all(|p| (p[0] - p[1]).abs() < 0.01 && (p[1] - p[2]).abs() < 0.01);
assert!(grey, "the program is black and white");
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(s.active_sequence().unwrap().find_item(clip).unwrap().1.effects.len(), before);
@@ -242,7 +242,7 @@ fn hdr_lumetri_and_mastering_metadata_through_commands() {
for k in 0..3 {
assert!((m1[k] / m0[k] - 2.0).abs() < 0.08, "channel {k}: {m0:?} → {m1:?}");
}
assert!(up.px.chunks_exact(4).any(|p| p[1] > 1.2), "values above reference white survive");
assert!(up.px.as_chunks::<4>().0.iter().any(|p| p[1] > 1.2), "values above reference white survive");
// HDR White / HDR Specular / HDR Range are ordinary Lumetri parameters
s.execute("effects.setParam", json!({"clip": clip.0, "effect": idx, "param": "hdr_white", "value": 2000.0})).unwrap();
s.execute("effects.setParam", json!({"clip": clip.0, "effect": idx, "param": "hdr_specular", "value": -50.0})).unwrap();
+44 -27
View File
@@ -362,7 +362,7 @@ pub(crate) fn place_item(
let frame = (seq.settings.width, seq.settings.height);
let mut placements = Vec::new();
let link = if has_v && has_a { Some(p.alloc_id()) } else { None };
if has_v {
if let Some(vdest) = vdest.filter(|_| has_v) {
let mut v = p.make_track_item(item, TrackKind::Video, at, range, rate).ok_or_else(|| bad(label, "bad item"))?;
v.link = link;
for e in &mut v.effects {
@@ -371,10 +371,9 @@ pub(crate) fn place_item(
if is_media && src_size != frame {
crate::settings::apply_media_scaling(&mut v, &scaling, frame, src_size);
}
placements.push((vdest.expect("checked"), v));
placements.push((vdest, v));
}
if has_a {
let adest = adest.expect("checked");
if let Some(adest) = adest.filter(|_| has_a) {
let (arange, aat) = audio_split.unwrap_or((range, at));
let mut a = p.make_track_item(item, TrackKind::Audio, aat, arange, rate).ok_or_else(|| bad(label, "bad item"))?;
// Modify ▸ Audio Channels with several audio clips: one per clip, on the tracks below
@@ -675,27 +674,27 @@ fn build() -> Vec<CommandSpec> {
Ok(id) => ids.push(id.0),
Err(e) => errors.push(format!("{path}: {e}")),
},
Ok(b) if crate::captions::detect(&path, &b).is_some() => {
let fmt = crate::captions::detect(&path, &b).expect("detected");
match crate::captions::import(s, &path, &b, fmt, None) {
Ok(r) => reports.push(r),
Err(e) => errors.push(format!("{path}: {e}")),
}
}
Ok(b) if crate::interchange::detect(&path, &b).is_some() => {
let fmt = crate::interchange::detect(&path, &b).expect("detected");
match crate::interchange::import(s, &path, &b, fmt) {
Ok(r) => {
sequences.extend(r["sequences"].as_array().cloned().unwrap_or_default());
reports.push(r);
Ok(b) => {
if let Some(fmt) = crate::captions::detect(&path, &b) {
match crate::captions::import(s, &path, &b, fmt, None) {
Ok(r) => reports.push(r),
Err(e) => errors.push(format!("{path}: {e}")),
}
} else if let Some(fmt) = crate::interchange::detect(&path, &b) {
match crate::interchange::import(s, &path, &b, fmt) {
Ok(r) => {
sequences.extend(r["sequences"].as_array().cloned().unwrap_or_default());
reports.push(r);
}
Err(e) => errors.push(format!("{path}: {e}")),
}
} else {
match import_bytes(s, &path, b.into(), bin) {
Ok(id) => ids.push(id.0),
Err(e) => errors.push(format!("{path}: {e}")),
}
Err(e) => errors.push(format!("{path}: {e}")),
}
}
Ok(b) => match import_bytes(s, &path, b.into(), bin) {
Ok(id) => ids.push(id.0),
Err(e) => errors.push(format!("{path}: {e}")),
},
Err(e) => errors.push(format!("{path}: {e}")),
}
}
@@ -1280,7 +1279,7 @@ fn build() -> Vec<CommandSpec> {
let p = p.clone();
s.edit("Sequence Settings", |pr, _| {
if let Some(n) = str_p(&p, "name") {
pr.item_mut(id).expect("seq").name = n.to_string();
pr.item_mut(id).ok_or(EngineError::NoSequence)?.name = n.to_string();
}
let q = pr.sequence_mut(id).ok_or(EngineError::NoSequence)?;
if let Some(w) = u64_p(&p, "width") {
@@ -1793,7 +1792,21 @@ fn build() -> Vec<CommandSpec> {
.map(Tick)
.or_else(|| p.get("deltaFrames").and_then(Value::as_i64).map(|f| s.sequence_rate().tick_of(f)))
.unwrap_or_default();
s.edit_sequence("Slide", |q, ctx, _| Ok(edit::slide(q, c, d, ctx)?))?;
let clips = with_links(s, &[c]);
s.edit_sequence("Slide", |q, ctx, _| {
// clamp across all linked partners, then slide them by the common delta
let mut dd = d;
for c in &clips {
let x = edit::slide(&mut q.clone(), *c, dd, ctx)?;
if x.abs() < dd.abs() {
dd = x;
}
}
for c in &clips {
edit::slide(q, *c, dd, ctx)?;
}
Ok(())
})?;
Ok(Value::Null)
}),
cmd!("timeline.rateStretch", "Rate Stretch", [], None, r#"{"clip":id,"edge":"in|out","delta":ticks}"#, has_seq, |s, p| {
@@ -2440,7 +2453,7 @@ fn nest(s: &mut Session, p: &Value) -> Result<Value> {
s.edit("Nest", |pr, st| {
let nid = pr.new_sequence(&name, q.settings.clone(), q.video_tracks.len(), q.audio_tracks.len(), None);
{
let nq = pr.sequence_mut(nid).expect("new");
let nq = pr.sequence_mut(nid).ok_or(EngineError::NoSequence)?;
for (k, ti, it) in &items {
let mut it = it.clone();
it.start -= start;
@@ -2451,11 +2464,15 @@ fn nest(s: &mut Session, p: &Value) -> Result<Value> {
}
}
let rate = q.settings.frame_rate;
let mut v = pr.make_track_item(nid, TrackKind::Video, start, TimeRange::new(Tick::ZERO, end - start), rate).expect("item");
let mut v = pr
.make_track_item(nid, TrackKind::Video, start, TimeRange::new(Tick::ZERO, end - start), rate)
.ok_or_else(|| EngineError::Other("cannot place the nested sequence".into()))?;
for e in &mut v.effects {
resolve_auto_points(e, (q.settings.width, q.settings.height), (q.settings.width, q.settings.height));
}
let a = pr.make_track_item(nid, TrackKind::Audio, start, TimeRange::new(Tick::ZERO, end - start), rate).expect("item");
let a = pr
.make_track_item(nid, TrackKind::Audio, start, TimeRange::new(Tick::ZERO, end - start), rate)
.ok_or_else(|| EngineError::Other("cannot place the nested sequence".into()))?;
let link = pr.alloc_id();
let seq = pr.sequence_mut(seq_id).ok_or(EngineError::NoSequence)?;
edit::delete_items(seq, &sel);
+20 -11
View File
@@ -44,6 +44,15 @@ pub fn add_generator(p: &mut Project, pool: &MediaPool, src: GeneratorSource, na
/// Build the demo project. Returns (project, main sequence).
pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
build_demo(pool).unwrap_or_else(|| {
// Only reachable if the built-in items could not be placed: an empty demo, not a crash.
let mut p = Project::new("FilmCraft Demo");
let seq = p.new_sequence("Main Edit", SequenceSettings::default(), 3, 3, None);
(p, seq)
})
}
fn build_demo(pool: &MediaPool) -> Option<(Project, ItemId)> {
let mut p = Project::new("FilmCraft Demo");
let footage = p.add_bin("Footage", None);
let audio_bin = p.add_bin("Audio", None);
@@ -108,8 +117,8 @@ pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
for (ci, sin, dur) in cuts {
let src_range = TimeRange::new(r.snap(Tick::from_seconds_f64(sin)), r.snap(Tick::from_seconds_f64(dur)));
let link = p.alloc_id();
let mut v = p.make_track_item(clips[ci], TrackKind::Video, t, src_range, r).expect("item");
let mut a = p.make_track_item(clips[ci], TrackKind::Audio, t, src_range, r).expect("item");
let mut v = p.make_track_item(clips[ci], TrackKind::Video, t, src_range, r)?;
let mut a = p.make_track_item(clips[ci], TrackKind::Audio, t, src_range, r)?;
v.link = Some(link);
a.link = Some(link);
for e in &mut v.effects {
@@ -118,14 +127,14 @@ pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
a.effect_mut("volume").map(|e| e.params.get_mut("level").map(|l| l.value = ParamValue::Float(-8.0)));
let dur_t = v.duration;
placed.push((v.id, a.id, t, dur_t));
let s = p.sequence_mut(seq).expect("seq");
let s = p.sequence_mut(seq)?;
s.video_tracks[0].items.push(v);
s.audio_tracks[0].items.push(a);
t += dur_t;
}
// A lower-third-ish overlay on V2: the plasma clip scaled down in the corner with a drop shadow.
let ov_range = TimeRange::new(Tick::ZERO, r.snap(Tick::from_seconds_f64(4.0)));
let mut ov = p.make_track_item(clips[4], TrackKind::Video, r.snap(Tick::from_seconds_f64(6.0)), ov_range, r).expect("item");
let mut ov = p.make_track_item(clips[4], TrackKind::Video, r.snap(Tick::from_seconds_f64(6.0)), ov_range, r)?;
for e in &mut ov.effects {
resolve_auto_points(e, (1920, 1080), (1920, 1080));
}
@@ -152,18 +161,18 @@ pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
lum.params.get_mut("temperature").map(|v| v.value = ParamValue::Float(18.0));
lum.params.get_mut("contrast").map(|v| v.value = ParamValue::Float(22.0));
lum.params.get_mut("vignette_amount").map(|v| v.value = ParamValue::Float(-1.4));
let s = p.sequence_mut(seq).expect("seq");
let s = p.sequence_mut(seq)?;
if let Some(it) = s.video_tracks[0].items.get_mut(3) {
it.effects.insert(0, lum);
}
}
let s = p.sequence_mut(seq).expect("seq");
let s = p.sequence_mut(seq)?;
s.video_tracks[1].items.push(ov);
// Transitions: cross dissolve between shots 1-2, dip to black 3-4, push 5-6, fade in at start.
let tr_len = r.tick_of(24);
let mk = |id: u64, eff: &str, from: Option<filmcraft_project::ClipId>, to: Option<filmcraft_project::ClipId>, start: Tick| Transition {
id: TransitionId(id),
effect: find_effect(eff).expect("fx").instance(),
effect: crate::presets::effect_instance(eff),
start,
duration: tr_len,
from,
@@ -181,7 +190,7 @@ pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
s.video_tracks[0].transitions = trs;
s.audio_tracks[0].transitions.push(Transition {
id: TransitionId(900_010),
effect: find_effect("constant_power").expect("fx").instance(),
effect: crate::presets::effect_instance("constant_power"),
start: cut(1) - tr_len.mul_ratio(1, 2),
duration: tr_len,
from: Some(placed[0].1),
@@ -207,13 +216,13 @@ pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
});
}
// Music on A2 (under the whole edit), ducked -14 dB.
let mut m = p.make_track_item(music, TrackKind::Audio, Tick::ZERO, TimeRange::new(Tick::ZERO, end), r).expect("music");
let mut m = p.make_track_item(music, TrackKind::Audio, Tick::ZERO, TimeRange::new(Tick::ZERO, end), r)?;
m.effect_mut("volume").map(|e| e.params.get_mut("level").map(|l| l.value = ParamValue::Float(-14.0)));
m.label = Label::Caribbean;
let s = p.sequence_mut(seq).expect("seq");
let s = p.sequence_mut(seq)?;
s.audio_tracks[1].items.push(m);
for tr in s.all_tracks_mut() {
tr.sort();
}
(p, seq)
Some((p, seq))
}
+3
View File
@@ -411,6 +411,9 @@ fn ranges_resolve_and_sidecar_captions_are_written() {
assert_eq!(frames(&mut s, json!({"preset": "PNG Sequence"})), io, "In/Out is the default");
assert!(s.execute("export.resolve", json!({"range": "workArea"})).is_err(), "no work area");
assert_eq!(frames(&mut s, json!({"preset": "Animated GIF 640×360", "range": "custom", "startSeconds": 0, "endSeconds": 2})), 30, "15 fps");
let bare = frames(&mut s, json!({"preset": "Animated GIF 640×360", "startSeconds": 0, "endSeconds": 2}));
assert_eq!(bare, 30, "start/end times without `range` are a custom range, not the In/Out or whole sequence");
assert!(s.execute("export.resolve", json!({"startSeconds": 1})).is_err(), "a start without an end is an error");
let res = s.execute("export.resolve", json!({"preset": "YouTube 2160p 4K Ultra HD"})).unwrap();
assert_eq!(res["output"]["width"], 3840);
assert!(res["summary"]["video"].as_str().unwrap().contains("3840x2160"));
+39 -1
View File
@@ -244,12 +244,14 @@ pub fn sequence_param(s: &Session, p: &Value, cmd: &str) -> Result<ItemId> {
}
/// Export ▸ Range: `entire` | `inOut` | `workArea` | `custom` (`start…` / `end…` times).
/// None = the default (In/Out when set, else the whole sequence).
/// None = the default (In/Out when set, else the whole sequence); bare `start…` / `end…` times
/// without a `range` mean `custom`.
pub fn range_param(s: &Session, project: &Project, seq: ItemId, v: Option<&Value>, p: &Value, cmd: &str) -> Result<Option<TimeRange>> {
let q = project.sequence(seq).ok_or(EngineError::NoSequence)?;
let fd = q.settings.frame_rate.frame_duration();
let whole = TimeRange::from_bounds(Tick::ZERO, q.duration().max(fd));
let mode = match v {
None | Some(Value::Null) if time_p(s, p, "start").is_some() || time_p(s, p, "end").is_some() => "custom".to_string(),
None | Some(Value::Null) => return Ok(None),
Some(Value::String(m)) => m.to_ascii_lowercase().replace(['-', '_', ' ', '/'], ""),
Some(o @ Value::Object(_)) => return range_param(s, project, seq, o.get("mode").or(Some(&json!("custom"))), o, cmd),
@@ -385,6 +387,7 @@ pub fn spawn_export(s: &mut Session, project: Arc<Project>, seq: ItemId, mut set
}
*res.lock().unwrap_or_else(|x| x.into_inner()) = Some(r);
};
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
s.jobs.push(job);
if wait || cfg!(target_arch = "wasm32") {
run();
@@ -394,6 +397,24 @@ pub fn spawn_export(s: &mut Session, project: Arc<Project>, seq: ItemId, mut set
Ok(id)
}
/// Last-resort guard for a background job body (export, previews, proxies, …): a panic marks the
/// job failed with an error instead of killing its thread and leaving the job "running" forever.
/// The panic hook has already logged where it happened.
pub(crate) fn guard_job(
progress: Arc<filmcraft_export::Progress>,
result: Arc<std::sync::Mutex<Option<std::result::Result<filmcraft_export::Report, String>>>>,
run: impl FnOnce() + Send + 'static,
) -> impl FnOnce() + Send + 'static {
move || {
if std::panic::catch_unwind(std::panic::AssertUnwindSafe(run)).is_err() {
let msg = "internal error (see the crash log); the job was stopped".to_string();
*progress.error.lock().unwrap_or_else(|x| x.into_inner()) = Some(msg.clone());
progress.finished.store(true, Ordering::Relaxed);
*result.lock().unwrap_or_else(|x| x.into_inner()) = Some(Err(msg));
}
}
}
fn file_name(path: &str) -> String {
Path::new(path).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_else(|| path.to_string())
}
@@ -1008,3 +1029,20 @@ pub(crate) fn commands() -> Vec<CommandSpec> {
spec("export.quick", "Quick Export", r#"{"preset":str?,"path":str?,"wait":bool?, …settings params}"#, has_seq, quick, true),
]
}
#[cfg(test)]
mod guard_tests {
use super::*;
/// A panicking job body must leave the job finished with an error, not "running" forever.
#[test]
fn panicking_job_is_reported_as_failed() {
let progress: Arc<filmcraft_export::Progress> = Arc::default();
let result: Arc<std::sync::Mutex<Option<std::result::Result<filmcraft_export::Report, String>>>> = Arc::default();
let run = guard_job(progress.clone(), result.clone(), || panic!("job bug"));
std::thread::spawn(run).join().expect("the guard catches the panic");
assert!(progress.finished.load(Ordering::Relaxed));
assert!(progress.error.lock().unwrap().is_some());
assert!(matches!(&*result.lock().unwrap(), Some(Err(_))));
}
}
+9 -9
View File
@@ -157,15 +157,15 @@ pub fn thumbnail(t: &GraphicsTemplate, w: u32) -> (u32, u32, Vec<u8>) {
let mut p = Project::new("thumb");
let rate = filmcraft_time::FrameRate::FPS_30;
let g = p.add_item("G", Label::Rose, ItemKind::Graphic { width: t.canvas[0], height: t.canvas[1], rate }, None);
let mut it = p
.make_track_item(
g,
filmcraft_project::TrackKind::Video,
Tick::ZERO,
filmcraft_time::TimeRange::new(Tick::ZERO, t.duration.max(rate.frame_duration())),
rate,
)
.expect("item");
let Some(mut it) = p.make_track_item(
g,
filmcraft_project::TrackKind::Video,
Tick::ZERO,
filmcraft_time::TimeRange::new(Tick::ZERO, t.duration.max(rate.frame_duration())),
rate,
) else {
return (w, h, vec![0; w as usize * h as usize * 4]);
};
it.effects.extend(t.layers.iter().cloned());
it.graphic = Some(Box::new(meta));
// a moment into the clip, so intro animations have settled a little
+5 -5
View File
@@ -551,11 +551,11 @@ macro_rules! arrange_cmd {
fn new_shape_cmd(s: &mut Session, p: &Value, shape: &str) -> Result<Value> {
let mut q = p.clone();
q["shape"] = json!(shape);
(find_spec("graphics.newShape").run)(s, &q)
(find_spec("graphics.newShape")?.run)(s, &q)
}
fn find_spec(id: &str) -> CommandSpec {
commands().into_iter().find(|c| c.id == id).expect("graphics command")
fn find_spec(id: &str) -> Result<CommandSpec> {
commands().into_iter().find(|c| c.id == id).ok_or_else(|| EngineError::UnknownCommand(id.to_string()))
}
/// Graphics menu commands (New Layer shapes, Align/Distribute/Arrange/Select submenus, resets).
@@ -576,7 +576,7 @@ fn menu_commands() -> Vec<CommandSpec> {
|s, p| {
let mut p = if p.is_object() { p.clone() } else { json!({}) };
p["vertical"] = json!(true);
(find_spec("graphics.newText").run)(s, &p)
(find_spec("graphics.newText")?.run)(s, &p)
},
),
spec(
@@ -731,7 +731,7 @@ fn menu_commands() -> Vec<CommandSpec> {
let rate = q.settings.frame_rate;
let want = rate.snap_nearest(Tick::from_seconds_f64(seconds)).max(rate.frame_duration());
let tr = q.video_tracks.iter_mut().find(|tr| tr.items.iter().any(|i| i.id == clip)).ok_or(filmcraft_edit::EditError::NoItem(clip))?;
let i = tr.items.iter().position(|i| i.id == clip).expect("found");
let i = tr.items.iter().position(|i| i.id == clip).ok_or(filmcraft_edit::EditError::NoItem(clip))?;
let start = tr.items[i].start;
let limit = tr.items.get(i + 1).map_or(want, |n| n.start - start);
let d = want.min(limit).max(rate.frame_duration());
+9 -1
View File
@@ -8,6 +8,8 @@
//! The project is an `Arc<Project>` edited copy-on-write; undo keeps whole-project snapshots (cheap
//! thanks to structural sharing of untouched items).
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
pub mod aaf_omf;
pub mod autosave;
pub mod captions;
@@ -475,6 +477,7 @@ impl Session {
pub fn start_autosave(&mut self, cfg: autosave::AutosaveConfig) -> std::io::Result<()> {
let prefs_path = cfg.data_dir.join("preferences.json");
self.prefs = autosave::Preferences::load(&prefs_path);
self.prefs.apply_hardware_decoding();
self.media.set_use_proxies(self.prefs.media.enable_proxies);
self.shortcuts.set_dir(&cfg.data_dir);
self.presets.set_dir(&cfg.data_dir);
@@ -546,6 +549,7 @@ impl Session {
pub fn set_prefs(&mut self, p: autosave::Preferences) -> std::io::Result<()> {
let cache_changed = self.prefs.media_cache != p.media_cache;
self.prefs = p;
self.prefs.apply_hardware_decoding();
if cache_changed {
self.apply_media_cache();
}
@@ -599,7 +603,11 @@ impl Session {
return Err(e);
}
self.exec_depth += 1;
let r = (spec.run)(self, &params);
// Last-resort guard: a panic in a command (a bug that escaped the never-crash rules)
// becomes an error for the caller (UI, CLI, control channel, MCP) instead of taking the
// session and its unsaved project down. The panic hook has logged where it happened.
let r = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| (spec.run)(self, &params)))
.unwrap_or_else(|_| Err(EngineError::Other(format!("internal error in `{id}` (see the crash log); the command did not complete"))));
self.exec_depth -= 1;
// source graphics: an edited instance updates the shared layers and the other instances
if r.is_ok() && spec.journal && id.starts_with("graphics.") && !self.project.source_graphics.is_empty() {
+1
View File
@@ -634,6 +634,7 @@ fn track(s: &mut Session, p: &Value) -> Result<Value> {
prog.finished.store(true, Ordering::Relaxed);
*lock(&res) = Some(r);
};
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
s.jobs.push(job);
s.mask_jobs.push(PendingTrack { job: id, target: MaskSel { clip, effect: ei, mask: mi }, seq: seq_id, keys, applied: 0 });
let wait = crate::commands::bool_p(p, "wait").unwrap_or(false);
+1 -1
View File
@@ -101,7 +101,7 @@ pub fn psnr(a: &[u8], b: &[u8]) -> f64 {
assert_eq!(a.len(), b.len());
let mut se = 0.0;
let mut n = 0.0;
for (x, y) in a.chunks_exact(4).zip(b.chunks_exact(4)) {
for (x, y) in a.as_chunks::<4>().0.iter().zip(b.as_chunks::<4>().0) {
for c in 0..3 {
let d = x[c] as f64 - y[c] as f64;
se += d * d;
+11 -7
View File
@@ -554,7 +554,7 @@ fn merge_clips(s: &mut Session, p: &Value) -> Result<Value> {
a.link = Some(link);
a_items.push(a);
}
let q = pr.sequence_mut(sid).expect("new sequence");
let q = pr.sequence_mut(sid).ok_or(EngineError::NoSequence)?;
for (t, it) in q.video_tracks.iter_mut().zip(v_items) {
t.items.push(it);
}
@@ -616,7 +616,7 @@ fn create_multicam(s: &mut Session, p: &Value) -> Result<Value> {
let (mut vi, mut ai) = (0usize, 0usize);
let mut placed: Vec<(TrackKind, usize, TrackItem)> = Vec::new();
let (vtracks, atracks): (Vec<TrackId>, Vec<TrackId>) = {
let q = pr.sequence(sid).expect("new sequence");
let q = pr.sequence(sid).ok_or(EngineError::NoSequence)?;
(q.video_tracks.iter().map(|t| t.id).collect(), q.audio_tracks.iter().map(|t| t.id).collect())
};
for (k, c) in clips.iter().enumerate() {
@@ -640,7 +640,7 @@ fn create_multicam(s: &mut Session, p: &Value) -> Result<Value> {
cameras.push(cam);
}
{
let q = pr.sequence_mut(sid).expect("new sequence");
let q = pr.sequence_mut(sid).ok_or(EngineError::NoSequence)?;
for (kind, idx, it) in placed {
q.tracks_mut(kind)[idx].items.push(it);
}
@@ -892,10 +892,14 @@ fn record_start(s: &mut Session, p: &Value) -> Result<Value> {
fn apply_pass(s: &mut Session, end: Tick) -> Result<usize> {
let Some(seq_id) = s.mcrec.seq else { return Ok(0) };
let merging = s.history.merge_key.as_deref() == Some(s.mcrec.key.as_str()) && s.history.undo.last().is_some_and(|u| u.0 == RECORD_LABEL);
if !merging || s.mcrec.base.is_none() {
s.mcrec.base = Some(s.project.clone());
}
let base = s.mcrec.base.clone().expect("set above");
let base = match s.mcrec.base.clone() {
Some(b) if merging => b,
_ => {
let b = s.project.clone();
s.mcrec.base = Some(b.clone());
b
}
};
let (tracks, cuts, key) = (s.mcrec.tracks.clone(), s.mcrec.cuts.clone(), s.mcrec.key.clone());
let Some(source) = s.mcrec.source else { return Ok(0) };
let media = s.media.clone();
+28 -1
View File
@@ -9,6 +9,7 @@ use crate::Session;
/// measure an interval).
pub fn decode_json() -> Value {
let g = filmcraft_codecs::gop_stats();
let hw = filmcraft_codecs::hw::hw_stats();
json!({
"requests": g.hits + g.misses,
"cacheHits": g.hits,
@@ -22,6 +23,17 @@ pub fn decode_json() -> Value {
"evicted": g.evicted,
"decodeMs": g.decode_ns as f64 / 1e6,
"decodeMsPerSample": g.decode_ms_per_sample(),
"framesDecoded": g.frames,
// Settings ▸ Playback ▸ Hardware decoding: pictures from OS hardware decoders vs ours,
// hardware decoders created, streams handed to software up front, mid-stream fallbacks.
"hardware": {
"enabled": filmcraft_codecs::hw::hardware_decoding(),
"frames": hw.frames,
"softwareFrames": g.frames.saturating_sub(hw.frames),
"sessions": hw.sessions,
"declined": hw.declined,
"fallbacks": hw.fallbacks,
},
})
}
@@ -48,9 +60,24 @@ mod tests {
s.execute("file.openDemoProject", json!({})).unwrap();
let undo = s.history.undo.len();
let v = s.execute("perf.stats", json!({})).unwrap();
for k in ["requests", "cacheHitRate", "seeks", "samplesDecoded", "samplesSkipped", "draftFrames", "h264Threads", "decodeMs", "decodeMsPerSample"] {
for k in [
"requests",
"cacheHitRate",
"seeks",
"samplesDecoded",
"samplesSkipped",
"draftFrames",
"h264Threads",
"decodeMs",
"decodeMsPerSample",
"framesDecoded",
] {
assert!(v["decode"][k].is_number(), "decode.{k} in {v}");
}
for k in ["frames", "softwareFrames", "sessions", "declined", "fallbacks"] {
assert!(v["decode"]["hardware"][k].is_number(), "decode.hardware.{k} in {v}");
}
assert!(v["decode"]["hardware"]["enabled"].is_boolean());
assert!(v["media"]["openSources"].is_number());
assert_eq!(v["jobs"]["running"], json!(0));
assert_eq!(s.history.undo.len(), undo, "a query adds no undo step");
+15 -1
View File
@@ -151,8 +151,22 @@ fn kf(t_secs: f64, v: ParamValue) -> Keyframe {
Keyframe::new(Tick((t_secs * TICKS_PER_SECOND as f64).round() as i64), v)
}
/// A fresh instance of a built-in effect (a bare instance named `id` if the registry lacks it,
/// which the preset tests rule out).
pub(crate) fn effect_instance(id: &str) -> EffectInstance {
filmcraft_project::find_effect(id).map(|d| d.instance()).unwrap_or_else(|| EffectInstance {
effect: id.to_string(),
enabled: true,
params: Default::default(),
masks: Vec::new(),
post_fader: false,
essential: false,
layer: None,
})
}
fn effect(id: &str, params: &[(&str, ParamValue)]) -> EffectInstance {
let mut e = filmcraft_project::find_effect(id).unwrap_or_else(|| panic!("built-in preset uses unknown effect {id}")).instance();
let mut e = effect_instance(id);
for (k, v) in params {
if let Some(p) = e.params.get_mut(*k) {
p.value = v.clone();
+3 -1
View File
@@ -543,6 +543,7 @@ pub fn render(s: &mut Session, mode: RenderMode, p: &Value) -> Result<Value> {
prog.finished.store(true, Ordering::Relaxed);
*res.lock().unwrap_or_else(|x| x.into_inner()) = Some(r);
};
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
s.jobs.push(job);
let wait = p.get("wait").and_then(Value::as_bool).unwrap_or(false);
if wait || cfg!(target_arch = "wasm32") {
@@ -629,6 +630,7 @@ pub fn render_audio(s: &mut Session, p: &Value) -> Result<Value> {
prog.finished.store(true, Ordering::Relaxed);
*res.lock().unwrap_or_else(|x| x.into_inner()) = Some(r);
};
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
s.jobs.push(job);
if p.get("wait").and_then(Value::as_bool).unwrap_or(false) || cfg!(target_arch = "wasm32") {
run();
@@ -680,7 +682,7 @@ pub fn read_wav_f32(b: &[u8]) -> Option<Vec<f32>> {
if !float_stereo {
return None;
}
return Some(body.chunks_exact(4).map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect());
return Some(body.as_chunks::<4>().0.iter().map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect());
}
i += 8 + len + (len & 1);
}
+1
View File
@@ -229,6 +229,7 @@ pub fn run(s: &mut Session, p: &Value) -> Result<Value> {
prog.finished.store(true, Ordering::Relaxed);
*res.lock().unwrap_or_else(|x| x.into_inner()) = Some(r);
};
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
s.jobs.push(job);
if bool_p(p, "wait").unwrap_or(false) || cfg!(target_arch = "wasm32") {
run();
+1
View File
@@ -241,6 +241,7 @@ pub fn start_job(s: &mut Session, label: String, work: Vec<(ItemId, String, &'st
prog.finished.store(true, std::sync::atomic::Ordering::Relaxed);
*res.lock().unwrap_or_else(|x| x.into_inner()) = Some(r);
};
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
s.jobs.push(job);
s.media_jobs.push(PendingJob { job: id, on_done, outputs });
if wait || cfg!(target_arch = "wasm32") {
+3 -4
View File
@@ -35,12 +35,11 @@ pub fn fingerprint(size: u64, head: &[u8], tail: &[u8]) -> u64 {
h ^= h >> 29;
};
for part in [head, tail] {
let mut chunks = part.chunks_exact(8);
for c in &mut chunks {
mix(u64::from_le_bytes([c[0], c[1], c[2], c[3], c[4], c[5], c[6], c[7]]));
let (chunks, r) = part.as_chunks::<8>();
for c in chunks {
mix(u64::from_le_bytes(*c));
}
let mut last = [0u8; 8];
let r = chunks.remainder();
last[..r.len()].copy_from_slice(r);
mix(u64::from_le_bytes(last) ^ (r.len() as u64) << 56);
mix(part.len() as u64);
+1
View File
@@ -206,6 +206,7 @@ fn detect(s: &mut Session, p: &Value) -> Result<Value> {
*lock(&res) = Some(r);
prog.finished.store(true, Ordering::Relaxed);
};
let run = crate::export_tools::guard_job(job.progress.clone(), job.result.clone(), run);
s.jobs.push(job);
s.scene_jobs.push(PendingScene { job: id, seq: seq_id, apply, results: results.clone() });
let wait = bool_p(p, "wait").unwrap_or(false);
+4 -1
View File
@@ -381,7 +381,10 @@ fn make_subsequence(s: &mut Session, p: &Value) -> Result<Value> {
let offset = range.map(|r| r.start).unwrap_or_else(|| items.iter().map(|i| i.2.start).min().unwrap_or_default());
let name = match str_p(p, "name") {
Some(n) => n.to_string(),
None => (1..).map(|k| format!("{seq_name}_Sub_{k:02}")).find(|n| !s.project.items.values().any(|i| &i.name == n)).expect("free name"),
None => (1..)
.map(|k| format!("{seq_name}_Sub_{k:02}"))
.find(|n| !s.project.items.values().any(|i| &i.name == n))
.unwrap_or_else(|| format!("{seq_name}_Sub")),
};
let id = s.edit("Make Subsequence", |pr, st| {
let nid = pr.new_sequence(&name, q.settings.clone(), q.video_tracks.len(), q.audio_tracks.len(), None);
+11
View File
@@ -653,6 +653,7 @@ const CACHE_MGMT: &[(&str, &str)] = &[
];
const AUTO_SCROLL: &[(&str, &str)] = &[("noScroll", "No Scroll"), ("pageScroll", "Page Scroll"), ("smoothScroll", "Smooth Scroll")];
const MOUSE_SCROLL: &[(&str, &str)] = &[("vertical", "Vertical"), ("horizontal", "Horizontal")];
const HW_DECODE: &[(&str, &str)] = &[("auto", "Auto"), ("off", "Off")];
const TRACKS: &[(&str, &str)] = &[("useFile", "Use File"), ("mono", "Mono"), ("stereo", "Stereo"), ("5.1", "5.1"), ("adaptive", "Adaptive")];
pub const DURATION_UNITS: &[(&str, &str)] = &[("frames", "Frames"), ("seconds", "Seconds")];
@@ -952,6 +953,10 @@ static CATEGORIES: &[Category] = &[
b("playback.enableTransmit", "Enable Mercury Transmit", false),
b("playback.disableVideoInBackground", "Disable video output when in the background", false),
b("playback.draftDecode", "Draft decoding at reduced playback resolution (H.264: faster, some frames less filtered)", true),
f("playback.hardwareDecoding", "Hardware decoding", Kind::Choice(HW_DECODE), true),
Row::Note(
"Hardware decoding: Auto uses the system's video decoder (VideoToolbox on macOS) for the H.264 and HEVC streams it supports, and FilmCraft's own decoder for everything else or if the hardware fails. Media that is already open keeps its decoder until it is reopened.",
),
],
},
Category {
@@ -1122,6 +1127,12 @@ pub fn migrate(v: &mut Value) {
}
impl crate::autosave::Preferences {
/// Settings ▸ Playback ▸ Hardware decoding as the process-wide switch the hardware decoder
/// factories consult (new decoders only).
pub fn apply_hardware_decoding(&self) {
filmcraft_codecs::hw::set_hardware_decoding(self.playback.hardware_decoding != "off");
}
/// Values of one category as a JSON object.
pub fn category_value(&self, id: &str) -> Option<Value> {
self.to_value().get(id).cloned()
+12
View File
@@ -480,3 +480,15 @@ fn auto_transcribe_on_import_and_transcription_defaults() {
set(&mut s, "mediaAnalysis.whisperModel", json!("whisper-tiny"));
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn hardware_decoding_setting_drives_the_decoder_switch() {
let mut s = Session::default();
assert_eq!(s.prefs.playback.hardware_decoding, "auto", "Auto by default");
set(&mut s, "playback.hardwareDecoding", json!("off"));
assert!(!filmcraft_codecs::hw::hardware_decoding(), "Off reaches the decoder registry");
assert!(s.execute("prefs.set", json!({"key": "playback.hardwareDecoding", "value": "gpu"})).is_err());
set(&mut s, "playback.hardwareDecoding", json!("auto"));
assert!(filmcraft_codecs::hw::hardware_decoding());
assert!(settings::field("playback.hardwareDecoding").is_some_and(|f| f.wired && matches!(f.kind, Kind::Choice(_))));
}
+4 -2
View File
@@ -441,7 +441,9 @@ fn bindable(c: &CommandSpec) -> bool {
}
_ => {}
}
fields.last_mut().expect("field").push(ch);
if let Some(f) = fields.last_mut() {
f.push(ch);
}
}
fields.iter().all(|f| f.contains('?') || f.contains('='))
}
@@ -505,7 +507,7 @@ impl Shortcuts {
dir: None,
session_start: None,
};
s.bindings = s.builtin(DEFAULT_PRESET).expect("default preset");
s.bindings = s.builtin(DEFAULT_PRESET).unwrap_or_default();
s
}
+12
View File
@@ -83,6 +83,18 @@ fn trim_linked_and_ripple_delete() {
assert_eq!(q.video_tracks[0].items[0].start, s.sequence_rate().tick_of(12), "the trim gap remains");
}
#[test]
fn slide_moves_linked_audio_with_the_video() {
let mut s = demo();
let v = s.active_sequence().unwrap().video_tracks[0].items[1].clone();
s.execute("timeline.slide", json!({"clip": v.id.0, "deltaFrames": 5})).unwrap();
let q = s.active_sequence().unwrap();
let (_, nv) = q.find_item(v.id).unwrap();
let a = q.audio_tracks[0].items.iter().find(|i| i.link == v.link).unwrap();
assert_eq!(nv.start, v.start + s.sequence_rate().tick_of(5));
assert_eq!((a.start, a.source_in), (nv.start, nv.source_in), "linked audio slides too and stays in sync");
}
#[test]
fn zero_fps_is_refused() {
let mut s = demo();
+1 -1
View File
@@ -433,7 +433,7 @@ pub fn shuttle(s: &mut Session, p: &Value) -> Result<Value> {
tick(s, clock)?;
}
let rate = s.sequence_rate();
let d = s.trim_play.dynamic.as_mut().expect("dynamic trim");
let d = s.trim_play.dynamic.as_mut().ok_or_else(|| EngineError::Other("dynamic trimming is not active".into()))?;
let speed = if slow {
0.25 * dir
} else if d.speed * dir > 0.0 && !d.at_limit {
+80
View File
@@ -0,0 +1,80 @@
//! An export depends only on the project and the settings: the same file on every machine, however
//! many cores render it and however the work is cut into steps.
use super::*;
use crate::tests::project;
fn tmp(name: &str) -> String {
let d = std::env::temp_dir().join(format!("fc-export-det-{}", std::process::id()));
std::fs::create_dir_all(&d).unwrap();
d.join(name).to_string_lossy().to_string()
}
/// Export on a machine with `threads` cores: `export` renders one frame per core of the pool it
/// runs in.
fn export_with_cores(threads: usize, settings: &ExportSettings) -> Vec<u8> {
let (p, seq, m) = project();
let pool = rayon::ThreadPoolBuilder::new().num_threads(threads).build().unwrap();
pool.install(|| export(&p, seq, settings, &m, &Progress::default())).unwrap();
std::fs::read(&settings.path).unwrap()
}
/// Export a step at a time with `batch` frames per step (the web app uses 1).
fn export_stepped(batch: i64, settings: &ExportSettings) -> Vec<u8> {
let (p, seq, m) = project();
let prog = Progress::default();
let mut ex = Exporter::new(p, seq, settings, &prog).unwrap();
ex.set_batch(batch);
while !matches!(ex.step(&m, &prog).unwrap(), Step::Done(_)) {}
std::fs::read(&settings.path).unwrap()
}
#[test]
fn same_file_whatever_the_core_count() {
// 24 frames: one full interleave group and a partial one
for (format, ext) in [(Format::ProRes, "mov"), (Format::H264, "mp4"), (Format::Mjpeg, "mov")] {
let settings = |name: &str| ExportSettings { format, path: tmp(&format!("{name}.{ext}")), ..Default::default() };
let reference = export_with_cores(2, &settings(&format!("{}-2", format.id())));
for threads in [3, 5, 16] {
let other = export_with_cores(threads, &settings(&format!("{}-{threads}", format.id())));
assert!(reference == other, "{}: {threads} cores give a different file than 2 cores", format.label());
}
for batch in [1, 7] {
let other = export_stepped(batch, &settings(&format!("{}-b{batch}", format.id())));
assert!(reference == other, "{}: {batch} frame(s) per step give a different file", format.label());
}
}
}
/// Slice NAL units (types 1 and 5) in a length-prefixed (4-byte) H.264 sample.
fn slices_in(sample: &[u8]) -> usize {
let (mut n, mut i) = (0, 0);
while i + 4 <= sample.len() {
let len = u32::from_be_bytes([sample[i], sample[i + 1], sample[i + 2], sample[i + 3]]) as usize;
if let Some(&h) = sample.get(i + 4)
&& matches!(h & 0x1f, 1 | 5)
{
n += 1;
}
i += 4 + len;
}
n
}
#[test]
fn h264_slices_follow_the_frame_size_not_the_cores() {
// 4096 lines = 256 macroblock rows: one slice per four rows on every machine, also on machines
// with fewer than 64 cores (the encoder's default caps the slices at the core count)
let path = tmp("tall.mp4");
let s = ExportSettings { format: Format::H264, path: path.clone(), frame_size: Some((64, 4096)), include_audio: false, ..Default::default() };
let (p, seq, m) = project();
export(&p, seq, &s, &m, &Progress::default()).unwrap();
let bytes = std::fs::read(&path).unwrap();
let file = filmcraft_isobmff::open(bytes.as_slice()).unwrap();
let vt = file.track_of_kind(filmcraft_isobmff::TrackKind::Video).unwrap();
assert_eq!(file.tracks[vt].samples.len(), 24);
for i in 0..file.tracks[vt].samples.len() {
let sample = file.read_sample(bytes.as_slice(), vt, i).unwrap();
assert_eq!(slices_in(&sample), 64, "picture {i}");
}
}
+47 -18
View File
@@ -11,6 +11,10 @@
//! When a source's bytes are not available yet (asynchronous web reads mark
//! [`filmcraft_media::pending`]), the batch is rendered again on the next call: nothing is encoded
//! from frames or audio with missing media.
//!
//! The file does not depend on how many frames a step renders (one per core natively, one on the
//! web): packets go to the muxer in fixed groups of [`INTERLEAVE`] output frames, each followed by
//! the audio up to its end, so the same project and settings give the same bytes on any machine.
use std::sync::Arc;
use std::sync::atomic::Ordering;
@@ -30,6 +34,11 @@ use crate::{
audio_factories, export_range, frame_span, video_factories,
};
/// Output frames per interleaved group: the video packets of these frames, then the audio up to
/// their end, form one chunk of each track. Fixed (not the batch size, which follows the core
/// count) so the file layout is the same on every machine.
pub const INTERLEAVE: i64 = 16;
/// What one [`Exporter::step`] did.
#[derive(Debug)]
pub enum Step {
@@ -58,7 +67,9 @@ pub struct Exporter {
measured: bool,
/// The first pass of a two-pass encode is running.
first_pass: bool,
/// Created after the first batch (encoders may finalise their codec config then).
/// Encoded video packets not written yet (the current [`INTERLEAVE`] group).
queued: Vec<EncodedPacket>,
/// Created after the first group was encoded (encoders may finalise their codec config then).
mux: Option<Mp4Writer<Out>>,
/// MXF exports write through this instead of `mux`.
mxf: Option<MxfMux>,
@@ -140,6 +151,7 @@ impl Exporter {
audio,
measured,
first_pass: two_pass,
queued: Vec::new(),
mux: None,
mxf: None,
seq,
@@ -177,7 +189,7 @@ impl Exporter {
if n == 0 {
return Ok(());
}
let mux = self.mux.as_mut().expect("mux created");
let mux = self.mux.as_mut().ok_or_else(|| ExportError::Encode("internal: the container writer was not created".into()))?;
match self.aenc.as_mut() {
Some(a) => {
for au in a.encode(&buf)? {
@@ -209,7 +221,7 @@ impl Exporter {
if let Some(m) = self.mxf.as_mut() {
return m.write_video(packets);
}
let mux = self.mux.as_mut().expect("mux created");
let mux = self.mux.as_mut().ok_or_else(|| ExportError::Encode("internal: the container writer was not created".into()))?;
for p in packets {
mux.write_sample(self.vt, WriteSample { data: &p.data, duration: p.duration, composition_offset: p.composition_offset, is_sync: p.key })
.map_err(|e| ExportError::Io(e.to_string()))?;
@@ -270,7 +282,7 @@ impl Exporter {
if !self.settings.part_of_batch {
progress.set_status("Measuring loudness");
}
let a = self.audio.as_mut().expect("audio");
let a = self.audio.as_mut().ok_or_else(|| ExportError::Encode("internal: the audio pipeline was not created".into()))?;
a.measure(&self.settings, sources, &|| progress.cancel.load(Ordering::Relaxed))?;
if filmcraft_media::pending::take() {
return Ok(Step::Pending);
@@ -295,22 +307,33 @@ impl Exporter {
self.next = end;
return Ok(Step::Progress);
}
let mixed = match self.audio.as_ref().map(|a| a.sr) {
Some(sr) if (self.mux.is_none() && self.mxf.is_none()) || self.at.is_some() || self.mxf.is_some() => {
let until = self.sample_at_frame(end, sr);
self.audio.as_mut().and_then(|a| a.pull(until, sources))
// the audio of every interleave group that ends in this batch (before encoding, so a
// pending source can still make the batch run again)
let cuts: Vec<i64> = (f + 1..=end).filter(|&g| (g - self.f0) % INTERLEAVE == 0).collect();
let mut mixed = Vec::with_capacity(cuts.len());
if let Some(sr) = self.audio.as_ref().map(|a| a.sr) {
for &g in &cuts {
let until = self.sample_at_frame(g, sr);
mixed.push(self.audio.as_mut().and_then(|a| a.pull(until, sources)));
}
_ => None,
};
}
if filmcraft_media::pending::take() {
return Ok(Step::Pending);
}
let packets = self.encode(&frames, f)?;
if self.mux.is_none() && self.mxf.is_none() {
self.open_mux()?;
let mut mixed = mixed.into_iter();
for (k, frame) in frames.iter().enumerate() {
let fi = f + k as i64;
let packets = self.encode(std::slice::from_ref(frame), fi)?;
self.queued.extend(packets);
if cuts.contains(&(fi + 1)) {
if self.mux.is_none() && self.mxf.is_none() {
self.open_mux()?;
}
let packets = std::mem::take(&mut self.queued);
self.write_video(packets)?;
self.write_audio(mixed.next().flatten())?;
}
}
self.write_video(packets)?;
self.write_audio(mixed)?;
progress.done.fetch_add((end - f) as u64, Ordering::Relaxed);
self.next = end;
return Ok(Step::Progress);
@@ -332,12 +355,13 @@ impl Exporter {
if self.mux.is_none() && self.mxf.is_none() {
self.open_mux()?;
}
let tail = self.venc.flush()?;
let mut tail = std::mem::take(&mut self.queued);
tail.extend(self.venc.flush()?);
self.write_video(tail)?;
self.write_audio(mixed)?;
if let (Some(at), Some(a)) = (self.at, self.aenc.as_mut()) {
let fs = a.frame_size();
let mux = self.mux.as_mut().expect("mux created");
let mux = self.mux.as_mut().ok_or_else(|| ExportError::Encode("internal: the container writer was not created".into()))?;
for au in a.flush()? {
mux.write_sample(at, WriteSample { data: &au, duration: fs, composition_offset: 0, is_sync: true })
.map_err(|e| ExportError::Io(e.to_string()))?;
@@ -346,7 +370,12 @@ impl Exporter {
let (bytes, extra_files) = match self.mxf.take() {
Some(m) => m.finish(&self.settings)?,
None => {
let w = self.mux.take().expect("mux created").finish().map_err(|e| ExportError::Io(e.to_string()))?;
let w = self
.mux
.take()
.ok_or_else(|| ExportError::Encode("internal: the container writer was not created".into()))?
.finish()
.map_err(|e| ExportError::Io(e.to_string()))?;
(w.finish(&self.settings)?, Vec::new())
}
};
+17 -2
View File
@@ -11,6 +11,7 @@
//! [`ExportSettings`] carries every Export-mode setting (frame size, rate, bitrate encoding, audio
//! format, multiplexer, captions, effects, metadata) as serde data; [`presets`] defines the
//! built-in presets.
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
use std::io::Write;
@@ -568,7 +569,7 @@ impl VideoEncoder for MjpegEncoder {
self.rate.num as u32
}
fn encode(&mut self, f: &EncoderFrame) -> Result<Vec<EncodedPacket>> {
let rgb: Vec<u8> = f.rgba.chunks_exact(4).flat_map(|p| [p[0], p[1], p[2]]).collect();
let rgb: Vec<u8> = f.rgba.as_chunks::<4>().0.iter().flat_map(|p| [p[0], p[1], p[2]]).collect();
let mut out = Vec::new();
let mut enc = image::codecs::jpeg::JpegEncoder::new_with_quality(&mut out, self.quality);
enc.encode(&rgb, f.width, f.height, image::ExtendedColorType::Rgb8).map_err(|e| ExportError::Encode(e.to_string()))?;
@@ -916,6 +917,13 @@ pub fn rgbf_to_yuv420_8(rgb: &[f32], w: usize, h: usize, kr: f32, kb: f32, y: &m
});
}
/// Slices per H.264 picture: one per four macroblock rows. The encoder's default follows the core
/// count, which would make the stream (and every decoded picture) depend on the machine; this is
/// what it picks on a machine with enough cores.
pub(crate) fn h264_slices(height: u32) -> usize {
(height.div_ceil(16) as usize).div_ceil(4).max(1)
}
fn h264_factory(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettings) -> Option<Result<Box<dyn VideoEncoder>>> {
if format != Format::H264 {
return None;
@@ -925,6 +933,7 @@ fn h264_factory(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSetti
cfg.format = if s.format.is_mxf() { filmcraft_h264enc::PacketFormat::AnnexB } else { filmcraft_h264enc::PacketFormat::LengthPrefixed };
cfg.aud = s.format.is_mxf();
cfg.keyint = s.keyframe_distance.filter(|k| *k > 0).unwrap_or_else(|| (rate.num as f64 / rate.den as f64 * 2.0).round().max(1.0) as u32);
cfg.slices = h264_slices(h);
let kbps = s.bitrate_kbps.max(100);
let max = s.max_bitrate_kbps.filter(|m| *m >= kbps).unwrap_or(kbps * 3 / 2);
cfg.rate = match s.bitrate_mode {
@@ -1108,7 +1117,10 @@ pub fn export(project: &Arc<Project>, seq: ItemId, settings: &ExportSettings, so
}
(total, count)
}
Format::H264 | Format::ProRes | Format::DnxHr | Format::Mjpeg | Format::MxfOp1a | Format::MxfOpAtom => unreachable!("stepped export"),
Format::H264 | Format::ProRes | Format::DnxHr | Format::Mjpeg | Format::MxfOp1a | Format::MxfOpAtom => {
// Handled by the stepped exporter above; reaching here would be a dispatch bug.
return Err(ExportError::Unsupported(format!("{:?} must run as a stepped export", settings.format)));
}
};
let secs = t0.elapsed().as_secs_f64();
if !settings.part_of_batch {
@@ -1129,3 +1141,6 @@ mod surround_tests;
#[cfg(test)]
mod mxf_tests;
#[cfg(test)]
mod determinism_tests;
+2 -2
View File
@@ -225,7 +225,7 @@ fn ffmpeg_oracle_op1a_and_op_atom() {
let at = mx.track_of_kind(filmcraft_mxf::TrackKind::Sound).unwrap();
let ours = mx.read_pcm(&bytes, at, 0, 48_000).unwrap();
assert_eq!(pcm.len(), 48_000 * 2 * 4, "{codec:?}");
for (i, c) in pcm.chunks_exact(4).enumerate() {
for (i, c) in pcm.as_chunks::<4>().0.iter().enumerate() {
let s = i32::from_le_bytes([c[0], c[1], c[2], c[3]]) >> 8;
assert_eq!(s as f32 / 8_388_608.0, ours[i % 2][i / 2], "{codec:?}: sample {i}");
}
@@ -255,7 +255,7 @@ fn ffmpeg_oracle_op1a_and_op_atom() {
assert_eq!((s["codec_name"].as_str(), s["channels"].as_u64(), s["sample_rate"].as_str()), (Some("pcm_s24le"), Some(1), Some("48000")), "{a}");
let mono = ffmpeg_raw(&ffmpeg, a, "0:a:0", &["-f", "s32le", "-acodec", "pcm_s32le"]);
assert_eq!(mono.len(), 48_000 * 4, "{a}");
let want: Vec<u8> = stereo.chunks_exact(8).flat_map(|f| f[k * 4..k * 4 + 4].to_vec()).collect();
let want: Vec<u8> = stereo.as_chunks::<8>().0.iter().flat_map(|f| f[k * 4..k * 4 + 4].to_vec()).collect();
assert!(mono == want, "{a}: channel {k} differs from the WAV export");
}
}
+4 -4
View File
@@ -111,7 +111,7 @@ impl Pipeline {
if let Some(tf) = self.out_tf.as_ref().filter(|_| self.hdr_out) {
let mut out = vec![0f32; img.w * img.h * 3];
out.par_chunks_mut(img.w * 3).zip(img.px.par_chunks(img.w * 4)).for_each(|(o, s)| {
for (o, p) in o.chunks_exact_mut(3).zip(s.chunks_exact(4)) {
for (o, p) in o.as_chunks_mut::<3>().0.iter_mut().zip(s.as_chunks::<4>().0) {
o.copy_from_slice(&tf.encode([p[0], p[1], p[2]]));
}
});
@@ -239,7 +239,7 @@ fn load_overlay(path: &str, width: f32) -> Result<Image> {
let rgba = image::load_from_memory(&bytes).map_err(|e| ExportError::Unsupported(format!("image overlay {path}: {e}")))?.to_rgba8();
let (iw, ih) = (rgba.width() as usize, rgba.height() as usize);
let mut img = Image::new(iw, ih);
for (o, p) in img.px.chunks_exact_mut(4).zip(rgba.as_raw().chunks_exact(4)) {
for (o, p) in img.px.as_chunks_mut::<4>().0.iter_mut().zip(rgba.as_raw().as_chunks::<4>().0) {
let a = p[3] as f32 / 255.0;
for c in 0..3 {
o[c] = filmcraft_color::srgb_to_linear(p[c] as f32 / 255.0) * a;
@@ -310,7 +310,7 @@ fn resample_axis(src: &Image, n: usize, horizontal: bool) -> Image {
pub fn limit_rgba8(rgba: &mut [u8], min_percent: f32, max_percent: f32) {
let (lo, hi) = (min_percent.clamp(0.0, 100.0) / 100.0, max_percent.clamp(0.0, 100.0) / 100.0);
rgba.par_chunks_mut(4 * 1024).for_each(|chunk| {
for p in chunk.chunks_exact_mut(4) {
for p in chunk.as_chunks_mut::<4>().0 {
let mut c = [p[0] as f32 / 255.0, p[1] as f32 / 255.0, p[2] as f32 / 255.0];
limit_px(&mut c, lo, hi);
for i in 0..3 {
@@ -323,7 +323,7 @@ pub fn limit_rgba8(rgba: &mut [u8], min_percent: f32, max_percent: f32) {
fn limit_f32(rgb: &mut [f32], min_percent: f32, max_percent: f32) {
let (lo, hi) = (min_percent.clamp(0.0, 100.0) / 100.0, max_percent.clamp(0.0, 100.0) / 100.0);
rgb.par_chunks_mut(3 * 1024).for_each(|chunk| {
for p in chunk.chunks_exact_mut(3) {
for p in chunk.as_chunks_mut::<3>().0 {
let mut c = [p[0], p[1], p[2]];
limit_px(&mut c, lo, hi);
p.copy_from_slice(&c);
+1 -1
View File
@@ -204,7 +204,7 @@ fn read_wav16(path: &str) -> Vec<Vec<f32>> {
let ch = u16::from_le_bytes([b[22], b[23]]) as usize;
let data = &b[44..];
let mut out = vec![Vec::new(); ch];
for (i, s) in data.chunks_exact(2).enumerate() {
for (i, s) in data.as_chunks::<2>().0.iter().enumerate() {
out[i % ch].push(i16::from_le_bytes([s[0], s[1]]) as f32 / 32768.0);
}
out
+1 -1
View File
@@ -110,7 +110,7 @@ fn decode(path: &str) -> Option<Vec<Vec<f64>>> {
let out =
std::process::Command::new(ffmpeg).args(["-v", "error", "-i", path, "-map", "0:a:0", "-f", "s32le", "-acodec", "pcm_s32le", "-"]).output().unwrap();
assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr));
let ints: Vec<i32> = out.stdout.chunks_exact(4).map(|b| i32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
let ints: Vec<i32> = out.stdout.as_chunks::<4>().0.iter().map(|b| i32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
let n = ints.len() / 6;
Some((0..6).map(|c| (0..n).map(|i| ints[i * 6 + c] as f64).collect()).collect())
}
+2
View File
@@ -34,6 +34,8 @@
//! per-character text styles (`TrackItem::graphic`, `EffectInstance::layer`) and source graphics
//! (`Project::source_graphics`). No-op step; older builds would drop these fields when saving.
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
pub mod atomic;
pub mod autosave;
+4 -2
View File
@@ -5,6 +5,8 @@
//! compositor. Pixel data is `Arc`-shared so caches, monitors and export share frames for free.
//! - [`AudioBuffer`]: planar f32 samples.
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
use std::sync::Arc;
use filmcraft_color::{ColorInfo, DecodeTable, Matrix, Range, linear_to_srgb_u8, normalize_c, normalize_y, srgb_u8_to_linear_table, to_linear, ycbcr_to_rgb};
@@ -383,14 +385,14 @@ impl VideoFrame {
pub fn luma8(&self) -> Vec<u8> {
match &self.data {
PixelData::Yuv8 { planes, .. } => planes[0].as_ref().clone(),
_ => self.to_rgba8().chunks_exact(4).map(|p| (0.2126 * p[0] as f32 + 0.7152 * p[1] as f32 + 0.0722 * p[2] as f32) as u8).collect(),
_ => self.to_rgba8().as_chunks::<4>().0.iter().map(|p| (0.2126 * p[0] as f32 + 0.7152 * p[1] as f32 + 0.0722 * p[2] as f32) as u8).collect(),
}
}
}
/// Convert a row of premultiplied linear f32 RGBA into straight sRGB RGBA8.
pub fn linear_premul_to_srgb8(src: &[f32], dst: &mut [u8]) {
for (s, o) in src.chunks_exact(4).zip(dst.chunks_exact_mut(4)) {
for (s, o) in src.as_chunks::<4>().0.iter().zip(dst.as_chunks_mut::<4>().0) {
let a = s[3].clamp(0.0, 1.0);
if a <= 0.0 {
o.copy_from_slice(&[0, 0, 0, 0]);
+2
View File
@@ -1,5 +1,7 @@
//! 2D geometry for FilmCraft: vectors, rectangles and affine transforms (Motion effect math).
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
use serde::{Deserialize, Serialize};
use std::ops::{Add, Mul, Sub};
+203 -3
View File
@@ -185,7 +185,9 @@ impl MediaSource for Encoded {
let m = filmcraft_color::spaces::to_f32(&filmcraft_color::spaces::gamut_matrix(Gamut::Bt709, self.target.gamut()));
let curve = self.target.curve();
let out: Vec<u8> = rgba
.chunks_exact(4)
.as_chunks::<4>()
.0
.iter()
.flat_map(|p| {
let lin = [0, 1, 2].map(|k| filmcraft_color::srgb_to_linear(p[k] as f32 / 255.0) * self.gain);
let c = filmcraft_color::spaces::apply3(&m, lin);
@@ -723,7 +725,7 @@ fn lumetri_presets_grid() {
fn scenes_are_distinct() {
let imgs: Vec<(&str, Rgba8)> = scenes().into_iter().map(|(n, _, f)| (n, render_cpu(&f()))).collect();
for (n, img) in &imgs {
let lit = img.px.chunks_exact(4).filter(|p| p[0] as u32 + p[1] as u32 + p[2] as u32 > 30).count();
let lit = img.px.as_chunks::<4>().0.iter().filter(|p| p[0] as u32 + p[1] as u32 + p[2] as u32 > 30).count();
assert!(lit > img.px.len() / 4 / 4, "{n}: mostly black");
}
for i in 0..imgs.len() {
@@ -758,7 +760,7 @@ fn gpu_matches_cpu_on_golden_scenes() {
let kind = if matches!(plan, filmcraft_render::plan::FramePlan::Layers { .. }) { "layers" } else { "cpu image" };
c.composite(&plan);
let (gw, gh, mut px) = c.read_output().expect("GPU readback");
px.chunks_exact_mut(4).for_each(|p| p[3] = 255);
px.as_chunks_mut::<4>().0.iter_mut().for_each(|p| p[3] = 255);
let gpu = Rgba8::new(gw, gh, px);
let d = diff(&cpu, &gpu).unwrap();
// mean over RGB only (alpha is equal by construction)
@@ -773,3 +775,201 @@ fn gpu_matches_cpu_on_golden_scenes() {
}
assert!(failures.is_empty(), "GPU differs from the CPU reference:\n{}", failures.join("\n"));
}
/// Per-pixel max RGB difference (99th percentile, mean) over the pixels at least 1.5 px from every
/// plan layer's quad outline: the CPU resampler fades a layer's edge over one more pixel than the
/// GPU rasterises, and with blend modes those edge pixels are a visible share of a 320×180 frame.
fn interior_diff(plan: &filmcraft_render::plan::FramePlan, a: &Rgba8, b: &Rgba8) -> (u32, f64) {
let filmcraft_render::plan::FramePlan::Layers { width, layers, .. } = plan else { return (u32::MAX, f64::MAX) };
let inv: Vec<_> = layers.iter().filter_map(|l| Some((l.matrix.inverse()?, l.size().0 as f64, l.size().1 as f64))).collect();
let mut d: Vec<u32> = Vec::new();
for (i, (pa, pb)) in a.px.chunks(4).zip(b.px.chunks(4)).enumerate() {
let (x, y) = ((i % width) as f64 + 0.5, (i / width) as f64 + 0.5);
let edge = inv.iter().any(|(m, fw, fh)| {
let c = [(-1.5, -1.5), (1.5, -1.5), (-1.5, 1.5), (1.5, 1.5)].map(|(dx, dy)| {
let p = m.apply(Vec2::new(x + dx, y + dy));
p.x >= 0.0 && p.y >= 0.0 && p.x <= *fw && p.y <= *fh
});
c.iter().any(|v| *v != c[0])
});
if !edge {
d.push((0..3).map(|k| (pa[k] as i32 - pb[k] as i32).unsigned_abs()).max().unwrap_or(0));
}
}
d.sort_unstable();
(d[d.len() * 99 / 100], d.iter().sum::<u32>() as f64 / d.len() as f64)
}
/// Every blend mode composited by the GPU from `plan_frame` (no CPU fallback): two overlapping
/// transformed clips at partial opacity in the mode over footage, plus a clip with a standard
/// effect (rendered on the CPU as a layer image) in the same mode, match the CPU render with the
/// GPU parity tolerance (p99 ≤ 6, mean < 1.5) away from layer outlines; whole-frame numbers are
/// printed too.
#[test]
fn gpu_blend_modes_match_cpu_render() {
let Some((dev, q)) = device() else {
eprintln!("SKIPPED (gpu blend parity): no GPU adapter");
return;
};
let mut c = filmcraft_gpu::GpuCompositor::new(&dev, &q);
let mut failures = Vec::new();
for mode in filmcraft_project::effect::BLEND_MODES {
let mut b = Builder::new(4);
let bg = b.demo(DemoScene::OceanSunset);
let (dunes, aurora, bars) = (b.demo(DemoScene::Dunes), b.demo(DemoScene::Aurora), b.media(Generator::BarsAndTone));
b.place(0, bg, 0, 48);
let v2 = b.place(1, dunes, 0, 48);
b.fixed(v2, "motion", &[("scale", fl(60.0)), ("rotation", fl(12.0)), ("position", pt(130.0, 80.0))]);
b.fixed(v2, "opacity", &[("opacity", fl(70.0)), ("blend", blend(mode))]);
let v3 = b.place(2, aurora, 0, 48);
b.fixed(v3, "motion", &[("scale", fl(45.0)), ("rotation", fl(-20.0)), ("position", pt(210.0, 110.0))]);
b.fixed(v3, "opacity", &[("opacity", fl(60.0)), ("blend", blend(mode))]);
let v4 = b.place(3, bars, 0, 48);
b.fixed(v4, "motion", &[("scale", fl(30.0)), ("position", pt(250.0, 50.0))]);
b.fixed(v4, "opacity", &[("opacity", fl(85.0)), ("blend", blend(mode))]);
b.effect(v4, "crop", &[("left", fl(10.0)), ("bottom", fl(20.0))]);
let s = b.at(12);
let t = RATE.tick_of(s.frame);
let cpu = render_cpu(&s);
let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, t, RenderOptions::default(), &s.sources);
let filmcraft_render::plan::FramePlan::Layers { layers, .. } = &plan else {
failures.push(format!("{mode}: planned as a CPU image"));
continue;
};
assert_eq!(layers.len(), 4, "{mode}");
c.composite(&plan);
let (gw, gh, mut px) = c.read_output().expect("GPU readback");
px.as_chunks_mut::<4>().0.iter_mut().for_each(|p| p[3] = 255);
let gpu = Rgba8::new(gw, gh, px);
let d = diff(&cpu, &gpu).unwrap();
let (p99, mean) = interior_diff(&plan, &cpu, &gpu);
eprintln!("{mode}: GPU vs CPU interior p99 {p99}, mean {mean:.3}; whole frame {d}");
if p99 > 6 || mean >= 1.5 {
failures.push(format!("{mode}: interior p99 {p99}, mean {mean}; {d}"));
}
}
assert!(failures.is_empty(), "GPU blend modes differ from the CPU render:\n{}", failures.join("\n"));
}
/// Inside a transition the CPU mixes the clips and composites the result Normal, ignoring their
/// blend modes: the plan does the same (Screen clips cross-dissolving on V2 over V1).
#[test]
fn gpu_transition_of_blended_clips_matches_cpu_render() {
let Some((dev, q)) = device() else {
eprintln!("SKIPPED (gpu blend parity): no GPU adapter");
return;
};
let mut b = Builder::new(2);
let (bg, x, y) = (b.demo(DemoScene::OceanSunset), b.demo(DemoScene::Aurora), b.demo(DemoScene::Dunes));
b.place(0, bg, 0, 48);
let ca = b.place(1, x, 0, 24);
let cb = b.place(1, y, 24, 24);
for id in [ca, cb] {
b.fixed(id, "opacity", &[("blend", blend("Screen"))]);
}
b.transition(1, "cross_dissolve", ca, cb, 24, 12);
let s = b.at(22);
let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, RATE.tick_of(s.frame), RenderOptions::default(), &s.sources);
let filmcraft_render::plan::FramePlan::Layers { layers, .. } = &plan else { panic!("planned as a CPU image") };
assert!(layers.iter().all(|l| l.blend == filmcraft_render::Blend::Normal));
let mut c = filmcraft_gpu::GpuCompositor::new(&dev, &q);
c.composite(&plan);
let (gw, gh, mut px) = c.read_output().expect("GPU readback");
px.as_chunks_mut::<4>().0.iter_mut().for_each(|p| p[3] = 255);
let d = diff(&render_cpu(&s), &Rgba8::new(gw, gh, px)).unwrap();
eprintln!("Screen clips in a cross dissolve: GPU vs CPU {d}");
assert!(d.p99 <= 6 && d.mean_abs * 4.0 / 3.0 < 1.5, "{d}");
}
/// Standard effect chains run on the GPU: every clip below plans as a GPU layer with its effects
/// (no CPU-rendered image), including keyframed parameters, transformed clips, partial opacity
/// and blend modes, and matches the CPU render with the GPU parity tolerance (p99 ≤ 6, mean < 1.5
/// away from layer outlines) at three times. A clip with an effect the GPU stage lacks (Emboss)
/// still falls back to a CPU layer image.
#[test]
fn gpu_effect_chains_match_cpu_render() {
let mut b = Builder::new(5);
let (bg, dunes, aurora, bars, sunset) =
(b.demo(DemoScene::OceanSunset), b.demo(DemoScene::Dunes), b.demo(DemoScene::Aurora), b.media(Generator::BarsAndTone), b.demo(DemoScene::OceanSunset));
let v1 = b.place(0, bg, 0, 48);
b.effect(v1, "brightness_contrast", &[("brightness", fl(-10.0)), ("contrast", fl(20.0))]);
b.effect(v1, "gaussian_blur", &[("repeat_edge", ParamValue::Bool(true))]);
// keyframed blur: evaluated at the frame's time
let p = b.clip(v1).effects.last_mut().and_then(|e| e.params.get_mut("blurriness")).expect("blurriness");
p.put_keyframe(Tick::ZERO, fl(2.0));
p.put_keyframe(RATE.tick_of(24), fl(20.0));
let v2 = b.place(1, dunes, 0, 48);
b.fixed(v2, "motion", &[("scale", fl(60.0)), ("rotation", fl(12.0)), ("position", pt(130.0, 80.0))]);
b.fixed(v2, "opacity", &[("opacity", fl(70.0)), ("blend", blend("Screen"))]);
b.effect(v2, "tint", &[("amount", fl(60.0))]);
b.effect(v2, "unsharp_mask", &[("amount", fl(120.0)), ("radius", fl(2.0))]);
b.effect(v2, "crop", &[("left", fl(8.0)), ("bottom", fl(12.0)), ("feather", fl(6.0))]);
let v3 = b.place(2, aurora, 0, 48);
b.fixed(v3, "motion", &[("scale", fl(45.0)), ("rotation", fl(-20.0)), ("position", pt(210.0, 110.0))]);
b.fixed(v3, "opacity", &[("opacity", fl(85.0))]);
b.effect(v3, "levels", &[("in_black", fl(20.0)), ("gamma", fl(130.0))]);
b.effect(v3, "mirror", &[("angle", fl(60.0))]);
b.effect(v3, "directional_blur", &[("length", fl(4.0)), ("direction", fl(45.0))]);
b.effect(v3, "transform", &[("rotation", fl(10.0)), ("scale_height", fl(110.0))]);
let v4 = b.place(3, bars, 0, 48);
b.fixed(v4, "motion", &[("scale", fl(30.0)), ("position", pt(250.0, 50.0))]);
b.fixed(v4, "opacity", &[("blend", blend("Multiply"))]);
b.effect(v4, "posterize", &[("levels", fl(5.0))]);
b.effect(v4, "invert", &[("blend", fl(40.0))]);
b.effect(v4, "offset", &[("shift", pt(400.0, 120.0))]);
b.effect(v4, "horizontal_flip", &[]);
b.effect(v4, "black_white", &[]);
let v5 = b.place(4, sunset, 0, 48);
b.fixed(v5, "motion", &[("scale", fl(25.0)), ("position", pt(60.0, 140.0))]);
b.effect(v5, "emboss", &[]);
let mut s = b.at(12);
let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, RATE.tick_of(s.frame), RenderOptions::default(), &s.sources);
let filmcraft_render::plan::FramePlan::Layers { layers, .. } = &plan else { panic!("planned as a CPU image") };
assert_eq!(layers.len(), 5);
for (i, l) in layers.iter().take(4).enumerate() {
let fx = l.fx.as_ref().unwrap_or_else(|| panic!("V{}: not a GPU effect layer", i + 1));
assert!(!fx.ops.is_empty());
}
assert!(layers[4].fx.is_none(), "Emboss is rendered on the CPU");
let Some((dev, q)) = device() else {
eprintln!("SKIPPED (gpu effect parity): no GPU adapter");
return;
};
let mut c = filmcraft_gpu::GpuCompositor::new(&dev, &q);
let mut failures = Vec::new();
for frame in [0, 12, 30] {
s.frame = frame;
let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, RATE.tick_of(frame), RenderOptions::default(), &s.sources);
c.composite(&plan);
let (gw, gh, mut px) = c.read_output().expect("GPU readback");
px.as_chunks_mut::<4>().0.iter_mut().for_each(|p| p[3] = 255);
let gpu = Rgba8::new(gw, gh, px);
let cpu = render_cpu(&s);
let d = diff(&cpu, &gpu).unwrap();
let (p99, mean) = interior_diff(&plan, &cpu, &gpu);
eprintln!("effect chains @ {frame}: GPU vs CPU interior p99 {p99}, mean {mean:.3}; whole frame {d}");
if p99 > 6 || mean >= 1.5 {
let dir = filmcraft_testkit::golden::failures_dir();
let _ = filmcraft_testkit::golden::write_png(&dir.join(format!("fx_chains_{frame}.gpu.png")), &gpu);
let _ = filmcraft_testkit::golden::write_png(&dir.join(format!("fx_chains_{frame}.cpu.png")), &cpu);
failures.push(format!("frame {frame}: interior p99 {p99}, mean {mean}; {d}; images in {}", dir.display()));
}
}
// ½ playback resolution: the working images (and the effects' pixel radii) shrink with it
let opts = RenderOptions { scale: 0.5, ..RenderOptions::default() };
let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, RATE.tick_of(s.frame), opts, &s.sources);
let filmcraft_render::plan::FramePlan::Layers { layers, .. } = &plan else { panic!("½: planned as a CPU image") };
assert!(layers.iter().take(4).all(|l| l.fx.is_some()));
c.composite(&plan);
let (gw, gh, mut px) = c.read_output().expect("GPU readback");
px.as_chunks_mut::<4>().0.iter_mut().for_each(|p| p[3] = 255);
let gpu = Rgba8::new(gw, gh, px);
let img = render_sequence(&s.project, s.seq, RATE.tick_of(s.frame), opts, &s.sources);
let cpu = Rgba8::new(img.w as u32, img.h as u32, img.over_black_rgba8());
let (p99, mean) = interior_diff(&plan, &cpu, &gpu);
eprintln!("effect chains @ ½: GPU vs CPU interior p99 {p99}, mean {mean:.3}; whole frame {}", diff(&cpu, &gpu).unwrap());
if p99 > 6 || mean >= 1.5 {
failures.push(format!("½ resolution: interior p99 {p99}, mean {mean}"));
}
assert!(failures.is_empty(), "GPU effect chains differ from the CPU render:\n{}", failures.join("\n"));
}
Loaded 100 of 246 files, more files were not shown because too many files have changed in this diff. Show more