Compare commits

..
753 Commits
Author SHA1 Message Date
Brandon ThomasandClaude Opus 5.5 2ff3f5e266 Release prep: 0.6.0 and a Linux riscv64 release build (#830)
- release.yml: new `linux-riscv64` job, cross-compiled on ubuntu-24.04 against
  ports.ubuntu.com riscv64 libs (ported from GridCraft), tar.gz only, CLI
  smoke-tested under QEMU. Installs libasound2-dev:riscv64 for cpal/ALSA; the
  VA-API and NVDEC/NVENC paths are loaded with libloading at run time, so they
  need no riscv64 libraries. The draft-release job now needs it, so its tarball
  is uploaded and covered by SHA256SUMS.txt.
- packaging/linux/package.sh: CROSS_ARCH / CROSS_TARGET / CROSS_COMPILE /
  EMULATOR support, as in GridCraft.
- README and docs/releasing.md list the riscv64 tarball.
- cargo xtask version set 0.6.0 (Cargo.toml, Cargo.lock); refresh contributors.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 19:28:41 -04:00
Brandon Thomas 2a9f39db63 Merge pull request #825 from vangel76/gap-aac-audio
#814: AAC Audio export (audio-only .m4a, or ADTS .aac)
2026-10-10 19:22:45 -04:00
Brandon Thomas c47e883c27 Merge pull request #826 from vangel76/gap-tga-dpx-import
#821: import Targa and DPX stills and image sequences
2026-10-10 19:15:39 -04:00
Brandon Thomas 55b07f29ff Merge pull request #818 from vangel76/gap-mask-rounded-corners
#813: masks round their corners (rounded rectangle masks)
2026-10-10 19:15:33 -04:00
Brandon Thomas 4ecdb6ea71 Merge pull request #824 from ReReddo/feat/monitor-wheel-zoom
monitors: the wheel zooms about the pointer, the middle button pans
2026-10-10 19:10:45 -04:00
Brandon Thomas 5ad352ed04 Merge pull request #815 from vbasky/fix/new-bin-clear
Clear leftover item selection when making a new bin
2026-10-10 19:10:39 -04:00
vangel76 f8f16b6eca Merge remote-tracking branch 'upstream/main' into gap-aac-audio
# Conflicts:
#	crates/engine/src/commands.rs
#	docs/gaps.md
2026-10-11 01:09:25 +02:00
Brandon ThomasandClaude Opus 5.5 f114b5f10c Revert #783 (Linux H.265 through NVENC): intermittent heap corruption in the CLI
With NVENC registered on Linux, `filmcraft-cli --test cli` aborts in 2 of 45 runs on an RTX 3090
(`malloc(): mismatching next->prev_size`, `malloc(): unaligned fastbin chunk detected`) right
after an export finishes; 0 of 30 runs fail with this reverted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 19:01:15 -04:00
vangel76 2ca656684e Merge remote-tracking branch 'upstream/main' into gap-mask-rounded-corners
# Conflicts:
#	docs/gaps.md
2026-10-11 01:00:22 +02:00
vangel76andClaude Opus 5.5 fe52bc0b2c #821: import Targa and DPX stills and image sequences
Problem: Premiere imports Targa and DPX as stills and numbered sequences.
FilmCraft exports both (#779) but could not read either: the extensions
were not importable and the image crate has no DPX reader, so even our
own Targa / DPX renders could not come back in.

Fix: clean-room decoders in filmcraft-media. Targa from the TGA 2.0 spec:
image types 1/2/3 and RLE 9/10/11, colour maps, 15/16/24/32-bit, 8/16-bit
grayscale, any origin. DPX from SMPTE ST 268: first element, luma / RGB /
RGBA / ABGR, 8/16-bit, filled 10/12-bit (methods A and B), either byte
order, padded lines, orientations 0-3. Both bound allocations by the
data present and reject hostile headers with errors. Stills and image
sequences share one decode_raster entry point; "tga" and "dpx" join the
importable still extensions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 00:58:15 +02:00
vangel76andClaude Opus 5.5 b9f60842bb #814: AAC Audio export (audio-only .m4a, or ADTS .aac)
Problem: Premiere's "AAC Audio" format has no FilmCraft equivalent. AAC
was only written inside H.264 / H.265 / AV1 movies; the only audio-only
exports were Waveform Audio and AIFF.

Fix: a new `aac` export format ("AAC Audio") that encodes the sequence
mix with the existing AAC-LC encoder into an MPEG-4 audio file (.m4a,
encoder delay in an edit list), or a raw ADTS stream when the path ends
in .aac. Bitrate, sample rate, channels and loudness normalisation follow
the audio settings. Adds an "AAC Audio 48 kHz 320 kbps" Audio Only
preset, CLI extension guessing (.m4a / .aac), es/fr/ja/zh-cn strings, and
updates G13 (also noting ProRes 4444 / XQ is already in the export UI).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 00:56:35 +02:00
ReReddo 5f9908905c monitors: the wheel zooms about the pointer, the middle button pans (#807)
Zooming a monitor took the zoom dropdown, which zooms about the middle
of the picture, and then scrolling or the Hand tool to bring the part
you wanted into view; at Fit the wheel did nothing.

Now, on the Program and Source monitors:

- The mouse wheel (or a pinch) zooms about the pointer, from Fit too,
  between the dropdown's 10% and 1600%: the point under the pointer
  stays under it. It zooms as fast as the Timeline's wheel zoom, through
  the same wheel_input.
- A middle-button drag that starts over the picture pans it, whatever
  the tool; the Hand tool still pans with the left button.
- Fit resets zoom and pan, and each monitor keeps its own, as before.
  Comparison View, which always fits both frames, ignores the wheel.

Tests: view_ui (wheel zoom about the pointer, middle-button pan, the 10%
floor, Fit, Source untouched).
2026-10-11 01:55:26 +03:00
vangel76andClaude Opus 5.5 16493a7f70 #813: masks round their corners (rounded rectangle masks)
Problem: Premiere 26's redesigned mask tools give rectangle masks
rounded corners; FilmCraft masks only had sharp corners.

Fix: MaskPath::with_rounded_corners replaces each sharp corner between
straight edges by a circular fillet (two vertices joined by a cubic arc,
handles 4/3·tan(φ/4)·r, shrunk to fit half of each edge; hostile radii
and indices are clamped or ignored). masks.add takes `cornerRadius` for
a rounded rectangle, and the new masks.roundCorners command rounds an
existing mask's corners (or the listed vertices) in one undo step, on
every Mask Path keyframe so animated paths keep interpolating.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 00:47:09 +02:00
Brandon Thomas 9efe8f3112 Merge pull request #812 from vangel76/gap-exr-sequences
#806: export OpenEXR image sequences
2026-10-10 18:44:25 -04:00
Brandon Thomas 2bbc9fdb35 Merge pull request #811 from vangel76/gap-morph-cut
#805: Morph Cut morphs along optical flow instead of cross-dissolving
2026-10-10 18:44:19 -04:00
Brandon Thomas b42b80b529 Merge pull request #808 from vangel76/gap-mp2-priming
#800: MP2 audio in MP4 and MOV decodes instead of playing silent
2026-10-10 18:44:13 -04:00
Vikram Bhaskaran d4ec47a4a3 Clear leftover Project item selection when making a new bin.
New Bin only selected the folder and started rename, so Clear still targeted clips that were selected before. The trash button and Delete then removed those items and left the bin. Clicking a bin already deselects items; creating one now does the same, and Clear drops the rename field so it does not keep editing a bin that is about to go.
2026-10-11 09:39:29 +11:00
Brandon Thomas 4a81aa1742 Merge pull request #804 from vangel76/gap-french-localization
#498: French (fr) interface translation
2026-10-10 18:35:20 -04:00
Brandon Thomas 71e25aa045 Merge pull request #803 from Ni-zav/fix/771-shuttle-audio-s20261010T214728Z-4f9c
fix(playback): keep audio during forward shuttle speeds (#771)
2026-10-10 18:35:14 -04:00
vangel76andClaude Opus 5.5 fb5863bc8a #806: export OpenEXR image sequences
Problem: Premiere offers OpenEXR as an image-sequence export format;
FilmCraft could not write EXR at all (docs/gaps.md G13).

Fix: a new export format (id exr, "OpenEXR") writing numbered .exr stills,
with a preset, the Include Alpha Channel option, the file.exportMedia format
list and es/ja/zh-cn strings. The writer in crates/export/src/raster.rs
follows the public OpenEXR file layout: version 2 single-part scan-line
file, half-float B/G/R(/A) channels, no compression, one scan line per
chunk, Rec.709 chromaticities; values are scene-linear (the pipeline's
transfer undone) with premultiplied alpha, as EXR defines. No new
dependency. ZIP / PIZ / DWA compression stays open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 00:34:28 +02:00
vangel76andClaude Opus 5.5 50ce8b090c #805: Morph Cut morphs along optical flow instead of cross-dissolving
Problem: Morph Cut rendered as a plain Cross Dissolve, so a jump cut in a
talking-head shot showed the subject twice at half opacity during the
transition, and the GPU plan mixed it as a dissolve too.

Fix: Morph Cut warps both frames along the dense optical flow between them
(the Lucas-Kanade flow of #780, crates/render/src/flow.rs) and blends, so the
subject moves from its place in the outgoing shot to its place in the incoming
one; where the frames do not correspond it stays the cross dissolve. Frames
inside a Morph Cut are planned whole on the CPU (no longer a GPU-mixable
"simple" transition) and cost as a complex transition in the preview estimate.
Morph Cut goldens re-blessed. No face tracking yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 00:32:08 +02:00
vangel76andClaude Opus 5.5 c9b9bf3aae #800: MP2 audio in MP4 and MOV decodes instead of playing silent
Problem: MP4 and QuickTime name MPEG audio without its layer (ffmpeg
stores MP2 as `mp4a` + esds object type 0x6B), and we always made the
layer III decoder for it. symphonia's decoder reads only the layer it
was made for, so every MP2 frame was rejected and the track played as
silence; MOV's `.mp2` sample entry was not recognised at all.

Fix: the MPEG audio decoder for MP4 / MOV is made on the first packet,
for the layer its frame header names (ISO/IEC 11172-3 2.4.2.3);
`.mp2` maps to the MPEG audio codec like `.mp3`; the track is labelled
"MPEG Audio" for layers I/II, as Matroska's is. Decoded tracks now match
ffmpeg sample-exactly (edit list trims the 481-sample encoder delay).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 00:26:39 +02:00
Brandon Thomas 120883d41c Merge pull request #802 from Ni-zav/fix/786-label-menu-swatches-s20261010T214728Z-4f9c
fix(menus): show configured label colors in Edit menu
2026-10-10 18:25:47 -04:00
vangel76andClaude Opus 5.5 73f9982c0c #498: French (fr) interface translation
Problem: French, one of Premiere's interface languages and one of the
12 key languages in docs/gaps.md G14, had no catalog; #498 asked for it.

Fix: a new fr.tsv catalog with every entry Spanish has (menus, panels,
dialogs, Settings pages, the Effects / Effect Controls registry, the
Keyboard Shortcuts dialog, status messages) plus the native macOS
Settings item, written from the plain meaning of the English text with
ordinary French editing vocabulary (infinitive commands, "vous", French
sentence case and spacing). Language::Fr is persisted as `fr`;
app.language.french joins Edit > Language (checkmark, native macOS menu),
Settings > General offers Français, and System Language picks French for
any fr-* locale. Like Japanese, tests require only the core menus, so
later strings show in English until translated; a French entry without
a Spanish one fails. A new test checks the bundled fonts cover every
French letter (the glyph check is shared with Ukrainian). Docs: G14,
localization-parity, ROADMAP language row, ui-egui README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 00:22:08 +02:00
Brandon Thomas 9f6d75350c Merge pull request #795 from vangel76/gap-ec-transition-timeline
#577: Effect Controls shows a selected transition's own timeline
2026-10-10 18:15:32 -04:00
Brandon Thomas bf7ef0e894 Merge pull request #794 from impostazioni/fix/alt-drag-duplicate
Timeline: Option-drag copies clips instead of moving them
2026-10-10 18:15:25 -04:00
Brandon Thomas de6dcc8ebf Merge pull request #789 from vangel76/gap-optical-flow
#780: Optical Flow time interpolation synthesises motion-compensated frames
2026-10-10 18:09:14 -04:00
Brandon Thomas 257d7af4ab Merge pull request #791 from vangel76/gap-image-sequences
#779: export JPEG, Targa and DPX image sequences
2026-10-10 18:09:10 -04:00
vangel76andClaude Opus 5.5 0b62ebb9ad #577: Effect Controls shows a selected transition's own timeline
Problem: with a transition selected, Effect Controls showed only its
settings. Premiere puts the transition's own timeline beside them (clip
A, clip B and the transition over their cut), where it can be slid
over the cut or resized by dragging; FilmCraft could do that only in the
Timeline.

Fix: right of the settings (the clip view's divider and width), a time
ruler over the cut, clip A above, the transition between, clip B below,
the cut and the playhead. Dragging the transition's middle slides it,
dragging an end changes its duration from that end (the Timeline's
rules, transitions::drag_span), measured against the view the drag began
in; each drag is one undoable sequence.setTransition. The ruler moves the
playhead. Every part has an automation id; no new UI strings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 00:08:40 +02:00
Nigel Ihza Firdausy c7ad44bc19 test(menus): normalize regression test spacing 2026-10-11 04:55:59 +07:00
Brandon Thomas ab7318e788 Merge pull request #788 from vangel76/gap-eac3
#647: decode E-AC-3 (Dolby Digital Plus) audio
2026-10-10 17:52:57 -04:00
vangel76andClaude Opus 5.5 c673ee85b9 #779: export JPEG, Targa and DPX image sequences
Problem: Premiere offers JPEG, Targa and DPX as image-sequence export
formats; FilmCraft only wrote PNG, TIFF and BMP sequences (docs/gaps.md G13).

Fix: three new export formats (ids jpg, tga, dpx), numbered like the other
sequences, with presets, the export UI (JPEG quality slider, Targa alpha)
and es/ja/zh-cn strings. JPEG uses the image crate at the export quality;
Targa (TGA 2.0, uncompressed 24/32-bit) and DPX (SMPTE ST 268, 10-bit RGB
method A, BT.709) are written by a small hostile-input-safe writer in
crates/export/src/raster.rs. Format::is_image_sequence replaces the
hand-kept format lists. OpenEXR stays open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 23:51:41 +02:00
Nigel Ihza Firdausy 022e645a91 fix(playback): keep audio during forward shuttle speeds (#771) 2026-10-11 04:50:20 +07:00
vangel76andClaude Opus 5.5 e56ddd0e36 #780: Optical Flow time interpolation synthesises motion-compensated frames
Problem: a speed-changed clip set to Time Interpolation > Optical Flow
rendered exactly like Frame Blending, a cross-fade of the two neighbouring
source frames, so moving objects showed double images.

Fix: new filmcraft_render::flow. It computes dense coarse-to-fine
Lucas-Kanade flow both ways on a luma pyramid at most 640 px wide, drops
motion wherever standing still explains the window as well (flicker on
static content), forward-splats both flows to the in-between time
(Baker et al. 2011), and warps both frames there. Each fetch is weighted
by how well its own flow agrees, so uncovered background comes from the
frame that shows it. Pixels that look the same in both frames, or where
neither fetch agrees, fade to the plain cross-fade. media_image uses it
for OpticalFlow clips. docs/gaps.md G18 and target-app-parity updated
(also drops the #415 / #458 items, already fixed on main).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 23:50:15 +02:00
Brandon Thomas f346da650f Merge pull request #784 from vangel76/gap-render-in-out-range
#424: Render In to Out renders only the frames between In and Out
2026-10-10 17:49:34 -04:00
Brandon Thomas 8ae2c641a3 Merge pull request #782 from vangel76/gap-parity-menus
#781: cargo xtask parity measures menu coverage
2026-10-10 17:49:28 -04:00
Brandon Thomas 662d9d07b1 Merge pull request #783 from SilentYeti/nvenc-hevc-linux
Linux: H.265 export through NVENC
2026-10-10 17:49:22 -04:00
vangel76andClaude Opus 5.5 be98b9d7f7 #647: decode E-AC-3 (Dolby Digital Plus) audio
Problem: files with E-AC-3 audio (MP4 ec-3, MOV, Matroska A_EAC3, MPEG-TS)
imported silent: filmcraft-ac3 refused bsid 11-16.

Fix: clean-room E-AC-3 decoding from ATSC A/52:2012 Annex E in filmcraft-ac3:
bsi and audfrm syntax, 1/2/3/6-block syncframes, reduced sample rates,
frame-based exponent strategies (exponents reused across short syncframes),
SNR offset strategies, default band structures, spectral extension
(translation, notch filter, noise blending). Independent substream 0 is
decoded; dependent substreams and other programs are skipped (7.1 plays as
its 5.1 core). AHT and enhanced coupling are refused as unsupported. The
audio blocks share the AC-3 path; AC-3 output is unchanged. filmcraft-codecs
routes E-AC-3 in MP4/MOV, Matroska and TS to the decoder; the TS splitter
joins dependent substream frames to their access unit.

Tests: ffmpeg oracle for raw E-AC-3 (64.8-91.0 dB SNR) and in MP4, MOV,
Matroska and TS (79.5-91.3 dB, random access identical), mutation fuzz,
synthetic frames for headers, substream skipping, refusals and exponent
reuse across 1-block syncframes, spectral extension unit tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 23:49:03 +02:00
Nigel Ihza Firdausy 2890a42616 fix(menus): show current label colours in Edit menu (#786) 2026-10-11 04:48:16 +07:00
Brandon Thomas 5b0ef8755f Merge pull request #778 from MarekSpodBiedry/issue-770
Timeline: draw each audio track's own waveform, not track 1's (#770)
2026-10-10 17:44:05 -04:00
Brandon Thomas 85c1c6cacc Merge pull request #777 from MarekSpodBiedry/issue-769
text: the CJK system-font search no longer keeps every probed font file in memory (#769)
2026-10-10 17:44:00 -04:00
vangel76andClaude Opus 5.5 d676fdc720 #424: Render In to Out renders only the frames between In and Out
Problem: Render Effects In to Out and Render In to Out rendered every
render-bar segment that In/Out touched, in full. Segments run between
edit points, so In/Out inside one long clip rendered the whole clip
(often the whole timeline).

Fix: render only the marked frames of each segment as a partial preview,
`<hash>-<off>-<len>.mov`, frames counted from the segment start (so a
ripple keeps it, like whole `<hash>.mov` previews). The store indexes
parts per hash; the render bar is green over rendered frames only,
playback reads the part holding a frame, later renders fill only the
gaps, and Delete Render Files In to Out removes only files holding
marked frames.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 23:41:36 +02:00
SilentYetiandClaude Opus 5.5 c635d9af9e Linux: H.265 export through NVENC
The NVENC HEVC encoder (Main 8-bit SDR and Main 10 HDR PQ / HLG) was gated to
Windows. It now also runs on 64-bit Linux, on the CUDA device the H.264 encoder
already uses: the H.265 format and HDR probes are registered there and the
HEVC tests run on both systems.

NVENC registration on Linux now happens after the VA-API probe. Started beside
it, the background HEVC probe's first cuInit could deadlock with
libva-nvidia-driver, which calls cuInit from its library constructor under the
loader lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 14:41:04 -07:00
vangel76andClaude Opus 5.5 6aca7c1798 #781: cargo xtask parity measures menu coverage
Problem: the menu parity figure in docs/target-app-parity.md came from an
ad hoc label-matching script over string literals; nothing repeatable
measured the real menu bar (docs/gaps.md G7).

Fix: `cargo xtask parity` dumps the shipped menu bar (new
`menus::default_menu_items`, deterministic: English, default prefs,
built-in workspaces; printed by `examples/menu_tree.rs`) and compares it
with a reference list of menu paths, by default the local, never-committed
`plan/premiere/menus.json` (any JSON of paths via --reference; parsing is
depth- and size-capped). Presence and placement are reported separately
(same path, elsewhere, missing); app/Help/recent/account/macOS items are
out of scope. Writes a markdown checklist (target/parity/ or --out) and
--min fails below a presence threshold. Without a reference it prints our
inventory only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 23:39:57 +02:00
MarekSpodBiedry 8417e46090 Timeline: draw each audio track's own waveform, not track 1's (#770) 2026-10-10 23:22:52 +02:00
MarekSpodBiedry 62358d8c4e text: the CJK system-font search no longer keeps every probed font file in memory (#769) 2026-10-10 23:22:09 +02:00
Brandon Thomas ed141c74d0 Merge pull request #739 from mutombopr/av1-export-format-only
HW6: AV1 export format, enabled where a hardware encoder registers one
2026-10-10 17:15:49 -04:00
Brandon ThomasandClaude Opus 5.5 36cc163b44 Merge origin/main into #739: keep #600's CRF validation, and AV1 (hardware, like H.265) has no CRF mode either
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 17:15:40 -04:00
Brandon Thomas 843faef795 Merge pull request #625 from KattKnight/refresh-changed-media
Media: Automatically refresh growing files works (and catches re-renders)
2026-10-10 17:09:45 -04:00
Brandon ThomasandClaude Opus 5.5 fe47da6fdf Merge origin/main into #625 (keep both ja.tsv additions)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 17:09:36 -04:00
Brandon Thomas c4bb63bb6d Merge pull request #772 from alex-brazh/fix/open-gop-leading-pictures
codecs: an open GOP's leading pictures decode from the GOP before, not as black frames
2026-10-10 17:01:42 -04:00
Brandon Thomas 77eeb5be29 Merge pull request #767 from vangel76/fix-313-timeline-double-click
#313: double-clicking a Timeline clip opens it in the Source monitor
2026-10-10 17:01:36 -04:00
Brandon Thomas b92fdde25b Merge pull request #765 from ReReddo/feat/rect-corner-handles
program monitor: drag a rectangle's corners round with on-canvas corner handles
2026-10-10 17:01:31 -04:00
Brandon Thomas 726ba5cf5c Merge pull request #763 from vangel76/fix-313-source-clip-history
#313: the Source monitor remembers its clips; Close and Close All
2026-10-10 17:01:25 -04:00
Brandon Thomas 47fc0677c7 Merge pull request #762 from vangel76/fix-626-unsupported-video-slate
#626: undecodable video shows the MEDIA UNREADABLE slate, not black
2026-10-10 17:01:19 -04:00
Brandon Thomas 88376964c3 Merge pull request #761 from maccagnit/gpu-export-no-backend
GPU export: a CLI built on its own exports on the CPU instead of failing
2026-10-10 17:01:13 -04:00
Brandon Thomas 74963e0e7e Merge pull request #760 from maccagnit/gpu-export-cpu-frames
GPU export: a frame planned as one CPU image skips the renderer pool (Auto was slower than Off)
2026-10-10 17:01:07 -04:00
Brandon Thomas 65a04ea2c0 Merge pull request #759 from vangel76/fix-284-maximize-restore
#284: panel menu Maximize Frame can be restored; an empty dock reopens panels
2026-10-10 17:01:01 -04:00
Brandon Thomas c1c2f91cfc Merge pull request #758 from vangel76/fix-266-save-prompt-on-quit
#266: quitting with unsaved changes asks to save first
2026-10-10 17:00:55 -04:00
Brandon Thomas 35527a38ba Merge pull request #757 from vangel76/fix-228-export-frame-pending
#228: Export Frame reports a frame still loading instead of saving it blank
2026-10-10 17:00:50 -04:00
Brandon Thomas c6dbc390a4 Merge pull request #756 from vangel76/fix-374-transition-follows-trim
#374: a clip's fades follow its edges on a regular trim
2026-10-10 17:00:43 -04:00
impostazioniandClaude Opus 5.5 a8b3d7f703 Timeline: Option-drag copies clips instead of moving them
Holding Option (Alt) while dragging timeline clips now leaves the
originals in place and drops copies where the pointer lets go, as in
Premiere. The copies are selected; copies of linked clips are linked to
each other, and transitions that would travel with a move are copied.

- edit: copy_items (overwrite / insert, new ids, links and transitions)
- engine: timeline.move takes `copy: true`, returns `copied`
- ui: the drag remembers Option, shows the copy under the pointer while
  the originals stay, copy cursor
- tests: engine copy/link/locked-track test, headless Option-drag UI test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:52:28 +01:00
Brandon Thomas 7a36a6114a Merge pull request #610 from ivandrozdov1505-coder/l10n-ru
feat: Add Russian (ru) localization
2026-10-10 16:42:13 -04:00
Brandon ThomasandClaude Opus 5.5 5a94f761ff Merge origin/main into #610: Russian alongside Simplified Chinese and German; ROADMAP from main; rows for Next Appearance Mode and Project Notes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 16:42:04 -04:00
Brandon Thomas aba95af675 Merge pull request #630 from keller-lukas/i18n-german
i18n: add German (de) interface translation
2026-10-10 16:34:10 -04:00
Brandon ThomasandClaude Opus 5.5 398befbeec Merge origin/main into #630: ROADMAP from main (restructured), contributors, German row for the new Project Notes menu
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 16:34:01 -04:00
Brandon Thomas 0a2f8dd9ca Merge pull request #704 from ossini/fix/ripple-delete
#648, #668: select a gap in the Timeline and close it with Delete (Ripple Delete)
2026-10-10 16:27:58 -04:00
Brandon ThomasandClaude Opus 5.5 e4f3cab5a0 Merge origin/main into #704 (gap and marker context menus side by side; contributors)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 16:27:48 -04:00
Alex Brazh 45f2b8e297 codecs: an open GOP's leading pictures decode from the GOP before, not as black frames
In an open GOP (iPhone HEVC; x265 `open-gop`) each CRA after the first is
followed in the file by RASL pictures shown before it, predicted from the
GOP before. GopCache started a decoding run at `sync_before(i)`, the sync
sample at or before the wanted sample in decoding order: for a RASL
picture that is the CRA after it, and the decoders (software and
VideoToolbox) rightly leave a run's leading RASL pictures out. The frame
was never produced, so the request failed ("frame not produced") and the
clip's layer was left out: a black frame (or the track below) wherever a
seek landed on one of the 3-in-24 RASL frames of iPhone video, such as a
clip trimmed to start on one, the parked playhead, and parallel export.

- `decode_start`: a sample shown before its sync sample starts the run at
  the sync sample before (the MPEG and MXF readers already did this in
  their own `sync_before`; this covers MP4/MOV and Matroska too).
- A running decoder that started after that point is not continued for
  it: it left the picture out.

Tests: a synthetic open-GOP stream in gop.rs, and an x265 open-GOP oracle
test (MP4 and Matroska, every frame from a fresh source and each CRA
before its leading pictures) against ffmpeg. Both fail without the fix.
2026-10-10 23:14:29 +03:00
Brandon Thomas 73723151ce Merge pull request #682 from vangel76/fix-342-prores-4444-export
#342: ProRes 4444 and 4444 XQ export, with alpha
2026-10-10 16:14:22 -04:00
Brandon ThomasandClaude Opus 5.5 fad5255dc9 media: importable tests after the items (clippy items_after_test_module, #677 follow-up)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 16:08:58 -04:00
vangel76 2f49774895 Merge remote-tracking branch 'upstream/main' into fix-374-transition-follows-trim
# Conflicts:
#	crates/edit/src/tests.rs
2026-10-10 22:08:23 +02:00
Brandon Thomas b20ec56ded Merge pull request #677 from vbasky/fix/593-import-extension-case
Treat media extensions as case-insensitive on import
2026-10-10 16:07:13 -04:00
Brandon ThomasandClaude Opus 5.5 c16e712e5a Merge origin/main into #677 (keep both import tests)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 16:07:04 -04:00
Brandon Thomas 413b4449f4 Merge pull request #627 from JordyUSA/fix/374-waveform-trim
#374: trim preview keeps the waveform in place instead of squeezing it
2026-10-10 16:02:42 -04:00
Brandon ThomasandClaude Opus 5.5 e82f9bca71 Merge origin/main into #627 (keep both trim UI tests)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 16:02:33 -04:00
vangel76andClaude Opus 5.5 ad80801596 #313: double-clicking a Timeline clip opens it in the Source monitor
Problem: the Timeline only reacted to double-clicks on transitions and
nested sequences, so double-clicking an ordinary clip did nothing instead
of Premiere's opening it in the Source monitor.

Fix: `source.open` takes an optional `clip` (a clip of the active
sequence): its media opens in the Source monitor, parked on the frame
under the playhead when the playhead is over the clip, otherwise on the
clip's first frame. The Timeline's Selection-tool double-click on a clip
body (not its edge, not a nested sequence or graphic) dispatches it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 22:01:54 +02:00
Brandon Thomas bd60bf4e57 Merge pull request #636 from JordyUSA/fix/project-dblclick-import
Project panel: double-click anywhere empty in Icon view opens Import
2026-10-10 15:58:49 -04:00
Brandon ThomasandClaude Opus 5.5 97c6e5fb92 Merge origin/main into #636: #607's view background already opens Import on a double-click anywhere empty; keep this PR's regression test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 15:58:39 -04:00
vangel76andClaude Opus 5.5 3190eab4ef #313: the Source monitor remembers its clips; Close and Close All
Problem: the Source monitor only knew the one clip it showed. Opening
another clip forgot the previous one, there was no list to switch back
from, and no way to close a Source clip (Premiere keeps a list of the
recently opened clips with Close / Close All).

Fix: EditorState keeps `source_history` (most recent first, at most 20),
updated after every command; items removed from the project drop out.
New commands `source.close` (the next most recent clip opens in its
place) and `source.closeAll`. The Source panel menu lists the recent
clips (the shown one ticked) to switch between, then Close / Close All.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:55:11 +02:00
Brandon Thomas a6b323056f Merge pull request #755 from vangel76/fix-227-transitions-follow-move
#227: transitions travel with the clips they sit on when moved
2026-10-10 15:53:26 -04:00
Brandon Thomas e43392a86c Merge pull request #754 from vangel76/fix-483-drop-new-track
#483: media dropped past the last track goes on a new track
2026-10-10 15:53:19 -04:00
Brandon Thomas 6d6492491d Merge pull request #753 from vangel76/fix-227-drag-lumetri-presets
#227: Lumetri presets can be dragged onto a Timeline clip
2026-10-10 15:53:14 -04:00
Brandon Thomas 6239d3d65e Merge pull request #751 from vangel76/fix-456-nest-bins
#456: a bin can be dragged into another bin and back out
2026-10-10 15:53:08 -04:00
Brandon Thomas 9fb10ded25 Merge pull request #750 from vangel76/fix-620-project-notes
#620: Project Notes panel, saved in the project
2026-10-10 15:53:02 -04:00
Brandon Thomas a270f5cf34 Merge pull request #749 from vangel76/fix-456-new-bin-name
#456: a new bin asks for its name
2026-10-10 15:52:55 -04:00
Brandon Thomas 7c919da28e Merge pull request #747 from vangel76/fix-719-lumetri-vignette-roundness
#719: positive Roundness rounds the Lumetri vignette
2026-10-10 15:52:49 -04:00
Brandon Thomas 4768d1a582 Merge pull request #745 from vangel76/fix-715-mkv-timestamp-origin
#715: a Matroska remux that keeps its timestamps starts at its first picture
2026-10-10 15:52:43 -04:00
Brandon Thomas 49948d2c65 Merge pull request #743 from ReReddo/fix/highlight-color
appearance: the Highlight Color drives the playhead, timecodes and focus
2026-10-10 15:52:38 -04:00
Brandon Thomas 735d1acbd7 Merge pull request #741 from vangel76/fix-695-clear-clicked-marker
#695: Timeline markers can be picked and cleared
2026-10-10 15:52:33 -04:00
Brandon Thomas e59fb46f77 Merge pull request #740 from vangel76/fix-712-keyboard-slide-linked
#712: keyboard Slide keeps linked picture and sound aligned at a neighbour limit
2026-10-10 15:52:26 -04:00
Brandon Thomas a1e0051060 Merge pull request #738 from mutombopr/av1-seqheader-format
HW5: AV1 sequence headers compare by picture format, so SVT-AV1 files stay on hardware
2026-10-10 15:52:20 -04:00
Brandon Thomas 7f772fc1c6 Merge pull request #681 from ReReddo/feat/effect-controls-row-lines
effect controls: row lines, a draggable divider, scrolling anywhere, larger keyframes
2026-10-10 15:52:14 -04:00
Brandon Thomas 1039e4830c Merge pull request #671 from vangel76/fix-284-reset-keeps-keyframes
#284: reset buttons keep keyframes; a reset button on every property
2026-10-10 15:52:08 -04:00
Brandon Thomas 8747305aa0 Merge pull request #663 from vangel76/fix-653-clamp-trim-drag
#653: a dragged trim edge stops at the end of the media
2026-10-10 15:52:03 -04:00
Brandon Thomas 9a13e7c8f3 Merge pull request #657 from vangel76/fix-642-open-links
#642: open Help/About links in the system browser on the desktop
2026-10-10 15:51:56 -04:00
Brandon Thomas cee18f1006 Merge pull request #651 from vangel76/fix-591-explicit-clip-errors
#591: named clips that cannot stand in for a selection say why
2026-10-10 15:51:49 -04:00
Brandon Thomas 93c86f1e0a Merge pull request #735 from vangel76/fix-710-version-after-options
#710: --version works after global options
2026-10-10 15:51:44 -04:00
Brandon Thomas e66fa7faf4 Merge pull request #727 from Ni-zav/fix/prefs-timeline-focus-schema-713-s20261010T1747Z-c64d9
fix(settings): mark inactive Timeline focus preference as unwired
2026-10-10 15:51:37 -04:00
Oskar 24d555eb7c Merge upstream/main into fix/ripple-delete
Conflicts:
- contributors/people.toml: keep both new entries (maxm11, ossini)
- timeline.rs: take upstream's Alt-click through timeline.select (it
  clears the gap selection), and its empty-space arm (#683); only an
  empty stretch of a track selects a gap
2026-10-10 21:48:36 +02:00
ReReddo 8f1b7c3494 program monitor: drag a rectangle's corners round (#744)
A rectangle's rounded corners could only be set by typing a Corner
Radius in Properties. As in Premiere, a selected rectangle now shows
four small circles just inside its corners: dragging one towards the
middle rounds all four corners (Corner Radius is one value), dragging it
back out squares them, and the radius stops at half the shorter side,
where the renderer caps it too.

The circles sit on each corner's diagonal at the radius, at least 12
points in, and show only on rectangles at least four times that across
on screen. While dragging, a white outline of the rounded rectangle
previews the result; the radius is set on release with graphics.set, one
undo step per drag. Automation ids: program.layer.<clip>.<layer>.corner.<n>.

Tests: graphics_handles_ui (the circles round all four corners, clamp
and undo; ellipses and tiny rectangles have none).
2026-10-10 22:46:27 +03:00
vangel76andClaude Opus 5.5 ba5727029a #626: undecodable video shows the MEDIA UNREADABLE slate, not black
Problem: a file that opens but whose picture FilmCraft cannot decode
(H.264 High 10 / 4:2:2 / 4:4:4, interlaced, a codec without a decoder)
rendered black with no sign of why. The H.264 and HEVC decoders'
"unsupported" errors reached the media source as generic decode errors,
and the renderer drops frames that fail.

Fix: the H.264 / HEVC wrappers report unsupported features as
CodecError::Unsupported. The media pool wraps every file source it opens
so a frame failing as unsupported renders the existing MEDIA UNREADABLE
slate and the item is marked Unreadable with the decoder's reason (as
for files that fail to open). Decodable frames, damaged frames and the
sound are untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:46:06 +02:00
Tommaso MaccagniandClaude Opus 5.5 d215d0e8a3 gpu: a build without a wgpu backend exports on the CPU instead of failing
filmcraft-gpu asks wgpu for no backend (`default-features = false`); a build
that includes the app gets them through eframe. A CLI built on its own
(`cargo run --release -p filmcraft-cli`, as the README and
docs/contributing.md do) has none, and `wgpu::Instance::default()` panics
there: an export with GPU rendering on ended as "internal error (see the
crash log); the job was stopped".

ExportRenderer::new returns None in such a build, like a host without an
adapter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:40:37 +02:00
Tommaso MaccagniandClaude Opus 5.5 13d8e73e8e export: a frame planned as one CPU image does not take a pooled GPU renderer
With GPU rendering on, a frame the planner hands back whole (an adjustment
layer, a complex transition, an HDR / wide-gamut sequence) was still rendered
by render_sequence, but inside RendererPool::with: as many frames at a time as
there are pooled renderers (2 to 4) instead of one per export thread. A
timeline graded on an adjustment layer exported slower with Auto than with Off.

Pipeline::frame asks the planner first (plan::is_cpu_frame, the test
plan_frame itself makes) and renders such a frame on the CPU directly. It
counts as a fallback now, not as a GPU frame. The picture does not change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:40:37 +02:00
vangel76andClaude Opus 5.5 1ff2b9fc89 #284: panel menu Maximize Frame can be restored; an empty dock reopens panels
Problem: the panel menu's Maximize Frame replaced the whole dock tree with a
one-panel group, so there was no way back short of resetting the workspace,
and closing that panel left an empty dock. DockNode::open_near only inserted
beside the requested neighbour, Project or Program, so with those gone (or
no panel left) every Window > panel command was a silent no-op.

Fix: the panel menu now toggles the same Maximize/Restore Frame state as
` / Shift+` (labelled Restore Frame while maximized) and keeps the tree;
a maximized panel that was closed counts as restored, and switching
workspace clears it. open_near falls back to the first group, even an
empty one, as its doc comment promised.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:37:09 +02:00
vangel76andClaude Opus 5.5 1580ebb520 #266: quitting with unsaved changes asks to save first
Problem: Quit, Cmd+Q or the window's close button exited at once with
unsaved changes, leaving them to the recovery journal, so every launch
after an unsaved session offered a recovery (docs/project-files.md: no
"Save changes?" prompt on quit yet).

Fix: a close request with unsaved changes is cancelled and the existing
Save Project prompt asks Cancel / Don't Save / Save (automation ids
quit.*), as in Premiere. Don't Save discards the changes and their
recovery snapshot; Save on a never-saved project (here and in Close
Project) opens the Save dialog and returns to the prompt when cancelled.
The control channel's app.quit still quits without asking.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:33:45 +02:00
vangel76andClaude Opus 5.5 7f59211912 #228: Export Frame reports a frame still loading instead of saving it blank
Problem: file.exportFrame rendered and wrote the still without looking at
filmcraft_media::pending. On the web, a frame WebCodecs is still decoding
(or whose Blob bytes are still loading) is left out of the render, so the
command succeeded with a transparent/blank picture passed off as the
requested frame.

Fix: clear the pending flag before rendering and check it afterwards; when
a read was deferred, fail with "the frame is still loading; export it
again in a moment" and write nothing. A later call exports the real frame.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:31:27 +02:00
vangel76andClaude Opus 5.5 2cab09e69b #374: a clip's fades follow its edges on a regular trim
Problem: dragging a clip's edge with the Selection tool (a regular trim)
moved the edge but left a fade in from nothing / fade out to nothing where
it was; only Ripple trims moved transitions along with the cut.

Fix: after a regular trim, the trimmed clip's one-sided transitions are put
back on its edges, and shortened when the clip gets shorter than them.
Transitions between two clips are untouched (a regular trim cannot move
their cut).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:29:55 +02:00
vangel76 95a9a7770a Merge upstream/main into fix-342-prores-4444-export
# Conflicts:
#	crates/engine/src/commands.rs
2026-10-10 21:27:52 +02:00
vangel76andClaude Opus 5.5 b86dcf7bb3 #227: transitions travel with the clips they sit on when moved
Problem: moving a clip dropped the transitions at its edges (#227, point 8
of #374). `edit::move_items` deletes the moved clips before placing them
again, and that deletion removes every transition that refers to them, so
a fade on a clip's edge, or the cut between two clips moved together,
vanished.

Fix: before the move, collect the transitions whose clips all move onto one
track by one offset, and put them back on that track shifted by the offset
once the clips have landed (overwrite and insert moves). A crossfade whose
partner stays behind is still dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:24:52 +02:00
Brandon Thomas 3665378384 Revert "Merge pull request #689 from Ni-zav/fix/654-track-header-rename-s20261010T1632Z-b8e4"
This reverts commit 826858dfec, reversing
changes made to 0a9c133b4e.
2026-10-10 15:24:48 -04:00
Brandon Thomas e78223db6d Revert "Merge pull request #662 from MrCloudy2/fix/export-hevc-stall-480"
This reverts commit aa8ad402cc, reversing
changes made to 5f9b5e1234.
2026-10-10 15:24:44 -04:00
Brandon Thomas b43e3828a9 Revert "Merge pull request #685 from vbasky/fix/linked-clip-delete"
This reverts commit 6145dcc568, reversing
changes made to f34a00c020.
2026-10-10 15:24:43 -04:00
Brandon ThomasandClaude Opus 5.5 4cdb15466e Captions: a TTML frame count that saturates the clock is rejected, not read as the end of time (#705 follow-up)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 15:24:43 -04:00
vangel76 c583157088 Merge upstream/main into fix-284-reset-keeps-keyframes
# Conflicts:
#	crates/engine/src/clip_ops_tests.rs
2026-10-10 21:24:34 +02:00
Brandon Thomas 334bccde88 Revert "Merge pull request #674 from vbasky/fix/542-limiter-lookahead"
This reverts commit 62a0eea519, reversing
changes made to 23ac344798.
2026-10-10 15:23:52 -04:00
vangel76andClaude Opus 5.5 cca7d5ef0e #483: media dropped past the last track goes on a new track
Problem: dropping a project item in the empty space above the top video
track or below the last audio track did nothing: the Timeline only
accepted drops on an existing track row, and timeline.place could only
target existing tracks.

Fix: timeline.place accepts "new" as `track` / `audioTrack`. It adds a
track after the last one of its kind for each stream the item carries,
then places the clip there, as one undo step (the new tracks are removed
again if the placement fails). The Timeline shows the drop ghost in the
empty space and places on a new video track (above) or audio track
(below); the other stream goes on the track of the same number, or on a
new one when that track does not exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:23:42 +02:00
vangel76andClaude Opus 5.5 78a3fea222 #227: Lumetri presets can be dragged onto a Timeline clip
Problem: in the Effects panel, the Lumetri Presets rows and the wide-panel
thumbnail grid only sensed clicks. They applied through double-click or the
Apply menu, never through a drag, unlike every other effect and preset.

Fix: rows and grid cells sense drags and start an effect payload
`lumetri:<name>`. The Timeline's effect drop applies it to the clip under the
pointer with `lumetri.applyPreset` (one undo step). The drag ghost shows the
preset's name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:21:26 +02:00
vangel76 ae488523a9 Merge remote-tracking branch 'upstream/main' into fix-620-project-notes
# Conflicts:
#	crates/ui-egui/src/i18n/es.tsv
#	crates/ui-egui/src/i18n/ja.tsv
#	crates/ui-egui/src/i18n/zh-cn.tsv
2026-10-10 21:15:04 +02:00
vangel76andClaude Opus 5.5 6f650a363a #456: a bin can be dragged into another bin and back out
Problem: a bin stayed wherever it was created. `project.moveToBin` only
moved project items, and bin rows and cards in the Project panel could not
be dragged, so the only way to nest bins was to create them inside each
other.

Fix: `project.moveToBin` also takes bin ids in `items` (as `project.delete`
does) and moves the bin with everything in it, skipping the root and any
move that would put a bin inside itself. In the Project panel, bin rows
(List view) and bin cards (Icon view) can be dragged: dropping one on
another bin nests it there, and dropping it on empty space moves it into
the bin the view shows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:13:42 +02:00
vangel76 6067462b88 Merge upstream/main into fix-591-explicit-clip-errors 2026-10-10 21:13:24 +02:00
vangel76 22de920fd6 Merge remote-tracking branch 'upstream/main' into fix-710-version-after-options
# Conflicts:
#	apps/filmcraft-cli/src/args.rs
2026-10-10 21:12:17 +02:00
vangel76andClaude Opus 5.5 110c1783c3 #620: Project Notes panel, saved in the project
Problem: a project had nowhere to keep free text (to-dos, instructions
for whoever opens it next, frame numbers).

Fix: plain-text `Project::notes` (schema v15, no-op migration so older
builds refuse instead of dropping it), `project.notes` /
`project.setNotes {text, merge?}` engine commands (undoable, capped at
1 MiB; calls with one `merge` key share an undo step), and a
Window > Project Notes panel whose editor (`notes.text`) commits every
change, one typing session per undo step. Translations for es, ja,
zh-cn (and the panel title for pt-br, uk).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:09:13 +02:00
Brandon ThomasandClaude Opus 5.5 2140d9c9ab Restore green after the community batch: rustfmt, clippy as_chunks_mut in webm_alpha, zh-cn caption-track and es shutter-angle rows
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 15:07:59 -04:00
Brandon Thomas e214d84e84 Revert "Merge pull request #589 from Ni-zav/fix/547-japanese-font-priority-s20261010T084251Z-a6c2d9"
This reverts commit b1f2973c9d, reversing
changes made to 8fa24de32b.
2026-10-10 15:05:04 -04:00
vangel76 2a1c92e15a Merge remote-tracking branch 'upstream/main' into fix-653-clamp-trim-drag
# Conflicts:
#	crates/ui-egui/tests/timeline_trim_ui.rs
2026-10-10 21:02:44 +02:00
ReReddo ade56c6277 effect controls: draggable divider, scroll the whole panel (#643)
The split between the effect list and the keyframe area was fixed. Now a
divider between them drags left or right, as in Premiere: it shows a
resize cursor and turns blue under the pointer, and the width it sets
stays while FilmCraft runs (UiState::effect_controls_split, automation id
effectControls.divider). The effect list keeps the 260 points its rows
are laid out for (values from 150 points in, the reset button at the
end), the keyframe area at least 60.

Scrolling was tied to the effect list. The scroll area now spans the list
and the keyframe area, so the mouse wheel scrolls anywhere over the panel
and the scrollbar sits at its right edge. The ruler's click and drag are
handled after the rows, which now reach under it, so they still move the
playhead.

At narrow widths labels no longer run into what follows them: property,
effect and mask names are cut off with "…" (effect_controls::row_label),
and a dropdown narrows to the space left in its row.

Smaller readability fixes in the same panel:

- Keyframes in the lane are drawn at about Premiere's size (a diamond ~9
  points wide instead of ~7; Hold squares and Bezier circles to match),
  with a grab area to match.
- The keyframe navigator's arrows are taller (11 points, clickable over 20
  of the row's 22) and its diamond larger; Properties uses the same
  navigator, so it gets them too.
- Checkboxes (Bypass, Uniform Scale, a mask's Inverted) sit centred between
  their row lines (effect_controls::fit_to_row, which also fits dropdowns).

Tests: keyframes_ui effect_controls_divider_and_scrolling_anywhere (the
divider drags, keeps its width and both minimum widths; the wheel scrolls
the rows over the keyframe area).
2026-10-10 22:00:54 +03:00
ReReddo 5d6f690e36 effect controls: lines between property rows (#640)
Effect Controls now draws a thin line under every row, as Premiere does,
across the names, the values and the keyframe lane, so a row can be
followed from its name to its value and keyframes. Each line runs through
the middle of the spacing between rows, so a row sits centred between its
two lines. It covers every kind of row: effect headers, properties, Custom
Setup, the value and velocity graphs, the mask rows, and a selected
transition's settings. The colour is the theme's separator, so it follows
the dark and light themes.

- effect_controls::row_line draws the line; effect_controls::
  fit_dropdown_to_row keeps a dropdown (Blend Mode, a mask's mode) inside
  its row, clear of the lines.
- The keyframe lane's ruler and clip bar are now painted after the rows, on
  their own background, so rows scrolled up under them no longer draw over
  them (their lines, and keyframe diamonds, which could already do so).
2026-10-10 22:00:37 +03:00
vangel76andClaude Opus 5.5 ddb3890e68 #456: a new bin asks for its name
Problem: every New Bin entry (Cmd+B, File > New > Bin, the Project panel
menu, the footer button and the context menus) ran `file.newBin` with the
fixed name "New Bin", so each bin had to be renamed by hand afterwards.

Fix: `file.newBin` without a `name` from the UI makes the bin in the shown
bin (or the given parent), selects it and opens its inline rename field;
agents that pass `name` still name it directly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 20:59:02 +02:00
vangel76 15bf7f9f48 Merge remote-tracking branch 'upstream/main' into fix-642-open-links
# Conflicts:
#	apps/filmcraft/src/main.rs
2026-10-10 20:57:56 +02:00
Brandon Thomas 794a25fc9e Merge pull request #658 from vbasky/fix/current-bugs
Fix silent WAV sample-rate substitution
2026-10-10 14:57:13 -04:00
Brandon Thomas ef52c3834f Merge pull request #737 from ReReddo/fix/shape-handles-size
graphics: shape handles change the shape's Size, Shift keeps the ratio
2026-10-10 14:57:06 -04:00
Brandon Thomas 8d737a4af2 Merge pull request #733 from vangel76/fix-721-aac-surround-order
#721: 5.1 AAC imports decode in WAV/SMPTE channel order
2026-10-10 14:57:00 -04:00
Brandon Thomas b04da9c426 Merge pull request #732 from vangel76/fix-714-mp4-no-edit-origin
#714: an MP4 without an edit list starts at its first picture
2026-10-10 14:56:54 -04:00
Brandon Thomas ba6668e942 Merge pull request #730 from vangel76/fix-709-gpu-rendering-value
#709: --gpu-rendering takes a space-separated value
2026-10-10 14:56:46 -04:00
Brandon Thomas 9686d1c49f Merge pull request #726 from Ni-zav/fix/edl-audio-transition-names-711-s20261010T1747Z-c64d9
fix(edl): retain named audio crossfades when importing EDL
2026-10-10 14:56:39 -04:00
Brandon Thomas c64df0171d Merge pull request #725 from Ni-zav/fix/tts-text-number-normalization-s20261010T1747Z-c64d9
fix(tts-text): preserve amounts and numeric units in narration
2026-10-10 14:56:33 -04:00
Brandon Thomas a1646560e6 Merge pull request #723 from impostazioni/fix/delete-caption-tracks
Captions: delete caption tracks from the timeline and Delete Tracks dialog
2026-10-10 14:56:28 -04:00
Brandon Thomas c6eea54dfc Merge pull request #722 from ReReddo/fix/settings-hex-field
settings: the hex colour field takes typing
2026-10-10 14:56:22 -04:00
Brandon Thomas c11f20cf9c Merge pull request #708 from impostazioni/fix/caption-font
Captions: render the track font and add a font picker
2026-10-10 14:56:17 -04:00
KattandClaude Opus 5.5 46b391957c Merge main: keep both new Japanese rows
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 00:26:12 +05:30
Brandon Thomas e7026e1b4c Merge pull request #706 from Ni-zav/fix/stl-literal-angle-text-s20261010T1655Z-d7c2
fix(captions): preserve literal angle brackets in EBU STL export
2026-10-10 14:56:11 -04:00
Brandon Thomas 2ed0295cce Merge pull request #705 from Ni-zav/fix/ttml-time-expression-overflow-s20261010T1650Z-d7c2
fix(captions): reject overflowing TTML times and round extreme clocks safely
2026-10-10 14:56:06 -04:00
Brandon Thomas ee1118bc21 Merge pull request #703 from ReReddo/fix/timeline-track-lines
timeline: lines between the tracks, a short playhead head
2026-10-10 14:56:01 -04:00
Brandon Thomas efb58176a4 Merge pull request #702 from vangel76/fix-411-folder-bins
#411: importing a folder keeps its structure as bins
2026-10-10 14:55:55 -04:00
Brandon Thomas 104355ae13 Merge pull request #701 from vangel76/fix-313-source-shuttle
#313: J/K/L shuttle the Source monitor both ways at up to 8x
2026-10-10 14:55:50 -04:00
Brandon Thomas 425c61a1ad Merge pull request #699 from ReReddo/fix/deselect-empty-space
timeline, text editing, program monitor: clicking empty space deselects
2026-10-10 14:55:44 -04:00
Brandon Thomas 33115e9a8a Merge pull request #698 from vangel76/fix-614-continuous-thumbnails
#614: Head, Head and Tail and Continuous video thumbnails in the Timeline
2026-10-10 14:55:39 -04:00
Brandon Thomas 23d1a37da2 Merge pull request #693 from vangel76/fix-67-file-reader-pool
#67: media files share a bounded pool of open descriptors
2026-10-10 14:55:33 -04:00
Brandon Thomas 12e714bf53 Merge pull request #692 from vangel76/fix-659-macos-microphone
#659: macOS app may use the microphone for voice-over
2026-10-10 14:55:26 -04:00
Brandon Thomas 0cf7a51ada Merge pull request #691 from vangel76/fix-279-streaming-wav
#279: WAV opens in place and reads only the samples asked for
2026-10-10 14:55:20 -04:00
Brandon Thomas b95a7f4eb5 Merge pull request #690 from vangel76/fix-680-shift-delete-ripple
#680: Shift+Delete ripple deletes with the Timeline focused
2026-10-10 14:55:15 -04:00
Brandon Thomas 826858dfec Merge pull request #689 from Ni-zav/fix/654-track-header-rename-s20261010T1632Z-b8e4
feat(timeline): rename video and audio tracks from their headers
2026-10-10 14:55:08 -04:00
Brandon Thomas 0a9c133b4e Merge pull request #688 from Ni-zav/fix/431-monitor-button-editor-s20261010T1632Z-b8e4
fix(monitor): make Button Editor customize transport controls
2026-10-10 14:55:03 -04:00
Brandon Thomas 7d424eb78f Merge pull request #686 from vbasky/fix/link-media-layout
Keep Link Media actions accessible for large lists and long paths
2026-10-10 14:54:57 -04:00
Brandon Thomas 6145dcc568 Merge pull request #685 from vbasky/fix/linked-clip-delete
Preserve individual linked-clip selections when deleting media
2026-10-10 14:54:50 -04:00
Brandon Thomas f34a00c020 Merge pull request #684 from vangel76/fix-239-position-graph
#239: Position keyframes get the value and velocity graphs
2026-10-10 14:54:45 -04:00
Brandon Thomas ea33e3cb00 Merge pull request #679 from vbasky/fix/415-transform-shutter
Apply Transform shutter angle when the override is on
2026-10-10 14:54:38 -04:00
Brandon Thomas b07df164fa Merge pull request #676 from vbasky/fix/592-import-replace
Add file.import replace mode for empty-project round-trips
2026-10-10 14:54:33 -04:00
Brandon Thomas db352fc963 Merge pull request #675 from vbasky/fix/458-ultra-key-setting
Apply Ultra Key Setting as a parameter preset
2026-10-10 14:54:27 -04:00
Brandon Thomas 62a0eea519 Merge pull request #674 from vbasky/fix/542-limiter-lookahead
Pass Hard Limiter look-ahead into the DSP limiter
2026-10-10 14:54:21 -04:00
Brandon Thomas 23ac344798 Merge pull request #673 from abdelkabirouadoukou/fix-monterey-constant-bitrate
platform: look up kVTCompressionPropertyKey_ConstantBitRate at run time
2026-10-10 14:54:16 -04:00
Brandon Thomas 66a7239115 Merge pull request #672 from vangel76/fix-457-ui-scale
#457: Settings > Appearance > UI Scale
2026-10-10 14:54:09 -04:00
Brandon Thomas b8cb56f815 Merge pull request #665 from vangel76/fix-333-fade-presets-merge
#333: Fade In and Fade Out presets add up on one clip
2026-10-10 14:54:04 -04:00
Brandon Thomas 087877fd72 Merge pull request #664 from vangel76/fix-461-interchange-stream-media
#461: interchange import links media through the streaming reader
2026-10-10 14:53:58 -04:00
Brandon Thomas aa8ad402cc Merge pull request #662 from MrCloudy2/fix/export-hevc-stall-480
#480: exports no longer stop for good when a clip's decoding uses rayon (rotated iPhone HEVC clips)
2026-10-10 14:53:51 -04:00
Brandon Thomas 5f9b5e1234 Merge pull request #656 from vangel76/fix-412-single-keyframe-select
#412: select a single keyframe in Effect Controls
2026-10-10 14:53:45 -04:00
Brandon Thomas 66a4208a6b Merge pull request #652 from vangel76/fix-456-project-select-all
#456: Select All in the Project panel takes only the shown bin
2026-10-10 14:53:40 -04:00
Brandon Thomas 6e3d5e7c77 Merge pull request #646 from maxm11/issue-536-bin-lookup
perf(export): index the bin tree once in closure() (#536 follow-up)
2026-10-10 14:53:34 -04:00
Brandon Thomas 1a8bdb43e4 Merge pull request #644 from maxm11/issue-460-rtmd-timecode
MP4: read the start timecode from a Sony rtmd track when there is no tmcd
2026-10-10 14:53:26 -04:00
vangel76andClaude Opus 5.5 dfbcadb88c #719: positive Roundness rounds the Lumetri vignette
Problem: the Lumetri vignette scaled its horizontal coordinate by
`1 + vround * 0.0` and applied aspect correction only for negative
Roundness, so 0 … +100 rendered identically (CPU path, the GPU op and the
GPU shader parameters alike). Negative values moved towards a circle,
the direction positive values should go.

Fix: one helper, `gpufx::lumetri_vignette_aspect`, scales the horizontal
coordinate by aspect^roundness: 0 keeps the ellipse that follows the
frame, +100 is a circle, negative values widen the oval. Non-finite or
out-of-range values fall back safely. The CPU path, the CPU FxOp and the
GPU parameter packing all use it, so they stay in parity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 20:53:19 +02:00
Brandon Thomas 70d0e22920 Merge pull request #633 from 4papuce/fix/gpu-software-rasterizer
ui: reject software rasterizers for GPU compositing and say so
2026-10-10 14:53:19 -04:00
Brandon Thomas 9842fc0c15 Merge pull request #624 from MarekSpodBiedry/issue-378
Space out browser downloads so image-sequence exports are not dropped (#378)
2026-10-10 14:53:13 -04:00
Brandon Thomas 6f366f7666 Merge pull request #622 from MarekSpodBiedry/issue-587
Let a bin be cleared from its context menu, the trash button, the panel menu and Delete (#587)
2026-10-10 14:53:08 -04:00
Brandon Thomas 1879a30db8 Merge pull request #615 from ReReddo/feat/clipboard-paste
timeline, project panel: paste images and copied files from the system clipboard
2026-10-10 14:53:02 -04:00
Brandon Thomas df5be6ad78 Merge pull request #607 from ReReddo/feat/project-marquee-select
project panel: drag in empty space to marquee-select, in List, Icon and Freeform views
2026-10-10 14:52:56 -04:00
Brandon Thomas 97c39725cb Merge pull request #606 from JordyUSA/fix/pcm-large-files
Export: never crash on WAV/AIFF over 4 GiB; stream PCM to disk
2026-10-10 14:52:50 -04:00
Brandon Thomas 159c654d54 Merge pull request #600 from JordyUSA/feat/export-crf
Export: CRF (constant quality) for H.264
2026-10-10 14:52:44 -04:00
Brandon Thomas 9504266326 Merge pull request #599 from gyroninja/fix/trim-monitor-drag-deadlock
Trim Monitor: Make trim drags no longer deadlock
2026-10-10 14:52:38 -04:00
Brandon Thomas e6372efce4 Merge pull request #597 from liangforstudy/feat/402-webm-alpha
#402: WebM transparency (VP9 alpha in BlockAdditions)
2026-10-10 14:52:32 -04:00
Brandon Thomas 525c3b2845 Merge pull request #596 from MarekSpodBiedry/issue-481
Audio Clip Mixer: click the dB readout to type a volume (#481)
2026-10-10 14:52:25 -04:00
Brandon Thomas 539759ef18 Merge pull request #595 from MarekSpodBiedry/issue-476
Reject unknown mcp options instead of silently ignoring them (#476)
2026-10-10 14:52:19 -04:00
Brandon Thomas e0fc43f267 Merge pull request #594 from MarekSpodBiedry/issue-561
Keyboard slip: clamp linked clips by one shared delta (#529)
2026-10-10 14:52:12 -04:00
Brandon Thomas b1f2973c9d Merge pull request #589 from Ni-zav/fix/547-japanese-font-priority-s20261010T084251Z-a6c2d9
fix(i18n): prefer Japanese system glyphs over Chinese fallback (#547)
2026-10-10 14:52:06 -04:00
Brandon Thomas 8fa24de32b Merge pull request #585 from haranishi/fix/cfb-bound-chain-walks
CFB: bound the DIFAT and FAT chain walks by the file's own size
2026-10-10 14:52:00 -04:00
Brandon Thomas 8b56e0fec6 Merge pull request #584 from haranishi/fix/aaf-zero-size-weak-index
Hardening: an AAF weak-reference collection with a zero key size can't exhaust memory
2026-10-10 14:51:54 -04:00
vangel76andClaude Opus 5.5 57cacfefa8 #715: a Matroska remux that keeps its timestamps starts at its first picture
Problem: MkvSource used the raw Segment timestamps as media time and the
absolute Info/Duration endpoint as the clip length. A remux that keeps its
source's first PTS (ffmpeg -copyts, 1.48 s in the report) repeated its
opening picture until then and made a 3.48 s clip out of 2 s of media.

Fix: after opening, every track moves back by the earliest start among the
playable tracks (an audio track's start before CodecDelay, never below
zero), so a delay between picture and sound survives. The duration is the
Segment end minus that origin, never less than the main track's samples.
Zero-origin files are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 20:48:37 +02:00
vangel76andClaude Opus 5.5 c13f2aeddb #695: Timeline markers can be picked and cleared
Problem: Markers > Clear Selected Marker removes only a marker exactly at
the playhead, but a click on a marker on the Timeline ruler left the
playhead on the frame under the pointer (with snapping off), and
right-clicking a marker opened the clip menu, so a marker placed on the
Timeline could not be removed from there.

Fix: a press on a marker's pentagon puts the playhead exactly on it, and
right-clicking a marker opens a marker menu (Clear Selected Marker, Clear
Markers; automation ids timeline.markerMenu.<command>). The pentagon's
rect is shared by drawing and hit-testing (timeline_hit::marker_rect).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 20:41:12 +02:00
ReReddo 1edd8d6e33 appearance: the Highlight Color drives the playhead, timecodes and focus (#707)
Settings > Appearance > Highlight Color replaced only the accent
(selections, primary buttons). The playhead, the timecodes, the focused
panel's outline, drag values and the track-targeting buttons stayed blue,
so a pink highlight left an interface that was mostly blue.

A highlight of one's own now colours those too: each keeps the
highlight's hue and is as much more or less saturated and bright as the
theme's own blue is than the theme's highlight (the playhead and
timecodes a little lighter, the targeting buttons a little deeper). With
the default highlight nothing is derived, so every theme looks exactly as
before. The track-targeting fill, written into the Timeline code, is a
theme token now (track_target).

Tests: theme (the default highlight leaves the five colours as designed
in every theme; a pink one gives them its hue, lighter and darker as the
default blues are).
2026-10-10 21:40:36 +03:00
vangel76andClaude Opus 5.5 ce6afffc6f #712: keyboard Slide keeps linked picture and sound aligned at a neighbour limit
Problem: timeline.slideLeft/Right(5) slid every expanded selection member by
the raw requested delta, so when a neighbour limited only one linked partner
(e.g. a short sound-only follower), picture and sound ended up at different
starts. timeline.slide already clamped to a common delta.

Fix: add edit::slide_items, which clamps the delta by the most restrictive
item and slides all items by it, and use it from both the keyboard Slide
commands and timeline.slide.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 20:39:48 +02:00
ReReddo fdb0ce944d graphics: shape handles change the shape's Size, Shift keeps the ratio
Dragging a handle of a rectangle, ellipse or polygon in the Program
monitor changed the layer's Scale, and a corner scaled width and height
alike, so a shape could not be made wider or taller from a corner.

As in Premiere, the handles now change the shape's Size (Premiere's
Path) and leave Scale alone: the dragged side or corner follows the
pointer, the opposite one stays, and the anchor point is renumbered so
that it and Position stay on the same spot of the picture. A corner
moves freely; Shift keeps the proportions. A path has no Size, so it
still stretches (and its corners follow the same Shift rule).

The paragraph-text box and the shape box share the side-dragging code
(drag_sides).

Tests: graphics_handles_ui (a shape's handles change its Size and Shift
keeps the ratio; a polygon's handles keep the opposite corner).
2026-10-10 21:29:53 +03:00
vangel76andClaude Opus 5.5 bc5965e42b #710: --version works after global options
Problem: `filmcraft-cli --demo --version` (or `--compact -V`, `--data-dir d
--version`) failed with "missing subcommand" and exit 2. The version check only
looked at the very first argument, so a preceding global option left
`--version` to the option parser and no subcommand behind.

Fix: parse the arguments first and treat `--version` anywhere, or `-V` in the
subcommand position, as a version query (`Args::wants_version`), still before
the hardware decoders are registered. `--help` keeps priority.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 20:22:46 +02:00
vangel76andClaude Opus 5.5 d0617c849b #721: 5.1 AAC imports decode in WAV/SMPTE channel order
Problem: our AAC decoder outputs channels in AAC element order (C, L, R,
Ls, Rs, LFE for channel configuration 6) and PacketDecoder passed that
through unchanged, while the mixer, panner, loudness and export all use
WAV/SMPTE order (L, R, C, LFE, Ls, Rs). A 5.1 AAC import therefore put
centre in L, L in R, R in C, the surrounds one slot early and LFE in Rs.
The export path already reorders for the encoder; the decode path did not.

Fix: PacketDecoder reorders AAC output for channel configurations 3-6
into WAV/SMPTE order, like the AC-3 decoder and Opus mapping family 1
already do. The Opus reorder now shares the same bounds-checked helper.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 20:20:59 +02:00
vangel76andClaude Opus 5.5 14f04c8c90 #714: an MP4 without an edit list starts at its first picture
Problem: a video track with no edit list (x264 MP4 written without
elst, fragmented MP4) kept its raw first PTS, the B-frame composition
delay, as its origin. Media time zero repeated the first picture for
that delay, and since the clip lasts the media duration, the last
pictures never appeared in a sequence made from it.

Fix: when the video track has no edit list, move every track back by
the earliest start among the video and audio tracks, as ffmpeg does
with an input's start time. Video-only files now start at their first
picture; a file whose audio starts at zero keeps its A/V sync, and
explicit edit lists (empty edits, delays) are untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 20:19:17 +02:00
KattandClaude Opus 5.5 121ae6958f Merge main: ROADMAP.md moved to the new docs layout
Our line goes where upstream now keeps it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 23:48:01 +05:30
vangel76andClaude Opus 5.5 62928e9db2 #709: --gpu-rendering takes a space-separated value
Problem: `--gpu-rendering` was missing from the CLI's list of
value-taking options, so `export --gpu-rendering off out.wav` stored the
option without a value and turned `off` into the output path (or dropped
the setting silently when the option came after the filename).

Fix: list `--gpu-rendering` among the valued options, as documented in
the CLI help.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 20:11:00 +02:00
Brandon ThomasandClaude Opus 5.5 3ad5a59321 Progress docs: target-app parity, gaps, alpha gate, readiness by audience (#700)
Progress docs per craftrules standards/progress-docs.md: target-app parity, gaps, roadmap with alpha gate, readiness by audience (full / mainstream / essentials) with Opus 5.5 hours, and feature, format, hardware, UI and localization parity docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 14:09:14 -04:00
Nigel Ihza Firdausy 30417a345d settings: reflect unwired Timeline focus preference in schema 2026-10-11 00:56:04 +07:00
impostazioniandClaude Opus 5.5 6d6ed3f33a Captions: delete caption tracks from the timeline and Delete Tracks dialog
captions.deleteTrack existed but nothing in the UI reached it, so extra
caption tracks could not be removed and piled up in the timeline.

- Caption track header right-click menu: Delete Caption Track, and
  Delete Empty Caption Tracks (disabled when none is empty). Automation
  ids `timeline.captionTrack.C<n>.menu.delete` / `.menu.deleteEmpty`.
- sequence.deleteTracks takes `"captions": "empty" | "C2" | id`. Unlike
  video and audio, every caption track may go (a sequence can have
  none). Unknown names/ids and other value types are errors. The
  selection of captions on deleted tracks is cleared; one undo step.
- Sequence > Delete Tracks... gets a Caption Tracks section (shown when
  the sequence has caption tracks), ids `deleteTracks.captions*`.
- Spanish strings for the new labels.

Tests: engine `delete_tracks_removes_caption_tracks`; headless UI tests
for the header menu and the dialog (delete_caption_tracks_ui.rs).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 18:55:01 +01:00
ReReddo c8accc4946 settings: the hex colour field takes typing
The `#rrggbb` field next to a colour swatch (Appearance ▸ Highlight
Color) was refilled from the saved colour on every frame and kept an
edit only when the whole text was already a valid colour. One typed or
deleted character never is, so the field snapped back and nothing could
be typed.

While the field has focus it now keeps what is typed, and each valid
colour is saved as it comes; leaving the field shows the saved colour.

Tests: settings_ui::hex_field_takes_typing_a_character_at_a_time.
2026-10-10 20:53:43 +03:00
Nigel Ihza Firdausy 06ebdfa08a interchange: preserve named audio dissolves on EDL import 2026-10-11 00:53:37 +07:00
Nigel Ihza Firdausy 758330822f tts-text: preserve numeric groups, percentage units and decimal money 2026-10-11 00:52:00 +07:00
Manuel CamachoandClaude Opus 5.5 7f4bf2157b HW6: AV1 export format, enabled where a hardware encoder registers one (#30)
Format::Av1 in export settings and jobs, SampleEntry::av1 (av01 + av1C) in
isobmff, and the Export panel's Format list, profile and encoder rows. No
encoder ships here: like H.265, AV1 is listed greyed out until a platform
encoder registers a probe, and a job without one fails with "AV1 encoder not
available yet". Split out of #409; the VA-API AV1 encoder follows on main's
vaapi::va design.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 10:46:43 -07:00
impostazioniandClaude Opus 5.5 6aa4146c16 Captions: render the track font and add a font picker
The caption style's `font` was stored and settable through
captions.setStyle, but the burn-in renderer always drew Inter SemiBold.
Lay out, wrap and measure captions with the track's family (Inter when
the name is empty or not installed), so the Program monitor and burned-in
exports use the chosen font.

Add a font dropdown to the caption style strip in the Text panel's
Captions tab (automation id `text.captions.style.font`, options
`text.captions.style.font.option.<family>`, registered only while
visible). macOS-private faces (names starting with '.') are not offered:
some lack a space glyph and run words together.

Tests: renderer regression test (a non-Inter family changes the pixels;
failed before), fallback test for empty/unknown names, and a headless UI
test that picks Noto Serif from the dropdown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 18:29:09 +01:00
ReReddo 6d46d4f315 timeline: lines between the tracks, a short playhead head
- The line under each track was drawn in the Timeline background colour,
  the same as the tracks, so the tracks ran into each other. It now uses
  the separator colour the track headers already use, and the top video
  track also gets a line above it, in the headers and the track area.
- The playhead's head ran the full height of the ruler and covered the
  markers and the timecodes. It is now the short head Effect Controls
  uses, on the tick row.
2026-10-10 20:24:18 +03:00
vangel76andClaude Opus 5.5 a4050b4ffc #411: importing a folder keeps its structure as bins
Problem: mediaBrowser.import flattened a folder into its first-level
files (not recursive) and created no bins, so sub-folders were skipped
and the folder hierarchy was lost in the project.

Fix: a folder becomes a bin named after it (in `bin`, else the root)
holding its files, and each sub-folder with media a bin inside that, as
Premiere's Media Browser does. Folders without media get no bin. The
walk is bounded (depth 32, 4096 folders) against symlink loops. The
result also lists the new `bins`. Import as Image Sequence and plain
file imports behave as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 19:22:52 +02:00
vangel76andClaude Opus 5.5 78e9221a34 #614: document timeline.thumbnails outside the ui.set row
The ui.set table row is one long line; #412 (PR #656) also edits it, so
the two PRs conflicted with each other in docs/control-protocol.md
whichever merged second. The thumbnails parameter is now described in
its own paragraph below the table, leaving that row untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 19:22:42 +02:00
vangel76andClaude Opus 5.5 0c2cc293bc #342: translate the ProRes 4444 export presets
i18n::tests::spanish_translates_dynamic_names lists every built-in
export preset's name and description; the three new ProRes 4444 presets
had no Spanish entry, so ui-egui's lib tests failed. Adds them to es, ja
and zh-cn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 19:22:28 +02:00
vangel76andClaude Opus 5.5 25d9bff1dd #457: translate the UI Scale setting
i18n::tests::spanish_translates_dynamic_names lists every settings
label and choice; "UI Scale" and "Auto (system)" had no Spanish entry,
so ui-egui's lib tests failed. Adds them to es, ja and zh-cn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 19:18:04 +02:00
Oskar ac8b777447 contributors: add myself (ossini), shown with my GitHub profile name 2026-10-10 19:17:47 +02:00
vangel76andClaude Opus 5.5 3feadebbc4 #313: J/K/L shuttle the Source monitor both ways at up to 8x
Problem: with the Source monitor focused, L only ever played at normal
speed, and J and the slow shuttle commands returned "Source playback
currently supports normal forward speed": the Source clock always ran at 1x.

Fix: the Source clock runs at a signed speed. L/J play forward/backward
and pressing the same key again doubles the speed up to 8x; Shuttle Slow
Right/Left play at 1/4 speed. As in the Program monitor, speeds other than
normal forward play without sound, and backward playback stops on the
first frame. docs/monitors.md updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 19:16:40 +02:00
Oskar ca44d85450 #648, #668: click a gap in the Timeline, Delete closes it
Clicking empty time between two clips selects the gap, drawn as a light
block across it. Delete or Backspace closes it; right-clicking it opens
a menu with Ripple Delete. A cut's edit point menu right-clicked before
no longer stands in for the gap's menu.
2026-10-10 19:15:30 +02:00
Oskar c8f85af5a1 #648, #668: select a gap and close it with Delete or Ripple Delete
timeline.selectGap selects the empty time on one track between two
clips (or before the first one). Delete, Backspace (edit.clear) and
Ripple Delete then close it as one "Ripple Delete" undo step, as in
Premiere Pro. The selection lives in the editor state (gapSelection in
sequence.inspect), is replaced by a clip, transition or edit point
selection, and is dropped by an edit that changes the gap.

sequence.closeGap shares the same code.
2026-10-10 19:15:30 +02:00
Oskar 86d113e557 edit: Close Gap refuses a locked track
Closing a gap on a locked track skipped that track but still moved the
sync-locked ones, pulling them out of sync with it. It is now refused
with the usual "locked" error.
2026-10-10 19:15:30 +02:00
vangel76andClaude Opus 5.5 c85d0ac973 #614: Head, Head and Tail and Continuous video thumbnails in the Timeline
Problem: a video clip only ever showed its first frame. Show Video
Thumbnails was a plain on/off switch, so a wide clip at high zoom stayed a
flat colour after its head.

Fix: the wrench menu keeps Show Video Thumbnails and adds Head (the
default, as before), Head and Tail and Continuous under it. Continuous
tiles aspect-correct frames across the clip on a grid anchored at its head,
only where the panel shows them (at most 256 per clip), each frame the
source time under the tile's left edge (speed, reverse, frame holds,
multicam angles), requested at thumbnail priority through the existing
frame cache. A clip too narrow for the tiles falls back to its head. The
mode is UI state (`thumbnail_mode`, serde default Head, so saved settings
keep working) and `ui.set {timeline: {thumbnails}}` sets it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 19:08:48 +02:00
ReReddo 1ed57e6d4a timeline, text editing, program monitor: clicking empty space deselects (#683)
Deselecting by clicking somewhere empty only worked in some spots.

- Timeline: a click on no track at all (below the tracks, between the
  video and audio tracks) did nothing; it now deselects clips, transitions
  and edit points, as a click on an empty stretch of a track does, and a
  drag from there draws a box selection.
- Text editing: a text layer being edited took the keyboard back on every
  frame and read every key, so after clicking another panel, typing there
  (the Effects search, say) went into the text (also #227). Moving to
  another panel now ends the edit; Properties and Essential Graphics, which
  style the text, keep it. While another field has the keyboard the keys
  are its own, and the text takes the keyboard back only when nothing has
  it. Only a move counts, so an edit an agent starts with another panel
  focused goes on.
- Program monitor: a click around the picture, or on the picture's empty
  space, ends the text edit and deselects the graphic layers and the mask
  (graphics::deselect_on_monitor); before, only the picture's empty space
  ended an edit or deselected layers, and a mask stayed selected.

The transform box (#639) follows in its own change once that lands.

Tests: deselect_ui (3: empty Timeline space deselects and starts a box;
clicking another panel ends a text edit while Properties keeps it; empty
monitor space deselects a layer and a mask).
2026-10-10 20:08:38 +03:00
vangel76andClaude Opus 5.5 d8f22e9908 #67: media files share a bounded pool of open descriptors
Problem: every FileReader owned its std::fs::File for its whole
lifetime, so each open media source pinned one descriptor. A project
with hundreds of sources hit the soft limit (256 on macOS, 1024 on
Linux) and the remaining media failed to open or showed as offline.

Fix: FileReader keeps its path and borrows the file from a shared LRU
pool of at most MAX_OPEN_FILES (128) open files. Past the cap the least
recently read file is closed and its reader reopens it on the next
read; a file that is now shorter than when it was opened is refused
instead of being read as the old one. Dropping a reader closes its file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 19:01:11 +02:00
vangel76andClaude Opus 5.5 379b9b5dec #659: macOS app may use the microphone for voice-over
Problem: the macOS bundle had no NSMicrophoneUsageDescription and empty
hardened-runtime entitlements, so macOS never asked for microphone access
and voice-over recording got no input.

Fix: declare NSMicrophoneUsageDescription in Info.plist.in and grant
com.apple.security.device.audio-input in entitlements.plist; an xtask test
guards both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 19:00:52 +02:00
Nigel Ihza Firdausy 0133f1d717 fix(captions): keep trim-tolerant tag recognition 2026-10-10 23:58:13 +07:00
Nigel Ihza Firdausy fc69e9cad9 fix(captions): preserve whitespace-tolerant styling tags 2026-10-10 23:57:43 +07:00
Nigel Ihza Firdausy f9f0268b65 fix(captions): keep literal angle-bracket text in EBU STL exports 2026-10-10 23:57:01 +07:00
vangel76andClaude Opus 5.5 878000df07 #279: WAV opens in place and reads only the samples asked for
Problem: WAV had no streaming reader, so open_reader fell back to reading
the whole file into memory, and WavSource kept those bytes for its
lifetime. Importing a 90-minute WAV cost about its size in memory and
blocked until the whole file was read.

Fix: WavSource reads through a SharedReader. Opening walks the chunk
headers (fmt, data, bext, iXML) without touching the samples, and
audio() reads only the frame range a request needs, in bounded blocks.
open_reader takes WAV this way after the registered reader openers;
WavSource::parse keeps working on in-memory bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 18:54:30 +02:00
Nigel Ihza Firdausy 5763460a62 fix(captions): reject overflowing TTML times and round extreme clocks safely 2026-10-10 23:52:06 +07:00
vangel76andClaude Opus 5.5 86856f18b9 #680: Shift+Delete ripple deletes with the Timeline focused
Problem: the focused panel's shortcuts were all tried before the
application-wide ones. egui ignores extra Shift and Alt when matching a
key press, so with the Timeline focused its `Delete` (Clear, #243)
swallowed the app-wide `Shift+Delete` (Ripple Delete) and the clip was
removed without closing the gap.

Fix: try the most specific chords first across both scopes; the sort is
stable, so a panel shortcut still overrides an application shortcut on
the same chord.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 18:49:48 +02:00
Vikram Bhaskaran 97243984d9 #496: Keep Link Media actions accessible for large lists and long paths. 2026-10-11 03:41:59 +11:00
Vikram Bhaskaran ff171b9c8c #680: Preserve individual linked-clip selections when deleting media. 2026-10-11 03:41:52 +11:00
Manuel CamachoandClaude Opus 5.5 5b9a909d47 HW5: AV1 sequence headers compare by picture format, so SVT-AV1 files stay on hardware (#30)
FrameStreamInfo::parameters_changed compared the in-band AV1 sequence header with
av1C's byte for byte. SVT-AV1 writes a provisional av1C header whose coding tool
flags (CDEF, restoration, warped motion...) differ from the in-band one, so every
such MP4 dropped to the software decoder from its first frame, on every hardware
backend. Headers now compare by what the picture format depends on: profile,
maximum size and colour configuration. A header that does not parse still counts
as a change. Regression test with an SVT-AV1 720p 10-bit av1C / in-band pair.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 09:41:45 -07:00
Nigel Ihza Firdausy 2fe4715d5c feat(timeline): rename tracks from their header (#654) 2026-10-10 23:41:13 +07:00
Nigel Ihza Firdausy 316435e54d fix(monitor): make Button Editor customize transport controls (#431) 2026-10-10 23:36:50 +07:00
vangel76andClaude Opus 5.5 5a243ea838 #239: Position keyframes get the value and velocity graphs
Problem: Effect Controls only offered the graph twirl-down for scalar
(Float) parameters. Position is a point (Vec2), so animating it showed
keyframes but never the value/velocity graphs that Scale has.

Fix: point parameters are graphable too. The value graph draws an X and
a Y curve with a draggable keyframe on each (a drag changes only that
coordinate); the influence handles sit on the X curve, since both
coordinates share them; the velocity graph shows the speed along the
path. NaN "auto" coordinates are skipped instead of drawn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 18:32:12 +02:00
vangel76andClaude Opus 5.5 4b67c44dbe #342: ProRes 4444 and 4444 XQ export, with alpha
Problem: the ProRes encoder already codes 4444 / 4444 XQ (optionally
with alpha), but export only knew the four 422 flavours: any other
proresProfile fell back to HQ and the encoder was always fed a 4:2:2
frame, so graphics could not round-trip with their alpha.

Fix: proresProfile accepts 4444 and 4444xq. The ProRes export encoder
builds a 4:4:4 frame for them and, with Include Alpha Channel, codes
the straight alpha of the unflattened picture (ExportSettings::
supports_alpha / keeps_alpha). Export Settings offers both profiles and
the alpha checkbox for them; new presets Apple ProRes 4444, 4444 with
Alpha and 4444 XQ; summary text and the MXF coding label name the
profile (and the MXF descriptor keeps its 12-bit 4:4:4 depth).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 18:12:54 +02:00
Abdelkabir Ouadoukou 0f08798af0 platform: look up kVTCompressionPropertyKey_ConstantBitRate at run time 2026-10-10 15:54:00 +00:00
Vikram Bhaskaran 9e77d9d786 Apply Transform shutter angle when the override is on.
The Transform effect listed a shutter angle but had no override, so motion blur never used it. Checking Use Effect Shutter Angle now samples the animated transform across that angle.
2026-10-11 02:48:52 +11:00
vangel76andClaude Opus 5.5 c4e61b3c83 #457: Settings > Appearance > UI Scale
Problem: FilmCraft had no UI scale preference. The interface always
followed the scale the desktop reported (150% on some X11 setups), and
only Ctrl+-/Ctrl++ could change it, for the session only.

Fix: add appearance.uiScale (Auto, 75%-300%) to Settings > Appearance.
A fixed value sets egui's zoom so a point is that many physical pixels
whatever the system reports; it is applied when the setting or the
system scale changes, so Ctrl+-/Ctrl++ keep working for the session,
and Auto gives the system's scale back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 17:45:50 +02:00
vangel76andClaude Opus 5.5 10ffc016b8 #284: reset buttons keep keyframes; a reset button on every property
Problem: Reset Effect replaced the effect with a fresh instance, so every
keyframe (and every mask) on it was deleted, and Effect Controls had no
way to reset a single property.

Fix: effects.reset now resets the parameters in place, as Premiere does:
a static parameter gets its default, an animated one keeps its keyframes
and gets the default as a keyframe at the playhead; masks stay. New
effects.resetParam does the same for one parameter, and every Effect
Controls property row has a reset button for it (automation id
effectControls.<effect>.<param>.reset).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 17:44:06 +02:00
Vikram Bhaskaran ec7c43eea2 Treat media extensions as case-insensitive on import.
Camera files such as CLIP.MOV were hidden or skipped on Linux because sibling matching and tests assumed lowercase suffixes. Import, the media browser, and image-sequence numbering now accept mixed-case extensions.
2026-10-11 02:39:53 +11:00
Vikram Bhaskaran 0a1f9730c8 Add file.import replace mode for empty-project round-trips.
Headless import merged into the current project, so a demo or leftover timeline mixed into export. replace:true starts from file.newProject with the same name, then imports, which is the documented empty-project-start workflow.
2026-10-11 02:36:46 +11:00
Vikram Bhaskaran 80db16ebd5 Apply Ultra Key Setting as a parameter preset.
Relaxed and Aggressive previously only scaled two internal multipliers, so the matte stayed at Default. Switching Setting now writes Premiere-like matte values and the renderer uses that set, including Custom for hand-tuned keys.
2026-10-11 02:34:08 +11:00
vangel76andClaude Opus 5.5 6a995d7c0a #333: Fade In and Fade Out presets add up on one clip
Problem: presets.apply replaced an intrinsic effect (Opacity) wholesale, so
applying Fade Out after Fade In discarded the Fade In keyframes (and vice
versa).

Fix: when a preset lands on an intrinsic effect, keep the clip's existing
keyframes that lie outside the span the preset animates for that parameter.
Re-applying the same preset still replaces its own ramp; static preset values
behave as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 17:30:17 +02:00
Vikram Bhaskaran 6d1c4fdb84 Pass Hard Limiter look-ahead into the DSP limiter.
The project Look-Ahead Time parameter was stored but never applied, so 0 ms and 30 ms mixed the same clip. The limiter now sizes its look-ahead from that parameter and pre-rolls for the chosen window.
2026-10-11 02:28:52 +11:00
vangel76andClaude Opus 5.5 3c0fa2ab0b #461: interchange import links media through the streaming reader
Problem: after merging an FCP7 XML / OTIO / EDL / AAF document, the import
linked each media file by reading it whole into memory (services.read_file +
open_bytes). Multi-GB 4K files took minutes and stayed resident, while a
direct import opens them in well under a second.

Fix: link through MediaPool::open_file, as import_streamed does: the host's
random-access reader reads only the container index now and samples on
demand (hosts without a reader still read the file whole, as before).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 17:21:56 +02:00
vangel76andClaude Opus 5.5 a902b11e80 #653: a dragged trim edge stops at the end of the media
Problem: dragging a clip edge in the Timeline previewed the raw pointer
delta, so the edge kept growing past the end of the clip's media (or into
the neighbouring clip) and only jumped back when the button came up and
`timeline.trim` clamped it (#653, point 2 of #374).

Fix: `commands::trim_delta` computes, without editing, the delta
`timeline.trim` would apply (the same clamp across linked partners, now
shared by both), and the Timeline's trim drag shows that delta.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 17:19:58 +02:00
MrCloudy2andClaude Opus 5.5 69bb1dd33c #480: exports no longer stop for good when a clip's decoding uses rayon
A GopCache decodes with its lock held, and that decoding starts rayon work
(rotating iPhone frames, cropping, slice-threaded decoders). On the global pool
a worker that took part of it could, waiting inside it, pick up another render
job asking the same cache for a frame and block on the lock, while the lock
holder waited for the work on that worker's stack: exports of rotated iPhone
HEVC clips stopped making frames in about half the runs.

The locked decoding now runs on a separate decode pool, whose threads never wait
for a cache's lock; a caller that is a rayon worker waits on a plain thread so it
does no other rayon work while holding the lock. The cancel/catch-up/draft/
background hints travel with it (filmcraft_media::cancel::Context). Intra-only
streams crop and rotate before taking the lock again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 17:14:34 +02:00
JordyUSAandClaude Opus 5.5 4ce807263d Project panel test: drop an unneeded mut (clippy unused_mut)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 22:06:45 +07:00
Vikram Bhaskaran 170852d8db #533: Preserve explicit export sample rates to avoid silent substitution. 2026-10-11 02:00:58 +11:00
vangel76andClaude Opus 5.5 b24225afd3 #642: open Help/About links in the system browser on the desktop
Problem: every link (Help menu, About dialog, header Discord button, Import
screen, Settings and Keyboard help) went through egui's `ctx.open_url`. The
workspace builds eframe without its `links` feature, so egui-winit only logs
"Cannot open url - feature links not enabled" and nothing opens on Linux,
Windows or macOS.

Fix: add a `HostHooks::open_url` hook, as PhotoCraft does. `links::open` now
takes the app and calls the hook, falling back to `ctx.open_url` when it is
unset or fails (the web build keeps opening a new tab). The desktop app sets
it to `open::that_detached` (open 5, MIT), which does not block the UI
thread. Test: every Help link reaches the hook, and a failing hook falls
back to `ctx.open_url`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 16:58:26 +02:00
vangel76andClaude Opus 5.5 b6ba78e300 #412: select a single keyframe in Effect Controls
Problem: the keyframe lane highlighted every keyframe whose time equals the
playhead. A click on a keyframe moves the playhead there, so every
parameter's keyframe at that frame lit up (point 4 of #412).

Fix: a serde keyframe selection in the UI state (clip, effect index, param,
mask, media time). A click (or drag start) selects exactly that keyframe and
still moves the playhead; highlighting reads the selection. A moved keyframe
stays selected; references to deleted keyframes or clips no longer shown are
dropped. Readable via ui.inspect, settable via ui.set keyframeSelection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 16:57:20 +02:00
JordyUSAandClaude Opus 5.5 360183e273 Merge main into feat/export-crf
Conflict in crates/engine/src/export_tools.rs: both sides added an
import (BitrateMode for CRF, GpuRendering from main); both kept.
main's new Chinese catalog requires every tl! string: added zh-CN for
"Quality (CRF)" and "CRF (constant quality)".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:45:31 +07:00
vangel76andClaude Opus 5.5 7eaba0ddd5 #456: Select All in the Project panel takes only the shown bin
Problem: with the Project panel focused, Cmd+A ran project.selectAll,
which selected every item in the project: items in closed bins and in
bins of other tabs too.

Fix: project.selectAll takes an optional `bin` (its direct items) and
`expanded` (sub-bins twirled open in List view, whose items are shown
and count). Unknown or malformed ids are an error and leave the
selection alone; without `bin` it still selects the whole project. The
UI passes the shown tab's bin and, in List view, the open bins; while a
search filters the panel it selects exactly the rows shown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 16:43:42 +02:00
Max Murphy 2472b890ac MP4 rtmd: reject drop-frame labels that counting skips
Drop-frame counting never emits the first base/15 frame labels of a minute
except each tenth (frames 00-01 at 29.97, 00-03 at 59.94). Accepting them let
a damaged 00:01:00;00 normalize to a different start time, so fields_to_frames
now rejects those labels and the start timecode is left unset.
2026-10-10 10:21:03 -04:00
Max Murphy 83fb73b9b1 MP4 rtmd: bound the header read and reject damaged timecode fields
Address review of #460:
- Read only the fixed-size rtmd header (18 bytes) instead of read_sample,
  which allocated the whole untrusted stsz size (a corrupt multi-GiB size
  could exhaust memory during import).
- Validate hours/minutes/seconds/frames against the frame base and drop the
  start timecode when they are out of range, instead of normalizing damaged
  metadata into a plausible but wrong offset.
2026-10-10 10:21:02 -04:00
Max Murphy cfebe8df14 Contributors: credit maxm11 (Max Murphy) 2026-10-10 10:20:45 -04:00
Max Murphy 23b4bf24b1 MP4: read the start timecode from a Sony rtmd track when there is no tmcd
Sony XAVC cameras such as the FX3 and a6400 keep the clip's start timecode
in an rtmd (real-time metadata) data track rather than a QuickTime tmcd
track, so probe reported start_timecode: null and timecode sync could not
use the footage. Read the hours/minutes/seconds/drop-frame/frames block from
the first rtmd sample and expose it as MediaInfo::start_timecode, falling
back only when no tmcd track is present (#460).
2026-10-10 10:20:33 -04:00
Max Murphy 4fcc437bf6 perf(export): index the bin tree once in closure()
The bin/item id split checked p.root.find_bin for every queued id, and
find_bin walks the whole tree for an item id, making a flat N-item export
O(N^2). Build the bin id set and each bin's direct children once, then
test membership and expand from the index, so closure() is O(N log N)
and bin expansion never re-walks the tree.
2026-10-10 10:05:01 -04:00
vangel76andClaude Opus 5.5 1169ef2a93 contributors: credit vangel76
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 15:49:08 +02:00
vangel76andClaude Opus 5.5 1e3d4a8a8c #591: named clips that cannot stand in for a selection say why
Problem: a command that takes `clips` / `clip` answered "no clips
selected" when the caller named clips that are not in the active
sequence. Over MCP or the control channel that reads as if the
explicit ids were ignored: a clip id of another sequence (for example
from `sequence.inspect {"item": …}`), an id that is no clip at all (a
track or project item id from the same inspect output) and a string
id like "29" all gave the same message.

Fix: when the selection check fails and the parameters named clips
under a key the command documents, but none is a clip of the active
sequence, the reason names the first one and what is wrong with it:
the sequence it is in (with the `sequence.open` call to make it
active), that no clip has that id, or that the value is not a clip id.
Enablement itself is unchanged: such a call is still refused, and a
call that names nothing keeps the selection's reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 15:48:55 +02:00
JordyUSAandClaude Opus 5.5 8f0f689c85 Project panel: double-click anywhere empty in Icon view opens Import
In Icon view each row of cards was laid out as a hover-only strip, so
only the strip under the last row sensed clicks. With an empty project
the whole panel is that strip and double-click opened Import; as soon
as the project had cards (for example after placing audio in the
timeline, which adds the clip and its sequence), most of the visible
blank space sat beside or between the cards and ignored double-clicks.

A click-sensing background now covers the visible Icon view behind the
cards (registered first, so cards keep their own clicks) with the same
handlers as the empty strip: click deselects, double-click imports,
right-click opens the panel menu.

Tests double-click with real pointer input at 60 fps: between and beside
the demo project's bins, and beside the cards after a tone is placed on
A1; both failed before (0 imports) and pass now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 20:10:39 +07:00
KattandClaude Opus 5.5 b92078ec7e Media: a file that can't be opened is tried again once it changes, not on every look
A media file the refresh couldn't open (still being written, or not media
FilmCraft reads) kept its old stamp, so every look saw it as changed and
opened it again on the UI thread. It now keeps its new stamp and is tried
again once it changes again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 18:30:21 +05:30
Mario PodolanandXiaomi MiMo 7123e1fc46 ui: detect software rasterizers and say GPU acceleration is unavailable
When wgpu falls back to a software rasterizer (llvmpipe / SwiftShader /
softpipe - containers and VMs without GPU device access, missing drivers),
set_wgpu accepted it as the GPU compositor (llvmpipe satisfies the Rgba16Float
checks), so the preview quietly composited through WGSL on a CPU rasterizer:
GPU usage 0 %, CPU saturated, laggy playback, no visible explanation.

software_rasterizer() now detects CPU rasterizers (device_type == Cpu, or
llvmpipe / SwiftShader / softpipe in the name) and gpu_compositor_unsupported
rejects them first, so the monitors use the native CPU compositor, which
renders the same frames faster than wgpu-on-llvmpipe. The status bar shows the
reason ("GPU acceleration is unavailable; preview renders on the CPU
(software rasterizer (...))") in addition to the log line. New tl! string with
Spanish, Japanese and Simplified Chinese translations.

Test: gpu_fallback_tests::software_rasterizers_are_detected.

Co-Authored-By: Xiaomi MiMo <noreply@xiaomi.com>
2026-10-10 14:43:28 +02:00
Brandon Thomas 98ba9acbce Merge pull request #421 from mutombopr/gpu-rendering
GPU export rendering (on by default) and Vignette, Video Limiter, Lumetri Color on the GPU compositor
2026-10-10 08:35:25 -04:00
Lukas KellerandClaude Opus 5.5 63c9b17454 contributors: credit @keller-lukas as Lukas Keller
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 14:35:14 +02:00
Lukas KellerandClaude Opus 5.5 09165a383d i18n: German (de) joins Edit > Language
Language::De is persisted as `de` and translated through a new de.tsv
catalog that covers every menu label and the native macOS Settings menu,
written from the plain meaning of the English labels with informal "du"
and infinitive commands. The new app.language.german command checks,
switches and restores the language like the others, the native macOS
menu maps it back, Settings > General offers Deutsch, and System Language
picks German for any de-* locale (de-DE, de-AT, de-CH ...).

A new test, built like the Ukrainian one, fails when German misses a core
menu label (the Brazilian Portuguese entries, the top-level menus, the
macOS Settings menu) or keeps an entry that is no longer a menu label;
menu labels added later use English until they are translated. The
System Language test no longer uses `de` as an unsupported example.

Closes #490

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 14:35:14 +02:00
ReReddo 06b835d5a8 timeline, project panel: paste images and copied files from the system clipboard (#611)
Ctrl+V with a screenshot or an image copied in another app, or with files
copied in Explorer / Finder, now pastes them. In the Timeline they are
imported and placed at the playhead, end to end, without overwriting
anything: each clip goes on the first free track from the targeted one
outward (up for video, down for audio), and a track is added when every
track is busy there. Ctrl+Shift+V inserts them on the targeted tracks, as
Paste Insert does. In the Project panel Ctrl+V imports into the shown bin.
One undo step takes the paste back.

Copying clips already puts their names on the system clipboard as text, so
media on the clipboard was copied after them; without media, Paste pastes
the copied clips as before.

- engine: edit.pasteMedia {paths, insert, place, bin}; paste_media::
  save_pasted_image writes "Pasted Image <n>.png" where voice-over takes and
  narrations go (voiceover::media_dir: Scratch Disks > Captured Audio and
  Video, else next to the project, else the data directory), trying a
  bounded number of names as voiceover::unique_wav_path does.
  timeline.place's default length moves to commands::full_duration so both
  share it.
- ui-egui: HostHooks::clipboard_media reads the clipboard (ui-egui stays
  OS-independent); menus::invoke turns edit.paste / edit.pasteInsert into a
  media paste when the Timeline or Project panel has focus. egui-winit drops
  the Ctrl+V press when the clipboard holds no text but passes the release,
  so a V release without its press puts the press back. That also makes
  Ctrl+V paste copied clips when the system clipboard is empty.
- filmcraft (desktop): the hook, with arboard (already in the tree through
  egui): copied files first, else an image.

Not covered: the web build, whose browser clipboard works differently.

Tests: paste_media_tests (5: image above the footage with nothing
overwritten and one undo step, tracks added when every track is busy,
Paste Insert, import only, saved image names) and clipboard_paste_ui (4:
an image pasted from a key release with no text, copied files and Paste
Insert, the Project panel imports, copied clips still paste and a plain V
pastes nothing).
2026-10-10 15:34:47 +03:00
Brandon ThomasandClaude Opus 5.5 58f768c472 Merge origin/main into gpu-rendering
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 08:27:57 -04:00
Brandon Thomas 1abacc4d56 Merge pull request #278 from mutombopr/tts-narration
Text to Speech narrations: panel, natural voices (Kokoro-82M in pure Rust), click-to-edit
2026-10-10 08:27:12 -04:00
JordyUSAandClaude Opus 5.5 696f428faf #374: trim preview keeps the waveform in place instead of squeezing it
While an edge was dragged, the Timeline drew the clip at its new length
from the untrimmed item, so draw_waveform spread the clip's whole source
range over the shorter (or longer) box: the waveform squeezed and the
sound under the pointer moved, which made it hard to trim to where the
music starts (#374, item 13). Trim and roll previews now draw from the
item the edit will leave, so every timeline time still covered shows the
same media and the waveform is cut off at the moving edge. Video
thumbnails show the new first frame the same way. Rate Stretch, which
really fits the same media into a new length, still squeezes.

The item update is factored out of trim and ripple_trim_group into
filmcraft_edit::move_edge, which the preview uses too, so preview and
edit agree for speed-changed, reversed and frame-held clips.

Rendered mid-drag against the released result, the music clip's
waveform matched in 15% of pixel columns before and 100% after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 19:24:37 +07:00
Brandon ThomasandClaude Opus 5.5 c030f78762 Merge origin/main into tts-narration
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 08:13:05 -04:00
Brandon Thomas b47248bbcf Merge pull request #246 from ahmedhosnypro/fix/linux-dev-desktop-icon
linux: install a dev desktop entry so the taskbar shows the app icon
2026-10-10 08:12:47 -04:00
Brandon ThomasandClaude Opus 5.5 be498865c3 Merge origin/main into fix/linux-dev-desktop-icon
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 08:12:41 -04:00
KattandClaude Opus 5.5 de6ab8d239 Media: translate Refresh Changed Media (es, ja, zh-CN)
Portuguese and Ukrainian catalogs only hold menu labels, so they keep the
English name like the other menu-less commands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 17:41:45 +05:30
Brandon Thomas d01440d950 Merge pull request #254 from ahmedhosnypro/fix/alsa-default-fallback-23
Playback: fall back to a working output device when the ALSA default cannot be opened
2026-10-10 08:10:09 -04:00
Brandon Thomas b638f94d7a Merge pull request #442 from cshapiro/fix/windows-upstream-validation
WINCI: Repair HEVC timing and CRLF translation validation
2026-10-10 08:10:03 -04:00
Brandon Thomas 9847a5cbf1 Merge pull request #357 from jnohclee-rgb/dharma/filmcraft-344-help
cli: make --help print usage successfully
2026-10-10 08:09:57 -04:00
Brandon Thomas 7f81d6b063 Merge pull request #390 from txiki739/fix/linux-file-dialog-extensions
#262: file dialogs list extensions in any capitalisation on Linux
2026-10-10 08:09:51 -04:00
MarekSpodBiedry b81792c453 Space out browser downloads so image-sequence exports are not dropped (#378) 2026-10-10 14:06:01 +02:00
MarekSpodBiedry b5965dc568 Let a bin be cleared from its context menu, the trash button, the panel menu and Delete (#587) 2026-10-10 13:59:04 +02:00
Brandon ThomasandClaude Opus 5.5 28b704ba82 #278: zh-cn rows for Text to Speech and Edit Narration
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 07:54:23 -04:00
KattandClaude Opus 5.5 a72d3fe1b9 Media: Automatically refresh growing files works (and catches re-renders)
Settings > Media > Automatically refresh growing files and Refresh
growing files every were listed but not wired. The desktop app now
stamps each file media item's file (size + modification time) on a
worker thread every N seconds and as soon as its window comes back to
the front, and reads changed files again the way import opens them: the
item gets the new MediaInfo (length, frame size) and identity, and the
media pool swaps in a new source (the old one indexes the file as it
was). Not an undo step and a clean project stays clean (bump_view). This
also covers a render written over the file a clip uses (an EffectCraft
or After Effects round trip). A file that can't be opened yet is tried
again. media.refreshChanged runs one look (CLI, MCP).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 17:07:46 +05:30
Brandon Thomas 5a5684484f Merge pull request #517 from mitchmckinstry/feat/drop-import-to-timeline
[feat] files dropped onto the Timeline are placed there, not just imported. Supports preview too
2026-10-10 07:28:19 -04:00
Brandon Thomas 4ab7134a46 Merge pull request #434 from David-T-Campos/fix/share-sequences-across-edit-snapshots
project: share sequences across edit snapshots
2026-10-10 07:28:12 -04:00
Brandon Thomas ef51ef569d Merge pull request #503 from swillunderscore/transitions-select-resize
#430, #224: select, edit, delete and resize transitions
2026-10-10 07:28:04 -04:00
Brandon Thomas ad10c6259c Merge pull request #520 from maccagnit/setparam-choice-names-keyframe
effects.setParam: choices by name, a `keyframe` option, and the keyframe count in the result
2026-10-10 07:27:56 -04:00
Brandon Thomas b962329b0d Merge pull request #515 from Hathor875/feat/clip-link-pairs
clip.link: pair video clips with overlapping audio clips (#507)
2026-10-10 07:27:49 -04:00
Brandon Thomas 409a8d34fa Merge pull request #522 from gyroninja/fix/graphics-drag-undo-step
Essential Graphics: Make a property drag  one undo step
2026-10-10 07:27:43 -04:00
Brandon Thomas 6fe9b5680e Merge pull request #470 from sanyamsingh001/codex/scalar-graph-current-main
Start scalar graph drags at the pointer press position
2026-10-10 07:27:36 -04:00
Brandon Thomas 106ab5e337 Merge pull request #509 from mjohnson7cd/fix-mcp-media-import-schema-499
fix(mcp): media_import schema mirrors file.import (#499)
2026-10-10 07:27:29 -04:00
Brandon Thomas 66122699a7 Merge pull request #465 from MarekSpodBiedry/issue-463
FCP7 XML: keep the source channel of split mono audio tracks (#463)
2026-10-10 07:27:23 -04:00
Brandon ThomasandClaude Opus 5.5 c9e803ff3b Merge origin/main
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 07:24:39 -04:00
Brandon ThomasandClaude Opus 5.5 cebe4ee9e0 #421: zh-cn rows for the GPU Rendering export strings
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 07:24:32 -04:00
Brandon ThomasandClaude Opus 5.5 ac103be552 Merge origin/main into fix-503; Chinese and Japanese rows for the transition strings
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 07:21:51 -04:00
Brandon ThomasandClaude Opus 5.5 762344769b Merge origin/main
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 07:21:39 -04:00
Brandon ThomasandClaude Opus 5.5 4304cd9382 Merge origin/main into #421; GPU export opt-in, wgpu error handler, drop panicked renderers
- GpuRendering defaults to Off (opt-in per export) until measured on Windows and macOS;
  docs and ROADMAP say so.
- The shared export device installs an uncaptured-error handler and a device-lost callback
  that log once and retire GPU export for the session (frames go to the CPU) instead of
  wgpu's default panic.
- RendererPool drops a renderer that panicked and retires the pool, so the rest of the
  export renders on the CPU (regression test in gpu_hook_scenarios); the duplicate
  hostile-size test that raced on the global registry is folded into it.
- Rebased onto main's render_plan/transitions: render_export_prepared reads the linear
  accumulator after render_plan; transition (Composite) plans fall back to the CPU.
- es/ja rows for the new Export strings; web_time::Instant in the export pipeline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 07:21:38 -04:00
Brandon ThomasandClaude Opus 5.5 ab227a1853 Timeline file drops: Chinese and Japanese strings
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 07:19:37 -04:00
grib_job 1b83994c93 feat: Add Russian (ru) localization (updated) 2026-10-10 14:14:11 +03:00
Brandon ThomasandClaude Opus 5.5 1bf590c73c Timeline file drops: merge origin/main; document the platform cursor exception and mixed-DPI limit
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 07:11:18 -04:00
Brandon Thomas e831d0f248 Merge remote-tracking branch 'origin/main' into fix-434 2026-10-10 07:08:57 -04:00
Brandon ThomasandClaude Opus 5.5 68c297b987 Arc<Sequence>: fix the missed &mut sites and closing braces; drop the unrelated key-dispatch test
source_monitor markers and format's audio-stream clamp now detach with Arc::make_mut;
restore the final braces in clip_ops.rs and project/src/lib.rs (cargo fmt);
scripted.rs is back to main (the test covered key dispatch, which this PR doesn't change).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 07:08:56 -04:00
Brandon Thomas 4b172dcede Merge pull request #505 from luniiya/feat/system-appearance
Standardise system appearance modes and FilmCraft theme previews
2026-10-10 07:07:48 -04:00
Brandon ThomasandClaude Opus 5.5 1dc42c7d0d Simplified Chinese rows for the appearance-mode strings
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 07:07:40 -04:00
Brandon Thomas 1ace90d191 Merge remote-tracking branch 'origin/main' into fix-505 2026-10-10 07:05:06 -04:00
Brandon Thomas df3df9d352 Merge pull request #550 from hadmark/main
Add Simplified Chinese (zh) translation
2026-10-10 07:04:58 -04:00
Brandon Thomas ac8d62c547 Merge pull request #549 from SilentYeti/nvdec-linux
Linux: NVDEC hardware decoding for H.264 and HEVC
2026-10-10 07:04:53 -04:00
Brandon Thomas 805705398a Merge pull request #445 from LloydNicholson/fix/anamorphic-par
Issue #551: Anamorphic media: honour pixel aspect ratio in placement, monitors and New Sequence From Clip
2026-10-10 07:04:47 -04:00
Brandon Thomas 2160911fc1 Merge pull request #492 from MrCloudy2/fix/h264-crop-size-288
#288: MP4 / MOV pictures are cropped as the file says (H.264 / HEVC cropping, clean aperture)
2026-10-10 07:04:41 -04:00
Brandon Thomas 33bc073743 Merge remote-tracking branch 'origin/main' into fix-505 2026-10-10 07:01:42 -04:00
Brandon ThomasandClaude Opus 5.5 1ffe0cf4e7 #505: merge origin/main; ja/uk rows, colorTheme kept in line, docs
- ja.tsv rows for the new appearance strings (no duplicate Dark/Light);
  uk.tsv gets Next Appearance Mode, which the pt-br menu catalog now has.
- appearance.colorTheme is legacy input but no longer goes stale: it follows the
  mode and theme choices (the theme a fixed mode shows, the dark theme in Auto).
- docs/project-files.md documents appearanceMode / darkTheme / lightTheme.
- #489's menu/settings label test compares with the theme cards' choices.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 07:01:41 -04:00
Brandon ThomasandClaude Opus 5.5 ae852a36da Merge origin/main
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 06:57:46 -04:00
Brandon ThomasandClaude Opus 5.5 664c13efeb Merge origin/main into #278; schema v14, neural-voices off by default, review fixes
- Narrations take schema v14 (main's v13 is audio streams): v13_to_v14, docs updated.
- neural-voices is off by default in the desktop app, like whisper; the model download stays
  an explicit, confirmed job.
- Removed the debug probe2 test module from tts-text.
- project_subset prunes narrations of items it drops.
- unique_wav_path is bounded (99 999 names) and returns an error instead of looping.
- narration.rs uses web_time::Instant (wasm clock test); tts.* commands get the
  "Text to Speech" shortcut category, with es/ja rows for it and Edit Narration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 06:55:59 -04:00
Brandon Thomas a20ac75ee6 Merge remote-tracking branch 'origin/main' into fix-434 2026-10-10 06:55:52 -04:00
Brandon Thomas 995a8b4bdf Merge remote-tracking branch 'origin/main' into fix-503 2026-10-10 06:55:44 -04:00
Brandon ThomasandClaude Opus 5.5 adcd28b11a chrome_ui test: the menu bar's automation ids (menu.*, from #454) sit in the header
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 06:51:18 -04:00
Brandon Thomas 5c4a07b148 Merge remote-tracking branch 'origin/main' into fix-505 2026-10-10 06:48:09 -04:00
8e9b805952 #550: merge origin/main; Simplified Chinese as zh-cn with #389's plumbing
- One code everywhere: Language::ZhCn, persisted as `zh-cn`, catalog zh-cn.tsv.
- general.interfaceLanguage accepts and lists zh-cn (Settings > General).
- macOS native menu follows app.language.chinese.
- System Language picks Simplified Chinese only for zh, zh-CN, zh-SG and zh-Hans
  (not zh-TW / zh-HK / zh-MO / zh-Hant).
- Drop the second CJK font loader: cjk.rs already installs the Chinese fallback,
  so switching needs only cjk::available(); without a face the switch is refused
  and a saved zh-cn preference falls back to English at startup, like Japanese.
- Remove the fork's build-zh.yml workflow.
- Tests: Chinese covers every menu label and tl! literal; switching persists or is
  refused without a font; locale mapping. New strings get es/ja/zh rows.

Plumbing from #389 by @demon820308.

Co-authored-by: demon820308 <zwbigi@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 06:45:45 -04:00
Brandon ThomasandClaude Opus 5.5 8e0a824dcf Merge origin/main into fix-503 (keep both sides: Volume line + transitions)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 06:44:25 -04:00
Brandon Thomas 959c0a5a8b Merge remote-tracking branch 'origin/main' into fix-520
# Conflicts:
#	crates/engine/src/tests.rs
2026-10-10 06:43:01 -04:00
Brandon ThomasandClaude Opus 5.5 dc325d34c1 clip.link: clips without an overlapping partner stay unlinked; track kind worked out once
Merge origin/main (keep both contributor entries).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 06:38:35 -04:00
Brandon ThomasandClaude Opus 5.5 0414a1d346 #549: NVDEC callback state from Box::into_raw, freed in Drop; setting test accepts NVDEC or VA-API
Session kept the parser's user data in a Box and later wrote through that Box,
which invalidates the raw pointer the callbacks use (Stacked/Tree Borrows).
Rust code now reaches the state only through the same into_raw pointer, never
across the parse call, and Drop frees it after the parser is destroyed.
The setting test accepts either Linux backend's decoder when NVDEC registered,
and the README notes perf.stats names NVDEC for streams VA-API decodes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 06:32:00 -04:00
JordyUSAandClaude Opus 5.5 2e21c44904 Export: never crash on WAV/AIFF over 4 GiB; stream PCM to disk
WAV and AIFF sizes were 32-bit with unchecked arithmetic: past 4 GiB
(24-bit 5.1 at 96 kHz after about 41 minutes) a debug build panicked
and a release build wrote a wrapped, corrupt header. The whole file was
also built in memory first.

- WAV past 4 GiB is written as RF64 (EBU Tech 3306) with a ds64 chunk;
  smaller files stay plain RIFF.
- AIFF has no 64-bit form: a longer one is refused before anything is
  written, naming WAV instead.
- The header is written from the known length, then the samples one
  second at a time, so memory stays flat; progress advances per second.
- Odd-sized data chunks get their pad byte; hostile lengths, channel
  counts and sample rates are errors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 17:31:36 +07:00
Brandon ThomasandClaude Opus 5.5 d0addce05f Graphics drag undo: cargo fmt; UI tests read params through typed accessors
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 06:28:34 -04:00
Brandon ThomasandClaude Opus 5.5 e19eaa760b Merge origin/main into #246; dev desktop entry is opt-in, icons not embedded
- Nothing is written unless FILMCRAFT_DEV_DESKTOP_ENTRY=1 (debug builds no longer write into
  ~/.local/share on every cargo run).
- The 256/512 PNGs are copied from this checkout's assets/app-icon at runtime instead of
  include_bytes! into every Linux binary; a missing icon is logged and skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 06:22:16 -04:00
Brandon Thomas 4181a50c58 Merge remote-tracking branch 'origin/main' into fix-549 2026-10-10 06:21:46 -04:00
Brandon Thomas fdfbc34c1a Merge remote-tracking branch 'origin/main' into fix-445 2026-10-10 06:20:10 -04:00
Brandon ThomasandClaude Opus 5.5 57bd528769 Merge origin/main into #445; PAR test fixture uses MediaInfo::audio_streams
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 06:20:09 -04:00
Brandon Thomas e2ebbe715e Merge remote-tracking branch 'origin/main' into fix-246 2026-10-10 06:18:20 -04:00
Brandon ThomasandClaude Opus 5.5 19c756c252 Merge origin/main into #254; prefer pipewire/pulse, cache the fallback, translate notes
- When the default output cannot be opened, probe pipewire/pulse first and raw hw:/plughw:
  last (those take the card exclusively).
- The fallback list is probed only after the default fails, and the pick is cached per host,
  so devices()/configure() no longer open every PCM.
- Status-bar notes go through i18n::t/fmt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 06:18:12 -04:00
Brandon Thomas 40f710ec0a Merge remote-tracking branch 'origin/main' into fix-522 2026-10-10 06:16:09 -04:00
Brandon Thomas c567006057 Merge remote-tracking branch 'origin/main' into fix-470 2026-10-10 06:16:01 -04:00
gyro 81268330e9 Essential Graphics: Make a property drag one undo step 2026-10-10 03:14:40 -07:00
gyro 48e5350d30 Trim Monitor: Make trim drags no longer deadlock
data_mut() and drag_delta() both take egui's non-reentrant context
write-lock; calling the second inside the first froze the app on the
first frame of a drag in the Trim Monitor. The delta is now read
before the lock is taken, and a regression test drags through the
real Trim Monitor.
2026-10-10 03:14:19 -07:00
JordyUSAandClaude Opus 5.5 593515e884 h264enc: CRF level fits its bitrate; stream records its settings
Constant quality has no bitrate cap, but the level was chosen from the
picture size and rate alone: CRF 0 exports of 1080p camera footage
averaged 105 Mb/s (147 Mb/s peak) while signalling High@L4 (25 Mb/s).
The level now allows a worst-case estimate of 3 bits per pixel at
CRF 0, halving every 6 steps, so CRF 0 at 1080p30 signals 5.1 and the
default CRF 23 stays at 4.0.

The user_data_unregistered SEI on the first access unit now lists the
settings x264-style (rc=crf crf=18.0 profile=high level=4.0 ...).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 17:09:05 +07:00
JordyUSAandClaude Opus 5.5 3a46156861 Export: CRF (constant quality) bitrate encoding for H.264
Adds BitrateMode::Crf with ExportSettings::crf (0-51, default 23) and
maps it to the built-in encoder's RateControl::Crf, which export never
used. The Export panel offers "CRF (constant quality)" with a Quality
(CRF) slider in place of the bitrate fields; file.exportMedia takes
`crf` and `bitrateMode: "crf"`. H.265 (hardware only) rejects it, and
VideoToolbox / NVENC decline it so the built-in encoder runs. Settings
saved before this read as CRF 23.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 17:09:05 +07:00
Brandon Thomas e63fecfc06 Merge remote-tracking branch 'origin/main' into fix-254 2026-10-10 06:04:51 -04:00
Brandon ThomasandClaude Opus 5.5 2ff51268f1 Merge origin/main into #442; pin HEVC VUI offsets, drop the CRLF i18n hunk
The i18n CRLF normalisation already landed with #488, so main's version is kept.
abi_tests now asserts the HEVC VUI field offsets (inside the VUI and within
NV_ENC_CONFIG_HEVC) to rule out a binding mismatch behind the reversed timing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 06:04:44 -04:00
Brandon ThomasandClaude Opus 5.5 54f27f1f05 Merge origin/main
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 05:57:23 -04:00
Brandon ThomasandClaude Opus 5.5 e3a00c3a25 Merge origin/main into #357; check --help after Args::parse in cli()
Rebased onto main's cli()/out! refactor: `--help` is checked as a parsed flag, so
`filmcraft-cli export --help` prints the reference too (test extended).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 05:57:22 -04:00
Brandon Thomas 814287858c Merge pull request #407 from dmatviichuk/i18n-ukrainian
i18n: add Ukrainian (uk) interface translation
2026-10-10 05:50:06 -04:00
Brandon Thomas ba0e66c6d3 Merge remote-tracking branch 'origin/main' into fix-470
# Conflicts:
#	crates/ui-egui/src/panels/effect_controls.rs
2026-10-10 05:49:49 -04:00
Brandon ThomasandClaude Opus 5.5 e0d2c9c73b MCP media_import: test the paths schema and param mapping; update docs/agents.md (#499)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 05:49:28 -04:00
Brandon Thomas a8e38c4e07 Merge remote-tracking branch 'origin/main' into fix-445 2026-10-10 05:47:31 -04:00
Brandon ThomasandClaude Opus 5.5 72a9a7b7aa #492: clippy is_multiple_of in the oracle test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 05:47:22 -04:00
Brandon ThomasandClaude Opus 5.5 3cb3dc13aa Merge origin/main
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 05:46:51 -04:00
Brandon ThomasandClaude Opus 5.5 634d6cf359 Merge origin/main into #390; case-insensitive filters on main's translated dialog labels
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 05:46:50 -04:00
Brandon Thomas d3dd2fd312 Merge origin/main 2026-10-10 05:45:05 -04:00
Brandon ThomasandClaude Opus 5.5 d8a7b28b2e Merge origin/main into #492; ffmpeg oracle: decode unaligned crops, skip clap cases on ffmpeg without clap
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 05:45:04 -04:00
Brandon Thomas 4e9a450acd Merge remote-tracking branch 'origin/main' into fix-407 2026-10-10 05:44:26 -04:00
MarekSpodBiedry 77becde085 Audio Clip Mixer: click the dB readout to type a volume (#481) 2026-10-10 11:44:16 +02:00
Brandon Thomas 8454634fb8 Merge remote-tracking branch 'origin/main' into fix-509 2026-10-10 05:43:04 -04:00
Brandon Thomas a46b70eac8 Merge remote-tracking branch 'origin/main' into fix-465 2026-10-10 05:42:54 -04:00
Brandon ThomasandClaude Opus 5.5 24f8279327 FCP7: trackindex 1 means channel 0 when the file is split across mono tracks (#463)
A split-stereo left clip exported trackindex 1 and re-imported as full stereo,
losing the left/right split. Treat trackindex 1 as channel 0 when another
audio clip item of the same file in the sequence uses a higher index.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 05:42:52 -04:00
MarekSpodBiedry dcf2d7f423 Reject unknown mcp options instead of silently ignoring them (#476) 2026-10-10 11:42:50 +02:00
MarekSpodBiedry 628d573f1d Keyboard slip: clamp linked clips by one shared delta (#529) 2026-10-10 11:41:12 +02:00
liangforstudyandClaude Opus 5.5 4f8dcdf587 #402: WebM transparency (VP9 alpha in BlockAdditions)
A WebM VP9 track with AlphaMode 1 stores its alpha as a second, independently coded VP9
stream in each block's BlockAdditional (BlockAddID 1). The Matroska reader detected
AlphaMode (has_alpha was true) but never read the additions, so transparent areas
rendered black.

- matroska: BlockGroups' BlockAdditions are parsed; the BlockAdditional with ID 1 of an
  unlaced block is kept as Sample::addition, read with MkvFile::read_sample_addition.
- codecs: a WebM VP9 track with AlphaMode and additions gets an AlphaDecoder
  (webm_alpha.rs): it decodes each sample's alpha frame with a second software VP9
  decoder and attaches its luma plane (full range, converted to the picture's depth)
  as the frame's alpha. After a seek the alpha decoder resynchronises from the nearest
  alpha key frame. Colour pictures are held until their alpha picture is out, since the
  two decoders may return pictures after different calls. A missing or undecodable alpha
  frame leaves that picture opaque; it never fails the colour decode.

Tests: vp9_webm_alpha_layer_matches_libvpx (frames out of order and forward, colour and
alpha exact against ffmpeg's libvpx decode; fails before: no alpha plane; passed 200 of
200 runs with four in parallel), set_alpha unit tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq8BK1LmohzE1ga4NgAdWM
2026-10-10 17:40:58 +08:00
Brandon ThomasandClaude Opus 5.5 55d10195fb main: green again after the batch merges
- marker_export test: build the Session with struct-update syntax (clippy field_reassign_with_default, #454)
- es/ja catalogs: Export Markers as CSV… (#454) and the gradient fill labels (#514); drop the duplicate
  "Gradient" row #514 added to es.tsv

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 05:39:39 -04:00
ReReddo d67583385b project panel: drag in empty space to marquee-select, in List, Icon and Freeform views (#578)
Dragging with the left button from empty space in the Project panel draws a
selection rectangle, and every clip whose row or card it touches is selected,
live. In Icon view the gaps between cards count as empty space too, so a click
there deselects. A plain drag replaces the selection, Shift or Ctrl adds to
it, and Escape cancels and restores the selection the drag began with. The
selection goes through project.select; the rectangle registers
project.marquee.

Synthetic input (control channel, MCP, UI tests) now sends ModifiersChanged
around a pointer press and release that carry modifiers, so a drag that egui
recognises frames after the press still sees Shift / Ctrl, as with real keys.
This also lets automated Shift-drags of the Timeline marquee add to the
selection.

Limits: the rectangle does not select bins (the panel tracks one selected
bin), and in List view, where rows span the full width, it starts in the
empty space below the rows.

Tests: five project_panel_ui tests. The ui-egui suite passes except the two
Windows CRLF failures fixed in #488.
2026-10-10 12:38:47 +03:00
Luniiya 9e3c9c9b65 Merge remote-tracking branch 'upstream/main' into feat/system-appearance
# Conflicts:
#	crates/ui-egui/src/i18n/es.tsv
#	crates/ui-egui/src/i18n/ja.tsv
#	crates/ui-egui/src/icons.rs
2026-10-10 11:32:04 +02:00
SilentYetiandClaude Opus 5.5 3df0b9bb36 Merge remote-tracking branch 'origin/main' into nvdec-linux
tests/setting.rs: keep the check against the backend register() reported
(main's fix expects VA-API on Linux, which NVDEC now sits in front of).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 02:30:22 -07:00
Dmytro MatviichukandCodex 2c2e651eba i18n: add Ukrainian (uk) interface translation (#405)
Translate all 347 Spanish/Portuguese menu keys plus Clip, Audio and Settings.
Add the Ukrainian language command, serialized UI state, native macOS
menu updates, and coverage and Cyrillic glyph tests.

Co-Authored-By: Codex <noreply@openai.com>
2026-10-10 11:26:42 +02:00
Brandon Thomas bd1c043e19 Merge origin/main into fix-465 2026-10-10 05:24:33 -04:00
SilentYetiandClaude Opus 5.5 b3d68fe0ae NVDEC: reject pictures without valid bitstream data before the driver call
The parser's picture parameters were passed to cuvidDecodePicture unread.
Their shared head is now checked first: a null data pointer, zero length, no
slices, or a slice offset past the end of the data is an error, so the
session fails over to the software decoder and the driver never sees the
picture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 02:17:16 -07:00
Brandon Thomas 1201458630 Merge pull request #489 from ReReddo/fix/appearance-theme-name-471
appearance: View > Appearance names the middle theme "Dark", as Settings does
2026-10-10 05:12:53 -04:00
Brandon ThomasandClaude Opus 5.5 63c10831ff Merge origin/main into fix/appearance-theme-name-471
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 05:12:48 -04:00
Brandon ThomasandClaude Opus 5.5 df7cc97729 Format: rustfmt the clip_ops tests that landed with #561
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 05:11:32 -04:00
Brandon Thomas 0b92df2288 Merge pull request #569 from MarekSpodBiedry/issue-539
CLI render: report the real reason for an invalid scale instead of no sequence (#539)
2026-10-10 05:11:06 -04:00
Brandon ThomasandClaude Opus 5.5 14d2171282 Merge origin/main into issue-539 (keep both CLI helpers)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 05:11:01 -04:00
Brandon Thomas f04651adc5 Merge pull request #427 from Tom1tk/windows-av1-oracle-skip
Windows: skip AV1 oracle tests when ffmpeg lacks libsvtav1/libdav1d
2026-10-10 05:03:39 -04:00
Brandon Thomas dfdccd91e1 Merge pull request #504 from sawizzle/docs/installed-release-mcp
docs: register the MCP server from an installed release
2026-10-10 05:03:33 -04:00
Brandon Thomas 701c433a96 Merge pull request #501 from dylanmtaylor/chore/update-actions
chore: bump github actions to fix Node 20 deprecation warnings
2026-10-10 05:03:26 -04:00
Brandon Thomas 53b62a12ea Merge pull request #420 from txiki739/fix/flatpak-source-cli
#346: the from-source Flatpak includes the CLI and MCP server
2026-10-10 05:03:18 -04:00
Brandon Thomas 303b1518f6 Merge pull request #446 from LloydNicholson/fix/mp4-color-from-bitstream
Issue #552: MP4 and Matroska colour from the bitstream when the container has none
2026-10-10 05:03:11 -04:00
Brandon Thomas 11f8ac9718 Merge pull request #486 from MasakatsuFunaki/nvenc-rgba-input
M-HW: NVENC takes the export's RGBA and converts it on the GPU (#30)
2026-10-10 05:03:06 -04:00
Brandon Thomas 606c355644 Merge pull request #475 from MrCloudy2/fix/vaapi-shared-display
#474: VA-API decoders share one display (no 0.3 s open / 70 ms close per decoder on NVIDIA)
2026-10-10 05:03:00 -04:00
Brandon Thomas 325d1aff42 Merge pull request #555 from adurr21/fix/issue-150-light-theme-inconsistencies
Fix Light theme inconsistencies
2026-10-10 05:02:53 -04:00
Brandon Thomas 76ac514c5a Merge pull request #449 from haranishi/i18n-japanese-whole-interface
i18n: Japanese for the whole interface
2026-10-10 05:02:46 -04:00
Brandon Thomas 0c04e194f8 Merge pull request #450 from haranishi/fix-text-tab-width-cjk
Text panel: tab widths follow the drawn label (Japanese tabs overlapped)
2026-10-10 05:02:41 -04:00
Brandon Thomas fc1b9e2930 Merge pull request #488 from ReReddo/fix/i18n-source-scan-crlf
i18n: the tl! source scan in tests passes on CRLF checkouts
2026-10-10 05:02:35 -04:00
Brandon Thomas c83197e4a3 Merge pull request #516 from chuygarcia92/make-missing-clip-list-scrollable
Implement scrollable areas for media dialog
2026-10-10 05:02:30 -04:00
Brandon Thomas 4d3fe1bdb0 Merge pull request #506 from liangforstudy/fix/save-playhead
Project files: each sequence reopens with its playhead where it was left
2026-10-10 05:02:24 -04:00
Brandon Thomas 8e908d0550 Merge pull request #521 from maccagnit/newshape-placement-docs
graphics: shape commands document `seconds` / `track` / `time`, and placement errors name the command that asked
2026-10-10 05:02:17 -04:00
Brandon Thomas a3d2399ae9 Merge pull request #518 from Guilhermeb-Ferrarezi/ui/color-param-eyedropper
ui: add an eyedropper to colour parameters (Ultra Key, Color Key...)
2026-10-10 05:02:10 -04:00
Brandon Thomas c76d7931ca Merge pull request #514 from vbasky/feature/graphic-gradient
Fill graphic shapes and text with a linear gradient
2026-10-10 05:02:04 -04:00
Brandon Thomas f8464f1c96 Merge pull request #511 from liangforstudy/fix/484-enable-flips-each-clip
#484: Enable flips each selected clip on its own
2026-10-10 05:01:58 -04:00
Brandon Thomas 51cfa83ec7 Merge pull request #576 from ReReddo/feat/timecode-scrub-edit
timeline, monitors: drag the current-time timecode to scrub, click to type a time
2026-10-10 05:01:52 -04:00
Brandon Thomas 9f98495728 Merge pull request #487 from DWAA1660/fix/482-transition-drop-preview
Preview transition placement while dragging onto timeline clips (#482)
2026-10-10 05:01:45 -04:00
Brandon Thomas 339198fbdb Merge pull request #472 from ossini/fix/223-audio-in-video-clips
Audio of video clips: volume in Effect Controls + draggable volume line (#223)
2026-10-10 05:01:38 -04:00
Brandon Thomas ef37ad3b40 Merge pull request #275 from carlitosteso3/feat/audio-scrubbing-211
Audio plays while scrubbing (#211)
2026-10-10 05:01:32 -04:00
Brandon Thomas 9a3692f239 Merge pull request #257 from MrCloudy2/feat/edit-point-menu-219
#219: right-clicking a cut opens the edit point menu
2026-10-10 05:01:25 -04:00
Brandon Thomas fed3820692 Merge pull request #494 from sanyamsingh001/codex/time-current-main
Keep frame navigation precise and time arithmetic bounded
2026-10-10 05:01:18 -04:00
Brandon Thomas 751857d715 Merge pull request #464 from MarekSpodBiedry/issue-459
MCP render_frame: max_side 0 means no limit (#459)
2026-10-10 05:01:12 -04:00
Brandon Thomas 8e2df153f4 Merge pull request #556 from LloydNicholson/fix/caption-list-track
Issue #545: captions.list with a track that names no caption track is an error
2026-10-10 05:01:06 -04:00
Brandon Thomas 913b45cb19 Merge pull request #557 from maxm11/issue-536-bin-export
fix(export): Selection as Project exports a selected bin's contents (#536)
2026-10-10 05:00:59 -04:00
Brandon Thomas 2ded373cdb Merge pull request #453 from MrCloudy2/fix/hybrid-replay-keeps-pictures
#447: HybridDecoder fallback replay goes on past samples the software decoder rejects
2026-10-10 05:00:48 -04:00
Brandon Thomas ad0e0ef5d4 Merge pull request #454 from vbasky/feature/marker-csv-export
Export sequence markers to a spreadsheet-safe CSV
2026-10-10 05:00:42 -04:00
Brandon Thomas 204cc16754 Merge pull request #497 from digitaldecaycreations-lgtm/fix/windows-verbatim-pathurl
interchange: fix file:// URLs for Windows verbatim paths (FCP7 XML media offline in Premiere)
2026-10-10 05:00:37 -04:00
Brandon Thomas 629d554e0d Merge pull request #467 from sanyamsingh001/codex/audio-stream-essence
Keep selected audio streams distinct in AAF/OMF exports
2026-10-10 05:00:30 -04:00
Brandon Thomas acf4ca06f7 Merge pull request #419 from txiki739/fix/aaf-metadictionary
#323: AAF files open in readers that load the meta-dictionary
2026-10-10 05:00:24 -04:00
Brandon ThomasandClaude Opus 5.5 0423916667 Marker CSV export test: use egui_kittest's wgpu() builder (with_wgpu_render does not exist in our version)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 04:51:01 -04:00
Brandon Thomas d92f6b516d Merge remote-tracking branch 'origin/main' into fix-454 2026-10-10 04:51:01 -04:00
Nigel Ihza Firdausy fb41e56195 fix(i18n): prioritize Japanese system font ahead of Chinese fallback (#547) 2026-10-10 15:47:24 +07:00
Brandon Thomas 45936c1d6c Merge pull request #575 from MarekSpodBiedry/issue-548
Slide: keep reversed neighbours' source ranges correct (#548)
2026-10-10 04:44:26 -04:00
Brandon Thomas d55d742026 Merge pull request #574 from MarekSpodBiedry/issue-546
Text panel: word navigation ignores a selection the transcript no longer has (#546)
2026-10-10 04:44:18 -04:00
Brandon Thomas fbb45f794b Merge pull request #573 from MarekSpodBiedry/issue-544
MCC export: end each ancillary packet with its checksum byte so FFmpeg reads the captions (#544)
2026-10-10 04:44:09 -04:00
Brandon Thomas 135f42163d Merge pull request #572 from MarekSpodBiedry/issue-543
masks.addVertex: clamp oversized index on animated paths, reject empty paths instead of panicking (#543)
2026-10-10 04:44:01 -04:00
Brandon Thomas 3db99e6817 Merge pull request #571 from MarekSpodBiedry/issue-541
Audio Gain: measure the peak of the audio stream each clip plays (#541)
2026-10-10 04:43:52 -04:00
Brandon Thomas 58925adc45 Merge pull request #570 from MarekSpodBiedry/issue-540
Parametric Equalizer: use the full EQ so master gain, bands 1/2/4/5 and pass filters reach the audio (#540)
2026-10-10 04:43:44 -04:00
Brandon Thomas 4509e6e217 Merge pull request #566 from MarekSpodBiedry/issue-535
Project Manager: keep every parent of a nested subclip when collecting (#535)
2026-10-10 04:43:35 -04:00
Brandon Thomas 4bb600ee36 Merge pull request #565 from MarekSpodBiedry/issue-534
Forced relink and Replace Footage keep the new file's properties (#534)
2026-10-10 04:43:26 -04:00
Brandon Thomas b5a0e1dd78 Merge pull request #564 from MarekSpodBiedry/issue-532
Queue move: saturate a huge relative offset instead of overflowing (#532)
2026-10-10 04:43:18 -04:00
Brandon Thomas 1dba0532a5 Merge pull request #563 from MarekSpodBiedry/issue-531
Export settings: accept camelCase for nested snake_case fields (#531)
2026-10-10 04:43:08 -04:00
Brandon Thomas 9e2ebd696e Merge pull request #562 from MarekSpodBiedry/issue-530
Frame-held Regular In trim stops at the previous clip instead of overlapping it (#530)
2026-10-10 04:42:59 -04:00
Brandon Thomas de030de569 Merge pull request #561 from MarekSpodBiedry/issue-529
Linked slip: clamp to one shared delta so picture and sound stay aligned (#529)
2026-10-10 04:42:51 -04:00
Brandon Thomas dd28b12f1f Merge pull request #560 from MarekSpodBiedry/issue-528
Trim: extend each selected edit point to the playhead by its own distance (#528)
2026-10-10 04:42:43 -04:00
Brandon Thomas 0e11974976 Merge pull request #559 from MarekSpodBiedry/issue-527
fix(edit): reversed trims use the media handle their playback direction consumes (#527)
2026-10-10 04:42:34 -04:00
Brandon Thomas 764723f439 Merge pull request #558 from MarekSpodBiedry/issue-526
Roll: reversed clips take the cut from the opposite source end (#526)
2026-10-10 04:42:27 -04:00
Brandon Thomas a790ca05ee Merge pull request #568 from MarekSpodBiedry/issue-538
CLI export: forward a lone --start or --end so the engine reports the missing bound (#538)
2026-10-10 04:42:20 -04:00
Brandon Thomas 8a32432f9b Merge pull request #567 from MarekSpodBiedry/issue-537
CLI: a closed stderr no longer panics with exit 101 (#537)
2026-10-10 04:42:12 -04:00
Ikki NishiharaandClaude Sonnet 5.5 c34090f2d2 CFB: bound the DIFAT and FAT chain walks by the file's own size
The DIFAT guard came from the header's sector count and the chain guard from
the FAT length, both of which a file can inflate. A chain cannot have more
sectors than the file holds: bound both walks by that.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-10 17:34:16 +09:00
Ikki NishiharaandClaude Sonnet 5.5 c33cf33fe0 AAF: a weak-reference collection with a zero key size can't exhaust memory
An index that declares a key size of 0 made the importer loop `count` times
(a u32 from the file) pushing an empty key each time. Reject a non-empty
collection with key size 0, and require that count * size fits in the index
stream before the loop starts.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-10 17:29:44 +09:00
ReReddo ab9e4e6ceb timeline, monitors: drag the current-time timecode to scrub, click to type a time (#554)
The Timeline's timecode and the Program and Source monitors' current-time
timecodes were display-only. They now share one widget,
widgets::timecode_field, which replaces the unused timecode_label:

- Dragging scrubs that panel's playhead, 1 frame per point (Shift x10), and
  stops at the start (for the Source monitor, the clip's start). The pointer
  is hidden while dragging and put back where it was pressed.
- A click opens a text field with the whole value selected. Enter or clicking
  away goes to the typed time through parse_timecode, so `1000` and `+15`
  work. Escape cancels. An invalid time is reported in the status bar.
  Left/Right on the selected value start from its last digit.

The Timeline and Program monitor use playhead.set, the Source monitor
source.setPlayhead.

Tests: timecode_ui (5 tests). The ui-egui suite passes except the two
Windows CRLF failures fixed in #488.
2026-10-10 10:35:44 +03:00
Max Murphy 5b6e1ce720 fix(export): Selection as Project exports a selected bin's contents
Selecting a nonempty bin reported success with items:[bin] while writing
an empty project: closure() kept the bin id (which names no item) and
project_subset() then pruned everything. Expand a named non-root bin to
the items it holds, as project.delete does, so only real items are kept
and reported. An empty bin now returns an actionable error instead of a
misleading empty destination.
2026-10-10 03:19:38 -04:00
MarekSpodBiedry f684440ef5 Trim: extend each selected edit point to the playhead by its own distance (#528) 2026-10-10 09:16:50 +02:00
MarekSpodBiedry b40964db91 masks.addVertex: clamp oversized index on animated paths, reject empty paths instead of panicking (#543) 2026-10-10 09:16:34 +02:00
MarekSpodBiedry 8337c1656e Text panel: word navigation ignores a selection the transcript no longer has (#546) 2026-10-10 09:16:33 +02:00
MarekSpodBiedry 96ae4c6244 Export settings: accept camelCase for nested snake_case fields (#531) 2026-10-10 09:16:10 +02:00
MarekSpodBiedry d1b83b9c97 Linked slip: clamp to one shared delta so picture and sound stay aligned (#529) 2026-10-10 09:16:01 +02:00
MarekSpodBiedry 769ead74a8 Audio Gain: measure the peak of the audio stream each clip plays (#541) 2026-10-10 09:15:51 +02:00
MarekSpodBiedry 09d5473645 Forced relink and Replace Footage keep the new file's properties (#534) 2026-10-10 09:15:37 +02:00
MarekSpodBiedry 64007e93b6 CLI: a closed stderr no longer panics with exit 101 (#537) 2026-10-10 09:15:24 +02:00
MarekSpodBiedry 544f561587 MCC export: end each ancillary packet with its checksum byte so FFmpeg reads the captions (#544) 2026-10-10 09:15:03 +02:00
MarekSpodBiedry ffbe2622da CLI render: report the real reason for an invalid scale instead of no sequence (#539) 2026-10-10 09:14:50 +02:00
MarekSpodBiedry 948c791db8 Project Manager: keep every parent of a nested subclip when collecting (#535) 2026-10-10 09:14:47 +02:00
MarekSpodBiedry 09f0cf1cc7 Parametric Equalizer: use the full EQ so master gain, bands 1/2/4/5 and pass filters reach the audio (#540) 2026-10-10 09:14:46 +02:00
MarekSpodBiedry d23b33a9fe Frame-held Regular In trim stops at the previous clip instead of overlapping it (#530) 2026-10-10 09:14:41 +02:00
MarekSpodBiedry 7484343012 fix(edit): reversed trims use the media handle their playback direction consumes (#527) 2026-10-10 09:14:40 +02:00
MarekSpodBiedry 1ccb51fb78 Slide: keep reversed neighbours' source ranges correct (#548) 2026-10-10 09:14:35 +02:00
MarekSpodBiedry aff59f15ec Queue move: saturate a huge relative offset instead of overflowing (#532) 2026-10-10 09:14:21 +02:00
MarekSpodBiedry 0c66f5ffd3 Roll: reversed clips take the cut from the opposite source end (#526) 2026-10-10 09:14:03 +02:00
MarekSpodBiedry 8ee37150e4 CLI export: forward a lone --start or --end so the engine reports the missing bound (#538) 2026-10-10 09:13:54 +02:00
AustinandCodex 2c049fe893 ui-egui: fix Light theme chrome inconsistencies (#150)
Route audio controls, meters, effect graphs, timeline controls, caption lanes, and effect badges through theme tokens so Light mode no longer retains dark-only surfaces. Add palette regression coverage and a three-theme mixer harness test while preserving Dark and Medium colors.

Co-Authored-By: Codex <noreply@openai.com>
2026-10-10 03:03:03 -04:00
lloydandClaude Opus 5.5 8175b16a47 captions: captions.list with a track that names no caption track is an error
`captions.list` resolved `track` with track_param and treated an
unresolved selector the same as an omitted one, so a missing numeric id
(or a label past the last track, such as `C3` with two tracks) returned
every caption track with success. It now fails with "no such caption
track", as captions.export, setTrack, setStyle, deleteTrack and add do.
Omitting `track` still lists every track, and valid ids and C-labels
still return just their track.

Test: two caption tracks; each id and each label returns only its own
track; a missing id, `C3`, `C0` and a non-label string are errors.

Fixes #545

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 09:02:57 +02:00
hadmark 6732ad9c94 Add files via upload 2026-10-10 14:32:49 +08:00
hadmark f067b709de Add Simplified Chinese language support
Add Simplified Chinese to language menu
2026-10-10 14:17:39 +08:00
SilentYetiandClaude Opus 5.5 03a840d28b Linux: NVDEC hardware decoding for H.264 and HEVC
Adds a second Linux decoder backend, crates/platform/src/nvdec, that talks to
NVIDIA's driver directly (libnvcuvid + the CUDA driver, both loaded at run
time). It is registered in front of VA-API: what NVDEC declines still reaches
VA-API, then the software decoders. H.264 8-bit and HEVC 8/10-bit, 4:2:0
progressive.

NVIDIA's parser takes Annex B access units, so there is no host-side DPB.
Pictures are read back as NV12 / P016 through the existing biplanar path and
are bit-exact with the software decoders. The CUDA context is the one
nvenc::device already retains; it gains push/pop and copy-to-host. A seek
recreates the parser and keeps the decoder.

Also fixes tests/setting.rs, which expected the Media Foundation backend name
on Linux, and updates the Hardware decoding tooltip and its Spanish row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 23:17:11 -07:00
hadmark 9679157fdd Add Simplified Chinese language support
Register Chinese language
2026-10-10 14:14:17 +08:00
hadmark c17013fca9 Add files via upload
Add Chinese translation catalog
2026-10-10 14:08:41 +08:00
SawizzleandClaude Opus 5.5 918a64317d docs: drop the third-party link; the Windows path is the default install folder
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 21:50:49 -07:00
Tommaso MaccagniandClaude Opus 5.5 28cde3168d graphics: document shape placement params; placement errors name their command
graphics.newShape, newRectangle, newEllipse and newPolygon already take
`seconds`, `track` and `time` through place_video_clip, as graphics.newText
does. Their parameter docs (what command_list shows) and the table in
docs/graphics.md listed only position, size and clip; they now list all of
them and say that position is the shape's centre.

place_video_clip takes the id of the command that called it for its errors.
It named graphics.newText in every placement error (V3 is not free here),
whether a shape, a file, a template or an upgraded caption had asked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 06:34:24 +02:00
Tommaso MaccagniandClaude Opus 5.5 ecfa560ed3 engine: effects.setParam takes a choice by name and can write a keyframe
A choice parameter takes its option name as well as its index, matched
without case, as transition parameters and graphics.set already do.

"keyframe": true writes a keyframe at `time` and animates the parameter (the
stopwatch and the value in one step, like clipMixer.set). Without it nothing
changes: a parameter that is not animated gets a static value whatever the
time. A parameter that cannot be animated refuses the keyframe.

The result is {"keyframes": n} instead of null, so a caller can tell that a
value sent with a time became a static value (0) and not a keyframe.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 06:32:32 +02:00
MitchandClaude Sonnet 5.5 88c9ec1701 #171: preview where dragged files will land on the Timeline
While files are dragged over the window from the OS, the Timeline shows a
placeholder at the snapped drop point on the row under the cursor, labelled
with the file count. The cursor is polled each frame because the OS sends no
motion events during such a drag.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-10 14:25:58 +11:00
MitchandClaude Sonnet 5.5 b630643ec9 #171: read the OS cursor for drops from the file manager
winit on Windows sends no cursor events while files are dragged in, so the
pointer was stale and the drop was never placed on the timeline. The desktop
app now passes a host hook that reads the cursor from the OS (GetCursorPos),
which the UI maps into window points.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-10 14:21:49 +11:00
MitchandClaude Sonnet 5.5 a0708ff243 #171: files dropped onto the Timeline are placed there, not just imported
Dropping OS files on the Timeline panel now imports them and places them
(overwrite) at the pointer, snapped, chained end to end. Controlled by
Settings > Timeline > "Place files dropped onto the Timeline directly on
the Timeline" (timeline.dropImportToTimeline, default on). Drops elsewhere
stay a plain bin import.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-10 14:14:19 +11:00
Guilhermeb-Ferrarezi 3180e3d2f2 ui: format the eyedropper 2026-10-09 23:48:16 -03:00
Guilhermeb-Ferrarezi c2bde2e6fd ui: add an eyedropper to colour parameters
Colour parameters in Effect Controls (the Ultra Key key colour, Color Key, mattes...) get a pipette
button. Armed, a click in the Program monitor fills the parameter with the colour under the pointer
(5x5 average, so a noisy green screen gives its true green); Esc disarms.

The colour is read from a render of the sequence at the playhead with the armed effect switched off,
so an Ultra Key that already removed the green can still pick it. The value is stored the way the
colour picker beside it stores it, through effects.setParam (one undo step).
2026-10-09 23:43:20 -03:00
Krzysztof Cieślik b6c604c104 clip.link: pair video clips with overlapping audio clips (#507)
When linking a selection containing multiple video clips, create one link
per video clip with its overlapping audio clips rather than a single link
for all clips, matching Premiere Pro behavior.
2026-10-10 04:21:06 +02:00
Chuy Garcia f2c42f9864 Implement scrollable areas for media dialog
Added vertical scroll areas for media rows and candidates to improve UI usability.
2026-10-09 20:20:17 -06:00
Vikram Bhaskaran ac83bae223 Blend graphic gradients in linear light and keep per-character fills solid.
Stops were mixed in encoded sRGB, and any run with its own fill turned the whole layer into a solid colour. The ramp now blends in linear light, turns with the layer, and leaves a per-character fill solid.
2026-10-10 13:18:45 +11:00
liangforstudyandClaude Opus 5.5 fcaa6bbdf4 #484: Enable flips each selected clip on its own
With one enabled and one disabled clip selected, Clip > Enable (Shift+E) set every
selected clip to the same state: the first press enabled both, the next disabled both.
Premiere flips each clip separately (confirmed by the reporter in #484), so a mixed
selection swaps. Each clip in the selection (with its linked clips) now flips on its
own; a clip named twice in the parameters still flips once.

Test: enable_flips_each_selected_clip (fails before the fix: the mixed press left
both clips enabled).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq8BK1LmohzE1ga4NgAdWM
2026-10-10 09:23:38 +08:00
Vikram Bhaskaran d55ebe94e5 Fill graphic shapes and text with a linear gradient.
Appearance › Fill Type paints a ramp across the layer bounds, and the ramp turns with the layer. A per-character fill colour still paints those characters as a solid.
2026-10-10 12:05:03 +11:00
Brandon ThomasandClaude Opus 5.5 7bd7621262 Bump version to 0.5.0 (from 0.4.0) for release; refresh contributors
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 21:02:11 -04:00
Mara Johnson 71dd56ff8f fix(mcp): media_import schema mirrors file.import (#499) 2026-10-09 20:47:33 -04:00
swillunderscoreandClaude Opus 5.5 f61669d208 #224: dragging a transition's end trims that end only
Adobe's Change transition duration page shows the Trim-In / Trim-Out icon on a transition's end
in the Timeline: the drag trims that end, and the other stays. A centred transition dragged by one
end therefore becomes Custom Start, rather than growing on both sides. The alignment rule ("how
the transition changes length depends on the alignment") stays for typed durations: Effect
Controls' Duration and Set Transition Duration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 19:20:25 -05:00
Luniiya 77fa159f49 ui: standardise system appearance modes and theme previews 2026-10-10 02:14:55 +02:00
Dylan M. Taylor 75559a8f41 chore: bump github actions to fix Node 20 deprecation warnings 2026-10-09 20:08:27 -04:00
SawizzleandClaude Opus 5.5 1286f135f4 docs: register the MCP server from an installed release
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 17:06:15 -07:00
swillunderscoreandClaude Opus 5.5 efd62f7bda #430, #224: select a transition, edit it in Effect Controls, Delete it, drag its ends or middle
Timeline: clicking a transition selects it (outlined; Shift adds / removes) and opens it in Effect
Controls; Delete removes it. Its ends are edge zones: dragging one changes the duration (a Center
at Cut transition grows on both sides and stays centred, any other keeps its other end), dragging
its middle slides it over the cut (Custom Start); each drag is previewed and committed as one
sequence.setTransition. Double-clicking it opens Set Transition Duration.

Effect Controls for a selected transition: Duration (timecode), Alignment (Center at Cut, Start
at Cut, End at Cut; Custom Start shown when dragged off-centre), Reverse, its own settings
(numbers, angles, choices, colours, points, checkboxes), Reset. Dragging a value is one undo step.

Tests: transitions_ui.rs drives it all over the control channel (select, Effect Controls, the Delete
key, undo, edge and middle drags, the Alignment menu, the dialog, Push's settings), with
FILMCRAFT_UI_SNAPSHOT_DIR snapshots. Spanish for the two new strings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 19:04:21 -05:00
digitaldecaycreations-lgtmandClaude Opus 5.5 5e87784af8 interchange: strip Windows verbatim prefix in file URLs
`path_to_file_url` treated `\\?\C:\x` (the form `canonicalize` returns on
Windows) as a UNC share named `?`, so FCP7 XML exports on Windows wrote
`file://%3F/C:/...` and Premiere opened the media offline. Drop the
`\\?\` prefix (and map `\\?\UNC\server\share` to `//server/share`) before
building the URL. Verified: the exported XML now imports into Premiere
Pro 2026 on Windows with all media online.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 16:20:58 -07:00
sanyamsingh001andCodex a3ad7a8ce7 TIME.FRAME_NAVIGATION: Preserve exact frame steps and bound time arithmetic
Co-authored-by: Codex <codex@openai.com>
2026-10-10 09:00:34 +10:00
swillunderscoreandClaude Opus 5.5 d4d1830d99 #430: transitions can be selected, removed, resized and moved (engine)
- filmcraft_edit::transitions: where an applied transition may sit (over its cut, within the
  clips it joins; a fade to or from nothing keeps its clip edge), its alignment as Effect
  Controls shows it (Center at Cut / Start at Cut / End at Cut / Custom Start), set_span, remove,
  and drag_span for the Timeline (an end changes the duration, the middle slides it).
- EditorState::transition_selection; timeline.select {"transitions": [...]} replaces the clip,
  edit point and caption selections, and a clip selection clears it.
- edit.clear (Delete) removes the selected transitions when no clips are selected;
  sequence.removeTransition removes given or selected ones; Ripple Delete stays clip-only.
- sequence.setTransition also takes duration / frames (following the alignment: Center at Cut and
  Custom Start move both ends, Start at Cut the end, End at Cut the beginning), align, start, and
  merge / begin so a drag is one undo step. Out-of-range times are refused before any frame
  arithmetic, atomically.
- sequence inspection reports each transition's alignment and the transition selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 17:59:58 -05:00
MrCloudy2andClaude Opus 5.5 e90508f16f #288: MP4 / MOV pictures are cropped as the file says (H.264 / HEVC cropping, clean aperture)
The MP4 source reported the sample entry's size, only swapped for the display rotation, so two
crops were lost: the codec's own (H.264 SPS frame cropping, HEVC conformance window), which the
decoder applies but the reported size and sequences made from the clip didn't, and the clean
aperture (`clap`), parsed but never used. An iPhone clip (1920x1440 coded, 1744x1308 clean
aperture, -90 degree display matrix) showed 1440x1920 instead of 1308x1744.

H.264 / HEVC sizes now come from the stream's parameter sets; the clean aperture is a rectangle
inside that picture (ignored when it makes no sense), reported as the size and cut from every
frame by the GOP cache before the display rotation, through the new VideoFrame::cropped.

Tests: bit-exact against ffmpeg (which applies both crops) for an H.264 SPS crop, an HEVC
conformance window, centred and off-centre clean apertures and a clean aperture with a rotation;
clean apertures that make no sense are ignored without a panic; VideoFrame::cropped on every pixel
format with odd sizes and misaligned corners.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 00:50:03 +02:00
DWAA1660 87a01072d0 #482: Preview transition placement while dragging 2026-10-09 16:01:08 -06:00
user.email 5b3d2c2a0b linux: cargo fmt the dev-icon BadExecPath arm
The to_str() refinement landed unformatted: rustfmt wants the long
error arm braced.

🤖 Generated with [Kimi Code](https://github.com/storyteller/kimi-code)
Co-Authored-By: Kimi <noreply@storyteller.dev>
2026-10-10 01:00:30 +03:00
MasakatsuFunakiandClaude 5001d6bf29 M-HW: NVENC takes the export's RGBA and converts it on the GPU (#30)
8-bit NVENC sessions (H.265 Main, hardware H.264) now take the export's RGBA
frames as ABGR input buffers and NVENC converts them to BT.709 limited 4:2:0
on the GPU, with the same codes as rgba_to_yuv420_8. The CPU conversion ran
on the encode thread, where it competed with the render of the next frames
for the thread pool (about 13 ms per 1080p frame instead of 1.5 ms) and held
NVENC back. A driver that refuses RGB input falls back to the CPU conversion
into NV12. HEVC Main 10 is unchanged.

60 s of a 1080p30 project as H.265 in the app: 18.7 s -> 15.0 s (96 -> 120
fps); headless 30.3 s -> 16.8 s.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-09 23:49:24 +02:00
user.email 4955515bcd linux: dev desktop entry only for source-tree runs; both Exec layers; exact Exec match
Review feedback on #246:

- Gate the install to dev runs: a debug build, or a release run with
  FILMCRAFT_DEV_DESKTOP_ENTRY set (the documented cargo run --release flow);
  never when APPIMAGE or FLATPAK_ID is set — an AppImage mounts at a fresh
  /tmp/.mount_* each launch, so its Exec path changes every time, and packaged
  installs ship their own entry and must not write into the profile.
- Escape Exec for both desktop-entry layers exactly like
  packaging/linux/install.sh: shell layer first (` " $ \ get a backslash),
  then the file-value layer (every backslash doubled), always double-quoted.
  Binary paths with =, % or control characters are rejected instead of written
  (install.sh rejects them for the same reason).
- Compare the Exec path exactly: parse the first Exec argument through both
  layers instead of a substring contains, so /a/filmcraft-old no longer
  counts as ours and a stale entry is rewritten.
- eprintln -> log::warn for the skip message (the logging module landed meanwhile).

🤖 Generated with [Kimi Code](https://github.com/storyteller/kimi-code)
Co-Authored-By: Kimi <noreply@storyteller.dev>
2026-10-10 00:41:44 +03:00
ReReddo 83da21acd1 contributors: credit ReReddo
Add ReReddo's own entry to contributors/people.toml, display name only, as
AGENTS.md asks of contributors. The name is shown once the maintainers'
contributors.py run verifies it.
2026-10-10 00:34:26 +03:00
ReReddo 63fe547f70 i18n: the tl! source scan in tests passes on CRLF checkouts
spanish_translates_every_tl_literal cuts each source file at
"#[cfg(test)]\nmod tests". On a Windows checkout with core.autocrlf=true the
files have CRLF, so the cut never matched. The scan then read the test module,
including its own assertion text, and failed with "use a plain string literal
in tl! and tlf!".

Normalise CRLF to LF when reading the sources. Checkouts with LF endings
(Linux, macOS, CI) are unaffected.

Tests: cargo test -p filmcraft-ui-egui --lib passes (71 tests). The test failed
before this change on a Windows checkout.
2026-10-10 00:31:41 +03:00
ReReddo 73f2727ee4 appearance: View > Appearance names the middle theme "Dark", as Settings does (#471)
View > Appearance called the middle color theme "Medium" while Settings >
Appearance > Color Theme called the same choice "Dark". Use the Settings
name, which is also the stored value and the name in docs/project-files.md.

Only the menu label changes. Command ids (view.theme.dark/medium/light) and
stored values (darkest/dark/light) are as before. In pt-br.tsv the "Medium"
entry becomes "Dark" (Escuro), because that catalog may only hold menu labels.

Adds a settings_ui test that each View > Appearance label equals the matching
Settings dropdown label.

Tests: settings_ui and the i18n tests pass, except
i18n::tests::spanish_translates_every_tl_literal. It fails on Windows
checkouts with core.autocrlf=true (the source scan looks for "\n") and is
unrelated to this change.
2026-10-10 00:30:29 +03:00
liangforstudyandClaude Opus 5.5 72e3eeb5de Project files: each sequence reopens with its playhead where it was left
The engine already keeps one playhead per sequence while a project is open, and the
project file's view already stores each sequence's zoom, scroll and track heights, but
not its playhead: after saving and reopening, every sequence started at zero.

ProjectView gains `playheads` (sequence id -> Tick), written beside the tabs and views
and restored with them when "Restore open sequences when opening projects" is on. Files
written before it have none and open as they did. Values from a file are not trusted:
ids that are not sequences are dropped, playheads are clamped to 0..ProjectView::MAX_PLAYHEAD
and snapped to a frame of their sequence.

Tests: each_sequence_reopens_with_its_playhead_where_it_was_left (fails without the
fix), hostile playheads in a_damaged_view_in_a_project_file_is_cleaned_up_or_ignored,
a deleted sequence's playhead is not written, and the format round trip carries it.

Also adds my contributor credit entry (people.toml).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq8BK1LmohzE1ga4NgAdWM
2026-10-10 05:22:05 +08:00
MrCloudy2andClaude Opus 5.5 30ff11e669 #474: VA-API decoders share one display instead of opening one each
Every VA-API session opened its own display and terminated it when dropped. With NVIDIA's driver
that is a CUDA context each: 0.2-0.3 s to open, 60-70 ms to terminate and about 100 MB of video
memory, on the thread creating or dropping the decoder (a frame worker at every clip change, the
UI thread when media is removed: a ~0.6 s freeze deleting five played clips, and dropped frames at
clip changes).

Sessions now take the process's shared display (the startup probe's, kept for the life of the
process like the libraries) and make only their own configuration, context and surfaces. Display
is Sync on the strength of libva's Multithreading Guide (va.h): its functions and drivers are
thread-safe, and every session uses only its own VA objects. A session that cannot be made on the
shared display is tried once on a new display, which is shared from then on if it works (a GPU
reset).

On NVIDIA (RTX 3060): deleting five played clips 650-690 ms -> 132-198 ms, dropped frames in 20 s
of playback 2-3 -> 0, peak video memory 1281 -> 1010 MiB. Intel and software unchanged.

Test: tests/vaapi_displays.rs (decoders one after another and six at once on six threads open no
display; pictures bit-exact). The VA-API hardware suite passes on NVIDIA and Intel, and the
concurrency tests 20 times each.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 22:58:47 +02:00
sanyamsingh001andCodex 23f79890fd UI.SCALAR_PRESS_ORIGIN: Start scalar graph drags at the press position
Co-authored-by: Codex <codex@openai.com>
2026-10-10 06:31:59 +10:00
Oskar e9baec0154 #223: audio waveforms follow the clip volume
Like Premiere, the waveform in an audio clip now grows and shrinks with
its Volume level, keyframes included, so dragging the volume line down
visibly lowers the waveform. A bypassed or disabled Volume leaves it as
it was.
2026-10-09 22:28:41 +02:00
Oskar cf694f86b3 #223: volume tooltip keeps one width while dragging
The dB readout next to the pointer resized itself on every value change
and sometimes wrapped into a column of single characters. It is now one
line in a monospace font with a fixed number format, so the box stays
put.
2026-10-09 22:18:03 +02:00
user.email d8ef953f16 Merge branch 'main' into fix/linux-dev-desktop-icon 2026-10-09 23:08:04 +03:00
sanyamsingh001andCodex dd121aeb9c AUDIO.STREAM_ESSENCE: Preserve selected audio streams in AAF/OMF exports
Co-authored-by: Codex <codex@openai.com>
2026-10-10 05:53:14 +10:00
Oskar d6e43bcf2c docs: volume grab kind for ui.timeline.hit, clip_audio_ui tests (#223) 2026-10-09 21:40:02 +02:00
Oskar b4b4ea6317 #223: drag the clip volume line and its keyframes in the timeline
With the Selection or Pen tool the line on audio clips is now a grab
target of its own (Grab::Volume): dragging it up or down changes the
clip's Volume level live, as one undo step. With keyframes it moves the
two keyframes around the pointer, like Premiere. The Pen tool or a
Cmd/Ctrl-click adds a keyframe on the line; keyframes drag in time and
level and right-click deletes them.

A plain click on the line still selects the clip and a right-click
still opens the clip menu, since presses go through the timeline's own
hit-testing.
2026-10-09 21:39:41 +02:00
Oskar 44c462d43d #223: clip volume line follows its keyframes
The white line on audio clips was drawn at the level of the first frame
only, squeezed above the waveform, and not at all with waveforms off.
It now spans the clip under the name on the track fader scale, bends
with Volume keyframes and shows them as diamonds. Tracks showing a
track keyframe lane hide it.
2026-10-09 21:12:48 +02:00
MarekSpodBiedry c25cc5b18b FCP7 XML: keep the source channel of split mono audio tracks (#463) 2026-10-09 21:02:47 +02:00
MarekSpodBiedry 684c0a0d2f MCP render_frame: max_side 0 means no limit (#459) 2026-10-09 20:55:36 +02:00
Oskar 5fbc5cf8e5 #223: keyframe commands can merge a drag into one undo step
effects.setKeyframe and effects.moveKeyframe take merge/begin like
effects.setParam does, and moveKeyframe can set the value on the way
(same as mixer.moveKeyframe). The timeline volume line needs this to
update keyframes live while dragging without flooding the undo history.
2026-10-09 20:47:28 +02:00
Oskar 01a8eb90a6 #223: Properties panel shows the audio of a linked video clip
Same as Effect Controls: with a video clip and its linked audio selected
the Audio section (Level, Pan) now edits the audio clip instead of being
hidden behind Transform and Crop.
2026-10-09 20:44:33 +02:00
Oskar 7a922ab139 #223: Effect Controls shows the audio of a linked video clip
Selecting a video clip with linked selection on also selects its audio,
but the panel only listed the video effects. Now the linked audio clip's
Volume, Channel Volume and Panner follow under an Audio heading, like in
Premiere.
2026-10-09 20:43:22 +02:00
Vikram Bhaskaran c855c50ba9 Export sequence markers to a spreadsheet-safe CSV.
The report is sorted, uses the sequence timecode, escapes formula-like text, and does not change the edit.
2026-10-10 04:37:44 +11:00
MrCloudy2andClaude Opus 5.5 c724bc5566 #447: HybridDecoder fallback replay goes on past samples the software decoder rejects
After a hardware decoder failed, the replay of the run through the software decoder stopped at the
first sample the software decoder rejected (`?` in the replay loop): the pictures it had already
output were dropped, the rest of the run was never fed, the error was returned for the failed
sample even when the software decoder accepted it, and the replay log was not restored. That
happens when the GPU tolerated a damaged sample our decoder rejects and then fails later in the
same run; one picture went missing on a damaged HEVC file through VA-API (Intel).

The replay now behaves like software decoding from the restart point: a rejected sample is skipped
(logged when the hardware had already decoded it); if the failed sample itself is rejected its
error is this call's, and the pictures replayed before it are kept in `carry` for the next output.

Regression test (every platform, no GPU): a stand-in hardware decoder that decodes one damaged
sample the software decoder rejects and fails at that sample or later; every software picture
comes out once and in order, errors are reported where the software decoder reports them, and the
pictures from the next restart point are the software decoder's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 19:36:54 +02:00
Ikki NishiharaandClaude Sonnet 5.5 a123da6d0d Text panel: tab widths follow the drawn label
The Transcript / Captions / Graphics tabs were laid out at 7 px per
character. That suits Latin text, but Japanese glyphs are about twice as
wide, so with the interface in Japanese the labels ran into each other.

A tab is now as wide as its label is drawn when that is wider than the
old estimate, measured in the semibold the active tab uses so selecting a
tab does not move its neighbours. Latin labels keep their old widths.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-10 02:19:01 +09:00
Ikki NishiharaandClaude Sonnet 5.5 cf614c1fa3 i18n: Japanese for the whole interface
ja.tsv now has the same 3,466 entries as es.tsv (menus, panels, dialogs,
the Settings pages, the Effects registry, Keyboard Shortcuts, tooltips and
status messages) instead of 89 menu-only entries.

The 46 old rows that came from PhotoCraft/PDFCraft and match no FilmCraft
string are dropped. Five rows that were wrong for a video editor are
fixed: Horizontal/Vertical are 水平/垂直 (every use is geometry, not
writing direction), Orientation is 向き, Type is 種類 and Effect is
エフェクト. The other 38 rows that were already in use are unchanged.

ROADMAP.md and crates/ui-egui/README.md now list Japanese next to Spanish
as covering the whole interface.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-10 02:14:19 +09:00
lloydandClaude Opus 5.5 ff8728af08 Anamorphic media: pixel aspect ratio in placement, monitors and New Sequence From Clip
A 1440x1080 clip with 4:3 pixels showed squeezed: the demuxers read its
pixel aspect ratio, but nothing after import used it. New Sequence From
Clip made a 1440x1080 sequence with square pixels, the Motion transform
sized clips from storage pixels (pillarboxed at 1440 wide in a 1920x1080
sequence), and the monitors drew the frame at its storage aspect.

- project: checked_par / sane_par / par_ratio / conformed_size treat
  file and project ratios as hostile (a zero term, or more than 8:1
  either way, is square). MediaClip::pixel_aspect (Interpret Footage
  wins over the file), Project::source_par (subclips follow the parent,
  nested sequences have their own, graphics and adjustment layers have
  none) and Project::conformed_source_size.
- render: motion_matrix takes the source's ratio and works in square
  display units (anchor in source pixels, position in sequence pixels,
  scale and rotation at the display aspect); Scale to Frame Size fits
  the display size. CPU, GPU plan, adjustment regions and the mask /
  graphics overlays share it.
- engine: New Sequence From Clip, New Sequence from an item and
  multicam sequences copy the clip's ratio; default media scaling and
  Fit / Fill frame use the display size; generators made for a
  non-square sequence get its pixels; transcodes render pixel for
  pixel. clip.interpretFootage takes pixelAspect ([num, den] or "file").
- ui: the Source and Program monitors draw the picture at its display
  aspect; overlays map both axes; Sequence Settings shows the display
  aspect (16:9 for 1440x1080 at 4:3).

Tests: render par_tests (fill, GPU plan parity, Scale to Frame Size,
non-square sequences, rotation, Interpret Footage, hostile ratios),
engine par_tests (New Sequence From Clip, overwrite into 1920x1080,
default scaling, Fit / Fill, Interpret Footage with undo and refused
values, hostile file ratios, generators, and ffmpeg-made MP4 and MKV
files), project unit tests, and the headless par_ui monitor test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 18:28:56 +02:00
MrCloudy2andClaude Opus 5.5 67ed539fbe #219: translate the edit point menu
The interface became translatable since this branch started. The edit point menu now draws its
labels (and automation labels) in the interface language like the clip menu: the trim types
through i18n::t, Apply Default Transitions and Join Through Edits with tl! (both already in
the Spanish catalog). Spanish entries for the five trim types use the catalog's existing
vocabulary (recorte de rizo, edición de rodillo, recortar), and the trim types are listed with
the clip menu's labels in the test that requires Spanish for every such name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 18:12:58 +02:00
MrCloudy2andClaude Opus 5.5 f80b6bb002 Merge origin/main into feat/edit-point-menu-219
Conflict in the timeline's right-click handling (crates/ui-egui/src/panels/timeline.rs):
#220 now remembers the right-clicked clip so Unlink keeps only it selected, and this branch
opens the edit point menu on a clip's edge. Both are kept: the clip is remembered on every
right-click, and an edge also stores the edit point menu's target.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 18:12:49 +02:00
Cathy Shapiro 626df5ac86 WINCI: Repair upstream HEVC timing and CRLF translation validation
Delegate HEVC VUI timing to NVENC's configured frame rate instead of the
explicit timing pair reversed by the affected driver. Preserve existing
stream quality/timing assertions and cover integer and fractional rates.
Normalize line endings in the translation test's source scanner so it
excludes test modules in Windows checkouts as well as LF checkouts.

Assisted-by: Codex
2026-10-09 10:59:56 -05:00
lloydandClaude Opus 5.5 d7b7027f42 M8: MP4 and Matroska colour from the bitstream when the container has none
ffmpeg writes MP4 without a `colr` box unless asked (`+write_colr`), and
color_from only read `colr` (or vpcC / apvC), so every such file fell back
to BT.709 limited: an HEVC PQ file was detected as rec709 / hdr false, a
full-range H.264 file as limited range, and BT.601 SD with BT.709
primaries. Decoded frames were right (the decoders read the VUI); the
media info, HDR detection and Interpret Footage defaults were not.
Matroska without a `Colour` element had the same gap.

New `stream_color` reads the colour a codec configuration carries in its
bitstream with the decoders' own parsers: the VUI of the SPS in avcC /
hvcC and the color_config of the AV1 sequence header in av1C. Precedence:
`colr` (nclx / nclc) or Matroska `Colour`, then vpcC / apvC / the
bitstream, field by field (what the container leaves unspecified falls
through), then the existing size defaults (BT.601 matrix for SD, BT.709
otherwise). nclc has no range flag, so the bitstream's range now stands;
an ICC `colr` no longer hides the bitstream's code points. A truncated or
corrupt parameter set gives no colour and the defaults.

Tests build avcC / hvcC / av1C bit by bit: HEVC PQ → rec2100-pq, full-range
H.264 → Full, BT.601 SD (NTSC and PAL) primaries, unspecified VUI keeps the
defaults, colr wins, Matroska without Colour, and every truncation and
single-bit flip of each configuration under catch_unwind. An ffmpeg-oracle
test checks real x264 / x265 files against their `+write_colr` remux.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 17:58:49 +02:00
carlitosteso3andClaude Opus 5.5 2d520d7a2f #211: audio plays while scrubbing
"Play audio while scrubbing in Source and Program Monitors" (audio.scrubAudio,
Shift+S) was saved and toggled but never read. Now, while the playhead moves
with a mouse button down and playback is stopped, an 80 ms grain of the
sequence's sound (4 ms fades at its ends) plays from the new position.

It is detected once per frame in the app loop, so the Timeline ruler, the
monitors' scrub bars and dragged timecodes all scrub audibly. One output
stream stays open for the whole scrub and each move only re-aims it; it closes
shortly after the button is released, when the setting is off, or when
playback starts and takes the device.

Rebased on main, where Source monitor playback (source_playback) arrived:
it owns the audio device while it plays too, so a scrub stream still
lingering when Source playback starts no longer stops its sound on release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:29:23 -05:00
David Campos 89474eaeb5 project: share sequences across edit snapshots (scripted.rs)
Add test for shortcut dispatch in a single input frame.
2026-10-09 19:27:55 +04:00
David Campos 510156ea01 project: share sequences across edit snapshots (fixtures.rs) 2026-10-09 19:27:15 +04:00
David Campos 101d2686ed project: share sequences across edit snapshots (clip_ops.rs) 2026-10-09 19:26:54 +04:00
David Campos e53ac7eff9 project: share sequences across edit snapshots (lib.rs) 2026-10-09 19:26:27 +04:00
David Campos 5b01f8720f project: share sequences across edit snapshots (sequence_extras.rs) 2026-10-09 19:26:06 +04:00
David Campos 5f6c57984a project: share sequences across edit snapshots (lib.rs) 2026-10-09 19:25:46 +04:00
David Campos 7a719fe411 project: share sequences across edit snapshots 2026-10-09 19:25:06 +04:00
Brandon Thomas 7b6c134472 Merge pull request #363 from jerrykimo/fix/chinese-ui-glyphs
Fix Chinese glyph rendering in native timeline and UI labels
2026-10-09 11:02:46 -04:00
Brandon ThomasandClaude Opus 5.5 12f2ea17d6 CJK fonts: Chinese faces after the Japanese ones, so Japanese keeps Japanese glyph forms
The two share code points; putting the Chinese faces first drew Japanese UI text's kanji
with Chinese shapes. Also clears the test's texture deltas (epaint 0.36 panics on
dropping them unapplied).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:02:38 -04:00
Brandon Thomas e9c09d243a Merge remote-tracking branch 'origin/main' into fix-363 2026-10-09 10:56:40 -04:00
Brandon Thomas cb03d34c15 Merge pull request #361 from wisegamesyt-droid/fix-multitrack-audio
feat(media, engine): implement multi-stream container audio support and schema v13
2026-10-09 10:56:24 -04:00
Brandon Thomas e06018e07b Merge remote-tracking branch 'origin/main' into fix-361 2026-10-09 10:53:43 -04:00
Brandon ThomasandClaude Opus 5.5 fc32a19bd1 Proxies with fewer audio streams keep the proxy picture, audio from the original
Rejecting the whole proxy lost proxy playback for every multi-track recording
(and video-only proxies of media with audio); only the audio needs the original.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:53:42 -04:00
Brandon Thomas ef6391df90 Merge pull request #428 from storytold/fix-clippy-as-chunks-2
CI: gpu Push test uses as_chunks (clippy on Rust 1.99)
2026-10-09 10:38:46 -04:00
Brandon ThomasandClaude Opus 5.5 1f5ee123b2 CI: gpu Push test uses as_chunks (clippy chunks_exact_to_as_chunks on Rust 1.99)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:38:39 -04:00
Brandon Thomas d19e1b2783 Merge remote-tracking branch 'origin/main' into fix-361 2026-10-09 10:34:54 -04:00
Brandon Thomas ade68437d7 Merge pull request #269 from carlitosteso3/feat/new-sequence-dialog
New Sequence… opens the Sequence Settings dialog as New Sequence (stacked on #422)
2026-10-09 10:33:02 -04:00
Brandon ThomasandClaude Opus 5.5 3cc03f1447 Source playback: has_audio() for multi-stream MediaInfo
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:22:48 -04:00
carlitosteso3andClaude Opus 5.5 f0e1cfcec5 New Sequence… opens the Sequence Settings dialog as New Sequence
File ▸ New ▸ Sequence… (Cmd+N) and the Project panel's New Item ▸ Sequence…
made a sequence with the default settings at once. They now open the
Sequence Settings dialog (#355) as New Sequence (`open_new`): the General
tab from the default settings, named like file.newSequence would name it,
and a Tracks tab with the number of video and audio tracks. OK runs
file.newSequence with them; drop-frame timecode and Maximum Render Quality,
which the command has no parameter for, follow with sequence.settings when
they aren't the defaults. Making a sequence from a clip, or by dropping one
on an empty timeline, still passes params and stays direct.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:22:47 -05:00
carlitosteso3andClaude Opus 5.5 632420a01e Sequence Settings: Sequence Name on the General tab
Follow-up to #355, as suggested in #225: the dialog's General tab gets a
Sequence Name field, filled with the sequence's name when it opens. OK
sends `name` to `sequence.settings` (which already accepts it) only when it
changed, so renaming is part of the dialog's single undo step; a blank name
keeps the old one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:21:45 -05:00
Tom 1eaeac9b1f Windows: skip AV1 oracle tests when ffmpeg lacks libsvtav1/libdav1d 2026-10-09 15:19:08 +01:00
Brandon Thomas b32d1cd5ed Merge remote-tracking branch 'origin/main' into fix-361 2026-10-09 10:17:58 -04:00
Brandon ThomasandClaude Opus 5.5 850ac4342f CLI test: probe reports audio as audio_streams (schema v13 MediaInfo)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:17:40 -04:00
Brandon Thomas 9c43a0d02f Merge pull request #425 from storytold/pr/353-monitor-export-frame
M3.12: Export frames from Source and Program monitors (#353 by @cshapiro)
2026-10-09 10:14:15 -04:00
Brandon Thomas 81eb842e2e Merge remote-tracking branch 'origin/main' into fix-353 2026-10-09 10:13:59 -04:00
Brandon Thomas a880fa9154 Merge pull request #386 from storybored379/feat/nvenc-export
platform: enable existing NVENC H.264 export on Linux
2026-10-09 10:13:43 -04:00
Brandon Thomas 38f4255735 Merge remote-tracking branch 'origin/main' into fix-386 2026-10-09 10:13:01 -04:00
Brandon ThomasandClaude Opus 5.5 f267a941fb Merge origin/main into feat/nvenc-export
main gained NVENC H.265 Main / Main 10 (#328, #360), export stage timings
(#370) and VA-API decoding (#358). Resolution: main's NVENC session, FFI and
export factory, with this PR's changes ported onto them:
- ffi: the portable Guid (now also for the HEVC GUIDs), NV_ENC_DEVICE_TYPE_CUDA;
- session: device.rs (Direct3D 11 / retained CUDA primary context), the
  Linux libnvidia-encode.so.1 loader, Device::KIND;
- export: factory_with and its tests, the even-size check, the saturating
  max bitrate; H.265 through NVENC stays Windows-only (the factory declines
  it on Linux, where nothing probed or tested it);
- Cargo.toml: main's Linux libloading dependency covers both users;
- docs: README / ROADMAP / architecture now say NVENC H.264 runs on Windows
  and Linux; ADR and platform README keep both addenda / test rows;
- abi_tests: note why the MSVC-generated expectations hold on LP64 Linux.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:12:46 -04:00
Brandon Thomas 5f4e2d7c99 Merge remote-tracking branch 'origin/main' into fix-353
# Conflicts:
#	crates/engine/src/keyboard.rs
#	crates/engine/src/lib.rs
#	crates/ui-egui/src/panels/monitor.rs
2026-10-09 10:11:15 -04:00
Brandon Thomas ab06c110d1 Merge pull request #423 from storytold/pr/338-source-playback
M3.12: Source playback, clip dragging and monitor range editing (#338 by @cshapiro)
2026-10-09 10:10:34 -04:00
Brandon Thomas 560355f3d2 Merge remote-tracking branch 'origin/main' into fix-338 2026-10-09 10:09:57 -04:00
Brandon Thomas 666e891538 Merge pull request #370 from SaymonMaster14/m8/export-stage-timings
Export: stage timings in perf.stats, and render the next batch while encoding
2026-10-09 10:08:49 -04:00
Brandon ThomasandClaude Opus 5.5 9526ba50e3 M3.12: Spanish strings for Source playback controls
#238's i18n landed on main; its catalog-coverage tests need Spanish for
the new Loop marked range checkbox and the Source play/stop command names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:08:42 -04:00
Brandon ThomasandClaude Opus 5.5 285e81b5da Merge origin/main into m8/export-stage-timings
NVENC export: keep main's Main 10 / Main dispatch and time both colour
conversions (rgbf_to_yuv420_10, rgba_to_yuv420_8) as Stage::Convert.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:08:35 -04:00
Brandon ThomasandClaude Opus 5.5 bb4c6d0bb2 Format: repair out-of-range audio stream data on load instead of refusing the project
A clip with audio_stream >= MAX_AUDIO_STREAMS is reset to stream 0 and media
with more than MAX_AUDIO_STREAMS streams keeps the first ones, each with a
warning: one bad index must never lose a whole project.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:08:10 -04:00
Brandon ThomasandClaude Opus 5.5 701d7c7111 Merge origin/main into multi-stream audio
Keeps #359's contiguous_starts audio timing, now per stream in Mp4Audio /
MkvAudio (starts and preroll in source sample frames). MP4 audio streams and
their decoders are built from the same (track, info) pairs so they stay
index-aligned. Adapts new main code to audio_streams / TrackItem::audio_stream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:08:09 -04:00
txiki739andClaude Opus 5.5 ccaa7212c1 #323: AAF files open in readers that load the meta-dictionary
The Root's MetaDictionary was written with no properties. pyaaf2 loads
its ClassDefinitions and TypeDefinitions sets to merge them with its
built-in model and refused every file: "cannot find index stream:
ClassDefinitions-3 index".

The MetaDictionary now has both sets (pids 0x0003, 0x0004, keyed by the
definition AUID 0x0005), empty: they only carry extensions to the
baseline model and everything FilmCraft writes is baseline. An empty
strong-reference set also writes its real key size (16 for AUIDs, 32
for MobIDs) instead of 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 16:06:49 +02:00
Brandon Thomas e196de8f64 Merge remote-tracking branch 'origin/main' into fix-338
# Conflicts:
#	ROADMAP.md
#	crates/engine/src/commands.rs
#	crates/engine/src/lib.rs
#	crates/ui-egui/README.md
#	crates/ui-egui/src/panels/monitor.rs
2026-10-09 10:06:01 -04:00
Brandon Thomas 95230b9072 Merge pull request #360 from SaymonMaster14/m3/nvenc-hevc-main10
Windows: NVENC H.265 Main 10 HDR (PQ / HLG) export
2026-10-09 10:05:10 -04:00
Brandon ThomasandClaude Opus 5.5 c9e2bfcac0 Merge origin/main into m3/nvenc-hevc-main10
Export panel: the HEVC profile labels go through tl! with Spanish entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:04:58 -04:00
Brandon Thomas 95e585d989 Merge pull request #418 from storytold/pr/337-windows-validation
WINCI: Fix Windows paths and portable validation (#337 by @cshapiro)
2026-10-09 10:04:57 -04:00
Brandon Thomas d84fdc1268 Merge remote-tracking branch 'origin/main' into fix-337 2026-10-09 10:04:46 -04:00
Brandon Thomas 9a61c7610b Merge pull request #308 from WavJaby/perf/shared-transitions
perf(gpu): composite Push transitions on the shared GPU path
2026-10-09 10:04:24 -04:00
Brandon ThomasandClaude Opus 5.5 faa4bf953d Render plan: restore the detailed module doc and add the Push / PlanStep paragraph
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:03:38 -04:00
Brandon Thomas a6ac2d7eff Merge remote-tracking branch 'origin/main' into fix-308
# Conflicts:
#	crates/ui-egui/src/perf.rs
2026-10-09 10:01:25 -04:00
Brandon ThomasandClaude Opus 5.5 afd5a6b347 M3: NVENC Main 10 review fixes: saturating plane sizes, as_chunks for clippy
encode / encode_10 compute the plane sizes they check with saturating_mul;
fill_p010, rgbf_to_yuv420_10 and the HDR export test use as_chunks instead of
chunks_exact with a constant size (clippy::chunks_exact_to_as_chunks).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:01:08 -04:00
Brandon Thomas 3931562101 Merge pull request #238 from txiki739/i18n-spanish-panels
i18n: Spanish for the whole interface, catalog files and a remembered language
2026-10-09 10:00:38 -04:00
Brandon ThomasandClaude Opus 5.5 a66be150e3 WINCI: ignore the mixer throughput benchmark only where it times wall clock
On Unix the benchmark measures the test thread's CPU time, so suite load
does not move it and it keeps running. Without a per-thread CPU clock
(Windows) it falls back to wall time, which is load-dependent and flaky on
shared CI runners; ignore it only there and document how to run it.
The GPU effect-source hunks were dropped in the merge in favour of #376's
fs_fx_source, now on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:59:45 -04:00
Brandon Thomas 156535a6e4 Merge origin/main into i18n-spanish-panels
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# Conflicts:
#	crates/codecs/tests/mpeg_oracle.rs
2026-10-09 09:59:28 -04:00
Brandon Thomas efa89a7d2d Merge remote-tracking branch 'origin/main' into fix-337
# Conflicts:
#	crates/gpu/src/fx_tests.rs
2026-10-09 09:59:26 -04:00
Brandon Thomas 4d3a814968 Merge pull request #376 from sanyamsingh001/codex/gpu-effect-source
GPU.EFFECT_SOURCE: preserve exact texel centers for effect inputs
2026-10-09 09:59:15 -04:00
Brandon Thomas ef46f64a28 Merge pull request #375 from WavJaby/perf/web-yuv-frames
perf(web): feed WebCodecs YUV planes to the shared compositor
2026-10-09 09:58:35 -04:00
Brandon Thomas 1dea07a85a Merge remote-tracking branch 'origin/main' into fix-376 2026-10-09 09:58:16 -04:00
Brandon Thomas 9c24f4419c Merge pull request #368 from sanyamsingh001/codex/linux-validation-baseline
LINUX.VALIDATION: honor preset shortcuts and retain every MPEG oracle timestamp
2026-10-09 09:58:10 -04:00
Brandon ThomasandClaude Opus 5.5 183884fd34 web: WebCodecs busy check honours STALE_MS; checked row offsets in plane()
A copyTo that never settles kept copying > 0 and bypassed the stale timeout,
wedging the session; now busy requires recent output. plane() computes each
row's start/end with checked arithmetic and bounds it by the validated end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:58:06 -04:00
Brandon Thomas 6893b19314 Merge remote-tracking branch 'origin/main' into fix-368
# Conflicts:
#	contributors/people.toml
2026-10-09 09:57:52 -04:00
Brandon ThomasandClaude Opus 5.5 de1320dc88 Merge origin/main into i18n-spanish-panels
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:57:17 -04:00
Brandon ThomasandClaude Opus 5.5 82cb6fbc74 i18n: Spanish for strings added on main (Sequence Settings, keyframe navigator, export)
Translate the new Sequence Settings dialog (labels, sections, tabs,
values, hints), the shared keyframe navigator's tooltips, the
non-macOS tool and mask hints, Include Alpha Channel and the export
status. Update the hardware decoding description to main's wording.
Automation ids and labels stay English.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:57:13 -04:00
Brandon Thomas 67634241c8 Merge remote-tracking branch 'origin/main' into fix-375 2026-10-09 09:56:15 -04:00
Brandon Thomas 0d2c21163c Merge pull request #299 from dcarnelutti/export/trim-pool-when-done
export: free the frame pool when an export ends
2026-10-09 09:56:07 -04:00
Brandon ThomasandClaude Opus 5.5 79a3b623fe Pool trim: free only the float shelf, and only after a standalone export
pool::trim() from Pipeline::drop flushed the whole process-wide pool after every
render-preview segment and proxy, and threw away playback's U8/U16 planes too.
Now Pipeline::drop calls pool::trim_f32() only when !settings.part_of_batch.
PoolStats gains f32_idle_bytes so the tests can check the float shelf alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:55:47 -04:00
Brandon Thomas 584f742bf4 Merge pull request #247 from switch87/readme-gentoo
README: install on Gentoo from the ::snakebyte overlay
2026-10-09 09:55:23 -04:00
Brandon ThomasandClaude Opus 5.5 7050055a58 README: move the Gentoo overlay into Downloads > Linux
The Gentoo paragraph sat after the build-from-source and control-protocol
notes; it belongs with the other Linux install options. Mark it clearly as
community-maintained.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:54:46 -04:00
Brandon Thomas c21c86089f Merge remote-tracking branch 'origin/main' into fix-247
# Conflicts:
#	contributors/people.toml
2026-10-09 09:52:32 -04:00
Brandon Thomas 6ebc21f093 Merge remote-tracking branch 'origin/main' into fix-360
# Conflicts:
#	README.md
#	ROADMAP.md
#	crates/platform/README.md
2026-10-09 09:46:48 -04:00
Brandon Thomas 18f5bad7a8 Merge pull request #328 from SaymonMaster14/m2/nvenc-hevc
Windows: NVENC H.265 (HEVC) Main hardware encoding for Export
2026-10-09 09:46:13 -04:00
Brandon Thomas 97a8eaffec Merge remote-tracking branch 'origin/main' into fix-299
# Conflicts:
#	docs/control-protocol.md
2026-10-09 09:45:27 -04:00
Brandon Thomas 66a74afc38 Merge pull request #365 from DWAA1660/fix/windows-dx12-startup
Fix #179: leave OpenGL out of the Windows wgpu instance (Vulkan | DX12 by default)
2026-10-09 09:45:03 -04:00
Brandon ThomasandClaude Opus 5.5 b3f2a89944 Windows: default to Vulkan | DX12 | Metal (drop only GL) instead of forcing DX12
Forcing DX12 compiles shaders through FXC, which rejects fx.wgsl (silent CPU
fallback), and leaves machines without DX12 with no backend. Leaving GL out is
what avoids the AMD atio6axx.dll crash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:44:20 -04:00
Brandon Thomas aff136ff67 Merge remote-tracking branch 'origin/main' into fix-365
# Conflicts:
#	apps/filmcraft/src/main.rs
2026-10-09 09:44:20 -04:00
Brandon ThomasandClaude Opus 5.5 cc5e3c1065 Revert shortcut-preset changes in favour of #337's approach
The maintainers chose #337's handling of Ctrl/Cmd shortcut collisions off
macOS (drop_control_collisions / audit_for / builtin_for), which audits the
collisions and keeps documented winners rather than keeping the first listed
binding. This reverts crates/engine/src/shortcuts.rs and its tests to main
so the two approaches don't overlap; the MPEG-oracle ffprobe-JSON change and
the frame-pool test rewrite from this PR are kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:40:15 -04:00
Brandon Thomas 1b4b6fbffa Merge remote-tracking branch 'origin/main' into fix-368
# Conflicts:
#	contributors/people.toml
2026-10-09 09:40:15 -04:00
Brandon Thomas 1879d472e1 Merge pull request #417 from storytold/fix-clippy-as-chunks
CI: gpu test uses as_chunks (clippy on Rust 1.99)
2026-10-09 09:38:52 -04:00
Brandon ThomasandClaude Opus 5.5 7a29eb4dff CI: gpu test uses as_chunks (clippy chunks_exact_to_as_chunks on Rust 1.99)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:38:45 -04:00
txiki739andClaude Opus 5.5 17921200d7 #346: the from-source Flatpak includes the CLI and MCP server
The release bundle manifest installs filmcraft and filmcraft-cli; the
from-source manifest built and installed only the desktop app, so a
Flatpak built from source had no headless CLI or stdio MCP server.

It now builds -p filmcraft -p filmcraft-cli and installs
/app/bin/filmcraft-cli. The default command stays filmcraft; the CLI
runs with `flatpak run --command=filmcraft-cli ai.storyteller.filmcraft`
(the manifest header and docs/releasing.md show it). Runtime, SDK and
finish-args are unchanged.

packaging-lint now also checks that both manifests install the same
programs, {filmcraft, filmcraft-cli}; it fails on the old manifest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 15:37:09 +02:00
Brandon Thomas 570d06d5bd Merge remote-tracking branch 'origin/main' into fix-328
# Conflicts:
#	Cargo.lock
#	README.md
#	ROADMAP.md
#	crates/platform/Cargo.toml
#	crates/platform/README.md
#	docs/architecture.md
2026-10-09 09:34:12 -04:00
Brandon ThomasandClaude Opus 5.5 c497b648f5 Merge origin/main into i18n-spanish-panels
Resolve conflicts: keep main's macOS/other hint text and mask vertex
tooltip variants (each wrapped in tl!), take main's shared keyframe_nav
helper and translate its tooltips.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:34:01 -04:00
Brandon ThomasandClaude Opus 5.5 33f3656071 #262: drop new apps/filmcraft/README.md; explain file filters in module docs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:33:38 -04:00
Brandon Thomas 45e951ff61 Merge remote-tracking branch 'origin/main' into fix-390
# Conflicts:
#	apps/filmcraft/src/main.rs
2026-10-09 09:33:32 -04:00
Manuel CamachoandClaude Opus 5.5 7b73c8e887 GPU3d: GPU export rendering is Auto by default (#30)
Merlin's effects-heavy bench scene (GPU3c) settles it: with layers and
effects the GPU compositor exports 2.4x faster at 1080p (13.0 -> 5.0-5.6 s,
737 -> 238 CPU ms/frame) and 3.2x at 4K (38.7-39.3 -> 12.0 s, 2955 -> 775
CPU ms/frame); a single plain clip takes the same time either way
(decoding / encoding bound). Every Auto row rendered all its frames on the
GPU, without fallbacks. The CPU reference renderer stays the fallback for
anything the GPU stage does not run.

Trade-off: the GPU matches the CPU within the compositor's parity
tolerance, not bit for bit, so an export's bytes can depend on the
machine's GPU; Off keeps them byte-reproducible (the determinism tests
register no GPU renderer and are unaffected).

- settings: GpuRendering::Auto is the #[default]; ExportSettings::default.
- test: the default, settings saved without the field, and "off".
- docs/performance.md: GPU3 results (1080p and 4K).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 06:31:57 -07:00
Brandon Thomas d63df6882b Merge pull request #256 from wstock/fix/245-ctrl-chords-control
control: Ctrl+ chords through ui.key fire shortcuts off macOS (#245)
2026-10-09 09:31:39 -04:00
Brandon Thomas a739ffea84 Merge remote-tracking branch 'origin/main' into fix-256
# Conflicts:
#	crates/ui-egui/tests/keyboard_ui.rs
#	docs/control-protocol.md
2026-10-09 09:29:27 -04:00
Brandon Thomas e5214ab7eb Merge pull request #355 from swillunderscore/sequence-settings-dialog
Sequence ▸ Sequence Settings… opens Premiere's dialog (#172, #287)
2026-10-09 09:28:59 -04:00
Brandon Thomas afe35e4c41 Merge pull request #240 from WavJaby/fix/web-import-collision
fix(web): preserve distinct same-name media imports
2026-10-09 09:28:53 -04:00
Brandon Thomas de0299bd1b Merge remote-tracking branch 'origin/main' into fix-240
# Conflicts:
#	apps/filmcraft-web/tests/smoke.mjs
2026-10-09 09:27:59 -04:00
Brandon Thomas 96f76bac2a Merge remote-tracking branch 'origin/main' into fix-355
# Conflicts:
#	contributors/people.toml
2026-10-09 09:27:39 -04:00
Brandon Thomas c2f2018707 Merge pull request #359 from IppSec/fix/236-audio-crackle
#236: audio in MKV and OBS-remuxed MP4 plays without crackling
2026-10-09 09:27:34 -04:00
Brandon Thomas 6e80bab6b8 Merge remote-tracking branch 'origin/main' into fix-359
# Conflicts:
#	contributors/people.toml
2026-10-09 09:26:08 -04:00
Brandon Thomas 0bae537ad0 Merge pull request #300 from dcarnelutti/platform/warm-hevc-probe
platform: ask for the hardware HEVC encoder in the background at startup
2026-10-09 09:25:32 -04:00
Brandon Thomas 22d2046c0a Merge pull request #270 from Tom1tk/windows-modifier-hints
Windows: platform modifier names in status-bar and mask hints
2026-10-09 09:25:26 -04:00
Brandon Thomas 9736b2a6ba Merge pull request #264 from carlitosteso3/fix/recovery-retry-197
#197, #193: web recovery retries failed snapshot writes and keeps skipped snapshots dirty
2026-10-09 09:25:20 -04:00
Brandon Thomas 48e0b65110 Merge pull request #237 from wstock/fix/230-wasm-clock
#230: jobs time themselves with web_time, so Create Proxies and mask tracking don't panic on the web
2026-10-09 09:25:14 -04:00
Brandon Thomas 202d5f061c Merge pull request #242 from wstock/fix/160-png-alpha
export: keep alpha in PNG/TIFF sequences (Include Alpha Channel) (#160)
2026-10-09 09:25:08 -04:00
Brandon Thomas 5cca829c39 Merge pull request #369 from storybored379/fix/mcp-cancel-numbered-movies
MCP: cancelling a movie export keeps numbered files beside it
2026-10-09 09:25:03 -04:00
Brandon Thomas 0409fdccf4 Merge pull request #372 from storybored379/fix/graphic-duration-placement
graphics: keep duration separate from timeline placement
2026-10-09 09:24:58 -04:00
Brandon Thomas d1f491823c Merge pull request #373 from storybored379/fix/audio-gain-result
mixer: report only the audio gains actually applied
2026-10-09 09:24:51 -04:00
Brandon Thomas bf0f2bd8ff Merge pull request #305 from Pathos0925/fix/296-mono-loudness
#296: Auto-Match measures a mono clip as one channel
2026-10-09 09:24:45 -04:00
Brandon Thomas bf21ee86c2 Merge pull request #282 from Tom1tk/windows-export-paths
Windows: export paths expand ~, accept trailing \ and verbatim paths
2026-10-09 09:24:38 -04:00
Brandon Thomas 996ccf72fd Merge pull request #349 from rossaai/rossa/embedding-hooks
Embedding: set_context on the web app, header and status bar flags, status bar colour token
2026-10-09 09:24:31 -04:00
Brandon Thomas 7cf322f19d Merge pull request #319 from WavJaby/fix/yuv-code-range
fix(gpu): normalize YUV code ranges independently of texture scaling
2026-10-09 09:24:25 -04:00
Brandon Thomas 275abb3193 Merge pull request #385 from SaymonMaster14/m4c/background-decoders
Thumbnails decode with small decoders that are not kept (#55)
2026-10-09 09:24:19 -04:00
Brandon Thomas cae8eb9f4f Merge pull request #381 from SaymonMaster14/m4/memory-stress
Decode caches: navigation bursts no longer keep every clip's decoder and frames (#55)
2026-10-09 09:24:13 -04:00
Brandon Thomas ae79acad8a Merge pull request #241 from WavJaby/fix/import-bin
fix(import): use the open Project bin as destination
2026-10-09 09:24:07 -04:00
Brandon Thomas 16ba523625 Merge pull request #253 from wstock/fix/244-project-delete-named-bins
engine: project.delete takes explicit bin ids without a Project-panel selection (#244)
2026-10-09 09:24:02 -04:00
Brandon Thomas 20ece5313e Merge pull request #261 from wstock/fix/243-delete-key
shortcuts: Delete clears the selected timeline clips (#243)
2026-10-09 09:23:56 -04:00
Brandon Thomas e7fe8f46b7 Merge pull request #250 from MrCloudy2/fix/unlink-keeps-clicked-clip-220
#220: Unlink leaves only the right-clicked clip selected
2026-10-09 09:23:49 -04:00
Brandon Thomas 5d69487925 Merge pull request #307 from justjohn12345/pr/bugfix
Keyframes: Properties gets Premiere's navigator, Effect Controls a time ruler
2026-10-09 09:23:44 -04:00
Brandon Thomas 354bb86332 Merge pull request #304 from justjohn12345/pr/monitor-follows-drags
Program monitor: the picture follows a value that is dragged
2026-10-09 09:23:37 -04:00
Brandon Thomas 55b6c0a916 Merge pull request #221 from carlitosteso3/fix/playhead-controls-208
Go to In / Go to Out move to the sequence start / end without marks (#208)
2026-10-09 09:23:31 -04:00
Brandon Thomas 929e05d996 Merge pull request #303 from MarekSpodBiedry/issue-139
Issue #139: portable.txt next to the exe keeps the data directory beside it
2026-10-09 09:23:25 -04:00
Brandon Thomas 0f3343cc4a Merge pull request #277 from midasdf/fix/vertical-japanese-punctuation
M10: use Japanese vertical forms and origins for titles
2026-10-09 09:23:20 -04:00
Brandon Thomas 298391574e Merge pull request #289 from midasdf/fix/vertical-grapheme-cells
M10: keep combining marks in one vertical text cell
2026-10-09 09:23:14 -04:00
Brandon Thomas e919d3f80b Merge pull request #249 from switch87/desktop-log-file
Desktop: write log records to standard error and a rotating log file
2026-10-09 09:23:08 -04:00
Brandon Thomas d16f9adb0a Merge pull request #387 from txiki739/fix/linux-tar-installer
#214: install script for the Linux tarball
2026-10-09 09:23:02 -04:00
Brandon Thomas bfa819a448 Merge pull request #358 from JordyUSA/vaapi-h264-decode
HW5: Linux VA-API hardware decoding for H.264 and HEVC (Main / Main 10)
2026-10-09 09:22:56 -04:00
Brandon Thomas 3e6043c46a Merge pull request #263 from XusBadia/dmg-background
macOS DMG: branded window background and icon layout
2026-10-09 09:20:31 -04:00
Manuel Camacho 1aa6ff6514 GPU3c: effects-heavy export bench scene (#30) 2026-10-09 06:18:18 -07:00
Manuel Camacho 5ee4faf2d2 GPU7: Lumetri GPU op matches fn lumetri exactly (test); doc comments restored (#30)
Add CPU parity test verifying FxOp::Lumetri::apply matches original fn lumetri
exactly (0.00e0 sample difference) across all parity cases. Restore doc comments
on fn on and fn grade_space.

🤖 Written by agy (Antigravity Agent, Gemini 3.8 Flash)
2026-10-09 06:18:18 -07:00
Manuel Camacho 029e9bb040 GPU7: Lumetri Color (basic, creative, vignette) on the GPU compositor (#30)
Port the common slice of Lumetri Color (Basic Correction, Creative without
sharpen, and Vignette) to the GPU effect stage (FxOp::Lumetri, OP_LUMETRI = 31).

Lumetri instances with unsupported sections (input LUT, Look / look LUT,
sharpen, curves, color wheels, HSL secondary, or HDR grading) fall back to CPU.

Parity tests pass within max relative difference 4.28e-7 and 0.000% flips.

🤖 Written by agy (Antigravity Agent, Gemini 3.8 Flash)
2026-10-09 06:18:18 -07:00
Manuel CamachoandClaude Opus 5.5 83e6fe7eda GPU6: invert alpha keeps the CPU's 1e-6 threshold (#30)
The previous commit raised the GPU's un-premultiply threshold for Invert
(alpha channel) to 1e-4, which the CPU reference does not use (it is
1e-6): alphas between the two inverted differently. The 6 % flipped
pixels came from the bilinear leak the same commit fixed in load4; with
it, the CPU's threshold passes gpu_effects_match_cpu_exactly (and the
gpu and golden suites, serially).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 06:18:18 -07:00
Manuel Camacho f2fefeeb4b GPU6: GPU effects match the CPU on AMD (gamma, extract, invert, color replace) (#30)
Fix subpixel bilinear interpolation leakage during 1:1 source texture
sampling in composite.wgsl load4, and guard against divide-by-zero /
subpixel noise amplification in OP_INVERT_ALPHA in fx.wgsl.

All fx_tests::gpu_effects_match_cpu_exactly test cases now pass on AMD
(Radeon RX 7900, Mesa 26.2.2, Vulkan) within EXACT (1e-4) and 0.000% flips.

🤖 Written by agy (Antigravity Agent, Gemini 3.8 Flash)
2026-10-09 06:18:17 -07:00
Manuel Camacho ab334b2adf GPU5: Video Limiter on the GPU compositor (#30)
Move Video Limiter from CPU (crates/render/src/vfx/color.rs) to the GPU
compositor (filmcraft_render::gpufx::FxOp, crates/gpu/src/fx.rs, crates/gpu/src/fx.wgsl).
Parity matches CPU reference within 9.28e-6 across all axis modes, clip levels,
compressions, and gamut warning color.

🤖 Written by agy (Antigravity Agent, Gemini 3.8 Flash)
2026-10-09 06:18:17 -07:00
Manuel Camacho d2b608605e GPU4: Vignette on the GPU compositor (#30)
Add Vignette effect to the GPU effect stage (render::gpufx, gpu::fx,
fx.wgsl). Evaluated parameters are shared between the CPU reference and
GPU compute passes. Replaces old CPU vignette body in vfx::color with
gpufx::FxOp::eval and gpufx::FxOp::apply.

Gates: fmt, clippy, test -p filmcraft-render, test -p filmcraft-gpu,
layers, wasm all verified.

🤖 Written by agy (Antigravity Agent, Gemini 3.8 Flash)
2026-10-09 06:18:17 -07:00
Manuel CamachoandClaude Opus 5.5 91056340ba GPU3b: pooled GPU export renderers, one-submission read-back (#30)
GPU export rendering was ~20 % slower in wall-clock time than CPU
rendering: one renderer behind a mutex made the export workers queue.

- Export workers share a pool of up to four renderers on one wgpu device
  (one per worker was no faster: the read-back then contends for memory
  bandwidth).
- render_export_prepared composites into the linear accumulator and
  copies it to a reused staging buffer in one submission (no sRGB resolve
  pass); read_accumulator shares the read-back, which now returns None
  instead of stale pooled values when a row is missing.
- f16_to_f32_fast: same result as f16_to_f32 for all 65536 inputs (test).
- Timing counters (export::export_lock_wait, gpu::export_renderer::timing)
  printed by the export bench.

RX 7900, same build, Off / Auto / Off: H.264 + AAC 4.67 / 4.71 / 4.76 s,
ProRes 422 4.20 / 4.21 / 4.21 s; CPU per frame 287-292 -> 277 (H.264),
316-317 -> 300 (ProRes). Was 4.86 -> 5.78 s. Parity, not faster: on this
single-clip scene decoding and encoding dominate (rendering is ~15 of
~290 CPU ms per frame), so GPU rendering stays Off by default.

Gates: fmt, clippy (gpu, export, ui-egui), layers, assets; gpu
export_pipeline, golden and export tests pass. Pre-existing on the
previous commit: fx_tests::gpu_effects_match_cpu_exactly fails on this
AMD GPU, and the gpu unit tests segfault in the driver when run in
parallel (pass serially apart from that test).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 06:18:17 -07:00
Manuel Camacho ba403212f6 GPU3: GPU export rendering off by default until it is faster (#30)
Set GpuRendering default to Off in ExportSettings so exports use the CPU
reference renderer by default until GPU export achieves lower wall-clock time.

🤖 Written by agy (Antigravity Agent, Gemini 3.8 Flash)
2026-10-09 06:18:17 -07:00
Manuel Camacho 2b0f263418 GPU3: render exports on the GPU compositor (#30)
Add off-screen GPU export renderer (filmcraft-gpu::ExportRenderer) reading back
the linear float accumulator as premultiplied RGBA f32, plug it into the
export pipeline via FrameRenderer hook and GpuRendering setting, register
in desktop app and CLI, and record export.gpu_frames statistics.

🤖 Completed by agy (Antigravity Agent, Gemini 3.8 Flash) based on draft by Merlin
2026-10-09 06:18:17 -07:00
Brandon Thomas 9c24778039 Merge pull request #399 from ItzBloxy/fix/259-trim-hit-testing
#259: the edge you press is the edge you trim
2026-10-09 08:17:17 -04:00
Brandon Thomas 6a240ca701 Merge pull request #391 from txiki739/fix/export-menu-paths
#382: File > Export entries work from the menus
2026-10-09 08:17:12 -04:00
Brandon Thomas 6a1145eb42 Merge pull request #324 from mjohnson7cd/fix/301-nonstandard-fps-stepping
fix: frame stepping broken for non-standard frame rates (#301)
2026-10-09 08:17:06 -04:00
Brandon Thomas 28235e25b8 Merge pull request #394 from txiki739/fix/reverse-source-frames
#318: a reversed clip shows exactly its source range
2026-10-09 08:17:01 -04:00
Brandon Thomas 7605a7a7ce Merge pull request #392 from txiki739/fix/fcp7-subframe-spans
#340: FCP7 XML clip source span always matches its record span
2026-10-09 08:16:56 -04:00
Brandon Thomas 475b30d793 Merge pull request #393 from MarekSpodBiedry/issue-379
#379: preset export and import go through the host, so the web build no longer traps
2026-10-09 08:16:51 -04:00
Brandon Thomas fee418e2b6 Merge pull request #252 from WavJaby/fix/workspace-lints
chore(lints): satisfy strict workspace Clippy on Rust 1.95
2026-10-09 08:16:46 -04:00
Brandon Thomas d51f6b1b7d Merge pull request #362 from JordyUSA/fix/341-export-message
#341: CLI export message names the file written, not the requested path
2026-10-09 08:16:41 -04:00
Brandon Thomas 0f409f3fb1 Merge pull request #297 from mjohnson7cd/fix/285-cli-broken-pipe
fix: filmcraft-cli exits quietly on broken pipe (#285)
2026-10-09 08:16:36 -04:00
BloxyandClaude Opus 5.5 7401f5038b contributors: credit @ItzBloxy
My own entry for the About window: display name "Bloxy".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 12:17:08 +02:00
BloxyandClaude Opus 5.5 af2b4601c8 #259: caption edges trim from where they were pressed
Caption blocks chose move / trim in / trim out where egui recognised the drag, about 9 px in
from a press 2 px inside the edge and past the 5 px edge zone, so they moved instead of
trimming. They now decide at the press origin and measure the drag from it. UI test that
failed before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:46:35 +02:00
BloxyandClaude Opus 5.5 42cc47e7db #259: the timeline shows the edge a press will grab
Hovering a clip edge draws the edit-point bracket at half strength on the edge a press would
grab: red for a ripple; yellow for a trim, roll, rate stretch or remix; on both sides of the
cut for a roll. It follows the tool and the modifiers, from the same grab_at as the press.
The bracket drawing is shared with the selected edit points (trim_monitor::paint_bracket).

An ignored test renders the four cases at a demo cut to PNGs for review:
cargo test -p filmcraft-ui-egui --test timeline_trim_ui -- --ignored hover_bracket_screenshots

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:40:07 +02:00
txiki739andClaude Opus 5.5 dfd67b2774 #318: a reversed clip shows exactly its source range
A reversed clip's source time is the instant just before its mirrored
position (source_out - rel - 1 tick). The MP4 reader rounds a request to
the nearest unit of the track timescale before picking the frame at or
before it, so one tick below a frame boundary landed on the next frame:
a clip trimmed to source frames 10-29 played 30 down to 11.

video_source_time snaps a reversed clip's video request down to the
start of the media frame that contains it (clamped to the source In and
the unsnapped time). Both places that ask a source for a clip's picture
use it: media_frame and the plan path. source_time_at is unchanged for
audio and nested sequences, and forward clips and frame holds keep
their request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:28:24 +02:00
BloxyandClaude Opus 5.5 4eb34ae6c7 #259: the edge you press is the edge you trim
Pressing a clip edge and dragging inward selected and moved the clip: the press was hit-tested
only once egui recognised a drag, 6 pt later, by when the pointer had left the 7 px edge zone.
And at a cut the left clip's Out edge claimed both sides, so the right clip's In edge could not
be grabbed.

What a press grabs is now worked out on the frame the button goes down (timeline_hit::grab_at,
shared by the cursor, the press and ui.timeline.hit) and acted on by both the click and the
drag. Inside a clip beats outside it, so the side of a cut picks the clip; zero-width clips
have no edges. Trims, rolls, rate stretches and remixes move by how far the pointer moved since
the press, and a moved clip stays under the point it was grabbed by. ui.timeline.hit takes
`modifiers` and reports `kind`, the trim a press would start.

Unit tests for the zones and tools; headless UI tests that drag a pixel or a few per frame at
60 fps (before: the clip moved, the wrong clip was trimmed, a wobbly click selected the clip, a
moved clip lagged).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:27:48 +02:00
txiki739andClaude Opus 5.5 43e3d26823 #218: System Language follows the operating system's language
Settings > General > Interface Language gains System Language
(`system`), now the default: a new installation shows the first of the
user's preferred languages that the interface has (es-419, pt_BR.UTF-8,
ja-JP...; any Portuguese gets pt-br), otherwise English. A language
already chosen, or saved in preferences.json, is kept.

The host supplies the preferred languages through
HostHooks::system_languages: sys-locale on the desktop (LANGUAGE,
LC_ALL, LC_MESSAGES, LANG on Linux; the OS lists on macOS/Windows) and
navigator.languages on the web. Without the hook (tests, headless),
System Language is English, so tests do not depend on the machine's
locale.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:06:43 +02:00
MarekSpodBiedry 55e43e6147 #379: preset export and import go through the host, so the web build no longer traps 2026-10-09 10:45:28 +02:00
txiki739andClaude Opus 5.5 d39692af63 #340: FCP7 XML clip source span always matches its record span
write_clip rounded start, end, in and out to frames independently, so
a clip placed by seconds at 23.976 (start 0, in 20 s, 3 s long) was
written as start 0 / end 72 / in 480 / out 551: 71 source frames for
72 record frames.

At normal speed and on a freeze frame the source span is the record
span, so out is now in + (end - start). Speed-changed clips keep their
independently rounded out. The master duration is raised to out when
rounding pushes out past it; pproTicksIn/Out still carry exact ticks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:37:12 +02:00
txiki739andClaude Opus 5.5 a076e50553 #382: File > Export entries work from the menus
A menu entry runs its command with no parameters, so every File >
Export entry whose command needs a `path` failed with "need `path`",
from the egui menu bar and the native macOS menu alike.

- Media… opens the Export mode, like Cmd+M (Premiere's File > Export >
  Media); with a `path` it still exports directly.
- EDL, Final Cut Pro XML, FCPXML, OpenTimelineIO, Avid Log Exchange and
  Selection as FilmCraft Project ask the host's save panel for the
  destination, suggest the sequence's (or project's) name with the
  right extension, then export. "Selection" suggests "<project>
  selection.fcproj" so it does not offer to overwrite the open project.
  Cancelling writes nothing and reports no error.

Disabled entries (no sequence, no selection) still fail before any
dialog opens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:27:36 +02:00
txiki739andClaude Opus 5.5 5f3682fe7c #262: file dialogs list extensions in any capitalisation on Linux
rfd hands filter extensions to the XDG portal, GTK and Zenity as
case-sensitive `*.ext` globs, so `CLIP.MP4` and `clip.Mp4` were hidden.
`file_filters::extensions` keeps the given extensions first (the save
dialogs' default suffix) and, outside Windows and macOS, adds one
bracket pattern per extension (`mp4` -> `[mM][pP]4`), the form the
portal's FileChooser docs recommend.

Every filtered dialog uses it: import, relink, Open file, Open
Project, Save Project and Save As.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:06:35 +02:00
txiki739andClaude Opus 5.5 ae09ba904c #214: install script for the Linux tarball
The tarball now ships install.sh. It installs filmcraft, filmcraft-cli,
the desktop entry, icons, MIME type, metainfo and docs under ~/.local
without sudo; --prefix takes any absolute path (sudo ./install.sh
--prefix /usr/local for all users). Running it again updates.

The desktop entry's Exec/TryExec point at the installed binary's
absolute path, escaped per the Desktop Entry spec, so the menu entry
works without ~/.local/bin on PATH. The script checks the bundle is
complete before writing, refuses /, prefixes inside the extracted
bundle and characters launchers cannot express (=, %, control
characters), and treats the cache refresh tools as optional.

packaging/linux/test_install.py covers install and update, launching
through gio, bad arguments, an incomplete bundle, --help and a real
package.sh tarball build; Packaging lint runs it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:54:20 +02:00
Davi AndreiandClaude Sonnet 5.5 b485e2c180 Thumbnails decode with small decoders that are not kept
Right after importing many 4K clips the Project panel and the timeline ask
every clip for a thumbnail. Up to six frame workers decoded them at once,
each with a frame-threaded software H.264 decoder (16 threads and 18
pictures in flight here, about 0.5 GB at 4K), and the decoders stayed: 42
4K clips with hardware decoding Off took the process past 6 GB during import.

Thumbnail jobs (item frames at the Media Browser / thumbnail priorities) now
run under filmcraft_media::cancel::with_background: H.264 decoders made for
them use 2 threads (thread_limited), parallelism comes from the workers
decoding different files, and the decoders of caches only thumbnails used
are released after each thumbnail (the clip served last keeps its own). A
monitor or playback request on such a cache replaces the small decoder with
a full one. Sequence frames are never background (the scopes ask for the
program frame at 40 while playing).

42 4K clips (Windows), peak private bytes: hardware Off 6.1-6.8 GB (killed)
-> 3.1-3.2 GB; hardware Auto 2.7-2.8 GB (#381) -> 2.3-2.4 GB. Thumbnails of
12 / 42 clips ready in 0.6 / 2.2 s instead of 2.2 s / not finishing.
Example crates/ui-egui/examples/thumb_burst.rs reproduces the burst.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 04:32:43 -03:00
Davi AndreiandClaude Sonnet 5.5 0c64e41b53 Decode caches: navigation bursts no longer keep every clip's decoder and frames (#55)
A cache asked for a frame within IDLE (3 s) counted as in use, and the caps
of #41 (4 live decoders, 1 GiB of frames) only bound caches idle for longer,
so moving the playhead across 42 4K clips kept 42 hardware decoders (6.5 GB
of VRAM) and up to 3.9 GB of frames. Now only caches touched within RECENT
(500 ms) are protected: caches idle 0.5-3 s give up decoders beyond
HARD_DECODERS (8) and frames beyond the shared budget, least recently used
first; the 3 s rule down to 4 decoders is unchanged. store() refreshes
last_used.

Measured with a 42-clip 4K reproduction (Windows, Media Foundation): peak
private bytes 6.6 -> 4.0 GB, frame cache 3.9 -> 1.1 GB, live decoders
42 -> 8-14, GPU memory 6.5 -> 4.3-4.6 GB, 0 dropped frames before and after.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 03:13:42 -03:00
WavJaby 6893cb4f03 perf(web): feed native YUV frames to the shared compositor 2026-10-09 13:47:06 +08:00
sanyamsingh001andCodex 892003f3d4 GPU.EFFECT_SOURCE: preserve exact texel centers for effect inputs
Draw effect sources from working pixel positions and integer decimation.
Cover identity, alpha inversion and box minification without changing tolerances.

Co-authored-by: Codex <codex@openai.com>
2026-10-09 15:44:39 +10:00
Xus BadiaandClaude Opus 5.5 10b71eec85 macOS DMG: branded window background and icon layout
The DMG opened as a plain Finder window. It now shows the app icon's figure on
the FilmCraft colour, with FilmCraft.app and the Applications link side by side.
Same approach as PhotoCraft's and VectorCraft's (storytold/photocraft#1201,
storytold/vectorcraft#493).

- packaging/macos/dmg/: background.svg (source; text outlined, no paper grain,
  the figure an <image> of assets/app-icon), background.tiff (1x + 2x HiDPI,
  16-colour palette, Deflate, sRGB; ~106 KB)
  and dmg-layout.DS_Store (Finder view settings: window, icon size 128,
  positions, background).
- generate.py writes the TIFF and the layout on any OS, byte-for-byte
  reproducible, with nothing from the machine that ran it (no paths, users,
  disk names or volume UUIDs).
- package.sh copies them into the staging folder; the image is still built with
  hdiutil makehybrid. Nothing extra runs in the signing job.
- The volume name drops the version: the layout finds the background through
  an alias that includes the volume name. The DMG file name keeps the version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 07:43:43 +02:00
storybored379andOpenAI Codex 7264594494 mixer: report only the audio gains actually applied
Fixes #343.

Co-Authored-By: OpenAI Codex <noreply@openai.com>
2026-10-08 21:35:58 -07:00
Davi AndreiandClaude Sonnet 5.5 9510e26584 Export: render the next batch while the current one is encoded
A step now renders batch k+1 on the rayon pool while the calling thread
encodes, muxes and mixes the audio of batch k. The file is byte-identical
with or without overlap. A prefetch that hits pending media is dropped and
rendered again, cancel frees it, a render panic becomes an error, and wasm
never overlaps. perf.stats export.stages gains waitMs.

Memory: up to 2 x batch x frame bytes of finished frames are in flight, so
the batch is capped to a 512 MiB budget (1080p keeps 16, 4K SDR 8, 4K HDR 2
frames). Measured peak RSS vs before: 1080p +67..+158 MB; 4K SDR -535..-927
MB; 4K HDR about -2.6 GB (5.2 -> 2.6 GB). Wall time 1.06-1.16x faster.

Known pre-existing web issue, untouched: AudioOut::pull advances before the
pending check, so a retried audio group is lost.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 01:26:41 -03:00
Davi AndreiandClaude Opus 5.5 1996f319c3 Export: stage wall times in perf.stats (export.stages)
filmcraft_export::Stage, stage_times(), note_stage() and timed(): the wall
time of setup, loudness, render, encode (and its RGB->YUV convert part),
audio, mux and finish, summed per process like export.hardware. Converters
in the H.264 / ProRes / NVENC / VideoToolbox encoders are timed as Convert.
Two clock reads per stage per batch; the exported file does not change
(determinism tests). Test: an export grows every stage counter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 01:26:09 -03:00
storybored379andOpenAI Codex 3971b2666f graphics: keep duration separate from timeline placement
Fixes #326.

Co-Authored-By: OpenAI Codex <noreply@openai.com>
2026-10-08 21:08:47 -07:00
sanyamsingh001andCodex 0e0f0485da LINUX.VALIDATION: honor preset shortcuts and retain MPEG oracle frames
Resolve inherited physical key collisions when loading built-in presets,
while preserving established explicit winners and deliberate custom conflicts.
Read every ffprobe display-frame timestamp and register the requested credit.
Verify exact frame-plane reuse without depending on concurrent global metrics.

Co-authored-by: Codex <codex@openai.com>
2026-10-09 14:06:37 +10:00
storybored379andClaude Opus 5.5 78053191ce MCP: cancelling a movie export keeps numbered files beside it
remove_partial treated any `<stem><digits>.<ext>` file as a still of an image sequence, so
cancelling an export to `clip.mp4` also deleted a `clip1.mp4` written in the same second (or
during the export). Only PNG/TIFF/BMP sequence exports write numbered stills, and their frame
numbers have at least three digits (`image_sequence_path`); match only those.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 21:05:31 -07:00
storybored379andOpenAI Codex ea79751a5c platform: enable existing NVENC H.264 export on Linux
Runtime-load the NVIDIA encode and CUDA driver libraries, retaining the primary context through session destruction. Preserve the Windows Direct3D path and default-off selection, and test driver rejection and deterministic software fallback.

Co-Authored-By: OpenAI Codex <noreply@openai.com>
2026-10-08 20:40:01 -07:00
Cathy Shapiro ac145dbaa8 M3.12: Dismiss Export Frame when clicking outside
Cancel the export draft when clicking outside the dialog, including top menus and mode tabs, while allowing the background control to receive the click. Keep the dialog's own format and depth dropdowns interactive.

Cover dismissal for both monitors, mode switching, dropdown selection, and background monitor clicks in a UI regression. No frame is exported when clicking away.

Assisted-by: Codex
2026-10-08 21:57:30 -05:00
JordyUSAandClaude Opus 5.5 9e8ad1d6a2 #341: CLI export message names the file written, not the requested path
When the format changes the extension (APV into `out.mp4` writes
`out.mp4.mov`), `filmcraft-cli export` said "exported out.mp4" on stderr
while stdout JSON and the filesystem had `out.mp4.mov`. The message now
takes the path(s) from the export result: `path` of a direct export, each
item's `path` with --queue; the requested path only when the result names
none. Unit test for the helper; end-to-end test (direct and queued APV
export) that failed before the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:53:28 +07:00
Davi AndreiandClaude Sonnet 5.5 dbae11b843 Windows: NVENC H.265 Main 10 HDR (PQ / HLG) export
HDR sequences export as real 10-bit HEVC Main 10 through NVENC where the GPU
reports 10-bit HEVC: float BT.2020 R'G'B' -> rgbf_to_yuv420_10 -> P010 input;
VUI BT.2020 + PQ/HLG; HDR10 mastering-display / content-light as SEI on every
IDR (PQ); colr/mdcv/clli in the sample entry. New register_hdr_probe /
hdr_available in the export crate; macOS and GPUs without 10-bit keep
tone-mapping HDR sequences to 8-bit SDR HEVC. ABI asserts extended and
re-verified against nvEncodeAPI.h 12.1 (0 drift). 8-bit SDR HEVC unchanged;
the two #328 tests that asserted HDR is declined now expect Main 10.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 23:38:14 -03:00
Idiotic f146d7ba9f feat(media,engine): implement multi-stream container audio support and schema v13
- Update MediaInfo to store all discovered audio streams in audio_streams

- Retain independent stream metadata and decoder state for MP4/MOV and Matroska/WebM

- Update audio_placement_specs to allocate successive tracks via ensure_audio_tracks

- Isolate clip rendering to read assigned audio stream rather than replaying stream 0

- Bump project schema to v13 with backwards compatibility for legacy v12 projects

Resolves #312

Fixes #200

Fixes #315
2026-10-08 22:36:45 -04:00
ippsecandClaude Opus 5.5 c43d1090c1 contributors: credit IppSec by GitHub profile name
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 22:30:49 -04:00
ippsecandClaude Opus 5.5 d51945c807 #236: audio in MKV and OBS-remuxed MP4 plays without crackling
Matroska timestamps are whole milliseconds and a 1024-sample AAC frame at 48 kHz lasts
21.33 ms, so frames are stamped at 0, 1008, 2064, 3072… samples; OBS's remux to MP4 carries
that rounding into the sample durations (1008, 1008, 1056…). The MKV and MP4 readers put each
decoded packet at its own timestamp, overwriting the end of one packet and leaving a hole of
silence after another: a click every few packets, crackling throughout playback and export.

Packets of codecs with a fixed frame length (AAC, MPEG audio, AC-3) now run on from one
another and resynchronise to the timestamp only across gaps of more than half a packet, the
rule the MPEG-TS/PS reader already uses. Laced Matroska frames without a duration (which
repeat the block's timestamp) follow on too, and MKV Opus uses the same helper. MP4 audio
packet starts are kept in samples instead of track timescale units.

Regression tests mux our own AAC into MKV (millisecond stamps, plain and laced, a real gap)
and into an MP4 with OBS-remux durations: block-by-block playback and seeks must equal the
packets decoded back to back (all fail before the fix). An ffmpeg oracle test plays
ffmpeg-made MKVs with AAC, MP3, MP2 and AC-3 and requires them as smooth as ffmpeg's decode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 22:29:23 -04:00
swillunderscoreandClaude Opus 5.5 57d8990765 contributors: display name for @swillunderscore
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 21:19:21 -05:00
jnohclee-rgb 0bfea22e1b cli: make --help print usage successfully
Fixes #344. Handle the leading help flag before command parsing and decoder registration; cover output and exit status with a native CLI regression.
2026-10-09 10:14:06 +08:00
Cathy Shapiro 6c0e3ecc39 M3.12: Match Export Frame timecode to its monitor
Display the captured frame with the same frame rate and drop-frame timecode format as Source or Program, instead of decimal seconds. Keep the captured timecode fixed while the export dialog is open.

Add a UI regression comparing both monitor labels at standard and NTSC rates, with Program drop-frame formatting and playhead movement after capture.

Assisted-by: Codex
2026-10-08 21:11:44 -05:00
JordyUSAandClaude Opus 5.5 e741267125 HW5: Linux VA-API HEVC hardware decoding, Main and Main 10 (#30)
HEVC through VA-API the same way as H.264: the software HEVC decoder's
parsers and DPB (now generic over the picture type in filmcraft-hevc:
POC, reference picture sets, reference lists, output order, RASL
handling) drive the stateless hardware. Main, Main 10 and Main Still
Picture, 4:2:0 8/10-bit (P010 read-back); scaling lists, tiles, WPP,
dependent slices, weighted prediction. Missing references and range
extensions go to the software decoder.

Tests: hardware parity on HEVC Main / Main 10 incl. open GOP, scaling
lists, slices+WPP, weighted prediction, tskip+AMP, 1080p/2160p (streamed
comparison to stay within memory); HEVC front end against a recording
stand-in on every OS; ABI checks extended to va_dec_hevc.h. Benchmark on
Iris Xe: 4K HEVC 203 -> 10 ms CPU per frame, 4K Main 10 212 -> 18 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:00:30 +07:00
swillunderscoreandClaude Opus 5.5 87bb39f5f5 #172: Sequence Settings… opens Premiere's dialog (General, Color Management, VR Properties)
The menu item ran sequence.settings with no parameters and changed nothing. It now opens the
dialog: Timebase, Frame Size with "Scale motion effects proportionally" (on by default), the
timecode Display Format (drop-frame / non-drop-frame for 29.97, 59.94, 119.88), Channel Format,
Sample Rate, Maximum Render Quality, and the Color Management tab, applied in one undo step; OK
with nothing changed adds no undo step. Settings FilmCraft has no equivalent for yet (Editing
Mode, Pixel Aspect Ratio, Fields, Frames / Feet + Frames, Number of Channels, audio Display
Format, Video Previews, Maximum Bit Depth, VR Properties) are shown greyed out with their value
and a "Not supported in FilmCraft yet" hint; Composite in Linear Color is shown on (the
compositor is always linear). Every control has an automation id.

sequence_ui.rs: two tests driving the dialog over the control channel, and
FILMCRAFT_UI_SNAPSHOT_DIR support like the other UI tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 20:58:37 -05:00
swillunderscoreandClaude Opus 5.5 1f6f7d4c7d #172: sequence.settings scales Motion with the frame size and takes timecode, render quality and colour
- filmcraft_edit::frame_size::scale_motion: Sequence Settings' "Scale motion effects
  proportionally when changing frame size". Every video clip's Motion Position maps through
  p' = c1 + k(p - c0) and Scale / Scale Width multiply by k = min(W1/W0, H1/H0), static values
  and keyframes: a plain proportional scale for the same aspect, and on an aspect change the old
  frame fits inside the new one, centred. Automatic positions stay automatic, Scale to Frame
  Size clips keep their Scale (the renderer refits them), Anchor Point and Rotation stay.
- sequence.settings: new dropFrame, maxRenderQuality, workingSpace, wideGamut, autoToneMap and
  scaleMotion (default false, so existing callers are unchanged) params, in the same undo step;
  returns {"scaledClips": n}. Drop-frame on a rate without it is refused atomically.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 20:58:37 -05:00
jerrykimo 34cf6005d0 ui: Render Chinese document names with font fallbacks 2026-10-08 20:50:09 -05:00
Cathy Shapiro 9f0ed51b49 M3.12: Export frames from Source and Program monitors
Wire both camera buttons to captured-monitor frame export with original media at full resolution. Add a movable settings dialog with format/depth selection, a visible destination folder, Browse, overwrite confirmation, and optional project import.

Reuse existing 8-bit still encoding; add JPEG and direct float-to-16-bit PNG/TIFF encoding. Cover monitor routing, captured frame time, proxy bypass, hostile parameters, depth, import, cancellation, overwrite, and fractional header movement with synthetic regressions.

Assisted-by: Codex
2026-10-08 20:46:12 -05:00
JordyUSAandClaude Opus 5.5 3496e553e7 HW5: Linux VA-API H.264 hardware decoding (#30)
Stateless VA-API decoding with FilmCraft's own H.264 host side: the
software decoder's SPS/PPS/slice-header parsers and its DPB (now generic
over the picture type in filmcraft-h264) drive the GPU, so reference
marking, reference lists and output order are shared and the pictures are
bit-exact with the software decoder. libva is loaded at run time; no libva
or driver means software decoding as before. frame_num gaps (open-GOP
seeks) get the software decoder's stand-in frames. HybridDecoder keeps the
transparent software fallback.

Tests: hardware parity on 7 H.264 streams incl. 1080p/2160p, seeks,
flushes, forced failures, fuzzing, concurrency; front end against a
recording stand-in on every OS; ABI checks against va.h. Docs: platform
README, ADR 0001 addendum, architecture, performance (H.264 2160p CPU per
frame 417 -> 11 ms on Iris Xe), roadmap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 08:35:57 +07:00
Davi AndreiandClaude Opus 5.5 a8f91e3ffe M2: H.265 stays in the Main tier (pick the level)
Left to choose, NVENC answered a peak bitrate above its level's Main tier
limit with the High tier: 1080p30 at 12/18 Mbit/s was written as level 4 High
tier, 2160p30/60 at 40/60 Mbit/s as 5.0/5.1 High tier, which many hardware
decoders refuse. The level is now the lowest whose Main tier limits hold the
picture size, sample rate and peak bitrate (H.265 Table A.8), e.g. 1080p30 at
18 Mbit/s is level 4.1 Main tier. Unit test of the table; hardware test that
the hvcC says Main tier at the expected level.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 22:23:25 -03:00
Idiotic 78ed3edd84 feat(engine): dynamically expand audio tracks on multi-clip placement 2026-10-08 21:11:12 -04:00
rossaaiandClaude Opus 5.5 0506a01f29 UI: the status bar fills with the header_bg token
The status bar was a fixed #1c1c1c, dark on the Light and Medium themes
and on a theme an embedding app sets through FilmcraftApp::tokens. It now
takes the header bar's colour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 19:57:43 -05:00
rossaaiandClaude Opus 5.5 1e9f37864b UI: UiState::show_header and show_status_bar, on by default
An app that embeds FilmCraft can hide the header bar and the status bar
and give the panels the whole window. Saved UI state without the fields
keeps both shown. With the status bar hidden, a finished preview render
still plays: the job watching moved out of job_status into watch_jobs,
which runs every frame.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 19:57:43 -05:00
rossaaiandClaude Opus 5.5 83501964f6 Web: public set_context for a host that runs its own WebApp
post and repaint wake the egui context that start stores in a private
thread-local. An app that embeds FilmCraft and builds WebApp in its own
eframe runner had no way to set it, so posted actions and window.filmcraft
requests waited for the next input event. start now calls the same
set_context.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 19:57:43 -05:00
Cathy Shapiro 8a7275c821 M3.12: Source playback and monitor range editing
Play source video and audio, drag marked ranges as video, audio, or a linked pair, and route Source commands to clip marks and markers. Add shared draggable In/Out handles and range movement, preserving scrubber priority and explicit-only Program marks.

Include original code-drawn controls, documentation, attribution, and regression coverage for playback, placement, routing, undo, cancellation, and hostile parameters.

Assisted-by: Codex
2026-10-08 18:58:46 -05:00
Cathy Shapiro 26d972ce01 WINCI: Merge current upstream 0.4.0 release and packaging
Preserve the Windows validation fixes and original #282 commit while bringing the contribution up to date with upstream main.

Assisted-by: Codex
2026-10-08 18:58:11 -05:00
DWAA1660andClaude Opus 5.5 8d6461705e Fix #179: default Windows startup to DirectX 12
eframe's default wgpu instance enables PRIMARY | GL on Windows. Creating
the OpenGL backend can crash inside AMD's atio6axx.dll before the window
appears, so the app flashes in Task Manager and exits. When WGPU_BACKEND
is not set, the Windows instance now enables only DirectX 12; an explicit
WGPU_BACKEND is still honoured. macOS, Linux and web are unchanged.

Same fix as storytold/designcraft#168.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LyVbT6Vv4BwWZ4hY8973qD
2026-10-08 17:52:30 -06:00
Cathy Shapiro 0b34c076ce WINCI: Windows path handling and portable validation
Normalize explicit relink prefixes, compare native paths in portable tests, close fixture sessions before directory moves, and preserve exact GPU effect-source pixel centres. Keep the hardware-dependent mixer benchmark opt-in with its 4x target unchanged.

Shortcut portability changes reused from storytold/filmcraft#37 (bd944ee and f2b36fe). The existing export fix from #282 remains in its original parent commit.

Assisted-by: Codex
2026-10-08 18:28:32 -05:00
mjohnson7cd 7048a7905a fix: frame stepping broken for non-standard frame rates (#301)
tick_of() used floor division (div_euclid) which truncated the frame
duration for non-standard rates like 37.516 fps (9379/250) where the
frame duration is 6770871094.999 ticks, not an integer. This caused
frame_at(tick_of(f)) to return f-1 instead of f, making step forward
stall at frame 0 and step back jump backward by 2+ frames.

Changed tick_of() to use ceiling division so the round-trip is exact
for all integer frame indices. Ceiling is correct because the frame
duration is always > 1 tick, so floor(ceil(f*D) / D) == f for every
integer f. frame_duration() remains floor division (unchanged) since
many callers do end - frame_duration() to get the previous frame.

All 11 standard broadcast rates are unaffected since their frame
durations are already exact integers (ceil == floor there).

Added regression tests:
- exotic_frame_rate_roundtrip: round-trip over -20k..20k for 5
  non-standard rates, snap invariant, step-back via frame_duration,
  forward/backward stepping simulation
- exotic_frame_rate_durations: frame_duration is floor for exotic,
  exact for standard
- exotic_frame_tick_roundtrip proptest: arbitrary FrameRate::new(num,
  den) round-trip

Thanks to @justjohn12345 for catching that round (not ceil) still
fails for ~50% of frames starting at f=938.
2026-10-08 18:37:41 -04:00
Oldrich333andClaude Opus 5.5 44248c060f CLI: a closed stdout no longer panics filmcraft-cli, and the work still finishes (#285)
`filmcraft-cli commands | head -c 10` panicked with "failed printing to stdout: Broken pipe
(os error 32)". Output now goes through `outln!` / `out!`. When stdout fails, later output is
dropped but the program does not stop: the rest of `exec`, `run` and `import` (later script
lines, `--save`, `--save-as`) still runs and the exit status still reports a failing command.
Exiting at the first failed print would report success while skipping the save, e.g.
`filmcraft-cli run edits.jsonl --project p.fcproj --save | head -1`. A closed pipe is not an
error; any other stdout write error is reported and makes the exit status 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 17:09:53 -04:00
WavJaby 4d74065aa6 fix(gpu): separate YUV code ranges from texture scaling
- Normalize full-range 8-bit and limited-range high-depth YUV
- Gate code ranges and CPU/GPU parity for inline/prepared uploads
- Existing main GPU effects/frame-pool failures reproduced
- Full workspace and app lifecycle checks remain unrun
2026-10-09 03:57:01 +08:00
Davi AndreiandClaude Opus 5.5 29b454ac17 M2: review fixes for NVENC H.265
- register() asks the HEVC probe on a thread of its own (the first probe is
  about half a second: D3D11 device + NVENC session), so the first draw of
  Export's format list does not wait for it; tests/nvenc_hevc_warm.rs.
- A machine with no HEVC encoder is not a declined hardware attempt: the
  factory returns None without counting `declined`.
- HEVC tests skip only when the probe says there is no HEVC encoder; on a
  machine that has one, a configuration NVENC refuses fails the test.
- Wording: NVENC is FilmCraft's only HEVC encoder on Windows (Windows itself
  ships others).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 16:48:43 -03:00
Davi AndreiandClaude Sonnet 5.5 deebe2e5b2 Windows: NVENC H.265 (HEVC) Main hardware encoding for Export
One NVENC session, ring and NV12 path now serve H.264 and HEVC (Codec enum,
Profile::HevcMain). HEVC Main 8-bit 4:2:0 SDR BT.709, MP4/MOV hvc1, parameter
sets only in an hvcC built from the SPS the encoder wrote. The H.265 format is
available through a cached NVENC probe; choosing it is the opt-in, declines are
counted errors. ABI asserts regenerated from nvEncodeAPI.h 12.1 (no drift) and
extended for HEVC; tests, docs and roadmap updated.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 16:28:50 -03:00
WavJaby 944d581cde perf(gpu): composite Push transitions on the shared GPU path 2026-10-09 01:41:43 +08:00
Pathos0925andClaude Fable 5.1 24cde5e9fe #296: Auto-Match measures a mono clip as one channel
Loudness Auto-Match metered every clip as the stereo pair clip_signal
plays, so a mono source (or a mono pick from Modify > Audio Channels /
Breakout to Mono), copied onto both sides, read 3.01 LU louder than the
file and got 3 dB too little gain. BS.1770 and ffmpeg's ebur128 measure
a mono programme as one channel; now so does clip_loudness, through a new
render::audio::clip_is_mono. Stereo and wider sources are unchanged, as
are the export loudness pass and the Meters panel (they measure the real
output mix).

Regression test: the same speech as a 1-channel WAV and as dual-mono
stereo measure 3.01 LU apart, the mono clip agrees with a one-channel
meter over the file within 0.1 LU, and a mono pick counts as mono.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 12:09:15 -05:00
justjohn12345andClaude Opus 5.5 a6b21fe7cb Keyframes: Properties gets Premiere's navigator, Effect Controls a time ruler
Properties panel: the diamond on a row was the stopwatch (effects.toggleAnimation),
so clicking it on a keyframe dropped every keyframe and kept the value at the
playhead. It is now Add/Remove Keyframe (effects.addKeyframe), as in Premiere:
it turns animation on with a keyframe at the playhead, adds one there, or removes
the one there, and the value falls back to what the remaining keyframes give in
Properties, Effect Controls and the Program monitor. An animated row shows the
navigator (previous / diamond / next); the diamond is filled only while the
playhead is on a keyframe. Crop's diamonds work too (the effect is applied first).

Effect Controls: the keyframe lane has a time ruler (ticks and sequence timecode
over the clip's stretch of the sequence) with the playhead's handle on it; click
or drag the ruler to move the playhead. The playhead is drawn only while it is on
the clip. Both panels share one navigator widget, with automation ids
<panel>.<effect>.<param>.addKeyframe / .prevKeyframe / .nextKeyframe, plus
effectControls.ruler and effectControls.playhead.

Tests: engine add/remove at the playhead with undo, redo and bad parameters;
headless UI tests of the navigator, Crop and the ruler; ruler labels and limits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:06:25 -04:00
justjohn12345andClaude Opus 5.5 d09e95663c Program monitor: the picture follows a value that is dragged
While a value was dragged in Effect Controls or Properties, the Program monitor
kept the picture from before the drag and only caught up when the mouse rested.

Two causes:

- Every step of a drag is a new project revision, so the frame the monitor asks
  for is never ready on the refresh that asks for it, and the stand-in lookup
  only knew frames of the same revision. The monitor now remembers the frames it
  asked for on its latest paused refreshes and, when neither the frame due nor a
  cached frame just before it is ready, shows the newest of those that is. A
  backward or fast scrub had the same gap and gets the same stand-in. Playback
  is unchanged.
- A frame still rendering was cancelled when the same frame was asked for at a
  newer revision. With frames slower than a refresh (CPU effects) each one was
  thrown away for the next. Such a job now finishes; a request for a different
  frame of the view (scrubbing) still cancels it.

perf.stats gains `monitor`: the frame and revision of the picture the Program
monitor shows, against the playhead's frame and the project's revision.

Tests (headless, perf_ui): a parameter changed on every UI pass and a backward
scrub leave the picture at most one step behind; a frame being rendered survives
a request for the same frame at a newer revision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:06:13 -04:00
MarekSpodBiedry 502c9a33a7 Issue #139: portable.txt next to the exe keeps the data directory beside it 2026-10-08 18:58:41 +02:00
Dario CarneluttiandClaude Sonnet 5.5 389cfefe92 platform: ask for the hardware HEVC encoder in the background at startup
`Format::Hevc` is available only where a hardware HEVC session can be created, and
`hardware_encode::hevc_available` finds out by creating a small one on the first question. That
question comes from the first draw of Export mode's format list, on the UI thread, and the session
takes about 0.1 s (88-100 ms in seven of eight fresh processes on an Apple M1, 222 ms in the first):
the app stalls for that long the first time Export is opened.

`register()` now starts the probe on a thread of its own (`warm_hevc_probe`, once however many times
`register` is called). Whoever asks while it runs waits for the same `OnceLock` answer, no longer than
asking first; a thread that cannot start, or a probe that panics (caught), leaves the question to the
first caller as before. `hevc_probed()` says whether the answer exists, for diagnostics and tests.

Test: `tests/hevc_probe.rs` registers, never calls `hevc_available`, and waits for `hevc_probed()` (it
fails after 60 s without the call in `register`).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 10:17:58 -06:00
Dario CarneluttiandClaude Sonnet 5.5 10879f385e export: free the frame pool when an export ends
The export pipeline hands the float image of every frame it converts back to
`filmcraft_frame::pool` (#184), which keeps up to 320 MiB of them, and the decoders' plane shelves
hold up to 192 MiB of each type. An export's images are the size of its frames, which nothing else
asks for, so a finished export left them idle until the next one.

`pool::trim()` frees every idle buffer. `Pipeline` calls it when it is dropped, one place for every
way an export ends (done, failed, cancelled, dropped half way by a host that steps it, as the web
build does) and for every caller of `export` / `Exporter` (previews, proxies and AAF/OMF mixdowns
share the pipeline). A job still rendering at that moment allocates a few images again: the pool is
only a cache.

Measured with a synthetic 3 s 1080p24 matte exported as PNG frames three times in one process
(Apple M1, `footprint`, 10 s after the last export): 575 MB without the trim, the pool holding
332 MB idle, and 258 MB with it. macOS returns the freed pages over a few seconds, not at once.

Tests: `crates/frame/tests/pool_trim.rs` (every shelf emptied, the pool works afterwards) and
`crates/export/tests/pool_trim.rs` (an image sequence, a finished and a cancelled video, an export
that could not start and a stepped export dropped half way each leave nothing idle; without the
trim they fail).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 10:13:07 -06:00
midasdfandCodex d77728008f Keep grapheme clusters together in vertical text
Co-Authored-By: Codex <noreply@openai.com>
2026-10-09 00:40:01 +09:00
Tom 2b843d3d81 Windows: export paths expand ~, accept trailing \ and verbatim paths
- expand_home uses std_home_dir (USERPROFILE on Windows) instead of
  HOME only; the Export panel reuses the engine function.
- std_home_dir prefers USERPROFILE on Windows, where Git Bash sets
  HOME to a Unix-style path.
- On Windows, / in an export path becomes \: after a verbatim
  prefix (what canonicalize returns) / is not a separator, giving
  os error 123.
- One is_folder check for output_path and export.queue.add, so a
  trailing \ no longer numbers several queued sequences.
2026-10-08 15:57:33 +01:00
Manuel CamachoandClaude Opus 5.5 abd62eefa9 Cargo.lock: tts crates at 0.4.0 after the version bump
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 07:38:19 -07:00
Manuel CamachoandClaude Opus 5.5 036daaaf12 Merge upstream main (0.4.0 version bump) into tts-narration
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 07:38:10 -07:00
midasdfandCodex d67d225da5 M10: use Japanese vertical forms and origins for titles
Co-Authored-By: Codex <noreply@openai.com>
2026-10-08 23:20:38 +09:00
Brandon ThomasandClaude Opus 5.5 5231852443 Bump version to 0.4.0 (from 0.3.0) for release
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:01:39 -04:00
Manuel CamachoandClaude Opus 5.5 beebdbcd39 M7.12: natural Text to Speech voices (Kokoro-82M in pure Rust)
- filmcraft-tts (feature kokoro): Kokoro-82M on candle, implemented from
  the StyleTTS 2 / iSTFTNet / ALBERT papers, config.json and the
  checkpoint's tensor shapes; official .pth read with candle's restricted
  pickle reader; voice packs from their zip. Neural Voice: tts-text
  phonemize -> Kokoro per chunk -> pauses -> pitch via audio-dsp.
- catalog: 9 US English voices (model card grade C+ or better) + CMUdict,
  every file pinned to a revision and SHA-256 (336 MB).
- speech: generic download_files (same verified downloader).
- engine: tts.render (background synthesis into Session::tts_cache),
  tts.downloadVoices (background job, cancellable), voices list with
  package state; feature neural-voices (default on in the desktop app).
- ui: natural/basic voice groups, consent box (size, sources, licences),
  download progress + Cancel, synthesis never blocks the window;
  ui.set {tts:{...}} for agents.

Measured: Whisper-base WER on 50 original sentences 2.1-3.6 % for all 9
voices (built-in robotic voice 96.6 %); 2.2-2.3x real time on CPU.
Gates: fmt, clippy, layers, assets, wasm pass; tests pass except
shortcuts_tests::assign_reassigns_conflicts_and_undoes (fails on main)
and the frame pool test (flaky under load, passes alone).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 06:52:30 -07:00
Manuel Camacho 6367fc1444 M7.11: tts-text: keep ? and ! for intonation
Fix round from Claude's P2b review. The normalizer's terminator run now picks
the strongest terminator (? over ! over .) and carries it through a new
backwards-compatible Token::SentenceEndWith(char) (SentenceEnd still means
"."); phonemize attaches ? and ! to the sentence's last word so Kokoro can
shape intonation: "Ready?" -> "ɹˈɛdɪ?", "Go!" -> "ɡˈO!", "Really?!" -> ?.

Also converts the remaining direct indexes in non-test code to
get()/get_mut()/iterators (letter_to_sound slices, the tion/sion and
first-vowel stress writes, the scanner's char access).

New cases: nine terminator tests in normalize (incl. "Is it 1999?",
"Call 555-0134!", "Wait... what?", "Are you there?!") and nine phonemize
assertions (Ready?, Go!, Done., No!!, Are you there?!, ...). All
SentenceEnd-matching tests updated.

Gates: fmt, clippy -p filmcraft-tts-text --all-targets -D warnings, tests
(37 suites), xtask layers, xtask wasm - all green.

🤖 Written by Merlin (Hermes Agent, profile merlin)
2026-10-08 06:52:30 -07:00
Manuel Camacho 82d411c7e2 M7.11: tts-text: English pronunciation (CMUdict + letter-to-sound) for Kokoro
New module phonemize in filmcraft-tts-text: normalize(script) -> words ->
Kokoro phoneme symbols, the interface Claude's Kokoro code (P3) calls. Public
API per the brief's contract: Lexicon (CMUdict parser, 16 MiB / 500 000-entry
caps, malformed lines skipped and counted), Overrides (case-insensitive,
raw /phonemes/ or respellings, capped at 1 000 entries of 200 chars),
Chunk::Phonemes/Pause with MAX_CHUNK_CHARS = 400, phonemize(), and SYMBOLS —
the exact character set emitted (ARPAbet→Kokoro table from the brief, stress
marks ˈ/ˌ before the vowel, function words unstressed, punctuation attached).

Lookup order: overrides → dictionary (first pronunciation) → morphology
(-s/-'s with voicing, -ed, -ing, -er, -ly, un-, re-) → letter-to-sound rules
written from scratch (tion/sion, ture/sure, ough, igh, digraphs, silent e,
soft c/g, doubled consonants, final le). Hyphenated words split per part.

Tests: CMUdict parse table (alternates, comments, malformed counting, caps),
46 exact phonemize cases, 36 letter-to-sound cases, chunk splitting at
commas ≤ 400 chars, a symbol-set test over all normalizer inputs plus
hostile ones, and catch_unwind hostiles (huge, empty, emoji, RTL, NUL,
10 000-letter word, 1 001 overrides). Optional measured-accuracy test
(FILMCRAFT_CMUDICT): held-out CMUdict words → 9.1% word accuracy, 37.9%
phoneme error rate (229 words, 2026-10-08). CMUdict is downloaded with the
voices at run time, never committed.

Gates: fmt, clippy -p filmcraft-tts-text --all-targets -D warnings, tests
(37 suites), xtask layers, xtask wasm (44 crates) — all green.

🤖 Written by Merlin (Hermes Agent, profile merlin)
2026-10-08 06:52:30 -07:00
Manuel Camacho a38ddcbda4 M7.10: tts-text: text normalizer for narration (English)
New L2 crate filmcraft-tts-text: normalize(script, lang) -> Vec<Token> for the
Text to Speech text front end (P2a). English only (EnUs/EnGb), per the amended
brief. Handles cardinals to 999 999 999 999, negatives, decimals, ordinals,
money ($, €, £), percent, years (1100-2099), clock times, numeric and
abbreviated dates, phone-like digit groups, Mr./Mrs./Dr./St./Mt./vs./etc./e.g./
i.e./a.m./p.m., spelled acronyms, &/+@#/, and [pause Ns|ms] markers capped at
10 s. Pure Rust, deps serde + thiserror only, wasm-clean.

Never crashes (AGENTS.md §0): >1 MiB is NormalizeError::TooLong, oversized and
leading-zero numbers are read digit by digit, malformed pause markers become
plain text, all slicing is get()/boundary-safe, and a catch_unwind suite covers
empty/whitespace/oversized/10k-digit/$$$$/mixed-script/emoji/RTL/combining/NUL
inputs plus determinism.

Fix round (Claude review): a '.' after a number now ends the sentence (year
rule and money/decimal follow-through), so "in 1999." reads as a year with a
SentenceEnd and "$1,234,567." gets its final SentenceEnd; new table cases for
sentence-final numbers and the year rule under punctuation (., ,, !, ()); the
remaining direct indexes in non-test code converted to get()/first()/last().

Gates: fmt, clippy -p filmcraft-tts-text --all-targets -D warnings, tests
(25 suites, ~140 table cases), cargo xtask layers, cargo xtask wasm all green.
cargo xtask ci not run this round (Claude runs it when merging); it stops on a
pre-existing filmcraft-engine shortcuts test failure unrelated to this crate.

🤖 Written by Merlin (Hermes Agent, profile merlin)
2026-10-08 06:52:30 -07:00
Manuel CamachoandClaude Opus 5.5 66ba5bbb3c M7.9: Text to Speech narrations (panel, tts.* commands, built-in voice)
Write what a narrator says, pick a voice, pitch and pace, and add it to
the timeline. Selecting a narration clip loads it back into the panel;
Save regenerates it in place as one undo step, keeping the clip's start
and length (longer speech kept whole in the file, shorter speech padded
with silence).

- filmcraft-tts (L2): Voice trait, voice catalogue, pause markers
  ([pause 1s]), and an original formant synthesizer (Basic Female/Male;
  robotic placeholder until the neural voices land).
- project: Narration + Project::narrations (schema v13, no-op migration).
- engine: tts.voices/preview/create/edit/inspect; narrations never
  overwrite clips (free targeted track, free track, else a new track).
- ui: Text to Speech panel beside Properties, preview playback,
  click-to-edit, automation ids.

Gates: fmt, clippy -D warnings, layers, assets and wasm (rustup
toolchain) pass. Tests: all pass except three that fail identically on
main (61238bf) on this machine: shortcuts_tests::assign_reassigns_
conflicts_and_undoes, gpu fx_tests::gpu_effects_match_cpu_exactly, and
the frame pool test, which passes when run alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 06:52:23 -07:00
207b179428 README Downloads: where to get it first, then builds with Windows before macOS (#271)
Co-authored-by: Brandon Thomas <shared2@storyteller.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 09:50:42 -04:00
Tom 0bf83a27c0 Windows: platform modifier names in status-bar and mask hints
The Selection and Zoom status-bar hints and the mask-vertex tooltip
named Mac keys (Opt, Cmd) on every platform. Show Alt/Ctrl on
Windows and Linux, matching the keys the code actually reads
(egui `alt` / `command`). macOS text is unchanged.
2026-10-08 14:47:40 +01:00
86a531ad22 Release: FreeBSD, Flatpak (x86_64 + aarch64) and AppImage zsync; README Downloads (#267)
- release.yml: new `flatpak` jobs (x86_64 on ubuntu-24.04, aarch64 on ubuntu-24.04-arm) that
  repackage the Linux tarball as filmcraft-<v>-linux-<arch>.flatpak, and a `freebsd` job that
  builds filmcraft-<v>-freebsd-x86_64.tar.gz in a FreeBSD 14.3 VM; both feed the draft release.
- AppImages embed gh-releases-zsync update information and ship a .zsync beside them; the
  Linux job checks both.
- Jobs that sign nothing (version, Linux, Flatpak, FreeBSD, web) no longer use the `release`
  environment, so a dispatch on a branch dry-runs them; the signing jobs and the draft-release
  job stay in the environment, so nothing is published from a branch.
- packaging/linux/flatpak-bundle.sh + bundle manifest (same runtime and finish-args as the
  Flathub manifest; packaging-lint checks they agree), packaging/freebsd/package.sh (with a
  --dry-run that packaging-lint runs), and a nightly FreeBSD build workflow.
- README: Downloads section listing every build; docs/releasing.md updated.

Co-authored-by: Brandon Thomas <shared2@storyteller.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 09:41:42 -04:00
wstockandClaude Opus 5.5 06abca1bb9 shortcuts: Delete clears the selected timeline clips (#243)
Windows and Linux keyboards' Delete key is the forward-delete key. In
FilmCraft Default it was bound only to Project ▸ Clear (application-
wide), so with the Timeline focused it ran project.delete, which does
nothing without a Project-panel selection: the selected clips stayed.
Backspace (the Mac "delete" key) already cleared them.

Add a Timeline-panel binding of Delete to edit.clear, as Premiere has
it; in the Project panel and application-wide Delete still runs
Project ▸ Clear.

Tests: shortcuts_tests::delete_clears_timeline_clips_and_project_items
(Delete resolves to edit.clear with the Timeline focused and to
project.delete otherwise, macOS and Windows) and
keyboard_ui::delete_key_clears_the_selected_timeline_clip (headless app:
D selects a clip, Delete removes it, undo restores it). Both failed
before the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:14:52 +00:00
Brandon ThomasandClaude Opus 5.5 08735f3103 Bump version to 0.3.0 (from 0.2.1) for release
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 08:12:49 -04:00
wstockandClaude Fable 5.1 32764392f1 control: Ctrl+ chords through ui.key fire shortcuts off macOS (#245)
Bindings are written Cmd+… and parse to `command`; egui's matching then
requires `command`. A physical Ctrl press off macOS carries it
(egui-winit sets command = ctrl there), but ui.key built its event from
parse_shortcut, which mapped "Ctrl" to `ctrl` alone, so an injected
Ctrl+Z matched nothing while Cmd+Z worked. ui.click's `ctrl` modifier
had the same gap.

Off macOS "Ctrl" and "Cmd" are one key (the engine's Chord::effective
already says so), so parse_shortcut maps both to `command` there, and
the control channel synthesises presses the way the OS reports them:
a press of either carries both flags. macOS is unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 11:08:19 +00:00
Ahmed HosnyandKimi 9bf6595ccd Playback: fall back to a working output device when the ALSA default cannot be opened
On a machine whose ALSA default PCM is misconfigured (missing 99-pipewire-default.conf symlink, #23/#106) the host default could not be opened: playback ran without sound and without any user-facing message. The device picker now falls back to the first output device that can open, logs why, and a new defaulted AudioOut::note() surfaces the note in the status bar (first frame and when playback starts); a machine with no device at all reports it too.

🤖 Generated with [Kimi Code](https://github.com/storyteller/kimi-code)
Co-Authored-By: Kimi <noreply@storyteller.dev>
2026-10-08 13:59:59 +03:00
wstockandClaude Fable 5.1 acf56bcd27 engine: project.delete takes explicit bin ids without a Project-panel selection (#244)
A caller that names its targets does not need a selection: enabled_for
re-runs the enablement check with the named ids standing in for it. But
named_items only recognised project items, and project.delete's `items`
take bin ids too (the Project panel's selection holds both), so
{"items":[<bin ids>]} with nothing selected was still "not available
right now: select an item in the Project panel".

Count a named bin as a target, except the project's own top bin, which
no command acts on.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 10:54:42 +00:00
WavJaby 92c6aa2b63 chore(lints): preserve behavior in workspace clippy cleanup
- Move conditional no-op match arms into guards

- Preserve renderer predicate order with De Morgan's law

- Strict clippy, fmt, layers, assets and WASM pass on Rust 1.95

- Clean engine runs match main: 379 pass, 10 fail, 3 ignored

- Full CI is not green; upstream baseline failures remain
2026-10-08 18:22:27 +08:00
Gert Pellin 2d6c4d3a3d Desktop: write log records to standard error and a rotating log file
The desktop app installed no logger, so every log::warn!/log::error! in
the workspace was discarded: GPU compositor fallbacks, audio output
problems, a panicked frame job or audio mixer, container warnings. A
launch from a desktop menu has no terminal either, so there was nothing
to attach to a bug report.

apps/filmcraft/src/logging.rs (no new dependency):
- standard error plus <data dir>/Logs/filmcraft.log, next to the crash
  logs (--data-dir / FILMCRAFT_DATA_DIR respected, Help > Reveal Log Files
  opens the folder); each launch rotates the previous log to
  filmcraft.1.log and .2.log, so the log of a run that crashed survives
  the next start; the file is capped at 16 MiB
- records logged before the data directory is known are buffered
  (bounded) and written once the file is attached
- default levels: info for filmcraft*, warn for everything else;
  RUST_LOG replaces them with env_logger-style directives
- never panics: an unusable directory or a failed write leaves standard
  error as the only sink; a poisoned lock is tolerated
- the realtime audio callbacks do not log; a record holds the lock for
  one write of an already formatted line

The logger is installed first in main (before the panic hook) and the
file is opened after argument parsing (--help, --version and usage
errors leave no file). The start-up eprintln!s in main (auto-save
unavailable, open/recovery errors, no window) now go through the logger.
Documented in docs/contributing.md (environment variables, Logs).
2026-10-08 11:37:02 +02:00
MrCloudy2andClaude Opus 5.5 fcdcc07359 #219: right-clicking a cut opens the edit point menu
Right-clicking a clip's edge on the timeline now selects that edit point and
opens Premiere's edit point menu instead of the clip menu: Ripple Trim In,
Ripple Trim Out, Roll Edit, Trim In and Trim Out (each re-selects the cut with
that trim type, greyed out when the needed side has no clip), Apply Default
Transitions, and Join Through Edits. The body of a clip still opens the clip
menu.

Join Through Edits from this menu joins only the right-clicked cut (and its
linked partner): `sequence.joinThroughEdits` takes a new `cut: [left, right]`
parameter, since `clips: [id]` also matched any other through edit touching
the same piece.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:33:19 +02:00
Gert Pellin d018fb9abb Contributors: credit switch87 as Gert Pellin 2026-10-08 11:27:34 +02:00
Gert Pellin 7a042cd72f README: install on Gentoo from the ::snakebyte overlay 2026-10-08 11:25:54 +02:00
MrCloudy2andClaude Opus 5.5 bee690f050 #220: Unlink leaves only the right-clicked clip selected
Unlinking from the timeline clip menu cleared the link, but the picture and
its sound stayed selected together, so dragging the picture still moved the
sound and Unlink looked like it did nothing. The clip that was right-clicked
is now the only one selected after Unlink, and can be dragged away straight
away. Link leaves the selection as it was.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:53:54 +02:00
Ahmed HosnyandKimi 58ce158ed0 contributors: credit ahhosnyas (people.ahmedhosnypro) as Ahmed Hosny
Own entry in contributors/people.toml for the About window, per AGENTS.md.

🤖 Generated with [Kimi Code](https://github.com/storyteller/kimi-code)
Co-Authored-By: Kimi <noreply@storyteller.dev>
2026-10-08 11:39:09 +03:00
Ahmed Hosny f14d8b7ec9 Merge branch 'main' into fix/linux-dev-desktop-icon 2026-10-08 11:37:31 +03:00
Ahmed Hosny 538a329935 linux: install a dev desktop entry so the taskbar shows the app icon
A dev run (cargo run) has no installed .desktop entry or hicolor icon for
the app id the window sets (ai.storyteller.filmcraft), so KDE (Wayland and
X11) falls back to the generic Wayland icon in the taskpanel and the
window topbar. Packaging installs both; development did not.

At startup on Linux, when no ai.storyteller.filmcraft.desktop exists in
any XDG data dir, install one into $XDG_DATA_HOME/~/.local/share
(applications/) plus the 256/512 hicolor PNGs, with Exec pointing at the
running binary. A packaged entry is never overridden; failures print a
warning and the app carries on (AGENTS.md §0).

🤖 Generated with [Kimi Code](https://github.com/storyteller/kimi-code)
Co-Authored-By: Kimi <noreply@storyteller.dev>
2026-10-08 11:37:31 +03:00
wstockandClaude Fable 5.1 6068bfc589 export: keep alpha in PNG/TIFF sequences with Include Alpha Channel (#160)
The export pipeline flattened every frame over black
(`over_black_rgba8`), so a PNG sequence of a sequence with transparency
came out on a black background, while Export Frame already wrote straight
RGBA. `ExportSettings.alpha` (serde default false, Premiere's default)
makes the pipeline emit straight sRGB RGBA for PNG and TIFF sequences and
`encode_still` keep the channel in TIFF; BMP and video formats ignore it.
Exposed as the `alpha` flat parameter of file.exportMedia and an
"Include Alpha Channel" checkbox in Export mode's Video section, shown
for PNG and TIFF only.

Fixes #160

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 08:31:37 +00:00
WavJaby aa66dd9c8d fix(import): use the open Project bin as destination
Resolve the shown Project bin for file-dialog and dropped imports. Capture the browser destination before asynchronous media reads so navigation cannot redirect an in-flight batch.
2026-10-08 16:27:48 +08:00
WavJaby 3e39952e0b fix(web): preserve distinct same-name media imports
Allocate an unused virtual path for every browser import so equal-size media cannot replace existing sources. Add an OPFS recovery regression for issue #194.
2026-10-08 16:18:07 +08:00
wstockandClaude Fable 5.1 0e129e2811 #230: jobs time themselves with web_time, so Create Proxies and mask tracking don't panic on the web
The web build runs every job synchronously, and four job bodies (proxies,
mask tracking, video and audio previews, Project Manager) still called
std::time::Instant::now() for their "Done in N s" report — which panics
with "time not implemented on this platform" on wasm32 and took the whole
app down. They now use web_time::Instant like the rest of the engine
(std::time::Instant on native targets). autosave imports its clock from
web_time too, and a regression test scans the crate's non-test sources
for std::time::Instant so the next copy-pasted job fails cargo test
instead of the web app.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 07:45:37 +00:00
txiki739andClaude Opus 5.5 4af4765278 Merge storytold/main into i18n-spanish-panels
Brazilian Portuguese (#176) moves to the catalog system as i18n/pt-br.tsv (the
menu entries unchanged) and joins the general.interfaceLanguage preference. The
new UI text from main is wrapped and translated into Spanish: the credits
window, Text Properties, H.265 export rows, Clip Speed / Duration, Color Matte
dialogs, the voice-over discard button, the time-left suffix and the audio
output status messages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 08:59:32 +02:00
Claude 7fbb223707 #197, #193: web recovery retries failed snapshot writes and keeps skipped snapshots dirty
The OPFS recovery snapshot (apps/filmcraft-web) had two problems:

- #197: opfs::write never reported failures, and Autosave recorded the
  revision as snapshotted before the write ran. A failed write was only
  retried after the next edit.
- #193: the first tick of a clean session wrote {"dirty": false} to the
  meta. Starting with ?norecover or ?fresh therefore discarded a
  snapshot that was never reopened.

The decision of when to snapshot or mark clean moves to a pure module,
recovery_policy, tested natively in tests/recovery_policy.rs. Failed
writes (snapshot or clean meta) are retried after the 5 s interval
without another edit, and the meta is only marked clean to retire a
snapshot this session wrote. opfs::write_then reports whether a queued
write landed (coalesced writes report the newest one's result).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012HrAqZmrrDLRpxBJatxvaJ
2026-10-08 04:37:53 +00:00
carlitosteso3andClaude Opus 5.5 7392a69d0f #208: Go to In / Go to Out move to the sequence start / end without marks
The Program monitor's Go to In and Go to Out buttons (Shift+I / Shift+O) did
nothing when the sequence had no In / Out mark. Like Premiere (and like the
Source monitor already did), they now go to the start / end of the sequence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:43:49 -05:00
Brandon ThomasandClaude Opus 5.5 61238bf8fc Release CI: macOS job on macos-15-xlarge
Custom macOS larger runners in a runner group do not pick up jobs; the
built-in M2 Pro label does. Only release.yml uses it, and the
actions_macos_xl budget caps the monthly spend.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 22:07:57 -04:00
Brandon ThomasandClaude Opus 5.5 08b769a92a Release CI: macOS job on the org macos-release runner
The standard macOS runners are shared with all CI and queue for an hour or
more. The macos-release runner (GitHub-hosted M2 Pro) is in a runner group
usable only by release.yml on the release branch, with a monthly spend cap
on the actions_macos_xl SKU.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:54:54 -04:00
Brandon Thomas 5812509d3b Merge pull request #213 from carlitosteso3/fix/speed-dialog-undo-201
Speed/Duration opens its dialog; dragging an effect parameter is one undo step (#201)
2026-10-07 21:25:58 -04:00
Brandon ThomasandClaude Opus 5.5 ee8f886332 #201: each press of a drag begins its own undo step
A drag value changes only once the mouse moves, after the press frame, so sending `begin`
only on the press frame merged two drags of the same parameter into one undo step.
Remember the press that began a step by its start time instead; unit test drives two drags
through `run` with a real egui context (fails on the press-frame version).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:25:44 -04:00
carlitosteso3andClaude Opus 5.5 70aa661586 #201: Speed/Duration opens its dialog; dragging an effect parameter is one undo step
- The clip menu's Speed/Duration… applied 50% at once, and Cmd+R / Clip ▸
  Speed/Duration… ran it at 100% with no dialog. It now opens Clip Speed /
  Duration (speed, the resulting duration, Reverse Speed, Ripple Edit, Time
  Interpolation), seeded from the selected clip.
- effects.setParam took one undo step per frame of a drag, so Undo after
  dragging Time Remapping's speed (or any Effect Controls value) crept back in
  small steps. It takes `merge` / `begin` like the mixer and masks, and Effect
  Controls sends them while the mouse button is down: a drag is one step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:25:44 -04:00
Brandon ThomasandClaude Opus 5.5 87c4d1516e About: contributor and model credits compiled in (grab bag + table, self-submitted names) (#215)
* About: contributors compiled in from contributors.json (grab bag + table, sortable, name toggle)

Help ▸ About FilmCraft gets tabs About · Contributors · Models, following the craftrules standard
(standards/contributors.md): contributors/contributors.json is generated by
craftrules/scripts/contributors.py and baked into the binary by crates/ui-egui/build.rs as static
tables (nothing read at run time; the web build has them too). Each contributor line: @username,
opt-in display/real name, PRs, commits, lines +/−/Δ, binary assets +/−, first/last commit.
Sortable by every column; alphabetical ignores case and the leading @. Tabs and credits controls
register automation ids (about.tab.*, about.credits.*).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Contributors docs: craftrules is two levels up (../../craftrules)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Contributors: self-submitted names in contributors/people.toml; AGENTS.md credit-yourself section

build.rs reads contributors/people.toml next to the generated contributors.json and shows a name only
for entries the generator verified as written by that person (craftrules standards/contributors.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Contributors: regenerate (verified people.toml entries)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:16:52 -04:00
Brandon Thomas c337dcce01 Merge pull request #210 from cirocueto2013-wq/fix/hostile-input-bounds
fix: validate hostile project, export and MXF inputs
2026-10-07 20:53:11 -04:00
Brandon ThomasandClaude Opus 5.5 de70468ab6 #210 review: bound MXF reads by the file, keep legacy projects opening, allow 16K
- mxf: read_pcm bounds a request by the sample frames the file actually stores (plus 16M
  samples of zero padding) instead of a fixed 16,777,216-sample cap, so whole-clip reads of
  long stereo MXF (transcription) work again; PCM/AES3 chunk buffers are clamped to the source
  length instead of a 256 MiB cap; decode_pcm's output is bounded by its input, so its
  fixed cap goes (the AES3 >8-channel refusal stays; a request is also bounded by the source
  length divided by the channel count). read_sample keeps only the source-length bound.
- mxf: index_entries no longer fails open() past 1,048,576 entries (the sparse BTreeMap
  already bounds memory by real entries); long recordings open.
- project/format: new Sequence/Track/CaptionTrack::check_bounds() hold only the hostile-value
  bounds (settings, marks, overflowing time ranges); project load and export ranges use it, so
  projects that break only structural rules (overlap, zero duration) still open and export.
  check() = check_bounds() + the structural rules, as before.
- project: frame-size cap raised to 268,435,456 pixels, 32768 per side (16K, 16384x8192).
- engine: checked_u32_p accepts integer-valued floats (1920.0); renderable_sequence returns
  an EngineError (new Session::try_render_program_at, used by automation render) instead of
  eprintln.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:52:53 -04:00
cirocueto2013-wqandCodex 07d85b0f02 fix: validate hostile project, export and MXF inputs
Validate staged speed edits before adopting their timeline state.

Co-authored-by: Codex <noreply@openai.com>
2026-10-07 20:38:43 -04:00
Brandon Thomas a847671dbb Merge pull request #212 from storytold/test-188
#188: regression test — a copied Graphic clip pastes at the playhead
2026-10-07 20:37:34 -04:00
5ee3ff64e9 Web: Link Media Locate… links the chosen file to the selected missing clip (#111) (#174)
* Web: Link Media Locate… links the chosen file to the selected missing clip (#111)

Link Media ▸ Locate…, `media.attachProxies` and `media.reconnectFullRes` all
need the chosen file's path back synchronously so they can call the engine
with `item` + `path`. The existing `pick_files` hook imports every chosen
file as a new project item, so on the web build (where the browser picker is
asynchronous and the hook returned `Vec::new()`) the command ran without a
path and the file silently landed as a new project item instead of being
relinked. On native the same hook worked because rfd returns paths
synchronously.

Add a separate `pick_file_for_relink` hook that returns the chosen file's
virtual path synchronously. Native hosts open an rfd dialog; the web host
registers the file in `/files/<name>` via a pending slot and runs
`media.relink` itself when the picker resolves. The three call sites prefer
the new hook and fall back to `pick_files` when the host has not installed
it (so the import flow used by every other dialog stays unchanged).

Fixes #111

Co-Authored-By: ninso112 <ninso112@pm.me>

* Co-ship: Luna161 ack'd and reviewed fix/111-link-media-locate

Reviewed the pick_file_for_relink hook routing, the take_files relink path,
and the regression test. Looks correct end to end; ready for upstream review.

Co-Authored-By: ninso112 <ninso112@pm.me>

* #111: relink hint names its command; no pending state after a cancelled picker

- RelinkHint carries the command (media.relink / attachProxies / reconnectFullRes), so the
  web host no longer runs media.relink for Attach Proxies and Reconnect Full Resolution.
- The web picker hands the hint to its own callback instead of a thread-local slot, so a
  cancelled picker can't turn the next ordinary import into a relink.
- Replace the hook-only unit tests with a UI test that drives Link Media ▸ Locate….

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: ninso112 <ninso112@pm.me>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:35:58 -04:00
Brandon Thomas 4f98721cab Merge pull request #209 from cirocueto2013-wq/fix/cpal-voiceover-recovery
fix: support integer CPAL devices and recover voice-over/audio failures
2026-10-07 20:35:54 -04:00
Brandon Thomas a4d65d3263 Merge pull request #204 from DWAA1660/fix/199-clipboard-shortcuts
#199: make Ctrl+C / Ctrl+X / Ctrl+V work on Windows and Linux
2026-10-07 20:35:50 -04:00
Brandon Thomas 6d904c1b37 Merge pull request #192 from cirocueto2013-wq/fix/editing-workflow-stability
Stabilize project lifecycle, timeline, audio and export workflows
2026-10-07 20:35:46 -04:00
Brandon Thomas d4ca3594ec Merge pull request #203 from carlitosteso3/fix/ripple-razor-snap-164
Ripple delete names the sync-locked track in the way; playhead snaps to cuts (#164, items 2, 3 and snap)
2026-10-07 20:35:42 -04:00
Brandon Thomas cd7395ec7c Merge pull request #207 from carlitosteso3/fix/ripple-trim-targets-164
Q/W ripple trim only the selected clips' tracks (#164)
2026-10-07 20:35:37 -04:00
carlitosteso3andClaude Opus 5.5 41659ae769 #164: Q/W ripple trim only the selected clips' tracks
With clips selected under the playhead, Ripple Trim Previous/Next Edit to
Playhead (and the plain trims) now trim only their tracks and linked partners,
like Add Edit; with none selected they use the targeted tracks as before. All
tracks are targeted by default, so W on a V1 clip also cut the music on A2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:50:39 -05:00
carlitosteso3andClaude Opus 5.5 e7e96b247f #164: ripple delete says which sync-locked track blocks it; playhead snaps to cuts
- A refused ripple delete / ripple trim now names the sync-locked track whose
  clip is in the way and how to proceed, instead of a generic "would break sync"
  (the edit is refused the same way Premiere refuses it).
- Dragging the playhead no longer snaps onto its own position (it stuck and moved
  in SNAP_PX jumps), snaps on the initial press too, and "Snap playhead in
  Timeline when Snap is enabled" is on by default, as in Premiere.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:47:21 -05:00
txiki739andClaude Opus 5.5 c7ddd4f314 tests: shortcut conflict test compares against the platform's own baseline
On Windows and Linux the macOS Control key folds into the primary modifier, so
the default set already reports conflicts there (documented in shortcuts.rs).
The test now checks that assigning adds exactly one conflict and that undo
restores the original set, on every platform.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 01:41:48 +02:00
txiki739andClaude Opus 5.5 3d67c4b73d tests: the MPEG oracle reads ffprobe's best-effort timestamps
Recent ffprobe versions report PTS=N/A for a program-stream picture whose
timestamp is inferred, so the oracle dropped that frame and compared 29 of 30.
best_effort_timestamp keeps the inferred time and all 30 frames are compared.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 01:41:48 +02:00
txiki739andClaude Opus 5.5 b6cdc9dd7a i18n: Spanish for the whole interface, catalogs and a remembered language
Translations move from Rust tables to one catalog per language
(crates/ui-egui/src/i18n/<code>.tsv), looked up through HashMaps. UI code
wraps its strings in tl!("…") and tlf!("…{name}", name); names from
registries (effects and parameters, settings, commands, panels, workspaces,
export and effect presets) are translated where they are drawn, so command
ids, automation ids and preference values stay English.

Spanish now covers menus, panels, dialogs, the Settings pages, the Effects and
Effect Controls registry, the Keyboard Shortcuts dialog and status messages
(about 3,350 entries). Tests fail when a tl!/tlf! literal, a menu label or a
registry name has no Spanish entry, and check catalog syntax, duplicates,
placeholders and ellipses. Searches match the Spanish label as well as the
English source. Window titles keep their English egui ids.

The language is stored in the general.interfaceLanguage preference (also in
Settings > General) and restored on startup; a failed preferences write still
switches the interface and says it could not be saved. Template values are
inserted literally, so a file name containing braces is not re-substituted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 01:41:48 +02:00
cirocueto2013-wqandCodex 7151d329d6 fix: support integer CPAL devices and recover voice-over/audio failures
Co-authored-by: Codex <noreply@openai.com>
2026-10-07 20:33:33 -03:00
DWAA1660andClaude Opus 5.5 f4221cd1b5 #199: make Ctrl+C / Ctrl+X / Ctrl+V work on Windows and Linux
egui-winit reports the clipboard shortcuts as Event::Copy / Cut / Paste
instead of key presses, so the shortcut bindings never matched them.
Outside text fields, add the key press each clipboard event stands for
before matching bindings, and put copied clip names on the system
clipboard so Ctrl+V is delivered right after a copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QQNunPf8nMeAQi6M92RT68
2026-10-07 16:57:38 -06:00
cirocueto2013-wqandCodex 1fd30aa29d engine: bound subclip media resolution and reject cycles
Co-authored-by: Codex <noreply@openai.com>
2026-10-07 19:56:58 -03:00
531 changed files with 83896 additions and 7096 deletions

No files matched your search

+58
View File
@@ -0,0 +1,58 @@
# Builds FilmCraft on FreeBSD, inside a FreeBSD VM on a Linux runner (GitHub has no FreeBSD
# runners), then runs packaging/freebsd/package.sh, the script the release job runs, and checks
# the tarball. A broken FreeBSD build or package fails here before it can fail a release. Nothing
# is uploaded and no secrets are used. (The test suite isn't run here yet: the ffmpeg-oracle and
# golden-image tests need tools and a GPU the VM doesn't have.)
name: FreeBSD
# The VM build takes an hour or more, so it runs nightly on main rather than on every push or
# dependency change: a drifted dependency still shows up within a day. PRs that change the
# FreeBSD packaging itself still run it.
on:
schedule:
- cron: "37 2 * * *"
pull_request:
paths:
- .github/workflows/freebsd.yml
- packaging/freebsd/**
- packaging/env.sh
workflow_dispatch:
concurrency:
group: freebsd-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
freebsd:
name: FreeBSD 14 (build + package)
runs-on: ubuntu-latest
timeout-minutes: 240
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Build and package in a FreeBSD VM
# vmactions/freebsd-vm v1.5.9, pinned by commit (it runs our build).
uses: vmactions/freebsd-vm@c46abacb49f09938ca4e1702d15d836285d694cc # v1.5.9
with:
release: "14.3"
usesh: true
cpu: 4
mem: 12288
# Nothing to bring back; copying the target dir out of the VM fails on large builds.
copyback: false
# The same packages as release.yml's freebsd job: winit/wgpu (X11, Wayland, Vulkan,
# Mesa), the file dialogs (GTK 3), fonts, and audio (cpal over ALSA: alsa-lib).
prepare: |
pkg install -y bash git curl pkgconf cmake python3 libxkbcommon wayland libX11 libXcursor libXrandr libXi libxcb mesa-libs vulkan-loader gtk3 fontconfig freetype2 alsa-lib
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain stable
run: |
set -e
. "$HOME/.cargo/env"
git config --global --add safe.directory "*"
rustc --version
export CARGO_TARGET_DIR=/var/tmp/filmcraft-target CARGO_BUILD_JOBS=4
bash packaging/freebsd/package.sh
tar -tzf dist/release/filmcraft-*-freebsd-x86_64.tar.gz \
| grep -E '/bin/filmcraft$|/bin/filmcraft-cli$|/share/applications/ai\.storyteller\.filmcraft\.desktop$|/share/doc/filmcraft/LICENSE-MIT$'
+20 -2
View File
@@ -23,7 +23,7 @@ jobs:
env:
ACTIONLINT_VERSION: 1.7.12
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install tools
run: |
sudo apt-get update
@@ -35,6 +35,8 @@ jobs:
"$RUNNER_TEMP/actionlint" -color
- name: shellcheck
run: shellcheck packaging/*.sh packaging/*/*.sh
- name: Linux tarball installer
run: python3 packaging/linux/test_install.py
- name: PowerShell
shell: pwsh
run: |
@@ -54,4 +56,20 @@ jobs:
desktop-file-validate packaging/linux/ai.storyteller.filmcraft.desktop
sed -e 's/@VERSION@/0.0.0/' -e 's/@DATE@/2026-01-01/' packaging/linux/ai.storyteller.filmcraft.metainfo.xml.in >"$RUNNER_TEMP/ai.storyteller.filmcraft.metainfo.xml"
appstreamcli validate --no-net --explain "$RUNNER_TEMP/ai.storyteller.filmcraft.metainfo.xml"
python3 -c "import yaml; m = yaml.safe_load(open('packaging/linux/flatpak/ai.storyteller.filmcraft.yml')); assert m['id'] == 'ai.storyteller.filmcraft' and m['command'] == 'filmcraft'"
python3 - <<'EOF'
import yaml
src = yaml.safe_load(open("packaging/linux/flatpak/ai.storyteller.filmcraft.yml"))
bundle = yaml.safe_load(open("packaging/linux/flatpak/ai.storyteller.filmcraft.bundle.yml"))
for m in (src, bundle):
assert m["id"] == "ai.storyteller.filmcraft" and m["command"] == "filmcraft", m["id"]
for key in ("runtime", "runtime-version", "sdk", "finish-args"):
assert src[key] == bundle[key], f"Flatpak manifests disagree on {key}: {src[key]} vs {bundle[key]}"
# Both install the app and the CLI/MCP server (#346).
import re
def programs(m):
return {p for c in m["modules"][0]["build-commands"] for p in re.findall(r"/app/bin/([\w.-]+)", c)}
assert programs(src) == programs(bundle) == {"filmcraft", "filmcraft-cli"}, f"Flatpak manifests install {programs(src)} vs {programs(bundle)}"
print("Flatpak manifests ok")
EOF
- name: FreeBSD package layout (stub binaries)
run: bash packaging/freebsd/package.sh --dry-run
+199 -35
View File
@@ -1,9 +1,12 @@
# Release pipeline: every push to `release` builds signed installers for every platform and
# creates or updates the draft GitHub Release "FilmCraft v<version>". See docs/releasing.md.
#
# All jobs run in the `release` environment (restricted to the `release` branch), which holds
# the signing secrets. Every secret is optional: missing ones produce unsigned artifacts and a
# warning, never a failed build.
# The jobs that sign (macOS, Windows) and the one that publishes the draft run in the `release`
# environment (restricted to the `release` branch), which holds the signing secrets. Every secret
# is optional: missing ones produce unsigned artifacts and a warning, never a failed build. The
# jobs that sign nothing (version, Linux incl. riscv64, Flatpak, FreeBSD, web) get no environment
# and no secrets, so a dispatch on any other branch dry-runs them; there the signing jobs are
# refused, so the draft-release job (which needs them) is skipped and nothing is published.
name: Release
on:
@@ -35,13 +38,12 @@ jobs:
version:
name: version
runs-on: ubuntu-latest
environment: release
outputs:
version: ${{ steps.v.outputs.version }}
date: ${{ steps.v.outputs.date }}
prerelease: ${{ steps.v.outputs.prerelease }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- id: v
name: Resolve version
run: |
@@ -63,23 +65,22 @@ jobs:
macos:
name: macOS universal
needs: version
runs-on: macos-15
runs-on: macos-15-xlarge # M2 Pro larger runner; use only in release.yml (actions_macos_xl budget caps spend)
environment: release
env:
FILMCRAFT_VERSION: ${{ needs.version.outputs.version }}
FILMCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@v4
# Fonts (optional build input, required for releases): docs/contributing.md#building-with-craft-fonts
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@686976e191b89faba57d3206551f0f330d8cb249 # stable
with:
targets: aarch64-apple-darwin, x86_64-apple-darwin
- uses: Swatinem/rust-cache@v2
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: release-macos-universal
- name: Apply version override
@@ -99,7 +100,7 @@ jobs:
CRAFT_FONTS_DIR: ${{ github.workspace }}/craft-fonts
CRAFT_FONTS_REQUIRED: "1"
run: packaging/macos/package.sh --arch universal
- uses: actions/upload-artifact@v4
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: release-macos
path: dist/release/*
@@ -127,17 +128,16 @@ jobs:
FILMCRAFT_VERSION: ${{ needs.version.outputs.version }}
FILMCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@v4
# Fonts (optional build input, required for releases): docs/contributing.md#building-with-craft-fonts
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@686976e191b89faba57d3206551f0f330d8cb249 # stable
with:
targets: ${{ matrix.target }}
- uses: Swatinem/rust-cache@v2
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: release-windows-${{ matrix.arch }}
- name: Apply version override
@@ -161,7 +161,7 @@ jobs:
CRAFT_FONTS_DIR: ${{ github.workspace }}/craft-fonts
CRAFT_FONTS_REQUIRED: "1"
run: ./packaging/windows/package.ps1 -Arch ${{ matrix.arch }}
- uses: actions/upload-artifact@v4
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: release-windows-${{ matrix.arch }}
path: dist/release/*
@@ -173,7 +173,7 @@ jobs:
# The oldest GitHub-hosted image, so the binaries only need glibc >= 2.35 (Ubuntu 22.04+,
# Debian 12+, Fedora 36+, RHEL 10). See docs/releasing.md.
runs-on: ${{ matrix.runner }}
environment: release
# No `environment: release`: this job signs nothing, so it gets no release secrets.
strategy:
fail-fast: false
matrix:
@@ -185,9 +185,8 @@ jobs:
FILMCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
NFPM_VERSION: 2.47.0
steps:
- uses: actions/checkout@v4
# Fonts (optional build input, required for releases): docs/contributing.md#building-with-craft-fonts
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
@@ -197,12 +196,12 @@ jobs:
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
libxkbcommon-dev libwayland-dev libx11-dev libxrandr-dev libxi-dev libgl1-mesa-dev libasound2-dev \
desktop-file-utils appstream python3-yaml
desktop-file-utils appstream python3-yaml zsync
curl -fsSL -o "$RUNNER_TEMP/nfpm.deb" \
"https://github.com/goreleaser/nfpm/releases/download/v${NFPM_VERSION}/nfpm_${NFPM_VERSION}_${{ matrix.deb }}.deb"
sudo dpkg -i "$RUNNER_TEMP/nfpm.deb"
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- uses: dtolnay/rust-toolchain@686976e191b89faba57d3206551f0f330d8cb249 # stable
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: release-linux-${{ matrix.arch }}
- name: Apply version override
@@ -222,34 +221,199 @@ jobs:
for appimage in dist/release/*.AppImage; do
chmod +x "$appimage"
APPIMAGE_EXTRACT_AND_RUN=1 "$appimage" --version
# Update information is embedded, and the .zsync for delta updates sits beside it.
"$appimage" --appimage-updateinformation | grep -F 'gh-releases-zsync|'
test -s "$appimage.zsync"
done
- uses: actions/upload-artifact@v4
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: release-linux-${{ matrix.arch }}
path: dist/release/*
if-no-files-found: error
linux-riscv64:
name: Linux riscv64
needs: version
# Cross-compiled on x86_64 (no hosted RISC-V runners) against the runner's own Ubuntu release
# from ports.ubuntu.com, so the binaries need that release's glibc (24.04: 2.39). 22.04's
# riscv64 ports lag its amd64 updates, which breaks the multiarch install. tar.gz only; the
# CLI is smoke-tested under QEMU. Audio links ALSA (cpal), hence libasound2-dev:riscv64; the
# hardware video paths (VA-API, NVDEC/NVENC) are loaded at run time with libloading, so they
# need no riscv64 libraries and fall back to the pure-Rust codecs when absent.
runs-on: ubuntu-24.04
# No `environment: release`: this job signs nothing, so it gets no release secrets.
env:
FILMCRAFT_VERSION: ${{ needs.version.outputs.version }}
FILMCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@686976e191b89faba57d3206551f0f330d8cb249 # stable
with:
targets: riscv64gc-unknown-linux-gnu
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: release-linux-riscv64
- name: Apply version override
if: inputs.version != ''
run: cargo xtask version set "$FILMCRAFT_VERSION"
- name: Install cross-compilation dependencies
run: |
. /etc/os-release
sudo dpkg --add-architecture riscv64
# Pin the existing (archive.ubuntu.com) sources to amd64; riscv64 comes from ports.
if [ -f /etc/apt/sources.list.d/ubuntu.sources ]; then
sudo sed -i '/^Types: deb/a Architectures: amd64' /etc/apt/sources.list.d/ubuntu.sources
fi
if [ -f /etc/apt/sources.list ]; then
sudo sed -i 's/^deb /deb [arch=amd64] /g' /etc/apt/sources.list
sudo sed -i 's/^deb-src /deb-src [arch=amd64] /g' /etc/apt/sources.list
fi
for suite in "$VERSION_CODENAME" "$VERSION_CODENAME-updates" "$VERSION_CODENAME-security"; do
echo "deb [arch=riscv64] http://ports.ubuntu.com/ubuntu-ports $suite main universe restricted multiverse"
done | sudo tee /etc/apt/sources.list.d/riscv64.list
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
gcc-riscv64-linux-gnu binutils-riscv64-linux-gnu libc6-dev-riscv64-cross libc6-dev:riscv64 qemu-user \
libxkbcommon-dev:riscv64 libwayland-dev:riscv64 libx11-dev:riscv64 libxrandr-dev:riscv64 \
libxi-dev:riscv64 libgl1-mesa-dev:riscv64 libasound2-dev:riscv64
- name: Build and package (tar.gz)
env:
CRAFT_FONTS_DIR: ${{ github.workspace }}/craft-fonts
CRAFT_FONTS_REQUIRED: "1"
CARGO_TARGET_RISCV64GC_UNKNOWN_LINUX_GNU_LINKER: riscv64-linux-gnu-gcc
PKG_CONFIG_SYSROOT_DIR: /
PKG_CONFIG_PATH: /usr/lib/riscv64-linux-gnu/pkgconfig:/usr/share/pkgconfig
PKG_CONFIG_ALLOW_CROSS: "1"
CROSS_ARCH: riscv64
CROSS_TARGET: riscv64gc-unknown-linux-gnu
CROSS_COMPILE: riscv64-linux-gnu-
EMULATOR: qemu-riscv64
QEMU_LD_PREFIX: /usr/riscv64-linux-gnu
run: packaging/linux/package.sh --formats tar
- name: Check the tarball
run: |
tar -tzf dist/release/filmcraft-*-linux-riscv64.tar.gz \
| grep -E '/bin/filmcraft$|/bin/filmcraft-cli$|/share/applications/ai\.storyteller\.filmcraft\.desktop$'
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: release-linux-riscv64
path: dist/release/*.tar.gz
if-no-files-found: error
flatpak:
name: Flatpak ${{ matrix.arch }}
needs: [version, linux]
# Repackages the Linux job's tarball (the same binaries as the AppImage/deb/rpm); no Rust
# build here. 24.04 for a newer flatpak-builder (1.4) than 22.04 ships.
runs-on: ${{ matrix.runner }}
# No `environment: release`: this job signs nothing, so it gets no release secrets.
strategy:
fail-fast: false
matrix:
include:
- { arch: x86_64, runner: ubuntu-24.04 }
- { arch: aarch64, runner: ubuntu-24.04-arm }
env:
FILMCRAFT_VERSION: ${{ needs.version.outputs.version }}
FILMCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: release-linux-${{ matrix.arch }}
path: dist/linux
- name: Install flatpak-builder
run: |
sudo apt-get update
# librsvg2-common: the SVG pixbuf loader. flatpak-builder runs the host's
# `appstreamcli compose`, which renders the scalable icon and fails without it.
sudo apt-get install -y --no-install-recommends flatpak flatpak-builder librsvg2-common
# Ubuntu 24.04's AppArmor blocks the unprivileged user namespaces bubblewrap needs.
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
- name: Build the Flatpak bundle
run: packaging/linux/flatpak-bundle.sh "dist/linux/filmcraft-$FILMCRAFT_VERSION-linux-${{ matrix.arch }}.tar.gz"
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: release-flatpak-${{ matrix.arch }}
path: dist/release/*.flatpak
if-no-files-found: error
freebsd:
name: FreeBSD x86_64
needs: version
# GitHub has no FreeBSD runners: build in a FreeBSD 14.3 VM, the same image and packages as
# .github/workflows/freebsd.yml.
runs-on: ubuntu-latest
timeout-minutes: 180
# No `environment: release`: this job signs nothing, so it gets no release secrets.
env:
FILMCRAFT_VERSION: ${{ needs.version.outputs.version }}
FILMCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
path: craft-fonts
- name: Build and package in a FreeBSD VM (tar.gz)
# vmactions/freebsd-vm v1.5.9, pinned by commit (it runs our build).
uses: vmactions/freebsd-vm@c46abacb49f09938ca4e1702d15d836285d694cc # v1.5.9
with:
release: "14.3"
usesh: true
cpu: 4
mem: 12288
# Passed into the VM by name; never templated into the script below.
envs: "FILMCRAFT_VERSION FILMCRAFT_BUILD_DATE FILMCRAFT_BUILD_SHA INPUT_VERSION"
# The FreeBSD CI packages (.github/workflows/freebsd.yml), plus bash for the script.
prepare: |
pkg install -y bash git curl pkgconf cmake python3 libxkbcommon wayland libX11 libXcursor libXrandr libXi libxcb mesa-libs vulkan-loader gtk3 fontconfig freetype2 alsa-lib
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain stable
run: |
set -e
. "$HOME/.cargo/env"
git config --global --add safe.directory "$PWD"
# Build outside the checkout so only dist/ is copied back out of the VM (copying the
# target dir back fails on large builds). 4 jobs: more runs the VM out of memory.
export CARGO_TARGET_DIR=/var/tmp/filmcraft-target CARGO_BUILD_JOBS=4
if [ -n "$INPUT_VERSION" ]; then cargo xtask version set "$FILMCRAFT_VERSION"; fi
# craft-fonts was checked out next to the sources and synced into the VM with them.
export CRAFT_FONTS_DIR="$PWD/craft-fonts" CRAFT_FONTS_REQUIRED=1
bash packaging/freebsd/package.sh
tar -tzf dist/release/filmcraft-*-freebsd-x86_64.tar.gz \
| grep -E '/bin/filmcraft$|/bin/filmcraft-cli$|/share/applications/ai\.storyteller\.filmcraft\.desktop$|/share/doc/filmcraft/LICENSE-MIT$'
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: release-freebsd-x86_64
path: dist/release/*
if-no-files-found: error
web:
name: Web (wasm)
needs: version
runs-on: ubuntu-latest
environment: release
# No `environment: release`: this job signs nothing, so it gets no release secrets.
env:
FILMCRAFT_VERSION: ${{ needs.version.outputs.version }}
FILMCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
WASM_BINDGEN_VERSION: 0.2.129
steps:
- uses: actions/checkout@v4
# Fonts (optional build input, required for releases): docs/contributing.md#building-with-craft-fonts
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@686976e191b89faba57d3206551f0f330d8cb249 # stable
with:
targets: wasm32-unknown-unknown
- uses: Swatinem/rust-cache@v2
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: release-web
- name: Install wasm-bindgen CLI
@@ -267,7 +431,7 @@ jobs:
CRAFT_FONTS_DIR: ${{ github.workspace }}/craft-fonts
CRAFT_FONTS_REQUIRED: "1"
run: packaging/web/package.sh
- uses: actions/upload-artifact@v4
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: release-web
path: dist/release/*
@@ -275,7 +439,7 @@ jobs:
release:
name: Draft GitHub Release
needs: [version, macos, windows, linux, web]
needs: [version, macos, windows, linux, linux-riscv64, flatpak, freebsd, web]
runs-on: ubuntu-latest
environment: release
permissions:
@@ -286,7 +450,7 @@ jobs:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_REPO: ${{ github.repository }}
steps:
- uses: actions/download-artifact@v4
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
pattern: release-*
path: artifacts
+5 -5
View File
@@ -36,11 +36,11 @@ jobs:
runs-on: windows-latest
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@89b12181fb390509a0842a86cc55eeb8eb928c1d # stable
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: dtolnay/rust-toolchain@686976e191b89faba57d3206551f0f330d8cb249 # stable
with:
targets: aarch64-pc-windows-msvc
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: windows-arm64-package
- name: Install WiX
@@ -51,7 +51,7 @@ jobs:
env:
DIST: ${{ github.workspace }}\dist\arm64
run: ./packaging/windows/package.ps1 -Arch arm64
- uses: actions/upload-artifact@v4
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: filmcraft-windows-arm64
path: dist/arm64/*
@@ -64,7 +64,7 @@ jobs:
runs-on: windows-11-arm
timeout-minutes: 30
steps:
- uses: actions/download-artifact@v4
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: filmcraft-windows-arm64
path: dist
+21 -1
View File
@@ -21,7 +21,7 @@ panic path, and fix a crash before building on top of it. The cross-app standard
code alone (a literal that always parses): a single-item `#[allow(clippy::expect_used)]` with the reason.
3. **No `unsafe`** (`unsafe_code = "forbid"` for the workspace). The single exception is
`crates/platform`, which needs `unsafe` to call the operating system's media APIs (hardware
video decoding through VideoToolbox and Media Foundation / Direct3D 11; later encoding and Linux) and does nothing else. It
video decoding through VideoToolbox, Media Foundation / Direct3D 11, VA-API and NVDEC; hardware encoding) and does nothing else. It
uses `unsafe_code = "deny"` with `#[allow(unsafe_code)]` only on its FFI modules, a
`// SAFETY:` comment on every `unsafe` block, a safe `Result`-returning public API, and the
pure-Rust decoder as the tested fallback ([ADR 0001](docs/adr/0001-platform-ffi.md)).
@@ -127,6 +127,7 @@ See [§0](#0-never-crash). It outranks every other engineering rule.
See `CLAUDE.md`: pure Rust, dependency layering (`cargo xtask layers`), exact `Tick` time, everything is
a command, everything is agent-drivable, and the quality gates (`cargo xtask ci`) before every commit.
Shared real-file test corpora (Photoshop-authored PSDs, etc.) live in
[`storytold/photocraft-corpus`](https://github.com/storytold/photocraft-corpus), explained in
[craftrules `standards/test-corpora.md`](https://github.com/storytold/craftrules/blob/main/standards/test-corpora.md).
@@ -141,3 +142,22 @@ as PhotoCraft does with `cargo xtask corpus`.
- [docs/agents.md](docs/agents.md): driving FilmCraft over MCP / the control channel, and the agent work loop
- [docs/control-protocol.md](docs/control-protocol.md): control-channel method reference
- [ATTRIBUTION.md](ATTRIBUTION.md): asset index
- [ROADMAP.md](ROADMAP.md), [docs/target-app-parity.md](docs/target-app-parity.md), [docs/gaps.md](docs/gaps.md): stage, parity with Premiere Pro and the ranked work list (craftrules `standards/progress-docs.md`)
## Contributor credits (About window)
- About ▸ Contributors/Models are compiled into the binary from `contributors/contributors.json`
(commit stats; generated, never hand-edit) and `contributors/people.toml` (names people chose for
themselves). See `docs/contributors.md`.
- **Agents working for a contributor:** when you prepare a PR, check whether your human's GitHub
username has a `[people.<username>]` entry in `contributors/people.toml`. If not, ask them once
whether they want to be credited by more than their username: a real name, a display name, and/or
their public GitHub profile name (`sync_github_name = true`). If yes, add **only their own** entry
(copy the template at the top of the file, or run
`python3 ../../craftrules/scripts/contributors.py --add-me . --real-name "…" --sync-github-name`)
and include it in their PR, committed as them. If no, change nothing: they are credited as
`@username` anyway.
- Never add, edit, guess or copy anyone else's entry or name (not from git config, commit authors or
GitHub profiles). Never hand-edit `contributors.json`.
- Maintainers refresh the stats with `python3 ../../craftrules/scripts/contributors.py .` (it also
re-verifies who wrote each `people.toml` entry; `--check` only verifies).
+7 -1
View File
@@ -9,6 +9,9 @@ iconography, images or other Adobe assets; open licences only; every asset attri
| Asset | Author | Source | Licence |
|---|---|---|---|
| `crates/ui-egui/src/i18n/fr.tsv` | FilmCraft contributors | Original work: clean-room French interface translations | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/ru.tsv` | FilmCraft contributors | Original work: clean-room Russian interface translations | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/uk.tsv` | FilmCraft contributors | Original work: clean-room Ukrainian interface translations | MIT OR Apache-2.0 |
| `assets/fonts/Inter-Regular.ttf` | The Inter Project Authors | https://github.com/rsms/inter | OFL-1.1 (`assets/fonts/OFL-Inter.txt`) |
| `assets/fonts/Inter-Medium.ttf` | The Inter Project Authors | https://github.com/rsms/inter | OFL-1.1 (`assets/fonts/OFL-Inter.txt`) |
| `assets/fonts/Inter-SemiBold.ttf` | The Inter Project Authors | https://github.com/rsms/inter | OFL-1.1 (`assets/fonts/OFL-Inter.txt`) |
@@ -26,6 +29,8 @@ iconography, images or other Adobe assets; open licences only; every asset attri
| `docs/images/filmcraft-effects.png` | FilmCraft contributors | Original work: FilmCraft screenshot crop; NASA Earth Views (US Government work, public domain) | MIT OR Apache-2.0; footage public domain |
| `docs/images/filmcraft-timeline.png` | FilmCraft contributors | Original work: FilmCraft UI crop; thumbnails from Night of the Living Dead (1968) and Carnival of Souls (1962), CC0 Chopin waveform | MIT OR Apache-2.0; film frames public domain |
| `assets/app-icon/filmcraft.svg` | Brandon Thomas (FilmCraft owner) | Original work: FilmCraft app icon (owl with ear tufts), drawn by the owner in ArtCraft and vectorised; canonical vector | MIT OR Apache-2.0 (`assets/app-icon/LICENSE.txt`) |
| `packaging/macos/dmg/background.svg` | @XusBadia | Original work: FilmCraft macOS DMG window background, derived from the FilmCraft app icon (`assets/app-icon/filmcraft.svg`); rendered to `background.tiff` by `packaging/macos/dmg/generate.py` | MIT OR Apache-2.0 (`LICENSE-MIT`, `LICENSE-APACHE`) |
| `packaging/macos/dmg/background.tiff` | @XusBadia | Rendered from `packaging/macos/dmg/background.svg` by `packaging/macos/dmg/generate.py` | MIT OR Apache-2.0 (`LICENSE-MIT`, `LICENSE-APACHE`) |
| `assets/app-icon/filmcraft-small.svg` | Brandon Thomas (FilmCraft owner) | Original work: FilmCraft app icon (owl with ear tufts), drawn by the owner in ArtCraft and vectorised; lighter vector | MIT OR Apache-2.0 (`assets/app-icon/LICENSE.txt`) |
| `assets/app-icon/filmcraft-1024.png` | Brandon Thomas (FilmCraft owner) | Original work: FilmCraft app icon (owl with ear tufts), drawn by the owner in ArtCraft and vectorised; 1024 px render | MIT OR Apache-2.0 (`assets/app-icon/LICENSE.txt`) |
| `assets/app-icon/filmcraft-macos-512.png` | Brandon Thomas (FilmCraft owner) | Original work: FilmCraft app icon (owl with ear tufts), drawn by the owner in ArtCraft and vectorised; macOS runtime Dock icon (512 px) | MIT OR Apache-2.0 (`assets/app-icon/LICENSE.txt`) |
@@ -99,6 +104,7 @@ is traced or derived from Adobe artwork.
| Asset | Where |
|---|---|
| UI icons (tools, transport, panels, header) | `crates/ui-egui/src/icons.rs`: vector paths on a 16×16 grid |
| Source and Program range braces and trim hover cues | `crates/ui-egui/src/panels/monitor.rs` and `source_range.rs`: original generic vector shapes by FilmCraft contributors, MIT OR Apache-2.0; no third-party artwork |
| Align / distribute / paragraph-alignment glyphs in the graphics panel | `crates/ui-egui/src/panels/graphics.rs` (`align_glyph`): bars and lines drawn with the egui painter |
| Demo footage (ocean sunset, aurora, city night, dunes, forest, plasma), bars and tone, counting leader, colour matte | `crates/media/src/generators.rs` |
| Built-in graphics templates (Lower Third – Slab / Rule, Ticker – Crawl, Title – Centered / Boxed, End Credits – Roll, Callout – Pointer / Tag) and their placeholder text | `crates/project/src/gtemplate.rs` (`builtin_templates`): original designs defined in code with the bundled Inter font; thumbnails are rendered at runtime, no template, image or font files |
@@ -146,7 +152,7 @@ the original copyright and permission notice in its header.
| File | Author | Source | Licence |
|---|---|---|---|
| `crates/platform/src/nvenc/ffi.rs` | NVIDIA Corporation (`nvEncodeAPI.h`); FilmCraft contributors (Rust transcription) | Structures, function table and constants transcribed from `nvEncodeAPI.h`, NVIDIA Video Codec SDK, API 12.1 (https://developer.nvidia.com/video-codec-sdk) | MIT (Copyright (c) 2010-2023 NVIDIA Corporation; notice kept in the file header) |
| `crates/platform/src/nvenc/ffi.rs` | NVIDIA Corporation (`nvEncodeAPI.h`); FilmCraft contributors (Rust transcription) | Structures (including `NV_ENC_CONFIG_HEVC` and its bit-field masks, `NV_ENC_PIC_PARAMS_HEVC`, `NV_ENC_SEI_PAYLOAD`, `NV_ENC_TIME_CODE`), function table, codec / profile GUIDs (H.264, HEVC Main, HEVC Main 10) and constants (`NV_ENC_LEVEL_HEVC_*`, tier, CU size, `NV_ENC_BUFFER_FORMAT_YUV420_10BIT`, `NV_ENC_CAPS_SUPPORT_10BIT_ENCODE`, BT.2020 / PQ / HLG VUI codes) transcribed from `nvEncodeAPI.h`, NVIDIA Video Codec SDK, API 12.1 (https://developer.nvidia.com/video-codec-sdk) | MIT (Copyright (c) 2010-2023 NVIDIA Corporation; notice kept in the file header) |
## Downloaded at runtime
+1 -1
View File
@@ -4,7 +4,7 @@ FilmCraft is a clean-room, open-source, pure-Rust non-linear video editor target
## Start every session here
`plan/` is maintainer-local (gitignored, not in the repo). Each step names the public equivalent to use if you don't have it.
1. Read `plan/STATUS.md` (current milestone, next unchecked task, blockers). Public: [`ROADMAP.md`](ROADMAP.md).
1. Read `plan/STATUS.md` (current milestone, next unchecked task, blockers). Public: [`ROADMAP.md`](ROADMAP.md), then [`docs/gaps.md`](docs/gaps.md) (ranked shortfalls) and [`docs/roadmap.md`](docs/roadmap.md) (current focus).
2. Read the task in `plan/execution-plan.md` §3, the relevant section of `plan/architecture.md`, and the README/docs of the crate you touch. Public: [`docs/architecture.md`](docs/architecture.md), [`docs/contributing.md`](docs/contributing.md), [`docs/testing.md`](docs/testing.md). Visual/behaviour reference: `plan/premiere/` (local only, never committed).
3. Follow the autonomous operation protocol (`plan/execution-plan.md` §7; public summary: [`docs/agents.md`](docs/agents.md) §5–7). Don't stop to ask unless it lists the decision as the user's.
Generated
+141 -47
View File
@@ -1665,29 +1665,39 @@ dependencies = [
[[package]]
name = "filmcraft"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"arboard",
"chrono",
"cpal",
"eframe",
"egui",
"filmcraft-engine",
"filmcraft-export",
"filmcraft-gpu",
"filmcraft-platform",
"filmcraft-project",
"filmcraft-render",
"filmcraft-time",
"filmcraft-ui-egui",
"glob",
"log",
"muda",
"objc2 0.6.4",
"objc2-app-kit 0.3.2",
"objc2-foundation 0.3.2",
"open",
"rfd",
"serde_json",
"sys-locale",
"winit",
"winresource",
"zbus",
]
[[package]]
name = "filmcraft-aac"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1697,7 +1707,7 @@ dependencies = [
[[package]]
name = "filmcraft-ac3"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-testkit",
"thiserror 2.0.21",
@@ -1705,7 +1715,7 @@ dependencies = [
[[package]]
name = "filmcraft-apv"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1715,14 +1725,14 @@ dependencies = [
[[package]]
name = "filmcraft-audio-dsp"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-testkit",
]
[[package]]
name = "filmcraft-automation"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"base64 0.22.1",
"filmcraft-engine",
@@ -1738,7 +1748,7 @@ dependencies = [
[[package]]
name = "filmcraft-av1"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1749,14 +1759,14 @@ dependencies = [
[[package]]
name = "filmcraft-bitstream"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"proptest",
]
[[package]]
name = "filmcraft-captions"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-color",
"filmcraft-project",
@@ -1769,20 +1779,24 @@ dependencies = [
[[package]]
name = "filmcraft-cfb"
version = "0.2.1"
version = "0.6.0"
[[package]]
name = "filmcraft-cli"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-automation",
"filmcraft-codecs",
"filmcraft-engine",
"filmcraft-export",
"filmcraft-gpu",
"filmcraft-media",
"filmcraft-mpegts",
"filmcraft-mxf",
"filmcraft-ogg",
"filmcraft-platform",
"filmcraft-project",
"filmcraft-render",
"filmcraft-time",
"serde_json",
"tokio",
@@ -1790,7 +1804,7 @@ dependencies = [
[[package]]
name = "filmcraft-codecs"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-aac",
"filmcraft-ac3",
@@ -1816,6 +1830,7 @@ dependencies = [
"filmcraft-vp9",
"image",
"log",
"serde_json",
"symphonia",
"thiserror 2.0.21",
"web-time",
@@ -1823,7 +1838,7 @@ dependencies = [
[[package]]
name = "filmcraft-color"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"proptest",
"serde",
@@ -1831,7 +1846,7 @@ dependencies = [
[[package]]
name = "filmcraft-dnx"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1841,8 +1856,9 @@ dependencies = [
[[package]]
name = "filmcraft-edit"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-geom",
"filmcraft-media",
"filmcraft-project",
"filmcraft-time",
@@ -1853,7 +1869,7 @@ dependencies = [
[[package]]
name = "filmcraft-engine"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-audio-dsp",
"filmcraft-captions",
@@ -1874,6 +1890,7 @@ dependencies = [
"filmcraft-testkit",
"filmcraft-text",
"filmcraft-time",
"filmcraft-tts",
"image",
"log",
"serde",
@@ -1884,7 +1901,7 @@ dependencies = [
[[package]]
name = "filmcraft-export"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-aac",
"filmcraft-apv",
@@ -1914,11 +1931,12 @@ dependencies = [
[[package]]
name = "filmcraft-format"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-media",
"filmcraft-project",
"filmcraft-time",
"log",
"serde",
"serde_json",
"thiserror 2.0.21",
@@ -1926,7 +1944,7 @@ dependencies = [
[[package]]
name = "filmcraft-frame"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-color",
"filmcraft-time",
@@ -1936,7 +1954,7 @@ dependencies = [
[[package]]
name = "filmcraft-geom"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"proptest",
"serde",
@@ -1944,7 +1962,7 @@ dependencies = [
[[package]]
name = "filmcraft-golden"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-color",
"filmcraft-frame",
@@ -1961,15 +1979,17 @@ dependencies = [
[[package]]
name = "filmcraft-gpu"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-color",
"filmcraft-export",
"filmcraft-frame",
"filmcraft-geom",
"filmcraft-media",
"filmcraft-project",
"filmcraft-render",
"filmcraft-time",
"image",
"log",
"pollster 0.4.0",
"rayon",
@@ -1978,7 +1998,7 @@ dependencies = [
[[package]]
name = "filmcraft-h264"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1988,7 +2008,7 @@ dependencies = [
[[package]]
name = "filmcraft-h264enc"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -1998,7 +2018,7 @@ dependencies = [
[[package]]
name = "filmcraft-hevc"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -2008,7 +2028,7 @@ dependencies = [
[[package]]
name = "filmcraft-interchange"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-cfb",
"filmcraft-color",
@@ -2025,7 +2045,7 @@ dependencies = [
[[package]]
name = "filmcraft-isobmff"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -2036,7 +2056,7 @@ dependencies = [
[[package]]
name = "filmcraft-matroska"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-testkit",
"serde_json",
@@ -2044,7 +2064,7 @@ dependencies = [
[[package]]
name = "filmcraft-media"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-color",
"filmcraft-frame",
@@ -2057,7 +2077,7 @@ dependencies = [
[[package]]
name = "filmcraft-mpeg2v"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-testkit",
"rayon",
@@ -2066,7 +2086,7 @@ dependencies = [
[[package]]
name = "filmcraft-mpegts"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-testkit",
"thiserror 2.0.21",
@@ -2074,39 +2094,43 @@ dependencies = [
[[package]]
name = "filmcraft-mxf"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-testkit",
]
[[package]]
name = "filmcraft-ogg"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-testkit",
]
[[package]]
name = "filmcraft-opus"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-testkit",
]
[[package]]
name = "filmcraft-platform"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-codecs",
"filmcraft-color",
"filmcraft-export",
"filmcraft-frame",
"filmcraft-h264",
"filmcraft-hevc",
"filmcraft-isobmff",
"filmcraft-media",
"filmcraft-project",
"filmcraft-render",
"filmcraft-testkit",
"filmcraft-time",
"libloading",
"log",
"objc2-core-foundation",
"objc2-core-media",
@@ -2117,7 +2141,7 @@ dependencies = [
[[package]]
name = "filmcraft-project"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-color",
"filmcraft-geom",
@@ -2131,7 +2155,7 @@ dependencies = [
[[package]]
name = "filmcraft-prores"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-bitstream",
"filmcraft-testkit",
@@ -2141,7 +2165,7 @@ dependencies = [
[[package]]
name = "filmcraft-render"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-audio-dsp",
"filmcraft-captions",
@@ -2161,7 +2185,7 @@ dependencies = [
[[package]]
name = "filmcraft-scopes"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-color",
"serde",
@@ -2170,7 +2194,7 @@ dependencies = [
[[package]]
name = "filmcraft-speech"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"candle-core",
"candle-nn",
@@ -2188,33 +2212,56 @@ dependencies = [
[[package]]
name = "filmcraft-testkit"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"png",
]
[[package]]
name = "filmcraft-text"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-color",
"harfrust",
"skrifa",
"unicode-bidi",
"unicode-linebreak",
"unicode-segmentation",
]
[[package]]
name = "filmcraft-time"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"proptest",
"serde",
]
[[package]]
name = "filmcraft-tts"
version = "0.6.0"
dependencies = [
"candle-core",
"candle-nn",
"filmcraft-audio-dsp",
"filmcraft-tts-text",
"serde",
"serde_json",
"thiserror 2.0.21",
"zip",
]
[[package]]
name = "filmcraft-tts-text"
version = "0.6.0"
dependencies = [
"serde",
"thiserror 2.0.21",
]
[[package]]
name = "filmcraft-ui-egui"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"cpu-time",
"eframe",
@@ -2225,6 +2272,7 @@ dependencies = [
"filmcraft-color",
"filmcraft-edit",
"filmcraft-engine",
"filmcraft-export",
"filmcraft-frame",
"filmcraft-geom",
"filmcraft-gpu",
@@ -2236,6 +2284,7 @@ dependencies = [
"filmcraft-scopes",
"filmcraft-text",
"filmcraft-time",
"filmcraft-tts",
"image",
"log",
"pollster 0.4.0",
@@ -2243,12 +2292,13 @@ dependencies = [
"rustix 1.1.5",
"serde",
"serde_json",
"toml 1.1.6+spec-1.1.0",
"web-time",
]
[[package]]
name = "filmcraft-vp9"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-bitstream",
"rayon",
@@ -2257,7 +2307,7 @@ dependencies = [
[[package]]
name = "filmcraft-web"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"eframe",
"egui",
@@ -2815,6 +2865,12 @@ dependencies = [
"vello_common",
]
[[package]]
name = "glob"
version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e4eba85ea1d0a966a983acd07deee566e67395d2d96b6fb39e62b5a833f1eb0b"
[[package]]
name = "glow"
version = "0.17.0"
@@ -3177,6 +3233,25 @@ dependencies = [
"generic-array",
]
[[package]]
name = "is-docker"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "928bae27f42bc99b60d9ac7334e3a21d10ad8f1835a4e12ec3ec0464765ed1b3"
dependencies = [
"once_cell",
]
[[package]]
name = "is-wsl"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "173609498df190136aa7dea1a91db051746d339e18476eed5ca40521f02d7aa5"
dependencies = [
"is-docker",
"once_cell",
]
[[package]]
name = "itertools"
version = "0.15.0"
@@ -4102,6 +4177,16 @@ version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c08d65885ee38876c4f86fa503fb49d7b507c2b62552df7c70b2fce627e06381"
[[package]]
name = "open"
version = "5.4.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aa576c76302b7b808eecc68061e67336c47833ef9d22caa74dda10fa9675eebc"
dependencies = [
"is-wsl",
"libc",
]
[[package]]
name = "openssl-probe"
version = "0.2.1"
@@ -5781,6 +5866,15 @@ dependencies = [
"syn 3.0.6",
]
[[package]]
name = "sys-locale"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8eab9a99a024a169fe8a903cf9d4a3b3601109bcc13bd9e3c6fff259138626c4"
dependencies = [
"libc",
]
[[package]]
name = "sysctl"
version = "0.6.0"
@@ -7238,7 +7332,7 @@ checksum = "e450f9b2ed1dff33c94c12589a87338689467b9c4f5d8a5710bd09a847d2c8a7"
[[package]]
name = "xtask"
version = "0.2.1"
version = "0.6.0"
dependencies = [
"filmcraft-testkit",
"serde_json",
+3 -1
View File
@@ -4,7 +4,7 @@ members = ["crates/*", "apps/*", "xtask"]
default-members = ["crates/*", "apps/*", "xtask"]
[workspace.package]
version = "0.2.1"
version = "0.6.0"
edition = "2024"
license = "MIT OR Apache-2.0"
rust-version = "1.95"
@@ -44,6 +44,8 @@ filmcraft-interchange = { path = "crates/interchange" }
filmcraft-text = { path = "crates/text" }
filmcraft-captions = { path = "crates/captions" }
filmcraft-speech = { path = "crates/speech" }
filmcraft-tts = { path = "crates/tts" }
filmcraft-tts-text = { path = "crates/tts-text" }
filmcraft-render = { path = "crates/render" }
filmcraft-gpu = { path = "crates/gpu" }
filmcraft-scopes = { path = "crates/scopes" }
+91 -9
View File
@@ -59,6 +59,7 @@
<a href="#interchange">Interchange</a> ·
<a href="#built-for-agents">Agents</a> ·
<a href="#get-started">Get started</a> ·
<a href="#downloads">Downloads</a> ·
<a href="#documentation">Docs</a> ·
<a href="#status">Status</a>
</p>
@@ -82,7 +83,7 @@ FilmCraft is a non-linear editor for people who know Premiere: the same panels,
- **Three-point editing.** Mark In and Out in the Source monitor, then insert (`,`) or overwrite (`.`) onto the patched tracks. Lift (`;`) and extract (`'`) take ranges back out.
- **Every trim.** Ripple, roll, slip, slide, rate stretch and razor tools. Trim mode selects edit points as ripple, roll or trim, nudges them a frame at a time (`⌥←` `⌥→`, ×5 with `⇧`), toggles the trim type with `⌃T` and extends them to the playhead with `E`. `Q` and `W` ripple-trim to the playhead. The **Trim Monitor** shows both sides of the edit, and **dynamic trimming** trims live while it plays: `L` forward, `J` back, `K` to stop and commit as one undo step.
- **Exact time.** Every edit is computed on integer ticks: 254,016,000,000 per second, which divides evenly by every common frame rate and sample rate. 23.976, 29.97 drop-frame and 59.94 are exact, not approximate.
- **The details pros rely on.** Markers with colours, names and durations; add edit (`⌘K`) on one or all tracks; nesting; copy, paste and paste insert; ripple delete and close gap; snapping; unlimited undo with a History panel.
- **The details pros rely on.** Markers with colours, names and durations; export sequence review notes with **Markers ▸ Export Markers as CSV…**; add edit (`⌘K`) on one or all tracks; nesting; copy, paste and paste insert; ripple delete and close gap; snapping; unlimited undo with a History panel.
- **Your keys.** A Keyboard Shortcuts editor (`⌥⌘K`) with a drawn keyboard, panel-specific shortcuts, conflict warnings and presets for FilmCraft, Premiere Pro, Final Cut Pro and Avid key layouts.
- **Never lose work.** Saves are atomic, auto-save keeps a rolling set of versions, and a crash-recovery journal written about a second after each edit brings back unsaved changes after a crash or power cut.
@@ -136,7 +137,7 @@ FilmCraft is a non-linear editor for people who know Premiere: the same panels,
- **Premiere 26's full Video Effects bin (93 effects in 16 folders), the Legacy bin and the obsolete effects old projects use, plus around 30 transitions.** Blurs (Bokeh, Focus, Compound), keys (Ultra Key, Track Matte), distortions (Corner Pin, Turbulent Displace, Warp Stabilizer), Lights & Glows, Immersive Video (VR) effects on equirectangular footage, Posterize Time, Echo and more. Cross dissolve, dip to black or white, film dissolve, wipes, irises, pushes, slides, zooms, page peel, cube spin and more.
- **Motion and opacity on every clip:** position, scale, rotation, anchor point and anti-flicker, plus 26 blend modes.
- **Keyframes like Premiere's:** linear, Bezier, auto and continuous Bezier, hold, ease in and ease out. Effect Controls shows a keyframe lane for every parameter, and each animated parameter opens into **value and velocity graphs** with draggable influence handles.
- **Keyframes like Premiere's:** linear, Bezier, auto and continuous Bezier, hold, ease in and ease out. Effect Controls shows a keyframe lane for every parameter under a time ruler with the playhead's handle, and each animated parameter opens into **value and velocity graphs** with draggable influence handles. Effect Controls and the Properties panel share the keyframe navigator (◀ ◆ ▶): add or remove the keyframe at the playhead, step to the previous or next one.
- **A GPU compositor** built on wgpu (Metal, Vulkan, DirectX 12, WebGPU). It samples YUV straight from the decoder with footprint supersampling and blends in linear light. A CPU path renders the same frames, and the two are tested against each other.
<p align="center">
@@ -184,7 +185,7 @@ No FFmpeg inside. The video codecs, AAC, Opus and the containers are our own Rus
| **Matroska / WebM** | ✓ | | Lacing, Cues, header stripping, HDR colour metadata |
| **Stills** | ✓ | ✓ | Import PNG, JPEG, GIF, WebP, TIFF and BMP; export PNG sequences and animated GIF |
Also imported: MPEG-2 / MPEG-1 video, AC-3, MP2, MXF (OP1a / OP-Atom), MPEG transport and program streams (AVCHD, broadcast, DVD), Ogg and image sequences. Not yet: HEVC and AV1 export, camera RAW, E-AC-3. Hardware decoding works on macOS (VideoToolbox) and Windows (Media Foundation), with Linux to follow; H.264 export can use NVIDIA's encoder on Windows (opt-in) ([#30](https://github.com/storytold/filmcraft/issues/30)).
Also imported: MPEG-2 / MPEG-1 video, AC-3, MP2, MXF (OP1a / OP-Atom), MPEG transport and program streams (AVCHD, broadcast, DVD), Ogg and image sequences. Not yet: AV1 export, camera RAW, E-AC-3. Hardware decoding works on macOS (VideoToolbox), Windows (Media Foundation) and, for H.264 and HEVC, Linux (VA-API); H.264 export can use NVIDIA's encoder on Windows and Linux (opt-in), and H.265 (HEVC Main, 8-bit) export works through it (NVENC on Windows, VideoToolbox on macOS) ([#30](https://github.com/storytold/filmcraft/issues/30)).
<br>
@@ -273,22 +274,25 @@ The control protocol is documented in [docs/control-protocol.md](docs/control-pr
| [docs/control-protocol.md](docs/control-protocol.md) | Control-channel and MCP reference |
| [docs/project-files.md](docs/project-files.md) | `.fcproj` format, schema migrations, auto-save and crash recovery |
| [docs/graphics.md](docs/graphics.md) · [docs/captions.md](docs/captions.md) | Text engine, graphic clips and tools; caption tracks and formats |
| [ROADMAP.md](ROADMAP.md) | Honest assessment, what's missing, milestones and estimates |
| [ROADMAP.md](ROADMAP.md) | Stage, parity numbers, languages, what's next and the progress log |
| [docs/target-app-parity.md](docs/target-app-parity.md) · [docs/gaps.md](docs/gaps.md) · [docs/roadmap.md](docs/roadmap.md) | Parity assessment against Premiere Pro, the ranked list of shortfalls, milestones and current focus |
| [docs/codec-parity.md](docs/codec-parity.md) · [docs/file-format-parity.md](docs/file-format-parity.md) · [docs/hardware-parity.md](docs/hardware-parity.md) · [docs/ui-parity.md](docs/ui-parity.md) · [docs/localization-parity.md](docs/localization-parity.md) | Parity checklists by dimension |
## Status
FilmCraft is young and moving fast. Editing, trimming, multicam, colour, keyframes, effects, titles, captions, mixing, codecs and export work today.
We track two numbers ([ROADMAP.md](ROADMAP.md#honest-assessment-2026-10-05)):
FilmCraft is in **alpha**. We track two numbers ([docs/target-app-parity.md](docs/target-app-parity.md), re-measured 2026-10-10 against Premiere Pro 26.5.2):
- **Feature checklist: ~87%.** Premiere Pro's menu items, effects, transitions, panels and formats that exist in FilmCraft.
- **Ready for real work: ~50–60%.** Our honest estimate of how close FilmCraft is to replacing Premiere on real projects.
- **Feature breadth: ~86%.** Premiere Pro's menu items, effects, transitions, panels and formats that exist in FilmCraft (menus 92% and effects 100% measured).
- **Ready for real work: ~56%.** Our honest estimate of how close FilmCraft is to replacing Premiere on real projects.
The biggest gaps today:
- **Speed on big footage.** Hardware decoding works on macOS and Windows; Linux has no hardware path yet. Blend modes and the most common effects run on the GPU, but Lumetri, keys and export rendering still run on the CPU; H.264 encoding can use the hardware encoder on macOS and on Windows with an NVIDIA GPU (opt-in, Export ▸ Hardware encoding), and H.265 export is hardware-only there ([#30](https://github.com/storytold/filmcraft/issues/30)).
- **Speed on big footage.** Hardware decoding works on macOS and Windows, and for H.264 and HEVC on Linux (VA-API, or NVDEC with NVIDIA's driver; VP9 and AV1 decode in software there). Blend modes and the most common effects run on the GPU, but Lumetri, keys and export rendering still run on the CPU; H.264 encoding can use the hardware encoder on macOS and on Windows and Linux with an NVIDIA GPU (opt-in, Export ▸ Hardware encoding), and H.265 export is hardware-only on both (HDR sequences export Main 10 PQ / HLG with NVENC; VideoToolbox writes 8-bit SDR) ([#30](https://github.com/storytold/filmcraft/issues/30)).
- **Premiere projects.** `.prproj` files can't be opened yet; bring sequences over as FCP7 XML, AAF or OTIO.
- **No plugins.** No VST3 / Audio Units or OpenFX hosting.
- **Delivery codecs.** H.264 is our only delivery-codec export; no HEVC or AV1 export yet.
- **Delivery codecs.** H.264 is our only software delivery-codec export; H.265 exports only through a hardware encoder (NVIDIA on Windows: Main 8-bit SDR and Main 10 HDR PQ / HLG; VideoToolbox on macOS: 8-bit Main, SDR); no AV1 export yet.
- **Real-world media and platforms.** Our decoders are bit-exact on conformance streams, but camera and phone files in the wild are less tested. Windows and Linux get far less testing than macOS.
- **AI features.** Few so far; speech to text is optional and off by default.
@@ -310,6 +314,84 @@ A layered Cargo workspace:
Nothing below the front ends depends on a UI toolkit or OS API. `cargo xtask ci` checks formatting, lints, tests, the layering rules, asset attribution and the wasm build.
## Downloads
**New to FilmCraft?** Download it from the [FilmCraft page on getartcraft.com](https://getartcraft.com/apps/filmcraft). That's the easiest way to install it.
**Want a specific build or format?** On GitHub, the [latest release](https://github.com/storytold/filmcraft/releases/latest) has every build listed below, and [all releases](https://github.com/storytold/filmcraft/releases) has earlier versions and their notes. `<ver>` in the file names is the version number, and `SHA256SUMS.txt` lists a checksum for every file.
### Windows
| Build | Installer | Portable |
|---|---|---|
| x64 (64-bit Intel/AMD) | `filmcraft-<ver>-windows-x64.msi` | `filmcraft-<ver>-windows-x64-portable.zip` |
| arm64 (Snapdragon and other ARM PCs) | `filmcraft-<ver>-windows-arm64.msi` | `filmcraft-<ver>-windows-arm64-portable.zip` |
| x86 (32-bit) | `filmcraft-<ver>-windows-x86.msi` | `filmcraft-<ver>-windows-x86-portable.zip` |
Installers and executables are code-signed.
**If the app doesn't open on Windows:** the desktop app initializes only Vulkan and DirectX 12 by
default, not OpenGL. Letting wgpu also create an OpenGL instance can crash some graphics drivers (AMD's
`atio6axx.dll`) before the window appears, so the app would flash in Task Manager and quit.
`WGPU_BACKEND` overrides the default for troubleshooting (for example `dx12` or `vulkan`). In
PowerShell, from the folder containing the executable:
```powershell
$env:WGPU_BACKEND = "vulkan"
& .\filmcraft.exe
Remove-Item Env:WGPU_BACKEND # restore the default for later launches
```
An explicit `gl` override can bring the driver crash back on affected systems. The macOS, Linux
and web backend defaults are unchanged.
### macOS
| Build | File | Notes |
|---|---|---|
| App, universal (Apple silicon + Intel) | `filmcraft-<ver>-macos-universal.dmg` | Signed and notarized |
| Command-line tool, universal | `filmcraft-cli-<ver>-macos-universal.zip` | Signed and notarized |
### Linux
| Format | x86_64 | aarch64 (ARM64) | Notes |
|---|---|---|---|
| AppImage | `filmcraft-<ver>-linux-x86_64.AppImage` | `filmcraft-<ver>-linux-aarch64.AppImage` | Runs anywhere; updates itself with [AppImageUpdate](https://github.com/AppImageCommunity/AppImageUpdate) (`.zsync` files) |
| Flatpak | `filmcraft-<ver>-linux-x86_64.flatpak` | `filmcraft-<ver>-linux-aarch64.flatpak` | Sandboxed; `flatpak install --user <file>` |
| Debian/Ubuntu | `filmcraft-<ver>-linux-x86_64.deb` | `filmcraft-<ver>-linux-aarch64.deb` | |
| Fedora/RHEL/openSUSE | `filmcraft-<ver>-linux-x86_64.rpm` | `filmcraft-<ver>-linux-aarch64.rpm` | |
| Tarball | `filmcraft-<ver>-linux-x86_64.tar.gz` | `filmcraft-<ver>-linux-aarch64.tar.gz` | Unpack and run `./install.sh` |
RISC-V (riscv64): `filmcraft-<ver>-linux-riscv64.tar.gz` (tarball only; needs glibc 2.39+, e.g. Ubuntu 24.04
or Debian 13).
The tarball installer places FilmCraft, its command-line tool and desktop integration in
`~/.local`, without administrator permissions. For all users, run
`sudo ./install.sh --prefix /usr/local` instead. Running the installer again updates the installation.
**Gentoo (community-maintained):** the [::snakebyte overlay](https://github.com/switch87/snakebyte-overlay)
packages the Linux release as `media-video/filmcraft-bin`. It is maintained by the community, not by the
FilmCraft team, so report packaging problems to the overlay:
```sh
eselect repository add snakebyte git https://github.com/switch87/snakebyte-overlay.git
emaint sync -r snakebyte
echo 'media-video/filmcraft-bin ~amd64' >> /etc/portage/package.accept_keywords/filmcraft
emerge --ask media-video/filmcraft-bin
```
### FreeBSD
| Build | File |
|---|---|
| x86_64 | `filmcraft-<ver>-freebsd-x86_64.tar.gz` |
### Web (WebAssembly)
| Build | File | Notes |
|---|---|---|
| Static site | `filmcraft-web-<ver>.zip` | Runs in a modern browser; host it on any static server |
## The Crafting Apps
FilmCraft is one of the **Crafting Apps**: free, open-source creative tools from the
+107 -146
View File
@@ -1,163 +1,117 @@
# FilmCraft Roadmap
Progress toward parity with Adobe Premiere Pro, with estimates. Updated as milestones land.
**Stage: alpha** · next: beta, ~19 points of ready-for-real-work and ~400–700 h away, plus opening Premiere's `.prproj` ([why](docs/target-app-parity.md#stage); passes the [core-workflow gate](docs/roadmap.md#alpha-gate))
**Last updated:** 2026-10-05 · **Feature checklist:** ~87% · **Ready for real work:** ~50–60% · **Code:** 35 crates, 1605+ tests
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (full re-measure against Premiere Pro 2026 26.5.2; scorecard, gaps, milestones and localization moved into docs/ per the craftrules progress-docs standard) · **Target:** Adobe Premiere Pro 2026 (26.5.2)
Two numbers, because they answer different questions. The **feature checklist** counts whether
Premiere's menu items, effects, panels and formats exist in FilmCraft. **Ready for real work** is
our honest estimate of how close FilmCraft is to replacing Premiere for an editor on real
projects: depth, correctness, robustness, speed and the pro ecosystem. The gap between the two
is the work that matters most now. Agents: read [Where we are lacking](#where-we-are-lacking)
before choosing work.
Progress toward parity with Adobe Premiere Pro. This page is the summary; the detail is in
[docs/target-app-parity.md](docs/target-app-parity.md) (assessment, methodology, calibration),
[docs/gaps.md](docs/gaps.md) (the ranked work list: **agents, read it before choosing work**) and
[docs/roadmap.md](docs/roadmap.md) (milestones and current focus).
## Honest assessment (2026-10-05)
## Headline numbers
### How the checklist number is made, and what it doesn't tell you
- **Partly measured.** The menu figure comes from diffing a dump of Premiere 26.5's menus against
`filmcraft-cli commands`. The effects, transitions and audio-effects counts are checked by unit
tests against Premiere's Effects panel tree (`premiere_26_video_effects_catalogue`,
`premiere_audio_set_is_complete_and_foldered`, `premiere_26_transition_tree`).
- **Partly agent-reported.** The other areas' percentages are written by the agents doing the work.
No automated parity check runs in CI (PhotoCraft has `cargo xtask parity`; we don't yet), so
nothing stops a number drifting upward.
- **Presence, not quality.** An item counts once it exists. An effect described as an
"approximation" counts the same as an exact one. Nothing compares our rendering with
Premiere's.
- **Weights understate what users hit first.** Performance carries 5%. Plugins and AI features
aren't in the scorecard at all.
### By dimension
| Dimension | Checklist says | Honest estimate | Evidence |
|---|---|---|---|
| **Feature breadth** (menus, commands, panels) | ~86–95% | **~85%** | Menu diff against Premiere 26.5; effect, transition and audio-effect catalogues complete by test. The long tail is mostly there. |
| **Correctness in depth** | not tracked | **~60–70%** | A first-time contributor found four real bugs in core paths within hours (#5–#8: AIFF import failed, Slide left linked audio behind, transitions didn't follow ripple trims, export ignored start/end times). User reports: Color Matte always grey with no picker (#29); ALSA audio error on Linux (#23). Settings are "most wired", not all. |
| **Codecs and media** | ~90% | **~75%** | Our decoders are bit-exact on conformance streams. Real camera and phone media (variable frame rate, damaged files, unusual containers) is far less tested. No camera RAW (RED, BRAW, ARRIRAW), no E-AC-3. |
| **Export** | ~92% | **~70%** | Delivery: H.264 only (ProRes, DNxHR and image sequences cover mastering). No HEVC or AV1 export. AAF and OMF have never been validated in Avid Media Composer or Pro Tools. Export renders and encodes on the CPU only. |
| **Performance and hardware** | ~62% | **~35–40%** | Premiere runs effects, decoding and encoding on the GPU and the hardware media engines. Here, hardware decode works on macOS (VideoToolbox, #33; active in the desktop app only since #41, so releases up to 0.2.1 decoded in software) and Windows (Media Foundation / Direct3D 11 DXVA, H.264, HEVC Main / Main 10, VP9 and AV1 4:2:0 8/10-bit, #30). The GPU does compositing, blend modes (#32) and 31 common effects (GPU2): colour adjustments, crop/flip/transform, blurs, sharpen. Lumetri, keys, masks, the remaining effects and export rendering are CPU; H.264 encoding can use VideoToolbox on macOS (opt-in per export, no B-frames) and NVENC on Windows with an NVIDIA GPU (opt-in per export), and H.265 (HEVC) export exists there only, through the same hardware encoder. Linux has no hardware path. 8K isn't real-time (AV1 is on Windows with a hardware decoder). The effect list shows Premiere's "GPU accelerated" badge, but only 31 effect ids run on the GPU here; the rest, Lumetri included, run on the CPU. |
| **Stability** | not tracked | **improving, unproven** | The never-crash pass (no panics in product code, last-resort guards) landed on 2026-10-04. There's no field record from real users' projects yet. |
| **Plugins and pro ecosystem** | not tracked | **~0–10%** | No audio plugin hosting (VST3 / Audio Units) and no third-party video effects (OpenFX). For many professional editors this alone rules FilmCraft out. Team Projects and Productions are out of scope by design. |
| **AI features** | partial | **~25–35%** | Speech to text exists but is off in default builds (the `whisper` feature). Enhance Speech is a DSP chain, not a model. Auto Reframe is approximate. No Generative Extend, and no media-intelligence search or auto colour. |
| **Platforms** | not tracked | **macOS solid; Windows and Linux thin** | Development and testing happen on macOS. Windows and Linux are built for releases but barely exercised at runtime. The web build is a compile target; the browser app shell isn't finished. |
| **UI fidelity** | ~85% | **~80%** | Agents compare with Premiere reference screenshots; this hasn't been checked independently. |
### Where we are lacking
In priority order. Agents should prefer this work over adding more checklist items.
1. **Hardware acceleration** (#30), the most visible gap to users:
- hardware decode on Linux (VA-API);
- zero-copy decoded frames into wgpu;
- the remaining effects on the GPU: Lumetri, keys, Vignette, Video Limiter, masks (31 common effects are done);
- GPU export, then hardware encode: H.264 and H.265 on macOS are in (H.264 opt-in; H.265 has no other encoder), H.264 on NVENC (Windows, opt-in); HEVC on Windows and Linux remain.
2. **A measured parity number.** Add `cargo xtask parity`, run in CI. It should cover menus,
commands, effects, transitions, panels, preferences and formats against the Premiere snapshot in
`plan/premiere/`. It should report presence and fidelity separately (exact, approximate, stub).
3. **Real-world media corpus.** Phone, camera and screen recordings, variable frame rate, damaged
files, long GOPs, odd containers. Each gets an import → edit → export round-trip test with an
ffprobe / ffmpeg oracle.
4. **Workflow acceptance tests.** Scripted MCP sessions that do real jobs end to end: assemble and
trim a scene, colour it, mix and loudness-normalise it, caption it, deliver it, round-trip
through XML / AAF.
5. **Windows and Linux at runtime.** CI that runs the test suite and a headless playback/export
smoke test on both, not just release builds. Fix the Linux audio path (#23).
6. **Delivery codecs.** HEVC export is in on macOS (VideoToolbox, 8-bit, no software encoder); still to do: HEVC elsewhere, 10-bit / HDR HEVC, and AV1 (`rav1e`, BSD-2).
7. **Plugin hosting** (VST3 / Audio Units, then OpenFX). Needs FFI, so it falls under the isolated
`unsafe` crate rule (AGENTS.md §0.3). **Owner decision needed** before starting.
8. **AI features** with openly licensed local models: transcription on by default, text-based
editing, speech enhancement, reframing.
9. **The web app shell**: file access, WebCodecs, audio.
### Where we're going
**Next:** close items 1–5. Hardware acceleration fixes the gap users notice first (CPU-bound 4K playback and export). The
parity tool, media corpus and workflow tests make the progress number trustworthy. Windows and
Linux CI makes "runs everywhere" true.
**After that:** delivery codecs, plugin hosting (pending the owner's decision) and AI features,
measured with the same tools.
**Goal:** "ready for real work" at 85%+, measured rather than estimated, on all three desktop
platforms.
## Feature checklist scorecard
Measured against Premiere Pro 26.5 on this machine. Menu items: the native menu bar dump, minus
Adobe-cloud-only items (Team Projects, Productions, Firefly, Stock, Dynamic Link, account/help pages),
matched against `filmcraft-cli commands` and the UI command table. Effects: the Effects panel tree.
| Area | Weight | Measured | Coverage |
|---|---|---|---|
| Editing, timeline, trimming, multicam | 20% | core edit algebra, trim modes, sync, multicam, menu long tail, Scene Edit Detection, Automate to Sequence, Premiere default keyboard (143 new commands; 39 cloud/absent-panel keys skipped) | ~86% |
| Menus / commands | (cross-check) | ~325 of 344 in-scope menu items | ~95% |
| Effects and transitions | 12% | video effects 93/93 (+Legacy, Obsolete), audio effects 53/53, video transitions 84/84 (+21 Legacy), audio transitions 3/3; some approximations (Warp Stabilizer 2-D, Morph Cut, Auto Reframe) | ~92% |
| Media I/O and codecs | 12% | H.264, HEVC, VP9, AV1, ProRes, DNx, MJPEG, MPEG-2/MPEG-1 (4:2:2 incl. IMX/XDCAM), Opus, AAC, AC-3, MP2; MP4/MOV/MKV/WebM, MXF (incl. MPEG-2), MPEG TS/PS (AVCHD, broadcast, DVD), Ogg, image sequences, BWF; no E-AC-3, camera raw, HW decode | ~90% |
| Panels and UI fidelity | 12% | all main panels incl. Metadata, Timecode, Events, Progress, Reference Monitor, full Lumetri Scopes, audio effect editors; Project panel List/Icon/Freeform with view presets and hover scrub; Media Browser with Favorites/navigation/ingest | ~85% |
| Audio | 10% | mixer, automation (track + clip), Essential Sound, meters, all 53 effects, 5.1 tracks/buses/panner/export, voice-over record, Remix | ~88% |
| Colour | 8% | Lumetri complete, LUTs, colour management, HDR, all Lumetri Scopes (+ `scopes.read` for agents) | ~88% |
| Graphics and captions | 8% | text engine, shapes, per-character styles, our own graphics templates (.fcgt, 8 built-ins, export/install/edit), rolls/crawls, responsive pins + time, captions SRT/VTT/SCC/MCC/STL/TTML/DFXP, transcripts | ~85% |
| Export | 8% | own H.264/AAC, ProRes, DNxHR, MXF OP1a/OP-Atom, image sequences, GIF, WAV, 5.1; preset library + user presets, full Export-mode settings, queue, Quick Export; AAF (Edit Protocol) import/export, OMF 2.0 export; no AAF/OMF validation against Avid/Pro Tools yet | ~92% |
| Preferences and project management | 5% | Settings dialog with 16 categories (most settings wired), project settings, scratch disks, search bins, templates | ~80% |
| Performance | 5% | 1080p real-time, 3×1080p; 4K H.264 real-time at load ≤140 (−23% decoder cycles, opt-in draft decoding at 1/2–1/4); HEVC 2.5× cheaper, catch-up decoding, `cargo xtask bench` + `perf.stats`; blend modes on the GPU (#32); 8K and AV1 not yet real-time on a loaded machine; no hardware path on Linux, and on Windows only decode and opt-in NVENC H.264 export (see the honest assessment: ~35–40% against Premiere) | ~62% |
| **Weighted total** | | | **~87%** |
## Estimate to parity
This estimate is for the **feature checklist**. Closing the honest-assessment gaps (above) is the
"Robust on real-world material" row, plus plugins and AI features, which it doesn't include.
Measured throughput in the last work block (2026-10-01 night → 10-02): five Opus 5.5 agents in
parallel for ~4.5 wall-clock hours (~18 agent-hours including integration) moved parity by ~4 points
(≈4–5 agent-hours per point), on a machine that was heavily overloaded (load 150–300 on 14 cores) and
once ran out of disk. The remaining points are a long tail (each effect, dialog and preference page is
small, but there are many), so the estimate applies a 1.3–1.5× tail factor.
| | Opus 5.5 agent-hours | Wall-clock (4–5 parallel agents + integrator) |
| | Value | How |
|---|---|---|
| Feature parity by checklist (~19 points left) | ~90–125 | **~20–32 h** |
| Robust on real-world material (codec edge cases, 4K/8K performance, pro workflows) | +120–200 | **+30–50 h** |
| **Total to "100% and better"** | **~210–325** | **~50–82 h (≈2–3.5 days, 24/7)** |
| **Feature breadth** | **~86%** | Does each Premiere feature exist? Menus 92% (measured: label diff against Premiere 26.5.2's menu dump), effects / transitions / audio effects 100% (measured by tests), other areas estimated |
| **Ready for real work** | **~56%** (51–61%) | How close FilmCraft is to replacing Premiere for an editor on real projects: depth, correctness, file compatibility, speed, stability, ecosystem. Estimated depth per area, combined by a [written formula](docs/target-app-parity.md#three-readiness-numbers) |
| **Mainstream practitioner** | **~55%** | A typical professional editor's weekly work only, discounted ×0.90 interaction, ×0.92 stability, ×0.88 file exchange with Premiere users ([method](docs/target-app-parity.md#mainstream-practitioner)) |
| **Essentials user** | **~60%** | Import, cut, title, music, export at default settings, discounted ×0.88 launch / stability, ×0.90 discoverability, ×0.95 opening files people send ([method](docs/target-app-parity.md#essentials-user)) |
| **To beta** | **~400–700 Opus 5.5 agent-hours** | Estimated, calibrated from this repo's PR history |
| **To full parity** | **~1,550–2,750 agent-hours** | ~70% parallelizes; plugins, AI models, native-speaker review and some hardware need a human |
| Code (measured) | 45 crates, ~343k lines of Rust, 2,520 tests, 268 merged PRs, 69 authors | `git`, `gh`, `grep` |
The 2026-10-02 block moved parity ~62% → ~72% with six agents in ~6 wall-clock hours, despite the
disk filling twice; the remaining work is mostly panels (scopes, Metadata, Media Browser), export
presets/queue, codecs (MXF, image sequences, hardware decode), performance and graphics templates.
Two numbers, because they answer different questions. The gap between them is the work that
matters most now: breadth is nearly done, depth is not.
Limits on speed: CPU and disk on one machine (more than ~5 agents slows everyone down: each worktree
build is 6–7 GB and a full `cargo xtask ci` takes 20–60 min under load), and a single integrator
merging, resolving conflicts (e.g. two agents bumping the project schema) and re-running CI. With one
agent and no parallelism, multiply wall-clock by ~3–4.
### Readiness by audience
Not reachable clean-room and locally: **Generative Extend** (needs a large video-generation model).
**Enhance Speech** and **Auto Reframe** are feasible only with openly licensed models we can ship.
| Audience | Ready % | Opus 5.5 agent wall-clock hours to ~95% | Work that dominates |
|---|---|---|---|
| Full target (ready for real work) | ~56% | ~1,450–2,600 (~70% parallelizes) | Plugin hosting, AI models, localization incl. RTL / Devanagari, camera RAW and delivery codecs, `.prproj`, hardware acceleration, UI fidelity |
| Mainstream practitioner | ~55% | ~500–850 (~65% parallelizes) | `.prproj` import (60–120), UI fidelity (120–200), stability and PR CI (40–80), 10-bit / 4:2:2 camera decode (30–50), playback / hardware (60–100), import and export gaps (60–100) |
| Essentials user | ~60% | ~180–350 (~60% parallelizes) | Launch crashes on Windows / Intel / older macOS (40–80), discoverability and timeline friction (60–110), playback lag (30–60), common import gaps (20–40), audio devices (15–30) |
## Milestones
Method and calibration: [docs/target-app-parity.md](docs/target-app-parity.md#three-readiness-numbers).
Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining agent-hours.
## By dimension
| # | Milestone | Status | Done | Remaining | Est. |
|---|---|---|---|---|---|
| M0 | Skeleton + visual shell | ✅ | Workspace, 20 crates, dock/workspaces, Premiere 26 look, native menus, control channel, MCP, xtask gates (layers, wasm) | — | — |
| M1 | Media I/O | ✅ | MP4/MOV demux+mux, WAV, stills, MJPEG, symphonia audio (MP3/FLAC/ALAC/Vorbis), GOP seek + frame cache, import | Media Browser polish | 2 |
| M2 | H.264 decoder | ✅ | Own decoder, bit-exact on 37+ streams, 500–600 fps 1080p | — | — |
| M3 | Editing core | 🟡 | Edit algebra (insert/overwrite/razor/lift/extract/ripple/roll/slip/slide/rate-stretch/nest/paste), tools, markers, trim mode + Trim Monitor + dynamic J/K/L trimming, Keyboard Shortcuts editor with FilmCraft/Premiere/FCP/Avid presets; Edit/Clip/File menu commands (Label colours and Select Label Group, Paste/Remove Attributes, Select All Matching, Remove Unused, Consolidate Duplicates, Sequence From Clip, Bin From Selection, Offline File, Close Project, Make/Edit Subclip, Modify Audio Channels/Timecode, Frame Hold Options/Add Frame Hold/Insert Frame Hold Segment, Time Interpolation with frame blending, Fit/Fill frame, Breakout to Mono, Extract Audio, Replace With Clip; M3.11: Scene Edit Detection (pure-Rust cut detection, background job), Normalize Mix Track, Simplify Sequence, Transcribe Sequence, Find/Find Next and search bins, Automate to Sequence, Edit Original, Edit Offline, Source Settings, Update Metadata (XMP), Generate Audio Waveform, Project Settings General/Scratch Disks, Get Media File Properties, Save as Template, Selection as FilmCraft Project, Avid Log Exchange export, Flash Cue markers, Dynamic Audio Waveforms, Reveal Log Files, System Compatibility Report); M3.12 keyboard parity: Premiere's default keyboard (~115 keyboard-only commands: edit-point navigation on targeted/any track, select clip at playhead/next/previous, extend edit to playhead, nudge/slip/slide selection, target and source-patch toggles, clip volume ±1 dB/many, frame maximize/full screen/panel cycling, monitor zoom, track heights, Project and Text panel keyboard navigation, text size/leading/alignment, Export Frame, poster frames; see docs/keyboard.md); multicam (Create Multi-Camera Source Sequence, Multi-Camera view with live switching on 1–9, angle switching, Enable/Flatten, Edit Cameras) and sync (Synchronize, Merge Clips; In/Out/timecode/marker/audio — GCC-PHAT, sample-accurate) | Multicam paging >16 angles and grid thumbnails, optical flow (renders as frame blending), 39 Premiere default shortcuts skipped with reasons in docs/keyboard.md (Productions, AI/cloud tools, work area bar, Production/Search panels) | 6–10 |
| M4 | Playback | 🟡 | Audio-clock master, prefetch with cancellation, J/K/L, correct dropped-frame stats, playback resolution, render bar + content-hashed render previews, App Nap opt-out; 1080p H.264 and 3 stacked 1080p streams play with 0 dropped frames | 4K under load, 8K, frame-threaded AV1 decode, reduced-resolution decode for multicam grids | 6–10 |
| M5 | Effects, keyframes, GPU | 🟡 | All 93 Premiere 26 video effects and 84 transitions (CPU), blend modes and 31 common effects on the GPU (#32, GPU2), keyframes + value/velocity graphs, wgpu compositor, effect + opacity masks (ellipse/polygon/pen, feather, expansion, modes; CPU/WGSL parity), mask tracking (Lucas–Kanade + RANSAC), adjustment layers, effect presets (built-in + user, JSON import/export) | Effects on the GPU (in progress, #30), GPU masks in the live path, GPU export, exact Warp Stabilizer (3-D) and Morph Cut | 20–30 |
| M6 | Export | ✅ | Own H.264 encoder (High/Main/Baseline, B-frames, VBR/CBR/2-pass) → MP4 or QuickTime + own AAC; ProRes, DNxHR, MJPEG, PNG/TIFF/BMP sequences (Premiere-style numbering), GIF, WAV, AIFF; background jobs; M6.5: Export mode parity (frame size / rate / scaling / pixel aspect, profile / level, CBR / VBR 1- and 2-pass, keyframe distance, audio codec / rate / channels / bitrate / sample size, multiplexer, burn-in or sidecar captions, image / name / timecode overlays, video limiter, loudness normalization with true-peak limiter, metadata, ranges, estimated size, summary), 24 built-in presets + user presets with favourites and import/export (Preset Manager), export queue (reorder, cancel, retry, several sequences / ranges), Quick Export; `export.*` commands and `filmcraft-cli export --preset` | Interlaced encoding, 5.1 audio, smart render, publishing destinations | 6–8 |
| M7 | Audio | 🟡 | Mixer graph (tracks → submixes → Mix, pre/post-fader inserts and sends, latency-compensated, sample-accurate, ~6× realtime for 24 tracks × 3 effects on one core), Audio Track Mixer + Audio Clip Mixer panels, track automation (Off/Read/Latch/Touch/Write, recorded live while playing, thinned to keyframes, timeline lanes with pen editing), solo/solo-safe, channel mapping basics, peak + BS.1770 loudness meters (match ffmpeg), DSP crate with 16 clip/track effects, Audio Gain (set/adjust/normalize), Constant Power / Constant Gain / Exponential Fade, Essential Sound (types, Loudness auto-match, Repair incl. DeEss/DeReverb, Clarity, Creative, Ducking keyframes, presets; full Dialogue chain 22× realtime) | Music duration remix, ML speech enhancement, 5.1 panner and multichannel buses, voice-over record, effect editor windows (EQ curve), remaining effects (multiband, convolution reverb), clip-mixer automation recording | 6–9 |
| M8 | Colour | 🟡 | Lumetri: basic, creative + looks, RGB & hue curves, wheels, HSL secondary, vignette, section bypass; Input LUT / Look LUT (.cube 1D/3D/shaper, .3dl; tetrahedral CPU + WGSL; project LUT library; built-in camera conversions); Colour Match (Oklab tonal-range statistics, skin protection, solved in Lumetri wheels); colour management: Rec. 709 / Rec. 2100 PQ / HLG working spaces + wide gamut, Interpret Footage colour space (S-Log3, V-Log, Canon Log 2/3, LogC3/4, Apple Log, D-Log from published specs), metadata auto-detect, BT.2390 tone mapping, gamut mapping, HDR export signalling (VUI/colr/mdcv/clli/SEI, ffprobe-verified); Lumetri Scopes: Vectorscope YUV (75/100 % targets, skin-tone line) and HLS, Histogram, Parade (RGB/YUV/RGB-White), Waveform (RGB/Luma/YC/YC no Chroma), presets, Rec. 601/709/2100, 8-bit/float/HDR (cd/m²) scales, multi-scope grid, `scopes.read` | HDR-aware Lumetri maths, macOS EDR monitors, mastering metadata → tone-map peak, D-Log M (no published formula), HSL Secondary refine | 3–5 |
| M9 | More codecs | 🟡 | ProRes decode+encode, AAC decode+encode, HEVC Main/Main 10 decoder (bit-exact on 41 fixtures, ~225 fps 1080p), VP9 decoder (profiles 0–3, 8/10/12-bit, bit-exact on 50+ fixtures; WebM/MKV `V_VP9` and MP4 `vp09` import with key-frame-checked seeking), Matroska/WebM import (H.264/HEVC/VP9/ProRes/MJPEG + AAC/Opus/FLAC/MP3/Vorbis/PCM), Opus decoder (SILK/CELT/hybrid, 5.1/7.1 multistream; all RFC 8251 vectors range-exact; WebM/MKV/MP4), DNxHD/DNxHR decoder (all SMPTE ST 2019-1 CIDs, 8/10/12-bit, 4:2:2/4:4:4, interlaced, alpha; within IDCT precision of ffmpeg on 22 fixtures) + DNxHR LB/SQ/HQ/HQX/444 encoder and MOV `AVdh` export, AV1 decoder (Main profile, 8/10-bit, all intra/inter tools, loop filters, superres, film grain, intra BC, spatial layers; bit-exact vs libdav1d on 19 SVT fixtures + 22 libaom vectors; MP4 `av01` / WebM `V_AV1` import), MXF import (OP1a/OP-Atom; AVC with POC-ordered B pictures bit-exact, DNxHD/DNxHR, ProRes, PCM/AES3; MPEG-2 reported unsupported; index-table seeking, material package timecode), Ogg Opus/Vorbis (granule seeking, pre-skip, end trimming), image sequences, Broadcast WAV timecode | VP9 frame threading, AV1 threading/SIMD (~4 fps 1080p today), AV1 High/Professional profiles, MPEG-2 decoder, hardware decode | 12–20 |
| M10 | Graphics & captions | 🟡 | Caption tracks (Subtitle/CEA-608/708/Teletext formats, track style), SRT/WebVTT/SCC import+export (frame-exact, property-tested), caption editing (add/split/merge/trim/move, sync-locked insert/extract), Text panel Captions tab, burn-in in Program monitor and export; text engine (`crates/text`: bundled + system fonts, harfrust shaping, bidi, line breaking, paragraph layout, glyph cache; 3-line 1080p title ≈ 0.15 ms warm); graphic clips with text + shape layers (fill, 2 strokes, background, shadow, keyframable transform), Type tool with on-monitor editing, shape/pen tools, Properties/Essential Graphics editor, align/distribute; graphics templates (own `.fcgt` format, 8 original built-ins, Browse tab with engine-rendered thumbnails, export / install / apply / editable properties; never reads .mogrt), rolls/crawls with ease and pre/postroll, responsive pins and intro/outro, per-character styles, Upgrade Caption to Graphic, source graphics, Replace Fonts; MCC (608+708), EBU STL, TTML/IMSC1, DFXP (frame-exact, property-tested) | 608/708 embedding in video streams, speech-to-text | 12–18 |
| M11 | Interchange & project management | ✅ | `.fcproj` schema versions + migrations, atomic saves, Save a Copy/Revert, auto-save ring + crash-recovery journal (Preferences ▸ Auto Save, recovery prompt), FCP7 XML, FCPXML, EDL, OTIO; offline media (own slate) + Link Media (fingerprint-checked relink, folder remap, search, Align Timecode, Make Offline); proxies (ProRes Proxy/LT, H.264 ¼/½ background jobs, attach/detach/reconnect, monitor toggle, export full-res) + ingest (copy/transcode/proxies); Project Manager (collect, consolidate + transcode with handles, size estimate) | Rename media to clip names, image-sequence conversion, Media Browser-driven relink, smart (cross-drive) path tracking | 1–2 |
| M12–M16 | Multicam, web (WASM), platform, long tail | 🟡 | Multicam: audio sync (GCC-PHAT, sub-sample), Merge Clips, multi-camera source sequences, Multi-Camera view with live switching (1–9); L0–L4 crates compile to wasm32 | Web app shell (file access, WebCodecs, audio), scene detection, auto reframe, ~800 remaining commands and dialogs, performance hardening | 35–55 |
| Dimension | Ready | Hours | Doc |
|---|---|---|---|
| Features (editing, effects, colour, audio, graphics, export settings) | ~66% | 350–620 | [target-app-parity.md](docs/target-app-parity.md#by-feature-area) |
| UI / UX fidelity | ~60% | 120–200 | [ui-parity.md](docs/ui-parity.md) |
| File formats (containers, `.prproj`, interchange, captions, stills) | ~60% | 140–240 | [file-format-parity.md](docs/file-format-parity.md) |
| Codecs | ~60% | 150–250 | [codec-parity.md](docs/codec-parity.md) |
| Hardware acceleration | ~45% | 120–200 | [hardware-parity.md](docs/hardware-parity.md) |
| Localization | ~35% | 150–230 | [localization-parity.md](docs/localization-parity.md) |
| Performance | ~55% | 60–120 | [performance.md](docs/performance.md) |
| Stability | ~45% | 60–120 | [gaps.md G4](docs/gaps.md#g4-crashes-at-start-up-and-no-test-ci-on-pull-requests) |
| Platforms (macOS, Windows, Linux, FreeBSD, web) | ~60% | 50–90 | [gaps.md G9](docs/gaps.md#g9-windows-and-linux-at-run-time) |
| Plugins and ecosystem | ~5% | 200–400 | [gaps.md G11](docs/gaps.md#g11-no-plugin-hosting) |
| AI features | ~25% | 150–300 | [gaps.md G12](docs/gaps.md#g12-ai-features) |
## Running now
## Features
- **Hardware acceleration (#30):** landed: blend modes on the GPU (#32); VideoToolbox hardware decode + the `platform` FFI crate (#33); 31 common effects on the GPU (GPU2). Windows: Media Foundation / Direct3D 11 H.264 + HEVC decode (HW2) and VP9 + AV1 (HW3), NVENC H.264 export (HW4, opt-in). Next: VA-API (Linux) decode, zero-copy upload, Lumetri on the GPU.
- **Next:** [Where we are lacking](#where-we-are-lacking) items 1–5. GPU export waits for the export-crate work (#19 and the never-crash follow-up) to land.
| Area | Breadth | Ready | Hours |
|---|---|---|---|
| Editing, timeline, trimming, multicam | ~90% | ~70% | 80–140 |
| Effects and transitions | 100% (measured) | ~60% | 80–140 |
| Media import and codecs | ~75% | ~60% | 150–250 |
| Audio | ~88% | ~65% | 50–90 |
| Panels and UI fidelity | ~88% | ~60% | 120–200 |
| Colour | ~90% | ~70% | 40–70 |
| Graphics, titles, captions | ~85% | ~65% | 50–90 |
| Export and delivery | ~80% | ~65% | 60–110 |
| Project files and interchange | ~70% | ~45% | 90–160 |
| Preferences and project management | ~80% | ~65% | 20–40 |
## Log
Evidence for every row: [docs/target-app-parity.md](docs/target-app-parity.md#by-feature-area).
## Languages
Premiere ships 10 interface languages. Measured from our catalogs (3,498 strings):
| Language | Status | Strings |
|---|---|---|
| English | full | 100% |
| Simplified Chinese | partial (whole interface; engine errors English) | 99.6% |
| Spanish | partial (whole interface) | 99.9% |
| Hindi | none (no Devanagari shaping in the interface) | 0% |
| Arabic | none (no RTL interface) | 0% |
| French | partial (whole interface; native review welcome) | ~100% |
| Portuguese (Brazil) | menus only | 9.9% |
| Indonesian | none | 0% |
| Japanese | partial (whole interface) | 99.7% |
| German | none | 0% |
| Korean | none | 0% |
| Vietnamese | none | 0% |
Also shipped: Ukrainian (menus only, 10%). Detail: [docs/localization-parity.md](docs/localization-parity.md).
## Upcoming
Ranked; detail and "done when" in [docs/roadmap.md](docs/roadmap.md) and [docs/gaps.md](docs/gaps.md).
1. **No launch crashes, and CI on pull requests** (macOS 12, Intel UHD, Windows 11 reports; no workflow runs the tests today). 40–80 h.
2. **Open Premiere's `.prproj`**, the beta blocker. 60–120 h.
3. **10-bit / 4:2:2 camera media without hardware** (H.264 High 10 / 4:2:2, HEVC 4:2:2; #626). 30–50 h.
4. **Measure fidelity against Premiere** and add `cargo xtask parity`. 50–90 h.
5. **Real-world media corpus and workflow acceptance tests.** 40–70 h.
6. **Hardware acceleration, continued** (#30): zero-copy upload, more effects on the GPU, Quick Sync / AMD encoders, VideoToolbox Main 10. 120–200 h.
7. **UI fidelity:** Program monitor transform handles, Effect Controls, docking. 120–200 h.
8. **Plugin hosting** (VST3 / AU, then OpenFX). Owner decision needed. 200–400 h.
## Progress log
- **2026-10-10 (changed media):** Settings ▸ Media ▸ Automatically refresh growing files works: media files that change on disk while the project is open (a recording still being written, a render written over the file a clip uses) are read again every *Refresh growing files every* seconds and as soon as the window comes back to the front: the new length, picture and identity, without an undo step (`media.refreshChanged`, `media_watch`).
- **2026-10-10 (re-measure):** full parity re-measure against Premiere Pro 26.5.2: menu label diff 338 / 366 (92%), installed bundle listings (10 UI languages, exporter and codec frameworks), 188 open issues. Breadth ~86% (file formats now counted), ready ~55%, stage **alpha**; progress docs reorganized per the craftrules standard ([target-app-parity](docs/target-app-parity.md), [gaps](docs/gaps.md), [roadmap](docs/roadmap.md), [codec](docs/codec-parity.md), [file-format](docs/file-format-parity.md), [hardware](docs/hardware-parity.md), [UI](docs/ui-parity.md), [localization](docs/localization-parity.md)).
- **2026-10-10:** ~75 community PRs merged: GPU export rendering (Auto) with Vignette, Video Limiter and Lumetri basic / creative / vignette on the GPU (#421); Linux NVDEC H.264 / HEVC decode (#549); text-to-speech narrations with Kokoro-82M (#278); Japanese for the whole interface (#449) and Ukrainian menus (#407); ALSA default-device fallback (#254, fixes #23); drop import into the timeline (#517); system appearance (#505); timecode scrub editing (#576); NVENC RGBA input (#486); marker CSV export (#454); many fixes (#526–#575).
- **2026-10-09:** release 0.5.0. NVENC H.265 export on Windows, Main and Main 10 HDR (#328, #360); NVENC H.264 export (#386); VA-API H.264 / HEVC decode (#358); Source monitor playback (#423); multi-stream container audio, schema v13 (#361); New Sequence and Sequence Settings dialogs (#269, #355); Spanish for the panels (#238); background decoders and memory stress tests (#385, #381).
- **2026-10-08 (Source range dragging):** implicit full-clip In/Out, video-only/audio-only/linked both controls, picture/waveform drag gestures and one-command undoable timeline placement. Shared Source/Program draggable frame-bounded handles and range translation, cancellation, Source marker/navigation routing and marked-range playback/looping.
- **2026-10-08 (Source playback):** normal forward playback for video/audio media and subclips in the Source monitor, an independent Source clock, Play/Pause and focused Space, frame stepping, device-clock audio with wall-clock fallback, seeking, and end-of-clip stopping. Windows desktop review verified a real video file and standalone WAV, including In/Out marking. Reverse/shuttle Source playback and source sequences remain unsupported; no parity percentage changed.
- **2026-10-08 (night):** HDR H.265 (HEVC Main 10, PQ and HLG, BT.2020, limited range) export on Windows through NVENC (#30): an HDR sequence now exports as real 10-bit HDR (`rgbf_to_yuv420_10` into P010 input buffers; VUI BT.2020 + PQ / HLG; the HDR10 mastering-display and content-light messages as SEI on every IDR for PQ, through the generic SEI payload array because NVENC 12.1 has no mastering-display field; `colr` / `mdcv` / `clli` in the sample entry), where the GPU reports 10-bit HEVC (new `register_hdr_probe` / `hdr_available` in the export crate; elsewhere, and on macOS, HDR sequences still tone-map to 8-bit SDR HEVC, and `settings.sdr` forces that). ffprobe reads `hevc` / `Main 10` / `hvc1` / `yuv420p10le` / `bt2020` / `smpte2084` or `arib-std-b67` / `bt2020nc` / `tv` with the mastering-display and content-light side data for PQ; ffmpeg's decode equals ours code for code; the in-process round trip is at 71.1 dB luma PSNR on the 10-bit scale with 803 distinct levels on a ramp and the peak code 940 intact (RTX 5060). 8-bit SDR H.265 is unchanged.
- **2026-10-08:** H.265 (HEVC Main, 8-bit 4:2:0, SDR) export on Windows through NVENC (#30): the NVENC session, ring and NV12 path of the H.264 encoder now serve both codecs (the H.264 export is byte-for-byte unchanged); the format is available where the GPU has an HEVC encoder, choosing it is the opt-in, and `hvcC` is built from the SPS the encoder wrote. ffprobe reads `hevc` / `Main` / `hvc1` / `yuv420p` / BT.709; our own HEVC decoder decodes it at 48.9 dB luma PSNR (RTX 5060). Main 10 and HDR remain.
- **2026-10-08 (#285):** `filmcraft-cli` no longer panics with "Broken pipe" when its stdout is closed early (`filmcraft-cli commands | head`): later output is dropped, the rest of the work (script lines, `--save`, `--save-as`) still runs, and the exit status still reports failures. Other stdout write errors are reported and make the exit status 1. No change to the checklist or estimates.
- **2026-10-06:** 21 community PRs landed (#19, #28, #38, #39, #41–#60). Highlights: hardware decoding is now actually on in the desktop app (#41: `register()` sat inside a `log::info!` that never ran, so #33's VideoToolbox path only worked in the CLI and benchmarks); decode memory on long timelines cut sharply (frame pool, shared GOP-cache budget, positional file reads); exports are byte-identical across machines (#19); crash fix for odd track names over MCP (#44); ripple trim, ripple delete and speed changes move split-edit clips once (#56–#58); Japanese interface using an installed system font (#38; no fonts bundled); MCP annotations, resources, export progress and cancellation (#28).
@@ -203,6 +157,13 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining
- **2026-09-30 (late morning):** keyframe value/velocity graphs; xtask gates; H.264 MP4 export; Lumetri curves, wheels, looks, HSL secondary.
- **2026-09-30 (early morning):** H.264 decoder, ProRes, AAC; GPU compositor; MCP; Premiere 26 visual fidelity pass.
### Interface localization
## Revision history
Edit > Language switches English/Japanese/Spanish/Portuguese (Brazilian) and persists in UI state. Commands `app.language.english`, `app.language.japanese`, `app.language.spanish` and `app.language.portuguese` are reachable through the control channel. Core native/in-window menu labels are translated; untranslated labels use English. Japanese text uses the Japanese fonts from [craft-fonts](https://github.com/storytold/craft-fonts) when FilmCraft is built with them (`CRAFT_FONTS_DIR`; all official releases, including the web build), otherwise a Japanese font already installed on the system; with neither, switching to Japanese is refused with a message. Dialog/panel translation coverage remains open. Existing vertical text support is preserved.
| Date | Change | Summary |
|---|---|---|
| 2026-10-10 | minor | Readiness-by-audience table with hours to ~95% per audience |
| 2026-10-10 | minor | Headline adds mainstream practitioner (~55%) and essentials user (~60%); ready for real work ~55% -> ~56% (formula written out, same inputs); beta distance ~19 points |
| 2026-10-10 | minor | Stage banner links the core-workflow gate (all six workflows pass on macOS; stays alpha) |
| 2026-10-10 | major | Restructured to the craftrules progress-docs shape: stage banner, headline numbers, dimension / feature / language tables, upcoming; re-measured against Premiere 26.5.2 (breadth ~87 → ~86%, ready ~50–60% → ~55%); honest assessment and scorecard moved to docs/target-app-parity.md, "Where we are lacking" to docs/gaps.md, milestones to docs/roadmap.md, interface localization to docs/localization-parity.md |
| 2026-10-08 | minor | Log entries for HEVC export, Source playback and range dragging |
| 2026-10-05 | major | Honest assessment added: checklist ~87% vs ready for real work ~50–60% |
+5 -1
View File
@@ -19,13 +19,17 @@ whisper = ["filmcraft-engine/whisper", "filmcraft-engine/speech-download"]
[dependencies]
filmcraft-engine = { workspace = true }
filmcraft-export = { workspace = true }
filmcraft-project = { workspace = true }
filmcraft-render = { workspace = true }
filmcraft-time = { workspace = true }
filmcraft-gpu = { workspace = true }
filmcraft-media = { workspace = true }
filmcraft-codecs = { workspace = true }
filmcraft-platform = { workspace = true }
filmcraft-mxf = { workspace = true }
filmcraft-mpegts = { workspace = true }
filmcraft-ogg = { workspace = true }
filmcraft-time = { workspace = true }
filmcraft-automation = { workspace = true }
serde_json = { workspace = true }
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }
+40
View File
@@ -19,6 +19,7 @@ const VALUED: &[&str] = &[
"--end",
"--data-dir",
"--settings",
"--gpu-rendering",
];
#[derive(Debug, Default)]
@@ -60,6 +61,17 @@ impl Args {
self.opts.iter().any(|(n, _)| n == k)
}
/// A version query: `--version` anywhere, or `-V` in the subcommand position, so global
/// options may come first (`--demo --version`, `--compact -V`).
pub fn wants_version(&self) -> bool {
self.flag("--version") || self.pos(0) == Some("-V")
}
/// The first option that is not in `allowed`, if any.
pub fn unknown_opt(&self, allowed: &[&str]) -> Option<&str> {
self.opts.iter().map(|(n, _)| n.as_str()).find(|n| !allowed.contains(n))
}
/// Command params from positionals `from..`: one JSON object, or `key=value` pairs
/// (dotted keys nest: `color.r=1`).
pub fn params_from(&self, from: usize) -> Result<Value, String> {
@@ -135,9 +147,37 @@ mod tests {
assert_eq!(a.params_from(2).unwrap(), json!({"a": {"b": [1]}}));
}
#[test]
fn version_after_global_options() {
for s in ["--version", "-V", "--demo --version", "--compact -V", "--data-dir d --version", "export --version"] {
assert!(args(s).wants_version(), "{s}");
}
for s in ["version", "--data-dir -V", "exec x -V", "-- --version"] {
assert!(!args(s).wants_version(), "{s}");
}
}
#[test]
fn unknown_opt_reports_first_unlisted_option() {
let allowed = ["--bridge", "--project"];
assert_eq!(args("mcp --automation-read-root X").unknown_opt(&allowed), Some("--automation-read-root"));
assert_eq!(args("mcp --bridge=a --typo=1").unknown_opt(&allowed), Some("--typo"));
assert_eq!(args("mcp --project p --bridge a").unknown_opt(&allowed), None);
}
#[test]
fn double_dash_ends_options() {
let a = args("import -- --weird-name.mov a.mov");
assert_eq!(a.positionals, ["import", "--weird-name.mov", "a.mov"]);
}
#[test]
fn gpu_rendering_takes_a_space_separated_value() {
let a = args("export --gpu-rendering off out.wav --start 0 --end 0.1");
assert_eq!(a.positionals, ["export", "out.wav"]);
assert_eq!(a.opt("--gpu-rendering"), Some("off"));
let b = args("export out.wav --gpu-rendering auto");
assert_eq!(b.positionals, ["export", "out.wav"]);
assert_eq!(b.opt("--gpu-rendering"), Some("auto"));
}
}
+222 -23
View File
@@ -4,6 +4,32 @@
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
/// `println!` that doesn't panic when stdout is closed (`filmcraft-cli commands | head`) with
/// "failed printing to stdout: Broken pipe (os error 32)". Once stdout has failed, later output is
/// dropped and the work goes on: see `stdout_failed`.
macro_rules! outln {
($($arg:tt)*) => {{
use std::io::Write as _;
if !$crate::stdout_gone() {
if let Err(e) = writeln!(std::io::stdout(), $($arg)*) {
$crate::stdout_failed(e);
}
}
}};
}
/// `print!` counterpart of `outln!`.
macro_rules! out {
($($arg:tt)*) => {{
use std::io::Write as _;
if !$crate::stdout_gone() {
if let Err(e) = write!(std::io::stdout(), $($arg)*) {
$crate::stdout_failed(e);
}
}
}};
}
mod args;
mod probe;
@@ -11,6 +37,28 @@ use args::{Args, parse_value};
use filmcraft_automation::BridgeClient;
use filmcraft_engine::Session;
use serde_json::{Value, json};
use std::sync::atomic::{AtomicBool, Ordering};
/// Set once a write to stdout has failed; later output is dropped.
static STDOUT_GONE: AtomicBool = AtomicBool::new(false);
/// Set when stdout failed for a reason other than a closed reader: the exit status becomes 1.
static STDOUT_ERROR: AtomicBool = AtomicBool::new(false);
fn stdout_gone() -> bool {
STDOUT_GONE.load(Ordering::Relaxed)
}
/// stdout went away. The program doesn't stop there: the rest of `exec`/`run`/`import` (later
/// script lines, `--save`, `--save-as`) still runs and the exit status still tells how it went, so
/// `exec … --save | head -1` keeps the edit. A reader that stopped early (a closed pipe) is not an
/// error, as in ripgrep; any other write error is reported and makes the exit status 1.
fn stdout_failed(e: std::io::Error) {
if STDOUT_GONE.swap(true, Ordering::Relaxed) || e.kind() == std::io::ErrorKind::BrokenPipe {
return;
}
diag(format_args!("filmcraft-cli: can't write to stdout: {e}"));
STDOUT_ERROR.store(true, Ordering::Relaxed);
}
const HELP: &str = "\
filmcraft-cli: drive FilmCraft from the shell (headless engine, or the live app with --bridge)
@@ -27,10 +75,11 @@ SUBCOMMANDS
inspect [project|sequence] project tree or active sequence as JSON (default: both)
import <file>... import media into the project
export <out> [--preset name] [--format f] [--range r] [--start s --end s] [--settings json]
[--gpu-rendering auto|off]
[--scale f] [--quality 0-100] [--no-audio] [--queue]
export the active sequence and wait for it to finish: with an
export preset (`export --list-presets`; built-in or the user's), or
a format (h264|hevc|prores|dnxhr|apv|mjpeg|mxf-op1a|mxf-opatom|png|tiff|bmp|gif|wav|aiff, guessed
a format (h264|hevc|prores|dnxhr|apv|mjpeg|mxf-op1a|mxf-opatom|png|tiff|bmp|gif|wav|aiff|aac, guessed
from the extension); --range entire|inOut|workArea, or a custom
range in seconds; --settings is ExportSettings JSON merged over the
preset; --scale renders at a fraction of the frame size (0.5 =
@@ -67,13 +116,23 @@ EXIT STATUS
0 success · 1 a command failed · 2 usage error
";
/// Writes one diagnostic line to `w`, dropping any write error. `eprintln!` panics (exit 101) when stderr's
/// reader is gone, which would turn a usage error, a failure or a finished export into a crash.
fn write_diag(w: &mut dyn std::io::Write, msg: std::fmt::Arguments) {
let _ = writeln!(w, "{msg}");
}
fn diag(msg: std::fmt::Arguments) {
write_diag(&mut std::io::stderr(), msg);
}
fn usage(msg: impl std::fmt::Display) -> ! {
eprintln!("filmcraft-cli: {msg}\nRun `filmcraft-cli help` for usage.");
diag(format_args!("filmcraft-cli: {msg}\nRun `filmcraft-cli help` for usage."));
std::process::exit(2)
}
fn fail(msg: impl std::fmt::Display) -> ! {
eprintln!("filmcraft-cli: {msg}");
diag(format_args!("filmcraft-cli: {msg}"));
std::process::exit(1)
}
@@ -134,7 +193,7 @@ impl Backend {
fn print(a: &Args, v: &Value) {
let s = if a.flag("--compact") { serde_json::to_string(v) } else { serde_json::to_string_pretty(v) };
println!("{}", s.unwrap_or_default());
outln!("{}", s.unwrap_or_default());
}
/// Export format from the output file's extension.
@@ -150,28 +209,68 @@ fn format_for(path: &str) -> Option<&'static str> {
"tif" | "tiff" => "tiff",
"bmp" => "bmp",
"aif" | "aiff" => "aiff",
"m4a" | "aac" => "aac",
_ => return None,
})
}
/// OS hardware video decoders (VideoToolbox on macOS, Media Foundation on Windows) in front of our
/// own, as in the desktop app. Also what `mcp` and the headless commands decode with.
/// The GPU export frame renderer (filmcraft-gpu's off-screen compositor behind filmcraft-export's
/// frame-renderer hook): exports with GPU rendering Auto composite on the GPU and fall back to the
/// CPU reference renderer wherever it cannot.
struct GpuFrameRenderer(filmcraft_gpu::ExportRenderer);
impl filmcraft_export::FrameRenderer for GpuFrameRenderer {
fn render(
&mut self,
project: &filmcraft_project::Project,
seq: filmcraft_project::ItemId,
t: filmcraft_time::Tick,
opts: filmcraft_render::RenderOptions,
sources: &dyn filmcraft_render::SourceProvider,
) -> Option<filmcraft_render::Image> {
self.0.render(project, seq, t, opts, sources)
}
}
fn register_gpu_frame_renderer() {
filmcraft_export::register_frame_renderer(|| {
filmcraft_gpu::ExportRenderer::new().map(|r| Box::new(GpuFrameRenderer(r)) as Box<dyn filmcraft_export::FrameRenderer>)
});
}
fn register_hardware_decoders() -> filmcraft_platform::Availability {
filmcraft_platform::register()
}
#[tokio::main]
async fn main() {
if matches!(std::env::args().nth(1).as_deref(), Some("--version" | "-V")) {
println!("filmcraft-cli {}", env!("CARGO_PKG_VERSION"));
cli().await;
if STDOUT_ERROR.load(Ordering::Relaxed) {
std::process::exit(1);
}
}
async fn cli() {
let a = Args::parse(std::env::args().skip(1));
// `--help` anywhere (`filmcraft-cli --help`, `filmcraft-cli export --help`) prints the reference:
// the parser takes it as a flag option, so it never reaches the subcommand match.
if a.flag("--help") {
out!("{HELP}");
return;
}
// `--version` anywhere and `-V` as the subcommand, also after global options (`--demo --version`, #710).
if a.wants_version() {
outln!("filmcraft-cli {}", env!("CARGO_PKG_VERSION"));
return;
}
register_hardware_decoders();
let a = Args::parse(std::env::args().skip(1));
register_gpu_frame_renderer();
let Some(cmd) = a.pos(0) else { usage("missing subcommand") };
match cmd {
"help" | "--help" | "-h" => print!("{HELP}"),
"version" => println!("filmcraft-cli {}", env!("CARGO_PKG_VERSION")),
"help" | "-h" => out!("{HELP}"),
"version" => outln!("filmcraft-cli {}", env!("CARGO_PKG_VERSION")),
"probe" => {
let path = a.pos(1).unwrap_or_else(|| usage("probe <media> [--image-sequence]"));
match probe::probe(path, a.flag("--image-sequence")) {
@@ -189,11 +288,11 @@ async fn main() {
src.video_frame(filmcraft_media::FrameRequest::full(rate.tick_of(f))).unwrap_or_else(|e| fail(e));
}
let dt = t0.elapsed().as_secs_f64();
println!("{n} frames in {dt:.2}s → {:.1} fps (sequential, via Mp4Source)", n as f64 / dt);
outln!("{n} frames in {dt:.2}s → {:.1} fps (sequential, via Mp4Source)", n as f64 / dt);
let t1 = std::time::Instant::now();
let f = src.video_frame(filmcraft_media::FrameRequest::full(rate.tick_of(n / 2))).unwrap_or_else(|e| fail(e));
let (w, h, _) = f.to_linear_f32_decimated(2);
println!("½-res linear conversion {w}x{h}: {:.1} ms", t1.elapsed().as_secs_f64() * 1000.0);
outln!("½-res linear conversion {w}x{h}: {:.1} ms", t1.elapsed().as_secs_f64() * 1000.0);
}
"commands" | "describe" => {
let mut b = Backend::open(&a);
@@ -219,7 +318,7 @@ async fn main() {
} else {
for c in &list {
let s = |k: &str| c[k].as_str().unwrap_or("").to_string();
println!("{:<32} {:<34} {:<14} {}", s("id"), s("label"), s("shortcut"), s("params"));
outln!("{:<32} {:<34} {:<14} {}", s("id"), s("label"), s("shortcut"), s("params"));
}
}
}
@@ -246,9 +345,9 @@ async fn main() {
let v: Value = serde_json::from_str(line).unwrap_or_else(|e| usage(format!("line {}: {e}", n + 1)));
let id = v["id"].as_str().unwrap_or_else(|| usage(format!("line {}: missing \"id\"", n + 1)));
match b.exec(id, v.get("params").cloned().unwrap_or(json!({}))).await {
Ok(r) => println!("{}", json!({"line": n + 1, "id": id, "ok": true, "result": r})),
Ok(r) => outln!("{}", json!({"line": n + 1, "id": id, "ok": true, "result": r})),
Err(e) => {
println!("{}", json!({"line": n + 1, "id": id, "ok": false, "error": e}));
outln!("{}", json!({"line": n + 1, "id": id, "ok": false, "error": e}));
failed += 1;
if !a.flag("--keep-going") {
std::process::exit(1);
@@ -318,17 +417,16 @@ async fn main() {
if let Some(r) = a.opt("--range") {
p["range"] = json!(r);
}
if let (Some(s0), Some(s1)) = (a.opt("--start"), a.opt("--end")) {
p["range"] = json!("custom");
p["startSeconds"] = parse_value(s0);
p["endSeconds"] = parse_value(s1);
}
set_custom_bounds(&mut p, a.opt("--start"), a.opt("--end"));
if let Some(js) = a.opt("--settings") {
p["settings"] = serde_json::from_str(js).unwrap_or_else(|e| usage(format!("--settings: {e}")));
}
if a.flag("--no-audio") {
p["audio"] = json!(false);
}
if let Some(v) = a.opt("--gpu-rendering") {
p["gpuRendering"] = json!(v); // off | auto (validated by the export command)
}
let mut b = Backend::open(&a);
let t0 = std::time::Instant::now();
let r = if a.flag("--queue") {
@@ -344,7 +442,7 @@ async fn main() {
match r {
Ok(v) => {
print(&a, &v);
eprintln!("exported {out} in {:.1}s", t0.elapsed().as_secs_f64());
diag(format_args!("exported {} in {:.1}s", written_paths(&v, out).join(", "), t0.elapsed().as_secs_f64()));
}
Err(e) => fail(format!("export: {e}")),
}
@@ -359,13 +457,16 @@ async fn main() {
let out = a.opt("--out").unwrap_or("frame.png");
s.set_playhead(filmcraft_time::Tick::from_seconds_f64(secs));
let t0 = std::time::Instant::now();
let Some(img) = s.render_program(scale) else { fail("no sequence") };
let img = render_at_playhead(&s, scale).unwrap_or_else(|e| fail(e));
let dt = t0.elapsed();
let png = filmcraft_automation::png_rgba(img.w as u32, img.h as u32, img.over_black_rgba8(), 0).unwrap_or_else(|e| fail(e));
std::fs::write(out, png).unwrap_or_else(|e| fail(format!("{out}: {e}")));
eprintln!("rendered {}x{} in {:.1} ms → {out}", img.w, img.h, dt.as_secs_f64() * 1000.0);
diag(format_args!("rendered {}x{} in {:.1} ms → {out}", img.w, img.h, dt.as_secs_f64() * 1000.0));
}
"mcp" => {
if let Some(opt) = a.unknown_opt(&["--bridge", "--project", "--demo", "--data-dir"]) {
usage(format!("unknown mcp option `{opt}`"));
}
let server = match a.opt("--bridge") {
Some(addr) => filmcraft_automation::FilmcraftMcp::bridge(addr).unwrap_or_else(|e| fail(e)),
None => match Backend::open(&a) {
@@ -381,14 +482,112 @@ async fn main() {
}
}
/// The files an export wrote, from its result: `path` of a direct export, or each queued item's
/// `path` (the format may have changed the extension: `out.mp4` exported as APV is `out.mp4.mov`).
/// `requested` when the result names none.
fn written_paths(result: &Value, requested: &str) -> Vec<String> {
let paths: Vec<String> = match result.get("path").and_then(Value::as_str) {
Some(p) => vec![p.to_string()],
None => result
.get("items")
.and_then(Value::as_array)
.into_iter()
.flatten()
.filter_map(|i| i.get("path").and_then(Value::as_str))
.map(str::to_string)
.collect(),
};
if paths.is_empty() { vec![requested.to_string()] } else { paths }
}
/// The active sequence at the playhead, or the reason it cannot be rendered (no sequence, bad scale).
fn render_at_playhead(s: &Session, scale: f32) -> Result<filmcraft_engine::render::Image, String> {
s.try_render_program_at(scale, s.playhead()).map_err(|e| e.to_string())
}
/// `--start` / `--end` ask for a custom range. A lone bound is forwarded too, so the engine reports the missing one
/// instead of the export silently running over the whole sequence.
fn set_custom_bounds(p: &mut Value, start: Option<&str>, end: Option<&str>) {
if start.is_none() && end.is_none() {
return;
}
p["range"] = json!("custom");
if let Some(s0) = start {
p["startSeconds"] = parse_value(s0);
}
if let Some(s1) = end {
p["endSeconds"] = parse_value(s1);
}
}
#[cfg(test)]
mod format_tests {
#[test]
fn render_reports_the_real_reason() {
let mut s = filmcraft_engine::Session::default();
assert!(super::render_at_playhead(&s, 0.5).unwrap_err().contains("sequence"));
s.execute("file.newSequence", serde_json::json!({"width": 16, "height": 16})).unwrap();
for scale in [0.0, -1.0, f32::NAN] {
let e = super::render_at_playhead(&s, scale).unwrap_err();
assert!(e.contains("finite and positive"), "{e}");
}
assert_eq!(super::render_at_playhead(&s, 0.5).unwrap().w, 8);
}
/// The CLI (and MCP / headless runs, which share its entry point) registers the hardware
/// decoders at start-up.
#[test]
fn startup_registers_the_hardware_decoders() {
let hardware = super::register_hardware_decoders();
assert_eq!(filmcraft_platform::registered(), cfg!(any(target_os = "macos", target_os = "windows")), "{hardware:?}");
// always on macOS and Windows; on Linux when a VA-API driver or NVIDIA's driver (NVDEC) is there
let expected = cfg!(any(target_os = "macos", target_os = "windows"))
|| (cfg!(target_os = "linux") && matches!(hardware, filmcraft_platform::Availability::Available(_)));
assert_eq!(filmcraft_platform::registered(), expected, "{hardware:?}");
}
#[test]
fn a_failing_diagnostic_stream_does_not_panic() {
struct Closed;
impl std::io::Write for Closed {
fn write(&mut self, _: &[u8]) -> std::io::Result<usize> {
Err(std::io::ErrorKind::BrokenPipe.into())
}
fn flush(&mut self) -> std::io::Result<()> {
Err(std::io::ErrorKind::BrokenPipe.into())
}
}
super::write_diag(&mut Closed, format_args!("exported {}", "a.wav"));
let mut ok = Vec::new();
super::write_diag(&mut ok, format_args!("hi"));
assert_eq!(ok, b"hi\n");
}
#[test]
fn export_message_names_the_written_files() {
use serde_json::json;
let direct = json!({"job": 1, "path": "out-apv.mp4.mov", "result": {"path": "out-apv.mp4.mov"}});
assert_eq!(super::written_paths(&direct, "out-apv.mp4"), ["out-apv.mp4.mov"]);
let queued = json!({"added": [1, 2], "items": [{"path": "a.mp4.mov"}, {"path": "b.wav"}]});
assert_eq!(super::written_paths(&queued, "a.mp4"), ["a.mp4.mov", "b.wav"]);
assert_eq!(super::written_paths(&json!({}), "x.mov"), ["x.mov"]);
assert_eq!(super::written_paths(&json!({"items": []}), "x.mov"), ["x.mov"]);
}
#[test]
fn export_lone_bound_is_forwarded_as_custom_range() {
use serde_json::json;
let mut p = json!({});
super::set_custom_bounds(&mut p, Some("0.25"), None);
assert_eq!(p["range"], "custom");
assert_eq!(p["startSeconds"], 0.25);
assert!(p.get("endSeconds").is_none());
let mut p = json!({});
super::set_custom_bounds(&mut p, None, Some("0.25"));
assert_eq!(p["range"], "custom");
assert_eq!(p["endSeconds"], 0.25);
let mut p = json!({});
super::set_custom_bounds(&mut p, None, None);
assert!(p.get("range").is_none());
}
#[test]
+107 -1
View File
@@ -24,6 +24,32 @@ fn help_and_usage_errors() {
assert_eq!(cli(&["exec", "file.newBin", "notkv"]).status.code(), Some(2));
}
#[test]
fn version_after_global_options() {
let want = format!("filmcraft-cli {}\n", env!("CARGO_PKG_VERSION"));
for args in [&["--version"][..], &["-V"], &["version"], &["--demo", "--version"], &["--compact", "-V"], &["--demo", "version"]] {
let o = cli(args);
assert!(o.status.success(), "{args:?}: {}", String::from_utf8_lossy(&o.stderr));
assert_eq!(String::from_utf8_lossy(&o.stdout), want, "{args:?}");
}
}
#[test]
fn long_help_matches_help_command() {
let reference = cli(&["help"]);
assert!(reference.status.success());
let long_help = cli(&["--help"]);
assert!(long_help.status.success(), "--help failed: {}", String::from_utf8_lossy(&long_help.stderr));
assert_eq!(long_help.stdout, reference.stdout);
assert!(long_help.stderr.is_empty());
let short_help = cli(&["-h"]);
assert!(short_help.status.success());
assert_eq!(short_help.stdout, reference.stdout);
let sub_help = cli(&["export", "--help"]);
assert!(sub_help.status.success(), "export --help failed: {}", String::from_utf8_lossy(&sub_help.stderr));
assert_eq!(sub_help.stdout, reference.stdout);
}
#[test]
fn probe_reports_mpeg_transport_and_program_streams() {
let Some(ffmpeg) = ["/opt/homebrew/bin/ffmpeg", "/usr/local/bin/ffmpeg", "/usr/bin/ffmpeg"].into_iter().find(|p| std::path::Path::new(p).exists()) else {
@@ -44,7 +70,7 @@ fn probe_reports_mpeg_transport_and_program_streams() {
assert!(st.success());
let v = json_out(&cli(&["probe", out.to_str().unwrap()]));
assert!(v["video"]["codec"].as_str().unwrap().starts_with("MPEG-2 Video"), "{v}");
assert_eq!(v["audio"]["codec"], "MPEG Audio");
assert_eq!(v["audio_streams"][0]["codec"], "MPEG Audio");
let streams = v["mpeg"]["streams"].as_array().unwrap();
assert_eq!(streams.len(), 2, "{v}");
assert_eq!(streams[0]["codec"], "MPEG-2 Video");
@@ -179,9 +205,89 @@ fn export_with_presets() {
let v = json_out(&cli(&["--demo", "--data-dir", data_s, "export", q.to_str().unwrap(), "--preset", "Tiny WAV", "--queue", "--range", "entire"]));
assert_eq!(v["items"][0]["status"], "done", "{v}");
assert!(q.exists());
// a format that changes the extension: the completion message names the file written (#341)
let apv = dir.join("apv.mp4");
let o =
cli(&["--demo", "--data-dir", data_s, "export", apv.to_str().unwrap(), "--preset", "APV 422-10", "--start", "0", "--end", "0.1", "--scale", "0.125"]);
let v = json_out(&o);
let written = v["path"].as_str().unwrap().to_string();
assert!(written.ends_with("apv.mp4.mov") && std::path::Path::new(&written).exists(), "{v}");
let msg = String::from_utf8_lossy(&o.stderr);
assert!(msg.contains(&format!("exported {written} in ")), "{msg}");
let qa = dir.join("queued-apv.mp4");
let o = cli(&[
"--demo",
"--data-dir",
data_s,
"export",
qa.to_str().unwrap(),
"--preset",
"APV 422-10",
"--queue",
"--start",
"0",
"--end",
"0.1",
"--scale",
"0.125",
]);
let v = json_out(&o);
let written = v["items"][0]["path"].as_str().unwrap().to_string();
assert!(written.ends_with("queued-apv.mp4.mov"), "{v}");
assert!(String::from_utf8_lossy(&o.stderr).contains(&format!("exported {written} in ")), "{}", String::from_utf8_lossy(&o.stderr));
// unknown preset: exit status 1 and a message naming it
let bad = cli(&["--demo", "--data-dir", data_s, "export", dir.join("x.mp4").to_str().unwrap(), "--preset", "No Such Preset"]);
assert_eq!(bad.status.code(), Some(1));
assert!(String::from_utf8_lossy(&bad.stderr).contains("No Such Preset"));
let _ = std::fs::remove_dir_all(&dir);
}
/// `filmcraft-cli commands | head -c 10` panicked with "failed printing to stdout: Broken pipe
/// (os error 32)" and exit status 101 (#285).
#[test]
fn closed_stdout_ends_quietly() {
use std::io::Read;
for args in [&["commands"][..], &["help"], &["--demo", "commands", "--json"]] {
let mut child = Command::new(env!("CARGO_BIN_EXE_filmcraft-cli")).args(args).stdout(Stdio::piped()).stderr(Stdio::piped()).spawn().unwrap();
let mut first = [0u8; 10];
child.stdout.as_mut().unwrap().read_exact(&mut first).unwrap();
drop(child.stdout.take());
let out = child.wait_with_output().unwrap();
let err = String::from_utf8_lossy(&out.stderr);
assert!(out.status.success() && err.is_empty(), "{args:?}: {:?}: {err}", out.status);
}
}
/// `run -` with stdout already closed: `run` reads its script to the end before it prints, so its
/// first write fails for sure. The rest of the script and the save still happen, and the exit status
/// still reports a failing command.
fn run_with_closed_stdout(args: &[&str], script: &[u8]) -> Output {
let mut child =
Command::new(env!("CARGO_BIN_EXE_filmcraft-cli")).args(args).stdin(Stdio::piped()).stdout(Stdio::piped()).stderr(Stdio::piped()).spawn().unwrap();
drop(child.stdout.take());
child.stdin.take().unwrap().write_all(script).unwrap();
child.wait_with_output().unwrap()
}
#[test]
fn closed_stdout_still_runs_the_script_and_saves() {
let dir = std::env::temp_dir().join(format!("fc-cli-pipe-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let proj = dir.join("p.fcproj");
let p = proj.to_str().unwrap();
let script = b"{\"id\":\"file.newBin\",\"params\":{\"name\":\"Piped One\"}}\n{\"id\":\"file.newBin\",\"params\":{\"name\":\"Piped Two\"}}\n";
let out = run_with_closed_stdout(&["--demo", "--save-as", p, "run", "-"], script);
assert!(out.status.success(), "{:?}: {}", out.status, String::from_utf8_lossy(&out.stderr));
let tree = json_out(&cli(&["--project", p, "inspect", "project"])).to_string();
assert!(tree.contains("Piped One") && tree.contains("Piped Two"), "{tree}");
// a failing command is still a failure, whether or not anyone reads stdout
let out = run_with_closed_stdout(&["--demo", "run", "-"], b"{\"id\":\"no.such.command\"}\n");
assert_eq!(out.status.code(), Some(1));
let out = run_with_closed_stdout(
&["--demo", "--keep-going", "run", "-"],
b"{\"id\":\"no.such.command\"}\n{\"id\":\"file.newBin\",\"params\":{\"name\":\"x\"}}\n",
);
assert_eq!(out.status.code(), Some(1));
let _ = std::fs::remove_dir_all(&dir);
}
+24
View File
@@ -0,0 +1,24 @@
//! Spacing of browser downloads (pure, so it is tested natively in `tests/download_pacing.rs`; the
//! anchor clicks live in [`crate::fs::download`]).
//!
//! Chrome drops anchor-click downloads that start in one tight burst, so an image-sequence export
//! (one file per frame) delivered only about half of its frames (#378). Each download is given a
//! slot [`GAP_MS`] after the previous one.
/// Milliseconds between two download clicks.
pub const GAP_MS: f64 = 125.0;
/// Hands out click times.
#[derive(Clone, Copy, Debug, Default, PartialEq)]
pub struct Pacer {
next_free: f64,
}
impl Pacer {
/// Reserve the next slot at time `now` (ms); returns how long to wait before clicking.
pub fn reserve(&mut self, now: f64) -> f64 {
let start = now.max(self.next_free);
self.next_free = start + GAP_MS;
start - now
}
}
+21 -17
View File
@@ -54,6 +54,7 @@ struct Fs {
thread_local! {
static FS: RefCell<Fs> = RefCell::new(Fs::default());
static DOWNLOADS: RefCell<crate::download_pacing::Pacer> = RefCell::new(crate::download_pacing::Pacer::default());
}
/// Wake-up hook for arriving data.
@@ -70,9 +71,8 @@ fn file_name(path: &str) -> &str {
path.rsplit('/').next().unwrap_or(path)
}
/// Register a browser `File` (or `Blob` with a name) and return its virtual path. A file with
/// the same name and size as an existing entry replaces it (the same file picked again keeps
/// its path, so projects referencing it come back online); otherwise a numbered name is used.
/// Register a browser `File` (or named `Blob`) at an unused virtual path; collisions get numbered names.
/// Name + size do not identify content: replacing an occupied path changes every project item referencing it.
pub fn register_blob(name: &str, blob: web_sys::Blob) -> String {
let size = blob.size() as u64;
let clean: String = name.chars().map(|c| if c == '/' || c == '\\' { '_' } else { c }).collect();
@@ -85,18 +85,11 @@ pub fn register_blob(name: &str, blob: web_sys::Blob) -> String {
let mut n = 1;
let path = loop {
let p = if n == 1 { format!("/files/{clean}") } else { format!("/files/{stem} ({n}){ext}") };
match fs.entries.get(&p) {
None => break p,
Some(Entry::Blob { size: s, .. }) if *s == size => break p,
Some(Entry::Mem(b)) if b.len() as u64 == size => break p,
_ => n += 1,
if !fs.entries.contains_key(&p) {
break p;
}
n += 1;
};
if let Some(Entry::Blob { id, .. }) = fs.entries.get(&path) {
let id = *id;
fs.blobs.remove(&id);
fs.chunks.retain(|k, _| k.0 != id);
}
let id = fs.next_id;
fs.next_id += 1;
fs.blobs.insert(id, blob);
@@ -413,7 +406,8 @@ impl Services for WebServices {
}
}
/// Offer bytes as a browser download named `name`.
/// Offer bytes as a browser download named `name`. Downloads are clicked at least
/// [`crate::download_pacing::GAP_MS`] apart: Chrome drops a burst of them (#378).
pub fn download(name: &str, data: &[u8]) -> Result<(), wasm_bindgen::JsValue> {
let doc = web_sys::window().and_then(|w| w.document()).ok_or("no document")?;
let blob = bytes_to_blob(data, "application/octet-stream")?;
@@ -422,10 +416,20 @@ pub fn download(name: &str, data: &[u8]) -> Result<(), wasm_bindgen::JsValue> {
a.set_href(&url);
a.set_download(file_name(name));
a.style().set_property("display", "none")?;
doc.body().ok_or("no body")?.append_child(&a)?;
a.click();
a.remove();
let delay = DOWNLOADS.with(|d| d.borrow_mut().reserve(js_sys::Date::now()));
let name = name.to_owned();
wasm_bindgen_futures::spawn_local(async move {
if delay > 0.0 {
sleep(delay as i32).await;
}
let clicked = match doc.body() {
Some(body) => body.append_child(&a).map(|_| a.click()),
None => Err("no body".into()),
};
a.remove();
if let Err(e) = clicked {
log::warn!("download {name}: {e:?}");
}
sleep(60_000).await;
let _ = web_sys::Url::revoke_object_url(&url);
});
+47 -6
View File
@@ -10,7 +10,7 @@ use std::cell::RefCell;
use std::rc::Rc;
use filmcraft_engine::Services;
use filmcraft_ui_egui::FilmcraftApp;
use filmcraft_ui_egui::{FilmcraftApp, RelinkHint};
use serde_json::{Value, json};
use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
@@ -81,12 +81,12 @@ async fn prewarm(path: &str, size: u64) {
}
/// `file.import` one path, retried while its bytes are loading.
async fn import_path(path: String) -> Result<Value, String> {
async fn import_path(path: String, bin: u64) -> Result<Value, String> {
for _ in 0..2000 {
let p = path.clone();
let (r, pending) = on_ui(move |w, _| {
let _ = filmcraft_media::pending::take();
let r = w.app.session.execute("file.import", json!({"paths": [p]})).map_err(|e| e.to_string());
let r = w.app.session.execute("file.import", json!({"paths": [p], "bin": bin})).map_err(|e| e.to_string());
(r, filmcraft_media::pending::take())
})
.await;
@@ -105,6 +105,8 @@ async fn import_path(path: String) -> Result<Value, String> {
/// Import browser files (no copies): returns `{items, errors, paths}`.
pub async fn import_files(files: Vec<web_sys::File>) -> Value {
// Capture once per batch: navigation while prewarming must not redirect pending imports.
let bin = on_ui(|w, _| w.app.import_bin().0).await;
let mut items = Vec::new();
let mut errors = Vec::new();
let mut paths = Vec::new();
@@ -113,7 +115,7 @@ pub async fn import_files(files: Vec<web_sys::File>) -> Value {
let size = f.size() as u64;
let path = fs::register_blob(&name, f.clone().into());
prewarm(&path, size).await;
match import_path(path.clone()).await {
match import_path(path.clone(), bin).await {
Ok(v) => {
items.extend(v.get("items").and_then(Value::as_array).cloned().unwrap_or_default());
errors.extend(v.get("errors").and_then(Value::as_array).cloned().unwrap_or_default());
@@ -153,12 +155,43 @@ pub fn take_files(files: Vec<web_sys::File>) {
});
}
/// Files a picker returned: imported, or with a relink hint, the first one is registered under
/// `/files/<name>` and the hint's command runs on it (#111).
fn picked(files: Vec<web_sys::File>, relink: Option<RelinkHint>) {
let Some(hint) = relink else { return take_files(files) };
let Some(f) = files.into_iter().next() else { return };
wasm_bindgen_futures::spawn_local(async move {
let path = fs::register_blob(&f.name(), f.clone().into());
crate::recovery::keep_media(&path, &f);
let mut params = hint.params;
if !params.is_object() {
params = json!({});
}
params["path"] = json!(path);
let command = hint.command;
let outcome = on_ui(move |w, _| w.app.session.execute(&command, params).map_err(|e| e.to_string())).await;
crate::post(move |w, _| match outcome {
Ok(v) => {
if let Some(n) = v.get("relinked").and_then(Value::as_array).map(Vec::len) {
w.app.status(format!("Linked {n} clip(s)"));
}
}
Err(e) => w.app.status(e),
});
});
}
fn has_fsa_picker() -> bool {
web_sys::window().is_some_and(|w| js_sys::Reflect::has(&w, &"showOpenFilePicker".into()).unwrap_or(false))
}
/// Let the user pick files: `showOpenFilePicker` when available, else a file input.
pub fn pick(exts: &[&str], multiple: bool) {
pick_then(exts, multiple, None);
}
/// [`pick`], handing the files to [`picked`]: a cancelled picker leaves nothing pending.
fn pick_then(exts: &[&str], multiple: bool, relink: Option<RelinkHint>) {
let accept: Vec<String> = exts.iter().map(|e| format!(".{e}")).collect();
if has_fsa_picker() {
let opts = js_sys::Object::new();
@@ -178,7 +211,7 @@ pub fn pick(exts: &[&str], multiple: bool) {
files.push(f);
}
}
take_files(files);
picked(files, relink);
});
return;
}
@@ -190,7 +223,7 @@ pub fn pick(exts: &[&str], multiple: bool) {
let inp = input.clone();
let on_change = Closure::<dyn FnMut()>::new(move || {
if let Some(list) = inp.files() {
take_files((0..list.length()).filter_map(|i| list.get(i)).collect());
picked((0..list.length()).filter_map(|i| list.get(i)).collect(), relink.clone());
}
});
input.set_onchange(Some(on_change.as_ref().unchecked_ref()));
@@ -231,4 +264,12 @@ pub fn install_hooks(app: &mut FilmcraftApp) {
// Saving writes the file in memory and offers it as a download (`fs::WebServices`).
app.hooks.pick_save = Some(Box::new(|name: &str| Some(format!("/projects/{name}"))));
app.hooks.pick_save_as = Some(Box::new(|_filter: &str, _exts: &[&str], name: &str| Some(format!("/exports/{name}"))));
// Link Media ▸ Locate…, Attach Proxies, Reconnect Full Resolution: the picker is async, so the
// hint's command runs once the user has chosen (#111).
app.hooks.pick_file_for_relink = Some(Box::new(|exts: &[&str], hint: Option<RelinkHint>| {
if hint.is_some() {
pick_then(exts, false, hint);
}
None
}));
}
+13 -1
View File
@@ -20,10 +20,12 @@
pub mod api;
pub mod audio;
pub mod download_pacing;
pub mod fs;
pub mod import;
pub mod opfs;
pub mod recovery;
pub mod recovery_policy;
pub mod webcodecs;
use std::cell::RefCell;
@@ -58,6 +60,13 @@ pub fn repaint() {
});
}
/// Give a host that builds its own [`WebApp`] the egui context that [`post`] and [`repaint`] wake.
/// [`start`] calls it; an embedding app that runs `WebApp` in its own eframe runner calls it once
/// from its app creator, or posted actions and JS API requests wait for the next input event.
pub fn set_context(ctx: egui::Context) {
CTX.with(|c| *c.borrow_mut() = Some(ctx));
}
/// Environment facts reported by `filmcraft.info()` (backend, isolation, codecs…).
pub fn set_info(key: &str, v: serde_json::Value) {
INFO.with(|i| {
@@ -150,7 +159,7 @@ pub async fn start(canvas_id: String) -> Result<(), JsValue> {
canvas,
web_options,
Box::new(move |cc| {
CTX.with(|c| *c.borrow_mut() = Some(cc.egui_ctx.clone()));
set_context(cc.egui_ctx.clone());
let ctx = cc.egui_ctx.clone();
fs::set_on_data(move || ctx.request_repaint());
fs::set_on_write(|path, data| {
@@ -177,6 +186,9 @@ pub async fn start(canvas_id: String) -> Result<(), JsValue> {
}
let mut app = FilmcraftApp::new(session);
import::install_hooks(&mut app);
// Settings ▸ General ▸ Interface Language ▸ System Language (#218): the browser's languages.
app.hooks.system_languages =
Some(Box::new(|| web_sys::window().map(|w| w.navigator().languages().iter().filter_map(|v| v.as_string()).collect()).unwrap_or_default()));
let (tx, rx) = std::sync::mpsc::channel();
api::set_sender(tx);
app = app.with_control(rx);
+21 -4
View File
@@ -10,8 +10,13 @@ use wasm_bindgen::{JsCast, JsValue};
use wasm_bindgen_futures::JsFuture;
use web_sys::{FileSystemDirectoryHandle, FileSystemFileHandle};
/// Called with whether a queued write reached OPFS.
pub type Done = Box<dyn FnOnce(bool)>;
thread_local! {
static QUEUE: RefCell<(BTreeMap<String, Vec<u8>>, bool)> = const { RefCell::new((BTreeMap::new(), false)) };
/// Pending writes per path (bytes and the callbacks of every write coalesced into them), and
/// whether the writer task is running.
static QUEUE: RefCell<(BTreeMap<String, (Vec<u8>, Vec<Done>)>, bool)> = const { RefCell::new((BTreeMap::new(), false)) };
}
/// Whether this browser has OPFS (`navigator.storage.getDirectory`).
@@ -60,9 +65,17 @@ async fn write_now(rel: &str, data: &[u8]) -> Result<(), JsValue> {
/// Queue a write of `data` to `rel` (relative to the OPFS root).
pub fn write(rel: String, data: Vec<u8>) {
write_then(rel, data, None);
}
/// [`write`], then call `done` with whether it succeeded. A write replaced by a newer one to the
/// same path reports the newer one's result.
pub fn write_then(rel: String, data: Vec<u8>, done: Option<Done>) {
let start = QUEUE.with(|q| {
let mut q = q.borrow_mut();
q.0.insert(rel, data);
let e = q.0.entry(rel).or_default();
e.0 = data;
e.1.extend(done);
!std::mem::replace(&mut q.1, true)
});
if !start {
@@ -81,10 +94,14 @@ pub fn write(rel: String, data: Vec<u8>) {
}
}
});
let Some((rel, data)) = next else { break };
if let Err(e) = write_now(&rel, &data).await {
let Some((rel, (data, done))) = next else { break };
let r = write_now(&rel, &data).await;
if let Err(e) = &r {
log::warn!("OPFS write {rel}: {e:?}");
}
for d in done {
d(r.is_ok());
}
}
});
}
+48 -18
View File
@@ -6,24 +6,32 @@
//! - Imported media are copied to `media/<name>` (the browser streams the copy; nothing goes
//! through wasm memory), so the snapshot's `/files/<name>` references resolve after a reload.
//! - At start-up a dirty snapshot is reopened (`?norecover` skips it, `?fresh` also skips the media).
//! A skipped snapshot stays dirty until this session writes its own ([`crate::recovery_policy`]).
use std::cell::{Cell, RefCell};
use std::rc::Rc;
use filmcraft_engine::Session;
use serde_json::{Value, json};
use crate::opfs;
use crate::recovery_policy::{Action, Policy};
const SNAPSHOT: &str = "recovery/snapshot.fcproj";
const META: &str = "recovery/meta.json";
/// Seconds between snapshots of a changing project.
const INTERVAL_S: f64 = 5.0;
/// Media larger than this are not copied to OPFS.
const KEEP_MAX: f64 = 4.0 * 1024.0 * 1024.0 * 1024.0;
/// A finished OPFS write, reported back to the next [`Autosave::tick`].
enum Outcome {
Snapshot { revision: u64, ok: bool },
Clean { ok: bool },
}
#[derive(Default)]
pub struct Autosave {
last_revision: u64,
last_time: f64,
last_dirty: Option<bool>,
policy: Policy,
outcomes: Rc<RefCell<Vec<Outcome>>>,
}
impl Autosave {
@@ -33,22 +41,44 @@ impl Autosave {
return;
}
let now = ctx.input(|i| i.time);
let dirty = s.revision != s.saved_revision;
if dirty {
if s.revision != self.last_revision && now - self.last_time >= INTERVAL_S {
opfs::write(SNAPSHOT.into(), filmcraft_format::encode(&s.project, false));
for o in self.outcomes.borrow_mut().drain(..) {
match o {
Outcome::Snapshot { revision, ok } => self.policy.written(revision, ok, now),
Outcome::Clean { ok } => self.policy.cleaned(ok, now),
}
}
match self.policy.tick(now, s.revision, s.saved_revision) {
Action::Nothing => {}
Action::WakeAfter(secs) => ctx.request_repaint_after(std::time::Duration::from_secs_f64(secs)),
Action::WriteSnapshot(revision) => {
// Snapshot and meta both have to land; either failing retries the pair.
let failed = Rc::new(Cell::new(false));
let left = Rc::new(Cell::new(2u8));
let done = |out: Rc<RefCell<Vec<Outcome>>>, failed: Rc<Cell<bool>>, left: Rc<Cell<u8>>, ctx: egui::Context| -> opfs::Done {
Box::new(move |ok| {
failed.set(failed.get() || !ok);
left.set(left.get().saturating_sub(1));
if left.get() == 0 {
out.borrow_mut().push(Outcome::Snapshot { revision, ok: !failed.get() });
ctx.request_repaint();
}
})
};
let meta = json!({"dirty": true, "name": s.project.name, "revision": s.revision, "time": js_sys::Date::now()});
opfs::write(META.into(), meta.to_string().into_bytes());
self.last_revision = s.revision;
self.last_time = now;
} else if s.revision != self.last_revision {
// the UI may go idle: wake up when the snapshot is due
ctx.request_repaint_after(std::time::Duration::from_secs_f64((INTERVAL_S - (now - self.last_time)).max(0.05)));
let d1 = done(self.outcomes.clone(), failed.clone(), left.clone(), ctx.clone());
let d2 = done(self.outcomes.clone(), failed, left, ctx.clone());
opfs::write_then(SNAPSHOT.into(), filmcraft_format::encode(&s.project, false), Some(d1));
opfs::write_then(META.into(), meta.to_string().into_bytes(), Some(d2));
}
Action::WriteClean => {
let (out, ctx) = (self.outcomes.clone(), ctx.clone());
let done: opfs::Done = Box::new(move |ok| {
out.borrow_mut().push(Outcome::Clean { ok });
ctx.request_repaint();
});
opfs::write_then(META.into(), json!({"dirty": false}).to_string().into_bytes(), Some(done));
}
} else if self.last_dirty != Some(false) {
opfs::write(META.into(), json!({"dirty": false}).to_string().into_bytes());
}
self.last_dirty = Some(dirty);
}
}
+88
View File
@@ -0,0 +1,88 @@
//! When the web recovery snapshot is written or marked clean (pure, so it is tested natively in
//! `tests/recovery_policy.rs`; the OPFS side lives in [`crate::recovery`]).
//!
//! - A dirty project is snapshotted at most every [`INTERVAL_S`] seconds, once per revision. A write
//! that fails is retried after the interval, without waiting for another edit (#197).
//! - The meta is marked clean only to retire a snapshot this session wrote. A snapshot left by an
//! earlier session that was not reopened (`?norecover`, `?fresh`) stays dirty (#193).
/// Seconds between snapshots of a changing project.
pub const INTERVAL_S: f64 = 5.0;
/// What the caller should do this frame.
#[derive(Clone, Copy, Debug, PartialEq)]
pub enum Action {
Nothing,
/// Write the snapshot and a dirty meta for this revision, then report with [`Policy::written`].
WriteSnapshot(u64),
/// Write a clean meta, then report with [`Policy::cleaned`].
WriteClean,
/// Nothing now; wake up after this many seconds.
WakeAfter(f64),
}
#[derive(Debug, Default)]
pub struct Policy {
/// Revision whose snapshot was last written or is being written.
last_revision: Option<u64>,
last_time: Option<f64>,
/// This session wrote a dirty meta that is not yet retired.
owns_dirty_meta: bool,
/// A clean-meta write is in flight.
cleaning: bool,
/// Snapshot writes started so far, and how many had started when the clean write did (a
/// snapshot started after it makes the meta dirty again).
snapshots: u64,
snapshots_at_clean: u64,
}
impl Policy {
/// Called every frame with the session's revisions and the time in seconds.
pub fn tick(&mut self, now: f64, revision: u64, saved_revision: u64) -> Action {
if revision != saved_revision {
if self.last_revision == Some(revision) {
return Action::Nothing;
}
let since = self.last_time.map_or(f64::INFINITY, |t| now - t);
if since >= INTERVAL_S {
self.last_revision = Some(revision);
self.last_time = Some(now);
self.owns_dirty_meta = true;
self.snapshots += 1;
return Action::WriteSnapshot(revision);
}
return Action::WakeAfter((INTERVAL_S - since).max(0.05));
}
if self.owns_dirty_meta && !self.cleaning {
let since = self.last_time.map_or(f64::INFINITY, |t| now - t);
if self.last_revision.is_some() || since >= INTERVAL_S {
self.cleaning = true;
self.snapshots_at_clean = self.snapshots;
return Action::WriteClean;
}
return Action::WakeAfter((INTERVAL_S - since).max(0.05));
}
Action::Nothing
}
/// The snapshot write for `revision` finished.
pub fn written(&mut self, revision: u64, ok: bool, now: f64) {
if !ok && self.last_revision == Some(revision) {
// retry after the interval, even if nothing else changes
self.last_revision = None;
self.last_time = Some(now);
}
}
/// The clean-meta write finished.
pub fn cleaned(&mut self, ok: bool, now: f64) {
self.cleaning = false;
if ok {
self.owns_dirty_meta = self.snapshots != self.snapshots_at_clean;
} else {
// retry after the interval
self.last_revision = None;
self.last_time = Some(now);
}
}
}
+109 -20
View File
@@ -12,8 +12,7 @@
//! the job on a later frame, when the decoder's output callback has delivered the picture.
//! - A session that has not delivered the frame it was started for is not restarted by other
//! requests (thumbnails and the monitor would otherwise keep resetting each other).
//! - Outputs are converted to RGBA by drawing the `VideoFrame` on an `OffscreenCanvas` (the
//! browser applies the stream's colour description).
//! - Native YUV planes feed the desktop frame/compositor contract. Unsupported browser formats or geometry use Canvas RGBA.
//! - Any decoder error or unsupported configuration switches the source to our decoder.
//!
//! `?nowebcodecs` disables it. Support is probed at start-up with `VideoDecoder.isConfigSupported`.
@@ -34,6 +33,20 @@ use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
use wasm_bindgen_futures::JsFuture;
mod pixels;
#[derive(Default)]
struct Transfers {
native: u64,
canvas: u64,
closed: u64,
stale: u64,
bytes: u64,
native_ms: f64,
canvas_ms: f64,
reasons: BTreeMap<&'static str, u64>,
}
/// Samples fed past the wanted one (decoders hold pictures for reordering).
const LOOKAHEAD: usize = 8;
/// Decoded frames kept per source.
@@ -52,6 +65,7 @@ thread_local! {
static DECODED: Cell<u64> = const { Cell::new(0) };
static FALLBACKS: Cell<u64> = const { Cell::new(0) };
static SOURCES: Cell<u64> = const { Cell::new(0) };
static TRANSFERS: RefCell<Transfers> = RefCell::new(Transfers::default());
}
fn now_ms() -> f64 {
@@ -160,11 +174,15 @@ pub fn stats() -> serde_json::Value {
.iter()
.map(|(id, s)| {
json!({"id": id, "start": s.start, "next": s.next, "target": s.target, "needsKey": s.needs_key, "flushing": s.flushing.get(),
"errored": s.errored.get(), "queue": s.queue(), "idleMs": now_ms() - s.last_output.get()})
"errored": s.errored.get(), "queue": s.queue(), "copies": s.copying.get(), "idleMs": now_ms() - s.last_output.get()})
})
.collect()
});
json!({"sources": SOURCES.with(Cell::get), "framesDecoded": DECODED.with(Cell::get), "fallbacks": FALLBACKS.with(Cell::get), "sessions": sessions})
let transfers = TRANSFERS.with(|s| {
let s = s.borrow();
json!({"nativeFrames":s.native,"canvasFrames":s.canvas,"closedFrames":s.closed,"staleFrames":s.stale,"nativeBytes":s.bytes,"nativeMs":s.native_ms,"canvasMs":s.canvas_ms,"canvasReasons":s.reasons})
});
json!({"sources": SOURCES.with(Cell::get), "framesDecoded": DECODED.with(Cell::get), "fallbacks": FALLBACKS.with(Cell::get), "sessions": sessions,"transfers":transfers})
}
/// A [`SharedReader`] as the demuxer's byte source.
@@ -210,6 +228,8 @@ pub fn reader_opener(name: &str, head: &[u8], reader: &SharedReader) -> Option<R
});
SOURCES.with(|c| c.set(c.get() + 1));
log::info!("{name}: video decoded with WebCodecs ({codec})");
let color = info.video.as_ref().map(|v| v.color).unwrap_or_default();
let par = info.video.as_ref().map(|v| v.par).unwrap_or((1, 1));
Some(Ok(Arc::new(WcSource {
inner,
info,
@@ -217,11 +237,12 @@ pub fn reader_opener(name: &str, head: &[u8], reader: &SharedReader) -> Option<R
file,
track,
id,
config: Config { codec, description, width: w, height: h, rotation },
config: Config { codec, description, width: w, height: h, rotation, color, par },
state: Mutex::new(Cache::default()),
})))
}
#[derive(Clone)]
struct Config {
codec: String,
description: Option<Vec<u8>>,
@@ -229,6 +250,8 @@ struct Config {
height: u32,
/// Clockwise quarter turns from the track matrix, applied to decoded frames.
rotation: u8,
color: filmcraft_color::ColorInfo,
par: (u32, u32),
}
#[derive(Default)]
@@ -253,6 +276,9 @@ struct Session {
/// Highest presentation time output since the session (re)started.
out_max: Rc<Cell<i64>>,
out: Rc<RefCell<Vec<(i64, VideoFrame)>>>,
copying: Rc<Cell<u32>>,
generation: Rc<Cell<u64>>,
alive: Rc<Cell<bool>>,
_closures: (Closure<dyn FnMut(JsValue)>, Closure<dyn FnMut(JsValue)>),
}
@@ -286,17 +312,65 @@ impl Session {
let errored = Rc::new(Cell::new(false));
let last_output = Rc::new(Cell::new(now_ms()));
let out_max = Rc::new(Cell::new(i64::MIN));
let copying = Rc::new(Cell::new(0u32));
let generation = Rc::new(Cell::new(0u64));
let alive = Rc::new(Cell::new(true));
let (o, lo, om) = (out.clone(), last_output.clone(), out_max.clone());
let (copies, epoch, active, config, errors) = (copying.clone(), generation.clone(), alive.clone(), cfg.clone(), errored.clone());
let on_output = Closure::<dyn FnMut(JsValue)>::new(move |frame: JsValue| {
let ts = get(&frame, "timestamp").as_f64().unwrap_or(0.0) as i64;
om.set(om.get().max(ts));
if let Some(f) = to_rgba(&frame) {
o.borrow_mut().push((ts, f));
DECODED.with(|c| c.set(c.get() + 1));
}
let _ = call(&frame, "close", &[]);
lo.set(now_ms());
crate::repaint();
let expected = epoch.get();
let (o, lo, om, copies, epoch, active, config, errors) =
(o.clone(), lo.clone(), om.clone(), copies.clone(), epoch.clone(), active.clone(), config.clone(), errors.clone());
copies.set(copies.get().saturating_add(1));
wasm_bindgen_futures::spawn_local(async move {
let begin = now_ms();
let converted = match pixels::read(&frame, &config).await {
Ok((f, bytes)) => {
TRANSFERS.with(|s| {
let mut s = s.borrow_mut();
s.native = s.native.saturating_add(1);
s.bytes = s.bytes.saturating_add(bytes as u64);
s.native_ms += now_ms() - begin;
});
Some(f)
}
Err(reason) => {
let begin = now_ms();
let f = to_rgba(&frame);
TRANSFERS.with(|s| {
let mut s = s.borrow_mut();
s.canvas = s.canvas.saturating_add(1);
s.canvas_ms += now_ms() - begin;
let count = s.reasons.entry(reason).or_default();
*count = count.saturating_add(1);
});
f
}
};
let _ = call(&frame, "close", &[]);
copies.set(copies.get().saturating_sub(1));
TRANSFERS.with(|s| {
let mut s = s.borrow_mut();
s.closed = s.closed.saturating_add(1);
});
if !active.get() || epoch.get() != expected {
TRANSFERS.with(|s| {
let mut s = s.borrow_mut();
s.stale = s.stale.saturating_add(1);
});
return;
}
if let Some(f) = converted {
o.borrow_mut().push((ts, f));
om.set(om.get().max(ts));
DECODED.with(|c| c.set(c.get().saturating_add(1)));
} else {
errors.set(true);
}
lo.set(now_ms());
crate::repaint();
});
});
let e = errored.clone();
let on_error = Closure::<dyn FnMut(JsValue)>::new(move |err: JsValue| {
@@ -317,6 +391,9 @@ impl Session {
last_output,
out_max,
out,
copying,
generation,
alive,
_closures: (on_output, on_error),
};
s.configure(cfg)?;
@@ -341,6 +418,14 @@ impl Session {
}
}
impl Drop for Session {
fn drop(&mut self) {
self.alive.set(false);
self.generation.set(self.generation.get().wrapping_add(1));
let _ = call(&self.decoder, "close", &[]);
}
}
/// An MP4/MOV source whose video is decoded by the browser.
pub struct WcSource {
inner: SharedSource,
@@ -388,12 +473,15 @@ impl WcSource {
// already output but the cache dropped needs a restart).
let covers = !s.needs_key && s.start <= i && key <= s.next && (i >= s.next || want > s.out_max.get());
if !covers {
// Don't interrupt a session that is still working towards its own frame.
let busy = !s.needs_key && s.target != i64::MIN && s.out_max.get() < s.target && now_ms() - s.last_output.get() < STALE_MS;
// Don't interrupt a session that is still working towards its own frame, unless it
// has gone quiet for STALE_MS (a copyTo that never settles must not wedge it).
let fresh = now_ms() - s.last_output.get() < STALE_MS;
let busy = fresh && (s.copying.get() > 0 || (!s.needs_key && s.target != i64::MIN && s.out_max.get() < s.target));
if busy {
return Drive::Waiting;
}
if !s.needs_key || s.next > 0 {
s.generation.set(s.generation.get().wrapping_add(1));
let _ = call(&s.decoder, "reset", &[]);
if s.configure(&self.config).is_err() {
return Drive::Failed;
@@ -410,7 +498,7 @@ impl WcSource {
s.target = want;
}
let upto = (i + LOOKAHEAD).min(n - 1);
while s.next <= upto && s.queue() < MAX_QUEUE {
while s.next <= upto && s.queue().saturating_add(s.copying.get()) < MAX_QUEUE {
let smp = &t.samples[s.next];
let mut data = vec![0u8; smp.size as usize];
match self.reader.read_at(smp.offset, &mut data) {
@@ -480,10 +568,8 @@ impl WcSource {
impl Drop for WcSource {
fn drop(&mut self) {
SESSIONS.with(|ss| {
if let Ok(mut ss) = ss.try_borrow_mut()
&& let Some(s) = ss.remove(&self.id)
{
let _ = call(&s.decoder, "close", &[]);
if let Ok(mut ss) = ss.try_borrow_mut() {
ss.remove(&self.id);
}
});
}
@@ -532,4 +618,7 @@ impl MediaSource for WcSource {
fn audio(&self, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
self.inner.audio(start, frames, sample_rate)
}
fn audio_stream(&self, stream: usize, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
self.inner.audio_stream(stream, start, frames, sample_rate)
}
}
+148
View File
@@ -0,0 +1,148 @@
//! Browser planes -> core frames; no color conversion or chroma reconstruction here.
use std::sync::Arc;
use filmcraft_color::{ColorInfo, Matrix, Primaries, Range, Transfer};
use filmcraft_frame::{Chroma, PixelData, VideoFrame};
use filmcraft_time::Tick;
use wasm_bindgen::{JsCast, JsValue};
use wasm_bindgen_futures::JsFuture;
use super::{Config, call, get, obj};
// One 8K 4:4:4 16-bit picture + alpha fits within 256 MiB; cap browser-driven allocations.
const MAX_COPY_BYTES: u32 = 256 * 1024 * 1024;
fn integer(value: JsValue) -> Option<u32> {
let n = value.as_f64()?;
if n.is_finite() && n >= 0.0 && n <= u32::MAX as f64 && n.fract() == 0.0 { Some(n as u32) } else { None }
}
fn color(frame: &JsValue, fallback: ColorInfo) -> Option<ColorInfo> {
let c = get(frame, "colorSpace");
let matrix = match get(&c, "matrix").as_string().as_deref() {
Some("bt709") => Matrix::Bt709,
Some("bt470bg" | "smpte170m") => Matrix::Bt601,
Some("bt2020-ncl") => Matrix::Bt2020Ncl,
None => fallback.matrix,
_ => return None,
};
let transfer = match get(&c, "transfer").as_string().as_deref() {
Some("bt709") => Transfer::Bt709,
Some("iec61966-2-1") => Transfer::Srgb,
Some("smpte2084") => Transfer::Pq,
Some("arib-std-b67") => Transfer::Hlg,
None => fallback.transfer,
_ => return None,
};
let primaries = match get(&c, "primaries").as_string().as_deref() {
Some("bt709") => Primaries::Bt709,
Some("bt470bg") => Primaries::Bt601_625,
Some("smpte170m") => Primaries::Bt601_525,
Some("bt2020") => Primaries::Bt2020,
None => fallback.primaries,
_ => return None,
};
let range = match get(&c, "fullRange").as_bool() {
Some(true) => Range::Full,
Some(false) => Range::Limited,
None => fallback.range,
};
Some(ColorInfo { matrix, transfer, primaries, range })
}
fn plane(raw: &js_sys::Uint8Array, layout: &JsValue, width: u32, height: u32) -> Result<Vec<u8>, &'static str> {
let offset = integer(get(layout, "offset")).ok_or("layout")?;
let stride = integer(get(layout, "stride")).filter(|s| *s >= width).ok_or("layout")?;
let end =
height.checked_sub(1).and_then(|h| h.checked_mul(stride)).and_then(|n| offset.checked_add(n)).and_then(|n| n.checked_add(width)).ok_or("layout")?;
if end > raw.length() {
return Err("layout");
}
let len = width.checked_mul(height).filter(|n| *n <= MAX_COPY_BYTES).ok_or("allocation")? as usize;
let mut out = Vec::new();
out.try_reserve_exact(len).map_err(|_| "memory")?;
out.resize(len, 0);
if stride == width {
raw.subarray(offset, end).copy_to(&mut out);
} else {
for (y, row) in out.chunks_exact_mut(width as usize).enumerate() {
// Checked for the last row above; earlier rows are smaller, but stay checked anyway.
let start = u32::try_from(y).ok().and_then(|y| y.checked_mul(stride)).and_then(|n| offset.checked_add(n)).ok_or("layout")?;
let row_end = start.checked_add(width).filter(|e| *e <= end).ok_or("layout")?;
raw.subarray(start, row_end).copy_to(row);
}
}
Ok(out)
}
/// Copy native 8-bit YUV planes; unsupported formats/geometry return a counted Canvas fallback reason.
pub(super) async fn read(frame: &JsValue, cfg: &Config) -> Result<(VideoFrame, u32), &'static str> {
let format = get(frame, "format").as_string().ok_or("format")?;
let (chroma, alpha, interleaved) = match format.as_str() {
"NV12" => (Chroma::C420, false, true),
"I420" => (Chroma::C420, false, false),
"I420A" => (Chroma::C420, true, false),
"I422" => (Chroma::C422, false, false),
"I422A" => (Chroma::C422, true, false),
"I444" => (Chroma::C444, false, false),
"I444A" => (Chroma::C444, true, false),
_ => return Err("format"),
};
let rect = get(frame, "visibleRect");
let w = integer(get(&rect, "width")).filter(|n| *n > 0).ok_or("geometry")?;
let h = integer(get(&rect, "height")).filter(|n| *n > 0).ok_or("geometry")?;
if integer(get(frame, "displayWidth")) != Some(w)
|| integer(get(frame, "displayHeight")) != Some(h)
|| get(frame, "rotation").as_f64().is_some_and(|r| r != 0.0)
|| get(frame, "flip").as_bool() == Some(true)
{
return Err("geometry");
}
let color = color(frame, cfg.color).ok_or("color")?;
let options = obj(&[("rect", rect)]);
let bytes =
integer(call(frame, "allocationSize", &[&options]).map_err(|_| "allocation")?).filter(|n| *n > 0 && *n <= MAX_COPY_BYTES).ok_or("allocation")?;
let array: js_sys::Function = get(&js_sys::global(), "Uint8Array").dyn_into().map_err(|_| "allocation")?;
let raw: js_sys::Uint8Array =
js_sys::Reflect::construct(&array, &js_sys::Array::of1(&bytes.into())).and_then(|v| v.dyn_into()).map_err(|_| "allocation")?;
let promise = call(frame, "copyTo", &[raw.as_ref(), &options]).map_err(|_| "copy")?.dyn_into::<js_sys::Promise>().map_err(|_| "copy")?;
let layout = js_sys::Array::from(&JsFuture::from(promise).await.map_err(|_| "copy")?);
let count = if interleaved {
2
} else if alpha {
4
} else {
3
};
if layout.length() != count {
return Err("layout");
}
let (sx, sy) = chroma.shifts();
let (cw, ch) = (w.div_ceil(1 << sx), h.div_ceil(1 << sy));
let y = plane(&raw, &layout.get(0), w, h)?;
let (u, v) = if interleaved {
let uv = plane(&raw, &layout.get(1), cw.checked_mul(2).ok_or("allocation")?, ch)?;
let pairs = uv.as_chunks::<2>().0;
let (mut u, mut v) = (Vec::new(), Vec::new());
u.try_reserve_exact(pairs.len()).map_err(|_| "memory")?;
v.try_reserve_exact(pairs.len()).map_err(|_| "memory")?;
u.extend(pairs.iter().map(|p| p[0]));
v.extend(pairs.iter().map(|p| p[1]));
(u, v)
} else {
(plane(&raw, &layout.get(1), cw, ch)?, plane(&raw, &layout.get(2), cw, ch)?)
};
let alpha = if alpha { Some(Arc::new(plane(&raw, &layout.get(3), w, h)?)) } else { None };
Ok((
VideoFrame {
width: w,
height: h,
data: PixelData::Yuv8 { planes: [Arc::new(y), Arc::new(u), Arc::new(v)], chroma, alpha },
color,
par: cfg.par,
pts: Tick::ZERO,
},
bytes,
))
}
@@ -0,0 +1,26 @@
//! Download spacing. It is pure, so it runs natively even though the rest of the crate is
//! wasm32-only.
#[path = "../src/download_pacing.rs"]
mod download_pacing;
use download_pacing::{GAP_MS, Pacer};
/// #378: 20 downloads requested in the same instant are spread out, not clicked in one burst.
#[test]
fn burst_of_downloads_is_spaced_out() {
let mut p = Pacer::default();
let delays: Vec<f64> = (0..20).map(|_| p.reserve(1000.0)).collect();
assert_eq!(delays[0], 0.0);
for (i, d) in delays.iter().enumerate() {
assert_eq!(*d, i as f64 * GAP_MS);
}
}
/// A download after a quiet period is not delayed.
#[test]
fn download_after_a_pause_is_immediate() {
let mut p = Pacer::default();
assert_eq!(p.reserve(0.0), 0.0);
assert_eq!(p.reserve(10_000.0), 0.0);
}
+67
View File
@@ -0,0 +1,67 @@
// Web import destinations: active bin tab, browser drop + navigation during pending reads.
// Synthetic browser drop event exercises the production handler; physical OS drag/picker UI is outside this test.
export async function checkImportBins({ evaluate, media, timeoutMs = 120000 }) {
const report = {gates: {}, cases: []};
await evaluate('filmcraft.execute("file.newProject", {name: "Import bins"})');
const parent = (await evaluate('filmcraft.execute("file.newBin", {name: "Footage"})')).bin;
const child = (await evaluate(`filmcraft.execute("file.newBin", {name: "Selects", parent: ${parent}})`)).bin;
await evaluate(`(async () => {window.__importBinFixture = new Uint8Array(await (await fetch(${JSON.stringify(media)})).arrayBuffer());})()`);
const setBin = bin => evaluate(`filmcraft.request("ui.set", {projectPanel: {bin: ${JSON.stringify(bin)}, activeTab: null}})`);
const importExpression = name => `filmcraft.importFiles([new File([window.__importBinFixture], ${JSON.stringify(name)}, {type: "video/mp4"})])`;
const locate = async name => {
const deadline = Date.now() + timeoutMs;
const pollMs = 100;
for (;;) {
const project = await evaluate('filmcraft.execute("project.inspect")');
const find = bin => {
for (const entry of bin.children) {
if (entry.item !== undefined && entry.name === name) return bin.id;
if (entry.children) {const found = find(entry); if (found !== undefined) return found;}
}
};
const found = find(project.root);
if (found !== undefined) return found;
if (Date.now() >= deadline) throw new Error(`Import never appeared: ${name}`);
await new Promise(resolve => setTimeout(resolve, pollMs));
}
};
const check = async (key, expected, name) => {
const actual = await locate(name);
report.cases.push({key, expected, actual, name}); report.gates[key] = actual === expected;
};
await evaluate(`filmcraft.request("ui.set", {projectPanel: {bin: ${child}, activeTab: 0, tabs: [{bin: ${parent}}]}})`);
await evaluate(importExpression("tab.mp4")); await check("activeTab", parent, "tab.mp4");
await setBin(child);
await evaluate(`(() => {
const transfer = new DataTransfer();
transfer.items.add(new File([window.__importBinFixture], "drop.mp4", {type: "video/mp4"}));
window.dispatchEvent(new DragEvent("drop", {dataTransfer: transfer, bubbles: true, cancelable: true}));
})()`);
await check("browserDrop", child, "drop.mp4");
await setBin(child);
await evaluate(`(() => {
const original = Blob.prototype.arrayBuffer;
let release;
const pending = new Promise(resolve => {release = resolve;});
const gate = window.__importBinGate = {hits: 0, release, original};
Blob.prototype.arrayBuffer = function () {gate.hits++; return pending.then(() => original.call(this));};
window.__pendingBinImport = ${importExpression("pending.mp4")};
})()`);
try {
const deadline = Date.now() + timeoutMs;
const pollMs = 100;
for (;;) {
if (await evaluate("window.__importBinGate.hits > 0")) break;
if (Date.now() >= deadline) throw new Error("Read-delay probe did not intercept any Blob reads");
await new Promise(resolve => setTimeout(resolve, pollMs));
}
report.delayedReadHits = await evaluate("window.__importBinGate.hits");
await setBin(parent);
await evaluate("window.__importBinGate.release(); window.__pendingBinImport");
await check("destinationCapturedBeforeRead", child, "pending.mp4");
} finally {
await evaluate("Blob.prototype.arrayBuffer = window.__importBinGate.original; window.__importBinGate.release(); delete window.__importBinGate; delete window.__pendingBinImport; delete window.__importBinFixture");
}
report.ok = Object.values(report.gates).every(Boolean);
return report;
}
@@ -0,0 +1,78 @@
// #194 regression: same-name/same-size imports must retain independent sources across OPFS recovery.
// evaluate(expression) awaits a browser expression; navigate(url) waits for FilmCraft readiness. Generated PCM16 mono WAVs only; no binary fixtures.
import { createHash } from "node:crypto";
function tone(frequency, seconds) {
// RIFF/WAVE PCM16 mono: 44-byte header + 2 bytes/sample; fixture matches #194 (48 kHz, amplitude 12000).
const rate = 48000, bytesPerSample = 2, samples = rate * seconds;
const b = Buffer.alloc(44 + samples * bytesPerSample);
b.write("RIFF", 0); b.writeUInt32LE(b.length - 8, 4); b.write("WAVEfmt ", 8);
b.writeUInt32LE(16, 16); b.writeUInt16LE(1, 20); b.writeUInt16LE(1, 22);
b.writeUInt32LE(rate, 24); b.writeUInt32LE(rate * bytesPerSample, 28);
b.writeUInt16LE(bytesPerSample, 32); b.writeUInt16LE(16, 34); b.write("data", 36); b.writeUInt32LE(samples * bytesPerSample, 40);
for (let i = 0; i < samples; i++) b.writeInt16LE(Math.round(12000 * Math.sin(2 * Math.PI * frequency * i / rate)), 44 + bytesPerSample * i);
return b;
}
export async function checkImportCollision({ evaluate, navigate, url, timeoutMs = 120000 }) {
const fixtures = [tone(440, 2), tone(880, 2)];
const hashes = fixtures.map(b => createHash("sha256").update(b).digest("hex"));
const report = { hashes, fixtureBytes: fixtures.map(b => b.length), gates: {} };
const hash = expression => evaluate(`(async () => Array.from(new Uint8Array(await crypto.subtle.digest("SHA-256", ${expression})), b => b.toString(16).padStart(2, "0")).join(""))()`);
const readHash = path => hash(`await filmcraft.readFile(${JSON.stringify(path)})`);
const importFile = index => evaluate(`filmcraft.importFiles([new File([Uint8Array.from(atob(${JSON.stringify(fixtures[index].toString("base64"))}), c => c.charCodeAt(0))], "collision.wav", {type: "audio/wav"})])`);
await evaluate('filmcraft.execute("file.newProject", {name: "Import collision"})');
const a = await importFile(0); report.a = a;
if (a.errors?.length || !a.items?.length) throw new Error("baseline import failed: " + JSON.stringify(a));
report.initialHash = await readHash(a.paths[0]);
report.gates.healthyInitial = report.initialHash === hashes[0];
await evaluate(`filmcraft.execute("file.newSequence", {name: "Original A", fromItem: ${a.items[0]}})`);
const b = await importFile(1); report.b = b;
if (b.errors?.length || !b.items?.length) throw new Error("second import failed: " + JSON.stringify(b));
report.afterBHash = await readHash(a.paths[0]);
report.gates.distinctPaths = a.paths[0] !== b.paths[0];
report.gates.originalPreserved = report.afterBHash === hashes[0];
report.beforeProject = await evaluate('filmcraft.execute("project.inspect")');
const revision = report.beforeProject.revision;
const deadline = Date.now() + timeoutMs;
const persistencePollMs = 100;
for (;;) {
const snapshot = await evaluate(`(async () => {
try {
const dir = await (await navigator.storage.getDirectory()).getDirectoryHandle("recovery");
const read = async name => JSON.parse(await (await (await dir.getFileHandle(name)).getFile()).text());
return {meta: await read("meta.json"), project: await read("snapshot.fcproj")};
} catch (e) {return {error: String(e)};}
})()`);
report.recoveryMeta = snapshot.meta;
if (snapshot.meta?.dirty && snapshot.meta.revision === revision && JSON.stringify(snapshot.project).includes("Original A")) break;
if (Date.now() >= deadline) throw new Error("Recovery persistence gate failed: " + JSON.stringify(snapshot));
await new Promise(resolve => setTimeout(resolve, persistencePollMs));
}
// Media writes run separately from the snapshot queue; gate their actual bytes before navigation.
for (;;) {
report.opfs = await evaluate(`(async () => {
const dir = await (await navigator.storage.getDirectory()).getDirectoryHandle("media");
const files = [];
for await (const [name, handle] of dir.entries()) {
const bytes = await (await handle.getFile()).arrayBuffer();
files.push({name, hash: Array.from(new Uint8Array(await crypto.subtle.digest("SHA-256", bytes)), b => b.toString(16).padStart(2, "0")).join("")});
}
return files;
})()`);
// Before the fix, A and B intentionally share the same path; qualify the last write there.
const expected = new Map([[a.paths[0], hashes[0]], [b.paths[0], hashes[1]]]);
if ([...expected].every(([path, digest]) => report.opfs.some(f => f.name === path.slice("/files/".length) && f.hash === digest))) break;
if (Date.now() >= deadline) throw new Error("Imported media did not reach OPFS: " + JSON.stringify(report.opfs));
await new Promise(resolve => setTimeout(resolve, persistencePollMs));
}
const recoveryUrl = new URL(url);
recoveryUrl.searchParams.delete("fresh"); recoveryUrl.searchParams.delete("norecover");
await navigate(recoveryUrl.href);
report.recovered = await evaluate('filmcraft.execute("project.inspect")');
report.afterReload = {a: await readHash(a.paths[0]), b: await readHash(b.paths[0])};
report.gates.recoveredProject = JSON.stringify(report.recovered).includes("Original A");
report.gates.recoveredSources = report.afterReload.a === hashes[0] && report.afterReload.b === hashes[1];
report.ok = Object.values(report.gates).every(Boolean);
return report;
}
@@ -0,0 +1,77 @@
//! The web recovery snapshot policy. It is pure, so it runs natively even though the rest of the
//! crate is wasm32-only.
#[path = "../src/recovery_policy.rs"]
mod recovery_policy;
use recovery_policy::{Action, INTERVAL_S, Policy};
/// #197: a snapshot write that fails is retried after the interval without another edit.
#[test]
fn failed_snapshot_write_is_retried_without_another_edit() {
let mut p = Policy::default();
assert_eq!(p.tick(0.0, 3, 0), Action::WriteSnapshot(3));
p.written(3, false, 0.1);
// not immediately (no spinning on a failing disk)…
assert!(matches!(p.tick(1.0, 3, 0), Action::WakeAfter(_)));
// …but once the interval has passed, with the revision unchanged
assert_eq!(p.tick(0.1 + INTERVAL_S, 3, 0), Action::WriteSnapshot(3));
p.written(3, true, 5.2);
assert_eq!(p.tick(20.0, 3, 0), Action::Nothing);
}
/// A failure reported for an older revision doesn't make the newer one be written again.
#[test]
fn stale_failure_is_ignored() {
let mut p = Policy::default();
assert_eq!(p.tick(0.0, 1, 0), Action::WriteSnapshot(1));
assert_eq!(p.tick(INTERVAL_S, 2, 0), Action::WriteSnapshot(2));
p.written(1, false, INTERVAL_S);
assert_eq!(p.tick(3.0 * INTERVAL_S, 2, 0), Action::Nothing);
}
/// #193: a clean session (started with `?norecover` / `?fresh`, so the snapshot left by an
/// earlier session was not reopened) never marks that snapshot clean.
#[test]
fn clean_start_leaves_an_unconsumed_snapshot_dirty() {
let mut p = Policy::default();
for t in 0..100 {
assert_eq!(p.tick(t as f64, 0, 0), Action::Nothing);
}
}
/// The snapshot this session wrote is retired once the project is saved.
#[test]
fn saving_marks_own_snapshot_clean() {
let mut p = Policy::default();
assert_eq!(p.tick(0.0, 1, 0), Action::WriteSnapshot(1));
p.written(1, true, 0.1);
assert_eq!(p.tick(1.0, 1, 1), Action::WriteClean);
assert_eq!(p.tick(1.1, 1, 1), Action::Nothing);
p.cleaned(true, 1.2);
assert_eq!(p.tick(30.0, 1, 1), Action::Nothing);
}
/// A failed clean write is retried after the interval.
#[test]
fn failed_clean_write_is_retried() {
let mut p = Policy::default();
assert_eq!(p.tick(0.0, 1, 0), Action::WriteSnapshot(1));
p.written(1, true, 0.1);
assert_eq!(p.tick(1.0, 1, 1), Action::WriteClean);
p.cleaned(false, 1.1);
assert!(matches!(p.tick(2.0, 1, 1), Action::WakeAfter(_)));
assert_eq!(p.tick(1.1 + INTERVAL_S, 1, 1), Action::WriteClean);
}
/// An edit made while the clean write is in flight keeps the meta dirty after it lands.
#[test]
fn edit_during_clean_write_keeps_meta_owned() {
let mut p = Policy::default();
assert_eq!(p.tick(0.0, 1, 0), Action::WriteSnapshot(1));
assert_eq!(p.tick(10.0, 1, 1), Action::WriteClean);
assert_eq!(p.tick(20.0, 2, 1), Action::WriteSnapshot(2));
p.cleaned(true, 20.1);
// saved again: the newer snapshot must be retired too
assert_eq!(p.tick(30.0, 2, 2), Action::WriteClean);
}
+15
View File
@@ -13,6 +13,8 @@ import { spawn } from "node:child_process";
import { mkdirSync, writeFileSync, readdirSync, statSync, mkdtempSync } from "node:fs";
import { join } from "node:path";
import { tmpdir } from "node:os";
import { checkImportBins } from "./import-bins.mjs";
import { checkImportCollision } from "./import-collision.mjs";
const arg = (k, d) => {
const i = process.argv.indexOf(`--${k}`);
@@ -162,6 +164,19 @@ try {
await sleep(500);
await send("Emulation.setDeviceMetricsOverride", { width: 1600, height: 1000, deviceScaleFactor: 1, mobile: false });
report.steps.modes = { screenshot: await shot("05-modes") };
report.steps.importBins = await checkImportBins({evaluate: js, media});
if (!report.steps.importBins.ok) throw new Error("import bin regression: " + JSON.stringify(report.steps.importBins.gates));
report.steps.importCollision = await checkImportCollision({
evaluate: js,
url,
navigate: async (nextUrl) => {
await send("Page.navigate", { url: nextUrl });
await until("!!(window.filmcraftLoad && (window.filmcraftLoad.readyMs || window.filmcraftLoad.error))", 120000);
const load = await js("window.filmcraftLoad");
if (load.error) throw new Error(load.error);
},
});
if (!report.steps.importCollision.ok) throw new Error("import collision regression: " + JSON.stringify(report.steps.importCollision.gates));
// still alive? (a panicked app never answers)
await Promise.race([js("filmcraft.inspect().then(() => true)"), sleep(5000).then(() => { throw new Error("app stopped answering"); })]);
const fatal = logs.filter((l) => /panicked at|RuntimeError|EXCEPTION/.test(l));
+20
View File
@@ -14,9 +14,16 @@ workspace = true
default = []
# Speech-to-text (Text panel ▸ Transcribe) with Whisper on the CPU, and model downloads.
whisper = ["filmcraft-engine/whisper", "filmcraft-engine/speech-download"]
# Natural Text to Speech voices (Kokoro-82M on the CPU, downloaded on first use).
neural-voices = ["filmcraft-engine/neural-voices"]
[dependencies]
filmcraft-engine = { workspace = true }
filmcraft-export = { workspace = true }
filmcraft-project = { workspace = true }
filmcraft-render = { workspace = true }
filmcraft-time = { workspace = true }
filmcraft-gpu = { workspace = true }
filmcraft-ui-egui = { workspace = true }
filmcraft-platform = { workspace = true }
egui = { workspace = true }
@@ -25,7 +32,15 @@ serde_json = { workspace = true }
log = { workspace = true }
rfd = { workspace = true }
cpal = "0.16"
# Files and images on the system clipboard for Paste (#611); egui already uses it for text.
arboard = "3"
chrono = { version = "0.4", default-features = false, features = ["clock"] }
sys-locale = "0.3"
# Help menu / About / Discord links in the system browser (MIT; as PhotoCraft).
open = "5"
[dev-dependencies]
glob = "0.3"
[target.'cfg(target_os = "macos")'.dependencies]
muda = "0.21"
@@ -35,6 +50,11 @@ objc2 = "0.6"
# Same winit as eframe: macOS event-loop options (no app activation when driven by an agent).
winit = { version = "0.30", default-features = false }
[target.'cfg(target_os = "linux")'.dependencies]
# Settings ▸ Appearance ▸ Sync with system: the XDG desktop portal's colour scheme. The zbus the
# portal file dialogs (rfd) and accesskit already use.
zbus = "5.19"
# Icon + VERSIONINFO for filmcraft.exe (build.rs); only needed when building on Windows.
[target.'cfg(windows)'.build-dependencies]
winresource = "0.1"
+188
View File
@@ -0,0 +1,188 @@
//! The system's light or dark appearance for Settings ▸ Appearance ▸ Appearance Mode ▸ Sync with
//! system, on Linux desktops where winit reports none (Wayland compositors send no theme).
//!
//! The XDG desktop portal is read once (`org.freedesktop.portal.Settings.Read` of
//! `org.freedesktop.appearance` / `color-scheme`), then its `SettingChanged` signal is followed on a
//! worker thread, which wakes the UI only when the value changes. There is no polling. Without a
//! portal answer, `gsettings` runs once at start-up, by absolute path. Elsewhere the host returns no
//! service and the UI uses what egui reports.
#[cfg(target_os = "linux")]
const PORTAL_NAMESPACE: &str = "org.freedesktop.appearance";
#[cfg(target_os = "linux")]
const PORTAL_KEY: &str = "color-scheme";
#[cfg(target_os = "linux")]
fn portal_proxy() -> Option<zbus::blocking::Proxy<'static>> {
let conn = zbus::blocking::connection::Builder::session().ok()?.method_timeout(std::time::Duration::from_secs(2)).build().ok()?;
zbus::blocking::Proxy::new(&conn, "org.freedesktop.portal.Desktop", "/org/freedesktop/portal/desktop", "org.freedesktop.portal.Settings").ok()
}
#[cfg(target_os = "linux")]
fn portal_theme(proxy: &zbus::blocking::Proxy<'_>) -> Option<egui::Theme> {
let value: zbus::zvariant::OwnedValue = proxy.call("Read", &(PORTAL_NAMESPACE, PORTAL_KEY)).ok()?;
decode(portal_code(value)?)
}
/// The `u32` inside a portal reply. `Read` returns a variant, and some portals nest another
/// variant inside it.
#[cfg(target_os = "linux")]
fn portal_code(value: zbus::zvariant::OwnedValue) -> Option<u32> {
let mut value: zbus::zvariant::Value<'_> = value.into();
for _ in 0..4 {
match value {
zbus::zvariant::Value::Value(inner) => value = *inner,
other => return u32::try_from(other).ok(),
}
}
None
}
/// The portal's `color-scheme`: 0 no preference, 1 prefer dark, 2 prefer light.
#[cfg(target_os = "linux")]
fn decode(code: u32) -> Option<egui::Theme> {
match code {
1 => Some(egui::Theme::Dark),
2 => Some(egui::Theme::Light),
_ => None,
}
}
/// One-shot fallback for desktops without the settings portal: runs once, by absolute path.
#[cfg(target_os = "linux")]
fn gtk_theme() -> Option<egui::Theme> {
let exe = ["/usr/bin/gsettings", "/bin/gsettings", "/usr/local/bin/gsettings"].into_iter().find(|p| std::path::Path::new(p).is_file())?;
let output = std::process::Command::new(exe).args(["get", "org.gnome.desktop.interface", "color-scheme"]).output().ok()?;
if !output.status.success() {
return None;
}
parse_gtk_scheme(std::str::from_utf8(&output.stdout).ok()?)
}
#[cfg(target_os = "linux")]
fn parse_gtk_scheme(s: &str) -> Option<egui::Theme> {
match s.trim().trim_matches('\'') {
"prefer-dark" => Some(egui::Theme::Dark),
"prefer-light" => Some(egui::Theme::Light),
_ => None,
}
}
#[cfg(target_os = "linux")]
fn code_of(theme: Option<egui::Theme>) -> u8 {
match theme {
Some(egui::Theme::Dark) => 1,
Some(egui::Theme::Light) => 2,
None => 0,
}
}
/// Publish initial and subsequent answers alike: the initial Read may finish after the UI's
/// bounded startup wait, so it must also wake an already-created context.
#[cfg(target_os = "linux")]
fn publish(value: &std::sync::atomic::AtomicU8, code: u8, repaint: impl FnOnce()) {
if value.swap(code, std::sync::atomic::Ordering::Relaxed) != code {
repaint();
}
}
/// The host hook (`HostHooks::system_theme`), or `None` where egui's own report is used.
pub fn service() -> Option<filmcraft_ui_egui::SystemThemeFn> {
#[cfg(target_os = "linux")]
{
use std::sync::atomic::{AtomicU8, Ordering};
use std::sync::{Arc, OnceLock};
let value = Arc::new(AtomicU8::new(0));
let wake: Arc<OnceLock<egui::Context>> = Arc::new(OnceLock::new());
let (worker_value, worker_wake) = (Arc::clone(&value), Arc::clone(&wake));
let (ready, wait) = std::sync::mpsc::channel();
let worker = move || {
let proxy = portal_proxy();
// Subscribe before Read so a change during startup cannot be missed.
let signals = proxy.as_ref().and_then(|p| p.receive_signal("SettingChanged").ok());
let initial = proxy.as_ref().and_then(portal_theme).or_else(gtk_theme);
publish(&worker_value, code_of(initial), || {
if let Some(ctx) = worker_wake.get() {
ctx.request_repaint();
}
});
let _ = ready.send(());
let Some(signals) = signals else { return };
// Sleeps in the signal iterator: no polling.
for message in signals {
let Ok((namespace, key, changed)) = message.body().deserialize::<(String, String, zbus::zvariant::OwnedValue)>() else {
continue;
};
if namespace != PORTAL_NAMESPACE || key != PORTAL_KEY {
continue;
}
let code = code_of(portal_code(changed).and_then(decode));
publish(&worker_value, code, || {
if let Some(ctx) = worker_wake.get() {
ctx.request_repaint();
}
});
}
};
let spawned = std::thread::Builder::new().name("appearance-portal".into()).spawn(move || {
if std::panic::catch_unwind(std::panic::AssertUnwindSafe(worker)).is_err() {
log::warn!("the system appearance watcher stopped; Sync with system keeps the last value");
}
});
if spawned.is_err() {
return None;
}
// The first reading, so the first frame already shows the right theme (bounded wait).
let _ = wait.recv_timeout(std::time::Duration::from_millis(250));
Some(Box::new(move |ctx: &egui::Context| {
let _ = wake.set(ctx.clone());
decode(u32::from(value.load(Ordering::Relaxed)))
}))
}
#[cfg(not(target_os = "linux"))]
{
None
}
}
#[cfg(test)]
mod tests {
#[cfg(target_os = "linux")]
#[test]
fn initial_and_signal_answers_repaint_only_on_changes() {
use std::sync::atomic::{AtomicU8, AtomicUsize, Ordering};
let repaints = AtomicUsize::new(0);
let value = AtomicU8::new(0);
let repaint = || {
repaints.fetch_add(1, Ordering::Relaxed);
};
// A late initial Read takes the same publication path as later signals.
super::publish(&value, 2, repaint);
assert_eq!(repaints.load(Ordering::Relaxed), 1);
super::publish(&value, 2, repaint);
assert_eq!(repaints.load(Ordering::Relaxed), 1);
super::publish(&value, 1, repaint);
assert_eq!(repaints.load(Ordering::Relaxed), 2);
assert_eq!(value.load(Ordering::Relaxed), 1);
}
#[cfg(target_os = "linux")]
#[test]
fn portal_and_gsettings_values_decode() {
assert_eq!(super::decode(0), None);
assert_eq!(super::decode(1), Some(egui::Theme::Dark));
assert_eq!(super::decode(2), Some(egui::Theme::Light));
assert_eq!(super::decode(9), None);
let nested = zbus::zvariant::Value::Value(Box::new(zbus::zvariant::Value::Value(Box::new(zbus::zvariant::Value::U32(2)))));
assert_eq!(super::portal_code(zbus::zvariant::OwnedValue::try_from(nested).unwrap()), Some(2));
let plain = zbus::zvariant::OwnedValue::from(1u32);
assert_eq!(super::portal_code(plain), Some(1));
let wrong = zbus::zvariant::OwnedValue::try_from(zbus::zvariant::Value::from("dark")).unwrap();
assert_eq!(super::portal_code(wrong), None);
assert_eq!(super::parse_gtk_scheme("'prefer-light'\n"), Some(egui::Theme::Light));
assert_eq!(super::parse_gtk_scheme("'prefer-dark'\n"), Some(egui::Theme::Dark));
assert_eq!(super::parse_gtk_scheme("'default'\n"), None);
assert_eq!(super::parse_gtk_scheme(""), None);
assert_eq!((super::code_of(None), super::code_of(Some(egui::Theme::Dark)), super::code_of(Some(egui::Theme::Light))), (0, 1, 2));
}
}
+224 -27
View File
@@ -1,8 +1,9 @@
//! cpal audio output: the playback master clock. Settings ▸ Audio Hardware picks the host
//! ("Device Class"), the output device, the I/O buffer size and the sample rate.
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::{Arc, Mutex, PoisonError};
use cpal::traits::{DeviceTrait, HostTrait, StreamTrait};
use filmcraft_engine::settings::AudioHardwarePrefs;
@@ -11,11 +12,37 @@ use filmcraft_ui_egui::{AudioDevices, AudioOut};
pub struct CpalOut {
stream: Option<cpal::Stream>,
played: Arc<AtomicU64>,
failed: Arc<AtomicBool>,
rate: u32,
channels: u16,
hw: AudioHardwarePrefs,
/// Sequence sample rate for "Attempt to force hardware to document sample rate".
document_rate: Option<u32>,
/// Status-bar note: the host's default device could not be opened and a fallback device is
/// in use, or no output device was found (playing without sound).
note: Option<Note>,
/// The fallback picked when the default device could not be opened, as (host, device) names,
/// so later lookups (`devices`, `configure`, `start`) reuse it instead of probing every PCM.
fallback: Mutex<Option<(String, String)>>,
}
/// A hardware note for the status bar, translated when it is shown.
#[derive(Clone, Debug, PartialEq)]
enum Note {
/// The default output device cannot be opened; this named device is used instead.
Fallback(String),
/// No output device at all: playback runs without sound.
NoDevice,
}
impl Note {
fn text(&self) -> String {
use filmcraft_ui_egui::i18n::{fmt, t};
match self {
Note::Fallback(device) => fmt(t("Audio hardware: the default output device cannot be opened; using '{device}'"), &[("device", device)]),
Note::NoDevice => t("No audio output device found: playing without sound (check Settings ▸ Audio Hardware)").to_string(),
}
}
}
/// The host named in the settings (empty or unknown = the default host).
@@ -29,55 +56,164 @@ fn host(name: &str) -> cpal::Host {
cpal::default_host()
}
/// The output device named in the settings (empty or missing = the host's default).
fn output_device(h: &cpal::Host, name: &str) -> Option<cpal::Device> {
/// The output device the settings ask for, with a status-bar note when a fallback is in use.
struct DeviceChoice {
device: cpal::Device,
/// The host's default device could not be opened: this device is the fallback.
fallback: Option<Note>,
}
/// Fallback order when the default device cannot be opened: the sound-server bridges first
/// (`pipewire`, `pulse`: they share the card with every other app), then other named PCMs, and
/// raw `hw:`/`plughw:` devices last (opening one takes the card exclusively and silences the
/// rest of the desktop).
fn fallback_rank(name: &str) -> u8 {
let n = name.to_ascii_lowercase();
if n == "pipewire" || n.starts_with("pipewire:") {
0
} else if n == "pulse" || n.starts_with("pulse:") {
1
} else if n.starts_with("hw:") || n.starts_with("plughw:") {
3
} else {
2
}
}
/// The output device named in the settings (empty or missing = the host's default). When the
/// default cannot be opened — a broken ALSA `default` (missing `99-pipewire-default.conf`,
/// #23/#106) — a fallback that can, so the app is not silently mute. The fallback list is probed
/// only after the default fails, in [`fallback_rank`] order, and the pick is cached in `cache`.
fn output_device(h: &cpal::Host, name: &str, cache: &Mutex<Option<(String, String)>>) -> Option<DeviceChoice> {
if !name.is_empty()
&& let Ok(mut devs) = h.output_devices()
&& let Some(d) = devs.find(|d| d.name().is_ok_and(|n| n == name))
{
return Some(d);
return Some(DeviceChoice { device: d, fallback: None });
}
h.default_output_device()
if let Some(d) = h.default_output_device()
&& d.default_output_config().is_ok()
{
return Some(DeviceChoice { device: d, fallback: None });
}
let host_name = h.id().name().to_string();
let mut cache = cache.lock().unwrap_or_else(PoisonError::into_inner);
let mut devs: Vec<(String, cpal::Device)> = h.output_devices().ok()?.filter_map(|d| d.name().ok().map(|n| (n, d))).collect();
if let Some((cached_host, cached)) = cache.as_ref()
&& *cached_host == host_name
&& let Some(i) = devs.iter().position(|(n, _)| n == cached)
{
let (found, d) = devs.swap_remove(i);
return Some(DeviceChoice { device: d, fallback: Some(Note::Fallback(found)) });
}
devs.sort_by_key(|(n, _)| fallback_rank(n));
let (found, d) = devs.into_iter().find(|(_, d)| d.default_output_config().is_ok())?;
log::warn!("the default audio output device cannot be opened; falling back to '{found}'");
*cache = Some((host_name, found.clone()));
Some(DeviceChoice { device: d, fallback: Some(Note::Fallback(found)) })
}
impl CpalOut {
pub fn new() -> Option<Self> {
pub fn new() -> Self {
let host = cpal::default_host();
let dev = host.default_output_device()?;
let cfg = dev.default_output_config().ok()?;
Some(Self {
let fallback = Mutex::new(None);
let choice = output_device(&host, "", &fallback);
let cfg = choice.as_ref().and_then(|c| c.device.default_output_config().ok());
let note = match choice {
Some(c) => c.fallback,
None => {
log::warn!("no audio output device found");
Some(Note::NoDevice)
}
};
Self {
stream: None,
played: Arc::new(AtomicU64::new(0)),
rate: cfg.sample_rate().0,
channels: cfg.channels(),
failed: Arc::new(AtomicBool::new(false)),
rate: cfg.as_ref().map_or(48_000, |c| c.sample_rate().0),
channels: cfg.as_ref().map_or(2, |c| c.channels()),
hw: AudioHardwarePrefs::default(),
document_rate: None,
})
note,
fallback,
}
}
/// The stream configuration the settings ask for, falling back to the device default.
fn config(&self, dev: &cpal::Device) -> Result<cpal::SupportedStreamConfig, String> {
let default = dev.default_output_config().map_err(|e| e.to_string())?;
let want = if self.hw.force_document_rate { self.document_rate.unwrap_or(self.hw.sample_rate) } else { self.hw.sample_rate };
let exact = dev.supported_output_configs().ok().and_then(|mut it| {
it.find(|c| {
c.sample_format() == cpal::SampleFormat::F32
&& c.channels() == default.channels()
&& (c.min_sample_rate().0..=c.max_sample_rate().0).contains(&want)
})
.and_then(|c| c.try_with_sample_rate(cpal::SampleRate(want)))
let exact = dev.supported_output_configs().ok().and_then(|it| {
it.filter(|c| c.channels() > 0 && (c.min_sample_rate().0..=c.max_sample_rate().0).contains(&want))
.max_by_key(|c| {
(c.channels() == default.channels(), c.sample_format() == cpal::SampleFormat::F32, c.sample_format() == default.sample_format())
})
.and_then(|c| c.try_with_sample_rate(cpal::SampleRate(want)))
});
Ok(exact.unwrap_or(default))
}
fn converted_stream<T>(
&self,
dev: &cpal::Device,
config: &cpal::StreamConfig,
mut fill: Box<dyn FnMut(&mut [f32], usize) + Send>,
) -> Result<cpal::Stream, cpal::BuildStreamError>
where
T: cpal::SizedSample + cpal::FromSample<f32>,
{
let channels = usize::from(config.channels);
let played = self.played.clone();
let failed = self.failed.clone();
let error_flag = failed.clone();
let mut scratch = Vec::new();
dev.build_output_stream(
config,
move |buf: &mut [T], _| {
// Reuse the conversion buffer. A pathological driver buffer must not panic the audio thread.
if buf.len() > 1_048_576 || scratch.try_reserve(buf.len().saturating_sub(scratch.len())).is_err() {
buf.fill(T::EQUILIBRIUM);
if !failed.swap(true, Ordering::SeqCst) {
eprintln!("filmcraft: unable to allocate the audio conversion buffer");
}
return;
}
scratch.resize(buf.len(), 0.0);
fill(&mut scratch, channels);
convert_samples(&scratch, buf);
played.fetch_add((buf.len() / channels) as u64, Ordering::SeqCst);
},
move |e| stream_error(&error_flag, e),
None,
)
}
}
fn stream_error(failed: &AtomicBool, e: cpal::StreamError) {
if !failed.swap(true, Ordering::SeqCst) {
eprintln!("filmcraft: audio stream error: {e}");
}
}
fn convert_samples<T: cpal::Sample + cpal::FromSample<f32>>(input: &[f32], output: &mut [T]) {
// Integer conversion represents [-1, 1). In particular I24 cannot represent +1 exactly.
let upper = f32::from_bits(1.0_f32.to_bits() - 1);
for (sample, converted) in input.iter().zip(output) {
*converted = T::from_sample(if sample.is_finite() { sample.clamp(-1.0, upper) } else { 0.0 });
}
}
impl AudioOut for CpalOut {
fn start(&mut self, mut fill: Box<dyn FnMut(&mut [f32], usize) + Send>) -> Result<u32, String> {
self.stop();
let host = host(&self.hw.device_class);
let dev = output_device(&host, &self.hw.default_output).ok_or("no output device")?;
let DeviceChoice { device: dev, fallback } = output_device(&host, &self.hw.default_output, &self.fallback).ok_or("no output device")?;
self.note = fallback;
let cfg = self.config(&dev)?;
let channels = cfg.channels() as usize;
if channels == 0 || cfg.sample_rate().0 == 0 {
return Err("the output device returned an invalid channel count or sample rate".into());
}
let mut config: cpal::StreamConfig = cfg.clone().into();
if let cpal::SupportedBufferSize::Range { min, max } = cfg.buffer_size()
&& (*min..=*max).contains(&self.hw.buffer_size)
@@ -87,8 +223,9 @@ impl AudioOut for CpalOut {
self.rate = cfg.sample_rate().0;
self.channels = cfg.channels();
self.played.store(0, Ordering::SeqCst);
self.failed.store(false, Ordering::SeqCst);
let played = self.played.clone();
let err = |e| eprintln!("filmcraft: audio stream error: {e}");
let failed = self.failed.clone();
let stream = match cfg.sample_format() {
cpal::SampleFormat::F32 => dev.build_output_stream(
&config,
@@ -96,9 +233,19 @@ impl AudioOut for CpalOut {
fill(buf, channels);
played.fetch_add((buf.len() / channels) as u64, Ordering::SeqCst);
},
err,
move |e| stream_error(&failed, e),
None,
),
cpal::SampleFormat::I8 => self.converted_stream::<i8>(&dev, &config, fill),
cpal::SampleFormat::I16 => self.converted_stream::<i16>(&dev, &config, fill),
cpal::SampleFormat::I24 => self.converted_stream::<cpal::I24>(&dev, &config, fill),
cpal::SampleFormat::I32 => self.converted_stream::<i32>(&dev, &config, fill),
cpal::SampleFormat::I64 => self.converted_stream::<i64>(&dev, &config, fill),
cpal::SampleFormat::U8 => self.converted_stream::<u8>(&dev, &config, fill),
cpal::SampleFormat::U16 => self.converted_stream::<u16>(&dev, &config, fill),
cpal::SampleFormat::U32 => self.converted_stream::<u32>(&dev, &config, fill),
cpal::SampleFormat::U64 => self.converted_stream::<u64>(&dev, &config, fill),
cpal::SampleFormat::F64 => self.converted_stream::<f64>(&dev, &config, fill),
other => return Err(format!("unsupported sample format {other:?}")),
}
.map_err(|e| e.to_string())?;
@@ -116,13 +263,14 @@ impl AudioOut for CpalOut {
self.channels as usize
}
fn played_frames(&self) -> Option<u64> {
self.stream.as_ref().map(|_| self.played.load(Ordering::SeqCst))
self.stream.as_ref().filter(|_| !self.failed.load(Ordering::SeqCst)).map(|_| self.played.load(Ordering::SeqCst))
}
fn devices(&self) -> AudioDevices {
let h = host(&self.hw.device_class);
let outputs = h.output_devices().map(|d| d.filter_map(|x| x.name().ok()).collect()).unwrap_or_default();
let inputs = h.input_devices().map(|d| d.filter_map(|x| x.name().ok()).collect()).unwrap_or_default();
let output_channels = output_device(&h, &self.hw.default_output).and_then(|d| self.config(&d).ok()).map(|c| c.channels()).unwrap_or(0);
let output_channels =
output_device(&h, &self.hw.default_output, &self.fallback).and_then(|c| self.config(&c.device).ok()).map(|cfg| cfg.channels()).unwrap_or(0);
AudioDevices { hosts: cpal::available_hosts().iter().map(|h| h.name().to_string()).collect(), inputs, outputs, output_channels }
}
fn configure(&mut self, hw: &AudioHardwarePrefs, document_rate: Option<u32>) {
@@ -130,11 +278,60 @@ impl AudioOut for CpalOut {
self.document_rate = document_rate;
// the rate playback mixes at must be known before `start`
let host = host(&self.hw.device_class);
if let Some(dev) = output_device(&host, &self.hw.default_output)
&& let Ok(cfg) = self.config(&dev)
if let Some(choice) = output_device(&host, &self.hw.default_output, &self.fallback)
&& let Ok(cfg) = self.config(&choice.device)
{
self.rate = cfg.sample_rate().0;
self.channels = cfg.channels();
self.note = choice.fallback;
}
}
fn note(&self) -> Option<String> {
self.note.as_ref().map(Note::text)
}
}
#[cfg(test)]
mod tests {
use super::*;
use cpal::Sample;
#[test]
fn fallback_prefers_sound_servers_over_raw_hardware() {
let mut names = vec!["plughw:CARD=PCH,DEV=0", "hw:CARD=PCH,DEV=0", "sysdefault:CARD=PCH", "pulse", "pipewire"];
names.sort_by_key(|n| fallback_rank(n));
assert_eq!(names, ["pipewire", "pulse", "sysdefault:CARD=PCH", "plughw:CARD=PCH,DEV=0", "hw:CARD=PCH,DEV=0"]);
}
#[test]
fn notes_name_the_fallback_device() {
assert!(Note::Fallback("pipewire".into()).text().contains("'pipewire'"));
assert!(!Note::NoDevice.text().is_empty());
}
#[test]
fn integer_conversion_clips_and_preserves_interleaved_channels() {
let mut signed = [0_i16; 8];
convert_samples(&[-2.0, 2.0, -0.5, 0.5, 0.0, f32::NAN, f32::INFINITY, f32::NEG_INFINITY], &mut signed);
assert_eq!(signed, [i16::MIN, i16::MAX, -16_384, 16_384, 0, 0, 0, 0]);
let mut unsigned = [0_u16; 3];
convert_samples(&[-1.0, 0.0, 1.0], &mut unsigned);
assert_eq!(unsigned, [0, 32_768, u16::MAX]);
}
#[test]
fn full_scale_24_bit_samples_remain_representable() {
let mut samples = [cpal::I24::EQUILIBRIUM; 3];
convert_samples(&[-1.0, 0.0, 1.0], &mut samples);
assert_eq!(samples.map(|s| s.inner()), [-8_388_608, 0, 8_388_607]);
}
#[test]
fn failed_stream_relinquishes_the_playback_clock() {
let failed = AtomicBool::new(false);
stream_error(&failed, cpal::StreamError::DeviceNotAvailable);
assert!(failed.load(Ordering::SeqCst));
let output = CpalOut::new();
assert_eq!(output.played_frames(), None);
}
}
+170 -56
View File
@@ -14,11 +14,13 @@ pub struct CpalIn {
host: String,
buf: Arc<Mutex<Vec<Vec<f32>>>>,
stop: Option<mpsc::Sender<()>>,
worker: Option<std::thread::JoinHandle<()>>,
error: Arc<Mutex<Option<String>>>,
}
impl CpalIn {
pub fn new(host: &str) -> Self {
Self { host: host.to_string(), buf: Arc::default(), stop: None }
Self { host: host.to_string(), buf: Arc::default(), stop: None, worker: None, error: Arc::default() }
}
}
@@ -46,7 +48,63 @@ fn lock<T>(m: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
m.lock().unwrap_or_else(|e| e.into_inner())
}
fn report_error(error: &Mutex<Option<String>>, message: String) {
let mut error = lock(error);
if error.is_none() {
eprintln!("filmcraft: audio input error: {message}");
*error = Some(message);
}
}
fn append<T: cpal::Sample>(data: &[T], channels: usize, buf: &Mutex<Vec<Vec<f32>>>, error: &Mutex<Option<String>>)
where
f32: cpal::FromSample<T>,
{
if lock(error).is_some() {
return;
}
let mut b = lock(buf);
if channels == 0 || channels > 256 || data.len() > 1_048_576 {
report_error(error, "invalid input device buffer".into());
return;
}
if b.len() != channels {
*b = vec![Vec::new(); channels];
}
// The recorder drains at stop. Bound accumulated samples and report exhaustion instead of
// allowing a long take or a broken/unpaced driver to exhaust the application heap.
let samples = b.first().map_or(0, Vec::len).saturating_mul(channels).saturating_add(data.len());
let frames = data.len() / channels;
if samples > 268_435_456 || b.iter_mut().any(|c| c.try_reserve(frames).is_err()) {
report_error(error, "voice-over input buffer is full; recording failed".into());
return;
}
for frame in data.chunks_exact(channels) {
for (channel, sample) in b.iter_mut().zip(frame) {
let sample = sample.to_sample::<f32>();
channel.push(if sample.is_finite() { sample.clamp(-1.0, 1.0) } else { 0.0 });
}
}
}
fn input_stream<T: cpal::SizedSample>(
dev: &cpal::Device,
config: &cpal::StreamConfig,
buf: Arc<Mutex<Vec<Vec<f32>>>>,
error: Arc<Mutex<Option<String>>>,
) -> Result<cpal::Stream, cpal::BuildStreamError>
where
f32: cpal::FromSample<T>,
{
let channels = usize::from(config.channels);
let stream_error = error.clone();
dev.build_input_stream(config, move |data: &[T], _| append(data, channels, &buf, &error), move |e| report_error(&stream_error, e.to_string()), None)
}
impl AudioInput for CpalIn {
fn configure_host(&mut self, name: &str) {
self.host = name.to_string();
}
fn devices(&self) -> Vec<String> {
host(&self.host).input_devices().map(|d| d.filter_map(|x| x.name().ok()).collect()).unwrap_or_default()
}
@@ -56,68 +114,77 @@ impl AudioInput for CpalIn {
fn start(&mut self, name: &str, sample_rate: u32) -> Result<InputFormat, String> {
self.stop();
lock(&self.buf).clear();
*lock(&self.error) = None;
let (host_name, name) = (self.host.clone(), name.to_string());
let buf = self.buf.clone();
let error = self.error.clone();
let (fmt_tx, fmt_rx) = mpsc::channel::<Result<InputFormat, String>>();
let (stop_tx, stop_rx) = mpsc::channel::<()>();
std::thread::Builder::new()
.name("voice-over input".into())
.spawn(move || {
let open = || -> Result<(cpal::Stream, InputFormat), String> {
let h = host(&host_name);
let dev = device(&h, &name).ok_or("no input device")?;
let default = dev.default_input_config().map_err(|e| e.to_string())?;
// the sequence rate when the device offers it in f32, else the device default
let cfg = dev
.supported_input_configs()
.ok()
.and_then(|mut it| {
it.find(|c| c.sample_format() == cpal::SampleFormat::F32 && (c.min_sample_rate().0..=c.max_sample_rate().0).contains(&sample_rate))
.and_then(|c| c.try_with_sample_rate(cpal::SampleRate(sample_rate)))
})
.unwrap_or(default);
if cfg.sample_format() != cpal::SampleFormat::F32 {
return Err(format!("unsupported input sample format {:?}", cfg.sample_format()));
}
let ch = cfg.channels() as usize;
let fmt = InputFormat { sample_rate: cfg.sample_rate().0, channels: cfg.channels() };
let stream = dev
.build_input_stream(
&cfg.into(),
move |data: &[f32], _| {
let mut b = lock(&buf);
if b.len() != ch {
*b = vec![Vec::new(); ch];
}
for frame in data.chunks_exact(ch.max(1)) {
for (c, x) in frame.iter().enumerate() {
b[c].push(*x);
}
}
},
|e| eprintln!("filmcraft: audio input error: {e}"),
None,
)
self.worker = Some(
std::thread::Builder::new()
.name("voice-over input".into())
.spawn(move || {
let open = || -> Result<(cpal::Stream, InputFormat), String> {
let h = host(&host_name);
let dev = device(&h, &name).ok_or("no input device")?;
let default = dev.default_input_config().map_err(|e| e.to_string())?;
// Prefer f32 at the sequence rate, retaining integer-only input devices.
let cfg = dev
.supported_input_configs()
.ok()
.and_then(|it| {
it.filter(|c| c.channels() > 0 && (c.min_sample_rate().0..=c.max_sample_rate().0).contains(&sample_rate))
.max_by_key(|c| (c.channels() == default.channels(), c.sample_format() == cpal::SampleFormat::F32))
.and_then(|c| c.try_with_sample_rate(cpal::SampleRate(sample_rate)))
})
.unwrap_or(default);
if cfg.channels() == 0 || cfg.channels() > 256 || cfg.sample_rate().0 == 0 {
return Err("the input device returned an invalid format".into());
}
let fmt = InputFormat { sample_rate: cfg.sample_rate().0, channels: cfg.channels() };
let format = cfg.sample_format();
let config = cfg.into();
let stream = match format {
cpal::SampleFormat::I8 => input_stream::<i8>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::I16 => input_stream::<i16>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::I24 => input_stream::<cpal::I24>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::I32 => input_stream::<i32>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::I64 => input_stream::<i64>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::U8 => input_stream::<u8>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::U16 => input_stream::<u16>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::U32 => input_stream::<u32>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::U64 => input_stream::<u64>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::F32 => input_stream::<f32>(&dev, &config, buf.clone(), error.clone()),
cpal::SampleFormat::F64 => input_stream::<f64>(&dev, &config, buf.clone(), error.clone()),
other => return Err(format!("unsupported input sample format {other:?}")),
}
.map_err(|e| e.to_string())?;
stream.play().map_err(|e| e.to_string())?;
Ok((stream, fmt))
};
match open() {
Ok((stream, fmt)) => {
let _ = fmt_tx.send(Ok(fmt));
// keep the stream alive until stop (or the sender is dropped)
let _ = stop_rx.recv();
drop(stream);
stream.play().map_err(|e| e.to_string())?;
Ok((stream, fmt))
};
let opened = std::panic::catch_unwind(std::panic::AssertUnwindSafe(open)).unwrap_or_else(|_| Err("audio input worker panicked".into()));
match opened {
Ok((stream, fmt)) => {
let _ = fmt_tx.send(Ok(fmt));
// keep the stream alive until stop (or the sender is dropped)
let _ = stop_rx.recv();
drop(stream);
}
Err(e) => {
let _ = fmt_tx.send(Err(e));
}
}
Err(e) => {
let _ = fmt_tx.send(Err(e));
}
}
})
.map_err(|e| e.to_string())?;
let fmt = fmt_rx.recv().map_err(|e| e.to_string())??;
})
.map_err(|e| e.to_string())?,
);
self.stop = Some(stop_tx);
Ok(fmt)
match fmt_rx.recv().map_err(|e| e.to_string()).and_then(|f| f) {
Ok(fmt) => Ok(fmt),
Err(e) => {
self.stop();
Err(e)
}
}
}
fn read(&mut self, frames: usize) -> Vec<Vec<f32>> {
let mut b = lock(&self.buf);
@@ -131,9 +198,56 @@ impl AudioInput for CpalIn {
fn discard(&mut self) {
lock(&self.buf).iter_mut().for_each(Vec::clear);
}
fn error(&self) -> Option<String> {
lock(&self.error).clone()
}
fn stop(&mut self) {
if let Some(tx) = self.stop.take() {
let _ = tx.send(());
}
if let Some(worker) = self.worker.take()
&& worker.join().is_err()
{
report_error(&self.error, "audio input worker panicked".into());
}
}
}
impl Drop for CpalIn {
fn drop(&mut self) {
self.stop();
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn signed_and_unsigned_input_is_deinterleaved_to_float() {
let buf = Mutex::new(Vec::new());
let error = Mutex::new(None);
append(&[i16::MIN, i16::MAX, 0, 16384], 2, &buf, &error);
let data = lock(&buf);
assert_eq!(data[0], [-1.0, 0.0]);
assert!((data[1][0] - 1.0).abs() < 0.0001);
assert_eq!(data[1][1], 0.5);
drop(data);
lock(&buf).clear();
append(&[0_u16, 32768, 65535, 32768], 2, &buf, &error);
let data = lock(&buf);
assert_eq!(data[1], [0.0, 0.0]);
assert_eq!(data[0][0], -1.0);
assert!(lock(&error).is_none());
}
#[test]
fn invalid_input_is_reported_without_panicking_or_appending() {
let buf = Mutex::new(Vec::new());
let error = Mutex::new(None);
append(&[0.0_f32], 0, &buf, &error);
assert!(lock(&error).is_some());
append(&[1.0_f32], 1, &buf, &error);
assert!(lock(&buf).is_empty());
}
}
+329
View File
@@ -0,0 +1,329 @@
//! Dev-time Linux desktop entry: the shell icon fallback.
//!
//! The Linux shell (Wayland *and* X11) resolves the window to a `.desktop` file by app id (set in
//! `main.rs` via `eframe`'s `with_app_id`) and takes the taskbar / window-list icon from its
//! `Icon=` entry in the hicolor theme. Packaging installs both
//! (`packaging/linux/ai.storyteller.filmcraft.desktop` and the hicolor PNGs) — but a source-tree
//! run has neither, so KDE/GNOME fall back to the generic Wayland icon. The fix is to ensure a
//! user-level entry for those runs: when no entry for the app id exists in any XDG data dir,
//! install one (plus the hicolor icon PNGs, read from this checkout's `assets/app-icon`, so
//! nothing is embedded in the binary) into `$XDG_DATA_HOME`/`~/.local/share`. An existing
//! packaged entry is never overridden; the user-owned dev copy is rewritten when it points at
//! another binary (a dev build moves between checkouts). The shell picks the icon up at the *next* app start;
//! `update-desktop-database`/`kbuildsycoca` is not needed for icon lookup.
//!
//! Opt-in only: nothing is written unless `FILMCRAFT_DEV_DESKTOP_ENTRY=1` is set, so ordinary
//! `cargo run`s (agents, worktrees, CI) never touch the user's profile. Packaged runs are
//! excluded even then — AppImage (`APPIMAGE`) mounts at a fresh `/tmp/.mount_*` each launch,
//! so its `Exec` path changes every time and it ships its own entry, and a Flatpak
//! (`FLATPAK_ID`) installs its entry through the manifest. `packaging/linux/install.sh` writes the packaged
//! `Exec=` the same two-layer way as here; keep the two in sync.
//!
//! Every failure is logged and skipped — a missing dev icon must never stop the app
//! ([`AGENTS.md`](/AGENTS.md) §0 "never crash").
/// Install the user-level desktop entry and icons for a source-tree run (Linux).
pub fn ensure_dev_desktop_entry() {
// Packaged and sandboxed runs never write into the user's profile: an AppImage mounts at a
// fresh /tmp/.mount_* every launch (the Exec path changes each time) and a Flatpak installs
// its own entry.
if std::env::var_os("APPIMAGE").is_some() || std::env::var_os("FLATPAK_ID").is_some() {
return;
}
// Opt-in only: a plain `cargo run` (agents, worktrees, other checkouts) must not write into
// the user's profile, and a packaged release never does (packages and install.sh install
// the real entry).
if std::env::var_os("FILMCRAFT_DEV_DESKTOP_ENTRY").is_none_or(|v| v != "1") {
return;
}
if let Err(e) = ensure() {
log::warn!("desktop entry (taskbar icon) not installed: {e}");
}
}
/// Install missing files; `Ok` whether anything was written or an entry already existed.
fn ensure() -> Result<(), DevIconError> {
let file_name = format!("{}.desktop", crate::APP_ID);
let user_dir = user_data_dir()?;
let exe = std::env::current_exe().map_err(DevIconError::Io)?;
// Not `to_string_lossy`: a substituted U+FFFD would install an `Exec` that can never start
// the app, and the same lossy string would match on every later run, so the broken entry
// would never be rewritten or warned about.
let exe = exe.to_str().ok_or(DevIconError::BadExecPath)?;
if exe.contains('=') || exe.contains('%') || exe.chars().any(|c| c.is_control()) {
// A desktop entry cannot carry these (install.sh rejects them for the same reason):
// '=' separates key and value, '%' introduces a field code, control characters are
// not valid in the string layer.
return Err(DevIconError::BadExecPath);
}
for dir in search_dirs(&user_dir) {
let candidate = dir.join("applications").join(&file_name);
if !candidate.is_file() {
continue;
}
// A packaged entry (any system dir) already gives the shell its icon; only our own
// user-level copy is kept fresh.
if dir != user_dir {
return Ok(());
}
let body = std::fs::read_to_string(&candidate).map_err(DevIconError::Io)?;
if desktop_file_exec_is_ours(&body, exe) {
return Ok(());
}
break; // a dev entry from another checkout: fall through and rewrite it
}
for size in ICON_SIZES {
let rel = std::path::Path::new("hicolor").join(format!("{size}x{size}")).join("apps").join(format!("{}.png", crate::APP_ID));
let dest = user_dir.join("icons").join(&rel);
if !dest.is_file()
&& let Some(parent) = dest.parent()
{
// The checkout this binary was built from; a missing icon only costs the icon.
let src = std::path::Path::new(ICON_DIR).join(&rel);
if !src.is_file() {
log::warn!("dev desktop entry: icon {} not found; skipping it", src.display());
continue;
}
std::fs::create_dir_all(parent).map_err(DevIconError::Io)?;
std::fs::copy(&src, &dest).map_err(DevIconError::Io)?;
}
}
let applications = user_dir.join("applications");
std::fs::create_dir_all(&applications).map_err(DevIconError::Io)?;
let body = desktop_file_content(crate::APP_ID, "FilmCraft", exe);
std::fs::write(applications.join(&file_name), body).map_err(DevIconError::Io)?;
Ok(())
}
/// The hicolor icon sizes installed for the taskbar (a subset large enough for panel and topbar;
/// the shell picks the nearest size, so 256 and 512 cover both).
const ICON_SIZES: [u16; 2] = [256, 512];
/// The app icons in the source tree this binary was built from (a path, not embedded bytes:
/// the entry is a dev-checkout convenience and release binaries must not carry the PNGs).
const ICON_DIR: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/../../assets/app-icon");
/// The user data dir we install into: `$XDG_DATA_HOME`, else `$HOME/.local/share` (XDG base
/// directory spec). Untrusted environment: an empty or relative `XDG_DATA_HOME` means the
/// default, and a missing or relative `HOME` is an error — never a guess that would write
/// into the working directory.
fn user_data_dir() -> Result<std::path::PathBuf, DevIconError> {
if let Some(v) = std::env::var_os("XDG_DATA_HOME")
&& !v.is_empty()
&& std::path::Path::new(&v).is_absolute()
{
return Ok(std::path::PathBuf::from(v));
}
match std::env::var_os("HOME").filter(|h| !h.is_empty() && std::path::Path::new(h).is_absolute()) {
Some(home) => Ok(std::path::PathBuf::from(home).join(".local").join("share")),
None => Err(DevIconError::NoHome),
}
}
/// The XDG data dirs to search for an existing entry, user dir first, then `$XDG_DATA_DIRS` (or
/// the spec default `/usr/local/share:/usr/share`). Only absolute paths are taken; the spec
/// requires them and a relative one would read the working directory.
fn search_dirs(user: &std::path::Path) -> Vec<std::path::PathBuf> {
let mut dirs = vec![user.to_path_buf()];
let system = std::env::var_os("XDG_DATA_DIRS")
.filter(|v| !v.is_empty())
.map(|v| v.to_string_lossy().into_owned())
.unwrap_or_else(|| "/usr/local/share:/usr/share".into());
for dir in system.split(':') {
if std::path::Path::new(dir).is_absolute() {
dirs.push(std::path::PathBuf::from(dir));
}
}
dirs
}
/// Whether the `.desktop` body's `Exec` points at our binary exactly (a stale dev entry from
/// another checkout — `/a/filmcraft-old` is not `/a/filmcraft` — does not, and is rewritten; a
/// packaged `Exec=filmcraft` never matches a source-tree path).
fn desktop_file_exec_is_ours(body: &str, exe: &str) -> bool {
exec_line(body).is_some_and(|arg| arg == exe)
}
/// The first `Exec=` argument (the executable), unescaped through both Desktop Entry layers:
/// first the string layer of the file value (`\\` `\s` `\n` `\t` `\r`, per
/// <https://specifications.freedesktop.org/desktop-entry/latest/>), then the shell layer of the
/// argument (double quotes, and `\` before `` ` `"` `\` `$` inside them). `None` when there is no
/// `Exec=` line or its first argument cannot be parsed.
fn exec_line(body: &str) -> Option<String> {
let value = body.lines().find(|l| l.starts_with("Exec=")).and_then(|l| l.strip_prefix("Exec="))?;
let value = unescape_string_value(value);
if let Some(rest) = value.strip_prefix('"') {
let mut out = String::new();
let mut chars = rest.chars();
while let Some(c) = chars.next() {
match c {
'"' => return Some(out),
'\\' => match chars.next() {
Some(e @ ('\\' | '"' | '`' | '$')) => out.push(e),
Some(e) => {
out.push('\\');
out.push(e);
}
None => return None, // a dangling escape ends the argument
},
c => out.push(c),
}
}
None // no closing quote
} else {
Some(value.split(' ').next().unwrap_or_default().to_string())
}
}
/// Undo the desktop entry's string layer: `\\` `\s` `\n` `\t` `\r` are the defined sequences; a
/// backslash before anything else is kept as the character itself (lenient, like desktop file
/// parsers in the wild).
fn unescape_string_value(value: &str) -> String {
let mut out = String::with_capacity(value.len());
let mut chars = value.chars();
while let Some(c) = chars.next() {
if c != '\\' {
out.push(c);
continue;
}
match chars.next() {
Some('\\') => out.push('\\'),
Some('s') => out.push(' '),
Some('n') => out.push('\n'),
Some('t') => out.push('\t'),
Some('r') => out.push('\r'),
Some(e) => out.push(e),
None => out.push('\\'),
}
}
out
}
/// The `.desktop` body: same fields as `packaging/linux/ai.storyteller.filmcraft.desktop`, with
/// `Exec` pointing at this exact binary (a dev build isn't on `$PATH`, so `TryExec` is left out)
/// and the icon by theme name.
fn desktop_file_content(app_id: &str, name: &str, exe: &str) -> String {
let mut out = String::new();
out.push_str("[Desktop Entry]\n");
out.push_str("Type=Application\n");
out.push_str(&format!("Name={name}\n"));
out.push_str("GenericName=Video Editor\n");
out.push_str("Comment=Edit video, color and sound\n");
out.push_str(&format!("Exec=\"{}\" %F\n", escape_exec_argument(exe)));
out.push_str(&format!("Icon={app_id}\n"));
out.push_str("Terminal=false\n");
out.push_str("StartupNotify=true\n");
out.push_str(&format!("StartupWMClass={app_id}\n"));
out.push_str("Categories=AudioVideo;Video;AudioVideoEditing;\n");
out.push_str("Keywords=video;editor;film;timeline;color;grading;nle;\n");
out
}
/// Escape `s` for the `Exec=` value's two layers, like `packaging/linux/install.sh`: first the
/// shell layer (`` ` `` `"` `$` `\` get a backslash, and the argument is double-quoted), then
/// the string layer of the file value (every backslash doubled).
fn escape_exec_argument(s: &str) -> String {
let mut shell = String::with_capacity(s.len() + 2);
for c in s.chars() {
if matches!(c, '`' | '"' | '$' | '\\') {
shell.push('\\');
}
shell.push(c);
}
shell.replace('\\', "\\\\")
}
/// Everything that can keep the dev icon out of the taskbar; logged, never fatal.
#[derive(Debug)]
enum DevIconError {
Io(std::io::Error),
NoHome,
BadExecPath,
}
impl std::fmt::Display for DevIconError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
DevIconError::Io(e) => write!(f, "{e}"),
DevIconError::NoHome => write!(f, "no home directory for the user data dir"),
DevIconError::BadExecPath => {
write!(f, "the binary path has a character a desktop entry cannot carry (=, % or a control character), or is not valid UTF-8")
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// The entry mirrors the packaged one, derived from the real file so the two cannot drift:
/// minus `TryExec` (a dev build isn't on `$PATH`), plus the quoted dev `Exec` path.
/// The icons the entry installs exist in the checkout.
#[test]
fn icon_sources_exist_in_the_checkout() {
for size in ICON_SIZES {
let p = std::path::Path::new(ICON_DIR).join(format!("hicolor/{size}x{size}/apps/ai.storyteller.filmcraft.png"));
assert!(p.is_file(), "{}", p.display());
}
}
#[test]
fn desktop_file_matches_the_packaged_fields() {
let packaged = include_str!("../../../packaging/linux/ai.storyteller.filmcraft.desktop");
let expected = packaged
.lines()
.filter(|l| !l.starts_with("TryExec="))
.map(|l| if l.starts_with("Exec=") { "Exec=\"/target/debug/filmcraft\" %F" } else { l })
.collect::<Vec<_>>()
.join("\n")
+ "\n";
let body = desktop_file_content("ai.storyteller.filmcraft", "FilmCraft", "/target/debug/filmcraft");
assert_eq!(body, expected);
}
/// Both layers are escaped, like install.sh: shell first (`` ` `` `"` `$` `\` get a
/// backslash), then *every* backslash doubled for the file value — including the escape
/// backslashes just added.
#[test]
fn exec_escapes_both_desktop_entry_layers() {
assert_eq!(escape_exec_argument("/opt/film craft/filmcraft"), "/opt/film craft/filmcraft");
// a\b → shell a\\b → string a\\\\b
assert_eq!(escape_exec_argument("a\\b"), "a\\\\\\\\b");
// a$b → shell a\$b → string a\\$b
assert_eq!(escape_exec_argument("a$b"), "a\\\\$b");
}
/// A written entry reads back as the exact binary it was written for, through both layers.
#[test]
fn exec_round_trips_through_both_layers() {
for exe in ["/home/dev/Projects/filmcraft/target/debug/filmcraft", "/opt/film craft/filmcraft", "/a$b\"c`d\\e"] {
let body = desktop_file_content("ai.storyteller.filmcraft", "FilmCraft", exe);
assert_eq!(exec_line(&body).as_deref(), Some(exe));
assert!(desktop_file_exec_is_ours(&body, exe));
}
}
/// Only the exact executable counts: another checkout's path, a path with our path as a
/// prefix, and a packaged `Exec=filmcraft` are all someone else's entry.
#[test]
fn desktop_file_exec_is_ours_matches_only_our_binary() {
let body = desktop_file_content("ai.storyteller.filmcraft", "FilmCraft", "/a/filmcraft");
assert!(!desktop_file_exec_is_ours(&body, "/a/filmcraft-old"));
assert!(!desktop_file_exec_is_ours(&body, "/b/filmcraft"));
assert!(!desktop_file_exec_is_ours("Exec=filmcraft %F\n", "/a/filmcraft"));
assert!(!desktop_file_exec_is_ours("Name=filmcraft\n", "/a/filmcraft"));
assert!(!desktop_file_exec_is_ours("Exec=\"dangling\n", "/a/filmcraft"));
}
/// The string layer is undone before the argument is split: `\s` becomes a space, which
/// ends a bare argument (the shell layer's quoting, not `\s`, is how a space travels), and
/// `\\` is one backslash. install.sh always quotes, so a space in our own `Exec` never
/// splits.
#[test]
fn exec_reads_the_string_layer() {
assert_eq!(exec_line("Exec=/a\\sb %F\n").as_deref(), Some("/a"));
assert_eq!(exec_line("Exec=\"/a\\sb\" %F\n").as_deref(), Some("/a b"));
assert_eq!(exec_line("Exec=/a\\\\b %F\n").as_deref(), Some("/a\\b"));
}
}
+70
View File
@@ -0,0 +1,70 @@
//! Extension filters for native file dialogs.
//!
//! Every filtered file dialog (import, relink, open file/preset, open/save project) builds
//! its filter with [`extensions`]. On Linux and the BSDs, rfd's XDG portal, GTK and Zenity
//! backends turn each extension into a case-sensitive `*.{ext}` glob, so camera files such
//! as `shot.MP4` or `shot.Mp4` would be hidden. The helper appends one bracket pattern per
//! extension (`mp4` -> `[mM][pP]4`) after the literal extensions, which stay first so save
//! dialogs keep their normal default suffix. macOS and Windows dialogs already ignore case
//! and receive the plain list.
pub fn extensions(exts: &[&str]) -> Vec<String> {
let extensions: Vec<_> = exts.iter().map(|ext| (*ext).to_owned()).collect();
// XDG portal/Zenity globs are case-sensitive. Keep the literal extensions first so
// Save dialogs can still infer a normal default suffix, then cover mixed case too.
// https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.FileChooser.html
#[cfg(not(any(target_os = "windows", target_os = "macos")))]
{
let mut extensions = extensions;
for ext in exts {
if !ext.bytes().any(|b| b.is_ascii_alphabetic()) {
continue;
}
let mut pattern = String::new();
for c in ext.chars() {
if c.is_ascii_alphabetic() {
pattern.extend(['[', c.to_ascii_lowercase(), c.to_ascii_uppercase(), ']']);
} else {
pattern.push(c);
}
}
extensions.push(pattern);
}
extensions
}
// Native macOS and Windows filters take ordinary extensions.
#[cfg(any(target_os = "windows", target_os = "macos"))]
extensions
}
#[cfg(test)]
mod tests {
use super::extensions;
// rfd's XDG portal and Zenity backends turn each extension into `*.{ext}`.
// Exercise those case-sensitive patterns against filenames, as the picker does.
#[cfg(not(any(target_os = "windows", target_os = "macos")))]
#[test]
fn camera_files_are_visible_in_every_capitalization() {
let filters: Vec<_> = extensions(&["mp4", "mov", "m2ts", "wav", "fcproj"]).iter().map(|ext| glob::Pattern::new(&format!("*.{ext}")).unwrap()).collect();
for name in ["camera.mp4", "camera.MP4", "camera.Mp4", "clip.MOV", "clip.moV", "take.M2TS", "sound.WaV", "Edit.FCPROJ", "Edit.FcProj"] {
assert!(filters.iter().any(|f| f.matches(name)), "the picker hides {name}");
}
for name in ["camera.mp4.bak", "camera.mp44", "camera.xmp4", "notes.txt", "mp4"] {
assert!(!filters.iter().any(|f| f.matches(name)), "the picker admits {name}");
}
}
#[test]
fn save_filters_keep_the_literal_default_extension() {
assert_eq!(extensions(&["mp4", "mov"]).first().map(String::as_str), Some("mp4"));
assert!(extensions(&[]).is_empty());
assert_eq!(extensions(&["*", "", "123"]), ["*", "", "123"]);
}
#[cfg(any(target_os = "windows", target_os = "macos"))]
#[test]
fn native_filters_receive_only_literal_extensions() {
assert_eq!(extensions(&["mp4", "mov", "fcproj"]), ["mp4", "mov", "fcproj"]);
}
}
+52
View File
@@ -0,0 +1,52 @@
//! Windows graphics instance configuration, applied before wgpu can initialize any driver.
use eframe::{NativeOptions, egui_wgpu::WgpuSetup, wgpu::Backends};
/// Restrict the wgpu instance to the primary backends (Vulkan, DirectX 12, Metal), i.e. leave
/// OpenGL out, unless `backend_override` (normally `Backends::from_env()`, i.e. `WGPU_BACKEND`)
/// names other backends.
pub fn configure(options: &mut NativeOptions, backend_override: Option<Backends>) {
if let WgpuSetup::CreateNew(create) = &mut options.wgpu_options.wgpu_setup {
// eframe's default includes GL. Creating that backend can crash inside AMD's
// atio6axx.dll before adapter selection or any Rust error handling runs, so the
// window never appears. Choosing an adapter afterwards is too late: only leaving GL
// out of the instance avoids it. Keep Vulkan and DX12 both: forcing DX12 alone sends
// shaders through FXC (which rejects fx.wgsl, silently falling back to the CPU) and
// leaves machines without DX12 with no backend at all.
create.instance_descriptor.backends = backend_override.unwrap_or(Backends::PRIMARY);
}
}
#[cfg(test)]
mod tests {
use super::*;
fn configured_backends(backend_override: Option<Backends>) -> Backends {
let mut options = NativeOptions::default();
configure(&mut options, backend_override);
let WgpuSetup::CreateNew(create) = options.wgpu_options.wgpu_setup else {
panic!("default native options must create a wgpu instance");
};
create.instance_descriptor.backends
}
#[test]
fn windows_default_leaves_out_gl() {
// eframe's own default (PRIMARY | GL) would initialize OpenGL as well.
let WgpuSetup::CreateNew(eframe_default) = NativeOptions::default().wgpu_options.wgpu_setup else {
panic!("default native options must create a wgpu instance");
};
assert!(eframe_default.instance_descriptor.backends.contains(Backends::GL));
let configured = configured_backends(None);
assert_eq!(configured, Backends::PRIMARY);
assert!(!configured.contains(Backends::GL));
assert!(configured.contains(Backends::DX12) && configured.contains(Backends::VULKAN));
}
#[test]
fn explicit_backend_override_is_preserved() {
for backends in [Backends::VULKAN, Backends::GL, Backends::DX12, Backends::VULKAN | Backends::DX12, Backends::empty()] {
assert_eq!(configured_backends(Some(backends)), backends);
}
}
}
+562
View File
@@ -0,0 +1,562 @@
//! The desktop app's logger: `log::` records go to standard error and to a log file.
//!
//! Without a logger every `log::warn!`/`log::error!` in the workspace (GPU compositor fallbacks,
//! audio output problems, a panicked frame job or audio mixer, container warnings) vanished. A
//! launch from a desktop menu has no terminal, so the file is what a bug report can attach:
//! `<data dir>/Logs/filmcraft.log`, next to the crash logs (`--data-dir`, `FILMCRAFT_DATA_DIR` or
//! `filmcraft_engine::autosave::default_data_dir`; Help ▸ Reveal Log Files opens the folder). Each
//! start moves the previous log to `filmcraft.1.log` (and that one to `.2`), so the log of a run
//! that crashed survives the next launch.
//!
//! Levels: `info` for FilmCraft's own crates, `warn` for everything else (wgpu and naga are
//! chatty). `RUST_LOG` replaces that with env_logger-style directives: `debug`,
//! `warn,filmcraft_codecs=trace`, `wgpu_core=info`. A directive ending in `*` matches every target
//! that starts with it (`filmcraft*=debug`).
//!
//! Records logged before the data directory is known are kept (up to [`MAX_PENDING`]) and written
//! once the file is attached. Writing never panics: a file that can't be created or written leaves
//! standard error as the only sink.
//!
//! Realtime audio: the cpal callbacks (`audio.rs`, `audio_in.rs`, the play-ahead reader) do not
//! log; they report through flags and a one-time `eprintln!`. A record takes the sink lock only for
//! one `write_all` of an already formatted line (standard error is written outside the lock), so a
//! logging thread such as the audio mixer never waits on more than one other line being written.
use std::fs::File;
use std::io::Write;
use std::path::{Path, PathBuf};
use std::sync::{Mutex, OnceLock, PoisonError};
use std::time::SystemTime;
use log::LevelFilter;
/// The current log file's name inside the log directory.
pub const LOG_FILE: &str = "filmcraft.log";
/// The log directory inside the data directory (shared with the crash logs and Reveal Log Files).
pub const LOG_DIR: &str = "Logs";
/// How many previous logs are kept (`filmcraft.1.log` … `filmcraft.<KEEP>.log`).
pub const KEEP: usize = 2;
/// The log file stops growing past this size (a runaway warning can't fill the disk).
pub const MAX_FILE_BYTES: u64 = 16 * 1024 * 1024;
/// Records kept in memory until the log file is attached.
pub const MAX_PENDING: usize = 512;
/// The built-in filter when `RUST_LOG` is unset or empty.
pub const DEFAULT_FILTER: &str = "warn,filmcraft*=info";
/// Per-target level filter parsed from env_logger-style directives.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Filter {
default: LevelFilter,
directives: Vec<(String, LevelFilter)>,
}
impl Filter {
/// Parse `spec`; unknown levels and empty parts are skipped. Without a bare level, targets no
/// directive names log errors only.
pub fn parse(spec: &str) -> Filter {
let mut f = Filter { default: LevelFilter::Error, directives: Vec::new() };
for part in spec.split(',').map(str::trim).filter(|p| !p.is_empty()) {
match part.split_once('=') {
Some((name, level)) => {
let name = name.trim();
if let (false, Some(level)) = (name.is_empty(), parse_level(level)) {
f.directives.push((name.to_owned(), level));
}
}
None => match parse_level(part) {
Some(level) => f.default = level,
// A bare target name: everything from it (env_logger does the same).
None => f.directives.push((part.to_owned(), LevelFilter::Trace)),
},
}
}
f
}
/// The level that applies to `target` (the most specific matching directive wins).
pub fn level_for(&self, target: &str) -> LevelFilter {
self.directives.iter().filter(|(name, _)| matches(name, target)).max_by_key(|(name, _)| name.len()).map_or(self.default, |(_, level)| *level)
}
/// The most verbose level any target can reach (for `log::set_max_level`).
pub fn max(&self) -> LevelFilter {
self.directives.iter().map(|(_, level)| *level).fold(self.default, Ord::max)
}
}
fn parse_level(s: &str) -> Option<LevelFilter> {
s.trim().parse().ok()
}
/// `name` matches `target` itself and its submodules (`a` matches `a` and `a::b`, not `ab`);
/// `name*` matches every target starting with `name`.
fn matches(name: &str, target: &str) -> bool {
match name.strip_suffix('*') {
Some(prefix) => target.starts_with(prefix),
None => target.strip_prefix(name).is_some_and(|rest| rest.is_empty() || rest.starts_with("::")),
}
}
/// `2026-10-08T07:59:17.728Z` (UTC, milliseconds). A clock before 1970 reads as the epoch.
pub fn timestamp(t: SystemTime) -> String {
let since = t.duration_since(std::time::UNIX_EPOCH).unwrap_or_default();
let secs = since.as_secs();
let (days, rem) = (secs / 86_400, secs % 86_400);
let (y, m, d) = civil_from_days(days);
format!("{y:04}-{m:02}-{d:02}T{:02}:{:02}:{:02}.{:03}Z", rem / 3600, rem % 3600 / 60, rem % 60, since.subsec_millis())
}
/// Days since 1970-01-01 to a proleptic Gregorian (year, month, day) (Howard Hinnant's
/// `civil_from_days`, unsigned because `timestamp` clamps at the epoch; saturating so an absurd
/// clock can't overflow).
fn civil_from_days(days: u64) -> (u64, u64, u64) {
let z = days.saturating_add(719_468);
let era = z / 146_097;
let doe = z % 146_097;
let yoe = (doe - doe / 1460 + doe / 36_524 - doe / 146_096) / 365;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let d = doy - (153 * mp + 2) / 5 + 1;
let m = if mp < 10 { mp + 3 } else { mp - 9 };
let y = era.saturating_mul(400).saturating_add(yoe).saturating_add(u64::from(m <= 2));
(y, m, d)
}
fn numbered(dir: &Path, n: usize) -> PathBuf {
if n == 0 { dir.join(LOG_FILE) } else { dir.join(format!("filmcraft.{n}.log")) }
}
/// Shift the previous logs up one (`.log` → `.1.log` → … → `.<KEEP>.log`, the oldest dropped)
/// and create a fresh, empty log file in `dir` (created if missing).
pub fn rotate(dir: &Path) -> std::io::Result<(PathBuf, File)> {
std::fs::create_dir_all(dir)?;
for n in (1..=KEEP).rev() {
match std::fs::rename(numbered(dir, n - 1), numbered(dir, n)) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => return Err(e),
_ => {}
}
}
let path = numbered(dir, 0);
let file = File::create(&path)?;
Ok((path, file))
}
/// One formatted log line: `<timestamp> <LEVEL> [<thread>] <target>: <message>\n`.
pub fn format_line(ts: &str, level: log::Level, thread: &str, target: &str, message: &str) -> String {
let message = message.strip_suffix('\n').unwrap_or(message);
format!("{ts} {level:<5} [{thread}] {target}: {message}\n")
}
/// Where formatted lines go besides standard error: kept in memory until a file is attached,
/// then appended to it until it reaches its size cap.
pub struct Sink {
file: Option<File>,
pending: Vec<String>,
dropped: usize,
written: u64,
max_bytes: u64,
capped: bool,
}
impl Sink {
pub fn new(max_bytes: u64) -> Sink {
Sink { file: None, pending: Vec::new(), dropped: 0, written: 0, max_bytes, capped: false }
}
/// Append `line` to the file, or keep it until [`Sink::attach`].
pub fn write(&mut self, line: &str) {
if self.file.is_none() {
if self.capped {
return;
}
if self.pending.len() < MAX_PENDING {
self.pending.push(line.to_owned());
} else {
self.dropped = self.dropped.saturating_add(1);
}
return;
}
self.append(line);
}
/// Start writing to `file`, first the lines kept so far.
pub fn attach(&mut self, file: File) {
self.file = Some(file);
for line in std::mem::take(&mut self.pending) {
self.append(&line);
}
if self.dropped > 0 {
let note = format!("{} earlier log lines were dropped before the log file was opened\n", self.dropped);
self.dropped = 0;
self.append(&note);
}
}
/// No file will come: drop the kept lines and keep none from now on.
pub fn close(&mut self) {
self.file = None;
self.pending = Vec::new();
self.capped = true;
}
fn append(&mut self, line: &str) {
if self.capped {
return;
}
let Some(file) = self.file.as_mut() else { return };
let len = u64::try_from(line.len()).unwrap_or(u64::MAX);
let text = if self.written.saturating_add(len) > self.max_bytes {
self.capped = true;
format!("log file reached {} bytes; later records go to standard error only\n", self.max_bytes)
} else {
self.written = self.written.saturating_add(len);
line.to_owned()
};
// A full disk or a vanished file must not take the app down: drop the file sink (and
// stop buffering, since no file will come).
if file.write_all(text.as_bytes()).is_err() {
self.file = None;
self.capped = true;
}
}
}
/// The installed logger: a [`Filter`], standard error and a [`Sink`].
pub struct AppLogger {
filter: Filter,
stderr: bool,
sink: Mutex<Sink>,
}
impl AppLogger {
pub fn new(filter: Filter, stderr: bool) -> AppLogger {
AppLogger { filter, stderr, sink: Mutex::new(Sink::new(MAX_FILE_BYTES)) }
}
/// Rotate the logs in `dir` and send records to the fresh file; returns its path.
/// An unusable directory leaves standard error as the only sink ([`AppLogger::stderr_only`]).
pub fn attach_dir(&self, dir: &Path) -> Result<PathBuf, String> {
match rotate(dir) {
Ok((path, file)) => {
self.sink.lock().unwrap_or_else(PoisonError::into_inner).attach(file);
Ok(path)
}
Err(e) => {
self.stderr_only();
Err(format!("{}: {e}", dir.display()))
}
}
}
/// No log file this run (no data directory, or it can't be used): stop keeping records for one.
pub fn stderr_only(&self) {
self.sink.lock().unwrap_or_else(PoisonError::into_inner).close();
}
}
impl log::Log for AppLogger {
fn enabled(&self, metadata: &log::Metadata<'_>) -> bool {
metadata.level() <= self.filter.level_for(metadata.target())
}
fn log(&self, record: &log::Record<'_>) {
if !self.enabled(record.metadata()) {
return;
}
// Format before taking the lock, so the lock covers one write only.
let thread = std::thread::current();
let line = format_line(&timestamp(SystemTime::now()), record.level(), thread.name().unwrap_or("?"), record.target(), &record.args().to_string());
if self.stderr {
let _ = std::io::stderr().write_all(line.as_bytes());
}
self.sink.lock().unwrap_or_else(PoisonError::into_inner).write(&line);
}
fn flush(&self) {}
}
/// Install the logger (filter from `RUST_LOG`, else [`DEFAULT_FILTER`]); `None` when another
/// logger was installed first. Call [`AppLogger::attach_dir`] once the data directory is known.
pub fn install() -> Option<&'static AppLogger> {
static LOGGER: OnceLock<AppLogger> = OnceLock::new();
let spec = std::env::var("RUST_LOG").ok().filter(|s| !s.trim().is_empty());
let filter = Filter::parse(spec.as_deref().unwrap_or(DEFAULT_FILTER));
let max = filter.max();
let logger = LOGGER.get_or_init(|| AppLogger::new(filter, true));
log::set_logger(logger).ok()?;
log::set_max_level(max);
Some(logger)
}
#[cfg(test)]
mod tests {
use super::*;
use log::Log;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::{Duration, UNIX_EPOCH};
fn temp_dir(tag: &str) -> PathBuf {
static N: AtomicUsize = AtomicUsize::new(0);
let d = std::env::temp_dir().join(format!("filmcraft-logging-{tag}-{}-{}", std::process::id(), N.fetch_add(1, Ordering::Relaxed)));
let _ = std::fs::remove_dir_all(&d);
d
}
fn read(path: &Path) -> String {
std::fs::read_to_string(path).expect("read log")
}
fn record(logger: &AppLogger, level: log::Level, target: &'static str, msg: &'static str) {
logger.log(&log::Record::builder().level(level).target(target).args(format_args!("{msg}")).build());
}
#[test]
fn the_default_filter_shows_filmcraft_info_and_other_crates_warnings() {
let f = Filter::parse(DEFAULT_FILTER);
assert_eq!(f.level_for("filmcraft"), LevelFilter::Info);
assert_eq!(f.level_for("filmcraft::audio"), LevelFilter::Info);
assert_eq!(f.level_for("filmcraft_ui_egui::play_ahead"), LevelFilter::Info);
assert_eq!(f.level_for("filmcraft_codecs::mxf"), LevelFilter::Info);
assert_eq!(f.level_for("wgpu_core::device"), LevelFilter::Warn);
assert_eq!(f.level_for("naga"), LevelFilter::Warn);
assert_eq!(f.max(), LevelFilter::Info);
}
#[test]
fn directives_follow_env_logger_and_the_most_specific_one_wins() {
let f = Filter::parse("info,wgpu_core=error,filmcraft_codecs=trace,filmcraft_codecs::mxf=off");
assert_eq!(f.level_for("eframe"), LevelFilter::Info);
assert_eq!(f.level_for("wgpu_core::instance"), LevelFilter::Error);
assert_eq!(f.level_for("filmcraft_codecs::ogg"), LevelFilter::Trace);
assert_eq!(f.level_for("filmcraft_codecs::mxf"), LevelFilter::Off);
assert_eq!(f.max(), LevelFilter::Trace);
// A module name is matched at `::` boundaries, not as a bare prefix.
assert_eq!(f.level_for("wgpu_core_extra"), LevelFilter::Info);
// A trailing `*` is a prefix.
assert_eq!(Filter::parse("warn,filmcraft_ui*=debug").level_for("filmcraft_ui_egui::frames"), LevelFilter::Debug);
assert_eq!(Filter::parse("warn,filmcraft_ui*=debug").level_for("filmcraft_engine"), LevelFilter::Warn);
// A bare target name sets that target to the most verbose level, as env_logger does.
assert_eq!(Filter::parse("naga").level_for("naga::front"), LevelFilter::Trace);
assert_eq!(Filter::parse("naga").level_for("eframe"), LevelFilter::Error);
// Levels are case-insensitive; the last bare level wins.
assert_eq!(Filter::parse("debug,WARN").level_for("x"), LevelFilter::Warn);
}
#[test]
fn hostile_specs_never_panic_and_fall_back_sensibly() {
let long = "a".repeat(100_000);
let specs = [
"",
",,,",
"=",
"==",
"=debug",
"filmcraft=",
"nonsense=loud",
"🦀=info",
"*",
"*=debug",
"a=b=c",
"warn,,filmcraft=DEBUG",
" \t ",
"\u{0}",
"é*=trace",
&long,
];
for spec in specs {
let f = Filter::parse(spec);
for target in ["filmcraft", "", "🦀", "é", "a::b"] {
let _ = f.level_for(target);
}
let _ = f.max();
}
assert_eq!(Filter::parse("warn,,filmcraft=DEBUG").level_for("filmcraft"), LevelFilter::Debug);
assert_eq!(Filter::parse("nonsense=loud").level_for("nonsense"), LevelFilter::Error);
assert_eq!(Filter::parse("=debug").max(), LevelFilter::Error, "a directive without a name is skipped");
assert_eq!(Filter::parse("*=debug").level_for("anything"), LevelFilter::Debug, "a lone `*` matches every target");
assert_eq!(Filter::parse("é*=trace").level_for("éa"), LevelFilter::Trace);
}
#[test]
fn timestamps_are_utc_with_milliseconds() {
assert_eq!(timestamp(UNIX_EPOCH), "1970-01-01T00:00:00.000Z");
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_millis(1_791_446_357_728)), "2026-10-08T07:59:17.728Z");
// Leap day, and the day after it.
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(1_709_164_800)), "2024-02-29T00:00:00.000Z");
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(1_709_251_199)), "2024-02-29T23:59:59.000Z");
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(1_709_251_200)), "2024-03-01T00:00:00.000Z");
// 2000 is a leap year (divisible by 400), 2100 is not.
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(951_782_400)), "2000-02-29T00:00:00.000Z");
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(4_107_542_400)), "2100-03-01T00:00:00.000Z");
// Before the epoch (a clock set wrong) clamps instead of panicking.
assert_eq!(timestamp(UNIX_EPOCH - Duration::from_secs(5)), "1970-01-01T00:00:00.000Z");
}
#[test]
fn rotation_keeps_the_previous_logs_and_starts_an_empty_file() {
let dir = temp_dir("rotate");
for run in 1..=4 {
let (path, _file) = rotate(&dir).expect("rotate");
assert_eq!(path, dir.join(LOG_FILE));
assert_eq!(read(&path), "");
std::fs::write(&path, format!("run {run}")).expect("write");
}
assert_eq!(read(&dir.join(LOG_FILE)), "run 4");
assert_eq!(read(&dir.join("filmcraft.1.log")), "run 3");
assert_eq!(read(&dir.join("filmcraft.2.log")), "run 2");
assert!(!dir.join("filmcraft.3.log").exists(), "only {KEEP} old logs are kept");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn rotation_into_an_unusable_directory_is_an_error_not_a_panic() {
let dir = temp_dir("blocked");
std::fs::create_dir_all(dir.parent().expect("parent")).expect("tmp");
std::fs::write(&dir, "a file where the directory should be").expect("block");
assert!(rotate(&dir).is_err());
assert!(rotate(&dir.join("below")).is_err());
let _ = std::fs::remove_file(&dir);
}
#[test]
fn a_line_carries_time_level_thread_target_and_message() {
assert_eq!(
format_line("2026-10-08T07:59:17.728Z", log::Level::Warn, "main", "filmcraft_ui_egui", "audio output unavailable: no output device"),
"2026-10-08T07:59:17.728Z WARN [main] filmcraft_ui_egui: audio output unavailable: no output device\n"
);
// A multi-line message keeps its lines; the record still ends in exactly one newline.
assert!(format_line("t", log::Level::Error, "w", "x", "a\nb\n").ends_with("x: a\nb\n"));
}
#[test]
fn lines_logged_before_the_file_exists_are_written_when_it_is_attached() {
let dir = temp_dir("pending");
let (path, file) = rotate(&dir).expect("rotate");
let mut sink = Sink::new(MAX_FILE_BYTES);
sink.write("early 1\n");
sink.write("early 2\n");
sink.attach(file);
sink.write("late\n");
assert_eq!(read(&path), "early 1\nearly 2\nlate\n");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn the_pending_buffer_is_bounded_and_says_how_much_it_dropped() {
let dir = temp_dir("bounded");
let (path, file) = rotate(&dir).expect("rotate");
let mut sink = Sink::new(MAX_FILE_BYTES);
for i in 0..MAX_PENDING + 5 {
sink.write(&format!("line {i}\n"));
}
sink.attach(file);
let text = read(&path);
assert_eq!(text.lines().filter(|l| l.starts_with("line ")).count(), MAX_PENDING);
assert_eq!(text.matches("5 earlier log lines were dropped").count(), 1, "{text}");
assert!(text.contains("line 0\n") && !text.contains(&format!("line {MAX_PENDING}\n")));
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn exactly_max_pending_lines_drop_nothing() {
let dir = temp_dir("exact");
let (path, file) = rotate(&dir).expect("rotate");
let mut sink = Sink::new(MAX_FILE_BYTES);
for i in 0..MAX_PENDING {
sink.write(&format!("line {i}\n"));
}
sink.attach(file);
let text = read(&path);
assert_eq!(text.lines().count(), MAX_PENDING);
assert!(!text.contains("dropped"), "{text}");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn the_file_stops_at_its_size_cap_with_one_note() {
let dir = temp_dir("cap");
let (path, file) = rotate(&dir).expect("rotate");
let mut sink = Sink::new(100);
sink.attach(file);
for i in 0..50 {
sink.write(&format!("line {i:02} with some padding\n"));
}
let text = read(&path);
assert_eq!(text.lines().filter(|l| l.starts_with("line ")).count(), 3, "{text}");
assert_eq!(text.matches("log file reached 100 bytes").count(), 1, "{text}");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn the_logger_filters_records_and_writes_them_to_the_attached_file() {
let dir = temp_dir("logger");
let logger = AppLogger::new(Filter::parse(DEFAULT_FILTER), false);
record(&logger, log::Level::Info, "filmcraft", "FilmCraft starting");
record(&logger, log::Level::Info, "wgpu_core::device", "too chatty");
let path = logger.attach_dir(&dir).expect("attach");
assert_eq!(path, dir.join(LOG_FILE));
record(&logger, log::Level::Warn, "wgpu_hal::vulkan", "a real warning");
record(&logger, log::Level::Debug, "filmcraft", "below info");
let text = read(&path);
let lines: Vec<&str> = text.lines().collect();
assert_eq!(lines.len(), 2, "{text}");
assert!(lines.first().is_some_and(|l| l.contains(" INFO [") && l.ends_with("filmcraft: FilmCraft starting")), "{text}");
assert!(lines.get(1).is_some_and(|l| l.ends_with("wgpu_hal::vulkan: a real warning")), "{text}");
assert!(logger.enabled(&log::Metadata::builder().level(log::Level::Info).target("filmcraft_engine::autosave").build()));
assert!(!logger.enabled(&log::Metadata::builder().level(log::Level::Info).target("naga").build()));
// A second launch rotates: the first log becomes `.1`.
let second = AppLogger::new(Filter::parse(DEFAULT_FILTER), false);
assert_eq!(second.attach_dir(&dir).expect("attach again"), dir.join(LOG_FILE));
assert_eq!(read(&dir.join("filmcraft.1.log")), text);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn records_from_many_threads_are_whole_lines() {
let dir = temp_dir("threads");
let logger = AppLogger::new(Filter::parse("info"), false);
let path = logger.attach_dir(&dir).expect("attach");
std::thread::scope(|s| {
for _ in 0..4 {
s.spawn(|| {
for _ in 0..100 {
record(&logger, log::Level::Info, "filmcraft", "from a worker");
}
});
}
});
let text = read(&path);
assert_eq!(text.lines().count(), 400);
assert!(text.lines().all(|l| l.ends_with("filmcraft: from a worker")), "{text}");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn attaching_to_an_unusable_directory_reports_an_error_and_keeps_logging() {
let dir = temp_dir("unusable");
std::fs::write(&dir, "not a directory").expect("block");
let logger = AppLogger::new(Filter::parse("info"), false);
let e = logger.attach_dir(&dir).expect_err("a file is not a log directory");
assert!(e.contains(&dir.display().to_string()), "{e}");
record(&logger, log::Level::Error, "x", "still fine");
let sink = logger.sink.lock().expect("sink");
assert!(sink.file.is_none() && sink.pending.is_empty(), "nothing is kept for a file that will not come");
drop(sink);
let _ = std::fs::remove_file(&dir);
}
#[test]
fn a_poisoned_lock_does_not_stop_logging() {
let dir = temp_dir("poison");
let logger = AppLogger::new(Filter::parse("info"), false);
let path = logger.attach_dir(&dir).expect("attach");
let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
let _guard = logger.sink.lock();
panic!("poison the sink");
}));
assert!(logger.sink.is_poisoned());
record(&logger, log::Level::Info, "filmcraft", "after the panic");
assert!(read(&path).contains("after the panic"));
let _ = std::fs::remove_dir_all(&dir);
}
}
+133 -18
View File
@@ -16,14 +16,24 @@
//! with unsaved changes the app asks to recover them; `--recover` recovers the newest without
//! asking, `--no-recover` starts without asking (the changes stay available via File ▸ Recover
//! Unsaved Changes…).
//!
//! `log` records go to standard error and `<data dir>/Logs/filmcraft.log` (`RUST_LOG` sets the
//! levels); see [`logging`].
#![cfg_attr(all(target_os = "windows", not(debug_assertions)), windows_subsystem = "windows")]
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
mod app_nap;
mod appearance;
mod args;
mod audio;
mod audio_in;
mod control_server;
#[cfg(target_os = "linux")]
mod dev_icon;
mod file_filters;
#[cfg(any(target_os = "windows", test))]
mod graphics;
mod logging;
#[cfg(target_os = "macos")]
mod native_menu;
mod window_raise;
@@ -66,6 +76,8 @@ fn tell(text: &str, error: bool) {
}
fn main() -> eframe::Result {
// First, so the panic hook and every start-up record are logged (`logging`).
let logger = logging::install();
// lossy, so a value that is not Unicode is reported like any other bad port instead of ignored
let env_port = std::env::var_os("FILMCRAFT_CONTROL_PORT").map(|p| p.to_string_lossy().into_owned());
let Launch { control_port, files, demo, startup_flag, recover, data_dir } = match args::parse(env_port, std::env::args().skip(1)) {
@@ -83,13 +95,30 @@ fn main() -> eframe::Result {
std::process::exit(2);
}
};
let log_dir = data_dir.clone().or_else(default_data_dir).map(|d| d.join(logging::LOG_DIR));
log::info!("FilmCraft {} ({} {})", env!("CARGO_PKG_VERSION"), std::env::consts::OS, std::env::consts::ARCH);
// The log file lives next to the crash logs; opened after the arguments, so `--help`,
// `--version` and usage errors leave no file behind. Records logged until now go in first.
if let Some(logger) = logger {
match log_dir.as_deref().map(|d| logger.attach_dir(d)) {
Some(Ok(path)) => log::info!("log file {}", path.display()),
Some(Err(e)) => log::warn!("no log file: {e}"),
None => {
logger.stderr_only();
log::warn!("no log file: no data directory (set FILMCRAFT_DATA_DIR or --data-dir)");
}
}
}
app_nap::disable();
#[cfg(target_os = "linux")]
dev_icon::ensure_dev_desktop_entry();
// Panics anywhere go to <data dir>/Logs/crash-<day>.log with a backtrace; the UI pass and
// frame workers catch them and keep running (see filmcraft_ui_egui::crash).
filmcraft_ui_egui::crash::install(data_dir.clone().or_else(default_data_dir).map(|d| d.join("Logs")));
filmcraft_ui_egui::crash::install(log_dir);
// OS hardware video decoders (VideoToolbox on macOS) in front of our own; Settings ▸ Playback ▸
// Hardware decoding switches them off. Unsupported streams and failures use our decoders.
register_hardware_decoders();
register_gpu_frame_renderer();
let options = eframe::NativeOptions {
viewport: egui::ViewportBuilder::default()
.with_title("FilmCraft")
@@ -107,6 +136,13 @@ fn main() -> eframe::Result {
event_loop_builder: agent_event_loop(control_port.is_some()),
..Default::default()
};
// Before eframe creates the wgpu instance: leave OpenGL out on Windows (see graphics.rs).
#[cfg(target_os = "windows")]
let options = {
let mut options = options;
graphics::configure(&mut options, eframe::wgpu::Backends::from_env());
options
};
let started = eframe::run_native(
"FilmCraft",
options,
@@ -116,7 +152,7 @@ fn main() -> eframe::Result {
let mut cfg = AutosaveConfig::new(dir);
cfg.local_offset = local_offset;
if let Err(e) = session.start_autosave(cfg) {
eprintln!("filmcraft: auto-save and crash recovery unavailable: {e}");
log::warn!("auto-save and crash recovery unavailable: {e}");
}
}
// voice-over recording reads the microphone through cpal
@@ -124,7 +160,7 @@ fn main() -> eframe::Result {
let project = files.iter().find(|f| f.ends_with(".fcproj")).cloned();
if let Some(p) = project {
if let Err(e) = session.execute("file.open", json!({"path": p})) {
eprintln!("filmcraft: {e}");
log::error!("{e}");
}
} else if !startup_flag && session.prefs.general.at_startup == "openMostRecent" {
// Settings ▸ General ▸ At Startup ▸ Open Most Recent
@@ -132,7 +168,7 @@ fn main() -> eframe::Result {
match recent {
Some(p) => {
if let Err(e) = session.execute("file.open", json!({"path": p})) {
eprintln!("filmcraft: {e}");
log::error!("{e}");
}
}
None => {
@@ -150,8 +186,8 @@ fn main() -> eframe::Result {
if recover == Some(true) && !session.recovery_candidates().is_empty() {
let id = session.recovery_candidates()[0].id.clone();
match session.execute("file.recover", json!({"id": id})) {
Ok(r) => eprintln!("filmcraft: recovered {r}"),
Err(e) => eprintln!("filmcraft: recovery failed: {e}"),
Ok(r) => log::info!("recovered {r}"),
Err(e) => log::error!("recovery failed: {e}"),
}
}
let mut app = FilmcraftApp::new(session);
@@ -164,29 +200,69 @@ fn main() -> eframe::Result {
{
app.set_wgpu(rs);
}
if let Some(out) = audio::CpalOut::new() {
// Settings ▸ Audio Hardware is applied on the first frame (`apply_prefs`)
app.audio = Some(Box::new(out));
}
// Keep device selection available even when the default device is unavailable.
// Settings ▸ Audio Hardware is applied on the first frame (`apply_prefs`).
app.audio = Some(Box::new(audio::CpalOut::new()));
app.hooks.pick_files = Some(Box::new(|exts: &[&str]| {
rfd::FileDialog::new().add_filter("Media", exts).pick_files().unwrap_or_default().into_iter().map(|p| p.to_string_lossy().to_string()).collect()
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t("Media"), &file_filters::extensions(exts))
.pick_files()
.unwrap_or_default()
.into_iter()
.map(|p| p.to_string_lossy().to_string())
.collect()
}));
// Link Media ▸ Locate…, Attach Proxies, Reconnect Full Resolution: one path, not imported.
app.hooks.pick_file_for_relink = Some(Box::new(|exts: &[&str], _hint| {
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t("Media"), &file_filters::extensions(exts))
.pick_file()
.map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_save = Some(Box::new(|name: &str| {
rfd::FileDialog::new().add_filter("FilmCraft Project", &["fcproj"]).set_file_name(name).save_file().map(|p| p.to_string_lossy().to_string())
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t("FilmCraft Project"), &file_filters::extensions(&["fcproj"]))
.set_file_name(name)
.save_file()
.map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_save_as = Some(Box::new(|filter: &str, exts: &[&str], name: &str| {
rfd::FileDialog::new().add_filter(filter, exts).set_file_name(name).save_file().map(|p| p.to_string_lossy().to_string())
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t(filter), &file_filters::extensions(exts))
.set_file_name(name)
.save_file()
.map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_folder = Some(Box::new(|| rfd::FileDialog::new().pick_folder().map(|p| p.to_string_lossy().to_string())));
app.hooks.pick_folder_at =
Some(Box::new(|dir: &str| rfd::FileDialog::new().set_directory(dir).pick_folder().map(|p| p.to_string_lossy().into_owned())));
app.hooks.open_path = Some(Box::new(open_path));
// Help menu, About and Discord links (#642): eframe is built without its `links`
// feature, so `ctx.open_url` does nothing on the desktop. Detached, so a launcher
// that waits for the browser never freezes the UI.
app.hooks.open_url = Some(Box::new(|url: &str| open::that_detached(url).map_err(|e| e.to_string())));
app.hooks.clipboard_media = Some(Box::new(clipboard_media));
// Settings ▸ General ▸ Interface Language ▸ System Language (#218).
app.hooks.cursor_screen_position = Some(Box::new(filmcraft_platform::cursor::cursor_screen_position));
app.hooks.system_languages = Some(Box::new(|| sys_locale::get_locales().collect()));
// Settings ▸ Appearance ▸ Appearance Mode ▸ Sync with system on Linux desktops whose
// compositor reports no theme to winit (no polling: see appearance.rs).
app.hooks.system_theme = appearance::service();
app.hooks.raise_without_focus = Some(Box::new(|| {
window_raise::raise_without_focus();
}));
app.hooks.pick_open_file = Some(Box::new(|filter: &str, exts: &[&str]| {
rfd::FileDialog::new().add_filter(filter, exts).pick_file().map(|p| p.to_string_lossy().to_string())
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t(filter), &file_filters::extensions(exts))
.pick_file()
.map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_open_project = Some(Box::new(|| {
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t("FilmCraft Project"), &file_filters::extensions(&["fcproj"]))
.pick_file()
.map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_open_project =
Some(Box::new(|| rfd::FileDialog::new().add_filter("FilmCraft Project", &["fcproj"]).pick_file().map(|p| p.to_string_lossy().to_string())));
#[cfg(target_os = "macos")]
{
let (rx, update) = native_menu::install(&app, cc.egui_ctx.clone());
@@ -205,13 +281,25 @@ fn main() -> eframe::Result {
// instead of exiting silently, and keep the reason in the crash log.
if let Err(e) = &started {
let msg = format!("FilmCraft could not start its window: {e}\n\nUpdating the graphics driver usually fixes this.");
eprintln!("filmcraft: {msg}");
log::error!("{msg}");
filmcraft_ui_egui::crash::record(&msg);
let _ = rfd::MessageDialog::new().set_title("FilmCraft").set_description(&msg).set_level(rfd::MessageLevel::Error).show();
}
started
}
/// Files copied in the file manager, else an image, on the system clipboard (Paste, #611).
fn clipboard_media() -> Option<filmcraft_ui_egui::ClipboardMedia> {
let mut clipboard = arboard::Clipboard::new().ok()?;
if let Ok(files) = clipboard.get().file_list()
&& !files.is_empty()
{
return Some(filmcraft_ui_egui::ClipboardMedia::Files(files.iter().map(|p| p.to_string_lossy().into_owned()).collect()));
}
let image = clipboard.get_image().ok()?;
Some(filmcraft_ui_egui::ClipboardMedia::Image { width: image.width, height: image.height, rgba: image.bytes.into_owned() })
}
/// Open a file in its default application, or reveal it in the file manager (Edit Original,
/// Reveal Log Files).
fn open_path(path: &str, reveal: bool) -> Result<(), String> {
@@ -239,6 +327,30 @@ fn open_path(path: &str, reveal: bool) -> Result<(), String> {
cmd.spawn().map(|_| ()).map_err(|e| format!("can't open {path}: {e}"))
}
/// The GPU export frame renderer (filmcraft-gpu's off-screen compositor behind filmcraft-export's
/// frame-renderer hook): exports with GPU rendering Auto composite on the GPU and fall back to the
/// CPU reference renderer wherever it cannot.
struct GpuFrameRenderer(filmcraft_gpu::ExportRenderer);
impl filmcraft_export::FrameRenderer for GpuFrameRenderer {
fn render(
&mut self,
project: &filmcraft_project::Project,
seq: filmcraft_project::ItemId,
t: filmcraft_time::Tick,
opts: filmcraft_render::RenderOptions,
sources: &dyn filmcraft_render::SourceProvider,
) -> Option<filmcraft_render::Image> {
self.0.render(project, seq, t, opts, sources)
}
}
fn register_gpu_frame_renderer() {
filmcraft_export::register_frame_renderer(|| {
filmcraft_gpu::ExportRenderer::new().map(|r| Box::new(GpuFrameRenderer(r)) as Box<dyn filmcraft_export::FrameRenderer>)
});
}
/// Put the OS hardware video decoders in front of our own. Registered in a statement of its own:
/// a log macro does not evaluate its arguments while no logger takes its level, which left the
/// hardware decoders out of every run.
@@ -275,6 +387,9 @@ mod tests {
fn startup_registers_the_hardware_decoders_without_a_logger() {
assert!(!log::log_enabled!(log::Level::Info));
let hardware = super::register_hardware_decoders();
assert_eq!(filmcraft_platform::registered(), cfg!(any(target_os = "macos", target_os = "windows")), "{hardware:?}");
// always on macOS and Windows; on Linux when a VA-API driver or NVIDIA's driver (NVDEC) is there
let expected = cfg!(any(target_os = "macos", target_os = "windows"))
|| (cfg!(target_os = "linux") && matches!(hardware, filmcraft_platform::Availability::Available(_)));
assert_eq!(filmcraft_platform::registered(), expected, "{hardware:?}");
}
}
+10
View File
@@ -99,6 +99,16 @@ pub fn install(app: &FilmcraftApp, ctx: egui::Context) -> (Receiver<String>, Sho
filmcraft_ui_egui::i18n::Language::Es
} else if checked("app.language.portuguese") {
filmcraft_ui_egui::i18n::Language::PtBr
} else if checked("app.language.ukrainian") {
filmcraft_ui_egui::i18n::Language::Uk
} else if checked("app.language.chinese") {
filmcraft_ui_egui::i18n::Language::ZhCn
} else if checked("app.language.german") {
filmcraft_ui_egui::i18n::Language::De
} else if checked("app.language.russian") {
filmcraft_ui_egui::i18n::Language::Ru
} else if checked("app.language.french") {
filmcraft_ui_egui::i18n::Language::Fr
} else {
filmcraft_ui_egui::i18n::Language::En
};
File diff suppressed because it is too large. Load diff
+76
View File
@@ -0,0 +1,76 @@
# Contributor names for the About window (craftrules standards/contributors.md).
#
# Everyone who contributes is credited by GitHub username automatically. Anything more is up to
# you: add YOUR OWN entry below, in your own PR (commit it yourself). Never add someone else; a
# maintainer may add a person only when that person asked, recording where in consent_source.
#
# Easiest: python3 ../../craftrules/scripts/contributors.py --add-me . \
# [--real-name "Ada Lovelace"] [--display-name "Ada"] [--sync-github-name]
#
# Or copy this block, replacing your-github-username (every field is optional):
#
# [people.your-github-username]
# real_name = "Your Name" # shown when the viewer picks "Real name"
# display_name = "Nickname" # shown when the viewer picks "Display name"
# sync_github_name = true # or: use the name on your public GitHub profile as display name
#
# To remove your names, delete your entry (or set fields empty) in a PR. The next
# contributors.py run, and the next build, drop them.
[people.echelon]
real_name = "Brandon Thomas"
display_name = "Brandon Thomas"
sync_github_name = false
[people.ahmedhosnypro]
real_name = "Ahmed Hosny"
display_name = "Ahmed Hosny"
sync_github_name = false
[people.sanyamsingh001]
display_name = "sanyamsingh001"
sync_github_name = false
[people.switch87]
real_name = "Gert Pellin"
display_name = "Gert Pellin"
sync_github_name = false
[people.swillunderscore]
display_name = "Swill Software"
[people.IppSec]
sync_github_name = true
[people.ItzBloxy]
display_name = "Bloxy"
[people.Hathor875]
real_name = "Krzysztof Cieślik"
sync_github_name = true
[people.ReReddo]
display_name = "ReReddo"
sync_github_name = false
[people.liangforstudy]
sync_github_name = true
[people.keller-lukas]
real_name = "Lukas Keller"
display_name = "Lukas Keller"
sync_github_name = false
[people.maxm11]
real_name = "Max Murphy"
display_name = "Max Murphy"
sync_github_name = false
[people.ossini]
sync_github_name = true
[people.vangel76]
real_name = "Valentin Klein"
display_name = "Valentin Klein"
sync_github_name = false
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "filmcraft-ac3"
description = "Clean-room AC-3 (Dolby Digital, ATSC A/52) audio decoder"
description = "Clean-room AC-3 and E-AC-3 (Dolby Digital / Dolby Digital Plus, ATSC A/52) audio decoder"
version.workspace = true
edition.workspace = true
license.workspace = true
+42 -6
View File
@@ -1,6 +1,6 @@
# filmcraft-ac3
Clean-room AC-3 (Dolby Digital) audio decoder in pure Rust. Layer L0: depends only on
Clean-room AC-3 (Dolby Digital) and E-AC-3 (Dolby Digital Plus) audio decoder in pure Rust. Layer L0: depends only on
`thiserror`. Builds for `wasm32-unknown-unknown`; no `unsafe`. No GPL/LGPL code (FFmpeg,
liba52) was consulted; FFmpeg is used only as an external fixture generator and test oracle.
@@ -11,14 +11,17 @@ publicly available standard: §5 bit stream syntax and semantics, §6 decoding o
exponents, §7.2 parametric bit allocation (Tables 7.6-7.16 extracted mechanically from the
standard's text), §7.3 mantissas and dither, §7.4 coupling, §7.5 rematrixing, §7.7.1 dynamic
range, §7.9 transforms and block switching. The window is computed as the Kaiser-Bessel derived
window (α = 5) and checked against Table 7.33 to 6·10⁻⁶.
window (α = 5) and checked against Table 7.33 to 6·10⁻⁶. E-AC-3: Annex E §E2 bit stream syntax
(syncinfo, bsi, audfrm, audblk; Tables E2.1-E2.13, frame exponent strategies from Table E2.10),
§E3.3 modified parameters, §E3.4.2 AHT helper variables (to detect AHT), §E3.6 spectral extension
(attenuation Table E3.14, which is 2^(-(code + 1)(bin + 1) / 15), checked against the table).
## API
```rust
let mut dec = filmcraft_ac3::Decoder::new();
let h = filmcraft_ac3::parse_header(frame)?; // rate, channels, frame size
let out = dec.decode(frame)?; // 1536 samples per channel, WAV channel order
let out = dec.decode(frame)?; // h.samples() per channel, WAV channel order
dec.reset(); // before decoding from another position
```
@@ -26,7 +29,7 @@ dec.reset(); // before decoding from another
- Every audio coding mode (1+1, 1/0, 2/0, 3/0, 2/1, 3/1, 2/2, 3/2) with or without LFE; 32 /
44.1 / 48 kHz and every frame size; bsid ≤ 8 (9 and 10, the half / quarter-rate variants, are
accepted). E-AC-3 (bsid 11-16) is refused with `Error::Unsupported`.
accepted).
- D15 / D25 / D45 exponents with reuse across blocks; the parametric bit allocation in the
standard's fixed-point steps, delta bit allocation, the all-zero SNR offset special case;
grouped 3-, 5- and 11-level mantissas (groups shared across exponent sets), 7- and 15-level
@@ -43,6 +46,26 @@ dec.reset(); // before decoding from another
- Errors (bad exponents, reserved codes, blocks running past the frame) return `Err`; garbage
never panics.
## E-AC-3 (bsid 11-16)
- Independent substream 0 is decoded (stream types 0 and 2); dependent substreams (the channels
beyond 5.1 of a 7.1 program) and further independent programs decode to no channels, so a 7.1
stream plays as its 5.1 core. `Header::is_primary` tells them apart.
- 1, 2, 3 or 6 blocks per syncframe (`Header::samples`), 32 / 44.1 / 48 kHz and the reduced rates
16 / 22.05 / 24 kHz, every acmod with or without LFE.
- The whole bsi (mixing and informational metadata, custom channel maps, addbsi) is parsed and
skipped; audfrm: per-block or frame-based exponent strategies (exponents may be reused across
syncframes of fewer than six blocks; the decoder keeps them until `reset`), the syntax-enable
flags and their defaults, SNR offset strategies 1-3, converter fields, block start information.
- Audio blocks share the AC-3 path: coupling with the default or transmitted band structure
(`firstcplcos` / `firstcplleak` states), rematrixing with the Annex E band count, end mantissas
derived from spectral extension.
- Spectral extension: band structure, coordinates, blending factors, translation with wrap,
banded RMS, the border / wrap-point notch filter, noise blending and scaling (§E3.6.4).
- Refused with `Error::Unsupported`: the adaptive hybrid transform (when a channel uses it; `ahte`
alone is fine) and enhanced coupling. Transient pre-noise processing data is read and not
applied.
## Accuracy (vs FFmpeg as an external oracle)
AC-3 output is not bit-exact across decoders: zero-bit mantissas are filled with
@@ -56,10 +79,17 @@ FFmpeg's decode (worst channel SNR):
| 1/0 32 kHz 96 kb/s | 94.1 dB | 95.7 dB |
| 3/2 + LFE 48 kHz 448 kb/s | 81.4 dB | 90.2 dB |
| 2/0 pink noise 128 kb/s (many zero-bit mantissas) | 27.7 dB | 27.9 dB |
| E-AC-3 2/0 48 kHz 192 kb/s (tones, chirp) | 64.8 dB | 74.7 dB |
| E-AC-3 2/0 44.1 kHz 64 kb/s with coupling and rematrixing | 65.5 dB | 66.1 dB |
| E-AC-3 1/0 32 kHz 64 kb/s | 91.0 dB | 90.1 dB |
| E-AC-3 3/2 + LFE 48 kHz 384 kb/s with mixing / informational metadata | 81.6 dB | 85.3 dB |
| E-AC-3 2/0 pink noise 96 kb/s | 48.6 dB | 47.9 dB |
The noise row shows the difference is dither: two of our own decodes with different dither
sequences differ as much as we differ from FFmpeg. In MPEG-2 TS (AVCHD-style `.mts`) and VOB the
maximum sample difference to FFmpeg is 2.3·10⁻³.
maximum sample difference to FFmpeg is 2.3·10⁻³. E-AC-3 through `filmcraft-codecs`
(`tests/eac3_oracle.rs`): 5.1 in MP4 and Matroska 91.3 dB, stereo in QuickTime 87.0 dB and
MPEG-TS 79.5 dB against FFmpeg's decode.
## Gaps
@@ -67,4 +97,10 @@ maximum sample difference to FFmpeg is 2.3·10⁻³.
§7.9.4.2 but not checked against an oracle.
- No downmixing, no heavy compression (`compr`) or dialogue normalisation (both are metadata
for playback systems), no CRC check (corrupt frames are caught by consistency checks).
- E-AC-3 is not decoded.
- FFmpeg's E-AC-3 encoder writes six-block syncframes with frame-based exponent strategies,
coupling with the default band structure and none of the optional per-block syntax. Spectral
extension (no free encoder writes it), fewer blocks per syncframe, the reduced rates and the
per-block syntax (shared with AC-3) are implemented from the standard and covered by unit tests
on synthetic data, not by an oracle.
- E-AC-3: no adaptive hybrid transform, enhanced coupling, transient pre-noise processing, or
channels from dependent substreams (7.1 plays as 5.1).
+634
View File
@@ -0,0 +1,634 @@
//! E-AC-3 (ATSC A/52:2012 Annex E): the syncframe header and bit stream information (§E2.2.2),
//! the audio frame (§E2.2.3) and spectral extension (§E2.2.4, §E3.6). The audio blocks share the
//! AC-3 decoder (`Decoder::audio_block`), which reads the E-AC-3 differences from [`FrameInfo`].
use crate::{Bits, Error, Header, NFCHANS, Result};
/// Sample rates by fscod, and by fscod2 when fscod is 3 (Tables E2.2, E2.3).
const RATES: [u32; 3] = [48_000, 44_100, 32_000];
const REDUCED_RATES: [u32; 3] = [24_000, 22_050, 16_000];
/// Audio blocks per syncframe by numblkscod (Table E2.4).
const BLOCKS: [u8; 4] = [1, 2, 3, 6];
/// Exponent strategies of the six blocks for each frame-based strategy code (Table E2.10):
/// 0 reuse, 1 D15, 2 D25, 3 D45.
const FRAME_EXPSTR: [[u8; 6]; 32] = [
[1, 0, 0, 0, 0, 0],
[1, 0, 0, 0, 0, 3],
[1, 0, 0, 0, 2, 0],
[1, 0, 0, 0, 3, 3],
[2, 0, 0, 2, 0, 0],
[2, 0, 0, 2, 0, 3],
[2, 0, 0, 3, 2, 0],
[2, 0, 0, 3, 3, 3],
[2, 0, 1, 0, 0, 0],
[2, 0, 2, 0, 0, 3],
[2, 0, 2, 0, 2, 0],
[2, 0, 2, 0, 3, 3],
[2, 0, 3, 2, 0, 0],
[2, 0, 3, 2, 0, 3],
[2, 0, 3, 3, 2, 0],
[2, 0, 3, 3, 3, 3],
[3, 1, 0, 0, 0, 0],
[3, 1, 0, 0, 0, 3],
[3, 2, 0, 0, 2, 0],
[3, 2, 0, 0, 3, 3],
[3, 2, 0, 2, 0, 0],
[3, 2, 0, 2, 0, 3],
[3, 2, 0, 3, 2, 0],
[3, 2, 0, 3, 3, 3],
[3, 3, 1, 0, 0, 0],
[3, 3, 2, 0, 0, 3],
[3, 3, 2, 0, 2, 0],
[3, 3, 2, 0, 3, 3],
[3, 3, 3, 2, 0, 0],
[3, 3, 3, 2, 0, 3],
[3, 3, 3, 3, 2, 0],
[3, 3, 3, 3, 3, 3],
];
/// Default coupling band structure by coupling sub-band (Table E2.12, "Default Coupling Banding
/// Structure"; sub-band 0 never starts a combined band).
const DEFAULT_CPLBNDSTRC: [bool; 18] = {
let mut t = [false; 18];
let ones = [8, 10, 11, 13, 14, 15, 16, 17];
let mut i = 0;
while i < ones.len() {
t[ones[i]] = true;
i += 1;
}
t
};
/// Default spectral extension band structure by sub-band (Table E2.11).
const DEFAULT_SPXBNDSTRC: [bool; 17] = {
let mut t = [false; 17];
let mut i = 8;
while i < 17 {
t[i] = true;
i += 2;
}
t
};
/// First transform coefficient of spectral extension sub-band `k` (Table E3.13, k ≤ 17).
const fn spx_band_bin(k: usize) -> usize {
25 + 12 * k
}
/// Spectral extension attenuation (Table E3.14): spxattentab[code][binindex] = 2^(-(code + 1) ·
/// (binindex + 1) / 15), which the table lists to nine decimals.
fn spx_atten(code: usize, binindex: usize) -> f32 {
let e = (code as f64 + 1.0) * (binindex as f64 + 1.0) / 15.0;
(-e).exp2() as f32
}
/// Parse the header of an E-AC-3 syncframe (bsid 11-16) at the start of `b` (at least 7 bytes).
pub(crate) fn parse_header(b: &[u8], bsid: u8) -> Result<Header> {
let mut r = Bits::new(b.get(2..).unwrap_or_default());
let strmtyp = r.read(2) as u8;
let substreamid = r.read(3) as u8;
let words = r.read(11) as usize + 1;
let fscod = r.read(2) as usize;
let (sample_rate, blocks) = match RATES.get(fscod) {
Some(&rate) => (rate, BLOCKS[r.read(2) as usize & 3]),
None => (*REDUCED_RATES.get(r.read(2) as usize).ok_or(Error::Invalid("reserved fscod2"))?, 6),
};
let acmod = r.read(3) as u8;
let lfeon = r.bit();
if strmtyp == 3 {
return Err(Error::Unsupported("reserved E-AC-3 stream type 3".into()));
}
let frame_bytes = words * 2;
if frame_bytes < 8 {
return Err(Error::Invalid("E-AC-3 syncframe too short"));
}
// frame bits per second of audio
let bits = frame_bytes as u64 * 8 * u64::from(sample_rate) / (256 * u64::from(blocks));
let bitrate_kbps = u32::try_from(bits / 1000).unwrap_or(u32::MAX);
Ok(Header { sample_rate, bitrate_kbps, frame_bytes, bsid, acmod, lfeon, blocks, strmtyp, substreamid })
}
/// What the audio frame and blocks need from the bit stream information.
pub(crate) struct Bsi {
pub strmtyp: u8,
/// Sample rate code for the bit allocation's hearing threshold: fscod, or fscod2 at the
/// reduced rates.
pub fscod: usize,
pub numblkscod: usize,
}
/// Read syncinfo and bsi (§E2.2.1-2), leaving `r` at the audio frame.
pub(crate) fn read_bsi(r: &mut Bits) -> Result<Bsi> {
r.skip(16); // syncword
let strmtyp = r.read(2) as u8;
r.skip(3 + 11); // substreamid, frmsiz
let fscod = r.read(2) as usize;
let (rate_code, numblkscod) = if fscod == 3 {
let fscod2 = r.read(2) as usize;
if fscod2 == 3 {
return Err(Error::Invalid("reserved fscod2"));
}
(fscod2, 3)
} else {
(fscod, r.read(2) as usize)
};
let nblocks = BLOCKS[numblkscod & 3] as usize;
let acmod = r.read(3) as usize;
let lfeon = r.bit();
r.skip(5 + 5); // bsid, dialnorm
if r.bit() {
r.skip(8); // compr
}
if acmod == 0 {
r.skip(5); // dialnorm2
if r.bit() {
r.skip(8); // compr2
}
}
if strmtyp == 1 && r.bit() {
r.skip(16); // chanmap
}
if r.bit() {
// mixing metadata
if acmod > 2 {
r.skip(2); // dmixmod
}
if acmod & 1 != 0 && acmod > 2 {
r.skip(6); // ltrtcmixlev, lorocmixlev
}
if acmod & 4 != 0 {
r.skip(6); // ltrtsurmixlev, lorosurmixlev
}
if lfeon && r.bit() {
r.skip(5); // lfemixlevcod
}
if strmtyp == 0 {
if r.bit() {
r.skip(6); // pgmscl
}
if acmod == 0 && r.bit() {
r.skip(6); // pgmscl2
}
if r.bit() {
r.skip(6); // extpgmscl
}
match r.read(2) {
1 => r.skip(5), // premixcmpsel, drcsrc, premixcmpscl
2 => r.skip(12),
// mixdeflen: the mixdata field (through mixdatafill) is mixdeflen + 2 bytes
3 => {
let len = r.read(5) as usize;
r.skip(8 * (len + 2));
}
_ => {}
}
if acmod < 2 {
if r.bit() {
r.skip(14); // panmean, paninfo
}
if acmod == 0 && r.bit() {
r.skip(14); // panmean2, paninfo2
}
}
if r.bit() {
// frame mixing configuration information
if numblkscod == 0 {
r.skip(5);
} else {
for _ in 0..nblocks {
if r.bit() {
r.skip(5);
}
}
}
}
}
}
if r.bit() {
// informational metadata
r.skip(5); // bsmod, copyrightb, origbs
if acmod == 2 {
r.skip(4); // dsurmod, dheadphonmod
}
if acmod >= 6 {
r.skip(2); // dsurexmod
}
if r.bit() {
r.skip(8); // mixlevel, roomtyp, adconvtyp
}
if acmod == 0 && r.bit() {
r.skip(8); // mixlevel2, roomtyp2, adconvtyp2
}
if fscod < 3 {
r.skip(1); // sourcefscod
}
}
if strmtyp == 0 && numblkscod != 3 {
r.skip(1); // convsync
}
if strmtyp == 2 && (numblkscod == 3 || r.bit()) {
r.skip(6); // blkid → frmsizecod
}
if r.bit() {
let l = r.read(6) as usize;
r.skip((l + 1) * 8); // addbsi
}
Ok(Bsi { strmtyp, fscod: rate_code, numblkscod })
}
/// The audio frame (§E2.2.3) and the state that runs from block to block within a syncframe.
pub(crate) struct FrameInfo {
pub strmtyp: u8,
pub blkswe: bool,
pub dithflage: bool,
pub bamode: bool,
pub frmfgaincode: bool,
pub dbaflde: bool,
pub skipflde: bool,
pub snroffststr: u8,
pub frmcsnroffst: i32,
pub frmfsnroffst: i32,
pub cplstre: [bool; 6],
pub cplinu: [bool; 6],
pub cplexpstr: [u8; 6],
pub chexpstr: [[u8; 5]; 6],
pub lfeexpstr: [u8; 6],
/// Spectral extension attenuation code per channel (spxattencod), when in use.
pub spxatten: [Option<usize>; 5],
pub firstcplcos: [bool; 5],
pub firstcplleak: bool,
/// Coupling band structure by absolute coupling sub-band.
pub cplbndstrc: [bool; 18],
pub spx: Spx,
}
impl FrameInfo {
/// Read the audio frame. `words` is the syncframe size in 16-bit words.
pub(crate) fn read(r: &mut Bits, bsi: &Bsi, acmod: usize, lfeon: bool, nblocks: usize, words: usize) -> Result<FrameInfo> {
let nf = NFCHANS[acmod & 7];
let six = bsi.numblkscod == 3;
let (expstre, ahte) = if six { (r.bit(), r.bit()) } else { (true, false) };
let snroffststr = r.read(2) as u8;
if snroffststr == 3 {
return Err(Error::Invalid("reserved SNR offset strategy"));
}
let transproce = r.bit();
let mut e = FrameInfo {
strmtyp: bsi.strmtyp,
blkswe: r.bit(),
dithflage: r.bit(),
bamode: r.bit(),
frmfgaincode: r.bit(),
dbaflde: r.bit(),
skipflde: r.bit(),
snroffststr,
frmcsnroffst: 0,
frmfsnroffst: 0,
cplstre: [false; 6],
cplinu: [false; 6],
cplexpstr: [0; 6],
chexpstr: [[0; 5]; 6],
lfeexpstr: [0; 6],
spxatten: [None; 5],
firstcplcos: [true; 5],
firstcplleak: true,
cplbndstrc: DEFAULT_CPLBNDSTRC,
spx: Spx::new(),
};
let spxattene = r.bit();
if acmod > 1 {
e.cplstre[0] = true;
e.cplinu[0] = r.bit();
for blk in 1..nblocks {
e.cplstre[blk] = r.bit();
e.cplinu[blk] = if e.cplstre[blk] { r.bit() } else { e.cplinu[blk - 1] };
}
}
if expstre {
for blk in 0..nblocks {
if e.cplinu[blk] {
e.cplexpstr[blk] = r.read(2) as u8;
}
for ch in 0..nf {
e.chexpstr[blk][ch] = r.read(2) as u8;
}
}
} else {
// frame-based strategies (six blocks only)
if acmod > 1 && e.cplinu.iter().any(|&c| c) {
e.cplexpstr = FRAME_EXPSTR[r.read(5) as usize & 31];
}
for ch in 0..nf {
let s = FRAME_EXPSTR[r.read(5) as usize & 31];
for (blk, &v) in s.iter().enumerate() {
e.chexpstr[blk][ch] = v;
}
}
}
if lfeon {
for blk in 0..nblocks {
e.lfeexpstr[blk] = r.read(1) as u8;
}
}
if bsi.strmtyp == 0 && (six || r.bit()) {
r.skip(5 * nf); // convexpstr
}
if ahte {
// AHT flags exist for channels whose exponents are sent once in the frame (§E3.4.2)
let unsupported = || Err(Error::Unsupported("E-AC-3 adaptive hybrid transform".into()));
let ncplblks = e.cplinu.iter().filter(|&&c| c).count();
let ncplregs = (0..6).filter(|&b| e.cplstre[b] || e.cplexpstr[b] != 0).count();
if ncplblks == 6 && ncplregs == 1 && r.bit() {
return unsupported();
}
for ch in 0..nf {
if (0..6).filter(|&b| e.chexpstr[b][ch] != 0).count() == 1 && r.bit() {
return unsupported();
}
}
if lfeon && e.lfeexpstr.iter().filter(|&&s| s != 0).count() == 1 && r.bit() {
return unsupported();
}
}
if snroffststr == 0 {
e.frmcsnroffst = r.read(6) as i32;
e.frmfsnroffst = r.read(4) as i32;
}
if transproce {
// transient pre-noise processing (§E3.7) is not applied
for _ in 0..nf {
if r.bit() {
r.skip(10 + 8); // transprocloc, transproclen
}
}
}
if spxattene {
for ch in 0..nf {
if r.bit() {
e.spxatten[ch] = Some(r.read(5) as usize);
}
}
}
if bsi.numblkscod != 0 && r.bit() {
// blkstrtinfo: (blocks - 1) × (4 + ceil(log2(words per frame))) bits (§E2.3.2.27)
let log2 = (usize::BITS - words.saturating_sub(1).leading_zeros()) as usize;
r.skip((nblocks - 1) * (4 + log2));
}
Ok(e)
}
}
/// Spectral extension parameters (§E2.2.4, §E3.6), carried from block to block of a syncframe.
pub(crate) struct Spx {
pub inu: bool,
pub chinspx: [bool; 5],
pub begf: usize,
strtf: usize,
/// First sub-band and one past the last (spx_begin_subbnd, spx_end_subbnd).
begin: usize,
end: usize,
bndstrc: [bool; 17],
/// Band sizes in coefficients (spxbndsztab) and their count (nspxbnds).
sizes: [usize; 17],
nbands: usize,
first: [bool; 5],
co: [[f32; 17]; 5],
nblend: [[f32; 17]; 5],
sblend: [[f32; 17]; 5],
}
impl Spx {
fn new() -> Spx {
Spx {
inu: false,
chinspx: [false; 5],
begf: 0,
strtf: 0,
begin: 0,
end: 0,
bndstrc: DEFAULT_SPXBNDSTRC,
sizes: [0; 17],
nbands: 0,
first: [true; 5],
co: [[0.0; 17]; 5],
nblend: [[0.0; 17]; 5],
sblend: [[1.0; 17]; 5],
}
}
/// First coefficient of the extension region.
pub fn begin_bin(&self) -> usize {
spx_band_bin(self.begin)
}
/// Read the strategy and coordinates of block `blk`.
pub(crate) fn read(&mut self, r: &mut Bits, blk: usize, acmod: usize, nf: usize) -> Result<()> {
if blk == 0 || r.bit() {
self.inu = r.bit();
if self.inu {
if acmod == 1 {
self.chinspx = [true, false, false, false, false];
} else {
for ch in 0..nf {
self.chinspx[ch] = r.bit();
}
}
self.strtf = r.read(2) as usize;
self.begf = r.read(3) as usize;
let endf = r.read(3) as usize;
self.begin = if self.begf < 6 { self.begf + 2 } else { self.begf * 2 - 3 };
self.end = if endf < 3 { endf + 5 } else { endf * 2 + 3 };
if r.bit() {
for b in self.begin + 1..self.end {
self.bndstrc[b] = r.bit();
}
}
if self.begin >= self.end {
return Err(Error::Invalid("spectral extension ends before it begins"));
}
if self.strtf >= self.begin {
return Err(Error::Invalid("spectral extension copies from above its start"));
}
self.nbands = 1;
self.sizes = [0; 17];
self.sizes[0] = 12;
for b in self.begin + 1..self.end {
if self.bndstrc[b] {
self.sizes[self.nbands - 1] += 12;
} else {
self.sizes[self.nbands] = 12;
self.nbands += 1;
}
}
} else {
self.chinspx = [false; 5];
self.first = [true; 5];
}
}
if !self.inu {
return Ok(());
}
for ch in 0..nf {
if !self.chinspx[ch] {
self.first[ch] = true;
continue;
}
let coe = std::mem::replace(&mut self.first[ch], false) || r.bit();
if !coe {
continue;
}
let blend = r.read(5) as f32 / 32.0;
let mstr = r.read(2) as i32;
// coordinates (§E3.6.3) and blending factors (§E3.6.4.2.1)
let mut mant = self.begin_bin() as f32;
let end = spx_band_bin(self.end) as f32;
for b in 0..self.nbands {
let e = r.read(4) as i32;
let m = r.read(2) as f32;
let v = if e == 15 { m / 4.0 } else { (m + 4.0) / 8.0 };
self.co[ch][b] = v * 2f32.powi(-(e + 3 * mstr));
let size = self.sizes[b] as f32;
let nratio = ((mant + 0.5 * size) / end - blend).clamp(0.0, 1.0);
self.nblend[ch][b] = nratio.sqrt();
self.sblend[ch][b] = (1.0 - nratio).sqrt();
mant += size;
}
}
Ok(())
}
/// Synthesize the extension region of one channel's coefficients (§E3.6.4): translate the
/// low band up, notch-filter the borders when attenuation is on, blend with `noise` (zero
/// mean, unit variance) scaled to the band's energy, and scale by the coordinates.
pub(crate) fn synthesize(&self, ch: usize, atten: Option<usize>, tc: &mut [f32; 256], mut noise: impl FnMut() -> f32) {
let copystart = spx_band_bin(self.strtf);
let copyend = self.begin_bin();
let sizes = &self.sizes[..self.nbands];
// translation
let mut wrap = [false; 17];
let (mut copy, mut insert) = (copystart, copyend);
for (b, &size) in sizes.iter().enumerate() {
if copy + size > copyend {
copy = copystart;
wrap[b] = true;
}
for _ in 0..size {
if copy == copyend {
copy = copystart;
}
if let (Some(&v), true) = (tc.get(copy), insert < tc.len()) {
tc[insert] = v;
}
insert += 1;
copy += 1;
}
}
// banded RMS energy of the translated coefficients
let mut rms = [0f32; 17];
let mut at = copyend;
for (b, &size) in sizes.iter().enumerate() {
let band = tc.get(at..at + size).unwrap_or_default();
rms[b] = (band.iter().map(|v| v * v).sum::<f32>() / size as f32).sqrt();
at += size;
}
// notch filter around the baseband / extension border and each wrap point
if let Some(code) = atten {
let g = [spx_atten(code, 0), spx_atten(code, 1), spx_atten(code, 2)];
let mut notch = |center: usize| {
for (k, gain) in [g[0], g[1], g[2], g[1], g[0]].into_iter().enumerate() {
if let Some(v) = (center + k).checked_sub(2).and_then(|i| tc.get_mut(i)) {
*v *= gain;
}
}
};
notch(copyend);
let mut start = copyend;
for (b, &size) in sizes.iter().enumerate() {
if b > 0 && wrap[b] {
notch(start);
}
start += size;
}
}
// noise blending and scaling
let mut at = copyend;
for (b, &size) in sizes.iter().enumerate() {
let nscale = rms[b] * self.nblend[ch][b];
let sscale = self.sblend[ch][b];
let co = self.co[ch][b] * 32.0;
for _ in 0..size {
let n = noise();
if let Some(v) = tc.get_mut(at) {
*v = (*v * sscale + n * nscale) * co;
}
at += 1;
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn attenuation_matches_table_e3_14() {
// rows 0, 4, 14 and 31 as printed
let rows: [(usize, [f64; 3]); 4] = [
(0, [0.954841604, 0.911722489, 0.870550563]),
(4, [0.793700526, 0.629960525, 0.500000000]),
(14, [0.500000000, 0.250000000, 0.125000000]),
(31, [0.227930622, 0.051952369, 0.011841536]),
];
for (code, want) in rows {
for (i, w) in want.into_iter().enumerate() {
assert!((f64::from(spx_atten(code, i)) - w).abs() < 1e-7, "{code} {i}");
}
}
}
#[test]
fn default_band_structures() {
assert_eq!(DEFAULT_CPLBNDSTRC.iter().filter(|&&b| b).count(), 8);
assert!(DEFAULT_CPLBNDSTRC[8] && !DEFAULT_CPLBNDSTRC[9] && !DEFAULT_CPLBNDSTRC[12] && DEFAULT_CPLBNDSTRC[17]);
assert_eq!(DEFAULT_SPXBNDSTRC.iter().enumerate().filter(|(_, b)| **b).map(|(i, _)| i).collect::<Vec<_>>(), vec![8, 10, 12, 14, 16]);
// every frame-based strategy sends new exponents in block 0
assert!(FRAME_EXPSTR.iter().all(|s| s[0] != 0));
}
#[test]
fn spectral_extension_translation_and_scaling() {
// begin at sub-band 2 (bin 49), end at 5 (bin 85), copy from sub-band 0 (bin 25): three
// 12-bin bands, the third wraps back to the copy start
let mut s = Spx::new();
s.inu = true;
(s.strtf, s.begin, s.end, s.nbands) = (0, 2, 5, 3);
s.sizes[..3].copy_from_slice(&[12, 12, 12]);
s.co[0][..3].copy_from_slice(&[1.0 / 32.0, 0.5 / 32.0, 0.25 / 32.0]);
s.nblend[0] = [0.0; 17];
s.sblend[0] = [1.0; 17];
let mut tc = [0f32; 256];
for (i, v) in tc[25..49].iter_mut().enumerate() {
*v = i as f32 + 1.0;
}
s.synthesize(0, None, &mut tc, || 0.0);
assert_eq!(tc[49], 1.0);
assert_eq!(tc[61], 13.0 * 0.5);
assert_eq!(tc[73], 1.0 * 0.25);
assert_eq!(tc[84], 12.0 * 0.25);
assert_eq!(tc[85], 0.0);
// pure noise: the band gets the energy of its translated coefficients
s.nblend[0] = [1.0; 17];
s.sblend[0] = [0.0; 17];
s.co[0] = [1.0 / 32.0; 17];
let mut tc2 = [0f32; 256];
tc2[25..49].fill(2.0);
let mut k = 0;
s.synthesize(0, None, &mut tc2, || {
k += 1;
if k % 2 == 0 { 1.0 } else { -1.0 }
});
assert!(tc2[49..85].iter().all(|v| v.abs() == 2.0));
}
}
+321 -80
View File
@@ -1,5 +1,6 @@
//! Clean-room AC-3 (Dolby Digital) audio decoder, written from ATSC A/52:2012 ("Digital Audio
//! Compression Standard", §5 bit stream syntax, §6-7 decoding).
//! Clean-room AC-3 (Dolby Digital) and E-AC-3 (Dolby Digital Plus) audio decoder, written from
//! ATSC A/52:2012 ("Digital Audio Compression Standard", §5 bit stream syntax, §6-7 decoding,
//! Annex E for E-AC-3).
//!
//! - All audio coding modes (1+1, 1/0 … 3/2) with or without LFE, 32 / 44.1 / 48 kHz, every
//! frame size (Table 5.18).
@@ -11,10 +12,15 @@
//! - Output: planar f32 at full scale ±1, channels in WAV / SMPTE order (L R C LFE Ls Rs); no
//! downmix.
//!
//! E-AC-3 (bsid 11-16) is recognised and refused.
//! E-AC-3 (bsid 11-16, Annex E): independent substream 0 (dependent substreams and further
//! programs are skipped), 1 / 2 / 3 / 6 blocks per syncframe, the reduced sample rates, frame-based
//! exponent strategies and the other frame-level syntax, and spectral extension (§E3.6). The
//! adaptive hybrid transform and enhanced coupling are refused with [`Error::Unsupported`];
//! transient pre-noise processing data is read and not applied.
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
mod eac3;
mod tables;
use std::sync::OnceLock;
@@ -47,12 +53,35 @@ pub struct Header {
pub bsid: u8,
pub acmod: u8,
pub lfeon: bool,
/// Audio blocks (of 256 samples per channel) in the syncframe: 6 for AC-3; 1, 2, 3 or 6 for
/// E-AC-3.
pub blocks: u8,
/// E-AC-3 stream type (Table E2.1): 0 independent, 1 dependent, 2 independent converted from
/// AC-3. 0 for AC-3.
pub strmtyp: u8,
/// E-AC-3 substream id (0 for AC-3).
pub substreamid: u8,
}
impl Header {
/// Output channels (full bandwidth + LFE).
pub fn channels(&self) -> usize {
NFCHANS[self.acmod as usize] + self.lfeon as usize
NFCHANS[self.acmod as usize & 7] + self.lfeon as usize
}
/// Samples per channel the syncframe decodes to.
pub fn samples(&self) -> usize {
256 * self.blocks as usize
}
/// Whether this is an E-AC-3 syncframe (bsid 11-16).
pub fn is_eac3(&self) -> bool {
self.bsid > 10
}
/// Whether the syncframe carries the program [`Decoder::decode`] outputs: AC-3, or E-AC-3
/// independent substream 0 (§E2.3.1.2). [`Decoder::decode`] returns no channels for the
/// others: dependent substreams (the channels beyond 5.1 of a 7.1 program) and further
/// independent programs.
pub fn is_primary(&self) -> bool {
self.strmtyp != 1 && self.substreamid == 0
}
}
@@ -64,8 +93,11 @@ pub fn parse_header(b: &[u8]) -> Result<Header> {
let fscod = b[4] >> 6;
let frmsizecod = b[4] & 0x3F;
let bsid = b[5] >> 3;
if bsid > 10 {
return Err(Error::Unsupported(format!("bsid {bsid} (E-AC-3)")));
if (11..=16).contains(&bsid) {
return eac3::parse_header(b, bsid);
}
if bsid > 16 {
return Err(Error::Unsupported(format!("bsid {bsid}")));
}
if fscod == 3 || frmsizecod >= 38 {
return Err(Error::Invalid("reserved fscod / frmsizecod"));
@@ -91,7 +123,7 @@ pub fn parse_header(b: &[u8]) -> Result<Header> {
r.skip(2);
}
let lfeon = r.read(1) == 1;
Ok(Header { sample_rate, bitrate_kbps: kbps >> shift, frame_bytes: words as usize * 2, bsid, acmod, lfeon })
Ok(Header { sample_rate, bitrate_kbps: kbps >> shift, frame_bytes: words as usize * 2, bsid, acmod, lfeon, blocks: 6, strmtyp: 0, substreamid: 0 })
}
/// MSB-first bit reader; reads past the end give zeros.
@@ -469,7 +501,9 @@ struct Frame {
chincpl: [bool; 5],
phsflginu: bool,
cplbegf: usize,
cplendf: usize,
/// cplendf + 3: one past the last coupling sub-band (with spectral extension cplendf may be
/// negative, §E3.3.1).
cplend: usize,
/// Coupling band of each coupling sub-band (relative to cplbegf).
sub_to_band: [usize; 18],
ncplbnd: usize,
@@ -500,7 +534,7 @@ impl Frame {
chincpl: [false; 5],
phsflginu: false,
cplbegf: 0,
cplendf: 0,
cplend: 0,
sub_to_band: [0; 18],
ncplbnd: 0,
cplco: [[0.0; 18]; 5],
@@ -523,7 +557,7 @@ impl Frame {
37 + 12 * self.cplbegf
}
fn cplendmant(&self) -> usize {
37 + 12 * (self.cplendf + 3)
37 + 12 * self.cplend
}
}
@@ -622,6 +656,9 @@ pub struct Decoder {
delay: [[f32; 256]; 6],
dither: u32,
seed: u32,
/// E-AC-3: the block state of the last syncframe (acmod, lfeon, sample rate code), whose
/// exponents a syncframe of fewer than six blocks may reuse.
carry: Option<(usize, bool, usize, Box<Frame>)>,
}
impl Default for Decoder {
@@ -630,7 +667,8 @@ impl Default for Decoder {
}
}
/// A decoded syncframe: 1536 samples per channel, WAV channel order.
/// A decoded syncframe: [`Header::samples`] samples per channel (1536 for AC-3), WAV channel
/// order. No channels for an E-AC-3 syncframe that is not [`Header::is_primary`].
#[derive(Clone, Debug)]
pub struct Decoded {
pub header: Header,
@@ -639,7 +677,7 @@ pub struct Decoded {
impl Decoder {
pub fn new() -> Decoder {
Decoder { delay: [[0.0; 256]; 6], dither: 0, seed: 0x1234_5678 }
Decoder { delay: [[0.0; 256]; 6], dither: 0, seed: 0x1234_5678, carry: None }
}
/// Seed the dither generator (dither is decoder-specific noise; tests compare two seeds).
@@ -650,6 +688,7 @@ impl Decoder {
/// Forget the overlap (before decoding from another position).
pub fn reset(&mut self) {
self.delay = [[0.0; 256]; 6];
self.carry = None;
}
fn dither(&mut self) -> f32 {
@@ -658,12 +697,20 @@ impl Decoder {
((self.dither >> 8) as f32 / (1u32 << 24) as f32 * 2.0 - 1.0) * 0.707
}
/// Zero-mean, unit-variance noise for spectral extension (§E3.6.4.2.4): uniform in ±√3.
fn noise(&mut self) -> f32 {
self.dither() * (3f32.sqrt() / 0.707)
}
/// Decode one syncframe.
pub fn decode(&mut self, frame: &[u8]) -> Result<Decoded> {
let h = parse_header(frame)?;
if frame.len() < h.frame_bytes {
return Err(Error::Invalid("truncated syncframe"));
}
if h.is_eac3() {
return self.decode_eac3(&frame[..h.frame_bytes], h);
}
let fscod = (frame[4] >> 6) as usize;
// Dither is reseeded per syncframe from its CRC words (plus the configured seed), so a
// frame decodes to the same samples however it was reached (seeking, render caches).
@@ -708,39 +755,83 @@ impl Decoder {
let l = r.read(6) as usize;
r.skip((l + 1) * 8);
}
let nfchans = NFCHANS[acmod];
let mut f = Frame::new(nfchans, fscod);
let mut f = Frame::new(NFCHANS[acmod], fscod);
let pcm = self.blocks(&mut r, &mut f, acmod, lfeon, 6, None)?;
Ok(Decoded { header: h, channels: wav_order(pcm, acmod, lfeon) })
}
/// Decode one E-AC-3 syncframe (`frame` is exactly the syncframe).
fn decode_eac3(&mut self, frame: &[u8], h: Header) -> Result<Decoded> {
if !h.is_primary() {
return Ok(Decoded { header: h, channels: Vec::new() });
}
// E-AC-3 has a single CRC, at the end (§E2.2.6): reseed the dither from the last words
let n = frame.len();
let tail = frame.get(n.saturating_sub(4)..).and_then(|t| <[u8; 4]>::try_from(t).ok()).unwrap_or_default();
self.dither = self.seed ^ u32::from_be_bytes(tail).wrapping_mul(0x9E37_79B9);
let mut r = Bits::new(frame);
let bsi = eac3::read_bsi(&mut r)?;
let (acmod, lfeon) = (h.acmod as usize & 7, h.lfeon);
let nblocks = h.blocks as usize;
let mut e = eac3::FrameInfo::read(&mut r, &bsi, acmod, lfeon, nblocks, n / 2)?;
let mut f = match self.carry.take() {
Some((a, l, fs, f)) if (a, l, fs) == (acmod, lfeon, bsi.fscod) => f,
_ => Box::new(Frame::new(NFCHANS[acmod], bsi.fscod)),
};
let pcm = self.blocks(&mut r, &mut f, acmod, lfeon, nblocks, Some(&mut e))?;
self.carry = Some((acmod, lfeon, bsi.fscod, f));
Ok(Decoded { header: h, channels: wav_order(pcm, acmod, lfeon) })
}
/// Decode `nblocks` audio blocks and synthesize them (inverse transform, overlap-add), in coded
/// channel order with the LFE last.
fn blocks(&mut self, r: &mut Bits, f: &mut Frame, acmod: usize, lfeon: bool, nblocks: usize, mut x: Option<&mut eac3::FrameInfo>) -> Result<Vec<Vec<f32>>> {
let nfchans = f.nfchans;
let nout = nfchans + lfeon as usize;
let mut pcm: Vec<Vec<f32>> = vec![Vec::with_capacity(1536); nout];
for blk in 0..6 {
let mut pcm: Vec<Vec<f32>> = vec![Vec::with_capacity(256 * nblocks); nout];
for blk in 0..nblocks {
let mut coefs = [[0f32; 256]; 6];
self.audio_block(&mut r, &mut f, acmod, lfeon, blk, &mut coefs)?;
self.audio_block(r, f, acmod, lfeon, blk, &mut coefs, x.as_deref_mut())?;
if r.overrun() {
return Err(Error::Invalid("audio block runs past the frame"));
}
// inverse transform, overlap-add
let mut buf = [0f32; 512];
for ch in 0..nout {
for (ch, out) in pcm.iter_mut().enumerate() {
let src = if ch < nfchans { ch } else { LFE };
let short = ch < nfchans && f.blksw[ch];
imdct(&coefs[src], short, &mut buf);
let d = &mut self.delay[src];
for n in 0..256 {
pcm[ch].push(2.0 * (buf[n] + d[n]));
out.push(2.0 * (buf[n] + d[n]));
d[n] = buf[256 + n];
}
}
}
Ok(Decoded { header: h, channels: wav_order(pcm, acmod, lfeon) })
Ok(pcm)
}
fn audio_block(&mut self, r: &mut Bits, f: &mut Frame, acmod: usize, lfeon: bool, blk: usize, coefs: &mut [[f32; 256]; 6]) -> Result<()> {
#[allow(clippy::too_many_arguments)]
fn audio_block(
&mut self,
r: &mut Bits,
f: &mut Frame,
acmod: usize,
lfeon: bool,
blk: usize,
coefs: &mut [[f32; 256]; 6],
mut x: Option<&mut eac3::FrameInfo>,
) -> Result<()> {
let nf = f.nfchans;
for ch in 0..nf {
f.blksw[ch] = r.bit();
let eac3 = x.is_some();
match x.as_deref() {
Some(e) if !e.blkswe => f.blksw = [false; 5],
_ => (0..nf).for_each(|ch| f.blksw[ch] = r.bit()),
}
for ch in 0..nf {
f.dithflag[ch] = r.bit();
match x.as_deref() {
// dither on when the flags are not sent (§E2.2.4)
Some(e) if !e.dithflage => f.dithflag = [true; 5],
_ => (0..nf).for_each(|ch| f.dithflag[ch] = r.bit()),
}
for k in 0..if acmod == 0 { 2 } else { 1 } {
if r.bit() {
@@ -749,26 +840,62 @@ impl Decoder {
f.dynrng[k] = 1.0;
}
}
// E-AC-3 spectral extension strategy and coordinates
if let Some(e) = x.as_deref_mut() {
e.spx.read(r, blk, acmod, nf)?;
}
let spx = x.as_deref().map(|e| &e.spx).filter(|s| s.inu);
let spxbegf = spx.map(|s| s.begf);
let spx_begin = spx.map(|s| s.begin_bin());
let spx_chans = spx.map(|s| s.chinspx).unwrap_or([false; 5]);
// coupling strategy
if r.bit() {
f.cplinu = r.bit();
let strategy = match x.as_deref() {
Some(e) => e.cplstre[blk].then_some(e.cplinu[blk]),
None => r.bit().then(|| r.bit()),
};
if let Some(inu) = strategy {
f.cplinu = inu;
if f.cplinu {
for ch in 0..nf {
f.chincpl[ch] = r.bit();
if eac3 && r.bit() {
return Err(Error::Unsupported("E-AC-3 enhanced coupling".into()));
}
if eac3 && acmod == 2 {
f.chincpl = [true, true, false, false, false];
} else {
for ch in 0..nf {
f.chincpl[ch] = r.bit();
}
}
if acmod == 2 {
f.phsflginu = r.bit();
}
f.cplbegf = r.read(4) as usize;
f.cplendf = r.read(4) as usize;
if f.cplendf + 3 <= f.cplbegf {
f.cplend = match spxbegf {
// the coupling region ends where spectral extension begins (§E2.2.4)
Some(b) if b < 6 => b + 1,
Some(b) => 2 * b - 4,
None => r.read(4) as usize + 3,
};
if f.cplend <= f.cplbegf {
return Err(Error::Invalid("coupling end below start"));
}
let nsub = 3 + f.cplendf - f.cplbegf;
// coupling band structure, by absolute sub-band
let mut strc = [false; 18];
match x.as_deref_mut() {
Some(e) => {
if r.bit() {
for s in f.cplbegf + 1..f.cplend {
e.cplbndstrc[s] = r.bit();
}
}
strc = e.cplbndstrc;
}
None => (f.cplbegf + 1..f.cplend).for_each(|s| strc[s] = r.bit()),
}
let mut band = 0;
f.sub_to_band[0] = 0;
for s in 1..nsub {
if !r.bit() {
for s in 1..f.cplend - f.cplbegf {
if !strc[f.cplbegf + s] {
band += 1;
}
f.sub_to_band[s] = band;
@@ -776,15 +903,33 @@ impl Decoder {
f.ncplbnd = band + 1;
} else {
f.chincpl = [false; 5];
if let Some(e) = x.as_deref_mut() {
e.firstcplcos = [true; 5];
e.firstcplleak = true;
f.phsflginu = false;
}
}
} else if blk == 0 {
} else if blk == 0 && !eac3 {
return Err(Error::Invalid("no coupling strategy in block 0"));
}
// coupling coordinates
if f.cplinu {
let mut any = false;
for ch in 0..nf {
if f.chincpl[ch] && r.bit() {
if !f.chincpl[ch] {
if let Some(e) = x.as_deref_mut() {
e.firstcplcos[ch] = true;
}
continue;
}
let coe = match x.as_deref_mut() {
Some(e) if e.firstcplcos[ch] => {
e.firstcplcos[ch] = false;
true
}
_ => r.bit(),
};
if coe {
any = true;
let mstr = r.read(2) as i32;
for bnd in 0..f.ncplbnd {
@@ -792,7 +937,7 @@ impl Decoder {
let m = r.read(4) as f32;
let v = if e == 15 { m / 16.0 } else { (m + 16.0) / 32.0 };
let co = v * 2f32.powi(-(e + 3 * mstr));
for s in 0..(3 + f.cplendf - f.cplbegf) {
for s in 0..(f.cplend - f.cplbegf) {
if f.sub_to_band[s] == bnd {
f.cplco[ch][s] = co;
}
@@ -803,7 +948,7 @@ impl Decoder {
if acmod == 2 && f.phsflginu && any {
for bnd in 0..f.ncplbnd {
let flag = r.bit();
for s in 0..(3 + f.cplendf - f.cplbegf) {
for s in 0..(f.cplend - f.cplbegf) {
if f.sub_to_band[s] == bnd {
f.phsflg[s] = flag;
}
@@ -811,34 +956,55 @@ impl Decoder {
}
}
}
// rematrixing
if acmod == 2 && r.bit() {
let n = if !f.cplinu || f.cplbegf > 2 {
4
} else if f.cplbegf > 0 {
3
// rematrixing (E-AC-3 always sends the flags in block 0)
if acmod == 2 && ((eac3 && blk == 0) || r.bit()) {
let n = if f.cplinu {
match f.cplbegf {
0 => 2,
1 | 2 => 3,
_ => 4,
}
} else {
2
match spxbegf {
Some(b) if b < 2 => 3,
_ => 4,
}
};
f.rematflg = [false; 4];
for k in 0..n {
f.rematflg[k] = r.bit();
}
}
// exponent strategies
if f.cplinu {
f.expstr[CPL] = r.read(2) as u8;
}
for ch in 0..nf {
f.expstr[ch] = r.read(2) as u8;
}
if lfeon {
f.expstr[LFE] = r.read(1) as u8;
// exponent strategies (E-AC-3: from the audio frame)
match x.as_deref() {
Some(e) => {
if f.cplinu {
f.expstr[CPL] = e.cplexpstr[blk];
}
f.expstr[..nf].copy_from_slice(&e.chexpstr[blk][..nf]);
if lfeon {
f.expstr[LFE] = e.lfeexpstr[blk];
}
}
None => {
if f.cplinu {
f.expstr[CPL] = r.read(2) as u8;
}
for ch in 0..nf {
f.expstr[ch] = r.read(2) as u8;
}
if lfeon {
f.expstr[LFE] = r.read(1) as u8;
}
}
}
for ch in 0..nf {
if f.expstr[ch] != 0 {
if f.chincpl[ch] && f.cplinu {
f.endmant[ch] = f.cplstrtmant();
} else if let (true, Some(begin)) = (spx_chans[ch], spx_begin) {
// §E3.3.3: the channel ends where spectral extension begins
f.endmant[ch] = begin;
} else {
let bw = r.read(6) as usize;
if bw > 60 {
@@ -870,40 +1036,105 @@ impl Decoder {
decode_exponents(r, abs, 2, 1, &mut f.exps[LFE], 0, false)?;
}
// bit allocation parametric information
if r.bit() {
f.g.sdcycod = r.read(2) as usize;
f.g.fdcycod = r.read(2) as usize;
f.g.sgaincod = r.read(2) as usize;
f.g.dbpbcod = r.read(2) as usize;
f.g.floorcod = r.read(3) as usize;
} else if blk == 0 {
return Err(Error::Invalid("no bit allocation information in block 0"));
match x.as_deref() {
Some(e) if !e.bamode => {
// §E2.2.4 defaults
(f.g.sdcycod, f.g.fdcycod, f.g.sgaincod, f.g.dbpbcod, f.g.floorcod) = (2, 1, 1, 2, 7);
}
_ => {
if r.bit() {
f.g.sdcycod = r.read(2) as usize;
f.g.fdcycod = r.read(2) as usize;
f.g.sgaincod = r.read(2) as usize;
f.g.dbpbcod = r.read(2) as usize;
f.g.floorcod = r.read(3) as usize;
} else if blk == 0 && !eac3 {
return Err(Error::Invalid("no bit allocation information in block 0"));
}
}
}
if r.bit() {
f.csnroffst = r.read(6) as i32;
if f.cplinu {
f.fsnroffst[CPL] = r.read(4) as i32;
f.fgaincod[CPL] = r.read(3) as usize;
match x.as_deref() {
Some(e) => {
// SNR offset strategies (Table E2.9), fast gain codes
let sets = [CPL, 0, 1, 2, 3, 4, LFE];
match e.snroffststr {
0 => {
f.csnroffst = e.frmcsnroffst;
sets.iter().for_each(|&c| f.fsnroffst[c] = e.frmfsnroffst);
}
s if blk == 0 || r.bit() => {
f.csnroffst = r.read(6) as i32;
if s == 1 {
let v = r.read(4) as i32;
sets.iter().for_each(|&c| f.fsnroffst[c] = v);
} else {
if f.cplinu {
f.fsnroffst[CPL] = r.read(4) as i32;
}
for ch in 0..nf {
f.fsnroffst[ch] = r.read(4) as i32;
}
if lfeon {
f.fsnroffst[LFE] = r.read(4) as i32;
}
}
}
_ => {}
}
if e.frmfgaincode && r.bit() {
if f.cplinu {
f.fgaincod[CPL] = r.read(3) as usize;
}
for ch in 0..nf {
f.fgaincod[ch] = r.read(3) as usize;
}
if lfeon {
f.fgaincod[LFE] = r.read(3) as usize;
}
} else {
f.fgaincod = [4; 7];
}
if e.strmtyp == 0 && r.bit() {
r.skip(10); // convsnroffst
}
}
for ch in 0..nf {
f.fsnroffst[ch] = r.read(4) as i32;
f.fgaincod[ch] = r.read(3) as usize;
None => {
if r.bit() {
f.csnroffst = r.read(6) as i32;
if f.cplinu {
f.fsnroffst[CPL] = r.read(4) as i32;
f.fgaincod[CPL] = r.read(3) as usize;
}
for ch in 0..nf {
f.fsnroffst[ch] = r.read(4) as i32;
f.fgaincod[ch] = r.read(3) as usize;
}
if lfeon {
f.fsnroffst[LFE] = r.read(4) as i32;
f.fgaincod[LFE] = r.read(3) as usize;
}
} else if blk == 0 {
return Err(Error::Invalid("no SNR offsets in block 0"));
}
}
if lfeon {
f.fsnroffst[LFE] = r.read(4) as i32;
f.fgaincod[LFE] = r.read(3) as usize;
}
} else if blk == 0 {
return Err(Error::Invalid("no SNR offsets in block 0"));
}
if f.cplinu && r.bit() {
f.cplleak = (((r.read(3) as i32) << 8) + 768, ((r.read(3) as i32) << 8) + 768);
if f.cplinu {
let leak = match x.as_deref_mut() {
Some(e) if e.firstcplleak => {
e.firstcplleak = false;
true
}
_ => r.bit(),
};
if leak {
f.cplleak = (((r.read(3) as i32) << 8) + 768, ((r.read(3) as i32) << 8) + 768);
}
}
// delta bit allocation
if blk == 0 {
f.deltbae = [2; 7];
}
if r.bit() {
if x.as_deref().is_none_or(|e| e.dbaflde) && r.bit() {
if f.cplinu {
f.deltbae[CPL] = r.read(2) as u8;
}
@@ -921,7 +1152,7 @@ impl Decoder {
}
}
// skip field
if r.bit() {
if x.as_deref().is_none_or(|e| e.skipflde) && r.bit() {
let l = r.read(9) as usize;
r.skip(l * 8);
}
@@ -1014,6 +1245,16 @@ impl Decoder {
}
}
}
// spectral extension: synthesize the high band of each channel from its low band
if let Some(e) = x.as_deref()
&& e.spx.inu
{
for ch in 0..nf {
if e.spx.chinspx[ch] {
e.spx.synthesize(ch, e.spxatten[ch], &mut coefs[ch], || self.noise());
}
}
}
// dynamic range
for ch in 0..nf {
let g = if acmod == 0 && ch == 1 { f.dynrng[1] } else { f.dynrng[0] };
+174 -1
View File
@@ -56,7 +56,7 @@ fn headers_and_gains() {
let b = [0x0B, 0x77, 0, 0, 30, 8 << 3, 0b1110_0001, 0b0100_0000];
let h = parse_header(&b).unwrap();
assert_eq!((h.sample_rate, h.bitrate_kbps, h.frame_bytes, h.acmod, h.lfeon, h.channels()), (48_000, 448, 1792, 7, true, 6));
assert_eq!(parse_header(&[0x0B, 0x77, 0, 0, 0, 16 << 3, 0, 0]), Err(Error::Unsupported("bsid 16 (E-AC-3)".into())));
assert_eq!(parse_header(&[0x0B, 0x77, 0, 0, 0, 17 << 3, 0, 0]), Err(Error::Unsupported("bsid 17".into())));
assert_eq!(dynrng_gain(0), 1.0);
assert!((dynrng_gain(0b1110_0000) - 0.5).abs() < 1e-7);
assert!((dynrng_gain(0b0111_1111) - 16.0 * 63.0 / 64.0).abs() < 1e-5);
@@ -96,3 +96,176 @@ fn garbage_never_panics() {
}
}
}
/// MSB-first bit writer for synthetic E-AC-3 syncframes.
#[derive(Default)]
struct W {
b: Vec<u8>,
n: usize,
}
impl W {
fn put(&mut self, bits: u32, v: u32) {
for i in (0..bits).rev() {
if self.n.is_multiple_of(8) {
self.b.push(0);
}
let bit = ((v >> i) & 1) as u8;
*self.b.last_mut().unwrap() |= bit << (7 - self.n % 8);
self.n += 1;
}
}
}
/// An E-AC-3 syncframe of `words` 16-bit words: syncinfo and a bsi with no optional metadata
/// (fscod < 3 with `numblkscod`), then `audio` writes the rest; zero padded.
fn eac3_frame(strmtyp: u32, substreamid: u32, words: u32, numblkscod: u32, acmod: u32, audio: impl FnOnce(&mut W)) -> Vec<u8> {
let mut w = W::default();
w.put(16, 0x0B77);
w.put(2, strmtyp);
w.put(3, substreamid);
w.put(11, words - 1);
w.put(2, 0); // 48 kHz
w.put(2, numblkscod);
w.put(3, acmod);
w.put(1, 0); // lfeon
w.put(5, 16); // bsid
w.put(5, 31); // dialnorm
w.put(1, 0); // compre
if strmtyp == 1 {
w.put(1, 0); // chanmape
}
w.put(2, 0); // mixmdate, infomdate
if strmtyp == 0 && numblkscod != 3 {
w.put(1, 0); // convsync
}
w.put(1, 0); // addbsie
audio(&mut w);
let mut b = w.b;
b.resize(words as usize * 2, 0);
b
}
#[test]
fn eac3_headers() {
let f = eac3_frame(0, 0, 384, 3, 2, |_| {});
let h = parse_header(&f).unwrap();
assert_eq!((h.sample_rate, h.blocks, h.samples(), h.frame_bytes, h.bitrate_kbps, h.channels()), (48_000, 6, 1536, 768, 192, 2));
assert!(h.is_eac3() && h.is_primary());
// one block per syncframe: 256 samples
let h = parse_header(&eac3_frame(0, 0, 100, 0, 7, |_| {})).unwrap();
assert_eq!((h.blocks, h.samples(), h.channels(), h.bitrate_kbps), (1, 256, 5, 300));
// reduced sample rate (fscod 3, fscod2 1): six blocks at 22.05 kHz
let mut f = eac3_frame(0, 0, 384, 0, 2, |_| {});
f[4] = 0b1101_0100;
let h = parse_header(&f).unwrap();
assert_eq!((h.sample_rate, h.blocks, h.acmod), (22_050, 6, 2));
f[4] = 0b1111_0100;
assert_eq!(parse_header(&f), Err(Error::Invalid("reserved fscod2")));
// a dependent substream and a second independent program are recognised and skipped
let mut d = Decoder::new();
for (strmtyp, id) in [(1, 0), (0, 1)] {
let f = eac3_frame(strmtyp, id, 64, 3, 2, |_| {});
let out = d.decode(&f).unwrap();
assert!(!out.header.is_primary() && out.channels.is_empty());
}
assert_eq!(parse_header(&eac3_frame(3, 0, 64, 3, 2, |_| {})), Err(Error::Unsupported("reserved E-AC-3 stream type 3".into())));
}
#[test]
fn eac3_tools_we_do_not_decode_are_refused() {
// mono, six blocks, AHT enabled and used: chexpstr D15 then reuse, so chahtinu is sent
let aht = eac3_frame(0, 0, 64, 3, 1, |w| {
w.put(1, 1); // expstre
w.put(1, 1); // ahte
w.put(2, 0); // snroffststr
w.put(8, 0); // transproce … spxattene
w.put(2, 1); // chexpstr[0] = D15
w.put(10, 0); // chexpstr[1..6] = reuse
w.put(5, 0); // convexpstr
w.put(1, 1); // chahtinu
});
assert_eq!(Decoder::new().decode(&aht).unwrap_err(), Error::Unsupported("E-AC-3 adaptive hybrid transform".into()));
// stereo with coupling in block 0 using enhanced coupling
let ecpl = eac3_frame(0, 0, 64, 3, 2, |w| {
w.put(1, 1); // expstre
w.put(1, 0); // ahte
w.put(2, 0); // snroffststr
w.put(8, 0);
w.put(1, 1); // cplinu[0]
w.put(5, 0); // cplstre[1..6]
w.put(6, 0b01_01_01); // block 0: cplexpstr, chexpstr × 2
w.put(30, 0); // blocks 1-5: reuse
w.put(10, 0); // convexpstr
w.put(10, 0); // frmcsnroffst, frmfsnroffst
w.put(1, 0); // blkstrtinfoe
w.put(1, 0); // dynrnge
w.put(1, 0); // spxinu
w.put(1, 1); // ecplinu
});
assert_eq!(Decoder::new().decode(&ecpl).unwrap_err(), Error::Unsupported("E-AC-3 enhanced coupling".into()));
}
#[test]
fn eac3_single_block_frames_reuse_exponents_across_syncframes() {
// mono, one block per syncframe; the first sends exponents (all 10, bandwidth code 0), the
// second reuses them. Zero SNR offsets allocate no bits: every coefficient is dither.
let frame = |new_exps: bool| {
eac3_frame(0, 0, 32, 0, 1, |w| {
// (expstre and ahte are not sent with fewer than six blocks)
w.put(2, 0); // snroffststr
w.put(8, 0); // transproce … spxattene
w.put(2, new_exps as u32); // chexpstr[0]: D15 or reuse
w.put(1, 0); // convexpstre
w.put(10, 0); // frmcsnroffst, frmfsnroffst
w.put(1, 0); // dynrnge
w.put(1, 0); // spxinu
if new_exps {
w.put(6, 0); // chbwcod → 73 coefficients
w.put(4, 10); // absolute exponent
for _ in 0..24 {
w.put(7, 62); // three zero deltas
}
w.put(2, 0); // gainrng
}
w.put(1, 0); // convsnroffste
})
};
let (first, second) = (frame(true), frame(false));
let mut d = Decoder::new();
let a = d.decode(&first).unwrap();
assert_eq!((a.header.samples(), a.channels.len(), a.channels[0].len()), (256, 1, 256));
let b = d.decode(&second).unwrap();
let peak = |c: &[f32]| c.iter().fold(0f32, |m, v| m.max(v.abs()));
assert!(peak(&b.channels[0]) > 1e-5 && peak(&b.channels[0]) < 0.1, "{}", peak(&b.channels[0]));
// after a reset there are no exponents to reuse: no coefficients, silence
d.reset();
let c = d.decode(&second).unwrap();
assert_eq!(peak(&c.channels[0]), 0.0);
}
#[test]
fn eac3_garbage_never_panics() {
let mut d = Decoder::new();
let mut x = 0x5EEDu32;
for len in [8usize, 40, 300, 1500, 4096] {
for _ in 0..300 {
let mut f: Vec<u8> = (0..len)
.map(|_| {
x ^= x << 13;
x ^= x >> 17;
x ^= x << 5;
x as u8
})
.collect();
f[0] = 0x0B;
f[1] = 0x77;
// independent substream 0, frmsiz within the buffer, bsid 11-16
let words = (len / 2).clamp(4, 2048) - 1;
f[2] = (words >> 8) as u8 & 7;
f[3] = words as u8;
f[5] = (f[5] & 7) | ((11 + x % 6) as u8) << 3;
let _ = d.decode(&f);
}
}
}
+125 -7
View File
@@ -1,5 +1,5 @@
//! AC-3 against ffmpeg: ffmpeg's AC-3 encoder writes raw syncframes from generated tones and
//! noise; ffmpeg's decode of the same file is the reference. AC-3 output is not bit-exact across
//! AC-3 and E-AC-3 against ffmpeg: ffmpeg's AC-3 and E-AC-3 encoders write raw syncframes from
//! generated tones and noise; ffmpeg's decode of the same file is the reference. AC-3 output is not bit-exact across
//! decoders (zero-bit mantissas are filled with decoder-specific dither, §7.3.4; float vs fixed
//! arithmetic), so the criterion is SNR against ffmpeg's decode.
@@ -8,6 +8,8 @@ use std::process::{Command, Stdio};
struct Spec {
name: &'static str,
/// ffmpeg encoder and raw muxer: "ac3" or "eac3".
codec: &'static str,
input: &'static str,
rate: u32,
channels: u32,
@@ -19,6 +21,7 @@ struct Spec {
const SPECS: &[Spec] = &[
Spec {
name: "ac3_stereo_192k",
codec: "ac3",
input: "aevalsrc=exprs=0.3*sin(2*PI*441*t)+0.1*sin(2*PI*3100*t)|0.25*sin(2*PI*660*t)+0.1*sin(2*PI*(500+4000*t)*t):s=48000",
rate: 48_000,
channels: 2,
@@ -28,16 +31,26 @@ const SPECS: &[Spec] = &[
// low bitrate stereo: coupling and rematrixing
Spec {
name: "ac3_stereo_96k_coupling",
codec: "ac3",
input: "aevalsrc=exprs=0.3*sin(2*PI*441*t)+0.05*sin(2*PI*9100*t)|0.3*sin(2*PI*441*t)+0.05*sin(2*PI*11000*t):s=44100",
rate: 44_100,
channels: 2,
enc: &["-b:a", "96k", "-channel_coupling", "1"],
min_snr: 40.0,
},
Spec { name: "ac3_mono_32k", input: "sine=frequency=1000:sample_rate=32000", rate: 32_000, channels: 1, enc: &["-b:a", "96k"], min_snr: 60.0 },
Spec {
name: "ac3_mono_32k",
codec: "ac3",
input: "sine=frequency=1000:sample_rate=32000",
rate: 32_000,
channels: 1,
enc: &["-b:a", "96k"],
min_snr: 60.0,
},
// 5.1 with LFE
Spec {
name: "ac3_51_448k",
codec: "ac3",
input: "aevalsrc=exprs=0.3*sin(2*PI*300*t)|0.3*sin(2*PI*400*t)|0.3*sin(2*PI*500*t)|0.3*sin(2*PI*60*t)|0.2*sin(2*PI*700*t)|0.2*sin(2*PI*800*t):s=48000:c=5.1",
rate: 48_000,
channels: 6,
@@ -47,22 +60,86 @@ const SPECS: &[Spec] = &[
// noise (wideband, many zero-bit mantissas): the SNR is limited by dither
Spec {
name: "ac3_noise_stereo",
codec: "ac3",
input: "anoisesrc=color=pink:sample_rate=48000:amplitude=0.3:seed=3,pan=stereo|c0=c0|c1=-0.5*c0",
rate: 48_000,
channels: 2,
enc: &["-b:a", "128k"],
min_snr: 18.0,
},
// E-AC-3 (Annex E): ffmpeg writes six-block syncframes with frame-based exponent strategies
Spec {
name: "eac3_stereo_192k",
codec: "eac3",
input: "aevalsrc=exprs=0.3*sin(2*PI*441*t)+0.1*sin(2*PI*3100*t)|0.25*sin(2*PI*660*t)+0.1*sin(2*PI*(500+4000*t)*t):s=48000",
rate: 48_000,
channels: 2,
enc: &["-b:a", "192k"],
min_snr: 60.0,
},
// low bitrate stereo: coupling (with the default band structure) and rematrixing
Spec {
name: "eac3_stereo_64k_coupling",
codec: "eac3",
input: "aevalsrc=exprs=0.3*sin(2*PI*441*t)+0.05*sin(2*PI*9100*t)|0.3*sin(2*PI*441*t)+0.05*sin(2*PI*11000*t):s=44100",
rate: 44_100,
channels: 2,
enc: &["-b:a", "64k", "-channel_coupling", "1"],
min_snr: 35.0,
},
Spec {
name: "eac3_mono_32k",
codec: "eac3",
input: "sine=frequency=1000:sample_rate=32000",
rate: 32_000,
channels: 1,
enc: &["-b:a", "64k"],
min_snr: 60.0,
},
// 5.1 with LFE and mixing / informational metadata in the bsi
Spec {
name: "eac3_51_384k_metadata",
codec: "eac3",
input: "aevalsrc=exprs=0.3*sin(2*PI*300*t)|0.3*sin(2*PI*400*t)|0.3*sin(2*PI*500*t)|0.3*sin(2*PI*60*t)|0.2*sin(2*PI*700*t)|0.2*sin(2*PI*800*t):s=48000:c=5.1",
rate: 48_000,
channels: 6,
enc: &[
"-b:a",
"384k",
"-dmix_mode",
"loro",
"-ltrt_cmixlev",
"0.707",
"-loro_surmixlev",
"0.5",
"-dsurex_mode",
"on",
"-mixing_level",
"100",
"-room_type",
"small",
],
min_snr: 55.0,
},
Spec {
name: "eac3_noise_stereo",
codec: "eac3",
input: "anoisesrc=color=pink:sample_rate=48000:amplitude=0.3:seed=3,pan=stereo|c0=c0|c1=-0.5*c0",
rate: 48_000,
channels: 2,
enc: &["-b:a", "96k", "-dsur_mode", "on", "-dheadphone_mode", "off"],
min_snr: 18.0,
},
];
fn make(ff: &Path, s: &Spec) -> Option<PathBuf> {
let out = filmcraft_testkit::fixtures_dir("ac3").join(format!("{}.ac3", s.name));
let out = filmcraft_testkit::fixtures_dir("ac3").join(format!("{}.{}", s.name, s.codec));
filmcraft_testkit::fixtures::generate(&out, |tmp| {
let st = Command::new(ff)
.args(["-hide_banner", "-v", "error", "-y", "-f", "lavfi", "-i", s.input, "-t", "1"])
.args(["-c:a", "ac3"])
.args(["-c:a", s.codec])
.args(s.enc)
.args(["-f", "ac3"])
.args(["-f", s.codec])
.arg(tmp)
.stdin(Stdio::null())
.status();
@@ -94,6 +171,7 @@ fn ffmpeg_encoded_streams_decode() {
for fr in frames(&data) {
let d = dec.decode(fr).unwrap();
assert_eq!((d.header.sample_rate, d.channels.len() as u32), (s.rate, s.channels), "{}", s.name);
assert_eq!(d.header.is_eac3(), s.codec == "eac3", "{}", s.name);
for (o, c) in ours.iter_mut().zip(d.channels) {
o.extend(c);
}
@@ -136,12 +214,52 @@ fn ffmpeg_encoded_streams_decode() {
}
}
/// Damaged E-AC-3 syncframes (bit flips, truncation, corrupt sizes) give errors or noise, never
/// a panic.
#[test]
fn damaged_eac3_frames_never_panic() {
let ff = filmcraft_testkit::require_ffmpeg!();
let mut x = 0x00C0_FFEEu32;
let mut rnd = move || {
x ^= x << 13;
x ^= x >> 17;
x ^= x << 5;
x
};
for s in SPECS.iter().filter(|s| s.codec == "eac3") {
let f = make(&ff, s).unwrap_or_else(|| panic!("could not generate {}", s.name));
let data = std::fs::read(&f).unwrap();
let mut dec = filmcraft_ac3::Decoder::new();
for fr in frames(&data).into_iter().take(8) {
for k in 0..300 {
let mut m = fr.to_vec();
match k % 3 {
0 => {
for _ in 0..1 + rnd() % 8 {
let bit = rnd() as usize % (m.len() * 8);
m[bit / 8] ^= 0x80 >> (bit % 8);
}
}
1 => m.truncate(rnd() as usize % m.len()),
_ => {
// keep the sync word, scramble frmsiz / fscod / numblkscod / acmod
m[2] ^= rnd() as u8;
m[3] ^= rnd() as u8;
m[4] ^= rnd() as u8;
}
}
let _ = dec.decode(&m);
}
}
}
}
#[test]
#[ignore]
fn generate_fixtures() {
let Some(ff) = filmcraft_testkit::ffmpeg_or_skip("ac3 fixtures") else { return };
for s in SPECS {
let out = filmcraft_testkit::fixtures_dir("ac3").join(format!("{}.ac3", s.name));
let out = filmcraft_testkit::fixtures_dir("ac3").join(format!("{}.{}", s.name, s.codec));
filmcraft_testkit::fixtures::generate_and_report(&format!("ac3/{}", s.name), &[out], || make(&ff, s));
}
}
+1 -1
View File
@@ -2,7 +2,7 @@
//!
//! **Channel order.** 5.1 audio is always in the SMPTE / WAVE order **L, R, C, LFE, Ls, Rs**
//! ([`L`], [`R`], [`C`], [`LFE`], [`LS`], [`RS`]); stereo is L, R; mono is one channel. Codecs with a
//! different native order (AAC: C, L, R, Ls, Rs, LFE) reorder at the encoder boundary.
//! different native order (AAC: C, L, R, Ls, Rs, LFE) reorder at the codec boundary.
//!
//! **Downmix** follows ITU-R BS.775 (Table 2, "3/2 source" rows), with `k = 1/√2` (−3 dB):
//!
+60 -5
View File
@@ -107,10 +107,11 @@ impl FilmcraftMcp {
}
/// Delete what an interrupted export wrote, and nothing else: the output file itself, the numbered
/// stills of an image sequence (`<stem><digits>.<same extension>`) and the caption sidecar
/// stills of an image sequence (`<stem><3+ digits>.<same extension>`, only when the output is a
/// PNG/TIFF/BMP/JPEG/Targa/DPX sequence) and the caption sidecar
/// (`<stem>.srt` / `<stem>.vtt`), each only if it was written since `since`. Other files that
/// merely share the name's stem (`<stem>.aep`, `<stem>2.psd`, …) are never touched, even when they
/// were saved during the export.
/// merely share the name's stem (`<stem>.aep`, `<stem>2.psd`, `<stem>1.mp4` beside a movie, …) are
/// never touched, even when they were saved during the export.
pub fn remove_partial(path: &str, since: SystemTime) {
let p = Path::new(path);
let (Some(dir), Some(stem), Some(file)) = (p.parent(), p.file_stem().and_then(|s| s.to_str()), p.file_name().and_then(|s| s.to_str())) else {
@@ -139,9 +140,19 @@ fn is_export_output(name: &str, file: &str, stem: &str, ext: &str) -> bool {
if let Some(sidecar) = rest.strip_prefix('.') {
return sidecar.eq_ignore_ascii_case("srt") || sidecar.eq_ignore_ascii_case("vtt");
}
// image sequence: one or more digits, then the output's own extension
// image sequence: the frame number (at least three digits, like `image_sequence_path`), then the
// output's own extension; movies and audio are a single file, so numbered neighbours are not theirs
if !is_sequence_extension(ext) {
return false;
}
let digits = rest.len() - rest.trim_start_matches(|c: char| c.is_ascii_digit()).len();
digits > 0 && !ext.is_empty() && rest.get(digits..).is_some_and(|r| r.strip_prefix('.').is_some_and(|x| x == ext))
digits >= 3 && rest.get(digits..).is_some_and(|r| r.strip_prefix('.').is_some_and(|x| x == ext))
}
/// Whether an export to a file with extension `ext` writes an image sequence (one still per frame).
fn is_sequence_extension(ext: &str) -> bool {
use filmcraft_engine::export::Format;
Format::from_name(ext).is_some_and(Format::is_image_sequence)
}
#[cfg(test)]
@@ -170,6 +181,50 @@ mod tests {
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn numbered_files_beside_a_movie_are_kept() {
let dir = std::env::temp_dir().join(format!("fc-remove-partial-movie-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let started = SystemTime::now();
// a movie export writes one file: `clip1.mp4` and `clip002.mp4` are other exports
let ours = ["clip.mp4", "clip.srt"];
let theirs = ["clip1.mp4", "clip002.mp4", "clip2.mov"];
for f in ours.iter().chain(theirs.iter()) {
std::fs::write(dir.join(f), b"x").unwrap();
}
remove_partial(&dir.join("clip.mp4").to_string_lossy(), started);
for f in ours {
assert!(!dir.join(f).exists(), "{f} should be removed");
}
for f in theirs {
assert!(dir.join(f).exists(), "{f} must not be removed");
}
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn sequence_frames_have_at_least_three_digits() {
let dir = std::env::temp_dir().join(format!("fc-remove-partial-digits-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let started = SystemTime::now();
// frames are numbered with at least three digits (`shot000.tiff`); `shot2.tiff` is not one
let ours = ["shot.tiff", "shot000.tiff", "shot1234.tiff"];
let theirs = ["shot2.tiff", "shot12.tiff"];
for f in ours.iter().chain(theirs.iter()) {
std::fs::write(dir.join(f), b"x").unwrap();
}
remove_partial(&dir.join("shot.tiff").to_string_lossy(), started);
for f in ours {
assert!(!dir.join(f).exists(), "{f} should be removed");
}
for f in theirs {
assert!(dir.join(f).exists(), "{f} must not be removed");
}
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn files_older_than_the_export_are_kept() {
let dir = std::env::temp_dir().join(format!("fc-remove-partial-old-{}", std::process::id()));
+59 -5
View File
@@ -139,6 +139,30 @@ pub struct TypeParams {
pub text: String,
}
#[derive(Debug, Deserialize, JsonSchema)]
pub struct MediaImportParams {
/// Absolute paths to media files (MP4/MOV, WAV/MP3/FLAC/AIFF/Ogg, PNG/JPEG/…).
pub paths: Vec<String>,
/// Bin ID to import into (optional).
#[serde(default)]
pub bin: Option<u64>,
/// Import numbered stills as an image sequence (optional, default false).
#[serde(default)]
pub image_sequence: Option<bool>,
}
/// `file.import` params for a `media_import` call.
fn media_import_params(p: MediaImportParams) -> Value {
let mut params = json!({"paths": p.paths});
if let Some(bin) = p.bin {
params["bin"] = json!(bin);
}
if let Some(seq) = p.image_sequence {
params["imageSequence"] = json!(seq);
}
params
}
#[derive(Debug, Default, Deserialize, JsonSchema)]
pub struct ElementsParams {
/// Only element ids starting with this prefix (e.g. `timeline.clip.`).
@@ -381,9 +405,8 @@ impl FilmcraftMcp {
description = "Import media files by absolute path (MP4/MOV, WAV/MP3/FLAC/AIFF/Ogg, PNG/JPEG/…).",
annotations(read_only_hint = false, destructive_hint = false, idempotent_hint = false, open_world_hint = false)
)]
async fn media_import(&self, Parameters(p): Parameters<TypeParams>) -> Result<CallToolResult, McpError> {
let paths: Vec<&str> = p.text.split('\n').map(str::trim).filter(|s| !s.is_empty()).collect();
wrap(self.run("file.import", json!({"paths": paths})).await)
async fn media_import(&self, Parameters(p): Parameters<MediaImportParams>) -> Result<CallToolResult, McpError> {
wrap(self.run("file.import", media_import_params(p)).await)
}
#[tool(
@@ -402,8 +425,8 @@ impl FilmcraftMcp {
let t = p.seconds.map_or_else(|| g.playhead(), filmcraft_time::Tick::from_seconds_f64);
let (w, h) =
g.active_sequence().map(|q| (q.settings.width, q.settings.height)).ok_or_else(|| AutomationError::Other("no sequence".into()))?;
let scale = (max as f32 / w.max(h) as f32).min(1.0);
let img = g.render_program_at(scale, t).ok_or_else(|| AutomationError::Other("no sequence".into()))?;
let scale = render_scale(max, w, h);
let img = g.try_render_program_at(scale, t).map_err(|e| AutomationError::Other(e.to_string()))?;
png_rgba(img.w as u32, img.h as u32, img.over_black_rgba8(), max)
})
.await
@@ -648,10 +671,26 @@ impl ServerHandler for FilmcraftMcp {
}
}
/// Render scale that fits the longest side of a `w` x `h` sequence into `max_side` (never upscales; 0 = no limit, as in `png_rgba`).
fn render_scale(max_side: u32, w: u32, h: u32) -> f32 {
if max_side == 0 {
return 1.0;
}
(max_side as f32 / w.max(h).max(1) as f32).min(1.0)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn render_scale_zero_means_no_limit() {
assert_eq!(render_scale(0, 1920, 1080), 1.0);
assert_eq!(render_scale(960, 1920, 1080), 0.5);
assert_eq!(render_scale(960, 640, 360), 1.0);
assert_eq!(render_scale(960, 0, 0), 1.0);
}
#[tokio::test]
async fn headless_commands_and_render() {
let mut s = Session::default();
@@ -719,6 +758,21 @@ mod tests {
}
}
/// #499: `media_import` takes `paths` (a required array), and forwards `bin` / `image_sequence` to `file.import`.
#[test]
fn media_import_schema_and_params() {
let m = FilmcraftMcp::headless(Session::default());
let def = m.tool_router.get("media_import").unwrap();
let schema = Value::Object((*def.input_schema).clone());
assert_eq!(schema["properties"]["paths"]["type"], "array", "{schema}");
assert!(schema["required"].as_array().unwrap().iter().any(|r| r == "paths"), "{schema}");
let p: MediaImportParams = serde_json::from_value(json!({"paths": ["/a.png"], "bin": 7, "image_sequence": true})).unwrap();
assert_eq!(media_import_params(p), json!({"paths": ["/a.png"], "bin": 7, "imageSequence": true}));
let p: MediaImportParams = serde_json::from_value(json!({"paths": ["/a.wav"]})).unwrap();
assert_eq!(media_import_params(p), json!({"paths": ["/a.wav"]}));
assert!(serde_json::from_value::<MediaImportParams>(json!({"text": "/a.wav"})).is_err());
}
fn demo() -> Session {
let mut s = Session::default();
s.execute("file.openDemoProject", json!({})).unwrap();
+1 -1
View File
@@ -7,7 +7,7 @@ use common::*;
#[test]
#[ignore]
fn adhoc_compare() {
let Some(ff) = ffmpeg() else { return };
let Some(ff) = ffmpeg_dav1d() else { return };
let Some(path) = std::env::var_os("AV1_IVF") else { return };
let path = std::path::PathBuf::from(path);
let pics = decode_all(&path).unwrap();
+20 -1
View File
@@ -6,8 +6,27 @@ use filmcraft_av1::{Decoder, Picture};
use std::path::{Path, PathBuf};
use std::process::Command;
/// ffmpeg with libsvtav1 (fixtures) and libdav1d (reference), or `None` after reporting a skip.
pub fn ffmpeg() -> Option<PathBuf> {
filmcraft_testkit::ffmpeg_or_skip("av1 oracle")
ffmpeg_with(&[("-encoders", "libsvtav1"), ("-decoders", "libdav1d")])
}
/// ffmpeg with libdav1d, for checks that decode existing files only.
pub fn ffmpeg_dav1d() -> Option<PathBuf> {
ffmpeg_with(&[("-decoders", "libdav1d")])
}
/// Builds such as the Windows "essentials" ones lack these libraries; skip rather than fail.
fn ffmpeg_with(needs: &[(&str, &str)]) -> Option<PathBuf> {
let ff = filmcraft_testkit::ffmpeg_or_skip("av1 oracle")?;
for &(list, lib) in needs {
let out = Command::new(&ff).args(["-hide_banner", list]).output().map(|o| String::from_utf8_lossy(&o.stdout).into_owned()).unwrap_or_default();
if !out.contains(lib) {
filmcraft_testkit::skip("av1 oracle", &format!("ffmpeg has no {lib} (see `ffmpeg {list}`)"));
return None;
}
}
Some(ff)
}
pub fn fixture_dir() -> PathBuf {
+2 -2
View File
@@ -9,7 +9,7 @@ const VECTORS: &[&str] =
#[test]
fn conformance_vectors() {
let Some(ff) = ffmpeg() else { return };
let Some(ff) = ffmpeg_dav1d() else { return };
for name in VECTORS {
let Some(path) = test_vector(name) else { return };
check_file(&ff, name, &path);
@@ -21,7 +21,7 @@ fn conformance_vectors() {
#[test]
#[ignore]
fn conformance_vectors_extended() {
let Some(ff) = ffmpeg() else { return };
let Some(ff) = ffmpeg_dav1d() else { return };
let list: Vec<String> = match std::env::var("AV1_VECTORS") {
Ok(v) => v.split(',').map(str::to_string).collect(),
Err(_) => EXTENDED.iter().map(|s| s.to_string()).collect(),
+32 -13
View File
@@ -49,13 +49,10 @@ impl Overlay {
}
}
/// The caption face: Inter SemiBold at `px`.
fn caption_style(px: f32) -> TextStyle {
TextStyle { style: "SemiBold".into(), size: px, ..Default::default() }
}
fn measure(text: &str, px: f32) -> f32 {
filmcraft_text::measure(text, &caption_style(px))
/// The caption face: the track's font family (Inter when empty or not installed) in SemiBold at `px`.
fn caption_style(font: &str, px: f32) -> TextStyle {
let family = if font.trim().is_empty() { filmcraft_text::fonts::DEFAULT_FAMILY } else { font.trim() };
TextStyle { family: family.into(), style: "SemiBold".into(), size: px, ..Default::default() }
}
fn lin(c: [u8; 4]) -> [f32; 4] {
@@ -64,12 +61,12 @@ fn lin(c: [u8; 4]) -> [f32; 4] {
}
/// Word-wrap one line to `max_w` pixels.
fn wrap(line: &str, px: f32, max_w: f32) -> Vec<String> {
fn wrap(line: &str, ts: &TextStyle, max_w: f32) -> Vec<String> {
let mut out = Vec::new();
let mut cur = String::new();
for word in line.split(' ') {
let cand = if cur.is_empty() { word.to_string() } else { format!("{cur} {word}") };
if !cur.is_empty() && measure(&cand, px) > max_w {
if !cur.is_empty() && filmcraft_text::measure(&cand, ts) > max_w {
out.push(std::mem::take(&mut cur));
cur = word.to_string();
} else {
@@ -115,16 +112,17 @@ pub fn render_caption(c: &Caption, style: &CaptionStyle, w: usize, h: usize) ->
let scale = h as f32 / 1080.0;
let px = (style.size * scale).max(4.0);
let max_w = w as f32 * 0.9;
let ts = caption_style(&style.font, px);
let mut lines: Vec<String> = Vec::new();
// (speaker names are metadata; like Premiere, they are not burned in)
for l in c.plain_lines() {
lines.extend(wrap(l.trim(), px, max_w));
lines.extend(wrap(l.trim(), &ts, max_w));
}
lines.retain(|l| !l.is_empty());
if lines.is_empty() {
return None;
}
let vm = filmcraft_text::fonts::face(filmcraft_text::resolve("Inter", "SemiBold").face).metrics(px);
let vm = filmcraft_text::fonts::face(filmcraft_text::resolve(&ts.family, &ts.style).face).metrics(px);
let (asc, desc) = (vm.ascent, vm.descent);
let lh = px * style.line_spacing.max(0.8);
let pad_x = (px * 0.3).round();
@@ -141,7 +139,7 @@ pub fn render_caption(c: &Caption, style: &CaptionStyle, w: usize, h: usize) ->
.max(0.0)
.round();
let side = w as f32 * 0.05;
let widths: Vec<f32> = lines.iter().map(|l| measure(l, px)).collect();
let widths: Vec<f32> = lines.iter().map(|l| filmcraft_text::measure(l, &ts)).collect();
let xs: Vec<f32> = widths
.iter()
.map(|&lw| match align {
@@ -176,7 +174,7 @@ pub fn render_caption(c: &Caption, style: &CaptionStyle, w: usize, h: usize) ->
}
}
let baseline = lt + (lh - (asc + desc)) / 2.0 + asc;
let l = layout(line, &caption_style(px), &ParagraphStyle::default());
let l = layout(line, &ts, &ParagraphStyle::default());
let mut m = filmcraft_text::Mask { w: ow, h: oh, a: std::mem::take(&mut cover) };
render::draw(&l, &render::at(xs[i], baseline.round()), &mut m, (ox, oy));
cover = m.a;
@@ -305,6 +303,27 @@ mod tests {
assert!(o.px.chunks(4).any(|p| p[0] > 0.9 && p[1] < 0.05 && p[3] > 0.9), "red outline pixels");
}
#[test]
fn track_font_is_used() {
// the style's font used to be ignored (always Inter)
let inter = render_caption(&cap("Hello world"), &CaptionStyle::default(), 1920, 1080).unwrap();
let serif = CaptionStyle { font: "Noto Serif".into(), ..Default::default() };
let serif = render_caption(&cap("Hello world"), &serif, 1920, 1080).unwrap();
assert_ne!((inter.w, &inter.px), (serif.w, &serif.px), "Noto Serif renders like Inter");
let mono = CaptionStyle { font: "jetbrains mono".into(), ..Default::default() };
let mono = render_caption(&cap("Hello world"), &mono, 1920, 1080).unwrap();
assert_ne!((inter.w, &inter.px), (mono.w, &mono.px), "JetBrains Mono renders like Inter");
}
#[test]
fn unknown_or_empty_font_falls_back_to_inter() {
let inter = render_caption(&cap("Hello"), &CaptionStyle::default(), 1280, 720).unwrap();
for font in ["", " ", "No Such Font 123"] {
let st = CaptionStyle { font: font.into(), ..Default::default() };
assert_eq!(render_caption(&cap("Hello"), &st, 1280, 720), Some(inter.clone()), "font {font:?}");
}
}
#[test]
fn hidden_track_draws_nothing() {
let mut t = CaptionTrack::new(TrackId(1), "S".into(), CaptionFormat::Subtitle);
+4 -1
View File
@@ -307,7 +307,8 @@ pub fn parse_clock(s: &str) -> Option<Tick> {
/// Format ticks as `HH:MM:SS<sep>mmm` (rounded to the nearest millisecond).
pub fn format_clock(t: Tick, sep: char) -> String {
let ms = (t.0.max(0) + TICKS_PER_MS / 2) / TICKS_PER_MS;
// The rounding addition must not overflow for an input-derived Tick near i64::MAX.
let ms = (i128::from(t.0.max(0)) + i128::from(TICKS_PER_MS) / 2) / i128::from(TICKS_PER_MS);
let (h, m, s, f) = (ms / 3_600_000, (ms / 60_000) % 60, (ms / 1000) % 60, ms % 1000);
format!("{h:02}:{m:02}:{s:02}{sep}{f:03}")
}
@@ -368,6 +369,8 @@ mod tests {
assert_eq!(parse_clock("00:61:00,000"), None);
assert_eq!(parse_clock("aa:00:00,000"), None);
assert_eq!(format_clock(Tick(TICKS_PER_SECOND * 3723 + TICKS_PER_MS * 45), ','), "01:02:03,045");
assert!(format_clock(Tick(i64::MAX), ',').ends_with(",077"));
assert_eq!(format_clock(Tick(i64::MIN), '.'), "00:00:00.000");
}
#[test]
+10 -2
View File
@@ -17,7 +17,8 @@
//! - **CDP (SMPTE ST 334-2):** identifier `96 69`, length, frame-rate code, flags, sequence
//! counter, a `cc_data` section (`72`, `E0 | cc_count`, cc_count × (marker/valid/type, two
//! bytes)), optional time code / service info sections, footer `74`, the counter again and a
//! checksum that makes the packet's byte sum 0 mod 256.
//! checksum that makes the packet's byte sum 0 mod 256. The ancillary packet then ends with one ST 291 checksum byte
//! (low 8 bits of DID + SDID + data count + CDP bytes).
//! - **Writing** (29.97 fps, `30DF` or `30` timecode labels): every frame from the first caption
//! data to one frame after the last carries a CDP with 20 cc_data triplets: field 1 CEA-608
//! pop-on data (the same schedule as [SCC](crate::scc)), a null field-2 pair, CEA-708
@@ -161,6 +162,9 @@ pub fn build_cdp(seq: u16, data: &FrameData) -> Vec<u8> {
cdp.push(((256 - sum % 256) % 256) as u8);
let mut out = vec![0x61, 0x01, cdp.len() as u8];
out.extend(cdp);
// SMPTE ST 291 ancillary checksum: low 8 bits of the sum of DID, SDID, data count and the user data words.
let anc_sum: u32 = out.iter().map(|&b| b as u32).sum();
out.push((anc_sum & 0xff) as u8);
out
}
@@ -661,7 +665,11 @@ mod tests {
let d = FrameData { f1: Some((0x14, 0x20)), dtvcc: vec![(true, 0x02, 0x21), (false, 0x89, 0x01)] };
let b = build_cdp(7, &d);
assert_eq!(&b[..3], &[0x61, 0x01, 73]);
let cdp = &b[3..];
// prefix (3) + CDP (73) + ancillary checksum byte (1)
assert_eq!(b.len(), 77);
let sum: u32 = b[..76].iter().map(|&x| x as u32).sum();
assert_eq!(b[76], (sum & 0xff) as u8, "ancillary checksum");
let cdp = &b[3..76];
assert_eq!(cdp.len(), 73);
assert_eq!(&cdp[..9], &[0x96, 0x69, 73, 0x4f, 0x43, 0, 7, 0x72, 0xf4]);
assert_eq!(cdp.iter().map(|&x| x as u32).sum::<u32>() % 256, 0, "checksum");
+31 -8
View File
@@ -215,15 +215,22 @@ fn encode_text(text: &str) -> Vec<u8> {
if ch == '<'
&& let Some(end) = rest.find('>')
{
match rest[1..end].trim().to_ascii_lowercase().as_str() {
"i" => out.push(0x80),
"/i" => out.push(0x81),
"u" => out.push(0x82),
"/u" => out.push(0x83),
_ => {}
let tag = &rest[1..end];
// Only a syntactically plausible tag is markup. Comparisons such as
// "2 < 3 > 1" and "x < y > z" must not lose their middle text.
let name = tag.strip_prefix('/').unwrap_or(tag);
let is_tag = name.as_bytes().first().is_some_and(u8::is_ascii_alphabetic);
if is_tag {
match tag.trim().to_ascii_lowercase().as_str() {
"i" => out.push(0x80),
"/i" => out.push(0x81),
"u" => out.push(0x82),
"/u" => out.push(0x83),
_ => {}
}
rest = &rest[end + 1..];
continue;
}
rest = &rest[end + 1..];
continue;
}
if ch == '\n' {
out.push(0x8a);
@@ -451,6 +458,22 @@ mod tests {
assert!(parse(&b[..500]).is_err());
}
#[test]
fn literal_angle_brackets_are_not_discarded_as_markup() {
let literal = "2 < 3 > 1, x < y > z, and 5 <7> 6";
let doc = Document {
cues: vec![Cue { start: FrameRate::FPS_25.tick_of(25), end: FrameRate::FPS_25.tick_of(75), text: literal.into(), ..Default::default() }],
..Default::default()
};
let encoded = write(&doc, Some(FrameRate::FPS_25));
let decoded = parse(&encoded).expect("valid EBU STL");
assert_eq!(decoded.cues.len(), 1);
assert_eq!(decoded.cues[0].text, literal);
// Recognized styling still becomes STL control bytes.
assert_eq!(encode_text("<i>Hi</i>"), vec![0x80, b'H', b'i', 0x81]);
assert_eq!(encode_text("<i >Hi</i >"), vec![0x80, b'H', b'i', 0x81]);
}
#[test]
fn programme_start_is_subtracted() {
let doc = Document {
+43 -5
View File
@@ -252,6 +252,10 @@ fn parse_time(s: &str, tm: Timing) -> Option<Tick> {
let (h, m) = (num(parts.first()?)?, num(parts.get(1)?)?);
let sec_part = parts.get(2)?;
let (si, sf) = sec_part.split_once('.').unwrap_or((sec_part, ""));
let seconds = num(si)?;
if m >= 60 || seconds >= 60 {
return None;
}
let mut t = (h * 3600 + m * 60) * tps + decimal_ticks(si, sf, tps)?;
if let Some(fp) = parts.get(3) {
if parts.len() > 4 {
@@ -259,13 +263,15 @@ fn parse_time(s: &str, tm: Timing) -> Option<Tick> {
}
let (fi, fs) = fp.split_once('.').unwrap_or((fp, ""));
let frames = num(fi)?;
let _ = tm.base;
let sub: i128 = if fs.is_empty() { 0 } else { num(fs)? };
if frames >= i128::from(tm.base) || sub >= i128::from(tm.sub_frame_rate) {
return None;
}
// frames at the effective rate; sub-frames divide a frame
let ft = tm.rate.tick_of(1).0 as i128;
t += frames * ft + sub * ft / tm.sub_frame_rate.max(1) as i128;
}
return Some(Tick(t as i64));
return i64::try_from(t).ok().map(Tick);
}
let unit_start = s.find(|c: char| c.is_ascii_alphabetic())?;
let (v, unit) = s.split_at(unit_start);
@@ -281,8 +287,12 @@ fn parse_time(s: &str, tm: Timing) -> Option<Tick> {
"f" => {
let n: i128 = vi.parse().ok()?;
let fd = tm.rate.tick_of(1).0 as i128;
let whole = tm.rate.tick_of(n as i64).0 as i128;
whole + decimal_ticks("0", vf, fd)?
let whole = tm.rate.tick_of(i64::try_from(n).ok()?);
// tick_of saturates; a saturated count overflowed rather than named a real time
if whole == Tick(i64::MAX) {
return None;
}
whole.0 as i128 + decimal_ticks("0", vf, fd)?
}
"t" => {
let n: i128 = vi.parse().ok()?;
@@ -290,7 +300,7 @@ fn parse_time(s: &str, tm: Timing) -> Option<Tick> {
}
_ => return None,
};
Some(Tick(t as i64))
i64::try_from(t).ok().map(Tick)
}
#[derive(Clone, Copy, Default, PartialEq)]
@@ -552,6 +562,34 @@ mod tests {
assert_eq!(parse_time("1:2", t), None);
}
#[test]
fn invalid_and_overflowed_time_expressions_are_ignored() {
let normal = tm(FrameRate::FPS_25);
for expr in [
"999999999999999h",
"999999999999999m",
"999999999999999s",
"999999999999999ms",
"999999999999999f",
"999999999999999t",
"00:60:00",
"00:00:60",
"00:00:01:25",
] {
assert_eq!(parse_time(expr, normal), None, "{expr}");
}
let subframes = Timing { sub_frame_rate: 2, ..normal };
assert!(parse_time("00:00:01:12.1", subframes).is_some());
assert_eq!(parse_time("00:00:01:12.2", subframes), None);
// Bad cues must not wrap to a plausible but unrelated timeline position.
let xml =
r#"<tt xmlns="http://www.w3.org/ns/ttml"><body><div><p begin="999999999999999s" end="2s">Bad</p><p begin="1s" end="2s">Good</p></div></body></tt>"#;
let doc = parse(xml).expect("well-formed TTML");
assert_eq!(doc.cues.len(), 1);
assert_eq!(doc.cues[0].text, "Good");
assert!(!doc.warnings.is_empty());
}
#[test]
fn reads_styles_spans_and_nesting() {
let x = r#"<?xml version="1.0"?>
+10 -5
View File
@@ -84,7 +84,6 @@ impl<'a> CompoundFile<'a> {
let mini_cutoff = u32_at(data, 56) as u64;
let first_minifat = u32_at(data, 60);
let first_difat = u32_at(data, 68);
let n_difat = u32_at(data, 72) as usize;
let mut header_clsid = [0u8; 16];
header_clsid.copy_from_slice(&data[8..24]);
let max_sectors = data.len() / sector_size + 1;
@@ -108,8 +107,10 @@ impl<'a> CompoundFile<'a> {
let mut seen = 0;
while d <= MAXREGSECT && fat_sectors.len() < n_fat {
seen += 1;
if seen > n_difat.max(1) + max_sectors {
return Err(invalid("DIFAT chain loops"));
// a DIFAT chain cannot have more sectors than the file holds; the header's DIFAT sector
// count (up to 2^32 - 1) is not trusted, or a self-pointing sector loops that often
if seen > max_sectors {
return Err(invalid(format!("DIFAT chain loops: more than {max_sectors} sectors in a {}-byte file", data.len())));
}
let s = cf.sector(d)?;
let per = sector_size / 4 - 1;
@@ -169,13 +170,17 @@ impl<'a> CompoundFile<'a> {
let mut out = Vec::new();
let mut s = start;
let mut steps = 0usize;
// A chain cannot visit more sectors than the file holds. Not `self.fat.len()`: the FAT can
// be far longer (one FAT sector named by many header / DIFAT slots), and a looping chain
// would then append a whole sector per FAT entry (hundreds of MB, or an abort).
let max_steps = self.data.len() / self.sector_size + 1;
while s != ENDOFCHAIN {
if s > MAXREGSECT {
return Err(invalid(format!("bad sector {s:#x} in a chain")));
}
steps += 1;
if steps > self.fat.len().max(1) {
return Err(invalid("sector chain loops"));
if steps > max_steps {
return Err(invalid(format!("sector chain loops: more than {max_steps} sectors in a {}-byte file", self.data.len())));
}
let b = self.sector(s)?;
out.extend_from_slice(b);
+92
View File
@@ -199,3 +199,95 @@ fn mutation_never_panics_or_hangs() {
}
}
}
fn put_u32(b: &mut [u8], at: usize, v: u32) {
b[at..at + 4].copy_from_slice(&v.to_le_bytes());
}
/// A 512-byte header (§2.2) with sector shift `shift`: no directory, mini FAT or DIFAT chain, and
/// all 109 FAT slots free. Tests fill in the fields they attack.
fn bare_header(shift: u16) -> Vec<u8> {
let mut h = vec![0u8; 512];
h[..8].copy_from_slice(&SIGNATURE);
h[24..26].copy_from_slice(&0x3Eu16.to_le_bytes());
h[26..28].copy_from_slice(&(if shift == 9 { 3u16 } else { 4 }).to_le_bytes());
h[28..30].copy_from_slice(&0xFFFEu16.to_le_bytes());
h[30..32].copy_from_slice(&shift.to_le_bytes());
h[32..34].copy_from_slice(&6u16.to_le_bytes());
for at in [48, 60, 68] {
put_u32(&mut h, at, ENDOFCHAIN);
}
put_u32(&mut h, 56, MINI_STREAM_CUTOFF);
h[76..].fill(0xFF);
h
}
/// A DIFAT sector that names no FAT sector and points to itself. The loop guard used to be the
/// header's DIFAT sector count (up to 2^32 - 1), so this 1 KB file kept `open` (and so AAF
/// sniffing) busy for 14-16 s here (minutes with `u32::MAX`), reading a sector each pass. A DIFAT
/// chain cannot be longer than the file, so the walk now stops after as many passes as the file
/// has sectors (1024 / 512 + 1 = 3 here) and fails on the 4th.
#[test]
fn difat_self_loop_fails_fast() {
let mut b = bare_header(9);
put_u32(&mut b, 44, 1); // one FAT sector, to be found through the DIFAT
put_u32(&mut b, 68, 0); // first DIFAT sector
put_u32(&mut b, 72, 100_000_000); // claimed DIFAT sector count (inflates the old loop bound)
let mut difat = vec![0xFF; 512]; // no FAT sector numbers...
put_u32(&mut difat, 508, 0); // ...and the next DIFAT sector is this one
b.extend_from_slice(&difat);
// The message names the bound the walk stopped at: the file's 3 sectors, not the header's
// 100,000,000 (the old guard allowed 100,000,003 passes).
let expect = format!("DIFAT chain loops: more than 3 sectors in a {}-byte file", b.len());
assert_eq!(CompoundFile::open(&b).unwrap_err(), Error::Invalid(expect));
}
/// All 109 header slots name the same FAT sector, so the FAT is 109 copies of it, and the
/// directory chain loops (sector 0 → 0). The loop guard used to be the FAT length, so this 450 KB
/// file appended a sector per FAT entry before failing: a 457 MB buffer per open (117 GB with
/// 64 KB sectors in a 7 MB file). A chain cannot visit more sectors than the file has.
#[test]
fn duplicated_fat_sector_chain_loop_fails_fast() {
let ss = 4096;
let mut b = bare_header(12);
put_u32(&mut b, 44, 109); // FAT sectors: the 109 header slots...
for i in 0..109 {
put_u32(&mut b, 76 + i * 4, 0); // ...all naming sector 0
}
put_u32(&mut b, 48, 0); // the directory starts at sector 0
b.resize(ss, 0);
let mut fat = vec![0xFF; ss];
put_u32(&mut fat, 0, 0); // FAT[0] = 0: sector 0 is followed by itself
b.extend_from_slice(&fat);
b.resize(110 * ss, 0); // big enough for 109 FAT sectors to pass the size check
// The message names the bound the walk stopped at: 111, this file's 110 sectors + 1, not the
// 109 * 1024 = 111,616 entries of the inflated FAT that the old guard allowed (a 457 MB buffer).
let expect = format!("sector chain loops: more than 111 sectors in a {}-byte file", b.len());
assert_eq!(CompoundFile::open(&b).unwrap_err(), Error::Invalid(expect));
}
/// Positive control for the chain bound: the longest chains a file can hold still read back, in
/// both versions: a directory (read without a size limit) and a stream, each filling every sector
/// but the header and the FAT (and the directory, for the stream).
#[test]
fn chains_filling_the_file_still_read() {
for v in [Version::V3, Version::V4] {
let ss = v_sector(v);
let mut w = Writer::new(v);
for i in 0..1000 {
w.stream(Writer::ROOT, &format!("e{i}"), Vec::new()).unwrap();
}
let b = w.finish();
assert!(b.len() / ss <= (1001 * 128usize).div_ceil(ss) + 3, "header, directory, FAT");
let cf = CompoundFile::open(&b).unwrap();
assert_eq!(cf.root().children.len(), 1000);
assert_eq!(cf.read_path("e999").unwrap(), Vec::<u8>::new());
let mut w = Writer::new(v);
let data = pattern(64 * ss, 3);
w.stream(Writer::ROOT, "big", data.clone()).unwrap();
let b = w.finish();
assert_eq!(b.len(), (1 + 64 + 2) * ss, "header, stream, directory, FAT");
assert_eq!(CompoundFile::open(&b).unwrap().read_path("big").unwrap(), data);
}
}
+1
View File
@@ -41,3 +41,4 @@ symphonia = { version = "0.5.5", default-features = false, features = ["aac", "m
[dev-dependencies]
filmcraft-testkit = { workspace = true }
serde_json = { workspace = true }
+1 -1
View File
@@ -114,7 +114,7 @@ impl ApvSource {
bitrate,
hdr: None,
}),
audio: None,
audio_streams: Vec::new(),
container: "APV".into(),
start_timecode: None,
file_size: Some(len),
+345 -14
View File
@@ -16,15 +16,19 @@ use crate::{CodecError, Result};
enum Inner {
/// Our own AAC-LC decoder. `up` interpolates HE-AAC's core to the output rate (see [`Upsample2x`]).
/// Surround output is reordered to WAV/SMPTE order (see [`aac_to_wav_order`]).
Aac { dec: Box<filmcraft_aac::Decoder>, asc: Vec<u8>, up: Option<Box<Upsample2x>> },
/// AAC whose configuration arrives with the first frame (ADTS / LATM).
AacPending,
/// MPEG audio at this rate whose layer arrives with the first frame (MP4 / QuickTime name the
/// codec, not the layer, and symphonia's decoder reads only the layer it was made for).
MpegPending(u32),
/// Our own Opus decoder (always 48 kHz output; pre-skip is left to container timestamps).
/// `order` maps output channel → decoded channel (Vorbis → WAV/SMPTE order for surround).
Opus { dec: Box<filmcraft_opus::Decoder>, order: Option<&'static [usize]> },
/// Bootstrap decoders (MP3, ALAC, FLAC, HE-AAC…) via symphonia.
Symphonia(Box<dyn Decoder>),
/// Our AC-3 decoder (ATSC A/52).
/// Our AC-3 and E-AC-3 decoder (ATSC A/52, Annex E).
Ac3(Box<filmcraft_ac3::Decoder>),
}
@@ -52,6 +56,29 @@ fn vorbis_to_wav_order(channels: usize) -> Option<&'static [usize]> {
})
}
/// Channel order for AAC surround (ISO/IEC 14496-3 Table 1.19, our decoder's element order: C, L R,
/// surrounds, LFE): output channel → decoded channel, giving the WAV/SMPTE order (L R C LFE Ls Rs)
/// used elsewhere, as the AC-3 decoder does and the AAC encoder's caller undoes on export. Only the
/// unambiguous channel configurations 3–6; 1, 2, 7 and PCE layouts keep the decoded order.
fn aac_to_wav_order(channel_config: u8) -> Option<&'static [usize]> {
Some(match channel_config {
3 => &[1, 2, 0],
4 => &[1, 2, 0, 3],
5 => &[1, 2, 0, 3, 4],
6 => &[1, 2, 0, 5, 3, 4],
_ => return None,
})
}
/// `channels` reordered by `order` (output channel → decoded channel). A decode whose channel count
/// does not match the order is returned unchanged.
fn reorder(mut channels: Vec<Vec<f32>>, order: Option<&[usize]>) -> Vec<Vec<f32>> {
match order {
Some(order) if order.len() == channels.len() => order.iter().map(|&c| channels.get_mut(c).map(std::mem::take).unwrap_or_default()).collect(),
_ => channels,
}
}
/// Duration of an Opus packet in 48 kHz samples from its TOC and frame count (RFC 6716 §3.1),
/// without parsing frame lengths. For a multistream packet this is the first stream's TOC, which
/// every stream shares. `None` for an empty or invalid (> 120 ms) packet.
@@ -87,7 +114,7 @@ impl PacketDecoder {
/// (HE-AAC, see [`aac_output_rate`]) the decoded core is upsampled to it.
pub fn aac(asc: &[u8], sample_rate: u32) -> Result<Self> {
if let Ok(dec) = filmcraft_aac::Decoder::new(asc) {
let up = (dec.sample_rate().checked_mul(2) == Some(sample_rate)).then(|| Box::new(Upsample2x::new(dec.channels())));
let up = upsamples_core(dec.sample_rate(), sample_rate).then(|| Box::new(Upsample2x::new(dec.channels())));
return Ok(Self { inner: Inner::Aac { dec: Box::new(dec), asc: asc.to_vec(), up }, channels: 0 });
}
Self::new(CODEC_TYPE_AAC, sample_rate, Some(asc.to_vec()))
@@ -109,6 +136,12 @@ impl PacketDecoder {
};
Self::new(codec, sample_rate, None)
}
/// MPEG-1/2 audio of a layer the frame headers name (see [`mpeg_audio_layer`]): MP4's
/// `esds` object types 0x69 / 0x6B and QuickTime's `.mp3` / `.mp2` cover layers I–III alike
/// (ffmpeg stores MP2 as `mp4a` + 0x6B).
pub fn lazy_mpeg_audio(sample_rate: u32) -> Self {
Self { inner: Inner::MpegPending(sample_rate), channels: 0 }
}
/// AAC configured by [`Self::ensure_aac`] from in-band headers (ADTS, LATM).
pub fn lazy_aac() -> Self {
Self { inner: Inner::AacPending, channels: 0 }
@@ -124,7 +157,7 @@ impl PacketDecoder {
self.inner = Inner::Aac { dec: Box::new(dec), asc: asc.to_vec(), up: None };
Ok(())
}
/// AC-3 (ATSC A/52).
/// AC-3 and E-AC-3 (ATSC A/52, Annex E).
pub fn ac3() -> Result<Self> {
Ok(Self { inner: Inner::Ac3(Box::new(filmcraft_ac3::Decoder::new())), channels: 0 })
}
@@ -136,8 +169,8 @@ impl PacketDecoder {
use filmcraft_isobmff::CodecConfig as C;
match c {
C::Aac(a) => Self::aac(&a.asc, if rate > 0 { rate } else { a.sample_rate }),
C::Mp3 => Self::new(CODEC_TYPE_MP3, rate, None),
C::Ac3 { .. } => Self::ac3(),
C::Mp3 => Ok(Self::lazy_mpeg_audio(rate)),
C::Ac3 { .. } | C::Eac3 { .. } => Self::ac3(),
C::Alac { cookie } => Self::new(CODEC_TYPE_ALAC, rate, Some(cookie.clone())),
C::Flac(_) => Self::new(CODEC_TYPE_FLAC, rate, None),
C::Opus(o) => Self::opus(filmcraft_opus::OpusHead::from_dops(&o.to_bytes()).map_err(|e| CodecError::Unsupported(format!("Opus: {e}")))?),
@@ -146,36 +179,44 @@ impl PacketDecoder {
}
/// Decode one packet into planar channels.
pub fn decode(&mut self, data: &[u8], ts: u64) -> Result<Vec<Vec<f32>>> {
if let Inner::MpegPending(rate) = self.inner {
let layer = mpeg_audio_layer(data).ok_or_else(|| CodecError::Decode("MPEG audio: no frame header".into()))?;
self.inner = Self::mpeg_audio(layer, rate)?.inner;
}
let dec = match &mut self.inner {
Inner::Aac { dec, up, .. } => {
let mut out = dec.decode(data).map_err(|e| CodecError::Decode(e.to_string()))?;
if let Some(up) = up {
out = up.process(out);
}
let out = reorder(out, aac_to_wav_order(dec.config().channel_config));
self.channels = out.len();
return Ok(out);
}
Inner::Opus { dec, order } => {
// A corrupt packet is concealed like a lost one (keeps timing and decoder state).
let mut out = match dec.decode(Some(data)) {
let out = match dec.decode(Some(data)) {
Ok(o) => o,
Err(_) => dec.decode(None).map_err(|e| CodecError::Decode(e.to_string()))?,
};
if let Some(order) = order {
out = order.iter().map(|&c| std::mem::take(&mut out[c])).collect();
}
let out = reorder(out, *order);
self.channels = out.len();
return Ok(out);
}
Inner::Symphonia(d) => d,
Inner::AacPending => return Err(CodecError::Decode("AAC: no configuration yet".into())),
Inner::MpegPending(_) => return Err(CodecError::Decode("MPEG audio: no frame header".into())),
Inner::Ac3(d) => {
// a packet may hold several syncframes (MP4 / Matroska)
// a packet may hold several syncframes (MP4 / Matroska); E-AC-3 dependent
// substreams and further programs decode to no channels and are skipped
let mut out: Vec<Vec<f32>> = Vec::new();
let mut p = 0;
while p < data.len() {
let f = d.decode(&data[p..]).map_err(|e| CodecError::Decode(e.to_string()))?;
p += f.header.frame_bytes;
if f.channels.is_empty() {
continue;
}
if out.is_empty() {
out = f.channels;
} else {
@@ -218,11 +259,21 @@ impl PacketDecoder {
Inner::Opus { dec, .. } => dec.reset(),
Inner::Symphonia(d) => d.reset(),
Inner::Ac3(d) => d.reset(),
Inner::AacPending => {}
Inner::AacPending | Inner::MpegPending(_) => {}
}
}
}
/// The layer (1, 2 or 3) named by the MPEG-1/2 audio frame header that starts `frame`: the 12-bit
/// sync word (11 bits for MPEG-2.5), then `ID` and the 2-bit `layer` field, 3 for layer I down to 1
/// for layer III (ISO/IEC 11172-3 §2.4.2.3). `None` without a sync word or for the reserved layer 0.
pub fn mpeg_audio_layer(frame: &[u8]) -> Option<u8> {
match *frame {
[0xFF, b1, ..] if b1 & 0xE0 == 0xE0 && (b1 >> 1) & 3 != 0 => Some(4 - ((b1 >> 1) & 3)),
_ => None,
}
}
/// The output rate of an AAC stream: twice the core rate for HE-AAC (ISO/IEC 14496-3 §1.6.5).
/// Explicit signalling names the rate in the `AudioSpecificConfig`; with implicit signalling (an
/// AAC-LC config at 24 kHz or less) SBR is only visible as fill-element data in the access units,
@@ -244,6 +295,95 @@ pub fn aac_output_rate<'a>(asc: &[u8], first_units: impl IntoIterator<Item = &'a
Some(core)
}
/// Whether [`PacketDecoder::aac`] upsamples a decoded AAC core at `core` Hz to `rate` (HE-AAC).
fn upsamples_core(core: u32, rate: u32) -> bool {
core.checked_mul(2) == Some(rate)
}
/// A codec whose packets all decode to the same number of samples (see [`fixed_packet_samples`]).
#[derive(Clone, Copy, Debug)]
pub enum FixedFrames<'a> {
/// AAC with this AudioSpecificConfig.
Aac(&'a [u8]),
/// MPEG-1/2 audio, layer I, II or III.
MpegAudio,
/// AC-3 (and E-AC-3 in a stream of single syncframes, MPEG-TS).
Ac3,
/// E-AC-3 packets of whole syncframes (MP4, Matroska): an access unit may hold several
/// syncframes and dependent substreams.
Eac3,
}
/// Samples each packet decodes to at the stream's output `rate`, for codecs where that is the same
/// for every packet: AAC 1024 (2048 when [`PacketDecoder::aac`] upsamples an HE-AAC core); MPEG
/// audio and AC-3 from the frame header of `first`, the stream's first packet; E-AC-3 from the
/// syncframes of the program we decode in `first`. `None` when it is not known: an AAC
/// configuration our decoder does not read, a header that does not parse, or a header whose rate
/// is not `rate`.
pub fn fixed_packet_samples(codec: FixedFrames, first: &[u8], rate: u32) -> Option<i64> {
let mpegts = match codec {
FixedFrames::Aac(asc) => {
let core = filmcraft_aac::Decoder::new(asc).ok()?.sample_rate();
return Some(if upsamples_core(core, rate) { 2048 } else { 1024 });
}
FixedFrames::Eac3 => {
let mut samples = 0usize;
let mut p = 0;
while let Some(h) = first.get(p..).and_then(|b| filmcraft_ac3::parse_header(b).ok()) {
if h.is_primary() {
if h.sample_rate != rate {
return None;
}
samples += h.samples();
}
p += h.frame_bytes;
}
return i64::try_from(samples).ok().filter(|&n| n > 0);
}
FixedFrames::MpegAudio => filmcraft_mpegts::Codec::MpegAudio,
FixedFrames::Ac3 => filmcraft_mpegts::Codec::Ac3,
};
let f = filmcraft_mpegts::frame_info(&mpegts, first)?;
(f.sample_rate == rate && f.samples > 0).then_some(f.samples as i64)
}
/// Where the container puts one audio packet, for [`contiguous_starts`].
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct PacketTime {
/// Its timestamp, in output samples.
pub stamp: i64,
/// Whether `stamp` is the packet's own. A Matroska frame after the first in a laced block
/// without a duration repeats the block's timestamp.
pub stamped: bool,
/// Samples it decodes to, when known.
pub samples: Option<i64>,
}
/// Packet start positions in output samples: each packet starts where the previous one ends,
/// unless its own timestamp is more than `tolerance` samples from there (a real gap or overlap in
/// the recording) or the previous packet's length is unknown; then it starts at its timestamp.
///
/// Container timestamps are coarser than a sample. Matroska stores milliseconds, and a remux to MP4
/// (OBS's) turns that rounding into uneven sample durations: a 1024-sample AAC frame at 48 kHz lasts
/// 21.33 ms, so frames are stamped at 0, 1008, 2064, 3072… Placing each decoded packet at its
/// stamp overwrote the end of one packet and left a hole of silence after another, a click every
/// few packets: crackling throughout playback and export (#236).
pub fn contiguous_starts(packets: impl IntoIterator<Item = PacketTime>, tolerance: i64) -> Vec<i64> {
let tolerance = tolerance.max(0).unsigned_abs();
let mut next: Option<i64> = None;
packets
.into_iter()
.map(|p| {
let start = match next {
Some(n) if !p.stamped || n.abs_diff(p.stamp) <= tolerance => n,
_ => p.stamp,
};
next = p.samples.and_then(|k| start.checked_add(k));
start
})
.collect()
}
/// Taps on each side of the interpolation point in [`Upsample2x`].
const UPSAMPLE_HALF: usize = 12;
@@ -311,7 +451,7 @@ impl Upsample2x {
}
}
/// Whether AC-3 audio decodes (our ATSC A/52 decoder).
/// Whether AC-3 and E-AC-3 audio decode (our ATSC A/52 decoder).
pub const AC3_DECODER: bool = true;
/// An AudioSpecificConfig for AAC with these parameters (ISO/IEC 14496-3 §1.6.2.1, GA specific
@@ -666,12 +806,12 @@ impl AudioFileSource {
kind: MediaKind::AudioOnly,
duration: Tick::from_units(frames as i64, rate as i64),
video: None,
audio: Some(AudioStreamInfo {
audio_streams: vec![AudioStreamInfo {
sample_rate: rate,
channels: samples.len() as u32,
codec: codec_name,
bits_per_sample: track.codec_params.bits_per_sample,
}),
}],
container: ext.to_uppercase(),
start_timecode: None,
file_size: Some(bytes.len() as u64),
@@ -739,3 +879,194 @@ mod he_aac_tests {
}
}
}
#[cfg(test)]
mod surround_order_tests {
use super::*;
/// Power of `x` at `hz` (Goertzel), normalised by length.
fn power(x: &[f32], hz: f32, rate: f32) -> f32 {
let k = 2.0 * (std::f32::consts::TAU * hz / rate).cos();
let (mut s1, mut s2) = (0.0f32, 0.0f32);
for &v in x {
let s0 = v + k * s1 - s2;
s2 = s1;
s1 = s0;
}
(s1 * s1 + s2 * s2 - k * s1 * s2) / x.len() as f32
}
/// #721: 5.1 AAC (configuration 6, coded C L R Ls Rs LFE) decodes to WAV/SMPTE order
/// L R C LFE Ls Rs, as the rest of FilmCraft and the AAC export path expect.
#[test]
fn aac_51_decodes_in_wav_order() {
let rate = 48_000u32;
// tones per speaker, in WAV order: L, R, C, LFE, Ls, Rs
let hz = [300.0f32, 400.0, 500.0, 60.0, 700.0, 800.0];
let tone = |f: f32| (0..rate as usize).map(|i| 0.25 * (std::f32::consts::TAU * f * i as f32 / rate as f32).sin()).collect::<Vec<f32>>();
let wav: Vec<Vec<f32>> = hz.iter().map(|&f| tone(f)).collect();
// the encoder takes AAC element order (as the exporter feeds it): C L R Ls Rs LFE
let coded: Vec<&[f32]> = [2usize, 0, 1, 4, 5, 3].iter().map(|&c| wav[c].as_slice()).collect();
let mut enc = filmcraft_aac::Encoder::new(filmcraft_aac::EncoderConfig::cbr(rate, 6, 384_000)).unwrap();
let mut units = enc.encode(&coded);
units.extend(enc.flush());
let mut dec = PacketDecoder::aac(&enc.audio_specific_config(), rate).unwrap();
let mut out = vec![Vec::new(); 6];
for u in &units {
for (o, c) in out.iter_mut().zip(dec.decode(u, 0).unwrap()) {
o.extend(c);
}
}
assert_eq!(dec.channels, 6);
for (ch, samples) in out.iter().enumerate() {
let window = &samples[4800..38_400];
let own = power(window, hz[ch], rate as f32);
for (other, &f) in hz.iter().enumerate().filter(|&(o, _)| o != ch) {
let p = power(window, f, rate as f32);
assert!(own > 100.0 * p, "channel {ch}: {} Hz {own} vs {f} Hz (channel {other}) {p}", hz[ch]);
}
}
}
#[test]
fn reorder_leaves_mismatched_channel_counts_alone() {
let chans = vec![vec![1.0], vec![2.0]];
assert_eq!(reorder(chans.clone(), aac_to_wav_order(6)), chans);
assert_eq!(reorder(vec![vec![0.0], vec![1.0], vec![2.0]], aac_to_wav_order(3)), vec![vec![1.0], vec![2.0], vec![0.0]]);
}
}
#[cfg(test)]
mod packet_time_tests {
use super::*;
/// 1024-sample frames at 48 kHz stamped to the nearest millisecond, then back in samples.
fn ms_stamps(count: i64) -> Vec<i64> {
(0..count).map(|k| (k * 1024 * 1000 + 24_000) / 48_000 * 48).collect()
}
fn timed(stamps: &[i64], samples: Option<i64>) -> Vec<PacketTime> {
stamps.iter().map(|&stamp| PacketTime { stamp, stamped: true, samples }).collect()
}
#[test]
fn rounded_stamps_give_back_to_back_packets() {
let stamps = ms_stamps(200);
assert_eq!(&stamps[..4], [0, 1008, 2064, 3072]);
let starts = contiguous_starts(timed(&stamps, Some(1024)), 512);
assert_eq!(starts, (0..200).map(|k| k * 1024).collect::<Vec<_>>());
}
#[test]
fn real_gaps_and_overlaps_resynchronise() {
// a 100 ms gap after packet 3, then a packet stamped 600 samples early
let mut stamps: Vec<i64> = (0..8).map(|k| k * 1024).collect();
for s in &mut stamps[4..] {
*s += 4800;
}
stamps[7] -= 600;
let starts = contiguous_starts(timed(&stamps, Some(1024)), 512);
assert_eq!(starts, [0, 1024, 2048, 3072, 8896, 9920, 10944, 11368]);
// within the tolerance: runs on
stamps[7] += 200;
assert_eq!(contiguous_starts(timed(&stamps, Some(1024)), 512)[7], 11968);
}
#[test]
fn unstamped_packets_follow_on_and_unknown_lengths_resynchronise() {
// a laced block of three frames that repeat its timestamp
let packets = [0, 1024, 1024, 1024, 4096].map(|stamp| PacketTime { stamp, stamped: true, samples: Some(1024) });
let mut laced = packets;
laced[2].stamped = false;
laced[3].stamped = false;
assert_eq!(contiguous_starts(laced, 512), [0, 1024, 2048, 3072, 4096]);
// a packet whose length is unknown: the next one starts at its own stamp
let mut unknown = timed(&[0, 1024, 2100, 3124], Some(1024));
unknown[1].samples = None;
assert_eq!(contiguous_starts(unknown, 512), [0, 1024, 2100, 3124]);
}
#[test]
fn hostile_values_stay_finite() {
let packets = [
PacketTime { stamp: i64::MAX, stamped: true, samples: Some(i64::MAX) },
PacketTime { stamp: i64::MIN, stamped: false, samples: Some(-5) },
PacketTime { stamp: 0, stamped: true, samples: Some(i64::MIN) },
PacketTime { stamp: 7, stamped: true, samples: None },
];
assert_eq!(contiguous_starts(packets, i64::MIN).len(), 4);
assert_eq!(contiguous_starts(packets, i64::MAX).len(), 4);
assert!(contiguous_starts([], 512).is_empty());
}
#[test]
fn fixed_frame_lengths() {
// AAC-LC 48 kHz stereo; HE-AAC (24 kHz core) played at 48 kHz is upsampled
assert_eq!(fixed_packet_samples(FixedFrames::Aac(&[0x11, 0x90]), &[], 48_000), Some(1024));
assert_eq!(fixed_packet_samples(FixedFrames::Aac(&[0x13, 0x10]), &[], 48_000), Some(2048));
assert_eq!(fixed_packet_samples(FixedFrames::Aac(&[]), &[], 48_000), None);
// MPEG-1 layer III 44.1 kHz, 128 kbps; MPEG-2 layer III 22.05 kHz
assert_eq!(fixed_packet_samples(FixedFrames::MpegAudio, &[0xFF, 0xFB, 0x90, 0x44], 44_100), Some(1152));
assert_eq!(fixed_packet_samples(FixedFrames::MpegAudio, &[0xFF, 0xF3, 0x90, 0x44], 22_050), Some(576));
// the header's rate must be the stream's
assert_eq!(fixed_packet_samples(FixedFrames::MpegAudio, &[0xFF, 0xFB, 0x90, 0x44], 48_000), None);
assert_eq!(fixed_packet_samples(FixedFrames::MpegAudio, &[0xFF], 44_100), None);
assert_eq!(fixed_packet_samples(FixedFrames::Ac3, &[0x0B, 0x77], 48_000), None);
// E-AC-3 access units: 8-byte syncframes (frmsiz 3) at 48 kHz, stereo. A six-block
// independent frame with a dependent substream (7.1) is 1536 samples; two one-block
// frames are 512
let frame = |strmtyp: u8, numblkscod: u8| [0x0B, 0x77, strmtyp << 6, 3, (numblkscod << 4) | (2 << 1), 16 << 3, 0, 0];
let au = |frames: &[[u8; 8]]| frames.concat();
assert_eq!(fixed_packet_samples(FixedFrames::Eac3, &au(&[frame(0, 3), frame(1, 3)]), 48_000), Some(1536));
assert_eq!(fixed_packet_samples(FixedFrames::Eac3, &au(&[frame(0, 0), frame(0, 0)]), 48_000), Some(512));
assert_eq!(fixed_packet_samples(FixedFrames::Eac3, &au(&[frame(0, 3)]), 44_100), None);
assert_eq!(fixed_packet_samples(FixedFrames::Eac3, &[0x0B, 0x77], 48_000), None);
}
}
#[cfg(test)]
mod mpeg_audio_tests {
use super::*;
/// A silent mono 48 kHz MPEG-1 frame of `layer` (2: 64 kbps, 192 bytes; 3: 56 kbps, 168
/// bytes): header, then all-zero bit allocation / side information.
fn silent_frame(layer: u8) -> Vec<u8> {
let (b1, len) = if layer == 2 { (0xFD, 192) } else { (0xFB, 168) };
let mut f = vec![0xFF, b1, 0x44, 0xC0];
f.resize(len, 0);
f
}
#[test]
fn layer_comes_from_the_frame_header() {
assert_eq!(mpeg_audio_layer(&[0xFF, 0xFF, 0x10, 0xC0]), Some(1));
assert_eq!(mpeg_audio_layer(&silent_frame(2)), Some(2));
assert_eq!(mpeg_audio_layer(&silent_frame(3)), Some(3));
// MPEG-2 layer III, MPEG-2.5 layer II
assert_eq!(mpeg_audio_layer(&[0xFF, 0xF3, 0x90, 0x44]), Some(3));
assert_eq!(mpeg_audio_layer(&[0xFF, 0xE5, 0x90, 0x44]), Some(2));
// reserved layer, no sync word, too short
for bad in [&[0xFF, 0xF9, 0x44, 0xC0][..], &[0xFF, 0x1D], &[0x00, 0xFD], &[0xFF], &[]] {
assert_eq!(mpeg_audio_layer(bad), None, "{bad:?}");
}
}
/// #800: ffmpeg stores MP2 in MP4 as `mp4a` + object type 0x6B, which names no layer. Handing
/// it to the layer III decoder rejected every frame and the track played as silence.
#[test]
fn mp4_mpeg_audio_decodes_the_layer_its_frames_name() {
for layer in [2u8, 3] {
let mut d = PacketDecoder::for_isobmff(&filmcraft_isobmff::CodecConfig::Mp3, 48_000).unwrap();
// a packet without a frame header is an error, not a crash, and configures nothing
assert!(d.decode(&[0u8; 16], 0).is_err());
d.reset();
for k in 0..3u64 {
let out = d.decode(&silent_frame(layer), k * 1152).unwrap_or_else(|e| panic!("layer {layer}: {e}"));
assert_eq!(out.len(), 1, "layer {layer}");
assert_eq!(out[0].len(), 1152, "layer {layer}");
assert!(out[0].iter().all(|&s| s == 0.0), "layer {layer}");
}
}
}
}
+178
View File
@@ -0,0 +1,178 @@
//! Audio packets placed by container timestamps (#236).
//!
//! An AAC access unit decodes to 1024 samples, 21.33 ms at 48 kHz. Matroska stores timestamps in
//! whole milliseconds, and OBS's remux to MP4 carries that rounding into sample durations of 1008,
//! 1008, 1056… Placing each decoded packet at its own timestamp overwrote the end of one packet and
//! left a hole of silence after another: playback and export crackled throughout. The streams here
//! are AAC from our own encoder muxed with such timestamps; read back block by block, as playback
//! reads them, they must be exactly the access units decoded back to back.
use std::io::Cursor;
use std::sync::Arc;
use filmcraft_isobmff::{Brand, Mp4Writer, SampleEntry, TrackConfig, WriteSample, WriterOptions};
use filmcraft_matroska::{LacingMode, MkvWriter, MuxOptions, TrackKind, TrackSpec};
use filmcraft_media::MediaSource;
use crate::audio::PacketDecoder;
use crate::{MkvSource, Mp4Source};
/// A second of a stereo tone (440 Hz left, 660 Hz right) encoded with our AAC encoder: the
/// AudioSpecificConfig and the access units.
fn aac(rate: u32) -> (Vec<u8>, Vec<Vec<u8>>) {
let tone = |hz: f32| (0..rate as usize).map(|i| 0.5 * (std::f32::consts::TAU * hz * i as f32 / rate as f32).sin()).collect::<Vec<f32>>();
let (l, r) = (tone(440.0), tone(660.0));
let mut enc = filmcraft_aac::Encoder::new(filmcraft_aac::EncoderConfig::cbr(rate, 2, 160_000)).unwrap();
let mut units = enc.encode(&[&l, &r]);
units.extend(enc.flush());
(enc.audio_specific_config(), units)
}
/// The access units decoded back to back.
fn continuous(asc: &[u8], rate: u32, units: &[Vec<u8>]) -> Vec<Vec<f32>> {
let mut dec = PacketDecoder::aac(asc, rate).unwrap();
let mut out = vec![Vec::new(), Vec::new()];
for u in units {
for (o, c) in out.iter_mut().zip(dec.decode(u, 0).unwrap()) {
o.extend(c);
}
}
out
}
/// Access unit `k`'s start to the nearest millisecond, as muxers stamp it.
fn stamp_ms(rate: u32, k: usize) -> i64 {
(k as i64 * 1024 * 2000 + rate as i64) / (2 * rate as i64)
}
/// Matroska (1 ms timestamps), with the units after `gap` delayed by `gap_ms` (a real gap).
fn mkv(asc: &[u8], rate: u32, units: &[Vec<u8>], lacing: LacingMode, gap: usize, gap_ms: i64) -> Arc<[u8]> {
let mut spec = TrackSpec::new(TrackKind::Audio, "A_AAC");
spec.codec_private = asc.to_vec();
spec.audio = Some((rate as f64, 2, None));
let opts = MuxOptions { lacing, ..MuxOptions::default() };
assert_eq!(opts.timestamp_scale, 1_000_000, "1 ms timestamps");
let mut w = MkvWriter::new(Cursor::new(Vec::new()), vec![spec], opts).unwrap();
for (k, u) in units.iter().enumerate() {
let ms = stamp_ms(rate, k) + if k >= gap { gap_ms } else { 0 };
w.write_frame(0, ms * 1_000_000, true, u, None).unwrap();
}
w.finish().unwrap().into_inner().into()
}
/// MP4 as OBS remuxes Matroska: each millisecond timestamp rescaled to the sample rate, the
/// differences written as sample durations.
fn mp4_remuxed(asc: &[u8], rate: u32, units: &[Vec<u8>]) -> Arc<[u8]> {
let mut w = Mp4Writer::new(Cursor::new(Vec::new()), WriterOptions::new(Brand::Mp4)).unwrap();
let t = w.add_track(TrackConfig::new(SampleEntry::aac(asc.to_vec(), 2, rate), rate)).unwrap();
let at = |k: usize| (stamp_ms(rate, k) * rate as i64 + 500) / 1000;
for (k, u) in units.iter().enumerate() {
let duration = (at(k + 1) - at(k)) as u32;
w.write_sample(t, WriteSample { data: u, duration, composition_offset: 0, is_sync: true }).unwrap();
}
w.finish().unwrap().into_inner().into()
}
/// `frames` samples per channel read in 512-sample blocks at `rate`, as playback reads them.
fn played(src: &dyn MediaSource, rate: u32, frames: usize) -> Vec<Vec<f32>> {
let mut out = vec![Vec::new(), Vec::new()];
let mut pos = 0;
while pos < frames {
let n = 512.min(frames - pos);
let b = src.audio(pos as i64, n, rate).unwrap();
for (o, c) in out.iter_mut().zip(&b.channels) {
o.extend_from_slice(c);
}
pos += n;
}
out
}
/// Samples of `got` that differ from `want[k - shift]` (silence where that is out of range), and
/// the first few.
fn mismatches(got: &[Vec<f32>], want: &[Vec<f32>], shift: usize) -> (usize, Vec<(usize, usize)>) {
let mut bad = Vec::new();
for (c, (g, w)) in got.iter().zip(want).enumerate() {
for (k, x) in g.iter().enumerate() {
let expect = k.checked_sub(shift).and_then(|j| w.get(j)).copied().unwrap_or(0.0);
if x.to_bits() != expect.to_bits() {
bad.push((c, k));
}
}
}
(bad.len(), bad.into_iter().take(4).collect())
}
/// Runs of two or more exact zeros inside the signal: holes left between packets.
fn holes(got: &[Vec<f32>]) -> usize {
got.iter().map(|c| c.windows(2).filter(|w| w[0] == 0.0 && w[1] == 0.0).count()).sum()
}
#[test]
fn matroska_millisecond_stamps_play_back_to_back() {
for rate in [48_000, 44_100] {
let (asc, units) = aac(rate);
let want = continuous(&asc, rate, &units);
let frames = units.len().saturating_sub(1) * 1024;
for lacing in [LacingMode::None, LacingMode::Xiph] {
let bytes = mkv(&asc, rate, &units, lacing, usize::MAX, 0);
let src = MkvSource::open("tone.mkv", bytes.clone()).unwrap();
let got = played(&src, rate, frames);
let (n, first) = mismatches(&got, &want, 0);
assert_eq!(n, 0, "{rate} Hz, {lacing:?}: {n} samples differ from the continuous decode, first (channel, sample) {first:?}");
// random access (a fresh source, so a cold decoder primed with the packet before) gives
// the same samples
let src = MkvSource::open("tone.mkv", bytes).unwrap();
for start in [30_011usize, 7_000] {
let got = src.audio(start as i64, 4_800, rate).unwrap().channels;
let at: Vec<Vec<f32>> = want.iter().map(|c| c[start..start + 4_800].to_vec()).collect();
assert_eq!(mismatches(&got, &at, 0).0, 0, "{rate} Hz, {lacing:?}: seek to {start}");
}
}
}
}
#[test]
fn obs_remuxed_mp4_plays_back_to_back() {
for rate in [48_000, 44_100] {
let (asc, units) = aac(rate);
let want = continuous(&asc, rate, &units);
let bytes = mp4_remuxed(&asc, rate, &units);
let src = Mp4Source::open("tone.mp4", bytes).unwrap();
let got = played(&src, rate, units.len().saturating_sub(1) * 1024);
let (n, first) = mismatches(&got, &want, 0);
assert_eq!(n, 0, "{rate} Hz: {n} samples differ from the continuous decode, first (channel, sample) {first:?}");
}
}
#[test]
fn a_real_gap_stays_where_the_timestamps_put_it() {
let rate = 48_000;
let (asc, units) = aac(rate);
let want = continuous(&asc, rate, &units);
// 100 ms of the recording missing before unit 20: it plays as silence, and the units after it
// keep their timestamps (to the millisecond)
let src = MkvSource::open("gap.mkv", mkv(&asc, rate, &units, LacingMode::None, 20, 100)).unwrap();
let shift = ((stamp_ms(rate, 20) + 100) * 48 - 20 * 1024) as usize;
assert!((4_700..4_900).contains(&shift), "{shift}");
let got = played(&src, rate, units.len().saturating_sub(1) * 1024 + shift);
let before: Vec<Vec<f32>> = got.iter().map(|c| c[..20 * 1024].to_vec()).collect();
assert_eq!(mismatches(&before, &want, 0).0, 0);
assert!(got.iter().all(|c| c[20 * 1024..20 * 1024 + shift].iter().all(|x| *x == 0.0)), "the gap is silent");
let after: Vec<Vec<f32>> = got.iter().map(|c| c[20 * 1024 + shift..].to_vec()).collect();
let rest: Vec<Vec<f32>> = want.iter().map(|c| c[20 * 1024..].to_vec()).collect();
let (n, first) = mismatches(&after, &rest, 0);
assert_eq!(n, 0, "{n} samples after the gap differ, first {first:?}");
}
#[test]
fn playing_at_another_rate_leaves_no_holes() {
// a 44.1 kHz recording in a 48 kHz sequence (export's default rate)
let (asc, units) = aac(44_100);
let src = MkvSource::open("tone.mkv", mkv(&asc, 44_100, &units, LacingMode::None, usize::MAX, 0)).unwrap();
let frames = (units.len().saturating_sub(2) * 1024) * 48_000 / 44_100;
let got = played(&src, 48_000, frames);
// skip the encoder's priming (the first frame decodes to near-silence)
let tail: Vec<Vec<f32>> = got.iter().map(|c| c[2048..].to_vec()).collect();
assert_eq!(holes(&tail), 0);
}
+330 -27
View File
@@ -12,7 +12,7 @@ use std::time::Duration;
use web_time::Instant;
use filmcraft_color::ColorInfo;
use filmcraft_frame::VideoFrame;
use filmcraft_frame::{Region, VideoFrame};
use crate::CodecError;
use crate::video::VideoDecoder;
@@ -122,6 +122,18 @@ pub trait VideoSamples {
fn make_decoder(&self) -> crate::Result<Box<dyn VideoDecoder>>;
}
/// The sync sample to start decoding at for sample `i`: its sync sample, or an earlier one when
/// `i` is a leading picture of an open GOP (stored after its random-access picture, shown before
/// it, like the RASL pictures after every HEVC CRA of iPhone video). Those are predicted from the
/// GOP before, so a decoder starting at their random-access picture leaves them out.
fn decode_start(s: &dyn VideoSamples, i: usize) -> usize {
let mut key = s.sync_before(i);
while key > 0 && s.pts(i) < s.pts(key) {
key = s.sync_before(key - 1).min(key - 1);
}
key
}
struct State {
decoder: Option<Box<dyn VideoDecoder>>,
/// The decoder only produces intra pictures: frames decode independently and in parallel.
@@ -140,6 +152,10 @@ struct State {
bytes: usize,
/// When a frame was last asked of this cache.
last_used: Instant,
/// Only background requests (thumbnails, [`filmcraft_media::cancel::with_background`]) have
/// used the decoder since it was made: it is given up as soon as another background request
/// is done, whatever the caches' recency.
background: bool,
}
impl State {
@@ -212,10 +228,27 @@ impl Drop for DecodingGuard {
/// new one when the playhead reaches them, which costs what a seek costs.
const MAX_LIVE_DECODERS: usize = 4;
/// A cache asked for a frame this recently is in use (a layer of the frame being composited,
/// the next clip being prefetched): it keeps its decoder whatever the count.
/// A cache asked for a frame this recently may be needed again soon (the next clip being
/// prefetched, a clip the user scrubs back to): it keeps its decoder up to [`HARD_DECODERS`], and
/// beyond [`MAX_LIVE_DECODERS`] only until it has been idle this long. Its frames are only kept
/// within [`FRAME_BUDGET`] once it has been idle for [`RECENT`]: going back to a clip left (or
/// paused) more than that long ago may cost a re-decode from its keyframe when other clips have
/// filled the budget meanwhile. Frames go least recently used cache first, earliest frames first.
const IDLE: Duration = Duration::from_secs(3);
/// A cache asked for a frame this recently is in use (a layer of the frame being composited): it
/// keeps its decoder and frames whatever the count. Caches idle for less than [`IDLE`] but longer
/// than this are a burst of navigation (thumbnails of every clip, the playhead jumping from clip
/// to clip) that the playhead has already left: they give up decoders beyond [`HARD_DECODERS`]
/// and frames beyond [`FRAME_BUDGET`]. Without this, everything touched within [`IDLE`] counted
/// as in use, so a burst held every clip's decoder (42 live hardware sessions, 6.5 GB of VRAM)
/// and hundreds of MB of frames per clip.
const RECENT: Duration = Duration::from_millis(500);
/// Most decoders caches idle for [`RECENT`] to [`IDLE`] keep together (a composite of a few
/// layers plus the clips just left behind).
const HARD_DECODERS: usize = 8;
/// Decoded frames all caches may hold together before idle ones give theirs up. Each cache also
/// has its own budget, which alone let every clip on a timeline keep hundreds of MB of frames
/// nobody was looking at.
@@ -228,6 +261,10 @@ struct Pool {
decoders: AtomicUsize,
max_decoders: usize,
idle: Duration,
/// Caches idle at least this long are no longer in use (see [`RECENT`]).
recent: Duration,
/// Decoders the caches idle for [`Self::recent`] but not [`Self::idle`] may keep.
hard_decoders: usize,
/// Bytes of the frames the caches hold.
bytes: AtomicUsize,
budget: usize,
@@ -258,13 +295,29 @@ fn try_state(c: &Shared) -> Option<MutexGuard<'_, State>> {
impl Pool {
fn new(max_decoders: usize, idle: Duration, budget: usize) -> Self {
Self { caches: Mutex::new(Vec::new()), decoders: AtomicUsize::new(0), max_decoders, idle, bytes: AtomicUsize::new(0), budget }
Self {
caches: Mutex::new(Vec::new()),
decoders: AtomicUsize::new(0),
max_decoders,
idle,
recent: idle,
hard_decoders: max_decoders,
bytes: AtomicUsize::new(0),
budget,
}
}
/// Caches idle for `recent` (< the idle time) are still trimmed, down to `hard_decoders`.
fn tiered(mut self, recent: Duration, hard_decoders: usize) -> Self {
self.recent = recent.min(self.idle);
self.hard_decoders = hard_decoders.max(self.max_decoders);
self
}
/// The pool of every cache in the process.
fn global() -> Arc<Pool> {
static POOL: std::sync::OnceLock<Arc<Pool>> = std::sync::OnceLock::new();
POOL.get_or_init(|| Arc::new(Pool::new(MAX_LIVE_DECODERS, IDLE, FRAME_BUDGET))).clone()
POOL.get_or_init(|| Arc::new(Pool::new(MAX_LIVE_DECODERS, IDLE, FRAME_BUDGET).tiered(RECENT, HARD_DECODERS))).clone()
}
fn register(&self, c: &Arc<Shared>) {
@@ -273,15 +326,15 @@ impl Pool {
g.push(Arc::downgrade(c));
}
/// The idle caches other than `me` that `keep` selects, least recently used first. Other
/// The caches other than `me` idle for at least `idle` that `keep` selects, least recently used first. Other
/// caches are only ever try-locked, so two caches trimming each other cannot deadlock.
fn idle_others(&self, me: &Shared, keep: impl Fn(&State) -> bool) -> Vec<Arc<Shared>> {
fn idle_others(&self, me: &Shared, idle: Duration, keep: impl Fn(&State) -> bool) -> Vec<Arc<Shared>> {
let all: Vec<Arc<Shared>> = self.caches.lock().unwrap_or_else(PoisonError::into_inner).iter().filter_map(Weak::upgrade).collect();
let mut idle: Vec<(Instant, Arc<Shared>)> = all
.into_iter()
.filter(|c| !std::ptr::eq(Arc::as_ptr(c), me))
.filter_map(|c| {
let used = try_state(&c).filter(|st| keep(st) && st.last_used.elapsed() >= self.idle).map(|st| st.last_used)?;
let used = try_state(&c).filter(|st| keep(st) && st.last_used.elapsed() >= idle).map(|st| st.last_used)?;
Some((used, c))
})
.collect();
@@ -295,29 +348,53 @@ impl Pool {
#[must_use = "drop the released decoders with no cache lock held"]
fn trim_decoders(&self, me: &Shared) -> Vec<Box<dyn VideoDecoder>> {
let mut released = Vec::new();
if self.decoders.load(Ordering::Relaxed) <= self.max_decoders {
return released;
}
for c in self.idle_others(me, |st| st.decoder.is_some()) {
if self.decoders.load(Ordering::Relaxed) <= self.max_decoders {
break;
// first the caches idle for long, down to the cap; then those a burst has left behind
for (idle, cap) in [(self.idle, self.max_decoders), (self.recent, self.hard_decoders)] {
if self.decoders.load(Ordering::Relaxed) <= cap {
continue;
}
if let Some(mut st) = try_state(&c) {
for c in self.idle_others(me, idle, |st| st.decoder.is_some()) {
if self.decoders.load(Ordering::Relaxed) <= cap {
break;
}
if let Some(mut st) = try_state(&c) {
released.extend(st.release_decoder(self));
}
}
}
released
}
/// Take the decoders of the caches other than `me` that only background requests have used
/// (see [`State::background`]) and that are not decoding now. A burst of thumbnails (an import
/// of many clips) touches every clip within moments, so the recency rules treat them all as
/// in use; each software 4K decoder holds ~0.5 GB. The cache `me` just served keeps its
/// decoder (hovering a clip asks for more of its frames). Drop the result with no cache lock
/// held.
#[must_use = "drop the released decoders with no cache lock held"]
fn release_background(&self, me: &Shared) -> Vec<Box<dyn VideoDecoder>> {
let all: Vec<Arc<Shared>> = self.caches.lock().unwrap_or_else(PoisonError::into_inner).iter().filter_map(Weak::upgrade).collect();
let mut released = Vec::new();
for c in all.iter().filter(|c| !std::ptr::eq(Arc::as_ptr(c), me)) {
if let Some(mut st) = try_state(c)
&& st.background
&& st.decoder.is_some()
{
released.extend(st.release_decoder(self));
}
}
released
}
/// Evict the frames of idle caches, least recently used first, down to the budget. Caches in
/// use keep theirs (each within its own budget): a frame evicted before it is shown costs a
/// Evict the frames of caches idle for [`RECENT`], least recently used first, down to the
/// budget. Caches in use keep theirs (each within its own budget): a frame evicted before it is shown costs a
/// re-decode from the keyframe.
fn trim_frames(&self, me: &Shared) {
let over = || self.bytes.load(Ordering::Relaxed) > self.budget;
if !over() {
return;
}
for c in self.idle_others(me, |st| !st.frames.is_empty()) {
for c in self.idle_others(me, self.recent, |st| !st.frames.is_empty()) {
if !over() {
break;
}
@@ -345,6 +422,8 @@ pub struct GopCache {
explicit_color: Option<ColorInfo>,
/// Clockwise quarter turns from the container's display matrix, applied to every frame.
rotation: u8,
/// The container's clean aperture, cut from every frame before the rotation.
crop: Option<Region>,
budget: usize,
}
@@ -361,7 +440,7 @@ const CONTINUE_THROUGH: usize = 48;
impl GopCache {
pub fn new(explicit_color: Option<ColorInfo>) -> Self {
// unit tests count decoder restarts: each of their caches gets a pool of its own
let pool = if cfg!(test) { Arc::new(Pool::new(MAX_LIVE_DECODERS, IDLE, FRAME_BUDGET)) } else { Pool::global() };
let pool = if cfg!(test) { Arc::new(Pool::new(MAX_LIVE_DECODERS, IDLE, FRAME_BUDGET).tiered(RECENT, HARD_DECODERS)) } else { Pool::global() };
Self::in_pool(explicit_color, pool)
}
@@ -378,11 +457,12 @@ impl GopCache {
drafts: Default::default(),
bytes: 0,
last_used: Instant::now(),
background: false,
}),
pool: pool.clone(),
});
pool.register(&shared);
Self { shared, explicit_color, rotation: 0, budget: 384 << 20 }
Self { shared, explicit_color, rotation: 0, crop: None, budget: 384 << 20 }
}
/// Turn every decoded frame clockwise by `quarter_turns` × 90° (the container's display
@@ -392,19 +472,30 @@ impl GopCache {
self
}
/// Cut every decoded frame to `region` (the container's clean aperture, in stored picture
/// coordinates) before the rotation, as cropping and the display matrix compose.
pub fn with_crop(mut self, region: Option<Region>) -> Self {
self.crop = region;
self
}
/// Colour signalled by the container wins over the bitstream's (YUV frames only); the
/// container's display rotation is applied.
/// container's clean aperture and then its display rotation are applied.
fn finish(&self, mut f: VideoFrame) -> VideoFrame {
if let Some(c) = self.explicit_color
&& !matches!(f.data, filmcraft_frame::PixelData::Rgba8(_) | filmcraft_frame::PixelData::RgbaF32(_))
{
f.color = c;
}
if let Some(r) = self.crop {
f = f.cropped(r);
}
if self.rotation != 0 { f.rotated(self.rotation) } else { f }
}
fn store(&self, st: &mut State, pts: i64, f: VideoFrame, draft: bool) {
let f = self.finish(f);
st.last_used = Instant::now();
if draft {
st.drafts.insert(pts);
DRAFT.fetch_add(1, Ordering::Relaxed);
@@ -467,6 +558,16 @@ impl GopCache {
/// ([`VideoDecoder::set_draft`]); its draft frames are cached for draft requests only, and an
/// exact request for one re-decodes it.
pub fn frame_late(&self, s: &dyn VideoSamples, target: i64, late_before: Option<i64>) -> crate::Result<Arc<VideoFrame>> {
let r = self.frame_late_locked(s, target, late_before);
if filmcraft_media::cancel::background() {
// Outside the cache's lock (see `release_background`). A thumbnail leaves the decoders
// of earlier thumbnails no reason to stay: each costs hundreds of MB at 4K.
drop(self.shared.pool.release_background(&self.shared));
}
r
}
fn frame_late_locked(&self, s: &dyn VideoSamples, target: i64, late_before: Option<i64>) -> crate::Result<Arc<VideoFrame>> {
let n = s.count();
let i = s.sample_at(target.max(0)).or_else(|| (n > 0).then(|| n - 1)).ok_or_else(|| CodecError::Decode("empty track".into()))?;
let want_pts = s.pts(i);
@@ -482,11 +583,23 @@ impl GopCache {
let mut st = self.shared.state.lock().unwrap_or_else(|e| e.into_inner());
let _decoding = DecodingGuard::enter(me);
st.last_used = Instant::now();
if let Some(f) = st.cached(want_pts, draft) {
let background = filmcraft_media::cancel::background();
if let Some(f) = st.cached(want_pts, draft).cloned() {
HITS.fetch_add(1, Ordering::Relaxed);
return Ok(f.clone());
// a monitor asking for a frame of a thumbnail's clip makes its decoder one in use
st.background &= background;
return Ok(f);
}
MISSES.fetch_add(1, Ordering::Relaxed);
if !background {
st.background = false;
} else if st.decoder.is_none() {
st.background = true;
}
if !background && st.decoder.as_ref().is_some_and(|d| d.thread_limited()) {
// made for a thumbnail (few threads): a monitor gets a full one
released.extend(st.release_decoder(&self.shared.pool));
}
if st.decoder.is_none() {
let d = s.make_decoder()?;
st.intra = d.intra_only();
@@ -503,12 +616,13 @@ impl GopCache {
if st.intra {
return self.intra_frame(st, s, i, want_pts);
}
let mut key = s.sync_before(i);
let mut key = decode_start(s, i);
// Continue the running decoder when it has passed the wanted sample's sync sample and
// either has not reached the sample yet or has been fed it without outputting it yet
// (a frame-threaded decoder holds many pictures in flight). Otherwise the frame was
// evicted or lies in another GOP: restart at the sync sample.
let running = st.next != usize::MAX && st.next <= n;
// evicted or lies in another GOP: restart at the sync sample. A run that started after
// `key` (at the random-access picture of a wanted leading picture) left the picture out.
let running = st.next != usize::MAX && st.next <= n && st.start <= key;
// Decoding on through a short stretch into the next GOP is cheaper than a restart, and
// playback wants those frames anyway.
let near = st.next <= key && key - st.next <= CONTINUE_THROUGH;
@@ -593,7 +707,7 @@ impl GopCache {
want_pts: i64,
n: usize,
) -> crate::Result<Vec<crate::video::DecodedFrame>> {
let mut key = s.sync_before(i);
let mut key = decode_start(s, i);
// Continue the running decoder when the wanted sample is ahead within this GOP run.
let continuing = *next != usize::MAX && *next > key && *next <= i + 16 && *next <= n;
if !continuing {
@@ -683,6 +797,8 @@ mod tests {
/// (pts, decoded as a draft picture)
held: VecDeque<(i64, bool)>,
draft: bool,
/// made for background work (few threads, like the H.264 decoder's)
limited: bool,
resets: Arc<AtomicUsize>,
decodes: Arc<AtomicUsize>,
}
@@ -717,6 +833,9 @@ mod tests {
fn name(&self) -> &str {
"test"
}
fn thread_limited(&self) -> bool {
self.limited
}
fn intra_only(&self) -> bool {
self.intra
}
@@ -749,6 +868,7 @@ mod tests {
intra: self.intra,
held: VecDeque::new(),
draft: false,
limited: filmcraft_media::cancel::background(),
resets: self.resets.clone(),
decodes: self.decodes.clone(),
}))
@@ -944,6 +1064,99 @@ mod tests {
assert!(s.iter().all(|s| s.resets.load(Ordering::Relaxed) == 1), "no decoder restarted");
}
/// A burst of navigation (every clip of a timeline asked once within moments, as when
/// thumbnails are made or the playhead jumps clip to clip) must not keep every clip's decoder
/// and frames: only the caches of the frame being composited are in use.
#[test]
fn a_burst_of_single_requests_does_not_keep_every_decoder_and_frame() {
// the real decoder caps and idle time; RECENT scaled down from 500 ms to the test's pace
// (3 ms per clip). With the old rule (recent = idle) all 24 decoders stay live.
let pool = Arc::new(Pool::new(MAX_LIVE_DECODERS, IDLE, FRAME_BUDGET).tiered(Duration::from_millis(2), HARD_DECODERS));
let n = 24;
let s: Vec<Samples> = (0..n).map(|_| samples(300, 250, 3, false)).collect();
let c: Vec<GopCache> = (0..n).map(|_| GopCache::in_pool(None, pool.clone())).collect();
for k in 0..n {
assert_eq!(index_of(&c[k].frame(&s[k], 10_000).expect("frame")), 10);
std::thread::sleep(Duration::from_millis(3));
}
assert!(
pool.decoders.load(Ordering::Relaxed) <= HARD_DECODERS + 1,
"{} live decoders after a burst over {n} clips",
pool.decoders.load(Ordering::Relaxed)
);
// the clip asked last, and a clip in use together with it, keep theirs and carry on
let resets = s[n - 1].resets.load(Ordering::Relaxed);
assert_eq!(index_of(&c[n - 1].frame(&s[n - 1], 11_000).expect("frame")), 11);
assert_eq!(s[n - 1].resets.load(Ordering::Relaxed), resets);
}
/// The thumbnails an import queues: every clip asked once, quickly (the real caps and recency,
/// no sleeping: all of them are "recent"). Each 4K software decoder costs ~0.5 GB, and 42
/// clips used to hold 6+ GB at once.
#[test]
fn background_requests_leave_no_decoders_behind() {
use filmcraft_media::cancel::with_background;
let pool = Arc::new(Pool::new(MAX_LIVE_DECODERS, IDLE, FRAME_BUDGET).tiered(RECENT, HARD_DECODERS));
let n = 24;
let s: Vec<Samples> = (0..n).map(|_| samples(300, 250, 3, false)).collect();
let c: Vec<GopCache> = (0..n).map(|_| GopCache::in_pool(None, pool.clone())).collect();
// a clip in use by a monitor
assert_eq!(index_of(&c[0].frame(&s[0], 10_000).expect("frame")), 10);
for k in 1..n {
assert_eq!(index_of(&with_background(true, || c[k].frame(&s[k], 10_000)).expect("frame")), 10);
assert!(pool.decoders.load(Ordering::Relaxed) <= 2, "{} live decoders after thumbnail {k}", pool.decoders.load(Ordering::Relaxed));
}
// the monitor's clip and the last thumbnail keep theirs and carry on without a seek
assert_eq!(pool.decoders.load(Ordering::Relaxed), 2);
assert_eq!(index_of(&c[0].frame(&s[0], 11_000).expect("frame")), 11);
assert_eq!(s[0].resets.load(Ordering::Relaxed), 1);
assert_eq!(index_of(&with_background(true, || c[n - 1].frame(&s[n - 1], 12_000)).expect("frame")), 12);
assert_eq!(s[n - 1].resets.load(Ordering::Relaxed), 1);
// a monitor asking for a frame of a thumbnail's clip makes its decoder one in use
assert_eq!(index_of(&c[n - 1].frame(&s[n - 1], 13_000).expect("frame")), 13);
for k in 1..4 {
with_background(true, || c[k].frame(&s[k], 20_000)).expect("frame");
}
assert_eq!(index_of(&c[n - 1].frame(&s[n - 1], 14_000).expect("frame")), 14);
assert!(c[n - 1].shared.state.lock().unwrap().decoder.is_some(), "the thumbnails left its decoder");
// a cache that lost its decoder still decodes (from the sync sample)
assert_eq!(index_of(&with_background(true, || c[1].frame(&s[1], 30_000)).expect("frame")), 30);
}
#[test]
fn a_monitor_replaces_a_decoder_made_for_a_thumbnail() {
use filmcraft_media::cancel::with_background;
let pool = Arc::new(Pool::new(MAX_LIVE_DECODERS, IDLE, FRAME_BUDGET).tiered(RECENT, HARD_DECODERS));
let (s, c) = (samples(300, 250, 3, false), GopCache::in_pool(None, pool.clone()));
assert_eq!(index_of(&with_background(true, || c.frame(&s, 10_000)).expect("frame")), 10);
// another thumbnail of the same clip carries on with the thumbnail decoder
assert_eq!(index_of(&with_background(true, || c.frame(&s, 12_000)).expect("frame")), 12);
assert_eq!(s.resets.load(Ordering::Relaxed), 1);
// a frame a monitor waits for (not cached: far ahead) is decoded by a full-thread decoder
assert_eq!(index_of(&c.frame(&s, 100_000).expect("frame")), 100);
assert!(!c.shared.state.lock().unwrap().decoder.as_ref().is_some_and(|d| d.thread_limited()));
assert_eq!(pool.decoders.load(Ordering::Relaxed), 1);
// and keeps it for the next frames
let resets = s.resets.load(Ordering::Relaxed);
assert_eq!(index_of(&c.frame(&s, 101_000).expect("frame")), 101);
assert_eq!(s.resets.load(Ordering::Relaxed), resets);
}
#[test]
fn a_burst_of_single_requests_does_not_keep_every_clips_frames() {
// 1×1 RGBA pictures (4 bytes): a shared budget of ten of them
let pool = Arc::new(Pool::new(usize::MAX, Duration::from_secs(3600), 40).tiered(Duration::from_millis(2), usize::MAX));
let n = 12;
let s: Vec<Samples> = (0..n).map(|_| samples(300, 250, 0, false)).collect();
let c: Vec<GopCache> = (0..n).map(|_| GopCache::in_pool(None, pool.clone())).collect();
for k in 0..n {
assert_eq!(index_of(&c[k].frame(&s[k], 10_000).expect("frame")), 10);
std::thread::sleep(Duration::from_millis(5));
}
let held = pool.bytes.load(Ordering::Relaxed);
assert!(held <= 40 + 11 * 4, "{held} bytes held after a burst over {n} clips");
}
#[test]
fn intra_caches_give_up_their_spare_decoders_too() {
let pool = Arc::new(Pool::new(1, Duration::ZERO, FRAME_BUDGET));
@@ -1061,4 +1274,94 @@ mod tests {
}
assert!(reused() - before >= more as u64 - 2, "{} of {more} planes recycled", reused() - before);
}
/// An open-GOP stream like iPhone HEVC: 8 pictures a GOP; after the first, each GOP stores
/// its random-access picture (display 8·g) first, then two leading pictures shown before it
/// (8·g − 2, 8·g − 1, predicted from the GOP before), then the rest. pts = 1000 · display.
struct OpenGop {
/// (display index, sync) in decode order
order: Vec<(i64, bool)>,
resets: Arc<AtomicUsize>,
}
impl OpenGop {
fn new(gops: i64) -> Self {
let mut order: Vec<(i64, bool)> = (0..6).map(|d| (d, d == 0)).collect();
for g in 1..gops {
order.extend([(8 * g, true), (8 * g - 2, false), (8 * g - 1, false)]);
order.extend((8 * g + 1..8 * g + 6).map(|d| (d, false)));
}
Self { order, resets: Default::default() }
}
}
/// Leaves out the leading pictures of the random-access picture a run starts at (RASL
/// pictures, as the HEVC decoders do), outputs the others at once.
struct OpenGopDec {
run_start: Option<i64>,
resets: Arc<AtomicUsize>,
}
impl VideoDecoder for OpenGopDec {
fn decode(&mut self, _sample: &[u8], pts: i64) -> crate::Result<Vec<DecodedFrame>> {
let start = *self.run_start.get_or_insert(pts);
Ok(if pts < start { Vec::new() } else { vec![picture((pts, false))] })
}
fn flush(&mut self) -> Vec<DecodedFrame> {
Vec::new()
}
fn reset(&mut self) {
self.resets.fetch_add(1, Ordering::Relaxed);
self.run_start = None;
}
fn name(&self) -> &str {
"open gop"
}
}
impl VideoSamples for OpenGop {
fn count(&self) -> usize {
self.order.len()
}
fn pts(&self, i: usize) -> i64 {
self.order[i].0 * 1000
}
fn sync_before(&self, i: usize) -> usize {
(0..=i).rev().find(|&k| self.order[k].1).unwrap_or(0)
}
fn sample_at(&self, t: i64) -> Option<usize> {
self.order.iter().position(|&(d, _)| d == t / 1000)
}
fn read(&self, i: usize) -> crate::Result<Vec<u8>> {
Ok(vec![i as u8])
}
fn make_decoder(&self) -> crate::Result<Box<dyn VideoDecoder>> {
Ok(Box::new(OpenGopDec { run_start: None, resets: self.resets.clone() }))
}
}
/// A leading picture of an open GOP, asked for first (a clip trimmed to start on it, the
/// playhead parked on it), came out black: decoding started at the random-access picture
/// after it, which leaves it out.
#[test]
fn leading_pictures_of_an_open_gop_decode_from_the_gop_before() {
let s = OpenGop::new(6);
for d in [6usize, 7, 14, 15, 39] {
let c = GopCache::new(None);
assert_eq!(index_of(&c.frame(&s, d as i64 * 1000).expect("frame")), d);
}
// the random-access picture first, then its leading pictures: the run that started at
// it left them out, so they restart a GOP earlier
let c = GopCache::new(None);
assert_eq!(index_of(&c.frame(&s, 16_000).expect("frame")), 16);
assert_eq!(index_of(&c.frame(&s, 15_000).expect("frame")), 15);
assert_eq!(index_of(&c.frame(&s, 14_000).expect("frame")), 14);
// playing through: one seek, every picture in turn
let s = OpenGop::new(6);
let c = GopCache::new(None);
for d in 0..46usize {
assert_eq!(index_of(&c.frame(&s, d as i64 * 1000).expect("frame")), d);
}
assert_eq!(s.resets.load(Ordering::Relaxed), 1);
}
}
+55 -4
View File
@@ -53,7 +53,7 @@ pub struct FrameStreamInfo {
}
/// The OBUs of a low-overhead AV1 sample: `(type, payload)`, stopping at the first malformed one.
fn obus(data: &[u8]) -> Vec<(u8, &[u8])> {
pub(crate) fn obus(data: &[u8]) -> Vec<(u8, &[u8])> {
let mut out = Vec::new();
let mut pos = 0usize;
while let Some(&h) = data.get(pos) {
@@ -82,6 +82,13 @@ fn obus(data: &[u8]) -> Vec<(u8, &[u8])> {
out
}
/// What an AV1 sequence header fixes about the pictures: profile, maximum size and the colour
/// configuration (bit depth, chroma format, colour description). `None` when it does not parse.
fn av1_format(seq: &[u8]) -> Option<(u8, u32, u32, filmcraft_av1::ColorConfig)> {
let h = filmcraft_av1::SequenceHeader::parse(seq).ok()?;
Some((h.profile, h.max_frame_width, h.max_frame_height, h.color))
}
/// VP9 colour as our decoder reports it: `color_space` and range from the bitstream, transfer and
/// primaries from the container when it says. (Shared with `Vp9Decoder`.)
pub(crate) fn vp9_color(width: u32, height: u32, color_space: u8, full_range: bool, transfer: Option<Transfer>, primaries: Option<Primaries>) -> ColorInfo {
@@ -243,14 +250,58 @@ impl FrameStreamInfo {
}
/// Whether `sample` declares parameters the hardware session was not set up for: an AV1
/// sequence header that differs from `av1C`'s, a VP9 key frame of another size, bit depth or
/// chroma format. The decoder then hands the stream to our software decoder.
/// sequence header whose picture format (profile, size, colour configuration) differs from
/// `av1C`'s, a VP9 key frame of another size, bit depth or chroma format. The decoder then
/// hands the stream to our software decoder.
pub fn parameters_changed(&self, sample: &[u8]) -> bool {
match self.codec {
FrameCodec::Av1 => obus(sample).into_iter().any(|(t, p)| t == 1 && self.sequence_header.as_deref() != Some(p)),
// Only what the picture format depends on counts: encoders such as SVT-AV1 put a
// provisional sequence header in `av1C` whose coding tool flags (CDEF, restoration,
// warped motion…) differ from the in-band one, and decoders follow the in-band one.
FrameCodec::Av1 => obus(sample).into_iter().any(|(t, p)| t == 1 && self.sequence_header.as_deref().is_none_or(|s| av1_format(s) != av1_format(p))),
FrameCodec::Vp9 => {
self.picture_params(sample).is_some_and(|p| p.size != self.size || p.bit_depth != self.bit_depth || p.subsampling != self.subsampling)
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// An `av1C` sequence header from SVT-AV1 (1280x720 10-bit) and the in-band one of the same
/// stream: they differ only in coding tool flags.
const AV1C: [u8; 14] = [0x0a, 0x0c, 0x02, 0x00, 0x00, 0x2d, 0x6a, 0x67, 0xfd, 0x9e, 0x01, 0x7c, 0x20, 0x20];
const IN_BAND: [u8; 14] = [0x0a, 0x0c, 0x02, 0x00, 0x00, 0x2d, 0x6a, 0x67, 0xfd, 0x9e, 0x35, 0x7c, 0xe0, 0x20];
fn info() -> FrameStreamInfo {
let c = filmcraft_isobmff::Av1Config {
seq_profile: 0,
seq_level_idx_0: 0,
seq_tier_0: false,
high_bitdepth: true,
twelve_bit: false,
monochrome: false,
chroma_subsampling_x: true,
chroma_subsampling_y: true,
chroma_sample_position: 0,
initial_presentation_delay_minus_one: None,
config_obus: AV1C.to_vec(),
};
FrameStreamInfo::from_av1(&c, (0, 0)).unwrap()
}
#[test]
fn av1_sequence_headers_compare_by_picture_format() {
let i = info();
assert_eq!((i.size, i.bit_depth), ((1280, 720), 10));
assert!(!i.parameters_changed(&AV1C), "the same header");
// used to send every SVT-AV1 MP4 to the software decoder from its first frame
assert!(!i.parameters_changed(&IN_BAND), "only coding tool flags differ");
// a header that does not parse is a change
assert!(i.parameters_changed(&[0x0a, 0x02, 0xe0, 0x00]));
// no sequence header in the sample: nothing changed
assert!(!i.parameters_changed(&[0x12, 0x00]));
}
}
+6
View File
@@ -28,7 +28,9 @@ pub mod mp4;
pub mod mpeg;
pub mod mxf;
pub mod ogg;
mod stream_color;
pub mod video;
mod webm_alpha;
use std::sync::{Arc, RwLock};
@@ -173,7 +175,11 @@ pub fn open_bytes(name: &str, bytes: Arc<[u8]>) -> std::result::Result<filmcraft
filmcraft_media::open_bytes(name, bytes, &openers())
}
#[cfg(test)]
mod audio_timing_tests;
#[cfg(test)]
mod rounded_pts_tests;
#[cfg(test)]
mod stream_color_tests;
#[cfg(test)]
mod tests;
+312 -134
View File
@@ -8,11 +8,14 @@
//! subtracts `CodecDelay` from timestamps, so the pre-skip samples land before zero and are never
//! read; random access decodes `SeekPreRoll` (at least [`crate::audio::OPUS_PRE_ROLL`]) of preceding
//! packets before the target. Packet starts are accumulated from TOC durations (see [`opus_starts`]).
//!
//! AAC, MPEG audio and AC-3 packets each decode to a fixed number of samples, which millisecond
//! timestamps can't express: their packets run on from one another (see [`audio_starts`]).
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use filmcraft_color::{ColorInfo, Matrix, Primaries, Range, Transfer};
use filmcraft_color::{ColorInfo, Matrix, Transfer};
use filmcraft_frame::{AudioBuffer, VideoFrame};
use filmcraft_isobmff::{AvcConfig, CodecConfig, FourCc, HevcConfig, PcmConfig, SampleEntry, VpcConfig};
use filmcraft_matroska::{Codec, MkvFile, TrackKind};
@@ -21,6 +24,7 @@ use filmcraft_time::{FrameRate, Tick};
use crate::audio::{PacketDecoder, decode_pcm};
use crate::gop::{GopCache, VideoSamples};
use crate::stream_color::ColorCodes;
use crate::video::VideoDecoder;
use crate::{CodecError, make_video_decoder};
@@ -35,51 +39,136 @@ struct AudioState {
last_decoded: Option<usize>,
}
/// One playable audio track with its own decoder state and timing tables.
struct MkvAudio {
/// Index into `MkvFile::tracks`.
track: usize,
state: Mutex<AudioState>,
/// Packet start positions in source sample frames.
starts: Vec<i64>,
/// Decoder pre-roll after a seek, in source sample frames (0: prime with one packet).
preroll: i64,
}
pub struct MkvSource {
info: MediaInfo,
bytes: crate::Src,
file: MkvFile,
vtrack: Option<usize>,
atrack: Option<usize>,
/// Video codec as an ISO-BMFF sample entry (for the decoder factories).
ventry: Option<SampleEntry>,
video: GopCache,
audio: Mutex<AudioState>,
/// Audio packet start positions in source sample frames.
audio_starts: Vec<i64>,
/// Decoder pre-roll after a seek, in source sample frames (0: prime with one packet).
audio_preroll: i64,
/// The alpha layer of a VP9 track with `AlphaMode` 1 (WebM transparency, #402).
alpha: Option<Arc<crate::webm_alpha::AlphaLayer>>,
/// The playable audio tracks in file order: `info.audio_streams[k]` describes `audios[k]`.
audios: Vec<MkvAudio>,
}
/// Audio packet start positions in source sample frames.
///
/// Block timestamps are quantised to `TimestampScale` (usually 1 ms), so they are only accurate to a
/// tick: a 1024-sample AAC frame at 48 kHz lasts 21.33 ms. Packets that all decode to the same
/// length (AAC, MPEG audio, AC-3, E-AC-3) therefore run on from one another, resynchronising to the
/// timestamp only across gaps of more than half a packet (and more than two ticks); see
/// [`crate::audio::contiguous_starts`]. Opus packets: [`opus_starts`]. Other codecs start at their
/// timestamps.
fn audio_starts(file: &MkvFile, bytes: &crate::Src, ti: usize, rate: i64) -> Vec<i64> {
use crate::audio::{FixedFrames, PacketTime, contiguous_starts, fixed_packet_samples};
let Some(t) = file.tracks.get(ti) else { return Vec::new() };
if let Some(h) = opus_head(&t.codec) {
return opus_starts(file, bytes, ti, &h, rate);
}
let (n, d) = tb(t);
let at = |pts: i64| (pts as i128 * n as i128 * rate as i128 / d as i128) as i64;
let first = || file.read_sample(bytes, ti, 0).unwrap_or_default();
let out_rate = u32::try_from(rate).unwrap_or(0);
let frame = match &t.codec {
Codec::Aac { asc } => fixed_packet_samples(FixedFrames::Aac(asc), &[], out_rate),
Codec::Mp3 | Codec::Mp2 => fixed_packet_samples(FixedFrames::MpegAudio, &first(), out_rate),
Codec::Ac3 => fixed_packet_samples(FixedFrames::Ac3, &first(), out_rate),
Codec::Eac3 => fixed_packet_samples(FixedFrames::Eac3, &first(), out_rate),
_ => None,
};
let Some(frame) = frame else {
return t.samples.iter().map(|s| at(s.pts)).collect();
};
let packets = t.samples.iter().map(|s| PacketTime { stamp: at(s.pts), stamped: own_stamp(s), samples: Some(frame) });
contiguous_starts(packets, (frame / 2).max(stamp_tolerance(t, rate)))
}
/// Sample-exact Opus packet start positions (48 kHz frames, pre-skip removed).
///
/// Block timestamps are quantised to `TimestampScale` (usually 1 ms) and the demuxer subtracts a
/// rounded `CodecDelay`, so they are only accurate to a tick. Starts are therefore accumulated from
/// each packet's TOC duration, resynchronising to the timestamp only across real gaps (more than
/// two ticks off). Pre-skip is the exact `CodecDelay` (or the header's pre-skip when it is absent).
/// The demuxer also subtracts a rounded `CodecDelay` from the timestamps. Starts are accumulated
/// from each packet's TOC duration, resynchronising to the timestamp only across real gaps (more
/// than two ticks off). Pre-skip is the exact `CodecDelay` (or the header's pre-skip when it is
/// absent).
fn opus_starts(file: &MkvFile, bytes: &crate::Src, ti: usize, head: &filmcraft_opus::OpusHead, rate: i64) -> Vec<i64> {
let t = &file.tracks[ti];
let Some(t) = file.tracks.get(ti) else { return Vec::new() };
let (n, d) = tb(t);
let delay_ns = t.codec_delay_ns as i128;
// The demuxer's rounding of CodecDelay to ticks, undone here.
let delay_ticks = codec_delay_ticks(t);
let skip = if t.codec_delay_ns > 0 { (delay_ns * rate as i128 / 1_000_000_000) as i64 } else { head.pre_skip as i64 };
let at = |pts: i64| ((pts.saturating_add(delay_ticks) as i128 * n as i128 * rate as i128 / d as i128) as i64).saturating_sub(skip);
let packets = t.samples.iter().enumerate().map(|(i, s)| crate::audio::PacketTime {
stamp: at(s.pts),
stamped: own_stamp(s),
samples: file.read_sample(bytes, ti, i).ok().and_then(|p| crate::audio::opus_packet_samples(&p)).map(|k| k as i64),
});
crate::audio::contiguous_starts(packets, stamp_tolerance(t, rate))
}
/// `CodecDelay` in timestamp ticks, rounded as the demuxer rounds it (half away from zero) before
/// subtracting it from every timestamp of the track.
fn codec_delay_ticks(t: &filmcraft_matroska::Track) -> i64 {
let (n, d) = tb(t);
let scale_ns = (n as i128 * 1_000_000_000 / d as i128).max(1);
let delay_ns = t.codec_delay_ns as i128;
// The demuxer's rounding of CodecDelay to ticks (half away from zero), undone here.
let delay_ticks = ((delay_ns + scale_ns / 2) / scale_ns) as i64;
let skip = if t.codec_delay_ns > 0 { (delay_ns * rate as i128 / 1_000_000_000) as i64 } else { head.pre_skip as i64 };
let at = |pts: i64| ((pts + delay_ticks) as i128 * n as i128 * rate as i128 / d as i128) as i64 - skip;
let tolerance = (2 * scale_ns * rate as i128 / 1_000_000_000) as i64 + 1;
let mut starts = Vec::with_capacity(t.samples.len());
let mut next: Option<i64> = None;
for (i, s) in t.samples.iter().enumerate() {
let stamped = at(s.pts);
let start = match next {
Some(p) if (p - stamped).abs() <= tolerance => p,
_ => stamped,
};
starts.push(start);
let dur = file.read_sample(bytes, ti, i).ok().and_then(|p| crate::audio::opus_packet_samples(&p));
next = dur.map(|k| start + k as i64);
i64::try_from((t.codec_delay_ns as i128 + scale_ns / 2) / scale_ns).unwrap_or(i64::MAX)
}
/// The track timestamp that media time zero stands for (#715): the earliest start among the
/// playable tracks. Block timestamps count from the start of the Segment, and a remux that keeps
/// its source's timestamps (`ffmpeg -copyts`, a cut from a recording) starts its first picture
/// seconds in; taken as media time, that repeated the opening picture until then. An audio track
/// starts where its first packet does before `CodecDelay` was taken off (the delay only hides
/// priming), and never before zero.
fn media_origin(file: &MkvFile, vtrack: Option<usize>, atracks: &[usize]) -> i64 {
let start = |i: usize| {
let t = file.tracks.get(i)?;
let first = t.samples.iter().map(|s| s.pts).min()?;
Some(if t.kind == TrackKind::Audio { first.saturating_add(codec_delay_ticks(t)).max(0) } else { first })
};
vtrack.into_iter().chain(atracks.iter().copied()).filter_map(start).min().unwrap_or(0).max(0)
}
/// Move every track back by `origin` timestamp ticks, so the streams keep their relative timing
/// (a delay between picture and sound survives).
fn rebase(file: &mut MkvFile, origin: i64) {
if origin <= 0 {
return;
}
starts
for s in file.tracks.iter_mut().flat_map(|t| t.samples.iter_mut()) {
s.pts = s.pts.saturating_sub(origin);
}
}
/// End of the last sample of track `i`, in its timebase.
fn track_end(file: &MkvFile, i: usize) -> Option<(i64, &filmcraft_matroska::Track)> {
let t = file.tracks.get(i)?;
Some((t.samples.iter().map(|s| s.pts.saturating_add(i64::try_from(s.duration).unwrap_or(i64::MAX))).max().unwrap_or(0), t))
}
/// Two timestamp ticks in sample frames at `rate`, plus one: how far a packet's Matroska timestamp
/// can be from its exact start.
fn stamp_tolerance(t: &filmcraft_matroska::Track, rate: i64) -> i64 {
let (n, d) = tb(t);
i64::try_from(2 * n as i128 * rate as i128 / d as i128).unwrap_or(i64::MAX).saturating_add(1)
}
/// Whether a sample's timestamp is its own: frames after the first in a laced block without a
/// duration repeat the block's timestamp.
fn own_stamp(s: &filmcraft_matroska::Sample) -> bool {
s.lace == 0 || s.duration > 0
}
/// The parsed `OpusHead` of an `A_OPUS` track.
@@ -97,7 +186,7 @@ fn tb(t: &filmcraft_matroska::Track) -> (i64, i64) {
fn to_tick(t: &filmcraft_matroska::Track, pts: i64) -> Tick {
let (n, d) = tb(t);
Tick::from_rational(pts * n, 1, d)
Tick::from_rational(pts, n, d)
}
/// A time as the track timestamp to look a frame up by: rounded to the nearest tick, as muxers
@@ -172,32 +261,24 @@ fn hdr_metadata(c: &filmcraft_matroska::Colour) -> Option<filmcraft_color::HdrMe
})
}
fn color_of(v: &filmcraft_matroska::VideoInfo, w: u32, h: u32) -> (ColorInfo, bool) {
let mut c = ColorInfo { matrix: filmcraft_frame::default_matrix(w, h), transfer: Transfer::Bt709, primaries: Primaries::Bt709, range: Range::Limited };
let Some(col) = &v.colour else { return (c, false) };
let mut explicit = false;
if let Some(m) = col.matrix_coefficients.and_then(|m| Matrix::from_code(m as u8)) {
c.matrix = m;
explicit = true;
}
if let Some(t) = col.transfer_characteristics.and_then(|t| Transfer::from_code(t as u8)) {
c.transfer = t;
explicit = true;
}
if let Some(p) = col.primaries {
c.primaries = match p {
9 => Primaries::Bt2020,
12 => Primaries::P3D65,
5 => Primaries::Bt601_625,
6 => Primaries::Bt601_525,
_ => Primaries::Bt709,
/// Colour of a video track: the `Colour` element wins; what it leaves unspecified, or all of it
/// when there is none, comes from the stream's own description (the SPS VUI in an `avcC` /
/// `hvcC`, the sequence header in an `av1C`); the rest defaults by frame size. The flag says
/// whether `Colour` signals anything that overrides the decoder.
fn color_of(v: &filmcraft_matroska::VideoInfo, w: u32, h: u32, stream: Option<ColorCodes>) -> (ColorInfo, bool) {
// H.273 code points; `Range`: 1 broadcast (limited), 2 full, 0 / 3 not signalled here
let code = |c: Option<u64>| c.unwrap_or(2);
let container = v.colour.as_ref().map(|col| {
let range = match col.range {
Some(1) => Some(false),
Some(2) => Some(true),
_ => None,
};
}
if col.full_range() {
c.range = Range::Full;
explicit = true;
}
(c, explicit)
ColorCodes::from_wide(code(col.primaries), code(col.transfer_characteristics), code(col.matrix_coefficients), range)
});
let explicit = container.is_some_and(|c| Matrix::from_code(c.matrix).is_some() || Transfer::from_code(c.transfer).is_some() || c.full_range == Some(true));
let sources: Vec<ColorCodes> = [container, stream].into_iter().flatten().collect();
(crate::stream_color::resolve(w, h, &sources), explicit)
}
fn codec_label(c: &Codec) -> String {
@@ -234,12 +315,38 @@ impl MkvSource {
/// Open from a random-access reader: only the index is read now, samples on demand.
pub fn open_reader(name: &str, reader: filmcraft_media::SharedReader) -> crate::Result<Self> {
let bytes = crate::Src(reader);
let file = filmcraft_matroska::open(&bytes).map_err(|e| CodecError::Container(e.to_string()))?;
let mut file = filmcraft_matroska::open(&bytes).map_err(|e| CodecError::Container(e.to_string()))?;
let vtrack = file.tracks.iter().position(|t| t.kind == TrackKind::Video && !t.samples.is_empty());
let atrack = file.tracks.iter().position(|t| t.kind == TrackKind::Audio && !t.samples.is_empty());
if vtrack.is_none() && atrack.is_none() {
// every audio track is a stream (the track count comes from the file: capped)
let atracks: Vec<usize> = file
.tracks
.iter()
.enumerate()
.filter(|(_, t)| t.kind == TrackKind::Audio && !t.samples.is_empty())
.map(|(i, _)| i)
.take(filmcraft_media::MAX_AUDIO_STREAMS)
.collect();
if vtrack.is_none() && atracks.is_empty() {
return Err(CodecError::Unsupported("no playable tracks".into()));
}
let main = vtrack.or(atracks.first().copied());
// The Segment's end (`Info/Duration` or the last sample), in Segment time.
let segment_end = match file.duration_ns() {
Some(ns) => Tick::from_rational(i64::try_from(ns).unwrap_or(i64::MAX), 1, 1_000_000_000),
None => main.and_then(|i| track_end(&file, i)).map(|(end, t)| to_tick(t, end)).unwrap_or_default(),
};
// Media time zero is the first picture (or sound): the streams start at their common origin.
let origin = media_origin(&file, vtrack, &atracks);
rebase(&mut file, origin);
// `Info/Duration` is the Segment's end: the media lasts from the origin to there (never less
// than its main track's samples).
let duration = match main.and_then(|i| track_end(&file, i)) {
Some((end, t)) if origin > 0 => {
let (n, d) = tb(t);
(segment_end - Tick::from_rational(origin, n, d)).max(to_tick(t, end))
}
_ => segment_end,
};
let mut explicit_color = None;
let mut ventry = None;
// display rotation from the track's Projection (portrait phone video is stored landscape)
@@ -258,12 +365,12 @@ impl MkvSource {
FrameRate::from_f64(1.0 / step)
}
};
let (color, explicit) = color_of(&v, w, h);
ventry = sample_entry(&t.codec, &t.codec_private, t.video.as_ref(), w as u16, h as u16);
let (color, explicit) = color_of(&v, w, h, ventry.as_ref().and_then(|e| crate::stream_color::from_codec_config(&e.codec)));
if explicit {
explicit_color = Some(color);
}
ventry = sample_entry(&t.codec, &t.codec_private, t.video.as_ref(), w as u16, h as u16);
let secs = file.duration_ns().unwrap_or(0) as f64 / 1e9;
let secs = duration.seconds();
let bitrate = (secs > 0.0).then(|| (t.samples.iter().map(|s| s.size as u64).sum::<u64>() as f64 * 8.0 / secs) as u64);
let ((w, h), par) = if rotation % 2 == 1 { ((h, w), (v.pixel_aspect().1, v.pixel_aspect().0)) } else { ((w, h), v.pixel_aspect()) };
VideoStreamInfo {
@@ -279,75 +386,55 @@ impl MkvSource {
hdr: v.colour.as_ref().and_then(hdr_metadata),
}
});
let audio = atrack.map(|i| {
let t = &file.tracks[i];
let a = t.audio.clone().unwrap_or_default();
let mut rate = a.output_sampling_frequency.unwrap_or(a.sampling_frequency).round().max(1.0) as u32;
let mut channels = (a.channels as u32).max(1);
if let Some(h) = opus_head(&t.codec) {
rate = crate::audio::OPUS_RATE;
channels = h.channels as u32;
}
AudioStreamInfo { sample_rate: rate, channels, codec: codec_label(&t.codec), bits_per_sample: a.bit_depth.map(|b| b as u32) }
});
let duration = match file.duration_ns() {
Some(ns) => Tick::from_rational(ns as i64, 1, 1_000_000_000),
None => vtrack
.or(atrack)
.map(|i| {
let t = &file.tracks[i];
let end = t.samples.iter().map(|s| s.pts + s.duration as i64).max().unwrap_or(0);
to_tick(t, end)
})
.unwrap_or_default(),
};
let audio_streams: Vec<AudioStreamInfo> = atracks
.iter()
.filter_map(|&i| file.tracks.get(i))
.map(|t| {
let a = t.audio.clone().unwrap_or_default();
let mut rate = a.output_sampling_frequency.unwrap_or(a.sampling_frequency).round().max(1.0) as u32;
let mut channels = (a.channels as u32).max(1);
if let Some(h) = opus_head(&t.codec) {
rate = crate::audio::OPUS_RATE;
channels = h.channels as u32;
}
AudioStreamInfo { sample_rate: rate, channels, codec: codec_label(&t.codec), bits_per_sample: a.bit_depth.map(|b| b as u32) }
})
.collect();
let info = MediaInfo {
name: name.to_string(),
kind: if video.is_some() { MediaKind::Movie } else { MediaKind::AudioOnly },
duration,
video,
audio,
audio_streams,
container: if file.is_webm() { "WebM".into() } else { "Matroska".into() },
start_timecode: None,
file_size: Some(bytes.0.len()),
};
let audio_starts = atrack
.map(|i| {
let t = &file.tracks[i];
let rate = info.audio.as_ref().map_or(48_000, |a| a.sample_rate) as i64;
let (n, d) = tb(t);
let at = |pts: i64| (pts as i128 * n as i128 * rate as i128 / d as i128) as i64;
match opus_head(&t.codec) {
Some(h) => opus_starts(&file, &bytes, i, &h, rate),
None => t.samples.iter().map(|s| at(s.pts)).collect(),
}
})
.unwrap_or_default();
let audio_preroll = atrack
.map(|i| {
let t = &file.tracks[i];
let rate = info.audio.as_ref().map_or(48_000, |a| a.sample_rate) as i64;
match t.codec {
// `info.audio_streams[k]` was made from `atracks[k]`, so the two stay index-aligned
let audios: Vec<MkvAudio> = atracks
.iter()
.zip(&info.audio_streams)
.filter_map(|(&i, ainfo)| {
let t = file.tracks.get(i)?;
let rate = ainfo.sample_rate as i64;
let starts = audio_starts(&file, &bytes, i, rate);
let preroll = match t.codec {
Codec::Opus { .. } => {
let container = (t.seek_pre_roll_ns as u128 * rate as u128).div_ceil(1_000_000_000) as i64;
container.max(crate::audio::OPUS_PRE_ROLL as i64 * rate / 48_000)
}
_ => 0,
}
};
let state = Mutex::new(AudioState { decoder: None, packets: HashMap::new(), order: Vec::new(), last_decoded: None });
Some(MkvAudio { track: i, state, starts, preroll })
})
.unwrap_or(0);
Ok(Self {
info,
bytes,
file,
vtrack,
atrack,
ventry,
video: GopCache::new(explicit_color).with_rotation(rotation),
audio: Mutex::new(AudioState { decoder: None, packets: HashMap::new(), order: Vec::new(), last_decoded: None }),
audio_starts,
audio_preroll,
})
.collect();
// WebM transparency: VP9 alpha in each block's BlockAdditional (ID 1)
let alpha = vtrack.and_then(|i| file.tracks.get(i)).filter(|t| {
matches!(t.codec, Codec::Vp9 { .. }) && t.video.as_ref().is_some_and(|v| v.alpha_mode != 0) && t.samples.iter().any(|s| s.addition.is_some())
});
let alpha = alpha.map(|t| Arc::new(crate::webm_alpha::AlphaLayer::new(bytes.clone(), t.samples.iter().map(|s| (s.pts, s.addition)))));
Ok(Self { info, bytes, file, vtrack, ventry, video: GopCache::new(explicit_color).with_rotation(rotation), alpha, audios })
}
fn read(&self, track: usize, i: usize) -> crate::Result<Vec<u8>> {
@@ -361,7 +448,7 @@ impl MkvSource {
Codec::Opus { head } => PacketDecoder::opus(filmcraft_opus::OpusHead::parse(head).map_err(|e| CodecError::Unsupported(format!("Opus: {e}")))?),
Codec::Mp3 => PacketDecoder::new(CODEC_TYPE_MP3, rate, None),
Codec::Mp2 => PacketDecoder::mpeg_audio(2, rate),
Codec::Ac3 => PacketDecoder::ac3(),
Codec::Ac3 | Codec::Eac3 => PacketDecoder::ac3(),
// symphonia wants the STREAMINFO block body: skip `fLaC` + the 4-byte block header.
Codec::Flac { private } => PacketDecoder::new(CODEC_TYPE_FLAC, rate, private.get(8..42).map(<[u8]>::to_vec)),
Codec::Vorbis { headers } if headers.len() == 3 => {
@@ -373,13 +460,14 @@ impl MkvSource {
}
}
fn audio_packet(&self, st: &mut AudioState, i: usize) -> crate::Result<Arc<Vec<Vec<f32>>>> {
/// Decoded packet `i` of audio stream `a` (`ainfo` is its `AudioStreamInfo`; `st` its decoder state).
fn audio_packet(&self, a: &MkvAudio, ainfo: &AudioStreamInfo, st: &mut AudioState, i: usize) -> crate::Result<Arc<Vec<Vec<f32>>>> {
if let Some(p) = st.packets.get(&i) {
return Ok(p.clone());
}
let ti = self.atrack.ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
let track = &self.file.tracks[ti];
let ainfo = self.info.audio.as_ref().ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
let ti = a.track;
let track = self.file.tracks.get(ti).ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
let pk_start = a.starts.get(i).copied().ok_or_else(|| CodecError::Container("audio packet out of range".into()))?;
let data = self.read(ti, i)?;
let decoded = match &track.codec {
Codec::Pcm { float, big_endian, bits } => {
@@ -402,8 +490,8 @@ impl MkvSource {
if st.last_decoded.is_none_or(|l| l + 1 != i) {
let d = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?;
d.reset();
let from = if self.audio_preroll > 0 {
self.audio_starts.partition_point(|&x| x <= self.audio_starts[i] - self.audio_preroll).saturating_sub(1)
let from = if a.preroll > 0 {
a.starts.partition_point(|&x| x <= pk_start.saturating_sub(a.preroll)).saturating_sub(1)
} else {
i.saturating_sub(1)
};
@@ -413,7 +501,7 @@ impl MkvSource {
}
}
}
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, self.audio_starts[i].max(0) as u64);
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, pk_start.max(0) as u64);
st.last_decoded = Some(i);
r.unwrap_or_default()
}
@@ -452,9 +540,15 @@ impl VideoSamples for MkvVideo<'_> {
self.src.read(self.track, i)
}
fn make_decoder(&self) -> crate::Result<Box<dyn VideoDecoder>> {
match &self.src.ventry {
Some(e) => make_video_decoder(e),
None => Err(CodecError::Unsupported(format!("no decoder for {} video", codec_label(&self.src.file.tracks[self.track].codec)))),
let Some(e) = &self.src.ventry else {
return Err(CodecError::Unsupported(format!("no decoder for {} video", codec_label(&self.src.file.tracks[self.track].codec))));
};
let color = make_video_decoder(e)?;
let Some(layer) = &self.src.alpha else { return Ok(color) };
// the alpha layer always goes through our own decoder (it is luma only)
match crate::software_video_decoder(e) {
Ok(alpha) => Ok(Box::new(crate::webm_alpha::AlphaDecoder::new(color, alpha, layer.clone()))),
Err(_) => Ok(color),
}
}
}
@@ -473,23 +567,27 @@ impl MediaSource for MkvSource {
}
fn audio(&self, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
let ti = self.atrack.ok_or(MediaError::NoStream("audio"))?;
let n_packets = self.file.tracks[ti].samples.len();
let ainfo = self.info.audio.as_ref().ok_or(MediaError::NoStream("audio"))?;
self.audio_stream(0, start, frames, sample_rate)
}
fn audio_stream(&self, stream: usize, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
let a = self.audios.get(stream).ok_or(MediaError::NoStream("audio"))?;
let n_packets = self.file.tracks.get(a.track).ok_or(MediaError::NoStream("audio"))?.samples.len();
let ainfo = self.info.audio_streams.get(stream).ok_or(MediaError::NoStream("audio"))?;
let src_rate = ainfo.sample_rate;
let ch = ainfo.channels.max(1) as usize;
let ratio = src_rate as f64 / sample_rate as f64;
let s0 = (start as f64 * ratio).floor() as i64;
let need = (frames as f64 * ratio).ceil() as i64 + 2;
let mut src: Vec<Vec<f32>> = vec![vec![0.0; need.max(0) as usize]; ch];
let mut st = self.audio.lock().unwrap_or_else(|e| e.into_inner());
let mut i = self.audio_starts.partition_point(|&x| x <= s0).saturating_sub(1);
let mut st = a.state.lock().unwrap_or_else(|e| e.into_inner());
let mut i = a.starts.partition_point(|&x| x <= s0).saturating_sub(1);
while i < n_packets {
let pk_start = self.audio_starts[i];
let Some(&pk_start) = a.starts.get(i) else { break };
if pk_start >= s0 + need {
break;
}
let pk = self.audio_packet(&mut st, i)?;
let pk = self.audio_packet(a, ainfo, &mut st, i)?;
for (c, dst) in src.iter_mut().enumerate() {
let Some(chan) = pk.get(c.min(pk.len().saturating_sub(1))) else { continue };
for (k, v) in chan.iter().enumerate() {
@@ -532,3 +630,83 @@ pub fn reader_opener(name: &str, head: &[u8], reader: &filmcraft_media::SharedRe
}
Some(MkvSource::open_reader(name, reader.clone()).map(|s| Arc::new(s) as SharedSource).map_err(Into::into))
}
#[cfg(test)]
mod multi_audio_tests {
use filmcraft_matroska::{MkvWriter, MuxOptions, TrackKind, TrackSpec};
use filmcraft_media::MediaSource;
#[test]
fn every_audio_track_is_inspected_and_decoded_independently() {
let tracks: Vec<TrackSpec> = [(48_000.0, "first"), (24_000.0, "second")]
.into_iter()
.map(|(rate, name)| {
let mut t = TrackSpec::new(TrackKind::Audio, "A_PCM/INT/LIT");
t.audio = Some((rate, 1, Some(16)));
t.name = Some(name.into());
t
})
.collect();
let mut writer = MkvWriter::new(std::io::Cursor::new(Vec::new()), tracks, MuxOptions::default()).unwrap();
for (stream, value) in [(0, 8192i16), (1, -16384)] {
let data: Vec<u8> = (0..100).flat_map(|_| value.to_le_bytes()).collect();
writer.write_frame(stream, 0, true, &data, Some(4_000_000)).unwrap();
}
let source = super::MkvSource::open("two.mkv", writer.finish().unwrap().into_inner().into()).unwrap();
assert_eq!(source.info().audio_streams.iter().map(|a| a.sample_rate).collect::<Vec<_>>(), vec![48_000, 24_000]);
let primary = source.audio(0, 50, 48_000).unwrap();
let secondary = source.audio_stream(1, 0, 50, 48_000).unwrap();
assert!((primary.channels[0][20] - 0.25).abs() < 0.001);
assert!((secondary.channels[0][20] + 0.5).abs() < 0.001);
assert!(source.audio_stream(2, 0, 50, 48_000).is_err());
}
}
#[cfg(test)]
mod origin_tests {
use filmcraft_frame::PixelData;
use filmcraft_matroska::{MkvWriter, MuxOptions, TrackKind, TrackSpec};
use filmcraft_media::{FrameRequest, MediaSource};
use filmcraft_time::Tick;
/// A flat grey Motion-JPEG picture: its level says which frame it is.
fn picture(k: u8) -> Vec<u8> {
let img = image::GrayImage::from_pixel(16, 16, image::Luma([20 + 20 * k]));
let mut out = Vec::new();
image::codecs::jpeg::JpegEncoder::new_with_quality(&mut out, 95).encode_image(&img).unwrap();
out
}
/// #715: a remux that keeps its source's timestamps (first picture at 1.48 s, sound 0.2 s after
/// it) starts at its first picture, lasts as long as its pictures, and keeps the sound's delay.
/// It repeated picture 0 until 1.48 s, and the clip ran to the Segment's absolute end.
#[test]
fn timestamp_preserving_remux_starts_at_the_first_picture() {
let mut v = TrackSpec::new(TrackKind::Video, "V_MJPEG");
v.video_size = Some((16, 16));
v.default_duration_ns = Some(40_000_000);
let mut a = TrackSpec::new(TrackKind::Audio, "A_PCM/INT/LIT");
a.audio = Some((48_000.0, 1, Some(16)));
let mut w = MkvWriter::new(std::io::Cursor::new(Vec::new()), vec![v, a], MuxOptions::default()).unwrap();
for k in 0..10u8 {
let pts = 1_480_000_000 + i64::from(k) * 40_000_000;
w.write_frame(0, pts, true, &picture(k), None).unwrap();
if k == 5 {
let pcm: Vec<u8> = (0..960).flat_map(|_| 16_384i16.to_le_bytes()).collect();
w.write_frame(1, 1_680_000_000, true, &pcm, Some(20_000_000)).unwrap();
}
}
let src = super::MkvSource::open("shifted.mkv", w.finish().unwrap().into_inner().into()).unwrap();
assert_eq!(src.info().duration, Tick::from_rational(10, 1, 25), "ten pictures at 25 fps");
let level = |k: i64| {
let f = src.video_frame(FrameRequest::full(Tick::from_rational(k, 1, 25))).unwrap();
let PixelData::Rgba8(px) = &f.data else { panic!("MJPEG decodes to RGBA8") };
px[0]
};
let seen: Vec<u8> = (0..10).map(level).collect();
assert!(seen.windows(2).all(|p| p[0] < p[1]), "one picture per frame, in order: {seen:?}");
let sound = src.audio(0, 48_000 * 2 / 5, 48_000).unwrap();
assert!(sound.channels[0][9_590].abs() < 0.001, "silent before the delay");
assert!((sound.channels[0][9_610] - 0.5).abs() < 0.001, "sound 0.2 s after the first picture");
}
}
+431 -137
View File
@@ -7,9 +7,9 @@
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use filmcraft_color::{ColorInfo, Matrix, Primaries, Range, Transfer};
use filmcraft_frame::{AudioBuffer, VideoFrame};
use filmcraft_isobmff::{CodecConfig, Mp4File, TrackKind};
use filmcraft_color::ColorInfo;
use filmcraft_frame::{AudioBuffer, Region, VideoFrame};
use filmcraft_isobmff::{ByteSource, CleanAperture, CodecConfig, Mp4File, TrackKind};
use filmcraft_media::{AudioStreamInfo, FrameRequest, MediaError, MediaInfo, MediaKind, MediaSource, SharedSource, VideoStreamInfo};
use filmcraft_time::{FrameRate, Tick};
@@ -29,26 +29,69 @@ struct AudioState {
last_decoded: Option<usize>,
}
/// One playable audio track with its own decoder state and timing tables.
struct Mp4Audio {
/// Index into `Mp4File::tracks`.
track: usize,
state: Mutex<AudioState>,
/// Packet start positions in source sample frames (see [`audio_starts`]).
starts: Vec<i64>,
/// Presentation offset of the track in its timescale (edit list, or Opus pre-skip).
offset: i64,
/// Decoder pre-roll after a seek, in source sample frames (0: prime with one packet).
preroll: i64,
}
pub struct Mp4Source {
info: MediaInfo,
bytes: crate::Src,
file: Mp4File,
vtrack: Option<usize>,
atrack: Option<usize>,
video: GopCache,
audio: Mutex<AudioState>,
/// Cumulative sample start frames for the audio track (for packet lookup).
audio_starts: Vec<i64>,
/// Presentation offset of the audio track in its timescale (edit list, or Opus pre-skip).
audio_offset: i64,
/// Decoder pre-roll after a seek, in audio track timescale units (0: prime with one packet).
audio_preroll: i64,
/// The playable audio tracks in file order: `info.audio_streams[k]` describes `audios[k]`.
audios: Vec<Mp4Audio>,
}
pub fn sniff(b: &[u8]) -> bool {
b.len() >= 12 && matches!(&b[4..8], b"ftyp" | b"moov" | b"mdat" | b"wide" | b"free" | b"skip")
}
/// Audio packet start positions in source sample frames at `rate`: the running total of the sample
/// durations, except that packets which all decode to the same length (AAC, MPEG audio, AC-3,
/// E-AC-3) run on from one another, resynchronising to the total only across gaps of more than half
/// a packet.
/// A remux from Matroska (OBS's) carries the millisecond rounding of its timestamps into the
/// durations: 1008, 1008, 1056… for 1024-sample AAC frames at 48 kHz (see
/// [`crate::audio::contiguous_starts`]).
fn audio_starts(file: &Mp4File, bytes: &crate::Src, ti: usize, rate: u32) -> Vec<i64> {
use crate::audio::{FixedFrames, PacketTime, contiguous_starts, fixed_packet_samples};
let Some(t) = file.tracks.get(ti) else { return Vec::new() };
let ts = i128::from(t.timescale.max(1));
let mut total = 0i128;
let stamps: Vec<i64> = t
.samples
.iter()
.map(|s| {
let at = total * i128::from(rate) / ts;
total += i128::from(s.duration);
i64::try_from(at).unwrap_or(i64::MAX)
})
.collect();
let first = || file.read_sample(bytes, ti, 0).unwrap_or_default();
let frame = match t.entries.first().map(|e| &e.codec) {
Some(CodecConfig::Aac(a)) => fixed_packet_samples(FixedFrames::Aac(&a.asc), &[], rate),
Some(CodecConfig::Mp3) => fixed_packet_samples(FixedFrames::MpegAudio, &first(), rate),
Some(CodecConfig::Ac3 { .. }) => fixed_packet_samples(FixedFrames::Ac3, &first(), rate),
Some(CodecConfig::Eac3 { .. }) => fixed_packet_samples(FixedFrames::Eac3, &first(), rate),
_ => None,
};
let Some(frame) = frame else { return stamps };
// samples per track timescale unit, rounded up: how coarse the running total is
let unit = i64::try_from((i128::from(rate) + ts - 1) / ts).unwrap_or(i64::MAX);
let packets = stamps.iter().map(|&stamp| PacketTime { stamp, stamped: true, samples: Some(frame) });
contiguous_starts(packets, (frame / 2).max(unit.saturating_mul(2).saturating_add(1)))
}
/// HDR static metadata from the sample entry's `mdcv` / `clli` boxes (0 = unknown).
fn hdr_metadata(md: Option<&filmcraft_isobmff::MasteringDisplay>, cll: Option<(u16, u16)>) -> Option<filmcraft_color::HdrMetadata> {
if md.is_none() && cll.is_none() {
@@ -63,48 +106,104 @@ fn hdr_metadata(md: Option<&filmcraft_isobmff::MasteringDisplay>, cll: Option<(u
})
}
/// The clean aperture (`clap`, ISO/IEC 14496-12 / QuickTime) of a `w`×`h` decoded picture as a
/// pixel rectangle: its size, centred on the picture and moved by its offsets, rounded to whole
/// samples. `None` (the whole picture is shown) when it is the whole picture or makes no sense: a
/// zero denominator, an empty or larger-than-the-picture size, a rectangle outside the picture.
/// [`VideoFrame::cropped`] moves an odd corner onto the chroma grid.
fn clean_aperture(c: CleanAperture, (w, h): (u32, u32)) -> Option<Region> {
let q = |(n, d): (u32, u32)| (d != 0).then(|| f64::from(n) / f64::from(d));
let qs = |(n, d): (i32, u32)| (d != 0).then(|| f64::from(n) / f64::from(d));
let (cw, ch, dx, dy) = (q(c.width)?.round(), q(c.height)?.round(), qs(c.horiz_offset)?, qs(c.vert_offset)?);
let (fw, fh) = (f64::from(w), f64::from(h));
if !(cw >= 1.0 && ch >= 1.0 && cw <= fw && ch <= fh) {
return None;
}
let (x, y) = (((fw - cw) / 2.0 + dx).round(), ((fh - ch) / 2.0 + dy).round());
// also false for NaN
if !(x >= 0.0 && y >= 0.0 && x + cw <= fw && y + ch <= fh) {
return None;
}
let r = Region { x: x as usize, y: y as usize, w: cw as usize, h: ch as usize };
(!r.is_full(w as usize, h as usize)).then_some(r)
}
/// Colour of a video sample entry. The `colr` box (`nclx` / `nclc`) wins; what it leaves
/// unspecified, or all of it when there is none (ffmpeg writes MP4 without `colr` by default),
/// comes from the stream's own description: `vpcC` / `apvC`, or the SPS VUI in `avcC` / `hvcC`
/// and the sequence header in `av1C`. The rest defaults by frame size.
fn color_from(entry: &filmcraft_isobmff::SampleEntry, w: u32, h: u32) -> ColorInfo {
let mut c = ColorInfo { matrix: filmcraft_frame::default_matrix(w, h), transfer: Transfer::Bt709, primaries: Primaries::Bt709, range: Range::Limited };
// VP9 and APV carry their colour description in vpcC / apvC (used when there is no colr box).
let vpc = match &entry.codec {
CodecConfig::Vp9(v) => Some(filmcraft_isobmff::ColorInfo::Nclx {
primaries: v.colour_primaries as u16,
transfer: v.transfer_characteristics as u16,
matrix: v.matrix_coefficients as u16,
full_range: v.full_range,
}),
CodecConfig::Apv(a) if a.color_description_present => Some(filmcraft_isobmff::ColorInfo::Nclx {
primaries: a.color_primaries as u16,
transfer: a.transfer_characteristics as u16,
matrix: a.matrix_coefficients as u16,
full_range: a.full_range,
}),
use crate::stream_color::ColorCodes;
use filmcraft_isobmff::ColorInfo as Colr;
let colr = match entry.video.as_ref().and_then(|v| v.color.as_ref()) {
Some(Colr::Nclx { primaries, transfer, matrix, full_range }) => {
Some(ColorCodes::from_wide((*primaries).into(), (*transfer).into(), (*matrix).into(), Some(*full_range)))
}
// QuickTime `nclc` has no range flag
Some(Colr::Nclc { primaries, transfer, matrix }) => Some(ColorCodes::from_wide((*primaries).into(), (*transfer).into(), (*matrix).into(), None)),
// an ICC profile, or no `colr`: no code points
_ => None,
};
if let Some(col) = entry.video.as_ref().and_then(|v| v.color.as_ref()).or(vpc.as_ref()) {
let (p, t, m, full) = match col {
filmcraft_isobmff::ColorInfo::Nclx { primaries, transfer, matrix, full_range } => (*primaries, *transfer, *matrix, *full_range),
filmcraft_isobmff::ColorInfo::Nclc { primaries, transfer, matrix } => (*primaries, *transfer, *matrix, false),
_ => return c,
};
if let Some(m) = Matrix::from_code(m as u8) {
c.matrix = m;
}
if let Some(t) = Transfer::from_code(t as u8) {
c.transfer = t;
}
c.primaries = match p {
9 => Primaries::Bt2020,
12 => Primaries::P3D65,
5 => Primaries::Bt601_625,
6 => Primaries::Bt601_525,
_ => Primaries::Bt709,
};
if full {
c.range = Range::Full;
}
let stream = match &entry.codec {
CodecConfig::Vp9(v) => Some(ColorCodes {
primaries: v.colour_primaries,
transfer: v.transfer_characteristics,
matrix: v.matrix_coefficients,
full_range: Some(v.full_range),
}),
CodecConfig::Apv(a) if a.color_description_present => Some(ColorCodes {
primaries: a.color_primaries,
transfer: a.transfer_characteristics,
matrix: a.matrix_coefficients,
full_range: Some(a.full_range),
}),
CodecConfig::Apv(_) => None,
codec => crate::stream_color::from_codec_config(codec),
};
let sources: Vec<ColorCodes> = [colr, stream].into_iter().flatten().collect();
crate::stream_color::resolve(w, h, &sources)
}
/// First bytes of a Sony `rtmd` sample needed for the start-timecode block (`frames` is last).
const RTMD_HEADER_LEN: usize = 0x12;
/// Sony XAVC `rtmd` (real-time metadata) start timecode fields, read from the first sample: hours,
/// minutes, seconds, a drop-frame flag and frames. Sony stores them as raw decimal bytes in the
/// sample's fixed header. The layout is proprietary and undocumented, so it is reproduced here from
/// real footage; it is the same block ffmpeg reports as the `timecode` tag of an `rtmd` stream.
///
/// Fields outside SMPTE bounds — including the labels drop-frame counting skips — are rejected
/// (no fabricated start time from damaged metadata).
fn rtmd_fields(sample: &[u8], rate: FrameRate) -> Option<(i64, i64, i64, i64, bool)> {
let f = sample.get(0x0d..0x12)?;
let (h, m, s) = (f[0] as i64, f[1] as i64, f[2] as i64);
let frames = f[4] as i64;
let drop = f[3] != 0;
let base = rate.timecode_base();
if h > 23 || m > 59 || s > 59 || frames >= base {
return None;
}
c
// Drop-frame counting skips the first `base / 15` frame labels of every minute except each
// tenth minute, so those labels never occur in valid footage (SMPTE ST 12-1).
if drop && rate.supports_drop_frame() && s == 0 && m % 10 != 0 && frames < base / 15 {
return None;
}
Some((h, m, s, frames, drop))
}
/// Start timecode from a Sony `rtmd` track as a frame count at `rate` (issue #460), used when the
/// file has no `tmcd` track. `None` if there is no `rtmd` track or it has no usable first sample.
/// Only the fixed-size header is read (a damaged `stsz` size can't drive a huge allocation).
fn rtmd_start_timecode(file: &Mp4File, bytes: &crate::Src, rate: FrameRate) -> Option<i64> {
let t = file.tracks.iter().find(|t| t.entries.first().is_some_and(|e| e.format.0 == *b"rtmd"))?;
let s = t.samples.first()?;
if s.size < RTMD_HEADER_LEN as u32 {
return None;
}
let mut buf = [0u8; RTMD_HEADER_LEN];
bytes.read_at(s.offset, &mut buf).ok()?;
let (h, m, s, f, drop) = rtmd_fields(&buf, rate)?;
Some(filmcraft_time::fields_to_frames(h, m, s, f, rate, drop))
}
impl Mp4Source {
@@ -115,24 +214,36 @@ impl Mp4Source {
/// Open from a random-access reader: only the index is read now, samples on demand.
pub fn open_reader(name: &str, reader: filmcraft_media::SharedReader) -> crate::Result<Self> {
let bytes = crate::Src(reader);
let file = filmcraft_isobmff::open(&bytes).map_err(|e| CodecError::Container(e.to_string()))?;
let mut file = filmcraft_isobmff::open(&bytes).map_err(|e| CodecError::Container(e.to_string()))?;
// A track needs samples, a sample description and a timescale to be playable (damaged
// files can lack them: indexing `entries[0]` or dividing by the timescale used to panic).
let playable = |t: &&filmcraft_isobmff::Track, kind| t.kind == kind && !t.samples.is_empty() && !t.entries.is_empty() && t.timescale > 0;
let vtrack = file.tracks.iter().position(|t| playable(&t, TrackKind::Video));
let atrack = file.tracks.iter().position(|t| playable(&t, TrackKind::Audio));
if vtrack.is_none() && atrack.is_none() {
// every audio track is a stream (the track count comes from the file: capped)
let atracks: Vec<usize> =
file.tracks.iter().enumerate().filter(|(_, t)| playable(t, TrackKind::Audio)).map(|(i, _)| i).take(filmcraft_media::MAX_AUDIO_STREAMS).collect();
if vtrack.is_none() && atracks.is_empty() {
return Err(CodecError::Unsupported("no playable tracks".into()));
}
let mut color = ColorInfo::REC709;
let mut explicit_color = None;
// display rotation from the track matrix (portrait phone video is stored landscape)
let rotation = vtrack.and_then(|i| file.tracks[i].display_rotation()).unwrap_or(0);
// the clean aperture ('clap') cut from every frame, before the rotation
let mut aperture = None;
let video = vtrack.map(|i| {
let t = &file.tracks[i];
let entry = &t.entries[0];
let vp = entry.video.clone().unwrap_or_default();
let (w, h) = (if vp.width > 0 { vp.width as u32 } else { t.width }, if vp.height > 0 { vp.height as u32 } else { t.height });
// H.264 and HEVC say their own output size (frame cropping / conformance window), which
// is what the decoder returns; a sample entry can give the coded size instead (#288)
let (w, h) = match crate::hw::NalStreamInfo::from_entry(entry) {
Some(Ok(n)) if n.crop.2 > 0 && n.crop.3 > 0 => (n.crop.2, n.crop.3),
_ => (w, h),
};
aperture = vp.clean_aperture.and_then(|c| clean_aperture(c, (w, h)));
let (w, h) = aperture.map_or((w, h), |r| (r.w as u32, r.h as u32));
// frame rate from the median sample duration
let mut durs: Vec<u32> = t.samples.iter().take(240).map(|s| s.duration).collect();
durs.sort_unstable();
@@ -195,90 +306,49 @@ impl Mp4Source {
}
info
});
let audio = atrack.map(|i| {
let t = &file.tracks[i];
let entry = &t.entries[0];
let ap = entry.audio.clone().unwrap_or_default();
let (rate, ch, bits) = match &entry.codec {
CodecConfig::Aac(a) => (
{
// HE-AAC plays at twice the core rate the AudioSpecificConfig starts with
// an AAC access unit is at most 6144 bits per channel: a hostile `stsz` can't make us read more
let units = t
.samples
.iter()
.take(8)
.filter_map(|x| filmcraft_media::reader::read_range(&*bytes.0, x.offset, (x.size as usize).min(MAX_AAC_PROBE_UNIT)).ok());
let units: Vec<Vec<u8>> = units.collect();
match crate::audio::aac_output_rate(&a.asc, units.iter().map(Vec::as_slice)) {
Some(r) => r,
None if a.sample_rate > 0 => a.sample_rate,
None => ap.sample_rate as u32,
}
},
if a.channel_config > 0 { a.channel_config as u32 } else { ap.channels },
None,
),
CodecConfig::Pcm(p) => (p.sample_rate as u32, p.channels, Some(p.bits as u32)),
CodecConfig::Opus(o) => (crate::audio::OPUS_RATE, (o.output_channels as u32).max(1), None),
_ => (if ap.sample_rate > 0.0 { ap.sample_rate as u32 } else { t.timescale }, ap.channels.max(1), None),
};
AudioStreamInfo { sample_rate: rate.max(1), channels: ch.max(1), codec: codec_label(&entry.codec), bits_per_sample: bits }
});
let duration = vtrack.or(atrack).and_then(|i| file.tracks.get(i)).map(|t| presentation_duration(&file, t)).unwrap_or_default();
// one entry per playable audio track (`playable` guarantees a sample description); a track
// that somehow has none is dropped so `audio_streams` and `audios` stay index-aligned
let (atracks, audio_streams): (Vec<usize>, Vec<AudioStreamInfo>) =
atracks.into_iter().filter_map(|i| Some((i, audio_stream_info(file.tracks.get(i)?, &bytes)?))).unzip();
let duration = vtrack.or(atracks.first().copied()).and_then(|i| file.tracks.get(i)).map(|t| presentation_duration(&file, t)).unwrap_or_default();
let start_timecode = file.tracks.iter().find_map(|t| match t.codec() {
Some(CodecConfig::Timecode(tc)) => tc.start_frame.map(|f| f as i64),
_ => None,
});
// Sony XAVC files with no `tmcd` track carry the start timecode in an `rtmd` metadata
// track; convert its fields to a frame count using the video frame rate.
let start_timecode = start_timecode.or_else(|| rtmd_start_timecode(&file, &bytes, video.as_ref()?.frame_rate));
let info = MediaInfo {
name: name.to_string(),
kind: if video.is_some() { MediaKind::Movie } else { MediaKind::AudioOnly },
duration,
video,
audio,
audio_streams,
container: if file.is_quicktime { "QuickTime".into() } else { "MPEG-4".into() },
start_timecode,
file_size: Some(bytes.0.len()),
};
let audio_starts = atrack
.map(|i| {
let t = &file.tracks[i];
let mut acc = 0i64;
t.samples
.iter()
.map(|s| {
let st = acc;
acc += s.duration as i64;
st
})
.collect()
})
.unwrap_or_default();
let (audio_offset, audio_preroll) = atrack
.map(|i| {
let t = &file.tracks[i];
let mut audios: Vec<Mp4Audio> = atracks
.iter()
.zip(&info.audio_streams)
.filter_map(|(&i, ainfo)| {
let t = file.tracks.get(i)?;
let starts = audio_starts(&file, &bytes, i, ainfo.sample_rate);
let ts = t.timescale.max(1) as i64;
match &t.entries[0].codec {
CodecConfig::Opus(o) => {
let (offset, preroll) = match t.entries.first().map(|e| &e.codec) {
Some(CodecConfig::Opus(o)) => {
let off = if t.edit_offset != 0 { t.edit_offset } else { -(o.pre_skip as i64) * ts / 48_000 };
(off, (crate::audio::OPUS_PRE_ROLL as u64 * ts as u64).div_ceil(48_000) as i64)
// (Opus plays at 48 kHz: the pre-roll is in its sample frames)
(off, crate::audio::OPUS_PRE_ROLL as i64)
}
_ => (t.edit_offset, 0),
}
};
let state = Mutex::new(AudioState { decoder: None, packets: HashMap::new(), order: Vec::new(), last_decoded: None });
Some(Mp4Audio { track: i, state, starts, offset, preroll })
})
.unwrap_or((0, 0));
Ok(Self {
info,
bytes,
file,
vtrack,
atrack,
video: GopCache::new(explicit_color).with_rotation(rotation),
audio: Mutex::new(AudioState { decoder: None, packets: HashMap::new(), order: Vec::new(), last_decoded: None }),
audio_starts,
audio_offset,
audio_preroll,
})
.collect();
rebase_unedited_video(&mut file, vtrack, &mut audios);
Ok(Self { info, bytes, file, vtrack, video: GopCache::new(explicit_color).with_rotation(rotation).with_crop(aperture), audios })
}
fn read(&self, track: usize, i: usize) -> crate::Result<Vec<u8>> {
@@ -294,27 +364,30 @@ impl Mp4Source {
self.video.frame_late(&Mp4Video { src: self, track: ti }, t.to_rational_round(1, ts), late)
}
fn audio_packet(&self, st: &mut AudioState, i: usize) -> crate::Result<Arc<Vec<Vec<f32>>>> {
/// Decoded packet `i` of audio stream `a` (`ainfo` is its `AudioStreamInfo`; `st` its decoder state).
fn audio_packet(&self, a: &Mp4Audio, ainfo: &AudioStreamInfo, st: &mut AudioState, i: usize) -> crate::Result<Arc<Vec<Vec<f32>>>> {
if let Some(p) = st.packets.get(&i) {
return Ok(p.clone());
}
let ti = self.atrack.ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
let track = &self.file.tracks[ti];
let ti = a.track;
let track = self.file.tracks.get(ti).ok_or_else(|| CodecError::Unsupported("no audio track".into()))?;
let entry = track.entries.first().ok_or_else(|| CodecError::Container("audio track without a sample description".into()))?;
let sample = track.samples.get(i).ok_or_else(|| CodecError::Container("audio packet out of range".into()))?;
let pk_start = a.starts.get(i).copied().unwrap_or(0);
let data = self.read(ti, i)?;
let decoded = match &entry.codec {
CodecConfig::Pcm(p) => decode_pcm(&data, p),
c => {
if st.decoder.is_none() {
st.decoder = Some(PacketDecoder::for_isobmff(c, self.info.audio.as_ref().map_or(48_000, |a| a.sample_rate))?);
st.decoder = Some(PacketDecoder::for_isobmff(c, ainfo.sample_rate)?);
}
// Non-sequential access: reset and prime with the preceding packets (codec pre-roll:
// one packet, or `OPUS_PRE_ROLL` worth for Opus).
if st.last_decoded.is_none_or(|l| l + 1 != i) {
let d = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?;
d.reset();
let from = if self.audio_preroll > 0 {
self.audio_starts.partition_point(|&x| x <= self.audio_starts[i] - self.audio_preroll).saturating_sub(1)
let from = if a.preroll > 0 {
a.starts.partition_point(|&x| x <= pk_start.saturating_sub(a.preroll)).saturating_sub(1)
} else {
i.saturating_sub(1)
};
@@ -324,11 +397,11 @@ impl Mp4Source {
}
}
}
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, self.audio_starts[i].max(0) as u64);
let r = st.decoder.as_mut().ok_or_else(|| CodecError::Decode("no audio decoder".into()))?.decode(&data, pk_start.max(0) as u64);
st.last_decoded = Some(i);
match r {
Ok(v) => v,
Err(_) => vec![vec![0.0; track.samples[i].duration as usize]; self.info.audio.as_ref().map_or(2, |a| a.channels as usize)],
Err(_) => vec![vec![0.0; sample.duration as usize]; ainfo.channels.max(1) as usize],
}
}
};
@@ -381,30 +454,33 @@ impl MediaSource for Mp4Source {
}
fn audio(&self, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
let ti = self.atrack.ok_or(MediaError::NoStream("audio"))?;
let track = &self.file.tracks[ti];
let ainfo = self.info.audio.as_ref().ok_or(MediaError::NoStream("audio"))?;
self.audio_stream(0, start, frames, sample_rate)
}
fn audio_stream(&self, stream: usize, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
let a = self.audios.get(stream).ok_or(MediaError::NoStream("audio"))?;
let ti = a.track;
let track = self.file.tracks.get(ti).ok_or(MediaError::NoStream("audio"))?;
let ainfo = self.info.audio_streams.get(stream).ok_or(MediaError::NoStream("audio"))?;
let src_rate = ainfo.sample_rate;
let ch = ainfo.channels.max(1) as usize;
// Map the requested window to source samples (edit list offset applied: presentation = pts + edit_offset).
let ratio = src_rate as f64 / sample_rate as f64;
let p0 = (start as f64 * ratio).floor() as i64;
// (in i128: a hostile edit list can make the offset large enough to overflow i64)
let offset = i128::from(self.audio_offset) * i128::from(src_rate) / i128::from(track.timescale.max(1));
let offset = i128::from(a.offset) * i128::from(src_rate) / i128::from(track.timescale.max(1));
let s0 = i64::try_from(i128::from(p0) - offset).unwrap_or(if offset > 0 { i64::MIN } else { i64::MAX });
let need = (frames as f64 * ratio).ceil() as i64 + 2;
// Samples-per-unit: the track timescale is usually the sample rate for audio.
let unit = src_rate as f64 / track.timescale.max(1) as f64;
let mut src: Vec<Vec<f32>> = vec![vec![0.0; need.max(0) as usize]; ch];
let mut st = self.audio.lock().unwrap_or_else(|e| e.into_inner());
let first = self.audio_starts.partition_point(|&x| (x as f64 * unit) as i64 <= s0.max(0)).saturating_sub(1);
let mut st = a.state.lock().unwrap_or_else(|e| e.into_inner());
let first = a.starts.partition_point(|&x| x <= s0.max(0)).saturating_sub(1);
let mut i = first;
while i < track.samples.len() {
let pk_start = (self.audio_starts[i] as f64 * unit) as i64;
let Some(&pk_start) = a.starts.get(i) else { break };
if pk_start >= s0.saturating_add(need) {
break;
}
let pk = self.audio_packet(&mut st, i)?;
let pk = self.audio_packet(a, ainfo, &mut st, i)?;
for (c, dst) in src.iter_mut().enumerate() {
let Some(chan) = pk.get(c.min(pk.len().saturating_sub(1))) else { continue };
for (k, v) in chan.iter().enumerate() {
@@ -443,6 +519,48 @@ impl MediaSource for Mp4Source {
}
}
/// What the file says about one audio track (`None`: the track has no sample description).
fn audio_stream_info(t: &filmcraft_isobmff::Track, bytes: &crate::Src) -> Option<AudioStreamInfo> {
let entry = t.entries.first()?;
let ap = entry.audio.clone().unwrap_or_default();
let (rate, ch, bits) = match &entry.codec {
CodecConfig::Aac(a) => (
{
// HE-AAC plays at twice the core rate the AudioSpecificConfig starts with
// an AAC access unit is at most 6144 bits per channel: a hostile `stsz` can't make us read more
let units = t
.samples
.iter()
.take(8)
.filter_map(|x| filmcraft_media::reader::read_range(&*bytes.0, x.offset, (x.size as usize).min(MAX_AAC_PROBE_UNIT)).ok());
let units: Vec<Vec<u8>> = units.collect();
match crate::audio::aac_output_rate(&a.asc, units.iter().map(Vec::as_slice)) {
Some(r) => r,
None if a.sample_rate > 0 => a.sample_rate,
None => ap.sample_rate as u32,
}
},
if a.channel_config > 0 { a.channel_config as u32 } else { ap.channels },
None,
),
CodecConfig::Pcm(p) => (p.sample_rate as u32, p.channels, Some(p.bits as u32)),
CodecConfig::Opus(o) => (crate::audio::OPUS_RATE, (o.output_channels as u32).max(1), None),
_ => (if ap.sample_rate > 0.0 { ap.sample_rate as u32 } else { t.timescale }, ap.channels.max(1), None),
};
let codec = match &entry.codec {
// these entries name no layer (ffmpeg stores MP2 as `mp4a` + 0x6B): the first frame does
CodecConfig::Mp3 => {
let head = t.samples.first().and_then(|s| filmcraft_media::reader::read_range(&*bytes.0, s.offset, (s.size as usize).min(4)).ok());
match head.as_deref().and_then(crate::audio::mpeg_audio_layer) {
Some(1 | 2) => "MPEG Audio".into(),
_ => codec_label(&entry.codec),
}
}
c => codec_label(c),
};
Some(AudioStreamInfo { sample_rate: rate.max(1), channels: ch.max(1), codec, bits_per_sample: bits })
}
fn codec_label(c: &CodecConfig) -> String {
match c {
CodecConfig::Avc(a) => format!(
@@ -546,6 +664,35 @@ pub fn opener(name: &str, bytes: Arc<[u8]>) -> Option<Result<SharedSource, Media
Some(Mp4Source::open(name, bytes).map(|s| Arc::new(s) as SharedSource).map_err(Into::into))
}
/// Start the file at its first picture when its video track has no edit list (#714). Without an
/// edit list the media timeline is the presentation, so a B-frame stream's first picture sits at its
/// composition delay (two frames on an x264 MP4 written without `elst`, and on fragmented MP4s):
/// media time zero repeated it and the clip, which lasts the media duration, lost its last pictures.
/// Every track moves back by the earliest start among them, video and audio alike, so the streams
/// stay in sync and a delay between them keeps its meaning; a track with an edit list is never
/// rebased on its own.
fn rebase_unedited_video(file: &mut Mp4File, vtrack: Option<usize>, audios: &mut [Mp4Audio]) {
let Some(vi) = vtrack else { return };
let Some(t) = file.tracks.get(vi).filter(|t| t.edits.is_empty() && t.timescale > 0) else { return };
let Some(first) = t.samples.iter().map(|s| s.pts).min().map(|p| p.saturating_add(t.edit_offset)).filter(|&p| p > 0) else { return };
let vts = i64::from(t.timescale);
let video_start = Tick::from_rational(first, 1, vts);
let audio_ts = |a: &Mp4Audio| file.tracks.get(a.track).map_or(1, |t| i64::from(t.timescale.max(1)));
// an audio track starts at its offset (a negative one only hides priming before zero)
let shift = audios.iter().fold(video_start, |m, a| m.min(Tick::from_rational(a.offset.max(0), 1, audio_ts(a))));
if shift <= Tick::ZERO {
return;
}
for a in audios.iter_mut() {
let d = shift.to_rational_round(1, audio_ts(a));
a.offset = a.offset.saturating_sub(d);
}
let v = if shift == video_start { first } else { shift.to_rational_round(1, vts) };
if let Some(t) = file.tracks.get_mut(vi) {
t.edit_offset = t.edit_offset.saturating_sub(v);
}
}
/// How long a track plays. With an edit list that is the sum of its edits (ISO/IEC 14496-12
/// §8.6.6, movie timescale), as `tkhd` records it. The media duration (`mdhd`) runs on the decode
/// timeline: it is longer than the presentation by the B-frame delay an edit skips (`media_time`, one
@@ -641,6 +788,30 @@ mod tests {
assert_eq!(frames(&Mp4Source::open("zero.mov", mov_with_edits(zero)).expect("open")), 4);
}
/// #714: a video track without an edit list whose pictures carry a composition delay (B-frame
/// style, as x264 writes them with no `elst`) starts at its first picture. Media time zero
/// repeated picture 0 for the length of the delay and the last pictures never showed.
#[test]
fn unedited_composition_delay_starts_at_the_first_picture() {
let mut mux = Mp4Writer::new(std::io::Cursor::new(Vec::new()), WriterOptions::new(Brand::Mov)).expect("writer");
let t = mux.add_track(TrackConfig::new(SampleEntry::prores(FourCc(*b"apch"), 64, 32), 25)).expect("track");
for k in 0..4u16 {
let mut fr = filmcraft_prores::Frame::new(64, 32, filmcraft_prores::ChromaFormat::Yuv422, 10, false);
fr.y.fill(100 + 200 * k);
let data = filmcraft_prores::Encoder::new(filmcraft_prores::Profile::Hq, 64, 32).encode(&fr).expect("encode");
mux.write_sample(t, WriteSample { data: &data, duration: 1, composition_offset: 2, is_sync: true }).expect("sample");
}
let src = Mp4Source::open("delay.mov", mux.finish().expect("finish").into_inner().into()).expect("open");
assert_eq!(src.info().duration, Tick::from_rational(4, 1, 25));
let luma = |k: i64| {
let f = src.video_frame(FrameRequest::full(Tick::from_rational(k, 1, 25))).expect("frame");
let l = f.luma8();
l.iter().map(|&v| u32::from(v)).sum::<u32>() / l.len().max(1) as u32
};
let seen: Vec<u32> = (0..4).map(luma).collect();
assert!(seen.windows(2).all(|w| w[0] < w[1]), "one picture per frame, in order: {seen:?}");
}
/// Overwrite the big-endian u32 `skip` bytes after the first `fourcc` box type.
fn patch_u32(b: &mut [u8], fourcc: &[u8; 4], skip: usize, v: u32) {
let k = b.windows(4).position(|x| x == fourcc).expect("box") + 4 + skip;
@@ -708,6 +879,60 @@ mod tests {
assert_eq!(s.info().video.as_ref().map(|v| (v.width, v.height)), Some((64, 32)));
}
/// A one-frame 25 fps ProRes MOV (64×32, left half bright) with clean aperture `clap`.
fn clap_mov(clap: filmcraft_isobmff::CleanAperture) -> Arc<[u8]> {
let (w, h) = (64u32, 32u32);
let mut fr = filmcraft_prores::Frame::new(w, h, filmcraft_prores::ChromaFormat::Yuv422, 10, false);
for (i, y) in fr.y.iter_mut().enumerate() {
*y = if (i as u32 % w) < w / 2 { 800 } else { 100 };
}
let data = filmcraft_prores::Encoder::new(filmcraft_prores::Profile::Hq, w, h).encode(&fr).expect("encode");
let mut entry = SampleEntry::prores(FourCc(*b"apch"), w as u16, h as u16);
entry.video.as_mut().expect("video").clean_aperture = Some(clap);
let mut mux = Mp4Writer::new(std::io::Cursor::new(Vec::new()), WriterOptions::new(Brand::Mov)).expect("writer");
let t = mux.add_track(TrackConfig::new(entry, 25)).expect("track");
mux.write_sample(t, WriteSample { data: &data, duration: 1, composition_offset: 0, is_sync: true }).expect("sample");
mux.finish().expect("finish").into_inner().into()
}
#[test]
fn clean_aperture_crops_frames_and_reported_size() {
use filmcraft_isobmff::CleanAperture;
let clap = |w: (u32, u32), h: (u32, u32), dx: (i32, u32), dy: (i32, u32)| CleanAperture { width: w, height: h, horiz_offset: dx, vert_offset: dy };
// 32x16 moved 8 left of centre: columns 8..40 (24 bright, then 8 dark), rows 8..24
let s = Mp4Source::open("clap.mov", clap_mov(clap((32, 1), (16, 1), (-8, 1), (0, 1)))).expect("open");
assert_eq!(s.info().video.as_ref().map(|v| (v.width, v.height)), Some((32, 16)));
let f = s.video_frame(FrameRequest::full(Tick::ZERO)).expect("frame");
assert_eq!((f.width, f.height), (32, 16));
let l = f.luma8();
let col = |x: usize| (0..16).map(|y| u32::from(l[y * 32 + x])).sum::<u32>() / 16;
assert!(col(0) > col(31) + 100 && col(23) > col(24) + 100, "bright up to column 23: {} {} {} {}", col(0), col(23), col(24), col(31));
// rational sizes and offsets (64/2 = 32): the same rectangle
let s = Mp4Source::open("clap2.mov", clap_mov(clap((64, 2), (48, 3), (-16, 2), (0, 7)))).expect("open");
assert_eq!(s.info().video.as_ref().map(|v| (v.width, v.height)), Some((32, 16)));
// a clean aperture that makes no sense is ignored (the whole picture), never a panic
for c in [
clap((0, 1), (16, 1), (0, 1), (0, 1)),
clap((32, 0), (16, 1), (0, 1), (0, 1)),
clap((32, 1), (16, 1), (0, 0), (0, 1)),
clap((65, 1), (16, 1), (0, 1), (0, 1)),
clap((32, 1), (33, 1), (0, 1), (0, 1)),
clap((32, 1), (16, 1), (17, 1), (0, 1)),
clap((32, 1), (16, 1), (0, 1), (-9, 1)),
clap((32, 1), (16, 1), (i32::MIN, 1), (i32::MAX, 1)),
clap((u32::MAX, 1), (u32::MAX, 1), (0, 1), (0, 1)),
clap((1, u32::MAX), (16, 1), (0, u32::MAX), (0, 1)),
clap((64, 1), (32, 1), (0, 1), (0, 1)),
] {
let s = Mp4Source::open("bad.mov", clap_mov(c)).expect("open");
assert_eq!(s.info().video.as_ref().map(|v| (v.width, v.height)), Some((64, 32)), "{c:?}");
let f = s.video_frame(FrameRequest::full(Tick::ZERO)).expect("frame");
assert_eq!((f.width, f.height), (64, 32), "{c:?}");
}
}
#[test]
fn mirrored_or_scaled_matrices_are_not_mistaken_for_rotations() {
const ONE: i32 = 0x10000;
@@ -801,6 +1026,75 @@ mod tests {
assert_eq!((v.pixel_format.as_str(), v.has_alpha), (want, alpha), "{}", String::from_utf8_lossy(fourcc));
}
}
#[test]
fn rtmd_fields_parse_and_reject_damaged_metadata() {
let rate = FrameRate::FPS_59_94;
// 03:37:12:34, non-drop (a Sony FX3/a6400 XAVC header).
let mut s = [0u8; 24];
s[0x0d..0x12].copy_from_slice(&[3, 37, 12, 0, 34]);
assert_eq!(rtmd_fields(&s, rate), Some((3, 37, 12, 34, false)));
s[0x10] = 1;
assert_eq!(rtmd_fields(&s, rate), Some((3, 37, 12, 34, true)));
// Too short to hold the block: no fields, no panic.
assert_eq!(rtmd_fields(&[0u8; 6], rate), None);
// Damaged fields are rejected rather than normalized into a plausible time.
for bad in [[3, 37, 12, 0, 60], [3, 60, 0, 0, 0], [24, 0, 0, 0, 0], [0, 0, 60, 0, 0]] {
let mut b = [0u8; 24];
b[0x0d..0x12].copy_from_slice(&bad);
assert_eq!(rtmd_fields(&b, rate), None, "{bad:?}");
}
}
#[test]
fn rtmd_fields_reject_drop_frame_skipped_labels() {
let df = |h, m, s, f| {
let mut b = [0u8; 24];
b[0x0d..0x12].copy_from_slice(&[h, m, s, 1, f]);
rtmd_fields(&b, FrameRate::FPS_59_94)
};
// At 59.94 DF, frames 00-03 of a non-tenth minute are skipped, so they never occur.
assert_eq!(df(0, 1, 0, 0), None);
assert_eq!(df(0, 1, 0, 3), None);
assert_eq!(df(0, 1, 0, 4), Some((0, 1, 0, 4, true)));
assert_eq!(df(0, 10, 0, 0), Some((0, 10, 0, 0, true))); // tenth minute: label exists
assert_eq!(df(0, 1, 1, 0), Some((0, 1, 1, 0, true))); // seconds > 0: label exists
// At 29.97 DF only frames 00-01 are skipped.
let df30 = |m, s, f| {
let mut b = [0u8; 24];
b[0x0d..0x12].copy_from_slice(&[0, m, s, 1, f]);
rtmd_fields(&b, FrameRate::FPS_29_97)
};
assert_eq!(df30(1, 0, 0), None);
assert_eq!(df30(1, 0, 2), Some((0, 1, 0, 2, true)));
}
}
#[cfg(test)]
mod multi_audio_tests {
use filmcraft_isobmff::{Brand, Mp4Writer, PcmConfig, SampleEntry, TrackConfig, WriteSample, WriterOptions};
use filmcraft_media::MediaSource;
/// Two mono PCM tracks with different rates and samples. Stream indices follow file order;
/// decoder caches, sample timing and resampling must not cross streams.
#[test]
fn every_audio_track_is_inspected_and_decoded_independently() {
let mut mux = Mp4Writer::new(std::io::Cursor::new(Vec::new()), WriterOptions::new(Brand::Mov)).unwrap();
for (rate, value) in [(48_000u32, 8192i16), (24_000, -16384)] {
let entry = SampleEntry::pcm(PcmConfig { bits: 16, float: false, big_endian: false, signed: true, channels: 1, sample_rate: rate as f64 });
let track = mux.add_track(TrackConfig::new(entry, rate)).unwrap();
let data: Vec<u8> = (0..rate / 10).flat_map(|_| value.to_le_bytes()).collect();
mux.write_sample(track, WriteSample { data: &data, duration: rate / 10, composition_offset: 0, is_sync: true }).unwrap();
}
let bytes = mux.finish().unwrap().into_inner();
let source = super::Mp4Source::open("two.mov", bytes.into()).unwrap();
assert_eq!(source.info().audio_streams.iter().map(|a| a.sample_rate).collect::<Vec<_>>(), vec![48_000, 24_000]);
let primary = source.audio(0, 100, 48_000).unwrap();
let secondary = source.audio_stream(1, 0, 100, 48_000).unwrap();
assert!((primary.channels[0][50] - 0.25).abs() < 0.001);
assert!((secondary.channels[0][50] + 0.5).abs() < 0.001);
assert!(source.audio_stream(2, 0, 100, 48_000).is_err());
}
}
#[cfg(test)]
@@ -829,7 +1123,7 @@ mod he_aac_tests {
}
let b = mux.finish().unwrap().into_inner();
let src = super::Mp4Source::open("he.mp4", b.into()).unwrap();
assert_eq!(src.info().audio.as_ref().unwrap().sample_rate, 44_100);
assert_eq!(src.info().audio().unwrap().sample_rate, 44_100);
// one second from 0.5 s at 44.1 kHz: the 1 kHz tone at its level (not silence, not shifted)
let buf = src.audio(22_050, 44_100, 44_100).unwrap();
let x = &buf.channels[0];
+8 -6
View File
@@ -120,7 +120,7 @@ fn vkind(c: &Codec) -> Option<VKind> {
}
fn audio_supported(c: &Codec) -> bool {
matches!(c, Codec::MpegAudio | Codec::AacAdts | Codec::AacLatm | Codec::LpcmBluray | Codec::LpcmDvd | Codec::Ac3)
matches!(c, Codec::MpegAudio | Codec::AacAdts | Codec::AacLatm | Codec::LpcmBluray | Codec::LpcmDvd | Codec::Ac3 | Codec::Eac3)
}
/// The coded rate unless the timestamps say otherwise (3:2 pulldown, field-rate coding). Never
@@ -211,7 +211,7 @@ impl MpegSource {
kind: MediaKind::Movie,
duration: Tick::ZERO,
video: None,
audio: None,
audio_streams: Vec::new(),
container: file.format.name().to_string(),
start_timecode: None,
file_size: Some(bytes.0.len()),
@@ -316,7 +316,7 @@ impl MpegSource {
kind: MediaKind::Movie,
duration: Tick::ZERO,
video: None,
audio: None,
audio_streams: Vec::new(),
container: "MPEG video elementary stream".into(),
start_timecode: None,
file_size: Some(bytes.0.len()),
@@ -485,7 +485,8 @@ impl MpegSource {
/// Mark the audio stream as not decodable (it still shows in the media info).
fn audio_unsupported(&mut self, codec_name: &str, why: String) -> Option<AudioTrack> {
self.unsupported_audio = Some(why);
self.info.audio = Some(AudioStreamInfo { sample_rate: 48_000, channels: 2, codec: format!("{codec_name} (unsupported)"), bits_per_sample: None });
self.info.audio_streams =
vec![AudioStreamInfo { sample_rate: 48_000, channels: 2, codec: format!("{codec_name} (unsupported)"), bits_per_sample: None }];
None
}
@@ -504,7 +505,7 @@ impl MpegSource {
};
let mut latm = None;
let (codec, rate, channels, per_frame, bits) = match &st.codec {
Codec::MpegAudio | Codec::AacAdts | Codec::Ac3 => {
Codec::MpegAudio | Codec::AacAdts | Codec::Ac3 | Codec::Eac3 => {
let Some(fi) = filmcraft_mpegts::frame_info(&st.codec, &first) else {
return self.audio_unsupported(&codec_name, format!("{codec_name}: bad frame header"));
};
@@ -599,7 +600,8 @@ impl MpegSource {
self.info.kind = MediaKind::AudioOnly;
}
}
self.info.audio = Some(AudioStreamInfo { sample_rate: rate.max(1), channels: channels.max(1) as u32, codec: codec_name, bits_per_sample: bits });
self.info.audio_streams =
vec![AudioStreamInfo { sample_rate: rate.max(1), channels: channels.max(1) as u32, codec: codec_name, bits_per_sample: bits }];
Some(AudioTrack {
stream: ai,
codec,
+2 -2
View File
@@ -361,7 +361,7 @@ impl MxfSource {
kind: if video.is_some() { MediaKind::Movie } else { MediaKind::AudioOnly },
duration,
video,
audio,
audio_streams: audio.into_iter().collect(),
container: format!("MXF {}", file.operational_pattern.name()),
start_timecode: None,
file_size: Some(bytes.0.len()),
@@ -457,7 +457,7 @@ impl MediaSource for MxfSource {
}
fn audio(&self, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
let ainfo = self.info.audio.as_ref().ok_or(MediaError::NoStream("audio"))?;
let ainfo = self.info.audio().ok_or(MediaError::NoStream("audio"))?;
let src_rate = ainfo.sample_rate;
let ch = ainfo.channels.max(1) as usize;
let mut out = AudioBuffer::silence(sample_rate, ch, frames);
+1 -1
View File
@@ -110,7 +110,7 @@ impl OggSource {
kind: MediaKind::AudioOnly,
duration: Tick::from_units(frames, rate as i64),
video: None,
audio: Some(AudioStreamInfo { sample_rate: rate, channels: channels.max(1), codec: codec.into(), bits_per_sample: None }),
audio_streams: vec![AudioStreamInfo { sample_rate: rate, channels: channels.max(1), codec: codec.into(), bits_per_sample: None }],
container: "Ogg".into(),
start_timecode: None,
file_size: Some(bytes.0.len()),
+92
View File
@@ -0,0 +1,92 @@
//! The colour description a video stream carries in its own bitstream, for containers that don't
//! repeat it: the H.264 / HEVC sequence parameter set's VUI (from `avcC` / `hvcC`) and the AV1
//! sequence header's `color_config` (from `av1C`), read with the decoders' own parsers.
//!
//! ffmpeg writes MP4 without a `colr` box unless asked (`-movflags +write_colr`), so for most
//! files the bitstream is the only place their colour is described. [`resolve`] merges it with
//! what the container signals, which wins.
use filmcraft_color::{ColorInfo, Matrix, Range, Transfer};
use filmcraft_isobmff::{AvcConfig, CodecConfig, HevcConfig};
use crate::video::primaries_from_code;
/// ITU-T H.273 `Unspecified`.
const UNSPECIFIED: u8 = 2;
/// Colour code points (ITU-T H.273 / ISO/IEC 23091-2) and range as one place in a file (a
/// container colour box, a sequence header) signals them.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct ColorCodes {
pub primaries: u8,
pub transfer: u8,
pub matrix: u8,
/// `None`: this place doesn't say.
pub full_range: Option<bool>,
}
impl ColorCodes {
/// From wider code points (a `colr` box or a Matroska `Colour` element): a value that doesn't
/// fit a byte is no H.273 code point, so it counts as unspecified.
pub fn from_wide(primaries: u64, transfer: u64, matrix: u64, full_range: Option<bool>) -> Self {
let code = |v: u64| u8::try_from(v).unwrap_or(UNSPECIFIED);
Self { primaries: code(primaries), transfer: code(transfer), matrix: code(matrix), full_range }
}
}
/// The colour a video codec configuration's bitstream signals: the VUI of the first SPS in an
/// `avcC` / `hvcC`, or the `color_config` of the sequence header in an `av1C`. `None` for other
/// codecs, an SPS without VUI, and a configuration that is truncated or corrupt.
pub fn from_codec_config(codec: &CodecConfig) -> Option<ColorCodes> {
match codec {
CodecConfig::Avc(c) => avc(c),
CodecConfig::Hevc(c) => hevc(c),
CodecConfig::Av1(c) => av1(&c.config_obus),
_ => None,
}
}
fn avc(c: &AvcConfig) -> Option<ColorCodes> {
// NAL header byte: nal_unit_type 7 is an SPS
let nal = c.sps.iter().find(|n| n.first().is_some_and(|h| h & 0x1f == 7))?;
let sps = filmcraft_h264::params::Sps::parse(&filmcraft_bitstream::unescape_rbsp(nal.get(1..)?)).ok()?;
let v = sps.vui?;
Some(ColorCodes { primaries: v.colour_primaries, transfer: v.transfer_characteristics, matrix: v.matrix_coefficients, full_range: Some(v.full_range) })
}
fn hevc(c: &HevcConfig) -> Option<ColorCodes> {
// two-byte NAL header: nal_unit_type 33 is an SPS (whatever array the record files it under)
let nal = c.arrays.iter().flat_map(|a| &a.nalus).find(|n| n.first().is_some_and(|h| (h >> 1) & 0x3f == 33))?;
let sps = filmcraft_hevc::params::Sps::parse(&filmcraft_bitstream::unescape_rbsp(nal.get(2..)?)).ok()?;
let v = sps.vui?;
Some(ColorCodes { primaries: v.colour_primaries, transfer: v.transfer_characteristics, matrix: v.matrix_coefficients, full_range: Some(v.full_range) })
}
fn av1(config_obus: &[u8]) -> Option<ColorCodes> {
// OBU type 1: the sequence header
let (_, payload) = crate::hw_frame::obus(config_obus).into_iter().find(|(t, _)| *t == 1)?;
let c = filmcraft_av1::SequenceHeader::parse(payload).ok()?.color;
Some(ColorCodes { primaries: c.color_primaries, transfer: c.transfer_characteristics, matrix: c.matrix_coefficients, full_range: Some(c.color_range) })
}
/// Colour from the code points a file signals, most authoritative first (the container's colour
/// box, then the bitstream's): the matrix, transfer, primaries and range each come from the first
/// source that signals a value we know. What no source signals keeps the defaults: the matrix by
/// frame size ([`filmcraft_frame::default_matrix`]: BT.601 for SD, BT.709 otherwise), BT.709
/// transfer and primaries, limited range.
pub fn resolve(width: u32, height: u32, sources: &[ColorCodes]) -> ColorInfo {
let mut c = ColorInfo { matrix: filmcraft_frame::default_matrix(width, height), ..ColorInfo::REC709 };
if let Some(m) = sources.iter().find_map(|s| Matrix::from_code(s.matrix)) {
c.matrix = m;
}
if let Some(t) = sources.iter().find_map(|s| Transfer::from_code(s.transfer)) {
c.transfer = t;
}
if let Some(p) = sources.iter().find_map(|s| primaries_from_code(s.primaries)) {
c.primaries = p;
}
if sources.iter().find_map(|s| s.full_range) == Some(true) {
c.range = Range::Full;
}
c
}
+340
View File
@@ -0,0 +1,340 @@
//! Colour from the bitstream when the container has none (`crate::stream_color`): ffmpeg writes MP4
//! without a `colr` box by default, and an HEVC PQ file imported as Rec. 709 SDR, a full-range
//! H.264 file as limited range, and a BT.601 SD file with BT.709 primaries. The parameter sets
//! are built here bit by bit; no media files.
use std::io::Cursor;
use std::sync::Arc;
use filmcraft_bitstream::{BitWriter, escape_rbsp};
use filmcraft_color::{ColorInfo, ColorSpace, Matrix, Primaries, Range, Transfer};
use filmcraft_isobmff::{Av1Config, AvcConfig, Brand, CodecConfig, HevcConfig, HevcNalArray, Mp4Writer, SampleEntry, TrackConfig, WriteSample, WriterOptions};
use filmcraft_matroska::{MkvWriter, MuxOptions, TrackKind, TrackSpec};
use filmcraft_media::MediaSource;
use crate::stream_color::{ColorCodes, from_codec_config, resolve};
/// The video signal description of a VUI: (colour_primaries, transfer_characteristics,
/// matrix_coefficients) when `colour_description_present_flag` is set, and the range flag.
#[derive(Clone, Copy)]
struct Signal {
codes: Option<(u8, u8, u8)>,
full_range: bool,
}
const PQ_2020: Signal = Signal { codes: Some((9, 16, 9)), full_range: false };
const SMPTE_170M: Signal = Signal { codes: Some((6, 6, 6)), full_range: false };
const FULL_ONLY: Signal = Signal { codes: None, full_range: true };
fn signal(w: &mut BitWriter, s: Signal) {
w.write_bit(true); // video_signal_type_present_flag
w.write_bits(5, 3); // video_format: unspecified
w.write_bit(s.full_range);
w.write_bit(s.codes.is_some()); // colour_description_present_flag
if let Some((p, t, m)) = s.codes {
w.write_bits(p.into(), 8);
w.write_bits(t.into(), 8);
w.write_bits(m.into(), 8);
}
}
/// A Baseline H.264 SPS NAL unit (ITU-T H.264 §7.3.2.1.1, VUI per Annex E.1.1) of
/// `mbs_w`×`mbs_h` macroblocks, with a VUI carrying `vui` (no VUI when `None`).
fn h264_sps(mbs_w: u32, mbs_h: u32, vui: Option<Signal>) -> Vec<u8> {
let mut w = BitWriter::new();
w.write_bits(66, 8); // profile_idc: Baseline
w.write_bits(0, 8); // constraint flags
w.write_bits(30, 8); // level_idc
w.write_ue(0); // seq_parameter_set_id
w.write_ue(0); // log2_max_frame_num_minus4
w.write_ue(2); // pic_order_cnt_type
w.write_ue(1); // max_num_ref_frames
w.write_bit(false); // gaps_in_frame_num_value_allowed_flag
w.write_ue(mbs_w - 1);
w.write_ue(mbs_h - 1);
w.write_bits(0b110, 3); // frame_mbs_only, direct_8x8_inference, no cropping
w.write_bit(vui.is_some()); // vui_parameters_present_flag
if let Some(s) = vui {
w.write_bit(false); // aspect_ratio_info_present_flag
w.write_bit(false); // overscan_info_present_flag
signal(&mut w, s);
w.write_bit(false); // chroma_loc_info_present_flag
w.write_bit(false); // timing_info_present_flag
w.write_bit(false); // nal_hrd_parameters_present_flag
w.write_bit(false); // vcl_hrd_parameters_present_flag
w.write_bit(false); // pic_struct_present_flag
w.write_bit(false); // bitstream_restriction_flag
}
w.rbsp_trailing();
let mut nal = vec![0x67];
nal.extend(escape_rbsp(&w.finish()));
nal
}
/// A Main 10 HEVC SPS NAL unit (ITU-T H.265 §7.3.2.2, VUI per Annex E.2.1) of `width`×`height`
/// (multiples of 8), with a VUI carrying `vui` (no VUI when `None`).
fn hevc_sps(width: u32, height: u32, vui: Option<Signal>) -> Vec<u8> {
let mut w = BitWriter::new();
w.write_bits(0, 4); // sps_video_parameter_set_id
w.write_bits(0, 3); // sps_max_sub_layers_minus1
w.write_bit(true); // sps_temporal_id_nesting_flag
// profile_tier_level: Main 10, level 4.1
w.write_bits(0, 2);
w.write_bit(false);
w.write_bits(2, 5);
w.write_bits(0x2000_0000, 32);
w.write_bits(0b1001, 4);
w.write_bits(0, 32);
w.write_bits(0, 12);
w.write_bits(123, 8);
w.write_ue(0); // sps_seq_parameter_set_id
w.write_ue(1); // chroma_format_idc 4:2:0
w.write_ue(width);
w.write_ue(height);
w.write_bit(false); // conformance_window_flag
w.write_ue(2); // bit_depth_luma_minus8
w.write_ue(2); // bit_depth_chroma_minus8
w.write_ue(4); // log2_max_pic_order_cnt_lsb_minus4
w.write_bit(true); // sps_sub_layer_ordering_info_present_flag
w.write_ue(4); // sps_max_dec_pic_buffering_minus1
w.write_ue(0); // sps_max_num_reorder_pics
w.write_ue(0); // sps_max_latency_increase_plus1
w.write_ue(0); // log2_min_luma_coding_block_size_minus3 (8)
w.write_ue(1); // log2_diff_max_min_luma_coding_block_size (CTB 16)
w.write_ue(0); // log2_min_luma_transform_block_size_minus2 (4)
w.write_ue(2); // log2_diff_max_min_luma_transform_block_size (16)
w.write_ue(0); // max_transform_hierarchy_depth_inter
w.write_ue(0); // max_transform_hierarchy_depth_intra
w.write_bit(false); // scaling_list_enabled_flag
w.write_bit(false); // amp_enabled_flag
w.write_bit(false); // sample_adaptive_offset_enabled_flag
w.write_bit(false); // pcm_enabled_flag
w.write_ue(0); // num_short_term_ref_pic_sets
w.write_bit(false); // long_term_ref_pics_present_flag
w.write_bit(false); // sps_temporal_mvp_enabled_flag
w.write_bit(false); // strong_intra_smoothing_enabled_flag
w.write_bit(vui.is_some()); // vui_parameters_present_flag
if let Some(s) = vui {
w.write_bit(false); // aspect_ratio_info_present_flag
w.write_bit(false); // overscan_info_present_flag
signal(&mut w, s);
w.write_bit(false); // chroma_loc_info_present_flag
w.write_bit(false); // neutral_chroma_indication_flag
w.write_bit(false); // field_seq_flag
w.write_bit(false); // frame_field_info_present_flag
w.write_bit(false); // default_display_window_flag
w.write_bit(false); // vui_timing_info_present_flag
w.write_bit(false); // bitstream_restriction_flag
}
w.write_bit(false); // sps_extension_present_flag
w.rbsp_trailing();
let mut nal = vec![33 << 1, 1];
nal.extend(escape_rbsp(&w.finish()));
nal
}
/// An AV1 sequence header OBU (AV1 bitstream spec §5.5; reduced still-picture header, profile 0,
/// 10-bit 4:2:0) of `width`×`height` with `color_config` signalling `codes` and `full_range`.
fn av1_sequence_header(width: u32, height: u32, codes: Option<(u8, u8, u8)>, full_range: bool) -> Vec<u8> {
let mut w = BitWriter::new();
w.write_bits(0, 3); // seq_profile
w.write_bit(true); // still_picture
w.write_bit(true); // reduced_still_picture_header
w.write_bits(0, 5); // seq_level_idx[0]
w.write_bits(15, 4); // frame_width_bits_minus_1
w.write_bits(15, 4); // frame_height_bits_minus_1
w.write_bits(width - 1, 16);
w.write_bits(height - 1, 16);
w.write_bits(0, 3); // use_128x128_superblock, enable_filter_intra, enable_intra_edge_filter
w.write_bits(0, 3); // enable_superres, enable_cdef, enable_restoration
w.write_bit(true); // high_bitdepth
w.write_bit(false); // mono_chrome
w.write_bit(codes.is_some()); // color_description_present_flag
if let Some((p, t, m)) = codes {
w.write_bits(p.into(), 8);
w.write_bits(t.into(), 8);
w.write_bits(m.into(), 8);
}
w.write_bit(full_range); // color_range
w.write_bits(0, 2); // chroma_sample_position
w.write_bit(false); // separate_uv_delta_q
w.write_bit(false); // film_grain_params_present
w.rbsp_trailing();
let payload = w.finish();
// OBU header: type 1 (OBU_SEQUENCE_HEADER), obu_has_size_field; one-byte leb128 size
let mut obu = vec![(1 << 3) | 2, u8::try_from(payload.len()).expect("short payload")];
obu.extend(payload);
obu
}
fn avc(sps: Vec<u8>) -> CodecConfig {
CodecConfig::Avc(AvcConfig::new(vec![sps], vec![vec![0x68, 0xCE, 0x38, 0x80]], 4))
}
fn hevc(sps: Vec<u8>) -> CodecConfig {
CodecConfig::Hevc(HevcConfig {
general_profile_idc: 2,
chroma_format_idc: 1,
bit_depth_luma: 10,
bit_depth_chroma: 10,
length_size: 4,
arrays: vec![HevcNalArray { completeness: true, nal_type: 33, nalus: vec![sps] }],
..Default::default()
})
}
fn av1(obus: Vec<u8>) -> CodecConfig {
CodecConfig::Av1(Av1Config { high_bitdepth: true, chroma_subsampling_x: true, chroma_subsampling_y: true, config_obus: obus, ..Default::default() })
}
/// A one-sample MP4 with `codec` (the sample is never decoded) and, when given, a `colr` box.
fn mp4(codec: CodecConfig, w: u16, h: u16, colr: Option<filmcraft_isobmff::ColorInfo>) -> Arc<[u8]> {
let mut entry = match codec {
CodecConfig::Avc(c) => SampleEntry::avc(c, w, h),
CodecConfig::Hevc(c) => SampleEntry::hevc(c, w, h),
other => SampleEntry::video(filmcraft_isobmff::FourCc(*b"av01"), other, w, h),
};
if let Some(v) = entry.video.as_mut() {
v.color = colr;
}
let mut mux = Mp4Writer::new(Cursor::new(Vec::new()), WriterOptions::new(Brand::Mp4)).expect("writer");
let t = mux.add_track(TrackConfig::new(entry, 30)).expect("track");
mux.write_sample(t, WriteSample { data: &[0, 0, 0, 1, 0x65], duration: 1, composition_offset: 0, is_sync: true }).expect("sample");
mux.finish().expect("finish").into_inner().into()
}
fn mp4_color(codec: CodecConfig, w: u16, h: u16, colr: Option<filmcraft_isobmff::ColorInfo>) -> ColorInfo {
let src = crate::Mp4Source::open("clip.mp4", mp4(codec, w, h, colr)).expect("open");
src.info().video.as_ref().expect("video").color
}
#[test]
fn hevc_pq_without_colr_is_rec2100_pq() {
let c = mp4_color(hevc(hevc_sps(1920, 1080, Some(PQ_2020))), 1920, 1080, None);
assert_eq!(c, ColorInfo { matrix: Matrix::Bt2020Ncl, transfer: Transfer::Pq, primaries: Primaries::Bt2020, range: Range::Limited });
assert_eq!(ColorSpace::from_info(&c), ColorSpace::Rec2100Pq);
assert!(ColorSpace::from_info(&c).is_hdr());
}
#[test]
fn full_range_h264_without_colr_is_full_range() {
// video_full_range_flag with no colour description: full range, the rest by size
let c = mp4_color(avc(h264_sps(80, 45, Some(FULL_ONLY))), 1280, 720, None);
assert_eq!(c, ColorInfo { range: Range::Full, ..ColorInfo::REC709 });
}
#[test]
fn bt601_sd_h264_without_colr_has_bt601_primaries() {
let c = mp4_color(avc(h264_sps(45, 30, Some(SMPTE_170M))), 720, 480, None);
assert_eq!(c, ColorInfo { matrix: Matrix::Bt601, transfer: Transfer::Bt709, primaries: Primaries::Bt601_525, range: Range::Limited });
// PAL (BT.470 BG primaries, BT.601 matrix)
let pal = Signal { codes: Some((5, 6, 5)), full_range: false };
let c = mp4_color(avc(h264_sps(45, 36, Some(pal))), 720, 576, None);
assert_eq!((c.primaries, c.matrix), (Primaries::Bt601_625, Matrix::Bt601));
}
#[test]
fn unspecified_or_missing_vui_keeps_the_size_defaults() {
let unspecified = Signal { codes: Some((2, 2, 2)), full_range: false };
for vui in [None, Some(unspecified)] {
assert_eq!(mp4_color(avc(h264_sps(80, 45, vui)), 1280, 720, None), ColorInfo::REC709);
assert_eq!(mp4_color(hevc(hevc_sps(1920, 1080, vui)), 1920, 1080, None), ColorInfo::REC709);
// SD: the BT.601 matrix, as before
assert_eq!(mp4_color(avc(h264_sps(45, 30, vui)), 720, 480, None), ColorInfo { matrix: Matrix::Bt601, ..ColorInfo::REC709 });
}
}
#[test]
fn colr_wins_over_the_bitstream() {
use filmcraft_isobmff::ColorInfo::{Nclc, Nclx};
let sps = || hevc(hevc_sps(1920, 1080, Some(PQ_2020)));
// a BT.709 nclx box beats a PQ VUI, and is what the decoder's frames are tagged with
let colr = Nclx { primaries: 1, transfer: 1, matrix: 1, full_range: false };
assert_eq!(mp4_color(sps(), 1920, 1080, Some(colr.clone())), ColorInfo::REC709);
// HLG in colr, PQ in the VUI: HLG
let hlg = Nclx { primaries: 9, transfer: 18, matrix: 9, full_range: false };
assert_eq!(mp4_color(sps(), 1920, 1080, Some(hlg)).transfer, Transfer::Hlg);
// limited range in colr beats a full-range VUI
assert_eq!(mp4_color(avc(h264_sps(80, 45, Some(FULL_ONLY))), 1280, 720, Some(colr)).range, Range::Limited);
// what colr leaves unspecified comes from the VUI
let partial = Nclx { primaries: 2, transfer: 16, matrix: 2, full_range: false };
assert_eq!(mp4_color(sps(), 1920, 1080, Some(partial)), mp4_color(sps(), 1920, 1080, None));
// QuickTime nclc has no range flag: the VUI's range stands
let nclc = Nclc { primaries: 1, transfer: 1, matrix: 1 };
assert_eq!(mp4_color(avc(h264_sps(80, 45, Some(FULL_ONLY))), 1280, 720, Some(nclc)).range, Range::Full);
// an ICC profile has no code points: the bitstream decides
let icc = filmcraft_isobmff::ColorInfo::Icc { kind: filmcraft_isobmff::FourCc(*b"prof"), profile: vec![0; 8] };
assert_eq!(mp4_color(sps(), 1920, 1080, Some(icc)).transfer, Transfer::Pq);
}
#[test]
fn av1_sequence_header_colour() {
let obu = av1_sequence_header(1920, 1080, Some((9, 16, 9)), false);
assert_eq!(from_codec_config(&av1(obu.clone())), Some(ColorCodes { primaries: 9, transfer: 16, matrix: 9, full_range: Some(false) }));
let c = mp4_color(av1(obu), 1920, 1080, None);
assert_eq!(ColorSpace::from_info(&c), ColorSpace::Rec2100Pq);
// no colour description: unspecified code points, the range flag still counts
let obu = av1_sequence_header(640, 360, None, true);
assert_eq!(from_codec_config(&av1(obu)), Some(ColorCodes { primaries: 2, transfer: 2, matrix: 2, full_range: Some(true) }));
}
#[test]
fn matroska_without_colour_uses_the_bitstream() {
let CodecConfig::Hevc(hvcc) = hevc(hevc_sps(1920, 1080, Some(PQ_2020))) else { panic!("an hvcC") };
let mut spec = TrackSpec::new(TrackKind::Video, "V_MPEGH/ISO/HEVC");
spec.codec_private = hvcc.to_bytes();
spec.video_size = Some((1920, 1080));
spec.default_duration_ns = Some(33_333_333);
let mut w = MkvWriter::new(Cursor::new(Vec::new()), vec![spec], MuxOptions::default()).expect("writer");
w.write_frame(0, 0, true, &[0, 0, 0, 1, 0x26], None).expect("frame");
let b: Arc<[u8]> = w.finish().expect("finish").into_inner().into();
let src = crate::MkvSource::open("clip.mkv", b).expect("open");
let c = src.info().video.as_ref().expect("video").color;
assert_eq!(ColorSpace::from_info(&c), ColorSpace::Rec2100Pq);
assert_eq!((c.primaries, c.matrix), (Primaries::Bt2020, Matrix::Bt2020Ncl));
}
#[test]
fn resolve_takes_each_field_from_the_first_source_that_knows_it() {
let colr = ColorCodes { primaries: 1, transfer: 2, matrix: 0, full_range: None };
let vui = ColorCodes { primaries: 9, transfer: 18, matrix: 9, full_range: Some(true) };
// matrix 0 (identity) is not one we know: the VUI's BT.2020 is used
let c = resolve(1920, 1080, &[colr, vui]);
assert_eq!(c, ColorInfo { matrix: Matrix::Bt2020Ncl, transfer: Transfer::Hlg, primaries: Primaries::Bt709, range: Range::Full });
assert_eq!(resolve(720, 480, &[]), ColorInfo { matrix: Matrix::Bt601, ..ColorInfo::REC709 });
// code points that don't fit a byte are unspecified, not truncated (0x109 is not 9)
assert_eq!(ColorCodes::from_wide(0x109, 0x110, 0x109, None), ColorCodes { primaries: 2, transfer: 2, matrix: 2, full_range: None });
}
/// Every truncation and many bit flips of each configuration: never a panic, and the result is
/// either some colour or none (the size defaults).
#[test]
fn hostile_parameter_sets_never_panic() {
let configs: Vec<(Vec<u8>, fn(Vec<u8>) -> CodecConfig)> = vec![
(h264_sps(45, 30, Some(SMPTE_170M)), avc),
(hevc_sps(1920, 1080, Some(PQ_2020)), hevc),
(av1_sequence_header(1920, 1080, Some((9, 16, 9)), false), av1),
];
for (good, make) in configs {
assert!(from_codec_config(&make(good.clone())).is_some());
let mut variants: Vec<Vec<u8>> = (0..good.len()).map(|n| good[..n].to_vec()).collect();
for bit in 0..good.len() * 8 {
let mut b = good.clone();
b[bit / 8] ^= 0x80 >> (bit % 8);
variants.push(b);
}
variants.push(vec![0xff; 64]);
variants.push(Vec::new());
for v in variants {
let r = std::panic::catch_unwind(|| from_codec_config(&make(v.clone())));
assert!(r.is_ok(), "panicked on {v:02x?}");
}
}
// configurations with no parameter sets at all
assert_eq!(from_codec_config(&CodecConfig::Avc(AvcConfig::default())), None);
assert_eq!(from_codec_config(&CodecConfig::Hevc(HevcConfig::default())), None);
assert_eq!(from_codec_config(&CodecConfig::Av1(Av1Config::default())), None);
// a garbage SPS imports with the size defaults
assert_eq!(mp4_color(avc(vec![0x67, 0xff, 0xff]), 1280, 720, None), ColorInfo::REC709);
assert_eq!(mp4_color(hevc(vec![33 << 1, 1, 0xff]), 1920, 1080, None), ColorInfo::REC709);
}
+93 -2
View File
@@ -70,7 +70,7 @@ fn mjpeg_mov_with_pcm() {
fn aac_mp4_decodes() {
let Some(b) = fixture("tone_aac.mp4", &["-f", "lavfi", "-i", "sine=frequency=440:sample_rate=48000:d=3", "-c:a", "aac", "-b:a", "128k"]) else { return };
let src = crate::open_bytes("tone_aac.mp4", b).unwrap();
assert!(src.info().audio.as_ref().unwrap().codec.contains("AAC"));
assert!(src.info().audio().unwrap().codec.contains("AAC"));
// read in the middle (random access) and sequentially
let a = src.audio(48_000, 4800, 48_000).unwrap();
let p = a.peaks()[0];
@@ -387,6 +387,68 @@ fn mkv_vp9_422_seeks_bit_exact() {
check_vp9_seeks("noise_vp9_422.mkv", "Matroska", "yuv422p", &["-profile:v", "1", "-deadline", "realtime", "-speed", "8"], 30);
}
/// #402: a WebM whose VP9 track carries an alpha layer (`AlphaMode` 1, a second VP9 stream in
/// each block's `BlockAdditional`) decodes with that alpha. Frames requested out of order (seeks
/// across key frames, then forward) match libvpx's decode exactly, colour and alpha.
#[test]
fn vp9_webm_alpha_layer_matches_libvpx() {
let (w, h, frames) = (176usize, 144usize, 30usize);
// the alpha changes every frame, so a picture paired with the wrong alpha frame shows
let Some(path) = fixture_path(
"alpha_vp9.webm",
&[
"-f",
"lavfi",
"-i",
"testsrc2=s=176x144:r=25,format=yuva420p,geq=lum='lum(X,Y)':cb='cb(X,Y)':cr='cr(X,Y)':a='mod(X*2+N*7\\,256)'",
"-frames:v",
"30",
"-c:v",
"libvpx-vp9",
"-pix_fmt",
"yuva420p",
"-g",
"10",
"-deadline",
"realtime",
"-speed",
"8",
"-b:v",
"600k",
],
) else {
return;
};
// the reference: ffmpeg's libvpx decoder (its native VP9 decoder ignores the alpha layer)
let src_path = path.to_string_lossy().into_owned();
let Some(reference) =
fixture("alpha_vp9.webm.yuva420p.yuv", &["-c:v", "libvpx-vp9", "-i", &src_path, "-fps_mode", "passthrough", "-f", "rawvideo", "-pix_fmt", "yuva420p"])
else {
return;
};
let (luma, chroma) = (w * h, (w / 2) * (h / 2));
let frame_len = luma * 2 + chroma * 2;
if reference.len() != frame_len * frames {
eprintln!("libvpx reference has {} bytes, not {}; skipping", reference.len(), frame_len * frames);
return;
}
let bytes: Arc<[u8]> = std::fs::read(&path).unwrap().into();
let src = crate::open_bytes("alpha_vp9.webm", bytes).unwrap();
let info = src.info().clone();
assert_eq!(info.container, "WebM");
assert!(info.video.as_ref().unwrap().has_alpha);
let rate = info.frame_rate();
let order: Vec<usize> = [29, 3, 17, 12, 0, 25, 21, 9].into_iter().chain(10..frames).collect();
for k in order {
let f = src.video_frame(FrameRequest::full(rate.tick_of(k as i64))).unwrap();
let r = &reference[k * frame_len..(k + 1) * frame_len];
let filmcraft_frame::PixelData::Yuv8 { planes, alpha, .. } = &f.data else { panic!("frame {k}: not 8-bit YUV: {}", f.format_label()) };
assert_eq!(planes[0].as_slice(), &r[..luma], "frame {k}: luma");
let alpha = alpha.as_ref().unwrap_or_else(|| panic!("frame {k}: no alpha plane"));
assert_eq!(alpha.as_slice(), &r[luma + 2 * chroma..], "frame {k}: alpha");
}
}
/// Every sample flagged as sync (as in an MP4 without `stss`): the GOP cache must still start
/// decoding at a real VP9 key frame.
#[test]
@@ -510,7 +572,7 @@ fn check_opus(name: &str, channels: usize, expr: &str, extra: &[&str], container
let src = crate::open_bytes(name, b).expect("open");
let info = src.info().clone();
assert_eq!(info.container, container);
let a = info.audio.as_ref().expect("audio");
let a = info.audio().expect("audio");
assert_eq!((a.codec.as_str(), a.sample_rate, a.channels as usize), ("Opus", 48_000, channels));
let d = info.duration.seconds();
assert!((d - 3.0).abs() < 0.03, "{name}: duration {d}");
@@ -615,3 +677,32 @@ fn apv_mp4_and_raw_bitstream_decode() {
let decoded_mp4 = mp4_src.video_frame(FrameRequest::full(Tick::ZERO)).expect("decode mp4 frame");
assert_eq!(decoded_mp4.to_rgba8(), decoded_raw.to_rgba8());
}
/// ffmpeg writes MP4 without a `colr` box unless asked: the colour comes from the x265 / x264
/// SPS VUI, and matches the same stream remuxed with `colr`.
#[test]
fn mp4_colour_comes_from_the_sps_without_colr() {
use filmcraft_color::{ColorSpace, Primaries, Range};
let src = |name: &str, args: &[&str]| fixture(name, args).map(|b| crate::open_bytes(name, b).expect("open"));
let color = |s: &filmcraft_media::SharedSource| s.info().video.as_ref().expect("video").color;
let lavfi = |size: &'static str| ["-f", "lavfi", "-i", size, "-t", "0.2"];
let x265 = "colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:log-level=error";
let mut pq = lavfi("testsrc2=s=640x360:r=25").to_vec();
pq.extend(["-c:v", "libx265", "-tag:v", "hvc1", "-pix_fmt", "yuv420p10le", "-x265-params", x265]);
let Some(plain) = src("hevc_pq_no_colr.mp4", &pq) else { return };
assert_eq!(ColorSpace::from_info(&color(&plain)), ColorSpace::Rec2100Pq);
let colr_path = fixture_path("hevc_pq_no_colr.mp4", &pq).expect("fixture");
if let Some(with_colr) = src("hevc_pq_colr.mp4", &["-i", colr_path.to_str().expect("path"), "-c", "copy", "-movflags", "+write_colr"]) {
assert_eq!(color(&with_colr), color(&plain));
}
let mut full = lavfi("testsrc2=s=640x360:r=25").to_vec();
full.extend(["-c:v", "libx264", "-pix_fmt", "yuvj420p", "-color_range", "pc"]);
if let Some(s) = src("h264_full_no_colr.mp4", &full) {
assert_eq!(color(&s).range, Range::Full);
}
let mut sd = lavfi("testsrc2=s=720x480:r=30000/1001").to_vec();
sd.extend(["-c:v", "libx264", "-pix_fmt", "yuv420p", "-x264-params", "colorprim=smpte170m:transfer=smpte170m:colormatrix=smpte170m"]);
if let Some(s) = src("h264_sd601_no_colr.mp4", &sd) {
assert_eq!(color(&s).primaries, Primaries::Bt601_525);
}
}
+56 -9
View File
@@ -45,8 +45,19 @@ pub trait VideoDecoder: Send {
/// shortcuts that only change pictures nothing references, which come out flagged
/// [`DecodedFrame::draft`]. Decoders without such a mode ignore it.
fn set_draft(&mut self, _on: bool) {}
/// Made with fewer worker threads than a decoder for playback gets, because it was made for
/// background work ([`filmcraft_media::cancel::with_background`]). A request for a monitor
/// replaces such a decoder.
fn thread_limited(&self) -> bool {
false
}
}
/// Worker threads of the H.264 decoders made for background work (thumbnails): a frame-threaded
/// 4K decoder holds `threads + 2` pictures in flight (~0.5 GB with 16 threads), and the frame
/// workers already decode different files in parallel.
const BACKGROUND_H264_THREADS: usize = 2;
/// NAL unit headers of a length-prefixed (avcC / hvcC) sample: the first two bytes of each unit.
fn nal_headers(sample: &[u8], length_size: usize) -> impl Iterator<Item = (u8, u8)> + '_ {
let mut pos = 0usize;
@@ -200,19 +211,36 @@ pub struct H264Decoder {
dec: filmcraft_h264::Decoder,
length_size: usize,
draft: bool,
/// Worker threads (a background decoder gets fewer).
threads: usize,
}
impl H264Decoder {
/// The worker threads for a decoder made now.
fn threads_now() -> usize {
if filmcraft_media::cancel::background() { BACKGROUND_H264_THREADS.min(h264_threads()) } else { h264_threads() }
}
fn make_inner(avcc: &[u8], threads: usize) -> Result<filmcraft_h264::Decoder> {
let mut dec = filmcraft_h264::Decoder::with_threads(threads);
if !avcc.is_empty() {
dec.configure_avcc(avcc).map_err(h264_error)?;
}
Ok(dec)
}
pub fn new(avcc: Vec<u8>) -> Result<Self> {
recycle_h264_planes();
let dec = filmcraft_h264::Decoder::from_avcc(&avcc).map_err(|e| CodecError::Decode(e.to_string()))?;
let threads = Self::threads_now();
let dec = Self::make_inner(&avcc, threads)?;
let length_size = avcc_length_size(&avcc);
Ok(Self { avcc, dec, length_size, draft: false })
Ok(Self { avcc, dec, length_size, draft: false, threads })
}
/// A decoder for Annex B byte-stream samples (start codes, in-band parameter sets: MXF, TS).
pub fn annexb() -> Self {
recycle_h264_planes();
Self { avcc: Vec::new(), dec: filmcraft_h264::Decoder::new(), length_size: 0, draft: false }
let threads = Self::threads_now();
Self { avcc: Vec::new(), dec: filmcraft_h264::Decoder::with_threads(threads), length_size: 0, draft: false, threads }
}
fn convert(p: filmcraft_h264::Picture) -> DecodedFrame {
use std::sync::Arc;
@@ -236,18 +264,26 @@ impl H264Decoder {
}
}
/// An H.264 decoder error as a [`CodecError`]: a feature the decoder does not implement (High
/// 4:2:2, field pictures...) is [`CodecError::Unsupported`], so the media pool shows the
/// unreadable slate instead of a black picture (#626); anything else is a decode error.
fn h264_error(e: filmcraft_h264::Error) -> CodecError {
match e {
filmcraft_h264::Error::Unsupported(what) => CodecError::Unsupported(format!("H.264: {what}")),
e => CodecError::Decode(e.to_string()),
}
}
impl VideoDecoder for H264Decoder {
fn decode(&mut self, sample: &[u8], pts: i64) -> Result<Vec<DecodedFrame>> {
let pics = self.dec.decode(sample, pts).map_err(|e| CodecError::Decode(e.to_string()))?;
let pics = self.dec.decode(sample, pts).map_err(h264_error)?;
Ok(pics.into_iter().map(Self::convert).collect())
}
fn flush(&mut self) -> Vec<DecodedFrame> {
self.dec.flush().into_iter().map(Self::convert).collect()
}
fn reset(&mut self) {
if self.avcc.is_empty() {
self.dec = filmcraft_h264::Decoder::new();
} else if let Ok(d) = filmcraft_h264::Decoder::from_avcc(&self.avcc) {
if let Ok(d) = Self::make_inner(&self.avcc, self.threads) {
self.dec = d;
}
self.dec.set_draft(self.draft);
@@ -255,6 +291,9 @@ impl VideoDecoder for H264Decoder {
fn name(&self) -> &str {
"FilmCraft H.264"
}
fn thread_limited(&self) -> bool {
self.threads < h264_threads()
}
fn set_draft(&mut self, on: bool) {
self.draft = on;
self.dec.set_draft(on);
@@ -299,7 +338,7 @@ pub struct HevcDecoder {
impl HevcDecoder {
pub fn new(hvcc: Vec<u8>) -> Result<Self> {
let dec = filmcraft_hevc::Decoder::from_hvcc(&hvcc).map_err(|e| CodecError::Decode(e.to_string()))?;
let dec = filmcraft_hevc::Decoder::from_hvcc(&hvcc).map_err(hevc_error)?;
let (length_size, highest_tid) = hvcc_length_size_and_tid(&hvcc);
Ok(Self { hvcc, dec, length_size, highest_tid })
}
@@ -339,9 +378,17 @@ impl HevcDecoder {
}
}
/// An HEVC decoder error as a [`CodecError`] (see [`h264_error`]).
fn hevc_error(e: filmcraft_hevc::Error) -> CodecError {
match e {
filmcraft_hevc::Error::Unsupported(what) => CodecError::Unsupported(format!("HEVC: {what}")),
e => CodecError::Decode(e.to_string()),
}
}
impl VideoDecoder for HevcDecoder {
fn decode(&mut self, sample: &[u8], pts: i64) -> Result<Vec<DecodedFrame>> {
let pics = self.dec.decode(sample, pts).map_err(|e| CodecError::Decode(e.to_string()))?;
let pics = self.dec.decode(sample, pts).map_err(hevc_error)?;
Ok(pics.into_iter().map(Self::convert).collect())
}
fn flush(&mut self) -> Vec<DecodedFrame> {
+290
View File
@@ -0,0 +1,290 @@
//! WebM alpha (#402): a VP9 track with `AlphaMode` 1 carries, in each block's `BlockAdditional`
//! with `BlockAddID` 1, a second VP9 stream coded independently of the colour one. Its luma plane,
//! full range, is the frame's alpha (the WebM convention, as libvpx-based decoders read it).
//!
//! [`AlphaDecoder`] wraps the colour decoder: every sample it decodes, it also decodes that
//! sample's alpha frame with a second (software) VP9 decoder and attaches the luma plane to the
//! colour picture with the same timestamp, holding a colour picture back until its alpha is out. The alpha stream has key frames of its own, which need
//! not coincide with the colour ones: after a seek (or any other break in the order samples are
//! fed), the alpha decoder restarts from the nearest alpha key frame at or before the sample.
//!
//! Alpha is best effort: a missing, damaged or undecodable alpha frame leaves that picture opaque,
//! never fails the colour decode.
use std::collections::{HashMap, VecDeque};
use std::sync::Arc;
use filmcraft_frame::{PixelData, VideoFrame};
use crate::video::{DecodedFrame, VideoDecoder};
/// How far back the alpha decoder looks for a key frame when it resynchronises.
const MAX_RESYNC: usize = 600;
/// Decoded alpha pictures waiting for their colour picture, and colour pictures waiting for their
/// alpha (VP9 pictures come out in order, so a few are plenty; past that a picture goes out opaque
/// rather than stall).
const MAX_PENDING: usize = 8;
/// Where each sample's alpha frame is stored.
pub(crate) struct AlphaLayer {
src: crate::Src,
/// Per sample, in file (decode) order.
additions: Vec<Option<(u64, u32)>>,
pts: Vec<i64>,
/// First sample with each pts.
by_pts: HashMap<i64, usize>,
}
impl AlphaLayer {
/// `samples`: (pts, addition) of every sample of the track, in file order.
pub(crate) fn new(src: crate::Src, samples: impl Iterator<Item = (i64, Option<(u64, u32)>)>) -> Self {
let (mut additions, mut all_pts) = (Vec::new(), Vec::new());
let mut by_pts = HashMap::new();
for (i, (pts, add)) in samples.enumerate() {
additions.push(add);
all_pts.push(pts);
by_pts.entry(pts).or_insert(i);
}
Self { src, additions, pts: all_pts, by_pts }
}
fn read(&self, i: usize) -> Option<Vec<u8>> {
let (offset, size) = (*self.additions.get(i)?)?;
let end = offset.checked_add(size as u64)?;
if end > self.src.0.len() {
return None;
}
let mut data = vec![0; size as usize];
self.src.0.read_at(offset, &mut data).ok()?;
Some(data)
}
}
/// A colour decoder plus the decoder of its alpha layer.
///
/// Both decoders may hand pictures back later than the sample that made them (frame threading),
/// and not necessarily after the same call. A colour picture is therefore held until its alpha
/// picture has come out, or until it is certain that it won't: the alpha decoder has moved past
/// it (pictures come out in order), it failed, or the sample has no alpha frame.
pub(crate) struct AlphaDecoder {
color: Box<dyn VideoDecoder>,
alpha: Box<dyn VideoDecoder>,
layer: Arc<AlphaLayer>,
/// Sample the alpha decoder expects next; `None` after a reset or a failure.
next: Option<usize>,
/// Decoded alpha pictures not yet paired, by pts.
pending: VecDeque<(i64, VideoFrame)>,
/// Latest pts the alpha decoder has produced since the last reset.
alpha_max: Option<i64>,
/// Colour pictures waiting for their alpha, in output order.
held: VecDeque<DecodedFrame>,
}
impl AlphaDecoder {
pub(crate) fn new(color: Box<dyn VideoDecoder>, alpha: Box<dyn VideoDecoder>, layer: Arc<AlphaLayer>) -> Self {
Self { color, alpha, layer, next: None, pending: VecDeque::new(), alpha_max: None, held: VecDeque::new() }
}
fn keep(&mut self, frames: Vec<DecodedFrame>) {
for f in frames {
self.alpha_max = Some(self.alpha_max.map_or(f.pts, |m| m.max(f.pts)));
self.pending.push_back((f.pts, f.frame));
}
while self.pending.len() > MAX_PENDING {
self.pending.pop_front();
}
}
/// Decode the alpha frame of the sample with `pts`, first catching up from an alpha key frame
/// when the sample doesn't follow the previous one.
fn feed_alpha(&mut self, pts: i64) {
let Some(&i) = self.layer.by_pts.get(&pts) else {
self.next = None;
return;
};
if self.next != Some(i) {
self.alpha.reset();
self.pending.clear();
self.alpha_max = None;
let is_key = |k: usize| self.layer.read(k).is_some_and(|d| filmcraft_vp9::is_keyframe(&d));
let start = (i.saturating_sub(MAX_RESYNC)..=i).rev().find(|&k| is_key(k));
if let Some(start) = start {
for k in start..i {
let (Some(d), Some(&p)) = (self.layer.read(k), self.layer.pts.get(k)) else { continue };
match self.alpha.decode(&d, p) {
Ok(frames) => self.keep(frames),
Err(_) => break,
}
}
}
}
self.next = Some(i + 1);
let Some(d) = self.layer.read(i) else { return };
match self.alpha.decode(&d, pts) {
Ok(frames) => self.keep(frames),
Err(e) => {
log::debug!("WebM alpha frame at pts {pts}: {e}");
self.next = None;
}
}
}
/// Whether the alpha picture for `pts` can still come out of the alpha decoder.
fn alpha_may_come(&self, pts: i64) -> bool {
let has_frame = self.layer.by_pts.get(&pts).is_some_and(|&i| self.layer.additions.get(i).is_some_and(|a| a.is_some()));
has_frame && self.next.is_some() && self.alpha_max.is_none_or(|m| m < pts)
}
/// Release held colour pictures, in order, as far as their alpha is settled. `all`: release
/// every one (flush), with or without alpha.
fn release(&mut self, all: bool) -> Vec<DecodedFrame> {
let mut out = Vec::new();
while let Some(f) = self.held.front() {
let pts = f.pts;
let found = self.pending.iter().position(|(p, _)| *p == pts);
if found.is_none() && !all && self.held.len() <= MAX_PENDING && self.alpha_may_come(pts) {
break;
}
let Some(mut f) = self.held.pop_front() else { break };
if let Some(k) = found
&& let Some((_, a)) = self.pending.remove(k)
{
set_alpha(&mut f.frame, &a);
}
out.push(f);
}
out
}
}
impl VideoDecoder for AlphaDecoder {
fn decode(&mut self, sample: &[u8], pts: i64) -> crate::Result<Vec<DecodedFrame>> {
let out = self.color.decode(sample, pts)?;
self.held.extend(out);
self.feed_alpha(pts);
Ok(self.release(false))
}
fn flush(&mut self) -> Vec<DecodedFrame> {
let out = self.color.flush();
self.held.extend(out);
let rest = self.alpha.flush();
self.keep(rest);
self.release(true)
}
fn reset(&mut self) {
self.color.reset();
self.alpha.reset();
self.next = None;
self.pending.clear();
self.alpha_max = None;
self.held.clear();
}
fn name(&self) -> &str {
self.color.name()
}
fn intra_only(&self) -> bool {
self.color.intra_only()
}
fn is_random_access(&self, sample: &[u8]) -> Option<bool> {
self.color.is_random_access(sample)
}
fn is_disposable(&self, sample: &[u8]) -> bool {
self.color.is_disposable(sample)
}
fn set_draft(&mut self, on: bool) {
self.color.set_draft(on);
self.alpha.set_draft(on);
}
fn thread_limited(&self) -> bool {
self.color.thread_limited()
}
}
/// The luma plane of `a` (an alpha picture) as `frame`'s alpha, converted to `frame`'s bit depth.
/// Nothing happens when the sizes differ or either picture has no usable planes.
pub(crate) fn set_alpha(frame: &mut VideoFrame, a: &VideoFrame) {
if (frame.width, frame.height) != (a.width, a.height) {
return;
}
let n = frame.width as usize * frame.height as usize;
// alpha codes as 16-bit with their depth, so every pairing converts through one path
let (codes, abits): (Vec<u16>, u32) = match &a.data {
PixelData::Yuv8 { planes, .. } => (planes[0].iter().map(|&v| v as u16).collect(), 8),
PixelData::Yuv16 { planes, bits, .. } => (planes[0].to_vec(), (*bits).clamp(1, 16)),
_ => return,
};
if codes.len() != n {
return;
}
let rescale = |v: u16, to: u32| -> u32 {
let (from_max, to_max) = ((1u32 << abits) - 1, (1u32 << to.clamp(1, 16)) - 1);
(v as u32 * to_max + from_max / 2) / from_max
};
match &mut frame.data {
PixelData::Yuv8 { alpha, .. } => *alpha = Some(Arc::new(codes.iter().map(|&v| rescale(v, 8) as u8).collect())),
PixelData::Yuv16 { alpha, bits, .. } => {
let to = *bits;
*alpha = Some(Arc::new(codes.iter().map(|&v| rescale(v, to) as u16).collect()));
}
PixelData::Rgba8(rgba) => {
let px = Arc::make_mut(rgba);
for (p, &v) in px.as_chunks_mut::<4>().0.iter_mut().zip(&codes) {
p[3] = rescale(v, 8) as u8;
}
}
PixelData::RgbaF32(_) => {}
}
}
#[cfg(test)]
mod tests {
use super::*;
use filmcraft_frame::Chroma;
fn yuv8(w: u32, h: u32, luma: Vec<u8>) -> VideoFrame {
let n = (w * h) as usize;
let c = n / 4;
let data = PixelData::Yuv8 { planes: [Arc::new(luma), Arc::new(vec![128; c]), Arc::new(vec![128; c])], chroma: Chroma::C420, alpha: None };
VideoFrame { width: w, height: h, data, color: Default::default(), par: (1, 1), pts: filmcraft_time::Tick::ZERO }
}
fn yuv16(w: u32, h: u32, bits: u32) -> VideoFrame {
let n = (w * h) as usize;
let data =
PixelData::Yuv16 { planes: [Arc::new(vec![0; n]), Arc::new(vec![0; n / 4]), Arc::new(vec![0; n / 4])], chroma: Chroma::C420, bits, alpha: None };
VideoFrame { width: w, height: h, data, color: Default::default(), par: (1, 1), pts: filmcraft_time::Tick::ZERO }
}
#[test]
fn alpha_luma_becomes_the_alpha_plane_at_the_frame_depth() {
let a = yuv8(2, 2, vec![0, 128, 255, 64]);
// 8-bit picture: the luma as is
let mut f = yuv8(2, 2, vec![9; 4]);
set_alpha(&mut f, &a);
let PixelData::Yuv8 { alpha, .. } = &f.data else { panic!() };
assert_eq!(alpha.as_deref(), Some(&vec![0, 128, 255, 64]));
// 10-bit picture: full range kept (255 → 1023)
let mut f = yuv16(2, 2, 10);
set_alpha(&mut f, &a);
let PixelData::Yuv16 { alpha, .. } = &f.data else { panic!() };
assert_eq!(alpha.as_deref(), Some(&vec![0, 514, 1023, 257]));
// RGBA picture (VP9 RGB profiles): the A channel
let mut f = VideoFrame::rgba8(2, 2, vec![7; 16]);
set_alpha(&mut f, &a);
let PixelData::Rgba8(px) = &f.data else { panic!() };
assert_eq!(px.chunks(4).map(|p| p[3]).collect::<Vec<_>>(), [0, 128, 255, 64]);
}
#[test]
fn an_alpha_picture_of_another_size_is_ignored() {
let mut f = yuv8(4, 2, vec![9; 8]);
set_alpha(&mut f, &yuv8(2, 2, vec![0; 4]));
let PixelData::Yuv8 { alpha, .. } = &f.data else { panic!() };
assert!(alpha.is_none());
// a luma plane shorter than the picture (damaged) is ignored too
let mut f = yuv8(2, 2, vec![9; 4]);
set_alpha(&mut f, &yuv8(2, 2, vec![0; 3]));
let PixelData::Yuv8 { alpha, .. } = &f.data else { panic!() };
assert!(alpha.is_none());
}
}
+79
View File
@@ -0,0 +1,79 @@
//! E-AC-3 (Dolby Digital Plus) audio in MP4, Matroska and MPEG-TS against ffmpeg: ffmpeg encodes
//! and muxes the files and decodes them as the oracle. Zero-bit mantissas carry decoder-specific
//! dither (A/52 §7.3.4), so the criterion is SNR against ffmpeg's decode, sample-aligned.
mod common;
use common::*;
const STEREO: &str = "aevalsrc=exprs=0.3*sin(2*PI*441*t)+0.1*sin(2*PI*3100*t)|0.25*sin(2*PI*660*t)+0.1*sin(2*PI*(500+4000*t)*t):s=48000";
const SURROUND: &str =
"aevalsrc=exprs=0.3*sin(2*PI*300*t)|0.3*sin(2*PI*400*t)|0.3*sin(2*PI*500*t)|0.3*sin(2*PI*60*t)|0.2*sin(2*PI*700*t)|0.2*sin(2*PI*800*t):s=48000:c=5.1";
/// Worst channel SNR (dB) of `ours` (planar) against ffmpeg's interleaved `want`, with ours
/// shifted by `lag` samples, over the samples both cover past the first 1000.
fn snr(ours: &[Vec<f32>], want: &[f32], ch: usize, lag: i64) -> f64 {
let n = want.len() / ch;
(0..ch)
.map(|c| {
let (mut sig, mut err) = (0f64, 0f64);
for i in 1000..n - 1000 {
let Some(&a) = ours[c].get((i as i64 + lag) as usize) else { continue };
let (a, b) = (a as f64, want[i * ch + c] as f64);
sig += b * b;
err += (a - b) * (a - b);
}
10.0 * (sig / err.max(1e-30)).log10()
})
.fold(f64::INFINITY, f64::min)
}
#[test]
fn eac3_in_mp4_matroska_and_transport_streams() {
let ff = filmcraft_testkit::require_ffmpeg!();
let cases: [(&str, &str, u32, &[&str]); 4] = [
("eac3_51.mp4", SURROUND, 6, &["-b:a", "384k"]),
("eac3_51.mkv", SURROUND, 6, &["-b:a", "384k"]),
("eac3_stereo.mov", STEREO, 2, &["-b:a", "192k"]),
("eac3_stereo.ts", STEREO, 2, &["-b:a", "192k"]),
];
for (name, input, channels, enc) in cases {
let args: Vec<&str> = ["-f", "lavfi", "-i", input, "-t", "2", "-c:a", "eac3"].into_iter().chain(enc.iter().copied()).collect();
let f = fixture(&ff, name, &args).unwrap_or_else(|| panic!("could not write {name}"));
let src = filmcraft_codecs::open_bytes(name, bytes(&f)).unwrap();
let a = src.info().audio().cloned().unwrap();
assert_eq!((a.sample_rate, a.channels), (48_000, channels), "{name}: {}", a.codec);
let want = ffmpeg_audio_f32(&ff, &f, &[]);
let ch = channels as usize;
let n = want.len() / ch;
// MPEG-TS audio starts at its first PTS relative to the presentation start
let start = match name.ends_with(".ts") {
true => filmcraft_codecs::MpegSource::open(name, bytes(&f)).unwrap().audio_extent().map_or(0, |e| e.0),
false => 0,
};
let ours = src.audio(start, n, 48_000).unwrap().channels;
assert_eq!(ours.len(), ch, "{name}");
// Matroska stores CodecDelay (the encoder's 256-sample priming) in ns and timestamps in
// ms, and we place packets at the rounded value (240 samples): a shared Matroska timing
// limit (AAC and AC-3 alike), not a decoding one. Elsewhere the samples line up exactly.
let (worst, lag) = (-32..=32).map(|lag| (snr(&ours, &want, ch, lag), lag)).fold((f64::MIN, 0), |a, b| if b.0 > a.0 { b } else { a });
let max_lag = if name.ends_with(".mkv") { 16 } else { 0 };
assert!(lag.abs() <= max_lag, "{name}: decode is {lag} samples off ffmpeg's");
// a window read on a fresh source (random access) gives the same samples
let fresh = filmcraft_codecs::open_bytes(name, bytes(&f)).unwrap();
let at = n / 3;
let win = fresh.audio(start + at as i64, 4800, 48_000).unwrap().channels;
let mut seek_err = 0f32;
for c in 0..ch {
for (k, v) in win[c].iter().enumerate() {
seek_err = seek_err.max((v - ours[c][at + k]).abs());
}
}
println!(
"{name}: {} ({} Hz, {ch} ch), worst channel SNR vs ffmpeg {worst:.1} dB (lag {lag}), random access max diff {seek_err:.1e}",
a.codec, a.sample_rate
);
assert!(worst >= 50.0, "{name}: SNR {worst:.1} dB");
assert!(seek_err <= 1e-6, "{name}: random-access audio differs by {seek_err}");
}
}
@@ -0,0 +1,50 @@
//! Matroska audio made by ffmpeg, against ffmpeg's decode (#236). ffmpeg stamps blocks to the
//! millisecond, so at 44.1 kHz AAC (1024-sample), MP3 (1152) and AC-3 (1536) packets are each
//! stamped up to half a millisecond off their exact start. Read back block by block, as playback
//! reads it, a tone must stay as smooth as ffmpeg's decode of it: no holes or overlaps between
//! packets.
mod common;
use common::*;
/// Largest |second difference| of `x` from `from` on: a click shows up as a spike.
fn roughness(x: &[f32], from: usize) -> f32 {
x.windows(3).skip(from).map(|w| (w[0] - 2.0 * w[1] + w[2]).abs()).fold(0.0, f32::max)
}
#[test]
fn matroska_audio_plays_without_holes() {
let ff = filmcraft_testkit::require_ffmpeg!();
let cases: [(&str, u32, &[&str]); 5] = [
("tone_aac_48k.mkv", 48_000, &["-c:a", "aac", "-b:a", "160k"]),
("tone_aac_44k.mkv", 44_100, &["-c:a", "aac", "-b:a", "160k"]),
("tone_mp3_44k.mkv", 44_100, &["-c:a", "libmp3lame", "-b:a", "192k"]),
("tone_mp2_44k.mkv", 44_100, &["-c:a", "mp2", "-b:a", "192k"]),
("tone_ac3_44k.mkv", 44_100, &["-c:a", "ac3", "-b:a", "192k"]),
];
for (name, rate, codec) in cases {
let input = format!("sine=frequency=440:sample_rate={rate}:duration=3");
let args: Vec<&str> = ["-f", "lavfi", "-i", input.as_str(), "-af", "volume=4"].iter().copied().chain(codec.iter().copied()).collect();
let Some(path) = fixture(&ff, name, &args) else {
eprintln!("SKIPPED ({name}): this ffmpeg can't encode it");
continue;
};
let src = filmcraft_codecs::open_bytes(name, bytes(&path)).unwrap();
assert_eq!(src.info().audio().map(|a| a.sample_rate), Some(rate), "{name}");
let frames = rate as usize * 29 / 10;
let mut ours = Vec::with_capacity(frames);
while ours.len() < frames {
let n = 512.min(frames - ours.len());
ours.extend_from_slice(&src.audio(ours.len() as i64, n, rate).unwrap().channels[0]);
}
let theirs = ffmpeg_audio_f32(&ff, &path, &[]);
// past the codecs' start-up (priming, MP3's first frame)
let skip = rate as usize / 10;
let (r_ours, r_theirs) = (roughness(&ours, skip), roughness(&theirs[..theirs.len().min(frames)], skip));
let peak = ours.iter().skip(skip).fold(0f32, |m, x| m.max(x.abs()));
println!("{name}: peak {peak:.3}, max |second difference| ours {r_ours:.5}, ffmpeg {r_theirs:.5}");
assert!(peak > 0.4, "{name}: the tone is missing (peak {peak})");
assert!(r_ours < 2.0 * r_theirs + 0.005, "{name}: clicks between packets (|Δ²| {r_ours} vs ffmpeg's {r_theirs})");
}
}
+60 -3
View File
@@ -1,6 +1,7 @@
//! Image sequences and Broadcast WAV against ffmpeg: ffmpeg writes numbered stills (PNG, JPEG,
//! TIFF, BMP, WebP when available) and BWF files; the sequence frames must equal ffmpeg's decode of
//! the same image2 sequence (lossless formats exactly, JPEG within a few levels), and the BWF
//! Image sequences, stills and Broadcast WAV against ffmpeg: ffmpeg writes numbered stills (PNG,
//! JPEG, TIFF, BMP, WebP when available, Targa, DPX), single Targa / DPX stills in every layout it
//! has, and BWF files; our frames must equal ffmpeg's decode of the same files (lossless formats
//! exactly, JPEG within a few levels, high bit depths within one 8-bit step), and the BWF
//! TimeReference must give the start timecode.
mod common;
@@ -114,6 +115,62 @@ fn webp_sequence_lossless() {
check(&ff, "webp", &["-c:v", "libwebp", "-lossless", "1"], 0);
}
#[test]
fn targa_and_dpx_sequences_exact() {
let ff = filmcraft_testkit::require_ffmpeg!();
check(&ff, "tga", &[], 0);
check(&ff, "dpx", &["-pix_fmt", "gbrp10be"], 1);
}
/// One `ext` still in `pix_fmt` (95 × 63, so scan lines need padding; translucent when the format
/// has alpha) opened like an imported file must match ffmpeg's RGBA decode within `tol` (exactly
/// when `tol` is 0).
fn still_matches(ff: &Path, ext: &str, pix_fmt: &str, extra: &[&str], tol: u8) {
let name = format!("still_{pix_fmt}{}.{ext}", extra.join(""));
let alpha = if pix_fmt.contains('a') && pix_fmt != "pal8" { ",format=rgba,geq=r='r(X,Y)':g='g(X,Y)':b='b(X,Y)':a='X*2+Y'" } else { "" };
let vf = format!("scale=95:63{alpha}");
let mut args = vec!["-f", "lavfi", "-i", "testsrc2=size=96x64", "-frames:v", "1", "-vf", &vf, "-pix_fmt", pix_fmt];
args.extend_from_slice(extra);
let Some(f) = fixture(ff, &name, &args) else {
eprintln!("SKIPPED: ffmpeg cannot write {name}");
return;
};
let src = filmcraft_codecs::open_bytes(&name, bytes(&f)).unwrap_or_else(|e| panic!("{name}: {e}"));
assert_eq!(src.info().kind, MediaKind::Still, "{name}");
let v = src.info().video.as_ref().unwrap();
assert_eq!((v.width, v.height), (95, 63), "{name}");
let ours = rgba(&src.video_frame(FrameRequest::full(filmcraft_time::Tick(0))).unwrap());
// deeper than 8 bits: ffmpeg's 16-bit decode, rounded to 8 bits (its own 8-bit conversion dithers)
let want: Vec<u8> = if tol == 0 {
ffmpeg_out(ff, &["-i", f.to_str().unwrap(), "-f", "rawvideo", "-pix_fmt", "rgba", "-"])
} else {
let wide = ffmpeg_out(ff, &["-i", f.to_str().unwrap(), "-f", "rawvideo", "-pix_fmt", "rgba64le", "-"]);
wide.as_chunks::<2>().0.iter().map(|c| ((u32::from(u16::from_le_bytes(*c)) * 255 + 32767) / 65535) as u8).collect()
};
assert_eq!(ours.len(), want.len(), "{name}");
let d = ours.iter().zip(&want).map(|(a, b)| a.abs_diff(*b)).max().unwrap();
assert!(d <= tol, "{name}: differs by {d}");
assert_eq!(v.has_alpha, want.as_chunks::<4>().0.iter().any(|p| p[3] != 255), "{name}");
eprintln!("{name}: max diff {d}");
}
#[test]
fn targa_stills_every_layout() {
let ff = filmcraft_testkit::require_ffmpeg!();
for pix_fmt in ["bgr24", "bgra", "rgb555le", "gray", "pal8"] {
still_matches(&ff, "tga", pix_fmt, &[], 0); // run-length encoded (ffmpeg's default)
still_matches(&ff, "tga", pix_fmt, &["-rle", "0"], 0);
}
}
#[test]
fn dpx_stills_every_layout() {
let ff = filmcraft_testkit::require_ffmpeg!();
for (pix_fmt, tol) in [("rgb24", 0), ("rgba", 0), ("gray", 0), ("rgb48be", 1), ("rgba64be", 1), ("gray16be", 1), ("gbrp10be", 1), ("gbrp12be", 1)] {
still_matches(&ff, "dpx", pix_fmt, &[], tol);
}
}
#[test]
fn missing_frames_hold_the_previous_frame() {
let ff = filmcraft_testkit::require_ffmpeg!();
+194
View File
@@ -53,3 +53,197 @@ fn bframe_mp4_duration_is_the_decoded_frame_count() {
let src = filmcraft_codecs::open_bytes("bframes_x264.mp4", bytes(&f)).unwrap();
assert_eq!(src.info().duration, filmcraft_time::FrameRate::FPS_30.tick_of(n as i64));
}
/// `mp4` (moov after mdat, as ffmpeg writes it) with a clean aperture (`clap`) of `w`×`h` moved by
/// (`dx`, `dy`) from the centre, inserted at the end of the first video sample entry.
fn with_clap(mp4: &[u8], (w, h, dx, dy): (u32, u32, i32, i32)) -> Vec<u8> {
let be = |b: &[u8], at: usize| u32::from_be_bytes([b[at], b[at + 1], b[at + 2], b[at + 3]]) as usize;
// the chain moov > trak > mdia > minf > stbl > stsd > avc1 (stsd has 8 bytes before its entries)
let mut chain = Vec::new();
let (mut off, mut end) = (0, mp4.len());
for (want, skip) in [(b"moov", 8), (b"trak", 8), (b"mdia", 8), (b"minf", 8), (b"stbl", 8), (b"stsd", 16), (b"avc1", 0)] {
loop {
assert!(off + 8 <= end, "no {}", String::from_utf8_lossy(want));
let size = be(mp4, off);
if &mp4[off + 4..off + 8] == want {
chain.push(off);
end = off + size;
off += skip;
break;
}
off += size;
}
}
let entry = *chain.last().unwrap();
assert!(mp4.windows(4).position(|x| x == b"mdat").unwrap() < chain[0], "moov must follow mdat");
let mut clap = Vec::new();
for v in [40u32, u32::from_be_bytes(*b"clap"), w, 1, h, 1, dx as u32, 1, dy as u32, 1] {
clap.extend_from_slice(&v.to_be_bytes());
}
let mut out = mp4.to_vec();
let at = entry + be(mp4, entry);
out.splice(at..at, clap);
for &b in &chain {
let size = (be(&out, b) + 40) as u32;
out[b..b + 4].copy_from_slice(&size.to_be_bytes());
}
out
}
/// Cropped pictures are the size and the samples ffmpeg decodes (#288): H.264 frame cropping (SPS)
/// and an HEVC conformance window while the sample entry gives the uncropped size, and a clean
/// aperture (`clap`, as iPhones write), centred, off-centre, and with a 90° display rotation
/// applied after it.
#[test]
fn cropped_pictures_match_ffmpeg() {
let ff = filmcraft_testkit::require_ffmpeg!();
let src = ["-f", "lavfi", "-i", "testsrc2=size=320x240:rate=30:duration=1,noise=alls=10:allf=t", "-c:v", "libx264", "-bf", "2", "-pix_fmt", "yuv420p"];
let Some(plain) = fixture(&ff, "crop_plain_x264.mp4", &src) else {
eprintln!("SKIPPED: ffmpeg without libx264");
return;
};
let sps =
fixture(&ff, "crop_sps_x264.mp4", &[&src[..], &["-bsf:v", "h264_metadata=crop_left=16:crop_right=16:crop_top=8:crop_bottom=8"]].concat()).unwrap();
// the display matrix added without re-encoding (an encode would turn the pixels instead)
let rotated = fixture(&ff, "crop_rotated_x264.mp4", &["-display_rotation", "90", "-i", plain.to_str().unwrap(), "-c", "copy"]).unwrap();
let dir = dir();
let mut cases = vec![("SPS crop", sps, (288, 224))];
// HEVC: the conformance window
let hevc = [
"-f",
"lavfi",
"-i",
"testsrc2=size=320x240:rate=30:duration=1,noise=alls=10:allf=t",
"-c:v",
"libx265",
"-x265-params",
"log-level=error:bframes=3",
"-tag:v",
"hvc1",
"-pix_fmt",
"yuv420p",
];
match fixture(&ff, "crop_conformance_x265.mp4", &[&hevc[..], &["-bsf:v", "hevc_metadata=crop_left=16:crop_right=16:crop_top=8:crop_bottom=8"]].concat()) {
Some(p) => cases.push(("HEVC conformance window", p, (288, 224))),
None => eprintln!("SKIPPED (HEVC case): ffmpeg without libx265"),
}
for (name, base, clap, size) in [
("centred clap", &plain, (280, 200, 0, 0), (280, 200)),
("off-centre clap", &plain, (200, 150, -30, 22), (200, 150)),
("clap then rotation", &rotated, (280, 200, 0, 0), (200, 280)),
] {
let path = dir.join(format!("crop_{}.mp4", name.replace(' ', "_")));
std::fs::write(&path, with_clap(&bytes(base), clap)).unwrap();
cases.push((name, path, size));
}
for (name, path, (w, h)) in cases {
let src = filmcraft_codecs::open_bytes(path.file_name().unwrap().to_str().unwrap(), bytes(&path)).unwrap();
let v = src.info().video.clone().unwrap();
assert_eq!((v.width, v.height), (w, h), "{name}: reported size");
let (cw, ch) = (w.div_ceil(2) as usize, h.div_ceil(2) as usize);
// `-flags unaligned`: ffmpeg otherwise skips a left crop that would misalign its frame
// pointers (on AVX-512 machines it needs 64-byte alignment) and returns wider frames
let raw = ffmpeg_out(
&ff,
&["-flags", "unaligned", "-i", path.to_str().unwrap(), "-map", "0:v:0", "-fps_mode", "passthrough", "-f", "rawvideo", "-pix_fmt", "yuv420p", "-"],
);
let frame_bytes = (w * h) as usize + 2 * cw * ch;
if name.contains("clap") && !raw.len().is_multiple_of(frame_bytes) {
// ffmpeg before 7.1 ignores `clap` and decodes the whole picture
eprintln!("SKIPPED ({name}): this ffmpeg does not apply the clean aperture");
continue;
}
assert_eq!(raw.len() % frame_bytes, 0, "{name}: ffmpeg's frames are {w}x{h}");
let n = raw.len() / frame_bytes;
assert_eq!(n, 30, "{name}");
for (i, want) in raw.chunks(frame_bytes).enumerate() {
let f = src.video_frame(filmcraft_media::FrameRequest::full(v.frame_rate.tick_of(i as i64))).unwrap();
assert_eq!((f.width, f.height), (w, h), "{name}: frame {i} size");
assert!(planes(&f) == raw_planes(want, w as usize, h as usize, cw, ch, 1), "{name}: frame {i} differs from ffmpeg's");
}
}
}
/// An open GOP (x265 `open-gop`, like iPhone HEVC): each CRA after the first is followed in the
/// file by RASL pictures shown before it, predicted from the GOP before. Asked for first (a clip
/// trimmed to start on one, the playhead parked on one), such a picture came out black: decoding
/// started at the CRA after it, which leaves its RASL pictures out. Every frame, asked of a fresh
/// source and in an order that visits each CRA before its leading pictures, is ffmpeg's, from MP4
/// and from Matroska.
#[test]
fn open_gop_leading_pictures_decode_alone() {
let ff = filmcraft_testkit::require_ffmpeg!();
let args = [
"-f",
"lavfi",
"-i",
"testsrc2=size=160x96:rate=30:duration=2,noise=alls=10:allf=t",
"-c:v",
"libx265",
"-x265-params",
"log-level=error:keyint=12:min-keyint=12:scenecut=0:open-gop=1:bframes=3",
"-tag:v",
"hvc1",
"-pix_fmt",
"yuv420p",
];
let Some(mp4) = fixture(&ff, "open_gop_x265.mp4", &args) else {
eprintln!("SKIPPED: ffmpeg without libx265");
return;
};
let mkv = fixture(&ff, "open_gop_x265.mkv", &["-i", mp4.to_str().unwrap(), "-c", "copy"]).unwrap();
let raw = ffmpeg_frames(&ff, &mp4, "yuv420p");
let frame_bytes = 160 * 96 * 3 / 2;
let want: Vec<_> = raw.chunks(frame_bytes).map(|f| raw_planes(f, 160, 96, 80, 48, 1)).collect();
assert_eq!(want.len(), 60);
for path in [&mp4, &mkv] {
let name = path.file_name().unwrap().to_str().unwrap();
let open = || filmcraft_codecs::open_bytes(name, bytes(path)).unwrap();
let rate = open().info().frame_rate();
let frame = |src: &filmcraft_media::SharedSource, i: usize| {
let f = src.video_frame(filmcraft_media::FrameRequest::full(rate.tick_of(i as i64))).unwrap_or_else(|e| panic!("{name}: frame {i}: {e}"));
assert!(planes(&f) == want[i], "{name}: frame {i} differs from ffmpeg's");
};
for i in 0..want.len() {
frame(&open(), i);
}
let src = open();
for cra in [12, 24, 36, 48] {
for i in [cra, cra - 1, cra - 2, cra - 3] {
frame(&src, i);
}
}
}
}
/// #800: MP2 in MP4 (`mp4a` + object type 0x6B, which names no layer) and in MOV (QuickTime
/// `.mp2`) decodes sample-exactly as ffmpeg does: same length, the encoder delay (481 samples)
/// trimmed by the edit list, and the same samples from the first one on, also after a seek. It
/// used to go to the layer III decoder (MP4: silence) or not decode at all (MOV).
#[test]
fn mp2_in_mp4_and_mov_matches_ffmpeg() {
let ff = filmcraft_testkit::require_ffmpeg!();
for (name, rate) in [("mp2_stereo_224k_48k.mp4", 48_000u32), ("mp2_stereo_224k_44k.mov", 44_100)] {
let input = format!("anoisesrc=d=1:c=pink:r={rate}:a=0.3");
let Some(f) = fixture(&ff, name, &["-f", "lavfi", "-i", &input, "-ac", "2", "-c:a", "mp2", "-b:a", "224k"]) else {
eprintln!("SKIPPED ({name}): this ffmpeg can't encode it");
continue;
};
let src = filmcraft_codecs::open_bytes(name, bytes(&f)).unwrap();
let a = src.info().audio().cloned().unwrap();
assert_eq!((a.sample_rate, a.channels, a.codec.as_str()), (rate, 2, "MPEG Audio"), "{name}");
let want = ffmpeg_audio_f32(&ff, &f, &[]);
let n = want.len() / 2;
assert!(n > rate as usize * 9 / 10, "{name}: ffmpeg decoded {n} samples");
let worst = |from: usize, len: usize| {
let got = src.audio(from as i64, len, rate).unwrap();
(0..len).map(|k| (0..2).map(|c| (got.channels[c][k] - want[2 * (from + k) + c]).abs()).fold(0f32, f32::max)).fold(0f32, f32::max)
};
let level = want.iter().fold(0f32, |m, v| m.max(v.abs()));
assert!(level > 0.1, "{name}: the source is silent ({level})");
// a seek into the middle first, then the whole track from the start
let (mid, all) = (worst(n / 2 + 77, 3000), worst(0, n));
println!("{name}: {n} samples, max error mid {mid:.2e}, whole {all:.2e}");
assert!(mid <= 1e-4 && all <= 1e-4, "{name}: differs from ffmpeg by {mid} (mid) / {all} (whole)");
}
}
+261
View File
@@ -0,0 +1,261 @@
//! Sony XAVC MP4 with no `tmcd` track: the start timecode lives in an `rtmd` metadata track
//! (issue #460). A hand-built file exercises the whole probe path; the byte layout of the `rtmd`
//! sample is the one real XAVC footage uses (checked against ffprobe's `timecode` tag).
use std::sync::Arc;
use filmcraft_time::FrameRate;
/// Minimal ISO-BMFF box writer (the demuxer only needs headers and sample tables here).
#[derive(Default)]
struct B(Vec<u8>);
impl B {
fn u16(&mut self, v: u16) {
self.0.extend_from_slice(&v.to_be_bytes());
}
fn u32(&mut self, v: u32) {
self.0.extend_from_slice(&v.to_be_bytes());
}
fn bytes(&mut self, b: &[u8]) {
self.0.extend_from_slice(b);
}
fn zeros(&mut self, n: usize) {
self.0.resize(self.0.len() + n, 0);
}
/// A container box (`size` + `kind` + children).
fn container(&mut self, kind: &[u8; 4], f: impl FnOnce(&mut B)) {
let at = self.0.len();
self.u32(0);
self.bytes(kind);
f(self);
let len = (self.0.len() - at) as u32;
self.0[at..at + 4].copy_from_slice(&len.to_be_bytes());
}
/// A full box (`size` + `kind` + version/flags + body).
fn full(&mut self, kind: &[u8; 4], f: impl FnOnce(&mut B)) {
self.container(kind, |b| {
b.zeros(4);
f(b);
});
}
fn leaf(&mut self, kind: &[u8; 4], payload: &[u8]) {
self.container(kind, |b| b.bytes(payload));
}
}
/// A `stsd` visual sample entry (JPEG keeps the video track decodable enough to open).
fn video_entry(b: &mut B) {
b.bytes(&[0; 6]);
b.u16(1);
b.u16(0);
b.u16(0);
b.bytes(&[0; 4]);
b.u32(0);
b.u32(0);
b.u16(1920);
b.u16(1080);
b.u32(0x0048_0000);
b.u32(0x0048_0000);
b.u32(0);
b.u16(1);
b.bytes(&[0; 32]);
b.u16(24);
b.u16(0xFFFF);
}
/// A video track at 59.94 fps, plus a Sony `rtmd` track whose first sample is `rtmd_sample`.
fn file_with_rtmd(rtmd_sample: &[u8]) -> Vec<u8> {
file_with_rtmd_claimed(rtmd_sample, rtmd_sample.len() as u32)
}
/// As [`file_with_rtmd`] but the `rtmd` sample's `stsz` size is `claimed` (which may exceed the
/// bytes actually present, as a corrupt file can).
fn file_with_rtmd_claimed(rtmd_sample: &[u8], claimed: u32) -> Vec<u8> {
let mut b = B::default();
b.leaf(b"ftyp", b"isom\x00\x00\x00\x00isom");
let mdat_start = b.0.len();
let mdat: Vec<u8> = [&[1, 2, 3, 4][..], rtmd_sample].concat();
b.leaf(b"mdat", &mdat);
let video_off = (mdat_start + 8) as u32;
let rtmd_off = video_off + 4;
b.container(b"moov", |b| {
b.full(b"mvhd", |b| {
b.zeros(8);
b.u32(1000);
b.u32(1001);
b.zeros(80);
});
// Video track (id 1): one sample of 1001/60000 s → 59.94 fps.
b.container(b"trak", |b| {
b.full(b"tkhd", |b| {
b.zeros(8);
b.u32(1);
b.zeros(4);
b.u32(1001);
b.zeros(52);
b.u32(1920 << 16);
b.u32(1080 << 16);
});
b.container(b"mdia", |b| {
b.full(b"mdhd", |b| {
b.zeros(8);
b.u32(60000);
b.u32(1001);
b.u16(0x55C4);
b.u16(0);
});
b.full(b"hdlr", |b| {
b.zeros(4);
b.bytes(b"vide");
b.zeros(12);
b.bytes(b"Video\0");
});
b.container(b"minf", |b| {
b.container(b"stbl", |b| {
b.full(b"stsd", |b| {
b.u32(1);
b.container(b"jpeg", video_entry);
});
b.full(b"stts", |b| {
b.u32(1);
b.u32(1);
b.u32(1001);
});
b.full(b"stsc", |b| {
b.u32(1);
b.u32(1);
b.u32(1);
b.u32(1);
});
b.full(b"stsz", |b| {
b.u32(0);
b.u32(1);
b.u32(4);
});
b.full(b"stco", |b| {
b.u32(1);
b.u32(video_off);
});
});
});
});
});
// Sony rtmd metadata track (id 2).
b.container(b"trak", |b| {
b.full(b"tkhd", |b| {
b.zeros(8);
b.u32(2);
b.zeros(4);
b.u32(1);
b.zeros(52);
b.zeros(8);
});
b.container(b"mdia", |b| {
b.full(b"mdhd", |b| {
b.zeros(8);
b.u32(1);
b.u32(1);
b.u16(0x55C4);
b.u16(0);
});
b.full(b"hdlr", |b| {
b.zeros(4);
b.bytes(b"meta");
b.zeros(12);
b.bytes(b"Timed Metadata\0");
});
b.container(b"minf", |b| {
b.container(b"stbl", |b| {
b.full(b"stsd", |b| {
b.u32(1);
b.leaf(b"rtmd", &[0, 0, 0, 0, 0, 0, 0, 1]);
});
b.full(b"stts", |b| {
b.u32(1);
b.u32(1);
b.u32(1);
});
b.full(b"stsc", |b| {
b.u32(1);
b.u32(1);
b.u32(1);
b.u32(1);
});
b.full(b"stsz", |b| {
b.u32(0);
b.u32(1);
b.u32(claimed);
});
b.full(b"stco", |b| {
b.u32(1);
b.u32(rtmd_off);
});
});
});
});
});
});
b.0
}
/// A Sony `rtmd` sample: hours, minutes, seconds at 0x0d–0x0f, a drop-frame flag at 0x10 and the
/// frame number at 0x11, then the start of the SMPTE KLV key.
fn rtmd_sample(h: u8, m: u8, s: u8, f: u8, drop: u8) -> Vec<u8> {
let mut v = vec![0u8; 24];
v[0x0d] = h;
v[0x0e] = m;
v[0x0f] = s;
v[0x10] = drop;
v[0x11] = f;
v[0x12..0x16].copy_from_slice(&[0x06, 0x0e, 0x2b, 0x34]);
v
}
#[test]
fn rtmd_start_timecode_is_read() {
let file = file_with_rtmd(&rtmd_sample(3, 37, 12, 34, 0));
let src = filmcraft_codecs::open_bytes("fx3.mp4", Arc::from(file.into_boxed_slice())).unwrap();
let rate = FrameRate::new(60000, 1001);
let tc = src.info().start_timecode.expect("rtmd start timecode");
assert_eq!(tc, 781_954);
assert_eq!(filmcraft_time::format_timecode_frames(tc, rate, false), "03:37:12:34");
}
#[test]
fn rtmd_without_usable_sample_is_ignored() {
// Too short to carry the timecode block: no start timecode, and no panic.
let file = file_with_rtmd(&[0u8; 6]);
let src = filmcraft_codecs::open_bytes("short.mp4", Arc::from(file.into_boxed_slice())).unwrap();
assert_eq!(src.info().start_timecode, None);
}
#[test]
fn huge_claimed_rtmd_sample_does_not_exhaust_memory() {
// The stsz size claims 16 MiB but only the header bytes exist: only the fixed-size header is
// read, so a corrupt size cannot drive a huge allocation during open (issue #460 review).
let file = file_with_rtmd_claimed(&rtmd_sample(1, 2, 3, 4, 0), 16 * 1024 * 1024);
let src = filmcraft_codecs::open_bytes("big.mp4", Arc::from(file.into_boxed_slice())).unwrap();
assert_eq!(src.info().start_timecode, Some(223_384)); // 01:02:03:04 at 59.94
}
#[test]
fn damaged_timecode_fields_are_not_published() {
// Frame 60 at 59.94 fps (valid range is 0–59): unusable metadata, not a fabricated time.
let file = file_with_rtmd(&rtmd_sample(3, 37, 12, 60, 0));
let src = filmcraft_codecs::open_bytes("bad.mp4", Arc::from(file.into_boxed_slice())).unwrap();
assert_eq!(src.info().start_timecode, None);
}
#[test]
fn drop_frame_skipped_labels_are_not_published() {
// 00:01:00;00 at 59.94 fps: frames 00-03 of a non-tenth minute don't exist in drop-frame
// counting, so this metadata is unusable rather than a start time.
let file = file_with_rtmd(&rtmd_sample(0, 1, 0, 0, 1));
let src = filmcraft_codecs::open_bytes("df_bad.mp4", Arc::from(file.into_boxed_slice())).unwrap();
assert_eq!(src.info().start_timecode, None);
// The first real label of that minute (frame 04) is accepted.
let file = file_with_rtmd(&rtmd_sample(0, 1, 0, 4, 1));
let src = filmcraft_codecs::open_bytes("df_ok.mp4", Arc::from(file.into_boxed_slice())).unwrap();
assert_eq!(src.info().start_timecode, Some(3600));
}
+65 -16
View File
@@ -177,15 +177,64 @@ fn open(p: &Path) -> MpegSource {
MpegSource::open(p.file_name().unwrap().to_str().unwrap(), bytes(p)).unwrap()
}
/// ffprobe's video frames in display order: pts (90 kHz).
/// ffprobe's video frames in display order: PTS (90 kHz). A program-stream picture
/// need not carry its own PTS; use the decoder's presentation timestamp for that picture.
fn ffprobe_frame_pts(file: &Path) -> Option<Vec<i64>> {
let fp = filmcraft_testkit::ffprobe()?;
let fp = filmcraft_testkit::ffprobe_or_skip("MPEG video frame presentation timestamps")?;
let o = std::process::Command::new(fp)
.args(["-v", "error", "-select_streams", "v:0", "-show_entries", "frame=pts", "-of", "csv=p=0"])
.args(["-v", "error", "-select_streams", "v:0", "-show_entries", "frame=pts,best_effort_timestamp", "-of", "json"])
.arg(file)
.output()
.ok()?;
Some(String::from_utf8_lossy(&o.stdout).lines().filter_map(|l| l.trim().trim_end_matches(',').parse().ok()).collect())
.unwrap_or_else(|error| panic!("{}: cannot run ffprobe: {error}", file.display()));
assert!(o.status.success(), "{}: ffprobe failed: {}", file.display(), String::from_utf8_lossy(&o.stderr));
Some(parse_ffprobe_frame_pts(&o.stdout).unwrap_or_else(|error| panic!("{}: {error}", file.display())))
}
fn parse_ffprobe_frame_pts(bytes: &[u8]) -> Result<Vec<i64>, String> {
let data: serde_json::Value = serde_json::from_slice(bytes).map_err(|error| format!("invalid ffprobe JSON: {error}"))?;
let frames = data.get("frames").and_then(serde_json::Value::as_array).ok_or("ffprobe output is missing its frames array")?;
if frames.is_empty() {
return Err("ffprobe returned no video frames".into());
}
frames
.iter()
.enumerate()
.map(|(index, frame)| {
let field = if frame.get("pts").is_some() { "pts" } else { "best_effort_timestamp" };
frame.get(field).and_then(serde_json::Value::as_i64).ok_or_else(|| format!("ffprobe video frame {index} is missing an integer {field}"))
})
.collect()
}
#[test]
fn ffprobe_pts_preserve_missing_raw_timestamps_and_side_data() {
let output = br#"{"frames":[
{"pts":131400,"best_effort_timestamp":131401,"side_data_list":[{},{}]},
{"best_effort_timestamp":135000,"side_data_list":[{}]},
{"pts":138600,"best_effort_timestamp":138600}
]}"#;
assert_eq!(parse_ffprobe_frame_pts(output).unwrap(), [131400, 135000, 138600], "keep every display frame, with raw PTS preferred when present");
assert_eq!(parse_ffprobe_frame_pts(br#"{"frames":[{"pts":-3600},{"pts":0},{"pts":9223372036854775807}]}"#).unwrap(), [-3600, 0, i64::MAX]);
}
#[test]
fn malformed_ffprobe_pts_fail_instead_of_dropping_frames() {
for output in [
"not JSON",
"{}",
r#"{"frames":{}}"#,
r#"{"frames":[]}"#,
r#"{"frames":[{}]}"#,
r#"{"frames":[null]}"#,
r#"{"frames":[{"pts":"N/A","best_effort_timestamp":135000}]}"#,
r#"{"frames":[{"pts":null,"best_effort_timestamp":135000}]}"#,
r#"{"frames":[{"pts":1.5}]}"#,
r#"{"frames":[{"pts":9223372036854775808}]}"#,
r#"{"frames":[{"best_effort_timestamp":"135000"}]}"#,
r#"{"frames":[{"pts":131400},{"best_effort_timestamp":null},{"pts":138600}]}"#,
] {
assert!(parse_ffprobe_frame_pts(output.as_bytes()).is_err(), "malformed oracle output must fail: {output}");
}
}
/// Decode every frame in order and `seeks` random frames; compare with ffmpeg within `tol`.
@@ -245,7 +294,7 @@ fn check_video(ff: &Path, file: &Path, tol: u16, seeks: usize) -> (u16, f64) {
/// Our decoded audio from the first audio frame against ffmpeg's decode. Returns the max error.
fn check_audio(ff: &Path, file: &Path, tol: f32) -> f32 {
let src = open(file);
let a = src.info().audio.clone().unwrap();
let a = src.info().audio().cloned().unwrap();
let (start, len) = src.audio_extent().unwrap();
let want = ffmpeg_audio_f32(ff, file, &[]);
let ch = a.channels as usize;
@@ -314,7 +363,7 @@ fn transport_stream_mpeg2_and_mp2() {
let (worst, psnr) = check_video(&ff, &f, 4, 12);
assert!(psnr >= 58.0, "PSNR {psnr}");
println!("ts_mpeg2_mp2: video max diff {worst}, PSNR {psnr:.2} dB");
let a = src.info().audio.clone().unwrap();
let a = src.info().audio().cloned().unwrap();
assert_eq!((a.codec.as_str(), a.sample_rate, a.channels), ("MPEG Audio", 48_000, 2));
let e = check_audio(&ff, &f, 2e-4);
println!("ts_mpeg2_mp2: MP2 audio max error {e:.2e}");
@@ -337,7 +386,7 @@ fn bdav_h264_with_lpcm_sample_exact() {
let f = make(&ff, "m2ts_h264_lpcm.m2ts");
let src = open(&f);
assert_eq!(src.info().container, "MPEG-2 TS (BDAV/AVCHD)");
let a = src.info().audio.clone().unwrap();
let a = src.info().audio().cloned().unwrap();
assert_eq!((a.codec.as_str(), a.channels, a.bits_per_sample), ("LPCM (Blu-ray)", 2, Some(24)));
check_video(&ff, &f, 0, 10);
assert_eq!(check_audio(&ff, &f, 0.0), 0.0);
@@ -349,7 +398,7 @@ fn avchd_ac3_audio() {
let f = make(&ff, "m2ts_h264_ac3.mts");
let src = open(&f);
check_video(&ff, &f, 0, 6);
let a = src.info().audio.clone().unwrap();
let a = src.info().audio().cloned().unwrap();
if filmcraft_codecs::audio::AC3_DECODER {
assert_eq!((a.codec.as_str(), a.channels), ("AC-3", 2));
// AC-3 zero-bit mantissas carry decoder-specific dither (A/52 §7.3.4)
@@ -370,7 +419,7 @@ fn transport_stream_hevc_and_latm() {
let f = make(&ff, "ts_hevc_latm.ts");
let src = open(&f);
assert_eq!(src.info().video.as_ref().unwrap().codec, "HEVC");
assert_eq!(src.info().audio.as_ref().unwrap().codec, "AAC (LATM)");
assert_eq!(src.info().audio().unwrap().codec, "AAC (LATM)");
check_video(&ff, &f, 0, 8);
let e = check_audio(&ff, &f, 1e-4);
println!("ts_hevc_latm: AAC (LATM) max error {e:.2e}");
@@ -389,17 +438,17 @@ fn program_streams() {
// DVD LPCM: sample-exact
let f = make(&ff, "ps_mpeg2_lpcm.vob");
let src = open(&f);
assert_eq!(src.info().audio.as_ref().unwrap().codec, "LPCM (DVD)");
assert_eq!(src.info().audio.as_ref().unwrap().bits_per_sample, Some(24));
assert_eq!(src.info().audio().unwrap().codec, "LPCM (DVD)");
assert_eq!(src.info().audio().unwrap().bits_per_sample, Some(24));
assert_eq!(check_audio(&ff, &f, 0.0), 0.0);
check_video(&ff, &f, 4, 4);
let f = make(&ff, "ps_mpeg2_lpcm16.vob");
assert_eq!(open(&f).info().audio.as_ref().unwrap().bits_per_sample, Some(16));
assert_eq!(open(&f).info().audio().unwrap().bits_per_sample, Some(16));
assert_eq!(check_audio(&ff, &f, 0.0), 0.0);
// AC-3 in private stream 1
let f = make(&ff, "ps_mpeg2_ac3.vob");
let src = open(&f);
assert_eq!(src.info().audio.as_ref().unwrap().codec, "AC-3");
assert_eq!(src.info().audio().unwrap().codec, "AC-3");
let e = check_audio(&ff, &f, 5e-3);
println!("ps_mpeg2_ac3: AC-3 max error {e:.2e}");
// MPEG-1 system stream
@@ -457,7 +506,7 @@ fn standalone_mp2_file() {
let ff = filmcraft_testkit::require_ffmpeg!();
let f = make(&ff, "audio.mp2");
let src = filmcraft_codecs::open_bytes("audio.mp2", bytes(&f)).unwrap();
let a = src.info().audio.clone().unwrap();
let a = src.info().audio().cloned().unwrap();
assert_eq!((a.sample_rate, a.channels), (48_000, 2));
let want = ffmpeg_audio_f32(&ff, &f, &[]);
let n = want.len() / 2;
@@ -483,7 +532,7 @@ fn truncated_and_corrupt_files_never_panic() {
let _ = s.video_frame(FrameRequest::full(v.frame_rate.tick_of(i)));
}
}
if let Some(a) = &info.audio {
if let Some(a) = info.audio() {
let _ = s.audio(rng.below(48_000) as i64, 2048, a.sample_rate);
}
};
+7 -7
View File
@@ -179,7 +179,7 @@ fn check_video(ff: &Path, file: &Path, pix_fmt: &str, chroma: (usize, usize), bp
/// Our audio, as interleaved f32, equals ffmpeg's decode exactly.
fn check_audio_exact(ff: &Path, file: &Path, extra: &[&str]) {
let src = open(file);
let a = src.info().audio.clone().unwrap();
let a = src.info().audio().cloned().unwrap();
let ch = a.channels as usize;
let want = ffmpeg_audio_f32(ff, file, extra);
let n = want.len() / ch;
@@ -236,7 +236,7 @@ fn h264_2997_drop_frame_timecode_and_24_bit_audio() {
assert!(tc.drop_frame);
assert_eq!(tc.format(), "01:00:00;00");
assert_eq!(info.start_timecode, Some(107_892));
assert_eq!(info.audio.as_ref().unwrap().bits_per_sample, Some(24));
assert_eq!(info.audio().unwrap().bits_per_sample, Some(24));
check_video(&ff, &f, "yuv420p", (2, 2), 1, 0, 10);
check_audio_exact(&ff, &f, &[]);
}
@@ -299,14 +299,14 @@ fn op_atom_video_and_audio_files() {
let v = make(&ff, "mxf_atom_dnxhr.mxf");
let src = open(&v);
assert_eq!(src.info().container, "MXF OP-Atom");
assert!(src.info().audio.is_none());
assert!(!src.info().has_audio());
assert_eq!(src.file().tracks[0].wrapping, filmcraft_mxf::Wrapping::Clip);
check_video(&ff, &v, "yuv422p", (2, 1), 1, 2, 10);
let a = make(&ff, "mxf_atom_pcm.mxf");
let src = open(&a);
assert_eq!(src.info().kind, MediaKind::AudioOnly);
assert_eq!(src.info().container, "MXF OP-Atom");
let ai = src.info().audio.clone().unwrap();
let ai = src.info().audio().cloned().unwrap();
assert_eq!((ai.sample_rate, ai.channels, ai.bits_per_sample), (48_000, 1, Some(24)));
assert_eq!(src.info().duration, filmcraft_time::Tick::from_units(48_000, 48_000));
check_audio_exact(&ff, &a, &[]);
@@ -317,7 +317,7 @@ fn d10_aes3_elements_sample_exact() {
let ff = filmcraft_testkit::require_ffmpeg!();
let f = make(&ff, "mxf_d10.mxf");
let src = open(&f);
let a = src.info().audio.clone().unwrap();
let a = src.info().audio().cloned().unwrap();
assert_eq!(a.codec, "AES3 PCM");
assert!(a.channels >= 4, "{}", a.channels);
let v = src.info().video.as_ref().unwrap();
@@ -347,7 +347,7 @@ fn truncated_and_corrupt_files_never_panic() {
let _ = s.video_frame(FrameRequest::full(t));
}
}
if let Some(a) = &info.audio {
if let Some(a) = info.audio() {
let _ = s.audio(rng.below(96_000) as i64, 4000, a.sample_rate);
}
}
@@ -364,7 +364,7 @@ fn truncated_and_corrupt_files_never_panic() {
let _ = s.video_frame(FrameRequest::full(s.info().frame_rate().tick_of(i)));
}
}
if s.info().audio.is_some() {
if s.info().has_audio() {
let _ = s.audio(0, 9600, 48_000);
}
}
+3 -3
View File
@@ -88,7 +88,7 @@ fn snr_db(reference: &[f32], ours: &[f32]) -> f64 {
/// (our continuous decode interleaved, ffmpeg's decode interleaved, channels, rate)
fn decode_both(ff: &Path, file: &Path, decoder: &[&str]) -> (OggSource, Vec<f32>, Vec<f32>, usize, u32) {
let src = OggSource::open(file.file_name().unwrap().to_str().unwrap(), bytes(file)).unwrap();
let a = src.info().audio.clone().unwrap();
let a = src.info().audio().cloned().unwrap();
let ch = a.channels as usize;
let mut args: Vec<&str> = decoder.to_vec();
args.extend_from_slice(&["-i", file.to_str().unwrap(), "-map", "0:a:0", "-f", "f32le", "-"]);
@@ -134,7 +134,7 @@ fn opus_celt_music_matches_libopus() {
assert_eq!(src.info().container, "Ogg");
assert_eq!(src.info().kind, MediaKind::AudioOnly);
assert_eq!((rate, ch), (48_000, 2));
assert_eq!(src.info().audio.as_ref().unwrap().codec, "Opus");
assert_eq!(src.info().audio().unwrap().codec, "Opus");
let snr = snr_db(&want, &ours);
eprintln!("{}: {} samples, SNR {snr:.1} dB vs libopus", f.display(), want.len() / ch);
assert!(snr >= 50.0, "SNR {snr:.1} dB");
@@ -173,7 +173,7 @@ fn vorbis_matches_ffmpeg() {
let ff = filmcraft_testkit::require_ffmpeg!();
let f = make(&ff, "ogg_vorbis.ogg");
let (src, ours, want, ch, rate) = decode_both(&ff, &f, &[]);
assert_eq!(src.info().audio.as_ref().unwrap().codec, "Vorbis");
assert_eq!(src.info().audio().unwrap().codec, "Vorbis");
assert_eq!((rate, ch), (44_100, 2));
let snr = snr_db(&want, &ours);
eprintln!("{}: SNR {snr:.1} dB vs ffmpeg's Vorbis decoder", f.display());
+1
View File
@@ -16,5 +16,6 @@ thiserror = { workspace = true }
[dev-dependencies]
proptest = { workspace = true }
filmcraft-geom = { workspace = true }
filmcraft-media = { workspace = true }
serde_json = { workspace = true }
+213
View File
@@ -0,0 +1,213 @@
//! Sequence Settings ▸ "Scale motion effects proportionally when changing frame size".
//!
//! Motion's Position is in sequence pixels and its Scale is relative to the source, so a new frame
//! size leaves every clip where it was in pixels: smaller and off-centre in a larger frame, cropped
//! in a smaller one. With the option on, the whole picture scales instead, by one factor about the
//! frame centre. On an aspect change (16:9 → 9:16) that factor is the one that fits the old frame
//! inside the new one (`min(W1/W0, H1/H0)`), so nothing is cropped and the old composition sits
//! centred, letterboxed or pillarboxed, rather than filling the new frame.
use filmcraft_project::{ParamValue, Sequence, TrackKind};
/// Motion Scale's upper bound (the parameter's range, in %).
const MAX_SCALE: f64 = 10_000.0;
/// The uniform factor `k` and the old and new frame centres for `from` → `to`, or `None` for an
/// empty frame on either side.
fn fit(from: (u32, u32), to: (u32, u32)) -> Option<(f64, (f64, f64), (f64, f64))> {
if from.0 == 0 || from.1 == 0 || to.0 == 0 || to.1 == 0 {
return None;
}
let (w0, h0, w1, h1) = (f64::from(from.0), f64::from(from.1), f64::from(to.0), f64::from(to.1));
let k = (w1 / w0).min(h1 / h0);
(k.is_finite() && k > 0.0).then_some((k, (w0 / 2.0, h0 / 2.0), (w1 / 2.0, h1 / 2.0)))
}
/// Rescale the Motion of every video clip in `seq` for a frame-size change from `from` to `to`:
/// static values and every keyframe. Returns how many clips were rescaled (0 when the size does not
/// change).
///
/// - Position maps through `p' = c1 + k·(p − c0)`. An automatic (NaN) position stays automatic: it
/// follows the frame centre on its own.
/// - Scale and Scale Width are multiplied by `k`, up to the parameter's 10 000 %. A clip with Scale to
/// Frame Size keeps its Scale, because the renderer refits it to the new frame anyway.
/// - Anchor Point (source pixels) and Rotation do not depend on the frame size and stay as they are.
/// - Only the intrinsic Motion effect changes, as the option's name says: the points of standard
/// effects (a Mirror centre, the Transform effect's Position) keep their pixel values.
pub fn scale_motion(seq: &mut Sequence, from: (u32, u32), to: (u32, u32)) -> usize {
if from == to {
return 0;
}
let Some((k, c0, c1)) = fit(from, to) else { return 0 };
let mut n = 0;
for track in seq.all_tracks_mut().filter(|t| t.kind == TrackKind::Video) {
for item in &mut track.items {
let refits = item.scale_to_frame;
let Some(motion) = item.effect_mut("motion") else { continue };
if let Some(p) = motion.param_mut("position") {
p.map_values(|v| match v {
ParamValue::Vec2(mut v) => {
v.x = c1.0 + k * (v.x - c0.0);
v.y = c1.1 + k * (v.y - c0.1);
ParamValue::Vec2(v)
}
other => other,
});
}
if !refits {
for id in ["scale", "scale_width"] {
if let Some(p) = motion.param_mut(id) {
p.map_values(|v| match v {
ParamValue::Float(s) => ParamValue::Float((s * k).clamp(0.0, MAX_SCALE)),
other => other,
});
}
}
}
n += 1;
}
}
n
}
#[cfg(test)]
mod tests {
use super::*;
use filmcraft_project::{ClipId, ItemId, Keyframe, Label, Project, SequenceSettings, TrackItem, find_effect};
use filmcraft_time::Tick;
fn seq_with(items: Vec<TrackItem>) -> Sequence {
let mut p = Project::new("t");
let s = p.new_sequence("s", SequenceSettings::default(), 2, 1, None);
let mut seq = p.sequence(s).unwrap().clone();
seq.video_tracks[0].items = items;
seq
}
fn clip(id: u64, pos: (f64, f64), scale: f64) -> TrackItem {
let mut motion = find_effect("motion").unwrap().instance();
motion.param_mut("position").unwrap().value = ParamValue::Vec2(filmcraft_geom::Vec2::new(pos.0, pos.1));
motion.param_mut("scale").unwrap().value = ParamValue::Float(scale);
TrackItem {
id: ClipId(id),
item: ItemId(1),
name: format!("c{id}"),
label: Label::Iris,
start: Tick(id as i64 * 1_000_000_000_000),
duration: Tick(1_000_000_000_000),
source_in: Tick::ZERO,
speed: 1.0,
reverse: false,
enabled: true,
link: None,
group: None,
effects: vec![motion],
markers: vec![],
gain_db: 0.0,
frame_hold: None,
scale_to_frame: false,
essential: None,
multicam: None,
time_interpolation: Default::default(),
hold_filters: false,
field_options: None,
source_channels: Vec::new(),
audio_stream: 0,
graphic: None,
}
}
fn pos(seq: &Sequence, i: usize) -> (f64, f64) {
match seq.video_tracks[0].items[i].effect("motion").unwrap().param("position").unwrap().value {
ParamValue::Vec2(v) => (v.x, v.y),
_ => panic!("position is a point"),
}
}
fn scale(seq: &Sequence, i: usize) -> f64 {
match seq.video_tracks[0].items[i].effect("motion").unwrap().param("scale").unwrap().value {
ParamValue::Float(s) => s,
_ => panic!("scale is a number"),
}
}
fn close(a: (f64, f64), b: (f64, f64)) -> bool {
(a.0 - b.0).abs() < 1e-9 && (a.1 - b.1).abs() < 1e-9
}
#[test]
fn same_aspect_scales_position_and_scale_together() {
let mut seq = seq_with(vec![clip(1, (960.0, 540.0), 100.0), clip(2, (300.0, 90.0), 50.0)]);
assert_eq!(scale_motion(&mut seq, (1920, 1080), (1280, 720)), 2);
assert!(close(pos(&seq, 0), (640.0, 360.0)), "{:?}", pos(&seq, 0));
assert!(close(pos(&seq, 1), (200.0, 60.0)), "{:?}", pos(&seq, 1));
assert!((scale(&seq, 0) - 100.0 * 2.0 / 3.0).abs() < 1e-9);
assert!((scale(&seq, 1) - 50.0 * 2.0 / 3.0).abs() < 1e-9);
}
#[test]
fn aspect_change_fits_the_old_frame_inside_the_new_one() {
// 1920x1080 -> 1080x1920: k = min(1080/1920, 1920/1080) = 0.5625, centred
let mut seq = seq_with(vec![clip(1, (960.0, 540.0), 100.0), clip(2, (0.0, 0.0), 100.0)]);
scale_motion(&mut seq, (1920, 1080), (1080, 1920));
assert!(close(pos(&seq, 0), (540.0, 960.0)), "{:?}", pos(&seq, 0));
// the old top-left corner lands on the new frame's left edge, letterboxed vertically
assert!(close(pos(&seq, 1), (0.0, 960.0 - 0.5625 * 540.0)), "{:?}", pos(&seq, 1));
assert!((scale(&seq, 0) - 56.25).abs() < 1e-9);
}
#[test]
fn keyframes_scale_too() {
let mut c = clip(1, (960.0, 540.0), 100.0);
let m = c.effect_mut("motion").unwrap();
m.param_mut("position").unwrap().keyframes = vec![
Keyframe::new(Tick::ZERO, ParamValue::Vec2(filmcraft_geom::Vec2::new(0.0, 0.0))),
Keyframe::new(Tick(10), ParamValue::Vec2(filmcraft_geom::Vec2::new(1920.0, 1080.0))),
];
m.param_mut("scale").unwrap().keyframes = vec![Keyframe::new(Tick::ZERO, ParamValue::Float(100.0)), Keyframe::new(Tick(10), ParamValue::Float(200.0))];
let mut seq = seq_with(vec![c]);
scale_motion(&mut seq, (1920, 1080), (3840, 2160));
let m = seq.video_tracks[0].items[0].effect("motion").unwrap();
let kp: Vec<_> = m.param("position").unwrap().keyframes.iter().map(|k| k.value.clone()).collect();
assert_eq!(kp, vec![ParamValue::Vec2(filmcraft_geom::Vec2::new(0.0, 0.0)), ParamValue::Vec2(filmcraft_geom::Vec2::new(3840.0, 2160.0))]);
let ks: Vec<_> = m.param("scale").unwrap().keyframes.iter().map(|k| k.value.clone()).collect();
assert_eq!(ks, vec![ParamValue::Float(200.0), ParamValue::Float(400.0)]);
}
#[test]
fn automatic_position_stays_automatic() {
let mut seq = seq_with(vec![clip(1, (f64::NAN, f64::NAN), 100.0)]);
scale_motion(&mut seq, (1920, 1080), (1280, 720));
let (x, y) = pos(&seq, 0);
assert!(x.is_nan() && y.is_nan());
assert!((scale(&seq, 0) - 100.0 * 2.0 / 3.0).abs() < 1e-9);
}
#[test]
fn scale_to_frame_clips_keep_their_scale_but_move() {
let mut c = clip(1, (480.0, 270.0), 100.0);
c.scale_to_frame = true;
let mut seq = seq_with(vec![c]);
scale_motion(&mut seq, (1920, 1080), (3840, 2160));
assert!(close(pos(&seq, 0), (960.0, 540.0)), "{:?}", pos(&seq, 0));
assert_eq!(scale(&seq, 0), 100.0);
}
#[test]
fn scale_is_capped_at_the_parameter_range() {
let mut seq = seq_with(vec![clip(1, (1.0, 1.0), 5000.0)]);
scale_motion(&mut seq, (2, 2), (8, 8));
assert_eq!(scale(&seq, 0), MAX_SCALE);
}
#[test]
fn same_size_or_empty_frame_changes_nothing() {
let mut seq = seq_with(vec![clip(1, (100.0, 100.0), 100.0)]);
let before = seq.clone();
assert_eq!(scale_motion(&mut seq, (1920, 1080), (1920, 1080)), 0);
assert_eq!(scale_motion(&mut seq, (0, 1080), (1280, 720)), 0);
assert_eq!(scale_motion(&mut seq, (1920, 1080), (1280, 0)), 0);
// Debug, not ==: the automatic anchor is NaN, which never equals itself
assert_eq!(format!("{:?}", seq.video_tracks[0].items[0].effects), format!("{:?}", before.video_tracks[0].items[0].effects));
}
}
+232 -51
View File
@@ -18,9 +18,11 @@
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
pub mod captions;
pub mod frame_size;
pub mod multicam;
pub mod through;
pub mod transcript;
pub mod transitions;
use std::collections::HashMap;
@@ -35,8 +37,9 @@ pub enum EditError {
NoTrack(TrackId),
#[error("track is locked")]
Locked,
#[error("this edit would break sync on a sync-locked track")]
SyncLockConflict,
/// Names the sync-locked track (`A2`) whose clip is in the way.
#[error("this edit would break sync: {0} is sync-locked and has a clip in the way (turn off its sync lock, or lock the track, to make this edit)")]
SyncLockConflict(String),
#[error("not enough media (handles) for this trim")]
NoHandles,
#[error("edit would make a clip shorter than one frame")]
@@ -211,6 +214,26 @@ fn transitions_follow_cuts(before: &Sequence, after: &mut Sequence) {
}
}
/// Keep a clip's one-sided transitions (a fade in from nothing, a fade out to nothing) on its edges
/// after a regular trim moved them, no longer than the clip (#374). Transitions between two clips
/// are left alone: a regular trim cannot move their cut.
fn fades_follow_edges(track: &mut Track, clip: ClipId) {
let Some((start, end, len)) = track.item(clip).map(|i| (i.start, i.end(), i.duration)) else { return };
for tr in &mut track.transitions {
match (tr.from, tr.to) {
(None, Some(to)) if to == clip => {
tr.duration = tr.duration.min(len);
tr.start = start;
}
(Some(from), None) if from == clip => {
tr.duration = tr.duration.min(len);
tr.start = end - tr.duration;
}
_ => {}
}
}
}
fn place(track: &mut Track, item: TrackItem) {
let idx = track.items.partition_point(|i| i.start <= item.start);
track.items.insert(idx, item);
@@ -381,6 +404,17 @@ fn ripple_delete_spans(seq: &Sequence, items: &[ClipId]) -> Vec<TimeRange> {
spans
}
/// How the UI names a track: `V1`, `A2` (by position), else its name.
pub fn track_label(seq: &Sequence, id: TrackId) -> String {
if let Some(i) = seq.video_tracks.iter().position(|t| t.id == id) {
return format!("V{}", i + 1);
}
if let Some(i) = seq.audio_tracks.iter().position(|t| t.id == id) {
return format!("A{}", i + 1);
}
seq.track(id).map(|t| t.name.clone()).unwrap_or_else(|| format!("track {}", id.0))
}
/// `ranges` in time order, with the ones that touch or overlap joined.
fn merged(mut ranges: Vec<TimeRange>) -> Vec<TimeRange> {
ranges.sort_by_key(|r| r.start);
@@ -422,7 +456,7 @@ pub fn ripple_delete_items(seq: &mut Sequence, items: &[ClipId]) -> Result<Vec<T
continue;
}
if !on_affected && !track_range_empty(tr, *span) {
return Err(EditError::SyncLockConflict);
return Err(EditError::SyncLockConflict(track_label(seq, tr.id)));
}
shift_track_from(tr, span.end(), -span.duration);
}
@@ -434,6 +468,10 @@ pub fn ripple_delete_items(seq: &mut Sequence, items: &[ClipId]) -> Result<Vec<T
/// Close the gap containing `t` on a track (Ripple Delete on a gap).
pub fn close_gap(seq: &mut Sequence, track: TrackId, t: Tick) -> Result<()> {
let tr = seq.track(track).ok_or(EditError::NoTrack(track))?;
if tr.locked {
// the other tracks would move while this one stayed put
return Err(EditError::Locked);
}
if tr.item_at(t).is_some() {
return Err(EditError::Nothing);
}
@@ -450,7 +488,7 @@ pub fn close_gap(seq: &mut Sequence, track: TrackId, t: Tick) -> Result<()> {
continue;
}
if !track_range_empty(tr, gap) {
return Err(EditError::SyncLockConflict);
return Err(EditError::SyncLockConflict(track_label(seq, tr.id)));
}
}
shift_track_from(tr, gap.end(), -gap.duration);
@@ -473,17 +511,99 @@ pub fn move_items(seq: &mut Sequence, moves: &[(ClipId, TrackId, Tick)], insert_
it.start = (*start).max(Tick::ZERO);
placed.push((*dest, it));
}
// carry transitions with moved clips? Premiere drops transitions whose partner is not moved.
let carried = carried_transitions(&work, &placed);
// a transition whose partner stays behind is dropped, as in Premiere
delete_items(&mut work, &moves.iter().map(|m| m.0).collect::<Vec<_>>());
if insert_mode {
insert(&mut work, placed, ctx)?;
} else {
overwrite(&mut work, placed, ctx)?;
}
for (dest, tr) in carried {
let t = track_mut(&mut work, dest)?;
if [tr.from, tr.to].into_iter().flatten().all(|c| t.item(c).is_some()) {
t.transitions.push(tr);
t.sort();
}
}
*seq = work;
Ok(())
}
/// Copy items to (track, time) with overwrite or insert semantics, leaving the originals where
/// they are (Option-drag). `moves`: (item, destination track, start of the copy). The copies get
/// new ids; copies of linked clips are linked to each other, not to the originals, and the
/// transitions that would travel with a move are copied with them. Returns the copies' ids in
/// the order of `moves`.
pub fn copy_items(seq: &mut Sequence, moves: &[(ClipId, TrackId, Tick)], insert_mode: bool, ctx: &mut EditCtx) -> Result<Vec<ClipId>> {
let mut work = seq.clone();
let mut placed = Vec::new();
for (c, dest, start) in moves {
let (_, it) = work.find_item(*c).ok_or(EditError::NoItem(*c))?;
let mut it = it.clone();
if work.track(*dest).ok_or(EditError::NoTrack(*dest))?.locked {
return Err(EditError::Locked);
}
it.start = (*start).max(Tick::ZERO);
placed.push((*dest, it));
}
// worked out on the original ids, then moved over to the copies
let carried = carried_transitions(&work, &placed);
let mut ids = HashMap::new();
let mut links = HashMap::new();
for (_, it) in &mut placed {
let id = ClipId(ctx.alloc());
ids.insert(it.id, id);
it.id = id;
if let Some(l) = it.link {
it.link = Some(*links.entry(l).or_insert_with(|| ctx.alloc()));
}
}
// a copy whose linked partner is not copied is not linked
let mut counts: HashMap<u64, usize> = HashMap::new();
for l in placed.iter().filter_map(|(_, it)| it.link) {
*counts.entry(l).or_default() += 1;
}
for (_, it) in &mut placed {
if it.link.is_some_and(|l| counts.get(&l).copied().unwrap_or(0) < 2) {
it.link = None;
}
}
let out = if insert_mode { insert(&mut work, placed, ctx)? } else { overwrite(&mut work, placed, ctx)? };
for (dest, mut tr) in carried {
tr.id = TransitionId(ctx.alloc());
tr.from = tr.from.and_then(|c| ids.get(&c).copied());
tr.to = tr.to.and_then(|c| ids.get(&c).copied());
let t = track_mut(&mut work, dest)?;
if [tr.from, tr.to].into_iter().flatten().all(|c| t.item(c).is_some()) && (tr.from.is_some() || tr.to.is_some()) {
t.transitions.push(tr);
t.sort();
}
}
*seq = work;
Ok(out)
}
/// The transitions that travel with a move (#227, #374): those whose clips all move, onto one
/// track and by one offset (a fade at a moved clip's edge, or the cut between two clips that move
/// together), shifted to where those clips land.
fn carried_transitions(seq: &Sequence, placed: &[(TrackId, TrackItem)]) -> Vec<(TrackId, Transition)> {
let landing = |c: ClipId| {
let (dest, it) = placed.iter().find(|(_, it)| it.id == c)?;
let (_, old) = seq.find_item(c)?;
Some((*dest, it.start - old.start))
};
let mut carried = Vec::new();
for tr in seq.all_tracks().filter(|t| !t.locked).flat_map(|t| t.transitions.iter()) {
let lands: Option<Vec<(TrackId, Tick)>> = [tr.from, tr.to].into_iter().flatten().map(landing).collect();
let Some((&(dest, delta), rest)) = lands.as_deref().and_then(<[_]>::split_first) else { continue };
if rest.iter().all(|l| *l == (dest, delta)) {
carried.push((dest, Transition { start: tr.start + delta, ..tr.clone() }));
}
}
carried
}
fn media_len(ctx: &EditCtx, item: &TrackItem) -> Option<Tick> {
if item.frame_hold.is_some() {
return None;
@@ -522,15 +642,30 @@ pub fn clamp_trim(seq: &Sequence, clip: ClipId, edge: Edge, mode: TrimMode, delt
match edge {
Edge::In => {
// extending left (d<0) needs media before source_in; shortening needs min duration
let max_ext = Tick((head(ctx, it).0 as f64 / speed).floor() as i64);
// a reversed clip plays source_out down to source_in, so its In edge consumes media after source_out
let handle = match (it.reverse, media) {
(true, Some(m)) => (m - it.source_out()).max(Tick::ZERO),
(true, None) => Tick::MAX,
(false, _) => head(ctx, it),
};
let max_ext = if handle == Tick::MAX { Tick::MAX } else { Tick((handle.0 as f64 / speed).floor() as i64) };
let lo = if mode == TrimMode::Regular { (-(it.start - prev_end)).max(-max_ext) } else { -max_ext };
let hi = it.duration - ctx.min_duration;
d = d.clamp(if it.frame_hold.is_some() { Tick::MIN } else { lo }, hi);
// a frame hold needs no source head, but a Regular trim still stops at the previous clip
let lo = match (it.frame_hold.is_some(), mode) {
(true, TrimMode::Regular) => -(it.start - prev_end),
(true, _) => Tick::MIN,
(false, _) => lo,
};
d = d.clamp(lo, hi);
}
Edge::Out => {
let lo = -(it.duration - ctx.min_duration);
let mut hi = if mode == TrimMode::Regular { next_start - it.end() } else { Tick::MAX };
if let Some(m) = media {
if it.reverse && it.frame_hold.is_none() {
// a reversed clip's Out edge consumes media before source_in
hi = hi.min(Tick((head(ctx, it).0 as f64 / speed).floor() as i64));
} else if let Some(m) = media {
let remain = Tick(((m - it.source_out()).0 as f64 / speed).floor() as i64);
hi = hi.min(remain);
}
@@ -540,6 +675,31 @@ pub fn clamp_trim(seq: &Sequence, clip: ClipId, edge: Edge, mode: TrimMode, delt
Ok(d)
}
/// What moving one edge by `d` (already clamped) does to the item itself: its length, the media
/// it starts at (a reversed clip consumes media from its other end; a frame hold has none to
/// move), and, for the In edge with `move_start`, where it starts. The trims and the timeline's
/// live trim preview share it, so the preview shows the media the edit will keep (#374).
pub fn move_edge(it: &mut TrackItem, edge: Edge, d: Tick, move_start: bool) {
let speed = it.speed.abs();
match edge {
Edge::In => {
if !it.reverse && it.frame_hold.is_none() {
it.source_in += src_of(d, speed);
}
it.duration -= d;
if move_start {
it.start += d;
}
}
Edge::Out => {
if it.reverse {
it.source_in -= src_of(d, speed);
}
it.duration += d;
}
}
}
/// Trim one edge of an item (Selection-tool edge drag = Regular; Ripple tool = Ripple).
/// Linked partners should be trimmed by the caller with the same delta.
pub fn trim(seq: &mut Sequence, clip: ClipId, edge: Edge, mode: TrimMode, delta: Tick, ctx: &mut EditCtx) -> Result<Tick> {
@@ -551,27 +711,10 @@ pub fn trim(seq: &mut Sequence, clip: ClipId, edge: Edge, mode: TrimMode, delta:
let old_end = it.end();
let old_start = it.start;
let own_link = it.link;
let speed = it.speed.abs();
let mut work = seq.clone();
{
let (_, it) = work.find_item_mut(clip).ok_or(EditError::NoItem(clip))?;
match edge {
Edge::In => {
if !it.reverse && it.frame_hold.is_none() {
it.source_in += src_of(d, speed);
}
it.duration -= d;
if mode == TrimMode::Regular {
it.start += d;
}
}
Edge::Out => {
if it.reverse {
it.source_in -= src_of(d, speed);
}
it.duration += d;
}
}
move_edge(it, edge, d, mode == TrimMode::Regular);
}
if mode == TrimMode::Ripple {
let (at, shift) = match edge {
@@ -589,7 +732,7 @@ pub fn trim(seq: &mut Sequence, clip: ClipId, edge: Edge, mode: TrimMode, delta:
// (the clip's own linked partners are the caller's to trim or leave)
let in_the_way = tr.items.iter().any(|i| i.range().overlaps(&closing) && (own_link.is_none() || i.link != own_link));
if tr.id != tid && shift < Tick::ZERO && in_the_way {
return Err(EditError::SyncLockConflict);
return Err(EditError::SyncLockConflict(track_label(seq, tr.id)));
}
let from = if edge == Edge::In { old_start + Tick(1) } else { old_end };
for i in &mut tr.items {
@@ -601,6 +744,8 @@ pub fn trim(seq: &mut Sequence, clip: ClipId, edge: Edge, mode: TrimMode, delta:
}
}
transitions_follow_cuts(seq, &mut work);
} else if let Ok(track) = track_mut(&mut work, tid) {
fades_follow_edges(track, clip);
}
work.check().map_err(EditError::Other)?;
*seq = work;
@@ -629,25 +774,11 @@ pub fn ripple_trim_group(seq: &mut Sequence, clips: &[ClipId], edge: Edge, delta
let mut origins: Vec<(TrackId, Tick)> = Vec::new();
for c in clips {
let (tid, it) = work.find_item_mut(*c).ok_or(EditError::NoItem(*c))?;
let speed = it.speed.abs();
let from = match edge {
Edge::In => it.start + Tick(1),
Edge::Out => it.end(),
};
match edge {
Edge::In => {
if !it.reverse && it.frame_hold.is_none() {
it.source_in += src_of(d, speed);
}
it.duration -= d;
}
Edge::Out => {
if it.reverse {
it.source_in -= src_of(d, speed);
}
it.duration += d;
}
}
move_edge(it, edge, d, false);
if !origins.iter().any(|(t, _)| *t == tid) {
origins.push((tid, from));
}
@@ -700,7 +831,7 @@ pub fn ripple_trim_group(seq: &mut Sequence, clips: &[ClipId], edge: Edge, delta
let closing = if edge == Edge::In { TimeRange::new(from - Tick(1), -shift) } else { TimeRange::new(from + shift, -shift) };
let own = |i: &TrackItem| i.link.is_some_and(|l| own_links.contains(&l));
if tr.items.iter().any(|i| i.range().overlaps(&closing) && !follows(i) && !own(i) && !stays(i)) {
return Err(EditError::SyncLockConflict);
return Err(EditError::SyncLockConflict(track_label(seq, tid)));
}
}
let followers: Vec<ClipId> = tr.items.iter().filter(|i| !clips.contains(&i.id) && follows(i)).map(|i| i.id).collect();
@@ -719,7 +850,7 @@ pub fn ripple_trim_group(seq: &mut Sequence, clips: &[ClipId], edge: Edge, delta
});
// a follower with no room (a clip in its way, or the sequence start) blocks the edit
if tr.items.first().is_some_and(|i| i.start < Tick::ZERO) || tr.items.windows(2).any(|w| w[0].end() > w[1].start) {
return Err(EditError::SyncLockConflict);
return Err(EditError::SyncLockConflict(track_label(seq, tr.id)));
}
}
transitions_follow_cuts(seq, &mut work);
@@ -738,10 +869,19 @@ pub fn roll(seq: &mut Sequence, left: ClipId, right: ClipId, delta: Tick, ctx: &
let mut d = delta;
// left out-point limits
d = d.max(-(l.duration - ctx.min_duration)).min(r.duration - ctx.min_duration);
if let Some(m) = media_len(ctx, l) {
// a reversed clip consumes media from the opposite end, so its media limit is the other one
if l.reverse {
d = d.min(Tick((head(ctx, l).0 as f64 / l.speed.abs()).floor() as i64));
} else if let Some(m) = media_len(ctx, l) {
d = d.min(Tick(((m - l.source_out()).0 as f64 / l.speed.abs()).floor() as i64));
}
d = d.max(-Tick((head(ctx, r).0 as f64 / r.speed.abs()).floor() as i64));
if r.reverse {
if let Some(m) = media_len(ctx, r) {
d = d.max(-Tick(((m - r.source_out()).0 as f64 / r.speed.abs()).floor() as i64));
}
} else {
d = d.max(-Tick((head(ctx, r).0 as f64 / r.speed.abs()).floor() as i64));
}
if d == Tick::ZERO {
return Ok(d);
}
@@ -749,15 +889,19 @@ pub fn roll(seq: &mut Sequence, left: ClipId, right: ClipId, delta: Tick, ctx: &
let before = seq.clone();
{
let (_, l) = seq.find_item_mut(left).ok_or(EditError::NoItem(left))?;
if l.reverse {
l.source_in -= src_of(d, ls);
}
l.duration += d;
}
{
let (_, r) = seq.find_item_mut(right).ok_or(EditError::NoItem(right))?;
r.start += d;
r.duration -= d;
r.source_in += src_of(d, rs);
if !r.reverse {
r.source_in += src_of(d, rs);
}
}
let _ = ls;
transitions_follow_cuts(&before, seq);
Ok(d)
}
@@ -786,7 +930,10 @@ pub fn slide(seq: &mut Sequence, clip: ClipId, delta: Tick, ctx: &mut EditCtx) -
let mut d = delta;
if let Some(p) = &prev {
d = d.max(-(p.duration - ctx.min_duration));
if let Some(m) = media_len(ctx, p) {
if p.reverse {
// a reversed neighbour grows by pulling earlier media in front of its source In
d = d.min(Tick((head(ctx, p).0 as f64 / p.speed.abs()).floor() as i64));
} else if let Some(m) = media_len(ctx, p) {
d = d.min(Tick(((m - p.source_out()).0 as f64 / p.speed.abs()).floor() as i64));
}
} else {
@@ -794,7 +941,14 @@ pub fn slide(seq: &mut Sequence, clip: ClipId, delta: Tick, ctx: &mut EditCtx) -
}
if let Some(n) = &next {
d = d.min(n.duration - ctx.min_duration);
d = d.max(-Tick((head(ctx, n).0 as f64 / n.speed.abs()).floor() as i64));
if n.reverse {
// a reversed neighbour grows leftwards by pulling later media behind its source Out
if let Some(m) = media_len(ctx, n) {
d = d.max(-Tick(((m - n.source_out()).0 as f64 / n.speed.abs()).floor() as i64));
}
} else {
d = d.max(-Tick((head(ctx, n).0 as f64 / n.speed.abs()).floor() as i64));
}
} else {
d = d.min(neighbours(tr, clip).1 - it.end());
}
@@ -806,12 +960,17 @@ pub fn slide(seq: &mut Sequence, clip: ClipId, delta: Tick, ctx: &mut EditCtx) -
if let Some(p) = prev {
let pi = t.item_mut(p.id).ok_or(EditError::NoItem(p.id))?;
pi.duration += d;
if p.reverse {
pi.source_in -= src_of(d, p.speed);
}
}
if let Some(n) = next {
let ni = t.item_mut(n.id).ok_or(EditError::NoItem(n.id))?;
ni.start += d;
ni.duration -= d;
ni.source_in += src_of(d, n.speed);
if !n.reverse {
ni.source_in += src_of(d, n.speed);
}
}
let me = t.item_mut(clip).ok_or(EditError::NoItem(clip))?;
me.start += d;
@@ -819,6 +978,27 @@ pub fn slide(seq: &mut Sequence, clip: ClipId, delta: Tick, ctx: &mut EditCtx) -
Ok(d)
}
/// Slide several items (a linked picture/sound pair, a selection) by one common delta: the most
/// restrictive item's neighbour limit clamps them all, so their relative timing is kept.
/// Returns the delta applied.
pub fn slide_items(seq: &mut Sequence, clips: &[ClipId], delta: Tick, ctx: &mut EditCtx) -> Result<Tick> {
let mut d = delta;
for c in clips {
let x = slide(&mut seq.clone(), *c, d, ctx)?;
if x.0.signum() != d.0.signum() {
d = Tick::ZERO;
} else if x.abs() < d.abs() {
d = x;
}
}
if d != Tick::ZERO {
for c in clips {
slide(seq, *c, d, ctx)?;
}
}
Ok(d)
}
/// Rate stretch: change an item's duration by dragging an edge, adjusting speed to keep the same media.
pub fn rate_stretch(seq: &mut Sequence, clip: ClipId, edge: Edge, delta: Tick, ctx: &mut EditCtx) -> Result<f64> {
let (tid, it) = seq.find_item(clip).ok_or(EditError::NoItem(clip))?;
@@ -908,6 +1088,7 @@ pub fn set_speed_group(seq: &mut Sequence, clips: &[ClipId], speed: f64, reverse
if ripple {
transitions_follow_cuts(seq, &mut work);
}
work.check().map_err(EditError::Other)?;
*seq = work;
Ok(())
}
+184 -9
View File
@@ -60,6 +60,7 @@ impl Fx {
hold_filters: false,
field_options: None,
source_channels: Vec::new(),
audio_stream: 0,
graphic: None,
}
}
@@ -144,7 +145,7 @@ fn ripple_delete_and_conflict() {
// audio under the gap blocks the ripple
let b = fx.put(v1, 20, 10, 0);
fx.put(a1, 22, 3, 0);
assert_eq!(ripple_delete_items(&mut fx.seq, &[b]), Err(EditError::SyncLockConflict));
assert_eq!(ripple_delete_items(&mut fx.seq, &[b]), Err(EditError::SyncLockConflict("A1".into())), "names the track in the way");
assert_eq!(fx.spans(v1), vec![(0, 10), (20, 10)], "unchanged on failure");
}
@@ -233,6 +234,27 @@ fn close_gap_works() {
assert_eq!(fx.spans(v1), vec![(0, 10), (10, 10)]);
}
#[test]
fn reversed_trim_uses_the_handle_its_playback_direction_consumes() {
let mut fx = Fx::new();
let v1 = fx.v(0);
// source frames [5, 15) played backwards; the media is 1000 frames long
let a = fx.put(v1, 30, 10, 5);
fx.seq.find_item_mut(a).unwrap().1.reverse = true;
let mut n = fx.next;
// out extends into the 5 frames before source_in, not into the 985 after source_out
let d = trim(&mut fx.seq, a, Edge::Out, TrimMode::Regular, f(20), &mut Fx::ctx(&mut n)).unwrap();
assert_eq!(d, f(5));
let it = fx.seq.find_item(a).unwrap().1;
assert_eq!((it.source_in, it.source_out()), (f(0), f(15)));
// in extends into the media after source_out
let d = trim(&mut fx.seq, a, Edge::In, TrimMode::Regular, -f(10), &mut Fx::ctx(&mut n)).unwrap();
assert_eq!(d, -f(10));
let it = fx.seq.find_item(a).unwrap().1;
assert_eq!((it.source_in, it.source_out()), (f(0), f(25)));
fx.seq.check().unwrap();
}
#[test]
fn regular_trim_respects_neighbours_and_handles() {
let mut fx = Fx::new();
@@ -252,6 +274,24 @@ fn regular_trim_respects_neighbours_and_handles() {
fx.seq.check().unwrap();
}
#[test]
fn held_regular_in_trim_stops_at_previous_clip() {
let mut fx = Fx::new();
let v1 = fx.v(0);
fx.put(v1, 0, 10, 0);
let b = fx.put(v1, 15, 10, 5);
fx.seq.find_item_mut(b).unwrap().1.frame_hold = Some(f(5));
let mut n = fx.next;
// 5 frames of gap: a 20 frame extension is clamped to 5 instead of overlapping the previous clip
let d = trim(&mut fx.seq, b, Edge::In, TrimMode::Regular, -f(20), &mut Fx::ctx(&mut n)).unwrap();
assert_eq!(d, -f(5));
assert_eq!(fx.spans(v1), vec![(0, 10), (10, 15)]);
// no space left: nothing to do, and no error
let d = trim(&mut fx.seq, b, Edge::In, TrimMode::Regular, -f(3), &mut Fx::ctx(&mut n)).unwrap();
assert_eq!(d, Tick::ZERO);
fx.seq.check().unwrap();
}
#[test]
fn ripple_trim_shifts_following() {
let mut fx = Fx::new();
@@ -284,10 +324,10 @@ fn ripple_trim_group_takes_a_split_edit_along() {
ripple_trim_group(&mut fx.seq, &[a], Edge::Out, -f(5), &mut Fx::ctx(&mut n)).unwrap();
assert_eq!((fx.spans(v1), fx.spans(a1)), (vec![(0, 8), (8, 10)], vec![(4, 14)]), "shorter: it follows too");
// the sound would start before the sequence does: refused, nothing changes
assert_eq!(ripple_trim_group(&mut fx.seq, &[a], Edge::Out, -f(5), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict));
assert!(matches!(ripple_trim_group(&mut fx.seq, &[a], Edge::Out, -f(5), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict(_))));
// another clip right before the sound leaves it no room either
fx.put(a1, 0, 4, 0);
assert_eq!(ripple_trim_group(&mut fx.seq, &[a], Edge::Out, -f(2), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict));
assert!(matches!(ripple_trim_group(&mut fx.seq, &[a], Edge::Out, -f(2), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict(_))));
assert_eq!((fx.spans(v1), fx.spans(a1)), (vec![(0, 8), (8, 10)], vec![(0, 4), (4, 14)]), "unchanged on failure");
// a linked clip that ends before the cut is not a split edit and stays where it is
let early = fx.put(fx.a(1), 0, 3, 0);
@@ -358,6 +398,25 @@ fn roll_slip_slide() {
fx.seq.check().unwrap();
}
#[test]
fn slide_keeps_reversed_neighbours_content() {
let mut fx = Fx::new();
let v1 = fx.v(0);
let a = fx.put(v1, 0, 30, 20);
let b = fx.put(v1, 30, 30, 50);
let c = fx.put(v1, 60, 30, 70);
for id in [a, c] {
fx.seq.find_item_mut(id).unwrap().1.reverse = true;
}
let mut n = fx.next;
slide(&mut fx.seq, b, f(10), &mut Fx::ctx(&mut n)).unwrap();
assert_eq!(fx.spans(v1), vec![(0, 40), (40, 30), (70, 20)]);
assert_eq!(fx.seq.find_item(a).unwrap().1.source_in, f(10));
assert_eq!(fx.seq.find_item(b).unwrap().1.source_in, f(50));
assert_eq!(fx.seq.find_item(c).unwrap().1.source_in, f(70));
fx.seq.check().unwrap();
}
#[test]
fn transitions_stay_on_their_cuts() {
let mut fx = Fx::new();
@@ -405,6 +464,67 @@ fn transitions_stay_on_their_cuts() {
fx.seq.check().unwrap();
}
#[test]
fn fades_follow_a_regular_trim() {
let mut fx = Fx::new();
let v1 = fx.v(0);
let a = fx.put(v1, 10, 20, 10);
let fade = |id, start, from, to| Transition {
id: TransitionId(id),
effect: filmcraft_project::find_effect("cross_dissolve").unwrap().instance(),
start: f(start),
duration: f(6),
from,
to,
align: Default::default(),
reverse: false,
};
fx.seq.track_mut(v1).unwrap().transitions = vec![fade(1, 10, None, Some(a)), fade(2, 24, Some(a), None)];
let spans = |fx: &Fx| fx.seq.track(v1).unwrap().transitions.iter().map(|t| (t.start, t.duration)).collect::<Vec<_>>();
let mut n = fx.next;
trim(&mut fx.seq, a, Edge::In, TrimMode::Regular, f(2), &mut Fx::ctx(&mut n)).unwrap();
trim(&mut fx.seq, a, Edge::Out, TrimMode::Regular, f(4), &mut Fx::ctx(&mut n)).unwrap();
// the fade in starts with the clip (12), the fade out ends with it (34)
assert_eq!(spans(&fx), vec![(f(12), f(6)), (f(28), f(6))]);
// a clip trimmed shorter than its fades keeps them inside it
trim(&mut fx.seq, a, Edge::Out, TrimMode::Regular, -f(18), &mut Fx::ctx(&mut n)).unwrap();
assert_eq!(spans(&fx), vec![(f(12), f(4)), (f(12), f(4))]);
fx.seq.check().unwrap();
}
#[test]
fn transitions_travel_with_moved_clips() {
// V1: a [0,10) fading in, b [10,20), c [20,30); b→c crossfade, c fading out
let mut fx = Fx::new();
let (v1, v2) = (fx.v(0), fx.v(1));
let a = fx.put(v1, 0, 10, 10);
let b = fx.put(v1, 10, 10, 10);
let c = fx.put(v1, 20, 10, 10);
let tr = |id, start, from, to| Transition {
id: TransitionId(id),
effect: filmcraft_project::find_effect("cross_dissolve").unwrap().instance(),
start: f(start),
duration: f(4),
from,
to,
align: Default::default(),
reverse: false,
};
fx.seq.track_mut(v1).unwrap().transitions = vec![tr(1, 0, None, Some(a)), tr(2, 18, Some(b), Some(c)), tr(3, 26, Some(c), None)];
let at = |fx: &Fx, track: TrackId| -> Vec<(u64, i64)> { fx.seq.track(track).unwrap().transitions.iter().map(|t| (t.id.0, R.frame_at(t.start))).collect() };
let mut n = fx.next;
// a fade at a clip's edge goes with it, to another track too
move_items(&mut fx.seq, &[(a, v2, f(40))], false, &mut Fx::ctx(&mut n)).unwrap();
assert_eq!((at(&fx, v1), at(&fx, v2)), (vec![(2, 18), (3, 26)], vec![(1, 40)]));
// a cut whose two clips move together keeps its transition, in an insert move as well
move_items(&mut fx.seq, &[(b, v1, f(60)), (c, v1, f(70))], true, &mut Fx::ctx(&mut n)).unwrap();
assert_eq!(at(&fx, v1), vec![(2, 68), (3, 76)]);
// a crossfade whose partner stays behind is dropped; the other clip's fade stays with it
move_items(&mut fx.seq, &[(c, v1, f(90))], false, &mut Fx::ctx(&mut n)).unwrap();
assert_eq!(at(&fx, v1), vec![(3, 96)]);
fx.seq.check().unwrap();
}
#[test]
fn rate_stretch_and_speed() {
let mut fx = Fx::new();
@@ -592,7 +712,7 @@ fn shorter_head_into_an_l_cut_keeps_the_earlier_sound_where_it_is() {
// later material on A1 moves up with the pictures; landing on the L cut's sound refuses
let later = fx.put(a1, 15, 5, 0);
assert_eq!(ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(3), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict));
assert!(matches!(ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(3), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict(_))));
assert_eq!(fx.spans(a1), vec![(0, 13), (15, 5)], "unchanged on failure");
fx.seq.find_item_mut(later).unwrap().1.start = f(20);
ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(2), &mut Fx::ctx(&mut n)).unwrap();
@@ -605,10 +725,10 @@ fn shorter_head_into_an_l_cut_keeps_the_earlier_sound_where_it_is() {
let b = fx.put(v1, 10, 10, 0);
fx.put(a1, 0, 13, 0);
let mut n = fx.next;
assert_eq!(ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(3), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict));
assert!(matches!(ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(3), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict(_))));
// so does a link with no partner left
fx.seq.track_mut(a1).unwrap().items[0].link = Some(9);
assert_eq!(ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(3), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict));
assert!(matches!(ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(3), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict(_))));
// a linked cutaway across the cut whose own partner reaches past it too is no L cut: it refuses
let mut fx = Fx::new();
@@ -622,7 +742,7 @@ fn shorter_head_into_an_l_cut_keeps_the_earlier_sound_where_it_is() {
fx.seq.find_item_mut(c).unwrap().1.link = Some(7);
}
let mut n = fx.next;
assert_eq!(ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(3), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict));
assert!(matches!(ripple_trim_group(&mut fx.seq, &[b], Edge::In, f(3), &mut Fx::ctx(&mut n)), Err(EditError::SyncLockConflict(_))));
}
#[test]
@@ -651,7 +771,7 @@ fn shorter_head_closes_the_stretch_after_the_cut_on_sync_locked_tracks() {
} else {
trim(&mut fx.seq, b, Edge::In, TrimMode::Ripple, f(2), &mut Fx::ctx(&mut n))
};
assert_eq!(r, Err(EditError::SyncLockConflict), "group {group}");
assert!(matches!(r, Err(EditError::SyncLockConflict(_))), "group {group}");
}
// a clip on a sync-locked track that starts at the cut stays put while the pictures after it move
// up: only the stretch after the cut sees it (the old stretch, before the cut, let it through)
@@ -668,7 +788,62 @@ fn shorter_head_closes_the_stretch_after_the_cut_on_sync_locked_tracks() {
} else {
trim(&mut fx.seq, b, Edge::In, TrimMode::Ripple, f(3), &mut Fx::ctx(&mut n))
};
assert_eq!(r, Err(EditError::SyncLockConflict), "group {group}");
assert!(matches!(r, Err(EditError::SyncLockConflict(_))), "group {group}");
assert_eq!(fx.spans(v1), vec![(0, 10), (10, 10), (20, 10)], "unchanged on failure");
}
}
#[test]
fn roll_reversed_clips_keeps_media_mapping() {
let mut fx = Fx::new();
let v = fx.v(0);
let mut a = fx.item(0, 30, 20);
a.reverse = true;
let mut b = fx.item(30, 30, 50);
b.reverse = true;
let (ia, ib) = (a.id, b.id);
fx.seq.track_mut(v).unwrap().items.extend([a, b]);
let mut n = 0;
roll(&mut fx.seq, ia, ib, f(10), &mut Fx::ctx(&mut n)).unwrap();
let (_, l) = fx.seq.find_item(ia).unwrap();
assert_eq!((l.start, l.duration, l.source_in), (f(0), f(40), f(10)));
let (_, r) = fx.seq.find_item(ib).unwrap();
assert_eq!((r.start, r.duration, r.source_in), (f(40), f(20), f(50)));
}
/// #374: moving an edge must not change which media sits under any timeline time the clip still
/// covers (the timeline's live trim preview draws its waveform from this item), and must leave the
/// item as the trim itself does.
#[test]
fn moving_an_edge_keeps_the_media_under_the_timeline() {
let fx = Fx::new();
let v1 = fx.v(0);
for (speed, reverse, hold) in [(1.0, false, None), (2.0, false, None), (0.5, false, None), (1.0, true, None), (2.0, true, None), (1.0, false, Some(f(50)))]
{
for (edge, d) in [(Edge::In, f(7)), (Edge::In, -f(4)), (Edge::Out, -f(9)), (Edge::Out, f(3))] {
let mut fresh = Fx::new();
let c = fresh.put(v1, 100, 40, 200);
{
let it = fresh.seq.find_item_mut(c).unwrap().1;
(it.speed, it.reverse, it.frame_hold) = (speed, reverse, hold);
}
let before = fresh.seq.find_item(c).unwrap().1.clone();
let mut moved = before.clone();
move_edge(&mut moved, edge, d, true);
let label = format!("speed {speed} reverse {reverse} hold {hold:?} {edge:?} {d:?}");
// every frame still covered shows the same media as before the move
let mut t = moved.start.max(before.start);
while t < moved.end().min(before.end()) {
let (a, b) = (before.source_time_at(t), moved.source_time_at(t));
assert!((a - b).0.abs() <= 1, "{label}: at {t:?} {a:?} became {b:?}");
t += f(1);
}
// and the item is what the real (unclamped) trim makes of it
let mut n = fresh.next;
let got = trim(&mut fresh.seq, c, edge, TrimMode::Regular, d, &mut Fx::ctx(&mut n)).unwrap();
assert_eq!(got, d, "{label}: not clamped");
let trimmed = fresh.seq.find_item(c).unwrap().1;
assert_eq!((trimmed.start, trimmed.duration, trimmed.source_in), (moved.start, moved.duration, moved.source_in), "{label}");
}
}
}
+7 -2
View File
@@ -59,6 +59,11 @@ pub fn through_edits(seq: &Sequence) -> Vec<ThroughEdit> {
/// piece is listed. The left piece absorbs the right one (keeping its own effects and attributes);
/// a transition sitting on the cut is removed. Returns the number of edits joined.
pub fn join_through_edits(seq: &mut Sequence, only: &[ClipId]) -> usize {
join_through_edits_where(seq, |a, b| only.is_empty() || only.contains(&a) || only.contains(&b))
}
/// Join the through edits on unlocked tracks whose (left, right) pieces `wanted` accepts.
pub fn join_through_edits_where(seq: &mut Sequence, wanted: impl Fn(ClipId, ClipId) -> bool) -> usize {
let mut n = 0;
for tr in seq.all_tracks_mut() {
if tr.locked {
@@ -67,8 +72,7 @@ pub fn join_through_edits(seq: &mut Sequence, only: &[ClipId]) -> usize {
let mut i = 0;
while i + 1 < tr.items.len() {
let (a, b) = (&tr.items[i], &tr.items[i + 1]);
let wanted = only.is_empty() || only.contains(&a.id) || only.contains(&b.id);
if !(wanted && is_through_edit(a, b)) {
if !(wanted(a.id, b.id) && is_through_edit(a, b)) {
i += 1;
continue;
}
@@ -180,6 +184,7 @@ mod tests {
hold_filters: false,
field_options: None,
source_channels: Vec::new(),
audio_stream: 0,
graphic: None,
}
}
+1
View File
@@ -57,6 +57,7 @@ fn clip(id: u64, start: f64, dur: f64, src_in: f64) -> TrackItem {
hold_filters: false,
field_options: None,
source_channels: Vec::new(),
audio_stream: 0,
graphic: None,
}
}
+401
View File
@@ -0,0 +1,401 @@
//! Applied transitions: where one may sit over its cut, its alignment, and moving / resizing /
//! removing one (Effect Controls' Duration and Alignment, dragging its edges or its middle in the
//! Timeline, Delete).
//!
//! Premiere (Align transitions; Change transition duration): a transition between two clips stays
//! over their cut, "Center at Cut", "Start at Cut", "End at Cut" or, once dragged off-centre,
//! "Custom Start". Changing its duration moves both ends equally for Center at Cut / Custom Start,
//! only the end for Start at Cut and only the beginning for End at Cut. A one-sided transition (a
//! fade from or to nothing) starts or ends at its clip's edge.
//!
//! Premiere limits a transition by the media its clips have beyond the cut (handles). FilmCraft
//! does not check handles when applying one, so here a transition may cover at most the clips it
//! joins: from the outgoing clip's start to the incoming clip's end.
use filmcraft_project::{Sequence, Track, Transition, TransitionAlign, TransitionId};
use filmcraft_time::Tick;
use crate::{Edge, EditError, Result};
/// How a transition sits on its cut, as Effect Controls' Alignment menu shows it.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Alignment {
Center,
Start,
End,
/// Premiere's "Custom Start": dragged to a place over the cut that is none of the others.
Custom,
}
impl Alignment {
pub const MENU: [Alignment; 3] = [Alignment::Center, Alignment::Start, Alignment::End];
pub fn id(self) -> &'static str {
match self {
Alignment::Center => "center",
Alignment::Start => "start",
Alignment::End => "end",
Alignment::Custom => "custom",
}
}
pub fn label(self) -> &'static str {
match self {
Alignment::Center => "Center at Cut",
Alignment::Start => "Start at Cut",
Alignment::End => "End at Cut",
Alignment::Custom => "Custom Start",
}
}
/// `center`, `start` or `end` (Custom Start is reached by moving a transition, not chosen).
pub fn parse(s: &str) -> Option<Alignment> {
Alignment::MENU.into_iter().find(|a| a.id().eq_ignore_ascii_case(s) || a.label().eq_ignore_ascii_case(s))
}
}
/// Where a transition may be: `(earliest start, cut, latest end)`. Between two clips that is the
/// outgoing clip's start, the cut and the incoming clip's end; for a fade to or from nothing the cut
/// is its clip's edge. `None` when its clips are gone.
pub fn bounds(track: &Track, tr: &Transition) -> Option<(Tick, Tick, Tick)> {
let from = tr.from.and_then(|c| track.item(c));
let to = tr.to.and_then(|c| track.item(c));
match (from, to) {
(Some(f), Some(t)) => Some((f.start, t.start, t.end())),
(None, Some(t)) => Some((t.start, t.start, t.end())),
(Some(f), None) => Some((f.start, f.end(), f.end())),
(None, None) => None,
}
}
/// A transition's alignment on `cut`. Centred means within half a `frame` of the cut, as
/// centring an odd number of frames lands on a frame boundary.
pub fn alignment(tr: &Transition, cut: Tick, frame: Tick) -> Alignment {
match (tr.from, tr.to) {
(None, Some(_)) => return Alignment::Start,
(Some(_), None) => return Alignment::End,
_ => {}
}
if tr.start == cut {
Alignment::Start
} else if tr.end() == cut {
Alignment::End
} else if (tr.start.0.saturating_mul(2).saturating_add(tr.duration.0) - cut.0.saturating_mul(2)).abs() <= frame.0.max(0) {
Alignment::Center
} else {
Alignment::Custom
}
}
fn find(seq: &Sequence, id: TransitionId) -> Option<(&Track, &Transition)> {
seq.all_tracks().find_map(|t| t.transitions.iter().find(|x| x.id == id).map(|x| (t, x)))
}
/// Where a transition would start for `duration` and `align`, keeping its cut (and, for Center
/// at Cut / Custom Start, its centre; `snap` puts a centred start on a frame).
pub fn start_for(seq: &Sequence, id: TransitionId, duration: Tick, align: Alignment, frame: Tick, snap: impl Fn(Tick) -> Tick) -> Option<Tick> {
let (track, tr) = find(seq, id)?;
let (_, cut, _) = bounds(track, tr)?;
let one_sided = alignment(tr, cut, frame);
let align = match one_sided {
Alignment::Start | Alignment::End if tr.from.is_none() || tr.to.is_none() => one_sided,
_ => align,
};
Some(match align {
Alignment::Start => cut,
Alignment::End => cut - duration,
Alignment::Center => snap(cut - duration.mul_ratio(1, 2)),
// Custom Start: the ends move equally, around where it is now
Alignment::Custom => snap(tr.start + tr.duration.mul_ratio(1, 2) - duration.mul_ratio(1, 2)),
})
}
/// Move / resize a transition to cover `start .. start + duration`. Refused (leaving the sequence
/// as it was) when its track is locked, it would be shorter than `min`, it would leave its clips,
/// or it would no longer touch its cut (a one-sided one must keep its edge).
pub fn set_span(seq: &mut Sequence, id: TransitionId, start: Tick, duration: Tick, min: Tick, frame: Tick) -> Result<()> {
let (track, tr) = find(seq, id).ok_or_else(|| EditError::Other(format!("no transition {}", id.0)))?;
if track.locked {
return Err(EditError::Locked);
}
let (lo, cut, hi) = bounds(track, tr).ok_or_else(|| EditError::Other("the transition's clips are gone".into()))?;
if duration < min.max(Tick(1)) {
return Err(EditError::Other("a transition lasts at least one frame".into()));
}
let end = start.0.checked_add(duration.0).map(Tick).ok_or_else(|| EditError::Other("transition out of range".into()))?;
if start < lo || end > hi {
return Err(EditError::Other("a transition can't extend past the clips it joins".into()));
}
let on_cut = match (tr.from, tr.to) {
(None, Some(_)) => start == cut,
(Some(_), None) => end == cut,
_ => start <= cut && cut <= end,
};
if !on_cut {
return Err(EditError::Other(if tr.from.is_some() && tr.to.is_some() {
"a transition between two clips must stay over their cut".into()
} else {
"a one-sided transition stays on its clip's edge".into()
}));
}
let tid = track.id;
let Some(x) = seq.track_mut(tid).and_then(|t| t.transitions.iter_mut().find(|x| x.id == id)) else {
return Err(EditError::Other(format!("no transition {}", id.0)));
};
x.start = start;
x.duration = duration;
// the stored alignment (interchange): Custom Start has no variant, its span says it all
x.align = match alignment(x, cut, frame) {
Alignment::Start => TransitionAlign::StartAtCut,
Alignment::End => TransitionAlign::EndAtCut,
Alignment::Center | Alignment::Custom => TransitionAlign::CenterAtCut,
};
Ok(())
}
/// Where dragging a transition by `delta` in the Timeline puts it, as `(start, duration)`. Its
/// middle (`edge` None) slides it over the cut. An end trims that end only (Premiere shows the
/// Trim-In / Trim-Out icon there): the other end stays, whatever the alignment, so a Center at
/// Cut transition becomes Custom Start. (A typed duration, in Effect Controls or Set Transition
/// Duration, follows the alignment instead: see [`start_for`].) A fade to or from nothing keeps
/// its clip-edge end, and does not slide. Always clamped to where it may be ([`bounds`], over the
/// cut, at least one `frame`). `None` when its clips are gone.
pub fn drag_span(track: &Track, tr: &Transition, edge: Option<Edge>, delta: Tick, frame: Tick) -> Option<(Tick, Tick)> {
let (lo, cut, hi) = bounds(track, tr)?;
let frame = frame.max(Tick(1));
let (s0, e0, d0) = (tr.start, tr.end(), tr.duration);
let one_sided = tr.from.is_none() || tr.to.is_none();
let unchanged = Some((s0, d0));
match edge {
None if one_sided => unchanged,
None => {
let (min_s, max_s) = (lo.max(cut - d0), (hi - d0).min(cut));
if min_s > max_s { unchanged } else { Some(((s0 + delta).clamp(min_s, max_s), d0)) }
}
Some(Edge::In) if tr.from.is_none() => unchanged,
Some(Edge::In) => {
let max_s = cut.min(e0 - frame).max(lo);
let st = (s0 + delta).clamp(lo, max_s);
Some((st, e0 - st))
}
Some(Edge::Out) if tr.to.is_none() => unchanged,
Some(Edge::Out) => {
let min_e = cut.max(s0 + frame).min(hi);
let en = (e0 + delta).clamp(min_e, hi);
Some((s0, en - s0))
}
}
}
/// Remove transitions (on unlocked tracks). Returns how many went.
pub fn remove(seq: &mut Sequence, ids: &[TransitionId]) -> Result<usize> {
if seq.all_tracks().any(|t| t.locked && t.transitions.iter().any(|x| ids.contains(&x.id))) {
return Err(EditError::Locked);
}
let mut n = 0;
for t in seq.all_tracks_mut() {
let before = t.transitions.len();
t.transitions.retain(|x| !ids.contains(&x.id));
n += before - t.transitions.len();
}
if n == 0 { Err(EditError::Nothing) } else { Ok(n) }
}
#[cfg(test)]
mod tests {
use super::*;
use filmcraft_project::{ClipId, ItemId, Label, Project, SequenceSettings, TrackItem, find_effect};
use filmcraft_time::FrameRate;
const R: FrameRate = FrameRate::FPS_24;
fn f(n: i64) -> Tick {
R.tick_of(n)
}
fn clip(id: u64, start: i64, dur: i64) -> TrackItem {
TrackItem {
id: ClipId(id),
item: ItemId(1),
name: format!("c{id}"),
label: Label::Iris,
start: f(start),
duration: f(dur),
source_in: Tick::ZERO,
speed: 1.0,
reverse: false,
enabled: true,
link: None,
group: None,
effects: vec![],
markers: vec![],
gain_db: 0.0,
frame_hold: None,
scale_to_frame: false,
essential: None,
multicam: None,
time_interpolation: Default::default(),
hold_filters: false,
field_options: None,
source_channels: Vec::new(),
audio_stream: 0,
graphic: None,
}
}
fn transition(id: u64, start: i64, dur: i64, from: Option<u64>, to: Option<u64>) -> Transition {
// any effect: these tests are about where the transition sits
let effect = find_effect("opacity").unwrap().instance();
Transition {
id: TransitionId(id),
effect,
start: f(start),
duration: f(dur),
from: from.map(ClipId),
to: to.map(ClipId),
align: TransitionAlign::CenterAtCut,
reverse: false,
}
}
/// V1: c1 0..48, c2 48..96, c3 96..144; a centred 24-frame transition on 48 (id 1), a fade in
/// at 0 (id 2) and a fade out at 144 (id 3).
fn seq() -> Sequence {
let mut p = Project::new("t");
let s = p.new_sequence("s", SequenceSettings { frame_rate: R, ..Default::default() }, 1, 0, None);
let mut q = p.sequence(s).unwrap().clone();
let t = &mut q.video_tracks[0];
t.items = vec![clip(1, 0, 48), clip(2, 48, 48), clip(3, 96, 48)];
t.transitions = vec![transition(1, 36, 24, Some(1), Some(2)), transition(2, 0, 12, None, Some(1)), transition(3, 132, 12, Some(3), None)];
q
}
fn span(q: &Sequence, id: u64) -> (Tick, Tick) {
let x = q.video_tracks[0].transitions.iter().find(|x| x.id.0 == id).unwrap();
(x.start, x.duration)
}
fn align(q: &Sequence, id: u64) -> Alignment {
let (t, x) = find(q, TransitionId(id)).unwrap();
let (_, cut, _) = bounds(t, x).unwrap();
alignment(x, cut, f(1))
}
#[test]
fn alignment_reads_the_span() {
let mut q = seq();
assert_eq!((align(&q, 1), align(&q, 2), align(&q, 3)), (Alignment::Center, Alignment::Start, Alignment::End));
set_span(&mut q, TransitionId(1), f(48), f(24), f(1), f(1)).unwrap();
assert_eq!(align(&q, 1), Alignment::Start);
set_span(&mut q, TransitionId(1), f(24), f(24), f(1), f(1)).unwrap();
assert_eq!(align(&q, 1), Alignment::End);
set_span(&mut q, TransitionId(1), f(40), f(24), f(1), f(1)).unwrap();
assert_eq!(align(&q, 1), Alignment::Custom);
// an odd centred duration sits half a frame off the cut and is still Center at Cut
set_span(&mut q, TransitionId(1), f(42), f(13), f(1), f(1)).unwrap();
assert_eq!(align(&q, 1), Alignment::Center);
}
#[test]
fn duration_changes_follow_the_alignment() {
let q = seq();
let snap = |t: Tick| R.snap(t);
// Center at Cut: both ends move
assert_eq!(start_for(&q, TransitionId(1), f(36), Alignment::Center, f(1), snap), Some(f(30)));
// Start at Cut: only the end moves; End at Cut: only the beginning
assert_eq!(start_for(&q, TransitionId(1), f(36), Alignment::Start, f(1), snap), Some(f(48)));
assert_eq!(start_for(&q, TransitionId(1), f(36), Alignment::End, f(1), snap), Some(f(12)));
// a one-sided transition keeps its edge whatever is asked
assert_eq!(start_for(&q, TransitionId(2), f(20), Alignment::Center, f(1), snap), Some(f(0)));
assert_eq!(start_for(&q, TransitionId(3), f(20), Alignment::Center, f(1), snap), Some(f(124)));
}
#[test]
fn custom_start_resizes_around_its_middle() {
let mut q = seq();
set_span(&mut q, TransitionId(1), f(40), f(20), f(1), f(1)).unwrap();
assert_eq!(start_for(&q, TransitionId(1), f(10), Alignment::Custom, f(1), |t| R.snap(t)), Some(f(45)));
}
#[test]
fn spans_stay_on_the_cut_and_inside_the_clips() {
let mut q = seq();
let before = q.clone();
for (id, start, dur) in [
(1, -1, 24), // before the outgoing clip
(1, 40, 60), // past the incoming clip's end
(1, 10, 20), // entirely before the cut
(1, 50, 10), // entirely after it
(1, 40, 0), // empty
(2, 1, 12), // a fade in must start at its clip's start
(3, 120, 12), // a fade out must end at its clip's end
(9, 0, 12), // no such transition
] {
assert!(set_span(&mut q, TransitionId(id), f(start), f(dur), f(1), f(1)).is_err(), "{id} {start} {dur}");
assert_eq!(q, before, "refused changes leave the sequence as it was");
}
// the whole of both clips is allowed
set_span(&mut q, TransitionId(1), f(0), f(96), f(1), f(1)).unwrap();
assert_eq!(span(&q, 1), (f(0), f(96)));
// Tick::MAX never overflows
assert!(set_span(&mut q, TransitionId(1), Tick(i64::MAX - 1), Tick(i64::MAX), f(1), f(1)).is_err());
}
fn drag(q: &Sequence, id: u64, edge: Option<Edge>, frames: i64) -> (i64, i64) {
let (t, x) = find(q, TransitionId(id)).unwrap();
let (st, d) = drag_span(t, x, edge, f(frames), f(1)).unwrap();
(R.frame_at(st), R.frame_at(d))
}
#[test]
fn dragging_a_transition() {
let mut q = seq();
// centred (36..60 over the cut at 48): an end trims that end only, the other stays
assert_eq!(drag(&q, 1, Some(Edge::In), -4), (32, 28));
assert_eq!(drag(&q, 1, Some(Edge::Out), -10), (36, 14));
assert_eq!(drag(&q, 1, Some(Edge::Out), -100), (36, 12), "the end stops at the cut");
assert_eq!(drag(&q, 1, Some(Edge::Out), 100), (36, 60), "and at the incoming clip's end");
assert_eq!(drag(&q, 1, Some(Edge::In), -100), (0, 60), "the start at the outgoing clip's start");
// the middle slides it, staying over the cut and inside the clips
assert_eq!(drag(&q, 1, None, 5), (41, 24));
assert_eq!(drag(&q, 1, None, 100), (48, 24));
assert_eq!(drag(&q, 1, None, -100), (24, 24));
// not centred: the other end stays
set_span(&mut q, TransitionId(1), f(40), f(20), f(1), f(1)).unwrap();
assert_eq!(drag(&q, 1, Some(Edge::In), -4), (36, 24));
assert_eq!(drag(&q, 1, Some(Edge::In), 100), (48, 12), "the start stops at the cut");
assert_eq!(drag(&q, 1, Some(Edge::Out), 6), (40, 26));
assert_eq!(drag(&q, 1, Some(Edge::Out), -100), (40, 8), "the end stops at the cut");
// a fade in keeps its start, a fade out its end; neither slides
assert_eq!(drag(&q, 2, Some(Edge::In), 3), (0, 12));
assert_eq!(drag(&q, 2, Some(Edge::Out), 3), (0, 15));
assert_eq!(drag(&q, 2, None, 3), (0, 12));
assert_eq!(drag(&q, 3, Some(Edge::Out), 3), (132, 12));
assert_eq!(drag(&q, 3, Some(Edge::In), -3), (129, 15));
// whatever comes out is a span set_span accepts
for (id, edge, n) in [(1, Some(Edge::In), -70), (1, None, 33), (2, Some(Edge::Out), 99), (3, Some(Edge::In), -99)] {
let (t, x) = find(&q, TransitionId(id)).unwrap();
let (st, d) = drag_span(t, x, edge, f(n), f(1)).unwrap();
let mut c = q.clone();
set_span(&mut c, TransitionId(id), st, d, f(1), f(1)).unwrap();
}
}
#[test]
fn locked_tracks_refuse() {
let mut q = seq();
q.video_tracks[0].locked = true;
assert_eq!(set_span(&mut q, TransitionId(1), f(40), f(20), f(1), f(1)), Err(EditError::Locked));
assert_eq!(remove(&mut q, &[TransitionId(1)]), Err(EditError::Locked));
}
#[test]
fn remove_takes_only_the_named_transitions() {
let mut q = seq();
assert_eq!(remove(&mut q, &[TransitionId(1), TransitionId(3)]), Ok(2));
let left: Vec<u64> = q.video_tracks[0].transitions.iter().map(|x| x.id.0).collect();
assert_eq!(left, vec![2]);
assert_eq!(q.video_tracks[0].items.len(), 3, "the clips stay");
assert_eq!(remove(&mut q, &[TransitionId(1)]), Err(EditError::Nothing));
}
}
+3
View File
@@ -15,6 +15,8 @@ default = []
whisper = ["filmcraft-speech/whisper"]
# Speech-model downloads over HTTPS.
speech-download = ["filmcraft-speech/download"]
# Natural voices for Text to Speech: Kokoro-82M on candle (CPU) and their download.
neural-voices = ["filmcraft-tts/kokoro", "speech-download"]
[dependencies]
filmcraft-time = { workspace = true }
@@ -35,6 +37,7 @@ filmcraft-format = { workspace = true }
filmcraft-interchange = { workspace = true }
filmcraft-captions = { workspace = true }
filmcraft-speech = { workspace = true }
filmcraft-tts = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
thiserror = { workspace = true }
+14 -2
View File
@@ -143,7 +143,7 @@ fn decode(s: &Session, need: &AudioNeed, sample_rate: u32) -> Result<Vec<Vec<f32
let mut done = 0usize;
while done < frames {
let n = chunk.min(frames - done);
let b = src.audio(start + done as i64, n, sample_rate).map_err(|e| EngineError::Other(e.to_string()))?;
let b = src.audio_stream(need.audio_stream, start + done as i64, n, sample_rate).map_err(|e| EngineError::Other(e.to_string()))?;
for (c, o) in out.iter_mut().enumerate() {
match b.channels.get(c).or(b.channels.first()) {
Some(x) => o.extend(x.iter().copied().chain(std::iter::repeat(0.0)).take(n)),
@@ -230,8 +230,19 @@ fn ensure_dir(s: &Session, dir: &str) {
}
}
fn check_linked_streams(s: &Session, seq: ItemId, plan: &AudioPlan) -> Result<()> {
if plan.mode == AudioMode::Linked && !plan.trim && !plan.render_effects {
let needs = audio_needs(&s.project, seq, &NeedOptions { nests: plan.nests, ..Default::default() });
if needs.iter().any(|need| need.audio_stream > 0) {
return Err(EngineError::Other("linked AAF media cannot select container audio streams; choose embedded or separate audio".into()));
}
}
Ok(())
}
/// Prepare the audio essence of an export; returns the essence and the files written.
pub fn prepare_audio(s: &Session, seq: ItemId, plan: &AudioPlan, media_dir: &str) -> Result<(Vec<AudioEssence>, Vec<String>)> {
check_linked_streams(s, seq, plan)?;
if plan.mode == AudioMode::Linked && !plan.trim && !plan.render_effects {
return Ok((Vec::new(), Vec::new()));
}
@@ -255,7 +266,7 @@ pub fn prepare_audio(s: &Session, seq: ItemId, plan: &AudioPlan, media_dir: &str
let name = audio_clip(&s.project, seq, c).map(|(_, _, ti)| ti.name.clone()).unwrap_or_default();
format!("{} {}", safe_name(&name), c.0)
}
EssenceKey::Media(_) => safe_name(
EssenceKey::Media(_) | EssenceKey::MediaStream { .. } => safe_name(
need.path
.as_deref()
.map(|p| std::path::Path::new(p).file_stem().map(|x| x.to_string_lossy().into_owned()).unwrap_or_default())
@@ -380,6 +391,7 @@ pub fn export_aaf(s: &mut Session, p: &Value) -> Result<Value> {
let path = str_p(p, "path").ok_or_else(|| bad(cmd, "need `path`"))?.to_string();
let seq = crate::export_tools::sequence_param(s, p, cmd)?;
let plan = plan_from(s, p, seq, cmd, false)?;
check_linked_streams(s, seq, &plan)?;
let (dir, stem) = split(&path);
let media_dir = join(&dir, &format!("{stem} Media"));
let mut files = Vec::new();
+193 -4
View File
@@ -69,7 +69,7 @@ fn aaf_embedded_trimmed_audio_round_trips_sample_exact() {
assert_eq!(a, b, "{kind:?}");
}
// video links to the original movies, audio to WAV files extracted next to the AAF
assert_eq!(path_of(&s, q.video_tracks[0].items[0].item), path_of(&s, items[0]));
assert_eq!(std::path::Path::new(&path_of(&s, q.video_tracks[0].items[0].item)), std::path::Path::new(&path_of(&s, items[0])));
for (c, oc) in q.audio_tracks[0].items.iter().zip(&orig.audio_tracks[0].items) {
let p = path_of(&s, c.item);
assert!(p.ends_with(".wav") && std::path::Path::new(&p).exists(), "{p}");
@@ -138,7 +138,7 @@ fn aaf_separate_aiff_breakout_and_linked() {
let n = imported_seq(&r);
let q = s.project.sequence(n).unwrap();
let a = &q.audio_tracks[0].items[0];
assert_eq!(path_of(&s, a.item), path_of(&s, items[0]));
assert_eq!(std::path::Path::new(&path_of(&s, a.item)), std::path::Path::new(&path_of(&s, items[0])));
assert_eq!(a.source_in, FrameRate::FPS_24.tick_of(12));
assert!(!s.project.item(a.item).unwrap().as_media().unwrap().offline);
let _ = std::fs::remove_dir_all(dir);
@@ -159,7 +159,7 @@ fn aaf_video_mixdown() {
assert_eq!(q.video_tracks.iter().map(|t| t.items.len()).sum::<usize>(), 1);
let v = &q.video_tracks[0].items[0];
assert_eq!(v.duration, FrameRate::FPS_24.tick_of(48));
assert_eq!(path_of(&s, v.item), mov);
assert_eq!(std::path::Path::new(&path_of(&s, v.item)), std::path::Path::new(mov));
let m = s.project.item(v.item).unwrap().as_media().unwrap();
assert!(!m.offline);
assert_eq!(m.info.video.as_ref().map(|v| (v.width, v.height)), Some((64, 36)));
@@ -284,7 +284,7 @@ fn aaf_keeps_a_nested_sequence_as_a_sequence() {
let want: Vec<_> = orig.tracks(kind)[0].items.iter().map(|c| (c.start, c.duration)).collect();
assert_eq!(got, want, "{kind:?}");
}
assert_eq!(path_of(&s, n.video_tracks[0].items[0].item), path_of(&s, items[0]));
assert_eq!(std::path::Path::new(&path_of(&s, n.video_tracks[0].items[0].item)), std::path::Path::new(&path_of(&s, items[0])));
for (c, oc) in n.audio_tracks[0].items.iter().zip(&orig.audio_tracks[0].items) {
let p = path_of(&s, c.item);
assert!(p.ends_with(".wav") && std::path::Path::new(&p).exists(), "{p}");
@@ -311,3 +311,192 @@ fn commands_are_registered_with_menus_and_validate_params() {
assert!(s.execute("file.importAaf", json!({"path": dir.join("a.mov").to_string_lossy()})).is_err());
let _ = std::fs::remove_dir_all(dir);
}
// Selected-stream essence uses only PR361 schema13 placement and decoder fields.
mod selected_streams {
use crate::{Services, Session};
use filmcraft_frame::{AudioBuffer, VideoFrame};
use filmcraft_media::{FrameRequest, MediaError, MediaInfo, MediaKind, MediaSource};
use filmcraft_project::{ClipId, ItemId, ItemKind, Label, MediaClip, MediaRef, SequenceSettings};
use filmcraft_time::{FrameRate, Tick};
use serde_json::json;
use std::sync::{Arc, Mutex};
type Calls = Arc<Mutex<Vec<(usize, i64, usize, u32)>>>;
struct Tones {
info: MediaInfo,
calls: Calls,
}
impl Tones {
fn new(streams: usize) -> Self {
let source = filmcraft_media::generators::GeneratorSource::new(
filmcraft_media::Generator::BarsAndTone,
32,
18,
FrameRate::FPS_24,
Tick::from_seconds_f64(2.0),
);
let mut info = source.info().clone();
info.kind = MediaKind::AudioOnly;
info.video = None;
info.audio_streams = vec![info.audio_streams[0].clone(); streams];
Self { info, calls: Arc::default() }
}
fn sample(stream: usize, channel: usize, frame: i64, rate: u32) -> f32 {
let frequency = (stream * 2 + channel + 1) as f64 * 240.0;
let amplitude = 0.05 * (stream + 1) as f64;
(amplitude * (std::f64::consts::TAU * frequency * frame as f64 / rate as f64).sin()) as f32
}
}
impl MediaSource for Tones {
fn info(&self) -> &MediaInfo {
&self.info
}
fn video_frame(&self, _: FrameRequest) -> filmcraft_media::Result<Arc<VideoFrame>> {
Err(MediaError::NoStream("video"))
}
fn audio(&self, start: i64, frames: usize, sample_rate: u32) -> filmcraft_media::Result<AudioBuffer> {
self.audio_stream(0, start, frames, sample_rate)
}
fn audio_stream(&self, stream: usize, start: i64, frames: usize, sample_rate: u32) -> filmcraft_media::Result<AudioBuffer> {
if stream >= self.info.audio_streams.len() {
return Err(MediaError::NoStream("audio"));
}
self.calls.lock().unwrap().push((stream, start, frames, sample_rate));
Ok(AudioBuffer {
sample_rate,
channels: (0..2).map(|channel| (0..frames).map(|i| Self::sample(stream, channel, start + i as i64, sample_rate)).collect()).collect(),
})
}
}
#[derive(Default)]
struct MemoryFiles {
writes: Mutex<Vec<(String, Vec<u8>)>>,
}
impl Services for MemoryFiles {
fn read_file(&self, _: &str) -> std::io::Result<Vec<u8>> {
Err(std::io::ErrorKind::NotFound.into())
}
fn write_file(&self, path: &str, bytes: &[u8]) -> std::io::Result<()> {
self.writes.lock().unwrap().push((path.to_owned(), bytes.to_vec()));
Ok(())
}
fn export_in_memory(&self) -> bool {
true
}
}
fn add_source(s: &mut Session, streams: usize) -> (ItemId, Calls) {
let source = Arc::new(Tones::new(streams));
let calls = source.calls.clone();
let mut project = (*s.project).clone();
let item = project.add_item(
"distinct tones",
Label::Iris,
ItemKind::Media(MediaClip {
media: MediaRef::File { path: format!("tone-source-{streams}.wav") },
info: source.info.clone(),
interpret: Default::default(),
mark_in: None,
mark_out: None,
markers: vec![],
offline: false,
proxy: None,
identity: None,
}),
None,
);
s.project = Arc::new(project);
s.media.insert(item, source);
(item, calls)
}
fn fixture() -> (Session, ItemId, ItemId, Calls, Arc<MemoryFiles>) {
let files = Arc::new(MemoryFiles::default());
let mut session = Session::new(files.clone());
let mut project = (*session.project).clone();
let sequence =
project.new_sequence("streams", SequenceSettings { width: 32, height: 18, frame_rate: FrameRate::FPS_24, ..Default::default() }, 0, 1, None);
session.project = Arc::new(project);
session.state.active_sequence = Some(sequence);
session.state.open_sequences = vec![sequence];
let (item, calls) = add_source(&mut session, 3);
session.state.project_selection = vec![item];
session.execute("sequence.linkedSelection", json!({"on": false})).unwrap();
(session, item, sequence, calls, files)
}
fn place(s: &mut Session, item: ItemId, seconds: f64) -> Vec<ClipId> {
let result = s.execute("timeline.place", json!({"item": item.0, "track": "A1", "seconds": seconds})).unwrap();
result["clips"].as_array().unwrap().iter().map(|id| ClipId(id.as_u64().unwrap())).collect()
}
#[test]
fn aaf_essence_distinguishes_streams_and_linked_refusal_precedes_writes() {
use crate::aaf_omf::{AudioMode, AudioPlan};
use filmcraft_interchange::essence::{EssenceData, EssenceKey, NestNeeds};
let (mut s, item, sequence, calls, files) = fixture();
place(&mut s, item, 0.0);
let plan = AudioPlan {
mode: AudioMode::Embedded,
aiff: false,
mxf: false,
sample_rate: 48_000,
bits: 16,
trim: true,
handles: Tick::ZERO,
render_effects: false,
breakout: false,
nests: NestNeeds::Inside,
};
let (essences, paths) = crate::aaf_omf::prepare_audio(&s, sequence, &plan, "ignored").unwrap();
assert!(paths.is_empty());
assert_eq!(essences.len(), 3);
for (stream, essence) in essences.iter().enumerate() {
assert_eq!(essence.key, EssenceKey::media(item, stream));
let EssenceData::Embedded(bytes) = &essence.data else { panic!("expected embedded PCM") };
for channel in 0..2 {
let offset = (25 * 2 + channel) * 2;
let actual = i16::from_le_bytes([bytes[offset], bytes[offset + 1]]);
let expected = (Tones::sample(stream, channel, 25, 48_000) * 32767.0).round() as i16;
assert_eq!(actual, expected);
}
}
let options = filmcraft_interchange::aaf::AafOptions {
media: filmcraft_interchange::essence::MediaOptions { essence: essences, ..Default::default() },
..Default::default()
};
let (aaf, _) = filmcraft_interchange::aaf::export(&s.project, sequence, &options).unwrap();
let omf_options = filmcraft_interchange::omf::OmfOptions { media: options.media.clone(), ..Default::default() };
let (omf, _) = filmcraft_interchange::omf::export(&s.project, sequence, &omf_options).unwrap();
for (kind, document) in [("AAF", aaf), ("OMF", omf)] {
let (imported, extracted, _) = if kind == "AAF" {
filmcraft_interchange::aaf::import(&document, &Default::default()).unwrap()
} else {
filmcraft_interchange::omf::import(&document, &Default::default()).unwrap()
};
assert_eq!(extracted.len(), 3, "{kind}: composition memo must retain distinct selected-stream essence");
let tracks = &imported.project.sequence(imported.sequences[0]).unwrap().audio_tracks;
assert_eq!(tracks.len(), 3, "{kind}");
let mut source_ids: Vec<_> = tracks.iter().map(|track| track.items[0].item).collect();
source_ids.sort();
source_ids.dedup();
assert_eq!(source_ids.len(), 3, "{kind}: each stream remains a distinct imported source");
let mut waveforms: Vec<_> = extracted.iter().map(|media| media.wav.clone()).collect();
waveforms.sort();
waveforms.dedup();
assert_eq!(waveforms.len(), 3, "{kind}");
}
assert!(calls.lock().unwrap().iter().any(|call| call.0 == 2));
assert!(files.writes.lock().unwrap().is_empty());
let error =
s.execute("file.exportAaf", json!({"path": "must-not-be-written.aaf", "sequence": sequence.0, "audio": "linked", "trimAudio": false})).unwrap_err();
assert!(error.to_string().contains("embedded") || error.to_string().contains("separate"));
assert!(files.writes.lock().unwrap().is_empty());
}
}
+179
View File
@@ -0,0 +1,179 @@
//! Multi-clip audio placement: `audio_placement_specs` and dynamic audio track allocation.
use filmcraft_media::MediaSource;
use filmcraft_project::{AudioChannels, ItemId, TrackKind};
use filmcraft_time::{FrameRate, Tick, TimeRange};
use serde_json::json;
use crate::Session;
use crate::commands::{AudioPlacementSpec, audio_placement_specs, ensure_audio_tracks};
fn demo() -> Session {
let mut s = Session::default();
s.execute("file.openDemoProject", json!({})).unwrap();
s
}
fn active(s: &Session) -> ItemId {
s.state.active_sequence.expect("demo session has an active sequence")
}
#[test]
fn specs_always_place_one_clip_on_the_destination() {
assert_eq!(audio_placement_specs(&[], 1), vec![AudioPlacementSpec { track_offset: 0, audio_stream: 0, source_channels: None }]);
assert_eq!(audio_placement_specs(&[vec![0, 1]], 1).len(), 1);
}
#[test]
fn specs_put_further_clips_on_the_tracks_below() {
let specs = audio_placement_specs(&[vec![0], vec![1], vec![2, 3]], 1);
let got: Vec<(usize, Option<Vec<u16>>)> = specs.into_iter().map(|s| (s.track_offset, s.source_channels)).collect();
assert_eq!(got, vec![(0, None), (1, Some(vec![1])), (2, Some(vec![2, 3]))]);
}
#[test]
fn specs_are_capped_for_hostile_channel_maps() {
let clips = vec![vec![0u16]; 100_000];
assert!(audio_placement_specs(&clips, usize::MAX).len() <= crate::sequence_tools::MAX_TRACKS);
}
#[test]
fn specs_append_container_streams_after_channel_map_clips() {
let specs = audio_placement_specs(&[vec![1], vec![0]], 3);
assert_eq!(
specs.iter().map(|s| (s.track_offset, s.audio_stream, s.source_channels.clone())).collect::<Vec<_>>(),
vec![(0, 0, None), (1, 0, Some(vec![0])), (2, 1, None), (3, 2, None)]
);
assert_eq!(audio_placement_specs(&[], 0).len(), 1);
}
#[test]
fn placing_media_with_two_streams_expands_tracks_and_keeps_stream_indices() {
let mut s = Session::default();
let mut p = (*s.project).clone();
let rate = FrameRate::FPS_24;
let seq = p.new_sequence("Two streams", Default::default(), 1, 1, None);
let source = filmcraft_media::generators::GeneratorSource::new(filmcraft_media::Generator::BarsAndTone, 32, 32, rate, rate.tick_of(24));
let mut info = source.info().clone();
info.audio_streams.push(info.audio_streams[0].clone());
let item = p.add_item(
"two streams",
filmcraft_project::Label::Iris,
filmcraft_project::ItemKind::Media(filmcraft_project::MediaClip {
media: filmcraft_project::MediaRef::Generator(filmcraft_media::Generator::BarsAndTone),
info,
interpret: Default::default(),
mark_in: None,
mark_out: None,
markers: vec![],
offline: false,
proxy: None,
identity: None,
}),
None,
);
let (vdest, adest) = {
let q = p.sequence(seq).unwrap();
(q.video_tracks[0].id, q.audio_tracks[0].id)
};
s.project = std::sync::Arc::new(p);
s.state.active_sequence = Some(seq);
let ids =
crate::commands::place_item(&mut s, item, TimeRange::new(Tick::ZERO, rate.tick_of(24)), Tick::ZERO, Some(vdest), Some(adest), false, "test", None)
.unwrap();
assert_eq!(ids.len(), 3);
let q = s.project.sequence(seq).unwrap();
assert_eq!(q.audio_tracks.len(), 2);
let (a, b) = (&q.audio_tracks[0].items[0], &q.audio_tracks[1].items[0]);
assert_eq!((a.audio_stream, b.audio_stream), (0, 1));
assert_eq!(a.link, b.link);
assert!(a.link.is_some());
assert_ne!(a.id, b.id);
assert_eq!(a.source_in, b.source_in);
assert_eq!(a.start, b.start);
}
#[test]
fn clip_peaks_measure_the_stream_the_clip_plays() {
let mut s = Session::default();
let mut p = (*s.project).clone();
let rate = FrameRate::FPS_24;
let seq = p.new_sequence("Two streams", Default::default(), 1, 1, None);
let source = filmcraft_media::generators::GeneratorSource::new(filmcraft_media::Generator::BarsAndTone, 32, 32, rate, rate.tick_of(24));
let mut info = source.info().clone();
info.audio_streams.push(info.audio_streams[0].clone());
let item = p.add_item(
"two streams",
filmcraft_project::Label::Iris,
filmcraft_project::ItemKind::Media(filmcraft_project::MediaClip {
media: filmcraft_project::MediaRef::Generator(filmcraft_media::Generator::BarsAndTone),
info,
interpret: Default::default(),
mark_in: None,
mark_out: None,
markers: vec![],
offline: false,
proxy: None,
identity: None,
}),
None,
);
let (vdest, adest) = {
let q = p.sequence(seq).unwrap();
(q.video_tracks[0].id, q.audio_tracks[0].id)
};
s.project = std::sync::Arc::new(p);
s.state.active_sequence = Some(seq);
let ids =
crate::commands::place_item(&mut s, item, TimeRange::new(Tick::ZERO, rate.tick_of(24)), Tick::ZERO, Some(vdest), Some(adest), false, "test", None)
.unwrap();
let q = s.project.sequence(seq).unwrap();
let (a, b) = (q.audio_tracks[0].items[0].id, q.audio_tracks[1].items[0].id);
assert!(ids.contains(&a) && ids.contains(&b));
let peaks = crate::mixer::clip_peaks(&s, &[a, b]);
let db = |c: filmcraft_project::ClipId| peaks.iter().find(|(id, _)| *id == c).map(|(_, d)| *d).unwrap();
// The generator only has stream 0, so the clip playing stream 1 is silent, not a copy of stream 0.
assert!(db(a) > -30.0, "{peaks:?}");
assert!(db(b) < -100.0, "{peaks:?}");
}
#[test]
fn missing_audio_tracks_are_cloned_from_the_destination() {
let s = demo();
let seq = active(&s);
let mut p = (*s.project).clone();
let have = p.sequence(seq).map(|q| q.audio_tracks.len()).unwrap_or(0);
assert!(have >= 1);
if let Some(t) = p.sequence_mut(seq).and_then(|q| q.audio_tracks.get_mut(0)) {
t.channels = AudioChannels::Mono;
t.volume_db = -3.0;
t.locked = true;
}
let ids = ensure_audio_tracks(&mut p, seq, 0, have + 3, "test").unwrap();
assert_eq!(ids.len(), have + 3);
let q = p.sequence(seq).unwrap();
for t in &q.audio_tracks[have..] {
assert_eq!(t.kind, TrackKind::Audio);
assert_eq!(t.channels, AudioChannels::Mono);
assert_eq!(t.volume_db, -3.0);
assert!(t.items.is_empty() && !t.locked);
}
let mut unique = ids.clone();
unique.sort_by_key(|t| t.0);
unique.dedup();
assert_eq!(unique.len(), ids.len(), "track ids are unique");
// enough tracks already: nothing is added
let again = ensure_audio_tracks(&mut p, seq, 0, have + 3, "test").unwrap();
assert_eq!(again, ids);
}
#[test]
fn hostile_track_counts_are_an_error_not_a_crash() {
let s = demo();
let seq = active(&s);
let mut p = (*s.project).clone();
assert!(ensure_audio_tracks(&mut p, seq, usize::MAX, usize::MAX, "test").is_err());
assert!(ensure_audio_tracks(&mut p, seq, 0, 1_000_000, "test").is_err());
assert!(ensure_audio_tracks(&mut p, ItemId(u64::MAX), 0, 1, "test").is_err());
}
+23 -2
View File
@@ -22,12 +22,13 @@ use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::sync::mpsc::{Receiver, RecvTimeoutError, Sender, channel};
use std::thread::JoinHandle;
use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};
use std::time::Duration;
use filmcraft_format::autosave as names;
use filmcraft_project::Project;
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
use web_time::{Instant, SystemTime, UNIX_EPOCH};
use crate::settings;
@@ -391,6 +392,7 @@ impl Preferences {
self.playback.postroll_seconds = secs(self.playback.postroll_seconds);
self.trim.large_trim_offset = self.trim.large_trim_offset.clamp(1, 1000);
self.labels.sanitize_labels();
self.appearance.sync_color_theme();
self.version = settings::PREFS_VERSION;
}
@@ -453,6 +455,12 @@ impl Preferences {
let _: Preferences = serde_json::from_value(v.clone()).map_err(|e| format!("`{key}`: {e}"))?;
settings::sanitize(&mut v, &Self::default().to_value());
let mut p: Preferences = serde_json::from_value(v).map_err(|e| format!("`{key}`: {e}"))?;
// Old clients that set the single Color Theme still pick the theme they name (before
// `clamp`, which brings `colorTheme` back in line with the mode and theme choices).
if key == "appearance.colorTheme" {
let theme = p.appearance.color_theme.clone();
p.appearance.select_theme(&theme);
}
p.clamp();
*self = p;
Ok(())
@@ -470,7 +478,17 @@ impl Preferences {
}
}
/// The per-user data directory (`FILMCRAFT_DATA_DIR` overrides):
/// A file with this name next to the executable makes the install portable: its data directory is
/// `<exe dir>/data` instead of the per-user one (the Windows portable zip ships it).
pub const PORTABLE_MARKER: &str = "portable.txt";
/// `<exe_dir>/data` when `exe_dir` holds [`PORTABLE_MARKER`], else `None`.
pub fn portable_data_dir(exe_dir: &Path) -> Option<PathBuf> {
exe_dir.join(PORTABLE_MARKER).is_file().then(|| exe_dir.join("data"))
}
/// The data directory. `FILMCRAFT_DATA_DIR` overrides; a portable install ([`PORTABLE_MARKER`] next
/// to the executable) uses `<exe dir>/data`; otherwise the per-user directory:
/// macOS `~/Library/Application Support/FilmCraft`, Windows `%APPDATA%\FilmCraft`,
/// elsewhere `$XDG_DATA_HOME/filmcraft` or `~/.local/share/filmcraft`.
pub fn default_data_dir() -> Option<PathBuf> {
@@ -478,6 +496,9 @@ pub fn default_data_dir() -> Option<PathBuf> {
if let Some(d) = env("FILMCRAFT_DATA_DIR") {
return Some(d);
}
if let Some(d) = std::env::current_exe().ok().and_then(|exe| exe.parent().and_then(portable_data_dir)) {
return Some(d);
}
if cfg!(target_os = "macos") {
return env("HOME").map(|h| h.join("Library/Application Support/FilmCraft"));
}
+13
View File
@@ -176,6 +176,19 @@ fn auto_save_ring_rotates_and_can_save_the_project() {
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn portable_marker_keeps_data_next_to_the_executable() {
// #139: a portable install must not write to %APPDATA%.
let d = temp_dir("portable");
assert_eq!(autosave::portable_data_dir(&d), None);
std::fs::create_dir_all(d.join(autosave::PORTABLE_MARKER)).unwrap();
assert_eq!(autosave::portable_data_dir(&d), None, "a directory is not the marker");
std::fs::remove_dir_all(d.join(autosave::PORTABLE_MARKER)).unwrap();
std::fs::write(d.join(autosave::PORTABLE_MARKER), "").unwrap();
assert_eq!(autosave::portable_data_dir(&d), Some(d.join("data")));
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn journal_costs_the_ui_thread_only_a_handoff() {
// Per change the UI thread pays an Arc clone and a channel send; encoding happens on the worker.
+5 -1
View File
@@ -513,7 +513,11 @@ pub fn commands() -> Vec<CommandSpec> {
enabled: always,
run: |s, p| {
let Some(seq) = s.active_sequence() else { return Ok(json!({"tracks": []})) };
let only = p.get("track").and_then(|_| track_param(s, p));
// a `track` that names no caption track is an error, never every track
let only = match p.get("track") {
Some(_) => Some(track_param(s, p).ok_or_else(|| bad("captions.list", "no such caption track"))?),
None => None,
};
let tracks: Vec<Value> =
seq.caption_tracks.iter().enumerate().filter(|(_, t)| only.is_none_or(|o| o == t.id)).map(|(i, t)| track_json(s, t, i)).collect();
Ok(json!({"tracks": tracks, "selection": s.state.caption_selection.iter().map(|c| c.0).collect::<Vec<_>>()}))
+21
View File
@@ -137,3 +137,24 @@ fn caption_commands_need_tracks_and_navigate() {
s.execute("captions.deleteTrack", json!({"track": "C1"})).unwrap();
assert!(s.active_sequence().unwrap().caption_tracks.is_empty());
}
#[test]
fn caption_list_track_filter_never_falls_back_to_every_track() {
let mut s = demo();
s.execute("captions.newTrack", json!({})).unwrap();
s.execute("captions.newTrack", json!({})).unwrap();
let all = s.execute("captions.list", json!({})).unwrap();
let ids: Vec<u64> = all["tracks"].as_array().unwrap().iter().map(|t| t["id"].as_u64().unwrap()).collect();
assert_eq!(ids.len(), 2, "{all}");
for (i, id) in ids.iter().enumerate() {
for sel in [json!(id), json!(format!("C{}", i + 1))] {
let r = s.execute("captions.list", json!({"track": sel})).unwrap();
let got: Vec<u64> = r["tracks"].as_array().unwrap().iter().map(|t| t["id"].as_u64().unwrap()).collect();
assert_eq!(got, vec![*id], "{sel}");
}
}
let missing = ids.iter().max().unwrap() + 999;
for sel in [json!(missing), json!("C3"), json!("C0"), json!("nope")] {
assert!(s.execute("captions.list", json!({"track": sel})).is_err(), "{sel} names no caption track");
}
}
Loaded 100 of 531 files, more files were not shown because too many files have changed in this diff. Show more