Disk cache: refuse writes below 64 MB of room, show this disk's reserve

Review follow-ups to the reserve fix:

- accepts_writes also refuses while free disk space leaves less than
  64 MB of room (a limit the setting asks for is always used): a few
  frames would fill it, and the rest would be compressed only to be
  refused.
- While writes are refused the idle passes ask for no repaint, so the
  UI keeps a frame coming every 5 s: the free space is read again and
  the RAM frames are written once there is room.
- Settings > Disk names the space kept free on this disk (the reserve
  the last reading applied; the rule from MIN/MAX_DISK_RESERVE when it
  is unknown), and the "no room" note follows accepts_writes.
- The settings.disk.cacheLimit automation label is machine-readable
  and the same in every language: limitMb, maxMb, reserveMb,
  spaceLimited, writes.
- cargo fmt.
This commit is contained in:
vangel76 committed 2026-11-07 23:38:49 +01:00
1 parent 4295717ca7
commit e1b59f3760
5 files changed
+102 -35

No files matched your search

+40 -9
View File
@@ -345,6 +345,8 @@ pub struct DiskCache {
disk_cap: AtomicU64,
/// Free space is limiting the cache (warned once per episode).
space_limited: AtomicBool,
/// The free space the cache leaves on its disk ([`disk_reserve`]); 0 = unknown.
reserve: AtomicU64,
hits: AtomicU64,
misses: AtomicU64,
writes: AtomicU64,
@@ -399,10 +401,14 @@ fn checksum(data: &[u8]) -> u64 {
}
/// The least free space the disk cache leaves on its disk.
const MIN_DISK_RESERVE: u64 = 2 << 30;
pub const MIN_DISK_RESERVE: u64 = 2 << 30;
/// The most free space the disk cache leaves on its disk: 5% of a large disk is far more than
/// the system needs to keep working (on a 4 TB disk it held a 20 GB cache at 0 with 190 GB free).
const MAX_DISK_RESERVE: u64 = 10 << 30;
pub const MAX_DISK_RESERVE: u64 = 10 << 30;
/// Below this much room left by free disk space, frame and layer writes are refused before they
/// are encoded ([`DiskCache::accepts_writes`]): a few frames would fill it, and the rest would be
/// compressed only to be refused.
const MIN_WRITE_ROOM: u64 = 64 << 20;
/// The free space the disk cache leaves on a disk of `total` bytes: 5% of it, at least 2 GB and
/// at most 10 GB. It keeps the cache from filling the disk; the cache's size is the user's
@@ -441,6 +447,7 @@ impl DiskCache {
max_bytes: AtomicU64::new(max_bytes),
disk_cap: AtomicU64::new(u64::MAX),
space_limited: AtomicBool::new(false),
reserve: AtomicU64::new(0),
hits: AtomicU64::new(0),
misses: AtomicU64::new(0),
writes: AtomicU64::new(0),
@@ -539,16 +546,18 @@ impl DiskCache {
/// [`disk_reserve`] free (5% of the disk, at least 2 GB and at most 10 GB), evicting when it
/// must.
pub fn set_disk_space(&self, space: Option<(u64, u64)>) {
let mut reserve = 0;
let cap = match space {
// (a disk reported as 0 bytes, such as an automount map, is unknown)
Some((total, available)) if total > 0 => {
let reserve = disk_reserve(total);
reserve = disk_reserve(total);
let used = self.index.lock().map(|g| g.total()).unwrap_or_default();
// The cache's own files count as free: below the reserve it gives space back.
used.saturating_add(available).saturating_sub(reserve)
}
_ => u64::MAX,
};
self.reserve.store(reserve, Ordering::Relaxed);
self.disk_cap.store(cap, Ordering::Relaxed);
let limited = cap < self.max_bytes();
if self.space_limited.swap(limited, Ordering::Relaxed) != limited && limited {
@@ -557,12 +566,20 @@ impl DiskCache {
self.evict();
}
/// Whether a write can be stored at all now: `false` while free disk space holds the limit
/// at 0. One atomic read, so frame and layer writes check it before they spend time
/// converting and compressing pixels that [`DiskCache::write_format`] would refuse anyway.
/// (Whether a given entry fits is known only once it is compressed: `write_format` decides.)
/// Whether frame and layer writes are worth encoding now: `false` while free disk space
/// holds the limit at 0 or under 64 MB (a limit the setting asks for is always used). Two
/// atomic reads, so writes check it before they spend time converting and compressing
/// pixels that [`DiskCache::write_format`] would refuse anyway. (Whether a given entry fits
/// is known only once it is compressed: `write_format` decides.)
pub fn accepts_writes(&self) -> bool {
self.limit_bytes() > 0
let (max, limit) = (self.max_bytes(), self.limit_bytes());
limit > 0 && (limit >= MIN_WRITE_ROOM || limit == max)
}
/// The free space the cache leaves on its disk ([`disk_reserve`]), once the disk's size is
/// known ([`DiskCache::set_disk_space`]).
pub fn reserve_bytes(&self) -> Option<u64> {
Some(self.reserve.load(Ordering::Relaxed)).filter(|r| *r > 0)
}
/// Frame and layer entries compressed for writing so far (tests: a write refused up front
@@ -1064,7 +1081,7 @@ impl DiskCache {
return;
}
self.count_encode();
let _ =self.enqueue_format(Kind::Layer, key, FORMAT_LAYER_FILES, with_files(files, encode_layer(buf)), Some((files, since)));
let _ = self.enqueue_format(Kind::Layer, key, FORMAT_LAYER_FILES, with_files(files, encode_layer(buf)), Some((files, since)));
}
/// Write a layer buffer synchronously (tests); a list that can't be stored is not.
@@ -1877,6 +1894,7 @@ mod tests {
let st = dc.stats();
assert_eq!((st.max_bytes, st.limit_bytes), (20 * GB, 20 * GB), "{st:?}");
assert!(dc.accepts_writes());
assert_eq!(dc.reserve_bytes(), Some(10 * GB));
// Just above the 10 GB reserve: the cache may use what is left above it.
dc.set_disk_space(Some((4096 * GB, 15 * GB)));
assert_eq!(dc.stats().limit_bytes, 5 * GB);
@@ -1908,6 +1926,19 @@ mod tests {
dc.flush();
assert_eq!(dc.encodes(), 0, "nothing compressed");
assert_eq!(dc.stats().entries, 0);
// 32 MB of room above the reserve: a few frames would fill it, the rest would be
// compressed only to be refused. Not worth encoding either.
dc.set_disk_space(Some((4096 * GB, 10 * GB + (32 << 20))));
assert_eq!(dc.stats().limit_bytes, 32 << 20);
assert!(!dc.accepts_writes());
assert!(!dc.put_frame(1, 16, 16, &rgba, &ReadFiles::default(), None, 0), "refused");
dc.put_layer(3, &buf(3, 16, 16), &EntryFiles::default(), 0);
assert_eq!(dc.encodes(), 0, "nothing compressed");
// A small limit the setting asks for (not free space) is used.
dc.set_disk_space(None);
dc.set_max_bytes(32 << 20);
assert!(dc.accepts_writes());
dc.set_max_bytes(20 * GB);
// Room again: the same writes are compressed and stored.
dc.set_disk_space(Some((4096 * GB, 500 * GB)));
assert!(dc.put_frame(1, 16, 16, &rgba, &ReadFiles::default(), None, 0));
+5 -4
View File
@@ -1058,13 +1058,14 @@ impl Frames {
let oh = opts_hash(opts);
self.note_disk_series(base.comp, base.scale, base.view ^ oh);
self.carry_disk(src);
// Free disk space holds the cache at 0: nothing would be stored, so nothing is converted
// (a refusal doesn't count as a try, so every idle pass would convert them all again).
// A later idle pass writes them once there is room.
// Free disk space leaves the cache (next to) no room: nothing would be stored, so nothing
// is converted (a refusal doesn't count as a try, so every idle pass would convert them
// all again). A later idle pass writes them once there is room (`prefs_live::memory_watch`
// keeps frames coming meanwhile).
if !dc.accepts_writes() {
return false;
}
let known =self.content_keys.lock().unwrap_or_else(std::sync::PoisonError::into_inner).get(&(base.revision, base.comp)).copied();
let known = self.content_keys.lock().unwrap_or_else(std::sync::PoisonError::into_inner).get(&(base.revision, base.comp)).copied();
let Some(content) = known else {
self.request_content_key(src.project.clone(), base.revision, base.comp);
return true;
+48 -20
View File
@@ -245,12 +245,14 @@ fn page_ui(app: &mut EffectcraftApp, ui: &mut egui::Ui, page: &Page, cur: &Value
/// Settings ▸ Disk, under Maximum Disk Cache Size: the limit in effect, and why it is lower when
/// free disk space holds the cache below the setting (it never fills the disk). Automation id
/// `settings.disk.cacheLimit`. Nothing without an open disk cache.
/// `settings.disk.cacheLimit`, labelled `limitMb=… maxMb=… reserveMb=… spaceLimited=… writes=…`
/// (machine-readable, the same in every interface language). Nothing without an open disk cache.
/// The maximum is the one applied: an edit applies at once (a dragged value on release).
fn disk_cache_limit_ui(app: &mut EffectcraftApp, ui: &mut egui::Ui, t: &Tokens) {
let Some(dc) = app.session.disk_cache.clone() else { return };
// (two atomic reads: drawn every frame)
let (max, limit) = (dc.max_bytes(), dc.limit_bytes());
let lines = disk_cache_limit_lines(max, limit);
// (atomic reads: drawn every frame)
let (max, limit, reserve, writes) = (dc.max_bytes(), dc.limit_bytes(), dc.reserve_bytes(), dc.accepts_writes());
let lines = disk_cache_limit_lines(max, limit, reserve, writes);
let color = if limit < max { t.warning } else { t.text_dim };
let r = ui
.vertical(|ui| {
@@ -259,18 +261,31 @@ fn disk_cache_limit_ui(app: &mut EffectcraftApp, ui: &mut egui::Ui, t: &Tokens)
}
})
.response;
app.auto.add("settings.disk.cacheLimit", r.rect, &lines.join(" "));
let label =
format!("limitMb={} maxMb={} reserveMb={} spaceLimited={} writes={writes}", limit.min(max) >> 20, max >> 20, reserve.unwrap_or(0) >> 20, limit < max);
app.auto.add("settings.disk.cacheLimit", r.rect, &label);
}
/// The lines [`disk_cache_limit_ui`] shows for a disk cache of `max` bytes (Maximum Disk Cache
/// Size) whose limit in effect is `limit` bytes.
pub(crate) fn disk_cache_limit_lines(max: u64, limit: u64) -> Vec<String> {
/// Size) whose limit in effect is `limit` bytes, leaving `reserve` bytes of its disk free (when
/// known), and taking frame and layer writes (`writes`, [`DiskCache::accepts_writes`]).
///
/// [`DiskCache::accepts_writes`]: effectcraft_engine::render::disk_cache::DiskCache::accepts_writes
pub(crate) fn disk_cache_limit_lines(max: u64, limit: u64, reserve: Option<u64>, writes: bool) -> Vec<String> {
use effectcraft_engine::render::disk_cache::{MAX_DISK_RESERVE, MIN_DISK_RESERVE};
const GB: f64 = (1u64 << 30) as f64;
let mut lines = vec![tr_args("Disk cache limit in effect: {} MB", &[&(limit.min(max) >> 20)])];
if limit < max {
lines.push(tr("Limited by free disk space: EffectCraft keeps 5% of the disk free (at least 2 GB, at most 10 GB).").to_string());
if limit == 0 {
lines.push(tr("No room left: nothing is written to the disk cache until disk space is freed.").to_string());
}
lines.push(match reserve {
Some(r) => tr_args("Limited by free disk space: EffectCraft keeps {} GB of this disk free.", &[&format!("{:.1}", r as f64 / GB)]),
None => tr_args(
"Limited by free disk space: EffectCraft keeps 5% of the disk free (at least {} GB, at most {} GB).",
&[&(MIN_DISK_RESERVE >> 30), &(MAX_DISK_RESERVE >> 30)],
),
});
}
if !writes {
lines.push(tr("No room left: nothing is written to the disk cache until disk space is freed.").to_string());
}
lines
}
@@ -1275,25 +1290,38 @@ mod tests {
let ctx = egui::Context::default();
let (tokens, cur) = (app.tokens, serde_json::to_value(&app.session.prefs).unwrap());
let mut actions = vec![];
let mut output = ctx.run_ui(
egui::RawInput { screen_rect: Some(egui::Rect::from_min_size(egui::Pos2::ZERO, vec2(900.0, 1200.0))), ..Default::default() },
|ui| page_ui(app, ui, &page, &cur, &tokens, &mut actions),
);
let mut output = ctx
.run_ui(egui::RawInput { screen_rect: Some(egui::Rect::from_min_size(egui::Pos2::ZERO, vec2(900.0, 1200.0))), ..Default::default() }, |ui| {
page_ui(app, ui, &page, &cur, &tokens, &mut actions)
});
output.textures_delta.clear();
app.auto.find("settings.disk.cacheLimit").map(|e| e.label.clone())
};
let text = || disk_cache_limit_lines(dc.max_bytes(), dc.limit_bytes(), dc.reserve_bytes(), dc.accepts_writes()).join(" ");
// A 4 TB disk with 190 GB free: the 20 GB setting holds.
dc.set_disk_space(Some((4096 * GB, 190 * GB)));
let label = shown(&mut app).expect("the limit is shown");
assert!(label.contains("20480 MB") && !label.contains("free disk space"), "{label}");
assert_eq!(label, "limitMb=20480 maxMb=20480 reserveMb=10240 spaceLimited=false writes=true");
assert_eq!(text(), "Disk cache limit in effect: 20480 MB");
// 3 GB free on a 4 TB disk: 10 GB stay free, nothing is left for the cache.
dc.set_disk_space(Some((4096 * GB, 3 * GB)));
let label = shown(&mut app).unwrap();
assert!(label.contains(": 0 MB") && label.contains("Limited by free disk space") && label.contains("No room left"), "{label}");
assert_eq!(shown(&mut app).unwrap(), "limitMb=0 maxMb=20480 reserveMb=10240 spaceLimited=true writes=false");
assert_eq!(
text(),
"Disk cache limit in effect: 0 MB Limited by free disk space: EffectCraft keeps 10.0 GB of this disk free. No room left: nothing is written to the disk cache until disk space is freed."
);
// 12 GB free: 2 GB for the cache.
dc.set_disk_space(Some((4096 * GB, 12 * GB)));
let label = shown(&mut app).unwrap();
assert!(label.contains("2048 MB") && label.contains("Limited by free disk space") && !label.contains("No room left"), "{label}");
assert_eq!(shown(&mut app).unwrap(), "limitMb=2048 maxMb=20480 reserveMb=10240 spaceLimited=true writes=true");
assert_eq!(text(), "Disk cache limit in effect: 2048 MB Limited by free disk space: EffectCraft keeps 10.0 GB of this disk free.");
// A 100 GB disk keeps 5 GB.
dc.set_disk_space(Some((100 * GB, 6 * GB)));
assert_eq!(shown(&mut app).unwrap(), "limitMb=1024 maxMb=20480 reserveMb=5120 spaceLimited=true writes=true");
// Without the disk's size, the rule from the constants.
assert_eq!(
disk_cache_limit_lines(20 * GB, GB, None, true)[1],
"Limited by free disk space: EffectCraft keeps 5% of the disk free (at least 2 GB, at most 10 GB)."
);
// No disk cache: no line.
app.session.disk_cache = None;
assert!(shown(&mut app).is_none());
+6
View File
@@ -141,6 +141,12 @@ pub fn dropped_files(app: &mut EffectcraftApp, ui: &egui::Ui, bounds: Rect) {
/// reading starts PowerShell in the background, so it isn't done more often).
pub(crate) fn memory_watch(app: &mut EffectcraftApp, ctx: &egui::Context, now: f64) {
let id = egui::Id::unique("prefs-memory-watch");
// While free disk space keeps frames and layers out of the disk cache, the idle passes ask
// for no repaint (nothing to write): keep a frame coming so its free space is read again and
// the RAM frames are written once there is room (the reading lands between frames).
if app.session.disk_cache.as_ref().is_some_and(|dc| !dc.accepts_writes()) {
ctx.request_repaint_after(std::time::Duration::from_secs(5));
}
let last = ctx.data(|d| d.get_temp::<f64>(id));
if last.is_some_and(|t| now - t < 30.0) {
return;
+3 -2
View File
@@ -145,8 +145,9 @@ Commands (CLI, MCP, control channel):
at most half the origin's quota; the folder setting does not apply there). On the desktop the
cache never fills its disk: it leaves 5% of the disk free, at least 2 GB and at most 10 GB, and
is smaller than `disk.diskCacheMaxGb` when free space requires it. The Disk page shows the limit
in effect in MB under Maximum Disk Cache Size, with a note when free disk space lowers it (at 0
nothing is written, and frames and layers are not compressed for it). The Disk page also
in effect in MB under Maximum Disk Cache Size, with a note naming the space kept free when free
disk space lowers it (with less than 64 MB of room nothing is written, and frames and layers
are not compressed for it). The Disk page also
shows **Browser Storage** in the web app: usage and quota, persistent storage, Clear buttons
(`storage.info` / `storage.persist` / `storage.clear`, see [web.md](web.md))
- `disk.mediaCacheFolder`: audio waveform summaries kept between sessions (`Peaks/`)