Commit Graph
220 Commits
Author SHA1 Message Date
FredApps 3951f6df01 sonarcloud: merge each shard's profiles before the scan downloads them (#210)
Each coverage shard now merges its own raw profiles into one sparse
.profdata with the archive toolchain's llvm-profdata and uploads only
that. The scan job merges the shard profiles instead of downloading
every raw profile beside the test archive, which on 11 Nov 2026 filled
the runner for vectorcraft (four ~7 GB zipped profraw artifacts).

A raw profile llvm-profdata cannot read, such as one cut short by a
test process that crashed while writing it (lightcraft, 11 Nov 2026),
now fails its shard with an error naming the file, instead of being
uploaded and failing the scan's merge.

Single-shard callers keep the cargo llvm-cov path unchanged.
2026-11-11 19:55:14 -05:00
Dylan M. Taylor ae29fbc6ec fix(sonarcloud): explicitly allow Ubuntu ESM HTTPS egress (#217) 2026-11-11 19:25:34 -05:00
FredApps 840a417e3d Release: scripts/cut_release.py cuts a release from green main without a bump pull request (#207)
* Release: scripts/cut_release.py cuts a release from green main without a bump pull request

On 11 Nov 2026 the release of 14 apps took most of the day. A large part went to version-bump pull
requests that changed only version strings yet waited on the full CI of a pull request (1-2 h of
FreeBSD, sharded SonarCloud coverage), and to rerunning it. The script commits the bump on the
newest main commit whose CI passed, merges it into release (which starts release.yml), opens the
same bump for main as its own pull request, and with --publish publishes the draft once the run
passed and has assets. Tests cover the bump: workspace and path-dependency versions, only the
workspace's own Cargo.lock entries, CRLF files, the repository hook.

* cut_release: review fixes

- A trailing comment after the workspace version stays; the value alone is replaced.
- Cargo.lock: only the workspace members' entries (cargo metadata --no-deps) change, not a local
  path dependency outside the workspace that shares the version.
- Cargo.lock is checked with a resolving cargo metadata --locked, on the bump, on release after
  the merge and on the sync branch; --no-deps never compared it with path dependency versions.
- --publish refuses a draft with fewer assets than the last release (a skipped build job leaves
  the run green), unless --allow-fewer-assets.
- The sync PR never moves main backwards: versions compare as numbers, and a main at or past the
  release gets none.
- A failed auto-merge says so, with the PR to merge by hand.
- Arguments that reach git or gh are checked first (owner/name, version, commit hash, workflow,
  title), so none can be read as an option. git, gh and cargo are required up front.

* Release: scripts/release_train.py releases many apps at once with cut_release.py

Every app in parallel, a log per app, and one summary line each (published, pushed, or stopped
and why); one app stopping doesn't stop the rest. Tested on figcraft and soundcraft with
--dry-run (both done in under a minute).

* tests: release_train options before -- in the test

* release_train: only owner/name repos and known cut_release.py options reach the subprocess
2026-11-11 23:02:43 +01:00
FredApps 603045e42d release.yml: a dry-run input builds and checks without releasing (#208)
* release.yml: a dry-run input builds and checks without releasing

For a nightly run on main that finds release-only breakage before release day. On 11 Nov 2026
gridcraft's riscv64 build, materialcraft's private-repo FreeBSD jobs, wordcraft's Print job and
the riscv64 apt mirrors each surfaced only during the release, one fix PR and release run at a
time. With dry-run: no job enters the release environment (no signing secrets; the environment
admits only the release branch), the macOS job and the draft are skipped, and the Flatpak OCI
layouts are not pushed to GHCR or Docker Hub.

* tests: the Flatpak OCI push and the release environment also stay out of a dry run
2026-11-11 22:46:06 +01:00
Dylan M. Taylor 15058dc71e ci: use StepSecurity's maintained actions (#206)
* ci: use StepSecurity's maintained actions

Each third-party action that StepSecurity publishes a secure drop-in
replacement for now uses that replacement, pinned to a commit with its
version as a comment. Versions follow what we used: the newest
StepSecurity release in the same major, or the closest major they
publish. dtolnay/rust-toolchain refs that named a toolchain (stable,
nightly, 1.76) become the replacement's `toolchain` input.

* tests: find the FreeBSD VM, SonarCloud scan and install-action steps by their StepSecurity names
2026-11-11 14:55:40 -05:00
Dylan M. Taylor 693c8dd084 sonarcloud: honour each repository's own sonar.cpd.exclusions (#205)
* sonarcloud: honour each repository's own sonar.cpd.exclusions

* sonarcloud: read sonar.cpd.exclusions with properties syntax and allow the ? wildcard
2026-11-11 14:42:26 -05:00
FredApps 5d42870ead apt: wait for cloud-init before rewriting the sources (#204)
During the ubuntu.com outage, cloud-init's apt_configure module (once per
instance, at boot) spent two 10-second DNS timeouts looking for a mirror
and then wrote ubuntu.sources from its template, after the job had
started. On those VMs configure-apt-mirrors had already pointed the
sources at its list: apt-get update fetched every index for that list,
cloud-init put the image's mirror list back, and apt-get install found
no indexes for it, so no Ubuntu package had a candidate (materialcraft's
Flatpak x86_64 job, the FreeBSD tests job on 26.04). A repro on 12
runners failed on 8, every round, and a file watcher caught cloud-init's
write landing mid-update.

wait-for-cloud-init.sh runs `cloud-init status --wait` (at most 300 s,
never failing the job) before configure-apt-mirrors and
install-riscv64-cross's select-mirrors.sh touch anything.
2026-11-11 17:57:29 +01:00
Dylan M. Taylor 95e8d67389 egress: allow *.kernel.org and *.dotsrc.org wherever apt mirrors are allowed (#203)
Block-mode Harden-Runner still flagged apt's lookups under these zones
(_https._tcp.mirrors.edge.kernel.org, _https._tcp.mirrors.dotsrc.org),
and apt failures are breaking CI across the org. Allow the whole zones on
443 next to the existing mirror hosts.
2026-11-11 11:20:38 -05:00
FredApps 3f541b8904 install-riscv64-cross: fall back to mirrors outside ubuntu.com (#202)
The release builds' Linux riscv64 job read amd64 from archive and
security.ubuntu.com and riscv64 from ports.ubuntu.com only. With no
ubuntu.com name resolving on the runners, apt spent 3m20s on failed
lookups and then found no libc6-dev:riscv64, so every app's release
failed there.

select-mirrors.sh now chooses one pair of mirrors for both architectures:
Ubuntu's own, then dotsrc (it carries both trees), then kernel.org with
TU Chemnitz's ports. It takes the first pair whose amd64 and riscv64
InRelease files answer for every suite with the same Date (Ubuntu signs
one Release for archive and ports), so Multi-Arch: same packages still
match. The image's own Ubuntu sources are disabled in that job: they
only added a second, lagging snapshot and, during the outage, the stall.

The riscv64 job runs Harden-Runner in audit mode, so release.yml needs
no allowlist change.
2026-11-11 17:01:34 +01:00
FredApps 0817d766c4 test-freebsd: show apt's view and the CPU when the package install fails (#201)
One run on 11 Nov 2026 found no candidate for tcsh or debsigs right after
an update that fetched every index from kernel.org; the runs before and
after it, under the same ubuntu.com outage, passed. Print apt-cache policy,
the architecture settings, the universe index files and whether the CPU
supports x86-64-v3, so the next failure shows the cause.
2026-11-11 16:36:11 +01:00
Dylan M. Taylor 3d49e352fa apt: stop SRV lookups; give the FreeBSD VM host the fallback mirrors (#200)
Harden-Runner blocked apt's SRV lookups (_https._tcp.mirrors.edge.kernel.org,
_https._tcp.mirrors.dotsrc.org): the allow lists name the hosts, not their
SRV records. configure-apt-mirrors now sets Acquire::EnableSrvRecords false;
apt connects to the hosts directly either way.

The FreeBSD workflow's VM action apt-installs QEMU on the host, which failed
during the ubuntu.com outage (sudo exit 100) and reached packages.microsoft.com.
Run configure-apt-mirrors first and drop the image's Microsoft source, which
the job never installs from.
2026-11-11 10:26:48 -05:00
FredApps d97dcd6dcf apt: fall back to mirrors outside ubuntu.com when it is down (#199)
On 11 Nov 2026 no ubuntu.com name resolved on the runners for about
half an hour. The images' mirror list (azure.archive, archive and
security.ubuntu.com) sits entirely in that zone, so every retry spent
its five minutes on 20-second lookups of the same dead names and the
SonarCloud archive builds failed with "downloading the packages failed
3 times".

configure-apt-mirrors points the image's Ubuntu sources at
/etc/apt/craft-apt-mirrors.txt: the image's mirrors in the image's
order, then mirrors.edge.kernel.org and mirrors.dotsrc.org (arm:
mirrors.dotsrc.org and ftp.tu-chemnitz.de). Each mirror is checked
first, in parallel; those that do not serve this release's InRelease
move to the end. install-apt-packages runs it before every network
attempt, and the plain apt-get steps (Flatpak, release, packaging lint,
RPM signing, Linux deps, FreeBSD test) run it before apt-get update.
Block-mode Harden-Runner lists allow the new hosts.
2026-11-11 16:21:03 +01:00
Dylan M. Taylor 91e2c76173 sonarcloud: opt-in SQLX_OFFLINE and extra cargo features for the coverage build (#195) 2026-11-10 23:08:00 -05:00
Dylan M. Taylor 0588474894 sonarcloud: optional JavaScript/TypeScript coverage (#194)
* sonarcloud: optional JavaScript/TypeScript coverage (node-coverage-script)

* sonarcloud: scan after failing JS/TS tests, then fail the job
2026-11-10 22:29:06 -05:00
Dylan M. Taylor 4b47a2a3c7 Sonar: [[ in sign-linux.sh, a linear trailer regex in refresh-contributors (S7688, S8786) (#193)
Clears both open SonarCloud issues.

- `actions/draft-release/sign-linux.sh`: `[[ ]]` in place of `[ ]` for bash tests (SonarCloud shell:S7688). Every script has a bash shebang; no test uses `-a`/`-o`, escaped parentheses or an unquoted pattern on the right of `=`.
- `actions/refresh-contributors/refresh_contributors.py` `model()`: `([^<>]*)<([^<>]+)>` on the stripped trailer, with the name stripped after, in place of `\s*(.*?)\s*<([^<>]+)>\s*` (python:S8786, backtracking). Same result for every Git identity: a name can't hold `<` or `>`.
2026-11-10 18:45:19 -05:00
Dylan M. Taylor 644a5d76b9 ci: harden workflows with credential isolation, template injection defenses, and dependabot cooldown (#191)
* ci: harden workflows with credential isolation, template injection defenses, and dependabot cooldown

- Add persist-credentials: false across all actions/checkout invocations
- Eliminate template injection by mapping matrix and input variables to env: in freebsd.yml, release.yml, and sonarcloud.yml
- Add 7-day cooldown to dependabot.yml to protect against supply-chain compromise of recently published releases
- Update test suite to validate hardened environment mappings

* ci: allow static.crates.io and registry endpoints in gate and windows-arm64 runners
2026-11-10 18:07:38 -05:00
FredApps 9b9e0f5128 Merge pull request #188 from storytold/sonar/docker-build-job-permissions
fix(docker-build): scope packages: write to the pushing jobs
2026-11-10 18:27:46 +01:00
FredApps 567530acb5 Merge branch 'main' into sonar/docker-build-job-permissions 2026-11-10 18:23:14 +01:00
FredApps b24513e795 fix(freebsd): cap the VM's ZFS ARC so big test crates aren't OOM-killed (#190)
* fix(freebsd): give the VM a swap file and cap the ZFS ARC

photocraft's FreeBSD job was SIGKILLed compiling photocraft-ui-egui's lib tests late in the run (release run 42609972735, dispatch 42805955303). Add an 8 GB swap file when the guest has none, cap the ARC at 2 GB when the image uses ZFS, and print disk, swap and memory so the next OOM is diagnosable.

* fix(freebsd): only touch swap and the ARC inside a FreeBSD guest

The workflow tests run the prepare script on Linux under sh -eu, where swapctl doesn't exist. Run the ARC cap, swap file and diagnostics only when uname is FreeBSD, and don't let the diagnostics fail the step.

* fix(freebsd): add the swap file only with room to spare, and remove a partial one

Per review: a dd that runs out of space must not leave a partial /var/swap0 that then starves the build. Require 16 GB free on /var first and delete the file if any step fails.

* fix(freebsd): keep only the ZFS ARC cap; drop the swap-file fallback

Per review: swap on a ZFS-backed file can hang FreeBSD under memory pressure, and today's image already has a swap partition, so the fallback never ran. The verified run passed on the ARC cap alone.
2026-11-10 16:40:36 +01:00
Dylan M. Taylor 969c32dcf2 ci: allow SonarCloud S3 analysis sensor cache and scanner data endpoints 2026-11-10 08:26:20 -05:00
Dylan M. Taylor 0912de799d ci: expand allowed-endpoints with wildcard domains for APT, GPG, and PyPI 2026-11-10 08:26:20 -05:00
Dylan M. Taylor 21875aea67 ci: add GPG keyservers to sonarcloud.yml allowed-endpoints 2026-11-10 08:26:20 -05:00
Dylan M. Taylor 2edf8b6175 ci: address review findings for Harden-Runner egress policy
- Add allowed-endpoints to ast-grep.yml matching test-ast-grep.yml
- Add pypi.org and files.pythonhosted.org to sonarcloud.yml for Python coverage
- Add packages.microsoft.com:443 and https mirrors to apt allowed-endpoints
- Set unvalidated reusable workflows (release, docker-build, gate, etc.) to audit mode
2026-11-10 08:26:20 -05:00
Dylan M. Taylor 9e571bd658 ci: add allowed-endpoints to packaging-lint and house-rules workflows 2026-11-10 08:26:20 -05:00
Dylan M. Taylor 5a43790c93 ci: replace anchore/sbom-action with step-security/sbom-action
Switch to the StepSecurity-maintained secure drop-in replacement
pinned to v0.24.2 (28bb7d0d2869c9ab0e5571c93591747945819d39).
2026-11-10 08:26:20 -05:00
Dylan M. Taylor df5f1fef9a ci: specify allowed-endpoints for block mode in workflows 2026-11-10 08:26:20 -05:00
Dylan M. Taylor 80cb1d6a31 ci: switch Harden-Runner egress-policy to block mode 2026-11-10 08:26:20 -05:00
Dylan M. Taylor 529d68aa05 test: account for Harden-Runner step in workflow assertions
Update unit test step inspections and unpacks to locate steps by name/action
or ignore the leading Harden-Runner security step.
2026-11-10 08:26:20 -05:00
Dylan M. Taylor 83e81167c6 ci: add StepSecurity Harden-Runner to every workflow job
Adds step-security/harden-runner (v2.22.4, SHA-pinned) as the first step of
every job in the reusable and repo workflows, in egress-policy: audit mode.
Audit mode only records network, file and process activity; it blocks nothing.
Unsupported environments (containers, FreeBSD VM guests, ubuntu-slim) are
skipped by the action itself. Dependabot's github-actions group keeps the pin
current.
2026-11-10 08:26:20 -05:00
FredApps 1a6c6c0dd5 fix(docker-build): scope packages: write to the pushing jobs 2026-11-10 11:10:27 +01:00
Dylan M. Taylor fd14193f06 fix(docker-build): set up Buildx before registry logins (#181) 2026-11-08 20:16:57 -05:00
Dylan M. Taylor c9f7255114 chore(docker-build): remove DigitalOcean Container Registry support (#180) 2026-11-08 20:10:10 -05:00
Dylan M. Taylor a5eda472ea feat(docker-build): support Docker Hub registry alongside DOCR and GHCR (#179) 2026-11-08 19:33:37 -05:00
Dylan M. Taylor 3b3ea7ed62 feat(release): dual-push Flatpak OCI layouts to Docker Hub (#178)
* feat(release): dual-push Flatpak OCI layouts to Docker Hub

* fix(release): derive Docker Hub repo name from REGISTRY
2026-11-08 19:11:04 -05:00
FredApps 4b23a68072 fix(packaging-lint): retry transient actionlint downloads (#176)
Bound archive and checksum download retries, including refused connections, while retaining HTTPS and SHA-256 validation. Report missing checksum entries using an exact archive filename lookup. Add nine real-curl regression tests for recovery, permanent failures, retry bounds and checksum validation.
2026-11-08 14:16:10 +01:00
FredApps 8fb29161c2 fix(sonar): reduce sharded coverage disk usage (#174)
* fix(sonar): bound sharded coverage disk usage

* test(signing): detect restricted GPG agent sockets
2026-11-08 14:15:27 +01:00
Dylan M. Taylor 8e9b8b96c5 fix: support rpm 4.20+ signing and pin release job to ubuntu-24.04 (#175)
* fix(draft-release): use rpmsign --resign to support RPM 4.20+

* fix(release): run Draft GitHub Release on ubuntu-24.04
2026-11-07 20:48:33 -05:00
Dylan M. Taylor 626427bb5f Build without provenance so digests match across registries (#173)
An attestation names the repository it was pushed to, so DOCR and GHCR received
different digests and the merge job could only tag one of them.
2026-11-07 15:22:47 -05:00
Dylan M. Taylor a9cdd6881c Push every image by digest in the docker build (#172)
buildx pushes a by-digest output to one repository per invocation, so with two
imagerepositories only the last one received the manifest and the merge job failed
with "not found". Loop over the image list and assert the digests agree.
2026-11-07 15:18:14 -05:00
Dylan M. Taylor 4acf645c17 Give the composed build output its image type (#171)
buildx requires a `type=` prefix in `outputs`; without it every caller failed
with "type is required for output" before the first layer was built.
2026-11-07 15:13:09 -05:00
Dylan M. Taylor 18c197bc08 Add a reusable docker build workflow (#170)
Builds every platform in `platforms` by digest into each repository in `images`,
then joins those digests into one multi-arch manifest per repository, logging in
to DOCR, GHCR or dhi.io only when an image needs it. It carries the inputs both
craft web images need: a custom Dockerfile and stage, build args, a GHA layer
cache, preinstalled-SDK pruning, and a hook for preparing large build contexts
such as the font sources.
2026-11-07 14:29:07 -05:00
Dylan M. Taylor 3e2ac193fd feat: drop Windows 32-bit builds (#163)
Drops i686-pc-windows-msvc (x86) builds from the Windows release matrix, retaining 64-bit x64 and arm64 targets.
2026-11-06 21:27:06 -05:00
Dylan M. Taylor 2f46576245 feat: allow uploading assets to an existing published release (#164) (#165)
Add allow-published (default false) to draft-release. When true, draft-release
permits uploading assets to an existing published release without failing or
attempting to convert it to a draft with --draft.
2026-11-06 20:40:48 -05:00
Dylan M. Taylor aa106c86ee feat: allow additional Arch runtime dependencies (#161) 2026-11-06 19:33:10 -05:00
Dylan M. Taylor 489222f309 feat: optionally retain existing draft release assets (#160) 2026-11-06 19:24:50 -05:00
Dylan M. Taylor dd3d987521 feat: allow a custom draft release tag prefix (#159) 2026-11-06 16:06:01 -05:00
dependabot[bot] 91428662bf ci(deps): bump the actions group with 2 updates (#154)
Bumps the actions group with 2 updates: [taiki-e/install-action](https://github.com/taiki-e/install-action) and [SonarSource/sonarqube-scan-action](https://github.com/sonarsource/sonarqube-scan-action).


Updates `taiki-e/install-action` from 2.89.1 to 2.90.0
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/install-action/compare/682ce6c3c696a4cd5168a05f98e51aa02b877941...b80e2bb2635f28eb98b9fce1e2f0887e1535c6b2)

Updates `SonarSource/sonarqube-scan-action` from 8.4.0 to 8.4.1
- [Release notes](https://github.com/sonarsource/sonarqube-scan-action/releases)
- [Commits](https://github.com/sonarsource/sonarqube-scan-action/compare/4d1dc0c91ee400dcf8a6dddf2f705ac36542e304...e805d875275f728fac1885b413934be6d7c63f65)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.90.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
- dependency-name: SonarSource/sonarqube-scan-action
  dependency-version: 8.4.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-06 07:36:43 -05:00
nnaannooandnnaannoo 13f3c2c99b release.yml: export each app's build variables from its env.sh, so a new app needs no change here (#148)
* release.yml: export each app's build variables from its env.sh, not a list

Eight release jobs set <APP>_VERSION, <APP>_BUILD_DATE and <APP>_BUILD_SHA for
every app by name (298 lines), and the FreeBSD VM step listed them all again,
so every new app meant editing this workflow. materialcraft and texturecraft
were never added: their packaging reads MATERIALCRAFT_* and TEXTURECRAFT_*,
so their release builds got no release version, date or commit, and a
dispatched run with a version override packaged them under Cargo.toml's.

actions/export-app-build-env reads the prefix the app's packaging/env.sh (or
packaging/common.sh) already declares in VERSION="${<PREFIX>_VERSION:-...}",
falling back to app-name in capitals, and writes the three variables to
GITHUB_ENV; its names output feeds the FreeBSD VM's envs. Each job runs it
right after checkout. On all 17 apps' env.sh it finds the prefix the old
list used.

Tests run the script on sample env.sh files and check release.yml sets no
app's variable by name, so the list can't come back.

* Export the build variables after the contributor snapshot, which must follow the checkout

test_contributor_refresh requires each build job to load the contributor snapshot directly after checkout, so the export step goes after it when a job has one. Also fix the line-break test passing a variable twice.

---------

Co-authored-by: nnaannoo <n4nosaur@gmail.com>
2026-11-06 07:36:30 -05:00
FredApps e8500913a3 Merge pull request #153 from storytold/house-rules-check
Check that AGENTS.md copies the shared house rules
2026-11-06 11:45:26 +01:00
FredAppsandCursor 3207fcf8ad Keep the house-rules rewrite on the checked AGENTS.md path.
The path is resolved and has to stay inside the named directory. The existing file's text is written back as text, so the path-traversal finding on that write is suppressed.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-11-06 11:42:33 +01:00