Keep DRM/KMS support in tribes-linux so the kexec installer can show a local console on Proxmox/QEMU and generic real hardware.
This restores VGA console output while retaining the other kernel slimming options.
Build HAProxy with Linux capability support and preserve cap_net_bind_service in the generated service configuration so QUIC can bind privileged port 443 after HAProxy drops to the haproxy user.
Do not eagerly load Hyper-V, VMware, or Xen frontend modules in the kexec installer initrd. The broadened compatibility list caused Hetzner CPX31 kexec boots to fall back to the Ubuntu disk before installer SSH became reachable; the KVM/QEMU and common emulated NIC subset boots successfully.\n\nAlso require cpio in the local kexec image build script so missing cpio fails before writing an incomplete image.
The deployed node materializes a real operating-system via `guix system
init`/reconfigure, so the authoritative substitute set is the closure of
that operating-system -- which CI already builds in substitute-system-jobs.
The hand-curated manifests/substitutes/{base,installer,tribes-node}.scm only
approximated a node's closure and drifted from it.
Add (tribes ci artifacts-substitutes), emitting the node OS closures and
bootloader files for the guix-tribes-substitutes jobset, and move those jobs
out of artifacts-master (now just the docker image and sender-runtime pack).
Building a node's full closure pulls every transitive upstream dependency
into our store, so the mirror keeps serving everything an install needs
without depending on bordeaux/ci. To preserve mirroring of the operator
toolkit -- which lived only in the old tribes-node manifest, not in any
system -- carry ripgrep/fd/tmux/neovim/btop in the node system profile, so
they are both installed on nodes and covered by the closure.
Drop the redundant manifests/substitutes/*.scm.
Include common Xen, Hyper-V, VMware, virtio, and emulated QEMU storage/network modules in the kexec installer initrd.\n\nAlso log to both VGA and serial consoles and raise kernel log verbosity so manual-host kexec failures are easier to diagnose.
Reconstruct public channel URLs for the substitute-channel-profile job before calling latest-channel-derivation. Cuirass passes custom jobs store-checkout URLs, which do not match the profile Legion installers compute with guix time-machine -C channels.scm.
Build the base-edge substitute system through the same host-configuration plus system-facts materialization path used by Legion installs. This keeps the existing Cuirass substitute-system-base-edge job but makes it warm the installer module graph more closely.
The channel-instance build compiles every .scm file in the channel as
a Guile module; a bare-script sources.scm with top-level use-modules +
(manifest ...) failed compilation with 'no code for module (manifests
sources)'. Mirror the dual-use shape of manifests/substitutes/*.scm:
declare define-module, bind the manifest to a name, and end with a
bare reference so primitive-load still returns the manifest value.
fold-packages lives in (gnu packages), not (guix packages); the
sources canary manifest was missing the import and aborted with an
unbound-variable on evaluation.
Restructure the Cuirass jobs so each concern lives in its own spec,
mirroring how ci.guix.gnu.org partitions things:
* tribes/ci/channel.scm: new entry point emitting only the
channel-instance derivation -- a cheap canary that fails fast
when channel modules don't compile or the checkout is unreachable.
* manifests/sources.scm: new manifest of upstream origins for every
package defined by the tribes (and nbde) channel modules. Drives
a network canary that catches stale upstream URLs / commit refs
(e.g. force-pushed pins) without depending on actual package builds.
* tribes/ci/artifacts-master.scm: drop channel-profile and
substitute-manifest jobs -- they move out to their own specs
consumed via Cuirass's built-in 'manifests build type.
* tribes/ci/artifacts.scm: remove the now-dead substitute-manifest
plumbing along with its imports and exports.
The accompanying Cuirass spec list (deployed separately) registers
guix-tribes-channel, guix-tribes-source, guix-tribes-substitutes,
guix-tribes, and guix-tribes-kexec-installer.
Cuirass rewrites each channel URL to the /gnu/store/<hash>-<name>-<commit>
checkout it has already fetched, but those derivation outputs are flat
working trees with no .git/ directory. latest-channel-derivation calls
latest-channel-instances, which runs libgit2's update-cached-checkout on
the URL and crashes with GIT_ENOTFOUND, aborting the entire evaluation.
Skip the (re-)fetch by constructing channel-instance records directly from
the already-resolved (url, commit) pair via the public
checkout->channel-instance helper, then feed those to
channel-instances->derivation -- the same path "guix pull" uses internally.
Add a Cuirass artifact job for the Guix channel profile produced from the evaluated channel arguments. This restores the pre-Cuirass warmup coverage for the time-machine profile used by installers.
Log PostgreSQL statements slower than one second on Tribes nodes. The existing syslog-ng configuration already routes postgres syslog entries to /var/log/postgresql.log and the combined JSONL log, which supertest remote log capture collects from /var/log.
Live mixed-provider supertest runs showed control-lane SYNC-PAGE handlers timing out while waiting for Parrhesia DB read connections under catch-up load. Set larger Parrhesia main/read pool sizes in the Tribes service environment so sync-page queries can proceed concurrently during node joins.
The kexec-installer jobset shared (tribes ci artifacts) cuirass-jobs
with master, so its evaluations re-emitted the docker/sender-runtime
and substitute jobs from whatever the master branch's checkout
happened to look like, while the kexec image itself stopped being
emitted once that helper was simplified.
Split into two thin per-jobset entry points:
- (tribes ci artifacts-master): docker + sender-runtime + substitutes
- (tribes ci artifacts-kexec): guix-kexec-installer only
The shared module now just exports the helpers and image builders.
Have the local-control helper report the pulled profile's guix describe channel data after guix pull. Generation records now store exact commits for branch-based channel plans, and generations responses expose current system channel provenance as a fallback for initial installs.
Add a local-control endpoint that uses Guix channel Git checkouts to fetch refs and list semver tag candidates for configured channels. Persist resolved channel pins on local-control generation records so Tribes can compare available updates with the commit actually prepared and activated.
Remove the legacy kexec-installer sexp pin and stop the normal master artifact jobset from producing kexec installer archives; the kexec-installer branch now selects the installer source commit.
Match the old kexec image builder by excluding already-copied boot artifacts from the embedded store squashfs. The Cuirass derivation was including the installer initrd and parameters store items from the system references graph, adding about 50 MiB to the archive.
Exclude CI-only modules from the tribes-command package build so it no longer tries to compile modules that depend on top-level substitute manifests.
Include the trust plugin in the kobold CI target because kobold requires the alliance trust capability, and add a regression test for the target plugin set.
Export Cuirass custom jobs for release artifacts, substitute warmup manifests, deployment system targets, and bootloader files. This lets the Guix container delegate artifact and substitute builds to Cuirass instead of the sync script.
Replace the generic Nixpkgs reference with per-option rationale tied to Tribes deployment behavior. Drop CONFIG_DEBUG_LIST because CONFIG_BUG_ON_DATA_CORRUPTION already selects the cheaper LIST_HARDENED checks without enabling heavier debug-list machinery.
Avoid embedding the already-copied kexec initrd and boot parameters in the squashfs store closure while keeping the system path and runtime closure available. Add conservative extra tribes-linux trims for niche mesh/IoT network stacks and rare local filesystems.
Add a shared tribes-linux package with conservative hardening and server/text-console slimming. Wire build-host, kexec installer, and installed NBDE systems to use it, and assert substitute targets use the shared kernel.
Add a sender-ffmpeg package inheriting FFmpeg 8.0 with the headless streaming codec/protocol subset and without display/GPU stacks. Use it for the Sender plugin extra package and the external sender runtime pack, while preserving fallback lookup for existing stock ffmpeg plugin packages.
Define a Guix tarball pack for external Sender runtime nodes, including ffmpeg, Vinyl, node_exporter, vmagent, and a private Shepherd launcher. The launcher defaults to /etc/tribes/sender-runtime.env and includes an example env file plus optional boot hook installation.
Refresh the Tribes source pin and plugin pins after fixing update-plugin-pin to hash Sender dependencies with the same setup used by the package definition.
Make update-plugin-pin derive its dependency hashing setup from the plugin package's reuse-host-libs and include-mix-deps settings instead of always injecting the host source setup.
Move the shared Guix base channel to guix-fork refactor/substituter-trace-framing at 83b0e7d44546968002fb0c0043004da4e9bedc0d and use that signed commit as the channel introduction.
Add a reusable tribes/development namespace with a development manifest and tribes-dev helper. Move the Node 24 package into guix-tribes so development shells and downstream packages can share it.
Adds (tribes packages npm) with fetch-npm-deps (network-enabled
fixed-output node_modules tree via npm ci, in the style of
fetch-mix-deps) and npm-binary-package (extracts an npm tarball or
source dir, drops in the prefetched node_modules, and creates /bin
wrappers that invoke node with PATH-INPUTS prepended).
Both helpers take a #:node keyword so callers can pin against a
specific node version (e.g. node-24).
Resolve disabled plugins into the Guix generation while passing their names to Tribes as runtime-disabled, preserving package/schema/data for later re-enable.
Bump Tribes, Kobold, and Trust to the commits that add generic access policy support and Trust-gated Kobold dataset access.\n\nPin hashes were refreshed with pguix after the local Guix build could not resolve cached Hex metadata offline.
When local-control writes rollout channels from a SystemTarget plan, preserve the current core guix channel if the plan only contains the Tribes plugin channel.\n\nGuix pull requires a channel named guix; SystemTarget channel entries currently model the Tribes channel selected for plugin/package resolution, so overwriting channels.scm with only that channel breaks prepare.
Limit plugin catalog discovery to tribes/plugins modules present on the active Guix load path.\n\nThe local-control catalog path runs under guix repl where channel modules are available on %load-path but not necessarily in %package-module-path, causing fold-packages to return an empty catalog.\n\nAlso update the deploy executor test expectation for the current aether registry version.
Pin tribes-plugin-supertest to the scheduler probe fixture commit and refresh its source and mix dependency hashes.\n\nValidated tribes-plugin-supertest on pguix.
Handle keyword-style #:provides and #:requires entries in plugin package definitions, including both pinned and local package variants.\n\nVerified update-plugin-pin with --use-build-host pguix for supertest and sender temp checkouts.
Move the Guix Tribes source pin to the monorepo commit that prepares tribes_plugin_api as a standalone Hex package.\n\nAdd a tribes-plugin-api Guix package from the monorepo subproject and install built-in plugin ebin files by manifest otp_app so tribes_ui assembles correctly.\n\nValidated on pguix with builds for tribes, tribes-plugin-api, external plugin packages, and an assembled tribes-full-with-plugins expression.
Derive plugin registry records from package metadata, reuse host release libs for plugin compilation, and make native/private Mix deps opt-in.\n\nUpdate the Tribes pin to the scheduler commit and verify plugin builds on pguix without shipping host OTP apps.
Add the Trust plugin package to guix-tribes, include it in plugin pin refreshes and registry substitute packages, and update the Kobold package pin for the dataset foundation.
Generate the plugin catalog dynamically from Guix package metadata exposed by the pulled channel environment. Remove the static catalog service path, write rollout target channels before pull, expose the local-control catalog endpoint, and update the Tribes source pin.
Validate id/slug manifests and fully-qualified capabilities during release assembly, add Kobold to the plugin registry, and
refresh Tribes/plugin pins.
Run the pinned Tribes e2e preparation in a non-login shell so devenv's PATH remains active. The login shell reset PATH and made mix unavailable before dependency preparation.
Install local Hex/Rebar and fetch Mix dependencies in the checked-out pinned Tribes harness before running the Docker e2e script. The e2e script invokes mix run for host-side assertions, which otherwise fails on a fresh checkout without deps.
Use store-qualified tools in the Guix-built Docker entrypoint, seed PATH for the Mix release launcher, and provide a default release cookie for e2e containers. Also normalize Tribes release launcher installation so debug package variants expose bin/tribes-app consistently.
Read the image reference reported by docker load and pass that exact tag to the pinned Tribes e2e harness. This avoids trying to pull an assumed repo:commit tag when guix pack loads the image as repo.commit:latest.
Rewrite Shepherd messages copied through /proc/kmsg to program=shepherd with notice severity and stripped kernel timestamp prefix. Keep the normalized entries in combined JSONL and conventional messages logs without duplicating raw kernel copies.
Run the pinned Docker E2E workflow on the host-Nix runner and reuse the host daemon/store instead of installing Nix inside the job container. This avoids unsandboxed in-container builds and the /homeless-shelter purity failure while keeping the existing gated E2E flow.
Compile Aether and Supertest against the already-built Tribes release libs instead of rebuilding the shared host dependency closure. Keep Sender on the existing standalone path until plugin-only dependency handling is added.
Cancel superseded pinned E2E runs and skip automatic jobs for unsigned agent pushes so only signed self pushes or manual dispatch run the expensive workflow.
The tribes-ci self-hosted label was never declared by the runner, so all
runs sat queued. Switch to the docker-backed ubuntu-22.04 label and
install Nix + devenv inside the job so the e2e harness can run.
The trivial-build-system does not propagate (guix build utils) into the
build sandbox by default, so the builder failed with "no code for module
(guix build utils)" when resolving mkdir-p. Wrap the gexp with
with-imported-modules and move chmod out of call-with-output-file so it
actually runs after the file is closed.
Default pin refresh scripts to local Guix and require an explicit --use-build-host HOST for remote hashing/builds. Update the README to describe current channel contents and pin maintenance commands without local host-specific names.
The channel package cache evaluates tribes-debug-docker-entrypoint during guix pull; import (gnu packages nss) so nss-certs is bound with current Guix.
Add a wrapper that refreshes the Tribes pin and each listed external plugin pin by calling the existing update scripts.
Support an optional --commit flag that stages only the affected pin files and commits them with the requested bump message.
Add a Guix-packed debug Docker image and Gitea workflow for pinned multi-node e2e validation. The debug image is built with admin debug methods enabled and published under debug-specific names so it is not confused with a real release image.
Mix validates Git dependencies with remote.origin.url as well as HEAD during offline builds. Keep a tiny sanitized .git directory for SCM deps so vendored dependency trees remain reproducible without tripping lock mismatch checks.
Update the fixed-output hashes for the raw and prepared Tribes Mix dependency trees.
Remove the guix-tribes sync-overlap-seconds service field and host JSON key so deployments use the Tribes release default instead of carrying a second drift-prone default.
This also removes the example and CI substitute configuration override for TRIBES_SYNC_OVERLAP_SECONDS.
Pin Tribes to the importer fix that recognizes source/type markers after release logger metadata so deployed log roundtrip entries import with structured metadata.
Move Chrony from a tailed file source to native syslog per-service output, and add the Prometheus node exporter Shepherd log to syslog-ng tail inputs. This keeps combined JSONL coverage aligned with the files produced by deployed nodes.
Make the syslog-ng combined JSONL importer input group-readable by the tribes group instead of world-readable. Activation now creates the file as root:tribes 0640, and the syslog-ng destination preserves that group and mode so the Tribes importer can read it without exposing per-service logs broadly.
Keep Sender's development default as ffmpeg on PATH, but when the sender plugin contributes Guix ffmpeg to the Tribes runtime, set SENDER_FFMPEG_EXECUTABLE to the exact store ffmpeg binary. This keeps the deployment model explicit without forcing ffmpeg into base nodes.
Set FILESYSTEM_FSINOTIFY_EXECUTABLE_FILE to the Guix inotify-tools binary and include inotify-tools in the Tribes profile/PATH so the Elixir file_system backend can start on deployed nodes instead of falling back to polling.
Update the Tribes pin to include the JSONL importer fix that treats an unavailable file watcher as a polling fallback on systems without inotify-tools.
Package a minimal syslog-ng build without Python runtime support and wire it in as the system syslogd provider. Write combined Tribes JSONL logs plus conventional and per-service logs directly under /var/log, and update the Tribes source pin for the amended importer default.
Teach the NBDE mapped-device initrd logic to mount the boot partition and try /boot/nbde/local-boot.key at boot time instead of baking local key material into the initrd.
Update system facts defaults, materialisation, tests, and NBDE documentation for the boot-key model.
Define a local Chrony service using Guix's chrony package and the default Guix NTP pool configuration.
Install the service on every Tribes node and add node tests for service inclusion plus rendered chrony.conf defaults.
Add a boot-time Shepherd one-shot that starts the Tribes service once Legion-managed secret files exist. This keeps the first secrets-free install boot explicit-start only, while allowing already-installed nodes to recover after provider reboots.
Document the behavior and assert the service auto-start shape in the node service test.
Install the NBDE boot-store staging hook into every generated node OS so Guix generation switches copy GRUB-referenced store items into the unencrypted /boot partition. This keeps new kernel and initrd store paths bootable when the real store lives on encrypted root.
Also move the provider NIC initrd module list into the shared installed base so immutable rollout snapshots keep the modules that Legion previously injected only during initial install.
Bump vinyl-exporter to v0.2.0 and expose trusted proxy configuration so HLS IP fallback can use X-Forwarded-For from the local HAProxy/Vinyl edge path.
Point the base and Legion channel pin files at guix-fork commit 5205cfb34c2e1c070bc026541ac9ab02319ee8c7 so deployments use the current diagnostic fixes.
Capture stderr together with stdout when invoking Guix commands so malformed progress/reporting failures remain visible to Legion and supertest logs.\n\nAlso log the exact Guix command, exit status, selected binary, profile root, and Guix-related load path environment for future provisioning failures.
Set GUIX_UNINSTALLED while invoking the selected current Guix profile so the packaged guix launcher does not prepend its own older module tree ahead of the profile load paths. This makes child commands such as system switch-generation actually run channel-provided Guix fixes.
Extend the current-guix environment test to cover the new variable and restoration behavior.
Preserve the captured guix system switch-generation output tail in local-control failure details. This keeps the actual Guix error path visible when a rollout fails before the profile switch, as seen in the cluster sender test.
Add explicit BIOS and EFI bootloader configuration targets for substitute prewarming. Plain guix system build does not lower the bootloader config or installer helper paths, so those GRUB artifacts could be missing from our cache even when the OS closure was present.
Use minimal synthetic operating systems for these targets so the bootloader artifacts can be built without pulling the full Tribes node closure.
Point the mirrored Legion kexec installer image at the current signed guix-tribes master commit so the Guix mirror builder publishes a fresh tarball for this channel baseline.
Move the guix-fork channel pin to the signed status trace hardening commit b73ddc8cc1dc530bcabc1227013b9e3be7a257db.
Keep the existing e75105240061d201b7d523018d07e3b230479a26 introduction commit as the channel authentication root.
Configure edge vinyl_exporter to strip the Sender HLS prefix before deriving stream labels so playlist metrics use the Sender stream id instead of collapsing under the generic sender path component.
Expose channel-level operating-system targets for phase1, a base edge node, and a Sender-enabled edge node so the substitute builder can warm full deployment closures, not just package manifests.
Package the Vinyl metrics exporter, run it alongside the edge Vinyl cache, add it to local vmagent scraping and the node substitute manifest, and update the Tribes and sender pins consumed by the node build.
Pin guix-tribes to the Alliance refactor commit and refresh Aether, Sender, and Supertest plugin source hashes.
Also carry the VictoriaMetrics scraper adjustment so vmagent scrapes the Tribes /metrics endpoint on port 4000.
Run Guix's Prometheus node exporter on every default Tribes node, bind it to loopback, and include it in the node substitute manifest. Add the exporter endpoint to vmagent's default scrape config and cover the service wiring in the node system tests.
Package VictoriaMetrics 1.143.0 with the Tribes Go helper, install the single-node server plus vmagent and backup tools, and enable loopback-only VictoriaMetrics/vmagent services on every Tribes node. Add the package to the node substitute manifest and cover the default service wiring in the node system tests.
Add a scoped AWS-LC 1.73.0 package for HAProxy and build HAProxy's QUIC support against it instead of OpenSSL compatibility mode.
Remove the limited-quic runtime knob and update the rendered node config test to assert native QUIC configuration.
Upgrade the HAProxy package to 3.3.10 and build it with QUIC support via the OpenSSL compatibility layer used by the current Guix OpenSSL 3.0 input.\n\nExpose default HTTP/3 UDP listeners, advertise h3 through Alt-Svc, and cover the generated edge proxy config in the node system tests.
Let HAProxy own both public edge listeners so ACME challenge paths are proxied directly to Lego's standalone challenge server while all other HTTP traffic redirects to HTTPS. Remove the HTTP-only Vinyl instance and keep Vinyl focused on the private cache backend.
Add HAProxy 3.3.9 to the Tribes Guix channel and introduce a Shepherd service for the public TLS edge. Wire node edge deployments and Lego reloads to HAProxy while keeping Vinyl's HTTP challenge and cache topology unchanged.
Move rollout dependency resolution off the old hard-coded host capability baseline and onto the same host/built-in plugin contract used by package assembly. The resolver now recognizes the built-in tribes_ui provider for ui@1 and rejects unsatisfied host/built-in contracts before planning external plugin dependencies.
Replace the assembled plugin package builder's get-string-all dependency with a local read-char loop. This keeps the builder module closure compatible with node-side Guix builds that use the rollout Guile package and avoids failures resolving (ice-9 custom-ports).
(cherry picked from commit 8b433f9d90)
Return a bounded tail of captured guix system build output in local-control build_failed frames. This keeps rollout diagnostics actionable when the node is destroyed before the full helper stderr can be inspected.
Treat capabilities provided by built-in Tribes plugins as baseline resolver capabilities so channel plugins that require ui@1 can be previewed and rolled out without adding a separate external UI plugin. Add a sender regression case to the deploy executor tests.
Bump the sender plugin pin to the current packaged source and assert that plugin builds containing muontrap include an executable priv/muontrap binary. The Guix package build now fails early if the native Mix dependency is compiled without its runtime helper.
Add the effective Tribes package and plugin extra package bin directories to the Tribes launcher PATH. This lets runtime plugin code find executables, such as sender resolving ffmpeg from its Guix extra package, after a rollout switches to the plugin-enabled system generation.
Prepare already builds the target system and lowers the Shepherd service-upgrade program, but substituted store items can still leave referenced outputs absent locally. That let activation or Shepherd service loading fetch or build large dependencies during the switch phase.
Realize the target system closure and the lowered service-upgrade gexp closure during prepare with Guix store topological closure walking plus ensure-path. Keep the same closure realization in local-eval as defense-in-depth for any post-switch gexp evaluation.
Compile the in-tree tribes_ui Mix project during the Tribes release build and install its ebin output into the packaged plugin directory so the runtime plugin loader can load the entry module.
Disable wx in the OTP 28 package used by Tribes builds and route Mix/Rebar through the matching Rebar package so server builds do not pull in the wx/GUI dependency graph.
Make plugin builds closer to the host build foundation by avoiding Node unless assets are built and vendoring libsecp256k1 for hermetic NIF compilation. Add diffutils for secp256k1 configure checks.
Add the supertest fixture plugin to the Guix plugin registry so rollout preview can resolve the plugin name from the baseline channel while development continues on the signed supertest-dev branch.
2026-04-30 13:13:01 +02:00
24 changed files with 236 additions and 241 deletions
;; Interactive toolkit installed into every node's system profile. These are
;; vanilla upstream packages an operator reaches for over SSH; shipping them in
;; the system closure means they are both present on the node and served by our
;; substitute mirror, instead of forcing a fallback to bordeaux/ci on first use.
(define %tribes-node-operator-tools
(map specification->package
'("ripgrep""fd""tmux""neovim""btop")))
(define (tribes-node-bbr-servicesconfig)
(if (tribes-node-configuration-enable-bbr?config)
(list
@@ -280,6 +294,9 @@
(simple-service'tribes-node-network-tools
profile-service-type
(list nftables))
(simple-service'tribes-node-operator-tools
profile-service-type
%tribes-node-operator-tools)
(serviceprometheus-node-exporter-service-type
(prometheus-node-exporter-configuration
(packageprometheus-node-exporter)
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
self
