Document gaps to real Guix FreeBSD builds
This commit is contained in:
@@ -302,3 +302,67 @@ Next recommended step:
|
||||
1. document the concrete remaining gap between these Scheme-driven phase-runner prototypes and a true Guix package/derivation/store-daemon build on FreeBSD, especially around store management, implicit inputs, and build isolation
|
||||
2. or choose a somewhat more demanding GNU package with non-trivial declared inputs to identify the first builder-side FreeBSD adaptation points
|
||||
3. continue keeping `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear
|
||||
|
||||
## 2026-04-01 — Phase 1.2 follow-up: documented the gap to a real Guix package build
|
||||
|
||||
Completed work:
|
||||
|
||||
- analyzed the concrete remaining gap between the current FreeBSD validation harnesses and a real Guix package/derivation/store-daemon build
|
||||
- wrote the analysis to:
|
||||
- `docs/reports/phase1-guix-build-gap-analysis.md`
|
||||
- based the analysis on the current local Guix source tree, including the relevant host-side and daemon-side code paths in:
|
||||
- `guix/packages.scm`
|
||||
- `guix/build-system/gnu.scm`
|
||||
- `guix/gexp.scm`
|
||||
- `guix/store.scm`
|
||||
- `guix/derivations.scm`
|
||||
- `gnu/packages/commencement.scm`
|
||||
- `doc/guix.texi`
|
||||
- `nix/libstore/build.cc`
|
||||
- compared those real Guix layers against the currently validated FreeBSD prototypes:
|
||||
- shell-driven native GNU Hello harness
|
||||
- Scheme-driven GNU Hello builder-phase runner
|
||||
- Scheme-driven GNU which builder-phase runner
|
||||
|
||||
Main conclusions recorded:
|
||||
|
||||
- the current FreeBSD work has validated a narrow but real builder-side slice of Guix execution:
|
||||
- Guile can run the needed Scheme code when using the fixed local build
|
||||
- `(guix build gnu-build-system)` phases can build small GNU packages on FreeBSD
|
||||
- however, the current prototypes still bypass several critical layers of a real Guix build:
|
||||
- package -> bag lowering
|
||||
- bag -> derivation lowering
|
||||
- imported module closure/store materialization
|
||||
- real daemon RPC and build submission
|
||||
- canonical `/gnu/store` management and metadata
|
||||
- build users, chroot/container or jail-style isolation
|
||||
- substitute/graft/offload handling
|
||||
- documented that the current phase runners rely on host tools already present on FreeBSD, whereas real `gnu-build-system` uses implicit inputs drawn from `%final-inputs`
|
||||
- identified the most actionable next milestone as a derivation-generation investigation for a tiny package, to locate the first failure boundary among:
|
||||
- host-side lowering
|
||||
- store interaction
|
||||
- derivation emission
|
||||
- daemon availability
|
||||
- daemon-side execution
|
||||
|
||||
Current assessment:
|
||||
|
||||
- Phase 1.2 now has both practical build validation and a clearer architectural map of what remains before a true Guix package build can work on FreeBSD
|
||||
- the project has reduced uncertainty around builder-side GNU phase portability
|
||||
- the next uncertainty is now specifically the host-side lowering/store/daemon boundary, not the builder-phase boundary
|
||||
|
||||
Recent commits:
|
||||
|
||||
- `e380e88` — `Add FreeBSD Guile verification harness`
|
||||
- `cd721b1` — `Update progress after Guile verification`
|
||||
- `27916cb` — `Diagnose Guile subprocess crash on FreeBSD`
|
||||
- `02f7a7f` — `Validate local Guile fix on FreeBSD`
|
||||
- `4aebea4` — `Add native GNU Hello FreeBSD build harness`
|
||||
- `c944cdb` — `Validate Guix builder phases on FreeBSD`
|
||||
- `0a2e48e` — `Validate GNU which builder phases on FreeBSD`
|
||||
|
||||
Next recommended step:
|
||||
|
||||
1. investigate whether a tiny package can be lowered far enough on FreeBSD to produce a real derivation, and capture the exact first failure point
|
||||
2. if derivation generation proves immediately blocked, document whether the blocker is generated Guix modules/configuration, store connectivity, or daemon assumptions
|
||||
3. continue keeping `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear
|
||||
|
||||
298
docs/reports/phase1-guix-build-gap-analysis.md
Normal file
298
docs/reports/phase1-guix-build-gap-analysis.md
Normal file
@@ -0,0 +1,298 @@
|
||||
# Phase 1.2 follow-up: gap analysis between current FreeBSD prototypes and a real Guix package build
|
||||
|
||||
Date: 2026-04-01
|
||||
|
||||
## Summary
|
||||
|
||||
This step documents the concrete remaining gap between the current FreeBSD validation harnesses and a true Guix package/derivation/store-daemon build.
|
||||
|
||||
The repository now has three relevant validation layers:
|
||||
|
||||
- native shell-driven GNU Hello build:
|
||||
- `tests/native-build/run-gnu-hello.sh`
|
||||
- Scheme-driven GNU Hello builder-phase prototype:
|
||||
- `tests/native-build/gnu-hello-guix-phase-runner.scm`
|
||||
- `tests/native-build/run-gnu-hello-guix-phase-runner.sh`
|
||||
- Scheme-driven GNU which builder-phase prototype:
|
||||
- `tests/native-build/gnu-which-guix-phase-runner.scm`
|
||||
- `tests/native-build/run-gnu-which-guix-phase-runner.sh`
|
||||
|
||||
Those prototypes prove that FreeBSD can already execute a small but real subset of Guix builder-side GNU build logic when paired with the fixed local Guile build.
|
||||
|
||||
They do **not** yet prove that FreeBSD can perform an actual Guix package build.
|
||||
|
||||
## Current validated surface
|
||||
|
||||
What is already validated on FreeBSD amd64:
|
||||
|
||||
1. Guile can run the necessary Scheme code, provided the locally fixed Guile build is used.
|
||||
2. Guix builder-side `(guix build gnu-build-system)` code can run a subset of `%standard-phases`.
|
||||
3. Two small GNU packages build successfully through that subset:
|
||||
- GNU Hello
|
||||
- GNU which
|
||||
4. The following builder phases have been exercised successfully:
|
||||
- `set-SOURCE-DATE-EPOCH`
|
||||
- `unpack`
|
||||
- `configure`
|
||||
- `build`
|
||||
- `check`
|
||||
- `install`
|
||||
|
||||
This means the question is no longer "can FreeBSD run any Guix builder-side GNU phase logic at all?"
|
||||
|
||||
The question is now "what still separates these prototypes from a real Guix package build?"
|
||||
|
||||
## What a real Guix package build does beyond the current prototypes
|
||||
|
||||
At a high level, a real Guix package build goes through these layers:
|
||||
|
||||
1. package object -> bag lowering
|
||||
- `package->bag` in `~/repos/guix/guix/packages.scm`
|
||||
2. bag -> derivation lowering
|
||||
- `bag->derivation` in `~/repos/guix/guix/packages.scm`
|
||||
3. build-system lowering and builder gexp creation
|
||||
- `lower` and `gnu-build` in `~/repos/guix/guix/build-system/gnu.scm`
|
||||
4. gexp residualization into store items and imported module closures
|
||||
- `lower-object` in `~/repos/guix/guix/gexp.scm`
|
||||
5. daemon connection and build request submission
|
||||
- `open-connection` and `set-build-options` in `~/repos/guix/guix/store.scm`
|
||||
- `build-derivations` in `~/repos/guix/guix/derivations.scm`
|
||||
6. daemon-side execution with store management, build users, and isolation
|
||||
- `guix-daemon` behavior documented in `~/repos/guix/doc/guix.texi`
|
||||
- daemon implementation details in `~/repos/guix/nix/libstore/build.cc`
|
||||
|
||||
The current FreeBSD prototypes only validate a slice of step 3, and they do so in a manually constructed environment.
|
||||
|
||||
## Gap matrix
|
||||
|
||||
### 1. Package lowering is not exercised yet
|
||||
|
||||
Current prototypes:
|
||||
|
||||
- call `gnu-build` directly
|
||||
- manually provide the source tarball URL/hash context
|
||||
- manually choose outputs and phases
|
||||
|
||||
Missing real Guix behavior:
|
||||
|
||||
- no `<package>` object is lowered through `package->bag`
|
||||
- no implicit inputs are attached by `lower`
|
||||
- no `package->derivation` call occurs
|
||||
- no real `.drv` file is generated
|
||||
|
||||
Why this matters:
|
||||
|
||||
A large amount of actual Guix behavior lives above the builder-phase layer. The package record, arguments, native inputs, propagated inputs, outputs, search paths, and implicit inputs are all resolved before the builder runs.
|
||||
|
||||
FreeBSD implication:
|
||||
|
||||
The current prototypes show that builder-side Scheme logic works, but they do not yet show that Guix's host-side package lowering logic can successfully target FreeBSD.
|
||||
|
||||
### 2. Implicit GNU build-system inputs are bypassed
|
||||
|
||||
In real Guix, `gnu-build-system` adds implicit inputs through `standard-packages` in `~/repos/guix/guix/build-system/gnu.scm`, which ultimately comes from `%final-inputs` in `~/repos/guix/gnu/packages/commencement.scm`.
|
||||
|
||||
That implicit input set includes toolchain and utility packages such as:
|
||||
|
||||
- `tar`
|
||||
- `gzip`
|
||||
- `bzip2`
|
||||
- `file`
|
||||
- `diffutils`
|
||||
- `patch`
|
||||
- `findutils`
|
||||
- `gawk`
|
||||
- `zstd`
|
||||
- `sed`
|
||||
- `grep`
|
||||
- `xz`
|
||||
- `coreutils`
|
||||
- `make`
|
||||
- `bash`
|
||||
- `ld-wrapper`
|
||||
- `binutils`
|
||||
- `gcc`
|
||||
- `libc`
|
||||
- `libc:static`
|
||||
|
||||
Current prototypes instead rely on host tools already present on FreeBSD, such as:
|
||||
|
||||
- `fetch`
|
||||
- `sha256`
|
||||
- `bsdtar`
|
||||
- `/bin/sh`
|
||||
- `/usr/bin/install`
|
||||
- host `cc`
|
||||
- host `make`
|
||||
|
||||
Why this matters:
|
||||
|
||||
A real Guix package build is not just "run the phases somehow". It runs them with a specific Guix-managed tool universe and search-path model.
|
||||
|
||||
FreeBSD implication:
|
||||
|
||||
The current prototypes do not yet answer whether Guix's implicit input model can be lowered, realized, and used coherently on FreeBSD.
|
||||
|
||||
### 3. Imported module closure and build-side module materialization are bypassed
|
||||
|
||||
Real `gnu-build` in `~/repos/guix/guix/build-system/gnu.scm` builds a gexp that uses `with-imported-modules` and `gexp->derivation` so that builder-side modules become store material.
|
||||
|
||||
Current prototypes instead:
|
||||
|
||||
- set `GUILE_LOAD_PATH` directly to `~/repos/guix`
|
||||
- load Guix modules straight from the checkout
|
||||
- do not build/store an imported module closure
|
||||
- do not validate generated Guix configuration modules such as `guix config`
|
||||
|
||||
Why this matters:
|
||||
|
||||
This is one of the biggest remaining gaps. Our prototypes prove that the builder-side code itself can run, but they do not prove that Guix can correctly package up the builder program, its modules, and its imported dependencies as a derivation input graph.
|
||||
|
||||
FreeBSD implication:
|
||||
|
||||
Before a true Guix package build can work, FreeBSD will need a functioning path for:
|
||||
|
||||
- imported Guix Scheme modules
|
||||
- build-side module resolution in a store context
|
||||
- derivation generation for those module closures
|
||||
|
||||
### 4. The store is still only simulated
|
||||
|
||||
Current prototypes use store-like output names under `/tmp`, for example:
|
||||
|
||||
- `/tmp/.../0000000000000000-hello-2.12.3`
|
||||
- `/tmp/.../0000000000000000-which-2.21`
|
||||
|
||||
Missing real Guix behavior:
|
||||
|
||||
- no canonical `/gnu/store`
|
||||
- no true content-addressed store path computation
|
||||
- no `.drv` path registration
|
||||
- no store database updates
|
||||
- no garbage-collection roots
|
||||
- no validity registration or reference graph maintenance
|
||||
|
||||
Why this matters:
|
||||
|
||||
Guix's reproducibility, dependency tracking, and garbage collection all depend on the real store, not just on an output directory that looks store-like.
|
||||
|
||||
FreeBSD implication:
|
||||
|
||||
A real FreeBSD port must eventually establish a functioning `/gnu/store` equivalent with daemon-managed metadata, not just staged output directories.
|
||||
|
||||
### 5. No daemon RPC path has been validated
|
||||
|
||||
Real Guix clients talk to the daemon using `open-connection` in `~/repos/guix/guix/store.scm` and submit builds through `build-derivations` in `~/repos/guix/guix/derivations.scm`.
|
||||
|
||||
Current prototypes:
|
||||
|
||||
- do not connect to a daemon
|
||||
- do not negotiate the daemon protocol
|
||||
- do not set build options via `set-build-options`
|
||||
- do not submit derivations for building
|
||||
|
||||
Why this matters:
|
||||
|
||||
Even if derivation generation were to succeed on FreeBSD, Guix still needs an operational daemon path to realize builds and manage the store.
|
||||
|
||||
FreeBSD implication:
|
||||
|
||||
At some point the port must stop being "Scheme code can run locally" and become "a Guix client can ask a daemon to build something".
|
||||
|
||||
### 6. No real build isolation or build-user model is in place
|
||||
|
||||
The Guix manual section on `guix-daemon` documents that builds normally run:
|
||||
|
||||
- under dedicated build users from `--build-users-group`
|
||||
- in a chroot containing only declared dependencies
|
||||
- and on GNU/Linux, inside additional namespaces/containers
|
||||
|
||||
The daemon code in `~/repos/guix/nix/libstore/build.cc` also clearly depends on:
|
||||
|
||||
- a build-users group
|
||||
- UID switching
|
||||
- temporary build directories
|
||||
- chroot setup
|
||||
- Linux container features when available
|
||||
|
||||
Current prototypes:
|
||||
|
||||
- run as the current user
|
||||
- do not switch to build users
|
||||
- do not restrict filesystem visibility to declared inputs
|
||||
- do not use chroot
|
||||
- do not use jails
|
||||
- do not isolate network access
|
||||
|
||||
Why this matters:
|
||||
|
||||
This is the largest architectural gap relative to the final FreeBSD goal.
|
||||
|
||||
FreeBSD implication:
|
||||
|
||||
The current results are strong evidence for builder-side portability, but they say almost nothing yet about the eventual jail/build-user/isolation design required for a proper port.
|
||||
|
||||
### 7. Substitutes, grafts, and offloading are untouched
|
||||
|
||||
Real Guix package builds may involve:
|
||||
|
||||
- substitutes
|
||||
- grafts
|
||||
- daemon-side build options
|
||||
- offloading
|
||||
|
||||
Relevant code paths appear in:
|
||||
|
||||
- `package->derivation` in `~/repos/guix/guix/packages.scm`
|
||||
- `set-build-options` in `~/repos/guix/guix/store.scm`
|
||||
- `guix-daemon` documentation in `~/repos/guix/doc/guix.texi`
|
||||
|
||||
Current prototypes:
|
||||
|
||||
- always build locally
|
||||
- do not use substitutes
|
||||
- do not test graft behavior
|
||||
- do not exercise offload logic
|
||||
|
||||
Why this matters:
|
||||
|
||||
These are not the first blockers for a minimal FreeBSD prototype, but they remain part of the full Guix execution model.
|
||||
|
||||
## Practical conclusion
|
||||
|
||||
The current FreeBSD work has successfully de-risked the following narrow question:
|
||||
|
||||
> Can FreeBSD run Guix builder-side GNU build phases for small packages when Guile is fixed?
|
||||
|
||||
Answer: yes.
|
||||
|
||||
The next blockers are now clearly above that layer:
|
||||
|
||||
1. package lowering to bags/derivations
|
||||
2. imported module/store materialization
|
||||
3. real store and daemon connectivity
|
||||
4. daemon-side isolation and build-user execution
|
||||
|
||||
## Most actionable next milestone
|
||||
|
||||
The smallest high-value milestone after this analysis is likely:
|
||||
|
||||
### Minimal milestone: derivation-generation validation without claiming full build support
|
||||
|
||||
Try to validate that FreeBSD can at least get through the host-side lowering path for a tiny package far enough to produce a derivation or identify the first concrete failure point.
|
||||
|
||||
That would answer a much sharper question than the current prototypes:
|
||||
|
||||
> Does FreeBSD fail first in host-side Guix lowering, in store interaction, or in daemon execution?
|
||||
|
||||
This is the narrowest next step that meaningfully approaches a real Guix package build.
|
||||
|
||||
## Recommended next step
|
||||
|
||||
Proceed with a targeted derivation-generation investigation for a tiny package, using the current GNU Hello package as the first candidate if practical, and record the exact first failure boundary among:
|
||||
|
||||
- missing generated Guix modules/configuration
|
||||
- store connection assumptions
|
||||
- derivation emission
|
||||
- daemon availability
|
||||
- daemon-side execution assumptions
|
||||
Reference in New Issue
Block a user