Validate Guix builder phases on FreeBSD

2026-04-01 09:39:18 +02:00
parent 4aebea4dab
commit c944cdba5f
4 changed files with 455 additions and 0 deletions
--- a/docs/PROGRESS.md
+++ b/docs/PROGRESS.md
@@ -169,3 +169,70 @@ Next recommended step:
 1. extend Phase 1.2 with at least one additional representative GNU/autotools package build on FreeBSD, or
 2. prototype a tiny Scheme-based `gnu-build-system`-like phase runner using the known-good local Guile path, starting from the GNU Hello flow
 3. continue keeping `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear
+
+## 2026-04-01 — Phase 1.2 follow-up: Guix builder-side GNU Hello phase runner validated
+
+Completed work:
+
+- added a Scheme-driven GNU Hello build prototype:
+  - `tests/native-build/gnu-hello-guix-phase-runner.scm`
+  - `tests/native-build/run-gnu-hello-guix-phase-runner.sh`
+- required the previously validated fixed local Guile build for this harness because it depends on subprocess-heavy Scheme operations
+- used Guix modules directly from `~/repos/guix`, including:
+  - `(guix base32)`
+  - `(guix build gnu-build-system)`
+  - `(guix build utils)`
+- fetched and hash-verified GNU Hello `2.12.3` again against the Guix package hash:
+  - nix-base32: `183a6rxnhixiyykd7qis0y9g9cfqhpkk872a245y3zl28can0pqd`
+  - SHA256: `0d5f60154382fee10b114a1c34e785d8b1f492073ae2d3a6f7b147687b366aa0`
+- successfully executed a subset of Guix builder-side `%standard-phases` on FreeBSD:
+  - `set-SOURCE-DATE-EPOCH`
+  - `unpack`
+  - `configure`
+  - `build`
+  - `check`
+  - `install`
+- installed GNU Hello into a store-like output path under the temporary work directory rather than using a `/usr/local` `DESTDIR` staging layout
+- executed the resulting binary successfully and confirmed output:
+  - `Hello, world!`
+- captured metadata including:
+  - host triplet
+  - selected phase list
+  - runtime dependencies
+  - test-suite summary
+- wrote the results to `docs/reports/phase1-guix-gnu-hello-phase-runner.md`
+
+Important findings:
+
+- this is the first validation step in the repo that successfully exercised actual Guix builder-side GNU build logic on FreeBSD instead of only a shell approximation
+- the harness works when driven by the fixed local Guile build, confirming that the earlier Guile subprocess-fix validation is directly useful for FreeBSD Guix build orchestration
+- GNU Hello's `make check` test suite also passed in this mode:
+  - total: `7`
+  - pass: `7`
+  - fail: `0`
+- the resulting binary's runtime dependencies differ from the earlier `/usr/local`-prefixed native shell harness; in this store-like output layout it only showed:
+  - `libc.so.7`
+  - `libsys.so.7`
+- that difference is a useful clue that Guix-style output layout/build invocation can materially affect FreeBSD runtime linkage behavior
+
+Current assessment:
+
+- Phase 1.2 now has both:
+  - a shell-driven native GNU Hello build harness, and
+  - a Scheme-driven prototype that uses real Guix builder-side GNU phases
+- this is still short of a true Guix package/derivation build, but it significantly narrows the gap between host validation and real `gnu-build-system` execution on FreeBSD
+- the known-good local Guile path is now validated as part of a practical Guix-adjacent build workflow, not just standalone subprocess diagnostics
+
+Recent commits:
+
+- `e380e88` — `Add FreeBSD Guile verification harness`
+- `cd721b1` — `Update progress after Guile verification`
+- `27916cb` — `Diagnose Guile subprocess crash on FreeBSD`
+- `02f7a7f` — `Validate local Guile fix on FreeBSD`
+- `4aebea4` — `Add native GNU Hello FreeBSD build harness`
+
+Next recommended step:
+
+1. run the Scheme-driven phase-runner pattern against at least one more small GNU/autotools package on FreeBSD, or
+2. document the concrete gaps between this prototype and a real Guix package/derivation build, especially around store management and build isolation
+3. continue keeping `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear
--- a/docs/reports/phase1-guix-gnu-hello-phase-runner.md
+++ b/docs/reports/phase1-guix-gnu-hello-phase-runner.md
@@ -0,0 +1,169 @@
+# Phase 1.2 follow-up: Scheme-driven GNU Hello build with Guix builder phases on FreeBSD
+
+Date: 2026-04-01
+
+## Summary
+
+This step extends the earlier native GNU Hello shell harness by validating a tiny Scheme-driven prototype that uses Guix's actual builder-side GNU build machinery on FreeBSD.
+
+Added files:
+
+- `tests/native-build/gnu-hello-guix-phase-runner.scm`
+- `tests/native-build/run-gnu-hello-guix-phase-runner.sh`
+
+The prototype does not yet build a real Guix package or derivation, but it does execute a subset of Guix's builder-side `%standard-phases` from `(guix build gnu-build-system)` against GNU Hello on FreeBSD.
+
+## Why this step matters
+
+The previous Phase 1.2 step showed that FreeBSD can perform a shell-driven native autotools build of GNU Hello.
+
+This step moves closer to actual Guix execution by validating that FreeBSD can run Guix's Scheme-side builder logic when using a fixed local Guile build.
+
+That is important because the earlier packaged FreeBSD `guile3` binary still crashes in subprocess helpers such as `system*`, `spawn`, and `open-pipe*`. A Scheme-driven build harness therefore needs the locally fixed Guile path validated in Phase 1.1.
+
+## Harness design
+
+The shell wrapper:
+
+- requires a fixed local Guile build
+- automatically prepends the matching sibling `lib` directory to `LD_LIBRARY_PATH`
+- exposes the Guix source tree through `GUILE_LOAD_PATH`
+- creates and cleans up a temporary work directory unless `KEEP_WORKDIR=1`
+
+The Scheme runner:
+
+1. fetches GNU Hello `2.12.3`
+2. translates the Guix nix-base32 hash to hexadecimal using `(guix base32)`
+3. verifies the downloaded tarball with `sha256(1)`
+4. invokes `gnu-build` from `(guix build gnu-build-system)`
+5. runs this phase subset from `%standard-phases`:
+   - `set-SOURCE-DATE-EPOCH`
+   - `unpack`
+   - `configure`
+   - `build`
+   - `check`
+   - `install`
+6. executes the installed `hello` binary
+7. records metadata including test results and runtime dependencies
+
+Unlike the earlier shell harness, this prototype installs directly into a store-like output directory under the temporary work tree, which is closer to actual Guix build behavior.
+
+## Source identity
+
+As in the previous step, this uses the GNU Hello source currently referenced by `~/repos/guix/gnu/packages/base.scm`:
+
+- version: `2.12.3`
+- Guix nix-base32: `183a6rxnhixiyykd7qis0y9g9cfqhpkk872a245y3zl28can0pqd`
+- verified SHA256: `0d5f60154382fee10b114a1c34e785d8b1f492073ae2d3a6f7b147687b366aa0`
+
+## Verification command
+
+```sh
+METADATA_OUT=/tmp/gnu-hello-guix-metadata.txt \
+  ./tests/native-build/run-gnu-hello-guix-phase-runner.sh
+```
+
+The wrapper used the previously validated local fixed Guile build:
+
+```text
+/tmp/guile-freebsd-validate-install/bin/guile
+```
+
+## Result
+
+The Scheme-driven build succeeded on `FreeBSD 15.0-STABLE` amd64.
+
+Observed outcomes:
+
+- source fetch: success
+- source hash verification: success
+- Guix builder phase execution: success
+- `configure`: success
+- `build`: success
+- `check`: success
+- `install`: success
+- runtime execution: success
+
+Observed program output:
+
+```text
+Hello, world!
+```
+
+Observed test summary:
+
+```text
+# TOTAL: 7
+# PASS:  7
+# SKIP:  0
+# XFAIL: 0
+# FAIL:  0
+# XPASS: 0
+# ERROR: 0
+```
+
+## Notable findings
+
+### 1. Guix builder-side GNU phases can run on FreeBSD
+
+This is the first repository step that successfully exercised actual Guix builder-side GNU build logic on FreeBSD rather than only a shell approximation.
+
+The harness successfully called `gnu-build` from `(guix build gnu-build-system)` with a selected subset of `%standard-phases`.
+
+### 2. The local fixed Guile build remains necessary for Scheme-driven build orchestration
+
+This harness depends on subprocess helpers such as `open-pipe*` and `invoke`, so it must not use the packaged FreeBSD `guile3` binary.
+
+Using the fixed local Guile build from earlier Phase 1.1 validation was sufficient.
+
+### 3. Store-like installation changes the resulting runtime linkage profile
+
+The installed binary ended up in a store-like output directory:
+
+```text
+/tmp/.../0000000000000000-hello-2.12.3/bin/hello
+```
+
+Observed runtime dependencies were:
+
+- `libc.so.7`
+- `libsys.so.7`
+
+This differs from the earlier `/usr/local`-prefixed native shell harness, where the staged binary also pulled in `libiconv.so.2`, `libintl.so.8`, and `libthr.so.3`.
+
+This is an important data point: a more Guix-like output layout and build path can produce materially different runtime linkage on FreeBSD.
+
+### 4. The test suite passes in this mode too
+
+Unlike the previous shell harness, this prototype also ran GNU Hello's `make check` phase successfully.
+
+That gives a stronger signal that the Guix builder-side phase sequence is usable on FreeBSD for at least a minimal GNU autotools package.
+
+## What this step demonstrates
+
+This step demonstrates that FreeBSD can support a small but real fragment of Guix build orchestration in Scheme, provided that Guile itself is built from a fixed revision containing the upstream subprocess crash fix.
+
+In practical terms, the host now has validated evidence for all of the following:
+
+1. native shell-driven GNU Hello build works
+2. local fixed Guile can safely drive subprocess-heavy Scheme code on FreeBSD
+3. Guix builder-side GNU phases can build, test, install, and run GNU Hello on FreeBSD
+
+## Remaining limitations
+
+This is still not a full Guix package build:
+
+- no derivation was created
+- no store daemon was involved
+- no build isolation/jail integration was used
+- no dependency graph resolution was performed by Guix package machinery
+- only a subset of `%standard-phases` was exercised
+
+## Recommended next step
+
+Continue Phase 1.2 by broadening coverage in one of two ways:
+
+1. run the same Scheme-driven phase-runner approach against another small GNU/autotools package, or
+2. start documenting the exact Linux-vs-FreeBSD gaps that remain between this prototype and a real `gnu-build-system` build driven by Guix package definitions
+
+Either direction would build directly on the harnesses now present in `tests/native-build/`.
--- a/tests/native-build/gnu-hello-guix-phase-runner.scm
+++ b/tests/native-build/gnu-hello-guix-phase-runner.scm
@@ -0,0 +1,143 @@
+(use-modules (guix base32)
+             (guix build gnu-build-system)
+             (guix build utils)
+             (ice-9 format)
+             (ice-9 match)
+             (ice-9 popen)
+             (srfi srfi-1)
+             (rnrs bytevectors)
+             (rnrs io ports))
+
+(define (getenv* name default)
+  (or (getenv name) default))
+
+(define (trim-trailing-newlines str)
+  (let loop ((len (string-length str)))
+    (if (and (> len 0)
+             (char=? (string-ref str (- len 1)) #\newline))
+        (loop (- len 1))
+        (substring str 0 len))))
+
+(define (command-output program . args)
+  (let* ((port   (apply open-pipe* OPEN_READ program args))
+         (output (get-string-all port))
+         (status (close-pipe port)))
+    (unless (zero? status)
+      (error (format #f "command failed: ~a ~s => ~a"
+                     program args status)))
+    (trim-trailing-newlines output)))
+
+(define (nix-base32->hex str)
+  (string-concatenate
+   (map (lambda (byte)
+          (format #f "~2,'0x" byte))
+        (bytevector->u8-list (nix-base32-string->bytevector str)))))
+
+(define (phase-subset names)
+  (filter (match-lambda
+            ((name . _)
+             (memq name names)))
+          %standard-phases))
+
+(define workdir
+  (or (getenv "WORKDIR")
+      (error "WORKDIR environment variable is required")))
+
+(define hello-version
+  (getenv* "HELLO_VERSION" "2.12.3"))
+(define expected-nix-base32
+  (getenv* "HELLO_NIX_BASE32" "183a6rxnhixiyykd7qis0y9g9cfqhpkk872a245y3zl28can0pqd"))
+(define source-url
+  (getenv* "HELLO_SOURCE_URL"
+           (string-append "https://ftp.gnu.org/gnu/hello/hello-"
+                          hello-version ".tar.gz")))
+(define guix-source-dir
+  (getenv* "GUIX_SOURCE_DIR"
+           (string-append (getenv "HOME") "/repos/guix")))
+(define output-dir
+  (string-append workdir "/0000000000000000-hello-" hello-version))
+(define source-tarball
+  (string-append workdir "/hello-" hello-version ".tar.gz"))
+(define source-dir
+  (string-append workdir "/hello-" hello-version))
+(define metadata-file
+  (string-append workdir "/gnu-hello-guix-phase-runner-metadata.txt"))
+(define expected-sha256-hex
+  (nix-base32->hex expected-nix-base32))
+(define selected-phase-names
+  '(set-SOURCE-DATE-EPOCH unpack configure build check install))
+(define selected-phases
+  (phase-subset selected-phase-names))
+
+(setenv "NIX_BUILD_TOP" workdir)
+(mkdir-p workdir)
+
+(format #t "Using workdir: ~a~%" workdir)
+(format #t "Using Guix source: ~a~%" guix-source-dir)
+(format #t "Fetching source: ~a~%" source-url)
+(invoke "fetch" "-o" source-tarball source-url)
+
+(let ((actual-sha256-hex (command-output "sha256" "-q" source-tarball)))
+  (unless (string=? actual-sha256-hex expected-sha256-hex)
+    (error (format #f "sha256 mismatch: expected ~a but got ~a"
+                   expected-sha256-hex actual-sha256-hex)))
+
+  (format #t "Verified SHA256: ~a~%" actual-sha256-hex)
+  (format #t "Running Guix builder phases: ~s~%" selected-phase-names)
+
+  (with-directory-excursion workdir
+    (gnu-build #:source source-tarball
+               #:outputs `(("out" . ,output-dir))
+               #:phases selected-phases
+               #:tests? #t))
+
+  (let* ((installed-binary (string-append output-dir "/bin/hello"))
+         (hello-output (command-output installed-binary))
+         (host-triplet (with-directory-excursion source-dir
+                         (command-output "sh" "build-aux/config.guess")))
+         (binary-file (command-output "file" installed-binary))
+         (runtime-deps (command-output "ldd" installed-binary))
+         (check-summary
+          (call-with-input-file (string-append source-dir "/test-suite.log")
+            get-string-all))
+         (guile-bin (or (getenv "GUILE_BIN") "<unset>"))
+         (uname-string (command-output "uname" "-a")))
+    (unless (string=? hello-output "Hello, world!")
+      (error (format #f "unexpected hello output: ~s" hello-output)))
+
+    (call-with-output-file metadata-file
+      (lambda (port)
+        (format port "hello_version=~a~%" hello-version)
+        (format port "source_url=~a~%" source-url)
+        (format port "expected_nix_base32=~a~%" expected-nix-base32)
+        (format port "expected_sha256_hex=~a~%" expected-sha256-hex)
+        (format port "actual_sha256_hex=~a~%" actual-sha256-hex)
+        (format port "guix_source_dir=~a~%" guix-source-dir)
+        (format port "guile_bin=~a~%" guile-bin)
+        (format port "workdir=~a~%" workdir)
+        (format port "source_tarball=~a~%" source-tarball)
+        (format port "source_dir=~a~%" source-dir)
+        (format port "output_dir=~a~%" output-dir)
+        (format port "installed_binary=~a~%" installed-binary)
+        (format port "hello_output=~a~%" hello-output)
+        (format port "host_triplet=~a~%" host-triplet)
+        (format port "selected_phases=~s~%" selected-phase-names)
+        (format port "binary_file=~a~%" binary-file)
+        (format port "uname=~a~%" uname-string)
+        (format port "check_summary_begin~%~a~%check_summary_end~%"
+                check-summary)
+        (format port "runtime_deps_begin~%~a~%runtime_deps_end~%"
+                runtime-deps)))
+
+    (when (getenv "METADATA_OUT")
+      (mkdir-p (dirname (getenv "METADATA_OUT")))
+      (copy-file metadata-file (getenv "METADATA_OUT")))
+
+    (format #t "PASS gnu-hello-guix-phase-runner~%")
+    (format #t "Hello output: ~a~%" hello-output)
+    (format #t "Installed binary: ~a~%" installed-binary)
+    (format #t "Metadata file: ~a~%" metadata-file)
+    (when (getenv "METADATA_OUT")
+      (format #t "Copied metadata to: ~a~%" (getenv "METADATA_OUT")))
+    (display "--- metadata ---\n")
+    (display (call-with-input-file metadata-file get-string-all))))
--- a/tests/native-build/run-gnu-hello-guix-phase-runner.sh
+++ b/tests/native-build/run-gnu-hello-guix-phase-runner.sh
@@ -0,0 +1,76 @@
+#!/bin/sh
+set -eu
+
+guix_source_dir=${GUIX_SOURCE_DIR:-"$HOME/repos/guix"}
+script_dir=$(CDPATH= cd -- "$(dirname "$0")" && pwd)
+runner_scm=$script_dir/gnu-hello-guix-phase-runner.scm
+
+if [ ! -d "$guix_source_dir/guix" ]; then
+  echo "Guix source tree not found at $guix_source_dir" >&2
+  exit 1
+fi
+
+if [ -n "${GUILE_BIN:-}" ]; then
+  guile_bin=$GUILE_BIN
+elif [ -x /tmp/guile-freebsd-validate-install/bin/guile ]; then
+  guile_bin=/tmp/guile-freebsd-validate-install/bin/guile
+else
+  cat >&2 <<'EOF'
+A fixed local Guile build is required for this harness.
+The packaged FreeBSD guile3 binary still crashes in system*/spawn/open-pipe*.
+Set GUILE_BIN to a locally built fixed Guile, for example:
+  GUILE_BIN=/tmp/guile-freebsd-validate-install/bin/guile
+EOF
+  exit 1
+fi
+
+if [ ! -x "$guile_bin" ]; then
+  echo "Guile binary is not executable: $guile_bin" >&2
+  exit 1
+fi
+
+guile_prefix=$(CDPATH= cd -- "$(dirname "$guile_bin")/.." && pwd)
+guile_lib_dir=$guile_prefix/lib
+if [ -e "$guile_lib_dir/libguile-3.0.so.1" ]; then
+  if [ -n "${LD_LIBRARY_PATH:-}" ]; then
+    export LD_LIBRARY_PATH="$guile_lib_dir:$LD_LIBRARY_PATH"
+  else
+    export LD_LIBRARY_PATH="$guile_lib_dir"
+  fi
+fi
+
+cleanup=0
+if [ -n "${WORKDIR:-}" ]; then
+  workdir=$WORKDIR
+  mkdir -p "$workdir"
+else
+  workdir=$(mktemp -d /tmp/fruix-gnu-hello-guix.XXXXXX)
+  cleanup=1
+fi
+
+if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
+  cleanup=0
+fi
+
+cleanup_workdir() {
+  if [ "$cleanup" -eq 1 ]; then
+    rm -rf "$workdir"
+  fi
+}
+trap cleanup_workdir EXIT INT TERM
+
+export GUILE_BIN="$guile_bin"
+export GUIX_SOURCE_DIR="$guix_source_dir"
+export WORKDIR="$workdir"
+export GUILE_AUTO_COMPILE=0
+if [ -n "${GUILE_LOAD_PATH:-}" ]; then
+  export GUILE_LOAD_PATH="$guix_source_dir:$GUILE_LOAD_PATH"
+else
+  export GUILE_LOAD_PATH="$guix_source_dir"
+fi
+
+printf 'Using Guile: %s\n' "$guile_bin"
+printf 'Using LD_LIBRARY_PATH: %s\n' "${LD_LIBRARY_PATH:-<unset>}"
+printf 'Working directory: %s\n' "$workdir"
+
+"$guile_bin" -s "$runner_scm"