Validate GNU which builder phases on FreeBSD

2026-04-01 09:49:54 +02:00
parent c944cdba5f
commit 0a2e48eda4
4 changed files with 474 additions and 0 deletions
--- a/docs/PROGRESS.md
+++ b/docs/PROGRESS.md
@@ -236,3 +236,69 @@ Next recommended step:
 1. run the Scheme-driven phase-runner pattern against at least one more small GNU/autotools package on FreeBSD, or
 2. document the concrete gaps between this prototype and a real Guix package/derivation build, especially around store management and build isolation
 3. continue keeping `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear
+
+## 2026-04-01 — Phase 1.2 follow-up: second Scheme-driven GNU package build validated with GNU which
+
+Completed work:
+
+- added a second Scheme-driven GNU package harness:
+  - `tests/native-build/gnu-which-guix-phase-runner.scm`
+  - `tests/native-build/run-gnu-which-guix-phase-runner.sh`
+- again used the previously validated fixed local Guile build because this harness depends on subprocess-heavy Guix/Scheme builder logic
+- used the current Guix package definition in `~/repos/guix/gnu/packages/base.scm` as the source of truth for:
+  - GNU which version `2.21`
+  - expected Guix nix-base32 source hash `1bgafvy3ypbhhfznwjv1lxmd6mci3x1byilnnkc7gcr486wlb8pl`
+- verified the downloaded tarball against the translated SHA256:
+  - `f4a245b94124b377d8b49646bf421f9155d36aa7614b6ebf83705d3ffc76eaad`
+- successfully executed the same subset of Guix builder-side `%standard-phases` on FreeBSD as used for GNU Hello:
+  - `set-SOURCE-DATE-EPOCH`
+  - `unpack`
+  - `configure`
+  - `build`
+  - `check`
+  - `install`
+- executed the resulting `which` binary successfully with a deterministic command:
+  - `PATH=/bin:/usr/bin ./which sh`
+- confirmed output:
+  - `/bin/sh`
+- captured metadata including:
+  - host triplet
+  - phase list
+  - runtime dependencies
+  - check-phase success status
+  - executed command output
+- wrote the results to `docs/reports/phase1-guix-which-phase-runner.md`
+
+Important findings:
+
+- this confirms the Scheme-driven Guix builder-side phase-runner pattern is not limited to GNU Hello; a second small GNU/autotools package also succeeds on FreeBSD
+- GNU which's `check` phase passed, but it did not leave behind an Automake-style `test-suite.log` or `testsuite.log`
+- GNU which emitted a non-fatal `configure` warning about Guix's standard `--enable-fast-install` flag being unrecognized
+- the source also emitted several clang warnings about deprecated non-prototype C declarations/definitions, but the build still completed successfully
+- the resulting `which` binary again showed a minimal store-like runtime linkage profile:
+  - `libc.so.7`
+  - `libsys.so.7`
+- unlike GNU Hello, the source tree did not present an obvious shipped `config.guess`, so the harness used `cc -dumpmachine` as a fallback for host-triplet metadata
+
+Current assessment:
+
+- Phase 1.2 now has two successful Scheme-driven Guix builder-side GNU package validations on FreeBSD:
+  - GNU Hello
+  - GNU which
+- this increases confidence that a narrow but real subset of `gnu-build-system` builder-side execution already works on FreeBSD when paired with the fixed local Guile build
+- the next uncertainty is now less about whether basic builder phases run at all, and more about where real Guix package/derivation/store integration and isolation will first require FreeBSD-specific adaptation
+
+Recent commits:
+
+- `e380e88` — `Add FreeBSD Guile verification harness`
+- `cd721b1` — `Update progress after Guile verification`
+- `27916cb` — `Diagnose Guile subprocess crash on FreeBSD`
+- `02f7a7f` — `Validate local Guile fix on FreeBSD`
+- `4aebea4` — `Add native GNU Hello FreeBSD build harness`
+- `c944cdb` — `Validate Guix builder phases on FreeBSD`
+
+Next recommended step:
+
+1. document the concrete remaining gap between these Scheme-driven phase-runner prototypes and a true Guix package/derivation/store-daemon build on FreeBSD, especially around store management, implicit inputs, and build isolation
+2. or choose a somewhat more demanding GNU package with non-trivial declared inputs to identify the first builder-side FreeBSD adaptation points
+3. continue keeping `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear
--- a/docs/reports/phase1-guix-which-phase-runner.md
+++ b/docs/reports/phase1-guix-which-phase-runner.md
@@ -0,0 +1,157 @@
+# Phase 1.2 follow-up: second Scheme-driven GNU package build validated with GNU which
+
+Date: 2026-04-01
+
+## Summary
+
+This step extends the Scheme-driven FreeBSD validation from GNU Hello to a second small GNU/autotools package: GNU which.
+
+Added files:
+
+- `tests/native-build/gnu-which-guix-phase-runner.scm`
+- `tests/native-build/run-gnu-which-guix-phase-runner.sh`
+
+Like the earlier GNU Hello phase-runner, this harness uses a fixed local Guile build plus Guix's builder-side `(guix build gnu-build-system)` code to execute a subset of `%standard-phases` on FreeBSD.
+
+## Source identity
+
+The source of truth was the current Guix package definition in `~/repos/guix/gnu/packages/base.scm`:
+
+- package: `which`
+- version: `2.21`
+- Guix nix-base32: `1bgafvy3ypbhhfznwjv1lxmd6mci3x1byilnnkc7gcr486wlb8pl`
+
+Translated and verified SHA256:
+
+```text
+f4a245b94124b377d8b49646bf421f9155d36aa7614b6ebf83705d3ffc76eaad
+```
+
+## Verification command
+
+```sh
+METADATA_OUT=/tmp/gnu-which-guix-metadata.txt \
+  ./tests/native-build/run-gnu-which-guix-phase-runner.sh
+```
+
+The harness used the previously validated fixed local Guile build:
+
+```text
+/tmp/guile-freebsd-validate-install/bin/guile
+```
+
+## Guix builder phases exercised
+
+The harness ran this subset of `(guix build gnu-build-system)` `%standard-phases`:
+
+- `set-SOURCE-DATE-EPOCH`
+- `unpack`
+- `configure`
+- `build`
+- `check`
+- `install`
+
+## Result
+
+The Scheme-driven GNU which build succeeded on `FreeBSD 15.0-STABLE` amd64.
+
+Observed outcomes:
+
+- source fetch: success
+- source hash verification: success
+- `configure`: success
+- `build`: success
+- `check`: success
+- `install`: success
+- runtime execution: success
+
+The installed binary was executed as:
+
+```sh
+PATH=/bin:/usr/bin ./which sh
+```
+
+Observed output:
+
+```text
+/bin/sh
+```
+
+## Notable findings
+
+### 1. The Scheme-driven phase-runner pattern is not limited to GNU Hello
+
+This confirms that the earlier GNU Hello success was not a one-off. A second small GNU/autotools package also builds successfully on FreeBSD when driven by Guix's builder-side GNU build logic.
+
+### 2. GNU which did not emit a `test-suite.log`, but `check` still succeeded
+
+The `check` phase completed successfully, but this package did not leave behind a `test-suite.log` or `testsuite.log` file.
+
+That is a useful reminder for future FreeBSD validation harnesses: passing `make check` cannot always be summarized by the presence of Automake-style test log files.
+
+### 3. Guix's default `--enable-fast-install` configure flag is not universally recognized
+
+During `configure`, GNU which reported:
+
+```text
+configure: WARNING: unrecognized options: --enable-fast-install
+```
+
+The build still succeeded. This is a small but useful data point for later FreeBSD/Guix compatibility work: some packages tolerate Guix's standard configure flag set while emitting non-fatal warnings.
+
+### 4. Older GNU package code triggers modern clang warnings on FreeBSD
+
+GNU which built successfully, but the build emitted several warnings about deprecated non-prototype C function declarations/definitions.
+
+These warnings did not prevent the build, but they are good evidence that older GNU packages may need warning-tolerance assumptions when validated with modern FreeBSD clang toolchains.
+
+### 5. Runtime linkage again matched a minimal store-like output profile
+
+The resulting `which` binary linked against:
+
+- `libc.so.7`
+- `libsys.so.7`
+
+That matches the store-like GNU Hello Scheme-runner result more closely than the earlier `/usr/local`-staged native shell GNU Hello build.
+
+## Additional metadata captured
+
+The harness recorded:
+
+- verified source hash
+- selected Guix phase list
+- host triplet
+- runtime dependencies
+- executed command and expected output
+- confirmation that the `check` phase passed
+
+The host triplet recorded was:
+
+```text
+x86_64-unknown-freebsd15.0
+```
+
+Because this source tree did not ship an obvious `config.guess`, the harness fell back to `cc -dumpmachine` for host-triplet metadata.
+
+## What this step demonstrates
+
+This step strengthens Phase 1.2 in two ways:
+
+1. it validates a second GNU/autotools package using Guix builder-side Scheme phases on FreeBSD
+2. it reveals a few practical compatibility details already visible at this stage:
+   - optional absence of Automake-style test logs
+   - non-fatal `--enable-fast-install` warnings
+   - clang warnings in older GNU source
+
+## Current implication for the porting effort
+
+With both GNU Hello and GNU which validated through Scheme-driven builder-side Guix phases, the project now has evidence that FreeBSD can already support a narrow but real subset of `gnu-build-system` execution, provided the locally fixed Guile build is used.
+
+That still falls short of a true package/derivation/store-daemon build, but it further reduces uncertainty around builder-side phase execution itself.
+
+## Recommended next step
+
+The next useful step is likely one of:
+
+1. document the concrete remaining gap between these phase-runner prototypes and a real Guix package/derivation build on FreeBSD, or
+2. choose a slightly more demanding GNU package with non-trivial inputs to see where builder-side execution first starts needing FreeBSD-specific adaptation
--- a/tests/native-build/gnu-which-guix-phase-runner.scm
+++ b/tests/native-build/gnu-which-guix-phase-runner.scm
@@ -0,0 +1,175 @@
+(use-modules (guix base32)
+             (guix build gnu-build-system)
+             (guix build utils)
+             (ice-9 format)
+             (ice-9 match)
+             (ice-9 popen)
+             (srfi srfi-1)
+             (rnrs bytevectors)
+             (rnrs io ports))
+
+(define (getenv* name default)
+  (or (getenv name) default))
+
+(define (trim-trailing-newlines str)
+  (let loop ((len (string-length str)))
+    (if (and (> len 0)
+             (char=? (string-ref str (- len 1)) #\newline))
+        (loop (- len 1))
+        (substring str 0 len))))
+
+(define (command-output program . args)
+  (let* ((port   (apply open-pipe* OPEN_READ program args))
+         (output (get-string-all port))
+         (status (close-pipe port)))
+    (unless (zero? status)
+      (error (format #f "command failed: ~a ~s => ~a"
+                     program args status)))
+    (trim-trailing-newlines output)))
+
+(define (nix-base32->hex str)
+  (string-concatenate
+   (map (lambda (byte)
+          (format #f "~2,'0x" byte))
+        (bytevector->u8-list (nix-base32-string->bytevector str)))))
+
+(define (phase-subset names)
+  (filter (match-lambda
+            ((name . _)
+             (memq name names)))
+          %standard-phases))
+
+(define (maybe-read-test-log source-dir)
+  (match (find-files source-dir "(^|/)(test-suite\\.log|testsuite\\.log)$")
+    ((file . _)
+     (call-with-input-file file get-string-all))
+    (()
+     "<no test-suite.log or testsuite.log found>")))
+
+(define (with-path path thunk)
+  (let ((original (getenv "PATH")))
+    (dynamic-wind
+      (lambda ()
+        (setenv "PATH" path))
+      thunk
+      (lambda ()
+        (if original
+            (setenv "PATH" original)
+            (unsetenv "PATH"))))))
+
+(define (host-triplet-from-source source-dir)
+  (with-directory-excursion source-dir
+    (cond
+     ((file-exists? "build-aux/config.guess")
+      (command-output "sh" "build-aux/config.guess"))
+     ((file-exists? "config.guess")
+      (command-output "sh" "config.guess"))
+     (else
+      (command-output "cc" "-dumpmachine")))))
+
+(define workdir
+  (or (getenv "WORKDIR")
+      (error "WORKDIR environment variable is required")))
+
+(define which-version
+  (getenv* "WHICH_VERSION" "2.21"))
+(define expected-nix-base32
+  (getenv* "WHICH_NIX_BASE32" "1bgafvy3ypbhhfznwjv1lxmd6mci3x1byilnnkc7gcr486wlb8pl"))
+(define source-url
+  (getenv* "WHICH_SOURCE_URL"
+           (string-append "https://ftp.gnu.org/gnu/which/which-"
+                          which-version ".tar.gz")))
+(define guix-source-dir
+  (getenv* "GUIX_SOURCE_DIR"
+           (string-append (getenv "HOME") "/repos/guix")))
+(define output-dir
+  (string-append workdir "/0000000000000000-which-" which-version))
+(define source-tarball
+  (string-append workdir "/which-" which-version ".tar.gz"))
+(define source-dir
+  (string-append workdir "/which-" which-version))
+(define metadata-file
+  (string-append workdir "/gnu-which-guix-phase-runner-metadata.txt"))
+(define expected-sha256-hex
+  (nix-base32->hex expected-nix-base32))
+(define selected-phase-names
+  '(set-SOURCE-DATE-EPOCH unpack configure build check install))
+(define selected-phases
+  (phase-subset selected-phase-names))
+(define run-path "/bin:/usr/bin")
+(define expected-which-output "/bin/sh")
+
+(setenv "NIX_BUILD_TOP" workdir)
+(mkdir-p workdir)
+
+(format #t "Using workdir: ~a~%" workdir)
+(format #t "Using Guix source: ~a~%" guix-source-dir)
+(format #t "Fetching source: ~a~%" source-url)
+(invoke "fetch" "-o" source-tarball source-url)
+
+(let ((actual-sha256-hex (command-output "sha256" "-q" source-tarball)))
+  (unless (string=? actual-sha256-hex expected-sha256-hex)
+    (error (format #f "sha256 mismatch: expected ~a but got ~a"
+                   expected-sha256-hex actual-sha256-hex)))
+
+  (format #t "Verified SHA256: ~a~%" actual-sha256-hex)
+  (format #t "Running Guix builder phases: ~s~%" selected-phase-names)
+
+  (with-directory-excursion workdir
+    (gnu-build #:source source-tarball
+               #:outputs `(("out" . ,output-dir))
+               #:phases selected-phases
+               #:tests? #t))
+
+  (let* ((installed-binary (string-append output-dir "/bin/which"))
+         (which-output (with-path run-path
+                         (lambda ()
+                           (command-output installed-binary "sh"))))
+         (host-triplet (host-triplet-from-source source-dir))
+         (binary-file (command-output "file" installed-binary))
+         (runtime-deps (command-output "ldd" installed-binary))
+         (check-summary (maybe-read-test-log source-dir))
+         (guile-bin (or (getenv "GUILE_BIN") "<unset>"))
+         (uname-string (command-output "uname" "-a")))
+    (unless (string=? which-output expected-which-output)
+      (error (format #f "unexpected which output: ~s" which-output)))
+
+    (call-with-output-file metadata-file
+      (lambda (port)
+        (format port "which_version=~a~%" which-version)
+        (format port "source_url=~a~%" source-url)
+        (format port "expected_nix_base32=~a~%" expected-nix-base32)
+        (format port "expected_sha256_hex=~a~%" expected-sha256-hex)
+        (format port "actual_sha256_hex=~a~%" actual-sha256-hex)
+        (format port "guix_source_dir=~a~%" guix-source-dir)
+        (format port "guile_bin=~a~%" guile-bin)
+        (format port "workdir=~a~%" workdir)
+        (format port "source_tarball=~a~%" source-tarball)
+        (format port "source_dir=~a~%" source-dir)
+        (format port "output_dir=~a~%" output-dir)
+        (format port "installed_binary=~a~%" installed-binary)
+        (format port "run_path=~a~%" run-path)
+        (format port "which_output=~a~%" which-output)
+        (format port "expected_which_output=~a~%" expected-which-output)
+        (format port "host_triplet=~a~%" host-triplet)
+        (format port "selected_phases=~s~%" selected-phase-names)
+        (format port "check_phase=passed~%")
+        (format port "binary_file=~a~%" binary-file)
+        (format port "uname=~a~%" uname-string)
+        (format port "check_summary_begin~%~a~%check_summary_end~%"
+                check-summary)
+        (format port "runtime_deps_begin~%~a~%runtime_deps_end~%"
+                runtime-deps)))
+
+    (when (getenv "METADATA_OUT")
+      (mkdir-p (dirname (getenv "METADATA_OUT")))
+      (copy-file metadata-file (getenv "METADATA_OUT")))
+
+    (format #t "PASS gnu-which-guix-phase-runner~%")
+    (format #t "Which output: ~a~%" which-output)
+    (format #t "Installed binary: ~a~%" installed-binary)
+    (format #t "Metadata file: ~a~%" metadata-file)
+    (when (getenv "METADATA_OUT")
+      (format #t "Copied metadata to: ~a~%" (getenv "METADATA_OUT")))
+    (display "--- metadata ---\n")
+    (display (call-with-input-file metadata-file get-string-all))))
--- a/tests/native-build/run-gnu-which-guix-phase-runner.sh
+++ b/tests/native-build/run-gnu-which-guix-phase-runner.sh
@@ -0,0 +1,76 @@
+#!/bin/sh
+set -eu
+
+guix_source_dir=${GUIX_SOURCE_DIR:-"$HOME/repos/guix"}
+script_dir=$(CDPATH= cd -- "$(dirname "$0")" && pwd)
+runner_scm=$script_dir/gnu-which-guix-phase-runner.scm
+
+if [ ! -d "$guix_source_dir/guix" ]; then
+  echo "Guix source tree not found at $guix_source_dir" >&2
+  exit 1
+fi
+
+if [ -n "${GUILE_BIN:-}" ]; then
+  guile_bin=$GUILE_BIN
+elif [ -x /tmp/guile-freebsd-validate-install/bin/guile ]; then
+  guile_bin=/tmp/guile-freebsd-validate-install/bin/guile
+else
+  cat >&2 <<'EOF'
+A fixed local Guile build is required for this harness.
+The packaged FreeBSD guile3 binary still crashes in system*/spawn/open-pipe*.
+Set GUILE_BIN to a locally built fixed Guile, for example:
+  GUILE_BIN=/tmp/guile-freebsd-validate-install/bin/guile
+EOF
+  exit 1
+fi
+
+if [ ! -x "$guile_bin" ]; then
+  echo "Guile binary is not executable: $guile_bin" >&2
+  exit 1
+fi
+
+guile_prefix=$(CDPATH= cd -- "$(dirname "$guile_bin")/.." && pwd)
+guile_lib_dir=$guile_prefix/lib
+if [ -e "$guile_lib_dir/libguile-3.0.so.1" ]; then
+  if [ -n "${LD_LIBRARY_PATH:-}" ]; then
+    export LD_LIBRARY_PATH="$guile_lib_dir:$LD_LIBRARY_PATH"
+  else
+    export LD_LIBRARY_PATH="$guile_lib_dir"
+  fi
+fi
+
+cleanup=0
+if [ -n "${WORKDIR:-}" ]; then
+  workdir=$WORKDIR
+  mkdir -p "$workdir"
+else
+  workdir=$(mktemp -d /tmp/fruix-gnu-which-guix.XXXXXX)
+  cleanup=1
+fi
+
+if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
+  cleanup=0
+fi
+
+cleanup_workdir() {
+  if [ "$cleanup" -eq 1 ]; then
+    rm -rf "$workdir"
+  fi
+}
+trap cleanup_workdir EXIT INT TERM
+
+export GUILE_BIN="$guile_bin"
+export GUIX_SOURCE_DIR="$guix_source_dir"
+export WORKDIR="$workdir"
+export GUILE_AUTO_COMPILE=0
+if [ -n "${GUILE_LOAD_PATH:-}" ]; then
+  export GUILE_LOAD_PATH="$guix_source_dir:$GUILE_LOAD_PATH"
+else
+  export GUILE_LOAD_PATH="$guix_source_dir"
+fi
+
+printf 'Using Guile: %s\n' "$guile_bin"
+printf 'Using LD_LIBRARY_PATH: %s\n' "${LD_LIBRARY_PATH:-<unset>}"
+printf 'Working directory: %s\n' "$workdir"
+
+"$guile_bin" -s "$runner_scm"