diff --git a/docs/PROGRESS.md b/docs/PROGRESS.md index ebd68ed..e5889dc 100644 --- a/docs/PROGRESS.md +++ b/docs/PROGRESS.md @@ -169,3 +169,70 @@ Next recommended step: 1. extend Phase 1.2 with at least one additional representative GNU/autotools package build on FreeBSD, or 2. prototype a tiny Scheme-based `gnu-build-system`-like phase runner using the known-good local Guile path, starting from the GNU Hello flow 3. continue keeping `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear + +## 2026-04-01 — Phase 1.2 follow-up: Guix builder-side GNU Hello phase runner validated + +Completed work: + +- added a Scheme-driven GNU Hello build prototype: + - `tests/native-build/gnu-hello-guix-phase-runner.scm` + - `tests/native-build/run-gnu-hello-guix-phase-runner.sh` +- required the previously validated fixed local Guile build for this harness because it depends on subprocess-heavy Scheme operations +- used Guix modules directly from `~/repos/guix`, including: + - `(guix base32)` + - `(guix build gnu-build-system)` + - `(guix build utils)` +- fetched and hash-verified GNU Hello `2.12.3` again against the Guix package hash: + - nix-base32: `183a6rxnhixiyykd7qis0y9g9cfqhpkk872a245y3zl28can0pqd` + - SHA256: `0d5f60154382fee10b114a1c34e785d8b1f492073ae2d3a6f7b147687b366aa0` +- successfully executed a subset of Guix builder-side `%standard-phases` on FreeBSD: + - `set-SOURCE-DATE-EPOCH` + - `unpack` + - `configure` + - `build` + - `check` + - `install` +- installed GNU Hello into a store-like output path under the temporary work directory rather than using a `/usr/local` `DESTDIR` staging layout +- executed the resulting binary successfully and confirmed output: + - `Hello, world!` +- captured metadata including: + - host triplet + - selected phase list + - runtime dependencies + - test-suite summary +- wrote the results to `docs/reports/phase1-guix-gnu-hello-phase-runner.md` + +Important findings: + +- this is the first validation step in the repo that successfully exercised actual Guix builder-side GNU build logic on FreeBSD instead of only a shell approximation +- the harness works when driven by the fixed local Guile build, confirming that the earlier Guile subprocess-fix validation is directly useful for FreeBSD Guix build orchestration +- GNU Hello's `make check` test suite also passed in this mode: + - total: `7` + - pass: `7` + - fail: `0` +- the resulting binary's runtime dependencies differ from the earlier `/usr/local`-prefixed native shell harness; in this store-like output layout it only showed: + - `libc.so.7` + - `libsys.so.7` +- that difference is a useful clue that Guix-style output layout/build invocation can materially affect FreeBSD runtime linkage behavior + +Current assessment: + +- Phase 1.2 now has both: + - a shell-driven native GNU Hello build harness, and + - a Scheme-driven prototype that uses real Guix builder-side GNU phases +- this is still short of a true Guix package/derivation build, but it significantly narrows the gap between host validation and real `gnu-build-system` execution on FreeBSD +- the known-good local Guile path is now validated as part of a practical Guix-adjacent build workflow, not just standalone subprocess diagnostics + +Recent commits: + +- `e380e88` — `Add FreeBSD Guile verification harness` +- `cd721b1` — `Update progress after Guile verification` +- `27916cb` — `Diagnose Guile subprocess crash on FreeBSD` +- `02f7a7f` — `Validate local Guile fix on FreeBSD` +- `4aebea4` — `Add native GNU Hello FreeBSD build harness` + +Next recommended step: + +1. run the Scheme-driven phase-runner pattern against at least one more small GNU/autotools package on FreeBSD, or +2. document the concrete gaps between this prototype and a real Guix package/derivation build, especially around store management and build isolation +3. continue keeping `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear diff --git a/docs/reports/phase1-guix-gnu-hello-phase-runner.md b/docs/reports/phase1-guix-gnu-hello-phase-runner.md new file mode 100644 index 0000000..3b2813e --- /dev/null +++ b/docs/reports/phase1-guix-gnu-hello-phase-runner.md @@ -0,0 +1,169 @@ +# Phase 1.2 follow-up: Scheme-driven GNU Hello build with Guix builder phases on FreeBSD + +Date: 2026-04-01 + +## Summary + +This step extends the earlier native GNU Hello shell harness by validating a tiny Scheme-driven prototype that uses Guix's actual builder-side GNU build machinery on FreeBSD. + +Added files: + +- `tests/native-build/gnu-hello-guix-phase-runner.scm` +- `tests/native-build/run-gnu-hello-guix-phase-runner.sh` + +The prototype does not yet build a real Guix package or derivation, but it does execute a subset of Guix's builder-side `%standard-phases` from `(guix build gnu-build-system)` against GNU Hello on FreeBSD. + +## Why this step matters + +The previous Phase 1.2 step showed that FreeBSD can perform a shell-driven native autotools build of GNU Hello. + +This step moves closer to actual Guix execution by validating that FreeBSD can run Guix's Scheme-side builder logic when using a fixed local Guile build. + +That is important because the earlier packaged FreeBSD `guile3` binary still crashes in subprocess helpers such as `system*`, `spawn`, and `open-pipe*`. A Scheme-driven build harness therefore needs the locally fixed Guile path validated in Phase 1.1. + +## Harness design + +The shell wrapper: + +- requires a fixed local Guile build +- automatically prepends the matching sibling `lib` directory to `LD_LIBRARY_PATH` +- exposes the Guix source tree through `GUILE_LOAD_PATH` +- creates and cleans up a temporary work directory unless `KEEP_WORKDIR=1` + +The Scheme runner: + +1. fetches GNU Hello `2.12.3` +2. translates the Guix nix-base32 hash to hexadecimal using `(guix base32)` +3. verifies the downloaded tarball with `sha256(1)` +4. invokes `gnu-build` from `(guix build gnu-build-system)` +5. runs this phase subset from `%standard-phases`: + - `set-SOURCE-DATE-EPOCH` + - `unpack` + - `configure` + - `build` + - `check` + - `install` +6. executes the installed `hello` binary +7. records metadata including test results and runtime dependencies + +Unlike the earlier shell harness, this prototype installs directly into a store-like output directory under the temporary work tree, which is closer to actual Guix build behavior. + +## Source identity + +As in the previous step, this uses the GNU Hello source currently referenced by `~/repos/guix/gnu/packages/base.scm`: + +- version: `2.12.3` +- Guix nix-base32: `183a6rxnhixiyykd7qis0y9g9cfqhpkk872a245y3zl28can0pqd` +- verified SHA256: `0d5f60154382fee10b114a1c34e785d8b1f492073ae2d3a6f7b147687b366aa0` + +## Verification command + +```sh +METADATA_OUT=/tmp/gnu-hello-guix-metadata.txt \ + ./tests/native-build/run-gnu-hello-guix-phase-runner.sh +``` + +The wrapper used the previously validated local fixed Guile build: + +```text +/tmp/guile-freebsd-validate-install/bin/guile +``` + +## Result + +The Scheme-driven build succeeded on `FreeBSD 15.0-STABLE` amd64. + +Observed outcomes: + +- source fetch: success +- source hash verification: success +- Guix builder phase execution: success +- `configure`: success +- `build`: success +- `check`: success +- `install`: success +- runtime execution: success + +Observed program output: + +```text +Hello, world! +``` + +Observed test summary: + +```text +# TOTAL: 7 +# PASS: 7 +# SKIP: 0 +# XFAIL: 0 +# FAIL: 0 +# XPASS: 0 +# ERROR: 0 +``` + +## Notable findings + +### 1. Guix builder-side GNU phases can run on FreeBSD + +This is the first repository step that successfully exercised actual Guix builder-side GNU build logic on FreeBSD rather than only a shell approximation. + +The harness successfully called `gnu-build` from `(guix build gnu-build-system)` with a selected subset of `%standard-phases`. + +### 2. The local fixed Guile build remains necessary for Scheme-driven build orchestration + +This harness depends on subprocess helpers such as `open-pipe*` and `invoke`, so it must not use the packaged FreeBSD `guile3` binary. + +Using the fixed local Guile build from earlier Phase 1.1 validation was sufficient. + +### 3. Store-like installation changes the resulting runtime linkage profile + +The installed binary ended up in a store-like output directory: + +```text +/tmp/.../0000000000000000-hello-2.12.3/bin/hello +``` + +Observed runtime dependencies were: + +- `libc.so.7` +- `libsys.so.7` + +This differs from the earlier `/usr/local`-prefixed native shell harness, where the staged binary also pulled in `libiconv.so.2`, `libintl.so.8`, and `libthr.so.3`. + +This is an important data point: a more Guix-like output layout and build path can produce materially different runtime linkage on FreeBSD. + +### 4. The test suite passes in this mode too + +Unlike the previous shell harness, this prototype also ran GNU Hello's `make check` phase successfully. + +That gives a stronger signal that the Guix builder-side phase sequence is usable on FreeBSD for at least a minimal GNU autotools package. + +## What this step demonstrates + +This step demonstrates that FreeBSD can support a small but real fragment of Guix build orchestration in Scheme, provided that Guile itself is built from a fixed revision containing the upstream subprocess crash fix. + +In practical terms, the host now has validated evidence for all of the following: + +1. native shell-driven GNU Hello build works +2. local fixed Guile can safely drive subprocess-heavy Scheme code on FreeBSD +3. Guix builder-side GNU phases can build, test, install, and run GNU Hello on FreeBSD + +## Remaining limitations + +This is still not a full Guix package build: + +- no derivation was created +- no store daemon was involved +- no build isolation/jail integration was used +- no dependency graph resolution was performed by Guix package machinery +- only a subset of `%standard-phases` was exercised + +## Recommended next step + +Continue Phase 1.2 by broadening coverage in one of two ways: + +1. run the same Scheme-driven phase-runner approach against another small GNU/autotools package, or +2. start documenting the exact Linux-vs-FreeBSD gaps that remain between this prototype and a real `gnu-build-system` build driven by Guix package definitions + +Either direction would build directly on the harnesses now present in `tests/native-build/`. diff --git a/tests/native-build/gnu-hello-guix-phase-runner.scm b/tests/native-build/gnu-hello-guix-phase-runner.scm new file mode 100644 index 0000000..ac2f7bc --- /dev/null +++ b/tests/native-build/gnu-hello-guix-phase-runner.scm @@ -0,0 +1,143 @@ +(use-modules (guix base32) + (guix build gnu-build-system) + (guix build utils) + (ice-9 format) + (ice-9 match) + (ice-9 popen) + (srfi srfi-1) + (rnrs bytevectors) + (rnrs io ports)) + +(define (getenv* name default) + (or (getenv name) default)) + +(define (trim-trailing-newlines str) + (let loop ((len (string-length str))) + (if (and (> len 0) + (char=? (string-ref str (- len 1)) #\newline)) + (loop (- len 1)) + (substring str 0 len)))) + +(define (command-output program . args) + (let* ((port (apply open-pipe* OPEN_READ program args)) + (output (get-string-all port)) + (status (close-pipe port))) + (unless (zero? status) + (error (format #f "command failed: ~a ~s => ~a" + program args status))) + (trim-trailing-newlines output))) + +(define (nix-base32->hex str) + (string-concatenate + (map (lambda (byte) + (format #f "~2,'0x" byte)) + (bytevector->u8-list (nix-base32-string->bytevector str))))) + +(define (phase-subset names) + (filter (match-lambda + ((name . _) + (memq name names))) + %standard-phases)) + +(define workdir + (or (getenv "WORKDIR") + (error "WORKDIR environment variable is required"))) + +(define hello-version + (getenv* "HELLO_VERSION" "2.12.3")) +(define expected-nix-base32 + (getenv* "HELLO_NIX_BASE32" "183a6rxnhixiyykd7qis0y9g9cfqhpkk872a245y3zl28can0pqd")) +(define source-url + (getenv* "HELLO_SOURCE_URL" + (string-append "https://ftp.gnu.org/gnu/hello/hello-" + hello-version ".tar.gz"))) +(define guix-source-dir + (getenv* "GUIX_SOURCE_DIR" + (string-append (getenv "HOME") "/repos/guix"))) +(define output-dir + (string-append workdir "/0000000000000000-hello-" hello-version)) +(define source-tarball + (string-append workdir "/hello-" hello-version ".tar.gz")) +(define source-dir + (string-append workdir "/hello-" hello-version)) +(define metadata-file + (string-append workdir "/gnu-hello-guix-phase-runner-metadata.txt")) +(define expected-sha256-hex + (nix-base32->hex expected-nix-base32)) +(define selected-phase-names + '(set-SOURCE-DATE-EPOCH unpack configure build check install)) +(define selected-phases + (phase-subset selected-phase-names)) + +(setenv "NIX_BUILD_TOP" workdir) +(mkdir-p workdir) + +(format #t "Using workdir: ~a~%" workdir) +(format #t "Using Guix source: ~a~%" guix-source-dir) +(format #t "Fetching source: ~a~%" source-url) +(invoke "fetch" "-o" source-tarball source-url) + +(let ((actual-sha256-hex (command-output "sha256" "-q" source-tarball))) + (unless (string=? actual-sha256-hex expected-sha256-hex) + (error (format #f "sha256 mismatch: expected ~a but got ~a" + expected-sha256-hex actual-sha256-hex))) + + (format #t "Verified SHA256: ~a~%" actual-sha256-hex) + (format #t "Running Guix builder phases: ~s~%" selected-phase-names) + + (with-directory-excursion workdir + (gnu-build #:source source-tarball + #:outputs `(("out" . ,output-dir)) + #:phases selected-phases + #:tests? #t)) + + (let* ((installed-binary (string-append output-dir "/bin/hello")) + (hello-output (command-output installed-binary)) + (host-triplet (with-directory-excursion source-dir + (command-output "sh" "build-aux/config.guess"))) + (binary-file (command-output "file" installed-binary)) + (runtime-deps (command-output "ldd" installed-binary)) + (check-summary + (call-with-input-file (string-append source-dir "/test-suite.log") + get-string-all)) + (guile-bin (or (getenv "GUILE_BIN") "")) + (uname-string (command-output "uname" "-a"))) + (unless (string=? hello-output "Hello, world!") + (error (format #f "unexpected hello output: ~s" hello-output))) + + (call-with-output-file metadata-file + (lambda (port) + (format port "hello_version=~a~%" hello-version) + (format port "source_url=~a~%" source-url) + (format port "expected_nix_base32=~a~%" expected-nix-base32) + (format port "expected_sha256_hex=~a~%" expected-sha256-hex) + (format port "actual_sha256_hex=~a~%" actual-sha256-hex) + (format port "guix_source_dir=~a~%" guix-source-dir) + (format port "guile_bin=~a~%" guile-bin) + (format port "workdir=~a~%" workdir) + (format port "source_tarball=~a~%" source-tarball) + (format port "source_dir=~a~%" source-dir) + (format port "output_dir=~a~%" output-dir) + (format port "installed_binary=~a~%" installed-binary) + (format port "hello_output=~a~%" hello-output) + (format port "host_triplet=~a~%" host-triplet) + (format port "selected_phases=~s~%" selected-phase-names) + (format port "binary_file=~a~%" binary-file) + (format port "uname=~a~%" uname-string) + (format port "check_summary_begin~%~a~%check_summary_end~%" + check-summary) + (format port "runtime_deps_begin~%~a~%runtime_deps_end~%" + runtime-deps))) + + (when (getenv "METADATA_OUT") + (mkdir-p (dirname (getenv "METADATA_OUT"))) + (copy-file metadata-file (getenv "METADATA_OUT"))) + + (format #t "PASS gnu-hello-guix-phase-runner~%") + (format #t "Hello output: ~a~%" hello-output) + (format #t "Installed binary: ~a~%" installed-binary) + (format #t "Metadata file: ~a~%" metadata-file) + (when (getenv "METADATA_OUT") + (format #t "Copied metadata to: ~a~%" (getenv "METADATA_OUT"))) + (display "--- metadata ---\n") + (display (call-with-input-file metadata-file get-string-all)))) diff --git a/tests/native-build/run-gnu-hello-guix-phase-runner.sh b/tests/native-build/run-gnu-hello-guix-phase-runner.sh new file mode 100755 index 0000000..b9f88e2 --- /dev/null +++ b/tests/native-build/run-gnu-hello-guix-phase-runner.sh @@ -0,0 +1,76 @@ +#!/bin/sh +set -eu + +guix_source_dir=${GUIX_SOURCE_DIR:-"$HOME/repos/guix"} +script_dir=$(CDPATH= cd -- "$(dirname "$0")" && pwd) +runner_scm=$script_dir/gnu-hello-guix-phase-runner.scm + +if [ ! -d "$guix_source_dir/guix" ]; then + echo "Guix source tree not found at $guix_source_dir" >&2 + exit 1 +fi + +if [ -n "${GUILE_BIN:-}" ]; then + guile_bin=$GUILE_BIN +elif [ -x /tmp/guile-freebsd-validate-install/bin/guile ]; then + guile_bin=/tmp/guile-freebsd-validate-install/bin/guile +else + cat >&2 <<'EOF' +A fixed local Guile build is required for this harness. +The packaged FreeBSD guile3 binary still crashes in system*/spawn/open-pipe*. +Set GUILE_BIN to a locally built fixed Guile, for example: + GUILE_BIN=/tmp/guile-freebsd-validate-install/bin/guile +EOF + exit 1 +fi + +if [ ! -x "$guile_bin" ]; then + echo "Guile binary is not executable: $guile_bin" >&2 + exit 1 +fi + +guile_prefix=$(CDPATH= cd -- "$(dirname "$guile_bin")/.." && pwd) +guile_lib_dir=$guile_prefix/lib +if [ -e "$guile_lib_dir/libguile-3.0.so.1" ]; then + if [ -n "${LD_LIBRARY_PATH:-}" ]; then + export LD_LIBRARY_PATH="$guile_lib_dir:$LD_LIBRARY_PATH" + else + export LD_LIBRARY_PATH="$guile_lib_dir" + fi +fi + +cleanup=0 +if [ -n "${WORKDIR:-}" ]; then + workdir=$WORKDIR + mkdir -p "$workdir" +else + workdir=$(mktemp -d /tmp/fruix-gnu-hello-guix.XXXXXX) + cleanup=1 +fi + +if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then + cleanup=0 +fi + +cleanup_workdir() { + if [ "$cleanup" -eq 1 ]; then + rm -rf "$workdir" + fi +} +trap cleanup_workdir EXIT INT TERM + +export GUILE_BIN="$guile_bin" +export GUIX_SOURCE_DIR="$guix_source_dir" +export WORKDIR="$workdir" +export GUILE_AUTO_COMPILE=0 +if [ -n "${GUILE_LOAD_PATH:-}" ]; then + export GUILE_LOAD_PATH="$guix_source_dir:$GUILE_LOAD_PATH" +else + export GUILE_LOAD_PATH="$guix_source_dir" +fi + +printf 'Using Guile: %s\n' "$guile_bin" +printf 'Using LD_LIBRARY_PATH: %s\n' "${LD_LIBRARY_PATH:-}" +printf 'Working directory: %s\n' "$workdir" + +"$guile_bin" -s "$runner_scm"