fruix/docs/PROGRESS.md

# Progress

## 2026-04-01 — Phase 1.1 started: Guile verified on FreeBSD amd64

Completed work:

- installed/confirmed `guile3-3.0.10`
- added a reusable verification harness:
  - `tests/guile/run-phase1-verification.sh`
  - `tests/guile/verify-phase1.scm`
  - `tests/guile/modules/phase1/sample.scm`
- verified the following on `FreeBSD 15.0-STABLE` amd64:
  - module loading
  - deterministic output generation
  - file I/O
  - process handling with `primitive-fork`/`waitpid`
  - loopback TCP sockets
  - FFI calls into libc
  - execution of Guix bootstrap-related code from `(guix build make-bootstrap)`
- wrote the results to `docs/reports/phase1-guile-freebsd.md`

Notable findings:

- `guile3` and `guile-3.0` are present, but there is no unversioned `guile` binary
- `system*` and `open-pipe*` currently segfault on this host (`exit 139`)
- despite that crash, the lower-level process primitives needed for further investigation do work

Current assessment:

- Phase 1.1 has a solid amd64 smoke-verification baseline
- Phase 1.1 is not fully complete yet because `i386` has not been checked and the subprocess crash needs investigation
- verification harness committed as `e380e88` (`Add FreeBSD Guile verification harness`)

## 2026-04-01 — Phase 1.1 follow-up: subprocess crash isolated

Completed work:

- added a dedicated subprocess diagnostic harness:
  - `tests/guile/run-subprocess-diagnostics.sh`
  - `tests/guile/posix-spawn-freebsd-diagnostics.c`
- reproduced crashes for:
  - `system*`
  - `spawn`
  - `open-pipe*`
- confirmed all three fail with `SIGSEGV` / `exit 139`
- confirmed native FreeBSD `posix_spawn` + `posix_spawn_file_actions_addclosefrom_np` works in a standalone C program
- confirmed FreeBSD behavior that triggers gnulib replacement logic:
  - `posix_spawn_file_actions_adddup2` accepts an invalid fd in the gnulib probe
  - `posix_spawn_file_actions_addopen` accepts an invalid fd in the gnulib probe
  - `posix_spawnp` accepts a shebang-less executable script in the gnulib security probe
- wrote the analysis to `docs/reports/phase1-guile-subprocess-crash.md`

Conclusion:

- this is most likely an upstream Guile/gnulib ABI bug on FreeBSD, not a Guix-specific problem
- likely sequence:
  1. gnulib enables `REPLACE_POSIX_SPAWN=1`
  2. Guile still enables `HAVE_POSIX_SPAWN_FILE_ACTIONS_ADDCLOSEFROM_NP`
  3. Guile passes a gnulib replacement `posix_spawn_file_actions_t` object to native `posix_spawn_file_actions_addclosefrom_np`
  4. libc interprets gnulib struct fields as a native pointer and crashes
- evidence from the lldb core matches this hypothesis (`*fa = 0x0000000600000008`, consistent with gnulib `_allocated=8`, `_used=6`)

Current assessment:

- Phase 1.1 amd64 investigation is now much stronger and has a concrete root-cause hypothesis
- the next practical step is to validate a workaround or patch in Guile so subprocess helpers stop crashing
- after that, continue with Phase 1.2 (minimal native build environment / GNU Hello)

## 2026-04-01 — Phase 1.1 follow-up: local Guile build validated the fix

Completed work:

- installed the additional build tooling needed for a local Guile checkout build:
  - `autoconf`
  - `automake`
  - `libtool`
  - `gettext-tools`
  - `texinfo`
  - `help2man`
  - `gperf`
  - `pkgconf`
- confirmed a FreeBSD-specific bootstrap quirk:
  - Guile `autogen.sh` needs GNU `m4`
  - FreeBSD base `/usr/bin/m4` is not sufficient
  - `M4=gm4 ./autogen.sh` works
- built a disposable validation copy from `~/repos/guile`
- confirmed `~/repos/guile` already contains upstream commit:
  - `eb828801f621d3e130b6fe88cfc4acaa69b98a03`
  - `Don't use posix_spawn_file_actions_addclosefrom_np with glib posix_spawn`
- updated the local test harnesses so they can test non-system Guile builds:
  - `tests/guile/run-phase1-verification.sh`
  - `tests/guile/run-subprocess-diagnostics.sh`
  - both now accept `GUILE_BIN`
  - both now prepend the sibling `../lib` directory to `LD_LIBRARY_PATH` when a matching local `libguile-3.0.so.1` exists
  - subprocess diagnostics now supports `EXPECT_GUILE_SUBPROCESS_CRASH=0` for fixed builds
- validated that the packaged Guile still reproduces the crash
- validated that the locally built Guile succeeds for:
  - `system*`
  - `spawn`
  - `open-pipe*`
- re-ran the broader Phase 1.1 Scheme verification suite successfully against the local Guile build
- wrote the results to `docs/reports/phase1-guile-local-build-validation.md`

Important findings:

- the local Guile executable initially still crashed until it was forced to load its matching local `libguile-3.0.so.1`
- once `LD_LIBRARY_PATH` pointed at the local install lib directory, subprocess helpers worked correctly
- this strongly supports the earlier diagnosis and shows that the upstream Guile fix resolves the problem in practice
- the local `~/repos/bdwgc` checkout was not needed for this step; packaged `boehm-gc-threaded` was sufficient so far

Current assessment:

- Phase 1.1 now has both a root-cause analysis and a working validated fix path on amd64
- no source changes were needed in `~/repos/guile` because the local checkout already contains the relevant upstream fix
- no source changes were needed in `~/repos/bdwgc` yet, but the earlier FreeBSD warning keeps it on the watch list
- the project can now move on to Phase 1.2 with a known-good local Guile fallback

## 2026-04-01 — Phase 1.2 started: native GNU Hello build validated on FreeBSD

Completed work:

- added a reusable native build harness:
  - `tests/native-build/run-gnu-hello.sh`
- used the current Guix package definition in `~/repos/guix/gnu/packages/base.scm` as the source of truth for:
  - GNU Hello version `2.12.3`
  - expected Guix nix-base32 source hash `183a6rxnhixiyykd7qis0y9g9cfqhpkk872a245y3zl28can0pqd`
- verified the downloaded tarball against the translated SHA256:
  - `0d5f60154382fee10b114a1c34e785d8b1f492073ae2d3a6f7b147687b366aa0`
- successfully executed the standard native build lifecycle on `FreeBSD 15.0-STABLE` amd64:
  - fetch
  - hash verification
  - extract
  - configure
  - build
  - staged install
  - runtime execution
- confirmed the staged binary runs and prints:
  - `Hello, world!`
- captured build metadata including:
  - compiler and make versions
  - host triplet
  - configure command
  - staged output path
  - runtime shared-library dependencies
- wrote the results to `docs/reports/phase1-native-gnu-hello.md`

Important findings:

- GNU Hello built successfully with FreeBSD base `make`, not just `gmake`
- that contrasts with the earlier local Guile build, which did require GNU `gmake`
- even this minimal GNU package links against FreeBSD-userland-provided libraries such as `libiconv` and `libintl`, which is useful data for later Guix package modeling
- this step is still a native shell-driven build exercise, not yet a real Guix package build

Current assessment:

- Phase 1.2 now has a concrete native autotools success case on FreeBSD
- the host can perform the basic fetch/verify/configure/build/install/run cycle needed for later `gnu-build-system` adaptation work
- Guix-specific build orchestration is still missing, but the environmental baseline is stronger now

Recent commits:

- `e380e88` — `Add FreeBSD Guile verification harness`
- `cd721b1` — `Update progress after Guile verification`
- `27916cb` — `Diagnose Guile subprocess crash on FreeBSD`
- `02f7a7f` — `Validate local Guile fix on FreeBSD`

Next recommended step:

1. extend Phase 1.2 with at least one additional representative GNU/autotools package build on FreeBSD, or
2. prototype a tiny Scheme-based `gnu-build-system`-like phase runner using the known-good local Guile path, starting from the GNU Hello flow
3. continue keeping `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear

## 2026-04-01 — Phase 1.2 follow-up: Guix builder-side GNU Hello phase runner validated

Completed work:

- added a Scheme-driven GNU Hello build prototype:
  - `tests/native-build/gnu-hello-guix-phase-runner.scm`
  - `tests/native-build/run-gnu-hello-guix-phase-runner.sh`
- required the previously validated fixed local Guile build for this harness because it depends on subprocess-heavy Scheme operations
- used Guix modules directly from `~/repos/guix`, including:
  - `(guix base32)`
  - `(guix build gnu-build-system)`
  - `(guix build utils)`
- fetched and hash-verified GNU Hello `2.12.3` again against the Guix package hash:
  - nix-base32: `183a6rxnhixiyykd7qis0y9g9cfqhpkk872a245y3zl28can0pqd`
  - SHA256: `0d5f60154382fee10b114a1c34e785d8b1f492073ae2d3a6f7b147687b366aa0`
- successfully executed a subset of Guix builder-side `%standard-phases` on FreeBSD:
  - `set-SOURCE-DATE-EPOCH`
  - `unpack`
  - `configure`
  - `build`
  - `check`
  - `install`
- installed GNU Hello into a store-like output path under the temporary work directory rather than using a `/usr/local` `DESTDIR` staging layout
- executed the resulting binary successfully and confirmed output:
  - `Hello, world!`
- captured metadata including:
  - host triplet
  - selected phase list
  - runtime dependencies
  - test-suite summary
- wrote the results to `docs/reports/phase1-guix-gnu-hello-phase-runner.md`

Important findings:

- this is the first validation step in the repo that successfully exercised actual Guix builder-side GNU build logic on FreeBSD instead of only a shell approximation
- the harness works when driven by the fixed local Guile build, confirming that the earlier Guile subprocess-fix validation is directly useful for FreeBSD Guix build orchestration
- GNU Hello's `make check` test suite also passed in this mode:
  - total: `7`
  - pass: `7`
  - fail: `0`
- the resulting binary's runtime dependencies differ from the earlier `/usr/local`-prefixed native shell harness; in this store-like output layout it only showed:
  - `libc.so.7`
  - `libsys.so.7`
- that difference is a useful clue that Guix-style output layout/build invocation can materially affect FreeBSD runtime linkage behavior

Current assessment:

- Phase 1.2 now has both:
  - a shell-driven native GNU Hello build harness, and
  - a Scheme-driven prototype that uses real Guix builder-side GNU phases
- this is still short of a true Guix package/derivation build, but it significantly narrows the gap between host validation and real `gnu-build-system` execution on FreeBSD
- the known-good local Guile path is now validated as part of a practical Guix-adjacent build workflow, not just standalone subprocess diagnostics

Recent commits:

- `e380e88` — `Add FreeBSD Guile verification harness`
- `cd721b1` — `Update progress after Guile verification`
- `27916cb` — `Diagnose Guile subprocess crash on FreeBSD`
- `02f7a7f` — `Validate local Guile fix on FreeBSD`
- `4aebea4` — `Add native GNU Hello FreeBSD build harness`

Next recommended step:

1. run the Scheme-driven phase-runner pattern against at least one more small GNU/autotools package on FreeBSD, or
2. document the concrete gaps between this prototype and a real Guix package/derivation build, especially around store management and build isolation
3. continue keeping `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear

## 2026-04-01 — Phase 1.2 follow-up: second Scheme-driven GNU package build validated with GNU which

Completed work:

- added a second Scheme-driven GNU package harness:
  - `tests/native-build/gnu-which-guix-phase-runner.scm`
  - `tests/native-build/run-gnu-which-guix-phase-runner.sh`
- again used the previously validated fixed local Guile build because this harness depends on subprocess-heavy Guix/Scheme builder logic
- used the current Guix package definition in `~/repos/guix/gnu/packages/base.scm` as the source of truth for:
  - GNU which version `2.21`
  - expected Guix nix-base32 source hash `1bgafvy3ypbhhfznwjv1lxmd6mci3x1byilnnkc7gcr486wlb8pl`
- verified the downloaded tarball against the translated SHA256:
  - `f4a245b94124b377d8b49646bf421f9155d36aa7614b6ebf83705d3ffc76eaad`
- successfully executed the same subset of Guix builder-side `%standard-phases` on FreeBSD as used for GNU Hello:
  - `set-SOURCE-DATE-EPOCH`
  - `unpack`
  - `configure`
  - `build`
  - `check`
  - `install`
- executed the resulting `which` binary successfully with a deterministic command:
  - `PATH=/bin:/usr/bin ./which sh`
- confirmed output:
  - `/bin/sh`
- captured metadata including:
  - host triplet
  - phase list
  - runtime dependencies
  - check-phase success status
  - executed command output
- wrote the results to `docs/reports/phase1-guix-which-phase-runner.md`

Important findings:

- this confirms the Scheme-driven Guix builder-side phase-runner pattern is not limited to GNU Hello; a second small GNU/autotools package also succeeds on FreeBSD
- GNU which's `check` phase passed, but it did not leave behind an Automake-style `test-suite.log` or `testsuite.log`
- GNU which emitted a non-fatal `configure` warning about Guix's standard `--enable-fast-install` flag being unrecognized
- the source also emitted several clang warnings about deprecated non-prototype C declarations/definitions, but the build still completed successfully
- the resulting `which` binary again showed a minimal store-like runtime linkage profile:
  - `libc.so.7`
  - `libsys.so.7`
- unlike GNU Hello, the source tree did not present an obvious shipped `config.guess`, so the harness used `cc -dumpmachine` as a fallback for host-triplet metadata

Current assessment:

- Phase 1.2 now has two successful Scheme-driven Guix builder-side GNU package validations on FreeBSD:
  - GNU Hello
  - GNU which
- this increases confidence that a narrow but real subset of `gnu-build-system` builder-side execution already works on FreeBSD when paired with the fixed local Guile build
- the next uncertainty is now less about whether basic builder phases run at all, and more about where real Guix package/derivation/store integration and isolation will first require FreeBSD-specific adaptation

Recent commits:

- `e380e88` — `Add FreeBSD Guile verification harness`
- `cd721b1` — `Update progress after Guile verification`
- `27916cb` — `Diagnose Guile subprocess crash on FreeBSD`
- `02f7a7f` — `Validate local Guile fix on FreeBSD`
- `4aebea4` — `Add native GNU Hello FreeBSD build harness`
- `c944cdb` — `Validate Guix builder phases on FreeBSD`

Next recommended step:

1. document the concrete remaining gap between these Scheme-driven phase-runner prototypes and a true Guix package/derivation/store-daemon build on FreeBSD, especially around store management, implicit inputs, and build isolation
2. or choose a somewhat more demanding GNU package with non-trivial declared inputs to identify the first builder-side FreeBSD adaptation points
3. continue keeping `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear