Investigate Guix derivation generation on FreeBSD

2026-04-01 10:10:24 +02:00
parent 245a47dfbc
commit d62e9b01ed
3 changed files with 360 additions and 0 deletions
--- a/docs/PROGRESS.md
+++ b/docs/PROGRESS.md
@@ -366,3 +366,68 @@ Next recommended step:
 1. investigate whether a tiny package can be lowered far enough on FreeBSD to produce a real derivation, and capture the exact first failure point
 2. if derivation generation proves immediately blocked, document whether the blocker is generated Guix modules/configuration, store connectivity, or daemon assumptions
 3. continue keeping `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear
+
+## 2026-04-01 — Phase 1.2 follow-up: derivation-generation investigation identified the first real checkout blockers
+
+Completed work:
+
+- added a reproducible checkout/bootstrap/configure investigation harness:
+  - `tests/guix/run-derivation-generation-investigation.sh`
+- used the previously validated fixed local Guile build for the investigation:
+  - `/tmp/guile-freebsd-validate-install/bin/guile`
+- followed the operator instruction for future store setup by parameterizing the checkout attempts to use:
+  - store directory: `/frx/store`
+  - local state directory: `/frx/var`
+  - sysconf directory: `/frx/etc`
+- created a disposable shared clone of `~/repos/guix`
+- successfully ran `./bootstrap` from that disposable checkout on FreeBSD
+- attempted `configure` without `--with-courage`
+- attempted `configure` again with `--with-courage`
+- confirmed directly that the local fixed Guile build currently cannot load:
+  - `(gnutls)`
+- wrote the results to:
+  - `docs/reports/phase1-guix-derivation-generation-investigation.md`
+
+Important findings:
+
+- a stock Guix checkout currently fails configuration on FreeBSD before any derivation/store/daemon work is reached, due to the explicit unsupported-platform gate:
+  - ``configure: error: `x86_64-freebsd15.0' is not a supported platform.``
+- re-running `configure` with `--with-courage` gets past that gate, but then stops on a second blocker:
+  - `configure: error: The Guile bindings of GnuTLS are missing; please install them.`
+- a direct module-load test with the same local Guile confirms the problem more concretely:
+  - `(use-modules (gnutls))` fails with `no code for module (gnutls)`
+- this means the current effort is still blocked before reaching:
+  - usable `pre-inst-env` generation for Guix commands
+  - package -> bag lowering in a live checkout
+  - bag -> derivation lowering
+  - daemon connectivity
+  - actual `/frx/store` population
+
+Current assessment:
+
+- the first practical boundary between the earlier FreeBSD builder-phase prototypes and a real Guix checkout has now been located more precisely
+- the project is no longer blocked by vague uncertainty at this stage; it is blocked by two concrete checkout-preparation issues:
+  1. unsupported-platform configure gating
+  2. missing Guile `(gnutls)` bindings
+- importantly, the derivation-generation investigation has not yet reached the store/daemon boundary, so the next step must first clear the `(gnutls)` dependency issue
+
+Recent commits:
+
+- `e380e88` — `Add FreeBSD Guile verification harness`
+- `cd721b1` — `Update progress after Guile verification`
+- `27916cb` — `Diagnose Guile subprocess crash on FreeBSD`
+- `02f7a7f` — `Validate local Guile fix on FreeBSD`
+- `4aebea4` — `Add native GNU Hello FreeBSD build harness`
+- `c944cdb` — `Validate Guix builder phases on FreeBSD`
+- `0a2e48e` — `Validate GNU which builder phases on FreeBSD`
+- `245a47d` — `Document gaps to real Guix FreeBSD builds`
+
+Next recommended step:
+
+1. obtain working Guile `(gnutls)` bindings compatible with the fixed local Guile build and re-run the derivation-generation investigation
+2. once configuration succeeds, continue until the next failure boundary is identified among:
+  - `pre-inst-env` usability
+  - derivation emission
+  - daemon connectivity
+  - daemon-side `/frx/store` assumptions
+3. continue keeping `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear
--- a/docs/reports/phase1-guix-derivation-generation-investigation.md
+++ b/docs/reports/phase1-guix-derivation-generation-investigation.md
@@ -0,0 +1,135 @@
+# Phase 1.2 follow-up: derivation-generation investigation from a Guix checkout on FreeBSD
+
+Date: 2026-04-01
+
+## Summary
+
+This step investigated how far FreeBSD can get toward a real Guix package/derivation build from a Guix source checkout, and captured the first concrete failure boundary.
+
+Added harness:
+
+- `tests/guix/run-derivation-generation-investigation.sh`
+
+Per operator instruction, all store-related configuration in this investigation used:
+
+- store directory: `/frx/store`
+
+and, to keep the experimental Guix state alongside that store prefix, the configure attempts also used:
+
+- local state directory: `/frx/var`
+- sysconf directory: `/frx/etc`
+
+## Goal
+
+The immediate goal was not to complete a full build, but to answer a narrower question left open by the previous reports:
+
+> What is the first actual blocker when trying to move from FreeBSD builder-phase prototypes toward a real Guix checkout capable of derivation generation?
+
+## What was attempted
+
+A disposable shared clone of `~/repos/guix` was created under `/tmp`, then:
+
+1. `./bootstrap` was run successfully
+2. `configure` was run without `--with-courage`
+3. `configure` was run again with `--with-courage`
+
+The harness used the previously validated fixed local Guile build:
+
+- `/tmp/guile-freebsd-validate-install/bin/guile`
+
+## Reproduction command
+
+```sh
+METADATA_OUT=/tmp/guix-derivation-investigation.txt \
+  ./tests/guix/run-derivation-generation-investigation.sh
+```
+
+## Result
+
+The investigation did not reach derivation generation yet.
+
+Instead, it identified two earlier gating failures.
+
+### Finding 1: unsupported-platform gate in `configure`
+
+Without `--with-courage`, configuration fails immediately on FreeBSD with:
+
+```text
+checking for the Guix system type... x86_64-freebsd15.0
+configure: error: `x86_64-freebsd15.0' is not a supported platform.
+```
+
+This confirms that a stock Guix checkout currently refuses to configure on FreeBSD before any daemon/store/derivation logic is exercised.
+
+### Finding 2: missing Guile `(gnutls)` bindings after bypassing the platform gate
+
+When `configure` is re-run with `--with-courage`, it gets past the unsupported-platform gate but then stops with:
+
+```text
+configure: error: The Guile bindings of GnuTLS are missing; please install them.
+```
+
+A direct check with the same local fixed Guile build confirmed that:
+
+```scheme
+(use-modules (gnutls))
+```
+
+currently fails on this host with:
+
+```text
+no code for module (gnutls)
+```
+
+## Interpretation
+
+This is useful because it pinpoints the current failure boundary more precisely than before.
+
+The first blockers toward a real Guix checkout capable of derivation generation on FreeBSD are currently:
+
+1. a deliberate unsupported-platform configure gate
+2. a missing mandatory Guile `(gnutls)` module dependency needed by the checkout configuration
+
+This means the project is still blocked **before**:
+
+- `pre-inst-env` generation usable for Guix commands
+- package -> bag lowering in a live Guix checkout
+- bag -> derivation lowering
+- store daemon connection
+- `/frx/store` population or management
+
+## What this does and does not prove
+
+This step proves:
+
+- a Guix checkout can be bootstrapped successfully on FreeBSD
+- Guix's configure logic explicitly rejects FreeBSD unless `--with-courage` is used
+- after bypassing that gate, the next immediate blocker is missing Guile GnuTLS bindings
+
+This step does **not** yet prove:
+
+- whether derivation generation itself works on FreeBSD
+- whether daemon connectivity works
+- whether `/frx/store` can be initialized and used successfully
+- whether the next blocker after GnuTLS would be store-related or daemon-related
+
+## Notes on `/frx`
+
+No actual `/frx/store` was created or populated in this step because the checkout did not get far enough.
+
+However, this investigation establishes the intended experimental store path for subsequent work:
+
+- `/frx/store`
+
+rather than Guix's usual `/gnu/store`.
+
+## Recommended next step
+
+The most direct next step is to obtain working Guile `(gnutls)` bindings compatible with the fixed local Guile build, then re-run this derivation-generation investigation.
+
+That should reveal the next actual failure boundary, which is likely to be one of:
+
+- generated checkout/runtime module issues
+- derivation emission
+- daemon connectivity
+- daemon-side assumptions about the store or build environment
--- a/tests/guix/run-derivation-generation-investigation.sh
+++ b/tests/guix/run-derivation-generation-investigation.sh
@@ -0,0 +1,160 @@
+#!/bin/sh
+set -eu
+
+source_repo=${GUIX_SOURCE_REPO:-"$HOME/repos/guix"}
+store_dir=${STORE_DIR:-/frx/store}
+localstatedir=${LOCALSTATEDIR:-/frx/var}
+sysconfdir=${SYSCONFDIR:-/frx/etc}
+
+guile_bin=${GUILE_BIN:-/tmp/guile-freebsd-validate-install/bin/guile}
+if [ ! -x "$guile_bin" ]; then
+  echo "Guile binary is not executable: $guile_bin" >&2
+  exit 1
+fi
+
+guile_bindir=$(CDPATH= cd -- "$(dirname "$guile_bin")" && pwd)
+export PATH="$guile_bindir:/usr/local/bin:$PATH"
+export ACLOCAL_PATH=/usr/local/share/aclocal${ACLOCAL_PATH:+:$ACLOCAL_PATH}
+
+cleanup=0
+if [ -n "${WORKDIR:-}" ]; then
+  workdir=$WORKDIR
+  mkdir -p "$workdir"
+else
+  workdir=$(mktemp -d /tmp/fruix-guix-derivation-investigation.XXXXXX)
+  cleanup=1
+fi
+
+if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
+  cleanup=0
+fi
+
+cleanup_workdir() {
+  if [ "$cleanup" -eq 1 ]; then
+    rm -rf "$workdir"
+  fi
+}
+trap cleanup_workdir EXIT INT TERM
+
+srcclone=$workdir/guix-src
+build_unsupported=$workdir/build-unsupported
+build_courage=$workdir/build-with-courage
+bootstrap_log=$workdir/bootstrap.log
+unsupported_log=$workdir/configure-unsupported.log
+courage_log=$workdir/configure-with-courage.log
+metadata_file=$workdir/derivation-generation-investigation.txt
+gnutls_check_out=$workdir/guile-gnutls-check.out
+gnutls_check_err=$workdir/guile-gnutls-check.err
+
+printf 'Working directory: %s\n' "$workdir"
+printf 'Cloning source from: %s\n' "$source_repo"
+git clone --shared "$source_repo" "$srcclone" >/dev/null 2>&1
+
+(
+  cd "$srcclone"
+  ./bootstrap
+) >"$bootstrap_log" 2>&1
+
+mkdir -p "$build_unsupported" "$build_courage"
+
+set +e
+(
+  cd "$build_unsupported"
+  PKG_CONFIG_PATH=/usr/local/libdata/pkgconfig:/usr/local/lib/pkgconfig \
+  CPPFLAGS='-I/usr/local/include' \
+  LDFLAGS='-L/usr/local/lib -Wl,-rpath,/usr/local/lib' \
+  GUILE="$guile_bin" \
+  GUILE_EFFECTIVE_VERSION=3.0 \
+  "$srcclone/configure" \
+    --with-store-dir="$store_dir" \
+    --localstatedir="$localstatedir" \
+    --sysconfdir="$sysconfdir"
+) >"$unsupported_log" 2>&1
+unsupported_rc=$?
+
+(
+  cd "$build_courage"
+  PKG_CONFIG_PATH=/usr/local/libdata/pkgconfig:/usr/local/lib/pkgconfig \
+  CPPFLAGS='-I/usr/local/include' \
+  LDFLAGS='-L/usr/local/lib -Wl,-rpath,/usr/local/lib' \
+  GUILE="$guile_bin" \
+  GUILE_EFFECTIVE_VERSION=3.0 \
+  "$srcclone/configure" \
+    --with-courage \
+    --with-store-dir="$store_dir" \
+    --localstatedir="$localstatedir" \
+    --sysconfdir="$sysconfdir"
+) >"$courage_log" 2>&1
+courage_rc=$?
+set -e
+
+unsupported_summary=$(grep -E "supported platform|Guix system type|configure: error" "$unsupported_log" | tail -n 5 || true)
+courage_summary=$(grep -E "supported platform|GnuTLS|guile 3.0|configure: error|configure: WARNING" "$courage_log" | tail -n 8 || true)
+
+local_gnutls_check=missing
+set +e
+LD_LIBRARY_PATH="$(CDPATH= cd -- "$guile_bindir/.." && pwd)/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" \
+  "$guile_bin" -c '(use-modules (gnutls)) (display "ok") (newline)' >"$gnutls_check_out" 2>"$gnutls_check_err"
+check_rc=$?
+set -e
+if [ "$check_rc" -eq 0 ]; then
+  local_gnutls_check=present
+fi
+
+if [ "$unsupported_rc" -eq 0 ]; then
+  echo "configure without --with-courage unexpectedly succeeded" >&2
+  exit 1
+fi
+if ! grep -q "not a supported platform" "$unsupported_log"; then
+  echo "configure without --with-courage failed, but not for the expected unsupported-platform reason" >&2
+  exit 1
+fi
+if [ "$courage_rc" -eq 0 ]; then
+  echo "configure with --with-courage unexpectedly succeeded; investigation expectations need updating" >&2
+  exit 1
+fi
+if ! grep -q "Guile bindings of GnuTLS are missing" "$courage_log"; then
+  echo "configure with --with-courage failed, but not for the expected missing-GnuTLS-bindings reason" >&2
+  exit 1
+fi
+
+cat >"$metadata_file" <<EOF
+source_repo=$source_repo
+srcclone=$srcclone
+guile_bin=$guile_bin
+store_dir=$store_dir
+localstatedir=$localstatedir
+sysconfdir=$sysconfdir
+bootstrap_log=$bootstrap_log
+configure_unsupported_log=$unsupported_log
+configure_with_courage_log=$courage_log
+configure_without_courage_rc=$unsupported_rc
+configure_with_courage_rc=$courage_rc
+local_guile_gnutls_module=$local_gnutls_check
+unsupported_summary_begin
+$unsupported_summary
+unsupported_summary_end
+courage_summary_begin
+$courage_summary
+courage_summary_end
+EOF
+
+if [ -n "${METADATA_OUT:-}" ]; then
+  mkdir -p "$(dirname "$METADATA_OUT")"
+  cp "$metadata_file" "$METADATA_OUT"
+fi
+
+printf 'PASS guix-derivation-generation-investigation\n'
+printf 'Store dir under test: %s\n' "$store_dir"
+printf 'Finding 1: configure without --with-courage is blocked by unsupported platform gating.\n'
+printf 'Finding 2: configure with --with-courage is currently blocked by missing Guile (gnutls) bindings.\n'
+if [ "$cleanup" -eq 0 ]; then
+  printf 'Metadata file: %s\n' "$metadata_file"
+else
+  printf '%s\n' 'Work directory will be removed on exit (set KEEP_WORKDIR=1 to preserve it).'
+fi
+if [ -n "${METADATA_OUT:-}" ]; then
+  printf 'Copied metadata to: %s\n' "$METADATA_OUT"
+fi
+printf '%s\n' '--- metadata ---'
+cat "$metadata_file"