diff --git a/docs/PROGRESS.md b/docs/PROGRESS.md index 501ad18..2f8d980 100644 --- a/docs/PROGRESS.md +++ b/docs/PROGRESS.md @@ -431,3 +431,71 @@ Next recommended step: - daemon connectivity - daemon-side `/frx/store` assumptions 3. continue keeping `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear + +## 2026-04-01 — Phase 1.2 follow-up: local Guile-GnuTLS built on FreeBSD; next blocker is Guile-Git + +Completed work: + +- installed the host-side C GnuTLS stack needed for Guile-GnuTLS builds: + - `gnutls` + - `libtasn1` + - `nettle` + - `p11-kit` +- added a reproducible local Guile-GnuTLS build harness: + - `tests/guix/build-local-guile-gnutls.sh` +- updated the derivation-generation investigation harness so it can consume extra Guile module prefixes through: + - `GUILE_EXTRA_PREFIX` +- used the current Guix package definition in `~/repos/guix/gnu/packages/tls.scm` as the source of truth for: + - `guile-gnutls` version `5.0.1` + - expected Guix nix-base32 source hash `0kqngyx4520gjk49l6whjd2ss994kaj9rm78lli6p3q6xry0945i` +- verified the downloaded Guile-GnuTLS tarball against the translated SHA256: + - `b190047cee068f6b22a5e8d49ca49a2425ad4593901b9ac8940f8842ba7f164f` +- built and installed a local Guile-GnuTLS validation copy against the previously validated fixed local Guile build under: + - `/tmp/guile-gnutls-freebsd-validate-install` +- validated successfully that the fixed local Guile can now load: + - `(gnutls)` +- re-ran the checkout derivation-generation investigation with: + - `GUILE_EXTRA_PREFIX=/tmp/guile-gnutls-freebsd-validate-install` + - store directory still set to `/frx/store` +- wrote the results to: + - `docs/reports/phase1-guile-gnutls-freebsd.md` + +Important findings: + +- Guile-GnuTLS does not build with FreeBSD base `make`; it requires GNU `gmake` +- a FreeBSD-specific source compatibility issue surfaced in `guile/src/core.c`: + - it includes `` unconditionally + - on this host, `alloca` is available through `` instead +- for local validation, the harness applies a small disposable-tree patch that uses `` on `__FreeBSD__` +- after that fix, the local Guile-GnuTLS build succeeded and `(use-modules (gnutls))` worked with the fixed local Guile build +- re-running the Guix checkout investigation confirms the earlier `(gnutls)` blocker is genuinely cleared +- the next configure-time blocker is now: + - `configure: error: Guile-Git is missing; please install it.` + +Current assessment: + +- the first checkout-preparation blocker after unsupported-platform gating has advanced from missing `(gnutls)` to missing `Guile-Git` +- this is meaningful progress because the project is now moving farther into the dependency chain required for a real Guix checkout on FreeBSD +- the requested experimental store path remains `/frx/store`, but the effort still has not yet reached actual store population or daemon interaction + +Recent commits: + +- `e380e88` — `Add FreeBSD Guile verification harness` +- `cd721b1` — `Update progress after Guile verification` +- `27916cb` — `Diagnose Guile subprocess crash on FreeBSD` +- `02f7a7f` — `Validate local Guile fix on FreeBSD` +- `4aebea4` — `Add native GNU Hello FreeBSD build harness` +- `c944cdb` — `Validate Guix builder phases on FreeBSD` +- `0a2e48e` — `Validate GNU which builder phases on FreeBSD` +- `245a47d` — `Document gaps to real Guix FreeBSD builds` +- `d62e9b0` — `Investigate Guix derivation generation on FreeBSD` + +Next recommended step: + +1. obtain `Guile-Git` compatible with the fixed local Guile build and re-run the derivation-generation investigation again +2. once checkout configuration succeeds, continue until the next failure boundary is identified among: + - `pre-inst-env` usability + - derivation emission + - daemon connectivity + - daemon-side `/frx/store` assumptions +3. continue keeping `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear diff --git a/docs/reports/phase1-guile-gnutls-freebsd.md b/docs/reports/phase1-guile-gnutls-freebsd.md new file mode 100644 index 0000000..a66270e --- /dev/null +++ b/docs/reports/phase1-guile-gnutls-freebsd.md @@ -0,0 +1,186 @@ +# Phase 1.2 follow-up: local Guile-GnuTLS built on FreeBSD and checkout investigation advanced + +Date: 2026-04-01 + +## Summary + +This step addressed the previously identified missing Guile `(gnutls)` blocker that prevented a Guix checkout from configuring on FreeBSD even when `--with-courage` was used. + +Added harnesses/updates: + +- `tests/guix/build-local-guile-gnutls.sh` +- updated `tests/guix/run-derivation-generation-investigation.sh` + +The result is: + +1. local Guile-GnuTLS can be built successfully on FreeBSD against the already validated fixed local Guile build +2. after making that module available, the Guix checkout configure step advances past the GnuTLS check +3. the next checkout blocker is now clearly identified as missing `Guile-Git` + +## Inputs used + +### Local Guile + +The build used the previously validated fixed local Guile install: + +- `/tmp/guile-freebsd-validate-install/bin/guile` + +### Guile-GnuTLS source identity + +The source of truth was the current Guix package definition in `~/repos/guix/gnu/packages/tls.scm`: + +- package: `guile-gnutls` +- version: `5.0.1` +- Guix nix-base32: `0kqngyx4520gjk49l6whjd2ss994kaj9rm78lli6p3q6xry0945i` + +Translated and verified SHA256: + +```text +b190047cee068f6b22a5e8d49ca49a2425ad4593901b9ac8940f8842ba7f164f +``` + +### FreeBSD package dependency + +The host did not have the C GnuTLS library installed, so this step installed: + +- `gnutls` +- `libtasn1` +- `nettle` +- `p11-kit` + +via: + +```sh +sudo pkg install -y gnutls +``` + +## FreeBSD-specific build findings + +### 1. GNU make is required + +Guile-GnuTLS uses GNU-make-specific pattern rules in `guile/src/Makefile.am`. + +Attempting to build with FreeBSD base `make` failed with: + +```text +make[3]: don't know how to make core.x. Stop +``` + +Using `gmake` resolved that issue. + +### 2. Guile-GnuTLS needs a FreeBSD-specific `alloca` include fix + +The build then failed in `guile/src/core.c` with: + +```text +fatal error: 'alloca.h' file not found +``` + +On this FreeBSD host, `alloca` is available via `stdlib.h`, not `alloca.h`. + +For validation, the harness applies a small source patch in the disposable tree: + +```c +#ifdef __FreeBSD__ +#include +#else +#include +#endif +``` + +With that patch applied, the build completed successfully. + +## Validation command + +```sh +ENV_OUT=/tmp/guile-gnutls-env.sh \ +METADATA_OUT=/tmp/guile-gnutls-metadata.txt \ +./tests/guix/build-local-guile-gnutls.sh +``` + +Default install prefix used by the harness: + +```text +/tmp/guile-gnutls-freebsd-validate-install +``` + +## Result + +The local Guile-GnuTLS build succeeded. + +Installed artifacts included: + +- `share/guile/site/3.0/gnutls.scm` +- `lib/guile/3.0/site-ccache/gnutls.go` +- `lib/guile/3.0/extensions/guile-gnutls-v-2.so.0.0.0` + +Module validation succeeded with the fixed local Guile build: + +```scheme +(use-modules (gnutls)) +``` + +Observed validation result: + +```text +ok +``` + +The harness also writes an environment file that can be used to expose the built module tree to later commands. + +## Re-running the checkout investigation + +After building Guile-GnuTLS, the derivation-generation investigation was re-run with: + +```sh +GUILE_EXTRA_PREFIX=/tmp/guile-gnutls-freebsd-validate-install \ +METADATA_OUT=/tmp/guix-derivation-investigation-with-gnutls.txt \ +./tests/guix/run-derivation-generation-investigation.sh +``` + +The investigation still uses the requested experimental store/state paths: + +- store: `/frx/store` +- local state: `/frx/var` +- sysconf: `/frx/etc` + +## Updated result + +With `(gnutls)` available, the Guix checkout gets past the previous GnuTLS blocker. + +The configure step now fails at the next missing Guile dependency: + +```text +configure: error: Guile-Git is missing; please install it. +``` + +This is a meaningful advancement because it proves the earlier GnuTLS blocker is genuinely cleared and that the checkout can proceed farther toward a usable `pre-inst-env` environment. + +## What this step demonstrates + +This step demonstrates that: + +1. missing Guile `(gnutls)` support on FreeBSD is not a dead end +2. a local, compatible Guile-GnuTLS build can be produced against the fixed local Guile build +3. FreeBSD now reaches the next concrete checkout blocker for Guix source-tree configuration: `Guile-Git` + +## Current blocker stack + +The current checkout path toward real derivation generation is now blocked by: + +1. unsupported-platform configure gating unless `--with-courage` is used +2. missing `Guile-Git` after GnuTLS support is supplied + +The earlier missing `(gnutls)` blocker has been cleared for local validation purposes. + +## Recommended next step + +Build or otherwise obtain `Guile-Git` compatible with the same fixed local Guile build, then re-run the derivation-generation investigation again. + +That should reveal whether the next boundary lies in: + +- another Guile module dependency +- `pre-inst-env` usability +- derivation emission +- daemon connectivity +- `/frx/store`/daemon assumptions diff --git a/tests/guix/build-local-guile-gnutls.sh b/tests/guix/build-local-guile-gnutls.sh new file mode 100755 index 0000000..1555cef --- /dev/null +++ b/tests/guix/build-local-guile-gnutls.sh @@ -0,0 +1,208 @@ +#!/bin/sh +set -eu + +source_url=${GUILE_GNUTLS_SOURCE_URL:-"https://ftp.gnu.org/gnu/gnutls/guile-gnutls-v5.0.1-src.tar.gz"} +version=${GUILE_GNUTLS_VERSION:-5.0.1} +expected_nix_base32=${GUILE_GNUTLS_NIX_BASE32:-0kqngyx4520gjk49l6whjd2ss994kaj9rm78lli6p3q6xry0945i} +install_prefix=${INSTALL_PREFIX:-/tmp/guile-gnutls-freebsd-validate-install} +guile_bin=${GUILE_BIN:-/tmp/guile-freebsd-validate-install/bin/guile} +guix_source_dir=${GUIX_SOURCE_DIR:-"$HOME/repos/guix"} +make_bin=${MAKE_BIN:-gmake} + +if [ ! -x "$guile_bin" ]; then + echo "Guile binary is not executable: $guile_bin" >&2 + exit 1 +fi +if [ ! -d "$guix_source_dir/guix" ]; then + echo "Guix source tree not found at $guix_source_dir" >&2 + exit 1 +fi +if ! command -v fetch >/dev/null 2>&1; then + echo "fetch not found in PATH" >&2 + exit 1 +fi +if ! command -v sha256 >/dev/null 2>&1; then + echo "sha256 not found in PATH" >&2 + exit 1 +fi +if ! command -v bsdtar >/dev/null 2>&1; then + echo "bsdtar not found in PATH" >&2 + exit 1 +fi +if ! command -v "$make_bin" >/dev/null 2>&1; then + echo "Required make tool not found: $make_bin" >&2 + exit 1 +fi + +cleanup=0 +if [ -n "${WORKDIR:-}" ]; then + workdir=$WORKDIR + mkdir -p "$workdir" +else + workdir=$(mktemp -d /tmp/fruix-guile-gnutls.XXXXXX) + cleanup=1 +fi + +if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then + cleanup=0 +fi + +cleanup_workdir() { + if [ "$cleanup" -eq 1 ]; then + rm -rf "$workdir" + fi +} +trap cleanup_workdir EXIT INT TERM + +guile_bindir=$(CDPATH= cd -- "$(dirname "$guile_bin")" && pwd) +guile_prefix=$(CDPATH= cd -- "$guile_bindir/.." && pwd) +guile_lib_dir=$guile_prefix/lib +tool_bindir=$workdir/guile-tools-bin +mkdir -p "$tool_bindir" +ln -sf "$guile_bin" "$tool_bindir/guile-3.0" +ln -sf "$guile_bin" "$tool_bindir/guile" +ln -sf "$guile_bindir/guild" "$tool_bindir/guild-3.0" +ln -sf "$guile_bindir/guild" "$tool_bindir/guild" +ln -sf "$guile_bindir/guile-config" "$tool_bindir/guile-config-3.0" +ln -sf "$guile_bindir/guile-config" "$tool_bindir/guile-config" +ln -sf "$guile_bindir/guile-snarf" "$tool_bindir/guile-snarf" +export PATH="$tool_bindir:$guile_bindir:/usr/local/bin:$PATH" +export ACLOCAL_PATH=/usr/local/share/aclocal${ACLOCAL_PATH:+:$ACLOCAL_PATH} +if [ -n "${LD_LIBRARY_PATH:-}" ]; then + export LD_LIBRARY_PATH="$guile_lib_dir:/usr/local/lib:$LD_LIBRARY_PATH" +else + export LD_LIBRARY_PATH="$guile_lib_dir:/usr/local/lib" +fi +export PKG_CONFIG_PATH=/usr/local/libdata/pkgconfig:/usr/local/lib/pkgconfig${PKG_CONFIG_PATH:+:$PKG_CONFIG_PATH} +export CPPFLAGS='-I/usr/local/include' +export LDFLAGS="-L/usr/local/lib -Wl,-rpath,/usr/local/lib -Wl,-rpath,$guile_lib_dir" + +guile_version=$("$guile_bin" -c '(display (effective-version))') +site_dir=$install_prefix/share/guile/site/$guile_version +site_ccache_dir=$install_prefix/lib/guile/$guile_version/site-ccache +extensions_dir=$install_prefix/lib/guile/$guile_version/extensions +env_file=$workdir/guile-gnutls-env.sh +metadata_file=$workdir/guile-gnutls-build-metadata.txt +src_tarball=$workdir/guile-gnutls-v$version-src.tar.gz +src_dir=$workdir/guile-gnutls-v$version +build_dir=$workdir/build +bootstrap_log=$workdir/bootstrap.log +configure_log=$workdir/configure.log +build_log=$workdir/build.log +install_log=$workdir/install.log + +expected_sha256_hex=$(GUILE_AUTO_COMPILE=0 \ + GUILE_LOAD_PATH="$guix_source_dir${GUILE_LOAD_PATH:+:$GUILE_LOAD_PATH}" \ + "$guile_bin" -c "(use-modules (guix base32) (rnrs bytevectors) (ice-9 format)) (for-each (lambda (b) (format #t \"~2,'0x\" b)) (bytevector->u8-list (nix-base32-string->bytevector (cadr (command-line))))) (newline)" \ + "$expected_nix_base32") + +printf 'Using Guile: %s\n' "$guile_bin" +printf 'Installing to: %s\n' "$install_prefix" +printf 'Working directory: %s\n' "$workdir" +printf 'Fetching: %s\n' "$source_url" +fetch -o "$src_tarball" "$source_url" + +actual_sha256_hex=$(sha256 -q "$src_tarball") +if [ "$actual_sha256_hex" != "$expected_sha256_hex" ]; then + echo "sha256 mismatch for $src_tarball" >&2 + echo "expected: $expected_sha256_hex" >&2 + echo "actual: $actual_sha256_hex" >&2 + exit 1 +fi + +bsdtar -xf "$src_tarball" -C "$workdir" +if [ ! -d "$src_dir" ]; then + echo "expected extracted source directory not found: $src_dir" >&2 + exit 1 +fi + +perl -0pi -e 's/#include /#ifdef __FreeBSD__\n#include \n#else\n#include \n#endif/' "$src_dir/guile/src/core.c" + +( + cd "$src_dir" + ./bootstrap +) >"$bootstrap_log" 2>&1 + +rm -rf "$install_prefix" +mkdir -p "$build_dir" +( + cd "$build_dir" + "$src_dir/configure" \ + --prefix="$install_prefix" \ + --disable-static \ + --with-guile-site-dir="$site_dir" \ + --with-guile-site-ccache-dir="$site_ccache_dir" \ + --with-guile-extension-dir="$extensions_dir" +) >"$configure_log" 2>&1 + +( + cd "$build_dir" + "$make_bin" -j"${JOBS:-$(sysctl -n hw.ncpu 2>/dev/null || echo 1)}" +) >"$build_log" 2>&1 + +( + cd "$build_dir" + "$make_bin" install +) >"$install_log" 2>&1 + +cat >"$env_file" <&2 + exit 1 +fi + +cat >"$metadata_file" <"$gnutls_check_out" 2>"$gnutls_check_err" +"$guile_bin" -c '(use-modules (gnutls)) (display "ok") (newline)' >"$gnutls_check_out" 2>"$gnutls_check_err" check_rc=$? set -e if [ "$check_rc" -eq 0 ]; then @@ -113,15 +141,24 @@ if [ "$courage_rc" -eq 0 ]; then echo "configure with --with-courage unexpectedly succeeded; investigation expectations need updating" >&2 exit 1 fi -if ! grep -q "Guile bindings of GnuTLS are missing" "$courage_log"; then - echo "configure with --with-courage failed, but not for the expected missing-GnuTLS-bindings reason" >&2 - exit 1 +if [ "$local_gnutls_check" = missing ]; then + if ! grep -q "Guile bindings of GnuTLS are missing" "$courage_log"; then + echo "configure with --with-courage failed, but not for the expected missing-GnuTLS-bindings reason" >&2 + exit 1 + fi +else + if grep -q "Guile bindings of GnuTLS are missing" "$courage_log"; then + echo "configure with --with-courage still reports missing GnuTLS bindings even though (gnutls) loads" >&2 + exit 1 + fi fi cat >"$metadata_file" <} store_dir=$store_dir localstatedir=$localstatedir sysconfdir=$sysconfdir @@ -147,7 +184,11 @@ fi printf 'PASS guix-derivation-generation-investigation\n' printf 'Store dir under test: %s\n' "$store_dir" printf 'Finding 1: configure without --with-courage is blocked by unsupported platform gating.\n' -printf 'Finding 2: configure with --with-courage is currently blocked by missing Guile (gnutls) bindings.\n' +if [ "$local_gnutls_check" = missing ]; then + printf 'Finding 2: configure with --with-courage is currently blocked by missing Guile (gnutls) bindings.\n' +else + printf 'Finding 2: Guile (gnutls) bindings are available; configure now advances to the next blocker.\n' +fi if [ "$cleanup" -eq 0 ]; then printf 'Metadata file: %s\n' "$metadata_file" else