From 15b9037034545f79ed5caa0d5439e16d6d7eefe6 Mon Sep 17 00:00:00 2001 From: Steffen Beyer Date: Wed, 1 Apr 2026 10:39:33 +0200 Subject: [PATCH] Build local Guile-Git on FreeBSD --- docs/PROGRESS.md | 72 ++++++ docs/reports/phase1-guile-git-freebsd.md | 197 +++++++++++++++ tests/guix/build-local-guile-git.sh | 227 ++++++++++++++++++ ...run-derivation-generation-investigation.sh | 26 +- 4 files changed, 520 insertions(+), 2 deletions(-) create mode 100644 docs/reports/phase1-guile-git-freebsd.md create mode 100755 tests/guix/build-local-guile-git.sh diff --git a/docs/PROGRESS.md b/docs/PROGRESS.md index 2f8d980..c232904 100644 --- a/docs/PROGRESS.md +++ b/docs/PROGRESS.md @@ -499,3 +499,75 @@ Next recommended step: - daemon connectivity - daemon-side `/frx/store` assumptions 3. continue keeping `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear + +## 2026-04-01 — Phase 1.2 follow-up: local Guile-Git stack built on FreeBSD; next blocker is Guile-JSON + +Completed work: + +- added a reproducible local Guile-Git dependency-stack build harness: + - `tests/guix/build-local-guile-git.sh` +- updated the derivation-generation investigation harness to probe and record both: + - local `(gnutls)` availability + - local `(git)` / `graph-descendant?` availability +- used the current Guix package definitions in `~/repos/guix/gnu/packages/guile.scm` as the source of truth for: + - `guile-bytestructures` version `1.0.10` + - `guile-git` version `0.10.0` +- built `guile-bytestructures` from the matching upstream tag and recorded resolved commit: + - `27cadba6b69a01b38b33bb39b9766d713eb90c1b` +- built `guile-git` from the matching upstream tag and recorded resolved commit: + - `05d4a48c811f29c8db80ee6697fe658950fb503e` +- installed both into the same local dependency prefix already used for the earlier Guile-GnuTLS validation: + - `/tmp/guile-gnutls-freebsd-validate-install` +- validated successfully that the fixed local Guile can now load: + - `(bytestructures guile)` + - `(git)` +- validated specifically that the Guile-Git export required by Guix `configure.ac` is present: + - `graph-descendant?` +- confirmed the host `libgit2` dependency used for the build is: + - `1.9.2` +- re-ran the checkout derivation-generation investigation with: + - `GUILE_EXTRA_PREFIX=/tmp/guile-gnutls-freebsd-validate-install` + - store directory still set to `/frx/store` +- wrote the results to: + - `docs/reports/phase1-guile-git-freebsd.md` + +Important findings: + +- both `guile-bytestructures` and `guile-git` were built from Git source layouts, so autotools regeneration was required: + - `guile-bytestructures`: `autoreconf -vfi` + - `guile-git`: `autoreconf -vfi` via harness fallback +- unlike the earlier Guile-GnuTLS step, no additional FreeBSD-specific source patch was needed for either package in this validation pass +- the Guix checkout now gets past both previously cleared dependency gates: + - `(gnutls)` + - `Guile-Git` +- after clearing those, the next configure-time blocker is now: + - `configure: error: Guile-JSON is missing; please install it.` + +Current assessment: + +- the checkout-preparation path on FreeBSD has progressed one dependency layer deeper +- the local validation prefix under `/tmp/guile-gnutls-freebsd-validate-install` now contains at least: + - Guile-GnuTLS + - Guile bytestructures + - Guile-Git +- despite that progress, the work still has not yet reached derivation emission, daemon connectivity, or actual `/frx/store` population +- the next concrete blocker is now `Guile-JSON`, as reported directly by the real Guix checkout configure step + +Recent commits: + +- `e380e88` — `Add FreeBSD Guile verification harness` +- `cd721b1` — `Update progress after Guile verification` +- `27916cb` — `Diagnose Guile subprocess crash on FreeBSD` +- `02f7a7f` — `Validate local Guile fix on FreeBSD` +- `4aebea4` — `Add native GNU Hello FreeBSD build harness` +- `c944cdb` — `Validate Guix builder phases on FreeBSD` +- `0a2e48e` — `Validate GNU which builder phases on FreeBSD` +- `245a47d` — `Document gaps to real Guix FreeBSD builds` +- `d62e9b0` — `Investigate Guix derivation generation on FreeBSD` +- `c0a85ed` — `Build local Guile-GnuTLS on FreeBSD` + +Next recommended step: + +1. obtain `Guile-JSON` compatible with the fixed local Guile build and install it into the same local dependency prefix +2. re-run the derivation-generation investigation again to identify the next configure-time or checkout-time blocker after `Guile-JSON` +3. continue keeping `/frx/store` as the intended experimental store root and keep `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear diff --git a/docs/reports/phase1-guile-git-freebsd.md b/docs/reports/phase1-guile-git-freebsd.md new file mode 100644 index 0000000..9a180be --- /dev/null +++ b/docs/reports/phase1-guile-git-freebsd.md @@ -0,0 +1,197 @@ +# Phase 1.2 follow-up: local Guile-Git stack built on FreeBSD and checkout investigation advanced again + +Date: 2026-04-01 + +## Summary + +This step addressed the next checkout blocker found after local Guile-GnuTLS support was added on FreeBSD. + +Added/updated files: + +- `tests/guix/build-local-guile-git.sh` +- updated `tests/guix/run-derivation-generation-investigation.sh` + +The new result is: + +1. `guile-bytestructures` and `guile-git` can be built successfully on FreeBSD against the previously validated fixed local Guile build +2. these modules can be installed into the same local prefix already used for the Guile-GnuTLS validation copy +3. after making both `(gnutls)` and `(git)` available, the Guix checkout configure step advances again +4. the next checkout blocker is now `Guile-JSON` + +## Inputs used + +### Local Guile + +The same fixed local Guile build was used: + +- `/tmp/guile-freebsd-validate-install/bin/guile` + +### Existing local dependency prefix + +To keep checkout prerequisites together, this step installed into the existing prefix already populated by the earlier Guile-GnuTLS validation step: + +- `/tmp/guile-gnutls-freebsd-validate-install` + +That prefix now contains at least: + +- Guile-GnuTLS +- Guile bytestructures +- Guile-Git + +### Guix source-of-truth package definitions + +This step used the current Guix package definitions in `~/repos/guix/gnu/packages/guile.scm`: + +- `guile-bytestructures` + - version `1.0.10` +- `guile-git` + - version `0.10.0` + +Because both Guix packages use `git-fetch`, the local harness cloned the matching upstream tags and recorded the resolved commits: + +- `guile-bytestructures` tag `1.0.10` + - commit `27cadba6b69a01b38b33bb39b9766d713eb90c1b` +- `guile-git` tag `v0.10.0` + - commit `05d4a48c811f29c8db80ee6697fe658950fb503e` + +### Host-side library dependency + +The host already had the needed `libgit2` package installed. The build used: + +- `libgit2` version `1.9.2` + +observed via: + +```text +pkg-config --modversion libgit2 +``` + +## Build findings + +### 1. Git checkouts need autotools regeneration + +Both packages were built from the Git source layout reflected by the Guix package definitions, so generated configure scripts were not assumed to exist. + +Observed working bootstrap paths: + +- `guile-bytestructures`: `autoreconf -vfi` +- `guile-git`: `autoreconf -vfi` via the harness fallback path + +This matches the fact that the Guix package definitions already list autotools components among native inputs. + +### 2. No new FreeBSD-specific source patch was needed here + +Unlike the earlier Guile-GnuTLS build, neither `guile-bytestructures` nor `guile-git` required a FreeBSD-specific source patch during this validation step. + +### 3. The Guile-Git version/export check now passes locally + +Guix `configure.ac` checks not only that `(git)` can be loaded, but also that the module exports a sufficiently recent symbol: + +- `graph-descendant?` + +The local validation explicitly checked that this export is present and callable enough to satisfy the configure-time requirement. + +## Validation command + +```sh +ENV_OUT=/tmp/guile-git-env.sh \ +METADATA_OUT=/tmp/guile-git-metadata.txt \ +./tests/guix/build-local-guile-git.sh +``` + +Default install prefix used by the harness: + +```text +/tmp/guile-gnutls-freebsd-validate-install +``` + +## Result + +The local Guile dependency stack build succeeded. + +Validated module loads: + +```scheme +(use-modules (bytestructures guile)) +(use-modules (git)) +``` + +Observed checks: + +```text +bytestructures module check: ok +guile-git module check: ok +Existing (gnutls) module in prefix: present +``` + +This means the same local prefix now satisfies at least the first two mandatory Guix checkout module requirements previously encountered on FreeBSD: + +- `(gnutls)` +- `(git)` with `graph-descendant?` + +## Re-running the checkout investigation + +After installing the Guile-Git stack, the derivation-generation investigation was re-run with: + +```sh +GUILE_EXTRA_PREFIX=/tmp/guile-gnutls-freebsd-validate-install \ +METADATA_OUT=/tmp/guix-derivation-investigation-with-git.txt \ +./tests/guix/run-derivation-generation-investigation.sh +``` + +The investigation still used the requested experimental directories: + +- store: `/frx/store` +- local state: `/frx/var` +- sysconf: `/frx/etc` + +## Updated result + +With both `(gnutls)` and `(git)` available, the Guix checkout configure step progresses farther and now fails at the next missing Guile dependency: + +```text +configure: error: Guile-JSON is missing; please install it. +``` + +Relevant configure summary lines observed: + +```text +checking whether Guile-JSON is available and recent enough... no +configure: error: Guile-JSON is missing; please install it. +``` + +## What this step demonstrates + +This step demonstrates that: + +1. the earlier `Guile-Git` blocker is genuinely cleared on FreeBSD when using the fixed local Guile build plus local dependency prefix +2. the checkout can now move one dependency layer deeper than before +3. the project is still in the checkout-prerequisite stage rather than the derivation/store/daemon stage +4. the next concrete prerequisite for reaching deeper Guix checkout usability on FreeBSD is `Guile-JSON` + +## Current blocker stack + +The current checkout path is now blocked by: + +1. unsupported-platform configure gating unless `--with-courage` is used +2. missing `Guile-JSON` after supplying: + - Guile-GnuTLS + - Guile-Git + +The earlier missing blockers have now been cleared locally for validation: + +- `(gnutls)` +- `(git)` / recent Guile-Git export requirement + +## Recommended next step + +Obtain or build `Guile-JSON` compatible with the same fixed local Guile build, install it into the same local dependency prefix, and re-run the derivation-generation investigation again. + +That will show whether the next boundary lies at: + +- `Guile-SQLite3` +- `Guile-Gcrypt` +- `Guile-zlib` +- `Guile-lzlib` +- `Guile-semver` +- or finally something beyond configure-time Guile module prerequisites diff --git a/tests/guix/build-local-guile-git.sh b/tests/guix/build-local-guile-git.sh new file mode 100755 index 0000000..07dbf32 --- /dev/null +++ b/tests/guix/build-local-guile-git.sh @@ -0,0 +1,227 @@ +#!/bin/sh +set -eu + +bytestructures_repo=${GUILE_BYTESTRUCTURES_REPO:-"https://github.com/TaylanUB/scheme-bytestructures"} +bytestructures_version=${GUILE_BYTESTRUCTURES_VERSION:-1.0.10} +guile_git_repo=${GUILE_GIT_REPO:-"https://gitlab.com/guile-git/guile-git.git"} +guile_git_version=${GUILE_GIT_VERSION:-0.10.0} +install_prefix=${INSTALL_PREFIX:-/tmp/guile-gnutls-freebsd-validate-install} +guile_bin=${GUILE_BIN:-/tmp/guile-freebsd-validate-install/bin/guile} +make_bin=${MAKE_BIN:-gmake} + +if [ ! -x "$guile_bin" ]; then + echo "Guile binary is not executable: $guile_bin" >&2 + exit 1 +fi +for tool in git autoreconf pkg-config "$make_bin"; do + if ! command -v "$tool" >/dev/null 2>&1; then + echo "Required tool not found: $tool" >&2 + exit 1 + fi +done + +cleanup=0 +if [ -n "${WORKDIR:-}" ]; then + workdir=$WORKDIR + mkdir -p "$workdir" +else + workdir=$(mktemp -d /tmp/fruix-guile-git.XXXXXX) + cleanup=1 +fi + +if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then + cleanup=0 +fi + +cleanup_workdir() { + if [ "$cleanup" -eq 1 ]; then + rm -rf "$workdir" + fi +} +trap cleanup_workdir EXIT INT TERM + +guile_bindir=$(CDPATH= cd -- "$(dirname "$guile_bin")" && pwd) +guile_prefix=$(CDPATH= cd -- "$guile_bindir/.." && pwd) +guile_lib_dir=$guile_prefix/lib +guile_version=$(LD_LIBRARY_PATH="$guile_lib_dir${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" "$guile_bin" -c '(display (effective-version))') +site_dir=$install_prefix/share/guile/site/$guile_version +site_ccache_dir=$install_prefix/lib/guile/$guile_version/site-ccache +extensions_dir=$install_prefix/lib/guile/$guile_version/extensions + +tool_bindir=$workdir/guile-tools-bin +mkdir -p "$tool_bindir" +ln -sf "$guile_bin" "$tool_bindir/guile-3.0" +ln -sf "$guile_bin" "$tool_bindir/guile" +ln -sf "$guile_bindir/guild" "$tool_bindir/guild-3.0" +ln -sf "$guile_bindir/guild" "$tool_bindir/guild" +ln -sf "$guile_bindir/guile-config" "$tool_bindir/guile-config-3.0" +ln -sf "$guile_bindir/guile-config" "$tool_bindir/guile-config" +ln -sf "$guile_bindir/guile-snarf" "$tool_bindir/guile-snarf" +export PATH="$tool_bindir:$guile_bindir:/usr/local/bin:$PATH" +export ACLOCAL_PATH=/usr/local/share/aclocal${ACLOCAL_PATH:+:$ACLOCAL_PATH} +export PKG_CONFIG_PATH=/usr/local/libdata/pkgconfig:/usr/local/lib/pkgconfig${PKG_CONFIG_PATH:+:$PKG_CONFIG_PATH} +export CPPFLAGS='-I/usr/local/include' +export LDFLAGS="-L/usr/local/lib -Wl,-rpath,/usr/local/lib -Wl,-rpath,$guile_lib_dir -Wl,-rpath,$install_prefix/lib" +if [ -n "${LD_LIBRARY_PATH:-}" ]; then + export LD_LIBRARY_PATH="$install_prefix/lib:$guile_lib_dir:/usr/local/lib:$LD_LIBRARY_PATH" +else + export LD_LIBRARY_PATH="$install_prefix/lib:$guile_lib_dir:/usr/local/lib" +fi +if [ -d "$site_dir" ]; then + export GUILE_LOAD_PATH="$site_dir${GUILE_LOAD_PATH:+:$GUILE_LOAD_PATH}" +fi +if [ -d "$site_ccache_dir" ]; then + export GUILE_LOAD_COMPILED_PATH="$site_ccache_dir${GUILE_LOAD_COMPILED_PATH:+:$GUILE_LOAD_COMPILED_PATH}" +fi +if [ -d "$extensions_dir" ]; then + export GUILE_EXTENSIONS_PATH="$extensions_dir${GUILE_EXTENSIONS_PATH:+:$GUILE_EXTENSIONS_PATH}" +fi + +bytes_src_dir=$workdir/bytestructures +git_src_dir=$workdir/guile-git +bytes_build_dir=$workdir/build-bytestructures +git_build_dir=$workdir/build-guile-git +metadata_file=$workdir/guile-git-build-metadata.txt +env_file=$workdir/guile-git-env.sh +bytes_autoreconf_log=$workdir/bytestructures-autoreconf.log +bytes_configure_log=$workdir/bytestructures-configure.log +bytes_build_log=$workdir/bytestructures-build.log +bytes_install_log=$workdir/bytestructures-install.log +git_bootstrap_log=$workdir/guile-git-bootstrap.log +git_configure_log=$workdir/guile-git-configure.log +git_build_log=$workdir/guile-git-build.log +git_install_log=$workdir/guile-git-install.log + +printf 'Using Guile: %s\n' "$guile_bin" +printf 'Installing into existing prefix: %s\n' "$install_prefix" +printf 'Working directory: %s\n' "$workdir" + +rm -rf \ + "$site_dir/bytestructures" \ + "$site_dir/git" \ + "$site_dir/git.scm" \ + "$site_ccache_dir/bytestructures" \ + "$site_ccache_dir/git" \ + "$site_ccache_dir/git.go" +mkdir -p "$install_prefix" + +git clone --depth 1 --branch "$bytestructures_version" "$bytestructures_repo" "$bytes_src_dir" >"$workdir/bytestructures-clone.log" 2>&1 +bytestructures_commit=$(git -C "$bytes_src_dir" rev-parse HEAD) +( + cd "$bytes_src_dir" + autoreconf -vfi +) >"$bytes_autoreconf_log" 2>&1 +mkdir -p "$bytes_build_dir" +( + cd "$bytes_build_dir" + "$bytes_src_dir/configure" --prefix="$install_prefix" +) >"$bytes_configure_log" 2>&1 +( + cd "$bytes_build_dir" + "$make_bin" GUILE_AUTO_COMPILE=0 -j"${JOBS:-$(sysctl -n hw.ncpu 2>/dev/null || echo 1)}" +) >"$bytes_build_log" 2>&1 +( + cd "$bytes_build_dir" + "$make_bin" GUILE_AUTO_COMPILE=0 install +) >"$bytes_install_log" 2>&1 + +export GUILE_LOAD_PATH="$site_dir${GUILE_LOAD_PATH:+:$GUILE_LOAD_PATH}" +export GUILE_LOAD_COMPILED_PATH="$site_ccache_dir${GUILE_LOAD_COMPILED_PATH:+:$GUILE_LOAD_COMPILED_PATH}" +bytestructures_check=$("$guile_bin" -c '(use-modules (bytestructures guile)) (display "ok") (newline)') +if [ "$bytestructures_check" != "ok" ]; then + echo "(bytestructures guile) module validation failed" >&2 + exit 1 +fi + +git clone --depth 1 --branch "v$guile_git_version" "$guile_git_repo" "$git_src_dir" >"$workdir/guile-git-clone.log" 2>&1 +guile_git_commit=$(git -C "$git_src_dir" rev-parse HEAD) +( + cd "$git_src_dir" + if [ -x ./autogen.sh ]; then + ./autogen.sh + else + autoreconf -vfi + fi +) >"$git_bootstrap_log" 2>&1 +mkdir -p "$git_build_dir" +( + cd "$git_build_dir" + "$git_src_dir/configure" --prefix="$install_prefix" +) >"$git_configure_log" 2>&1 +( + cd "$git_build_dir" + "$make_bin" GUILE_AUTO_COMPILE=0 -j"${JOBS:-$(sysctl -n hw.ncpu 2>/dev/null || echo 1)}" +) >"$git_build_log" 2>&1 +( + cd "$git_build_dir" + "$make_bin" GUILE_AUTO_COMPILE=0 install +) >"$git_install_log" 2>&1 + +guile_git_check=$("$guile_bin" -c '(use-modules (git)) (display (if (procedure? graph-descendant?) "ok" "missing-export")) (newline)') +if [ "$guile_git_check" != "ok" ]; then + echo "(git) module validation failed" >&2 + exit 1 +fi + +gnutls_check=$("$guile_bin" -c '(catch #t (lambda () (use-modules (gnutls)) (display "present") (newline)) (lambda _ (display "missing") (newline)))') +libgit2_version=$(pkg-config --modversion libgit2) + +cat >"$env_file" <"$metadata_file" <"$git_check_out" 2>"$git_check_err" +check_rc=$? +set -e +if [ "$check_rc" -eq 0 ] && [ "$(tr -d '\n' < "$git_check_out")" = ok ]; then + local_git_check=present +fi + if [ "$unsupported_rc" -eq 0 ]; then echo "configure without --with-courage unexpectedly succeeded" >&2 exit 1 @@ -152,6 +164,12 @@ else exit 1 fi fi +if [ "$local_gnutls_check" = present ] && [ "$local_git_check" = missing ]; then + if ! grep -q "Guile-Git is missing" "$courage_log"; then + echo "configure with --with-courage failed, but not for the expected missing-Guile-Git reason" >&2 + exit 1 + fi +fi cat >"$metadata_file" <} local_guile_gnutls_module=$local_gnutls_check +local_guile_git_module=$local_git_check unsupported_summary_begin $unsupported_summary unsupported_summary_end @@ -186,8 +206,10 @@ printf 'Store dir under test: %s\n' "$store_dir" printf 'Finding 1: configure without --with-courage is blocked by unsupported platform gating.\n' if [ "$local_gnutls_check" = missing ]; then printf 'Finding 2: configure with --with-courage is currently blocked by missing Guile (gnutls) bindings.\n' +elif [ "$local_git_check" = missing ]; then + printf 'Finding 2: Guile (gnutls) is available, but configure is now blocked by missing Guile-Git.\n' else - printf 'Finding 2: Guile (gnutls) bindings are available; configure now advances to the next blocker.\n' + printf 'Finding 2: Guile (gnutls) and Guile-Git are available; next blocker: %s\n' "${courage_error:-unknown configure failure}" fi if [ "$cleanup" -eq 0 ]; then printf 'Metadata file: %s\n' "$metadata_file"