self/fruix
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Build local Guile-JSON on FreeBSD

Browse Source
This commit is contained in:
Steffen
2026-04-01 10:48:54 +02:00
parent 15b9037034
commit 47d31e8992
4 changed files with 467 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

66
docs/PROGRESS.md
View File

@@ -571,3 +571,69 @@ Next recommended step:
1. obtain `Guile-JSON` compatible with the fixed local Guile build and install it into the same local dependency prefix 1. obtain `Guile-JSON` compatible with the fixed local Guile build and install it into the same local dependency prefix
2. re-run the derivation-generation investigation again to identify the next configure-time or checkout-time blocker after `Guile-JSON` 2. re-run the derivation-generation investigation again to identify the next configure-time or checkout-time blocker after `Guile-JSON`
3. continue keeping `/frx/store` as the intended experimental store root and keep `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear 3. continue keeping `/frx/store` as the intended experimental store root and keep `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear
## 2026-04-01 — Phase 1.2 follow-up: local Guile-JSON built on FreeBSD; next blocker is Guile-SQLite3
Completed work:
- added a reproducible local Guile-JSON build harness:
- `tests/guix/build-local-guile-json.sh`
- updated the derivation-generation investigation harness to probe and record local recent-enough `(json)` availability in addition to the earlier `(gnutls)` and `(git)` checks
- used the current Guix package definition in `~/repos/guix/gnu/packages/guile.scm` as the source of truth for:
- `guile-json` version `4.7.3`
- expected Guix nix-base32 source hash `127k2xc07w1gnyqs40z4865l8p3ra5xgpcn569dz04lxsa709fiq`
- verified the downloaded Guile-JSON tarball against the translated SHA256:
- `38ba048ed29d12f05b32c5b2fb7a51795c448b41e403a2b1b72ff0035817f388`
- built and installed a local Guile-JSON validation copy into the same local dependency prefix already used for checkout prerequisites:
- `/tmp/guile-gnutls-freebsd-validate-install`
- validated successfully that the fixed local Guile now satisfies the Guix configure-time JSON requirement by:
- loading `(json)`
- using `define-json-mapping`
- decoding a small JSON object successfully
- re-ran the checkout derivation-generation investigation with:
- `GUILE_EXTRA_PREFIX=/tmp/guile-gnutls-freebsd-validate-install`
- store directory still set to `/frx/store`
- wrote the results to:
- `docs/reports/phase1-guile-json-freebsd.md`
Important findings:
- unlike the earlier Guile-Git step, Guile-JSON built cleanly from the release tarball and did not require autotools regeneration in this validation pass
- unlike the earlier Guile-GnuTLS step, no FreeBSD-specific source patch was needed here
- the shared local validation prefix now contains at least:
- Guile-GnuTLS
- Guile bytestructures
- Guile-Git
- Guile-JSON
- the Guix checkout now gets past the previously cleared dependency gates for:
- `(gnutls)`
- `Guile-Git`
- `Guile-JSON`
- after clearing those, the next configure-time blocker is now:
- `configure: error: A recent Guile-SQLite3 could not be found; please install it.`
Current assessment:
- the checkout-preparation path on FreeBSD has progressed another dependency layer deeper
- the project still has not yet reached derivation emission, daemon connectivity, or actual `/frx/store` population
- the next concrete blocker is now recent `Guile-SQLite3`, as reported directly by the real Guix checkout configure step
Recent commits:
- `e380e88` — `Add FreeBSD Guile verification harness`
- `cd721b1` — `Update progress after Guile verification`
- `27916cb` — `Diagnose Guile subprocess crash on FreeBSD`
- `02f7a7f` — `Validate local Guile fix on FreeBSD`
- `4aebea4` — `Add native GNU Hello FreeBSD build harness`
- `c944cdb` — `Validate Guix builder phases on FreeBSD`
- `0a2e48e` — `Validate GNU which builder phases on FreeBSD`
- `245a47d` — `Document gaps to real Guix FreeBSD builds`
- `d62e9b0` — `Investigate Guix derivation generation on FreeBSD`
- `c0a85ed` — `Build local Guile-GnuTLS on FreeBSD`
- `15b9037` — `Build local Guile-Git on FreeBSD`
Next recommended step:
1. obtain recent `Guile-SQLite3` compatible with the fixed local Guile build and install it into the same local dependency prefix
2. re-run the derivation-generation investigation again to identify the next configure-time or checkout-time blocker after `Guile-SQLite3`
3. continue keeping `/frx/store` as the intended experimental store root and keep `~/repos/bdwgc` in reserve if later FreeBSD-specific GC/thread issues appear

181
docs/reports/phase1-guile-json-freebsd.md Normal file
View File

@@ -0,0 +1,181 @@
# Phase 1.2 follow-up: local Guile-JSON built on FreeBSD and checkout investigation advanced again
Date: 2026-04-01
## Summary
This step addressed the next Guix checkout blocker found after adding local Guile-GnuTLS and Guile-Git support on FreeBSD.
Added/updated files:
- `tests/guix/build-local-guile-json.sh`
- updated `tests/guix/run-derivation-generation-investigation.sh`
The result is:
1. `guile-json` can be built successfully on FreeBSD against the previously validated fixed local Guile build
2. it can be installed into the same local dependency prefix already used for the earlier checkout prerequisite work
3. after making `(json)` available, the Guix checkout configure step advances again
4. the next checkout blocker is now `Guile-SQLite3`
## Inputs used
### Local Guile
The same fixed local Guile build was used:
- `/tmp/guile-freebsd-validate-install/bin/guile`
### Existing local dependency prefix
This step reused the same local dependency prefix already populated by earlier prerequisite builds:
- `/tmp/guile-gnutls-freebsd-validate-install`
At this point that prefix contains at least:
- Guile-GnuTLS
- Guile bytestructures
- Guile-Git
- Guile-JSON
### Guix source-of-truth package definition
This step used the current Guix package definition in `~/repos/guix/gnu/packages/guile.scm`:
- `guile-json`
- version `4.7.3`
- Guix nix-base32 source hash:
- `127k2xc07w1gnyqs40z4865l8p3ra5xgpcn569dz04lxsa709fiq`
Translated and verified SHA256:
```text
38ba048ed29d12f05b32c5b2fb7a51795c448b41e403a2b1b72ff0035817f388
```
## Build findings
### 1. The release tarball builds cleanly on FreeBSD with the fixed local Guile
Unlike the earlier Guile-Git step, `guile-json` was built from the upstream release tarball rather than a Git checkout, so no autotools regeneration step was required in this validation pass.
### 2. No new FreeBSD-specific source patch was needed
Unlike the earlier Guile-GnuTLS step, this build did not require any FreeBSD-specific source patch.
### 3. The validation matched Guix's actual configure-time expectation
Guix does not merely check that `(json)` loads; `configure.ac` calls `GUIX_CHECK_GUILE_JSON`, which requires a recent-enough Guile-JSON providing `define-json-mapping`.
The local validation therefore used the same kind of test:
- load `(json)`
- define a small JSON mapping with `define-json-mapping`
- decode a JSON object and confirm the mapped record matches the expected value
Observed validation result:
```text
ok
```
## Validation command
```sh
ENV_OUT=/tmp/guile-json-env.sh \
METADATA_OUT=/tmp/guile-json-metadata.txt \
./tests/guix/build-local-guile-json.sh
```
Default install prefix used by the harness:
```text
/tmp/guile-gnutls-freebsd-validate-install
```
## Result
The local Guile-JSON build succeeded.
Observed checks:
```text
json module check: ok
Existing (gnutls) module in prefix: present
Existing (git) module in prefix: present
```
This means the same local prefix now satisfies at least the following mandatory Guix checkout module requirements previously encountered on FreeBSD:
- `(gnutls)`
- `(git)` with recent Guile-Git export support
- `(json)` with recent-enough `define-json-mapping` support
## Re-running the checkout investigation
After installing Guile-JSON, the derivation-generation investigation was re-run with:
```sh
GUILE_EXTRA_PREFIX=/tmp/guile-gnutls-freebsd-validate-install \
METADATA_OUT=/tmp/guix-derivation-investigation-with-json.txt \
./tests/guix/run-derivation-generation-investigation.sh
```
The investigation continued to use the requested experimental directories:
- store: `/frx/store`
- local state: `/frx/var`
- sysconf: `/frx/etc`
## Updated result
With `(gnutls)`, `(git)`, and `(json)` available, the Guix checkout configure step progresses farther and now fails at the next missing Guile dependency:
```text
configure: error: A recent Guile-SQLite3 could not be found; please install it.
```
Relevant configure summary lines observed:
```text
checking whether Guile-JSON is available and recent enough... yes
configure: error: A recent Guile-SQLite3 could not be found; please install it.
```
## What this step demonstrates
This step demonstrates that:
1. the earlier `Guile-JSON` blocker is genuinely cleared on FreeBSD when using the fixed local Guile build plus the local dependency prefix
2. the checkout can move one more dependency layer deeper than before
3. the project is still in the checkout-prerequisite stage rather than the derivation/store/daemon stage
4. the next concrete prerequisite for reaching deeper Guix checkout usability on FreeBSD is `Guile-SQLite3`
## Current blocker stack
The current checkout path is now blocked by:
1. unsupported-platform configure gating unless `--with-courage` is used
2. missing recent `Guile-SQLite3` after supplying:
- Guile-GnuTLS
- Guile-Git
- Guile-JSON
The earlier missing blockers have now been cleared locally for validation:
- `(gnutls)`
- `(git)` / recent Guile-Git export requirement
- recent `(json)` support
## Recommended next step
Obtain or build `Guile-SQLite3` compatible with the same fixed local Guile build, install it into the same local dependency prefix, and re-run the derivation-generation investigation again.
That should show whether the next boundary lies at:
- `Guile-Gcrypt`
- `Guile-zlib`
- `Guile-lzlib`
- `Guile-semver`
- or finally something beyond configure-time Guile module prerequisites

194
tests/guix/build-local-guile-json.sh Executable file
View File

@@ -0,0 +1,194 @@
#!/bin/sh
set -eu
source_url=${GUILE_JSON_SOURCE_URL:-"https://download.savannah.gnu.org/releases/guile-json/guile-json-4.7.3.tar.gz"}
version=${GUILE_JSON_VERSION:-4.7.3}
expected_nix_base32=${GUILE_JSON_NIX_BASE32:-127k2xc07w1gnyqs40z4865l8p3ra5xgpcn569dz04lxsa709fiq}
install_prefix=${INSTALL_PREFIX:-/tmp/guile-gnutls-freebsd-validate-install}
guile_bin=${GUILE_BIN:-/tmp/guile-freebsd-validate-install/bin/guile}
guix_source_dir=${GUIX_SOURCE_DIR:-"$HOME/repos/guix"}
make_bin=${MAKE_BIN:-gmake}
if [ ! -x "$guile_bin" ]; then
echo "Guile binary is not executable: $guile_bin" >&2
exit 1
fi
if [ ! -d "$guix_source_dir/guix" ]; then
echo "Guix source tree not found at $guix_source_dir" >&2
exit 1
fi
for tool in fetch sha256 bsdtar "$make_bin"; do
if ! command -v "$tool" >/dev/null 2>&1; then
echo "Required tool not found: $tool" >&2
exit 1
fi
done
cleanup=0
if [ -n "${WORKDIR:-}" ]; then
workdir=$WORKDIR
mkdir -p "$workdir"
else
workdir=$(mktemp -d /tmp/fruix-guile-json.XXXXXX)
cleanup=1
fi
if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
cleanup=0
fi
cleanup_workdir() {
if [ "$cleanup" -eq 1 ]; then
rm -rf "$workdir"
fi
}
trap cleanup_workdir EXIT INT TERM
guile_bindir=$(CDPATH= cd -- "$(dirname "$guile_bin")" && pwd)
guile_prefix=$(CDPATH= cd -- "$guile_bindir/.." && pwd)
guile_lib_dir=$guile_prefix/lib
guile_version=$(LD_LIBRARY_PATH="$guile_lib_dir${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" "$guile_bin" -c '(display (effective-version))')
site_dir=$install_prefix/share/guile/site/$guile_version
site_ccache_dir=$install_prefix/lib/guile/$guile_version/site-ccache
extensions_dir=$install_prefix/lib/guile/$guile_version/extensions
tool_bindir=$workdir/guile-tools-bin
mkdir -p "$tool_bindir"
ln -sf "$guile_bin" "$tool_bindir/guile-3.0"
ln -sf "$guile_bin" "$tool_bindir/guile"
ln -sf "$guile_bindir/guild" "$tool_bindir/guild-3.0"
ln -sf "$guile_bindir/guild" "$tool_bindir/guild"
ln -sf "$guile_bindir/guile-config" "$tool_bindir/guile-config-3.0"
ln -sf "$guile_bindir/guile-config" "$tool_bindir/guile-config"
ln -sf "$guile_bindir/guile-snarf" "$tool_bindir/guile-snarf"
export PATH="$tool_bindir:$guile_bindir:/usr/local/bin:$PATH"
export ACLOCAL_PATH=/usr/local/share/aclocal${ACLOCAL_PATH:+:$ACLOCAL_PATH}
export PKG_CONFIG_PATH=/usr/local/libdata/pkgconfig:/usr/local/lib/pkgconfig${PKG_CONFIG_PATH:+:$PKG_CONFIG_PATH}
export CPPFLAGS='-I/usr/local/include'
export LDFLAGS="-L/usr/local/lib -Wl,-rpath,/usr/local/lib -Wl,-rpath,$guile_lib_dir -Wl,-rpath,$install_prefix/lib"
if [ -n "${LD_LIBRARY_PATH:-}" ]; then
export LD_LIBRARY_PATH="$install_prefix/lib:$guile_lib_dir:/usr/local/lib:$LD_LIBRARY_PATH"
else
export LD_LIBRARY_PATH="$install_prefix/lib:$guile_lib_dir:/usr/local/lib"
fi
if [ -d "$site_dir" ]; then
export GUILE_LOAD_PATH="$site_dir${GUILE_LOAD_PATH:+:$GUILE_LOAD_PATH}"
fi
if [ -d "$site_ccache_dir" ]; then
export GUILE_LOAD_COMPILED_PATH="$site_ccache_dir${GUILE_LOAD_COMPILED_PATH:+:$GUILE_LOAD_COMPILED_PATH}"
fi
if [ -d "$extensions_dir" ]; then
export GUILE_EXTENSIONS_PATH="$extensions_dir${GUILE_EXTENSIONS_PATH:+:$GUILE_EXTENSIONS_PATH}"
fi
src_tarball=$workdir/guile-json-$version.tar.gz
src_dir=$workdir/guile-json-$version
build_dir=$workdir/build
metadata_file=$workdir/guile-json-build-metadata.txt
env_file=$workdir/guile-json-env.sh
configure_log=$workdir/configure.log
build_log=$workdir/build.log
install_log=$workdir/install.log
expected_sha256_hex=$(GUILE_AUTO_COMPILE=0 \
GUILE_LOAD_PATH="$guix_source_dir${GUILE_LOAD_PATH:+:$GUILE_LOAD_PATH}" \
"$guile_bin" -c "(use-modules (guix base32) (rnrs bytevectors) (ice-9 format)) (for-each (lambda (b) (format #t \"~2,'0x\" b)) (bytevector->u8-list (nix-base32-string->bytevector (cadr (command-line))))) (newline)" \
"$expected_nix_base32")
printf 'Using Guile: %s\n' "$guile_bin"
printf 'Installing into existing prefix: %s\n' "$install_prefix"
printf 'Working directory: %s\n' "$workdir"
printf 'Fetching: %s\n' "$source_url"
fetch -o "$src_tarball" "$source_url"
actual_sha256_hex=$(sha256 -q "$src_tarball")
if [ "$actual_sha256_hex" != "$expected_sha256_hex" ]; then
echo "sha256 mismatch for $src_tarball" >&2
echo "expected: $expected_sha256_hex" >&2
echo "actual: $actual_sha256_hex" >&2
exit 1
fi
bsdtar -xf "$src_tarball" -C "$workdir"
if [ ! -d "$src_dir" ]; then
echo "expected extracted source directory not found: $src_dir" >&2
exit 1
fi
rm -rf "$site_dir/json" "$site_dir/json.scm" "$site_ccache_dir/json" "$site_ccache_dir/json.go"
mkdir -p "$build_dir"
(
cd "$build_dir"
"$src_dir/configure" --prefix="$install_prefix"
) >"$configure_log" 2>&1
(
cd "$build_dir"
"$make_bin" GUILE_AUTO_COMPILE=0 -j"${JOBS:-$(sysctl -n hw.ncpu 2>/dev/null || echo 1)}"
) >"$build_log" 2>&1
(
cd "$build_dir"
"$make_bin" GUILE_AUTO_COMPILE=0 install
) >"$install_log" 2>&1
export GUILE_LOAD_PATH="$site_dir${GUILE_LOAD_PATH:+:$GUILE_LOAD_PATH}"
export GUILE_LOAD_COMPILED_PATH="$site_ccache_dir${GUILE_LOAD_COMPILED_PATH:+:$GUILE_LOAD_COMPILED_PATH}"
json_check=$("$guile_bin" -c '(use-modules (json)) (define-json-mapping <frob> make-frob frob? json->frob (a frob-a) (b frob-b "bee")) (display (if (equal? (json->frob (open-input-string "{ \"a\": 1, \"bee\": 2 }")) (make-frob 1 2)) "ok" "mismatch")) (newline)')
if [ "$json_check" != "ok" ]; then
echo "(json) module validation failed" >&2
exit 1
fi
gnutls_check=$("$guile_bin" -c '(catch #t (lambda () (use-modules (gnutls)) (display "present") (newline)) (lambda _ (display "missing") (newline)))')
git_check=$("$guile_bin" -c '(catch #t (lambda () (use-modules (git)) (display (if (procedure? graph-descendant?) "present" "missing-export")) (newline)) (lambda _ (display "missing") (newline)))')
cat >"$env_file" <<EOF
export GUILE_EXTRA_PREFIX='$install_prefix'
export GUILE_LOAD_PATH='$site_dir'
export GUILE_LOAD_COMPILED_PATH='$site_ccache_dir'
export GUILE_EXTENSIONS_PATH='$extensions_dir'
export LD_LIBRARY_PATH='$install_prefix/lib:$guile_lib_dir:/usr/local/lib'
EOF
cat >"$metadata_file" <<EOF
version=$version
source_url=$source_url
expected_nix_base32=$expected_nix_base32
expected_sha256_hex=$expected_sha256_hex
actual_sha256_hex=$actual_sha256_hex
guile_bin=$guile_bin
guile_version=$guile_version
install_prefix=$install_prefix
site_dir=$site_dir
site_ccache_dir=$site_ccache_dir
extensions_dir=$extensions_dir
configure_log=$configure_log
build_log=$build_log
install_log=$install_log
json_module_check=$json_check
gnutls_module_already_present=$gnutls_check
guile_git_module_already_present=$git_check
env_file=$env_file
EOF
if [ -n "${METADATA_OUT:-}" ]; then
mkdir -p "$(dirname "$METADATA_OUT")"
cp "$metadata_file" "$METADATA_OUT"
fi
if [ -n "${ENV_OUT:-}" ]; then
mkdir -p "$(dirname "$ENV_OUT")"
cp "$env_file" "$ENV_OUT"
fi
printf 'PASS local-guile-json-build\n'
printf 'json module check: %s\n' "$json_check"
printf 'Existing (gnutls) module in prefix: %s\n' "$gnutls_check"
printf 'Existing (git) module in prefix: %s\n' "$git_check"
printf 'Environment file: %s\n' "$env_file"
printf 'Metadata file: %s\n' "$metadata_file"
if [ -n "${ENV_OUT:-}" ]; then
printf 'Copied environment file to: %s\n' "$ENV_OUT"
fi
if [ -n "${METADATA_OUT:-}" ]; then
printf 'Copied metadata file to: %s\n' "$METADATA_OUT"
fi
printf '%s\n' '--- metadata ---'
cat "$metadata_file"

27
tests/guix/run-derivation-generation-investigation.sh
View File

@@ -141,6 +141,17 @@ if [ "$check_rc" -eq 0 ] && [ "$(tr -d '\n' < "$git_check_out")" = ok ]; then
local_git_check=present local_git_check=present
fi fi
json_check_out=$workdir/guile-json-check.out
json_check_err=$workdir/guile-json-check.err
local_json_check=missing
set +e
"$guile_bin" -c '(use-modules (json)) (define-json-mapping <frob> make-frob frob? json->frob (a frob-a) (b frob-b "bee")) (display (if (equal? (json->frob (open-input-string "{ \"a\": 1, \"bee\": 2 }")) (make-frob 1 2)) "ok" "mismatch")) (newline)' >"$json_check_out" 2>"$json_check_err"
check_rc=$?
set -e
if [ "$check_rc" -eq 0 ] && [ "$(tr -d '\n' < "$json_check_out")" = ok ]; then
local_json_check=present
fi
if [ "$unsupported_rc" -eq 0 ]; then if [ "$unsupported_rc" -eq 0 ]; then
echo "configure without --with-courage unexpectedly succeeded" >&2 echo "configure without --with-courage unexpectedly succeeded" >&2
exit 1 exit 1
@@ -170,6 +181,17 @@ if [ "$local_gnutls_check" = present ] && [ "$local_git_check" = missing ]; then
exit 1 exit 1
fi fi
fi fi
if [ "$local_gnutls_check" = present ] && [ "$local_git_check" = present ] && [ "$local_json_check" = missing ]; then
if ! grep -q "Guile-JSON is missing" "$courage_log"; then
echo "configure with --with-courage failed, but not for the expected missing-Guile-JSON reason" >&2
exit 1
fi
elif [ "$local_json_check" = present ]; then
if grep -q "Guile-JSON is missing" "$courage_log"; then
echo "configure with --with-courage still reports missing Guile-JSON even though (json) works" >&2
exit 1
fi
fi
cat >"$metadata_file" <<EOF cat >"$metadata_file" <<EOF
source_repo=$source_repo source_repo=$source_repo
@@ -188,6 +210,7 @@ configure_with_courage_rc=$courage_rc
configure_with_courage_error=${courage_error:-<none>} configure_with_courage_error=${courage_error:-<none>}
local_guile_gnutls_module=$local_gnutls_check local_guile_gnutls_module=$local_gnutls_check
local_guile_git_module=$local_git_check local_guile_git_module=$local_git_check
local_guile_json_module=$local_json_check
unsupported_summary_begin unsupported_summary_begin
$unsupported_summary $unsupported_summary
unsupported_summary_end unsupported_summary_end
@@ -208,8 +231,10 @@ if [ "$local_gnutls_check" = missing ]; then
printf 'Finding 2: configure with --with-courage is currently blocked by missing Guile (gnutls) bindings.\n' printf 'Finding 2: configure with --with-courage is currently blocked by missing Guile (gnutls) bindings.\n'
elif [ "$local_git_check" = missing ]; then elif [ "$local_git_check" = missing ]; then
printf 'Finding 2: Guile (gnutls) is available, but configure is now blocked by missing Guile-Git.\n' printf 'Finding 2: Guile (gnutls) is available, but configure is now blocked by missing Guile-Git.\n'
elif [ "$local_json_check" = missing ]; then
printf 'Finding 2: Guile (gnutls) and Guile-Git are available, but configure is now blocked by missing Guile-JSON.\n'
else else
printf 'Finding 2: Guile (gnutls) and Guile-Git are available; next blocker: %s\n' "${courage_error:-unknown configure failure}" printf 'Finding 2: Guile (gnutls), Guile-Git, and Guile-JSON are available; next blocker: %s\n' "${courage_error:-unknown configure failure}"
fi fi
if [ "$cleanup" -eq 0 ]; then if [ "$cleanup" -eq 0 ]; then
printf 'Metadata file: %s\n' "$metadata_file" printf 'Metadata file: %s\n' "$metadata_file"