Generate FreeBSD system closures in /frx/store

2026-04-01 18:42:33 +02:00
parent 13963e7f62
commit e4288dc330
6 changed files with 314 additions and 2 deletions
--- a/docs/PROGRESS.md
+++ b/docs/PROGRESS.md
@@ -1906,7 +1906,7 @@ Important findings:
  - `host_name=fruix-freebsd`
  - `kernel_package=freebsd-kernel`
  - `bootloader_package=freebsd-bootloader`
-  - `base_packages=freebsd-runtime,freebsd-userland,freebsd-libc,freebsd-rc-scripts,freebsd-bash`
+  - `base_packages=freebsd-runtime,freebsd-userland,freebsd-libc,freebsd-rc-scripts,freebsd-sh,freebsd-bash`
  - `users=root,operator`
  - `groups=wheel,operator`
  - `generated_files=boot/loader.conf,etc/rc.conf,etc/fstab,etc/hosts,etc/passwd,etc/group,etc/shells,etc/motd,activate,shepherd/init.scm`
@@ -1916,3 +1916,51 @@ Current assessment:

 - Phase 7.1 is now satisfied on the current FreeBSD prototype track
 - the next step is to materialize this operating-system description into a reproducible system closure under `/frx/store`
+
+## 2026-04-01 — Phase 7.2 completed: minimal system closure generated under `/frx/store`
+
+Completed work:
+
+- added the Phase 7.2 closure materialization harnesses:
+  - `tests/system/materialize-phase7-system-closure.scm`
+  - `tests/system/run-phase7-system-closure.sh`
+- refined the minimal operating-system example so the generated system profile also contains `/bin/sh`
+- wrote the Phase 7.2 report:
+  - `docs/reports/phase7-system-closure-freebsd.md`
+- ran the system-closure harness successfully and captured metadata under:
+  - `/tmp/phase7-system-closure-metadata.txt`
+
+Important findings:
+
+- the declarative FreeBSD Fruix operating-system object now materializes into a real system closure under `/frx/store`
+- that closure contains:
+  - boot assets
+  - a merged system profile
+  - generated `/etc` files
+  - a generated activation script
+  - a generated Shepherd configuration
+  - a generated `rc.d` launcher for Shepherd
+- the closure now embeds the concrete first init integration choice for the FreeBSD track:
+  - `freebsd-init+rc.d-shepherd`
+- rerunning the same materialization produced the same closure path, which is the current prototype proof of reproducible closure generation for this phase
+- observed metadata confirmed:
+  - `closure_path=/frx/store/...-fruix-system-fruix-freebsd`
+  - `closure_rebuild_path=/frx/store/...-fruix-system-fruix-freebsd`
+  - `kernel_store=/frx/store/...-freebsd-kernel-15.0-STABLE`
+  - `bootloader_store=/frx/store/...-freebsd-bootloader-15.0-STABLE`
+  - `guile_store=/frx/store/...-fruix-guile-runtime-3.0`
+  - `guile_extra_store=/frx/store/...-fruix-guile-extra-3.0`
+  - `shepherd_store=/frx/store/...-fruix-shepherd-runtime-1.0.9`
+  - `profile_bin_sh=/frx/store/...-fruix-system-fruix-freebsd/profile/bin/sh`
+  - `profile_sbin_init=/frx/store/...-fruix-system-fruix-freebsd/profile/sbin/init`
+  - `profile_rc=/frx/store/...-fruix-system-fruix-freebsd/profile/etc/rc`
+
+Current assessment:
+
+- Phase 7.2 is now satisfied on the current FreeBSD prototype track
+- the next step is to materialize and statically validate an installable root filesystem tree from this system closure
+
+Current assessment:
+
+- Phase 7.1 is now satisfied on the current FreeBSD prototype track
+- the next step is to materialize this operating-system description into a reproducible system closure under `/frx/store`
--- a/docs/reports/phase7-operating-system-model-freebsd.md
+++ b/docs/reports/phase7-operating-system-model-freebsd.md
@@ -56,7 +56,7 @@ Observed metadata included:
 - `host_name=fruix-freebsd`
 - `kernel_package=freebsd-kernel`
 - `bootloader_package=freebsd-bootloader`
- `base_packages=freebsd-runtime,freebsd-userland,freebsd-libc,freebsd-rc-scripts,freebsd-bash`
+- `base_packages=freebsd-runtime,freebsd-userland,freebsd-libc,freebsd-rc-scripts,freebsd-sh,freebsd-bash`
 - `users=root,operator`
 - `groups=wheel,operator`
 - `file_system_count=3`
--- a/docs/reports/phase7-system-closure-freebsd.md
+++ b/docs/reports/phase7-system-closure-freebsd.md
@@ -0,0 +1,74 @@
+# Phase 7.2: Minimal FreeBSD system closure generated from the Fruix system model
+
+Date: 2026-04-01
+
+## Summary
+
+This step materializes the Phase 7.1 Fruix operating-system description into a reproducible system closure under `/frx/store`.
+
+Added files:
+
+- `tests/system/materialize-phase7-system-closure.scm`
+- `tests/system/run-phase7-system-closure.sh`
+
+## Validation command
+
+Run command:
+
+```sh
+METADATA_OUT=/tmp/phase7-system-closure-metadata.txt \
+./tests/system/run-phase7-system-closure.sh
+```
+
+## What the harness does
+
+The harness:
+
+1. ensures the local Guile/Fibers/Shepherd runtime needed for the generated system is available
+2. loads the declarative Phase 7 operating-system object
+3. materializes the referenced FreeBSD base packages into `/frx/store`
+4. materializes local Guile and Shepherd prefixes into `/frx/store`
+5. generates a Fruix system closure store item containing:
+   - boot assets
+   - a merged system profile
+   - generated `/etc` files
+   - an activation script
+   - a Shepherd configuration
+   - a generated FreeBSD `rc.d` launcher for Shepherd
+6. reruns the same materialization a second time and verifies that the same closure path is produced
+
+## Observed results
+
+Observed metadata included:
+
+- `closure_path=/frx/store/...-fruix-system-fruix-freebsd`
+- `closure_rebuild_path=/frx/store/...-fruix-system-fruix-freebsd`
+- `kernel_store=/frx/store/...-freebsd-kernel-15.0-STABLE`
+- `bootloader_store=/frx/store/...-freebsd-bootloader-15.0-STABLE`
+- `guile_store=/frx/store/...-fruix-guile-runtime-3.0`
+- `guile_extra_store=/frx/store/...-fruix-guile-extra-3.0`
+- `shepherd_store=/frx/store/...-fruix-shepherd-runtime-1.0.9`
+- `profile_bin_sh=/frx/store/...-fruix-system-fruix-freebsd/profile/bin/sh`
+- `profile_sbin_init=/frx/store/...-fruix-system-fruix-freebsd/profile/sbin/init`
+- `profile_rc=/frx/store/...-fruix-system-fruix-freebsd/profile/etc/rc`
+- `init_integration=freebsd-init+rc.d-shepherd`
+
+## Important findings
+
+- the FreeBSD system definition now materializes into a single reproducible system closure store item rather than remaining only a logical specification
+- the closure uses a Guix-like split between:
+  - referenced store items for package/runtime content
+  - generated configuration and activation payload in the system closure itself
+- the first integrated init strategy is now concretely encoded in the closure contents:
+  - FreeBSD init and rc infrastructure from the base system profile
+  - a generated `fruix_shepherd` rc script
+  - a generated Shepherd configuration that writes a deterministic ready marker
+- rerunning the same materialization produced the same closure path, which is the current prototype proof of reproducible closure generation for this phase
+
+## Conclusion
+
+Phase 7.2 is satisfied on the current FreeBSD prototype track:
+
+- a declarative Fruix system description now generates a real system closure under `/frx/store`
+- that closure contains boot files, `/etc` payloads, activation payload, and Shepherd launch integration
+- the next step is to materialize a root filesystem tree from this closure and statically validate it for later image construction work
--- a/tests/system/materialize-phase7-system-closure.scm
+++ b/tests/system/materialize-phase7-system-closure.scm
@@ -0,0 +1,102 @@
+(use-modules (fruix system freebsd)
+             (ice-9 format)
+             (srfi srfi-13)
+             (rnrs io ports))
+
+(define workdir
+  (or (getenv "WORKDIR")
+      (error "WORKDIR environment variable is required")))
+(define os-file
+  (or (getenv "OS_FILE")
+      (error "OS_FILE environment variable is required")))
+(define store-dir
+  (or (getenv "STORE_DIR")
+      "/frx/store"))
+(define guile-prefix
+  (or (getenv "GUILE_PREFIX")
+      "/tmp/guile-freebsd-validate-install"))
+(define guile-extra-prefix
+  (or (getenv "GUILE_EXTRA_PREFIX")
+      "/tmp/guile-gnutls-freebsd-validate-install"))
+(define shepherd-prefix
+  (or (getenv "SHEPHERD_PREFIX")
+      "/tmp/shepherd-freebsd-validate-install"))
+(define metadata-file
+  (string-append workdir "/phase7-system-closure-metadata.txt"))
+
+(primitive-load os-file)
+(validate-operating-system phase7-operating-system)
+
+(define (assert-exists path)
+  (unless (file-exists? path)
+    (error "required path missing" path)))
+
+(let* ((closure-a (materialize-operating-system phase7-operating-system
+                                                #:store-dir store-dir
+                                                #:guile-prefix guile-prefix
+                                                #:guile-extra-prefix guile-extra-prefix
+                                                #:shepherd-prefix shepherd-prefix))
+       (closure-b (materialize-operating-system phase7-operating-system
+                                                #:store-dir store-dir
+                                                #:guile-prefix guile-prefix
+                                                #:guile-extra-prefix guile-extra-prefix
+                                                #:shepherd-prefix shepherd-prefix))
+       (closure-path (assoc-ref closure-a 'closure-path))
+       (closure-rebuild-path (assoc-ref closure-b 'closure-path))
+       (kernel-store (assoc-ref closure-a 'kernel-store))
+       (bootloader-store (assoc-ref closure-a 'bootloader-store))
+       (guile-store (assoc-ref closure-a 'guile-store))
+       (guile-extra-store (assoc-ref closure-a 'guile-extra-store))
+       (shepherd-store (assoc-ref closure-a 'shepherd-store))
+       (base-package-stores (assoc-ref closure-a 'base-package-stores))
+       (rc-script (string-append closure-path "/usr/local/etc/rc.d/fruix-shepherd"))
+       (shepherd-config (string-append closure-path "/shepherd/init.scm"))
+       (activation-script (string-append closure-path "/activate"))
+       (loader-conf (string-append closure-path "/boot/loader.conf"))
+       (profile-bin-sh (string-append closure-path "/profile/bin/sh"))
+       (profile-sbin-init (string-append closure-path "/profile/sbin/init"))
+       (profile-rc (string-append closure-path "/profile/etc/rc"))
+       (ready-marker (assoc-ref (operating-system-closure-spec phase7-operating-system)
+                                'ready-marker))
+       (rc-script-target (readlink (string-append closure-path "/boot/loader")))
+       (kernel-link (readlink (string-append closure-path "/boot/kernel/kernel"))))
+  (for-each assert-exists
+            (list closure-path kernel-store bootloader-store guile-store
+                  guile-extra-store shepherd-store rc-script shepherd-config
+                  activation-script loader-conf profile-bin-sh profile-sbin-init
+                  profile-rc (string-append closure-path "/parameters.scm")))
+  (unless (string=? closure-path closure-rebuild-path)
+    (error "closure path was not reproducible" closure-path closure-rebuild-path))
+  (call-with-output-file metadata-file
+    (lambda (port)
+      (format port "store_dir=~a~%" store-dir)
+      (format port "closure_path=~a~%" closure-path)
+      (format port "closure_rebuild_path=~a~%" closure-rebuild-path)
+      (format port "kernel_store=~a~%" kernel-store)
+      (format port "bootloader_store=~a~%" bootloader-store)
+      (format port "guile_store=~a~%" guile-store)
+      (format port "guile_extra_store=~a~%" guile-extra-store)
+      (format port "shepherd_store=~a~%" shepherd-store)
+      (format port "base_package_store_count=~a~%" (length base-package-stores))
+      (format port "base_package_stores=~a~%" (string-join base-package-stores ","))
+      (format port "rc_script=~a~%" rc-script)
+      (format port "shepherd_config=~a~%" shepherd-config)
+      (format port "activation_script=~a~%" activation-script)
+      (format port "loader_conf=~a~%" loader-conf)
+      (format port "boot_loader_target=~a~%" rc-script-target)
+      (format port "kernel_link_target=~a~%" kernel-link)
+      (format port "profile_bin_sh=~a~%" profile-bin-sh)
+      (format port "profile_sbin_init=~a~%" profile-sbin-init)
+      (format port "profile_rc=~a~%" profile-rc)
+      (format port "ready_marker=~a~%" ready-marker)
+      (format port "init_integration=freebsd-init+rc.d-shepherd~%")))
+
+  (when (getenv "METADATA_OUT")
+    (copy-file metadata-file (getenv "METADATA_OUT")))
+
+  (format #t "PASS phase7-system-closure~%")
+  (format #t "Metadata file: ~a~%" metadata-file)
+  (when (getenv "METADATA_OUT")
+    (format #t "Copied metadata to: ~a~%" (getenv "METADATA_OUT")))
+  (display "--- metadata ---\n")
+  (display (call-with-input-file metadata-file get-string-all)))
--- a/tests/system/phase7-minimal-operating-system.scm
+++ b/tests/system/phase7-minimal-operating-system.scm
@@ -10,6 +10,7 @@
                         freebsd-userland
                         freebsd-libc
                         freebsd-rc-scripts
+                         freebsd-sh
                         freebsd-bash)
   #:groups (list (user-group #:name "wheel" #:gid 0 #:system? #t)
                  (user-group #:name "operator" #:gid 1000 #:system? #f))
--- a/tests/system/run-phase7-system-closure.sh
+++ b/tests/system/run-phase7-system-closure.sh
@@ -0,0 +1,87 @@
+#!/bin/sh
+set -eu
+
+project_root=${PROJECT_ROOT:-$(pwd)}
+guix_source_dir=${GUIX_SOURCE_DIR:-"$HOME/repos/guix"}
+script_dir=$(CDPATH= cd -- "$(dirname "$0")" && pwd)
+runner_scm=$script_dir/materialize-phase7-system-closure.scm
+os_file=$script_dir/phase7-minimal-operating-system.scm
+guile_bin=${GUILE_BIN:-/tmp/guile-freebsd-validate-install/bin/guile}
+guile_extra_prefix=${GUILE_EXTRA_PREFIX:-/tmp/guile-gnutls-freebsd-validate-install}
+shepherd_prefix=${SHEPHERD_PREFIX:-/tmp/shepherd-freebsd-validate-install}
+store_dir=${STORE_DIR:-/frx/store}
+metadata_target=${METADATA_OUT:-}
+
+if [ ! -x "$guile_bin" ]; then
+  echo "Guile binary is not executable: $guile_bin" >&2
+  exit 1
+fi
+
+ensure_built() {
+  if [ ! -d "$guile_extra_prefix/share/guile/site" ] || \
+     ! GUILE_LOAD_PATH="$guile_extra_prefix/share/guile/site/3.0${GUILE_LOAD_PATH:+:$GUILE_LOAD_PATH}" \
+       GUILE_LOAD_COMPILED_PATH="$guile_extra_prefix/lib/guile/3.0/site-ccache${GUILE_LOAD_COMPILED_PATH:+:$GUILE_LOAD_COMPILED_PATH}" \
+       GUILE_EXTENSIONS_PATH="$guile_extra_prefix/lib/guile/3.0/extensions${GUILE_EXTENSIONS_PATH:+:$GUILE_EXTENSIONS_PATH}" \
+       LD_LIBRARY_PATH="$guile_extra_prefix/lib:/tmp/guile-freebsd-validate-install/lib:/usr/local/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" \
+       "$guile_bin" -c '(catch #t (lambda () (use-modules (fibers)) (display "ok") (newline)) (lambda _ (display "missing") (newline)))' | grep -qx ok; then
+    METADATA_OUT= ENV_OUT= "$project_root/tests/shepherd/build-local-guile-fibers.sh"
+  fi
+
+  if [ ! -x "$shepherd_prefix/bin/shepherd" ] || [ ! -x "$shepherd_prefix/bin/herd" ]; then
+    METADATA_OUT= ENV_OUT= GUILE_EXTRA_PREFIX="$guile_extra_prefix" "$project_root/tests/shepherd/build-local-shepherd.sh"
+  fi
+}
+
+ensure_built
+
+guile_prefix=$(CDPATH= cd -- "$(dirname "$guile_bin")/.." && pwd)
+guile_lib_dir=$guile_prefix/lib
+cleanup=0
+if [ -n "${WORKDIR:-}" ]; then
+  workdir=$WORKDIR
+  mkdir -p "$workdir"
+else
+  workdir=$(mktemp -d /tmp/fruix-phase7-system-closure.XXXXXX)
+  cleanup=1
+fi
+if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
+  cleanup=0
+fi
+
+cleanup_workdir() {
+  if [ "$cleanup" -eq 1 ]; then
+    rm -rf "$workdir"
+  fi
+}
+trap cleanup_workdir EXIT INT TERM
+
+export GUILE_AUTO_COMPILE=0
+export WORKDIR="$workdir"
+export OS_FILE="$os_file"
+export STORE_DIR="$store_dir"
+export GUILE_PREFIX="$guile_prefix"
+export GUILE_EXTRA_PREFIX="$guile_extra_prefix"
+export SHEPHERD_PREFIX="$shepherd_prefix"
+
+if [ -n "${GUILE_LOAD_PATH:-}" ]; then
+  gui_load_path="$project_root/modules:$guix_source_dir:$GUILE_LOAD_PATH"
+else
+  gui_load_path="$project_root/modules:$guix_source_dir"
+fi
+
+printf 'Using Guile: %s\n' "$guile_bin"
+printf 'Working directory: %s\n' "$workdir"
+printf 'Store directory: %s\n' "$store_dir"
+
+sudo env \
+  GUILE_AUTO_COMPILE=0 \
+  GUILE_LOAD_PATH="$gui_load_path" \
+  LD_LIBRARY_PATH="$guile_lib_dir${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" \
+  WORKDIR="$workdir" \
+  OS_FILE="$os_file" \
+  STORE_DIR="$store_dir" \
+  GUILE_PREFIX="$guile_prefix" \
+  GUILE_EXTRA_PREFIX="$guile_extra_prefix" \
+  SHEPHERD_PREFIX="$shepherd_prefix" \
+  METADATA_OUT="$metadata_target" \
+  "$guile_bin" -s "$runner_scm"