diff --git a/docs/PROGRESS.md b/docs/PROGRESS.md index 4989ce3..6f0d973 100644 --- a/docs/PROGRESS.md +++ b/docs/PROGRESS.md @@ -2541,3 +2541,81 @@ Next recommended step: 1. focus directly on eliminating the remaining Guile / Shepherd compatibility-prefix shims from the guest runtime 2. preserve `shepherd-pid1` as an experimental selectable boot mode while that cleanup proceeds 3. once the runtime-prefix issue is reduced, reassess whether `shepherd-pid1` should replace the older `freebsd-init+rc.d-shepherd` path as the preferred Fruix boot architecture + +## 2026-04-02 — Post-Phase-10: removed runtime dependence on `/tmp` Guile / Shepherd compatibility-prefix shims + +Completed work: + +- removed the generated guest's runtime dependence on the old `/tmp` compatibility-prefix symlinks for Guile, guile-extra, and Shepherd +- wrote the subphase report: + - `docs/reports/postphase10-runtime-prefix-shims-freebsd.md` +- updated the prefix materializer in: + - `modules/fruix/system/freebsd.scm` + - bumped the prefix-materializer revision + - added deterministic post-copy sanitation for staged runtime prefixes +- removed activation-time recreation of these guest-side shims from the generated activation path: + - `/tmp/guile-freebsd-validate-install` + - `/tmp/guile-gnutls-freebsd-validate-install` + - `/tmp/shepherd-freebsd-validate-install` +- sanitized the staged guile-extra runtime so it no longer depends on those old prefixes for key module loading: + - patched `fibers/config.scm` to use `GUILE_EXTENSIONS_PATH` + - patched `gnutls.scm` to fall back to `GUILE_EXTENSIONS_PATH` + - removed stale compiled cache files that would otherwise retain the old prefix behavior: + - `fibers/config.go` + - `gnutls.go` +- sanitized the staged Shepherd runtime so it no longer depends on the old temporary prefix for `shepherd config`: + - patched `share/guile/site/3.0/shepherd/config.scm` + - removed stale compiled cache file: + - `shepherd/config.go` +- extended the real XCP-ng validation harnesses so they now explicitly check for: + - absence of the `/tmp` compatibility-prefix trees + - successful Guile module loading from the store-backed runtime +- updated: + - `tests/system/run-phase9-xcpng-boot.sh` + - `tests/system/run-phase11-shepherd-pid1-xcpng.sh` + +Validation: + +- `tests/system/run-phase9-xcpng-boot.sh` passes on the real VM with: + - workdir: `/tmp/noshim-phase9-smoke-1775143001` + - `compat_prefix_shims=absent` + - `guile_module_smoke=ok` + - `ready_marker=ready` + - `shepherd_status=running` + - `sshd_status=running` +- `tests/system/run-phase11-shepherd-pid1-xcpng.sh` passes on the real VM with: + - workdir: `/tmp/noshim-phase11-smoke-1775142712` + - `compat_prefix_shims=absent` + - `guile_module_smoke=ok` + - `ready_marker=ready` + - `shepherd_pid=1` + - `shepherd_status=running` + - `sshd_status=running` +- a direct manual guest probe also confirmed that all three `/tmp` compatibility-prefix paths are absent while Guile can still load: + - `(fibers config)` + - `(gnutls)` + - `(shepherd config)` + +Important findings: + +- the remaining native-runtime problem was narrower than the earlier boot-manager issue: + - boot was already solved + - PID 1 was already solved + - the next real dependency to remove was the guest's reliance on temporary compatibility aliases +- deleting the stale compiled cache files for the affected modules was important; otherwise Guile could continue using prefix-baked compiled forms even after the source modules were patched +- this subphase removes runtime dependence on the old `/tmp` compatibility shims, but it does not yet guarantee that every embedded historical prefix string has disappeared from every binary or metadata artifact + +Current assessment: + +- Fruix now boots and runs from a store-backed Guile / Shepherd runtime arrangement on FreeBSD without needing guest-side `/tmp` compatibility-prefix symlinks +- this now holds for both validated real-VM boot modes: + - `freebsd-init+rc.d-shepherd` + - `shepherd-pid1` +- the main remaining cleanup is deeper and lower-level: + - move the local Guile / guile-extra / Shepherd build/install flow itself closer to a truly store-native prefix so the remaining baked strings disappear from the artifacts rather than merely becoming runtime-irrelevant + +Next recommended step: + +1. keep `shepherd-pid1` available as the stronger experimental boot architecture +2. start pushing the local Guile / guile-extra / Shepherd build/install process itself toward a truly store-native prefix layout +3. clean up the remaining historical prefix strings still present in binaries, libtool metadata, and pkg-config metadata where they still matter for developer/operator workflows diff --git a/docs/reports/postphase10-runtime-prefix-shims-freebsd.md b/docs/reports/postphase10-runtime-prefix-shims-freebsd.md new file mode 100644 index 0000000..f902923 --- /dev/null +++ b/docs/reports/postphase10-runtime-prefix-shims-freebsd.md @@ -0,0 +1,160 @@ +# Post-Phase-10: removed runtime dependence on `/tmp` Guile/Shepherd compatibility-prefix shims + +Date: 2026-04-02 + +## Goal + +After validating both the `rc.d` bridge path and the new Shepherd-as-PID-1 path, the next priority was the remaining runtime-artifact problem: + +- the guest still carried activation-time `/tmp/...` compatibility symlinks for locally built Guile, guile-extra, and Shepherd prefixes +- those shims were compensating for baked-in historical paths in staged runtime artifacts + +The goal of this subphase was narrower than fully rebuilding Guile/Shepherd in a store-native way. The immediate target was: + +- remove the guest's *runtime dependence* on those `/tmp` compatibility-prefix symlinks, +- keep both validated boot modes working, +- and prove that Guile can still load the relevant modules from the store-backed runtime layout without those `/tmp` aliases. + +## Result + +This subphase succeeded. + +The generated guest no longer needs activation to create: + +- `/tmp/guile-freebsd-validate-install` +- `/tmp/guile-gnutls-freebsd-validate-install` +- `/tmp/shepherd-freebsd-validate-install` + +Those shims are now absent in the booted guest, while both of these boot paths still pass on the real XCP-ng VM: + +1. `freebsd-init+rc.d-shepherd` +2. `shepherd-pid1` + +In both cases, the guest now also passes an explicit in-guest Guile module smoke test using the staged store paths and *without* the `/tmp` prefix aliases. + +## What changed + +### 1. Activation no longer creates `/tmp` compatibility-prefix symlinks + +`modules/fruix/system/freebsd.scm` no longer emits activation logic that recreates the old `/tmp/...` prefix trees. + +That means the guest is no longer depending on those aliases as part of ordinary first boot. + +### 2. The staged runtime prefixes are sanitized as they are materialized into `/frx/store` + +The prefix materializer now post-processes the copied runtime trees so the most relevant runtime-facing Scheme modules no longer require those `/tmp` prefix aliases. + +This required a new prefix-materializer revision and deterministic post-copy sanitation. + +### 3. Guile-extra runtime fixes + +The staged guile-extra runtime now: + +- patches `fibers/config.scm` so it can use `GUILE_EXTENSIONS_PATH` instead of falling back to the old `/tmp/guile-gnutls-freebsd-validate-install/...` path +- patches `gnutls.scm` so it can fall back to `GUILE_EXTENSIONS_PATH` +- removes stale compiled cache files that still embedded the old prefix values and would otherwise override the corrected source modules: + - `fibers/config.go` + - `gnutls.go` + +### 4. Shepherd runtime fixes + +The staged Shepherd runtime now: + +- patches `share/guile/site/3.0/shepherd/config.scm` to use real runtime-facing paths instead of the old temporary install prefix: + - `Prefix-dir` => `/frx` + - `%localstatedir` => `/var` + - `%runstatedir` => `/var/run` + - `%sysconfdir` => `/etc` + - `%localedir` => `/usr/share/locale` + - `%pkglibdir` => `/usr/local/lib/shepherd` +- removes the stale compiled cache file that still embedded the old prefix values: + - `shepherd/config.go` + +### 5. Real-VM validation harnesses now assert shim absence directly + +The main real-VM validation scripts now check that the old `/tmp` compatibility-prefix trees are absent in the guest: + +- `tests/system/run-phase9-xcpng-boot.sh` +- `tests/system/run-phase11-shepherd-pid1-xcpng.sh` + +They also run an in-guest Guile module smoke test using the store-backed runtime environment and require: + +- `guile_module_smoke=ok` + +## Validation + +### Real VM, `freebsd-init+rc.d-shepherd` + +Passing run: + +- `PASS phase9-xcpng-boot` +- workdir: `/tmp/noshim-phase9-smoke-1775143001` + +Key metadata: + +```text +compat_prefix_shims=absent +guile_module_smoke=ok +ready_marker=ready +shepherd_status=running +sshd_status=running +``` + +### Real VM, `shepherd-pid1` + +Passing run: + +- `PASS phase11-shepherd-pid1-xcpng` +- workdir: `/tmp/noshim-phase11-smoke-1775142712` + +Key metadata: + +```text +compat_prefix_shims=absent +guile_module_smoke=ok +ready_marker=ready +shepherd_pid=1 +shepherd_status=running +sshd_status=running +``` + +### Manual guest confirmation + +A direct SSH probe on the real guest confirmed that all three historical `/tmp` compatibility prefixes are absent while Guile can still load: + +- `(fibers config)` +- `(gnutls)` +- `(shepherd config)` + +with output including: + +```text +ABSENT:/tmp/guile-freebsd-validate-install +ABSENT:/tmp/guile-gnutls-freebsd-validate-install +ABSENT:/tmp/shepherd-freebsd-validate-install +1.4.2 +/var/run +``` + +## Assessment + +This removes an important transitional runtime crutch. + +Fruix is no longer merely *booting despite* historical `/tmp` prefix aliases; it now boots and runs Guile/Shepherd services from the staged store-backed runtime arrangement without those guest-side compatibility symlinks. + +## Remaining limitation + +This does **not** mean every historical prefix string has disappeared from every staged artifact. + +Some compiled binaries and libraries still contain old build/install strings internally, especially in places such as: + +- `libguile` +- libtool metadata +- pkg-config metadata +- extension shared objects built upstream with the original prefixes + +However, the important practical milestone is that the guest no longer depends on `/tmp` compatibility-prefix symlinks at runtime. + +## Recommended next step + +The next, deeper cleanup would be to eliminate the remaining baked prefix strings from the runtime artifacts themselves by moving the local Guile/guile-extra/Shepherd build/install flow closer to a genuinely store-native prefix from the start, rather than relying on post-copy sanitation. diff --git a/modules/fruix/system/freebsd.scm b/modules/fruix/system/freebsd.scm index 5f1e6ea..dac7787 100644 --- a/modules/fruix/system/freebsd.scm +++ b/modules/fruix/system/freebsd.scm @@ -298,7 +298,65 @@ (hash-set! cache (package-cache-key package) output-path) output-path)))) -(define prefix-materializer-version "2") +(define prefix-materializer-version "3") + +(define (string-replace-all str old new) + (let ((old-len (string-length old))) + (let loop ((start 0) (chunks '())) + (let ((index (string-contains str old start))) + (if index + (loop (+ index old-len) + (cons new + (cons (substring str start index) chunks))) + (apply string-append + (reverse (cons (substring str start) chunks)))))))) + +(define (rewrite-text-file path replacements) + (when (file-exists? path) + (let* ((mode (stat:perms (stat path))) + (original (call-with-input-file path get-string-all)) + (updated (fold (lambda (replacement text) + (string-replace-all text (car replacement) (cdr replacement))) + original + replacements))) + (unless (string=? original updated) + (write-file path updated) + (chmod path mode))))) + +(define (delete-file-if-exists path) + (when (file-exists? path) + (delete-file path))) + +(define (sanitize-materialized-prefix name output-path) + (cond + ((string=? name "fruix-guile-extra") + (rewrite-text-file + (string-append output-path "/share/guile/site/3.0/fibers/config.scm") + '(("((getenv \"FIBERS_BUILD_DIR\")\n => (lambda (builddir) (in-vicinity builddir \".libs\")))\n (else \"/tmp/guile-gnutls-freebsd-validate-install/lib/guile/3.0/extensions\"))" + . "((getenv \"FIBERS_BUILD_DIR\")\n => (lambda (builddir) (in-vicinity builddir \".libs\")))\n ((getenv \"GUILE_EXTENSIONS_PATH\"))\n (else \"/usr/local/lib/guile/3.0/extensions\"))"))) + (rewrite-text-file + (string-append output-path "/share/guile/site/3.0/gnutls.scm") + '(("\"/tmp/guile-gnutls-freebsd-validate-install/lib/guile/3.0/extensions\"" + . "(or (getenv \"GUILE_EXTENSIONS_PATH\") \"/usr/local/lib/guile/3.0/extensions\")"))) + (delete-file-if-exists (string-append output-path "/lib/guile/3.0/site-ccache/fibers/config.go")) + (delete-file-if-exists (string-append output-path "/lib/guile/3.0/site-ccache/gnutls.go"))) + ((string=? name "fruix-shepherd-runtime") + (rewrite-text-file + (string-append output-path "/share/guile/site/3.0/shepherd/config.scm") + '(("(define Prefix-dir \"/tmp/shepherd-freebsd-validate-install\")" + . "(define Prefix-dir \"/frx\")") + ("(define %localstatedir \"/tmp/shepherd-freebsd-validate-install/var\")" + . "(define %localstatedir \"/var\")") + ("(define %runstatedir \"/tmp/shepherd-freebsd-validate-install/var/run\")" + . "(define %runstatedir \"/var/run\")") + ("(define %sysconfdir \"/tmp/shepherd-freebsd-validate-install/etc\")" + . "(define %sysconfdir \"/etc\")") + ("(define %localedir \"/tmp/shepherd-freebsd-validate-install/share/locale\")" + . "(define %localedir \"/usr/share/locale\")") + ("(define %pkglibdir \"/tmp/shepherd-freebsd-validate-install/lib/shepherd\")" + . "(define %pkglibdir \"/usr/local/lib/shepherd\")"))) + (delete-file-if-exists (string-append output-path "/lib/guile/3.0/site-ccache/shepherd/config.go")))) + #t) (define (prefix-manifest-string source-path extra-files) (string-append @@ -345,6 +403,7 @@ (copy-extra-node (car entry) (string-append output-path "/" (cdr entry)))) extra-files) + (sanitize-materialized-prefix name output-path) (write-file (string-append output-path "/.fruix-package") manifest)) output-path)) @@ -578,23 +637,7 @@ uid gid home))))) users) "")) - (compat-prefixes - (string-append - (if guile-store - (string-append - "rm -rf /tmp/guile-freebsd-validate-install\n" - "ln -s " guile-store " /tmp/guile-freebsd-validate-install\n") - "") - (if guile-extra-store - (string-append - "rm -rf /tmp/guile-gnutls-freebsd-validate-install\n" - "ln -s " guile-extra-store " /tmp/guile-gnutls-freebsd-validate-install\n") - "") - (if shepherd-store - (string-append - "rm -rf /tmp/shepherd-freebsd-validate-install\n" - "ln -s " shepherd-store " /tmp/shepherd-freebsd-validate-install\n") - ""))) + (compat-prefixes "") (ssh-section (string-append "mkdir -p /var/empty /etc/ssh /root/.ssh\n" diff --git a/tests/system/run-phase11-shepherd-pid1-xcpng.sh b/tests/system/run-phase11-shepherd-pid1-xcpng.sh index bc2c9c4..1f27cdb 100755 --- a/tests/system/run-phase11-shepherd-pid1-xcpng.sh +++ b/tests/system/run-phase11-shepherd-pid1-xcpng.sh @@ -75,6 +75,9 @@ closure_path=$(sed -n 's/^closure_path=//p' "$phase8_metadata") closure_base=$(basename "$closure_path") raw_sha256=$(sed -n 's/^raw_sha256=//p' "$phase8_metadata") image_store_path=$(sed -n 's/^image_store_path=//p' "$phase8_metadata") +guile_store=$(grep 'fruix-guile-runtime-3.0$' "$closure_path/.references" | head -n 1) +guile_extra_store=$(grep 'fruix-guile-extra-3.0$' "$closure_path/.references" | head -n 1) +shepherd_store=$(grep 'fruix-shepherd-runtime-1.0.9$' "$closure_path/.references" | head -n 1) command -v qemu-img >/dev/null 2>&1 || { echo "qemu-img is required to convert the raw Fruix image to XCP-ng-compatible VHD" >&2 @@ -149,6 +152,8 @@ logger_log=$(ssh_guest 'cat /var/log/fruix-shepherd.log' | tr '\n' ' ') sshd_status=$(ssh_guest 'service sshd onestatus >/dev/null 2>&1 && echo running || echo stopped') uname_output=$(ssh_guest 'uname -sr') operator_home_listing=$(ssh_guest 'ls -d /home/operator') +compat_prefix_shims=$(ssh_guest 'for p in /tmp/guile-freebsd-validate-install /tmp/guile-gnutls-freebsd-validate-install /tmp/shepherd-freebsd-validate-install; do if [ -e "$p" ] || [ -L "$p" ]; then echo present; exit 0; fi; done; echo absent') +guile_module_smoke=$(ssh_guest "env LANG='C.UTF-8' LC_ALL='C.UTF-8' LD_LIBRARY_PATH='$guile_extra_store/lib:$guile_store/lib:/usr/local/lib' GUILE_SYSTEM_PATH='$guile_store/share/guile/3.0:$guile_store/share/guile/site/3.0:$guile_store/share/guile/site:$guile_store/share/guile' GUILE_LOAD_PATH='$shepherd_store/share/guile/site/3.0:$guile_extra_store/share/guile/site/3.0' GUILE_SYSTEM_COMPILED_PATH='$guile_store/lib/guile/3.0/ccache:$guile_store/lib/guile/3.0/site-ccache' GUILE_LOAD_COMPILED_PATH='$shepherd_store/lib/guile/3.0/site-ccache:$guile_extra_store/lib/guile/3.0/site-ccache' GUILE_SYSTEM_EXTENSIONS_PATH='$guile_store/lib/guile/3.0/extensions' GUILE_EXTENSIONS_PATH='$guile_extra_store/lib/guile/3.0/extensions' '$guile_store/bin/guile' --no-auto-compile -c '(use-modules (fibers config) (gnutls) (shepherd config)) (display \"ok\") (newline)'") activate_preview=$(ssh_guest 'head -n 5 /run/current-system/activate' | tr '\n' ' ') [ "$ready_marker" = ready ] || { echo "unexpected ready marker contents: $ready_marker" >&2; exit 1; } @@ -156,6 +161,8 @@ activate_preview=$(ssh_guest 'head -n 5 /run/current-system/activate' | tr '\n' [ "$shepherd_socket" = present ] || { echo "shepherd socket is missing" >&2; exit 1; } [ "$shepherd_status" = running ] || { echo "shepherd is not running" >&2; exit 1; } [ "$sshd_status" = running ] || { echo "sshd is not running" >&2; exit 1; } +[ "$compat_prefix_shims" = absent ] || { echo "compatibility prefix shims are still present in /tmp" >&2; exit 1; } +[ "$guile_module_smoke" = ok ] || { echo "guest Guile module smoke failed: $guile_module_smoke" >&2; exit 1; } [ "$run_current_system_target" = "/frx/store/$closure_base" ] || { echo "unexpected /run/current-system target in guest: $run_current_system_target" >&2 exit 1 @@ -194,6 +201,8 @@ sshd_status=$sshd_status logger_log=$logger_log uname_output=$uname_output operator_home_listing=$operator_home_listing +compat_prefix_shims=$compat_prefix_shims +guile_module_smoke=$guile_module_smoke activate_preview=$activate_preview boot_backend=xcp-ng-xo-cli init_mode=shepherd-pid1 diff --git a/tests/system/run-phase9-xcpng-boot.sh b/tests/system/run-phase9-xcpng-boot.sh index ab0dbc3..3a1fb0c 100755 --- a/tests/system/run-phase9-xcpng-boot.sh +++ b/tests/system/run-phase9-xcpng-boot.sh @@ -75,6 +75,9 @@ closure_path=$(sed -n 's/^closure_path=//p' "$phase8_metadata") closure_base=$(basename "$closure_path") raw_sha256=$(sed -n 's/^raw_sha256=//p' "$phase8_metadata") image_store_path=$(sed -n 's/^image_store_path=//p' "$phase8_metadata") +guile_store=$(grep 'fruix-guile-runtime-3.0$' "$closure_path/.references" | head -n 1) +guile_extra_store=$(grep 'fruix-guile-extra-3.0$' "$closure_path/.references" | head -n 1) +shepherd_store=$(grep 'fruix-shepherd-runtime-1.0.9$' "$closure_path/.references" | head -n 1) command -v qemu-img >/dev/null 2>&1 || { echo "qemu-img is required to convert the raw Fruix image to XCP-ng-compatible VHD" >&2 @@ -148,11 +151,15 @@ logger_log=$(ssh_guest 'cat /var/log/fruix-shepherd.log' | tr '\n' ' ') sshd_status=$(ssh_guest 'service sshd onestatus >/dev/null 2>&1 && echo running || echo stopped') uname_output=$(ssh_guest 'uname -sr') operator_home_listing=$(ssh_guest 'ls -d /home/operator') +compat_prefix_shims=$(ssh_guest 'for p in /tmp/guile-freebsd-validate-install /tmp/guile-gnutls-freebsd-validate-install /tmp/shepherd-freebsd-validate-install; do if [ -e "$p" ] || [ -L "$p" ]; then echo present; exit 0; fi; done; echo absent') +guile_module_smoke=$(ssh_guest "env LANG='C.UTF-8' LC_ALL='C.UTF-8' LD_LIBRARY_PATH='$guile_extra_store/lib:$guile_store/lib:/usr/local/lib' GUILE_SYSTEM_PATH='$guile_store/share/guile/3.0:$guile_store/share/guile/site/3.0:$guile_store/share/guile/site:$guile_store/share/guile' GUILE_LOAD_PATH='$shepherd_store/share/guile/site/3.0:$guile_extra_store/share/guile/site/3.0' GUILE_SYSTEM_COMPILED_PATH='$guile_store/lib/guile/3.0/ccache:$guile_store/lib/guile/3.0/site-ccache' GUILE_LOAD_COMPILED_PATH='$shepherd_store/lib/guile/3.0/site-ccache:$guile_extra_store/lib/guile/3.0/site-ccache' GUILE_SYSTEM_EXTENSIONS_PATH='$guile_store/lib/guile/3.0/extensions' GUILE_EXTENSIONS_PATH='$guile_extra_store/lib/guile/3.0/extensions' '$guile_store/bin/guile' --no-auto-compile -c '(use-modules (fibers config) (gnutls) (shepherd config)) (display \"ok\") (newline)'") activate_preview=$(ssh_guest 'head -n 5 /run/current-system/activate' | tr '\n' ' ') [ "$ready_marker" = ready ] || { echo "unexpected ready marker contents: $ready_marker" >&2; exit 1; } [ "$shepherd_status" = running ] || { echo "fruix_shepherd is not running" >&2; exit 1; } [ "$sshd_status" = running ] || { echo "sshd is not running" >&2; exit 1; } +[ "$compat_prefix_shims" = absent ] || { echo "compatibility prefix shims are still present in /tmp" >&2; exit 1; } +[ "$guile_module_smoke" = ok ] || { echo "guest Guile module smoke failed: $guile_module_smoke" >&2; exit 1; } [ "$run_current_system_target" = "/frx/store/$closure_base" ] || { echo "unexpected /run/current-system target in guest: $run_current_system_target" >&2 exit 1 @@ -189,6 +196,8 @@ sshd_status=$sshd_status logger_log=$logger_log uname_output=$uname_output operator_home_listing=$operator_home_listing +compat_prefix_shims=$compat_prefix_shims +guile_module_smoke=$guile_module_smoke activate_preview=$activate_preview boot_backend=xcp-ng-xo-cli operator_access=ssh-root-key