Remove guest runtime prefix shim dependency

docs/PROGRESS.md

@@ -2541,3 +2541,81 @@ Next recommended step:
 1. focus directly on eliminating the remaining Guile / Shepherd compatibility-prefix shims from the guest runtime
 2. preserve `shepherd-pid1` as an experimental selectable boot mode while that cleanup proceeds
 3. once the runtime-prefix issue is reduced, reassess whether `shepherd-pid1` should replace the older `freebsd-init+rc.d-shepherd` path as the preferred Fruix boot architecture
+
+## 2026-04-02 — Post-Phase-10: removed runtime dependence on `/tmp` Guile / Shepherd compatibility-prefix shims
+
+Completed work:
+
+- removed the generated guest's runtime dependence on the old `/tmp` compatibility-prefix symlinks for Guile, guile-extra, and Shepherd
+- wrote the subphase report:
+  - `docs/reports/postphase10-runtime-prefix-shims-freebsd.md`
+- updated the prefix materializer in:
+  - `modules/fruix/system/freebsd.scm`
+  - bumped the prefix-materializer revision
+  - added deterministic post-copy sanitation for staged runtime prefixes
+- removed activation-time recreation of these guest-side shims from the generated activation path:
+  - `/tmp/guile-freebsd-validate-install`
+  - `/tmp/guile-gnutls-freebsd-validate-install`
+  - `/tmp/shepherd-freebsd-validate-install`
+- sanitized the staged guile-extra runtime so it no longer depends on those old prefixes for key module loading:
+  - patched `fibers/config.scm` to use `GUILE_EXTENSIONS_PATH`
+  - patched `gnutls.scm` to fall back to `GUILE_EXTENSIONS_PATH`
+  - removed stale compiled cache files that would otherwise retain the old prefix behavior:
+    - `fibers/config.go`
+    - `gnutls.go`
+- sanitized the staged Shepherd runtime so it no longer depends on the old temporary prefix for `shepherd config`:
+  - patched `share/guile/site/3.0/shepherd/config.scm`
+  - removed stale compiled cache file:
+    - `shepherd/config.go`
+- extended the real XCP-ng validation harnesses so they now explicitly check for:
+  - absence of the `/tmp` compatibility-prefix trees
+  - successful Guile module loading from the store-backed runtime
+- updated:
+  - `tests/system/run-phase9-xcpng-boot.sh`
+  - `tests/system/run-phase11-shepherd-pid1-xcpng.sh`
+
+Validation:
+
+- `tests/system/run-phase9-xcpng-boot.sh` passes on the real VM with:
+  - workdir: `/tmp/noshim-phase9-smoke-1775143001`
+  - `compat_prefix_shims=absent`
+  - `guile_module_smoke=ok`
+  - `ready_marker=ready`
+  - `shepherd_status=running`
+  - `sshd_status=running`
+- `tests/system/run-phase11-shepherd-pid1-xcpng.sh` passes on the real VM with:
+  - workdir: `/tmp/noshim-phase11-smoke-1775142712`
+  - `compat_prefix_shims=absent`
+  - `guile_module_smoke=ok`
+  - `ready_marker=ready`
+  - `shepherd_pid=1`
+  - `shepherd_status=running`
+  - `sshd_status=running`
+- a direct manual guest probe also confirmed that all three `/tmp` compatibility-prefix paths are absent while Guile can still load:
+  - `(fibers config)`
+  - `(gnutls)`
+  - `(shepherd config)`
+
+Important findings:
+
+- the remaining native-runtime problem was narrower than the earlier boot-manager issue:
+  - boot was already solved
+  - PID 1 was already solved
+  - the next real dependency to remove was the guest's reliance on temporary compatibility aliases
+- deleting the stale compiled cache files for the affected modules was important; otherwise Guile could continue using prefix-baked compiled forms even after the source modules were patched
+- this subphase removes runtime dependence on the old `/tmp` compatibility shims, but it does not yet guarantee that every embedded historical prefix string has disappeared from every binary or metadata artifact
+
+Current assessment:
+
+- Fruix now boots and runs from a store-backed Guile / Shepherd runtime arrangement on FreeBSD without needing guest-side `/tmp` compatibility-prefix symlinks
+- this now holds for both validated real-VM boot modes:
+  - `freebsd-init+rc.d-shepherd`
+  - `shepherd-pid1`
+- the main remaining cleanup is deeper and lower-level:
+  - move the local Guile / guile-extra / Shepherd build/install flow itself closer to a truly store-native prefix so the remaining baked strings disappear from the artifacts rather than merely becoming runtime-irrelevant
+
+Next recommended step:
+
+1. keep `shepherd-pid1` available as the stronger experimental boot architecture
+2. start pushing the local Guile / guile-extra / Shepherd build/install process itself toward a truly store-native prefix layout
+3. clean up the remaining historical prefix strings still present in binaries, libtool metadata, and pkg-config metadata where they still matter for developer/operator workflows

docs/reports/postphase10-runtime-prefix-shims-freebsd.md

@@ -0,0 +1,160 @@
+# Post-Phase-10: removed runtime dependence on `/tmp` Guile/Shepherd compatibility-prefix shims
+
+Date: 2026-04-02
+
+## Goal
+
+After validating both the `rc.d` bridge path and the new Shepherd-as-PID-1 path, the next priority was the remaining runtime-artifact problem:
+
+- the guest still carried activation-time `/tmp/...` compatibility symlinks for locally built Guile, guile-extra, and Shepherd prefixes
+- those shims were compensating for baked-in historical paths in staged runtime artifacts
+
+The goal of this subphase was narrower than fully rebuilding Guile/Shepherd in a store-native way.  The immediate target was:
+
+- remove the guest's *runtime dependence* on those `/tmp` compatibility-prefix symlinks,
+- keep both validated boot modes working,
+- and prove that Guile can still load the relevant modules from the store-backed runtime layout without those `/tmp` aliases.
+
+## Result
+
+This subphase succeeded.
+
+The generated guest no longer needs activation to create:
+
+- `/tmp/guile-freebsd-validate-install`
+- `/tmp/guile-gnutls-freebsd-validate-install`
+- `/tmp/shepherd-freebsd-validate-install`
+
+Those shims are now absent in the booted guest, while both of these boot paths still pass on the real XCP-ng VM:
+
+1. `freebsd-init+rc.d-shepherd`
+2. `shepherd-pid1`
+
+In both cases, the guest now also passes an explicit in-guest Guile module smoke test using the staged store paths and *without* the `/tmp` prefix aliases.
+
+## What changed
+
+### 1. Activation no longer creates `/tmp` compatibility-prefix symlinks
+
+`modules/fruix/system/freebsd.scm` no longer emits activation logic that recreates the old `/tmp/...` prefix trees.
+
+That means the guest is no longer depending on those aliases as part of ordinary first boot.
+
+### 2. The staged runtime prefixes are sanitized as they are materialized into `/frx/store`
+
+The prefix materializer now post-processes the copied runtime trees so the most relevant runtime-facing Scheme modules no longer require those `/tmp` prefix aliases.
+
+This required a new prefix-materializer revision and deterministic post-copy sanitation.
+
+### 3. Guile-extra runtime fixes
+
+The staged guile-extra runtime now:
+
+- patches `fibers/config.scm` so it can use `GUILE_EXTENSIONS_PATH` instead of falling back to the old `/tmp/guile-gnutls-freebsd-validate-install/...` path
+- patches `gnutls.scm` so it can fall back to `GUILE_EXTENSIONS_PATH`
+- removes stale compiled cache files that still embedded the old prefix values and would otherwise override the corrected source modules:
+  - `fibers/config.go`
+  - `gnutls.go`
+
+### 4. Shepherd runtime fixes
+
+The staged Shepherd runtime now:
+
+- patches `share/guile/site/3.0/shepherd/config.scm` to use real runtime-facing paths instead of the old temporary install prefix:
+  - `Prefix-dir` => `/frx`
+  - `%localstatedir` => `/var`
+  - `%runstatedir` => `/var/run`
+  - `%sysconfdir` => `/etc`
+  - `%localedir` => `/usr/share/locale`
+  - `%pkglibdir` => `/usr/local/lib/shepherd`
+- removes the stale compiled cache file that still embedded the old prefix values:
+  - `shepherd/config.go`
+
+### 5. Real-VM validation harnesses now assert shim absence directly
+
+The main real-VM validation scripts now check that the old `/tmp` compatibility-prefix trees are absent in the guest:
+
+- `tests/system/run-phase9-xcpng-boot.sh`
+- `tests/system/run-phase11-shepherd-pid1-xcpng.sh`
+
+They also run an in-guest Guile module smoke test using the store-backed runtime environment and require:
+
+- `guile_module_smoke=ok`
+
+## Validation
+
+### Real VM, `freebsd-init+rc.d-shepherd`
+
+Passing run:
+
+- `PASS phase9-xcpng-boot`
+- workdir: `/tmp/noshim-phase9-smoke-1775143001`
+
+Key metadata:
+
+```text
+compat_prefix_shims=absent
+guile_module_smoke=ok
+ready_marker=ready
+shepherd_status=running
+sshd_status=running
+```
+
+### Real VM, `shepherd-pid1`
+
+Passing run:
+
+- `PASS phase11-shepherd-pid1-xcpng`
+- workdir: `/tmp/noshim-phase11-smoke-1775142712`
+
+Key metadata:
+
+```text
+compat_prefix_shims=absent
+guile_module_smoke=ok
+ready_marker=ready
+shepherd_pid=1
+shepherd_status=running
+sshd_status=running
+```
+
+### Manual guest confirmation
+
+A direct SSH probe on the real guest confirmed that all three historical `/tmp` compatibility prefixes are absent while Guile can still load:
+
+- `(fibers config)`
+- `(gnutls)`
+- `(shepherd config)`
+
+with output including:
+
+```text
+ABSENT:/tmp/guile-freebsd-validate-install
+ABSENT:/tmp/guile-gnutls-freebsd-validate-install
+ABSENT:/tmp/shepherd-freebsd-validate-install
+1.4.2
+/var/run
+```
+
+## Assessment
+
+This removes an important transitional runtime crutch.
+
+Fruix is no longer merely *booting despite* historical `/tmp` prefix aliases; it now boots and runs Guile/Shepherd services from the staged store-backed runtime arrangement without those guest-side compatibility symlinks.
+
+## Remaining limitation
+
+This does **not** mean every historical prefix string has disappeared from every staged artifact.
+
+Some compiled binaries and libraries still contain old build/install strings internally, especially in places such as:
+
+- `libguile`
+- libtool metadata
+- pkg-config metadata
+- extension shared objects built upstream with the original prefixes
+
+However, the important practical milestone is that the guest no longer depends on `/tmp` compatibility-prefix symlinks at runtime.
+
+## Recommended next step
+
+The next, deeper cleanup would be to eliminate the remaining baked prefix strings from the runtime artifacts themselves by moving the local Guile/guile-extra/Shepherd build/install flow closer to a genuinely store-native prefix from the start, rather than relying on post-copy sanitation.

modules/fruix/system/freebsd.scm

@@ -298,7 +298,65 @@
           (hash-set! cache (package-cache-key package) output-path)
           output-path))))
 
-(define prefix-materializer-version "2")
+(define prefix-materializer-version "3")
+
+(define (string-replace-all str old new)
+  (let ((old-len (string-length old)))
+    (let loop ((start 0) (chunks '()))
+      (let ((index (string-contains str old start)))
+        (if index
+            (loop (+ index old-len)
+                  (cons new
+                        (cons (substring str start index) chunks)))
+            (apply string-append
+                   (reverse (cons (substring str start) chunks))))))))
+
+(define (rewrite-text-file path replacements)
+  (when (file-exists? path)
+    (let* ((mode (stat:perms (stat path)))
+           (original (call-with-input-file path get-string-all))
+           (updated (fold (lambda (replacement text)
+                            (string-replace-all text (car replacement) (cdr replacement)))
+                          original
+                          replacements)))
+      (unless (string=? original updated)
+        (write-file path updated)
+        (chmod path mode)))))
+
+(define (delete-file-if-exists path)
+  (when (file-exists? path)
+    (delete-file path)))
+
+(define (sanitize-materialized-prefix name output-path)
+  (cond
+   ((string=? name "fruix-guile-extra")
+    (rewrite-text-file
+     (string-append output-path "/share/guile/site/3.0/fibers/config.scm")
+     '(("((getenv \"FIBERS_BUILD_DIR\")\n    => (lambda (builddir) (in-vicinity builddir \".libs\")))\n   (else \"/tmp/guile-gnutls-freebsd-validate-install/lib/guile/3.0/extensions\"))"
+        . "((getenv \"FIBERS_BUILD_DIR\")\n    => (lambda (builddir) (in-vicinity builddir \".libs\")))\n   ((getenv \"GUILE_EXTENSIONS_PATH\"))\n   (else \"/usr/local/lib/guile/3.0/extensions\"))")))
+    (rewrite-text-file
+     (string-append output-path "/share/guile/site/3.0/gnutls.scm")
+     '(("\"/tmp/guile-gnutls-freebsd-validate-install/lib/guile/3.0/extensions\""
+        . "(or (getenv \"GUILE_EXTENSIONS_PATH\") \"/usr/local/lib/guile/3.0/extensions\")")))
+    (delete-file-if-exists (string-append output-path "/lib/guile/3.0/site-ccache/fibers/config.go"))
+    (delete-file-if-exists (string-append output-path "/lib/guile/3.0/site-ccache/gnutls.go")))
+   ((string=? name "fruix-shepherd-runtime")
+    (rewrite-text-file
+     (string-append output-path "/share/guile/site/3.0/shepherd/config.scm")
+     '(("(define Prefix-dir \"/tmp/shepherd-freebsd-validate-install\")"
+        . "(define Prefix-dir \"/frx\")")
+       ("(define %localstatedir \"/tmp/shepherd-freebsd-validate-install/var\")"
+        . "(define %localstatedir \"/var\")")
+       ("(define %runstatedir \"/tmp/shepherd-freebsd-validate-install/var/run\")"
+        . "(define %runstatedir \"/var/run\")")
+       ("(define %sysconfdir \"/tmp/shepherd-freebsd-validate-install/etc\")"
+        . "(define %sysconfdir \"/etc\")")
+       ("(define %localedir \"/tmp/shepherd-freebsd-validate-install/share/locale\")"
+        . "(define %localedir \"/usr/share/locale\")")
+       ("(define %pkglibdir \"/tmp/shepherd-freebsd-validate-install/lib/shepherd\")"
+        . "(define %pkglibdir \"/usr/local/lib/shepherd\")")))
+    (delete-file-if-exists (string-append output-path "/lib/guile/3.0/site-ccache/shepherd/config.go"))))
+  #t)
 
 (define (prefix-manifest-string source-path extra-files)
   (string-append
@@ -345,6 +403,7 @@
                   (copy-extra-node (car entry)
                                    (string-append output-path "/" (cdr entry))))
                 extra-files)
+      (sanitize-materialized-prefix name output-path)
       (write-file (string-append output-path "/.fruix-package") manifest))
     output-path))
 
@@ -578,23 +637,7 @@
                                  uid gid home)))))
                 users)
            ""))
-         (compat-prefixes
+         (compat-prefixes "")
-          (string-append
-           (if guile-store
-               (string-append
-                "rm -rf /tmp/guile-freebsd-validate-install\n"
-                "ln -s " guile-store " /tmp/guile-freebsd-validate-install\n")
-               "")
-           (if guile-extra-store
-               (string-append
-                "rm -rf /tmp/guile-gnutls-freebsd-validate-install\n"
-                "ln -s " guile-extra-store " /tmp/guile-gnutls-freebsd-validate-install\n")
-               "")
-           (if shepherd-store
-               (string-append
-                "rm -rf /tmp/shepherd-freebsd-validate-install\n"
-                "ln -s " shepherd-store " /tmp/shepherd-freebsd-validate-install\n")
-               "")))
          (ssh-section
           (string-append
            "mkdir -p /var/empty /etc/ssh /root/.ssh\n"

tests/system/run-phase11-shepherd-pid1-xcpng.sh

@@ -75,6 +75,9 @@ closure_path=$(sed -n 's/^closure_path=//p' "$phase8_metadata")
 closure_base=$(basename "$closure_path")
 raw_sha256=$(sed -n 's/^raw_sha256=//p' "$phase8_metadata")
 image_store_path=$(sed -n 's/^image_store_path=//p' "$phase8_metadata")
+guile_store=$(grep 'fruix-guile-runtime-3.0$' "$closure_path/.references" | head -n 1)
+guile_extra_store=$(grep 'fruix-guile-extra-3.0$' "$closure_path/.references" | head -n 1)
+shepherd_store=$(grep 'fruix-shepherd-runtime-1.0.9$' "$closure_path/.references" | head -n 1)
 
 command -v qemu-img >/dev/null 2>&1 || {
   echo "qemu-img is required to convert the raw Fruix image to XCP-ng-compatible VHD" >&2
@@ -149,6 +152,8 @@ logger_log=$(ssh_guest 'cat /var/log/fruix-shepherd.log' | tr '\n' ' ')
 sshd_status=$(ssh_guest 'service sshd onestatus >/dev/null 2>&1 && echo running || echo stopped')
 uname_output=$(ssh_guest 'uname -sr')
 operator_home_listing=$(ssh_guest 'ls -d /home/operator')
+compat_prefix_shims=$(ssh_guest 'for p in /tmp/guile-freebsd-validate-install /tmp/guile-gnutls-freebsd-validate-install /tmp/shepherd-freebsd-validate-install; do if [ -e "$p" ] || [ -L "$p" ]; then echo present; exit 0; fi; done; echo absent')
+guile_module_smoke=$(ssh_guest "env LANG='C.UTF-8' LC_ALL='C.UTF-8' LD_LIBRARY_PATH='$guile_extra_store/lib:$guile_store/lib:/usr/local/lib' GUILE_SYSTEM_PATH='$guile_store/share/guile/3.0:$guile_store/share/guile/site/3.0:$guile_store/share/guile/site:$guile_store/share/guile' GUILE_LOAD_PATH='$shepherd_store/share/guile/site/3.0:$guile_extra_store/share/guile/site/3.0' GUILE_SYSTEM_COMPILED_PATH='$guile_store/lib/guile/3.0/ccache:$guile_store/lib/guile/3.0/site-ccache' GUILE_LOAD_COMPILED_PATH='$shepherd_store/lib/guile/3.0/site-ccache:$guile_extra_store/lib/guile/3.0/site-ccache' GUILE_SYSTEM_EXTENSIONS_PATH='$guile_store/lib/guile/3.0/extensions' GUILE_EXTENSIONS_PATH='$guile_extra_store/lib/guile/3.0/extensions' '$guile_store/bin/guile' --no-auto-compile -c '(use-modules (fibers config) (gnutls) (shepherd config)) (display \"ok\") (newline)'")
 activate_preview=$(ssh_guest 'head -n 5 /run/current-system/activate' | tr '\n' ' ')
 
 [ "$ready_marker" = ready ] || { echo "unexpected ready marker contents: $ready_marker" >&2; exit 1; }
@@ -156,6 +161,8 @@ activate_preview=$(ssh_guest 'head -n 5 /run/current-system/activate' | tr '\n'
 [ "$shepherd_socket" = present ] || { echo "shepherd socket is missing" >&2; exit 1; }
 [ "$shepherd_status" = running ] || { echo "shepherd is not running" >&2; exit 1; }
 [ "$sshd_status" = running ] || { echo "sshd is not running" >&2; exit 1; }
+[ "$compat_prefix_shims" = absent ] || { echo "compatibility prefix shims are still present in /tmp" >&2; exit 1; }
+[ "$guile_module_smoke" = ok ] || { echo "guest Guile module smoke failed: $guile_module_smoke" >&2; exit 1; }
 [ "$run_current_system_target" = "/frx/store/$closure_base" ] || {
   echo "unexpected /run/current-system target in guest: $run_current_system_target" >&2
   exit 1
@@ -194,6 +201,8 @@ sshd_status=$sshd_status
 logger_log=$logger_log
 uname_output=$uname_output
 operator_home_listing=$operator_home_listing
+compat_prefix_shims=$compat_prefix_shims
+guile_module_smoke=$guile_module_smoke
 activate_preview=$activate_preview
 boot_backend=xcp-ng-xo-cli
 init_mode=shepherd-pid1

tests/system/run-phase9-xcpng-boot.sh

@@ -75,6 +75,9 @@ closure_path=$(sed -n 's/^closure_path=//p' "$phase8_metadata")
 closure_base=$(basename "$closure_path")
 raw_sha256=$(sed -n 's/^raw_sha256=//p' "$phase8_metadata")
 image_store_path=$(sed -n 's/^image_store_path=//p' "$phase8_metadata")
+guile_store=$(grep 'fruix-guile-runtime-3.0$' "$closure_path/.references" | head -n 1)
+guile_extra_store=$(grep 'fruix-guile-extra-3.0$' "$closure_path/.references" | head -n 1)
+shepherd_store=$(grep 'fruix-shepherd-runtime-1.0.9$' "$closure_path/.references" | head -n 1)
 
 command -v qemu-img >/dev/null 2>&1 || {
   echo "qemu-img is required to convert the raw Fruix image to XCP-ng-compatible VHD" >&2
@@ -148,11 +151,15 @@ logger_log=$(ssh_guest 'cat /var/log/fruix-shepherd.log' | tr '\n' ' ')
 sshd_status=$(ssh_guest 'service sshd onestatus >/dev/null 2>&1 && echo running || echo stopped')
 uname_output=$(ssh_guest 'uname -sr')
 operator_home_listing=$(ssh_guest 'ls -d /home/operator')
+compat_prefix_shims=$(ssh_guest 'for p in /tmp/guile-freebsd-validate-install /tmp/guile-gnutls-freebsd-validate-install /tmp/shepherd-freebsd-validate-install; do if [ -e "$p" ] || [ -L "$p" ]; then echo present; exit 0; fi; done; echo absent')
+guile_module_smoke=$(ssh_guest "env LANG='C.UTF-8' LC_ALL='C.UTF-8' LD_LIBRARY_PATH='$guile_extra_store/lib:$guile_store/lib:/usr/local/lib' GUILE_SYSTEM_PATH='$guile_store/share/guile/3.0:$guile_store/share/guile/site/3.0:$guile_store/share/guile/site:$guile_store/share/guile' GUILE_LOAD_PATH='$shepherd_store/share/guile/site/3.0:$guile_extra_store/share/guile/site/3.0' GUILE_SYSTEM_COMPILED_PATH='$guile_store/lib/guile/3.0/ccache:$guile_store/lib/guile/3.0/site-ccache' GUILE_LOAD_COMPILED_PATH='$shepherd_store/lib/guile/3.0/site-ccache:$guile_extra_store/lib/guile/3.0/site-ccache' GUILE_SYSTEM_EXTENSIONS_PATH='$guile_store/lib/guile/3.0/extensions' GUILE_EXTENSIONS_PATH='$guile_extra_store/lib/guile/3.0/extensions' '$guile_store/bin/guile' --no-auto-compile -c '(use-modules (fibers config) (gnutls) (shepherd config)) (display \"ok\") (newline)'")
 activate_preview=$(ssh_guest 'head -n 5 /run/current-system/activate' | tr '\n' ' ')
 
 [ "$ready_marker" = ready ] || { echo "unexpected ready marker contents: $ready_marker" >&2; exit 1; }
 [ "$shepherd_status" = running ] || { echo "fruix_shepherd is not running" >&2; exit 1; }
 [ "$sshd_status" = running ] || { echo "sshd is not running" >&2; exit 1; }
+[ "$compat_prefix_shims" = absent ] || { echo "compatibility prefix shims are still present in /tmp" >&2; exit 1; }
+[ "$guile_module_smoke" = ok ] || { echo "guest Guile module smoke failed: $guile_module_smoke" >&2; exit 1; }
 [ "$run_current_system_target" = "/frx/store/$closure_base" ] || {
   echo "unexpected /run/current-system target in guest: $run_current_system_target" >&2
   exit 1
@@ -189,6 +196,8 @@ sshd_status=$sshd_status
 logger_log=$logger_log
 uname_output=$uname_output
 operator_home_listing=$operator_home_listing
+compat_prefix_shims=$compat_prefix_shims
+guile_module_smoke=$guile_module_smoke
 activate_preview=$activate_preview
 boot_backend=xcp-ng-xo-cli
 operator_access=ssh-root-key