system: validate development environment overlay

docs/GUIX_DIFFERENCES.md

@@ -1,6 +1,6 @@
 # Fruix differences for Guix sysadmins
 
-Date: 2026-04-04
+Date: 2026-04-05
 
 This document is aimed at operators who already know Guix well and want a quick map of where Fruix behaves similarly and where it intentionally differs.
 
@@ -48,13 +48,12 @@ That path remains the active runtime boundary used by activation and service wir
 
 Fruix avoids in-place mutation of an older deployed closure.
 
-The validated rollback story today is:
+The validated rollback story now has two layers:
 
-- keep the earlier declaration
-- rebuild or rematerialize it
-- boot or redeploy that earlier closure again
+- declaration-level rollback by rebuilding/redeploying an earlier declaration
+- installed-system rollback between already-recorded generations on the target itself
 
-That is Guix-like in spirit even though Fruix does not yet expose the same installed-system rollback command surface.
+That is Guix-like in spirit, although Fruix still exposes a smaller installed-system workflow than Guix's more mature `reconfigure` model.
 
 ### Generation-style metadata and roots
 
@@ -78,7 +77,7 @@ Guix heavily reuses its profile-generation model and represents a lot of meaning
 
 Fruix keeps the **semantics** but uses a more explicit metadata-oriented layout for installed systems.
 
-Current Fruix layout:
+Current Fruix layout starts as:
 
 ```text
 /var/lib/fruix/system/
@@ -92,6 +91,20 @@ Current Fruix layout:
       install.scm
 ```
 
+After a validated installed-system switch, Fruix also records:
+
+```text
+/var/lib/fruix/system/
+  rollback -> generations/1
+  rollback-generation
+  generations/
+    2/
+      closure -> /frx/store/...-fruix-system-...
+      metadata.scm
+      provenance.scm
+      install.scm
+```
+
 Why Fruix does this:
 
 - it makes deployment state easier to inspect directly
@@ -154,27 +167,98 @@ Validated examples:
 
 So compared with Guix-on-Linux intuition, Fruix operators should be more explicit about target-device selection during installation and installer-media validation.
 
-## 6. Fruix does not yet have Guix-equivalent installed-system generation commands
+## 6. Fruix now has a minimal installed-system generation command surface, but it is still smaller than Guix's
 
-This is the biggest current operational gap.
+This remains the biggest operational gap, but it is no longer a complete gap.
 
-Fruix does **not** yet provide a mature equivalent of the familiar Guix System operator flow around in-place generation switching and rollback commands.
+Installed Fruix systems now provide a small in-guest helper:
 
-Today, Fruix rollback is mostly:
+- `fruix system status`
+- `fruix system switch /frx/store/...-fruix-system-...`
+- `fruix system rollback`
 
-- declaration-driven
-- rebuild/redeploy based
+What this gives you today:
 
-rather than:
+- explicit current-generation tracking
+- explicit rollback-generation tracking
+- in-place switching between already-staged closures on the installed target
+- rollback without reinstalling the whole system image again
 
-- switch current system generation in place through a dedicated command
+What it still does **not** give you yet compared with Guix:
 
-So if you come from Guix, assume that Fruix currently has:
+- a mature `reconfigure`-style workflow that builds and stages the new closure from inside the target system
+- automatic closure transfer/fetch as part of `switch`
+- the broader generation-management UX Guix operators expect
+
+So if you come from Guix, assume that Fruix now has:
 
 - strong closure/store semantics
 - explicit install artifacts
 - explicit generation metadata roots
-- but a less mature installed-system generation UX
+- a real but still modest installed-system switch/rollback UX
+
+## 7. Fruix keeps Guix-like store semantics, but not Guix/Nix hash-prefix machinery exactly
+
+Fruix still uses immutable store paths under:
+
+- `/frx/store`
+
+and it still treats a store path as a deployment identity boundary.
+
+But Fruix now intentionally differs from Guix/Nix in how the visible store-path prefix is constructed.
+
+Current Fruix policy is:
+
+- centralize store-path naming behind shared helpers
+- hash a small semantic identity record rather than copying Nix's historical path-hash formula exactly
+- include at least:
+  - object kind
+  - logical/display name
+  - output name
+  - payload or manifest identity
+  - hash-scheme version marker
+- truncate the visible SHA-256 prefix to **160 bits**
+- render that visible prefix as **40 hex characters**
+
+Why Fruix does this instead of copying Guix/Nix exactly:
+
+- the main goal was shorter store prefixes, not Nix compatibility for its own sake
+- distinct outputs should have distinct identities because `out`, `lib`, `debug`, and `doc` are semantically different artifacts
+- Fruix wanted one central policy point that can be swapped later without touching every materializer again
+- Fruix did **not** want to inherit Nix's legacy details unless they provide clear value here
+  - custom base32 alphabet and bit ordering
+  - compressed/XOR-folded path hashes
+  - exact historical `output:out` / `source` path-hash conventions
+
+So compared with Guix:
+
+- the important semantic property is the same:
+  - different store objects should get different immutable identities
+- the exact printable prefix algorithm is intentionally simpler in Fruix today
+
+For Guix-familiar operators, the practical takeaway is:
+
+- still think of `/frx/store/...` paths as immutable deployment identities
+- do **not** assume Fruix store prefixes are byte-for-byte comparable to Guix/Nix ones
+- expect Fruix to prefer a simpler, centralized naming policy unless exact Guix/Nix behavior becomes necessary later
+
+## 8. Fruix can expose a separate in-system development profile overlay
+
+For the validated Phase 20.1 path, Fruix can now expose development tooling separately from the main runtime profile.
+
+On those systems, Fruix exposes:
+
+- `/run/current-system/development-profile`
+- `/run/current-development`
+- `/usr/local/bin/fruix-development-environment`
+
+The intent is:
+
+- keep the main runtime profile lean
+- expose headers, `usr/share/mk`, and selected toolchain commands explicitly
+- avoid treating a development-heavy system image as the default runtime shape
+
+Compared with Guix, this is conceptually similar to keeping development-oriented state separate from the main runtime identity, but Fruix currently expresses it as a system-attached development overlay rather than through Guix's broader profile/tooling model.
 
 ## Where Fruix is intentionally trying to improve on Guix's representation
 
@@ -202,10 +286,11 @@ If you are already comfortable with Guix, the safest Fruix mental model today is
    - closure path
    - source provenance metadata
    - install metadata
-5. think of rollback today as:
-   - “redeploy the earlier declaration again”
-   rather than:
-   - “switch to an already-managed previous generation in place”
+5. think of rollback in two layers:
+   - if the target already has the desired closure staged locally:
+     - use `fruix system rollback`
+   - otherwise:
+     - redeploy the earlier declaration again
 
 ## Status summary
 
@@ -222,4 +307,4 @@ It differs most from Guix in:
 - source-provenance emphasis
 - installer-medium-oriented workflows
 - generation-layout representation
-- and the still-maturing installed-system generation command surface
+- and an installed-system generation command surface that now exists, but is still much smaller than Guix's

docs/PROGRESS.md

@@ -30,6 +30,14 @@ Fruix currently has:
   - `/var/lib/fruix/system`
 - explicit installed-system retention roots under:
   - `/frx/var/fruix/gcroots`
+- a validated installed-system generation switch/rollback workflow via:
+  - `fruix system status`
+  - `fruix system switch`
+  - `fruix system rollback`
+- a validated separate in-system development environment overlay via:
+  - `/run/current-system/development-profile`
+  - `/run/current-development`
+  - `/usr/local/bin/fruix-development-environment`
 
 Validated boot modes still are:
 
@@ -42,38 +50,37 @@ The validated Phase 18 installation work currently uses:
 
 ## Latest completed achievement
 
-### 2026-04-04 — Phase 19.2 completed
+### 2026-04-05 — Phase 20.1 completed
 
-Fruix now records an explicit installed-system generation layout and retention-root model instead of relying mainly on harness knowledge.
+Fruix now has a validated real-VM path where a booted Fruix-managed FreeBSD system exposes a separate development environment for native base work without collapsing the runtime/development split.
 
 Highlights:
 
-- added explicit installed-system generation layout under:
-  - `/var/lib/fruix/system`
-- added explicit installed-system retention roots under:
-  - `/frx/var/fruix/gcroots`
-- installed targets now record a first-generation deployment directory containing:
-  - `closure`
-  - `metadata.scm`
-  - `provenance.scm`
-  - `install.scm`
-- `/run/current-system` remains the runtime boundary and still points directly at the active closure path
-- added Guix-oriented operator notes in:
-  - `docs/GUIX_DIFFERENCES.md`
-- updated deployment workflow documentation to reflect the new explicit generation model
+- operating-system declarations now support:
+  - `#:development-packages`
+- system closures can now carry a separate development profile at:
+  - `/run/current-system/development-profile`
+  - `/run/current-development`
+- opt-in systems now ship an in-guest helper at:
+  - `/usr/local/bin/fruix-development-environment`
+- the validated Phase 20.1 guest path exposes:
+  - native headers
+  - `usr/share/mk` for `bsd.*.mk`
+  - Clang toolchain commands such as `cc`, `c++`, `ar`, `ranlib`, and `nm`
+- the validated guest workflow now supports:
+  - `eval "$(/usr/local/bin/fruix-development-environment)"`
+  - direct compilation with the Fruix-provided toolchain
+  - a simple `bsd.prog.mk` build on the running Fruix guest
 
 Validation:
 
-- `PASS phase19-generation-layout-qemu`
-- regression re-check:
-  - `PASS phase18-installer-iso`
+- `PASS phase20-development-environment-xcpng`
 
 Reports:
 
 - `docs/system-deployment-workflow.md`
 - `docs/GUIX_DIFFERENCES.md`
-- `docs/reports/phase19-deployment-workflow-freebsd.md`
-- `docs/reports/phase19-generation-layout-freebsd.md`
+- `docs/reports/phase20-development-environment-freebsd.md`
 
 ## Recent major milestones
 
@@ -99,6 +106,6 @@ Reports:
 
 Per `docs/PLAN_4.md`, the next planned step is:
 
-- **Phase 19.3** — validate installed-system rollback through the intended operator-facing workflow
+- **Phase 20.2** — run host-initiated native base builds inside a Fruix-managed environment
 
-Phase 19.2 is now complete: Fruix has an explicit installed-system generation layout and retention-root model on FreeBSD.
+Phase 20.1 is now complete: Fruix validates a separate in-system development environment for native FreeBSD base work on the approved real XCP-ng path.

docs/reports/phase19-installed-system-rollback-freebsd.md

@@ -0,0 +1,226 @@
+# Phase 19.3: installed-system rollback workflow on FreeBSD
+
+Date: 2026-04-04
+
+## Goal
+
+Phase 19.3 is about validating installed-system rollback through the intended operator-facing workflow, not only through host-side build/image redeploy harnesses.
+
+The key question was:
+
+- can an already-installed Fruix system move between recorded generations coherently, using an operator-facing command surface on the target itself?
+
+## Decision
+
+The current Fruix solution is intentionally modest.
+
+Fruix now provides a small installed-system helper on the target itself:
+
+- `/usr/local/bin/fruix`
+
+Validated in-guest commands:
+
+- `fruix system status`
+- `fruix system switch /frx/store/...-fruix-system-...`
+- `fruix system rollback`
+
+Important scope choice:
+
+- `switch` assumes the candidate closure is already present on the target's `/frx/store`
+- Fruix does **not** yet fetch or transfer that closure onto the target automatically
+
+That keeps Phase 19.3 focused on generation-state correctness rather than introducing a larger store-transfer story prematurely.
+
+## Implemented model
+
+Installed systems now support the following validated operator pattern:
+
+1. build a candidate closure with the host-side Fruix frontend
+2. stage that closure into the installed system's `/frx/store`
+3. run:
+   - `fruix system switch /frx/store/...candidate...`
+4. reboot into the candidate generation
+5. if needed, run:
+   - `fruix system rollback`
+6. reboot into the recorded rollback generation
+
+The installed system now records explicit rollback state under:
+
+```text
+/var/lib/fruix/system/
+  current -> generations/N
+  current-generation
+  rollback -> generations/M
+  rollback-generation
+  generations/
+    ...
+```
+
+and explicit rollback reachability under:
+
+```text
+/frx/var/fruix/gcroots/
+  current-system -> /frx/store/...current...
+  rollback-system -> /frx/store/...rollback...
+  system-1 -> ...
+  system-2 -> ...
+```
+
+## Code changes
+
+### `modules/fruix/system/freebsd/render.scm`
+
+Added a generated in-guest Fruix deployment helper script under:
+
+- `usr/local/bin/fruix`
+
+That helper now:
+
+- reports installed-system state with `fruix system status`
+- stages a new current generation with `fruix system switch`
+- stages the recorded rollback generation with `fruix system rollback`
+- updates:
+  - `/var/lib/fruix/system/current`
+  - `/var/lib/fruix/system/current-generation`
+  - `/var/lib/fruix/system/rollback`
+  - `/var/lib/fruix/system/rollback-generation`
+  - `/frx/var/fruix/gcroots/current-system`
+  - `/frx/var/fruix/gcroots/rollback-system`
+  - `/frx/var/fruix/gcroots/system-N`
+- refreshes the ESP bootloader file from the selected closure's `boot/loader.efi`
+
+A practical implementation detail mattered here:
+
+- replacing `/run/current-system` with a remove-then-recreate strategy caused the live shell environment to break while the link was absent
+- switching that update to an atomic symlink replacement path for `/run/current-system` avoided that gap and made the in-guest operator command reliable
+
+### `modules/fruix/system/freebsd/media.scm`
+
+Updated installed rootfs staging so that installed targets expose:
+
+- `/usr/local/bin/fruix -> /run/current-system/usr/local/bin/fruix`
+
+Also bumped the explicit generation-layout version from:
+
+- `1` to `2`
+
+because the installed-system model now includes operator-driven switch/rollback state as part of the validated layout story.
+
+### `modules/fruix/system/freebsd/model.scm`
+
+Updated generated-file metadata so the system closure records:
+
+- `usr/local/bin/fruix`
+
+as part of the generated operating-system file set.
+
+## New validation harness
+
+Added:
+
+- `tests/system/run-phase19-installed-system-rollback-qemu.sh`
+
+This harness validates the actual installed-system operator flow on local `QEMU/UEFI/TCG`.
+
+## Validation flow
+
+The harness now performs all of the following:
+
+1. installs a current system image directly to a target disk image
+2. builds a distinct candidate closure
+   - in the validated harness this differs by host name so the closure identity changes cleanly without needing a heavier base-version rebuild
+3. stages the candidate closure and its referenced store items into the installed target's `/frx/store`
+4. boots the installed current system
+5. validates initial state:
+   - current generation = `1`
+   - current closure = current installed closure
+   - no rollback generation yet recorded
+6. runs:
+   - `fruix system switch /frx/store/...candidate...`
+7. validates staged switch state:
+   - current generation = `2`
+   - rollback generation = `1`
+   - current closure = candidate closure
+   - rollback closure = original current closure
+   - generation 2 metadata/install files were written
+8. reboots and validates boot into the candidate closure
+9. runs:
+   - `fruix system rollback`
+10. validates staged rollback state:
+    - current generation = `1`
+    - rollback generation = `2`
+    - current closure = original closure
+    - rollback closure = candidate closure
+11. reboots and validates boot back into the original current system
+12. confirms post-rollback service state:
+    - `fruix-shepherd` running
+    - `sshd` running
+    - activation log still shows success
+
+## Passing validation
+
+Passing result:
+
+- `PASS phase19-installed-system-rollback-qemu`
+
+Validated metadata summary:
+
+```text
+current_closure_path=/frx/store/4debd106d62f14594ba1612e1e7105f1658bf5f4075d6e5db5436efeaf929d90-fruix-system-fruix-freebsd-current
+candidate_closure_path=/frx/store/54fb14e6071b8e5704a5dc75e2881c2f0533767771c26c4181f57afea88d1e8b-fruix-system-fruix-freebsd-canary
+current_host_name=fruix-freebsd-current
+candidate_host_name=fruix-freebsd-canary
+final_current_generation=1
+final_current_closure=/frx/store/4debd106d62f14594ba1612e1e7105f1658bf5f4075d6e5db5436efeaf929d90-fruix-system-fruix-freebsd-current
+final_rollback_generation=2
+final_rollback_closure=/frx/store/54fb14e6071b8e5704a5dc75e2881c2f0533767771c26c4181f57afea88d1e8b-fruix-system-fruix-freebsd-canary
+installed_system_switch=ok
+installed_system_rollback=ok
+```
+
+## Regression checks
+
+After landing the installed-system switch/rollback workflow, the following regression checks still pass:
+
+- `PASS phase19-generation-layout-qemu`
+- `PASS phase18-installer-iso`
+
+That means the new in-guest generation-management path did not regress:
+
+- the previously validated explicit generation layout
+- or the UEFI installer ISO boot/install path
+
+## Relationship to Guix
+
+This phase does **not** claim that Fruix now matches Guix's full installed-system UX.
+
+What Fruix now has is:
+
+- explicit generation state on disk
+- explicit current/rollback pointers
+- a minimal installed-system operator command surface
+- validated switching and rollback between already-staged closures
+
+What still remains compared with Guix:
+
+- building/staging the candidate closure from inside the target system itself
+- automatic closure transfer/fetch as part of `switch`
+- a richer long-term generation lifecycle policy
+
+## Conclusion
+
+Phase 19.3 is complete.
+
+Fruix now validates an actual installed-system rollback workflow on FreeBSD:
+
+- the target system itself can report current/rollback state
+- it can switch to a staged candidate generation
+- it can reboot into that candidate generation
+- it can roll back to the recorded prior generation
+- and it can reboot into the restored current system
+
+That closes the Phase 19 deployment story from:
+
+- documented deployment workflow
+- to explicit generation layout
+- to validated installed-system operator rollback behavior

docs/reports/phase20-development-environment-freebsd.md

@@ -0,0 +1,160 @@
+# Phase 20.1: Fruix-managed development environment for native FreeBSD base work
+
+Date: 2026-04-05
+
+## Goal
+
+Validate that a booted Fruix-managed FreeBSD system can expose a usable development environment for deeper native base work without collapsing the runtime/development boundary back into one broad profile.
+
+This step explicitly builds on the Phase 14 split between:
+
+- native runtime/boot artifacts for the running system
+- separate development-facing artifacts such as headers and toolchain pieces
+
+The goal is **not** full self-hosting yet.
+
+It is to prove that a running Fruix system can expose the tools and paths needed for native FreeBSD build work in a controlled way.
+
+## Implementation
+
+### New operating-system field
+
+`modules/fruix/system/freebsd/model.scm` now supports:
+
+- `#:development-packages`
+- `operating-system-development-packages`
+
+The default remains empty, so existing systems do not change unless they opt in.
+
+### Separate development profile inside the system closure
+
+`modules/fruix/system/freebsd/media.scm` now materializes an additional profile tree when `development-packages` is non-empty:
+
+- `/frx/store/...-fruix-system-.../development-profile`
+
+The main runtime tree remains:
+
+- `/frx/store/...-fruix-system-.../profile`
+
+This preserves the runtime/development split:
+
+- runtime stays under `profile`
+- development tooling stays under `development-profile`
+
+The development stores are also now part of the closure references and recorded in `metadata/store-layout.scm`.
+
+### In-guest development environment helper
+
+Opt-in systems with development packages now ship:
+
+- `/usr/local/bin/fruix-development-environment`
+
+and expose a stable runtime link:
+
+- `/run/current-development -> /run/current-system/development-profile`
+
+The helper emits shell exports for the active development profile, including at least:
+
+- `FRUIX_DEVELOPMENT_PROFILE`
+- `FRUIX_DEVELOPMENT_INCLUDE`
+- `FRUIX_DEVELOPMENT_SHARE_MK`
+- `FRUIX_CC`
+- `FRUIX_CXX`
+- `FRUIX_AR`
+- `FRUIX_RANLIB`
+- `FRUIX_NM`
+- `FRUIX_BMAKE`
+- `CPPFLAGS`
+- `MAKEFLAGS`
+- `PATH`
+
+Intended use:
+
+```sh
+eval "$(/usr/local/bin/fruix-development-environment)"
+```
+
+### Chosen development overlay for this phase
+
+For the validated Phase 20.1 guest path, the development overlay is intentionally narrow:
+
+- `freebsd-native-headers`
+- `freebsd-clang-toolchain`
+
+This was chosen deliberately.
+
+The earlier standalone Phase 14 development-profile package set was designed for broad profile composition, but for a running Fruix system it unnecessarily duplicated runtime pieces already present in the system profile.
+
+For native base work, the important additions here are:
+
+- `usr/include`
+- `usr/share/mk`
+- Clang/binutils-style frontend tools
+
+while the running system already supplies:
+
+- base runtime
+- `/usr/bin/make`
+- system libraries and boot/runtime state
+
+That keeps the Phase 20.1 environment focused on native base development rather than reintroducing a broad mixed profile.
+
+## New files
+
+Added:
+
+- `tests/system/phase20-development-operating-system.scm.in`
+- `tests/system/run-phase20-development-environment-xcpng.sh`
+
+## Validation
+
+Passing run:
+
+- `PASS phase20-development-environment-xcpng`
+- workdir: `/tmp/fruix-phase20-development-xcpng`
+
+Validated on the approved real XCP-ng path:
+
+- VM `90490f2e-e8fc-4b7a-388e-5c26f0157289`
+- VDI `0f1f90d3-48ca-4fa2-91d8-fc6339b95743`
+
+Representative result:
+
+```text
+closure_path=/frx/store/c0ad43d7ef72323d4270a4f1e96ca1f5cc99566c-fruix-system-fruix-freebsd
+development_profile_path=/frx/store/c0ad43d7ef72323d4270a4f1e96ca1f5cc99566c-fruix-system-fruix-freebsd/development-profile
+development_profile_guest=/run/current-system/development-profile
+cc_version=FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)
+hello_direct=hello-from-direct-dev-env
+hello_make=hello-from-make-dev-env
+development_environment=ok
+```
+
+The harness verified all of the following on the real booted Fruix guest:
+
+- runtime boot/regression still succeeds on XCP-ng
+- the closure contains a separate `development-profile`
+- runtime `profile` does **not** regain headers or `usr/share/mk`
+- `/usr/local/bin/fruix-development-environment` exists and emits the expected exports
+- `/run/current-development` points at `/run/current-system/development-profile`
+- direct compilation works with the exported Clang toolchain and headers
+- a simple native FreeBSD `make` build using `.include <bsd.prog.mk>` also succeeds
+
+## Result
+
+Phase 20.1 is complete.
+
+Fruix now validates a real booted FreeBSD system path where:
+
+- the running system remains lean and runtime-focused
+- native development artifacts are exposed separately and explicitly
+- the guest can compile code directly with the Fruix-provided toolchain
+- the guest can also drive a simple `bsd.prog.mk` build using the exported development environment
+
+This is enough to say that Fruix can host a controlled native FreeBSD base-development environment, without yet claiming full self-hosting.
+
+## Next step
+
+Per `docs/PLAN_4.md`, the next planned step is:
+
+- **Phase 20.2** — run host-initiated native base builds inside a Fruix-managed environment

docs/system-deployment-workflow.md

@@ -1,6 +1,6 @@
 # Fruix system deployment workflow
 
-Date: 2026-04-04
+Date: 2026-04-05
 
 ## Purpose
 
@@ -11,9 +11,10 @@ This document defines the current canonical Fruix workflow for:
 - installing a declarative system onto an image or disk
 - booting through installer media
 - rolling forward to a candidate system
-- rolling back to an earlier declared system
+- switching an installed system to a staged candidate generation
+- rolling an installed system back to an earlier recorded generation
 
-This is the Phase 19.1 operator-facing view of the system model already implemented in earlier phases.
+This is now the Phase 19 operator-facing view of the system model as validated through explicit installed-system generation switching and rollback.
 
 ## Core model
 
@@ -168,6 +169,60 @@ Use this when you want to:
 - install from an ISO-attached Fruix environment
 - test the same install model on more realistic VM paths
 
+### Installed-system generation commands
+
+Installed Fruix systems now also ship a small in-guest deployment helper at:
+
+- `/usr/local/bin/fruix`
+
+Current validated in-guest commands are:
+
+```sh
+fruix system status
+fruix system switch /frx/store/...-fruix-system-...
+fruix system rollback
+```
+
+Current intended usage:
+
+1. build a candidate closure on the operator side with `./bin/fruix system build`
+2. ensure that candidate closure is present on the installed target's `/frx/store`
+3. run `fruix system switch /frx/store/...` on the installed system
+4. reboot into the staged candidate generation
+5. if needed, run `fruix system rollback`
+6. reboot back into the recorded rollback generation
+
+Important current limitation:
+
+- `fruix system switch` does **not** yet fetch or copy the candidate closure onto the target for you
+- it assumes the selected closure is already present in the installed system's `/frx/store`
+
+### In-guest development environment
+
+Opt-in systems can also expose a separate development overlay under:
+
+- `/run/current-system/development-profile`
+- `/run/current-development`
+
+Those systems now ship a helper at:
+
+- `/usr/local/bin/fruix-development-environment`
+
+Intended use:
+
+```sh
+eval "$(/usr/local/bin/fruix-development-environment)"
+```
+
+That helper exports a development-oriented environment while keeping the main runtime profile separate.  The validated Phase 20.1 path currently uses this to expose at least:
+
+- native headers under `usr/include`
+- FreeBSD `share/mk` files for `bsd.*.mk`
+- Clang toolchain commands such as `cc`, `c++`, `ar`, `ranlib`, and `nm`
+- `MAKEFLAGS` pointing at the development profile's `usr/share/mk`
+
+This is the current Fruix-native way to make a running system suitable for controlled native base-development work without merging development content back into the main runtime profile.
+
 ## Deployment patterns
 
 ### 1. Build-first workflow
@@ -261,7 +316,7 @@ Installed Fruix systems now record an explicit first-generation deployment layou
 
 - `/var/lib/fruix/system`
 
-Current validated shape:
+Initial installed shape:
 
 ```text
 /var/lib/fruix/system/
@@ -275,6 +330,24 @@ Current validated shape:
       install.scm   # present on installed targets
 ```
 
+After a validated in-place switch, the layout extends to:
+
+```text
+/var/lib/fruix/system/
+  current -> generations/2
+  current-generation
+  rollback -> generations/1
+  rollback-generation
+  generations/
+    1/
+      ...
+    2/
+      closure -> /frx/store/...-fruix-system-...
+      metadata.scm
+      provenance.scm
+      install.scm   # deployment metadata for the switch operation
+```
+
 Installed systems also now create explicit GC-root-style deployment links under:
 
 - `/frx/var/fruix/gcroots`
@@ -284,7 +357,9 @@ Current validated shape:
 ```text
 /frx/var/fruix/gcroots/
   current-system -> /frx/store/...-fruix-system-...
+  rollback-system -> /frx/store/...-fruix-system-...
   system-1 -> /frx/store/...-fruix-system-...
+  system-2 -> /frx/store/...-fruix-system-...
 ```
 
 Important detail:
@@ -294,7 +369,9 @@ Important detail:
 
 ## Roll-forward workflow
 
-The current Fruix roll-forward model is declaration-driven.
+The current Fruix roll-forward model now has two validated layers.
+
+### Declaration/deployment roll-forward
 
 Canonical process:
 
@@ -314,47 +391,61 @@ Canonical process:
 5. boot or install the candidate
 6. validate the candidate closure in the booted system
 
-The important property is that the candidate closure appears beside the earlier one in `/frx/store` rather than mutating it in place.
+### Installed-system generation roll-forward
+
+When the candidate closure is already present on an installed target:
+
+1. run `fruix system switch /frx/store/...candidate...`
+2. confirm the staged state with `fruix system status`
+3. reboot into the candidate generation
+4. validate the new active closure after reboot
+
+The important property is still that the candidate closure appears beside the earlier one in `/frx/store` rather than mutating it in place.
 
 ## Rollback workflow
 
-The current canonical rollback workflow is also declaration-driven.
+The current canonical rollback workflow also now has two validated layers.
 
-Today, rollback means:
+### Declaration/deployment rollback
+
+You can still roll back by redeploying the earlier declaration:
 
 1. retain the earlier declaration that produced the known-good closure
 2. rebuild or rematerialize that earlier declaration
 3. redeploy or reboot that earlier artifact again
 
-Concretely, the usual rollback choices are:
+Concretely, the usual declaration-level rollback choices are:
 
 - rebuild the earlier declaration with `fruix system build` and confirm the old closure path reappears
 - boot the earlier declaration again through `fruix system image`
 - reinstall the earlier declaration through `fruix system install`, `installer`, or `installer-iso` if the deployment medium itself must change
 
-This rollback story has already been validated at the closure/image/deployment level:
+### Installed-system generation rollback
 
-- side-by-side base-version coexistence in `/frx/store`
-- roll-forward to a candidate closure
-- rollback by rebuilding and booting the earlier declaration again
-- validation on both local QEMU and the approved XCP-ng VM path
+When an installed target already has both the current and rollback generations recorded:
+
+1. run `fruix system rollback`
+2. confirm the staged state with `fruix system status`
+3. reboot into the rollback generation
+4. validate the restored active closure after reboot
+
+This installed-system rollback path is now validated on local `QEMU/UEFI/TCG`.
 
 ### Important scope note
 
-This is not yet the same thing as a first-class installed-system generation switch command.
+This is still not yet the same thing as Guix's full `reconfigure`/generation UX.
 
-Current rollback is:
+Current installed-system rollback is intentionally modest:
 
-- **redeploy the earlier declaration again**
-
-What still remains for later Phase 19 work is making rollback itself operator-driven at the installed-system layer, rather than only declaration/redeploy driven.
+- it switches between already-recorded generations on the target
+- it does not yet fetch candidate closures onto the machine for you
+- it does not yet expose a richer history-management or generation-pruning policy
 
 Still pending:
 
-- previous-generation tracking beyond the initial explicit generation-1 layout
-- an explicit rollback target link distinct from `current`
-- an operator-facing installed-system rollback workflow
-- generation switching without full redeploy
+- operator-facing closure transfer or fetch onto installed systems
+- multi-generation lifecycle policy beyond the validated `current` and `rollback` pointers
+- a fuller `reconfigure`-style installed-system UX
 
 ## Provenance and deployment identity
 
@@ -375,16 +466,16 @@ Operators should retain metadata from successful candidate and current deploymen
 
 ## Current limitations
 
-The deployment workflow is now coherent, but it is not yet the final generation-management story.
+The deployment workflow is now coherent, and Fruix now has a validated installed-system switch/rollback path, but it is still not the final generation-management story.
 
 Not yet first-class:
 
-- a dedicated `switch` or `reconfigure` command
-- an installed-system rollback command that moves between generations in place
-- multi-generation retention and previous-generation tracking beyond generation 1
-- generation switching policy independent of full redeploy
+- host-side closure transfer/fetch onto installed systems as part of `fruix system switch`
+- a fuller `reconfigure` workflow that builds and stages the new closure from inside the target environment
+- multi-generation lifecycle policy beyond the validated `current` and `rollback` pointers
+- generation pruning and retention policy independent of full redeploy
 
-Those are the next logical steps after the current explicit-generation layout.
+Those are the next logical steps after the current explicit-generation switch/rollback model.
 
 ## Summary
 
@@ -395,6 +486,8 @@ The current canonical Fruix deployment model is:
 - **materialize** the artifact appropriate to the deployment target
 - **boot or install** that artifact
 - **identify deployments by closure path and provenance metadata**
-- **roll back by rebuilding/redeploying the earlier declaration**, not by mutating the current closure in place
+- on installed systems, **switch** to a staged candidate with `fruix system switch`
+- on installed systems, **roll back** to the recorded rollback generation with `fruix system rollback`
+- still use declaration/redeploy rollback when the target does not already have the desired closure staged locally
 
-That is the operator-facing workflow Fruix should document and use while installed-system generation switching remains more limited than Guix's mature in-place system-generation workflow.
+That is the operator-facing workflow Fruix should document and use while its installed-system generation UX remains simpler than Guix's mature in-place system-generation workflow.

modules/fruix/system/freebsd.scm

@@ -31,6 +31,7 @@
                operating-system-kernel
                operating-system-bootloader
                operating-system-base-packages
+               operating-system-development-packages
                operating-system-users
                operating-system-groups
                operating-system-file-systems

modules/fruix/system/freebsd/build.scm

@@ -364,11 +364,11 @@
          (cached (hash-ref cache cache-key #f)))
     (if cached
         cached
-        (let* ((hash (sha256-string manifest))
-               (output-path (string-append store-dir "/" hash "-"
-                                           (freebsd-package-name prepared-package)
-                                           "-"
-                                           (freebsd-package-version prepared-package))))
+        (let* ((display-name (string-append (freebsd-package-name prepared-package)
+                                            "-"
+                                            (freebsd-package-version prepared-package)))
+               (output-path (make-store-path store-dir display-name manifest
+                                             #:kind 'freebsd-package)))
           (unless (file-exists? output-path)
             (case (freebsd-package-build-system prepared-package)
               ((copy-build-system)
@@ -454,8 +454,9 @@
 
 (define* (materialize-prefix source-path name version store-dir #:key (extra-files '()))
   (let* ((manifest (prefix-manifest-string source-path extra-files))
-         (hash (sha256-string manifest))
-         (output-path (string-append store-dir "/" hash "-" name "-" version)))
+         (display-name (string-append name "-" version))
+         (output-path (make-store-path store-dir display-name manifest
+                                       #:kind 'prefix)))
     (unless (file-exists? output-path)
       (mkdir-p output-path)
       (for-each (lambda (entry)

modules/fruix/system/freebsd/media.scm

@@ -79,11 +79,16 @@
          (kernel-package (operating-system-kernel os))
          (bootloader-package (operating-system-bootloader os))
          (base-packages (operating-system-base-packages os))
+         (development-packages (operating-system-development-packages os))
          (kernel-store (materialize-freebsd-package kernel-package store-dir cache source-cache))
          (bootloader-store (materialize-freebsd-package bootloader-package store-dir cache source-cache))
          (base-package-stores (map (lambda (package)
                                      (materialize-freebsd-package package store-dir cache source-cache))
                                    base-packages))
+         (development-package-stores
+          (map (lambda (package)
+                 (materialize-freebsd-package package store-dir cache source-cache))
+               development-packages))
          (base-package-pairs (map cons base-packages base-package-stores))
          (store-classification
           (append (list (cons kernel-package kernel-store)
@@ -148,6 +153,8 @@
                    (host-base-stores . ,host-base-stores)
                    (native-base-store-count . ,(length native-base-stores))
                    (native-base-stores . ,native-base-stores)
+                   (development-package-store-count . ,(length development-package-stores))
+                   (development-package-stores . ,development-package-stores)
                    (fruix-runtime-store-count . ,(length fruix-runtime-stores))
                    (fruix-runtime-stores . ,fruix-runtime-stores)
                    (host-base-replacement-order . ,%freebsd-host-staged-replacement-order)
@@ -161,7 +168,12 @@
                                      . ,(render-activation-rc-script))
                                     ("usr/local/etc/rc.d/fruix-shepherd"
                                      . ,(render-rc-script shepherd-store guile-store guile-extra-store)))))
-         (references (delete-duplicates (append materialized-source-stores host-base-stores native-base-stores fruix-runtime-stores)))
+         (references (delete-duplicates
+                      (append materialized-source-stores
+                              host-base-stores
+                              native-base-stores
+                              development-package-stores
+                              fruix-runtime-stores)))
          (manifest (string-append
                     "closure-spec=\n"
                     (object->string (operating-system-closure-spec os))
@@ -172,9 +184,12 @@
                                  "\n")
                     "\nreferences=\n"
                     (string-join references "\n")))
-         (hash (sha256-string manifest))
-         (closure-path (string-append store-dir "/" hash "-fruix-system-"
-                                      (operating-system-host-name os))))
+         (display-name (string-append "fruix-system-"
+                                      (operating-system-host-name os)))
+         (closure-path (make-store-path store-dir display-name manifest
+                                        #:kind 'operating-system))
+         (development-profile-path (and (not (null? development-package-stores))
+                                        (string-append closure-path "/development-profile"))))
     (unless (file-exists? closure-path)
       (mkdir-p closure-path)
       (mkdir-p (string-append closure-path "/boot/kernel"))
@@ -192,6 +207,11 @@
       (for-each (lambda (output)
                   (merge-output-into-tree output (string-append closure-path "/profile")))
                 base-package-stores)
+      (when development-profile-path
+        (mkdir-p development-profile-path)
+        (for-each (lambda (output)
+                    (merge-output-into-tree output development-profile-path))
+                  development-package-stores))
       (for-each
        (lambda (entry)
          (write-file (string-append closure-path "/" (car entry)) (cdr entry)))
@@ -201,6 +221,10 @@
         (chmod (string-append closure-path "/etc/master.passwd") #o600))
       (chmod (string-append closure-path "/usr/local/etc/rc.d/fruix-activate") #o555)
       (chmod (string-append closure-path "/usr/local/etc/rc.d/fruix-shepherd") #o555)
+      (when (file-exists? (string-append closure-path "/usr/local/bin/fruix"))
+        (chmod (string-append closure-path "/usr/local/bin/fruix") #o555))
+      (when (file-exists? (string-append closure-path "/usr/local/bin/fruix-development-environment"))
+        (chmod (string-append closure-path "/usr/local/bin/fruix-development-environment") #o555))
       (when (file-exists? (string-append closure-path "/boot/fruix-pid1"))
         (chmod (string-append closure-path "/boot/fruix-pid1") #o555))
       (write-file (string-append closure-path "/parameters.scm")
@@ -215,6 +239,8 @@
       (guile-extra-store . ,guile-extra-store)
       (shepherd-store . ,shepherd-store)
       (base-package-stores . ,base-package-stores)
+      (development-package-stores . ,development-package-stores)
+      (development-profile-path . ,development-profile-path)
       (host-base-stores . ,host-base-stores)
       (native-base-stores . ,native-base-stores)
       (fruix-runtime-stores . ,fruix-runtime-stores)
@@ -233,7 +259,7 @@
   (mkdir-p (dirname link-name))
   (symlink target link-name))
 
-(define system-generation-layout-version "1")
+(define system-generation-layout-version "2")
 
 (define* (system-generation-metadata-object os closure-path
                                             #:key
@@ -308,8 +334,9 @@
   (mkdir-p rootfs)
   (for-each (lambda (dir)
               (mkdir-p (string-append rootfs dir)))
-            '("/run" "/boot" "/etc" "/etc/ssh" "/usr" "/usr/share" "/usr/local" "/usr/local/etc"
-              "/usr/local/etc/rc.d" "/var" "/var/cron" "/var/db" "/var/lib" "/var/lib/fruix"
+            '("/run" "/boot" "/etc" "/etc/ssh" "/usr" "/usr/share" "/usr/local"
+              "/usr/local/bin" "/usr/local/etc" "/usr/local/etc/rc.d" "/var"
+              "/var/cron" "/var/db" "/var/lib" "/var/lib/fruix"
               "/var/log" "/var/run" "/tmp" "/dev" "/root" "/home"))
   (chmod (string-append rootfs "/tmp") #o1777)
   (symlink-force closure-path (string-append rootfs "/run/current-system"))
@@ -345,6 +372,14 @@
               (symlink-force (string-append "/run/current-system/boot/" path)
                              (string-append rootfs "/boot/" path)))
             '("kernel" "loader" "loader.efi" "device.hints" "defaults" "lua" "loader.conf"))
+  (symlink-force "/run/current-system/usr/local/bin/fruix"
+                 (string-append rootfs "/usr/local/bin/fruix"))
+  (when (file-exists? (string-append closure-path "/development-profile"))
+    (symlink-force "/run/current-system/development-profile"
+                   (string-append rootfs "/run/current-development")))
+  (when (file-exists? (string-append closure-path "/usr/local/bin/fruix-development-environment"))
+    (symlink-force "/run/current-system/usr/local/bin/fruix-development-environment"
+                   (string-append rootfs "/usr/local/bin/fruix-development-environment")))
   (symlink-force "/run/current-system/usr/local/etc/rc.d/fruix-activate"
                  (string-append rootfs "/usr/local/etc/rc.d/fruix-activate"))
   (symlink-force "/run/current-system/usr/local/etc/rc.d/fruix-shepherd"
@@ -865,9 +900,10 @@
                     "\nstore-items=\n"
                     (string-join store-items "\n")
                     "\n"))
-         (hash (sha256-string manifest))
-         (image-store-path (string-append store-dir "/" hash "-fruix-bhyve-image-"
-                                          (operating-system-host-name os)))
+         (display-name (string-append "fruix-bhyve-image-"
+                                      (operating-system-host-name os)))
+         (image-store-path (make-store-path store-dir display-name manifest
+                                            #:kind 'bhyve-image))
          (disk-image (string-append image-store-path "/disk.img"))
          (esp-image (string-append image-store-path "/esp.img"))
          (root-image (string-append image-store-path "/root.ufs")))
@@ -1019,9 +1055,10 @@
                     "\ninstall-metadata=\n"
                     (object->string install-metadata)
                     "\n"))
-         (hash (sha256-string manifest))
-         (image-store-path (string-append store-dir "/" hash "-fruix-installer-image-"
-                                          (operating-system-host-name installer-os)))
+         (display-name (string-append "fruix-installer-image-"
+                                      (operating-system-host-name installer-os)))
+         (image-store-path (make-store-path store-dir display-name manifest
+                                            #:kind 'installer-image))
          (disk-image (string-append image-store-path "/disk.img"))
          (esp-image (string-append image-store-path "/esp.img"))
          (root-image (string-append image-store-path "/root.ufs")))
@@ -1324,9 +1361,10 @@
                     "\ninstall-metadata=\n"
                     (object->string install-metadata)
                     "\n"))
-         (hash (sha256-string manifest))
-         (iso-store-path (string-append store-dir "/" hash "-fruix-installer-iso-"
-                                        (operating-system-host-name installer-os)))
+         (display-name (string-append "fruix-installer-iso-"
+                                      (operating-system-host-name installer-os)))
+         (iso-store-path (make-store-path store-dir display-name manifest
+                                          #:kind 'installer-iso))
          (iso-image (string-append iso-store-path "/installer.iso"))
          (boot-efi-image (string-append iso-store-path "/efiboot.img"))
          (root-image (string-append iso-store-path "/root.img")))

modules/fruix/system/freebsd/model.scm

@@ -33,6 +33,7 @@
             operating-system-kernel
             operating-system-bootloader
             operating-system-base-packages
+            operating-system-development-packages
             operating-system-users
             operating-system-groups
             operating-system-file-systems
@@ -95,8 +96,8 @@
   (make-file-system device mount-point type options needed-for-boot?))
 
 (define-record-type <operating-system>
-  (make-operating-system host-name freebsd-base kernel bootloader base-packages users groups
-                         file-systems services loader-entries rc-conf-entries
+  (make-operating-system host-name freebsd-base kernel bootloader base-packages development-packages
+                         users groups file-systems services loader-entries rc-conf-entries
                          init-mode ready-marker root-authorized-keys)
   operating-system?
   (host-name operating-system-host-name)
@@ -104,6 +105,7 @@
   (kernel operating-system-kernel)
   (bootloader operating-system-bootloader)
   (base-packages operating-system-base-packages)
+  (development-packages operating-system-development-packages)
   (users operating-system-users)
   (groups operating-system-groups)
   (file-systems operating-system-file-systems)
@@ -120,6 +122,7 @@
                            (kernel freebsd-kernel)
                            (bootloader freebsd-bootloader)
                            (base-packages %freebsd-system-packages)
+                           (development-packages '())
                            (users (list (user-account #:name "root"
                                                       #:uid 0
                                                       #:group "wheel"
@@ -161,8 +164,8 @@
                            (init-mode 'freebsd-init+rc.d-shepherd)
                            (ready-marker "/var/lib/fruix/ready")
                            (root-authorized-keys '()))
-  (make-operating-system host-name freebsd-base kernel bootloader base-packages users groups
-                         file-systems services loader-entries rc-conf-entries
+  (make-operating-system host-name freebsd-base kernel bootloader base-packages development-packages
+                         users groups file-systems services loader-entries rc-conf-entries
                          init-mode ready-marker root-authorized-keys))
 
 (define default-minimal-operating-system (operating-system))
@@ -231,6 +234,8 @@
 (define (validate-operating-system os)
   (let* ((host-name (operating-system-host-name os))
          (base (operating-system-freebsd-base os))
+         (base-packages (operating-system-base-packages os))
+         (development-packages (operating-system-development-packages os))
          (users (operating-system-users os))
          (groups (operating-system-groups os))
          (file-systems (operating-system-file-systems os))
@@ -242,6 +247,10 @@
       (error "operating-system host-name must not be empty"))
     (unless (freebsd-base? base)
       (error "operating-system freebsd-base must be a <freebsd-base> record"))
+    (unless (every freebsd-package? base-packages)
+      (error "operating-system base-packages must be a list of <freebsd-package> records"))
+    (unless (every freebsd-package? development-packages)
+      (error "operating-system development-packages must be a list of <freebsd-package> records"))
     (validate-freebsd-source (freebsd-base-source base))
     (let ((dups (duplicate-elements user-names)))
       (unless (null? dups)
@@ -295,7 +304,11 @@
      "metadata/host-base-provenance.scm"
      "metadata/store-layout.scm"
      "activate"
-     "shepherd/init.scm")
+     "shepherd/init.scm"
+     "usr/local/bin/fruix")
+   (if (null? (operating-system-development-packages os))
+       '()
+       '("usr/local/bin/fruix-development-environment"))
    (if (pid1-init-mode? os)
        '("boot/fruix-pid1")
        '())
@@ -315,6 +328,8 @@
     (bootloader-package . ,(freebsd-package-name (operating-system-bootloader os)))
     (base-package-count . ,(length (operating-system-base-packages os)))
     (base-packages . ,(package-names (operating-system-base-packages os)))
+    (development-package-count . ,(length (operating-system-development-packages os)))
+    (development-packages . ,(package-names (operating-system-development-packages os)))
     (user-count . ,(length (operating-system-users os)))
     (users . ,(map user-account-name (operating-system-users os)))
     (group-count . ,(length (operating-system-groups os)))

modules/fruix/system/freebsd/render.scm

@@ -467,6 +467,324 @@
      "}\n\n"
      "load_rc_config $name\n"
      "run_rc_command \"$1\"\n")))
+(define (render-installed-system-fruix os)
+  (string-append
+   "#!/bin/sh\n"
+   "set -eu\n"
+   "PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin\n"
+   "tool_closure=$(readlink /run/current-system 2>/dev/null || true)\n"
+   "if [ -n \"$tool_closure\" ]; then\n"
+   "  PATH=\"$tool_closure/profile/bin:$tool_closure/profile/sbin:$tool_closure/profile/usr/bin:$tool_closure/profile/usr/sbin:$PATH\"\n"
+   "fi\n"
+   "export PATH\n\n"
+   "system_root=/var/lib/fruix/system\n"
+   "generations_root=\"$system_root/generations\"\n"
+   "current_link=\"$system_root/current\"\n"
+   "current_generation_file=\"$system_root/current-generation\"\n"
+   "rollback_link=\"$system_root/rollback\"\n"
+   "rollback_generation_file=\"$system_root/rollback-generation\"\n"
+   "gcroots_root=/frx/var/fruix/gcroots\n"
+   "run_current_link=/run/current-system\n"
+   "layout_version=2\n"
+   "host_name='" (operating-system-host-name os) "'\n"
+   "ready_marker='" (operating-system-ready-marker os) "'\n"
+   "init_mode='" (symbol->string (operating-system-init-mode os)) "'\n\n"
+   "usage()\n"
+   "{\n"
+   "  cat <<'EOF'\n"
+   "Usage: fruix system status\n"
+   "       fruix system switch /frx/store/...-fruix-system-...\n"
+   "       fruix system rollback\n"
+   "EOF\n"
+   "}\n\n"
+   "die()\n"
+   "{\n"
+   "  echo \"fruix: $*\" >&2\n"
+   "  exit 1\n"
+   "}\n\n"
+   "read_link_maybe()\n"
+   "{\n"
+   "  if [ -L \"$1\" ]; then\n"
+   "    readlink \"$1\"\n"
+   "  fi\n"
+   "}\n\n"
+   "read_file_maybe()\n"
+   "{\n"
+   "  if [ -f \"$1\" ]; then\n"
+   "    tr -d '\\n' < \"$1\"\n"
+   "  fi\n"
+   "}\n\n"
+   "symlink_force()\n"
+   "{\n"
+   "  target=$1\n"
+   "  link_name=$2\n"
+   "  tmp_link=\"${link_name}.new.$$\"\n"
+   "  mkdir -p \"$(dirname \"$link_name\")\"\n"
+   "  if [ \"$link_name\" = \"$run_current_link\" ]; then\n"
+   "    rm -f \"$tmp_link\"\n"
+   "    ln -s \"$target\" \"$tmp_link\"\n"
+   "    mv -h -f \"$tmp_link\" \"$link_name\"\n"
+   "  else\n"
+   "    rm -f \"$link_name\"\n"
+   "    ln -s \"$target\" \"$link_name\"\n"
+   "  fi\n"
+   "}\n\n"
+   "validate_closure()\n"
+   "{\n"
+   "  closure=$1\n"
+   "  [ -d \"$closure\" ] || die \"missing closure directory: $closure\"\n"
+   "  [ -f \"$closure/activate\" ] || die \"closure is missing activate script: $closure\"\n"
+   "  [ -f \"$closure/shepherd/init.scm\" ] || die \"closure is missing shepherd config: $closure\"\n"
+   "  [ -f \"$closure/boot/loader.efi\" ] || die \"closure is missing loader.efi: $closure\"\n"
+   "}\n\n"
+   "max_generation_number()\n"
+   "{\n"
+   "  max=0\n"
+   "  if [ -d \"$generations_root\" ]; then\n"
+   "    for path in \"$generations_root\"/*; do\n"
+   "      [ -d \"$path\" ] || continue\n"
+   "      base=$(basename \"$path\")\n"
+   "      case \"$base\" in\n"
+   "        ''|*[!0-9]*)\n"
+   "          continue\n"
+   "          ;;\n"
+   "      esac\n"
+   "      if [ \"$base\" -gt \"$max\" ]; then\n"
+   "        max=$base\n"
+   "      fi\n"
+   "    done\n"
+   "  fi\n"
+   "  printf '%s\\n' \"$max\"\n"
+   "}\n\n"
+   "next_generation_number()\n"
+   "{\n"
+   "  max=$(max_generation_number)\n"
+   "  printf '%s\\n' $((max + 1))\n"
+   "}\n\n"
+   "write_generation_metadata()\n"
+   "{\n"
+   "  generation=$1\n"
+   "  closure=$2\n"
+   "  action=$3\n"
+   "  previous_generation=$4\n"
+   "  previous_closure=$5\n"
+   "  generation_dir=\"$generations_root/$generation\"\n"
+   "  install_metadata_path=\"/var/lib/fruix/system/generations/$generation/install.scm\"\n"
+   "  cat > \"$generation_dir/metadata.scm\" <<EOF\n"
+   "((system-generation-version . \"$layout_version\")\n"
+   " (generation-number . $generation)\n"
+   " (host-name . \"$host_name\")\n"
+   " (ready-marker . \"$ready_marker\")\n"
+   " (init-mode . $init_mode)\n"
+   " (closure-path . \"$closure\")\n"
+   " (parameters-file . \"$closure/parameters.scm\")\n"
+   " (freebsd-base-file . \"$closure/metadata/freebsd-base.scm\")\n"
+   " (freebsd-source-file . \"$closure/metadata/freebsd-source.scm\")\n"
+   " (freebsd-source-materializations-file . \"$closure/metadata/freebsd-source-materializations.scm\")\n"
+   " (host-base-provenance-file . \"$closure/metadata/host-base-provenance.scm\")\n"
+   " (store-layout-file . \"$closure/metadata/store-layout.scm\")\n"
+   " (install-metadata-path . \"$install_metadata_path\")\n"
+   " (deployment-action . \"$action\")\n"
+   " (previous-generation-number . \"$previous_generation\")\n"
+   " (previous-closure-path . \"$previous_closure\"))\n"
+   "EOF\n"
+   "  chmod 644 \"$generation_dir/metadata.scm\"\n"
+   "}\n\n"
+   "write_generation_provenance()\n"
+   "{\n"
+   "  generation=$1\n"
+   "  closure=$2\n"
+   "  generation_dir=\"$generations_root/$generation\"\n"
+   "  cat > \"$generation_dir/provenance.scm\" <<EOF\n"
+   "((closure-path . \"$closure\")\n"
+   " (parameters-file . \"$closure/parameters.scm\")\n"
+   " (freebsd-base-file . \"$closure/metadata/freebsd-base.scm\")\n"
+   " (freebsd-source-file . \"$closure/metadata/freebsd-source.scm\")\n"
+   " (freebsd-source-materializations-file . \"$closure/metadata/freebsd-source-materializations.scm\")\n"
+   " (host-base-provenance-file . \"$closure/metadata/host-base-provenance.scm\")\n"
+   " (store-layout-file . \"$closure/metadata/store-layout.scm\"))\n"
+   "EOF\n"
+   "  chmod 644 \"$generation_dir/provenance.scm\"\n"
+   "}\n\n"
+   "write_generation_install_metadata()\n"
+   "{\n"
+   "  generation=$1\n"
+   "  closure=$2\n"
+   "  action=$3\n"
+   "  previous_generation=$4\n"
+   "  previous_closure=$5\n"
+   "  generation_dir=\"$generations_root/$generation\"\n"
+   "  cat > \"$generation_dir/install.scm\" <<EOF\n"
+   "((deployment-kind . \"$action\")\n"
+   " (generation-number . $generation)\n"
+   " (closure-path . \"$closure\")\n"
+   " (previous-generation-number . \"$previous_generation\")\n"
+   " (previous-closure-path . \"$previous_closure\")\n"
+   " (freebsd-base-file . \"$closure/metadata/freebsd-base.scm\")\n"
+   " (freebsd-source-file . \"$closure/metadata/freebsd-source.scm\")\n"
+   " (freebsd-source-materializations-file . \"$closure/metadata/freebsd-source-materializations.scm\")\n"
+   " (store-layout-file . \"$closure/metadata/store-layout.scm\"))\n"
+   "EOF\n"
+   "  chmod 644 \"$generation_dir/install.scm\"\n"
+   "}\n\n"
+   "prepare_generation()\n"
+   "{\n"
+   "  generation=$1\n"
+   "  closure=$2\n"
+   "  action=$3\n"
+   "  previous_generation=$4\n"
+   "  previous_closure=$5\n"
+   "  generation_dir=\"$generations_root/$generation\"\n"
+   "  mkdir -p \"$generation_dir\"\n"
+   "  symlink_force \"$closure\" \"$generation_dir/closure\"\n"
+   "  write_generation_metadata \"$generation\" \"$closure\" \"$action\" \"$previous_generation\" \"$previous_closure\"\n"
+   "  write_generation_provenance \"$generation\" \"$closure\"\n"
+   "  write_generation_install_metadata \"$generation\" \"$closure\" \"$action\" \"$previous_generation\" \"$previous_closure\"\n"
+   "}\n\n"
+   "update_efi_loader()\n"
+   "{\n"
+   "  closure=$1\n"
+   "  [ -e /dev/gpt/efiboot ] || return 0\n"
+   "  esp_mount=$(mktemp -d /tmp/fruix-efiboot.XXXXXX)\n"
+   "  if /sbin/mount -t msdosfs /dev/gpt/efiboot \"$esp_mount\" >/dev/null 2>&1; then\n"
+   "    mkdir -p \"$esp_mount/EFI/BOOT\"\n"
+   "    cp \"$closure/boot/loader.efi\" \"$esp_mount/EFI/BOOT/BOOTX64.EFI\"\n"
+   "    sync\n"
+   "    /sbin/umount \"$esp_mount\" >/dev/null 2>&1 || true\n"
+   "  fi\n"
+   "  rmdir \"$esp_mount\" >/dev/null 2>&1 || true\n"
+   "}\n\n"
+   "status()\n"
+   "{\n"
+   "  current_generation=$(read_file_maybe \"$current_generation_file\")\n"
+   "  current_generation_link=$(read_link_maybe \"$current_link\")\n"
+   "  current_closure=$(read_link_maybe \"$run_current_link\")\n"
+   "  rollback_generation=$(read_file_maybe \"$rollback_generation_file\")\n"
+   "  rollback_generation_link=$(read_link_maybe \"$rollback_link\")\n"
+   "  rollback_closure=\"\"\n"
+   "  if [ -n \"$rollback_generation_link\" ] && [ -L \"$system_root/$rollback_generation_link/closure\" ]; then\n"
+   "    rollback_closure=$(readlink \"$system_root/$rollback_generation_link/closure\")\n"
+   "  fi\n"
+   "  printf 'current_generation=%s\\n' \"$current_generation\"\n"
+   "  printf 'current_link=%s\\n' \"$current_generation_link\"\n"
+   "  printf 'current_closure=%s\\n' \"$current_closure\"\n"
+   "  printf 'rollback_generation=%s\\n' \"$rollback_generation\"\n"
+   "  printf 'rollback_link=%s\\n' \"$rollback_generation_link\"\n"
+   "  printf 'rollback_closure=%s\\n' \"$rollback_closure\"\n"
+   "}\n\n"
+   "switch_to_closure()\n"
+   "{\n"
+   "  target_closure=$1\n"
+   "  validate_closure \"$target_closure\"\n"
+   "  current_generation=$(read_file_maybe \"$current_generation_file\")\n"
+   "  current_closure=$(read_link_maybe \"$run_current_link\")\n"
+   "  [ -n \"$current_generation\" ] || die \"missing current generation metadata\"\n"
+   "  [ -n \"$current_closure\" ] || die \"missing /run/current-system target\"\n"
+   "  if [ \"$target_closure\" = \"$current_closure\" ]; then\n"
+   "    status\n"
+   "    return 0\n"
+   "  fi\n"
+   "  new_generation=$(next_generation_number)\n"
+   "  prepare_generation \"$new_generation\" \"$target_closure\" switch \"$current_generation\" \"$current_closure\"\n"
+   "  symlink_force \"generations/$current_generation\" \"$rollback_link\"\n"
+   "  printf '%s\\n' \"$current_generation\" > \"$rollback_generation_file\"\n"
+   "  symlink_force \"$current_closure\" \"$gcroots_root/rollback-system\"\n"
+   "  symlink_force \"generations/$new_generation\" \"$current_link\"\n"
+   "  printf '%s\\n' \"$new_generation\" > \"$current_generation_file\"\n"
+   "  symlink_force \"$target_closure\" \"$gcroots_root/system-$new_generation\"\n"
+   "  symlink_force \"$target_closure\" \"$gcroots_root/current-system\"\n"
+   "  symlink_force \"$target_closure\" \"$run_current_link\"\n"
+   "  update_efi_loader \"$target_closure\"\n"
+   "  status\n"
+   "}\n\n"
+   "rollback_current_generation()\n"
+   "{\n"
+   "  rollback_generation=$(read_file_maybe \"$rollback_generation_file\")\n"
+   "  rollback_generation_link=$(read_link_maybe \"$rollback_link\")\n"
+   "  [ -n \"$rollback_generation\" ] || die \"no rollback generation is recorded\"\n"
+   "  [ -n \"$rollback_generation_link\" ] || die \"no rollback link is recorded\"\n"
+   "  rollback_closure=$(read_link_maybe \"$system_root/$rollback_generation_link/closure\")\n"
+   "  [ -n \"$rollback_closure\" ] || die \"rollback generation has no closure link\"\n"
+   "  current_generation=$(read_file_maybe \"$current_generation_file\")\n"
+   "  current_closure=$(read_link_maybe \"$run_current_link\")\n"
+   "  [ -n \"$current_generation\" ] || die \"missing current generation metadata\"\n"
+   "  [ -n \"$current_closure\" ] || die \"missing current closure link\"\n"
+   "  symlink_force \"generations/$current_generation\" \"$rollback_link\"\n"
+   "  printf '%s\\n' \"$current_generation\" > \"$rollback_generation_file\"\n"
+   "  symlink_force \"$current_closure\" \"$gcroots_root/rollback-system\"\n"
+   "  symlink_force \"$rollback_generation_link\" \"$current_link\"\n"
+   "  printf '%s\\n' \"$rollback_generation\" > \"$current_generation_file\"\n"
+   "  symlink_force \"$rollback_closure\" \"$gcroots_root/current-system\"\n"
+   "  symlink_force \"$rollback_closure\" \"$run_current_link\"\n"
+   "  update_efi_loader \"$rollback_closure\"\n"
+   "  status\n"
+   "}\n\n"
+   "case \"${1:-}\" in\n"
+   "  system)\n"
+   "    case \"${2:-}\" in\n"
+   "      status)\n"
+   "        [ $# -eq 2 ] || { usage >&2; exit 1; }\n"
+   "        status\n"
+   "        ;;\n"
+   "      switch)\n"
+   "        [ $# -eq 3 ] || { usage >&2; exit 1; }\n"
+   "        switch_to_closure \"$3\"\n"
+   "        ;;\n"
+   "      rollback)\n"
+   "        [ $# -eq 2 ] || { usage >&2; exit 1; }\n"
+   "        rollback_current_generation\n"
+   "        ;;\n"
+   "      --help|-h|'')\n"
+   "        usage\n"
+   "        ;;\n"
+   "      *)\n"
+   "        usage >&2\n"
+   "        exit 1\n"
+   "        ;;\n"
+   "    esac\n"
+   "    ;;\n"
+   "  --help|-h|'')\n"
+   "    usage\n"
+   "    ;;\n"
+   "  *)\n"
+   "    usage >&2\n"
+   "    exit 1\n"
+   "    ;;\n"
+   "esac\n"))
+(define (render-development-environment-script os)
+  (string-append
+   "#!/bin/sh\n"
+   "set -eu\n"
+   "profile=/run/current-system/development-profile\n"
+   "[ -d \"$profile\" ] || {\n"
+   "  echo \"fruix-development-environment: development profile is not available\" >&2\n"
+   "  exit 1\n"
+   "}\n"
+   "cat <<EOF\n"
+   "export FRUIX_DEVELOPMENT_PROFILE=\"$profile\"\n"
+   "export FRUIX_DEVELOPMENT_INCLUDE=\"$profile/usr/include\"\n"
+   "export FRUIX_DEVELOPMENT_LIB=\"$profile/lib\"\n"
+   "export FRUIX_DEVELOPMENT_SHARE_MK=\"$profile/usr/share/mk\"\n"
+   "export FRUIX_DEVELOPMENT_BIN=\"$profile/bin\"\n"
+   "export FRUIX_DEVELOPMENT_USR_BIN=\"$profile/usr/bin\"\n"
+   "export FRUIX_CC=\"$profile/bin/cc\"\n"
+   "export FRUIX_CXX=\"$profile/bin/c++\"\n"
+   "export FRUIX_AR=\"$profile/bin/ar\"\n"
+   "export FRUIX_RANLIB=\"$profile/bin/ranlib\"\n"
+   "export FRUIX_NM=\"$profile/bin/nm\"\n"
+   "export FRUIX_BMAKE=\"/usr/bin/make\"\n"
+   "export CC=\"$profile/bin/cc\"\n"
+   "export CXX=\"$profile/bin/c++\"\n"
+   "export AR=\"$profile/bin/ar\"\n"
+   "export RANLIB=\"$profile/bin/ranlib\"\n"
+   "export NM=\"$profile/bin/nm\"\n"
+   "export CPPFLAGS=\"-I$profile/usr/include\"\n"
+   "export CFLAGS=\"-I$profile/usr/include\"\n"
+   "export CXXFLAGS=\"-I$profile/usr/include\"\n"
+   "export LDFLAGS=\"-L$profile/lib\"\n"
+   "export MAKEFLAGS=\"-m $profile/usr/share/mk\"\n"
+   "export PATH=\"/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:$profile/bin:$profile/sbin:$profile/usr/bin:$profile/usr/sbin\"\n"
+   "EOF\n"))
 
 
 (define* (operating-system-generated-files os #:key guile-store guile-extra-store shepherd-store)
@@ -486,7 +804,12 @@
                                                 #:guile-store guile-store
                                                 #:guile-extra-store guile-extra-store
                                                 #:shepherd-store shepherd-store))
-     ("shepherd/init.scm" . ,(render-shepherd-config os)))
+     ("shepherd/init.scm" . ,(render-shepherd-config os))
+     ("usr/local/bin/fruix" . ,(render-installed-system-fruix os)))
+   (if (null? (operating-system-development-packages os))
+       '()
+       `(("usr/local/bin/fruix-development-environment"
+          . ,(render-development-environment-script os))))
    (if (pid1-init-mode? os)
        `(("boot/fruix-pid1" . ,(render-pid1-script os shepherd-store guile-store guile-extra-store)))
        '())

modules/fruix/system/freebsd/source.scm

@@ -150,9 +150,10 @@
          (effective-source (assoc-ref resolution 'effective-source))
          (identity (assoc-ref resolution 'identity))
          (manifest (freebsd-source-manifest source effective-source identity))
-         (hash (sha256-string manifest))
-         (output-path (string-append store-dir "/" hash "-freebsd-source-"
-                                     (safe-name-fragment (freebsd-source-name source))))
+         (display-name (string-append "freebsd-source-"
+                                      (safe-name-fragment (freebsd-source-name source))))
+         (output-path (make-store-path store-dir display-name manifest
+                                       #:kind 'freebsd-source))
          (info-file (string-append output-path "/.freebsd-source-info.scm"))
          (cache-path (assoc-ref resolution 'cache-path))
          (populate-tree (assoc-ref resolution 'populate-tree)))

modules/fruix/system/freebsd/utils.scm

@@ -14,6 +14,8 @@
             safe-command-output
             write-file
             sha256-string
+            store-hash-string
+            make-store-path
             file-hash
             directory-entries
             path-signature
@@ -68,6 +70,40 @@
     (write-file tmp text)
     (command-output "sha256" "-q" tmp)))
 
+(define store-hash-visible-length 40)
+(define store-hash-scheme-version "1")
+
+(define (store-identity-field value)
+  (cond ((symbol? value)
+         (symbol->string value))
+        ((string? value)
+         value)
+        (else
+         (object->string value))))
+
+(define* (store-hash-string payload #:key (kind 'item) name (output "out"))
+  (let* ((identity `((scheme . "fruix-store-path")
+                     (version . ,store-hash-scheme-version)
+                     (kind . ,(store-identity-field kind))
+                     (name . ,(store-identity-field (or name "")))
+                     (output . ,(store-identity-field output))
+                     (payload . ,payload)))
+         (digest (sha256-string (object->string identity))))
+    (string-take digest store-hash-visible-length)))
+
+(define* (make-store-path store-dir display-name payload
+                          #:key
+                          (kind 'item)
+                          name
+                          (output "out"))
+  (string-append store-dir "/"
+                 (store-hash-string payload
+                                    #:kind kind
+                                    #:name (or name display-name)
+                                    #:output output)
+                 "-"
+                 display-name))
+
 (define (file-hash path)
   (command-output "sha256" "-q" path))
 

tests/system/phase19-generation-rollback-operating-system.scm.in

@@ -0,0 +1,91 @@
+(use-modules (fruix system freebsd)
+             (fruix packages freebsd))
+
+(define phase19-source
+  (freebsd-source
+   #:name "__SOURCE_NAME__"
+   #:kind 'git
+   #:ref "__SOURCE_REF__"
+   #:commit "__SOURCE_COMMIT__"))
+
+(define phase19-base
+  (freebsd-base
+   #:name "__BASE_NAME__"
+   #:version-label "__BASE_VERSION_LABEL__"
+   #:release "__BASE_RELEASE__"
+   #:branch "__BASE_BRANCH__"
+   #:source phase19-source
+   #:source-root "__DECLARED_SOURCE_ROOT__"
+   #:target "amd64"
+   #:target-arch "amd64"
+   #:kernconf "GENERIC"))
+
+(define phase19-operating-system
+  (operating-system
+   #:host-name "__HOST_NAME__"
+   #:freebsd-base phase19-base
+   #:kernel (freebsd-native-kernel-for phase19-base)
+   #:bootloader (freebsd-native-bootloader-for phase19-base)
+   #:base-packages (freebsd-native-system-packages-for phase19-base)
+   #:groups (list (user-group #:name "wheel" #:gid 0 #:system? #t)
+                  (user-group #:name "sshd" #:gid 22 #:system? #t)
+                  (user-group #:name "_dhcp" #:gid 65 #:system? #t)
+                  (user-group #:name "operator" #:gid 1000 #:system? #f))
+   #:users (list (user-account #:name "root"
+                               #:uid 0
+                               #:group "wheel"
+                               #:comment "Charlie &"
+                               #:home "/root"
+                               #:shell "/bin/sh"
+                               #:system? #t)
+                 (user-account #:name "sshd"
+                               #:uid 22
+                               #:group "sshd"
+                               #:comment "Secure Shell Daemon"
+                               #:home "/var/empty"
+                               #:shell "/usr/sbin/nologin"
+                               #:system? #t)
+                 (user-account #:name "_dhcp"
+                               #:uid 65
+                               #:group "_dhcp"
+                               #:comment "dhcp programs"
+                               #:home "/var/empty"
+                               #:shell "/usr/sbin/nologin"
+                               #:system? #t)
+                 (user-account #:name "operator"
+                               #:uid 1000
+                               #:group "operator"
+                               #:supplementary-groups '("wheel")
+                               #:comment "Fruix Operator"
+                               #:home "/home/operator"
+                               #:shell "/bin/sh"
+                               #:system? #f))
+   #:file-systems (list (file-system #:device "/dev/gpt/fruix-root"
+                                     #:mount-point "/"
+                                     #:type "ufs"
+                                     #:options "rw"
+                                     #:needed-for-boot? #t)
+                        (file-system #:device "devfs"
+                                     #:mount-point "/dev"
+                                     #:type "devfs"
+                                     #:options "rw"
+                                     #:needed-for-boot? #t)
+                        (file-system #:device "tmpfs"
+                                     #:mount-point "/tmp"
+                                     #:type "tmpfs"
+                                     #:options "rw,size=64m"))
+   #:services '(shepherd ready-marker sshd)
+   #:loader-entries '(("autoboot_delay" . "1")
+                      ("boot_multicons" . "YES")
+                      ("boot_serial" . "YES")
+                      ("console" . "comconsole,vidconsole"))
+   #:rc-conf-entries '(("clear_tmp_enable" . "NO")
+                       ("hostid_enable" . "NO")
+                       ("sendmail_enable" . "NONE")
+                       ("sshd_enable" . "YES")
+                       ("ifconfig_xn0" . "SYNCDHCP")
+                       ("ifconfig_em0" . "SYNCDHCP")
+                       ("ifconfig_vtnet0" . "SYNCDHCP"))
+   #:init-mode 'freebsd-init+rc.d-shepherd
+   #:ready-marker "/var/lib/fruix/ready"
+   #:root-authorized-keys '("__ROOT_AUTHORIZED_KEY__")))

tests/system/phase20-development-operating-system.scm.in

@@ -0,0 +1,73 @@
+(use-modules (fruix system freebsd)
+             (fruix packages freebsd))
+
+(define phase20-operating-system
+  (operating-system
+   #:host-name "fruix-freebsd"
+   #:kernel freebsd-native-kernel
+   #:bootloader freebsd-native-bootloader
+   #:base-packages %freebsd-native-system-packages
+   #:development-packages (list freebsd-native-headers
+                                freebsd-clang-toolchain)
+   #:groups (list (user-group #:name "wheel" #:gid 0 #:system? #t)
+                  (user-group #:name "sshd" #:gid 22 #:system? #t)
+                  (user-group #:name "_dhcp" #:gid 65 #:system? #t)
+                  (user-group #:name "operator" #:gid 1000 #:system? #f))
+   #:users (list (user-account #:name "root"
+                               #:uid 0
+                               #:group "wheel"
+                               #:comment "Charlie &"
+                               #:home "/root"
+                               #:shell "/bin/sh"
+                               #:system? #t)
+                 (user-account #:name "sshd"
+                               #:uid 22
+                               #:group "sshd"
+                               #:comment "Secure Shell Daemon"
+                               #:home "/var/empty"
+                               #:shell "/usr/sbin/nologin"
+                               #:system? #t)
+                 (user-account #:name "_dhcp"
+                               #:uid 65
+                               #:group "_dhcp"
+                               #:comment "dhcp programs"
+                               #:home "/var/empty"
+                               #:shell "/usr/sbin/nologin"
+                               #:system? #t)
+                 (user-account #:name "operator"
+                               #:uid 1000
+                               #:group "operator"
+                               #:supplementary-groups '("wheel")
+                               #:comment "Fruix Operator"
+                               #:home "/home/operator"
+                               #:shell "/bin/sh"
+                               #:system? #f))
+   #:file-systems (list (file-system #:device "/dev/gpt/fruix-root"
+                                     #:mount-point "/"
+                                     #:type "ufs"
+                                     #:options "rw"
+                                     #:needed-for-boot? #t)
+                        (file-system #:device "devfs"
+                                     #:mount-point "/dev"
+                                     #:type "devfs"
+                                     #:options "rw"
+                                     #:needed-for-boot? #t)
+                        (file-system #:device "tmpfs"
+                                     #:mount-point "/tmp"
+                                     #:type "tmpfs"
+                                     #:options "rw,size=64m"))
+   #:services '(shepherd ready-marker sshd)
+   #:loader-entries '(("autoboot_delay" . "1")
+                      ("boot_multicons" . "YES")
+                      ("boot_serial" . "YES")
+                      ("console" . "comconsole,vidconsole"))
+   #:rc-conf-entries '(("clear_tmp_enable" . "NO")
+                       ("hostid_enable" . "NO")
+                       ("sendmail_enable" . "NONE")
+                       ("sshd_enable" . "YES")
+                       ("ifconfig_xn0" . "SYNCDHCP")
+                       ("ifconfig_em0" . "SYNCDHCP")
+                       ("ifconfig_vtnet0" . "SYNCDHCP"))
+   #:init-mode 'shepherd-pid1
+   #:ready-marker "/var/lib/fruix/ready"
+   #:root-authorized-keys '("__ROOT_AUTHORIZED_KEY__")))

tests/system/run-phase19-installed-system-rollback-qemu.sh

@@ -0,0 +1,388 @@
+#!/bin/sh
+set -eu
+
+project_root=${PROJECT_ROOT:-$(pwd)}
+script_dir=$(CDPATH= cd -- "$(dirname "$0")" && pwd)
+fruix_cmd=$project_root/bin/fruix
+os_template=${OS_TEMPLATE:-$script_dir/phase19-generation-rollback-operating-system.scm.in}
+system_name=${SYSTEM_NAME:-phase19-operating-system}
+store_dir=${STORE_DIR:-/frx/store}
+disk_capacity=${DISK_CAPACITY:-12g}
+root_size=${ROOT_SIZE:-10g}
+qemu_smp=${QEMU_SMP:-2}
+ssh_port=${QEMU_SSH_PORT:-10024}
+base_name=${BASE_NAME:-phase19-generation-layout}
+base_version_label=${BASE_VERSION_LABEL:-15.0-STABLE-generation-layout}
+base_release=${BASE_RELEASE:-15.0-STABLE}
+base_branch=${BASE_BRANCH:-stable/15}
+source_name=${SOURCE_NAME:-stable15-generation-layout-source}
+source_ref=${SOURCE_REF:-stable/15}
+source_commit=${SOURCE_COMMIT:-332708a606f6bf0841c1d4a74c0d067f5640fe89}
+declared_source_root=${DECLARED_SOURCE_ROOT:-/var/empty/fruix-unused-source-root-generation-layout}
+current_host_name=${CURRENT_HOST_NAME:-fruix-freebsd-current}
+candidate_host_name=${CANDIDATE_HOST_NAME:-fruix-freebsd-canary}
+metadata_target=${METADATA_OUT:-}
+root_authorized_key_file=${ROOT_AUTHORIZED_KEY_FILE:-$HOME/.ssh/id_ed25519.pub}
+root_ssh_private_key_file=${ROOT_SSH_PRIVATE_KEY_FILE:-$HOME/.ssh/id_ed25519}
+cleanup=0
+
+[ -x "$fruix_cmd" ] || {
+  echo "fruix command is not executable: $fruix_cmd" >&2
+  exit 1
+}
+[ -f "$os_template" ] || {
+  echo "missing operating-system template: $os_template" >&2
+  exit 1
+}
+[ -f "$root_authorized_key_file" ] || {
+  echo "missing root authorized key file: $root_authorized_key_file" >&2
+  exit 1
+}
+[ -f "$root_ssh_private_key_file" ] || {
+  echo "missing root SSH private key file: $root_ssh_private_key_file" >&2
+  exit 1
+}
+command -v qemu-system-x86_64 >/dev/null 2>&1 || {
+  echo "qemu-system-x86_64 is required" >&2
+  exit 1
+}
+[ -f /usr/local/share/edk2-qemu/QEMU_UEFI_CODE-x86_64.fd ] || {
+  echo "missing QEMU UEFI firmware" >&2
+  exit 1
+}
+
+if [ -n "${WORKDIR:-}" ]; then
+  workdir=$WORKDIR
+  mkdir -p "$workdir"
+else
+  workdir=$(mktemp -d /tmp/fruix-phase19-installed-system-rollback-qemu.XXXXXX)
+  cleanup=1
+fi
+if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
+  cleanup=0
+fi
+
+current_os_file=$workdir/current-operating-system.scm
+candidate_os_file=$workdir/candidate-operating-system.scm
+current_install_out=$workdir/current-install.txt
+candidate_build_out=$workdir/candidate-build.txt
+target_image=$workdir/installed.img
+candidate_store_items=$workdir/candidate-store-items.txt
+stage_log=$workdir/stage-candidate-store.txt
+serial_log=$workdir/serial.log
+qemu_pidfile=$workdir/qemu.pid
+uefi_vars=$workdir/QEMU_UEFI_VARS.fd
+metadata_file=$workdir/phase19-installed-system-rollback-qemu-metadata.txt
+switch_status_file=$workdir/switch-status.txt
+rollback_status_file=$workdir/rollback-status.txt
+boot_current_status_file=$workdir/boot-current-status.txt
+boot_candidate_status_file=$workdir/boot-candidate-status.txt
+boot_rollback_status_file=$workdir/boot-rollback-status.txt
+generation2_metadata_file=$workdir/generation2-metadata.scm
+generation2_install_file=$workdir/generation2-install.scm
+mnt_root=$workdir/mnt-root
+md_unit=
+
+cleanup_workdir() {
+  if [ -f "$qemu_pidfile" ]; then
+    sudo kill "$(sudo cat "$qemu_pidfile")" >/dev/null 2>&1 || true
+  fi
+  if [ -n "$md_unit" ]; then
+    sudo umount "$mnt_root" >/dev/null 2>&1 || true
+    sudo mdconfig -d -u "$md_unit" >/dev/null 2>&1 || true
+  fi
+  if [ "$cleanup" -eq 1 ]; then
+    rm -rf "$workdir" 2>/dev/null || sudo rm -rf "$workdir"
+  fi
+}
+trap cleanup_workdir EXIT INT TERM
+
+render_os() {
+  output=$1
+  host_name=$2
+  root_authorized_key=$(tr -d '\n' < "$root_authorized_key_file")
+  sed \
+    -e "s|__BASE_NAME__|$base_name|g" \
+    -e "s|__BASE_VERSION_LABEL__|$base_version_label|g" \
+    -e "s|__BASE_RELEASE__|$base_release|g" \
+    -e "s|__BASE_BRANCH__|$base_branch|g" \
+    -e "s|__SOURCE_NAME__|$source_name|g" \
+    -e "s|__SOURCE_REF__|$source_ref|g" \
+    -e "s|__SOURCE_COMMIT__|$source_commit|g" \
+    -e "s|__DECLARED_SOURCE_ROOT__|$declared_source_root|g" \
+    -e "s|__HOST_NAME__|$host_name|g" \
+    -e "s|__ROOT_AUTHORIZED_KEY__|$root_authorized_key|g" \
+    "$os_template" > "$output"
+}
+
+render_os "$current_os_file" "$current_host_name"
+render_os "$candidate_os_file" "$candidate_host_name"
+cp /usr/local/share/edk2-qemu/QEMU_UEFI_VARS-x86_64.fd "$uefi_vars"
+mkdir -p "$mnt_root"
+
+action_env() {
+  sudo env \
+    HOME="$HOME" \
+    GUILE_AUTO_COMPILE=0 \
+    FRUIX_FREEBSD_BUILD_JOBS="${FRUIX_FREEBSD_BUILD_JOBS:-8}" \
+    GUIX_SOURCE_DIR="${GUIX_SOURCE_DIR:-$HOME/repos/guix}" \
+    GUILE_BIN="${GUILE_BIN:-/tmp/guile-freebsd-validate-install/bin/guile}" \
+    GUILE_EXTRA_PREFIX="${GUILE_EXTRA_PREFIX:-/tmp/guile-gnutls-freebsd-validate-install}" \
+    SHEPHERD_PREFIX="${SHEPHERD_PREFIX:-/tmp/shepherd-freebsd-validate-install}" \
+    "$@"
+}
+
+field() {
+  name=$1
+  file=$2
+  sed -n "s/^$name=//p" "$file" | tail -n 1
+}
+
+status_field() {
+  name=$1
+  file=$2
+  sed -n "s/^$name=//p" "$file" | tail -n 1
+}
+
+wait_for_ssh() {
+  for attempt in $(jot 120 1 120); do
+    if ssh_guest 'service sshd onestatus >/dev/null 2>&1' >/dev/null 2>&1; then
+      return 0
+    fi
+    sleep 2
+  done
+  return 1
+}
+
+ssh_guest() {
+  ssh -p "$ssh_port" -i "$root_ssh_private_key_file" \
+    -o BatchMode=yes \
+    -o StrictHostKeyChecking=no \
+    -o UserKnownHostsFile=/dev/null \
+    -o ConnectTimeout=5 \
+    root@127.0.0.1 "$@"
+}
+
+reboot_guest() {
+  ssh_guest 'shutdown -r now >/dev/null 2>&1 || reboot >/dev/null 2>&1 || true' >/dev/null 2>&1 || true
+  sleep 5
+  wait_for_ssh || {
+    echo "guest did not return over SSH after reboot" >&2
+    exit 1
+  }
+}
+
+capture_status() {
+  output_file=$1
+  ssh_guest '/usr/local/bin/fruix system status' > "$output_file"
+}
+
+action_env "$fruix_cmd" system install "$current_os_file" \
+  --system "$system_name" \
+  --store "$store_dir" \
+  --target "$target_image" \
+  --disk-capacity "$disk_capacity" \
+  --root-size "$root_size" > "$current_install_out"
+
+current_closure_path=$(field closure_path "$current_install_out")
+install_metadata_path=$(field install_metadata_path "$current_install_out")
+materialized_source_store=$(field materialized_source_stores "$current_install_out")
+[ -n "$current_closure_path" ] || { echo "missing current closure path" >&2; exit 1; }
+[ "$install_metadata_path" = /var/lib/fruix/install.scm ] || { echo "unexpected install metadata path: $install_metadata_path" >&2; exit 1; }
+
+action_env "$fruix_cmd" system build "$candidate_os_file" \
+  --system "$system_name" \
+  --store "$store_dir" > "$candidate_build_out"
+
+candidate_closure_path=$(field closure_path "$candidate_build_out")
+[ -n "$candidate_closure_path" ] || { echo "missing candidate closure path" >&2; exit 1; }
+[ "$candidate_closure_path" != "$current_closure_path" ] || { echo "candidate closure unexpectedly matches current closure" >&2; exit 1; }
+
+{
+  printf '%s\n' "$candidate_closure_path"
+  cat "$candidate_closure_path/.references"
+} | awk 'NF { print }' | sort -u > "$candidate_store_items"
+
+candidate_store_item_count=$(wc -l < "$candidate_store_items" | tr -d ' ')
+
+md=$(sudo mdconfig -a -t vnode -f "$target_image")
+md_unit=${md#md}
+sudo mount -t ufs "/dev/${md}p2" "$mnt_root"
+sudo mkdir -p "$mnt_root/frx/store"
+: > "$stage_log"
+while IFS= read -r item; do
+  [ -n "$item" ] || continue
+  item_base=$(basename "$item")
+  if [ -e "$mnt_root/frx/store/$item_base" ]; then
+    printf 'already-present=%s\n' "$item_base" >> "$stage_log"
+    continue
+  fi
+  printf 'copy=%s\n' "$item_base" >> "$stage_log"
+  sudo sh -c "cd '$store_dir' && pax -rw -pe '$item_base' '$mnt_root/frx/store'"
+done < "$candidate_store_items"
+sudo sync
+sudo umount "$mnt_root"
+sudo mdconfig -d -u "$md_unit"
+md_unit=
+
+sudo qemu-system-x86_64 \
+  -machine q35,accel=tcg \
+  -cpu max \
+  -m 2048 \
+  -smp "$qemu_smp" \
+  -display none \
+  -serial "file:$serial_log" \
+  -monitor none \
+  -pidfile "$qemu_pidfile" \
+  -daemonize \
+  -drive if=pflash,format=raw,readonly=on,file=/usr/local/share/edk2-qemu/QEMU_UEFI_CODE-x86_64.fd \
+  -drive if=pflash,format=raw,file="$uefi_vars" \
+  -drive if=virtio,format=raw,file="$target_image" \
+  -netdev user,id=net0,hostfwd=tcp::${ssh_port}-:22 \
+  -device virtio-net-pci,netdev=net0
+
+wait_for_ssh || {
+  echo "guest never became reachable over SSH" >&2
+  exit 1
+}
+
+capture_status "$boot_current_status_file"
+boot_current_closure=$(ssh_guest 'readlink /run/current-system')
+boot_current_hostname=$(ssh_guest 'hostname')
+boot_current_generation=$(status_field current_generation "$boot_current_status_file")
+boot_current_link=$(status_field current_link "$boot_current_status_file")
+boot_current_rollback_generation=$(status_field rollback_generation "$boot_current_status_file")
+[ "$boot_current_closure" = "$current_closure_path" ] || { echo "unexpected current closure after initial boot: $boot_current_closure" >&2; exit 1; }
+[ "$boot_current_hostname" = "$current_host_name" ] || { echo "unexpected hostname after initial boot: $boot_current_hostname" >&2; exit 1; }
+[ "$boot_current_generation" = 1 ] || { echo "unexpected initial generation: $boot_current_generation" >&2; exit 1; }
+[ "$boot_current_link" = generations/1 ] || { echo "unexpected initial current link: $boot_current_link" >&2; exit 1; }
+[ -z "$boot_current_rollback_generation" ] || { echo "rollback generation should be empty before switch" >&2; exit 1; }
+ssh_guest 'test -x /usr/local/bin/fruix'
+
+ssh_guest "/usr/local/bin/fruix system switch $candidate_closure_path" > "$switch_status_file"
+switch_current_generation=$(status_field current_generation "$switch_status_file")
+switch_current_link=$(status_field current_link "$switch_status_file")
+switch_current_closure=$(status_field current_closure "$switch_status_file")
+switch_rollback_generation=$(status_field rollback_generation "$switch_status_file")
+switch_rollback_link=$(status_field rollback_link "$switch_status_file")
+switch_rollback_closure=$(status_field rollback_closure "$switch_status_file")
+[ "$switch_current_generation" = 2 ] || { echo "unexpected generation after switch: $switch_current_generation" >&2; exit 1; }
+[ "$switch_current_link" = generations/2 ] || { echo "unexpected current link after switch: $switch_current_link" >&2; exit 1; }
+[ "$switch_current_closure" = "$candidate_closure_path" ] || { echo "unexpected current closure after switch: $switch_current_closure" >&2; exit 1; }
+[ "$switch_rollback_generation" = 1 ] || { echo "unexpected rollback generation after switch: $switch_rollback_generation" >&2; exit 1; }
+[ "$switch_rollback_link" = generations/1 ] || { echo "unexpected rollback link after switch: $switch_rollback_link" >&2; exit 1; }
+[ "$switch_rollback_closure" = "$current_closure_path" ] || { echo "unexpected rollback closure after switch: $switch_rollback_closure" >&2; exit 1; }
+[ "$(ssh_guest 'readlink /frx/var/fruix/gcroots/current-system')" = "$candidate_closure_path" ] || { echo "unexpected current-system gc root after switch" >&2; exit 1; }
+[ "$(ssh_guest 'readlink /frx/var/fruix/gcroots/rollback-system')" = "$current_closure_path" ] || { echo "unexpected rollback-system gc root after switch" >&2; exit 1; }
+[ "$(ssh_guest 'readlink /frx/var/fruix/gcroots/system-2')" = "$candidate_closure_path" ] || { echo "unexpected system-2 gc root after switch" >&2; exit 1; }
+ssh_guest 'test -f /var/lib/fruix/system/generations/2/metadata.scm'
+ssh_guest 'test -f /var/lib/fruix/system/generations/2/provenance.scm'
+ssh_guest 'test -f /var/lib/fruix/system/generations/2/install.scm'
+ssh_guest "cat /var/lib/fruix/system/generations/2/metadata.scm" > "$generation2_metadata_file"
+ssh_guest "cat /var/lib/fruix/system/generations/2/install.scm" > "$generation2_install_file"
+case "$(cat "$generation2_metadata_file")" in
+  *"$candidate_closure_path"*"$current_closure_path"*) : ;;
+  *) echo "generation 2 metadata does not record both candidate and previous closure paths" >&2; exit 1 ;;
+esac
+case "$(cat "$generation2_install_file")" in
+  *"(deployment-kind . \"switch\")"*"$candidate_closure_path"*) : ;;
+  *) echo "generation 2 install metadata does not record switch provenance" >&2; exit 1 ;;
+esac
+
+reboot_guest
+capture_status "$boot_candidate_status_file"
+boot_candidate_closure=$(ssh_guest 'readlink /run/current-system')
+boot_candidate_hostname=$(ssh_guest 'hostname')
+boot_candidate_generation=$(status_field current_generation "$boot_candidate_status_file")
+boot_candidate_rollback_generation=$(status_field rollback_generation "$boot_candidate_status_file")
+boot_candidate_rollback_closure=$(status_field rollback_closure "$boot_candidate_status_file")
+[ "$boot_candidate_closure" = "$candidate_closure_path" ] || { echo "unexpected closure after switch reboot: $boot_candidate_closure" >&2; exit 1; }
+[ "$boot_candidate_hostname" = "$candidate_host_name" ] || { echo "unexpected hostname after switch reboot: $boot_candidate_hostname" >&2; exit 1; }
+[ "$boot_candidate_generation" = 2 ] || { echo "unexpected generation after switch reboot: $boot_candidate_generation" >&2; exit 1; }
+[ "$boot_candidate_rollback_generation" = 1 ] || { echo "unexpected rollback generation after switch reboot: $boot_candidate_rollback_generation" >&2; exit 1; }
+[ "$boot_candidate_rollback_closure" = "$current_closure_path" ] || { echo "unexpected rollback closure after switch reboot: $boot_candidate_rollback_closure" >&2; exit 1; }
+
+ssh_guest '/usr/local/bin/fruix system rollback' > "$rollback_status_file"
+rollback_current_generation=$(status_field current_generation "$rollback_status_file")
+rollback_current_link=$(status_field current_link "$rollback_status_file")
+rollback_current_closure=$(status_field current_closure "$rollback_status_file")
+rollback_rollback_generation=$(status_field rollback_generation "$rollback_status_file")
+rollback_rollback_link=$(status_field rollback_link "$rollback_status_file")
+rollback_rollback_closure=$(status_field rollback_closure "$rollback_status_file")
+[ "$rollback_current_generation" = 1 ] || { echo "unexpected generation after rollback: $rollback_current_generation" >&2; exit 1; }
+[ "$rollback_current_link" = generations/1 ] || { echo "unexpected current link after rollback: $rollback_current_link" >&2; exit 1; }
+[ "$rollback_current_closure" = "$current_closure_path" ] || { echo "unexpected current closure after rollback: $rollback_current_closure" >&2; exit 1; }
+[ "$rollback_rollback_generation" = 2 ] || { echo "unexpected rollback generation after rollback: $rollback_rollback_generation" >&2; exit 1; }
+[ "$rollback_rollback_link" = generations/2 ] || { echo "unexpected rollback link after rollback: $rollback_rollback_link" >&2; exit 1; }
+[ "$rollback_rollback_closure" = "$candidate_closure_path" ] || { echo "unexpected rollback closure after rollback: $rollback_rollback_closure" >&2; exit 1; }
+[ "$(ssh_guest 'readlink /frx/var/fruix/gcroots/current-system')" = "$current_closure_path" ] || { echo "unexpected current-system gc root after rollback" >&2; exit 1; }
+[ "$(ssh_guest 'readlink /frx/var/fruix/gcroots/rollback-system')" = "$candidate_closure_path" ] || { echo "unexpected rollback-system gc root after rollback" >&2; exit 1; }
+
+reboot_guest
+capture_status "$boot_rollback_status_file"
+boot_rollback_closure=$(ssh_guest 'readlink /run/current-system')
+boot_rollback_hostname=$(ssh_guest 'hostname')
+boot_rollback_generation=$(status_field current_generation "$boot_rollback_status_file")
+boot_rollback_rollback_generation=$(status_field rollback_generation "$boot_rollback_status_file")
+boot_rollback_rollback_closure=$(status_field rollback_closure "$boot_rollback_status_file")
+shepherd_status=$(ssh_guest '/usr/local/etc/rc.d/fruix-shepherd onestatus >/dev/null 2>&1 && echo running || echo stopped')
+sshd_status=$(ssh_guest 'service sshd onestatus >/dev/null 2>&1 && echo running || echo stopped')
+activate_log=$(ssh_guest 'cat /var/log/fruix-activate.log 2>/dev/null || true' | tr '\n' ' ')
+[ "$boot_rollback_closure" = "$current_closure_path" ] || { echo "unexpected closure after rollback reboot: $boot_rollback_closure" >&2; exit 1; }
+[ "$boot_rollback_hostname" = "$current_host_name" ] || { echo "unexpected hostname after rollback reboot: $boot_rollback_hostname" >&2; exit 1; }
+[ "$boot_rollback_generation" = 1 ] || { echo "unexpected generation after rollback reboot: $boot_rollback_generation" >&2; exit 1; }
+[ "$boot_rollback_rollback_generation" = 2 ] || { echo "unexpected rollback generation after rollback reboot: $boot_rollback_rollback_generation" >&2; exit 1; }
+[ "$boot_rollback_rollback_closure" = "$candidate_closure_path" ] || { echo "unexpected rollback closure after rollback reboot: $boot_rollback_rollback_closure" >&2; exit 1; }
+[ "$shepherd_status" = running ] || { echo "fruix-shepherd is not running after rollback reboot" >&2; exit 1; }
+[ "$sshd_status" = running ] || { echo "sshd is not running after rollback reboot" >&2; exit 1; }
+case "$activate_log" in
+  *fruix-activate:done*) : ;;
+  *) echo "activation log does not show success after rollback workflow" >&2; exit 1 ;;
+esac
+
+cat >"$metadata_file" <<EOF
+workdir=$workdir
+current_os_file=$current_os_file
+candidate_os_file=$candidate_os_file
+target_image=$target_image
+current_closure_path=$current_closure_path
+candidate_closure_path=$candidate_closure_path
+current_host_name=$current_host_name
+candidate_host_name=$candidate_host_name
+candidate_store_item_count=$candidate_store_item_count
+candidate_store_items=$candidate_store_items
+stage_log=$stage_log
+serial_log=$serial_log
+install_metadata_path=$install_metadata_path
+materialized_source_store=$materialized_source_store
+switch_status_file=$switch_status_file
+rollback_status_file=$rollback_status_file
+boot_current_status_file=$boot_current_status_file
+boot_candidate_status_file=$boot_candidate_status_file
+boot_rollback_status_file=$boot_rollback_status_file
+generation2_metadata_file=$generation2_metadata_file
+generation2_install_file=$generation2_install_file
+final_current_generation=$boot_rollback_generation
+final_current_closure=$boot_rollback_closure
+final_rollback_generation=$boot_rollback_rollback_generation
+final_rollback_closure=$boot_rollback_rollback_closure
+shepherd_status=$shepherd_status
+sshd_status=$sshd_status
+boot_backend=qemu-uefi-tcg
+installed_system_switch=ok
+installed_system_rollback=ok
+EOF
+
+if [ -n "$metadata_target" ]; then
+  mkdir -p "$(dirname "$metadata_target")"
+  cp "$metadata_file" "$metadata_target"
+fi
+
+printf 'PASS phase19-installed-system-rollback-qemu\n'
+printf 'Work directory: %s\n' "$workdir"
+printf 'Metadata file: %s\n' "$metadata_file"
+if [ -n "$metadata_target" ]; then
+  printf 'Copied metadata to: %s\n' "$metadata_target"
+fi
+printf '%s\n' '--- metadata ---'
+cat "$metadata_file"

tests/system/run-phase20-development-environment-xcpng.sh

@@ -0,0 +1,220 @@
+#!/bin/sh
+set -eu
+
+repo_root=${PROJECT_ROOT:-$(pwd)}
+os_template=${OS_TEMPLATE:-$repo_root/tests/system/phase20-development-operating-system.scm.in}
+system_name=${SYSTEM_NAME:-phase20-operating-system}
+root_size=${ROOT_SIZE:-8g}
+metadata_target=${METADATA_OUT:-}
+root_authorized_key_file=${ROOT_AUTHORIZED_KEY_FILE:-$HOME/.ssh/id_ed25519.pub}
+root_ssh_private_key_file=${ROOT_SSH_PRIVATE_KEY_FILE:-$HOME/.ssh/id_ed25519}
+cleanup=0
+
+if [ -n "${WORKDIR:-}" ]; then
+  workdir=$WORKDIR
+  mkdir -p "$workdir"
+else
+  workdir=$(mktemp -d /tmp/fruix-phase20-development-xcpng.XXXXXX)
+  cleanup=1
+fi
+if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
+  cleanup=0
+fi
+
+inner_metadata=$workdir/phase20-development-inner-metadata.txt
+metadata_file=$workdir/phase20-development-environment-xcpng-metadata.txt
+
+action_cleanup() {
+  if [ "$cleanup" -eq 1 ]; then
+    rm -rf "$workdir"
+  fi
+}
+trap action_cleanup EXIT INT TERM
+
+KEEP_WORKDIR=1 WORKDIR="$workdir/inner" METADATA_OUT="$inner_metadata" \
+  ROOT_AUTHORIZED_KEY_FILE="$root_authorized_key_file" \
+  ROOT_SSH_PRIVATE_KEY_FILE="$root_ssh_private_key_file" \
+  OS_TEMPLATE="$os_template" SYSTEM_NAME="$system_name" ROOT_SIZE="$root_size" \
+  "$repo_root/tests/system/run-phase11-shepherd-pid1-xcpng.sh"
+
+phase8_metadata=$(sed -n 's/^phase8_metadata=//p' "$inner_metadata")
+closure_path=$(sed -n 's/^closure_path=//p' "$inner_metadata")
+closure_base=$(sed -n 's/^closure_base=//p' "$inner_metadata")
+guest_ip=$(sed -n 's/^guest_ip=//p' "$inner_metadata")
+vm_id=$(sed -n 's/^vm_id=//p' "$inner_metadata")
+vdi_id=$(sed -n 's/^vdi_id=//p' "$inner_metadata")
+shepherd_pid=$(sed -n 's/^shepherd_pid=//p' "$inner_metadata")
+sshd_status=$(sed -n 's/^sshd_status=//p' "$inner_metadata")
+compat_prefix_shims=$(sed -n 's/^compat_prefix_shims=//p' "$inner_metadata")
+guile_module_smoke=$(sed -n 's/^guile_module_smoke=//p' "$inner_metadata")
+activate_log=$(sed -n 's/^activate_log=//p' "$inner_metadata")
+
+development_profile_path=$closure_path/development-profile
+runtime_profile_path=$closure_path/profile
+development_env_script=$closure_path/usr/local/bin/fruix-development-environment
+
+[ "$shepherd_pid" = 1 ] || { echo "shepherd was not PID 1" >&2; exit 1; }
+[ "$sshd_status" = running ] || { echo "sshd is not running" >&2; exit 1; }
+[ "$compat_prefix_shims" = absent ] || { echo "compatibility prefix shims reappeared" >&2; exit 1; }
+[ "$guile_module_smoke" = ok ] || { echo "guest Guile module smoke failed" >&2; exit 1; }
+case "$activate_log" in
+  *fruix-activate:done*) : ;;
+  *) echo "activation log does not show success" >&2; exit 1 ;;
+esac
+
+for path in \
+  "$development_profile_path/bin/cc" \
+  "$development_profile_path/bin/c++" \
+  "$development_profile_path/bin/ar" \
+  "$development_profile_path/usr/include/sys/param.h" \
+  "$development_profile_path/usr/share/mk/bsd.prog.mk" \
+  "$development_env_script"
+do
+  [ -e "$path" ] || {
+    echo "required development environment path missing: $path" >&2
+    exit 1
+  }
+done
+[ ! -e "$runtime_profile_path/include" ] || { echo "runtime profile unexpectedly contains headers" >&2; exit 1; }
+[ ! -e "$runtime_profile_path/usr/share/mk" ] || { echo "runtime profile unexpectedly contains /usr/share/mk" >&2; exit 1; }
+[ ! -e "$runtime_profile_path/bin/cc" ] || { echo "runtime profile unexpectedly contains cc" >&2; exit 1; }
+
+ssh_guest() {
+  ssh -i "$root_ssh_private_key_file" \
+    -o BatchMode=yes \
+    -o StrictHostKeyChecking=no \
+    -o UserKnownHostsFile=/dev/null \
+    -o ConnectTimeout=5 \
+    root@"$guest_ip" "$@"
+}
+
+guest_dev_metadata=$(ssh -i "$root_ssh_private_key_file" \
+  -o BatchMode=yes \
+  -o StrictHostKeyChecking=no \
+  -o UserKnownHostsFile=/dev/null \
+  -o ConnectTimeout=5 \
+  root@"$guest_ip" 'sh -s' <<'EOF'
+set -eu
+[ -x /usr/local/bin/fruix-development-environment ]
+[ -L /run/current-development ]
+[ "$(readlink /run/current-development)" = "/run/current-system/development-profile" ]
+exports=$(/usr/local/bin/fruix-development-environment)
+printf '%s\n' "$exports" | grep '^export FRUIX_DEVELOPMENT_PROFILE="/run/current-system/development-profile"$' >/dev/null
+printf '%s\n' "$exports" | grep '^export MAKEFLAGS="-m /run/current-system/development-profile/usr/share/mk"$' >/dev/null
+eval "$exports"
+[ -d "$FRUIX_DEVELOPMENT_PROFILE" ]
+[ -x "$FRUIX_CC" ]
+[ -x "$FRUIX_CXX" ]
+[ -x "$FRUIX_AR" ]
+[ -f "$FRUIX_DEVELOPMENT_INCLUDE/sys/param.h" ]
+[ -f "$FRUIX_DEVELOPMENT_SHARE_MK/bsd.prog.mk" ]
+cc_version=$($FRUIX_CC --version | awk 'NR==1 { print; exit }')
+tmp=/tmp/fruix-phase20-development-env
+rm -rf "$tmp"
+mkdir -p "$tmp/direct" "$tmp/mk"
+cat > "$tmp/direct/hello.c" <<'EOS'
+#include <stdio.h>
+int main(void){puts("hello-from-direct-dev-env");return 0;}
+EOS
+$FRUIX_CC $CPPFLAGS $LDFLAGS "$tmp/direct/hello.c" -o "$tmp/direct/hello"
+hello_direct=$($tmp/direct/hello)
+cat > "$tmp/mk/hello.c" <<'EOS'
+#include <stdio.h>
+int main(void){puts("hello-from-make-dev-env");return 0;}
+EOS
+cat > "$tmp/mk/Makefile" <<'EOS'
+PROG=hello
+SRCS=hello.c
+NO_MAN=yes
+.include <bsd.prog.mk>
+EOS
+cat > "$tmp/mk/hello.1" <<'EOS'
+.Dd April 5, 2026
+.Dt HELLO 1
+.Sh NAME
+.Nm hello
+.Nd phase20 development environment smoke binary
+EOS
+cd "$tmp/mk"
+make clean >/dev/null 2>&1 || true
+make > "$tmp/mk/make.log" 2>&1
+hello_make=$(./hello)
+make_log_tail=$(tail -n 20 "$tmp/mk/make.log" | tr '\n' ' ')
+exports_flat=$(printf '%s' "$exports" | tr '\n' ' ')
+printf 'development_profile=%s\n' "$FRUIX_DEVELOPMENT_PROFILE"
+printf 'cc_version=%s\n' "$cc_version"
+printf 'hello_direct=%s\n' "$hello_direct"
+printf 'hello_make=%s\n' "$hello_make"
+printf 'exports=%s\n' "$exports_flat"
+printf 'make_log_tail=%s\n' "$make_log_tail"
+EOF
+)
+
+development_profile=$(printf '%s\n' "$guest_dev_metadata" | sed -n 's/^development_profile=//p')
+cc_version=$(printf '%s\n' "$guest_dev_metadata" | sed -n 's/^cc_version=//p')
+hello_direct=$(printf '%s\n' "$guest_dev_metadata" | sed -n 's/^hello_direct=//p')
+hello_make=$(printf '%s\n' "$guest_dev_metadata" | sed -n 's/^hello_make=//p')
+development_exports=$(printf '%s\n' "$guest_dev_metadata" | sed -n 's/^exports=//p')
+make_log_tail=$(printf '%s\n' "$guest_dev_metadata" | sed -n 's/^make_log_tail=//p')
+
+[ "$development_profile" = "/run/current-system/development-profile" ] || {
+  echo "unexpected guest development profile path: $development_profile" >&2
+  exit 1
+}
+case "$cc_version" in
+  *"FreeBSD clang version"*) : ;;
+  *) echo "unexpected cc version output: $cc_version" >&2; exit 1 ;;
+esac
+[ "$hello_direct" = hello-from-direct-dev-env ] || {
+  echo "unexpected direct compile output: $hello_direct" >&2
+  exit 1
+}
+[ "$hello_make" = hello-from-make-dev-env ] || {
+  echo "unexpected make-based compile output: $hello_make" >&2
+  exit 1
+}
+case "$development_exports" in
+  *'export FRUIX_CC="/run/current-system/development-profile/bin/cc"'*) : ;;
+  *) echo "development environment exports do not include FRUIX_CC" >&2; exit 1 ;;
+esac
+
+cat >"$metadata_file" <<EOF
+workdir=$workdir
+inner_metadata=$inner_metadata
+phase8_metadata=$phase8_metadata
+closure_path=$closure_path
+closure_base=$closure_base
+vm_id=$vm_id
+vdi_id=$vdi_id
+guest_ip=$guest_ip
+root_size=$root_size
+development_profile_path=$development_profile_path
+development_env_script=$development_env_script
+shepherd_pid=$shepherd_pid
+sshd_status=$sshd_status
+compat_prefix_shims=$compat_prefix_shims
+guile_module_smoke=$guile_module_smoke
+development_profile_guest=$development_profile
+cc_version=$cc_version
+hello_direct=$hello_direct
+hello_make=$hello_make
+development_exports=$development_exports
+make_log_tail=$make_log_tail
+boot_backend=xcp-ng-xo-cli
+init_mode=shepherd-pid1
+development_environment=ok
+EOF
+
+if [ -n "$metadata_target" ]; then
+  mkdir -p "$(dirname "$metadata_target")"
+  cp "$metadata_file" "$metadata_target"
+fi
+
+printf 'PASS phase20-development-environment-xcpng\n'
+printf 'Work directory: %s\n' "$workdir"
+printf 'Metadata file: %s\n' "$metadata_file"
+if [ -n "$metadata_target" ]; then
+  printf 'Copied metadata to: %s\n' "$metadata_target"
+fi
+printf '%s\n' '--- metadata ---'
+cat "$metadata_file"