self/fruix
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: self:b3b1ba248906d1b4a7a7544930235f2d340f0d3f
Branches Tags
self:master
...
compare: self:master
Branches Tags
self:master

11 Commits
b3b1ba2489 ... master

Author SHA1 Message Date
Steffen Beyer
2a1a6c3b81 docs: record build-profile revalidation 2026-04-06 19:04:18 +02:00
Steffen Beyer
cb9e7332f4 system: separate build profile from development 2026-04-06 12:38:35 +02:00
Steffen Beyer
db4d5bdf4c system: add in-node build and reconfigure 2026-04-06 08:55:35 +02:00
Steffen Beyer
0e8b30434f system: consume promoted native base results 2026-04-06 06:38:23 +02:00
Steffen Beyer
f41a916f45 native-build: introduce executor model 2026-04-06 05:26:34 +02:00
Steffen Beyer
006ffee615 native-build: promote results into store objects 2026-04-06 01:16:44 +02:00
Steffen Beyer
4614592a25 system: prototype self-hosted native builds 2026-04-05 17:59:56 +02:00
Steffen Beyer
a3dd5556ae system: validate host-initiated native base builds 2026-04-05 16:32:08 +02:00
Steffen Beyer
9e9a0b59fc system: validate development environment overlay 2026-04-05 11:56:06 +02:00
Steffen Beyer
4975084baa store: centralize Fruix path naming 2026-04-05 08:58:40 +02:00
Steffen Beyer
9dae4e5c84 system: validate installed rollback workflow 2026-04-05 01:39:24 +02:00
34 changed files with 7028 additions and 201 deletions
Expand all files Collapse all files

203
docs/GUIX_DIFFERENCES.md
View File

@@ -1,6 +1,6 @@
# Fruix differences for Guix sysadmins # Fruix differences for Guix sysadmins
Date: 2026-04-04 Date: 2026-04-06
This document is aimed at operators who already know Guix well and want a quick map of where Fruix behaves similarly and where it intentionally differs. This document is aimed at operators who already know Guix well and want a quick map of where Fruix behaves similarly and where it intentionally differs.
@@ -48,13 +48,12 @@ That path remains the active runtime boundary used by activation and service wir
Fruix avoids in-place mutation of an older deployed closure. Fruix avoids in-place mutation of an older deployed closure.
The validated rollback story today is: The validated rollback story now has two layers:
- keep the earlier declaration - declaration-level rollback by rebuilding/redeploying an earlier declaration
- rebuild or rematerialize it - installed-system rollback between already-recorded generations on the target itself
- boot or redeploy that earlier closure again
That is Guix-like in spirit even though Fruix does not yet expose the same installed-system rollback command surface. That is Guix-like in spirit, although Fruix still exposes a smaller installed-system workflow than Guix's more mature `reconfigure` model.
### Generation-style metadata and roots ### Generation-style metadata and roots
@@ -78,7 +77,7 @@ Guix heavily reuses its profile-generation model and represents a lot of meaning
Fruix keeps the **semantics** but uses a more explicit metadata-oriented layout for installed systems. Fruix keeps the **semantics** but uses a more explicit metadata-oriented layout for installed systems.
Current Fruix layout: Current Fruix layout starts as:
```text ```text
/var/lib/fruix/system/ /var/lib/fruix/system/
@@ -92,6 +91,20 @@ Current Fruix layout:
install.scm install.scm
``` ```
After a validated installed-system switch, Fruix also records:
```text
/var/lib/fruix/system/
rollback -> generations/1
rollback-generation
generations/
2/
closure -> /frx/store/...-fruix-system-...
metadata.scm
provenance.scm
install.scm
```
Why Fruix does this: Why Fruix does this:
- it makes deployment state easier to inspect directly - it makes deployment state easier to inspect directly
@@ -154,27 +167,172 @@ Validated examples:
So compared with Guix-on-Linux intuition, Fruix operators should be more explicit about target-device selection during installation and installer-media validation. So compared with Guix-on-Linux intuition, Fruix operators should be more explicit about target-device selection during installation and installer-media validation.
## 6. Fruix does not yet have Guix-equivalent installed-system generation commands ## 6. Fruix now has a minimal installed-system generation command surface, but it is still smaller than Guix's
This is the biggest current operational gap. This remains the biggest operational gap, but it is no longer a complete gap.
Fruix does **not** yet provide a mature equivalent of the familiar Guix System operator flow around in-place generation switching and rollback commands. Installed Fruix systems now provide a larger in-guest helper surface:
Today, Fruix rollback is mostly: - `fruix system build`
- `fruix system reconfigure`
- `fruix system status`
- `fruix system switch /frx/store/...-fruix-system-...`
- `fruix system rollback`
- declaration-driven What this gives you today:
- rebuild/redeploy based
rather than: - explicit current-generation tracking
- explicit rollback-generation tracking
- in-place switching between already-staged closures on the installed target
- rollback without reinstalling the whole system image again
- a validated in-node build path that can read the embedded current declaration inputs
- a validated in-node `reconfigure` path that can build a candidate closure locally and stage it as the next generation
- switch current system generation in place through a dedicated command What it still does **not** give you yet compared with Guix:
So if you come from Guix, assume that Fruix currently has: - a mature end-to-end `upgrade` story for advancing declared inputs automatically
- automatic closure transfer/fetch as part of `switch`
- a higher-level `deploy` workflow across multiple machines or targets
- the broader generation-management UX Guix operators expect
So if you come from Guix, assume that Fruix now has:
- strong closure/store semantics - strong closure/store semantics
- explicit install artifacts - explicit install artifacts
- explicit generation metadata roots - explicit generation metadata roots
- but a less mature installed-system generation UX - a real installed-system build/reconfigure/switch/rollback surface
- but not yet the fuller long-term node/deployment UX that Guix users may expect
## 7. Fruix keeps Guix-like store semantics, but not Guix/Nix hash-prefix machinery exactly
Fruix still uses immutable store paths under:
- `/frx/store`
and it still treats a store path as a deployment identity boundary.
But Fruix now intentionally differs from Guix/Nix in how the visible store-path prefix is constructed.
Current Fruix policy is:
- centralize store-path naming behind shared helpers
- hash a small semantic identity record rather than copying Nix's historical path-hash formula exactly
- include at least:
- object kind
- logical/display name
- output name
- payload or manifest identity
- hash-scheme version marker
- truncate the visible SHA-256 prefix to **160 bits**
- render that visible prefix as **40 hex characters**
Why Fruix does this instead of copying Guix/Nix exactly:
- the main goal was shorter store prefixes, not Nix compatibility for its own sake
- distinct outputs should have distinct identities because `out`, `lib`, `debug`, and `doc` are semantically different artifacts
- Fruix wanted one central policy point that can be swapped later without touching every materializer again
- Fruix did **not** want to inherit Nix's legacy details unless they provide clear value here
- custom base32 alphabet and bit ordering
- compressed/XOR-folded path hashes
- exact historical `output:out` / `source` path-hash conventions
So compared with Guix:
- the important semantic property is the same:
- different store objects should get different immutable identities
- the exact printable prefix algorithm is intentionally simpler in Fruix today
For Guix-familiar operators, the practical takeaway is:
- still think of `/frx/store/...` paths as immutable deployment identities
- do **not** assume Fruix store prefixes are byte-for-byte comparable to Guix/Nix ones
- expect Fruix to prefer a simpler, centralized naming policy unless exact Guix/Nix behavior becomes necessary later
## 8. Fruix can expose separate in-system development and build overlays
For the validated Phase 20 path, Fruix now distinguishes three layers instead of two:
- runtime profile
- development profile
- build profile
On those systems, Fruix exposes:
- development:
- `/run/current-system/development-profile`
- `/run/current-development`
- `/usr/local/bin/fruix-development-environment`
- build:
- `/run/current-system/build-profile`
- `/run/current-build`
- `/usr/local/bin/fruix-build-environment`
- canonical base-build compatibility links:
- `/usr/include -> /run/current-system/build-profile/usr/include`
- `/usr/share/mk -> /run/current-system/build-profile/usr/share/mk`
The intent is:
- keep the main runtime profile lean
- keep the development helper interactive and convenient
- keep the build helper narrower, more sanitized, and closer to the actual `buildworld` / `buildkernel` contract
- avoid treating a development-heavy or build-heavy image as the default runtime shape
Compared with Guix, this is conceptually similar to keeping development-oriented and build-oriented state separate from the main runtime identity, but Fruix currently expresses it as system-attached overlays rather than through Guix's broader profile/tooling model.
Fruix still has the guest self-hosted native-build prototype helper at:
- `/usr/local/bin/fruix-self-hosted-native-build`
But that helper now explicitly uses the build helper contract instead of trying to reuse the full interactive development shell. The practical Fruix takeaway is:
- the development overlay makes native base work possible inside the system
- the build overlay gives Fruix a stricter, more reproducible contract for real base builds
Fruix now also makes an explicit distinction between:
- mutable native-build staging/results under:
- `/var/lib/fruix/native-builds`
- immutable promoted native-build identities under:
- `/frx/store`
So a guest self-hosted run can stage a result tree locally, while the host can later promote that result into first-class Fruix store objects such as:
- `/frx/store/...-fruix-native-world-...`
- `/frx/store/...-fruix-native-kernel-...`
- `/frx/store/...-fruix-native-headers-...`
- `/frx/store/...-fruix-native-bootloader-...`
- `/frx/store/...-fruix-native-build-result-...`
Compared with Guix, this is a more explicit split between:
- a mutable result/staging area for native build execution
- and the immutable store identities that Fruix treats as the real promoted result
Fruix also now models native-build placement more explicitly as an executor choice.
Current executor kinds are:
- `host`
- `ssh-guest`
- `self-hosted`
So instead of treating:
- host-driven builds
- host-initiated guest builds
- guest self-hosted builds
as three unrelated architectural forks, Fruix is moving toward one native-build result model with different executor policies attached to it.
Fruix also now lets system declarations consume a promoted native-build result bundle directly.
That means Fruix can now distinguish more explicitly between:
- a mutable build/result root used during execution
- an immutable promoted native-build identity in `/frx/store`
- and a later system declaration that points at that promoted identity as an input
Compared with Guix, this is still Guix-like in the important sense that the deployed system is identified by immutable store objects, but Fruix is making the FreeBSD native-base result set itself a more explicit first-class declaration input than it was before.
## Where Fruix is intentionally trying to improve on Guix's representation ## Where Fruix is intentionally trying to improve on Guix's representation
@@ -202,10 +360,11 @@ If you are already comfortable with Guix, the safest Fruix mental model today is
- closure path - closure path
- source provenance metadata - source provenance metadata
- install metadata - install metadata
5. think of rollback today as: 5. think of rollback in two layers:
- “redeploy the earlier declaration again” - if the target already has the desired closure staged locally:
rather than: - use `fruix system rollback`
- “switch to an already-managed previous generation in place” - otherwise:
- redeploy the earlier declaration again
## Status summary ## Status summary
@@ -222,4 +381,4 @@ It differs most from Guix in:
- source-provenance emphasis - source-provenance emphasis
- installer-medium-oriented workflows - installer-medium-oriented workflows
- generation-layout representation - generation-layout representation
- and the still-maturing installed-system generation command surface - and an installed-system generation command surface that now exists, but is still much smaller than Guix's

162
docs/PROGRESS.md
View File

@@ -30,6 +30,51 @@ Fruix currently has:
- `/var/lib/fruix/system` - `/var/lib/fruix/system`
- explicit installed-system retention roots under: - explicit installed-system retention roots under:
- `/frx/var/fruix/gcroots` - `/frx/var/fruix/gcroots`
- a validated installed-system generation switch/rollback workflow via:
- `fruix system status`
- `fruix system switch`
- `fruix system rollback`
- a validated separate in-system development environment overlay via:
- `/run/current-system/development-profile`
- `/run/current-development`
- `/usr/local/bin/fruix-development-environment`
- a validated separate in-system build environment overlay via:
- `/run/current-system/build-profile`
- `/run/current-build`
- `/usr/local/bin/fruix-build-environment`
- `/usr/include -> /run/current-system/build-profile/usr/include`
- `/usr/share/mk -> /run/current-system/build-profile/usr/share/mk`
- a validated host-initiated native base-build path inside a Fruix-managed guest via:
- real XCP-ng boot of a development-enabled Fruix system
- in-guest `buildworld` / `buildkernel`
- staged `installworld` / `distribution` / `installkernel`
- a validated controlled guest self-hosted native base-build prototype via:
- `/usr/local/bin/fruix-self-hosted-native-build`
- result roots under `/var/lib/fruix/native-builds`
- in-guest source recovery from current-system metadata
- a validated promotion path that turns native-build result roots into first-class Fruix store objects via:
- `fruix native-build promote`
- `/frx/store/...-fruix-native-world-...`
- `/frx/store/...-fruix-native-kernel-...`
- `/frx/store/...-fruix-native-headers-...`
- `/frx/store/...-fruix-native-bootloader-...`
- `/frx/store/...-fruix-native-build-result-...`
- an explicit executor model for native base builds with current executor kinds:
- `host`
- `ssh-guest`
- `self-hosted`
- end-to-end validated staged-result-plus-promotion paths for executor policies:
- `ssh-guest`
- `self-hosted`
- system declarations that can now consume a promoted native-build result bundle directly via:
- `promoted-native-build-result`
- `operating-system-from-promoted-native-build-result`
- a real XCP-ng boot validation of a system materialized from a promoted native-build result identity
- installed systems that now carry their own canonical declaration inputs and bundled Fruix node CLI sources
- a real XCP-ng validation of in-node:
- `fruix system build`
- `fruix system reconfigure`
- `fruix system rollback`
Validated boot modes still are: Validated boot modes still are:
@@ -42,38 +87,98 @@ The validated Phase 18 installation work currently uses:
## Latest completed achievement ## Latest completed achievement
### 2026-04-04 — Phase 19.2 completed ### 2026-04-06 — Fruix now separates interactive development from strict native base-build environment
Fruix now records an explicit installed-system generation layout and retention-root model instead of relying mainly on harness knowledge. Fruix now has a more explicit three-layer model for build-capable FreeBSD systems:
- runtime profile
- development profile
- build profile
Highlights: Highlights:
- added explicit installed-system generation layout under: - `<operating-system>` now supports separate `build-packages`
- `/var/lib/fruix/system` - system closures can now materialize both:
- added explicit installed-system retention roots under: - `development-profile`
- `/frx/var/fruix/gcroots` - `build-profile`
- installed targets now record a first-generation deployment directory containing: - build-capable systems now expose:
- `closure` - `/run/current-system/build-profile`
- `metadata.scm` - `/run/current-build`
- `provenance.scm` - `/usr/local/bin/fruix-build-environment`
- `install.scm` - canonical compatibility links for native base builds now come from the build profile:
- `/run/current-system` remains the runtime boundary and still points directly at the active closure path - `/usr/include -> /run/current-system/build-profile/usr/include`
- added Guix-oriented operator notes in: - `/usr/share/mk -> /run/current-system/build-profile/usr/share/mk`
- `docs/GUIX_DIFFERENCES.md` - the new build helper intentionally clears development-shell variables such as:
- updated deployment workflow documentation to reflect the new explicit generation model - `MAKEFLAGS`
- `CPPFLAGS`
- `CFLAGS`
- `CXXFLAGS`
- `LDFLAGS`
- the self-hosted native-build helper now uses this stricter build-helper contract instead of manually reconstructing that sanitization ad hoc
- promotion metadata for native-build results now records `build-profile` explicitly
Validation: Validation:
- `PASS phase19-generation-layout-qemu` - `PASS phase20-development-environment-xcpng`
- regression re-check: - `PASS phase20-self-hosted-native-build-xcpng`
- `PASS phase18-installer-iso` - `PASS phase20-native-build-store-promotion-xcpng`
- `PASS phase20-host-initiated-native-build-xcpng`
- `PASS phase20-host-initiated-native-build-store-promotion-xcpng`
Representative validated metadata included:
- `build_profile_guest=/run/current-system/build-profile`
- `build_profile=/run/current-system/build-profile`
- `helper_version=5`
- `executor_version=5`
Report:
- `docs/reports/postphase20-build-profile-separation-freebsd.md`
- `docs/system-deployment-workflow.md`
- `docs/GUIX_DIFFERENCES.md`
### 2026-04-06 — Installed systems can now build and reconfigure themselves from local declaration state
Fruix-installed systems are now meaningfully closer to real Fruix nodes.
Highlights:
- system closures now carry canonical declaration metadata in:
- `metadata/system-declaration.scm`
- `metadata/system-declaration-info.scm`
- `metadata/system-declaration-system`
- system closures now also carry bundled Fruix node CLI sources in:
- `share/fruix/node/scripts/fruix.scm`
- `share/fruix/node/modules/...`
- `share/fruix/node/guix/guix/build/utils.scm`
- the installed helper at `/usr/local/bin/fruix` now supports:
- `fruix system build`
- `fruix system reconfigure`
- `fruix system status`
- `fruix system switch`
- `fruix system rollback`
- no-argument in-node `build` and `reconfigure` now use the node's own embedded declaration inputs
- in-node Fruix builds now reuse the installed Guile/Shepherd runtime stores already referenced by the system instead of assuming host-only `/tmp/...` build prefixes
- the real XCP-ng validation proved the full installed-node flow:
- boot current system
- build from local declaration state
- build a candidate declaration on-node
- `reconfigure` into that candidate generation
- reboot into the candidate generation
- `rollback`
- reboot back into the original generation
Validation:
- `PASS postphase20-installed-node-build-reconfigure-xcpng`
Reports: Reports:
- `docs/reports/postphase20-promoted-native-base-declarations-freebsd.md`
- `docs/reports/postphase20-installed-node-management-freebsd.md`
- `docs/system-deployment-workflow.md` - `docs/system-deployment-workflow.md`
- `docs/GUIX_DIFFERENCES.md` - `docs/GUIX_DIFFERENCES.md`
- `docs/reports/phase19-deployment-workflow-freebsd.md`
- `docs/reports/phase19-generation-layout-freebsd.md`
## Recent major milestones ## Recent major milestones
@@ -97,8 +202,19 @@ Reports:
## Next step ## Next step
Per `docs/PLAN_4.md`, the next planned step is: `docs/PLAN_4.md` currently ends at Phase 20.3, and that milestone sequence is now complete.
- **Phase 19.3** — validate installed-system rollback through the intended operator-facing workflow The next practical follow-up is now clearer:
Phase 19.2 is now complete: Fruix has an explicit installed-system generation layout and retention-root model on FreeBSD. - grow the installed-node command surface from validated `build`/`reconfigure`/`rollback` toward:
- `upgrade`
- `build-base`
- `deploy`
- decide how executor policy, native-base promotion, and installed-node reconfiguration should compose in one operator-facing workflow
- determine how much of native-build request/promotion should remain explicit versus being absorbed into higher-level Fruix node actions
The immediate architectural direction is no longer just “can guest self-hosting work?”
It is now:
- how should Fruix expose real managed-node behavior across declaration inputs, native-base results, generation switching, and deployment actions?

226
docs/reports/phase19-installed-system-rollback-freebsd.md Normal file
View File

@@ -0,0 +1,226 @@
# Phase 19.3: installed-system rollback workflow on FreeBSD
Date: 2026-04-04
## Goal
Phase 19.3 is about validating installed-system rollback through the intended operator-facing workflow, not only through host-side build/image redeploy harnesses.
The key question was:
- can an already-installed Fruix system move between recorded generations coherently, using an operator-facing command surface on the target itself?
## Decision
The current Fruix solution is intentionally modest.
Fruix now provides a small installed-system helper on the target itself:
- `/usr/local/bin/fruix`
Validated in-guest commands:
- `fruix system status`
- `fruix system switch /frx/store/...-fruix-system-...`
- `fruix system rollback`
Important scope choice:
- `switch` assumes the candidate closure is already present on the target's `/frx/store`
- Fruix does **not** yet fetch or transfer that closure onto the target automatically
That keeps Phase 19.3 focused on generation-state correctness rather than introducing a larger store-transfer story prematurely.
## Implemented model
Installed systems now support the following validated operator pattern:
1. build a candidate closure with the host-side Fruix frontend
2. stage that closure into the installed system's `/frx/store`
3. run:
- `fruix system switch /frx/store/...candidate...`
4. reboot into the candidate generation
5. if needed, run:
- `fruix system rollback`
6. reboot into the recorded rollback generation
The installed system now records explicit rollback state under:
```text
/var/lib/fruix/system/
current -> generations/N
current-generation
rollback -> generations/M
rollback-generation
generations/
...
```
and explicit rollback reachability under:
```text
/frx/var/fruix/gcroots/
current-system -> /frx/store/...current...
rollback-system -> /frx/store/...rollback...
system-1 -> ...
system-2 -> ...
```
## Code changes
### `modules/fruix/system/freebsd/render.scm`
Added a generated in-guest Fruix deployment helper script under:
- `usr/local/bin/fruix`
That helper now:
- reports installed-system state with `fruix system status`
- stages a new current generation with `fruix system switch`
- stages the recorded rollback generation with `fruix system rollback`
- updates:
- `/var/lib/fruix/system/current`
- `/var/lib/fruix/system/current-generation`
- `/var/lib/fruix/system/rollback`
- `/var/lib/fruix/system/rollback-generation`
- `/frx/var/fruix/gcroots/current-system`
- `/frx/var/fruix/gcroots/rollback-system`
- `/frx/var/fruix/gcroots/system-N`
- refreshes the ESP bootloader file from the selected closure's `boot/loader.efi`
A practical implementation detail mattered here:
- replacing `/run/current-system` with a remove-then-recreate strategy caused the live shell environment to break while the link was absent
- switching that update to an atomic symlink replacement path for `/run/current-system` avoided that gap and made the in-guest operator command reliable
### `modules/fruix/system/freebsd/media.scm`
Updated installed rootfs staging so that installed targets expose:
- `/usr/local/bin/fruix -> /run/current-system/usr/local/bin/fruix`
Also bumped the explicit generation-layout version from:
- `1` to `2`
because the installed-system model now includes operator-driven switch/rollback state as part of the validated layout story.
### `modules/fruix/system/freebsd/model.scm`
Updated generated-file metadata so the system closure records:
- `usr/local/bin/fruix`
as part of the generated operating-system file set.
## New validation harness
Added:
- `tests/system/run-phase19-installed-system-rollback-qemu.sh`
This harness validates the actual installed-system operator flow on local `QEMU/UEFI/TCG`.
## Validation flow
The harness now performs all of the following:
1. installs a current system image directly to a target disk image
2. builds a distinct candidate closure
- in the validated harness this differs by host name so the closure identity changes cleanly without needing a heavier base-version rebuild
3. stages the candidate closure and its referenced store items into the installed target's `/frx/store`
4. boots the installed current system
5. validates initial state:
- current generation = `1`
- current closure = current installed closure
- no rollback generation yet recorded
6. runs:
- `fruix system switch /frx/store/...candidate...`
7. validates staged switch state:
- current generation = `2`
- rollback generation = `1`
- current closure = candidate closure
- rollback closure = original current closure
- generation 2 metadata/install files were written
8. reboots and validates boot into the candidate closure
9. runs:
- `fruix system rollback`
10. validates staged rollback state:
- current generation = `1`
- rollback generation = `2`
- current closure = original closure
- rollback closure = candidate closure
11. reboots and validates boot back into the original current system
12. confirms post-rollback service state:
- `fruix-shepherd` running
- `sshd` running
- activation log still shows success
## Passing validation
Passing result:
- `PASS phase19-installed-system-rollback-qemu`
Validated metadata summary:
```text
current_closure_path=/frx/store/4debd106d62f14594ba1612e1e7105f1658bf5f4075d6e5db5436efeaf929d90-fruix-system-fruix-freebsd-current
candidate_closure_path=/frx/store/54fb14e6071b8e5704a5dc75e2881c2f0533767771c26c4181f57afea88d1e8b-fruix-system-fruix-freebsd-canary
current_host_name=fruix-freebsd-current
candidate_host_name=fruix-freebsd-canary
final_current_generation=1
final_current_closure=/frx/store/4debd106d62f14594ba1612e1e7105f1658bf5f4075d6e5db5436efeaf929d90-fruix-system-fruix-freebsd-current
final_rollback_generation=2
final_rollback_closure=/frx/store/54fb14e6071b8e5704a5dc75e2881c2f0533767771c26c4181f57afea88d1e8b-fruix-system-fruix-freebsd-canary
installed_system_switch=ok
installed_system_rollback=ok
```
## Regression checks
After landing the installed-system switch/rollback workflow, the following regression checks still pass:
- `PASS phase19-generation-layout-qemu`
- `PASS phase18-installer-iso`
That means the new in-guest generation-management path did not regress:
- the previously validated explicit generation layout
- or the UEFI installer ISO boot/install path
## Relationship to Guix
This phase does **not** claim that Fruix now matches Guix's full installed-system UX.
What Fruix now has is:
- explicit generation state on disk
- explicit current/rollback pointers
- a minimal installed-system operator command surface
- validated switching and rollback between already-staged closures
What still remains compared with Guix:
- building/staging the candidate closure from inside the target system itself
- automatic closure transfer/fetch as part of `switch`
- a richer long-term generation lifecycle policy
## Conclusion
Phase 19.3 is complete.
Fruix now validates an actual installed-system rollback workflow on FreeBSD:
- the target system itself can report current/rollback state
- it can switch to a staged candidate generation
- it can reboot into that candidate generation
- it can roll back to the recorded prior generation
- and it can reboot into the restored current system
That closes the Phase 19 deployment story from:
- documented deployment workflow
- to explicit generation layout
- to validated installed-system operator rollback behavior

160
docs/reports/phase20-development-environment-freebsd.md Normal file
View File

@@ -0,0 +1,160 @@
# Phase 20.1: Fruix-managed development environment for native FreeBSD base work
Date: 2026-04-05
## Goal
Validate that a booted Fruix-managed FreeBSD system can expose a usable development environment for deeper native base work without collapsing the runtime/development boundary back into one broad profile.
This step explicitly builds on the Phase 14 split between:
- native runtime/boot artifacts for the running system
- separate development-facing artifacts such as headers and toolchain pieces
The goal is **not** full self-hosting yet.
It is to prove that a running Fruix system can expose the tools and paths needed for native FreeBSD build work in a controlled way.
## Implementation
### New operating-system field
`modules/fruix/system/freebsd/model.scm` now supports:
- `#:development-packages`
- `operating-system-development-packages`
The default remains empty, so existing systems do not change unless they opt in.
### Separate development profile inside the system closure
`modules/fruix/system/freebsd/media.scm` now materializes an additional profile tree when `development-packages` is non-empty:
- `/frx/store/...-fruix-system-.../development-profile`
The main runtime tree remains:
- `/frx/store/...-fruix-system-.../profile`
This preserves the runtime/development split:
- runtime stays under `profile`
- development tooling stays under `development-profile`
The development stores are also now part of the closure references and recorded in `metadata/store-layout.scm`.
### In-guest development environment helper
Opt-in systems with development packages now ship:
- `/usr/local/bin/fruix-development-environment`
and expose a stable runtime link:
- `/run/current-development -> /run/current-system/development-profile`
The helper emits shell exports for the active development profile, including at least:
- `FRUIX_DEVELOPMENT_PROFILE`
- `FRUIX_DEVELOPMENT_INCLUDE`
- `FRUIX_DEVELOPMENT_SHARE_MK`
- `FRUIX_CC`
- `FRUIX_CXX`
- `FRUIX_AR`
- `FRUIX_RANLIB`
- `FRUIX_NM`
- `FRUIX_BMAKE`
- `CPPFLAGS`
- `MAKEFLAGS`
- `PATH`
Intended use:
```sh
eval "$(/usr/local/bin/fruix-development-environment)"
```
### Chosen development overlay for this phase
For the validated Phase 20.1 guest path, the development overlay is intentionally narrow:
- `freebsd-native-headers`
- `freebsd-clang-toolchain`
This was chosen deliberately.
The earlier standalone Phase 14 development-profile package set was designed for broad profile composition, but for a running Fruix system it unnecessarily duplicated runtime pieces already present in the system profile.
For native base work, the important additions here are:
- `usr/include`
- `usr/share/mk`
- Clang/binutils-style frontend tools
while the running system already supplies:
- base runtime
- `/usr/bin/make`
- system libraries and boot/runtime state
That keeps the Phase 20.1 environment focused on native base development rather than reintroducing a broad mixed profile.
## New files
Added:
- `tests/system/phase20-development-operating-system.scm.in`
- `tests/system/run-phase20-development-environment-xcpng.sh`
## Validation
Passing run:
- `PASS phase20-development-environment-xcpng`
- workdir: `/tmp/fruix-phase20-development-xcpng`
Validated on the approved real XCP-ng path:
- VM `90490f2e-e8fc-4b7a-388e-5c26f0157289`
- VDI `0f1f90d3-48ca-4fa2-91d8-fc6339b95743`
Representative result:
```text
closure_path=/frx/store/c0ad43d7ef72323d4270a4f1e96ca1f5cc99566c-fruix-system-fruix-freebsd
development_profile_path=/frx/store/c0ad43d7ef72323d4270a4f1e96ca1f5cc99566c-fruix-system-fruix-freebsd/development-profile
development_profile_guest=/run/current-system/development-profile
cc_version=FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)
hello_direct=hello-from-direct-dev-env
hello_make=hello-from-make-dev-env
development_environment=ok
```
The harness verified all of the following on the real booted Fruix guest:
- runtime boot/regression still succeeds on XCP-ng
- the closure contains a separate `development-profile`
- runtime `profile` does **not** regain headers or `usr/share/mk`
- `/usr/local/bin/fruix-development-environment` exists and emits the expected exports
- `/run/current-development` points at `/run/current-system/development-profile`
- direct compilation works with the exported Clang toolchain and headers
- a simple native FreeBSD `make` build using `.include <bsd.prog.mk>` also succeeds
## Result
Phase 20.1 is complete.
Fruix now validates a real booted FreeBSD system path where:
- the running system remains lean and runtime-focused
- native development artifacts are exposed separately and explicitly
- the guest can compile code directly with the Fruix-provided toolchain
- the guest can also drive a simple `bsd.prog.mk` build using the exported development environment
This is enough to say that Fruix can host a controlled native FreeBSD base-development environment, without yet claiming full self-hosting.
## Next step
Per `docs/PLAN_4.md`, the next planned step is:
- **Phase 20.2** — run host-initiated native base builds inside a Fruix-managed environment

169
docs/reports/phase20-host-initiated-native-builds-freebsd.md Normal file
View File

@@ -0,0 +1,169 @@
# Phase 20.2: host-initiated native base builds inside a Fruix-managed environment
Date: 2026-04-05
## Goal
Validate the next step after Phase 20.1:
- the host still orchestrates the outer loop
- the actual FreeBSD native base build work runs inside a booted Fruix-managed system
This is the intermediate path between:
- purely host-side native base builds
- any future claim of guest self-hosting
The target here was not a new self-hosted package manager story.
It was narrower:
- boot a Fruix-managed FreeBSD system on the approved real XCP-ng path
- expose the development environment required for native base work
- run real `buildworld` / `buildkernel` / staged install steps inside that Fruix guest
- confirm that the resulting staged artifacts are the expected FreeBSD base slices
## Implementation
### Canonical development-path compatibility links
Phase 20.1 proved that Fruix could keep development content separate in:
- `/run/current-system/development-profile`
- `/run/current-development`
Phase 20.2 exposed an additional practical requirement:
- FreeBSD native base builds still expect canonical system paths such as:
- `/usr/include`
- `/usr/share/mk`
For development-enabled systems, `populate-rootfs-from-closure` now also exposes:
- `/usr/include -> /run/current-system/development-profile/usr/include`
- `/usr/share/mk -> /run/current-system/development-profile/usr/share/mk`
This keeps the development profile separate while still satisfying the buildworld/buildkernel assumptions of the native FreeBSD build system.
### Media builder invalidation
Because this changed the visible rootfs layout of booted systems, the media builder versions were bumped in `modules/fruix/system/freebsd/media.scm`:
- `image-builder-version`
- `install-builder-version`
- `installer-image-builder-version`
- `installer-iso-builder-version`
That ensured booted images and future installed targets actually pick up the new compatibility links.
### New validation harness
Added:
- `tests/system/run-phase20-host-initiated-native-build-xcpng.sh`
This harness reuses the validated Phase 20.1 XCP-ng path first, then performs the 20.2-native-build step over SSH from the host.
The guest build flow is:
1. boot the development-enabled Fruix guest on XCP-ng
2. recover the materialized source store from `/run/current-system/metadata/store-layout.scm`
3. run real FreeBSD native build commands inside the guest:
- `make -j8 buildworld`
- `make -j8 buildkernel`
- `make DESTDIR=... installworld`
- `make DESTDIR=... distribution`
- `make DESTDIR=... installkernel`
4. stage narrower artifact slices from the staged output:
- headers slice
- bootloader slice
- kernel stage
### Why `DB_FROM_SRC=yes` is used for staged install steps
The development-enabled Fruix guest is intentionally lean and does not carry the full ambient host account database.
`installworld` on modern FreeBSD checks for required users/groups unless `DB_FROM_SRC` is defined. For staged installs into `DESTDIR`, the appropriate controlled input is the source tree's own account database under `etc/`, not the minimal running guest's `/etc/master.passwd`.
So the validated Phase 20.2 staged install path uses:
- `DB_FROM_SRC=yes`
for:
- `installworld`
- `distribution`
- `installkernel`
That keeps the staged install driven by the declared source input rather than by accidental guest-local account state.
## Validation
Passing run:
- `PASS phase20-host-initiated-native-build-xcpng`
- workdir: `/tmp/fruix-phase20-host-initiated-native-build-xcpng`
Validated on the approved real XCP-ng path:
- VM `90490f2e-e8fc-4b7a-388e-5c26f0157289`
- VDI `0f1f90d3-48ca-4fa2-91d8-fc6339b95743`
Representative result:
```text
build_jobs=8
source_store=/frx/store/12d7704362e95afc2697db63f168b878e082b372-freebsd-source-default
source_root=/frx/store/12d7704362e95afc2697db63f168b878e082b372-freebsd-source-default/tree
build_root=/var/tmp/fruix-phase20-native-build
world_stage=/var/tmp/fruix-phase20-native-build/stage-world
kernel_stage=/var/tmp/fruix-phase20-native-build/stage-kernel
headers_stage=/var/tmp/fruix-phase20-native-build/artifact-headers
bootloader_stage=/var/tmp/fruix-phase20-native-build/artifact-bootloader
build_root_size=7.6G
world_stage_size=672M
kernel_stage_size=739M
headers_stage_size=32M
bootloader_stage_size=1.3M
sha_kernel=16950f116a52134b98e2f8e0dacc556e18fe254e4a0ac2c1741422dde281a341
sha_loader=ea417846167ece270ada611624dca622ca38bd30125b9a125cd8ebb8b3600313
sha_param=9eb140ca7d9666f3d484a4174c9acd94b45427db6292b4e17de19af2c6aa5219
host_initiated_native_build=ok
```
The harness verified all of the following:
- the guest still boots and passes the Phase 20.1 development-environment checks first
- development-enabled systems expose canonical native-build compatibility links at:
- `/usr/include`
- `/usr/share/mk`
- the guest can recover the declared materialized FreeBSD source store from system metadata
- real FreeBSD `buildworld` succeeds inside the booted Fruix guest
- real FreeBSD `buildkernel` succeeds inside the booted Fruix guest
- staged `installworld`, `distribution`, and `installkernel` also succeed inside the guest
- the staged outputs contain the expected artifact shapes:
- `boot/kernel/kernel`
- `usr/include/sys/param.h`
- `usr/share/mk/bsd.prog.mk`
- `boot/loader.efi`
- `boot/defaults/loader.conf`
- `boot/lua/loader.lua`
## Result
Phase 20.2 is complete.
Fruix now validates a real host-orchestrated path where:
- the host boots and reaches a Fruix-managed development-enabled guest
- the guest uses its own Fruix-exposed development paths and declared source store
- the native FreeBSD base build work runs inside that Fruix-managed environment
- the host remains the outer orchestrator and result collector
This materially narrows the gap to any future self-hosting experiment while still avoiding the complexity jump to a full guest-driven package/deployment loop.
## Next step
Per `docs/PLAN_4.md`, the next planned step is:
- **Phase 20.3** — reassess and potentially prototype guest self-hosted base builds

202
docs/reports/phase20-native-build-executor-model-freebsd.md Normal file
View File

@@ -0,0 +1,202 @@
# Post-Phase 20: native build executor model
Date: 2026-04-06
## Goal
Turn the now-proven native base-build placement options into one Fruix abstraction instead of treating them as unrelated paths.
The desired model is:
- same declared source identity
- same expected artifact kinds
- same staged-result shape
- same promotion/provenance shape
- different executor policy
So the question becomes:
- where should the build run?
and the answer is expressed as executor policy rather than as a separate architecture each time.
## Executor model
Fruix now has an explicit native-build executor model with current executor kinds:
- `host`
- `ssh-guest`
- `self-hosted`
and intended future extension points:
- `jail`
- `remote-builder`
The new executor model is implemented in:
- `modules/fruix/system/freebsd/executor.scm`
It defines a structured executor object that records at least:
- `kind`
- `name`
- `version`
- `properties`
## What changed
### 1. Structured executor metadata
Native-build result metadata is no longer limited to a flat string such as:
- `guest-self-hosted`
Instead, result/promotion objects now carry a structured executor description.
Representative executor objects now look like:
```scheme
((kind . self-hosted)
(name . "guest-self-hosted")
(version . "4")
(properties . (...)))
```
or:
```scheme
((kind . ssh-guest)
(name . "ssh-guest")
(version . "1")
(properties . (...)))
```
Promoted store metadata also now records compatibility fields derived from that executor object:
- `executor-kind`
- `executor-name`
- `executor-version`
### 2. Shared staged-result shape across executors
Both validated executor paths now converge on the same mutable staging layout under:
- `/var/lib/fruix/native-builds/<run-id>`
with promoted artifacts staged under:
- `artifacts/world`
- `artifacts/kernel`
- `artifacts/headers`
- `artifacts/bootloader`
and promotion metadata recorded in:
- `promotion.scm`
The heavy build work remains executor-specific, but the result shape is now shared.
### 3. Shared immutable promotion path
Both validated executor paths now promote through the same command:
```sh
fruix native-build promote RESULT_ROOT
```
That promotion creates immutable store identities for:
- `world`
- `kernel`
- `headers`
- `bootloader`
- result bundle
under `/frx/store/...`.
## Validated executor policies
### `self-hosted`
The self-hosted guest helper now emits the structured executor metadata directly from:
- `/usr/local/bin/fruix-self-hosted-native-build`
Validated self-hosted promotion flow:
- `PASS phase20-self-hosted-native-build-xcpng`
- `PASS phase20-native-build-store-promotion-xcpng`
Representative promoted result:
```text
result_store=/frx/store/0423193b9bd5e652bdb9d94d077e40dfcc3e9e78-fruix-native-build-result-15.0-STABLE-guest-self-hosted
world_store=/frx/store/5f67e95058186147206ff6f5da2243a09212e358-fruix-native-world-15.0-STABLE-guest-self-hosted
kernel_store=/frx/store/3a32797cc187a90e8273f205eababae6246568d9-fruix-native-kernel-15.0-STABLE-guest-self-hosted
headers_store=/frx/store/1d54d9814461f003e91add8cd37e94ac5f3d04ce-fruix-native-headers-15.0-STABLE-guest-self-hosted
bootloader_store=/frx/store/49f4885f0f05a324ac826f2618d4c7a923ca30d2-fruix-native-bootloader-15.0-STABLE-guest-self-hosted
```
### `ssh-guest`
The host-initiated guest path now also stages a shared native-build result root under:
- `/var/lib/fruix/native-builds/<run-id>`
instead of stopping at temporary build directories alone.
Its promotion metadata records executor policy as:
- `kind = ssh-guest`
- `name = ssh-guest`
- `version = 1`
with executor properties including:
- `transport = ssh`
- `orchestrator = host`
- guest addressing / VM identity metadata
Validated host-initiated promotion flow:
- `PASS phase20-host-initiated-native-build-xcpng`
- `PASS phase20-host-initiated-native-build-store-promotion-xcpng`
Representative promoted result:
```text
result_store=/frx/store/ffe44f5d1ba576e1f811ad3fe3a526a242b5c4a5-fruix-native-build-result-15.0-STABLE-ssh-guest
world_store=/frx/store/89c7a71c3df148a1f99b13d57fd6be88243eb2cb-fruix-native-world-15.0-STABLE-ssh-guest
kernel_store=/frx/store/93bac81122022b40438d356146a6854b4ee48513-fruix-native-kernel-15.0-STABLE-ssh-guest
headers_store=/frx/store/dd7f39f526bca4849caf1eaf96ae25d29b43493c-fruix-native-headers-15.0-STABLE-ssh-guest
bootloader_store=/frx/store/78b1c6b0b5c0c2c1549f5f42f3d64b6d9293669b-fruix-native-bootloader-15.0-STABLE-ssh-guest
```
## Important result
The same declared source and artifact contract now works across two different executor policies:
- `ssh-guest`
- `self-hosted`
while preserving the same Fruix-native staging/promotion split:
- mutable result roots under `/var/lib/fruix/native-builds/...`
- immutable promoted identities under `/frx/store/...`
That is the core architectural win of the executor model.
## What is not yet fully unified
The `host` executor kind now exists in the model, but the fully shared staged-result-plus-promotion workflow is not yet wired through the existing host-local native build path.
So the executor model is introduced and real-VM validated across two policies, but not yet uniformly productized across every native-build entry point.
## Result
Fruix now treats native-build placement as executor policy rather than as an architectural fork.
That means the next step is no longer to invent another build path from scratch.
It is to keep extending the same executor/result/promotion model so additional execution policies can plug into the same Fruix-native object story.

190
docs/reports/phase20-native-build-store-promotion-freebsd.md Normal file
View File

@@ -0,0 +1,190 @@
# Post-Phase 20: native build result promotion into first-class Fruix store objects
Date: 2026-04-05
## Goal
Make native FreeBSD base-build results feel like real Fruix objects instead of stopping at mutable staged files under:
- `/var/lib/fruix/native-builds/...`
The desired model is:
- `/var/lib/fruix/native-builds/...` remains a staging/result area
- `/frx/store/...` remains the real immutable identity
Validated artifact identities:
- `world`
- `kernel`
- `headers`
- `bootloader`
## What changed
### Promotion metadata in guest result roots
The guest self-hosted helper now emits a promotion description file at:
- `/var/lib/fruix/native-builds/<run-id>/promotion.scm`
That metadata records at least:
- executor / executor-version
- run-id / guest-host-name
- closure path
- development profile path
- declared FreeBSD base metadata
- source store provenance
- build policy
- artifact entries for:
- `world`
- `kernel`
- `headers`
- `bootloader`
The helper also stages a promotable `world` artifact tree in addition to the already validated narrower artifacts.
### Host-side promotion API
Fruix now exports:
- `promote-native-build-result`
and the CLI now exposes:
- `fruix native-build promote RESULT_ROOT [--store DIR]`
### Promoted store object layout
Promotion now creates immutable store objects for:
- `/frx/store/...-fruix-native-world-...`
- `/frx/store/...-fruix-native-kernel-...`
- `/frx/store/...-fruix-native-headers-...`
- `/frx/store/...-fruix-native-bootloader-...`
Each promoted artifact store records:
- `.fruix-native-build-object.scm`
- `.references`
Promotion also creates a result-bundle store object:
- `/frx/store/...-fruix-native-build-result-...`
That bundle records:
- `.fruix-native-build-result.scm`
- `artifacts/world`
- `artifacts/kernel`
- `artifacts/headers`
- `artifacts/bootloader`
where the `artifacts/*` entries are symlinks to the promoted artifact stores.
### Identity policy
Artifact identity is now based on Fruix metadata plus a tree-content signature of the staged artifact tree.
That means promotion identity depends on both:
- the explicit Fruix-native build metadata
- the actual content of the promoted artifact tree
## Validation harness
Added:
- `tests/system/run-phase20-native-build-store-promotion-xcpng.sh`
This harness:
1. boots the approved real XCP-ng guest path
2. runs the validated in-guest self-hosted native build helper
3. imports the guest result root back to the host
4. runs `fruix native-build promote`
5. verifies promoted store paths, metadata, symlink structure, and representative hashes
## Validation
Passing run:
- `PASS phase20-native-build-store-promotion-xcpng`
- workdir: `/tmp/current-phase20-native-build-store-promotion-xcpng`
Approved real XCP-ng path:
- VM `90490f2e-e8fc-4b7a-388e-5c26f0157289`
- VDI `0f1f90d3-48ca-4fa2-91d8-fc6339b95743`
Representative metadata:
```text
run_id=20260405T213444Z
source_store=/frx/store/12d7704362e95afc2697db63f168b878e082b372-freebsd-source-default
guest_result_root=/var/lib/fruix/native-builds/20260405T213444Z
result_store=/frx/store/c6329a0053720b05aff3274b8b1d522c909f475d-fruix-native-build-result-15.0-STABLE-guest-self-hosted
world_store=/frx/store/dfe37b36f6537a95ceea16ea62001b2ca5617eb7-fruix-native-world-15.0-STABLE-guest-self-hosted
kernel_store=/frx/store/0ab7cbceca240ab2c3b91e83e059844ea792e49e-fruix-native-kernel-15.0-STABLE-guest-self-hosted
headers_store=/frx/store/5bbeae9266687a229f1c6d176a08886c35243ff0-fruix-native-headers-15.0-STABLE-guest-self-hosted
bootloader_store=/frx/store/bd49a508bd7a3b94a2535d6774f31c993c406552-fruix-native-bootloader-15.0-STABLE-guest-self-hosted
artifact_store_count=4
sha_kernel=16950f116a52134b98e2f8e0dacc556e18fe254e4a0ac2c1741422dde281a341
sha_loader=ea417846167ece270ada611624dca622ca38bd30125b9a125cd8ebb8b3600313
sha_param=9eb140ca7d9666f3d484a4174c9acd94b45427db6292b4e17de19af2c6aa5219
promoted_kernel_sha=16950f116a52134b98e2f8e0dacc556e18fe254e4a0ac2c1741422dde281a341
promoted_loader_sha=ea417846167ece270ada611624dca622ca38bd30125b9a125cd8ebb8b3600313
promoted_param_sha=9eb140ca7d9666f3d484a4174c9acd94b45427db6292b4e17de19af2c6aa5219
native_build_store_promotion=ok
```
Validated facts:
- guest self-hosted native-build results remain available under `/var/lib/fruix/native-builds/...` as mutable staging/results
- those staged results are now sufficient to promote first-class Fruix identities on the host
- promotion creates four immutable artifact store objects plus one immutable result-bundle store object
- promoted metadata retains executor/source/build-policy/provenance information explicitly
- promoted kernel, bootloader, and headers hashes match the already validated staged artifacts
- the promoted `world` artifact is now also preserved as a first-class Fruix store object instead of remaining only as an unpromoted staged tree
## Important implementation note
The first full promotion attempt exposed an incorrect assumption in the validation surface:
- the promoted/staged `world` artifact should be checked at:
- `bin/sh`
- not at:
- `usr/bin/sh`
That path expectation was corrected in:
- helper validation
- emitted promotion metadata
- the end-to-end promotion harness
## Result
Fruix now has a validated native-build object model with a clear split:
- mutable native-build result roots under `/var/lib/fruix/native-builds/...`
- immutable promoted identities under `/frx/store/...`
That makes native FreeBSD base builds feel substantially more Fruix-native:
- build outputs have explicit immutable identities
- metadata is Fruix-native rather than implied only by ad hoc directory layout
- executor/source/provenance/build-policy remain attached to the promoted result
- the staged result area and the real store identity are now intentionally distinct
## Next direction
This suggests the next product step is not merely “more self-hosting”.
It is to generalize this result model so different execution modes can converge on the same promoted object story, for example:
- host-initiated in-guest builds
- guest self-hosted builds
- future executor variants
That would move Fruix closer to a shared executor model rather than treating each validation path as a one-off harness.

201
docs/reports/phase20-self-hosted-native-builds-freebsd.md Normal file
View File

@@ -0,0 +1,201 @@
# Phase 20.3: controlled guest self-hosted native base-build prototype
Date: 2026-04-05
## Goal
Reassess guest self-hosting now that Fruix has already completed the earlier source, installation, generation-layout, rollback, development-overlay, and host-initiated in-guest native-build steps.
Phase 20.3 asked for real evidence about:
- what self-hosting would improve
- what it would cost in complexity
- how it fits with the Fruix source/deployment model already in place
## What changed
### New in-guest helper
Development-enabled systems now also ship:
- `/usr/local/bin/fruix-self-hosted-native-build`
This helper performs a controlled in-guest native FreeBSD base build using the system's own declared materialized source store recorded in:
- `/run/current-system/metadata/store-layout.scm`
The helper:
1. verifies the development overlay is present
2. verifies the canonical compatibility links exist:
- `/usr/include`
- `/usr/share/mk`
3. recovers the materialized FreeBSD source store from current-system metadata
4. runs:
- `buildworld`
- `buildkernel`
- `installworld`
- `distribution`
- `installkernel`
5. stages narrower artifact outputs under:
- `/var/lib/fruix/native-builds/<run-id>/artifacts/`
6. records metadata and status under:
- `/var/lib/fruix/native-builds/<run-id>/`
- `/var/lib/fruix/native-builds/latest`
The heavy object/stage work stays under:
- `/var/tmp/fruix-self-hosted-native-builds/<run-id>`
so the installed-system result area remains smaller and more legible.
### Important environment fix discovered during prototyping
The first prototype attempt failed even though Phase 20.2 had already succeeded.
Cause:
- directly evaluating `fruix-development-environment` before `buildworld` exported development-oriented variables like:
- `MAKEFLAGS`
- `CPPFLAGS`
- `CFLAGS`
- `CXXFLAGS`
- `LDFLAGS`
- those are appropriate for smaller development builds, but they polluted FreeBSD's world/kernel bootstrap environment and broke the LLVM bootstrap phase
Representative failure:
- missing generated LLVM config headers during bootstrap (`llvm/Config/abi-breaking.h`)
The validated fix was to make the self-hosted helper explicitly sanitize that environment first:
- reset `PATH` to the normal base paths
- unset development-shell variables such as:
- `MAKEFLAGS`
- `CC`, `CXX`, `AR`, `RANLIB`, `NM`
- `CPPFLAGS`, `CFLAGS`, `CXXFLAGS`, `LDFLAGS`
- `FRUIX_DEVELOPMENT_*`
- `FRUIX_*` tool variables
So the final 20.3 result is not “just reuse the development shell wholesale”.
It is more precise:
- use the development overlay for canonical paths and available content
- but run the real base-build steps in a cleaner, purpose-built helper environment
### Closure invalidation
To ensure the updated helper actually affects generated system closures, the operating-system closure spec now also records helper-version markers for development-enabled systems.
That ensures guest images pick up helper changes instead of silently reusing an older cached closure path.
### Validation harness
Added:
- `tests/system/run-phase20-self-hosted-native-build-xcpng.sh`
This harness:
1. boots the validated development-enabled Fruix guest on the approved XCP-ng path
2. verifies the new helper exists in the guest
3. invokes the helper from inside the guest
4. verifies the recorded result/status/`latest` pointer
5. validates the resulting staged artifact metadata and hashes
## Validation
Passing run:
- `PASS phase20-self-hosted-native-build-xcpng`
- workdir: `/tmp/fruix-phase20-self-hosted-native-build-xcpng`
Validated on the approved real XCP-ng path:
- VM `90490f2e-e8fc-4b7a-388e-5c26f0157289`
- VDI `0f1f90d3-48ca-4fa2-91d8-fc6339b95743`
Representative metadata:
```text
run_id=20260405T150359Z
helper_version=2
build_jobs=8
source_store=/frx/store/12d7704362e95afc2697db63f168b878e082b372-freebsd-source-default
build_root=/var/tmp/fruix-self-hosted-native-builds/20260405T150359Z
result_root=/var/lib/fruix/native-builds/20260405T150359Z
latest_link=/var/lib/fruix/native-builds/latest
latest_target=/var/lib/fruix/native-builds/20260405T150359Z
status_value=ok
build_root_size=7.5G
result_root_size=343M
kernel_artifact_size=158M
headers_artifact_size=32M
bootloader_artifact_size=1.3M
sha_kernel=16950f116a52134b98e2f8e0dacc556e18fe254e4a0ac2c1741422dde281a341
sha_loader=ea417846167ece270ada611624dca622ca38bd30125b9a125cd8ebb8b3600313
sha_param=9eb140ca7d9666f3d484a4174c9acd94b45427db6292b4e17de19af2c6aa5219
self_hosted_native_build=ok
```
Validated facts:
- the development-enabled Fruix guest can now run a controlled self-hosted native base-build helper from inside the installed system itself
- the helper can recover the declared source store from current-system metadata without host-side parsing
- `buildworld` and `buildkernel` succeed in the guest
- staged `installworld`, `distribution`, and `installkernel` succeed in the guest
- the helper records a stable result directory and `latest` pointer under:
- `/var/lib/fruix/native-builds`
- the resulting artifact hashes match the earlier validated Phase 20.2 host-initiated in-guest path
## What self-hosting improved
The prototype demonstrates a few real improvements:
- the build recipe itself now lives inside the Fruix-managed system, not only in a host-side SSH harness
- the guest can derive its own declared source input from current-system metadata
- result/state recording now has a Fruix-native installed-system location:
- `/var/lib/fruix/native-builds`
- the host no longer needs to spell out every `make` phase just to validate the in-guest path
## What it cost in complexity
The prototype also made the extra complexity visible:
- the guest helper needs its own controlled environment contract
- a naive reuse of the development-shell exports was wrong for real `buildworld`
- helper-version invalidation had to be made explicit so closure caching would not hide helper changes
- the in-guest result/staging model now needs its own operator-facing conventions
So the experiment did not eliminate complexity.
It mostly moved some of it from the host harness into an explicit in-guest helper contract.
## Decision after the prototype
Phase 20.3 is complete because Fruix now has a **first controlled guest self-hosted native base-build prototype**.
However, the evidence does **not** suggest replacing the Phase 20.2 path as the default operator workflow yet.
The current recommendation is:
- keep the **host-initiated in-guest native-build path** as the simpler default validation and orchestration flow
- keep the new **self-hosted helper** as a controlled prototype and stepping stone toward deeper guest-driven workflows
That fits the existing Fruix model well:
- source identity still comes from declared store-backed metadata
- deployment identity still comes from immutable closures under `/frx/store`
- the guest-side prototype adds a narrower in-system build/result workflow without replacing the existing deployment story
## Result
Phase 20.3 is complete.
Fruix now has:
- a validated host-orchestrated in-guest native base-build workflow
- and a validated first controlled guest self-hosted native base-build prototype
That answers the Phase 20.3 question with real evidence instead of only prior caution.

223
docs/reports/postphase20-build-profile-separation-freebsd.md Normal file
View File

@@ -0,0 +1,223 @@
# Post-Phase 20: separate development from native base-build environment
Date: 2026-04-06
## Goal
Tighten the distinction that Phase 20.3 exposed in practice:
- an interactive development shell is not the same thing as a reliable native base-build environment
Fruix now models three layers instead of two:
- runtime profile
- development profile
- build profile
## Why this change was needed
The earlier self-hosted native-build prototype proved that simply reusing the exported development environment for `buildworld` / `buildkernel` was too loose.
Development-oriented exports like:
- `MAKEFLAGS`
- `CPPFLAGS`
- `CFLAGS`
- `CXXFLAGS`
- `LDFLAGS`
are useful for interactive compilation work, but they are the wrong contract for a real FreeBSD world/kernel bootstrap.
Phase 20.3 previously worked around that by manually sanitizing the shell before running the build.
This change makes that separation explicit in the product model instead of keeping it as an ad hoc helper detail.
## Implementation
### New operating-system layer
`modules/fruix/system/freebsd/model.scm` now supports:
- `#:build-packages`
- `operating-system-build-packages`
So a build-capable system can now carry both:
- `development-packages`
- `build-packages`
separately.
### New build profile inside the system closure
When `build-packages` is non-empty, Fruix now materializes:
- `/frx/store/...-fruix-system-.../build-profile`
alongside the existing:
- `/frx/store/...-fruix-system-.../profile`
- `/frx/store/...-fruix-system-.../development-profile`
Store-layout metadata now records both development-package stores and build-package stores explicitly.
### New in-guest build helper
Build-capable systems now ship:
- `/usr/local/bin/fruix-build-environment`
and expose a stable runtime link:
- `/run/current-build -> /run/current-system/build-profile`
That helper intentionally emits a stricter environment contract than the interactive development helper.
It clears development-shell variables such as:
- `MAKEOBJDIRPREFIX`
- `MAKEFLAGS`
- `CC`
- `CXX`
- `AR`
- `RANLIB`
- `NM`
- `CPPFLAGS`
- `CFLAGS`
- `CXXFLAGS`
- `LDFLAGS`
and also clears development-helper exports such as:
- `FRUIX_DEVELOPMENT_PROFILE`
- `FRUIX_CC`
- `FRUIX_CXX`
- `FRUIX_AR`
- `FRUIX_RANLIB`
- `FRUIX_NM`
- `FRUIX_BMAKE`
Then it exports build-specific values including at least:
- `FRUIX_BUILD_PROFILE`
- `FRUIX_BUILD_INCLUDE`
- `FRUIX_BUILD_SHARE_MK`
- `FRUIX_BUILD_BIN`
- `FRUIX_BUILD_USR_BIN`
- `FRUIX_BUILD_CC`
- `FRUIX_BUILD_CXX`
- `FRUIX_BUILD_AR`
- `FRUIX_BUILD_RANLIB`
- `FRUIX_BUILD_NM`
- `FRUIX_BMAKE`
- `PATH`
Intended use:
```sh
eval "$(/usr/local/bin/fruix-build-environment)"
```
### Canonical build compatibility links now come from the build profile
For native base-build compatibility, build-capable systems now expose:
- `/usr/include -> /run/current-system/build-profile/usr/include`
- `/usr/share/mk -> /run/current-system/build-profile/usr/share/mk`
This means the running system can still keep development and build content separate while offering the canonical paths that FreeBSD native build machinery expects.
### Self-hosted native-build helper now uses the build helper contract
`/usr/local/bin/fruix-self-hosted-native-build` now:
- requires `build-profile`
- requires `/usr/local/bin/fruix-build-environment`
- evaluates the build helper contract before `buildworld`
- verifies the canonical compatibility links point at `build-profile`
This replaces the earlier approach where the helper had to reconstruct sanitization manually around a development-oriented environment.
### Promotion metadata now records build profile provenance
Promotion metadata emitted for self-hosted native-build results now records:
- `build-profile`
Promoted artifact/result metadata also now preserves that field.
## Validation
Validated on the approved real XCP-ng path:
- VM `90490f2e-e8fc-4b7a-388e-5c26f0157289`
- VDI `0f1f90d3-48ca-4fa2-91d8-fc6339b95743`
Passing runs:
- `PASS phase20-development-environment-xcpng`
- `PASS phase20-self-hosted-native-build-xcpng`
- `PASS phase20-native-build-store-promotion-xcpng`
- `PASS phase20-host-initiated-native-build-xcpng`
- `PASS phase20-host-initiated-native-build-store-promotion-xcpng`
## Representative validated results
Development/build environment validation:
```text
development_profile_guest=/run/current-system/development-profile
build_profile_guest=/run/current-system/build-profile
development_env_script=/frx/store/...-fruix-system-.../usr/local/bin/fruix-development-environment
build_env_script=/frx/store/...-fruix-system-.../usr/local/bin/fruix-build-environment
development_environment=ok
```
Self-hosted native-build validation:
```text
helper_version=5
executor_kind=self-hosted
executor_name=guest-self-hosted
executor_version=5
build_profile=/run/current-system/build-profile
source_store=/frx/store/12d7704362e95afc2697db63f168b878e082b372-freebsd-source-default
self_hosted_native_build=ok
```
Promotion validation:
```text
executor_kind=self-hosted
executor_name=guest-self-hosted
executor_version=5
result_store=/frx/store/3ce6aefd564bc51f2465dcbb5c261355be4c7076-fruix-native-build-result-15.0-STABLE-guest-self-hosted
native_build_store_promotion=ok
```
Host-initiated path revalidation with the new build-profile split also succeeded:
```text
executor_kind=ssh-guest
executor_name=ssh-guest
executor_version=1
result_store=/frx/store/93a4837d2588acfde2262010b296df9c7b7b367a-fruix-native-build-result-15.0-STABLE-ssh-guest
host_initiated_native_build_store_promotion=ok
```
## Result
Fruix now expresses an important product distinction more honestly:
- development is for interactive work
- build is for a stricter native base-build contract
That reduces special-case cleanup inside the self-hosted helper and gives Fruix a clearer path for future operator-facing commands such as:
- `fruix system build-base`
- `fruix system upgrade`
because the system model now has an explicit place to say:
- what is available for interactive development
- what is available for real native base builds

178
docs/reports/postphase20-installed-node-management-freebsd.md Normal file
View File

@@ -0,0 +1,178 @@
# Post-Phase 20: installed systems as real Fruix nodes
Date: 2026-04-06
## Goal
Make a Fruix-installed machine feel like a managed Fruix node instead of only a deployed image with switch/rollback helpers.
The immediate target was not yet the full long-term command surface of:
- `fruix system upgrade`
- `fruix system build-base`
- `fruix system deploy`
but it was enough to cross an important product threshold:
- installed nodes can now remember their own declaration inputs
- installed nodes can build from those inputs locally
- installed nodes can reconfigure themselves into a newly built generation
- installed nodes can still roll back cleanly
## What changed
### Canonical declaration state now ships inside the system closure
Fruix system closures now carry explicit declaration metadata:
- `metadata/system-declaration.scm`
- `metadata/system-declaration-info.scm`
- `metadata/system-declaration-system`
That gives an installed system a canonical local answer to:
- what declaration source produced me?
- what top-level system variable should be used?
This declaration metadata is also recorded through the installed generation layout metadata.
### Bundled Fruix node CLI sources now ship inside the closure
Installed system closures now also carry a self-contained Fruix node CLI source bundle under:
- `share/fruix/node/scripts/fruix.scm`
- `share/fruix/node/modules/...`
- `share/fruix/node/guix/guix/build/utils.scm`
This gives the installed node enough local Fruix/Guix Scheme source to run Fruix system actions from the node itself.
### Installed `fruix` helper gained local build/reconfigure support
The installed helper at:
- `/usr/local/bin/fruix`
now supports:
- `fruix system build`
- `fruix system reconfigure`
- `fruix system status`
- `fruix system switch`
- `fruix system rollback`
Current behavior:
- `fruix system build` with no extra arguments uses:
- `/run/current-system/metadata/system-declaration.scm`
- `/run/current-system/metadata/system-declaration-system`
- `fruix system reconfigure` with no extra arguments builds from that same embedded declaration and then stages a switch to the resulting closure
- both commands can also take an explicit declaration file plus `--system NAME`, using the same general CLI shape as the host-side Fruix frontend
### In-node builds now reuse the installed Fruix runtime stores
A crucial implementation fix was needed here.
An installed node should not try to reconstruct the Guile/Shepherd runtime prefixes from host-side `/tmp/...` build roots or host `/usr/local/lib/...` assumptions.
Instead, in-node Fruix builds now explicitly reuse the installed runtime stores already referenced by the current system closure.
That allows the in-node build path to work on the real installed system rather than depending on build-host-only paths.
## Validation harness
Added:
- `tests/system/postphase20-installed-node-operating-system.scm.in`
- `tests/system/run-postphase20-installed-node-build-reconfigure-xcpng.sh`
This harness validates the new installed-node workflow on the approved real XCP-ng path.
## Validation performed
Approved real XCP-ng path:
- VM `90490f2e-e8fc-4b7a-388e-5c26f0157289`
- VDI `0f1f90d3-48ca-4fa2-91d8-fc6339b95743`
Promoted native-base result consumed by the installed node declaration:
- `/frx/store/ffe44f5d1ba576e1f811ad3fe3a526a242b5c4a5-fruix-native-build-result-15.0-STABLE-ssh-guest`
Validated flow:
1. boot a Fruix-installed node built from the promoted native-base result
2. confirm the installed node exposes:
- embedded declaration metadata
- bundled node CLI sources
3. run in-guest:
- `fruix system build`
using the node's own embedded declaration inputs
4. copy a candidate declaration to the guest
5. run in-guest:
- `fruix system build /root/candidate.scm --system postphase20-installed-node-operating-system`
6. run in-guest:
- `fruix system reconfigure /root/candidate.scm --system postphase20-installed-node-operating-system`
7. reboot and verify the candidate generation boots
8. run in-guest:
- `fruix system rollback`
9. reboot and verify the original generation boots again
Passing run:
- `PASS postphase20-installed-node-build-reconfigure-xcpng`
Representative metadata:
```text
closure_path=/frx/store/cd4e52d8bff348953939401c8623d4189d7c9432-fruix-system-fruix-node-current
current_built_closure=/frx/store/18ee10925a15b48c676463a3359c45ff766e16a0-fruix-system-fruix-node-current
candidate_closure=/frx/store/46fc9631faf556c30a1a5f39f718d5d38a3f6ba8-fruix-system-fruix-node-canary
reconfigure_closure=/frx/store/46fc9631faf556c30a1a5f39f718d5d38a3f6ba8-fruix-system-fruix-node-canary
reconfigure_current_generation=2
reconfigure_current_closure=/frx/store/46fc9631faf556c30a1a5f39f718d5d38a3f6ba8-fruix-system-fruix-node-canary
reconfigure_rollback_generation=1
reconfigure_rollback_closure=/frx/store/cd4e52d8bff348953939401c8623d4189d7c9432-fruix-system-fruix-node-current
candidate_hostname=fruix-node-canary
rollback_current_generation=1
rollback_current_closure=/frx/store/cd4e52d8bff348953939401c8623d4189d7c9432-fruix-system-fruix-node-current
rollback_rollback_generation=2
rollback_rollback_closure=/frx/store/46fc9631faf556c30a1a5f39f718d5d38a3f6ba8-fruix-system-fruix-node-canary
rollback_hostname=fruix-node-current
installed_node_build_reconfigure=ok
```
## Important observation
The no-argument in-node build of the current declaration did not necessarily reproduce the exact original current closure path.
That is expected with the current model because closure metadata still records builder-local provenance such as the current build host context.
So the significant validated fact is not strict bit-for-bit closure reuse.
It is that the installed node can now:
- read its own declaration inputs locally
- build a valid local candidate closure from them
- build a different candidate declaration locally
- reconfigure itself into that candidate generation
- boot the candidate generation
- roll back and boot the prior generation again
## Result
Fruix-installed machines are now meaningfully closer to real Fruix nodes.
They no longer only support:
- `status`
- `switch`
- `rollback`
They now also support a validated local node workflow for:
- reading embedded declaration inputs
- building a local candidate closure
- reconfiguring into that locally built candidate
- rolling back through the same installed generation model
This is the first concrete step toward a fuller operator-facing Fruix node command surface.

165
docs/reports/postphase20-promoted-native-base-declarations-freebsd.md Normal file
View File

@@ -0,0 +1,165 @@
# Post-Phase 20: promoted native base result sets as declaration inputs
Date: 2026-04-06
## Goal
Move Fruix from:
- “the guest built a kernel”
to:
- “Fruix materialized a native-base result set with identity X”
and then prove that a normal Fruix system declaration can consume that promoted result set directly.
Concretely, the desired product behavior was:
1. build runs somewhere
2. result is recorded in a staging/result root
3. Fruix validates the result shape and metadata
4. Fruix promotes it into store objects
5. system declarations can refer to those promoted artifacts
Before this step, Fruix already had validated steps 1 through 4.
This step implemented and validated step 5.
## What changed
### First-class promoted result reference in system declarations
Fruix now has a first-class promoted native-build result reference object.
Current declaration-level entry points are:
- `promoted-native-build-result`
- `operating-system-from-promoted-native-build-result`
A declaration can now point at a promoted native-build result bundle in `/frx/store`, for example:
```scheme
(define promoted
(promoted-native-build-result
#:store-path "/frx/store/...-fruix-native-build-result-..."))
(define os
(operating-system-from-promoted-native-build-result
promoted
...))
```
### Promoted result bundles now drive system composition
From a promoted result bundle, Fruix now derives:
- the effective FreeBSD base metadata used by the system declaration
- kernel package identity from the promoted kernel artifact store
- bootloader package identity from the promoted bootloader artifact store
- base-package identity from the promoted world artifact store
- optional development-package identity from the promoted headers artifact store
This means a system declaration can now be based on a promoted native-build result set rather than only on source-driven base-package reconstruction.
### Closure metadata now records promoted-result provenance explicitly
When a system declaration uses a promoted native-build result set, Fruix now records that explicitly in the resulting closure metadata.
New closure metadata includes:
- `metadata/promoted-native-build-result.scm`
- `metadata/store-layout.scm` entries for:
- promoted result summary
- promoted artifact stores
The system closure also retains the promoted result bundle itself as an explicit reference in:
- `/frx/store/...-fruix-system-.../.references`
So the resulting deployed closure does not merely happen to contain files copied from a promoted artifact; it explicitly records which promoted result set it came from.
## Implementation notes
The key plumbing additions were:
- a promoted native-build result record in:
- `modules/fruix/system/freebsd/model.scm`
- promoted-result readers/helpers in:
- `modules/fruix/system/freebsd/build.scm`
- direct system-construction helper:
- `operating-system-from-promoted-native-build-result`
- support for consuming promoted artifact stores directly when materializing packages
- closure/store-layout metadata wiring in:
- `modules/fruix/system/freebsd/media.scm`
The important product-level effect is that promoted native-build results are no longer only post-build artifacts. They are now declaration inputs.
## Validation harness
Added:
- `tests/system/phase20-promoted-native-base-operating-system.scm.in`
- `tests/system/run-phase20-promoted-native-base-declaration-xcpng.sh`
The harness supports two modes:
- use an explicit `RESULT_STORE=/frx/store/...-fruix-native-build-result-...`
- or, if no result store is supplied, first run the validated host-initiated promotion path and consume that result
## Validation performed
Validated on the approved real XCP-ng path using the promoted `ssh-guest` result bundle:
- result bundle:
- `/frx/store/ffe44f5d1ba576e1f811ad3fe3a526a242b5c4a5-fruix-native-build-result-15.0-STABLE-ssh-guest`
- VM:
- `90490f2e-e8fc-4b7a-388e-5c26f0157289`
- VDI:
- `0f1f90d3-48ca-4fa2-91d8-fc6339b95743`
Passing run:
- `PASS phase20-promoted-native-base-declaration-xcpng`
Representative metadata:
```text
closure_path=/frx/store/ac1e6dcfe67d3cde49d4fd5da97740f6244276b4-fruix-system-fruix-freebsd
result_store=/frx/store/ffe44f5d1ba576e1f811ad3fe3a526a242b5c4a5-fruix-native-build-result-15.0-STABLE-ssh-guest
executor_kind=ssh-guest
executor_name=ssh-guest
executor_version=1
world_store=/frx/store/89c7a71c3df148a1f99b13d57fd6be88243eb2cb-fruix-native-world-15.0-STABLE-ssh-guest
kernel_store=/frx/store/93bac81122022b40438d356146a6854b4ee48513-fruix-native-kernel-15.0-STABLE-ssh-guest
headers_store=/frx/store/dd7f39f526bca4849caf1eaf96ae25d29b43493c-fruix-native-headers-15.0-STABLE-ssh-guest
bootloader_store=/frx/store/78b1c6b0b5c0c2c1549f5f42f3d64b6d9293669b-fruix-native-bootloader-15.0-STABLE-ssh-guest
kernel_link=/frx/store/93bac81122022b40438d356146a6854b4ee48513-fruix-native-kernel-15.0-STABLE-ssh-guest/boot/kernel/kernel
bootloader_link=/frx/store/78b1c6b0b5c0c2c1549f5f42f3d64b6d9293669b-fruix-native-bootloader-15.0-STABLE-ssh-guest/boot/loader.efi
guest_kernel_link=/frx/store/93bac81122022b40438d356146a6854b4ee48513-fruix-native-kernel-15.0-STABLE-ssh-guest/boot/kernel/kernel
guest_bootloader_link=/frx/store/78b1c6b0b5c0c2c1549f5f42f3d64b6d9293669b-fruix-native-bootloader-15.0-STABLE-ssh-guest/boot/loader.efi
guest_uname=FreeBSD 15.0-STABLE
promoted_native_base_declaration=ok
```
## Validated facts
The real-VM validation proved that Fruix can now:
- consume a promoted native-build result bundle directly from a declaration
- materialize a normal system closure from that promoted identity
- retain explicit promoted-result provenance in closure metadata
- boot the resulting system successfully on the approved XCP-ng path
- keep the closure's kernel and bootloader linked directly to the promoted artifact stores
- preserve the promoted result-bundle store itself as an explicit closure reference
So the operator-facing story is now materially better:
- not only “a build happened somewhere”
- but “this system declaration is based on promoted native-base result X”
## Result
Promoted native-build result sets are now first-class Fruix declaration inputs.
That is the point where the native FreeBSD base path stops being only a build/promotion harness and starts acting more like real product behavior.

456
docs/system-deployment-workflow.md
View File

@@ -1,6 +1,6 @@
# Fruix system deployment workflow # Fruix system deployment workflow
Date: 2026-04-04 Date: 2026-04-06
## Purpose ## Purpose
@@ -11,9 +11,10 @@ This document defines the current canonical Fruix workflow for:
- installing a declarative system onto an image or disk - installing a declarative system onto an image or disk
- booting through installer media - booting through installer media
- rolling forward to a candidate system - rolling forward to a candidate system
- rolling back to an earlier declared system - switching an installed system to a staged candidate generation
- rolling an installed system back to an earlier recorded generation
This is the Phase 19.1 operator-facing view of the system model already implemented in earlier phases. This is now the Phase 19 operator-facing view of the system model as validated through explicit installed-system generation switching and rollback.
## Core model ## Core model
@@ -168,6 +169,361 @@ Use this when you want to:
- install from an ISO-attached Fruix environment - install from an ISO-attached Fruix environment
- test the same install model on more realistic VM paths - test the same install model on more realistic VM paths
### Installed-system generation commands
Installed Fruix systems now also ship a small in-guest deployment helper at:
- `/usr/local/bin/fruix`
Current validated in-guest commands are:
```sh
fruix system build
fruix system reconfigure
fruix system status
fruix system switch /frx/store/...-fruix-system-...
fruix system rollback
```
Installed systems now carry canonical declaration state in:
- `/run/current-system/metadata/system-declaration.scm`
- `/run/current-system/metadata/system-declaration-info.scm`
- `/run/current-system/metadata/system-declaration-system`
So the in-guest helper can now build from the node's own embedded declaration inputs.
Current validated build/reconfigure behavior is:
- `fruix system build`
- with no extra arguments, builds from the embedded current declaration
- `fruix system reconfigure`
- with no extra arguments, builds from the embedded current declaration and stages a switch to the resulting closure
- both commands can also take an explicit declaration file plus `--system NAME`
Current intended usage now has two validated patterns.
### Pattern A: build elsewhere, then switch/rollback locally
1. build a candidate closure on the operator side with `./bin/fruix system build`
2. ensure that candidate closure is present on the installed target's `/frx/store`
3. run `fruix system switch /frx/store/...` on the installed system
4. reboot into the staged candidate generation
5. if needed, run `fruix system rollback`
6. reboot back into the recorded rollback generation
Important current limitation of this lower-level pattern:
- `fruix system switch` does **not** yet fetch or copy the candidate closure onto the target for you
- it assumes the selected closure is already present in the installed system's `/frx/store`
### Pattern B: build and reconfigure from the node itself
1. inspect or edit the node declaration inputs
- embedded current declaration, or
- an explicit replacement declaration file
2. run:
```sh
fruix system build
```
or:
```sh
fruix system build /path/to/candidate.scm --system my-operating-system
```
3. stage a local generation update with:
```sh
fruix system reconfigure
```
or:
```sh
fruix system reconfigure /path/to/candidate.scm --system my-operating-system
```
4. reboot into the staged generation
5. if needed, run `fruix system rollback`
6. reboot back into the recorded prior generation
### In-guest development and build environments
Opt-in systems can now expose two separate overlays above the main runtime profile:
- development:
- `/run/current-system/development-profile`
- `/run/current-development`
- `/usr/local/bin/fruix-development-environment`
- build:
- `/run/current-system/build-profile`
- `/run/current-build`
- `/usr/local/bin/fruix-build-environment`
Intended use:
```sh
eval "$(/usr/local/bin/fruix-development-environment)"
```
for interactive development work, and:
```sh
eval "$(/usr/local/bin/fruix-build-environment)"
```
for a narrower native base-build contract.
The current split is:
- runtime profile
- development profile
- build profile
The development helper remains intentionally interactive and currently exposes at least:
- native headers under `usr/include`
- FreeBSD `share/mk` files for `bsd.*.mk`
- Clang toolchain commands such as `cc`, `c++`, `ar`, `ranlib`, and `nm`
- `MAKEFLAGS` pointing at the development profile's `usr/share/mk`
The build helper is intentionally more sanitized and less interactive. It clears development-shell variables such as:
- `MAKEFLAGS`
- `CPPFLAGS`
- `CFLAGS`
- `CXXFLAGS`
- `LDFLAGS`
and then exposes build-oriented paths such as:
- `FRUIX_BUILD_PROFILE`
- `FRUIX_BUILD_INCLUDE`
- `FRUIX_BUILD_SHARE_MK`
- `FRUIX_BUILD_CC`
- `FRUIX_BUILD_CXX`
- `FRUIX_BUILD_AR`
- `FRUIX_BUILD_RANLIB`
- `FRUIX_BUILD_NM`
- `FRUIX_BMAKE`
For native base-build compatibility, build-enabled systems now expose canonical links at:
- `/usr/include -> /run/current-system/build-profile/usr/include`
- `/usr/share/mk -> /run/current-system/build-profile/usr/share/mk`
So Fruix now separates interactive development support from the stricter environment used for `buildworld` / `buildkernel` style work, instead of treating them as one overlay.
### Host-initiated native base builds inside a Fruix-managed guest
The currently validated intermediate path toward self-hosting is still host-orchestrated.
The host:
1. boots a development-enabled Fruix guest
2. connects over SSH
3. recovers the materialized FreeBSD source store from system metadata
4. runs native FreeBSD build commands inside the guest
5. collects and records the staged outputs
The validated build sequence inside the guest is:
- `make -jN buildworld`
- `make -jN buildkernel`
- `make DESTDIR=... installworld`
- `make DESTDIR=... distribution`
- `make DESTDIR=... installkernel`
For staged install steps, the validated path uses:
- `DB_FROM_SRC=yes`
so the staged install is driven by the declared source tree's account database rather than by accidental guest-local `/etc/master.passwd` contents.
This is the current Phase 20.2 answer to “where should native base builds run?”
- **inside** a Fruix-managed FreeBSD environment
- but still with the **host** driving the outer orchestration loop
### Controlled guest self-hosted native-build prototype
Fruix now also has a narrower in-guest prototype helper at:
- `/usr/local/bin/fruix-self-hosted-native-build`
Intended use:
```sh
FRUIX_SELF_HOSTED_NATIVE_BUILD_JOBS=8 \
/usr/local/bin/fruix-self-hosted-native-build
```
That helper:
1. evaluates the build helper and verifies the build overlay plus canonical compatibility links
2. recovers the materialized FreeBSD source store from:
- `/run/current-system/metadata/store-layout.scm`
3. runs the native FreeBSD build/install phases inside the guest
4. records staged results under:
- `/var/lib/fruix/native-builds/<run-id>`
- `/var/lib/fruix/native-builds/latest`
5. emits promotion metadata for first-class artifact identities covering:
- `world`
- `kernel`
- `headers`
- `bootloader`
6. keeps the heavier object/stage work under:
- `/var/tmp/fruix-self-hosted-native-builds/<run-id>`
Important current detail:
- the self-hosted helper now uses the separate `fruix-build-environment` contract instead of reusing the interactive development helper wholesale
- that build helper intentionally clears development-shell exports such as `MAKEFLAGS`, `CPPFLAGS`, `CFLAGS`, `CXXFLAGS`, and `LDFLAGS` before `buildworld`
- this keeps the base-build path closer to the exact contract needed for real world/kernel bootstrap work
So the validated Phase 20.3 answer is:
- a controlled guest self-hosted base-build prototype now works
- but the simpler default operator flow should still be the Phase 20.2 host-initiated in-guest path unless there is a specific reason to push the build loop farther into the guest
### Promoting native-build results into first-class Fruix store objects
The guest-side result root is now explicitly a **staging/result area**, not the final immutable identity.
Current validated flow:
1. run the in-guest helper so the guest records a result under:
- `/var/lib/fruix/native-builds/<run-id>`
2. copy that result root back to the host
3. run:
```sh
fruix native-build promote RESULT_ROOT
```
The promotion step creates immutable `/frx/store` identities for:
- `world`
- `kernel`
- `headers`
- `bootloader`
and also creates a result-bundle store object that references those promoted artifact stores.
Current metadata split:
- mutable staging/result root:
- `/var/lib/fruix/native-builds/<run-id>`
- immutable artifact stores:
- `/frx/store/...-fruix-native-world-...`
- `/frx/store/...-fruix-native-kernel-...`
- `/frx/store/...-fruix-native-headers-...`
- `/frx/store/...-fruix-native-bootloader-...`
- immutable result bundle:
- `/frx/store/...-fruix-native-build-result-...`
The promoted store objects record explicit Fruix-native metadata including at least:
- executor kind / name / version
- run-id / guest-host-name
- closure path
- source store provenance
- build policy
- artifact kind
- required-file expectations
- recorded content signatures and hashes
This is the current Fruix-native answer to the question:
- where should mutable native-build state live?
- `/var/lib/fruix/native-builds/...`
- where should immutable native-build identity live?
- `/frx/store/...`
### Using promoted native-build results in system declarations
Fruix system declarations can now refer directly to a promoted native-build result bundle.
Current declaration-level helpers are:
- `promoted-native-build-result`
- `operating-system-from-promoted-native-build-result`
Representative pattern:
```scheme
(define promoted
(promoted-native-build-result
#:store-path "/frx/store/...-fruix-native-build-result-..."))
(define os
(operating-system-from-promoted-native-build-result
promoted
#:host-name "fruix-freebsd"
...))
```
That now gives Fruix a more product-like story:
1. a build runs under some executor policy
2. Fruix records the staged mutable result
3. Fruix promotes it into immutable store identities
4. a later system declaration can point at that promoted result identity
5. Fruix materializes and boots a normal system from that promoted identity
The resulting closure now records that provenance explicitly through:
- `metadata/promoted-native-build-result.scm`
- `metadata/store-layout.scm`
- closure references that retain the selected result-bundle store path
So the operator-facing statement is now:
- “this Fruix system is based on promoted native-base result X”
not only:
- “some earlier build happened and its files were copied somewhere.”
### Native-build executor model
Fruix now has an explicit executor model for native base builds.
Current executor kinds are:
- `host`
- `ssh-guest`
- `self-hosted`
and the intended future extension points are:
- `jail`
- `remote-builder`
The important change is architectural:
- declared source identity stays the same
- expected artifact kinds stay the same
- result/promotion metadata shape stays the same
- only the executor policy changes
So “where the build runs” is now treated as executor policy rather than as a separate native-build architecture each time.
Current end-to-end validated executors for the staged-result-plus-promotion model are:
- `ssh-guest`
- `self-hosted`
Both now converge on the same Fruix-native flow:
1. run the build under a selected executor
2. stage a result root under `/var/lib/fruix/native-builds/...`
3. emit the same promotion/provenance shape
4. promote the result into immutable `/frx/store/...` objects
## Deployment patterns ## Deployment patterns
### 1. Build-first workflow ### 1. Build-first workflow
@@ -261,7 +617,7 @@ Installed Fruix systems now record an explicit first-generation deployment layou
- `/var/lib/fruix/system` - `/var/lib/fruix/system`
Current validated shape: Initial installed shape:
```text ```text
/var/lib/fruix/system/ /var/lib/fruix/system/
@@ -275,6 +631,24 @@ Current validated shape:
install.scm # present on installed targets install.scm # present on installed targets
``` ```
After a validated in-place switch, the layout extends to:
```text
/var/lib/fruix/system/
current -> generations/2
current-generation
rollback -> generations/1
rollback-generation
generations/
1/
...
2/
closure -> /frx/store/...-fruix-system-...
metadata.scm
provenance.scm
install.scm # deployment metadata for the switch operation
```
Installed systems also now create explicit GC-root-style deployment links under: Installed systems also now create explicit GC-root-style deployment links under:
- `/frx/var/fruix/gcroots` - `/frx/var/fruix/gcroots`
@@ -284,7 +658,9 @@ Current validated shape:
```text ```text
/frx/var/fruix/gcroots/ /frx/var/fruix/gcroots/
current-system -> /frx/store/...-fruix-system-... current-system -> /frx/store/...-fruix-system-...
rollback-system -> /frx/store/...-fruix-system-...
system-1 -> /frx/store/...-fruix-system-... system-1 -> /frx/store/...-fruix-system-...
system-2 -> /frx/store/...-fruix-system-...
``` ```
Important detail: Important detail:
@@ -294,7 +670,9 @@ Important detail:
## Roll-forward workflow ## Roll-forward workflow
The current Fruix roll-forward model is declaration-driven. The current Fruix roll-forward model now has two validated layers.
### Declaration/deployment roll-forward
Canonical process: Canonical process:
@@ -314,47 +692,61 @@ Canonical process:
5. boot or install the candidate 5. boot or install the candidate
6. validate the candidate closure in the booted system 6. validate the candidate closure in the booted system
The important property is that the candidate closure appears beside the earlier one in `/frx/store` rather than mutating it in place. ### Installed-system generation roll-forward
When the candidate closure is already present on an installed target:
1. run `fruix system switch /frx/store/...candidate...`
2. confirm the staged state with `fruix system status`
3. reboot into the candidate generation
4. validate the new active closure after reboot
The important property is still that the candidate closure appears beside the earlier one in `/frx/store` rather than mutating it in place.
## Rollback workflow ## Rollback workflow
The current canonical rollback workflow is also declaration-driven. The current canonical rollback workflow also now has two validated layers.
Today, rollback means: ### Declaration/deployment rollback
You can still roll back by redeploying the earlier declaration:
1. retain the earlier declaration that produced the known-good closure 1. retain the earlier declaration that produced the known-good closure
2. rebuild or rematerialize that earlier declaration 2. rebuild or rematerialize that earlier declaration
3. redeploy or reboot that earlier artifact again 3. redeploy or reboot that earlier artifact again
Concretely, the usual rollback choices are: Concretely, the usual declaration-level rollback choices are:
- rebuild the earlier declaration with `fruix system build` and confirm the old closure path reappears - rebuild the earlier declaration with `fruix system build` and confirm the old closure path reappears
- boot the earlier declaration again through `fruix system image` - boot the earlier declaration again through `fruix system image`
- reinstall the earlier declaration through `fruix system install`, `installer`, or `installer-iso` if the deployment medium itself must change - reinstall the earlier declaration through `fruix system install`, `installer`, or `installer-iso` if the deployment medium itself must change
This rollback story has already been validated at the closure/image/deployment level: ### Installed-system generation rollback
- side-by-side base-version coexistence in `/frx/store` When an installed target already has both the current and rollback generations recorded:
- roll-forward to a candidate closure
- rollback by rebuilding and booting the earlier declaration again 1. run `fruix system rollback`
- validation on both local QEMU and the approved XCP-ng VM path 2. confirm the staged state with `fruix system status`
3. reboot into the rollback generation
4. validate the restored active closure after reboot
This installed-system rollback path is now validated on local `QEMU/UEFI/TCG`.
### Important scope note ### Important scope note
This is not yet the same thing as a first-class installed-system generation switch command. This is still not yet the same thing as Guix's full `reconfigure`/generation UX.
Current rollback is: Current installed-system rollback is intentionally modest:
- **redeploy the earlier declaration again** - it switches between already-recorded generations on the target
- it does not yet fetch candidate closures onto the machine for you
What still remains for later Phase 19 work is making rollback itself operator-driven at the installed-system layer, rather than only declaration/redeploy driven. - it does not yet expose a richer history-management or generation-pruning policy
Still pending: Still pending:
- previous-generation tracking beyond the initial explicit generation-1 layout - operator-facing closure transfer or fetch onto installed systems
- an explicit rollback target link distinct from `current` - multi-generation lifecycle policy beyond the validated `current` and `rollback` pointers
- an operator-facing installed-system rollback workflow - a fuller `reconfigure`-style installed-system UX
- generation switching without full redeploy
## Provenance and deployment identity ## Provenance and deployment identity
@@ -375,16 +767,16 @@ Operators should retain metadata from successful candidate and current deploymen
## Current limitations ## Current limitations
The deployment workflow is now coherent, but it is not yet the final generation-management story. The deployment workflow is now coherent, and Fruix now has a validated installed-system switch/rollback path, but it is still not the final generation-management story.
Not yet first-class: Not yet first-class:
- a dedicated `switch` or `reconfigure` command - host-side closure transfer/fetch onto installed systems as part of `fruix system switch`
- an installed-system rollback command that moves between generations in place - a fuller `reconfigure` workflow that builds and stages the new closure from inside the target environment
- multi-generation retention and previous-generation tracking beyond generation 1 - multi-generation lifecycle policy beyond the validated `current` and `rollback` pointers
- generation switching policy independent of full redeploy - generation pruning and retention policy independent of full redeploy
Those are the next logical steps after the current explicit-generation layout. Those are the next logical steps after the current explicit-generation switch/rollback model.
## Summary ## Summary
@@ -395,6 +787,8 @@ The current canonical Fruix deployment model is:
- **materialize** the artifact appropriate to the deployment target - **materialize** the artifact appropriate to the deployment target
- **boot or install** that artifact - **boot or install** that artifact
- **identify deployments by closure path and provenance metadata** - **identify deployments by closure path and provenance metadata**
- **roll back by rebuilding/redeploying the earlier declaration**, not by mutating the current closure in place - on installed systems, **switch** to a staged candidate with `fruix system switch`
- on installed systems, **roll back** to the recorded rollback generation with `fruix system rollback`
- still use declaration/redeploy rollback when the target does not already have the desired closure staged locally
That is the operator-facing workflow Fruix should document and use while installed-system generation switching remains more limited than Guix's mature in-place system-generation workflow. That is the operator-facing workflow Fruix should document and use while its installed-system generation UX remains simpler than Guix's mature in-place system-generation workflow.

30
modules/fruix/system/freebsd.scm
View File

@@ -1,6 +1,8 @@
(define-module (fruix system freebsd) (define-module (fruix system freebsd)
#:use-module (fruix system freebsd model) #:use-module (fruix system freebsd model)
#:use-module (fruix system freebsd source) #:use-module (fruix system freebsd source)
#:use-module (fruix system freebsd executor)
#:use-module (fruix system freebsd build)
#:use-module (fruix system freebsd media) #:use-module (fruix system freebsd media)
#:re-export (user-group #:re-export (user-group
user-group? user-group?
@@ -24,13 +26,21 @@
file-system-type file-system-type
file-system-options file-system-options
file-system-needed-for-boot? file-system-needed-for-boot?
promoted-native-build-result?
promoted-native-build-result-store-path
promoted-native-build-result-metadata-file
promoted-native-build-result-metadata
promoted-native-build-result-spec
operating-system operating-system
operating-system? operating-system?
operating-system-host-name operating-system-host-name
operating-system-freebsd-base operating-system-freebsd-base
operating-system-native-build-result
operating-system-kernel operating-system-kernel
operating-system-bootloader operating-system-bootloader
operating-system-base-packages operating-system-base-packages
operating-system-development-packages
operating-system-build-packages
operating-system-users operating-system-users
operating-system-groups operating-system-groups
operating-system-file-systems operating-system-file-systems
@@ -42,6 +52,26 @@
operating-system-root-authorized-keys operating-system-root-authorized-keys
validate-operating-system validate-operating-system
materialize-freebsd-source materialize-freebsd-source
native-build-executor
native-build-executor?
native-build-executor-ref
native-build-executor-kind
native-build-executor-name
native-build-executor-version
native-build-executor-properties
normalize-native-build-executor
host-native-build-executor
ssh-guest-native-build-executor
self-hosted-native-build-executor
promoted-native-build-result
promoted-native-build-result->freebsd-base
promoted-native-build-result-artifact-store
promoted-native-build-result-kernel-package
promoted-native-build-result-bootloader-package
promoted-native-build-result-base-packages
promoted-native-build-result-development-packages
operating-system-from-promoted-native-build-result
promote-native-build-result
operating-system-closure-spec operating-system-closure-spec
operating-system-install-spec operating-system-install-spec
operating-system-image-spec operating-system-image-spec

559
modules/fruix/system/freebsd/build.scm
View File

@@ -2,6 +2,7 @@
#:use-module (fruix packages freebsd) #:use-module (fruix packages freebsd)
#:use-module (fruix system freebsd model) #:use-module (fruix system freebsd model)
#:use-module (fruix system freebsd source) #:use-module (fruix system freebsd source)
#:use-module (fruix system freebsd executor)
#:use-module (fruix system freebsd utils) #:use-module (fruix system freebsd utils)
#:use-module (guix build utils) #:use-module (guix build utils)
#:use-module (ice-9 format) #:use-module (ice-9 format)
@@ -10,7 +11,16 @@
#:use-module (srfi srfi-1) #:use-module (srfi srfi-1)
#:use-module (srfi srfi-13) #:use-module (srfi srfi-13)
#:export (host-freebsd-provenance #:export (host-freebsd-provenance
promoted-native-build-result
promoted-native-build-result->freebsd-base
promoted-native-build-result-artifact-store
promoted-native-build-result-kernel-package
promoted-native-build-result-bootloader-package
promoted-native-build-result-base-packages
promoted-native-build-result-development-packages
operating-system-from-promoted-native-build-result
materialize-freebsd-package materialize-freebsd-package
promote-native-build-result
materialize-prefix)) materialize-prefix))
(define (host-freebsd-provenance) (define (host-freebsd-provenance)
@@ -339,55 +349,505 @@
(append overrides plan))) (append overrides plan)))
(define* (materialize-freebsd-package package store-dir cache #:optional source-cache) (define* (materialize-freebsd-package package store-dir cache #:optional source-cache)
(let* ((source-cache (or source-cache (make-hash-table))) (if (existing-store-package? package)
(input-paths (map (lambda (input) (validate-existing-store-package package)
(materialize-freebsd-package input store-dir cache source-cache)) (let* ((source-cache (or source-cache (make-hash-table)))
(freebsd-package-inputs package))) (input-paths (map (lambda (input)
(prepared-package (materialize-freebsd-package input store-dir cache source-cache))
(if (freebsd-native-build-package? package) (freebsd-package-inputs package)))
(let* ((source (plan-freebsd-source (freebsd-package-install-plan package))) (prepared-package
(source-result (materialize-freebsd-source/cached source store-dir source-cache)) (if (freebsd-native-build-package? package)
(plan (plan-with-materialized-source (freebsd-package-install-plan package) (let* ((source (plan-freebsd-source (freebsd-package-install-plan package)))
source-result))) (source-result (materialize-freebsd-source/cached source store-dir source-cache))
(package-with-install-plan package plan)) (plan (plan-with-materialized-source (freebsd-package-install-plan package)
package)) source-result)))
(effective-input-paths (package-with-install-plan package plan))
(if (freebsd-native-build-package? package) package))
(cons (build-plan-ref (freebsd-package-install-plan prepared-package) (effective-input-paths
'materialized-source-store (if (freebsd-native-build-package? package)
#f) (cons (build-plan-ref (freebsd-package-install-plan prepared-package)
input-paths) 'materialized-source-store
input-paths)) #f)
(effective-input-paths (filter identity effective-input-paths)) input-paths)
(manifest (package-manifest-string prepared-package effective-input-paths)) input-paths))
(cache-key (sha256-string manifest)) (effective-input-paths (filter identity effective-input-paths))
(cached (hash-ref cache cache-key #f))) (manifest (package-manifest-string prepared-package effective-input-paths))
(if cached (cache-key (sha256-string manifest))
cached (cached (hash-ref cache cache-key #f)))
(let* ((hash (sha256-string manifest)) (if cached
(output-path (string-append store-dir "/" hash "-" cached
(freebsd-package-name prepared-package) (let* ((display-name (string-append (freebsd-package-name prepared-package)
"-" "-"
(freebsd-package-version prepared-package)))) (freebsd-package-version prepared-package)))
(unless (file-exists? output-path) (output-path (make-store-path store-dir display-name manifest
(case (freebsd-package-build-system prepared-package) #:kind 'freebsd-package)))
((copy-build-system) (unless (file-exists? output-path)
(mkdir-p output-path) (case (freebsd-package-build-system prepared-package)
(for-each (lambda (entry) ((copy-build-system)
(materialize-plan-entry output-path entry)) (mkdir-p output-path)
(freebsd-package-install-plan prepared-package)) (for-each (lambda (entry)
(write-file (string-append output-path "/.references") (materialize-plan-entry output-path entry))
(string-join effective-input-paths "\n")) (freebsd-package-install-plan prepared-package))
(write-file (string-append output-path "/.fruix-package") manifest)) (write-file (string-append output-path "/.references")
((freebsd-world-build-system freebsd-kernel-build-system) (string-join effective-input-paths "\n"))
(materialize-native-freebsd-package prepared-package effective-input-paths manifest output-path)) (write-file (string-append output-path "/.fruix-package") manifest))
(else ((freebsd-world-build-system freebsd-kernel-build-system)
(error (format #f "unsupported package build system: ~a" (materialize-native-freebsd-package prepared-package effective-input-paths manifest output-path))
(freebsd-package-build-system prepared-package)))))) (else
(hash-set! cache cache-key output-path) (error (format #f "unsupported package build system: ~a"
output-path)))) (freebsd-package-build-system prepared-package))))))
(hash-set! cache cache-key output-path)
output-path)))))
(define native-build-result-promotion-version "1")
(define (native-build-result-ref data key default)
(match (assoc key data)
((_ . value) value)
(#f default)))
(define (native-build-result-executor result)
(let* ((executor (native-build-result-ref result 'executor #f))
(legacy-version (native-build-result-ref result 'executor-version "legacy")))
(cond
((native-build-executor? executor)
executor)
((string? executor)
(let ((normalized (normalize-native-build-executor executor)))
`((kind . ,(native-build-executor-kind normalized))
(name . ,(native-build-executor-name normalized))
(version . ,legacy-version)
(properties . ,(native-build-executor-properties normalized)))))
(else
(native-build-executor #:kind 'unknown
#:name "unknown"
#:version legacy-version)))))
(define (native-build-result-executor-kind result)
(native-build-executor-kind (native-build-result-executor result)))
(define (native-build-result-executor-name result)
(native-build-executor-name (native-build-result-executor result)))
(define (native-build-result-executor-version result)
(native-build-executor-version (native-build-result-executor result)))
(define (read-native-build-result result-root)
(let ((promotion-file (string-append result-root "/promotion.scm")))
(unless (file-exists? promotion-file)
(error "native build result is missing promotion.scm" result-root))
(let ((result (call-with-input-file promotion-file read)))
(unless (equal? (native-build-result-ref result 'native-build-result-version #f)
native-build-result-promotion-version)
(error "unsupported native build result promotion version" promotion-file))
result)))
(define existing-store-package-build-system 'existing-store-item-build-system)
(define (existing-store-package? package)
(eq? (freebsd-package-build-system package)
existing-store-package-build-system))
(define (existing-store-package-ref package key default)
(build-plan-ref (freebsd-package-install-plan package) key default))
(define (validate-existing-store-package package)
(let* ((store-path (existing-store-package-ref package 'store-path #f))
(required-file (existing-store-package-ref package 'required-file #f))
(metadata-file (existing-store-package-ref package 'metadata-file #f)))
(unless (and (string? store-path) (file-exists? store-path))
(error "existing-store package is missing a valid store path" package store-path))
(when metadata-file
(unless (file-exists? metadata-file)
(error "existing-store package metadata file is missing" package metadata-file)))
(when required-file
(unless (file-exists? (string-append store-path "/" required-file))
(error "existing-store package is missing required file"
package
(string-append store-path "/" required-file))))
store-path))
(define (normalize-promoted-native-build-result value)
(cond
((promoted-native-build-result? value)
value)
((string? value)
(promoted-native-build-result #:store-path value))
(else
(error "expected a promoted native build result or store path" value))))
(define (read-promoted-native-build-artifact-metadata metadata-file)
(unless (file-exists? metadata-file)
(error "promoted native build artifact metadata file is missing" metadata-file))
(let ((metadata (call-with-input-file metadata-file read)))
(unless (equal? (native-build-result-ref metadata 'native-build-object-version #f)
native-build-result-promotion-version)
(error "unsupported promoted native build object version" metadata-file))
(unless (eq? (native-build-result-ref metadata 'object-kind #f) 'artifact)
(error "promoted native build object is not an artifact" metadata-file))
metadata))
(define (promoted-native-build-result-artifact-entry result artifact-kind)
(let* ((metadata (promoted-native-build-result-metadata result))
(artifacts (native-build-result-ref metadata 'artifacts '()))
(entry (find (lambda (item)
(eq? (native-build-result-ref item 'artifact-kind #f)
artifact-kind))
artifacts)))
(unless entry
(error "promoted native build result is missing artifact entry" artifact-kind))
entry))
(define (promoted-native-build-result-artifact-store result artifact-kind)
(let* ((result (normalize-promoted-native-build-result result))
(entry (promoted-native-build-result-artifact-entry result artifact-kind))
(store-path (native-build-result-ref entry 'store-path #f))
(metadata-file (native-build-result-ref entry 'metadata-file #f))
(artifact-metadata (and metadata-file
(read-promoted-native-build-artifact-metadata metadata-file)))
(required-file (and artifact-metadata
(native-build-result-ref artifact-metadata 'required-file #f))))
(unless (and (string? store-path) (file-exists? store-path))
(error "promoted native build result is missing artifact store" artifact-kind store-path))
(when artifact-metadata
(unless (eq? (native-build-result-ref artifact-metadata 'artifact-kind #f)
artifact-kind)
(error "promoted native build artifact metadata kind mismatch"
artifact-kind
metadata-file)))
(when required-file
(unless (file-exists? (string-append store-path "/" required-file))
(error "promoted native build artifact store is missing required file"
artifact-kind
(string-append store-path "/" required-file))))
store-path))
(define* (promoted-native-build-result #:key store-path)
(unless (and (string? store-path) (file-exists? store-path))
(error "promoted native build result store path does not exist" store-path))
(let* ((metadata-file (string-append store-path "/.fruix-native-build-result.scm")))
(unless (file-exists? metadata-file)
(error "promoted native build result store is missing metadata" metadata-file))
(let* ((metadata (call-with-input-file metadata-file read))
(result (make-promoted-native-build-result store-path metadata-file metadata)))
(unless (equal? (native-build-result-ref metadata 'native-build-result-version #f)
native-build-result-promotion-version)
(error "unsupported promoted native build result version" metadata-file))
(unless (eq? (native-build-result-ref metadata 'object-kind #f) 'result-bundle)
(error "promoted native build result store does not contain a result bundle" metadata-file))
(for-each (lambda (artifact-kind)
(promoted-native-build-result-artifact-store result artifact-kind))
'(world kernel headers bootloader))
result)))
(define (promoted-native-build-result->freebsd-base result)
(let* ((result (normalize-promoted-native-build-result result))
(metadata (promoted-native-build-result-metadata result))
(base (native-build-result-ref metadata 'freebsd-base '()))
(source (native-build-result-ref metadata 'source '()))
(source-root (or (native-build-result-ref source 'source-root #f)
(native-build-result-ref base 'source-root #f)
"/usr/src"))
(source-name (string-append "promoted-native-build-result-source-"
(path-basename
(promoted-native-build-result-store-path result))))
(synthetic-source (freebsd-source #:name source-name
#:kind 'local-tree
#:path source-root)))
(freebsd-base #:name (native-build-result-ref base 'name "promoted-native-build-result")
#:version-label (native-build-result-ref base 'version-label "unknown")
#:release (native-build-result-ref base 'release "unknown")
#:branch (native-build-result-ref base 'branch "unknown")
#:source synthetic-source
#:source-root (native-build-result-ref base 'source-root source-root)
#:target (native-build-result-ref base 'target "amd64")
#:target-arch (native-build-result-ref base 'target-arch "amd64")
#:kernconf (native-build-result-ref base 'kernconf "GENERIC"))))
(define (promoted-native-build-result-artifact-package result artifact-kind
package-name synopsis description)
(let* ((result (normalize-promoted-native-build-result result))
(metadata (promoted-native-build-result-metadata result))
(base (native-build-result-ref metadata 'freebsd-base '()))
(entry (promoted-native-build-result-artifact-entry result artifact-kind))
(store-path (promoted-native-build-result-artifact-store result artifact-kind))
(metadata-file (native-build-result-ref entry 'metadata-file #f))
(artifact-metadata (and metadata-file
(read-promoted-native-build-artifact-metadata metadata-file)))
(required-file (and artifact-metadata
(native-build-result-ref artifact-metadata 'required-file #f))))
(freebsd-package
#:name package-name
#:version (native-build-result-ref base 'version-label "unknown")
#:build-system existing-store-package-build-system
#:home-page "https://www.freebsd.org/"
#:synopsis synopsis
#:description description
#:license 'bsd-2
#:install-plan `((store-path . ,store-path)
(metadata-file . ,metadata-file)
(required-file . ,required-file)
(artifact-kind . ,artifact-kind)
(result-store . ,(promoted-native-build-result-store-path result))))))
(define (promoted-native-build-result-world-package result)
(promoted-native-build-result-artifact-package
result
'world
"freebsd-promoted-world"
"Promoted Fruix-native FreeBSD world artifact"
"FreeBSD world artifact imported from a promoted Fruix native-build result bundle."))
(define (promoted-native-build-result-kernel-package result)
(promoted-native-build-result-artifact-package
result
'kernel
"freebsd-promoted-kernel"
"Promoted Fruix-native FreeBSD kernel artifact"
"FreeBSD kernel artifact imported from a promoted Fruix native-build result bundle."))
(define (promoted-native-build-result-bootloader-package result)
(promoted-native-build-result-artifact-package
result
'bootloader
"freebsd-promoted-bootloader"
"Promoted Fruix-native FreeBSD bootloader artifact"
"FreeBSD bootloader artifact imported from a promoted Fruix native-build result bundle."))
(define (promoted-native-build-result-headers-package result)
(promoted-native-build-result-artifact-package
result
'headers
"freebsd-promoted-headers"
"Promoted Fruix-native FreeBSD headers artifact"
"FreeBSD headers artifact imported from a promoted Fruix native-build result bundle."))
(define (promoted-native-build-result-base-packages result)
(list (promoted-native-build-result-world-package result)))
(define (promoted-native-build-result-development-packages result)
(list (promoted-native-build-result-headers-package result)))
(define* (operating-system-from-promoted-native-build-result result
#:key
(host-name #f)
(freebsd-base #f)
(kernel #f)
(bootloader #f)
(base-packages #f)
(development-packages #f)
(users #f)
(groups #f)
(file-systems #f)
(services #f)
(loader-entries #f)
(rc-conf-entries #f)
(init-mode #f)
(ready-marker #f)
(root-authorized-keys #f))
(let* ((result (normalize-promoted-native-build-result result))
(defaults default-minimal-operating-system)
(fallback (lambda (value thunk)
(if (eq? value #f) (thunk) value))))
(operating-system
#:host-name (fallback host-name (lambda () (operating-system-host-name defaults)))
#:freebsd-base (fallback freebsd-base (lambda ()
(promoted-native-build-result->freebsd-base result)))
#:native-build-result result
#:kernel (fallback kernel (lambda ()
(promoted-native-build-result-kernel-package result)))
#:bootloader (fallback bootloader (lambda ()
(promoted-native-build-result-bootloader-package result)))
#:base-packages (fallback base-packages (lambda ()
(promoted-native-build-result-base-packages result)))
#:development-packages (fallback development-packages (lambda ()
(operating-system-development-packages defaults)))
#:users (fallback users (lambda () (operating-system-users defaults)))
#:groups (fallback groups (lambda () (operating-system-groups defaults)))
#:file-systems (fallback file-systems (lambda () (operating-system-file-systems defaults)))
#:services (fallback services (lambda () (operating-system-services defaults)))
#:loader-entries (fallback loader-entries (lambda () (operating-system-loader-entries defaults)))
#:rc-conf-entries (fallback rc-conf-entries (lambda () (operating-system-rc-conf-entries defaults)))
#:init-mode (fallback init-mode (lambda () (operating-system-init-mode defaults)))
#:ready-marker (fallback ready-marker (lambda () (operating-system-ready-marker defaults)))
#:root-authorized-keys (fallback root-authorized-keys (lambda ()
(operating-system-root-authorized-keys defaults))))))
(define (native-build-artifact-entry result artifact-kind)
(let* ((artifacts (native-build-result-ref result 'artifacts '()))
(entry (assoc artifact-kind artifacts)))
(unless entry
(error "native build result is missing artifact entry" artifact-kind))
(cdr entry)))
(define (native-build-artifact-root result-root result artifact-kind)
(let* ((entry (native-build-artifact-entry result artifact-kind))
(relative-path (native-build-result-ref entry 'path #f))
(required-file (native-build-result-ref entry 'required-file #f))
(artifact-root (and relative-path
(string-append result-root "/" relative-path))))
(unless (and artifact-root (file-exists? artifact-root))
(error "native build result is missing artifact tree" artifact-kind artifact-root))
(when required-file
(unless (file-exists? (string-append artifact-root "/" required-file))
(error "native build artifact is missing required file"
artifact-kind
(string-append artifact-root "/" required-file))))
artifact-root))
(define (native-build-existing-store-references result store-dir)
(filter identity
(map (lambda (path)
(and (string? path)
(string-prefix? (string-append store-dir "/") path)
(file-exists? path)
path))
(list (native-build-result-ref result 'closure-path #f)
(let ((source (native-build-result-ref result 'source '())))
(native-build-result-ref source 'store-path #f))))))
(define (native-build-artifact-display-name result artifact-kind)
(let* ((base (native-build-result-ref result 'freebsd-base '()))
(version-label (native-build-result-ref base 'version-label "unknown"))
(executor-name (native-build-result-executor-name result)))
(string-append "fruix-native-"
(symbol->string artifact-kind)
"-"
version-label
"-"
executor-name)))
(define (native-build-promoted-artifact-metadata result artifact-kind content-signature)
(let* ((entry (native-build-artifact-entry result artifact-kind))
(executor (native-build-result-executor result))
(build-profile (native-build-result-ref result 'build-profile
(native-build-result-ref result 'development-profile ""))))
`((native-build-object-version . ,native-build-result-promotion-version)
(object-kind . artifact)
(artifact-kind . ,artifact-kind)
(executor . ,executor)
(executor-kind . ,(native-build-result-executor-kind result))
(executor-name . ,(native-build-result-executor-name result))
(executor-version . ,(native-build-result-executor-version result))
(run-id . ,(native-build-result-ref result 'run-id "unknown"))
(guest-host-name . ,(native-build-result-ref result 'guest-host-name "unknown"))
(closure-path . ,(native-build-result-ref result 'closure-path ""))
(development-profile . ,(native-build-result-ref result 'development-profile ""))
(build-profile . ,build-profile)
(freebsd-base . ,(native-build-result-ref result 'freebsd-base '()))
(source . ,(native-build-result-ref result 'source '()))
(build-policy . ,(native-build-result-ref result 'build-policy '()))
(required-file . ,(native-build-result-ref entry 'required-file ""))
(recorded-sha256 . ,(native-build-result-ref entry 'recorded-sha256 ""))
(content-signature . ,content-signature))))
(define (promote-native-build-artifact result-root result store-dir artifact-kind)
(let* ((artifact-root (native-build-artifact-root result-root result artifact-kind))
(content-signature (tree-content-signature artifact-root))
(metadata (native-build-promoted-artifact-metadata result artifact-kind content-signature))
(payload (object->string metadata))
(display-name (native-build-artifact-display-name result artifact-kind))
(output-path (make-store-path store-dir display-name payload
#:kind 'native-build-artifact
#:output artifact-kind))
(references (native-build-existing-store-references result store-dir)))
(unless (file-exists? output-path)
(mkdir-p output-path)
(stage-tree-into-output artifact-root output-path)
(write-file (string-append output-path "/.references")
(string-join references "\n"))
(write-file (string-append output-path "/.fruix-native-build-object.scm")
payload))
`((artifact-kind . ,artifact-kind)
(artifact-root . ,artifact-root)
(store-path . ,output-path)
(content-signature . ,content-signature)
(metadata-file . ,(string-append output-path "/.fruix-native-build-object.scm")))) )
(define (native-build-result-display-name result)
(let* ((base (native-build-result-ref result 'freebsd-base '()))
(version-label (native-build-result-ref base 'version-label "unknown"))
(executor-name (native-build-result-executor-name result)))
(string-append "fruix-native-build-result-" version-label "-" executor-name)))
(define (native-build-promoted-result-object result promoted-artifacts)
(let ((executor (native-build-result-executor result))
(build-profile (native-build-result-ref result 'build-profile
(native-build-result-ref result 'development-profile ""))))
`((native-build-result-version . ,native-build-result-promotion-version)
(object-kind . result-bundle)
(executor . ,executor)
(executor-kind . ,(native-build-result-executor-kind result))
(executor-name . ,(native-build-result-executor-name result))
(executor-version . ,(native-build-result-executor-version result))
(run-id . ,(native-build-result-ref result 'run-id "unknown"))
(guest-host-name . ,(native-build-result-ref result 'guest-host-name "unknown"))
(closure-path . ,(native-build-result-ref result 'closure-path ""))
(development-profile . ,(native-build-result-ref result 'development-profile ""))
(build-profile . ,build-profile)
(freebsd-base . ,(native-build-result-ref result 'freebsd-base '()))
(source . ,(native-build-result-ref result 'source '()))
(build-policy . ,(native-build-result-ref result 'build-policy '()))
(artifact-count . ,(length promoted-artifacts))
(artifacts . ,(map (lambda (entry)
`((artifact-kind . ,(assoc-ref entry 'artifact-kind))
(store-path . ,(assoc-ref entry 'store-path))
(content-signature . ,(assoc-ref entry 'content-signature))
(metadata-file . ,(assoc-ref entry 'metadata-file))))
promoted-artifacts)))))
(define* (promote-native-build-result result-root #:key (store-dir "/frx/store"))
(let* ((result (read-native-build-result result-root))
(promoted-artifacts (map (lambda (artifact-kind)
(promote-native-build-artifact result-root result store-dir artifact-kind))
'(world kernel headers bootloader)))
(result-object (native-build-promoted-result-object result promoted-artifacts))
(payload (object->string result-object))
(display-name (native-build-result-display-name result))
(result-store (make-store-path store-dir display-name payload
#:kind 'native-build-result))
(result-references (append (map (lambda (entry)
(assoc-ref entry 'store-path))
promoted-artifacts)
(native-build-existing-store-references result store-dir))))
(unless (file-exists? result-store)
(mkdir-p (string-append result-store "/artifacts"))
(for-each (lambda (entry)
(symlink (assoc-ref entry 'store-path)
(string-append result-store
"/artifacts/"
(symbol->string (assoc-ref entry 'artifact-kind)))))
promoted-artifacts)
(write-file (string-append result-store "/.references")
(string-join result-references "\n"))
(write-file (string-append result-store "/.fruix-native-build-result.scm")
payload))
`((result-root . ,result-root)
(executor-kind . ,(native-build-result-executor-kind result))
(executor-name . ,(native-build-result-executor-name result))
(executor-version . ,(native-build-result-executor-version result))
(result-store . ,result-store)
(result-metadata-file . ,(string-append result-store "/.fruix-native-build-result.scm"))
(artifact-store-count . ,(length promoted-artifacts))
(artifact-stores . ,(map (lambda (entry) (assoc-ref entry 'store-path)) promoted-artifacts))
(world-store . ,(assoc-ref (find (lambda (entry)
(eq? (assoc-ref entry 'artifact-kind) 'world))
promoted-artifacts)
'store-path))
(kernel-store . ,(assoc-ref (find (lambda (entry)
(eq? (assoc-ref entry 'artifact-kind) 'kernel))
promoted-artifacts)
'store-path))
(headers-store . ,(assoc-ref (find (lambda (entry)
(eq? (assoc-ref entry 'artifact-kind) 'headers))
promoted-artifacts)
'store-path))
(bootloader-store . ,(assoc-ref (find (lambda (entry)
(eq? (assoc-ref entry 'artifact-kind) 'bootloader))
promoted-artifacts)
'store-path)))))
(define (sanitize-materialized-prefix name output-path) (define (sanitize-materialized-prefix name output-path)
(cond (cond
((string=? name "fruix-guile-extra") ((string=? name "fruix-guile-extra")
@@ -454,8 +914,9 @@
(define* (materialize-prefix source-path name version store-dir #:key (extra-files '())) (define* (materialize-prefix source-path name version store-dir #:key (extra-files '()))
(let* ((manifest (prefix-manifest-string source-path extra-files)) (let* ((manifest (prefix-manifest-string source-path extra-files))
(hash (sha256-string manifest)) (display-name (string-append name "-" version))
(output-path (string-append store-dir "/" hash "-" name "-" version))) (output-path (make-store-path store-dir display-name manifest
#:kind 'prefix)))
(unless (file-exists? output-path) (unless (file-exists? output-path)
(mkdir-p output-path) (mkdir-p output-path)
(for-each (lambda (entry) (for-each (lambda (entry)

121
modules/fruix/system/freebsd/executor.scm Normal file
View File

@@ -0,0 +1,121 @@
(define-module (fruix system freebsd executor)
#:use-module (ice-9 match)
#:use-module (srfi srfi-1)
#:export (native-build-executor-model-version
native-build-executor
native-build-executor?
native-build-executor-ref
native-build-executor-kind
native-build-executor-name
native-build-executor-version
native-build-executor-properties
normalize-native-build-executor
host-native-build-executor
ssh-guest-native-build-executor
self-hosted-native-build-executor))
(define native-build-executor-model-version "1")
(define (association-list? value)
(and (list? value)
(every pair? value)))
(define (executor-name kind provided-name)
(or provided-name
(symbol->string kind)))
(define* (native-build-executor #:key kind name
(version native-build-executor-model-version)
(properties '()))
(unless (symbol? kind)
(error "native build executor kind must be a symbol" kind))
(unless (string? (executor-name kind name))
(error "native build executor name must be a string" name))
(unless (string? version)
(error "native build executor version must be a string" version))
(unless (association-list? properties)
(error "native build executor properties must be an association list" properties))
`((kind . ,kind)
(name . ,(executor-name kind name))
(version . ,version)
(properties . ,properties)))
(define (native-build-executor-ref executor key default)
(match (assoc key executor)
((_ . value) value)
(#f default)))
(define (native-build-executor? value)
(and (association-list? value)
(symbol? (native-build-executor-ref value 'kind #f))
(string? (native-build-executor-ref value 'name #f))
(string? (native-build-executor-ref value 'version #f))
(association-list? (native-build-executor-ref value 'properties '()))))
(define (native-build-executor-kind executor)
(native-build-executor-ref executor 'kind 'unknown))
(define (native-build-executor-name executor)
(native-build-executor-ref executor 'name "unknown"))
(define (native-build-executor-version executor)
(native-build-executor-ref executor 'version "unknown"))
(define (native-build-executor-properties executor)
(native-build-executor-ref executor 'properties '()))
(define (legacy-executor-kind name)
(cond
((member name '("host")) 'host)
((member name '("ssh-guest" "guest-ssh" "guest-host-initiated")) 'ssh-guest)
((member name '("self-hosted" "guest-self-hosted")) 'self-hosted)
((member name '("jail")) 'jail)
((member name '("remote-builder")) 'remote-builder)
(else 'legacy)))
(define (normalize-native-build-executor value)
(cond
((native-build-executor? value)
value)
((string? value)
(native-build-executor #:kind (legacy-executor-kind value)
#:name value
#:version "legacy"))
(else
(error "unsupported native build executor representation" value))))
(define* (host-native-build-executor #:key (name "host")
host-name working-directory)
(native-build-executor
#:kind 'host
#:name name
#:properties (filter-map identity
`((host-name . ,host-name)
(working-directory . ,working-directory)))))
(define* (ssh-guest-native-build-executor #:key (name "ssh-guest")
transport orchestrator
guest-host-name guest-ip vm-id vdi-id)
(native-build-executor
#:kind 'ssh-guest
#:name name
#:properties (filter-map identity
`((transport . ,(or transport "ssh"))
(orchestrator . ,(or orchestrator "host"))
(guest-host-name . ,guest-host-name)
(guest-ip . ,guest-ip)
(vm-id . ,vm-id)
(vdi-id . ,vdi-id)))))
(define* (self-hosted-native-build-executor #:key (name "self-hosted")
helper-path helper-version
guest-host-name build-root-base result-root-base)
(native-build-executor
#:kind 'self-hosted
#:name name
#:version (or helper-version native-build-executor-model-version)
#:properties (filter-map identity
`((helper-path . ,helper-path)
(guest-host-name . ,guest-host-name)
(build-root-base . ,build-root-base)
(result-root-base . ,result-root-base)))))

305
modules/fruix/system/freebsd/media.scm
View File

@@ -72,18 +72,35 @@
(store-dir "/frx/store") (store-dir "/frx/store")
(guile-prefix "/tmp/guile-freebsd-validate-install") (guile-prefix "/tmp/guile-freebsd-validate-install")
(guile-extra-prefix "/tmp/guile-gnutls-freebsd-validate-install") (guile-extra-prefix "/tmp/guile-gnutls-freebsd-validate-install")
(shepherd-prefix "/tmp/shepherd-freebsd-validate-install")) (shepherd-prefix "/tmp/shepherd-freebsd-validate-install")
(guile-store-path #f)
(guile-extra-store-path #f)
(shepherd-store-path #f)
(declaration-source #f)
(declaration-origin #f)
(declaration-system-symbol #f))
(validate-operating-system os) (validate-operating-system os)
(let* ((cache (make-hash-table)) (let* ((cache (make-hash-table))
(source-cache (make-hash-table)) (source-cache (make-hash-table))
(native-build-result (operating-system-native-build-result os))
(kernel-package (operating-system-kernel os)) (kernel-package (operating-system-kernel os))
(bootloader-package (operating-system-bootloader os)) (bootloader-package (operating-system-bootloader os))
(base-packages (operating-system-base-packages os)) (base-packages (operating-system-base-packages os))
(development-packages (operating-system-development-packages os))
(build-packages (operating-system-build-packages os))
(kernel-store (materialize-freebsd-package kernel-package store-dir cache source-cache)) (kernel-store (materialize-freebsd-package kernel-package store-dir cache source-cache))
(bootloader-store (materialize-freebsd-package bootloader-package store-dir cache source-cache)) (bootloader-store (materialize-freebsd-package bootloader-package store-dir cache source-cache))
(base-package-stores (map (lambda (package) (base-package-stores (map (lambda (package)
(materialize-freebsd-package package store-dir cache source-cache)) (materialize-freebsd-package package store-dir cache source-cache))
base-packages)) base-packages))
(development-package-stores
(map (lambda (package)
(materialize-freebsd-package package store-dir cache source-cache))
development-packages))
(build-package-stores
(map (lambda (package)
(materialize-freebsd-package package store-dir cache source-cache))
build-packages))
(base-package-pairs (map cons base-packages base-package-stores)) (base-package-pairs (map cons base-packages base-package-stores))
(store-classification (store-classification
(append (list (cons kernel-package kernel-store) (append (list (cons kernel-package kernel-store)
@@ -104,12 +121,15 @@
("/usr/local/lib/libtasn1.so.6" . "lib/libtasn1.so.6") ("/usr/local/lib/libtasn1.so.6" . "lib/libtasn1.so.6")
("/usr/local/lib/libhogweed.so.6" . "lib/libhogweed.so.6") ("/usr/local/lib/libhogweed.so.6" . "lib/libhogweed.so.6")
("/usr/local/lib/libnettle.so.8" . "lib/libnettle.so.8"))) ("/usr/local/lib/libnettle.so.8" . "lib/libnettle.so.8")))
(guile-store (materialize-prefix guile-prefix "fruix-guile-runtime" "3.0" store-dir (guile-store (or guile-store-path
#:extra-files guile-runtime-extra-files)) (materialize-prefix guile-prefix "fruix-guile-runtime" "3.0" store-dir
(guile-extra-store (materialize-prefix guile-extra-prefix "fruix-guile-extra" "3.0" store-dir #:extra-files guile-runtime-extra-files)))
#:extra-files (append guile-runtime-extra-files (guile-extra-store (or guile-extra-store-path
guile-extra-runtime-files))) (materialize-prefix guile-extra-prefix "fruix-guile-extra" "3.0" store-dir
(shepherd-store (materialize-prefix shepherd-prefix "fruix-shepherd-runtime" "1.0.9" store-dir)) #:extra-files (append guile-runtime-extra-files
guile-extra-runtime-files))))
(shepherd-store (or shepherd-store-path
(materialize-prefix shepherd-prefix "fruix-shepherd-runtime" "1.0.9" store-dir)))
(host-base-stores (host-base-stores
(delete-duplicates (delete-duplicates
(map cdr (map cdr
@@ -129,29 +149,76 @@
(delete-duplicates (map (lambda (result) (delete-duplicates (map (lambda (result)
(assoc-ref result 'source-store-path)) (assoc-ref result 'source-store-path))
source-materializations))) source-materializations)))
(promoted-native-build-result-summary
(and native-build-result
(promoted-native-build-result-spec native-build-result)))
(promoted-native-build-result-store
(and native-build-result
(promoted-native-build-result-store-path native-build-result)))
(promoted-native-build-artifact-stores
(delete-duplicates
(filter identity
(if native-build-result
(map (lambda (artifact-kind)
(promoted-native-build-result-artifact-store native-build-result artifact-kind))
'(world kernel headers bootloader))
'()))))
(declaration-source-text
(or declaration-source
";; Fruix declaration source is unavailable for this closure.\n"))
(declaration-origin-text (or declaration-origin ""))
(declaration-system-text
(cond ((symbol? declaration-system-symbol)
(symbol->string declaration-system-symbol))
((string? declaration-system-symbol)
declaration-system-symbol)
(else "")))
(declaration-info-object
`((available? . ,(not (not declaration-source)))
(system-variable . ,declaration-system-text)))
(metadata-files (metadata-files
`(("metadata/freebsd-base.scm" (append
. ,(object->string (freebsd-base-spec (operating-system-freebsd-base os)))) (list (cons "metadata/freebsd-base.scm"
("metadata/freebsd-source.scm" (object->string (freebsd-base-spec (operating-system-freebsd-base os))))
. ,(object->string (freebsd-source-spec (freebsd-base-source (operating-system-freebsd-base os))))) (cons "metadata/freebsd-source.scm"
("metadata/freebsd-source-materializations.scm" (object->string (freebsd-source-spec (freebsd-base-source (operating-system-freebsd-base os)))))
. ,(object->string (map freebsd-source-materialization-spec source-materializations))) (cons "metadata/freebsd-source-materializations.scm"
("metadata/host-base-provenance.scm" (object->string (map freebsd-source-materialization-spec source-materializations)))
. ,(object->string (host-freebsd-provenance))) (cons "metadata/host-base-provenance.scm"
("metadata/store-layout.scm" (object->string (host-freebsd-provenance)))
. ,(object->string (cons "metadata/system-declaration.scm"
`((freebsd-base . ,(freebsd-base-spec (operating-system-freebsd-base os))) declaration-source-text)
(freebsd-source . ,(freebsd-source-spec (freebsd-base-source (operating-system-freebsd-base os)))) (cons "metadata/system-declaration-info.scm"
(materialized-source-store-count . ,(length materialized-source-stores)) (object->string declaration-info-object))
(materialized-source-stores . ,materialized-source-stores) (cons "metadata/system-declaration-system"
(host-base-store-count . ,(length host-base-stores)) (string-append declaration-system-text "\n"))
(host-base-stores . ,host-base-stores) (cons "metadata/store-layout.scm"
(native-base-store-count . ,(length native-base-stores)) (object->string
(native-base-stores . ,native-base-stores) `((freebsd-base . ,(freebsd-base-spec (operating-system-freebsd-base os)))
(fruix-runtime-store-count . ,(length fruix-runtime-stores)) (freebsd-source . ,(freebsd-source-spec (freebsd-base-source (operating-system-freebsd-base os))))
(fruix-runtime-stores . ,fruix-runtime-stores) (system-declaration-available? . ,(not (not declaration-source)))
(host-base-replacement-order . ,%freebsd-host-staged-replacement-order) (system-declaration-system-variable . ,declaration-system-text)
(init-mode . ,(operating-system-init-mode os))))))) (promoted-native-build-result . ,promoted-native-build-result-summary)
(promoted-native-build-artifact-store-count . ,(length promoted-native-build-artifact-stores))
(promoted-native-build-artifact-stores . ,promoted-native-build-artifact-stores)
(materialized-source-store-count . ,(length materialized-source-stores))
(materialized-source-stores . ,materialized-source-stores)
(host-base-store-count . ,(length host-base-stores))
(host-base-stores . ,host-base-stores)
(native-base-store-count . ,(length native-base-stores))
(native-base-stores . ,native-base-stores)
(development-package-store-count . ,(length development-package-stores))
(development-package-stores . ,development-package-stores)
(build-package-store-count . ,(length build-package-stores))
(build-package-stores . ,build-package-stores)
(fruix-runtime-store-count . ,(length fruix-runtime-stores))
(fruix-runtime-stores . ,fruix-runtime-stores)
(host-base-replacement-order . ,%freebsd-host-staged-replacement-order)
(init-mode . ,(operating-system-init-mode os))))))
(if promoted-native-build-result-summary
(list (cons "metadata/promoted-native-build-result.scm"
(object->string promoted-native-build-result-summary)))
'())))
(generated-files (append (operating-system-generated-files os (generated-files (append (operating-system-generated-files os
#:guile-store guile-store #:guile-store guile-store
#:guile-extra-store guile-extra-store #:guile-extra-store guile-extra-store
@@ -161,7 +228,16 @@
. ,(render-activation-rc-script)) . ,(render-activation-rc-script))
("usr/local/etc/rc.d/fruix-shepherd" ("usr/local/etc/rc.d/fruix-shepherd"
. ,(render-rc-script shepherd-store guile-store guile-extra-store))))) . ,(render-rc-script shepherd-store guile-store guile-extra-store)))))
(references (delete-duplicates (append materialized-source-stores host-base-stores native-base-stores fruix-runtime-stores))) (references (delete-duplicates
(append (if promoted-native-build-result-store
(list promoted-native-build-result-store)
'())
materialized-source-stores
host-base-stores
native-base-stores
development-package-stores
build-package-stores
fruix-runtime-stores)))
(manifest (string-append (manifest (string-append
"closure-spec=\n" "closure-spec=\n"
(object->string (operating-system-closure-spec os)) (object->string (operating-system-closure-spec os))
@@ -172,9 +248,14 @@
"\n") "\n")
"\nreferences=\n" "\nreferences=\n"
(string-join references "\n"))) (string-join references "\n")))
(hash (sha256-string manifest)) (display-name (string-append "fruix-system-"
(closure-path (string-append store-dir "/" hash "-fruix-system-" (operating-system-host-name os)))
(operating-system-host-name os)))) (closure-path (make-store-path store-dir display-name manifest
#:kind 'operating-system))
(development-profile-path (and (not (null? development-package-stores))
(string-append closure-path "/development-profile")))
(build-profile-path (and (not (null? build-package-stores))
(string-append closure-path "/build-profile"))))
(unless (file-exists? closure-path) (unless (file-exists? closure-path)
(mkdir-p closure-path) (mkdir-p closure-path)
(mkdir-p (string-append closure-path "/boot/kernel")) (mkdir-p (string-append closure-path "/boot/kernel"))
@@ -192,6 +273,16 @@
(for-each (lambda (output) (for-each (lambda (output)
(merge-output-into-tree output (string-append closure-path "/profile"))) (merge-output-into-tree output (string-append closure-path "/profile")))
base-package-stores) base-package-stores)
(when development-profile-path
(mkdir-p development-profile-path)
(for-each (lambda (output)
(merge-output-into-tree output development-profile-path))
development-package-stores))
(when build-profile-path
(mkdir-p build-profile-path)
(for-each (lambda (output)
(merge-output-into-tree output build-profile-path))
build-package-stores))
(for-each (for-each
(lambda (entry) (lambda (entry)
(write-file (string-append closure-path "/" (car entry)) (cdr entry))) (write-file (string-append closure-path "/" (car entry)) (cdr entry)))
@@ -201,6 +292,14 @@
(chmod (string-append closure-path "/etc/master.passwd") #o600)) (chmod (string-append closure-path "/etc/master.passwd") #o600))
(chmod (string-append closure-path "/usr/local/etc/rc.d/fruix-activate") #o555) (chmod (string-append closure-path "/usr/local/etc/rc.d/fruix-activate") #o555)
(chmod (string-append closure-path "/usr/local/etc/rc.d/fruix-shepherd") #o555) (chmod (string-append closure-path "/usr/local/etc/rc.d/fruix-shepherd") #o555)
(when (file-exists? (string-append closure-path "/usr/local/bin/fruix"))
(chmod (string-append closure-path "/usr/local/bin/fruix") #o555))
(when (file-exists? (string-append closure-path "/usr/local/bin/fruix-development-environment"))
(chmod (string-append closure-path "/usr/local/bin/fruix-development-environment") #o555))
(when (file-exists? (string-append closure-path "/usr/local/bin/fruix-build-environment"))
(chmod (string-append closure-path "/usr/local/bin/fruix-build-environment") #o555))
(when (file-exists? (string-append closure-path "/usr/local/bin/fruix-self-hosted-native-build"))
(chmod (string-append closure-path "/usr/local/bin/fruix-self-hosted-native-build") #o555))
(when (file-exists? (string-append closure-path "/boot/fruix-pid1")) (when (file-exists? (string-append closure-path "/boot/fruix-pid1"))
(chmod (string-append closure-path "/boot/fruix-pid1") #o555)) (chmod (string-append closure-path "/boot/fruix-pid1") #o555))
(write-file (string-append closure-path "/parameters.scm") (write-file (string-append closure-path "/parameters.scm")
@@ -215,6 +314,10 @@
(guile-extra-store . ,guile-extra-store) (guile-extra-store . ,guile-extra-store)
(shepherd-store . ,shepherd-store) (shepherd-store . ,shepherd-store)
(base-package-stores . ,base-package-stores) (base-package-stores . ,base-package-stores)
(development-package-stores . ,development-package-stores)
(build-package-stores . ,build-package-stores)
(development-profile-path . ,development-profile-path)
(build-profile-path . ,build-profile-path)
(host-base-stores . ,host-base-stores) (host-base-stores . ,host-base-stores)
(native-base-stores . ,native-base-stores) (native-base-stores . ,native-base-stores)
(fruix-runtime-stores . ,fruix-runtime-stores) (fruix-runtime-stores . ,fruix-runtime-stores)
@@ -223,7 +326,13 @@
(freebsd-source-materializations-file . ,(string-append closure-path "/metadata/freebsd-source-materializations.scm")) (freebsd-source-materializations-file . ,(string-append closure-path "/metadata/freebsd-source-materializations.scm"))
(materialized-source-stores . ,materialized-source-stores) (materialized-source-stores . ,materialized-source-stores)
(host-base-provenance-file . ,(string-append closure-path "/metadata/host-base-provenance.scm")) (host-base-provenance-file . ,(string-append closure-path "/metadata/host-base-provenance.scm"))
(system-declaration-file . ,(string-append closure-path "/metadata/system-declaration.scm"))
(system-declaration-info-file . ,(string-append closure-path "/metadata/system-declaration-info.scm"))
(system-declaration-system-file . ,(string-append closure-path "/metadata/system-declaration-system"))
(store-layout-file . ,(string-append closure-path "/metadata/store-layout.scm")) (store-layout-file . ,(string-append closure-path "/metadata/store-layout.scm"))
(promoted-native-build-result-file
. ,(and promoted-native-build-result-summary
(string-append closure-path "/metadata/promoted-native-build-result.scm")))
(generated-files . ,(map car generated-files)) (generated-files . ,(map car generated-files))
(references . ,references)))) (references . ,references))))
@@ -233,7 +342,7 @@
(mkdir-p (dirname link-name)) (mkdir-p (dirname link-name))
(symlink target link-name)) (symlink target link-name))
(define system-generation-layout-version "1") (define system-generation-layout-version "2")
(define* (system-generation-metadata-object os closure-path (define* (system-generation-metadata-object os closure-path
#:key #:key
@@ -252,6 +361,9 @@
(freebsd-source-materializations-file (freebsd-source-materializations-file
. ,(string-append closure-path "/metadata/freebsd-source-materializations.scm")) . ,(string-append closure-path "/metadata/freebsd-source-materializations.scm"))
(host-base-provenance-file . ,(string-append closure-path "/metadata/host-base-provenance.scm")) (host-base-provenance-file . ,(string-append closure-path "/metadata/host-base-provenance.scm"))
(system-declaration-file . ,(string-append closure-path "/metadata/system-declaration.scm"))
(system-declaration-info-file . ,(string-append closure-path "/metadata/system-declaration-info.scm"))
(system-declaration-system-file . ,(string-append closure-path "/metadata/system-declaration-system"))
(store-layout-file . ,(string-append closure-path "/metadata/store-layout.scm")) (store-layout-file . ,(string-append closure-path "/metadata/store-layout.scm"))
(install-metadata-path . ,install-metadata-path) (install-metadata-path . ,install-metadata-path)
(install-spec . ,install-spec))) (install-spec . ,install-spec)))
@@ -264,6 +376,9 @@
(freebsd-source-materializations-file (freebsd-source-materializations-file
. ,(string-append closure-path "/metadata/freebsd-source-materializations.scm")) . ,(string-append closure-path "/metadata/freebsd-source-materializations.scm"))
(host-base-provenance-file . ,(string-append closure-path "/metadata/host-base-provenance.scm")) (host-base-provenance-file . ,(string-append closure-path "/metadata/host-base-provenance.scm"))
(system-declaration-file . ,(string-append closure-path "/metadata/system-declaration.scm"))
(system-declaration-info-file . ,(string-append closure-path "/metadata/system-declaration-info.scm"))
(system-declaration-system-file . ,(string-append closure-path "/metadata/system-declaration-system"))
(store-layout-file . ,(string-append closure-path "/metadata/store-layout.scm")))) (store-layout-file . ,(string-append closure-path "/metadata/store-layout.scm"))))
(define* (populate-system-generation-layout os rootfs closure-path (define* (populate-system-generation-layout os rootfs closure-path
@@ -308,8 +423,9 @@
(mkdir-p rootfs) (mkdir-p rootfs)
(for-each (lambda (dir) (for-each (lambda (dir)
(mkdir-p (string-append rootfs dir))) (mkdir-p (string-append rootfs dir)))
'("/run" "/boot" "/etc" "/etc/ssh" "/usr" "/usr/share" "/usr/local" "/usr/local/etc" '("/run" "/boot" "/etc" "/etc/ssh" "/usr" "/usr/share" "/usr/local"
"/usr/local/etc/rc.d" "/var" "/var/cron" "/var/db" "/var/lib" "/var/lib/fruix" "/usr/local/bin" "/usr/local/etc" "/usr/local/etc/rc.d" "/var"
"/var/cron" "/var/db" "/var/lib" "/var/lib/fruix"
"/var/log" "/var/run" "/tmp" "/dev" "/root" "/home")) "/var/log" "/var/run" "/tmp" "/dev" "/root" "/home"))
(chmod (string-append rootfs "/tmp") #o1777) (chmod (string-append rootfs "/tmp") #o1777)
(symlink-force closure-path (string-append rootfs "/run/current-system")) (symlink-force closure-path (string-append rootfs "/run/current-system"))
@@ -345,6 +461,37 @@
(symlink-force (string-append "/run/current-system/boot/" path) (symlink-force (string-append "/run/current-system/boot/" path)
(string-append rootfs "/boot/" path))) (string-append rootfs "/boot/" path)))
'("kernel" "loader" "loader.efi" "device.hints" "defaults" "lua" "loader.conf")) '("kernel" "loader" "loader.efi" "device.hints" "defaults" "lua" "loader.conf"))
(symlink-force "/run/current-system/usr/local/bin/fruix"
(string-append rootfs "/usr/local/bin/fruix"))
(when (file-exists? (string-append closure-path "/development-profile"))
(symlink-force "/run/current-system/development-profile"
(string-append rootfs "/run/current-development")))
(when (file-exists? (string-append closure-path "/build-profile"))
(symlink-force "/run/current-system/build-profile"
(string-append rootfs "/run/current-build"))
(when (file-exists? (string-append closure-path "/build-profile/usr/include"))
(symlink-force "/run/current-system/build-profile/usr/include"
(string-append rootfs "/usr/include")))
(when (file-exists? (string-append closure-path "/build-profile/usr/share/mk"))
(symlink-force "/run/current-system/build-profile/usr/share/mk"
(string-append rootfs "/usr/share/mk"))))
(when (and (not (file-exists? (string-append closure-path "/build-profile")))
(file-exists? (string-append closure-path "/development-profile")))
(when (file-exists? (string-append closure-path "/development-profile/usr/include"))
(symlink-force "/run/current-system/development-profile/usr/include"
(string-append rootfs "/usr/include")))
(when (file-exists? (string-append closure-path "/development-profile/usr/share/mk"))
(symlink-force "/run/current-system/development-profile/usr/share/mk"
(string-append rootfs "/usr/share/mk"))))
(when (file-exists? (string-append closure-path "/usr/local/bin/fruix-development-environment"))
(symlink-force "/run/current-system/usr/local/bin/fruix-development-environment"
(string-append rootfs "/usr/local/bin/fruix-development-environment")))
(when (file-exists? (string-append closure-path "/usr/local/bin/fruix-build-environment"))
(symlink-force "/run/current-system/usr/local/bin/fruix-build-environment"
(string-append rootfs "/usr/local/bin/fruix-build-environment")))
(when (file-exists? (string-append closure-path "/usr/local/bin/fruix-self-hosted-native-build"))
(symlink-force "/run/current-system/usr/local/bin/fruix-self-hosted-native-build"
(string-append rootfs "/usr/local/bin/fruix-self-hosted-native-build")))
(symlink-force "/run/current-system/usr/local/etc/rc.d/fruix-activate" (symlink-force "/run/current-system/usr/local/etc/rc.d/fruix-activate"
(string-append rootfs "/usr/local/etc/rc.d/fruix-activate")) (string-append rootfs "/usr/local/etc/rc.d/fruix-activate"))
(symlink-force "/run/current-system/usr/local/etc/rc.d/fruix-shepherd" (symlink-force "/run/current-system/usr/local/etc/rc.d/fruix-shepherd"
@@ -362,12 +509,18 @@
(store-dir "/frx/store") (store-dir "/frx/store")
(guile-prefix "/tmp/guile-freebsd-validate-install") (guile-prefix "/tmp/guile-freebsd-validate-install")
(guile-extra-prefix "/tmp/guile-gnutls-freebsd-validate-install") (guile-extra-prefix "/tmp/guile-gnutls-freebsd-validate-install")
(shepherd-prefix "/tmp/shepherd-freebsd-validate-install")) (shepherd-prefix "/tmp/shepherd-freebsd-validate-install")
(declaration-source #f)
(declaration-origin #f)
(declaration-system-symbol #f))
(let* ((closure (materialize-operating-system os (let* ((closure (materialize-operating-system os
#:store-dir store-dir #:store-dir store-dir
#:guile-prefix guile-prefix #:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix #:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix)) #:shepherd-prefix shepherd-prefix
#:declaration-source declaration-source
#:declaration-origin declaration-origin
#:declaration-system-symbol declaration-system-symbol))
(closure-path (assoc-ref closure 'closure-path))) (closure-path (assoc-ref closure 'closure-path)))
(populate-rootfs-from-closure os rootfs closure-path))) (populate-rootfs-from-closure os rootfs closure-path)))
@@ -563,10 +716,10 @@
(installer-root-partition-label . ,installer-root-partition-label) (installer-root-partition-label . ,installer-root-partition-label)
(target-install . ,target-install-spec)))) (target-install . ,target-install-spec))))
(define image-builder-version "2") (define image-builder-version "3")
(define install-builder-version "1") (define install-builder-version "2")
(define installer-image-builder-version "1") (define installer-image-builder-version "2")
(define installer-iso-builder-version "2") (define installer-iso-builder-version "3")
(define (operating-system-install-metadata-object install-spec closure-path store-items) (define (operating-system-install-metadata-object install-spec closure-path store-items)
`((install-version . ,install-builder-version) `((install-version . ,install-builder-version)
@@ -708,6 +861,9 @@
(guile-prefix "/tmp/guile-freebsd-validate-install") (guile-prefix "/tmp/guile-freebsd-validate-install")
(guile-extra-prefix "/tmp/guile-gnutls-freebsd-validate-install") (guile-extra-prefix "/tmp/guile-gnutls-freebsd-validate-install")
(shepherd-prefix "/tmp/shepherd-freebsd-validate-install") (shepherd-prefix "/tmp/shepherd-freebsd-validate-install")
(declaration-source #f)
(declaration-origin #f)
(declaration-system-symbol #f)
(efi-size "64m") (efi-size "64m")
(root-size #f) (root-size #f)
(disk-capacity #f) (disk-capacity #f)
@@ -720,7 +876,10 @@
#:store-dir store-dir #:store-dir store-dir
#:guile-prefix guile-prefix #:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix #:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix)) #:shepherd-prefix shepherd-prefix
#:declaration-source declaration-source
#:declaration-origin declaration-origin
#:declaration-system-symbol declaration-system-symbol))
(closure-path (assoc-ref closure 'closure-path)) (closure-path (assoc-ref closure 'closure-path))
(store-items (store-reference-closure (list closure-path))) (store-items (store-reference-closure (list closure-path)))
(target-kind (if (string-prefix? "/dev/" target) (target-kind (if (string-prefix? "/dev/" target)
@@ -835,6 +994,9 @@
(guile-prefix "/tmp/guile-freebsd-validate-install") (guile-prefix "/tmp/guile-freebsd-validate-install")
(guile-extra-prefix "/tmp/guile-gnutls-freebsd-validate-install") (guile-extra-prefix "/tmp/guile-gnutls-freebsd-validate-install")
(shepherd-prefix "/tmp/shepherd-freebsd-validate-install") (shepherd-prefix "/tmp/shepherd-freebsd-validate-install")
(declaration-source #f)
(declaration-origin #f)
(declaration-system-symbol #f)
(efi-size "64m") (efi-size "64m")
(root-size "256m") (root-size "256m")
(disk-capacity #f) (disk-capacity #f)
@@ -845,7 +1007,10 @@
#:store-dir store-dir #:store-dir store-dir
#:guile-prefix guile-prefix #:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix #:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix)) #:shepherd-prefix shepherd-prefix
#:declaration-source declaration-source
#:declaration-origin declaration-origin
#:declaration-system-symbol declaration-system-symbol))
(closure-path (assoc-ref closure 'closure-path)) (closure-path (assoc-ref closure 'closure-path))
(image-spec (operating-system-image-spec os (image-spec (operating-system-image-spec os
#:efi-size efi-size #:efi-size efi-size
@@ -865,9 +1030,10 @@
"\nstore-items=\n" "\nstore-items=\n"
(string-join store-items "\n") (string-join store-items "\n")
"\n")) "\n"))
(hash (sha256-string manifest)) (display-name (string-append "fruix-bhyve-image-"
(image-store-path (string-append store-dir "/" hash "-fruix-bhyve-image-" (operating-system-host-name os)))
(operating-system-host-name os))) (image-store-path (make-store-path store-dir display-name manifest
#:kind 'bhyve-image))
(disk-image (string-append image-store-path "/disk.img")) (disk-image (string-append image-store-path "/disk.img"))
(esp-image (string-append image-store-path "/esp.img")) (esp-image (string-append image-store-path "/esp.img"))
(root-image (string-append image-store-path "/root.ufs"))) (root-image (string-append image-store-path "/root.ufs")))
@@ -887,7 +1053,10 @@
#:store-dir store-dir #:store-dir store-dir
#:guile-prefix guile-prefix #:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix #:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix) #:shepherd-prefix shepherd-prefix
#:declaration-source declaration-source
#:declaration-origin declaration-origin
#:declaration-system-symbol declaration-system-symbol)
(copy-rootfs-for-image rootfs image-rootfs) (copy-rootfs-for-image rootfs image-rootfs)
(copy-store-items-into-rootfs image-rootfs store-dir store-items) (copy-store-items-into-rootfs image-rootfs store-dir store-items)
(mkdir-p (string-append esp-stage "/EFI/BOOT")) (mkdir-p (string-append esp-stage "/EFI/BOOT"))
@@ -955,6 +1124,9 @@
(guile-prefix "/tmp/guile-freebsd-validate-install") (guile-prefix "/tmp/guile-freebsd-validate-install")
(guile-extra-prefix "/tmp/guile-gnutls-freebsd-validate-install") (guile-extra-prefix "/tmp/guile-gnutls-freebsd-validate-install")
(shepherd-prefix "/tmp/shepherd-freebsd-validate-install") (shepherd-prefix "/tmp/shepherd-freebsd-validate-install")
(declaration-source #f)
(declaration-origin #f)
(declaration-system-symbol #f)
(install-target-device "/dev/vtbd1") (install-target-device "/dev/vtbd1")
(efi-size "64m") (efi-size "64m")
(root-size "10g") (root-size "10g")
@@ -973,12 +1145,18 @@
#:store-dir store-dir #:store-dir store-dir
#:guile-prefix guile-prefix #:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix #:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix)) #:shepherd-prefix shepherd-prefix
#:declaration-source declaration-source
#:declaration-origin declaration-origin
#:declaration-system-symbol declaration-system-symbol))
(installer-closure (materialize-operating-system installer-os (installer-closure (materialize-operating-system installer-os
#:store-dir store-dir #:store-dir store-dir
#:guile-prefix guile-prefix #:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix #:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix)) #:shepherd-prefix shepherd-prefix
#:declaration-source declaration-source
#:declaration-origin declaration-origin
#:declaration-system-symbol declaration-system-symbol))
(target-closure-path (assoc-ref target-closure 'closure-path)) (target-closure-path (assoc-ref target-closure 'closure-path))
(installer-closure-path (assoc-ref installer-closure 'closure-path)) (installer-closure-path (assoc-ref installer-closure 'closure-path))
(target-store-items (store-reference-closure (list target-closure-path))) (target-store-items (store-reference-closure (list target-closure-path)))
@@ -1019,9 +1197,10 @@
"\ninstall-metadata=\n" "\ninstall-metadata=\n"
(object->string install-metadata) (object->string install-metadata)
"\n")) "\n"))
(hash (sha256-string manifest)) (display-name (string-append "fruix-installer-image-"
(image-store-path (string-append store-dir "/" hash "-fruix-installer-image-" (operating-system-host-name installer-os)))
(operating-system-host-name installer-os))) (image-store-path (make-store-path store-dir display-name manifest
#:kind 'installer-image))
(disk-image (string-append image-store-path "/disk.img")) (disk-image (string-append image-store-path "/disk.img"))
(esp-image (string-append image-store-path "/esp.img")) (esp-image (string-append image-store-path "/esp.img"))
(root-image (string-append image-store-path "/root.ufs"))) (root-image (string-append image-store-path "/root.ufs")))
@@ -1262,6 +1441,9 @@
(guile-prefix "/tmp/guile-freebsd-validate-install") (guile-prefix "/tmp/guile-freebsd-validate-install")
(guile-extra-prefix "/tmp/guile-gnutls-freebsd-validate-install") (guile-extra-prefix "/tmp/guile-gnutls-freebsd-validate-install")
(shepherd-prefix "/tmp/shepherd-freebsd-validate-install") (shepherd-prefix "/tmp/shepherd-freebsd-validate-install")
(declaration-source #f)
(declaration-origin #f)
(declaration-system-symbol #f)
(install-target-device "/dev/vtbd0") (install-target-device "/dev/vtbd0")
(root-size #f) (root-size #f)
(installer-host-name (string-append (operating-system-host-name os) (installer-host-name (string-append (operating-system-host-name os)
@@ -1278,12 +1460,18 @@
#:store-dir store-dir #:store-dir store-dir
#:guile-prefix guile-prefix #:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix #:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix)) #:shepherd-prefix shepherd-prefix
#:declaration-source declaration-source
#:declaration-origin declaration-origin
#:declaration-system-symbol declaration-system-symbol))
(installer-closure (materialize-operating-system installer-os (installer-closure (materialize-operating-system installer-os
#:store-dir store-dir #:store-dir store-dir
#:guile-prefix guile-prefix #:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix #:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix)) #:shepherd-prefix shepherd-prefix
#:declaration-source declaration-source
#:declaration-origin declaration-origin
#:declaration-system-symbol declaration-system-symbol))
(target-closure-path (assoc-ref target-closure 'closure-path)) (target-closure-path (assoc-ref target-closure 'closure-path))
(installer-closure-path (assoc-ref installer-closure 'closure-path)) (installer-closure-path (assoc-ref installer-closure 'closure-path))
(target-closure-store-items (store-reference-closure (list target-closure-path))) (target-closure-store-items (store-reference-closure (list target-closure-path)))
@@ -1324,9 +1512,10 @@
"\ninstall-metadata=\n" "\ninstall-metadata=\n"
(object->string install-metadata) (object->string install-metadata)
"\n")) "\n"))
(hash (sha256-string manifest)) (display-name (string-append "fruix-installer-iso-"
(iso-store-path (string-append store-dir "/" hash "-fruix-installer-iso-" (operating-system-host-name installer-os)))
(operating-system-host-name installer-os))) (iso-store-path (make-store-path store-dir display-name manifest
#:kind 'installer-iso))
(iso-image (string-append iso-store-path "/installer.iso")) (iso-image (string-append iso-store-path "/installer.iso"))
(boot-efi-image (string-append iso-store-path "/efiboot.img")) (boot-efi-image (string-append iso-store-path "/efiboot.img"))
(root-image (string-append iso-store-path "/root.img"))) (root-image (string-append iso-store-path "/root.img")))

126
modules/fruix/system/freebsd/model.scm
View File

@@ -26,13 +26,22 @@
file-system-type file-system-type
file-system-options file-system-options
file-system-needed-for-boot? file-system-needed-for-boot?
make-promoted-native-build-result
promoted-native-build-result?
promoted-native-build-result-store-path
promoted-native-build-result-metadata-file
promoted-native-build-result-metadata
promoted-native-build-result-spec
operating-system operating-system
operating-system? operating-system?
operating-system-host-name operating-system-host-name
operating-system-freebsd-base operating-system-freebsd-base
operating-system-native-build-result
operating-system-kernel operating-system-kernel
operating-system-bootloader operating-system-bootloader
operating-system-base-packages operating-system-base-packages
operating-system-development-packages
operating-system-build-packages
operating-system-users operating-system-users
operating-system-groups operating-system-groups
operating-system-file-systems operating-system-file-systems
@@ -94,16 +103,71 @@
(needed-for-boot? #f)) (needed-for-boot? #f))
(make-file-system device mount-point type options needed-for-boot?)) (make-file-system device mount-point type options needed-for-boot?))
(define-record-type <promoted-native-build-result>
(make-promoted-native-build-result store-path metadata-file metadata)
promoted-native-build-result?
(store-path promoted-native-build-result-store-path)
(metadata-file promoted-native-build-result-metadata-file)
(metadata promoted-native-build-result-metadata))
(define (promoted-native-build-result-metadata-ref metadata key default)
(match (assoc key metadata)
((_ . value) value)
(#f default)))
(define (promoted-native-build-result-artifact-spec metadata artifact-kind)
(find (lambda (entry)
(eq? (promoted-native-build-result-metadata-ref entry 'artifact-kind #f)
artifact-kind))
(promoted-native-build-result-metadata-ref metadata 'artifacts '())))
(define (promoted-native-build-result-spec result)
(let* ((metadata (promoted-native-build-result-metadata result))
(base (promoted-native-build-result-metadata-ref metadata 'freebsd-base '()))
(source (promoted-native-build-result-metadata-ref metadata 'source '())))
`((store-path . ,(promoted-native-build-result-store-path result))
(metadata-file . ,(promoted-native-build-result-metadata-file result))
(executor-kind . ,(promoted-native-build-result-metadata-ref metadata 'executor-kind #f))
(executor-name . ,(promoted-native-build-result-metadata-ref metadata 'executor-name #f))
(executor-version . ,(promoted-native-build-result-metadata-ref metadata 'executor-version #f))
(run-id . ,(promoted-native-build-result-metadata-ref metadata 'run-id #f))
(version-label . ,(promoted-native-build-result-metadata-ref base 'version-label #f))
(release . ,(promoted-native-build-result-metadata-ref base 'release #f))
(branch . ,(promoted-native-build-result-metadata-ref base 'branch #f))
(source-store . ,(promoted-native-build-result-metadata-ref source 'store-path #f))
(source-root . ,(promoted-native-build-result-metadata-ref source 'source-root #f))
(artifact-count . ,(promoted-native-build-result-metadata-ref metadata 'artifact-count 0))
(world-store . ,(promoted-native-build-result-metadata-ref
(promoted-native-build-result-artifact-spec metadata 'world)
'store-path
#f))
(kernel-store . ,(promoted-native-build-result-metadata-ref
(promoted-native-build-result-artifact-spec metadata 'kernel)
'store-path
#f))
(headers-store . ,(promoted-native-build-result-metadata-ref
(promoted-native-build-result-artifact-spec metadata 'headers)
'store-path
#f))
(bootloader-store . ,(promoted-native-build-result-metadata-ref
(promoted-native-build-result-artifact-spec metadata 'bootloader)
'store-path
#f)))))
(define-record-type <operating-system> (define-record-type <operating-system>
(make-operating-system host-name freebsd-base kernel bootloader base-packages users groups (make-operating-system host-name freebsd-base native-build-result kernel bootloader
file-systems services loader-entries rc-conf-entries base-packages development-packages build-packages users groups file-systems
init-mode ready-marker root-authorized-keys) services loader-entries rc-conf-entries init-mode ready-marker
root-authorized-keys)
operating-system? operating-system?
(host-name operating-system-host-name) (host-name operating-system-host-name)
(freebsd-base operating-system-freebsd-base) (freebsd-base operating-system-freebsd-base)
(native-build-result operating-system-native-build-result)
(kernel operating-system-kernel) (kernel operating-system-kernel)
(bootloader operating-system-bootloader) (bootloader operating-system-bootloader)
(base-packages operating-system-base-packages) (base-packages operating-system-base-packages)
(development-packages operating-system-development-packages)
(build-packages operating-system-build-packages)
(users operating-system-users) (users operating-system-users)
(groups operating-system-groups) (groups operating-system-groups)
(file-systems operating-system-file-systems) (file-systems operating-system-file-systems)
@@ -117,9 +181,12 @@
(define* (operating-system #:key (define* (operating-system #:key
(host-name "fruix-freebsd") (host-name "fruix-freebsd")
(freebsd-base %default-freebsd-base) (freebsd-base %default-freebsd-base)
(native-build-result #f)
(kernel freebsd-kernel) (kernel freebsd-kernel)
(bootloader freebsd-bootloader) (bootloader freebsd-bootloader)
(base-packages %freebsd-system-packages) (base-packages %freebsd-system-packages)
(development-packages '())
(build-packages '())
(users (list (user-account #:name "root" (users (list (user-account #:name "root"
#:uid 0 #:uid 0
#:group "wheel" #:group "wheel"
@@ -161,9 +228,10 @@
(init-mode 'freebsd-init+rc.d-shepherd) (init-mode 'freebsd-init+rc.d-shepherd)
(ready-marker "/var/lib/fruix/ready") (ready-marker "/var/lib/fruix/ready")
(root-authorized-keys '())) (root-authorized-keys '()))
(make-operating-system host-name freebsd-base kernel bootloader base-packages users groups (make-operating-system host-name freebsd-base native-build-result kernel bootloader
file-systems services loader-entries rc-conf-entries base-packages development-packages build-packages users groups file-systems
init-mode ready-marker root-authorized-keys)) services loader-entries rc-conf-entries init-mode ready-marker
root-authorized-keys))
(define default-minimal-operating-system (operating-system)) (define default-minimal-operating-system (operating-system))
@@ -231,6 +299,10 @@
(define (validate-operating-system os) (define (validate-operating-system os)
(let* ((host-name (operating-system-host-name os)) (let* ((host-name (operating-system-host-name os))
(base (operating-system-freebsd-base os)) (base (operating-system-freebsd-base os))
(native-build-result (operating-system-native-build-result os))
(base-packages (operating-system-base-packages os))
(development-packages (operating-system-development-packages os))
(build-packages (operating-system-build-packages os))
(users (operating-system-users os)) (users (operating-system-users os))
(groups (operating-system-groups os)) (groups (operating-system-groups os))
(file-systems (operating-system-file-systems os)) (file-systems (operating-system-file-systems os))
@@ -242,6 +314,15 @@
(error "operating-system host-name must not be empty")) (error "operating-system host-name must not be empty"))
(unless (freebsd-base? base) (unless (freebsd-base? base)
(error "operating-system freebsd-base must be a <freebsd-base> record")) (error "operating-system freebsd-base must be a <freebsd-base> record"))
(when native-build-result
(unless (promoted-native-build-result? native-build-result)
(error "operating-system native-build-result must be a <promoted-native-build-result> record")))
(unless (every freebsd-package? base-packages)
(error "operating-system base-packages must be a list of <freebsd-package> records"))
(unless (every freebsd-package? development-packages)
(error "operating-system development-packages must be a list of <freebsd-package> records"))
(unless (every freebsd-package? build-packages)
(error "operating-system build-packages must be a list of <freebsd-package> records"))
(validate-freebsd-source (freebsd-base-source base)) (validate-freebsd-source (freebsd-base-source base))
(let ((dups (duplicate-elements user-names))) (let ((dups (duplicate-elements user-names)))
(unless (null? dups) (unless (null? dups)
@@ -294,8 +375,23 @@
"metadata/freebsd-base.scm" "metadata/freebsd-base.scm"
"metadata/host-base-provenance.scm" "metadata/host-base-provenance.scm"
"metadata/store-layout.scm" "metadata/store-layout.scm"
"metadata/system-declaration.scm"
"metadata/system-declaration-info.scm"
"metadata/system-declaration-system"
"activate" "activate"
"shepherd/init.scm") "shepherd/init.scm"
"share/fruix/node/scripts/fruix.scm"
"usr/local/bin/fruix")
(if (operating-system-native-build-result os)
'("metadata/promoted-native-build-result.scm")
'())
(if (null? (operating-system-development-packages os))
'()
'("usr/local/bin/fruix-development-environment"))
(if (null? (operating-system-build-packages os))
'()
'("usr/local/bin/fruix-build-environment"
"usr/local/bin/fruix-self-hosted-native-build"))
(if (pid1-init-mode? os) (if (pid1-init-mode? os)
'("boot/fruix-pid1") '("boot/fruix-pid1")
'()) '())
@@ -311,10 +407,26 @@
(validate-operating-system os) (validate-operating-system os)
`((host-name . ,(operating-system-host-name os)) `((host-name . ,(operating-system-host-name os))
(freebsd-base . ,(freebsd-base-spec (operating-system-freebsd-base os))) (freebsd-base . ,(freebsd-base-spec (operating-system-freebsd-base os)))
(promoted-native-build-result
. ,(and (operating-system-native-build-result os)
(promoted-native-build-result-spec
(operating-system-native-build-result os))))
(kernel-package . ,(freebsd-package-name (operating-system-kernel os))) (kernel-package . ,(freebsd-package-name (operating-system-kernel os)))
(bootloader-package . ,(freebsd-package-name (operating-system-bootloader os))) (bootloader-package . ,(freebsd-package-name (operating-system-bootloader os)))
(base-package-count . ,(length (operating-system-base-packages os))) (base-package-count . ,(length (operating-system-base-packages os)))
(base-packages . ,(package-names (operating-system-base-packages os))) (base-packages . ,(package-names (operating-system-base-packages os)))
(development-package-count . ,(length (operating-system-development-packages os)))
(development-packages . ,(package-names (operating-system-development-packages os)))
(build-package-count . ,(length (operating-system-build-packages os)))
(build-packages . ,(package-names (operating-system-build-packages os)))
(installed-system-command-surface-version . "2")
(bundled-fruix-node-cli-version . "1")
(development-environment-helper-version
. ,(if (null? (operating-system-development-packages os)) #f "1"))
(build-environment-helper-version
. ,(if (null? (operating-system-build-packages os)) #f "1"))
(self-hosted-native-build-helper-version
. ,(if (null? (operating-system-build-packages os)) #f "5"))
(user-count . ,(length (operating-system-users os))) (user-count . ,(length (operating-system-users os)))
(users . ,(map user-account-name (operating-system-users os))) (users . ,(map user-account-name (operating-system-users os)))
(group-count . ,(length (operating-system-groups os))) (group-count . ,(length (operating-system-groups os)))

736
modules/fruix/system/freebsd/render.scm
View File

@@ -4,6 +4,7 @@
#:use-module (ice-9 match) #:use-module (ice-9 match)
#:use-module (srfi srfi-1) #:use-module (srfi srfi-1)
#:use-module (srfi srfi-13) #:use-module (srfi srfi-13)
#:use-module (rnrs io ports)
#:export (operating-system-generated-files #:export (operating-system-generated-files
render-activation-rc-script render-activation-rc-script
render-rc-script)) render-rc-script))
@@ -468,6 +469,726 @@
"load_rc_config $name\n" "load_rc_config $name\n"
"run_rc_command \"$1\"\n"))) "run_rc_command \"$1\"\n")))
(define (path-parent path)
(let ((index (string-rindex path #\/)))
(cond
((not index) ".")
((zero? index) "/")
(else (substring path 0 index)))))
(define (read-source-file-string path)
(call-with-input-file path get-string-all))
(define (bundled-fruix-node-files)
(let* ((repo-root (or (getenv "FRUIX_PROJECT_ROOT")
(let ((render-file (current-filename)))
(and render-file
(path-parent
(path-parent
(path-parent
(path-parent
(path-parent render-file)))))))
(getcwd)))
(guix-root (or (getenv "GUIX_SOURCE_DIR")
(string-append (getenv "HOME") "/repos/guix")))
(specs `((,(string-append repo-root "/scripts/fruix.scm")
. "share/fruix/node/scripts/fruix.scm")
(,(string-append repo-root "/modules/fruix/packages/freebsd.scm")
. "share/fruix/node/modules/fruix/packages/freebsd.scm")
(,(string-append repo-root "/modules/fruix/system/freebsd.scm")
. "share/fruix/node/modules/fruix/system/freebsd.scm")
(,(string-append repo-root "/modules/fruix/system/freebsd/build.scm")
. "share/fruix/node/modules/fruix/system/freebsd/build.scm")
(,(string-append repo-root "/modules/fruix/system/freebsd/executor.scm")
. "share/fruix/node/modules/fruix/system/freebsd/executor.scm")
(,(string-append repo-root "/modules/fruix/system/freebsd/media.scm")
. "share/fruix/node/modules/fruix/system/freebsd/media.scm")
(,(string-append repo-root "/modules/fruix/system/freebsd/model.scm")
. "share/fruix/node/modules/fruix/system/freebsd/model.scm")
(,(string-append repo-root "/modules/fruix/system/freebsd/render.scm")
. "share/fruix/node/modules/fruix/system/freebsd/render.scm")
(,(string-append repo-root "/modules/fruix/system/freebsd/source.scm")
. "share/fruix/node/modules/fruix/system/freebsd/source.scm")
(,(string-append repo-root "/modules/fruix/system/freebsd/utils.scm")
. "share/fruix/node/modules/fruix/system/freebsd/utils.scm")
(,(string-append guix-root "/guix/build/utils.scm")
. "share/fruix/node/guix/guix/build/utils.scm"))))
(map (lambda (entry)
(cons (cdr entry)
(read-source-file-string (car entry))))
specs)))
(define (render-installed-system-fruix os guile-store guile-extra-store shepherd-store)
(string-append
"#!/bin/sh\n"
"set -eu\n"
"PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin\n"
"tool_closure=$(readlink /run/current-system 2>/dev/null || true)\n"
"if [ -n \"$tool_closure\" ]; then\n"
" PATH=\"$tool_closure/profile/bin:$tool_closure/profile/sbin:$tool_closure/profile/usr/bin:$tool_closure/profile/usr/sbin:$PATH\"\n"
"fi\n"
"export PATH\n\n"
"system_root=/var/lib/fruix/system\n"
"generations_root=\"$system_root/generations\"\n"
"current_link=\"$system_root/current\"\n"
"current_generation_file=\"$system_root/current-generation\"\n"
"rollback_link=\"$system_root/rollback\"\n"
"rollback_generation_file=\"$system_root/rollback-generation\"\n"
"gcroots_root=/frx/var/fruix/gcroots\n"
"run_current_link=/run/current-system\n"
"node_root=/run/current-system/share/fruix/node\n"
"node_script=\"$node_root/scripts/fruix.scm\"\n"
"node_module_root=\"$node_root/modules\"\n"
"node_guix_root=\"$node_root/guix\"\n"
"declaration_file=/run/current-system/metadata/system-declaration.scm\n"
"declaration_info_file=/run/current-system/metadata/system-declaration-info.scm\n"
"declaration_system_file=/run/current-system/metadata/system-declaration-system\n"
"default_store_dir=/frx/store\n"
"guile_store='" guile-store "'\n"
"guile_extra_store='" guile-extra-store "'\n"
"shepherd_store='" shepherd-store "'\n"
"layout_version=2\n"
"host_name='" (operating-system-host-name os) "'\n"
"ready_marker='" (operating-system-ready-marker os) "'\n"
"init_mode='" (symbol->string (operating-system-init-mode os)) "'\n\n"
"usage()\n"
"{\n"
" cat <<'EOF'\n"
"Usage: fruix system status\n"
" fruix system build [DECLARATION [--system NAME] ...]\n"
" fruix system reconfigure [DECLARATION [--system NAME] ...]\n"
" fruix system switch /frx/store/...-fruix-system-...\n"
" fruix system rollback\n"
"EOF\n"
"}\n\n"
"die()\n"
"{\n"
" echo \"fruix: $*\" >&2\n"
" exit 1\n"
"}\n\n"
"read_link_maybe()\n"
"{\n"
" if [ -L \"$1\" ]; then\n"
" readlink \"$1\"\n"
" fi\n"
"}\n\n"
"read_file_maybe()\n"
"{\n"
" if [ -f \"$1\" ]; then\n"
" tr -d '\\n' < \"$1\"\n"
" fi\n"
"}\n\n"
"default_system_name()\n"
"{\n"
" read_file_maybe \"$declaration_system_file\"\n"
"}\n\n"
"symlink_force()\n"
"{\n"
" target=$1\n"
" link_name=$2\n"
" tmp_link=\"${link_name}.new.$$\"\n"
" mkdir -p \"$(dirname \"$link_name\")\"\n"
" if [ \"$link_name\" = \"$run_current_link\" ]; then\n"
" rm -f \"$tmp_link\"\n"
" ln -s \"$target\" \"$tmp_link\"\n"
" mv -h -f \"$tmp_link\" \"$link_name\"\n"
" else\n"
" rm -f \"$link_name\"\n"
" ln -s \"$target\" \"$link_name\"\n"
" fi\n"
"}\n\n"
"validate_closure()\n"
"{\n"
" closure=$1\n"
" [ -d \"$closure\" ] || die \"missing closure directory: $closure\"\n"
" [ -f \"$closure/activate\" ] || die \"closure is missing activate script: $closure\"\n"
" [ -f \"$closure/shepherd/init.scm\" ] || die \"closure is missing shepherd config: $closure\"\n"
" [ -f \"$closure/boot/loader.efi\" ] || die \"closure is missing loader.efi: $closure\"\n"
"}\n\n"
"ensure_default_declaration()\n"
"{\n"
" [ -f \"$declaration_file\" ] || die \"current declaration file is missing: $declaration_file\"\n"
" [ -f \"$declaration_info_file\" ] || die \"current declaration info file is missing: $declaration_info_file\"\n"
" current_system_name=$(default_system_name)\n"
" [ -n \"$current_system_name\" ] || die \"current declaration is missing a system variable name\"\n"
"}\n\n"
"run_node_cli()\n"
"{\n"
" [ -x \"$guile_store/bin/guile\" ] || die \"missing Guile runtime: $guile_store/bin/guile\"\n"
" [ -f \"$node_script\" ] || die \"missing bundled Fruix node CLI: $node_script\"\n"
" [ -d \"$node_module_root\" ] || die \"missing bundled Fruix modules: $node_module_root\"\n"
" [ -d \"$node_guix_root\" ] || die \"missing bundled Guix modules: $node_guix_root\"\n"
" guile_load_path=\"$node_module_root:$node_guix_root:$shepherd_store/share/guile/site/3.0:$guile_extra_store/share/guile/site/3.0\"\n"
" guile_system_path=\"$guile_store/share/guile/3.0:$guile_store/share/guile/site/3.0:$guile_store/share/guile/site:$guile_store/share/guile\"\n"
" guile_system_compiled_path=\"$guile_store/lib/guile/3.0/ccache:$guile_store/lib/guile/3.0/site-ccache\"\n"
" guile_load_compiled_path=\"$shepherd_store/lib/guile/3.0/site-ccache:$guile_extra_store/lib/guile/3.0/site-ccache\"\n"
" guile_system_extensions_path=\"$guile_store/lib/guile/3.0/extensions\"\n"
" guile_extensions_path=\"$guile_extra_store/lib/guile/3.0/extensions\"\n"
" ld_library_path=\"$guile_extra_store/lib:$guile_store/lib:/usr/local/lib\"\n"
" env \\\n"
" GUILE_AUTO_COMPILE=0 \\\n"
" GUILE_SYSTEM_PATH=\"$guile_system_path\" \\\n"
" GUILE_LOAD_PATH=\"$guile_load_path\" \\\n"
" GUILE_SYSTEM_COMPILED_PATH=\"$guile_system_compiled_path\" \\\n"
" GUILE_LOAD_COMPILED_PATH=\"$guile_load_compiled_path\" \\\n"
" GUILE_SYSTEM_EXTENSIONS_PATH=\"$guile_system_extensions_path\" \\\n"
" GUILE_EXTENSIONS_PATH=\"$guile_extensions_path\" \\\n"
" LD_LIBRARY_PATH=\"$ld_library_path\" \\\n"
" GUILE_PREFIX=\"$guile_store\" \\\n"
" GUILE_EXTRA_PREFIX=\"$guile_extra_store\" \\\n"
" SHEPHERD_PREFIX=\"$shepherd_store\" \\\n"
" FRUIX_GUILE_STORE=\"$guile_store\" \\\n"
" FRUIX_GUILE_EXTRA_STORE=\"$guile_extra_store\" \\\n"
" FRUIX_SHEPHERD_STORE=\"$shepherd_store\" \\\n"
" GUIX_SOURCE_DIR=\"$node_guix_root\" \\\n"
" FRUIX_PROJECT_ROOT=\"$node_root\" \\\n"
" \"$guile_store/bin/guile\" --no-auto-compile -s \"$node_script\" \"$@\"\n"
"}\n\n"
"system_build()\n"
"{\n"
" if [ $# -eq 0 ]; then\n"
" ensure_default_declaration\n"
" run_node_cli system build \"$declaration_file\" --system \"$current_system_name\" --store \"$default_store_dir\"\n"
" else\n"
" run_node_cli system build \"$@\"\n"
" fi\n"
"}\n\n"
"reconfigure_system()\n"
"{\n"
" build_output=$(mktemp /tmp/fruix-system-reconfigure.XXXXXX)\n"
" if [ $# -eq 0 ]; then\n"
" ensure_default_declaration\n"
" if ! run_node_cli system build \"$declaration_file\" --system \"$current_system_name\" --store \"$default_store_dir\" > \"$build_output\"; then\n"
" cat \"$build_output\" >&2 || true\n"
" rm -f \"$build_output\"\n"
" exit 1\n"
" fi\n"
" else\n"
" if ! run_node_cli system build \"$@\" > \"$build_output\"; then\n"
" cat \"$build_output\" >&2 || true\n"
" rm -f \"$build_output\"\n"
" exit 1\n"
" fi\n"
" fi\n"
" closure=$(sed -n 's/^closure_path=//p' \"$build_output\" | tail -n 1)\n"
" [ -n \"$closure\" ] || die \"failed to recover closure_path from in-system build output\"\n"
" cat \"$build_output\"\n"
" rm -f \"$build_output\"\n"
" switch_to_closure \"$closure\"\n"
" printf 'reconfigure_closure=%s\\n' \"$closure\"\n"
" printf 'reboot_required=true\\n'\n"
"}\n\n"
"max_generation_number()\n"
"{\n"
" max=0\n"
" if [ -d \"$generations_root\" ]; then\n"
" for path in \"$generations_root\"/*; do\n"
" [ -d \"$path\" ] || continue\n"
" base=$(basename \"$path\")\n"
" case \"$base\" in\n"
" ''|*[!0-9]*)\n"
" continue\n"
" ;;\n"
" esac\n"
" if [ \"$base\" -gt \"$max\" ]; then\n"
" max=$base\n"
" fi\n"
" done\n"
" fi\n"
" printf '%s\\n' \"$max\"\n"
"}\n\n"
"next_generation_number()\n"
"{\n"
" max=$(max_generation_number)\n"
" printf '%s\\n' $((max + 1))\n"
"}\n\n"
"write_generation_metadata()\n"
"{\n"
" generation=$1\n"
" closure=$2\n"
" action=$3\n"
" previous_generation=$4\n"
" previous_closure=$5\n"
" generation_dir=\"$generations_root/$generation\"\n"
" install_metadata_path=\"/var/lib/fruix/system/generations/$generation/install.scm\"\n"
" cat > \"$generation_dir/metadata.scm\" <<EOF\n"
"((system-generation-version . \"$layout_version\")\n"
" (generation-number . $generation)\n"
" (host-name . \"$host_name\")\n"
" (ready-marker . \"$ready_marker\")\n"
" (init-mode . $init_mode)\n"
" (closure-path . \"$closure\")\n"
" (parameters-file . \"$closure/parameters.scm\")\n"
" (freebsd-base-file . \"$closure/metadata/freebsd-base.scm\")\n"
" (freebsd-source-file . \"$closure/metadata/freebsd-source.scm\")\n"
" (freebsd-source-materializations-file . \"$closure/metadata/freebsd-source-materializations.scm\")\n"
" (host-base-provenance-file . \"$closure/metadata/host-base-provenance.scm\")\n"
" (store-layout-file . \"$closure/metadata/store-layout.scm\")\n"
" (install-metadata-path . \"$install_metadata_path\")\n"
" (deployment-action . \"$action\")\n"
" (previous-generation-number . \"$previous_generation\")\n"
" (previous-closure-path . \"$previous_closure\"))\n"
"EOF\n"
" chmod 644 \"$generation_dir/metadata.scm\"\n"
"}\n\n"
"write_generation_provenance()\n"
"{\n"
" generation=$1\n"
" closure=$2\n"
" generation_dir=\"$generations_root/$generation\"\n"
" cat > \"$generation_dir/provenance.scm\" <<EOF\n"
"((closure-path . \"$closure\")\n"
" (parameters-file . \"$closure/parameters.scm\")\n"
" (freebsd-base-file . \"$closure/metadata/freebsd-base.scm\")\n"
" (freebsd-source-file . \"$closure/metadata/freebsd-source.scm\")\n"
" (freebsd-source-materializations-file . \"$closure/metadata/freebsd-source-materializations.scm\")\n"
" (host-base-provenance-file . \"$closure/metadata/host-base-provenance.scm\")\n"
" (store-layout-file . \"$closure/metadata/store-layout.scm\"))\n"
"EOF\n"
" chmod 644 \"$generation_dir/provenance.scm\"\n"
"}\n\n"
"write_generation_install_metadata()\n"
"{\n"
" generation=$1\n"
" closure=$2\n"
" action=$3\n"
" previous_generation=$4\n"
" previous_closure=$5\n"
" generation_dir=\"$generations_root/$generation\"\n"
" cat > \"$generation_dir/install.scm\" <<EOF\n"
"((deployment-kind . \"$action\")\n"
" (generation-number . $generation)\n"
" (closure-path . \"$closure\")\n"
" (previous-generation-number . \"$previous_generation\")\n"
" (previous-closure-path . \"$previous_closure\")\n"
" (freebsd-base-file . \"$closure/metadata/freebsd-base.scm\")\n"
" (freebsd-source-file . \"$closure/metadata/freebsd-source.scm\")\n"
" (freebsd-source-materializations-file . \"$closure/metadata/freebsd-source-materializations.scm\")\n"
" (store-layout-file . \"$closure/metadata/store-layout.scm\"))\n"
"EOF\n"
" chmod 644 \"$generation_dir/install.scm\"\n"
"}\n\n"
"prepare_generation()\n"
"{\n"
" generation=$1\n"
" closure=$2\n"
" action=$3\n"
" previous_generation=$4\n"
" previous_closure=$5\n"
" generation_dir=\"$generations_root/$generation\"\n"
" mkdir -p \"$generation_dir\"\n"
" symlink_force \"$closure\" \"$generation_dir/closure\"\n"
" write_generation_metadata \"$generation\" \"$closure\" \"$action\" \"$previous_generation\" \"$previous_closure\"\n"
" write_generation_provenance \"$generation\" \"$closure\"\n"
" write_generation_install_metadata \"$generation\" \"$closure\" \"$action\" \"$previous_generation\" \"$previous_closure\"\n"
"}\n\n"
"update_efi_loader()\n"
"{\n"
" closure=$1\n"
" [ -e /dev/gpt/efiboot ] || return 0\n"
" esp_mount=$(mktemp -d /tmp/fruix-efiboot.XXXXXX)\n"
" if /sbin/mount -t msdosfs /dev/gpt/efiboot \"$esp_mount\" >/dev/null 2>&1; then\n"
" mkdir -p \"$esp_mount/EFI/BOOT\"\n"
" cp \"$closure/boot/loader.efi\" \"$esp_mount/EFI/BOOT/BOOTX64.EFI\"\n"
" sync\n"
" /sbin/umount \"$esp_mount\" >/dev/null 2>&1 || true\n"
" fi\n"
" rmdir \"$esp_mount\" >/dev/null 2>&1 || true\n"
"}\n\n"
"status()\n"
"{\n"
" current_generation=$(read_file_maybe \"$current_generation_file\")\n"
" current_generation_link=$(read_link_maybe \"$current_link\")\n"
" current_closure=$(read_link_maybe \"$run_current_link\")\n"
" rollback_generation=$(read_file_maybe \"$rollback_generation_file\")\n"
" rollback_generation_link=$(read_link_maybe \"$rollback_link\")\n"
" rollback_closure=\"\"\n"
" if [ -n \"$rollback_generation_link\" ] && [ -L \"$system_root/$rollback_generation_link/closure\" ]; then\n"
" rollback_closure=$(readlink \"$system_root/$rollback_generation_link/closure\")\n"
" fi\n"
" printf 'current_generation=%s\\n' \"$current_generation\"\n"
" printf 'current_link=%s\\n' \"$current_generation_link\"\n"
" printf 'current_closure=%s\\n' \"$current_closure\"\n"
" printf 'rollback_generation=%s\\n' \"$rollback_generation\"\n"
" printf 'rollback_link=%s\\n' \"$rollback_generation_link\"\n"
" printf 'rollback_closure=%s\\n' \"$rollback_closure\"\n"
"}\n\n"
"switch_to_closure()\n"
"{\n"
" target_closure=$1\n"
" validate_closure \"$target_closure\"\n"
" current_generation=$(read_file_maybe \"$current_generation_file\")\n"
" current_closure=$(read_link_maybe \"$run_current_link\")\n"
" [ -n \"$current_generation\" ] || die \"missing current generation metadata\"\n"
" [ -n \"$current_closure\" ] || die \"missing /run/current-system target\"\n"
" if [ \"$target_closure\" = \"$current_closure\" ]; then\n"
" status\n"
" return 0\n"
" fi\n"
" new_generation=$(next_generation_number)\n"
" prepare_generation \"$new_generation\" \"$target_closure\" switch \"$current_generation\" \"$current_closure\"\n"
" symlink_force \"generations/$current_generation\" \"$rollback_link\"\n"
" printf '%s\\n' \"$current_generation\" > \"$rollback_generation_file\"\n"
" symlink_force \"$current_closure\" \"$gcroots_root/rollback-system\"\n"
" symlink_force \"generations/$new_generation\" \"$current_link\"\n"
" printf '%s\\n' \"$new_generation\" > \"$current_generation_file\"\n"
" symlink_force \"$target_closure\" \"$gcroots_root/system-$new_generation\"\n"
" symlink_force \"$target_closure\" \"$gcroots_root/current-system\"\n"
" symlink_force \"$target_closure\" \"$run_current_link\"\n"
" update_efi_loader \"$target_closure\"\n"
" status\n"
"}\n\n"
"rollback_current_generation()\n"
"{\n"
" rollback_generation=$(read_file_maybe \"$rollback_generation_file\")\n"
" rollback_generation_link=$(read_link_maybe \"$rollback_link\")\n"
" [ -n \"$rollback_generation\" ] || die \"no rollback generation is recorded\"\n"
" [ -n \"$rollback_generation_link\" ] || die \"no rollback link is recorded\"\n"
" rollback_closure=$(read_link_maybe \"$system_root/$rollback_generation_link/closure\")\n"
" [ -n \"$rollback_closure\" ] || die \"rollback generation has no closure link\"\n"
" current_generation=$(read_file_maybe \"$current_generation_file\")\n"
" current_closure=$(read_link_maybe \"$run_current_link\")\n"
" [ -n \"$current_generation\" ] || die \"missing current generation metadata\"\n"
" [ -n \"$current_closure\" ] || die \"missing current closure link\"\n"
" symlink_force \"generations/$current_generation\" \"$rollback_link\"\n"
" printf '%s\\n' \"$current_generation\" > \"$rollback_generation_file\"\n"
" symlink_force \"$current_closure\" \"$gcroots_root/rollback-system\"\n"
" symlink_force \"$rollback_generation_link\" \"$current_link\"\n"
" printf '%s\\n' \"$rollback_generation\" > \"$current_generation_file\"\n"
" symlink_force \"$rollback_closure\" \"$gcroots_root/current-system\"\n"
" symlink_force \"$rollback_closure\" \"$run_current_link\"\n"
" update_efi_loader \"$rollback_closure\"\n"
" status\n"
"}\n\n"
"case \"${1:-}\" in\n"
" system)\n"
" case \"${2:-}\" in\n"
" status)\n"
" [ $# -eq 2 ] || { usage >&2; exit 1; }\n"
" status\n"
" ;;\n"
" build)\n"
" shift 2\n"
" system_build \"$@\"\n"
" ;;\n"
" reconfigure)\n"
" shift 2\n"
" reconfigure_system \"$@\"\n"
" ;;\n"
" switch)\n"
" [ $# -eq 3 ] || { usage >&2; exit 1; }\n"
" switch_to_closure \"$3\"\n"
" ;;\n"
" rollback)\n"
" [ $# -eq 2 ] || { usage >&2; exit 1; }\n"
" rollback_current_generation\n"
" ;;\n"
" --help|-h|'')\n"
" usage\n"
" ;;\n"
" *)\n"
" usage >&2\n"
" exit 1\n"
" ;;\n"
" esac\n"
" ;;\n"
" --help|-h|'')\n"
" usage\n"
" ;;\n"
" *)\n"
" usage >&2\n"
" exit 1\n"
" ;;\n"
"esac\n"))
(define (render-development-environment-script os)
(string-append
"#!/bin/sh\n"
"set -eu\n"
"profile=/run/current-system/development-profile\n"
"[ -d \"$profile\" ] || {\n"
" echo \"fruix-development-environment: development profile is not available\" >&2\n"
" exit 1\n"
"}\n"
"cat <<EOF\n"
"export FRUIX_DEVELOPMENT_PROFILE=\"$profile\"\n"
"export FRUIX_DEVELOPMENT_INCLUDE=\"$profile/usr/include\"\n"
"export FRUIX_DEVELOPMENT_LIB=\"$profile/lib\"\n"
"export FRUIX_DEVELOPMENT_SHARE_MK=\"$profile/usr/share/mk\"\n"
"export FRUIX_DEVELOPMENT_BIN=\"$profile/bin\"\n"
"export FRUIX_DEVELOPMENT_USR_BIN=\"$profile/usr/bin\"\n"
"export FRUIX_CC=\"$profile/bin/cc\"\n"
"export FRUIX_CXX=\"$profile/bin/c++\"\n"
"export FRUIX_AR=\"$profile/bin/ar\"\n"
"export FRUIX_RANLIB=\"$profile/bin/ranlib\"\n"
"export FRUIX_NM=\"$profile/bin/nm\"\n"
"export FRUIX_BMAKE=\"/usr/bin/make\"\n"
"export CC=\"$profile/bin/cc\"\n"
"export CXX=\"$profile/bin/c++\"\n"
"export AR=\"$profile/bin/ar\"\n"
"export RANLIB=\"$profile/bin/ranlib\"\n"
"export NM=\"$profile/bin/nm\"\n"
"export CPPFLAGS=\"-I$profile/usr/include\"\n"
"export CFLAGS=\"-I$profile/usr/include\"\n"
"export CXXFLAGS=\"-I$profile/usr/include\"\n"
"export LDFLAGS=\"-L$profile/lib\"\n"
"export MAKEFLAGS=\"-m $profile/usr/share/mk\"\n"
"export PATH=\"/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:$profile/bin:$profile/sbin:$profile/usr/bin:$profile/usr/sbin\"\n"
"EOF\n"))
(define (render-build-environment-script os)
(string-append
"#!/bin/sh\n"
"set -eu\n"
"profile=/run/current-system/build-profile\n"
"[ -d \"$profile\" ] || {\n"
" echo \"fruix-build-environment: build profile is not available\" >&2\n"
" exit 1\n"
"}\n"
"cat <<EOF\n"
"unset MAKEOBJDIRPREFIX MAKEFLAGS CC CXX AR RANLIB NM CPPFLAGS CFLAGS CXXFLAGS LDFLAGS\n"
"unset FRUIX_DEVELOPMENT_PROFILE FRUIX_DEVELOPMENT_INCLUDE FRUIX_DEVELOPMENT_LIB FRUIX_DEVELOPMENT_SHARE_MK\n"
"unset FRUIX_DEVELOPMENT_BIN FRUIX_DEVELOPMENT_USR_BIN FRUIX_CC FRUIX_CXX FRUIX_AR FRUIX_RANLIB FRUIX_NM FRUIX_BMAKE\n"
"export FRUIX_BUILD_PROFILE=\"$profile\"\n"
"export FRUIX_BUILD_INCLUDE=\"$profile/usr/include\"\n"
"export FRUIX_BUILD_LIB=\"$profile/lib\"\n"
"export FRUIX_BUILD_SHARE_MK=\"$profile/usr/share/mk\"\n"
"export FRUIX_BUILD_BIN=\"$profile/bin\"\n"
"export FRUIX_BUILD_USR_BIN=\"$profile/usr/bin\"\n"
"export FRUIX_BUILD_CC=\"$profile/bin/cc\"\n"
"export FRUIX_BUILD_CXX=\"$profile/bin/c++\"\n"
"export FRUIX_BUILD_AR=\"$profile/bin/ar\"\n"
"export FRUIX_BUILD_RANLIB=\"$profile/bin/ranlib\"\n"
"export FRUIX_BUILD_NM=\"$profile/bin/nm\"\n"
"export FRUIX_BMAKE=\"make\"\n"
"export PATH=\"/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:$profile/bin:$profile/sbin:$profile/usr/bin:$profile/usr/sbin\"\n"
"EOF\n"))
(define (render-self-hosted-native-build-script os)
(let* ((base-spec (freebsd-base-spec (operating-system-freebsd-base os)))
(base-name (assoc-ref base-spec 'name))
(version-label (assoc-ref base-spec 'version-label))
(release (assoc-ref base-spec 'release))
(branch (assoc-ref base-spec 'branch))
(declared-source-root (assoc-ref base-spec 'source-root))
(target (assoc-ref base-spec 'target))
(target-arch (assoc-ref base-spec 'target-arch))
(kernconf (assoc-ref base-spec 'kernconf))
(make-flags (or (assoc-ref base-spec 'make-flags) '()))
(build-common (string-join
(append (list (format #f "TARGET=~a" target)
(format #f "TARGET_ARCH=~a" target-arch)
(format #f "KERNCONF=~a" kernconf))
make-flags)
" "))
(install-common (string-append build-common " DB_FROM_SRC=yes")))
(string-append
"#!/bin/sh\n"
"set -eu\n"
"umask 022\n"
"profile=/run/current-system/build-profile\n"
"guest_host_name='" (operating-system-host-name os) "'\n"
"[ -d \"$profile\" ] || {\n"
" echo \"fruix-self-hosted-native-build: build profile is not available\" >&2\n"
" exit 1\n"
"}\n"
"[ -x /usr/local/bin/fruix-build-environment ] || {\n"
" echo \"fruix-self-hosted-native-build: build environment helper is missing\" >&2\n"
" exit 1\n"
"}\n"
"eval \"$(/usr/local/bin/fruix-build-environment)\"\n"
"[ \"${FRUIX_BUILD_PROFILE:-}\" = \"$profile\" ] || {\n"
" echo \"fruix-self-hosted-native-build: build environment helper exported an unexpected profile\" >&2\n"
" exit 1\n"
"}\n"
"[ -L /usr/include ] || {\n"
" echo \"fruix-self-hosted-native-build: /usr/include compatibility link is missing\" >&2\n"
" exit 1\n"
"}\n"
"[ \"$(readlink /usr/include)\" = \"/run/current-system/build-profile/usr/include\" ] || {\n"
" echo \"fruix-self-hosted-native-build: /usr/include points at the wrong target\" >&2\n"
" exit 1\n"
"}\n"
"[ -L /usr/share/mk ] || {\n"
" echo \"fruix-self-hosted-native-build: /usr/share/mk compatibility link is missing\" >&2\n"
" exit 1\n"
"}\n"
"[ \"$(readlink /usr/share/mk)\" = \"/run/current-system/build-profile/usr/share/mk\" ] || {\n"
" echo \"fruix-self-hosted-native-build: /usr/share/mk points at the wrong target\" >&2\n"
" exit 1\n"
"}\n"
"make_cmd=${FRUIX_BMAKE:-make}\n"
"jobs=${FRUIX_SELF_HOSTED_NATIVE_BUILD_JOBS:-$(sysctl -n hw.ncpu)}\n"
"case \"$jobs\" in\n"
" ''|*[!0-9]*)\n"
" echo \"fruix-self-hosted-native-build: invalid job count: $jobs\" >&2\n"
" exit 1\n"
" ;;\n"
"esac\n"
"run_id=${FRUIX_SELF_HOSTED_NATIVE_BUILD_ID:-$(date -u +%Y%m%dT%H%M%SZ)}\n"
"build_root_base=${FRUIX_SELF_HOSTED_NATIVE_BUILD_ROOT_BASE:-/var/tmp/fruix-self-hosted-native-builds}\n"
"result_root_base=${FRUIX_SELF_HOSTED_NATIVE_BUILD_OUTPUT_BASE:-/var/lib/fruix/native-builds}\n"
"build_root=$build_root_base/$run_id\n"
"result_root=$result_root_base/$run_id\n"
"logdir=$result_root/logs\n"
"status_file=$result_root/status\n"
"metadata_file=$result_root/metadata.txt\n"
"promotion_file=$result_root/promotion.scm\n"
"world_stage=$build_root/stage-world\n"
"kernel_stage=$build_root/stage-kernel\n"
"world_artifact=$result_root/artifacts/world\n"
"headers_artifact=$result_root/artifacts/headers\n"
"kernel_artifact=$result_root/artifacts/kernel\n"
"bootloader_artifact=$result_root/artifacts/bootloader\n"
"latest_link=$result_root_base/latest\n"
"mkdir -p \"$build_root\" \"$result_root\" \"$logdir\"\n"
"printf 'running\\n' > \"$status_file\"\n"
"fail_mark() {\n"
" rc=$?\n"
" if [ \"$rc\" -ne 0 ]; then\n"
" printf 'failed\\n' > \"$status_file\"\n"
" fi\n"
"}\n"
"trap fail_mark EXIT HUP INT TERM\n"
"closure=$(readlink /run/current-system)\n"
"store_layout=$closure/metadata/store-layout.scm\n"
"[ -f \"$store_layout\" ] || {\n"
" echo \"fruix-self-hosted-native-build: store layout metadata is missing\" >&2\n"
" exit 1\n"
"}\n"
"source_store=$(sed -n 's/.*\"\\(\\/frx\\/store\\/[^\"]*-freebsd-source-[^\"]*\\)\".*/\\1/p' \"$store_layout\" | head -n 1)\n"
"[ -n \"$source_store\" ] || {\n"
" echo \"fruix-self-hosted-native-build: failed to recover source store from store-layout.scm\" >&2\n"
" exit 1\n"
"}\n"
"source_root=$source_store/tree\n"
"[ -d \"$source_root\" ] || {\n"
" echo \"fruix-self-hosted-native-build: source root is missing: $source_root\" >&2\n"
" exit 1\n"
"}\n"
"mkdir -p \"$world_artifact\" \"$headers_artifact/usr\" \"$kernel_artifact/boot\" \"$bootloader_artifact/boot\"\n"
"export MAKEOBJDIRPREFIX=\"$build_root/obj\"\n"
"\"$make_cmd\" -j\"$jobs\" -C \"$source_root\" " build-common " buildworld > \"$logdir/buildworld.log\" 2>&1\n"
"\"$make_cmd\" -j\"$jobs\" -C \"$source_root\" " build-common " buildkernel > \"$logdir/buildkernel.log\" 2>&1\n"
"\"$make_cmd\" -C \"$source_root\" " install-common " DESTDIR=\"$world_stage\" installworld > \"$logdir/installworld.log\" 2>&1\n"
"\"$make_cmd\" -C \"$source_root\" " install-common " DESTDIR=\"$world_stage\" distribution > \"$logdir/distribution.log\" 2>&1\n"
"\"$make_cmd\" -C \"$source_root\" " install-common " DESTDIR=\"$kernel_stage\" installkernel > \"$logdir/installkernel.log\" 2>&1\n"
"cp -a \"$world_stage/.\" \"$world_artifact/\"\n"
"cp -a \"$kernel_stage/boot/kernel\" \"$kernel_artifact/boot/kernel\"\n"
"cp -a \"$world_stage/usr/include\" \"$headers_artifact/usr/include\"\n"
"mkdir -p \"$headers_artifact/usr/share\"\n"
"cp -a \"$world_stage/usr/share/mk\" \"$headers_artifact/usr/share/mk\"\n"
"cp -a \"$world_stage/boot/loader\" \"$bootloader_artifact/boot/loader\"\n"
"cp -a \"$world_stage/boot/loader.efi\" \"$bootloader_artifact/boot/loader.efi\"\n"
"cp -a \"$world_stage/boot/device.hints\" \"$bootloader_artifact/boot/device.hints\"\n"
"cp -a \"$world_stage/boot/defaults\" \"$bootloader_artifact/boot/defaults\"\n"
"cp -a \"$world_stage/boot/lua\" \"$bootloader_artifact/boot/lua\"\n"
"[ -f \"$world_artifact/bin/sh\" ]\n"
"[ -f \"$kernel_artifact/boot/kernel/kernel\" ]\n"
"[ -f \"$headers_artifact/usr/include/sys/param.h\" ]\n"
"[ -f \"$headers_artifact/usr/share/mk/bsd.prog.mk\" ]\n"
"[ -f \"$bootloader_artifact/boot/loader.efi\" ]\n"
"[ -f \"$bootloader_artifact/boot/defaults/loader.conf\" ]\n"
"[ -f \"$bootloader_artifact/boot/lua/loader.lua\" ]\n"
"sha_kernel=$(sha256 -q \"$kernel_artifact/boot/kernel/kernel\")\n"
"sha_loader=$(sha256 -q \"$bootloader_artifact/boot/loader.efi\")\n"
"sha_param=$(sha256 -q \"$headers_artifact/usr/include/sys/param.h\")\n"
"buildworld_tail=$(tail -n 20 \"$logdir/buildworld.log\" | tr '\\n' ' ')\n"
"buildkernel_tail=$(tail -n 20 \"$logdir/buildkernel.log\" | tr '\\n' ' ')\n"
"installworld_tail=$(tail -n 20 \"$logdir/installworld.log\" | tr '\\n' ' ')\n"
"distribution_tail=$(tail -n 20 \"$logdir/distribution.log\" | tr '\\n' ' ')\n"
"installkernel_tail=$(tail -n 20 \"$logdir/installkernel.log\" | tr '\\n' ' ')\n"
"root_df=$(df -h / | tail -n 1 | tr -s ' ' | tr '\\t' ' ')\n"
"build_root_size=$(du -sh \"$build_root\" | awk '{print $1}')\n"
"result_root_size=$(du -sh \"$result_root\" | awk '{print $1}')\n"
"world_artifact_size=$(du -sh \"$world_artifact\" | awk '{print $1}')\n"
"kernel_artifact_size=$(du -sh \"$kernel_artifact\" | awk '{print $1}')\n"
"headers_artifact_size=$(du -sh \"$headers_artifact\" | awk '{print $1}')\n"
"bootloader_artifact_size=$(du -sh \"$bootloader_artifact\" | awk '{print $1}')\n"
"rm -f \"$latest_link\"\n"
"ln -s \"$result_root\" \"$latest_link\"\n"
"cat >\"$promotion_file\" <<EOF\n"
"((native-build-result-version . \"1\")\n"
" (executor . ((kind . self-hosted)\n"
" (name . \"guest-self-hosted\")\n"
" (version . \"5\")\n"
" (properties . ((helper-path . \"/usr/local/bin/fruix-self-hosted-native-build\")\n"
" (guest-host-name . \"$guest_host_name\")\n"
" (build-root-base . \"$build_root_base\")\n"
" (result-root-base . \"$result_root_base\")))))\n"
" (run-id . \"$run_id\")\n"
" (guest-host-name . \"$guest_host_name\")\n"
" (closure-path . \"$closure\")\n"
" (build-profile . \"$profile\")\n"
" (freebsd-base . ((name . \"" base-name "\")\n"
" (version-label . \"" version-label "\")\n"
" (release . \"" release "\")\n"
" (branch . \"" branch "\")\n"
" (source-root . \"" declared-source-root "\")\n"
" (target . \"" target "\")\n"
" (target-arch . \"" target-arch "\")\n"
" (kernconf . \"" kernconf "\")))\n"
" (source . ((store-path . \"$source_store\")\n"
" (source-root . \"$source_root\")))\n"
" (build-policy . ((jobs . \"$jobs\")\n"
" (build-common . \"" build-common "\")\n"
" (install-common . \"" install-common "\")))\n"
" (artifacts . ((world . ((path . \"artifacts/world\")\n"
" (required-file . \"bin/sh\")))\n"
" (kernel . ((path . \"artifacts/kernel\")\n"
" (required-file . \"boot/kernel/kernel\")\n"
" (recorded-sha256 . \"$sha_kernel\")))\n"
" (headers . ((path . \"artifacts/headers\")\n"
" (required-file . \"usr/include/sys/param.h\")\n"
" (recorded-sha256 . \"$sha_param\")))\n"
" (bootloader . ((path . \"artifacts/bootloader\")\n"
" (required-file . \"boot/loader.efi\")\n"
" (recorded-sha256 . \"$sha_loader\"))))))\n"
"EOF\n"
"cat >\"$metadata_file\" <<EOF\n"
"run_id=$run_id\n"
"helper_version=5\n"
"executor_kind=self-hosted\n"
"executor_name=guest-self-hosted\n"
"executor_version=5\n"
"closure_path=$closure\n"
"guest_host_name=$guest_host_name\n"
"build_profile=$profile\n"
"source_store=$source_store\n"
"source_root=$source_root\n"
"build_jobs=$jobs\n"
"build_common=" build-common "\n"
"install_common=" install-common "\n"
"build_root=$build_root\n"
"result_root=$result_root\n"
"logdir=$logdir\n"
"status_file=$status_file\n"
"metadata_file=$metadata_file\n"
"promotion_file=$promotion_file\n"
"world_stage=$world_stage\n"
"kernel_stage=$kernel_stage\n"
"world_artifact=$world_artifact\n"
"kernel_artifact=$kernel_artifact\n"
"headers_artifact=$headers_artifact\n"
"bootloader_artifact=$bootloader_artifact\n"
"latest_link=$latest_link\n"
"root_df=$root_df\n"
"build_root_size=$build_root_size\n"
"result_root_size=$result_root_size\n"
"world_artifact_size=$world_artifact_size\n"
"kernel_artifact_size=$kernel_artifact_size\n"
"headers_artifact_size=$headers_artifact_size\n"
"bootloader_artifact_size=$bootloader_artifact_size\n"
"sha_kernel=$sha_kernel\n"
"sha_loader=$sha_loader\n"
"sha_param=$sha_param\n"
"buildworld_tail=$buildworld_tail\n"
"buildkernel_tail=$buildkernel_tail\n"
"installworld_tail=$installworld_tail\n"
"distribution_tail=$distribution_tail\n"
"installkernel_tail=$installkernel_tail\n"
"self_hosted_native_build=ok\n"
"EOF\n"
"printf 'ok\\n' > \"$status_file\"\n"
"cat \"$metadata_file\"\n")))
(define* (operating-system-generated-files os #:key guile-store guile-extra-store shepherd-store) (define* (operating-system-generated-files os #:key guile-store guile-extra-store shepherd-store)
(append (append
@@ -486,7 +1207,20 @@
#:guile-store guile-store #:guile-store guile-store
#:guile-extra-store guile-extra-store #:guile-extra-store guile-extra-store
#:shepherd-store shepherd-store)) #:shepherd-store shepherd-store))
("shepherd/init.scm" . ,(render-shepherd-config os))) ("shepherd/init.scm" . ,(render-shepherd-config os))
("usr/local/bin/fruix"
. ,(render-installed-system-fruix os guile-store guile-extra-store shepherd-store)))
(bundled-fruix-node-files)
(if (null? (operating-system-development-packages os))
'()
`(("usr/local/bin/fruix-development-environment"
. ,(render-development-environment-script os))))
(if (null? (operating-system-build-packages os))
'()
`(("usr/local/bin/fruix-build-environment"
. ,(render-build-environment-script os))
("usr/local/bin/fruix-self-hosted-native-build"
. ,(render-self-hosted-native-build-script os))))
(if (pid1-init-mode? os) (if (pid1-init-mode? os)
`(("boot/fruix-pid1" . ,(render-pid1-script os shepherd-store guile-store guile-extra-store))) `(("boot/fruix-pid1" . ,(render-pid1-script os shepherd-store guile-store guile-extra-store)))
'()) '())

7
modules/fruix/system/freebsd/source.scm
View File

@@ -150,9 +150,10 @@
(effective-source (assoc-ref resolution 'effective-source)) (effective-source (assoc-ref resolution 'effective-source))
(identity (assoc-ref resolution 'identity)) (identity (assoc-ref resolution 'identity))
(manifest (freebsd-source-manifest source effective-source identity)) (manifest (freebsd-source-manifest source effective-source identity))
(hash (sha256-string manifest)) (display-name (string-append "freebsd-source-"
(output-path (string-append store-dir "/" hash "-freebsd-source-" (safe-name-fragment (freebsd-source-name source))))
(safe-name-fragment (freebsd-source-name source)))) (output-path (make-store-path store-dir display-name manifest
#:kind 'freebsd-source))
(info-file (string-append output-path "/.freebsd-source-info.scm")) (info-file (string-append output-path "/.freebsd-source-info.scm"))
(cache-path (assoc-ref resolution 'cache-path)) (cache-path (assoc-ref resolution 'cache-path))
(populate-tree (assoc-ref resolution 'populate-tree))) (populate-tree (assoc-ref resolution 'populate-tree)))

61
modules/fruix/system/freebsd/utils.scm
View File

@@ -14,9 +14,12 @@
safe-command-output safe-command-output
write-file write-file
sha256-string sha256-string
store-hash-string
make-store-path
file-hash file-hash
directory-entries directory-entries
path-signature path-signature
tree-content-signature
install-plan-signature install-plan-signature
native-build-source-tree-sha256 native-build-source-tree-sha256
copy-regular-file copy-regular-file
@@ -68,6 +71,40 @@
(write-file tmp text) (write-file tmp text)
(command-output "sha256" "-q" tmp))) (command-output "sha256" "-q" tmp)))
(define store-hash-visible-length 40)
(define store-hash-scheme-version "1")
(define (store-identity-field value)
(cond ((symbol? value)
(symbol->string value))
((string? value)
value)
(else
(object->string value))))
(define* (store-hash-string payload #:key (kind 'item) name (output "out"))
(let* ((identity `((scheme . "fruix-store-path")
(version . ,store-hash-scheme-version)
(kind . ,(store-identity-field kind))
(name . ,(store-identity-field (or name "")))
(output . ,(store-identity-field output))
(payload . ,payload)))
(digest (sha256-string (object->string identity))))
(string-take digest store-hash-visible-length)))
(define* (make-store-path store-dir display-name payload
#:key
(kind 'item)
name
(output "out"))
(string-append store-dir "/"
(store-hash-string payload
#:kind kind
#:name (or name display-name)
#:output output)
"-"
display-name))
(define (file-hash path) (define (file-hash path)
(command-output "sha256" "-q" path)) (command-output "sha256" "-q" path))
@@ -96,6 +133,30 @@
(else (else
(string-append "other:" path ":" (symbol->string (stat:type st))))))) (string-append "other:" path ":" (symbol->string (stat:type st)))))))
(define (tree-content-signature root)
(define (walk path relative)
(let ((st (lstat path)))
(case (stat:type st)
((regular)
(string-append "file:" relative ":" (file-hash path)))
((symlink)
(string-append "symlink:" relative ":" (readlink path)))
((directory)
(string-join
(cons (string-append "directory:" relative)
(apply append
(map (lambda (entry)
(let ((child-relative (if (string=? relative ".")
entry
(string-append relative "/" entry))))
(list (walk (string-append path "/" entry)
child-relative))))
(directory-entries path))))
"\n"))
(else
(string-append "other:" relative ":" (symbol->string (stat:type st)))))))
(walk root "."))
(define (install-plan-signature entry) (define (install-plan-signature entry)
(match entry (match entry
(('file source target) (('file source target)

97
scripts/fruix.scm
View File

@@ -6,7 +6,8 @@
(ice-9 format) (ice-9 format)
(ice-9 match) (ice-9 match)
(srfi srfi-1) (srfi srfi-1)
(srfi srfi-13)) (srfi srfi-13)
(rnrs io ports))
(define (usage code) (define (usage code)
(format (if (= code 0) #t (current-error-port)) (format (if (= code 0) #t (current-error-port))
@@ -15,6 +16,7 @@
Commands:\n\ Commands:\n\
system ACTION ... Build or materialize Fruix system artifacts.\n\ system ACTION ... Build or materialize Fruix system artifacts.\n\
source ACTION ... Fetch or snapshot declarative FreeBSD source inputs.\n\ source ACTION ... Fetch or snapshot declarative FreeBSD source inputs.\n\
native-build ACTION ... Promote native build results into Fruix store objects.\n\
\n\ \n\
System actions:\n\ System actions:\n\
build Materialize the Fruix system closure in /frx/store.\n\ build Materialize the Fruix system closure in /frx/store.\n\
@@ -37,6 +39,12 @@ System options:\n\
Source actions:\n\ Source actions:\n\
materialize Materialize a declared FreeBSD source tree in /frx/store.\n\ materialize Materialize a declared FreeBSD source tree in /frx/store.\n\
\n\ \n\
Native-build actions:\n\
promote Promote a native build result root into /frx/store.\n\
\n\
Native-build options:\n\
--store DIR Store directory to use (default: /frx/store).\n\
\n\
Source options:\n\ Source options:\n\
--source NAME Scheme variable holding the freebsd-source object.\n\ --source NAME Scheme variable holding the freebsd-source object.\n\
--store DIR Store directory to use (default: /frx/store).\n\ --store DIR Store directory to use (default: /frx/store).\n\
@@ -62,6 +70,9 @@ Common options:\n\
(format #t "~a=~a~%" (car field) (stringify (cdr field)))) (format #t "~a=~a~%" (car field) (stringify (cdr field))))
fields)) fields))
(define (read-file-string file)
(call-with-input-file file get-string-all))
(define (lookup-bound-value module symbol) (define (lookup-bound-value module symbol)
(let ((var (module-variable module symbol))) (let ((var (module-variable module symbol)))
(and var (variable-ref var)))) (and var (variable-ref var))))
@@ -216,6 +227,28 @@ Common options:\n\
((arg . tail) ((arg . tail)
(loop tail (cons arg positional) source-name store-dir cache-dir))))) (loop tail (cons arg positional) source-name store-dir cache-dir)))))
(define (parse-native-build-arguments action rest)
(let loop ((args rest)
(positional '())
(store-dir "/frx/store"))
(match args
(()
(let ((positional (reverse positional)))
`((command . "native-build")
(action . ,action)
(positional . ,positional)
(store-dir . ,store-dir))))
(("--help")
(usage 0))
(((? (lambda (arg) (string-prefix? "--store=" arg)) arg) . tail)
(loop tail positional (option-value arg "--store=")))
(("--store" value . tail)
(loop tail positional value))
(((? (lambda (arg) (string-prefix? "--" arg)) arg) . _)
(error "unknown option" arg))
((arg . tail)
(loop tail (cons arg positional) store-dir)))))
(define (parse-arguments argv) (define (parse-arguments argv)
(match argv (match argv
((_) ((_)
@@ -228,10 +261,14 @@ Common options:\n\
(usage 0)) (usage 0))
((_ "source" "--help") ((_ "source" "--help")
(usage 0)) (usage 0))
((_ "native-build" "--help")
(usage 0))
((_ "system" action . rest) ((_ "system" action . rest)
(parse-system-arguments action rest)) (parse-system-arguments action rest))
((_ "source" action . rest) ((_ "source" action . rest)
(parse-source-arguments action rest)) (parse-source-arguments action rest))
((_ "native-build" action . rest)
(parse-native-build-arguments action rest))
((_ . _) ((_ . _)
(usage 1)))) (usage 1))))
@@ -542,6 +579,23 @@ Common options:\n\
(target_store_item_count . ,(length target-store-items)) (target_store_item_count . ,(length target-store-items))
(installer_store_item_count . ,(length installer-store-items)))))) (installer_store_item_count . ,(length installer-store-items))))))
(define (emit-native-build-promotion-metadata store-dir result-root result)
(emit-metadata
`((action . "promote")
(result_root . ,result-root)
(store_dir . ,store-dir)
(executor_kind . ,(assoc-ref result 'executor-kind))
(executor_name . ,(assoc-ref result 'executor-name))
(executor_version . ,(assoc-ref result 'executor-version))
(result_store . ,(assoc-ref result 'result-store))
(result_metadata_file . ,(assoc-ref result 'result-metadata-file))
(artifact_store_count . ,(assoc-ref result 'artifact-store-count))
(artifact_stores . ,(string-join (assoc-ref result 'artifact-stores) ","))
(world_store . ,(assoc-ref result 'world-store))
(kernel_store . ,(assoc-ref result 'kernel-store))
(headers_store . ,(assoc-ref result 'headers-store))
(bootloader_store . ,(assoc-ref result 'bootloader-store)))))
(define (main argv) (define (main argv)
(let* ((parsed (parse-arguments argv)) (let* ((parsed (parse-arguments argv))
(command (assoc-ref parsed 'command)) (command (assoc-ref parsed 'command))
@@ -579,7 +633,11 @@ Common options:\n\
(lambda (os resolved-symbol) (lambda (os resolved-symbol)
(let* ((guile-prefix (or (getenv "GUILE_PREFIX") "/tmp/guile-freebsd-validate-install")) (let* ((guile-prefix (or (getenv "GUILE_PREFIX") "/tmp/guile-freebsd-validate-install"))
(guile-extra-prefix (or (getenv "GUILE_EXTRA_PREFIX") "/tmp/guile-gnutls-freebsd-validate-install")) (guile-extra-prefix (or (getenv "GUILE_EXTRA_PREFIX") "/tmp/guile-gnutls-freebsd-validate-install"))
(shepherd-prefix (or (getenv "SHEPHERD_PREFIX") "/tmp/shepherd-freebsd-validate-install"))) (shepherd-prefix (or (getenv "SHEPHERD_PREFIX") "/tmp/shepherd-freebsd-validate-install"))
(guile-store-path (getenv "FRUIX_GUILE_STORE"))
(guile-extra-store-path (getenv "FRUIX_GUILE_EXTRA_STORE"))
(shepherd-store-path (getenv "FRUIX_SHEPHERD_STORE"))
(declaration-source (read-file-string os-file)))
(cond (cond
((string=? action "build") ((string=? action "build")
(emit-system-build-metadata (emit-system-build-metadata
@@ -588,7 +646,13 @@ Common options:\n\
#:store-dir store-dir #:store-dir store-dir
#:guile-prefix guile-prefix #:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix #:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix))) #:shepherd-prefix shepherd-prefix
#:guile-store-path guile-store-path
#:guile-extra-store-path guile-extra-store-path
#:shepherd-store-path shepherd-store-path
#:declaration-source declaration-source
#:declaration-origin os-file
#:declaration-system-symbol resolved-symbol)))
((string=? action "rootfs") ((string=? action "rootfs")
(unless rootfs (unless rootfs
(error "rootfs action requires ROOTFS-DIR or --rootfs DIR")) (error "rootfs action requires ROOTFS-DIR or --rootfs DIR"))
@@ -596,7 +660,10 @@ Common options:\n\
#:store-dir store-dir #:store-dir store-dir
#:guile-prefix guile-prefix #:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix #:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix))) #:shepherd-prefix shepherd-prefix
#:declaration-source declaration-source
#:declaration-origin os-file
#:declaration-system-symbol resolved-symbol)))
(emit-metadata (emit-metadata
`((action . "rootfs") `((action . "rootfs")
(os_file . ,os-file) (os_file . ,os-file)
@@ -614,6 +681,9 @@ Common options:\n\
#:guile-prefix guile-prefix #:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix #:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix #:shepherd-prefix shepherd-prefix
#:declaration-source declaration-source
#:declaration-origin os-file
#:declaration-system-symbol resolved-symbol
#:root-size (or root-size "256m") #:root-size (or root-size "256m")
#:disk-capacity disk-capacity))) #:disk-capacity disk-capacity)))
((string=? action "installer") ((string=? action "installer")
@@ -624,6 +694,9 @@ Common options:\n\
#:guile-prefix guile-prefix #:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix #:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix #:shepherd-prefix shepherd-prefix
#:declaration-source declaration-source
#:declaration-origin os-file
#:declaration-system-symbol resolved-symbol
#:install-target-device (or install-target-device "/dev/vtbd1") #:install-target-device (or install-target-device "/dev/vtbd1")
#:root-size (or root-size "10g") #:root-size (or root-size "10g")
#:disk-capacity disk-capacity))) #:disk-capacity disk-capacity)))
@@ -635,6 +708,9 @@ Common options:\n\
#:guile-prefix guile-prefix #:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix #:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix #:shepherd-prefix shepherd-prefix
#:declaration-source declaration-source
#:declaration-origin os-file
#:declaration-system-symbol resolved-symbol
#:install-target-device (or install-target-device "/dev/vtbd0") #:install-target-device (or install-target-device "/dev/vtbd0")
#:root-size root-size))) #:root-size root-size)))
((string=? action "install") ((string=? action "install")
@@ -648,6 +724,9 @@ Common options:\n\
#:guile-prefix guile-prefix #:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix #:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix #:shepherd-prefix shepherd-prefix
#:declaration-source declaration-source
#:declaration-origin os-file
#:declaration-system-symbol resolved-symbol
#:root-size root-size #:root-size root-size
#:disk-capacity disk-capacity)))))))))) #:disk-capacity disk-capacity))))))))))
((string=? command "source") ((string=? command "source")
@@ -692,6 +771,16 @@ Common options:\n\
(materialized_source_ref . ,(or (assoc-ref effective 'ref) "")) (materialized_source_ref . ,(or (assoc-ref effective 'ref) ""))
(materialized_source_commit . ,(or (assoc-ref result 'effective-commit) "")) (materialized_source_commit . ,(or (assoc-ref result 'effective-commit) ""))
(materialized_source_sha256 . ,(or (assoc-ref result 'effective-sha256) "")))))))))) (materialized_source_sha256 . ,(or (assoc-ref result 'effective-sha256) ""))))))))))
((string=? command "native-build")
(let ((positional (assoc-ref parsed 'positional)))
(unless (string=? action "promote")
(error "unknown native-build action" action))
(let ((result-root (match positional
((path . _) path)
(() (error "missing native build result root argument")))))
(emit-native-build-promotion-metadata
store-dir result-root
(promote-native-build-result result-root #:store-dir store-dir)))))
(else (else
(usage 1))))) (usage 1)))))

91
tests/system/phase19-generation-rollback-operating-system.scm.in Normal file
View File

@@ -0,0 +1,91 @@
(use-modules (fruix system freebsd)
(fruix packages freebsd))
(define phase19-source
(freebsd-source
#:name "__SOURCE_NAME__"
#:kind 'git
#:ref "__SOURCE_REF__"
#:commit "__SOURCE_COMMIT__"))
(define phase19-base
(freebsd-base
#:name "__BASE_NAME__"
#:version-label "__BASE_VERSION_LABEL__"
#:release "__BASE_RELEASE__"
#:branch "__BASE_BRANCH__"
#:source phase19-source
#:source-root "__DECLARED_SOURCE_ROOT__"
#:target "amd64"
#:target-arch "amd64"
#:kernconf "GENERIC"))
(define phase19-operating-system
(operating-system
#:host-name "__HOST_NAME__"
#:freebsd-base phase19-base
#:kernel (freebsd-native-kernel-for phase19-base)
#:bootloader (freebsd-native-bootloader-for phase19-base)
#:base-packages (freebsd-native-system-packages-for phase19-base)
#:groups (list (user-group #:name "wheel" #:gid 0 #:system? #t)
(user-group #:name "sshd" #:gid 22 #:system? #t)
(user-group #:name "_dhcp" #:gid 65 #:system? #t)
(user-group #:name "operator" #:gid 1000 #:system? #f))
#:users (list (user-account #:name "root"
#:uid 0
#:group "wheel"
#:comment "Charlie &"
#:home "/root"
#:shell "/bin/sh"
#:system? #t)
(user-account #:name "sshd"
#:uid 22
#:group "sshd"
#:comment "Secure Shell Daemon"
#:home "/var/empty"
#:shell "/usr/sbin/nologin"
#:system? #t)
(user-account #:name "_dhcp"
#:uid 65
#:group "_dhcp"
#:comment "dhcp programs"
#:home "/var/empty"
#:shell "/usr/sbin/nologin"
#:system? #t)
(user-account #:name "operator"
#:uid 1000
#:group "operator"
#:supplementary-groups '("wheel")
#:comment "Fruix Operator"
#:home "/home/operator"
#:shell "/bin/sh"
#:system? #f))
#:file-systems (list (file-system #:device "/dev/gpt/fruix-root"
#:mount-point "/"
#:type "ufs"
#:options "rw"
#:needed-for-boot? #t)
(file-system #:device "devfs"
#:mount-point "/dev"
#:type "devfs"
#:options "rw"
#:needed-for-boot? #t)
(file-system #:device "tmpfs"
#:mount-point "/tmp"
#:type "tmpfs"
#:options "rw,size=64m"))
#:services '(shepherd ready-marker sshd)
#:loader-entries '(("autoboot_delay" . "1")
("boot_multicons" . "YES")
("boot_serial" . "YES")
("console" . "comconsole,vidconsole"))
#:rc-conf-entries '(("clear_tmp_enable" . "NO")
("hostid_enable" . "NO")
("sendmail_enable" . "NONE")
("sshd_enable" . "YES")
("ifconfig_xn0" . "SYNCDHCP")
("ifconfig_em0" . "SYNCDHCP")
("ifconfig_vtnet0" . "SYNCDHCP"))
#:init-mode 'freebsd-init+rc.d-shepherd
#:ready-marker "/var/lib/fruix/ready"
#:root-authorized-keys '("__ROOT_AUTHORIZED_KEY__")))

75
tests/system/phase20-development-operating-system.scm.in Normal file
View File

@@ -0,0 +1,75 @@
(use-modules (fruix system freebsd)
(fruix packages freebsd))
(define phase20-operating-system
(operating-system
#:host-name "fruix-freebsd"
#:kernel freebsd-native-kernel
#:bootloader freebsd-native-bootloader
#:base-packages %freebsd-native-system-packages
#:development-packages (list freebsd-native-headers
freebsd-clang-toolchain)
#:build-packages (list freebsd-native-headers
freebsd-clang-toolchain)
#:groups (list (user-group #:name "wheel" #:gid 0 #:system? #t)
(user-group #:name "sshd" #:gid 22 #:system? #t)
(user-group #:name "_dhcp" #:gid 65 #:system? #t)
(user-group #:name "operator" #:gid 1000 #:system? #f))
#:users (list (user-account #:name "root"
#:uid 0
#:group "wheel"
#:comment "Charlie &"
#:home "/root"
#:shell "/bin/sh"
#:system? #t)
(user-account #:name "sshd"
#:uid 22
#:group "sshd"
#:comment "Secure Shell Daemon"
#:home "/var/empty"
#:shell "/usr/sbin/nologin"
#:system? #t)
(user-account #:name "_dhcp"
#:uid 65
#:group "_dhcp"
#:comment "dhcp programs"
#:home "/var/empty"
#:shell "/usr/sbin/nologin"
#:system? #t)
(user-account #:name "operator"
#:uid 1000
#:group "operator"
#:supplementary-groups '("wheel")
#:comment "Fruix Operator"
#:home "/home/operator"
#:shell "/bin/sh"
#:system? #f))
#:file-systems (list (file-system #:device "/dev/gpt/fruix-root"
#:mount-point "/"
#:type "ufs"
#:options "rw"
#:needed-for-boot? #t)
(file-system #:device "devfs"
#:mount-point "/dev"
#:type "devfs"
#:options "rw"
#:needed-for-boot? #t)
(file-system #:device "tmpfs"
#:mount-point "/tmp"
#:type "tmpfs"
#:options "rw,size=64m"))
#:services '(shepherd ready-marker sshd)
#:loader-entries '(("autoboot_delay" . "1")
("boot_multicons" . "YES")
("boot_serial" . "YES")
("console" . "comconsole,vidconsole"))
#:rc-conf-entries '(("clear_tmp_enable" . "NO")
("hostid_enable" . "NO")
("sendmail_enable" . "NONE")
("sshd_enable" . "YES")
("ifconfig_xn0" . "SYNCDHCP")
("ifconfig_em0" . "SYNCDHCP")
("ifconfig_vtnet0" . "SYNCDHCP"))
#:init-mode 'shepherd-pid1
#:ready-marker "/var/lib/fruix/ready"
#:root-authorized-keys '("__ROOT_AUTHORIZED_KEY__")))

73
tests/system/phase20-promoted-native-base-operating-system.scm.in Normal file
View File

@@ -0,0 +1,73 @@
(use-modules (fruix system freebsd)
(fruix packages freebsd))
(define phase20-promoted-native-build-result
(promoted-native-build-result
#:store-path "__PROMOTED_RESULT_STORE__"))
(define phase20-promoted-native-base-operating-system
(operating-system-from-promoted-native-build-result
phase20-promoted-native-build-result
#:host-name "fruix-freebsd"
#:groups (list (user-group #:name "wheel" #:gid 0 #:system? #t)
(user-group #:name "sshd" #:gid 22 #:system? #t)
(user-group #:name "_dhcp" #:gid 65 #:system? #t)
(user-group #:name "operator" #:gid 1000 #:system? #f))
#:users (list (user-account #:name "root"
#:uid 0
#:group "wheel"
#:comment "Charlie &"
#:home "/root"
#:shell "/bin/sh"
#:system? #t)
(user-account #:name "sshd"
#:uid 22
#:group "sshd"
#:comment "Secure Shell Daemon"
#:home "/var/empty"
#:shell "/usr/sbin/nologin"
#:system? #t)
(user-account #:name "_dhcp"
#:uid 65
#:group "_dhcp"
#:comment "dhcp programs"
#:home "/var/empty"
#:shell "/usr/sbin/nologin"
#:system? #t)
(user-account #:name "operator"
#:uid 1000
#:group "operator"
#:supplementary-groups '("wheel")
#:comment "Fruix Operator"
#:home "/home/operator"
#:shell "/bin/sh"
#:system? #f))
#:file-systems (list (file-system #:device "/dev/gpt/fruix-root"
#:mount-point "/"
#:type "ufs"
#:options "rw"
#:needed-for-boot? #t)
(file-system #:device "devfs"
#:mount-point "/dev"
#:type "devfs"
#:options "rw"
#:needed-for-boot? #t)
(file-system #:device "tmpfs"
#:mount-point "/tmp"
#:type "tmpfs"
#:options "rw,size=64m"))
#:services '(shepherd ready-marker sshd)
#:loader-entries '(("autoboot_delay" . "1")
("boot_multicons" . "YES")
("boot_serial" . "YES")
("console" . "comconsole,vidconsole"))
#:rc-conf-entries '(("clear_tmp_enable" . "NO")
("hostid_enable" . "NO")
("sendmail_enable" . "NONE")
("sshd_enable" . "YES")
("ifconfig_xn0" . "SYNCDHCP")
("ifconfig_em0" . "SYNCDHCP")
("ifconfig_vtnet0" . "SYNCDHCP"))
#:init-mode 'shepherd-pid1
#:ready-marker "/var/lib/fruix/ready"
#:root-authorized-keys '("__ROOT_AUTHORIZED_KEY__")))

73
tests/system/postphase20-installed-node-operating-system.scm.in Normal file
View File

@@ -0,0 +1,73 @@
(use-modules (fruix system freebsd)
(fruix packages freebsd))
(define postphase20-promoted-native-build-result
(promoted-native-build-result
#:store-path "__PROMOTED_RESULT_STORE__"))
(define postphase20-installed-node-operating-system
(operating-system-from-promoted-native-build-result
postphase20-promoted-native-build-result
#:host-name "__HOST_NAME__"
#:groups (list (user-group #:name "wheel" #:gid 0 #:system? #t)
(user-group #:name "sshd" #:gid 22 #:system? #t)
(user-group #:name "_dhcp" #:gid 65 #:system? #t)
(user-group #:name "operator" #:gid 1000 #:system? #f))
#:users (list (user-account #:name "root"
#:uid 0
#:group "wheel"
#:comment "Charlie &"
#:home "/root"
#:shell "/bin/sh"
#:system? #t)
(user-account #:name "sshd"
#:uid 22
#:group "sshd"
#:comment "Secure Shell Daemon"
#:home "/var/empty"
#:shell "/usr/sbin/nologin"
#:system? #t)
(user-account #:name "_dhcp"
#:uid 65
#:group "_dhcp"
#:comment "dhcp programs"
#:home "/var/empty"
#:shell "/usr/sbin/nologin"
#:system? #t)
(user-account #:name "operator"
#:uid 1000
#:group "operator"
#:supplementary-groups '("wheel")
#:comment "Fruix Operator"
#:home "/home/operator"
#:shell "/bin/sh"
#:system? #f))
#:file-systems (list (file-system #:device "/dev/gpt/fruix-root"
#:mount-point "/"
#:type "ufs"
#:options "rw"
#:needed-for-boot? #t)
(file-system #:device "devfs"
#:mount-point "/dev"
#:type "devfs"
#:options "rw"
#:needed-for-boot? #t)
(file-system #:device "tmpfs"
#:mount-point "/tmp"
#:type "tmpfs"
#:options "rw,size=64m"))
#:services '(shepherd ready-marker sshd)
#:loader-entries '(("autoboot_delay" . "1")
("boot_multicons" . "YES")
("boot_serial" . "YES")
("console" . "comconsole,vidconsole"))
#:rc-conf-entries '(("clear_tmp_enable" . "NO")
("hostid_enable" . "NO")
("sendmail_enable" . "NONE")
("sshd_enable" . "YES")
("ifconfig_xn0" . "SYNCDHCP")
("ifconfig_em0" . "SYNCDHCP")
("ifconfig_vtnet0" . "SYNCDHCP"))
#:init-mode 'shepherd-pid1
#:ready-marker "/var/lib/fruix/ready"
#:root-authorized-keys '("__ROOT_AUTHORIZED_KEY__")))

388
tests/system/run-phase19-installed-system-rollback-qemu.sh Executable file
View File

@@ -0,0 +1,388 @@
#!/bin/sh
set -eu
project_root=${PROJECT_ROOT:-$(pwd)}
script_dir=$(CDPATH= cd -- "$(dirname "$0")" && pwd)
fruix_cmd=$project_root/bin/fruix
os_template=${OS_TEMPLATE:-$script_dir/phase19-generation-rollback-operating-system.scm.in}
system_name=${SYSTEM_NAME:-phase19-operating-system}
store_dir=${STORE_DIR:-/frx/store}
disk_capacity=${DISK_CAPACITY:-12g}
root_size=${ROOT_SIZE:-10g}
qemu_smp=${QEMU_SMP:-2}
ssh_port=${QEMU_SSH_PORT:-10024}
base_name=${BASE_NAME:-phase19-generation-layout}
base_version_label=${BASE_VERSION_LABEL:-15.0-STABLE-generation-layout}
base_release=${BASE_RELEASE:-15.0-STABLE}
base_branch=${BASE_BRANCH:-stable/15}
source_name=${SOURCE_NAME:-stable15-generation-layout-source}
source_ref=${SOURCE_REF:-stable/15}
source_commit=${SOURCE_COMMIT:-332708a606f6bf0841c1d4a74c0d067f5640fe89}
declared_source_root=${DECLARED_SOURCE_ROOT:-/var/empty/fruix-unused-source-root-generation-layout}
current_host_name=${CURRENT_HOST_NAME:-fruix-freebsd-current}
candidate_host_name=${CANDIDATE_HOST_NAME:-fruix-freebsd-canary}
metadata_target=${METADATA_OUT:-}
root_authorized_key_file=${ROOT_AUTHORIZED_KEY_FILE:-$HOME/.ssh/id_ed25519.pub}
root_ssh_private_key_file=${ROOT_SSH_PRIVATE_KEY_FILE:-$HOME/.ssh/id_ed25519}
cleanup=0
[ -x "$fruix_cmd" ] || {
echo "fruix command is not executable: $fruix_cmd" >&2
exit 1
}
[ -f "$os_template" ] || {
echo "missing operating-system template: $os_template" >&2
exit 1
}
[ -f "$root_authorized_key_file" ] || {
echo "missing root authorized key file: $root_authorized_key_file" >&2
exit 1
}
[ -f "$root_ssh_private_key_file" ] || {
echo "missing root SSH private key file: $root_ssh_private_key_file" >&2
exit 1
}
command -v qemu-system-x86_64 >/dev/null 2>&1 || {
echo "qemu-system-x86_64 is required" >&2
exit 1
}
[ -f /usr/local/share/edk2-qemu/QEMU_UEFI_CODE-x86_64.fd ] || {
echo "missing QEMU UEFI firmware" >&2
exit 1
}
if [ -n "${WORKDIR:-}" ]; then
workdir=$WORKDIR
mkdir -p "$workdir"
else
workdir=$(mktemp -d /tmp/fruix-phase19-installed-system-rollback-qemu.XXXXXX)
cleanup=1
fi
if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
cleanup=0
fi
current_os_file=$workdir/current-operating-system.scm
candidate_os_file=$workdir/candidate-operating-system.scm
current_install_out=$workdir/current-install.txt
candidate_build_out=$workdir/candidate-build.txt
target_image=$workdir/installed.img
candidate_store_items=$workdir/candidate-store-items.txt
stage_log=$workdir/stage-candidate-store.txt
serial_log=$workdir/serial.log
qemu_pidfile=$workdir/qemu.pid
uefi_vars=$workdir/QEMU_UEFI_VARS.fd
metadata_file=$workdir/phase19-installed-system-rollback-qemu-metadata.txt
switch_status_file=$workdir/switch-status.txt
rollback_status_file=$workdir/rollback-status.txt
boot_current_status_file=$workdir/boot-current-status.txt
boot_candidate_status_file=$workdir/boot-candidate-status.txt
boot_rollback_status_file=$workdir/boot-rollback-status.txt
generation2_metadata_file=$workdir/generation2-metadata.scm
generation2_install_file=$workdir/generation2-install.scm
mnt_root=$workdir/mnt-root
md_unit=
cleanup_workdir() {
if [ -f "$qemu_pidfile" ]; then
sudo kill "$(sudo cat "$qemu_pidfile")" >/dev/null 2>&1 || true
fi
if [ -n "$md_unit" ]; then
sudo umount "$mnt_root" >/dev/null 2>&1 || true
sudo mdconfig -d -u "$md_unit" >/dev/null 2>&1 || true
fi
if [ "$cleanup" -eq 1 ]; then
rm -rf "$workdir" 2>/dev/null || sudo rm -rf "$workdir"
fi
}
trap cleanup_workdir EXIT INT TERM
render_os() {
output=$1
host_name=$2
root_authorized_key=$(tr -d '\n' < "$root_authorized_key_file")
sed \
-e "s|__BASE_NAME__|$base_name|g" \
-e "s|__BASE_VERSION_LABEL__|$base_version_label|g" \
-e "s|__BASE_RELEASE__|$base_release|g" \
-e "s|__BASE_BRANCH__|$base_branch|g" \
-e "s|__SOURCE_NAME__|$source_name|g" \
-e "s|__SOURCE_REF__|$source_ref|g" \
-e "s|__SOURCE_COMMIT__|$source_commit|g" \
-e "s|__DECLARED_SOURCE_ROOT__|$declared_source_root|g" \
-e "s|__HOST_NAME__|$host_name|g" \
-e "s|__ROOT_AUTHORIZED_KEY__|$root_authorized_key|g" \
"$os_template" > "$output"
}
render_os "$current_os_file" "$current_host_name"
render_os "$candidate_os_file" "$candidate_host_name"
cp /usr/local/share/edk2-qemu/QEMU_UEFI_VARS-x86_64.fd "$uefi_vars"
mkdir -p "$mnt_root"
action_env() {
sudo env \
HOME="$HOME" \
GUILE_AUTO_COMPILE=0 \
FRUIX_FREEBSD_BUILD_JOBS="${FRUIX_FREEBSD_BUILD_JOBS:-8}" \
GUIX_SOURCE_DIR="${GUIX_SOURCE_DIR:-$HOME/repos/guix}" \
GUILE_BIN="${GUILE_BIN:-/tmp/guile-freebsd-validate-install/bin/guile}" \
GUILE_EXTRA_PREFIX="${GUILE_EXTRA_PREFIX:-/tmp/guile-gnutls-freebsd-validate-install}" \
SHEPHERD_PREFIX="${SHEPHERD_PREFIX:-/tmp/shepherd-freebsd-validate-install}" \
"$@"
}
field() {
name=$1
file=$2
sed -n "s/^$name=//p" "$file" | tail -n 1
}
status_field() {
name=$1
file=$2
sed -n "s/^$name=//p" "$file" | tail -n 1
}
wait_for_ssh() {
for attempt in $(jot 120 1 120); do
if ssh_guest 'service sshd onestatus >/dev/null 2>&1' >/dev/null 2>&1; then
return 0
fi
sleep 2
done
return 1
}
ssh_guest() {
ssh -p "$ssh_port" -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=5 \
root@127.0.0.1 "$@"
}
reboot_guest() {
ssh_guest 'shutdown -r now >/dev/null 2>&1 || reboot >/dev/null 2>&1 || true' >/dev/null 2>&1 || true
sleep 5
wait_for_ssh || {
echo "guest did not return over SSH after reboot" >&2
exit 1
}
}
capture_status() {
output_file=$1
ssh_guest '/usr/local/bin/fruix system status' > "$output_file"
}
action_env "$fruix_cmd" system install "$current_os_file" \
--system "$system_name" \
--store "$store_dir" \
--target "$target_image" \
--disk-capacity "$disk_capacity" \
--root-size "$root_size" > "$current_install_out"
current_closure_path=$(field closure_path "$current_install_out")
install_metadata_path=$(field install_metadata_path "$current_install_out")
materialized_source_store=$(field materialized_source_stores "$current_install_out")
[ -n "$current_closure_path" ] || { echo "missing current closure path" >&2; exit 1; }
[ "$install_metadata_path" = /var/lib/fruix/install.scm ] || { echo "unexpected install metadata path: $install_metadata_path" >&2; exit 1; }
action_env "$fruix_cmd" system build "$candidate_os_file" \
--system "$system_name" \
--store "$store_dir" > "$candidate_build_out"
candidate_closure_path=$(field closure_path "$candidate_build_out")
[ -n "$candidate_closure_path" ] || { echo "missing candidate closure path" >&2; exit 1; }
[ "$candidate_closure_path" != "$current_closure_path" ] || { echo "candidate closure unexpectedly matches current closure" >&2; exit 1; }
{
printf '%s\n' "$candidate_closure_path"
cat "$candidate_closure_path/.references"
} | awk 'NF { print }' | sort -u > "$candidate_store_items"
candidate_store_item_count=$(wc -l < "$candidate_store_items" | tr -d ' ')
md=$(sudo mdconfig -a -t vnode -f "$target_image")
md_unit=${md#md}
sudo mount -t ufs "/dev/${md}p2" "$mnt_root"
sudo mkdir -p "$mnt_root/frx/store"
: > "$stage_log"
while IFS= read -r item; do
[ -n "$item" ] || continue
item_base=$(basename "$item")
if [ -e "$mnt_root/frx/store/$item_base" ]; then
printf 'already-present=%s\n' "$item_base" >> "$stage_log"
continue
fi
printf 'copy=%s\n' "$item_base" >> "$stage_log"
sudo sh -c "cd '$store_dir' && pax -rw -pe '$item_base' '$mnt_root/frx/store'"
done < "$candidate_store_items"
sudo sync
sudo umount "$mnt_root"
sudo mdconfig -d -u "$md_unit"
md_unit=
sudo qemu-system-x86_64 \
-machine q35,accel=tcg \
-cpu max \
-m 2048 \
-smp "$qemu_smp" \
-display none \
-serial "file:$serial_log" \
-monitor none \
-pidfile "$qemu_pidfile" \
-daemonize \
-drive if=pflash,format=raw,readonly=on,file=/usr/local/share/edk2-qemu/QEMU_UEFI_CODE-x86_64.fd \
-drive if=pflash,format=raw,file="$uefi_vars" \
-drive if=virtio,format=raw,file="$target_image" \
-netdev user,id=net0,hostfwd=tcp::${ssh_port}-:22 \
-device virtio-net-pci,netdev=net0
wait_for_ssh || {
echo "guest never became reachable over SSH" >&2
exit 1
}
capture_status "$boot_current_status_file"
boot_current_closure=$(ssh_guest 'readlink /run/current-system')
boot_current_hostname=$(ssh_guest 'hostname')
boot_current_generation=$(status_field current_generation "$boot_current_status_file")
boot_current_link=$(status_field current_link "$boot_current_status_file")
boot_current_rollback_generation=$(status_field rollback_generation "$boot_current_status_file")
[ "$boot_current_closure" = "$current_closure_path" ] || { echo "unexpected current closure after initial boot: $boot_current_closure" >&2; exit 1; }
[ "$boot_current_hostname" = "$current_host_name" ] || { echo "unexpected hostname after initial boot: $boot_current_hostname" >&2; exit 1; }
[ "$boot_current_generation" = 1 ] || { echo "unexpected initial generation: $boot_current_generation" >&2; exit 1; }
[ "$boot_current_link" = generations/1 ] || { echo "unexpected initial current link: $boot_current_link" >&2; exit 1; }
[ -z "$boot_current_rollback_generation" ] || { echo "rollback generation should be empty before switch" >&2; exit 1; }
ssh_guest 'test -x /usr/local/bin/fruix'
ssh_guest "/usr/local/bin/fruix system switch $candidate_closure_path" > "$switch_status_file"
switch_current_generation=$(status_field current_generation "$switch_status_file")
switch_current_link=$(status_field current_link "$switch_status_file")
switch_current_closure=$(status_field current_closure "$switch_status_file")
switch_rollback_generation=$(status_field rollback_generation "$switch_status_file")
switch_rollback_link=$(status_field rollback_link "$switch_status_file")
switch_rollback_closure=$(status_field rollback_closure "$switch_status_file")
[ "$switch_current_generation" = 2 ] || { echo "unexpected generation after switch: $switch_current_generation" >&2; exit 1; }
[ "$switch_current_link" = generations/2 ] || { echo "unexpected current link after switch: $switch_current_link" >&2; exit 1; }
[ "$switch_current_closure" = "$candidate_closure_path" ] || { echo "unexpected current closure after switch: $switch_current_closure" >&2; exit 1; }
[ "$switch_rollback_generation" = 1 ] || { echo "unexpected rollback generation after switch: $switch_rollback_generation" >&2; exit 1; }
[ "$switch_rollback_link" = generations/1 ] || { echo "unexpected rollback link after switch: $switch_rollback_link" >&2; exit 1; }
[ "$switch_rollback_closure" = "$current_closure_path" ] || { echo "unexpected rollback closure after switch: $switch_rollback_closure" >&2; exit 1; }
[ "$(ssh_guest 'readlink /frx/var/fruix/gcroots/current-system')" = "$candidate_closure_path" ] || { echo "unexpected current-system gc root after switch" >&2; exit 1; }
[ "$(ssh_guest 'readlink /frx/var/fruix/gcroots/rollback-system')" = "$current_closure_path" ] || { echo "unexpected rollback-system gc root after switch" >&2; exit 1; }
[ "$(ssh_guest 'readlink /frx/var/fruix/gcroots/system-2')" = "$candidate_closure_path" ] || { echo "unexpected system-2 gc root after switch" >&2; exit 1; }
ssh_guest 'test -f /var/lib/fruix/system/generations/2/metadata.scm'
ssh_guest 'test -f /var/lib/fruix/system/generations/2/provenance.scm'
ssh_guest 'test -f /var/lib/fruix/system/generations/2/install.scm'
ssh_guest "cat /var/lib/fruix/system/generations/2/metadata.scm" > "$generation2_metadata_file"
ssh_guest "cat /var/lib/fruix/system/generations/2/install.scm" > "$generation2_install_file"
case "$(cat "$generation2_metadata_file")" in
*"$candidate_closure_path"*"$current_closure_path"*) : ;;
*) echo "generation 2 metadata does not record both candidate and previous closure paths" >&2; exit 1 ;;
esac
case "$(cat "$generation2_install_file")" in
*"(deployment-kind . \"switch\")"*"$candidate_closure_path"*) : ;;
*) echo "generation 2 install metadata does not record switch provenance" >&2; exit 1 ;;
esac
reboot_guest
capture_status "$boot_candidate_status_file"
boot_candidate_closure=$(ssh_guest 'readlink /run/current-system')
boot_candidate_hostname=$(ssh_guest 'hostname')
boot_candidate_generation=$(status_field current_generation "$boot_candidate_status_file")
boot_candidate_rollback_generation=$(status_field rollback_generation "$boot_candidate_status_file")
boot_candidate_rollback_closure=$(status_field rollback_closure "$boot_candidate_status_file")
[ "$boot_candidate_closure" = "$candidate_closure_path" ] || { echo "unexpected closure after switch reboot: $boot_candidate_closure" >&2; exit 1; }
[ "$boot_candidate_hostname" = "$candidate_host_name" ] || { echo "unexpected hostname after switch reboot: $boot_candidate_hostname" >&2; exit 1; }
[ "$boot_candidate_generation" = 2 ] || { echo "unexpected generation after switch reboot: $boot_candidate_generation" >&2; exit 1; }
[ "$boot_candidate_rollback_generation" = 1 ] || { echo "unexpected rollback generation after switch reboot: $boot_candidate_rollback_generation" >&2; exit 1; }
[ "$boot_candidate_rollback_closure" = "$current_closure_path" ] || { echo "unexpected rollback closure after switch reboot: $boot_candidate_rollback_closure" >&2; exit 1; }
ssh_guest '/usr/local/bin/fruix system rollback' > "$rollback_status_file"
rollback_current_generation=$(status_field current_generation "$rollback_status_file")
rollback_current_link=$(status_field current_link "$rollback_status_file")
rollback_current_closure=$(status_field current_closure "$rollback_status_file")
rollback_rollback_generation=$(status_field rollback_generation "$rollback_status_file")
rollback_rollback_link=$(status_field rollback_link "$rollback_status_file")
rollback_rollback_closure=$(status_field rollback_closure "$rollback_status_file")
[ "$rollback_current_generation" = 1 ] || { echo "unexpected generation after rollback: $rollback_current_generation" >&2; exit 1; }
[ "$rollback_current_link" = generations/1 ] || { echo "unexpected current link after rollback: $rollback_current_link" >&2; exit 1; }
[ "$rollback_current_closure" = "$current_closure_path" ] || { echo "unexpected current closure after rollback: $rollback_current_closure" >&2; exit 1; }
[ "$rollback_rollback_generation" = 2 ] || { echo "unexpected rollback generation after rollback: $rollback_rollback_generation" >&2; exit 1; }
[ "$rollback_rollback_link" = generations/2 ] || { echo "unexpected rollback link after rollback: $rollback_rollback_link" >&2; exit 1; }
[ "$rollback_rollback_closure" = "$candidate_closure_path" ] || { echo "unexpected rollback closure after rollback: $rollback_rollback_closure" >&2; exit 1; }
[ "$(ssh_guest 'readlink /frx/var/fruix/gcroots/current-system')" = "$current_closure_path" ] || { echo "unexpected current-system gc root after rollback" >&2; exit 1; }
[ "$(ssh_guest 'readlink /frx/var/fruix/gcroots/rollback-system')" = "$candidate_closure_path" ] || { echo "unexpected rollback-system gc root after rollback" >&2; exit 1; }
reboot_guest
capture_status "$boot_rollback_status_file"
boot_rollback_closure=$(ssh_guest 'readlink /run/current-system')
boot_rollback_hostname=$(ssh_guest 'hostname')
boot_rollback_generation=$(status_field current_generation "$boot_rollback_status_file")
boot_rollback_rollback_generation=$(status_field rollback_generation "$boot_rollback_status_file")
boot_rollback_rollback_closure=$(status_field rollback_closure "$boot_rollback_status_file")
shepherd_status=$(ssh_guest '/usr/local/etc/rc.d/fruix-shepherd onestatus >/dev/null 2>&1 && echo running || echo stopped')
sshd_status=$(ssh_guest 'service sshd onestatus >/dev/null 2>&1 && echo running || echo stopped')
activate_log=$(ssh_guest 'cat /var/log/fruix-activate.log 2>/dev/null || true' | tr '\n' ' ')
[ "$boot_rollback_closure" = "$current_closure_path" ] || { echo "unexpected closure after rollback reboot: $boot_rollback_closure" >&2; exit 1; }
[ "$boot_rollback_hostname" = "$current_host_name" ] || { echo "unexpected hostname after rollback reboot: $boot_rollback_hostname" >&2; exit 1; }
[ "$boot_rollback_generation" = 1 ] || { echo "unexpected generation after rollback reboot: $boot_rollback_generation" >&2; exit 1; }
[ "$boot_rollback_rollback_generation" = 2 ] || { echo "unexpected rollback generation after rollback reboot: $boot_rollback_rollback_generation" >&2; exit 1; }
[ "$boot_rollback_rollback_closure" = "$candidate_closure_path" ] || { echo "unexpected rollback closure after rollback reboot: $boot_rollback_rollback_closure" >&2; exit 1; }
[ "$shepherd_status" = running ] || { echo "fruix-shepherd is not running after rollback reboot" >&2; exit 1; }
[ "$sshd_status" = running ] || { echo "sshd is not running after rollback reboot" >&2; exit 1; }
case "$activate_log" in
*fruix-activate:done*) : ;;
*) echo "activation log does not show success after rollback workflow" >&2; exit 1 ;;
esac
cat >"$metadata_file" <<EOF
workdir=$workdir
current_os_file=$current_os_file
candidate_os_file=$candidate_os_file
target_image=$target_image
current_closure_path=$current_closure_path
candidate_closure_path=$candidate_closure_path
current_host_name=$current_host_name
candidate_host_name=$candidate_host_name
candidate_store_item_count=$candidate_store_item_count
candidate_store_items=$candidate_store_items
stage_log=$stage_log
serial_log=$serial_log
install_metadata_path=$install_metadata_path
materialized_source_store=$materialized_source_store
switch_status_file=$switch_status_file
rollback_status_file=$rollback_status_file
boot_current_status_file=$boot_current_status_file
boot_candidate_status_file=$boot_candidate_status_file
boot_rollback_status_file=$boot_rollback_status_file
generation2_metadata_file=$generation2_metadata_file
generation2_install_file=$generation2_install_file
final_current_generation=$boot_rollback_generation
final_current_closure=$boot_rollback_closure
final_rollback_generation=$boot_rollback_rollback_generation
final_rollback_closure=$boot_rollback_rollback_closure
shepherd_status=$shepherd_status
sshd_status=$sshd_status
boot_backend=qemu-uefi-tcg
installed_system_switch=ok
installed_system_rollback=ok
EOF
if [ -n "$metadata_target" ]; then
mkdir -p "$(dirname "$metadata_target")"
cp "$metadata_file" "$metadata_target"
fi
printf 'PASS phase19-installed-system-rollback-qemu\n'
printf 'Work directory: %s\n' "$workdir"
printf 'Metadata file: %s\n' "$metadata_file"
if [ -n "$metadata_target" ]; then
printf 'Copied metadata to: %s\n' "$metadata_target"
fi
printf '%s\n' '--- metadata ---'
cat "$metadata_file"

260
tests/system/run-phase20-development-environment-xcpng.sh Executable file
View File

@@ -0,0 +1,260 @@
#!/bin/sh
set -eu
repo_root=${PROJECT_ROOT:-$(pwd)}
os_template=${OS_TEMPLATE:-$repo_root/tests/system/phase20-development-operating-system.scm.in}
system_name=${SYSTEM_NAME:-phase20-operating-system}
root_size=${ROOT_SIZE:-8g}
metadata_target=${METADATA_OUT:-}
root_authorized_key_file=${ROOT_AUTHORIZED_KEY_FILE:-$HOME/.ssh/id_ed25519.pub}
root_ssh_private_key_file=${ROOT_SSH_PRIVATE_KEY_FILE:-$HOME/.ssh/id_ed25519}
cleanup=0
if [ -n "${WORKDIR:-}" ]; then
workdir=$WORKDIR
mkdir -p "$workdir"
else
workdir=$(mktemp -d /tmp/fruix-phase20-development-xcpng.XXXXXX)
cleanup=1
fi
if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
cleanup=0
fi
inner_metadata=$workdir/phase20-development-inner-metadata.txt
metadata_file=$workdir/phase20-development-environment-xcpng-metadata.txt
action_cleanup() {
if [ "$cleanup" -eq 1 ]; then
rm -rf "$workdir"
fi
}
trap action_cleanup EXIT INT TERM
KEEP_WORKDIR=1 WORKDIR="$workdir/inner" METADATA_OUT="$inner_metadata" \
ROOT_AUTHORIZED_KEY_FILE="$root_authorized_key_file" \
ROOT_SSH_PRIVATE_KEY_FILE="$root_ssh_private_key_file" \
OS_TEMPLATE="$os_template" SYSTEM_NAME="$system_name" ROOT_SIZE="$root_size" \
"$repo_root/tests/system/run-phase11-shepherd-pid1-xcpng.sh"
phase8_metadata=$(sed -n 's/^phase8_metadata=//p' "$inner_metadata")
closure_path=$(sed -n 's/^closure_path=//p' "$inner_metadata")
closure_base=$(sed -n 's/^closure_base=//p' "$inner_metadata")
guest_ip=$(sed -n 's/^guest_ip=//p' "$inner_metadata")
vm_id=$(sed -n 's/^vm_id=//p' "$inner_metadata")
vdi_id=$(sed -n 's/^vdi_id=//p' "$inner_metadata")
shepherd_pid=$(sed -n 's/^shepherd_pid=//p' "$inner_metadata")
sshd_status=$(sed -n 's/^sshd_status=//p' "$inner_metadata")
compat_prefix_shims=$(sed -n 's/^compat_prefix_shims=//p' "$inner_metadata")
guile_module_smoke=$(sed -n 's/^guile_module_smoke=//p' "$inner_metadata")
activate_log=$(sed -n 's/^activate_log=//p' "$inner_metadata")
development_profile_path=$closure_path/development-profile
build_profile_path=$closure_path/build-profile
runtime_profile_path=$closure_path/profile
development_env_script=$closure_path/usr/local/bin/fruix-development-environment
build_env_script=$closure_path/usr/local/bin/fruix-build-environment
[ "$shepherd_pid" = 1 ] || { echo "shepherd was not PID 1" >&2; exit 1; }
[ "$sshd_status" = running ] || { echo "sshd is not running" >&2; exit 1; }
[ "$compat_prefix_shims" = absent ] || { echo "compatibility prefix shims reappeared" >&2; exit 1; }
[ "$guile_module_smoke" = ok ] || { echo "guest Guile module smoke failed" >&2; exit 1; }
case "$activate_log" in
*fruix-activate:done*) : ;;
*) echo "activation log does not show success" >&2; exit 1 ;;
esac
for path in \
"$development_profile_path/bin/cc" \
"$development_profile_path/bin/c++" \
"$development_profile_path/bin/ar" \
"$development_profile_path/usr/include/sys/param.h" \
"$development_profile_path/usr/share/mk/bsd.prog.mk" \
"$build_profile_path/bin/cc" \
"$build_profile_path/usr/include/sys/param.h" \
"$build_profile_path/usr/share/mk/bsd.prog.mk" \
"$development_env_script" \
"$build_env_script"
do
[ -e "$path" ] || {
echo "required development environment path missing: $path" >&2
exit 1
}
done
[ ! -e "$runtime_profile_path/include" ] || { echo "runtime profile unexpectedly contains headers" >&2; exit 1; }
[ ! -e "$runtime_profile_path/usr/share/mk" ] || { echo "runtime profile unexpectedly contains /usr/share/mk" >&2; exit 1; }
[ ! -e "$runtime_profile_path/bin/cc" ] || { echo "runtime profile unexpectedly contains cc" >&2; exit 1; }
ssh_guest() {
ssh -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=5 \
root@"$guest_ip" "$@"
}
guest_dev_metadata=$(ssh -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=5 \
root@"$guest_ip" 'sh -s' <<'EOF'
set -eu
[ -x /usr/local/bin/fruix-development-environment ]
[ -x /usr/local/bin/fruix-build-environment ]
[ -L /run/current-development ]
[ "$(readlink /run/current-development)" = "/run/current-system/development-profile" ]
[ -L /run/current-build ]
[ "$(readlink /run/current-build)" = "/run/current-system/build-profile" ]
exports=$(/usr/local/bin/fruix-development-environment)
printf '%s\n' "$exports" | grep '^export FRUIX_DEVELOPMENT_PROFILE="/run/current-system/development-profile"$' >/dev/null
printf '%s\n' "$exports" | grep '^export MAKEFLAGS="-m /run/current-system/development-profile/usr/share/mk"$' >/dev/null
eval "$exports"
[ -d "$FRUIX_DEVELOPMENT_PROFILE" ]
[ -x "$FRUIX_CC" ]
[ -x "$FRUIX_CXX" ]
[ -x "$FRUIX_AR" ]
[ -f "$FRUIX_DEVELOPMENT_INCLUDE/sys/param.h" ]
[ -f "$FRUIX_DEVELOPMENT_SHARE_MK/bsd.prog.mk" ]
cc_version=$($FRUIX_CC --version | awk 'NR==1 { print; exit }')
build_exports=$(/usr/local/bin/fruix-build-environment)
printf '%s\n' "$build_exports" | grep '^unset MAKEOBJDIRPREFIX MAKEFLAGS CC CXX AR RANLIB NM CPPFLAGS CFLAGS CXXFLAGS LDFLAGS$' >/dev/null
printf '%s\n' "$build_exports" | grep '^export FRUIX_BUILD_PROFILE="/run/current-system/build-profile"$' >/dev/null
eval "$build_exports"
[ -d "$FRUIX_BUILD_PROFILE" ]
[ -f "$FRUIX_BUILD_INCLUDE/sys/param.h" ]
[ -f "$FRUIX_BUILD_SHARE_MK/bsd.prog.mk" ]
[ "${MAKEFLAGS-unset}" = unset ]
[ "${CPPFLAGS-unset}" = unset ]
[ "${CFLAGS-unset}" = unset ]
[ "${LDFLAGS-unset}" = unset ]
[ "$FRUIX_BMAKE" = make ]
build_profile_value=$FRUIX_BUILD_PROFILE
eval "$exports"
tmp=/tmp/fruix-phase20-development-env
rm -rf "$tmp"
mkdir -p "$tmp/direct" "$tmp/mk"
cat > "$tmp/direct/hello.c" <<'EOS'
#include <stdio.h>
int main(void){puts("hello-from-direct-dev-env");return 0;}
EOS
$FRUIX_CC $CPPFLAGS $LDFLAGS "$tmp/direct/hello.c" -o "$tmp/direct/hello"
hello_direct=$($tmp/direct/hello)
cat > "$tmp/mk/hello.c" <<'EOS'
#include <stdio.h>
int main(void){puts("hello-from-make-dev-env");return 0;}
EOS
cat > "$tmp/mk/Makefile" <<'EOS'
PROG=hello
SRCS=hello.c
NO_MAN=yes
.include <bsd.prog.mk>
EOS
cat > "$tmp/mk/hello.1" <<'EOS'
.Dd April 5, 2026
.Dt HELLO 1
.Sh NAME
.Nm hello
.Nd phase20 development environment smoke binary
EOS
cd "$tmp/mk"
make clean >/dev/null 2>&1 || true
make > "$tmp/mk/make.log" 2>&1
hello_make=$(./hello)
make_log_tail=$(tail -n 20 "$tmp/mk/make.log" | tr '\n' ' ')
exports_flat=$(printf '%s' "$exports" | tr '\n' ' ')
printf 'development_profile=%s\n' "$FRUIX_DEVELOPMENT_PROFILE"
printf 'build_profile=%s\n' "$build_profile_value"
printf 'cc_version=%s\n' "$cc_version"
printf 'hello_direct=%s\n' "$hello_direct"
printf 'hello_make=%s\n' "$hello_make"
build_exports_flat=$(printf '%s' "$build_exports" | tr '\n' ' ')
printf 'exports=%s\n' "$exports_flat"
printf 'build_exports=%s\n' "$build_exports_flat"
printf 'make_log_tail=%s\n' "$make_log_tail"
EOF
)
development_profile=$(printf '%s\n' "$guest_dev_metadata" | sed -n 's/^development_profile=//p')
build_profile=$(printf '%s\n' "$guest_dev_metadata" | sed -n 's/^build_profile=//p')
cc_version=$(printf '%s\n' "$guest_dev_metadata" | sed -n 's/^cc_version=//p')
hello_direct=$(printf '%s\n' "$guest_dev_metadata" | sed -n 's/^hello_direct=//p')
hello_make=$(printf '%s\n' "$guest_dev_metadata" | sed -n 's/^hello_make=//p')
development_exports=$(printf '%s\n' "$guest_dev_metadata" | sed -n 's/^exports=//p')
build_exports=$(printf '%s\n' "$guest_dev_metadata" | sed -n 's/^build_exports=//p')
make_log_tail=$(printf '%s\n' "$guest_dev_metadata" | sed -n 's/^make_log_tail=//p')
[ "$development_profile" = "/run/current-system/development-profile" ] || {
echo "unexpected guest development profile path: $development_profile" >&2
exit 1
}
[ "$build_profile" = "/run/current-system/build-profile" ] || {
echo "unexpected guest build profile path: $build_profile" >&2
exit 1
}
case "$cc_version" in
*"FreeBSD clang version"*) : ;;
*) echo "unexpected cc version output: $cc_version" >&2; exit 1 ;;
esac
[ "$hello_direct" = hello-from-direct-dev-env ] || {
echo "unexpected direct compile output: $hello_direct" >&2
exit 1
}
[ "$hello_make" = hello-from-make-dev-env ] || {
echo "unexpected make-based compile output: $hello_make" >&2
exit 1
}
case "$development_exports" in
*'export FRUIX_CC="/run/current-system/development-profile/bin/cc"'*) : ;;
*) echo "development environment exports do not include FRUIX_CC" >&2; exit 1 ;;
esac
case "$build_exports" in
*'export FRUIX_BUILD_PROFILE="/run/current-system/build-profile"'*) : ;;
*) echo "build environment exports do not include FRUIX_BUILD_PROFILE" >&2; exit 1 ;;
esac
cat >"$metadata_file" <<EOF
workdir=$workdir
inner_metadata=$inner_metadata
phase8_metadata=$phase8_metadata
closure_path=$closure_path
closure_base=$closure_base
vm_id=$vm_id
vdi_id=$vdi_id
guest_ip=$guest_ip
root_size=$root_size
development_profile_path=$development_profile_path
build_profile_path=$build_profile_path
development_env_script=$development_env_script
build_env_script=$build_env_script
shepherd_pid=$shepherd_pid
sshd_status=$sshd_status
compat_prefix_shims=$compat_prefix_shims
guile_module_smoke=$guile_module_smoke
development_profile_guest=$development_profile
build_profile_guest=$build_profile
cc_version=$cc_version
hello_direct=$hello_direct
hello_make=$hello_make
development_exports=$development_exports
build_exports=$build_exports
make_log_tail=$make_log_tail
boot_backend=xcp-ng-xo-cli
init_mode=shepherd-pid1
development_environment=ok
EOF
if [ -n "$metadata_target" ]; then
mkdir -p "$(dirname "$metadata_target")"
cp "$metadata_file" "$metadata_target"
fi
printf 'PASS phase20-development-environment-xcpng\n'
printf 'Work directory: %s\n' "$workdir"
printf 'Metadata file: %s\n' "$metadata_file"
if [ -n "$metadata_target" ]; then
printf 'Copied metadata to: %s\n' "$metadata_target"
fi
printf '%s\n' '--- metadata ---'
cat "$metadata_file"

239
tests/system/run-phase20-host-initiated-native-build-store-promotion-xcpng.sh Executable file
View File

@@ -0,0 +1,239 @@
#!/bin/sh
set -eu
repo_root=${PROJECT_ROOT:-$(pwd)}
os_template=${OS_TEMPLATE:-$repo_root/tests/system/phase20-development-operating-system.scm.in}
system_name=${SYSTEM_NAME:-phase20-operating-system}
root_size=${ROOT_SIZE:-20g}
store_dir=${STORE_DIR:-/frx/store}
metadata_target=${METADATA_OUT:-}
root_authorized_key_file=${ROOT_AUTHORIZED_KEY_FILE:-$HOME/.ssh/id_ed25519.pub}
root_ssh_private_key_file=${ROOT_SSH_PRIVATE_KEY_FILE:-$HOME/.ssh/id_ed25519}
cleanup=0
if [ -n "${WORKDIR:-}" ]; then
workdir=$WORKDIR
mkdir -p "$workdir"
else
workdir=$(mktemp -d /tmp/fruix-phase20-host-initiated-native-build-store-promotion-xcpng.XXXXXX)
cleanup=1
fi
if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
cleanup=0
fi
inner_metadata=$workdir/phase20-host-initiated-native-build-store-promotion-inner-metadata.txt
promotion_out=$workdir/native-build-promote.txt
metadata_file=$workdir/phase20-host-initiated-native-build-store-promotion-xcpng-metadata.txt
import_root=$workdir/import
action_cleanup() {
if [ "$cleanup" -eq 1 ]; then
rm -rf "$workdir"
fi
}
trap action_cleanup EXIT INT TERM
KEEP_WORKDIR=1 WORKDIR="$workdir/inner" METADATA_OUT="$inner_metadata" \
ROOT_AUTHORIZED_KEY_FILE="$root_authorized_key_file" \
ROOT_SSH_PRIVATE_KEY_FILE="$root_ssh_private_key_file" \
OS_TEMPLATE="$os_template" SYSTEM_NAME="$system_name" ROOT_SIZE="$root_size" \
"$repo_root/tests/system/run-phase20-host-initiated-native-build-xcpng.sh"
guest_ip=$(sed -n 's/^guest_ip=//p' "$inner_metadata")
vm_id=$(sed -n 's/^vm_id=//p' "$inner_metadata")
vdi_id=$(sed -n 's/^vdi_id=//p' "$inner_metadata")
closure_path=$(sed -n 's/^closure_path=//p' "$inner_metadata")
closure_base=$(sed -n 's/^closure_base=//p' "$inner_metadata")
run_id=$(sed -n 's/^run_id=//p' "$inner_metadata")
source_store=$(sed -n 's/^source_store=//p' "$inner_metadata")
result_root=$(sed -n 's/^result_root=//p' "$inner_metadata")
promotion_file=$(sed -n 's/^promotion_file=//p' "$inner_metadata")
world_artifact=$(sed -n 's/^world_artifact=//p' "$inner_metadata")
kernel_artifact=$(sed -n 's/^kernel_artifact=//p' "$inner_metadata")
headers_artifact=$(sed -n 's/^headers_artifact=//p' "$inner_metadata")
bootloader_artifact=$(sed -n 's/^bootloader_artifact=//p' "$inner_metadata")
sha_kernel=$(sed -n 's/^sha_kernel=//p' "$inner_metadata")
sha_loader=$(sed -n 's/^sha_loader=//p' "$inner_metadata")
sha_param=$(sed -n 's/^sha_param=//p' "$inner_metadata")
ssh_guest() {
ssh -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=5 \
root@"$guest_ip" "$@"
}
mkdir -p "$import_root"
result_base=$(basename "$result_root")
ssh -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=5 \
root@"$guest_ip" "tar -C '$(dirname "$result_root")' -cf - '$result_base'" | tar -C "$import_root" -xf -
local_result_root=$import_root/$result_base
[ -d "$local_result_root" ] || { echo "failed to import native build result root" >&2; exit 1; }
[ -f "$local_result_root/promotion.scm" ] || { echo "imported result is missing promotion.scm" >&2; exit 1; }
[ -f "$local_result_root/artifacts/world/bin/sh" ] || { echo "imported result is missing world artifact" >&2; exit 1; }
[ -f "$local_result_root/artifacts/kernel/boot/kernel/kernel" ] || { echo "imported result is missing kernel artifact" >&2; exit 1; }
[ -f "$local_result_root/artifacts/headers/usr/include/sys/param.h" ] || { echo "imported result is missing headers artifact" >&2; exit 1; }
[ -f "$local_result_root/artifacts/bootloader/boot/loader.efi" ] || { echo "imported result is missing bootloader artifact" >&2; exit 1; }
action_env() {
sudo env \
HOME="$HOME" \
GUILE_AUTO_COMPILE=0 \
GUIX_SOURCE_DIR="${GUIX_SOURCE_DIR:-$HOME/repos/guix}" \
GUILE_BIN="${GUILE_BIN:-/tmp/guile-freebsd-validate-install/bin/guile}" \
GUILE_EXTRA_PREFIX="${GUILE_EXTRA_PREFIX:-/tmp/guile-gnutls-freebsd-validate-install}" \
SHEPHERD_PREFIX="${SHEPHERD_PREFIX:-/tmp/shepherd-freebsd-validate-install}" \
"$@"
}
action_env "$repo_root/bin/fruix" native-build promote "$local_result_root" --store "$store_dir" >"$promotion_out"
field() {
sed -n "s/^$1=//p" "$promotion_out" | tail -n 1
}
executor_kind=$(field executor_kind)
executor_name=$(field executor_name)
executor_version=$(field executor_version)
result_store=$(field result_store)
result_metadata_file=$(field result_metadata_file)
artifact_store_count=$(field artifact_store_count)
artifact_stores=$(field artifact_stores)
world_store=$(field world_store)
kernel_store=$(field kernel_store)
headers_store=$(field headers_store)
bootloader_store=$(field bootloader_store)
[ "$executor_kind" = ssh-guest ] || { echo "unexpected executor kind: $executor_kind" >&2; exit 1; }
[ "$executor_name" = ssh-guest ] || { echo "unexpected executor name: $executor_name" >&2; exit 1; }
[ "$executor_version" = 1 ] || { echo "unexpected executor version: $executor_version" >&2; exit 1; }
[ "$artifact_store_count" = 4 ] || { echo "unexpected artifact store count: $artifact_store_count" >&2; exit 1; }
case "$result_store" in
/frx/store/*-fruix-native-build-result-*-ssh-guest) : ;;
*) echo "unexpected result store path: $result_store" >&2; exit 1 ;;
esac
case "$world_store" in
/frx/store/*-fruix-native-world-*-ssh-guest) : ;;
*) echo "unexpected world store path: $world_store" >&2; exit 1 ;;
esac
case "$kernel_store" in
/frx/store/*-fruix-native-kernel-*-ssh-guest) : ;;
*) echo "unexpected kernel store path: $kernel_store" >&2; exit 1 ;;
esac
case "$headers_store" in
/frx/store/*-fruix-native-headers-*-ssh-guest) : ;;
*) echo "unexpected headers store path: $headers_store" >&2; exit 1 ;;
esac
case "$bootloader_store" in
/frx/store/*-fruix-native-bootloader-*-ssh-guest) : ;;
*) echo "unexpected bootloader store path: $bootloader_store" >&2; exit 1 ;;
esac
[ -f "$result_metadata_file" ] || { echo "missing result metadata file: $result_metadata_file" >&2; exit 1; }
[ -f "$world_store/.fruix-native-build-object.scm" ] || { echo "missing world store metadata" >&2; exit 1; }
[ -f "$kernel_store/.fruix-native-build-object.scm" ] || { echo "missing kernel store metadata" >&2; exit 1; }
[ -f "$headers_store/.fruix-native-build-object.scm" ] || { echo "missing headers store metadata" >&2; exit 1; }
[ -f "$bootloader_store/.fruix-native-build-object.scm" ] || { echo "missing bootloader store metadata" >&2; exit 1; }
[ -L "$result_store/artifacts/world" ] || { echo "missing promoted world artifact link" >&2; exit 1; }
[ -L "$result_store/artifacts/kernel" ] || { echo "missing promoted kernel artifact link" >&2; exit 1; }
[ -L "$result_store/artifacts/headers" ] || { echo "missing promoted headers artifact link" >&2; exit 1; }
[ -L "$result_store/artifacts/bootloader" ] || { echo "missing promoted bootloader artifact link" >&2; exit 1; }
[ "$(readlink "$result_store/artifacts/world")" = "$world_store" ] || { echo "world artifact link mismatch" >&2; exit 1; }
[ "$(readlink "$result_store/artifacts/kernel")" = "$kernel_store" ] || { echo "kernel artifact link mismatch" >&2; exit 1; }
[ "$(readlink "$result_store/artifacts/headers")" = "$headers_store" ] || { echo "headers artifact link mismatch" >&2; exit 1; }
[ "$(readlink "$result_store/artifacts/bootloader")" = "$bootloader_store" ] || { echo "bootloader artifact link mismatch" >&2; exit 1; }
[ -f "$world_store/bin/sh" ] || { echo "promoted world store missing /bin/sh" >&2; exit 1; }
[ -f "$kernel_store/boot/kernel/kernel" ] || { echo "promoted kernel store missing kernel" >&2; exit 1; }
[ -f "$headers_store/usr/include/sys/param.h" ] || { echo "promoted headers store missing param.h" >&2; exit 1; }
[ -f "$bootloader_store/boot/loader.efi" ] || { echo "promoted bootloader store missing loader.efi" >&2; exit 1; }
promoted_kernel_sha=$(sha256 -q "$kernel_store/boot/kernel/kernel")
promoted_loader_sha=$(sha256 -q "$bootloader_store/boot/loader.efi")
promoted_param_sha=$(sha256 -q "$headers_store/usr/include/sys/param.h")
[ "$promoted_kernel_sha" = "$sha_kernel" ] || { echo "kernel sha mismatch after promotion" >&2; exit 1; }
[ "$promoted_loader_sha" = "$sha_loader" ] || { echo "loader sha mismatch after promotion" >&2; exit 1; }
[ "$promoted_param_sha" = "$sha_param" ] || { echo "param.h sha mismatch after promotion" >&2; exit 1; }
grep -F '(executor-kind . ssh-guest)' "$result_metadata_file" >/dev/null || {
echo "result metadata file is missing ssh-guest executor kind" >&2
exit 1
}
grep -F '(executor-name . "ssh-guest")' "$result_metadata_file" >/dev/null || {
echo "result metadata file is missing ssh-guest executor name" >&2
exit 1
}
grep -F "$source_store" "$result_metadata_file" >/dev/null || {
echo "result metadata file is missing source store provenance" >&2
exit 1
}
grep -F '(artifact-kind . kernel)' "$kernel_store/.fruix-native-build-object.scm" >/dev/null || {
echo "kernel store metadata is missing artifact kind" >&2
exit 1
}
grep -F '(artifact-kind . world)' "$world_store/.fruix-native-build-object.scm" >/dev/null || {
echo "world store metadata is missing artifact kind" >&2
exit 1
}
cat >"$metadata_file" <<EOF
workdir=$workdir
inner_metadata=$inner_metadata
promotion_out=$promotion_out
closure_path=$closure_path
closure_base=$closure_base
vm_id=$vm_id
vdi_id=$vdi_id
guest_ip=$guest_ip
root_size=$root_size
run_id=$run_id
source_store=$source_store
guest_result_root=$result_root
guest_promotion_file=$promotion_file
guest_world_artifact=$world_artifact
guest_kernel_artifact=$kernel_artifact
guest_headers_artifact=$headers_artifact
guest_bootloader_artifact=$bootloader_artifact
local_result_root=$local_result_root
store_dir=$store_dir
executor_kind=$executor_kind
executor_name=$executor_name
executor_version=$executor_version
result_store=$result_store
result_metadata_file=$result_metadata_file
artifact_store_count=$artifact_store_count
artifact_stores=$artifact_stores
world_store=$world_store
kernel_store=$kernel_store
headers_store=$headers_store
bootloader_store=$bootloader_store
sha_kernel=$sha_kernel
sha_loader=$sha_loader
sha_param=$sha_param
promoted_kernel_sha=$promoted_kernel_sha
promoted_loader_sha=$promoted_loader_sha
promoted_param_sha=$promoted_param_sha
boot_backend=xcp-ng-xo-cli
init_mode=shepherd-pid1
host_initiated_native_build_store_promotion=ok
EOF
if [ -n "$metadata_target" ]; then
mkdir -p "$(dirname "$metadata_target")"
cp "$metadata_file" "$metadata_target"
fi
printf 'PASS phase20-host-initiated-native-build-store-promotion-xcpng\n'
printf 'Work directory: %s\n' "$workdir"
printf 'Metadata file: %s\n' "$metadata_file"
if [ -n "$metadata_target" ]; then
printf 'Copied metadata to: %s\n' "$metadata_target"
fi
printf '%s\n' '--- metadata ---'
cat "$metadata_file"

435
tests/system/run-phase20-host-initiated-native-build-xcpng.sh Executable file
View File

@@ -0,0 +1,435 @@
#!/bin/sh
set -eu
repo_root=${PROJECT_ROOT:-$(pwd)}
os_template=${OS_TEMPLATE:-$repo_root/tests/system/phase20-development-operating-system.scm.in}
system_name=${SYSTEM_NAME:-phase20-operating-system}
root_size=${ROOT_SIZE:-20g}
metadata_target=${METADATA_OUT:-}
root_authorized_key_file=${ROOT_AUTHORIZED_KEY_FILE:-$HOME/.ssh/id_ed25519.pub}
root_ssh_private_key_file=${ROOT_SSH_PRIVATE_KEY_FILE:-$HOME/.ssh/id_ed25519}
cleanup=0
if [ -n "${WORKDIR:-}" ]; then
workdir=$WORKDIR
mkdir -p "$workdir"
else
workdir=$(mktemp -d /tmp/fruix-phase20-native-build-xcpng.XXXXXX)
cleanup=1
fi
if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
cleanup=0
fi
inner_metadata=$workdir/phase20-native-build-inner-metadata.txt
metadata_file=$workdir/phase20-host-initiated-native-build-xcpng-metadata.txt
action_cleanup() {
if [ "$cleanup" -eq 1 ]; then
rm -rf "$workdir"
fi
}
trap action_cleanup EXIT INT TERM
KEEP_WORKDIR=1 WORKDIR="$workdir/inner" METADATA_OUT="$inner_metadata" \
ROOT_AUTHORIZED_KEY_FILE="$root_authorized_key_file" \
ROOT_SSH_PRIVATE_KEY_FILE="$root_ssh_private_key_file" \
OS_TEMPLATE="$os_template" SYSTEM_NAME="$system_name" ROOT_SIZE="$root_size" \
"$repo_root/tests/system/run-phase20-development-environment-xcpng.sh"
phase8_metadata=$(sed -n 's/^phase8_metadata=//p' "$inner_metadata")
closure_path=$(sed -n 's/^closure_path=//p' "$inner_metadata")
closure_base=$(sed -n 's/^closure_base=//p' "$inner_metadata")
guest_ip=$(sed -n 's/^guest_ip=//p' "$inner_metadata")
vm_id=$(sed -n 's/^vm_id=//p' "$inner_metadata")
vdi_id=$(sed -n 's/^vdi_id=//p' "$inner_metadata")
shepherd_pid=$(sed -n 's/^shepherd_pid=//p' "$inner_metadata")
sshd_status=$(sed -n 's/^sshd_status=//p' "$inner_metadata")
compat_prefix_shims=$(sed -n 's/^compat_prefix_shims=//p' "$inner_metadata")
guile_module_smoke=$(sed -n 's/^guile_module_smoke=//p' "$inner_metadata")
[ "$shepherd_pid" = 1 ] || { echo "shepherd was not PID 1" >&2; exit 1; }
[ "$sshd_status" = running ] || { echo "sshd is not running" >&2; exit 1; }
[ "$compat_prefix_shims" = absent ] || { echo "compatibility prefix shims reappeared" >&2; exit 1; }
[ "$guile_module_smoke" = ok ] || { echo "guest Guile module smoke failed" >&2; exit 1; }
ssh_guest() {
ssh -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=5 \
root@"$guest_ip" "$@"
}
guest_build_jobs=${GUEST_BUILD_JOBS:-$(ssh_guest 'sysctl -n hw.ncpu')}
case "$guest_build_jobs" in
''|*[!0-9]*)
echo "invalid guest build job count: $guest_build_jobs" >&2
exit 1
;;
esac
native_build_metadata=$(ssh_guest env BUILD_JOBS="$guest_build_jobs" GUEST_IP="$guest_ip" VM_ID="$vm_id" VDI_ID="$vdi_id" sh -s <<'EOF'
set -eu
[ -L /run/current-development ]
[ -L /run/current-build ]
[ -L /usr/include ]
[ "$(readlink /usr/include)" = "/run/current-system/build-profile/usr/include" ]
[ -L /usr/share/mk ]
[ "$(readlink /usr/share/mk)" = "/run/current-system/build-profile/usr/share/mk" ]
guest_host_name=$(hostname)
closure=$(readlink /run/current-system)
source_store=$(sed -n 's/.*"\(\/frx\/store\/[^"]*-freebsd-source-[^"]*\)".*/\1/p' "$closure/metadata/store-layout.scm" | head -n 1)
source_root="$source_store/tree"
build_common='TARGET=amd64 TARGET_ARCH=amd64 KERNCONF=GENERIC __MAKE_CONF=/dev/null SRCCONF=/dev/null SRC_ENV_CONF=/dev/null MK_DEBUG_FILES=no MK_TESTS=no'
install_common="$build_common DB_FROM_SRC=yes"
build_root=/var/tmp/fruix-phase20-native-build
run_id=${FRUIX_HOST_INITIATED_NATIVE_BUILD_ID:-$(date -u +%Y%m%dT%H%M%SZ)}
result_root_base=/var/lib/fruix/native-builds
result_root=$result_root_base/$run_id
logdir=$build_root/logs
status_file=$result_root/status
guest_metadata_file=$result_root/metadata.txt
promotion_file=$result_root/promotion.scm
world_stage=$build_root/stage-world
kernel_stage=$build_root/stage-kernel
headers_stage=$build_root/artifact-headers
bootloader_stage=$build_root/artifact-bootloader
world_artifact=$result_root/artifacts/world
kernel_artifact=$result_root/artifacts/kernel
headers_artifact=$result_root/artifacts/headers
bootloader_artifact=$result_root/artifacts/bootloader
latest_link=$result_root_base/latest
rm -rf "$build_root" "$result_root"
mkdir -p "$logdir" "$result_root" "$world_artifact" "$kernel_artifact/boot" "$headers_artifact/usr" "$bootloader_artifact/boot"
printf 'running\n' > "$status_file"
fail_mark() {
rc=$?
if [ "$rc" -ne 0 ]; then
printf 'failed\n' > "$status_file"
fi
}
trap fail_mark EXIT HUP INT TERM
export MAKEOBJDIRPREFIX="$build_root/obj"
make -j"$BUILD_JOBS" -C "$source_root" TARGET=amd64 TARGET_ARCH=amd64 KERNCONF=GENERIC __MAKE_CONF=/dev/null SRCCONF=/dev/null SRC_ENV_CONF=/dev/null MK_DEBUG_FILES=no MK_TESTS=no buildworld > "$logdir/buildworld.log" 2>&1
make -j"$BUILD_JOBS" -C "$source_root" TARGET=amd64 TARGET_ARCH=amd64 KERNCONF=GENERIC __MAKE_CONF=/dev/null SRCCONF=/dev/null SRC_ENV_CONF=/dev/null MK_DEBUG_FILES=no MK_TESTS=no buildkernel > "$logdir/buildkernel.log" 2>&1
make -C "$source_root" $install_common DESTDIR="$world_stage" installworld > "$logdir/installworld.log" 2>&1
make -C "$source_root" $install_common DESTDIR="$world_stage" distribution > "$logdir/distribution.log" 2>&1
make -C "$source_root" $install_common DESTDIR="$kernel_stage" installkernel > "$logdir/installkernel.log" 2>&1
mkdir -p "$headers_stage/usr" "$bootloader_stage/boot"
cp -a "$world_stage/." "$world_artifact/"
cp -a "$kernel_stage/boot/kernel" "$kernel_artifact/boot/kernel"
cp -a "$world_stage/usr/include" "$headers_stage/usr/include"
mkdir -p "$headers_stage/usr/share"
cp -a "$world_stage/usr/share/mk" "$headers_stage/usr/share/mk"
cp -a "$headers_stage/usr/." "$headers_artifact/usr/"
cp -a "$world_stage/boot/loader" "$bootloader_stage/boot/loader"
cp -a "$world_stage/boot/loader.efi" "$bootloader_stage/boot/loader.efi"
cp -a "$world_stage/boot/device.hints" "$bootloader_stage/boot/device.hints"
cp -a "$world_stage/boot/defaults" "$bootloader_stage/boot/defaults"
cp -a "$world_stage/boot/lua" "$bootloader_stage/boot/lua"
cp -a "$bootloader_stage/boot/." "$bootloader_artifact/boot/"
[ -f "$world_artifact/bin/sh" ]
[ -f "$kernel_artifact/boot/kernel/kernel" ]
[ -f "$headers_artifact/usr/include/sys/param.h" ]
[ -f "$headers_artifact/usr/share/mk/bsd.prog.mk" ]
[ -f "$bootloader_artifact/boot/loader.efi" ]
[ -f "$bootloader_artifact/boot/defaults/loader.conf" ]
[ -f "$bootloader_artifact/boot/lua/loader.lua" ]
sha_kernel=$(sha256 -q "$kernel_artifact/boot/kernel/kernel")
sha_loader=$(sha256 -q "$bootloader_artifact/boot/loader.efi")
sha_param=$(sha256 -q "$headers_artifact/usr/include/sys/param.h")
buildworld_tail=$(tail -n 20 "$logdir/buildworld.log" | tr '\n' ' ')
buildkernel_tail=$(tail -n 20 "$logdir/buildkernel.log" | tr '\n' ' ')
installworld_tail=$(tail -n 20 "$logdir/installworld.log" | tr '\n' ' ')
distribution_tail=$(tail -n 20 "$logdir/distribution.log" | tr '\n' ' ')
installkernel_tail=$(tail -n 20 "$logdir/installkernel.log" | tr '\n' ' ')
root_df=$(df -h / | tail -n 1 | tr -s ' ' | tr '\t' ' ')
build_root_size=$(du -sh "$build_root" | awk '{print $1}')
result_root_size=$(du -sh "$result_root" | awk '{print $1}')
world_stage_size=$(du -sh "$world_stage" | awk '{print $1}')
kernel_stage_size=$(du -sh "$kernel_stage" | awk '{print $1}')
headers_stage_size=$(du -sh "$headers_stage" | awk '{print $1}')
bootloader_stage_size=$(du -sh "$bootloader_stage" | awk '{print $1}')
world_artifact_size=$(du -sh "$world_artifact" | awk '{print $1}')
kernel_artifact_size=$(du -sh "$kernel_artifact" | awk '{print $1}')
headers_artifact_size=$(du -sh "$headers_artifact" | awk '{print $1}')
bootloader_artifact_size=$(du -sh "$bootloader_artifact" | awk '{print $1}')
rm -f "$latest_link"
ln -s "$result_root" "$latest_link"
cat >"$promotion_file" <<EOF2
((native-build-result-version . "1")
(executor . ((kind . ssh-guest)
(name . "ssh-guest")
(version . "1")
(properties . ((transport . "ssh")
(orchestrator . "host")
(guest-host-name . "$guest_host_name")
(guest-ip . "$GUEST_IP")
(vm-id . "$VM_ID")
(vdi-id . "$VDI_ID")))))
(run-id . "$run_id")
(guest-host-name . "$guest_host_name")
(closure-path . "$closure")
(build-profile . "/run/current-system/build-profile")
(freebsd-base . ((name . "default")
(version-label . "15.0-STABLE")
(release . "15.0-STABLE")
(branch . "stable/15")
(source-root . "/usr/src")
(target . "amd64")
(target-arch . "amd64")
(kernconf . "GENERIC")))
(source . ((store-path . "$source_store")
(source-root . "$source_root")))
(build-policy . ((jobs . "$BUILD_JOBS")
(build-common . "$build_common")
(install-common . "$install_common")))
(artifacts . ((world . ((path . "artifacts/world")
(required-file . "bin/sh")))
(kernel . ((path . "artifacts/kernel")
(required-file . "boot/kernel/kernel")
(recorded-sha256 . "$sha_kernel")))
(headers . ((path . "artifacts/headers")
(required-file . "usr/include/sys/param.h")
(recorded-sha256 . "$sha_param")))
(bootloader . ((path . "artifacts/bootloader")
(required-file . "boot/loader.efi")
(recorded-sha256 . "$sha_loader"))))))
EOF2
cat >"$guest_metadata_file" <<EOF2
run_id=$run_id
executor_kind=ssh-guest
executor_name=ssh-guest
executor_version=1
guest_host_name=$guest_host_name
closure_path=$closure
source_store=$source_store
source_root=$source_root
build_jobs=$BUILD_JOBS
build_common=$build_common
install_common=$install_common
build_root=$build_root
result_root=$result_root
logdir=$logdir
status_file=$status_file
metadata_file=$guest_metadata_file
promotion_file=$promotion_file
world_stage=$world_stage
kernel_stage=$kernel_stage
headers_stage=$headers_stage
bootloader_stage=$bootloader_stage
world_artifact=$world_artifact
kernel_artifact=$kernel_artifact
headers_artifact=$headers_artifact
bootloader_artifact=$bootloader_artifact
latest_link=$latest_link
root_df=$root_df
build_root_size=$build_root_size
result_root_size=$result_root_size
world_stage_size=$world_stage_size
kernel_stage_size=$kernel_stage_size
headers_stage_size=$headers_stage_size
bootloader_stage_size=$bootloader_stage_size
world_artifact_size=$world_artifact_size
kernel_artifact_size=$kernel_artifact_size
headers_artifact_size=$headers_artifact_size
bootloader_artifact_size=$bootloader_artifact_size
buildworld_log=$logdir/buildworld.log
buildkernel_log=$logdir/buildkernel.log
installworld_log=$logdir/installworld.log
distribution_log=$logdir/distribution.log
installkernel_log=$logdir/installkernel.log
sha_kernel=$sha_kernel
sha_loader=$sha_loader
sha_param=$sha_param
buildworld_tail=$buildworld_tail
buildkernel_tail=$buildkernel_tail
installworld_tail=$installworld_tail
distribution_tail=$distribution_tail
installkernel_tail=$installkernel_tail
host_initiated_native_build=ok
EOF2
printf 'ok\n' > "$status_file"
cat "$guest_metadata_file"
EOF
)
run_id=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^run_id=//p')
executor_kind=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^executor_kind=//p')
executor_name=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^executor_name=//p')
executor_version=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^executor_version=//p')
guest_host_name=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^guest_host_name=//p')
source_store=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^source_store=//p')
source_root=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^source_root=//p')
build_jobs=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^build_jobs=//p')
build_common=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^build_common=//p')
install_common=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^install_common=//p')
build_root=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^build_root=//p')
result_root=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^result_root=//p')
logdir=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^logdir=//p')
status_file=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^status_file=//p')
guest_metadata_file=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^metadata_file=//p')
promotion_file=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^promotion_file=//p')
buildworld_log=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^buildworld_log=//p')
buildkernel_log=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^buildkernel_log=//p')
installworld_log=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^installworld_log=//p')
distribution_log=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^distribution_log=//p')
installkernel_log=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^installkernel_log=//p')
world_stage=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^world_stage=//p')
kernel_stage=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^kernel_stage=//p')
headers_stage=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^headers_stage=//p')
bootloader_stage=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^bootloader_stage=//p')
world_artifact=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^world_artifact=//p')
kernel_artifact=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^kernel_artifact=//p')
headers_artifact=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^headers_artifact=//p')
bootloader_artifact=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^bootloader_artifact=//p')
latest_link=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^latest_link=//p')
root_df=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^root_df=//p')
build_root_size=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^build_root_size=//p')
result_root_size=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^result_root_size=//p')
world_stage_size=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^world_stage_size=//p')
kernel_stage_size=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^kernel_stage_size=//p')
headers_stage_size=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^headers_stage_size=//p')
bootloader_stage_size=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^bootloader_stage_size=//p')
world_artifact_size=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^world_artifact_size=//p')
kernel_artifact_size=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^kernel_artifact_size=//p')
headers_artifact_size=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^headers_artifact_size=//p')
bootloader_artifact_size=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^bootloader_artifact_size=//p')
sha_kernel=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^sha_kernel=//p')
sha_loader=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^sha_loader=//p')
sha_param=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^sha_param=//p')
buildworld_tail=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^buildworld_tail=//p')
buildkernel_tail=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^buildkernel_tail=//p')
installworld_tail=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^installworld_tail=//p')
distribution_tail=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^distribution_tail=//p')
installkernel_tail=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^installkernel_tail=//p')
status_value=$(ssh_guest "cat '$status_file'")
latest_target=$(ssh_guest "readlink '$latest_link'")
ssh_guest "[ -f '$promotion_file' ]"
ssh_guest "[ -f '$world_artifact/bin/sh' ]"
[ "$executor_kind" = ssh-guest ] || { echo "unexpected executor kind: $executor_kind" >&2; exit 1; }
[ "$executor_name" = ssh-guest ] || { echo "unexpected executor name: $executor_name" >&2; exit 1; }
[ "$executor_version" = 1 ] || { echo "unexpected executor version: $executor_version" >&2; exit 1; }
[ "$status_value" = ok ] || { echo "host-initiated build status is not ok: $status_value" >&2; exit 1; }
[ "$latest_target" = "$result_root" ] || { echo "latest link target mismatch: $latest_target" >&2; exit 1; }
case "$source_store" in
/frx/store/*-freebsd-source-*) : ;;
*) echo "unexpected source store path: $source_store" >&2; exit 1 ;;
esac
case "$source_root" in
/frx/store/*-freebsd-source-*/*) : ;;
*) echo "unexpected source root path: $source_root" >&2; exit 1 ;;
esac
case "$build_root" in
/var/tmp/fruix-phase20-native-build) : ;;
*) echo "unexpected build root: $build_root" >&2; exit 1 ;;
esac
case "$result_root" in
/var/lib/fruix/native-builds/*) : ;;
*) echo "unexpected result root: $result_root" >&2; exit 1 ;;
esac
printf '%s\n' "$sha_kernel" | grep -E '^[0-9a-f]{64}$' >/dev/null || {
echo "invalid kernel sha256: $sha_kernel" >&2
exit 1
}
printf '%s\n' "$sha_loader" | grep -E '^[0-9a-f]{64}$' >/dev/null || {
echo "invalid loader sha256: $sha_loader" >&2
exit 1
}
printf '%s\n' "$sha_param" | grep -E '^[0-9a-f]{64}$' >/dev/null || {
echo "invalid param.h sha256: $sha_param" >&2
exit 1
}
case "$buildworld_tail" in
*'World build completed on'*) : ;;
*) echo "buildworld log does not show completion" >&2; exit 1 ;;
esac
case "$buildkernel_tail" in
*'Kernel(s) GENERIC built in'*) : ;;
*) echo "buildkernel log does not show successful kernel build" >&2; exit 1 ;;
esac
cat >"$metadata_file" <<EOF
workdir=$workdir
inner_metadata=$inner_metadata
phase8_metadata=$phase8_metadata
closure_path=$closure_path
closure_base=$closure_base
vm_id=$vm_id
vdi_id=$vdi_id
guest_ip=$guest_ip
root_size=$root_size
run_id=$run_id
executor_kind=$executor_kind
executor_name=$executor_name
executor_version=$executor_version
guest_host_name=$guest_host_name
build_jobs=$build_jobs
source_store=$source_store
source_root=$source_root
build_common=$build_common
install_common=$install_common
build_root=$build_root
result_root=$result_root
logdir=$logdir
status_file=$status_file
guest_metadata_file=$guest_metadata_file
promotion_file=$promotion_file
buildworld_log=$buildworld_log
buildkernel_log=$buildkernel_log
installworld_log=$installworld_log
distribution_log=$distribution_log
installkernel_log=$installkernel_log
world_stage=$world_stage
kernel_stage=$kernel_stage
headers_stage=$headers_stage
bootloader_stage=$bootloader_stage
world_artifact=$world_artifact
kernel_artifact=$kernel_artifact
headers_artifact=$headers_artifact
bootloader_artifact=$bootloader_artifact
latest_link=$latest_link
latest_target=$latest_target
status_value=$status_value
root_df=$root_df
build_root_size=$build_root_size
result_root_size=$result_root_size
world_stage_size=$world_stage_size
kernel_stage_size=$kernel_stage_size
headers_stage_size=$headers_stage_size
bootloader_stage_size=$bootloader_stage_size
world_artifact_size=$world_artifact_size
kernel_artifact_size=$kernel_artifact_size
headers_artifact_size=$headers_artifact_size
bootloader_artifact_size=$bootloader_artifact_size
sha_kernel=$sha_kernel
sha_loader=$sha_loader
sha_param=$sha_param
buildworld_tail=$buildworld_tail
buildkernel_tail=$buildkernel_tail
installworld_tail=$installworld_tail
distribution_tail=$distribution_tail
installkernel_tail=$installkernel_tail
boot_backend=xcp-ng-xo-cli
init_mode=shepherd-pid1
host_initiated_native_build=ok
EOF
if [ -n "$metadata_target" ]; then
mkdir -p "$(dirname "$metadata_target")"
cp "$metadata_file" "$metadata_target"
fi
printf 'PASS phase20-host-initiated-native-build-xcpng\n'
printf 'Work directory: %s\n' "$workdir"
printf 'Metadata file: %s\n' "$metadata_file"
if [ -n "$metadata_target" ]; then
printf 'Copied metadata to: %s\n' "$metadata_target"
fi
printf '%s\n' '--- metadata ---'
cat "$metadata_file"

248
tests/system/run-phase20-native-build-store-promotion-xcpng.sh Executable file
View File

@@ -0,0 +1,248 @@
#!/bin/sh
set -eu
repo_root=${PROJECT_ROOT:-$(pwd)}
os_template=${OS_TEMPLATE:-$repo_root/tests/system/phase20-development-operating-system.scm.in}
system_name=${SYSTEM_NAME:-phase20-operating-system}
root_size=${ROOT_SIZE:-20g}
store_dir=${STORE_DIR:-/frx/store}
metadata_target=${METADATA_OUT:-}
root_authorized_key_file=${ROOT_AUTHORIZED_KEY_FILE:-$HOME/.ssh/id_ed25519.pub}
root_ssh_private_key_file=${ROOT_SSH_PRIVATE_KEY_FILE:-$HOME/.ssh/id_ed25519}
cleanup=0
if [ -n "${WORKDIR:-}" ]; then
workdir=$WORKDIR
mkdir -p "$workdir"
else
workdir=$(mktemp -d /tmp/fruix-phase20-native-build-store-promotion-xcpng.XXXXXX)
cleanup=1
fi
if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
cleanup=0
fi
inner_metadata=$workdir/phase20-native-build-store-promotion-inner-metadata.txt
existing_inner_metadata=${EXISTING_INNER_METADATA:-}
promotion_out=$workdir/native-build-promote.txt
metadata_file=$workdir/phase20-native-build-store-promotion-xcpng-metadata.txt
import_root=$workdir/import
action_cleanup() {
if [ "$cleanup" -eq 1 ]; then
rm -rf "$workdir"
fi
}
trap action_cleanup EXIT INT TERM
if [ -n "$existing_inner_metadata" ]; then
cp "$existing_inner_metadata" "$inner_metadata"
else
KEEP_WORKDIR=1 WORKDIR="$workdir/inner" METADATA_OUT="$inner_metadata" \
ROOT_AUTHORIZED_KEY_FILE="$root_authorized_key_file" \
ROOT_SSH_PRIVATE_KEY_FILE="$root_ssh_private_key_file" \
OS_TEMPLATE="$os_template" SYSTEM_NAME="$system_name" ROOT_SIZE="$root_size" \
"$repo_root/tests/system/run-phase20-self-hosted-native-build-xcpng.sh"
fi
guest_ip=$(sed -n 's/^guest_ip=//p' "$inner_metadata")
vm_id=$(sed -n 's/^vm_id=//p' "$inner_metadata")
vdi_id=$(sed -n 's/^vdi_id=//p' "$inner_metadata")
closure_path=$(sed -n 's/^closure_path=//p' "$inner_metadata")
closure_base=$(sed -n 's/^closure_base=//p' "$inner_metadata")
run_id=$(sed -n 's/^run_id=//p' "$inner_metadata")
source_store=$(sed -n 's/^source_store=//p' "$inner_metadata")
result_root=$(sed -n 's/^result_root=//p' "$inner_metadata")
promotion_file=$(sed -n 's/^promotion_file=//p' "$inner_metadata")
world_artifact=$(sed -n 's/^world_artifact=//p' "$inner_metadata")
kernel_artifact=$(sed -n 's/^kernel_artifact=//p' "$inner_metadata")
headers_artifact=$(sed -n 's/^headers_artifact=//p' "$inner_metadata")
bootloader_artifact=$(sed -n 's/^bootloader_artifact=//p' "$inner_metadata")
sha_kernel=$(sed -n 's/^sha_kernel=//p' "$inner_metadata")
sha_loader=$(sed -n 's/^sha_loader=//p' "$inner_metadata")
sha_param=$(sed -n 's/^sha_param=//p' "$inner_metadata")
ssh_guest() {
ssh -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=5 \
root@"$guest_ip" "$@"
}
mkdir -p "$import_root"
result_base=$(basename "$result_root")
ssh -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=5 \
root@"$guest_ip" "tar -C '$(dirname "$result_root")' -cf - '$result_base'" | tar -C "$import_root" -xf -
local_result_root=$import_root/$result_base
[ -d "$local_result_root" ] || { echo "failed to import native build result root" >&2; exit 1; }
[ -f "$local_result_root/promotion.scm" ] || { echo "imported result is missing promotion.scm" >&2; exit 1; }
[ -f "$local_result_root/artifacts/world/bin/sh" ] || { echo "imported result is missing world artifact" >&2; exit 1; }
[ -f "$local_result_root/artifacts/kernel/boot/kernel/kernel" ] || { echo "imported result is missing kernel artifact" >&2; exit 1; }
[ -f "$local_result_root/artifacts/headers/usr/include/sys/param.h" ] || { echo "imported result is missing headers artifact" >&2; exit 1; }
[ -f "$local_result_root/artifacts/bootloader/boot/loader.efi" ] || { echo "imported result is missing bootloader artifact" >&2; exit 1; }
action_env() {
sudo env \
HOME="$HOME" \
GUILE_AUTO_COMPILE=0 \
GUIX_SOURCE_DIR="${GUIX_SOURCE_DIR:-$HOME/repos/guix}" \
GUILE_BIN="${GUILE_BIN:-/tmp/guile-freebsd-validate-install/bin/guile}" \
GUILE_EXTRA_PREFIX="${GUILE_EXTRA_PREFIX:-/tmp/guile-gnutls-freebsd-validate-install}" \
SHEPHERD_PREFIX="${SHEPHERD_PREFIX:-/tmp/shepherd-freebsd-validate-install}" \
"$@"
}
action_env "$repo_root/bin/fruix" native-build promote "$local_result_root" --store "$store_dir" >"$promotion_out"
field() {
sed -n "s/^$1=//p" "$promotion_out" | tail -n 1
}
executor_kind=$(field executor_kind)
executor_name=$(field executor_name)
executor_version=$(field executor_version)
result_store=$(field result_store)
result_metadata_file=$(field result_metadata_file)
artifact_store_count=$(field artifact_store_count)
artifact_stores=$(field artifact_stores)
world_store=$(field world_store)
kernel_store=$(field kernel_store)
headers_store=$(field headers_store)
bootloader_store=$(field bootloader_store)
[ "$executor_kind" = self-hosted ] || { echo "unexpected executor kind: $executor_kind" >&2; exit 1; }
[ "$executor_name" = guest-self-hosted ] || { echo "unexpected executor name: $executor_name" >&2; exit 1; }
[ "$executor_version" = 5 ] || { echo "unexpected executor version: $executor_version" >&2; exit 1; }
[ "$artifact_store_count" = 4 ] || { echo "unexpected artifact store count: $artifact_store_count" >&2; exit 1; }
case "$result_store" in
/frx/store/*-fruix-native-build-result-*-guest-self-hosted) : ;;
*) echo "unexpected result store path: $result_store" >&2; exit 1 ;;
esac
case "$world_store" in
/frx/store/*-fruix-native-world-*-guest-self-hosted) : ;;
*) echo "unexpected world store path: $world_store" >&2; exit 1 ;;
esac
case "$kernel_store" in
/frx/store/*-fruix-native-kernel-*-guest-self-hosted) : ;;
*) echo "unexpected kernel store path: $kernel_store" >&2; exit 1 ;;
esac
case "$headers_store" in
/frx/store/*-fruix-native-headers-*-guest-self-hosted) : ;;
*) echo "unexpected headers store path: $headers_store" >&2; exit 1 ;;
esac
case "$bootloader_store" in
/frx/store/*-fruix-native-bootloader-*-guest-self-hosted) : ;;
*) echo "unexpected bootloader store path: $bootloader_store" >&2; exit 1 ;;
esac
[ -f "$result_metadata_file" ] || { echo "missing result metadata file: $result_metadata_file" >&2; exit 1; }
[ -f "$world_store/.fruix-native-build-object.scm" ] || { echo "missing world store metadata" >&2; exit 1; }
[ -f "$kernel_store/.fruix-native-build-object.scm" ] || { echo "missing kernel store metadata" >&2; exit 1; }
[ -f "$headers_store/.fruix-native-build-object.scm" ] || { echo "missing headers store metadata" >&2; exit 1; }
[ -f "$bootloader_store/.fruix-native-build-object.scm" ] || { echo "missing bootloader store metadata" >&2; exit 1; }
[ -L "$result_store/artifacts/world" ] || { echo "missing promoted world artifact link" >&2; exit 1; }
[ -L "$result_store/artifacts/kernel" ] || { echo "missing promoted kernel artifact link" >&2; exit 1; }
[ -L "$result_store/artifacts/headers" ] || { echo "missing promoted headers artifact link" >&2; exit 1; }
[ -L "$result_store/artifacts/bootloader" ] || { echo "missing promoted bootloader artifact link" >&2; exit 1; }
[ "$(readlink "$result_store/artifacts/world")" = "$world_store" ] || { echo "world artifact link mismatch" >&2; exit 1; }
[ "$(readlink "$result_store/artifacts/kernel")" = "$kernel_store" ] || { echo "kernel artifact link mismatch" >&2; exit 1; }
[ "$(readlink "$result_store/artifacts/headers")" = "$headers_store" ] || { echo "headers artifact link mismatch" >&2; exit 1; }
[ "$(readlink "$result_store/artifacts/bootloader")" = "$bootloader_store" ] || { echo "bootloader artifact link mismatch" >&2; exit 1; }
[ -f "$world_store/bin/sh" ] || { echo "promoted world store missing /bin/sh" >&2; exit 1; }
[ -f "$kernel_store/boot/kernel/kernel" ] || { echo "promoted kernel store missing kernel" >&2; exit 1; }
[ -f "$headers_store/usr/include/sys/param.h" ] || { echo "promoted headers store missing param.h" >&2; exit 1; }
[ -f "$bootloader_store/boot/loader.efi" ] || { echo "promoted bootloader store missing loader.efi" >&2; exit 1; }
promoted_kernel_sha=$(sha256 -q "$kernel_store/boot/kernel/kernel")
promoted_loader_sha=$(sha256 -q "$bootloader_store/boot/loader.efi")
promoted_param_sha=$(sha256 -q "$headers_store/usr/include/sys/param.h")
[ "$promoted_kernel_sha" = "$sha_kernel" ] || { echo "kernel sha mismatch after promotion" >&2; exit 1; }
[ "$promoted_loader_sha" = "$sha_loader" ] || { echo "loader sha mismatch after promotion" >&2; exit 1; }
[ "$promoted_param_sha" = "$sha_param" ] || { echo "param.h sha mismatch after promotion" >&2; exit 1; }
grep -F '(executor-kind . self-hosted)' "$result_metadata_file" >/dev/null || {
echo "result metadata file is missing self-hosted executor kind" >&2
exit 1
}
grep -F '(executor-name . "guest-self-hosted")' "$result_metadata_file" >/dev/null || {
echo "result metadata file is missing guest-self-hosted executor name" >&2
exit 1
}
grep -F "$source_store" "$result_metadata_file" >/dev/null || {
echo "result metadata file is missing source store provenance" >&2
exit 1
}
grep -F '(build-profile . "/run/current-system/build-profile")' "$result_metadata_file" >/dev/null || {
echo "result metadata file is missing build-profile provenance" >&2
exit 1
}
grep -F '(artifact-kind . kernel)' "$kernel_store/.fruix-native-build-object.scm" >/dev/null || {
echo "kernel store metadata is missing artifact kind" >&2
exit 1
}
grep -F '(artifact-kind . world)' "$world_store/.fruix-native-build-object.scm" >/dev/null || {
echo "world store metadata is missing artifact kind" >&2
exit 1
}
cat >"$metadata_file" <<EOF
workdir=$workdir
inner_metadata=$inner_metadata
promotion_out=$promotion_out
closure_path=$closure_path
closure_base=$closure_base
vm_id=$vm_id
vdi_id=$vdi_id
guest_ip=$guest_ip
root_size=$root_size
run_id=$run_id
source_store=$source_store
guest_result_root=$result_root
guest_promotion_file=$promotion_file
guest_world_artifact=$world_artifact
guest_kernel_artifact=$kernel_artifact
guest_headers_artifact=$headers_artifact
guest_bootloader_artifact=$bootloader_artifact
local_result_root=$local_result_root
store_dir=$store_dir
executor_kind=$executor_kind
executor_name=$executor_name
executor_version=$executor_version
result_store=$result_store
result_metadata_file=$result_metadata_file
artifact_store_count=$artifact_store_count
artifact_stores=$artifact_stores
world_store=$world_store
kernel_store=$kernel_store
headers_store=$headers_store
bootloader_store=$bootloader_store
sha_kernel=$sha_kernel
sha_loader=$sha_loader
sha_param=$sha_param
promoted_kernel_sha=$promoted_kernel_sha
promoted_loader_sha=$promoted_loader_sha
promoted_param_sha=$promoted_param_sha
boot_backend=xcp-ng-xo-cli
init_mode=shepherd-pid1
native_build_store_promotion=ok
EOF
if [ -n "$metadata_target" ]; then
mkdir -p "$(dirname "$metadata_target")"
cp "$metadata_file" "$metadata_target"
fi
printf 'PASS phase20-native-build-store-promotion-xcpng\n'
printf 'Work directory: %s\n' "$workdir"
printf 'Metadata file: %s\n' "$metadata_file"
if [ -n "$metadata_target" ]; then
printf 'Copied metadata to: %s\n' "$metadata_target"
fi
printf '%s\n' '--- metadata ---'
cat "$metadata_file"

227
tests/system/run-phase20-promoted-native-base-declaration-xcpng.sh Executable file
View File

@@ -0,0 +1,227 @@
#!/bin/sh
set -eu
repo_root=${PROJECT_ROOT:-$(pwd)}
os_template=${OS_TEMPLATE:-$repo_root/tests/system/phase20-promoted-native-base-operating-system.scm.in}
system_name=${SYSTEM_NAME:-phase20-promoted-native-base-operating-system}
root_size=${ROOT_SIZE:-12g}
result_store=${RESULT_STORE:-}
metadata_target=${METADATA_OUT:-}
root_authorized_key_file=${ROOT_AUTHORIZED_KEY_FILE:-$HOME/.ssh/id_ed25519.pub}
root_ssh_private_key_file=${ROOT_SSH_PRIVATE_KEY_FILE:-$HOME/.ssh/id_ed25519}
cleanup=0
if [ -n "${WORKDIR:-}" ]; then
workdir=$WORKDIR
mkdir -p "$workdir"
else
workdir=$(mktemp -d /tmp/fruix-phase20-promoted-native-base-xcpng.XXXXXX)
cleanup=1
fi
if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
cleanup=0
fi
promotion_metadata=$workdir/phase20-promoted-native-base-promotion-metadata.txt
prepared_template=$workdir/phase20-promoted-native-base-operating-system.scm.in
inner_metadata=$workdir/phase20-promoted-native-base-inner-metadata.txt
metadata_file=$workdir/phase20-promoted-native-base-declaration-xcpng-metadata.txt
cleanup_workdir() {
if [ "$cleanup" -eq 1 ]; then
rm -rf "$workdir"
fi
}
trap cleanup_workdir EXIT INT TERM
if [ -z "$result_store" ]; then
KEEP_WORKDIR=1 WORKDIR="$workdir/promotion" METADATA_OUT="$promotion_metadata" \
ROOT_AUTHORIZED_KEY_FILE="$root_authorized_key_file" \
ROOT_SSH_PRIVATE_KEY_FILE="$root_ssh_private_key_file" \
ROOT_SIZE=20g \
"$repo_root/tests/system/run-phase20-host-initiated-native-build-store-promotion-xcpng.sh"
result_store=$(sed -n 's/^result_store=//p' "$promotion_metadata")
fi
[ -n "$result_store" ] || { echo "missing promoted result store" >&2; exit 1; }
[ -d "$result_store" ] || { echo "promoted result store does not exist: $result_store" >&2; exit 1; }
[ -f "$result_store/.fruix-native-build-result.scm" ] || {
echo "promoted result store is missing .fruix-native-build-result.scm" >&2
exit 1
}
[ -L "$result_store/artifacts/world" ] || { echo "promoted result store is missing world artifact link" >&2; exit 1; }
[ -L "$result_store/artifacts/kernel" ] || { echo "promoted result store is missing kernel artifact link" >&2; exit 1; }
[ -L "$result_store/artifacts/headers" ] || { echo "promoted result store is missing headers artifact link" >&2; exit 1; }
[ -L "$result_store/artifacts/bootloader" ] || { echo "promoted result store is missing bootloader artifact link" >&2; exit 1; }
world_store=$(readlink "$result_store/artifacts/world")
kernel_store=$(readlink "$result_store/artifacts/kernel")
headers_store=$(readlink "$result_store/artifacts/headers")
bootloader_store=$(readlink "$result_store/artifacts/bootloader")
result_metadata_file=$result_store/.fruix-native-build-result.scm
executor_kind=$(grep -o '(executor-kind \. [^)]*)' "$result_metadata_file" | head -n 1 | sed 's/(executor-kind \. //; s/)$//')
executor_name=$(grep -o '(executor-name \. "[^"]*")' "$result_metadata_file" | head -n 1 | sed 's/(executor-name \. "//; s/")$//')
executor_version=$(grep -o '(executor-version \. "[^"]*")' "$result_metadata_file" | head -n 1 | sed 's/(executor-version \. "//; s/")$//')
[ -f "$world_store/bin/sh" ] || { echo "promoted world store is missing /bin/sh" >&2; exit 1; }
[ -f "$kernel_store/boot/kernel/kernel" ] || { echo "promoted kernel store is missing kernel" >&2; exit 1; }
[ -f "$headers_store/usr/include/sys/param.h" ] || { echo "promoted headers store is missing param.h" >&2; exit 1; }
[ -f "$bootloader_store/boot/loader.efi" ] || { echo "promoted bootloader store is missing loader.efi" >&2; exit 1; }
sed "s|__PROMOTED_RESULT_STORE__|$result_store|g" "$os_template" > "$prepared_template"
action_metadata=${promotion_metadata:-}
ROOT_SIZE="$root_size" KEEP_WORKDIR=1 WORKDIR="$workdir/boot" METADATA_OUT="$inner_metadata" \
ROOT_AUTHORIZED_KEY_FILE="$root_authorized_key_file" \
ROOT_SSH_PRIVATE_KEY_FILE="$root_ssh_private_key_file" \
OS_TEMPLATE="$prepared_template" SYSTEM_NAME="$system_name" \
"$repo_root/tests/system/run-phase11-shepherd-pid1-xcpng.sh"
phase8_metadata=$(sed -n 's/^phase8_metadata=//p' "$inner_metadata")
closure_path=$(sed -n 's/^closure_path=//p' "$inner_metadata")
closure_base=$(sed -n 's/^closure_base=//p' "$inner_metadata")
guest_ip=$(sed -n 's/^guest_ip=//p' "$inner_metadata")
vm_id=$(sed -n 's/^vm_id=//p' "$inner_metadata")
vdi_id=$(sed -n 's/^vdi_id=//p' "$inner_metadata")
shepherd_pid=$(sed -n 's/^shepherd_pid=//p' "$inner_metadata")
sshd_status=$(sed -n 's/^sshd_status=//p' "$inner_metadata")
compat_prefix_shims=$(sed -n 's/^compat_prefix_shims=//p' "$inner_metadata")
guile_module_smoke=$(sed -n 's/^guile_module_smoke=//p' "$inner_metadata")
activate_log=$(sed -n 's/^activate_log=//p' "$inner_metadata")
promoted_result_file=$closure_path/metadata/promoted-native-build-result.scm
store_layout_file=$closure_path/metadata/store-layout.scm
[ "$shepherd_pid" = 1 ] || { echo "shepherd was not PID 1" >&2; exit 1; }
[ "$sshd_status" = running ] || { echo "sshd is not running" >&2; exit 1; }
[ "$compat_prefix_shims" = absent ] || { echo "compatibility prefix shims reappeared" >&2; exit 1; }
[ "$guile_module_smoke" = ok ] || { echo "guest Guile module smoke failed" >&2; exit 1; }
case "$activate_log" in
*fruix-activate:done*) : ;;
*) echo "activation log does not show success" >&2; exit 1 ;;
esac
[ -f "$promoted_result_file" ] || { echo "closure is missing promoted native build result metadata" >&2; exit 1; }
[ -f "$store_layout_file" ] || { echo "closure is missing store layout metadata" >&2; exit 1; }
grep -F "$result_store" "$promoted_result_file" >/dev/null || {
echo "closure promoted result metadata does not reference the selected result store" >&2
exit 1
}
grep -F "$result_store" "$closure_path/.references" >/dev/null || {
echo "closure references do not retain the promoted result store" >&2
exit 1
}
grep -F "$kernel_store" "$promoted_result_file" >/dev/null || {
echo "closure promoted result metadata does not reference the promoted kernel store" >&2
exit 1
}
grep -F "$bootloader_store" "$promoted_result_file" >/dev/null || {
echo "closure promoted result metadata does not reference the promoted bootloader store" >&2
exit 1
}
grep -F "$result_store" "$store_layout_file" >/dev/null || {
echo "store layout metadata does not record the promoted result store" >&2
exit 1
}
kernel_link=$(readlink "$closure_path/boot/kernel/kernel")
bootloader_link=$(readlink "$closure_path/boot/loader.efi")
[ "$kernel_link" = "$kernel_store/boot/kernel/kernel" ] || {
echo "closure kernel link does not target the promoted kernel store" >&2
exit 1
}
[ "$bootloader_link" = "$bootloader_store/boot/loader.efi" ] || {
echo "closure bootloader link does not target the promoted bootloader store" >&2
exit 1
}
ssh_guest() {
ssh -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=5 \
root@"$guest_ip" "$@"
}
guest_promoted_metadata=$(ssh -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=5 \
root@"$guest_ip" 'sh -s' <<EOF
set -eu
[ -f /run/current-system/metadata/promoted-native-build-result.scm ]
grep -F '$result_store' /run/current-system/metadata/promoted-native-build-result.scm >/dev/null
grep -F '$result_store' /run/current-system/.references >/dev/null
kernel_link=$(readlink /run/current-system/boot/kernel/kernel)
bootloader_link=$(readlink /run/current-system/boot/loader.efi)
[ "$kernel_link" = "$kernel_store/boot/kernel/kernel" ]
[ "$bootloader_link" = "$bootloader_store/boot/loader.efi" ]
[ -x /bin/sh ]
[ -x /usr/sbin/sshd ]
printf 'kernel_link=%s\n' "$kernel_link"
printf 'bootloader_link=%s\n' "$bootloader_link"
printf 'uname=%s\n' "$(uname -sr)"
printf 'promoted_result_store=%s\n' '$result_store'
EOF
)
guest_kernel_link=$(printf '%s\n' "$guest_promoted_metadata" | sed -n 's/^kernel_link=//p')
guest_bootloader_link=$(printf '%s\n' "$guest_promoted_metadata" | sed -n 's/^bootloader_link=//p')
guest_uname=$(printf '%s\n' "$guest_promoted_metadata" | sed -n 's/^uname=//p')
[ "$guest_kernel_link" = "$kernel_store/boot/kernel/kernel" ] || {
echo "guest kernel link does not target the promoted kernel store" >&2
exit 1
}
[ "$guest_bootloader_link" = "$bootloader_store/boot/loader.efi" ] || {
echo "guest bootloader link does not target the promoted bootloader store" >&2
exit 1
}
cat >"$metadata_file" <<EOF
workdir=$workdir
promotion_metadata=$promotion_metadata
inner_metadata=$inner_metadata
phase8_metadata=$phase8_metadata
closure_path=$closure_path
closure_base=$closure_base
vm_id=$vm_id
vdi_id=$vdi_id
guest_ip=$guest_ip
root_size=$root_size
result_store=$result_store
result_metadata_file=$result_metadata_file
executor_kind=$executor_kind
executor_name=$executor_name
executor_version=$executor_version
world_store=$world_store
kernel_store=$kernel_store
headers_store=$headers_store
bootloader_store=$bootloader_store
promoted_result_file=$promoted_result_file
store_layout_file=$store_layout_file
kernel_link=$kernel_link
bootloader_link=$bootloader_link
guest_kernel_link=$guest_kernel_link
guest_bootloader_link=$guest_bootloader_link
guest_uname=$guest_uname
boot_backend=xcp-ng-xo-cli
init_mode=shepherd-pid1
promoted_native_base_declaration=ok
EOF
if [ -n "$metadata_target" ]; then
mkdir -p "$(dirname "$metadata_target")"
cp "$metadata_file" "$metadata_target"
fi
printf 'PASS phase20-promoted-native-base-declaration-xcpng\n'
printf 'Work directory: %s\n' "$workdir"
printf 'Metadata file: %s\n' "$metadata_file"
if [ -n "$metadata_target" ]; then
printf 'Copied metadata to: %s\n' "$metadata_target"
fi
printf '%s\n' '--- metadata ---'
cat "$metadata_file"

266
tests/system/run-phase20-self-hosted-native-build-xcpng.sh Executable file
View File

@@ -0,0 +1,266 @@
#!/bin/sh
set -eu
repo_root=${PROJECT_ROOT:-$(pwd)}
os_template=${OS_TEMPLATE:-$repo_root/tests/system/phase20-development-operating-system.scm.in}
system_name=${SYSTEM_NAME:-phase20-operating-system}
root_size=${ROOT_SIZE:-20g}
metadata_target=${METADATA_OUT:-}
root_authorized_key_file=${ROOT_AUTHORIZED_KEY_FILE:-$HOME/.ssh/id_ed25519.pub}
root_ssh_private_key_file=${ROOT_SSH_PRIVATE_KEY_FILE:-$HOME/.ssh/id_ed25519}
cleanup=0
if [ -n "${WORKDIR:-}" ]; then
workdir=$WORKDIR
mkdir -p "$workdir"
else
workdir=$(mktemp -d /tmp/fruix-phase20-self-hosted-native-build-xcpng.XXXXXX)
cleanup=1
fi
if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
cleanup=0
fi
inner_metadata=$workdir/phase20-self-hosted-inner-metadata.txt
metadata_file=$workdir/phase20-self-hosted-native-build-xcpng-metadata.txt
action_cleanup() {
if [ "$cleanup" -eq 1 ]; then
rm -rf "$workdir"
fi
}
trap action_cleanup EXIT INT TERM
KEEP_WORKDIR=1 WORKDIR="$workdir/inner" METADATA_OUT="$inner_metadata" \
ROOT_AUTHORIZED_KEY_FILE="$root_authorized_key_file" \
ROOT_SSH_PRIVATE_KEY_FILE="$root_ssh_private_key_file" \
OS_TEMPLATE="$os_template" SYSTEM_NAME="$system_name" ROOT_SIZE="$root_size" \
"$repo_root/tests/system/run-phase20-development-environment-xcpng.sh"
phase8_metadata=$(sed -n 's/^phase8_metadata=//p' "$inner_metadata")
closure_path=$(sed -n 's/^closure_path=//p' "$inner_metadata")
closure_base=$(sed -n 's/^closure_base=//p' "$inner_metadata")
guest_ip=$(sed -n 's/^guest_ip=//p' "$inner_metadata")
vm_id=$(sed -n 's/^vm_id=//p' "$inner_metadata")
vdi_id=$(sed -n 's/^vdi_id=//p' "$inner_metadata")
shepherd_pid=$(sed -n 's/^shepherd_pid=//p' "$inner_metadata")
sshd_status=$(sed -n 's/^sshd_status=//p' "$inner_metadata")
compat_prefix_shims=$(sed -n 's/^compat_prefix_shims=//p' "$inner_metadata")
guile_module_smoke=$(sed -n 's/^guile_module_smoke=//p' "$inner_metadata")
[ "$shepherd_pid" = 1 ] || { echo "shepherd was not PID 1" >&2; exit 1; }
[ "$sshd_status" = running ] || { echo "sshd is not running" >&2; exit 1; }
[ "$compat_prefix_shims" = absent ] || { echo "compatibility prefix shims reappeared" >&2; exit 1; }
[ "$guile_module_smoke" = ok ] || { echo "guest Guile module smoke failed" >&2; exit 1; }
ssh_guest() {
ssh -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=5 \
root@"$guest_ip" "$@"
}
guest_build_jobs=${GUEST_BUILD_JOBS:-$(ssh_guest 'sysctl -n hw.ncpu')}
case "$guest_build_jobs" in
''|*[!0-9]*)
echo "invalid guest build job count: $guest_build_jobs" >&2
exit 1
;;
esac
ssh_guest '[ -x /usr/local/bin/fruix-build-environment ]'
ssh_guest '[ -x /usr/local/bin/fruix-self-hosted-native-build ]'
ssh_guest '[ -L /run/current-build ]'
ssh_guest '[ -L /usr/include ]'
ssh_guest '[ -L /usr/share/mk ]'
self_hosted_metadata=$(ssh_guest env FRUIX_SELF_HOSTED_NATIVE_BUILD_JOBS="$guest_build_jobs" /usr/local/bin/fruix-self-hosted-native-build)
run_id=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^run_id=//p')
helper_version=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^helper_version=//p')
executor_kind=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^executor_kind=//p')
executor_name=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^executor_name=//p')
executor_version=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^executor_version=//p')
build_profile=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^build_profile=//p')
source_store=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^source_store=//p')
source_root=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^source_root=//p')
build_jobs=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^build_jobs=//p')
build_common=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^build_common=//p')
install_common=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^install_common=//p')
build_root=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^build_root=//p')
result_root=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^result_root=//p')
logdir=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^logdir=//p')
status_file=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^status_file=//p')
guest_metadata_file=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^metadata_file=//p')
promotion_file=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^promotion_file=//p')
world_stage=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^world_stage=//p')
kernel_stage=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^kernel_stage=//p')
world_artifact=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^world_artifact=//p')
kernel_artifact=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^kernel_artifact=//p')
headers_artifact=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^headers_artifact=//p')
bootloader_artifact=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^bootloader_artifact=//p')
latest_link=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^latest_link=//p')
root_df=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^root_df=//p')
build_root_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^build_root_size=//p')
result_root_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^result_root_size=//p')
world_artifact_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^world_artifact_size=//p')
kernel_artifact_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^kernel_artifact_size=//p')
headers_artifact_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^headers_artifact_size=//p')
bootloader_artifact_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^bootloader_artifact_size=//p')
sha_kernel=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^sha_kernel=//p')
sha_loader=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^sha_loader=//p')
sha_param=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^sha_param=//p')
buildworld_tail=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^buildworld_tail=//p')
buildkernel_tail=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^buildkernel_tail=//p')
installworld_tail=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^installworld_tail=//p')
distribution_tail=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^distribution_tail=//p')
installkernel_tail=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^installkernel_tail=//p')
self_hosted_native_build=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^self_hosted_native_build=//p')
status_value=$(ssh_guest "cat '$status_file'")
latest_target=$(ssh_guest "readlink '$latest_link'")
ssh_guest "[ -f '$promotion_file' ]"
ssh_guest "[ -f '$world_artifact/bin/sh' ]"
[ "$helper_version" = 5 ] || { echo "unexpected helper version: $helper_version" >&2; exit 1; }
[ "$executor_kind" = self-hosted ] || { echo "unexpected executor kind: $executor_kind" >&2; exit 1; }
[ "$executor_name" = guest-self-hosted ] || { echo "unexpected executor name: $executor_name" >&2; exit 1; }
[ "$executor_version" = 5 ] || { echo "unexpected executor version: $executor_version" >&2; exit 1; }
[ "$build_profile" = /run/current-system/build-profile ] || { echo "unexpected build profile: $build_profile" >&2; exit 1; }
[ "$build_jobs" = "$guest_build_jobs" ] || { echo "unexpected build job count: $build_jobs" >&2; exit 1; }
[ "$status_value" = ok ] || { echo "self-hosted build status is not ok: $status_value" >&2; exit 1; }
[ "$latest_target" = "$result_root" ] || { echo "latest link target mismatch: $latest_target" >&2; exit 1; }
[ "$self_hosted_native_build" = ok ] || { echo "self-hosted build marker missing" >&2; exit 1; }
case "$source_store" in
/frx/store/*-freebsd-source-*) : ;;
*) echo "unexpected source store path: $source_store" >&2; exit 1 ;;
esac
case "$source_root" in
/frx/store/*-freebsd-source-*/*) : ;;
*) echo "unexpected source root path: $source_root" >&2; exit 1 ;;
esac
case "$build_root" in
/var/tmp/fruix-self-hosted-native-builds/*) : ;;
*) echo "unexpected build root: $build_root" >&2; exit 1 ;;
esac
case "$result_root" in
/var/lib/fruix/native-builds/*) : ;;
*) echo "unexpected result root: $result_root" >&2; exit 1 ;;
esac
case "$latest_link" in
/var/lib/fruix/native-builds/latest) : ;;
*) echo "unexpected latest link path: $latest_link" >&2; exit 1 ;;
esac
printf '%s\n' "$run_id" | grep -E '^[0-9]{8}T[0-9]{6}Z$' >/dev/null || {
echo "unexpected run id: $run_id" >&2
exit 1
}
printf '%s\n' "$sha_kernel" | grep -E '^[0-9a-f]{64}$' >/dev/null || {
echo "invalid kernel sha256: $sha_kernel" >&2
exit 1
}
printf '%s\n' "$sha_loader" | grep -E '^[0-9a-f]{64}$' >/dev/null || {
echo "invalid loader sha256: $sha_loader" >&2
exit 1
}
printf '%s\n' "$sha_param" | grep -E '^[0-9a-f]{64}$' >/dev/null || {
echo "invalid param.h sha256: $sha_param" >&2
exit 1
}
case "$buildworld_tail" in
*'World build completed on'*) : ;;
*) echo "buildworld log does not show completion" >&2; exit 1 ;;
esac
case "$buildkernel_tail" in
*'Kernel(s) GENERIC built in'*) : ;;
*) echo "buildkernel log does not show successful kernel build" >&2; exit 1 ;;
esac
case "$installworld_tail" in
*'Install world completed in'*) : ;;
*) echo "installworld log does not show successful completion" >&2; exit 1 ;;
esac
case "$installkernel_tail" in
*'Install kernel(s) GENERIC completed in'*) : ;;
*) echo "installkernel log does not show successful completion" >&2; exit 1 ;;
esac
printf '%s\n' "$build_common" | grep -F 'TARGET=amd64 TARGET_ARCH=amd64 KERNCONF=GENERIC' >/dev/null || {
echo "unexpected build_common flags: $build_common" >&2
exit 1
}
printf '%s\n' "$install_common" | grep -F 'DB_FROM_SRC=yes' >/dev/null || {
echo "install_common is missing DB_FROM_SRC=yes" >&2
exit 1
}
cat >"$metadata_file" <<EOF
workdir=$workdir
inner_metadata=$inner_metadata
phase8_metadata=$phase8_metadata
closure_path=$closure_path
closure_base=$closure_base
vm_id=$vm_id
vdi_id=$vdi_id
guest_ip=$guest_ip
root_size=$root_size
run_id=$run_id
helper_version=$helper_version
executor_kind=$executor_kind
executor_name=$executor_name
executor_version=$executor_version
build_jobs=$build_jobs
build_profile=$build_profile
source_store=$source_store
source_root=$source_root
build_common=$build_common
install_common=$install_common
build_root=$build_root
result_root=$result_root
logdir=$logdir
status_file=$status_file
guest_metadata_file=$guest_metadata_file
promotion_file=$promotion_file
world_stage=$world_stage
kernel_stage=$kernel_stage
world_artifact=$world_artifact
kernel_artifact=$kernel_artifact
headers_artifact=$headers_artifact
bootloader_artifact=$bootloader_artifact
latest_link=$latest_link
latest_target=$latest_target
status_value=$status_value
root_df=$root_df
build_root_size=$build_root_size
result_root_size=$result_root_size
world_artifact_size=$world_artifact_size
kernel_artifact_size=$kernel_artifact_size
headers_artifact_size=$headers_artifact_size
bootloader_artifact_size=$bootloader_artifact_size
sha_kernel=$sha_kernel
sha_loader=$sha_loader
sha_param=$sha_param
buildworld_tail=$buildworld_tail
buildkernel_tail=$buildkernel_tail
installworld_tail=$installworld_tail
distribution_tail=$distribution_tail
installkernel_tail=$installkernel_tail
boot_backend=xcp-ng-xo-cli
init_mode=shepherd-pid1
self_hosted_native_build=ok
EOF
if [ -n "$metadata_target" ]; then
mkdir -p "$(dirname "$metadata_target")"
cp "$metadata_file" "$metadata_target"
fi
printf 'PASS phase20-self-hosted-native-build-xcpng\n'
printf 'Work directory: %s\n' "$workdir"
printf 'Metadata file: %s\n' "$metadata_file"
if [ -n "$metadata_target" ]; then
printf 'Copied metadata to: %s\n' "$metadata_target"
fi
printf '%s\n' '--- metadata ---'
cat "$metadata_file"

6
tests/system/run-phase8-system-image.sh
View File

@@ -104,11 +104,11 @@ image_size_bytes=$(stat -f '%z' "$disk_image")
closure_base=$(basename "$closure_path") closure_base=$(basename "$closure_path")
case "$image_store_path" in case "$image_store_path" in
/frx/store/*-fruix-bhyve-image-fruix-freebsd) : ;; /frx/store/*-fruix-bhyve-image-*) : ;;
*) echo "unexpected image store path: $image_store_path" >&2; exit 1 ;; *) echo "unexpected image store path: $image_store_path" >&2; exit 1 ;;
esac esac
case "$disk_image" in case "$disk_image" in
/frx/store/*-fruix-bhyve-image-fruix-freebsd/disk.img) : ;; /frx/store/*-fruix-bhyve-image-*/disk.img) : ;;
*) echo "unexpected disk image path: $disk_image" >&2; exit 1 ;; *) echo "unexpected disk image path: $disk_image" >&2; exit 1 ;;
esac esac
@@ -142,7 +142,7 @@ if [ -L "$mnt_root/etc/master.passwd" ]; then master_passwd_kind=symlink; elif [
loader_conf_image=$mnt_root/frx/store/$closure_base/boot/loader.conf loader_conf_image=$mnt_root/frx/store/$closure_base/boot/loader.conf
rc_conf_image=$mnt_root/frx/store/$closure_base/etc/rc.conf rc_conf_image=$mnt_root/frx/store/$closure_base/etc/rc.conf
grep -F 'comconsole' "$loader_conf_image" >/dev/null || { echo "loader.conf is missing serial console config" >&2; exit 1; } grep -F 'comconsole' "$loader_conf_image" >/dev/null || { echo "loader.conf is missing serial console config" >&2; exit 1; }
grep -F 'hostname="fruix-freebsd"' "$rc_conf_image" >/dev/null || { echo "rc.conf is missing hostname" >&2; exit 1; } grep -E '^hostname=".+"$' "$rc_conf_image" >/dev/null || { echo "rc.conf is missing hostname" >&2; exit 1; }
[ -f "$host_base_provenance_file" ] || { echo "missing host base provenance file: $host_base_provenance_file" >&2; exit 1; } [ -f "$host_base_provenance_file" ] || { echo "missing host base provenance file: $host_base_provenance_file" >&2; exit 1; }
[ -f "$store_layout_file" ] || { echo "missing store layout file: $store_layout_file" >&2; exit 1; } [ -f "$store_layout_file" ] || { echo "missing store layout file: $store_layout_file" >&2; exit 1; }
[ -n "$host_freebsd_version" ] || { echo "missing host freebsd version provenance" >&2; exit 1; } [ -n "$host_freebsd_version" ] || { echo "missing host freebsd version provenance" >&2; exit 1; }

271
tests/system/run-postphase20-installed-node-build-reconfigure-xcpng.sh Executable file
View File

@@ -0,0 +1,271 @@
#!/bin/sh
set -eu
repo_root=${PROJECT_ROOT:-$(pwd)}
os_template=${OS_TEMPLATE:-$repo_root/tests/system/postphase20-installed-node-operating-system.scm.in}
system_name=${SYSTEM_NAME:-postphase20-installed-node-operating-system}
result_store=${RESULT_STORE:-/frx/store/ffe44f5d1ba576e1f811ad3fe3a526a242b5c4a5-fruix-native-build-result-15.0-STABLE-ssh-guest}
root_size=${ROOT_SIZE:-12g}
current_host_name=${CURRENT_HOST_NAME:-fruix-node-current}
candidate_host_name=${CANDIDATE_HOST_NAME:-fruix-node-canary}
metadata_target=${METADATA_OUT:-}
root_authorized_key_file=${ROOT_AUTHORIZED_KEY_FILE:-$HOME/.ssh/id_ed25519.pub}
root_ssh_private_key_file=${ROOT_SSH_PRIVATE_KEY_FILE:-$HOME/.ssh/id_ed25519}
cleanup=0
if [ -n "${WORKDIR:-}" ]; then
workdir=$WORKDIR
mkdir -p "$workdir"
else
workdir=$(mktemp -d /tmp/fruix-postphase20-installed-node-xcpng.XXXXXX)
cleanup=1
fi
if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
cleanup=0
fi
current_os_file=$workdir/current-operating-system.scm
candidate_os_file=$workdir/candidate-operating-system.scm
inner_metadata=$workdir/postphase20-installed-node-inner-metadata.txt
current_build_out=$workdir/current-build.txt
candidate_build_out=$workdir/candidate-build.txt
reconfigure_out=$workdir/reconfigure.txt
rollback_out=$workdir/rollback.txt
post_reconfigure_status=$workdir/post-reconfigure-status.txt
post_boot_candidate_status=$workdir/post-boot-candidate-status.txt
post_boot_rollback_status=$workdir/post-boot-rollback-status.txt
metadata_file=$workdir/postphase20-installed-node-build-reconfigure-xcpng-metadata.txt
cleanup_workdir() {
if [ "$cleanup" -eq 1 ]; then
rm -rf "$workdir"
fi
}
trap cleanup_workdir EXIT INT TERM
[ -f "$os_template" ] || { echo "missing operating-system template: $os_template" >&2; exit 1; }
[ -f "$root_authorized_key_file" ] || { echo "missing root authorized key file: $root_authorized_key_file" >&2; exit 1; }
[ -f "$root_ssh_private_key_file" ] || { echo "missing root SSH private key file: $root_ssh_private_key_file" >&2; exit 1; }
[ -d "$result_store" ] || { echo "promoted result store does not exist: $result_store" >&2; exit 1; }
root_authorized_key=$(tr -d '\n' < "$root_authorized_key_file")
render_os() {
output=$1
host_name=$2
sed \
-e "s|__PROMOTED_RESULT_STORE__|$result_store|g" \
-e "s|__HOST_NAME__|$host_name|g" \
-e "s|__ROOT_AUTHORIZED_KEY__|$root_authorized_key|g" \
"$os_template" > "$output"
}
render_os "$current_os_file" "$current_host_name"
render_os "$candidate_os_file" "$candidate_host_name"
KEEP_WORKDIR=1 WORKDIR="$workdir/boot" METADATA_OUT="$inner_metadata" \
ROOT_AUTHORIZED_KEY_FILE="$root_authorized_key_file" \
ROOT_SSH_PRIVATE_KEY_FILE="$root_ssh_private_key_file" \
OS_TEMPLATE="$current_os_file" SYSTEM_NAME="$system_name" ROOT_SIZE="$root_size" \
"$repo_root/tests/system/run-phase11-shepherd-pid1-xcpng.sh"
phase8_metadata=$(sed -n 's/^phase8_metadata=//p' "$inner_metadata")
closure_path=$(sed -n 's/^closure_path=//p' "$inner_metadata")
closure_base=$(sed -n 's/^closure_base=//p' "$inner_metadata")
guest_ip=$(sed -n 's/^guest_ip=//p' "$inner_metadata")
vm_id=$(sed -n 's/^vm_id=//p' "$inner_metadata")
vdi_id=$(sed -n 's/^vdi_id=//p' "$inner_metadata")
shepherd_pid=$(sed -n 's/^shepherd_pid=//p' "$inner_metadata")
sshd_status=$(sed -n 's/^sshd_status=//p' "$inner_metadata")
compat_prefix_shims=$(sed -n 's/^compat_prefix_shims=//p' "$inner_metadata")
guile_module_smoke=$(sed -n 's/^guile_module_smoke=//p' "$inner_metadata")
activate_log=$(sed -n 's/^activate_log=//p' "$inner_metadata")
[ "$shepherd_pid" = 1 ] || { echo "shepherd was not PID 1" >&2; exit 1; }
[ "$sshd_status" = running ] || { echo "sshd is not running" >&2; exit 1; }
[ "$compat_prefix_shims" = absent ] || { echo "compatibility prefix shims reappeared" >&2; exit 1; }
[ "$guile_module_smoke" = ok ] || { echo "guest Guile module smoke failed" >&2; exit 1; }
case "$activate_log" in
*fruix-activate:done*) : ;;
*) echo "activation log does not show success" >&2; exit 1 ;;
esac
for path in \
"$closure_path/metadata/system-declaration.scm" \
"$closure_path/metadata/system-declaration-info.scm" \
"$closure_path/metadata/system-declaration-system" \
"$closure_path/share/fruix/node/scripts/fruix.scm" \
"$closure_path/share/fruix/node/modules/fruix/system/freebsd/render.scm" \
"$closure_path/share/fruix/node/guix/guix/build/utils.scm"
do
[ -f "$path" ] || {
echo "required installed-node path missing: $path" >&2
exit 1
}
done
grep -F "$current_host_name" "$closure_path/metadata/system-declaration.scm" >/dev/null || {
echo "embedded declaration does not mention current host name" >&2
exit 1
}
grep -F "$system_name" "$closure_path/metadata/system-declaration-system" >/dev/null || {
echo "embedded declaration system name is missing" >&2
exit 1
}
ssh_guest() {
ssh -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=5 \
root@"$guest_ip" "$@"
}
scp_guest() {
scp -O -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=5 \
"$@"
}
wait_for_ssh() {
for attempt in $(jot 120 1 120); do
if ssh_guest 'service sshd onestatus >/dev/null 2>&1' >/dev/null 2>&1; then
return 0
fi
sleep 2
done
return 1
}
reboot_guest() {
ssh_guest 'shutdown -r now >/dev/null 2>&1 || reboot >/dev/null 2>&1 || true' >/dev/null 2>&1 || true
sleep 5
wait_for_ssh || {
echo "guest did not return over SSH after reboot" >&2
exit 1
}
}
ssh_guest 'sh -s' <<EOF
set -eu
[ -x /usr/local/bin/fruix ]
[ -f /run/current-system/metadata/system-declaration.scm ]
[ -f /run/current-system/metadata/system-declaration-info.scm ]
[ -f /run/current-system/metadata/system-declaration-system ]
[ -f /run/current-system/share/fruix/node/scripts/fruix.scm ]
[ -f /run/current-system/share/fruix/node/modules/fruix/system/freebsd/media.scm ]
[ -f /run/current-system/share/fruix/node/guix/guix/build/utils.scm ]
grep -F '$system_name' /run/current-system/metadata/system-declaration-system >/dev/null
EOF
ssh_guest '/usr/local/bin/fruix system build' > "$current_build_out"
current_built_closure=$(sed -n 's/^closure_path=//p' "$current_build_out" | tail -n 1)
[ -n "$current_built_closure" ] || { echo "missing closure_path from in-system build output" >&2; exit 1; }
case "$current_built_closure" in
/frx/store/*-fruix-system-$current_host_name) : ;;
*)
echo "in-system build of current declaration produced an unexpected closure path: $current_built_closure" >&2
exit 1
;;
esac
scp_guest "$candidate_os_file" root@"$guest_ip":/root/candidate.scm >/dev/null
ssh_guest "/usr/local/bin/fruix system build /root/candidate.scm --system $system_name" > "$candidate_build_out"
candidate_closure=$(sed -n 's/^closure_path=//p' "$candidate_build_out" | tail -n 1)
[ -n "$candidate_closure" ] || { echo "missing candidate closure_path from in-system build output" >&2; exit 1; }
[ "$candidate_closure" != "$closure_path" ] || {
echo "candidate closure unexpectedly matches current closure" >&2
exit 1
}
ssh_guest "/usr/local/bin/fruix system reconfigure /root/candidate.scm --system $system_name" > "$reconfigure_out"
reconfigure_closure=$(sed -n 's/^reconfigure_closure=//p' "$reconfigure_out" | tail -n 1)
reconfigure_current_generation=$(sed -n 's/^current_generation=//p' "$reconfigure_out" | tail -n 1)
reconfigure_current_closure=$(sed -n 's/^current_closure=//p' "$reconfigure_out" | tail -n 1)
reconfigure_rollback_generation=$(sed -n 's/^rollback_generation=//p' "$reconfigure_out" | tail -n 1)
reconfigure_rollback_closure=$(sed -n 's/^rollback_closure=//p' "$reconfigure_out" | tail -n 1)
[ "$reconfigure_closure" = "$candidate_closure" ] || { echo "reconfigure closure mismatch" >&2; exit 1; }
[ "$reconfigure_current_generation" = 2 ] || { echo "unexpected current generation after reconfigure: $reconfigure_current_generation" >&2; exit 1; }
[ "$reconfigure_current_closure" = "$candidate_closure" ] || { echo "unexpected current closure after reconfigure" >&2; exit 1; }
[ "$reconfigure_rollback_generation" = 1 ] || { echo "unexpected rollback generation after reconfigure: $reconfigure_rollback_generation" >&2; exit 1; }
[ "$reconfigure_rollback_closure" = "$closure_path" ] || { echo "unexpected rollback closure after reconfigure" >&2; exit 1; }
ssh_guest '/usr/local/bin/fruix system status' > "$post_reconfigure_status"
reboot_guest
candidate_hostname=$(ssh_guest 'hostname')
candidate_run_current=$(ssh_guest 'readlink /run/current-system')
ssh_guest '/usr/local/bin/fruix system status' > "$post_boot_candidate_status"
[ "$candidate_hostname" = "$candidate_host_name" ] || { echo "unexpected host name after candidate boot: $candidate_hostname" >&2; exit 1; }
[ "$candidate_run_current" = "$candidate_closure" ] || { echo "unexpected current closure after candidate boot: $candidate_run_current" >&2; exit 1; }
ssh_guest '/usr/local/bin/fruix system rollback' > "$rollback_out"
rollback_current_generation=$(sed -n 's/^current_generation=//p' "$rollback_out" | tail -n 1)
rollback_current_closure=$(sed -n 's/^current_closure=//p' "$rollback_out" | tail -n 1)
rollback_rollback_generation=$(sed -n 's/^rollback_generation=//p' "$rollback_out" | tail -n 1)
rollback_rollback_closure=$(sed -n 's/^rollback_closure=//p' "$rollback_out" | tail -n 1)
[ "$rollback_current_generation" = 1 ] || { echo "unexpected current generation after rollback: $rollback_current_generation" >&2; exit 1; }
[ "$rollback_current_closure" = "$closure_path" ] || { echo "unexpected current closure after rollback" >&2; exit 1; }
[ "$rollback_rollback_generation" = 2 ] || { echo "unexpected rollback generation after rollback: $rollback_rollback_generation" >&2; exit 1; }
[ "$rollback_rollback_closure" = "$candidate_closure" ] || { echo "unexpected rollback closure after rollback" >&2; exit 1; }
reboot_guest
rollback_hostname=$(ssh_guest 'hostname')
rollback_run_current=$(ssh_guest 'readlink /run/current-system')
ssh_guest '/usr/local/bin/fruix system status' > "$post_boot_rollback_status"
[ "$rollback_hostname" = "$current_host_name" ] || { echo "unexpected host name after rollback boot: $rollback_hostname" >&2; exit 1; }
[ "$rollback_run_current" = "$closure_path" ] || { echo "unexpected current closure after rollback boot: $rollback_run_current" >&2; exit 1; }
cat > "$metadata_file" <<EOF
workdir=$workdir
inner_metadata=$inner_metadata
phase8_metadata=$phase8_metadata
closure_path=$closure_path
closure_base=$closure_base
vm_id=$vm_id
vdi_id=$vdi_id
guest_ip=$guest_ip
root_size=$root_size
result_store=$result_store
current_host_name=$current_host_name
candidate_host_name=$candidate_host_name
current_build_out=$current_build_out
current_built_closure=$current_built_closure
candidate_build_out=$candidate_build_out
candidate_closure=$candidate_closure
reconfigure_out=$reconfigure_out
reconfigure_closure=$reconfigure_closure
reconfigure_current_generation=$reconfigure_current_generation
reconfigure_current_closure=$reconfigure_current_closure
reconfigure_rollback_generation=$reconfigure_rollback_generation
reconfigure_rollback_closure=$reconfigure_rollback_closure
candidate_hostname=$candidate_hostname
candidate_run_current=$candidate_run_current
rollback_out=$rollback_out
rollback_current_generation=$rollback_current_generation
rollback_current_closure=$rollback_current_closure
rollback_rollback_generation=$rollback_rollback_generation
rollback_rollback_closure=$rollback_rollback_closure
rollback_hostname=$rollback_hostname
rollback_run_current=$rollback_run_current
boot_backend=xcp-ng-xo-cli
init_mode=shepherd-pid1
installed_node_build_reconfigure=ok
EOF
if [ -n "$metadata_target" ]; then
mkdir -p "$(dirname "$metadata_target")"
cp "$metadata_file" "$metadata_target"
fi
printf 'PASS postphase20-installed-node-build-reconfigure-xcpng\n'
printf 'Work directory: %s\n' "$workdir"
printf 'Metadata file: %s\n' "$metadata_file"
if [ -n "$metadata_target" ]; then
printf 'Copied metadata to: %s\n' "$metadata_target"
fi
printf '%s\n' '--- metadata ---'
cat "$metadata_file"