self/fruix
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

system: prototype self-hosted native builds

Browse Source
This commit is contained in:
Steffen
2026-04-05 17:59:56 +02:00
parent a3dd5556ae
commit 4614592a25
8 changed files with 701 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
docs/GUIX_DIFFERENCES.md
View File

@@ -263,6 +263,17 @@ The intent is:
Compared with Guix, this is conceptually similar to keeping development-oriented state separate from the main runtime identity, but Fruix currently expresses it as a system-attached development overlay rather than through Guix's broader profile/tooling model.
Fruix now also has a narrow guest self-hosted native-build prototype helper at:
- `/usr/local/bin/fruix-self-hosted-native-build`
That helper does **not** just reuse the whole exported development shell wholesale. The validated prototype had to sanitize development-oriented variables such as `MAKEFLAGS`, `CPPFLAGS`, `CFLAGS`, `CXXFLAGS`, and `LDFLAGS` before `buildworld`, because those are convenient for smaller development tasks but can poison the FreeBSD world/kernel bootstrap path.
The practical Fruix takeaway is:
- the development overlay makes native base work possible inside the system
- but real guest self-hosted base builds still need their own stricter build contract
## Where Fruix is intentionally trying to improve on Guix's representation
Fruix is not trying to improve on Guix's core semantics. Guix already got those right.

47
docs/PROGRESS.md
View File

@@ -42,6 +42,10 @@ Fruix currently has:
- real XCP-ng boot of a development-enabled Fruix system
- in-guest `buildworld` / `buildkernel`
- staged `installworld` / `distribution` / `installkernel`
- a validated controlled guest self-hosted native base-build prototype via:
- `/usr/local/bin/fruix-self-hosted-native-build`
- result roots under `/var/lib/fruix/native-builds`
- in-guest source recovery from current-system metadata
Validated boot modes still are:
@@ -54,33 +58,32 @@ The validated Phase 18 installation work currently uses:
## Latest completed achievement
### 2026-04-05 — Phase 20.2 completed
### 2026-04-05 — Phase 20.3 completed
Fruix now has a validated intermediate path where the host still orchestrates the workflow, but real FreeBSD native base-build work runs inside a booted Fruix-managed FreeBSD guest.
Fruix now has a validated first controlled guest self-hosted native base-build prototype, on top of the already validated host-initiated in-guest build path.
Highlights:
- development-enabled systems now expose canonical native-build compatibility links at:
- `/usr/include -> /run/current-system/development-profile/usr/include`
- `/usr/share/mk -> /run/current-system/development-profile/usr/share/mk`
- media builder versions were bumped so booted images and future installed targets pick up that rootfs layout change
- the validated guest build path now runs real FreeBSD native build steps inside the Fruix-managed guest:
- `buildworld`
- `buildkernel`
- `installworld`
- `distribution`
- `installkernel`
- staged install steps use:
- `DB_FROM_SRC=yes`
- so the staged install is driven by the declared source tree's account database rather than by the guest's minimal local `/etc` state
- the validated result now includes staged native artifact outputs for:
- development-enabled systems now ship an in-guest helper at:
- `/usr/local/bin/fruix-self-hosted-native-build`
- the helper recovers the declared materialized FreeBSD source store from:
- `/run/current-system/metadata/store-layout.scm`
- the helper records self-hosted build results under:
- `/var/lib/fruix/native-builds/<run-id>`
- `/var/lib/fruix/native-builds/latest`
- heavy object/stage work remains under:
- `/var/tmp/fruix-self-hosted-native-builds/<run-id>`
- the prototype exposed an important contract detail:
- a naive reuse of the development-shell exports polluted `buildworld`
- the validated helper therefore sanitizes development-oriented variables such as `MAKEFLAGS`, `CPPFLAGS`, `CFLAGS`, `CXXFLAGS`, and `LDFLAGS` before world/kernel bootstrap
- the validated result includes guest-recorded native artifact outputs for:
- kernel
- bootloader slice
- headers / `usr/share/mk`
Validation:
- `PASS phase20-host-initiated-native-build-xcpng`
- `PASS phase20-self-hosted-native-build-xcpng`
Reports:
@@ -88,6 +91,7 @@ Reports:
- `docs/GUIX_DIFFERENCES.md`
- `docs/reports/phase20-development-environment-freebsd.md`
- `docs/reports/phase20-host-initiated-native-builds-freebsd.md`
- `docs/reports/phase20-self-hosted-native-builds-freebsd.md`
## Recent major milestones
@@ -111,8 +115,11 @@ Reports:
## Next step
Per `docs/PLAN_4.md`, the next planned step is:
`docs/PLAN_4.md` currently ends at Phase 20.3, and that milestone sequence is now complete.
- **Phase 20.3** — reassess and potentially prototype guest self-hosted base builds
The next practical follow-up is therefore no longer another planned `PLAN_4` phase, but a product decision:
Phase 20.2 is now complete: Fruix validates host-initiated native FreeBSD base builds running inside the approved real XCP-ng Fruix guest path.
- whether to keep the new guest self-hosted helper as a narrow prototype while Phase 20.2 remains the default operator path
- or whether to invest in a broader guest-driven Fruix-native native-build workflow
Phase 20.3 is now complete: Fruix validates a first controlled guest self-hosted native FreeBSD base-build prototype on the approved real XCP-ng path.

201
docs/reports/phase20-self-hosted-native-builds-freebsd.md Normal file
View File

@@ -0,0 +1,201 @@
# Phase 20.3: controlled guest self-hosted native base-build prototype
Date: 2026-04-05
## Goal
Reassess guest self-hosting now that Fruix has already completed the earlier source, installation, generation-layout, rollback, development-overlay, and host-initiated in-guest native-build steps.
Phase 20.3 asked for real evidence about:
- what self-hosting would improve
- what it would cost in complexity
- how it fits with the Fruix source/deployment model already in place
## What changed
### New in-guest helper
Development-enabled systems now also ship:
- `/usr/local/bin/fruix-self-hosted-native-build`
This helper performs a controlled in-guest native FreeBSD base build using the system's own declared materialized source store recorded in:
- `/run/current-system/metadata/store-layout.scm`
The helper:
1. verifies the development overlay is present
2. verifies the canonical compatibility links exist:
- `/usr/include`
- `/usr/share/mk`
3. recovers the materialized FreeBSD source store from current-system metadata
4. runs:
- `buildworld`
- `buildkernel`
- `installworld`
- `distribution`
- `installkernel`
5. stages narrower artifact outputs under:
- `/var/lib/fruix/native-builds/<run-id>/artifacts/`
6. records metadata and status under:
- `/var/lib/fruix/native-builds/<run-id>/`
- `/var/lib/fruix/native-builds/latest`
The heavy object/stage work stays under:
- `/var/tmp/fruix-self-hosted-native-builds/<run-id>`
so the installed-system result area remains smaller and more legible.
### Important environment fix discovered during prototyping
The first prototype attempt failed even though Phase 20.2 had already succeeded.
Cause:
- directly evaluating `fruix-development-environment` before `buildworld` exported development-oriented variables like:
- `MAKEFLAGS`
- `CPPFLAGS`
- `CFLAGS`
- `CXXFLAGS`
- `LDFLAGS`
- those are appropriate for smaller development builds, but they polluted FreeBSD's world/kernel bootstrap environment and broke the LLVM bootstrap phase
Representative failure:
- missing generated LLVM config headers during bootstrap (`llvm/Config/abi-breaking.h`)
The validated fix was to make the self-hosted helper explicitly sanitize that environment first:
- reset `PATH` to the normal base paths
- unset development-shell variables such as:
- `MAKEFLAGS`
- `CC`, `CXX`, `AR`, `RANLIB`, `NM`
- `CPPFLAGS`, `CFLAGS`, `CXXFLAGS`, `LDFLAGS`
- `FRUIX_DEVELOPMENT_*`
- `FRUIX_*` tool variables
So the final 20.3 result is not “just reuse the development shell wholesale”.
It is more precise:
- use the development overlay for canonical paths and available content
- but run the real base-build steps in a cleaner, purpose-built helper environment
### Closure invalidation
To ensure the updated helper actually affects generated system closures, the operating-system closure spec now also records helper-version markers for development-enabled systems.
That ensures guest images pick up helper changes instead of silently reusing an older cached closure path.
### Validation harness
Added:
- `tests/system/run-phase20-self-hosted-native-build-xcpng.sh`
This harness:
1. boots the validated development-enabled Fruix guest on the approved XCP-ng path
2. verifies the new helper exists in the guest
3. invokes the helper from inside the guest
4. verifies the recorded result/status/`latest` pointer
5. validates the resulting staged artifact metadata and hashes
## Validation
Passing run:
- `PASS phase20-self-hosted-native-build-xcpng`
- workdir: `/tmp/fruix-phase20-self-hosted-native-build-xcpng`
Validated on the approved real XCP-ng path:
- VM `90490f2e-e8fc-4b7a-388e-5c26f0157289`
- VDI `0f1f90d3-48ca-4fa2-91d8-fc6339b95743`
Representative metadata:
```text
run_id=20260405T150359Z
helper_version=2
build_jobs=8
source_store=/frx/store/12d7704362e95afc2697db63f168b878e082b372-freebsd-source-default
build_root=/var/tmp/fruix-self-hosted-native-builds/20260405T150359Z
result_root=/var/lib/fruix/native-builds/20260405T150359Z
latest_link=/var/lib/fruix/native-builds/latest
latest_target=/var/lib/fruix/native-builds/20260405T150359Z
status_value=ok
build_root_size=7.5G
result_root_size=343M
kernel_artifact_size=158M
headers_artifact_size=32M
bootloader_artifact_size=1.3M
sha_kernel=16950f116a52134b98e2f8e0dacc556e18fe254e4a0ac2c1741422dde281a341
sha_loader=ea417846167ece270ada611624dca622ca38bd30125b9a125cd8ebb8b3600313
sha_param=9eb140ca7d9666f3d484a4174c9acd94b45427db6292b4e17de19af2c6aa5219
self_hosted_native_build=ok
```
Validated facts:
- the development-enabled Fruix guest can now run a controlled self-hosted native base-build helper from inside the installed system itself
- the helper can recover the declared source store from current-system metadata without host-side parsing
- `buildworld` and `buildkernel` succeed in the guest
- staged `installworld`, `distribution`, and `installkernel` succeed in the guest
- the helper records a stable result directory and `latest` pointer under:
- `/var/lib/fruix/native-builds`
- the resulting artifact hashes match the earlier validated Phase 20.2 host-initiated in-guest path
## What self-hosting improved
The prototype demonstrates a few real improvements:
- the build recipe itself now lives inside the Fruix-managed system, not only in a host-side SSH harness
- the guest can derive its own declared source input from current-system metadata
- result/state recording now has a Fruix-native installed-system location:
- `/var/lib/fruix/native-builds`
- the host no longer needs to spell out every `make` phase just to validate the in-guest path
## What it cost in complexity
The prototype also made the extra complexity visible:
- the guest helper needs its own controlled environment contract
- a naive reuse of the development-shell exports was wrong for real `buildworld`
- helper-version invalidation had to be made explicit so closure caching would not hide helper changes
- the in-guest result/staging model now needs its own operator-facing conventions
So the experiment did not eliminate complexity.
It mostly moved some of it from the host harness into an explicit in-guest helper contract.
## Decision after the prototype
Phase 20.3 is complete because Fruix now has a **first controlled guest self-hosted native base-build prototype**.
However, the evidence does **not** suggest replacing the Phase 20.2 path as the default operator workflow yet.
The current recommendation is:
- keep the **host-initiated in-guest native-build path** as the simpler default validation and orchestration flow
- keep the new **self-hosted helper** as a controlled prototype and stepping stone toward deeper guest-driven workflows
That fits the existing Fruix model well:
- source identity still comes from declared store-backed metadata
- deployment identity still comes from immutable closures under `/frx/store`
- the guest-side prototype adds a narrower in-system build/result workflow without replacing the existing deployment story
## Result
Phase 20.3 is complete.
Fruix now has:
- a validated host-orchestrated in-guest native base-build workflow
- and a validated first controlled guest self-hosted native base-build prototype
That answers the Phase 20.3 question with real evidence instead of only prior caution.

35
docs/system-deployment-workflow.md
View File

@@ -259,6 +259,41 @@ This is the current Phase 20.2 answer to “where should native base builds run?
- **inside** a Fruix-managed FreeBSD environment
- but still with the **host** driving the outer orchestration loop
### Controlled guest self-hosted native-build prototype
Fruix now also has a narrower in-guest prototype helper at:
- `/usr/local/bin/fruix-self-hosted-native-build`
Intended use:
```sh
FRUIX_SELF_HOSTED_NATIVE_BUILD_JOBS=8 \
/usr/local/bin/fruix-self-hosted-native-build
```
That helper:
1. verifies the development overlay and canonical compatibility links
2. recovers the materialized FreeBSD source store from:
- `/run/current-system/metadata/store-layout.scm`
3. runs the native FreeBSD build/install phases inside the guest
4. records results under:
- `/var/lib/fruix/native-builds/<run-id>`
- `/var/lib/fruix/native-builds/latest`
5. keeps the heavier object/stage work under:
- `/var/tmp/fruix-self-hosted-native-builds/<run-id>`
Important current detail:
- the self-hosted helper intentionally **sanitizes** development-shell exports such as `MAKEFLAGS`, `CPPFLAGS`, `CFLAGS`, `CXXFLAGS`, and `LDFLAGS` before `buildworld`
- directly reusing the full development-shell environment polluted FreeBSD's bootstrap path and was not reliable enough for real world/kernel builds
So the validated Phase 20.3 answer is:
- a controlled guest self-hosted base-build prototype now works
- but the simpler default operator flow should still be the Phase 20.2 host-initiated in-guest path unless there is a specific reason to push the build loop farther into the guest
## Deployment patterns
### 1. Build-first workflow