Compare commits
3 Commits
9e9a0b59fc
...
006ffee615
| Author | SHA1 | Date | |
|---|---|---|---|
| 006ffee615 | |||
| 4614592a25 | |||
| a3dd5556ae |
@@ -251,15 +251,49 @@ On those systems, Fruix exposes:
|
||||
- `/run/current-system/development-profile`
|
||||
- `/run/current-development`
|
||||
- `/usr/local/bin/fruix-development-environment`
|
||||
- `/usr/include -> /run/current-system/development-profile/usr/include`
|
||||
- `/usr/share/mk -> /run/current-system/development-profile/usr/share/mk`
|
||||
|
||||
The intent is:
|
||||
|
||||
- keep the main runtime profile lean
|
||||
- expose headers, `usr/share/mk`, and selected toolchain commands explicitly
|
||||
- satisfy native FreeBSD buildworld/buildkernel expectations for canonical system paths when development support is enabled
|
||||
- avoid treating a development-heavy system image as the default runtime shape
|
||||
|
||||
Compared with Guix, this is conceptually similar to keeping development-oriented state separate from the main runtime identity, but Fruix currently expresses it as a system-attached development overlay rather than through Guix's broader profile/tooling model.
|
||||
|
||||
Fruix now also has a narrow guest self-hosted native-build prototype helper at:
|
||||
|
||||
- `/usr/local/bin/fruix-self-hosted-native-build`
|
||||
|
||||
That helper does **not** just reuse the whole exported development shell wholesale. The validated prototype had to sanitize development-oriented variables such as `MAKEFLAGS`, `CPPFLAGS`, `CFLAGS`, `CXXFLAGS`, and `LDFLAGS` before `buildworld`, because those are convenient for smaller development tasks but can poison the FreeBSD world/kernel bootstrap path.
|
||||
|
||||
The practical Fruix takeaway is:
|
||||
|
||||
- the development overlay makes native base work possible inside the system
|
||||
- but real guest self-hosted base builds still need their own stricter build contract
|
||||
|
||||
Fruix now also makes an explicit distinction between:
|
||||
|
||||
- mutable native-build staging/results under:
|
||||
- `/var/lib/fruix/native-builds`
|
||||
- immutable promoted native-build identities under:
|
||||
- `/frx/store`
|
||||
|
||||
So a guest self-hosted run can stage a result tree locally, while the host can later promote that result into first-class Fruix store objects such as:
|
||||
|
||||
- `/frx/store/...-fruix-native-world-...`
|
||||
- `/frx/store/...-fruix-native-kernel-...`
|
||||
- `/frx/store/...-fruix-native-headers-...`
|
||||
- `/frx/store/...-fruix-native-bootloader-...`
|
||||
- `/frx/store/...-fruix-native-build-result-...`
|
||||
|
||||
Compared with Guix, this is a more explicit split between:
|
||||
|
||||
- a mutable result/staging area for native build execution
|
||||
- and the immutable store identities that Fruix treats as the real promoted result
|
||||
|
||||
## Where Fruix is intentionally trying to improve on Guix's representation
|
||||
|
||||
Fruix is not trying to improve on Guix's core semantics. Guix already got those right.
|
||||
|
||||
@@ -38,6 +38,21 @@ Fruix currently has:
|
||||
- `/run/current-system/development-profile`
|
||||
- `/run/current-development`
|
||||
- `/usr/local/bin/fruix-development-environment`
|
||||
- a validated host-initiated native base-build path inside a Fruix-managed guest via:
|
||||
- real XCP-ng boot of a development-enabled Fruix system
|
||||
- in-guest `buildworld` / `buildkernel`
|
||||
- staged `installworld` / `distribution` / `installkernel`
|
||||
- a validated controlled guest self-hosted native base-build prototype via:
|
||||
- `/usr/local/bin/fruix-self-hosted-native-build`
|
||||
- result roots under `/var/lib/fruix/native-builds`
|
||||
- in-guest source recovery from current-system metadata
|
||||
- a validated promotion path that turns native-build result roots into first-class Fruix store objects via:
|
||||
- `fruix native-build promote`
|
||||
- `/frx/store/...-fruix-native-world-...`
|
||||
- `/frx/store/...-fruix-native-kernel-...`
|
||||
- `/frx/store/...-fruix-native-headers-...`
|
||||
- `/frx/store/...-fruix-native-bootloader-...`
|
||||
- `/frx/store/...-fruix-native-build-result-...`
|
||||
|
||||
Validated boot modes still are:
|
||||
|
||||
@@ -50,37 +65,48 @@ The validated Phase 18 installation work currently uses:
|
||||
|
||||
## Latest completed achievement
|
||||
|
||||
### 2026-04-05 — Phase 20.1 completed
|
||||
### 2026-04-05 — Native base builds promoted into first-class Fruix store objects
|
||||
|
||||
Fruix now has a validated real-VM path where a booted Fruix-managed FreeBSD system exposes a separate development environment for native base work without collapsing the runtime/development split.
|
||||
Fruix now has a validated end-to-end path that treats guest native base-build results as **staged mutable results first**, and then promotes them into **immutable Fruix store identities**.
|
||||
|
||||
Highlights:
|
||||
|
||||
- operating-system declarations now support:
|
||||
- `#:development-packages`
|
||||
- system closures can now carry a separate development profile at:
|
||||
- `/run/current-system/development-profile`
|
||||
- `/run/current-development`
|
||||
- opt-in systems now ship an in-guest helper at:
|
||||
- `/usr/local/bin/fruix-development-environment`
|
||||
- the validated Phase 20.1 guest path exposes:
|
||||
- native headers
|
||||
- `usr/share/mk` for `bsd.*.mk`
|
||||
- Clang toolchain commands such as `cc`, `c++`, `ar`, `ranlib`, and `nm`
|
||||
- the validated guest workflow now supports:
|
||||
- `eval "$(/usr/local/bin/fruix-development-environment)"`
|
||||
- direct compilation with the Fruix-provided toolchain
|
||||
- a simple `bsd.prog.mk` build on the running Fruix guest
|
||||
- guest self-hosted runs still record staged results under:
|
||||
- `/var/lib/fruix/native-builds/<run-id>`
|
||||
- `/var/lib/fruix/native-builds/latest`
|
||||
- those result roots now carry promotion metadata describing:
|
||||
- executor / executor-version
|
||||
- closure path
|
||||
- source store provenance
|
||||
- build policy
|
||||
- artifact entries for:
|
||||
- `world`
|
||||
- `kernel`
|
||||
- `headers`
|
||||
- `bootloader`
|
||||
- the host can now run:
|
||||
- `fruix native-build promote RESULT_ROOT`
|
||||
- promotion creates immutable `/frx/store` objects for:
|
||||
- `world`
|
||||
- `kernel`
|
||||
- `headers`
|
||||
- `bootloader`
|
||||
- promotion also creates a result-bundle store object that references those artifact stores
|
||||
- the validated promotion metadata now makes Fruix-native native-build identity explicit instead of leaving results only as ad hoc files under `/var/lib/fruix/native-builds/...`
|
||||
|
||||
Validation:
|
||||
|
||||
- `PASS phase20-development-environment-xcpng`
|
||||
- `PASS phase20-self-hosted-native-build-xcpng`
|
||||
- `PASS phase20-native-build-store-promotion-xcpng`
|
||||
|
||||
Reports:
|
||||
|
||||
- `docs/system-deployment-workflow.md`
|
||||
- `docs/GUIX_DIFFERENCES.md`
|
||||
- `docs/reports/phase20-development-environment-freebsd.md`
|
||||
- `docs/reports/phase20-host-initiated-native-builds-freebsd.md`
|
||||
- `docs/reports/phase20-self-hosted-native-builds-freebsd.md`
|
||||
- `docs/reports/phase20-native-build-store-promotion-freebsd.md`
|
||||
|
||||
## Recent major milestones
|
||||
|
||||
@@ -104,8 +130,16 @@ Reports:
|
||||
|
||||
## Next step
|
||||
|
||||
Per `docs/PLAN_4.md`, the next planned step is:
|
||||
`docs/PLAN_4.md` currently ends at Phase 20.3, and that milestone sequence is now complete.
|
||||
|
||||
- **Phase 20.2** — run host-initiated native base builds inside a Fruix-managed environment
|
||||
The next practical follow-up is now clearer:
|
||||
|
||||
Phase 20.1 is now complete: Fruix validates a separate in-system development environment for native FreeBSD base work on the approved real XCP-ng path.
|
||||
- unify host-initiated and self-hosted native-build execution behind a shared Fruix executor/result model
|
||||
- make the same first-class promotion story available regardless of whether the outer loop is host-driven or guest-driven
|
||||
- decide how much of result import/promotion should remain host-side versus become a more integrated Fruix deployment action
|
||||
|
||||
The immediate architectural direction is no longer just “can guest self-hosting work?”
|
||||
|
||||
It is now:
|
||||
|
||||
- how should Fruix represent native base builds as real first-class objects and workflows across different executors?
|
||||
|
||||
169
docs/reports/phase20-host-initiated-native-builds-freebsd.md
Normal file
169
docs/reports/phase20-host-initiated-native-builds-freebsd.md
Normal file
@@ -0,0 +1,169 @@
|
||||
# Phase 20.2: host-initiated native base builds inside a Fruix-managed environment
|
||||
|
||||
Date: 2026-04-05
|
||||
|
||||
## Goal
|
||||
|
||||
Validate the next step after Phase 20.1:
|
||||
|
||||
- the host still orchestrates the outer loop
|
||||
- the actual FreeBSD native base build work runs inside a booted Fruix-managed system
|
||||
|
||||
This is the intermediate path between:
|
||||
|
||||
- purely host-side native base builds
|
||||
- any future claim of guest self-hosting
|
||||
|
||||
The target here was not a new self-hosted package manager story.
|
||||
|
||||
It was narrower:
|
||||
|
||||
- boot a Fruix-managed FreeBSD system on the approved real XCP-ng path
|
||||
- expose the development environment required for native base work
|
||||
- run real `buildworld` / `buildkernel` / staged install steps inside that Fruix guest
|
||||
- confirm that the resulting staged artifacts are the expected FreeBSD base slices
|
||||
|
||||
## Implementation
|
||||
|
||||
### Canonical development-path compatibility links
|
||||
|
||||
Phase 20.1 proved that Fruix could keep development content separate in:
|
||||
|
||||
- `/run/current-system/development-profile`
|
||||
- `/run/current-development`
|
||||
|
||||
Phase 20.2 exposed an additional practical requirement:
|
||||
|
||||
- FreeBSD native base builds still expect canonical system paths such as:
|
||||
- `/usr/include`
|
||||
- `/usr/share/mk`
|
||||
|
||||
For development-enabled systems, `populate-rootfs-from-closure` now also exposes:
|
||||
|
||||
- `/usr/include -> /run/current-system/development-profile/usr/include`
|
||||
- `/usr/share/mk -> /run/current-system/development-profile/usr/share/mk`
|
||||
|
||||
This keeps the development profile separate while still satisfying the buildworld/buildkernel assumptions of the native FreeBSD build system.
|
||||
|
||||
### Media builder invalidation
|
||||
|
||||
Because this changed the visible rootfs layout of booted systems, the media builder versions were bumped in `modules/fruix/system/freebsd/media.scm`:
|
||||
|
||||
- `image-builder-version`
|
||||
- `install-builder-version`
|
||||
- `installer-image-builder-version`
|
||||
- `installer-iso-builder-version`
|
||||
|
||||
That ensured booted images and future installed targets actually pick up the new compatibility links.
|
||||
|
||||
### New validation harness
|
||||
|
||||
Added:
|
||||
|
||||
- `tests/system/run-phase20-host-initiated-native-build-xcpng.sh`
|
||||
|
||||
This harness reuses the validated Phase 20.1 XCP-ng path first, then performs the 20.2-native-build step over SSH from the host.
|
||||
|
||||
The guest build flow is:
|
||||
|
||||
1. boot the development-enabled Fruix guest on XCP-ng
|
||||
2. recover the materialized source store from `/run/current-system/metadata/store-layout.scm`
|
||||
3. run real FreeBSD native build commands inside the guest:
|
||||
- `make -j8 buildworld`
|
||||
- `make -j8 buildkernel`
|
||||
- `make DESTDIR=... installworld`
|
||||
- `make DESTDIR=... distribution`
|
||||
- `make DESTDIR=... installkernel`
|
||||
4. stage narrower artifact slices from the staged output:
|
||||
- headers slice
|
||||
- bootloader slice
|
||||
- kernel stage
|
||||
|
||||
### Why `DB_FROM_SRC=yes` is used for staged install steps
|
||||
|
||||
The development-enabled Fruix guest is intentionally lean and does not carry the full ambient host account database.
|
||||
|
||||
`installworld` on modern FreeBSD checks for required users/groups unless `DB_FROM_SRC` is defined. For staged installs into `DESTDIR`, the appropriate controlled input is the source tree's own account database under `etc/`, not the minimal running guest's `/etc/master.passwd`.
|
||||
|
||||
So the validated Phase 20.2 staged install path uses:
|
||||
|
||||
- `DB_FROM_SRC=yes`
|
||||
|
||||
for:
|
||||
|
||||
- `installworld`
|
||||
- `distribution`
|
||||
- `installkernel`
|
||||
|
||||
That keeps the staged install driven by the declared source input rather than by accidental guest-local account state.
|
||||
|
||||
## Validation
|
||||
|
||||
Passing run:
|
||||
|
||||
- `PASS phase20-host-initiated-native-build-xcpng`
|
||||
- workdir: `/tmp/fruix-phase20-host-initiated-native-build-xcpng`
|
||||
|
||||
Validated on the approved real XCP-ng path:
|
||||
|
||||
- VM `90490f2e-e8fc-4b7a-388e-5c26f0157289`
|
||||
- VDI `0f1f90d3-48ca-4fa2-91d8-fc6339b95743`
|
||||
|
||||
Representative result:
|
||||
|
||||
```text
|
||||
build_jobs=8
|
||||
source_store=/frx/store/12d7704362e95afc2697db63f168b878e082b372-freebsd-source-default
|
||||
source_root=/frx/store/12d7704362e95afc2697db63f168b878e082b372-freebsd-source-default/tree
|
||||
build_root=/var/tmp/fruix-phase20-native-build
|
||||
world_stage=/var/tmp/fruix-phase20-native-build/stage-world
|
||||
kernel_stage=/var/tmp/fruix-phase20-native-build/stage-kernel
|
||||
headers_stage=/var/tmp/fruix-phase20-native-build/artifact-headers
|
||||
bootloader_stage=/var/tmp/fruix-phase20-native-build/artifact-bootloader
|
||||
build_root_size=7.6G
|
||||
world_stage_size=672M
|
||||
kernel_stage_size=739M
|
||||
headers_stage_size=32M
|
||||
bootloader_stage_size=1.3M
|
||||
sha_kernel=16950f116a52134b98e2f8e0dacc556e18fe254e4a0ac2c1741422dde281a341
|
||||
sha_loader=ea417846167ece270ada611624dca622ca38bd30125b9a125cd8ebb8b3600313
|
||||
sha_param=9eb140ca7d9666f3d484a4174c9acd94b45427db6292b4e17de19af2c6aa5219
|
||||
host_initiated_native_build=ok
|
||||
```
|
||||
|
||||
The harness verified all of the following:
|
||||
|
||||
- the guest still boots and passes the Phase 20.1 development-environment checks first
|
||||
- development-enabled systems expose canonical native-build compatibility links at:
|
||||
- `/usr/include`
|
||||
- `/usr/share/mk`
|
||||
- the guest can recover the declared materialized FreeBSD source store from system metadata
|
||||
- real FreeBSD `buildworld` succeeds inside the booted Fruix guest
|
||||
- real FreeBSD `buildkernel` succeeds inside the booted Fruix guest
|
||||
- staged `installworld`, `distribution`, and `installkernel` also succeed inside the guest
|
||||
- the staged outputs contain the expected artifact shapes:
|
||||
- `boot/kernel/kernel`
|
||||
- `usr/include/sys/param.h`
|
||||
- `usr/share/mk/bsd.prog.mk`
|
||||
- `boot/loader.efi`
|
||||
- `boot/defaults/loader.conf`
|
||||
- `boot/lua/loader.lua`
|
||||
|
||||
## Result
|
||||
|
||||
Phase 20.2 is complete.
|
||||
|
||||
Fruix now validates a real host-orchestrated path where:
|
||||
|
||||
- the host boots and reaches a Fruix-managed development-enabled guest
|
||||
- the guest uses its own Fruix-exposed development paths and declared source store
|
||||
- the native FreeBSD base build work runs inside that Fruix-managed environment
|
||||
- the host remains the outer orchestrator and result collector
|
||||
|
||||
This materially narrows the gap to any future self-hosting experiment while still avoiding the complexity jump to a full guest-driven package/deployment loop.
|
||||
|
||||
## Next step
|
||||
|
||||
Per `docs/PLAN_4.md`, the next planned step is:
|
||||
|
||||
- **Phase 20.3** — reassess and potentially prototype guest self-hosted base builds
|
||||
190
docs/reports/phase20-native-build-store-promotion-freebsd.md
Normal file
190
docs/reports/phase20-native-build-store-promotion-freebsd.md
Normal file
@@ -0,0 +1,190 @@
|
||||
# Post-Phase 20: native build result promotion into first-class Fruix store objects
|
||||
|
||||
Date: 2026-04-05
|
||||
|
||||
## Goal
|
||||
|
||||
Make native FreeBSD base-build results feel like real Fruix objects instead of stopping at mutable staged files under:
|
||||
|
||||
- `/var/lib/fruix/native-builds/...`
|
||||
|
||||
The desired model is:
|
||||
|
||||
- `/var/lib/fruix/native-builds/...` remains a staging/result area
|
||||
- `/frx/store/...` remains the real immutable identity
|
||||
|
||||
Validated artifact identities:
|
||||
|
||||
- `world`
|
||||
- `kernel`
|
||||
- `headers`
|
||||
- `bootloader`
|
||||
|
||||
## What changed
|
||||
|
||||
### Promotion metadata in guest result roots
|
||||
|
||||
The guest self-hosted helper now emits a promotion description file at:
|
||||
|
||||
- `/var/lib/fruix/native-builds/<run-id>/promotion.scm`
|
||||
|
||||
That metadata records at least:
|
||||
|
||||
- executor / executor-version
|
||||
- run-id / guest-host-name
|
||||
- closure path
|
||||
- development profile path
|
||||
- declared FreeBSD base metadata
|
||||
- source store provenance
|
||||
- build policy
|
||||
- artifact entries for:
|
||||
- `world`
|
||||
- `kernel`
|
||||
- `headers`
|
||||
- `bootloader`
|
||||
|
||||
The helper also stages a promotable `world` artifact tree in addition to the already validated narrower artifacts.
|
||||
|
||||
### Host-side promotion API
|
||||
|
||||
Fruix now exports:
|
||||
|
||||
- `promote-native-build-result`
|
||||
|
||||
and the CLI now exposes:
|
||||
|
||||
- `fruix native-build promote RESULT_ROOT [--store DIR]`
|
||||
|
||||
### Promoted store object layout
|
||||
|
||||
Promotion now creates immutable store objects for:
|
||||
|
||||
- `/frx/store/...-fruix-native-world-...`
|
||||
- `/frx/store/...-fruix-native-kernel-...`
|
||||
- `/frx/store/...-fruix-native-headers-...`
|
||||
- `/frx/store/...-fruix-native-bootloader-...`
|
||||
|
||||
Each promoted artifact store records:
|
||||
|
||||
- `.fruix-native-build-object.scm`
|
||||
- `.references`
|
||||
|
||||
Promotion also creates a result-bundle store object:
|
||||
|
||||
- `/frx/store/...-fruix-native-build-result-...`
|
||||
|
||||
That bundle records:
|
||||
|
||||
- `.fruix-native-build-result.scm`
|
||||
- `artifacts/world`
|
||||
- `artifacts/kernel`
|
||||
- `artifacts/headers`
|
||||
- `artifacts/bootloader`
|
||||
|
||||
where the `artifacts/*` entries are symlinks to the promoted artifact stores.
|
||||
|
||||
### Identity policy
|
||||
|
||||
Artifact identity is now based on Fruix metadata plus a tree-content signature of the staged artifact tree.
|
||||
|
||||
That means promotion identity depends on both:
|
||||
|
||||
- the explicit Fruix-native build metadata
|
||||
- the actual content of the promoted artifact tree
|
||||
|
||||
## Validation harness
|
||||
|
||||
Added:
|
||||
|
||||
- `tests/system/run-phase20-native-build-store-promotion-xcpng.sh`
|
||||
|
||||
This harness:
|
||||
|
||||
1. boots the approved real XCP-ng guest path
|
||||
2. runs the validated in-guest self-hosted native build helper
|
||||
3. imports the guest result root back to the host
|
||||
4. runs `fruix native-build promote`
|
||||
5. verifies promoted store paths, metadata, symlink structure, and representative hashes
|
||||
|
||||
## Validation
|
||||
|
||||
Passing run:
|
||||
|
||||
- `PASS phase20-native-build-store-promotion-xcpng`
|
||||
- workdir: `/tmp/current-phase20-native-build-store-promotion-xcpng`
|
||||
|
||||
Approved real XCP-ng path:
|
||||
|
||||
- VM `90490f2e-e8fc-4b7a-388e-5c26f0157289`
|
||||
- VDI `0f1f90d3-48ca-4fa2-91d8-fc6339b95743`
|
||||
|
||||
Representative metadata:
|
||||
|
||||
```text
|
||||
run_id=20260405T213444Z
|
||||
source_store=/frx/store/12d7704362e95afc2697db63f168b878e082b372-freebsd-source-default
|
||||
guest_result_root=/var/lib/fruix/native-builds/20260405T213444Z
|
||||
result_store=/frx/store/c6329a0053720b05aff3274b8b1d522c909f475d-fruix-native-build-result-15.0-STABLE-guest-self-hosted
|
||||
world_store=/frx/store/dfe37b36f6537a95ceea16ea62001b2ca5617eb7-fruix-native-world-15.0-STABLE-guest-self-hosted
|
||||
kernel_store=/frx/store/0ab7cbceca240ab2c3b91e83e059844ea792e49e-fruix-native-kernel-15.0-STABLE-guest-self-hosted
|
||||
headers_store=/frx/store/5bbeae9266687a229f1c6d176a08886c35243ff0-fruix-native-headers-15.0-STABLE-guest-self-hosted
|
||||
bootloader_store=/frx/store/bd49a508bd7a3b94a2535d6774f31c993c406552-fruix-native-bootloader-15.0-STABLE-guest-self-hosted
|
||||
artifact_store_count=4
|
||||
sha_kernel=16950f116a52134b98e2f8e0dacc556e18fe254e4a0ac2c1741422dde281a341
|
||||
sha_loader=ea417846167ece270ada611624dca622ca38bd30125b9a125cd8ebb8b3600313
|
||||
sha_param=9eb140ca7d9666f3d484a4174c9acd94b45427db6292b4e17de19af2c6aa5219
|
||||
promoted_kernel_sha=16950f116a52134b98e2f8e0dacc556e18fe254e4a0ac2c1741422dde281a341
|
||||
promoted_loader_sha=ea417846167ece270ada611624dca622ca38bd30125b9a125cd8ebb8b3600313
|
||||
promoted_param_sha=9eb140ca7d9666f3d484a4174c9acd94b45427db6292b4e17de19af2c6aa5219
|
||||
native_build_store_promotion=ok
|
||||
```
|
||||
|
||||
Validated facts:
|
||||
|
||||
- guest self-hosted native-build results remain available under `/var/lib/fruix/native-builds/...` as mutable staging/results
|
||||
- those staged results are now sufficient to promote first-class Fruix identities on the host
|
||||
- promotion creates four immutable artifact store objects plus one immutable result-bundle store object
|
||||
- promoted metadata retains executor/source/build-policy/provenance information explicitly
|
||||
- promoted kernel, bootloader, and headers hashes match the already validated staged artifacts
|
||||
- the promoted `world` artifact is now also preserved as a first-class Fruix store object instead of remaining only as an unpromoted staged tree
|
||||
|
||||
## Important implementation note
|
||||
|
||||
The first full promotion attempt exposed an incorrect assumption in the validation surface:
|
||||
|
||||
- the promoted/staged `world` artifact should be checked at:
|
||||
- `bin/sh`
|
||||
- not at:
|
||||
- `usr/bin/sh`
|
||||
|
||||
That path expectation was corrected in:
|
||||
|
||||
- helper validation
|
||||
- emitted promotion metadata
|
||||
- the end-to-end promotion harness
|
||||
|
||||
## Result
|
||||
|
||||
Fruix now has a validated native-build object model with a clear split:
|
||||
|
||||
- mutable native-build result roots under `/var/lib/fruix/native-builds/...`
|
||||
- immutable promoted identities under `/frx/store/...`
|
||||
|
||||
That makes native FreeBSD base builds feel substantially more Fruix-native:
|
||||
|
||||
- build outputs have explicit immutable identities
|
||||
- metadata is Fruix-native rather than implied only by ad hoc directory layout
|
||||
- executor/source/provenance/build-policy remain attached to the promoted result
|
||||
- the staged result area and the real store identity are now intentionally distinct
|
||||
|
||||
## Next direction
|
||||
|
||||
This suggests the next product step is not merely “more self-hosting”.
|
||||
|
||||
It is to generalize this result model so different execution modes can converge on the same promoted object story, for example:
|
||||
|
||||
- host-initiated in-guest builds
|
||||
- guest self-hosted builds
|
||||
- future executor variants
|
||||
|
||||
That would move Fruix closer to a shared executor model rather than treating each validation path as a one-off harness.
|
||||
201
docs/reports/phase20-self-hosted-native-builds-freebsd.md
Normal file
201
docs/reports/phase20-self-hosted-native-builds-freebsd.md
Normal file
@@ -0,0 +1,201 @@
|
||||
# Phase 20.3: controlled guest self-hosted native base-build prototype
|
||||
|
||||
Date: 2026-04-05
|
||||
|
||||
## Goal
|
||||
|
||||
Reassess guest self-hosting now that Fruix has already completed the earlier source, installation, generation-layout, rollback, development-overlay, and host-initiated in-guest native-build steps.
|
||||
|
||||
Phase 20.3 asked for real evidence about:
|
||||
|
||||
- what self-hosting would improve
|
||||
- what it would cost in complexity
|
||||
- how it fits with the Fruix source/deployment model already in place
|
||||
|
||||
## What changed
|
||||
|
||||
### New in-guest helper
|
||||
|
||||
Development-enabled systems now also ship:
|
||||
|
||||
- `/usr/local/bin/fruix-self-hosted-native-build`
|
||||
|
||||
This helper performs a controlled in-guest native FreeBSD base build using the system's own declared materialized source store recorded in:
|
||||
|
||||
- `/run/current-system/metadata/store-layout.scm`
|
||||
|
||||
The helper:
|
||||
|
||||
1. verifies the development overlay is present
|
||||
2. verifies the canonical compatibility links exist:
|
||||
- `/usr/include`
|
||||
- `/usr/share/mk`
|
||||
3. recovers the materialized FreeBSD source store from current-system metadata
|
||||
4. runs:
|
||||
- `buildworld`
|
||||
- `buildkernel`
|
||||
- `installworld`
|
||||
- `distribution`
|
||||
- `installkernel`
|
||||
5. stages narrower artifact outputs under:
|
||||
- `/var/lib/fruix/native-builds/<run-id>/artifacts/`
|
||||
6. records metadata and status under:
|
||||
- `/var/lib/fruix/native-builds/<run-id>/`
|
||||
- `/var/lib/fruix/native-builds/latest`
|
||||
|
||||
The heavy object/stage work stays under:
|
||||
|
||||
- `/var/tmp/fruix-self-hosted-native-builds/<run-id>`
|
||||
|
||||
so the installed-system result area remains smaller and more legible.
|
||||
|
||||
### Important environment fix discovered during prototyping
|
||||
|
||||
The first prototype attempt failed even though Phase 20.2 had already succeeded.
|
||||
|
||||
Cause:
|
||||
|
||||
- directly evaluating `fruix-development-environment` before `buildworld` exported development-oriented variables like:
|
||||
- `MAKEFLAGS`
|
||||
- `CPPFLAGS`
|
||||
- `CFLAGS`
|
||||
- `CXXFLAGS`
|
||||
- `LDFLAGS`
|
||||
- those are appropriate for smaller development builds, but they polluted FreeBSD's world/kernel bootstrap environment and broke the LLVM bootstrap phase
|
||||
|
||||
Representative failure:
|
||||
|
||||
- missing generated LLVM config headers during bootstrap (`llvm/Config/abi-breaking.h`)
|
||||
|
||||
The validated fix was to make the self-hosted helper explicitly sanitize that environment first:
|
||||
|
||||
- reset `PATH` to the normal base paths
|
||||
- unset development-shell variables such as:
|
||||
- `MAKEFLAGS`
|
||||
- `CC`, `CXX`, `AR`, `RANLIB`, `NM`
|
||||
- `CPPFLAGS`, `CFLAGS`, `CXXFLAGS`, `LDFLAGS`
|
||||
- `FRUIX_DEVELOPMENT_*`
|
||||
- `FRUIX_*` tool variables
|
||||
|
||||
So the final 20.3 result is not “just reuse the development shell wholesale”.
|
||||
|
||||
It is more precise:
|
||||
|
||||
- use the development overlay for canonical paths and available content
|
||||
- but run the real base-build steps in a cleaner, purpose-built helper environment
|
||||
|
||||
### Closure invalidation
|
||||
|
||||
To ensure the updated helper actually affects generated system closures, the operating-system closure spec now also records helper-version markers for development-enabled systems.
|
||||
|
||||
That ensures guest images pick up helper changes instead of silently reusing an older cached closure path.
|
||||
|
||||
### Validation harness
|
||||
|
||||
Added:
|
||||
|
||||
- `tests/system/run-phase20-self-hosted-native-build-xcpng.sh`
|
||||
|
||||
This harness:
|
||||
|
||||
1. boots the validated development-enabled Fruix guest on the approved XCP-ng path
|
||||
2. verifies the new helper exists in the guest
|
||||
3. invokes the helper from inside the guest
|
||||
4. verifies the recorded result/status/`latest` pointer
|
||||
5. validates the resulting staged artifact metadata and hashes
|
||||
|
||||
## Validation
|
||||
|
||||
Passing run:
|
||||
|
||||
- `PASS phase20-self-hosted-native-build-xcpng`
|
||||
- workdir: `/tmp/fruix-phase20-self-hosted-native-build-xcpng`
|
||||
|
||||
Validated on the approved real XCP-ng path:
|
||||
|
||||
- VM `90490f2e-e8fc-4b7a-388e-5c26f0157289`
|
||||
- VDI `0f1f90d3-48ca-4fa2-91d8-fc6339b95743`
|
||||
|
||||
Representative metadata:
|
||||
|
||||
```text
|
||||
run_id=20260405T150359Z
|
||||
helper_version=2
|
||||
build_jobs=8
|
||||
source_store=/frx/store/12d7704362e95afc2697db63f168b878e082b372-freebsd-source-default
|
||||
build_root=/var/tmp/fruix-self-hosted-native-builds/20260405T150359Z
|
||||
result_root=/var/lib/fruix/native-builds/20260405T150359Z
|
||||
latest_link=/var/lib/fruix/native-builds/latest
|
||||
latest_target=/var/lib/fruix/native-builds/20260405T150359Z
|
||||
status_value=ok
|
||||
build_root_size=7.5G
|
||||
result_root_size=343M
|
||||
kernel_artifact_size=158M
|
||||
headers_artifact_size=32M
|
||||
bootloader_artifact_size=1.3M
|
||||
sha_kernel=16950f116a52134b98e2f8e0dacc556e18fe254e4a0ac2c1741422dde281a341
|
||||
sha_loader=ea417846167ece270ada611624dca622ca38bd30125b9a125cd8ebb8b3600313
|
||||
sha_param=9eb140ca7d9666f3d484a4174c9acd94b45427db6292b4e17de19af2c6aa5219
|
||||
self_hosted_native_build=ok
|
||||
```
|
||||
|
||||
Validated facts:
|
||||
|
||||
- the development-enabled Fruix guest can now run a controlled self-hosted native base-build helper from inside the installed system itself
|
||||
- the helper can recover the declared source store from current-system metadata without host-side parsing
|
||||
- `buildworld` and `buildkernel` succeed in the guest
|
||||
- staged `installworld`, `distribution`, and `installkernel` succeed in the guest
|
||||
- the helper records a stable result directory and `latest` pointer under:
|
||||
- `/var/lib/fruix/native-builds`
|
||||
- the resulting artifact hashes match the earlier validated Phase 20.2 host-initiated in-guest path
|
||||
|
||||
## What self-hosting improved
|
||||
|
||||
The prototype demonstrates a few real improvements:
|
||||
|
||||
- the build recipe itself now lives inside the Fruix-managed system, not only in a host-side SSH harness
|
||||
- the guest can derive its own declared source input from current-system metadata
|
||||
- result/state recording now has a Fruix-native installed-system location:
|
||||
- `/var/lib/fruix/native-builds`
|
||||
- the host no longer needs to spell out every `make` phase just to validate the in-guest path
|
||||
|
||||
## What it cost in complexity
|
||||
|
||||
The prototype also made the extra complexity visible:
|
||||
|
||||
- the guest helper needs its own controlled environment contract
|
||||
- a naive reuse of the development-shell exports was wrong for real `buildworld`
|
||||
- helper-version invalidation had to be made explicit so closure caching would not hide helper changes
|
||||
- the in-guest result/staging model now needs its own operator-facing conventions
|
||||
|
||||
So the experiment did not eliminate complexity.
|
||||
|
||||
It mostly moved some of it from the host harness into an explicit in-guest helper contract.
|
||||
|
||||
## Decision after the prototype
|
||||
|
||||
Phase 20.3 is complete because Fruix now has a **first controlled guest self-hosted native base-build prototype**.
|
||||
|
||||
However, the evidence does **not** suggest replacing the Phase 20.2 path as the default operator workflow yet.
|
||||
|
||||
The current recommendation is:
|
||||
|
||||
- keep the **host-initiated in-guest native-build path** as the simpler default validation and orchestration flow
|
||||
- keep the new **self-hosted helper** as a controlled prototype and stepping stone toward deeper guest-driven workflows
|
||||
|
||||
That fits the existing Fruix model well:
|
||||
|
||||
- source identity still comes from declared store-backed metadata
|
||||
- deployment identity still comes from immutable closures under `/frx/store`
|
||||
- the guest-side prototype adds a narrower in-system build/result workflow without replacing the existing deployment story
|
||||
|
||||
## Result
|
||||
|
||||
Phase 20.3 is complete.
|
||||
|
||||
Fruix now has:
|
||||
|
||||
- a validated host-orchestrated in-guest native base-build workflow
|
||||
- and a validated first controlled guest self-hosted native base-build prototype
|
||||
|
||||
That answers the Phase 20.3 question with real evidence instead of only prior caution.
|
||||
@@ -214,15 +214,145 @@ Intended use:
|
||||
eval "$(/usr/local/bin/fruix-development-environment)"
|
||||
```
|
||||
|
||||
That helper exports a development-oriented environment while keeping the main runtime profile separate. The validated Phase 20.1 path currently uses this to expose at least:
|
||||
That helper exports a development-oriented environment while keeping the main runtime profile separate. The validated Phase 20 path currently uses this to expose at least:
|
||||
|
||||
- native headers under `usr/include`
|
||||
- FreeBSD `share/mk` files for `bsd.*.mk`
|
||||
- Clang toolchain commands such as `cc`, `c++`, `ar`, `ranlib`, and `nm`
|
||||
- `MAKEFLAGS` pointing at the development profile's `usr/share/mk`
|
||||
|
||||
For native base-build compatibility, development-enabled systems also now expose canonical links at:
|
||||
|
||||
- `/usr/include -> /run/current-system/development-profile/usr/include`
|
||||
- `/usr/share/mk -> /run/current-system/development-profile/usr/share/mk`
|
||||
|
||||
This is the current Fruix-native way to make a running system suitable for controlled native base-development work without merging development content back into the main runtime profile.
|
||||
|
||||
### Host-initiated native base builds inside a Fruix-managed guest
|
||||
|
||||
The currently validated intermediate path toward self-hosting is still host-orchestrated.
|
||||
|
||||
The host:
|
||||
|
||||
1. boots a development-enabled Fruix guest
|
||||
2. connects over SSH
|
||||
3. recovers the materialized FreeBSD source store from system metadata
|
||||
4. runs native FreeBSD build commands inside the guest
|
||||
5. collects and records the staged outputs
|
||||
|
||||
The validated build sequence inside the guest is:
|
||||
|
||||
- `make -jN buildworld`
|
||||
- `make -jN buildkernel`
|
||||
- `make DESTDIR=... installworld`
|
||||
- `make DESTDIR=... distribution`
|
||||
- `make DESTDIR=... installkernel`
|
||||
|
||||
For staged install steps, the validated path uses:
|
||||
|
||||
- `DB_FROM_SRC=yes`
|
||||
|
||||
so the staged install is driven by the declared source tree's account database rather than by accidental guest-local `/etc/master.passwd` contents.
|
||||
|
||||
This is the current Phase 20.2 answer to “where should native base builds run?”
|
||||
|
||||
- **inside** a Fruix-managed FreeBSD environment
|
||||
- but still with the **host** driving the outer orchestration loop
|
||||
|
||||
### Controlled guest self-hosted native-build prototype
|
||||
|
||||
Fruix now also has a narrower in-guest prototype helper at:
|
||||
|
||||
- `/usr/local/bin/fruix-self-hosted-native-build`
|
||||
|
||||
Intended use:
|
||||
|
||||
```sh
|
||||
FRUIX_SELF_HOSTED_NATIVE_BUILD_JOBS=8 \
|
||||
/usr/local/bin/fruix-self-hosted-native-build
|
||||
```
|
||||
|
||||
That helper:
|
||||
|
||||
1. verifies the development overlay and canonical compatibility links
|
||||
2. recovers the materialized FreeBSD source store from:
|
||||
- `/run/current-system/metadata/store-layout.scm`
|
||||
3. runs the native FreeBSD build/install phases inside the guest
|
||||
4. records staged results under:
|
||||
- `/var/lib/fruix/native-builds/<run-id>`
|
||||
- `/var/lib/fruix/native-builds/latest`
|
||||
5. emits promotion metadata for first-class artifact identities covering:
|
||||
- `world`
|
||||
- `kernel`
|
||||
- `headers`
|
||||
- `bootloader`
|
||||
6. keeps the heavier object/stage work under:
|
||||
- `/var/tmp/fruix-self-hosted-native-builds/<run-id>`
|
||||
|
||||
Important current detail:
|
||||
|
||||
- the self-hosted helper intentionally **sanitizes** development-shell exports such as `MAKEFLAGS`, `CPPFLAGS`, `CFLAGS`, `CXXFLAGS`, and `LDFLAGS` before `buildworld`
|
||||
- directly reusing the full development-shell environment polluted FreeBSD's bootstrap path and was not reliable enough for real world/kernel builds
|
||||
|
||||
So the validated Phase 20.3 answer is:
|
||||
|
||||
- a controlled guest self-hosted base-build prototype now works
|
||||
- but the simpler default operator flow should still be the Phase 20.2 host-initiated in-guest path unless there is a specific reason to push the build loop farther into the guest
|
||||
|
||||
### Promoting native-build results into first-class Fruix store objects
|
||||
|
||||
The guest-side result root is now explicitly a **staging/result area**, not the final immutable identity.
|
||||
|
||||
Current validated flow:
|
||||
|
||||
1. run the in-guest helper so the guest records a result under:
|
||||
- `/var/lib/fruix/native-builds/<run-id>`
|
||||
2. copy that result root back to the host
|
||||
3. run:
|
||||
|
||||
```sh
|
||||
fruix native-build promote RESULT_ROOT
|
||||
```
|
||||
|
||||
The promotion step creates immutable `/frx/store` identities for:
|
||||
|
||||
- `world`
|
||||
- `kernel`
|
||||
- `headers`
|
||||
- `bootloader`
|
||||
|
||||
and also creates a result-bundle store object that references those promoted artifact stores.
|
||||
|
||||
Current metadata split:
|
||||
|
||||
- mutable staging/result root:
|
||||
- `/var/lib/fruix/native-builds/<run-id>`
|
||||
- immutable artifact stores:
|
||||
- `/frx/store/...-fruix-native-world-...`
|
||||
- `/frx/store/...-fruix-native-kernel-...`
|
||||
- `/frx/store/...-fruix-native-headers-...`
|
||||
- `/frx/store/...-fruix-native-bootloader-...`
|
||||
- immutable result bundle:
|
||||
- `/frx/store/...-fruix-native-build-result-...`
|
||||
|
||||
The promoted store objects record explicit Fruix-native metadata including at least:
|
||||
|
||||
- executor / executor-version
|
||||
- run-id / guest-host-name
|
||||
- closure path
|
||||
- source store provenance
|
||||
- build policy
|
||||
- artifact kind
|
||||
- required-file expectations
|
||||
- recorded content signatures and hashes
|
||||
|
||||
This is the current Fruix-native answer to the question:
|
||||
|
||||
- where should mutable native-build state live?
|
||||
- `/var/lib/fruix/native-builds/...`
|
||||
- where should immutable native-build identity live?
|
||||
- `/frx/store/...`
|
||||
|
||||
## Deployment patterns
|
||||
|
||||
### 1. Build-first workflow
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
(define-module (fruix system freebsd)
|
||||
#:use-module (fruix system freebsd model)
|
||||
#:use-module (fruix system freebsd source)
|
||||
#:use-module (fruix system freebsd build)
|
||||
#:use-module (fruix system freebsd media)
|
||||
#:re-export (user-group
|
||||
user-group?
|
||||
@@ -43,6 +44,7 @@
|
||||
operating-system-root-authorized-keys
|
||||
validate-operating-system
|
||||
materialize-freebsd-source
|
||||
promote-native-build-result
|
||||
operating-system-closure-spec
|
||||
operating-system-install-spec
|
||||
operating-system-image-spec
|
||||
|
||||
@@ -11,6 +11,7 @@
|
||||
#:use-module (srfi srfi-13)
|
||||
#:export (host-freebsd-provenance
|
||||
materialize-freebsd-package
|
||||
promote-native-build-result
|
||||
materialize-prefix))
|
||||
|
||||
(define (host-freebsd-provenance)
|
||||
@@ -388,6 +389,182 @@
|
||||
output-path))))
|
||||
|
||||
|
||||
(define native-build-result-promotion-version "1")
|
||||
|
||||
(define (native-build-result-ref data key default)
|
||||
(match (assoc key data)
|
||||
((_ . value) value)
|
||||
(#f default)))
|
||||
|
||||
(define (read-native-build-result result-root)
|
||||
(let ((promotion-file (string-append result-root "/promotion.scm")))
|
||||
(unless (file-exists? promotion-file)
|
||||
(error "native build result is missing promotion.scm" result-root))
|
||||
(let ((result (call-with-input-file promotion-file read)))
|
||||
(unless (equal? (native-build-result-ref result 'native-build-result-version #f)
|
||||
native-build-result-promotion-version)
|
||||
(error "unsupported native build result promotion version" promotion-file))
|
||||
result)))
|
||||
|
||||
(define (native-build-artifact-entry result artifact-kind)
|
||||
(let* ((artifacts (native-build-result-ref result 'artifacts '()))
|
||||
(entry (assoc artifact-kind artifacts)))
|
||||
(unless entry
|
||||
(error "native build result is missing artifact entry" artifact-kind))
|
||||
(cdr entry)))
|
||||
|
||||
(define (native-build-artifact-root result-root result artifact-kind)
|
||||
(let* ((entry (native-build-artifact-entry result artifact-kind))
|
||||
(relative-path (native-build-result-ref entry 'path #f))
|
||||
(required-file (native-build-result-ref entry 'required-file #f))
|
||||
(artifact-root (and relative-path
|
||||
(string-append result-root "/" relative-path))))
|
||||
(unless (and artifact-root (file-exists? artifact-root))
|
||||
(error "native build result is missing artifact tree" artifact-kind artifact-root))
|
||||
(when required-file
|
||||
(unless (file-exists? (string-append artifact-root "/" required-file))
|
||||
(error "native build artifact is missing required file"
|
||||
artifact-kind
|
||||
(string-append artifact-root "/" required-file))))
|
||||
artifact-root))
|
||||
|
||||
(define (native-build-existing-store-references result store-dir)
|
||||
(filter identity
|
||||
(map (lambda (path)
|
||||
(and (string? path)
|
||||
(string-prefix? (string-append store-dir "/") path)
|
||||
(file-exists? path)
|
||||
path))
|
||||
(list (native-build-result-ref result 'closure-path #f)
|
||||
(let ((source (native-build-result-ref result 'source '())))
|
||||
(native-build-result-ref source 'store-path #f))))))
|
||||
|
||||
(define (native-build-artifact-display-name result artifact-kind)
|
||||
(let* ((base (native-build-result-ref result 'freebsd-base '()))
|
||||
(version-label (native-build-result-ref base 'version-label "unknown"))
|
||||
(executor (native-build-result-ref result 'executor "unknown")))
|
||||
(string-append "fruix-native-"
|
||||
(symbol->string artifact-kind)
|
||||
"-"
|
||||
version-label
|
||||
"-"
|
||||
executor)))
|
||||
|
||||
(define (native-build-promoted-artifact-metadata result artifact-kind content-signature)
|
||||
(let* ((entry (native-build-artifact-entry result artifact-kind)))
|
||||
`((native-build-object-version . ,native-build-result-promotion-version)
|
||||
(object-kind . artifact)
|
||||
(artifact-kind . ,artifact-kind)
|
||||
(executor . ,(native-build-result-ref result 'executor "unknown"))
|
||||
(executor-version . ,(native-build-result-ref result 'executor-version "unknown"))
|
||||
(run-id . ,(native-build-result-ref result 'run-id "unknown"))
|
||||
(guest-host-name . ,(native-build-result-ref result 'guest-host-name "unknown"))
|
||||
(closure-path . ,(native-build-result-ref result 'closure-path ""))
|
||||
(development-profile . ,(native-build-result-ref result 'development-profile ""))
|
||||
(freebsd-base . ,(native-build-result-ref result 'freebsd-base '()))
|
||||
(source . ,(native-build-result-ref result 'source '()))
|
||||
(build-policy . ,(native-build-result-ref result 'build-policy '()))
|
||||
(required-file . ,(native-build-result-ref entry 'required-file ""))
|
||||
(recorded-sha256 . ,(native-build-result-ref entry 'recorded-sha256 ""))
|
||||
(content-signature . ,content-signature))))
|
||||
|
||||
(define (promote-native-build-artifact result-root result store-dir artifact-kind)
|
||||
(let* ((artifact-root (native-build-artifact-root result-root result artifact-kind))
|
||||
(content-signature (tree-content-signature artifact-root))
|
||||
(metadata (native-build-promoted-artifact-metadata result artifact-kind content-signature))
|
||||
(payload (object->string metadata))
|
||||
(display-name (native-build-artifact-display-name result artifact-kind))
|
||||
(output-path (make-store-path store-dir display-name payload
|
||||
#:kind 'native-build-artifact
|
||||
#:output artifact-kind))
|
||||
(references (native-build-existing-store-references result store-dir)))
|
||||
(unless (file-exists? output-path)
|
||||
(mkdir-p output-path)
|
||||
(stage-tree-into-output artifact-root output-path)
|
||||
(write-file (string-append output-path "/.references")
|
||||
(string-join references "\n"))
|
||||
(write-file (string-append output-path "/.fruix-native-build-object.scm")
|
||||
payload))
|
||||
`((artifact-kind . ,artifact-kind)
|
||||
(artifact-root . ,artifact-root)
|
||||
(store-path . ,output-path)
|
||||
(content-signature . ,content-signature)
|
||||
(metadata-file . ,(string-append output-path "/.fruix-native-build-object.scm")))) )
|
||||
|
||||
(define (native-build-result-display-name result)
|
||||
(let* ((base (native-build-result-ref result 'freebsd-base '()))
|
||||
(version-label (native-build-result-ref base 'version-label "unknown"))
|
||||
(executor (native-build-result-ref result 'executor "unknown")))
|
||||
(string-append "fruix-native-build-result-" version-label "-" executor)))
|
||||
|
||||
(define (native-build-promoted-result-object result promoted-artifacts)
|
||||
`((native-build-result-version . ,native-build-result-promotion-version)
|
||||
(object-kind . result-bundle)
|
||||
(executor . ,(native-build-result-ref result 'executor "unknown"))
|
||||
(executor-version . ,(native-build-result-ref result 'executor-version "unknown"))
|
||||
(run-id . ,(native-build-result-ref result 'run-id "unknown"))
|
||||
(guest-host-name . ,(native-build-result-ref result 'guest-host-name "unknown"))
|
||||
(closure-path . ,(native-build-result-ref result 'closure-path ""))
|
||||
(development-profile . ,(native-build-result-ref result 'development-profile ""))
|
||||
(freebsd-base . ,(native-build-result-ref result 'freebsd-base '()))
|
||||
(source . ,(native-build-result-ref result 'source '()))
|
||||
(build-policy . ,(native-build-result-ref result 'build-policy '()))
|
||||
(artifact-count . ,(length promoted-artifacts))
|
||||
(artifacts . ,(map (lambda (entry)
|
||||
`((artifact-kind . ,(assoc-ref entry 'artifact-kind))
|
||||
(store-path . ,(assoc-ref entry 'store-path))
|
||||
(content-signature . ,(assoc-ref entry 'content-signature))
|
||||
(metadata-file . ,(assoc-ref entry 'metadata-file))))
|
||||
promoted-artifacts))))
|
||||
|
||||
(define* (promote-native-build-result result-root #:key (store-dir "/frx/store"))
|
||||
(let* ((result (read-native-build-result result-root))
|
||||
(promoted-artifacts (map (lambda (artifact-kind)
|
||||
(promote-native-build-artifact result-root result store-dir artifact-kind))
|
||||
'(world kernel headers bootloader)))
|
||||
(result-object (native-build-promoted-result-object result promoted-artifacts))
|
||||
(payload (object->string result-object))
|
||||
(display-name (native-build-result-display-name result))
|
||||
(result-store (make-store-path store-dir display-name payload
|
||||
#:kind 'native-build-result))
|
||||
(result-references (append (map (lambda (entry)
|
||||
(assoc-ref entry 'store-path))
|
||||
promoted-artifacts)
|
||||
(native-build-existing-store-references result store-dir))))
|
||||
(unless (file-exists? result-store)
|
||||
(mkdir-p (string-append result-store "/artifacts"))
|
||||
(for-each (lambda (entry)
|
||||
(symlink (assoc-ref entry 'store-path)
|
||||
(string-append result-store
|
||||
"/artifacts/"
|
||||
(symbol->string (assoc-ref entry 'artifact-kind)))))
|
||||
promoted-artifacts)
|
||||
(write-file (string-append result-store "/.references")
|
||||
(string-join result-references "\n"))
|
||||
(write-file (string-append result-store "/.fruix-native-build-result.scm")
|
||||
payload))
|
||||
`((result-root . ,result-root)
|
||||
(result-store . ,result-store)
|
||||
(result-metadata-file . ,(string-append result-store "/.fruix-native-build-result.scm"))
|
||||
(artifact-store-count . ,(length promoted-artifacts))
|
||||
(artifact-stores . ,(map (lambda (entry) (assoc-ref entry 'store-path)) promoted-artifacts))
|
||||
(world-store . ,(assoc-ref (find (lambda (entry)
|
||||
(eq? (assoc-ref entry 'artifact-kind) 'world))
|
||||
promoted-artifacts)
|
||||
'store-path))
|
||||
(kernel-store . ,(assoc-ref (find (lambda (entry)
|
||||
(eq? (assoc-ref entry 'artifact-kind) 'kernel))
|
||||
promoted-artifacts)
|
||||
'store-path))
|
||||
(headers-store . ,(assoc-ref (find (lambda (entry)
|
||||
(eq? (assoc-ref entry 'artifact-kind) 'headers))
|
||||
promoted-artifacts)
|
||||
'store-path))
|
||||
(bootloader-store . ,(assoc-ref (find (lambda (entry)
|
||||
(eq? (assoc-ref entry 'artifact-kind) 'bootloader))
|
||||
promoted-artifacts)
|
||||
'store-path)))))
|
||||
|
||||
(define (sanitize-materialized-prefix name output-path)
|
||||
(cond
|
||||
((string=? name "fruix-guile-extra")
|
||||
|
||||
@@ -225,6 +225,8 @@
|
||||
(chmod (string-append closure-path "/usr/local/bin/fruix") #o555))
|
||||
(when (file-exists? (string-append closure-path "/usr/local/bin/fruix-development-environment"))
|
||||
(chmod (string-append closure-path "/usr/local/bin/fruix-development-environment") #o555))
|
||||
(when (file-exists? (string-append closure-path "/usr/local/bin/fruix-self-hosted-native-build"))
|
||||
(chmod (string-append closure-path "/usr/local/bin/fruix-self-hosted-native-build") #o555))
|
||||
(when (file-exists? (string-append closure-path "/boot/fruix-pid1"))
|
||||
(chmod (string-append closure-path "/boot/fruix-pid1") #o555))
|
||||
(write-file (string-append closure-path "/parameters.scm")
|
||||
@@ -376,10 +378,19 @@
|
||||
(string-append rootfs "/usr/local/bin/fruix"))
|
||||
(when (file-exists? (string-append closure-path "/development-profile"))
|
||||
(symlink-force "/run/current-system/development-profile"
|
||||
(string-append rootfs "/run/current-development")))
|
||||
(string-append rootfs "/run/current-development"))
|
||||
(when (file-exists? (string-append closure-path "/development-profile/usr/include"))
|
||||
(symlink-force "/run/current-system/development-profile/usr/include"
|
||||
(string-append rootfs "/usr/include")))
|
||||
(when (file-exists? (string-append closure-path "/development-profile/usr/share/mk"))
|
||||
(symlink-force "/run/current-system/development-profile/usr/share/mk"
|
||||
(string-append rootfs "/usr/share/mk"))))
|
||||
(when (file-exists? (string-append closure-path "/usr/local/bin/fruix-development-environment"))
|
||||
(symlink-force "/run/current-system/usr/local/bin/fruix-development-environment"
|
||||
(string-append rootfs "/usr/local/bin/fruix-development-environment")))
|
||||
(when (file-exists? (string-append closure-path "/usr/local/bin/fruix-self-hosted-native-build"))
|
||||
(symlink-force "/run/current-system/usr/local/bin/fruix-self-hosted-native-build"
|
||||
(string-append rootfs "/usr/local/bin/fruix-self-hosted-native-build")))
|
||||
(symlink-force "/run/current-system/usr/local/etc/rc.d/fruix-activate"
|
||||
(string-append rootfs "/usr/local/etc/rc.d/fruix-activate"))
|
||||
(symlink-force "/run/current-system/usr/local/etc/rc.d/fruix-shepherd"
|
||||
@@ -598,10 +609,10 @@
|
||||
(installer-root-partition-label . ,installer-root-partition-label)
|
||||
(target-install . ,target-install-spec))))
|
||||
|
||||
(define image-builder-version "2")
|
||||
(define install-builder-version "1")
|
||||
(define installer-image-builder-version "1")
|
||||
(define installer-iso-builder-version "2")
|
||||
(define image-builder-version "3")
|
||||
(define install-builder-version "2")
|
||||
(define installer-image-builder-version "2")
|
||||
(define installer-iso-builder-version "3")
|
||||
|
||||
(define (operating-system-install-metadata-object install-spec closure-path store-items)
|
||||
`((install-version . ,install-builder-version)
|
||||
|
||||
@@ -308,7 +308,8 @@
|
||||
"usr/local/bin/fruix")
|
||||
(if (null? (operating-system-development-packages os))
|
||||
'()
|
||||
'("usr/local/bin/fruix-development-environment"))
|
||||
'("usr/local/bin/fruix-development-environment"
|
||||
"usr/local/bin/fruix-self-hosted-native-build"))
|
||||
(if (pid1-init-mode? os)
|
||||
'("boot/fruix-pid1")
|
||||
'())
|
||||
@@ -330,6 +331,10 @@
|
||||
(base-packages . ,(package-names (operating-system-base-packages os)))
|
||||
(development-package-count . ,(length (operating-system-development-packages os)))
|
||||
(development-packages . ,(package-names (operating-system-development-packages os)))
|
||||
(development-environment-helper-version
|
||||
. ,(if (null? (operating-system-development-packages os)) #f "1"))
|
||||
(self-hosted-native-build-helper-version
|
||||
. ,(if (null? (operating-system-development-packages os)) #f "3"))
|
||||
(user-count . ,(length (operating-system-users os)))
|
||||
(users . ,(map user-account-name (operating-system-users os)))
|
||||
(group-count . ,(length (operating-system-groups os)))
|
||||
|
||||
@@ -785,6 +785,223 @@
|
||||
"export MAKEFLAGS=\"-m $profile/usr/share/mk\"\n"
|
||||
"export PATH=\"/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:$profile/bin:$profile/sbin:$profile/usr/bin:$profile/usr/sbin\"\n"
|
||||
"EOF\n"))
|
||||
(define (render-self-hosted-native-build-script os)
|
||||
(let* ((base-spec (freebsd-base-spec (operating-system-freebsd-base os)))
|
||||
(base-name (assoc-ref base-spec 'name))
|
||||
(version-label (assoc-ref base-spec 'version-label))
|
||||
(release (assoc-ref base-spec 'release))
|
||||
(branch (assoc-ref base-spec 'branch))
|
||||
(declared-source-root (assoc-ref base-spec 'source-root))
|
||||
(target (assoc-ref base-spec 'target))
|
||||
(target-arch (assoc-ref base-spec 'target-arch))
|
||||
(kernconf (assoc-ref base-spec 'kernconf))
|
||||
(make-flags (or (assoc-ref base-spec 'make-flags) '()))
|
||||
(build-common (string-join
|
||||
(append (list (format #f "TARGET=~a" target)
|
||||
(format #f "TARGET_ARCH=~a" target-arch)
|
||||
(format #f "KERNCONF=~a" kernconf))
|
||||
make-flags)
|
||||
" "))
|
||||
(install-common (string-append build-common " DB_FROM_SRC=yes")))
|
||||
(string-append
|
||||
"#!/bin/sh\n"
|
||||
"set -eu\n"
|
||||
"umask 022\n"
|
||||
"profile=/run/current-system/development-profile\n"
|
||||
"guest_host_name='" (operating-system-host-name os) "'\n"
|
||||
"[ -d \"$profile\" ] || {\n"
|
||||
" echo \"fruix-self-hosted-native-build: development profile is not available\" >&2\n"
|
||||
" exit 1\n"
|
||||
"}\n"
|
||||
"[ -x /usr/local/bin/fruix-development-environment ] || {\n"
|
||||
" echo \"fruix-self-hosted-native-build: development environment helper is missing\" >&2\n"
|
||||
" exit 1\n"
|
||||
"}\n"
|
||||
"PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin\n"
|
||||
"unset MAKEOBJDIRPREFIX MAKEFLAGS CC CXX AR RANLIB NM CPPFLAGS CFLAGS CXXFLAGS LDFLAGS\n"
|
||||
"unset FRUIX_DEVELOPMENT_PROFILE FRUIX_DEVELOPMENT_INCLUDE FRUIX_DEVELOPMENT_LIB FRUIX_DEVELOPMENT_SHARE_MK\n"
|
||||
"unset FRUIX_DEVELOPMENT_BIN FRUIX_DEVELOPMENT_USR_BIN FRUIX_CC FRUIX_CXX FRUIX_AR FRUIX_RANLIB FRUIX_NM FRUIX_BMAKE\n"
|
||||
"[ -L /usr/include ] || {\n"
|
||||
" echo \"fruix-self-hosted-native-build: /usr/include compatibility link is missing\" >&2\n"
|
||||
" exit 1\n"
|
||||
"}\n"
|
||||
"[ \"$(readlink /usr/include)\" = \"/run/current-system/development-profile/usr/include\" ] || {\n"
|
||||
" echo \"fruix-self-hosted-native-build: /usr/include points at the wrong target\" >&2\n"
|
||||
" exit 1\n"
|
||||
"}\n"
|
||||
"[ -L /usr/share/mk ] || {\n"
|
||||
" echo \"fruix-self-hosted-native-build: /usr/share/mk compatibility link is missing\" >&2\n"
|
||||
" exit 1\n"
|
||||
"}\n"
|
||||
"[ \"$(readlink /usr/share/mk)\" = \"/run/current-system/development-profile/usr/share/mk\" ] || {\n"
|
||||
" echo \"fruix-self-hosted-native-build: /usr/share/mk points at the wrong target\" >&2\n"
|
||||
" exit 1\n"
|
||||
"}\n"
|
||||
"jobs=${FRUIX_SELF_HOSTED_NATIVE_BUILD_JOBS:-$(sysctl -n hw.ncpu)}\n"
|
||||
"case \"$jobs\" in\n"
|
||||
" ''|*[!0-9]*)\n"
|
||||
" echo \"fruix-self-hosted-native-build: invalid job count: $jobs\" >&2\n"
|
||||
" exit 1\n"
|
||||
" ;;\n"
|
||||
"esac\n"
|
||||
"run_id=${FRUIX_SELF_HOSTED_NATIVE_BUILD_ID:-$(date -u +%Y%m%dT%H%M%SZ)}\n"
|
||||
"build_root_base=${FRUIX_SELF_HOSTED_NATIVE_BUILD_ROOT_BASE:-/var/tmp/fruix-self-hosted-native-builds}\n"
|
||||
"result_root_base=${FRUIX_SELF_HOSTED_NATIVE_BUILD_OUTPUT_BASE:-/var/lib/fruix/native-builds}\n"
|
||||
"build_root=$build_root_base/$run_id\n"
|
||||
"result_root=$result_root_base/$run_id\n"
|
||||
"logdir=$result_root/logs\n"
|
||||
"status_file=$result_root/status\n"
|
||||
"metadata_file=$result_root/metadata.txt\n"
|
||||
"promotion_file=$result_root/promotion.scm\n"
|
||||
"world_stage=$build_root/stage-world\n"
|
||||
"kernel_stage=$build_root/stage-kernel\n"
|
||||
"world_artifact=$result_root/artifacts/world\n"
|
||||
"headers_artifact=$result_root/artifacts/headers\n"
|
||||
"kernel_artifact=$result_root/artifacts/kernel\n"
|
||||
"bootloader_artifact=$result_root/artifacts/bootloader\n"
|
||||
"latest_link=$result_root_base/latest\n"
|
||||
"mkdir -p \"$build_root\" \"$result_root\" \"$logdir\"\n"
|
||||
"printf 'running\\n' > \"$status_file\"\n"
|
||||
"fail_mark() {\n"
|
||||
" rc=$?\n"
|
||||
" if [ \"$rc\" -ne 0 ]; then\n"
|
||||
" printf 'failed\\n' > \"$status_file\"\n"
|
||||
" fi\n"
|
||||
"}\n"
|
||||
"trap fail_mark EXIT HUP INT TERM\n"
|
||||
"closure=$(readlink /run/current-system)\n"
|
||||
"store_layout=$closure/metadata/store-layout.scm\n"
|
||||
"[ -f \"$store_layout\" ] || {\n"
|
||||
" echo \"fruix-self-hosted-native-build: store layout metadata is missing\" >&2\n"
|
||||
" exit 1\n"
|
||||
"}\n"
|
||||
"source_store=$(sed -n 's/.*\"\\(\\/frx\\/store\\/[^\"]*-freebsd-source-[^\"]*\\)\".*/\\1/p' \"$store_layout\" | head -n 1)\n"
|
||||
"[ -n \"$source_store\" ] || {\n"
|
||||
" echo \"fruix-self-hosted-native-build: failed to recover source store from store-layout.scm\" >&2\n"
|
||||
" exit 1\n"
|
||||
"}\n"
|
||||
"source_root=$source_store/tree\n"
|
||||
"[ -d \"$source_root\" ] || {\n"
|
||||
" echo \"fruix-self-hosted-native-build: source root is missing: $source_root\" >&2\n"
|
||||
" exit 1\n"
|
||||
"}\n"
|
||||
"mkdir -p \"$world_artifact\" \"$headers_artifact/usr\" \"$kernel_artifact/boot\" \"$bootloader_artifact/boot\"\n"
|
||||
"export MAKEOBJDIRPREFIX=\"$build_root/obj\"\n"
|
||||
"make -j\"$jobs\" -C \"$source_root\" " build-common " buildworld > \"$logdir/buildworld.log\" 2>&1\n"
|
||||
"make -j\"$jobs\" -C \"$source_root\" " build-common " buildkernel > \"$logdir/buildkernel.log\" 2>&1\n"
|
||||
"make -C \"$source_root\" " install-common " DESTDIR=\"$world_stage\" installworld > \"$logdir/installworld.log\" 2>&1\n"
|
||||
"make -C \"$source_root\" " install-common " DESTDIR=\"$world_stage\" distribution > \"$logdir/distribution.log\" 2>&1\n"
|
||||
"make -C \"$source_root\" " install-common " DESTDIR=\"$kernel_stage\" installkernel > \"$logdir/installkernel.log\" 2>&1\n"
|
||||
"cp -a \"$world_stage/.\" \"$world_artifact/\"\n"
|
||||
"cp -a \"$kernel_stage/boot/kernel\" \"$kernel_artifact/boot/kernel\"\n"
|
||||
"cp -a \"$world_stage/usr/include\" \"$headers_artifact/usr/include\"\n"
|
||||
"mkdir -p \"$headers_artifact/usr/share\"\n"
|
||||
"cp -a \"$world_stage/usr/share/mk\" \"$headers_artifact/usr/share/mk\"\n"
|
||||
"cp -a \"$world_stage/boot/loader\" \"$bootloader_artifact/boot/loader\"\n"
|
||||
"cp -a \"$world_stage/boot/loader.efi\" \"$bootloader_artifact/boot/loader.efi\"\n"
|
||||
"cp -a \"$world_stage/boot/device.hints\" \"$bootloader_artifact/boot/device.hints\"\n"
|
||||
"cp -a \"$world_stage/boot/defaults\" \"$bootloader_artifact/boot/defaults\"\n"
|
||||
"cp -a \"$world_stage/boot/lua\" \"$bootloader_artifact/boot/lua\"\n"
|
||||
"[ -f \"$world_artifact/bin/sh\" ]\n"
|
||||
"[ -f \"$kernel_artifact/boot/kernel/kernel\" ]\n"
|
||||
"[ -f \"$headers_artifact/usr/include/sys/param.h\" ]\n"
|
||||
"[ -f \"$headers_artifact/usr/share/mk/bsd.prog.mk\" ]\n"
|
||||
"[ -f \"$bootloader_artifact/boot/loader.efi\" ]\n"
|
||||
"[ -f \"$bootloader_artifact/boot/defaults/loader.conf\" ]\n"
|
||||
"[ -f \"$bootloader_artifact/boot/lua/loader.lua\" ]\n"
|
||||
"sha_kernel=$(sha256 -q \"$kernel_artifact/boot/kernel/kernel\")\n"
|
||||
"sha_loader=$(sha256 -q \"$bootloader_artifact/boot/loader.efi\")\n"
|
||||
"sha_param=$(sha256 -q \"$headers_artifact/usr/include/sys/param.h\")\n"
|
||||
"buildworld_tail=$(tail -n 20 \"$logdir/buildworld.log\" | tr '\\n' ' ')\n"
|
||||
"buildkernel_tail=$(tail -n 20 \"$logdir/buildkernel.log\" | tr '\\n' ' ')\n"
|
||||
"installworld_tail=$(tail -n 20 \"$logdir/installworld.log\" | tr '\\n' ' ')\n"
|
||||
"distribution_tail=$(tail -n 20 \"$logdir/distribution.log\" | tr '\\n' ' ')\n"
|
||||
"installkernel_tail=$(tail -n 20 \"$logdir/installkernel.log\" | tr '\\n' ' ')\n"
|
||||
"root_df=$(df -h / | tail -n 1 | tr -s ' ' | tr '\\t' ' ')\n"
|
||||
"build_root_size=$(du -sh \"$build_root\" | awk '{print $1}')\n"
|
||||
"result_root_size=$(du -sh \"$result_root\" | awk '{print $1}')\n"
|
||||
"world_artifact_size=$(du -sh \"$world_artifact\" | awk '{print $1}')\n"
|
||||
"kernel_artifact_size=$(du -sh \"$kernel_artifact\" | awk '{print $1}')\n"
|
||||
"headers_artifact_size=$(du -sh \"$headers_artifact\" | awk '{print $1}')\n"
|
||||
"bootloader_artifact_size=$(du -sh \"$bootloader_artifact\" | awk '{print $1}')\n"
|
||||
"rm -f \"$latest_link\"\n"
|
||||
"ln -s \"$result_root\" \"$latest_link\"\n"
|
||||
"cat >\"$promotion_file\" <<EOF\n"
|
||||
"((native-build-result-version . \"1\")\n"
|
||||
" (executor . \"guest-self-hosted\")\n"
|
||||
" (executor-version . \"3\")\n"
|
||||
" (run-id . \"$run_id\")\n"
|
||||
" (guest-host-name . \"$guest_host_name\")\n"
|
||||
" (closure-path . \"$closure\")\n"
|
||||
" (development-profile . \"$profile\")\n"
|
||||
" (freebsd-base . ((name . \"" base-name "\")\n"
|
||||
" (version-label . \"" version-label "\")\n"
|
||||
" (release . \"" release "\")\n"
|
||||
" (branch . \"" branch "\")\n"
|
||||
" (source-root . \"" declared-source-root "\")\n"
|
||||
" (target . \"" target "\")\n"
|
||||
" (target-arch . \"" target-arch "\")\n"
|
||||
" (kernconf . \"" kernconf "\")))\n"
|
||||
" (source . ((store-path . \"$source_store\")\n"
|
||||
" (source-root . \"$source_root\")))\n"
|
||||
" (build-policy . ((jobs . \"$jobs\")\n"
|
||||
" (build-common . \"" build-common "\")\n"
|
||||
" (install-common . \"" install-common "\")))\n"
|
||||
" (artifacts . ((world . ((path . \"artifacts/world\")\n"
|
||||
" (required-file . \"bin/sh\")))\n"
|
||||
" (kernel . ((path . \"artifacts/kernel\")\n"
|
||||
" (required-file . \"boot/kernel/kernel\")\n"
|
||||
" (recorded-sha256 . \"$sha_kernel\")))\n"
|
||||
" (headers . ((path . \"artifacts/headers\")\n"
|
||||
" (required-file . \"usr/include/sys/param.h\")\n"
|
||||
" (recorded-sha256 . \"$sha_param\")))\n"
|
||||
" (bootloader . ((path . \"artifacts/bootloader\")\n"
|
||||
" (required-file . \"boot/loader.efi\")\n"
|
||||
" (recorded-sha256 . \"$sha_loader\"))))))\n"
|
||||
"EOF\n"
|
||||
"cat >\"$metadata_file\" <<EOF\n"
|
||||
"run_id=$run_id\n"
|
||||
"helper_version=3\n"
|
||||
"closure_path=$closure\n"
|
||||
"guest_host_name=$guest_host_name\n"
|
||||
"development_profile=$profile\n"
|
||||
"source_store=$source_store\n"
|
||||
"source_root=$source_root\n"
|
||||
"build_jobs=$jobs\n"
|
||||
"build_common=" build-common "\n"
|
||||
"install_common=" install-common "\n"
|
||||
"build_root=$build_root\n"
|
||||
"result_root=$result_root\n"
|
||||
"logdir=$logdir\n"
|
||||
"status_file=$status_file\n"
|
||||
"metadata_file=$metadata_file\n"
|
||||
"promotion_file=$promotion_file\n"
|
||||
"world_stage=$world_stage\n"
|
||||
"kernel_stage=$kernel_stage\n"
|
||||
"world_artifact=$world_artifact\n"
|
||||
"kernel_artifact=$kernel_artifact\n"
|
||||
"headers_artifact=$headers_artifact\n"
|
||||
"bootloader_artifact=$bootloader_artifact\n"
|
||||
"latest_link=$latest_link\n"
|
||||
"root_df=$root_df\n"
|
||||
"build_root_size=$build_root_size\n"
|
||||
"result_root_size=$result_root_size\n"
|
||||
"world_artifact_size=$world_artifact_size\n"
|
||||
"kernel_artifact_size=$kernel_artifact_size\n"
|
||||
"headers_artifact_size=$headers_artifact_size\n"
|
||||
"bootloader_artifact_size=$bootloader_artifact_size\n"
|
||||
"sha_kernel=$sha_kernel\n"
|
||||
"sha_loader=$sha_loader\n"
|
||||
"sha_param=$sha_param\n"
|
||||
"buildworld_tail=$buildworld_tail\n"
|
||||
"buildkernel_tail=$buildkernel_tail\n"
|
||||
"installworld_tail=$installworld_tail\n"
|
||||
"distribution_tail=$distribution_tail\n"
|
||||
"installkernel_tail=$installkernel_tail\n"
|
||||
"self_hosted_native_build=ok\n"
|
||||
"EOF\n"
|
||||
"printf 'ok\\n' > \"$status_file\"\n"
|
||||
"cat \"$metadata_file\"\n")))
|
||||
|
||||
|
||||
(define* (operating-system-generated-files os #:key guile-store guile-extra-store shepherd-store)
|
||||
@@ -809,7 +1026,9 @@
|
||||
(if (null? (operating-system-development-packages os))
|
||||
'()
|
||||
`(("usr/local/bin/fruix-development-environment"
|
||||
. ,(render-development-environment-script os))))
|
||||
. ,(render-development-environment-script os))
|
||||
("usr/local/bin/fruix-self-hosted-native-build"
|
||||
. ,(render-self-hosted-native-build-script os))))
|
||||
(if (pid1-init-mode? os)
|
||||
`(("boot/fruix-pid1" . ,(render-pid1-script os shepherd-store guile-store guile-extra-store)))
|
||||
'())
|
||||
|
||||
@@ -19,6 +19,7 @@
|
||||
file-hash
|
||||
directory-entries
|
||||
path-signature
|
||||
tree-content-signature
|
||||
install-plan-signature
|
||||
native-build-source-tree-sha256
|
||||
copy-regular-file
|
||||
@@ -132,6 +133,30 @@
|
||||
(else
|
||||
(string-append "other:" path ":" (symbol->string (stat:type st)))))))
|
||||
|
||||
(define (tree-content-signature root)
|
||||
(define (walk path relative)
|
||||
(let ((st (lstat path)))
|
||||
(case (stat:type st)
|
||||
((regular)
|
||||
(string-append "file:" relative ":" (file-hash path)))
|
||||
((symlink)
|
||||
(string-append "symlink:" relative ":" (readlink path)))
|
||||
((directory)
|
||||
(string-join
|
||||
(cons (string-append "directory:" relative)
|
||||
(apply append
|
||||
(map (lambda (entry)
|
||||
(let ((child-relative (if (string=? relative ".")
|
||||
entry
|
||||
(string-append relative "/" entry))))
|
||||
(list (walk (string-append path "/" entry)
|
||||
child-relative))))
|
||||
(directory-entries path))))
|
||||
"\n"))
|
||||
(else
|
||||
(string-append "other:" relative ":" (symbol->string (stat:type st)))))))
|
||||
(walk root "."))
|
||||
|
||||
(define (install-plan-signature entry)
|
||||
(match entry
|
||||
(('file source target)
|
||||
|
||||
@@ -15,6 +15,7 @@
|
||||
Commands:\n\
|
||||
system ACTION ... Build or materialize Fruix system artifacts.\n\
|
||||
source ACTION ... Fetch or snapshot declarative FreeBSD source inputs.\n\
|
||||
native-build ACTION ... Promote native build results into Fruix store objects.\n\
|
||||
\n\
|
||||
System actions:\n\
|
||||
build Materialize the Fruix system closure in /frx/store.\n\
|
||||
@@ -37,6 +38,12 @@ System options:\n\
|
||||
Source actions:\n\
|
||||
materialize Materialize a declared FreeBSD source tree in /frx/store.\n\
|
||||
\n\
|
||||
Native-build actions:\n\
|
||||
promote Promote a native build result root into /frx/store.\n\
|
||||
\n\
|
||||
Native-build options:\n\
|
||||
--store DIR Store directory to use (default: /frx/store).\n\
|
||||
\n\
|
||||
Source options:\n\
|
||||
--source NAME Scheme variable holding the freebsd-source object.\n\
|
||||
--store DIR Store directory to use (default: /frx/store).\n\
|
||||
@@ -216,6 +223,28 @@ Common options:\n\
|
||||
((arg . tail)
|
||||
(loop tail (cons arg positional) source-name store-dir cache-dir)))))
|
||||
|
||||
(define (parse-native-build-arguments action rest)
|
||||
(let loop ((args rest)
|
||||
(positional '())
|
||||
(store-dir "/frx/store"))
|
||||
(match args
|
||||
(()
|
||||
(let ((positional (reverse positional)))
|
||||
`((command . "native-build")
|
||||
(action . ,action)
|
||||
(positional . ,positional)
|
||||
(store-dir . ,store-dir))))
|
||||
(("--help")
|
||||
(usage 0))
|
||||
(((? (lambda (arg) (string-prefix? "--store=" arg)) arg) . tail)
|
||||
(loop tail positional (option-value arg "--store=")))
|
||||
(("--store" value . tail)
|
||||
(loop tail positional value))
|
||||
(((? (lambda (arg) (string-prefix? "--" arg)) arg) . _)
|
||||
(error "unknown option" arg))
|
||||
((arg . tail)
|
||||
(loop tail (cons arg positional) store-dir)))))
|
||||
|
||||
(define (parse-arguments argv)
|
||||
(match argv
|
||||
((_)
|
||||
@@ -228,10 +257,14 @@ Common options:\n\
|
||||
(usage 0))
|
||||
((_ "source" "--help")
|
||||
(usage 0))
|
||||
((_ "native-build" "--help")
|
||||
(usage 0))
|
||||
((_ "system" action . rest)
|
||||
(parse-system-arguments action rest))
|
||||
((_ "source" action . rest)
|
||||
(parse-source-arguments action rest))
|
||||
((_ "native-build" action . rest)
|
||||
(parse-native-build-arguments action rest))
|
||||
((_ . _)
|
||||
(usage 1))))
|
||||
|
||||
@@ -542,6 +575,20 @@ Common options:\n\
|
||||
(target_store_item_count . ,(length target-store-items))
|
||||
(installer_store_item_count . ,(length installer-store-items))))))
|
||||
|
||||
(define (emit-native-build-promotion-metadata store-dir result-root result)
|
||||
(emit-metadata
|
||||
`((action . "promote")
|
||||
(result_root . ,result-root)
|
||||
(store_dir . ,store-dir)
|
||||
(result_store . ,(assoc-ref result 'result-store))
|
||||
(result_metadata_file . ,(assoc-ref result 'result-metadata-file))
|
||||
(artifact_store_count . ,(assoc-ref result 'artifact-store-count))
|
||||
(artifact_stores . ,(string-join (assoc-ref result 'artifact-stores) ","))
|
||||
(world_store . ,(assoc-ref result 'world-store))
|
||||
(kernel_store . ,(assoc-ref result 'kernel-store))
|
||||
(headers_store . ,(assoc-ref result 'headers-store))
|
||||
(bootloader_store . ,(assoc-ref result 'bootloader-store)))))
|
||||
|
||||
(define (main argv)
|
||||
(let* ((parsed (parse-arguments argv))
|
||||
(command (assoc-ref parsed 'command))
|
||||
@@ -692,6 +739,16 @@ Common options:\n\
|
||||
(materialized_source_ref . ,(or (assoc-ref effective 'ref) ""))
|
||||
(materialized_source_commit . ,(or (assoc-ref result 'effective-commit) ""))
|
||||
(materialized_source_sha256 . ,(or (assoc-ref result 'effective-sha256) ""))))))))))
|
||||
((string=? command "native-build")
|
||||
(let ((positional (assoc-ref parsed 'positional)))
|
||||
(unless (string=? action "promote")
|
||||
(error "unknown native-build action" action))
|
||||
(let ((result-root (match positional
|
||||
((path . _) path)
|
||||
(() (error "missing native build result root argument")))))
|
||||
(emit-native-build-promotion-metadata
|
||||
store-dir result-root
|
||||
(promote-native-build-result result-root #:store-dir store-dir)))))
|
||||
(else
|
||||
(usage 1)))))
|
||||
|
||||
|
||||
275
tests/system/run-phase20-host-initiated-native-build-xcpng.sh
Executable file
275
tests/system/run-phase20-host-initiated-native-build-xcpng.sh
Executable file
@@ -0,0 +1,275 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
repo_root=${PROJECT_ROOT:-$(pwd)}
|
||||
os_template=${OS_TEMPLATE:-$repo_root/tests/system/phase20-development-operating-system.scm.in}
|
||||
system_name=${SYSTEM_NAME:-phase20-operating-system}
|
||||
root_size=${ROOT_SIZE:-20g}
|
||||
metadata_target=${METADATA_OUT:-}
|
||||
root_authorized_key_file=${ROOT_AUTHORIZED_KEY_FILE:-$HOME/.ssh/id_ed25519.pub}
|
||||
root_ssh_private_key_file=${ROOT_SSH_PRIVATE_KEY_FILE:-$HOME/.ssh/id_ed25519}
|
||||
cleanup=0
|
||||
|
||||
if [ -n "${WORKDIR:-}" ]; then
|
||||
workdir=$WORKDIR
|
||||
mkdir -p "$workdir"
|
||||
else
|
||||
workdir=$(mktemp -d /tmp/fruix-phase20-native-build-xcpng.XXXXXX)
|
||||
cleanup=1
|
||||
fi
|
||||
if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
|
||||
cleanup=0
|
||||
fi
|
||||
|
||||
inner_metadata=$workdir/phase20-native-build-inner-metadata.txt
|
||||
metadata_file=$workdir/phase20-host-initiated-native-build-xcpng-metadata.txt
|
||||
|
||||
action_cleanup() {
|
||||
if [ "$cleanup" -eq 1 ]; then
|
||||
rm -rf "$workdir"
|
||||
fi
|
||||
}
|
||||
trap action_cleanup EXIT INT TERM
|
||||
|
||||
KEEP_WORKDIR=1 WORKDIR="$workdir/inner" METADATA_OUT="$inner_metadata" \
|
||||
ROOT_AUTHORIZED_KEY_FILE="$root_authorized_key_file" \
|
||||
ROOT_SSH_PRIVATE_KEY_FILE="$root_ssh_private_key_file" \
|
||||
OS_TEMPLATE="$os_template" SYSTEM_NAME="$system_name" ROOT_SIZE="$root_size" \
|
||||
"$repo_root/tests/system/run-phase20-development-environment-xcpng.sh"
|
||||
|
||||
phase8_metadata=$(sed -n 's/^phase8_metadata=//p' "$inner_metadata")
|
||||
closure_path=$(sed -n 's/^closure_path=//p' "$inner_metadata")
|
||||
closure_base=$(sed -n 's/^closure_base=//p' "$inner_metadata")
|
||||
guest_ip=$(sed -n 's/^guest_ip=//p' "$inner_metadata")
|
||||
vm_id=$(sed -n 's/^vm_id=//p' "$inner_metadata")
|
||||
vdi_id=$(sed -n 's/^vdi_id=//p' "$inner_metadata")
|
||||
shepherd_pid=$(sed -n 's/^shepherd_pid=//p' "$inner_metadata")
|
||||
sshd_status=$(sed -n 's/^sshd_status=//p' "$inner_metadata")
|
||||
compat_prefix_shims=$(sed -n 's/^compat_prefix_shims=//p' "$inner_metadata")
|
||||
guile_module_smoke=$(sed -n 's/^guile_module_smoke=//p' "$inner_metadata")
|
||||
|
||||
[ "$shepherd_pid" = 1 ] || { echo "shepherd was not PID 1" >&2; exit 1; }
|
||||
[ "$sshd_status" = running ] || { echo "sshd is not running" >&2; exit 1; }
|
||||
[ "$compat_prefix_shims" = absent ] || { echo "compatibility prefix shims reappeared" >&2; exit 1; }
|
||||
[ "$guile_module_smoke" = ok ] || { echo "guest Guile module smoke failed" >&2; exit 1; }
|
||||
|
||||
ssh_guest() {
|
||||
ssh -i "$root_ssh_private_key_file" \
|
||||
-o BatchMode=yes \
|
||||
-o StrictHostKeyChecking=no \
|
||||
-o UserKnownHostsFile=/dev/null \
|
||||
-o ConnectTimeout=5 \
|
||||
root@"$guest_ip" "$@"
|
||||
}
|
||||
|
||||
guest_build_jobs=${GUEST_BUILD_JOBS:-$(ssh_guest 'sysctl -n hw.ncpu')}
|
||||
case "$guest_build_jobs" in
|
||||
''|*[!0-9]*)
|
||||
echo "invalid guest build job count: $guest_build_jobs" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
native_build_metadata=$(ssh_guest env BUILD_JOBS="$guest_build_jobs" sh -s <<'EOF'
|
||||
set -eu
|
||||
[ -L /run/current-development ]
|
||||
[ -L /usr/include ]
|
||||
[ "$(readlink /usr/include)" = "/run/current-system/development-profile/usr/include" ]
|
||||
[ -L /usr/share/mk ]
|
||||
[ "$(readlink /usr/share/mk)" = "/run/current-system/development-profile/usr/share/mk" ]
|
||||
closure=$(readlink /run/current-system)
|
||||
source_store=$(sed -n 's/.*"\(\/frx\/store\/[^\"]*-freebsd-source-[^\"]*\)".*/\1/p' "$closure/metadata/store-layout.scm" | head -n 1)
|
||||
source_root="$source_store/tree"
|
||||
build_root=/var/tmp/fruix-phase20-native-build
|
||||
logdir=$build_root/logs
|
||||
world_stage=$build_root/stage-world
|
||||
kernel_stage=$build_root/stage-kernel
|
||||
headers_stage=$build_root/artifact-headers
|
||||
bootloader_stage=$build_root/artifact-bootloader
|
||||
rm -rf "$build_root"
|
||||
mkdir -p "$logdir"
|
||||
export MAKEOBJDIRPREFIX="$build_root/obj"
|
||||
common='TARGET=amd64 TARGET_ARCH=amd64 KERNCONF=GENERIC __MAKE_CONF=/dev/null SRCCONF=/dev/null SRC_ENV_CONF=/dev/null MK_DEBUG_FILES=no MK_TESTS=no DB_FROM_SRC=yes'
|
||||
make -j"$BUILD_JOBS" -C "$source_root" TARGET=amd64 TARGET_ARCH=amd64 KERNCONF=GENERIC __MAKE_CONF=/dev/null SRCCONF=/dev/null SRC_ENV_CONF=/dev/null MK_DEBUG_FILES=no MK_TESTS=no buildworld > "$logdir/buildworld.log" 2>&1
|
||||
make -j"$BUILD_JOBS" -C "$source_root" TARGET=amd64 TARGET_ARCH=amd64 KERNCONF=GENERIC __MAKE_CONF=/dev/null SRCCONF=/dev/null SRC_ENV_CONF=/dev/null MK_DEBUG_FILES=no MK_TESTS=no buildkernel > "$logdir/buildkernel.log" 2>&1
|
||||
make -C "$source_root" $common DESTDIR="$world_stage" installworld > "$logdir/installworld.log" 2>&1
|
||||
make -C "$source_root" $common DESTDIR="$world_stage" distribution > "$logdir/distribution.log" 2>&1
|
||||
make -C "$source_root" $common DESTDIR="$kernel_stage" installkernel > "$logdir/installkernel.log" 2>&1
|
||||
mkdir -p "$headers_stage/usr" "$bootloader_stage/boot"
|
||||
cp -a "$world_stage/usr/include" "$headers_stage/usr/include"
|
||||
mkdir -p "$headers_stage/usr/share"
|
||||
cp -a "$world_stage/usr/share/mk" "$headers_stage/usr/share/mk"
|
||||
cp -a "$world_stage/boot/loader" "$bootloader_stage/boot/loader"
|
||||
cp -a "$world_stage/boot/loader.efi" "$bootloader_stage/boot/loader.efi"
|
||||
cp -a "$world_stage/boot/device.hints" "$bootloader_stage/boot/device.hints"
|
||||
cp -a "$world_stage/boot/defaults" "$bootloader_stage/boot/defaults"
|
||||
cp -a "$world_stage/boot/lua" "$bootloader_stage/boot/lua"
|
||||
[ -f "$kernel_stage/boot/kernel/kernel" ]
|
||||
[ -f "$headers_stage/usr/include/sys/param.h" ]
|
||||
[ -f "$headers_stage/usr/share/mk/bsd.prog.mk" ]
|
||||
[ -f "$bootloader_stage/boot/loader.efi" ]
|
||||
[ -f "$bootloader_stage/boot/defaults/loader.conf" ]
|
||||
[ -f "$bootloader_stage/boot/lua/loader.lua" ]
|
||||
sha_kernel=$(sha256 -q "$kernel_stage/boot/kernel/kernel")
|
||||
sha_loader=$(sha256 -q "$bootloader_stage/boot/loader.efi")
|
||||
sha_param=$(sha256 -q "$headers_stage/usr/include/sys/param.h")
|
||||
buildworld_tail=$(tail -n 20 "$logdir/buildworld.log" | tr '\n' ' ')
|
||||
buildkernel_tail=$(tail -n 20 "$logdir/buildkernel.log" | tr '\n' ' ')
|
||||
installworld_tail=$(tail -n 20 "$logdir/installworld.log" | tr '\n' ' ')
|
||||
distribution_tail=$(tail -n 20 "$logdir/distribution.log" | tr '\n' ' ')
|
||||
installkernel_tail=$(tail -n 20 "$logdir/installkernel.log" | tr '\n' ' ')
|
||||
root_df=$(df -h / | tail -n 1 | tr -s ' ' | tr '\t' ' ')
|
||||
build_root_size=$(du -sh "$build_root" | awk '{print $1}')
|
||||
world_stage_size=$(du -sh "$world_stage" | awk '{print $1}')
|
||||
kernel_stage_size=$(du -sh "$kernel_stage" | awk '{print $1}')
|
||||
headers_stage_size=$(du -sh "$headers_stage" | awk '{print $1}')
|
||||
bootloader_stage_size=$(du -sh "$bootloader_stage" | awk '{print $1}')
|
||||
printf 'build_jobs=%s\n' "$BUILD_JOBS"
|
||||
printf 'source_store=%s\n' "$source_store"
|
||||
printf 'source_root=%s\n' "$source_root"
|
||||
printf 'build_root=%s\n' "$build_root"
|
||||
printf 'logdir=%s\n' "$logdir"
|
||||
printf 'buildworld_log=%s\n' "$logdir/buildworld.log"
|
||||
printf 'buildkernel_log=%s\n' "$logdir/buildkernel.log"
|
||||
printf 'installworld_log=%s\n' "$logdir/installworld.log"
|
||||
printf 'distribution_log=%s\n' "$logdir/distribution.log"
|
||||
printf 'installkernel_log=%s\n' "$logdir/installkernel.log"
|
||||
printf 'world_stage=%s\n' "$world_stage"
|
||||
printf 'kernel_stage=%s\n' "$kernel_stage"
|
||||
printf 'headers_stage=%s\n' "$headers_stage"
|
||||
printf 'bootloader_stage=%s\n' "$bootloader_stage"
|
||||
printf 'root_df=%s\n' "$root_df"
|
||||
printf 'build_root_size=%s\n' "$build_root_size"
|
||||
printf 'world_stage_size=%s\n' "$world_stage_size"
|
||||
printf 'kernel_stage_size=%s\n' "$kernel_stage_size"
|
||||
printf 'headers_stage_size=%s\n' "$headers_stage_size"
|
||||
printf 'bootloader_stage_size=%s\n' "$bootloader_stage_size"
|
||||
printf 'sha_kernel=%s\n' "$sha_kernel"
|
||||
printf 'sha_loader=%s\n' "$sha_loader"
|
||||
printf 'sha_param=%s\n' "$sha_param"
|
||||
printf 'buildworld_tail=%s\n' "$buildworld_tail"
|
||||
printf 'buildkernel_tail=%s\n' "$buildkernel_tail"
|
||||
printf 'installworld_tail=%s\n' "$installworld_tail"
|
||||
printf 'distribution_tail=%s\n' "$distribution_tail"
|
||||
printf 'installkernel_tail=%s\n' "$installkernel_tail"
|
||||
EOF
|
||||
)
|
||||
|
||||
build_jobs=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^build_jobs=//p')
|
||||
source_store=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^source_store=//p')
|
||||
source_root=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^source_root=//p')
|
||||
build_root=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^build_root=//p')
|
||||
logdir=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^logdir=//p')
|
||||
buildworld_log=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^buildworld_log=//p')
|
||||
buildkernel_log=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^buildkernel_log=//p')
|
||||
installworld_log=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^installworld_log=//p')
|
||||
distribution_log=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^distribution_log=//p')
|
||||
installkernel_log=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^installkernel_log=//p')
|
||||
world_stage=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^world_stage=//p')
|
||||
kernel_stage=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^kernel_stage=//p')
|
||||
headers_stage=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^headers_stage=//p')
|
||||
bootloader_stage=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^bootloader_stage=//p')
|
||||
root_df=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^root_df=//p')
|
||||
build_root_size=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^build_root_size=//p')
|
||||
world_stage_size=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^world_stage_size=//p')
|
||||
kernel_stage_size=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^kernel_stage_size=//p')
|
||||
headers_stage_size=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^headers_stage_size=//p')
|
||||
bootloader_stage_size=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^bootloader_stage_size=//p')
|
||||
sha_kernel=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^sha_kernel=//p')
|
||||
sha_loader=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^sha_loader=//p')
|
||||
sha_param=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^sha_param=//p')
|
||||
buildworld_tail=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^buildworld_tail=//p')
|
||||
buildkernel_tail=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^buildkernel_tail=//p')
|
||||
installworld_tail=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^installworld_tail=//p')
|
||||
distribution_tail=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^distribution_tail=//p')
|
||||
installkernel_tail=$(printf '%s\n' "$native_build_metadata" | sed -n 's/^installkernel_tail=//p')
|
||||
|
||||
case "$source_store" in
|
||||
/frx/store/*-freebsd-source-*) : ;;
|
||||
*) echo "unexpected source store path: $source_store" >&2; exit 1 ;;
|
||||
esac
|
||||
case "$source_root" in
|
||||
/frx/store/*-freebsd-source-*/*) : ;;
|
||||
*) echo "unexpected source root path: $source_root" >&2; exit 1 ;;
|
||||
esac
|
||||
case "$build_root" in
|
||||
/var/tmp/fruix-phase20-native-build) : ;;
|
||||
*) echo "unexpected build root: $build_root" >&2; exit 1 ;;
|
||||
esac
|
||||
printf '%s\n' "$sha_kernel" | grep -E '^[0-9a-f]{64}$' >/dev/null || {
|
||||
echo "invalid kernel sha256: $sha_kernel" >&2
|
||||
exit 1
|
||||
}
|
||||
printf '%s\n' "$sha_loader" | grep -E '^[0-9a-f]{64}$' >/dev/null || {
|
||||
echo "invalid loader sha256: $sha_loader" >&2
|
||||
exit 1
|
||||
}
|
||||
printf '%s\n' "$sha_param" | grep -E '^[0-9a-f]{64}$' >/dev/null || {
|
||||
echo "invalid param.h sha256: $sha_param" >&2
|
||||
exit 1
|
||||
}
|
||||
case "$buildworld_tail" in
|
||||
*'World build completed on'*) : ;;
|
||||
*) echo "buildworld log does not show completion" >&2; exit 1 ;;
|
||||
esac
|
||||
case "$buildkernel_tail" in
|
||||
*'Kernel(s) GENERIC built in'*) : ;;
|
||||
*) echo "buildkernel log does not show successful kernel build" >&2; exit 1 ;;
|
||||
esac
|
||||
|
||||
cat >"$metadata_file" <<EOF
|
||||
workdir=$workdir
|
||||
inner_metadata=$inner_metadata
|
||||
phase8_metadata=$phase8_metadata
|
||||
closure_path=$closure_path
|
||||
closure_base=$closure_base
|
||||
vm_id=$vm_id
|
||||
vdi_id=$vdi_id
|
||||
guest_ip=$guest_ip
|
||||
root_size=$root_size
|
||||
build_jobs=$build_jobs
|
||||
source_store=$source_store
|
||||
source_root=$source_root
|
||||
build_root=$build_root
|
||||
logdir=$logdir
|
||||
buildworld_log=$buildworld_log
|
||||
buildkernel_log=$buildkernel_log
|
||||
installworld_log=$installworld_log
|
||||
distribution_log=$distribution_log
|
||||
installkernel_log=$installkernel_log
|
||||
world_stage=$world_stage
|
||||
kernel_stage=$kernel_stage
|
||||
headers_stage=$headers_stage
|
||||
bootloader_stage=$bootloader_stage
|
||||
root_df=$root_df
|
||||
build_root_size=$build_root_size
|
||||
world_stage_size=$world_stage_size
|
||||
kernel_stage_size=$kernel_stage_size
|
||||
headers_stage_size=$headers_stage_size
|
||||
bootloader_stage_size=$bootloader_stage_size
|
||||
sha_kernel=$sha_kernel
|
||||
sha_loader=$sha_loader
|
||||
sha_param=$sha_param
|
||||
buildworld_tail=$buildworld_tail
|
||||
buildkernel_tail=$buildkernel_tail
|
||||
installworld_tail=$installworld_tail
|
||||
distribution_tail=$distribution_tail
|
||||
installkernel_tail=$installkernel_tail
|
||||
boot_backend=xcp-ng-xo-cli
|
||||
init_mode=shepherd-pid1
|
||||
host_initiated_native_build=ok
|
||||
EOF
|
||||
|
||||
if [ -n "$metadata_target" ]; then
|
||||
mkdir -p "$(dirname "$metadata_target")"
|
||||
cp "$metadata_file" "$metadata_target"
|
||||
fi
|
||||
|
||||
printf 'PASS phase20-host-initiated-native-build-xcpng\n'
|
||||
printf 'Work directory: %s\n' "$workdir"
|
||||
printf 'Metadata file: %s\n' "$metadata_file"
|
||||
if [ -n "$metadata_target" ]; then
|
||||
printf 'Copied metadata to: %s\n' "$metadata_target"
|
||||
fi
|
||||
printf '%s\n' '--- metadata ---'
|
||||
cat "$metadata_file"
|
||||
231
tests/system/run-phase20-native-build-store-promotion-xcpng.sh
Executable file
231
tests/system/run-phase20-native-build-store-promotion-xcpng.sh
Executable file
@@ -0,0 +1,231 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
repo_root=${PROJECT_ROOT:-$(pwd)}
|
||||
os_template=${OS_TEMPLATE:-$repo_root/tests/system/phase20-development-operating-system.scm.in}
|
||||
system_name=${SYSTEM_NAME:-phase20-operating-system}
|
||||
root_size=${ROOT_SIZE:-20g}
|
||||
store_dir=${STORE_DIR:-/frx/store}
|
||||
metadata_target=${METADATA_OUT:-}
|
||||
root_authorized_key_file=${ROOT_AUTHORIZED_KEY_FILE:-$HOME/.ssh/id_ed25519.pub}
|
||||
root_ssh_private_key_file=${ROOT_SSH_PRIVATE_KEY_FILE:-$HOME/.ssh/id_ed25519}
|
||||
cleanup=0
|
||||
|
||||
if [ -n "${WORKDIR:-}" ]; then
|
||||
workdir=$WORKDIR
|
||||
mkdir -p "$workdir"
|
||||
else
|
||||
workdir=$(mktemp -d /tmp/fruix-phase20-native-build-store-promotion-xcpng.XXXXXX)
|
||||
cleanup=1
|
||||
fi
|
||||
if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
|
||||
cleanup=0
|
||||
fi
|
||||
|
||||
inner_metadata=$workdir/phase20-native-build-store-promotion-inner-metadata.txt
|
||||
existing_inner_metadata=${EXISTING_INNER_METADATA:-}
|
||||
promotion_out=$workdir/native-build-promote.txt
|
||||
metadata_file=$workdir/phase20-native-build-store-promotion-xcpng-metadata.txt
|
||||
import_root=$workdir/import
|
||||
|
||||
action_cleanup() {
|
||||
if [ "$cleanup" -eq 1 ]; then
|
||||
rm -rf "$workdir"
|
||||
fi
|
||||
}
|
||||
trap action_cleanup EXIT INT TERM
|
||||
|
||||
if [ -n "$existing_inner_metadata" ]; then
|
||||
cp "$existing_inner_metadata" "$inner_metadata"
|
||||
else
|
||||
KEEP_WORKDIR=1 WORKDIR="$workdir/inner" METADATA_OUT="$inner_metadata" \
|
||||
ROOT_AUTHORIZED_KEY_FILE="$root_authorized_key_file" \
|
||||
ROOT_SSH_PRIVATE_KEY_FILE="$root_ssh_private_key_file" \
|
||||
OS_TEMPLATE="$os_template" SYSTEM_NAME="$system_name" ROOT_SIZE="$root_size" \
|
||||
"$repo_root/tests/system/run-phase20-self-hosted-native-build-xcpng.sh"
|
||||
fi
|
||||
|
||||
guest_ip=$(sed -n 's/^guest_ip=//p' "$inner_metadata")
|
||||
vm_id=$(sed -n 's/^vm_id=//p' "$inner_metadata")
|
||||
vdi_id=$(sed -n 's/^vdi_id=//p' "$inner_metadata")
|
||||
closure_path=$(sed -n 's/^closure_path=//p' "$inner_metadata")
|
||||
closure_base=$(sed -n 's/^closure_base=//p' "$inner_metadata")
|
||||
run_id=$(sed -n 's/^run_id=//p' "$inner_metadata")
|
||||
source_store=$(sed -n 's/^source_store=//p' "$inner_metadata")
|
||||
result_root=$(sed -n 's/^result_root=//p' "$inner_metadata")
|
||||
promotion_file=$(sed -n 's/^promotion_file=//p' "$inner_metadata")
|
||||
world_artifact=$(sed -n 's/^world_artifact=//p' "$inner_metadata")
|
||||
kernel_artifact=$(sed -n 's/^kernel_artifact=//p' "$inner_metadata")
|
||||
headers_artifact=$(sed -n 's/^headers_artifact=//p' "$inner_metadata")
|
||||
bootloader_artifact=$(sed -n 's/^bootloader_artifact=//p' "$inner_metadata")
|
||||
sha_kernel=$(sed -n 's/^sha_kernel=//p' "$inner_metadata")
|
||||
sha_loader=$(sed -n 's/^sha_loader=//p' "$inner_metadata")
|
||||
sha_param=$(sed -n 's/^sha_param=//p' "$inner_metadata")
|
||||
|
||||
ssh_guest() {
|
||||
ssh -i "$root_ssh_private_key_file" \
|
||||
-o BatchMode=yes \
|
||||
-o StrictHostKeyChecking=no \
|
||||
-o UserKnownHostsFile=/dev/null \
|
||||
-o ConnectTimeout=5 \
|
||||
root@"$guest_ip" "$@"
|
||||
}
|
||||
|
||||
mkdir -p "$import_root"
|
||||
result_base=$(basename "$result_root")
|
||||
ssh -i "$root_ssh_private_key_file" \
|
||||
-o BatchMode=yes \
|
||||
-o StrictHostKeyChecking=no \
|
||||
-o UserKnownHostsFile=/dev/null \
|
||||
-o ConnectTimeout=5 \
|
||||
root@"$guest_ip" "tar -C '$(dirname "$result_root")' -cf - '$result_base'" | tar -C "$import_root" -xf -
|
||||
local_result_root=$import_root/$result_base
|
||||
[ -d "$local_result_root" ] || { echo "failed to import native build result root" >&2; exit 1; }
|
||||
[ -f "$local_result_root/promotion.scm" ] || { echo "imported result is missing promotion.scm" >&2; exit 1; }
|
||||
[ -f "$local_result_root/artifacts/world/bin/sh" ] || { echo "imported result is missing world artifact" >&2; exit 1; }
|
||||
[ -f "$local_result_root/artifacts/kernel/boot/kernel/kernel" ] || { echo "imported result is missing kernel artifact" >&2; exit 1; }
|
||||
[ -f "$local_result_root/artifacts/headers/usr/include/sys/param.h" ] || { echo "imported result is missing headers artifact" >&2; exit 1; }
|
||||
[ -f "$local_result_root/artifacts/bootloader/boot/loader.efi" ] || { echo "imported result is missing bootloader artifact" >&2; exit 1; }
|
||||
|
||||
action_env() {
|
||||
sudo env \
|
||||
HOME="$HOME" \
|
||||
GUILE_AUTO_COMPILE=0 \
|
||||
GUIX_SOURCE_DIR="${GUIX_SOURCE_DIR:-$HOME/repos/guix}" \
|
||||
GUILE_BIN="${GUILE_BIN:-/tmp/guile-freebsd-validate-install/bin/guile}" \
|
||||
GUILE_EXTRA_PREFIX="${GUILE_EXTRA_PREFIX:-/tmp/guile-gnutls-freebsd-validate-install}" \
|
||||
SHEPHERD_PREFIX="${SHEPHERD_PREFIX:-/tmp/shepherd-freebsd-validate-install}" \
|
||||
"$@"
|
||||
}
|
||||
|
||||
action_env "$repo_root/bin/fruix" native-build promote "$local_result_root" --store "$store_dir" >"$promotion_out"
|
||||
|
||||
field() {
|
||||
sed -n "s/^$1=//p" "$promotion_out" | tail -n 1
|
||||
}
|
||||
|
||||
result_store=$(field result_store)
|
||||
result_metadata_file=$(field result_metadata_file)
|
||||
artifact_store_count=$(field artifact_store_count)
|
||||
artifact_stores=$(field artifact_stores)
|
||||
world_store=$(field world_store)
|
||||
kernel_store=$(field kernel_store)
|
||||
headers_store=$(field headers_store)
|
||||
bootloader_store=$(field bootloader_store)
|
||||
|
||||
[ "$artifact_store_count" = 4 ] || { echo "unexpected artifact store count: $artifact_store_count" >&2; exit 1; }
|
||||
case "$result_store" in
|
||||
/frx/store/*-fruix-native-build-result-*-guest-self-hosted) : ;;
|
||||
*) echo "unexpected result store path: $result_store" >&2; exit 1 ;;
|
||||
esac
|
||||
case "$world_store" in
|
||||
/frx/store/*-fruix-native-world-*-guest-self-hosted) : ;;
|
||||
*) echo "unexpected world store path: $world_store" >&2; exit 1 ;;
|
||||
esac
|
||||
case "$kernel_store" in
|
||||
/frx/store/*-fruix-native-kernel-*-guest-self-hosted) : ;;
|
||||
*) echo "unexpected kernel store path: $kernel_store" >&2; exit 1 ;;
|
||||
esac
|
||||
case "$headers_store" in
|
||||
/frx/store/*-fruix-native-headers-*-guest-self-hosted) : ;;
|
||||
*) echo "unexpected headers store path: $headers_store" >&2; exit 1 ;;
|
||||
esac
|
||||
case "$bootloader_store" in
|
||||
/frx/store/*-fruix-native-bootloader-*-guest-self-hosted) : ;;
|
||||
*) echo "unexpected bootloader store path: $bootloader_store" >&2; exit 1 ;;
|
||||
esac
|
||||
|
||||
[ -f "$result_metadata_file" ] || { echo "missing result metadata file: $result_metadata_file" >&2; exit 1; }
|
||||
[ -f "$world_store/.fruix-native-build-object.scm" ] || { echo "missing world store metadata" >&2; exit 1; }
|
||||
[ -f "$kernel_store/.fruix-native-build-object.scm" ] || { echo "missing kernel store metadata" >&2; exit 1; }
|
||||
[ -f "$headers_store/.fruix-native-build-object.scm" ] || { echo "missing headers store metadata" >&2; exit 1; }
|
||||
[ -f "$bootloader_store/.fruix-native-build-object.scm" ] || { echo "missing bootloader store metadata" >&2; exit 1; }
|
||||
[ -L "$result_store/artifacts/world" ] || { echo "missing promoted world artifact link" >&2; exit 1; }
|
||||
[ -L "$result_store/artifacts/kernel" ] || { echo "missing promoted kernel artifact link" >&2; exit 1; }
|
||||
[ -L "$result_store/artifacts/headers" ] || { echo "missing promoted headers artifact link" >&2; exit 1; }
|
||||
[ -L "$result_store/artifacts/bootloader" ] || { echo "missing promoted bootloader artifact link" >&2; exit 1; }
|
||||
[ "$(readlink "$result_store/artifacts/world")" = "$world_store" ] || { echo "world artifact link mismatch" >&2; exit 1; }
|
||||
[ "$(readlink "$result_store/artifacts/kernel")" = "$kernel_store" ] || { echo "kernel artifact link mismatch" >&2; exit 1; }
|
||||
[ "$(readlink "$result_store/artifacts/headers")" = "$headers_store" ] || { echo "headers artifact link mismatch" >&2; exit 1; }
|
||||
[ "$(readlink "$result_store/artifacts/bootloader")" = "$bootloader_store" ] || { echo "bootloader artifact link mismatch" >&2; exit 1; }
|
||||
[ -f "$world_store/bin/sh" ] || { echo "promoted world store missing /bin/sh" >&2; exit 1; }
|
||||
[ -f "$kernel_store/boot/kernel/kernel" ] || { echo "promoted kernel store missing kernel" >&2; exit 1; }
|
||||
[ -f "$headers_store/usr/include/sys/param.h" ] || { echo "promoted headers store missing param.h" >&2; exit 1; }
|
||||
[ -f "$bootloader_store/boot/loader.efi" ] || { echo "promoted bootloader store missing loader.efi" >&2; exit 1; }
|
||||
|
||||
promoted_kernel_sha=$(sha256 -q "$kernel_store/boot/kernel/kernel")
|
||||
promoted_loader_sha=$(sha256 -q "$bootloader_store/boot/loader.efi")
|
||||
promoted_param_sha=$(sha256 -q "$headers_store/usr/include/sys/param.h")
|
||||
[ "$promoted_kernel_sha" = "$sha_kernel" ] || { echo "kernel sha mismatch after promotion" >&2; exit 1; }
|
||||
[ "$promoted_loader_sha" = "$sha_loader" ] || { echo "loader sha mismatch after promotion" >&2; exit 1; }
|
||||
[ "$promoted_param_sha" = "$sha_param" ] || { echo "param.h sha mismatch after promotion" >&2; exit 1; }
|
||||
|
||||
grep -F '(executor . "guest-self-hosted")' "$result_metadata_file" >/dev/null || {
|
||||
echo "result metadata file is missing guest-self-hosted executor" >&2
|
||||
exit 1
|
||||
}
|
||||
grep -F "$source_store" "$result_metadata_file" >/dev/null || {
|
||||
echo "result metadata file is missing source store provenance" >&2
|
||||
exit 1
|
||||
}
|
||||
grep -F '(artifact-kind . kernel)' "$kernel_store/.fruix-native-build-object.scm" >/dev/null || {
|
||||
echo "kernel store metadata is missing artifact kind" >&2
|
||||
exit 1
|
||||
}
|
||||
grep -F '(artifact-kind . world)' "$world_store/.fruix-native-build-object.scm" >/dev/null || {
|
||||
echo "world store metadata is missing artifact kind" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
cat >"$metadata_file" <<EOF
|
||||
workdir=$workdir
|
||||
inner_metadata=$inner_metadata
|
||||
promotion_out=$promotion_out
|
||||
closure_path=$closure_path
|
||||
closure_base=$closure_base
|
||||
vm_id=$vm_id
|
||||
vdi_id=$vdi_id
|
||||
guest_ip=$guest_ip
|
||||
root_size=$root_size
|
||||
run_id=$run_id
|
||||
source_store=$source_store
|
||||
guest_result_root=$result_root
|
||||
guest_promotion_file=$promotion_file
|
||||
guest_world_artifact=$world_artifact
|
||||
guest_kernel_artifact=$kernel_artifact
|
||||
guest_headers_artifact=$headers_artifact
|
||||
guest_bootloader_artifact=$bootloader_artifact
|
||||
local_result_root=$local_result_root
|
||||
store_dir=$store_dir
|
||||
result_store=$result_store
|
||||
result_metadata_file=$result_metadata_file
|
||||
artifact_store_count=$artifact_store_count
|
||||
artifact_stores=$artifact_stores
|
||||
world_store=$world_store
|
||||
kernel_store=$kernel_store
|
||||
headers_store=$headers_store
|
||||
bootloader_store=$bootloader_store
|
||||
sha_kernel=$sha_kernel
|
||||
sha_loader=$sha_loader
|
||||
sha_param=$sha_param
|
||||
promoted_kernel_sha=$promoted_kernel_sha
|
||||
promoted_loader_sha=$promoted_loader_sha
|
||||
promoted_param_sha=$promoted_param_sha
|
||||
boot_backend=xcp-ng-xo-cli
|
||||
init_mode=shepherd-pid1
|
||||
native_build_store_promotion=ok
|
||||
EOF
|
||||
|
||||
if [ -n "$metadata_target" ]; then
|
||||
mkdir -p "$(dirname "$metadata_target")"
|
||||
cp "$metadata_file" "$metadata_target"
|
||||
fi
|
||||
|
||||
printf 'PASS phase20-native-build-store-promotion-xcpng\n'
|
||||
printf 'Work directory: %s\n' "$workdir"
|
||||
printf 'Metadata file: %s\n' "$metadata_file"
|
||||
if [ -n "$metadata_target" ]; then
|
||||
printf 'Copied metadata to: %s\n' "$metadata_target"
|
||||
fi
|
||||
printf '%s\n' '--- metadata ---'
|
||||
cat "$metadata_file"
|
||||
252
tests/system/run-phase20-self-hosted-native-build-xcpng.sh
Executable file
252
tests/system/run-phase20-self-hosted-native-build-xcpng.sh
Executable file
@@ -0,0 +1,252 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
repo_root=${PROJECT_ROOT:-$(pwd)}
|
||||
os_template=${OS_TEMPLATE:-$repo_root/tests/system/phase20-development-operating-system.scm.in}
|
||||
system_name=${SYSTEM_NAME:-phase20-operating-system}
|
||||
root_size=${ROOT_SIZE:-20g}
|
||||
metadata_target=${METADATA_OUT:-}
|
||||
root_authorized_key_file=${ROOT_AUTHORIZED_KEY_FILE:-$HOME/.ssh/id_ed25519.pub}
|
||||
root_ssh_private_key_file=${ROOT_SSH_PRIVATE_KEY_FILE:-$HOME/.ssh/id_ed25519}
|
||||
cleanup=0
|
||||
|
||||
if [ -n "${WORKDIR:-}" ]; then
|
||||
workdir=$WORKDIR
|
||||
mkdir -p "$workdir"
|
||||
else
|
||||
workdir=$(mktemp -d /tmp/fruix-phase20-self-hosted-native-build-xcpng.XXXXXX)
|
||||
cleanup=1
|
||||
fi
|
||||
if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
|
||||
cleanup=0
|
||||
fi
|
||||
|
||||
inner_metadata=$workdir/phase20-self-hosted-inner-metadata.txt
|
||||
metadata_file=$workdir/phase20-self-hosted-native-build-xcpng-metadata.txt
|
||||
|
||||
action_cleanup() {
|
||||
if [ "$cleanup" -eq 1 ]; then
|
||||
rm -rf "$workdir"
|
||||
fi
|
||||
}
|
||||
trap action_cleanup EXIT INT TERM
|
||||
|
||||
KEEP_WORKDIR=1 WORKDIR="$workdir/inner" METADATA_OUT="$inner_metadata" \
|
||||
ROOT_AUTHORIZED_KEY_FILE="$root_authorized_key_file" \
|
||||
ROOT_SSH_PRIVATE_KEY_FILE="$root_ssh_private_key_file" \
|
||||
OS_TEMPLATE="$os_template" SYSTEM_NAME="$system_name" ROOT_SIZE="$root_size" \
|
||||
"$repo_root/tests/system/run-phase20-development-environment-xcpng.sh"
|
||||
|
||||
phase8_metadata=$(sed -n 's/^phase8_metadata=//p' "$inner_metadata")
|
||||
closure_path=$(sed -n 's/^closure_path=//p' "$inner_metadata")
|
||||
closure_base=$(sed -n 's/^closure_base=//p' "$inner_metadata")
|
||||
guest_ip=$(sed -n 's/^guest_ip=//p' "$inner_metadata")
|
||||
vm_id=$(sed -n 's/^vm_id=//p' "$inner_metadata")
|
||||
vdi_id=$(sed -n 's/^vdi_id=//p' "$inner_metadata")
|
||||
shepherd_pid=$(sed -n 's/^shepherd_pid=//p' "$inner_metadata")
|
||||
sshd_status=$(sed -n 's/^sshd_status=//p' "$inner_metadata")
|
||||
compat_prefix_shims=$(sed -n 's/^compat_prefix_shims=//p' "$inner_metadata")
|
||||
guile_module_smoke=$(sed -n 's/^guile_module_smoke=//p' "$inner_metadata")
|
||||
|
||||
[ "$shepherd_pid" = 1 ] || { echo "shepherd was not PID 1" >&2; exit 1; }
|
||||
[ "$sshd_status" = running ] || { echo "sshd is not running" >&2; exit 1; }
|
||||
[ "$compat_prefix_shims" = absent ] || { echo "compatibility prefix shims reappeared" >&2; exit 1; }
|
||||
[ "$guile_module_smoke" = ok ] || { echo "guest Guile module smoke failed" >&2; exit 1; }
|
||||
|
||||
ssh_guest() {
|
||||
ssh -i "$root_ssh_private_key_file" \
|
||||
-o BatchMode=yes \
|
||||
-o StrictHostKeyChecking=no \
|
||||
-o UserKnownHostsFile=/dev/null \
|
||||
-o ConnectTimeout=5 \
|
||||
root@"$guest_ip" "$@"
|
||||
}
|
||||
|
||||
guest_build_jobs=${GUEST_BUILD_JOBS:-$(ssh_guest 'sysctl -n hw.ncpu')}
|
||||
case "$guest_build_jobs" in
|
||||
''|*[!0-9]*)
|
||||
echo "invalid guest build job count: $guest_build_jobs" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
ssh_guest '[ -x /usr/local/bin/fruix-self-hosted-native-build ]'
|
||||
ssh_guest '[ -L /usr/include ]'
|
||||
ssh_guest '[ -L /usr/share/mk ]'
|
||||
|
||||
self_hosted_metadata=$(ssh_guest env FRUIX_SELF_HOSTED_NATIVE_BUILD_JOBS="$guest_build_jobs" /usr/local/bin/fruix-self-hosted-native-build)
|
||||
|
||||
run_id=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^run_id=//p')
|
||||
helper_version=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^helper_version=//p')
|
||||
source_store=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^source_store=//p')
|
||||
source_root=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^source_root=//p')
|
||||
build_jobs=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^build_jobs=//p')
|
||||
build_common=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^build_common=//p')
|
||||
install_common=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^install_common=//p')
|
||||
build_root=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^build_root=//p')
|
||||
result_root=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^result_root=//p')
|
||||
logdir=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^logdir=//p')
|
||||
status_file=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^status_file=//p')
|
||||
guest_metadata_file=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^metadata_file=//p')
|
||||
promotion_file=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^promotion_file=//p')
|
||||
world_stage=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^world_stage=//p')
|
||||
kernel_stage=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^kernel_stage=//p')
|
||||
world_artifact=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^world_artifact=//p')
|
||||
kernel_artifact=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^kernel_artifact=//p')
|
||||
headers_artifact=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^headers_artifact=//p')
|
||||
bootloader_artifact=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^bootloader_artifact=//p')
|
||||
latest_link=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^latest_link=//p')
|
||||
root_df=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^root_df=//p')
|
||||
build_root_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^build_root_size=//p')
|
||||
result_root_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^result_root_size=//p')
|
||||
world_artifact_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^world_artifact_size=//p')
|
||||
kernel_artifact_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^kernel_artifact_size=//p')
|
||||
headers_artifact_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^headers_artifact_size=//p')
|
||||
bootloader_artifact_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^bootloader_artifact_size=//p')
|
||||
sha_kernel=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^sha_kernel=//p')
|
||||
sha_loader=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^sha_loader=//p')
|
||||
sha_param=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^sha_param=//p')
|
||||
buildworld_tail=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^buildworld_tail=//p')
|
||||
buildkernel_tail=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^buildkernel_tail=//p')
|
||||
installworld_tail=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^installworld_tail=//p')
|
||||
distribution_tail=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^distribution_tail=//p')
|
||||
installkernel_tail=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^installkernel_tail=//p')
|
||||
self_hosted_native_build=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^self_hosted_native_build=//p')
|
||||
|
||||
status_value=$(ssh_guest "cat '$status_file'")
|
||||
latest_target=$(ssh_guest "readlink '$latest_link'")
|
||||
ssh_guest "[ -f '$promotion_file' ]"
|
||||
ssh_guest "[ -f '$world_artifact/bin/sh' ]"
|
||||
|
||||
[ "$helper_version" = 3 ] || { echo "unexpected helper version: $helper_version" >&2; exit 1; }
|
||||
[ "$build_jobs" = "$guest_build_jobs" ] || { echo "unexpected build job count: $build_jobs" >&2; exit 1; }
|
||||
[ "$status_value" = ok ] || { echo "self-hosted build status is not ok: $status_value" >&2; exit 1; }
|
||||
[ "$latest_target" = "$result_root" ] || { echo "latest link target mismatch: $latest_target" >&2; exit 1; }
|
||||
[ "$self_hosted_native_build" = ok ] || { echo "self-hosted build marker missing" >&2; exit 1; }
|
||||
|
||||
case "$source_store" in
|
||||
/frx/store/*-freebsd-source-*) : ;;
|
||||
*) echo "unexpected source store path: $source_store" >&2; exit 1 ;;
|
||||
esac
|
||||
case "$source_root" in
|
||||
/frx/store/*-freebsd-source-*/*) : ;;
|
||||
*) echo "unexpected source root path: $source_root" >&2; exit 1 ;;
|
||||
esac
|
||||
case "$build_root" in
|
||||
/var/tmp/fruix-self-hosted-native-builds/*) : ;;
|
||||
*) echo "unexpected build root: $build_root" >&2; exit 1 ;;
|
||||
esac
|
||||
case "$result_root" in
|
||||
/var/lib/fruix/native-builds/*) : ;;
|
||||
*) echo "unexpected result root: $result_root" >&2; exit 1 ;;
|
||||
esac
|
||||
case "$latest_link" in
|
||||
/var/lib/fruix/native-builds/latest) : ;;
|
||||
*) echo "unexpected latest link path: $latest_link" >&2; exit 1 ;;
|
||||
esac
|
||||
printf '%s\n' "$run_id" | grep -E '^[0-9]{8}T[0-9]{6}Z$' >/dev/null || {
|
||||
echo "unexpected run id: $run_id" >&2
|
||||
exit 1
|
||||
}
|
||||
printf '%s\n' "$sha_kernel" | grep -E '^[0-9a-f]{64}$' >/dev/null || {
|
||||
echo "invalid kernel sha256: $sha_kernel" >&2
|
||||
exit 1
|
||||
}
|
||||
printf '%s\n' "$sha_loader" | grep -E '^[0-9a-f]{64}$' >/dev/null || {
|
||||
echo "invalid loader sha256: $sha_loader" >&2
|
||||
exit 1
|
||||
}
|
||||
printf '%s\n' "$sha_param" | grep -E '^[0-9a-f]{64}$' >/dev/null || {
|
||||
echo "invalid param.h sha256: $sha_param" >&2
|
||||
exit 1
|
||||
}
|
||||
case "$buildworld_tail" in
|
||||
*'World build completed on'*) : ;;
|
||||
*) echo "buildworld log does not show completion" >&2; exit 1 ;;
|
||||
esac
|
||||
case "$buildkernel_tail" in
|
||||
*'Kernel(s) GENERIC built in'*) : ;;
|
||||
*) echo "buildkernel log does not show successful kernel build" >&2; exit 1 ;;
|
||||
esac
|
||||
case "$installworld_tail" in
|
||||
*'Install world completed in'*) : ;;
|
||||
*) echo "installworld log does not show successful completion" >&2; exit 1 ;;
|
||||
esac
|
||||
case "$installkernel_tail" in
|
||||
*'Install kernel(s) GENERIC completed in'*) : ;;
|
||||
*) echo "installkernel log does not show successful completion" >&2; exit 1 ;;
|
||||
esac
|
||||
printf '%s\n' "$build_common" | grep -F 'TARGET=amd64 TARGET_ARCH=amd64 KERNCONF=GENERIC' >/dev/null || {
|
||||
echo "unexpected build_common flags: $build_common" >&2
|
||||
exit 1
|
||||
}
|
||||
printf '%s\n' "$install_common" | grep -F 'DB_FROM_SRC=yes' >/dev/null || {
|
||||
echo "install_common is missing DB_FROM_SRC=yes" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
cat >"$metadata_file" <<EOF
|
||||
workdir=$workdir
|
||||
inner_metadata=$inner_metadata
|
||||
phase8_metadata=$phase8_metadata
|
||||
closure_path=$closure_path
|
||||
closure_base=$closure_base
|
||||
vm_id=$vm_id
|
||||
vdi_id=$vdi_id
|
||||
guest_ip=$guest_ip
|
||||
root_size=$root_size
|
||||
run_id=$run_id
|
||||
helper_version=$helper_version
|
||||
build_jobs=$build_jobs
|
||||
source_store=$source_store
|
||||
source_root=$source_root
|
||||
build_common=$build_common
|
||||
install_common=$install_common
|
||||
build_root=$build_root
|
||||
result_root=$result_root
|
||||
logdir=$logdir
|
||||
status_file=$status_file
|
||||
guest_metadata_file=$guest_metadata_file
|
||||
promotion_file=$promotion_file
|
||||
world_stage=$world_stage
|
||||
kernel_stage=$kernel_stage
|
||||
world_artifact=$world_artifact
|
||||
kernel_artifact=$kernel_artifact
|
||||
headers_artifact=$headers_artifact
|
||||
bootloader_artifact=$bootloader_artifact
|
||||
latest_link=$latest_link
|
||||
latest_target=$latest_target
|
||||
status_value=$status_value
|
||||
root_df=$root_df
|
||||
build_root_size=$build_root_size
|
||||
result_root_size=$result_root_size
|
||||
world_artifact_size=$world_artifact_size
|
||||
kernel_artifact_size=$kernel_artifact_size
|
||||
headers_artifact_size=$headers_artifact_size
|
||||
bootloader_artifact_size=$bootloader_artifact_size
|
||||
sha_kernel=$sha_kernel
|
||||
sha_loader=$sha_loader
|
||||
sha_param=$sha_param
|
||||
buildworld_tail=$buildworld_tail
|
||||
buildkernel_tail=$buildkernel_tail
|
||||
installworld_tail=$installworld_tail
|
||||
distribution_tail=$distribution_tail
|
||||
installkernel_tail=$installkernel_tail
|
||||
boot_backend=xcp-ng-xo-cli
|
||||
init_mode=shepherd-pid1
|
||||
self_hosted_native_build=ok
|
||||
EOF
|
||||
|
||||
if [ -n "$metadata_target" ]; then
|
||||
mkdir -p "$(dirname "$metadata_target")"
|
||||
cp "$metadata_file" "$metadata_target"
|
||||
fi
|
||||
|
||||
printf 'PASS phase20-self-hosted-native-build-xcpng\n'
|
||||
printf 'Work directory: %s\n' "$workdir"
|
||||
printf 'Metadata file: %s\n' "$metadata_file"
|
||||
if [ -n "$metadata_target" ]; then
|
||||
printf 'Copied metadata to: %s\n' "$metadata_target"
|
||||
fi
|
||||
printf '%s\n' '--- metadata ---'
|
||||
cat "$metadata_file"
|
||||
Reference in New Issue
Block a user