self/fruix
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

native-build: promote results into store objects

Browse Source
This commit is contained in:
Steffen
2026-04-06 01:16:44 +02:00
parent 4614592a25
commit 006ffee615
12 changed files with 863 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
docs/GUIX_DIFFERENCES.md
View File

@@ -274,6 +274,26 @@ The practical Fruix takeaway is:
- the development overlay makes native base work possible inside the system - the development overlay makes native base work possible inside the system
- but real guest self-hosted base builds still need their own stricter build contract - but real guest self-hosted base builds still need their own stricter build contract
Fruix now also makes an explicit distinction between:
- mutable native-build staging/results under:
- `/var/lib/fruix/native-builds`
- immutable promoted native-build identities under:
- `/frx/store`
So a guest self-hosted run can stage a result tree locally, while the host can later promote that result into first-class Fruix store objects such as:
- `/frx/store/...-fruix-native-world-...`
- `/frx/store/...-fruix-native-kernel-...`
- `/frx/store/...-fruix-native-headers-...`
- `/frx/store/...-fruix-native-bootloader-...`
- `/frx/store/...-fruix-native-build-result-...`
Compared with Guix, this is a more explicit split between:
- a mutable result/staging area for native build execution
- and the immutable store identities that Fruix treats as the real promoted result
## Where Fruix is intentionally trying to improve on Guix's representation ## Where Fruix is intentionally trying to improve on Guix's representation
Fruix is not trying to improve on Guix's core semantics. Guix already got those right. Fruix is not trying to improve on Guix's core semantics. Guix already got those right.

60
docs/PROGRESS.md
View File

@@ -46,6 +46,13 @@ Fruix currently has:
- `/usr/local/bin/fruix-self-hosted-native-build` - `/usr/local/bin/fruix-self-hosted-native-build`
- result roots under `/var/lib/fruix/native-builds` - result roots under `/var/lib/fruix/native-builds`
- in-guest source recovery from current-system metadata - in-guest source recovery from current-system metadata
- a validated promotion path that turns native-build result roots into first-class Fruix store objects via:
- `fruix native-build promote`
- `/frx/store/...-fruix-native-world-...`
- `/frx/store/...-fruix-native-kernel-...`
- `/frx/store/...-fruix-native-headers-...`
- `/frx/store/...-fruix-native-bootloader-...`
- `/frx/store/...-fruix-native-build-result-...`
Validated boot modes still are: Validated boot modes still are:
@@ -58,32 +65,39 @@ The validated Phase 18 installation work currently uses:
## Latest completed achievement ## Latest completed achievement
### 2026-04-05 — Phase 20.3 completed ### 2026-04-05 — Native base builds promoted into first-class Fruix store objects
Fruix now has a validated first controlled guest self-hosted native base-build prototype, on top of the already validated host-initiated in-guest build path. Fruix now has a validated end-to-end path that treats guest native base-build results as **staged mutable results first**, and then promotes them into **immutable Fruix store identities**.
Highlights: Highlights:
- development-enabled systems now ship an in-guest helper at: - guest self-hosted runs still record staged results under:
- `/usr/local/bin/fruix-self-hosted-native-build`
- the helper recovers the declared materialized FreeBSD source store from:
- `/run/current-system/metadata/store-layout.scm`
- the helper records self-hosted build results under:
- `/var/lib/fruix/native-builds/<run-id>` - `/var/lib/fruix/native-builds/<run-id>`
- `/var/lib/fruix/native-builds/latest` - `/var/lib/fruix/native-builds/latest`
- heavy object/stage work remains under: - those result roots now carry promotion metadata describing:
- `/var/tmp/fruix-self-hosted-native-builds/<run-id>` - executor / executor-version
- the prototype exposed an important contract detail: - closure path
- a naive reuse of the development-shell exports polluted `buildworld` - source store provenance
- the validated helper therefore sanitizes development-oriented variables such as `MAKEFLAGS`, `CPPFLAGS`, `CFLAGS`, `CXXFLAGS`, and `LDFLAGS` before world/kernel bootstrap - build policy
- the validated result includes guest-recorded native artifact outputs for: - artifact entries for:
- kernel - `world`
- bootloader slice - `kernel`
- headers / `usr/share/mk` - `headers`
- `bootloader`
- the host can now run:
- `fruix native-build promote RESULT_ROOT`
- promotion creates immutable `/frx/store` objects for:
- `world`
- `kernel`
- `headers`
- `bootloader`
- promotion also creates a result-bundle store object that references those artifact stores
- the validated promotion metadata now makes Fruix-native native-build identity explicit instead of leaving results only as ad hoc files under `/var/lib/fruix/native-builds/...`
Validation: Validation:
- `PASS phase20-self-hosted-native-build-xcpng` - `PASS phase20-self-hosted-native-build-xcpng`
- `PASS phase20-native-build-store-promotion-xcpng`
Reports: Reports:
@@ -92,6 +106,7 @@ Reports:
- `docs/reports/phase20-development-environment-freebsd.md` - `docs/reports/phase20-development-environment-freebsd.md`
- `docs/reports/phase20-host-initiated-native-builds-freebsd.md` - `docs/reports/phase20-host-initiated-native-builds-freebsd.md`
- `docs/reports/phase20-self-hosted-native-builds-freebsd.md` - `docs/reports/phase20-self-hosted-native-builds-freebsd.md`
- `docs/reports/phase20-native-build-store-promotion-freebsd.md`
## Recent major milestones ## Recent major milestones
@@ -117,9 +132,14 @@ Reports:
`docs/PLAN_4.md` currently ends at Phase 20.3, and that milestone sequence is now complete. `docs/PLAN_4.md` currently ends at Phase 20.3, and that milestone sequence is now complete.
The next practical follow-up is therefore no longer another planned `PLAN_4` phase, but a product decision: The next practical follow-up is now clearer:
- whether to keep the new guest self-hosted helper as a narrow prototype while Phase 20.2 remains the default operator path - unify host-initiated and self-hosted native-build execution behind a shared Fruix executor/result model
- or whether to invest in a broader guest-driven Fruix-native native-build workflow - make the same first-class promotion story available regardless of whether the outer loop is host-driven or guest-driven
- decide how much of result import/promotion should remain host-side versus become a more integrated Fruix deployment action
Phase 20.3 is now complete: Fruix validates a first controlled guest self-hosted native FreeBSD base-build prototype on the approved real XCP-ng path. The immediate architectural direction is no longer just “can guest self-hosting work?”
It is now:
- how should Fruix represent native base builds as real first-class objects and workflows across different executors?

190
docs/reports/phase20-native-build-store-promotion-freebsd.md Normal file
View File

@@ -0,0 +1,190 @@
# Post-Phase 20: native build result promotion into first-class Fruix store objects
Date: 2026-04-05
## Goal
Make native FreeBSD base-build results feel like real Fruix objects instead of stopping at mutable staged files under:
- `/var/lib/fruix/native-builds/...`
The desired model is:
- `/var/lib/fruix/native-builds/...` remains a staging/result area
- `/frx/store/...` remains the real immutable identity
Validated artifact identities:
- `world`
- `kernel`
- `headers`
- `bootloader`
## What changed
### Promotion metadata in guest result roots
The guest self-hosted helper now emits a promotion description file at:
- `/var/lib/fruix/native-builds/<run-id>/promotion.scm`
That metadata records at least:
- executor / executor-version
- run-id / guest-host-name
- closure path
- development profile path
- declared FreeBSD base metadata
- source store provenance
- build policy
- artifact entries for:
- `world`
- `kernel`
- `headers`
- `bootloader`
The helper also stages a promotable `world` artifact tree in addition to the already validated narrower artifacts.
### Host-side promotion API
Fruix now exports:
- `promote-native-build-result`
and the CLI now exposes:
- `fruix native-build promote RESULT_ROOT [--store DIR]`
### Promoted store object layout
Promotion now creates immutable store objects for:
- `/frx/store/...-fruix-native-world-...`
- `/frx/store/...-fruix-native-kernel-...`
- `/frx/store/...-fruix-native-headers-...`
- `/frx/store/...-fruix-native-bootloader-...`
Each promoted artifact store records:
- `.fruix-native-build-object.scm`
- `.references`
Promotion also creates a result-bundle store object:
- `/frx/store/...-fruix-native-build-result-...`
That bundle records:
- `.fruix-native-build-result.scm`
- `artifacts/world`
- `artifacts/kernel`
- `artifacts/headers`
- `artifacts/bootloader`
where the `artifacts/*` entries are symlinks to the promoted artifact stores.
### Identity policy
Artifact identity is now based on Fruix metadata plus a tree-content signature of the staged artifact tree.
That means promotion identity depends on both:
- the explicit Fruix-native build metadata
- the actual content of the promoted artifact tree
## Validation harness
Added:
- `tests/system/run-phase20-native-build-store-promotion-xcpng.sh`
This harness:
1. boots the approved real XCP-ng guest path
2. runs the validated in-guest self-hosted native build helper
3. imports the guest result root back to the host
4. runs `fruix native-build promote`
5. verifies promoted store paths, metadata, symlink structure, and representative hashes
## Validation
Passing run:
- `PASS phase20-native-build-store-promotion-xcpng`
- workdir: `/tmp/current-phase20-native-build-store-promotion-xcpng`
Approved real XCP-ng path:
- VM `90490f2e-e8fc-4b7a-388e-5c26f0157289`
- VDI `0f1f90d3-48ca-4fa2-91d8-fc6339b95743`
Representative metadata:
```text
run_id=20260405T213444Z
source_store=/frx/store/12d7704362e95afc2697db63f168b878e082b372-freebsd-source-default
guest_result_root=/var/lib/fruix/native-builds/20260405T213444Z
result_store=/frx/store/c6329a0053720b05aff3274b8b1d522c909f475d-fruix-native-build-result-15.0-STABLE-guest-self-hosted
world_store=/frx/store/dfe37b36f6537a95ceea16ea62001b2ca5617eb7-fruix-native-world-15.0-STABLE-guest-self-hosted
kernel_store=/frx/store/0ab7cbceca240ab2c3b91e83e059844ea792e49e-fruix-native-kernel-15.0-STABLE-guest-self-hosted
headers_store=/frx/store/5bbeae9266687a229f1c6d176a08886c35243ff0-fruix-native-headers-15.0-STABLE-guest-self-hosted
bootloader_store=/frx/store/bd49a508bd7a3b94a2535d6774f31c993c406552-fruix-native-bootloader-15.0-STABLE-guest-self-hosted
artifact_store_count=4
sha_kernel=16950f116a52134b98e2f8e0dacc556e18fe254e4a0ac2c1741422dde281a341
sha_loader=ea417846167ece270ada611624dca622ca38bd30125b9a125cd8ebb8b3600313
sha_param=9eb140ca7d9666f3d484a4174c9acd94b45427db6292b4e17de19af2c6aa5219
promoted_kernel_sha=16950f116a52134b98e2f8e0dacc556e18fe254e4a0ac2c1741422dde281a341
promoted_loader_sha=ea417846167ece270ada611624dca622ca38bd30125b9a125cd8ebb8b3600313
promoted_param_sha=9eb140ca7d9666f3d484a4174c9acd94b45427db6292b4e17de19af2c6aa5219
native_build_store_promotion=ok
```
Validated facts:
- guest self-hosted native-build results remain available under `/var/lib/fruix/native-builds/...` as mutable staging/results
- those staged results are now sufficient to promote first-class Fruix identities on the host
- promotion creates four immutable artifact store objects plus one immutable result-bundle store object
- promoted metadata retains executor/source/build-policy/provenance information explicitly
- promoted kernel, bootloader, and headers hashes match the already validated staged artifacts
- the promoted `world` artifact is now also preserved as a first-class Fruix store object instead of remaining only as an unpromoted staged tree
## Important implementation note
The first full promotion attempt exposed an incorrect assumption in the validation surface:
- the promoted/staged `world` artifact should be checked at:
- `bin/sh`
- not at:
- `usr/bin/sh`
That path expectation was corrected in:
- helper validation
- emitted promotion metadata
- the end-to-end promotion harness
## Result
Fruix now has a validated native-build object model with a clear split:
- mutable native-build result roots under `/var/lib/fruix/native-builds/...`
- immutable promoted identities under `/frx/store/...`
That makes native FreeBSD base builds feel substantially more Fruix-native:
- build outputs have explicit immutable identities
- metadata is Fruix-native rather than implied only by ad hoc directory layout
- executor/source/provenance/build-policy remain attached to the promoted result
- the staged result area and the real store identity are now intentionally distinct
## Next direction
This suggests the next product step is not merely “more self-hosting”.
It is to generalize this result model so different execution modes can converge on the same promoted object story, for example:
- host-initiated in-guest builds
- guest self-hosted builds
- future executor variants
That would move Fruix closer to a shared executor model rather than treating each validation path as a one-off harness.

63
docs/system-deployment-workflow.md
View File

@@ -278,10 +278,15 @@ That helper:
2. recovers the materialized FreeBSD source store from: 2. recovers the materialized FreeBSD source store from:
- `/run/current-system/metadata/store-layout.scm` - `/run/current-system/metadata/store-layout.scm`
3. runs the native FreeBSD build/install phases inside the guest 3. runs the native FreeBSD build/install phases inside the guest
4. records results under: 4. records staged results under:
- `/var/lib/fruix/native-builds/<run-id>` - `/var/lib/fruix/native-builds/<run-id>`
- `/var/lib/fruix/native-builds/latest` - `/var/lib/fruix/native-builds/latest`
5. keeps the heavier object/stage work under: 5. emits promotion metadata for first-class artifact identities covering:
- `world`
- `kernel`
- `headers`
- `bootloader`
6. keeps the heavier object/stage work under:
- `/var/tmp/fruix-self-hosted-native-builds/<run-id>` - `/var/tmp/fruix-self-hosted-native-builds/<run-id>`
Important current detail: Important current detail:
@@ -294,6 +299,60 @@ So the validated Phase 20.3 answer is:
- a controlled guest self-hosted base-build prototype now works - a controlled guest self-hosted base-build prototype now works
- but the simpler default operator flow should still be the Phase 20.2 host-initiated in-guest path unless there is a specific reason to push the build loop farther into the guest - but the simpler default operator flow should still be the Phase 20.2 host-initiated in-guest path unless there is a specific reason to push the build loop farther into the guest
### Promoting native-build results into first-class Fruix store objects
The guest-side result root is now explicitly a **staging/result area**, not the final immutable identity.
Current validated flow:
1. run the in-guest helper so the guest records a result under:
- `/var/lib/fruix/native-builds/<run-id>`
2. copy that result root back to the host
3. run:
```sh
fruix native-build promote RESULT_ROOT
```
The promotion step creates immutable `/frx/store` identities for:
- `world`
- `kernel`
- `headers`
- `bootloader`
and also creates a result-bundle store object that references those promoted artifact stores.
Current metadata split:
- mutable staging/result root:
- `/var/lib/fruix/native-builds/<run-id>`
- immutable artifact stores:
- `/frx/store/...-fruix-native-world-...`
- `/frx/store/...-fruix-native-kernel-...`
- `/frx/store/...-fruix-native-headers-...`
- `/frx/store/...-fruix-native-bootloader-...`
- immutable result bundle:
- `/frx/store/...-fruix-native-build-result-...`
The promoted store objects record explicit Fruix-native metadata including at least:
- executor / executor-version
- run-id / guest-host-name
- closure path
- source store provenance
- build policy
- artifact kind
- required-file expectations
- recorded content signatures and hashes
This is the current Fruix-native answer to the question:
- where should mutable native-build state live?
- `/var/lib/fruix/native-builds/...`
- where should immutable native-build identity live?
- `/frx/store/...`
## Deployment patterns ## Deployment patterns
### 1. Build-first workflow ### 1. Build-first workflow

2
modules/fruix/system/freebsd.scm
View File

@@ -1,6 +1,7 @@
(define-module (fruix system freebsd) (define-module (fruix system freebsd)
#:use-module (fruix system freebsd model) #:use-module (fruix system freebsd model)
#:use-module (fruix system freebsd source) #:use-module (fruix system freebsd source)
#:use-module (fruix system freebsd build)
#:use-module (fruix system freebsd media) #:use-module (fruix system freebsd media)
#:re-export (user-group #:re-export (user-group
user-group? user-group?
@@ -43,6 +44,7 @@
operating-system-root-authorized-keys operating-system-root-authorized-keys
validate-operating-system validate-operating-system
materialize-freebsd-source materialize-freebsd-source
promote-native-build-result
operating-system-closure-spec operating-system-closure-spec
operating-system-install-spec operating-system-install-spec
operating-system-image-spec operating-system-image-spec

177
modules/fruix/system/freebsd/build.scm
View File

@@ -11,6 +11,7 @@
#:use-module (srfi srfi-13) #:use-module (srfi srfi-13)
#:export (host-freebsd-provenance #:export (host-freebsd-provenance
materialize-freebsd-package materialize-freebsd-package
promote-native-build-result
materialize-prefix)) materialize-prefix))
(define (host-freebsd-provenance) (define (host-freebsd-provenance)
@@ -388,6 +389,182 @@
output-path)))) output-path))))
(define native-build-result-promotion-version "1")
(define (native-build-result-ref data key default)
(match (assoc key data)
((_ . value) value)
(#f default)))
(define (read-native-build-result result-root)
(let ((promotion-file (string-append result-root "/promotion.scm")))
(unless (file-exists? promotion-file)
(error "native build result is missing promotion.scm" result-root))
(let ((result (call-with-input-file promotion-file read)))
(unless (equal? (native-build-result-ref result 'native-build-result-version #f)
native-build-result-promotion-version)
(error "unsupported native build result promotion version" promotion-file))
result)))
(define (native-build-artifact-entry result artifact-kind)
(let* ((artifacts (native-build-result-ref result 'artifacts '()))
(entry (assoc artifact-kind artifacts)))
(unless entry
(error "native build result is missing artifact entry" artifact-kind))
(cdr entry)))
(define (native-build-artifact-root result-root result artifact-kind)
(let* ((entry (native-build-artifact-entry result artifact-kind))
(relative-path (native-build-result-ref entry 'path #f))
(required-file (native-build-result-ref entry 'required-file #f))
(artifact-root (and relative-path
(string-append result-root "/" relative-path))))
(unless (and artifact-root (file-exists? artifact-root))
(error "native build result is missing artifact tree" artifact-kind artifact-root))
(when required-file
(unless (file-exists? (string-append artifact-root "/" required-file))
(error "native build artifact is missing required file"
artifact-kind
(string-append artifact-root "/" required-file))))
artifact-root))
(define (native-build-existing-store-references result store-dir)
(filter identity
(map (lambda (path)
(and (string? path)
(string-prefix? (string-append store-dir "/") path)
(file-exists? path)
path))
(list (native-build-result-ref result 'closure-path #f)
(let ((source (native-build-result-ref result 'source '())))
(native-build-result-ref source 'store-path #f))))))
(define (native-build-artifact-display-name result artifact-kind)
(let* ((base (native-build-result-ref result 'freebsd-base '()))
(version-label (native-build-result-ref base 'version-label "unknown"))
(executor (native-build-result-ref result 'executor "unknown")))
(string-append "fruix-native-"
(symbol->string artifact-kind)
"-"
version-label
"-"
executor)))
(define (native-build-promoted-artifact-metadata result artifact-kind content-signature)
(let* ((entry (native-build-artifact-entry result artifact-kind)))
`((native-build-object-version . ,native-build-result-promotion-version)
(object-kind . artifact)
(artifact-kind . ,artifact-kind)
(executor . ,(native-build-result-ref result 'executor "unknown"))
(executor-version . ,(native-build-result-ref result 'executor-version "unknown"))
(run-id . ,(native-build-result-ref result 'run-id "unknown"))
(guest-host-name . ,(native-build-result-ref result 'guest-host-name "unknown"))
(closure-path . ,(native-build-result-ref result 'closure-path ""))
(development-profile . ,(native-build-result-ref result 'development-profile ""))
(freebsd-base . ,(native-build-result-ref result 'freebsd-base '()))
(source . ,(native-build-result-ref result 'source '()))
(build-policy . ,(native-build-result-ref result 'build-policy '()))
(required-file . ,(native-build-result-ref entry 'required-file ""))
(recorded-sha256 . ,(native-build-result-ref entry 'recorded-sha256 ""))
(content-signature . ,content-signature))))
(define (promote-native-build-artifact result-root result store-dir artifact-kind)
(let* ((artifact-root (native-build-artifact-root result-root result artifact-kind))
(content-signature (tree-content-signature artifact-root))
(metadata (native-build-promoted-artifact-metadata result artifact-kind content-signature))
(payload (object->string metadata))
(display-name (native-build-artifact-display-name result artifact-kind))
(output-path (make-store-path store-dir display-name payload
#:kind 'native-build-artifact
#:output artifact-kind))
(references (native-build-existing-store-references result store-dir)))
(unless (file-exists? output-path)
(mkdir-p output-path)
(stage-tree-into-output artifact-root output-path)
(write-file (string-append output-path "/.references")
(string-join references "\n"))
(write-file (string-append output-path "/.fruix-native-build-object.scm")
payload))
`((artifact-kind . ,artifact-kind)
(artifact-root . ,artifact-root)
(store-path . ,output-path)
(content-signature . ,content-signature)
(metadata-file . ,(string-append output-path "/.fruix-native-build-object.scm")))) )
(define (native-build-result-display-name result)
(let* ((base (native-build-result-ref result 'freebsd-base '()))
(version-label (native-build-result-ref base 'version-label "unknown"))
(executor (native-build-result-ref result 'executor "unknown")))
(string-append "fruix-native-build-result-" version-label "-" executor)))
(define (native-build-promoted-result-object result promoted-artifacts)
`((native-build-result-version . ,native-build-result-promotion-version)
(object-kind . result-bundle)
(executor . ,(native-build-result-ref result 'executor "unknown"))
(executor-version . ,(native-build-result-ref result 'executor-version "unknown"))
(run-id . ,(native-build-result-ref result 'run-id "unknown"))
(guest-host-name . ,(native-build-result-ref result 'guest-host-name "unknown"))
(closure-path . ,(native-build-result-ref result 'closure-path ""))
(development-profile . ,(native-build-result-ref result 'development-profile ""))
(freebsd-base . ,(native-build-result-ref result 'freebsd-base '()))
(source . ,(native-build-result-ref result 'source '()))
(build-policy . ,(native-build-result-ref result 'build-policy '()))
(artifact-count . ,(length promoted-artifacts))
(artifacts . ,(map (lambda (entry)
`((artifact-kind . ,(assoc-ref entry 'artifact-kind))
(store-path . ,(assoc-ref entry 'store-path))
(content-signature . ,(assoc-ref entry 'content-signature))
(metadata-file . ,(assoc-ref entry 'metadata-file))))
promoted-artifacts))))
(define* (promote-native-build-result result-root #:key (store-dir "/frx/store"))
(let* ((result (read-native-build-result result-root))
(promoted-artifacts (map (lambda (artifact-kind)
(promote-native-build-artifact result-root result store-dir artifact-kind))
'(world kernel headers bootloader)))
(result-object (native-build-promoted-result-object result promoted-artifacts))
(payload (object->string result-object))
(display-name (native-build-result-display-name result))
(result-store (make-store-path store-dir display-name payload
#:kind 'native-build-result))
(result-references (append (map (lambda (entry)
(assoc-ref entry 'store-path))
promoted-artifacts)
(native-build-existing-store-references result store-dir))))
(unless (file-exists? result-store)
(mkdir-p (string-append result-store "/artifacts"))
(for-each (lambda (entry)
(symlink (assoc-ref entry 'store-path)
(string-append result-store
"/artifacts/"
(symbol->string (assoc-ref entry 'artifact-kind)))))
promoted-artifacts)
(write-file (string-append result-store "/.references")
(string-join result-references "\n"))
(write-file (string-append result-store "/.fruix-native-build-result.scm")
payload))
`((result-root . ,result-root)
(result-store . ,result-store)
(result-metadata-file . ,(string-append result-store "/.fruix-native-build-result.scm"))
(artifact-store-count . ,(length promoted-artifacts))
(artifact-stores . ,(map (lambda (entry) (assoc-ref entry 'store-path)) promoted-artifacts))
(world-store . ,(assoc-ref (find (lambda (entry)
(eq? (assoc-ref entry 'artifact-kind) 'world))
promoted-artifacts)
'store-path))
(kernel-store . ,(assoc-ref (find (lambda (entry)
(eq? (assoc-ref entry 'artifact-kind) 'kernel))
promoted-artifacts)
'store-path))
(headers-store . ,(assoc-ref (find (lambda (entry)
(eq? (assoc-ref entry 'artifact-kind) 'headers))
promoted-artifacts)
'store-path))
(bootloader-store . ,(assoc-ref (find (lambda (entry)
(eq? (assoc-ref entry 'artifact-kind) 'bootloader))
promoted-artifacts)
'store-path)))))
(define (sanitize-materialized-prefix name output-path) (define (sanitize-materialized-prefix name output-path)
(cond (cond
((string=? name "fruix-guile-extra") ((string=? name "fruix-guile-extra")

2
modules/fruix/system/freebsd/model.scm
View File

@@ -334,7 +334,7 @@
(development-environment-helper-version (development-environment-helper-version
. ,(if (null? (operating-system-development-packages os)) #f "1")) . ,(if (null? (operating-system-development-packages os)) #f "1"))
(self-hosted-native-build-helper-version (self-hosted-native-build-helper-version
. ,(if (null? (operating-system-development-packages os)) #f "2")) . ,(if (null? (operating-system-development-packages os)) #f "3"))
(user-count . ,(length (operating-system-users os))) (user-count . ,(length (operating-system-users os)))
(users . ,(map user-account-name (operating-system-users os))) (users . ,(map user-account-name (operating-system-users os)))
(group-count . ,(length (operating-system-groups os))) (group-count . ,(length (operating-system-groups os)))

52
modules/fruix/system/freebsd/render.scm
View File

@@ -787,6 +787,11 @@
"EOF\n")) "EOF\n"))
(define (render-self-hosted-native-build-script os) (define (render-self-hosted-native-build-script os)
(let* ((base-spec (freebsd-base-spec (operating-system-freebsd-base os))) (let* ((base-spec (freebsd-base-spec (operating-system-freebsd-base os)))
(base-name (assoc-ref base-spec 'name))
(version-label (assoc-ref base-spec 'version-label))
(release (assoc-ref base-spec 'release))
(branch (assoc-ref base-spec 'branch))
(declared-source-root (assoc-ref base-spec 'source-root))
(target (assoc-ref base-spec 'target)) (target (assoc-ref base-spec 'target))
(target-arch (assoc-ref base-spec 'target-arch)) (target-arch (assoc-ref base-spec 'target-arch))
(kernconf (assoc-ref base-spec 'kernconf)) (kernconf (assoc-ref base-spec 'kernconf))
@@ -803,6 +808,7 @@
"set -eu\n" "set -eu\n"
"umask 022\n" "umask 022\n"
"profile=/run/current-system/development-profile\n" "profile=/run/current-system/development-profile\n"
"guest_host_name='" (operating-system-host-name os) "'\n"
"[ -d \"$profile\" ] || {\n" "[ -d \"$profile\" ] || {\n"
" echo \"fruix-self-hosted-native-build: development profile is not available\" >&2\n" " echo \"fruix-self-hosted-native-build: development profile is not available\" >&2\n"
" exit 1\n" " exit 1\n"
@@ -846,8 +852,10 @@
"logdir=$result_root/logs\n" "logdir=$result_root/logs\n"
"status_file=$result_root/status\n" "status_file=$result_root/status\n"
"metadata_file=$result_root/metadata.txt\n" "metadata_file=$result_root/metadata.txt\n"
"promotion_file=$result_root/promotion.scm\n"
"world_stage=$build_root/stage-world\n" "world_stage=$build_root/stage-world\n"
"kernel_stage=$build_root/stage-kernel\n" "kernel_stage=$build_root/stage-kernel\n"
"world_artifact=$result_root/artifacts/world\n"
"headers_artifact=$result_root/artifacts/headers\n" "headers_artifact=$result_root/artifacts/headers\n"
"kernel_artifact=$result_root/artifacts/kernel\n" "kernel_artifact=$result_root/artifacts/kernel\n"
"bootloader_artifact=$result_root/artifacts/bootloader\n" "bootloader_artifact=$result_root/artifacts/bootloader\n"
@@ -877,13 +885,14 @@
" echo \"fruix-self-hosted-native-build: source root is missing: $source_root\" >&2\n" " echo \"fruix-self-hosted-native-build: source root is missing: $source_root\" >&2\n"
" exit 1\n" " exit 1\n"
"}\n" "}\n"
"mkdir -p \"$headers_artifact/usr\" \"$kernel_artifact/boot\" \"$bootloader_artifact/boot\"\n" "mkdir -p \"$world_artifact\" \"$headers_artifact/usr\" \"$kernel_artifact/boot\" \"$bootloader_artifact/boot\"\n"
"export MAKEOBJDIRPREFIX=\"$build_root/obj\"\n" "export MAKEOBJDIRPREFIX=\"$build_root/obj\"\n"
"make -j\"$jobs\" -C \"$source_root\" " build-common " buildworld > \"$logdir/buildworld.log\" 2>&1\n" "make -j\"$jobs\" -C \"$source_root\" " build-common " buildworld > \"$logdir/buildworld.log\" 2>&1\n"
"make -j\"$jobs\" -C \"$source_root\" " build-common " buildkernel > \"$logdir/buildkernel.log\" 2>&1\n" "make -j\"$jobs\" -C \"$source_root\" " build-common " buildkernel > \"$logdir/buildkernel.log\" 2>&1\n"
"make -C \"$source_root\" " install-common " DESTDIR=\"$world_stage\" installworld > \"$logdir/installworld.log\" 2>&1\n" "make -C \"$source_root\" " install-common " DESTDIR=\"$world_stage\" installworld > \"$logdir/installworld.log\" 2>&1\n"
"make -C \"$source_root\" " install-common " DESTDIR=\"$world_stage\" distribution > \"$logdir/distribution.log\" 2>&1\n" "make -C \"$source_root\" " install-common " DESTDIR=\"$world_stage\" distribution > \"$logdir/distribution.log\" 2>&1\n"
"make -C \"$source_root\" " install-common " DESTDIR=\"$kernel_stage\" installkernel > \"$logdir/installkernel.log\" 2>&1\n" "make -C \"$source_root\" " install-common " DESTDIR=\"$kernel_stage\" installkernel > \"$logdir/installkernel.log\" 2>&1\n"
"cp -a \"$world_stage/.\" \"$world_artifact/\"\n"
"cp -a \"$kernel_stage/boot/kernel\" \"$kernel_artifact/boot/kernel\"\n" "cp -a \"$kernel_stage/boot/kernel\" \"$kernel_artifact/boot/kernel\"\n"
"cp -a \"$world_stage/usr/include\" \"$headers_artifact/usr/include\"\n" "cp -a \"$world_stage/usr/include\" \"$headers_artifact/usr/include\"\n"
"mkdir -p \"$headers_artifact/usr/share\"\n" "mkdir -p \"$headers_artifact/usr/share\"\n"
@@ -893,6 +902,7 @@
"cp -a \"$world_stage/boot/device.hints\" \"$bootloader_artifact/boot/device.hints\"\n" "cp -a \"$world_stage/boot/device.hints\" \"$bootloader_artifact/boot/device.hints\"\n"
"cp -a \"$world_stage/boot/defaults\" \"$bootloader_artifact/boot/defaults\"\n" "cp -a \"$world_stage/boot/defaults\" \"$bootloader_artifact/boot/defaults\"\n"
"cp -a \"$world_stage/boot/lua\" \"$bootloader_artifact/boot/lua\"\n" "cp -a \"$world_stage/boot/lua\" \"$bootloader_artifact/boot/lua\"\n"
"[ -f \"$world_artifact/bin/sh\" ]\n"
"[ -f \"$kernel_artifact/boot/kernel/kernel\" ]\n" "[ -f \"$kernel_artifact/boot/kernel/kernel\" ]\n"
"[ -f \"$headers_artifact/usr/include/sys/param.h\" ]\n" "[ -f \"$headers_artifact/usr/include/sys/param.h\" ]\n"
"[ -f \"$headers_artifact/usr/share/mk/bsd.prog.mk\" ]\n" "[ -f \"$headers_artifact/usr/share/mk/bsd.prog.mk\" ]\n"
@@ -910,15 +920,50 @@
"root_df=$(df -h / | tail -n 1 | tr -s ' ' | tr '\\t' ' ')\n" "root_df=$(df -h / | tail -n 1 | tr -s ' ' | tr '\\t' ' ')\n"
"build_root_size=$(du -sh \"$build_root\" | awk '{print $1}')\n" "build_root_size=$(du -sh \"$build_root\" | awk '{print $1}')\n"
"result_root_size=$(du -sh \"$result_root\" | awk '{print $1}')\n" "result_root_size=$(du -sh \"$result_root\" | awk '{print $1}')\n"
"world_artifact_size=$(du -sh \"$world_artifact\" | awk '{print $1}')\n"
"kernel_artifact_size=$(du -sh \"$kernel_artifact\" | awk '{print $1}')\n" "kernel_artifact_size=$(du -sh \"$kernel_artifact\" | awk '{print $1}')\n"
"headers_artifact_size=$(du -sh \"$headers_artifact\" | awk '{print $1}')\n" "headers_artifact_size=$(du -sh \"$headers_artifact\" | awk '{print $1}')\n"
"bootloader_artifact_size=$(du -sh \"$bootloader_artifact\" | awk '{print $1}')\n" "bootloader_artifact_size=$(du -sh \"$bootloader_artifact\" | awk '{print $1}')\n"
"rm -f \"$latest_link\"\n" "rm -f \"$latest_link\"\n"
"ln -s \"$result_root\" \"$latest_link\"\n" "ln -s \"$result_root\" \"$latest_link\"\n"
"cat >\"$promotion_file\" <<EOF\n"
"((native-build-result-version . \"1\")\n"
" (executor . \"guest-self-hosted\")\n"
" (executor-version . \"3\")\n"
" (run-id . \"$run_id\")\n"
" (guest-host-name . \"$guest_host_name\")\n"
" (closure-path . \"$closure\")\n"
" (development-profile . \"$profile\")\n"
" (freebsd-base . ((name . \"" base-name "\")\n"
" (version-label . \"" version-label "\")\n"
" (release . \"" release "\")\n"
" (branch . \"" branch "\")\n"
" (source-root . \"" declared-source-root "\")\n"
" (target . \"" target "\")\n"
" (target-arch . \"" target-arch "\")\n"
" (kernconf . \"" kernconf "\")))\n"
" (source . ((store-path . \"$source_store\")\n"
" (source-root . \"$source_root\")))\n"
" (build-policy . ((jobs . \"$jobs\")\n"
" (build-common . \"" build-common "\")\n"
" (install-common . \"" install-common "\")))\n"
" (artifacts . ((world . ((path . \"artifacts/world\")\n"
" (required-file . \"bin/sh\")))\n"
" (kernel . ((path . \"artifacts/kernel\")\n"
" (required-file . \"boot/kernel/kernel\")\n"
" (recorded-sha256 . \"$sha_kernel\")))\n"
" (headers . ((path . \"artifacts/headers\")\n"
" (required-file . \"usr/include/sys/param.h\")\n"
" (recorded-sha256 . \"$sha_param\")))\n"
" (bootloader . ((path . \"artifacts/bootloader\")\n"
" (required-file . \"boot/loader.efi\")\n"
" (recorded-sha256 . \"$sha_loader\"))))))\n"
"EOF\n"
"cat >\"$metadata_file\" <<EOF\n" "cat >\"$metadata_file\" <<EOF\n"
"run_id=$run_id\n" "run_id=$run_id\n"
"helper_version=2\n" "helper_version=3\n"
"closure_path=$closure\n" "closure_path=$closure\n"
"guest_host_name=$guest_host_name\n"
"development_profile=$profile\n" "development_profile=$profile\n"
"source_store=$source_store\n" "source_store=$source_store\n"
"source_root=$source_root\n" "source_root=$source_root\n"
@@ -930,8 +975,10 @@
"logdir=$logdir\n" "logdir=$logdir\n"
"status_file=$status_file\n" "status_file=$status_file\n"
"metadata_file=$metadata_file\n" "metadata_file=$metadata_file\n"
"promotion_file=$promotion_file\n"
"world_stage=$world_stage\n" "world_stage=$world_stage\n"
"kernel_stage=$kernel_stage\n" "kernel_stage=$kernel_stage\n"
"world_artifact=$world_artifact\n"
"kernel_artifact=$kernel_artifact\n" "kernel_artifact=$kernel_artifact\n"
"headers_artifact=$headers_artifact\n" "headers_artifact=$headers_artifact\n"
"bootloader_artifact=$bootloader_artifact\n" "bootloader_artifact=$bootloader_artifact\n"
@@ -939,6 +986,7 @@
"root_df=$root_df\n" "root_df=$root_df\n"
"build_root_size=$build_root_size\n" "build_root_size=$build_root_size\n"
"result_root_size=$result_root_size\n" "result_root_size=$result_root_size\n"
"world_artifact_size=$world_artifact_size\n"
"kernel_artifact_size=$kernel_artifact_size\n" "kernel_artifact_size=$kernel_artifact_size\n"
"headers_artifact_size=$headers_artifact_size\n" "headers_artifact_size=$headers_artifact_size\n"
"bootloader_artifact_size=$bootloader_artifact_size\n" "bootloader_artifact_size=$bootloader_artifact_size\n"

25
modules/fruix/system/freebsd/utils.scm
View File

@@ -19,6 +19,7 @@
file-hash file-hash
directory-entries directory-entries
path-signature path-signature
tree-content-signature
install-plan-signature install-plan-signature
native-build-source-tree-sha256 native-build-source-tree-sha256
copy-regular-file copy-regular-file
@@ -132,6 +133,30 @@
(else (else
(string-append "other:" path ":" (symbol->string (stat:type st))))))) (string-append "other:" path ":" (symbol->string (stat:type st)))))))
(define (tree-content-signature root)
(define (walk path relative)
(let ((st (lstat path)))
(case (stat:type st)
((regular)
(string-append "file:" relative ":" (file-hash path)))
((symlink)
(string-append "symlink:" relative ":" (readlink path)))
((directory)
(string-join
(cons (string-append "directory:" relative)
(apply append
(map (lambda (entry)
(let ((child-relative (if (string=? relative ".")
entry
(string-append relative "/" entry))))
(list (walk (string-append path "/" entry)
child-relative))))
(directory-entries path))))
"\n"))
(else
(string-append "other:" relative ":" (symbol->string (stat:type st)))))))
(walk root "."))
(define (install-plan-signature entry) (define (install-plan-signature entry)
(match entry (match entry
(('file source target) (('file source target)

57
scripts/fruix.scm
View File

@@ -15,6 +15,7 @@
Commands:\n\ Commands:\n\
system ACTION ... Build or materialize Fruix system artifacts.\n\ system ACTION ... Build or materialize Fruix system artifacts.\n\
source ACTION ... Fetch or snapshot declarative FreeBSD source inputs.\n\ source ACTION ... Fetch or snapshot declarative FreeBSD source inputs.\n\
native-build ACTION ... Promote native build results into Fruix store objects.\n\
\n\ \n\
System actions:\n\ System actions:\n\
build Materialize the Fruix system closure in /frx/store.\n\ build Materialize the Fruix system closure in /frx/store.\n\
@@ -37,6 +38,12 @@ System options:\n\
Source actions:\n\ Source actions:\n\
materialize Materialize a declared FreeBSD source tree in /frx/store.\n\ materialize Materialize a declared FreeBSD source tree in /frx/store.\n\
\n\ \n\
Native-build actions:\n\
promote Promote a native build result root into /frx/store.\n\
\n\
Native-build options:\n\
--store DIR Store directory to use (default: /frx/store).\n\
\n\
Source options:\n\ Source options:\n\
--source NAME Scheme variable holding the freebsd-source object.\n\ --source NAME Scheme variable holding the freebsd-source object.\n\
--store DIR Store directory to use (default: /frx/store).\n\ --store DIR Store directory to use (default: /frx/store).\n\
@@ -216,6 +223,28 @@ Common options:\n\
((arg . tail) ((arg . tail)
(loop tail (cons arg positional) source-name store-dir cache-dir))))) (loop tail (cons arg positional) source-name store-dir cache-dir)))))
(define (parse-native-build-arguments action rest)
(let loop ((args rest)
(positional '())
(store-dir "/frx/store"))
(match args
(()
(let ((positional (reverse positional)))
`((command . "native-build")
(action . ,action)
(positional . ,positional)
(store-dir . ,store-dir))))
(("--help")
(usage 0))
(((? (lambda (arg) (string-prefix? "--store=" arg)) arg) . tail)
(loop tail positional (option-value arg "--store=")))
(("--store" value . tail)
(loop tail positional value))
(((? (lambda (arg) (string-prefix? "--" arg)) arg) . _)
(error "unknown option" arg))
((arg . tail)
(loop tail (cons arg positional) store-dir)))))
(define (parse-arguments argv) (define (parse-arguments argv)
(match argv (match argv
((_) ((_)
@@ -228,10 +257,14 @@ Common options:\n\
(usage 0)) (usage 0))
((_ "source" "--help") ((_ "source" "--help")
(usage 0)) (usage 0))
((_ "native-build" "--help")
(usage 0))
((_ "system" action . rest) ((_ "system" action . rest)
(parse-system-arguments action rest)) (parse-system-arguments action rest))
((_ "source" action . rest) ((_ "source" action . rest)
(parse-source-arguments action rest)) (parse-source-arguments action rest))
((_ "native-build" action . rest)
(parse-native-build-arguments action rest))
((_ . _) ((_ . _)
(usage 1)))) (usage 1))))
@@ -542,6 +575,20 @@ Common options:\n\
(target_store_item_count . ,(length target-store-items)) (target_store_item_count . ,(length target-store-items))
(installer_store_item_count . ,(length installer-store-items)))))) (installer_store_item_count . ,(length installer-store-items))))))
(define (emit-native-build-promotion-metadata store-dir result-root result)
(emit-metadata
`((action . "promote")
(result_root . ,result-root)
(store_dir . ,store-dir)
(result_store . ,(assoc-ref result 'result-store))
(result_metadata_file . ,(assoc-ref result 'result-metadata-file))
(artifact_store_count . ,(assoc-ref result 'artifact-store-count))
(artifact_stores . ,(string-join (assoc-ref result 'artifact-stores) ","))
(world_store . ,(assoc-ref result 'world-store))
(kernel_store . ,(assoc-ref result 'kernel-store))
(headers_store . ,(assoc-ref result 'headers-store))
(bootloader_store . ,(assoc-ref result 'bootloader-store)))))
(define (main argv) (define (main argv)
(let* ((parsed (parse-arguments argv)) (let* ((parsed (parse-arguments argv))
(command (assoc-ref parsed 'command)) (command (assoc-ref parsed 'command))
@@ -692,6 +739,16 @@ Common options:\n\
(materialized_source_ref . ,(or (assoc-ref effective 'ref) "")) (materialized_source_ref . ,(or (assoc-ref effective 'ref) ""))
(materialized_source_commit . ,(or (assoc-ref result 'effective-commit) "")) (materialized_source_commit . ,(or (assoc-ref result 'effective-commit) ""))
(materialized_source_sha256 . ,(or (assoc-ref result 'effective-sha256) "")))))))))) (materialized_source_sha256 . ,(or (assoc-ref result 'effective-sha256) ""))))))))))
((string=? command "native-build")
(let ((positional (assoc-ref parsed 'positional)))
(unless (string=? action "promote")
(error "unknown native-build action" action))
(let ((result-root (match positional
((path . _) path)
(() (error "missing native build result root argument")))))
(emit-native-build-promotion-metadata
store-dir result-root
(promote-native-build-result result-root #:store-dir store-dir)))))
(else (else
(usage 1))))) (usage 1)))))

231
tests/system/run-phase20-native-build-store-promotion-xcpng.sh Executable file
View File

@@ -0,0 +1,231 @@
#!/bin/sh
set -eu
repo_root=${PROJECT_ROOT:-$(pwd)}
os_template=${OS_TEMPLATE:-$repo_root/tests/system/phase20-development-operating-system.scm.in}
system_name=${SYSTEM_NAME:-phase20-operating-system}
root_size=${ROOT_SIZE:-20g}
store_dir=${STORE_DIR:-/frx/store}
metadata_target=${METADATA_OUT:-}
root_authorized_key_file=${ROOT_AUTHORIZED_KEY_FILE:-$HOME/.ssh/id_ed25519.pub}
root_ssh_private_key_file=${ROOT_SSH_PRIVATE_KEY_FILE:-$HOME/.ssh/id_ed25519}
cleanup=0
if [ -n "${WORKDIR:-}" ]; then
workdir=$WORKDIR
mkdir -p "$workdir"
else
workdir=$(mktemp -d /tmp/fruix-phase20-native-build-store-promotion-xcpng.XXXXXX)
cleanup=1
fi
if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
cleanup=0
fi
inner_metadata=$workdir/phase20-native-build-store-promotion-inner-metadata.txt
existing_inner_metadata=${EXISTING_INNER_METADATA:-}
promotion_out=$workdir/native-build-promote.txt
metadata_file=$workdir/phase20-native-build-store-promotion-xcpng-metadata.txt
import_root=$workdir/import
action_cleanup() {
if [ "$cleanup" -eq 1 ]; then
rm -rf "$workdir"
fi
}
trap action_cleanup EXIT INT TERM
if [ -n "$existing_inner_metadata" ]; then
cp "$existing_inner_metadata" "$inner_metadata"
else
KEEP_WORKDIR=1 WORKDIR="$workdir/inner" METADATA_OUT="$inner_metadata" \
ROOT_AUTHORIZED_KEY_FILE="$root_authorized_key_file" \
ROOT_SSH_PRIVATE_KEY_FILE="$root_ssh_private_key_file" \
OS_TEMPLATE="$os_template" SYSTEM_NAME="$system_name" ROOT_SIZE="$root_size" \
"$repo_root/tests/system/run-phase20-self-hosted-native-build-xcpng.sh"
fi
guest_ip=$(sed -n 's/^guest_ip=//p' "$inner_metadata")
vm_id=$(sed -n 's/^vm_id=//p' "$inner_metadata")
vdi_id=$(sed -n 's/^vdi_id=//p' "$inner_metadata")
closure_path=$(sed -n 's/^closure_path=//p' "$inner_metadata")
closure_base=$(sed -n 's/^closure_base=//p' "$inner_metadata")
run_id=$(sed -n 's/^run_id=//p' "$inner_metadata")
source_store=$(sed -n 's/^source_store=//p' "$inner_metadata")
result_root=$(sed -n 's/^result_root=//p' "$inner_metadata")
promotion_file=$(sed -n 's/^promotion_file=//p' "$inner_metadata")
world_artifact=$(sed -n 's/^world_artifact=//p' "$inner_metadata")
kernel_artifact=$(sed -n 's/^kernel_artifact=//p' "$inner_metadata")
headers_artifact=$(sed -n 's/^headers_artifact=//p' "$inner_metadata")
bootloader_artifact=$(sed -n 's/^bootloader_artifact=//p' "$inner_metadata")
sha_kernel=$(sed -n 's/^sha_kernel=//p' "$inner_metadata")
sha_loader=$(sed -n 's/^sha_loader=//p' "$inner_metadata")
sha_param=$(sed -n 's/^sha_param=//p' "$inner_metadata")
ssh_guest() {
ssh -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=5 \
root@"$guest_ip" "$@"
}
mkdir -p "$import_root"
result_base=$(basename "$result_root")
ssh -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=5 \
root@"$guest_ip" "tar -C '$(dirname "$result_root")' -cf - '$result_base'" | tar -C "$import_root" -xf -
local_result_root=$import_root/$result_base
[ -d "$local_result_root" ] || { echo "failed to import native build result root" >&2; exit 1; }
[ -f "$local_result_root/promotion.scm" ] || { echo "imported result is missing promotion.scm" >&2; exit 1; }
[ -f "$local_result_root/artifacts/world/bin/sh" ] || { echo "imported result is missing world artifact" >&2; exit 1; }
[ -f "$local_result_root/artifacts/kernel/boot/kernel/kernel" ] || { echo "imported result is missing kernel artifact" >&2; exit 1; }
[ -f "$local_result_root/artifacts/headers/usr/include/sys/param.h" ] || { echo "imported result is missing headers artifact" >&2; exit 1; }
[ -f "$local_result_root/artifacts/bootloader/boot/loader.efi" ] || { echo "imported result is missing bootloader artifact" >&2; exit 1; }
action_env() {
sudo env \
HOME="$HOME" \
GUILE_AUTO_COMPILE=0 \
GUIX_SOURCE_DIR="${GUIX_SOURCE_DIR:-$HOME/repos/guix}" \
GUILE_BIN="${GUILE_BIN:-/tmp/guile-freebsd-validate-install/bin/guile}" \
GUILE_EXTRA_PREFIX="${GUILE_EXTRA_PREFIX:-/tmp/guile-gnutls-freebsd-validate-install}" \
SHEPHERD_PREFIX="${SHEPHERD_PREFIX:-/tmp/shepherd-freebsd-validate-install}" \
"$@"
}
action_env "$repo_root/bin/fruix" native-build promote "$local_result_root" --store "$store_dir" >"$promotion_out"
field() {
sed -n "s/^$1=//p" "$promotion_out" | tail -n 1
}
result_store=$(field result_store)
result_metadata_file=$(field result_metadata_file)
artifact_store_count=$(field artifact_store_count)
artifact_stores=$(field artifact_stores)
world_store=$(field world_store)
kernel_store=$(field kernel_store)
headers_store=$(field headers_store)
bootloader_store=$(field bootloader_store)
[ "$artifact_store_count" = 4 ] || { echo "unexpected artifact store count: $artifact_store_count" >&2; exit 1; }
case "$result_store" in
/frx/store/*-fruix-native-build-result-*-guest-self-hosted) : ;;
*) echo "unexpected result store path: $result_store" >&2; exit 1 ;;
esac
case "$world_store" in
/frx/store/*-fruix-native-world-*-guest-self-hosted) : ;;
*) echo "unexpected world store path: $world_store" >&2; exit 1 ;;
esac
case "$kernel_store" in
/frx/store/*-fruix-native-kernel-*-guest-self-hosted) : ;;
*) echo "unexpected kernel store path: $kernel_store" >&2; exit 1 ;;
esac
case "$headers_store" in
/frx/store/*-fruix-native-headers-*-guest-self-hosted) : ;;
*) echo "unexpected headers store path: $headers_store" >&2; exit 1 ;;
esac
case "$bootloader_store" in
/frx/store/*-fruix-native-bootloader-*-guest-self-hosted) : ;;
*) echo "unexpected bootloader store path: $bootloader_store" >&2; exit 1 ;;
esac
[ -f "$result_metadata_file" ] || { echo "missing result metadata file: $result_metadata_file" >&2; exit 1; }
[ -f "$world_store/.fruix-native-build-object.scm" ] || { echo "missing world store metadata" >&2; exit 1; }
[ -f "$kernel_store/.fruix-native-build-object.scm" ] || { echo "missing kernel store metadata" >&2; exit 1; }
[ -f "$headers_store/.fruix-native-build-object.scm" ] || { echo "missing headers store metadata" >&2; exit 1; }
[ -f "$bootloader_store/.fruix-native-build-object.scm" ] || { echo "missing bootloader store metadata" >&2; exit 1; }
[ -L "$result_store/artifacts/world" ] || { echo "missing promoted world artifact link" >&2; exit 1; }
[ -L "$result_store/artifacts/kernel" ] || { echo "missing promoted kernel artifact link" >&2; exit 1; }
[ -L "$result_store/artifacts/headers" ] || { echo "missing promoted headers artifact link" >&2; exit 1; }
[ -L "$result_store/artifacts/bootloader" ] || { echo "missing promoted bootloader artifact link" >&2; exit 1; }
[ "$(readlink "$result_store/artifacts/world")" = "$world_store" ] || { echo "world artifact link mismatch" >&2; exit 1; }
[ "$(readlink "$result_store/artifacts/kernel")" = "$kernel_store" ] || { echo "kernel artifact link mismatch" >&2; exit 1; }
[ "$(readlink "$result_store/artifacts/headers")" = "$headers_store" ] || { echo "headers artifact link mismatch" >&2; exit 1; }
[ "$(readlink "$result_store/artifacts/bootloader")" = "$bootloader_store" ] || { echo "bootloader artifact link mismatch" >&2; exit 1; }
[ -f "$world_store/bin/sh" ] || { echo "promoted world store missing /bin/sh" >&2; exit 1; }
[ -f "$kernel_store/boot/kernel/kernel" ] || { echo "promoted kernel store missing kernel" >&2; exit 1; }
[ -f "$headers_store/usr/include/sys/param.h" ] || { echo "promoted headers store missing param.h" >&2; exit 1; }
[ -f "$bootloader_store/boot/loader.efi" ] || { echo "promoted bootloader store missing loader.efi" >&2; exit 1; }
promoted_kernel_sha=$(sha256 -q "$kernel_store/boot/kernel/kernel")
promoted_loader_sha=$(sha256 -q "$bootloader_store/boot/loader.efi")
promoted_param_sha=$(sha256 -q "$headers_store/usr/include/sys/param.h")
[ "$promoted_kernel_sha" = "$sha_kernel" ] || { echo "kernel sha mismatch after promotion" >&2; exit 1; }
[ "$promoted_loader_sha" = "$sha_loader" ] || { echo "loader sha mismatch after promotion" >&2; exit 1; }
[ "$promoted_param_sha" = "$sha_param" ] || { echo "param.h sha mismatch after promotion" >&2; exit 1; }
grep -F '(executor . "guest-self-hosted")' "$result_metadata_file" >/dev/null || {
echo "result metadata file is missing guest-self-hosted executor" >&2
exit 1
}
grep -F "$source_store" "$result_metadata_file" >/dev/null || {
echo "result metadata file is missing source store provenance" >&2
exit 1
}
grep -F '(artifact-kind . kernel)' "$kernel_store/.fruix-native-build-object.scm" >/dev/null || {
echo "kernel store metadata is missing artifact kind" >&2
exit 1
}
grep -F '(artifact-kind . world)' "$world_store/.fruix-native-build-object.scm" >/dev/null || {
echo "world store metadata is missing artifact kind" >&2
exit 1
}
cat >"$metadata_file" <<EOF
workdir=$workdir
inner_metadata=$inner_metadata
promotion_out=$promotion_out
closure_path=$closure_path
closure_base=$closure_base
vm_id=$vm_id
vdi_id=$vdi_id
guest_ip=$guest_ip
root_size=$root_size
run_id=$run_id
source_store=$source_store
guest_result_root=$result_root
guest_promotion_file=$promotion_file
guest_world_artifact=$world_artifact
guest_kernel_artifact=$kernel_artifact
guest_headers_artifact=$headers_artifact
guest_bootloader_artifact=$bootloader_artifact
local_result_root=$local_result_root
store_dir=$store_dir
result_store=$result_store
result_metadata_file=$result_metadata_file
artifact_store_count=$artifact_store_count
artifact_stores=$artifact_stores
world_store=$world_store
kernel_store=$kernel_store
headers_store=$headers_store
bootloader_store=$bootloader_store
sha_kernel=$sha_kernel
sha_loader=$sha_loader
sha_param=$sha_param
promoted_kernel_sha=$promoted_kernel_sha
promoted_loader_sha=$promoted_loader_sha
promoted_param_sha=$promoted_param_sha
boot_backend=xcp-ng-xo-cli
init_mode=shepherd-pid1
native_build_store_promotion=ok
EOF
if [ -n "$metadata_target" ]; then
mkdir -p "$(dirname "$metadata_target")"
cp "$metadata_file" "$metadata_target"
fi
printf 'PASS phase20-native-build-store-promotion-xcpng\n'
printf 'Work directory: %s\n' "$workdir"
printf 'Metadata file: %s\n' "$metadata_file"
if [ -n "$metadata_target" ]; then
printf 'Copied metadata to: %s\n' "$metadata_target"
fi
printf '%s\n' '--- metadata ---'
cat "$metadata_file"

10
tests/system/run-phase20-self-hosted-native-build-xcpng.sh
View File

@@ -88,8 +88,10 @@ result_root=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^result_root=//p
logdir=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^logdir=//p') logdir=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^logdir=//p')
status_file=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^status_file=//p') status_file=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^status_file=//p')
guest_metadata_file=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^metadata_file=//p') guest_metadata_file=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^metadata_file=//p')
promotion_file=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^promotion_file=//p')
world_stage=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^world_stage=//p') world_stage=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^world_stage=//p')
kernel_stage=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^kernel_stage=//p') kernel_stage=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^kernel_stage=//p')
world_artifact=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^world_artifact=//p')
kernel_artifact=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^kernel_artifact=//p') kernel_artifact=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^kernel_artifact=//p')
headers_artifact=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^headers_artifact=//p') headers_artifact=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^headers_artifact=//p')
bootloader_artifact=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^bootloader_artifact=//p') bootloader_artifact=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^bootloader_artifact=//p')
@@ -97,6 +99,7 @@ latest_link=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^latest_link=//p
root_df=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^root_df=//p') root_df=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^root_df=//p')
build_root_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^build_root_size=//p') build_root_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^build_root_size=//p')
result_root_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^result_root_size=//p') result_root_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^result_root_size=//p')
world_artifact_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^world_artifact_size=//p')
kernel_artifact_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^kernel_artifact_size=//p') kernel_artifact_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^kernel_artifact_size=//p')
headers_artifact_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^headers_artifact_size=//p') headers_artifact_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^headers_artifact_size=//p')
bootloader_artifact_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^bootloader_artifact_size=//p') bootloader_artifact_size=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^bootloader_artifact_size=//p')
@@ -112,8 +115,10 @@ self_hosted_native_build=$(printf '%s\n' "$self_hosted_metadata" | sed -n 's/^se
status_value=$(ssh_guest "cat '$status_file'") status_value=$(ssh_guest "cat '$status_file'")
latest_target=$(ssh_guest "readlink '$latest_link'") latest_target=$(ssh_guest "readlink '$latest_link'")
ssh_guest "[ -f '$promotion_file' ]"
ssh_guest "[ -f '$world_artifact/bin/sh' ]"
[ "$helper_version" = 2 ] || { echo "unexpected helper version: $helper_version" >&2; exit 1; } [ "$helper_version" = 3 ] || { echo "unexpected helper version: $helper_version" >&2; exit 1; }
[ "$build_jobs" = "$guest_build_jobs" ] || { echo "unexpected build job count: $build_jobs" >&2; exit 1; } [ "$build_jobs" = "$guest_build_jobs" ] || { echo "unexpected build job count: $build_jobs" >&2; exit 1; }
[ "$status_value" = ok ] || { echo "self-hosted build status is not ok: $status_value" >&2; exit 1; } [ "$status_value" = ok ] || { echo "self-hosted build status is not ok: $status_value" >&2; exit 1; }
[ "$latest_target" = "$result_root" ] || { echo "latest link target mismatch: $latest_target" >&2; exit 1; } [ "$latest_target" = "$result_root" ] || { echo "latest link target mismatch: $latest_target" >&2; exit 1; }
@@ -202,8 +207,10 @@ result_root=$result_root
logdir=$logdir logdir=$logdir
status_file=$status_file status_file=$status_file
guest_metadata_file=$guest_metadata_file guest_metadata_file=$guest_metadata_file
promotion_file=$promotion_file
world_stage=$world_stage world_stage=$world_stage
kernel_stage=$kernel_stage kernel_stage=$kernel_stage
world_artifact=$world_artifact
kernel_artifact=$kernel_artifact kernel_artifact=$kernel_artifact
headers_artifact=$headers_artifact headers_artifact=$headers_artifact
bootloader_artifact=$bootloader_artifact bootloader_artifact=$bootloader_artifact
@@ -213,6 +220,7 @@ status_value=$status_value
root_df=$root_df root_df=$root_df
build_root_size=$build_root_size build_root_size=$build_root_size
result_root_size=$result_root_size result_root_size=$result_root_size
world_artifact_size=$world_artifact_size
kernel_artifact_size=$kernel_artifact_size kernel_artifact_size=$kernel_artifact_size
headers_artifact_size=$headers_artifact_size headers_artifact_size=$headers_artifact_size
bootloader_artifact_size=$bootloader_artifact_size bootloader_artifact_size=$bootloader_artifact_size