self/fruix
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add Fruix bootable installer environment

Browse Source
This commit is contained in:
Steffen
2026-04-04 03:09:25 +02:00
parent 2517710282
commit 1d0090752d
7 changed files with 1313 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files

92
docs/PROGRESS.md
View File

@@ -1,5 +1,97 @@
# Progress
## 2026-04-04 — Phase 18.2 completed: Fruix now boots a minimal installer environment and installs a target system from inside it
Completed work:
- added in `modules/fruix/system/freebsd.scm`:
- `installer-operating-system`
- `operating-system-installer-image-spec`
- `materialize-installer-image`
- the installer environment is now derived from the selected target operating system, while forcing the currently most stable installer boot path:
- `freebsd-init+rc.d-shepherd`
- the installer image now carries:
- its own installer-system closure
- the selected target-system closure
- the selected target-system store closure
- a staged target rootfs under:
- `/var/lib/fruix/installer/target-rootfs`
- installer plan/state files under:
- `/var/lib/fruix/installer`
- installer helper scripts:
- `/usr/local/libexec/fruix-installer-run`
- `/usr/local/etc/rc.d/fruix-installer`
- the booted installer environment now performs the install from inside the guest by:
- partitioning the selected target disk
- creating EFI + UFS filesystems
- copying the staged target rootfs
- copying only the selected target system's required store items into the target `/frx/store`
- installing the target `loader.efi`
- writing target install metadata to:
- `/var/lib/fruix/install.scm`
- recording installer state in:
- `/var/lib/fruix/installer/state`
- logging to:
- `/var/log/fruix-installer.log`
- added user-facing CLI support in `scripts/fruix.scm`:
- `fruix system installer`
- new option:
- `--install-target-device DEVICE`
- corrected the default FreeBSD virtio install target device for QEMU validation to:
- `/dev/vtbd1`
rather than the earlier incorrect Linux-flavored guess:
- `/dev/vtblk1`
- fixed `materialize-bhyve-image` so the generated UFS filesystem label now respects:
- `root-partition-label`
rather than always hardcoding:
- `fruix-root`
- added validation artifacts:
- `tests/system/phase18-installer-target-operating-system.scm.in`
- `tests/system/run-phase18-installer-environment.sh`
- wrote:
- `docs/reports/phase18-installer-environment-freebsd.md`
Validation:
- `PASS phase18-installer-environment`
- regression re-checks:
- `PASS phase18-system-install`
- `PASS phase17-source-revisions-qemu`
- validated a full two-disk workflow:
- booted Fruix installer image under QEMU/UEFI/TCG
- installer environment became reachable over SSH
- installer environment completed in-guest install with:
- `installer_state=done`
- installer log recorded:
- `fruix-installer:done`
- installed target disk contained:
- GPT partition table
- ESP filesystem:
- `msdosfs`
- root filesystem:
- `ufs`
- `EFI/BOOT/BOOTX64.EFI`
- `/var/lib/fruix/install.scm`
- validated source-driven provenance through the installer environment from:
- Git ref:
- `stable/15`
- pinned commit:
- `332708a606f6bf0841c1d4a74c0d067f5640fe89`
- materialized source store:
- `/frx/store/7563df2714ae7fa9bd40b83c74512ffe2cb2ad91b297915591b55c76edbb2fcb-freebsd-source-stable15-installer-target-source`
- validated boot of the installed target after in-guest installation:
- `/run/current-system` points at the installed target closure
- `/usr/local/etc/rc.d/fruix-shepherd onestatus` reports running
- `sshd` running
- activation completed successfully
Current assessment:
- Phase 18.2 is complete
- Fruix now has a minimal Fruix-managed installer environment, not just a host-driven installation primitive
- the next step is Phase 18.3:
- produce a bootable installer ISO for UEFI systems
## 2026-04-03 — Phase 18.1 completed: Fruix now has a minimal non-interactive installation flow
Completed work:

5
docs/PROG_SUMMARY.md
View File

@@ -32,6 +32,7 @@ Completed milestones include:
- **Source-driven boot validation**: Fruix can now also boot systems built from distinct declared FreeBSD source revisions while preserving those source identities in image/build metadata.
- **Explicit source policy**: the repo now records how FreeBSD source objects are fetched, cached, identified, invalidated, and consumed by native base builds in `docs/freebsd-source-policy.md`.
- **Minimal installation workflow**: Fruix now has a non-interactive `fruix system install` path that can partition, format, populate, and boot a target image or disk from a declarative system closure.
- **Minimal installer environment**: Fruix can now also build and boot a dedicated installer image that carries a selected target closure, installs it onto a second disk from inside the guest, and leaves the installed target bootable.
- **Base upgrade story**: Fruix can now keep distinct declared base versions side by side in `/frx/store` and roll forward / back between them through the normal system deployment flow.
## Major pain points now behind us
@@ -46,7 +47,7 @@ Completed milestones include:
## Major pain points still ahead
- **True store-native runtime artifacts**: some historical build/install prefixes are still embedded in binaries and metadata. They are no longer required at runtime, but the local Guile/guile-extra/Shepherd build/install flow should still be moved to a genuinely store-native prefix from the start.
- **Installer environment**: Fruix now has a host-driven non-interactive install path, but it still lacks a dedicated Fruix-managed installer environment that can boot into an install context and run that workflow from within the target environment.
- **Installer media beyond disk images**: Fruix now has both a host-driven install path and a bootable installer environment, but it still lacks a UEFI installer ISO and a more polished operator-facing installation medium.
- **Boot-path simplification**: Fruix now supports both the legacy `freebsd-init+rc.d-shepherd` path and the more Guix-like `shepherd-pid1` path. We still need to decide whether Shepherd PID 1 becomes the preferred/default architecture.
- **Reduce transitional FreeBSD glue**: more of the current bootstrap/activation/runtime setup should become cleaner and less prototype-specific over time.
- **Tooling and platform constraints**: local bhyve remains blocked by missing nested virtualization under Xen, and XO permissions still prevent creating/importing new VDIs; current validation must keep reusing the approved VM/VDI path.
@@ -54,4 +55,4 @@ Completed milestones include:
## Bottom line
Fruix has crossed the most important threshold: it is no longer just a collection of isolated FreeBSD experiments. It can now build declarative FreeBSD system artifacts, boot them on the real target VM, reach the network, serve SSH, run Shepherd as PID 1, operate from `/frx` without depending on temporary runtime-prefix shims, build native FreeBSD base artifacts into `/frx/store`, roll forward / back between declared base versions, materialize declared FreeBSD source inputs into `/frx/store`, drive native base builds from those materialized source snapshots, boot systems from distinct source revisions, explain the source provenance/invalidation rules explicitly, and install a declarative system onto a target image through a repeatable Fruix workflow. The biggest remaining work is no longer “can this build/install at all?” but “how does this become a fuller installer/deployment/generation story?”
Fruix has crossed the most important threshold: it is no longer just a collection of isolated FreeBSD experiments. It can now build declarative FreeBSD system artifacts, boot them on the real target VM, reach the network, serve SSH, run Shepherd as PID 1, operate from `/frx` without depending on temporary runtime-prefix shims, build native FreeBSD base artifacts into `/frx/store`, roll forward / back between declared base versions, materialize declared FreeBSD source inputs into `/frx/store`, drive native base builds from those materialized source snapshots, boot systems from distinct source revisions, explain the source provenance/invalidation rules explicitly, install a declarative system onto a target image through a repeatable Fruix workflow, and boot a dedicated Fruix-managed installer environment that performs that installation from inside the guest. The biggest remaining work is no longer “can this build/install at all?” but “how does this become a fuller installer/deployment/generation/installer-media story?”

191
docs/reports/phase18-installer-environment-freebsd.md Normal file
View File

@@ -0,0 +1,191 @@
# Phase 18.2: minimal Fruix-managed installer environment on FreeBSD
Date: 2026-04-04
## Goal
Phase 18.2 builds on the Phase 18.1 host-driven install primitive.
The goal here is not a polished live installer. The goal is a small Fruix-managed environment that can:
- boot as its own Fruix system,
- carry a selected target Fruix system closure and rootfs payload,
- install that target system onto a second disk from inside the booted environment,
- and leave the installed target bootable.
## Implementation
### New installer-environment API
Added in `modules/fruix/system/freebsd.scm`:
- `installer-operating-system`
- `operating-system-installer-image-spec`
- `materialize-installer-image`
The installer environment is derived from the selected target operating system, but with installer-specific behavior:
- host name defaults to:
- `<target-host-name>-installer`
- init mode is kept on the currently most stable installer path:
- `freebsd-init+rc.d-shepherd`
- the installer image root label is distinct:
- `fruix-installer-root`
- `sshd` is enabled for operator/debug access
- installer accounts needed for SSH/DHCP are ensured if absent:
- `sshd`
- `_dhcp`
### Bootable installer image contents
`materialize-installer-image` now produces a bootable image that contains:
- the installer system closure and its runtime store closure
- the selected target system closure
- the selected target system's referenced store items
- a prebuilt target rootfs tree staged under:
- `/var/lib/fruix/installer/target-rootfs`
- installer plan/state files under:
- `/var/lib/fruix/installer`
- installer helper scripts:
- `/usr/local/libexec/fruix-installer-run`
- `/usr/local/etc/rc.d/fruix-installer`
The booted installer environment runs a background rc.d job that:
- partitions the selected target disk
- creates EFI + UFS filesystems
- copies the staged target rootfs onto the target
- copies only the target system's required store items into the target `/frx/store`
- installs the target's `loader.efi`
- writes `/var/lib/fruix/install.scm` on the target
- records installer state in:
- `/var/lib/fruix/installer/state`
- logs to:
- `/var/log/fruix-installer.log`
### New CLI action
Added in `scripts/fruix.scm`:
- `fruix system installer`
Added option:
- `--install-target-device DEVICE`
This action materializes a bootable installer image in `/frx/store` and emits metadata for:
- installer image paths
- installer closure path
- target closure path
- target install device
- installer state/log paths
- declared/materialized FreeBSD source metadata
- target/native/runtime store metadata
### FreeBSD virtio target-device detail
A practical detail surfaced during validation:
- the correct FreeBSD virtio block device node for the second QEMU disk is:
- `/dev/vtbd1`
The earlier Linux-flavored guess:
- `/dev/vtblk1`
was wrong for the actual FreeBSD device node namespace in this environment.
The installer defaults were updated accordingly.
### Small image-builder correctness fix
While doing this work I also fixed `materialize-bhyve-image` so its generated UFS filesystem label respects the requested:
- `root-partition-label`
instead of always hardcoding:
- `fruix-root`
This matters for the installer image because it needs a distinct root label while the target disk still uses the normal target label.
## Validation
Added validation artifacts:
- `tests/system/phase18-installer-target-operating-system.scm.in`
- `tests/system/run-phase18-installer-environment.sh`
Passing validations:
- `PASS phase18-installer-environment`
- regression re-check:
- `PASS phase18-system-install`
- regression re-check:
- `PASS phase17-source-revisions-qemu`
Validated installer-environment result:
```text
installer_image_store_path=/frx/store/fb038dbf5dac2ad1bb767a264d3a268915f489b936dc5dd32425645102d3da48-fruix-installer-image-fruix-freebsd-installer
installer_disk_image=/frx/store/fb038dbf5dac2ad1bb767a264d3a268915f489b936dc5dd32425645102d3da48-fruix-installer-image-fruix-freebsd-installer/disk.img
installer_disk_capacity=16g
installer_root_size=14g
target_disk_capacity=12g
install_target_device=/dev/vtbd1
installer_closure_path=/frx/store/ea821f20b579684877fdc86a2a1e80485cf2b12d9d32f74f42e368d738c2ad4d-fruix-system-fruix-freebsd-installer
target_closure_path=/frx/store/7ee225db532b6973e385f8507d2d61aec3cd3aeb0864f983c2ae4b6e149ef3b0-fruix-system-fruix-freebsd
freebsd_source_kind=git
freebsd_source_ref=stable/15
freebsd_source_commit=332708a606f6bf0841c1d4a74c0d067f5640fe89
materialized_source_store=/frx/store/7563df2714ae7fa9bd40b83c74512ffe2cb2ad91b297915591b55c76edbb2fcb-freebsd-source-stable15-installer-target-source
installer_state=done
installer_sshd_status=running
target_esp_fstype=msdosfs
target_root_fstype=ufs
target_shepherd_status=running
target_sshd_status=running
installer_environment_boot=ok
installer_environment_install=ok
installed_target_boot=ok
```
The harness verified all of the following:
1. `fruix system installer` produces a bootable installer image in `/frx/store`
2. validation boots a workdir copy of that installer disk image so the store artifact itself is not mutated during the boot/install run
3. the installer environment boots successfully under QEMU/UEFI/TCG
4. the installer environment becomes reachable over SSH
5. `/run/current-system` inside the installer environment points at the installer closure
6. the installer rc.d job reaches:
- `state=done`
7. the installer log records:
- `fruix-installer:done`
8. the target raw disk is transformed into a valid GPT-installed Fruix target with:
- EFI filesystem: `msdosfs`
- root filesystem: `ufs`
- `EFI/BOOT/BOOTX64.EFI` present
- `/var/lib/fruix/install.scm` present
9. the installed target then boots successfully as its own Fruix system under QEMU/UEFI/TCG
10. after target boot:
- `/run/current-system` points at the target closure
- `/usr/local/etc/rc.d/fruix-shepherd onestatus` reports running
- `sshd` is running
- activation completed successfully
## Result
Phase 18.2 is complete.
Fruix now has a real installer substrate on FreeBSD:
- a bootable Fruix-managed installer image
- a target closure bundled inside that installer environment
- in-guest non-interactive installation onto a second disk
- validated boot of the installed result
The next step is Phase 18.3:
- produce a bootable installer ISO for UEFI systems, rather than only a disk-image-style installer environment.

438
modules/fruix/system/freebsd.scm
View File

@@ -53,10 +53,13 @@
operating-system-closure-spec
operating-system-install-spec
operating-system-image-spec
operating-system-installer-image-spec
installer-operating-system
materialize-operating-system
materialize-rootfs
install-operating-system
materialize-bhyve-image
materialize-installer-image
default-minimal-operating-system))
(define-record-type <user-group>
@@ -1901,6 +1904,81 @@
(closure-path (assoc-ref closure 'closure-path)))
(populate-rootfs-from-closure os rootfs closure-path)))
(define (assoc-remove keys entries)
(filter (lambda (entry)
(not (member (car entry) keys)))
entries))
(define (ensure-installer-groups groups)
(append groups
(filter (lambda (group)
(not (any (lambda (existing)
(string=? (user-group-name existing)
(user-group-name group)))
groups)))
(list (user-group #:name "sshd" #:gid 22 #:system? #t)
(user-group #:name "_dhcp" #:gid 65 #:system? #t)))))
(define (ensure-installer-users users)
(append users
(filter (lambda (user)
(not (any (lambda (existing)
(string=? (user-account-name existing)
(user-account-name user)))
users)))
(list (user-account #:name "sshd"
#:uid 22
#:group "sshd"
#:comment "Secure Shell Daemon"
#:home "/var/empty"
#:shell "/usr/sbin/nologin"
#:system? #t)
(user-account #:name "_dhcp"
#:uid 65
#:group "_dhcp"
#:comment "dhcp programs"
#:home "/var/empty"
#:shell "/usr/sbin/nologin"
#:system? #t)))))
(define* (installer-operating-system os
#:key
(host-name (string-append (operating-system-host-name os)
"-installer"))
(root-partition-label "fruix-installer-root")
(ready-marker "/var/lib/fruix/installer/ready"))
(operating-system
#:host-name host-name
#:freebsd-base (operating-system-freebsd-base os)
#:kernel (operating-system-kernel os)
#:bootloader (operating-system-bootloader os)
#:base-packages (operating-system-base-packages os)
#:users (ensure-installer-users (operating-system-users os))
#:groups (ensure-installer-groups (operating-system-groups os))
#:file-systems (list (file-system #:device (string-append "/dev/gpt/" root-partition-label)
#:mount-point "/"
#:type "ufs"
#:options "rw"
#:needed-for-boot? #t)
(file-system #:device "devfs"
#:mount-point "/dev"
#:type "devfs"
#:options "rw"
#:needed-for-boot? #t)
(file-system #:device "tmpfs"
#:mount-point "/tmp"
#:type "tmpfs"
#:options "rw,size=64m"))
#:services '(shepherd ready-marker sshd)
#:loader-entries (operating-system-loader-entries os)
#:rc-conf-entries (append (assoc-remove '("sshd_enable" "fruix_installer_enable")
(operating-system-rc-conf-entries os))
'(("sshd_enable" . "YES")
("fruix_installer_enable" . "YES")))
#:init-mode 'freebsd-init+rc.d-shepherd
#:ready-marker ready-marker
#:root-authorized-keys (operating-system-root-authorized-keys os)))
(define* (operating-system-install-spec os
#:key
target
@@ -1952,6 +2030,43 @@
(serial-console . ,serial-console)
(init-mode . ,(operating-system-init-mode os))))
(define* (operating-system-installer-image-spec os
#:key
(install-target-device "/dev/vtbd1")
(installer-host-name (string-append (operating-system-host-name os)
"-installer"))
(efi-size "64m")
(root-size "10g")
(disk-capacity #f)
(installer-efi-partition-label "efiboot")
(installer-root-partition-label "fruix-installer-root")
(target-efi-partition-label "efiboot")
(target-root-partition-label "fruix-root")
(serial-console "comconsole"))
(let* ((installer-os (installer-operating-system os
#:host-name installer-host-name
#:root-partition-label installer-root-partition-label))
(target-install-spec (operating-system-install-spec os
#:target install-target-device
#:target-kind 'block-device
#:efi-size efi-size
#:root-size #f
#:disk-capacity #f
#:efi-partition-label target-efi-partition-label
#:root-partition-label target-root-partition-label
#:serial-console serial-console)))
`((installer-host-name . ,installer-host-name)
(install-target-device . ,install-target-device)
(installer-root-partition-label . ,installer-root-partition-label)
(installer-image . ,(operating-system-image-spec installer-os
#:efi-size efi-size
#:root-size root-size
#:disk-capacity disk-capacity
#:efi-partition-label installer-efi-partition-label
#:root-partition-label installer-root-partition-label
#:serial-console serial-console))
(target-install . ,target-install-spec))))
(define (path-basename path)
(let ((parts (filter (lambda (part) (not (string-null? part)))
(string-split path #\/))))
@@ -1999,6 +2114,128 @@
(define image-builder-version "2")
(define install-builder-version "1")
(define installer-image-builder-version "1")
(define (operating-system-install-metadata-object install-spec closure-path store-items)
`((install-version . ,install-builder-version)
(install-spec . ,install-spec)
(closure-path . ,closure-path)
(store-item-count . ,(length store-items))
(store-items . ,store-items)))
(define (render-installer-run-script store-dir plan-directory)
(let ((target-rootfs (string-append plan-directory "/target-rootfs"))
(store-items-file (string-append plan-directory "/store-items"))
(install-metadata-source (string-append plan-directory "/install.scm"))
(target-loader-efi (string-append plan-directory "/loader.efi"))
(state-file (string-append plan-directory "/state"))
(log-file "/var/log/fruix-installer.log")
(target-device-file (string-append plan-directory "/target-device"))
(efi-size-file (string-append plan-directory "/efi-size"))
(efi-label-file (string-append plan-directory "/efi-partition-label"))
(root-label-file (string-append plan-directory "/root-partition-label")))
(string-append
"#!/bin/sh\n"
"set -eu\n"
"PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin\n"
"umask 022\n"
"target_rootfs='" target-rootfs "'\n"
"store_items_file='" store-items-file "'\n"
"install_metadata_source='" install-metadata-source "'\n"
"target_loader_efi='" target-loader-efi "'\n"
"state_file='" state-file "'\n"
"log_file='" log-file "'\n"
"target_device=$(cat '" target-device-file "')\n"
"efi_size=$(cat '" efi-size-file "')\n"
"efi_partition_label=$(cat '" efi-label-file "')\n"
"root_partition_label=$(cat '" root-label-file "')\n"
"esp_device=\"${target_device}p1\"\n"
"root_device=\"${target_device}p2\"\n"
"mnt_root=/var/run/fruix-installer/target-root\n"
"mnt_esp=/var/run/fruix-installer/target-esp\n"
"write_state()\n"
"{\n"
" mkdir -p \"$(dirname \"$state_file\")\"\n"
" printf '%s\\n' \"$1\" >\"$state_file\"\n"
"}\n"
"cleanup()\n"
"{\n"
" umount \"$mnt_esp\" >/dev/null 2>&1 || true\n"
" umount \"$mnt_root\" >/dev/null 2>&1 || true\n"
"}\n"
"cleanup_and_record()\n"
"{\n"
" status=\"$1\"\n"
" if [ \"$status\" -ne 0 ]; then\n"
" echo \"fruix-installer:failed status=$status\"\n"
" write_state failed\n"
" fi\n"
" cleanup\n"
"}\n"
"trap 'status=$?; cleanup_and_record \"$status\"' EXIT\n"
"trap 'exit 1' INT TERM\n"
"mkdir -p \"$(dirname \"$log_file\")\" /var/run/fruix-installer\n"
"exec >>\"$log_file\" 2>&1\n"
"echo 'fruix-installer:start'\n"
"write_state starting\n"
"[ -e \"$target_device\" ] || { echo \"fruix-installer:error missing target device $target_device\"; exit 1; }\n"
"gpart destroy -F \"$target_device\" >/dev/null 2>&1 || true\n"
"gpart create -s gpt \"$target_device\"\n"
"gpart add -a 1m -s \"$efi_size\" -t efi -l \"$efi_partition_label\" \"$target_device\"\n"
"gpart add -a 1m -t freebsd-ufs -l \"$root_partition_label\" \"$target_device\"\n"
"newfs_msdos -L EFISYS \"$esp_device\"\n"
"newfs -U -L \"$root_partition_label\" \"$root_device\"\n"
"mkdir -p \"$mnt_root\" \"$mnt_esp\"\n"
"mount -t ufs \"$root_device\" \"$mnt_root\"\n"
"mount -t msdosfs \"$esp_device\" \"$mnt_esp\"\n"
"write_state copying-rootfs\n"
"(cd \"$target_rootfs\" && pax -rw -pe . \"$mnt_root\")\n"
"mkdir -p \"$mnt_root" store-dir "\"\n"
"write_state copying-store\n"
"while IFS= read -r item_base || [ -n \"$item_base\" ]; do\n"
" [ -n \"$item_base\" ] || continue\n"
" (cd '" store-dir "' && pax -rw -pe \"$item_base\" \"$mnt_root" store-dir "\")\n"
"done <\"$store_items_file\"\n"
"mkdir -p \"$mnt_root/var/lib/fruix\" \"$mnt_esp/EFI/BOOT\"\n"
"cp \"$target_loader_efi\" \"$mnt_esp/EFI/BOOT/BOOTX64.EFI\"\n"
"cp \"$install_metadata_source\" \"$mnt_root/var/lib/fruix/install.scm\"\n"
"sync\n"
"echo 'fruix-installer:done'\n"
"write_state done\n")))
(define (render-installer-rc-script plan-directory)
(string-append
"#!/bin/sh\n"
"# PROVIDE: fruix_installer\n"
"# REQUIRE: NETWORKING sshd fruix_shepherd\n"
"# KEYWORD: shutdown\n\n"
". /etc/rc.subr\n\n"
"name=fruix_installer\n"
"rcvar=fruix_installer_enable\n"
": ${fruix_installer_enable:=YES}\n"
"pidfile=/var/run/fruix-installer.pid\n"
"command=/usr/sbin/daemon\n"
"command_args='-c -f -p /var/run/fruix-installer.pid -o /var/log/fruix-installer-bootstrap.out /usr/local/libexec/fruix-installer-run'\n"
"start_cmd=fruix_installer_start\n"
"stop_cmd=fruix_installer_stop\n"
"status_cmd=fruix_installer_status\n\n"
"fruix_installer_start()\n"
"{\n"
" mkdir -p '" plan-directory "' /var/run\n"
" $command $command_args\n"
"}\n\n"
"fruix_installer_stop()\n"
"{\n"
" [ -f \"$pidfile\" ] && kill \"$(cat \"$pidfile\")\" >/dev/null 2>&1 || true\n"
" rm -f \"$pidfile\"\n"
" return 0\n"
"}\n\n"
"fruix_installer_status()\n"
"{\n"
" [ -f '" plan-directory "/state' ]\n"
"}\n\n"
"load_rc_config $name\n"
"run_rc_command \"$1\"\n"))
(define (resize-gpt-image image disk-capacity)
(when disk-capacity
@@ -2100,11 +2337,7 @@
(let ((install-metadata-file (string-append mnt-root install-metadata-relative-path)))
(write-file install-metadata-file
(object->string
`((install-version . ,install-builder-version)
(install-spec . ,install-spec)
(closure-path . ,closure-path)
(store-item-count . ,(length store-items))
(store-items . ,store-items))))
(operating-system-install-metadata-object install-spec closure-path store-items)))
(chmod install-metadata-file #o644))
(run-command "sync")
`((target . ,target)
@@ -2204,7 +2437,8 @@
(string-append esp-stage "/EFI/BOOT/BOOTX64.EFI"))
(run-command "makefs" "-t" "ffs" "-T" "0" "-B" "little"
"-s" root-size
"-o" "label=fruix-root,version=2,bsize=32768,fsize=4096,density=16384"
"-o" (string-append "label=" root-partition-label
",version=2,bsize=32768,fsize=4096,density=16384")
temp-root image-rootfs)
(run-command "makefs" "-t" "msdos" "-T" "0"
"-o" "fat_type=32"
@@ -2256,3 +2490,195 @@
(store-layout-file . ,(assoc-ref closure 'store-layout-file))
(image-spec . ,image-spec)
(store-items . ,store-items))))
(define* (materialize-installer-image os
#:key
(store-dir "/frx/store")
(guile-prefix "/tmp/guile-freebsd-validate-install")
(guile-extra-prefix "/tmp/guile-gnutls-freebsd-validate-install")
(shepherd-prefix "/tmp/shepherd-freebsd-validate-install")
(install-target-device "/dev/vtbd1")
(efi-size "64m")
(root-size "10g")
(disk-capacity #f)
(installer-host-name (string-append (operating-system-host-name os)
"-installer"))
(installer-efi-partition-label "efiboot")
(installer-root-partition-label "fruix-installer-root")
(target-efi-partition-label "efiboot")
(target-root-partition-label "fruix-root")
(serial-console "comconsole"))
(let* ((installer-os (installer-operating-system os
#:host-name installer-host-name
#:root-partition-label installer-root-partition-label))
(target-closure (materialize-operating-system os
#:store-dir store-dir
#:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix))
(installer-closure (materialize-operating-system installer-os
#:store-dir store-dir
#:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix))
(target-closure-path (assoc-ref target-closure 'closure-path))
(installer-closure-path (assoc-ref installer-closure 'closure-path))
(target-store-items (store-reference-closure (list target-closure-path)))
(installer-store-items (store-reference-closure (list installer-closure-path)))
(combined-store-items (delete-duplicates (append installer-store-items target-store-items)))
(installer-image-spec (operating-system-installer-image-spec os
#:install-target-device install-target-device
#:installer-host-name installer-host-name
#:efi-size efi-size
#:root-size root-size
#:disk-capacity disk-capacity
#:installer-efi-partition-label installer-efi-partition-label
#:installer-root-partition-label installer-root-partition-label
#:target-efi-partition-label target-efi-partition-label
#:target-root-partition-label target-root-partition-label
#:serial-console serial-console))
(image-spec (assoc-ref installer-image-spec 'installer-image))
(target-install-spec (assoc-ref installer-image-spec 'target-install))
(install-metadata (operating-system-install-metadata-object target-install-spec
target-closure-path
target-store-items))
(installer-plan-directory "/var/lib/fruix/installer")
(installer-state-path (string-append installer-plan-directory "/state"))
(installer-log-path "/var/log/fruix-installer.log")
(manifest (string-append
"installer-image-builder-version=\n"
installer-image-builder-version
"\ninstaller-image-spec=\n"
(object->string installer-image-spec)
"installer-closure-path=\n"
installer-closure-path
"\ntarget-closure-path=\n"
target-closure-path
"\ncombined-store-items=\n"
(string-join combined-store-items "\n")
"\ntarget-store-items=\n"
(string-join target-store-items "\n")
"\ninstall-metadata=\n"
(object->string install-metadata)
"\n"))
(hash (string-hash manifest))
(image-store-path (string-append store-dir "/" hash "-fruix-installer-image-"
(operating-system-host-name installer-os)))
(disk-image (string-append image-store-path "/disk.img"))
(esp-image (string-append image-store-path "/esp.img"))
(root-image (string-append image-store-path "/root.ufs")))
(unless (file-exists? image-store-path)
(let* ((build-root (mktemp-directory "/tmp/fruix-installer-image-build.XXXXXX"))
(installer-rootfs (string-append build-root "/installer-rootfs"))
(target-rootfs (string-append build-root "/target-rootfs"))
(image-rootfs (string-append build-root "/image-rootfs"))
(esp-stage (string-append build-root "/esp-stage"))
(temp-output (mktemp-directory (string-append store-dir "/.fruix-installer-image.XXXXXX")))
(temp-disk (string-append build-root "/disk.img"))
(temp-esp (string-append build-root "/esp.img"))
(temp-root (string-append build-root "/root.ufs"))
(plan-root (string-append image-rootfs installer-plan-directory)))
(dynamic-wind
(lambda () #t)
(lambda ()
(populate-rootfs-from-closure installer-os installer-rootfs installer-closure-path)
(populate-rootfs-from-closure os target-rootfs target-closure-path)
(copy-rootfs-for-image installer-rootfs image-rootfs)
(mkdir-p plan-root)
(mkdir-p (string-append image-rootfs "/usr/local/libexec"))
(mkdir-p (string-append image-rootfs "/usr/local/etc/rc.d"))
(mkdir-p (string-append plan-root "/target-rootfs"))
(copy-tree-contents target-rootfs (string-append plan-root "/target-rootfs"))
(copy-store-items-into-rootfs image-rootfs store-dir combined-store-items)
(write-file (string-append plan-root "/store-items")
(string-append (string-join (map path-basename target-store-items) "\n") "\n"))
(write-file (string-append plan-root "/install.scm")
(object->string install-metadata))
(copy-regular-file (string-append target-closure-path "/boot/loader.efi")
(string-append plan-root "/loader.efi"))
(write-file (string-append plan-root "/target-device")
(string-append install-target-device "\n"))
(write-file (string-append plan-root "/efi-size")
(string-append efi-size "\n"))
(write-file (string-append plan-root "/efi-partition-label")
(string-append target-efi-partition-label "\n"))
(write-file (string-append plan-root "/root-partition-label")
(string-append target-root-partition-label "\n"))
(write-file (string-append plan-root "/state") "pending\n")
(write-file (string-append image-rootfs "/usr/local/libexec/fruix-installer-run")
(render-installer-run-script store-dir installer-plan-directory))
(write-file (string-append image-rootfs "/usr/local/etc/rc.d/fruix-installer")
(render-installer-rc-script installer-plan-directory))
(chmod (string-append image-rootfs "/usr/local/libexec/fruix-installer-run") #o555)
(chmod (string-append image-rootfs "/usr/local/etc/rc.d/fruix-installer") #o555)
(mkdir-p (string-append esp-stage "/EFI/BOOT"))
(copy-regular-file (string-append installer-closure-path "/boot/loader.efi")
(string-append esp-stage "/EFI/BOOT/BOOTX64.EFI"))
(run-command "makefs" "-t" "ffs" "-T" "0" "-B" "little"
"-s" root-size
"-o" (string-append "label=" installer-root-partition-label
",version=2,bsize=32768,fsize=4096,density=16384")
temp-root image-rootfs)
(run-command "makefs" "-t" "msdos" "-T" "0"
"-o" "fat_type=32"
"-o" "sectors_per_cluster=1"
"-o" "volume_label=EFISYS"
"-o" "volume_id=305419896"
"-s" efi-size
temp-esp esp-stage)
(run-command "mkimg" "-s" "gpt" "-f" "raw" "-t" "0"
"-p" (string-append "efi/" installer-efi-partition-label ":=" temp-esp)
"-p" (string-append "freebsd-ufs/" installer-root-partition-label ":=" temp-root)
"-o" temp-disk)
(resize-gpt-image temp-disk disk-capacity)
(mkdir-p temp-output)
(copy-regular-file temp-disk (string-append temp-output "/disk.img"))
(copy-regular-file temp-esp (string-append temp-output "/esp.img"))
(copy-regular-file temp-root (string-append temp-output "/root.ufs"))
(write-file (string-append temp-output "/installer-image-spec.scm")
(object->string installer-image-spec))
(write-file (string-append temp-output "/installer-closure-path") installer-closure-path)
(write-file (string-append temp-output "/target-closure-path") target-closure-path)
(write-file (string-append temp-output "/.references")
(string-join combined-store-items "\n"))
(write-file (string-append temp-output "/.fruix-package") manifest)
(chmod temp-output #o755)
(for-each (lambda (path)
(chmod path #o644))
(list (string-append temp-output "/disk.img")
(string-append temp-output "/esp.img")
(string-append temp-output "/root.ufs")
(string-append temp-output "/installer-image-spec.scm")
(string-append temp-output "/installer-closure-path")
(string-append temp-output "/target-closure-path")
(string-append temp-output "/.references")
(string-append temp-output "/.fruix-package")))
(rename-file temp-output image-store-path))
(lambda ()
(when (file-exists? build-root)
(delete-file-recursively build-root))))))
`((image-store-path . ,image-store-path)
(disk-image . ,disk-image)
(esp-image . ,esp-image)
(root-image . ,root-image)
(installer-closure-path . ,installer-closure-path)
(target-closure-path . ,target-closure-path)
(closure-path . ,installer-closure-path)
(image-spec . ,image-spec)
(installer-image-spec . ,installer-image-spec)
(install-spec . ,target-install-spec)
(installer-state-path . ,installer-state-path)
(installer-log-path . ,installer-log-path)
(install-target-device . ,install-target-device)
(host-base-stores . ,(assoc-ref target-closure 'host-base-stores))
(native-base-stores . ,(assoc-ref target-closure 'native-base-stores))
(fruix-runtime-stores . ,(assoc-ref target-closure 'fruix-runtime-stores))
(freebsd-base-file . ,(assoc-ref target-closure 'freebsd-base-file))
(freebsd-source-file . ,(assoc-ref target-closure 'freebsd-source-file))
(freebsd-source-materializations-file . ,(assoc-ref target-closure 'freebsd-source-materializations-file))
(materialized-source-stores . ,(assoc-ref target-closure 'materialized-source-stores))
(host-base-provenance-file . ,(assoc-ref target-closure 'host-base-provenance-file))
(store-layout-file . ,(assoc-ref target-closure 'store-layout-file))
(store-items . ,combined-store-items)
(target-store-items . ,target-store-items)
(installer-store-items . ,installer-store-items))))

121
scripts/fruix.scm
View File

@@ -19,15 +19,18 @@ Commands:\n\
System actions:\n\
build Materialize the Fruix system closure in /frx/store.\n\
image Materialize the Fruix disk image in /frx/store.\n\
installer Materialize a bootable Fruix installer image in /frx/store.\n\
install Install the Fruix system onto --target PATH.\n\
rootfs Materialize a rootfs tree at --rootfs DIR or ROOTFS-DIR.\n\
\n\
System options:\n\
--system NAME Scheme variable holding the operating-system object.\n\
--store DIR Store directory to use (default: /frx/store).\n\
--disk-capacity SIZE Disk capacity for 'image' or raw-file 'install' targets.\n\
--root-size SIZE Root filesystem size for 'image' or 'install' (example: 6g).\n\
--disk-capacity SIZE Disk capacity for 'image', 'installer', or raw-file 'install' targets.\n\
--root-size SIZE Root filesystem size for 'image', 'installer', or 'install' (example: 6g).\n\
--target PATH Install target for 'install' (raw image file or /dev/... device).\n\
--install-target-device DEVICE\n\
Target block device used by the booted 'installer' environment.\n\
--rootfs DIR Rootfs target for 'rootfs'.\n\
\n\
Source actions:\n\
@@ -126,6 +129,7 @@ Common options:\n\
(disk-capacity #f)
(root-size #f)
(target #f)
(install-target-device #f)
(rootfs #f))
(match args
(()
@@ -138,37 +142,44 @@ Common options:\n\
(disk-capacity . ,disk-capacity)
(root-size . ,root-size)
(target . ,target)
(install-target-device . ,install-target-device)
(rootfs . ,rootfs))))
(("--help")
(usage 0))
(((? (lambda (arg) (string-prefix? "--system=" arg)) arg) . tail)
(loop tail positional (option-value arg "--system=") store-dir disk-capacity root-size target rootfs))
(loop tail positional (option-value arg "--system=") store-dir disk-capacity root-size target install-target-device rootfs))
(("--system" value . tail)
(loop tail positional value store-dir disk-capacity root-size target rootfs))
(loop tail positional value store-dir disk-capacity root-size target install-target-device rootfs))
(((? (lambda (arg) (string-prefix? "--store=" arg)) arg) . tail)
(loop tail positional system-name (option-value arg "--store=") disk-capacity root-size target rootfs))
(loop tail positional system-name (option-value arg "--store=") disk-capacity root-size target install-target-device rootfs))
(("--store" value . tail)
(loop tail positional system-name value disk-capacity root-size target rootfs))
(loop tail positional system-name value disk-capacity root-size target install-target-device rootfs))
(((? (lambda (arg) (string-prefix? "--disk-capacity=" arg)) arg) . tail)
(loop tail positional system-name store-dir (option-value arg "--disk-capacity=") root-size target rootfs))
(loop tail positional system-name store-dir (option-value arg "--disk-capacity=") root-size target install-target-device rootfs))
(("--disk-capacity" value . tail)
(loop tail positional system-name store-dir value root-size target rootfs))
(loop tail positional system-name store-dir value root-size target install-target-device rootfs))
(((? (lambda (arg) (string-prefix? "--root-size=" arg)) arg) . tail)
(loop tail positional system-name store-dir disk-capacity (option-value arg "--root-size=") target rootfs))
(loop tail positional system-name store-dir disk-capacity (option-value arg "--root-size=") target install-target-device rootfs))
(("--root-size" value . tail)
(loop tail positional system-name store-dir disk-capacity value target rootfs))
(loop tail positional system-name store-dir disk-capacity value target install-target-device rootfs))
(((? (lambda (arg) (string-prefix? "--target=" arg)) arg) . tail)
(loop tail positional system-name store-dir disk-capacity root-size (option-value arg "--target=") rootfs))
(loop tail positional system-name store-dir disk-capacity root-size (option-value arg "--target=") install-target-device rootfs))
(("--target" value . tail)
(loop tail positional system-name store-dir disk-capacity root-size value rootfs))
(loop tail positional system-name store-dir disk-capacity root-size value install-target-device rootfs))
(((? (lambda (arg) (string-prefix? "--install-target-device=" arg)) arg) . tail)
(loop tail positional system-name store-dir disk-capacity root-size target
(option-value arg "--install-target-device=") rootfs))
(("--install-target-device" value . tail)
(loop tail positional system-name store-dir disk-capacity root-size target value rootfs))
(((? (lambda (arg) (string-prefix? "--rootfs=" arg)) arg) . tail)
(loop tail positional system-name store-dir disk-capacity root-size target (option-value arg "--rootfs=")))
(loop tail positional system-name store-dir disk-capacity root-size target install-target-device
(option-value arg "--rootfs=")))
(("--rootfs" value . tail)
(loop tail positional system-name store-dir disk-capacity root-size target value))
(loop tail positional system-name store-dir disk-capacity root-size target install-target-device value))
(((? (lambda (arg) (string-prefix? "--" arg)) arg) . _)
(error "unknown option" arg))
((arg . tail)
(loop tail (cons arg positional) system-name store-dir disk-capacity root-size target rootfs)))))
(loop tail (cons arg positional) system-name store-dir disk-capacity root-size target install-target-device rootfs)))))
(define (parse-source-arguments action rest)
(let loop ((args rest)
@@ -399,6 +410,72 @@ Common options:\n\
(usr_src_newvers_sha256 . ,(assoc-ref host-provenance 'usr-src-newvers-sha256))
(store_item_count . ,(length store-items))))))
(define (emit-system-installer-metadata os-file resolved-symbol store-dir os result)
(let* ((installer-image-spec (assoc-ref result 'installer-image-spec))
(image-spec (assoc-ref result 'image-spec))
(store-items (assoc-ref result 'store-items))
(target-store-items (assoc-ref result 'target-store-items))
(installer-store-items (assoc-ref result 'installer-store-items))
(host-base-stores (assoc-ref result 'host-base-stores))
(native-base-stores (assoc-ref result 'native-base-stores))
(fruix-runtime-stores (assoc-ref result 'fruix-runtime-stores))
(base (operating-system-freebsd-base os))
(source (freebsd-base-source base))
(host-provenance (call-with-input-file (assoc-ref result 'host-base-provenance-file) read)))
(emit-metadata
`((action . "installer")
(os_file . ,os-file)
(system_variable . ,resolved-symbol)
(store_dir . ,store-dir)
(freebsd_base_name . ,(freebsd-base-name base))
(freebsd_base_version_label . ,(freebsd-base-version-label base))
(freebsd_base_release . ,(freebsd-base-release base))
(freebsd_base_branch . ,(freebsd-base-branch base))
(freebsd_base_source_root . ,(freebsd-base-source-root base))
(freebsd_base_target . ,(freebsd-base-target base))
(freebsd_base_target_arch . ,(freebsd-base-target-arch base))
(freebsd_base_kernconf . ,(freebsd-base-kernconf base))
(freebsd_base_file . ,(assoc-ref result 'freebsd-base-file))
(freebsd_source_name . ,(freebsd-source-name source))
(freebsd_source_kind . ,(freebsd-source-kind source))
(freebsd_source_url . ,(or (freebsd-source-url source) ""))
(freebsd_source_path . ,(or (freebsd-source-path source) ""))
(freebsd_source_ref . ,(or (freebsd-source-ref source) ""))
(freebsd_source_commit . ,(or (freebsd-source-commit source) ""))
(freebsd_source_sha256 . ,(or (freebsd-source-sha256 source) ""))
(freebsd_source_file . ,(assoc-ref result 'freebsd-source-file))
(freebsd_source_materializations_file . ,(assoc-ref result 'freebsd-source-materializations-file))
(materialized_source_store_count . ,(length (assoc-ref result 'materialized-source-stores)))
(materialized_source_stores . ,(string-join (assoc-ref result 'materialized-source-stores) ","))
(disk_capacity . ,(assoc-ref image-spec 'disk-capacity))
(root_size . ,(assoc-ref image-spec 'root-size))
(installer_host_name . ,(assoc-ref installer-image-spec 'installer-host-name))
(install_target_device . ,(assoc-ref result 'install-target-device))
(installer_state_path . ,(assoc-ref result 'installer-state-path))
(installer_log_path . ,(assoc-ref result 'installer-log-path))
(image_store_path . ,(assoc-ref result 'image-store-path))
(disk_image . ,(assoc-ref result 'disk-image))
(esp_image . ,(assoc-ref result 'esp-image))
(root_image . ,(assoc-ref result 'root-image))
(installer_closure_path . ,(assoc-ref result 'installer-closure-path))
(target_closure_path . ,(assoc-ref result 'target-closure-path))
(host_base_store_count . ,(length host-base-stores))
(host_base_stores . ,(string-join host-base-stores ","))
(native_base_store_count . ,(length native-base-stores))
(native_base_stores . ,(string-join native-base-stores ","))
(fruix_runtime_store_count . ,(length fruix-runtime-stores))
(fruix_runtime_stores . ,(string-join fruix-runtime-stores ","))
(host_base_provenance_file . ,(assoc-ref result 'host-base-provenance-file))
(store_layout_file . ,(assoc-ref result 'store-layout-file))
(host_freebsd_version . ,(assoc-ref host-provenance 'freebsd-version-kru))
(host_uname . ,(assoc-ref host-provenance 'uname))
(usr_src_git_revision . ,(assoc-ref host-provenance 'usr-src-git-revision))
(usr_src_git_branch . ,(assoc-ref host-provenance 'usr-src-git-branch))
(usr_src_newvers_sha256 . ,(assoc-ref host-provenance 'usr-src-newvers-sha256))
(store_item_count . ,(length store-items))
(target_store_item_count . ,(length target-store-items))
(installer_store_item_count . ,(length installer-store-items))))))
(define (main argv)
(let* ((parsed (parse-arguments argv))
(command (assoc-ref parsed 'command))
@@ -410,10 +487,11 @@ Common options:\n\
(disk-capacity (assoc-ref parsed 'disk-capacity))
(root-size (assoc-ref parsed 'root-size))
(target-opt (assoc-ref parsed 'target))
(install-target-device (assoc-ref parsed 'install-target-device))
(rootfs-opt (assoc-ref parsed 'rootfs))
(system-name (assoc-ref parsed 'system-name))
(requested-symbol (and system-name (string->symbol system-name))))
(unless (member action '("build" "image" "install" "rootfs"))
(unless (member action '("build" "image" "installer" "install" "rootfs"))
(error "unknown system action" action))
(let* ((os-file (match positional
((file . _) file)
@@ -472,6 +550,17 @@ Common options:\n\
#:shepherd-prefix shepherd-prefix
#:root-size (or root-size "256m")
#:disk-capacity disk-capacity)))
((string=? action "installer")
(emit-system-installer-metadata
os-file resolved-symbol store-dir os
(materialize-installer-image os
#:store-dir store-dir
#:guile-prefix guile-prefix
#:guile-extra-prefix guile-extra-prefix
#:shepherd-prefix shepherd-prefix
#:install-target-device (or install-target-device "/dev/vtbd1")
#:root-size (or root-size "10g")
#:disk-capacity disk-capacity)))
((string=? action "install")
(unless target
(error "install action requires TARGET or --target PATH"))

91
tests/system/phase18-installer-target-operating-system.scm.in Normal file
View File

@@ -0,0 +1,91 @@
(use-modules (fruix system freebsd)
(fruix packages freebsd))
(define phase18-source
(freebsd-source
#:name "__SOURCE_NAME__"
#:kind 'git
#:ref "__SOURCE_REF__"
#:commit "__SOURCE_COMMIT__"))
(define phase18-base
(freebsd-base
#:name "__BASE_NAME__"
#:version-label "__BASE_VERSION_LABEL__"
#:release "__BASE_RELEASE__"
#:branch "__BASE_BRANCH__"
#:source phase18-source
#:source-root "__DECLARED_SOURCE_ROOT__"
#:target "amd64"
#:target-arch "amd64"
#:kernconf "GENERIC"))
(define phase18-target-operating-system
(operating-system
#:host-name "fruix-freebsd"
#:freebsd-base phase18-base
#:kernel (freebsd-native-kernel-for phase18-base)
#:bootloader (freebsd-native-bootloader-for phase18-base)
#:base-packages (freebsd-native-system-packages-for phase18-base)
#:groups (list (user-group #:name "wheel" #:gid 0 #:system? #t)
(user-group #:name "sshd" #:gid 22 #:system? #t)
(user-group #:name "_dhcp" #:gid 65 #:system? #t)
(user-group #:name "operator" #:gid 1000 #:system? #f))
#:users (list (user-account #:name "root"
#:uid 0
#:group "wheel"
#:comment "Charlie &"
#:home "/root"
#:shell "/bin/sh"
#:system? #t)
(user-account #:name "sshd"
#:uid 22
#:group "sshd"
#:comment "Secure Shell Daemon"
#:home "/var/empty"
#:shell "/usr/sbin/nologin"
#:system? #t)
(user-account #:name "_dhcp"
#:uid 65
#:group "_dhcp"
#:comment "dhcp programs"
#:home "/var/empty"
#:shell "/usr/sbin/nologin"
#:system? #t)
(user-account #:name "operator"
#:uid 1000
#:group "operator"
#:supplementary-groups '("wheel")
#:comment "Fruix Operator"
#:home "/home/operator"
#:shell "/bin/sh"
#:system? #f))
#:file-systems (list (file-system #:device "/dev/gpt/fruix-root"
#:mount-point "/"
#:type "ufs"
#:options "rw"
#:needed-for-boot? #t)
(file-system #:device "devfs"
#:mount-point "/dev"
#:type "devfs"
#:options "rw"
#:needed-for-boot? #t)
(file-system #:device "tmpfs"
#:mount-point "/tmp"
#:type "tmpfs"
#:options "rw,size=64m"))
#:services '(shepherd ready-marker sshd)
#:loader-entries '(("autoboot_delay" . "1")
("boot_multicons" . "YES")
("boot_serial" . "YES")
("console" . "comconsole,vidconsole"))
#:rc-conf-entries '(("clear_tmp_enable" . "NO")
("hostid_enable" . "NO")
("sendmail_enable" . "NONE")
("sshd_enable" . "YES")
("ifconfig_xn0" . "SYNCDHCP")
("ifconfig_em0" . "SYNCDHCP")
("ifconfig_vtnet0" . "SYNCDHCP"))
#:init-mode 'freebsd-init+rc.d-shepherd
#:ready-marker "/var/lib/fruix/ready"
#:root-authorized-keys '("__ROOT_AUTHORIZED_KEY__")))

399
tests/system/run-phase18-installer-environment.sh Executable file
View File

@@ -0,0 +1,399 @@
#!/bin/sh
set -eu
project_root=${PROJECT_ROOT:-$(pwd)}
script_dir=$(CDPATH= cd -- "$(dirname "$0")" && pwd)
fruix_cmd=$project_root/bin/fruix
os_template=${OS_TEMPLATE:-$script_dir/phase18-installer-target-operating-system.scm.in}
system_name=${SYSTEM_NAME:-phase18-target-operating-system}
store_dir=${STORE_DIR:-/frx/store}
installer_disk_capacity=${INSTALLER_DISK_CAPACITY:-16g}
installer_root_size=${INSTALLER_ROOT_SIZE:-14g}
target_disk_capacity=${TARGET_DISK_CAPACITY:-12g}
install_target_device=${INSTALL_TARGET_DEVICE:-/dev/vtbd1}
qemu_smp=${QEMU_SMP:-2}
installer_ssh_port=${INSTALLER_SSH_PORT:-10025}
target_ssh_port=${TARGET_SSH_PORT:-10026}
base_name=${BASE_NAME:-phase18-installer-target}
base_version_label=${BASE_VERSION_LABEL:-15.0-STABLE-installer-target}
base_release=${BASE_RELEASE:-15.0-STABLE}
base_branch=${BASE_BRANCH:-stable/15}
source_name=${SOURCE_NAME:-stable15-installer-target-source}
source_ref=${SOURCE_REF:-stable/15}
source_commit=${SOURCE_COMMIT:-332708a606f6bf0841c1d4a74c0d067f5640fe89}
declared_source_root=${DECLARED_SOURCE_ROOT:-/var/empty/fruix-unused-source-root-installer-target}
metadata_target=${METADATA_OUT:-}
root_authorized_key_file=${ROOT_AUTHORIZED_KEY_FILE:-$HOME/.ssh/id_ed25519.pub}
root_ssh_private_key_file=${ROOT_SSH_PRIVATE_KEY_FILE:-$HOME/.ssh/id_ed25519}
[ -x "$fruix_cmd" ] || {
echo "fruix command is not executable: $fruix_cmd" >&2
exit 1
}
[ -f "$os_template" ] || {
echo "missing operating-system template: $os_template" >&2
exit 1
}
[ -f "$root_authorized_key_file" ] || {
echo "missing root authorized key file: $root_authorized_key_file" >&2
exit 1
}
[ -f "$root_ssh_private_key_file" ] || {
echo "missing root SSH private key file: $root_ssh_private_key_file" >&2
exit 1
}
command -v qemu-system-x86_64 >/dev/null 2>&1 || {
echo "qemu-system-x86_64 is required" >&2
exit 1
}
[ -f /usr/local/share/edk2-qemu/QEMU_UEFI_CODE-x86_64.fd ] || {
echo "missing QEMU UEFI firmware" >&2
exit 1
}
cleanup=0
if [ -n "${WORKDIR:-}" ]; then
workdir=$WORKDIR
mkdir -p "$workdir"
else
workdir=$(mktemp -d /tmp/fruix-phase18-installer.XXXXXX)
cleanup=1
fi
if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
cleanup=0
fi
target_os_file=$workdir/phase18-installer-target-operating-system.scm
installer_out=$workdir/installer.txt
metadata_file=$workdir/phase18-installer-environment-metadata.txt
installer_serial_log=$workdir/installer-serial.log
target_serial_log=$workdir/target-serial.log
installer_qemu_pidfile=$workdir/installer-qemu.pid
target_qemu_pidfile=$workdir/target-qemu.pid
installer_uefi_vars=$workdir/installer-vars.fd
target_uefi_vars=$workdir/target-vars.fd
installer_boot_image=$workdir/installer-boot.img
target_image=$workdir/installed-target.img
gpart_log=$workdir/gpart-show.txt
mnt_esp=$workdir/mnt-esp
mnt_root=$workdir/mnt-root
md_unit=
cleanup_workdir() {
if [ -f "$installer_qemu_pidfile" ]; then
sudo kill "$(sudo cat "$installer_qemu_pidfile")" >/dev/null 2>&1 || true
fi
if [ -f "$target_qemu_pidfile" ]; then
sudo kill "$(sudo cat "$target_qemu_pidfile")" >/dev/null 2>&1 || true
fi
if [ -n "$md_unit" ]; then
sudo umount "$mnt_esp" >/dev/null 2>&1 || true
sudo umount "$mnt_root" >/dev/null 2>&1 || true
sudo mdconfig -d -u "$md_unit" >/dev/null 2>&1 || true
fi
if [ "$cleanup" -eq 1 ]; then
rm -rf "$workdir" 2>/dev/null || sudo rm -rf "$workdir"
fi
}
trap cleanup_workdir EXIT INT TERM
root_authorized_key=$(tr -d '\n' < "$root_authorized_key_file")
sed \
-e "s|__BASE_NAME__|$base_name|g" \
-e "s|__BASE_VERSION_LABEL__|$base_version_label|g" \
-e "s|__BASE_RELEASE__|$base_release|g" \
-e "s|__BASE_BRANCH__|$base_branch|g" \
-e "s|__SOURCE_NAME__|$source_name|g" \
-e "s|__SOURCE_REF__|$source_ref|g" \
-e "s|__SOURCE_COMMIT__|$source_commit|g" \
-e "s|__DECLARED_SOURCE_ROOT__|$declared_source_root|g" \
-e "s|__ROOT_AUTHORIZED_KEY__|$root_authorized_key|g" \
"$os_template" > "$target_os_file"
cp /usr/local/share/edk2-qemu/QEMU_UEFI_VARS-x86_64.fd "$installer_uefi_vars"
cp /usr/local/share/edk2-qemu/QEMU_UEFI_VARS-x86_64.fd "$target_uefi_vars"
truncate -s "$target_disk_capacity" "$target_image"
mkdir -p "$mnt_esp" "$mnt_root"
action_env() {
sudo env \
HOME="$HOME" \
GUILE_AUTO_COMPILE=0 \
FRUIX_FREEBSD_BUILD_JOBS="${FRUIX_FREEBSD_BUILD_JOBS:-8}" \
GUIX_SOURCE_DIR="${GUIX_SOURCE_DIR:-$HOME/repos/guix}" \
GUILE_BIN="${GUILE_BIN:-/tmp/guile-freebsd-validate-install/bin/guile}" \
GUILE_EXTRA_PREFIX="${GUILE_EXTRA_PREFIX:-/tmp/guile-gnutls-freebsd-validate-install}" \
SHEPHERD_PREFIX="${SHEPHERD_PREFIX:-/tmp/shepherd-freebsd-validate-install}" \
"$@"
}
action_env "$fruix_cmd" system installer "$target_os_file" \
--system "$system_name" \
--store "$store_dir" \
--install-target-device "$install_target_device" \
--disk-capacity "$installer_disk_capacity" \
--root-size "$installer_root_size" >"$installer_out"
field() {
sed -n "s/^$1=//p" "$installer_out" | tail -n 1
}
image_store_path=$(field image_store_path)
installer_disk_image=$(field disk_image)
installer_esp_image=$(field esp_image)
installer_root_image=$(field root_image)
installer_closure_path=$(field installer_closure_path)
target_closure_path=$(field target_closure_path)
installer_host_name=$(field installer_host_name)
install_target_device_out=$(field install_target_device)
installer_state_path=$(field installer_state_path)
installer_log_path=$(field installer_log_path)
freebsd_source_kind_out=$(field freebsd_source_kind)
freebsd_source_ref_out=$(field freebsd_source_ref)
freebsd_source_commit_out=$(field freebsd_source_commit)
freebsd_source_file=$(field freebsd_source_file)
freebsd_source_materializations_file=$(field freebsd_source_materializations_file)
materialized_source_store_count=$(field materialized_source_store_count)
materialized_source_stores=$(field materialized_source_stores)
host_base_store_count=$(field host_base_store_count)
native_base_store_count=$(field native_base_store_count)
native_base_stores=$(field native_base_stores)
store_item_count=$(field store_item_count)
target_store_item_count=$(field target_store_item_count)
installer_store_item_count=$(field installer_store_item_count)
store_layout_file=$(field store_layout_file)
[ -d "$image_store_path" ] || { echo "missing installer image store path: $image_store_path" >&2; exit 1; }
[ -f "$installer_disk_image" ] || { echo "missing installer disk image: $installer_disk_image" >&2; exit 1; }
[ -f "$installer_esp_image" ] || { echo "missing installer ESP image: $installer_esp_image" >&2; exit 1; }
[ -f "$installer_root_image" ] || { echo "missing installer root image: $installer_root_image" >&2; exit 1; }
[ -n "$installer_closure_path" ] || { echo "missing installer closure path" >&2; exit 1; }
[ -n "$target_closure_path" ] || { echo "missing target closure path" >&2; exit 1; }
[ "$install_target_device_out" = "$install_target_device" ] || { echo "unexpected install target device: $install_target_device_out" >&2; exit 1; }
[ "$installer_host_name" = fruix-freebsd-installer ] || { echo "unexpected installer host name: $installer_host_name" >&2; exit 1; }
[ "$freebsd_source_kind_out" = git ] || { echo "unexpected source kind: $freebsd_source_kind_out" >&2; exit 1; }
[ "$freebsd_source_ref_out" = "$source_ref" ] || { echo "unexpected source ref: $freebsd_source_ref_out" >&2; exit 1; }
[ "$freebsd_source_commit_out" = "$source_commit" ] || { echo "unexpected source commit: $freebsd_source_commit_out" >&2; exit 1; }
[ "$materialized_source_store_count" = 1 ] || { echo "unexpected materialized source store count: $materialized_source_store_count" >&2; exit 1; }
[ "$host_base_store_count" = 0 ] || { echo "expected zero host base stores, got: $host_base_store_count" >&2; exit 1; }
[ "$native_base_store_count" = 3 ] || { echo "expected three native base stores, got: $native_base_store_count" >&2; exit 1; }
[ -f "$freebsd_source_file" ] || { echo "missing freebsd source file: $freebsd_source_file" >&2; exit 1; }
[ -f "$freebsd_source_materializations_file" ] || { echo "missing source materializations file: $freebsd_source_materializations_file" >&2; exit 1; }
[ -f "$store_layout_file" ] || { echo "missing store layout file: $store_layout_file" >&2; exit 1; }
case "$materialized_source_stores" in
/frx/store/*-freebsd-source-$source_name) : ;;
*) echo "unexpected materialized source store path: $materialized_source_stores" >&2; exit 1 ;;
esac
[ "$store_item_count" -ge "$target_store_item_count" ] || { echo "combined store item count smaller than target store item count" >&2; exit 1; }
[ "$installer_store_item_count" -ge 1 ] || { echo "expected installer store items" >&2; exit 1; }
cp "$installer_disk_image" "$installer_boot_image"
target_closure_base=$(basename "$target_closure_path")
installer_closure_base=$(basename "$installer_closure_path")
sudo qemu-system-x86_64 \
-machine q35,accel=tcg \
-cpu max \
-m 2048 \
-smp "$qemu_smp" \
-display none \
-serial "file:$installer_serial_log" \
-monitor none \
-pidfile "$installer_qemu_pidfile" \
-daemonize \
-drive if=pflash,format=raw,readonly=on,file=/usr/local/share/edk2-qemu/QEMU_UEFI_CODE-x86_64.fd \
-drive if=pflash,format=raw,file="$installer_uefi_vars" \
-drive if=virtio,format=raw,file="$installer_boot_image" \
-drive if=virtio,format=raw,file="$target_image" \
-netdev user,id=net0,hostfwd=tcp::${installer_ssh_port}-:22 \
-device virtio-net-pci,netdev=net0
installer_guest() {
ssh -p "$installer_ssh_port" -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o LogLevel=ERROR \
-o ConnectTimeout=5 \
root@127.0.0.1 "$@"
}
installer_ssh_reached=0
installer_state=missing
for attempt in $(jot 150 1 150); do
if installer_guest 'service sshd onestatus >/dev/null 2>&1' >/dev/null 2>&1; then
installer_ssh_reached=1
installer_state=$(installer_guest "cat '$installer_state_path' 2>/dev/null || echo missing")
[ "$installer_state" = done ] && break
fi
sleep 2
done
[ "$installer_ssh_reached" = 1 ] || { echo "installer environment never became reachable over SSH" >&2; exit 1; }
[ "$installer_state" = done ] || { echo "installer environment did not finish installation: $installer_state" >&2; exit 1; }
installer_run_current_system=$(installer_guest 'readlink /run/current-system')
installer_sshd_status=$(installer_guest 'service sshd onestatus >/dev/null 2>&1 && echo running || echo stopped')
installer_activate_log=$(installer_guest 'cat /var/log/fruix-activate.log 2>/dev/null || true' | tr '\n' ' ')
installer_log=$(installer_guest "cat '$installer_log_path' 2>/dev/null || true" | tr '\n' ' ')
[ "$installer_run_current_system" = "/frx/store/$installer_closure_base" ] || { echo "unexpected installer current-system target: $installer_run_current_system" >&2; exit 1; }
[ "$installer_sshd_status" = running ] || { echo "installer sshd is not running" >&2; exit 1; }
case "$installer_activate_log" in
*fruix-activate:done*) : ;;
*) echo "installer activation log does not show success" >&2; exit 1 ;;
esac
case "$installer_log" in
*fruix-installer:done*) : ;;
*) echo "installer log does not show completion" >&2; exit 1 ;;
esac
sudo kill "$(sudo cat "$installer_qemu_pidfile")" >/dev/null 2>&1 || true
rm -f "$installer_qemu_pidfile"
sleep 2
md=$(sudo mdconfig -a -t vnode -f "$target_image")
md_unit=${md#md}
sudo gpart show -lp "/dev/$md" >"$gpart_log"
esp_fstype=$(sudo fstyp "/dev/${md}p1")
root_fstype=$(sudo fstyp "/dev/${md}p2")
[ "$esp_fstype" = msdosfs ] || { echo "unexpected target ESP filesystem: $esp_fstype" >&2; exit 1; }
[ "$root_fstype" = ufs ] || { echo "unexpected target root filesystem: $root_fstype" >&2; exit 1; }
sudo mount -t msdosfs "/dev/${md}p1" "$mnt_esp"
sudo mount -t ufs -o ro "/dev/${md}p2" "$mnt_root"
[ -f "$mnt_esp/EFI/BOOT/BOOTX64.EFI" ] || { echo "missing EFI boot file on installed target" >&2; exit 1; }
target_run_current_system=$(readlink "$mnt_root/run/current-system")
target_boot_loader=$(readlink "$mnt_root/boot/loader")
install_metadata_host=$(cat "$mnt_root/var/lib/fruix/install.scm")
[ "$target_run_current_system" = "/frx/store/$target_closure_base" ] || { echo "unexpected target /run/current-system target: $target_run_current_system" >&2; exit 1; }
[ "$target_boot_loader" = /run/current-system/boot/loader ] || { echo "unexpected target boot loader link: $target_boot_loader" >&2; exit 1; }
[ -d "$mnt_root/frx/store/$target_closure_base" ] || { echo "installed target closure missing from target root" >&2; exit 1; }
case "$install_metadata_host" in
*"$target_closure_path"*) : ;;
*) echo "installed target metadata does not record target closure path" >&2; exit 1 ;;
esac
case "$install_metadata_host" in
*"$materialized_source_stores"*) : ;;
*) echo "installed target metadata does not record materialized source store" >&2; exit 1 ;;
esac
sudo umount "$mnt_esp"
sudo umount "$mnt_root"
sudo mdconfig -d -u "$md_unit"
md_unit=
sudo qemu-system-x86_64 \
-machine q35,accel=tcg \
-cpu max \
-m 2048 \
-smp "$qemu_smp" \
-display none \
-serial "file:$target_serial_log" \
-monitor none \
-pidfile "$target_qemu_pidfile" \
-daemonize \
-drive if=pflash,format=raw,readonly=on,file=/usr/local/share/edk2-qemu/QEMU_UEFI_CODE-x86_64.fd \
-drive if=pflash,format=raw,file="$target_uefi_vars" \
-drive if=virtio,format=raw,file="$target_image" \
-netdev user,id=net0,hostfwd=tcp::${target_ssh_port}-:22 \
-device virtio-net-pci,netdev=net0
target_guest() {
ssh -p "$target_ssh_port" -i "$root_ssh_private_key_file" \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o LogLevel=ERROR \
-o ConnectTimeout=5 \
root@127.0.0.1 "$@"
}
for attempt in $(jot 120 1 120); do
if target_guest 'service sshd onestatus >/dev/null 2>&1' >/dev/null 2>&1; then
break
fi
sleep 2
done
target_run_current_system_guest=$(target_guest 'readlink /run/current-system')
target_shepherd_status=$(target_guest '/usr/local/etc/rc.d/fruix-shepherd onestatus >/dev/null 2>&1 && echo running || echo stopped')
target_sshd_status=$(target_guest 'service sshd onestatus >/dev/null 2>&1 && echo running || echo stopped')
target_install_metadata_guest=$(target_guest 'cat /var/lib/fruix/install.scm')
target_activate_log=$(target_guest 'cat /var/log/fruix-activate.log 2>/dev/null || true' | tr '\n' ' ')
[ "$target_run_current_system_guest" = "/frx/store/$target_closure_base" ] || { echo "unexpected booted target current-system: $target_run_current_system_guest" >&2; exit 1; }
[ "$target_shepherd_status" = running ] || { echo "fruix-shepherd is not running in booted target" >&2; exit 1; }
[ "$target_sshd_status" = running ] || { echo "sshd is not running in booted target" >&2; exit 1; }
case "$target_install_metadata_guest" in
*"$target_closure_path"*) : ;;
*) echo "booted target metadata does not record target closure path" >&2; exit 1 ;;
esac
case "$target_install_metadata_guest" in
*"$materialized_source_stores"*) : ;;
*) echo "booted target metadata does not record materialized source store" >&2; exit 1 ;;
esac
case "$target_activate_log" in
*fruix-activate:done*) : ;;
*) echo "booted target activation log does not show success" >&2; exit 1 ;;
esac
cat >"$metadata_file" <<EOF
workdir=$workdir
target_os_file=$target_os_file
installer_image_store_path=$image_store_path
installer_disk_image=$installer_disk_image
installer_boot_image=$installer_boot_image
installer_disk_capacity=$installer_disk_capacity
installer_root_size=$installer_root_size
target_image=$target_image
target_disk_capacity=$target_disk_capacity
install_target_device=$install_target_device
qemu_smp=$qemu_smp
freebsd_source_kind=$freebsd_source_kind_out
freebsd_source_ref=$freebsd_source_ref_out
freebsd_source_commit=$freebsd_source_commit_out
freebsd_source_file=$freebsd_source_file
freebsd_source_materializations_file=$freebsd_source_materializations_file
materialized_source_store_count=$materialized_source_store_count
materialized_source_store=$materialized_source_stores
installer_closure_path=$installer_closure_path
target_closure_path=$target_closure_path
native_base_store_count=$native_base_store_count
native_base_stores=$native_base_stores
store_item_count=$store_item_count
target_store_item_count=$target_store_item_count
installer_store_item_count=$installer_store_item_count
installer_state_path=$installer_state_path
installer_log_path=$installer_log_path
installer_state=$installer_state
installer_run_current_system=$installer_run_current_system
installer_sshd_status=$installer_sshd_status
installer_serial_log=$installer_serial_log
target_esp_fstype=$esp_fstype
target_root_fstype=$root_fstype
gpart_log=$gpart_log
target_run_current_system=$target_run_current_system_guest
target_shepherd_status=$target_shepherd_status
target_sshd_status=$target_sshd_status
target_serial_log=$target_serial_log
installer_environment_boot=ok
installer_environment_install=ok
installed_target_boot=ok
EOF
if [ -n "$metadata_target" ]; then
mkdir -p "$(dirname "$metadata_target")"
cp "$metadata_file" "$metadata_target"
fi
printf 'PASS phase18-installer-environment\n'
printf 'Work directory: %s\n' "$workdir"
printf 'Metadata file: %s\n' "$metadata_file"
if [ -n "$metadata_target" ]; then
printf 'Copied metadata to: %s\n' "$metadata_target"
fi
printf '%s\n' '--- metadata ---'
cat "$metadata_file"