diff --git a/docs/PROGRESS.md b/docs/PROGRESS.md index dfb6291..257536d 100644 --- a/docs/PROGRESS.md +++ b/docs/PROGRESS.md @@ -2041,7 +2041,44 @@ Next recommended step: - `/frx` as the canonical store root - stable upstream-derived internal names unless there is strong architectural value in renaming them +## 2026-04-01 — Phase 8.1 completed: reproducible bhyve-compatible raw disk image generated + +Completed work: + +- added the first Phase 8 image-generation harness: + - `tests/system/run-phase8-bhyve-image.sh` +- wrote the Phase 8.1 report: + - `docs/reports/phase8-bhyve-image-freebsd.md` +- ran the image-generation harness successfully and captured metadata under: + - `/tmp/phase8-bhyve-image-metadata.txt` + +Important findings: + +- the current Fruix FreeBSD track now has a reproducible raw disk-image build path using: + - GPT + - UEFI boot + - a FAT EFI system partition + - a UFS root partition + - serial-console-friendly loader settings +- the earlier Phase 7 rootfs tree was not sufficient by itself for an installable image because it still referenced `/frx/store` content that only existed on the host; the image builder therefore had to stage the system closure and its recursively declared store references inside the image rootfs under `/frx/store` +- rebuilding the same image a second time with fixed timestamps and explicit filesystem parameters produced the same SHA256, which is the current prototype proof of reproducible image generation on this host +- observed metadata confirmed: + - `raw_sha256=08605d738021cb6fb5b87c270e1eafde57e1acb5159d3a2257aad4c560e2efc5` + - `image_size_bytes=335578624` + - `esp_fstype=msdosfs` + - `root_fstype=ufs` + - `run_current_system_target=/frx/store/...-fruix-system-fruix-freebsd` + - `boot_loader_target=/run/current-system/boot/loader` + - `boot_loader_conf_target=/run/current-system/boot/loader.conf` + - `rc_conf_target=/run/current-system/etc/rc.conf` + - `rc_script_target=/run/current-system/usr/local/etc/rc.d/fruix-shepherd` + - `store_item_count=13` + - `boot_mode=uefi` + - `image_format=raw` + - `partition_scheme=gpt` + - `serial_console=comconsole` + Current assessment: -- Phase 7.1 is now satisfied on the current FreeBSD prototype track -- the next step is to materialize this operating-system description into a reproducible system closure under `/frx/store` +- Phase 8.1 is now satisfied on the current FreeBSD prototype track +- the next step is to integrate this image generation path into the declarative Fruix system-composition layer so that a single operating-system description can drive image generation end-to-end diff --git a/docs/reports/phase8-bhyve-image-freebsd.md b/docs/reports/phase8-bhyve-image-freebsd.md new file mode 100644 index 0000000..932cf4a --- /dev/null +++ b/docs/reports/phase8-bhyve-image-freebsd.md @@ -0,0 +1,84 @@ +# Phase 8.1: Reproducible bhyve-compatible disk image generation on FreeBSD + +Date: 2026-04-01 + +## Summary + +This step adds the first reproducible raw disk-image build path for the FreeBSD Fruix prototype. The target format is intentionally simple and debuggable: + +- raw disk image +- GPT partition map +- UEFI boot +- FreeBSD UFS root partition +- serial-console-friendly loader configuration + +Added file: + +- `tests/system/run-phase8-bhyve-image.sh` + +## Validation command + +Run command: + +```sh +METADATA_OUT=/tmp/phase8-bhyve-image-metadata.txt \ +./tests/system/run-phase8-bhyve-image.sh +``` + +## What the harness does + +The harness: + +1. reuses the Phase 7 rootfs materialization path +2. discovers the generated system closure under `/frx/store` +3. creates an image-oriented staging rootfs +4. copies the closure and its recursively declared store references into: + - `rootfs/frx/store` +5. builds: + - a FAT EFI system partition image containing `EFI/BOOT/BOOTX64.EFI` + - a UFS root partition image from the staged rootfs + - a final GPT raw disk image with labeled partitions +6. rebuilds the same image a second time with fixed timestamps and filesystem parameters +7. verifies that both resulting raw images have the same SHA256 +8. attaches the raw image through `mdconfig` and validates its partition/filesystem structure + +## Observed results + +Observed metadata included: + +- `raw_sha256=08605d738021cb6fb5b87c270e1eafde57e1acb5159d3a2257aad4c560e2efc5` +- `image_size_bytes=335578624` +- `esp_fstype=msdosfs` +- `root_fstype=ufs` +- `run_current_system_target=/frx/store/...-fruix-system-fruix-freebsd` +- `boot_loader_target=/run/current-system/boot/loader` +- `boot_loader_conf_target=/run/current-system/boot/loader.conf` +- `rc_conf_target=/run/current-system/etc/rc.conf` +- `rc_script_target=/run/current-system/usr/local/etc/rc.d/fruix-shepherd` +- `store_item_count=13` +- `boot_mode=uefi` +- `image_format=raw` +- `partition_scheme=gpt` +- `root_partition_label=fruix-root` +- `efi_partition_label=efiboot` +- `serial_console=comconsole` + +## Important findings + +- the current Fruix FreeBSD track now has a reproducible raw disk-image build path suitable for later bhyve work +- the earlier Phase 7 rootfs tree was not sufficient by itself for an installable image because it referenced `/frx/store` content that was still only present on the host; the image builder therefore had to stage the closure and its store references inside the image rootfs under `/frx/store` +- fixed timestamps and explicit filesystem parameters were sufficient to make repeated image builds byte-for-byte reproducible on this host +- boot-structure sanity checks succeeded for: + - GPT partitioning + - EFI partition population + - UFS root partition creation + - serial-console loader configuration + - preserved `run/current-system` system-link topology + +## Conclusion + +Phase 8.1 is satisfied on the current FreeBSD prototype track: + +- a reproducible bhyve-compatible raw disk image can now be generated from the Fruix system outputs +- the resulting image passes static boot-structure sanity checks +- the next step is to move this image builder into the Fruix system-composition layer so image generation becomes an output of the declarative system description itself diff --git a/tests/system/run-phase8-bhyve-image.sh b/tests/system/run-phase8-bhyve-image.sh new file mode 100755 index 0000000..3112eab --- /dev/null +++ b/tests/system/run-phase8-bhyve-image.sh @@ -0,0 +1,261 @@ +#!/bin/sh +set -eu + +repo_root=$(CDPATH= cd -- "$(dirname "$0")/../.." && pwd) +metadata_target=${METADATA_OUT:-} + +cleanup=0 +if [ -n "${WORKDIR:-}" ]; then + workdir=$WORKDIR + mkdir -p "$workdir" +else + workdir=$(mktemp -d /tmp/fruix-phase8-bhyve-image.XXXXXX) + cleanup=1 +fi +if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then + cleanup=0 +fi + +phase7_rootfs_dir=$workdir/phase7-rootfs +phase7_rootfs_log=$workdir/phase7-rootfs.log +phase7_rootfs_metadata=$workdir/phase7-rootfs-metadata.txt +image_rootfs=$workdir/image-rootfs +queue_file=$workdir/store-queue.txt +esp_stage=$workdir/esp-stage +esp_image_a=$workdir/esp-a.img +esp_image_b=$workdir/esp-b.img +root_image_a=$workdir/root-a.ufs +root_image_b=$workdir/root-b.ufs +disk_image_a=$workdir/fruix-bhyve-a.img +disk_image_b=$workdir/fruix-bhyve-b.img +root_makefs_a_log=$workdir/root-makefs-a.log +root_makefs_b_log=$workdir/root-makefs-b.log +esp_makefs_a_log=$workdir/esp-makefs-a.log +esp_makefs_b_log=$workdir/esp-makefs-b.log +mkimg_a_log=$workdir/mkimg-a.log +mkimg_b_log=$workdir/mkimg-b.log +gpart_log=$workdir/gpart-show.txt +metadata_file=$workdir/phase8-bhyve-image-metadata.txt +mnt_esp=$workdir/mnt-esp +mnt_root=$workdir/mnt-root +md_unit= + +cleanup_workdir() { + if [ -n "$md_unit" ]; then + sudo umount "$mnt_esp" >/dev/null 2>&1 || true + sudo umount "$mnt_root" >/dev/null 2>&1 || true + sudo mdconfig -d -u "$md_unit" >/dev/null 2>&1 || true + fi + if [ "$cleanup" -eq 1 ]; then + rm -rf "$workdir" 2>/dev/null || sudo rm -rf "$workdir" + fi +} +trap cleanup_workdir EXIT INT TERM + +copy_store_closure() { + closure_path=$1 + stage_root=$2 + + : > "$queue_file" + printf '%s\n' "$closure_path" > "$queue_file" + + while IFS= read -r item || [ -n "$item" ]; do + [ -n "$item" ] || continue + staged_item=$stage_root/frx/store/$(basename "$item") + if [ ! -e "$staged_item" ]; then + sudo cp -a "$item" "$staged_item" + if [ -f "$item/.references" ]; then + while IFS= read -r ref || [ -n "$ref" ]; do + [ -n "$ref" ] || continue + if [ ! -e "$stage_root/frx/store/$(basename "$ref")" ]; then + printf '%s\n' "$ref" >> "$queue_file" + fi + done < "$item/.references" + fi + fi + done < "$queue_file" +} + +KEEP_WORKDIR=1 WORKDIR=$phase7_rootfs_dir METADATA_OUT=$phase7_rootfs_metadata \ + "$repo_root/tests/system/run-phase7-rootfs.sh" >"$phase7_rootfs_log" 2>&1 + +rootfs=$phase7_rootfs_dir/rootfs +closure_path=$(readlink "$rootfs/run/current-system") +closure_base=$(basename "$closure_path") + +sudo rm -rf "$image_rootfs" +sudo cp -a "$rootfs" "$image_rootfs" +sudo mkdir -p "$image_rootfs/frx/store" +copy_store_closure "$closure_path" "$image_rootfs" + +rm -rf "$esp_stage" +mkdir -p "$esp_stage/EFI/BOOT" +sudo cp -f "$closure_path/boot/loader.efi" "$esp_stage/EFI/BOOT/BOOTX64.EFI" + +sudo makefs -t ffs -T 0 -B little -s 256m \ + -o label=fruix-root,version=2,bsize=32768,fsize=4096,density=16384 \ + "$root_image_a" "$image_rootfs" >"$root_makefs_a_log" 2>&1 +sudo makefs -t ffs -T 0 -B little -s 256m \ + -o label=fruix-root,version=2,bsize=32768,fsize=4096,density=16384 \ + "$root_image_b" "$image_rootfs" >"$root_makefs_b_log" 2>&1 +makefs -t msdos -T 0 \ + -o fat_type=32 \ + -o sectors_per_cluster=1 \ + -o volume_label=EFISYS \ + -o volume_id=305419896 \ + -s 64m "$esp_image_a" "$esp_stage" >"$esp_makefs_a_log" 2>&1 +makefs -t msdos -T 0 \ + -o fat_type=32 \ + -o sectors_per_cluster=1 \ + -o volume_label=EFISYS \ + -o volume_id=305419896 \ + -s 64m "$esp_image_b" "$esp_stage" >"$esp_makefs_b_log" 2>&1 +mkimg -s gpt -f raw -t 0 \ + -p efi/efiboot:="$esp_image_a" \ + -p freebsd-ufs/fruix-root:="$root_image_a" \ + -o "$disk_image_a" >"$mkimg_a_log" 2>&1 +mkimg -s gpt -f raw -t 0 \ + -p efi/efiboot:="$esp_image_b" \ + -p freebsd-ufs/fruix-root:="$root_image_b" \ + -o "$disk_image_b" >"$mkimg_b_log" 2>&1 + +raw_sha256_a=$(sha256 -q "$disk_image_a") +raw_sha256_b=$(sha256 -q "$disk_image_b") +[ "$raw_sha256_a" = "$raw_sha256_b" ] || { + echo "raw image reproducibility check failed" >&2 + exit 1 +} + +md=$(sudo mdconfig -a -t vnode -f "$disk_image_a") +md_unit=${md#md} +sudo mkdir -p "$mnt_esp" "$mnt_root" +sudo gpart show -lp "/dev/$md" >"$gpart_log" +esp_fstype=$(sudo fstyp "/dev/${md}p1") +root_fstype=$(sudo fstyp "/dev/${md}p2") +[ "$esp_fstype" = msdosfs ] || { echo "unexpected ESP filesystem: $esp_fstype" >&2; exit 1; } +[ "$root_fstype" = ufs ] || { echo "unexpected root filesystem: $root_fstype" >&2; exit 1; } +sudo mount -t msdosfs "/dev/${md}p1" "$mnt_esp" +sudo mount -t ufs -o ro "/dev/${md}p2" "$mnt_root" + +[ -f "$mnt_esp/EFI/BOOT/BOOTX64.EFI" ] || { + echo "missing EFI bootloader in mounted ESP" >&2 + exit 1 +} +run_current_system_target=$(readlink "$mnt_root/run/current-system") +activate_target=$(readlink "$mnt_root/activate") +bin_target=$(readlink "$mnt_root/bin") +boot_loader_target=$(readlink "$mnt_root/boot/loader") +boot_loader_conf_target=$(readlink "$mnt_root/boot/loader.conf") +rc_conf_target=$(readlink "$mnt_root/etc/rc.conf") +rc_script_target=$(readlink "$mnt_root/usr/local/etc/rc.d/fruix-shepherd") +[ "$run_current_system_target" = "/frx/store/$closure_base" ] || { + echo "unexpected /run/current-system target: $run_current_system_target" >&2 + exit 1 +} +[ "$activate_target" = /run/current-system/activate ] || { + echo "unexpected /activate target: $activate_target" >&2 + exit 1 +} +[ "$bin_target" = /run/current-system/profile/bin ] || { + echo "unexpected /bin target: $bin_target" >&2 + exit 1 +} +[ "$boot_loader_target" = /run/current-system/boot/loader ] || { + echo "unexpected /boot/loader target: $boot_loader_target" >&2 + exit 1 +} +[ "$boot_loader_conf_target" = /run/current-system/boot/loader.conf ] || { + echo "unexpected /boot/loader.conf target: $boot_loader_conf_target" >&2 + exit 1 +} +[ "$rc_conf_target" = /run/current-system/etc/rc.conf ] || { + echo "unexpected /etc/rc.conf target: $rc_conf_target" >&2 + exit 1 +} +[ "$rc_script_target" = /run/current-system/usr/local/etc/rc.d/fruix-shepherd ] || { + echo "unexpected fruix_shepherd rc script target: $rc_script_target" >&2 + exit 1 +} +[ -d "$mnt_root/frx/store/$closure_base" ] || { + echo "closure missing from image store population" >&2 + exit 1 +} +loader_conf_image=$mnt_root/frx/store/$closure_base/boot/loader.conf +rc_conf_image=$mnt_root/frx/store/$closure_base/etc/rc.conf +rc_script_image=$mnt_root/frx/store/$closure_base/usr/local/etc/rc.d/fruix-shepherd +shepherd_config_image=$mnt_root/frx/store/$closure_base/shepherd/init.scm +activate_image=$mnt_root/frx/store/$closure_base/activate +for required in "$loader_conf_image" "$rc_conf_image" "$rc_script_image" "$shepherd_config_image" "$activate_image"; do + [ -f "$required" ] || { + echo "required image content missing: $required" >&2 + exit 1 + } +done +grep -F 'console="comconsole"' "$loader_conf_image" >/dev/null || { + echo "loader.conf is missing the serial console setting" >&2 + exit 1 +} +grep -F 'hostname="fruix-freebsd"' "$rc_conf_image" >/dev/null || { + echo "rc.conf is missing the expected hostname" >&2 + exit 1 +} +grep -F 'fruix_shepherd_enable="YES"' "$rc_conf_image" >/dev/null || { + echo "rc.conf does not enable fruix_shepherd" >&2 + exit 1 +} +grep -F '/var/lib/fruix/ready' "$shepherd_config_image" >/dev/null || { + echo "shepherd configuration does not contain the ready marker" >&2 + exit 1 +} +store_item_count=$(find "$mnt_root/frx/store" -mindepth 1 -maxdepth 1 | wc -l | awk '{print $1}') +image_size_bytes=$(stat -f %z "$disk_image_a") + +cat >"$metadata_file" <