Build reproducible FreeBSD bhyve images
This commit is contained in:
@@ -2041,7 +2041,44 @@ Next recommended step:
|
||||
- `/frx` as the canonical store root
|
||||
- stable upstream-derived internal names unless there is strong architectural value in renaming them
|
||||
|
||||
## 2026-04-01 — Phase 8.1 completed: reproducible bhyve-compatible raw disk image generated
|
||||
|
||||
Completed work:
|
||||
|
||||
- added the first Phase 8 image-generation harness:
|
||||
- `tests/system/run-phase8-bhyve-image.sh`
|
||||
- wrote the Phase 8.1 report:
|
||||
- `docs/reports/phase8-bhyve-image-freebsd.md`
|
||||
- ran the image-generation harness successfully and captured metadata under:
|
||||
- `/tmp/phase8-bhyve-image-metadata.txt`
|
||||
|
||||
Important findings:
|
||||
|
||||
- the current Fruix FreeBSD track now has a reproducible raw disk-image build path using:
|
||||
- GPT
|
||||
- UEFI boot
|
||||
- a FAT EFI system partition
|
||||
- a UFS root partition
|
||||
- serial-console-friendly loader settings
|
||||
- the earlier Phase 7 rootfs tree was not sufficient by itself for an installable image because it still referenced `/frx/store` content that only existed on the host; the image builder therefore had to stage the system closure and its recursively declared store references inside the image rootfs under `/frx/store`
|
||||
- rebuilding the same image a second time with fixed timestamps and explicit filesystem parameters produced the same SHA256, which is the current prototype proof of reproducible image generation on this host
|
||||
- observed metadata confirmed:
|
||||
- `raw_sha256=08605d738021cb6fb5b87c270e1eafde57e1acb5159d3a2257aad4c560e2efc5`
|
||||
- `image_size_bytes=335578624`
|
||||
- `esp_fstype=msdosfs`
|
||||
- `root_fstype=ufs`
|
||||
- `run_current_system_target=/frx/store/...-fruix-system-fruix-freebsd`
|
||||
- `boot_loader_target=/run/current-system/boot/loader`
|
||||
- `boot_loader_conf_target=/run/current-system/boot/loader.conf`
|
||||
- `rc_conf_target=/run/current-system/etc/rc.conf`
|
||||
- `rc_script_target=/run/current-system/usr/local/etc/rc.d/fruix-shepherd`
|
||||
- `store_item_count=13`
|
||||
- `boot_mode=uefi`
|
||||
- `image_format=raw`
|
||||
- `partition_scheme=gpt`
|
||||
- `serial_console=comconsole`
|
||||
|
||||
Current assessment:
|
||||
|
||||
- Phase 7.1 is now satisfied on the current FreeBSD prototype track
|
||||
- the next step is to materialize this operating-system description into a reproducible system closure under `/frx/store`
|
||||
- Phase 8.1 is now satisfied on the current FreeBSD prototype track
|
||||
- the next step is to integrate this image generation path into the declarative Fruix system-composition layer so that a single operating-system description can drive image generation end-to-end
|
||||
|
||||
84
docs/reports/phase8-bhyve-image-freebsd.md
Normal file
84
docs/reports/phase8-bhyve-image-freebsd.md
Normal file
@@ -0,0 +1,84 @@
|
||||
# Phase 8.1: Reproducible bhyve-compatible disk image generation on FreeBSD
|
||||
|
||||
Date: 2026-04-01
|
||||
|
||||
## Summary
|
||||
|
||||
This step adds the first reproducible raw disk-image build path for the FreeBSD Fruix prototype. The target format is intentionally simple and debuggable:
|
||||
|
||||
- raw disk image
|
||||
- GPT partition map
|
||||
- UEFI boot
|
||||
- FreeBSD UFS root partition
|
||||
- serial-console-friendly loader configuration
|
||||
|
||||
Added file:
|
||||
|
||||
- `tests/system/run-phase8-bhyve-image.sh`
|
||||
|
||||
## Validation command
|
||||
|
||||
Run command:
|
||||
|
||||
```sh
|
||||
METADATA_OUT=/tmp/phase8-bhyve-image-metadata.txt \
|
||||
./tests/system/run-phase8-bhyve-image.sh
|
||||
```
|
||||
|
||||
## What the harness does
|
||||
|
||||
The harness:
|
||||
|
||||
1. reuses the Phase 7 rootfs materialization path
|
||||
2. discovers the generated system closure under `/frx/store`
|
||||
3. creates an image-oriented staging rootfs
|
||||
4. copies the closure and its recursively declared store references into:
|
||||
- `rootfs/frx/store`
|
||||
5. builds:
|
||||
- a FAT EFI system partition image containing `EFI/BOOT/BOOTX64.EFI`
|
||||
- a UFS root partition image from the staged rootfs
|
||||
- a final GPT raw disk image with labeled partitions
|
||||
6. rebuilds the same image a second time with fixed timestamps and filesystem parameters
|
||||
7. verifies that both resulting raw images have the same SHA256
|
||||
8. attaches the raw image through `mdconfig` and validates its partition/filesystem structure
|
||||
|
||||
## Observed results
|
||||
|
||||
Observed metadata included:
|
||||
|
||||
- `raw_sha256=08605d738021cb6fb5b87c270e1eafde57e1acb5159d3a2257aad4c560e2efc5`
|
||||
- `image_size_bytes=335578624`
|
||||
- `esp_fstype=msdosfs`
|
||||
- `root_fstype=ufs`
|
||||
- `run_current_system_target=/frx/store/...-fruix-system-fruix-freebsd`
|
||||
- `boot_loader_target=/run/current-system/boot/loader`
|
||||
- `boot_loader_conf_target=/run/current-system/boot/loader.conf`
|
||||
- `rc_conf_target=/run/current-system/etc/rc.conf`
|
||||
- `rc_script_target=/run/current-system/usr/local/etc/rc.d/fruix-shepherd`
|
||||
- `store_item_count=13`
|
||||
- `boot_mode=uefi`
|
||||
- `image_format=raw`
|
||||
- `partition_scheme=gpt`
|
||||
- `root_partition_label=fruix-root`
|
||||
- `efi_partition_label=efiboot`
|
||||
- `serial_console=comconsole`
|
||||
|
||||
## Important findings
|
||||
|
||||
- the current Fruix FreeBSD track now has a reproducible raw disk-image build path suitable for later bhyve work
|
||||
- the earlier Phase 7 rootfs tree was not sufficient by itself for an installable image because it referenced `/frx/store` content that was still only present on the host; the image builder therefore had to stage the closure and its store references inside the image rootfs under `/frx/store`
|
||||
- fixed timestamps and explicit filesystem parameters were sufficient to make repeated image builds byte-for-byte reproducible on this host
|
||||
- boot-structure sanity checks succeeded for:
|
||||
- GPT partitioning
|
||||
- EFI partition population
|
||||
- UFS root partition creation
|
||||
- serial-console loader configuration
|
||||
- preserved `run/current-system` system-link topology
|
||||
|
||||
## Conclusion
|
||||
|
||||
Phase 8.1 is satisfied on the current FreeBSD prototype track:
|
||||
|
||||
- a reproducible bhyve-compatible raw disk image can now be generated from the Fruix system outputs
|
||||
- the resulting image passes static boot-structure sanity checks
|
||||
- the next step is to move this image builder into the Fruix system-composition layer so image generation becomes an output of the declarative system description itself
|
||||
261
tests/system/run-phase8-bhyve-image.sh
Executable file
261
tests/system/run-phase8-bhyve-image.sh
Executable file
@@ -0,0 +1,261 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
repo_root=$(CDPATH= cd -- "$(dirname "$0")/../.." && pwd)
|
||||
metadata_target=${METADATA_OUT:-}
|
||||
|
||||
cleanup=0
|
||||
if [ -n "${WORKDIR:-}" ]; then
|
||||
workdir=$WORKDIR
|
||||
mkdir -p "$workdir"
|
||||
else
|
||||
workdir=$(mktemp -d /tmp/fruix-phase8-bhyve-image.XXXXXX)
|
||||
cleanup=1
|
||||
fi
|
||||
if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
|
||||
cleanup=0
|
||||
fi
|
||||
|
||||
phase7_rootfs_dir=$workdir/phase7-rootfs
|
||||
phase7_rootfs_log=$workdir/phase7-rootfs.log
|
||||
phase7_rootfs_metadata=$workdir/phase7-rootfs-metadata.txt
|
||||
image_rootfs=$workdir/image-rootfs
|
||||
queue_file=$workdir/store-queue.txt
|
||||
esp_stage=$workdir/esp-stage
|
||||
esp_image_a=$workdir/esp-a.img
|
||||
esp_image_b=$workdir/esp-b.img
|
||||
root_image_a=$workdir/root-a.ufs
|
||||
root_image_b=$workdir/root-b.ufs
|
||||
disk_image_a=$workdir/fruix-bhyve-a.img
|
||||
disk_image_b=$workdir/fruix-bhyve-b.img
|
||||
root_makefs_a_log=$workdir/root-makefs-a.log
|
||||
root_makefs_b_log=$workdir/root-makefs-b.log
|
||||
esp_makefs_a_log=$workdir/esp-makefs-a.log
|
||||
esp_makefs_b_log=$workdir/esp-makefs-b.log
|
||||
mkimg_a_log=$workdir/mkimg-a.log
|
||||
mkimg_b_log=$workdir/mkimg-b.log
|
||||
gpart_log=$workdir/gpart-show.txt
|
||||
metadata_file=$workdir/phase8-bhyve-image-metadata.txt
|
||||
mnt_esp=$workdir/mnt-esp
|
||||
mnt_root=$workdir/mnt-root
|
||||
md_unit=
|
||||
|
||||
cleanup_workdir() {
|
||||
if [ -n "$md_unit" ]; then
|
||||
sudo umount "$mnt_esp" >/dev/null 2>&1 || true
|
||||
sudo umount "$mnt_root" >/dev/null 2>&1 || true
|
||||
sudo mdconfig -d -u "$md_unit" >/dev/null 2>&1 || true
|
||||
fi
|
||||
if [ "$cleanup" -eq 1 ]; then
|
||||
rm -rf "$workdir" 2>/dev/null || sudo rm -rf "$workdir"
|
||||
fi
|
||||
}
|
||||
trap cleanup_workdir EXIT INT TERM
|
||||
|
||||
copy_store_closure() {
|
||||
closure_path=$1
|
||||
stage_root=$2
|
||||
|
||||
: > "$queue_file"
|
||||
printf '%s\n' "$closure_path" > "$queue_file"
|
||||
|
||||
while IFS= read -r item || [ -n "$item" ]; do
|
||||
[ -n "$item" ] || continue
|
||||
staged_item=$stage_root/frx/store/$(basename "$item")
|
||||
if [ ! -e "$staged_item" ]; then
|
||||
sudo cp -a "$item" "$staged_item"
|
||||
if [ -f "$item/.references" ]; then
|
||||
while IFS= read -r ref || [ -n "$ref" ]; do
|
||||
[ -n "$ref" ] || continue
|
||||
if [ ! -e "$stage_root/frx/store/$(basename "$ref")" ]; then
|
||||
printf '%s\n' "$ref" >> "$queue_file"
|
||||
fi
|
||||
done < "$item/.references"
|
||||
fi
|
||||
fi
|
||||
done < "$queue_file"
|
||||
}
|
||||
|
||||
KEEP_WORKDIR=1 WORKDIR=$phase7_rootfs_dir METADATA_OUT=$phase7_rootfs_metadata \
|
||||
"$repo_root/tests/system/run-phase7-rootfs.sh" >"$phase7_rootfs_log" 2>&1
|
||||
|
||||
rootfs=$phase7_rootfs_dir/rootfs
|
||||
closure_path=$(readlink "$rootfs/run/current-system")
|
||||
closure_base=$(basename "$closure_path")
|
||||
|
||||
sudo rm -rf "$image_rootfs"
|
||||
sudo cp -a "$rootfs" "$image_rootfs"
|
||||
sudo mkdir -p "$image_rootfs/frx/store"
|
||||
copy_store_closure "$closure_path" "$image_rootfs"
|
||||
|
||||
rm -rf "$esp_stage"
|
||||
mkdir -p "$esp_stage/EFI/BOOT"
|
||||
sudo cp -f "$closure_path/boot/loader.efi" "$esp_stage/EFI/BOOT/BOOTX64.EFI"
|
||||
|
||||
sudo makefs -t ffs -T 0 -B little -s 256m \
|
||||
-o label=fruix-root,version=2,bsize=32768,fsize=4096,density=16384 \
|
||||
"$root_image_a" "$image_rootfs" >"$root_makefs_a_log" 2>&1
|
||||
sudo makefs -t ffs -T 0 -B little -s 256m \
|
||||
-o label=fruix-root,version=2,bsize=32768,fsize=4096,density=16384 \
|
||||
"$root_image_b" "$image_rootfs" >"$root_makefs_b_log" 2>&1
|
||||
makefs -t msdos -T 0 \
|
||||
-o fat_type=32 \
|
||||
-o sectors_per_cluster=1 \
|
||||
-o volume_label=EFISYS \
|
||||
-o volume_id=305419896 \
|
||||
-s 64m "$esp_image_a" "$esp_stage" >"$esp_makefs_a_log" 2>&1
|
||||
makefs -t msdos -T 0 \
|
||||
-o fat_type=32 \
|
||||
-o sectors_per_cluster=1 \
|
||||
-o volume_label=EFISYS \
|
||||
-o volume_id=305419896 \
|
||||
-s 64m "$esp_image_b" "$esp_stage" >"$esp_makefs_b_log" 2>&1
|
||||
mkimg -s gpt -f raw -t 0 \
|
||||
-p efi/efiboot:="$esp_image_a" \
|
||||
-p freebsd-ufs/fruix-root:="$root_image_a" \
|
||||
-o "$disk_image_a" >"$mkimg_a_log" 2>&1
|
||||
mkimg -s gpt -f raw -t 0 \
|
||||
-p efi/efiboot:="$esp_image_b" \
|
||||
-p freebsd-ufs/fruix-root:="$root_image_b" \
|
||||
-o "$disk_image_b" >"$mkimg_b_log" 2>&1
|
||||
|
||||
raw_sha256_a=$(sha256 -q "$disk_image_a")
|
||||
raw_sha256_b=$(sha256 -q "$disk_image_b")
|
||||
[ "$raw_sha256_a" = "$raw_sha256_b" ] || {
|
||||
echo "raw image reproducibility check failed" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
md=$(sudo mdconfig -a -t vnode -f "$disk_image_a")
|
||||
md_unit=${md#md}
|
||||
sudo mkdir -p "$mnt_esp" "$mnt_root"
|
||||
sudo gpart show -lp "/dev/$md" >"$gpart_log"
|
||||
esp_fstype=$(sudo fstyp "/dev/${md}p1")
|
||||
root_fstype=$(sudo fstyp "/dev/${md}p2")
|
||||
[ "$esp_fstype" = msdosfs ] || { echo "unexpected ESP filesystem: $esp_fstype" >&2; exit 1; }
|
||||
[ "$root_fstype" = ufs ] || { echo "unexpected root filesystem: $root_fstype" >&2; exit 1; }
|
||||
sudo mount -t msdosfs "/dev/${md}p1" "$mnt_esp"
|
||||
sudo mount -t ufs -o ro "/dev/${md}p2" "$mnt_root"
|
||||
|
||||
[ -f "$mnt_esp/EFI/BOOT/BOOTX64.EFI" ] || {
|
||||
echo "missing EFI bootloader in mounted ESP" >&2
|
||||
exit 1
|
||||
}
|
||||
run_current_system_target=$(readlink "$mnt_root/run/current-system")
|
||||
activate_target=$(readlink "$mnt_root/activate")
|
||||
bin_target=$(readlink "$mnt_root/bin")
|
||||
boot_loader_target=$(readlink "$mnt_root/boot/loader")
|
||||
boot_loader_conf_target=$(readlink "$mnt_root/boot/loader.conf")
|
||||
rc_conf_target=$(readlink "$mnt_root/etc/rc.conf")
|
||||
rc_script_target=$(readlink "$mnt_root/usr/local/etc/rc.d/fruix-shepherd")
|
||||
[ "$run_current_system_target" = "/frx/store/$closure_base" ] || {
|
||||
echo "unexpected /run/current-system target: $run_current_system_target" >&2
|
||||
exit 1
|
||||
}
|
||||
[ "$activate_target" = /run/current-system/activate ] || {
|
||||
echo "unexpected /activate target: $activate_target" >&2
|
||||
exit 1
|
||||
}
|
||||
[ "$bin_target" = /run/current-system/profile/bin ] || {
|
||||
echo "unexpected /bin target: $bin_target" >&2
|
||||
exit 1
|
||||
}
|
||||
[ "$boot_loader_target" = /run/current-system/boot/loader ] || {
|
||||
echo "unexpected /boot/loader target: $boot_loader_target" >&2
|
||||
exit 1
|
||||
}
|
||||
[ "$boot_loader_conf_target" = /run/current-system/boot/loader.conf ] || {
|
||||
echo "unexpected /boot/loader.conf target: $boot_loader_conf_target" >&2
|
||||
exit 1
|
||||
}
|
||||
[ "$rc_conf_target" = /run/current-system/etc/rc.conf ] || {
|
||||
echo "unexpected /etc/rc.conf target: $rc_conf_target" >&2
|
||||
exit 1
|
||||
}
|
||||
[ "$rc_script_target" = /run/current-system/usr/local/etc/rc.d/fruix-shepherd ] || {
|
||||
echo "unexpected fruix_shepherd rc script target: $rc_script_target" >&2
|
||||
exit 1
|
||||
}
|
||||
[ -d "$mnt_root/frx/store/$closure_base" ] || {
|
||||
echo "closure missing from image store population" >&2
|
||||
exit 1
|
||||
}
|
||||
loader_conf_image=$mnt_root/frx/store/$closure_base/boot/loader.conf
|
||||
rc_conf_image=$mnt_root/frx/store/$closure_base/etc/rc.conf
|
||||
rc_script_image=$mnt_root/frx/store/$closure_base/usr/local/etc/rc.d/fruix-shepherd
|
||||
shepherd_config_image=$mnt_root/frx/store/$closure_base/shepherd/init.scm
|
||||
activate_image=$mnt_root/frx/store/$closure_base/activate
|
||||
for required in "$loader_conf_image" "$rc_conf_image" "$rc_script_image" "$shepherd_config_image" "$activate_image"; do
|
||||
[ -f "$required" ] || {
|
||||
echo "required image content missing: $required" >&2
|
||||
exit 1
|
||||
}
|
||||
done
|
||||
grep -F 'console="comconsole"' "$loader_conf_image" >/dev/null || {
|
||||
echo "loader.conf is missing the serial console setting" >&2
|
||||
exit 1
|
||||
}
|
||||
grep -F 'hostname="fruix-freebsd"' "$rc_conf_image" >/dev/null || {
|
||||
echo "rc.conf is missing the expected hostname" >&2
|
||||
exit 1
|
||||
}
|
||||
grep -F 'fruix_shepherd_enable="YES"' "$rc_conf_image" >/dev/null || {
|
||||
echo "rc.conf does not enable fruix_shepherd" >&2
|
||||
exit 1
|
||||
}
|
||||
grep -F '/var/lib/fruix/ready' "$shepherd_config_image" >/dev/null || {
|
||||
echo "shepherd configuration does not contain the ready marker" >&2
|
||||
exit 1
|
||||
}
|
||||
store_item_count=$(find "$mnt_root/frx/store" -mindepth 1 -maxdepth 1 | wc -l | awk '{print $1}')
|
||||
image_size_bytes=$(stat -f %z "$disk_image_a")
|
||||
|
||||
cat >"$metadata_file" <<EOF
|
||||
workdir=$workdir
|
||||
phase7_rootfs_dir=$phase7_rootfs_dir
|
||||
phase7_rootfs_log=$phase7_rootfs_log
|
||||
phase7_rootfs_metadata=$phase7_rootfs_metadata
|
||||
image_rootfs=$image_rootfs
|
||||
closure_path=$closure_path
|
||||
closure_base=$closure_base
|
||||
esp_stage=$esp_stage
|
||||
esp_image_a=$esp_image_a
|
||||
esp_image_b=$esp_image_b
|
||||
root_image_a=$root_image_a
|
||||
root_image_b=$root_image_b
|
||||
disk_image_a=$disk_image_a
|
||||
disk_image_b=$disk_image_b
|
||||
raw_sha256=$raw_sha256_a
|
||||
image_size_bytes=$image_size_bytes
|
||||
gpart_log=$gpart_log
|
||||
esp_fstype=$esp_fstype
|
||||
root_fstype=$root_fstype
|
||||
run_current_system_target=$run_current_system_target
|
||||
activate_target=$activate_target
|
||||
bin_target=$bin_target
|
||||
boot_loader_target=$boot_loader_target
|
||||
boot_loader_conf_target=$boot_loader_conf_target
|
||||
rc_conf_target=$rc_conf_target
|
||||
rc_script_target=$rc_script_target
|
||||
store_item_count=$store_item_count
|
||||
boot_mode=uefi
|
||||
image_format=raw
|
||||
partition_scheme=gpt
|
||||
root_partition_label=fruix-root
|
||||
efi_partition_label=efiboot
|
||||
serial_console=comconsole
|
||||
EOF
|
||||
|
||||
if [ -n "$metadata_target" ]; then
|
||||
mkdir -p "$(dirname "$metadata_target")"
|
||||
cp "$metadata_file" "$metadata_target"
|
||||
fi
|
||||
|
||||
printf 'PASS phase8-bhyve-image\n'
|
||||
printf 'Work directory: %s\n' "$workdir"
|
||||
printf 'Metadata file: %s\n' "$metadata_file"
|
||||
if [ -n "$metadata_target" ]; then
|
||||
printf 'Copied metadata to: %s\n' "$metadata_target"
|
||||
fi
|
||||
printf '%s\n' '--- metadata ---'
|
||||
cat "$metadata_file"
|
||||
Reference in New Issue
Block a user