diff --git a/docs/PROGRESS.md b/docs/PROGRESS.md index 7d85c6e..da60d8b 100644 --- a/docs/PROGRESS.md +++ b/docs/PROGRESS.md @@ -3538,3 +3538,89 @@ Current assessment: - Phase 15.1 is complete - Fruix now models the FreeBSD base as an explicit declarative system input instead of leaving it implicit in the builder host alone - the next step is to use that new declaration to prove side-by-side base versions and rollback-friendly rebuild/redeploy behavior + +## 2026-04-03 — Phase 15.2: validated side-by-side base versions and rollback-friendly redeploy + +Completed work: + +- wrote the Phase 15.2 report: + - `docs/reports/phase15-base-upgrades-freebsd.md` +- added validation harnesses: + - `tests/system/run-phase15-base-coexistence.sh` + - `tests/system/run-phase15-base-rollback-qemu.sh` + - `tests/system/run-phase15-base-rollback-xcpng.sh` +- used two explicit declarative base identities against the current validated native Phase 14 package split: + - current base: + - `name=stable-default` + - `version-label=15.0-STABLE` + - `release=15.0-STABLE` + - `branch=stable/15` + - candidate base: + - `name=stable-canary` + - `version-label=15.0-STABLE-p1` + - `release=15.0-STABLE` + - `branch=stable/15` +- both declarations still use the same local `/usr/src`, but now produce distinct declared base/store/deployment identities + +Validation: + +- side-by-side base-coexistence harness passes: + - `tests/system/run-phase15-base-coexistence.sh` + - workdir: `/tmp/phase15-2-coexist-1775202833` + - result: `PASS phase15-base-coexistence` + - confirmed: + - `current_closure=/frx/store/9f57ecc6481e271811ceb53ac21a3b2aef4ef329f82b7d4788622315db1f0e43-fruix-system-fruix-freebsd` + - `candidate_closure=/frx/store/dc40b1b7a76084e140d0457f3b7f6c5d4acc185f0d6cee0b161c9775d5fb3bec-fruix-system-fruix-freebsd` + - `current_base_version_label=15.0-STABLE` + - `candidate_base_version_label=15.0-STABLE-p1` + - `side_by_side_base_versions=ok` + - `rollback_rebuild_path=ok` + - this also confirmed that: + - both closures exist side by side in `/frx/store` + - rebuilding the current declaration returns the exact original current closure path + - current native base stores remain separate from candidate native base stores +- local QEMU rollback harness passes: + - `tests/system/run-phase15-base-rollback-qemu.sh` + - workdir: `/tmp/phase15-2-qemu2-1775204321` + - result: `PASS phase15-base-rollback-qemu` + - validation sequence: + 1. boot current base + 2. boot candidate base + 3. boot current base again + - confirmed: + - `current_first_closure=/frx/store/9f57ecc6481e271811ceb53ac21a3b2aef4ef329f82b7d4788622315db1f0e43-fruix-system-fruix-freebsd` + - `candidate_closure=/frx/store/dc40b1b7a76084e140d0457f3b7f6c5d4acc185f0d6cee0b161c9775d5fb3bec-fruix-system-fruix-freebsd` + - `rollback_closure=/frx/store/9f57ecc6481e271811ceb53ac21a3b2aef4ef329f82b7d4788622315db1f0e43-fruix-system-fruix-freebsd` + - `current_base_version_label=15.0-STABLE` + - `candidate_base_version_label=15.0-STABLE-p1` + - `rollback_base_version_label=15.0-STABLE` + - `base_rollforward_and_rollback=ok` +- real XCP-ng rollback harness passes: + - `tests/system/run-phase15-base-rollback-xcpng.sh` + - workdir: `/tmp/phase15-2-xcpng-1775204839` + - result: `PASS phase15-base-rollback-xcpng` + - validation sequence: + 1. boot candidate base on the approved VM/VDI + 2. boot current base again on the same approved VM/VDI + - confirmed: + - `candidate_closure=/frx/store/dc40b1b7a76084e140d0457f3b7f6c5d4acc185f0d6cee0b161c9775d5fb3bec-fruix-system-fruix-freebsd` + - `rollback_closure=/frx/store/9f57ecc6481e271811ceb53ac21a3b2aef4ef329f82b7d4788622315db1f0e43-fruix-system-fruix-freebsd` + - `candidate_base_version_label=15.0-STABLE-p1` + - `rollback_base_version_label=15.0-STABLE` + - `vm_id=90490f2e-e8fc-4b7a-388e-5c26f0157289` + - `vdi_id=0f1f90d3-48ca-4fa2-91d8-fc6339b95743` + - `base_rollforward_and_rollback=ok` + - guest invariants stayed intact for both boots: + - `shepherd_pid=1` + - `sshd_status=running` + - `compat_prefix_shims=absent` + - `guile_module_smoke=ok` + +Current assessment: + +- Phase 15.2 is complete +- Fruix now supports a real upgrade-style FreeBSD base workflow at the store/deployment level: + - explicit current vs. candidate base declarations + - side-by-side native base outputs in `/frx/store` + - rollback to the earlier closure without mutating it in place +- the remaining Phase 15 work is to document the evidence-based decision on whether self-hosted base builds should be the next step, or whether host-built native base artifacts should remain the near-term path while reproducibility/source acquisition improve diff --git a/docs/reports/phase15-base-upgrades-freebsd.md b/docs/reports/phase15-base-upgrades-freebsd.md new file mode 100644 index 0000000..bcf706e --- /dev/null +++ b/docs/reports/phase15-base-upgrades-freebsd.md @@ -0,0 +1,179 @@ +# Phase 15.2: side-by-side native base versions and rollback-friendly redeploy + +Date: 2026-04-03 + +## Goal + +Phase 15.2 demonstrated that Fruix can keep at least two distinct declarative FreeBSD base builds in `/frx/store` at the same time and switch between them through the normal system rebuild/image/boot flow. + +For this first upgrade-story validation, both declared bases still point at the same local `/usr/src`, but they carry distinct declarative version labels: + +- current base: `15.0-STABLE` +- candidate base: `15.0-STABLE-p1` + +That is enough to prove the Fruix properties needed here: + +- distinct content-addressed outputs +- side-by-side coexistence +- no in-place mutation of the older base closure +- rollback to the earlier closure using the normal deployment path + +## New files + +Added: + +- `tests/system/run-phase15-base-coexistence.sh` +- `tests/system/run-phase15-base-rollback-qemu.sh` +- `tests/system/run-phase15-base-rollback-xcpng.sh` + +## Validation model + +### Current base declaration + +```scheme +(freebsd-base + #:name "stable-default" + #:version-label "15.0-STABLE" + #:release "15.0-STABLE" + #:branch "stable/15" + ...) +``` + +### Candidate base declaration + +```scheme +(freebsd-base + #:name "stable-canary" + #:version-label "15.0-STABLE-p1" + #:release "15.0-STABLE" + #:branch "stable/15" + ...) +``` + +Both declarations use the same validated native Phase 14 package composition: + +- kernel from `freebsd-native-kernel-for` +- bootloader from `freebsd-native-bootloader-for` +- runtime from `freebsd-native-system-packages-for` +- `shepherd-pid1` + +## Side-by-side build validation + +Passing run: + +- `PASS phase15-base-coexistence` +- workdir: `/tmp/phase15-2-coexist-1775202833` + +Confirmed: + +```text +current_closure=/frx/store/9f57ecc6481e271811ceb53ac21a3b2aef4ef329f82b7d4788622315db1f0e43-fruix-system-fruix-freebsd +candidate_closure=/frx/store/dc40b1b7a76084e140d0457f3b7f6c5d4acc185f0d6cee0b161c9775d5fb3bec-fruix-system-fruix-freebsd +current_base_version_label=15.0-STABLE +candidate_base_version_label=15.0-STABLE-p1 +side_by_side_base_versions=ok +rollback_rebuild_path=ok +``` + +Current native base stores: + +```text +/frx/store/d9785661ea4829d51fbf545c2607a5691af2cc33c8ef3cd44de7ad5626685098-freebsd-native-kernel-15.0-STABLE +/frx/store/b448c822302ccdfb2f06da811fb224a044c51a9935bbfcd77a71a25d02f228f1-freebsd-native-bootloader-15.0-STABLE +/frx/store/ac3ba684020e70d3c76e593fd687cef8ab5e148958baabb477b7ef3d2647c5cd-freebsd-native-runtime-15.0-STABLE +``` + +Candidate native base stores: + +```text +/frx/store/05bee8ffbe8c43242ffd97da4dc305f2921612a660cbcb48c3a3536bfac07079-freebsd-native-kernel-15.0-STABLE-p1 +/frx/store/8955f1bfe89321e6e1e628c59376f2092547523f48a773974cc259963adac184-freebsd-native-bootloader-15.0-STABLE-p1 +/frx/store/30314f17fd8ff4a1a3eff31c8c5048f15f67c46d1132d5b8c45fd9768742665e-freebsd-native-runtime-15.0-STABLE-p1 +``` + +Important result: + +- the older current closure stayed in `/frx/store` +- the candidate closure appeared beside it +- rebuilding the current declaration returned the exact original current closure path again + +## Local QEMU rollback validation + +Passing run: + +- `PASS phase15-base-rollback-qemu` +- workdir: `/tmp/phase15-2-qemu2-1775204321` + +Validation sequence: + +1. boot current base +2. boot candidate base +3. boot current base again + +Confirmed: + +```text +current_first_closure=/frx/store/9f57ecc6481e271811ceb53ac21a3b2aef4ef329f82b7d4788622315db1f0e43-fruix-system-fruix-freebsd +candidate_closure=/frx/store/dc40b1b7a76084e140d0457f3b7f6c5d4acc185f0d6cee0b161c9775d5fb3bec-fruix-system-fruix-freebsd +rollback_closure=/frx/store/9f57ecc6481e271811ceb53ac21a3b2aef4ef329f82b7d4788622315db1f0e43-fruix-system-fruix-freebsd +current_base_version_label=15.0-STABLE +candidate_base_version_label=15.0-STABLE-p1 +rollback_base_version_label=15.0-STABLE +base_rollforward_and_rollback=ok +``` + +This showed that the booted system could move forward to the candidate base and then return to the earlier closure without mutating it in place. + +## Real XCP-ng rollback validation + +Passing run: + +- `PASS phase15-base-rollback-xcpng` +- workdir: `/tmp/phase15-2-xcpng-1775204839` + +Validation sequence: + +1. boot candidate base on the approved VM/VDI +2. boot current base again on the same approved VM/VDI + +Confirmed: + +```text +candidate_closure=/frx/store/dc40b1b7a76084e140d0457f3b7f6c5d4acc185f0d6cee0b161c9775d5fb3bec-fruix-system-fruix-freebsd +rollback_closure=/frx/store/9f57ecc6481e271811ceb53ac21a3b2aef4ef329f82b7d4788622315db1f0e43-fruix-system-fruix-freebsd +candidate_base_version_label=15.0-STABLE-p1 +rollback_base_version_label=15.0-STABLE +vm_id=90490f2e-e8fc-4b7a-388e-5c26f0157289 +vdi_id=0f1f90d3-48ca-4fa2-91d8-fc6339b95743 +base_rollforward_and_rollback=ok +``` + +Both boots also preserved the already-hardened guest properties: + +- `shepherd_pid=1` +- `sshd_status=running` +- `compat_prefix_shims=absent` +- `guile_module_smoke=ok` + +## Result + +Phase 15.2 is complete. + +Fruix now has a real declarative rebuild/redeploy/rollback story for the FreeBSD base at the store/model layer: + +- two declared base versions can coexist side by side in `/frx/store` +- the candidate deployment does not overwrite the current one in place +- rebuilding the earlier declaration returns to the earlier closure path +- the same story works both locally under QEMU and on the approved XCP-ng VM/VDI path + +## Scope note + +This first upgrade-story validation still uses the same local `/usr/src` as the underlying source tree for both declarations. What changed is the declared base identity and therefore the store/model/deployment identity. + +That is sufficient for this phase because the requirement was to establish the upgrade semantics: + +- explicit base declaration +- side-by-side outputs +- rollback-friendly closures + +The next improvement beyond Phase 15 would be to make acquiring or selecting distinct source trees/releases more reproducible and less tied to a single host checkout. diff --git a/tests/system/run-phase15-base-coexistence.sh b/tests/system/run-phase15-base-coexistence.sh new file mode 100755 index 0000000..204d39d --- /dev/null +++ b/tests/system/run-phase15-base-coexistence.sh @@ -0,0 +1,192 @@ +#!/bin/sh +set -eu + +project_root=${PROJECT_ROOT:-$(pwd)} +script_dir=$(CDPATH= cd -- "$(dirname "$0")" && pwd) +fruix_cmd=$project_root/bin/fruix +os_template=${OS_TEMPLATE:-$script_dir/phase15-declarative-base-pid1-operating-system.scm.in} +system_name=${SYSTEM_NAME:-phase15-operating-system} +store_dir=${STORE_DIR:-/frx/store} +current_base_name=${CURRENT_BASE_NAME:-stable-default} +current_base_version=${CURRENT_BASE_VERSION:-15.0-STABLE} +current_base_release=${CURRENT_BASE_RELEASE:-15.0-STABLE} +current_base_branch=${CURRENT_BASE_BRANCH:-stable/15} +candidate_base_name=${CANDIDATE_BASE_NAME:-stable-canary} +candidate_base_version=${CANDIDATE_BASE_VERSION:-15.0-STABLE-p1} +candidate_base_release=${CANDIDATE_BASE_RELEASE:-15.0-STABLE} +candidate_base_branch=${CANDIDATE_BASE_BRANCH:-stable/15} +metadata_target=${METADATA_OUT:-} +root_authorized_key_file=${ROOT_AUTHORIZED_KEY_FILE:-$HOME/.ssh/id_ed25519.pub} + +[ -x "$fruix_cmd" ] || { + echo "fruix command is not executable: $fruix_cmd" >&2 + exit 1 +} +[ -f "$os_template" ] || { + echo "missing operating-system template: $os_template" >&2 + exit 1 +} +[ -f "$root_authorized_key_file" ] || { + echo "missing root authorized key file: $root_authorized_key_file" >&2 + exit 1 +} + +cleanup=0 +if [ -n "${WORKDIR:-}" ]; then + workdir=$WORKDIR + mkdir -p "$workdir" +else + workdir=$(mktemp -d /tmp/fruix-phase15-base-coexistence.XXXXXX) + cleanup=1 +fi +if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then + cleanup=0 +fi + +cleanup_workdir() { + if [ "$cleanup" -eq 1 ]; then + rm -rf "$workdir" 2>/dev/null || sudo rm -rf "$workdir" + fi +} +trap cleanup_workdir EXIT INT TERM + +render_os() { + output=$1 + base_name=$2 + base_version=$3 + base_release=$4 + base_branch=$5 + root_authorized_key=$(tr -d '\n' < "$root_authorized_key_file") + sed \ + -e "s|__BASE_NAME__|$base_name|g" \ + -e "s|__BASE_VERSION_LABEL__|$base_version|g" \ + -e "s|__BASE_RELEASE__|$base_release|g" \ + -e "s|__BASE_BRANCH__|$base_branch|g" \ + -e "s|__ROOT_AUTHORIZED_KEY__|$root_authorized_key|g" \ + "$os_template" > "$output" +} + +action_env() { + sudo env \ + HOME="$HOME" \ + GUILE_AUTO_COMPILE=0 \ + FRUIX_FREEBSD_BUILD_JOBS="${FRUIX_FREEBSD_BUILD_JOBS:-8}" \ + GUIX_SOURCE_DIR="${GUIX_SOURCE_DIR:-$HOME/repos/guix}" \ + GUILE_BIN="${GUILE_BIN:-/tmp/guile-freebsd-validate-install/bin/guile}" \ + GUILE_EXTRA_PREFIX="${GUILE_EXTRA_PREFIX:-/tmp/guile-gnutls-freebsd-validate-install}" \ + SHEPHERD_PREFIX="${SHEPHERD_PREFIX:-/tmp/shepherd-freebsd-validate-install}" \ + "$@" +} + +build_os() { + os_file=$1 + out_file=$2 + action_env "$fruix_cmd" system build "$os_file" --system "$system_name" --store "$store_dir" >"$out_file" +} + +current_os=$workdir/current-operating-system.scm +candidate_os=$workdir/candidate-operating-system.scm +current_out_a=$workdir/current-build-a.txt +current_out_b=$workdir/current-build-b.txt +candidate_out=$workdir/candidate-build.txt +metadata_file=$workdir/phase15-base-coexistence-metadata.txt + +render_os "$current_os" "$current_base_name" "$current_base_version" "$current_base_release" "$current_base_branch" +render_os "$candidate_os" "$candidate_base_name" "$candidate_base_version" "$candidate_base_release" "$candidate_base_branch" + +build_os "$current_os" "$current_out_a" +build_os "$candidate_os" "$candidate_out" +build_os "$current_os" "$current_out_b" + +current_closure=$(sed -n 's/^closure_path=//p' "$current_out_a") +current_closure_rebuild=$(sed -n 's/^closure_path=//p' "$current_out_b") +candidate_closure=$(sed -n 's/^closure_path=//p' "$candidate_out") +current_native_stores=$(sed -n 's/^native_base_stores=//p' "$current_out_a") +candidate_native_stores=$(sed -n 's/^native_base_stores=//p' "$candidate_out") +current_host_count=$(sed -n 's/^host_base_store_count=//p' "$current_out_a") +candidate_host_count=$(sed -n 's/^host_base_store_count=//p' "$candidate_out") +current_base_version_out=$(sed -n 's/^freebsd_base_version_label=//p' "$current_out_a") +candidate_base_version_out=$(sed -n 's/^freebsd_base_version_label=//p' "$candidate_out") +current_base_file=$(sed -n 's/^freebsd_base_file=//p' "$current_out_a") +candidate_base_file=$(sed -n 's/^freebsd_base_file=//p' "$candidate_out") + +[ -n "$current_closure" ] || { echo "missing current closure" >&2; exit 1; } +[ -n "$candidate_closure" ] || { echo "missing candidate closure" >&2; exit 1; } +[ "$current_closure" = "$current_closure_rebuild" ] || { + echo "current closure path was not reproducible: $current_closure != $current_closure_rebuild" >&2 + exit 1 +} +[ "$current_closure" != "$candidate_closure" ] || { + echo "current and candidate closures unexpectedly match" >&2 + exit 1 +} +[ "$current_host_count" = 0 ] || { echo "current build has host base stores" >&2; exit 1; } +[ "$candidate_host_count" = 0 ] || { echo "candidate build has host base stores" >&2; exit 1; } +[ "$current_base_version_out" = "$current_base_version" ] || { echo "unexpected current base version label" >&2; exit 1; } +[ "$candidate_base_version_out" = "$candidate_base_version" ] || { echo "unexpected candidate base version label" >&2; exit 1; } +[ -f "$current_base_file" ] || { echo "missing current base file" >&2; exit 1; } +[ -f "$candidate_base_file" ] || { echo "missing candidate base file" >&2; exit 1; } + +for path in "$current_closure" "$candidate_closure"; do + [ -d "$path" ] || { + echo "expected closure directory missing: $path" >&2 + exit 1 + } +done + +printf '%s\n' "$current_native_stores" | tr ',' '\n' | grep "freebsd-native-kernel-$current_base_version$" >/dev/null || { + echo "current native stores do not contain the expected kernel version label" >&2 + exit 1 +} +printf '%s\n' "$candidate_native_stores" | tr ',' '\n' | grep "freebsd-native-kernel-$candidate_base_version$" >/dev/null || { + echo "candidate native stores do not contain the expected kernel version label" >&2 + exit 1 +} +[ "$current_native_stores" != "$candidate_native_stores" ] || { + echo "current and candidate native store sets unexpectedly match" >&2 + exit 1 +} + +grep -F "(version-label . \"$current_base_version\")" "$current_base_file" >/dev/null || { + echo "current base file missing version label" >&2 + exit 1 +} +grep -F "(version-label . \"$candidate_base_version\")" "$candidate_base_file" >/dev/null || { + echo "candidate base file missing version label" >&2 + exit 1 +} + +current_closure_base=$(basename "$current_closure") +candidate_closure_base=$(basename "$candidate_closure") +cat >"$metadata_file" </dev/null || sudo rm -rf "$workdir" + fi +} +trap cleanup_workdir EXIT INT TERM + +render_template() { + output=$1 + base_name=$2 + base_version=$3 + base_release=$4 + base_branch=$5 + sed \ + -e "s|__BASE_NAME__|$base_name|g" \ + -e "s|__BASE_VERSION_LABEL__|$base_version|g" \ + -e "s|__BASE_RELEASE__|$base_release|g" \ + -e "s|__BASE_BRANCH__|$base_branch|g" \ + "$os_template" > "$output" +} + +render_template "$current_template" "$current_base_name" "$current_base_version" "$current_base_release" "$current_base_branch" +render_template "$candidate_template" "$candidate_base_name" "$candidate_base_version" "$candidate_base_release" "$candidate_base_branch" + +run_boot() { + name=$1 + template=$2 + metadata_out=$3 + KEEP_WORKDIR=1 WORKDIR="$workdir/$name" METADATA_OUT="$metadata_out" \ + OS_TEMPLATE="$template" SYSTEM_NAME="$system_name" DISK_CAPACITY="$disk_capacity" ROOT_SIZE="$root_size" \ + "$repo_root/tests/system/run-phase11-shepherd-pid1-qemu.sh" >/dev/null +} + +current_first_metadata=$workdir/current-first-metadata.txt +candidate_metadata=$workdir/candidate-metadata.txt +rollback_metadata=$workdir/rollback-metadata.txt + +run_boot current-first "$current_template" "$current_first_metadata" +run_boot candidate "$candidate_template" "$candidate_metadata" +run_boot rollback "$current_template" "$rollback_metadata" + +current_first_phase8=$(sed -n 's/^phase8_metadata=//p' "$current_first_metadata") +candidate_phase8=$(sed -n 's/^phase8_metadata=//p' "$candidate_metadata") +rollback_phase8=$(sed -n 's/^phase8_metadata=//p' "$rollback_metadata") + +current_first_closure=$(sed -n 's/^closure_path=//p' "$current_first_metadata") +candidate_closure=$(sed -n 's/^closure_path=//p' "$candidate_metadata") +rollback_closure=$(sed -n 's/^closure_path=//p' "$rollback_metadata") + +current_first_version=$(sed -n 's/^freebsd_base_version_label=//p' "$current_first_phase8") +candidate_version=$(sed -n 's/^freebsd_base_version_label=//p' "$candidate_phase8") +rollback_version=$(sed -n 's/^freebsd_base_version_label=//p' "$rollback_phase8") + +current_first_shepherd_pid=$(sed -n 's/^shepherd_pid=//p' "$current_first_metadata") +candidate_shepherd_pid=$(sed -n 's/^shepherd_pid=//p' "$candidate_metadata") +rollback_shepherd_pid=$(sed -n 's/^shepherd_pid=//p' "$rollback_metadata") +current_first_sshd=$(sed -n 's/^sshd_status=//p' "$current_first_metadata") +candidate_sshd=$(sed -n 's/^sshd_status=//p' "$candidate_metadata") +rollback_sshd=$(sed -n 's/^sshd_status=//p' "$rollback_metadata") + +[ "$current_first_version" = "$current_base_version" ] || { echo "unexpected current version label" >&2; exit 1; } +[ "$candidate_version" = "$candidate_base_version" ] || { echo "unexpected candidate version label" >&2; exit 1; } +[ "$rollback_version" = "$current_base_version" ] || { echo "unexpected rollback version label" >&2; exit 1; } +[ "$current_first_closure" != "$candidate_closure" ] || { echo "candidate closure matches current closure" >&2; exit 1; } +[ "$current_first_closure" = "$rollback_closure" ] || { echo "rollback closure did not return to the original current closure" >&2; exit 1; } +for value in "$current_first_shepherd_pid" "$candidate_shepherd_pid" "$rollback_shepherd_pid"; do + [ "$value" = 1 ] || { echo "shepherd was not PID 1" >&2; exit 1; } +done +for value in "$current_first_sshd" "$candidate_sshd" "$rollback_sshd"; do + [ "$value" = running ] || { echo "sshd is not running in one of the rollback boots" >&2; exit 1; } +done + +current_first_native=$(sed -n 's/^native_base_stores=//p' "$current_first_phase8") +candidate_native=$(sed -n 's/^native_base_stores=//p' "$candidate_phase8") +rollback_native=$(sed -n 's/^native_base_stores=//p' "$rollback_phase8") + +printf '%s\n' "$current_first_native" | tr ',' '\n' | grep "freebsd-native-kernel-$current_base_version$" >/dev/null || { + echo "current native store set missing expected kernel version" >&2 + exit 1 +} +printf '%s\n' "$candidate_native" | tr ',' '\n' | grep "freebsd-native-kernel-$candidate_base_version$" >/dev/null || { + echo "candidate native store set missing expected kernel version" >&2 + exit 1 +} +[ "$current_first_native" = "$rollback_native" ] || { + echo "rollback native store set did not return to the original current set" >&2 + exit 1 +} + +cat >"$metadata_file" < "$output" +} + +render_template "$current_template" "$current_base_name" "$current_base_version" "$current_base_release" "$current_base_branch" +render_template "$candidate_template" "$candidate_base_name" "$candidate_base_version" "$candidate_base_release" "$candidate_base_branch" + +run_boot() { + name=$1 + template=$2 + metadata_out=$3 + KEEP_WORKDIR=1 WORKDIR="$workdir/$name" METADATA_OUT="$metadata_out" \ + OS_TEMPLATE="$template" SYSTEM_NAME="$system_name" ROOT_SIZE="$root_size" \ + "$repo_root/tests/system/run-phase11-shepherd-pid1-xcpng.sh" >/dev/null +} + +candidate_metadata=$workdir/candidate-metadata.txt +rollback_metadata=$workdir/rollback-metadata.txt + +run_boot candidate "$candidate_template" "$candidate_metadata" +run_boot rollback "$current_template" "$rollback_metadata" + +candidate_phase8=$(sed -n 's/^phase8_metadata=//p' "$candidate_metadata") +rollback_phase8=$(sed -n 's/^phase8_metadata=//p' "$rollback_metadata") + +candidate_closure=$(sed -n 's/^closure_path=//p' "$candidate_metadata") +rollback_closure=$(sed -n 's/^closure_path=//p' "$rollback_metadata") +candidate_guest_ip=$(sed -n 's/^guest_ip=//p' "$candidate_metadata") +rollback_guest_ip=$(sed -n 's/^guest_ip=//p' "$rollback_metadata") +vm_id=$(sed -n 's/^vm_id=//p' "$rollback_metadata") +vdi_id=$(sed -n 's/^vdi_id=//p' "$rollback_metadata") +candidate_version=$(sed -n 's/^freebsd_base_version_label=//p' "$candidate_phase8") +rollback_version=$(sed -n 's/^freebsd_base_version_label=//p' "$rollback_phase8") +candidate_shepherd_pid=$(sed -n 's/^shepherd_pid=//p' "$candidate_metadata") +rollback_shepherd_pid=$(sed -n 's/^shepherd_pid=//p' "$rollback_metadata") +candidate_sshd=$(sed -n 's/^sshd_status=//p' "$candidate_metadata") +rollback_sshd=$(sed -n 's/^sshd_status=//p' "$rollback_metadata") +candidate_compat=$(sed -n 's/^compat_prefix_shims=//p' "$candidate_metadata") +rollback_compat=$(sed -n 's/^compat_prefix_shims=//p' "$rollback_metadata") +candidate_smoke=$(sed -n 's/^guile_module_smoke=//p' "$candidate_metadata") +rollback_smoke=$(sed -n 's/^guile_module_smoke=//p' "$rollback_metadata") + +[ "$candidate_version" = "$candidate_base_version" ] || { echo "unexpected candidate version label" >&2; exit 1; } +[ "$rollback_version" = "$current_base_version" ] || { echo "unexpected rollback version label" >&2; exit 1; } +[ "$candidate_closure" != "$rollback_closure" ] || { echo "candidate and rollback closures unexpectedly match" >&2; exit 1; } +for value in "$candidate_shepherd_pid" "$rollback_shepherd_pid"; do + [ "$value" = 1 ] || { echo "shepherd was not PID 1" >&2; exit 1; } +done +for value in "$candidate_sshd" "$rollback_sshd"; do + [ "$value" = running ] || { echo "sshd is not running during XCP-ng rollback validation" >&2; exit 1; } +done +for value in "$candidate_compat" "$rollback_compat"; do + [ "$value" = absent ] || { echo "compatibility prefix shims reappeared" >&2; exit 1; } +done +for value in "$candidate_smoke" "$rollback_smoke"; do + [ "$value" = ok ] || { echo "guest Guile module smoke failed" >&2; exit 1; } +done + +candidate_native=$(sed -n 's/^native_base_stores=//p' "$candidate_phase8") +rollback_native=$(sed -n 's/^native_base_stores=//p' "$rollback_phase8") +printf '%s\n' "$candidate_native" | tr ',' '\n' | grep "freebsd-native-kernel-$candidate_base_version$" >/dev/null || { + echo "candidate native store set missing expected kernel version" >&2 + exit 1 +} +printf '%s\n' "$rollback_native" | tr ',' '\n' | grep "freebsd-native-kernel-$current_base_version$" >/dev/null || { + echo "rollback native store set missing expected kernel version" >&2 + exit 1 +} + +cat >"$metadata_file" <