Prototype FreeBSD store management

docs/PROGRESS.md

@@ -972,3 +972,110 @@ Next recommended step:
 2. if possible, use outputs or dependency relationships realistic enough to model how a future FreeBSD Guix daemon would retain referenced store items
 3. continue carrying the separate Guix checkout runtime blocker:
   - investigate the `leave-on-EPIPE` failure in `./pre-inst-env guix --version`
+
+## 2026-04-01 — Phase 2.3 completed: `/frx/store` prototype validated on FreeBSD
+
+Completed work:
+
+- added a runnable `/frx/store` prototype harness:
+  - `tests/store/run-freebsd-store-prototype.sh`
+- wrote the Phase 2.3 report:
+  - `docs/reports/phase2-freebsd-store-prototype.md`
+- created and exercised the operator-requested `/frx` layout on-host:
+  - `/frx/store`
+  - `/frx/var`
+  - `/frx/etc`
+  - `/frx/var/fruix/gcroots`
+- created a store group for the prototype path:
+  - `fruixbuild`
+- ran the store prototype successfully and captured metadata under:
+  - `/tmp/freebsd-store-prototype-metadata.txt`
+
+Important findings:
+
+- the current host now has a working `/frx/store` prototype owned as:
+  - `root:fruixbuild`
+  with mode:
+  - `drwxrwxr-t`
+- the prototype successfully created content-addressed demo store items under `/frx/store` using hash-based names
+- the demo item set included:
+  - rooted greeting data
+  - a rooted app referencing that data through an absolute store path
+  - an unrooted orphan item intended for collection
+- an unprivileged user (`nobody`) could:
+  - read store data
+  - execute the demo app from the store
+- the same unprivileged user could not:
+  - create files directly in `/frx/store`
+  and the observed failure was:
+  - `Permission denied`
+- the prototype GC logic followed rooted references successfully:
+  - with a GC root present, the app and its referenced data survived while the orphan item was collected
+  - after removing the GC root, the remaining demo items were collected as well
+- the demo store returned to an empty state after the second GC pass, so the host is left with the `/frx` skeleton but without lingering prototype payloads
+
+Current assessment:
+
+- Phase 2.3 is now satisfied on the current FreeBSD prototype track
+- the core store assumptions needed for a FreeBSD Guix-daemon design have practical validation now:
+  - `/frx/store` path viability
+  - root-controlled mutation
+  - unprivileged read access
+  - immutable absolute store references
+  - root-managed GC roots and mark/sweep retention behavior
+- remaining gaps are now above this architectural layer rather than below it:
+  - real derivation registration
+  - SQLite-backed store metadata
+  - daemon RPC integration
+  - actual package lowering/build submission using these mechanisms
+
+## 2026-04-01 — Phase 2 completed on the current FreeBSD prototype track
+
+Phase 2 is now considered complete for the active FreeBSD amd64 prototype path.
+
+Why this milestone is satisfied:
+
+- **Phase 2.1** success criteria were met:
+  - a detailed jail-first build-isolation design was produced
+  - a runnable prototype successfully executed a build command in a restricted FreeBSD jail
+- **Phase 2.2** success criteria were met:
+  - a concrete C privilege-dropping implementation was added
+  - build-user credential drop, inability to regain root, and concurrent cross-build isolation were demonstrated
+- **Phase 2.3** success criteria were met on the prototype track:
+  - a working `/frx/store` equivalent was established
+  - content-addressed demo store items were created and consumed
+  - unprivileged read vs. privileged write behavior was validated
+  - garbage-collection behavior over rooted references was demonstrated
+
+Important scope note:
+
+- this completes the **core daemon architecture adaptation** milestone, not a full Guix-daemon port
+- the separate real-checkout blocker from Phase 1 remains relevant for later integration work:
+  - `./pre-inst-env guix --version` still fails with `Wrong type to apply: #<syntax-transformer leave-on-EPIPE>`
+- however, that runtime issue no longer blocks the specific Phase 2 architectural deliverables because the jail, privilege, and store assumptions have now been validated independently on FreeBSD
+
+Recent commits:
+
+- `e380e88` — `Add FreeBSD Guile verification harness`
+- `cd721b1` — `Update progress after Guile verification`
+- `27916cb` — `Diagnose Guile subprocess crash on FreeBSD`
+- `02f7a7f` — `Validate local Guile fix on FreeBSD`
+- `4aebea4` — `Add native GNU Hello FreeBSD build harness`
+- `c944cdb` — `Validate Guix builder phases on FreeBSD`
+- `0a2e48e` — `Validate GNU which builder phases on FreeBSD`
+- `245a47d` — `Document gaps to real Guix FreeBSD builds`
+- `d62e9b0` — `Investigate Guix derivation generation on FreeBSD`
+- `c0a85ed` — `Build local Guile-GnuTLS on FreeBSD`
+- `15b9037` — `Build local Guile-Git on FreeBSD`
+- `47d31e8` — `Build local Guile-JSON on FreeBSD`
+- `d82195b` — `Advance Guix checkout on FreeBSD`
+- `9bf3d30` — `Document FreeBSD syscall mapping`
+- `7621798` — `Prototype FreeBSD jail build isolation`
+- `d65b2af` — `Prototype FreeBSD build user isolation`
+
+Next recommended step:
+
+1. begin Phase 3.1 by adapting Guix build-system expectations to the now-validated jail/privilege/store model on FreeBSD
+2. carry forward the concrete real-checkout runtime blocker for later integration work:
+  - investigate the `leave-on-EPIPE` failure in `./pre-inst-env guix --version`
+3. continue using `/frx/store` rather than `/gnu/store` for FreeBSD store experiments

docs/reports/phase2-freebsd-store-prototype.md

@@ -0,0 +1,189 @@
+# Phase 2.3: `/frx/store` prototype for FreeBSD
+
+Date: 2026-04-01
+
+## Summary
+
+This step establishes a working FreeBSD prototype for the Guix store concept under the operator-requested `/frx` prefix.
+
+Added file:
+
+- `tests/store/run-freebsd-store-prototype.sh`
+
+The prototype creates and validates:
+
+- `/frx/store`
+- `/frx/var`
+- `/frx/etc`
+- `/frx/var/fruix/gcroots`
+
+It then installs a few content-addressed demo store items, verifies that an unprivileged user can read and execute them but cannot write to the store, and exercises a simple reference-following garbage-collection pass.
+
+## Run command
+
+```sh
+METADATA_OUT=/tmp/freebsd-store-prototype-metadata.txt \
+./tests/store/run-freebsd-store-prototype.sh
+```
+
+## Store layout and permission model
+
+Observed store metadata:
+
+- store root: `/frx/store`
+- store state root: `/frx/var/fruix`
+- gc roots: `/frx/var/fruix/gcroots`
+- store group: `fruixbuild`
+- observed store root mode: `drwxrwxr-t`
+- observed store root ownership: `root:fruixbuild`
+
+This matches the intended high-level Guix model closely enough for the prototype track:
+
+- daemon/root can create and remove store items
+- ordinary users can traverse and read store items
+- ordinary users cannot write new store entries directly
+
+## Content-addressed demo items
+
+The prototype creates three demo store items using hashes derived from explicit manifests:
+
+1. **greeting data**
+   - path shape: `/frx/store/<hash>-demo-greeting-data`
+   - contains `share/greeting.txt`
+
+2. **hello app**
+   - path shape: `/frx/store/<hash>-demo-hello-app`
+   - contains `bin/demo-hello`
+   - references the greeting-data store path through an absolute store reference
+
+3. **orphan data**
+   - path shape: `/frx/store/<hash>-demo-orphan-data`
+   - intentionally left unrooted so GC should collect it
+
+Each store item also carries prototype metadata files:
+
+- `.fruix-demo`
+- `.references`
+
+The `.references` file is used by the prototype GC pass to retain transitive dependencies.
+
+## Readability and write protection checks
+
+The prototype validates store access as the unprivileged `nobody` user.
+
+Observed successful unprivileged reads/execution:
+
+```text
+hello-from-frx-store
+hello-from-frx-store
+```
+
+Those two lines come from:
+
+- directly reading the data file
+- executing the app script, which reads the referenced data store path
+
+Observed failed unprivileged write attempt:
+
+```text
+touch: /frx/store/should-not-write: Permission denied
+```
+
+This demonstrates the critical store property needed for later daemon work:
+
+- store contents are consumable by unprivileged users
+- store mutation remains daemon/root controlled
+
+## Garbage-collection prototype
+
+### Rooted GC pass
+
+The prototype creates a GC root symlink:
+
+- `/frx/var/fruix/gcroots/demo-root` -> `demo-hello-app`
+
+The GC algorithm then:
+
+1. walks GC roots
+2. marks the rooted item reachable
+3. follows `.references` transitively
+4. removes demo items not in the reachable set
+
+Observed store listing after the rooted GC pass:
+
+```text
+/frx/store/5aa68380f937120deaf9befe3390a563a3631672e2ab5d19b5498e2c288d7277-demo-hello-app
+/frx/store/d182f3489191dae9d4b1768e0e7afbfae0143b6d08750a058589437ccd5e0af3-demo-greeting-data
+```
+
+Result:
+
+- rooted app retained
+- referenced greeting data retained
+- orphan data removed
+
+### Unrooted GC pass
+
+The prototype then removes the GC root and reruns the same mark-and-sweep logic.
+
+Observed store listing after the second GC pass:
+
+```text
+<empty>
+```
+
+Result:
+
+- app removed
+- transitive dependency removed
+- demo store returned to an empty state
+
+## What this demonstrates for the FreeBSD port
+
+### 1. `/frx/store` works as a practical stand-in for `/gnu/store`
+
+The operator-requested `/frx` prefix is now exercised as a real on-host store root with:
+
+- correct directory creation
+- permissions resembling Guix store expectations
+- content-addressed path naming
+- shared read access
+- root-controlled mutation
+
+### 2. Store references can be modeled directly on FreeBSD
+
+The `demo-hello-app` item refers to the exact absolute path of its dependency under `/frx/store`, showing that the usual Guix model of immutable absolute store references maps cleanly onto FreeBSD path semantics.
+
+### 3. Garbage collection can be expressed with ordinary FreeBSD filesystem primitives
+
+No Linux-specific kernel mechanism was required for the prototype GC behavior.  A root/daemon process can manage roots, references, and deletion using normal filesystem traversal and metadata files.
+
+## Limitations versus real Guix
+
+This is still a prototype, not full Guix store integration.  It does **not** yet provide:
+
+- real derivation registration
+- SQLite-backed store metadata
+- daemon RPC interfaces
+- narinfo/substitute handling
+- signature verification
+- graft handling
+- real package database integration
+
+However, it does validate the central FreeBSD-side store assumptions needed before deeper integration:
+
+- `/frx/store` can exist with an appropriate permission model
+- absolute immutable store paths are viable
+- root-managed GC roots and dependency retention are viable
+- unprivileged consumers can read/execute store content without being able to mutate it
+
+## Conclusion
+
+Phase 2.3 is satisfied on the current prototype track:
+
+- a working `/frx/store` equivalent now exists on the host
+- content-addressed demo store items were created successfully
+- unprivileged readability and write protection were validated
+- reference-preserving garbage collection was demonstrated
+
+With Phase 2.1, 2.2, and 2.3 now all satisfied on the prototype track, the project has completed the core FreeBSD Guix-daemon architecture adaptation milestone needed before moving on to build-system adaptation work.

tests/store/run-freebsd-store-prototype.sh

@@ -0,0 +1,234 @@
+#!/bin/sh
+set -eu
+
+store_root=${STORE_ROOT:-/frx/store}
+state_root=${STATE_ROOT:-/frx/var}
+sysconf_root=${SYSCONF_ROOT:-/frx/etc}
+fruix_state_dir=$state_root/fruix
+fruix_gcroots_dir=$fruix_state_dir/gcroots
+store_group=${STORE_GROUP:-fruixbuild}
+workdir=${WORKDIR:-$(mktemp -d /tmp/fruix-store-prototype.XXXXXX)}
+cleanup_workdir=1
+metadata_file=$workdir/freebsd-store-prototype-metadata.txt
+access_read_out=$workdir/access-read.out
+access_write_err=$workdir/access-write.err
+first_gc_listing=$workdir/first-gc-listing.txt
+second_gc_listing=$workdir/second-gc-listing.txt
+store_root_stat=$workdir/store-root.stat
+gcroots_stat=$workdir/gcroots.stat
+
+if [ -n "${WORKDIR:-}" ]; then
+  cleanup_workdir=0
+fi
+if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
+  cleanup_workdir=0
+fi
+
+cleanup() {
+  set +e
+  if [ "$cleanup_workdir" -eq 1 ]; then
+    rm -rf "$workdir"
+  fi
+}
+trap cleanup EXIT INT TERM
+
+ensure_store_group() {
+  if ! pw groupshow "$store_group" >/dev/null 2>&1; then
+    if ! sudo pw groupadd "$store_group" -g 35000 >/dev/null 2>&1; then
+      sudo pw groupadd "$store_group" >/dev/null
+    fi
+    group_created=yes
+  else
+    group_created=no
+  fi
+  group_gid=$(pw groupshow "$store_group" | awk -F: '{print $3}')
+}
+
+clean_demo_items() {
+  sudo find "$store_root" -mindepth 2 -maxdepth 2 -name .fruix-demo -print 2>/dev/null |
+    while IFS= read -r marker; do
+      [ -n "$marker" ] || continue
+      sudo rm -rf "$(dirname "$marker")"
+    done
+  sudo rm -f "$fruix_gcroots_dir/demo-root"
+}
+
+install_store_item() {
+  item_name=$1
+  stage_dir=$2
+  item_hash=$3
+  destination=$store_root/${item_hash}-${item_name}
+
+  sudo rm -rf "$destination"
+  sudo mkdir -p "$destination"
+  sudo cp -a "$stage_dir/." "$destination/"
+  sudo find "$destination" -type d -exec chmod 0555 {} +
+  sudo find "$destination" -type f -exec chmod 0444 {} +
+  if [ -d "$destination/bin" ]; then
+    sudo find "$destination/bin" -type f -exec chmod 0555 {} +
+  fi
+  printf '%s\n' "$destination"
+}
+
+mark_reachable() {
+  item=$1
+  [ -n "$item" ] || return 0
+  [ -d "$item" ] || return 0
+  if grep -Fxq "$item" "$reachable_file" 2>/dev/null; then
+    return 0
+  fi
+  printf '%s\n' "$item" >> "$reachable_file"
+  if [ -f "$item/.references" ]; then
+    while IFS= read -r ref; do
+      [ -n "$ref" ] || continue
+      mark_reachable "$ref"
+    done < "$item/.references"
+  fi
+}
+
+run_gc() {
+  reachable_file=$1
+  : > "$reachable_file"
+  if [ -d "$fruix_gcroots_dir" ]; then
+    for root in "$fruix_gcroots_dir"/*; do
+      [ -L "$root" ] || continue
+      mark_reachable "$(readlink -f "$root")"
+    done
+  fi
+
+  sudo find "$store_root" -mindepth 2 -maxdepth 2 -name .fruix-demo -print |
+    while IFS= read -r marker; do
+      item=$(dirname "$marker")
+      if ! grep -Fxq "$item" "$reachable_file" 2>/dev/null; then
+        sudo rm -rf "$item"
+      fi
+    done
+}
+
+mkdir -p "$workdir/stage"
+ensure_store_group
+
+sudo install -d -m 0755 -o root -g wheel /frx
+sudo install -d -m 1775 -o root -g "$store_group" "$store_root"
+sudo install -d -m 0755 -o root -g wheel "$state_root" "$sysconf_root" "$fruix_state_dir" "$fruix_gcroots_dir"
+
+clean_demo_items
+sudo stat -f '%Su %Sg %OLp %Sp' "$store_root" > "$store_root_stat"
+sudo stat -f '%Su %Sg %OLp %Sp' "$fruix_gcroots_dir" > "$gcroots_stat"
+
+data_stage=$workdir/stage/greeting-data
+app_stage=$workdir/stage/hello-app
+orphan_stage=$workdir/stage/orphan-data
+mkdir -p "$data_stage/share" "$app_stage/bin" "$orphan_stage/share"
+
+printf 'hello-from-frx-store\n' > "$data_stage/share/greeting.txt"
+printf 'demo-item\n' > "$data_stage/.fruix-demo"
+: > "$data_stage/.references"
+data_manifest=$(printf 'name=demo-greeting-data\nfile=share/greeting.txt\ncontent=%s\n' "hello-from-frx-store")
+data_hash=$(printf '%s' "$data_manifest" | sha256 -q)
+data_path=$(install_store_item demo-greeting-data "$data_stage" "$data_hash")
+
+cat > "$app_stage/bin/demo-hello" <<EOF
+#!/bin/sh
+exec /bin/cat "$data_path/share/greeting.txt"
+EOF
+printf '%s\n' "$data_path" > "$app_stage/.references"
+printf 'demo-item\n' > "$app_stage/.fruix-demo"
+app_manifest=$(printf 'name=demo-hello-app\nfile=bin/demo-hello\nref=%s\n' "$data_path")
+app_hash=$(printf '%s' "$app_manifest" | sha256 -q)
+app_path=$(install_store_item demo-hello-app "$app_stage" "$app_hash")
+
+printf 'this-should-be-collected\n' > "$orphan_stage/share/orphan.txt"
+: > "$orphan_stage/.references"
+printf 'demo-item\n' > "$orphan_stage/.fruix-demo"
+orphan_manifest=$(printf 'name=demo-orphan-data\nfile=share/orphan.txt\ncontent=%s\n' "this-should-be-collected")
+orphan_hash=$(printf '%s' "$orphan_manifest" | sha256 -q)
+orphan_path=$(install_store_item demo-orphan-data "$orphan_stage" "$orphan_hash")
+
+sudo ln -sfn "$app_path" "$fruix_gcroots_dir/demo-root"
+
+sudo -u nobody /bin/sh -eu -c "
+  cat '$data_path/share/greeting.txt'
+  '$app_path/bin/demo-hello'
+" > "$access_read_out"
+
+set +e
+sudo -u nobody /bin/sh -eu -c "touch '$store_root/should-not-write'" >/dev/null 2> "$access_write_err"
+write_rc=$?
+set -e
+if [ "$write_rc" -eq 0 ]; then
+  echo "unexpected unprivileged store write succeeded" >&2
+  exit 1
+fi
+
+reachable_after_first_gc=$workdir/reachable-first.txt
+run_gc "$reachable_after_first_gc"
+find "$store_root" -maxdepth 1 -mindepth 1 | sort > "$first_gc_listing"
+
+if [ -d "$orphan_path" ]; then
+  echo "orphan store item survived rooted GC unexpectedly" >&2
+  exit 1
+fi
+if [ ! -d "$app_path" ] || [ ! -d "$data_path" ]; then
+  echo "reachable store items missing after rooted GC" >&2
+  exit 1
+fi
+
+sudo rm -f "$fruix_gcroots_dir/demo-root"
+reachable_after_second_gc=$workdir/reachable-second.txt
+run_gc "$reachable_after_second_gc"
+find "$store_root" -maxdepth 1 -mindepth 1 | sort > "$second_gc_listing"
+
+if [ -d "$app_path" ] || [ -d "$data_path" ]; then
+  echo "unrooted store items survived GC unexpectedly" >&2
+  exit 1
+fi
+
+cat > "$metadata_file" <<EOF
+store_root=$store_root
+state_root=$state_root
+sysconf_root=$sysconf_root
+fruix_state_dir=$fruix_state_dir
+fruix_gcroots_dir=$fruix_gcroots_dir
+store_group=$store_group
+store_group_gid=$group_gid
+group_created=$group_created
+store_root_stat=$(cat "$store_root_stat")
+gcroots_stat=$(cat "$gcroots_stat")
+data_hash=$data_hash
+app_hash=$app_hash
+orphan_hash=$orphan_hash
+data_path=$data_path
+app_path=$app_path
+orphan_path=$orphan_path
+nobody_read_output=$(tr '\n' '|' < "$access_read_out")
+unprivileged_write_rc=$write_rc
+unprivileged_write_err=$(tr '\n' '|' < "$access_write_err")
+first_gc_listing_file=$first_gc_listing
+second_gc_listing_file=$second_gc_listing
+first_gc_listing_begin
+$(cat "$first_gc_listing")
+first_gc_listing_end
+second_gc_listing_begin
+$(cat "$second_gc_listing")
+second_gc_listing_end
+EOF
+
+if [ -n "${METADATA_OUT:-}" ]; then
+  mkdir -p "$(dirname "$METADATA_OUT")"
+  cp "$metadata_file" "$METADATA_OUT"
+fi
+
+printf 'PASS freebsd-store-prototype\n'
+printf 'Metadata file: %s\n' "$metadata_file"
+if [ -n "${METADATA_OUT:-}" ]; then
+  printf 'Copied metadata to: %s\n' "$METADATA_OUT"
+fi
+printf '%s\n' '--- nobody read output ---'
+cat "$access_read_out"
+printf '%s\n' '--- first GC listing ---'
+cat "$first_gc_listing"
+printf '%s\n' '--- second GC listing ---'
+cat "$second_gc_listing"
+printf '%s\n' '--- metadata ---'
+cat "$metadata_file"