self/fruix
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Prototype FreeBSD build user isolation

Browse Source
This commit is contained in:
Steffen
2026-04-01 11:43:56 +02:00
parent 7621798ef5
commit d65b2afb27
4 changed files with 724 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

76
docs/PROGRESS.md
View File

@@ -896,3 +896,79 @@ Next recommended step:
2. then establish a `/frx/store`-based store-management prototype covering permissions, package readability, and garbage-collection behavior
3. continue carrying the separate Guix checkout runtime blocker:
- investigate the `leave-on-EPIPE` failure in `./pre-inst-env guix --version`
## 2026-04-01 — Phase 2.2 completed: privilege dropping and concurrent build-user isolation validated
Completed work:
- added a C helper implementing the core daemon-side privilege drop mechanics:
- `tests/daemon/freebsd-build-user-helper.c`
- added a harness that combines that helper with the new jail model and runs two jobs concurrently:
- `tests/daemon/run-freebsd-privilege-drop-prototype.sh`
- wrote the Phase 2.2 report:
- `docs/reports/phase2-freebsd-privilege-drop.md`
- ran the concurrent build-user prototype successfully and captured metadata under:
- `/tmp/freebsd-privdrop-metadata.txt`
Important findings:
- a root-launched FreeBSD helper can successfully perform the expected daemon-side transition:
- `setgroups`
- `setgid`
- `setuid`
into a dedicated build identity
- once dropped, the helper cannot regain root with `setuid(0)`:
- `Operation not permitted`
- each build job can create files in its own writable directory and those files end up owned by the dropped build UID/GID rather than by root
- two concurrent jobs using distinct numeric build identities succeeded with:
- job 1 UID/GID `35001:35001`
- job 2 UID/GID `35002:35002`
- host-side result files were observed with the matching ownership and restrictive permissions:
- `0600`
- the two jobs were deliberately held for two seconds each and the measured wall-clock elapsed time was also about two seconds, demonstrating actual concurrent execution rather than serialized execution
- two complementary denial modes were validated at the same time:
- peer build files mounted but blocked by permissions: `Permission denied`
- host path not mounted into the jail at all: `No such file or directory`
- the dropped build user also could not:
- create files in a protected root-owned directory
- `chown` its own output back to root
Current assessment:
- Phase 2.2 is now satisfied on the current FreeBSD prototype track
- the combined jail + build-user model now has practical validation for the most important security properties required by a future FreeBSD Guix daemon:
- root-controlled setup
- permanent drop to build credentials
- per-build writable areas
- cross-build isolation
- concurrent execution under distinct identities
- the remaining Phase 2 work is now centered on the store itself: permissions, readability, content-addressed layout, and garbage-collection behavior under `/frx/store`
Recent commits:
- `e380e88` — `Add FreeBSD Guile verification harness`
- `cd721b1` — `Update progress after Guile verification`
- `27916cb` — `Diagnose Guile subprocess crash on FreeBSD`
- `02f7a7f` — `Validate local Guile fix on FreeBSD`
- `4aebea4` — `Add native GNU Hello FreeBSD build harness`
- `c944cdb` — `Validate Guix builder phases on FreeBSD`
- `0a2e48e` — `Validate GNU which builder phases on FreeBSD`
- `245a47d` — `Document gaps to real Guix FreeBSD builds`
- `d62e9b0` — `Investigate Guix derivation generation on FreeBSD`
- `c0a85ed` — `Build local Guile-GnuTLS on FreeBSD`
- `15b9037` — `Build local Guile-Git on FreeBSD`
- `47d31e8` — `Build local Guile-JSON on FreeBSD`
- `d82195b` — `Advance Guix checkout on FreeBSD`
- `9bf3d30` — `Document FreeBSD syscall mapping`
- `7621798` — `Prototype FreeBSD jail build isolation`
Next recommended step:
1. complete Phase 2.3 by establishing a `/frx/store`-based store prototype with:
- correct root/daemon write restrictions
- unprivileged read access
- content-addressed path naming
- garbage-collection behavior
2. if possible, use outputs or dependency relationships realistic enough to model how a future FreeBSD Guix daemon would retain referenced store items
3. continue carrying the separate Guix checkout runtime blocker:
- investigate the `leave-on-EPIPE` failure in `./pre-inst-env guix --version`

159
docs/reports/phase2-freebsd-privilege-drop.md Normal file
View File

@@ -0,0 +1,159 @@
# Phase 2.2: FreeBSD privilege dropping and build-user isolation prototype
Date: 2026-04-01
## Summary
This step adds a concrete FreeBSD privilege-dropping prototype that complements the Phase 2.1 jail-isolation work.
Added files:
- `tests/daemon/freebsd-build-user-helper.c`
- `tests/daemon/run-freebsd-privilege-drop-prototype.sh`
The prototype demonstrates that a root-launched daemon-side process can:
- enter a FreeBSD jail build environment
- drop permanently to a designated numeric build UID/GID
- lose the ability to regain root with `setuid(0)`
- write only to its own build directory
- fail to read a peer build user's files
- fail to access a host path that is not mounted into the jail
- fail to create files in a protected root-owned directory
- run two builds concurrently under different build identities
## Prototype design
### Build-user model
For this prototype, build users are represented by dedicated numeric IDs rather than pre-created named accounts:
- job 1: UID/GID `35001`
- job 2: UID/GID `35002`
This is enough to validate the essential daemon-side mechanics:
- root can allocate a build identity
- root can prepare directories owned by that identity
- the build process can drop into that identity before executing untrusted work
For a real FreeBSD Guix daemon, these numeric identities would likely come from a reserved build-user pool, whether or not they are backed by visible passwd entries.
### Isolation structure
Each job gets its own jail and its own writable work directory. Inside each jail, the helper sees:
- `/lib`
- `/libexec`
- `/tools/freebsd-build-user-helper`
- `/work` (owned by that job's UID/GID)
- `/peer` (mounted from the other job's work directory, but inaccessible because of permissions)
- `/protected` (root-owned and not writable)
A host-side sentinel file is left outside the jail entirely and is referenced as `/outside-sentinel`; the helper confirms it is not visible.
### Concurrency check
Both jailed helper processes are started in parallel and intentionally sleep for two seconds after their checks. The observed elapsed wall-clock time was also two seconds, not four, showing that the prototype was running the two build-user jobs concurrently rather than serially.
## Run command
```sh
METADATA_OUT=/tmp/freebsd-privdrop-metadata.txt \
./tests/daemon/run-freebsd-privilege-drop-prototype.sh
```
## Observed results
Observed helper output for job 1:
```text
job=job1
target.uid=35001
target.gid=35001
post_drop.euid=35001
post_drop.egid=35001
drop.regain_root=Operation not permitted
access.allowed_read=job1-allowed
access.own_output_uid=35001
access.own_output_gid=35001
access.peer_file=Permission denied
access.hidden_file=No such file or directory
access.protected_create=Permission denied
drop.chown_root=Operation not permitted
```
Observed helper output for job 2 was identical in structure, with UID/GID `35002`.
Observed host-side result file ownership:
- job 1 result file: UID/GID `35001:35001`, mode `0600`
- job 2 result file: UID/GID `35002:35002`, mode `0600`
Observed concurrency metadata:
- configured hold time per job: `2` seconds
- measured elapsed wall-clock time: `2` seconds
Metadata was captured in:
- `/tmp/freebsd-privdrop-metadata.txt`
## What this demonstrates for the FreeBSD port
### 1. Permanent privilege dropping works in the required direction
The helper was started as root by the harness, then used:
- `setgroups`
- `setgid`
- `setuid`
After the drop, `setuid(0)` failed with `Operation not permitted`, which is the basic property Guix needs from a build-worker launch path.
### 2. Build-user ownership boundaries behave as expected
Each job could write its own result file, and the file ended up owned by the dropped build UID/GID rather than root. Each job then failed to read the peer job's file due to permission checks.
### 3. Jail filesystem containment and credential isolation compose correctly
Two distinct denial modes were observed:
- peer work directory mounted but inaccessible: `Permission denied`
- host path not mounted into the jail: `No such file or directory`
This is exactly the combined model the FreeBSD daemon design needs:
- jail layout restricts visibility
- build-user credentials restrict access where visibility alone is not enough
### 4. Protected daemon-owned paths remain protected after the drop
The dropped build user could not create files under the root-owned `/protected` directory and could not `chown` its output back to root. This is the essential privilege-separation behavior needed to keep store/daemon-owned paths outside build control.
## Mapping to Guix-daemon needs
| Guix-daemon requirement | Prototype result |
|---|---|
| launch build as root-controlled setup step | yes |
| drop to dedicated build UID/GID before untrusted work | yes |
| prevent build from regaining root | yes |
| keep per-build writable areas separate | yes |
| prevent one build from reading another build's files | yes |
| combine credential isolation with jail filesystem containment | yes |
| support concurrent builds under distinct identities | yes |
## Conclusion
Phase 2.2 is satisfied on the current prototype track:
- a concrete FreeBSD privilege-dropping implementation now exists in C
- it integrates cleanly with the jail-first model established in Phase 2.1
- it demonstrates the core properties required for a future FreeBSD Guix daemon:
- root-controlled setup
- permanent drop to build credentials
- cross-build isolation
- blocked access to protected daemon-owned paths
- concurrent build execution under separate identities
This reduces the remaining Phase 2 uncertainty mainly to store management and lifecycle, not to whether FreeBSD can enforce the basic build-user security model.

261
tests/daemon/freebsd-build-user-helper.c Normal file
View File

@@ -0,0 +1,261 @@
#include <errno.h>
#include <fcntl.h>
#include <grp.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
struct options {
const char *job_name;
uid_t uid;
gid_t gid;
const char *allowed_file;
const char *own_output;
const char *peer_file;
const char *hidden_file;
const char *protected_file;
unsigned int hold_seconds;
};
static int failures = 0;
static void usage(const char *argv0)
{
fprintf(stderr,
"usage: %s --job-name NAME --uid UID --gid GID --allowed-file PATH "
"--own-output PATH --peer-file PATH --hidden-file PATH "
"--protected-file PATH [--hold-seconds N]\n",
argv0);
}
static void print_value(const char *key, const char *value)
{
printf("%s=%s\n", key, value);
}
static void print_number(const char *key, long value)
{
printf("%s=%ld\n", key, value);
}
static void report_failure(const char *key, const char *detail)
{
failures++;
printf("%s=fail:%s\n", key, detail);
}
static bool parse_ulong(const char *text, unsigned long *value)
{
char *end = NULL;
unsigned long parsed;
errno = 0;
parsed = strtoul(text, &end, 10);
if (errno != 0 || end == text || *end != '\0')
return false;
*value = parsed;
return true;
}
static int parse_options(int argc, char **argv, struct options *opts)
{
int i;
memset(opts, 0, sizeof(*opts));
opts->hold_seconds = 0;
for (i = 1; i < argc; i++) {
if (strcmp(argv[i], "--job-name") == 0 && i + 1 < argc) {
opts->job_name = argv[++i];
} else if (strcmp(argv[i], "--uid") == 0 && i + 1 < argc) {
unsigned long value;
if (!parse_ulong(argv[++i], &value))
return -1;
opts->uid = (uid_t)value;
} else if (strcmp(argv[i], "--gid") == 0 && i + 1 < argc) {
unsigned long value;
if (!parse_ulong(argv[++i], &value))
return -1;
opts->gid = (gid_t)value;
} else if (strcmp(argv[i], "--allowed-file") == 0 && i + 1 < argc) {
opts->allowed_file = argv[++i];
} else if (strcmp(argv[i], "--own-output") == 0 && i + 1 < argc) {
opts->own_output = argv[++i];
} else if (strcmp(argv[i], "--peer-file") == 0 && i + 1 < argc) {
opts->peer_file = argv[++i];
} else if (strcmp(argv[i], "--hidden-file") == 0 && i + 1 < argc) {
opts->hidden_file = argv[++i];
} else if (strcmp(argv[i], "--protected-file") == 0 && i + 1 < argc) {
opts->protected_file = argv[++i];
} else if (strcmp(argv[i], "--hold-seconds") == 0 && i + 1 < argc) {
unsigned long value;
if (!parse_ulong(argv[++i], &value))
return -1;
opts->hold_seconds = (unsigned int)value;
} else {
return -1;
}
}
if (opts->job_name == NULL || opts->allowed_file == NULL || opts->own_output == NULL ||
opts->peer_file == NULL || opts->hidden_file == NULL || opts->protected_file == NULL)
return -1;
return 0;
}
static void require_drop(uid_t uid, gid_t gid)
{
gid_t groups[1];
groups[0] = gid;
if (setgroups(1, groups) != 0) {
report_failure("drop.setgroups", strerror(errno));
return;
}
if (setgid(gid) != 0) {
report_failure("drop.setgid", strerror(errno));
return;
}
if (setuid(uid) != 0) {
report_failure("drop.setuid", strerror(errno));
return;
}
print_number("post_drop.euid", (long)geteuid());
print_number("post_drop.egid", (long)getegid());
}
static void check_setuid_regain_root(void)
{
if (setuid(0) == 0) {
report_failure("drop.regain_root", "unexpected-success");
return;
}
print_value("drop.regain_root", strerror(errno));
}
static void check_read_allowed(const char *path)
{
int fd;
char buffer[256];
ssize_t nread;
fd = open(path, O_RDONLY);
if (fd < 0) {
report_failure("access.allowed_read", strerror(errno));
return;
}
nread = read(fd, buffer, sizeof(buffer) - 1);
close(fd);
if (nread < 0) {
report_failure("access.allowed_read", strerror(errno));
return;
}
buffer[nread] = '\0';
print_value("access.allowed_read", buffer);
}
static void check_write_own_output(const char *path, const char *job_name)
{
int fd;
char buffer[256];
struct stat st;
fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0600);
if (fd < 0) {
report_failure("access.own_output", strerror(errno));
return;
}
snprintf(buffer, sizeof(buffer), "%s-ran-as-build-user\n", job_name);
if (write(fd, buffer, strlen(buffer)) < 0) {
close(fd);
report_failure("access.own_output", strerror(errno));
return;
}
close(fd);
if (stat(path, &st) != 0) {
report_failure("access.own_output_stat", strerror(errno));
return;
}
print_number("access.own_output_uid", (long)st.st_uid);
print_number("access.own_output_gid", (long)st.st_gid);
}
static void expect_denied(const char *key, const char *path)
{
int fd;
errno = 0;
fd = open(path, O_RDONLY);
if (fd >= 0) {
close(fd);
report_failure(key, "unexpected-success");
return;
}
print_value(key, strerror(errno));
}
static void expect_protected_create_denied(const char *path)
{
int fd;
errno = 0;
fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0600);
if (fd >= 0) {
close(fd);
unlink(path);
report_failure("access.protected_create", "unexpected-success");
return;
}
print_value("access.protected_create", strerror(errno));
}
static void expect_chown_denied(const char *path)
{
if (chown(path, 0, 0) == 0) {
report_failure("drop.chown_root", "unexpected-success");
return;
}
print_value("drop.chown_root", strerror(errno));
}
int main(int argc, char **argv)
{
struct options opts;
if (parse_options(argc, argv, &opts) != 0) {
usage(argv[0]);
return 2;
}
print_value("job", opts.job_name);
print_number("target.uid", (long)opts.uid);
print_number("target.gid", (long)opts.gid);
require_drop(opts.uid, opts.gid);
check_setuid_regain_root();
check_read_allowed(opts.allowed_file);
check_write_own_output(opts.own_output, opts.job_name);
expect_denied("access.peer_file", opts.peer_file);
expect_denied("access.hidden_file", opts.hidden_file);
expect_protected_create_denied(opts.protected_file);
expect_chown_denied(opts.own_output);
if (opts.hold_seconds > 0)
sleep(opts.hold_seconds);
return failures == 0 ? 0 : 1;
}

228
tests/daemon/run-freebsd-privilege-drop-prototype.sh Executable file
View File

@@ -0,0 +1,228 @@
#!/bin/sh
set -eu
cc_bin=${CC_BIN:-/usr/bin/cc}
uid1=${BUILD_UID1:-35001}
gid1=${BUILD_GID1:-35001}
uid2=${BUILD_UID2:-35002}
gid2=${BUILD_GID2:-35002}
hold_seconds=${HOLD_SECONDS:-2}
cleanup_workdir=0
mount_points=
jail_ids=
if [ -n "${WORKDIR:-}" ]; then
workdir=$WORKDIR
mkdir -p "$workdir"
else
workdir=$(mktemp -d /tmp/fruix-privdrop-prototype.XXXXXX)
cleanup_workdir=1
fi
if [ "${KEEP_WORKDIR:-0}" -eq 1 ]; then
cleanup_workdir=0
fi
record_mount() {
mount_points="$1
$mount_points"
}
record_jail() {
jail_ids="$1
$jail_ids"
}
cleanup() {
set +e
old_ifs=$IFS
IFS='
'
for jail_id in $jail_ids; do
[ -n "$jail_id" ] || continue
sudo jail -r "$jail_id" >/dev/null 2>&1 || true
done
for mount_point in $mount_points; do
[ -n "$mount_point" ] || continue
sudo umount "$mount_point" >/dev/null 2>&1 || true
done
IFS=$old_ifs
if [ "$cleanup_workdir" -eq 1 ]; then
sudo rm -rf "$workdir"
fi
}
trap cleanup EXIT INT TERM
helper_src=tests/daemon/freebsd-build-user-helper.c
helper_bin=$workdir/tools/freebsd-build-user-helper
protected_dir=$workdir/protected
outside_sentinel=$workdir/outside-sentinel
metadata_file=$workdir/freebsd-privdrop-metadata.txt
helper_compile_log=$workdir/helper-compile.log
helper_compile_err=$workdir/helper-compile.err
job1_out=$workdir/job1.out
job1_err=$workdir/job1.err
job2_out=$workdir/job2.out
job2_err=$workdir/job2.err
job1_stat=$workdir/job1.stat
job2_stat=$workdir/job2.stat
mkdir -p "$workdir/tools" "$protected_dir" "$workdir/job1-host" "$workdir/job2-host"
printf 'host-only-outside-sentinel\n' > "$outside_sentinel"
printf 'protected-root-owned\n' > "$protected_dir/root-note.txt"
chmod 0755 "$protected_dir"
chmod 0644 "$protected_dir/root-note.txt"
"$cc_bin" -Wall -Wextra -std=c11 "$helper_src" -o "$helper_bin" >"$helper_compile_log" 2>"$helper_compile_err"
prepare_job_tree() {
job_name=$1
job_uid=$2
job_gid=$3
peer_uid=$4
peer_gid=$5
job_host=$workdir/$job_name-host
root=$job_host/root
work_mount=$job_host/work
peer_mount=$job_host/peer
mkdir -p "$root/tools" "$root/protected" "$root/work" "$root/peer" "$root/tmp" "$work_mount" "$peer_mount"
chmod 1777 "$root/tmp"
printf '%s-allowed\n' "$job_name" > "$work_mount/allowed.txt"
printf '%s-secret\n' "$job_name" > "$work_mount/secret.txt"
chmod 0600 "$work_mount/allowed.txt" "$work_mount/secret.txt"
chmod 0700 "$work_mount"
sudo chown -R "$job_uid:$job_gid" "$work_mount"
printf 'peer-for-%s\n' "$job_name" > "$peer_mount/secret.txt"
chmod 0600 "$peer_mount/secret.txt"
chmod 0700 "$peer_mount"
sudo chown -R "$peer_uid:$peer_gid" "$peer_mount"
for host_path in /lib /libexec; do
sudo mkdir -p "$root$host_path"
sudo mount_nullfs -o ro "$host_path" "$root$host_path"
record_mount "$root$host_path"
done
sudo mount_nullfs -o ro "$workdir/tools" "$root/tools"
record_mount "$root/tools"
sudo mount_nullfs "$work_mount" "$root/work"
record_mount "$root/work"
sudo mount_nullfs -o ro "$peer_mount" "$root/peer"
record_mount "$root/peer"
sudo mount_nullfs "$protected_dir" "$root/protected"
record_mount "$root/protected"
jail_name=fruix-privdrop-$job_name-$$
jail_id=$(sudo jail -i -c \
name="$jail_name" \
path="$root" \
host.hostname="$jail_name" \
persist \
ip4=disable \
ip6=disable)
record_jail "$jail_id"
printf '%s\n' "$jail_id" > "$job_host/jid"
}
prepare_job_tree job1 "$uid1" "$gid1" "$uid2" "$gid2"
prepare_job_tree job2 "$uid2" "$gid2" "$uid1" "$gid1"
jid1=$(cat "$workdir/job1-host/jid")
jid2=$(cat "$workdir/job2-host/jid")
start_epoch=$(date +%s)
sudo jexec "$jid1" /tools/freebsd-build-user-helper \
--job-name job1 \
--uid "$uid1" \
--gid "$gid1" \
--allowed-file /work/allowed.txt \
--own-output /work/result.txt \
--peer-file /peer/secret.txt \
--hidden-file /outside-sentinel \
--protected-file /protected/escape-job1 \
--hold-seconds "$hold_seconds" >"$job1_out" 2>"$job1_err" &
pid1=$!
sudo jexec "$jid2" /tools/freebsd-build-user-helper \
--job-name job2 \
--uid "$uid2" \
--gid "$gid2" \
--allowed-file /work/allowed.txt \
--own-output /work/result.txt \
--peer-file /peer/secret.txt \
--hidden-file /outside-sentinel \
--protected-file /protected/escape-job2 \
--hold-seconds "$hold_seconds" >"$job2_out" 2>"$job2_err" &
pid2=$!
set +e
wait "$pid1"
rc1=$?
wait "$pid2"
rc2=$?
set -e
end_epoch=$(date +%s)
elapsed=$((end_epoch - start_epoch))
if [ "$rc1" -ne 0 ] || [ "$rc2" -ne 0 ]; then
echo "freebsd privilege-drop prototype failed" >&2
cat "$job1_out" >&2 || true
cat "$job1_err" >&2 || true
cat "$job2_out" >&2 || true
cat "$job2_err" >&2 || true
exit 1
fi
sudo stat -f '%Su %Sg %u %g %Sp' "$workdir/job1-host/work/result.txt" > "$job1_stat"
sudo stat -f '%Su %Sg %u %g %Sp' "$workdir/job2-host/work/result.txt" > "$job2_stat"
cat > "$metadata_file" <<EOF
workdir=$workdir
helper_src=$helper_src
helper_bin=$helper_bin
helper_compile_log=$helper_compile_log
helper_compile_err=$helper_compile_err
jail1_id=$jid1
jail2_id=$jid2
build_uid1=$uid1
build_gid1=$gid1
build_uid2=$uid2
build_gid2=$gid2
hold_seconds=$hold_seconds
elapsed_seconds=$elapsed
job1_rc=$rc1
job2_rc=$rc2
job1_result_stat=$(cat "$job1_stat")
job2_result_stat=$(cat "$job2_stat")
job1_output_begin
$(cat "$job1_out")
job1_output_end
job2_output_begin
$(cat "$job2_out")
job2_output_end
EOF
if [ -n "${METADATA_OUT:-}" ]; then
mkdir -p "$(dirname "$METADATA_OUT")"
cp "$metadata_file" "$METADATA_OUT"
fi
printf 'PASS freebsd-privilege-drop-prototype\n'
printf 'Working directory: %s\n' "$workdir"
printf 'Metadata file: %s\n' "$metadata_file"
if [ -n "${METADATA_OUT:-}" ]; then
printf 'Copied metadata to: %s\n' "$METADATA_OUT"
fi
printf '%s\n' '--- job1 output ---'
cat "$job1_out"
printf '%s\n' '--- job2 output ---'
cat "$job2_out"
printf 'Elapsed seconds: %s\n' "$elapsed"
printf '%s\n' '--- metadata ---'
cat "$metadata_file"