docs: PLAN

docs/PLAN.md

@@ -0,0 +1,208 @@
+# Porting GNU Guix Package Management and Shepherd Init System to FreeBSD: A Comprehensive Implementation Plan
+
+This report presents a detailed technical roadmap for porting GNU Guix, including its package management system, system builder, and Shepherd init system, to run on top of the FreeBSD kernel. The project requires adapting fundamental system-level abstractions while maintaining Guix's core functional programming paradigm and reproducibility guarantees. Success will be achieved through a phased approach with clearly defined intermediate milestones, each building upon verified foundational components. Rather than attempting full package compatibility, the strategy focuses on establishing a minimal but functional environment that demonstrates core Guix functionality on FreeBSD, including basic package management, declarative system configuration, and service management through Shepherd.
+
+## Understanding the Current Architectures and Compatibility Landscape
+
+### GNU Guix's Dependence on Linux Abstractions
+
+GNU Guix as currently implemented relies heavily on Linux-specific kernel features and abstractions that have been developed over two decades of Linux system design[10][32][17]. The most critical of these dependencies involves process isolation mechanisms. **Guix-daemon**, the system-wide daemon responsible for building packages and managing the `/gnu/store` directory, achieves its isolation guarantees through a combination of techniques that are deeply rooted in Linux kernel capabilities[17][32]. When a user requests package compilation through commands like `guix build`, the daemon spawns build processes under separate build users using the `setuid` system call, executes them within a `chroot` environment with only declared dependencies visible, and runs processes under separate PID, networking, and mount "namespaces"[32][17].
+
+These Linux namespaces represent a fundamental architectural departure from FreeBSD's containerization model. Linux namespaces allow users to selectively isolate specific aspects of the system environment—mount namespaces for filesystem visibility, PID namespaces for process visibility, network namespaces for network isolation, user namespaces for user remapping, and IPC namespaces for inter-process communication[32]. This granular approach enables building unprivileged processes with isolation guarantees. Guix recently achieved a significant milestone by implementing unprivileged `guix-daemon` operation through Linux's unprivileged user namespaces, eliminating the previous requirement for root privileges[17][17][17]. This capability allows non-root users to create isolated build environments without any privilege escalation mechanism, fundamentally changing the security model.
+
+Beyond process isolation, Guix depends on specific Linux filesystem capabilities and semantics. The `/gnu/store` directory requires particular permission models and behavior that are assumed to work in specific ways on Linux[32][43]. Fixed-output derivations, which model operations like file downloads where content hashes are known in advance, depend on Linux's ability to verify build reproducibility. The build daemon's privilege dropping mechanism, as recently enhanced, relies on Linux's capability system where specific capabilities like `CAP_CHOWN` and `CAP_SETUID` can be granted or revoked independently[17][17][17].
+
+### FreeBSD's Containerization and Process Isolation Model
+
+FreeBSD approaches system-level process isolation through a fundamentally different architectural philosophy compared to Linux[24][31][31]. Rather than providing selective, granular namespace isolation, FreeBSD implements **jails**, which create a more opaque boundary between isolated environments[24][31]. A jail is not simply a collection of namespace restrictions but rather represents what might be called a "security baseline"—when you create a jail, processes within it are isolated in multiple dimensions simultaneously by default[31][31]. The filesystem root can be isolated or shared, networking can be fully isolated or restricted to specific IP addresses, and user space is always isolated[24][31].
+
+The fundamental difference in architectural philosophy manifests in how isolation is achieved. In Linux namespaces, isolation is opt-in—you start with no isolation and selectively apply the isolation mechanisms you need[31]. In FreeBSD jails, isolation is the default baseline, and you selectively disable it where needed[31]. This difference has profound implications for port design. Where Linux represents isolation through kernel features that can be enabled independently, FreeBSD represents isolation through modifications to the process table itself, adding an additional column for Jail ID (JID), similar to how traditional Unix systems add user ID (UID) and process ID (PID)[31][31]. The host system itself is Jail ID 0.
+
+FreeBSD provides several variants of jails suited for different use cases[24]. Thick jails contain a complete copy of the FreeBSD base system within the jail's environment, creating an almost standalone FreeBSD installation with its own libraries, executables, and configuration files[24]. Thin jails, by contrast, share the base system with the host. VNET jails provide network virtualization through isolated network stacks, allowing each jail to have separate IP addresses, routing tables, and network interfaces[24]. Additionally, FreeBSD supports Linux jails, which enable execution of Linux binaries within a FreeBSD jail through a Linux Binary Compatibility layer[24].
+
+The implications for Guix porting are significant. FreeBSD jails cannot directly replicate the permission model that Guix assumes. While jails can provide strong isolation for security purposes, they do not provide the fine-grained capability system that allows Linux to selectively drop specific privileges like `CAP_SETUID` or `CAP_CHOWN`[17]. FreeBSD's approach relies on different permission primitives altogether. Additionally, the build user isolation strategy that Guix employs—creating separate unprivileged build users and using `setuid` to switch between them for concurrent builds—does not map cleanly onto FreeBSD's permission model[32][17].
+
+### The Shepherd Init System Architecture
+
+The GNU Shepherd is Guix's initialization system, designed to serve as PID 1—the first process started when the system boots. Unlike traditional Unix init systems like SysV init or modern systemd, Shepherd is written entirely in GNU Guile Scheme, making it scriptable and extensible using the same language and paradigms as the rest of Guix[11][37]. This tight integration is intentional and represents a core design principle of Guix: the entire system, from package definitions to service management, is expressed in a single, consistent programming language and functional paradigm.
+
+Shepherd services are defined through Guile Scheme records that specify service names, procedures to start and stop services, and dependencies between services[11]. The flexibility of this approach is demonstrated by Guix's recent addition of containerization features for Shepherd services—this was implemented entirely within Guix without requiring any modifications to Shepherd itself, because the system's extensibility through Scheme enables such customization at the distribution level[11]. Services are managed through the `(gnu services shepherd)` module, which provides abstractions for defining how services should be started, stopped, and monitored.
+
+FreeBSD traditionally uses the rcng (run-command next generation) init system, which implements a shell-script-based framework for service management[3][37]. This represents a completely different paradigm from Shepherd. The rcng system is based on shell scripts, configuration flags, and traditional Unix conventions, whereas Shepherd is a programmable, scheme-based system manager. Additionally, FreeBSD's init utilities follow traditional POSIX conventions as documented in the init manual page[21], whereas Shepherd provides a modern, declarative approach to system initialization.
+
+## Phase 1: Foundation Assessment and Toolchain Preparation
+
+### Intermediate Goal 1.1: Verify Guile Scheme Availability on FreeBSD
+
+The first critical step is confirming that GNU Guile, the Scheme implementation that forms the linguistic foundation of Guix, can run reliably on FreeBSD. Guile is available in FreeBSD's ports collection as the `lang/guile` meta-port and various versioned Guile ports[14][47]. The Guile interpreter is documented to work for interactive use, basic scripting, and extension of larger applications[14], making it suitable as the foundation for Guix.
+
+**Verification Goal 1.1:** Successfully compile and execute the latest stable version of GNU Guile on FreeBSD for all target architectures (at minimum, amd64 and i386). Verify that Guile can execute non-trivial Scheme programs, including programs that use modules, perform file I/O, handle system processes through Guile's procedures for creating and managing processes, and manipulate network sockets. Create a test suite that exercises Guile's foreign function interface (FFI) for calling C library functions, as Guix depends heavily on this capability for accessing system APIs. Document any performance differences or missing features compared to Linux Guile.
+
+**Success Criteria:** Guile successfully executes the Guix package manager's bootstrap code (specifically code from the bootstrap phase described in `make-bootstrap.scm`), produces identical output when executing deterministic test programs on both Linux and FreeBSD, and Guile's module system correctly loads and manages dependencies across multiple packages.
+
+### Intermediate Goal 1.2: Establish Cross-Platform Build Environment
+
+Before modifying Guix itself, establish a complete GNU toolchain and build environment on FreeBSD. This includes GCC or Clang, GNU Make, Autoconf, Automake, Libtool, and other tools that Guix uses during package building. Additionally, identify any GNU tools that Guix depends on that may have different behavior or availability on FreeBSD.
+
+**Verification Goal 1.2:** Create a minimal Guix expression that can be built on FreeBSD using native build tools. This expression should build a simple C program (e.g., GNU Hello) using the `gnu-build-system` with necessary adaptations for FreeBSD. The build process must successfully:
+
+- Download source code from a remote repository
+- Extract and verify the source integrity
+- Configure the package using GNU autotools (even if slightly modified for FreeBSD)
+- Compile the package to produce working binaries
+- Install binaries to a staging directory
+- Generate proper metadata about build inputs and outputs
+
+Document all differences between the FreeBSD build process and the Linux equivalent, including differences in available build tools, default compilation flags, and system-specific requirements.
+
+**Success Criteria:** Successfully build and execute a simple GNU Hello package on FreeBSD. The compiled binary must work correctly and produce output identical to a Linux-compiled version. All necessary build dependencies must be tracked and available through FreeBSD's ports collection or through locally-defined Guix packages.
+
+### Intermediate Goal 1.3: Map FreeBSD System Call Interface
+
+Create comprehensive documentation of how FreeBSD's system call interface differs from Linux in areas critical to Guix operation. This includes detailed analysis of process creation, process isolation, filesystem operations, and capability-related system calls.
+
+**Verification Goal 1.3:** Produce a detailed technical specification documenting:
+
+- All Linux system calls that Guix currently uses, organized by functional category (process management, filesystem access, capability manipulation, etc.)
+- Corresponding FreeBSD equivalents or alternative approaches for achieving the same functionality
+- Gaps where FreeBSD provides no direct equivalent and requires architectural workarounds
+- Performance implications of any required workarounds
+- Compatibility layers or alternative system calls available on FreeBSD
+
+Create a test program in C that exercises the most critical system call mappings and verify correct behavior. Document areas where FreeBSD's behavior fundamentally differs from Linux in ways that cannot be automatically bridged through system call translation (such as the absence of user namespaces).
+
+**Success Criteria:** The specification document must be thorough enough that developers unfamiliar with FreeBSD can understand exactly how Guix must be modified to work with FreeBSD's kernel interface. All critical system calls must have identified solutions.
+
+## Phase 2: Core Guix-Daemon Architecture Adaptation
+
+### Intermediate Goal 2.1: Design Process Isolation Strategy for FreeBSD
+
+The most fundamental change required is reimplementing Guix-daemon's process isolation without access to Linux namespaces. This is the single most critical architectural challenge for the port. Rather than using separate namespaces for mount, PID, and networking, the implementation must use FreeBSD jails as the primary isolation mechanism.
+
+**Verification Goal 2.1:** Produce a detailed architectural design document that specifies:
+
+- How FreeBSD jails will replace Linux namespace isolation for build processes
+- How build users will be managed within the jail context
+- Whether to use thin or thick jails for build environments, with justification
+- How filesystem isolation will be achieved (mapping only declared dependencies into build environments)
+- How networking isolation will be configured (whether to use VNET jails for full network isolation)
+- How the design handles concurrent builds (whether each build gets its own jail or if builds share jail infrastructure)
+- Security implications of the proposed approach compared to Linux-based isolation
+
+Create a prototype implementation of a single build operation within a jail, demonstrating that basic build commands can execute correctly within a jail with restricted filesystem visibility.
+
+**Success Criteria:** The architectural design must clearly specify how each current Guix isolation feature maps to FreeBSD jail features, with identified gaps and proposed solutions. The prototype must successfully execute a build command within a FreeBSD jail with restricted filesystem access, demonstrating that the jail can be configured to see only explicitly mounted paths.
+
+### Intermediate Goal 2.2: Implement Privilege Dropping Mechanism for FreeBSD
+
+Develop a privilege-dropping mechanism equivalent to Guix's current Linux implementation. Linux uses `setuid`, separate build users, and capability dropping. FreeBSD has different privilege primitives.
+
+**Verification Goal 2.2:** Implement a modified version of the Guix-daemon that:
+
+- Operates with minimal required privileges (rather than requiring root as traditionally necessary)
+- Manages build user creation and switching on FreeBSD (whether through FreeBSD's user management system or within jails)
+- Implements privilege dropping equivalent to Linux's capability system, using FreeBSD's available mechanisms
+- Ensures that concurrent builds cannot interfere with each other
+- Ensures that build processes cannot access files outside their declared dependencies
+
+Create a C implementation that demonstrates the core privilege-dropping logic working correctly on FreeBSD. This implementation should be capable of creating a restricted build environment, executing a process within it, and ensuring that the process cannot escape the restrictions.
+
+**Success Criteria:** Successfully execute an untrusted build process with restricted filesystem and privilege access. Verify that escaped capabilities are minimized. Demonstrate that concurrent build processes running under different build users cannot access each other's files.
+
+### Intermediate Goal 2.3: Adapt the Store Concept for FreeBSD
+
+The `/gnu/store` directory and Guix's store management are core to Guix's model. Verify that this model can work on FreeBSD and adapt as necessary. The store should be accessible to all build processes through mounted paths but not directly modifiable.
+
+**Verification Goal 2.3:** Establish a working `/gnu/store` equivalent on FreeBSD that:
+
+- Can store built packages with appropriate permission models
+- Permits the Guix-daemon to write to the store
+- Prevents unprivileged users from writing to the store
+- Permits processes to read from the store
+- Uses content-addressable storage with hash-based naming to ensure reproducibility
+- Properly handles garbage collection to remove unused packages
+
+Create a minimal functional `/gnu/store` directory on FreeBSD, populate it with a few test packages, and verify that Guix-daemon can correctly manage these packages.
+
+**Success Criteria:** `/gnu/store` is functional on FreeBSD with the same permission model and content-addressable structure as on Linux. Guix-daemon can build and store packages. Multiple users can coexist on the system and access packages from the store according to proper permission rules.
+
+## Phase 3: Build System Adaptation
+
+### Intermediate Goal 3.1: Adapt Core Build Systems to FreeBSD
+
+The Guix `gnu-build-system` and other build systems need to be adapted to work with FreeBSD's build environment. This includes adapting the standard build phases (unpack, configure, build, check, install)[50].
+
+**Verification Goal 3.1:** Modify the `gnu-build-system` implementation to:
+
+- Correctly detect and use FreeBSD's build tools (clang instead of gcc if necessary, FreeBSD-specific make)
+- Adapt the configure phase to accept FreeBSD-specific configuration options
+- Ensure build flags and compiler options work on FreeBSD
+- Adapt the install phase to work with FreeBSD's filesystem conventions
+- Handle differences in library paths and naming conventions
+
+Successfully build five representative packages using the adapted `gnu-build-system` on FreeBSD, including at least one package that currently works on Linux Guix. The packages should include programs with runtime dependencies to verify that the build system correctly handles input and output dependencies.
+
+**Success Criteria:** Five standard GNU packages build successfully on FreeBSD using Guix. The built packages execute correctly and produce output identical to Linux-built versions where functionality is equivalent.
+
+### Intermediate Goal 3.2: Create FreeBSD-Specific Package Definitions
+
+Develop core package definitions for FreeBSD system software. These do not need to be comprehensive but should cover the minimum set needed for a functional system.
+
+**Verification Goal 3.2:** Create Guix package definitions for:
+
+- FreeBSD kernel and kernel headers
+- FreeBSD libc (libc rather than glibc)
+- FreeBSD userland utilities (coreutils alternatives)
+- Development tools (gcc/clang, make, autotools)
+- Minimum system libraries (openssl, zlib, etc.)
+- A basic shell (sh, bash)
+
+Each package definition should follow Guix's package definition conventions and specify correct inputs, build systems, and arguments for FreeBSD.
+
+**Success Criteria:** All core packages build successfully. Dependency relationships are correctly specified. Packages can be installed into profiles and are usable for building other packages.
+
+## Phase 4: Shepherd Init System Porting
+
+### Intermediate Goal 4.1: Port Shepherd to FreeBSD as a System Service
+
+Shepherd currently runs on Linux as an alternative to systemd. Adapt Shepherd to run on FreeBSD, initially as a regular service rather than as PID 1.
+
+**Verification Goal 4.1:** Successfully compile and run Shepherd on FreeBSD. Implement the basic service management capabilities:
+
+- Start and stop services through Shepherd's command interface
+- Manage service dependencies
+- Monitor service status
+- Gracefully handle service crashes and restarts
+- Execute services with appropriate privileges
+
+Create several test services (for example, a simple web server service, a logging service, a file monitoring service) and verify they can be managed through Shepherd. Services should be definable using Scheme expressions.
+
+**Success Criteria:** Shepherd compiles and runs on FreeBSD. Multiple services can be defined and managed simultaneously. Services can be started, stopped, and restarted through Shepherd's interface. Service dependencies work correctly (starting service A automatically starts service B if A depends on B).
+
+### Intermediate Goal 4.2: Adapt Shepherd for FreeBSD Init Integration
+
+Modify Shepherd to work as the system init process (PID 1) on FreeBSD. This requires handling system initialization, shutdown procedures, and signal management appropriate for FreeBSD.
+
+**Verification Goal 4.2:** Configure Shepherd as the system init process and successfully:
+
+- Boot a FreeBSD system with Shepherd as PID 1
+- Execute essential system initialization tasks
+- Manage essential system services (network, filesystem, logging)
+- Handle system shutdown signals gracefully
+- Execute defined services in the correct order
+- Handle service dependencies correctly during boot
+
+Create a minimal but complete Shepherd configuration that can boot a FreeBSD system to a usable login prompt.
+
+**Success Criteria:** A FreeBSD system successfully boots with Shepherd as PID 1. All system services come up in the correct order. The system remains stable and responsive. Shutdown procedures execute cleanly.
+
+### Intermediate Goal 4.3: Bridge Shepherd and FreeBSD Service Concepts
+
+Create mechanisms to integrate Shepherd services with FreeBSD's existing service concepts where beneficial, or clearly document where integration is unnecessary.
+
+**Verification Goal 4.3:** Demonstrate that Shepherd services can:
+
+- Provide equivalent functionality to FreeBSD's rcng services
+- Manage network interfaces and configuration
+- Manage user and group administration
+- Manage filesystem mounting and permissions

docs/PROMPT.md

@@ -0,0 +1,9 @@
+Your task is described in ./docs/PLAN.md. Current progress is (supposed to be) stored in ./docs/PROGRESS.md.
+
+Perform the next step towards the final goal. Git commit and update the progress file afterwards.
+
+You can use `sudo` freely, install missing software via `sudo pkg install`. If you still miss helpful tooling or other resources, stop and ask the operator, don't work around it.
+
+Guix sources are in ~/repos/guix. FreeBSD sources are installed in /usr/src. Clone other helpful repos to ~/repos/ as required.
+
+Good luck!