Current implementation printed metadata supposed to be present just once per
.trs file on the end of each test group. According to the automake's manual
that is undefined behavior. This commit fixes it by printing that metadata
just once, after all tests did run.
Since there is no built-in hook that could be used for
that (test-runner-on-final runs on *each* outermost test-end), I introduced
new `finalize' procedure that need to be called by the user. Possibly not the
most elegant solution, but since we are the only user, it works fine and
produces actually valid .trs file.
That also means there is no longer any use for test-runner-on-test-end!.
* build-aux/test-driver.scm (test-runner-gnu): Define new procedure `finalize'
and return it together with the runner. Do not call
test-runner-on-group-end!.
(main): Call the `finalize' after all tests are done.
Signed-off-by: Maxim Cournoyer <maxim@guixotic.coop>
Guile 3.0.11 ships with a new SRFI 64.
* build-aux/test-driver.scm (%test-match-all): Delete procedure.
(main): Replace %test-match-all with test-match-all.
Change-Id: Ibcd7d98194694c1de679491be4f5a603a79f6e29
I'm assuming this doesn't have any meaningful effect, and it seems to reduce
the number of readlink calls by a lot.
* build-aux/build-self.scm (build-program): Set
%file-port-name-canonicalization to #f.
(build): Ditto.
* build-aux/compile-as-derivation.scm: Ditto.
Change-Id: If43415c46c3911e84c76d1a9828b9c417a140a1b
To avoid one character per syscall output from within the build procedure.
* build-aux/compile-as-derivation.scm: Buffer the current-error-port.
Change-Id: I7725ef0cb1f3ad4e9f5fdd284734f376fd57b253
In the case where port->connection is used, since this doesn't setup
buffering.
* build-aux/build-self.scm (build-program): Setup buffering for the store
socket.
Change-Id: I822bb628e92d1070e78a2ad9e95665ca2ff4a351
There is no reason this should be syntax.
* build-aux/update-guix-package.scm (with-input-pipe-to-string): Make it a
procedure.
Change-Id: I96daeb40eb9202c542a7906926f2e25765d5078f
With srfi-64 from guile 3.0.11 the test name is no longer in the
test-result-alist.
* build-aux/test-driver.scm (test-runner-gnu): Use test-runner-test-name to
access test-name.
Change-Id: I53d0fdd8db0d1af5e636e3f2a68280cd2bddfe4c
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Merges: #5836
Signed-off-by: Rutherther <rutherther@ditigal.xyz>
The check script has been failing, %final-inputs have been changed
to a procedure that takes the system.
* build-aux/check-final-inputs-self-contained.scm
(final-inputs): Call %final-inputs procedure with system.
Change-Id: Id4d40387e669c996a380f64c73432d916915ead5
Signed-off-by: Rutherther <rutherther@ditigal.xyz>
To save a lot of disk space, keep only actually translated messages in
Gettext PO files. Ignore the guix domain, which is more complicated
and is tiny.
* build-aux/keep-only-translated.scm: New file.
* Makefile.am (download-po): Run it.
Change-Id: I6442ce0ef8d62f7e48e667c766b86d0ebf9c5415
This reverts commit a7db92d9b3, this time with
the more careful command, to avoid a world rebuild:
grep -rl --exclude-dir=build --exclude-dir=patches --exclude=ld-wrapper.in \
--exclude=sitecustomize.py --exclude=guix-emacs.el maxim.cournoyer@gmail.com |
xargs sed -i 's/maxim.cournoyer@gmail.com/maxim@guixotic.coop/g'
Change-Id: I0871f5cf9ace79e34e78e24154d0da6ea95a3a91
The change was automated via:
git grep -l maxim.cournoyer@gmail.com |
xargs sed -i 's/maxim.cournoyer@gmail.com/maxim@guixotic.coop/g'
* .mailmap: New entry.
Change-Id: Iaa88b1e51c94159c49138fb43910badb990eb169
* build-aux/build-self.scm (guile-gcrypt): Remove fallback case for when the
host Guix lacks ‘guile-gcrypt’.
Change-Id: I840333a72a45f8582bb26271d41e3fadf1c59f3e
The container that slirp4netns runs in should already be quite difficult to do
anything malicious in beyond basic denial of service or sending of network
traffic. There is, however, one hole remaining in the case in which there is
an adversary able to run code locally: abstract unix sockets. Because these
are governed by network namespaces, not IPC namespaces, and slirp4netns is in
the root network namespace, any process in the root network namespace can
cooperate with the slirp4netns process to take over its user.
To close this, we use seccomp to block the creation of unix-domain sockets by
slirp4netns. This requires some finesse, since slirp4netns absolutely needs
to be able to create other types of sockets - at minimum AF_INET and AF_INET6
Seccomp has many, many pitfalls. To name a few:
1. Seccomp provides you with an "arch" field, but this does not uniquely
determine the ABI being used; the actual meaning of a system call number
depends on both the number (which is often the result of ORing a related
system call with a flag for an alternate ABI) and the architecture.
2. Seccomp provides no direct way of knowing what the native value for the
arch field should be; the user must do configure/compile-time testing for
every architecture+ABI combination they want to support. Amusingly enough,
the linux-internal header files have this exact information
(SECCOMP_ARCH_NATIVE), but they aren't sharing it.
3. The only system call numbers we naturally have are the native ones in
asm/unistd.h. __NR_socket will always refer to the system call number for
the target system's ABI.
4. Seccomp can only manipulate 32-bit words, but represents every system call
argument as a uint64.
5. New system call numbers with as-yet-unknown semantics can be added to the
kernel at any time.
6. Based on this comment in arch/x86/entry/syscalls/syscall_32.tbl:
# 251 is available for reuse (was briefly sys_set_zone_reclaim)
previously-invalid system call numbers may later be reused for new system
calls.
7. Most architecture+ABI combinations have system call tables with many gaps
in them. arm-eabi, for example, has 35 such gaps (note: this is just the
number of distinct gaps, not the number of system call numbers contained in
those gaps).
8. Seccomp's BPF filters require a fully-acyclic control flow graph.
Any operation on a data structure must therefore first be fully
unrolled before it can be run.
9. Seccomp cannot dereference pointers. Only the raw bits provided to the
system calls can be inspected.
10. Some architecture+ABI combos have multiplexer system calls. For example,
socketcall can perform any socket-related system call. The arguments to
the multiplexed system call are passed indirectly, via a pointer to user
memory. They therefore cannot be inspected by seccomp.
11. Some valid system calls are not listed in any table in the kernel source.
For example, __ARM_NR_cacheflush is an "ARM private" system call. It does
not appear in any *.tbl file.
12. Conditional branches are limited to relative jumps of at most 256
instructions forward.
13. Prior to Linux 4.8, any process able to spawn another process and call
ptrace could bypass seccomp restrictions.
To address (1), (2), and (3), we include preprocessor checks to identify the
native architecture value, and reject all system calls that don't use the
native architecture.
To address (4), we use the AC_C_BIGENDIAN autoconf check to conditionally
define WORDS_BIGENDIAN, and match up the proper portions of any uint64 we test
for with the value in the accumulator being tested against.
To address (5) and (6), we use system call pinning. That is, we hardcode a
snapshot of all the valid system call numbers at the time of writing, and
reject any system call numbers not in the recorded set. A set is recorded for
every architecture+ABI combo, and the native one is chosen at compile-time.
This ensures that not only are non-native architectures rejected, but so are
non-native ABIs. For the sake of conciseness, we represent these sets as sets
of disjoint ranges. Due to (7), checking each range in turn could add a lot
of overhead to each system call, so we instead binary search through the
ranges. Due to (8), this binary search has to be fully unrolled, so we do
that too.
It can be tedious and error-prone to manually produce the syscall ranges by
looking at linux's *.tbl files, since the gaps are often small and
uncommented. To address this, a script, build-aux/extract-syscall-ranges.sh,
is added that will produce them given a *.tbl filename and an ABI regex (some
tables seem to abuse the ABI field with strange values like "memfd_secret").
Note that producing the final values still requires looking at the proper
asm/unistd.h file to find any private numbers and to identify any offsets and
ABI variants used.
(10) used to have no good solution, but in the past decade most architectures
have gained dedicated system call alternatives to at least socketcall, so we
can (hopefully) just block it entirely.
To address (13), we block ptrace also.
* build-aux/extract-syscall-ranges.sh: new script.
* Makefile.am (EXTRA_DIST): register it.
* config-daemon.ac: use AC_C_BIGENDIAN.
* nix/libutil/spawn.cc (setNoNewPrivsAction, addSeccompFilterAction): new
functions.
* nix/libutil/spawn.hh (setNoNewPrivsAction, addSeccompFilterAction): new
declarations.
(SpawnContext)[setNoNewPrivs, addSeccompFilter]: new fields.
* nix/libutil/seccomp.hh: new header file.
* nix/libutil/seccomp.cc: new file.
* nix/local.mk (libutil_a_SOURCES, libutil_headers): register them.
* nix/libstore/build.cc (slirpSeccompFilter, writeSeccompFilterDot):
new functions.
(spawnSlirp4netns): use them, set seccomp filter for slirp4netns.
Change-Id: Ic92c7f564ab12596b87ed0801b22f88fbb543b95
Signed-off-by: John Kehayias <john.kehayias@protonmail.com>
* build-aux/test-env.in: Pass ‘--disable-chroot’ only when unprivileged
user namespace support is lacking and warn in that case.
* tests/store.scm ("build-things, check mode"): Use ‘gettimeofday’
rather than a shared file as a source of entropy.
("symlink is symlink")
("isolated environment", "inputs are read-only")
("inputs cannot be remounted read-write")
("build root cannot be made world-readable")
("/tmp, store, and /dev/{null,full} are writable")
("network is unreachable"): New tests.
* tests/processes.scm ("client + lock"): Skip when
‘unprivileged-user-namespace-supported?’ returns true.
Change-Id: I3b3c3ebdf6db5fd36ee70251d07b893c17ca1b84
With the weak isolation available to the test daemon, it is essential to
disallow untrusted access to it, as otherwise another local user can gain our
user's credentials easily.
* build-aux/test-env.in: ensure the daemon-socket directory is freshly-created
with 0700 permissions.
Change-Id: I742f70fc6fc28e5b4dc88d590eef3daf1b964670
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
When computing channel instance derivations.
This is useful when you want to generate compatible derivations that can be
run with a daemon that potentially doesn't support builtin builders that the
daemon you're using to generate the derivations has.
I'm looking at this in particular because I want to use this in the data
service, since it provides substitutes for derivations, and since these can be
built on other machines, it's useful to control which builtin builders they
depend on.
Fixes: <https://issues.guix.gnu.org/67250>.
* build-aux/build-self.scm (build-program): Accept
#:built-in-builders and pass along to port->connection or
open-connection as approriate.
(build): Accept and pass on #:built-in-builders.
* guix/channels.scm (build-from-source, build-channel-instance,
channel-instance-derivations, channel-instances->manifest,
channel-instances->derivation): Accept and pass on
#:built-in-builders.
Change-Id: I315c990de66c6f7dca25a859165a5568abe385ea
This replaces Automake's `build-aux/mdate-sh' with our own
`build-aux/mdate-from-git.scm' to use reproducible timestamps from Git
instead.
* build-aux/mdate-from-git.scm: New script.
* bootstrap: Use it to replace build-aux/mdate-sh.
* Makefile.am (EXTRA_DIST): Add it.
Change-Id: I17d0a7de9ffea397129c0db1728f86e28a4e245f
* build-aux/xgettext.scm: Move setting of environment variables to shell
header.
(main): Use SOURCE_DATE_EPOCH as fallback for timestamp. This fixes running
from a tarball.
* Makefile.am (EXTRA_DIST): Add it.
Change-Id: Ic487587b22495868fd2a21545a13dc9e3458299c
* build-aux/xgettext.scm: New script.
* po/guix/Makevars (XGETTEXT): Set it.
(XGETTEXT_OPTIONS): Add --xgettext option to `real' xgettext.
* po/packages/Makevars (XGETTEXT): Set it.
(XGETTEXT_OPTIONS): Add --xgettext option to `real' xgettext.
Change-Id: I71b6b843970090f765f46ac346b92a346560e3f0
Emacs auto-detects the build-aux/test-driver.scm as sh-mode, due to the
shebang. So override the auto-detection using a local variable forcing
scheme-mode.
* build-aux/test-driver.scm: Set mode to scheme.
Change-Id: I07f12656dba5fb7b52447b94569307a2a605ffc1
Signed-off-by: Maxim Cournoyer <maxim.cournoyer@gmail.com>
* build-aux/cuirass/hurd-manifest.scm: Include full build of python-minimal
and gettext-minimal (no longer without-tests). Add grub-minimal, grub,
guix-without-tests and guile-3.0 (no longer delete it from guix dependencies).
Works around <https://issues.guix.gnu.org/55441#12>.
Start from commit bd86bbd300,
'open-inferior' uses 'primitive-fork' instead of 'open-pipe*'. As a
result, child process could potentially hang before calling 'execl' due
to undefined behavior when forking a multi-threaded process.
* build-aux/cuirass/evaluate.scm <top level>: Call 'open-inferior'
before 'n-par-for-each'.
The third key will be used in an upcoming commit.
Rename public keys to .pub.
* guix/tests/gnupg.scm (%ed25519-3-public-key-file): New variable.
(%ed25519-3-secret-key-file): New variable.
(%ed25519-2-public-key-file): Renamed from %ed25519bis-public-key-file.
(%ed25519-2-secret-key-file): Renamed from %ed25519bis-secret-key-file.
* tests/keys/ed25519-3.key: New file.
* tests/keys/ed25519-3.sec: New file.
Signed-off-by: Mathieu Othacehe <othacehe@gnu.org>
This ensures we use the same method in "make" as in "guix/self.scm".
* Makefile.am: Build guix/build/po.scm.
* build-aux/convert-xref.scm: New file.
* doc/local.mk (xref_command): Use it.
* guix/self.scm (translate-cross-references): Move it...
* guix/build/po.scm: Parse comments and flags separately to find fuzzy
flags.
(translate-cross-references): ...here.
(parse-tree->assoc): Ignore fuzzy entries.
Fixes <https://bugs.gnu.org/48963>.
Reported by Julien Lepiller <julien@lepiller.eu>.
This reduces peak memory consumption to something less unreasonable.
* Makefile.am (make-go): Depend on 'make-*-go' targets; remove body.
(guile-compilation-rule): New function.
(MODULES_CORE, MODULES_PACKAGES, MODULES_SYSTEM, MODULES_CLI): New
variables.
<top level>: Call 'guile-compilation-rule' 4 times.
* build-aux/compile-all.scm <top level>: Expect "--total" and
"--processed". Take them into account when displaying progress
reports.
This is a fixed version of 66b14dccdd,
which was reverted in e74250c3c5.
* guix/download.scm (%disarchive-mirrors): New variable.
(%disarchive-mirror-file): New variable.
(built-in-download): Add 'disarchive-mirrors' keyword argument and
pass its value along to the 'builtin:download' derivation.
(url-fetch): Pass '%disarchive-mirror-file' to 'built-in-download'.
* guix/scripts/perform-download.scm (perform-download): Read
Disarchive mirrors from the environment and pass them to
'url-fetch'.
* guix/build/download.scm (disarchive-fetch/any): New procedure.
(url-fetch): Add 'disarchive-mirrors' keyword argument, use it to
make a list of URIs, and use the new procedure to fetch the file if
all other methods fail.
* build-aux/build-self.scm (build-program)[select?]: Exclude '(guix
build download)'.
* guix/self.scm (compiled-guix)[*core-modules*]: Add 'guile-json' to
the list of extensions.
This importer has suffered from bitrot and no longer works with current
Nix and Nixpkgs. See <https://bugs.gnu.org/32339> and
<https://bugs.gnu.org/36255>.
* guix/import/snix.scm, guix/scripts/import/nix.scm,
tests/snix.scm: Remove.
* Makefile.am (MODULES, SCM_TESTS): Remove them.
* guix/scripts/import.scm (importers): Remove "nix".
* build-aux/test-env.in: Remove NIXPKGS variable.
* configure.ac: Remove '--with-nixpkgs' option.
* doc/guix.texi (Invoking guix import): Remove bit about "guix import
nix".
* etc/completion/fish/guix.fish: Likewise.
A number of packages doesn't really make sense in the name of the section to
be substituted. This change allows using simply '*** new packages' instead of
'*** 1999 new packages', for example, and have the update-NEWS.scm script
update it.
* build-aux/update-NEWS.scm (write-packages-added) <regexp>: Do not care about
leading white space in the name of the section.
This importer has suffered from bitrot and no longer works with current
Nix and Nixpkgs. See <https://bugs.gnu.org/32339> and
<https://bugs.gnu.org/36255>.
* guix/import/snix.scm, guix/scripts/import/nix.scm,
tests/snix.scm: Remove.
* Makefile.am (MODULES, SCM_TESTS): Remove them.
* guix/scripts/import.scm (importers): Remove "nix".
* build-aux/test-env.in: Remove NIXPKGS variable.
* configure.ac: Remove '--with-nixpkgs' option.
* doc/guix.texi (Invoking guix import): Remove bit about "guix import
nix".
* etc/completion/fish/guix.fish: Likewise.
This simplifies code and mostly ensures we don't print a spinner while
there's build activity going on.
* build-aux/build-self.scm (build-program): Remove 'spin' and
'call-with-new-thread' call from "compute-guix-derivation" body. Remove
"Computing Guix derivation" message.
(proxy): Pass extra argument to 'select'. Display a spinner when
'select' returns empty lists.
(build): Print "Computing Guix derivation" message here.
Fixes <https://bugs.gnu.org/41930>.
* build-aux/build-self.scm (build-program): Add extra 'build-output'
parameter. Interpret it as a socket name and connect to it; use it as
the CURRENT-BUILD-OUTPUT-PORT.
(proxy): New procedure.
(build): Open a named socket. Accept connections and call 'proxy' on it.
In particular, silence the Guile-Git autoload warnings introduced by
c1940fde43.
* build-aux/build-self.scm (build-program): Wrap 'guix-derivation' call
in 'parameterize'.
