forked from tribes/guix
self
83b0e7d445
Substituter progress events (@ download-started, @ download-progress,
@ download-succeeded and friends) used to flow from the substituter
subprocess to the client unframed: the daemon forwarded each chunk of the
substituter's stderr verbatim, mixed with whatever else was being written
to its own standard error. The pipe-read size, not the record boundary,
decided where a chunk ended, and with several substitutions or builds in
flight the chunks could interleave at byte granularity. The client status
parser then saw torn or fused records and either crashed or fed nonsense
into the renderer.
Wrap the substituter's stderr the same way builder output is wrapped:
when multiplexedBuildOutput is enabled, prefix each chunk with
"@ build-log PID LENGTH\n" framing keyed on the substituter's pid. The
client already understands this framing for build logs; it now drives
substituter traces through the same path.
On the client side, carry partial trace records across consecutive frames
for the same PID instead of treating each frame's payload as a complete
unit. Only fragments that look like the head of a trace event ("@ "
prefix) are held back; free-form build log fragments are still flushed
eagerly so terminals see partial output in real time. Treat \r as a
record delimiter inside framed payloads to accommodate guix substitute,
which redraws in-line progress that way.
Drain pending substituter stderr before waking a substitution goal in
reaction to its fd 4 status reply, so trailing progress traces reach the
client ahead of the @ substituter-succeeded event the goal will emit
once its state machine advances. Recheck descriptor readiness in
Worker::waitForInput before each read, since the drain may have consumed
bytes the outer select had already flagged as ready.
* nix/libstore/build.cc (SubstitutionGoal::writeFramedToStderr): New
method.
(SubstitutionGoal::drainFromAgent): New method.
(SubstitutionGoal::handleChildOutput): Use writeFramedToStderr for
fromAgent output; drain before wake on builderOut status.
(Worker::waitForInput): Recheck FD readiness before reading.
* guix/status.scm (line-delimiter?): New procedure.
(split-lines): Split on \r as well as \n.
(build-event-output-port): Add %build-output-fragments, a per-PID
carry-over buffer for partial trace records straddling frames.
Rework process-build-output around it.
* tests/status.scm (frame): New helper.
("compute-status, substituter trace inside @ build-log frame")
("compute-status, trace record split across frames for same PID")
("compute-status, CR-delimited progress inside a frame")
("compute-status, partial lines do not cross PID boundaries"): New
tests.
("compute-status, multiplexed build output", "compute-status, build
completion"): Terminate the Hello frame's payload with \n to reflect
realistic builder output.