Parallelize file linting with n-par-map

Use Guile's n-par-map from (ice-9 threads) to lint files in parallel,
one thread per CPU core. Results are collected per-file then output
sequentially to maintain deterministic ordering.

Performance on Guix tree (1336 files, 12 cores):

  Before (sequential)     After (parallel)
  full:    21s             full:    14s   (1.5x)
  surface: 15s             surface:  8s   (1.9x)
  line:    14s             line:     7s   (2.0x)
This commit is contained in:
2026-04-04 13:51:25 +02:00
parent 1734ebe381
commit f5d5919943

View File

@@ -7,6 +7,7 @@
(define-module (gulie engine)
#:use-module (ice-9 ftw)
#:use-module (ice-9 textual-ports)
#:use-module (ice-9 threads)
#:use-module (srfi srfi-1)
#:use-module (gulie diagnostic)
#:use-module (gulie rule)
@@ -127,14 +128,22 @@ and '%min-severity with value 'error, 'warning, or 'info (default)."
(sort diagnostics diagnostic<?)))
(define (lint-files files config)
"Lint multiple FILES. Returns total diagnostic count."
(let ((total 0))
"Lint multiple FILES in parallel. Returns total diagnostic count.
Uses n-par-map to distribute work across threads, then outputs
diagnostics sequentially to maintain deterministic file order."
(let* ((ncpus (max 1 (total-processor-count)))
(results (n-par-map ncpus
(lambda (file)
(cons file (lint-file file config)))
files))
(total 0))
;; Output in original file order (n-par-map preserves it)
(for-each
(lambda (file)
(let ((diags (lint-file file config)))
(lambda (result)
(let ((diags (cdr result)))
(set! total (+ total (length diags)))
(format-diagnostics diags (current-output-port))))
files)
results)
total))
(define (scheme-file? path)