tribes/guix
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/guix.git synced 2026-04-10 07:00:37 +02:00
Code Activity

gnu: llama-cpp: Use OpenBLAS.

Browse Source 
For faster prompt processing, OpenBLAS is recommended by
https://github.com/ggerganov/llama.cpp

* gnu/packages/machine-learning.scm (llama-cpp)[arguments]: Add
 #:configure-flags.
[native-inputs]: Add pkg-config.
[propagated-inputs]: Add openblas.

Change-Id: Iaf6f22252da13e2d6f503992878b35b0da7de0aa
Signed-off-by: Christopher Baines <mail@cbaines.net>
This commit is contained in:
John Fremlin
2024-04-03 23:46:25 -04:00
committed by Christopher Baines
parent 227dee63b0
commit d8a63bbcee
1 changed files with 4 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
gnu/packages/machine-learning.scm
View File

@@ -541,6 +541,8 @@ Performance is achieved by using the LLVM JIT compiler.")
(build-system cmake-build-system)
(arguments
(list
#:configure-flags
'(list "-DLLAMA_BLAS=ON" "-DLLAMA_BLAS_VENDOR=OpenBLAS")
#:modules '((ice-9 textual-ports)
(guix build utils)
((guix build python-build-system) #:prefix python:)
@@ -575,8 +577,9 @@ Performance is achieved by using the LLVM JIT compiler.")
(lambda _
(copy-file "bin/main" (string-append #$output "/bin/llama")))))))
(inputs (list python))
(native-inputs (list pkg-config))
(propagated-inputs
(list python-numpy python-pytorch python-sentencepiece))
(list python-numpy python-pytorch python-sentencepiece openblas))
(home-page "https://github.com/ggerganov/llama.cpp")
(synopsis "Port of Facebook's LLaMA model in C/C++")
(description "This package provides a port to Facebook's LLaMA collection