On 2025/01/31 20:31, Chris Cappuccio wrote: > This might be a way to do it. Does anyone have a card to test against?
No idea, descriptions for the vulkan-related ports don't give away much information about what's supported or how to use them. Had a thought earlier, it would be quite interesting to have llama.cpp use pledge (with a no-network set of promises). Looking through nm -s it seems like it may well be possible. (Not sure if supporting vulkan would affect that, i.e. whether it needs anything that simply wouldn't be allowed by any pledge). > Index: Makefile > =================================================================== > RCS file: /cvs/ports/misc/llama.cpp/Makefile,v > retrieving revision 1.1 > diff -u -p -u -r1.1 Makefile > --- Makefile 30 Jan 2025 22:55:11 -0000 1.1 > +++ Makefile 1 Feb 2025 04:20:02 -0000 > @@ -10,6 +10,7 @@ SHARED_LIBS += ggml-cpu 0.0 > SHARED_LIBS += ggml 0.0 > SHARED_LIBS += llama 0.0 > SHARED_LIBS += llava_shared 0.0 > +SHARED_LIBS += ggml-vulkan 0.0 > > CATEGORIES = misc > > @@ -18,11 +19,15 @@ HOMEPAGE = https://github.com/ggerganov > # MIT > PERMIT_PACKAGE = Yes > > -WANTLIB += m pthread ${COMPILER_LIBCXX} > +WANTLIB += m pthread vulkan ${COMPILER_LIBCXX} > > MODULES = devel/cmake > > +LIB_DEPENDS = graphics/vulkan-loader > +BUILD_DEPENDS = graphics/shaderc > + > CONFIGURE_ARGS = -DGGML_CCACHE=Off \ > - -DGGML_NATIVE=Off > + -DGGML_NATIVE=Off \ > + -DGGML_VULKAN=On > > .include <bsd.port.mk> > Index: pkg/PLIST > =================================================================== > RCS file: /cvs/ports/misc/llama.cpp/pkg/PLIST,v > retrieving revision 1.1 > diff -u -p -u -r1.1 PLIST > --- pkg/PLIST 30 Jan 2025 22:55:11 -0000 1.1 > +++ pkg/PLIST 1 Feb 2025 04:20:02 -0000 > @@ -58,6 +58,7 @@ bin/convert_hf_to_gguf.py > @bin bin/test-tokenizer-0 > @bin bin/test-tokenizer-1-bpe > @bin bin/test-tokenizer-1-spm > +@bin bin/vulkan-shaders-gen > include/ggml-alloc.h > include/ggml-backend.h > include/ggml-blas.h > @@ -74,7 +75,6 @@ include/ggml.h > include/gguf.h > include/llama-cpp.h > include/llama.h > -lib/cmake/ > lib/cmake/ggml/ > lib/cmake/ggml/ggml-config.cmake > lib/cmake/ggml/ggml-version.cmake > @@ -83,6 +83,7 @@ lib/cmake/llama/llama-config.cmake > lib/cmake/llama/llama-version.cmake > @lib lib/libggml-base.so.${LIBggml-base_VERSION} > @lib lib/libggml-cpu.so.${LIBggml-cpu_VERSION} > +@lib lib/libggml-vulkan.so.${LIBggml-vulkan_VERSION} > @lib lib/libggml.so.${LIBggml_VERSION} > @lib lib/libllama.so.${LIBllama_VERSION} > @lib lib/libllava_shared.so.${LIBllava_shared_VERSION} >
