Tested following intel recommandation to run Qwen/Qwen2-7B model in LNL based
on
https://docs.openvino.ai/2024/learn-openvino/llm_inference_guide/genai-guide-npu.html
using latest Linux NPU driver -
https://github.com/intel/linux-npu-driver/releases/tag/v1.10.1
:
$ uname -a
Linux tucano-806928 6.8.0-57-generic
$ sudo -E npu-env/bin/python
>>> import openvino_genai as ov_genai
>>> model_path="Llama-2-7B-Chat-GPTQ"
>>> pipe = ov_genai.LLMPipeline(model_path, "NPU")
>>> print(pipe.generate("The Sun is yellow because", max_new_tokens=100))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
RuntimeError: Exception from src/inference/src/cpp/infer_request.cpp:223:
Exception from
src/plugins/intel_npu/src/plugin/npuw/just_sync_infer_request.cpp:659:
Failed to compile. No more devices are left!
======
$ uname -a
Linux tucano-806928 6.8.0-58-generic #60-Ubuntu
$ sudo -E npu-env/bin/python
Python 3.12.3 (main, Feb 4 2025, 14:48:35) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import openvino_genai as ov_genai
>>> pipe = ov_genai.LLMPipeline("Llama-2-7B-Chat-GPTQ", "NPU")
>>> print(pipe.generate("The Sun is yellow because", max_new_tokens=100))
of the way it absorbs and scatters light.
The Sun appears yellow to us because of the way it absorbs and scatters light.
When light from the Sun hits an object, some of it is absorbed and some of it
is scattered in all directions. The amount of light that is scattered depends
on the color of the object and the wavelength of the light.
The Sun absorbs light in the yellow and orange parts of the spectrum, which is
why it
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/2098972
Title:
Enable Large Language Model (LLM) workloads using Intel NPU
Status in linux package in Ubuntu:
Fix Released
Status in linux source package in Noble:
Fix Committed
Status in linux source package in Oracular:
Fix Committed
Bug description:
[ Impact ]
* Enable larger LLM using intel NPU.
[ Fix ]
* Upstream commit:
83b6fa5844b5 accel/ivpu: Increase DMA address range
[ Test Plan ]
* Built and boot tested
[ Where problems could occur ]
* Memory management issue when using intel NPU.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2098972/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
