https://bugs.kde.org/show_bug.cgi?id=499563
Bug ID: 499563
Summary: Kdenlive mask generation plugin crashes when using
CUDA
Classification: Applications
Product: kdenlive
Version: git-master
Platform: Flatpak
OS: Linux
Status: REPORTED
Severity: normal
Priority: NOR
Component: Video Effects & Transitions
Assignee: j...@kdenlive.org
Reporter: paul.br...@kde.org
Target Milestone: ---
SUMMARY
I have a 4060 TI Nvidia GPU card I am using to test Kdenlive's AI mask
rendering plugin. The 4060 has 8GBs of RAM.
The plugin detects the card correctly and CUDA works well on Krita and Blender.
On Kdenlive, however, the mask generating process crashes due to "lack of
memory" trying to allocate 48 MBs.
STEPS TO REPRODUCE
1. Set card as Device in Configure > Plugins > Object Detextion
2. Use a clip with a moving object in the foreground
3. Select object with a rectangle around it
4. When the masked object is highlighted, click "generate mask"
OBSERVED RESULT
The processes crashes with the following error(s):
---
Traceback (most recent call last):
File "/app/share/kdenlive/scripts/automask/sam-objectmask.py", line 209, in
<module>
for out_frame_idx, out_obj_ids, out_mask_logits in
predictor.propagate_in_video(inference_state):
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/torch/utils/_contextlib.py",
line 57, in generator_context
response = gen.send(request)
^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/sam2/sam2_video_predictor.py",
line 603, in propagate_in_video
current_out, pred_masks = self._run_single_frame_inference(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/sam2/sam2_video_predictor.py",
line 758, in _run_single_frame_inference
) = self._get_image_feature(inference_state, frame_idx, batch_size)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/sam2/sam2_video_predictor.py",
line 714, in _get_image_feature
backbone_out = self.forward_image(image)
^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/sam2/modeling/sam2_base.py",
line 469, in forward_image
backbone_out = self.image_encoder(img_batch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/torch/nn/modules/module.py",
line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/torch/nn/modules/module.py",
line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/sam2/modeling/backbones/image_encoder.py",
line 31, in forward
features, pos = self.neck(self.trunk(sample))
^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/torch/nn/modules/module.py",
line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/torch/nn/modules/module.py",
line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/sam2/modeling/backbones/hieradet.py",
line 292, in forward
x = blk(x)
^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/torch/nn/modules/module.py",
line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/torch/nn/modules/module.py",
line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/sam2/modeling/backbones/hieradet.py",
line 165, in forward
x = x + self.drop_path(self.mlp(self.norm2(x)))
^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/torch/nn/modules/module.py",
line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/torch/nn/modules/module.py",
line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/sam2/modeling/sam2_utils.py",
line 133, in forward
x = self.act(layer(x)) if i < self.num_layers - 1 else layer(x)
^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/torch/nn/modules/module.py",
line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/torch/nn/modules/module.py",
line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/paul/.var/app/org.kde.kdenlive/data/kdenlive/venv-sam/lib/python3.11/site-packages/torch/nn/modules/activation.py",
line 734, in forward
return F.gelu(input, approximate=self.approximate)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 48.00 MiB. GPU 0
has a total capacity of 7.63 GiB of which 44.06 MiB is free. Process 105157 has
7.57 GiB memory in use. Of the allocated memory 7.26 GiB is allocated by
PyTorch, and 155.32 MiB is reserved by PyTorch but unallocated. If reserved but
unallocated memory is large try setting
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See
documentation for Memory Management
(https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
---
EXPECTED RESULT
The masked object should be generated for all frames
SOFTWARE/OS VERSIONS
Operating System: Arch Linux
KDE Plasma Version: 6.2.5
KDE Frameworks Version: 6.10.0
Qt Version: 6.8.2
Kernel Version: 6.13.1-arch1-1 (64-bit)
Graphics Platform: Wayland
--
You are receiving this mail because:
You are watching all bug changes.
