On Mon, Mar 24, 2025 at 5:07 PM Eric Huang <[email protected]> wrote: > > > On 2025-03-24 15:32, Alex Deucher wrote: > > On Mon, Mar 24, 2025 at 1:26 PM Eric Huang <[email protected]> wrote: > >> kfdtest KFDQMTest.OverSubscribeCpQueues with multiple > >> gpu mode fails on gfx v9.4.3+NPS4+CPX which has 64 gpu > >> nodes, the queues created are 65x64=4160, but the number > >> 1024 0f KFD_MAX_NUM_OF_QUEUES_PER_PROCESS is not enough > >> and test fails at function find_available_queue_slot(). > >> So increasing the nubmer will make the test passed. > >> > >> Signed-off-by: Eric Huang <[email protected]> > >> --- > >> drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 2 +- > >> 1 file changed, 1 insertion(+), 1 deletion(-) > >> > >> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h > >> b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h > >> index f6aedf69c644..054a78207ffe 100644 > >> --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h > >> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h > >> @@ -94,7 +94,7 @@ > >> ((typeof(ptr_to_struct)) kzalloc(sizeof(*ptr_to_struct), > >> GFP_KERNEL)) > >> > >> #define KFD_MAX_NUM_OF_PROCESSES 512 > >> -#define KFD_MAX_NUM_OF_QUEUES_PER_PROCESS 1024 > >> +#define KFD_MAX_NUM_OF_QUEUES_PER_PROCESS 4160 > > Doesn't this limit have more to do with the number of doorbells you > > can fit into a 4K page? If you only allocate 4K for doorbells how can > > you increase this? > > The doorbells size is allocated dynamically as multiple pages based on > KFD_MAX_NUM_OF_QUEUES_PER_PROCESS in KFD. Currently with 1024 of this > macro 2 pages are allocated, and after changing to 4160, 9 pages will be > allocated. Please refer in function kfd_allocate_process_doorbells().
Thanks for the details. Since most apps don't use that many, it seems like a waste of doorbells. Should this be limited to certain partition modes? Alex > > Thanks, > Eric > > > > > Alex > > > >> /* > >> * Size of the per-process TBA+TMA buffer: 2 pages > >> -- > >> 2.34.1 > >> >
