On Wed, 3 Sept 2025 at 21:10, Maarten Lankhorst <[email protected]> wrote: > > Hey, > > Den 2025-09-02 kl. 06:06, skrev Dave Airlie: > > From: Dave Airlie <[email protected]> > > > > This uses the newly introduced per-node gpu tracking stats, > > to track GPU memory allocated via TTM and reclaimable memory in > > the TTM page pools. > > > > These stats will be useful later for system information and > > later when mem cgroups are integrated. > > > > Cc: Christian Koenig <[email protected]> > > Cc: Matthew Brost <[email protected]> > > Cc: Johannes Weiner <[email protected]> > > Cc: [email protected] > > Cc: Andrew Morton <[email protected]> > > Signed-off-by: Dave Airlie <[email protected]> > > > > --- > > v2: add reclaim parameters and adjust the right counters. > > v3: drop the nid helper and get it from page. > > v4: use mod_lruvec_page_state (Shakeel) > > --- > > drivers/gpu/drm/ttm/ttm_pool.c | 21 +++++++++++++++------ > > 1 file changed, 15 insertions(+), 6 deletions(-) > > > > diff --git a/drivers/gpu/drm/ttm/ttm_pool.c b/drivers/gpu/drm/ttm/ttm_pool.c > > index baf27c70a419..148c7530738d 100644 > > --- a/drivers/gpu/drm/ttm/ttm_pool.c > > +++ b/drivers/gpu/drm/ttm/ttm_pool.c > > @@ -150,8 +150,10 @@ static struct page *ttm_pool_alloc_page(struct > > ttm_pool *pool, gfp_t gfp_flags, > > > > if (!pool->use_dma_alloc) { > > p = alloc_pages_node(pool->nid, gfp_flags, order); > > - if (p) > > + if (p) { > > p->private = order; > > + mod_lruvec_page_state(p, NR_GPU_ACTIVE, 1 << order); > > + } > > return p; > > } > > > > @@ -186,7 +188,7 @@ static struct page *ttm_pool_alloc_page(struct ttm_pool > > *pool, gfp_t gfp_flags, > > > > /* Reset the caching and pages of size 1 << order */ > > static void ttm_pool_free_page(struct ttm_pool *pool, enum ttm_caching > > caching, > > - unsigned int order, struct page *p) > > + unsigned int order, struct page *p, bool > > reclaim) > > { > > unsigned long attr = DMA_ATTR_FORCE_CONTIGUOUS; > > struct ttm_pool_dma *dma; > > @@ -201,6 +203,8 @@ static void ttm_pool_free_page(struct ttm_pool *pool, > > enum ttm_caching caching, > > #endif > > > > if (!pool || !pool->use_dma_alloc) { > > + mod_lruvec_page_state(p, reclaim ? NR_GPU_RECLAIM : > > NR_GPU_ACTIVE, > > + -(1 << order)); > > __free_pages(p, order); > > return; > > } > > @@ -288,6 +292,9 @@ static void ttm_pool_type_give(struct ttm_pool_type > > *pt, struct page *p) > > list_add(&p->lru, &pt->pages); > > spin_unlock(&pt->lock); > > atomic_long_add(1 << pt->order, &allocated_pages); > > + > > + mod_lruvec_page_state(p, NR_GPU_ACTIVE, -num_pages); > > + mod_lruvec_page_state(p, NR_GPU_RECLAIM, num_pages); > > } > > > > /* Take pages from a specific pool_type, return NULL when nothing > > available */ > > @@ -299,6 +306,8 @@ static struct page *ttm_pool_type_take(struct > > ttm_pool_type *pt) > > p = list_first_entry_or_null(&pt->pages, typeof(*p), lru); > > if (p) { > > atomic_long_sub(1 << pt->order, &allocated_pages); > > + mod_lruvec_page_state(p, NR_GPU_ACTIVE, (1 << pt->order)); > > + mod_lruvec_page_state(p, NR_GPU_RECLAIM, -(1 << pt->order)); > > list_del(&p->lru); > > } > > spin_unlock(&pt->lock); > > @@ -331,7 +340,7 @@ static void ttm_pool_type_fini(struct ttm_pool_type *pt) > > spin_unlock(&shrinker_lock); > > > > while ((p = ttm_pool_type_take(pt))) > > - ttm_pool_free_page(pt->pool, pt->caching, pt->order, p); > > + ttm_pool_free_page(pt->pool, pt->caching, pt->order, p, true); > > } > > > > /* Return the pool_type to use for the given caching and order */ > > @@ -383,7 +392,7 @@ static unsigned int ttm_pool_shrink(void) > > > > p = ttm_pool_type_take(pt); > > if (p) { > > - ttm_pool_free_page(pt->pool, pt->caching, pt->order, p); > > + ttm_pool_free_page(pt->pool, pt->caching, pt->order, p, true); > > num_pages = 1 << pt->order; > > } else { > > num_pages = 0; > > @@ -475,7 +484,7 @@ static pgoff_t ttm_pool_unmap_and_free(struct ttm_pool > > *pool, struct page *page, > > if (pt) > > ttm_pool_type_give(pt, page); > > else > > - ttm_pool_free_page(pool, caching, order, page); > > + ttm_pool_free_page(pool, caching, order, page, false); > > > > return nr; > > } > > @@ -780,7 +789,7 @@ static int __ttm_pool_alloc(struct ttm_pool *pool, > > struct ttm_tt *tt, > > return 0; > > > > error_free_page: > > - ttm_pool_free_page(pool, page_caching, order, p); > > + ttm_pool_free_page(pool, page_caching, order, p, false); > > > > error_free_all: > > if (tt->restore) > > Looking at the patch, there are 2 users of ttm_pool_free_page, one frees an > active page (various error paths), the other during page reclaim at pool > teardown and shrinking. > > So perhaps split into ttm_pool_free_active_page() and ttm_pool_reclaim_page()?
They would be 90% the same function though, with just hitting a different mod page state path, and I also change this later to use the cgroup mechanisms instead, so I'm not sure it would be effective. Dave. > > Kind regards, > ~Maarten Lankhorst
