Re: [PATCH v3] drm/doc: add rfc section for small BAR uapi
On 16/05/2022 19:11, Matthew Auld wrote: Add an entry for the new uapi needed for small BAR on DG2+. v2: - Some spelling fixes and other small tweaks. (Akeem & Thomas) - Rework error capture interactions, including no longer needing NEEDS_CPU_ACCESS for objects marked for capture. (Thomas) - Add probed_cpu_visible_size. (Lionel) v3: - Drop the vma query for now. - Add unallocated_cpu_visible_size as part of the region query. - Improve the docs some more, including documenting the expected behaviour on older kernels, since this came up in some offline discussion. Signed-off-by: Matthew Auld Cc: Thomas Hellström Cc: Lionel Landwerlin Cc: Tvrtko Ursulin Cc: Jon Bloomfield Cc: Daniel Vetter Cc: Jon Bloomfield Cc: Jordan Justen Cc: Kenneth Graunke Cc: Akeem G Abodunrin Cc: mesa-dev@lists.freedesktop.org --- Documentation/gpu/rfc/i915_small_bar.h | 164 +++ Documentation/gpu/rfc/i915_small_bar.rst | 47 +++ Documentation/gpu/rfc/index.rst | 4 + 3 files changed, 215 insertions(+) create mode 100644 Documentation/gpu/rfc/i915_small_bar.h create mode 100644 Documentation/gpu/rfc/i915_small_bar.rst diff --git a/Documentation/gpu/rfc/i915_small_bar.h b/Documentation/gpu/rfc/i915_small_bar.h new file mode 100644 index ..4079d287750b --- /dev/null +++ b/Documentation/gpu/rfc/i915_small_bar.h @@ -0,0 +1,164 @@ +/** + * struct __drm_i915_memory_region_info - Describes one region as known to the + * driver. + * + * Note this is using both struct drm_i915_query_item and struct drm_i915_query. + * For this new query we are adding the new query id DRM_I915_QUERY_MEMORY_REGIONS + * at &drm_i915_query_item.query_id. + */ +struct __drm_i915_memory_region_info { + /** @region: The class:instance pair encoding */ + struct drm_i915_gem_memory_class_instance region; + + /** @rsvd0: MBZ */ + __u32 rsvd0; + + /** @probed_size: Memory probed by the driver (-1 = unknown) */ + __u64 probed_size; Is -1 possible today or when it will be? For system memory it appears zeroes are returned today so that has to stay I think. Does it effectively mean userspace has to consider both 0 and -1 as unknown is the question. + + /** +* @unallocated_size: Estimate of memory remaining (-1 = unknown) +* +* Note this is only currently tracked for I915_MEMORY_CLASS_DEVICE +* regions, and also requires CAP_PERFMON or CAP_SYS_ADMIN to get +* reliable accounting. Without this(or if this an older kernel) the s/if this an/if this is an/ Also same question as above about -1. +* value here will always match the @probed_size. +*/ + __u64 unallocated_size; + + union { + /** @rsvd1: MBZ */ + __u64 rsvd1[8]; + struct { + /** +* @probed_cpu_visible_size: Memory probed by the driver +* that is CPU accessible. (-1 = unknown). Also question about -1. In this case this could be done since the field is yet to be added but I am curious if it ever can be -1. +* +* This will be always be <= @probed_size, and the +* remainder(if there is any) will not be CPU +* accessible. +* +* On systems without small BAR, the @probed_size will +* always equal the @probed_cpu_visible_size, since all +* of it will be CPU accessible. +* +* Note that if the value returned here is zero, then +* this must be an old kernel which lacks the relevant +* small-bar uAPI support(including I have noticed you prefer no space before parentheses throughout the text so I guess it's just my preference to have it. Very nitpicky even if I am right so up to you. +* I915_GEM_CREATE_EXT_FLAG_NEEDS_CPU_ACCESS), but on +* such systems we should never actually end up with a +* small BAR configuration, assuming we are able to load +* the kernel module. Hence it should be safe to treat +* this the same as when @probed_cpu_visible_size == +* @probed_size. +*/ + __u64 probed_cpu_visible_size; + + /** +* @unallocated_cpu_visible_size: Estimate of CPU +* visible memory remaining (-1 = unknown). +* +* Note this is only currently tracked for +* I915_MEMORY_CLASS_DEVICE regions, and also requires +* CAP_PERFMON or CAP_SYS_ADMIN to get reliab
Re: [PATCH v3] drm/doc: add rfc section for small BAR uapi
On 17/05/2022 11:29, Tvrtko Ursulin wrote: On 16/05/2022 19:11, Matthew Auld wrote: Add an entry for the new uapi needed for small BAR on DG2+. v2: - Some spelling fixes and other small tweaks. (Akeem & Thomas) - Rework error capture interactions, including no longer needing NEEDS_CPU_ACCESS for objects marked for capture. (Thomas) - Add probed_cpu_visible_size. (Lionel) v3: - Drop the vma query for now. - Add unallocated_cpu_visible_size as part of the region query. - Improve the docs some more, including documenting the expected behaviour on older kernels, since this came up in some offline discussion. Signed-off-by: Matthew Auld Cc: Thomas Hellström Cc: Lionel Landwerlin Cc: Tvrtko Ursulin Cc: Jon Bloomfield Cc: Daniel Vetter Cc: Jon Bloomfield Cc: Jordan Justen Cc: Kenneth Graunke Cc: Akeem G Abodunrin Cc: mesa-dev@lists.freedesktop.org --- Documentation/gpu/rfc/i915_small_bar.h | 164 +++ Documentation/gpu/rfc/i915_small_bar.rst | 47 +++ Documentation/gpu/rfc/index.rst | 4 + 3 files changed, 215 insertions(+) create mode 100644 Documentation/gpu/rfc/i915_small_bar.h create mode 100644 Documentation/gpu/rfc/i915_small_bar.rst diff --git a/Documentation/gpu/rfc/i915_small_bar.h b/Documentation/gpu/rfc/i915_small_bar.h new file mode 100644 index ..4079d287750b --- /dev/null +++ b/Documentation/gpu/rfc/i915_small_bar.h @@ -0,0 +1,164 @@ +/** + * struct __drm_i915_memory_region_info - Describes one region as known to the + * driver. + * + * Note this is using both struct drm_i915_query_item and struct drm_i915_query. + * For this new query we are adding the new query id DRM_I915_QUERY_MEMORY_REGIONS + * at &drm_i915_query_item.query_id. + */ +struct __drm_i915_memory_region_info { + /** @region: The class:instance pair encoding */ + struct drm_i915_gem_memory_class_instance region; + + /** @rsvd0: MBZ */ + __u32 rsvd0; + + /** @probed_size: Memory probed by the driver (-1 = unknown) */ + __u64 probed_size; Is -1 possible today or when it will be? For system memory it appears zeroes are returned today so that has to stay I think. Does it effectively mean userspace has to consider both 0 and -1 as unknown is the question. I raised this on v2. As far as I can tell there are no situation where we would get -1. Is it really probed_size=0 on smem?? It's not the case on the internal branch. Anv is not currently handling that case. I would very much like to not deal with 0 for smem. It really makes it easier for userspace rather than having to fish information from 2 different places and on top of dealing with multiple kernel versions. -Lionel + + /** + * @unallocated_size: Estimate of memory remaining (-1 = unknown) + * + * Note this is only currently tracked for I915_MEMORY_CLASS_DEVICE + * regions, and also requires CAP_PERFMON or CAP_SYS_ADMIN to get + * reliable accounting. Without this(or if this an older kernel) the s/if this an/if this is an/ Also same question as above about -1. + * value here will always match the @probed_size. + */ + __u64 unallocated_size; + + union { + /** @rsvd1: MBZ */ + __u64 rsvd1[8]; + struct { + /** + * @probed_cpu_visible_size: Memory probed by the driver + * that is CPU accessible. (-1 = unknown). Also question about -1. In this case this could be done since the field is yet to be added but I am curious if it ever can be -1. + * + * This will be always be <= @probed_size, and the + * remainder(if there is any) will not be CPU + * accessible. + * + * On systems without small BAR, the @probed_size will + * always equal the @probed_cpu_visible_size, since all + * of it will be CPU accessible. + * + * Note that if the value returned here is zero, then + * this must be an old kernel which lacks the relevant + * small-bar uAPI support(including I have noticed you prefer no space before parentheses throughout the text so I guess it's just my preference to have it. Very nitpicky even if I am right so up to you. + * I915_GEM_CREATE_EXT_FLAG_NEEDS_CPU_ACCESS), but on + * such systems we should never actually end up with a + * small BAR configuration, assuming we are able to load + * the kernel module. Hence it should be safe to treat + * this the same as when @probed_cpu_visible_size == + * @probed_size. + */ + __u64 probed_cpu_visible_size; + + /** + * @unallocated_cpu_visible_size: Estimate of CPU + * visible memory remaining (-1 = unknown). + * + * Note this is only currently tracked for + * I915_MEMORY_CLASS_
Re: [PATCH v3] drm/doc: add rfc section for small BAR uapi
On 17/05/2022 09:29, Tvrtko Ursulin wrote: On 16/05/2022 19:11, Matthew Auld wrote: Add an entry for the new uapi needed for small BAR on DG2+. v2: - Some spelling fixes and other small tweaks. (Akeem & Thomas) - Rework error capture interactions, including no longer needing NEEDS_CPU_ACCESS for objects marked for capture. (Thomas) - Add probed_cpu_visible_size. (Lionel) v3: - Drop the vma query for now. - Add unallocated_cpu_visible_size as part of the region query. - Improve the docs some more, including documenting the expected behaviour on older kernels, since this came up in some offline discussion. Signed-off-by: Matthew Auld Cc: Thomas Hellström Cc: Lionel Landwerlin Cc: Tvrtko Ursulin Cc: Jon Bloomfield Cc: Daniel Vetter Cc: Jon Bloomfield Cc: Jordan Justen Cc: Kenneth Graunke Cc: Akeem G Abodunrin Cc: mesa-dev@lists.freedesktop.org --- Documentation/gpu/rfc/i915_small_bar.h | 164 +++ Documentation/gpu/rfc/i915_small_bar.rst | 47 +++ Documentation/gpu/rfc/index.rst | 4 + 3 files changed, 215 insertions(+) create mode 100644 Documentation/gpu/rfc/i915_small_bar.h create mode 100644 Documentation/gpu/rfc/i915_small_bar.rst diff --git a/Documentation/gpu/rfc/i915_small_bar.h b/Documentation/gpu/rfc/i915_small_bar.h new file mode 100644 index ..4079d287750b --- /dev/null +++ b/Documentation/gpu/rfc/i915_small_bar.h @@ -0,0 +1,164 @@ +/** + * struct __drm_i915_memory_region_info - Describes one region as known to the + * driver. + * + * Note this is using both struct drm_i915_query_item and struct drm_i915_query. + * For this new query we are adding the new query id DRM_I915_QUERY_MEMORY_REGIONS + * at &drm_i915_query_item.query_id. + */ +struct __drm_i915_memory_region_info { + /** @region: The class:instance pair encoding */ + struct drm_i915_gem_memory_class_instance region; + + /** @rsvd0: MBZ */ + __u32 rsvd0; + + /** @probed_size: Memory probed by the driver (-1 = unknown) */ + __u64 probed_size; Is -1 possible today or when it will be? For system memory it appears zeroes are returned today so that has to stay I think. Does it effectively mean userspace has to consider both 0 and -1 as unknown is the question. It looks like it just returns the totalram_pages(). So at the moment nothing ever currently returns -1 or 0. Maybe that was a mistake for I915_MEMORY_SYSTEM. + + /** + * @unallocated_size: Estimate of memory remaining (-1 = unknown) + * + * Note this is only currently tracked for I915_MEMORY_CLASS_DEVICE + * regions, and also requires CAP_PERFMON or CAP_SYS_ADMIN to get + * reliable accounting. Without this(or if this an older kernel) the s/if this an/if this is an/ Also same question as above about -1. This should be the same as above, since this will give the same value as probed_size, or give the real avail value for lmem. + * value here will always match the @probed_size. + */ + __u64 unallocated_size; + + union { + /** @rsvd1: MBZ */ + __u64 rsvd1[8]; + struct { + /** + * @probed_cpu_visible_size: Memory probed by the driver + * that is CPU accessible. (-1 = unknown). Also question about -1. In this case this could be done since the field is yet to be added but I am curious if it ever can be -1. I was just going to make this the same as probed_size for system memory. But we could use -1 here instead. What do you think? Same for unallocated below. + * + * This will be always be <= @probed_size, and the + * remainder(if there is any) will not be CPU + * accessible. + * + * On systems without small BAR, the @probed_size will + * always equal the @probed_cpu_visible_size, since all + * of it will be CPU accessible. + * + * Note that if the value returned here is zero, then + * this must be an old kernel which lacks the relevant + * small-bar uAPI support(including I have noticed you prefer no space before parentheses throughout the text so I guess it's just my preference to have it. Very nitpicky even if I am right so up to you. Ok, will change :) + * I915_GEM_CREATE_EXT_FLAG_NEEDS_CPU_ACCESS), but on + * such systems we should never actually end up with a + * small BAR configuration, assuming we are able to load + * the kernel module. Hence it should be safe to treat + * this the same as when @probed_cpu_visible_size == + * @probed_size. + */ + __u64 probed_cpu_visible_size; + + /** + * @unallocated_cpu_visible_size: Estimate of CPU + * visible memory remaining (-1 = unknown). + * + * Note this is only currently tracked for +
Re: [PATCH v3] drm/doc: add rfc section for small BAR uapi
On 17/05/2022 09:55, Lionel Landwerlin wrote: On 17/05/2022 11:29, Tvrtko Ursulin wrote: On 16/05/2022 19:11, Matthew Auld wrote: Add an entry for the new uapi needed for small BAR on DG2+. v2: - Some spelling fixes and other small tweaks. (Akeem & Thomas) - Rework error capture interactions, including no longer needing NEEDS_CPU_ACCESS for objects marked for capture. (Thomas) - Add probed_cpu_visible_size. (Lionel) v3: - Drop the vma query for now. - Add unallocated_cpu_visible_size as part of the region query. - Improve the docs some more, including documenting the expected behaviour on older kernels, since this came up in some offline discussion. Signed-off-by: Matthew Auld Cc: Thomas Hellström Cc: Lionel Landwerlin Cc: Tvrtko Ursulin Cc: Jon Bloomfield Cc: Daniel Vetter Cc: Jon Bloomfield Cc: Jordan Justen Cc: Kenneth Graunke Cc: Akeem G Abodunrin Cc: mesa-dev@lists.freedesktop.org --- Documentation/gpu/rfc/i915_small_bar.h | 164 +++ Documentation/gpu/rfc/i915_small_bar.rst | 47 +++ Documentation/gpu/rfc/index.rst | 4 + 3 files changed, 215 insertions(+) create mode 100644 Documentation/gpu/rfc/i915_small_bar.h create mode 100644 Documentation/gpu/rfc/i915_small_bar.rst diff --git a/Documentation/gpu/rfc/i915_small_bar.h b/Documentation/gpu/rfc/i915_small_bar.h new file mode 100644 index ..4079d287750b --- /dev/null +++ b/Documentation/gpu/rfc/i915_small_bar.h @@ -0,0 +1,164 @@ +/** + * struct __drm_i915_memory_region_info - Describes one region as known to the + * driver. + * + * Note this is using both struct drm_i915_query_item and struct drm_i915_query. + * For this new query we are adding the new query id DRM_I915_QUERY_MEMORY_REGIONS + * at &drm_i915_query_item.query_id. + */ +struct __drm_i915_memory_region_info { + /** @region: The class:instance pair encoding */ + struct drm_i915_gem_memory_class_instance region; + + /** @rsvd0: MBZ */ + __u32 rsvd0; + + /** @probed_size: Memory probed by the driver (-1 = unknown) */ + __u64 probed_size; Is -1 possible today or when it will be? For system memory it appears zeroes are returned today so that has to stay I think. Does it effectively mean userspace has to consider both 0 and -1 as unknown is the question. I raised this on v2. As far as I can tell there are no situation where we would get -1. Is it really probed_size=0 on smem?? It's not the case on the internal branch. My bad, I misread the arguments to intel_memory_region_create while grepping: struct intel_memory_region *i915_gem_shmem_setup(struct drm_i915_private *i915, u16 type, u16 instance) { return intel_memory_region_create(i915, 0, totalram_pages() << PAGE_SHIFT, PAGE_SIZE, 0, 0, type, instance, &shmem_region_ops); I saw "0, 0" and wrongly assumed that would be the data, since it matched with my mental model and the comment against unallocated_size saying it's only tracked for device memory. Although I'd say it is questionable for i915 to return this data. I wonder it use case is possible where it would even be wrong but don't know. I guess the cat is out of the bag now. If the situation is -1 for unknown and some valid size (not zero) I don't think there is a problem here. Regards, Tvrtko Anv is not currently handling that case. I would very much like to not deal with 0 for smem. It really makes it easier for userspace rather than having to fish information from 2 different places and on top of dealing with multiple kernel versions. -Lionel + + /** + * @unallocated_size: Estimate of memory remaining (-1 = unknown) + * + * Note this is only currently tracked for I915_MEMORY_CLASS_DEVICE + * regions, and also requires CAP_PERFMON or CAP_SYS_ADMIN to get + * reliable accounting. Without this(or if this an older kernel) the s/if this an/if this is an/ Also same question as above about -1. + * value here will always match the @probed_size. + */ + __u64 unallocated_size; + + union { + /** @rsvd1: MBZ */ + __u64 rsvd1[8]; + struct { + /** + * @probed_cpu_visible_size: Memory probed by the driver + * that is CPU accessible. (-1 = unknown). Also question about -1. In this case this could be done since the field is yet to be added but I am curious if it ever can be -1. + * + * This will be always be <= @probed_size, and the + * remainder(if there is any) will not be CPU + * accessible. + * + * On systems without small BAR, the @probed_size will + * always equal the @probed_cpu_visible_size, since all +
Re: [PATCH v3] drm/doc: add rfc section for small BAR uapi
On 17/05/2022 12:23, Tvrtko Ursulin wrote: On 17/05/2022 09:55, Lionel Landwerlin wrote: On 17/05/2022 11:29, Tvrtko Ursulin wrote: On 16/05/2022 19:11, Matthew Auld wrote: Add an entry for the new uapi needed for small BAR on DG2+. v2: - Some spelling fixes and other small tweaks. (Akeem & Thomas) - Rework error capture interactions, including no longer needing NEEDS_CPU_ACCESS for objects marked for capture. (Thomas) - Add probed_cpu_visible_size. (Lionel) v3: - Drop the vma query for now. - Add unallocated_cpu_visible_size as part of the region query. - Improve the docs some more, including documenting the expected behaviour on older kernels, since this came up in some offline discussion. Signed-off-by: Matthew Auld Cc: Thomas Hellström Cc: Lionel Landwerlin Cc: Tvrtko Ursulin Cc: Jon Bloomfield Cc: Daniel Vetter Cc: Jon Bloomfield Cc: Jordan Justen Cc: Kenneth Graunke Cc: Akeem G Abodunrin Cc: mesa-dev@lists.freedesktop.org --- Documentation/gpu/rfc/i915_small_bar.h | 164 +++ Documentation/gpu/rfc/i915_small_bar.rst | 47 +++ Documentation/gpu/rfc/index.rst | 4 + 3 files changed, 215 insertions(+) create mode 100644 Documentation/gpu/rfc/i915_small_bar.h create mode 100644 Documentation/gpu/rfc/i915_small_bar.rst diff --git a/Documentation/gpu/rfc/i915_small_bar.h b/Documentation/gpu/rfc/i915_small_bar.h new file mode 100644 index ..4079d287750b --- /dev/null +++ b/Documentation/gpu/rfc/i915_small_bar.h @@ -0,0 +1,164 @@ +/** + * struct __drm_i915_memory_region_info - Describes one region as known to the + * driver. + * + * Note this is using both struct drm_i915_query_item and struct drm_i915_query. + * For this new query we are adding the new query id DRM_I915_QUERY_MEMORY_REGIONS + * at &drm_i915_query_item.query_id. + */ +struct __drm_i915_memory_region_info { + /** @region: The class:instance pair encoding */ + struct drm_i915_gem_memory_class_instance region; + + /** @rsvd0: MBZ */ + __u32 rsvd0; + + /** @probed_size: Memory probed by the driver (-1 = unknown) */ + __u64 probed_size; Is -1 possible today or when it will be? For system memory it appears zeroes are returned today so that has to stay I think. Does it effectively mean userspace has to consider both 0 and -1 as unknown is the question. I raised this on v2. As far as I can tell there are no situation where we would get -1. Is it really probed_size=0 on smem?? It's not the case on the internal branch. My bad, I misread the arguments to intel_memory_region_create while grepping: struct intel_memory_region *i915_gem_shmem_setup(struct drm_i915_private *i915, u16 type, u16 instance) { return intel_memory_region_create(i915, 0, totalram_pages() << PAGE_SHIFT, PAGE_SIZE, 0, 0, type, instance, &shmem_region_ops); I saw "0, 0" and wrongly assumed that would be the data, since it matched with my mental model and the comment against unallocated_size saying it's only tracked for device memory. Although I'd say it is questionable for i915 to return this data. I wonder it use case is possible where it would even be wrong but don't know. I guess the cat is out of the bag now. Not sure how questionable that is. There are a bunch of tools reporting the amount of memory available (free, top, htop, etc...). It might not be totalram_pages() but probably something close to it. Having a non 0 & non -1 value is useful. -Lionel If the situation is -1 for unknown and some valid size (not zero) I don't think there is a problem here. Regards, Tvrtko Anv is not currently handling that case. I would very much like to not deal with 0 for smem. It really makes it easier for userspace rather than having to fish information from 2 different places and on top of dealing with multiple kernel versions. -Lionel + + /** + * @unallocated_size: Estimate of memory remaining (-1 = unknown) + * + * Note this is only currently tracked for I915_MEMORY_CLASS_DEVICE + * regions, and also requires CAP_PERFMON or CAP_SYS_ADMIN to get + * reliable accounting. Without this(or if this an older kernel) the s/if this an/if this is an/ Also same question as above about -1. + * value here will always match the @probed_size. + */ + __u64 unallocated_size; + + union { + /** @rsvd1: MBZ */ + __u64 rsvd1[8]; + struct { + /** + * @probed_cpu_visible_size: Memory probed by the driver + * that is CPU accessible. (-1 = unknown). Also question about -1. In this case this could be done since the field is yet to be added but I am curious if it ever can be -1. + * + * This will be always be <= @probed_size, and the + * remainder(if there is any) will
[PATCH v4] drm/doc: add rfc section for small BAR uapi
Add an entry for the new uapi needed for small BAR on DG2+. v2: - Some spelling fixes and other small tweaks. (Akeem & Thomas) - Rework error capture interactions, including no longer needing NEEDS_CPU_ACCESS for objects marked for capture. (Thomas) - Add probed_cpu_visible_size. (Lionel) v3: - Drop the vma query for now. - Add unallocated_cpu_visible_size as part of the region query. - Improve the docs some more, including documenting the expected behaviour on older kernels, since this came up in some offline discussion. v4: - Various improvements all over. (Tvrtko) Signed-off-by: Matthew Auld Cc: Thomas Hellström Cc: Lionel Landwerlin Cc: Tvrtko Ursulin Cc: Jon Bloomfield Cc: Daniel Vetter Cc: Jon Bloomfield Cc: Jordan Justen Cc: Kenneth Graunke Cc: Akeem G Abodunrin Cc: mesa-dev@lists.freedesktop.org --- Documentation/gpu/rfc/i915_small_bar.h | 189 +++ Documentation/gpu/rfc/i915_small_bar.rst | 47 ++ Documentation/gpu/rfc/index.rst | 4 + 3 files changed, 240 insertions(+) create mode 100644 Documentation/gpu/rfc/i915_small_bar.h create mode 100644 Documentation/gpu/rfc/i915_small_bar.rst diff --git a/Documentation/gpu/rfc/i915_small_bar.h b/Documentation/gpu/rfc/i915_small_bar.h new file mode 100644 index ..c676640b23ef --- /dev/null +++ b/Documentation/gpu/rfc/i915_small_bar.h @@ -0,0 +1,189 @@ +/** + * struct __drm_i915_memory_region_info - Describes one region as known to the + * driver. + * + * Note this is using both struct drm_i915_query_item and struct drm_i915_query. + * For this new query we are adding the new query id DRM_I915_QUERY_MEMORY_REGIONS + * at &drm_i915_query_item.query_id. + */ +struct __drm_i915_memory_region_info { + /** @region: The class:instance pair encoding */ + struct drm_i915_gem_memory_class_instance region; + + /** @rsvd0: MBZ */ + __u32 rsvd0; + + /** +* @probed_size: Memory probed by the driver (-1 = unknown) +* +* Note that it should not be possible to ever encounter a zero value +* here, also note that no current region type will ever return -1 here. +* Although for future region types, this might be a possibility. The +* same applies to the other size fields. +*/ + __u64 probed_size; + + /** +* @unallocated_size: Estimate of memory remaining (-1 = unknown) +* +* Requires CAP_PERFMON or CAP_SYS_ADMIN to get reliable accounting. +* Without this (or if this is an older kernel) the value here will +* always equal the @probed_size. Note this is only currently tracked +* for I915_MEMORY_CLASS_DEVICE regions (for other types the value here +* will always equal the @probed_size). +*/ + __u64 unallocated_size; + + union { + /** @rsvd1: MBZ */ + __u64 rsvd1[8]; + struct { + /** +* @probed_cpu_visible_size: Memory probed by the driver +* that is CPU accessible. (-1 = unknown). +* +* This will be always be <= @probed_size, and the +* remainder (if there is any) will not be CPU +* accessible. +* +* On systems without small BAR, the @probed_size will +* always equal the @probed_cpu_visible_size, since all +* of it will be CPU accessible. +* +* Note this is only tracked for +* I915_MEMORY_CLASS_DEVICE regions (for other types the +* value here will always equal the @probed_size). +* +* Note that if the value returned here is zero, then +* this must be an old kernel which lacks the relevant +* small-bar uAPI support (including +* I915_GEM_CREATE_EXT_FLAG_NEEDS_CPU_ACCESS), but on +* such systems we should never actually end up with a +* small BAR configuration, assuming we are able to load +* the kernel module. Hence it should be safe to treat +* this the same as when @probed_cpu_visible_size == +* @probed_size. +*/ + __u64 probed_cpu_visible_size; + + /** +* @unallocated_cpu_visible_size: Estimate of CPU +* visible memory remaining (-1 = unknown). +* +* Note this is only tracked for +* I915_MEMORY_CLASS_DEVICE regions (for other types the +* value here will always equal the +
Re: [PATCH v3] drm/doc: add rfc section for small BAR uapi
On 17/05/2022 10:39, Lionel Landwerlin wrote: On 17/05/2022 12:23, Tvrtko Ursulin wrote: On 17/05/2022 09:55, Lionel Landwerlin wrote: On 17/05/2022 11:29, Tvrtko Ursulin wrote: On 16/05/2022 19:11, Matthew Auld wrote: Add an entry for the new uapi needed for small BAR on DG2+. v2: - Some spelling fixes and other small tweaks. (Akeem & Thomas) - Rework error capture interactions, including no longer needing NEEDS_CPU_ACCESS for objects marked for capture. (Thomas) - Add probed_cpu_visible_size. (Lionel) v3: - Drop the vma query for now. - Add unallocated_cpu_visible_size as part of the region query. - Improve the docs some more, including documenting the expected behaviour on older kernels, since this came up in some offline discussion. Signed-off-by: Matthew Auld Cc: Thomas Hellström Cc: Lionel Landwerlin Cc: Tvrtko Ursulin Cc: Jon Bloomfield Cc: Daniel Vetter Cc: Jon Bloomfield Cc: Jordan Justen Cc: Kenneth Graunke Cc: Akeem G Abodunrin Cc: mesa-dev@lists.freedesktop.org --- Documentation/gpu/rfc/i915_small_bar.h | 164 +++ Documentation/gpu/rfc/i915_small_bar.rst | 47 +++ Documentation/gpu/rfc/index.rst | 4 + 3 files changed, 215 insertions(+) create mode 100644 Documentation/gpu/rfc/i915_small_bar.h create mode 100644 Documentation/gpu/rfc/i915_small_bar.rst diff --git a/Documentation/gpu/rfc/i915_small_bar.h b/Documentation/gpu/rfc/i915_small_bar.h new file mode 100644 index ..4079d287750b --- /dev/null +++ b/Documentation/gpu/rfc/i915_small_bar.h @@ -0,0 +1,164 @@ +/** + * struct __drm_i915_memory_region_info - Describes one region as known to the + * driver. + * + * Note this is using both struct drm_i915_query_item and struct drm_i915_query. + * For this new query we are adding the new query id DRM_I915_QUERY_MEMORY_REGIONS + * at &drm_i915_query_item.query_id. + */ +struct __drm_i915_memory_region_info { + /** @region: The class:instance pair encoding */ + struct drm_i915_gem_memory_class_instance region; + + /** @rsvd0: MBZ */ + __u32 rsvd0; + + /** @probed_size: Memory probed by the driver (-1 = unknown) */ + __u64 probed_size; Is -1 possible today or when it will be? For system memory it appears zeroes are returned today so that has to stay I think. Does it effectively mean userspace has to consider both 0 and -1 as unknown is the question. I raised this on v2. As far as I can tell there are no situation where we would get -1. Is it really probed_size=0 on smem?? It's not the case on the internal branch. My bad, I misread the arguments to intel_memory_region_create while grepping: struct intel_memory_region *i915_gem_shmem_setup(struct drm_i915_private *i915, u16 type, u16 instance) { return intel_memory_region_create(i915, 0, totalram_pages() << PAGE_SHIFT, PAGE_SIZE, 0, 0, type, instance, &shmem_region_ops); I saw "0, 0" and wrongly assumed that would be the data, since it matched with my mental model and the comment against unallocated_size saying it's only tracked for device memory. Although I'd say it is questionable for i915 to return this data. I wonder it use case is possible where it would even be wrong but don't know. I guess the cat is out of the bag now. Not sure how questionable that is. There are a bunch of tools reporting the amount of memory available (free, top, htop, etc...). It might not be totalram_pages() but probably something close to it. Questionable as it being a resource driver does not own so it reporting it is a pure userspace convenience and maintenance burden for the driver. :) Not sure I understand why userspace even wants to know? Only reliable use could be to decide whether to even try and run a workload? But in that case too perhaps users wants to run it with swapping so let them. There is also the question memory accounting and a process could be running in a container with a limited view of the world (totalram_pages would be wrong). Although granted, we bypass memory accounting at the moment. Not sure what was the discussion on that before. Probably because one process can create an object and another can instantiate the backing store. Whether or not we could, or should, therefore account against the real creator is the question. But if we one day did, then we'd have to fiddle with the memory query to stop returning totalram_pages() and that would potentially be complicated. Well "would".. we probably will since this is already shipping. Regards, Tvrtko Having a non 0 & non -1 value is useful. -Lionel If the situation is -1 for unknown and some valid size (not zero) I don't think there is a problem here. Regards, Tvrtko Anv is not currently handling that case. I would very much like to not deal with 0 for smem. It really makes it eas
Re: [PATCH v3] drm/doc: add rfc section for small BAR uapi
On 17/05/2022 10:12, Matthew Auld wrote: On 17/05/2022 09:29, Tvrtko Ursulin wrote: On 16/05/2022 19:11, Matthew Auld wrote: Add an entry for the new uapi needed for small BAR on DG2+. v2: - Some spelling fixes and other small tweaks. (Akeem & Thomas) - Rework error capture interactions, including no longer needing NEEDS_CPU_ACCESS for objects marked for capture. (Thomas) - Add probed_cpu_visible_size. (Lionel) v3: - Drop the vma query for now. - Add unallocated_cpu_visible_size as part of the region query. - Improve the docs some more, including documenting the expected behaviour on older kernels, since this came up in some offline discussion. Signed-off-by: Matthew Auld Cc: Thomas Hellström Cc: Lionel Landwerlin Cc: Tvrtko Ursulin Cc: Jon Bloomfield Cc: Daniel Vetter Cc: Jon Bloomfield Cc: Jordan Justen Cc: Kenneth Graunke Cc: Akeem G Abodunrin Cc: mesa-dev@lists.freedesktop.org --- Documentation/gpu/rfc/i915_small_bar.h | 164 +++ Documentation/gpu/rfc/i915_small_bar.rst | 47 +++ Documentation/gpu/rfc/index.rst | 4 + 3 files changed, 215 insertions(+) create mode 100644 Documentation/gpu/rfc/i915_small_bar.h create mode 100644 Documentation/gpu/rfc/i915_small_bar.rst diff --git a/Documentation/gpu/rfc/i915_small_bar.h b/Documentation/gpu/rfc/i915_small_bar.h new file mode 100644 index ..4079d287750b --- /dev/null +++ b/Documentation/gpu/rfc/i915_small_bar.h @@ -0,0 +1,164 @@ +/** + * struct __drm_i915_memory_region_info - Describes one region as known to the + * driver. + * + * Note this is using both struct drm_i915_query_item and struct drm_i915_query. + * For this new query we are adding the new query id DRM_I915_QUERY_MEMORY_REGIONS + * at &drm_i915_query_item.query_id. + */ +struct __drm_i915_memory_region_info { + /** @region: The class:instance pair encoding */ + struct drm_i915_gem_memory_class_instance region; + + /** @rsvd0: MBZ */ + __u32 rsvd0; + + /** @probed_size: Memory probed by the driver (-1 = unknown) */ + __u64 probed_size; Is -1 possible today or when it will be? For system memory it appears zeroes are returned today so that has to stay I think. Does it effectively mean userspace has to consider both 0 and -1 as unknown is the question. It looks like it just returns the totalram_pages(). So at the moment nothing ever currently returns -1 or 0. Maybe that was a mistake for I915_MEMORY_SYSTEM. Yes I figured out my confusion later. + + /** + * @unallocated_size: Estimate of memory remaining (-1 = unknown) + * + * Note this is only currently tracked for I915_MEMORY_CLASS_DEVICE + * regions, and also requires CAP_PERFMON or CAP_SYS_ADMIN to get + * reliable accounting. Without this(or if this an older kernel) the s/if this an/if this is an/ Also same question as above about -1. This should be the same as above, since this will give the same value as probed_size, or give the real avail value for lmem. + * value here will always match the @probed_size. + */ + __u64 unallocated_size; + + union { + /** @rsvd1: MBZ */ + __u64 rsvd1[8]; + struct { + /** + * @probed_cpu_visible_size: Memory probed by the driver + * that is CPU accessible. (-1 = unknown). Also question about -1. In this case this could be done since the field is yet to be added but I am curious if it ever can be -1. I was just going to make this the same as probed_size for system memory. But we could use -1 here instead. What do you think? Same for unallocated below. I don't completely follow the instead part, when you already have documented it? -1 will start appearing with changes that require CAP_SYS_PERFMON right? Regards, Tvrtko + * + * This will be always be <= @probed_size, and the + * remainder(if there is any) will not be CPU + * accessible. + * + * On systems without small BAR, the @probed_size will + * always equal the @probed_cpu_visible_size, since all + * of it will be CPU accessible. + * + * Note that if the value returned here is zero, then + * this must be an old kernel which lacks the relevant + * small-bar uAPI support(including I have noticed you prefer no space before parentheses throughout the text so I guess it's just my preference to have it. Very nitpicky even if I am right so up to you. Ok, will change :) + * I915_GEM_CREATE_EXT_FLAG_NEEDS_CPU_ACCESS), but on + * such systems we should never actually end up with a + * small BAR configuration, assuming we are able to load + * the kernel module. Hence it should be safe to treat + * this the same as when @probed_cpu_visible_size == + * @probed_size. + *
Re: [PATCH v4] drm/doc: add rfc section for small BAR uapi
On 17/05/2022 11:52, Matthew Auld wrote: Add an entry for the new uapi needed for small BAR on DG2+. v2: - Some spelling fixes and other small tweaks. (Akeem & Thomas) - Rework error capture interactions, including no longer needing NEEDS_CPU_ACCESS for objects marked for capture. (Thomas) - Add probed_cpu_visible_size. (Lionel) v3: - Drop the vma query for now. - Add unallocated_cpu_visible_size as part of the region query. - Improve the docs some more, including documenting the expected behaviour on older kernels, since this came up in some offline discussion. v4: - Various improvements all over. (Tvrtko) You can ignore my previous reply, the clarifications from v4 read good to me. Acked-by: Tvrtko Ursulin Regards, Tvrtko Signed-off-by: Matthew Auld Cc: Thomas Hellström Cc: Lionel Landwerlin Cc: Tvrtko Ursulin Cc: Jon Bloomfield Cc: Daniel Vetter Cc: Jon Bloomfield Cc: Jordan Justen Cc: Kenneth Graunke Cc: Akeem G Abodunrin Cc: mesa-dev@lists.freedesktop.org --- Documentation/gpu/rfc/i915_small_bar.h | 189 +++ Documentation/gpu/rfc/i915_small_bar.rst | 47 ++ Documentation/gpu/rfc/index.rst | 4 + 3 files changed, 240 insertions(+) create mode 100644 Documentation/gpu/rfc/i915_small_bar.h create mode 100644 Documentation/gpu/rfc/i915_small_bar.rst diff --git a/Documentation/gpu/rfc/i915_small_bar.h b/Documentation/gpu/rfc/i915_small_bar.h new file mode 100644 index ..c676640b23ef --- /dev/null +++ b/Documentation/gpu/rfc/i915_small_bar.h @@ -0,0 +1,189 @@ +/** + * struct __drm_i915_memory_region_info - Describes one region as known to the + * driver. + * + * Note this is using both struct drm_i915_query_item and struct drm_i915_query. + * For this new query we are adding the new query id DRM_I915_QUERY_MEMORY_REGIONS + * at &drm_i915_query_item.query_id. + */ +struct __drm_i915_memory_region_info { + /** @region: The class:instance pair encoding */ + struct drm_i915_gem_memory_class_instance region; + + /** @rsvd0: MBZ */ + __u32 rsvd0; + + /** +* @probed_size: Memory probed by the driver (-1 = unknown) +* +* Note that it should not be possible to ever encounter a zero value +* here, also note that no current region type will ever return -1 here. +* Although for future region types, this might be a possibility. The +* same applies to the other size fields. +*/ + __u64 probed_size; + + /** +* @unallocated_size: Estimate of memory remaining (-1 = unknown) +* +* Requires CAP_PERFMON or CAP_SYS_ADMIN to get reliable accounting. +* Without this (or if this is an older kernel) the value here will +* always equal the @probed_size. Note this is only currently tracked +* for I915_MEMORY_CLASS_DEVICE regions (for other types the value here +* will always equal the @probed_size). +*/ + __u64 unallocated_size; + + union { + /** @rsvd1: MBZ */ + __u64 rsvd1[8]; + struct { + /** +* @probed_cpu_visible_size: Memory probed by the driver +* that is CPU accessible. (-1 = unknown). +* +* This will be always be <= @probed_size, and the +* remainder (if there is any) will not be CPU +* accessible. +* +* On systems without small BAR, the @probed_size will +* always equal the @probed_cpu_visible_size, since all +* of it will be CPU accessible. +* +* Note this is only tracked for +* I915_MEMORY_CLASS_DEVICE regions (for other types the +* value here will always equal the @probed_size). +* +* Note that if the value returned here is zero, then +* this must be an old kernel which lacks the relevant +* small-bar uAPI support (including +* I915_GEM_CREATE_EXT_FLAG_NEEDS_CPU_ACCESS), but on +* such systems we should never actually end up with a +* small BAR configuration, assuming we are able to load +* the kernel module. Hence it should be safe to treat +* this the same as when @probed_cpu_visible_size == +* @probed_size. +*/ + __u64 probed_cpu_visible_size; + + /** +* @unallocated_cpu_visible_size: Estimate of CPU +* visible memory remaining (-1 = unknown). +* +
RE: [PATCH v4] drm/doc: add rfc section for small BAR uapi
> -Original Message- > From: Tvrtko Ursulin > Sent: Tuesday, May 17, 2022 6:49 AM > To: Auld, Matthew ; intel-...@lists.freedesktop.org > Cc: dri-de...@lists.freedesktop.org; Thomas Hellström > ; Landwerlin, Lionel G > ; Bloomfield, Jon ; > Daniel Vetter ; Justen, Jordan L > ; Kenneth Graunke ; > Abodunrin, Akeem G ; mesa- > d...@lists.freedesktop.org > Subject: Re: [PATCH v4] drm/doc: add rfc section for small BAR uapi > > > On 17/05/2022 11:52, Matthew Auld wrote: > > Add an entry for the new uapi needed for small BAR on DG2+. > > > > v2: > >- Some spelling fixes and other small tweaks. (Akeem & Thomas) > >- Rework error capture interactions, including no longer needing > > NEEDS_CPU_ACCESS for objects marked for capture. (Thomas) > >- Add probed_cpu_visible_size. (Lionel) > > v3: > >- Drop the vma query for now. > >- Add unallocated_cpu_visible_size as part of the region query. > >- Improve the docs some more, including documenting the expected > > behaviour on older kernels, since this came up in some offline > > discussion. > > v4: > >- Various improvements all over. (Tvrtko) > > You can ignore my previous reply, the clarifications from v4 read good to me. > > Acked-by: Tvrtko Ursulin The patch looks good to me as well... Acked-by: Akeem G Abodunrin > > Regards, > > Tvrtko > > > > > Signed-off-by: Matthew Auld > > Cc: Thomas Hellström > > Cc: Lionel Landwerlin > > Cc: Tvrtko Ursulin > > Cc: Jon Bloomfield > > Cc: Daniel Vetter > > Cc: Jon Bloomfield > > Cc: Jordan Justen > > Cc: Kenneth Graunke > > Cc: Akeem G Abodunrin > > Cc: mesa-dev@lists.freedesktop.org > > --- > > Documentation/gpu/rfc/i915_small_bar.h | 189 > +++ > > Documentation/gpu/rfc/i915_small_bar.rst | 47 ++ > > Documentation/gpu/rfc/index.rst | 4 + > > 3 files changed, 240 insertions(+) > > create mode 100644 Documentation/gpu/rfc/i915_small_bar.h > > create mode 100644 Documentation/gpu/rfc/i915_small_bar.rst > > > > diff --git a/Documentation/gpu/rfc/i915_small_bar.h > > b/Documentation/gpu/rfc/i915_small_bar.h > > new file mode 100644 > > index ..c676640b23ef > > --- /dev/null > > +++ b/Documentation/gpu/rfc/i915_small_bar.h > > @@ -0,0 +1,189 @@ > > +/** > > + * struct __drm_i915_memory_region_info - Describes one region as > > +known to the > > + * driver. > > + * > > + * Note this is using both struct drm_i915_query_item and struct > drm_i915_query. > > + * For this new query we are adding the new query id > > +DRM_I915_QUERY_MEMORY_REGIONS > > + * at &drm_i915_query_item.query_id. > > + */ > > +struct __drm_i915_memory_region_info { > > + /** @region: The class:instance pair encoding */ > > + struct drm_i915_gem_memory_class_instance region; > > + > > + /** @rsvd0: MBZ */ > > + __u32 rsvd0; > > + > > + /** > > +* @probed_size: Memory probed by the driver (-1 = unknown) > > +* > > +* Note that it should not be possible to ever encounter a zero value > > +* here, also note that no current region type will ever return -1 here. > > +* Although for future region types, this might be a possibility. The > > +* same applies to the other size fields. > > +*/ > > + __u64 probed_size; > > + > > + /** > > +* @unallocated_size: Estimate of memory remaining (-1 = unknown) > > +* > > +* Requires CAP_PERFMON or CAP_SYS_ADMIN to get reliable > accounting. > > +* Without this (or if this is an older kernel) the value here will > > +* always equal the @probed_size. Note this is only currently tracked > > +* for I915_MEMORY_CLASS_DEVICE regions (for other types the value > here > > +* will always equal the @probed_size). > > +*/ > > + __u64 unallocated_size; > > + > > + union { > > + /** @rsvd1: MBZ */ > > + __u64 rsvd1[8]; > > + struct { > > + /** > > +* @probed_cpu_visible_size: Memory probed by the > driver > > +* that is CPU accessible. (-1 = unknown). > > +* > > +* This will be always be <= @probed_size, and the > > +* remainder (if there is any) will not be CPU > > +* accessible. > > +* > > +* On systems without small BAR, the @probed_size will > > +* always equal the @probed_cpu_visible_size, since all > > +* of it will be CPU accessible. > > +* > > +* Note this is only tracked for > > +* I915_MEMORY_CLASS_DEVICE regions (for other > types the > > +* value here will always equal the @probed_size). > > +* > > +* Note that if the value returned here is zero, then > > +* this must be an old kernel which lacks the relevant > > +* smal
Re: [PATCH v3] uapi/drm/i915: Document memory residency and Flat-CCS capability of obj
On 2022-05-13 at 14:06:11 -0700, Jordan Justen wrote: > On 2022-05-13 05:31:00, Lionel Landwerlin wrote: > > On 02/05/2022 17:15, Ramalingam C wrote: > > > Capture the impact of memory region preference list of the objects, on > > > their memory residency and Flat-CCS capability. > > > > > > v2: > > >Fix the Flat-CCS capability of an obj with {lmem, smem} preference > > >list [Thomas] > > > v3: > > >Reworded the doc [Matt] > > > > > > Signed-off-by: Ramalingam C > > > cc: Matthew Auld > > > cc: Thomas Hellstrom > > > cc: Daniel Vetter > > > cc: Jon Bloomfield > > > cc: Lionel Landwerlin > > > cc: Kenneth Graunke > > > cc: mesa-dev@lists.freedesktop.org > > > cc: Jordan Justen > > > cc: Tony Ye > > > Reviewed-by: Matthew Auld > > > --- > > > include/uapi/drm/i915_drm.h | 16 > > > 1 file changed, 16 insertions(+) > > > > > > diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h > > > index a2def7b27009..b7e1c2fe08dc 100644 > > > --- a/include/uapi/drm/i915_drm.h > > > +++ b/include/uapi/drm/i915_drm.h > > > @@ -3443,6 +3443,22 @@ struct drm_i915_gem_create_ext { > > >* At which point we get the object handle in > > > &drm_i915_gem_create_ext.handle, > > >* along with the final object size in &drm_i915_gem_create_ext.size, > > > which > > >* should account for any rounding up, if required. > > > + * > > > + * Note that userspace has no means of knowing the current backing region > > > + * for objects where @num_regions is larger than one. The kernel will > > > only > > > + * ensure that the priority order of the @regions array is honoured, > > > either > > > + * when initially placing the object, or when moving memory around due to > > > + * memory pressure > > > + * > > > + * On Flat-CCS capable HW, compression is supported for the objects > > > residing > > > + * in I915_MEMORY_CLASS_DEVICE. When such objects (compressed) has other > > > + * memory class in @regions and migrated (by I915, due to memory > > > + * constrain) to the non I915_MEMORY_CLASS_DEVICE region, then I915 > > > needs to > > > + * decompress the content. But I915 dosen't have the required > > > information to > > > + * decompress the userspace compressed objects. > > > + * > > > + * So I915 supports Flat-CCS, only on the objects which can reside only > > > on > > > + * I915_MEMORY_CLASS_DEVICE regions. > > > > I think it's fine to assume Flat-CSS surface will always be in lmem. > > > > I see no issue for the Anv Vulkan driver. > > > > Maybe Nanley or Ken can speak for the Iris GL driver? > > > > Acked-by: Jordan Justen Thank you Jordan for the Ack! Ram > > I think Nanley has accounted for this on iris with: > > https://gitlab.freedesktop.org/mesa/mesa/-/commit/42a865730ef72574e179b56a314f30fdccc6cba8 > > -Jordan