Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-28 Thread Jakub Kicinski
On Thu, 28 Jan 2021 16:03:02 -0800 Saeed Mahameed wrote: > On Tue, 2021-01-26 at 17:34 -0800, Jakub Kicinski wrote: > > On Fri, 22 Jan 2021 11:36:44 -0800 Saeed Mahameed wrote: > > > This series form Parav was the theme of this mlx5 release cycle, > > > we've been waiting anxiously for the auxbus

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-28 Thread Saeed Mahameed
On Tue, 2021-01-26 at 17:34 -0800, Jakub Kicinski wrote: > On Fri, 22 Jan 2021 11:36:44 -0800 Saeed Mahameed wrote: > > This series form Parav was the theme of this mlx5 release cycle, > > we've been waiting anxiously for the auxbus infrastructure to make > > it into > > the kernel, and now as the

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-26 Thread Jakub Kicinski
On Fri, 22 Jan 2021 11:36:44 -0800 Saeed Mahameed wrote: > This series form Parav was the theme of this mlx5 release cycle, > we've been waiting anxiously for the auxbus infrastructure to make it into > the kernel, and now as the auxbus is in and all the stars are aligned, I > can finally submit th

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Edwin Peer
On Mon, Jan 25, 2021 at 5:22 AM Jason Gunthorpe wrote: > SRIOV and SF's require a simple linear lookup to learn the "function" > because the BAR space is required to be linear. Isn't this still true even for NumVF's > 256? Wouldn't there still be a contiguous VF BAR space? Don't the routing IDs

RE: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Parav Pandit
Hi Edwin, > From: Edwin Peer > Sent: Monday, January 25, 2021 2:17 AM > > On Fri, Jan 22, 2021 at 11:37 AM Saeed Mahameed > wrote: > > > For more detailed information about subfunctions please see detailed tag > > log below. > > Apologies for the tardy question out of left field, but I've bee

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Jason Gunthorpe
On Mon, Jan 25, 2021 at 10:57:14AM +, Parav Pandit wrote: > Hi Edwin, > > > From: Edwin Peer > > Sent: Monday, January 25, 2021 2:17 AM > > > > On Fri, Jan 22, 2021 at 11:37 AM Saeed Mahameed > > wrote: > > > > > For more detailed information about subfunctions please see detailed tag > >

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Edwin Peer
On Mon, Jan 25, 2021 at 10:35 AM Edwin Peer wrote: > > Several weeks back, Jason already answered this VF scaling question from > > you at discussion [1]. > > > > [1] https://lore.kernel.org/netdev/20201216023928.gg552...@nvidia.com/ Regarding these costs: > A lot of the trappings that PCI-SIG

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Jason Gunthorpe
On Mon, Jan 25, 2021 at 01:23:04PM -0800, Edwin Peer wrote: > On Mon, Jan 25, 2021 at 12:41 PM Jason Gunthorpe wrote: > > > > That's an implementation decision. Nothing mandates that the state has > > > to physically exist in the same structure, only that reads and writes > > > are appropriately

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Edwin Peer
On Mon, Jan 25, 2021 at 12:41 PM Jason Gunthorpe wrote: > > That's an implementation decision. Nothing mandates that the state has > > to physically exist in the same structure, only that reads and writes > > are appropriately responded to. > > Yes, PCI does mandate this, you can't store the data

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Jason Gunthorpe
On Mon, Jan 25, 2021 at 11:23:56AM -0800, Edwin Peer wrote: > On Mon, Jan 25, 2021 at 5:22 AM Jason Gunthorpe wrote: > > > SRIOV and SF's require a simple linear lookup to learn the "function" > > because the BAR space is required to be linear. > > Isn't this still true even for NumVF's > 256? W

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Jason Gunthorpe
On Mon, Jan 25, 2021 at 12:22:13PM -0800, Edwin Peer wrote: > On Mon, Jan 25, 2021 at 11:59 AM Jason Gunthorpe wrote: > > > Every writable data mandated by the PCI spec requires very expensive > > on-die SRAM to store it. > > That's an implementation decision. Nothing mandates that the state has

RE: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Parav Pandit
> From: Edwin Peer > Sent: Tuesday, January 26, 2021 1:36 AM > > On Mon, Jan 25, 2021 at 11:49 AM Jason Gunthorpe > wrote: > > > I've never seen someone implement a NumVF > 256 by co-opting the bus > > number. > > Usually the VF offset already places the VF routing IDs into a > different bus

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Michael Chan
On Mon, Jan 25, 2021 at 12:09 PM Edwin Peer wrote: > > On Mon, Jan 25, 2021 at 11:49 AM Jason Gunthorpe wrote: > > > I've never seen someone implement a NumVF > 256 by co-opting the bus > > number. > > Usually the VF offset already places the VF routing IDs into a > different bus number range fro

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Edwin Peer
On Mon, Jan 25, 2021 at 11:59 AM Jason Gunthorpe wrote: > Every writable data mandated by the PCI spec requires very expensive > on-die SRAM to store it. That's an implementation decision. Nothing mandates that the state has to physically exist in the same structure, only that reads and writes a

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Edwin Peer
On Mon, Jan 25, 2021 at 11:49 AM Jason Gunthorpe wrote: > I've never seen someone implement a NumVF > 256 by co-opting the bus > number. Usually the VF offset already places the VF routing IDs into a different bus number range from the PF. That much at least works today. > Can Linux even assign

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Jason Gunthorpe
On Mon, Jan 25, 2021 at 11:34:49AM -0800, Edwin Peer wrote: > What do these amount to in practice? Presumably config space is backed > by normal memory controlled by firmware. Do VF's need to expose ECAM? > Also, don't MSI tables come out of the BAR budget? Is the required BAR > space necessarily

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Edwin Peer
On Mon, Jan 25, 2021 at 2:57 AM Parav Pandit wrote: > > Apologies for the tardy question out of left field, but I've been > > thinking about this some more. If I recall, the primary motivation for > > this was a means to effectively address more VFs? But, why can't the > > device simply expose mo

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-24 Thread Edwin Peer
On Fri, Jan 22, 2021 at 11:37 AM Saeed Mahameed wrote: > This series form Parav was the theme of this mlx5 release cycle, > we've been waiting anxiously for the auxbus infrastructure to make it into > the kernel, and now as the auxbus is in and all the stars are aligned, I > can finally submit th