On Wed, 27 Jun 2018 23:07:29 +0300, Or Gerlitz wrote: > On Wed, Jun 27, 2018 at 1:31 AM, Jakub Kicinski > <jakub.kicin...@netronome.com> wrote: > > On Tue, 26 Jun 2018 17:57:08 +0300, Or Gerlitz wrote: > > >> 2. re the egress side of things. Some NIC HWs can't just use LAG > >> as the egress port destination of an ACL (tc rule) and the HW rule > >> needs to be duplicated to both HW ports. So... in that case, you > >> see the HW driver doing the duplication (:() or we can somehow > >> make it happen from user-space? > > > It's the TC core that does the duplication. Drivers which don't need > > the duplication (e.g. mlxsw) will not register a new callback for each > > port on which shared block is bound. They will keep one list of rules, > > and a list of ports that those rules apply to. > > [snip] > > > Drivers which need duplication (multiplication) (all NICs?) have to > > register a new callback for each port bound to a shared block. And TC > > will call those drivers as many times as they have callbacks registered > > == as many times as they have ports bound to the block. Each time > > callback is invoked the driver will figure out the ingress port based > > on the cb_priv and use <ingress, cookie> as the key in its rule table > > (or have a separate rule table per ingress port). > > [snip snip] > > > I may be wrong, but I think you split the rules tables per port for mlx5 > > correct, currently I have a rule table per physical port. > > > So again you just register a callback every time shared block is bound, > > and then TC core will send add/remove rule commands down to the driver, > > relaying existing rules as well if needed. > > Let's see, the NIC uplink rep port devices were bounded (say) by ovs to > a shared-block because they are the lower devices (hate the slavish jargon) > of a bond device. > > Next, the TC stack will invoke the callback over these ports, when ingress > rule is added on the bond. > > But we are talking on ingress rule set on a non-uplink rep (VF rep) port, > where bonding is the egress of the rule. I guess the callback which you > probably > refer to (you hinted there below) is the egdev one, correct? you are > suggesting > that bonding will do egdev registration... I am a bit confused.
Ah, you really meant egress. We don't have this problem, but yes, I think you could register an egdev callback for each lower. You won't get the nice rule replay from egdev as of today, though :( > > Does that clarify things or were you asking more about the active > > passive thing John mentioned or some way to save rule space? > > no (didn't refer to active-passive) and no (didn't look to save rule space) > yes for active-active in a HW that needs duplicated rules (NICs). > > >> 3. for the case of overlay networks, e.g OVS based vxlan tunnel, the > >> ingress (decap) rule is set on the vxlan device. Jakub, you mentioned > >> a possible kernel patch to the HW (nfp, mlx5) drivers to have them bind > >> to the tunnel device for ingress rules. If we have agreed way to identify > >> uplink representors, can we do that from ovs too? > > > > I'm not sure, there can be multiple tunnel devices. Plus we really > > want to know the tunnel type, e.g. vxlan vs geneve, so simple shared > > block propagation will probably not cut it. If that's what you're > > referring to. > > isn't knowing the tunnel type already missing today? I saw you > started patches the tunnel key set action for Geneve, does upstream > + the patches you sent works or more is missing to get geneve encap > through the TC stack? Yes, knowing tunnel type missing today, but hopefully it won't be once we get to redesign of egdev :) Today we only support decap on standard ports :/ Encap is fine, though. FWIW Geneve already works on the nfp, the work from Simon & Pieter we posted is adding support for the options.
