On 20/07/21(Tue) 14:26, Alexander Bluhm wrote: > On Tue, Jul 20, 2021 at 10:08:09AM +0200, Martin Pieuchot wrote: > > On 19/07/21(Mon) 17:53, Alexander Bluhm wrote: > > > Hi, > > > > > > I found why the IPsec workaround did not work. > > > > > > At init time we set ifiq->ifiq_softnet = net_tq(ifp->if_index + > > > idx), but the workaround modifies net_tq() at runtime. Modifying > > > net_tq() at runtime is bad anyway as task_add() and task_del() could > > > be called with different task queues. > > > > > > So better use exclusive lock if IPsec is in use. For me this is > > > running stable. > > > > Note that having multiple threads competing for an exclusive rwlock will > > generate unnecessary wakeup/sleep cycles every time the lock is released. > > It is valuable to keep this in mind as it might add extra latency when > > processing packets. > > Of course. What do you recommend?
What you find easier to make progress with :) > - Develop outside of the tree until all problems are fixed. > - Delay work on parallel forwarding until IPsec is MP safe. > - Accept a possible slowdown of IPsec. In my measurements it gets > faster even with the exclusive lock. > - Concentrate on making IPsec faster. By removing the crypto > queues you gain much more performance than the exclusive lock may > cost. Did you see the massive kernel locks in my graph? > > http://bluhm.genua.de/perform/results/latest/patch-sys-ip-multiqueue.1/btrace/ssh_perform%40lt13_iperf3_-c10.4.56.36_-P10_-t10_-R-btrace-kstack.0.svg I didn't see this, nice to know. > - Make ARP MP safe. Currently we need the kernel lock there or > it crashes. This creates latency for all kind of packets. > - Convert the rwlock in pf to mutex. I think your argument counts > much more there. But I cannot prove it. My argument about what? Didn't we all agree about converting the rwlock to a mutex for now? > My plan is to commit what we have and improve where most pain is. > This makes incremental steps easier. Makes sense.
