On 2026-09-17 13:14:05, Ido Schimmel wrote: > On Mon, Sep 14, 2026 at 09:03:36PM -0500, Chris J Arges wrote: > > rt6_uncached_list_flush_dev() currently walks every per-CPU uncached route > > list for each device being removed. Hash uncached routes by their inet6 > > device so ordinary device teardown only visits the matching bucket on each > > CPU. > > The code is doing something else and hashing using dst_dev(): > > struct net_device *rt_dev = dst_dev(&rt->dst); > [...] > ul = &table->buckets[hash_ptr(rt_dev, > > CONFIG_IPV6_UNCACHED_ROUTE_HASH_BITS)]; > > > > > ip6_rt_get_dev_rcu() can return loopback or an L3 master while rt6i_idev > > still refers to the original interface, so such a route must be reachable > > from either device. Place those routes on a separate per-CPU list that is > > always visited in addition to the keyed bucket. > > > > This avoids growing struct rt6_info while filtering most unrelated routes > > from ordinary device teardown. > > > > The table has 2^CONFIG_IPV6_UNCACHED_ROUTE_HASH_BITS buckets and defaults > > to 64. Larger values shorten each bucket, but every additional bit doubles > > the per-CPU memory used by the table. The default costs approximately > > 1.5 KiB per possible CPU on x86-64. > > > > Signed-off-by: Chris J Arges <[email protected]> > > --- > > net/ipv6/Kconfig | 13 +++++++ > > Same comment as in patch 1 about the Kconfig. > > > net/ipv6/route.c | 102 > > +++++++++++++++++++++++++++++++++++++------------------ > > 2 files changed, 82 insertions(+), 33 deletions(-) > > > > diff --git a/net/ipv6/Kconfig b/net/ipv6/Kconfig > > index c3806c6ac96f..0253178668bd 100644 > > --- a/net/ipv6/Kconfig > > +++ b/net/ipv6/Kconfig > > @@ -18,6 +18,19 @@ menuconfig IPV6 > > > > if IPV6 > > > > +config IPV6_UNCACHED_ROUTE_HASH_BITS > > + int "IPv6 uncached route hash bits" > > + range 1 10 > > + default 6 > > + help > > + This option sets the number of buckets used in the IPv6 uncached > > + route hash table to 2^IPV6_UNCACHED_ROUTE_HASH_BITS buckets. The > > + allowed values select between 2 and 1024 buckets. Larger values > > + reduce collisions, but each additional bit doubles the per-CPU > > + memory used by the table. > > + > > + If unsure, use the default of 6 bits (64 buckets). > > + > > config IPV6_ROUTER_PREF > > bool "IPv6: Router Preference (RFC 4191) support" > > help > > diff --git a/net/ipv6/route.c b/net/ipv6/route.c > > index 7535b09068a0..080dce329168 100644 > > --- a/net/ipv6/route.c > > +++ b/net/ipv6/route.c > > @@ -40,6 +40,7 @@ > > #include <linux/seq_file.h> > > #include <linux/nsproxy.h> > > #include <linux/slab.h> > > +#include <linux/hash.h> > > #include <linux/jhash.h> > > #include <linux/siphash.h> > > #include <net/net_namespace.h> > > @@ -133,11 +134,27 @@ struct uncached_list { > > struct list_head head; > > }; > > > > -static DEFINE_PER_CPU_ALIGNED(struct uncached_list, rt6_uncached_list); > > +#define RT6_UNCACHED_HASH_SIZE > > BIT(CONFIG_IPV6_UNCACHED_ROUTE_HASH_BITS) > > + > > +struct rt6_uncached_table { > > + struct uncached_list buckets[RT6_UNCACHED_HASH_SIZE]; > > + /* Routes that must be discoverable through two different devices. */ > > + struct uncached_list mismatch; > > +}; > > + > > +static DEFINE_PER_CPU_ALIGNED(struct rt6_uncached_table, > > rt6_uncached_table); > > > > void rt6_uncached_list_add(struct rt6_info *rt) > > { > > - struct uncached_list *ul = raw_cpu_ptr(&rt6_uncached_list); > > + struct rt6_uncached_table *table = raw_cpu_ptr(&rt6_uncached_table); > > + struct net_device *rt_dev = dst_dev(&rt->dst); > > + struct uncached_list *ul; > > + > > + if (rt->rt6i_idev && rt->rt6i_idev->dev != rt_dev) > > + ul = &table->mismatch; > > + else > > + ul = &table->buckets[hash_ptr(rt_dev, > > + > > CONFIG_IPV6_UNCACHED_ROUTE_HASH_BITS)]; > > > > rt->dst.rt_uncached_list = ul; > > > > @@ -157,40 +174,51 @@ void rt6_uncached_list_del(struct rt6_info *rt) > > } > > } > > > > +static void rt6_uncached_list_flush(struct uncached_list *ul, > > + struct net_device *dev) > > +{ > > + struct rt6_info *rt, *safe; > > + > > + if (list_empty(&ul->head)) > > + return; > > + > > + spin_lock_bh(&ul->lock); > > + list_for_each_entry_safe(rt, safe, &ul->head, dst.rt_uncached) { > > + struct inet6_dev *rt_idev = rt->rt6i_idev; > > + struct net_device *rt_dev = dst_dev(&rt->dst); > > + bool handled = false; > > https://docs.kernel.org/next/process/maintainer-netdev.html#local-variable-ordering-reverse-xmas-tree-rcs > > > + > > + if (rt_idev && rt_idev->dev == dev) { > > + rt->rt6i_idev = in6_dev_get(blackhole_netdev); > > + in6_dev_put(rt_idev); > > + handled = true; > > + } > > + > > + if (rt_dev == dev) { > > + rt->dst.dev = blackhole_netdev; > > Please use: > > rcu_assign_pointer(rt->dst.dev_rcu, blackhole_netdev); > > See commit 1469773b246a ("ipv4: use rcu_assign_pointer() in > rt_flush_dev()") > > > + netdev_ref_replace(rt_dev, blackhole_netdev, > > + &rt->dst.dev_tracker, GFP_ATOMIC); > > + handled = true; > > + } > > + if (handled) > > + list_del_init(&rt->dst.rt_uncached); > > + } > > + spin_unlock_bh(&ul->lock); > > +} > > + > > static void rt6_uncached_list_flush_dev(struct net_device *dev) > > { > > int cpu; > > > > for_each_possible_cpu(cpu) { > > - struct uncached_list *ul = per_cpu_ptr(&rt6_uncached_list, cpu); > > - struct rt6_info *rt, *safe; > > + struct rt6_uncached_table *table; > > + struct uncached_list *ul; > > > > - if (list_empty(&ul->head)) > > - continue; > > - > > - spin_lock_bh(&ul->lock); > > - list_for_each_entry_safe(rt, safe, &ul->head, dst.rt_uncached) { > > - struct inet6_dev *rt_idev = rt->rt6i_idev; > > - struct net_device *rt_dev = rt->dst.dev; > > - bool handled = false; > > - > > - if (rt_idev && rt_idev->dev == dev) { > > - rt->rt6i_idev = in6_dev_get(blackhole_netdev); > > - in6_dev_put(rt_idev); > > - handled = true; > > - } > > - > > - if (rt_dev == dev) { > > - rt->dst.dev = blackhole_netdev; > > - netdev_ref_replace(rt_dev, blackhole_netdev, > > - &rt->dst.dev_tracker, > > - GFP_ATOMIC); > > - handled = true; > > - } > > - if (handled) > > - list_del_init(&rt->dst.rt_uncached); > > - } > > - spin_unlock_bh(&ul->lock); > > + table = per_cpu_ptr(&rt6_uncached_table, cpu); > > + ul = &table->buckets[hash_ptr(dev, > > + > > CONFIG_IPV6_UNCACHED_ROUTE_HASH_BITS)]; > > + rt6_uncached_list_flush(ul, dev); > > + rt6_uncached_list_flush(&table->mismatch, dev); > > The mismatch list can be quite long depending on the workload and every > device needs to walk it for every CPU. > > AFAICT, when there is a mismatch, dst_dev() is either loopback or a VRF > device. Can you instead hash based on rt6i_idev->dev (fallback to > dst_dev() when not available) and only iterate over all the buckets when > the device that is going away is loopback / VRF? > > That way, in the common case, you only need to walk one list per-CPU.
Ido, Yea I like this approach much better. Thank you for the reviews. I've re-tested and implemented your feedback into v3: https://lore.kernel.org/all/20260917-hash-bucket-route-lists-v3-0-30493a37b...@cloudflare.com/ --chris
