Hi Stephen,
On 10/14/2026 11:55 PM, Stephen Hemminger wrote:
On Wed, 14 Oct 2026 18:39:14 +0000
Vladimir Medvedkin <[email protected]> wrote:
This series adds multi-VRF support to the IPv4 FIB, allowing a single FIB
instance to host multiple isolated routing tables.
Today a FIB instance is a single routing table, so an application serving
several VRFs needs one FIB per VRF. In a burst oriented datapath packets of
one burst can belong to different VRFs, and the application either does
per-packet lookups in different FIBs or regroups the burst by VRF before
the lookup. Both are expensive. With this series all the VRFs live in one
FIB, the bulk lookup takes a VRF ID per address, and the existing API
operates on VRF 0.
v1 replicated the 16M entry tbl24 per VRF. That kept the lookup on par with
the single VRF case, but memory grew linearly with the number of VRFs,
which was the main concern raised on v1. v2 resolves the issue by adding
of two dataplane algorithms with different memory/lookup trade-offs.
DIR8-16-8 is a new three level variant of DIR-24-8 with 8, 16 and 8 bit
strides. Only the 256 entry root table (2 KB) is replicated per VRF; the
tbl16 and tbl8 groups come from pools shared by all the VRFs, so memory
follows the routes actually installed rather than the number of VRFs. The
cost is an extra memory access for prefixes longer than /8.
DIR-24-8 gets multi-VRF support by resolving the VRF after the longest
prefix match, along the lines Konstantin suggested on v1. Instead of a
vector of next hops per prefix, a single tbl24_8 shared by all the VRFs
returns an internal prefix ID, and a hash table keyed by {VRF, prefix ID}
returns the next hop, a miss meaning the VRF default next hop. The aim is
to resolve the VRF with a single memory access, so buckets hold 8 records
and there is no probing beyond the bucket.
The problem with existing FIB lookups is they are space intensive.
comparing to what?
They tradeoff memory for cache misses. Every level of indirection adds
another cache miss.
This is not always a case. Also that's why we are using prefetching.
Not sure they work well for VRF case at all.
I suspect this objection is applied for v1? v2 should resolve the issue
Normal LPM is more compact.
more compact compared to what? Under "Normal LPM" I assume rte_lpm
library, if so, then comparing to existing FIB, which uses DIR24_8 with
4 byte nexthops (apples to apples), the memory footprint is exactly the
same.
For normal mortal size machines having multiple VRF's is going to be huge.
How big are you envisioning this?
From my experience when I was working as a network engineer in network
operators, an average PE router could have 2 big VRFs (about 800k
prefixes for v4) and thousands small VRFs with about hundreds prefixes each.
With this input an average size of a dataplane struct with:
DIR8_16_8 could be around 2Kb*1000 (VRFs) + 2*256*64k*8b(TBL16 for 2
VRFs with full view) + 1000*4*64k*8b(small VRFs needs around 4 TBL16 - 3
for RFC1918 + 1 customer public net) + 1000*100(prefixes more specific
than /24, off the top of my head)*256*8b. This corresponds to TBL0: 2Mb,
TBL16: ~2.5Gb, TBL8:~200Mb, so under 3Gb of RAM.
DIR24_8: single DIR24_8 with joint prefixes for all VRFs and 4b entries
- 64mb for TBL24 + a few Mb for TBL8 (since all prefixes are joint) +
hash table of size from 8Mb to 1Gb.
This sounds like mortal machine RAM usage to me.
--
Regards,
Vladimir
