Changelog:
This series changes the xdp_redirect helper to use a hidden default map. The
redirect_map() helper also uses the map structure to batch packets, which
results in a significant (around 50%) performance boost for the _map variant.
However, the xdp_redirect() API is simpler if one just wants to redirect to
another interface, which means people tend to use this interface and then wonder
why they getter worse performance than expected.
This series seeks to close this performance difference between the two APIs. It
achieves this by changing xdp_redirect() to use a hidden devmap for looking up
destination interfaces, thus gaining the batching benefit with no visible
difference from the user API point of view.
Allocation of the default map is done dynamically as programs using the
xdp_redirect helper are loaded and attached to interfaces, and the maps are
freed again when no longer needed. Because of tail calls, this requires two
levels of refcounting: One global level that keeps track of whether any XDP
programs using the xdp_redirect() helper are loaded in the system at all. And
another that keeps track of whether any programs that could potentially result
in a call to xdp_redirect (i.e., either programs using the helper, or programs
using tail calls) are loaded in a given namespace.
The default maps are dynamically sized to the nearest power-of-two size that can
contain all interfaces present in each interface. If allocation fails, the user
action that triggered the allocation (either attaching an XDP program, or moving
an interface with a program attached) is rejected. If new interfaces appear in
a namespace which causes the default map to become too small, a new one is
allocated with the correct size; this allocation is the only one that cannot
lead to a rejection of the userspace action, so if it fails a warning is emitted
instead.
The first patch in the series refactors devmap.c to prepare for the subsequent
patches. The second patch adds the default map handling using the existing
array-based devmap structure. The third patch adds a new map type (devmap_idx)
that hashes devices on ifindex.
v1 -> v2:
- Add refcounting to only allocate default maps when needed
- Using said refcounting, also deallocate default maps
- Add dynamic sizing of default maps
- Handle moving of interfaces between namespaces
- Split out refactoring of devmap.c to separate patch
- Use hashmap semantics for update_elem of devmap_idx type maps
---
Toke Høiland-Jørgensen (3):
xdp: Refactor devmap code in preparation for subsequent additions
xdp: Always use a devmap for XDP_REDIRECT to a device
xdp: Add devmap_idx map type for looking up devices by ifindex
include/linux/bpf.h | 46 ++
include/linux/bpf_types.h | 1
include/linux/filter.h | 2
include/net/net_namespace.h | 2
include/net/netns/xdp.h | 11 +
include/trace/events/xdp.h | 3
include/uapi/linux/bpf.h | 1
kernel/bpf/devmap.c | 609 +++++++++++++++++++++++++++----
kernel/bpf/syscall.c | 27 +
kernel/bpf/verifier.c | 14 +
net/core/dev.c | 59 +++
net/core/filter.c | 69 +---
tools/bpf/bpftool/map.c | 1
tools/include/uapi/linux/bpf.h | 1
tools/lib/bpf/libbpf_probes.c | 1
tools/testing/selftests/bpf/test_maps.c | 16 +
16 files changed, 720 insertions(+), 143 deletions(-)
