Hi David,
The following pull-request contains BPF updates for your *net-next* tree.
The main changes are:
1) Teach verifier dead code removal, this also allows for optimizing /
removing conditional branches around dead code and to shrink the
resulting image. Code store constrained architectures like nfp would
have hard time doing this at JIT level, from Jakub.
2) Add JMP32 instructions to BPF ISA in order to allow for optimizing
code generation for 32-bit sub-registers. Evaluation shows that this
can result in code reduction of ~5-20% compared to 64 bit-only code
generation. Also add implementation for most JITs, from Jiong.
3) Add support for __int128 types in BTF which is also needed for
vmlinux's BTF conversion to work, from Yonghong.
4) Add a new command to bpftool in order to dump a list of BPF-related
parameters from the system or for a specific network device e.g. in
terms of available prog/map types or helper functions, from Quentin.
5) Add AF_XDP sock_diag interface for querying sockets from user
space which provides information about the RX/TX/fill/completion
rings, umem, memory usage etc, from Björn.
6) Add skb context access for skb_shared_info->gso_segs field, from Eric.
7) Add support for testing flow dissector BPF programs by extending
existing BPF_PROG_TEST_RUN infrastructure, from Stanislav.
8) Split BPF kselftest's test_verifier into various subgroups of tests
in order better deal with merge conflicts in this area, from Jakub.
9) Add support for queue/stack manipulations in bpftool, from Stanislav.
10) Document BTF, from Yonghong.
11) Dump supported ELF section names in libbpf on program load
failure, from Taeung.
12) Silence a false positive compiler warning in verifier's BTF
handling, from Peter.
13) Fix help string in bpftool's feature probing, from Prashant.
14) Remove duplicate includes in BPF kselftests, from Yue.
Please consider pulling these changes from:
git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
Thanks a lot!
----------------------------------------------------------------
The following changes since commit ae5220c672180765615458ae54dbcff9abe6a01d:
networking: Documentation: fix snmp_counters.rst Sphinx warnings (2019-01-16
13:29:54 -0800)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
for you to fetch changes up to 3d2af27a84a8474e510f5d8362303bfbee946308:
Merge branch 'bpf-flow-dissector-tests' (2019-01-29 01:08:30 +0100)
----------------------------------------------------------------
Alexei Starovoitov (4):
Merge branch 'bpftool-probes'
Merge branch 'dead-code-elimination'
Merge branch 'jmp32-insns'
Merge branch 'split-test_verifier'
Björn Töpel (3):
net: xsk: track AF_XDP sockets on a per-netns list
xsk: add id to umem
xsk: add sock_diag interface for AF_XDP
Daniel Borkmann (4):
Merge branch 'bpf-int128-btf'
Merge branch 'bpf-bpftool-queue-stack'
Merge branch 'af-xdp-sock-diag'
Merge branch 'bpf-flow-dissector-tests'
Eric Dumazet (1):
bpf: allow BPF programs access skb_shared_info->gso_segs field
Jakub Kicinski (16):
bpf: change parameters of call/branch offset adjustment
bpf: verifier: hard wire branches to dead code
bpf: verifier: remove dead code
bpf: verifier: remove unconditional branches by 0
selftests: bpf: add tests for dead code removal
bpf: verifier: record original instruction index
bpf: notify offload JITs about optimizations
nfp: bpf: don't use instruction number for jump target
nfp: bpf: split up the skip flag
nfp: bpf: save original program length
nfp: bpf: support optimizing dead branches
nfp: bpf: support removing dead code
selftests: bpf: prepare for break up of verifier tests
selftests: bpf: break up test_verifier
selftests: bpf: break up the rest of test_verifier
tools: bpftool: warn about risky prog array updates
Jiong Wang (16):
bpf: allocate 0x06 to new eBPF instruction class JMP32
bpf: refactor verifier min/max code for condition jump
bpf: verifier support JMP32
bpf: disassembler support JMP32
tools: bpftool: teach cfg code about JMP32
bpf: interpreter support for JMP32
bpf: JIT blinds support JMP32
x86_64: bpf: implement jitting of JMP32
x32: bpf: implement jitting of JMP32
arm64: bpf: implement jitting of JMP32
arm: bpf: implement jitting of JMP32
ppc: bpf: implement jitting of JMP32
s390: bpf: implement jitting of JMP32
nfp: bpf: implement jitting of JMP32
selftests: bpf: functional and min/max reasoning unit tests for JMP32
selftests: bpf: makefile support sub-register code-gen test mode
Peter Oskolkov (1):
bpf: fix a (false) compiler warning
Prashant Bhole (1):
bpftool: feature probing, change default action
Quentin Monnet (9):
tools: bpftool: add basic probe capability, probe syscall availability
tools: bpftool: add probes for /proc/ eBPF parameters
tools: bpftool: add probes for kernel configuration options
tools: bpftool: add probes for eBPF program types
tools: bpftool: add probes for eBPF map types
tools: bpftool: add probes for eBPF helper functions
tools: bpftool: add C-style "#define" output for probes
tools: bpftool: add probes for a network device
tools: bpftool: add bash completion for bpftool probes
Stanislav Fomichev (13):
libbpf: don't define CC and AR
bpftool: make key and value optional in update command
bpftool: make key optional in lookup command
bpftool: don't print empty key/value for maps
bpftool: add peek command
bpftool: add push and enqueue commands
bpftool: add pop and dequeue commands
bpftool: add bash completion for peek/push/enqueue/pop/dequeue
selftests/bpf: don't hardcode iptables/nc path in test_tcpnotify_user
selftests/bpf: suppress readelf stderr when probing for BTF support
net/flow_dissector: move bpf case into __skb_flow_bpf_dissect
bpf: add BPF_PROG_TEST_RUN support for flow dissector
selftests/bpf: add simple BPF_PROG_TEST_RUN examples for flow dissector
Taeung Song (1):
libbpf: Show supported ELF section names when failing to guess
prog/attach type
Yonghong Song (6):
bpf: btf: support 128 bit integer type
tools/bpf: add int128 raw test in test_btf
tools/bpf: refactor test_btf pretty printing for multiple map value
formats
tools/bpf: add bpffs pretty print test for int128
tools/bpf: support __int128 in bpftool map pretty dumper
bpf: btf: add btf documentation
YueHaibing (1):
selftests: bpf: remove duplicated include
Documentation/bpf/btf.rst | 870 ++
Documentation/bpf/index.rst | 7 +
Documentation/networking/filter.txt | 15 +-
arch/arm/net/bpf_jit_32.c | 53 +-
arch/arm/net/bpf_jit_32.h | 2 +
arch/arm64/net/bpf_jit_comp.c | 37 +-
arch/powerpc/include/asm/ppc-opcode.h | 1 +
arch/powerpc/net/bpf_jit.h | 4 +
arch/powerpc/net/bpf_jit_comp64.c | 120 +-
arch/s390/net/bpf_jit_comp.c | 66 +-
arch/x86/net/bpf_jit_comp.c | 46 +-
arch/x86/net/bpf_jit_comp32.c | 121 +-
drivers/net/ethernet/netronome/nfp/bpf/jit.c | 139 +-
drivers/net/ethernet/netronome/nfp/bpf/main.h | 51 +-
drivers/net/ethernet/netronome/nfp/bpf/offload.c | 9 +-
drivers/net/ethernet/netronome/nfp/bpf/verifier.c | 74 +-
include/linux/bpf.h | 10 +
include/linux/bpf_verifier.h | 6 +
include/linux/filter.h | 21 +
include/linux/skbuff.h | 5 +
include/net/net_namespace.h | 4 +
include/net/netns/xdp.h | 13 +
include/net/xdp_sock.h | 1 +
include/uapi/linux/bpf.h | 2 +
include/uapi/linux/xdp_diag.h | 72 +
kernel/bpf/btf.c | 104 +-
kernel/bpf/core.c | 273 +-
kernel/bpf/disasm.c | 34 +-
kernel/bpf/offload.c | 35 +
kernel/bpf/verifier.c | 624 +-
net/bpf/test_run.c | 82 +
net/core/filter.c | 22 +
net/core/flow_dissector.c | 92 +-
net/xdp/Kconfig | 8 +
net/xdp/Makefile | 1 +
net/xdp/xdp_umem.c | 13 +
net/xdp/xsk.c | 36 +-
net/xdp/xsk.h | 12 +
net/xdp/xsk_diag.c | 191 +
samples/bpf/bpf_insn.h | 20 +
tools/bpf/bpftool/Documentation/bpftool-cgroup.rst | 1 +
.../bpf/bpftool/Documentation/bpftool-feature.rst | 85 +
tools/bpf/bpftool/Documentation/bpftool-map.rst | 29 +-
tools/bpf/bpftool/Documentation/bpftool-net.rst | 1 +
tools/bpf/bpftool/Documentation/bpftool-perf.rst | 1 +
tools/bpf/bpftool/Documentation/bpftool-prog.rst | 1 +
tools/bpf/bpftool/Documentation/bpftool.rst | 1 +
tools/bpf/bpftool/bash-completion/bpftool | 110 +-
tools/bpf/bpftool/btf_dumper.c | 98 +-
tools/bpf/bpftool/cfg.c | 9 +-
tools/bpf/bpftool/feature.c | 764 +
tools/bpf/bpftool/main.c | 3 +-
tools/bpf/bpftool/main.h | 4 +
tools/bpf/bpftool/map.c | 232 +-
tools/bpf/bpftool/prog.c | 10 +-
tools/include/linux/filter.h | 20 +
tools/include/uapi/linux/bpf.h | 2 +
tools/lib/bpf/Build | 2 +-
tools/lib/bpf/Makefile | 17 +-
tools/lib/bpf/libbpf.c | 46 +-
tools/lib/bpf/libbpf.h | 14 +
tools/lib/bpf/libbpf.map | 7 +
tools/lib/bpf/libbpf_probes.c | 242 +
tools/testing/selftests/bpf/Makefile | 112 +-
tools/testing/selftests/bpf/flow_dissector_load.c | 43 +-
tools/testing/selftests/bpf/flow_dissector_load.h | 55 +
tools/testing/selftests/bpf/test_btf.c | 680 +-
tools/testing/selftests/bpf/test_flow_dissector.c | 2 -
tools/testing/selftests/bpf/test_maps.c | 1 -
tools/testing/selftests/bpf/test_progs.c | 78 +-
tools/testing/selftests/bpf/test_socket_cookie.c | 4 +-
tools/testing/selftests/bpf/test_sockmap.c | 1 -
tools/testing/selftests/bpf/test_tcpnotify_user.c | 6 +-
tools/testing/selftests/bpf/test_verifier.c | 15426 +------------------
tools/testing/selftests/bpf/verifier/.gitignore | 1 +
tools/testing/selftests/bpf/verifier/and.c | 50 +
.../testing/selftests/bpf/verifier/array_access.c | 219 +
tools/testing/selftests/bpf/verifier/basic.c | 23 +
tools/testing/selftests/bpf/verifier/basic_call.c | 50 +
tools/testing/selftests/bpf/verifier/basic_instr.c | 134 +
tools/testing/selftests/bpf/verifier/basic_stack.c | 64 +
.../testing/selftests/bpf/verifier/basic_stx_ldx.c | 45 +
tools/testing/selftests/bpf/verifier/bounds.c | 508 +
.../selftests/bpf/verifier/bounds_deduction.c | 124 +
.../bpf/verifier/bounds_mix_sign_unsign.c | 406 +
.../testing/selftests/bpf/verifier/bpf_get_stack.c | 44 +
tools/testing/selftests/bpf/verifier/calls.c | 1942 +++
tools/testing/selftests/bpf/verifier/cfg.c | 70 +
.../selftests/bpf/verifier/cgroup_inv_retcode.c | 72 +
tools/testing/selftests/bpf/verifier/cgroup_skb.c | 197 +
.../selftests/bpf/verifier/cgroup_storage.c | 220 +
tools/testing/selftests/bpf/verifier/const_or.c | 60 +
tools/testing/selftests/bpf/verifier/ctx.c | 93 +
tools/testing/selftests/bpf/verifier/ctx_sk_msg.c | 180 +
tools/testing/selftests/bpf/verifier/ctx_skb.c | 1033 ++
tools/testing/selftests/bpf/verifier/dead_code.c | 159 +
.../selftests/bpf/verifier/direct_packet_access.c | 633 +
.../bpf/verifier/direct_stack_access_wraparound.c | 40 +
tools/testing/selftests/bpf/verifier/div0.c | 184 +
.../testing/selftests/bpf/verifier/div_overflow.c | 104 +
.../selftests/bpf/verifier/helper_access_var_len.c | 614 +
.../selftests/bpf/verifier/helper_packet_access.c | 460 +
.../selftests/bpf/verifier/helper_value_access.c | 953 ++
tools/testing/selftests/bpf/verifier/jit.c | 88 +
tools/testing/selftests/bpf/verifier/jmp32.c | 724 +
tools/testing/selftests/bpf/verifier/jset.c | 165 +
tools/testing/selftests/bpf/verifier/jump.c | 180 +
tools/testing/selftests/bpf/verifier/junk_insn.c | 45 +
tools/testing/selftests/bpf/verifier/ld_abs.c | 286 +
tools/testing/selftests/bpf/verifier/ld_dw.c | 36 +
tools/testing/selftests/bpf/verifier/ld_imm64.c | 141 +
tools/testing/selftests/bpf/verifier/ld_ind.c | 72 +
tools/testing/selftests/bpf/verifier/leak_ptr.c | 67 +
tools/testing/selftests/bpf/verifier/lwt.c | 189 +
tools/testing/selftests/bpf/verifier/map_in_map.c | 62 +
.../selftests/bpf/verifier/map_ptr_mixing.c | 100 +
tools/testing/selftests/bpf/verifier/map_ret_val.c | 65 +
tools/testing/selftests/bpf/verifier/masking.c | 322 +
tools/testing/selftests/bpf/verifier/meta_access.c | 235 +
.../bpf/verifier/perf_event_sample_period.c | 59 +
.../selftests/bpf/verifier/prevent_map_lookup.c | 74 +
tools/testing/selftests/bpf/verifier/raw_stack.c | 305 +
.../testing/selftests/bpf/verifier/ref_tracking.c | 607 +
tools/testing/selftests/bpf/verifier/runtime_jit.c | 80 +
.../selftests/bpf/verifier/search_pruning.c | 156 +
tools/testing/selftests/bpf/verifier/spill_fill.c | 75 +
tools/testing/selftests/bpf/verifier/stack_ptr.c | 317 +
tools/testing/selftests/bpf/verifier/uninit.c | 39 +
tools/testing/selftests/bpf/verifier/unpriv.c | 521 +
tools/testing/selftests/bpf/verifier/value.c | 104 +
.../selftests/bpf/verifier/value_adj_spill.c | 43 +
.../selftests/bpf/verifier/value_illegal_alu.c | 94 +
.../testing/selftests/bpf/verifier/value_or_null.c | 152 +
.../selftests/bpf/verifier/value_ptr_arith.c | 792 +
tools/testing/selftests/bpf/verifier/var_off.c | 66 +
tools/testing/selftests/bpf/verifier/xadd.c | 97 +
tools/testing/selftests/bpf/verifier/xdp.c | 14 +
.../bpf/verifier/xdp_direct_packet_access.c | 900 ++
138 files changed, 21229 insertions(+), 16128 deletions(-)
create mode 100644 Documentation/bpf/btf.rst
create mode 100644 include/net/netns/xdp.h
create mode 100644 include/uapi/linux/xdp_diag.h
create mode 100644 net/xdp/xsk.h
create mode 100644 net/xdp/xsk_diag.c
create mode 100644 tools/bpf/bpftool/Documentation/bpftool-feature.rst
create mode 100644 tools/bpf/bpftool/feature.c
create mode 100644 tools/lib/bpf/libbpf_probes.c
create mode 100644 tools/testing/selftests/bpf/flow_dissector_load.h
create mode 100644 tools/testing/selftests/bpf/verifier/.gitignore
create mode 100644 tools/testing/selftests/bpf/verifier/and.c
create mode 100644 tools/testing/selftests/bpf/verifier/array_access.c
create mode 100644 tools/testing/selftests/bpf/verifier/basic.c
create mode 100644 tools/testing/selftests/bpf/verifier/basic_call.c
create mode 100644 tools/testing/selftests/bpf/verifier/basic_instr.c
create mode 100644 tools/testing/selftests/bpf/verifier/basic_stack.c
create mode 100644 tools/testing/selftests/bpf/verifier/basic_stx_ldx.c
create mode 100644 tools/testing/selftests/bpf/verifier/bounds.c
create mode 100644 tools/testing/selftests/bpf/verifier/bounds_deduction.c
create mode 100644
tools/testing/selftests/bpf/verifier/bounds_mix_sign_unsign.c
create mode 100644 tools/testing/selftests/bpf/verifier/bpf_get_stack.c
create mode 100644 tools/testing/selftests/bpf/verifier/calls.c
create mode 100644 tools/testing/selftests/bpf/verifier/cfg.c
create mode 100644 tools/testing/selftests/bpf/verifier/cgroup_inv_retcode.c
create mode 100644 tools/testing/selftests/bpf/verifier/cgroup_skb.c
create mode 100644 tools/testing/selftests/bpf/verifier/cgroup_storage.c
create mode 100644 tools/testing/selftests/bpf/verifier/const_or.c
create mode 100644 tools/testing/selftests/bpf/verifier/ctx.c
create mode 100644 tools/testing/selftests/bpf/verifier/ctx_sk_msg.c
create mode 100644 tools/testing/selftests/bpf/verifier/ctx_skb.c
create mode 100644 tools/testing/selftests/bpf/verifier/dead_code.c
create mode 100644 tools/testing/selftests/bpf/verifier/direct_packet_access.c
create mode 100644
tools/testing/selftests/bpf/verifier/direct_stack_access_wraparound.c
create mode 100644 tools/testing/selftests/bpf/verifier/div0.c
create mode 100644 tools/testing/selftests/bpf/verifier/div_overflow.c
create mode 100644 tools/testing/selftests/bpf/verifier/helper_access_var_len.c
create mode 100644 tools/testing/selftests/bpf/verifier/helper_packet_access.c
create mode 100644 tools/testing/selftests/bpf/verifier/helper_value_access.c
create mode 100644 tools/testing/selftests/bpf/verifier/jit.c
create mode 100644 tools/testing/selftests/bpf/verifier/jmp32.c
create mode 100644 tools/testing/selftests/bpf/verifier/jset.c
create mode 100644 tools/testing/selftests/bpf/verifier/jump.c
create mode 100644 tools/testing/selftests/bpf/verifier/junk_insn.c
create mode 100644 tools/testing/selftests/bpf/verifier/ld_abs.c
create mode 100644 tools/testing/selftests/bpf/verifier/ld_dw.c
create mode 100644 tools/testing/selftests/bpf/verifier/ld_imm64.c
create mode 100644 tools/testing/selftests/bpf/verifier/ld_ind.c
create mode 100644 tools/testing/selftests/bpf/verifier/leak_ptr.c
create mode 100644 tools/testing/selftests/bpf/verifier/lwt.c
create mode 100644 tools/testing/selftests/bpf/verifier/map_in_map.c
create mode 100644 tools/testing/selftests/bpf/verifier/map_ptr_mixing.c
create mode 100644 tools/testing/selftests/bpf/verifier/map_ret_val.c
create mode 100644 tools/testing/selftests/bpf/verifier/masking.c
create mode 100644 tools/testing/selftests/bpf/verifier/meta_access.c
create mode 100644
tools/testing/selftests/bpf/verifier/perf_event_sample_period.c
create mode 100644 tools/testing/selftests/bpf/verifier/prevent_map_lookup.c
create mode 100644 tools/testing/selftests/bpf/verifier/raw_stack.c
create mode 100644 tools/testing/selftests/bpf/verifier/ref_tracking.c
create mode 100644 tools/testing/selftests/bpf/verifier/runtime_jit.c
create mode 100644 tools/testing/selftests/bpf/verifier/search_pruning.c
create mode 100644 tools/testing/selftests/bpf/verifier/spill_fill.c
create mode 100644 tools/testing/selftests/bpf/verifier/stack_ptr.c
create mode 100644 tools/testing/selftests/bpf/verifier/uninit.c
create mode 100644 tools/testing/selftests/bpf/verifier/unpriv.c
create mode 100644 tools/testing/selftests/bpf/verifier/value.c
create mode 100644 tools/testing/selftests/bpf/verifier/value_adj_spill.c
create mode 100644 tools/testing/selftests/bpf/verifier/value_illegal_alu.c
create mode 100644 tools/testing/selftests/bpf/verifier/value_or_null.c
create mode 100644 tools/testing/selftests/bpf/verifier/value_ptr_arith.c
create mode 100644 tools/testing/selftests/bpf/verifier/var_off.c
create mode 100644 tools/testing/selftests/bpf/verifier/xadd.c
create mode 100644 tools/testing/selftests/bpf/verifier/xdp.c
create mode 100644
tools/testing/selftests/bpf/verifier/xdp_direct_packet_access.c
