Introduce tracking of bpf program stack depth in the verifier and use that info to reduce bpf program stack consumption in the interpreter and x64 JIT. Other JITs can take advantage of it as well in the future. Most of the programs consume very little stack, so it's good optimization in general and it's the first step toward bpf to bpf function calls.
Also use internal opcode for bpf_tail_call() marking to make clear that jmp|call|x opcode is not uapi and may be used for actual indirect call opcode in the future. Alexei Starovoitov (9): bpf: free up BPF_JMP | BPF_CALL | BPF_X opcode bpf: split bpf core interpreter bpf: teach verifier to track stack depth bpf: reconcile bpf_tail_call and stack_depth bpf: track stack depth of classic bpf programs bpf: fix stack_depth usage by test_bpf.ko bpf: use different interpreter depending on required stack size bpf: change x86 JITed program stack layout bpf: take advantage of stack_depth tracking in x64 JIT arch/arm64/net/bpf_jit_comp.c | 2 +- arch/powerpc/net/bpf_jit_comp64.c | 2 +- arch/s390/net/bpf_jit_comp.c | 2 +- arch/sparc/net/bpf_jit_comp_64.c | 2 +- arch/x86/net/bpf_jit.S | 20 ++++++------ arch/x86/net/bpf_jit_comp.c | 65 +++++++++++++++++++++------------------ include/linux/bpf.h | 1 + include/linux/filter.h | 3 ++ kernel/bpf/core.c | 47 ++++++++++++++++++++++------ kernel/bpf/verifier.c | 13 ++++++-- lib/test_bpf.c | 25 ++++++++++++++- net/core/filter.c | 36 +++++++++++++--------- 12 files changed, 147 insertions(+), 71 deletions(-) -- 2.9.3
