[PATCH bpf-next 00/17] Improve BPF syscall command documentation

2021-02-16 Thread Joe Stringer
From: Joe Stringer The state of bpf(2) manual pages today is not exactly ideal. For the most part, it was written several years ago and has not kept up with the pace of development in the kernel tree. For instance, out of a total of ~35 commands to the BPF syscall available today, when I pull

[PATCH bpf-next 03/17] bpf: Document BPF_F_LOCK in syscall commands

2021-02-16 Thread Joe Stringer
From: Joe Stringer Document the meaning of the BPF_F_LOCK flag for the map lookup/update descriptions. Based on commit 96049f3afd50 ("bpf: introduce BPF_F_LOCK flag"). Reviewed-by: Quentin Monnet Signed-off-by: Joe Stringer --- CC: Alexei Starovoitov --- include/uapi/linux/

[PATCH bpf-next 01/17] bpf: Import syscall arg documentation

2021-02-16 Thread Joe Stringer
From: Joe Stringer These descriptions are present in the man-pages project from the original submissions around 2015-2016. Import them so that they can be kept up to date as developers extend the bpf syscall commands. These descriptions follow the pattern used by scripts/bpf_helpers_doc.py so

[PATCH bpf-next 04/17] bpf: Document BPF_PROG_PIN syscall command

2021-02-16 Thread Joe Stringer
From: Joe Stringer Commit b2197755b263 ("bpf: add support for persistent maps/progs") contains the original implementation and git logs, used as reference for this documentation. Also pull in the filename restriction as documented in commit 6d8cb045cde6 ("bpf: comment why do

[PATCH bpf-next 02/17] bpf: Add minimal bpf() command documentation

2021-02-16 Thread Joe Stringer
From: Joe Stringer Introduce high-level descriptions of the intent and return codes of the bpf() syscall commands. Subsequent patches may further flesh out the content to provide a more useful programming reference. Reviewed-by: Quentin Monnet Signed-off-by: Joe Stringer --- include/uapi

[PATCH bpf-next 06/17] bpf: Document BPF_PROG_TEST_RUN syscall command

2021-02-16 Thread Joe Stringer
From: Joe Stringer Based on a brief read of the corresponding source code. Reviewed-by: Quentin Monnet Signed-off-by: Joe Stringer --- include/uapi/linux/bpf.h | 14 +++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux

[PATCH bpf-next 05/17] bpf: Document BPF_PROG_ATTACH syscall command

2021-02-16 Thread Joe Stringer
From: Joe Stringer Document the prog attach command in more detail, based on git commits: * commit f4324551489e ("bpf: add BPF_PROG_ATTACH and BPF_PROG_DETACH commands") * commit 4f738adba30a ("bpf: create tcp_bpf_ulp allowing BPF to monitor socket TX/RX data") * commit

[PATCH bpf-next 10/17] scripts/bpf: Abstract eBPF API target parameter

2021-02-16 Thread Joe Stringer
From: Joe Stringer Abstract out the target parameter so that upcoming commits, more than just the existing "helpers" target can be called to generate specific portions of docs from the eBPF UAPI headers. Reviewed-by: Quentin Monnet Signed-off-by: Joe Stringer --- scripts/bpf_d

[PATCH bpf-next 07/17] bpf: Document BPF_PROG_QUERY syscall command

2021-02-16 Thread Joe Stringer
From: Joe Stringer Commit 468e2f64d220 ("bpf: introduce BPF_PROG_QUERY command") originally introduced this, but there have been several additions since then. Unlike BPF_PROG_ATTACH, it appears that the sockmap progs are not able to be queried so far. Reviewed-by: Quentin Monnet Sig

[PATCH bpf-next 12/17] tools/bpf: Rename Makefile.{helpers,docs}

2021-02-16 Thread Joe Stringer
From: Joe Stringer In anticipation of including make targets for other manual pages in this makefile, rename it to something a bit more generic. Reviewed-by: Quentin Monnet Signed-off-by: Joe Stringer --- tools/bpf/{Makefile.helpers => Makefile.docs} | 2 +- tools/bpf/bpftool/Documentat

[PATCH bpf-next 14/17] tools/bpf: Build bpf-sycall.2 in Makefile.docs

2021-02-16 Thread Joe Stringer
From: Joe Stringer Add building of the bpf(2) syscall commands documentation as part of the docs building step in the build. This allows us to pick up on potential parse errors from the docs generator script as part of selftests. Reviewed-by: Quentin Monnet Signed-off-by: Joe Stringer

[PATCH bpf-next 08/17] bpf: Document BPF_MAP_*_BATCH syscall commands

2021-02-16 Thread Joe Stringer
From: Joe Stringer Based roughly on the following commits: * Commit cb4d03ab499d ("bpf: Add generic support for lookup batch op") * Commit 057996380a42 ("bpf: Add batch ops to all htab bpf map") * Commit aa2e93b8e58e ("bpf: Add generic support for update and delete

[PATCH bpf-next 16/17] docs/bpf: Add bpf() syscall command reference

2021-02-16 Thread Joe Stringer
From: Joe Stringer Generate the syscall command reference from the UAPI header file and include it in the main bpf docs page. Reviewed-by: Quentin Monnet Signed-off-by: Joe Stringer --- Documentation/Makefile | 2 ++ Documentation/bpf/Makefile | 28

[PATCH bpf-next 13/17] tools/bpf: Templatize man page generation

2021-02-16 Thread Joe Stringer
From: Joe Stringer Previously, the Makefile here was only targeting a single manual page so it just hardcoded a bunch of individual rules to specifically handle build, clean, install, uninstall for that particular page. Upcoming commits will generate manual pages for an additional section, so

[PATCH bpf-next 09/17] scripts/bpf: Rename bpf_helpers_doc.py -> bpf_doc.py

2021-02-16 Thread Joe Stringer
From: Joe Stringer Rename this file in anticipation of it being used for generating more than just helper man pages. Reviewed-by: Quentin Monnet Signed-off-by: Joe Stringer --- include/uapi/linux/bpf.h | 2 +- scripts/{bpf_helpers_doc.py => bpf_doc.py} | 4 ++-- tools/

[PATCH bpf-next 15/17] selftests/bpf: Add docs target

2021-02-16 Thread Joe Stringer
From: Joe Stringer This docs target will run the scripts/bpf_doc.py against the BPF UAPI headers to ensure that the parser used for generating manual pages from the headers doesn't trip on any newly added API documentation. While we're at it, remove the bpftool-specific docs check ta

[PATCH bpf-next 11/17] scripts/bpf: Add syscall commands printer

2021-02-16 Thread Joe Stringer
From: Joe Stringer Add a new target to bpf_doc.py to support generating the list of syscall commands directly from the UAPI headers. Assuming that developer submissions keep the main header up to date, this should allow the man pages to be automatically generated based on the latest API changes

[PATCH bpf-next 17/17] tools: Sync uapi bpf.h header with latest changes

2021-02-16 Thread Joe Stringer
From: Joe Stringer Synchronize the header after all of the recent changes. Reviewed-by: Quentin Monnet Signed-off-by: Joe Stringer --- tools/include/uapi/linux/bpf.h | 707 - 1 file changed, 706 insertions(+), 1 deletion(-) diff --git a/tools/include/uapi

Re: [PATCH bpf-next 00/17] Improve BPF syscall command documentation

2021-02-17 Thread Joe Stringer
On Wed, Feb 17, 2021 at 5:55 AM Toke Høiland-Jørgensen wrote: > > Joe Stringer writes: > > Given the relative success of the process around bpf-helpers(7) to > > encourage developers to document their user-facing changes, in this > > patch series I explore applying thi

Re: [PATCH bpf-next 00/17] Improve BPF syscall command documentation

2021-02-17 Thread Joe Stringer
On Wed, Feb 17, 2021 at 9:32 AM Jonathan Corbet wrote: > > [CC += linux-doc] > > Joe Stringer writes: > > > From: Joe Stringer > > > > The state of bpf(2) manual pages today is not exactly ideal. For the > > most part, it was written several years ago a

Re: [PATCH bpf-next 00/17] Improve BPF syscall command documentation

2021-02-18 Thread Joe Stringer
On Thu, Feb 18, 2021 at 11:49 AM Jonathan Corbet wrote: > > Joe Stringer writes: > > * The changes in patch 16 here extended Documentation/bpf/index.rst, > > but to assist in improving the overall kernel documentation > > organisation / hierarchy, you would prefe

Re: [PATCH bpf-next] libbpf: clarify flags in ringbuf helpers

2021-04-07 Thread Joe Stringer
Hi Pedro, On Tue, Apr 6, 2021 at 11:58 AM Pedro Tammela wrote: > > In 'bpf_ringbuf_reserve()' we require the flag to '0' at the moment. > > For 'bpf_ringbuf_{discard,submit,output}' a flag of '0' might send a > notification to the process if needed. > > Signed-off-by: Pedro Tammela > --- > incl

Re: [PATCH bpf-next] bpf: fix missing * in bpf.h

2021-03-02 Thread Joe Stringer
On Fri, Feb 26, 2021 at 8:51 AM Quentin Monnet wrote: > > 2021-02-24 10:59 UTC-0800 ~ Andrii Nakryiko > > On Wed, Feb 24, 2021 at 7:55 AM Daniel Borkmann > > wrote: > >> > >> On 2/23/21 3:43 PM, Jesper Dangaard Brouer wrote: > >>> On Tue, 23 Feb 2021 20:45:54 +0800 > >>> Hangbin Liu wrote: > >

Re: [PATCH] openvswitch: perform refragmentation for packets which pass through conntrack

2021-03-21 Thread Joe Stringer
Hey Aaron, long time no chat :) On Fri, Mar 19, 2021 at 1:43 PM Aaron Conole wrote: > > When a user instructs a flow pipeline to perform connection tracking, > there is an implicit L3 operation that occurs - namely the IP fragments > are reassembled and then processed as a single unit. After thi

Re: [RFC bpf-next 0/7] Programming socket lookup with BPF

2019-06-20 Thread Joe Stringer
On Wed, Jun 19, 2019 at 2:14 AM Jakub Sitnicki wrote: > > Hey Florian, > > Thanks for taking a look at it. > > On Tue, Jun 18, 2019 at 03:52 PM CEST, Florian Westphal wrote: > > Jakub Sitnicki wrote: > >> - XDP programs using bpf_sk_lookup helpers, like load balancers, can't > >>find the lis

Re: [RFC bpf-next 0/7] Programming socket lookup with BPF

2019-06-21 Thread Joe Stringer
On Fri, Jun 21, 2019 at 1:44 AM Jakub Sitnicki wrote: > > On Fri, Jun 21, 2019, 00:20 Joe Stringer wrote: >> >> On Wed, Jun 19, 2019 at 2:14 AM Jakub Sitnicki wrote: >> > >> > Hey Florian, >> > >> > Thanks for taking a look at it. &g

Removing skb_orphan() from ip_rcv_core()

2019-06-21 Thread Joe Stringer
Hi folks, picking this up again.. As discussed during LSFMM, I've been looking at adding something like an `skb_sk_assign()` helper to BPF so that logic similar to TPROXY can be implemented with integration into other BPF logic, however currently any attempts to do so are blocked by the skb_orphan

Re: Removing skb_orphan() from ip_rcv_core()

2019-06-24 Thread Joe Stringer
On Fri, Jun 21, 2019 at 1:59 PM Florian Westphal wrote: > > Joe Stringer wrote: > > As discussed during LSFMM, I've been looking at adding something like > > an `skb_sk_assign()` helper to BPF so that logic similar to TPROXY can > > be implemented with integration

Re: Removing skb_orphan() from ip_rcv_core()

2019-06-24 Thread Joe Stringer
On Mon, Jun 24, 2019 at 7:47 AM Jamal Hadi Salim wrote: > > On 2019-06-21 1:58 p.m., Joe Stringer wrote: > > Hi folks, picking this up again.. > [..] > > During LSFMM, it seemed like no-one knew quite why the skb_orphan() is > > necessary in that path in the current ve

Re: Removing skb_orphan() from ip_rcv_core()

2019-06-25 Thread Joe Stringer
On Mon, Jun 24, 2019 at 11:37 PM Eric Dumazet wrote: > On 6/24/19 8:17 PM, Joe Stringer wrote: > > On Fri, Jun 21, 2019 at 1:59 PM Florian Westphal wrote: > >> Joe Stringer wrote: > >>> However, if I drop these lines then I end up causing sockets to > >

Re: Removing skb_orphan() from ip_rcv_core()

2019-06-25 Thread Joe Stringer
On Tue, Jun 25, 2019 at 4:07 AM Jamal Hadi Salim wrote: > > On 2019-06-24 11:26 p.m., Joe Stringer wrote: > [..] > > > > I haven't got as far as UDP yet, but I didn't see any need for a > > dependency on netfilter. > > I'd be curious to see wh

[RFC bpf-next 02/11] bpf: Simplify ptr_min_max_vals adjustment

2018-05-09 Thread Joe Stringer
An upcoming commit will add another two pointer types that need very similar behaviour, so generalise this function now. Signed-off-by: Joe Stringer --- kernel/bpf/verifier.c | 22 ++ tools/testing/selftests/bpf/test_verifier.c | 14 +++--- 2

[RFC bpf-next 09/11] libbpf: Support loading individual progs

2018-05-09 Thread Joe Stringer
individually loaded, each program can be independently checked against its expected verification result. Signed-off-by: Joe Stringer --- tools/lib/bpf/libbpf.c | 4 ++-- tools/lib/bpf/libbpf.h | 3 +++ 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/tools/lib/bpf/libbpf.c b/tools

[RFC bpf-next 04/11] bpf: Add PTR_TO_SOCKET verifier type

2018-05-09 Thread Joe Stringer
Teach the verifier a little bit about a new type of pointer, a PTR_TO_SOCKET. This pointer type is accessed from BPF through the 'struct bpf_sock' structure. Signed-off-by: Joe Stringer --- include/linux/bpf.h | 19 +- include/linux/bpf_verifier.h | 2 ++ kernel/bpf/

[RFC bpf-next 10/11] selftests/bpf: Add C tests for reference tracking

2018-05-09 Thread Joe Stringer
Signed-off-by: Joe Stringer --- tools/testing/selftests/bpf/Makefile | 2 +- tools/testing/selftests/bpf/test_progs.c | 38 +++ tools/testing/selftests/bpf/test_sk_lookup_kern.c | 127 ++ 3 files changed, 166 insertions(+), 1 deletion(-) create

[RFC bpf-next 08/11] selftests/bpf: Add tests for reference tracking

2018-05-09 Thread Joe Stringer
_ptr spill into caller stack Signed-off-by: Joe Stringer --- tools/testing/selftests/bpf/test_verifier.c | 359 1 file changed, 359 insertions(+) diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c index 53439f40

[RFC bpf-next 05/11] bpf: Macrofy stack state copy

2018-05-09 Thread Joe Stringer
An upcoming commit will need very similar copy/realloc boilerplate, so refactor the existing stack copy/realloc functions into macros to simplify it. Signed-off-by: Joe Stringer --- kernel/bpf/verifier.c | 104 -- 1 file changed, 59 insertions

[RFC bpf-next 07/11] bpf: Add helper to retrieve socket in BPF

2018-05-09 Thread Joe Stringer
p. return TC_ACT_SHOT; } bpf_sk_release(sk, 0); return TC_ACT_OK; Signed-off-by: Joe Stringer --- include/uapi/linux/bpf.h | 39 +++- kernel/bpf/verifier.c | 8 ++- net/core/filter.c | 102

[RFC bpf-next 11/11] Documentation: Describe bpf reference tracking

2018-05-09 Thread Joe Stringer
Signed-off-by: Joe Stringer --- Documentation/networking/filter.txt | 64 + 1 file changed, 64 insertions(+) diff --git a/Documentation/networking/filter.txt b/Documentation/networking/filter.txt index 5032e1263bc9..77be17977bc5 100644 --- a/Documentation

[RFC bpf-next 00/11] Add socket lookup support

2018-05-09 Thread Joe Stringer
in future, we could encode an ordering preference in the flags field. * Currently this helper is only defined for TC hook point, but it should also be valid at XDP and perhaps some other hooks. Joe Stringer (11): bpf: Add iterator for spilled registers bpf: Simplify ptr_min_max_v

[RFC bpf-next 06/11] bpf: Add reference tracking to verifier

2018-05-09 Thread Joe Stringer
_state` and all existing copies of the pointer in registers are marked invalid. Signed-off-by: Joe Stringer --- include/linux/bpf_verifier.h | 18 ++- kernel/bpf/verifier.c| 295 --- 2 files changed, 292 insertions(+), 21 deletions(-) diff --git

[RFC bpf-next 01/11] bpf: Add iterator for spilled registers

2018-05-09 Thread Joe Stringer
Add this iterator for spilled registers, it concentrates the details of how to get the current frame's spilled registers into a single macro while clarifying the intention of the code which is calling the macro. Signed-off-by: Joe Stringer --- include/linux/bpf_verifier.h

[RFC bpf-next 03/11] bpf: Generalize ptr_or_null regs check

2018-05-09 Thread Joe Stringer
This check will be reused by an upcoming commit for conditional jump checks for sockets. Refactor it a bit to simplify the later commit. Signed-off-by: Joe Stringer --- kernel/bpf/verifier.c | 43 +-- 1 file changed, 25 insertions(+), 18 deletions

[PATCH bpf-next] selftests/bpf: Fix bash reference in Makefile

2018-05-10 Thread Joe Stringer
'|& ...' is a bash 4.0+ construct which is not guaranteed to be available when using '$(shell ...)' in a Makefile. Fall back to the more portable '2>&1 | ...'. Fixes the following warning during compilation: /bin/sh: 1: Syntax error: &q

Re: [RFC bpf-next 07/11] bpf: Add helper to retrieve socket in BPF

2018-05-11 Thread Joe Stringer
On 10 May 2018 at 22:00, Martin KaFai Lau wrote: > On Wed, May 09, 2018 at 02:07:05PM -0700, Joe Stringer wrote: >> This patch adds a new BPF helper function, sk_lookup() which allows BPF >> programs to find out if there is a socket listening on this host, and >> returns a

Re: [RFC bpf-next 07/11] bpf: Add helper to retrieve socket in BPF

2018-05-11 Thread Joe Stringer
On 11 May 2018 at 14:41, Martin KaFai Lau wrote: > On Fri, May 11, 2018 at 02:08:01PM -0700, Joe Stringer wrote: >> On 10 May 2018 at 22:00, Martin KaFai Lau wrote: >> > On Wed, May 09, 2018 at 02:07:05PM -0700, Joe Stringer wrote: >> >> This patch adds a new

Re: [ovs-dev] openvswitch crash on i386

2019-03-05 Thread Joe Stringer
pecifying which > > conntrack events (IPCT_*) should be delivered via the Netfilter > > netlink multicast groups. Default behavior depends on the system > > configuration, but typically a lot of events are delivered. This can be > > very chatty for the NF

Re: RFC: Fixing SK_REUSEPORT from sk_lookup_* helpers

2019-05-15 Thread Joe Stringer
On Wed, May 15, 2019 at 8:11 AM Lorenz Bauer wrote: > > In the BPF-based TPROXY session with Joe Stringer [1], I mentioned > that the sk_lookup_* helpers currently return inconsistent results if > SK_REUSEPORT programs are in play. > > SK_REUSEPORT programs are a hook point in

Re: RFC: Fixing SK_REUSEPORT from sk_lookup_* helpers

2019-05-18 Thread Joe Stringer
On Fri, May 17, 2019 at 7:15 AM Lorenz Bauer wrote: > > On Thu, 16 May 2019 at 21:33, Alexei Starovoitov > wrote: > > > > On Thu, May 16, 2019 at 09:41:34AM +0100, Lorenz Bauer wrote: > > > On Wed, 15 May 2019 at 18:16, Joe Stringer wrote: > > > > >

Re: [PATCH bpf] bpf: Check sk_fullsock() before returning from bpf_sk_lookup()

2019-05-18 Thread Joe Stringer
On Sat, May 18, 2019, 09:05 Martin Lau wrote: > > On Sat, May 18, 2019 at 08:38:46AM -1000, Joe Stringer wrote: > > On Fri, May 17, 2019, 12:02 Martin Lau wrote: > > > > > On Fri, May 17, 2019 at 02:51:48PM -0700, Eric Dumazet wrote: > > > > > > >

Re: [PATCH bpf] bpf: Check sk_fullsock() before returning from bpf_sk_lookup()

2019-05-20 Thread Joe Stringer
On Sat, May 18, 2019 at 7:08 PM Martin Lau wrote: > > On Sat, May 18, 2019 at 06:52:48PM -0700, Joe Stringer wrote: > > On Sat, May 18, 2019, 09:05 Martin Lau wrote: > > > > > > On Sat, May 18, 2019 at 08:38:46AM -1000, Joe Stringer wrote: > > > > On

Re: [PATCH bpf] bpf: Check sk_fullsock() before returning from bpf_sk_lookup()

2019-05-20 Thread Joe Stringer
P_TIME_WAIT. > > This patch checks for sk_fullsock() before returning. If it is not > a fullsock, sock_gen_put() is called if needed and then returns NULL. > > Fixes: 6acc9b432e67 ("bpf: Add helper to retrieve socket in BPF") > Cc: Joe Stringer > Signed-off-by: Martin KaFai Lau > --- Acked-by: Joe Stringer

Re: [PATCH net-next] openvswitch: add ct_clear action

2017-10-10 Thread Joe Stringer
On 9 October 2017 at 21:41, Pravin Shelar wrote: > On Fri, Oct 6, 2017 at 9:44 AM, Eric Garver wrote: >> This adds a ct_clear action for clearing conntrack state. ct_clear is >> currently implemented in OVS userspace, but is not backed by an action >> in the kernel datapath. This is useful for fl

Re: [PATCH net-next] openvswitch: add ct_clear action

2017-10-10 Thread Joe Stringer
On 10 October 2017 at 08:09, Eric Garver wrote: > On Tue, Oct 10, 2017 at 05:33:48AM -0700, Joe Stringer wrote: >> On 9 October 2017 at 21:41, Pravin Shelar wrote: >> > On Fri, Oct 6, 2017 at 9:44 AM, Eric Garver wrote: >> >> This adds a ct_clear action for clearin

Re: [ovs-dev] [PATCH net-next] openvswitch: add ct_clear action

2017-10-10 Thread Joe Stringer
On 10 October 2017 at 12:13, Eric Garver wrote: > On Tue, Oct 10, 2017 at 10:24:20AM -0700, Joe Stringer wrote: >> On 10 October 2017 at 08:09, Eric Garver wrote: >> > On Tue, Oct 10, 2017 at 05:33:48AM -0700, Joe Stringer wrote: >> >> On 9 October 2017 at 21:41, Pr

[PATCH iproute2] bpf: Print section name when hitting non ld64 issue

2018-02-28 Thread Joe Stringer
It's useful to be able to tell which section is being processed in the ELF when this error is triggered, so print that detail. Signed-off-by: Joe Stringer --- lib/bpf.c | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/bpf.c b/lib/bpf.c index 2db151e4dd3c..c38d92d87759 100644 ---

Re: [RFC PATCH bpf-next v2 0/4] Implement bpf queue/stack maps

2018-09-06 Thread Joe Stringer
On Thu, 6 Sep 2018 at 17:13, Alexei Starovoitov wrote: > bpf_map_pop_elem() is trying to do lookup_and_delete and preserve > validity of value without races. > With pcpu_freelist I don't think there is a solution. > We can have this queue/stack map without prealloc and use kmalloc/kfree > back and

[PATCH bpf-next 03/11] bpf: Generalize ptr_or_null regs check

2018-09-11 Thread Joe Stringer
This check will be reused by an upcoming commit for conditional jump checks for sockets. Refactor it a bit to simplify the later commit. Signed-off-by: Joe Stringer Acked-by: Alexei Starovoitov --- kernel/bpf/verifier.c | 43 +-- 1 file changed, 25

[PATCH bpf-next 05/11] bpf: Macrofy stack state copy

2018-09-11 Thread Joe Stringer
An upcoming commit will need very similar copy/realloc boilerplate, so refactor the existing stack copy/realloc functions into macros to simplify it. Signed-off-by: Joe Stringer --- kernel/bpf/verifier.c | 106 -- 1 file changed, 60 insertions(+), 46

[PATCH bpf-next 01/11] bpf: Add iterator for spilled registers

2018-09-11 Thread Joe Stringer
Add this iterator for spilled registers, it concentrates the details of how to get the current frame's spilled registers into a single macro while clarifying the intention of the code which is calling the macro. Signed-off-by: Joe Stringer Acked-by: Alexei Starovoitov --- include/

[PATCH bpf-next 02/11] bpf: Simplify ptr_min_max_vals adjustment

2018-09-11 Thread Joe Stringer
An upcoming commit will add another two pointer types that need very similar behaviour, so generalise this function now. Signed-off-by: Joe Stringer Acked-by: Alexei Starovoitov --- kernel/bpf/verifier.c | 22 ++--- tools/testing/selftests/bpf

[PATCH bpf-next 10/11] selftests/bpf: Add C tests for reference tracking

2018-09-11 Thread Joe Stringer
Signed-off-by: Joe Stringer --- tools/testing/selftests/bpf/Makefile | 2 +- tools/testing/selftests/bpf/test_progs.c | 38 ++ .../selftests/bpf/test_sk_lookup_kern.c | 128 ++ 3 files changed, 167 insertions(+), 1 deletion(-) create mode 100644 tools

[PATCH bpf-next 09/11] libbpf: Support loading individual progs

2018-09-11 Thread Joe Stringer
individually loaded, each program can be independently checked against its expected verification result. Signed-off-by: Joe Stringer --- tools/lib/bpf/libbpf.c | 4 ++-- tools/lib/bpf/libbpf.h | 3 +++ 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/tools/lib/bpf/libbpf.c b/tools

[PATCH bpf-next 00/11] Add socket lookup support

2018-09-11 Thread Joe Stringer
y of Nitin * Rebase This tree is also available at: https://github.com/joestringer/linux/commits/submit/sk-lookup-v1 Joe Stringer (11): bpf: Add iterator for spilled registers bpf: Simplify ptr_min_max_vals adjustment bpf: Generalize ptr_or_null regs check bpf: Add PTR_TO_SOCKET verifier type bp

[PATCH bpf-next 07/11] bpf: Add helper to retrieve socket in BPF

2018-09-11 Thread Joe Stringer
find a socket listening for this traffic. Drop. return TC_ACT_SHOT; } bpf_sk_release(sk, 0); return TC_ACT_OK; Signed-off-by: Joe Stringer --- include/uapi/linux/bpf.h | 54 +++- kernel/bpf/verifier.c | 8 +- net/core/filter.c

[PATCH bpf-next 06/11] bpf: Add reference tracking to verifier

2018-09-11 Thread Joe Stringer
_state` and all existing copies of the pointer in registers are marked invalid. Signed-off-by: Joe Stringer --- include/linux/bpf_verifier.h | 24 ++- kernel/bpf/verifier.c| 303 --- 2 files changed, 306 insertions(+), 21 deletions(-) diff --git a/include/linux

[PATCH bpf-next 04/11] bpf: Add PTR_TO_SOCKET verifier type

2018-09-11 Thread Joe Stringer
Teach the verifier a little bit about a new type of pointer, a PTR_TO_SOCKET. This pointer type is accessed from BPF through the 'struct bpf_sock' structure. Signed-off-by: Joe Stringer --- include/linux/bpf.h | 17 + include/linux/bpf_verifier.h | 2 + kernel/bpf/

[PATCH bpf-next 11/11] Documentation: Describe bpf reference tracking

2018-09-11 Thread Joe Stringer
Signed-off-by: Joe Stringer --- Documentation/networking/filter.txt | 64 + 1 file changed, 64 insertions(+) diff --git a/Documentation/networking/filter.txt b/Documentation/networking/filter.txt index e6b4ebb2b243..4443ce958862 100644 --- a/Documentation/networking

[PATCH bpf-next 08/11] selftests/bpf: Add tests for reference tracking

2018-09-11 Thread Joe Stringer
_ptr spill into caller stack Signed-off-by: Joe Stringer --- tools/testing/selftests/bpf/test_verifier.c | 359 1 file changed, 359 insertions(+) diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c index ceb55a9f

Re: [PATCH bpf-next 04/11] bpf: Add PTR_TO_SOCKET verifier type

2018-09-13 Thread Joe Stringer
On Wed, 12 Sep 2018 at 15:50, Alexei Starovoitov wrote: > > On Tue, Sep 11, 2018 at 05:36:33PM -0700, Joe Stringer wrote: > > ... > > +static bool reg_type_mismatch(enum bpf_reg_type src, enum bpf_reg_type > > prev) > > +{ > > + return src !

Re: [PATCH bpf-next 06/11] bpf: Add reference tracking to verifier

2018-09-13 Thread Joe Stringer
On Wed, 12 Sep 2018 at 16:17, Alexei Starovoitov wrote: > > On Tue, Sep 11, 2018 at 05:36:35PM -0700, Joe Stringer wrote: > > ... > > + > > +/* release function corresponding to acquire_reference_state(). > > Idempotent. */ > > +static int __release_referenc

Re: [PATCH bpf-next 07/11] bpf: Add helper to retrieve socket in BPF

2018-09-13 Thread Joe Stringer
On Thu, 13 Sep 2018 at 12:06, Alexei Starovoitov wrote: > > On Wed, Sep 12, 2018 at 5:06 PM, Alexei Starovoitov > wrote: > > On Tue, Sep 11, 2018 at 05:36:36PM -0700, Joe Stringer wrote: > >> This patch adds new BPF helper functions, bpf_sk_lookup_tcp() and > >>

Re: [PATCH bpf-next 10/11] selftests/bpf: Add C tests for reference tracking

2018-09-13 Thread Joe Stringer
On Wed, 12 Sep 2018 at 17:11, Alexei Starovoitov wrote: > > On Tue, Sep 11, 2018 at 05:36:39PM -0700, Joe Stringer wrote: > > Signed-off-by: Joe Stringer > > really nice set of tests. > please describe them briefly in commit log. > > Acked-by: Alexei Starovoitov Ack, will do.

Re: [PATCH bpf-next 11/11] Documentation: Describe bpf reference tracking

2018-09-13 Thread Joe Stringer
On Wed, 12 Sep 2018 at 17:13, Alexei Starovoitov wrote: > > On Tue, Sep 11, 2018 at 05:36:40PM -0700, Joe Stringer wrote: > > Signed-off-by: Joe Stringer > > just few words in commit log would be better than nothing. > > Acked-by: Alexei Starovoitov Ack, thanks for the review!

Re: [PATCH bpf-next 07/11] bpf: Add helper to retrieve socket in BPF

2018-09-13 Thread Joe Stringer
On Thu, 13 Sep 2018 at 13:55, Joe Stringer wrote: > struct bpf_sock_tuple4 { > __be32 saddr; > __be32 daddr; > __be16 sport; > __be16 dport; > __u8 family; > }; > > struct bpf_sock_tuple6 { > __be32 saddr[4]; > __be

Re: [PATCH bpf-next 07/11] bpf: Add helper to retrieve socket in BPF

2018-09-13 Thread Joe Stringer
On Thu, 13 Sep 2018 at 14:02, Alexei Starovoitov wrote: > > On Thu, Sep 13, 2018 at 01:55:01PM -0700, Joe Stringer wrote: > > On Thu, 13 Sep 2018 at 12:06, Alexei Starovoitov > > wrote: > > > > > > On Wed, Sep 12, 2018 at 5:06 PM, Alexei Starovoitov > >

Re: [PATCH bpf-next 07/11] bpf: Add helper to retrieve socket in BPF

2018-09-13 Thread Joe Stringer
On Thu, 13 Sep 2018 at 14:22, Alexei Starovoitov wrote: > > On Thu, Sep 13, 2018 at 02:17:17PM -0700, Joe Stringer wrote: > > On Thu, 13 Sep 2018 at 14:02, Alexei Starovoitov > > wrote: > > > > > > On Thu, Sep 13, 2018 at 01:55:01PM -0700, Joe Stringer wrote:

[PATCHv2 bpf-next 00/11] Add socket lookup support

2018-09-21 Thread Joe Stringer
x kbuild compilation warnings with particular configs. * Improve code comments describing the new verifier pieces. * Testing courtesy of Nitin * Rebase This tree is also available at: https://github.com/joestringer/linux/commits/submit/sk-lookup-v2 Joe Stringer (11): bpf: Add iterator for spi

[PATCHv2 bpf-next 03/11] bpf: Generalize ptr_or_null regs check

2018-09-21 Thread Joe Stringer
This check will be reused by an upcoming commit for conditional jump checks for sockets. Refactor it a bit to simplify the later commit. Signed-off-by: Joe Stringer Acked-by: Alexei Starovoitov --- kernel/bpf/verifier.c | 43 +-- 1 file changed, 25

[PATCHv2 bpf-next 01/11] bpf: Add iterator for spilled registers

2018-09-21 Thread Joe Stringer
Add this iterator for spilled registers, it concentrates the details of how to get the current frame's spilled registers into a single macro while clarifying the intention of the code which is calling the macro. Signed-off-by: Joe Stringer Acked-by: Alexei Starovoitov --- include/

[PATCHv2 bpf-next 09/11] libbpf: Support loading individual progs

2018-09-21 Thread Joe Stringer
individually loaded, each program can be independently checked against its expected verification result. Signed-off-by: Joe Stringer Acked-by: Alexei Starovoitov --- tools/lib/bpf/libbpf.c | 4 ++-- tools/lib/bpf/libbpf.h | 3 +++ 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a

[PATCHv2 bpf-next 04/11] bpf: Add PTR_TO_SOCKET verifier type

2018-09-21 Thread Joe Stringer
Teach the verifier a little bit about a new type of pointer, a PTR_TO_SOCKET. This pointer type is accessed from BPF through the 'struct bpf_sock' structure. Signed-off-by: Joe Stringer --- v2: Reuse reg_type_mismatch() in more places Reduce the number of passes at convert_

[PATCHv2 bpf-next 05/11] bpf: Macrofy stack state copy

2018-09-21 Thread Joe Stringer
An upcoming commit will need very similar copy/realloc boilerplate, so refactor the existing stack copy/realloc functions into macros to simplify it. Signed-off-by: Joe Stringer Acked-by: Alexei Starovoitov --- kernel/bpf/verifier.c | 106 -- 1 file

[PATCHv2 bpf-next 07/11] bpf: Add helper to retrieve socket in BPF

2018-09-21 Thread Joe Stringer
find a socket listening for this traffic. Drop. return TC_ACT_SHOT; } bpf_sk_release(sk, 0); return TC_ACT_OK; Signed-off-by: Joe Stringer --- v2: Rework 'struct bpf_sock_tuple' to allow passing a packet pointer Limit netns_id field to 32 bits Fix compile error w

[PATCHv2 bpf-next 02/11] bpf: Simplify ptr_min_max_vals adjustment

2018-09-21 Thread Joe Stringer
An upcoming commit will add another two pointer types that need very similar behaviour, so generalise this function now. Signed-off-by: Joe Stringer Acked-by: Alexei Starovoitov --- kernel/bpf/verifier.c | 22 ++--- tools/testing/selftests/bpf

[PATCHv2 bpf-next 06/11] bpf: Add reference tracking to verifier

2018-09-21 Thread Joe Stringer
_state` and all existing copies of the pointer in registers are marked invalid. Signed-off-by: Joe Stringer Acked-by: Alexei Starovoitov --- v2: Replace ptr_id defensive coding when releasing reference state with an internal error (-EFAULT) --- include/linux/bpf_verifier.h | 24 ++- kernel

[PATCHv2 bpf-next 11/11] Documentation: Describe bpf reference tracking

2018-09-21 Thread Joe Stringer
Document the new pointer types in the verifier and how the pointer ID tracking works to ensure that references which are taken are later released. Signed-off-by: Joe Stringer Acked-by: Alexei Starovoitov --- Documentation/networking/filter.txt | 64 + 1 file changed

[PATCHv2 bpf-next 10/11] selftests/bpf: Add C tests for reference tracking

2018-09-21 Thread Joe Stringer
Add some tests that demonstrate and test the balanced lookup/free nature of socket lookup. Section names that start with "fail" represent programs that are expected to fail verification; all others should succeed. Signed-off-by: Joe Stringer Acked-by: Alexei Starovoitov --- too

[PATCHv2 bpf-next 08/11] selftests/bpf: Add tests for reference tracking

2018-09-21 Thread Joe Stringer
_ptr spill into caller stack Signed-off-by: Joe Stringer Acked-by: Alexei Starovoitov --- tools/testing/selftests/bpf/test_verifier.c | 359 1 file changed, 359 insertions(+) diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifi

Re: netns_id in bpf_sk_lookup_{tcp,udp}

2018-11-19 Thread Joe Stringer
Hi David, thanks for pointing this out. This is more of an oversight through iterations, the runtime lookup will fail to find a socket if the netns value is greater than the range of a uint32 so I think it would actually make more sense to drop the parameter size to u32 rather than u64 so that thi

Re: netns_id in bpf_sk_lookup_{tcp,udp}

2018-11-19 Thread Joe Stringer
On Mon, 19 Nov 2018 at 10:39, David Ahern wrote: > > On 11/19/18 11:36 AM, Joe Stringer wrote: > > Hi David, thanks for pointing this out. > > > > This is more of an oversight through iterations, the runtime lookup > > will fail to find a socket if the netns value is

Re: netns_id in bpf_sk_lookup_{tcp,udp}

2018-11-19 Thread Joe Stringer
On Mon, 19 Nov 2018 at 12:29, Nicolas Dichtel wrote: > > Le 19/11/2018 à 20:54, David Ahern a écrit : > > On 11/19/18 12:47 PM, Joe Stringer wrote: > >> On Mon, 19 Nov 2018 at 10:39, David Ahern wrote: > >>> > >>> On 11/19/18 11:36 AM, Joe Stringe

Re: netns_id in bpf_sk_lookup_{tcp,udp}

2018-11-19 Thread Joe Stringer
On Mon, 19 Nov 2018 at 12:54, Joe Stringer wrote: > > On Mon, 19 Nov 2018 at 12:29, Nicolas Dichtel > wrote: > > > > Le 19/11/2018 à 20:54, David Ahern a écrit : > > > On 11/19/18 12:47 PM, Joe Stringer wrote: > > >> On Mon, 19 Nov 2018 at 10:39, David

[PATCH bpf] bpf: Support sk lookup in netns with id 0

2018-11-26 Thread Joe Stringer
. Signed-off-by: Joe Stringer --- include/uapi/linux/bpf.h | 29 +--- net/core/filter.c | 16 - tools/include/uapi/linux/bpf.h| 33 --- .../selftests/bpf/test_sk_lookup_kern.c | 18

Re: [PATCH bpf] bpf: Support sk lookup in netns with id 0

2018-11-27 Thread Joe Stringer
On Tue, 27 Nov 2018 at 06:49, Nicolas Dichtel wrote: > > Le 26/11/2018 à 23:08, David Ahern a écrit : > > On 11/26/18 2:27 PM, Joe Stringer wrote: > >> @@ -2405,6 +2407,9 @@ enum bpf_func_id { > >> /* BPF_FUNC_perf_event_output for sk_buff input context. */ &

Re: [PATCH bpf] bpf: Support sk lookup in netns with id 0

2018-11-28 Thread Joe Stringer
On Tue, 27 Nov 2018 at 13:12, Alexei Starovoitov wrote: > > On Tue, Nov 27, 2018 at 10:01:40AM -0800, Joe Stringer wrote: > > On Tue, 27 Nov 2018 at 06:49, Nicolas Dichtel > > wrote: > > > > > > Le 26/11/2018 ą 23:08, David Ahern a écrit : > > &

[PATCHv2 bpf 2/2] bpf: Improve socket lookup reuseport documentation

2018-11-29 Thread Joe Stringer
Improve the wording around socket lookup for reuseport sockets, and ensure that both bpf.h headers are in sync. Signed-off-by: Joe Stringer --- include/uapi/linux/bpf.h | 4 tools/include/uapi/linux/bpf.h | 8 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a

[PATCHv2 bpf 1/2] bpf: Support sk lookup in netns with id 0

2018-11-29 Thread Joe Stringer
any bits are set in the upper 32-bits, then no socket will be found. Signed-off-by: Joe Stringer --- include/uapi/linux/bpf.h | 35 ++--- net/core/filter.c | 11 +++--- tools/include/uapi/linux/bpf.h| 39

Re: [PATCHv2 bpf 1/2] bpf: Support sk lookup in netns with id 0

2018-11-30 Thread Joe Stringer
On Thu, 29 Nov 2018 at 16:30, Joe Stringer wrote: > > David Ahern and Nicolas Dichtel report that the handling of the netns id > 0 is incorrect for the BPF socket lookup helpers: rather than finding > the netns with id 0, it is resolving to the current netns. This renders >

Re: [PATCHv2 bpf 1/2] bpf: Support sk lookup in netns with id 0

2018-11-30 Thread Joe Stringer
On Fri, 30 Nov 2018 at 14:42, Alexei Starovoitov wrote: > > On Thu, Nov 29, 2018 at 04:29:33PM -0800, Joe Stringer wrote: > > David Ahern and Nicolas Dichtel report that the handling of the netns id > > 0 is incorrect for the BPF socket lookup helpers: rather than finding >

  1   2   3   4   5   6   >