Re: Errors compiling BPF programs from Linux selftests/bpf with GCC
On Thu, 2025-01-02 at 10:47 +0100, Jose E. Marchesi wrote:
> Hi Ihor.
> Thanks for working on this! :)
>
> > [...]
> > Older versions compile the dummy program without errors, however on
> > attempt to build the selftests there is a different issue: conflicting
> > int64 definitions (full log at [6]).
> >
> > In file included from /usr/include/x86_64-linux-gnu/sys/types.h:155,
> > from /usr/include/x86_64-linux-gnu/bits/socket.h:29,
> > from /usr/include/x86_64-linux-gnu/sys/socket.h:33,
> > from /usr/include/linux/if.h:28,
> > from /usr/include/linux/icmp.h:23,
> > from progs/test_cls_redirect_dynptr.c:10:
> > /usr/include/x86_64-linux-gnu/bits/stdint-intn.h:27:19: error:
> > conflicting types for ‘int64_t’; have ‘__int64_t’ {aka ‘long long int’}
> >27 | typedef __int64_t int64_t;
> > | ^~~
> > In file included from progs/test_cls_redirect_dynptr.c:6:
> >
> > /ci/workspace/bpfgcc.20240922/lib/gcc/bpf-unknown-none/15.0.0/include/stdint.h:43:24:
> > note: previous declaration of ‘int64_t’ with type ‘int64_t’ {aka ‘long
> > int’}
> >43 | typedef __INT64_TYPE__ int64_t;
> > |^~~
>
> I think this is what is going on:
>
> The BPF selftest is indirectly including glibc headers from the host
> where it is being compiled. In this case your x86_64 ubuntu system.
>
> Many glibc headers include bits/wordsize.h, which in the case of x86_64
> is:
>
> #if defined __x86_64__ && !defined __ILP32__
> # define __WORDSIZE 64
> #else
> # define __WORDSIZE 32
> #define __WORDSIZE32_SIZE_ULONG 0
> #define __WORDSIZE32_PTRDIFF_LONG 0
> #endif
>
> and then in bits/types.h:
>
> #if __WORDSIZE == 64
> typedef signed long int __int64_t;
> typedef unsigned long int __uint64_t;
> #else
> __extension__ typedef signed long long int __int64_t;
> __extension__ typedef unsigned long long int __uint64_t;
> #endif
>
> i.e. your BPF program ends using __WORDSIZE 32. This eventually leads
> to int64_t being defined as `signed long long int' in stdint-intn.h, as
> it would correspond to a x86_64 program running in 32-bit mode.
>
> GCC BPF, on the other hand, is a "baremetal" compiler and it provides a
> small set of headers (including stdint.h) that implement standard C99
> types like int64_t, adjusted to the BPF architecture.
>
> In this case there is a conflict between the 32-bit x86_64 definition of
> int64_t and the one of BPF.
>
> PS: the other headers installed by GCC BPF are:
> float.h iso646.h limits.h stdalign.h stdarg.h stdatomic.h stdbool.h
> stdckdint.h stddef.h stdfix.h stdint.h stdnoreturn.h syslimits.h
> tgmath.h unwind.h varargs.h
I wondered how this works with clang, because it does not define
__x86_64__ for bpf target. After staring and the output of -E:
- for clang int64_t is defined once and definition originate from
/usr/include/bits/stdint-intn.h included from /usr/include/stdint.h;
- for gcc int64_t is defined two times, definitions originate from:
- /bpf-unknown-none/15.0.0/include/stdint.h
- /usr/include/bits/stdint-intn.h included from /usr/include/sys/types.h.
So, both refer to stdint-intn.h, but only gcc refers to
compiler-specific stdint.h. This is so because of the structure of the
clang's /usr/lib/clang/19/include/stdint.h:
...
#if __STDC_HOSTED__ && __has_include_next()
...
# include_next
...
#else
...
typedef __INT64_TYPE__ int64_t;
...
#endif
...
The __STDC_HOSTED__ is defined as 1, thus when clang compiles the test case,
compiler-specific stdint.h is included, but it's content is ifdef'ed out and
it refers to system stdint.h instead. On the other hand, gcc-specific stdint.h
unconditionally typedefs int64_t.
Links:
- test case pre-processed by clang and gcc:
https://gist.github.com/eddyz87/d381094d67979291bd8338655b15dd5e
- LLVM source code for stdint.h:
https://github.com/llvm/llvm-project/blob/c703b4645c79e889fd6a0f3f64f01f957d981aa4/clang/lib/Headers/stdint.h#L24
Re: Errors compiling BPF programs from Linux selftests/bpf with GCC
On Fri, 2025-01-03 at 01:16 +0100, Jose E. Marchesi wrote:
[...]
> Yes, in the GCC BPF backend we are using
>
> use_gcc_stdint=provide
>
> which makes GCC to provide the version of stdint.h that assumes
> freestanding ("baremetal") mode. If we changed it to use
>
> use_gcc_stdint=wrap
>
> then it would install a stdint.h that does somethins similar to what
> clang does, at least in hosts providing C99 headers (note the lack of
> __has_include_next):
>
> #ifndef _GCC_WRAP_STDINT_H
> #if __STDC_HOSTED__
> # if defined __cplusplus && __cplusplus >= 201103L
> # undef __STDC_LIMIT_MACROS
> # define __STDC_LIMIT_MACROS
> # undef __STDC_CONSTANT_MACROS
> # define __STDC_CONSTANT_MACROS
> # endif
> #pragma GCC diagnostic push
> #pragma GCC diagnostic ignored "-Wpedantic" // include_next
> # include_next
> #pragma GCC diagnostic pop
> #else
> # include "stdint-gcc.h"
> #endif
> #define _GCC_WRAP_STDINT_H
> #endif
>
> We could switch to "wrap" to align with clang, but in that case it would
> be up to the user to provide a "host" stdint.h that contains sensible
> definitions for BPF. The kernel selftests, for example, would need to
> do so to avoid including /usr/include/stdint.h that more likely than not
> will provide incorrect definitions for int64_t and friends...
Would it be possible to push a branch that uses '=wrap' thing somewhere?
So that it could be further tested to see if there are more issues with
selftests.
Re: Errors compiling BPF programs from Linux selftests/bpf with GCC
On Thu, 2025-01-02 at 19:24 +0100, Jose E. Marchesi wrote:
[...]
> IMO the BPP selftest (and BPF programs in general) must not include host
> glibc headers at all, regardless of what BPF compiler is used. The
> glibc headers installed in the host are tailored to some particular
> architecture, be it x86_64 or whatever, not necessarily compatible with
> what the compilers assume for the BPF target.
>
> This particular case shows the problem well: all the glibc headers
> included by that BPF selftest assume that `long' is 32 bits, not 64
> bits, because x86_64 is not defined. This conflicts with both clang's
> and GCC's assumption that in BPF a `long' is 64 bits. This may or may
> not be a problem, depending on whether the BPF program uses the stuff
> defined in the headers and how it uses it. Had you be using an arm or
> sparc host instead of x86_64, you may be including macros and stuff that
> assume chars are unsigned. But chars are signed in bpf.
This makes sense, but might cause some friction.
The following glibc headers are included directly from selftests:
- errno.h
- features.h
- inttypes.h
- limits.h
- netinet/in.h
- netinet/udp.h
- sched.h
- stdint.h
- stdlib.h
- string.h
- sys/socket.h
- sys/types.h
- time.h
- unistd.h
However, removing includes for these headers does not help the test in
question, because some linux UAPI headers include libc headers when exported:
In file included from /usr/include/netinet/udp.h:51,
from progs/test_cls_redirect_dynptr.c:20:
/home/eddy/work/tmp/gccbpf/lib/gcc/bpf-unknown-none/15.0.0/include/stdint.h:43:24:
error: conflicting types for ‘int64_t’; have ‘long int’
43 | typedef __INT64_TYPE__ int64_t;
|^~~
In file included from /usr/include/sys/types.h:155,
from /usr/include/bits/socket.h:29,
from /usr/include/sys/socket.h:33,
from /usr/include/linux/if.h:28,
from /usr/include/linux/icmp.h:23,
from progs/test_cls_redirect_dynptr.c:12:
/usr/include/bits/stdint-intn.h:27:19: note: previous declaration of
‘int64_t’ with type ‘int64_t’ {aka ‘long long int’}
27 | typedef __int64_t int64_t;
| ^~~
On my system (Fedora 41) the linux/{icmp,if}.h UAPI headers are
provided by kernel-headers package, sys/socket.h is provided by
glibc-devel package.
The UAPI headers have two modes depending whether __KERNEL__ is
defined. When used during kernel build the __KERNEL__ is defined and
there are no outside references. When exported for packages like
kernel-headers (via 'make headers' target) the __KERNEL__ is not
defined and there are some references to libc includes
(in fact, references to '#ifdef __KERNEL__' blocks are cut out during
headers export).
E.g. here is a fragment of linux/if.h, when viewed from kernel source:
#ifndef _LINUX_IF_H
#define _LINUX_IF_H
#include /* for compatibility with glibc */
#include /* for "__kernel_caddr_t" et al */
#include/* for "struct sockaddr" et al */
#include /* for "__user" et al */
#ifndef __KERNEL__
#include /* for struct sockaddr.
*/
#endif
And here is the same fragment as part of the kernel-headers package
(/usr/include/linux/if.h):
#ifndef _LINUX_IF_H
#define _LINUX_IF_H
#include /* for compatibility with glibc */
#include /* for "__kernel_caddr_t" et al */
#include/* for "struct sockaddr" et al */
/* for "__user" et al */
#include /* for struct sockaddr.
*/
As far as I understand, the idea right now is that BPF users can
install the kernel-headers package (or its equivalent) and start
hacking. If we declare that this is no longer a blessed way, people
would need to switch to packages like kernel-devel that provide full
set of kernel headers for use with dkms etc, e.g. for my system the
if.h would be located here:
/usr/src/kernels/6.12.6-200.fc41.x86_64/include/uapi/linux/if.h .
To me this seems logical, however potentially such change might have
implications for existing BPF code-base.
