Re: [PATCH] bpf: Improvements in CO-RE builtins implementation.

[David Faust]

On 10/25/23 11:51, Cupertino Miranda wrote:
> Hi everyone,
> 
> This patch contains some more recent improvements to BPF CO-RE builtins.
> Please find further details of the changes on the patch header.
> 
> Looking forward for your review and comments.
> 
> Best regards,
> Cupertino Miranda

Hi Cupertino,

Thanks for continuing to work on these BPF builtins.

Comments inline below.

> 
> 
> bpf_builtins_upstream.patch
> 
> commit 6054209c0a8af9c3e6363550bf2ba4f4f2172eba
> Author: Cupertino Miranda 
> Date:   Tue Aug 8 09:22:41 2023 +0100
> 
> bpf: Improvements in CO-RE builtins implementation.
> 
> This patch moved the processing of attribute preserve_access_index to
> its own independent pass in a gimple lowering pass.
> This approach is more consistent with the implementation of the CO-RE
> builtins when used explicitly in the code.  The attributed type accesses
> are now early converted to __builtin_core_reloc builtin instead of being
> kept as an expression in code through out all of the middle-end.
> This disables the compiler to optimize out or manipulate the expression
> using the local defined type, instead of assuming nothing is known about
> this expression, as it should be the case in all of the CO-RE
> relocations.
> 
> In the process, also the __builtin_preserve_access_index has been
> improved to generate code for more complex expressions that would
> require more then one CO-RE relocation.
> This turned out to be a requirement, since bpf-next selftests would rely 
> on
> loop unrolling in order to convert an undefined index array access into a
> defined one. This seemed extreme to expect for the unroll to happen, and 
> for
> that reason GCC still generates correct code in such scenarios, even when 
> index
> access is never predictable or unrolling does not occur.
> 
> gcc/ChangeLog:
> * config/bpf/bpf-passes.def (pass_lower_bpf_core): Added pass.

It may only be due to how the patch is formatted in the attachment
(everything above the first diff seems to be indented?), but each entry
here should start with a tab rather than spaces.

Please double-check with contrib/gcc-changelog/git_check_commit.py, it
will complain if the indentation is wrong.

> * config/bpf/bpf-protos.h: Added prototype for new pass.
> * config/bpf/bpf.cc (bpf_const_not_ok_for_debug_p): New function.
> * config/bpf/bpf.md (mov_reloc_core): Changed

Looks like an entry along the lines of "Renamed to..." would be
more fitting.

> * config/bpf/core-builtins.cc (cr_builtins, 
> is_attr_preserve_access,
> core_field_info, bpf_core_get_index, compute_field_expr,
> process_field_expr, pack_type, make_core_relo,
> bpf_handle_plugin_finish_type, core_buintin_helpers,
> construct_builtin_core_reloc, bpf_resolve_overloaded_core_builtin,
> bpf_add_core_reloc): Changed.

"Changed." as a ChangeLog entry is not useful. Please be at least a
little bit more descriptive here so someone only reading the ChangeLog
has an idea of what they're looking at.

Also, I'm not sure this is a proper way to note multiple functions
with the same entry, usually this is done with a new pair of () for
each new line, like:

* config/bpf/core-builtins.cc (cr_builtins, is_attr_preserve_access)
(core_field_info, bpf_core_get_index): Changed to do blah.

contrib/gcc-changelog/git_check_commit.py might complain about this;
please check.

> (root_for_core_field_info, pack_field_expr,
> core_expr_with_field_expr_plus_base, make_core_safe_access_index,
> replace_core_access_index_comp_expr, 
> maybe_get_base_for_field_expr,
> core_access_clean, core_is_access_index, 
> core_mark_as_access_index,
> make_gimple_core_safe_access_index, execute_lower_bpf_core,
> make_pass_lower_bpf_core): Added functions.

Same thing about the multi-line entry within a single () pair.

> (pass_data_lower_bpf_core): New pass struct.
> (pass_lower_bpf_core): New gimple_opt_pass class.
> (pack_field_expr_for_preserve_field, 
> bpf_replace_core_move_operands): Removed function.
> (bpf_enum_value_kind): Added GTY(()).
> * config/bpf/core-builtins.h (bpf_field_info_kind, 
> bpf_type_id_kind,
> bpf_type_info_kind, bpf_enum_value_kind): New enum.
> * config/bpf/t-bpf: Added pass bpf-passes.def to PASSES_EXTRA.
> 
> gcc/testsuite/ChangeLog:
> * gcc.target/bpf/core-attr-5.c: New test.
> * gcc.target/bpf/core-attr-6.c: New test.
> * gcc.target/bpf/core-builtin-1.c: Corrected
> * gcc.target/bpf/core-builtin-enumvalue-opt.c: Corrected regular
> expression.
> * gcc.target/bpf/core-builtin-enumvalue.c: Corrected regular
> expres

Re: [PATCH v4] bpf: Improvements in CO-RE builtins implementation.

[David Faust]

On 10/26/23 08:08, Cupertino Miranda wrote:
> 
> Changes from v1:
>  - Fixed Davids remarks on initial patch.
>  - Fixed mistake with deleted '*'.
> 
> Changes from v2:
>  - Reversed return value for bpf_const_not_ok_for_debug_p function.

Hmm..

> +static bool
> +bpf_const_not_ok_for_debug_p (rtx p)
> +{
> +  if (GET_CODE (p) == UNSPEC
> +  && XINT (p, 1) == UNSPEC_CORE_RELOC)
> +return false;
> +
> +  return true;
> +}
> +
> +#undef TARGET_CONST_NOT_OK_FOR_DEBUG_P
> +#define TARGET_CONST_NOT_OK_FOR_DEBUG_P bpf_const_not_ok_for_debug_p

 -- Target Hook: bool TARGET_CONST_NOT_OK_FOR_DEBUG_P (rtx X)
 This hook should return true if X should not be emitted into debug
 sections.

As written now, won't this cause all ordinary (non-UNSPEC_CORE_RELOC)
consts to get rejected for debug? ("regular" debug i.e. DWARF, not to
be confused with the BTF.ext holding CO-RE relocs).

I see other targets implementing the hook returning true only in
specific cases and false otherwise.  The implementation in v1 makes
more sense to me.  Could you explain why flip the return value?

> 
> Changes from v3:
>  - Fixed ICE in two bpf-next tests:
>  -  if (!wi->is_lhs)
>  -   core_mark_as_access_index (gimple_get_lhs (wi->stmt));
>  +  tree lhs;
>  +  if (!wi->is_lhs
>  + && (lhs = gimple_get_lhs (wi->stmt)) != NULL_TREE)
>  +   core_mark_as_access_index (lhs);
> 

Re: [PATCH v5] bpf: Improvements in CO-RE builtins implementation.

[David Faust]

Hi Cupertino,

On 10/30/23 12:39, Cupertino Miranda wrote:
> 
> Hi everyone,
> 
> Please find a new version for the review as inline attachment.
> 
> Best regards,
> Cupertino
> 

This version LGTM.
Thanks!

> 
> Changes from v4:
>  - Implemented TARGET_DELEGITIMIZE_ADDRESS target hook as the proper
>  solution to the the warning for UNSPEC_CORE_RELOC being
>  non-delegitimize.
> 

Re: [PATCH v5] bpf: Improvements in CO-RE builtins implementation.

[David Faust]

On 10/31/23 09:58, David Faust wrote:
> Hi Cupertino,
> 
> On 10/30/23 12:39, Cupertino Miranda wrote:
>>
>> Hi everyone,
>>
>> Please find a new version for the review as inline attachment.
>>
>> Best regards,
>> Cupertino
>>
> 
> This version LGTM.
> Thanks!

OK for trunk.
Thanks.

> 
>>
>> Changes from v4:
>>  - Implemented TARGET_DELEGITIMIZE_ADDRESS target hook as the proper
>>  solution to the the warning for UNSPEC_CORE_RELOC being
>>  non-delegitimize.
>>
> 
> 

[PATCH] testsuite: ctf: make array in ctf-file-scope-1 fixed length

[David Faust]

The array member of struct SFOO in the ctf-file-scope-1 caused the test
to fail for the BPF target, since BPF does not support dynamic stack
allocation. The array does not need to variable length for the sake of
the test, so make it fixed length instead to allow the test to run
successfully for the bpf-unknown-none target.

Tested on x86_64-linux-gnu, and on x86_64-linux-gnu host for
bpf-unknown-none target.

gcc/testsuite/

* gcc.dg/debug/ctf/ctf-file-scope-1.c (SFOO): Make array member
fixed-length.
---
 gcc/testsuite/gcc.dg/debug/ctf/ctf-file-scope-1.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gcc/testsuite/gcc.dg/debug/ctf/ctf-file-scope-1.c 
b/gcc/testsuite/gcc.dg/debug/ctf/ctf-file-scope-1.c
index a683113e505..ddfb31da405 100644
--- a/gcc/testsuite/gcc.dg/debug/ctf/ctf-file-scope-1.c
+++ b/gcc/testsuite/gcc.dg/debug/ctf/ctf-file-scope-1.c
@@ -9,7 +9,7 @@
 
 int foo (int n)
 {
-  typedef struct { int a[n]; } SFOO;
+  typedef struct { int a[6]; } SFOO;
 
   SFOO a;
   __attribute__ ((noinline)) SFOO gfoo (void) { return a; }
-- 
2.43.0

[PATCH] bpf: add inline memset expansion

[David Faust]

Similar to memmove and memcpy, the BPF backend cannot fall back on a
library call to implement __builtin_memset, and should always expand
calls to it inline if possible.

This patch implements simple inline expansion of memset in the BPF
backend in a verifier-friendly way. Similar to memcpy and memmove, the
size must be an integer constant, as is also required by clang.

Tested for bpf-unknown-none target on x86_64-linux-gnu host.
Also testetd against kernel BPF verifier by compiling and loading a
test program using the inline memset expansion.

gcc/
* config/bpf/bpf-protos.h (bpf_expand_setmem): New prototype.
* config/bpf/bpf.cc (bpf_expand_setmem): New.
* config/bpf/bpf.md (setmemdi): New define_expand.

gcc/testsuite/
* gcc.target/bpf/memset-1.c: New test.
---
 gcc/config/bpf/bpf-protos.h |  1 +
 gcc/config/bpf/bpf.cc   | 66 +
 gcc/config/bpf/bpf.md   | 17 +++
 gcc/testsuite/gcc.target/bpf/memset-1.c | 39 +++
 4 files changed, 123 insertions(+)
 create mode 100644 gcc/testsuite/gcc.target/bpf/memset-1.c

diff --git a/gcc/config/bpf/bpf-protos.h b/gcc/config/bpf/bpf-protos.h
index 366acb87ae4..ac0c2f4038f 100644
--- a/gcc/config/bpf/bpf-protos.h
+++ b/gcc/config/bpf/bpf-protos.h
@@ -36,5 +36,6 @@ class gimple_opt_pass;
 gimple_opt_pass *make_pass_lower_bpf_core (gcc::context *ctxt);
 
 bool bpf_expand_cpymem (rtx *, bool);
+bool bpf_expand_setmem (rtx *);
 
 #endif /* ! GCC_BPF_PROTOS_H */
diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc
index 22b0cf2dc46..0e33f4347ba 100644
--- a/gcc/config/bpf/bpf.cc
+++ b/gcc/config/bpf/bpf.cc
@@ -1309,6 +1309,72 @@ bpf_expand_cpymem (rtx *operands, bool is_move)
   return true;
 }
 
+/* Expand setmem, as from __builtin_memset.
+   OPERANDS are the same as the setmem pattern.
+   Return true if the expansion was successful, false otherwise.  */
+
+bool
+bpf_expand_setmem (rtx *operands)
+{
+  /* Size must be constant for this expansion to work.  */
+  if (!CONST_INT_P (operands[1]))
+{
+  if (flag_building_libgcc)
+   warning (0, "could not inline call to %<__builtin_memset%>: "
+"size must be constant");
+  else
+   error ("could not inline call to %<__builtin_memset%>: "
+  "size must be constant");
+  return false;
+}
+
+  /* Alignment is a CONST_INT.  */
+  gcc_assert (CONST_INT_P (operands[3]));
+
+  rtx dst = operands[0];
+  rtx size = operands[1];
+  rtx val = operands[2];
+  unsigned HOST_WIDE_INT size_bytes = UINTVAL (size);
+  unsigned align = UINTVAL (operands[3]);
+  enum machine_mode mode;
+  switch (align)
+{
+case 1: mode = QImode; break;
+case 2: mode = HImode; break;
+case 4: mode = SImode; break;
+case 8: mode = DImode; break;
+default:
+  gcc_unreachable ();
+}
+
+  unsigned iters = size_bytes >> ceil_log2 (align);
+  unsigned remainder = size_bytes & (align - 1);
+  unsigned inc = GET_MODE_SIZE (mode);
+  unsigned offset = 0;
+
+  for (unsigned int i = 0; i < iters; i++)
+{
+  emit_move_insn (adjust_address (dst, mode, offset), val);
+  offset += inc;
+}
+  if (remainder & 4)
+{
+  emit_move_insn (adjust_address (dst, SImode, offset), val);
+  offset += 4;
+  remainder -= 4;
+}
+  if (remainder & 2)
+{
+  emit_move_insn (adjust_address (dst, HImode, offset), val);
+  offset += 2;
+  remainder -= 2;
+}
+  if (remainder & 1)
+emit_move_insn (adjust_address (dst, QImode, offset), val);
+
+  return true;
+}
+
 /* Finally, build the GCC target.  */
 
 struct gcc_target targetm = TARGET_INITIALIZER;
diff --git a/gcc/config/bpf/bpf.md b/gcc/config/bpf/bpf.md
index ca677bc6b50..ea688aadf91 100644
--- a/gcc/config/bpf/bpf.md
+++ b/gcc/config/bpf/bpf.md
@@ -663,4 +663,21 @@ (define_expand "movmemdi"
   FAIL;
 })
 
+;; memset
+;; 0 is dst
+;; 1 is length
+;; 2 is value
+;; 3 is alignment
+(define_expand "setmemdi"
+  [(set (match_operand:BLK 0 "memory_operand")
+   (match_operand:QI  2 "nonmemory_operand"))
+   (use (match_operand:DI  1 "general_operand"))
+   (match_operand 3 "immediate_operand")]
+ ""
+ {
+  if (bpf_expand_setmem (operands))
+DONE;
+  FAIL;
+})
+
 (include "atomic.md")
diff --git a/gcc/testsuite/gcc.target/bpf/memset-1.c 
b/gcc/testsuite/gcc.target/bpf/memset-1.c
new file mode 100644
index 000..9e9f8eff028
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/memset-1.c
@@ -0,0 +1,39 @@
+/* Ensure memset is expanded inline rather than emitting a libcall.  */
+
+/* { dg-do compile } */
+/* { dg-options "-O2" } */
+
+struct context {
+ unsigned int data;
+ unsigned int data_end;
+ unsigned int data_meta;
+ unsigned int ingress;
+ unsigned int queue_index;
+ unsigned int egress;
+};
+
+void
+set_small (struct context *ctx)
+{
+  void *data = (void *)(long)ctx->data;
+  char *dest = data;
+  __builtin_memset (dest + 4, 0, sizeof (struct context) - 4);
+}
+
+void
+

Re: [PATCH,V2] ctf: fix incorrect CTF for multi-dimensional array types

[David Faust]

Hi Indu, Cupertino,

On 3/4/24 10:00, Indu Bhagat wrote:
> From: Cupertino Miranda 
> 
> [Changes from V1]
>   - Refactor the code a bit.
> [End of changes from V1]
> 
> PR debug/114186
> 
> DWARF DIEs of type DW_TAG_subrange_type are linked together to represent
> the information about the subsequent dimensions.  The CTF processing was
> so far working through them in the opposite (incorrect) order.
> 
> While fixing the issue, refactor the code a bit for readability.
> 
> co-authored-By: Indu Bhagat 

Thanks for the patch and refactor. I do find v2 easier to follow.
Two very minor typos in comments, noted inline below.

Otherwise, LGTM and OK.
Thanks!

> 
> gcc/
>   PR debug/114186
>   * dwarf2ctf.cc (gen_ctf_array_type): Invoke the ctf_add_array ()
>   in the correct order of the dimensions.
> (gen_ctf_subrange_type): Refactor out handling of
>   DW_TAG_subrange_type DIE to here.
> 
> gcc/testsuite/
>   PR debug/114186
>   * gcc.dg/debug/ctf/ctf-array-6.c: Add test.
> ---
> 
> Testing notes:
> 
> Regression tested on x86_64-linux-gnu default target.
> Regression tested for target bpf-unknown-none (btf.exp, ctf.exp, bpf.exp).
> 
> ---
>  gcc/dwarf2ctf.cc | 153 +--
>  gcc/testsuite/gcc.dg/debug/ctf/ctf-array-6.c |  14 ++
>  2 files changed, 84 insertions(+), 83 deletions(-)
>  create mode 100644 gcc/testsuite/gcc.dg/debug/ctf/ctf-array-6.c
> 
> diff --git a/gcc/dwarf2ctf.cc b/gcc/dwarf2ctf.cc
> index dca86edfffa9..3985de115a79 100644
> --- a/gcc/dwarf2ctf.cc
> +++ b/gcc/dwarf2ctf.cc
> @@ -349,105 +349,92 @@ gen_ctf_pointer_type (ctf_container_ref ctfc, 
> dw_die_ref ptr_type)
>return ptr_type_id;
>  }
>  
> -/* Generate CTF for an array type.  */
> +/* Recursively generate CTF for array dimensions starting at DIE C (of type
> +   DW_TAG_subrange_type) until DIE LAST (of type DW_TAG_subrange_type) is
> +   reached.  ARRAY_ELEMS_TYPE_ID is base type for the array.  */
>  
>  static ctf_id_t
> -gen_ctf_array_type (ctf_container_ref ctfc, dw_die_ref array_type)
> +gen_ctf_subrange_type (ctf_container_ref ctfc, ctf_id_t array_elems_type_id,
> +dw_die_ref c, dw_die_ref last)
>  {
> -  dw_die_ref c;
> -  ctf_id_t array_elems_type_id = CTF_NULL_TYPEID;
> +  ctf_arinfo_t arinfo;
> +  ctf_id_t array_node_type_id = CTF_NULL_TYPEID;
> +
> +  dw_attr_node *upper_bound_at;
> +  dw_die_ref array_index_type;
> +  uint32_t array_num_elements;
> +
> +  /* When DW_AT_upper_bound is used to specify the size of an
> + array in DWARF, it is usually an unsigned constant
> + specifying the upper bound index of the array.  However,
> + for unsized arrays, such as foo[] or bar[0],
> + DW_AT_upper_bound is a signed integer constant
> + instead.  */
> +
> +  upper_bound_at = get_AT (c, DW_AT_upper_bound);
> +  if (upper_bound_at
> +  && AT_class (upper_bound_at) == dw_val_class_unsigned_const)
> +/* This is the ound index.  */

typo, I guess this is meant to be "bound" ? 

> +array_num_elements = get_AT_unsigned (c, DW_AT_upper_bound) + 1;
> +  else if (get_AT (c, DW_AT_count))
> +array_num_elements = get_AT_unsigned (c, DW_AT_count);
> +  else
> +{
> +  /* This is a VLA of some kind.  */
> +  array_num_elements = 0;
> +}
>  
> -  int vector_type_p = get_AT_flag (array_type, DW_AT_GNU_vector);
> -  if (vector_type_p)
> -return array_elems_type_id;
> +  /* Ok, mount and register the array type.  Note how the array
> + type we register here is the type of the elements in
> + subsequent "dimensions", if there are any.  */
> +  arinfo.ctr_nelems = array_num_elements;
>  
> -  dw_die_ref array_elems_type = ctf_get_AT_type (array_type);
> +  array_index_type = ctf_get_AT_type (c);
> +  arinfo.ctr_index = gen_ctf_type (ctfc, array_index_type);
>  
> -  /* First, register the type of the array elements if needed.  */
> -  array_elems_type_id = gen_ctf_type (ctfc, array_elems_type);
> +  if (c == last)
> +arinfo.ctr_contents = array_elems_type_id;
> +  else
> +arinfo.ctr_contents = gen_ctf_subrange_type (ctfc, array_elems_type_id,
> +  dw_get_die_sib (c), last);
>  
> -  /* DWARF array types pretend C supports multi-dimensional arrays.
> - So for the type int[N][M], the array type DIE contains two
> - subrange_type children, the first with upper bound N-1 and the
> - second with upper bound M-1.
> +  if (!ctf_type_exists (ctfc, c, &array_node_type_id))
> +array_node_type_id = ctf_add_array (ctfc, CTF_ADD_ROOT, &arinfo, c);
>  
> - CTF, on the other hand, just encodes each array type in its own
> - array type CTF struct.  Therefore we have to iterate on the
> - children and create all the needed types.  */
> +  return array_node_type_id;
> +}
>  
> -  c = dw_get_die_child (array_type);
> -  gcc_assert (c);
> -  do
> -{
> -  ctf_arinfo_t arinfo;
> -  dw_die_ref array_index_type;
> -  uint

Re: [PATCH, V3] ctf: fix incorrect CTF for multi-dimensional array types

[David Faust]

On 3/5/24 00:47, Indu Bhagat wrote:
> From: Cupertino Miranda 
> 
> [Changes from V2]
>   - Fixed aarch64 new FAILs reported by Linaro CI.
>   - Fixed typos and other nits pointed out in V2.
> [End of changes from V2]

OK, thanks.

> 
> PR debug/114186
> 
> DWARF DIEs of type DW_TAG_subrange_type are linked together to represent
> the information about the subsequent dimensions.  The CTF processing was
> so far working through them in the opposite (incorrect) order.
> 
> While fixing the issue, refactor the code a bit for readability.
> 
> co-authored-By: Indu Bhagat 
> 
> gcc/
>   PR debug/114186
>   * dwarf2ctf.cc (gen_ctf_array_type): Invoke the ctf_add_array ()
>   in the correct order of the dimensions.
> (gen_ctf_subrange_type): Refactor out handling of
>   DW_TAG_subrange_type DIE to here.
> 
> gcc/testsuite/
>   PR debug/114186
>   * gcc.dg/debug/ctf/ctf-array-6.c: Add test.
> ---
> 
> Testing notes:
>  - Linaro CI reported three new FAILs introduced by ctf-array-6.c due to
>presence of char '#' on aarch64 where the ASM_COMMENT_START differs.
>Fixed and regression tested on aarch64.
>  - Regression tested on x86_64-linux-gnu default target.
>  - Regression tested for target bpf-unknown-none (btf.exp, ctf.exp, bpf.exp).
>  - Kernel build with -gctf shows healthier CTF types for arrays.
> 
> ---
>  gcc/dwarf2ctf.cc | 158 +--
>  gcc/testsuite/gcc.dg/debug/ctf/ctf-array-6.c |  14 ++
>  2 files changed, 89 insertions(+), 83 deletions(-)
>  create mode 100644 gcc/testsuite/gcc.dg/debug/ctf/ctf-array-6.c
> 
> diff --git a/gcc/dwarf2ctf.cc b/gcc/dwarf2ctf.cc
> index dca86edfffa9..77d6bf896893 100644
> --- a/gcc/dwarf2ctf.cc
> +++ b/gcc/dwarf2ctf.cc
> @@ -349,105 +349,97 @@ gen_ctf_pointer_type (ctf_container_ref ctfc, 
> dw_die_ref ptr_type)
>return ptr_type_id;
>  }
>  
> -/* Generate CTF for an array type.  */
> +/* Recursively generate CTF for array dimensions starting at DIE C (of type
> +   DW_TAG_subrange_type) until DIE LAST (of type DW_TAG_subrange_type) is
> +   reached.  ARRAY_ELEMS_TYPE_ID is base type for the array.  */
>  
>  static ctf_id_t
> -gen_ctf_array_type (ctf_container_ref ctfc, dw_die_ref array_type)
> +gen_ctf_subrange_type (ctf_container_ref ctfc, ctf_id_t array_elems_type_id,
> +dw_die_ref c, dw_die_ref last)
>  {
> -  dw_die_ref c;
> -  ctf_id_t array_elems_type_id = CTF_NULL_TYPEID;
> +  ctf_arinfo_t arinfo;
> +  ctf_id_t array_node_type_id = CTF_NULL_TYPEID;
>  
> -  int vector_type_p = get_AT_flag (array_type, DW_AT_GNU_vector);
> -  if (vector_type_p)
> -return array_elems_type_id;
> +  dw_attr_node *upper_bound_at;
> +  dw_die_ref array_index_type;
> +  uint32_t array_num_elements;
>  
> -  dw_die_ref array_elems_type = ctf_get_AT_type (array_type);
> +  if (dw_get_die_tag (c) == DW_TAG_subrange_type)
> +{
> +  /* When DW_AT_upper_bound is used to specify the size of an
> +  array in DWARF, it is usually an unsigned constant
> +  specifying the upper bound index of the array.  However,
> +  for unsized arrays, such as foo[] or bar[0],
> +  DW_AT_upper_bound is a signed integer constant
> +  instead.  */
> +
> +  upper_bound_at = get_AT (c, DW_AT_upper_bound);
> +  if (upper_bound_at
> +   && AT_class (upper_bound_at) == dw_val_class_unsigned_const)
> + /* This is the upper bound index.  */
> + array_num_elements = get_AT_unsigned (c, DW_AT_upper_bound) + 1;
> +  else if (get_AT (c, DW_AT_count))
> + array_num_elements = get_AT_unsigned (c, DW_AT_count);
> +  else
> + {
> +   /* This is a VLA of some kind.  */
> +   array_num_elements = 0;
> + }
> +}
> +  else
> +gcc_unreachable ();
>  
> -  /* First, register the type of the array elements if needed.  */
> -  array_elems_type_id = gen_ctf_type (ctfc, array_elems_type);
> +  /* Ok, mount and register the array type.  Note how the array
> + type we register here is the type of the elements in
> + subsequent "dimensions", if there are any.  */
> +  arinfo.ctr_nelems = array_num_elements;
>  
> -  /* DWARF array types pretend C supports multi-dimensional arrays.
> - So for the type int[N][M], the array type DIE contains two
> - subrange_type children, the first with upper bound N-1 and the
> - second with upper bound M-1.
> +  array_index_type = ctf_get_AT_type (c);
> +  arinfo.ctr_index = gen_ctf_type (ctfc, array_index_type);
>  
> - CTF, on the other hand, just encodes each array type in its own
> - array type CTF struct.  Therefore we have to iterate on the
> - children and create all the needed types.  */
> +  if (c == last)
> +arinfo.ctr_contents = array_elems_type_id;
> +  else
> +arinfo.ctr_contents = gen_ctf_subrange_type (ctfc, array_elems_type_id,
> +  dw_get_die_sib (c), last);
>  
> -  c = dw_get_die_child (array_type);
> -  gcc_assert (c

[PATCH] bpf: add size threshold for inlining mem builtins

[David Faust]

BPF cannot fall back on library calls to implement memmove, memcpy and
memset, so we attempt to expand these inline always if possible.
However, this inline expansion was being attempted even for excessively
large operations, which could result in gcc consuming huge amounts of
memory and hanging.

Add a size threshold in the BPF backend below which to always expand
these operations inline, and introduce an option
-minline-memops-threshold= to control the threshold. Defaults to
1024 bytes.

Tested on x86_64-linux-gnu host for bpf-unknown-none target.
Fixes hang in test gcc.c-torture/compile/20050622-1.c for BPF, which
returns to (correctly) failing due to exceeding the eBPF stack limit.

gcc/

* config/bpf/bpf.cc (bpf_expand_cpymem, bpf_expand_setmem): Do
not attempt inline expansion if size is above threshold.
* config/bpf/bpf.opt (-minline-memops-threshold): New option.
* doc/invoke.texi (eBPF Options) <-minline-memops-threshold>:
Document.

gcc/testsuite/

* gcc.target/bpf/inline-memops-threshold-1.c: New test.
* gcc.target/bpf/inline-memops-threshold-2.c: New test.
---
 gcc/config/bpf/bpf.cc |  8 
 gcc/config/bpf/bpf.opt|  4 
 gcc/doc/invoke.texi   |  9 -
 .../gcc.target/bpf/inline-memops-threshold-1.c| 15 +++
 .../gcc.target/bpf/inline-memops-threshold-2.c| 14 ++
 5 files changed, 49 insertions(+), 1 deletion(-)
 create mode 100644 gcc/testsuite/gcc.target/bpf/inline-memops-threshold-1.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/inline-memops-threshold-2.c

diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc
index 0e33f4347ba..3f3dcb0a46b 100644
--- a/gcc/config/bpf/bpf.cc
+++ b/gcc/config/bpf/bpf.cc
@@ -1275,6 +1275,10 @@ bpf_expand_cpymem (rtx *operands, bool is_move)
   gcc_unreachable ();
 }
 
+  /* For sizes above threshold, always use a libcall.  */
+  if (size_bytes > (unsigned HOST_WIDE_INT) bpf_inline_memops_threshold)
+return false;
+
   unsigned iters = size_bytes >> ceil_log2 (align);
   unsigned remainder = size_bytes & (align - 1);
 
@@ -1347,6 +1351,10 @@ bpf_expand_setmem (rtx *operands)
   gcc_unreachable ();
 }
 
+  /* For sizes above threshold, always use a libcall.  */
+  if (size_bytes > (unsigned HOST_WIDE_INT) bpf_inline_memops_threshold)
+return false;
+
   unsigned iters = size_bytes >> ceil_log2 (align);
   unsigned remainder = size_bytes & (align - 1);
   unsigned inc = GET_MODE_SIZE (mode);
diff --git a/gcc/config/bpf/bpf.opt b/gcc/config/bpf/bpf.opt
index acfddebdad7..541ebe4dfc4 100644
--- a/gcc/config/bpf/bpf.opt
+++ b/gcc/config/bpf/bpf.opt
@@ -108,3 +108,7 @@ Enum(asm_dialect) String(normal) Value(ASM_NORMAL)
 
 EnumValue
 Enum(asm_dialect) String(pseudoc) Value(ASM_PSEUDOC)
+
+minline-memops-threshold=
+Target RejectNegative Joined UInteger Var(bpf_inline_memops_threshold) 
Init(1024)
+-minline-memops-threshold= Maximum size of memset/memmove/memcpy to 
inline, larger sizes will use a library call.
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index 2390d478121..7a965631123 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -971,7 +971,7 @@ Objective-C and Objective-C++ Dialects}.
 @gccoptlist{-mbig-endian -mlittle-endian
 -mframe-limit=@var{bytes} -mxbpf -mco-re -mno-co-re -mjmpext
 -mjmp32 -malu32 -mv3-atomics -mbswap -msdiv -msmov -mcpu=@var{version}
--masm=@var{dialect}}
+-masm=@var{dialect} -minline-memops-threshold=@var{bytes}}
 
 @emph{FR30 Options}
 @gccoptlist{-msmall-model  -mno-lsim}
@@ -25700,6 +25700,13 @@ Outputs pseudo-c assembly dialect.
 
 @end table
 
+@opindex -minline-memops-threshold
+@item -minline-memops-threshold=@var{bytes}
+Specifies a size threshold in bytes at or below which memmove, memcpy
+and memset shall always be expanded inline.  Operations dealing with
+sizes larger than this threshold will be implemented using a library
+call instead of being expanded inline.  The default is @samp{1024}.
+
 @end table
 
 @node FR30 Options
diff --git a/gcc/testsuite/gcc.target/bpf/inline-memops-threshold-1.c 
b/gcc/testsuite/gcc.target/bpf/inline-memops-threshold-1.c
new file mode 100644
index 000..c2ba4db5b7b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/inline-memops-threshold-1.c
@@ -0,0 +1,15 @@
+
+/* { dg-do compile } */
+/* { dg-options "-O2" "-minline-memops-threshold=256"} */
+
+char buf[512];
+
+void
+mov_small (void)
+{
+  __builtin_memmove (buf, buf + 2, 255);
+}
+
+/* { dg-final { scan-assembler-not "call" } } */
+/* { dg-final { scan-assembler "ldxb" } } */
+/* { dg-final { scan-assembler "stxb" } } */
diff --git a/gcc/testsuite/gcc.target/bpf/inline-memops-threshold-2.c 
b/gcc/testsuite/gcc.target/bpf/inline-memops-threshold-2.c
new file mode 100644
index 000..190b10b579c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/inline-memops-threshold-2.c
@@ -0,0 +1,14 @@
+/* { dg

[PATCH] bpf: testsuite: fix unresolved test in memset-1.c

[David Faust]

The test was trying to do too much by both checking for an error, and
checking the resulting assembly. Of course, due to the error no asm was
produced, so the scan-asm went unresolved. Split it into two separate
tests to fix the issue.

Tested on x86_64-linux-gnu host for bpf-unknown-none target.

gcc/testsuite/

* gcc.target/bpf/memset-1.c: Move error test case to...
* gcc.target/bpf/memset-2.c: ... here. New test.
---
 gcc/testsuite/gcc.target/bpf/memset-1.c |  8 
 gcc/testsuite/gcc.target/bpf/memset-2.c | 22 ++
 2 files changed, 22 insertions(+), 8 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/bpf/memset-2.c

diff --git a/gcc/testsuite/gcc.target/bpf/memset-1.c 
b/gcc/testsuite/gcc.target/bpf/memset-1.c
index 9e9f8eff028..7c4768c6e73 100644
--- a/gcc/testsuite/gcc.target/bpf/memset-1.c
+++ b/gcc/testsuite/gcc.target/bpf/memset-1.c
@@ -28,12 +28,4 @@ set_large (struct context *ctx)
   __builtin_memset (dest, 0xfe, 130);
 }
 
-void
-set_variable (struct context *ctx)
-{
-  void *data = (void *)(long)ctx->data;
-  char *dest = data;
-  __builtin_memset (dest, 0xbc, ctx->data_meta); /* { dg-error "could not 
inline call" } */
-}
-
 /* { dg-final { scan-assembler-times "call" 0 } } */
diff --git a/gcc/testsuite/gcc.target/bpf/memset-2.c 
b/gcc/testsuite/gcc.target/bpf/memset-2.c
new file mode 100644
index 000..0602a1a277c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/memset-2.c
@@ -0,0 +1,22 @@
+/* Test that we error if memset cannot be expanded inline.  */
+
+/* { dg-do compile } */
+/* { dg-options "-O2" } */
+
+struct context {
+ unsigned int data;
+ unsigned int data_end;
+ unsigned int data_meta;
+ unsigned int ingress;
+ unsigned int queue_index;
+ unsigned int egress;
+};
+
+
+void
+set_variable (struct context *ctx)
+{
+  void *data = (void *)(long)ctx->data;
+  char *dest = data;
+  __builtin_memset (dest, 0xbc, ctx->data_meta); /* { dg-error "could not 
inline call" } */
+}
-- 
2.43.0

[PATCH v2] bpf: add size threshold for inlining mem builtins

[David Faust]

[Changes from v1:
 - Error if threshold is exceeded instead of silently emitting libcall
 - Update test accordingly
 - Expand documentation to explain this behavior  ]

BPF cannot fall back on library calls to implement memmove, memcpy and
memset, so we attempt to expand these inline always if possible.
However, this inline expansion was being attempted even for excessively
large operations, which could result in gcc consuming huge amounts of
memory and hanging.

Add a size threshold in the BPF backend below which to always expand
these operations inline, and introduce an option
-minline-memops-threshold= to control the threshold. Defaults to
1024 bytes.

gcc/

* config/bpf/bpf.cc (bpf_expand_cpymem, bpf_expand_setmem): Do
not attempt inline expansion if size is above threshold.
* config/bpf/bpf.opt (-minline-memops-threshold): New option.
* doc/invoke.texi (eBPF Options) <-minline-memops-threshold>:
Document.

gcc/testsuite/

* gcc.target/bpf/inline-memops-threshold-1.c: New test.
* gcc.target/bpf/inline-memops-threshold-2.c: New test.
---
 gcc/config/bpf/bpf.cc | 26 ++-
 gcc/config/bpf/bpf.opt|  4 +++
 gcc/doc/invoke.texi   | 11 +++-
 .../bpf/inline-memops-threshold-1.c   | 15 +++
 .../bpf/inline-memops-threshold-2.c   | 11 
 5 files changed, 65 insertions(+), 2 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/bpf/inline-memops-threshold-1.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/inline-memops-threshold-2.c

diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc
index 0e33f4347ba..8365cd9fcb1 100644
--- a/gcc/config/bpf/bpf.cc
+++ b/gcc/config/bpf/bpf.cc
@@ -1244,9 +1244,9 @@ bool
 bpf_expand_cpymem (rtx *operands, bool is_move)
 {
   /* Size must be constant for this expansion to work.  */
+  const char *name = is_move ? "memmove" : "memcpy";
   if (!CONST_INT_P (operands[2]))
 {
-  const char *name = is_move ? "memmove" : "memcpy";
   if (flag_building_libgcc)
warning (0, "could not inline call to %<__builtin_%s%>: "
 "size must be constant", name);
@@ -1275,6 +1275,18 @@ bpf_expand_cpymem (rtx *operands, bool is_move)
   gcc_unreachable ();
 }
 
+  /* For sizes above threshold, always use a libcall.  */
+  if (size_bytes > (unsigned HOST_WIDE_INT) bpf_inline_memops_threshold)
+{
+  if (flag_building_libgcc)
+   warning (0, "could not inline call to %<__builtin_%s%>: "
+"too many bytes, use -minline-memops-threshold", name);
+  else
+   error ("could not inline call to %<__builtin_%s%>: "
+  "too many bytes, use -minline-memops-threshold", name);
+  return false;
+}
+
   unsigned iters = size_bytes >> ceil_log2 (align);
   unsigned remainder = size_bytes & (align - 1);
 
@@ -1347,6 +1359,18 @@ bpf_expand_setmem (rtx *operands)
   gcc_unreachable ();
 }
 
+  /* For sizes above threshold, always use a libcall.  */
+  if (size_bytes > (unsigned HOST_WIDE_INT) bpf_inline_memops_threshold)
+{
+  if (flag_building_libgcc)
+   warning (0, "could not inline call to %<__builtin_memset%>: "
+"too many bytes, use -minline-memops-threshold");
+  else
+   error ("could not inline call to %<__builtin_memset%>: "
+  "too many bytes, use -minline-memops-threshold");
+  return false;
+}
+
   unsigned iters = size_bytes >> ceil_log2 (align);
   unsigned remainder = size_bytes & (align - 1);
   unsigned inc = GET_MODE_SIZE (mode);
diff --git a/gcc/config/bpf/bpf.opt b/gcc/config/bpf/bpf.opt
index acfddebdad7..541ebe4dfc4 100644
--- a/gcc/config/bpf/bpf.opt
+++ b/gcc/config/bpf/bpf.opt
@@ -108,3 +108,7 @@ Enum(asm_dialect) String(normal) Value(ASM_NORMAL)
 
 EnumValue
 Enum(asm_dialect) String(pseudoc) Value(ASM_PSEUDOC)
+
+minline-memops-threshold=
+Target RejectNegative Joined UInteger Var(bpf_inline_memops_threshold) 
Init(1024)
+-minline-memops-threshold= Maximum size of memset/memmove/memcpy to 
inline, larger sizes will use a library call.
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index c0d604a2c5c..85c938d4a14 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -971,7 +971,7 @@ Objective-C and Objective-C++ Dialects}.
 @gccoptlist{-mbig-endian -mlittle-endian
 -mframe-limit=@var{bytes} -mxbpf -mco-re -mno-co-re -mjmpext
 -mjmp32 -malu32 -mv3-atomics -mbswap -msdiv -msmov -mcpu=@var{version}
--masm=@var{dialect}}
+-masm=@var{dialect} -minline-memops-threshold=@var{bytes}}
 
 @emph{FR30 Options}
 @gccoptlist{-msmall-model  -mno-lsim}
@@ -25701,6 +25701,15 @@ Outputs pseudo-c assembly dialect.
 
 @end table
 
+@opindex -minline-memops-threshold
+@item -minline-memops-threshold=@var{bytes}
+Specifies a size threshold in bytes at or below which memmove, memcpy
+and memset shall always be expanded inline.  Operations dealing with
+sizes l

[PATCH] bpf: define INT8_TYPE as signed char

[David Faust]

Change the BPF backend to define INT8_TYPE with an explicit sign, rather
than a plain char.  This is in line with other targets and removes the
risk of int8_t being affected by the signedness of the plain char type
of the host system.

Tested on x86_64-linux-gnu host for bpf-unknown-none.
Sanity checked compiling Linux kernel BPF selftests.

gcc/

* config/bpf/bpf.h (INT8_TYPE): Change to signed char.
---
 gcc/config/bpf/bpf.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gcc/config/bpf/bpf.h b/gcc/config/bpf/bpf.h
index f107a5a4c34..3cc5daa1b58 100644
--- a/gcc/config/bpf/bpf.h
+++ b/gcc/config/bpf/bpf.h
@@ -99,7 +99,7 @@
 
 #define SIG_ATOMIC_TYPE "char"
 
-#define INT8_TYPE "char"
+#define INT8_TYPE "signed char"
 #define INT16_TYPE "short int"
 #define INT32_TYPE "int"
 #define INT64_TYPE "long int"
-- 
2.43.0

Re: [PATCH] bpf: define INT8_TYPE as signed char

[David Faust]

On 3/14/24 10:16, Jose E. Marchesi wrote:
> 
> Hi David.
> 
>> Change the BPF backend to define INT8_TYPE with an explicit sign, rather
>> than a plain char.  This is in line with other targets and removes the
>> risk of int8_t being affected by the signedness of the plain char type
>> of the host system.
> 
> OK.
> 
> I would add to the commit message that the motivation for this change is
> that even if `char' is defined to be signed in BPF targets, some BPF
> programs use the (mal)practice of including internal libc headers
> indirectly via kernel headers and that may trigger compilation errors
> regarding redefinitions of types.

Thanks, added this to the commit message and pushed.

> 
> Thanks for the patch!
> 
>>
>> Tested on x86_64-linux-gnu host for bpf-unknown-none.
>> Sanity checked compiling Linux kernel BPF selftests.
>>
>> gcc/
>>
>>  * config/bpf/bpf.h (INT8_TYPE): Change to signed char.
>> ---
>>  gcc/config/bpf/bpf.h | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/gcc/config/bpf/bpf.h b/gcc/config/bpf/bpf.h
>> index f107a5a4c34..3cc5daa1b58 100644
>> --- a/gcc/config/bpf/bpf.h
>> +++ b/gcc/config/bpf/bpf.h
>> @@ -99,7 +99,7 @@
>>  
>>  #define SIG_ATOMIC_TYPE "char"
>>  
>> -#define INT8_TYPE "char"
>> +#define INT8_TYPE "signed char"
>>  #define INT16_TYPE "short int"
>>  #define INT32_TYPE "int"
>>  #define INT64_TYPE "long int"

Re: [PATCH 2/3] bpf: Fix access string default for CO-RE type based relocations

[David Faust]

On 3/13/24 07:24, Cupertino Miranda wrote:
> Although part of all CO-RE relocation data, type based relocations do
> not require an access string.
> Initial implementation defined it as an empty string.
> On the other hand, libbpf when parsing the CO-RE relocations verifies
> that those strings would contain "0", otherwise reports an error.
> This patch makes GCC compliant with libbpf expectations.

OK, thanks.

> 
> gcc/Changelog:
>   * config/bpf/btfext-out.cc (cpf_core_reloc_add): Correct for new code.
>   Add assert to validate the string is set.
>   * config/bpf/core-builtins.cc (cr_final): Make string struct
>   field as const.
>   (process_enum_value): Correct for field type change.
>   (process_type): Set access string to "0".
> 
> gcc/testsuite/ChangeLog:
>   * gcc.target/bpf/core-builtin-type-based.c: Correct.
>   * gcc.target/bpf/core-builtin-type-id.c: Correct.
> ---
>  gcc/config/bpf/btfext-out.cc   |  5 +++--
>  gcc/config/bpf/core-builtins.cc| 10 ++
>  gcc/testsuite/gcc.target/bpf/core-builtin-type-based.c |  1 +
>  gcc/testsuite/gcc.target/bpf/core-builtin-type-id.c|  1 +
>  4 files changed, 11 insertions(+), 6 deletions(-)
> 
> diff --git a/gcc/config/bpf/btfext-out.cc b/gcc/config/bpf/btfext-out.cc
> index 57c0dc323812..ff1fd0739f1e 100644
> --- a/gcc/config/bpf/btfext-out.cc
> +++ b/gcc/config/bpf/btfext-out.cc
> @@ -299,8 +299,9 @@ bpf_core_reloc_add (const tree type, const char * 
> section_name,
>  
>/* Buffer the access string in the auxiliary strtab.  */
>bpfcr->bpfcr_astr_off = 0;
> -  if (accessor != NULL)
> -bpfcr->bpfcr_astr_off = btf_ext_add_string (accessor);
> +  gcc_assert (accessor != NULL);
> +  bpfcr->bpfcr_astr_off = btf_ext_add_string (accessor);
> +
>bpfcr->bpfcr_type = get_btf_id (ctf_lookup_tree_type (ctfc, type));
>bpfcr->bpfcr_insn_label = label;
>bpfcr->bpfcr_kind = kind;
> diff --git a/gcc/config/bpf/core-builtins.cc b/gcc/config/bpf/core-builtins.cc
> index 4256fea15e49..70b14e48e6e5 100644
> --- a/gcc/config/bpf/core-builtins.cc
> +++ b/gcc/config/bpf/core-builtins.cc
> @@ -205,7 +205,7 @@ struct cr_local
>  /* Core Relocation Final data */
>  struct cr_final
>  {
> -  char *str;
> +  const char *str;
>tree type;
>enum btf_core_reloc_kind kind;
>  };
> @@ -868,8 +868,10 @@ process_enum_value (struct cr_builtins *data)
>   {
> if (TREE_VALUE (l) == expr)
>   {
> -   ret.str = (char *) ggc_alloc_atomic ((index / 10) + 1);
> -   sprintf (ret.str, "%d", index);
> +   char *tmp = (char *) ggc_alloc_atomic ((index / 10) + 1);
> +   sprintf (tmp, "%d", index);
> +   ret.str = (const char *) tmp;
> +
> break;
>   }
> index++;
> @@ -987,7 +989,7 @@ process_type (struct cr_builtins *data)
> || data->kind == BPF_RELO_TYPE_MATCHES);
>  
>struct cr_final ret;
> -  ret.str = NULL;
> +  ret.str = ggc_strdup ("0");
>ret.type = data->type;
>ret.kind = data->kind;
>  
> diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-type-based.c 
> b/gcc/testsuite/gcc.target/bpf/core-builtin-type-based.c
> index 74a8d5a14d9d..9d818133c084 100644
> --- a/gcc/testsuite/gcc.target/bpf/core-builtin-type-based.c
> +++ b/gcc/testsuite/gcc.target/bpf/core-builtin-type-based.c
> @@ -56,3 +56,4 @@ int foo(void *data)
>  /* { dg-final { scan-assembler-times "0x8\[\t \]+\[^\n\]*bpfcr_kind" 13 } } 
> BPF_TYPE_EXISTS */
>  /* { dg-final { scan-assembler-times "0x9\[\t \]+\[^\n\]*bpfcr_kind" 11 } } 
> BPF_TYPE_SIZE */
>  /* { dg-final { scan-assembler-times "0xc\[\t \]+\[^\n\]*bpfcr_kind" 13 } } 
> BPF_TYPE_MATCHES */
> +/* { dg-final { scan-assembler-times "bpfcr_astr_off \[(\"\]+0\[(\"\]+" 37 } 
> } */
> diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-type-id.c 
> b/gcc/testsuite/gcc.target/bpf/core-builtin-type-id.c
> index 4b23288eac08..9576b91bc940 100644
> --- a/gcc/testsuite/gcc.target/bpf/core-builtin-type-id.c
> +++ b/gcc/testsuite/gcc.target/bpf/core-builtin-type-id.c
> @@ -38,3 +38,4 @@ int foo(void *data)
>  /* { dg-final { scan-assembler-times "0\[\t \]+\[^\n\]*bpfcr_type" 0  { 
> xfail *-*-* } } } */
>  /* { dg-final { scan-assembler-times "0x6\[\t \]+\[^\n\]*bpfcr_kind" 13 } } 
> BPF_TYPE_ID_LOCAL */
>  /* { dg-final { scan-assembler-times "0x7\[\t \]+\[^\n\]*bpfcr_kind" 7 } } 
> BPF_TYPE_ID_TARGET */
> +/* { dg-final { scan-assembler-times "bpfcr_astr_off \[(\"\]+0\[(\"\]+" 20 } 
> } */

Re: [PATCH 3/3] bpf: Corrected index computation when present with unnamed struct fields

[David Faust]

On 3/13/24 07:24, Cupertino Miranda wrote:
> Any unnamed structure field if not a member of the BTF_KIND_STRUCT.
typo: if -> is

I'd suggest to clarify that "any unnamed structure field" is really
any unnamed non-struct-or-union field, since anonymous inner structs
and unions certainly are present in BTF (and you handle them here).

> For that reason, CO-RE access strings indexes should take that in
> consideration. This patch adds a condition to the incrementer that
> computes the index for the field access.

Otherwise, OK.
Thanks.

> 
> gcc/ChangeLog:
>   * config/bpf/core-builtins.cc (bpf_core_get_index): Check if
>   field contains a DECL_NAME.
> 
> gcc/testsuite/ChangeLog:
>   * gcc.target/bpf/core-builtin-fieldinfo-offset-1.c: Add
>   testcase for unnamed fields.
> ---
>  gcc/config/bpf/core-builtins.cc|  6 +-
>  .../gcc.target/bpf/core-builtin-fieldinfo-offset-1.c   | 10 --
>  2 files changed, 13 insertions(+), 3 deletions(-)
> 
> diff --git a/gcc/config/bpf/core-builtins.cc b/gcc/config/bpf/core-builtins.cc
> index 70b14e48e6e5..8333ad81d0e0 100644
> --- a/gcc/config/bpf/core-builtins.cc
> +++ b/gcc/config/bpf/core-builtins.cc
> @@ -553,7 +553,11 @@ bpf_core_get_index (const tree node, bool *valid)
>   {
> if (l == node)
>   return i;
> -   i++;
> +   /* Skip unnamed padding, not represented by BTF.  */
> +   if (DECL_NAME(l) != NULL_TREE
> +   || TREE_CODE (TREE_TYPE (l)) == UNION_TYPE
> +   || TREE_CODE (TREE_TYPE (l)) == RECORD_TYPE)
> + i++;
>   }
>  }
>else if (code == ARRAY_REF || code == ARRAY_RANGE_REF || code == MEM_REF)
> diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-fieldinfo-offset-1.c 
> b/gcc/testsuite/gcc.target/bpf/core-builtin-fieldinfo-offset-1.c
> index 27654205287d..8b1d8b012a2a 100644
> --- a/gcc/testsuite/gcc.target/bpf/core-builtin-fieldinfo-offset-1.c
> +++ b/gcc/testsuite/gcc.target/bpf/core-builtin-fieldinfo-offset-1.c
> @@ -14,6 +14,9 @@ struct T {
>struct S s[2];
>char c;
>char d;
> +  int a: 1;
> +  int:31;
> +  int f;
>  };
>  
>  enum {
> @@ -38,7 +41,9 @@ unsigned int foo (struct T *t)
>unsigned e1 = __builtin_preserve_field_info (bar()->d, FIELD_BYTE_OFFSET);
>unsigned e2 = __builtin_preserve_field_info (bar()->s[1].a4, 
> FIELD_BYTE_OFFSET);
>  
> -  return s0a1 + s0a4 + s0x + s1a1 + s1a4 + s1x + c + d + e1 + e2;
> +  unsigned f1 = __builtin_preserve_field_info (t->f, FIELD_BYTE_OFFSET);
> +
> +  return s0a1 + s0a4 + s0x + s1a1 + s1a4 + s1x + c + d + e1 + e2 + f1;
>  }
>  
>  /* { dg-final { scan-assembler-times "\[\t \]mov\[\t \]%r\[0-9\],4" 2 } } */
> @@ -65,5 +70,6 @@ unsigned int foo (struct T *t)
>  /* { dg-final { scan-assembler-times "bpfcr_astr_off \\(\"0:1:1:4\"\\)" 1 } 
> } */
>  /* { dg-final { scan-assembler-times "bpfcr_astr_off \\(\"0:2\"\\)" 1 } } */
>  /* { dg-final { scan-assembler-times "bpfcr_astr_off \\(\"0:3\"\\)" 2 } } */
> +/* { dg-final { scan-assembler-times "bpfcr_astr_off \\(\"0:5\"\\)" 1 } } */
>  
> -/* { dg-final { scan-assembler-times "0\[\t \]+\[^\n\]*bpfcr_kind" 10 } } */
> +/* { dg-final { scan-assembler-times "0\[\t \]+\[^\n\]*bpfcr_kind" 11 } } */

Re: [PATCH] btf: Emit labels in DATASEC bts_offset entries.

[David Faust]

On 3/26/24 07:28, Cupertino Miranda wrote:
> Hi everyone,
> 
> This patch is an expected fix for the issue reported by systemd in:
> https://github.com/systemd/systemd/issues/31888
> Also, Jose Marchesi opened the following bugzilla to report it:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114431
> 
> Please notice that the function created was inspired by existing code
> in the function btf_asm_type_ref.
> 
> Looking forward to your comments.

OK, LGTM.
Thanks for the fix!

> 
> Cupertino
> 
> 
> GCC was defining bts_offset entry to always contain 0.
> When comparing with clang, the same entry is instead a label to the
> respective variable or function. The assembler emits relocations for
> those labels.
> 
> gcc/ChangeLog:
>   PR target/114431
>   * btfout.cc (get_name_for_datasec_entry): Add function.
>   (btf_asm_datasec_entry): Print label when possible.
> 
> gcc/testsuite/ChangeLog:
>   gcc.dg/debug/btf/btf-datasec-1.c: Correct for new
>   implementation.
>   gcc.dg/debug/btf/btf-datasec-2.c: Likewise
>   gcc.dg/debug/btf/btf-pr106773.c: Likewise
> ---
>  gcc/btfout.cc | 30 ++-
>  .../gcc.dg/debug/btf/btf-datasec-1.c  |  6 ++--
>  .../gcc.dg/debug/btf/btf-datasec-2.c  |  7 +++--
>  gcc/testsuite/gcc.dg/debug/btf/btf-pr106773.c |  2 +-
>  4 files changed, 39 insertions(+), 6 deletions(-)
> 
> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
> index e63289bc554..5f708630e04 100644
> --- a/gcc/btfout.cc
> +++ b/gcc/btfout.cc
> @@ -1014,13 +1014,41 @@ btf_asm_func_type (ctf_container_ref ctfc, 
> ctf_dtdef_ref dtd, ctf_id_t id)
>btf_asm_type_ref ("btt_type", ctfc, get_btf_id (ref_id));
>  }
>  
> +/* Collect the name for the DATASEC reference required to be output as a
> +   symbol. */
> +
> +static const char *
> +get_name_for_datasec_entry (ctf_container_ref ctfc, ctf_id_t ref_id)
> +{
> +  if (ref_id >= num_types_added + 1
> +  && ref_id < num_types_added + num_vars_added + 1)
> +{
> +  /* Ref to a variable.  Should only appear in DATASEC entries.  */
> +  ctf_id_t var_id = btf_relative_var_id (ref_id);
> +  ctf_dvdef_ref dvd = ctfc->ctfc_vars_list[var_id];
> +  return dvd->dvd_name;
> +}
> +  else if (ref_id >= num_types_added + num_vars_added + 1)
> +{
> +  /* Ref to a FUNC record.  */
> +  size_t func_id = btf_relative_func_id (ref_id);
> +  ctf_dtdef_ref ref_type = (*funcs)[func_id];
> +  return get_btf_type_name (ref_type);
> +}
> +  return NULL;
> +}
> +
>  /* Asm'out a variable entry following a BTF_KIND_DATASEC.  */
>  
>  static void
>  btf_asm_datasec_entry (ctf_container_ref ctfc, struct btf_var_secinfo info)
>  {
> +  const char *symbol_name = get_name_for_datasec_entry (ctfc, info.type);
>btf_asm_type_ref ("bts_type", ctfc, info.type);
> -  dw2_asm_output_data (4, info.offset, "bts_offset");
> +  if (symbol_name == NULL)
> +dw2_asm_output_data (4, info.offset, "bts_offset");
> +  else
> +dw2_asm_output_offset (4, symbol_name, NULL, "bts_offset");
>dw2_asm_output_data (4, info.size, "bts_size");
>  }
>  
> diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-1.c 
> b/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-1.c
> index 77df88648c5..8557c38c20d 100644
> --- a/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-1.c
> +++ b/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-1.c
> @@ -19,8 +19,10 @@
>  /* { dg-final { scan-assembler-times "0xf03\[\t \]+\[^\n\]*btt_info" 2 } 
> } */
>  /* { dg-final { scan-assembler-times "0xf01\[\t \]+\[^\n\]*btt_info" 1 } 
> } */
>  
> -/* The offset entry for each variable in a DATSEC should be 0 at compile 
> time.  */
> -/* { dg-final { scan-assembler-times "0\[\t \]+\[^\n\]*bts_offset" 7 } } */
> +/* The offset entry for each variable in a DATSEC should contain a label.  */
> +/* { dg-final { scan-assembler-times ".4byte\[\t \]\[a-e\]\[\t 
> \]+\[^\n\]*bts_offset" 5 } } */
> +/* { dg-final { scan-assembler-times "my_cstruct\[\t \]+\[^\n\]*bts_offset" 
> 1 } } */
> +/* { dg-final { scan-assembler-times "bigarr\[\t \]+\[^\n\]*bts_offset" 1 } 
> } */
>  
>  /* Check that strings for each DATASEC have been added to the BTF string 
> table.  */
>  /* { dg-final { scan-assembler-times "ascii \".data.0\"\[\t 
> \]+\[^\n\]*btf_aux_string" 1 } } */
> diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-2.c 
> b/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-2.c
> index d6f3358d6fb..857d830e446 100644
> --- a/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-2.c
> +++ b/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-2.c
> @@ -9,8 +9,11 @@
>  /* { dg-final { scan-assembler-times " BTF_KIND_DATASEC 
> '.foo_sec'\[\\r\\n\]+\[^\\r\\n\]*0xf01\[\t \]+\[^\n\]*btt_info" 1 } } */
>  /* { dg-final { scan-assembler-times " BTF_KIND_DATASEC 
> '.bar_sec'\[\\r\\n\]+\[^\\r\\n\]*0xf02\[\t \]+\[^\n\]*btt_info" 1 } } */
>  
> -/* Function entries should have offset and size of 0 at compile time.  */
> -/* { dg-final { scan-assemb

Re: [PATCH] btf: Fix up btf-datasec-1.c test on x86

[David Faust]

Hi Jakub,

On 3/27/24 04:16, Jakub Jelinek wrote:
> On Wed, Mar 27, 2024 at 11:18:49AM +0100, Jakub Jelinek wrote:
>>> -/* The offset entry for each variable in a DATSEC should be 0 at compile 
>>> time.  */
>>> -/* { dg-final { scan-assembler-times "0\[\t \]+\[^\n\]*bts_offset" 7 } } */
>>> +/* The offset entry for each variable in a DATSEC should contain a label.  
>>> */
>>> +/* { dg-final { scan-assembler-times ".4byte\[\t \]\[a-e\]\[\t 
>>> \]+\[^\n\]*bts_offset" 5 } } */
>>
>> 4byte is used only on some targets, what exact assembler directive is used
>> for 4byte unaligned data is heavily target dependent.
> 
> Now in patch form, tested on x86_64-linux with
> make check-gcc RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} btf.exp'
> ok for trunk?

OK.
Thank you for fixing these.

> 
> 2024-03-27  Jakub Jelinek  
> 
>   * gcc.dg/debug/btf/btf-cvr-quals-1.c: Use dg-additional-options
>   instead of multiple dg-options.
>   * gcc.dg/debug/btf/btf-datasec-1.c: Likewise.  Accept all supported
>   unaligned 4 byte assembler directives rather than assuming it must
>   be .4byte.
> 
> --- gcc/testsuite/gcc.dg/debug/btf/btf-cvr-quals-1.c.jj   2021-07-12 
> 22:57:26.056103708 +0200
> +++ gcc/testsuite/gcc.dg/debug/btf/btf-cvr-quals-1.c  2024-03-27 
> 12:09:25.383278017 +0100
> @@ -23,7 +23,7 @@
>  
>  /* { dg-do compile } */
>  /* { dg-options "-O0 -gbtf -dA" } */
> -/* { dg-options "-O0 -gbtf -gdwarf-4 -dA" { target { *-*-darwin* } } } */
> +/* { dg-additional-options "-gdwarf-4" { target { *-*-darwin* } } } */
>  
>  /* { dg-final { scan-assembler-times "ascii \"int.0\"\[\t 
> \]+\[^\n\]*btf_string" 1 } } */
>  
> --- gcc/testsuite/gcc.dg/debug/btf/btf-datasec-1.c.jj 2024-03-26 
> 22:34:47.925757342 +0100
> +++ gcc/testsuite/gcc.dg/debug/btf/btf-datasec-1.c2024-03-27 
> 12:12:09.761045873 +0100
> @@ -11,16 +11,16 @@
>  
>  /* { dg-do compile )  */
>  /* { dg-options "-O0 -gbtf -dA" } */
> -/* { dg-options "-O0 -gbtf -dA -msdata=none" { target { { powerpc*-*-* } && 
> ilp32 } } } */
> -/* { dg-options "-O0 -gbtf -dA -msmall-data-limit=0" { target { riscv*-*-* } 
> } } */
> -/* { dg-options "-O0 -gbtf -dA -G0" { target { nios2-*-* } } } */
> +/* { dg-additional-options "-msdata=none" { target { { powerpc*-*-* } && 
> ilp32 } } } */
> +/* { dg-additional-options "-msmall-data-limit=0" { target { riscv*-*-* } } 
> } */
> +/* { dg-additional-options "-G0" { target { nios2-*-* } } } */
>  
>  /* Check for two DATASEC entries with vlen 3, and one with vlen 1.  */
>  /* { dg-final { scan-assembler-times "0xf03\[\t \]+\[^\n\]*btt_info" 2 } 
> } */
>  /* { dg-final { scan-assembler-times "0xf01\[\t \]+\[^\n\]*btt_info" 1 } 
> } */
>  
>  /* The offset entry for each variable in a DATSEC should contain a label.  */
> -/* { dg-final { scan-assembler-times ".4byte\[\t \]\[a-e\]\[\t 
> \]+\[^\n\]*bts_offset" 5 } } */
> +/* { dg-final { scan-assembler-times 
> "(?:(?:\\.4byte|\\.long|data4\\.ua|\\.ualong|\\.uaword|\\.dword|long|dc\\.l|\\.word)\[\t
>  \]|\\.vbyte\t4,\[\t \]?)\[a-e\]\[\t \]+\[^\n\]*bts_offset" 5 } } */
>  /* { dg-final { scan-assembler-times "my_cstruct\[\t \]+\[^\n\]*bts_offset" 
> 1 } } */
>  /* { dg-final { scan-assembler-times "bigarr\[\t \]+\[^\n\]*bts_offset" 1 } 
> } */
>  
> 
>   Jakub
> 

[PATCH] btf: emit symbol refs in DATASEC entries only for BPF [PR114608]

[David Faust]

The behavior introduced in
  fa60ac54964 btf: Emit labels in DATASEC bts_offset entries.

is only fully correct when compiling for the BPF target with BPF CO-RE
enabled.  In other cases, depending on optimizations, it can result in
an incorrect symbol reference in the entry bts_offset field for a symbol
which may not be emitted at all, causing link-time undefined symbol
reference errors like in PR114608.

The offending bts_offset field of BTF_KIND_DATASEC entries is in reality
only currently useful to consumers of BTF information for BPF programs
anyway.  Correct the regression by only emitting symbol references in
these entries when compiling for the BPF target.  For other targets, the
behavior returns to that prior to fa60ac54964.

The underlying cause is related to PR 113566 "btf: incorrect
BTF_KIND_DATASEC entries for variables which are optimized out." A
complete fix for 113566 is more involved and unsuitable for stage 4,
but will be addressed in the near future.

Tested on x86_64-linux-gnu and on x86_64-linux-gnu host for
bpf-unknown-none target.

OK?
Thanks.

gcc/
PR debug/114608
* btfout.cc (btf_asm_datasec_entry): Only emit a symbol reference when
generating BTF for BPF CO-RE target.

gcc/testsuite/
PR debug/114608
* gcc.dg/debug/btf/btf-datasec-1.c: Check bts_offset symbol references
only for BPF target.
* gcc.dg/debug/btf/btf-datasec-2.c: Likewise.
* gcc.dg/debug/btf/btf-pr106773.c: Likewise.
---
 gcc/btfout.cc  |  2 +-
 gcc/testsuite/gcc.dg/debug/btf/btf-datasec-1.c | 10 ++
 gcc/testsuite/gcc.dg/debug/btf/btf-datasec-2.c |  7 ---
 gcc/testsuite/gcc.dg/debug/btf/btf-pr106773.c  |  3 ++-
 4 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index 2e2b3524e83..4a8ec4d1ff0 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -1045,7 +1045,7 @@ btf_asm_datasec_entry (ctf_container_ref ctfc, struct 
btf_var_secinfo info)
 {
   const char *symbol_name = get_name_for_datasec_entry (ctfc, info.type);
   btf_asm_type_ref ("bts_type", ctfc, info.type);
-  if (symbol_name == NULL)
+  if (!btf_with_core_debuginfo_p () || symbol_name == NULL)
 dw2_asm_output_data (4, info.offset, "bts_offset");
   else
 dw2_asm_output_offset (4, symbol_name, NULL, "bts_offset");
diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-1.c 
b/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-1.c
index c8ebe5d07ca..15b76259218 100644
--- a/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-1.c
+++ b/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-1.c
@@ -19,10 +19,12 @@
 /* { dg-final { scan-assembler-times "0xf03\[\t \]+\[^\n\]*btt_info" 2 } } 
*/
 /* { dg-final { scan-assembler-times "0xf01\[\t \]+\[^\n\]*btt_info" 1 } } 
*/
 
-/* The offset entry for each variable in a DATSEC should contain a label.  */
-/* { dg-final { scan-assembler-times 
"(?:(?:\\.4byte|\\.long|data4\\.ua|\\.ualong|\\.uaword|\\.dword|long|dc\\.l|\\.word)\[\t
 \]|\\.vbyte\t4,\[\t \]?)_?\[a-e\]\[\t \]+\[^\n\]*bts_offset" 5 } } */
-/* { dg-final { scan-assembler-times "my_cstruct\[\t \]+\[^\n\]*bts_offset" 1 
} } */
-/* { dg-final { scan-assembler-times "bigarr\[\t \]+\[^\n\]*bts_offset" 1 } } 
*/
+/* { dg-final { scan-assembler-times "0\[\t \]+\[^\n\]*bts_offset" 7 { target 
{ ! bpf-*-* } } } } */
+
+/* For BPF target the offset entry for each variable in a DATSEC should 
contain a label.  */
+/* { dg-final { scan-assembler-times ".4byte\[\t \]\[a-e\]\[\t 
\]+\[^\n\]*bts_offset" 5 { target bpf-*-* } } } */
+/* { dg-final { scan-assembler-times "my_cstruct\[\t \]+\[^\n\]*bts_offset" 1 
{ target bpf-*-* } } } */
+/* { dg-final { scan-assembler-times "bigarr\[\t \]+\[^\n\]*bts_offset" 1 { 
target bpf-*-* } } } */
 
 /* Check that strings for each DATASEC have been added to the BTF string 
table.  */
 /* { dg-final { scan-assembler-times "ascii \".data.0\"\[\t 
\]+\[^\n\]*btf_aux_string" 1 } } */
diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-2.c 
b/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-2.c
index 857d830e446..a89a239a504 100644
--- a/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-2.c
+++ b/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-2.c
@@ -10,9 +10,10 @@
 /* { dg-final { scan-assembler-times " BTF_KIND_DATASEC 
'.bar_sec'\[\\r\\n\]+\[^\\r\\n\]*0xf02\[\t \]+\[^\n\]*btt_info" 1 } } */
 
 /* Function entries should have offset with a label and size of 0 at compile 
time.  */
-/* { dg-final { scan-assembler-times "chacha\[\t \]+\[^\n\]*bts_offset" 1 } } 
*/
-/* { dg-final { scan-assembler-times "bar\[\t \]+\[^\n\]*bts_offset" 1 } } */
-/* { dg-final { scan-assembler-times "foo\[\t \]+\[^\n\]*bts_offset" 1 } } */
+/* { dg-final { scan-assembler-times "chacha\[\t \]+\[^\n\]*bts_offset" 1 { 
target { bpf-*-* } }} } */
+/* { dg-final { scan-assembler-times "bar\[\t \]+\[^\n\]*bts_offset" 1 { 
target { bpf-*-* } } } } */
+/* { dg-final { scan-assembler-times "foo\[\t \]+\[^\n\]*bts_offset" 1 { 
target { bpf-*-* } } 

[PATCH] btf: improve btf-datasec-3.c test [PR 114642]

[David Faust]

This test failed on powerpc --target_board=unix'{-m32}' because two
variables were not placed in sections where the test silently (and
incorrectly) assumed they would be.

The important thing for the test is only that BTF_KIND_DATASEC entries
are NOT generated for the extern variable declarations without an
explicit section attribute.  Make the test more robust by placing the
non-extern variables in explicit sections, and invert the checks to
more accurately verify what we care about in this test.

Tested on x86_64-linux-gnu and x86_64-linux-gnu host for
powerpc64-linux-gnu and bpf-unkown-none targets.

OK?

gcc/testsuite/
PR testsuite/114642
* gcc.dg/debug/btf/btf-datasec-3.c: Make test more robust on different
architectures.
---
 gcc/testsuite/gcc.dg/debug/btf/btf-datasec-3.c | 16 
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-3.c 
b/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-3.c
index 297340cabfa..6b127aa14da 100644
--- a/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-3.c
+++ b/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-3.c
@@ -7,22 +7,22 @@
 
 extern int VERSION __attribute__((section (".version")));
 
-extern int test_bss1;
-extern int test_data1;
+extern int ext1;
+extern int ext2;
 
-int test_bss2;
-int test_data2 = 2;
+int var1 __attribute__((section (".sec_a")));
+int var2 __attribute__((section (".sec_b"))) = 2;
 
 int
 foo (void)
 {
-  test_bss2 = VERSION;
-  return test_bss1 + test_data1 + test_data2;
+  ext2 = VERSION;
+  return ext1 + var1 + var2;
 }
 
 /* There should be 3 DATASEC entries total.  Of the extern decls, only VERSION
has a known section; entries are not created for the other two.  */
 /* { dg-final { scan-assembler-times "bts_type" 3 } } */
-/* { dg-final { scan-assembler-times "bts_type: \\(BTF_KIND_VAR 
'test_data2'\\)" 1 } } */
-/* { dg-final { scan-assembler-times "bts_type: \\(BTF_KIND_VAR 
'test_bss2'\\)" 1 } } */
 /* { dg-final { scan-assembler-times "bts_type: \\(BTF_KIND_VAR 'VERSION'\\)" 
1 } } */
+/* { dg-final { scan-assembler-not "bts_type: \\(BTF_KIND_VAR 'ext1'\\)" } } */
+/* { dg-final { scan-assembler-not "bts_type: \\(BTF_KIND_VAR 'ext2'\\)" } } */
-- 
2.43.0

Re: [PATCH 1/2] ctf: fix PR debug/112878

[David Faust]

On 4/10/24 11:25, Indu Bhagat wrote:
> PR debug/112878: ICE: in ctf_add_slice, at ctfc.cc:499 with _BitInt > 255 in 
> a struct and -gctf1
> 
> The CTF generation in GCC does not have a mechanism to roll-back an
> already added type.  In this testcase presented in the PR, we hit a
> representation limit in CTF slices (for a member of a struct) and ICE,
> after the type for struct (CTF_K_STRUCT) has already been added to the
> container.
> 
> To exit gracefully instead, we now check for both the offset and size of
> the bitfield to be explicitly <= 255.  If the check fails, we emit the
> member with type CTF_K_UNKNOWN.  Note that, the value 255 stems from the
> existing binutils libctf checks which were motivated to guard against
> malformed inputs.
> 
> Although it is not accurate to say that this is a CTF representation
> limit, mark the code with TBD_CTF_REPRESENTATION_LIMIT for now so that
> this can be taken care of with the next format version bump, when
> libctf's checks for the slice data can be lifted as well.

OK.

> 
> gcc/ChangeLog:
>   PR debug/112878
>   * dwarf2ctf.cc (gen_ctf_sou_type): Check for conditions before
>   call to ctf_add_slice.  Use CTF_K_UNKNOWN type if fail.
> 
> gcc/testsuite/ChangeLog:
>   PR debug/112878
>   * gcc.dg/debug/ctf/ctf-bitfields-5.c: New test.
> ---
>  gcc/dwarf2ctf.cc| 15 ++-
>  .../gcc.dg/debug/ctf/ctf-bitfields-5.c  | 17 +
>  2 files changed, 27 insertions(+), 5 deletions(-)
>  create mode 100644 gcc/testsuite/gcc.dg/debug/ctf/ctf-bitfields-5.c
> 
> diff --git a/gcc/dwarf2ctf.cc b/gcc/dwarf2ctf.cc
> index 77d6bf89689..dc59569fe56 100644
> --- a/gcc/dwarf2ctf.cc
> +++ b/gcc/dwarf2ctf.cc
> @@ -606,11 +606,16 @@ gen_ctf_sou_type (ctf_container_ref ctfc, dw_die_ref 
> sou, uint32_t kind)
> if (attr)
>   bitpos += AT_unsigned (attr);
>  
> -   field_type_id = ctf_add_slice (ctfc, CTF_ADD_NONROOT,
> -  field_type_id,
> -  bitpos - field_location,
> -  bitsize,
> -  c);
> +   /* This is not precisely a TBD_CTF_REPRESENTATION_LIMIT, but
> +  surely something to look at for the next format version bump
> +  for CTF.  */
> +   if (bitsize <= 255 && (bitpos - field_location) <= 255)
> + field_type_id = ctf_add_slice (ctfc, CTF_ADD_NONROOT,
> +field_type_id,
> +bitpos - field_location,
> +bitsize, c);
> +   else
> + field_type_id = gen_ctf_unknown_type (ctfc);
>   }
>  
> /* Add the field type to the struct or union type.  */
> diff --git a/gcc/testsuite/gcc.dg/debug/ctf/ctf-bitfields-5.c 
> b/gcc/testsuite/gcc.dg/debug/ctf/ctf-bitfields-5.c
> new file mode 100644
> index 000..fee8228647c
> --- /dev/null
> +++ b/gcc/testsuite/gcc.dg/debug/ctf/ctf-bitfields-5.c
> @@ -0,0 +1,17 @@
> +/* Bitfield where the bit offset is > 255 is not allowed in CTF.
> +
> +   PR debug/112878.
> +   This testcase is to ensure graceful handling. No slices are expected.  */
> +
> +/* { dg-do compile { target bitint } } */
> +/* { dg-options "-O0 -gctf -dA" } */
> +
> +/* No slices are expected, but a struct with one member is expected.
> +   CTF_K_UNKNOWN is also expected.  */
> +/* { dg-final { scan-assembler-times "cts_type" 0 } } */
> +/* { dg-final { scan-assembler-times "\[\t \]0x1a01\[\t 
> \]+\[^\n\]*ctt_info" 1 } } */
> +/* { dg-final { scan-assembler-times "ascii \"unknown.0\"\[\t 
> \]+\[^\n\]*ctf_string" 1 } } */
> +
> +struct {
> +  _BitInt(282) a : 280;
> +} b;

Re: [PATCH 2/2] btf: do not skip members of data type with type id BTF_VOID_TYPEID

[David Faust]

Hi Indu,

On 4/10/24 11:25, Indu Bhagat wrote:
> Testing the previous fix in gen_ctf_sou_type () reveals an issue in BTF
> generation, however: BTF emission was currently decrementing the vlen
> (indicating the number of members) to skip members of type CTF_K_UNKNOWN
> altogether, but still emitting the BTF for the corresponding member (in
> output_asm_btf_sou_fields ()).
> 
> One can see malformed BTF by executing the newly added CTF testcase
> (gcc.dg/debug/ctf/ctf-bitfields-5.c) with -gbtf instead or even existing
> btf-struct-2.c without this patch.
> 
> To fix the issue, it makes sense to rather _not_ skip members of data
> type of type id BTF_VOID_TYPEID.

Thank you for catching and fixing this.

FWIW, what to do in such cases for a struct with a member that has no
representation is undefined behavior in BTF.  I certainly agree it's
better not to emit something malformed, and using 'void' is a good
choice.  Better to know there was a member there that could not be
represented than to skip it altogether, and the total struct size
shall still be correct.

OK.
Thanks!

> 
> gcc/ChangeLog:
>   * btfout.cc (btf_asm_type): Do not skip emitting members of
>   unknown type.
> 
> gcc/testsuite/ChangeLog:
>   * btf-bitfields-4.c: Update the vlen check.
>   * btf-struct-2.c: Check that member named 'f' with void data
>   type is emitted.
> ---
>  gcc/btfout.cc| 5 -
>  gcc/testsuite/gcc.dg/debug/btf/btf-bitfields-4.c | 6 +++---
>  gcc/testsuite/gcc.dg/debug/btf/btf-struct-2.c| 9 +
>  3 files changed, 8 insertions(+), 12 deletions(-)
> 
> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
> index 4a8ec4d1ff0..ab491f0297f 100644
> --- a/gcc/btfout.cc
> +++ b/gcc/btfout.cc
> @@ -820,11 +820,6 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd)
> /* Set kflag if this member is a representable bitfield.  */
> if (btf_dmd_representable_bitfield_p (ctfc, dmd))
>   btf_kflag = 1;
> -
> -   /* Struct members that refer to unsupported types or bitfield formats
> -  shall be skipped. These are marked during preprocessing.  */
> -   else if (!btf_emit_id_p (dmd->dmd_type))
> - btf_vlen -= 1;
>   }
>  }
>  
> diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-bitfields-4.c 
> b/gcc/testsuite/gcc.dg/debug/btf/btf-bitfields-4.c
> index c00c8b3d87f..d4a6ef6a1eb 100644
> --- a/gcc/testsuite/gcc.dg/debug/btf/btf-bitfields-4.c
> +++ b/gcc/testsuite/gcc.dg/debug/btf/btf-bitfields-4.c
> @@ -6,14 +6,14 @@
> In this test, we construct a structure such that the bitfield will have
> an offset so large as to be unrepresentable in BTF. We expect that the
> resulting BTF will describe the rest of the structure, ignoring the
> -   non-representable bitfield.  */
> +   non-representable bitfield by simply using void data type for the same.  
> */
>  
>  /* { dg-do compile } */
>  /* { dg-options "-O0 -gbtf -dA" } */
>  /* { dg-require-effective-target size32plus } */
>  
> -/* Struct with 3 members and no bitfield (kind_flag not set).  */
> -/* { dg-final { scan-assembler-times "\[\t \]0x403\[\t 
> \]+\[^\n\]*btt_info" 1 } } */
> +/* Struct with 4 members and no bitfield (kind_flag not set).  */
> +/* { dg-final { scan-assembler-times "\[\t \]0x404\[\t 
> \]+\[^\n\]*btt_info" 1 } } */
>  
>  struct bigly
>  {
> diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-struct-2.c 
> b/gcc/testsuite/gcc.dg/debug/btf/btf-struct-2.c
> index e9ff06883db..fa7231be75c 100644
> --- a/gcc/testsuite/gcc.dg/debug/btf/btf-struct-2.c
> +++ b/gcc/testsuite/gcc.dg/debug/btf/btf-struct-2.c
> @@ -2,14 +2,15 @@
> unsupported type.
>  
> BTF does not support vector types (among other things). When
> -   generating BTF for a struct (or union) type, members which refer to
> -   unsupported types should be skipped.  */
> +   generating BTF for a struct (or union) type.  Members which refer to
> +   unsupported types should not be skipped, however.  */
>  
>  /* { dg-do compile } */
>  /* { dg-options "-O0 -gbtf -dA" } */
>  
> -/* Expect a struct with only 2 members - 'f' should not be present.  */
> -/* { dg-final { scan-assembler-times "\[\t \]0x402\[\t 
> \]+\[^\n\]*btt_info" 1 } } */
> +/* Expect a struct with 3 members - 'f' is present but is of data type void. 
>  */
> +/* { dg-final { scan-assembler-times "\[\t \]0x403\[\t 
> \]+\[^\n\]*btt_info" 1 } } */
> +/* { dg-final { scan-assembler-times " MEMBER 'f' 
> idx=1\[\\r\\n\]+\[^\\r\\n\]*0\[\t \]+\[^\n\]*btm_type: void" 1 } } */
>  
>  struct with_float
>  {

Re: [PATCH 2/3] bpf: remove huge memory waste with string allocation.

[David Faust]

Hi Cupertino,

On 4/11/24 04:11, Cupertino Miranda wrote:
> Code was allocating way too much space for the string.

A little bit more description would not hurt. Perhaps you could say
something like:
"The BPF backend was allocating an unnecessarily large string when
 constructing CO-RE relocations for enum types."

> 
> gcc/ChangeLog:
>   * config/bpf/core-builtins.cc (process_enum_value): Corrected
>   string allocation.

nit: present tense, i.e. "Correct" rather than "Corrected"

> ---
>  gcc/config/bpf/core-builtins.cc | 7 ---
>  1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/gcc/config/bpf/core-builtins.cc b/gcc/config/bpf/core-builtins.cc
> index e03e986e2c1..ead1777d465 100644
> --- a/gcc/config/bpf/core-builtins.cc
> +++ b/gcc/config/bpf/core-builtins.cc
> @@ -872,10 +872,11 @@ process_enum_value (struct cr_builtins *data)
>   {
> if (TREE_VALUE (l) == expr)
>   {
> -   char *tmp = (char *) ggc_alloc_atomic ((index / 10) + 1);
> +   /* Array size is 21 = ceil(log_10(2^64)) + 1 to hold string
> +  representations of 64 bit integers.  */
> +   char tmp[21];
> sprintf (tmp, "%d", index);

It looks like `index' is an `unsigned int', so this sprintf should use
%u rather %d, no?

Also, it occurs to me that the `vlen' of BTF types is only 16 bits, so
BTF has no way currently to represent enums with more than 65535
enumerators. It may be worth adding a sanity check to bail out (error)
here if we're going to claim an index higher than that. And if that is
validated before the printf, the buffer can be 6 bytes ("65535\0").

> -   ret.str = (const char *) tmp;
> -
> +   ret.str = CONST_CAST (char *, ggc_strdup(tmp));
> break;
>   }
> index++;

[PATCH] btf: emit non-representable bitfield as void

[David Faust]

This patch fixes an issue with mangled BTF that could occur when
a struct type contains a bitfield member which cannot be represented
in BTF.  It is undefined what should happen in such cases, but we can
at least do something reasonable.

Commit

  936dd627cd9 "btf: do not skip members of data type with type id
  BTF_VOID_TYPEID"

made a similar change for un-representable non-bitfield members, but
had an unintended side-effect of mangling BTF for un-representable
bitfields: the struct (or union) would account for the offending
bitfield in its member count but the bitfield member itself was
not emitted, making the member count incorrect.

This change ensures that non-representable bitfield members of struct
and union types are always emitted with BTF_VOID_TYPEID.  This avoids
corrupting the BTF information for the entire struct or union type.

Tested on x86_64-linux-gnu and x86_64-linux-gnu host for
bpf-unknown-none target.

gcc/
* btfout.cc (btf_asm_sou_member): Always emit non-representable
bitfield members as having 'void' type.  Refactor slightly.

gcc/testsuite/
* gcc.dg/debug/btf/btf-bitfields-4.c: Add two new checks.
---
 gcc/btfout.cc | 48 +--
 .../gcc.dg/debug/btf/btf-bitfields-4.c|  2 +
 2 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index ab491f0297f..e1ada41b1f4 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -922,41 +922,39 @@ static void
 btf_asm_sou_member (ctf_container_ref ctfc, ctf_dmdef_t * dmd, unsigned int 
idx)
 {
   ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[dmd->dmd_type];
+  ctf_id_t base_type = get_btf_id (dmd->dmd_type);
+  uint64_t sou_offset = dmd->dmd_offset;
+
+  dw2_asm_output_data (4, dmd->dmd_name_offset,
+  "MEMBER '%s' idx=%u",
+  dmd->dmd_name, idx);
 
   /* Re-encode bitfields to BTF representation.  */
   if (CTF_V2_INFO_KIND (ref_type->dtd_data.ctti_info) == CTF_K_SLICE)
 {
-  ctf_id_t base_type = ref_type->dtd_u.dtu_slice.cts_type;
   unsigned short word_offset = ref_type->dtd_u.dtu_slice.cts_offset;
   unsigned short bits = ref_type->dtd_u.dtu_slice.cts_bits;
-  uint64_t sou_offset = dmd->dmd_offset;
-
-  /* Pack the bit offset and bitfield size together.  */
-  sou_offset += word_offset;
 
-  /* If this bitfield cannot be represented, do not output anything.
-The parent struct/union 'vlen' field has already been updated.  */
   if ((bits > 0xff) || (sou_offset > 0xff))
-   return;
-
-  sou_offset &= 0x00ff;
-  sou_offset |= ((bits & 0xff) << 24);
+   {
+ /* Bitfield cannot be represented in BTF.  Emit the member as having
+'void' type.  */
+ base_type = BTF_VOID_TYPEID;
+   }
+  else
+   {
+ /* Pack the bit offset and bitfield size together.  */
+ sou_offset += word_offset;
+ sou_offset &= 0x00ff;
+ sou_offset |= ((bits & 0xff) << 24);
 
-  dw2_asm_output_data (4, dmd->dmd_name_offset,
-  "MEMBER '%s' idx=%u",
-  dmd->dmd_name, idx);
-  /* Refer to the base type of the slice.  */
-  btf_asm_type_ref ("btm_type", ctfc, get_btf_id (base_type));
-  dw2_asm_output_data (4, sou_offset, "btm_offset");
-}
-  else
-{
-  dw2_asm_output_data (4, dmd->dmd_name_offset,
-  "MEMBER '%s' idx=%u",
-  dmd->dmd_name, idx);
-  btf_asm_type_ref ("btm_type", ctfc, get_btf_id (dmd->dmd_type));
-  dw2_asm_output_data (4, dmd->dmd_offset, "btm_offset");
+ /* Refer to the base type of the slice.  */
+ base_type = get_btf_id (ref_type->dtd_u.dtu_slice.cts_type);
+   }
 }
+
+  btf_asm_type_ref ("btm_type", ctfc, base_type);
+  dw2_asm_output_data (4, sou_offset, "btm_offset");
 }
 
 /* Asm'out an enum constant following a BTF_KIND_ENUM{,64}.  */
diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-bitfields-4.c 
b/gcc/testsuite/gcc.dg/debug/btf/btf-bitfields-4.c
index d4a6ef6a1eb..20cdfaa057a 100644
--- a/gcc/testsuite/gcc.dg/debug/btf/btf-bitfields-4.c
+++ b/gcc/testsuite/gcc.dg/debug/btf/btf-bitfields-4.c
@@ -14,6 +14,8 @@
 
 /* Struct with 4 members and no bitfield (kind_flag not set).  */
 /* { dg-final { scan-assembler-times "\[\t \]0x404\[\t 
\]+\[^\n\]*btt_info" 1 } } */
+/* { dg-final { scan-assembler-times " MEMBER" 4 } } */
+/* { dg-final { scan-assembler-times " MEMBER 'unsup' 
idx=2\[\\r\\n\]+\[^\\r\\n\]*0\[\t \]+\[^\n\]*btm_type: void" 1 } } */
 
 struct bigly
 {
-- 
2.43.0

Re: [PATCH] btf: emit non-representable bitfield as void

[David Faust]

On 4/11/24 13:40, Indu Bhagat wrote:
> On 4/11/24 11:53, David Faust wrote:
>> This patch fixes an issue with mangled BTF that could occur when
>> a struct type contains a bitfield member which cannot be represented
>> in BTF.  It is undefined what should happen in such cases, but we can
>> at least do something reasonable.
>>
>> Commit
>>
>>936dd627cd9 "btf: do not skip members of data type with type id
>>BTF_VOID_TYPEID"
>>
>> made a similar change for un-representable non-bitfield members, but
>> had an unintended side-effect of mangling BTF for un-representable
>> bitfields: the struct (or union) would account for the offending
>> bitfield in its member count but the bitfield member itself was
>> not emitted, making the member count incorrect.
>>
>> This change ensures that non-representable bitfield members of struct
>> and union types are always emitted with BTF_VOID_TYPEID.  This avoids
>> corrupting the BTF information for the entire struct or union type.
>>
>> Tested on x86_64-linux-gnu and x86_64-linux-gnu host for
>> bpf-unknown-none target.
>>
> 
> Hi David,
> 
> Thanks for fixing this.  One comment below.
> 
>> gcc/
>>  * btfout.cc (btf_asm_sou_member): Always emit non-representable
>>  bitfield members as having 'void' type.  Refactor slightly.
>>
>> gcc/testsuite/
>>  * gcc.dg/debug/btf/btf-bitfields-4.c: Add two new checks.
>> ---
>>   gcc/btfout.cc | 48 +--
>>   .../gcc.dg/debug/btf/btf-bitfields-4.c|  2 +
>>   2 files changed, 25 insertions(+), 25 deletions(-)
>>
>> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
>> index ab491f0297f..e1ada41b1f4 100644
>> --- a/gcc/btfout.cc
>> +++ b/gcc/btfout.cc
>> @@ -922,41 +922,39 @@ static void
>>   btf_asm_sou_member (ctf_container_ref ctfc, ctf_dmdef_t * dmd, unsigned 
>> int idx)
>>   {
>> ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[dmd->dmd_type];
>> +  ctf_id_t base_type = get_btf_id (dmd->dmd_type);
>> +  uint64_t sou_offset = dmd->dmd_offset;
>> +
>> +  dw2_asm_output_data (4, dmd->dmd_name_offset,
>> +   "MEMBER '%s' idx=%u",
>> +   dmd->dmd_name, idx);
>>   
>> /* Re-encode bitfields to BTF representation.  */
>> if (CTF_V2_INFO_KIND (ref_type->dtd_data.ctti_info) == CTF_K_SLICE)
>>   {
>> -  ctf_id_t base_type = ref_type->dtd_u.dtu_slice.cts_type;
>> unsigned short word_offset = ref_type->dtd_u.dtu_slice.cts_offset;
>> unsigned short bits = ref_type->dtd_u.dtu_slice.cts_bits;
>> -  uint64_t sou_offset = dmd->dmd_offset;
>> -
>> -  /* Pack the bit offset and bitfield size together.  */
>> -  sou_offset += word_offset;
>>   
>> -  /* If this bitfield cannot be represented, do not output anything.
>> - The parent struct/union 'vlen' field has already been updated.  */
>> if ((bits > 0xff) || (sou_offset > 0xff))
> 
> Why dont we reuse the btf_dmd_representable_bitfield_p () function here?
> 
> This one here checks for sou_off > 0xff, while that in 
> btf_dmd_representable_bitfield_p checks for sou_offset + word_offset > 
> 0xff.  The latter is more precise.

Oops. Good catch. We should be doing the same check here, but I moved
the sou_offset += word_offset into the else below, so the word_offset
isn't included.

I avoided using btf_dmd_representable_bitfield_p only because it does
some redundant work. Clearly that was a bad idea :D

Will use btf_dmd_representable_bitfield_p in v2 so we can keep these
checks contained.

> 
> 
>> -return;
>> -
>> -  sou_offset &= 0x00ff;
>> -  sou_offset |= ((bits & 0xff) << 24);
>> +{
>> +  /* Bitfield cannot be represented in BTF.  Emit the member as having
>> + 'void' type.  */
>> +  base_type = BTF_VOID_TYPEID;
>> +}
>> +  else
>> +{
>> +  /* Pack the bit offset and bitfield size together.  */
>> +  sou_offset += word_offset;
>> +  sou_offset &= 0x00ff;
>> +  sou_offset |= ((bits & 0xff) << 24);
>>   
>> -  dw2_asm_output_data (4, dmd->dmd_name_offset,
>> -   "MEMBER '%s' idx=%u",
>> -   dmd->dmd_name, idx);
>> -  /* Refer to the base type of the slice.  */
>> -  btf_asm_type_ref ("btm_type"

[PATCH v2] btf: emit non-representable bitfield as void

[David Faust]

[Changes from v1: use btf_dmd_representable_bitfield_p in
 btf_asm_sou_member, instead of a slightly incorrect equivalent check.]

This patch fixes an issue with mangled BTF that could occur when
a struct type contains a bitfield member which cannot be represented
in BTF.  It is undefined what should happen in such cases, but we can
at least do something reasonable.

Commit

  936dd627cd9 "btf: do not skip members of data type with type id
  BTF_VOID_TYPEID"

made a similar change for un-representable non-bitfield members, but
had an unintended side-effect of mangling BTF for un-representable
bitfields: the struct (or union) would account for the offending
bitfield in its member count but the bitfield member itself was
not emitted, making the member count incorrect.

This change ensures that non-representable bitfield members of struct
and union types are always emitted with BTF_VOID_TYPEID.  This avoids
corrupting the BTF information for the entire struct or union type.

Tested on x86_64-linux-gnu and x86_64-linux-gnu host for
bpf-unknown-none target.

gcc/
* btfout.cc (btf_asm_sou_member): Always emit non-representable
bitfield members as having 'void' type.  Refactor slightly.

gcc/testsuite/
* gcc.dg/debug/btf/btf-bitfields-4.c: Add two new checks.
---
 gcc/btfout.cc | 54 +--
 .../gcc.dg/debug/btf/btf-bitfields-4.c|  2 +
 2 files changed, 28 insertions(+), 28 deletions(-)

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index ab491f0297f..a1510574a93 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -922,41 +922,39 @@ static void
 btf_asm_sou_member (ctf_container_ref ctfc, ctf_dmdef_t * dmd, unsigned int 
idx)
 {
   ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[dmd->dmd_type];
+  ctf_id_t base_type = get_btf_id (dmd->dmd_type);
+  uint64_t sou_offset = dmd->dmd_offset;
+
+  dw2_asm_output_data (4, dmd->dmd_name_offset,
+  "MEMBER '%s' idx=%u",
+  dmd->dmd_name, idx);
 
   /* Re-encode bitfields to BTF representation.  */
   if (CTF_V2_INFO_KIND (ref_type->dtd_data.ctti_info) == CTF_K_SLICE)
 {
-  ctf_id_t base_type = ref_type->dtd_u.dtu_slice.cts_type;
-  unsigned short word_offset = ref_type->dtd_u.dtu_slice.cts_offset;
-  unsigned short bits = ref_type->dtd_u.dtu_slice.cts_bits;
-  uint64_t sou_offset = dmd->dmd_offset;
-
-  /* Pack the bit offset and bitfield size together.  */
-  sou_offset += word_offset;
-
-  /* If this bitfield cannot be represented, do not output anything.
-The parent struct/union 'vlen' field has already been updated.  */
-  if ((bits > 0xff) || (sou_offset > 0xff))
-   return;
+  if (btf_dmd_representable_bitfield_p (ctfc, dmd))
+   {
+ unsigned short word_offset = ref_type->dtd_u.dtu_slice.cts_offset;
+ unsigned short bits = ref_type->dtd_u.dtu_slice.cts_bits;
 
-  sou_offset &= 0x00ff;
-  sou_offset |= ((bits & 0xff) << 24);
+ /* Pack the bit offset and bitfield size together.  */
+ sou_offset += word_offset;
+ sou_offset &= 0x00ff;
+ sou_offset |= ((bits & 0xff) << 24);
 
-  dw2_asm_output_data (4, dmd->dmd_name_offset,
-  "MEMBER '%s' idx=%u",
-  dmd->dmd_name, idx);
-  /* Refer to the base type of the slice.  */
-  btf_asm_type_ref ("btm_type", ctfc, get_btf_id (base_type));
-  dw2_asm_output_data (4, sou_offset, "btm_offset");
-}
-  else
-{
-  dw2_asm_output_data (4, dmd->dmd_name_offset,
-  "MEMBER '%s' idx=%u",
-  dmd->dmd_name, idx);
-  btf_asm_type_ref ("btm_type", ctfc, get_btf_id (dmd->dmd_type));
-  dw2_asm_output_data (4, dmd->dmd_offset, "btm_offset");
+ /* Refer to the base type of the slice.  */
+ base_type = get_btf_id (ref_type->dtd_u.dtu_slice.cts_type);
+   }
+  else
+   {
+ /* Bitfield cannot be represented in BTF.  Emit the member as having
+'void' type.  */
+ base_type = BTF_VOID_TYPEID;
+   }
 }
+
+  btf_asm_type_ref ("btm_type", ctfc, base_type);
+  dw2_asm_output_data (4, sou_offset, "btm_offset");
 }
 
 /* Asm'out an enum constant following a BTF_KIND_ENUM{,64}.  */
diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-bitfields-4.c 
b/gcc/testsuite/gcc.dg/debug/btf/btf-bitfields-4.c
index d4a6ef6a1eb..20cdfaa057a 100644
--- a/gcc/testsuite/gcc.dg/debug/btf/btf-bitfields-4.c
+++ b/gcc/testsuite/gcc.dg/debug/btf/btf-bitfields-4.c
@@ -14,6 +14,8 @@
 
 /* Struct with 4 members and no bitfield (kind_flag not set).  */
 /* { dg-final { scan-assembler-times "\[\t \]0x404\[\t 
\]+\[^\n\]*btt_info" 1 } } */
+/* { dg-final { scan-assembler-times " MEMBER" 4 } } */
+/* { dg-final { scan-assembler-times " MEMBER 'unsup' 
idx=2\[\\r\\n\]+\[^\\r\\n\]*0\[\t \]+\[^\n\]*btm_type: void" 1 } } */
 
 struct bigly
 {
-- 
2.43.0

[PATCH] btf: fix a possibly misleading asm debug comment

[David Faust]

This patch fixes a small error that could occur in the debug comment
when emitting a type reference with btf_asm_type_ref.

While working on a previous patch, I noticed the following in the asm
output for the test btf-bitfields-4.c:

...
.long   0x39# MEMBER 'c' idx=3
.long   0x6 # btm_type: (BTF_KIND_UNKN '')
...
.long   0x34# TYPE 6 BTF_KIND_INT 'char'

The type for member 'c' is correct, but the comment for the member
incorrectly reads "BTF_KIND_UNKN ''".  This was caused by an
incorrect type lookup in btf_asm_type_ref that could happen if the
source file has types which can be represented in CTF but not in BTF.

This patch fixes the issue by changing btf_asm_type_ref to work fully
in the CTF ID space until writing out the final BTF ID.  That ensures
types are correctly identified when writing the asm debug comments,
like the following fixed comment for the above case.

...
.long   0x39# MEMBER 'c' idx=3
.long   0x6 # btm_type: (BTF_KIND_INT 'char')
...

Note that there was no problem with the actual BTF information, the
only error was in the comment.  This patch does not change the output
BTF information, and no tests were affected.

Tested on x86_64-linux-gnu and x86_64-linux-gnu host for
bpf-unknown-none target. Sanity checked by compiling kernel BPF
selftests.

gcc/
* btfout.cc (btf_asm_type_ref): Convert IDs to BTF internally and
fix potentially looking up wrong type for asm debug comment info.
Split into...
(btf_asm_datasec_type_ref): ... This. New.
(btf_asm_datasec_entry): Call it here, instead of btf_asm_type_ref.
(btf_asm_type, btf_asm_array, btf_asm_varent, btf_asm_sou_member)
(btf_asm_func_arg, btf_asm_func_type): Adapt btf_asm_type_ref call.
---
 gcc/btfout.cc | 84 ++-
 1 file changed, 50 insertions(+), 34 deletions(-)

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index e1ada41b1f4..24bef3acfd1 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -738,36 +738,22 @@ btf_dmd_representable_bitfield_p (ctf_container_ref ctfc, 
ctf_dmdef_t *dmd)
 /* Asm'out a reference to another BTF type.  */
 
 static void
-btf_asm_type_ref (const char *prefix, ctf_container_ref ctfc, ctf_id_t ref_id)
+btf_asm_type_ref (const char *prefix, ctf_container_ref ctfc, ctf_id_t ctf_id)
 {
-  if (ref_id == BTF_VOID_TYPEID || ref_id == BTF_INVALID_TYPEID)
+  ctf_id_t btf_id = get_btf_id (ctf_id);
+  if (btf_id == BTF_VOID_TYPEID || btf_id == BTF_INVALID_TYPEID)
 {
   /* There is no explicit void type.
 Also handle any invalid refs that made it this far, just in case.  */
-  dw2_asm_output_data (4, ref_id, "%s: void", prefix);
-}
-  else if (ref_id >= num_types_added + 1
-  && ref_id < num_types_added + num_vars_added + 1)
-{
-  /* Ref to a variable.  Should only appear in DATASEC entries.  */
-  ctf_id_t var_id = btf_relative_var_id (ref_id);
-  ctf_dvdef_ref dvd = ctfc->ctfc_vars_list[var_id];
-  dw2_asm_output_data (4, ref_id, "%s: (BTF_KIND_VAR '%s')",
-  prefix, dvd->dvd_name);
-
-}
-  else if (ref_id >= num_types_added + num_vars_added + 1)
-{
-  /* Ref to a FUNC record.  */
-  size_t func_id = btf_relative_func_id (ref_id);
-  ctf_dtdef_ref ref_type = (*funcs)[func_id];
-  dw2_asm_output_data (4, ref_id, "%s: (BTF_KIND_FUNC '%s')",
-  prefix, get_btf_type_name (ref_type));
+  dw2_asm_output_data (4, btf_id, "%s: void", prefix);
 }
   else
 {
-  /* Ref to a standard type in the types list.  */
-  ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[ref_id];
+  gcc_assert (btf_id <= num_types_added);
+
+  /* Ref to a standard type in the types list.  Note: take care that we
+must index the type list by the original CTF id, not the BTF id.  */
+  ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[ctf_id];
   uint32_t ref_kind
= get_btf_kind (CTF_V2_INFO_KIND (ref_type->dtd_data.ctti_info));
 
@@ -775,12 +761,43 @@ btf_asm_type_ref (const char *prefix, ctf_container_ref 
ctfc, ctf_id_t ref_id)
? btf_kind_name (BTF_KIND_ENUM)
: btf_kind_name (ref_kind);
 
-  dw2_asm_output_data (4, ref_id, "%s: (BTF_KIND_%s '%s')",
+  dw2_asm_output_data (4, btf_id, "%s: (BTF_KIND_%s '%s')",
   prefix, kind_name,
   get_btf_type_name (ref_type));
 }
 }
 
+/* Asm'out a reference to a BTF_KIND_VAR or BTF_KIND_FUNC type.  These type
+   kinds are BTF-specific, and should only be referred to by entries in
+   BTF_KIND_DATASEC records.  */
+
+static void
+btf_asm_datasec_type_ref (const char *prefix, ctf_container_ref ctfc,
+ ctf_id_t btf_id)
+{
+  if (btf_id >= num_types_added + 1
+  && btf_id < num_types_added + num_vars_added + 1)
+{
+  /* Ref to a variable.  Should only appea

[PATCH] bpf: fix zero_extendqidi2 ldx template

[David Faust]

Commit 77d0f9ec3809b4d2e32c36069b6b9239d301c030 inadvertently changed
the normal asm dialect instruction template for zero_extendqidi2 from
ldxb to ldxh. Fix that.

Tested for bpf-unknown-none on x86_64-linux-gnu host.

gcc/

* config/bpf/bpf.md (zero_extendqidi2): Correct asm template to
use ldxb instead of ldxh.
---
 gcc/config/bpf/bpf.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gcc/config/bpf/bpf.md b/gcc/config/bpf/bpf.md
index 080a63cd970..50df1aaa3e2 100644
--- a/gcc/config/bpf/bpf.md
+++ b/gcc/config/bpf/bpf.md
@@ -292,7 +292,7 @@ (define_insn "zero_extendqidi2"
   "@
{and\t%0,0xff|%0 &= 0xff}
{mov\t%0,%1\;and\t%0,0xff|%0 = %1;%0 &= 0xff}
-   {ldxh\t%0,%1|%0 = *(u8 *) (%1)}"
+   {ldxb\t%0,%1|%0 = *(u8 *) (%1)}"
   [(set_attr "type" "alu,alu,ldx")])
 
 (define_insn "zero_extendsidi2"
-- 
2.43.0

[PATCH] bpf: add inline memmove and memcpy expansion

[David Faust]

BPF programs are not typically linked, which means we cannot fall back
on library calls to implement __builtin_{memmove,memcpy} and should
always expand them inline if possible.

GCC already successfully expands these builtins inline in many cases,
but failed to do so for a few for simple cases involving overlapping
memmove in the kernel BPF selftests and was instead emitting a libcall.

This patch implments a simple inline expansion of memcpy and memmove in
the BPF backend in a verifier-friendly way, with the caveat that the
size must be an integer constant, which is also required by clang.

Tested for bpf-unknown-none on x86_64-linux-gnu host.

Also tested against the BPF verifier by compiling and loading a test
program with overlapping memmove (essentially the memmove-1.c test)
which failed before due to a libcall, and now successfully loads and
passes the verifier.

gcc/

* config/bpf/bpf-protos.h (bpf_expand_cpymem): New.
* config/bpf/bpf.cc: (emit_move_loop, CPYMEM_EXPAND_ERR)
(bpf_expand_cpymem): New.
* config/bpf/bpf.md: (cpymemdi, movmemdi): New define_expands.

gcc/testsuite/

* gcc.target/bpf/memcpy-1.c: New test.
* gcc.target/bpf/memmove-1.c: New test.
* gcc.target/bpf/memmove-2.c: New test.
---
 gcc/config/bpf/bpf-protos.h  |   2 +
 gcc/config/bpf/bpf.cc| 129 +++
 gcc/config/bpf/bpf.md|  36 +++
 gcc/testsuite/gcc.target/bpf/memcpy-1.c  |  26 +
 gcc/testsuite/gcc.target/bpf/memmove-1.c |  46 
 gcc/testsuite/gcc.target/bpf/memmove-2.c |  23 
 6 files changed, 262 insertions(+)
 create mode 100644 gcc/testsuite/gcc.target/bpf/memcpy-1.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/memmove-1.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/memmove-2.c

diff --git a/gcc/config/bpf/bpf-protos.h b/gcc/config/bpf/bpf-protos.h
index 46d950bd990..366acb87ae4 100644
--- a/gcc/config/bpf/bpf-protos.h
+++ b/gcc/config/bpf/bpf-protos.h
@@ -35,4 +35,6 @@ const char *bpf_add_core_reloc (rtx *operands, const char 
*templ);
 class gimple_opt_pass;
 gimple_opt_pass *make_pass_lower_bpf_core (gcc::context *ctxt);
 
+bool bpf_expand_cpymem (rtx *, bool);
+
 #endif /* ! GCC_BPF_PROTOS_H */
diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc
index d6ca47eeecb..c90d29d12ff 100644
--- a/gcc/config/bpf/bpf.cc
+++ b/gcc/config/bpf/bpf.cc
@@ -1184,6 +1184,135 @@ bpf_use_by_pieces_infrastructure_p (unsigned 
HOST_WIDE_INT size,
 #define TARGET_USE_BY_PIECES_INFRASTRUCTURE_P \
   bpf_use_by_pieces_infrastructure_p
 
+/* Helper for bpf_expand_cpymem.  Emit an unrolled loop moving the bytes
+   from SRC to DST.  */
+
+static void
+emit_move_loop (rtx src, rtx dst, machine_mode mode, int offset, int inc,
+   unsigned iters, unsigned remainder)
+{
+  rtx reg = gen_reg_rtx (mode);
+
+  /* First copy in chunks as large as alignment permits.  */
+  for (unsigned int i = 0; i < iters; i++)
+{
+  emit_move_insn (reg, adjust_address (src, mode, offset));
+  emit_move_insn (adjust_address (dst, mode, offset), reg);
+  offset += inc;
+}
+
+  /* Handle remaining bytes which might be smaller than the chunks
+ used above.  */
+  if (remainder & 4)
+{
+  emit_move_insn (reg, adjust_address (src, SImode, offset));
+  emit_move_insn (adjust_address (dst, SImode, offset), reg);
+  offset += (inc < 0 ? -4 : 4);
+  remainder -= 4;
+}
+  if (remainder & 2)
+{
+  emit_move_insn (reg, adjust_address (src, HImode, offset));
+  emit_move_insn (adjust_address (dst, HImode, offset), reg);
+  offset += (inc < 0 ? -2 : 2);
+  remainder -= 2;
+}
+  if (remainder & 1)
+{
+  emit_move_insn (reg, adjust_address (src, QImode, offset));
+  emit_move_insn (adjust_address (dst, QImode, offset), reg);
+}
+}
+
+/* Error if we cannot completely expand the memcpy/memmove inline,
+   unless we are building libgcc.  */
+#define CPYMEM_EXPAND_ERR(LOC, MSG)\
+  do   \
+{  \
+  if (flag_building_libgcc)
\
+   warning (LOC, MSG " for BPF memcpy/memmove expansion; " \
+"a libcall will be emitted");  \
+  else \
+   error (MSG " for BPF memcpy/memmove expansion");\
+} while (0);
+
+/* Expand cpymem/movmem, as from __builtin_memcpy/memmove.
+   OPERANDS are the same as the cpymem/movmem patterns.
+   IS_MOVE is true if this is a memmove, false for memcpy.
+   Return true if we successfully expanded, or false if we cannot
+   and must punt to a libcall.  */
+
+bool
+bpf_expand_cpymem (rtx *operands, bool is_move)
+{
+  /* Size and alignment must be constant.  */
+ 

[PATCH v2] bpf: add inline memmove and memcpy expansion

[David Faust]

[Changes from v1: Jose's review comments, all of which I agree with.
 - Fix 'implments' typo in commit message.
 - Change check that alignment is CONST_INT to gcc_assert ().
 - Change default case in alignment switch to gcc_unreachable ().
 - Reword error message for non-constant size memmove/memcpy, and
   update test for the error accordingly.
 - Delete CPYMEM_EXPAND_ERR macro, since it was now only used in
   one place.  ]

BPF programs are not typically linked, which means we cannot fall back
on library calls to implement __builtin_{memmove,memcpy} and should
always expand them inline if possible.

GCC already successfully expands these builtins inline in many cases,
but failed to do so for a few for simple cases involving overlapping
memmove in the kernel BPF selftests and was instead emitting a libcall.

This patch implements a simple inline expansion of memcpy and memmove in
the BPF backend in a verifier-friendly way, with the caveat that the
size must be an integer constant, which is also required by clang.

Tested for bpf-unknown-none on x86_64-linux-gnu host.

Also tested against the BPF verifier by compiling and loading a test
program with overlapping memmove (essentially the memmove-1.c test)
which failed before due to a libcall, and now successfully loads and
passes the verifier.

gcc/

* config/bpf/bpf-protos.h (bpf_expand_cpymem): New.
* config/bpf/bpf.cc: (emit_move_loop, bpf_expand_cpymem): New.
* config/bpf/bpf.md: (cpymemdi, movmemdi): New define_expands.

gcc/testsuite/

* gcc.target/bpf/memcpy-1.c: New test.
* gcc.target/bpf/memmove-1.c: New test.
* gcc.target/bpf/memmove-2.c: New test.
---
 gcc/config/bpf/bpf-protos.h  |   2 +
 gcc/config/bpf/bpf.cc| 116 +++
 gcc/config/bpf/bpf.md|  36 +++
 gcc/testsuite/gcc.target/bpf/memcpy-1.c  |  26 +
 gcc/testsuite/gcc.target/bpf/memmove-1.c |  46 +
 gcc/testsuite/gcc.target/bpf/memmove-2.c |  23 +
 6 files changed, 249 insertions(+)
 create mode 100644 gcc/testsuite/gcc.target/bpf/memcpy-1.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/memmove-1.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/memmove-2.c

diff --git a/gcc/config/bpf/bpf-protos.h b/gcc/config/bpf/bpf-protos.h
index 46d950bd990..366acb87ae4 100644
--- a/gcc/config/bpf/bpf-protos.h
+++ b/gcc/config/bpf/bpf-protos.h
@@ -35,4 +35,6 @@ const char *bpf_add_core_reloc (rtx *operands, const char 
*templ);
 class gimple_opt_pass;
 gimple_opt_pass *make_pass_lower_bpf_core (gcc::context *ctxt);
 
+bool bpf_expand_cpymem (rtx *, bool);
+
 #endif /* ! GCC_BPF_PROTOS_H */
diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc
index d6ca47eeecb..5db031a4551 100644
--- a/gcc/config/bpf/bpf.cc
+++ b/gcc/config/bpf/bpf.cc
@@ -1184,6 +1184,122 @@ bpf_use_by_pieces_infrastructure_p (unsigned 
HOST_WIDE_INT size,
 #define TARGET_USE_BY_PIECES_INFRASTRUCTURE_P \
   bpf_use_by_pieces_infrastructure_p
 
+/* Helper for bpf_expand_cpymem.  Emit an unrolled loop moving the bytes
+   from SRC to DST.  */
+
+static void
+emit_move_loop (rtx src, rtx dst, machine_mode mode, int offset, int inc,
+   unsigned iters, unsigned remainder)
+{
+  rtx reg = gen_reg_rtx (mode);
+
+  /* First copy in chunks as large as alignment permits.  */
+  for (unsigned int i = 0; i < iters; i++)
+{
+  emit_move_insn (reg, adjust_address (src, mode, offset));
+  emit_move_insn (adjust_address (dst, mode, offset), reg);
+  offset += inc;
+}
+
+  /* Handle remaining bytes which might be smaller than the chunks
+ used above.  */
+  if (remainder & 4)
+{
+  emit_move_insn (reg, adjust_address (src, SImode, offset));
+  emit_move_insn (adjust_address (dst, SImode, offset), reg);
+  offset += (inc < 0 ? -4 : 4);
+  remainder -= 4;
+}
+  if (remainder & 2)
+{
+  emit_move_insn (reg, adjust_address (src, HImode, offset));
+  emit_move_insn (adjust_address (dst, HImode, offset), reg);
+  offset += (inc < 0 ? -2 : 2);
+  remainder -= 2;
+}
+  if (remainder & 1)
+{
+  emit_move_insn (reg, adjust_address (src, QImode, offset));
+  emit_move_insn (adjust_address (dst, QImode, offset), reg);
+}
+}
+
+/* Expand cpymem/movmem, as from __builtin_memcpy/memmove.
+   OPERANDS are the same as the cpymem/movmem patterns.
+   IS_MOVE is true if this is a memmove, false for memcpy.
+   Return true if we successfully expanded, or false if we cannot
+   and must punt to a libcall.  */
+
+bool
+bpf_expand_cpymem (rtx *operands, bool is_move)
+{
+  /* Size must be constant for this expansion to work.  */
+  if (!CONST_INT_P (operands[2]))
+{
+  const char *name = is_move ? "memmove" : "memcpy";
+  if (flag_building_libgcc)
+   warning (RTL_LOCATION (operands[2]),
+"could not expand call to %<__builtin_%s%> inline: "
+"size must be constant", name);
+  else
+ 

Re: [PATCH 1/5] btf: fixed type id in BTF_KIND_FUNC struct data.

[David Faust]

On 2/20/24 02:24, Cupertino Miranda wrote:
> This patch correct the aditition of +1 on the type id, which originally
> was done in the wrong location and leaded to func_sts->dtd_type for
> BTF_KIND_FUNCS struct data to contain the type id of the previous entry.
> 
> gcc/ChangeLog:
>   * btfout.cc (btf_collect_dataset): Corrected BTF type id.

OK, thanks.

> ---
>  gcc/btfout.cc | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
> index dcf751f8fe0d..7e114e224449 100644
> --- a/gcc/btfout.cc
> +++ b/gcc/btfout.cc
> @@ -457,7 +457,8 @@ btf_collect_datasec (ctf_container_ref ctfc)
>func_dtd->dtd_data.ctti_type = dtd->dtd_type;
>func_dtd->linkage = dtd->linkage;
>func_dtd->dtd_name = dtd->dtd_name;
> -  func_dtd->dtd_type = num_types_added + num_types_created;
> +  /* +1 for the sentinel type not in the types map.  */
> +  func_dtd->dtd_type = num_types_added + num_types_created + 1;
>  
>/* Only the BTF_KIND_FUNC type actually references the name. The
>BTF_KIND_FUNC_PROTO is always anonymous.  */
> @@ -480,8 +481,7 @@ btf_collect_datasec (ctf_container_ref ctfc)
>  
> struct btf_var_secinfo info;
>  
> -   /* +1 for the sentinel type not in the types map.  */
> -   info.type = func_dtd->dtd_type + 1;
> +   info.type = func_dtd->dtd_type;
>  
> /* Both zero at compile time.  */
> info.size = 0;

Re: [PATCH 3/5] btf: moved btf deallocation to final.

[David Faust]

Hi Cupertino,

On 2/20/24 02:24, Cupertino Miranda wrote:
> Dissociated .BTF.ext from the CO-RE relocations creation. Improvement of
> allocation/deallocation of BTF structures. Moving deallocation to final
> when needed.
> 
> gcc/ChangeLog:
> 
>   * config/bpf/bpf.cc (bpf_option_override): Make BTF.ext enabled
>   by default for BPF.
>   (btf_asm_init_sections): Add btf deallocation.
>   * dwarf2ctf.cc (ctf_debug_finalize): Fixed btf deallocation.

I find the commit message and ChangeLog here overly brief and a little
bit confusing.

You refer to moving the BTF deallocation to 'final', but IMO in this
context 'final' has the particular meaning of referring to pass_final,
which is not where the deallocation is moved. Rather, the patch moves it
to bpf_file_end, which implements the TARGET_ASM_FILE_END hook and is
called after pass_final. So I suggest to avoid calling it 'final', and
to explain a little in the commit message under what circumstances the
deallocation must be moved.

Also, I think the ChangeLog is missing an entry for bpf_file_end.

Please find a little note on a typo inline below.

> ---
>  gcc/config/bpf/bpf.cc | 20 +---
>  gcc/dwarf2ctf.cc  |  5 -
>  2 files changed, 13 insertions(+), 12 deletions(-)
> 
> diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc
> index d6ca47eeecbe..4318b26b9cda 100644
> --- a/gcc/config/bpf/bpf.cc
> +++ b/gcc/config/bpf/bpf.cc
> @@ -195,10 +195,8 @@ bpf_option_override (void)
>if (TARGET_BPF_CORE && !btf_debuginfo_p ())
>  error ("BPF CO-RE requires BTF debugging information, use %<-gbtf%>");
>  
> -  /* To support the portability needs of BPF CO-RE approach, BTF debug
> - information includes the BPF CO-RE relocations.  */
> -  if (TARGET_BPF_CORE)
> -write_symbols |= BTF_WITH_CORE_DEBUG;
> +  /* BPF applications always generate .BTF.ext.  */
> +  write_symbols |= BTF_WITH_CORE_DEBUG;
>  
>/* Unlike much of the other BTF debug information, the information 
> necessary
>   for CO-RE relocations is added to the CTF container by the BPF backend.
> @@ -218,10 +216,7 @@ bpf_option_override (void)
>/* -gbtf implies -mcore when using the BPF backend, unless -mno-co-re
>   is specified.  */
>if (btf_debuginfo_p () && !(target_flags_explicit & MASK_BPF_CORE))
> -{
> -  target_flags |= MASK_BPF_CORE;
> -  write_symbols |= BTF_WITH_CORE_DEBUG;
> -}
> +target_flags |= MASK_BPF_CORE;
>  
>/* Determine available features from ISA setting (-mcpu=).  */
>if (bpf_has_jmpext == -1)
> @@ -267,7 +262,7 @@ bpf_option_override (void)
>  static void
>  bpf_asm_init_sections (void)
>  {
> -  if (TARGET_BPF_CORE)
> +  if (btf_debuginfo_p () && btf_with_core_debuginfo_p ())
>  btf_ext_init ();
>  }
>  
> @@ -279,8 +274,11 @@ bpf_asm_init_sections (void)
>  static void
>  bpf_file_end (void)
>  {
> -  if (TARGET_BPF_CORE)
> -btf_ext_output ();
> +  if (btf_debuginfo_p () && btf_with_core_debuginfo_p ())
> +{
> +  btf_ext_output ();
> +  btf_finalize ();
> +}
>  }
>  
>  #undef TARGET_ASM_FILE_END
> diff --git a/gcc/dwarf2ctf.cc b/gcc/dwarf2ctf.cc
> index 93e5619933fa..b9dfecf2c1c4 100644
> --- a/gcc/dwarf2ctf.cc
> +++ b/gcc/dwarf2ctf.cc
> @@ -944,7 +944,10 @@ ctf_debug_finalize (const char *filename, bool btf)
>if (btf)
>  {
>btf_output (filename);
> -  btf_finalize ();
> +  /* btf_finalize when compiling BPF applciations gets deallocated by the
> +  BPF target in bpf_file_end.  */

typo: applications

> +  if (btf_debuginfo_p () && !btf_with_core_debuginfo_p ())
> + btf_finalize ();
>  }
>  
>else

Re: [PATCH v2] bpf: add inline memmove and memcpy expansion

[David Faust]

On 2/20/24 12:37, Jose E. Marchesi wrote:
> 
> Hi Faust.
> 
>> +bool
>> +bpf_expand_cpymem (rtx *operands, bool is_move)
>> +{
>> +  /* Size must be constant for this expansion to work.  */
>> +  if (!CONST_INT_P (operands[2]))
>> +{
>> +  const char *name = is_move ? "memmove" : "memcpy";
>> +  if (flag_building_libgcc)
>> +warning (RTL_LOCATION (operands[2]),
>> + "could not expand call to %<__builtin_%s%> inline: "
>> + "size must be constant", name);
>> +  else
>> +error ("could not expand call to %<__builtin_%s%> inline: "
>> +   "size must be constant", name);
>> +  return false;
>> +}
> 
> I think you want to use warning_at and error_at above...  the first
> argument to `warning' (and second to warning_at) is not a location but
> an OPT_W* value.  You should pass 0 to the opt argument in this case as
> there is no -W option to handle this warning.

Oops. Thanks.

After fixing the arguments, `warning' and `error' generally give better
messages than `warning_at' and `error_at' since the
RTL_LOCATION (operands[2]) seems to be NULL for most (all?) cases, which
causes the diagnostic message to refer only to the containing function.
The non-`_at' versions highlight the builtin call being expanded, which
is much nicer.

> 
> Also, I would not mention 'expand' in the error/warning message, as it
> is a GCC internal concept.  I would use perhaps "could not inline call
> to %<__builtin_%s%>".

OK.

> 
>> +  /* Alignment is a CONST_INT.  */
>> +  gcc_assert (CONST_INT_P (operands[3]));
>> +
>> +  rtx dst = operands[0];
>> +  rtx src = operands[1];
>> +  rtx size = operands[2];
>> +  unsigned HOST_WIDE_INT size_bytes = UINTVAL (size);
>> +  unsigned align = UINTVAL (operands[3]);
>> +  enum machine_mode mode;
>> +  switch (align)
>> +{
>> +case 1: mode = QImode; break;
>> +case 2: mode = HImode; break;
>> +case 4: mode = SImode; break;
>> +case 8: mode = DImode; break;
>> +default:
>> +  gcc_unreachable ();
>> +}
>> +
>> +  unsigned iters = size_bytes >> ceil_log2 (align);
>> +  unsigned remainder = size_bytes & (align - 1);
>> +
>> +  int inc = GET_MODE_SIZE (mode);
>> +  rtx_code_label *fwd_label, *done_label;
>> +  if (is_move)
>> +{
>> +  /* For memmove, be careful of overlap.  It is not a concern for 
>> memcpy.
>> + To handle overlap, we check (at runtime) if SRC < DST, and if so do
>> + the move "backwards" starting from SRC + SIZE.  */
>> +  fwd_label = gen_label_rtx ();
>> +  done_label = gen_label_rtx ();
>> +
>> +  rtx dst_addr = copy_to_mode_reg (Pmode, XEXP (dst, 0));
>> +  rtx src_addr = copy_to_mode_reg (Pmode, XEXP (src, 0));
>> +  emit_cmp_and_jump_insns (src_addr, dst_addr, GEU, NULL_RTX, Pmode,
>> +   true, fwd_label, profile_probability::even ());
>> +
>> +  /* Emit the "backwards" unrolled loop.  */
>> +  emit_move_loop (src, dst, mode, size_bytes, -inc, iters, remainder);
>> +  emit_jump_insn (gen_jump (done_label));
>> +  emit_barrier ();
>> +
>> +  emit_label (fwd_label);
>> +}
>> +
>> +  emit_move_loop (src, dst, mode, 0, inc, iters, remainder);
>> +
>> +  if (is_move)
>> +emit_label (done_label);
>> +
>> +  return true;
>> +}
>> +
>>  /* Finally, build the GCC target.  */
>>  
>>  struct gcc_target targetm = TARGET_INITIALIZER;
>> diff --git a/gcc/config/bpf/bpf.md b/gcc/config/bpf/bpf.md
>> index 50df1aaa3e2..ca677bc6b50 100644
>> --- a/gcc/config/bpf/bpf.md
>> +++ b/gcc/config/bpf/bpf.md
>> @@ -627,4 +627,40 @@ (define_insn "ldabs"
>>"{ldabs\t%0|r0 = *( *) skb[%0]}"
>>[(set_attr "type" "ld")])
>>  
>> +;;; memmove and memcopy
>> +
>> +;; 0 is dst
>> +;; 1 is src
>> +;; 2 is size of copy in bytes
>> +;; 3 is alignment
>> +
>> +(define_expand "cpymemdi"
>> +  [(match_operand:BLK 0 "memory_operand")
>> +   (match_operand:BLK 1 "memory_operand")
>> +   (match_operand:DI 2 "general_operand")
>> +   (match_operand:DI 3 "immediate_operand")]
>> +   ""
>> +{
>> +  if (bpf_expand_cpymem (operands, false))
>> +DONE;
>> +  FAIL;
>> +})
>> +
>> +;; 0 is dst
>> +;; 1 is src
>> +;; 2 is size of copy in bytes
>> +;; 3 is alignment
>> +
>> +(define_expand "movmemdi"
>> +  [(match_operand:BLK 0 "memory_operand")
>> +   (match_operand:BLK 1 "memory_operand")
>> +   (match_operand:DI 2 "general_operand")
>> +   (match_operand:DI 3 "immediate_operand")]
>> +   ""
>> +{
>> +  if (bpf_expand_cpymem (operands, true))
>> +DONE;
>> +  FAIL;
>> +})
>> +
>>  (include "atomic.md")
>> diff --git a/gcc/testsuite/gcc.target/bpf/memcpy-1.c 
>> b/gcc/testsuite/gcc.target/bpf/memcpy-1.c
>> new file mode 100644
>> index 000..6c9707f24e8
>> --- /dev/null
>> +++ b/gcc/testsuite/gcc.target/bpf/memcpy-1.c
>> @@ -0,0 +1,26 @@
>> +/* Ensure memcpy is expanded inline rather than emitting a libcall.  */
>> +
>> +/* { dg-do compile } */
>> +/* { dg-options "-O2" } */
>> +
>> +struct context {
>> + unsigned int data;
>> 

[PATCH v3] bpf: add inline memmove and memcpy expansion

[David Faust]

[Changes from v2: 
 - Fix incorrectly passing a location instead of OPT_W* for warning ().
 - Reword warning/error message and test accordingly.  ]
 
[Changes from v1: Jose's review comments, all of which I agree with.
 - Fix 'implments' typo in commit message.
 - Change check that alignment is CONST_INT to gcc_assert ().
 - Change default case in alignment switch to gcc_unreachable ().
 - Reword error message for non-constant size memmove/memcpy, and
   update test for the error accordingly.
 - Delete CPYMEM_EXPAND_ERR macro, since it was now only used in
   one place.  ]

BPF programs are not typically linked, which means we cannot fall back
on library calls to implement __builtin_{memmove,memcpy} and should
always expand them inline if possible.

GCC already successfully expands these builtins inline in many cases,
but failed to do so for a few for simple cases involving overlapping
memmove in the kernel BPF selftests and was instead emitting a libcall.

This patch implements a simple inline expansion of memcpy and memmove in
the BPF backend in a verifier-friendly way, with the caveat that the
size must be an integer constant, which is also required by clang.

Tested for bpf-unknown-none on x86_64-linux-gnu host.

Also tested against the BPF verifier by compiling and loading a test
program with overlapping memmove (essentially the memmove-1.c test)
which failed before due to a libcall, and now successfully loads and
passes the verifier.

gcc/

* config/bpf/bpf-protos.h (bpf_expand_cpymem): New.
* config/bpf/bpf.cc: (emit_move_loop, bpf_expand_cpymem): New.
* config/bpf/bpf.md: (cpymemdi, movmemdi): New define_expands.

gcc/testsuite/

* gcc.target/bpf/memcpy-1.c: New test.
* gcc.target/bpf/memmove-1.c: New test.
* gcc.target/bpf/memmove-2.c: New test.
---
 gcc/config/bpf/bpf-protos.h  |   2 +
 gcc/config/bpf/bpf.cc| 115 +++
 gcc/config/bpf/bpf.md|  36 +++
 gcc/testsuite/gcc.target/bpf/memcpy-1.c  |  26 +
 gcc/testsuite/gcc.target/bpf/memmove-1.c |  46 +
 gcc/testsuite/gcc.target/bpf/memmove-2.c |  23 +
 6 files changed, 248 insertions(+)
 create mode 100644 gcc/testsuite/gcc.target/bpf/memcpy-1.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/memmove-1.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/memmove-2.c

diff --git a/gcc/config/bpf/bpf-protos.h b/gcc/config/bpf/bpf-protos.h
index 46d950bd990..366acb87ae4 100644
--- a/gcc/config/bpf/bpf-protos.h
+++ b/gcc/config/bpf/bpf-protos.h
@@ -35,4 +35,6 @@ const char *bpf_add_core_reloc (rtx *operands, const char 
*templ);
 class gimple_opt_pass;
 gimple_opt_pass *make_pass_lower_bpf_core (gcc::context *ctxt);
 
+bool bpf_expand_cpymem (rtx *, bool);
+
 #endif /* ! GCC_BPF_PROTOS_H */
diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc
index d6ca47eeecb..f9ac263613a 100644
--- a/gcc/config/bpf/bpf.cc
+++ b/gcc/config/bpf/bpf.cc
@@ -1184,6 +1184,121 @@ bpf_use_by_pieces_infrastructure_p (unsigned 
HOST_WIDE_INT size,
 #define TARGET_USE_BY_PIECES_INFRASTRUCTURE_P \
   bpf_use_by_pieces_infrastructure_p
 
+/* Helper for bpf_expand_cpymem.  Emit an unrolled loop moving the bytes
+   from SRC to DST.  */
+
+static void
+emit_move_loop (rtx src, rtx dst, machine_mode mode, int offset, int inc,
+   unsigned iters, unsigned remainder)
+{
+  rtx reg = gen_reg_rtx (mode);
+
+  /* First copy in chunks as large as alignment permits.  */
+  for (unsigned int i = 0; i < iters; i++)
+{
+  emit_move_insn (reg, adjust_address (src, mode, offset));
+  emit_move_insn (adjust_address (dst, mode, offset), reg);
+  offset += inc;
+}
+
+  /* Handle remaining bytes which might be smaller than the chunks
+ used above.  */
+  if (remainder & 4)
+{
+  emit_move_insn (reg, adjust_address (src, SImode, offset));
+  emit_move_insn (adjust_address (dst, SImode, offset), reg);
+  offset += (inc < 0 ? -4 : 4);
+  remainder -= 4;
+}
+  if (remainder & 2)
+{
+  emit_move_insn (reg, adjust_address (src, HImode, offset));
+  emit_move_insn (adjust_address (dst, HImode, offset), reg);
+  offset += (inc < 0 ? -2 : 2);
+  remainder -= 2;
+}
+  if (remainder & 1)
+{
+  emit_move_insn (reg, adjust_address (src, QImode, offset));
+  emit_move_insn (adjust_address (dst, QImode, offset), reg);
+}
+}
+
+/* Expand cpymem/movmem, as from __builtin_memcpy/memmove.
+   OPERANDS are the same as the cpymem/movmem patterns.
+   IS_MOVE is true if this is a memmove, false for memcpy.
+   Return true if we successfully expanded, or false if we cannot
+   and must punt to a libcall.  */
+
+bool
+bpf_expand_cpymem (rtx *operands, bool is_move)
+{
+  /* Size must be constant for this expansion to work.  */
+  if (!CONST_INT_P (operands[2]))
+{
+  const char *name = is_move ? "memmove" : "memcpy";
+  if (flag_building_libgcc)
+   warning (0, "could n

Re: [PATCH 2/5] btf: added KIND_FUNC traversal function.

[David Faust]

On 2/20/24 02:24, Cupertino Miranda wrote:
> Added a traversal function to traverse all BTF_KIND_FUNC nodes with a
> callback function. Used for .BTF.ext section content creation.
> 
> gcc/ChangeLog
> 
>   * btfout.cc (output_btf_func_types): use FOR_EACH_VEC_ELT.
>   (traverse_btf_func_types): Defined function.
>   * ctfc.h (funcs_traverse_callback): typedef for function
>   prototype.
>   (traverse_btf_func_types): Added prototype.

OK, thanks.

> ---
>  gcc/btfout.cc | 22 --
>  gcc/ctfc.h|  3 +++
>  2 files changed, 23 insertions(+), 2 deletions(-)
> 
> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
> index 7e114e224449..7aabd99f3e7c 100644
> --- a/gcc/btfout.cc
> +++ b/gcc/btfout.cc
> @@ -1276,8 +1276,10 @@ output_btf_types (ctf_container_ref ctfc)
>  static void
>  output_btf_func_types (ctf_container_ref ctfc)
>  {
> -  for (size_t i = 0; i < vec_safe_length (funcs); i++)
> -btf_asm_func_type (ctfc, (*funcs)[i], i);
> +  ctf_dtdef_ref ref;
> +  unsigned i;
> +  FOR_EACH_VEC_ELT (*funcs, i, ref)
> +btf_asm_func_type (ctfc, ref, i);
>  }
>  
>  /* Output all BTF_KIND_DATASEC records.  */
> @@ -1452,4 +1454,20 @@ btf_finalize (void)
>tu_ctfc = NULL;
>  }
>  
> +/* Traversal function for all BTF_KIND_FUNC type records.  */
> +
> +bool
> +traverse_btf_func_types (funcs_traverse_callback callback, void *data)
> +{
> +  ctf_dtdef_ref ref;
> +  unsigned i;
> +  FOR_EACH_VEC_ELT (*funcs, i, ref)
> +{
> +  bool stop = callback (ref, data);
> +  if (stop == true)
> + return true;
> +}
> +  return false;
> +}
> +
>  #include "gt-btfout.h"
> diff --git a/gcc/ctfc.h b/gcc/ctfc.h
> index 7aac57edac55..fa188bf2f5a4 100644
> --- a/gcc/ctfc.h
> +++ b/gcc/ctfc.h
> @@ -441,6 +441,9 @@ extern int ctf_add_variable (ctf_container_ref, const 
> char *, ctf_id_t,
>  extern ctf_id_t ctf_lookup_tree_type (ctf_container_ref, const tree);
>  extern ctf_id_t get_btf_id (ctf_id_t);
>  
> +typedef bool (*funcs_traverse_callback) (ctf_dtdef_ref, void *);
> +bool traverse_btf_func_types (funcs_traverse_callback, void *);
> +
>  /* CTF section does not emit location information; at this time, location
> information is needed for BTF CO-RE use-cases.  */
>  

Re: [PATCH 5/5] bpf: renamed coreout.* files to btfext-out.*.

[David Faust]

On 2/20/24 02:24, Cupertino Miranda wrote:
> gcc/ChangeLog:
>   * config.gcc (target_gtfiles): changed coreout to btfext-out.
>   (extra_objs): changed coreout to btfext-out.

I think these entries should start with a capital letter ("Changed...").

>   * config/bpf/coreout.cc: Renamed to btfext-out.cc
>   * config/bpf/btfext-out.cc: Added
>   * config/bpf/coreout.h: Renamed to btfext-out.h
>   * config/bpf/btfext-out.h: Added
>   * config/bpf/core-builtins.cc: Changed include
>   * config/bpf/core-builtins.h: Changed include
>   * config/bpf/t-bpf: Renamed file.

But, otherwise OK. Thanks.

> ---
>  gcc/config.gcc   | 4 ++--
>  gcc/config/bpf/{coreout.cc => btfext-out.cc} | 4 ++--
>  gcc/config/bpf/{coreout.h => btfext-out.h}   | 2 +-
>  gcc/config/bpf/core-builtins.cc  | 2 +-
>  gcc/config/bpf/core-builtins.h   | 2 +-
>  gcc/config/bpf/t-bpf | 4 ++--
>  6 files changed, 9 insertions(+), 9 deletions(-)
>  rename gcc/config/bpf/{coreout.cc => btfext-out.cc} (99%)
>  rename gcc/config/bpf/{coreout.h => btfext-out.h} (98%)
> 
> diff --git a/gcc/config.gcc b/gcc/config.gcc
> index a0f9c6723083..1ca033d75b66 100644
> --- a/gcc/config.gcc
> +++ b/gcc/config.gcc
> @@ -1653,8 +1653,8 @@ bpf-*-*)
>  tmake_file="${tmake_file} bpf/t-bpf"
>  use_collect2=no
>  use_gcc_stdint=provide
> -extra_objs="coreout.o core-builtins.o"
> -target_gtfiles="$target_gtfiles \$(srcdir)/config/bpf/coreout.cc 
> \$(srcdir)/config/bpf/core-builtins.cc"
> +extra_objs="btfext-out.o core-builtins.o"
> +target_gtfiles="$target_gtfiles \$(srcdir)/config/bpf/btfext-out.cc 
> \$(srcdir)/config/bpf/core-builtins.cc"
>  ;;
>  cris-*-elf | cris-*-none)
>   tm_file="elfos.h newlib-stdint.h ${tm_file}"
> diff --git a/gcc/config/bpf/coreout.cc b/gcc/config/bpf/btfext-out.cc
> similarity index 99%
> rename from gcc/config/bpf/coreout.cc
> rename to gcc/config/bpf/btfext-out.cc
> index 31b2abc3151b..4281cca83e13 100644
> --- a/gcc/config/bpf/coreout.cc
> +++ b/gcc/config/bpf/btfext-out.cc
> @@ -33,7 +33,7 @@
>  #include "tree-pretty-print.h"
>  #include "cgraph.h"
>  
> -#include "coreout.h"
> +#include "btfext-out.h"
>  
>  /* This file contains data structures and routines for construction and 
> output
> of BPF Compile Once - Run Everywhere (BPF CO-RE) information.
> @@ -618,4 +618,4 @@ btf_ext_output (void)
>dw2_asm_output_data (4, 0, "Required padding by libbpf structs");
>  }
>  
> -#include "gt-coreout.h"
> +#include "gt-btfext-out.h"
> diff --git a/gcc/config/bpf/coreout.h b/gcc/config/bpf/btfext-out.h
> similarity index 98%
> rename from gcc/config/bpf/coreout.h
> rename to gcc/config/bpf/btfext-out.h
> index 1c26b9274739..b36309475c97 100644
> --- a/gcc/config/bpf/coreout.h
> +++ b/gcc/config/bpf/btfext-out.h
> @@ -1,4 +1,4 @@
> -/* coreout.h - Declarations and definitions related to
> +/* btfext-out.h - Declarations and definitions related to
> BPF Compile Once - Run Everywhere (CO-RE) support.
> Copyright (C) 2021-2024 Free Software Foundation, Inc.
>  
> diff --git a/gcc/config/bpf/core-builtins.cc b/gcc/config/bpf/core-builtins.cc
> index aa75fd68cae6..8d8c54c1fb3d 100644
> --- a/gcc/config/bpf/core-builtins.cc
> +++ b/gcc/config/bpf/core-builtins.cc
> @@ -45,7 +45,7 @@ along with GCC; see the file COPYING3.  If not see
>  
>  #include "ctfc.h"
>  #include "btf.h"
> -#include "coreout.h"
> +#include "btfext-out.h"
>  #include "core-builtins.h"
>  
>  /* BPF CO-RE builtins definition.
> diff --git a/gcc/config/bpf/core-builtins.h b/gcc/config/bpf/core-builtins.h
> index c54f6ddac812..e56b55b94e0c 100644
> --- a/gcc/config/bpf/core-builtins.h
> +++ b/gcc/config/bpf/core-builtins.h
> @@ -1,7 +1,7 @@
>  #ifndef BPF_CORE_BUILTINS_H
>  #define BPF_CORE_BUILTINS_H
>  
> -#include "coreout.h"
> +#include "btfext-out.h"
>  
>  enum bpf_builtins
>  {
> diff --git a/gcc/config/bpf/t-bpf b/gcc/config/bpf/t-bpf
> index 18f1fa67794d..dc50332350c4 100644
> --- a/gcc/config/bpf/t-bpf
> +++ b/gcc/config/bpf/t-bpf
> @@ -1,7 +1,7 @@
>  
> -TM_H += $(srcdir)/config/bpf/coreout.h $(srcdir)/config/bpf/core-builtins.h
> +TM_H += $(srcdir)/config/bpf/btfext-out.h 
> $(srcdir)/config/bpf/core-builtins.h
>  
> -coreout.o: $(srcdir)/config/bpf/coreout.cc
> +btfext-out.o: $(srcdir)/config/bpf/btfext-out.cc
>   $(COMPILE) $<
>   $(POSTCOMPILE)
>  

Re: [PATCH v2 1/5] btf: fixed type id in BTF_KIND_FUNC struct data.

[David Faust]

Hi Cupertino,

Just some nits below. Apologies for incoming pedantry.

On 2/27/24 11:04, Cupertino Miranda wrote:
> This patch correct the aditition of +1 on the type id, which originally
> was done in the wrong location and leaded to func_sts->dtd_type for
> BTF_KIND_FUNCS struct data to contain the type id of the previous entry.

Multiple typos here:
  correct -> corrects
  aditition -> addition
  ...leaded to.. -> ..led to..
  func_sts -> func_dtd
  BTF_KIND_FUNCS -> BTF_KIND_FUNC

> 
> gcc/ChangeLog:
> 
>   * btfout.cc (btf_collect_dataset): Corrected BTF type id.

Please use present tense in the ChangeLog entries, to match GNU style
guidelines and existing entries,
i.e. "Correct..." instead of "Corrected..."

The same goes for the commit header, please use present tense to match
the style of existing commits,
i.e. "btf: fix type id..." instead of "fixed".

The patch itself LGTM, so OK with above changes.
Thanks!

> ---
>  gcc/btfout.cc | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
> index dcf751f8fe0d..7e114e224449 100644
> --- a/gcc/btfout.cc
> +++ b/gcc/btfout.cc
> @@ -457,7 +457,8 @@ btf_collect_datasec (ctf_container_ref ctfc)
>func_dtd->dtd_data.ctti_type = dtd->dtd_type;
>func_dtd->linkage = dtd->linkage;
>func_dtd->dtd_name = dtd->dtd_name;
> -  func_dtd->dtd_type = num_types_added + num_types_created;
> +  /* +1 for the sentinel type not in the types map.  */
> +  func_dtd->dtd_type = num_types_added + num_types_created + 1;
>  
>/* Only the BTF_KIND_FUNC type actually references the name. The
>BTF_KIND_FUNC_PROTO is always anonymous.  */
> @@ -480,8 +481,7 @@ btf_collect_datasec (ctf_container_ref ctfc)
>  
> struct btf_var_secinfo info;
>  
> -   /* +1 for the sentinel type not in the types map.  */
> -   info.type = func_dtd->dtd_type + 1;
> +   info.type = func_dtd->dtd_type;
>  
> /* Both zero at compile time.  */
> info.size = 0;

Re: [PATCH v2 2/5] btf: added KIND_FUNC traversal function.

[David Faust]

Hi Cupertino,

Similar to patch 1, please use present tense to match the style of
existing commits, in commit message and in ChangeLog.

On 2/27/24 11:04, Cupertino Miranda wrote:
> Added a traversal function to traverse all BTF_KIND_FUNC nodes with a
> callback function. Used for .BTF.ext section content creation.

Added -> Add

> 
> gcc/ChangeLog:
> 
>   * btfout.cc (output_btf_func_types): Use FOR_EACH_VEC_ELT.
>   (traverse_btf_func_types): Defined function.
>   * ctfc.h (funcs_traverse_callback): Typedef for function
>   prototype.
>   (traverse_btf_func_types): Added prototype.

Mix of present and past tenses here, please stick to the present:
Defined -> Define
Added -> Add

The code changes LGTM, so OK with those nits fixed.
Thanks.

> ---
>  gcc/btfout.cc | 22 --
>  gcc/ctfc.h|  3 +++
>  2 files changed, 23 insertions(+), 2 deletions(-)
> 
> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
> index 7e114e224449..7aabd99f3e7c 100644
> --- a/gcc/btfout.cc
> +++ b/gcc/btfout.cc
> @@ -1276,8 +1276,10 @@ output_btf_types (ctf_container_ref ctfc)
>  static void
>  output_btf_func_types (ctf_container_ref ctfc)
>  {
> -  for (size_t i = 0; i < vec_safe_length (funcs); i++)
> -btf_asm_func_type (ctfc, (*funcs)[i], i);
> +  ctf_dtdef_ref ref;
> +  unsigned i;
> +  FOR_EACH_VEC_ELT (*funcs, i, ref)
> +btf_asm_func_type (ctfc, ref, i);
>  }
>  
>  /* Output all BTF_KIND_DATASEC records.  */
> @@ -1452,4 +1454,20 @@ btf_finalize (void)
>tu_ctfc = NULL;
>  }
>  
> +/* Traversal function for all BTF_KIND_FUNC type records.  */
> +
> +bool
> +traverse_btf_func_types (funcs_traverse_callback callback, void *data)
> +{
> +  ctf_dtdef_ref ref;
> +  unsigned i;
> +  FOR_EACH_VEC_ELT (*funcs, i, ref)
> +{
> +  bool stop = callback (ref, data);
> +  if (stop == true)
> + return true;
> +}
> +  return false;
> +}
> +
>  #include "gt-btfout.h"
> diff --git a/gcc/ctfc.h b/gcc/ctfc.h
> index 7aac57edac55..fa188bf2f5a4 100644
> --- a/gcc/ctfc.h
> +++ b/gcc/ctfc.h
> @@ -441,6 +441,9 @@ extern int ctf_add_variable (ctf_container_ref, const 
> char *, ctf_id_t,
>  extern ctf_id_t ctf_lookup_tree_type (ctf_container_ref, const tree);
>  extern ctf_id_t get_btf_id (ctf_id_t);
>  
> +typedef bool (*funcs_traverse_callback) (ctf_dtdef_ref, void *);
> +bool traverse_btf_func_types (funcs_traverse_callback, void *);
> +
>  /* CTF section does not emit location information; at this time, location
> information is needed for BTF CO-RE use-cases.  */
>  

Re: [PATCH v2 3/5] bpf: Always emit .BTF.ext section if generating BTF

[David Faust]

On 2/27/24 11:04, Cupertino Miranda wrote:
> BPF applications, when generating BTF information should always create a
> .BTF.ext section.
> Current implementation was only creating it when -mco-re option was used.
> This patch makes .BTF.ext always be generated for BPF target objects.
> The patch also adds conditions around btf_finalize function call
> such that BTF deallocation happens later for BPF target.
> For BPF, btf_finalize is only called after .BTF.ext is generated.

Thank you, this version makes it much more clear what the patch does.

> 
> gcc/ChangeLog:
> 
>   * config/bpf/bpf.cc (bpf_option_override): Make .BTF.ext
>   enabled by default for BPF.
>   (bpf_file_end): Call BTF deallocation.
>   * dwarf2ctf.cc (ctf_debug_finalize): Conditionally execute BTF
>   deallocation.

You are missing ChangeLog entries for bpf_asm_init_sections and
ctf_debug_finish.

The script contrib/gcc-changelog/git_check_commit.py may help
to catch those.

The code changes LGTM, so OK with the ChangeLog fixed.
Thanks.

> ---
>  gcc/config/bpf/bpf.cc | 20 +---
>  gcc/dwarf2ctf.cc  | 12 ++--
>  2 files changed, 15 insertions(+), 17 deletions(-)
> 
> diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc
> index d6ca47eeecbe..4318b26b9cda 100644
> --- a/gcc/config/bpf/bpf.cc
> +++ b/gcc/config/bpf/bpf.cc
> @@ -195,10 +195,8 @@ bpf_option_override (void)
>if (TARGET_BPF_CORE && !btf_debuginfo_p ())
>  error ("BPF CO-RE requires BTF debugging information, use %<-gbtf%>");
>  
> -  /* To support the portability needs of BPF CO-RE approach, BTF debug
> - information includes the BPF CO-RE relocations.  */
> -  if (TARGET_BPF_CORE)
> -write_symbols |= BTF_WITH_CORE_DEBUG;
> +  /* BPF applications always generate .BTF.ext.  */
> +  write_symbols |= BTF_WITH_CORE_DEBUG;
>  
>/* Unlike much of the other BTF debug information, the information 
> necessary
>   for CO-RE relocations is added to the CTF container by the BPF backend.
> @@ -218,10 +216,7 @@ bpf_option_override (void)
>/* -gbtf implies -mcore when using the BPF backend, unless -mno-co-re
>   is specified.  */
>if (btf_debuginfo_p () && !(target_flags_explicit & MASK_BPF_CORE))
> -{
> -  target_flags |= MASK_BPF_CORE;
> -  write_symbols |= BTF_WITH_CORE_DEBUG;
> -}
> +target_flags |= MASK_BPF_CORE;
>  
>/* Determine available features from ISA setting (-mcpu=).  */
>if (bpf_has_jmpext == -1)
> @@ -267,7 +262,7 @@ bpf_option_override (void)
>  static void
>  bpf_asm_init_sections (void)
>  {
> -  if (TARGET_BPF_CORE)
> +  if (btf_debuginfo_p () && btf_with_core_debuginfo_p ())
>  btf_ext_init ();
>  }
>  
> @@ -279,8 +274,11 @@ bpf_asm_init_sections (void)
>  static void
>  bpf_file_end (void)
>  {
> -  if (TARGET_BPF_CORE)
> -btf_ext_output ();
> +  if (btf_debuginfo_p () && btf_with_core_debuginfo_p ())
> +{
> +  btf_ext_output ();
> +  btf_finalize ();
> +}
>  }
>  
>  #undef TARGET_ASM_FILE_END
> diff --git a/gcc/dwarf2ctf.cc b/gcc/dwarf2ctf.cc
> index 93e5619933fa..dca86edfffa9 100644
> --- a/gcc/dwarf2ctf.cc
> +++ b/gcc/dwarf2ctf.cc
> @@ -944,7 +944,10 @@ ctf_debug_finalize (const char *filename, bool btf)
>if (btf)
>  {
>btf_output (filename);
> -  btf_finalize ();
> +  /* btf_finalize when compiling BPF applciations gets deallocated by the
> +  BPF target in bpf_file_end.  */
> +  if (btf_debuginfo_p () && !btf_with_core_debuginfo_p ())
> + btf_finalize ();
>  }
>  
>else
> @@ -1027,11 +1030,8 @@ ctf_debug_finish (const char * filename)
>/* Emit BTF debug info here when CO-RE relocations need to be generated.
>   BTF with CO-RE relocations needs to be generated when CO-RE is in effect
>   for the BPF target.  */
> -  if (btf_with_core_debuginfo_p ())
> -{
> -  gcc_assert (btf_debuginfo_p ());
> -  ctf_debug_finalize (filename, btf_debuginfo_p ());
> -}
> +  if (btf_debuginfo_p () && btf_with_core_debuginfo_p ())
> +ctf_debug_finalize (filename, btf_debuginfo_p ());
>  }
>  
>  #include "gt-dwarf2ctf.h"

Re: [PATCH v2 5/5] bpf: renamed coreout.* files to btfext-out.*.

[David Faust]

On 2/27/24 11:04, Cupertino Miranda wrote:
> gcc/ChangeLog:
> 
>   * config.gcc (target_gtfiles): Changes coreout to btfext-out.
>   (extra_objs): Changes coreout to btfext-out.
>   * config/bpf/coreout.cc: Renamed to btfext-out.cc.
>   * config/bpf/btfext-out.cc: Added.
>   * config/bpf/coreout.h: Renamed to btfext-out.h.
>   * config/bpf/btfext-out.h: Added.
>   * config/bpf/core-builtins.cc: Changes include.
>   * config/bpf/core-builtins.h: Changes include.
>   * config/bpf/t-bpf: Renamed file.
This last entry is confusing, sounds like t-bpf is renamed, which it
isn't. I'd suggest to just say "accomodate renamed files" or so.

Similar to prior patches, there is a mix of present and past tenses here.
Please stick with the present.

Changes -> Change
Added -> Add  (or just "New.")
Renamed -> Rename.

OK with those changes.
Thanks.

> ---
>  gcc/config.gcc   | 4 ++--
>  gcc/config/bpf/{coreout.cc => btfext-out.cc} | 4 ++--
>  gcc/config/bpf/{coreout.h => btfext-out.h}   | 2 +-
>  gcc/config/bpf/core-builtins.cc  | 2 +-
>  gcc/config/bpf/core-builtins.h   | 2 +-
>  gcc/config/bpf/t-bpf | 4 ++--
>  6 files changed, 9 insertions(+), 9 deletions(-)
>  rename gcc/config/bpf/{coreout.cc => btfext-out.cc} (99%)
>  rename gcc/config/bpf/{coreout.h => btfext-out.h} (98%)
> 
> diff --git a/gcc/config.gcc b/gcc/config.gcc
> index a0f9c6723083..1ca033d75b66 100644
> --- a/gcc/config.gcc
> +++ b/gcc/config.gcc
> @@ -1653,8 +1653,8 @@ bpf-*-*)
>  tmake_file="${tmake_file} bpf/t-bpf"
>  use_collect2=no
>  use_gcc_stdint=provide
> -extra_objs="coreout.o core-builtins.o"
> -target_gtfiles="$target_gtfiles \$(srcdir)/config/bpf/coreout.cc 
> \$(srcdir)/config/bpf/core-builtins.cc"
> +extra_objs="btfext-out.o core-builtins.o"
> +target_gtfiles="$target_gtfiles \$(srcdir)/config/bpf/btfext-out.cc 
> \$(srcdir)/config/bpf/core-builtins.cc"
>  ;;
>  cris-*-elf | cris-*-none)
>   tm_file="elfos.h newlib-stdint.h ${tm_file}"
> diff --git a/gcc/config/bpf/coreout.cc b/gcc/config/bpf/btfext-out.cc
> similarity index 99%
> rename from gcc/config/bpf/coreout.cc
> rename to gcc/config/bpf/btfext-out.cc
> index 31b2abc3151b..4281cca83e13 100644
> --- a/gcc/config/bpf/coreout.cc
> +++ b/gcc/config/bpf/btfext-out.cc
> @@ -33,7 +33,7 @@
>  #include "tree-pretty-print.h"
>  #include "cgraph.h"
>  
> -#include "coreout.h"
> +#include "btfext-out.h"
>  
>  /* This file contains data structures and routines for construction and 
> output
> of BPF Compile Once - Run Everywhere (BPF CO-RE) information.
> @@ -618,4 +618,4 @@ btf_ext_output (void)
>dw2_asm_output_data (4, 0, "Required padding by libbpf structs");
>  }
>  
> -#include "gt-coreout.h"
> +#include "gt-btfext-out.h"
> diff --git a/gcc/config/bpf/coreout.h b/gcc/config/bpf/btfext-out.h
> similarity index 98%
> rename from gcc/config/bpf/coreout.h
> rename to gcc/config/bpf/btfext-out.h
> index 1c26b9274739..b36309475c97 100644
> --- a/gcc/config/bpf/coreout.h
> +++ b/gcc/config/bpf/btfext-out.h
> @@ -1,4 +1,4 @@
> -/* coreout.h - Declarations and definitions related to
> +/* btfext-out.h - Declarations and definitions related to
> BPF Compile Once - Run Everywhere (CO-RE) support.
> Copyright (C) 2021-2024 Free Software Foundation, Inc.
>  
> diff --git a/gcc/config/bpf/core-builtins.cc b/gcc/config/bpf/core-builtins.cc
> index aa75fd68cae6..8d8c54c1fb3d 100644
> --- a/gcc/config/bpf/core-builtins.cc
> +++ b/gcc/config/bpf/core-builtins.cc
> @@ -45,7 +45,7 @@ along with GCC; see the file COPYING3.  If not see
>  
>  #include "ctfc.h"
>  #include "btf.h"
> -#include "coreout.h"
> +#include "btfext-out.h"
>  #include "core-builtins.h"
>  
>  /* BPF CO-RE builtins definition.
> diff --git a/gcc/config/bpf/core-builtins.h b/gcc/config/bpf/core-builtins.h
> index c54f6ddac812..e56b55b94e0c 100644
> --- a/gcc/config/bpf/core-builtins.h
> +++ b/gcc/config/bpf/core-builtins.h
> @@ -1,7 +1,7 @@
>  #ifndef BPF_CORE_BUILTINS_H
>  #define BPF_CORE_BUILTINS_H
>  
> -#include "coreout.h"
> +#include "btfext-out.h"
>  
>  enum bpf_builtins
>  {
> diff --git a/gcc/config/bpf/t-bpf b/gcc/config/bpf/t-bpf
> index 18f1fa67794d..dc50332350c4 100644
> --- a/gcc/config/bpf/t-bpf
> +++ b/gcc/config/bpf/t-bpf
> @@ -1,7 +1,7 @@
>  
> -TM_H += $(srcdir)/config/bpf/coreout.h $(srcdir)/config/bpf/core-builtins.h
> +TM_H += $(srcdir)/config/bpf/btfext-out.h 
> $(srcdir)/config/bpf/core-builtins.h
>  
> -coreout.o: $(srcdir)/config/bpf/coreout.cc
> +btfext-out.o: $(srcdir)/config/bpf/btfext-out.cc
>   $(COMPILE) $<
>   $(POSTCOMPILE)
>  

Re: [PATCH v2 4/5] bpf: implementation of func_info in .BTF.ext.

[David Faust]

Hi Cupertino,

On 2/27/24 11:04, Cupertino Miranda wrote:
> Kernel verifier complains in some particular cases for missing func_info
> implementation in .BTF.ext. This patch implements it.
> 
> Strings are cached locally in coreout.cc to avoid adding duplicated
> strings in the string list. This string deduplication should eventually
> be moved to the CTFC functions such that this happens widely.
> 
> With this implementation, the CO-RE relocations information was also
> simplified and integrated with the FuncInfo structures.
>

I have just a couple small comments inline in the patch below, but they
are very minor and only suggestions/nits.

The ChangeLog has the same past/present tense issue as the other patches
in the series, but apart from that I see no issues. Great work! Thanks
for implementing this.

Patch is OK with the ChangeLog fixed up, and the inline nits - if 
you agree.
Thanks!
 
> gcc/Changelog:
> 
>   PR target/113453
>   * config/bpf/bpf.cc (bpf_function_prologue): Defined target
>   hook.
>   * config/bpf/coreout.cc (brf_ext_info_section)
>   (btf_ext_info): Moved from coreout.h
>   (btf_ext_funcinfo, btf_ext_lineinfo): Added struct.
>   (bpf_core_reloc): Renamed to btf_ext_core_reloc.
>   (btf_ext): Added static variable.
>   (btfext_info_sec_find_or_add, SEARCH_NODE_AND_RETURN)
>   (bpf_create_or_find_funcinfo, bpt_create_core_reloc)
>   (btf_ext_add_string, btf_funcinfo_type_callback)
>   (btf_add_func_info_for, btf_validate_funcinfo)
>   (btf_ext_info_len, output_btfext_func_info): Added function.
>   (output_btfext_header, bpf_core_reloc_add)
>   (output_btfext_core_relocs, btf_ext_init, btf_ext_output):
>   Changed to support new structs.
>   * config/bpf/coreout.h (btf_ext_funcinfo, btf_ext_lineinfo):
>   Moved and changed in coreout.cc.
>   (btf_add_func_info_for, btf_ext_add_string): Added prototypes.
> 
> gcc/testsuite/ChangeLog:
>   PR target/113453
>   * gcc.target/bpf/btfext-funcinfo-nocore.c: Added.
>   * gcc.target/bpf/btfext-funcinfo.c: Added.
>   * gcc.target/bpf/core-attr-5.c: Fixed regexp.
>   * gcc.target/bpf/core-attr-6.c: Fixed regexp.
>   * gcc.target/bpf/core-builtin-fieldinfo-offset-1.c: Fixed regexp.
>   * gcc.target/bpf/core-section-1.c: Fixed regexp
> ---
>  gcc/config/bpf/bpf.cc |  12 +
>  gcc/config/bpf/coreout.cc | 518 +-
>  gcc/config/bpf/coreout.h  |  20 +-
>  .../gcc.target/bpf/btfext-funcinfo-nocore.c   |  42 ++
>  .../gcc.target/bpf/btfext-funcinfo.c  |  46 ++
>  gcc/testsuite/gcc.target/bpf/core-attr-5.c|   9 +-
>  gcc/testsuite/gcc.target/bpf/core-attr-6.c|   6 +-
>  .../bpf/core-builtin-fieldinfo-offset-1.c |  13 +-
>  gcc/testsuite/gcc.target/bpf/core-section-1.c |   2 +-
>  9 files changed, 506 insertions(+), 162 deletions(-)
>  create mode 100644 gcc/testsuite/gcc.target/bpf/btfext-funcinfo-nocore.c
>  create mode 100644 gcc/testsuite/gcc.target/bpf/btfext-funcinfo.c
> 
> diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc
> index 4318b26b9cda..ea47e3a8dbfb 100644
> --- a/gcc/config/bpf/bpf.cc
> +++ b/gcc/config/bpf/bpf.cc
> @@ -385,6 +385,18 @@ bpf_compute_frame_layout (void)
>  #undef TARGET_COMPUTE_FRAME_LAYOUT
>  #define TARGET_COMPUTE_FRAME_LAYOUT bpf_compute_frame_layout
>  
> +/* Defined to initialize data for func_info region in .BTF.ext section.  */
> +
> +static void
> +bpf_function_prologue (FILE *f ATTRIBUTE_UNUSED)
> +{
> +  if (btf_debuginfo_p ())
> +btf_add_func_info_for (cfun->decl, current_function_func_begin_label);
> +}
> +
> +#undef TARGET_ASM_FUNCTION_PROLOGUE
> +#define TARGET_ASM_FUNCTION_PROLOGUE bpf_function_prologue
> +
>  /* Expand to the instructions in a function prologue.  This function
> is called when expanding the 'prologue' pattern in bpf.md.  */
>  
> diff --git a/gcc/config/bpf/coreout.cc b/gcc/config/bpf/coreout.cc
> index 2f06ec2a0f29..31b2abc3151b 100644
> --- a/gcc/config/bpf/coreout.cc
> +++ b/gcc/config/bpf/coreout.cc
> @@ -31,6 +31,7 @@
>  #include "btf.h"
>  #include "rtl.h"
>  #include "tree-pretty-print.h"
> +#include "cgraph.h"
>  
>  #include "coreout.h"
>  
> @@ -95,64 +96,193 @@
> result, a single .BTF.ext section can contain CO-RE relocations for 
> multiple
> programs in distinct sections.  */
>  
> -/* Internal representation of a BPF CO-RE relocation record.  */
> +/* BTF.ext debug info section.  */
> +static GTY (()) section * btf_ext_info_section;
> +
> +#ifndef BTF_EXT_INFO_SECTION_NAME
> +#define BTF_EXT_INFO_SECTION_NAME ".BTF.ext"
> +#endif
> +#define BTF_EXT_INFO_SECTION_FLAGS (SECTION_DEBUG)
> +
> +#ifndef BTF_EXT_INFO_SECTION_LABEL
> +#define BTF_EXT_INFO_SECTION_LABEL "Lbtfext"
> +#endif
> +
> +#define MAX_BTF_EXT_LABEL_BYTES 40
> +static char btf_ext_info_section_label[MAX_BTF_EXT_LABEL_BYTES];
> +
> +/* A funcinfo record, in the .BTF.ext funcinfo section.  */
>

Re: [PATCH] btf: print string position as comment for validation and testing purposes.

[David Faust]

Hi Cupertino,

On 1/8/24 02:55, Cupertino Miranda wrote:
> Hi everyone,
> 
> This patch adds a comment to the BTF strings regarding their position
> within the section. This is useful for assembly inspection purposes.
> 
> Regards,
> Cupertino
> 
> When using -dA, this function was only printing as comment btf_string or
> btf_aux_string.
> This patch changes the comment to also include the position of the
> string within the section in hexadecimal format.
> 
> gcc/ChangeLog:
>   * btfout.cc (output_btf_strs): Changed.

Please be a little bit more expressive in the ChangeLog.
Something along the lines of "print string offset in comment" will be
much more useful.

LGTM with that change, please apply.
Thanks!

> ---
>  gcc/btfout.cc | 7 +--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
> index db4f1084f85c..04218adc9e66 100644
> --- a/gcc/btfout.cc
> +++ b/gcc/btfout.cc
> @@ -1081,17 +1081,20 @@ static void
>  output_btf_strs (ctf_container_ref ctfc)
>  {
>ctf_string_t * ctf_string = ctfc->ctfc_strtable.ctstab_head;
> +  static int str_pos = 0;
>  
>while (ctf_string)
>  {
> -  dw2_asm_output_nstring (ctf_string->cts_str, -1, "btf_string");
> +  dw2_asm_output_nstring (ctf_string->cts_str, -1, "btf_string, str_pos 
> = 0x%x", str_pos);
> +  str_pos += strlen(ctf_string->cts_str) + 1;
>ctf_string = ctf_string->cts_next;
>  }
>  
>ctf_string = ctfc->ctfc_aux_strtable.ctstab_head;
>while (ctf_string)
>  {
> -  dw2_asm_output_nstring (ctf_string->cts_str, -1, "btf_aux_string");
> +  dw2_asm_output_nstring (ctf_string->cts_str, -1, "btf_aux_string, 
> str_pos = 0x%x", str_pos);
> +  str_pos += strlen(ctf_string->cts_str) + 1;
>ctf_string = ctf_string->cts_next;
>  }
>  }

Re: [PATCH] bpf: Correct BTF for kernel_helper attributed decls.

[David Faust]

Hi Cupetino,

On 1/8/24 03:05, Cupertino Miranda wrote:
> Hi everyone,
> 
> This patch address the problem reported in:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113225
> 
> Looking forward to your review.

LGTM, thanks. Please apply.

> 
> Cheers,
> Cupertino
> 
> 
> This patch fix a problem with kernel_helper attribute BTF information,
> which incorrectly generates BTF_KIND_FUNC entry.
> This BTF entry although accurate with traditional extern function
> declarations, once the function is attributed with kernel_helper, it is
> semantically incompatible of the kernel helpers in BPF infrastructure.
> 
> gcc/ChangeLog:
>   PR target/113225
>   * btfout.cc (btf_collect_datasec): Skip creating BTF info for
>   extern and kernel_helper attributed function decls.
> gcc/testsuite/ChangeLog:
>   * gcc.target/bpf/attr-kernel-helper.c: New test.
> ---
>  gcc/btfout.cc |  7 +++
>  gcc/testsuite/gcc.target/bpf/attr-kernel-helper.c | 15 +++
>  2 files changed, 22 insertions(+)
>  create mode 100644 gcc/testsuite/gcc.target/bpf/attr-kernel-helper.c
> 
> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
> index 04218adc9e66..39e7bec43bfb 100644
> --- a/gcc/btfout.cc
> +++ b/gcc/btfout.cc
> @@ -35,6 +35,8 @@ along with GCC; see the file COPYING3.  If not see
>  #include "diagnostic-core.h"
>  #include "cgraph.h"
>  #include "varasm.h"
> +#include "stringpool.h"
> +#include "attribs.h"
>  #include "dwarf2out.h" /* For lookup_decl_die.  */
>  
>  static int btf_label_num;
> @@ -429,6 +431,11 @@ btf_collect_datasec (ctf_container_ref ctfc)
>if (dtd == NULL)
>   continue;
>  
> +  if (DECL_EXTERNAL (func->decl)
> +   && (lookup_attribute ("kernel_helper",
> + DECL_ATTRIBUTES (func->decl))) != NULL_TREE)
> + continue;
> +
>/* Functions actually get two types: a BTF_KIND_FUNC_PROTO, and
>also a BTF_KIND_FUNC.  But the CTF container only allocates one
>type per function, which matches closely with BTF_KIND_FUNC_PROTO.
> diff --git a/gcc/testsuite/gcc.target/bpf/attr-kernel-helper.c 
> b/gcc/testsuite/gcc.target/bpf/attr-kernel-helper.c
> new file mode 100644
> index ..7c5a0007c979
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/bpf/attr-kernel-helper.c
> @@ -0,0 +1,15 @@
> +/* Basic test for kernel_helper attribute BTF information.  */
> +
> +/* { dg-do compile } */
> +/* { dg-options "-O0 -dA -gbtf" } */
> +
> +extern int foo_helper(int) __attribute((kernel_helper(42)));
> +extern int foo_nohelper(int);
> +
> +int bar (int arg)
> +{
> +  return foo_helper (arg) + foo_nohelper (arg);
> +}
> +
> +/* { dg-final { scan-assembler-times "BTF_KIND_FUNC 'foo_nohelper'" 1 } } */
> +/* { dg-final { scan-assembler-times "BTF_KIND_FUNC 'foo_helper'" 0 } } */

Re: [PATCH] btf: fix PR debug/112656

[David Faust]

Hi Indu,

On 11/30/23 14:18, Indu Bhagat wrote:
> PR debug/112656 - btf: function prototypes generated with name
> 
> With this patch, all BTF_KIND_FUNC_PROTO will appear anonymous in the
> generated BTF section.
> 
> As noted in the discussion in the bugzilla, the number of
> BTF_KIND_FUNC_PROTO types output varies across targets (BPF with -mco-re
> vs non-BPF targets).  Hence the check in the test case merely checks
> that all BTF_KIND_FUNC_PROTO appear anonymous.

Looks good to me. OK to apply.
Thanks!

> 
> gcc/ChangeLog:
> 
>   PR debug/112656
> * btfout.cc (btf_asm_type): Fixup ctti_name for all
>   BTF types of kind BTF_KIND_FUNC_PROTO.
> 
> gcc/testsuite/ChangeLog:
> 
>   PR debug/112656
> * gcc.dg/debug/btf/btf-function-7.c: New test.
> 
> 
> Testing notes:
>   - bootstrapped and reg tested on x86_64
>   - No regressions in btf.exp on BPF target
> 
> ---
>  gcc/btfout.cc |  4 
>  .../gcc.dg/debug/btf/btf-function-7.c | 19 +++
>  2 files changed, 23 insertions(+)
>  create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-function-7.c
> 
> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
> index 1c25404b2c0..a5e0d640e19 100644
> --- a/gcc/btfout.cc
> +++ b/gcc/btfout.cc
> @@ -820,6 +820,10 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd)
>   btf_kind = BTF_KIND_ENUM64;
> }
>  
> +  /* PR debug/112656.  BTF_KIND_FUNC_PROTO is always anonymous.  */
> +  if (btf_kind == BTF_KIND_FUNC_PROTO)
> +dtd->dtd_data.ctti_name = 0;
> +
>dw2_asm_output_data (4, dtd->dtd_data.ctti_name,
>  "TYPE %" PRIu64 " BTF_KIND_%s '%s'",
>  get_btf_id (dtd->dtd_type), btf_kind_name (btf_kind),
> diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-function-7.c 
> b/gcc/testsuite/gcc.dg/debug/btf/btf-function-7.c
> new file mode 100644
> index 000..b560dc75650
> --- /dev/null
> +++ b/gcc/testsuite/gcc.dg/debug/btf/btf-function-7.c
> @@ -0,0 +1,19 @@
> +/* Test BTF for inlined functions.
> +
> +   See PR/112656 - btf: function prototypes generated with name
> +   BTF_KIND_FUNC_PROTO must be anonymous.  */
> +
> +/* { dg-do compile } */
> +/* { dg-options "-O2 -gbtf -dA" } */
> +
> +/* { dg-final { scan-assembler-times "BTF_KIND_FUNC_PROTO 
> ''\\(\[0-9a-z\]*\\)'" 0 } } */
> +
> +static int log_event(const char *event_name, void *dev_ptr)
> +{
> +  return 666;
> +}
> +
> +int foo ()
> +{
> +  return log_event ("foobar", ((void *)0));
> +}

Re: [PATCH] btf: fix PR debug/112768

[David Faust]

On 11/30/23 14:17, Indu Bhagat wrote:
> PR debug/112768 - btf: fix asm comment output for BTF_KIND_FUNC* kinds
> 
> The patch adds a small function to abstract out the detail and return
> the name of the type.  The patch also fixes the issue of BTF_KIND_FUNC
> appearing in the comments with a 'null' string.

OK, thanks.

> 
> For btf-function-6.c testcase, after the patch:
> 
> .long   0   # TYPE 2 BTF_KIND_FUNC_PROTO ''
> .long   0xd02   # btt_info: kind=13, kflag=0, vlen=2
> .long   0x1 # btt_type: (BTF_KIND_INT 'int')
> .long   0   # farg_name
> .long   0x1 # farg_type: (BTF_KIND_INT 'int')
> .long   0   # farg_name
> .long   0x1 # farg_type: (BTF_KIND_INT 'int')
> .long   0   # TYPE 3 BTF_KIND_FUNC_PROTO ''
> .long   0xd01   # btt_info: kind=13, kflag=0, vlen=1
> .long   0x1 # btt_type: (BTF_KIND_INT 'int')
> .long   0x68# farg_name
> .long   0x1 # farg_type: (BTF_KIND_INT 'int')
> .long   0x5 # TYPE 4 BTF_KIND_FUNC 'extfunc'
> .long   0xc02   # btt_info: kind=12, kflag=0, linkage=2
> .long   0x2 # btt_type: (BTF_KIND_FUNC_PROTO '')
> .long   0xd # TYPE 5 BTF_KIND_FUNC 'foo'
> .long   0xc01   # btt_info: kind=12, kflag=0, linkage=1
> .long   0x3 # btt_type: (BTF_KIND_FUNC_PROTO '')
> 
> gcc/ChangeLog:
> 
>   PR debug/112768
>   * btfout.cc (get_btf_type_name): New definition.
>   (btf_collect_datasec): Update dtd_name to the original type name
>   string.
>   (btf_asm_type_ref): Use the new get_btf_type_name function
>   instead.
>   (btf_asm_type): Likewise.
>   (btf_asm_func_type): Likewise.
> 
> gcc/testsuite/ChangeLog:
> 
>   PR debug/112768
>   * gcc.dg/debug/btf/btf-function-6.c: Empty string expected with
>   BTF_KIND_FUNC_PROTO.
> 
> Testing notes:
>   - bootstrapped and reg tested on x86_64
>   - No regressions in btf.exp on BPF target
> 
> ---
>  gcc/btfout.cc | 22 +++
>  .../gcc.dg/debug/btf/btf-function-6.c |  4 ++--
>  2 files changed, 20 insertions(+), 6 deletions(-)
> 
> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
> index 5f2e99ce472..1c25404b2c0 100644
> --- a/gcc/btfout.cc
> +++ b/gcc/btfout.cc
> @@ -158,6 +158,19 @@ get_btf_kind (uint32_t ctf_kind)
>return BTF_KIND_UNKN;
>  }
>  
> +/* Some BTF types, like BTF_KIND_FUNC_PROTO, are anonymous.  The machinery
> +   in btfout to emit BTF, may reset dtd_data->ctti_name, but does not update
> +   the name in the ctf_dtdef_ref type object (deliberate choice).  This
> +   interface helps abstract out that state of affairs, while giving access to
> +   the name of the type as intended.  */
> +
> +static const char *
> +get_btf_type_name (ctf_dtdef_ref dtd)
> +{
> +  const char *anon = "";
> +  return (dtd->dtd_data.ctti_name) ? dtd->dtd_name : anon;
> +}
> +
>  /* Helper routines to map between 'relative' and 'absolute' IDs.
>  
> In BTF all records (including variables) are output in one long list, and 
> all
> @@ -425,6 +438,7 @@ btf_collect_datasec (ctf_container_ref ctfc)
>func_dtd->dtd_data = dtd->dtd_data;
>func_dtd->dtd_data.ctti_type = dtd->dtd_type;
>func_dtd->linkage = dtd->linkage;
> +  func_dtd->dtd_name = dtd->dtd_name;
>func_dtd->dtd_type = num_types_added + num_types_created;
>  
>/* Only the BTF_KIND_FUNC type actually references the name. The
> @@ -722,7 +736,7 @@ btf_asm_type_ref (const char *prefix, ctf_container_ref 
> ctfc, ctf_id_t ref_id)
>size_t func_id = btf_relative_func_id (ref_id);
>ctf_dtdef_ref ref_type = (*funcs)[func_id];
>dw2_asm_output_data (4, ref_id, "%s: (BTF_KIND_FUNC '%s')",
> -prefix, ref_type->dtd_name);
> +prefix, get_btf_type_name (ref_type));
>  }
>else
>  {
> @@ -733,7 +747,7 @@ btf_asm_type_ref (const char *prefix, ctf_container_ref 
> ctfc, ctf_id_t ref_id)
>  
>dw2_asm_output_data (4, ref_id, "%s: (BTF_KIND_%s '%s')",
>  prefix, btf_kind_name (ref_kind),
> -ref_type->dtd_name);
> +get_btf_type_name (ref_type));
>  }
>  }
>  
> @@ -809,7 +823,7 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd)
>dw2_asm_output_data (4, dtd->dtd_data.ctti_name,
>  "TYPE %" PRIu64 " BTF_KIND_%s '%s'",
>  get_btf_id (dtd->dtd_type), btf_kind_name (btf_kind),
> -dtd->dtd_name);
> +get_btf_type_name (dtd));
>dw2_asm_output_data (4, BTF_TYPE_INFO (btf_kind, btf_kflag, btf_vlen),
>  "btt_info: kind=%u, kflag=%u, vlen=%u",
>  btf_kind, btf_kflag, btf_vlen);
> @@ -950,7 +964,7 @@ btf_asm_func_type (ctf_container_ref ctfc, ctf_dtdef_ref 
> dtd, ct

[PATCH] btf: avoid wrong DATASEC entries for extern vars [PR112849]

[David Faust]

The process of creating BTF_KIND_DATASEC records involves iterating
through variable declarations, determining which section they will be
placed in, and creating an entry in the appropriate DATASEC record
accordingly.

For variables without e.g. an explicit __attribute__((section)), we use
categorize_decl_for_section () to identify the appropriate named section
and corresponding BTF_KIND_DATASEC record.

This was incorrectly being done for 'extern' variable declarations as
well as non-extern ones, which meant that extern variable declarations
could result in BTF_KIND_DATASEC entries claiming the variable is
allocated in some section such as '.bss' without any knowledge whether
that is actually true. That resulted in errors building the Linux kernel
BPF selftests.

This patch corrects btf_collect_datasec () to avoid assuming a section
for extern variables, and only emit BTF_KIND_DATASEC entries for them if
they have a known section.

Bootstrapped + tested on x86_64-linux-gnu.
Tested on x86_64-linux-gnu host for bpf-unknown-none.

gcc/
PR debug/112849
* btfout.cc (btf_collect_datasec): Avoid incorrectly creating an
entry in a BTF_KIND_DATASEC record for extern variable decls without
a known section.

gcc/testsuite/
PR debug/112849
* gcc.dg/debug/btf/btf-datasec-3.c: New test.
---
 gcc/btfout.cc | 10 ++-
 .../gcc.dg/debug/btf/btf-datasec-3.c  | 27 +++
 2 files changed, 36 insertions(+), 1 deletion(-)
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-datasec-3.c

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index a5e0d640e19..db4f1084f85 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -486,7 +486,15 @@ btf_collect_datasec (ctf_container_ref ctfc)
 
   /* Mark extern variables.  */
   if (DECL_EXTERNAL (node->decl))
-   dvd->dvd_visibility = BTF_VAR_GLOBAL_EXTERN;
+   {
+ dvd->dvd_visibility = BTF_VAR_GLOBAL_EXTERN;
+
+ /* PR112849: avoid assuming a section for extern decls without
+an explicit section, which would result in incorrectly
+emitting a BTF_KIND_DATASEC entry for them.  */
+ if (node->get_section () == NULL)
+   continue;
+   }
 
   const char *section_name = get_section_name (node);
   if (section_name == NULL)
diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-3.c 
b/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-3.c
new file mode 100644
index 000..3c1c7a28c2a
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-3.c
@@ -0,0 +1,27 @@
+/* PR debug/112849
+   Test that we do not incorrectly create BTF_KIND_DATASEC entries for
+   extern decls with no known section.  */
+
+/* { dg-do compile } */
+/* { dg-options "-O0 -gbtf -dA" } */
+
+extern int VERSION __attribute__((section (".version")));
+
+extern int test_bss1;
+extern int test_data1;
+
+int test_bss2;
+int test_data2 = 2;
+
+int
+foo (void)
+{
+  test_bss2 = VERSION;
+  return test_bss1 + test_data1 + test_data2;
+}
+
+/* There should only be a DATASEC entries for VERSION out of the extern decls. 
 */
+/* { dg-final { scan-assembler-times "bts_type" 3 } } */
+/* { dg-final { scan-assembler-times "bts_type: \\(BTF_KIND_VAR 
'test_data2'\\)" 1 } } */
+/* { dg-final { scan-assembler-times "bts_type: \\(BTF_KIND_VAR 
'test_bss2'\\)" 1 } } */
+/* { dg-final { scan-assembler-times "bts_type: \\(BTF_KIND_VAR 'VERSION'\\)" 
1 } } */
-- 
2.42.0

Re: [PATCH] btf: avoid wrong DATASEC entries for extern vars [PR112849]

[David Faust]

On 12/5/23 13:28, Indu Bhagat wrote:
> On 12/4/23 15:47, David Faust wrote:
>> The process of creating BTF_KIND_DATASEC records involves iterating
>> through variable declarations, determining which section they will be
>> placed in, and creating an entry in the appropriate DATASEC record
>> accordingly.
>>
>> For variables without e.g. an explicit __attribute__((section)), we use
>> categorize_decl_for_section () to identify the appropriate named section
>> and corresponding BTF_KIND_DATASEC record.
>>
>> This was incorrectly being done for 'extern' variable declarations as
>> well as non-extern ones, which meant that extern variable declarations
>> could result in BTF_KIND_DATASEC entries claiming the variable is
>> allocated in some section such as '.bss' without any knowledge whether
>> that is actually true. That resulted in errors building the Linux kernel
>> BPF selftests.
>>
>> This patch corrects btf_collect_datasec () to avoid assuming a section
>> for extern variables, and only emit BTF_KIND_DATASEC entries for them if
>> they have a known section.
>>
>> Bootstrapped + tested on x86_64-linux-gnu.
>> Tested on x86_64-linux-gnu host for bpf-unknown-none.
>>
> 
> One comment below.
> 
> LGTM, otherwise.
> Thanks
> 
>> gcc/
>>  PR debug/112849
>>  * btfout.cc (btf_collect_datasec): Avoid incorrectly creating an
>>  entry in a BTF_KIND_DATASEC record for extern variable decls without
>>  a known section.
>>
>> gcc/testsuite/
>>  PR debug/112849
>>  * gcc.dg/debug/btf/btf-datasec-3.c: New test.
>> ---
>>   gcc/btfout.cc | 10 ++-
>>   .../gcc.dg/debug/btf/btf-datasec-3.c  | 27 +++
>>   2 files changed, 36 insertions(+), 1 deletion(-)
>>   create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-datasec-3.c
>>
>> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
>> index a5e0d640e19..db4f1084f85 100644
>> --- a/gcc/btfout.cc
>> +++ b/gcc/btfout.cc
>> @@ -486,7 +486,15 @@ btf_collect_datasec (ctf_container_ref ctfc)
>>   
>> /* Mark extern variables.  */
>> if (DECL_EXTERNAL (node->decl))
>> -dvd->dvd_visibility = BTF_VAR_GLOBAL_EXTERN;
>> +{
>> +  dvd->dvd_visibility = BTF_VAR_GLOBAL_EXTERN;
>> +
>> +  /* PR112849: avoid assuming a section for extern decls without
>> + an explicit section, which would result in incorrectly
>> + emitting a BTF_KIND_DATASEC entry for them.  */
>> +  if (node->get_section () == NULL)
>> +continue;
>> +}
>>   
>> const char *section_name = get_section_name (node);
>> if (section_name == NULL)
>> diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-3.c 
>> b/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-3.c
>> new file mode 100644
>> index 000..3c1c7a28c2a
>> --- /dev/null
>> +++ b/gcc/testsuite/gcc.dg/debug/btf/btf-datasec-3.c
>> @@ -0,0 +1,27 @@
>> +/* PR debug/112849
>> +   Test that we do not incorrectly create BTF_KIND_DATASEC entries for
>> +   extern decls with no known section.  */
>> +
>> +/* { dg-do compile } */
>> +/* { dg-options "-O0 -gbtf -dA" } */
>> +
>> +extern int VERSION __attribute__((section (".version")));
>> +
>> +extern int test_bss1;
>> +extern int test_data1;
>> +
>> +int test_bss2;
>> +int test_data2 = 2;
>> +
>> +int
>> +foo (void)
>> +{
>> +  test_bss2 = VERSION;
>> +  return test_bss1 + test_data1 + test_data2;
>> +}
>> +
>> +/* There should only be a DATASEC entries for VERSION out of the extern 
>> decls.  */
> 
> The statement is unclear as is. Perhaps you wanted to say "There should 
> only be 3 DATASEC entries; including one for VERSION even though it is 
> extern decl" ?

Thanks. I reworded parts of that once or twice ended up with a garbled mess.

Changed to:

/* There should be 3 DATASEC entries total.  Of the extern decls, only VERSION
   has a known section; entries are not created for the other two.  */

and pushed.

> 
>> +/* { dg-final { scan-assembler-times "bts_type" 3 } } */
>> +/* { dg-final { scan-assembler-times "bts_type: \\(BTF_KIND_VAR 
>> 'test_data2'\\)" 1 } } */
>> +/* { dg-final { scan-assembler-times "bts_type: \\(BTF_KIND_VAR 
>> 'test_bss2'\\)" 1 } } */
>> +/* { dg-final { scan-assembler-times "bts_type: \\(BTF_KIND_VAR 
>> 'VERSION'\\)" 1 } } */
> 

[PATCH] btf: change encoding of forward-declared enums [PR111735]

[David Faust]

The BTF specification does not formally define a representation for
forward-declared enum types such as:

  enum Foo;

Forward-declarations for struct and union types are represented by
BTF_KIND_FWD, which has a 1-bit flag distinguishing the two.

The de-facto standard format used by other tools like clang and pahole
is to represent forward-declared enums as BTF_KIND_ENUM with vlen=0,
i.e. as a regular enum type with no enumerators.  This patch changes
GCC to adopt that format, and makes a couple of minor cleanups in
btf_asm_type ().

Bootstrapped and tested on x86_64-linux-gnu.
Also tested on x86_64-linux-gnu host for bpf-unknown-none target.

gcc/

PR debug/111735
* btfout.cc (btf_fwd_to_enum_p): New.
(btf_asm_type_ref): Special case references to enum forwards.
(btf_asm_type): Special case enum forwards. Rename btf_size_type to
btf_size, and change chained ifs switching on btf_kind into else ifs.

gcc/testsuite/

PR debug/111735
* gcc.dg/debug/btf/btf-forward-2.c: New test.
---
 gcc/btfout.cc | 46 ++-
 .../gcc.dg/debug/btf/btf-forward-2.c  | 18 
 2 files changed, 53 insertions(+), 11 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-forward-2.c

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index db4f1084f85..3ec938874b6 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -268,6 +268,17 @@ btf_emit_id_p (ctf_id_t id)
  && (btf_id_map[id] <= BTF_MAX_TYPE));
 }
 
+/* Return true if DTD is a forward-declared enum.  The BTF representation
+   of forward declared enums is not formally defined.  */
+
+static bool
+btf_fwd_to_enum_p (ctf_dtdef_ref dtd)
+{
+  uint32_t btf_kind = get_btf_kind (CTF_V2_INFO_KIND 
(dtd->dtd_data.ctti_info));
+
+  return (btf_kind == BTF_KIND_FWD && dtd->dtd_data.ctti_type == CTF_K_ENUM);
+}
+
 /* Each BTF type can be followed additional, variable-length information
completing the description of the type. Calculate the number of bytes
of variable information required to encode a given type.  */
@@ -753,8 +764,12 @@ btf_asm_type_ref (const char *prefix, ctf_container_ref 
ctfc, ctf_id_t ref_id)
   uint32_t ref_kind
= get_btf_kind (CTF_V2_INFO_KIND (ref_type->dtd_data.ctti_info));
 
+  const char *kind_name = btf_fwd_to_enum_p (ref_type)
+   ? btf_kind_name (BTF_KIND_ENUM)
+   : btf_kind_name (ref_kind);
+
   dw2_asm_output_data (4, ref_id, "%s: (BTF_KIND_%s '%s')",
-  prefix, btf_kind_name (ref_kind),
+  prefix, kind_name,
   get_btf_type_name (ref_type));
 }
 }
@@ -765,11 +780,11 @@ btf_asm_type_ref (const char *prefix, ctf_container_ref 
ctfc, ctf_id_t ref_id)
 static void
 btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd)
 {
-  uint32_t btf_kind, btf_kflag, btf_vlen, btf_size_type;
+  uint32_t btf_kind, btf_kflag, btf_vlen, btf_size;
   uint32_t ctf_info = dtd->dtd_data.ctti_info;
 
   btf_kind = get_btf_kind (CTF_V2_INFO_KIND (ctf_info));
-  btf_size_type = dtd->dtd_data.ctti_type;
+  btf_size = dtd->dtd_data.ctti_size;
   btf_vlen = CTF_V2_INFO_VLEN (ctf_info);
 
   /* By now any unrepresentable types have been removed.  */
@@ -777,7 +792,7 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd)
 
   /* Size 0 integers are redundant definitions of void. None should remain
  in the types list by this point.  */
-  gcc_assert (btf_kind != BTF_KIND_INT || btf_size_type >= 1);
+  gcc_assert (btf_kind != BTF_KIND_INT || btf_size >= 1);
 
   /* Re-encode the ctti_info to BTF.  */
   /* kflag is 1 for structs/unions with a bitfield member.
@@ -810,16 +825,26 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd)
  structs and forwards to unions. The dwarf2ctf conversion process stores
  the kind of the forward in ctti_type, but for BTF this must be 0 for
  forwards, with only the KIND_FLAG to distinguish.
- At time of writing, BTF forwards to enums are unspecified.  */
-  if (btf_kind == BTF_KIND_FWD)
+ Forwards to enum types are special-cased below.  */
+  else if (btf_kind == BTF_KIND_FWD)
 {
   if (dtd->dtd_data.ctti_type == CTF_K_UNION)
btf_kflag = 1;
 
-  btf_size_type = 0;
+  /* PR debug/111735.  Encode foward-declared enums as BTF_KIND_ENUM
+with vlen=0.  A representation for these is not formally defined;
+this is the de-facto standard used by other tools like clang
+and pahole.  */
+  else if (dtd->dtd_data.ctti_type == CTF_K_ENUM)
+   {
+ btf_kind = BTF_KIND_ENUM;
+ btf_vlen = 0;
+   }
+
+  btf_size = 0;
 }
 
-  if (btf_kind == BTF_KIND_ENUM)
+  else if (btf_kind == BTF_KIND_ENUM)
 {
   btf_kflag = dtd->dtd_enum_unsigned
? BTF_KF_ENUM_UNSIGNED
@@ -829,7 +854,7 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd)
}
 
   /* PR debug/112656.  BTF_KIND_FUNC_PROTO is alway

Re: [PATCH] bpf: Corrected condition in core_mark_as_access_index.

[David Faust]

On 11/13/23 14:36, Cupertino Miranda wrote:
> gcc/ChangeLog:
>   * config/bpf/core-builtins.cc (core_mark_as_access_index):
>   Corrected check.

OK, thanks.

> ---
>  gcc/config/bpf/core-builtins.cc | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/gcc/config/bpf/core-builtins.cc b/gcc/config/bpf/core-builtins.cc
> index 2ba78d7aed2e..1376c9309035 100644
> --- a/gcc/config/bpf/core-builtins.cc
> +++ b/gcc/config/bpf/core-builtins.cc
> @@ -1611,7 +1611,7 @@ core_mark_as_access_index (tree expr)
>|| TREE_CODE (expr) == INDIRECT_REF)
>  expr = TREE_OPERAND (expr, 0);
>  
> -  if (bpf_enum_mappings->get (expr) == NULL)
> +  if (core_access_index_map->get (expr) == NULL)
>  core_access_index_map->put (expr, NULL_TREE);
>  }
>  

Re: [PATCH] Fixed problem with BTF defining smaller enums.

[David Faust]

Hi Cupertino,

A couple of comments inline.

On 11/13/23 14:37, Cupertino Miranda wrote:
> This patch fixes a BTF, which would become invalid when having
> smaller then 4 byte definitions of enums.
> For example, when using the __attribute__((mode(byte))) in the enum
> definition.

Please add a test for this case in the BTF testsuite.

> 
> Two problems were identified:
>  - it would incorrectly create an entry for enum64 when the size of the
>enum was different then 4.
>  - it would allocate less then 4 bytes for the value entry in BTF, in
>case the type was smaller.
> 
> BTF generated was validated against clang.
> 
> gcc/ChangeLog:
>   * bpfout.cc (btf_calc_num_vbytes): Fixed logic for enum64.
>   (btf_asm_enum_const): Corrected logic for enum64 and smaller
>   than 4 bytes values.
> ---
>  gcc/btfout.cc | 10 +-
>  1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
> index e07fed302c24..d2263ec6eec3 100644
> --- a/gcc/btfout.cc
> +++ b/gcc/btfout.cc
> @@ -299,7 +299,7 @@ btf_calc_num_vbytes (ctf_dtdef_ref dtd)
>break;
>  
>  case BTF_KIND_ENUM:
> -  vlen_bytes += (dtd->dtd_data.ctti_size == 0x8)
> +  vlen_bytes += (dtd->dtd_data.ctti_size > 4)
>   ? vlen * sizeof (struct btf_enum64)
>   : vlen * sizeof (struct btf_enum);
>break;
> @@ -914,13 +914,13 @@ btf_asm_enum_const (unsigned int size, ctf_dmdef_t * 
> dmd, unsigned int idx)
>  {
>dw2_asm_output_data (4, dmd->dmd_name_offset, "ENUM_CONST '%s' idx=%u",
>  dmd->dmd_name, idx);
> -  if (size == 4)
> -dw2_asm_output_data (size, dmd->dmd_value, "bte_value");
> -  else
> +  if (size > 4)
>  {
> -  dw2_asm_output_data (4, dmd->dmd_value & 0x, "bte_value_lo32");
> +  dw2_asm_output_data (4, dmd->dmd_value & 0xfffe, "bte_value_lo32");

I don't understand the mask change here.
Why clear the low bit of the enum value?

>dw2_asm_output_data (4, (dmd->dmd_value >> 32) & 0x, 
> "bte_value_hi32");
>  }
> +  else
> +dw2_asm_output_data (size < 4 ? 4 : size, dmd->dmd_value, "bte_value");

In the else case isn't size guaranteed <= 4?
In which case, 'size < 4 ? 4 : size' always evaluates to 4.
So I would suggest to just write a literal 4 to keep it simple.


>  }
>  
>  /* Asm'out a function parameter description following a BTF_KIND_FUNC_PROTO. 
>  */

Re: [PATCH] bpf: Delayed the removal of the parser enum plugin handler.

[David Faust]

On 11/13/23 14:35, Cupertino Miranda wrote:
> The parser plugin handler that is responsible for collecting enum values
> information was being removed way too early.
> bpf_resolve_overloaded_core_builtin is called by the parser.
> It was moved to the function execute_lower_bpf_core.
> 

OK, thanks.

> gcc/ChangeLog:
>   * config/bpf/core-builtins.cc
>   (bpf_resolve_overloaded_core_builtin): Removed call.
>   (execute_lower_bpf_core): Added all to remove_parser_plugin.
> ---
>  gcc/config/bpf/core-builtins.cc | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/gcc/config/bpf/core-builtins.cc b/gcc/config/bpf/core-builtins.cc
> index a224847d5d91..2ba78d7aed2e 100644
> --- a/gcc/config/bpf/core-builtins.cc
> +++ b/gcc/config/bpf/core-builtins.cc
> @@ -1473,8 +1473,6 @@ tree
>  bpf_resolve_overloaded_core_builtin (location_t loc, tree fndecl,
>void *arglist)
>  {
> -  remove_parser_plugin ();
> -
>if (!bpf_require_core_support ())
>  return error_mark_node;
>  
> @@ -1688,6 +1686,7 @@ make_gimple_core_safe_access_index (tree *tp,
>  static unsigned int
>  execute_lower_bpf_core (void)
>  {
> +  remove_parser_plugin ();
>if (!TARGET_BPF_CORE)
>  return 0;
>  

Re: [PATCH v2] Fixed problem with BTF defining smaller enums.

[David Faust]

Hi Cupertino,

On 11/27/23 09:21, Cupertino Miranda wrote:
> Hi everyone,
> 
> David: Thanks for the v1 review.
> 
> This version adds the following;
>  - test case,
>  - improves condition logic,
>  - fixes mask typo.
> 
> Looking forward to your review.

v2 LGTM, please apply.
Thanks!

> 
> v1 at: https://gcc.gnu.org/pipermail/gcc-patches/2023-November/636391.html
> 
> Cheers,
> Cupertino
> 
> 
> 0004-Fixed-problem-with-BTF-defining-smaller-enums.patch
> 
> commit 3f89d352a4ee90882089142d743f8a748013b5fe
> Author: Cupertino Miranda 
> Date:   Fri Nov 10 14:02:30 2023 +
> 
> Fixed problem with BTF defining smaller enums.
> 
> This patch fixes a BTF, which would become invalid when having
> smaller then 4 byte definitions of enums.
> For example, when using the __attribute__((mode(byte))) in the enum
> definition.
> 
> Two problems were identified:
>  - it would incorrectly create an entry for enum64 when the size of the
>enum was different then 4.
>  - it would allocate less then 4 bytes for the value entry in BTF, in
>case the type was smaller.
> 
> BTF generated was validated against clang.
> 
> gcc/ChangeLog:
> * bpfout.cc (btf_calc_num_vbytes): Fixed logic for enum64.
> (btf_asm_enum_const): Corrected logic for enum64 and smaller
> than 4 bytes values.
> 
> gcc/testsuite/ChangeLog:
> gcc.dg/debug/btf/btf-enum-small.c: Added test.
> 
> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
> index e07fed302c24..5f2e99ce4725 100644
> --- a/gcc/btfout.cc
> +++ b/gcc/btfout.cc
> @@ -299,7 +299,7 @@ btf_calc_num_vbytes (ctf_dtdef_ref dtd)
>break;
>  
>  case BTF_KIND_ENUM:
> -  vlen_bytes += (dtd->dtd_data.ctti_size == 0x8)
> +  vlen_bytes += (dtd->dtd_data.ctti_size > 4)
>   ? vlen * sizeof (struct btf_enum64)
>   : vlen * sizeof (struct btf_enum);
>break;
> @@ -914,8 +914,8 @@ btf_asm_enum_const (unsigned int size, ctf_dmdef_t * dmd, 
> unsigned int idx)
>  {
>dw2_asm_output_data (4, dmd->dmd_name_offset, "ENUM_CONST '%s' idx=%u",
>  dmd->dmd_name, idx);
> -  if (size == 4)
> -dw2_asm_output_data (size, dmd->dmd_value, "bte_value");
> +  if (size <= 4)
> +dw2_asm_output_data (size < 4 ? 4 : size, dmd->dmd_value, "bte_value");
>else
>  {
>dw2_asm_output_data (4, dmd->dmd_value & 0x, "bte_value_lo32");
> diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-enum-small.c 
> b/gcc/testsuite/gcc.dg/debug/btf/btf-enum-small.c
> new file mode 100644
> index ..eb8a1bd2c438
> --- /dev/null
> +++ b/gcc/testsuite/gcc.dg/debug/btf/btf-enum-small.c
> @@ -0,0 +1,28 @@
> +/* Test BTF generation for small enums.  */
> +
> +/* { dg-do compile } */
> +/* { dg-options "-O2 -gbtf -dA" } */
> +
> +/* { dg-final { scan-assembler-not "bte_value_lo32" } } */
> +/* { dg-final { scan-assembler-not "bte_value_hi32" } } */
> +/* { dg-final { scan-assembler-times "\[\t \]0x602\[\t 
> \]+\[^\n\]*btt_info" 1 } } */
> +/* { dg-final { scan-assembler-times " ENUM_CONST 'eSMALL' idx=0" 1 } } */
> +/* { dg-final { scan-assembler-times " ENUM_CONST 'eSMALLY' idx=1" 1 } } */
> +/* { dg-final { scan-assembler-times "ascii \"eSMALL.0\"\[\t 
> \]+\[^\n\]*btf_string" 1 } } */
> +/* { dg-final { scan-assembler-times "ascii \"eSMALLY.0\"\[\t 
> \]+\[^\n\]*btf_string" 1 } } */
> +/* { dg-final { scan-assembler-times "bte_value" 2 } } */
> +
> +enum smalled_enum
> +{
> +  eSMALL,
> +  eSMALLY,
> +} __attribute__((mode(byte)));
> +
> +struct root_struct {
> +  enum smalled_enum esmall;
> +};
> +
> +enum smalled_enum
> +foo(struct root_struct *root) {
> +  return root->esmall;
> +}
> 

[PATCH] bpf: change ASM_COMMENT_START to '#'

[David Faust]

The BPF "pseudo-C" assembly dialect uses semi-colon (;) to separate
statements, not to begin line comments. The GNU assembler was recently
changed accordingly:

  https://sourceware.org/pipermail/binutils/2023-November/130867.html

This patch adapts the BPF backend in GCC accordingly, to use a hash (#)
instead of semi-colon (;) for ASM_COMMENT_START. This is supported
already in clang.

Tested on x86_64-linux-gnu host for bpf-unknown-none target.

gcc/
* config/bpf/bpf.h (ASM_COMMENT_START): Change from ';' to '#'.

gcc/testsuite/
* gcc.target/bpf/core-builtin-enumvalue-opt.c: Change dg-final
scans to not assume a specific comment character.
* gcc.target/bpf/core-builtin-enumvalue.c: Likewise.
* gcc.target/bpf/core-builtin-type-based.c: Likewise.
* gcc.target/bpf/core-builtin-type-id.c: Likewise.
---
 gcc/config/bpf/bpf.h |  2 +-
 .../gcc.target/bpf/core-builtin-enumvalue-opt.c  |  8 
 .../gcc.target/bpf/core-builtin-enumvalue.c  | 12 ++--
 .../gcc.target/bpf/core-builtin-type-based.c |  8 
 gcc/testsuite/gcc.target/bpf/core-builtin-type-id.c  |  6 +++---
 5 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/gcc/config/bpf/bpf.h b/gcc/config/bpf/bpf.h
index 1f177ec4c4e..d175e99046c 100644
--- a/gcc/config/bpf/bpf.h
+++ b/gcc/config/bpf/bpf.h
@@ -393,7 +393,7 @@ enum reg_class
 
 /*** The Overall Framework of an Assembler File.  */
 
-#define ASM_COMMENT_START ";"
+#define ASM_COMMENT_START "#"
 
 /* Output to assembler file text saying following lines
may contain character constants, extra white space, comments, etc.  */
diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-enumvalue-opt.c 
b/gcc/testsuite/gcc.target/bpf/core-builtin-enumvalue-opt.c
index c87e1a3ba3b..fc3c299fe9c 100644
--- a/gcc/testsuite/gcc.target/bpf/core-builtin-enumvalue-opt.c
+++ b/gcc/testsuite/gcc.target/bpf/core-builtin-enumvalue-opt.c
@@ -26,10 +26,10 @@ unsigned long foo(void *data)
  return 0;
 }
 
-/* { dg-final { scan-assembler-times "\t.4byte\t0x\[0-9a-f\]+\t; bpfcr_type 
\\(named_ue64\\)" 2 } } */
-/* { dg-final { scan-assembler-times "\t.4byte\t0x\[0-9a-f\]+\t; bpfcr_type 
\\(named_se64\\)" 2} } */
-/* { dg-final { scan-assembler-times "\t.4byte\t0xa\t; bpfcr_kind" 2 } } 
BPF_ENUMVAL_EXISTS */
-/* { dg-final { scan-assembler-times "\t.4byte\t0xb\t; bpfcr_kind" 2 } } 
BPF_ENUMVAL_VALUE */
+/* { dg-final { scan-assembler-times "bpfcr_type \\(named_ue64\\)" 2 } } */
+/* { dg-final { scan-assembler-times "bpfcr_type \\(named_se64\\)" 2} } */
+/* { dg-final { scan-assembler-times "0xa\[\t \]+\[^\n\]*bpfcr_kind" 2 } } 
BPF_ENUMVAL_EXISTS */
+/* { dg-final { scan-assembler-times "0xb\[\t \]+\[^\n\]*bpfcr_kind" 2 } } 
BPF_ENUMVAL_VALUE */
 
 /* { dg-final { scan-assembler-times "bpfcr_astr_off \\(\"0\"\\)" 4 } } */
 
diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-enumvalue.c 
b/gcc/testsuite/gcc.target/bpf/core-builtin-enumvalue.c
index 2f16903b8d6..23dfd8a10bf 100644
--- a/gcc/testsuite/gcc.target/bpf/core-builtin-enumvalue.c
+++ b/gcc/testsuite/gcc.target/bpf/core-builtin-enumvalue.c
@@ -40,12 +40,12 @@ int foo(void *data)
  return 0;
 }
 
-/* { dg-final { scan-assembler-times "\t.4byte\t0x\[0-9a-f\]+\t; bpfcr_type 
\\(named_ue64\\)" 5 } } */
-/* { dg-final { scan-assembler-times "\t.4byte\t0x\[0-9a-f\]+\t; bpfcr_type 
\\(named_se64\\)" 5} } */
-/* { dg-final { scan-assembler-times "\t.4byte\t0x\[0-9a-f\]+\t; bpfcr_type 
\\(named_ue\\)" 5 } } */
-/* { dg-final { scan-assembler-times "\t.4byte\t0x\[0-9a-f\]+\t; bpfcr_type 
\\(named_se\\)" 5} } */
-/* { dg-final { scan-assembler-times "\t.4byte\t0xa\t; bpfcr_kind" 12 } } 
BPF_ENUMVAL_EXISTS */
-/* { dg-final { scan-assembler-times "\t.4byte\t0xb\t; bpfcr_kind" 8 } } 
BPF_ENUMVAL_VALUE */
+/* { dg-final { scan-assembler-times "bpfcr_type \\(named_ue64\\)" 5 } } */
+/* { dg-final { scan-assembler-times "bpfcr_type \\(named_se64\\)" 5} } */
+/* { dg-final { scan-assembler-times "bpfcr_type \\(named_ue\\)" 5 } } */
+/* { dg-final { scan-assembler-times "bpfcr_type \\(named_se\\)" 5} } */
+/* { dg-final { scan-assembler-times "0xa\[\t \]+\[^\n\]*bpfcr_kind" 12 } } 
BPF_ENUMVAL_EXISTS */
+/* { dg-final { scan-assembler-times "0xb\[\t \]+\[^\n\]*bpfcr_kind" 8 } } 
BPF_ENUMVAL_VALUE */
 
 /* { dg-final { scan-assembler-times "bpfcr_astr_off \\(\"0\"\\)" 8 } } */
 /* { dg-final { scan-assembler-times "bpfcr_astr_off \\(\"1\"\\)" 8 } } */
diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-type-based.c 
b/gcc/testsuite/gcc.target/bpf/core-builtin-type-based.c
index 16b48ae0a00..74a8d5a14d9 100644
--- a/gcc/testsuite/gcc.target/bpf/core-builtin-type-based.c
+++ b/gcc/testsuite/gcc.target/bpf/core-builtin-type-based.c
@@ -52,7 +52,7 @@ int foo(void *data)
   return 0;
 }
 
-/* { dg-final { scan-assembler-times "\t.4byte\t0x0\t; bpfcr_type" 0 } } */
-/* { dg-final { scan-assembler-times "\t.4byte\t0x8\t; bpfcr_kind" 13 } } 
BPF_TYPE_EXISTS */

[COMMITTED] Regenerate common.opt.urls

[David Faust]

I was not aware of the requirement to regenerate the opt urls files
when adding a new option until the autobuilder complained.

Regenerate common.opt.urls for the -gprune-btf option added in:
  b8977d928a7a btf: add -gprune-btf option

Pushed as obvious.

gcc/
* common.opt.urls: Regenerate.
---
 gcc/common.opt.urls | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/gcc/common.opt.urls b/gcc/common.opt.urls
index 1ec326706332..0e71bce27c49 100644
--- a/gcc/common.opt.urls
+++ b/gcc/common.opt.urls
@@ -1642,6 +1642,9 @@ UrlSuffix(gcc/Debugging-Options.html#index-gctf)
 gbtf
 UrlSuffix(gcc/Debugging-Options.html#index-gbtf)
 
+gprune-btf
+UrlSuffix(gcc/Debugging-Options.html#index-gprune-btf)
+
 gdwarf
 UrlSuffix(gcc/Debugging-Options.html#index-gdwarf)
 
-- 
2.43.0

[PATCH] opts: allow -gctf, -gbtf, -gdwarf simultaneously

[David Faust]

[This is a resend of a patch previously sent as:
   PATCH v4 6/6 opts: allow any combination of DWARF,CTF,BTF
   https://gcc.gnu.org/pipermail/gcc-patches/2024-June/654253.html]

Previously it was not supported to generate both CTF and BTF debug info
in the same compiler run, as both formats made incompatible changes to
the same internal data structures.

With the structural changes to CTF and BTF generation made in:

  d3f586ec50d3 ctf, btf: restructure CTF/BTF emission

in particular, with the guarantee that CTF will always be fully emitted
before any BTF translation occurs, there is no longer anything
preventing generation of both CTF and BTF at the same time.

This patch changes option parsing to lift the restriction on specifying
both -gbtf and -gctf at the same time, allowing for any combination of
-gdwarf, -gctf, and -gbtf to be active in the same compiler invocation.

Bootstrapped and tested on x86_64-linux-gnu.
Also tested on x86_64-linux-gnu for bpf-unknown-none.

gcc/
* opts.cc (set_debug_level): Allow any combination of -gdwarf,
-gctf and -gbtf to be enabled at the same time.

gcc/testsuite/
* gcc.dg/debug/btf/btf-3.c: New test.
* gcc.dg/debug/btf/btf-4.c: Likewise.
* gcc.dg/debug/btf/btf-5.c: Likewise.
---
 gcc/opts.cc| 20 +---
 gcc/testsuite/gcc.dg/debug/btf/btf-3.c |  8 
 gcc/testsuite/gcc.dg/debug/btf/btf-4.c |  8 
 gcc/testsuite/gcc.dg/debug/btf/btf-5.c |  9 +
 4 files changed, 30 insertions(+), 15 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-3.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-4.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-5.c

diff --git a/gcc/opts.cc b/gcc/opts.cc
index d7e0126e11f8..735d0dd8accf 100644
--- a/gcc/opts.cc
+++ b/gcc/opts.cc
@@ -3508,21 +3508,11 @@ set_debug_level (uint32_t dinfo, int extended, const 
char *arg,
 }
   else
 {
-  /* Make and retain the choice if both CTF and DWARF debug info are to
-be generated.  */
-  if (((dinfo == DWARF2_DEBUG) || (dinfo == CTF_DEBUG))
- && ((opts->x_write_symbols == (DWARF2_DEBUG|CTF_DEBUG))
- || (opts->x_write_symbols == DWARF2_DEBUG)
- || (opts->x_write_symbols == CTF_DEBUG)))
-   {
- opts->x_write_symbols |= dinfo;
- opts_set->x_write_symbols |= dinfo;
-   }
-  /* However, CTF and BTF are not allowed together at this time.  */
-  else if (((dinfo == DWARF2_DEBUG) || (dinfo == BTF_DEBUG))
-  && ((opts->x_write_symbols == (DWARF2_DEBUG|BTF_DEBUG))
-  || (opts->x_write_symbols == DWARF2_DEBUG)
-  || (opts->x_write_symbols == BTF_DEBUG)))
+  /* Any combination of DWARF, CTF and BTF is allowed.  */
+  if (((dinfo == DWARF2_DEBUG) || (dinfo == CTF_DEBUG)
+  || (dinfo == BTF_DEBUG))
+ && ((opts->x_write_symbols | (DWARF2_DEBUG | CTF_DEBUG | BTF_DEBUG))
+  == (DWARF2_DEBUG | CTF_DEBUG | BTF_DEBUG)))
{
  opts->x_write_symbols |= dinfo;
  opts_set->x_write_symbols |= dinfo;
diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-3.c 
b/gcc/testsuite/gcc.dg/debug/btf/btf-3.c
new file mode 100644
index ..93c8164a2a54
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/debug/btf/btf-3.c
@@ -0,0 +1,8 @@
+/* Verify that BTF debug info can co-exist with DWARF.  */
+/* { dg-do compile } */
+/* { dg-options "-gdwarf -gbtf -dA" } */
+/* { dg-final { scan-assembler "0xeb9f.*btf_magic" } } */
+/* { dg-final { scan-assembler "DWARF version number" } } */
+
+void func (void)
+{ }
diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-4.c 
b/gcc/testsuite/gcc.dg/debug/btf/btf-4.c
new file mode 100644
index ..b087917188bb
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/debug/btf/btf-4.c
@@ -0,0 +1,8 @@
+/* Verify that BTF debug info can co-exist with CTF.  */
+/* { dg-do compile } */
+/* { dg-options "-gctf -gbtf -dA" } */
+/* { dg-final { scan-assembler "0xeb9f.*btf_magic" } } */
+/* { dg-final { scan-assembler "0xdff2.*CTF preamble magic number" } } */
+
+void func (void)
+{ }
diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-5.c 
b/gcc/testsuite/gcc.dg/debug/btf/btf-5.c
new file mode 100644
index ..45267b5fc422
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/debug/btf/btf-5.c
@@ -0,0 +1,9 @@
+/* Verify that BTF, CTF and DWARF can all co-exist happily.  */
+/* { dg-do compile } */
+/* { dg-options "-gctf -gbtf -gdwarf -dA" } */
+/* { dg-final { scan-assembler "0xeb9f.*btf_magic" } } */
+/* { dg-final { scan-assembler "0xdff2.*CTF preamble magic number" } } */
+/* { dg-final { scan-assembler "DWARF version number" } } */
+
+void func (void)
+{ }
-- 
2.43.0

Re: [PATCH] bpf: create modifier for mem operand for xchg and cmpxchg

[David Faust]

On 7/15/24 08:33, Cupertino Miranda wrote:
> Both xchg and cmpxchg instructions, in the pseudo-C dialect, do not
> expect their memory address operand to be surrounded by parentheses.
> For example, it should be output as "w0 =cmpxchg32_32(r8+8,w0,w2)"
> instead of "w0 =cmpxchg32_32((r8+8),w0,w2)".

After some brief searching, looks like these extra parens in [cmp]xchg*
are an unintended consequence of:

 bb5f619a938 bpf: fix printing of memory operands in pseudoc asm dialect

> 
> This patch implements an operand modifier 'M' which marks the
> instruction templates that do not expect the parentheses, and adds it do
> xchg and cmpxchg templates.

One very minor nit below, otherwise LGTM. Please apply.
Thanks for the fix!

> 
> gcc/ChangeLog:
>   * config/bpf/atomic.md (atomic_compare_and_swap,
>   atomic_exchange): Add operand modifier %M to the first
>   operand.
>   * config/bpf/bpf.cc (no_parentheses_mem_operand): Create
>   variable.
>   (bpf_print_operand): Set no_parentheses_mem_operand variable if
>   %M operand is used.
>   (bpf_print_operand_address): Conditionally output parentheses.
> 
> gcc/testsuite/ChangeLog:
>   * gcc.target/bpf/pseudoc-atomic-memaddr-op.c: Add test.
> ---
>  gcc/config/bpf/atomic.md  |  4 +--
>  gcc/config/bpf/bpf.cc | 20 ---
>  .../bpf/pseudoc-atomic-memaddr-op.c   | 36 +++
>  3 files changed, 54 insertions(+), 6 deletions(-)
>  create mode 100644 gcc/testsuite/gcc.target/bpf/pseudoc-atomic-memaddr-op.c
> 
> diff --git a/gcc/config/bpf/atomic.md b/gcc/config/bpf/atomic.md
> index 65bd5f266a1..be4511bb51b 100644
> --- a/gcc/config/bpf/atomic.md
> +++ b/gcc/config/bpf/atomic.md
> @@ -129,7 +129,7 @@
> (set (match_dup 1)
>  (match_operand:AMO 2 "nonmemory_operand" "0"))]
>"bpf_has_v3_atomics"
> -  "{axchg\t%1,%0|%w0 = xchg(%1, %w0)}")
> +  "{axchg\t%1,%0|%w0 = xchg(%M1, %w0)}")
>  
>  ;; The eBPF atomic-compare-and-exchange instruction has the form
>  ;;   acmp [%dst+offset], %src
> @@ -182,4 +182,4 @@
>(match_operand:AMO 3 "register_operand")]   ;; desired
>   UNSPEC_ACMP))]
>"bpf_has_v3_atomics"
> -  "{acmp\t%1,%3|%w0 = cmpxchg(%1, %w0, %w3)}")
> +  "{acmp\t%1,%3|%w0 = cmpxchg(%M1, %w0, %w3)}")
> diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc
> index 687e7ba49c1..a6b6e20731c 100644
> --- a/gcc/config/bpf/bpf.cc
> +++ b/gcc/config/bpf/bpf.cc
> @@ -835,6 +835,11 @@ bpf_print_register (FILE *file, rtx op, int code)
>  }
>  }
>  
> +/* Variable defined to implement 'M' operand modifier for the special cases
> +   where the parentheses should not be printed surrounding a memory address
> +   operand. */
> +static bool no_parentheses_mem_operand;
> +
>  /* Print an instruction operand.  This function is called in the macro
> PRINT_OPERAND defined in bpf.h */
>  
> @@ -847,6 +852,7 @@ bpf_print_operand (FILE *file, rtx op, int code)
>bpf_print_register (file, op, code);
>break;
>  case MEM:
> +  no_parentheses_mem_operand = (code == 'M');
>output_address (GET_MODE (op), XEXP (op, 0));
>break;
>  case CONST_DOUBLE:
> @@ -891,6 +897,9 @@ bpf_print_operand (FILE *file, rtx op, int code)
>  }
>  }
>  
> +#define PAREN_OPEN  (asm_dialect == ASM_NORMAL ? "[" : 
> no_parentheses_mem_operand ? "" : "(")
> +#define PAREN_CLOSE (asm_dialect == ASM_NORMAL ? "]" : 
> no_parentheses_mem_operand ? "" : ")")
> +
>  /* Print an operand which is an address.  This function should handle
> any legit address, as accepted by bpf_legitimate_address_p, and
> also addresses that are valid in CALL instructions.
> @@ -904,9 +913,9 @@ bpf_print_operand_address (FILE *file, rtx addr)
>switch (GET_CODE (addr))
>  {
>  case REG:
> -  fprintf (file, asm_dialect == ASM_NORMAL ? "[" : "(");
> +  fprintf (file, "%s", PAREN_OPEN);
>bpf_print_register (file, addr, 0);
> -  fprintf (file, asm_dialect == ASM_NORMAL ? "+0]" : "+0)");
> +  fprintf (file, "+0%s", PAREN_CLOSE);
>break;
>  case PLUS:
>{
> @@ -923,14 +932,14 @@ bpf_print_operand_address (FILE *file, rtx addr)
>   || (GET_CODE (op1) == UNSPEC
>   && XINT (op1, 1) == UNSPEC_CORE_RELOC)))
> {
> -fprintf (file, asm_dialect == ASM_NORMAL ? "[" : "(");
> + fprintf (file, "%s", PAREN_OPEN);
>   bpf_print_register (file, op0, 0);
>   fprintf (file, "+");
>   if (GET_CODE (op1) == UNSPEC)
> output_addr_const (file, XVECEXP (op1, 0, 0));
>   else
> output_addr_const (file, op1);
> -fprintf (file, asm_dialect == ASM_NORMAL ? "]" : ")");
> + fprintf (file, "%s", PAREN_CLOSE);
> }
>   else
> fatal_insn ("invalid address in operand", addr);
> @@ -948,6 +957,9 @@ bpf_print_operand_address (FILE *file, rtx addr)
>  }
>  }
>  
> +#undef PAREN

[PATCH v3 0/6] btf: refactor and add pruning option

[David Faust]

[v2: https://gcc.gnu.org/pipermail/gcc-patches/2024-May/650482.html
 Changes from v2:
 - Handle -flto when generating BTF (in patch 1).  For LTO builds, BTF
   is emitted at early_finish as before.  For non-LTO builds, BTF will
   be emitted at (late) finish for all targets.
 - Move the option parsing change to allow -gbtf and -gctf together to
   the end of the series.  This patch is relatively separate from the
   other changes, and only depends upon patch 1.
 - Include Indu's various comments and suggestions throughout the
   series.
 - Fix a few GNU style issues with indentation and long lines.
 - Fix the special handling for .maps section variables in BTF in
   patch 4.  Previously the type marking was too aggressive, and undid
   nearly all the pruning in some kernel selftests.  In one test this
   fix reduced the emitted BTF from over 6000 types to about 200 types,
   very similar to clang's output for the same test.  ]

This patch series signficantly refactors the BTF generation in gcc,
making it simpler and easier to understand, extend and maintain.

It also introduces an optional algorithm to "prune" BTF information
before emission.  This pruning is meant to be used for BPF programs, to
alleviate the massive bloating of BTF information caused by including
Linux kernel internal headers.  The pruning is designed to be compatible
with the unconditional pruning performed  by the LLVM BPF backend when
generating BTF information.

While the changes are fairly significant, there is no actual change in
emitted BTF information (unless pruning is enabled), other than bug
fixes and small additions to the assembler debug comments.

Patch 1 restructures the emission of CTF and BTF information, with the
result that CTF is always completely generated and emitted before any
BTF-related procedures are run.  BTF emission is moved to late finish
for all targets, except when building with -flto.

Patch 2 changes the data structures shared by CTF and BTF to use
pointers rather than type IDs for all inter-type references.  This
change is completely transparent to both CTF and BTF.

Patch 3 heavily refactors btfout.cc to take advantage of the prior
changes and significantly simplify the BTF implementation.  The changes
are nearly transparent, however some small but important improvements
are also made possible by the refactor, such as fixing PR113566 for
non-LTO builds.

Patch 4 adds a new option to perform pruning of the BTF information
before emission.  This is intended to be used for BPF programs which
often include kernel headers, and in many cases reduces the size of
the resulting BTF information by a factor of 10.

Patch 5 makes BTF pruning work with BPF CO-RE, and enables the pruning
by default in the BPF backend.

Patch 6 takes advantage of the prior changes, and removes the
restriction on generating both CTF and BTF in the same compiler run,
allowing for any combinaion of -gdwarf, -gctf and -gbtf.

Tested on x86_64-linux-gnu, and on x86_64-linux-gnu host for
bpf-unknown-none target.

Also tested by compiling and runninng Linux kernel BPF selftests.
No known regressions.

David Faust (6):
  ctf, btf: restructure CTF/BTF emission
  ctf: use pointers instead of IDs internally
  btf: refactor and simplify implementation
  btf: add -fprune-btf option
  bpf,btf: enable BTF pruning by default for BPF
  opts: allow any combination of DWARF, CTF, BTF

 gcc/btfout.cc | 1613 +
 gcc/common.opt|4 +
 gcc/config/bpf/bpf.cc |5 +
 gcc/config/bpf/btfext-out.cc  |   14 +-
 gcc/config/bpf/core-builtins.cc   |   74 +-
 gcc/ctfc.cc   |  141 +-
 gcc/ctfc.h|  113 +-
 gcc/ctfout.cc |   22 +-
 gcc/doc/invoke.texi   |   23 +
 gcc/dwarf2ctf.cc  |  311 ++--
 gcc/dwarf2ctf.h   |2 +-
 gcc/dwarf2out.cc  |4 +-
 gcc/opts.cc   |   20 +-
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c  |   25 +
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c  |   33 +
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c  |   35 +
 .../gcc.dg/debug/btf/btf-prune-maps.c |   20 +
 .../gcc.dg/debug/btf/btf-variables-5.c|6 +-
 include/btf.h |5 +
 19 files changed, 1420 insertions(+), 1050 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-maps.c

-- 
2.43.0

[PATCH v3 1/6] ctf, btf: restructure CTF/BTF emission

[David Faust]

This commit makes some structural changes to the CTF/BTF debug info
emission.  In particular:

 a) CTF is new always fully generated and emitted before any
BTF-related procedures are run.  This means that BTF-related
functions can change, even irreversibly, the shared in-memory
representation used by the two formats without issue.

 b) BTF generation has fewer entry points, and is cleanly divided
into early_finish and finish.

 c) BTF is now always emitted at finish (called from dwarf2out_finish),
for all targets in non-LTO builds, rather than being emitted at
early_finish for targets other than BPF CO-RE.  In LTO builds,
BTF is emitted at early_finish as before.

Note that this change alone does not alter the contents of BTF at
all, regardless of whether it would have previously been emitted at
early_finish or finish, because the calculation of the BTF to be
emitted is not moved by this patch, only the write-out.

The changes are transparent to both CTF and BTF emission.

gcc/
* btfout.cc (btf_init_postprocess): Rename to...
(btf_early_finish): ...this.
(btf_output): Rename to...
(btf_finish): ...this.
* ctfc.h: Analogous changes.
* dwarf2ctf.cc (ctf_debug_early_finish): Conditionally call
btf_early_finish, or ctf_finalize as appropriate.  Emit BTF
here for LTO builds.
(ctf_debug_finish): Always call btf_finish here if generating
BTF info in non-LTO builds.
(ctf_debug_finalize, ctf_debug_init_postprocess): Delete.
* dwarf2out.cc (dwarf2out_early_finish): Remove call to
ctf_debug_init_postprocess.
---
 gcc/btfout.cc| 28 +
 gcc/ctfc.h   |  4 +--
 gcc/dwarf2ctf.cc | 65 +++-
 gcc/dwarf2out.cc |  2 --
 4 files changed, 50 insertions(+), 49 deletions(-)

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index 07f066a47068..1b6a9ed811f0 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -1491,6 +1491,34 @@ btf_finalize (void)
   tu_ctfc = NULL;
 }
 
+/* Initial entry point of BTF generation, called at early_finish () after
+   CTF information has possibly been output.  Translate all CTF information
+   to BTF, and do any processing that must be done early, such as creating
+   BTF_KIND_FUNC records.  */
+
+void
+btf_early_finish (void)
+{
+  btf_init_postprocess ();
+}
+
+/* Late entry point for BTF generation, called from dwarf2out_finish ().
+   Complete and emit BTF information.  */
+
+void
+btf_finish (const char * filename)
+{
+  btf_output (filename);
+
+  /* If compiling for BPF with CO-RE info, we cannot deallocate until after
+ CO-RE information is created, which happens very late in BPF backend.
+ Therefore, the deallocation (i.e. btf_finalize ()) is delayed until
+ TARGET_ASM_FILE_END for BPF CO-RE.  */
+  if (!btf_with_core_debuginfo_p ())
+btf_finalize ();
+}
+
+
 /* Traversal function for all BTF_KIND_FUNC type records.  */
 
 bool
diff --git a/gcc/ctfc.h b/gcc/ctfc.h
index fa188bf2f5a4..e7bd93901cfa 100644
--- a/gcc/ctfc.h
+++ b/gcc/ctfc.h
@@ -384,8 +384,8 @@ extern void ctf_init (void);
 extern void ctf_output (const char * filename);
 extern void ctf_finalize (void);
 
-extern void btf_output (const char * filename);
-extern void btf_init_postprocess (void);
+extern void btf_early_finish (void);
+extern void btf_finish (const char * filename);
 extern void btf_finalize (void);
 
 extern ctf_container_ref ctf_get_tu_ctfc (void);
diff --git a/gcc/dwarf2ctf.cc b/gcc/dwarf2ctf.cc
index dc59569fe560..8f9e2fada9e3 100644
--- a/gcc/dwarf2ctf.cc
+++ b/gcc/dwarf2ctf.cc
@@ -933,30 +933,6 @@ gen_ctf_type (ctf_container_ref ctfc, dw_die_ref die)
   return type_id;
 }
 
-/* Prepare for output and write out the CTF debug information.  */
-
-static void
-ctf_debug_finalize (const char *filename, bool btf)
-{
-  if (btf)
-{
-  btf_output (filename);
-  /* btf_finalize when compiling BPF applciations gets deallocated by the
-BPF target in bpf_file_end.  */
-  if (btf_debuginfo_p () && !btf_with_core_debuginfo_p ())
-   btf_finalize ();
-}
-
-  else
-{
-  /* Emit the collected CTF information.  */
-  ctf_output (filename);
-
-  /* Reset the CTF state.  */
-  ctf_finalize ();
-}
-}
-
 bool
 ctf_do_die (dw_die_ref die)
 {
@@ -996,27 +972,27 @@ ctf_debug_init (void)
   add_name_attribute (ctf_unknown_die, "unknown");
 }
 
-/* Preprocess the CTF debug information after initialization.  */
-
-void
-ctf_debug_init_postprocess (bool btf)
-{
-  /* Only BTF requires postprocessing right after init.  */
-  if (btf)
-btf_init_postprocess ();
-}
-
 /* Early finish CTF/BTF debug info.  */
 
 void
 ctf_debug_early_finish (const char * filename)
 {
-  /* Emit CTF debug info early always.  */
-  if (ctf_debug_info_level > CTFINFO_LEVEL_NONE
-  /* Emit BTF debug info early if CO-RE relocations are not
-required.  */
-  || (btf_debu

[PATCH v3 2/6] ctf: use pointers instead of IDs internally

[David Faust]

This patch replaces all inter-type references in the ctfc internal data
structures with pointers, rather than the references-by-ID which were
used previously.

A couple of small updates in the BPF backend are included to make it
compatible with the change.

This change is only to the in-memory representation of various CTF
structures to make them easier to work with in various cases.  It is
outwardly transparent; there is no change in emitted CTF.

gcc/
* btfout.cc (BTF_VOID_TYPEID, BTF_INIT_TYPEID): Move defines to
include/btf.h.
(btf_dvd_emit_preprocess_cb, btf_emit_preprocess)
(btf_dmd_representable_bitfield_p, btf_asm_array, btf_asm_varent)
(btf_asm_sou_member, btf_asm_func_arg, btf_init_postprocess):
Adapt to structural changes in ctf_* structs.
* ctfc.h (struct ctf_dtdef): Add forward declaration.
(ctf_dtdef_t, ctf_dtdef_ref): Move typedefs earlier.
(struct ctf_arinfo, struct ctf_funcinfo, struct ctf_sliceinfo)
(struct ctf_itype, struct ctf_dmdef, struct ctf_func_arg)
(struct ctf_dvdef): Use pointers instead of type IDs for
references to other types and use typedefs where appropriate.
(struct ctf_dtdef): Add ref_type member.
(ctf_type_exists): Use pointer instead of type ID.
(ctf_add_reftype, ctf_add_enum, ctf_add_slice, ctf_add_float)
(ctf_add_integer, ctf_add_unknown, ctf_add_pointer)
(ctf_add_array, ctf_add_forward, ctf_add_typedef)
(ctf_add_function, ctf_add_sou, ctf_add_enumerator)
(ctf_add_variable): Likewise. Return pointer instead of ID.
(ctf_lookup_tree_type): Return pointer to type instead of ID.
* ctfc.cc: Analogous changes.
* ctfout.cc (ctf_asm_type, ctf_asm_slice, ctf_asm_varent)
(ctf_asm_sou_lmember, ctf_asm_sou_member, ctf_asm_func_arg)
(output_ctf_objt_info): Adapt to changes.
* dwarf2ctf.cc (gen_ctf_type, gen_ctf_void_type)
(gen_ctf_unknown_type, gen_ctf_base_type, gen_ctf_pointer_type)
(gen_ctf_subrange_type, gen_ctf_array_type, gen_ctf_typedef)
(gen_ctf_modifier_type, gen_ctf_sou_type, gen_ctf_function_type)
(gen_ctf_enumeration_type, gen_ctf_variable, gen_ctf_function)
(gen_ctf_type, ctf_do_die): Likewise.
* config/bpf/btfext-out.cc (struct btf_ext_core_reloc): Use
pointer instead of type ID.
(bpf_core_reloc_add, bpf_core_get_sou_member_index)
(output_btfext_core_sections): Adapt to above changes.
* config/bpf/core-builtins.cc (process_type): Likewise.

include/
* btf.h (BTF_VOID_TYPEID, BTF_INIT_TYPEID): Move defines here,
from gcc/btfout.cc.
---
 gcc/btfout.cc   |  40 +++---
 gcc/config/bpf/btfext-out.cc|  14 +-
 gcc/config/bpf/core-builtins.cc |   3 +-
 gcc/ctfc.cc | 139 +-
 gcc/ctfc.h  |  90 ++--
 gcc/ctfout.cc   |  22 +--
 gcc/dwarf2ctf.cc| 244 +++-
 include/btf.h   |   5 +
 8 files changed, 278 insertions(+), 279 deletions(-)

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index 1b6a9ed811f0..40e8d8c5c01b 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -61,11 +61,6 @@ static char btf_info_section_label[MAX_BTF_LABEL_BYTES];
 #define BTF_INFO_SECTION_LABEL  "Lbtf"
 #endif
 
-/* BTF encodes void as type id 0.  */
-
-#define BTF_VOID_TYPEID 0
-#define BTF_INIT_TYPEID 1
-
 #define BTF_INVALID_TYPEID 0x
 
 /* Mapping of CTF variables to the IDs they will be assigned when they are
@@ -626,7 +621,8 @@ btf_dvd_emit_preprocess_cb (ctf_dvdef_ref *slot, 
ctf_container_ref arg_ctfc)
 return 1;
 
   /* Do not add variables which refer to unsupported types.  */
-  if (!voids.contains (var->dvd_type) && btf_removed_type_p (var->dvd_type))
+  if (!voids.contains (var->dvd_type->dtd_type)
+  && btf_removed_type_p (var->dvd_type->dtd_type))
 return 1;
 
   arg_ctfc->ctfc_vars_list[num_vars_added] = var;
@@ -716,7 +712,7 @@ btf_emit_preprocess (ctf_container_ref ctfc)
 static bool
 btf_dmd_representable_bitfield_p (ctf_container_ref ctfc, ctf_dmdef_t *dmd)
 {
-  ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[dmd->dmd_type];
+  ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[dmd->dmd_type->dtd_type];
 
   if (CTF_V2_INFO_KIND (ref_type->dtd_data.ctti_info) == CTF_K_SLICE)
 {
@@ -913,8 +909,8 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd)
 static void
 btf_asm_array (ctf_container_ref ctfc, ctf_arinfo_t arr)
 {
-  btf_asm_type_ref ("bta_elem_type", ctfc, arr.ctr_contents);
-  btf_asm_type_ref ("bta_index_type", ctfc, arr.ctr_index);
+  btf_asm_type_ref ("bta_elem_type", ctfc, arr.ctr_contents->dtd_type);
+  btf_asm_type_ref ("bta_index_type", ctfc, arr.ctr_index->dtd_type);
   dw2_asm_output_data (4, arr.ctr_nelems, "bta_nelems");
 }
 
@@ -927,7 +923,7 @@ btf_asm_varent (ctf_container_ref ctfc, ctf_dvdef_ref var)
 

[PATCH v3 4/6] btf: add -fprune-btf option

[David Faust]

This patch adds a new option, -fprune-btf, to control BTF debug info
generation.

As the name implies, this option enables a kind of "pruning" of the BTF
information before it is emitted.  When enabled, rather than emitting
all type information translated from DWARF, only information for types
directly used in the source program is emitted.

The primary purpose of this pruning is to reduce the amount of
unnecessary BTF information emitted, especially for BPF programs.  It is
very common for BPF programs to incldue Linux kernel internal headers in
order to have access to kernel data structures.  However, doing so often
has the side effect of also adding type definitions for a large number
of types which are not actually used by nor relevant to the program.
In these cases, -fprune-btf commonly reduces the size of the resulting
BTF information by 10x or more, as seen on average when compiling Linux
kernel BPF selftests.  This both slims down the size of the resulting
object and reduces the time required by the BPF loader to verify the
program and its BTF information.

Note that the pruning implemented in this patch follows the same rules
as the BTF pruning performed unconditionally by LLVM's BPF backend when
generating BTF.  In particular, the main sources of pruning are:

  1) Only generate BTF for types used by variables and functions at
 the file scope.  Note that with or without pruning, BTF_KIND_VAR
 entries are only generated for variables present in the final
 object - unused static variables or variables completely optimized
 away must not have VAR entries in BTF.

  2) Avoid emitting full BTF for struct and union types which are only
 pointed-to by members of other struct/union types.  In these cases,
 the full BTF_KIND_STRUCT or BTF_KIND_UNION which would normally
 be emitted is replaced with a BTF_KIND_FWD, as though the
 underlying type was a forward-declared struct or union type.

gcc/
* btfout.cc (btf_used_types): New hash set.
(struct btf_fixup): New.
(fixups, forwards): New vecs.
(btf_output): Calculate num_types depending on flag_prune_btf.
(btf_early_finsih): New initialization for flag_prune_btf.
(btf_add_used_type): New function.
(btf_used_type_list_cb): Likewise.
(btf_late_collect_pruned_types): Likewise.
(btf_late_add_vars): Handle special case for variables in ".maps"
section when generating BTF for BPF CO-RE target.
(btf_late_finish): Use btf_late_collect_pruned_types when
flag_prune_btf in effect.  Move some initialization to btf_early_finish.
(btf_finalize): Additional deallocation for flag_prune_btf.
* common.opt (fprune-btf): New flag.
* ctfc.cc (init_ctf_strtable): Make non-static.
* ctfc.h (struct ctf_dtdef): Add visited_children_p boolean flag.
(init_ctf_strtable, ctfc_delete_strtab): Make extern.
* doc/invoke.texi (Debugging Options): Document -fprune-btf.

gcc/testsuite/
* gcc.dg/debug/btf/btf-prune-1.c: New test.
* gcc.dg/debug/btf/btf-prune-2.c: Likewise.
* gcc.dg/debug/btf/btf-prune-3.c: Likewise.
* gcc.dg/debug/btf/btf-prune-maps.c: Likewise.
---
 gcc/btfout.cc | 359 +-
 gcc/common.opt|   4 +
 gcc/ctfc.cc   |   2 +-
 gcc/ctfc.h|   3 +
 gcc/doc/invoke.texi   |  20 +
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c  |  25 ++
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c  |  33 ++
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c  |  35 ++
 .../gcc.dg/debug/btf/btf-prune-maps.c |  20 +
 9 files changed, 494 insertions(+), 7 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-maps.c

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index 32fda14f704b..a7da164f6b31 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -828,7 +828,10 @@ output_btf_types (ctf_container_ref ctfc)
 {
   size_t i;
   size_t num_types;
-  num_types = ctfc->ctfc_types->elements ();
+  if (flag_prune_btf)
+num_types = max_translated_id;
+  else
+num_types = ctfc->ctfc_types->elements ();
 
   if (num_types)
 {
@@ -957,6 +960,212 @@ btf_early_add_func_records (ctf_container_ref ctfc)
 }
 }
 
+/* The set of types used directly in the source program, and any types manually
+   marked as used.  This is the set of types which will be emitted when
+   flag_prune_btf is set.  */
+static GTY (()) hash_set *btf_used_types;
+
+/* Fixup used to avoid unnecessary pointer chasing for types.  A fixup is
+   created when a structure or union member is a pointer to another struct
+   or union type.  In such cases, avoid emitt

[PATCH v3 6/6] opts: allow any combination of DWARF, CTF, BTF

[David Faust]

Previously it was not supported to generate both CTF and BTF debug info
in the same compiler run, as both formats made incompatible changes to
the same internal data structures.

With the structural change in the prior patches, in particular the
guarantee that CTF will always be fully emitted before any BTF
translation occurs, there is no longer anything preventing generation
of both CTF and BTF at the same time.  This patch changes option parsing
to allow any combination of -gdwarf, -gctf, and -gbtf at the same time.

gcc/
* opts.cc (set_debug_level): Allow any combination of -gdwarf,
-gctf and -gbtf at the same time.
---
 gcc/opts.cc | 20 +++-
 1 file changed, 7 insertions(+), 13 deletions(-)

diff --git a/gcc/opts.cc b/gcc/opts.cc
index f80d5d4ba8f9..d58bea096a5f 100644
--- a/gcc/opts.cc
+++ b/gcc/opts.cc
@@ -3505,21 +3505,15 @@ set_debug_level (uint32_t dinfo, int extended, const 
char *arg,
 }
   else
 {
-  /* Make and retain the choice if both CTF and DWARF debug info are to
-be generated.  */
-  if (((dinfo == DWARF2_DEBUG) || (dinfo == CTF_DEBUG))
+  /* Any combination of DWARF, CTF and BTF is allowed together.  */
+  if (((dinfo == DWARF2_DEBUG) || (dinfo == CTF_DEBUG)
+  || (dinfo == BTF_DEBUG))
  && ((opts->x_write_symbols == (DWARF2_DEBUG|CTF_DEBUG))
+ || (opts->x_write_symbols == (DWARF2_DEBUG|BTF_DEBUG))
+ || (opts->x_write_symbols == (CTF_DEBUG|BTF_DEBUG))
  || (opts->x_write_symbols == DWARF2_DEBUG)
- || (opts->x_write_symbols == CTF_DEBUG)))
-   {
- opts->x_write_symbols |= dinfo;
- opts_set->x_write_symbols |= dinfo;
-   }
-  /* However, CTF and BTF are not allowed together at this time.  */
-  else if (((dinfo == DWARF2_DEBUG) || (dinfo == BTF_DEBUG))
-  && ((opts->x_write_symbols == (DWARF2_DEBUG|BTF_DEBUG))
-  || (opts->x_write_symbols == DWARF2_DEBUG)
-  || (opts->x_write_symbols == BTF_DEBUG)))
+ || (opts->x_write_symbols == CTF_DEBUG)
+ || (opts->x_write_symbols == BTF_DEBUG)))
{
  opts->x_write_symbols |= dinfo;
  opts_set->x_write_symbols |= dinfo;
-- 
2.43.0

[PATCH v3 5/6] bpf,btf: enable BTF pruning by default for BPF

[David Faust]

This patch enables -fprune-btf by default in the BPF backend when
generating BTF information, and fixes BPF CO-RE generation when using
-fprune-btf.

When generating BPF CO-RE information, we must ensure that types used
in CO-RE relocations always have sufficient BTF information emited so
that the CO-RE relocations can be processed by a BPF loader.  The BTF
pruning algorithm on its own does not have sufficient information to
determine which types are used in a BPF CO-RE relocation, so this
information must be supplied by the BPF backend, using a new
btf_mark_type_used function.

Co-authored-by: Cupertino Miranda 

gcc/
* btfout.cc (btf_mark_type_used): New.
* ctfc.h (btf_mark_type_used): Declare it here.
* config/bpf/bpf.cc (bpf_option_override): Enable -fprune-btf
by default if -gbtf is enabled.
* config/bpf/core-builtins.cc (extra_fn): New typedef.
(compute_field_expr): Add callback parameter, and call it if supplied.
Fix computation for MEM_REF.
(mark_component_type_as_used): New.
(bpf_mark_types_as_used): Likewise.
(bpf_expand_core_builtin): Call here.
* doc/invoke.texi (Debugging Options): Note that -fprune-btf is
enabled by default for BPF target when generating BTF.

gcc/testsuite/
* gcc.dg/debug/btf/btf-variables-5.c: Add -fno-prune-btf to dg-options.
---
 gcc/btfout.cc | 22 ++
 gcc/config/bpf/bpf.cc |  5 ++
 gcc/config/bpf/core-builtins.cc   | 71 +--
 gcc/ctfc.h|  1 +
 gcc/doc/invoke.texi   |  3 +
 .../gcc.dg/debug/btf/btf-variables-5.c|  6 +-
 6 files changed, 100 insertions(+), 8 deletions(-)

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index a7da164f6b31..35d2875e3f61 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -1501,6 +1501,28 @@ btf_late_assign_datasec_ids (ctf_container_ref ctfc)
 }
 }
 
+
+/* Manually mark that type T is used to ensure it will not be pruned.
+   Used by the BPF backend when generating BPF CO-RE to mark types used
+   in CO-RE relocations.  */
+
+void
+btf_mark_type_used (tree t)
+{
+  /* If we are not going to prune anyway, this is a no-op.  */
+  if (!flag_prune_btf)
+return;
+
+  gcc_assert (TYPE_P (t));
+  ctf_container_ref ctfc = ctf_get_tu_ctfc ();
+  ctf_dtdef_ref dtd = ctf_lookup_tree_type (ctfc, t);
+
+  if (!dtd)
+return;
+
+  btf_add_used_type (ctfc, dtd, false, false, true);
+}
+
 /* Callback used for assembling the only-used-types list.  Note that this is
the same as btf_type_list_cb above, but the hash_set traverse requires a
different function signature.  */
diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc
index dd1bfe38d29b..775730700eba 100644
--- a/gcc/config/bpf/bpf.cc
+++ b/gcc/config/bpf/bpf.cc
@@ -221,6 +221,11 @@ bpf_option_override (void)
   && !(target_flags_explicit & MASK_BPF_CORE))
 target_flags |= MASK_BPF_CORE;
 
+  /* -gbtf implies -fprune-btf for BPF target.  */
+  if (btf_debuginfo_p ())
+SET_OPTION_IF_UNSET (&global_options, &global_options_set,
+flag_prune_btf, true);
+
   /* Determine available features from ISA setting (-mcpu=).  */
   if (bpf_has_jmpext == -1)
 bpf_has_jmpext = (bpf_isa >= ISA_V2);
diff --git a/gcc/config/bpf/core-builtins.cc b/gcc/config/bpf/core-builtins.cc
index 232bebcadbd5..86e2e9d6e39f 100644
--- a/gcc/config/bpf/core-builtins.cc
+++ b/gcc/config/bpf/core-builtins.cc
@@ -624,13 +624,20 @@ bpf_core_get_index (const tree node, bool *valid)
 
ALLOW_ENTRY_CAST is an input arguments and specifies if the function should
consider as valid expressions in which NODE entry is a cast expression (or
-   tree code nop_expr).  */
+   tree code nop_expr).
+
+   EXTRA_FN is a callback function to allow extra functionality with this
+   function traversal.  Currently used for marking used type during expand
+   pass.  */
+
+typedef void (*extra_fn) (tree);
 
 static unsigned char
 compute_field_expr (tree node, unsigned int *accessors,
bool *valid,
tree *access_node,
-   bool allow_entry_cast = true)
+   bool allow_entry_cast = true,
+   extra_fn callback = NULL)
 {
   unsigned char n = 0;
   unsigned int fake_accessors[MAX_NR_ACCESSORS];
@@ -647,6 +654,9 @@ compute_field_expr (tree node, unsigned int *accessors,
 
   *access_node = node;
 
+  if (callback != NULL)
+callback (node);
+
   switch (TREE_CODE (node))
 {
 case INDIRECT_REF:
@@ -664,17 +674,19 @@ compute_field_expr (tree node, unsigned int *accessors,
 case COMPONENT_REF:
   n = compute_field_expr (TREE_OPERAND (node, 0), accessors,
  valid,
- access_node, false);
+ access_node, false, callback);
   accessors[n] = bpf_core_get_index (TREE_OPE

[PATCH v3 3/6] btf: refactor and simplify implementation

[David Faust]

This patch heavily refactors btfout.cc to take advantage of the
structural changes in the prior commits.

Now that inter-type references are internally stored as simply pointers,
all the painful, brittle, confusing infrastructure that was used in the
process of converting CTF type IDs to BTF type IDs can be thrown out.
This greatly simplifies the entire process of converting from CTF to
BTF, making the code cleaner, easier to read, and easier to maintain.

In addition, we no longer need to worry about destructive changes in
internal data structures used commonly by CTF and BTF, which allows
deleting several ancillary data structures previously used in btfout.cc.

This is nearly transparent, but a few improvements have also been made:

 1) BTF_KIND_FUNC records are now _always_ constructed at early_finish,
allowing us to construct records even for functions which are later
inlined by optimizations. DATASEC entries for functions are only
constructed at late_finish, to avoid incorrectly generating entries
for functions which get inlined.

 2) BTF_KIND_VAR records and DATASEC entries for them are now always
constructed at (late) finish, which avoids cases where we could
incorrectly create records for variables which were completely
optimized away.  This fixes PR debug/113566 for non-LTO builds.
In LTO builds, BTF must be emitted at early_finish, so some VAR
records may be emitted for variables which are later optimized away.

 3) Some additional assembler comments have been added with more
information for debugging.

gcc/
* btfout.cc (struct btf_datasec_entry): New.
(struct btf_datasec): Add `id' member.  Change `entries' to use
new struct btf_datasec_entry.
(func_map): New hash_map.
(max_translated_id): New.
(btf_var_ids, btf_id_map, holes, voids, num_vars_added)
(num_types_added, num_types_created): Delete.
(btf_absolute_var_id, btf_relative_var_id, btf_absolute_func_id)
(btf_relative_func_id, btf_absolute_datasec_id, init_btf_id_map)
(get_btf_id, set_btf_id, btf_emit_id_p): Delete.
(btf_removed_type_p): Delete.
(btf_dtd_kind, btf_emit_type_p): New helpers.
(btf_fwd_to_enum_p, btf_calc_num_vbytes): Use them.
(btf_collect_datasec): Delete.
(btf_dtd_postprocess_cb, btf_dvd_emit_preprocess_cb)
(btf_dtd_emit_preprocess_cb, btf_emit_preprocess): Delete.
(btf_dmd_representable_bitfield_p): Adapt to type reference changes
and delete now-unused ctfc argument.
(btf_asm_datasec_type_ref): Delete.
(btf_asm_type_ref): Adapt to type reference changes, simplify.
(btf_asm_type): Likewise. Mark struct/union types with bitfield
members.
(btf_asm_array): Adapt to data structure changes.
(btf_asm_varent): Likewise.
(btf_asm_sou_member): Likewise. Ensure non-bitfield members are
correctly re-encoded if struct or union contains any bitfield.
(btf_asm_func_arg, btf_asm_func_type, btf_asm_datasec_entry)
(btf_asm_datasec_type): Adapt to data structure changes.
(output_btf_header): Adapt to other changes, simplify type
length calculation, add info to assembler comments.
(output_btf_vars): Adapt to other changes.
(output_btf_strs): Fix overlong lines.
(output_asm_btf_sou_fields, output_asm_btf_enum_list)
(output_asm_btf_func_args_list, output_asm_btf_vlen_bytes)
(output_asm_btf_type, output_btf_types, output_btf_func_types)
(output_btf_datasec_types): Adapt to other changes.
(btf_init_postprocess): Delete.
(btf_output): Change to only perform output.
(btf_early_add_const_void, btf_early_add_func_records): New.
(btf_early_finish): Use them here. New.
(btf_datasec_push_entry): Adapt to data structure changes.
(btf_datasec_add_func, btf_datasec_add_var): New.
(btf_late_add_func_datasec_entries): New.
(btf_emit_variable_p): New helper.
(btf_late_add_vars): Use it here. New.
(btf_type_list_cb, btf_late_collect_translated_types): New.
(btf_late_assign_func_ids, btf_late_assign_var_ids)
(btf_late_assign_datasec_ids): New.
(btf_finish): Remove unused argument. Call new btf_late*
functions and btf_output.
(btf_finalize): Adapt to data structure changes.
* ctfc.h (struct ctf_dtdef): Convert existing boolean flags to
BOOL_BITFIELD and reorder.
(struct ctf_dvdef): Add dvd_id member.
(btf_finish): Remove argument from prototype.
(get_btf_id): Delete prototype.
(funcs_traverse_callback, traverse_btf_func_types): Add an
explanatory comment.
* dwarf2ctf.cc (ctf_debug_finish): Remove unused argument.
* dwarf2ctf.h: Analogous change.
* dwarf2out.cc: Likewise.
---
 gcc/btfout.cc| 1254 +++---

Re: [PATCH v3 4/6] btf: add -fprune-btf option

[David Faust]

On 5/31/24 00:07, Richard Biener wrote:
> On Thu, May 30, 2024 at 11:34 PM David Faust  wrote:
>>
>> This patch adds a new option, -fprune-btf, to control BTF debug info
>> generation.
> 
> Can you name it -gprune-btf instead?

Yes, sure.

I think I followed -feliminate-unused-debug-types, but I see the large
majority of options controlling e.g. DWARF use -g instead.

Happy to change it to -gprune-btf.

> 
>> As the name implies, this option enables a kind of "pruning" of the BTF
>> information before it is emitted.  When enabled, rather than emitting
>> all type information translated from DWARF, only information for types
>> directly used in the source program is emitted.
>>
>> The primary purpose of this pruning is to reduce the amount of
>> unnecessary BTF information emitted, especially for BPF programs.  It is
>> very common for BPF programs to incldue Linux kernel internal headers in
>> order to have access to kernel data structures.  However, doing so often
>> has the side effect of also adding type definitions for a large number
>> of types which are not actually used by nor relevant to the program.
>> In these cases, -fprune-btf commonly reduces the size of the resulting
>> BTF information by 10x or more, as seen on average when compiling Linux
>> kernel BPF selftests.  This both slims down the size of the resulting
>> object and reduces the time required by the BPF loader to verify the
>> program and its BTF information.
>>
>> Note that the pruning implemented in this patch follows the same rules
>> as the BTF pruning performed unconditionally by LLVM's BPF backend when
>> generating BTF.  In particular, the main sources of pruning are:
>>
>>   1) Only generate BTF for types used by variables and functions at
>>  the file scope.  Note that with or without pruning, BTF_KIND_VAR
>>  entries are only generated for variables present in the final
>>  object - unused static variables or variables completely optimized
>>  away must not have VAR entries in BTF.
>>
>>   2) Avoid emitting full BTF for struct and union types which are only
>>  pointed-to by members of other struct/union types.  In these cases,
>>  the full BTF_KIND_STRUCT or BTF_KIND_UNION which would normally
>>  be emitted is replaced with a BTF_KIND_FWD, as though the
>>  underlying type was a forward-declared struct or union type.
>>
>> gcc/
>> * btfout.cc (btf_used_types): New hash set.
>> (struct btf_fixup): New.
>> (fixups, forwards): New vecs.
>> (btf_output): Calculate num_types depending on flag_prune_btf.
>> (btf_early_finsih): New initialization for flag_prune_btf.
>> (btf_add_used_type): New function.
>> (btf_used_type_list_cb): Likewise.
>> (btf_late_collect_pruned_types): Likewise.
>> (btf_late_add_vars): Handle special case for variables in ".maps"
>> section when generating BTF for BPF CO-RE target.
>> (btf_late_finish): Use btf_late_collect_pruned_types when
>> flag_prune_btf in effect.  Move some initialization to 
>> btf_early_finish.
>> (btf_finalize): Additional deallocation for flag_prune_btf.
>> * common.opt (fprune-btf): New flag.
>> * ctfc.cc (init_ctf_strtable): Make non-static.
>> * ctfc.h (struct ctf_dtdef): Add visited_children_p boolean flag.
>> (init_ctf_strtable, ctfc_delete_strtab): Make extern.
>> * doc/invoke.texi (Debugging Options): Document -fprune-btf.
>>
>> gcc/testsuite/
>> * gcc.dg/debug/btf/btf-prune-1.c: New test.
>> * gcc.dg/debug/btf/btf-prune-2.c: Likewise.
>> * gcc.dg/debug/btf/btf-prune-3.c: Likewise.
>> * gcc.dg/debug/btf/btf-prune-maps.c: Likewise.
>> ---
>>  gcc/btfout.cc | 359 +-
>>  gcc/common.opt|   4 +
>>  gcc/ctfc.cc   |   2 +-
>>  gcc/ctfc.h|   3 +
>>  gcc/doc/invoke.texi   |  20 +
>>  gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c  |  25 ++
>>  gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c  |  33 ++
>>  gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c  |  35 ++
>>  .../gcc.dg/debug/btf/btf-prune-maps.c |  20 +
>>  9 files changed, 494 insertions(+), 7 deletions(-)
>>  create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c
>>  create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c
>>  create mode 1006

Re: [PATCH] btf: Protect BTF_KIND_INFO against invalid kind

[David Faust]

On 8/7/24 10:19, Will Hawkins wrote:
> On Mon, Jul 29, 2024 at 2:14 PM David Faust  wrote:
>>
>>
>> On 7/29/24 07:42, Will Hawkins wrote:
>>> If the user provides a kind value that is more than 5 bits, the
>>> BTF_KIND_INFO macro would emit incorrect values for info (by clobbering
>>> values of the kind flag).
>>>
>>> Tested on x86_64-redhat-linux.
>>
>> OK, thanks.
> 
> Just let me know if there is anything else that you need from me!
> Will

I just checked this in on your behalf.  Apologies for the delay.
Thanks!

> 
>>
>>>
>>> include/ChangeLog:
>>>
>>>   * btf.h (BTF_TYPE_INFO): Protect against user providing invalid
>>> kind.
>>>
>>> Signed-off-by: Will Hawkins 
>>> ---
>>>
>>> Notes:
>>>  I have a small out-of-tree test but was not sure whether a) it should
>>>be included and/or b) where it should be included. If you 
>>> would
>>>like me to include it, please just let me know where it 
>>> should go!
>>>
>>>  include/btf.h | 4 ++--
>>>  1 file changed, 2 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/include/btf.h b/include/btf.h
>>> index 3f45ffb0b6b..0c3e1a1cf51 100644
>>> --- a/include/btf.h
>>> +++ b/include/btf.h
>>> @@ -82,7 +82,7 @@ struct btf_type
>>>};
>>>  };
>>>
>>> -/* The folloing macros access the information encoded in btf_type.info.  */
>>> +/* The following macros access the information encoded in btf_type.info.  
>>> */
>>>  /* Type kind. See below.  */
>>>  #define BTF_INFO_KIND(info)  (((info) >> 24) & 0x1f)
>>>  /* Number of entries of variable length data following certain type kinds.
>>> @@ -95,7 +95,7 @@ struct btf_type
>>>
>>>  /* Encoding for struct btf_type.info.  */
>>>  #define BTF_TYPE_INFO(kind, kflag, vlen) \
>>> -  kflag) ? 1 : 0 ) << 31) | ((kind) << 24) | ((vlen) & 0x))
>>> +  kflag) ? 1 : 0 ) << 31) | ((kind & 0x1f) << 24) | ((vlen) & 0x))
>>>
>>>  #define BTF_KIND_UNKN0   /* Unknown or invalid.  */
>>>  #define BTF_KIND_INT 1   /* Integer.  */

[PATCH v4 0/6] btf: refactor and add pruning option

[David Faust]

[v3: https://gcc.gnu.org/pipermail/gcc-patches/2024-May/653165.html
 Changes from v3:
 - Address typos, comment fixes and other minor nits pointed out by
   Indu in patches 1-3 and 5.
 - Rename option added in patch 4 from -fprune-btf to -gprune-btf.
 - Reword commit message in patch 4 to better describe behavior of
   -gprune-btf between LTO and non-LTO builds.
 - Improve the option parsing in patch 6 using Indu's suggested
   approach.  Add a few tests for -gdwarf, -gbtf, -gctf interaction. ]

 Review status:
 - Patches 1-3 have been approved, not yet pushed.
 - The CTF/BTF portion of patch 5 has been approved; the remaining
   portion needs BPF backend reviewer approval.
 - Patches 4 and 6 need review.  Both have portions which I think
   need global maintainer approval. ]

This patch series signficantly refactors the BTF generation in gcc,
making it simpler and easier to understand, extend and maintain.

It also introduces an optional algorithm to "prune" BTF information
before emission.  This pruning is meant to be used for BPF programs, to
alleviate the massive bloating of BTF information caused by including
Linux kernel internal headers.  The pruning is designed to be compatible
with the unconditional pruning performed  by the LLVM BPF backend when
generating BTF information.

While the changes are fairly significant, there is no actual change in
emitted BTF information (unless pruning is enabled), other than bug
fixes and small additions to the assembler debug comments.

Patch 1 restructures the emission of CTF and BTF information, with the
result that CTF is always completely generated and emitted before any
BTF-related procedures are run.  BTF emission is moved to late finish
for all targets, except when building with -flto.

Patch 2 changes the data structures shared by CTF and BTF to use
pointers rather than type IDs for all inter-type references.  This
change is completely transparent to both CTF and BTF.

Patch 3 heavily refactors btfout.cc to take advantage of the prior
changes and significantly simplify the BTF implementation.  The changes
are nearly transparent, however some small but important improvements
are also made possible by the refactor, such as fixing PR113566 for
non-LTO builds.

Patch 4 adds a new option to perform pruning of the BTF information
before emission.  This is intended to be used for BPF programs which
often include kernel headers, and in many cases reduces the size of
the resulting BTF information by a factor of 10.

Patch 5 makes BTF pruning work with BPF CO-RE, and enables the pruning
by default in the BPF backend.

Patch 6 takes advantage of the prior changes, and removes the
restriction on generating both CTF and BTF in the same compiler run,
allowing for any combinaion of -gdwarf, -gctf and -gbtf.

Tested on x86_64-linux-gnu, and on x86_64-linux-gnu host for
bpf-unknown-none target.

Also tested by compiling and runninng Linux kernel BPF selftests.
No known regressions.

David Faust (6):
  ctf, btf: restructure CTF/BTF emission
  ctf: use pointers instead of IDs internally
  btf: refactor and simplify implementation
  btf: add -gprune-btf option
  bpf,btf: enable BTF pruning by default for BPF
  opts: allow any combination of DWARF, CTF, BTF

 gcc/btfout.cc | 1615 +
 gcc/common.opt|4 +
 gcc/config/bpf/bpf.cc |5 +
 gcc/config/bpf/btfext-out.cc  |   14 +-
 gcc/config/bpf/core-builtins.cc   |   74 +-
 gcc/ctfc.cc   |  153 +-
 gcc/ctfc.h|  113 +-
 gcc/ctfout.cc |   22 +-
 gcc/doc/invoke.texi   |   23 +
 gcc/dwarf2ctf.cc  |  324 ++--
 gcc/dwarf2ctf.h   |2 +-
 gcc/dwarf2out.cc  |4 +-
 gcc/opts.cc   |   20 +-
 gcc/testsuite/gcc.dg/debug/btf/btf-3.c|8 +
 gcc/testsuite/gcc.dg/debug/btf/btf-4.c|8 +
 gcc/testsuite/gcc.dg/debug/btf/btf-5.c|9 +
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c  |   25 +
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c  |   33 +
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c  |   35 +
 .../gcc.dg/debug/btf/btf-prune-maps.c |   20 +
 .../gcc.dg/debug/btf/btf-variables-5.c|6 +-
 include/btf.h |5 +
 22 files changed, 1457 insertions(+), 1065 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-3.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-4.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-5.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c
 create mode 100644 gcc/testsuite/gcc.dg/debu

[PATCH v4 1/6] ctf, btf: restructure CTF/BTF emission

[David Faust]

This commit makes some structural changes to the CTF/BTF debug info
emission.  In particular:

 a) CTF is new always fully generated and emitted before any
BTF-related procedures are run.  This means that BTF-related
functions can change, even irreversibly, the shared in-memory
representation used by the two formats without issue.

 b) BTF generation has fewer entry points, and is cleanly divided
into early_finish and finish.

 c) BTF is now always emitted at finish (called from dwarf2out_finish),
for all targets in non-LTO builds, rather than being emitted at
early_finish for targets other than BPF CO-RE.  In LTO builds,
BTF is emitted at early_finish as before.

Note that this change alone does not alter the contents of BTF at
all, regardless of whether it would have previously been emitted at
early_finish or finish, because the calculation of the BTF to be
emitted is not moved by this patch, only the write-out.

The changes are transparent to both CTF and BTF emission.

gcc/
* btfout.cc (btf_init_postprocess): Rename to...
(btf_early_finish): ...this.
(btf_output): Rename to...
(btf_finish): ...this.
* ctfc.h: Analogous changes.
* dwarf2ctf.cc (ctf_debug_early_finish): Conditionally call
btf_early_finish, or ctf_finalize as appropriate.  Emit BTF
here for LTO builds.
(ctf_debug_finish): Always call btf_finish here if generating
BTF info in non-LTO builds.
(ctf_debug_finalize, ctf_debug_init_postprocess): Delete.
* dwarf2out.cc (dwarf2out_early_finish): Remove call to
ctf_debug_init_postprocess.
---
 gcc/btfout.cc| 28 +
 gcc/ctfc.h   |  4 +--
 gcc/dwarf2ctf.cc | 65 +++-
 gcc/dwarf2out.cc |  2 --
 4 files changed, 50 insertions(+), 49 deletions(-)

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index 07f066a47068..d5e9f3bd43d3 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -1491,6 +1491,34 @@ btf_finalize (void)
   tu_ctfc = NULL;
 }
 
+/* Initial entry point of BTF generation, called at early_finish () after
+   CTF information has possibly been output.  Translate all CTF information
+   to BTF, and do any processing that must be done early, such as creating
+   BTF_KIND_FUNC records.  */
+
+void
+btf_early_finish (void)
+{
+  btf_init_postprocess ();
+}
+
+/* Late entry point for BTF generation, called from dwarf2out_finish ().
+   Complete and emit BTF information.  */
+
+void
+btf_finish (const char * filename)
+{
+  btf_output (filename);
+
+  /* If compiling for BPF with CO-RE info, we cannot deallocate until after the
+ contents of the .BTF.ext section are finalized, which happens very late in
+ BPF backend.  Therefore, the deallocation (i.e. btf_finalize ()) is 
delayed
+ until TARGET_ASM_FILE_END for BPF CO-RE.  */
+  if (!btf_with_core_debuginfo_p ())
+btf_finalize ();
+}
+
+
 /* Traversal function for all BTF_KIND_FUNC type records.  */
 
 bool
diff --git a/gcc/ctfc.h b/gcc/ctfc.h
index fa188bf2f5a4..e7bd93901cfa 100644
--- a/gcc/ctfc.h
+++ b/gcc/ctfc.h
@@ -384,8 +384,8 @@ extern void ctf_init (void);
 extern void ctf_output (const char * filename);
 extern void ctf_finalize (void);
 
-extern void btf_output (const char * filename);
-extern void btf_init_postprocess (void);
+extern void btf_early_finish (void);
+extern void btf_finish (const char * filename);
 extern void btf_finalize (void);
 
 extern ctf_container_ref ctf_get_tu_ctfc (void);
diff --git a/gcc/dwarf2ctf.cc b/gcc/dwarf2ctf.cc
index dc59569fe560..8f9e2fada9e3 100644
--- a/gcc/dwarf2ctf.cc
+++ b/gcc/dwarf2ctf.cc
@@ -933,30 +933,6 @@ gen_ctf_type (ctf_container_ref ctfc, dw_die_ref die)
   return type_id;
 }
 
-/* Prepare for output and write out the CTF debug information.  */
-
-static void
-ctf_debug_finalize (const char *filename, bool btf)
-{
-  if (btf)
-{
-  btf_output (filename);
-  /* btf_finalize when compiling BPF applciations gets deallocated by the
-BPF target in bpf_file_end.  */
-  if (btf_debuginfo_p () && !btf_with_core_debuginfo_p ())
-   btf_finalize ();
-}
-
-  else
-{
-  /* Emit the collected CTF information.  */
-  ctf_output (filename);
-
-  /* Reset the CTF state.  */
-  ctf_finalize ();
-}
-}
-
 bool
 ctf_do_die (dw_die_ref die)
 {
@@ -996,27 +972,27 @@ ctf_debug_init (void)
   add_name_attribute (ctf_unknown_die, "unknown");
 }
 
-/* Preprocess the CTF debug information after initialization.  */
-
-void
-ctf_debug_init_postprocess (bool btf)
-{
-  /* Only BTF requires postprocessing right after init.  */
-  if (btf)
-btf_init_postprocess ();
-}
-
 /* Early finish CTF/BTF debug info.  */
 
 void
 ctf_debug_early_finish (const char * filename)
 {
-  /* Emit CTF debug info early always.  */
-  if (ctf_debug_info_level > CTFINFO_LEVEL_NONE
-  /* Emit BTF debug info early if CO-RE relocations are not
-required.

[PATCH v4 5/6] bpf,btf: enable BTF pruning by default for BPF

[David Faust]

This patch enables -gprune-btf by default in the BPF backend when
generating BTF information, and fixes BPF CO-RE generation when using
-gprune-btf.

When generating BPF CO-RE information, we must ensure that types used
in CO-RE relocations always have sufficient BTF information emited so
that the CO-RE relocations can be processed by a BPF loader.  The BTF
pruning algorithm on its own does not have sufficient information to
determine which types are used in a BPF CO-RE relocation, so this
information must be supplied by the BPF backend, using a new
btf_mark_type_used function.

Co-authored-by: Cupertino Miranda 

gcc/
* btfout.cc (btf_mark_type_used): New.
* ctfc.h (btf_mark_type_used): Declare it here.
* config/bpf/bpf.cc (bpf_option_override): Enable -gprune-btf
by default if -gbtf is enabled.
* config/bpf/core-builtins.cc (extra_fn): New typedef.
(compute_field_expr): Add callback parameter, and call it if supplied.
Fix computation for MEM_REF.
(mark_component_type_as_used): New.
(bpf_mark_types_as_used): Likewise.
(bpf_expand_core_builtin): Call here.
* doc/invoke.texi (Debugging Options): Note that -gprune-btf is
enabled by default for BPF target when generating BTF.

gcc/testsuite/
* gcc.dg/debug/btf/btf-variables-5.c: Adjust one test for bpf-*-*
target.
---
 gcc/btfout.cc | 22 ++
 gcc/config/bpf/bpf.cc |  5 ++
 gcc/config/bpf/core-builtins.cc   | 71 +--
 gcc/ctfc.h|  1 +
 gcc/doc/invoke.texi   |  3 +
 .../gcc.dg/debug/btf/btf-variables-5.c|  6 +-
 6 files changed, 100 insertions(+), 8 deletions(-)

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index 34d8cec0a2e3..083ca48d6279 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -1503,6 +1503,28 @@ btf_assign_datasec_ids (ctf_container_ref ctfc)
 }
 }
 
+
+/* Manually mark that type T is used to ensure it will not be pruned.
+   Used by the BPF backend when generating BPF CO-RE to mark types used
+   in CO-RE relocations.  */
+
+void
+btf_mark_type_used (tree t)
+{
+  /* If we are not going to prune anyway, this is a no-op.  */
+  if (!debug_prune_btf)
+return;
+
+  gcc_assert (TYPE_P (t));
+  ctf_container_ref ctfc = ctf_get_tu_ctfc ();
+  ctf_dtdef_ref dtd = ctf_lookup_tree_type (ctfc, t);
+
+  if (!dtd)
+return;
+
+  btf_add_used_type (ctfc, dtd, false, false, true);
+}
+
 /* Callback used for assembling the only-used-types list.  Note that this is
the same as btf_type_list_cb above, but the hash_set traverse requires a
different function signature.  */
diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc
index dd1bfe38d29b..c62af7a6efa7 100644
--- a/gcc/config/bpf/bpf.cc
+++ b/gcc/config/bpf/bpf.cc
@@ -221,6 +221,11 @@ bpf_option_override (void)
   && !(target_flags_explicit & MASK_BPF_CORE))
 target_flags |= MASK_BPF_CORE;
 
+  /* -gbtf implies -gprune-btf for BPF target.  */
+  if (btf_debuginfo_p ())
+SET_OPTION_IF_UNSET (&global_options, &global_options_set,
+debug_prune_btf, true);
+
   /* Determine available features from ISA setting (-mcpu=).  */
   if (bpf_has_jmpext == -1)
 bpf_has_jmpext = (bpf_isa >= ISA_V2);
diff --git a/gcc/config/bpf/core-builtins.cc b/gcc/config/bpf/core-builtins.cc
index 232bebcadbd5..86e2e9d6e39f 100644
--- a/gcc/config/bpf/core-builtins.cc
+++ b/gcc/config/bpf/core-builtins.cc
@@ -624,13 +624,20 @@ bpf_core_get_index (const tree node, bool *valid)
 
ALLOW_ENTRY_CAST is an input arguments and specifies if the function should
consider as valid expressions in which NODE entry is a cast expression (or
-   tree code nop_expr).  */
+   tree code nop_expr).
+
+   EXTRA_FN is a callback function to allow extra functionality with this
+   function traversal.  Currently used for marking used type during expand
+   pass.  */
+
+typedef void (*extra_fn) (tree);
 
 static unsigned char
 compute_field_expr (tree node, unsigned int *accessors,
bool *valid,
tree *access_node,
-   bool allow_entry_cast = true)
+   bool allow_entry_cast = true,
+   extra_fn callback = NULL)
 {
   unsigned char n = 0;
   unsigned int fake_accessors[MAX_NR_ACCESSORS];
@@ -647,6 +654,9 @@ compute_field_expr (tree node, unsigned int *accessors,
 
   *access_node = node;
 
+  if (callback != NULL)
+callback (node);
+
   switch (TREE_CODE (node))
 {
 case INDIRECT_REF:
@@ -664,17 +674,19 @@ compute_field_expr (tree node, unsigned int *accessors,
 case COMPONENT_REF:
   n = compute_field_expr (TREE_OPERAND (node, 0), accessors,
  valid,
- access_node, false);
+ access_node, false, callback);
   accessors[n] = bpf_core_get_index (T

[PATCH v4 3/6] btf: refactor and simplify implementation

[David Faust]

This patch heavily refactors btfout.cc to take advantage of the
structural changes in the prior commits.

Now that inter-type references are internally stored as simply pointers,
all the painful, brittle, confusing infrastructure that was used in the
process of converting CTF type IDs to BTF type IDs can be thrown out.
This greatly simplifies the entire process of converting from CTF to
BTF, making the code cleaner, easier to read, and easier to maintain.

In addition, we no longer need to worry about destructive changes in
internal data structures used commonly by CTF and BTF, which allows
deleting several ancillary data structures previously used in btfout.cc.

This is nearly transparent, but a few improvements have also been made:

 1) BTF_KIND_FUNC records are now _always_ constructed at early_finish,
allowing us to construct records even for functions which are later
inlined by optimizations. DATASEC entries for functions are only
constructed at late_finish, to avoid incorrectly generating entries
for functions which get inlined.

 2) BTF_KIND_VAR records and DATASEC entries for them are now always
constructed at (late) finish, which avoids cases where we could
incorrectly create records for variables which were completely
optimized away.  This fixes PR debug/113566 for non-LTO builds.
In LTO builds, BTF must be emitted at early_finish, so some VAR
records may be emitted for variables which are later optimized away.

 3) Some additional assembler comments have been added with more
information for debugging.

gcc/
* btfout.cc (struct btf_datasec_entry): New.
(struct btf_datasec): Add `id' member.  Change `entries' to use
new struct btf_datasec_entry.
(func_map): New hash_map.
(max_translated_id): New.
(btf_var_ids, btf_id_map, holes, voids, num_vars_added)
(num_types_added, num_types_created): Delete.
(btf_absolute_var_id, btf_relative_var_id, btf_absolute_func_id)
(btf_relative_func_id, btf_absolute_datasec_id, init_btf_id_map)
(get_btf_id, set_btf_id, btf_emit_id_p): Delete.
(btf_removed_type_p): Delete.
(btf_dtd_kind, btf_emit_type_p): New helpers.
(btf_fwd_to_enum_p, btf_calc_num_vbytes): Use them.
(btf_collect_datasec): Delete.
(btf_dtd_postprocess_cb, btf_dvd_emit_preprocess_cb)
(btf_dtd_emit_preprocess_cb, btf_emit_preprocess): Delete.
(btf_dmd_representable_bitfield_p): Adapt to type reference changes
and delete now-unused ctfc argument.
(btf_asm_datasec_type_ref): Delete.
(btf_asm_type_ref): Adapt to type reference changes, simplify.
(btf_asm_type): Likewise. Mark struct/union types with bitfield
members.
(btf_asm_array): Adapt to data structure changes.
(btf_asm_varent): Likewise.
(btf_asm_sou_member): Likewise. Ensure non-bitfield members are
correctly re-encoded if struct or union contains any bitfield.
(btf_asm_func_arg, btf_asm_func_type, btf_asm_datasec_entry)
(btf_asm_datasec_type): Adapt to data structure changes.
(output_btf_header): Adapt to other changes, simplify type
length calculation, add info to assembler comments.
(output_btf_vars): Adapt to other changes.
(output_btf_strs): Fix overlong lines.
(output_asm_btf_sou_fields, output_asm_btf_enum_list)
(output_asm_btf_func_args_list, output_asm_btf_vlen_bytes)
(output_asm_btf_type, output_btf_types, output_btf_func_types)
(output_btf_datasec_types): Adapt to other changes.
(btf_init_postprocess): Delete.
(btf_output): Change to only perform output.
(btf_add_const_void, btf_add_func_records): New.
(btf_early_finish): Use them here. New.
(btf_datasec_push_entry): Adapt to data structure changes.
(btf_datasec_add_func, btf_datasec_add_var): New.
(btf_add_func_datasec_entries): New.
(btf_emit_variable_p): New helper.
(btf_add_vars): Use it here. New.
(btf_type_list_cb, btf_collect_translated_types): New.
(btf_assign_func_ids, btf_late_assign_var_ids)
(btf_assign_datasec_ids): New.
(btf_finish): Remove unused argument. Call new btf_late*
functions and btf_output.
(btf_finalize): Adapt to data structure changes.
* ctfc.h (struct ctf_dtdef): Convert existing boolean flags to
BOOL_BITFIELD and reorder.
(struct ctf_dvdef): Add dvd_id member.
(btf_finish): Remove argument from prototype.
(get_btf_id): Delete prototype.
(funcs_traverse_callback, traverse_btf_func_types): Add an
explanatory comment.
* dwarf2ctf.cc (ctf_debug_finish): Remove unused argument.
* dwarf2ctf.h: Analogous change.
* dwarf2out.cc: Likewise.
---
 gcc/btfout.cc| 1257 +++---
 gcc/ctfc.h   |   17 +-
 gcc/dwa

[PATCH v4 4/6] btf: add -gprune-btf option

[David Faust]

This patch adds a new option, -gprune-btf, to control BTF debug info
generation.

As the name implies, this option enables a kind of "pruning" of the BTF
information before it is emitted.  When enabled, rather than emitting
all type information translated from DWARF, only information for types
directly used in the source program is emitted.

The primary purpose of this pruning is to reduce the amount of
unnecessary BTF information emitted, especially for BPF programs.  It is
very common for BPF programs to include Linux kernel internal headers in
order to have access to kernel data structures.  However, doing so often
has the side effect of also adding type definitions for a large number
of types which are not actually used by nor relevant to the program.
In these cases, -gprune-btf commonly reduces the size of the resulting
BTF information by 10x or more, as seen on average when compiling Linux
kernel BPF selftests.  This both slims down the size of the resulting
object and reduces the time required by the BPF loader to verify the
program and its BTF information.

Note that the pruning implemented in this patch follows the same rules
as the BTF pruning performed unconditionally by LLVM's BPF backend when
generating BTF.  In particular, the main sources of pruning are:

  1) Only generate BTF for types used by variables and functions at the
 file scope.

 Note that which variables are known to be "used" may differ
 slightly between LTO and non-LTO builds due to optimizations.  For
 non-LTO builds (and always for the BPF target), variables which are
 optimized away during compilation are considered to be unused, and
 they (along with their types) are pruned.  For LTO builds, such
 variables are not known to be optimized away by the time pruning
 occurs, so VAR records for them and information for their types may
 be present in the emitted BTF information.  This is a missed
 optimization that may be fixed in the future.

  2) Avoid emitting full BTF for struct and union types which are only
 pointed-to by members of other struct/union types.  In these cases,
 the full BTF_KIND_STRUCT or BTF_KIND_UNION which would normally
 be emitted is replaced with a BTF_KIND_FWD, as though the
 underlying type was a forward-declared struct or union type.

gcc/
* btfout.cc (btf_used_types): New hash set.
(struct btf_fixup): New.
(fixups, forwards): New vecs.
(btf_output): Calculate num_types depending on debug_prune_btf.
(btf_early_finsih): New initialization for debug_prune_btf.
(btf_add_used_type): New function.
(btf_used_type_list_cb): Likewise.
(btf_collect_pruned_types): Likewise.
(btf_add_vars): Handle special case for variables in ".maps" section
when generating BTF for BPF CO-RE target.
(btf_late_finish): Use btf_collect_pruned_types when debug_prune_btf
is in effect.  Move some initialization to btf_early_finish.
(btf_finalize): Additional deallocation for debug_prune_btf.
* common.opt (gprune-btf): New flag.
* ctfc.cc (init_ctf_strtable): Make non-static.
* ctfc.h (init_ctf_strtable, ctfc_delete_strtab): Make extern.
* doc/invoke.texi (Debugging Options): Document -gprune-btf.

gcc/testsuite/
* gcc.dg/debug/btf/btf-prune-1.c: New test.
* gcc.dg/debug/btf/btf-prune-2.c: Likewise.
* gcc.dg/debug/btf/btf-prune-3.c: Likewise.
* gcc.dg/debug/btf/btf-prune-maps.c: Likewise.
---
 gcc/btfout.cc | 358 +-
 gcc/common.opt|   4 +
 gcc/ctfc.cc   |   2 +-
 gcc/ctfc.h|   3 +
 gcc/doc/invoke.texi   |  20 +
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c  |  25 ++
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c  |  33 ++
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c  |  35 ++
 .../gcc.dg/debug/btf/btf-prune-maps.c |  20 +
 9 files changed, 493 insertions(+), 7 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-maps.c

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index 89f148de9650..34d8cec0a2e3 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -828,7 +828,10 @@ output_btf_types (ctf_container_ref ctfc)
 {
   size_t i;
   size_t num_types;
-  num_types = ctfc->ctfc_types->elements ();
+  if (debug_prune_btf)
+num_types = max_translated_id;
+  else
+num_types = ctfc->ctfc_types->elements ();
 
   if (num_types)
 {
@@ -957,6 +960,212 @@ btf_add_func_records (ctf_container_ref ctfc)
 }
 }
 
+/* The set of types used directly in the source program, and any types manually
+   marked as used.  This is the set of type

[PATCH v4 2/6] ctf: use pointers instead of IDs internally

[David Faust]

This patch replaces all inter-type references in the ctfc internal data
structures with pointers, rather than the references-by-ID which were
used previously.

A couple of small updates in the BPF backend are included to make it
compatible with the change.

This change is only to the in-memory representation of various CTF
structures to make them easier to work with in various cases.  It is
outwardly transparent; there is no change in emitted CTF.

gcc/
* btfout.cc (BTF_VOID_TYPEID, BTF_INIT_TYPEID): Move defines to
include/btf.h.
(btf_dvd_emit_preprocess_cb, btf_emit_preprocess)
(btf_dmd_representable_bitfield_p, btf_asm_array, btf_asm_varent)
(btf_asm_sou_member, btf_asm_func_arg, btf_init_postprocess):
Adapt to structural changes in ctf_* structs.
* ctfc.h (struct ctf_dtdef): Add forward declaration.
(ctf_dtdef_t, ctf_dtdef_ref): Move typedefs earlier.
(struct ctf_arinfo, struct ctf_funcinfo, struct ctf_sliceinfo)
(struct ctf_itype, struct ctf_dmdef, struct ctf_func_arg)
(struct ctf_dvdef): Use pointers instead of type IDs for
references to other types and use typedefs where appropriate.
(struct ctf_dtdef): Add ref_type member.
(ctf_type_exists): Use pointer instead of type ID.
(ctf_add_reftype, ctf_add_enum, ctf_add_slice, ctf_add_float)
(ctf_add_integer, ctf_add_unknown, ctf_add_pointer)
(ctf_add_array, ctf_add_forward, ctf_add_typedef)
(ctf_add_function, ctf_add_sou, ctf_add_enumerator)
(ctf_add_variable): Likewise. Return pointer instead of ID.
(ctf_lookup_tree_type): Return pointer to type instead of ID.
* ctfc.cc: Analogous changes.
* ctfout.cc (ctf_asm_type, ctf_asm_slice, ctf_asm_varent)
(ctf_asm_sou_lmember, ctf_asm_sou_member, ctf_asm_func_arg)
(output_ctf_objt_info): Adapt to changes.
* dwarf2ctf.cc (gen_ctf_type, gen_ctf_void_type)
(gen_ctf_unknown_type, gen_ctf_base_type, gen_ctf_pointer_type)
(gen_ctf_subrange_type, gen_ctf_array_type, gen_ctf_typedef)
(gen_ctf_modifier_type, gen_ctf_sou_type, gen_ctf_function_type)
(gen_ctf_enumeration_type, gen_ctf_variable, gen_ctf_function)
(gen_ctf_type, ctf_do_die): Likewise.
* config/bpf/btfext-out.cc (struct btf_ext_core_reloc): Use
pointer instead of type ID.
(bpf_core_reloc_add, bpf_core_get_sou_member_index)
(output_btfext_core_sections): Adapt to above changes.
* config/bpf/core-builtins.cc (process_type): Likewise.

include/
* btf.h (BTF_VOID_TYPEID, BTF_INIT_TYPEID): Move defines here,
from gcc/btfout.cc.
---
 gcc/btfout.cc   |  40 +++--
 gcc/config/bpf/btfext-out.cc|  14 +-
 gcc/config/bpf/core-builtins.cc |   3 +-
 gcc/ctfc.cc | 151 +--
 gcc/ctfc.h  |  90 +--
 gcc/ctfout.cc   |  22 ++-
 gcc/dwarf2ctf.cc| 257 +++-
 include/btf.h   |   5 +
 8 files changed, 290 insertions(+), 292 deletions(-)

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index d5e9f3bd43d3..9d73478ba9a6 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -61,11 +61,6 @@ static char btf_info_section_label[MAX_BTF_LABEL_BYTES];
 #define BTF_INFO_SECTION_LABEL  "Lbtf"
 #endif
 
-/* BTF encodes void as type id 0.  */
-
-#define BTF_VOID_TYPEID 0
-#define BTF_INIT_TYPEID 1
-
 #define BTF_INVALID_TYPEID 0x
 
 /* Mapping of CTF variables to the IDs they will be assigned when they are
@@ -626,7 +621,8 @@ btf_dvd_emit_preprocess_cb (ctf_dvdef_ref *slot, 
ctf_container_ref arg_ctfc)
 return 1;
 
   /* Do not add variables which refer to unsupported types.  */
-  if (!voids.contains (var->dvd_type) && btf_removed_type_p (var->dvd_type))
+  if (!voids.contains (var->dvd_type->dtd_type)
+  && btf_removed_type_p (var->dvd_type->dtd_type))
 return 1;
 
   arg_ctfc->ctfc_vars_list[num_vars_added] = var;
@@ -716,7 +712,7 @@ btf_emit_preprocess (ctf_container_ref ctfc)
 static bool
 btf_dmd_representable_bitfield_p (ctf_container_ref ctfc, ctf_dmdef_t *dmd)
 {
-  ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[dmd->dmd_type];
+  ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[dmd->dmd_type->dtd_type];
 
   if (CTF_V2_INFO_KIND (ref_type->dtd_data.ctti_info) == CTF_K_SLICE)
 {
@@ -913,8 +909,8 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd)
 static void
 btf_asm_array (ctf_container_ref ctfc, ctf_arinfo_t arr)
 {
-  btf_asm_type_ref ("bta_elem_type", ctfc, arr.ctr_contents);
-  btf_asm_type_ref ("bta_index_type", ctfc, arr.ctr_index);
+  btf_asm_type_ref ("bta_elem_type", ctfc, arr.ctr_contents->dtd_type);
+  btf_asm_type_ref ("bta_index_type", ctfc, arr.ctr_index->dtd_type);
   dw2_asm_output_data (4, arr.ctr_nelems, "bta_nelems");
 }
 
@@ -927,7 +923,7 @@ btf_asm_varent (ctf_container_ref ctfc, ctf_dvdef_ref var)
  

[PATCH v4 6/6] opts: allow any combination of DWARF, CTF, BTF

[David Faust]

Previously it was not supported to generate both CTF and BTF debug info
in the same compiler run, as both formats made incompatible changes to
the same internal data structures.

With the structural change in the prior patches, in particular the
guarantee that CTF will always be fully emitted before any BTF
translation occurs, there is no longer anything preventing generation
of both CTF and BTF at the same time.  This patch changes option parsing
to allow any combination of -gdwarf, -gctf, and -gbtf at the same time.

gcc/
* opts.cc (set_debug_level): Allow any combination of -gdwarf,
-gctf and -gbtf to be enabled at the same time.

gcc/testsuite/
* gcc.dg/debug/btf/btf-3.c: New test.
* gcc.dg/debug/btf/btf-4.c: Likewise.
* gcc.dg/debug/btf/btf-5.c: Likewise.
---
 gcc/opts.cc| 20 +---
 gcc/testsuite/gcc.dg/debug/btf/btf-3.c |  8 
 gcc/testsuite/gcc.dg/debug/btf/btf-4.c |  8 
 gcc/testsuite/gcc.dg/debug/btf/btf-5.c |  9 +
 4 files changed, 30 insertions(+), 15 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-3.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-4.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-5.c

diff --git a/gcc/opts.cc b/gcc/opts.cc
index 1b1b46455af6..7e9f2d91172b 100644
--- a/gcc/opts.cc
+++ b/gcc/opts.cc
@@ -3506,21 +3506,11 @@ set_debug_level (uint32_t dinfo, int extended, const 
char *arg,
 }
   else
 {
-  /* Make and retain the choice if both CTF and DWARF debug info are to
-be generated.  */
-  if (((dinfo == DWARF2_DEBUG) || (dinfo == CTF_DEBUG))
- && ((opts->x_write_symbols == (DWARF2_DEBUG|CTF_DEBUG))
- || (opts->x_write_symbols == DWARF2_DEBUG)
- || (opts->x_write_symbols == CTF_DEBUG)))
-   {
- opts->x_write_symbols |= dinfo;
- opts_set->x_write_symbols |= dinfo;
-   }
-  /* However, CTF and BTF are not allowed together at this time.  */
-  else if (((dinfo == DWARF2_DEBUG) || (dinfo == BTF_DEBUG))
-  && ((opts->x_write_symbols == (DWARF2_DEBUG|BTF_DEBUG))
-  || (opts->x_write_symbols == DWARF2_DEBUG)
-  || (opts->x_write_symbols == BTF_DEBUG)))
+  /* Any combination of DWARF, CTF and BTF is allowed.  */
+  if (((dinfo == DWARF2_DEBUG) || (dinfo == CTF_DEBUG)
+  || (dinfo == BTF_DEBUG))
+ && ((opts->x_write_symbols | (DWARF2_DEBUG | CTF_DEBUG | BTF_DEBUG))
+  == (DWARF2_DEBUG | CTF_DEBUG | BTF_DEBUG)))
{
  opts->x_write_symbols |= dinfo;
  opts_set->x_write_symbols |= dinfo;
diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-3.c 
b/gcc/testsuite/gcc.dg/debug/btf/btf-3.c
new file mode 100644
index ..93c8164a2a54
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/debug/btf/btf-3.c
@@ -0,0 +1,8 @@
+/* Verify that BTF debug info can co-exist with DWARF.  */
+/* { dg-do compile } */
+/* { dg-options "-gdwarf -gbtf -dA" } */
+/* { dg-final { scan-assembler "0xeb9f.*btf_magic" } } */
+/* { dg-final { scan-assembler "DWARF version number" } } */
+
+void func (void)
+{ }
diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-4.c 
b/gcc/testsuite/gcc.dg/debug/btf/btf-4.c
new file mode 100644
index ..b087917188bb
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/debug/btf/btf-4.c
@@ -0,0 +1,8 @@
+/* Verify that BTF debug info can co-exist with CTF.  */
+/* { dg-do compile } */
+/* { dg-options "-gctf -gbtf -dA" } */
+/* { dg-final { scan-assembler "0xeb9f.*btf_magic" } } */
+/* { dg-final { scan-assembler "0xdff2.*CTF preamble magic number" } } */
+
+void func (void)
+{ }
diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-5.c 
b/gcc/testsuite/gcc.dg/debug/btf/btf-5.c
new file mode 100644
index ..45267b5fc422
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/debug/btf/btf-5.c
@@ -0,0 +1,9 @@
+/* Verify that BTF, CTF and DWARF can all co-exist happily.  */
+/* { dg-do compile } */
+/* { dg-options "-gctf -gbtf -gdwarf -dA" } */
+/* { dg-final { scan-assembler "0xeb9f.*btf_magic" } } */
+/* { dg-final { scan-assembler "0xdff2.*CTF preamble magic number" } } */
+/* { dg-final { scan-assembler "DWARF version number" } } */
+
+void func (void)
+{ }
-- 
2.43.0

Re: [PATCH v4 5/6] bpf,btf: enable BTF pruning by default for BPF

[David Faust]

On 6/12/24 09:55, Jose E. Marchesi wrote:
> 
> Hi Faust.
> Thanks for the patch.
> Please see a question below.
> 
>> This patch enables -gprune-btf by default in the BPF backend when
>> generating BTF information, and fixes BPF CO-RE generation when using
>> -gprune-btf.
>>
>> When generating BPF CO-RE information, we must ensure that types used
>> in CO-RE relocations always have sufficient BTF information emited so
>> that the CO-RE relocations can be processed by a BPF loader.  The BTF
>> pruning algorithm on its own does not have sufficient information to
>> determine which types are used in a BPF CO-RE relocation, so this
>> information must be supplied by the BPF backend, using a new
>> btf_mark_type_used function.
>>
>> Co-authored-by: Cupertino Miranda 
>>
>> gcc/
>>  * btfout.cc (btf_mark_type_used): New.
>>  * ctfc.h (btf_mark_type_used): Declare it here.
>>  * config/bpf/bpf.cc (bpf_option_override): Enable -gprune-btf
>>  by default if -gbtf is enabled.
>>  * config/bpf/core-builtins.cc (extra_fn): New typedef.
>>  (compute_field_expr): Add callback parameter, and call it if supplied.
>>  Fix computation for MEM_REF.
>>  (mark_component_type_as_used): New.
>>  (bpf_mark_types_as_used): Likewise.
>>  (bpf_expand_core_builtin): Call here.
>>  * doc/invoke.texi (Debugging Options): Note that -gprune-btf is
>>  enabled by default for BPF target when generating BTF.
>>
>> gcc/testsuite/
>>  * gcc.dg/debug/btf/btf-variables-5.c: Adjust one test for bpf-*-*
>>  target.
>> ---
>>  gcc/btfout.cc | 22 ++
>>  gcc/config/bpf/bpf.cc |  5 ++
>>  gcc/config/bpf/core-builtins.cc   | 71 +--
>>  gcc/ctfc.h|  1 +
>>  gcc/doc/invoke.texi   |  3 +
>>  .../gcc.dg/debug/btf/btf-variables-5.c|  6 +-
>>  6 files changed, 100 insertions(+), 8 deletions(-)
>>
>> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
>> index 34d8cec0a2e3..083ca48d6279 100644
>> --- a/gcc/btfout.cc
>> +++ b/gcc/btfout.cc
>> @@ -1503,6 +1503,28 @@ btf_assign_datasec_ids (ctf_container_ref ctfc)
>>  }
>>  }
>>  
>> +
>> +/* Manually mark that type T is used to ensure it will not be pruned.
>> +   Used by the BPF backend when generating BPF CO-RE to mark types used
>> +   in CO-RE relocations.  */
>> +
>> +void
>> +btf_mark_type_used (tree t)
>> +{
>> +  /* If we are not going to prune anyway, this is a no-op.  */
>> +  if (!debug_prune_btf)
>> +return;
>> +
>> +  gcc_assert (TYPE_P (t));
>> +  ctf_container_ref ctfc = ctf_get_tu_ctfc ();
>> +  ctf_dtdef_ref dtd = ctf_lookup_tree_type (ctfc, t);
>> +
>> +  if (!dtd)
>> +return;
>> +
>> +  btf_add_used_type (ctfc, dtd, false, false, true);
>> +}
>> +
>>  /* Callback used for assembling the only-used-types list.  Note that this is
>> the same as btf_type_list_cb above, but the hash_set traverse requires a
>> different function signature.  */
>> diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc
>> index dd1bfe38d29b..c62af7a6efa7 100644
>> --- a/gcc/config/bpf/bpf.cc
>> +++ b/gcc/config/bpf/bpf.cc
>> @@ -221,6 +221,11 @@ bpf_option_override (void)
>>&& !(target_flags_explicit & MASK_BPF_CORE))
>>  target_flags |= MASK_BPF_CORE;
>>  
>> +  /* -gbtf implies -gprune-btf for BPF target.  */
>> +  if (btf_debuginfo_p ())
>> +SET_OPTION_IF_UNSET (&global_options, &global_options_set,
>> + debug_prune_btf, true);
>> +
>>/* Determine available features from ISA setting (-mcpu=).  */
>>if (bpf_has_jmpext == -1)
>>  bpf_has_jmpext = (bpf_isa >= ISA_V2);
>> diff --git a/gcc/config/bpf/core-builtins.cc 
>> b/gcc/config/bpf/core-builtins.cc
>> index 232bebcadbd5..86e2e9d6e39f 100644
>> --- a/gcc/config/bpf/core-builtins.cc
>> +++ b/gcc/config/bpf/core-builtins.cc
>> @@ -624,13 +624,20 @@ bpf_core_get_index (const tree node, bool *valid)
>>  
>> ALLOW_ENTRY_CAST is an input arguments and specifies if the function 
>> should
>> consider as valid expressions in which NODE entry is a cast expression 
>> (or
>> -   tree code nop_expr).  */
>> +   tree code nop_expr).
>> +
>> +   EXTRA_FN is a callback function to allow extra functionality with this
>> +   function traversal.  Currently used for marking used type during expand
>> +   pass.  */
>> +
>> +typedef void (*extra_fn) (tree);
>>  
>>  static unsigned char
>>  compute_field_expr (tree node, unsigned int *accessors,
>>  bool *valid,
>>  tree *access_node,
>> -bool allow_entry_cast = true)
>> +bool allow_entry_cast = true,
>> +extra_fn callback = NULL)
>>  {
>>unsigned char n = 0;
>>unsigned int fake_accessors[MAX_NR_ACCESSORS];
>> @@ -647,6 +654,9 @@ compute_field_expr (tree node, unsigned int *accessors,
>>  
>>*access_node = node;
>>  
>> +  if (callback != NULL)
>> +callback 

Re: [PATCH v4 4/6] btf: add -gprune-btf option

[David Faust]

Ping.

Richard: I changed the option name as you asked but forgot to CC you on
the updated patch.  Is the new option OK?

Indu: You had some minor comments on the prior version which I have
addressed, not sure whether you meant the rest of the patch was OK or
not, or if you had a chance to review it.

Thanks!

archive: https://gcc.gnu.org/pipermail/gcc-patches/2024-June/654252.html

On 6/11/24 12:01, David Faust wrote:
> This patch adds a new option, -gprune-btf, to control BTF debug info
> generation.
> 
> As the name implies, this option enables a kind of "pruning" of the BTF
> information before it is emitted.  When enabled, rather than emitting
> all type information translated from DWARF, only information for types
> directly used in the source program is emitted.
> 
> The primary purpose of this pruning is to reduce the amount of
> unnecessary BTF information emitted, especially for BPF programs.  It is
> very common for BPF programs to include Linux kernel internal headers in
> order to have access to kernel data structures.  However, doing so often
> has the side effect of also adding type definitions for a large number
> of types which are not actually used by nor relevant to the program.
> In these cases, -gprune-btf commonly reduces the size of the resulting
> BTF information by 10x or more, as seen on average when compiling Linux
> kernel BPF selftests.  This both slims down the size of the resulting
> object and reduces the time required by the BPF loader to verify the
> program and its BTF information.
> 
> Note that the pruning implemented in this patch follows the same rules
> as the BTF pruning performed unconditionally by LLVM's BPF backend when
> generating BTF.  In particular, the main sources of pruning are:
> 
>   1) Only generate BTF for types used by variables and functions at the
>  file scope.
> 
>  Note that which variables are known to be "used" may differ
>  slightly between LTO and non-LTO builds due to optimizations.  For
>  non-LTO builds (and always for the BPF target), variables which are
>  optimized away during compilation are considered to be unused, and
>  they (along with their types) are pruned.  For LTO builds, such
>  variables are not known to be optimized away by the time pruning
>  occurs, so VAR records for them and information for their types may
>  be present in the emitted BTF information.  This is a missed
>  optimization that may be fixed in the future.
> 
>   2) Avoid emitting full BTF for struct and union types which are only
>  pointed-to by members of other struct/union types.  In these cases,
>  the full BTF_KIND_STRUCT or BTF_KIND_UNION which would normally
>  be emitted is replaced with a BTF_KIND_FWD, as though the
>  underlying type was a forward-declared struct or union type.
> 
> gcc/
>   * btfout.cc (btf_used_types): New hash set.
>   (struct btf_fixup): New.
>   (fixups, forwards): New vecs.
>   (btf_output): Calculate num_types depending on debug_prune_btf.
>   (btf_early_finsih): New initialization for debug_prune_btf.
>   (btf_add_used_type): New function.
>   (btf_used_type_list_cb): Likewise.
>   (btf_collect_pruned_types): Likewise.
>   (btf_add_vars): Handle special case for variables in ".maps" section
>   when generating BTF for BPF CO-RE target.
>   (btf_late_finish): Use btf_collect_pruned_types when debug_prune_btf
>   is in effect.  Move some initialization to btf_early_finish.
>   (btf_finalize): Additional deallocation for debug_prune_btf.
>   * common.opt (gprune-btf): New flag.
>   * ctfc.cc (init_ctf_strtable): Make non-static.
>   * ctfc.h (init_ctf_strtable, ctfc_delete_strtab): Make extern.
>   * doc/invoke.texi (Debugging Options): Document -gprune-btf.
> 
> gcc/testsuite/
>   * gcc.dg/debug/btf/btf-prune-1.c: New test.
>   * gcc.dg/debug/btf/btf-prune-2.c: Likewise.
>   * gcc.dg/debug/btf/btf-prune-3.c: Likewise.
>   * gcc.dg/debug/btf/btf-prune-maps.c: Likewise.
> ---
>  gcc/btfout.cc | 358 +-
>  gcc/common.opt|   4 +
>  gcc/ctfc.cc   |   2 +-
>  gcc/ctfc.h|   3 +
>  gcc/doc/invoke.texi   |  20 +
>  gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c  |  25 ++
>  gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c  |  33 ++
>  gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c  |  35 ++
>  .../gcc.dg/debug/btf/btf-prune-maps.c |  20 +
>  9 files changed, 493 insertions(+), 7 deletions(-)
>  create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c

Re: [PATCH v4 6/6] opts: allow any combination of DWARF, CTF, BTF

[David Faust]

Ping.

archive: https://gcc.gnu.org/pipermail/gcc-patches/2024-June/654253.html

On 6/11/24 12:01, David Faust wrote:
> Previously it was not supported to generate both CTF and BTF debug info
> in the same compiler run, as both formats made incompatible changes to
> the same internal data structures.
> 
> With the structural change in the prior patches, in particular the
> guarantee that CTF will always be fully emitted before any BTF
> translation occurs, there is no longer anything preventing generation
> of both CTF and BTF at the same time.  This patch changes option parsing
> to allow any combination of -gdwarf, -gctf, and -gbtf at the same time.
> 
> gcc/
>   * opts.cc (set_debug_level): Allow any combination of -gdwarf,
>   -gctf and -gbtf to be enabled at the same time.
> 
> gcc/testsuite/
>   * gcc.dg/debug/btf/btf-3.c: New test.
>   * gcc.dg/debug/btf/btf-4.c: Likewise.
>   * gcc.dg/debug/btf/btf-5.c: Likewise.
> ---
>  gcc/opts.cc| 20 +---
>  gcc/testsuite/gcc.dg/debug/btf/btf-3.c |  8 
>  gcc/testsuite/gcc.dg/debug/btf/btf-4.c |  8 
>  gcc/testsuite/gcc.dg/debug/btf/btf-5.c |  9 +
>  4 files changed, 30 insertions(+), 15 deletions(-)
>  create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-3.c
>  create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-4.c
>  create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-5.c
> 
> diff --git a/gcc/opts.cc b/gcc/opts.cc
> index 1b1b46455af6..7e9f2d91172b 100644
> --- a/gcc/opts.cc
> +++ b/gcc/opts.cc
> @@ -3506,21 +3506,11 @@ set_debug_level (uint32_t dinfo, int extended, const 
> char *arg,
>  }
>else
>  {
> -  /* Make and retain the choice if both CTF and DWARF debug info are to
> -  be generated.  */
> -  if (((dinfo == DWARF2_DEBUG) || (dinfo == CTF_DEBUG))
> -   && ((opts->x_write_symbols == (DWARF2_DEBUG|CTF_DEBUG))
> -   || (opts->x_write_symbols == DWARF2_DEBUG)
> -   || (opts->x_write_symbols == CTF_DEBUG)))
> - {
> -   opts->x_write_symbols |= dinfo;
> -   opts_set->x_write_symbols |= dinfo;
> - }
> -  /* However, CTF and BTF are not allowed together at this time.  */
> -  else if (((dinfo == DWARF2_DEBUG) || (dinfo == BTF_DEBUG))
> -&& ((opts->x_write_symbols == (DWARF2_DEBUG|BTF_DEBUG))
> -|| (opts->x_write_symbols == DWARF2_DEBUG)
> -|| (opts->x_write_symbols == BTF_DEBUG)))
> +  /* Any combination of DWARF, CTF and BTF is allowed.  */
> +  if (((dinfo == DWARF2_DEBUG) || (dinfo == CTF_DEBUG)
> +|| (dinfo == BTF_DEBUG))
> +   && ((opts->x_write_symbols | (DWARF2_DEBUG | CTF_DEBUG | BTF_DEBUG))
> +== (DWARF2_DEBUG | CTF_DEBUG | BTF_DEBUG)))
>   {
> opts->x_write_symbols |= dinfo;
> opts_set->x_write_symbols |= dinfo;
> diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-3.c 
> b/gcc/testsuite/gcc.dg/debug/btf/btf-3.c
> new file mode 100644
> index ..93c8164a2a54
> --- /dev/null
> +++ b/gcc/testsuite/gcc.dg/debug/btf/btf-3.c
> @@ -0,0 +1,8 @@
> +/* Verify that BTF debug info can co-exist with DWARF.  */
> +/* { dg-do compile } */
> +/* { dg-options "-gdwarf -gbtf -dA" } */
> +/* { dg-final { scan-assembler "0xeb9f.*btf_magic" } } */
> +/* { dg-final { scan-assembler "DWARF version number" } } */
> +
> +void func (void)
> +{ }
> diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-4.c 
> b/gcc/testsuite/gcc.dg/debug/btf/btf-4.c
> new file mode 100644
> index ..b087917188bb
> --- /dev/null
> +++ b/gcc/testsuite/gcc.dg/debug/btf/btf-4.c
> @@ -0,0 +1,8 @@
> +/* Verify that BTF debug info can co-exist with CTF.  */
> +/* { dg-do compile } */
> +/* { dg-options "-gctf -gbtf -dA" } */
> +/* { dg-final { scan-assembler "0xeb9f.*btf_magic" } } */
> +/* { dg-final { scan-assembler "0xdff2.*CTF preamble magic number" } } */
> +
> +void func (void)
> +{ }
> diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-5.c 
> b/gcc/testsuite/gcc.dg/debug/btf/btf-5.c
> new file mode 100644
> index ..45267b5fc422
> --- /dev/null
> +++ b/gcc/testsuite/gcc.dg/debug/btf/btf-5.c
> @@ -0,0 +1,9 @@
> +/* Verify that BTF, CTF and DWARF can all co-exist happily.  */
> +/* { dg-do compile } */
> +/* { dg-options "-gctf -gbtf -gdwarf -dA" } */
> +/* { dg-final { scan-assembler "0xeb9f.*btf_magic" } } */
> +/* { dg-final { scan-assembler "0xdff2.*CTF preamble magic number" } } */
> +/* { dg-final { scan-assembler "DWARF version number" } } */
> +
> +void func (void)
> +{ }

Re: [PATCH] btf: Protect BTF_KIND_INFO against invalid kind

[David Faust]

On 7/29/24 07:42, Will Hawkins wrote:
> If the user provides a kind value that is more than 5 bits, the
> BTF_KIND_INFO macro would emit incorrect values for info (by clobbering
> values of the kind flag).
> 
> Tested on x86_64-redhat-linux.

OK, thanks.

> 
> include/ChangeLog:
> 
>   * btf.h (BTF_TYPE_INFO): Protect against user providing invalid
> kind.
> 
> Signed-off-by: Will Hawkins 
> ---
> 
> Notes:
>  I have a small out-of-tree test but was not sure whether a) it should
>be included and/or b) where it should be included. If you would
>like me to include it, please just let me know where it should 
> go!
> 
>  include/btf.h | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/include/btf.h b/include/btf.h
> index 3f45ffb0b6b..0c3e1a1cf51 100644
> --- a/include/btf.h
> +++ b/include/btf.h
> @@ -82,7 +82,7 @@ struct btf_type
>};
>  };
>  
> -/* The folloing macros access the information encoded in btf_type.info.  */
> +/* The following macros access the information encoded in btf_type.info.  */
>  /* Type kind. See below.  */
>  #define BTF_INFO_KIND(info)  (((info) >> 24) & 0x1f)
>  /* Number of entries of variable length data following certain type kinds.
> @@ -95,7 +95,7 @@ struct btf_type
>  
>  /* Encoding for struct btf_type.info.  */
>  #define BTF_TYPE_INFO(kind, kflag, vlen) \
> -  kflag) ? 1 : 0 ) << 31) | ((kind) << 24) | ((vlen) & 0x))
> +  kflag) ? 1 : 0 ) << 31) | ((kind & 0x1f) << 24) | ((vlen) & 0x))
>  
>  #define BTF_KIND_UNKN0   /* Unknown or invalid.  */
>  #define BTF_KIND_INT 1   /* Integer.  */

Re: [PATCH v2] bpf: remove huge memory waste with string allocation.

[David Faust]

On 4/17/24 11:44, Cupertino Miranda wrote:
> The BPF backend was allocating an unnecessarily large string when
> constructing CO-RE relocations for enum types.
> 
> gcc/ChangeLog:
>   * config/bpf/core-builtins.cc (process_enum_value): Correct
>   string allocation.
> ---
>  gcc/config/bpf/core-builtins.cc | 10 ++
>  1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/gcc/config/bpf/core-builtins.cc b/gcc/config/bpf/core-builtins.cc
> index e03e986e2c1..beb039ea6e0 100644
> --- a/gcc/config/bpf/core-builtins.cc
> +++ b/gcc/config/bpf/core-builtins.cc
> @@ -870,12 +870,14 @@ process_enum_value (struct cr_builtins *data)
>unsigned int index = 0;
>for (tree l = TYPE_VALUES (type); l; l = TREE_CHAIN (l))
>   {
> +   gcc_assert (index < (1 << 16));

Since the index here is computed from the TREE node and not BTF, is it
somehow already enforced that this code won't be run if the enum can't
be represented in BTF? 

If not, I don't think the assert is a good idea. The TREE node could be
an enum too large for BTF, but not too large for gcc.  So the assert
will cause an ICE for something that is a representation limit (in BTF
and by extension .BTF.ext i.e. CO-RE) rather than a compiler bug.
IMO an error or warning makes more sense here, something like:

  if (index >= (1 << 16))
{
  bpf_error ("enumerator in CO-RE relocation cannot be represented");
  break;
}

WDYT?

> if (TREE_VALUE (l) == expr)
>   {
> -   char *tmp = (char *) ggc_alloc_atomic ((index / 10) + 1);
> -   sprintf (tmp, "%d", index);
> -   ret.str = (const char *) tmp;
> -
> +   /* Index can only be a value up to 2^16.  Should always fit
> +  in 6 chars.  */
> +   char tmp[6];
> +   sprintf (tmp, "%u", index);
> +   ret.str = CONST_CAST (char *, ggc_strdup(tmp));
> break;
>   }
> index++;

Re: [PATCH v3] bpf: remove huge memory waste with string allocation.

[David Faust]

Hi Cupertino,

On 4/18/24 13:58, Cupertino Miranda wrote:
> Hi David, everyone,
> 
> Following Davids last review I decided to properly detect error cases,
> as suggested.
> The error however should be reported earlier in compilation in
> pack_enum_valud function, where all the errors are reported.
> 
> Thanks for the quick and detailed reviews.
> 
> Regards,
> Cupertino

Thanks for taking the time on this.
This version is nice, just one little comment:

> 
> The BPF backend was allocating an unnecessarily large string when
> constructing CO-RE relocations for enum types.
> This patch further verifies if an enumerator is valid for CO-RE
> representability and returns an error in those cases.

The second sentence is a little awkward and seems to imply the error is
returned when the enumerator is valid :)
Perhaps "...verifies that an enumerator is valid for CO-RE, and returns
an error if it is not" or similar would be more clear?

Otherwise, OK.
Thanks!

> 
> gcc/ChangeLog:
>   * config/bpf/core-builtins.cc (get_index_for_enum_value): Create
>   function.
>   (pack_enum_value): Check for enumerator and error out.
>   (process_enum_value): Correct string allocation.
> ---
>  gcc/config/bpf/core-builtins.cc | 57 ++---
>  1 file changed, 38 insertions(+), 19 deletions(-)
> 
> diff --git a/gcc/config/bpf/core-builtins.cc b/gcc/config/bpf/core-builtins.cc
> index e03e986e2c1..829acea98f7 100644
> --- a/gcc/config/bpf/core-builtins.cc
> +++ b/gcc/config/bpf/core-builtins.cc
> @@ -795,6 +795,23 @@ process_field_expr (struct cr_builtins *data)
>  static GTY(()) hash_map *bpf_enum_mappings;
>  tree enum_value_type = NULL_TREE;
>  
> +static int
> +get_index_for_enum_value (tree type, tree expr)
> +{
> +  gcc_assert (TREE_CODE (expr) == CONST_DECL
> +   && TREE_CODE (type) == ENUMERAL_TYPE);
> +
> +  unsigned int index = 0;
> +  for (tree l = TYPE_VALUES (type); l; l = TREE_CHAIN (l))
> +{
> +  gcc_assert (index < (1 << 16));
> +  if (TREE_VALUE (l) == expr)
> + return index;
> +  index++;
> +}
> +  return -1;
> +}
> +
>  /* Pack helper for the __builtin_preserve_enum_value.  */
>  
>  static struct cr_local
> @@ -846,6 +863,16 @@ pack_enum_value_fail:
>   ret.reloc_data.default_value = integer_one_node;
>  }
>  
> +  if (ret.fail == false )
> +{
> +  int index = get_index_for_enum_value (type, tmp);
> +  if (index == -1 || index >= (1 << 16))
> + {
> +   bpf_error ("enum value in CO-RE builtin cannot be represented");
> +   ret.fail = true;
> + }
> +}
> +
>ret.reloc_data.type = type;
>ret.reloc_data.kind = kind;
>return ret;
> @@ -864,25 +891,17 @@ process_enum_value (struct cr_builtins *data)
>  
>struct cr_final ret = { NULL, type, data->kind };
>  
> -  if (TREE_CODE (expr) == CONST_DECL
> - && TREE_CODE (type) == ENUMERAL_TYPE)
> -{
> -  unsigned int index = 0;
> -  for (tree l = TYPE_VALUES (type); l; l = TREE_CHAIN (l))
> - {
> -   if (TREE_VALUE (l) == expr)
> - {
> -   char *tmp = (char *) ggc_alloc_atomic ((index / 10) + 1);
> -   sprintf (tmp, "%d", index);
> -   ret.str = (const char *) tmp;
> -
> -   break;
> - }
> -   index++;
> - }
> -}
> -  else
> -gcc_unreachable ();
> +  gcc_assert (TREE_CODE (expr) == CONST_DECL
> +   && TREE_CODE (type) == ENUMERAL_TYPE);
> +
> +  int index = get_index_for_enum_value (type, expr);
> +  gcc_assert (index != -1 && index < (1 << 16));
> +
> +  /* Index can only be a value up to 2^16.  Should always fit
> + in 6 chars.  */
> +  char tmp[6];
> +  sprintf (tmp, "%u", index);
> +  ret.str = CONST_CAST (char *, ggc_strdup(tmp));
>  
>return ret;
>  }

[PATCH] bpf: avoid issues with CO-RE and -gtoggle

[David Faust]

Compiling a BPF program with CO-RE relocations (and BTF) while also
passing -gtoggle led to an inconsistent state where CO-RE support was
enabled but BTF would not be generated, and this was not caught by the
existing option parsing.  This led to an ICE when generating the CO-RE
relocation info, since BTF is required for CO-RE.

Update bpf_option_override to avoid this case, and add a few tests for
the interactions of these options.

Tested on x86_64-linux-gnu host for bpf-unknown-none target.

gcc/
* config/bpf/bpf.cc (bpf_option_override): Improve handling of CO-RE
options to avoid issues with -gtoggle.

gcc/testsuite/
* gcc.target/bpf/core-options-1.c: New test.
* gcc.target/bpf/core-options-2.c: Likewise.
* gcc.target/bpf/core-options-3.c: Likewise.
---
 gcc/config/bpf/bpf.cc |  7 +--
 gcc/testsuite/gcc.target/bpf/core-options-1.c | 15 +++
 gcc/testsuite/gcc.target/bpf/core-options-2.c | 14 ++
 gcc/testsuite/gcc.target/bpf/core-options-3.c |  5 +
 4 files changed, 39 insertions(+), 2 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-options-1.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-options-2.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-options-3.c

diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc
index 98fb755bb8b..e6ea211a2c6 100644
--- a/gcc/config/bpf/bpf.cc
+++ b/gcc/config/bpf/bpf.cc
@@ -192,7 +192,8 @@ bpf_option_override (void)
   init_machine_status = bpf_init_machine_status;
 
   /* BPF CO-RE support requires BTF debug info generation.  */
-  if (TARGET_BPF_CORE && !btf_debuginfo_p ())
+  if (TARGET_BPF_CORE
+  && (!btf_debuginfo_p () || (debug_info_level < DINFO_LEVEL_NORMAL)))
 error ("BPF CO-RE requires BTF debugging information, use %<-gbtf%>");
 
   /* BPF applications always generate .BTF.ext.  */
@@ -215,7 +216,9 @@ bpf_option_override (void)
 
   /* -gbtf implies -mcore when using the BPF backend, unless -mno-co-re
  is specified.  */
-  if (btf_debuginfo_p () && !(target_flags_explicit & MASK_BPF_CORE))
+  if (btf_debuginfo_p ()
+  && (debug_info_level >= DINFO_LEVEL_NORMAL)
+  && !(target_flags_explicit & MASK_BPF_CORE))
 target_flags |= MASK_BPF_CORE;
 
   /* Determine available features from ISA setting (-mcpu=).  */
diff --git a/gcc/testsuite/gcc.target/bpf/core-options-1.c 
b/gcc/testsuite/gcc.target/bpf/core-options-1.c
new file mode 100644
index 000..7d8c677f239
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/core-options-1.c
@@ -0,0 +1,15 @@
+/* -gbtf for the BPF target should enable CO-RE support automatically.  */
+/* { dg-do compile } */
+/* { dg-options "-gbtf" } */
+
+struct A {
+  int x;
+  int y;
+  char c;
+};
+
+int
+foo (struct A *a) {
+  int y = __builtin_preserve_access_index (a->y);
+  return y;
+}
diff --git a/gcc/testsuite/gcc.target/bpf/core-options-2.c 
b/gcc/testsuite/gcc.target/bpf/core-options-2.c
new file mode 100644
index 000..8f466258e29
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/core-options-2.c
@@ -0,0 +1,14 @@
+/* { dg-do compile } */
+/* { dg-options "-gbtf -gtoggle" } */
+
+struct A {
+  int x;
+  int y;
+  char c;
+};
+
+int
+foo (struct A *a) {
+  int y = __builtin_preserve_access_index (a->y); /* { dg-error "BPF CO-RE is 
required" } */
+  return y;
+}
diff --git a/gcc/testsuite/gcc.target/bpf/core-options-3.c 
b/gcc/testsuite/gcc.target/bpf/core-options-3.c
new file mode 100644
index 000..ca32a7c4012
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/core-options-3.c
@@ -0,0 +1,5 @@
+/* This combination of options tries to enable CO-RE without BTF, and should
+   produce an error.  */
+/* { dg-do compile } */
+/* { dg-options "-gbtf -gtoggle -mco-re" } */
+/* { dg-excess-errors "BPF CO-RE requires BTF debugging information" } */
-- 
2.43.0

[PATCH] bpf: set PREFERRED_DEBUGGING_TYPE to BTF_DEBUG

[David Faust]

BTF is the standard debug info used with BPF programs, so it makes sense
to default to BTF rather than DWARF.

Tested on x86_64-linux-gnu host for bpf-unknown-none target.

gcc/
* config/bpf/bpf.h (PREFERRED_DEBUGGING_TYPE): Set to BTF_DEBUG.

gcc/testsuite/
* gcc.target/bpf/bpf-debug-options-1.c: New test.
* gcc.target/bpf/bpf-debug-options-2.c: Likewise.
* gcc.target/bpf/bpf-debug-options-3.c: Likewise.
* gcc.target/bpf/core-options-4.c: Likewise.
---
 gcc/config/bpf/bpf.h   |  5 +
 .../gcc.target/bpf/bpf-debug-options-1.c   | 17 +
 .../gcc.target/bpf/bpf-debug-options-2.c   | 18 ++
 .../gcc.target/bpf/bpf-debug-options-3.c   | 14 ++
 gcc/testsuite/gcc.target/bpf/core-options-4.c  |  4 
 5 files changed, 58 insertions(+)
 create mode 100644 gcc/testsuite/gcc.target/bpf/bpf-debug-options-1.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/bpf-debug-options-2.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/bpf-debug-options-3.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-options-4.c

diff --git a/gcc/config/bpf/bpf.h b/gcc/config/bpf/bpf.h
index c67e17526bf..e163fbf688d 100644
--- a/gcc/config/bpf/bpf.h
+++ b/gcc/config/bpf/bpf.h
@@ -245,6 +245,11 @@ enum reg_class
 
 / Debugging Info /
 
+/* Use BTF debug info by default.  */
+
+#undef  PREFERRED_DEBUGGING_TYPE
+#define PREFERRED_DEBUGGING_TYPE BTF_DEBUG
+
 /* In eBPF it is not possible to unwind frames. Disable CFA.  */
 
 #define DWARF2_FRAME_INFO 0
diff --git a/gcc/testsuite/gcc.target/bpf/bpf-debug-options-1.c 
b/gcc/testsuite/gcc.target/bpf/bpf-debug-options-1.c
new file mode 100644
index 000..409466c4ead
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/bpf-debug-options-1.c
@@ -0,0 +1,17 @@
+/* Default to BTF debug info.  */
+/* { dg-do compile } */
+/* { dg-options "-g -dA" }*/
+
+struct A {
+  int x;
+  int y;
+};
+
+int
+foo (struct A *a)
+{
+  return a->x;
+}
+
+/* { dg-final { scan-assembler-not "DWARF version" } } */
+/* { dg-final { scan-assembler "btf_version" } } */
diff --git a/gcc/testsuite/gcc.target/bpf/bpf-debug-options-2.c 
b/gcc/testsuite/gcc.target/bpf/bpf-debug-options-2.c
new file mode 100644
index 000..03bde12315b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/bpf-debug-options-2.c
@@ -0,0 +1,18 @@
+/* -g defaults to BTF, which in turn implies -mco-re.  */
+/* { dg-do compile } */
+/* { dg-options "-g -dA" }*/
+
+struct A {
+  int x;
+  int y;
+};
+
+int
+foo (struct A *a)
+{
+  return __builtin_preserve_access_index (a->x);
+}
+
+/* { dg-final { scan-assembler-not "DWARF version" } } */
+/* { dg-final { scan-assembler "btf_version" } } */
+/* { dg-final { scan-assembler "btfext_version" } } */
diff --git a/gcc/testsuite/gcc.target/bpf/bpf-debug-options-3.c 
b/gcc/testsuite/gcc.target/bpf/bpf-debug-options-3.c
new file mode 100644
index 000..d41790e0928
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/bpf-debug-options-3.c
@@ -0,0 +1,14 @@
+/* Using -g does not incorrectly force CO-RE enabled.  */
+/* { dg-do compile } */
+/* { dg-options "-g -dA -mno-co-re" }*/
+
+struct A {
+  int x;
+  int y;
+};
+
+int
+foo (struct A *a)
+{
+  return __builtin_preserve_access_index (a->x); /* { dg-error "BPF CO-RE is 
required" } */
+}
diff --git a/gcc/testsuite/gcc.target/bpf/core-options-4.c 
b/gcc/testsuite/gcc.target/bpf/core-options-4.c
new file mode 100644
index 000..fde4195da42
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/core-options-4.c
@@ -0,0 +1,4 @@
+/* -g implies BTF, -gtoggle turns it off.  CO-RE should not work.  */
+/* { dg-do compile } */
+/* { dg-options "-g -mco-re -gtoggle" } */
+/* { dg-excess-errors "BPF CO-RE requires BTF debugging information" } */
-- 
2.43.0

[PATCH 0/6] btf: refactor and add pruning option

[David Faust]

This patch series signficantly refactors the BTF generation in gcc,
making it simpler and easier to understand, extend and maintain.

It also introduces an optional algorithm to "prune" BTF information
before emission.  This pruning is meant to be used for BPF programs,
to alleviate the massive bloating of BPF progams caused by including
Linux kernel internal headers.  The pruning is designed to be almost
equivalent to that performed unconditionally by the LLVM BPF backend
when generating BTF information, for compatibility purposes.

While the changes are fairly significant, there is very little actual
change in emitted BTF information (unless pruning is enabled), other
than bug fixes and additions to the assembler debug comments.

Patch 1 restructures the emission of CTF and BTF information, with the
result that CTF is always completely generated and emitted before any
BTF-related procedures are run.

Patch 2 takes advantage of that change, and removes the restriction on
generating both CTF and BTF in the same compiler run, allowing for any
combinaion of -gdwarf, -gctf and -gbtf.

Patch 3 changes the data structures shared by CTF and BTF to use
pointers rather than type IDs for all inter-type references.  This
change is completely transparent to both CTF and BTF.

Patch 4 heavily refactors btfout.cc to take advantage of the prior
changes and significantly simplify the BTF implementation.  The changes
are nearly transparent, however some small but important improvements
are also made possible by the refactor, such as fixing PR113566.

Patch 5 adds a new option to perform pruning of the BTF information
before emission.  This is intended to be used for BPF programs which
often include kernel headers, and in many cases reduces the size of
the resulting BTF information by a factor of 10.

Patch 6 makes BTF pruning work with BPF CO-RE, and enables the pruning
by default in the BPF backend.

Tested on x86_64-linux-gnu, and on x86_64-linux-gnu host for
bpf-unknown-none target.

Also heavily tested with Linux kernel BPF selftests.
No known regressions.

David Faust (6):
  ctf, btf: restructure CTF/BTF emission
  opts: allow any combination of DWARF, CTF, BTF
  ctf: use pointers instead of IDs internally
  btf: refactor and simplify implementation
  btf: add -fprune-btf option
  bpf,btf: enable BTF pruning by default for BPF

 gcc/btfout.cc | 1645 ++---
 gcc/common.opt|4 +
 gcc/config/bpf/bpf.cc |5 +
 gcc/config/bpf/btfext-out.cc  |   12 +-
 gcc/config/bpf/core-builtins.cc   |   73 +-
 gcc/ctfc.cc   |  139 +-
 gcc/ctfc.h|   86 +-
 gcc/ctfout.cc |   19 +-
 gcc/doc/invoke.texi   |   23 +
 gcc/dwarf2ctf.cc  |  300 ++-
 gcc/dwarf2ctf.h   |2 +-
 gcc/dwarf2out.cc  |4 +-
 gcc/opts.cc   |   19 +-
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c  |   25 +
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c  |   33 +
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c  |   35 +
 .../gcc.dg/debug/btf/btf-variables-5.c|2 +-
 17 files changed, 1399 insertions(+), 1027 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c

-- 
2.43.0

[PATCH 2/6] opts: allow any combination of DWARF, CTF, BTF

[David Faust]

Previously it was not supported to generate both CTF and BTF debug info
in the same compiler run, as both formats made incompatible changes to
the same internal data structures.

With the structural change in the prior patch, in particular the
guarantee that CTF will always be fully emitted before any BTF
translation occurs, there is no longer anything preventing generation
of both CTF and BTF at the same time.  This patch changes option parsing
to allow any combination of -gdwarf, -gctf, and -gbtf at the same time.

gcc/
* opts.cc (set_debug_level): Allow any combination of -gdwarf,
-gctf and -gbtf at the same time.
---
 gcc/opts.cc | 19 ++-
 1 file changed, 6 insertions(+), 13 deletions(-)

diff --git a/gcc/opts.cc b/gcc/opts.cc
index 600e0ea..0a00eb6e2da 100644
--- a/gcc/opts.cc
+++ b/gcc/opts.cc
@@ -3488,21 +3488,14 @@ set_debug_level (uint32_t dinfo, int extended, const 
char *arg,
 }
   else
 {
-  /* Make and retain the choice if both CTF and DWARF debug info are to
-be generated.  */
-  if (((dinfo == DWARF2_DEBUG) || (dinfo == CTF_DEBUG))
+  /* Any combination of DWARF, CTF and BTF is allowed together.  */
+  if (((dinfo == DWARF2_DEBUG) || (dinfo == CTF_DEBUG) || (dinfo == 
BTF_DEBUG))
  && ((opts->x_write_symbols == (DWARF2_DEBUG|CTF_DEBUG))
+ || (opts->x_write_symbols == (DWARF2_DEBUG|BTF_DEBUG))
+ || (opts->x_write_symbols == (CTF_DEBUG|BTF_DEBUG))
  || (opts->x_write_symbols == DWARF2_DEBUG)
- || (opts->x_write_symbols == CTF_DEBUG)))
-   {
- opts->x_write_symbols |= dinfo;
- opts_set->x_write_symbols |= dinfo;
-   }
-  /* However, CTF and BTF are not allowed together at this time.  */
-  else if (((dinfo == DWARF2_DEBUG) || (dinfo == BTF_DEBUG))
-  && ((opts->x_write_symbols == (DWARF2_DEBUG|BTF_DEBUG))
-  || (opts->x_write_symbols == DWARF2_DEBUG)
-  || (opts->x_write_symbols == BTF_DEBUG)))
+ || (opts->x_write_symbols == CTF_DEBUG)
+ || (opts->x_write_symbols == BTF_DEBUG)))
{
  opts->x_write_symbols |= dinfo;
  opts_set->x_write_symbols |= dinfo;
-- 
2.43.0

[PATCH 1/6] ctf, btf: restructure CTF/BTF emission

[David Faust]

This commit makes some structural changes to the CTF/BTF debug info
emission.  In particular:

 a) CTF is new always fully generated and emitted before any
BTF-related procedures are run.  This means that BTF-related
functions can change, even irreversibly, the shared in-memory
representation used by the two formats without issue.

 b) BTF generation has fewer entry points, and is cleanly divided
into early_finish and finish.

 c) BTF is now always emitted at finish (called from dwarf2out_finish),
rather than being emitted at early_finish for targets other than
BPF CO-RE.  Note that this change alone does not alter the contents
of BTF at all, regardless of whether it would have previously been
emitted at early_finish or finish.

The changes are transparent to both CTF and BTF emission.

gcc/
* btfout.cc (btf_init_postprocess): Rename to...
(btf_early_finish): ...this.
(btf_output): Rename to...
(btf_finish): ...this.
* ctfc.h: Analogous changes.
* dwarf2ctf.cc (ctf_debug_early_finish): Conditionally call
btf_early_finish or ctf_finalize as appropriate.
(ctf_debug_finish): Always call btf_finish here if generating
BTF info.
(ctf_debug_finalize, ctf_debug_init_postprocess): Delete.
* dwarf2out.cc (dwarf2out_early_finish): Remove call to
ctf_debug_init_postprocess.
---
 gcc/btfout.cc| 28 +
 gcc/ctfc.h   |  4 ++--
 gcc/dwarf2ctf.cc | 54 +++-
 gcc/dwarf2out.cc |  2 --
 4 files changed, 42 insertions(+), 46 deletions(-)

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index 07f066a4706..1b6a9ed811f 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -1491,6 +1491,34 @@ btf_finalize (void)
   tu_ctfc = NULL;
 }
 
+/* Initial entry point of BTF generation, called at early_finish () after
+   CTF information has possibly been output.  Translate all CTF information
+   to BTF, and do any processing that must be done early, such as creating
+   BTF_KIND_FUNC records.  */
+
+void
+btf_early_finish (void)
+{
+  btf_init_postprocess ();
+}
+
+/* Late entry point for BTF generation, called from dwarf2out_finish ().
+   Complete and emit BTF information.  */
+
+void
+btf_finish (const char * filename)
+{
+  btf_output (filename);
+
+  /* If compiling for BPF with CO-RE info, we cannot deallocate until after
+ CO-RE information is created, which happens very late in BPF backend.
+ Therefore, the deallocation (i.e. btf_finalize ()) is delayed until
+ TARGET_ASM_FILE_END for BPF CO-RE.  */
+  if (!btf_with_core_debuginfo_p ())
+btf_finalize ();
+}
+
+
 /* Traversal function for all BTF_KIND_FUNC type records.  */
 
 bool
diff --git a/gcc/ctfc.h b/gcc/ctfc.h
index fa188bf2f5a..e7bd93901cf 100644
--- a/gcc/ctfc.h
+++ b/gcc/ctfc.h
@@ -384,8 +384,8 @@ extern void ctf_init (void);
 extern void ctf_output (const char * filename);
 extern void ctf_finalize (void);
 
-extern void btf_output (const char * filename);
-extern void btf_init_postprocess (void);
+extern void btf_early_finish (void);
+extern void btf_finish (const char * filename);
 extern void btf_finalize (void);
 
 extern ctf_container_ref ctf_get_tu_ctfc (void);
diff --git a/gcc/dwarf2ctf.cc b/gcc/dwarf2ctf.cc
index dc59569fe56..ec94982e4b1 100644
--- a/gcc/dwarf2ctf.cc
+++ b/gcc/dwarf2ctf.cc
@@ -933,30 +933,6 @@ gen_ctf_type (ctf_container_ref ctfc, dw_die_ref die)
   return type_id;
 }
 
-/* Prepare for output and write out the CTF debug information.  */
-
-static void
-ctf_debug_finalize (const char *filename, bool btf)
-{
-  if (btf)
-{
-  btf_output (filename);
-  /* btf_finalize when compiling BPF applciations gets deallocated by the
-BPF target in bpf_file_end.  */
-  if (btf_debuginfo_p () && !btf_with_core_debuginfo_p ())
-   btf_finalize ();
-}
-
-  else
-{
-  /* Emit the collected CTF information.  */
-  ctf_output (filename);
-
-  /* Reset the CTF state.  */
-  ctf_finalize ();
-}
-}
-
 bool
 ctf_do_die (dw_die_ref die)
 {
@@ -996,27 +972,21 @@ ctf_debug_init (void)
   add_name_attribute (ctf_unknown_die, "unknown");
 }
 
-/* Preprocess the CTF debug information after initialization.  */
-
-void
-ctf_debug_init_postprocess (bool btf)
-{
-  /* Only BTF requires postprocessing right after init.  */
-  if (btf)
-btf_init_postprocess ();
-}
-
 /* Early finish CTF/BTF debug info.  */
 
 void
 ctf_debug_early_finish (const char * filename)
 {
-  /* Emit CTF debug info early always.  */
-  if (ctf_debug_info_level > CTFINFO_LEVEL_NONE
-  /* Emit BTF debug info early if CO-RE relocations are not
-required.  */
-  || (btf_debuginfo_p () && !btf_with_core_debuginfo_p ()))
-ctf_debug_finalize (filename, btf_debuginfo_p ());
+  /* Emit the collected CTF information.  */
+  if (ctf_debug_info_level > CTFINFO_LEVEL_NONE)
+ctf_output (filename);
+
+  if (btf_debuginfo_p ())
+/

[PATCH 6/6] bpf,btf: enable BTF pruning by default for BPF

[David Faust]

This patch enables -fprune-btf by default in the BPF backend when
generating BTF information, and fixes BPF CO-RE generation when using
-fprune-btf.

When generating BPF CO-RE information, we must ensure that types used
in CO-RE relocations always have sufficient BTF information emited so
that the CO-RE relocations can be processed by a BPF loader.  The BTF
pruning algorithm on its own does not have sufficient information to
determine which types are used in a BPF CO-RE relocation, so this
information must be supplied by the BPF backend, using a new
btf_mark_type_used function.

Co-authored-by: Cupertino Miranda 

gcc/
* btfout.cc (btf_mark_type_used): New.
* ctfc.h (btf_mark_type_used): Declare it here.
* config/bpf/bpf.cc (bpf_option_override): Enable -fprune-btf
by default if -gbtf is enabled.
* config/bpf/bcore-builtins.cc (extra_fn): New typedef.
(compute_field_expr): Add callback parameter, and call it if supplied.
Fix computation for MEM_REF.
(mark_component_type_as_used): New.
(bpf_mark_types_as_used): Likewise.
(bpf_expand_core_builtin): Call here.
* doc/invoke.texi (Debugging Options): Note that -fprune-btf is
enabled by default for BPF target when generating BTF.

gcc/testsuite/
* gcc.dg/debug/btf/btf-variables-5.c: Add -fno-prune-btf to dg-options.
---
 gcc/btfout.cc | 22 ++
 gcc/config/bpf/bpf.cc |  5 ++
 gcc/config/bpf/core-builtins.cc   | 70 +--
 gcc/ctfc.h|  1 +
 gcc/doc/invoke.texi   |  3 +
 .../gcc.dg/debug/btf/btf-variables-5.c|  2 +-
 6 files changed, 96 insertions(+), 7 deletions(-)

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index 93d56492bbe..da2c9d35be9 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -1539,6 +1539,28 @@ btf_late_assign_datasec_ids (ctf_container_ref ctfc)
 }
 }
 
+
+/* Manually mark that type T is used to ensure it will not be pruned.
+   Used by the BPF backend when generating BPF CO-RE to mark types used
+   in CO-RE relocations.  */
+
+void
+btf_mark_type_used (tree t)
+{
+  /* If we are not going to prune anyway, this is a no-op.  */
+  if (!flag_prune_btf)
+return;
+
+  gcc_assert (TYPE_P (t));
+  ctf_container_ref ctfc = ctf_get_tu_ctfc ();
+  ctf_dtdef_ref dtd = ctf_lookup_tree_type (ctfc, t);
+
+  if (!dtd)
+return;
+
+  btf_minimal_add_type (ctfc, dtd, false, false);
+}
+
 /* Callback used for assembling the only-used-types list.  Note that this is
the same as btf_type_list_cb above, but the hash_set traverse requires a
different function signature.  */
diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc
index e6ea211a2c6..75303ce8f46 100644
--- a/gcc/config/bpf/bpf.cc
+++ b/gcc/config/bpf/bpf.cc
@@ -221,6 +221,11 @@ bpf_option_override (void)
   && !(target_flags_explicit & MASK_BPF_CORE))
 target_flags |= MASK_BPF_CORE;
 
+  /* -gbtf implies -fprune-btf for BPF target.  */
+  if (btf_debuginfo_p ())
+SET_OPTION_IF_UNSET (&global_options, &global_options_set,
+flag_prune_btf, true);
+
   /* Determine available features from ISA setting (-mcpu=).  */
   if (bpf_has_jmpext == -1)
 bpf_has_jmpext = (bpf_isa >= ISA_V2);
diff --git a/gcc/config/bpf/core-builtins.cc b/gcc/config/bpf/core-builtins.cc
index d5a7de825ad..1b91b1c0d25 100644
--- a/gcc/config/bpf/core-builtins.cc
+++ b/gcc/config/bpf/core-builtins.cc
@@ -624,13 +624,20 @@ bpf_core_get_index (const tree node, bool *valid)
 
ALLOW_ENTRY_CAST is an input arguments and specifies if the function should
consider as valid expressions in which NODE entry is a cast expression (or
-   tree code nop_expr).  */
+   tree code nop_expr).
+
+   EXTRA_FN is a callback function to allow extra functionality with this
+   function traversal. Currently used for marking used type during expand
+   pass.  */
+
+typedef void (*extra_fn) (tree);
 
 static unsigned char
 compute_field_expr (tree node, unsigned int *accessors,
bool *valid,
tree *access_node,
-   bool allow_entry_cast = true)
+   bool allow_entry_cast = true,
+   extra_fn callback = NULL)
 {
   unsigned char n = 0;
   unsigned int fake_accessors[MAX_NR_ACCESSORS];
@@ -647,6 +654,9 @@ compute_field_expr (tree node, unsigned int *accessors,
 
   *access_node = node;
 
+  if (callback != NULL)
+callback (node);
+
   switch (TREE_CODE (node))
 {
 case INDIRECT_REF:
@@ -664,17 +674,19 @@ compute_field_expr (tree node, unsigned int *accessors,
 case COMPONENT_REF:
   n = compute_field_expr (TREE_OPERAND (node, 0), accessors,
  valid,
- access_node, false);
+ access_node, false, callback);
   accessors[n] = bpf_core_get_index (TREE_OPERAND (node

[PATCH 5/6] btf: add -fprune-btf option

[David Faust]

This patch adds a new option, -fprune-btf, to control BTF debug info
generation.

As the name implies, this option enables a kind of "pruning" of the BTF
information before it is emitted.  When enabled, rather than emitting
all type information translated from DWARF, only information for types
directly used in the source program is emitted.

The primary purpose of this pruning is to reduce the amount of
unnecessary BTF information emitted, especially for BPF programs.  It is
very common for BPF programs to incldue Linux kernel internal headers in
order to have access to kernel data structures.  However, doing so often
has the side effect of also adding type definitions for a large number
of types which are not actually used by nor relevant to the program.
In these cases, -fprune-btf commonly reduces the size of the resulting
BTF information by approximately 10x.  This both slims down the size of
the resulting object and reduces the time required by the BPF loader to
verify the program and its BTF information.

Note that the pruning implemented in this patch follows the same rules
as the BTF pruning performed unconditionally by LLVM's BPF backend when
generating BTF.  In particular, the main sources of pruning are:

  1) Only generate BTF for types used by variables and functions at
 the file scope.

  2) Avoid emitting full BTF for struct and union types which are only
 pointed-to by members of other struct/union types.  In these cases,
 the full BTF_KIND_STRUCT or BTF_KIND_UNION which would normally
 be emitted is replaced with a BTF_KIND_FWD, as though the
 underlying type was a forward-declared struct or union type.

gcc/
* btfout.cc (btf_minimal_types): New hash set.
(struct btf_fixup): New.
(fixups, forwards): New vecs.
(btf_output): Calculate num_types depending on flag_prune_btf.
(btf_early_finsih): New initialization for flag_prune_btf.
(btf_mark_full_type_used): Likewise.
(btf_minimal_add_type): New function.
(btf_minimal_type_list_cb): Likewise.
(btf_late_collect_pruned_types): Likewise.
(btf_late_add_vars): Handle special case for variables in ".maps"
section when generating BTF for BPF CO-RE target.
(btf_late_finish): Use btf_late_collect_pruned_types when
flag_prune_btf in effect.  Move some initialization to btf_early_finish.
(btf_finalize): Additional deallocation for flag_prune_btf.
* common.opt (fprune-btf): New flag.
* ctfc.cc (init_ctf_strtable): Make non-static.
* ctfc.h (struct ctf_dtdef): Add visited_children_p boolean flag.
(init_ctf_strtable, ctfc_delete_strtab): Make extern.
* doc/invoke.texi (Debugging Options): Document -fprune-btf.

gcc/testsuite/
* gcc.dg/debug/btf/btf-prune-1.c: New test.
* gcc.dg/debug/btf/btf-prune-2.c: Likewise.
* gcc.dg/debug/btf/btf-prune-3.c: Likewise.
---
 gcc/btfout.cc| 394 ++-
 gcc/common.opt   |   4 +
 gcc/ctfc.cc  |   2 +-
 gcc/ctfc.h   |   5 +
 gcc/doc/invoke.texi  |  20 +
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c |  25 ++
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c |  33 ++
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c |  35 ++
 8 files changed, 511 insertions(+), 7 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index 0af0bd39fc7..93d56492bbe 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -833,7 +833,10 @@ output_btf_types (ctf_container_ref ctfc)
 {
   size_t i;
   size_t num_types;
-  num_types = ctfc->ctfc_types->elements ();
+  if (flag_prune_btf)
+num_types = max_translated_id;
+  else
+num_types = ctfc->ctfc_types->elements ();
 
   if (num_types)
 {
@@ -962,6 +965,211 @@ btf_early_add_func_records (ctf_container_ref ctfc)
 }
 }
 
+/* The set of types used directly in the source program, and any types manually
+   marked as used.  This is the set of types which will be emitted when
+   pruning (-fprune-btf) is enabled.  */
+static GTY (()) hash_set *btf_minimal_types;
+
+/* Fixup used to avoid unnecessary pointer chasing for types.  A fixup is
+   created when a structure or union member is a pointer to another struct
+   or union type.  In such cases, avoid emitting full type information for
+   the pointee struct or union type (which may be quite large), unless that
+   type is used directly elsewhere.  */
+struct btf_fixup
+{
+  ctf_dtdef_ref pointer_dtd; /* Type node to which the fixup is applied.  */
+  ctf_dtdef_ref pointee_dtd; /* Original type node referred to by pointer_dtd.
+   If this concrete type is not otherwise used,
+  

[PATCH 3/6] ctf: use pointers instead of IDs internally

[David Faust]

This patch replaces all inter-type references in the ctfc internal data
structures with pointers, rather than the references-by-ID which were
used previously.

A couple of small updates in the BPF backend are included to make it
compatible with the change.

This change is only to the in-memory representation of various CTF
structures to make them easier to work with in various cases.  It is
outwardly transparent; there is no change in emitted CTF.

gcc/
* ctfc.h (struct ctf_dtdef): Add forward declaration.
(struct ctf_arinfo, struct ctf_funcinfo, struct ctf_sliceinfo)
(struct ctf_itype, struct ctf_dmdef, struct ctf_func_arg)
(struct ctf_dvdef): Use pointers instead of type IDs for
references to other types.
(struct ctf_dtdef): Add ref_type member.
(ctf_type_exists): Use pointer instead of type ID.
(ctf_add_reftype, ctf_add_enum, ctf_add_slice, ctf_add_float)
(ctf_add_integer, ctf_add_unknown, ctf_add_pointer)
(ctf_add_array, ctf_add_forward, ctf_add_typedef)
(ctf_add_function, ctf_add_sou, ctf_add_enumerator)
(ctf_add_variable): Likewise. Return pointer instead of ID.
(ctf_lookup_tree_type): Return pointer to type instead of ID.
* ctfc.cc: Analogous changes.
* ctfout.cc (ctf_asm_type, ctf_asm_slice, ctf_asm_varent)
(ctf_asm_sou_lmember, ctf_asm_sou_member, ctf_asm_func_arg)
(output_ctf_objt_info): Adapt to changes.
* dwarf2ctf.cc (gen_ctf_type, gen_ctf_void_type)
(gen_ctf_unknown_type, gen_ctf_base_type, gen_ctf_pointer_type)
(gen_ctf_subrange_type, gen_ctf_array_type, gen_ctf_typedef)
(gen_ctf_modifier_type, gen_ctf_sou_type, gen_ctf_function_type)
(gen_ctf_enumeration_type, gen_ctf_variable, gen_ctf_function)
(gen_ctf_type, ctf_do_die): Likewise.
* config/bpf/btfext-out.cc (struct btf_ext_core_reloc): Use
pointer instead of type ID.
(bpf_core_reloc_add, bpf_core_get_sou_member_index)
(output_btfext_core_sections): Adapt to above changes.
* config/bpf/core-builtins.cc (process_type): Likewise.
---
 gcc/config/bpf/btfext-out.cc|  12 +-
 gcc/config/bpf/core-builtins.cc |   3 +-
 gcc/ctfc.cc | 137 +-
 gcc/ctfc.h  |  61 
 gcc/ctfout.cc   |  19 +--
 gcc/dwarf2ctf.cc| 244 +++-
 6 files changed, 233 insertions(+), 243 deletions(-)

diff --git a/gcc/config/bpf/btfext-out.cc b/gcc/config/bpf/btfext-out.cc
index 7ec438fd1d1..ce596e33643 100644
--- a/gcc/config/bpf/btfext-out.cc
+++ b/gcc/config/bpf/btfext-out.cc
@@ -134,7 +134,7 @@ struct GTY ((chain_next ("%h.next"))) btf_ext_lineinfo
 
 /* Internal representation of a BPF CO-RE relocation record.  */
 struct GTY ((chain_next ("%h.next"))) btf_ext_core_reloc {
-  unsigned int bpfcr_type; /* BTF type ID of container.  */
+  ctf_dtdef_ref bpfcr_type;/* BTF type of container.  */
   unsigned int  bpfcr_astr_off;/* Offset of access string in 
.BTF
   string table.  */
   rtx_code_label * bpfcr_insn_label;   /* RTX label attached to instruction
@@ -296,13 +296,14 @@ bpf_core_reloc_add (const tree type, const char * 
section_name,
   struct btf_ext_core_reloc *bpfcr = bpf_create_core_reloc (section_name, 
&sec);
 
   ctf_container_ref ctfc = ctf_get_tu_ctfc ();
+  ctf_dtdef_ref dtd = ctf_lookup_tree_type (ctfc, type);
 
   /* Buffer the access string in the auxiliary strtab.  */
   bpfcr->bpfcr_astr_off = 0;
   gcc_assert (accessor != NULL);
   bpfcr->bpfcr_astr_off = btf_ext_add_string (accessor);
 
-  bpfcr->bpfcr_type = get_btf_id (ctf_lookup_tree_type (ctfc, type));
+  bpfcr->bpfcr_type = dtd;
   bpfcr->bpfcr_insn_label = label;
   bpfcr->bpfcr_kind = kind;
 
@@ -341,7 +342,7 @@ bpf_core_get_sou_member_index (ctf_container_ref ctfc, 
const tree node)
   for (dmd = dtd->dtd_u.dtu_members;
dmd != NULL; dmd = (ctf_dmdef_t *) ctf_dmd_list_next (dmd))
 {
- bool field_has_btf = get_btf_id (dmd->dmd_type) <= BTF_MAX_TYPE;
+ bool field_has_btf = (dmd->dmd_type && dmd->dmd_type->dtd_type <= 
BTF_MAX_TYPE);
 
  if (field == node)
return field_has_btf ? i : -1;
@@ -574,8 +575,9 @@ output_btfext_core_sections (void)
 false);
  char *str = xstrdup (pp_formatted_text (&pp));
 
- dw2_asm_output_data (4, bpfcr->bpfcr_type, "bpfcr_type (%s)",
-  str);
+ uint32_t type_id =
+   bpfcr->bpfcr_type ? bpfcr->bpfcr_type->dtd_type : 0;
+ dw2_asm_output_data (4, type_id, "bpfcr_type (%s)", str);
  dw2_asm_output_data (4, bpfcr->bpfcr_astr_off + str_aux_off,
   "bpfcr_astr_off (\"%s\")",
   bpfcr->info.accessor_st

[PATCH 4/6] btf: refactor and simplify implementation

[David Faust]

This patch heavily refactors btfout.cc to take advantage of the
structural changes in the prior commits.

Now that inter-type references are internally stored as simply pointers,
all the painful, brittle, confusing infrastructure that was used in the
process of converting CTF type IDs to BTF type IDs can be thrown out.
This greatly simplifies the entire process of converting from CTF to
BTF, making the code cleaner, easier to read, and easier to maintain.

In addition, we no longer need to worry about destructive changes in
internal data structures used commonly by CTF and BTF, which allows
deleting several ancillary data structures previously used in btfout.cc.

This is nearly transparent, but a few improvements have also been made:

 1) BTF_KIND_FUNC records are now _always_ constructed at early_finish,
allowing us to construct records even for functions which are later
inlined by optimizations. DATASEC entries for functions are only
constructed at late_finish, to avoid incorrectly generating entries
for functions which get inlined.

 2) BTF_KIND_VAR records and DATASEC entries for them are now always
constructed at (late) finish, which avoids cases where we could
incorrectly create records for variables which were completely
optimized away. This fixes PR debug/113566.

 3) Some additional assembler comments have been added with more
information for debugging.

gcc/
* btfout.cc (struct btf_datasec_entry): New.
(struct btf_datasec): Add `id' member.  Change `entries' to use
new struct btf_datasec_entry.
(func_map): New hash_map.
(max_translated_id): New.
(btf_var_ids, btf_id_map, holes, voids, num_vars_added)
(num_types_added, num_types_created): Delete.
(btf_absolute_var_id, btf_relative_var_id, btf_absolute_func_id)
(btf_relative_func_id, btf_absolute_datasec_id, init_btf_id_map)
(get_btf_id, set_btf_id, btf_emit_id_p): Delete.
(btf_removed_type_p): Delete.
(btf_dtd_kind, btf_emit_type_p): New helpers.
(btf_fwd_to_enum_p, btf_calc_num_vbytes): Use them.
(btf_collect_datasec): Delete.
(btf_dtd_postprocess_cb, btf_dvd_emit_preprocess_cb)
(btf_dtd_emit_preprocess_cb, btf_emit_preprocess): Delete.
(btf_dmd_representable_bitfield_p): Adapt to type reference changes
and delete now-unused ctfc argument.
(btf_asm_datasec_type_ref): Delete.
(btf_asm_type_ref): Adapt to type reference changes, simplify.
(btf_asm_type): Likewise. Mark struct/union types with bitfield
members.
(btf_asm_array): Adapt to data structure changes.
(btf_asm_varent): Likewise.
(btf_asm_sou_member): Likewise. Ensure non-bitfield members are
correctly re-encoded if struct or union contains any bitfield.
(btf_asm_func_arg, btf_asm_func_type, btf_asm_datasec_entry)
(btf_asm_datasec_type): Adapt to data structure changes.
(output_btf_header): Adapt to other changes, simplify type
length calculation, add info to assembler comments.
(output_btf_vars): Adapt to other changes.
(output_btf_strs): Fix overlong lines.
(output_asm_btf_sou_fields, output_asm_btf_enum_list)
(output_asm_btf_func_args_list, output_asm_btf_vlen_bytes)
(output_asm_btf_type, output_btf_types, output_btf_func_types)
(output_btf_datasec_types): Adapt to other changes.
(btf_init_postprocess): Delete.
(btf_output): Change to only perform output.
(btf_early_add_const_void, btf_early_add_func_records): New.
(btf_early_finish): Use them here. New.
(btf_datasec_push_entry): Adapt to data structure changes.
(btf_datasec_add_func, btf_datasec_add_var): New.
(btf_late_add_func_datasec_entries): New.
(btf_emit_variable_p): New helper.
(btf_late_add_vars): Use it here. New.
(btf_type_list_cb, btf_late_collect_translated_types): New.
(btf_late_assign_func_ids, btf_late_assign_var_ids)
(btf_late_assign_datasec_ids): New.
(btf_finish): Remove unused argument. Call new btf_late*
functions and btf_output.
(btf_finalize): Adapt to data structure changes.
* ctfc.h (struct ctf_dtdef): Convert existing boolean flags to
BOOL_BITFIELD and reorder.
(struct ctf_dvdef): Add dvd_id member.
(btf_finish): Remove argument from prototype.
(get_btf_id): Delete prototype.
(funcs_traverse_callback, traverse_btf_func_types): Add an
explanatory comment.
* dwarf2ctf.cc (ctf_debug_finish): Remove unused argument.
* dwarf2ctf.h: Analogous change.
* dwarf2out.cc: Likewise.
---
 gcc/btfout.cc| 1261 +++---
 gcc/ctfc.h   |   17 +-
 gcc/dwarf2ctf.cc |4 +-
 gcc/dwarf2ctf.h  |2 +-
 gcc/dwarf2out.cc |2 +-
 5 files changed, 543 insertions(+), 743 deletio

Re: [PATCH 3/6] ctf: use pointers instead of IDs internally

[David Faust]

On 5/1/24 12:15, David Faust wrote:
> This patch replaces all inter-type references in the ctfc internal data
> structures with pointers, rather than the references-by-ID which were
> used previously.
> 
> A couple of small updates in the BPF backend are included to make it
> compatible with the change.
> 
> This change is only to the in-memory representation of various CTF
> structures to make them easier to work with in various cases.  It is
> outwardly transparent; there is no change in emitted CTF.

Linaro CI complains (rightly) about this patch.
Looks like I left out some changes here when I prepared the series
from my development brach and they went into patch 4 instead, or are
removed by the changes in patch 4.

Will figure this out and send a v2 soon.
Apologies for the noise.

> 
> gcc/
>   * ctfc.h (struct ctf_dtdef): Add forward declaration.
>   (struct ctf_arinfo, struct ctf_funcinfo, struct ctf_sliceinfo)
>   (struct ctf_itype, struct ctf_dmdef, struct ctf_func_arg)
>   (struct ctf_dvdef): Use pointers instead of type IDs for
>   references to other types.
>   (struct ctf_dtdef): Add ref_type member.
>   (ctf_type_exists): Use pointer instead of type ID.
>   (ctf_add_reftype, ctf_add_enum, ctf_add_slice, ctf_add_float)
>   (ctf_add_integer, ctf_add_unknown, ctf_add_pointer)
>   (ctf_add_array, ctf_add_forward, ctf_add_typedef)
>   (ctf_add_function, ctf_add_sou, ctf_add_enumerator)
>   (ctf_add_variable): Likewise. Return pointer instead of ID.
>   (ctf_lookup_tree_type): Return pointer to type instead of ID.
>   * ctfc.cc: Analogous changes.
>   * ctfout.cc (ctf_asm_type, ctf_asm_slice, ctf_asm_varent)
>   (ctf_asm_sou_lmember, ctf_asm_sou_member, ctf_asm_func_arg)
>   (output_ctf_objt_info): Adapt to changes.
>   * dwarf2ctf.cc (gen_ctf_type, gen_ctf_void_type)
>   (gen_ctf_unknown_type, gen_ctf_base_type, gen_ctf_pointer_type)
>   (gen_ctf_subrange_type, gen_ctf_array_type, gen_ctf_typedef)
>   (gen_ctf_modifier_type, gen_ctf_sou_type, gen_ctf_function_type)
>   (gen_ctf_enumeration_type, gen_ctf_variable, gen_ctf_function)
>   (gen_ctf_type, ctf_do_die): Likewise.
>   * config/bpf/btfext-out.cc (struct btf_ext_core_reloc): Use
>   pointer instead of type ID.
>   (bpf_core_reloc_add, bpf_core_get_sou_member_index)
>   (output_btfext_core_sections): Adapt to above changes.
>   * config/bpf/core-builtins.cc (process_type): Likewise.
> ---
>  gcc/config/bpf/btfext-out.cc|  12 +-
>  gcc/config/bpf/core-builtins.cc |   3 +-
>  gcc/ctfc.cc | 137 +-
>  gcc/ctfc.h  |  61 
>  gcc/ctfout.cc   |  19 +--
>  gcc/dwarf2ctf.cc| 244 +++-
>  6 files changed, 233 insertions(+), 243 deletions(-)
> 
> diff --git a/gcc/config/bpf/btfext-out.cc b/gcc/config/bpf/btfext-out.cc
> index 7ec438fd1d1..ce596e33643 100644
> --- a/gcc/config/bpf/btfext-out.cc
> +++ b/gcc/config/bpf/btfext-out.cc
> @@ -134,7 +134,7 @@ struct GTY ((chain_next ("%h.next"))) btf_ext_lineinfo
>  
>  /* Internal representation of a BPF CO-RE relocation record.  */
>  struct GTY ((chain_next ("%h.next"))) btf_ext_core_reloc {
> -  unsigned int bpfcr_type;   /* BTF type ID of container.  */
> +  ctf_dtdef_ref bpfcr_type;  /* BTF type of container.  */
>unsigned int  bpfcr_astr_off;  /* Offset of access string in 
> .BTF
>  string table.  */
>rtx_code_label * bpfcr_insn_label; /* RTX label attached to instruction
> @@ -296,13 +296,14 @@ bpf_core_reloc_add (const tree type, const char * 
> section_name,
>struct btf_ext_core_reloc *bpfcr = bpf_create_core_reloc (section_name, 
> &sec);
>  
>ctf_container_ref ctfc = ctf_get_tu_ctfc ();
> +  ctf_dtdef_ref dtd = ctf_lookup_tree_type (ctfc, type);
>  
>/* Buffer the access string in the auxiliary strtab.  */
>bpfcr->bpfcr_astr_off = 0;
>gcc_assert (accessor != NULL);
>bpfcr->bpfcr_astr_off = btf_ext_add_string (accessor);
>  
> -  bpfcr->bpfcr_type = get_btf_id (ctf_lookup_tree_type (ctfc, type));
> +  bpfcr->bpfcr_type = dtd;
>bpfcr->bpfcr_insn_label = label;
>bpfcr->bpfcr_kind = kind;
>  
> @@ -341,7 +342,7 @@ bpf_core_get_sou_member_index (ctf_container_ref ctfc, 
> const tree node)
>for (dmd = dtd->dtd_u.dtu_members;
> dmd != NULL; dmd = (ctf_dmdef_t *) ctf_dmd_list_next (dmd))
>  {
> -   bool field_has_btf = get_btf_id (dmd->dmd_type) <= BTF_MAX_TYPE;
> +   bool field_has_btf = (dmd->dmd_type && d

[PATCH v2 1/6] ctf, btf: restructure CTF/BTF emission

[David Faust]

This commit makes some structural changes to the CTF/BTF debug info
emission.  In particular:

 a) CTF is new always fully generated and emitted before any
BTF-related procedures are run.  This means that BTF-related
functions can change, even irreversibly, the shared in-memory
representation used by the two formats without issue.

 b) BTF generation has fewer entry points, and is cleanly divided
into early_finish and finish.

 c) BTF is now always emitted at finish (called from dwarf2out_finish),
rather than being emitted at early_finish for targets other than
BPF CO-RE.  Note that this change alone does not alter the contents
of BTF at all, regardless of whether it would have previously been
emitted at early_finish or finish.

The changes are transparent to both CTF and BTF emission.

gcc/
* btfout.cc (btf_init_postprocess): Rename to...
(btf_early_finish): ...this.
(btf_output): Rename to...
(btf_finish): ...this.
* ctfc.h: Analogous changes.
* dwarf2ctf.cc (ctf_debug_early_finish): Conditionally call
btf_early_finish or ctf_finalize as appropriate.
(ctf_debug_finish): Always call btf_finish here if generating
BTF info.
(ctf_debug_finalize, ctf_debug_init_postprocess): Delete.
* dwarf2out.cc (dwarf2out_early_finish): Remove call to
ctf_debug_init_postprocess.
---
 gcc/btfout.cc| 28 +
 gcc/ctfc.h   |  4 ++--
 gcc/dwarf2ctf.cc | 54 +++-
 gcc/dwarf2out.cc |  2 --
 4 files changed, 42 insertions(+), 46 deletions(-)

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index 07f066a4706..1b6a9ed811f 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -1491,6 +1491,34 @@ btf_finalize (void)
   tu_ctfc = NULL;
 }
 
+/* Initial entry point of BTF generation, called at early_finish () after
+   CTF information has possibly been output.  Translate all CTF information
+   to BTF, and do any processing that must be done early, such as creating
+   BTF_KIND_FUNC records.  */
+
+void
+btf_early_finish (void)
+{
+  btf_init_postprocess ();
+}
+
+/* Late entry point for BTF generation, called from dwarf2out_finish ().
+   Complete and emit BTF information.  */
+
+void
+btf_finish (const char * filename)
+{
+  btf_output (filename);
+
+  /* If compiling for BPF with CO-RE info, we cannot deallocate until after
+ CO-RE information is created, which happens very late in BPF backend.
+ Therefore, the deallocation (i.e. btf_finalize ()) is delayed until
+ TARGET_ASM_FILE_END for BPF CO-RE.  */
+  if (!btf_with_core_debuginfo_p ())
+btf_finalize ();
+}
+
+
 /* Traversal function for all BTF_KIND_FUNC type records.  */
 
 bool
diff --git a/gcc/ctfc.h b/gcc/ctfc.h
index fa188bf2f5a..e7bd93901cf 100644
--- a/gcc/ctfc.h
+++ b/gcc/ctfc.h
@@ -384,8 +384,8 @@ extern void ctf_init (void);
 extern void ctf_output (const char * filename);
 extern void ctf_finalize (void);
 
-extern void btf_output (const char * filename);
-extern void btf_init_postprocess (void);
+extern void btf_early_finish (void);
+extern void btf_finish (const char * filename);
 extern void btf_finalize (void);
 
 extern ctf_container_ref ctf_get_tu_ctfc (void);
diff --git a/gcc/dwarf2ctf.cc b/gcc/dwarf2ctf.cc
index dc59569fe56..ec94982e4b1 100644
--- a/gcc/dwarf2ctf.cc
+++ b/gcc/dwarf2ctf.cc
@@ -933,30 +933,6 @@ gen_ctf_type (ctf_container_ref ctfc, dw_die_ref die)
   return type_id;
 }
 
-/* Prepare for output and write out the CTF debug information.  */
-
-static void
-ctf_debug_finalize (const char *filename, bool btf)
-{
-  if (btf)
-{
-  btf_output (filename);
-  /* btf_finalize when compiling BPF applciations gets deallocated by the
-BPF target in bpf_file_end.  */
-  if (btf_debuginfo_p () && !btf_with_core_debuginfo_p ())
-   btf_finalize ();
-}
-
-  else
-{
-  /* Emit the collected CTF information.  */
-  ctf_output (filename);
-
-  /* Reset the CTF state.  */
-  ctf_finalize ();
-}
-}
-
 bool
 ctf_do_die (dw_die_ref die)
 {
@@ -996,27 +972,21 @@ ctf_debug_init (void)
   add_name_attribute (ctf_unknown_die, "unknown");
 }
 
-/* Preprocess the CTF debug information after initialization.  */
-
-void
-ctf_debug_init_postprocess (bool btf)
-{
-  /* Only BTF requires postprocessing right after init.  */
-  if (btf)
-btf_init_postprocess ();
-}
-
 /* Early finish CTF/BTF debug info.  */
 
 void
 ctf_debug_early_finish (const char * filename)
 {
-  /* Emit CTF debug info early always.  */
-  if (ctf_debug_info_level > CTFINFO_LEVEL_NONE
-  /* Emit BTF debug info early if CO-RE relocations are not
-required.  */
-  || (btf_debuginfo_p () && !btf_with_core_debuginfo_p ()))
-ctf_debug_finalize (filename, btf_debuginfo_p ());
+  /* Emit the collected CTF information.  */
+  if (ctf_debug_info_level > CTFINFO_LEVEL_NONE)
+ctf_output (filename);
+
+  if (btf_debuginfo_p ())
+/

[PATCH v2 0/6] btf: refactor and add pruning option

[David Faust]

[Changes from v1: Fix missing btfout.cc changes in patch 3 which
 prevent the patch from building as reported by Linaro CI.  Those
 changes are then removed/overwritten in patch 4, so the resulting
 code before patch 3 and after patch 4 is identical to v1. ]

This patch series signficantly refactors the BTF generation in gcc,
making it simpler and easier to understand, extend and maintain.

It also introduces an optional algorithm to "prune" BTF information
before emission.  This pruning is meant to be used for BPF programs,
to alleviate the massive bloating of BPF progams caused by including
Linux kernel internal headers.  The pruning is designed to be almost
equivalent to that performed unconditionally by the LLVM BPF backend
when generating BTF information, for compatibility purposes.

While the changes are fairly significant, there is very little actual
change in emitted BTF information (unless pruning is enabled), other
than bug fixes and additions to the assembler debug comments.

Patch 1 restructures the emission of CTF and BTF information, with the
result that CTF is always completely generated and emitted before any
BTF-related procedures are run.

Patch 2 takes advantage of that change, and removes the restriction on
generating both CTF and BTF in the same compiler run, allowing for any
combinaion of -gdwarf, -gctf and -gbtf.

Patch 3 changes the data structures shared by CTF and BTF to use
pointers rather than type IDs for all inter-type references.  This
change is completely transparent to both CTF and BTF.

Patch 4 heavily refactors btfout.cc to take advantage of the prior
changes and significantly simplify the BTF implementation.  The changes
are nearly transparent, however some small but important improvements
are also made possible by the refactor, such as fixing PR113566.

Patch 5 adds a new option to perform pruning of the BTF information
before emission.  This is intended to be used for BPF programs which
often include kernel headers, and in many cases reduces the size of
the resulting BTF information by a factor of 10.

Patch 6 makes BTF pruning work with BPF CO-RE, and enables the pruning
by default in the BPF backend.

Tested on x86_64-linux-gnu, and on x86_64-linux-gnu host for
bpf-unknown-none target.

Also heavily tested with Linux kernel BPF selftests.
No known regressions.

David Faust (6):
  ctf, btf: restructure CTF/BTF emission
  opts: allow any combination of DWARF, CTF, BTF
  ctf: use pointers instead of IDs internally
  btf: refactor and simplify implementation
  btf: add -fprune-btf option
  bpf,btf: enable BTF pruning by default for BPF

 gcc/btfout.cc | 1645 ++---
 gcc/common.opt|4 +
 gcc/config/bpf/bpf.cc |5 +
 gcc/config/bpf/btfext-out.cc  |   12 +-
 gcc/config/bpf/core-builtins.cc   |   73 +-
 gcc/ctfc.cc   |  139 +-
 gcc/ctfc.h|   86 +-
 gcc/ctfout.cc |   19 +-
 gcc/doc/invoke.texi   |   23 +
 gcc/dwarf2ctf.cc  |  300 ++-
 gcc/dwarf2ctf.h   |2 +-
 gcc/dwarf2out.cc  |4 +-
 gcc/opts.cc   |   19 +-
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c  |   25 +
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c  |   33 +
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c  |   35 +
 .../gcc.dg/debug/btf/btf-variables-5.c|2 +-
 17 files changed, 1399 insertions(+), 1027 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c

-- 
2.43.0

[PATCH v2 3/6] ctf: use pointers instead of IDs internally

[David Faust]

This patch replaces all inter-type references in the ctfc internal data
structures with pointers, rather than the references-by-ID which were
used previously.

A couple of small updates in the BPF backend are included to make it
compatible with the change.

This change is only to the in-memory representation of various CTF
structures to make them easier to work with in various cases.  It is
outwardly transparent; there is no change in emitted CTF.

gcc/
* btfout.cc (btf_dvd_emit_preprocess_cb, btf_emit_preprocess)
(btf_dmd_representable_bitfield_p, btf_asm_array, btf_asm_varent)
(btf_asm_sou_member, btf_asm_func_arg, btf_init_postprocess):
Adapt to structural changes in ctf_* structs.
* ctfc.h (struct ctf_dtdef): Add forward declaration.
(struct ctf_arinfo, struct ctf_funcinfo, struct ctf_sliceinfo)
(struct ctf_itype, struct ctf_dmdef, struct ctf_func_arg)
(struct ctf_dvdef): Use pointers instead of type IDs for
references to other types.
(struct ctf_dtdef): Add ref_type member.
(ctf_type_exists): Use pointer instead of type ID.
(ctf_add_reftype, ctf_add_enum, ctf_add_slice, ctf_add_float)
(ctf_add_integer, ctf_add_unknown, ctf_add_pointer)
(ctf_add_array, ctf_add_forward, ctf_add_typedef)
(ctf_add_function, ctf_add_sou, ctf_add_enumerator)
(ctf_add_variable): Likewise. Return pointer instead of ID.
(ctf_lookup_tree_type): Return pointer to type instead of ID.
* ctfc.cc: Analogous changes.
* ctfout.cc (ctf_asm_type, ctf_asm_slice, ctf_asm_varent)
(ctf_asm_sou_lmember, ctf_asm_sou_member, ctf_asm_func_arg)
(output_ctf_objt_info): Adapt to changes.
* dwarf2ctf.cc (gen_ctf_type, gen_ctf_void_type)
(gen_ctf_unknown_type, gen_ctf_base_type, gen_ctf_pointer_type)
(gen_ctf_subrange_type, gen_ctf_array_type, gen_ctf_typedef)
(gen_ctf_modifier_type, gen_ctf_sou_type, gen_ctf_function_type)
(gen_ctf_enumeration_type, gen_ctf_variable, gen_ctf_function)
(gen_ctf_type, ctf_do_die): Likewise.
* config/bpf/btfext-out.cc (struct btf_ext_core_reloc): Use
pointer instead of type ID.
(bpf_core_reloc_add, bpf_core_get_sou_member_index)
(output_btfext_core_sections): Adapt to above changes.
* config/bpf/core-builtins.cc (process_type): Likewise.
---
 gcc/btfout.cc   |  35 ++---
 gcc/config/bpf/btfext-out.cc|  12 +-
 gcc/config/bpf/core-builtins.cc |   3 +-
 gcc/ctfc.cc | 137 +-
 gcc/ctfc.h  |  61 
 gcc/ctfout.cc   |  19 +--
 gcc/dwarf2ctf.cc| 244 +++-
 7 files changed, 252 insertions(+), 259 deletions(-)

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index 1b6a9ed811f..14a503a4f80 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -626,7 +626,8 @@ btf_dvd_emit_preprocess_cb (ctf_dvdef_ref *slot, 
ctf_container_ref arg_ctfc)
 return 1;
 
   /* Do not add variables which refer to unsupported types.  */
-  if (!voids.contains (var->dvd_type) && btf_removed_type_p (var->dvd_type))
+  if (!voids.contains (var->dvd_type->dtd_type)
+  && btf_removed_type_p (var->dvd_type->dtd_type))
 return 1;
 
   arg_ctfc->ctfc_vars_list[num_vars_added] = var;
@@ -716,7 +717,7 @@ btf_emit_preprocess (ctf_container_ref ctfc)
 static bool
 btf_dmd_representable_bitfield_p (ctf_container_ref ctfc, ctf_dmdef_t *dmd)
 {
-  ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[dmd->dmd_type];
+  ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[dmd->dmd_type->dtd_type];
 
   if (CTF_V2_INFO_KIND (ref_type->dtd_data.ctti_info) == CTF_K_SLICE)
 {
@@ -913,8 +914,8 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd)
 static void
 btf_asm_array (ctf_container_ref ctfc, ctf_arinfo_t arr)
 {
-  btf_asm_type_ref ("bta_elem_type", ctfc, arr.ctr_contents);
-  btf_asm_type_ref ("bta_index_type", ctfc, arr.ctr_index);
+  btf_asm_type_ref ("bta_elem_type", ctfc, arr.ctr_contents->dtd_type);
+  btf_asm_type_ref ("bta_index_type", ctfc, arr.ctr_index->dtd_type);
   dw2_asm_output_data (4, arr.ctr_nelems, "bta_nelems");
 }
 
@@ -927,7 +928,7 @@ btf_asm_varent (ctf_container_ref ctfc, ctf_dvdef_ref var)
   (*(btf_var_ids->get (var)) + num_types_added + 1),
   var->dvd_name);
   dw2_asm_output_data (4, BTF_TYPE_INFO (BTF_KIND_VAR, 0, 0), "btv_info");
-  btf_asm_type_ref ("btv_type", ctfc, var->dvd_type);
+  btf_asm_type_ref ("btv_type", ctfc, var->dvd_type->dtd_type);
   dw2_asm_output_data (4, var->dvd_visibility, "btv_linkage");
 }
 
@@ -937,8 +938,8 @@ btf_asm_varent (ctf_container_ref ctfc, ctf_dvdef_ref var)
 static void
 btf_asm_sou_member (ctf_container_ref ctfc, ctf_dmdef_t * dmd, unsigned int 
idx)
 {
-  ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[dmd->dmd_type];
-  ctf_id_t base_type = dmd->dmd_type;
+  ctf_dtdef_ref

[PATCH v2 2/6] opts: allow any combination of DWARF, CTF, BTF

[David Faust]

Previously it was not supported to generate both CTF and BTF debug info
in the same compiler run, as both formats made incompatible changes to
the same internal data structures.

With the structural change in the prior patch, in particular the
guarantee that CTF will always be fully emitted before any BTF
translation occurs, there is no longer anything preventing generation
of both CTF and BTF at the same time.  This patch changes option parsing
to allow any combination of -gdwarf, -gctf, and -gbtf at the same time.

gcc/
* opts.cc (set_debug_level): Allow any combination of -gdwarf,
-gctf and -gbtf at the same time.
---
 gcc/opts.cc | 19 ++-
 1 file changed, 6 insertions(+), 13 deletions(-)

diff --git a/gcc/opts.cc b/gcc/opts.cc
index 600e0ea..0a00eb6e2da 100644
--- a/gcc/opts.cc
+++ b/gcc/opts.cc
@@ -3488,21 +3488,14 @@ set_debug_level (uint32_t dinfo, int extended, const 
char *arg,
 }
   else
 {
-  /* Make and retain the choice if both CTF and DWARF debug info are to
-be generated.  */
-  if (((dinfo == DWARF2_DEBUG) || (dinfo == CTF_DEBUG))
+  /* Any combination of DWARF, CTF and BTF is allowed together.  */
+  if (((dinfo == DWARF2_DEBUG) || (dinfo == CTF_DEBUG) || (dinfo == 
BTF_DEBUG))
  && ((opts->x_write_symbols == (DWARF2_DEBUG|CTF_DEBUG))
+ || (opts->x_write_symbols == (DWARF2_DEBUG|BTF_DEBUG))
+ || (opts->x_write_symbols == (CTF_DEBUG|BTF_DEBUG))
  || (opts->x_write_symbols == DWARF2_DEBUG)
- || (opts->x_write_symbols == CTF_DEBUG)))
-   {
- opts->x_write_symbols |= dinfo;
- opts_set->x_write_symbols |= dinfo;
-   }
-  /* However, CTF and BTF are not allowed together at this time.  */
-  else if (((dinfo == DWARF2_DEBUG) || (dinfo == BTF_DEBUG))
-  && ((opts->x_write_symbols == (DWARF2_DEBUG|BTF_DEBUG))
-  || (opts->x_write_symbols == DWARF2_DEBUG)
-  || (opts->x_write_symbols == BTF_DEBUG)))
+ || (opts->x_write_symbols == CTF_DEBUG)
+ || (opts->x_write_symbols == BTF_DEBUG)))
{
  opts->x_write_symbols |= dinfo;
  opts_set->x_write_symbols |= dinfo;
-- 
2.43.0

[PATCH v2 5/6] btf: add -fprune-btf option

[David Faust]

This patch adds a new option, -fprune-btf, to control BTF debug info
generation.

As the name implies, this option enables a kind of "pruning" of the BTF
information before it is emitted.  When enabled, rather than emitting
all type information translated from DWARF, only information for types
directly used in the source program is emitted.

The primary purpose of this pruning is to reduce the amount of
unnecessary BTF information emitted, especially for BPF programs.  It is
very common for BPF programs to incldue Linux kernel internal headers in
order to have access to kernel data structures.  However, doing so often
has the side effect of also adding type definitions for a large number
of types which are not actually used by nor relevant to the program.
In these cases, -fprune-btf commonly reduces the size of the resulting
BTF information by approximately 10x.  This both slims down the size of
the resulting object and reduces the time required by the BPF loader to
verify the program and its BTF information.

Note that the pruning implemented in this patch follows the same rules
as the BTF pruning performed unconditionally by LLVM's BPF backend when
generating BTF.  In particular, the main sources of pruning are:

  1) Only generate BTF for types used by variables and functions at
 the file scope.

  2) Avoid emitting full BTF for struct and union types which are only
 pointed-to by members of other struct/union types.  In these cases,
 the full BTF_KIND_STRUCT or BTF_KIND_UNION which would normally
 be emitted is replaced with a BTF_KIND_FWD, as though the
 underlying type was a forward-declared struct or union type.

gcc/
* btfout.cc (btf_minimal_types): New hash set.
(struct btf_fixup): New.
(fixups, forwards): New vecs.
(btf_output): Calculate num_types depending on flag_prune_btf.
(btf_early_finsih): New initialization for flag_prune_btf.
(btf_mark_full_type_used): Likewise.
(btf_minimal_add_type): New function.
(btf_minimal_type_list_cb): Likewise.
(btf_late_collect_pruned_types): Likewise.
(btf_late_add_vars): Handle special case for variables in ".maps"
section when generating BTF for BPF CO-RE target.
(btf_late_finish): Use btf_late_collect_pruned_types when
flag_prune_btf in effect.  Move some initialization to btf_early_finish.
(btf_finalize): Additional deallocation for flag_prune_btf.
* common.opt (fprune-btf): New flag.
* ctfc.cc (init_ctf_strtable): Make non-static.
* ctfc.h (struct ctf_dtdef): Add visited_children_p boolean flag.
(init_ctf_strtable, ctfc_delete_strtab): Make extern.
* doc/invoke.texi (Debugging Options): Document -fprune-btf.

gcc/testsuite/
* gcc.dg/debug/btf/btf-prune-1.c: New test.
* gcc.dg/debug/btf/btf-prune-2.c: Likewise.
* gcc.dg/debug/btf/btf-prune-3.c: Likewise.
---
 gcc/btfout.cc| 394 ++-
 gcc/common.opt   |   4 +
 gcc/ctfc.cc  |   2 +-
 gcc/ctfc.h   |   5 +
 gcc/doc/invoke.texi  |  20 +
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c |  25 ++
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c |  33 ++
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c |  35 ++
 8 files changed, 511 insertions(+), 7 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index 0af0bd39fc7..93d56492bbe 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -833,7 +833,10 @@ output_btf_types (ctf_container_ref ctfc)
 {
   size_t i;
   size_t num_types;
-  num_types = ctfc->ctfc_types->elements ();
+  if (flag_prune_btf)
+num_types = max_translated_id;
+  else
+num_types = ctfc->ctfc_types->elements ();
 
   if (num_types)
 {
@@ -962,6 +965,211 @@ btf_early_add_func_records (ctf_container_ref ctfc)
 }
 }
 
+/* The set of types used directly in the source program, and any types manually
+   marked as used.  This is the set of types which will be emitted when
+   pruning (-fprune-btf) is enabled.  */
+static GTY (()) hash_set *btf_minimal_types;
+
+/* Fixup used to avoid unnecessary pointer chasing for types.  A fixup is
+   created when a structure or union member is a pointer to another struct
+   or union type.  In such cases, avoid emitting full type information for
+   the pointee struct or union type (which may be quite large), unless that
+   type is used directly elsewhere.  */
+struct btf_fixup
+{
+  ctf_dtdef_ref pointer_dtd; /* Type node to which the fixup is applied.  */
+  ctf_dtdef_ref pointee_dtd; /* Original type node referred to by pointer_dtd.
+   If this concrete type is not otherwise used,
+  

[PATCH v2 4/6] btf: refactor and simplify implementation

[David Faust]

This patch heavily refactors btfout.cc to take advantage of the
structural changes in the prior commits.

Now that inter-type references are internally stored as simply pointers,
all the painful, brittle, confusing infrastructure that was used in the
process of converting CTF type IDs to BTF type IDs can be thrown out.
This greatly simplifies the entire process of converting from CTF to
BTF, making the code cleaner, easier to read, and easier to maintain.

In addition, we no longer need to worry about destructive changes in
internal data structures used commonly by CTF and BTF, which allows
deleting several ancillary data structures previously used in btfout.cc.

This is nearly transparent, but a few improvements have also been made:

 1) BTF_KIND_FUNC records are now _always_ constructed at early_finish,
allowing us to construct records even for functions which are later
inlined by optimizations. DATASEC entries for functions are only
constructed at late_finish, to avoid incorrectly generating entries
for functions which get inlined.

 2) BTF_KIND_VAR records and DATASEC entries for them are now always
constructed at (late) finish, which avoids cases where we could
incorrectly create records for variables which were completely
optimized away. This fixes PR debug/113566.

 3) Some additional assembler comments have been added with more
information for debugging.

gcc/
* btfout.cc (struct btf_datasec_entry): New.
(struct btf_datasec): Add `id' member.  Change `entries' to use
new struct btf_datasec_entry.
(func_map): New hash_map.
(max_translated_id): New.
(btf_var_ids, btf_id_map, holes, voids, num_vars_added)
(num_types_added, num_types_created): Delete.
(btf_absolute_var_id, btf_relative_var_id, btf_absolute_func_id)
(btf_relative_func_id, btf_absolute_datasec_id, init_btf_id_map)
(get_btf_id, set_btf_id, btf_emit_id_p): Delete.
(btf_removed_type_p): Delete.
(btf_dtd_kind, btf_emit_type_p): New helpers.
(btf_fwd_to_enum_p, btf_calc_num_vbytes): Use them.
(btf_collect_datasec): Delete.
(btf_dtd_postprocess_cb, btf_dvd_emit_preprocess_cb)
(btf_dtd_emit_preprocess_cb, btf_emit_preprocess): Delete.
(btf_dmd_representable_bitfield_p): Adapt to type reference changes
and delete now-unused ctfc argument.
(btf_asm_datasec_type_ref): Delete.
(btf_asm_type_ref): Adapt to type reference changes, simplify.
(btf_asm_type): Likewise. Mark struct/union types with bitfield
members.
(btf_asm_array): Adapt to data structure changes.
(btf_asm_varent): Likewise.
(btf_asm_sou_member): Likewise. Ensure non-bitfield members are
correctly re-encoded if struct or union contains any bitfield.
(btf_asm_func_arg, btf_asm_func_type, btf_asm_datasec_entry)
(btf_asm_datasec_type): Adapt to data structure changes.
(output_btf_header): Adapt to other changes, simplify type
length calculation, add info to assembler comments.
(output_btf_vars): Adapt to other changes.
(output_btf_strs): Fix overlong lines.
(output_asm_btf_sou_fields, output_asm_btf_enum_list)
(output_asm_btf_func_args_list, output_asm_btf_vlen_bytes)
(output_asm_btf_type, output_btf_types, output_btf_func_types)
(output_btf_datasec_types): Adapt to other changes.
(btf_init_postprocess): Delete.
(btf_output): Change to only perform output.
(btf_early_add_const_void, btf_early_add_func_records): New.
(btf_early_finish): Use them here. New.
(btf_datasec_push_entry): Adapt to data structure changes.
(btf_datasec_add_func, btf_datasec_add_var): New.
(btf_late_add_func_datasec_entries): New.
(btf_emit_variable_p): New helper.
(btf_late_add_vars): Use it here. New.
(btf_type_list_cb, btf_late_collect_translated_types): New.
(btf_late_assign_func_ids, btf_late_assign_var_ids)
(btf_late_assign_datasec_ids): New.
(btf_finish): Remove unused argument. Call new btf_late*
functions and btf_output.
(btf_finalize): Adapt to data structure changes.
* ctfc.h (struct ctf_dtdef): Convert existing boolean flags to
BOOL_BITFIELD and reorder.
(struct ctf_dvdef): Add dvd_id member.
(btf_finish): Remove argument from prototype.
(get_btf_id): Delete prototype.
(funcs_traverse_callback, traverse_btf_func_types): Add an
explanatory comment.
* dwarf2ctf.cc (ctf_debug_finish): Remove unused argument.
* dwarf2ctf.h: Analogous change.
* dwarf2out.cc: Likewise.
---
 gcc/btfout.cc| 1256 +++---
 gcc/ctfc.h   |   17 +-
 gcc/dwarf2ctf.cc |4 +-
 gcc/dwarf2ctf.h  |2 +-
 gcc/dwarf2out.cc |2 +-
 5 files changed, 539 insertions(+), 742 deletio

[PATCH v2 6/6] bpf,btf: enable BTF pruning by default for BPF

[David Faust]

This patch enables -fprune-btf by default in the BPF backend when
generating BTF information, and fixes BPF CO-RE generation when using
-fprune-btf.

When generating BPF CO-RE information, we must ensure that types used
in CO-RE relocations always have sufficient BTF information emited so
that the CO-RE relocations can be processed by a BPF loader.  The BTF
pruning algorithm on its own does not have sufficient information to
determine which types are used in a BPF CO-RE relocation, so this
information must be supplied by the BPF backend, using a new
btf_mark_type_used function.

Co-authored-by: Cupertino Miranda 

gcc/
* btfout.cc (btf_mark_type_used): New.
* ctfc.h (btf_mark_type_used): Declare it here.
* config/bpf/bpf.cc (bpf_option_override): Enable -fprune-btf
by default if -gbtf is enabled.
* config/bpf/bcore-builtins.cc (extra_fn): New typedef.
(compute_field_expr): Add callback parameter, and call it if supplied.
Fix computation for MEM_REF.
(mark_component_type_as_used): New.
(bpf_mark_types_as_used): Likewise.
(bpf_expand_core_builtin): Call here.
* doc/invoke.texi (Debugging Options): Note that -fprune-btf is
enabled by default for BPF target when generating BTF.

gcc/testsuite/
* gcc.dg/debug/btf/btf-variables-5.c: Add -fno-prune-btf to dg-options.
---
 gcc/btfout.cc | 22 ++
 gcc/config/bpf/bpf.cc |  5 ++
 gcc/config/bpf/core-builtins.cc   | 70 +--
 gcc/ctfc.h|  1 +
 gcc/doc/invoke.texi   |  3 +
 .../gcc.dg/debug/btf/btf-variables-5.c|  2 +-
 6 files changed, 96 insertions(+), 7 deletions(-)

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index 93d56492bbe..da2c9d35be9 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -1539,6 +1539,28 @@ btf_late_assign_datasec_ids (ctf_container_ref ctfc)
 }
 }
 
+
+/* Manually mark that type T is used to ensure it will not be pruned.
+   Used by the BPF backend when generating BPF CO-RE to mark types used
+   in CO-RE relocations.  */
+
+void
+btf_mark_type_used (tree t)
+{
+  /* If we are not going to prune anyway, this is a no-op.  */
+  if (!flag_prune_btf)
+return;
+
+  gcc_assert (TYPE_P (t));
+  ctf_container_ref ctfc = ctf_get_tu_ctfc ();
+  ctf_dtdef_ref dtd = ctf_lookup_tree_type (ctfc, t);
+
+  if (!dtd)
+return;
+
+  btf_minimal_add_type (ctfc, dtd, false, false);
+}
+
 /* Callback used for assembling the only-used-types list.  Note that this is
the same as btf_type_list_cb above, but the hash_set traverse requires a
different function signature.  */
diff --git a/gcc/config/bpf/bpf.cc b/gcc/config/bpf/bpf.cc
index e6ea211a2c6..75303ce8f46 100644
--- a/gcc/config/bpf/bpf.cc
+++ b/gcc/config/bpf/bpf.cc
@@ -221,6 +221,11 @@ bpf_option_override (void)
   && !(target_flags_explicit & MASK_BPF_CORE))
 target_flags |= MASK_BPF_CORE;
 
+  /* -gbtf implies -fprune-btf for BPF target.  */
+  if (btf_debuginfo_p ())
+SET_OPTION_IF_UNSET (&global_options, &global_options_set,
+flag_prune_btf, true);
+
   /* Determine available features from ISA setting (-mcpu=).  */
   if (bpf_has_jmpext == -1)
 bpf_has_jmpext = (bpf_isa >= ISA_V2);
diff --git a/gcc/config/bpf/core-builtins.cc b/gcc/config/bpf/core-builtins.cc
index d5a7de825ad..1b91b1c0d25 100644
--- a/gcc/config/bpf/core-builtins.cc
+++ b/gcc/config/bpf/core-builtins.cc
@@ -624,13 +624,20 @@ bpf_core_get_index (const tree node, bool *valid)
 
ALLOW_ENTRY_CAST is an input arguments and specifies if the function should
consider as valid expressions in which NODE entry is a cast expression (or
-   tree code nop_expr).  */
+   tree code nop_expr).
+
+   EXTRA_FN is a callback function to allow extra functionality with this
+   function traversal. Currently used for marking used type during expand
+   pass.  */
+
+typedef void (*extra_fn) (tree);
 
 static unsigned char
 compute_field_expr (tree node, unsigned int *accessors,
bool *valid,
tree *access_node,
-   bool allow_entry_cast = true)
+   bool allow_entry_cast = true,
+   extra_fn callback = NULL)
 {
   unsigned char n = 0;
   unsigned int fake_accessors[MAX_NR_ACCESSORS];
@@ -647,6 +654,9 @@ compute_field_expr (tree node, unsigned int *accessors,
 
   *access_node = node;
 
+  if (callback != NULL)
+callback (node);
+
   switch (TREE_CODE (node))
 {
 case INDIRECT_REF:
@@ -664,17 +674,19 @@ compute_field_expr (tree node, unsigned int *accessors,
 case COMPONENT_REF:
   n = compute_field_expr (TREE_OPERAND (node, 0), accessors,
  valid,
- access_node, false);
+ access_node, false, callback);
   accessors[n] = bpf_core_get_index (TREE_OPERAND (node

Re: [PATCH v2 1/6] ctf, btf: restructure CTF/BTF emission

[David Faust]

On 5/3/24 2:02 PM, Indu Bhagat wrote:
> On 5/2/24 10:11, David Faust wrote:
>> This commit makes some structural changes to the CTF/BTF debug info
>> emission.  In particular:
>>
>>   a) CTF is new always fully generated and emitted before any
>>  BTF-related procedures are run.  This means that BTF-related
>>  functions can change, even irreversibly, the shared in-memory
>>  representation used by the two formats without issue.
>>
>>   b) BTF generation has fewer entry points, and is cleanly divided
>>  into early_finish and finish.
>>
>>   c) BTF is now always emitted at finish (called from dwarf2out_finish),
>>  rather than being emitted at early_finish for targets other than
>>  BPF CO-RE.  Note that this change alone does not alter the contents
>>  of BTF at all, regardless of whether it would have previously been
>>  emitted at early_finish or finish.
>>
> 
> This will necessitate that we disallow -gbtf with -flto for non-BPF 
> targets.  Emitting BTF always at dwarf2out_finish will not work with LTO.

Yes, you're right.  I was not thinking about LTO when I wrote this.

I suppose the obvious fix is to undo "c)" above.  That change is not
really strictly necessary for the rest of the refactor in this series,
though we would lose the fix for PR debug/113566 in patch 4, since
fixing that requires us to generate some of the BTF at late finish time.
 Then BTF emission would be the same as it is currently:

BTF for BPF target, non-LTO-> (late) finish
BTF for BPF target, LTO-> forbidden by BPF back-end
BTF for non-BPF target -> early_finish

Alternatively, we could undo "c)" only for LTO builds, i.e.:

BTF for BPF target  non-LTO-> (late) finish
BTF for BPF target, LTO-> forbidden by BPF back-end
BTF for other targets, non-LTO -> (late) finish
BTF for other targets, LTO -> early_finish

which would allow fixing debug/113566 for non-LTO builds.  Then the
differences in BTF related to debug/113566 should only appear between
LTO and non-LTO builds, rather than between BPF and non-BPF targets.

IMO the latter (move emission back to early_finish for LTO builds) is a
little better.

Either way, I think the majority of code related to BTF generation from
later in this series should not need to change much if at all - only
whether those routines end up called at early or late finish.

> 
>> The changes are transparent to both CTF and BTF emission.
>>
>> gcc/
>>  * btfout.cc (btf_init_postprocess): Rename to...
>>  (btf_early_finish): ...this.
>>  (btf_output): Rename to...
>>  (btf_finish): ...this.
>>  * ctfc.h: Analogous changes.
>>  * dwarf2ctf.cc (ctf_debug_early_finish): Conditionally call
>>  btf_early_finish or ctf_finalize as appropriate.
>>  (ctf_debug_finish): Always call btf_finish here if generating
>>  BTF info.
>>  (ctf_debug_finalize, ctf_debug_init_postprocess): Delete.
>>  * dwarf2out.cc (dwarf2out_early_finish): Remove call to
>>  ctf_debug_init_postprocess.
>> ---
>>   gcc/btfout.cc| 28 +
>>   gcc/ctfc.h   |  4 ++--
>>   gcc/dwarf2ctf.cc | 54 +++-
>>   gcc/dwarf2out.cc |  2 --
>>   4 files changed, 42 insertions(+), 46 deletions(-)
>>
>> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
>> index 07f066a4706..1b6a9ed811f 100644
>> --- a/gcc/btfout.cc
>> +++ b/gcc/btfout.cc
>> @@ -1491,6 +1491,34 @@ btf_finalize (void)
>> tu_ctfc = NULL;
>>   }
>>   
>> +/* Initial entry point of BTF generation, called at early_finish () after
>> +   CTF information has possibly been output.  Translate all CTF information
>> +   to BTF, and do any processing that must be done early, such as creating
>> +   BTF_KIND_FUNC records.  */
>> +
>> +void
>> +btf_early_finish (void)
>> +{
>> +  btf_init_postprocess ();
>> +}
>> +
>> +/* Late entry point for BTF generation, called from dwarf2out_finish ().
>> +   Complete and emit BTF information.  */
>> +
>> +void
>> +btf_finish (const char * filename)
>> +{
>> +  btf_output (filename);
>> +
>> +  /* If compiling for BPF with CO-RE info, we cannot deallocate until after
>> + CO-RE information is created, which happens very late in BPF backend.
>> + Therefore, the deallocation (i.e. btf_finalize ()) is delayed until
>> + TARGET_ASM_FILE_END for BPF CO-RE.  */
>> +  if (!btf_with_core_debuginfo_p ())
>> +btf_finalize ();
>> +}
>> +
>> +
>>   /* Traversal funct

Re: [PATCH v2] bpf: make sure CO-RE relocs are typed with struct BTF_KIND_STRUCT

[David Faust]

On 10/14/24 11:04, Cupertino Miranda wrote:
> Hi everyone,
> 
> Here is the v2 for the patch in this thread:
>   https://gcc.gnu.org/pipermail/gcc-patches/2024-October/665378.html
> Please noticed that commit message was adapted to new content.
> 
> Regards,
> Cupertino

Hi Cupertino,

Thanks for the patch. The fix LGTM.

I have some comments about the new test below. But it is ok for now, and
I think addressing them properly is out of scope of this patch. This fix
will be good to have. So the patch is OK.

> 
> Based on observation within bpf-next selftests and comparisson of GCC
> and clang compiled code, the BPF loader expects all CO-RE relocations to
> point to BTF non const and non volatile type nodes.
> ---
>  gcc/btfout.cc |  2 +-
>  gcc/config/bpf/btfext-out.cc  |  7 
>  gcc/ctfc.h|  2 +
>  .../gcc.target/bpf/core-attr-const.c  | 40 +++
>  4 files changed, 50 insertions(+), 1 deletion(-)
>  create mode 100644 gcc/testsuite/gcc.target/bpf/core-attr-const.c
> 
> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
> index 8b91bde8798..24f62ec1a52 100644
> --- a/gcc/btfout.cc
> +++ b/gcc/btfout.cc
> @@ -167,7 +167,7 @@ get_btf_kind (uint32_t ctf_kind)
>  
>  /* Convenience wrapper around get_btf_kind for the common case.  */
>  
> -static uint32_t
> +uint32_t
>  btf_dtd_kind (ctf_dtdef_ref dtd)
>  {
>if (!dtd)
> diff --git a/gcc/config/bpf/btfext-out.cc b/gcc/config/bpf/btfext-out.cc
> index 095c35b894b..30266a2d384 100644
> --- a/gcc/config/bpf/btfext-out.cc
> +++ b/gcc/config/bpf/btfext-out.cc
> @@ -320,6 +320,13 @@ bpf_core_reloc_add (const tree type, const char * 
> section_name,
>ctf_container_ref ctfc = ctf_get_tu_ctfc ();
>ctf_dtdef_ref dtd = ctf_lookup_tree_type (ctfc, type);
>  
> +  /* Make sure CO-RE type is never the const or volatile version.  */
> +  if ((btf_dtd_kind (dtd) == BTF_KIND_CONST
> +   || btf_dtd_kind (dtd) == BTF_KIND_VOLATILE)
> +  && kind >= BPF_RELO_FIELD_BYTE_OFFSET
> +  && kind <= BPF_RELO_FIELD_RSHIFT_U64)
> +dtd = dtd->ref_type;
> +
>/* Buffer the access string in the auxiliary strtab.  */
>bpfcr->bpfcr_astr_off = 0;
>gcc_assert (accessor != NULL);
> diff --git a/gcc/ctfc.h b/gcc/ctfc.h
> index 41e1169f271..e5967f590f9 100644
> --- a/gcc/ctfc.h
> +++ b/gcc/ctfc.h
> @@ -465,4 +465,6 @@ extern void btf_mark_type_used (tree);
>  extern int ctfc_get_dtd_srcloc (ctf_dtdef_ref, ctf_srcloc_ref);
>  extern int ctfc_get_dvd_srcloc (ctf_dvdef_ref, ctf_srcloc_ref);
>  
> +extern uint32_t btf_dtd_kind (ctf_dtdef_ref dtd);
> +
>  #endif /* GCC_CTFC_H */
> diff --git a/gcc/testsuite/gcc.target/bpf/core-attr-const.c 
> b/gcc/testsuite/gcc.target/bpf/core-attr-const.c
> new file mode 100644
> index 000..da6113a3faf
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/bpf/core-attr-const.c
> @@ -0,0 +1,40 @@
> +/* Test to make sure CO-RE access relocs point to non const versions of the
> +   type.  */
> +
> +/* { dg-do compile } */
> +/* { dg-options "-O0 -dA -gbtf -mco-re -masm=normal" } */
> +
> +struct S {
> +  int a;
> +  int b;
> +  int c;
> +} __attribute__((preserve_access_index));
> +
> +void
> +func (struct S * s)
> +{
> +  int *x;
> +  int *y;
> +  int *z;
> +  struct S tmp;
> +  const struct S *cs = s;
> +  volatile struct S *vs = s;
> +
> +  /* 0:2 */
> +  x = &(s->c);
> +
> +  /* 0:1 */
> +  y = (int *) &(cs->b);
> +
> +  /* 0:1 */
> +  z = (int *) &(vs->b);
> +
> +  *x = 4;
> +  *y = 4;
> +  *z = 4;
> +}
> +
> +/* Both const and non const struct type should have the same bpfcr_type. */
> +/* { dg-final { scan-assembler-times "0x1\t# bpfcr_type \\(struct S\\)" 1 } 
> } */
> +/* { dg-final { scan-assembler-times "0x1\t# bpfcr_type \\(const struct 
> S\\)" 1 } } */
> +/* { dg-final { scan-assembler-times "0x1\t# bpfcr_type \\(volatile struct 
> S\\)" 1 } } */

The issue with these checks is that you are checking the exact BTF ID,
which is, in theory, not guaranteed to remain consistent between
different runs of the compiler. Even though in practice it does for now,
this could be broken by completely unrelated changes in the
DWARF-CTF-BTF pipeline.

But, I also understand that the purpose is to make sure these three
relocation records point to exactly the same BTF type. And currently we
do not have a good way to check that.

In addition, it is misleading that the asm comments refer to the
qualified types, which no longer match the actual BTF ID in the record.
I see that this is coming from btfext_out.cc:output_btfext_core_sections
which does the pretty-print from the original type expression rather
than the BTF ID that is used.

I wonder if it would instead make sense to use the btf_asm_type_ref
function from btfout.cc to asm the type ref (and the comment), which
will do so based on the type ID rather than the original type
expression.  Then for this test we can take a similar approach to the
BTF tests: drop the check on the conc

[PATCH 3/5] ctf: translate annotation DIEs to internal ctf

[David Faust]

Translate DW_TAG_GNU_annotation DIEs created for C attributes
btf_decl_tag and btf_type_tag into an in-memory representation in the
CTF/BTF container.  They will be output in BTF as BTF_KIND_DECL_TAG and
BTF_KIND_TYPE_TAG records.

The new CTF kinds used to represent these annotations, CTF_K_DECL_TAG
and CTF_K_TYPE_TAG, are expected to be formalized in the next version of
the CTF specification.  For now they only exist in memory as a
translation step to BTF, and are not emitted when generating CTF
information.

gcc/

* ctfc.cc (ctf_dtu_d_union_selector): Handle CTF_K_DECL_TAG and
CTF_K_TYPE_TAG.
(ctf_add_type_tag, ctf_add_decl_tag): New.
(ctf_add_variable): Return the new ctf_dvdef_ref rather than zero.
(new_ctf_container): Initialize new members.
(ctfc_delete_container): Deallocate new members.
* ctfc.h (ctf_dvdef, ctf_dvdef_t, ctf_dvdef_ref): Move forward
declarations earlier in file.
(ctf_tag_t): New typedef.
(ctf_dtdef): Add ctf_tag_t member to dtd_u union.
(ctf_dtu_d_union_enum): Add new CTF_DTU_D_TAG enumerator.
(ctf_container): Add ctfc_tags vector and ctfc_tags_map hash_map
members.
(ctf_add_type_tag, ctf_add_decl_tag): New function protos.
(ctf_add_variable): Change prototype return type to ctf_dvdef_ref.
* dwarf2ctf.cc (gen_ctf_type_tags, gen_ctf_decl_tags)
(gen_ctf_decl_tags_for_var): New static functions.
(gen_ctf_modifier_type): Handle type tags on types with cv-quals.
(gen_ctf_sou_type): Handle decl tags.
(gen_ctf_function_type): Likewise.
(gen_ctf_variable): Likewise.
(gen_ctf_function): Likewise.
(is_cvr_die): New helper function.
(gen_ctf_type): Handle type tags.

include/

* ctf.h (CTF_K_DECL_TAG, CTF_K_TYPE_TAG): New defines.
---
 gcc/ctfc.cc  |  66 -
 gcc/ctfc.h   |  41 +--
 gcc/dwarf2ctf.cc | 180 +--
 include/ctf.h|   4 ++
 4 files changed, 277 insertions(+), 14 deletions(-)

diff --git a/gcc/ctfc.cc b/gcc/ctfc.cc
index 8f531ffebf8..8fca36caa1e 100644
--- a/gcc/ctfc.cc
+++ b/gcc/ctfc.cc
@@ -107,6 +107,9 @@ ctf_dtu_d_union_selector (ctf_dtdef_ref ctftype)
   return CTF_DTU_D_ARGUMENTS;
 case CTF_K_SLICE:
   return CTF_DTU_D_SLICE;
+case CTF_K_DECL_TAG:
+case CTF_K_TYPE_TAG:
+  return CTF_DTU_D_TAG;
 default:
   /* The largest member as default.  */
   return CTF_DTU_D_ARRAY;
@@ -445,6 +448,54 @@ ctf_add_reftype (ctf_container_ref ctfc, uint32_t flag, 
ctf_dtdef_ref ref,
   return dtd;
 }
 
+ctf_dtdef_ref
+ctf_add_type_tag (ctf_container_ref ctfc, uint32_t flag, const char *value,
+ ctf_dtdef_ref ref_dtd)
+{
+  ctf_dtdef_ref dtd;
+   /* Create a DTD for the tag, but do not place it in the regular types list;
+  ctf format does not (yet) encode tags.  */
+  dtd = ggc_cleared_alloc ();
+
+  dtd->dtd_name = ctf_add_string (ctfc, value, &(dtd->dtd_data.ctti_name),
+ CTF_AUX_STRTAB);
+  /* Tags uniquely have a one-to-many relationship where one die may translate
+ to many ctf types.  Therefore we cannot key the tags by die.  */
+  dtd->dtd_key = NULL;
+  dtd->ref_type = ref_dtd;
+  dtd->dtd_data.ctti_info = CTF_TYPE_INFO (CTF_K_TYPE_TAG, flag, 0);
+  dtd->dtd_u.dtu_tag.ref_var = NULL; /* Not used for type tags.  */
+  dtd->dtd_u.dtu_tag.component_idx = -1U; /* Not used for type tags.  */
+
+  /* Insert tag directly into the tag list.  Type ID will be assigned later.  
*/
+  vec_safe_push (ctfc->ctfc_tags, dtd);
+  return dtd;
+}
+
+ctf_dtdef_ref
+ctf_add_decl_tag (ctf_container_ref ctfc, uint32_t flag, const char *value,
+ ctf_dtdef_ref ref_dtd, uint32_t comp_idx)
+{
+   ctf_dtdef_ref dtd;
+   /* Create a DTD for the tag, but do not place it in the regular types list;
+  ctf format does not (yet) encode tags.  */
+  dtd = ggc_cleared_alloc ();
+
+  dtd->dtd_name = ctf_add_string (ctfc, value, &(dtd->dtd_data.ctti_name),
+ CTF_AUX_STRTAB);
+  /* Tags uniquely have a one-to-many relationship where one die may translate
+ to many ctf types.  Therefore we cannot key the tags by die.  */
+  dtd->dtd_key = NULL;
+  dtd->ref_type = ref_dtd;
+  dtd->dtd_data.ctti_info = CTF_TYPE_INFO (CTF_K_DECL_TAG, flag, 0);
+  dtd->dtd_u.dtu_tag.ref_var = NULL;
+  dtd->dtd_u.dtu_tag.component_idx = comp_idx;
+
+  /* Insert tag directly into the tag list.  Type ID will be assigned later.  
*/
+  vec_safe_push (ctfc->ctfc_tags, dtd);
+  return dtd;
+}
+
 ctf_dtdef_ref
 ctf_add_forward (ctf_container_ref ctfc, uint32_t flag, const char * name,
 uint32_t kind, dw_die_ref die)
@@ -691,12 +742,12 @@ ctf_add_member_offset (ctf_container_ref ctfc, dw_die_ref 
sou,
   return 0;
 }
 
-int
+ctf_dvdef_ref
 ctf_add_variable (ctf_container_ref ctfc, const char * name, ctf_dtdef_ref ref,
 

[PATCH 1/5] c-family: add btf_type_tag and btf_decl_tag attributes

[David Faust]

Add two new c-family attributes, "btf_decl_tag" and "btf_type_tag" along
with a simple shared handler for them.

gcc/c-family/

* c-attribs.cc (c_common_attribute_table): Add btf_decl_tag and
btf_type_tag attributes.
(handle_btf_tag_attribute): New handler for both new attributes.
---
 gcc/c-family/c-attribs.cc | 25 -
 1 file changed, 24 insertions(+), 1 deletion(-)

diff --git a/gcc/c-family/c-attribs.cc b/gcc/c-family/c-attribs.cc
index 4dd2eecbea5..76374413f9e 100644
--- a/gcc/c-family/c-attribs.cc
+++ b/gcc/c-family/c-attribs.cc
@@ -187,6 +187,8 @@ static tree handle_fd_arg_attribute (tree *, tree, tree, 
int, bool *);
 static tree handle_flag_enum_attribute (tree *, tree, tree, int, bool *);
 static tree handle_null_terminated_string_arg_attribute (tree *, tree, tree, 
int, bool *);
 
+static tree handle_btf_tag_attribute (tree *, tree, tree, int, bool *);
+
 /* Helper to define attribute exclusions.  */
 #define ATTR_EXCL(name, function, type, variable)  \
   { name, function, type, variable }
@@ -635,7 +637,11 @@ const struct attribute_spec c_common_gnu_attributes[] =
   { "flag_enum", 0, 0, false, true, false, false,
  handle_flag_enum_attribute, NULL },
   { "null_terminated_string_arg", 1, 1, false, true, true, false,
- handle_null_terminated_string_arg_attribute, NULL}
+ handle_null_terminated_string_arg_attribute, 
NULL},
+  { "btf_type_tag",  1, 1, false, true, false, false,
+ handle_btf_tag_attribute, NULL},
+  { "btf_decl_tag",  1, 1, true, false, false, false,
+ handle_btf_tag_attribute, NULL}
 };
 
 const struct scoped_attribute_specs c_common_gnu_attribute_table =
@@ -5069,6 +5075,23 @@ handle_null_terminated_string_arg_attribute (tree *node, 
tree name, tree args,
   return NULL_TREE;
 }
 
+/* Handle the "btf_decl_tag" and "btf_type_tag" attributes.  */
+
+static tree
+handle_btf_tag_attribute (tree * ARG_UNUSED (node), tree name, tree args,
+ int ARG_UNUSED (flags), bool *no_add_attrs)
+{
+  if (!args)
+*no_add_attrs = true;
+  else if (TREE_CODE (TREE_VALUE (args)) != STRING_CST)
+{
+  error ("%qE attribute requires a string", name);
+  *no_add_attrs = true;
+}
+
+  return NULL_TREE;
+}
+
 /* Handle the "nonstring" variable attribute.  */
 
 static tree
-- 
2.45.2

[PATCH 5/5] doc: document btf_type_tag and btf_decl_tag attributes

[David Faust]

gcc/

* doc/extend.texi (Common Variable Attributes): Document new
btf_decl_tag attribute.
(Common Type Attributes): Document new btf_type_tag attribute.
---
 gcc/doc/extend.texi | 68 +
 1 file changed, 68 insertions(+)

diff --git a/gcc/doc/extend.texi b/gcc/doc/extend.texi
index 42bd567119d..fd8f2425947 100644
--- a/gcc/doc/extend.texi
+++ b/gcc/doc/extend.texi
@@ -7920,6 +7920,41 @@ align them on any target.
 The @code{aligned} attribute can also be used for functions
 (@pxref{Common Function Attributes}.)
 
+@cindex @code{btf_decl_tag} variable attribute
+@item btf_decl_tag (@var{argument})
+The @code{btf_decl_tag} attribute may be used to associate variable
+declarations, struct or union member declarations, function
+declarations, or function parameter declarations with arbitrary strings.
+These strings are not interpreted by the compiler in any way, and have
+no effect on code generation.  Instead, these user-provided strings
+are recorded in DWARF (via @code{DW_AT_GNU_annotation} and
+@code{DW_TAG_GNU_annotation} extensions) and BTF information (via
+@code{BTF_KIND_DECL_TAG} records), and associated to the attributed
+declaration.  If neither DWARF nor BTF information is generated, the
+attribute has no effect.
+
+The argument is treated as an ordinary string in the source language
+with no additional special rules.
+
+The attribute may be supplied multiple times for a single declaration,
+in which case each distinct argument string will be recorded in a
+separate DIE or BTF record, each associated to the declaration.  For
+a single declaration with multiple @code{btf_decl_tag} attributes,
+the order of the @code{DW_TAG_GNU_annotation} DIEs produced is not
+guaranteed to maintain the order of attributes in the source code.
+
+For example:
+
+@smallexample
+int * foo __attribute__ ((btf_decl_tag ("__percpu")));
+@end smallexample
+
+@noindent
+when compiled with @code{-gbtf} results in an additional
+@code{BTF_KIND_DECL_TAG} BTF record to be emitted in the BTF info,
+associating the string ``__rcu'' with the normal @code{BTF_KIND_VAR}
+record for the variable ``foo''.
+
 @cindex @code{counted_by} variable attribute
 @item counted_by (@var{count})
 The @code{counted_by} attribute may be attached to the C99 flexible array
@@ -9109,6 +9144,39 @@ is given by the product of arguments 1 and 2, and that
 @code{malloc_type}, like the standard C function @code{malloc},
 returns an object whose size is given by argument 1 to the function.
 
+@cindex @code{btf_type_tag} type attribute
+@item btf_type_tag (@var{argument})
+The @code{btf_type_tag} attribute may be used to associate (to ``tag'')
+particular types with arbitrary string annotations.  These annotations
+are recorded in debugging info by supported debug formats, currently
+DWARF (via @code{DW_AT_GNU_annotation} and @code{DW_TAG_GNU_annotation}
+extensions) and BTF (via @code{BTF_KIND_TYPE_TAG} records).  These
+annotation string are not interpreted by the compiler in any way, and
+have no effect on code generation.  If neither DWARF nor BTF
+information is generated, the attribute has no effect.
+
+The argument is treated as an ordinary string in the source language
+with no additional special rules.
+
+The attribute may be supplied multiple times for a single declaration,
+in which case each distinct argument string will be recorded in a
+separate DIE or BTF record, each associated to the type.  For a single
+type with multiple @code{btf_decl_tag} attributes, the order of the
+@code{DW_TAG_GNU_annotation} DIEs produced is not guaranteed to
+maintain the order of attributes in the source code.
+
+For example
+
+@smallexample
+int * __attribute__ ((btf_type_tag ("__user"))) foo;
+@end smallexample
+
+@noindent
+associates the string ``__user'' to the pointer-to-integer type of
+the declaration.  This string will be recorded in DWARF and/or BTF
+information associated with the appropriate pointer type DIE or
+@code{BTF_KIND_PTR} record.
+
 @cindex @code{copy} type attribute
 @item copy
 @itemx copy (@var{expression})
-- 
2.45.2

[PATCH 4/5] btf: generate and output DECL_TAG and TYPE_TAG records

[David Faust]

Support the btf_decl_tag and btf_type_tag attributes in BTF by creating
and emitting BTF_KIND_DECL_TAG and BTF_KIND_TYPE_TAG records,
respectively, for them.

Some care is required when -gprune-btf is in effect to avoid emitting
decl or type tags for declarations or types which have been pruned and
will not be emitted in BTF.

gcc/
* btfout.cc (get_btf_kind): Handle DECL_TAG and TYPE_TAG kinds.
(btf_calc_num_vbytes): Likewise.
(btf_asm_type): Likewise.
(output_asm_btf_vlen_bytes): Likewise.
(output_btf_tags): New.
(btf_output): Call it here.
(btf_add_used_type): Replace with simple wrapper around...
(btf_add_used_type_1): ...the implementation.  Handle
BTF_KIND_DECL_TAG and BTF_KIND_TYPE_TAG.
(btf_add_vars): Update btf_add_used_type call.
(btf_assign_tag_ids): New.
(btf_mark_type_used): Update btf_add_used_type call.
(btf_collect_pruned_types): Likewise.  Handle type and decl tags.
(btf_finish): Call btf_assign_tag_ids.

gcc/testsuite/
* gcc.dg/debug/btf/btf-decl-tag-1.c: New test.
* gcc.dg/debug/btf/btf-decl-tag-2.c: New test.
* gcc.dg/debug/btf/btf-decl-tag-3.c: New test.
* gcc.dg/debug/btf/btf-decl-tag-4.c: New test.
* gcc.dg/debug/btf/btf-type-tag-1.c: New test.
* gcc.dg/debug/btf/btf-type-tag-2.c: New test.
* gcc.dg/debug/btf/btf-type-tag-3.c: New test.
* gcc.dg/debug/btf/btf-type-tag-4.c: New test.
* gcc.dg/debug/btf/btf-type-tag-c2x-1.c: New test.

include/
* btf.h (BTF_KIND_DECL_TAG, BTF_KIND_TYPE_TAG) New defines.
(struct btf_decl_tag): New.
---
 gcc/btfout.cc | 176 +++---
 .../gcc.dg/debug/btf/btf-decl-tag-1.c |  14 ++
 .../gcc.dg/debug/btf/btf-decl-tag-2.c |  22 +++
 .../gcc.dg/debug/btf/btf-decl-tag-3.c |  22 +++
 .../gcc.dg/debug/btf/btf-decl-tag-4.c |  34 
 .../gcc.dg/debug/btf/btf-type-tag-1.c |  27 +++
 .../gcc.dg/debug/btf/btf-type-tag-2.c |  17 ++
 .../gcc.dg/debug/btf/btf-type-tag-3.c |  21 +++
 .../gcc.dg/debug/btf/btf-type-tag-4.c |  25 +++
 .../gcc.dg/debug/btf/btf-type-tag-c2x-1.c |  23 +++
 include/btf.h |  14 ++
 11 files changed, 371 insertions(+), 24 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-decl-tag-1.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-decl-tag-2.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-decl-tag-3.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-decl-tag-4.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-type-tag-1.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-type-tag-2.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-type-tag-3.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-type-tag-4.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-type-tag-c2x-1.c

diff --git a/gcc/btfout.cc b/gcc/btfout.cc
index 083ca48d627..e8190f685f9 100644
--- a/gcc/btfout.cc
+++ b/gcc/btfout.cc
@@ -141,6 +141,8 @@ get_btf_kind (uint32_t ctf_kind)
 case CTF_K_VOLATILE: return BTF_KIND_VOLATILE;
 case CTF_K_CONST:return BTF_KIND_CONST;
 case CTF_K_RESTRICT: return BTF_KIND_RESTRICT;
+case CTF_K_DECL_TAG: return BTF_KIND_DECL_TAG;
+case CTF_K_TYPE_TAG: return BTF_KIND_TYPE_TAG;
 default:;
 }
   return BTF_KIND_UNKN;
@@ -217,6 +219,7 @@ btf_calc_num_vbytes (ctf_dtdef_ref dtd)
 case BTF_KIND_CONST:
 case BTF_KIND_RESTRICT:
 case BTF_KIND_FUNC:
+case BTF_KIND_TYPE_TAG:
 /* These kinds have no vlen data.  */
   break;
 
@@ -256,6 +259,10 @@ btf_calc_num_vbytes (ctf_dtdef_ref dtd)
   vlen_bytes += vlen * sizeof (struct btf_var_secinfo);
   break;
 
+case BTF_KIND_DECL_TAG:
+  vlen_bytes += sizeof (struct btf_decl_tag);
+  break;
+
 default:
   break;
 }
@@ -452,6 +459,20 @@ btf_asm_type (ctf_dtdef_ref dtd)
 and should write 0.  */
   dw2_asm_output_data (4, 0, "(unused)");
   return;
+case BTF_KIND_DECL_TAG:
+  {
+   if (dtd->ref_type)
+ break;
+   else if (dtd->dtd_u.dtu_tag.ref_var)
+ {
+   /* ref_type is NULL for decl tag attached to a variable.  */
+   ctf_dvdef_ref dvd = dtd->dtd_u.dtu_tag.ref_var;
+   dw2_asm_output_data (4, dvd->dvd_id,
+"btt_type: (BTF_KIND_VAR '%s')",
+dvd->dvd_name);
+   return;
+ }
+  }
 default:
   break;
 }
@@ -801,6 +822,12 @@ output_asm_btf_vlen_bytes (ctf_container_ref ctfc, 
ctf_dtdef_ref dtd)
 at this point.  */
   gcc_unreachable ();
 
+case BTF_KIND_DECL_TAG:
+  dw2_asm_output_data (4, dtd->dtd_u.dtu_tag.component_idx,
+  "component_idx=%d",
+  dtd->dtd_u.dtu_tag.component_idx);
+  break;
+