[pushed] diagnostics: fix missing init of set_locations_cb

[David Malcolm]

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-4366-gc64693fb885f21.

gcc/ChangeLog:
* diagnostic.cc (diagnostic_initialize): Initialize
set_locations_cb to nullptr.
---
 gcc/diagnostic.cc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/gcc/diagnostic.cc b/gcc/diagnostic.cc
index 00183b10700..28ab74ff23e 100644
--- a/gcc/diagnostic.cc
+++ b/gcc/diagnostic.cc
@@ -245,6 +245,7 @@ diagnostic_initialize (diagnostic_context *context, int 
n_opts)
   context->begin_group_cb = NULL;
   context->end_group_cb = NULL;
   context->final_cb = default_diagnostic_final_cb;
+  context->set_locations_cb = nullptr;
   context->ice_handler_cb = NULL;
   context->includes_seen = NULL;
   context->m_client_data_hooks = NULL;
-- 
2.26.3

[pushed] diagnostics: add diagnostic_output_format class

[David Malcolm]

Eliminate various global variables in the json/sarif output code by
bundling together callbacks and state into a new diagnostic_output_format
class, with per-output-format subclasses.

No functional change intended.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-4368-g140820265d96b0.

gcc/ChangeLog:
* diagnostic-format-json.cc (toplevel_array): Remove global in
favor of json_output_format::m_top_level_array.
(cur_group): Likewise, for json_output_format::m_cur_group.
(cur_children_array): Likewise, for
json_output_format::m_cur_children_array.
(class json_output_format): New.
(json_begin_diagnostic): Remove, in favor of
json_output_format::on_begin_diagnostic.
(json_end_diagnostic): Convert to...
(json_output_format::on_end_diagnostic): ...this.
(json_begin_group): Remove, in favor of
json_output_format::on_begin_group.
(json_end_group): Remove, in favor of
json_output_format::on_end_group.
(json_flush_to_file): Remove, in favor of
json_output_format::flush_to_file.
(json_stderr_final_cb): Remove, in favor of json_output_format
dtor.
(json_output_base_file_name): Remove global.
(class json_stderr_output_format): New.
(json_file_final_cb): Remove.
(class json_file_output_format): New.
(json_emit_diagram): Remove.
(diagnostic_output_format_init_json): Update.
(diagnostic_output_format_init_json_file): Update.
* diagnostic-format-sarif.cc (the_builder): Remove this global,
moving to a field of the sarif_output_format.
(sarif_builder::maybe_make_artifact_content_object): Use the
context's m_file_cache.
(get_source_lines): Convert to...
(sarif_builder::get_source_lines): ...this, using context's
m_file_cache.
(sarif_begin_diagnostic): Remove, in favor of
sarif_output_format::on_begin_diagnostic.
(sarif_end_diagnostic): Remove, in favor of
sarif_output_format::on_end_diagnostic.
(sarif_begin_group): Remove, in favor of
sarif_output_format::on_begin_group.
(sarif_end_group): Remove, in favor of
sarif_output_format::on_end_group.
(sarif_flush_to_file): Delete.
(sarif_stderr_final_cb): Delete.
(sarif_output_base_file_name): Delete.
(sarif_file_final_cb): Delete.
(class sarif_output_format): New.
(sarif_emit_diagram): Delete.
(class sarif_stream_output_format): New.
(class sarif_file_output_format): New.
(diagnostic_output_format_init_sarif): Update.
(diagnostic_output_format_init_sarif_stderr): Update.
(diagnostic_output_format_init_sarif_file): Update.
(diagnostic_output_format_init_sarif_stream): Update.
* diagnostic-show-locus.cc (diagnostic_show_locus): Update.
* diagnostic.cc (default_diagnostic_final_cb): Delete, moving to
diagnostic_text_output_format's dtor.
(diagnostic_initialize): Update, making a new instance of
diagnostic_text_output_format.
(diagnostic_finish): Delete m_output_format, rather than calling
final_cb.
(diagnostic_report_diagnostic): Assert that m_output_format is
non-NULL.  Replace call to begin_group_cb with call to
m_output_format->on_begin_group.  Replace call to
diagnostic_starter with call to
m_output_format->on_begin_diagnostic.  Replace call to
diagnostic_finalizer with call to
m_output_format->on_end_diagnostic.
(diagnostic_emit_diagram): Replace both optional call to
m_diagrams.m_emission_cb and default implementation with call to
m_output_format->on_diagram.  Move default implementation to
diagnostic_text_output_format::on_diagram.
(auto_diagnostic_group::~auto_diagnostic_group): Replace call to
end_group_cb with call to m_output_format->on_end_group.
(diagnostic_text_output_format::~diagnostic_text_output_format):
New, based on default_diagnostic_final_cb.
(diagnostic_text_output_format::on_begin_diagnostic): New, based
on code from diagnostic_report_diagnostic.
(diagnostic_text_output_format::on_end_diagnostic): Likewise.
(diagnostic_text_output_format::on_diagram): New, based on code
from diagnostic_emit_diagram.
* diagnostic.h (class diagnostic_output_format): New.
(class diagnostic_text_output_format): New.
(diagnostic_context::begin_diagnostic): Move to...
(diagnostic_context::m_text_callbacks::begin_diagnostic): ...here.
(diagnostic_context::start_span): Move to...
(diagnostic_context::m_text_callbacks::start_span): ...here.
(diagnostic_context::end_diagnostic): Move to...
(diagnostic_context::m_text_callbacks::end_diagnostic): ...here.
(diagn

[pushed] diagnostics: group together source printing fields of diagnostic_context

[David Malcolm]

struct diagnostic_context has > 60 fields.

Try to tame some of the complexity by grouping together the 8
source-printing fields into a struct, the "m_source_printing" field.

No functional change intended.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-4367-gc5c565eff6277a.

gcc/ada/ChangeLog:
* gcc-interface/misc.cc (gnat_post_options): Update for renaming
of diagnostic_context's show_caret to m_source_printing.enabled.

gcc/analyzer/ChangeLog:
* program-point.cc: Update for grouping of source printing fields
within diagnostic_context.

gcc/c-family/ChangeLog:
* c-common.cc (maybe_add_include_fixit): Update for renaming of
diagnostic_context's show_caret to m_source_printing.enabled.
* c-opts.cc (c_common_init_options): Update for renaming of
diagnostic_context's colorize_source_p to
m_source_printing.colorize_source_p.

gcc/ChangeLog:
* diagnostic-show-locus.cc: Update for reorganization of
source-printing fields of diagnostic_context.
* diagnostic.cc (diagnostic_set_caret_max_width): Likewise.
(diagnostic_initialize): Likewise.
* diagnostic.h (diagnostic_context::show_caret): Move to...
(diagnostic_context::m_source_printing::enabled): ...here.
(diagnostic_context::caret_max_width): Move to...
(diagnostic_context::m_source_printing::max_width): ...here.
(diagnostic_context::caret_chars): Move to...
(diagnostic_context::m_source_printing::caret_chars): ...here.
(diagnostic_context::colorize_source_p): Move to...
(diagnostic_context::m_source_printing::colorize_source_p): ...here.
(diagnostic_context::show_labels_p): Move to...
(diagnostic_context::m_source_printing::show_labels_p): ...here.
(diagnostic_context::show_line_numbers_p): Move to...
(diagnostic_context::m_source_printing::show_line_numbers_p): ...here.
(diagnostic_context::min_margin_width): Move to...
(diagnostic_context::m_source_printing::min_margin_width): ...here.
(diagnostic_context::show_ruler_p): Move to...
(diagnostic_context::m_source_printing::show_ruler_p): ...here.
(diagnostic_same_line): Update for above changes.
* opts.cc (common_handle_option): Update for reorganization of
source-printing fields of diagnostic_context.
* selftest-diagnostic.cc
(test_diagnostic_context::test_diagnostic_context): Likewise.
* toplev.cc (general_init): Likewise.
* tree-diagnostic-path.cc (struct event_range): Likewise.

gcc/fortran/ChangeLog:
* error.cc (gfc_diagnostic_starter): Update for reorganization of
source-printing fields of diagnostic_context.
(gfc_diagnostics_init): Likewise.
(gfc_diagnostics_finish): Likewise.

gcc/testsuite/ChangeLog:
* gcc.dg/plugin/diagnostic_plugin_show_trees.c: Update for
reorganization of source-printing fields of diagnostic_context.
* gcc.dg/plugin/diagnostic_plugin_test_inlining.c: Likewise.
* gcc.dg/plugin/diagnostic_plugin_test_paths.c: Likewise.
* gcc.dg/plugin/diagnostic_plugin_test_show_locus.c: Likewise.
* gcc.dg/plugin/diagnostic_plugin_test_string_literals.c: Likewise.
* gcc.dg/plugin/diagnostic_plugin_test_tree_expression_range.c:
Likewise.
---
 gcc/ada/gcc-interface/misc.cc |  2 +-
 gcc/analyzer/program-point.cc |  4 +-
 gcc/c-family/c-common.cc  |  2 +-
 gcc/c-family/c-opts.cc|  2 +-
 gcc/diagnostic-show-locus.cc  | 93 ++-
 gcc/diagnostic.cc | 16 ++--
 gcc/diagnostic.h  | 73 ---
 gcc/fortran/error.cc  | 10 +-
 gcc/opts.cc   |  8 +-
 gcc/selftest-diagnostic.cc|  8 +-
 .../plugin/diagnostic_plugin_show_trees.c |  2 +-
 .../plugin/diagnostic_plugin_test_inlining.c  |  2 +-
 .../plugin/diagnostic_plugin_test_paths.c |  2 +-
 .../diagnostic_plugin_test_show_locus.c   | 26 +++---
 .../diagnostic_plugin_test_string_literals.c  |  2 +-
 ...nostic_plugin_test_tree_expression_range.c |  2 +-
 gcc/toplev.cc |  8 +-
 gcc/tree-diagnostic-path.cc   |  2 +-
 18 files changed, 140 insertions(+), 124 deletions(-)

diff --git a/gcc/ada/gcc-interface/misc.cc b/gcc/ada/gcc-interface/misc.cc
index 3b21bf5b43a..269c15e4b0d 100644
--- a/gcc/ada/gcc-interface/misc.cc
+++ b/gcc/ada/gcc-interface/misc.cc
@@ -269,7 +269,7 @@ gnat_post_options (const char **pfilename ATTRIBUTE_UNUSED)
 
   /* No caret by default for Ada.  */
   if (!OPTION_SET_P (flag_diagnostics_show_caret))
-global_dc->show_caret = false;
+global_dc->m_source_printing.enabled = false;
 
   /* Copy global settings to local versions.

[pushed] diagnostics: add ctors to text_info; add m_ prefixes to fields

[David Malcolm]

No functional change intended.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-4379-gc44ca7c01226e0.

gcc/ada/ChangeLog:
* gcc-interface/misc.cc: Use text_info ctor.

gcc/analyzer/ChangeLog:
* analyzer-logging.cc (logger::log_va_partial): Use text_info
ctor.
* analyzer.cc (make_label_text): Likewise.
(make_label_text_n): Likewise.
* pending-diagnostic.cc (evdesc::event_desc::formatted_print):
Likewise.

gcc/c/ChangeLog:
* c-objc-common.cc (c_tree_printer): Update for "m_" prefixes to
text_info fields.

gcc/cp/ChangeLog:
* error.cc: Update for "m_" prefixes to text_info fields.

gcc/d/ChangeLog:
* d-diagnostic.cc (d_diagnostic_report_diagnostic): Use text_info
ctor.

gcc/ChangeLog:
* diagnostic.cc (diagnostic_set_info_translated): Update for "m_"
prefixes to text_info fields.
(diagnostic_report_diagnostic): Likewise.
(verbatim): Use text_info ctor.
(simple_diagnostic_path::add_event): Likewise.
(simple_diagnostic_path::add_thread_event): Likewise.
* dumpfile.cc (dump_pretty_printer::decode_format): Update for
"m_" prefixes to text_info fields.
(dump_context::dump_printf_va): Use text_info ctor.
* graphviz.cc (graphviz_out::graphviz_out): Use text_info ctor.
(graphviz_out::print): Likewise.
* opt-problem.cc (opt_problem::opt_problem): Likewise.
* pretty-print.cc (pp_format): Update for "m_" prefixes to
text_info fields.
(pp_printf): Use text_info ctor.
(pp_verbatim): Likewise.
(assert_pp_format_va): Likewise.
* pretty-print.h (struct text_info): Add ctors.  Add "m_" prefix
to all fields.
* text-art/styled-string.cc (styled_string::from_fmt_va): Use
text_info ctor.
* tree-diagnostic.cc (default_tree_printer): Update for "m_"
prefixes to text_info fields.
* tree-pretty-print.h (pp_ti_abstract_origin): Likewise.

gcc/fortran/ChangeLog:
* error.cc (gfc_format_decoder): Update for "m_" prefixes to
text_info fields.
---
 gcc/ada/gcc-interface/misc.cc  |  5 +--
 gcc/analyzer/analyzer-logging.cc   |  5 +--
 gcc/analyzer/analyzer.cc   | 15 +--
 gcc/analyzer/pending-diagnostic.cc |  7 +---
 gcc/c/c-objc-common.cc |  4 +-
 gcc/cp/error.cc|  8 ++--
 gcc/d/d-diagnostic.cc  |  6 +--
 gcc/diagnostic.cc  | 33 +---
 gcc/dumpfile.cc| 13 +++---
 gcc/fortran/error.cc   |  2 +-
 gcc/graphviz.cc| 10 +
 gcc/opt-problem.cc |  6 +--
 gcc/pretty-print.cc| 63 --
 gcc/pretty-print.h | 24 +---
 gcc/text-art/styled-string.cc  |  5 +--
 gcc/tree-diagnostic.cc |  6 +--
 gcc/tree-pretty-print.h|  2 +-
 17 files changed, 81 insertions(+), 133 deletions(-)

diff --git a/gcc/ada/gcc-interface/misc.cc b/gcc/ada/gcc-interface/misc.cc
index 269c15e4b0d..453ae8087a6 100644
--- a/gcc/ada/gcc-interface/misc.cc
+++ b/gcc/ada/gcc-interface/misc.cc
@@ -293,7 +293,6 @@ static void
 internal_error_function (diagnostic_context *context, const char *msgid,
 va_list *ap)
 {
-  text_info tinfo;
   char *buffer, *p, *loc;
   String_Template temp, temp_loc;
   String_Pointer sp, sp_loc;
@@ -309,9 +308,7 @@ internal_error_function (diagnostic_context *context, const 
char *msgid,
   pp_clear_output_area (context->printer);
 
   /* Format the message into the pretty-printer.  */
-  tinfo.format_spec = msgid;
-  tinfo.args_ptr = ap;
-  tinfo.err_no = errno;
+  text_info tinfo (msgid, ap, errno);
   pp_format_verbatim (context->printer, &tinfo);
 
   /* Extract a (writable) pointer to the formatted text.  */
diff --git a/gcc/analyzer/analyzer-logging.cc b/gcc/analyzer/analyzer-logging.cc
index b78481c4098..ddfbb5b4c04 100644
--- a/gcc/analyzer/analyzer-logging.cc
+++ b/gcc/analyzer/analyzer-logging.cc
@@ -144,10 +144,7 @@ logger::log_partial (const char *fmt, ...)
 void
 logger::log_va_partial (const char *fmt, va_list *ap)
 {
-  text_info text;
-  text.format_spec = fmt;
-  text.args_ptr = ap;
-  text.err_no = 0;
+  text_info text (fmt, ap, 0);
   pp_format (m_pp, &text);
   pp_output_formatted_text (m_pp);
 }
diff --git a/gcc/analyzer/analyzer.cc b/gcc/analyzer/analyzer.cc
index 94c5cf242b2..9d4bc788f31 100644
--- a/gcc/analyzer/analyzer.cc
+++ b/gcc/analyzer/analyzer.cc
@@ -425,19 +425,13 @@ make_label_text (bool can_colorize, const char *fmt, ...)
   if (!can_colorize)
 pp_show_color (pp) = false;
 
-  text_info ti;
   rich_location rich_loc (line_table, UNKNOWN_LOCATION);
 
   va_list ap;
 
   va_start (ap, fmt);
 
-  ti.format_spec = _(fmt);
-  ti.args_ptr = ≈
-  ti.err_no = 0;
-  ti.x_data = NULL;
-  ti.m_richloc = &rich_loc;
-
+  text_info t

[PATCH] c++: print source code in print_instantiation_partial_context_line

[David Malcolm]

As mentioned in my Cauldron talk, this patch adds a call to
diagnostic_show_locus to the "required from here" messages
in print_instantiation_partial_context_line, so that e.g., rather
than the rather mystifying:

In file included from ../x86_64-pc-linux-gnu/libstdc++-v3/include/memory:78,
 from ../../src/demo-1.C:1:
../x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h: In instantiation 
of ‘std::__detail::__unique_ptr_t<_Tp> std::make_unique(_Args&& ...) [with _Tp 
= bar; _Args = {}; __detail::__unique_ptr_t<_Tp> = 
__detail::__unique_ptr_t]’:
../../src/demo-1.C:15:32:   required from here
../x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h:1066:30: error: 
no matching function for call to ‘bar::bar()’
 1066 | { return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
  |  ^~~
../../src/demo-1.C:10:3: note: candidate: ‘bar::bar(int)’
   10 |   bar (int);
  |   ^~~
../../src/demo-1.C:10:3: note:   candidate expects 1 argument, 0 provided
../../src/demo-1.C:7:7: note: candidate: ‘constexpr bar::bar(const bar&)’
7 | class bar : public foo
  |   ^~~
../../src/demo-1.C:7:7: note:   candidate expects 1 argument, 0 provided
../../src/demo-1.C:7:7: note: candidate: ‘constexpr bar::bar(bar&&)’
../../src/demo-1.C:7:7: note:   candidate expects 1 argument, 0 provided

we emit:

In file included from ../x86_64-pc-linux-gnu/libstdc++-v3/include/memory:78,
 from ../../src/demo-1.C:1:
../x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h: In instantiation 
of ‘std::__detail::__unique_ptr_t<_Tp> std::make_unique(_Args&& ...) [with _Tp 
= bar; _Args = {}; __detail::__unique_ptr_t<_Tp> = 
__detail::__unique_ptr_t]’:
../../src/demo-1.C:15:32:   required from here
   15 |   return std::make_unique ();
  |  ~~^~
../x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h:1066:30: error: 
no matching function for call to ‘bar::bar()’
 1066 | { return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
  |  ^~~
../../src/demo-1.C:10:3: note: candidate: ‘bar::bar(int)’
   10 |   bar (int);
  |   ^~~
../../src/demo-1.C:10:3: note:   candidate expects 1 argument, 0 provided
../../src/demo-1.C:7:7: note: candidate: ‘constexpr bar::bar(const bar&)’
7 | class bar : public foo
  |   ^~~
../../src/demo-1.C:7:7: note:   candidate expects 1 argument, 0 provided
../../src/demo-1.C:7:7: note: candidate: ‘constexpr bar::bar(bar&&)’
../../src/demo-1.C:7:7: note:   candidate expects 1 argument, 0 provided

which shows the code that's leading to the error (the bad call to
std::make_unique).


Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.

OK for trunk?


gcc/cp/ChangeLog:
* error.cc (print_instantiation_partial_context_line): Call
diagnostic_show_locus.

gcc/testsuite/ChangeLog:
* g++.dg/diagnostic/static_assert3.C: Add directives for
    additional source printing.
* g++.dg/template/error60.C: New test.

Signed-off-by: David Malcolm 
---
 gcc/cp/error.cc   |  2 +
 .../g++.dg/diagnostic/static_assert3.C|  7 +++-
 gcc/testsuite/g++.dg/template/error60.C   | 37 +++
 3 files changed, 45 insertions(+), 1 deletion(-)
 create mode 100644 gcc/testsuite/g++.dg/template/error60.C

diff --git a/gcc/cp/error.cc b/gcc/cp/error.cc
index ef96e140f24..767478cf5fd 100644
--- a/gcc/cp/error.cc
+++ b/gcc/cp/error.cc
@@ -3774,6 +3774,8 @@ print_instantiation_partial_context_line 
(diagnostic_context *context,
   ? _("recursively required from here\n")
   : _("required from here\n"));
 }
+  gcc_rich_location rich_loc (loc);
+  diagnostic_show_locus (context, &rich_loc, DK_NOTE);
 }
 
 /* Same as print_instantiation_full_context but less verbose.  */
diff --git a/gcc/testsuite/g++.dg/diagnostic/static_assert3.C 
b/gcc/testsuite/g++.dg/diagnostic/static_assert3.C
index 5d363884508..4ec53f17120 100644
--- a/gcc/testsuite/g++.dg/diagnostic/static_assert3.C
+++ b/gcc/testsuite/g++.dg/diagnostic/static_assert3.C
@@ -5,6 +5,11 @@
 template  struct is_same { static constexpr bool value 
= false; };
 template  struct is_same { static constexpr bool value = 
true; };
 
+/* { dg-begin-multiline-output "" }
+  f(0, 1.3);
+  ~^~~~
+   { dg-end-multiline-output "" } */
+
 template 
 void f(T, U)
 {
@@ -32,5 +37,5 @@ void f(T, U)
 
 void g()
 {
- f(0, 1.3);
+ f(0, 1.3); // { dg-message " required from here" }
 }
diff --git a/gcc/testsuite/g++.dg/template/error60.C 
b/gcc/testsuite/g++.dg/template/error60.C
new file mode 100644
index 000..8c2139b207c
--- /dev/null
+++ b/gcc/tests

Re: [COMMITTED] Remove pass counting in VRP.

[David Malcolm]

On Tue, 2023-10-03 at 10:32 -0400, Andrew MacLeod wrote:
> Pass counting in VRP is used to decide when to call early VRP, pass
> the 
> flag to enable warnings, and when the final pass is.
> 
> If you try to add additional passes, this becomes quite fragile. This
> patch simply chooses the pass based on the data pointer passed in,
> and 
> remove the pass counter.   The first FULL VRP pass invokes the
> warning 
> code, and the flag passed in now represents the FINAL pass of VRP.  
> There is no longer a global flag which, as it turns out, wasn't
> working 
> well with the JIT compiler, but when undetected.  (Thanks to dmalcolm
> for helping me sort out what was going on there)
> 
> 
> Bootstraps  on x86_64-pc-linux-gnu with no regressions.   Pushed.

[CCing jit mailing list]

I'm worried that this patch may have "papered over" an issue with
libgccjit.  Specifically:

[...snip...]

> diff --git a/gcc/tree-vrp.cc b/gcc/tree-vrp.cc
> index d7b194f5904..05266dfe34a 100644
> --- a/gcc/tree-vrp.cc
> +++ b/gcc/tree-vrp.cc
> @@ -1120,36 +1120,44 @@ const pass_data pass_data_early_vrp =
>( TODO_cleanup_cfg | TODO_update_ssa | TODO_verify_all ),
>  };
>  
> -static int vrp_pass_num = 0;
> +static bool run_warning_pass = true;

I see the global variable "run_warning_pass" starts out true here

>  class pass_vrp : public gimple_opt_pass
>  {
>  public:
>pass_vrp (gcc::context *ctxt, const pass_data &data_)
> -: gimple_opt_pass (data_, ctxt), data (data_), warn_array_bounds_p 
> (false),
> -  my_pass (vrp_pass_num++)
> -  {}
> +: gimple_opt_pass (data_, ctxt), data (data_),
> +  warn_array_bounds_p (false), final_p (false)
> +  {
> +// Only the frst VRP pass should run warnings.
> +if (&data == &pass_data_vrp)
> +  {
> + warn_array_bounds_p = run_warning_pass;
> + run_warning_pass = false;

...and run_warning_pass affects the member data
pass_vrp::warn_array_bounds_p here, and then becomes false, but nothing
seems to ever reset run_warning_pass back to true.

It seems that with this patch, if libgccjit compiles more than one
gcc_jit_context in the same process, the first context compilation will
warn, whereas subsequent ones in that process won't.

Or did I miss something?

[...snip...]

Thoughts?
Dave

Re: [COMMITTED] Remove pass counting in VRP.

[David Malcolm]

On Tue, 2023-10-03 at 13:11 -0400, Andrew MacLeod wrote:
> 
> On 10/3/23 13:02, David Malcolm wrote:
> > On Tue, 2023-10-03 at 10:32 -0400, Andrew MacLeod wrote:
> > > Pass counting in VRP is used to decide when to call early VRP,
> > > pass
> > > the
> > > flag to enable warnings, and when the final pass is.
> > > 
> > > If you try to add additional passes, this becomes quite fragile.
> > > This
> > > patch simply chooses the pass based on the data pointer passed
> > > in,
> > > and
> > > remove the pass counter.   The first FULL VRP pass invokes the
> > > warning
> > > code, and the flag passed in now represents the FINAL pass of
> > > VRP.
> > > There is no longer a global flag which, as it turns out, wasn't
> > > working
> > > well with the JIT compiler, but when undetected.  (Thanks to
> > > dmalcolm
> > > for helping me sort out what was going on there)
> > > 
> > > 
> > > Bootstraps  on x86_64-pc-linux-gnu with no regressions.   Pushed.
> > [CCing jit mailing list]
> > 
> > I'm worried that this patch may have "papered over" an issue with
> > libgccjit.  Specifically:
> 
> well, that isnt the patch that was checked in :-P

Aha!  That makes much more sense.  I took a look at
https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=7eb5ce7f58ed4a48641e1786e4fdeb2f7fb8c5ff
and yes, that looks like it will work with libgccjit

Thanks for clarifying (and for fixing the issue)
Dave

> 
> Im not sure how the old version got into the commit note.
> 
> Attached is the version checked in.
> 

[pushed] libcpp: "const" and other cleanups

[David Malcolm]

No functional change intended.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-4475-g25af7c1a806c0c.

gcc/ChangeLog:
* input.cc (make_location): Move implementation to
line_maps::make_location.

libcpp/ChangeLog:
* include/line-map.h (line_maps::pure_location_p): New decl.
(line_maps::get_pure_location): New decl.
(line_maps::get_range_from_loc): New decl.
(line_maps::get_start): New.
(line_maps::get_finish): New.
(line_maps::make_location): New decl.
(get_range_from_loc): Make line_maps param const.
(get_discriminator_from_loc): Likewise.
(pure_location_p): Likewise.
(get_pure_location): Likewise.
(linemap_check_files_exited): Likewise.
(linemap_tracks_macro_expansion_locs_p): Likewise.
(linemap_location_in_system_header_p): Likewise.
(linemap_location_from_macro_definition_p): Likewise.
(linemap_macro_map_loc_unwind_toward_spelling): Likewise.
(linemap_included_from_linemap): Likewise.
(first_map_in_common): Likewise.
(linemap_compare_locations): Likewise.
(linemap_location_before_p): Likewise.
(linemap_resolve_location): Likewise.
(linemap_unwind_toward_expansion): Likewise.
(linemap_unwind_to_first_non_reserved_loc): Likewise.
(linemap_expand_location): Likewise.
(linemap_get_file_highest_location): Likewise.
(linemap_get_statistics): Likewise.
(linemap_dump_location): Likewise.
(linemap_dump): Likewise.
(line_table_dump): Likewise.
* internal.h (linemap_get_expansion_line): Likewise.
(linemap_get_expansion_filename): Likewise.
* line-map.cc (can_be_stored_compactly_p): Likewise.
(get_data_from_adhoc_loc): Drop redundant "class".
(get_discriminator_from_adhoc_loc): Likewise.
(get_location_from_adhoc_loc): Likewise.
(get_range_from_adhoc_loc): Likewise.
(get_range_from_loc): Make const and move implementation to...
(line_maps::get_range_from_loc): ...this new function.
(get_discriminator_from_loc): Make line_maps param const.
(pure_location_p): Make const and move implementation to...
(line_maps::pure_location_p): ...this new function.
(get_pure_location): Make const and move implementation to...
(line_maps::get_pure_location): ...this new function.
(linemap_included_from_linemap): Make line_maps param const.
(linemap_check_files_exited): Likewise.
(linemap_tracks_macro_expansion_locs_p): Likewise.
(linemap_macro_map_loc_unwind_toward_spelling): Likewise.
(linemap_get_expansion_line): Likewise.
(linemap_get_expansion_filename): Likewise.
(linemap_location_in_system_header_p): Likewise.
(first_map_in_common_1): Likewise.
(linemap_compare_locations): Likewise.
(linemap_macro_loc_to_spelling_point): Likewise.
(linemap_macro_loc_to_def_point): Likewise.
(linemap_macro_loc_to_exp_point): Likewise.
(linemap_resolve_location): Likewise.
(linemap_location_from_macro_definition_p): Likewise.
(linemap_unwind_toward_expansion): Likewise.
(linemap_unwind_to_first_non_reserved_loc): Likewise.
(linemap_expand_location): Likewise.
(linemap_dump): Likewise.
(linemap_dump_location): Likewise.
(linemap_get_file_highest_location): Likewise.
(linemap_get_statistics): Likewise.
(line_table_dump): Likewise.
---
 gcc/input.cc  |  15 ++---
 libcpp/include/line-map.h |  69 ---
 libcpp/internal.h |   4 +-
 libcpp/line-map.cc| 135 ++
 4 files changed, 143 insertions(+), 80 deletions(-)

diff --git a/gcc/input.cc b/gcc/input.cc
index 1956f2fcbad..a0e7cb17455 100644
--- a/gcc/input.cc
+++ b/gcc/input.cc
@@ -1204,7 +1204,9 @@ expansion_point_location (location_t location)
 }
 
 /* Construct a location with caret at CARET, ranging from START to
-   finish e.g.
+   FINISH.
+
+   For example, consider:
 
  112
 12345678901234567890
@@ -1220,16 +1222,7 @@ expansion_point_location (location_t location)
 location_t
 make_location (location_t caret, location_t start, location_t finish)
 {
-  location_t pure_loc = get_pure_location (caret);
-  source_range src_range;
-  src_range.m_start = get_start (start);
-  src_range.m_finish = get_finish (finish);
-  location_t combined_loc = COMBINE_LOCATION_DATA (line_table,
-  pure_loc,
-  src_range,
-  NULL,
-  0);
-  return combined_loc;
+  return line_table->make_location (caret, start, finish);
 }
 
 /* Same as above, but taking a source ran

[pushed] diagnostics: fix ICE on sarif output when source file is unreadable [PR111700]

[David Malcolm]

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-4474-g94caa6a6b4bd73.

gcc/ChangeLog:
PR driver/111700
* input.cc (file_cache::add_file): Update leading comment to
clarify that it can fail.
(file_cache::lookup_or_add_file): Likewise.
(file_cache::get_source_file_content): Gracefully handle
lookup_or_add_file failing.

gcc/testsuite/ChangeLog:
PR driver/111700
* c-c++-common/diagnostic-format-sarif-file-pr111700.c: New test.
---
 gcc/input.cc | 12 ++--
 .../diagnostic-format-sarif-file-pr111700.c  | 12 
 2 files changed, 22 insertions(+), 2 deletions(-)
 create mode 100644 
gcc/testsuite/c-c++-common/diagnostic-format-sarif-file-pr111700.c

diff --git a/gcc/input.cc b/gcc/input.cc
index 0b00b8923db..1956f2fcbad 100644
--- a/gcc/input.cc
+++ b/gcc/input.cc
@@ -443,7 +443,10 @@ file_cache::evicted_cache_tab_entry (unsigned 
*highest_use_count)
accessed by caret diagnostic.  This cache is added to an array of
cache and can be retrieved by lookup_file_in_cache_tab.  This
function returns the created cache.  Note that only the last
-   num_file_slots files are cached.  */
+   num_file_slots files are cached.
+
+   This can return nullptr if the FILE_PATH can't be opened for
+   reading, or if the content can't be converted to the input_charset.  */
 
 file_cache_slot*
 file_cache::add_file (const char *file_path)
@@ -547,7 +550,10 @@ file_cache::~file_cache ()
 /* Lookup the cache used for the content of a given file accessed by
caret diagnostic.  If no cached file was found, create a new cache
for this file, add it to the array of cached file and return
-   it.  */
+   it.
+
+   This can return nullptr on a cache miss if FILE_PATH can't be opened for
+   reading, or if the content can't be converted to the input_charset.  */
 
 file_cache_slot*
 file_cache::lookup_or_add_file (const char *file_path)
@@ -1072,6 +1078,8 @@ char_span
 file_cache::get_source_file_content (const char *file_path)
 {
   file_cache_slot *c = lookup_or_add_file (file_path);
+  if (c == nullptr)
+return char_span (nullptr, 0);
   return c->get_full_file_content ();
 }
 
diff --git a/gcc/testsuite/c-c++-common/diagnostic-format-sarif-file-pr111700.c 
b/gcc/testsuite/c-c++-common/diagnostic-format-sarif-file-pr111700.c
new file mode 100644
index 000..06605accf6e
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/diagnostic-format-sarif-file-pr111700.c
@@ -0,0 +1,12 @@
+/* { dg-do compile } */
+/* { dg-options "-fdiagnostics-format=sarif-file" } */
+
+# 0 "this-file-does-not-exist.c"
+#warning message
+
+/* Verify that some JSON was written to a file with the expected name.  */
+/* { dg-final { verify-sarif-file } } */
+
+/* ...and that it at least includes the warning
+   { dg-final { scan-sarif-file "\"message\": " } }
+ { dg-final { scan-sarif-file "\"text\": \"#warning message" } } */
-- 
2.26.3

[pushed] libcpp: eliminate COMBINE_LOCATION_DATA

[David Malcolm]

This patch eliminates the function "COMBINE_LOCATION_DATA" (which hasn't
been a macro since r6-739-g0501dbd932a7e9) and the function
"get_combined_adhoc_loc" in favor of a new
line_maps::get_or_create_combined_loc member function.

No functional change intended.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-4476-g1f68a3e8727f36.

gcc/cp/ChangeLog:
* module.cc (module_state::read_location): Update for renaming of
get_combined_adhoc_loc.

gcc/ChangeLog:
* genmatch.cc (main): Update for "m_" prefix of some fields of
line_maps.
* input.cc (make_location): Update for removal of
COMBINE_LOCATION_DATA.
(dump_line_table_statistics): Update for "m_" prefix of some
fields of line_maps.
(location_with_discriminator): Update for removal of
COMBINE_LOCATION_DATA.
(line_table_test::line_table_test): Update for "m_" prefix of some
fields of line_maps.
* toplev.cc (general_init): Likewise.
* tree.cc (set_block): Update for removal of
COMBINE_LOCATION_DATA.
(set_source_range): Likewise.

libcpp/ChangeLog:
* include/line-map.h (line_maps::reallocator): Rename to...
(line_maps::m_reallocator): ...this.
(line_maps::round_alloc_size): Rename to...
(line_maps::m_round_alloc_size): ...this.
(line_maps::location_adhoc_data_map): Rename to...
(line_maps::m_location_adhoc_data_map): ...this.
(line_maps::num_optimized_ranges): Rename to...
(line_maps::m_num_optimized_ranges): ..this.
(line_maps::num_unoptimized_ranges): Rename to...
(line_maps::m_num_unoptimized_ranges): ...this.
(get_combined_adhoc_loc): Delete decl.
(COMBINE_LOCATION_DATA): Delete.
* lex.cc (get_location_for_byte_range_in_cur_line): Update for
removal of COMBINE_LOCATION_DATA.
(warn_about_normalization): Likewise.
(_cpp_lex_direct): Likewise.
* line-map.cc (line_maps::~line_maps): Update for "m_" prefix of
some fields of line_maps.
(rebuild_location_adhoc_htab): Likewise.
(can_be_stored_compactly_p): Convert to...
(line_maps::can_be_stored_compactly_p): ...this private member
function.
(get_combined_adhoc_loc): Convert to...
(line_maps::get_or_create_combined_loc): ...this public member
function.
(line_maps::make_location): Update for removal of
COMBINE_LOCATION_DATA.
(get_data_from_adhoc_loc): Update for "m_" prefix of some fields
of line_maps.
(get_discriminator_from_adhoc_loc): Likewise.
(get_location_from_adhoc_loc): Likewise.
(get_range_from_adhoc_loc): Convert to...
(line_maps::get_range_from_adhoc_loc): ...this private member
function.
(line_maps::get_range_from_loc): Update for conversion of
get_range_from_adhoc_loc to a member function.
(linemap_init): Update for "m_" prefix of some fields of
line_maps.
(line_map_new_raw): Likewise.
(linemap_enter_macro): Likewise.
(linemap_get_statistics): Likewise.
---
 gcc/cp/module.cc  |   3 +-
 gcc/genmatch.cc   |   4 +-
 gcc/input.cc  |  26 
 gcc/toplev.cc |   4 +-
 gcc/tree.cc   |  12 ++--
 libcpp/include/line-map.h |  40 ++--
 libcpp/lex.cc |  20 +++---
 libcpp/line-map.cc| 128 ++
 8 files changed, 118 insertions(+), 119 deletions(-)

diff --git a/gcc/cp/module.cc b/gcc/cp/module.cc
index 77c9edcbc04..e3fb2299d93 100644
--- a/gcc/cp/module.cc
+++ b/gcc/cp/module.cc
@@ -15969,7 +15969,8 @@ module_state::read_location (bytes_in &sec) const
range.m_finish = read_location (sec);
unsigned discriminator = sec.u ();
if (locus != loc && range.m_start != loc && range.m_finish != loc)
- locus = get_combined_adhoc_loc (line_table, locus, range, NULL, 
discriminator);
+ locus = line_table->get_or_create_combined_loc (locus, range,
+ nullptr, 
discriminator);
   }
   break;
 
diff --git a/gcc/genmatch.cc b/gcc/genmatch.cc
index 03d325efdf6..e9d7afa7728 100644
--- a/gcc/genmatch.cc
+++ b/gcc/genmatch.cc
@@ -5458,8 +5458,8 @@ main (int argc, char **argv)
 
   line_table = XCNEW (class line_maps);
   linemap_init (line_table, 0);
-  line_table->reallocator = xrealloc;
-  line_table->round_alloc_size = round_alloc_size;
+  line_table->m_reallocator = xrealloc;
+  line_table->m_round_alloc_size = round_alloc_size;
 
   r = cpp_create_reader (CLK_GNUC99, NULL, line_table);
   cpp_callbacks *cb = cpp_get_callbacks (r);
diff --git a/gcc/input.cc b/gcc/input.cc
index a0e7cb17455..fd09fccb0e3 100644
--- a/gcc/input.cc
+++ b/gcc/input.cc
@@ -1231,7 +1231,8 @@ location_t
 make_location (location_t caret, source_range src_range)

[pushed] libcpp: eliminate LINEMAPS_LAST_ALLOCATED{, _ORDINARY, _MACRO}_MAP

[David Malcolm]

Nothing uses these; delete them.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-4478-ga73c80d99736f0.

libcpp/ChangeLog:
* include/line-map.h (LINEMAPS_LAST_ALLOCATED_MAP): Delete.
(LINEMAPS_LAST_ALLOCATED_ORDINARY_MAP): Delete.
(LINEMAPS_LAST_ALLOCATED_MACRO_MAP): Delete.
---
 libcpp/include/line-map.h | 25 -
 1 file changed, 25 deletions(-)

diff --git a/libcpp/include/line-map.h b/libcpp/include/line-map.h
index b353df45209..7a172f4c846 100644
--- a/libcpp/include/line-map.h
+++ b/libcpp/include/line-map.h
@@ -936,16 +936,6 @@ LINEMAPS_LAST_MAP (const line_maps *set, bool map_kind)
  LINEMAPS_USED (set, map_kind) - 1);
 }
 
-/* Returns the last map that was allocated in the line table SET.
-   MAP_KIND shall be TRUE if we are interested in macro maps, FALSE
-   otherwise.*/
-inline line_map *
-LINEMAPS_LAST_ALLOCATED_MAP (const line_maps *set, bool map_kind)
-{
-  return LINEMAPS_MAP_AT (set, map_kind,
- LINEMAPS_ALLOCATED (set, map_kind) - 1);
-}
-
 /* Returns a pointer to the memory region where ordinary maps are
allocated in the line table SET.  */
 inline line_map_ordinary *
@@ -994,14 +984,6 @@ LINEMAPS_LAST_ORDINARY_MAP (const line_maps *set)
   return (line_map_ordinary *)LINEMAPS_LAST_MAP (set, false);
 }
 
-/* Returns a pointer to the last ordinary map allocated the line table
-   SET.  */
-inline line_map_ordinary *
-LINEMAPS_LAST_ALLOCATED_ORDINARY_MAP (const line_maps *set)
-{
-  return (line_map_ordinary *)LINEMAPS_LAST_ALLOCATED_MAP (set, false);
-}
-
 /* Returns a pointer to the beginning of the region where macro maps
are allocated.  */
 inline line_map_macro *
@@ -1059,13 +1041,6 @@ LINEMAPS_MACRO_LOWEST_LOCATION (const line_maps *set)
  : MAX_LOCATION_T + 1;
 }
 
-/* Returns the last macro map allocated in the line table SET.  */
-inline line_map_macro *
-LINEMAPS_LAST_ALLOCATED_MACRO_MAP (const line_maps *set)
-{
-  return (line_map_macro *)LINEMAPS_LAST_ALLOCATED_MAP (set, true);
-}
-
 extern void *get_data_from_adhoc_loc (const line_maps *, location_t);
 extern unsigned get_discriminator_from_adhoc_loc (const line_maps *, 
location_t);
 extern location_t get_location_from_adhoc_loc (const line_maps *,
-- 
2.26.3

[pushed] libcpp: eliminate LINEMAPS_{,ORDINARY_,MACRO_}CACHE

[David Malcolm]

It's simpler to use field access than to go through these inline
functions that look as if they are macros.

No functional change intended.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-4479-g45bae1809c3919.

libcpp/ChangeLog:
* include/line-map.h (maps_info_ordinary::cache): Rename to...
(maps_info_ordinary::m_cache): ...this.
(maps_info_macro::cache): Rename to...
(maps_info_macro::m_cache): ...this.
(LINEMAPS_CACHE): Delete.
(LINEMAPS_ORDINARY_CACHE): Delete.
(LINEMAPS_MACRO_CACHE): Delete.
* init.cc (read_original_filename): Update for adding "m_" prefix.
* line-map.cc (linemap_add): Eliminate LINEMAPS_ORDINARY_CACHE in
favor of a simple field access.
(linemap_enter_macro): Likewise for LINEMAPS_MACRO_CACHE.
(linemap_ordinary_map_lookup): Likewise for
LINEMAPS_ORDINARY_CACHE, twice.
(linemap_lookup_macro_index): Likewise for LINEMAPS_MACRO_CACHE.
---
 libcpp/include/line-map.h | 36 ++--
 libcpp/init.cc|  2 +-
 libcpp/line-map.cc| 12 ++--
 3 files changed, 13 insertions(+), 37 deletions(-)

diff --git a/libcpp/include/line-map.h b/libcpp/include/line-map.h
index 7a172f4c846..30f2284b5d1 100644
--- a/libcpp/include/line-map.h
+++ b/libcpp/include/line-map.h
@@ -733,7 +733,9 @@ struct GTY(()) maps_info_ordinary {
  or equal to ALLOCATED.  */
   unsigned int used;
 
-  mutable unsigned int cache;
+  /* The index of the last ordinary map that was looked up with
+ linemap_lookup.  */
+  mutable unsigned int m_cache;
 };
 
 struct GTY(()) maps_info_macro {
@@ -748,7 +750,9 @@ struct GTY(()) maps_info_macro {
  or equal to ALLOCATED.  */
   unsigned int used;
 
-  mutable unsigned int cache;
+  /* The index of the last macro map that was looked up with
+ linemap_lookup.  */
+  mutable unsigned int m_cache;
 };
 
 /* Data structure to associate a source_range together with an arbitrary
@@ -904,18 +908,6 @@ LINEMAPS_USED (line_maps *set, bool map_kind)
 return set->info_ordinary.used;
 }
 
-/* Returns the index of the last map that was looked up with
-   linemap_lookup. MAP_KIND shall be TRUE if we are interested in
-   macro maps, FALSE otherwise.  */
-inline unsigned int &
-LINEMAPS_CACHE (const line_maps *set, bool map_kind)
-{
-  if (map_kind)
-return set->info_macro.cache;
-  else
-return set->info_ordinary.cache;
-}
-
 /* Return the map at a given index.  */
 inline line_map *
 LINEMAPS_MAP_AT (const line_maps *set, bool map_kind, int index)
@@ -968,14 +960,6 @@ LINEMAPS_ORDINARY_USED (const line_maps *set)
   return LINEMAPS_USED (set, false);
 }
 
-/* Return the index of the last ordinary map that was looked up with
-   linemap_lookup.  */
-inline unsigned int &
-LINEMAPS_ORDINARY_CACHE (const line_maps *set)
-{
-  return LINEMAPS_CACHE (set, false);
-}
-
 /* Returns a pointer to the last ordinary map used in the line table
SET.  */
 inline line_map_ordinary *
@@ -1016,14 +1000,6 @@ LINEMAPS_MACRO_USED (const line_maps *set)
   return LINEMAPS_USED (set, true);
 }
 
-/* Return the index of the last macro map that was looked up with
-   linemap_lookup.  */
-inline unsigned int &
-LINEMAPS_MACRO_CACHE (const line_maps *set)
-{
-  return LINEMAPS_CACHE (set, true);
-}
-
 /* Returns the last macro map used in the line table SET.  */
 inline line_map_macro *
 LINEMAPS_LAST_MACRO_MAP (const line_maps *set)
diff --git a/libcpp/init.cc b/libcpp/init.cc
index 9a20f8d8176..b97d7a7b00e 100644
--- a/libcpp/init.cc
+++ b/libcpp/init.cc
@@ -788,7 +788,7 @@ read_original_filename (cpp_reader *pfile)
  penult[1].reason = penult[0].reason;
  penult[0] = penult[1];
  pfile->line_table->info_ordinary.used--;
- pfile->line_table->info_ordinary.cache = 0;
+ pfile->line_table->info_ordinary.m_cache = 0;
}
 
  return true;
diff --git a/libcpp/line-map.cc b/libcpp/line-map.cc
index 5b67a70429f..385d54c53b7 100644
--- a/libcpp/line-map.cc
+++ b/libcpp/line-map.cc
@@ -638,7 +638,7 @@ linemap_add (line_maps *set, enum lc_reason reason,
   map->sysp = sysp;
   map->to_file = to_file;
   map->to_line = to_line;
-  LINEMAPS_ORDINARY_CACHE (set) = LINEMAPS_ORDINARY_USED (set) - 1;
+  set->info_ordinary.m_cache = LINEMAPS_ORDINARY_USED (set) - 1;
   /* Do not store range_bits here.  That's readjusted in
  linemap_line_start.  */
   map->m_range_bits = map->m_column_and_range_bits = 0;
@@ -786,7 +786,7 @@ linemap_enter_macro (class line_maps *set, struct 
cpp_hashnode *macro_node,
   memset (MACRO_MAP_LOCATIONS (map), 0,
  2 * num_tokens * sizeof (location_t));
 
-  LINEMAPS_MACRO_CACHE (set) = LINEMAPS_MACRO_USED (set) - 1;
+  set->info_macro.m_cache = LINEMAPS_MACRO_USED (set) - 1;
 
   return map;
 }
@@ -1116,7 +1116,7 @@ linemap_ordinary_map_lookup (const line_maps *set, 
location_t line)
   if (set ==

[pushed] analyzer: improvements to out-of-bounds diagrams [PR111155]

[David Malcolm]

Update out-of-bounds diagrams to show existing string values,
and the initial write index within a string buffer.

For example, given the out-of-bounds write in strcat in:

void test (void)
{
  char buf[10];
  strcpy (buf, "hello");
  strcat (buf, " world!");
}

the diagram improves from:

   ┌─┬─┬┬┬┐┌─┬─┬─┐
   │ [0] │ [1] │[2] │[3] │[4] ││ [5] │ [6] │ [7] │
   ├─┼─┼┼┼┤├─┼─┼─┤
   │ ' ' │ 'w' │'o' │'r' │'l' ││ 'd' │ '!' │ NUL │
   ├─┴─┴┴┴┴┴─┴─┴─┤
   │  string literal (type: 'char[8]')   │
   └─┘
  │ ││││  │ │ │
  │ ││││  │ │ │
  v vvvv  v v v
  ┌─┬┬┐┌─┐
  │ [0] │  ...   │[9] ││ │
  ├─┴┴┤│after valid range│
  │ 'buf' (type: 'char[10]')  ││ │
  └───┘└─┘
  ├─┬─┤├┬┤
│   │
  ╭─┴╮╭─┴─╮
  │capacity: 10 bytes││overflow of 3 bytes│
  ╰──╯╰───╯

to:

 ┌┬┬┬┬┐┌─┬─┬─┐
 │[0] │[1] │[2] │[3] │[4] ││ [5] │ [6] │ [7] │
 ├┼┼┼┼┤├─┼─┼─┤
 │' ' │'w' │'o' │'r' │'l' ││ 'd' │ '!' │ NUL │
 ├┴┴┴┴┴┴─┴─┴─┤
 │ string literal (type: 'char[8]')  │
 └───┘
   │││││  │ │ │
   │││││  │ │ │
   vvvvv  v v v
  ┌─┬┬┬──┬┐┌─┐
  │ [0] │... │[5] │ ...  │[9] ││ │
  ├─┼┬┬┬┬┼┼──┴┘│ │
  │ 'h' │'e' │'l' │'l' │'o' ││NUL ││after valid range│
  ├─┴┴┴┴┴┴┴───┐│ │
  │ 'buf' (type: 'char[10]')  ││ │
  └───┘└─┘
  ├─┬─┤├┬┤
│   │
  ╭─┴╮╭─┴─╮
  │capacity: 10 bytes││overflow of 3 bytes│
  ╰──╯╰───╯

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-4477-gb365e9d57ad445.

gcc/analyzer/ChangeLog:
PR analyzer/55
* access-diagram.cc (boundaries::boundaries): Add logger param
(boundaries::add): Add logging.
(boundaries::get_hard_boundaries_in_range): New.
(boundaries::m_logger): New field.
(boundaries::get_table_x_for_offset): Make public.
(class svalue_spatial_item): New.
(class compound_svalue_spatial_item): New.
(add_ellipsis_to_gaps): New.
(valid_region_spatial_item::valid_region_spatial_item): Add theme
param.  Initialize m_boundaries, m_existing_sval, and
m_existing_sval_spatial_item.
(valid_region_spatial_item::add_boundaries): Set m_boundaries.
Add boundaries for any m_existing_sval_spatial_item.
(valid_region_spatial_item::add_array_elements_to_table): Rewrite
creation of min/max index in terms of
maybe_add_array_index_to_table.  Rewrite ellipsis code using
add_ellipsis_to_gaps. Add index values for any hard boundaries
within the valid region.
(valid_region_spatial_item::maybe_add_array_index_to_table): New,
based on code formerly in add_array_elements_to_table.
(valid_region_spatial_item::make_table): Make use of
m_existing_sval_spatial_item, if any.
(valid_region_spatial_item::m_boundaries): New field.
(valid_region_spatial_item::m_existing_sval): New field.
(valid_region_spatial_item::m_existing_sval

[pushed] libcpp: eliminate LINEMAPS_{ORDINARY,MACRO}_MAPS

[David Malcolm]

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-4480-g0a0ceb7a72fe0b.

libcpp/ChangeLog:
* include/line-map.h (LINEMAPS_ORDINARY_MAPS): Delete.
(LINEMAPS_MACRO_MAPS): Delete.
* line-map.cc (linemap_tracks_macro_expansion_locs_p): Update for
deletion of LINEMAPS_MACRO_MAPS.
(linemap_get_statistics): Likewise.
---
 libcpp/include/line-map.h | 16 
 libcpp/line-map.cc|  4 ++--
 2 files changed, 2 insertions(+), 18 deletions(-)

diff --git a/libcpp/include/line-map.h b/libcpp/include/line-map.h
index 30f2284b5d1..cb934e6b9b3 100644
--- a/libcpp/include/line-map.h
+++ b/libcpp/include/line-map.h
@@ -928,14 +928,6 @@ LINEMAPS_LAST_MAP (const line_maps *set, bool map_kind)
  LINEMAPS_USED (set, map_kind) - 1);
 }
 
-/* Returns a pointer to the memory region where ordinary maps are
-   allocated in the line table SET.  */
-inline line_map_ordinary *
-LINEMAPS_ORDINARY_MAPS (const line_maps *set)
-{
-  return set->info_ordinary.maps;
-}
-
 /* Returns the INDEXth ordinary map.  */
 inline line_map_ordinary *
 LINEMAPS_ORDINARY_MAP_AT (const line_maps *set, int index)
@@ -968,14 +960,6 @@ LINEMAPS_LAST_ORDINARY_MAP (const line_maps *set)
   return (line_map_ordinary *)LINEMAPS_LAST_MAP (set, false);
 }
 
-/* Returns a pointer to the beginning of the region where macro maps
-   are allocated.  */
-inline line_map_macro *
-LINEMAPS_MACRO_MAPS (const line_maps *set)
-{
-  return set->info_macro.maps;
-}
-
 /* Returns the INDEXth macro map.  */
 inline line_map_macro *
 LINEMAPS_MACRO_MAP_AT (const line_maps *set, int index)
diff --git a/libcpp/line-map.cc b/libcpp/line-map.cc
index 385d54c53b7..cd173530149 100644
--- a/libcpp/line-map.cc
+++ b/libcpp/line-map.cc
@@ -738,7 +738,7 @@ linemap_module_restore (line_maps *set, unsigned lwm)
 bool
 linemap_tracks_macro_expansion_locs_p (const line_maps *set)
 {
-  return LINEMAPS_MACRO_MAPS (set) != NULL;
+  return set->info_macro.maps != nullptr;
 }
 
 /* Create a macro map.  A macro map encodes source locations of tokens
@@ -2076,7 +2076,7 @@ linemap_get_statistics (const line_maps *set,
   macro_maps_allocated_size =
 LINEMAPS_MACRO_ALLOCATED (set) * sizeof (struct line_map_macro);
 
-  for (cur_map = LINEMAPS_MACRO_MAPS (set);
+  for (cur_map = set->info_macro.maps;
cur_map && cur_map <= LINEMAPS_LAST_MACRO_MAP (set);
++cur_map)
 {
-- 
2.26.3

Re: [pushed] analyzer: improvements to out-of-bounds diagrams [PR111155]

[David Malcolm]

On Mon, 2023-10-09 at 12:09 +0200, Tobias Burnus wrote:
> Hi David,
> 
> your commit breaks compilation with GCC < 6, here with GCC 5.2:
> 
> gcc/analyzer/access-diagram.cc: In member function 'void
> ana::boundaries::add(const ana::access_range&,
> ana::boundaries::kind)':
> gcc/analyzer/access-diagram.cc:655:20: error: 'kind' is not a class,
> namespace, or enumeration
>     (kind == kind::HARD) ? "HARD" : "soft");
>      ^
> The problem is ...
> 
> On 09.10.23 00:58, David Malcolm wrote:
> 
> > Update out-of-bounds diagrams to show existing string values,
> > diff --git a/gcc/analyzer/access-diagram.cc b/gcc/analyzer/access-
> > diagram.cc
> > index a51d594b5b2..2197ec63f53 100644
> > --- a/gcc/analyzer/access-diagram.cc
> > +++ b/gcc/analyzer/access-diagram.cc
> > @@ -630,8 +630,8 @@ class boundaries
> >   public:
> >     enum class kind { HARD, SOFT};
> 
> ...
> 
> > @@ -646,6 +646,15 @@ public:
> 
> Just above the following diff is the line:
> 
>    void add (const access_range &range, enum kind kind)
> 
> >     {
> >   add (range.m_start, kind);
> >   add (range.m_next, kind);
> > +    if (m_logger)
> > +  {
> > + m_logger->start_log_line ();
> > + m_logger->log_partial ("added access_range: ");
> > + range.dump_to_pp (m_logger->get_printer (), true);
> > + m_logger->log_partial (" (%s)",
> > +    (kind == kind::HARD) ? "HARD" :
> > "soft");
> > + m_logger->end_log_line ();
> 
> Actual problem:
> 
> Playing around also with the compiler explorer shows that GCC 5.2 or
> likewise 5.5
> do not like the variable (PARAM_DECL) name "kind" combined with 
> "kind::HARD".
> 
> The following works:
> (A) Using "kind == boundaries::kind::HARD" - i.e. adding
> "boundaries::"
> (B) Renaming the parameter name "kind" to something else - like "k"
> as used
>  in the other functions.
> 
> Can you fix it?

Sorry about the breakage, and thanks for the investigation.

Does the following patch fix the build for you?
Thanks


gcc/analyzer/ChangeLog:
* access-diagram.cc (boundaries::add): Explicitly state
"boundaries::" scope for "kind" enum.
---
 gcc/analyzer/access-diagram.cc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/gcc/analyzer/access-diagram.cc b/gcc/analyzer/access-diagram.cc
index 2197ec63f53..c7d190e3188 100644
--- a/gcc/analyzer/access-diagram.cc
+++ b/gcc/analyzer/access-diagram.cc
@@ -652,7 +652,8 @@ public:
m_logger->log_partial ("added access_range: ");
range.dump_to_pp (m_logger->get_printer (), true);
m_logger->log_partial (" (%s)",
-  (kind == kind::HARD) ? "HARD" : "soft");
+  (kind == boundaries::kind::HARD)
+  ? "HARD" : "soft");
m_logger->end_log_line ();
   }
   }
-- 
2.26.3

Re: [pushed] analyzer: improvements to out-of-bounds diagrams [PR111155]

[David Malcolm]

On Mon, 2023-10-09 at 17:01 +0200, Tobias Burnus wrote:
> Hi David,
> 
> On 09.10.23 16:08, David Malcolm wrote:
> > On Mon, 2023-10-09 at 12:09 +0200, Tobias Burnus wrote:
> > > The following works:
> > > (A) Using "kind == boundaries::kind::HARD" - i.e. adding
> > > "boundaries::"
> > > (B) Renaming the parameter name "kind" to something else - like
> > > "k"
> > > as used
> > >   in the other functions.
> > > 
> > > Can you fix it?
> > Sorry about the breakage, and thanks for the investigation.
> Well, without an older compiler, one does not see it. It also worked
> flawlessly on my laptop today.
> > Does the following patch fix the build for you?
> 
> Yes – as mentioned either of the variants above should work and (A)
> is
> what you have in your patch.
> 
> And it is what I actually tried for the full build. Hence, yes, it
> works :-)

Thanks!

I've pushed this to trunk as r14-4521-g08d0f840dc7ad2.

[pushed] diagnostics: fix missing initialization of context->extra_output_kind

[David Malcolm]

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-4668-gf8644b678285cf.

gcc/ChangeLog:
* diagnostic.cc (diagnostic_initialize): Ensure
context->extra_output_kind is initialized.
---
 gcc/diagnostic.cc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gcc/diagnostic.cc b/gcc/diagnostic.cc
index 9c15f0d8c50..03637459c56 100644
--- a/gcc/diagnostic.cc
+++ b/gcc/diagnostic.cc
@@ -204,7 +204,7 @@ diagnostic_initialize (diagnostic_context *context, int 
n_opts)
   context->m_source_printing.min_margin_width = 0;
   context->m_source_printing.show_ruler_p = false;
   context->report_bug = false;
-
+  context->extra_output_kind = EXTRA_DIAGNOSTIC_OUTPUT_none;
   if (const char *var = getenv ("GCC_EXTRA_DIAGNOSTIC_OUTPUT"))
 {
   if (!strcmp (var, "fixits-v1"))
-- 
2.26.3

[pushed] diagnostics: special-case -fdiagnostics-text-art-charset=ascii for LANG=C

[David Malcolm]

In the LWN discussion of the "ASCII" art in GCC 14
  https://lwn.net/Articles/946733/#Comments
there was some concern about the use of non-ASCII characters in the
output.

Currently -fdiagnostics-text-art-charset defaults to "emoji".
To better handle older terminals by default, this patch special-cases
LANG=C to use -fdiagnostics-text-art-charset=ascii.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-4669-g04013e4464020b.

gcc/ChangeLog:
* diagnostic.cc (diagnostic_initialize): When LANG=C, update
default for -fdiagnostics-text-art-charset from emoji to ascii.
* doc/invoke.texi (fdiagnostics-text-art-charset): Document the above.
---
 gcc/diagnostic.cc   | 13 +++--
 gcc/doc/invoke.texi |  3 ++-
 2 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/gcc/diagnostic.cc b/gcc/diagnostic.cc
index 03637459c56..6e46371b3b4 100644
--- a/gcc/diagnostic.cc
+++ b/gcc/diagnostic.cc
@@ -226,8 +226,17 @@ diagnostic_initialize (diagnostic_context *context, int 
n_opts)
   context->includes_seen = NULL;
   context->m_client_data_hooks = NULL;
   context->m_diagrams.m_theme = NULL;
-  diagnostics_text_art_charset_init (context,
-DIAGNOSTICS_TEXT_ART_CHARSET_DEFAULT);
+
+  enum diagnostic_text_art_charset text_art_charset
+= DIAGNOSTICS_TEXT_ART_CHARSET_DEFAULT;
+  if (const char *lang = getenv ("LANG"))
+{
+  /* For LANG=C, don't assume the terminal supports anything
+other than ASCII.  */
+  if (!strcmp (lang, "C"))
+   text_art_charset = DIAGNOSTICS_TEXT_ART_CHARSET_ASCII;
+}
+  diagnostics_text_art_charset_init (context, text_art_charset);
 }
 
 /* Maybe initialize the color support. We require clients to do this
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index 7c5f81d9783..ef9d1fb8fe6 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -5681,7 +5681,8 @@ value further adds the possibility of emoji in the output 
(such as emitting
 U+26A0 WARNING SIGN followed by U+FE0F VARIATION SELECTOR-16 to select the
 emoji variant of the character).
 
-The default is @samp{emoji}.
+The default is @samp{emoji}, except when the environment variable @env{LANG}
+is set to @samp{C}, in which case the default is @samp{ascii}.
 
 @opindex fdiagnostics-format
 @item -fdiagnostics-format=@var{FORMAT}
-- 
2.26.3

[RFC] Add function attribute: null_terminated_string_arg(PARAM_IDX)

[David Malcolm]

This patch adds a new function attribute to GCC for marking that an
argument is expected to be a null-terminated string.

For example, consider:

  void test_a (const char *p)
__attribute__((null_terminated_string_arg (1)));

which would indicate to humans and compilers that argument 1 of "test_a"
is expected to be a null-terminated string, with the idea:

- we should complain if it's not valid to read from *p up to the first
  '\0' character in the buffer

- we should complain if *p is not terminated, or if it's uninitialized
  before the first '\0' character

This is independent of the nonnull-ness of the pointer: if you also want
to express that the argument must be non-null, we already have
__attribute__((nonnull (N))), so the user can write e.g.:

  void test_b (const char *p)
__attribute__((null_terminated_string_arg (1))
__attribute__((nonnull (1)));

which can also be spelled as:

  void test_b (const char *p)
 __attribute__((null_terminated_string_arg (1),
nonnull (1)));

For a function similar to strncpy, we can use the "access" attribute to
express a maximum size of the read:

  void test_c (const char *p, size_t sz)
 __attribute__((null_terminated_string_arg (1),
nonnull (1),
access (read_only, 1, 2)));

The patch implements:
(a) C/C++ frontends: recognition of this attribute
(b) analyzer: usage of this attribute

The name is rather long; a shorter name might be "c_string_arg".

Does anything like this already exist in GCC, or in any other
compilers or analysis tools?

Thoughts?

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.

gcc/analyzer/ChangeLog:
* region-model.cc
(region_model::check_external_function_for_access_attr): Split
out, replacing with...
(region_model::check_function_attr_access): ...this new function
and...
(region_model::check_function_attrs): ...this new function.
(region_model::check_one_function_attr_null_terminated_string_arg):
New.
(region_model::check_function_attr_null_terminated_string_arg):
New.
(region_model::handle_unrecognized_call): Update for renaming of
check_external_function_for_access_attr to check_function_attrs.
(region_model::check_for_null_terminated_string_arg): Add return
value to one overload.  Make both overloads const.
* region-model.h: Include "stringpool.h" and "attribs.h".
(region_model::check_for_null_terminated_string_arg): Add return
value to one overload.  Make both overloads const.
(region_model::check_external_function_for_access_attr): Delete
decl.
(region_model::check_function_attr_access): New decl.
(region_model::check_function_attr_null_terminated_string_arg):
New decl.
(region_model::check_one_function_attr_null_terminated_string_arg):
New decl.
(region_model::check_function_attrs): New decl.

gcc/c-family/ChangeLog:
* c-attribs.cc (c_common_attribute_table): Add
"null_terminated_string_arg".
(handle_null_terminated_string_arg_attribute): New.

gcc/ChangeLog:
* doc/extend.texi (Common Function Attributes): Add
null_terminated_string_arg.

gcc/testsuite/ChangeLog:
* 
c-c++-common/analyzer/attr-null_terminated_string_arg-access-read_write.c:
New test.
* 
c-c++-common/analyzer/attr-null_terminated_string_arg-access-without-size.c:
New test.
* c-c++-common/analyzer/attr-null_terminated_string_arg-multiple.c:
New test.
* c-c++-common/analyzer/attr-null_terminated_string_arg-nonnull-2.c:
New test.
* c-c++-common/analyzer/attr-null_terminated_string_arg-nonnull-sized.c:
New test.
* c-c++-common/analyzer/attr-null_terminated_string_arg-nonnull.c:
New test.
* 
c-c++-common/analyzer/attr-null_terminated_string_arg-nullable-sized.c:
New test.
* c-c++-common/analyzer/attr-null_terminated_string_arg-nullable.c:
New test.
    * c-c++-common/attr-null_terminated_string_arg.c: New test.

Signed-off-by: David Malcolm 
---
 gcc/analyzer/region-model.cc  | 180 +++---
 gcc/analyzer/region-model.h   |  27 ++-
 gcc/c-family/c-attribs.cc |  17 ++
 gcc/doc/extend.texi   |  57 ++
 ..._terminated_string_arg-access-read_write.c |  15 ++
 ...erminated_string_arg-access-without-size.c |  54 ++
 ...attr-null_terminated_string_arg-multiple.c |  52 +
 ...ttr-null_terminated_string_arg-nonnull-2.c |  33 
 ...null_terminated_string_arg-nonnull-sized.c |  69 +++
 .../attr-null_terminated_string_arg-nonnull.c |  34 
 ...ull_terminated_string_arg-nullable-sized.c |  69 +++
 ...attr-null_terminated_stri

[committed] Reinstate dump_generic_expr_loc

[David Malcolm]

In r262149 ("Introduce dump_location_t", aka
c309657f69df19eaa590b6650acf4d3bea9ac9e6), I removed dump_generic_expr_loc
from dumpfile.h/c as it was unused in the source tree.

It looks like this was overzealous of me, as Richard wants to use it in
  "Re: [14/n] PR85694: Rework overwidening detection"
https://gcc.gnu.org/ml/gcc-patches/2018-06/msg01872.html
(which also reinstates the function, porting it to dump_location_t).

Sorry about the breakage.

This is a minimal patch to reinstate it, as I have followup patches to
dumpfile.c which need to touch it (and the other dump_* functions),
towards being able to save optimization records to a file.

I believe it's a subset of Richard's patch above.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.

Committed to trunk as r262295 under the "obvious" rule.

gcc/ChangeLog:
* dumpfile.c (dump_generic_expr_loc): Undo removal of this
function in r262149, changing "loc" param from source_location to
const dump_location_t &.
* dumpfile.h (dump_generic_expr_loc): Undo removal of this
declaration, as above.
---
 gcc/dumpfile.c | 22 ++
 gcc/dumpfile.h |  2 ++
 2 files changed, 24 insertions(+)

diff --git a/gcc/dumpfile.c b/gcc/dumpfile.c
index 93bc651..5f69f9b 100644
--- a/gcc/dumpfile.c
+++ b/gcc/dumpfile.c
@@ -506,6 +506,28 @@ dump_generic_expr (dump_flags_t dump_kind, dump_flags_t 
extra_dump_flags,
   print_generic_expr (alt_dump_file, t, dump_flags | extra_dump_flags);
 }
 
+
+/* Similar to dump_generic_expr, except additionally print the source
+   location.  */
+
+void
+dump_generic_expr_loc (dump_flags_t dump_kind, const dump_location_t &loc,
+  dump_flags_t extra_dump_flags, tree t)
+{
+  location_t srcloc = loc.get_location_t ();
+  if (dump_file && (dump_kind & pflags))
+{
+  dump_loc (dump_kind, dump_file, srcloc);
+  print_generic_expr (dump_file, t, dump_flags | extra_dump_flags);
+}
+
+  if (alt_dump_file && (dump_kind & alt_flags))
+{
+  dump_loc (dump_kind, alt_dump_file, srcloc);
+  print_generic_expr (alt_dump_file, t, dump_flags | extra_dump_flags);
+}
+}
+
 /* Output a formatted message using FORMAT on appropriate dump streams.  */
 
 void
diff --git a/gcc/dumpfile.h b/gcc/dumpfile.h
index 9828a3f..0e588a6 100644
--- a/gcc/dumpfile.h
+++ b/gcc/dumpfile.h
@@ -425,6 +425,8 @@ extern void dump_printf_loc (dump_flags_t, const 
dump_location_t &,
 const char *, ...) ATTRIBUTE_PRINTF_3;
 extern void dump_function (int phase, tree fn);
 extern void dump_basic_block (dump_flags_t, basic_block, int);
+extern void dump_generic_expr_loc (dump_flags_t, const dump_location_t &,
+  dump_flags_t, tree);
 extern void dump_generic_expr (dump_flags_t, dump_flags_t, tree);
 extern void dump_gimple_stmt_loc (dump_flags_t, const dump_location_t &,
  dump_flags_t, gimple *, int);
-- 
1.8.5.3

Re: [PATCH] -fopt-info: add indentation via DUMP_VECT_SCOPE

[David Malcolm]

On Mon, 2018-07-02 at 14:23 +0200, Christophe Lyon wrote:
> On Fri, 29 Jun 2018 at 10:09, Richard Biener  com> wrote:
> > 
> > On Tue, Jun 26, 2018 at 5:43 PM David Malcolm 
> > wrote:
> > > 
> > > This patch adds a concept of nested "scopes" to dumpfile.c's
> > > dump_*_loc
> > > calls, and wires it up to the DUMP_VECT_SCOPE macro in tree-
> > > vectorizer.h,
> > > so that the nested structure is shown in -fopt-info by
> > > indentation.
> > > 
> > > For example, this converts -fopt-info-all e.g. from:
> > > 
> > > test.c:8:3: note: === analyzing loop ===
> > > test.c:8:3: note: === analyze_loop_nest ===
> > > test.c:8:3: note: === vect_analyze_loop_form ===
> > > test.c:8:3: note: === get_loop_niters ===
> > > test.c:8:3: note: symbolic number of iterations is (unsigned int)
> > > n_9(D)
> > > test.c:8:3: note: not vectorized: loop contains function calls or
> > > data references that cannot be analyzed
> > > test.c:8:3: note: vectorized 0 loops in function
> > > 
> > > to:
> > > 
> > > test.c:8:3: note: === analyzing loop ===
> > > test.c:8:3: note:  === analyze_loop_nest ===
> > > test.c:8:3: note:   === vect_analyze_loop_form ===
> > > test.c:8:3: note:=== get_loop_niters ===
> > > test.c:8:3: note:   symbolic number of iterations is (unsigned
> > > int) n_9(D)
> > > test.c:8:3: note:   not vectorized: loop contains function calls
> > > or data references that cannot be analyzed
> > > test.c:8:3: note: vectorized 0 loops in function
> > > 
> > > showing that the "symbolic number of iterations" message is
> > > within
> > > the "=== analyze_loop_nest ===" (and not within the
> > > "=== vect_analyze_loop_form ===").
> > > 
> > > This is also enabling work for followups involving optimization
> > > records
> > > (allowing the records to directly capture the nested structure of
> > > the
> > > dump messages).
> > > 
> > > Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
> > > 
> > > OK for trunk?
> 
> Hi,
> 
> I've noticed that this patch (r262246) caused regressions on aarch64:
> gcc.dg/vect/slp-perm-1.c -flto -ffat-lto-objects  scan-tree-dump
> vect "note: Built SLP cancelled: can use load/store-lanes"
> gcc.dg/vect/slp-perm-1.c scan-tree-dump vect "note: Built SLP
> cancelled: can use load/store-lanes"
> gcc.dg/vect/slp-perm-2.c -flto -ffat-lto-objects  scan-tree-dump
> vect "note: Built SLP cancelled: can use load/store-lanes"
> gcc.dg/vect/slp-perm-2.c scan-tree-dump vect "note: Built SLP
> cancelled: can use load/store-lanes"
> gcc.dg/vect/slp-perm-3.c -flto -ffat-lto-objects  scan-tree-dump
> vect "note: Built SLP cancelled: can use load/store-lanes"
> gcc.dg/vect/slp-perm-3.c scan-tree-dump vect "note: Built SLP
> cancelled: can use load/store-lanes"
> gcc.dg/vect/slp-perm-5.c -flto -ffat-lto-objects  scan-tree-dump
> vect "note: Built SLP cancelled: can use load/store-lanes"
> gcc.dg/vect/slp-perm-5.c scan-tree-dump vect "note: Built SLP
> cancelled: can use load/store-lanes"
> gcc.dg/vect/slp-perm-6.c -flto -ffat-lto-objects  scan-tree-dump
> vect "note: Built SLP cancelled: can use load/store-lanes"
> gcc.dg/vect/slp-perm-6.c scan-tree-dump vect "note: Built SLP
> cancelled: can use load/store-lanes"
> gcc.dg/vect/slp-perm-7.c -flto -ffat-lto-objects  scan-tree-dump
> vect "note: Built SLP cancelled: can use load/store-lanes"
> gcc.dg/vect/slp-perm-7.c scan-tree-dump vect "note: Built SLP
> cancelled: can use load/store-lanes"
> gcc.dg/vect/slp-perm-8.c -flto -ffat-lto-objects  scan-tree-dump
> vect "note: Built SLP cancelled: can use load/store-lanes"
> gcc.dg/vect/slp-perm-8.c scan-tree-dump vect "note: Built SLP
> cancelled: can use load/store-lanes"
> 
> The problem is that now there are more spaces between "note:" and
> "Built", the attached small patch does that for slp-perm-1.c.

Sorry about the breakage.

> Is it the right way of fixing it or do we want to accept any amount
> of
> spaces for instance?

I don't think we want to hardcode the amount of space in the dumpfile. 
The idea of my patch was to make the dump more human-readable (I hope)
by visualizing the nesting structure of the dump messages, but I think
we shouldn't "bake" that int

Re: [C++ Patch] Adjust one more error message to use rich_location::add_range

[David Malcolm]

On Mon, 2018-07-02 at 12:58 +0200, Paolo Carlini wrote:
> Hi,
> 
> I was double checking my pending patch and going through the errors
> we 
> emit in decl.c and elsewhere about thread_local and __thread and
> noticed 
> another place, in parser.c, where using rich_location::add_range
> seems 
> natural. Note, we could in principle swap location and 
> decl_specs->locations[ds_thread] in the error basing on the gnu bool
> and 
> ensure that the caret always points to __thread. All in all, I don't 
> think it's worth it...
> 
> Thanks, Paolo.

The patch looks good to me (with my "diagnostic messages" maintainer
hat on)

Thanks
Dave

[committed] selftest: introduce class auto_fix_quotes

[David Malcolm]

This patch moves a workaround for locale differences from a
selftest in pretty-print.c to selftest.h/c to make it reusable; I need
this for a selftest in a followup patch.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.

Committed to trunk as r262317 (with my "diagnostic messages" maintainer
hat on)

gcc/ChangeLog:
* pretty-print.c (selftest::test_pp_format): Move save and restore
of quotes to class auto_fix_quotes, and add an instance.
* selftest.c: Include "intl.h".
(selftest::auto_fix_quotes::auto_fix_quotes): New ctor.
(selftest::auto_fix_quotes::~auto_fix_quotes): New dtor.
* selftest.h (selftest::auto_fix_quotes): New class.
---
 gcc/pretty-print.c |  9 +
 gcc/selftest.c | 20 
 gcc/selftest.h | 20 
 3 files changed, 41 insertions(+), 8 deletions(-)

diff --git a/gcc/pretty-print.c b/gcc/pretty-print.c
index 8babbffd..df3ee81 100644
--- a/gcc/pretty-print.c
+++ b/gcc/pretty-print.c
@@ -2102,10 +2102,7 @@ test_pp_format ()
 {
   /* Avoid introducing locale-specific differences in the results
  by hardcoding open_quote and close_quote.  */
-  const char *old_open_quote = open_quote;
-  const char *old_close_quote = close_quote;
-  open_quote = "`";
-  close_quote = "'";
+  auto_fix_quotes fix_quotes;
 
   /* Verify that plain text is passed through unchanged.  */
   assert_pp_format (SELFTEST_LOCATION, "unformatted", "unformatted");
@@ -2187,10 +2184,6 @@ test_pp_format ()
   assert_pp_format (SELFTEST_LOCATION, "item 3 of 7", "item %i of %i", 3, 7);
   assert_pp_format (SELFTEST_LOCATION, "problem with `bar' at line 10",
"problem with %qs at line %i", "bar", 10);
-
-  /* Restore old values of open_quote and close_quote.  */
-  open_quote = old_open_quote;
-  close_quote = old_close_quote;
 }
 
 /* Run all of the selftests within this file.  */
diff --git a/gcc/selftest.c b/gcc/selftest.c
index 27de9a4..dc90557 100644
--- a/gcc/selftest.c
+++ b/gcc/selftest.c
@@ -21,6 +21,7 @@ along with GCC; see the file COPYING3.  If not see
 #include "system.h"
 #include "coretypes.h"
 #include "selftest.h"
+#include "intl.h"
 
 #if CHECKING_P
 
@@ -192,6 +193,25 @@ temp_source_file::temp_source_file (const location &loc,
   fclose (out);
 }
 
+/* Avoid introducing locale-specific differences in the results
+   by hardcoding open_quote and close_quote.  */
+
+auto_fix_quotes::auto_fix_quotes ()
+{
+  m_saved_open_quote = open_quote;
+  m_saved_close_quote = close_quote;
+  open_quote = "`";
+  close_quote = "'";
+}
+
+/* Restore old values of open_quote and close_quote.  */
+
+auto_fix_quotes::~auto_fix_quotes ()
+{
+  open_quote = m_saved_open_quote;
+  close_quote = m_saved_close_quote;
+}
+
 /* Read the contents of PATH into memory, returning a 0-terminated buffer
that must be freed by the caller.
Fail (and abort) if there are any problems, with LOC as the reported
diff --git a/gcc/selftest.h b/gcc/selftest.h
index d66fb93..54fc488 100644
--- a/gcc/selftest.h
+++ b/gcc/selftest.h
@@ -113,6 +113,26 @@ class temp_source_file : public named_temp_file
const char *content);
 };
 
+/* RAII-style class for avoiding introducing locale-specific differences
+   in strings containing localized quote marks, by temporarily overriding
+   the "open_quote" and "close_quote" globals to something hardcoded.
+
+   Specifically, the C locale's values are used:
+   - open_quote becomes "`"
+   - close_quote becomes "'"
+   for the lifetime of the object.  */
+
+class auto_fix_quotes
+{
+ public:
+  auto_fix_quotes ();
+  ~auto_fix_quotes ();
+
+ private:
+  const char *m_saved_open_quote;
+  const char *m_saved_close_quote;
+};
+
 /* Various selftests involving location-handling require constructing a
line table and one or more line maps within it.
 
-- 
1.8.5.3

[PATCH 1/2] Add "optinfo" framework

[David Malcolm]

tinfo.h| 175 +
 gcc/selftest-run-tests.c |   1 +
 gcc/selftest.h   |   1 +
 11 files changed, 1233 insertions(+), 61 deletions(-)
 create mode 100644 gcc/dump-context.h
 create mode 100644 gcc/optinfo-internal.h
 create mode 100644 gcc/optinfo.cc
 create mode 100644 gcc/optinfo.h

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 66c8b6e..7d36a77 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -1426,6 +1426,7 @@ OBJS = \
optabs-libfuncs.o \
optabs-query.o \
optabs-tree.o \
+   optinfo.o \
options-save.o \
opts-global.o \
passes.o \
diff --git a/gcc/coretypes.h b/gcc/coretypes.h
index 283b4eb..ed0e825 100644
--- a/gcc/coretypes.h
+++ b/gcc/coretypes.h
@@ -134,6 +134,13 @@ struct gomp_single;
 struct gomp_target;
 struct gomp_teams;
 
+/* Subclasses of symtab_node, using indentation to show the class
+   hierarchy.  */
+
+class symtab_node;
+  struct cgraph_node;
+  class varpool_node;
+
 union section;
 typedef union section section;
 struct gcc_options;
diff --git a/gcc/dump-context.h b/gcc/dump-context.h
new file mode 100644
index 000..753f714
--- /dev/null
+++ b/gcc/dump-context.h
@@ -0,0 +1,128 @@
+/* Support code for handling the various dump_* calls in dumpfile.h
+   Copyright (C) 2018 Free Software Foundation, Inc.
+   Contributed by David Malcolm .
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 3, or (at your option)
+any later version.
+
+GCC is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3.  If not see
+<http://www.gnu.org/licenses/>.  */
+
+
+#ifndef GCC_DUMP_CONTEXT_H
+#define GCC_DUMP_CONTEXT_H 1
+
+/* A class for handling the various dump_* calls.
+
+   In particular, this class has responsibility for consolidating
+   the "dump_*" calls into optinfo instances (delimited by "dump_*_loc"
+   calls), and emitting them.
+
+   Putting this in a class (rather than as global state) allows
+   for selftesting of this code.  */
+
+class dump_context
+{
+  friend class temp_dump_context;
+ public:
+  static dump_context &get () { return *s_current; }
+
+  ~dump_context ();
+
+  void dump_gimple_stmt (dump_flags_t dump_kind, dump_flags_t extra_dump_flags,
+gimple *gs, int spc);
+
+  void dump_gimple_stmt_loc (dump_flags_t dump_kind,
+const dump_location_t &loc,
+dump_flags_t extra_dump_flags,
+gimple *gs, int spc);
+
+  void dump_generic_expr (dump_flags_t dump_kind,
+ dump_flags_t extra_dump_flags,
+ tree t);
+
+  void dump_generic_expr_loc (dump_flags_t dump_kind,
+ const dump_location_t &loc,
+ dump_flags_t extra_dump_flags,
+ tree t);
+
+  void dump_printf_va (dump_flags_t dump_kind, const char *format,
+  va_list ap) ATTRIBUTE_PRINTF (3, 0);
+
+  void dump_printf_loc_va (dump_flags_t dump_kind, const dump_location_t &loc,
+  const char *format, va_list ap)
+ATTRIBUTE_PRINTF (4, 0);
+
+  template
+  void dump_dec (dump_flags_t dump_kind, const poly_int &value);
+
+  void dump_symtab_node (dump_flags_t dump_kind, symtab_node *node);
+
+  /* Managing nested scopes.  */
+  unsigned int get_scope_depth () const;
+  void begin_scope (const char *name, const dump_location_t &loc);
+  void end_scope ();
+
+  /* For use in selftests; if true then optinfo_enabled_p is true.  */
+  bool forcibly_enable_optinfo_p () const
+  {
+return m_forcibly_enable_optinfo;
+  }
+
+  void end_any_optinfo ();
+
+ private:
+  optinfo &ensure_pending_optinfo ();
+  optinfo &begin_next_optinfo (const dump_location_t &loc);
+
+  /* For use in selftests; if true then optinfo_enabled_p is true.  */
+  bool m_forcibly_enable_optinfo;
+
+  /* The current nesting depth of dump scopes, for showing nesting
+ via indentation).  */
+  unsigned int m_scope_depth;
+
+  /* The optinfo currently being accumulated since the last dump_*_loc call,
+ if any.  */
+  optinfo *m_pending;
+
+  /* The currently active dump_context, for use by the dump_* API calls.  */
+  static dump_context *s_current;
+
+  /* The default active context.  */
+  static dump_context s_default;
+};
+
+#if CHECKING_P
+
+/* An RAII-style class for use in selftests for temporarily using a different
+   dump_context.  */
+
+class temp_dump_context
+{
+ public:
+  temp_dump_context (bool fo

[PATCH 0/2] v4: optinfo framework and remarks

[David Malcolm]

On Fri, 2018-06-29 at 09:09 +0200, Richard Biener wrote:
> On Thu, Jun 28, 2018 at 4:29 PM David Malcolm 
> wrote:
> > 
> > On Thu, 2018-06-28 at 13:29 +0200, Richard Biener wrote:
> > > On Tue, Jun 26, 2018 at 3:54 PM David Malcolm  > > m>
> > > wrote:
> > > > 
> > > > On Mon, 2018-06-25 at 15:34 +0200, Richard Biener wrote:
> > > > > On Wed, Jun 20, 2018 at 6:34 PM David Malcolm  > > > > t.co
> > > > > m>
> > > > > wrote:
> > > > > > 
> > > > > > Here's v3 of the patch (one big patch this time, rather
> > > > > > than a
> > > > > > kit).
> > > > > > 
> > > > > > Like the v2 patch kit, this patch reuses the existing dump
> > > > > > API,
> > > > > > rather than inventing its own.
> > > > > > 
> > > > > > Specifically, it uses the dump_* functions in dumpfile.h
> > > > > > that
> > > > > > don't
> > > > > > take a FILE *, the ones that implicitly write to dump_file
> > > > > > and/or
> > > > > > alt_dump_file.  I needed a name for them, so I've taken to
> > > > > > calling
> > > > > > them the "structured dump API" (better name ideas welcome).
> > > > > > 
> > > > > > v3 eliminates v2's optinfo_guard class, instead using
> > > > > > "dump_*_loc"
> > > > > > calls as delimiters when consolidating "dump_*"
> > > > > > calls.  There's
> > > > > > a
> > > > > > new dump_context class which has responsibility for
> > > > > > consolidating
> > > > > > them into optimization records.
> > > > > > 
> > > > > > The dump_*_loc calls now capture more than just a
> > > > > > location_t:
> > > > > > they
> > > > > > capture the profile_count and the location in GCC's own
> > > > > > sources
> > > > > > where
> > > > > > the dump is being emitted from.
> > > > > > 
> > > > > > This works by introducing a new "dump_location_t" class as
> > > > > > the
> > > > > > argument of those dump_*_loc calls.  The dump_location_t
> > > > > > can
> > > > > > be constructed from a gimple * or from an rtx_insn *, so
> > > > > > that
> > > > > > rather than writing:
> > > > > > 
> > > > > >   dump_printf_loc (MSG_NOTE, gimple_location (stmt),
> > > > > >"some message: %i", 42);
> > > > > > 
> > > > > > you can write:
> > > > > > 
> > > > > >   dump_printf_loc (MSG_NOTE, stmt,
> > > > > >"some message: %i", 42);
> > > > > > 
> > > > > > and the dump_location_t constructor will grab the
> > > > > > location_t
> > > > > > and
> > > > > > profile_count of stmt, and the location of the
> > > > > > "dump_printf_loc"
> > > > > > callsite (and gracefully handle "stmt" being NULL).
> > > > > > 
> > > > > > Earlier versions of the patch captured the location of the
> > > > > > dump_*_loc call via preprocessor hacks, or didn't work
> > > > > > properly;
> > > > > > this version of the patch works more cleanly: internally,
> > > > > > dump_location_t is split into two new classes:
> > > > > >   * dump_user_location_t: the location_t and profile_count
> > > > > > within
> > > > > > the *user's code*, and
> > > > > >   * dump_impl_location_t: the __builtin_FILE/LINE/FUNCTION
> > > > > > within
> > > > > > the *implementation* code (i.e. GCC or a plugin),
> > > > > > captured
> > > > > > "automagically" via default params
> > > > > > 
> > > > > > These classes are sometimes used elsewhere in the
> > > > > > code.  For
> > > > > > example, "vect_location" becomes a dump_user_location_t
> > > > > &

[PATCH 2/2] optinfo: add diagnostic remarks

[David Malcolm]

 dump_enabled_p (void)
(a) the active dump_file, if any
(b) the -fopt-info destination, if any
(c) to the "optinfo" destinations, if any:
+   (c.1) as diagnostic "remarks"
 
dump_* (MSG_*) --> dumpfile.c --+--> (a) dump_file
|
@@ -449,6 +450,7 @@ dump_enabled_p (void)
|
`--> (c) optinfo
 `---> optinfo destinations
+  (c.1) diagnostic "remarks"
 
For optinfos, the dump_*_loc mark the beginning of an optinfo
instance: all subsequent dump_* calls are consolidated into
diff --git a/gcc/fortran/gfc-diagnostic.def b/gcc/fortran/gfc-diagnostic.def
index 565fa83..a10b6aa 100644
--- a/gcc/fortran/gfc-diagnostic.def
+++ b/gcc/fortran/gfc-diagnostic.def
@@ -37,6 +37,7 @@ DEFINE_DIAGNOSTIC_KIND (DK_SORRY, "sorry, unimplemented", 
"error")
 DEFINE_DIAGNOSTIC_KIND (DK_WARNING, "Warning", "warning")
 DEFINE_DIAGNOSTIC_KIND (DK_ANACHRONISM, "anachronism", "warning")
 DEFINE_DIAGNOSTIC_KIND (DK_NOTE, "note", "note")
+DEFINE_DIAGNOSTIC_KIND (DK_REMARK, "remark ", "remark")
 DEFINE_DIAGNOSTIC_KIND (DK_DEBUG, "debug", "note")
 /* These two would be re-classified as DK_WARNING or DK_ERROR, so the
 prefix does not matter.  */
diff --git a/gcc/opt-functions.awk b/gcc/opt-functions.awk
index 2c371e5..48ecac5 100644
--- a/gcc/opt-functions.awk
+++ b/gcc/opt-functions.awk
@@ -105,6 +105,7 @@ function switch_flags (flags)
  test_flag("Undocumented", flags,  " | CL_UNDOCUMENTED") \
  test_flag("NoDWARFRecord", flags,  " | CL_NO_DWARF_RECORD") \
  test_flag("Warning", flags,  " | CL_WARNING") \
+ test_flag("Remark", flags,  " | CL_REMARK") \
  test_flag("(Optimization|PerFunction)", flags,  " | CL_OPTIMIZATION")
sub( "^0 \\| ", "", result )
return result
diff --git a/gcc/optinfo-emit-diagnostics.cc b/gcc/optinfo-emit-diagnostics.cc
new file mode 100644
index 000..5320379
--- /dev/null
+++ b/gcc/optinfo-emit-diagnostics.cc
@@ -0,0 +1,317 @@
+/* Emit optimization information as "remark" diagnostics.
+   Copyright (C) 2018 Free Software Foundation, Inc.
+   Contributed by David Malcolm .
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3.  If not see
+<http://www.gnu.org/licenses/>.  */
+
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+
+#include "backend.h"
+#include "tree.h"
+#include "tree-pass.h"
+#include "gimple.h"
+#include "pretty-print.h"
+#include "tree-pretty-print.h"
+#include "gimple-pretty-print.h"
+#include "diagnostic.h"
+#include "diagnostic-color.h"
+#include "options.h"
+#include "cgraph.h"
+
+#include "optinfo.h"
+#include "optinfo-emit-diagnostics.h"
+#include "optinfo-internal.h"
+#include "dump-context.h"
+#include "selftest.h"
+#include "context.h"
+#include "pass_manager.h"
+
+/* Print the items within OPTINFO to PP (as part of a remark).  */
+
+static void
+print_optinfo_items (pretty_printer *pp, const optinfo *optinfo)
+{
+  bool show_color = pp_show_color (pp);
+
+  for (unsigned i = 0; i < optinfo->num_items (); i++)
+{
+  const optinfo_item *item = optinfo->get_item (i);
+  switch (item->get_kind ())
+   {
+   default:
+ gcc_unreachable ();
+   case OPTINFO_ITEM_KIND_TEXT:
+ {
+   const optinfo_item_text *as_text
+ = (const optinfo_item_text *)item;
+   pp_string (pp, as_text->get_text ());
+ }
+ break;
+   case OPTINFO_ITEM_KIND_TREE:
+ {
+   const optinfo_item_tree *as_tree
+ = (const optinfo_item_tree *)item;
+   pp_begin_quote (pp, show_color);
+   dump_generic_node (pp, as_tree->get_node (), 0, TDF_DETAILS,
+  false);
+   pp_end_quote (pp, show_color);
+ }
+ break;
+  

Re: [PATCH] -fopt-info: add indentation via DUMP_VECT_SCOPE

[David Malcolm]

On Tue, 2018-07-03 at 09:37 +0200, Richard Biener wrote:
> On Mon, Jul 2, 2018 at 7:00 PM David Malcolm 
> wrote:
> > 
> > On Mon, 2018-07-02 at 14:23 +0200, Christophe Lyon wrote:
> > > On Fri, 29 Jun 2018 at 10:09, Richard Biener  > > ail.
> > > com> wrote:
> > > > 
> > > > On Tue, Jun 26, 2018 at 5:43 PM David Malcolm  > > > com>
> > > > wrote:
> > > > > 
> > > > > This patch adds a concept of nested "scopes" to dumpfile.c's
> > > > > dump_*_loc
> > > > > calls, and wires it up to the DUMP_VECT_SCOPE macro in tree-
> > > > > vectorizer.h,
> > > > > so that the nested structure is shown in -fopt-info by
> > > > > indentation.
> > > > > 
> > > > > For example, this converts -fopt-info-all e.g. from:
> > > > > 
> > > > > test.c:8:3: note: === analyzing loop ===
> > > > > test.c:8:3: note: === analyze_loop_nest ===
> > > > > test.c:8:3: note: === vect_analyze_loop_form ===
> > > > > test.c:8:3: note: === get_loop_niters ===
> > > > > test.c:8:3: note: symbolic number of iterations is (unsigned
> > > > > int)
> > > > > n_9(D)
> > > > > test.c:8:3: note: not vectorized: loop contains function
> > > > > calls or
> > > > > data references that cannot be analyzed
> > > > > test.c:8:3: note: vectorized 0 loops in function
> > > > > 
> > > > > to:
> > > > > 
> > > > > test.c:8:3: note: === analyzing loop ===
> > > > > test.c:8:3: note:  === analyze_loop_nest ===
> > > > > test.c:8:3: note:   === vect_analyze_loop_form ===
> > > > > test.c:8:3: note:=== get_loop_niters ===
> > > > > test.c:8:3: note:   symbolic number of iterations is
> > > > > (unsigned
> > > > > int) n_9(D)
> > > > > test.c:8:3: note:   not vectorized: loop contains function
> > > > > calls
> > > > > or data references that cannot be analyzed
> > > > > test.c:8:3: note: vectorized 0 loops in function
> > > > > 
> > > > > showing that the "symbolic number of iterations" message is
> > > > > within
> > > > > the "=== analyze_loop_nest ===" (and not within the
> > > > > "=== vect_analyze_loop_form ===").
> > > > > 
> > > > > This is also enabling work for followups involving
> > > > > optimization
> > > > > records
> > > > > (allowing the records to directly capture the nested
> > > > > structure of
> > > > > the
> > > > > dump messages).
> > > > > 
> > > > > Successfully bootstrapped & regrtested on x86_64-pc-linux-
> > > > > gnu.
> > > > > 
> > > > > OK for trunk?
> > > 
> > > Hi,
> > > 
> > > I've noticed that this patch (r262246) caused regressions on
> > > aarch64:
> > > gcc.dg/vect/slp-perm-1.c -flto -ffat-lto-objects  scan-tree-
> > > dump
> > > vect "note: Built SLP cancelled: can use load/store-lanes"
> > > gcc.dg/vect/slp-perm-1.c scan-tree-dump vect "note: Built SLP
> > > cancelled: can use load/store-lanes"
> > > gcc.dg/vect/slp-perm-2.c -flto -ffat-lto-objects  scan-tree-
> > > dump
> > > vect "note: Built SLP cancelled: can use load/store-lanes"
> > > gcc.dg/vect/slp-perm-2.c scan-tree-dump vect "note: Built SLP
> > > cancelled: can use load/store-lanes"
> > > gcc.dg/vect/slp-perm-3.c -flto -ffat-lto-objects  scan-tree-
> > > dump
> > > vect "note: Built SLP cancelled: can use load/store-lanes"
> > > gcc.dg/vect/slp-perm-3.c scan-tree-dump vect "note: Built SLP
> > > cancelled: can use load/store-lanes"
> > > gcc.dg/vect/slp-perm-5.c -flto -ffat-lto-objects  scan-tree-
> > > dump
> > > vect "note: Built SLP cancelled: can use load/store-lanes"
> > > gcc.dg/vect/slp-perm-5.c scan-tree-dump vect "note: Built SLP
> > > cancelled: can use load/store-lanes"
> > > gcc.dg/vect/slp-perm-6.c -flto -ffat-lto-objects  scan-tree-
> > > dump
> > > vect "note: Built SLP cancelled: can use load/store-lanes"
> > > gcc.dg/vect/slp-per

[PATCH] Remove "note: " prefix from some scan-tree-dump directives

[David Malcolm]

On Tue, 2018-07-03 at 15:53 +0200, Richard Biener wrote:
> On Tue, Jul 3, 2018 at 3:52 PM David Malcolm 
> wrote:
> >
> > On Tue, 2018-07-03 at 09:37 +0200, Richard Biener wrote:
> > > On Mon, Jul 2, 2018 at 7:00 PM David Malcolm  > > >
> > > wrote:
> > > >
> > > > On Mon, 2018-07-02 at 14:23 +0200, Christophe Lyon wrote:
> > > > > On Fri, 29 Jun 2018 at 10:09, Richard Biener  > > > > r@gm
> > > > > ail.
> > > > > com> wrote:
> > > > > >
> > > > > > On Tue, Jun 26, 2018 at 5:43 PM David Malcolm  > > > > > hat.
> > > > > > com>
> > > > > > wrote:
> > > > > > >
> > > > > > > This patch adds a concept of nested "scopes" to
> > > > > > > dumpfile.c's
> > > > > > > dump_*_loc
> > > > > > > calls, and wires it up to the DUMP_VECT_SCOPE macro in
> > > > > > > tree-
> > > > > > > vectorizer.h,
> > > > > > > so that the nested structure is shown in -fopt-info by
> > > > > > > indentation.
> > > > > > >
> > > > > > > For example, this converts -fopt-info-all e.g. from:
> > > > > > >
> > > > > > > test.c:8:3: note: === analyzing loop ===
> > > > > > > test.c:8:3: note: === analyze_loop_nest ===
> > > > > > > test.c:8:3: note: === vect_analyze_loop_form ===
> > > > > > > test.c:8:3: note: === get_loop_niters ===
> > > > > > > test.c:8:3: note: symbolic number of iterations is
> > > > > > > (unsigned
> > > > > > > int)
> > > > > > > n_9(D)
> > > > > > > test.c:8:3: note: not vectorized: loop contains function
> > > > > > > calls or
> > > > > > > data references that cannot be analyzed
> > > > > > > test.c:8:3: note: vectorized 0 loops in function
> > > > > > >
> > > > > > > to:
> > > > > > >
> > > > > > > test.c:8:3: note: === analyzing loop ===
> > > > > > > test.c:8:3: note:  === analyze_loop_nest ===
> > > > > > > test.c:8:3: note:   === vect_analyze_loop_form ===
> > > > > > > test.c:8:3: note:=== get_loop_niters ===
> > > > > > > test.c:8:3: note:   symbolic number of iterations is
> > > > > > > (unsigned
> > > > > > > int) n_9(D)
> > > > > > > test.c:8:3: note:   not vectorized: loop contains
> > > > > > > function
> > > > > > > calls
> > > > > > > or data references that cannot be analyzed
> > > > > > > test.c:8:3: note: vectorized 0 loops in function
> > > > > > >
> > > > > > > showing that the "symbolic number of iterations" message
> > > > > > > is
> > > > > > > within
> > > > > > > the "=== analyze_loop_nest ===" (and not within the
> > > > > > > "=== vect_analyze_loop_form ===").
> > > > > > >
> > > > > > > This is also enabling work for followups involving
> > > > > > > optimization
> > > > > > > records
> > > > > > > (allowing the records to directly capture the nested
> > > > > > > structure of
> > > > > > > the
> > > > > > > dump messages).
> > > > > > >
> > > > > > > Successfully bootstrapped & regrtested on x86_64-pc-
> > > > > > > linux-
> > > > > > > gnu.
> > > > > > >
> > > > > > > OK for trunk?
> > > > >
> > > > > Hi,
> > > > >
> > > > > I've noticed that this patch (r262246) caused regressions on
> > > > > aarch64:
> > > > > gcc.dg/vect/slp-perm-1.c -flto -ffat-lto-objects  scan-
> > > > > tree-
> > > > > dump
> > > > > vect "note: Built SLP cancelled: can use load/store-lanes"
> > > > > gcc.dg/vect/slp-perm-1.c scan-tree-dump vect "note: Built
> > > > > SLP
> > > > > cancelled: can use load/store-lanes"

Re: Add support for dumping multiple dump files under one name

[David Malcolm]

On Tue, 2018-07-03 at 11:00 +0100, Andre Vieira (lists) wrote:
> On 29/06/18 11:13, David Malcolm wrote:
> > On Fri, 2018-06-29 at 10:15 +0200, Richard Biener wrote:
> > > On Fri, 22 Jun 2018, Jan Hubicka wrote:
> > > 
> > > > Hi,
> > > > this patch adds dumpfile support for dumps that come in
> > > > multiple
> > > > parts.  This
> > > > is needed for WPA stream-out dump since we stream partitions in
> > > > parallel and
> > > > the dumps would come up in random order.  Parts are added by
> > > > new
> > > > parameter that
> > > > is initialzed by default to -1 (no parts). 
> > > > 
> > > > One thing I skipped is any support for duplicated opening of
> > > > file
> > > > with parts since I do not need it.
> > > > 
> > > > Bootstrapped/regtested x86_64-linux, OK?
> > > 
> > > Looks reasonable - David, anything you want to add / have
> > > changed?
> > 
> > No worries from my side; I don't think it interacts with the
> > optimization records stuff I'm working on - presumably this is just
> > for
> > dumping the WPA stream-out, rather than for dumping specific
> > optimizations.
> > 
> > [...snip...]
> > 
> > Dave
> > 
> 
> Hi David,
> 
> I believe r262245 is causing the following failures on aarch64 and
> arm:
> 
> FAIL: gcc.dg/vect/slp-perm-1.c -flto -ffat-lto-objects  scan-tree-
> dump
> vect "note: Built SLP cancelled: can use load/store-lanes"
> FAIL: gcc.dg/vect/slp-perm-1.c scan-tree-dump vect "note: Built SLP
> cancelled: can use load/store-lanes"
> FAIL: gcc.dg/vect/slp-perm-2.c -flto -ffat-lto-objects  scan-tree-
> dump
> vect "note: Built SLP cancelled: can use load/store-lanes"
> FAIL: gcc.dg/vect/slp-perm-2.c scan-tree-dump vect "note: Built SLP
> cancelled: can use load/store-lanes"
> FAIL: gcc.dg/vect/slp-perm-3.c -flto -ffat-lto-objects  scan-tree-
> dump
> vect "note: Built SLP cancelled: can use load/store-lanes"
> FAIL: gcc.dg/vect/slp-perm-3.c scan-tree-dump vect "note: Built SLP
> cancelled: can use load/store-lanes"
> FAIL: gcc.dg/vect/slp-perm-5.c -flto -ffat-lto-objects  scan-tree-
> dump
> vect "note: Built SLP cancelled: can use load/store-lanes"
> FAIL: gcc.dg/vect/slp-perm-5.c scan-tree-dump vect "note: Built SLP
> cancelled: can use load/store-lanes"
> FAIL: gcc.dg/vect/slp-perm-6.c -flto -ffat-lto-objects  scan-tree-
> dump
> vect "note: Built SLP cancelled: can use load/store-lanes"
> FAIL: gcc.dg/vect/slp-perm-6.c scan-tree-dump vect "note: Built SLP
> cancelled: can use load/store-lanes"
> FAIL: gcc.dg/vect/slp-perm-7.c -flto -ffat-lto-objects  scan-tree-
> dump
> vect "note: Built SLP cancelled: can use load/store-lanes"
> FAIL: gcc.dg/vect/slp-perm-7.c scan-tree-dump vect "note: Built SLP
> cancelled: can use load/store-lanes"
> FAIL: gcc.dg/vect/slp-perm-8.c -flto -ffat-lto-objects  scan-tree-
> dump
> vect "note: Built SLP cancelled: can use load/store-lanes"
> FAIL: gcc.dg/vect/slp-perm-8.c scan-tree-dump vect "note: Built SLP
> cancelled: can use load/store-lanes"
> 
> Could you please have a look?

Sorry about this; I think my r262246 ("dumpfile.c: add indentation via
DUMP_VECT_SCOPE") caused this.

Does https://gcc.gnu.org/ml/gcc-patches/2018-07/msg00122.html help?

Dave

[PATCH 0/5] [RFC v2] Higher-level reporting of vectorization problems

[David Malcolm]

On Mon, 2018-06-25 at 11:10 +0200, Richard Biener wrote:
> On Fri, 22 Jun 2018, David Malcolm wrote:
> 
> > NightStrike and I were chatting on IRC last week about
> > issues with trying to vectorize the following code:
> > 
> > #include 
> > std::size_t f(std::vector> const & v) {
> > std::size_t ret = 0;
> > for (auto const & w: v)
> > ret += w.size();
> > return ret;
> > }
> > 
> > icc could vectorize it, but gcc couldn't, but neither of us could
> > immediately figure out what the problem was.
> > 
> > Using -fopt-info leads to a wall of text.
> > 
> > I tried using my patch here:
> > 
> >  "[PATCH] v3 of optinfo, remarks and optimization records"
> >   https://gcc.gnu.org/ml/gcc-patches/2018-06/msg01267.html
> > 
> > It improved things somewhat, by showing:
> > (a) the nesting structure via indentation, and
> > (b) the GCC line at which each message is emitted (by using the
> > "remark" output)
> > 
> > but it's still a wall of text:
> > 
> >   https://dmalcolm.fedorapeople.org/gcc/2018-06-18/test.cc.remarks.
> > html
> >   https://dmalcolm.fedorapeople.org/gcc/2018-06-18/test.cc.d/..%7C.
> > .%7Csrc%7Ctest.cc.html#line-4
> > 
> > It doesn't yet provide a simple high-level message to a
> > tech-savvy user on what they need to do to get GCC to
> > vectorize their loop.
> 
> Yeah, in particular the vectorizer is way too noisy in its low-level
> functions.  IIRC -fopt-info-vec-missed is "somewhat" better:
> 
> t.C:4:26: note: step unknown.
> t.C:4:26: note: vector alignment may not be reachable
> t.C:4:26: note: not ssa-name.
> t.C:4:26: note: use not simple.
> t.C:4:26: note: not ssa-name.
> t.C:4:26: note: use not simple.
> t.C:4:26: note: no array mode for V2DI[3]
> t.C:4:26: note: Data access with gaps requires scalar epilogue loop
> t.C:4:26: note: can't use a fully-masked loop because the target
> doesn't 
> have the appropriate masked load or store.
> t.C:4:26: note: not ssa-name.
> t.C:4:26: note: use not simple.
> t.C:4:26: note: not ssa-name.
> t.C:4:26: note: use not simple.
> t.C:4:26: note: no array mode for V2DI[3]
> t.C:4:26: note: Data access with gaps requires scalar epilogue loop
> t.C:4:26: note: op not supported by target.
> t.C:4:26: note: not vectorized: relevant stmt not supported: _15 =
> _14 
> /[ex] 4;
> t.C:4:26: note: bad operation or unsupported loop bound.
> t.C:4:26: note: not vectorized: no grouped stores in basic block.
> t.C:4:26: note: not vectorized: no grouped stores in basic block.
> t.C:6:12: note: not vectorized: not enough data-refs in basic block.
> 
> 
> > The pertinent dump messages are:
> > 
> > test.cc:4:23: remark: === try_vectorize_loop_1 ===
> > [../../src/gcc/tree-vectorizer.c:674:try_vectorize_loop_1]
> > cc1plus: remark:
> > Analyzing loop at test.cc:4
> > [../../src/gcc/dumpfile.c:735:ensure_pending_optinfo]
> > test.cc:4:23: remark:  === analyze_loop_nest ===
> > [../../src/gcc/tree-vect-loop.c:2299:vect_analyze_loop]
> > [...snip...]
> > test.cc:4:23: remark:   === vect_analyze_loop_operations ===
> > [../../src/gcc/tree-vect-loop.c:1520:vect_analyze_loop_operations]
> > [...snip...]
> > test.cc:4:23: remark:==> examining statement: ‘_15 = _14 /[ex]
> > 4;’ [../../src/gcc/tree-vect-stmts.c:9382:vect_analyze_stmt]
> > test.cc:4:23: remark:vect_is_simple_use: operand ‘_14’
> > [../../src/gcc/tree-vect-stmts.c:10064:vect_is_simple_use]
> > test.cc:4:23: remark:def_stmt: ‘_14 = _8 - _7;’
> > [../../src/gcc/tree-vect-stmts.c:10098:vect_is_simple_use]
> > test.cc:4:23: remark:type of def: internal [../../src/gcc/tree-
> > vect-stmts.c:10112:vect_is_simple_use]
> > test.cc:4:23: remark:vect_is_simple_use: operand ‘4’
> > [../../src/gcc/tree-vect-stmts.c:10064:vect_is_simple_use]
> > test.cc:4:23: remark:op not supported by target.
> > [../../src/gcc/tree-vect-stmts.c:5932:vectorizable_operation]
> > test.cc:4:23: remark:not vectorized: relevant stmt not
> > supported: ‘_15 = _14 /[ex] 4;’ [../../src/gcc/tree-vect-
> > stmts.c:9565:vect_analyze_stmt]
> > test.cc:4:23: remark:   bad operation or unsupported loop bound.
> > [../../src/gcc/tree-vect-loop.c:2043:vect_analyze_loop_2]
> > cc1plus: remark: vectorized 0 loops in function.
> > [../../src/gcc/tree-vectorizer.c:904:vectorize_loops]
> > 
> > In particular, that complaint from
> >   [../../src/gcc/tree-vect-stmts.c:9565:vect_analyze_stmt]
> > is coming from:

[PATCH 2/5] Use opt-problem.h in try_vectorize_loop_1

[David Malcolm]

gcc/ChangeLog:
* tree-vectorizer.c: Include "opt-problem.h".
(try_vectorize_loop_1): Convert "loop_vinfo" from loop_vec_info to
opt_loop_vec_info loop_vinfo, and use opt_report to report any
opt_problem thus captured.  Use opt_report ro report on successful
loop vectorization.
---
 gcc/tree-vectorizer.c | 34 --
 1 file changed, 28 insertions(+), 6 deletions(-)

diff --git a/gcc/tree-vectorizer.c b/gcc/tree-vectorizer.c
index 8d54fbb..c60d0d9 100644
--- a/gcc/tree-vectorizer.c
+++ b/gcc/tree-vectorizer.c
@@ -79,6 +79,7 @@ along with GCC; see the file COPYING3.  If not see
 #include "stringpool.h"
 #include "attribs.h"
 #include "gimple-pretty-print.h"
+#include "opt-problem.h"
 
 
 /* Loop or bb location, with hotness information.  */
@@ -705,9 +706,17 @@ try_vectorize_loop_1 (hash_table 
*&simduid_to_vf_htab,
 LOCATION_FILE (vect_location.get_location_t ()),
 LOCATION_LINE (vect_location.get_location_t ()));
 
-  loop_vec_info loop_vinfo = vect_analyze_loop (loop, orig_loop_vinfo, 
&shared);
+  opt_loop_vec_info loop_vinfo = vect_analyze_loop (loop, orig_loop_vinfo, 
&shared);
   loop->aux = loop_vinfo;
 
+  if (!loop_vinfo)
+if (loop_vinfo.get_problem ())
+  {
+   opt_report report;
+   if (report.remark (vect_location, "couldn't vectorize loop"))
+ loop_vinfo.get_problem ()->report_reason (report);
+  }
+
   if (!loop_vinfo || !LOOP_VINFO_VECTORIZABLE_P (loop_vinfo))
 {
   /* Free existing information if loop is analyzed with some
@@ -775,13 +784,26 @@ try_vectorize_loop_1 (hash_table 
*&simduid_to_vf_htab,
 
   unsigned HOST_WIDE_INT bytes;
   if (current_vector_size.is_constant (&bytes))
-dump_printf_loc (MSG_OPTIMIZED_LOCATIONS, vect_location,
-"loop vectorized vectorized using "
+{
+  dump_printf_loc (MSG_OPTIMIZED_LOCATIONS, vect_location,
+  "loop vectorized vectorized using "
+  HOST_WIDE_INT_PRINT_UNSIGNED " byte "
+  "vectors\n", bytes);
+  opt_report report;
+  // FIXME: is this the correct format code?
+  report.remark (vect_location,
+"loop vectorized using "
 HOST_WIDE_INT_PRINT_UNSIGNED " byte "
-"vectors\n", bytes);
+"vectors", bytes);
+}
   else
-dump_printf_loc (MSG_OPTIMIZED_LOCATIONS, vect_location,
-"loop vectorized using variable length vectors\n");
+{
+  dump_printf_loc (MSG_OPTIMIZED_LOCATIONS, vect_location,
+  "loop vectorized using variable length vectors\n");
+  opt_report report;
+  report.remark (vect_location,
+"loop vectorized using variable length vectors");
+}
 
   loop_p new_loop = vect_transform_loop (loop_vinfo);
   (*num_vectorized_loops)++;
-- 
1.8.5.3

[PATCH 3/5] Add some test coverage

[David Malcolm]

In particular, note how this allows us to highlight specific loops in
testcases (via dg-remark), and to highlight the specific lines that cause
problems to the vectorizer (via dg-message).

gcc/testsuite/ChangeLog:
* gcc.dg/vect/vect-alias-check-4.c: Add -fremarks to options.  Add
dg-remark and dg-message directives to the cover the expected
vectorization failure reports.
* gcc.dg/vect/vect-remarks-1.c: New test.
---
 gcc/testsuite/gcc.dg/vect/vect-alias-check-4.c | 11 +++
 gcc/testsuite/gcc.dg/vect/vect-remarks-1.c | 18 ++
 2 files changed, 25 insertions(+), 4 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/vect/vect-remarks-1.c

diff --git a/gcc/testsuite/gcc.dg/vect/vect-alias-check-4.c 
b/gcc/testsuite/gcc.dg/vect/vect-alias-check-4.c
index 1e5fc27..b08b4b4 100644
--- a/gcc/testsuite/gcc.dg/vect/vect-alias-check-4.c
+++ b/gcc/testsuite/gcc.dg/vect/vect-alias-check-4.c
@@ -1,6 +1,6 @@
 /* { dg-do compile } */
 /* { dg-require-effective-target vect_int } */
-/* { dg-additional-options "--param vect-max-version-for-alias-checks=0" } */
+/* { dg-additional-options "--param vect-max-version-for-alias-checks=0 
-fremarks" } */
 
 #define N 16
 
@@ -14,22 +14,25 @@ union u { struct s2 f; struct s3 g; };
 void
 f1 (int a[][N], int b[][N])
 {
-  for (int i = 0; i < N; ++i)
+  for (int i = 0; i < N; ++i) /* { dg-remark "couldn't vectorize loop" } */
 a[0][i] += b[0][i];
+  /* { dg-message "will not create alias checks, as --param 
vect-max-version-for-alias-checks == 0" "" { target *-*-* } .-2 } */
 }
 
 void
 f2 (union u *a, union u *b)
 {
-  for (int i = 0; i < N; ++i)
+  for (int i = 0; i < N; ++i) /* { dg-remark "couldn't vectorize loop" } */
 a->f.b.a[i] += b->g.e.a[i];
+  /* { dg-message "will not create alias checks, as --param 
vect-max-version-for-alias-checks == 0" "" { target *-*-* } .-2 } */
 }
 
 void
 f3 (struct s1 *a, struct s1 *b)
 {
-  for (int i = 0; i < N - 1; ++i)
+  for (int i = 0; i < N - 1; ++i) /* { dg-remark "couldn't vectorize loop" } */
 a->a[i + 1] += b->a[i];
+  /* { dg-message "possible dependence between data-refs" "" { target *-*-* } 
.-2 } */
 }
 
 /* { dg-final { scan-tree-dump-not "LOOP VECTORIZED" "vect" } } */
diff --git a/gcc/testsuite/gcc.dg/vect/vect-remarks-1.c 
b/gcc/testsuite/gcc.dg/vect/vect-remarks-1.c
new file mode 100644
index 000..5006742
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/vect/vect-remarks-1.c
@@ -0,0 +1,18 @@
+/* { dg-do compile } */
+/* { dg-options "-fremarks -O3" } */
+
+extern void accumulate (int x, int *a);
+
+int test_1 (int arr[], int n)
+{
+  int sum = 0;
+  for (int i = 0; i < n; ++i) /* { dg-remark "couldn't vectorize loop" } */
+accumulate (arr[i], &sum); /* { dg-message "statement clobbers memory: 
'accumulate .*'" } */
+  return sum;
+}
+
+void test_2 (int *dst, int *src_a, int *src_b, int n)
+{
+  for (int i = 0; i < n; i++) /* { dg-remark "loop vectorized" } */
+dst[i] = src_a[i] + src_b[i];
+}
-- 
1.8.5.3

[PATCH 1/5] Add opt-problem.h

[David Malcolm]

gcc/ChangeLog:
* opt-problem.h: New file.
* tree-vectorizer.h (opt_loop_vec_info): New typedef.
---
 gcc/opt-problem.h | 326 ++
 gcc/tree-vectorizer.h |   6 +
 2 files changed, 332 insertions(+)
 create mode 100644 gcc/opt-problem.h

diff --git a/gcc/opt-problem.h b/gcc/opt-problem.h
new file mode 100644
index 000..100eed0
--- /dev/null
+++ b/gcc/opt-problem.h
@@ -0,0 +1,326 @@
+/* Rich information on why an optimization wasn't possible.
+   Copyright (C) 2018 Free Software Foundation, Inc.
+   Contributed by David Malcolm .
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3.  If not see
+<http://www.gnu.org/licenses/>.  */
+
+#ifndef GCC_OPT_PROBLEM_H
+#define GCC_OPT_PROBLEM_H
+
+#include "diagnostic-core.h" /* for ATTRIBUTE_GCC_DIAG.  */
+
+/* This header declares a family of wrapper classes for tracking a
+   success/failure value, while optionally supporting propagating an
+   opt_problem * describing any failure back up the call stack.
+
+   For instance, at the deepest point of the callstack where the failure
+   happens, rather than:
+
+ if (!check_something ())
+   {
+ if (dump_enabled_p ())
+   dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,
+"foo is unsupported.\n");
+ return false;
+   }
+ // [...more checks...]
+
+ // All checks passed:
+ return true;
+
+   we can capture the cause of the failure via:
+
+ if (!check_something ())
+   {
+ if (dump_enabled_p ())
+   dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,
+"foo is unsupported.\n");
+ return opt_result::failure ("foo is unsupported",
+ stmt);
+   }
+ // [...more checks...]
+
+ // All checks passed:
+ return opt_result::success ();
+
+   which effectively returns true or false, whilst recording any problem.
+
+   opt_result::success and opt_result::failure return opt_result values
+   which "looks like" true/false respectively, via operator bool().
+   If dump_enabled_p, then opt_result::failure also creates an opt_problem *,
+   capturing the pertinent data (here, "foo is unsupported " and "stmt").
+   If dumps are disabled, then opt_problem instances aren't
+   created, and it's equivalent to just returning a bool.
+
+   The opt_problem can be propagated via opt_result values back up
+   the call stack to where it makes most sense to the user.
+   For instance, rather than:
+
+ bool ok = try_something_that_might_fail ();
+ if (!ok)
+   {
+ if (dump_enabled_p ())
+   dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,
+"some message.\n");
+ return false;
+   }
+
+   we can replace the bool with an opt_result, so if dump_enabled_p, we
+   assume that if try_something_that_might_fail, an opt_problem * will be
+   created, and we can propagate it up the call chain:
+
+ opt_result ok = try_something_that_might_fail ();
+ if (!ok)
+   {
+ if (dump_enabled_p ())
+   dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,
+"some message.\n");
+ return ok; // propagating the opt_result
+   }
+
+   opt_result is an opt_wrapper, where opt_wrapper is a base
+   class for wrapping a T, optionally propagating an opt_problem in
+   case of failure (when dumps are enabled).  Similarly,
+   opt_pointer_wrapper can be used to wrap pointer types (where non-NULL
+   signifies success, NULL signifies failure).
+
+   In all cases, opt_wrapper acts as if the opt_problem were one of its
+   fields, but the opt_problem is actually stored in a global, so that when
+   compiled, an opt_wrapper is effectively just a T, so that we're
+   still just passing e.g. a bool around; the opt_wrapper classes
+   simply provide type-checking and an API to ensure that we provide
+   error-messages deep in the callstack at the places where problems
+   occur, and that we propagate them.  This also avoids having
+   to manage the ownership of the opt_problem instances.
+
+   Using opt_result and opt_wrapper documents the intent of the code
+   for the places where we represent success values, and allows the C++ type
+

[PATCH 5/5] Add opt-problem.cc

[David Malcolm]

gcc/ChangeLog:
* Makefile.in (OBJS): Add opt-problem.o.
* opt-problem.cc: New file.
---
 gcc/Makefile.in|  1 +
 gcc/opt-problem.cc | 96 ++
 2 files changed, 97 insertions(+)
 create mode 100644 gcc/opt-problem.cc

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index c5c3d3c..fb262da 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -1425,6 +1425,7 @@ OBJS = \
omp-grid.o \
omp-low.o \
omp-simd-clone.o \
+   opt-problem.o \
optabs.o \
optabs-libfuncs.o \
optabs-query.o \
diff --git a/gcc/opt-problem.cc b/gcc/opt-problem.cc
new file mode 100644
index 000..f518b16
--- /dev/null
+++ b/gcc/opt-problem.cc
@@ -0,0 +1,96 @@
+/* Rich information on why an optimization wasn't possible.
+   Copyright (C) 2018 Free Software Foundation, Inc.
+   Contributed by David Malcolm .
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3.  If not see
+<http://www.gnu.org/licenses/>.  */
+
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "backend.h"
+#include "diagnostic.h"
+#include "tree.h"
+#include "gimple.h"
+#include "pretty-print.h"
+#include "gimple-pretty-print.h"
+#include "opt-problem.h"
+
+/* Emit a remark about an optimization.  */
+
+bool
+opt_report::remark (dump_location_t loc, const char *gmsgid, ...)
+{
+  if (!flag_remarks)
+return false;
+
+  va_list ap;
+  va_start (ap, gmsgid);
+  bool ret = emit_diagnostic_valist (DK_REMARK, loc.get_location_t (), -1,
+gmsgid, &ap);
+  va_end (ap);
+
+  // FIXME: how should this interact with optinfo? (we don't want a duplicate 
remark)
+  return ret;
+}
+
+/* Emit a note about an optimization.  */
+
+bool
+opt_report::note (dump_location_t loc, const char *gmsgid, ...)
+{
+  va_list ap;
+  va_start (ap, gmsgid);
+  bool ret = emit_diagnostic_valist (DK_NOTE, loc.get_location_t (), -1,
+gmsgid, &ap);
+  va_end (ap);
+
+  // FIXME: how should this interact with optinfo? (we don't want a duplicate 
note)
+  return ret;
+}
+
+opt_problem *opt_problem::s_the_problem;
+
+// FIXME: some refactoring is needed, based on exactly where remarks land,
+// and how this interacts with the optinfo stuff.  For now, this decl:
+
+extern void
+print_impl_location (pretty_printer *pp, const dump_impl_location_t &impl_loc);
+
+/* Emit a diagnostic note describing why an optimization wasn't possible.  */
+
+void
+opt_problem::report_reason (opt_report &report)
+{
+  bool show_color = pp_show_color (global_dc->printer);
+
+  pretty_printer pp;
+  pp_show_color (&pp) = pp_show_color (global_dc->printer);
+  pp_string (&pp, m_text);
+  if (m_stmt)
+{
+  pp_string (&pp, ": ");
+  pp_begin_quote (&pp, show_color);
+  pp_gimple_stmt_1 (&pp, m_stmt, 0, TDF_SLIM);
+  pp_end_quote (&pp, show_color);
+}
+
+  print_impl_location (&pp, m_location.get_impl_location ());
+
+  const char *msg = pp_formatted_text (&pp);
+
+  report.note (m_location, "%s", msg);
+}
-- 
1.8.5.3

[PATCH 4/5] Use opt_result throughout vectorizer

[David Malcolm]

This patch uses the opt_result framework throughout the vectorizer
(and a few other places) so that specific details on problems are
propagated back up to the top-level of the vectorizer.

The changes are mostly fairly mechanical.

There are various FIXMEs in the code: for example, in various places
it feels like it would be useful to have a formatted printing API
for opt_result::failure etc, or other ways to capture the information
of interest (e.g. if there's a dependence between two data references
that's blocking vectorization, we probably should capture them so
we can highlight them to the user).

Almost everywhere using "bool" to handle error-checking uses "true"
for success, but there was one place where "true" meant "failure":
vect_analyze_data_ref_dependence, so the patch changes the sense
of that function's return value (as well as converting it from
bool to opt_result).

Placeholder ChangeLog follows:

gcc/ChangeLog:
* tree-data-ref.c: Use opt-problem.h in many places.
* tree-data-ref.h: Likewise.
* tree-vect-data-refs.c: Likewise.
* tree-vect-loop.c: Likewise.
* tree-vect-slp.c: Likewise.
* tree-vect-stmts.c: Likewise.
* tree-vectorizer.h: Likewise.
---
 gcc/tree-data-ref.c   |  33 +---
 gcc/tree-data-ref.h   |  10 ++-
 gcc/tree-vect-data-refs.c | 189 -
 gcc/tree-vect-loop.c  | 212 +++---
 gcc/tree-vect-slp.c   |   4 +-
 gcc/tree-vect-stmts.c | 130 
 gcc/tree-vectorizer.h |  35 
 7 files changed, 360 insertions(+), 253 deletions(-)

diff --git a/gcc/tree-data-ref.c b/gcc/tree-data-ref.c
index b163eaf..a6319f83 100644
--- a/gcc/tree-data-ref.c
+++ b/gcc/tree-data-ref.c
@@ -827,7 +827,7 @@ canonicalize_base_object_address (tree addr)
Return true if the analysis succeeded and store the results in DRB if so.
BB analysis can only fail for bitfield or reversed-storage accesses.  */
 
-bool
+opt_result
 dr_analyze_innermost (innermost_loop_behavior *drb, tree ref,
  struct loop *loop)
 {
@@ -851,14 +851,14 @@ dr_analyze_innermost (innermost_loop_behavior *drb, tree 
ref,
 {
   if (dump_file && (dump_flags & TDF_DETAILS))
fprintf (dump_file, "failed: bit offset alignment.\n");
-  return false;
+  return opt_result::failure ("bit offset alignment", NULL);
 }
 
   if (preversep)
 {
   if (dump_file && (dump_flags & TDF_DETAILS))
fprintf (dump_file, "failed: reverse storage order.\n");
-  return false;
+  return opt_result::failure ("reverse storage order", NULL);
 }
 
   /* Calculate the alignment and misalignment for the inner reference.  */
@@ -898,7 +898,9 @@ dr_analyze_innermost (innermost_loop_behavior *drb, tree 
ref,
 {
  if (dump_file && (dump_flags & TDF_DETAILS))
fprintf (dump_file, "failed: evolution of base is not affine.\n");
- return false;
+ // FIXME: should this propagate an opt_problem from simple_iv?
+ return opt_result::failure ("evolution of base is not affine",
+ NULL);
 }
 }
   else
@@ -924,7 +926,9 @@ dr_analyze_innermost (innermost_loop_behavior *drb, tree 
ref,
 {
  if (dump_file && (dump_flags & TDF_DETAILS))
fprintf (dump_file, "failed: evolution of offset is not affine.\n");
- return false;
+ // FIXME: should this propagate an opt_problem from simple_iv?
+ return opt_result::failure ("evolution of offset is not affine",
+ NULL);
 }
 }
 
@@ -981,7 +985,7 @@ dr_analyze_innermost (innermost_loop_behavior *drb, tree 
ref,
   if (dump_file && (dump_flags & TDF_DETAILS))
 fprintf (dump_file, "success.\n");
 
-  return true;
+  return opt_result::success ();
 }
 
 /* Return true if OP is a valid component reference for a DR access
@@ -1318,7 +1322,7 @@ data_ref_compare_tree (tree t1, tree t2)
 /* Return TRUE it's possible to resolve data dependence DDR by runtime alias
check.  */
 
-bool
+opt_result
 runtime_alias_check_p (ddr_p ddr, struct loop *loop, bool speed_p)
 {
   if (dump_enabled_p ())
@@ -1336,7 +1340,8 @@ runtime_alias_check_p (ddr_p ddr, struct loop *loop, bool 
speed_p)
dump_printf (MSG_MISSED_OPTIMIZATION,
 "runtime alias check not supported when optimizing "
 "for size.\n");
-  return false;
+  return opt_result::failure ("runtime alias check not supported when"
+ " optimizing for size", NULL);
 }
 
   /* FORNOW: We don't support versioning with outer-loop in either
@@ -1346,10 +1351,11 @@ runtime_alias_check_p (ddr_p ddr, struct loop *loop, 
bool speed_p)
   if (dump_enabled_p ())
dump_printf (MSG_MISSED_OPTIMIZATION,
 "runtime alias check not supported for outer 

Re: [PATCH 1/2] Add "optinfo" framework

[David Malcolm]

On Mon, 2018-07-09 at 15:00 +0200, Richard Biener wrote:
> On Mon, Jul 2, 2018 at 10:51 PM David Malcolm 
> wrote:
> > 
> > This patch implements a way to consolidate dump_* calls into
> > optinfo objects, as enabling work towards being able to write out
> > optimization records to a file, or emit them as diagnostic
> > "remarks".
> > 
> > The patch adds the support for building optinfo instances from
> > dump_*
> > calls, but leaves implementing any *users* of them to followup
> > patches.
> > 
> > Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
> > 
> > OK for trunk?
> 
> Looks good overall, but ...
> 
> To "fix" the GC issue you'd need to capture all possibly interesting
> information from tree/gimple while it is still in flight.  This _may_
> be
> necessary anyway since I remember writing code like
> 
>   fprintf (dump_file, "old: ");
>   print_gimple_stmt (..., stmt);
>   gimple_set_rhs1 (stmt, op);
>   fprintf (dump_file, "new: ");
>   print_gmple_stmt (..., stmt);
>   fprintf (dump_file, "\n");
> 
> capturing interesting information means we know all targeted
> optinfo channels, right?  And the optinfo consumers
> need to handle "streams" of input and may not look back.

> I've yet have to look at the 2nd patch but can you comment on
> this?  How difficult is it to re-wire how the data flows to make
> stmt re-use like the above possible?

I *think* it's doable: rather than capture, say, a gimple *, the
optinfo_item would capture the result of pp_gimple_stmt_1, plus some
metadata.  In fact, it would probably allow for removing the
optinfo_item subclasses, making optinfo_item concrete, containing
something like:

  /* Textual form.  */
  char *m_text;
  bool m_ownership_of_text;

  /* Metadata for optimization records.  */
  enum optinfo_item_kind m_kind;
  location_t m_location;

or somesuch.

I'll have a go at implementing this.

Thanks
Dave

> Thanks,
> Richard.
> 
> > gcc/ChangeLog:
> > * Makefile.in (OBJS): Add optinfo.o.
> > * coretypes.h (class symtab_node): New forward decl.
> > (struct cgraph_node): New forward decl.
> > (class varpool_node): New forward decl.
> > * dump-context.h: New file.
> > * dumpfile.c: Include "optinfo.h", "dump-context.h",
> > "cgraph.h",
> > "tree-pass.h", "optinfo-internal.h".
> > (refresh_dumps_are_enabled): Use optinfo_enabled_p.
> > (set_dump_file): Call
> > dumpfile_ensure_any_optinfo_are_flushed.
> > (set_alt_dump_file): Likewise.
> > (dump_context::~dump_context): New dtor.
> > (dump_gimple_stmt): Move implementation to...
> > (dump_context::dump_gimple_stmt): ...this new member
> > function.
> > Add the stmt to any pending optinfo, creating one if need
> > be.
> > (dump_gimple_stmt_loc): Move implementation to...
> > (dump_context::dump_gimple_stmt_loc): ...this new member
> > function.
> > Convert param "loc" from location_t to const
> > dump_location_t &.
> > Start a new optinfo and add the stmt to it.
> > (dump_generic_expr): Move implementation to...
> > (dump_context::dump_generic_expr): ...this new member
> > function.
> > Add the tree to any pending optinfo, creating one if need
> > be.
> > (dump_generic_expr_loc): Move implementation to...
> > (dump_context::dump_generic_expr_loc): ...this new member
> > function.  Add the tree to any pending optinfo, creating
> > one if
> > need be.
> > (dump_printf): Move implementation to...
> > (dump_context::dump_printf_va): ...this new member
> > function.  Add
> > the text to any pending optinfo, creating one if need be.
> > (dump_printf_loc): Move implementation to...
> > (dump_context::dump_printf_loc_va): ...this new member
> > function.
> > Convert param "loc" from location_t to const
> > dump_location_t &.
> > Start a new optinfo and add the stmt to it.
> > (dump_dec): Move implementation to...
> > (dump_context::dump_dec): ...this new member function.  Add
> > the
> > value to any pending optinfo, creating one if need be.
> > (dump_context::dump_symtab_node): New member function.
> > (dump_context::get_scope_depth): New member function.
> > (dump_context::begin_scope

[PATCH 2/2] Add "-fsave-optimization-record"

[David Malcolm]

   /* the value of the name */
+};
+
 #else
 
 struct _dont_use_rtx_here_;
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index 56cd122..c239c53 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -417,7 +417,8 @@ Objective-C and Objective-C++ Dialects}.
 -freorder-blocks-algorithm=@var{algorithm} @gol
 -freorder-blocks-and-partition  -freorder-functions @gol
 -frerun-cse-after-loop  -freschedule-modulo-scheduled-loops @gol
--frounding-math  -fsched2-use-superblocks  -fsched-pressure @gol
+-frounding-math  -fsave-optimization-record @gol
+-fsched2-use-superblocks  -fsched-pressure @gol
 -fsched-spec-load  -fsched-spec-load-dangerous @gol
 -fsched-stalled-insns-dep[=@var{n}]  -fsched-stalled-insns[=@var{n}] @gol
 -fsched-group-heuristic  -fsched-critical-path-heuristic @gol
@@ -9907,6 +9908,11 @@ Future versions of GCC may provide finer control of this 
setting
 using C99's @code{FENV_ACCESS} pragma.  This command-line option
 will be used to specify the default state for @code{FENV_ACCESS}.
 
+@item -fsave-optimization-record
+@opindex fsave-optimization-record
+Write a SRCFILE.opt-record.json file detailing what optimizations
+were performed.
+
 @item -fsignaling-nans
 @opindex fsignaling-nans
 Compile code assuming that IEEE signaling NaNs may generate user-visible
diff --git a/gcc/dumpfile.c b/gcc/dumpfile.c
index 38f9539..331b7fb 100644
--- a/gcc/dumpfile.c
+++ b/gcc/dumpfile.c
@@ -37,6 +37,7 @@ along with GCC; see the file COPYING3.  If not see
 #include "dump-context.h"
 #include "cgraph.h"
 #include "tree-pass.h" /* for "current_pass".  */
+#include "optinfo-emit-json.h"
 
 /* If non-NULL, return one past-the-end of the matching SUBPART of
the WHOLE string.  */
@@ -118,14 +119,6 @@ static struct dump_file_info dump_files[TDI_end] =
   DUMP_FILE_INFO (NULL, "ipa-all", DK_ipa, 0),
 };
 
-/* Define a name->number mapping for a dump flag value.  */
-template 
-struct kv_pair
-{
-  const char *const name;  /* the name of the value */
-  const ValueType value;   /* the value of the name */
-};
-
 /* Table of dump options. This must be consistent with the TDF_* flags
in dumpfile.h and opt_info_options below. */
 static const kv_pair dump_options[] =
@@ -176,7 +169,7 @@ static const kv_pair 
optinfo_verbosity_options[] =
 };
 
 /* Flags used for -fopt-info groups.  */
-static const kv_pair optgroup_options[] =
+const kv_pair optgroup_options[] =
 {
   {"ipa", OPTGROUP_IPA},
   {"loop", OPTGROUP_LOOP},
@@ -794,6 +787,7 @@ dump_context::end_scope ()
 {
   end_any_optinfo ();
   m_scope_depth--;
+  optimization_records_maybe_pop_dump_scope ();
 }
 
 /* Return the optinfo currently being accumulated, creating one if
@@ -820,8 +814,7 @@ dump_context::begin_next_optinfo (const dump_location_t 
&loc)
 }
 
 /* End any optinfo that has been accumulated within this context; emitting
-   it to any destinations as appropriate - though none have currently been
-   implemented.  */
+   it to any destinations as appropriate, such as optimization records.  */
 
 void
 dump_context::end_any_optinfo ()
diff --git a/gcc/dumpfile.h b/gcc/dumpfile.h
index 3096a89..e4823f8 100644
--- a/gcc/dumpfile.h
+++ b/gcc/dumpfile.h
@@ -420,6 +420,7 @@ extern FILE *dump_begin (int, dump_flags_t *);
 extern void dump_end (int, FILE *);
 extern int opt_info_switch_p (const char *);
 extern const char *dump_flag_name (int);
+extern const kv_pair optgroup_options[];
 
 /* Global variables used to communicate with passes.  */
 extern FILE *dump_file;
@@ -442,6 +443,7 @@ dump_enabled_p (void)
(a) the active dump_file, if any
(b) the -fopt-info destination, if any
(c) to the "optinfo" destinations, if any:
+   (c.1) as optimization records
 
dump_* (MSG_*) --> dumpfile.c --+--> (a) dump_file
|
@@ -449,6 +451,7 @@ dump_enabled_p (void)
|
`--> (c) optinfo
 `---> optinfo destinations
+  (c.1) optimization records
 
For optinfos, the dump_*_loc mark the beginning of an optinfo
instance: all subsequent dump_* calls are consolidated into
diff --git a/gcc/json.cc b/gcc/json.cc
new file mode 100644
index 000..3c2aa77
--- /dev/null
+++ b/gcc/json.cc
@@ -0,0 +1,293 @@
+/* JSON trees
+   Copyright (C) 2017-2018 Free Software Foundation, Inc.
+   Contributed by David Malcolm .
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PA

[PATCH 1/2] v5: Add "optinfo" framework

[David Malcolm]

6 +
 gcc/optinfo.h  | 203 ++
 7 files changed, 1200 insertions(+), 66 deletions(-)
 create mode 100644 gcc/dump-context.h
 create mode 100644 gcc/optinfo.cc
 create mode 100644 gcc/optinfo.h

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 2a05a66..dd1dfc1 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -1427,6 +1427,7 @@ OBJS = \
optabs-libfuncs.o \
optabs-query.o \
optabs-tree.o \
+   optinfo.o \
options-save.o \
opts-global.o \
passes.o \
diff --git a/gcc/coretypes.h b/gcc/coretypes.h
index 283b4eb..ed0e825 100644
--- a/gcc/coretypes.h
+++ b/gcc/coretypes.h
@@ -134,6 +134,13 @@ struct gomp_single;
 struct gomp_target;
 struct gomp_teams;
 
+/* Subclasses of symtab_node, using indentation to show the class
+   hierarchy.  */
+
+class symtab_node;
+  struct cgraph_node;
+  class varpool_node;
+
 union section;
 typedef union section section;
 struct gcc_options;
diff --git a/gcc/dump-context.h b/gcc/dump-context.h
new file mode 100644
index 000..a191e3a
--- /dev/null
+++ b/gcc/dump-context.h
@@ -0,0 +1,138 @@
+/* Support code for handling the various dump_* calls in dumpfile.h
+   Copyright (C) 2018 Free Software Foundation, Inc.
+   Contributed by David Malcolm .
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 3, or (at your option)
+any later version.
+
+GCC is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3.  If not see
+<http://www.gnu.org/licenses/>.  */
+
+
+#ifndef GCC_DUMP_CONTEXT_H
+#define GCC_DUMP_CONTEXT_H 1
+
+/* A class for handling the various dump_* calls.
+
+   In particular, this class has responsibility for consolidating
+   the "dump_*" calls into optinfo instances (delimited by "dump_*_loc"
+   calls), and emitting them.
+
+   Putting this in a class (rather than as global state) allows
+   for selftesting of this code.  */
+
+class dump_context
+{
+  friend class temp_dump_context;
+ public:
+  static dump_context &get () { return *s_current; }
+
+  ~dump_context ();
+
+  void dump_gimple_stmt (dump_flags_t dump_kind, dump_flags_t extra_dump_flags,
+gimple *gs, int spc);
+
+  void dump_gimple_stmt_loc (dump_flags_t dump_kind,
+const dump_location_t &loc,
+dump_flags_t extra_dump_flags,
+gimple *gs, int spc);
+
+  void dump_gimple_expr (dump_flags_t dump_kind,
+dump_flags_t extra_dump_flags,
+gimple *gs, int spc);
+
+  void dump_gimple_expr_loc (dump_flags_t dump_kind,
+   const dump_location_t &loc,
+   dump_flags_t extra_dump_flags,
+   gimple *gs,
+   int spc);
+
+  void dump_generic_expr (dump_flags_t dump_kind,
+ dump_flags_t extra_dump_flags,
+ tree t);
+
+  void dump_generic_expr_loc (dump_flags_t dump_kind,
+ const dump_location_t &loc,
+ dump_flags_t extra_dump_flags,
+ tree t);
+
+  void dump_printf_va (dump_flags_t dump_kind, const char *format,
+  va_list ap) ATTRIBUTE_PRINTF (3, 0);
+
+  void dump_printf_loc_va (dump_flags_t dump_kind, const dump_location_t &loc,
+  const char *format, va_list ap)
+ATTRIBUTE_PRINTF (4, 0);
+
+  template
+  void dump_dec (dump_flags_t dump_kind, const poly_int &value);
+
+  void dump_symtab_node (dump_flags_t dump_kind, symtab_node *node);
+
+  /* Managing nested scopes.  */
+  unsigned int get_scope_depth () const;
+  void begin_scope (const char *name, const dump_location_t &loc);
+  void end_scope ();
+
+  /* For use in selftests; if true then optinfo_enabled_p is true.  */
+  bool forcibly_enable_optinfo_p () const
+  {
+return m_forcibly_enable_optinfo;
+  }
+
+  void end_any_optinfo ();
+
+ private:
+  optinfo &ensure_pending_optinfo ();
+  optinfo &begin_next_optinfo (const dump_location_t &loc);
+
+  /* For use in selftests; if true then optinfo_enabled_p is true.  */
+  bool m_forcibly_enable_optinfo;
+
+  /* The current nesting depth of dump scopes, for showing nesting
+ via indentation).  */
+  unsigned int m_scope_depth;
+
+  /* The optinfo currently being accumulated since the last dump_*_loc call,
+ if any.  */
+  optinfo *m_pending;
+
+  /* The currently active dump_context, for use by the dump_* API cal

Re: [PATCH 0/5] [RFC v2] Higher-level reporting of vectorization problems

[David Malcolm]

On Wed, 2018-07-11 at 16:56 +0100, Richard Sandiford wrote:
> David Malcolm  writes:
> > On Mon, 2018-06-25 at 11:10 +0200, Richard Biener wrote:
> > > On Fri, 22 Jun 2018, David Malcolm wrote:
> > > 
> > > > NightStrike and I were chatting on IRC last week about
> > > > issues with trying to vectorize the following code:
> > > > 
> > > > #include 
> > > > std::size_t f(std::vector> const & v) {
> > > > std::size_t ret = 0;
> > > > for (auto const & w: v)
> > > > ret += w.size();
> > > > return ret;
> > > > }
> > > > 
> > > > icc could vectorize it, but gcc couldn't, but neither of us
> > > > could
> > > > immediately figure out what the problem was.
> > > > 
> > > > Using -fopt-info leads to a wall of text.
> > > > 
> > > > I tried using my patch here:
> > > > 
> > > >  "[PATCH] v3 of optinfo, remarks and optimization records"
> > > >   https://gcc.gnu.org/ml/gcc-patches/2018-06/msg01267.html
> > > > 
> > > > It improved things somewhat, by showing:
> > > > (a) the nesting structure via indentation, and
> > > > (b) the GCC line at which each message is emitted (by using the
> > > > "remark" output)
> > > > 
> > > > but it's still a wall of text:
> > > > 
> > > >   https://dmalcolm.fedorapeople.org/gcc/2018-06-18/test.cc.rema
> > > > rks.
> > > > html
> > > >   https://dmalcolm.fedorapeople.org/gcc/2018-06-18/test.cc.d/..
> > > > %7C.
> > > > .%7Csrc%7Ctest.cc.html#line-4
> > > > 
> > > > It doesn't yet provide a simple high-level message to a
> > > > tech-savvy user on what they need to do to get GCC to
> > > > vectorize their loop.
> > > 
> > > Yeah, in particular the vectorizer is way too noisy in its low-
> > > level
> > > functions.  IIRC -fopt-info-vec-missed is "somewhat" better:
> > > 
> > > t.C:4:26: note: step unknown.
> > > t.C:4:26: note: vector alignment may not be reachable
> > > t.C:4:26: note: not ssa-name.
> > > t.C:4:26: note: use not simple.
> > > t.C:4:26: note: not ssa-name.
> > > t.C:4:26: note: use not simple.
> > > t.C:4:26: note: no array mode for V2DI[3]
> > > t.C:4:26: note: Data access with gaps requires scalar epilogue
> > > loop
> > > t.C:4:26: note: can't use a fully-masked loop because the target
> > > doesn't 
> > > have the appropriate masked load or store.
> > > t.C:4:26: note: not ssa-name.
> > > t.C:4:26: note: use not simple.
> > > t.C:4:26: note: not ssa-name.
> > > t.C:4:26: note: use not simple.
> > > t.C:4:26: note: no array mode for V2DI[3]
> > > t.C:4:26: note: Data access with gaps requires scalar epilogue
> > > loop
> > > t.C:4:26: note: op not supported by target.
> > > t.C:4:26: note: not vectorized: relevant stmt not supported: _15
> > > =
> > > _14 
> > > /[ex] 4;
> > > t.C:4:26: note: bad operation or unsupported loop bound.
> > > t.C:4:26: note: not vectorized: no grouped stores in basic block.
> > > t.C:4:26: note: not vectorized: no grouped stores in basic block.
> > > t.C:6:12: note: not vectorized: not enough data-refs in basic
> > > block.
> > > 
> > > 
> > > > The pertinent dump messages are:
> > > > 
> > > > test.cc:4:23: remark: === try_vectorize_loop_1 ===
> > > > [../../src/gcc/tree-vectorizer.c:674:try_vectorize_loop_1]
> > > > cc1plus: remark:
> > > > Analyzing loop at test.cc:4
> > > > [../../src/gcc/dumpfile.c:735:ensure_pending_optinfo]
> > > > test.cc:4:23: remark:  === analyze_loop_nest ===
> > > > [../../src/gcc/tree-vect-loop.c:2299:vect_analyze_loop]
> > > > [...snip...]
> > > > test.cc:4:23: remark:   === vect_analyze_loop_operations ===
> > > > [../../src/gcc/tree-vect-
> > > > loop.c:1520:vect_analyze_loop_operations]
> > > > [...snip...]
> > > > test.cc:4:23: remark:==> examining statement: ‘_15 = _14
> > > > /[ex]
> > > > 4;’ [../../src/gcc/tree-vect-stmts.c:9382:vect_analyze_stmt]
> > > > test.cc:4:23: remark:vect_is_simple_use: operand ‘_14’
> > > > [../../src/gcc/tree-vect-stmts.c

[PATCH] C++: suggestions for misspelled private members (PR c++/84993)

[David Malcolm]

PR c++/84993 identifies a problem with our suggestions for
misspelled member names in the C++ FE for the case where the
member is private.

For example, given:

class foo
{
public:
  double get_ratio() const { return m_ratio; }

private:
  double m_ratio;
};

void test(foo *ptr)
{
  if (ptr->ratio >= 0.5)
;// etc
}

...we currently emit this suggestion:

: In function 'void test(foo*)':
:12:12: error: 'class foo' has no member named 'ratio'; did you mean 
'm_ratio'?
   if (ptr->ratio >= 0.5)
^
m_ratio

...but if the user follows this suggestion, they get:

: In function 'void test(foo*)':
:12:12: error: 'double foo::m_ratio' is private within this context
   if (ptr->m_ratio >= 0.5)
^~~
:7:10: note: declared private here
   double m_ratio;
  ^~~
:12:12: note: field 'double foo::m_ratio' can be accessed via 'double 
foo::get_ratio() const'
   if (ptr->m_ratio >= 0.5)
^~~
get_ratio()

It feels wrong to be emitting a fix-it hint that doesn't compile, so this
patch adds the accessor fix-it hint logic to this case, so that we directly
offer a valid suggestion:

: In function 'void test(foo*)':
:12:12: error: 'class foo' has no member named 'ratio'; did you mean
'double foo::m_ratio'? (accessible via 'double foo::get_ratio() const')
   if (ptr->ratio >= 0.5)
^
get_ratio()

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu;
adds 105 PASS results to g++.sum.

OK for trunk?

gcc/cp/ChangeLog:
PR c++/84993
* call.c (enforce_access): Move diagnostics to...
(complain_about_access): ...this new function.
* cp-tree.h (class access_failure_info): Rename split out field
"m_field_decl" into "m_decl" and "m_diag_decl".
(access_failure_info::record_access_failure): Add tree param.
(access_failure_info::was_inaccessible_p): New accessor.
(access_failure_info::get_decl): New accessor.
(access_failure_info::get_diag_decl): New accessor.
(access_failure_info::get_any_accessor): New member function.
(access_failure_info::add_fixit_hint): New static member function.
(complain_about_access): New decl.
* typeck.c (access_failure_info::record_access_failure): Update
for change to fields.
(access_failure_info::maybe_suggest_accessor): Split out into...
(access_failure_info::get_any_accessor): ...this new function...
(access_failure_info::add_fixit_hint): ...and this new function.
(finish_class_member_access_expr): Split out "has no member named"
error-handling into...
(complain_about_unrecognized_member): ...this new function, and
check that the guessed name is accessible along the access path.
Only provide a spell-correction fix-it hint if it is; otherwise,
attempt to issue an accessor fix-it hint.

gcc/testsuite/ChangeLog:
PR c++/84993
* g++.dg/torture/accessor-fixits-9.C: New test.
---
 gcc/cp/call.c|  64 ++
 gcc/cp/cp-tree.h |  17 ++-
 gcc/cp/typeck.c  | 150 +--
 gcc/testsuite/g++.dg/torture/accessor-fixits-9.C | 119 ++
 4 files changed, 282 insertions(+), 68 deletions(-)
 create mode 100644 gcc/testsuite/g++.dg/torture/accessor-fixits-9.C

diff --git a/gcc/cp/call.c b/gcc/cp/call.c
index 209c1fd..121f0ca 100644
--- a/gcc/cp/call.c
+++ b/gcc/cp/call.c
@@ -6456,6 +6456,38 @@ build_op_delete_call (enum tree_code code, tree addr, 
tree size,
   return error_mark_node;
 }
 
+/* Issue diagnostics about a disallowed access of DECL, using DIAG_DECL
+   in the diagnostics.
+
+   If ISSUE_ERROR is true, then issue an error about the
+   access, followed by a note showing the declaration.
+   Otherwise, just show the note.  */
+
+void
+complain_about_access (tree decl, tree diag_decl, bool issue_error)
+{
+  if (TREE_PRIVATE (decl))
+{
+  if (issue_error)
+   error ("%q#D is private within this context", diag_decl);
+  inform (DECL_SOURCE_LOCATION (diag_decl),
+ "declared private here");
+}
+  else if (TREE_PROTECTED (decl))
+{
+  if (issue_error)
+   error ("%q#D is protected within this context", diag_decl);
+  inform (DECL_SOURCE_LOCATION (diag_decl),
+ "declared protected here");
+}
+  else
+{
+  if (issue_error)
+   error ("%q#D is inaccessible within this context", diag_decl);
+  inform (DECL_SOURCE_LOCATION (diag_decl), "declared here");
+}
+}
+
 /* If the current scope isn't allowed to access DECL along
BASETYPE_PATH, give an error.  The most derived class in
BASETYPE_PATH is the one used to qualify DECL. DIAG_DECL is
@@ -6480,34 +6512,12 @@ enforce_access (tree basetype_path, tree decl, tree 
diag_decl,
 
   if (!accessible_p (basetype_path, decl, true))
 {
+  if (flag_n

Re: [PATCH] Fix _Pragma GCC diagnostic in macro expansions

[David Malcolm]

On Wed, 2018-07-04 at 10:53 +, Bernd Edlinger wrote:

Sorry for the delay in reviewing this.

> Hi,
> 
> currently _Pragma("GCC diagnostic ...") does not properly
> work in macro expansions.
> 
> Consider the following code:
> 
> #define B _Pragma("GCC diagnostic push") \
> _Pragma("GCC diagnostic ignored \"-Wattributes\"")
> #define E _Pragma("GCC diagnostic pop")
> 
> #define X() B int __attribute((unknown_attr)) x; E /* { dg-bogus
> "attribute directive ignored" } */
> 
> void test1(void)
> {
> X()  /* { dg-bogus "in expansion of macro" } */
> }
> 
> 
> Warnings happen in C++ despite the _Pragma, while C happens to
> suppress the warnings
> more or less by accident.
> 
> This is connected to the fact that GCC uses the location of the
> closing parenthesis of the
> function-like macro expansion in the _Pragma, while the rest of the
> locations are relative
> to the macro expansion point, which is the letter X in this case.
> 
> This patch changes the location of builtin macros and _Pragma to use
> the macro expansion
> point instead of the closing parenthesis.

While reviewing the various PRs for the testcases this touches, I
noticed:
  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61817#c3
which has a link to:
  http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1911.htm
which has:

  Suggested Technical Corrigendum

  Add to 6.10.8.1, paragraph 1, item __LINE__:

  The line number of a pp token is implementation defined to
  be the (physical) line number of either the first character
  or the last character of the pp token. The line number of a
  __LINE__ that spans multiple physical lines is implementation
  defined to be either the first line or the last line of that
  __LINE__. The line number of a __LINE__ in a macro body is the
  line number of the macro invocation. The line number of a macro
  invocation that spans multiple (physical or logical) lines is
  implementation defined to be either the line number of the
  first character of the macro name, the last character of the
  macro name or the closing ')' (if there is one).

I don't know the status of that suggested corrigendum, but if I'm
reading it right, changing the location from that of the closing ')' to
that of the macro name would keep us in compliance with that final
sentence (after extracting the location's line number).

> A few test cases had to be adjusted, most changes were necessary
> because the __LINE__
> location moved to the macro expansion point, which looks like a
> straight improvement.
> 
> In pr61817-2.c the location of __LINE__ depends on -ftrack-macro-
> expansion,
> when enabled the location of the macro argument is the spelling
> location, while all other
> locations change to the macro expansion point.
> 
> The C++ pagma plugin.c is also affected by the change, because the
> input_location is now
> the spelling location of _Pragma in DO_PRAGMA and has to be converted
> to the expansion
> point of the macro to get the expected result.
> 
> 
> Bootstrapped and reg-tested on x86_64-pc-linux-gnu.
> Is it OK for trunk?

Some nits:

> libcpp:
> 2018-07-04  Bernd Edlinger  
> 
> * macro.c (enter_macro_context): Change the location info for builtin
> macros from location of the closing parenthesis to location of the 
> macro
> expansion point.

Please update to say "builtin macros and _Pragma" here, rather than just
"builtin macros" here.

> testsuite:
> 2018-07-04  Bernd Edlinger  
> 
> * c-c++-common/cpp/diagnostic-pragma-2.c: New test.
> * c-c++-common/pr69558.c: Remove xfail.
> * gcc.dg/cpp/builtin-macro-1.c: Adjust test expectations.
> * gcc.dg/pr61817-1.c: Likewise.
> * gcc.dg/pr61817-2.c: Likewise.
> * g++.dg/plugin/pragma_plugin.c: Warn at expansion_point_location.

Please reference PR 69558 in both ChangeLog entries (given that this
fixes an XFAIL).

OK for trunk with the above changes; thanks.

It looks like with this change we can remove Jakub's r233058 hack.  I
briefly tested with it, and it seems to work.  But that can wait for a
followup.

Dave

[PING] Re: [PATCH 1/2] v5: Add "optinfo" framework

[David Malcolm]

Ping, re these patches:

"[PATCH 1/2] v5: Add "optinfo" framework"
  https://gcc.gnu.org/ml/gcc-patches/2018-07/msg00535.html
 
"[PATCH 2/2] Add "-fsave-optimization-record""
  https://gcc.gnu.org/ml/gcc-patches/2018-07/msg00536.html

Thanks
Dave

On Wed, 2018-07-11 at 07:37 -0400, David Malcolm wrote:
> Changes relative to v4:
> * eliminated optinfo subclasses as discussed
> * eliminated optinfo-internal.h, moving what remained into optinfo.h
> * added support for dump_gimple_expr_loc and dump_gimple_expr
> * more selftests
> 
> This patch implements a way to consolidate dump_* calls into
> optinfo objects, as enabling work towards being able to write out
> optimization records to a file (I'm focussing on that destination
> in this patch kit, rather than diagnostic remarks).
> 
> The patch adds the support for building optinfo instances from dump_*
> calls, but leaves implementing any *users* of them to followup
> patches.
> 
> Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
> 
> OK for trunk?
> 
> gcc/ChangeLog:
>   * Makefile.in (OBJS): Add optinfo.o.
>   * coretypes.h (class symtab_node): New forward decl.
>   (struct cgraph_node): New forward decl.
>   (class varpool_node): New forward decl.
>   * dump-context.h: New file.
>   * dumpfile.c: Include "optinfo.h", "dump-context.h",
> "cgraph.h",
>   "tree-pass.h".
>   (refresh_dumps_are_enabled): Use optinfo_enabled_p.
>   (set_dump_file): Call dumpfile_ensure_any_optinfo_are_flushed.
>   (set_alt_dump_file): Likewise.
>   (dump_context::~dump_context): New dtor.
>   (dump_gimple_stmt): Move implementation to...
>   (dump_context::dump_gimple_stmt): ...this new member function.
>   Add the stmt to any pending optinfo, creating one if need be.
>   (dump_gimple_stmt_loc): Move implementation to...
>   (dump_context::dump_gimple_stmt_loc): ...this new member
> function.
>   Start a new optinfo and add the stmt to it.
>   (dump_gimple_expr): Move implementation to...
>   (dump_context::dump_gimple_expr): ...this new member function.
>   Add the stmt to any pending optinfo, creating one if need be.
>   (dump_gimple_expr_loc): Move implementation to...
>   (dump_context::dump_gimple_expr_loc): ...this new member
> function.
>   Start a new optinfo and add the stmt to it.
>   (dump_generic_expr): Move implementation to...
>   (dump_context::dump_generic_expr): ...this new member function.
>   Add the tree to any pending optinfo, creating one if need be.
>   (dump_generic_expr_loc): Move implementation to...
>   (dump_context::dump_generic_expr_loc): ...this new member
>   function.  Add the tree to any pending optinfo, creating one if
>   need be.
>   (dump_printf): Move implementation to...
>   (dump_context::dump_printf_va): ...this new member
> function.  Add
>   the text to any pending optinfo, creating one if need be.
>   (dump_printf_loc): Move implementation to...
>   (dump_context::dump_printf_loc_va): ...this new member
> function.
>   Start a new optinfo and add the stmt to it.
>   (dump_dec): Move implementation to...
>   (dump_context::dump_dec): ...this new member function.  Add the
>   value to any pending optinfo, creating one if need be.
>   (dump_context::dump_symtab_node): New member function.
>   (dump_context::get_scope_depth): New member function.
>   (dump_context::begin_scope): New member function.
>   (dump_context::end_scope): New member function.
>   (dump_context::ensure_pending_optinfo): New member function.
>   (dump_context::begin_next_optinfo): New member function.
>   (dump_context::end_any_optinfo): New member function.
>   (dump_context::s_current): New global.
>   (dump_context::s_default): New global.
>   (dump_scope_depth): Delete global.
>   (dumpfile_ensure_any_optinfo_are_flushed): New function.
>   (dump_symtab_node): New function.
>   (get_dump_scope_depth): Reimplement in terms of dump_context.
>   (dump_begin_scope): Likewise.
>   (dump_end_scope): Likewise.
>   (selftest::temp_dump_context::temp_dump_context): New ctor.
>   (selftest::temp_dump_context::~temp_dump_context): New dtor.
>   (selftest::verify_item): New function.
>   (ASSERT_IS_TEXT): New macro.
>   (ASSERT_IS_TREE): New macro.
>   (ASSERT_IS_GIMPLE): New macro.
>   (selftest::test_capture_of_dump_calls): New test.
>   (selftest::dumpfile_c_tests): Call it.
>   * dumpfile.h (dump_printf, dump_printf_loc, dump_basic_block)
>  

Re: [PATCH 2/2] Add "-fsave-optimization-record"

[David Malcolm]

On Thu, 2018-07-19 at 14:39 +0200, Richard Biener wrote:
> On Wed, Jul 11, 2018 at 12:53 PM David Malcolm 
> wrote:
> > 
> > This patch implements a -fsave-optimization-record option, which
> > leads to a JSON file being written out, recording the dump_* calls
> > made (via the optinfo infrastructure in the previous patch).
> > 
> > The patch includes a minimal version of the JSON patch I posted
> > last
> > year, with just enough support needed for optimization records (I
> > removed all of the parser code, leaving just the code for building
> > in-memory JSON trees and writing them to a pretty_printer).
> > 
> > Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
> > 
> > OK for trunk?
> 
> +@item -fsave-optimization-record
> +@opindex fsave-optimization-record
> +Write a SRCFILE.opt-record.json file detailing what optimizations
> +were performed.
> +
> 
> I guess this should note that it is experimental and in no way
> complete.  Maybe list areas where reports will be generated,
> like vectorization?

Indeed. I also noticed that for some reason my patch had this option
between
  @item -frounding-math
and
  @item -fsignaling-nans
in the section on floating-point arithmetic options, so I've moved it
to the "GCC Developer Options" section for now, immediately after the
documentation of -fopt-info.

Here's what I've committed to invoke.texi (r262905):

BEGIN QUOTE:

@item -fsave-optimization-record
@opindex fsave-optimization-record
Write a SRCFILE.opt-record.json file detailing what optimizations
were performed, for those optimizations that support @option{-fopt-info}.

This option is experimental and the format of the data within the JSON
file is subject to change.

It is roughly equivalent to a machine-readable version of
@option{-fopt-info-all}, as a collection of messages with source file,
line number and column number, with the following additional data for
each message:

@itemize @bullet

@item
the execution count of the code being optimized, along with metadata about
whether this was from actual profile data, or just an estimate, allowing
consumers to prioritize messages by code hotness,

@item
the function name of the code being optimized, where applicable,

@item
the ``inlining chain'' for the code being optimized, so that when
a function is inlined into several different places (which might
themselves be inlined), the reader can distinguish between the copies,

@item
objects identifying those parts of the message that refer to expressions,
statements or symbol-table nodes, which of these categories they are, and,
when available, their source code location,

@item
the GCC pass that emitted the message, and

@item
the location in GCC's own code from which the message was emitted

@end itemize

Additionally, some messages are logically nested within other
messages, reflecting implementation details of the optimization
passes.

END QUOTE


> Did you check what happens with -flto -fsave-optimization-record?
> Will the compile-phase emit a json file for each source (expected,
> like with early inlining decisions)?  Will the WPA phase emit one
> (IPA decisions?) or will IPA decisions be recorded in the LTRANS
> one?  How will the LTRANS ones be named and where can they
> be found?  You don't need to solve all the issues with this patch
> but they should be eventually addressed somehow.

I'm looking at that now.

> I don't question the use or implementation of JSON, I'll just
> approve it.
> 
> The rest looks obvious enough, thus OK.

> 
> Some overall blurb in the documentation or changes.html
> on how to use this would be nice of course.

Thanks; will do.

Dave

> Thanks,
> Richard.
> 
> 
> > gcc/ChangeLog:
> > * Makefile.in (OBJS): Add json.o and optinfo-emit-json.o.
> > (CFLAGS-optinfo-emit-json.o): Define TARGET_NAME.
> > * common.opt (fsave-optimization-record): New option.
> > * coretypes.h (struct kv_pair): Move here from dumpfile.c.
> > * doc/invoke.texi (-fsave-optimization-record): New option.
> > * dumpfile.c: Include "optinfo-emit-json.h".
> > (struct kv_pair): Move to coretypes.h.
> > (optgroup_options): Make non-static.
> > (dump_context::end_scope): Call
> > optimization_records_maybe_pop_dump_scope.
> > * dumpfile.h (optgroup_options): New decl.
> > * json.cc: New file.
> > * json.h: New file.
> > * optinfo-emit-json.cc: New file.
> > * optinfo-emit-json.h: New file.
> > * optinfo.cc: Include "optinfo-emit-json.h".
> > (optinfo::emit): Call
> > optimization_records_mayb

[PATCH] -fsave-optimization-record: add contrib/optrecord.py

[David Malcolm]

This patch adds a Python 3 module to "contrib" for reading the output of
-fsave-optimization-record.

It can be imported from other Python code, or run standalone as a script,
in which case it prints the saved messages in a form resembling GCC
diagnostics.

OK for trunk?

contrib/ChangeLog:
* optrecord.py: New file.
---
 contrib/optrecord.py | 295 +++
 1 file changed, 295 insertions(+)
 create mode 100755 contrib/optrecord.py

diff --git a/contrib/optrecord.py b/contrib/optrecord.py
new file mode 100755
index 000..b07488e
--- /dev/null
+++ b/contrib/optrecord.py
@@ -0,0 +1,295 @@
+#!/usr/bin/env python3
+#
+# Python module for working with the result of -fsave-optimization-record
+# Contributed by David Malcolm .
+#
+# Copyright (C) 2018 Free Software Foundation, Inc.
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License as published by the Free
+# Software Foundation; either version 3, or (at your option) any later
+# version.
+#
+# GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+# for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3.  If not see
+# <http://www.gnu.org/licenses/>.  */
+
+import argparse
+import json
+import os
+import sys
+
+class TranslationUnit:
+"""Top-level class for containing optimization records"""
+@staticmethod
+def from_filename(filename):
+with open(filename) as f:
+root_obj = json.load(f)
+return TranslationUnit(filename, root_obj)
+
+def __init__(self, filename, json_obj):
+self.filename = filename
+self.pass_by_id = {}
+
+# Expect a 3-tuple
+metadata, passes, records = json_obj
+
+self.format = metadata['format']
+self.generator = metadata['generator']
+self.passes = [Pass(obj, self) for obj in passes]
+self.records = [Record(obj, self) for obj in records]
+
+def __repr__(self):
+return ('TranslationUnit(%r, %r, %r, %r)'
+% (self.filename, self.generator, self.passes, self.records))
+
+class Pass:
+"""An optimization pass"""
+def __init__(self, json_obj, tu):
+self.id_ = json_obj['id']
+self.name = json_obj['name']
+self.num = json_obj['num']
+self.optgroups = set(json_obj['optgroups']) # list of strings
+self.type = json_obj['type']
+tu.pass_by_id[self.id_] = self
+self.children = [Pass(child, tu)
+ for child in json_obj.get('children', [])]
+
+def __repr__(self):
+return ('Pass(%r, %r, %r, %r)'
+% (self.name, self.num, self.optgroups, self.type))
+
+def from_optional_json_field(cls, jsonobj, field):
+if field not in jsonobj:
+return None
+return cls(jsonobj[field])
+
+class ImplLocation:
+"""An implementation location (within the compiler itself)"""
+def __init__(self, json_obj):
+self.file = json_obj['file']
+self.line = json_obj['line']
+self.function = json_obj['function']
+
+def __repr__(self):
+return ('ImplLocation(%r, %r, %r)'
+% (self.file, self.line, self.function))
+
+class Location:
+"""A source location"""
+def __init__(self, json_obj):
+self.file = json_obj['file']
+self.line = json_obj['line']
+self.column = json_obj['column']
+
+def __str__(self):
+return '%s:%i:%i' % (self.file, self.line, self.column)
+
+def __repr__(self):
+return ('Location(%r, %r, %r)'
+% (self.file, self.line, self.column))
+
+class Count:
+"""An execution count"""
+def __init__(self, json_obj):
+self.quality = json_obj['quality']
+self.value = json_obj['value']
+
+def __repr__(self):
+return ('Count(%r, %r)'
+% (self.quality, self.value))
+
+def is_precise(self):
+return self.quality in ('precise', 'adjusted')
+
+class Record:
+"""A optimization record: success/failure/note"""
+def __init__(self, json_obj, tu):
+self.kind = json_obj['kind']
+if 'pass' in json_obj:
+self.pass_ = tu.pass_by_id[json_obj['pass']]
+else:
+ 

[committed] libcpp: remove redundant parameter from rich_location::set_range

[David Malcolm]

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.

Committed to trunk as r262913.

gcc/c-family/ChangeLog:
* c-common.c (c_cpp_error): Remove redundant "line_table"
parameter from call to rich_location::set_range.
(maybe_suggest_missing_token_insertion): Likewise.

gcc/ChangeLog:
* pretty-print.c (text_info::set_location): Remove redundant
"line_table" parameter from call to rich_location::set_range.

libcpp/ChangeLog:
* include/line-map.h (rich_location::set_range): Remove redundant
line_maps * parameter.
* line-map.c (rich_location::set_range): Likewise.
---
 gcc/c-family/c-common.c   | 4 ++--
 gcc/pretty-print.c| 2 +-
 libcpp/include/line-map.h | 3 +--
 libcpp/line-map.c | 4 ++--
 4 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/gcc/c-family/c-common.c b/gcc/c-family/c-common.c
index f5e..422d668 100644
--- a/gcc/c-family/c-common.c
+++ b/gcc/c-family/c-common.c
@@ -6133,7 +6133,7 @@ c_cpp_error (cpp_reader *pfile ATTRIBUTE_UNUSED, int 
level, int reason,
   gcc_unreachable ();
 }
   if (done_lexing)
-richloc->set_range (line_table, 0, input_location, true);
+richloc->set_range (0, input_location, true);
   diagnostic_set_info_translated (&diagnostic, msg, ap,
  richloc, dlevel);
   diagnostic_override_option_index (&diagnostic,
@@ -8174,7 +8174,7 @@ maybe_suggest_missing_token_insertion (rich_location 
*richloc,
   location_t hint_loc = hint->get_start_loc ();
   location_t old_loc = richloc->get_loc ();
 
-  richloc->set_range (line_table, 0, hint_loc, true);
+  richloc->set_range (0, hint_loc, true);
   richloc->add_range (old_loc, false);
 }
 }
diff --git a/gcc/pretty-print.c b/gcc/pretty-print.c
index df3ee81..dc7791a 100644
--- a/gcc/pretty-print.c
+++ b/gcc/pretty-print.c
@@ -699,7 +699,7 @@ void
 text_info::set_location (unsigned int idx, location_t loc, bool show_caret_p)
 {
   gcc_checking_assert (m_richloc);
-  m_richloc->set_range (line_table, idx, loc, show_caret_p);
+  m_richloc->set_range (idx, loc, show_caret_p);
 }
 
 location_t
diff --git a/libcpp/include/line-map.h b/libcpp/include/line-map.h
index ba1750d..a4baa49 100644
--- a/libcpp/include/line-map.h
+++ b/libcpp/include/line-map.h
@@ -1639,8 +1639,7 @@ class rich_location
   add_range (source_location loc,  bool show_caret_p);
 
   void
-  set_range (line_maps *set, unsigned int idx, source_location loc,
-bool show_caret_p);
+  set_range (unsigned int idx, source_location loc, bool show_caret_p);
 
   unsigned int get_num_locations () const { return m_ranges.count (); }
 
diff --git a/libcpp/line-map.c b/libcpp/line-map.c
index 1051022..a1a765f 100644
--- a/libcpp/line-map.c
+++ b/libcpp/line-map.c
@@ -2104,8 +2104,8 @@ rich_location::add_range (source_location loc, bool 
show_caret_p)
- the "%C" and "%L" format codes in the Fortran frontend.  */
 
 void
-rich_location::set_range (line_maps * /*set*/, unsigned int idx,
- source_location loc, bool show_caret_p)
+rich_location::set_range (unsigned int idx, source_location loc,
+ bool show_caret_p)
 {
   /* We can either overwrite an existing range, or add one exactly
  on the end of the array.  */
-- 
1.8.5.3

Re: [PATCH] -fsave-optimization-record: add contrib/optrecord.py

[David Malcolm]

On Mon, 2018-07-23 at 11:46 +0200, Richard Biener wrote:
> On Fri, Jul 20, 2018 at 6:27 PM David Malcolm 
> wrote:
> > 
> > This patch adds a Python 3 module to "contrib" for reading the
> > output of
> > -fsave-optimization-record.
> > 
> > It can be imported from other Python code, or run standalone as a
> > script,
> > in which case it prints the saved messages in a form resembling GCC
> > diagnostics.
> > 
> > OK for trunk?
> 
> OK, but shouldn't there maybe a user-visible (and thus installed)
> tool for
> this kind of stuff?  Which would mean to place it somewhere else.

As well as this support code, I've got code that uses it to generate
HTML reports.  I'm thinking that all this Python code might be better
to maintain in an entirely separate repository, as a third-party
project (maintained by me, under some suitable Free Software license,
accessible via PyPI), since I suspect that the release cycle ought to
be different from that of gcc itself.

Would that be a better approach?

Dave

> Richard.
> 
> > contrib/ChangeLog:
> > * optrecord.py: New file.
> > ---
> >  contrib/optrecord.py | 295
> > +++
> >  1 file changed, 295 insertions(+)
> >  create mode 100755 contrib/optrecord.py
> > 
> > diff --git a/contrib/optrecord.py b/contrib/optrecord.py
> > new file mode 100755
> > index 000..b07488e
> > --- /dev/null
> > +++ b/contrib/optrecord.py
> > @@ -0,0 +1,295 @@
> > +#!/usr/bin/env python3
> > +#
> > +# Python module for working with the result of -fsave-
> > optimization-record
> > +# Contributed by David Malcolm .
> > +#
> > +# Copyright (C) 2018 Free Software Foundation, Inc.
> > +# This file is part of GCC.
> > +#
> > +# GCC is free software; you can redistribute it and/or modify it
> > under
> > +# the terms of the GNU General Public License as published by the
> > Free
> > +# Software Foundation; either version 3, or (at your option) any
> > later
> > +# version.
> > +#
> > +# GCC is distributed in the hope that it will be useful, but
> > WITHOUT ANY
> > +# WARRANTY; without even the implied warranty of MERCHANTABILITY
> > or
> > +# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public
> > License
> > +# for more details.
> > +#
> > +# You should have received a copy of the GNU General Public
> > License
> > +# along with GCC; see the file COPYING3.  If not see
> > +# <http://www.gnu.org/licenses/>.  */
> > +
> > +import argparse
> > +import json
> > +import os
> > +import sys
> > +
> > +class TranslationUnit:
> > +"""Top-level class for containing optimization records"""
> > +@staticmethod
> > +def from_filename(filename):
> > +with open(filename) as f:
> > +root_obj = json.load(f)
> > +return TranslationUnit(filename, root_obj)
> > +
> > +def __init__(self, filename, json_obj):
> > +self.filename = filename
> > +self.pass_by_id = {}
> > +
> > +# Expect a 3-tuple
> > +metadata, passes, records = json_obj
> > +
> > +self.format = metadata['format']
> > +self.generator = metadata['generator']
> > +self.passes = [Pass(obj, self) for obj in passes]
> > +self.records = [Record(obj, self) for obj in records]
> > +
> > +def __repr__(self):
> > +return ('TranslationUnit(%r, %r, %r, %r)'
> > +% (self.filename, self.generator, self.passes,
> > self.records))
> > +
> > +class Pass:
> > +"""An optimization pass"""
> > +def __init__(self, json_obj, tu):
> > +self.id_ = json_obj['id']
> > +self.name = json_obj['name']
> > +self.num = json_obj['num']
> > +self.optgroups = set(json_obj['optgroups']) # list of
> > strings
> > +self.type = json_obj['type']
> > +tu.pass_by_id[self.id_] = self
> > +self.children = [Pass(child, tu)
> > + for child in json_obj.get('children',
> > [])]
> > +
> > +def __repr__(self):
> > +return ('Pass(%r, %r, %r, %r)'
> > +% (self.name, self.num, self.optgroups,
> > self.type))
> > +
> > +def from_optional_json_field(cls, json

[PATCH] Fix segfault in -fsave-optimization-record (PR tree-optimization/86636)

[David Malcolm]

There are various ways that it's possible for a gimple statement to
have an UNKNOWN_LOCATION, and for that UNKNOWN_LOCATION to be wrapped
in an ad-hoc location to capture inlining information.

For such a location, LOCATION_FILE (loc) is NULL.

Various places in -fsave-optimization-record were checking for
  loc != UNKNOWN_LOCATION
and were passing LOCATION_FILE (loc) to code that assumed a non-NULL
filename, thus leading to segfaults for the above cases.

This patch updates the tests to use
  LOCATION_LOCUS (loc) != UNKNOWN_LOCATION
instead, to look through ad-hoc location wrappers, fixing the segfaults.

It also adds various assertions to the affected code.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu; adds
8 PASS results to gcc.sum.

OK for trunk?

gcc/ChangeLog:
PR tree-optimization/86636
* json.cc (json::object::set): Fix comment.  Add assertions.
(json::array::append): Move here from json.h.  Add comment and an
assertion.
(json::string::string): Likewise.
* json.h (json::array::append): Move to json.cc.
(json::string::string): Likewise.
* optinfo-emit-json.cc
(optrecord_json_writer::impl_location_to_json): Assert that we
aren't attempting to write out UNKNOWN_LOCATION, or an ad-hoc
wrapper around it.  Expand the location once, rather than three
times.
(optrecord_json_writer::inlining_chain_to_json): Fix the check for
UNKNOWN_LOCATION, to use LOCATION_LOCUS to look through ad-hoc
wrappers.
(optrecord_json_writer::optinfo_to_json): Likewise, in four
places.  Fix some overlong lines.

gcc/testsuite/ChangeLog:
PR tree-optimization/86636
* gcc.c-torture/compile/pr86636.c: New test.
---
 gcc/json.cc   | 24 +++-
 gcc/json.h|  4 ++--
 gcc/optinfo-emit-json.cc  | 25 +++--
 gcc/testsuite/gcc.c-torture/compile/pr86636.c |  8 
 4 files changed, 48 insertions(+), 13 deletions(-)
 create mode 100644 gcc/testsuite/gcc.c-torture/compile/pr86636.c

diff --git a/gcc/json.cc b/gcc/json.cc
index 3c2aa77..3ead980 100644
--- a/gcc/json.cc
+++ b/gcc/json.cc
@@ -76,12 +76,15 @@ object::print (pretty_printer *pp) const
   pp_character (pp, '}');
 }
 
-/* Set the json::value * for KEY, taking ownership of VALUE
+/* Set the json::value * for KEY, taking ownership of V
(and taking a copy of KEY if necessary).  */
 
 void
 object::set (const char *key, value *v)
 {
+  gcc_assert (key);
+  gcc_assert (v);
+
   value **ptr = m_map.get (key);
   if (ptr)
 {
@@ -126,6 +129,15 @@ array::print (pretty_printer *pp) const
   pp_character (pp, ']');
 }
 
+/* Append non-NULL value V to a json::array, taking ownership of V.  */
+
+void
+array::append (value *v)
+{
+  gcc_assert (v);
+  m_elements.safe_push (v);
+}
+
 /* class json::number, a subclass of json::value, wrapping a double.  */
 
 /* Implementation of json::value::print for json::number.  */
@@ -140,6 +152,16 @@ number::print (pretty_printer *pp) const
 
 /* class json::string, a subclass of json::value.  */
 
+/* json::string's ctor.  */
+
+string::string (const char *utf8)
+{
+  gcc_assert (utf8);
+  m_utf8 = xstrdup (utf8);
+}
+
+/* Implementation of json::value::print for json::string.  */
+
 void
 string::print (pretty_printer *pp) const
 {
diff --git a/gcc/json.h b/gcc/json.h
index 5c3274c..154d9e1 100644
--- a/gcc/json.h
+++ b/gcc/json.h
@@ -107,7 +107,7 @@ class array : public value
   enum kind get_kind () const FINAL OVERRIDE { return JSON_ARRAY; }
   void print (pretty_printer *pp) const FINAL OVERRIDE;
 
-  void append (value *v) { m_elements.safe_push (v); }
+  void append (value *v);
 
  private:
   auto_vec m_elements;
@@ -134,7 +134,7 @@ class number : public value
 class string : public value
 {
  public:
-  string (const char *utf8) : m_utf8 (xstrdup (utf8)) {}
+  string (const char *utf8);
   ~string () { free (m_utf8); }
 
   enum kind get_kind () const FINAL OVERRIDE { return JSON_STRING; }
diff --git a/gcc/optinfo-emit-json.cc b/gcc/optinfo-emit-json.cc
index bf1172a..6460a81 100644
--- a/gcc/optinfo-emit-json.cc
+++ b/gcc/optinfo-emit-json.cc
@@ -202,10 +202,12 @@ optrecord_json_writer::impl_location_to_json 
(dump_impl_location_t loc)
 json::object *
 optrecord_json_writer::location_to_json (location_t loc)
 {
+  gcc_assert (LOCATION_LOCUS (loc) != UNKNOWN_LOCATION);
+  expanded_location exploc = expand_location (loc);
   json::object *obj = new json::object ();
-  obj->set ("file", new json::string (LOCATION_FILE (loc)));
-  obj->set ("line", new json::number (LOCATION_LINE (loc)));
-  obj->set ("column", new json::number (LOCATION_COLUMN (loc)));
+  obj->set ("file", new json::string (exploc.file));
+  obj->set ("line", new json::number (exploc.line));
+  obj->set ("column", new json::number (exploc.column));
   return obj;
 }
 
@@ -330,7 +332,7

Re: [PATCH] include more detail in -Warray-bounds (PR 86650)

[David Malcolm]

On Mon, 2018-07-23 at 17:49 -0600, Martin Sebor wrote:
> (David, I'm hoping your your help here.  Please see the end.)
> 
> While looking into a recent -Warray-bounds instance in Glibc
> involving inlining of large functions it became apparent that
> GCC could do a better job of pinpointing the source of
> the problem.
> 
> The attached patch makes a few adjustments to both
> the pretty printer infrastructure and to VRP to make this
> possible.  The diagnostic pretty printer already has directives
> to print the inlining context for both tree and gcall* arguments,
> so most of the changes just adjust things to be able to pass in
> gimple* argument instead.
> 
> The only slightly interesting change is to print the declaration
> to which the out-of-bounds array refers if one is known.
> 
> Tested on x86_64-linux with one regression.
> 
> The regression is in the gcc.dg/Warray-bounds.c test: the column
> numbers of the warnings are off.  Adding the %G specifier to
> the array bounds warnings in VRP has the unexpected effect of
> expanding the extent of the underling. For instance, for a test
> case like this:
> 
>int a[10];
> 
>void f (void)
>{
>  a[-1] = 0;
>}
> 
> from the expected:
> 
> a[-1] = 0;
> ~^~~~
> 
> to this:
> 
>a[-1] = 0;
> ~~^~~
> 
> David, do you have any idea how to avoid this?

Are you referring to the the various places in your patch (in e.g.
  vrp_prop::check_array_ref
  vrp_prop::check_mem_ref
  vrp_prop::search_for_addr_array
) where the patch changed things from this form:

  warning_at (location, OPT_Warray_bounds,
  "[...format string...]", ARGS...);

to this form:

  warning_at (location, OPT_Warray_bounds,
  "%G[...format string...]", stmt, ARGS...);

If so, there are two location_t values of interest here:
(a) the "location" value, and
(b) gimple_location (stmt)

My recollection is that %G and %K override the "location" value passed
in as the first param to the diagnostic call, overwriting it within the
diagnostic_info's text_info with the location value from the %K/%G
(which also set up the pp_ti_abstract_origin of the text_info from the
block information stashed in the ad-hoc data part of the location, so
that the pretty-printer prints the inlining chain).

[aside, why don't we always just print the inlining chain?  IIRC, %K
and %G feel too much like having to jump through hoops to me, given
that gimple_block is looking at gimple_location anyway, why not just
use the location in the location_t's ad-hoc data; I have a feeling
there's a PR open about this, but I don't have it to hand right now].

It looks like the old location was (a), and now you're seeing (b),
since (b) is the location of the statement.

Whether or not this is a problem is difficult to tell; what does the
full diagnostic look like? (you only quoted the diagnostic_show_locus
part of it).

Hope this is helpful
Dave

Re: [PATCH] include more detail in -Warray-bounds (PR 86650)

[David Malcolm]

On Mon, 2018-07-23 at 20:56 -0600, Martin Sebor wrote:
> On 07/23/2018 07:20 PM, David Malcolm wrote:
> > On Mon, 2018-07-23 at 17:49 -0600, Martin Sebor wrote:
> > > (David, I'm hoping your your help here.  Please see the end.)
> > > 
> > > While looking into a recent -Warray-bounds instance in Glibc
> > > involving inlining of large functions it became apparent that
> > > GCC could do a better job of pinpointing the source of
> > > the problem.
> > > 
> > > The attached patch makes a few adjustments to both
> > > the pretty printer infrastructure and to VRP to make this
> > > possible.  The diagnostic pretty printer already has directives
> > > to print the inlining context for both tree and gcall* arguments,
> > > so most of the changes just adjust things to be able to pass in
> > > gimple* argument instead.
> > > 
> > > The only slightly interesting change is to print the declaration
> > > to which the out-of-bounds array refers if one is known.
> > > 
> > > Tested on x86_64-linux with one regression.
> > > 
> > > The regression is in the gcc.dg/Warray-bounds.c test: the column
> > > numbers of the warnings are off.  Adding the %G specifier to
> > > the array bounds warnings in VRP has the unexpected effect of
> > > expanding the extent of the underling. For instance, for a test
> > > case like this:
> > > 
> > >int a[10];
> > > 
> > >void f (void)
> > >{
> > >  a[-1] = 0;
> > >}
> > > 
> > > from the expected:
> > > 
> > > a[-1] = 0;
> > > ~^~~~
> > > 
> > > to this:
> > > 
> > >a[-1] = 0;
> > > ~~^~~
> > > 
> > > David, do you have any idea how to avoid this?
> > 
> > Are you referring to the the various places in your patch (in e.g.
> >   vrp_prop::check_array_ref
> >   vrp_prop::check_mem_ref
> >   vrp_prop::search_for_addr_array
> > ) where the patch changed things from this form:
> > 
> >   warning_at (location, OPT_Warray_bounds,
> >   "[...format string...]", ARGS...);
> > 
> > to this form:
> > 
> >   warning_at (location, OPT_Warray_bounds,
> >   "%G[...format string...]", stmt, ARGS...);
> 
> Yes.
> 
> > 
> > If so, there are two location_t values of interest here:
> > (a) the "location" value, and
> > (b) gimple_location (stmt)
> > 
> > My recollection is that %G and %K override the "location" value
> > passed
> > in as the first param to the diagnostic call, overwriting it within
> > the
> > diagnostic_info's text_info with the location value from the %K/%G
> > (which also set up the pp_ti_abstract_origin of the text_info from
> > the
> > block information stashed in the ad-hoc data part of the location,
> > so
> > that the pretty-printer prints the inlining chain).
> 
> Would having the pretty printer restore the location and
> the block after it's done printing the context and before
> processing the rest of the format string fix it?  (I have
> only a vague idea how this all works so I'm not sure if
> this even makes sense.)

Structurally, it looks like this:

Temporaries during the emission of   |  Long-lived stuff:
the diagnostic:  |
 |+-+
++   ||global_dc|
|diagnostic_info |   |+-+
|++  |   |
||text_info:  |  |   |
||  m_richloc-+--+---> rich_location |
||  x_data+--+---+--> block (via pp_ti_abstract_origin)
|++  |   |
++   |
 |

The location_t of the diagnostic is stored in the rich_location.

Calling:
  warning_at (location)
creates a rich_location wrapping "location" and uses it as above.

During formatting, the %K/%G codes set text_info.x_data via
pp_ti_abstract_origin and overwrite the location_t in the
rich_location.

So in theory we could have a format code that sets the block and
doesn't touch the rich_location.  But that seems like overkill to me.
   

> > 
> > [aside, why don't we always just print the inlining chain?  IIRC,
> > %K
> > and %G feel too much like having to jump through hoops to me, given
> > that gimple_block is looking at gimple_location anyway, why not
> > just
> > use the location in

Re: [PATCH] combine: Allow combining two insns to two insns

[David Malcolm]

On Tue, 2018-07-24 at 17:18 +, Segher Boessenkool wrote:
> This patch allows combine to combine two insns into two.  This helps
> in many cases, by reducing instruction path length, and also allowing
> further combinations to happen.  PR85160 is a typical example of code
> that it can improve.
> 
> This patch does not allow such combinations if either of the original
> instructions was a simple move instruction.  In those cases combining
> the two instructions increases register pressure without improving
> the
> code.  With this move test register pressure does no longer increase
> noticably as far as I can tell.
> 
> (At first I also didn't allow either of the resulting insns to be a
> move instruction.  But that is actually a very good thing to have, as
> should have been obvious).
> 
> Tested for many months; tested on about 30 targets.
> 
> I'll commit this later this week if there are no objections.
> 
> 
> Segher
> 
> 
> 2018-07-24  Segher Boessenkool  
> 
>   PR rtl-optimization/85160
>   * combine.c (is_just_move): New function.
>   (try_combine): Allow combining two instructions into two if
> neither of
>   the original instructions was a move.
> 
> ---
>  gcc/combine.c | 22 --
>  1 file changed, 20 insertions(+), 2 deletions(-)
> 
> diff --git a/gcc/combine.c b/gcc/combine.c
> index cfe0f19..d64e84d 100644
> --- a/gcc/combine.c
> +++ b/gcc/combine.c
> @@ -2604,6 +2604,17 @@ can_split_parallel_of_n_reg_sets (rtx_insn
> *insn, int n)
>return true;
>  }
>  
> +/* Return whether X is just a single set, with the source
> +   a general_operand.  */
> +static bool
> +is_just_move (rtx x)
> +{
> +  if (INSN_P (x))
> +x = PATTERN (x);
> +
> +  return (GET_CODE (x) == SET && general_operand (SET_SRC (x),
> VOIDmode));
> +}

If I'm reading it right, the patch only calls this function on i2 and
i3, which are known to be rtx_insn *, rather than just rtx.

Hence the only way in which GET_CODE (x) can be SET is if the INSN_P
pattern test sets x to PATTERN (x) immediately above: it can't be a SET
otherwise - but this isn't obvious from the code.

Can this function take an rtx_insn * instead?  Maybe something like:

/* Return whether INSN's pattern is just a single set, with the source
   a general_operand.  */
static bool
is_just_move_p (rtx_insn *insn)
{
  if (!INSN_P (insn))
return false;

  rtx x = PATTERN (insn);
  return (GET_CODE (x) == SET && general_operand (SET_SRC (x), VOIDmode));
}

or similar?

[...snip...]

Thanks; I hope this is constructive.
Dave

[committed] optinfo-emit-json.cc: fix trivial memory leak

[David Malcolm]

There's a small leak in class optrecord_json_writer, which shows
up as a new leak in "make selftest-valgrind" as:

==50133== 40 bytes in 1 blocks are definitely lost in loss record 27 of 672
==50133==at 0x4A0645D: malloc (in 
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==50133==by 0x1EF5CAF: xrealloc (xmalloc.c:177)
==50133==by 0xEDDA56: void va_heap::reserve(vec*&, unsigned int, bool) (vec.h:288)
==50133==by 0xEDD74D: vec::reserve(unsigned 
int, bool) (vec.h:1621)
==50133==by 0xEDD515: vec::safe_push(json::array* const&) (vec.h:1730)
==50133==by 0xEDB060: optrecord_json_writer::optrecord_json_writer() 
(optinfo-emit-json.cc:124)
==50133==by 0xEDD141: selftest::test_building_json_from_dump_calls() 
(optinfo-emit-json.cc:935)
==50133==by 0xEDD3AF: selftest::optinfo_emit_json_cc_tests() 
(optinfo-emit-json.cc:955)
==50133==by 0x1DEB3AA: selftest::run_tests() (selftest-run-tests.c:78)
==50133==by 0x1065264: toplev::run_self_tests() (toplev.c:2225)
==50133==by 0x1065486: toplev::main(int, char**) (toplev.c:2303)
==50133==by 0x1E4A092: main (main.c:39)

The fix is trivial.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu;
manually checked "make selftest-valgrind" (which with this fix is now
back from four leaks down to three).

Committed to trunk as r262967 (under the "obvious" rule).

gcc/ChangeLog:
* optinfo-emit-json.cc (class optrecord_json_writer): Convert
field "m_scopes" from vec to auto_vec.
---
 gcc/optinfo-emit-json.cc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gcc/optinfo-emit-json.cc b/gcc/optinfo-emit-json.cc
index 6460a81..2199d52 100644
--- a/gcc/optinfo-emit-json.cc
+++ b/gcc/optinfo-emit-json.cc
@@ -75,7 +75,7 @@ private:
   json::array *m_root_tuple;
 
   /* The currently open scopes, for expressing nested optimization records.  */
-  vec m_scopes;
+  auto_vec m_scopes;
 };
 
 /* optrecord_json_writer's ctor.  Populate the top-level parts of the
-- 
1.8.5.3

Re: [PATCH] -fsave-optimization-record: add contrib/optrecord.py

[David Malcolm]

On Tue, 2018-07-24 at 16:11 +0200, Richard Biener wrote:
> On Mon, Jul 23, 2018 at 9:20 PM David Malcolm 
> wrote:
> > 
> > On Mon, 2018-07-23 at 11:46 +0200, Richard Biener wrote:
> > > On Fri, Jul 20, 2018 at 6:27 PM David Malcolm  > > m>
> > > wrote:
> > > > 
> > > > This patch adds a Python 3 module to "contrib" for reading the
> > > > output of
> > > > -fsave-optimization-record.
> > > > 
> > > > It can be imported from other Python code, or run standalone as
> > > > a
> > > > script,
> > > > in which case it prints the saved messages in a form resembling
> > > > GCC
> > > > diagnostics.
> > > > 
> > > > OK for trunk?
> > > 
> > > OK, but shouldn't there maybe a user-visible (and thus installed)
> > > tool for
> > > this kind of stuff?  Which would mean to place it somewhere else.
> > 
> > As well as this support code, I've got code that uses it to
> > generate
> > HTML reports.  I'm thinking that all this Python code might be
> > better
> > to maintain in an entirely separate repository, as a third-party
> > project (maintained by me, under some suitable Free Software
> > license,
> > accessible via PyPI), since I suspect that the release cycle ought
> > to
> > be different from that of gcc itself.
> > 
> > Would that be a better approach?
> 
> Possibly.
> 
> Richard.

[CCing Rainer and Mike]

A related matter that may affect this: currently there's not much test
coverage for -fsave-optimization-record in trunk (sorry).

"trunk" currently has:

(a) selftest::test_building_json_from_dump_calls, which captures the
results of some dump calls, and does very minimal textual verification
of the JSON that would be emitted by -fsave-optimization-record.

(b) gcc.c-torture/compile/pr86636.c, which merely verifies that we
don't ICE with a particular usage of -fsave-optimization-record.

Ideally we'd have some test coverage of the file written out by -fsave-
optimization-record: that it's valid JSON, that it conforms to the
expected internal structure, and that the expected data is correct and
complete (relative to some known dump calls; I have a plugin for
testing this if need be, in gcc.dg/plugins).

I don't know if Tcl has any JSON support, but in Python, JSON support
is built-in to the standard library, so I wonder if there's a case for
having a DejaGnu directive to (optionally) call out to a Python script
to check the JSON file that's been written, using this optrecord.py
module to handle loading the JSON.  Doing so would implicitly check
that that the emitted adheres to the expected internal structure, and
the script could add additional testcase-specific verifications.

The directive would have to check for the presence of Python, and emit
an UNSUPPORTED if unavailable.

If that sounds sane (and I'm willing to try implementing it), then that
suggests that optrecord.py should live in the gcc source tree, and be
installed somewhere (though I'm not sure where).

Alternatively, this could be done as a selftest, by adding some
functions to json.h/cc for inspecting and traversing the in-memory JSON
tree.  Though that wouldn't be as effective as an "end-to-
end"/integration test.

Thoughts?

Dave


> > Dave
> > 
> > > Richard.
> > > 
> > > > contrib/ChangeLog:
> > > > * optrecord.py: New file.
> > > > ---
> > > >  contrib/optrecord.py | 295
> > > > +++++++
> > > >  1 file changed, 295 insertions(+)
> > > >  create mode 100755 contrib/optrecord.py
> > > > 
> > > > diff --git a/contrib/optrecord.py b/contrib/optrecord.py
> > > > new file mode 100755
> > > > index 000..b07488e
> > > > --- /dev/null
> > > > +++ b/contrib/optrecord.py
> > > > @@ -0,0 +1,295 @@
> > > > +#!/usr/bin/env python3
> > > > +#
> > > > +# Python module for working with the result of -fsave-
> > > > optimization-record
> > > > +# Contributed by David Malcolm .
> > > > +#
> > > > +# Copyright (C) 2018 Free Software Foundation, Inc.
> > > > +# This file is part of GCC.
> > > > +#
> > > > +# GCC is free software; you can redistribute it and/or modify
> > > > it
> > > > under
> > > > +# the terms of the GNU General Public License as published by
> > > > the
> > > > Free
> > > > +# S

[PATCH] RFC: Prototype of "rich vectorization hints" idea

[David Malcolm]

element::make_vert ());
+vect->set_child (0, 2,
+new box_element (new text_element ("vectorized loop")));
+vect->set_child (0, 3, wiring_element::make_vert ());
+vect->set_child (0, 4, new box_element (new text_element ("epilogue")));
+  }
+  grid.set_child (0, 2, wiring_element::make_ne ());
+  grid.set_child (1, 2, wiring_element::make_sew ());
+  grid.set_child (2, 2, wiring_element::make_nw ());
+
+  ASSERT_ELEMENT_PRINT (&grid,
+   " |\n"
+   "++-+\n"
+   "|  |\n"
+   "+--+   +---+\n"
+   "|scalar|   |prologue   |\n"
+   "|  |   +---+\n"
+   "|  |       |\n"
+   "|  |   +---+\n"
+   "|  |   |vectorized loop|\n"
+   "|  |   +---+\n"
+   "|  |   |\n"
+   "|  |   +---+\n"
+   "|  |   |epilogue   |\n"
+   "+--+   +---+\n"
+   "|  |\n"
+   "++-+\n"
+   " |\n");
+}
+
+/* Run all of the selftests within this file.  */
+
+void
+diagram_cc_tests ()
+{
+  test_text_buffer ();
+
+  /* Tests of individual element subclasses.  */
+  test_text_element ();
+  test_grid_layout ();
+  test_box_element ();
+  test_wiring_elements ();
+
+  test_complex_diagram ();
+}
+
+} // namespace selftest
+
+#endif /* CHECKING_P */
diff --git a/gcc/diagram.h b/gcc/diagram.h
new file mode 100644
index 000..cbd4679
--- /dev/null
+++ b/gcc/diagram.h
@@ -0,0 +1,287 @@
+/* Classes for building diagrams.
+   Copyright (C) 2018 Free Software Foundation, Inc.
+   Contributed by David Malcolm .
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3.  If not see
+<http://www.gnu.org/licenses/>.  */
+
+#ifndef GCC_DIAGRAM_H
+#define GCC_DIAGRAM_H
+
+#include "json.h"
+
+/* The size of an element within a diagram, or of the diagram itself.  */
+
+struct size
+{
+  int w;
+  int h;
+
+  size (int width, int height) : w (width), h (height) {}
+
+  int get_area () const { return w * h; }
+};
+
+/* A coordinate within a diagram.  */
+
+struct point
+{
+  int x;
+  int y;
+
+  point (int x_, int y_) : x (x_) , y (y_) {}
+};
+
+/* A cell within a text buffer.  Currently just a "char", but could be
+   extended to represent a unicode code point and color information.  */
+
+struct cell
+{
+  char ch;
+
+  cell () : ch (' ') {}
+  cell (char ch_) : ch (ch_) {}
+};
+
+/* A text buffer, to which a diagram can be rendered.  */
+
+class text_buffer
+{
+ public:
+  text_buffer (size wh);
+  ~text_buffer () { delete m_buffer; }
+
+  void print (pretty_printer *pp) const;
+
+  cell get_cell (point xy) const;
+  void set_cell (point xy, char ch);
+
+  void left_aligned_text_at (point xy, const char *str);
+  void right_aligned_text_at (point xy, const char *str);
+
+ private:
+  int xy_to_index (point xy) const;
+
+ private:
+  size m_wh;
+  int m_area;
+  cell *m_buffer;
+};
+
+/* A base class representing an element within a diagram (or the
+   whole diagram), which can be printed to a location within a
+   text_buffer via the "print" virtual function.
+
+   Elements can also be serialized to JSON, and thus saved
+   in optimization records.
+   TODO: generate HTML and/or SVG from them in (I have this mostly
+   working, but the wiring_element subclass is problematic).
+
+   Elements support two-phase size negotiation:
+
+   (i) client code uses "get_requisition" to ask the element how
+   big it needs to be.  This recurses through children, generating
+   a requested size that's big enough for them all
+
+   (ii) client code calls "set_allocation" on the element, which sets
+   the size, and calls "handle_allocation" to perform any recursive
+   size-setting on the children.  */
+
+class element
+{
+ public:
+  element () : m_wh (0, 0) {}
+  virtual ~element () {}
+
+  void set_allocation 

[PATCH] Fixes to testcase for PR tree-optimization/86636

[David Malcolm]

On Thu, 2018-07-26 at 13:22 +0100, Andre Vieira (lists) wrote:

[...snip...]

> > > diff --git a/gcc/testsuite/gcc.c-torture/compile/pr86636.c
> > > b/gcc/testsuite/gcc.c-torture/compile/pr86636.c
> > > new file mode 100644
> > > index 000..2fe2f70
> > > --- /dev/null
> > > +++ b/gcc/testsuite/gcc.c-torture/compile/pr86636.c
> > > @@ -0,0 +1,8 @@
> > > +/* { dg-options "-fsave-optimization-record -ftree-loop-
> > > vectorize -ftree-parallelize-loops=2" } */
> > > +
> > > +void
> > > +n2 (int ih)
> > > +{
> > > +  while (ih < 1)
> > > +++ih;
> > > +}
> > > --
> > > 1.8.5.3
> > > 
> 
> Hi David,
> 
> I believe the test in this patch needs a "{ dg-require-effective-
> target
> pthread }" as -ftree-parallelize-loops seems to be turning on
> -pthread.
> 
> Cheers,
> Andre

Thanks; indeed I now see that every test case using
  -ftree-parallelize-loops
has such a directive.

Also, the test is missing a dg-do compile.

Does the following look correct?  (borderline covered by "obvious fix"
rule).

Dave


gcc/testsuite/ChangeLog:
PR tree-optimization/86636
* gcc.c-torture/compile/pr86636.c: Add dg-do compile.
Add dg-require-effective-target pthread.
---
 gcc/testsuite/gcc.c-torture/compile/pr86636.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/gcc/testsuite/gcc.c-torture/compile/pr86636.c 
b/gcc/testsuite/gcc.c-torture/compile/pr86636.c
index 2fe2f70..8527a0f 100644
--- a/gcc/testsuite/gcc.c-torture/compile/pr86636.c
+++ b/gcc/testsuite/gcc.c-torture/compile/pr86636.c
@@ -1,3 +1,5 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target pthread } */
 /* { dg-options "-fsave-optimization-record -ftree-loop-vectorize 
-ftree-parallelize-loops=2" } */
 
 void
-- 
1.8.5.3

[PING] Re: [PATCH] C++: suggestions for misspelled private members (PR c++/84993)

[David Malcolm]

Ping:
  https://gcc.gnu.org/ml/gcc-patches/2018-07/msg00594.html

On Wed, 2018-07-11 at 22:42 -0400, David Malcolm wrote:
> PR c++/84993 identifies a problem with our suggestions for
> misspelled member names in the C++ FE for the case where the
> member is private.
> 
> For example, given:
> 
> class foo
> {
> public:
>   double get_ratio() const { return m_ratio; }
> 
> private:
>   double m_ratio;
> };
> 
> void test(foo *ptr)
> {
>   if (ptr->ratio >= 0.5)
> ;// etc
> }
> 
> ...we currently emit this suggestion:
> 
> : In function 'void test(foo*)':
> :12:12: error: 'class foo' has no member named 'ratio'; did
> you mean 'm_ratio'?
>if (ptr->ratio >= 0.5)
> ^
> m_ratio
> 
> ...but if the user follows this suggestion, they get:
> 
> : In function 'void test(foo*)':
> :12:12: error: 'double foo::m_ratio' is private within this
> context
>if (ptr->m_ratio >= 0.5)
> ^~~
> :7:10: note: declared private here
>double m_ratio;
>   ^~~
> :12:12: note: field 'double foo::m_ratio' can be accessed via
> 'double foo::get_ratio() const'
>if (ptr->m_ratio >= 0.5)
> ^~~
> get_ratio()
> 
> It feels wrong to be emitting a fix-it hint that doesn't compile, so
> this
> patch adds the accessor fix-it hint logic to this case, so that we
> directly
> offer a valid suggestion:
> 
> : In function 'void test(foo*)':
> :12:12: error: 'class foo' has no member named 'ratio'; did
> you mean
> 'double foo::m_ratio'? (accessible via 'double foo::get_ratio()
> const')
>if (ptr->ratio >= 0.5)
> ^
> get_ratio()
> 
> Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu;
> adds 105 PASS results to g++.sum.
> 
> OK for trunk?
> 
> gcc/cp/ChangeLog:
>   PR c++/84993
>   * call.c (enforce_access): Move diagnostics to...
>   (complain_about_access): ...this new function.
>   * cp-tree.h (class access_failure_info): Rename split out field
>   "m_field_decl" into "m_decl" and "m_diag_decl".
>   (access_failure_info::record_access_failure): Add tree param.
>   (access_failure_info::was_inaccessible_p): New accessor.
>   (access_failure_info::get_decl): New accessor.
>   (access_failure_info::get_diag_decl): New accessor.
>   (access_failure_info::get_any_accessor): New member function.
>   (access_failure_info::add_fixit_hint): New static member
> function.
>   (complain_about_access): New decl.
>   * typeck.c (access_failure_info::record_access_failure): Update
>   for change to fields.
>   (access_failure_info::maybe_suggest_accessor): Split out
> into...
>   (access_failure_info::get_any_accessor): ...this new
> function...
>   (access_failure_info::add_fixit_hint): ...and this new
> function.
>   (finish_class_member_access_expr): Split out "has no member
> named"
>   error-handling into...
>   (complain_about_unrecognized_member): ...this new function, and
>   check that the guessed name is accessible along the access
> path.
>   Only provide a spell-correction fix-it hint if it is;
> otherwise,
>   attempt to issue an accessor fix-it hint.
> 
> gcc/testsuite/ChangeLog:
>   PR c++/84993
>   * g++.dg/torture/accessor-fixits-9.C: New test.
> ---
>  gcc/cp/call.c|  64 ++
>  gcc/cp/cp-tree.h |  17 ++-
>  gcc/cp/typeck.c  | 150
> +--
>  gcc/testsuite/g++.dg/torture/accessor-fixits-9.C | 119
> ++
>  4 files changed, 282 insertions(+), 68 deletions(-)
>  create mode 100644 gcc/testsuite/g++.dg/torture/accessor-fixits-9.C
> 
> diff --git a/gcc/cp/call.c b/gcc/cp/call.c
> index 209c1fd..121f0ca 100644
> --- a/gcc/cp/call.c
> +++ b/gcc/cp/call.c
> @@ -6456,6 +6456,38 @@ build_op_delete_call (enum tree_code code,
> tree addr, tree size,
>return error_mark_node;
>  }
>  
> +/* Issue diagnostics about a disallowed access of DECL, using
> DIAG_DECL
> +   in the diagnostics.
> +
> +   If ISSUE_ERROR is true, then issue an error about the
> +   access, followed by a note showing the declaration.
> +   Otherwise, just show the note.  */
> +
> +void
> +complain_about_access (tree decl, tree diag_decl, bool issue_error)
> +{
> +  if (TREE_PRIVATE (decl))
> +{
> +   

Re: [PATCH] Fixes to testcase for PR tree-optimization/86636

[David Malcolm]

On Fri, 2018-07-27 at 17:43 +0100, Andre Vieira (lists) wrote:
> On 27/07/18 16:39, David Malcolm wrote:
> > On Thu, 2018-07-26 at 13:22 +0100, Andre Vieira (lists) wrote:
> > 
> > [...snip...]
> > 
> > > > > diff --git a/gcc/testsuite/gcc.c-torture/compile/pr86636.c
> > > > > b/gcc/testsuite/gcc.c-torture/compile/pr86636.c
> > > > > new file mode 100644
> > > > > index 000..2fe2f70
> > > > > --- /dev/null
> > > > > +++ b/gcc/testsuite/gcc.c-torture/compile/pr86636.c
> > > > > @@ -0,0 +1,8 @@
> > > > > +/* { dg-options "-fsave-optimization-record -ftree-loop-
> > > > > vectorize -ftree-parallelize-loops=2" } */
> > > > > +
> > > > > +void
> > > > > +n2 (int ih)
> > > > > +{
> > > > > +  while (ih < 1)
> > > > > +++ih;
> > > > > +}
> > > > > --
> > > > > 1.8.5.3
> > > > > 
> > > 
> > > Hi David,
> > > 
> > > I believe the test in this patch needs a "{ dg-require-effective-
> > > target
> > > pthread }" as -ftree-parallelize-loops seems to be turning on
> > > -pthread.
> > > 
> > > Cheers,
> > > Andre
> > 
> > Thanks; indeed I now see that every test case using
> >   -ftree-parallelize-loops
> > has such a directive.
> > 
> > Also, the test is missing a dg-do compile.
> > 
> > Does the following look correct?  (borderline covered by "obvious
> > fix"
> > rule).
> > 
> > Dave
> > 
> > 
> > gcc/testsuite/ChangeLog:
> > PR tree-optimization/86636
> > * gcc.c-torture/compile/pr86636.c: Add dg-do compile.
> > Add dg-require-effective-target pthread.
> > ---
> >  gcc/testsuite/gcc.c-torture/compile/pr86636.c | 2 ++
> >  1 file changed, 2 insertions(+)
> > 
> > diff --git a/gcc/testsuite/gcc.c-torture/compile/pr86636.c
> > b/gcc/testsuite/gcc.c-torture/compile/pr86636.c
> > index 2fe2f70..8527a0f 100644
> > --- a/gcc/testsuite/gcc.c-torture/compile/pr86636.c
> > +++ b/gcc/testsuite/gcc.c-torture/compile/pr86636.c
> > @@ -1,3 +1,5 @@
> > +/* { dg-do compile } */
> > +/* { dg-require-effective-target pthread } */
> >  /* { dg-options "-fsave-optimization-record -ftree-loop-vectorize
> > -ftree-parallelize-loops=2" } */
> >  
> >  void
> > 
> 
> LGTM, but I am not a maintainer. I too would think you might would be
> ok
> with a "obvious fix" here.

Thanks for giving it another pair of eyes; given that, I've committed
it to trunk (r263031) under the "obvious fix" rule.

Dave

[PATCH 1/5] Simplify dump_context by adding a dump_loc member function

[David Malcolm]

This patch removes some duplicated code in dumpfile.c by
reimplementing the various dump_foo_loc calls in terms of dump_foo.

gcc/ChangeLog:
* dump-context.h (dump_context::dump_loc): New decl.
* dumpfile.c (dump_context::dump_loc): New member function.
(dump_context::dump_gimple_stmt_loc): Reimplement using dump_loc
and dump_gimple_stmt.
(dump_context::dump_gimple_expr_loc): Likewise, using
dump_gimple_expr.
(dump_context::dump_generic_expr_loc): Likewise, using
dump_generic_expr.
(dump_context::dump_printf_loc_va): Likewise, using
dump_printf_va.
(dump_context::begin_scope): Explicitly using the global function
"dump_loc", rather than the member function.
---
 gcc/dump-context.h |   2 +
 gcc/dumpfile.c | 119 ++---
 2 files changed, 33 insertions(+), 88 deletions(-)

diff --git a/gcc/dump-context.h b/gcc/dump-context.h
index a191e3a..f6df0b4 100644
--- a/gcc/dump-context.h
+++ b/gcc/dump-context.h
@@ -39,6 +39,8 @@ class dump_context
 
   ~dump_context ();
 
+  void dump_loc (dump_flags_t dump_kind, const dump_location_t &loc);
+
   void dump_gimple_stmt (dump_flags_t dump_kind, dump_flags_t extra_dump_flags,
 gimple *gs, int spc);
 
diff --git a/gcc/dumpfile.c b/gcc/dumpfile.c
index 176c9b8..3c8bc38 100644
--- a/gcc/dumpfile.c
+++ b/gcc/dumpfile.c
@@ -474,6 +474,27 @@ dump_context::~dump_context ()
   delete m_pending;
 }
 
+/* Print LOC to the appropriate dump destinations, given DUMP_KIND.
+   If optinfos are enabled, begin a new optinfo.  */
+
+void
+dump_context::dump_loc (dump_flags_t dump_kind, const dump_location_t &loc)
+{
+  location_t srcloc = loc.get_location_t ();
+
+  if (dump_file && (dump_kind & pflags))
+::dump_loc (dump_kind, dump_file, srcloc);
+
+  if (alt_dump_file && (dump_kind & alt_flags))
+::dump_loc (dump_kind, alt_dump_file, srcloc);
+
+  if (optinfo_enabled_p ())
+{
+  optinfo &info = begin_next_optinfo (loc);
+  info.handle_dump_file_kind (dump_kind);
+}
+}
+
 /* Dump gimple statement GS with SPC indentation spaces and
EXTRA_DUMP_FLAGS on the dump streams if DUMP_KIND is enabled.  */
 
@@ -504,25 +525,8 @@ dump_context::dump_gimple_stmt_loc (dump_flags_t dump_kind,
dump_flags_t extra_dump_flags,
gimple *gs, int spc)
 {
-  location_t srcloc = loc.get_location_t ();
-  if (dump_file && (dump_kind & pflags))
-{
-  dump_loc (dump_kind, dump_file, srcloc);
-  print_gimple_stmt (dump_file, gs, spc, dump_flags | extra_dump_flags);
-}
-
-  if (alt_dump_file && (dump_kind & alt_flags))
-{
-  dump_loc (dump_kind, alt_dump_file, srcloc);
-  print_gimple_stmt (alt_dump_file, gs, spc, dump_flags | 
extra_dump_flags);
-}
-
-  if (optinfo_enabled_p ())
-{
-  optinfo &info = begin_next_optinfo (loc);
-  info.handle_dump_file_kind (dump_kind);
-  info.add_gimple_stmt (gs, spc, dump_flags | extra_dump_flags);
-}
+  dump_loc (dump_kind, loc);
+  dump_gimple_stmt (dump_kind, extra_dump_flags, gs, spc);
 }
 
 /* Dump gimple statement GS with SPC indentation spaces and
@@ -557,25 +561,8 @@ dump_context::dump_gimple_expr_loc (dump_flags_t dump_kind,
gimple *gs,
int spc)
 {
-  location_t srcloc = loc.get_location_t ();
-  if (dump_file && (dump_kind & pflags))
-{
-  dump_loc (dump_kind, dump_file, srcloc);
-  print_gimple_expr (dump_file, gs, spc, dump_flags | extra_dump_flags);
-}
-
-  if (alt_dump_file && (dump_kind & alt_flags))
-{
-  dump_loc (dump_kind, alt_dump_file, srcloc);
-  print_gimple_expr (alt_dump_file, gs, spc, dump_flags | 
extra_dump_flags);
-}
-
-  if (optinfo_enabled_p ())
-{
-  optinfo &info = begin_next_optinfo (loc);
-  info.handle_dump_file_kind (dump_kind);
-  info.add_gimple_expr (gs, spc, dump_flags | extra_dump_flags);
-}
+  dump_loc (dump_kind, loc);
+  dump_gimple_expr (dump_kind, extra_dump_flags, gs, spc);
 }
 
 
@@ -611,25 +598,8 @@ dump_context::dump_generic_expr_loc (dump_flags_t 
dump_kind,
 dump_flags_t extra_dump_flags,
 tree t)
 {
-  location_t srcloc = loc.get_location_t ();
-  if (dump_file && (dump_kind & pflags))
-{
-  dump_loc (dump_kind, dump_file, srcloc);
-  print_generic_expr (dump_file, t, dump_flags | extra_dump_flags);
-}
-
-  if (alt_dump_file && (dump_kind & alt_flags))
-{
-  dump_loc (dump_kind, alt_dump_file, srcloc);
-  print_generic_expr (alt_dump_file, t, dump_flags | extra_dump_flags);
-}
-
-  if (optinfo_enabled_p ())
-{
-  optinfo &info = begin_next_optinfo (loc);
-  info.handle_dump_file_kind (dump_kind);
-  info.add_tree (t, dump_flags | extra_dump_flags);
-}
+  dump_loc (dump_

[PATCH 0/5] dump_printf support for middle-end types

[David Malcolm]

This patch kit converts dump_print and dump_printf_loc from using
fprintf etc internally to using a new pretty-printer
based on pp_format, which supports formatting middle-end types.

In particular, the following codes are implemented (in addition
to the standard pretty_printer ones):

   %E: gimple *:
   Equivalent to: dump_gimple_expr (MSG_*, TDF_SLIM, stmt, 0)
   %G: gimple *:
   Equivalent to: dump_gimple_stmt (MSG_*, TDF_SLIM, stmt, 0)
   %T: tree:
   Equivalent to: dump_generic_expr (MSG_*, arg, TDF_SLIM).

Hence it becomes possible to convert e.g.:

  if (dump_enabled_p ())
{
  dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,
   "not vectorized: different sized vector "
   "types in statement, ");
  dump_generic_expr (MSG_MISSED_OPTIMIZATION, TDF_SLIM, vectype);
  dump_printf (MSG_MISSED_OPTIMIZATION, " and ");
  dump_generic_expr (MSG_MISSED_OPTIMIZATION, TDF_SLIM, nunits_vectype);
  dump_printf (MSG_MISSED_OPTIMIZATION, "\n");
}
  return false;

into a single call to dump_printf_loc:

  if (dump_enabled_p ())
dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,
 "not vectorized: different sized vector "
 "types in statement, %T and %T\n",
 vectype, nunits_vectype);
  return false;

Unlike regular pretty-printers, this captures metadata for the
formatted chunks as appropriate, so that when written out to a
JSON optimization record, the relevant parts of the message are
labelled by type, and by source location (so that
e.g. %G is entirely equivalent to using dump_gimple_stmt).

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.

OK for trunk?

I'm hoping to use this in a v3 of:
  "[PATCH 0/5] [RFC v2] Higher-level reporting of vectorization problems"
 https://gcc.gnu.org/ml/gcc-patches/2018-07/msg00446.html
where the above might become:
  return opt_result::failure_at (stmt,
 "not vectorized: different sized vector "
 "types in statement, %T and %T\n",
 vectype, nunits_vectype);
where opt_result::failure_at would encapsulate the "false", and
capture an opt_problem * (when dumps are enabled), for the reasons
discussed in that other kit.

David Malcolm (5):
  Simplify dump_context by adding a dump_loc member function
  dumpfile.c: eliminate special-casing of dump_file/alt_dump_file
  C++: clean up cp_printer
  c-family: clean up the data tables in c-format.c
  Formatted printing for dump_* in the middle-end

 gcc/c-family/c-format.c   |  159 +++--
 gcc/c-family/c-format.h   |1 +
 gcc/cp/error.c|   46 +-
 gcc/dump-context.h|   25 +-
 gcc/dumpfile.c| 1011 ++---
 gcc/dumpfile.h|   54 +-
 gcc/optinfo-emit-json.cc  |2 +-
 gcc/optinfo.cc|  135 +---
 gcc/optinfo.h |   38 +-
 gcc/testsuite/gcc.dg/format/gcc_diag-1.c  |   19 +-
 gcc/testsuite/gcc.dg/format/gcc_diag-10.c |   33 +-
 11 files changed, 998 insertions(+), 525 deletions(-)

-- 
1.8.5.3

[PATCH 3/5] C++: clean up cp_printer

[David Malcolm]

This makes it easier to compare cp_printer with gcc_cxxdiag_char_table
in c-format.c.

No functional change intended.

gcc/cp/ChangeLog:
* error.c (cp_printer): In the leading comment, move "%H" and "I"
into alphabetical order, and add missing "%G" and "K".  Within the
switch statement, move cases 'G', 'H', 'I' and 'K" so that the
cases are in alphabetical order.
---
 gcc/cp/error.c | 46 --
 1 file changed, 20 insertions(+), 26 deletions(-)

diff --git a/gcc/cp/error.c b/gcc/cp/error.c
index b0d8e32..7a644fd 100644
--- a/gcc/cp/error.c
+++ b/gcc/cp/error.c
@@ -4025,6 +4025,10 @@ defer_phase_2_of_type_diff (deferred_printed_type 
*deferred,
%D   declaration.
%E   expression.
%F   function declaration.
+   %G   gcall *
+   %H   type difference (from).
+   %I   type difference (to).
+   %K   tree
%L  language as used in extern "lang".
%O  binary operator.
%P   function parameter whose position is indicated by an integer.
@@ -4032,9 +4036,7 @@ defer_phase_2_of_type_diff (deferred_printed_type 
*deferred,
%S   substitution (template + args)
%T   type.
%V   cv-qualifier.
-   %X   exception-specification.
-   %H   type difference (from)
-   %I   type difference (to).  */
+   %X   exception-specification.  */
 static bool
 cp_printer (pretty_printer *pp, text_info *text, const char *spec,
int precision, bool wide, bool set_locus, bool verbose,
@@ -4076,6 +4078,21 @@ cp_printer (pretty_printer *pp, text_info *text, const 
char *spec,
   break;
 case 'E': result = expr_to_string (next_tree); break;
 case 'F': result = fndecl_to_string (next_tree, verbose);  break;
+case 'G':
+  percent_G_format (text);
+  return true;
+case 'H':
+  defer_phase_2_of_type_diff (&postprocessor->m_type_a, next_tree,
+ buffer_ptr, verbose, *quoted);
+  return true;
+case 'I':
+  defer_phase_2_of_type_diff (&postprocessor->m_type_b, next_tree,
+ buffer_ptr, verbose, *quoted);
+  return true;
+case 'K':
+  t = va_arg (*text->args_ptr, tree);
+  percent_K_format (text, t);
+  return true;
 case 'L': result = language_to_string (next_lang); break;
 case 'O': result = op_to_string (false, next_tcode);   break;
 case 'P': result = parm_to_string (next_int);  break;
@@ -4090,29 +4107,6 @@ cp_printer (pretty_printer *pp, text_info *text, const 
char *spec,
 case 'V': result = cv_to_string (next_tree, verbose);  break;
 case 'X': result = eh_spec_to_string (next_tree, verbose);  break;
 
-case 'G':
-  percent_G_format (text);
-  return true;
-
-case 'K':
-  t = va_arg (*text->args_ptr, tree);
-  percent_K_format (text, t);
-  return true;
-
-case 'H':
-  {
-   defer_phase_2_of_type_diff (&postprocessor->m_type_a, next_tree,
-   buffer_ptr, verbose, *quoted);
-   return true;
-  }
-
-case 'I':
-  {
-   defer_phase_2_of_type_diff (&postprocessor->m_type_b, next_tree,
-   buffer_ptr, verbose, *quoted);
-   return true;
-  }
-
 default:
   return false;
 }
-- 
1.8.5.3

[PATCH 4/5] c-family: clean up the data tables in c-format.c

[David Malcolm]

The format_char_info tables in c-format.c for our own formats contain
a lot of repetition.

This patch adds a macro to express the conversion specifiers implemented
within pp_format, making it clearer which are custom ones added by the
various diagnostic_format_decoder callbacks.

Doing so uncovered a few mistakes in the data (based on comparison with
the source of the diagnostic_format_decoder callbacks, and the notes
below), which the patch fixes:

- gcc_diag_char_table didn't have 'Z', but it *is* implemented by pp_format.

- removed erroneous 'G' and 'K' entries from gcc_diag_char_table: they're
  implemented by default_tree_printer (and thus in "tdiag") and by the
  C/C++ FEs, but not in pp_format.

- removed "v" (lower case) from gcc_tdiag_char_table and
  gcc_cxxdiag_char_table

Notes:

pretty-print.h uses this for ATTRIBUTE_GCC_PPDIAG, used by pp_printf
and pp_verbatim:

whereas diagnostic-core.h uses this for ATTRIBUTE_GCC_DIAG, used by
the various diagnostic functions:

/* If we haven't already defined a front-end-specific diagnostics
   style, use the generic one.  */

Hence I'm assuming that __gcc_diag__ is for use for when we don't
know what kind of diagnostic_format_decoder we have, and we can
only rely on pp_format's core functionality, where __gcc_tdiag__
is allowed to assume default_tree_printer.

gcc/c-family/ChangeLog:
* c-format.c (PP_FORMAT_CHAR_TABLE): New macro, based on existing
table entries for gcc_diag_char_table, and the 'Z' entry from
gcc_tdiag_char_table, changing the "chain" entry for 'Z' from
&gcc_tdiag_char_table[0] to &gcc_diag_char_table[0].
(gcc_diag_char_table): Use PP_FORMAT_CHAR_TABLE, implicitly
adding missing "Z" for this table.  Remove erroneous "G" and "K"
entries.
(gcc_tdiag_char_table): Use PP_FORMAT_CHAR_TABLE.  Remove "v".
(gcc_cdiag_char_table): Use PP_FORMAT_CHAR_TABLE.
(gcc_cxxdiag_char_table): Use PP_FORMAT_CHAR_TABLE.  Remove "v".

gcc/testsuite/ChangeLog:
* gcc.dg/format/gcc_diag-1.c (foo): Update the %v tests for
tdiag and cxxdiag.
* gcc.dg/format/gcc_diag-10.c (test_diag): Update tests of %G
and %K.
---
 gcc/c-family/c-format.c   | 99 ++-
 gcc/testsuite/gcc.dg/format/gcc_diag-1.c  |  4 +-
 gcc/testsuite/gcc.dg/format/gcc_diag-10.c |  7 +--
 3 files changed, 35 insertions(+), 75 deletions(-)

diff --git a/gcc/c-family/c-format.c b/gcc/c-family/c-format.c
index a0192dd..82841e4 100644
--- a/gcc/c-family/c-format.c
+++ b/gcc/c-family/c-format.c
@@ -679,43 +679,40 @@ static const format_char_info asm_fprintf_char_table[] =
   { NULL,  0, STD_C89, NOLENGTHS, NULL, NULL, NULL }
 };
 
+/* GCC-specific format_char_info arrays.  */
+
+/* The conversion specifiers implemented within pp_format, and thus supported
+   by all pretty_printer instances within GCC.  */
+
+#define PP_FORMAT_CHAR_TABLE \
+  { "di",  0, STD_C89, { T89_I,   BADLEN,  BADLEN,  T89_L,   T9L_LL,  BADLEN,  
BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN  }, "q",  "",   NULL }, \
+  { "ox",  0, STD_C89, { T89_UI,  BADLEN,  BADLEN,  T89_UL,  T9L_ULL, BADLEN,  
BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN  }, "q",  "",   NULL }, \
+  { "u",   0, STD_C89, { T89_UI,  BADLEN,  BADLEN,  T89_UL,  T9L_ULL, BADLEN,  
BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN  }, "q",  "",   NULL }, \
+  { "c",   0, STD_C89, { T89_I,   BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN,  
BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN  }, "q",  "",   NULL }, \
+  { "s",   1, STD_C89, { T89_C,   BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN,  
BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN  }, "pq", "cR", NULL }, \
+  { "p",   1, STD_C89, { T89_V,   BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN,  
BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN  }, "q",  "c",  NULL }, \
+  { "r",   1, STD_C89, { T89_C,   BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN,  
BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN  }, "","//cR",   NULL 
}, \
+  { "<",   0, STD_C89, NOARGUMENTS, "",  "<",   NULL }, \
+  { ">",   0, STD_C89, NOARGUMENTS, "",  ">",   NULL }, \
+  { "'" ,  0, STD_C89, NOARGUMENTS, "",  "",NULL }, \
+  { "R",   0, STD_C89, NOARGUMENTS, "", "\\",   NULL }, \
+  { "m",   0, STD_C89, NOARGUMENTS, "q", "",   NULL }, \
+  { "Z",   1, STD_C89, { T89_I,   BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN,  
BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN  }, "","", 
&gcc_diag_char_table[0] }
+
 static const format_char_info gcc_diag_char_table[] =
 {
-  /* C89 conversion specifiers.  */
-  { "di",  0, STD_C89, { T89_I,   BADLEN,  BADLEN,  T89_L,   T9L_LL,  BADLEN,  
BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN  }, "q",  "",   NULL },
-  { "ox",  0, STD_C89, { T89_UI,  BADLEN,  BADLEN,  T89_UL,  T9L_ULL, BADLEN,  
BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN,  BADLEN  }, "q",  "",   NULL },
-  { "u",   0, STD_C89, { T89_UI,

[PATCH 5/5] Formatted printing for dump_* in the middle-end

[David Malcolm]

This patch converts dump_print and dump_printf_loc from using
printf (and thus ATTRIBUTE_PRINTF) to using a new pretty-printer
based on pp_format, which supports formatting middle-end types.

In particular, the following codes are implemented (in addition
to the standard pretty_printer ones):

   %E: gimple *:
   Equivalent to: dump_gimple_expr (MSG_*, TDF_SLIM, stmt, 0)
   %G: gimple *:
   Equivalent to: dump_gimple_stmt (MSG_*, TDF_SLIM, stmt, 0)
   %T: tree:
   Equivalent to: dump_generic_expr (MSG_*, arg, TDF_SLIM).

Hence it becomes possible to convert e.g.:

  if (dump_enabled_p ())
{
  dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,
   "not vectorized: different sized vector "
   "types in statement, ");
  dump_generic_expr (MSG_MISSED_OPTIMIZATION, TDF_SLIM, vectype);
  dump_printf (MSG_MISSED_OPTIMIZATION, " and ");
  dump_generic_expr (MSG_MISSED_OPTIMIZATION, TDF_SLIM, nunits_vectype);
  dump_printf (MSG_MISSED_OPTIMIZATION, "\n");
}

into a one-liner:

  if (dump_enabled_p ())
dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,
 "not vectorized: different sized vector "
 "types in statement, %T and %T\n",
 vectype, nunits_vectype);

Unlike regular pretty-printers, this one captures optinfo_item
instances for the formatted chunks as appropriate, so that when
written out to a JSON optimization record, the relevant parts of
the message are labelled by type, and by source location (so that
e.g. %G is entirely equivalent to using dump_gimple_stmt).

dump_printf and dump_printf_loc become marked with
ATTRIBUTE_GCC_DUMP_PRINTF, which the patch also implements.

gcc/c-family/ChangeLog:
* c-format.c (enum format_type): Add gcc_dump_printf_format_type.
(local_gimple_ptr_node): New decl.
(gcc_dump_printf_length_specs): New.
(gcc_dump_printf_flag_pairs): New.
(gcc_dump_printf_flag_specs): New.
(gcc_dump_printf_char_table): New.
(format_types_orig): Add entry for "gcc_dump_printf".
(init_dynamic_diag_info): Create local_gimple_ptr_node.
Set up length_char_specs and conversion_specs for
gcc_dump_printf_format_type.
(handle_format_attribute): Handle gcc_dump_printf_format_type.
* c-format.h (T89_GIMPLE): New macro.

gcc/ChangeLog:
* dump-context.h: Include "dumpfile.h".
(dump_context::dump_printf_va): Convert final param from va_list
to va_list *.  Convert from ATTRIBUTE_PRINTF to
ATTRIBUTE_GCC_DUMP_PRINTF.
(dump_context::dump_printf_loc_va): Likewise.
* dumpfile.c: Include "stringpool.h".
(make_item_for_dump_printf_va): Delete.
(make_item_for_dump_printf): Delete.
(class dump_pretty_printer): New class.
(dump_pretty_printer::dump_pretty_printer): New ctor.
(dump_pretty_printer::emit_items): New member function.
(dump_pretty_printer::emit_any_pending_textual_chunks): New member
function.
(dump_pretty_printer::emit_item): New member function.
(dump_pretty_printer::stash_item): New member function.
(dump_pretty_printer::format_decoder_cb): New member function.
(dump_pretty_printer::decode_format): New member function.
(dump_context::dump_printf_va): Reimplement in terms of
dump_pretty_printer.
(dump_context::dump_printf_loc_va): Convert final param from va_list
to va_list *.
(dump_context::begin_scope): Reimplement call to
make_item_for_dump_printf.
(dump_printf): Update for change to dump_printf_va.
(dump_printf_loc): Likewise.
(selftest::test_capture_of_dump_calls): Convert "stmt" from
greturn * to gimple *.  Add a test_decl.  Add tests of dump_printf
with %T, %E, and %G.
* dumpfile.h (ATTRIBUTE_GCC_DUMP_PRINTF): New macro.
(dump_printf): Replace ATTRIBUTE_PRINTF_2 with
ATTRIBUTE_GCC_DUMP_PRINTF (2, 3).
(dump_printf_loc): Replace ATTRIBUTE_PRINTF_3 with
ATTRIBUTE_GCC_DUMP_PRINTF (3, 0).

gcc/testsuite/ChangeLog:
* gcc.dg/format/gcc_diag-1.c: Fix typo.  Add test coverage for
gcc_dump_printf.
* gcc.dg/format/gcc_diag-10.c: Add gimple typedef.  Add test
coverage for gcc_dump_printf.
---
 gcc/c-family/c-format.c   |  60 -
 gcc/c-family/c-format.h   |   1 +
 gcc/dump-context.h|   7 +-
 gcc/dumpfile.c| 358 +++---
 gcc/dumpfile.h|  20 +-
 gcc/testsuite/gcc.dg/format/gcc_diag-1.c  |  15 +-
 gcc/testsuite/gcc.dg/format/gcc_diag-10.c |  26 +++
 7 files changed, 448 insertions(+), 39 deletions(-)

diff --git a/gcc/c-family/c-format.c b/gcc/c-family/c-format.c
index 82841e4..b524468 100644
--- a/gcc/c-family/c-format.c
+++ b/gcc/c-family/c-form

[PATCH 2/5] dumpfile.c: eliminate special-casing of dump_file/alt_dump_file

[David Malcolm]

With the addition of optinfo, the various dump_* calls had three parts:
- optionally print to dump_file
- optionally print to alt_dump_file
- optionally make an optinfo_item and add it to the pending optinfo,
  creating it for dump_*_loc calls.

However, this split makes it difficult to implement the formatted dumps
later in patch kit, so as enabling work towards that, this patch removes
the above split, so that all dumping within the dump_* API goes through
optinfo_item.

In order to ensure that the dumps to dump_file and alt_dump_file are
processed immediately (rather than being buffered within the pending
optinfo for consolidation), this patch introduces the idea of "immediate"
optinfo_item destinations vs "non-immediate" destinations.

The patch also adds selftest coverage of what's printed, and of scopes.

This adds two allocations per dump_* call when dumping is enabled.
I'm assuming that this isn't a problem, as dump_enabled_p is normally
false.  There are ways of optimizing it if it is an issue (by making
optinfo_item instances become temporaries that borrow the underlying
buffer), but they require nontrivial changes, so I'd prefer to leave
that for another patch kit, if it becomes necessary.

gcc/ChangeLog:
* dump-context.h: Include "pretty-print.h".
(dump_context::refresh_dumps_are_enabled): New decl.
(dump_context::emit_item): New decl.
(class dump_context): Add fields "m_test_pp" and
"m_test_pp_flags".
(temp_dump_context::temp_dump_context): Add param "test_pp_flags".
(temp_dump_context::get_dumped_text): New decl.
(class temp_dump_context): Add field "m_pp".
* dumpfile.c (refresh_dumps_are_enabled): Convert to...
(dump_context::refresh_dumps_are_enabled): ...and add a test for
m_test_pp.
(set_dump_file): Update for above change.
(set_alt_dump_file): Likewise.
(dump_loc): New overload, taking a pretty_printer *.
(dump_context::dump_loc): Call end_any_optinfo.  Dump the location
to any test pretty-printer.
(make_item_for_dump_gimple_stmt): New function, adapted from
optinfo::add_gimple_stmt.
(dump_context::dump_gimple_stmt): Call it, and use the result,
eliminating the direct usage of dump_file and alt_dump_file in
favor of indirectly using them via emit_item.
(make_item_for_dump_gimple_expr): New function, adapted from
optinfo::add_gimple_expr.
(dump_context::dump_gimple_expr): Call it, and use the result,
eliminating the direct usage of dump_file and alt_dump_file in
favor of indirectly using them via emit_item.
(make_item_for_dump_generic_expr): New function, adapted from
optinfo::add_tree.
(dump_context::dump_generic_expr): Call it, and use the result,
eliminating the direct usage of dump_file and alt_dump_file in
favor of indirectly using them via emit_item.
(make_item_for_dump_printf_va): New function, adapted from
optinfo::add_printf_va.
(make_item_for_dump_printf): New function.
(dump_context::dump_printf_va): Call make_item_for_dump_printf_va,
and use the result, eliminating the direct usage of dump_file and
alt_dump_file in favor of indirectly using them via emit_item.
(make_item_for_dump_dec): New function.
(dump_context::dump_dec): Call it, and use the result,
eliminating the direct usage of dump_file and alt_dump_file in
favor of indirectly using them via emit_item.
(make_item_for_dump_symtab_node): New function, adapted from
optinfo::add_symtab_node.
(dump_context::dump_symtab_node): Call it, and use the result,
eliminating the direct usage of dump_file and alt_dump_file in
favor of indirectly using them via emit_item.
(dump_context::begin_scope): Reimplement, avoiding direct usage
of dump_file and alt_dump_file in favor of indirectly using them
via emit_item.
(dump_context::emit_item): New member function.
(temp_dump_context::temp_dump_context): Add param "test_pp_flags".
Set up test pretty-printer on the underlying context.  Call
refresh_dumps_are_enabled.
(temp_dump_context::~temp_dump_context): Call
refresh_dumps_are_enabled.
(temp_dump_context::get_dumped_text): New member function.
(selftest::verify_dumped_text): New function.
(ASSERT_DUMPED_TEXT_EQ): New macro.
(selftest::test_capture_of_dump_calls): Run all tests twice, with
and then without optinfo enabled.  Add uses of
ASSERT_DUMPED_TEXT_EQ to all tests.  Add test of nested scopes.
* dumpfile.h: Update comment for the dump_* API.
* optinfo-emit-json.cc
(selftest::test_building_json_from_dump_calls): Update for new
param for temp_dump_context ctor.
* optinfo.cc (optinfo_item::optinfo_item): Remove "owned"

Re: [PATCH 0/5] dump_printf support for middle-end types

[David Malcolm]

On Tue, 2018-07-31 at 14:50 +0200, Richard Biener wrote:
> On Fri, Jul 27, 2018 at 11:47 PM David Malcolm 
> wrote:
> > 
> > This patch kit converts dump_print and dump_printf_loc from using
> > fprintf etc internally to using a new pretty-printer
> > based on pp_format, which supports formatting middle-end types.
> > 
> > In particular, the following codes are implemented (in addition
> > to the standard pretty_printer ones):
> > 
> >%E: gimple *:
> >Equivalent to: dump_gimple_expr (MSG_*, TDF_SLIM, stmt, 0)
> >%G: gimple *:
> >Equivalent to: dump_gimple_stmt (MSG_*, TDF_SLIM, stmt, 0)
> >%T: tree:
> >Equivalent to: dump_generic_expr (MSG_*, arg, TDF_SLIM).
> > 
> > Hence it becomes possible to convert e.g.:
> > 
> >   if (dump_enabled_p ())
> > {
> >   dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,
> >"not vectorized: different sized vector "
> >"types in statement, ");
> >   dump_generic_expr (MSG_MISSED_OPTIMIZATION, TDF_SLIM,
> > vectype);
> >   dump_printf (MSG_MISSED_OPTIMIZATION, " and ");
> >   dump_generic_expr (MSG_MISSED_OPTIMIZATION, TDF_SLIM,
> > nunits_vectype);
> >   dump_printf (MSG_MISSED_OPTIMIZATION, "\n");
> > }
> >   return false;
> > 
> > into a single call to dump_printf_loc:
> > 
> >   if (dump_enabled_p ())
> > dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,
> >  "not vectorized: different sized vector "
> >  "types in statement, %T and %T\n",
> >  vectype, nunits_vectype);
> >   return false;
> > 
> > Unlike regular pretty-printers, this captures metadata for the
> > formatted chunks as appropriate, so that when written out to a
> > JSON optimization record, the relevant parts of the message are
> > labelled by type, and by source location (so that
> > e.g. %G is entirely equivalent to using dump_gimple_stmt).
> > 
> > Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
> > 
> > OK for trunk?
> 
> Nice!  I'm somehow missing 3/5?  Will look into the other ones now.

Thanks.

FWIW, 3/5 was:
  "[PATCH 3/5] C++: clean up cp_printer"
 https://gcc.gnu.org/ml/gcc-patches/2018-07/msg01766.html

which Jason has already approved (and I've committed as r263046, after
a fresh bootstrap®rtest).

Dave

> Richard.
> 
> > I'm hoping to use this in a v3 of:
> >   "[PATCH 0/5] [RFC v2] Higher-level reporting of vectorization
> > problems"
> >  https://gcc.gnu.org/ml/gcc-patches/2018-07/msg00446.html
> > where the above might become:
> >   return opt_result::failure_at (stmt,
> >  "not vectorized: different sized
> > vector "
> >  "types in statement, %T and %T\n",
> >  vectype, nunits_vectype);
> > where opt_result::failure_at would encapsulate the "false", and
> > capture an opt_problem * (when dumps are enabled), for the reasons
> > discussed in that other kit.
> > 
> > David Malcolm (5):
> >   Simplify dump_context by adding a dump_loc member function
> >   dumpfile.c: eliminate special-casing of dump_file/alt_dump_file
> >   C++: clean up cp_printer
> >   c-family: clean up the data tables in c-format.c
> >   Formatted printing for dump_* in the middle-end
> > 
> >  gcc/c-family/c-format.c   |  159 +++--
> >  gcc/c-family/c-format.h   |1 +
> >  gcc/cp/error.c|   46 +-
> >  gcc/dump-context.h|   25 +-
> >  gcc/dumpfile.c| 1011
> > ++---
> >  gcc/dumpfile.h|   54 +-
> >  gcc/optinfo-emit-json.cc  |2 +-
> >  gcc/optinfo.cc|  135 +---
> >  gcc/optinfo.h |   38 +-
> >  gcc/testsuite/gcc.dg/format/gcc_diag-1.c  |   19 +-
> >  gcc/testsuite/gcc.dg/format/gcc_diag-10.c |   33 +-
> >  11 files changed, 998 insertions(+), 525 deletions(-)
> > 
> > --
> > 1.8.5.3
> > 

Re: [PATCH 5/5] Formatted printing for dump_* in the middle-end

[David Malcolm]

On Tue, 2018-07-31 at 15:03 +0200, Richard Biener wrote:
> On Fri, Jul 27, 2018 at 11:49 PM David Malcolm 
> wrote:
> > 
> > This patch converts dump_print and dump_printf_loc from using
> > printf (and thus ATTRIBUTE_PRINTF) to using a new pretty-printer
> > based on pp_format, which supports formatting middle-end types.
> > 
> > In particular, the following codes are implemented (in addition
> > to the standard pretty_printer ones):
> > 
> >%E: gimple *:
> >Equivalent to: dump_gimple_expr (MSG_*, TDF_SLIM, stmt, 0)
> >%G: gimple *:
> >Equivalent to: dump_gimple_stmt (MSG_*, TDF_SLIM, stmt, 0)
> >%T: tree:
> >Equivalent to: dump_generic_expr (MSG_*, arg, TDF_SLIM).
> > 
> > Hence it becomes possible to convert e.g.:
> > 
> >   if (dump_enabled_p ())
> > {
> >   dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,
> >"not vectorized: different sized vector "
> >"types in statement, ");
> >   dump_generic_expr (MSG_MISSED_OPTIMIZATION, TDF_SLIM,
> > vectype);
> >   dump_printf (MSG_MISSED_OPTIMIZATION, " and ");
> >   dump_generic_expr (MSG_MISSED_OPTIMIZATION, TDF_SLIM,
> > nunits_vectype);
> >   dump_printf (MSG_MISSED_OPTIMIZATION, "\n");
> > }
> > 
> > into a one-liner:
> > 
> >   if (dump_enabled_p ())
> > dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,
> >  "not vectorized: different sized vector "
> >  "types in statement, %T and %T\n",
> >  vectype, nunits_vectype);
> > 
> > Unlike regular pretty-printers, this one captures optinfo_item
> > instances for the formatted chunks as appropriate, so that when
> > written out to a JSON optimization record, the relevant parts of
> > the message are labelled by type, and by source location (so that
> > e.g. %G is entirely equivalent to using dump_gimple_stmt).
> > 
> > dump_printf and dump_printf_loc become marked with
> > ATTRIBUTE_GCC_DUMP_PRINTF, which the patch also implements.
> > 
> > gcc/c-family/ChangeLog:
> > * c-format.c (enum format_type): Add
> > gcc_dump_printf_format_type.
> > (local_gimple_ptr_node): New decl.
> > (gcc_dump_printf_length_specs): New.
> > (gcc_dump_printf_flag_pairs): New.
> > (gcc_dump_printf_flag_specs): New.
> > (gcc_dump_printf_char_table): New.
> > (format_types_orig): Add entry for "gcc_dump_printf".
> > (init_dynamic_diag_info): Create local_gimple_ptr_node.
> > Set up length_char_specs and conversion_specs for
> > gcc_dump_printf_format_type.
> > (handle_format_attribute): Handle
> > gcc_dump_printf_format_type.
> > * c-format.h (T89_GIMPLE): New macro.
> 
> Iff the c-family changes are neccessary (are they?) then how does
> this
> work for non-c-family languages which do not link c-family/c-
> format.o?

The c-family changes are necessary for bootstrap, so that -Wformat
works cleanly after changing dump_printf_loc etc from
  ATTRIBUTE_PRINTF_3;
to
  ATTRIBUTE_GCC_DUMP_PRINTF (3, 0);
i.e. they're just the changes to -Wformat to teach it how to verify the
new:
  __attribute__ ((__format__ (__gcc_dump_printf__, m ,n)))
(hence the cleanups to c-format.c earlier in the patch kit, to avoid
yet more copy-and-paste there for the new format decoder callback).

The implementation itself is all within dumpfile.c, hence the non-c-
family languages ought to work.  My testing was with:
  --enable-languages=c,c++,objc,obj-c++,fortran,ada,go,lto,jit,brig
(and with cloog and isl, fwiw).

(I kept the alphabetization of the ChangeLog files from my generate-
changelog.py script, which put the gcc/c-family/ChangeLog before the
gcc/ChangeLog and thus may have made this confusing to read, sorry).

I didn't exhaustively check every callsite to the changed calls; I'm
assuming that -Wformat during bootstrap has effectively checked that
for me.  Though now I think about it, I note that we use
HOST_WIDE_INT_PRINT_DEC in many places: is this guaranteed to be a
valid input to pp_format on all of our configurations?

Dave

> 
> > gcc/ChangeLog:
> > * dump-context.h: Include "dumpfile.h".
> > (dump_context::dump_printf_va): Convert final param from
> > va_list
> > to va_list *.  Convert from ATTRIBUTE_PRINTF to
> > ATTRIBUTE_GCC_DUMP_PRINTF.
> > (dump_context::dump_printf_loc_va): Likewise.
> >

Re: [PATCH 2/5] dumpfile.c: eliminate special-casing of dump_file/alt_dump_file

[David Malcolm]

On Tue, 2018-07-31 at 14:53 +0200, Richard Biener wrote:
> On Fri, Jul 27, 2018 at 11:48 PM David Malcolm 
> wrote:
> > 
> > With the addition of optinfo, the various dump_* calls had three
> > parts:
> > - optionally print to dump_file
> > - optionally print to alt_dump_file
> > - optionally make an optinfo_item and add it to the pending
> > optinfo,
> >   creating it for dump_*_loc calls.
> > 
> > However, this split makes it difficult to implement the formatted
> > dumps
> > later in patch kit, so as enabling work towards that, this patch
> > removes
> > the above split, so that all dumping within the dump_* API goes
> > through
> > optinfo_item.
> > 
> > In order to ensure that the dumps to dump_file and alt_dump_file
> > are
> > processed immediately (rather than being buffered within the
> > pending
> > optinfo for consolidation), this patch introduces the idea of
> > "immediate"
> > optinfo_item destinations vs "non-immediate" destinations.
> > 
> > The patch also adds selftest coverage of what's printed, and of
> > scopes.
> > 
> > This adds two allocations per dump_* call when dumping is enabled.
> > I'm assuming that this isn't a problem, as dump_enabled_p is
> > normally
> > false.  There are ways of optimizing it if it is an issue (by
> > making
> > optinfo_item instances become temporaries that borrow the
> > underlying
> > buffer), but they require nontrivial changes, so I'd prefer to
> > leave
> > that for another patch kit, if it becomes necessary.
> 
> Yeah, I guess that's OK given we can consolidate quite some calls
> after
> your patch anyways. 

We can, but FWIW my plan is to only touch the calls that I need to to
implement the  "Higher-level reporting of vectorization problems" idea
here:
   https://gcc.gnu.org/ml/gcc-patches/2018-07/msg00446.html

where the explicit dump calls become implicit within calls to things
like:

   return opt_result::failure_at (stmt,
  "not vectorized: different sized vector "
  "types in statement, %T and %T\n",
  vectype, nunits_vectype);

But if you think it's worthwhile, I can do a big patch that uses these
format codes throughout.

> Using alloca + placement new would be possible
> as well I guess?

Maybe.  I think the underlying question here is "what should the
lifetimes of the optinfo_items (and their text buffers) be?"

In the initial version of the optinfo patch kit, I had optinfo_items
being created in response to the various dump_* calls, and them being
added to an optinfo (which takes ownership of them), before the optinfo
is eventually emitted to various destinations; the optinfo is then
deleted, deleting the owned items.

This lifetime approach (having the optinfos own the optinfo_items) was
necessary because one of the destinations was through the diagnostics
system; they needed consolidation so that all of the items could be
alive at the point of emission.  (I think the JSON output also required
it at one point).

Hence the above approach needs the items and thus their underlying text
strings to live as long as the optinfo that owns them - the
destinations assume that the optinfo_items are all alive at the point
of emission.

Hence this requires new/delete pairs for the items, and also the
xstrdup around the text buffer, so that the items can own a copy.

But the dump_file and alt_dump_file destinations don't need the items
to be long-lived: they can be temporary wrappers.

Similarly, the optimization record destination could simply work in
terms of temporary items: when an optinfo_item is added, the
corresponding JSON could be added immediately.

So I think the only things that are requiring optinfo_items to be long-
lived are:
* the -fremarks idea from an earlier patch kit - and I'm not sure what
our plans for that should be, in terms of how it should interact with
alt_dump_file/-fopt-info
* the selftests within dumpfile.c itself.

So the other approach would be to rewrite dumpfile.c so that
optinfo_item instances (or maybe "dump_item" instances) are implicitly
temporary wrappers around a text buffer; the various emit destinations
make no assumptions that the items will stick around; any that do need 
them to (e.g. for dumpfile.c's selftests) make a copy, perhaps with a
optinfo_items_need_saving_p () function to guard adding a copy of each
item into the optinfo.

That would avoid the new/delete pair for all of the optinfo_item
instances, and the xstrdup for each one, apart from during selftests.

But it's a rewrite of this code (and has interactions with the

Re: [RFC] Add function attribute: null_terminated_string_arg(PARAM_IDX)

[David Malcolm]

On Thu, 2023-10-19 at 16:13 +0200, Andreas Schwab wrote:
> On Okt 19 2023, David Malcolm wrote:

[...]

> > +   /* First, check for a null-terminated string *without*
> > +  emitting emitting warnings (via a null context), to
> 
> -emitting
> 

Thanks; I've fixed this in my working copy.

Dave

Re: [PATCH] libcpp: Improve the diagnostic for poisoned identifiers [PR36887]

[David Malcolm]

On Wed, 2023-09-20 at 00:12 -0400, Lewis Hyatt wrote:
> Hello-
> 
> This patch implements the PR's request to add more information to the
> diagnostic issued for using a poisoned identifier. Bootstrapped +
> regtested
> all languages on x86-64 Linux. Does it look OK please? Thanks!

Thanks!

Patch looks good to me; please go ahead and push it.

Dave

Re: [PATCH v5 2/5] libcpp: add a function to determine UTF-8 validity of a C string

[David Malcolm]

On Wed, Jan 25, 2023 at 4:09 PM Ben Boeckel via Gcc  wrote:
>
> This simplifies the interface for other UTF-8 validity detections when a
> simple "yes" or "no" answer is sufficient.
>
> libcpp/
>
> * charset.cc: Add `_cpp_valid_utf8_str` which determines whether
> a C string is valid UTF-8 or not.
> * internal.h: Add prototype for `_cpp_valid_utf8_str`.
>
> Signed-off-by: Ben Boeckel 

[going through patches in patchwork]

What's the status of this patch; did this ever get committed?

I see that Jason preapproved this via his review of "[PATCH v3 2/3]
libcpp: add a function to determine UTF-8 validity of a C string"

Thanks
Dave

Re: [PATCH v5 2/5] libcpp: add a function to determine UTF-8 validity of a C string

[David Malcolm]

On Mon, 2023-10-23 at 11:24 -0400, Jason Merrill wrote:
> On 10/23/23 11:16, David Malcolm wrote:
> > On Wed, Jan 25, 2023 at 4:09 PM Ben Boeckel via Gcc
> >  wrote:
> > > 
> > > This simplifies the interface for other UTF-8 validity detections
> > > when a
> > > simple "yes" or "no" answer is sufficient.
> > > 
> > > libcpp/
> > > 
> > >  * charset.cc: Add `_cpp_valid_utf8_str` which determines
> > > whether
> > >  a C string is valid UTF-8 or not.
> > >  * internal.h: Add prototype for `_cpp_valid_utf8_str`.
> > > 
> > > Signed-off-by: Ben Boeckel 
> > 
> > [going through patches in patchwork]
> > 
> > What's the status of this patch; did this ever get committed?
> 
> It was superseded.

Thanks; closed out in patchwork.

Dave

Re: [PATCH] gcc/jit/jit-recording.cc: recording::global::write_to_dump: Avoid crashes when writing psuedo-C for globals with string initializers.

[David Malcolm]

On Fri, 2022-11-25 at 02:13 +0530, Vibhav Pant via Jit wrote:
> If a char * global was initialized with a rvalue from
> `gcc_jit_context_new_string_literal` containing a format string,
> dumping the context causes libgccjit to SIGSEGV due to an improperly
> constructed call to vasprintf. The following code snippet can
> reproduce
> the crash:
> 
> int main(int argc, char **argv)
> {
>  gcc_jit_context *ctxt = gcc_jit_context_acquire ();
>  gcc_jit_lvalue *var = gcc_jit_context_new_global(
>  ctxt, NULL, GCC_JIT_GLOBAL_EXPORTED,
>  gcc_jit_context_get_type(ctxt, GCC_JIT_TYPE_CONST_CHAR_PTR),
>  "var");
>  gcc_jit_global_set_initializer_rvalue(
>  var, gcc_jit_context_new_string_literal(ctxt, "%s"));
>  gcc_jit_context_dump_to_file (ctxt, "output", 0);
>  return 0;
> }
> 
> The offending line is jit-recording.cc:4922, where a call to d.write
> passes the initializer rvalue's debug string to `write` without a
> format specifier. The attached patch fixes this issue.

Thanks for spotting this, and sorry about missing your patch.

I've gone ahead and pushed this to trunk (as r14-4923-gac66744d94226a),
and will backport it.

Dave

Re: [RFC] Add function attribute: null_terminated_string_arg(PARAM_IDX)

[David Malcolm]

On Thu, 2023-10-19 at 10:02 -0400, David Malcolm wrote:
> This patch adds a new function attribute to GCC for marking that an
> argument is expected to be a null-terminated string.
> 
> For example, consider:
> 
>   void test_a (const char *p)
>     __attribute__((null_terminated_string_arg (1)));
> 
> which would indicate to humans and compilers that argument 1 of
> "test_a"
> is expected to be a null-terminated string, with the idea:
> 
> - we should complain if it's not valid to read from *p up to the
> first
>   '\0' character in the buffer
> 
> - we should complain if *p is not terminated, or if it's
> uninitialized
>   before the first '\0' character
> 
> This is independent of the nonnull-ness of the pointer: if you also
> want
> to express that the argument must be non-null, we already have
> __attribute__((nonnull (N))), so the user can write e.g.:
> 
>   void test_b (const char *p)
>     __attribute__((null_terminated_string_arg (1))
>     __attribute__((nonnull (1)));
> 
> which can also be spelled as:
> 
>   void test_b (const char *p)
>  __attribute__((null_terminated_string_arg (1),
>     nonnull (1)));
> 
> For a function similar to strncpy, we can use the "access" attribute
> to
> express a maximum size of the read:
> 
>   void test_c (const char *p, size_t sz)
>  __attribute__((null_terminated_string_arg (1),
>     nonnull (1),
>     access (read_only, 1, 2)));
> 
> The patch implements:
> (a) C/C++ frontends: recognition of this attribute
> (b) analyzer: usage of this attribute
> 
> The name is rather long; a shorter name might be "c_string_arg".
> 
> Does anything like this already exist in GCC, or in any other
> compilers or analysis tools?
> 
> Thoughts?
> 
> Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.

Given a week of no comments (apart from Andreas' typo fix), I've gone
ahead and pushed this to trunk as r14-4958-gcd7dadcd2759d1 (with the
typo fix).

I kept the rather long but explicit spelling
("null_terminated_string_arg") since projects tend to use macros for
function attributes rather than spelling them out each time, and so I
don't see the length being a problem to end-users.

Dave

[pushed] pretty-print: gracefully handle null URLs

[David Malcolm]

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-5047-gb9e2088d297744.

gcc/ChangeLog:
* pretty-print.cc (pretty_printer::pretty_printer): Initialize
m_skipping_null_url.
(pp_begin_url): Handle URL being null.
(pp_end_url): Likewise.
(selftest::test_null_urls): New.
(selftest::pretty_print_cc_tests): Call it.
* pretty-print.h (pretty_printer::m_skipping_null_url): New.
---
 gcc/pretty-print.cc | 57 +++--
 gcc/pretty-print.h  |  4 
 2 files changed, 59 insertions(+), 2 deletions(-)

diff --git a/gcc/pretty-print.cc b/gcc/pretty-print.cc
index 75446cc73a1..80780cfd7b8 100644
--- a/gcc/pretty-print.cc
+++ b/gcc/pretty-print.cc
@@ -1664,7 +1664,8 @@ pretty_printer::pretty_printer (int maximum_length)
 need_newline (),
 translate_identifiers (true),
 show_color (),
-url_format (URL_FORMAT_NONE)
+url_format (URL_FORMAT_NONE),
+m_skipping_null_url (false)
 {
   pp_line_cutoff (this) = maximum_length;
   /* By default, we emit prefixes once per message.  */
@@ -1687,7 +1688,8 @@ pretty_printer::pretty_printer (const pretty_printer 
&other)
   need_newline (other.need_newline),
   translate_identifiers (other.translate_identifiers),
   show_color (other.show_color),
-  url_format (other.url_format)
+  url_format (other.url_format),
+  m_skipping_null_url (false)
 {
   pp_line_cutoff (this) = maximum_length;
   /* By default, we emit prefixes once per message.  */
@@ -2211,6 +2213,13 @@ identifier_to_locale (const char *ident)
 void
 pp_begin_url (pretty_printer *pp, const char *url)
 {
+  if (!url)
+{
+  /* Handle null URL by skipping all output here,
+and in the next pp_end_url.  */
+  pp->m_skipping_null_url = true;
+  return;
+}
   switch (pp->url_format)
 {
 case URL_FORMAT_NONE:
@@ -2254,6 +2263,13 @@ get_end_url_string (pretty_printer *pp)
 void
 pp_end_url (pretty_printer *pp)
 {
+  if (pp->m_skipping_null_url)
+{
+  /* We gracefully handle pp_begin_url (NULL) by omitting output for
+both begin and end.  Here we handle the latter.  */
+  pp->m_skipping_null_url = false;
+  return;
+}
   if (pp->url_format != URL_FORMAT_NONE)
 pp_string (pp, get_end_url_string (pp));
 }
@@ -2588,6 +2604,42 @@ test_urls ()
   }
 }
 
+/* Verify that we gracefully reject null URLs.  */
+
+void
+test_null_urls ()
+{
+  {
+pretty_printer pp;
+pp.url_format = URL_FORMAT_NONE;
+pp_begin_url (&pp, nullptr);
+pp_string (&pp, "This isn't a link");
+pp_end_url (&pp);
+ASSERT_STREQ ("This isn't a link",
+ pp_formatted_text (&pp));
+  }
+
+  {
+pretty_printer pp;
+pp.url_format = URL_FORMAT_ST;
+pp_begin_url (&pp, nullptr);
+pp_string (&pp, "This isn't a link");
+pp_end_url (&pp);
+ASSERT_STREQ ("This isn't a link",
+ pp_formatted_text (&pp));
+  }
+
+  {
+pretty_printer pp;
+pp.url_format = URL_FORMAT_BEL;
+pp_begin_url (&pp, nullptr);
+pp_string (&pp, "This isn't a link");
+pp_end_url (&pp);
+ASSERT_STREQ ("This isn't a link",
+ pp_formatted_text (&pp));
+  }
+}
+
 /* Test multibyte awareness.  */
 static void test_utf8 ()
 {
@@ -2637,6 +2689,7 @@ pretty_print_cc_tests ()
   test_pp_format ();
   test_prefixes_and_wrapping ();
   test_urls ();
+  test_null_urls ();
   test_utf8 ();
 }
 
diff --git a/gcc/pretty-print.h b/gcc/pretty-print.h
index 02658c8afad..8759f0def38 100644
--- a/gcc/pretty-print.h
+++ b/gcc/pretty-print.h
@@ -295,6 +295,10 @@ public:
 
   /* Whether URLs should be emitted, and which terminator to use.  */
   diagnostic_url_format url_format;
+
+  /* If true, then we've had a pp_begin_url (nullptr), and so the
+ next pp_end_url should be a no-op.  */
+  bool m_skipping_null_url;
 };
 
 inline const char *
-- 
2.26.3

[pushed] opts.cc: fix comment about DOCUMENTATION_ROOT_URL

[David Malcolm]

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-5048-g8b4ac021cd1f63.

gcc/ChangeLog:
* opts.cc (get_option_url): Update comment; the requirement to
pass DOCUMENTATION_ROOT_URL's value via -D was removed in
r10-8065-ge33a1eae25b8a8.
---
 gcc/opts.cc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/gcc/opts.cc b/gcc/opts.cc
index 8015cb7556a..f54cf8305ca 100644
--- a/gcc/opts.cc
+++ b/gcc/opts.cc
@@ -3679,9 +3679,9 @@ char *
 get_option_url (diagnostic_context *, int option_index)
 {
   if (option_index)
-return concat (/* DOCUMENTATION_ROOT_URL should be supplied via -D by
- the Makefile (see --with-documentation-root-url), and
- should have a trailing slash.  */
+return concat (/* DOCUMENTATION_ROOT_URL should be supplied via
+ #include "config.h" (see --with-documentation-root-url),
+ and should have a trailing slash.  */
   DOCUMENTATION_ROOT_URL,
 
   /* get_option_html_page will return something like
-- 
2.26.3

[pushed] libcpp: eliminate MACRO_MAP_EXPANSION_POINT_LOCATION

[David Malcolm]

This patch eliminates the function "MACRO_MAP_EXPANSION_POINT_LOCATION"
(which hasn't been a macro since r6-739-g0501dbd932a7e9) in favor of
a new line_map_macro::get_expansion_point_location accessor.

No functional change intended.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-5049-gb0f19336f247c6.

gcc/c-family/ChangeLog:
* c-warn.cc (warn_for_multistatement_macros): Update for removal
of MACRO_MAP_EXPANSION_POINT_LOCATION.

gcc/cp/ChangeLog:
* module.cc (ordinary_loc_of): Update for removal of
MACRO_MAP_EXPANSION_POINT_LOCATION.
(module_state::note_location): Update for renaming of field.
(module_state::write_macro_maps): Likewise.

gcc/ChangeLog:
* input.cc (dump_location_info): Update for removal of
MACRO_MAP_EXPANSION_POINT_LOCATION.
* tree-diagnostic.cc (maybe_unwind_expanded_macro_loc):
Likewise.

libcpp/ChangeLog:
* include/line-map.h
(line_map_macro::get_expansion_point_location): New accessor.
(line_map_macro::expansion): Rename field to...
(line_map_macro::mexpansion): Rename field to...
(MACRO_MAP_EXPANSION_POINT_LOCATION): Delete this function.
* line-map.cc (linemap_enter_macro): Update for renaming of field.
(linemap_macro_map_loc_to_exp_point): Update for removal of
MACRO_MAP_EXPANSION_POINT_LOCATION.
---
 gcc/c-family/c-warn.cc|  2 +-
 gcc/cp/module.cc  |  6 +++---
 gcc/input.cc  |  4 ++--
 gcc/tree-diagnostic.cc|  2 +-
 libcpp/include/line-map.h | 19 ++-
 libcpp/line-map.cc|  4 ++--
 6 files changed, 19 insertions(+), 18 deletions(-)

diff --git a/gcc/c-family/c-warn.cc b/gcc/c-family/c-warn.cc
index 9ab83a9a84a..bc889cee6b9 100644
--- a/gcc/c-family/c-warn.cc
+++ b/gcc/c-family/c-warn.cc
@@ -2951,7 +2951,7 @@ warn_for_multistatement_macros (location_t body_loc, 
location_t next_loc,
   while (linemap_macro_expansion_map_p (guard_map))
 {
   const line_map_macro *mm = linemap_check_macro (guard_map);
-  guard_loc_exp = MACRO_MAP_EXPANSION_POINT_LOCATION (mm);
+  guard_loc_exp = mm->get_expansion_point_location ();
   guard_map = linemap_lookup (line_table, guard_loc_exp);
   if (guard_map == body_map)
return;
diff --git a/gcc/cp/module.cc b/gcc/cp/module.cc
index 539518d7923..c1c8c226bc1 100644
--- a/gcc/cp/module.cc
+++ b/gcc/cp/module.cc
@@ -13937,7 +13937,7 @@ ordinary_loc_of (line_maps *lmaps, location_t from)
  /* Find the ordinary location nearest FROM.  */
  const line_map *map = linemap_lookup (lmaps, from);
  const line_map_macro *mac_map = linemap_check_macro (map);
- from = MACRO_MAP_EXPANSION_POINT_LOCATION (mac_map);
+ from = mac_map->get_expansion_point_location ();
}
 }
   return from;
@@ -15779,7 +15779,7 @@ module_state::note_location (location_t loc)
  slot->remap = 0;
  // Expansion locations could themselves be from a
  // macro, we need to note them all.
- note_location (mac_map->expansion);
+ note_location (mac_map->m_expansion);
  gcc_checking_assert (mac_map->n_tokens);
  location_t tloc = UNKNOWN_LOCATION;
  for (unsigned ix = mac_map->n_tokens * 2; ix--;)
@@ -16375,7 +16375,7 @@ module_state::write_macro_maps (elf_out *to, range_t 
&info, unsigned *crc_p)
   sec.u (iter->remap);
   sec.u (mac->n_tokens);
   sec.cpp_node (mac->macro);
-  write_location (sec, mac->expansion);
+  write_location (sec, mac->m_expansion);
   const location_t *locs = mac->macro_locations;
   /* There are lots of identical runs.  */
   location_t prev = UNKNOWN_LOCATION;
diff --git a/gcc/input.cc b/gcc/input.cc
index fd09fccb0e3..6256d81f531 100644
--- a/gcc/input.cc
+++ b/gcc/input.cc
@@ -1530,9 +1530,9 @@ dump_location_info (FILE *stream)
   map->start_location,
   (map->start_location
+ MACRO_MAP_NUM_MACRO_TOKENS (map)));
-  inform (MACRO_MAP_EXPANSION_POINT_LOCATION (map),
+  inform (map->get_expansion_point_location (),
  "expansion point is location %i",
- MACRO_MAP_EXPANSION_POINT_LOCATION (map));
+ map->get_expansion_point_location ());
   fprintf (stream, "  map->start_location: %u\n",
   map->start_location);
 
diff --git a/gcc/tree-diagnostic.cc b/gcc/tree-diagnostic.cc
index a600f0e9f64..cae400cf372 100644
--- a/gcc/tree-diagnostic.cc
+++ b/gcc/tree-diagnostic.cc
@@ -217,7 +217,7 @@ maybe_unwind_expanded_macro_loc (diagnostic_context 
*context,
This is the locus 2/ of the earlier comment.  */
 location_t resolved_exp_loc =
   linemap_resolve_location (line_table,
-MACRO_MAP_EXPANSION_POINT_LOCATION 
(iter->map),
+

[pushed] analyzer: move class record_layout to its own .h/.cc

[David Malcolm]

No functional change intended.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-5050-g37e1634ef1a6f1.

gcc/ChangeLog:
* Makefile.in (ANALYZER_OBJS): Add analyzer/record-layout.o.

gcc/analyzer/ChangeLog:
* record-layout.cc: New file, based on material in region-model.cc.
* record-layout.h: Likewise.
* region-model.cc: Include "analyzer/record-layout.h".
(class record_layout): Move to record-layout.cc and .h
---
 gcc/Makefile.in   |   1 +
 gcc/analyzer/record-layout.cc | 125 
 gcc/analyzer/record-layout.h  |  91 +++
 gcc/analyzer/region-model.cc  | 132 +-
 4 files changed, 218 insertions(+), 131 deletions(-)
 create mode 100644 gcc/analyzer/record-layout.cc
 create mode 100644 gcc/analyzer/record-layout.h

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 91d6bfbea4d..41ed8163cd8 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -1324,6 +1324,7 @@ ANALYZER_OBJS = \
analyzer/program-point.o \
analyzer/program-state.o \
analyzer/ranges.o \
+   analyzer/record-layout.o \
analyzer/region.o \
analyzer/region-model.o \
analyzer/region-model-asm.o \
diff --git a/gcc/analyzer/record-layout.cc b/gcc/analyzer/record-layout.cc
new file mode 100644
index 000..1369bfb5eff
--- /dev/null
+++ b/gcc/analyzer/record-layout.cc
@@ -0,0 +1,125 @@
+/* Implementation of class record_layout.
+   Copyright (C) 2022-2023 Free Software Foundation, Inc.
+   Contributed by David Malcolm .
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 3, or (at your option)
+any later version.
+
+GCC is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3.  If not see
+<http://www.gnu.org/licenses/>.  */
+
+#include "config.h"
+#define INCLUDE_MEMORY
+#include "system.h"
+#include "coretypes.h"
+#include "tree.h"
+#include "function.h"
+#include "basic-block.h"
+#include "gimple.h"
+#include "diagnostic.h"
+#include "tree-diagnostic.h"
+#include "analyzer/analyzer.h"
+#include "analyzer/record-layout.h"
+
+#if ENABLE_ANALYZER
+
+namespace ana {
+
+/* class record_layout.  */
+
+record_layout::record_layout (tree record_type)
+{
+  gcc_assert (TREE_CODE (record_type) == RECORD_TYPE);
+
+  for (tree iter = TYPE_FIELDS (record_type); iter != NULL_TREE;
+   iter = DECL_CHAIN (iter))
+{
+  if (TREE_CODE (iter) == FIELD_DECL)
+   {
+ int iter_field_offset = int_bit_position (iter);
+ bit_size_t size_in_bits;
+ if (!int_size_in_bits (TREE_TYPE (iter), &size_in_bits))
+   size_in_bits = 0;
+
+ maybe_pad_to (iter_field_offset);
+
+ /* Add field.  */
+ m_items.safe_push (item (bit_range (iter_field_offset,
+ size_in_bits),
+  iter, false));
+   }
+}
+
+  /* Add any trailing padding.  */
+  bit_size_t size_in_bits;
+  if (int_size_in_bits (record_type, &size_in_bits))
+maybe_pad_to (size_in_bits);
+}
+
+void
+record_layout::dump_to_pp (pretty_printer *pp) const
+{
+  unsigned i;
+  item *it;
+  FOR_EACH_VEC_ELT (m_items, i, it)
+{
+  it->dump_to_pp (pp);
+  pp_newline (pp);
+}
+}
+
+void
+record_layout::dump () const
+{
+  pretty_printer pp;
+  pp_format_decoder (&pp) = default_tree_printer;
+  pp.buffer->stream = stderr;
+  dump_to_pp (&pp);
+  pp_flush (&pp);
+}
+
+const record_layout::item *
+record_layout::get_item_at (bit_offset_t offset) const
+{
+  unsigned i;
+  item *it;
+  FOR_EACH_VEC_ELT (m_items, i, it)
+if (it->contains_p (offset))
+  return it;
+  return NULL;
+}
+
+/* Subroutine of ctor.  Add padding item to NEXT_OFFSET if necessary.  */
+
+void
+record_layout::maybe_pad_to (bit_offset_t next_offset)
+{
+  if (m_items.length () > 0)
+{
+  const item &last_item = m_items[m_items.length () - 1];
+  bit_offset_t offset_after_last_item
+   = last_item.get_next_bit_offset ();
+  if (next_offset > offset_after_last_item)
+   {
+ bit_size_t padding_size
+   = next_offset - offset_after_last_item;
+ m_items.safe_push (item (bit_range (offset_after_last_item,
+ padding_size),
+  last_item.m_field, true));
+   }

[pushed] analyzer: fix clang warnings [PR112317]

[David Malcolm]

No functional change intended.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-5080-gc71028c979d55f.

gcc/analyzer/ChangeLog:
PR analyzer/112317
* access-diagram.cc (class x_aligned_x_ruler_widget): Eliminate
unused field "m_col_widths".
(access_diagram_impl::add_valid_vs_invalid_ruler): Update for
above change.
* region-model.cc
(check_one_function_attr_null_terminated_string_arg): Remove
unused variables "cd_unchecked", "strlen_sval", and
"limited_sval".
* region-model.h (region_model_context_decorator::warn): Add
missing "override".
---
 gcc/analyzer/access-diagram.cc |  9 +++--
 gcc/analyzer/region-model.cc   | 21 +
 gcc/analyzer/region-model.h|  2 +-
 3 files changed, 9 insertions(+), 23 deletions(-)

diff --git a/gcc/analyzer/access-diagram.cc b/gcc/analyzer/access-diagram.cc
index c7d190e3188..fb8c0282e75 100644
--- a/gcc/analyzer/access-diagram.cc
+++ b/gcc/analyzer/access-diagram.cc
@@ -919,11 +919,9 @@ class x_aligned_x_ruler_widget : public leaf_widget
 {
 public:
   x_aligned_x_ruler_widget (const access_diagram_impl &dia_impl,
-   const theme &theme,
-   table_dimension_sizes &col_widths)
+   const theme &theme)
   : m_dia_impl (dia_impl),
-m_theme (theme),
-m_col_widths (col_widths)
+m_theme (theme)
   {
   }
 
@@ -973,7 +971,6 @@ private:
 
   const access_diagram_impl &m_dia_impl;
   const theme &m_theme;
-  table_dimension_sizes &m_col_widths;
   std::vector m_labels;
 };
 
@@ -2361,7 +2358,7 @@ private:
 LOG_SCOPE (m_logger);
 
 x_aligned_x_ruler_widget *w
-  = new x_aligned_x_ruler_widget (*this, m_theme, *m_col_widths);
+  = new x_aligned_x_ruler_widget (*this, m_theme);
 
 access_range invalid_before_bits;
 if (m_op.maybe_get_invalid_before_bits (&invalid_before_bits))
diff --git a/gcc/analyzer/region-model.cc b/gcc/analyzer/region-model.cc
index 9479bcf380c..dc834406520 100644
--- a/gcc/analyzer/region-model.cc
+++ b/gcc/analyzer/region-model.cc
@@ -1877,23 +1877,13 @@ check_one_function_attr_null_terminated_string_arg 
(const gcall *call,
 || access->mode == access_read_write)
&& access->sizarg != UINT_MAX)
   {
-   /* First, check for a null-terminated string *without*
-  emitting warnings (via a null context), to get an
-  svalue for the strlen of the buffer (possibly
-  nullptr if there would be an issue).  */
-   call_details cd_unchecked (call, this, nullptr);
-   const svalue *strlen_sval
- = check_for_null_terminated_string_arg (cd_unchecked,
- arg_idx);
-
-   /* Get svalue for the size limit argument.  */
call_details cd_checked (call, this, ctxt);
const svalue *limit_sval
  = cd_checked.get_arg_svalue (access->sizarg);
const svalue *ptr_sval
  = cd_checked.get_arg_svalue (arg_idx);
/* Try reading all of the bytes expressed by the size param,
-  but without checking (via a null context).  */
+  but without emitting warnings (via a null context).  */
const svalue *limited_sval
  = read_bytes (deref_rvalue (ptr_sval, NULL_TREE, nullptr),
NULL_TREE,
@@ -1912,11 +1902,10 @@ check_one_function_attr_null_terminated_string_arg 
(const gcall *call,
  {
/* Reading up to the truncation limit seems OK; repeat
   the read, but with checking enabled.  */
-   const svalue *limited_sval
- = read_bytes (deref_rvalue (ptr_sval, NULL_TREE, ctxt),
-   NULL_TREE,
-   limit_sval,
-   ctxt);
+   read_bytes (deref_rvalue (ptr_sval, NULL_TREE, ctxt),
+   NULL_TREE,
+   limit_sval,
+   ctxt);
  }
return;
   }
diff --git a/gcc/analyzer/region-model.h b/gcc/analyzer/region-model.h
index 8bfb06880ff..4d8480df141 100644
--- a/gcc/analyzer/region-model.h
+++ b/gcc/analyzer/region-model.h
@@ -890,7 +890,7 @@ class region_model_context_decorator : public 
region_model_context
 {
  public:
   bool warn (std::unique_ptr d,
-const stmt_finder *custom_finder)
+const stmt_finder *custom_finder) override
   {
 if (m_inner)
   return m_inner->warn (std::move (d), custom_finder);
-- 
2.26.3

[PATCH/RFC 0/4] C/C++/diagnostics: various UX improvements

[David Malcolm]

The following patch kit implements the:
  #pragma GCC show_layout (struct foo)
idea I mentioned in my Cauldron talk (in patch 2),  and the other
patches implement various related user experience changes I came
across when implementing it.

Patch 1 reworks how c-pragma.cc parses pragmas, and experiments with
adding links to documentation to the diagnostics messages (on a
suitably capable terminal).

Patch 2 implements the new "show_layout" pragma

Patch 3 adds a new mechanism to the diagnostics subsytem for
automatically adding documentation links to messages, with enough
data to handle the pragmas from patch 1.

Patch 4 attempts to automatically populate the URL data for our docs by
parsing the results of "make html".

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.

I'd like to go ahead with patch 1 and patch 3; patch 2 and patch 4 may
need more work, but posting here for feedback.

Thoughts?

David Malcolm (4):
  c/c++: rework pragma parsing
  c: add #pragma GCC show_layout
  diagnostics: add automatic URL-ification within messages
  RFC: add contrib/regenerate-index-urls.py

 contrib/regenerate-index-urls.py  |  245 ++
 gcc/Makefile.in   |3 +-
 gcc/analyzer/record-layout.cc |  235 ++
 gcc/analyzer/record-layout.h  |4 +
 gcc/c-family/c-pragma.cc  |  641 -
 gcc/c-family/c-pragma.h   |5 +-
 gcc/diagnostic.cc |8 +-
 gcc/diagnostic.h  |4 +
 gcc/doc/extend.texi   |   49 +
 gcc/gcc-urlifier.cc   |  159 ++
 gcc/gcc-urlifier.def  | 2532 +
 gcc/gcc-urlifier.h|   26 +
 gcc/gcc.cc|2 +
 gcc/pretty-print-urlifier.h   |   33 +
 gcc/pretty-print.cc   |  242 +-
 gcc/pretty-print.h|5 +-
 gcc/selftest-run-tests.cc |1 +
 gcc/selftest.h|1 +
 gcc/stor-layout.h |3 +
 .../c-c++-common/pragma-message-parsing.c |   21 +
 .../c-c++-common/pragma-optimize-parsing.c|   16 +
 .../c-c++-common/pragma-pack-parsing-1.c  |   19 +
 .../c-c++-common/pragma-pack-parsing-2.c  |4 +
 .../pragma-redefine_extname-parsing.c |9 +
 .../c-c++-common/pragma-target-parsing.c  |   14 +
 .../c-c++-common/pragma-visibility-parsing.c  |   13 +
 .../c-c++-common/pragma-weak-parsing.c|   24 +
 gcc/testsuite/gcc.dg/bad-pragma-locations.c   |   22 +-
 .../gcc.dg/parsing-pragma-show_layout.c   |   15 +
 .../pragma-scalar_storate_order-parsing.c |8 +
 gcc/testsuite/gcc.dg/pragma-show_layout-1.c   |   12 +
 gcc/testsuite/gcc.dg/pragma-show_layout-2.c   |  184 ++
 ...agma-show_layout-infoleak-CVE-2017-18550.c |  175 ++
 gcc/testsuite/gcc.dg/sso-6.c  |2 +-
 gcc/toplev.cc |2 +
 35 files changed, 4589 insertions(+), 149 deletions(-)
 create mode 100755 contrib/regenerate-index-urls.py
 create mode 100644 gcc/gcc-urlifier.cc
 create mode 100644 gcc/gcc-urlifier.def
 create mode 100644 gcc/gcc-urlifier.h
 create mode 100644 gcc/pretty-print-urlifier.h
 create mode 100644 gcc/testsuite/c-c++-common/pragma-message-parsing.c
 create mode 100644 gcc/testsuite/c-c++-common/pragma-optimize-parsing.c
 create mode 100644 gcc/testsuite/c-c++-common/pragma-pack-parsing-1.c
 create mode 100644 gcc/testsuite/c-c++-common/pragma-pack-parsing-2.c
 create mode 100644 gcc/testsuite/c-c++-common/pragma-redefine_extname-parsing.c
 create mode 100644 gcc/testsuite/c-c++-common/pragma-target-parsing.c
 create mode 100644 gcc/testsuite/c-c++-common/pragma-visibility-parsing.c
 create mode 100644 gcc/testsuite/c-c++-common/pragma-weak-parsing.c
 create mode 100644 gcc/testsuite/gcc.dg/parsing-pragma-show_layout.c
 create mode 100644 gcc/testsuite/gcc.dg/pragma-scalar_storate_order-parsing.c
 create mode 100644 gcc/testsuite/gcc.dg/pragma-show_layout-1.c
 create mode 100644 gcc/testsuite/gcc.dg/pragma-show_layout-2.c
 create mode 100644 
gcc/testsuite/gcc.dg/pragma-show_layout-infoleak-CVE-2017-18550.c

-- 
2.26.3

[PATCH 3/4] diagnostics: add automatic URL-ification within messages

[David Malcolm]

  "ignoring malformed %<#pragma scalar_storage_order%>:"
+   " expected %, %, or %");
   }
 };
 
@@ -962,12 +948,8 @@ push_visibility (const char *str, int kind, const 
pragma_parser *p)
  " %, %, %"
  " or %"))
if (p)
- {
-   label_text doc_url (p->get_doc_url ());
-   inform (loc,
-   "ignoring malformed %<%{#pragma GCC visibility%}%>",
-   doc_url.get ());
- }
+ inform (loc,
+ "ignoring malformed %<#pragma GCC visibility%>");
   return;
 }
   visibility_options.inpragma = 1;
@@ -1014,11 +996,9 @@ handle_pragma_visibility (cpp_reader *)
 }
   if (bad == action)
 {
-  label_text doc_url (p.get_doc_url ());
   warning (OPT_Wpragmas,
-  "%<%{#pragma GCC visibility%}%> must be followed by % "
-  "or %",
-  doc_url.get ());
+  "%<#pragma GCC visibility%> must be followed by % "
+  "or %");
   return;
 }
   else
@@ -1027,11 +1007,8 @@ handle_pragma_visibility (cpp_reader *)
{
  if (! pop_visibility (0))
{
- label_text doc_url (p.get_doc_url ());
  warning (OPT_Wpragmas,
-  "no matching push for"
-  " %<%{#pragma GCC visibility pop%}%>",
-  doc_url.get ());
+  "no matching push for %<#pragma GCC visibility pop%>");
  return;
}
}
@@ -1511,7 +1488,7 @@ handle_pragma_push_options (cpp_reader *)
   token = pragma_lex (&x);
   if (token != CPP_EOF)
 {
-  warning (OPT_Wpragmas, "junk at end of %<#pragma push_options%>");
+  warning (OPT_Wpragmas, "junk at end of %<#pragma GCC push_options%>");
   return;
 }
 
@@ -1548,7 +1525,7 @@ handle_pragma_pop_options (cpp_reader *)
   token = pragma_lex (&x);
   if (token != CPP_EOF)
 {
-  warning (OPT_Wpragmas, "junk at end of %<#pragma pop_options%>");
+  warning (OPT_Wpragmas, "junk at end of %<#pragma GCC pop_options%>");
   return;
 }
 
@@ -1653,10 +1630,8 @@ handle_pragma_message (cpp_reader *)
 message = x;
   else
{
- label_text doc_url (p.get_doc_url ());
  warning_at (loc, OPT_Wpragmas,
- "expected a string after %<%{#pragma message%}%>",
- doc_url.get ());
+ "expected a string after %<#pragma message%>");
  return;
}
   if (!p.require_close_paren ())
@@ -1673,10 +1648,8 @@ handle_pragma_message (cpp_reader *)
 }
   else
 {
-  label_text doc_url (p.get_doc_url ());
   warning_at (loc, OPT_Wpragmas,
- "expected a string after %<%{#pragma message%}%>",
- doc_url.get ());
+ "expected a string after %<#pragma message%>");
   return;
 }
 
diff --git a/gcc/diagnostic.cc b/gcc/diagnostic.cc
index 0f392358aef..e3c440a9e8d 100644
--- a/gcc/diagnostic.cc
+++ b/gcc/diagnostic.cc
@@ -44,6 +44,7 @@ along with GCC; see the file COPYING3.  If not see
 #include "opts.h"
 #include "cpplib.h"
 #include "text-art/theme.h"
+#include "pretty-print-urlifier.h"
 
 #ifdef HAVE_TERMIOS_H
 # include 
@@ -193,6 +194,7 @@ diagnostic_initialize (diagnostic_context *context, int 
n_opts)
   context->option_state = NULL;
   context->option_name = NULL;
   context->get_option_url = NULL;
+  context->m_urlifier = nullptr;
   context->last_location = UNKNOWN_LOCATION;
   context->last_module = 0;
   context->x_data = NULL;
@@ -347,6 +349,9 @@ diagnostic_finish (diagnostic_context *context)
   delete context->m_client_data_hooks;
   context->m_client_data_hooks = NULL;
 }
+
+  delete context->m_urlifier;
+  context->m_urlifier = nullptr;
 }
 
 /* Initialize DIAGNOSTIC, where the message MSG has already been
@@ -1574,7 +1579,8 @@ diagnostic_report_diagnostic (diagnostic_context *context,
 context->m_output_format->on_begin_group ();
   context->diagnostic_group_emission_count++;
 
-  pp_format (context->printer, &diagnostic->message);
+  pp_format (context->printer, &diagnostic->message,
+context->m_urlifier);
   context->m_output_format->on_begin_diagnostic (diagnostic);
   pp_output_formatted_text (context->printer);
   if (context->show_cwe)
diff --git a/gcc/diagnostic.h b/gcc/diagnostic.h
index a2c8740cbd0..d79369289bc 100644
--- a/gcc/diagnostic.h
+++ b/gcc/diagnostic.h
@@ -354,6 +354,10 @@ struct diagnostic_context
  par

[PATCH 2/4] c: add #pragma GCC show_layout

[David Malcolm]

This patch adds a new pragma to the C frontend that will
make it emit a human-readable diagram of a struct's layout.

For example, given this contrived usage:

struct example {
  char foo : 7;
  char bar;
  char visible : 1;
  char active  : 1;
};

the compiler will emit output similar to the following:

note: 'sizeof(struct example)' == 3; layout:

  
┌───┬┬───┬─┬─┬───┐
  │Offsets│Byte│   0   │  1  │2
│   3   │
  
├───┼┼─┬─┬─┬─┬─┬─┬─┬─┼─┬─┬──┬──┬──┬──┬──┬──┼───┬───┬──┬──┬──┬──┬──┬──┼──┬──┬──┬──┬──┬──┬──┬──┤
  │ Byte  │Bit │0│1│2│3│4│5│6│7│8│9│10│11│12│13│14│15│16 │17 
│18│19│20│21│22│23│24│25│26│27│28│29│30│31│
  
├───┼┼─┴─┴─┴─┴─┴─┴─┼─┼─┴─┴──┴──┴──┴──┴──┴──┼───┼───┼──┴──┴──┴──┴──┴──┼──┴──┴──┴──┴──┴──┴──┴──┘
  │   0   │ 0  │'foo'│*│'bar'│(1)│(2)│ padding │
  └───┴┴─┴─┴─┴───┴───┴─┘
  *: padding
  (1): 'visible'
  (2): 'active'

The output is intended for humans, rather than scripts, and is
subject to change.

One wart is that it uses some analyzer internals, and thus requires
GCC to have been configured without disabling the analyzer.

Caveat: only tested on x86_64, and probably has some endianness and
packing assumptions in the testcases.

Thoughts?

gcc/analyzer/ChangeLog:
* record-layout.cc: Define INCLUDE_ALGORITHM and
INCLUDE_VECTOR.  Include "intl.h", "text-art/table.h",
"text-art/widget.h", and "diagnostic-diagram.h".
(class layout_diagram): New.
(layout_diagram::layout_diagram): New.
(layout_diagram::bit_to_table_coord): New.
(layout_diagram::ensure_table_rows): New.
(layout_diagram::get_string_for_item): New.
(impl_show_record_layout): New.
(show_record_layout): New.
* record-layout.h (class layout_diagram): New forward decl.
(class record_layout): Add friend class layout_diagram.

gcc/c-family/ChangeLog:
* c-pragma.cc: Include "stor-layout.h".
(class pragma_parser_show_layout): New.
(handle_pragma_show_layout): New.
(init_pragma): Register it.

gcc/ChangeLog:
* doc/extend.texi (Other Pragmas): New subsection,
with '#pragma GCC show_layout'.
* stor-layout.h (show_record_layout): New decl.

gcc/testsuite/ChangeLog:
* gcc.dg/parsing-pragma-show_layout.c: New test.
* gcc.dg/pragma-show_layout-1.c: New test.
* gcc.dg/pragma-show_layout-2.c: New test.
* gcc.dg/pragma-show_layout-infoleak-CVE-2017-18550.c: New test.
---
 gcc/analyzer/record-layout.cc | 235 ++
 gcc/analyzer/record-layout.h  |   4 +
 gcc/c-family/c-pragma.cc  |  95 +++
 gcc/doc/extend.texi   |  49 
 gcc/stor-layout.h |   3 +
 .../gcc.dg/parsing-pragma-show_layout.c   |  15 ++
 gcc/testsuite/gcc.dg/pragma-show_layout-1.c   |  12 +
 gcc/testsuite/gcc.dg/pragma-show_layout-2.c   | 184 ++
 ...agma-show_layout-infoleak-CVE-2017-18550.c | 175 +
 9 files changed, 772 insertions(+)
 create mode 100644 gcc/testsuite/gcc.dg/parsing-pragma-show_layout.c
 create mode 100644 gcc/testsuite/gcc.dg/pragma-show_layout-1.c
 create mode 100644 gcc/testsuite/gcc.dg/pragma-show_layout-2.c
 create mode 100644 
gcc/testsuite/gcc.dg/pragma-show_layout-infoleak-CVE-2017-18550.c

diff --git a/gcc/analyzer/record-layout.cc b/gcc/analyzer/record-layout.cc
index 1369bfb5eff..242a9895309 100644
--- a/gcc/analyzer/record-layout.cc
+++ b/gcc/analyzer/record-layout.cc
@@ -19,7 +19,9 @@ along with GCC; see the file COPYING3.  If not see
 .  */
 
 #include "config.h"
+#define INCLUDE_ALGORITHM
 #define INCLUDE_MEMORY
+#define INCLUDE_VECTOR
 #include "system.h"
 #include "coretypes.h"
 #include "tree.h"
@@ -28,8 +30,13 @@ along with GCC; see the file COPYING3.  If not see
 #include "gimple.h"
 #include "diagnostic.h"
 #include "tree-diagnostic.h"
+#include "intl.h"
+#include "make-unique.h"
 #include "analyzer/analyzer.h"
 #include "analyzer/record-layout.h"
+#include "text-art/table.h"
+#include "text-art/widget.h"
+#include "diagnostic-diagram.h"
 
 #if ENABLE_ANALYZER
 
@@ -120,6 +127,234 @@ record_layout::maybe_pad_to (bit_offset_t next_offset)
 }
 }
 
+class layout_diagram : public text_art::vbox_widget
+{
+public:
+  layout_diagram (const ana::record_layout &layout,
+ text_art::style_manager &sm,
+ const text_art::theme &theme);
+
+private:
+  text_art::table::coord_t bit_to_table_coord (ana::bit_offset_t bit);
+
+  void ensure_table_rows (text_art::style_manager &sm,
+ text_art::table &table,
+ int table_y);
+
+  text_art::styled_string
+  get_string_for_item (const ana::rec

[PATCH 1/4] c/c++: rework pragma parsing

[David Malcolm]

This patch reworks pragma parsing in c-pragma.cc, with the
following improvements:

- it replaces the GCC_BAD* macros (that contained "return") in favor
of helper classes and functions for emitting diagnostics, making control
flow more explicit

- the -Wpragmas diagnostics are reworded from the form e.g.:
  DESCRIPTION OF PROBLEM; ignored
to:
  ignoring malformed '#pragma FOO': DESCRIPTION OF PROBLEM

- the locations of the warnings are fixed to more accurately
reflect the location of the problem

- the names of the pragmas are URLified into links to the
documentation for the pragma.  For example, in:

  warning: ignoring malformed '#pragma weak': expected name [-Wpragmas]

in a suitable terminal, the "#pragma weak" within quotes is a link
to https://gcc.gnu.org/onlinedocs/gcc/Weak-Pragmas.html; similarly with

  warning: '#pragma pack' has no effect with '-fpack-struct' - ignored 
[-Wpragmas]

the "#pragma pack" text is linkified to
  https://gcc.gnu.org/onlinedocs/gcc/Structure-Layout-Pragmas.html
and the "-fpack-struct" text is linkified to:
  https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html#index-fpack-struct

I have a more general and maintainable approach to adding URLs to
diagnostics which is in a followup.

gcc/c-family/ChangeLog:
* c-pragma.cc (GCC_BAD): Delete.
(GCC_BAD2): Delete.
(GCC_BAD_AT): Delete.
(GCC_BAD2_AT): Delete.
(get_doc_url): New.
(class pragma_parser): New.
(handle_pragma_pack): Delete redundant forward decl.
(pop_alignment): Add param "p" and use it to get doc urls.
(enum class pack_action): Move here from within
handle_pragma_pack.
(class pragma_pack_parser): New.
(handle_pragma_pack): Rewrite using pragma_pack_parser
and enum class pack_action, eliminating uses of GCC_BAD*,
rewording diagnostics.
(handle_pragma_weak): Rewrite using pragma_parser, eliminating
uses of GCC_BAD*, rewording diagnostics.
(class pragma_scalar_storage_order_parser): New.
(handle_pragma_scalar_storage_order): Rewrite using above,
eliminating uses of GCC_BAD*, rewording diagnostics.
(handle_pragma_redefine_extname): Rewrite using pragma_parser,
eliminating uses of GCC_BAD*, rewording diagnostics.  Fix overlong
line.
(handle_pragma_visibility): Remove redundant forward decl.
(push_visibility): Add "const pragma_parser *" param.  Rewrite to
eliminate uses of GCC_BAD*.  Add note that warning was ignored.
(handle_pragma_visibility): Rewrite using pragma_parser,
eliminating uses of GCC_BAD*, rewording diagnostics.
(handle_pragma_target): Fix name of pragma in "error".  Eliminate
uses of GCC_BAD*.
(handle_pragma_optimize): Eliminate uses of GCC_BAD.
(handle_pragma_message): Rewrite using pragma_parser, eliminating
uses of GCC_BAD*, rewording diagnostics.
* c-pragma.h (class pragma_parser): New forward decl.
(push_visibility): Add optional "const pragma_parser *" param.

gcc/testsuite/ChangeLog:
* c-c++-common/pragma-message-parsing.c: New test.
* c-c++-common/pragma-optimize-parsing.c: New test.
* c-c++-common/pragma-pack-parsing-1.c: New test.
* c-c++-common/pragma-pack-parsing-2.c: New test.
* c-c++-common/pragma-redefine_extname-parsing.c: New test.
* c-c++-common/pragma-target-parsing.c: New test.
* c-c++-common/pragma-visibility-parsing.c: New test.
* c-c++-common/pragma-weak-parsing.c: New test.
* gcc.dg/bad-pragma-locations.c: Update for changes to wording and
location of -Wpragmas.
* gcc.dg/pragma-scalar_storate_order-parsing.c: New test.
* gcc.dg/sso-6.c: Update for changes to wording of -Wpragmas.
---
 gcc/c-family/c-pragma.cc  | 569 ++
 gcc/c-family/c-pragma.h   |   5 +-
 .../c-c++-common/pragma-message-parsing.c |  21 +
 .../c-c++-common/pragma-optimize-parsing.c|  16 +
 .../c-c++-common/pragma-pack-parsing-1.c  |  19 +
 .../c-c++-common/pragma-pack-parsing-2.c  |   4 +
 .../pragma-redefine_extname-parsing.c |   9 +
 .../c-c++-common/pragma-target-parsing.c  |  14 +
 .../c-c++-common/pragma-visibility-parsing.c  |  13 +
 .../c-c++-common/pragma-weak-parsing.c|  24 +
 gcc/testsuite/gcc.dg/bad-pragma-locations.c   |  22 +-
 .../pragma-scalar_storate_order-parsing.c |   8 +
 gcc/testsuite/gcc.dg/sso-6.c  |   2 +-
 13 files changed, 588 insertions(+), 138 deletions(-)
 create mode 100644 gcc/testsuite/c-c++-common/pragma-message-parsing.c
 create mode 100644 gcc/testsuite/c-c++-common/pragma-optimize-parsing.c
 create mode 100644 gcc/testsuite/c-c++-common/pragma-pack-parsing-1.c
 create mode 100644 gcc/testsuite/c-c++-common/pragma-pack-parsing-2.c
 create mode 100644 gcc/testsuite/c-c++-common/pragma-redefine_extname-parsi

[pushed] diagnostics: consolidate group-handling fields in diagnostic_context

[David Malcolm]

No functional change intended.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-5112-gae8abcb81ed814.

gcc/ChangeLog:
* diagnostic.cc (diagnostic_initialize): Update for consolidation
of group-based fields.
(diagnostic_report_diagnostic): Likewise.
(diagnostic_context::begin_group): New, based on body of
auto_diagnostic_group's ctor.
(diagnostic_context::end_group): New, based on body of
auto_diagnostic_group's dtor.
(auto_diagnostic_group::auto_diagnostic_group): Convert to a call
to begin_group.
(auto_diagnostic_group::~auto_diagnostic_group): Convert to a call
to end_group.
* diagnostic.h (diagnostic_context::begin_group): New decl.
(diagnostic_context::end_group): New decl.
(diagnostic_context::diagnostic_group_nesting_depth): Rename to...
(diagnostic_context::m_diagnostic_groups.m_nesting_depth):
...this.
(diagnostic_context::diagnostic_group_emission_count): Rename
to...
(diagnostic_context::m_diagnostic_groups::m_emission_count):
...this.
---
 gcc/diagnostic.cc | 42 --
 gcc/diagnostic.h  | 19 ++-
 2 files changed, 42 insertions(+), 19 deletions(-)

diff --git a/gcc/diagnostic.cc b/gcc/diagnostic.cc
index 0f392358aef..0759fae916e 100644
--- a/gcc/diagnostic.cc
+++ b/gcc/diagnostic.cc
@@ -218,8 +218,8 @@ diagnostic_initialize (diagnostic_context *context, int 
n_opts)
   context->tabstop = 8;
   context->escape_format = DIAGNOSTICS_ESCAPE_FORMAT_UNICODE;
   context->edit_context_ptr = NULL;
-  context->diagnostic_group_nesting_depth = 0;
-  context->diagnostic_group_emission_count = 0;
+  context->m_diagnostic_groups.m_nesting_depth = 0;
+  context->m_diagnostic_groups.m_emission_count = 0;
   context->m_output_format = new diagnostic_text_output_format (*context);
   context->set_locations_cb = nullptr;
   context->ice_handler_cb = NULL;
@@ -1570,9 +1570,9 @@ diagnostic_report_diagnostic (diagnostic_context *context,
 ++diagnostic_kind_count (context, diagnostic->kind);
 
   /* Is this the initial diagnostic within the stack of groups?  */
-  if (context->diagnostic_group_emission_count == 0)
+  if (context->m_diagnostic_groups.m_emission_count == 0)
 context->m_output_format->on_begin_group ();
-  context->diagnostic_group_emission_count++;
+  context->m_diagnostic_groups.m_emission_count++;
 
   pp_format (context->printer, &diagnostic->message);
   context->m_output_format->on_begin_diagnostic (diagnostic);
@@ -2296,28 +2296,42 @@ fancy_abort (const char *file, int line, const char 
*function)
   internal_error ("in %s, at %s:%d", function, trim_filename (file), line);
 }
 
+/* struct diagnostic_context.  */
+
+void
+diagnostic_context::begin_group ()
+{
+  m_diagnostic_groups.m_nesting_depth++;
+}
+
+void
+diagnostic_context::end_group ()
+{
+  if (--m_diagnostic_groups.m_nesting_depth == 0)
+{
+  /* Handle the case where we've popped the final diagnostic group.
+If any diagnostics were emitted, give the context a chance
+to do something.  */
+  if (m_diagnostic_groups.m_emission_count > 0)
+   m_output_format->on_end_group ();
+  m_diagnostic_groups.m_emission_count = 0;
+}
+}
+
 /* class auto_diagnostic_group.  */
 
 /* Constructor: "push" this group into global_dc.  */
 
 auto_diagnostic_group::auto_diagnostic_group ()
 {
-  global_dc->diagnostic_group_nesting_depth++;
+  global_dc->begin_group ();
 }
 
 /* Destructor: "pop" this group from global_dc.  */
 
 auto_diagnostic_group::~auto_diagnostic_group ()
 {
-  if (--global_dc->diagnostic_group_nesting_depth == 0)
-{
-  /* Handle the case where we've popped the final diagnostic group.
-If any diagnostics were emitted, give the context a chance
-to do something.  */
-  if (global_dc->diagnostic_group_emission_count > 0)
-   global_dc->m_output_format->on_end_group ();
-  global_dc->diagnostic_group_emission_count = 0;
-}
+  global_dc->end_group ();
 }
 
 /* class diagnostic_text_output_format : public diagnostic_output_format.  */
diff --git a/gcc/diagnostic.h b/gcc/diagnostic.h
index a2c8740cbd0..ed1b6c0c7b1 100644
--- a/gcc/diagnostic.h
+++ b/gcc/diagnostic.h
@@ -226,6 +226,12 @@ public:
the context of a diagnostic message.  */
 struct diagnostic_context
 {
+public:
+  void begin_group ();
+  void end_group ();
+
+public:
+
   /* Where most of the diagnostic formatting work is done.  */
   pretty_printer *printer;
 
@@ -443,12 +449,15 @@ struct diagnostic_context
  applied, for generating patches.  */
   edit_context *edit_context_ptr;
 
-  /* How many diagnostic_group instances are currently alive.  */
-  int diagnostic_group_nesting_depth;
+  /* Fields relating to diagnostic groups.  */
+  struct {
+/* How many diagnostic_group instances are currently alive.  */
+int m_nesting_depth;
 
- 

[pushed] diagnostics: convert diagnostic_context to a class

[David Malcolm]

This patch:
- converts "struct diagnostic_context" to "class diagnostic_context".
- ensures all data members have an "m_" prefix, except for "printer",
  which has so many uses that renaming would be painful.
- makes most of the data members private
- converts much of the diagnostic_* functions to member functions of
  diagnostic_context, adding compatibility wrappers for users such as
  the Fortran frontend, and making as many as possible private.

No functional change intended.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-5117-g8200cd97c9c57d.

gcc/ChangeLog:
* common.opt (fdiagnostics-text-art-charset=): Remove refererence
to diagnostic-text-art.h.
* coretypes.h (struct diagnostic_context): Replace forward decl
with...
(class diagnostic_context): ...this.
* diagnostic-format-json.cc: Update for changes to
diagnostic_context.
* diagnostic-format-sarif.cc: Likewise.
* diagnostic-show-locus.cc: Likewise.
* diagnostic-text-art.h: Deleted file, moving content...
(enum diagnostic_text_art_charset): ...to diagnostic.h,
(DIAGNOSTICS_TEXT_ART_CHARSET_DEFAULT): ...deleting,
(diagnostics_text_art_charset_init): ...deleting in favor of
diagnostic_context::set_text_art_charset.
* diagnostic.cc: Remove include of "diagnostic-text-art.h".
(pedantic_warning_kind): Update for field renaming.
(permissive_error_kind): Likewise.
(permissive_error_option): Likewise.
(diagnostic_initialize): Convert to...
(diagnostic_context::initialize): ...this, updating for field
renamings.
(diagnostic_color_init): Convert to...
(diagnostic_context::color_init): ...this.
(diagnostic_urls_init): Convert to...
(diagnostic_context::urls_init): ...this.
(diagnostic_initialize_input_context): Convert to...
(diagnostic_context::initialize_input_context): ...this.
(diagnostic_finish): Convert to...
(diagnostic_context::finish): ...this, updating for field
renamings.
(diagnostic_context::set_output_format): New.
(diagnostic_context::set_client_data_hooks): New.
(diagnostic_context::create_edit_context): New.
(diagnostic_converted_column): Convert to...
(diagnostic_context::converted_column): ...this.
(diagnostic_get_location_text): Update for field renaming.
(diagnostic_check_max_errors): Convert to...
(diagnostic_context::check_max_errors): ...this, updating for
field renamings.
(diagnostic_action_after_output): Convert to...
(diagnostic_context::action_after_output): ...this, updating for
field renamings.
(last_module_changed_p): Delete.
(set_last_module): Delete.
(includes_seen): Convert to...
(diagnostic_context::includes_seen_p): ...this, updating for field
renamings.
(diagnostic_report_current_module): Convert to...
(diagnostic_context::report_current_module): ...this, updating for
field renamings, and replacing uses of last_module_changed_p and
set_last_module to simple field accesses.
(diagnostic_show_any_path): Convert to...
(diagnostic_context::show_any_path): ...this.
(diagnostic_classify_diagnostic): Convert to...
(diagnostic_context::classify_diagnostic): ...this, updating for
field renamings.
(diagnostic_push_diagnostics): Convert to...
(diagnostic_context::push_diagnostics): ...this, updating for field
renamings.
(diagnostic_pop_diagnostics): Convert to...
(diagnostic_context::pop_diagnostics): ...this, updating for field
renamings.
(get_any_inlining_info): Convert to...
(diagnostic_context::get_any_inlining_info): ...this, updating for
field renamings.
(update_effective_level_from_pragmas): Convert to...
(diagnostic_context::update_effective_level_from_pragmas):
...this, updating for field renamings.
(print_any_cwe): Convert to...
(diagnostic_context::print_any_cwe): ...this.
(print_any_rules): Convert to...
(diagnostic_context::print_any_rules): ...this.
(print_option_information): Convert to...
(diagnostic_context::print_option_information): ...this, updating
for field renamings.
(diagnostic_enabled): Convert to...
(diagnostic_context::diagnostic_enabled): ...this, updating for
field renamings.
(warning_enabled_at): Convert to...
(diagnostic_context::warning_enabled_at): ...this.
(diagnostic_report_diagnostic): Convert to...
(diagnostic_context::report_diagnostic): ...this, updating for
field renamings and conversions to member functions.
(diagnostic_append_note): Update for field renaming.
(diagnostic_impl): Use diagnos

[pushed] diagnostics: add automatic URL-ification within messages

[David Malcolm]

On Thu, 2023-11-02 at 09:19 -0400, David Malcolm wrote:

[...snip...]

I eliminated the dependency on the c-pragma.cc changes from this patch,
updated it for diagnostic_context becoming a class, and pushed it to
trunk as r14-5118-gc5db4d8ba5f3de (after re-testing it).

For reference, here's what I pushed:

In r10-3781-gd26082357676a3 GCC's pretty-print framework gained
the ability to emit embedding URLs via escape sequences
for marking up text output..

In r10-3783-gb4c7ca2ef3915a GCC started using this for the
[-Wname-of-option] emitted at the end of each diagnostic so that it
becomes a hyperlink to the documentation for that option on the GCC
website.

This makes it much more convenient for the user to locate pertinent
documentation when a diagnostic is emitted.

The above involved special-casing in one specific place, but there is
plenty of quoted text throughout GCC's diagnostic messages that could
usefully have a documentation URL: references to options, pragmas, etc

This patch adds a new optional "urlifier" parameter to pp_format.
The idea is that a urlifier object has responsibility for mapping from
quoted strings in diagnostic messages to URLs, and pp_format has the
ability to automatically add URL escapes for strings that the urlifier
gives it URLs for.

For example, given the format string:

  "%<#pragma pack%> has no effect with %<-fpack-struct%>"

with this patch GCC is able to automatically linkify the "#pragma pack"
text to
  https://gcc.gnu.org/onlinedocs/gcc/Structure-Layout-Pragmas.html
and the "-fpack-struct" text to:
  https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html#index-fpack-struct

and we don't have to modify the format string itself.

This is only done for the pp_format within
diagnostic_context::report_diagnostic i.e. just for the primary message
in each diagnostics, and not for other places within GCC that use pp
format internally.

"urlifier" is an abstract base class, with a GCC-specific subclass
implementing the logic for generating URLs into GCC's HTML
documentation via binary search in a data table.  This patch implements
the gcc_urlifier with a small table generated by hand; the data table in
this patch only covers various pragmas and the option referenced by the
above pragma message.

I have a followup patch that scripts the creation of this data by
directly scraping the output of "make html", thus automating all this,
and (I hope) minimizing the work of ensuring that documentation URLs
emitted by GCC match the generated documentation.

gcc/ChangeLog:
* Makefile.in (GCC_OBJS): Add gcc-urlifier.o.
(OBJS): Likewise.

gcc/c-family/ChangeLog:
* c-pragma.cc:: (handle_pragma_push_options): Fix missing "GCC" in
name of pragma in "junk" message.
(handle_pragma_pop_options): Likewise.

gcc/ChangeLog:
* diagnostic.cc: Include "pretty-print-urlifier.h".
(diagnostic_context::initialize): Initialize m_urlifier.
(diagnostic_context::finish): Clean up m_urlifier
(diagnostic_report::diagnostic): m_urlifier to pp_format.
* diagnostic.h (diagnostic_context::m_urlifier): New field.
* gcc-urlifier.cc: New file.
* gcc-urlifier.def: New file.
* gcc-urlifier.h: New file.
* gcc.cc: Include "gcc-urlifier.h".
(driver::global_initializations): Initialize global_dc->m_urlifier.
* pretty-print-urlifier.h: New file.
* pretty-print.cc: Include "pretty-print-urlifier.h".
(obstack_append_string): New.
(urlify_quoted_string): New.
(pp_format): Add "urlifier" param and use it to implement optional
urlification of quoted text strings.
(pp_output_formatted_text): Make buffer a const pointer.
(selftest::pp_printf_with_urlifier): New.
(selftest::test_urlification): New.
(selftest::pretty_print_cc_tests): Call it.
* pretty-print.h (class urlifier): New forward declaration.
(pp_format): Add optional urlifier param.
* selftest-run-tests.cc (selftest::run_tests): Call
selftest::gcc_urlifier_cc_tests .
* selftest.h (selftest::gcc_urlifier_cc_tests): New decl.
* toplev.cc: Include "gcc-urlifier.h".
(general_init): Initialize global_dc->m_urlifier.
---
 gcc/Makefile.in |   3 +-
 gcc/c-family/c-pragma.cc|   4 +-
 gcc/diagnostic.cc   |   7 +-
 gcc/diagnostic.h|   4 +
 gcc/gcc-urlifier.cc | 159 +++
 gcc/gcc-urlifier.def|  20 +++
 gcc/gcc-urlifier.h  |  26 
 gcc/gcc.cc  |   2 +
 gcc/pretty-print-urlifier.h |  33 +
 gcc/pretty-print.cc | 242 +++-
 gcc/pretty-print.h  |   5 +-
 gcc/selftest-run-tests.cc   |   1 +
 gcc/selftest.h   

Re: [PATCH] diagnostics: fix gcc-urlifier.cc bootstrap failure [PR112379]

[David Malcolm]

On Sat, 2023-11-04 at 16:23 +, Sergei Trofimovich wrote:
> From: Sergei Trofimovich 
> 
> Without the change `./configure --enable-checking=release` bootstrap
> fails as:
> 
>     gcc/gcc-urlifier.cc:100:1: error:
> 'get_url_suffix_for_quoted_text()'
>     defined but not used [-Werror=unused-function]
> 
> This happens because the helper is used only in `ASSERT` macros which
> don't always get expanded to executable code.
> 
> The fix marks helper function with `ATTRIBUTE_UNUSED`.

Sorry for the breakage.

Patch looks good to me; thanks
Dave

> 
> gcc/
> PR bootstrap/112379
> * gcc-urlifier.cc (get_url_suffix_for_quoted_text): Mark as
> ATTRIBUTE_UNUSED.
> ---
>  gcc/gcc-urlifier.cc | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/gcc/gcc-urlifier.cc b/gcc/gcc-urlifier.cc
> index 269246bc703..0dbff985313 100644
> --- a/gcc/gcc-urlifier.cc
> +++ b/gcc/gcc-urlifier.cc
> @@ -37,7 +37,8 @@ public:
>    char *get_url_for_quoted_text (const char *p, size_t sz) const
> final override;
>  
>    const char *get_url_suffix_for_quoted_text (const char *p, size_t
> sz) const;
> -  const char *get_url_suffix_for_quoted_text (const char *p) const;
> +  /* We use ATTRIBUTE_UNUSED as this helper is called only from
> ASSERTs.  */
> +  const char *get_url_suffix_for_quoted_text (const char *p) const
> ATTRIBUTE_UNUSED;
>  
>  private:
>    static char *

[pushed 2/4] diagnostics: make diagnostic_context::m_urlifier private

[David Malcolm]

No functional change intended.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-5167-ga526cc6ff32e22.

gcc/ChangeLog:
* diagnostic.cc (diagnostic_context::set_urlifier): New.
* diagnostic.h (diagnostic_context::set_urlifier): New decl.
(diagnostic_context::m_urlifier): Make private.
* gcc.cc (driver::global_initializations): Use set_urlifier rather
than directly setting field.
* toplev.cc (general_init): Likewise.
---
 gcc/diagnostic.cc | 8 
 gcc/diagnostic.h  | 3 +++
 gcc/gcc.cc| 2 +-
 gcc/toplev.cc | 2 +-
 4 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/gcc/diagnostic.cc b/gcc/diagnostic.cc
index 90103e150f7..c617b34f02b 100644
--- a/gcc/diagnostic.cc
+++ b/gcc/diagnostic.cc
@@ -373,6 +373,14 @@ diagnostic_context::set_client_data_hooks 
(diagnostic_client_data_hooks *hooks)
   m_client_data_hooks = hooks;
 }
 
+void
+diagnostic_context::set_urlifier (urlifier *urlifier)
+{
+  /* Ideally we'd use a std::unique_ptr here.  */
+  delete m_urlifier;
+  m_urlifier = urlifier;
+}
+
 void
 diagnostic_context::create_edit_context ()
 {
diff --git a/gcc/diagnostic.h b/gcc/diagnostic.h
index 4ef031b5d1c..f9950ec2cf8 100644
--- a/gcc/diagnostic.h
+++ b/gcc/diagnostic.h
@@ -297,6 +297,7 @@ public:
   void set_output_format (diagnostic_output_format *output_format);
   void set_text_art_charset (enum diagnostic_text_art_charset charset);
   void set_client_data_hooks (diagnostic_client_data_hooks *hooks);
+  void set_urlifier (urlifier *);
   void create_edit_context ();
   void set_warning_as_error_requested (bool val)
   {
@@ -518,10 +519,12 @@ public:
  particular option.  */
   char *(*m_get_option_url) (diagnostic_context *, int);
 
+private:
   /* An optional hook for adding URLs to quoted text strings in
  diagnostics.  Only used for the main diagnostic message.  */
   urlifier *m_urlifier;
 
+public:
   void (*m_print_path) (diagnostic_context *, const diagnostic_path *);
   json::value *(*m_make_json_for_path) (diagnostic_context *,
const diagnostic_path *);
diff --git a/gcc/gcc.cc b/gcc/gcc.cc
index 02464958f36..51120c1489e 100644
--- a/gcc/gcc.cc
+++ b/gcc/gcc.cc
@@ -8292,7 +8292,7 @@ driver::global_initializations ()
   diagnostic_initialize (global_dc, 0);
   diagnostic_color_init (global_dc);
   diagnostic_urls_init (global_dc);
-  global_dc->m_urlifier = make_gcc_urlifier ();
+  global_dc->set_urlifier (make_gcc_urlifier ());
 
 #ifdef GCC_DRIVER_HOST_INITIALIZATION
   /* Perform host dependent initialization when needed.  */
diff --git a/gcc/toplev.cc b/gcc/toplev.cc
index e39162a3e49..d8e8978dd55 100644
--- a/gcc/toplev.cc
+++ b/gcc/toplev.cc
@@ -1049,7 +1049,7 @@ general_init (const char *argv0, bool init_signals)
   global_dc->m_option_state = &global_options;
   global_dc->m_option_name = option_name;
   global_dc->m_get_option_url = get_option_url;
-  global_dc->m_urlifier = make_gcc_urlifier ();
+  global_dc->set_urlifier (make_gcc_urlifier ());
 
   if (init_signals)
 {
-- 
2.26.3

[pushed 1/4] diagnostics: eliminate diagnostic_kind_count

[David Malcolm]

No functional change intended.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-5166-g579bb65cdd35a4.

gcc/ChangeLog:
* diagnostic.cc (diagnostic_context::check_max_errors): Replace
uses of diagnostic_kind_count with simple field acesss.
(diagnostic_context::report_diagnostic): Likewise.
(diagnostic_text_output_format::~diagnostic_text_output_format):
Replace use of diagnostic_kind_count with
diagnostic_context::diagnostic_count.
* diagnostic.h (diagnostic_kind_count): Delete.
(errorcount): Replace use of diagnostic_kind_count with
diagnostic_context::diagnostic_count.
(warningcount): Likewise.
(werrorcount): Likewise.
(sorrycount): Likewise.
---
 gcc/diagnostic.cc | 16 
 gcc/diagnostic.h  | 17 -
 2 files changed, 12 insertions(+), 21 deletions(-)

diff --git a/gcc/diagnostic.cc b/gcc/diagnostic.cc
index e917e6ce4ac..90103e150f7 100644
--- a/gcc/diagnostic.cc
+++ b/gcc/diagnostic.cc
@@ -651,9 +651,9 @@ diagnostic_context::check_max_errors (bool flush)
   if (!m_max_errors)
 return;
 
-  int count = (diagnostic_kind_count (this, DK_ERROR)
-  + diagnostic_kind_count (this, DK_SORRY)
-  + diagnostic_kind_count (this, DK_WERROR));
+  int count = (m_diagnostic_count[DK_ERROR]
+  + m_diagnostic_count[DK_SORRY]
+  + m_diagnostic_count[DK_WERROR]);
 
   if (count >= m_max_errors)
 {
@@ -1547,8 +1547,8 @@ diagnostic_context::report_diagnostic (diagnostic_info 
*diagnostic)
 error has already occurred.  This is counteracted by
 abort_on_error.  */
   if (!CHECKING_P
- && (diagnostic_kind_count (this, DK_ERROR) > 0
- || diagnostic_kind_count (this, DK_SORRY) > 0)
+ && (m_diagnostic_count[DK_ERROR] > 0
+ || m_diagnostic_count[DK_SORRY] > 0)
  && !m_abort_on_error)
{
  expanded_location s 
@@ -1563,9 +1563,9 @@ diagnostic_context::report_diagnostic (diagnostic_info 
*diagnostic)
 diagnostic->message.m_args_ptr);
 }
   if (diagnostic->kind == DK_ERROR && orig_diag_kind == DK_WARNING)
-++diagnostic_kind_count (this, DK_WERROR);
+++m_diagnostic_count[DK_WERROR];
   else
-++diagnostic_kind_count (this, diagnostic->kind);
+++m_diagnostic_count[diagnostic->kind];
 
   /* Is this the initial diagnostic within the stack of groups?  */
   if (m_diagnostic_groups.m_emission_count == 0)
@@ -2336,7 +2336,7 @@ auto_diagnostic_group::~auto_diagnostic_group ()
 diagnostic_text_output_format::~diagnostic_text_output_format ()
 {
   /* Some of the errors may actually have been warnings.  */
-  if (diagnostic_kind_count (&m_context, DK_WERROR))
+  if (m_context.diagnostic_count (DK_WERROR))
 {
   /* -Werror was given.  */
   if (m_context.warning_as_error_requested_p ())
diff --git a/gcc/diagnostic.h b/gcc/diagnostic.h
index cf21558c7b2..4ef031b5d1c 100644
--- a/gcc/diagnostic.h
+++ b/gcc/diagnostic.h
@@ -701,24 +701,15 @@ extern diagnostic_context *global_dc;
ready for use.  */
 #define diagnostic_ready_p() (global_dc->printer != NULL)
 
-/* The total count of a KIND of diagnostics emitted so far.  */
-
-inline int &
-diagnostic_kind_count (diagnostic_context *context,
-  diagnostic_t kind)
-{
-  return context->diagnostic_count (kind);
-}
-
 /* The number of errors that have been issued so far.  Ideally, these
would take a diagnostic_context as an argument.  */
-#define errorcount diagnostic_kind_count (global_dc, DK_ERROR)
+#define errorcount global_dc->diagnostic_count (DK_ERROR)
 /* Similarly, but for warnings.  */
-#define warningcount diagnostic_kind_count (global_dc, DK_WARNING)
+#define warningcount global_dc->diagnostic_count (DK_WARNING)
 /* Similarly, but for warnings promoted to errors.  */
-#define werrorcount diagnostic_kind_count (global_dc, DK_WERROR)
+#define werrorcount global_dc->diagnostic_count (DK_WERROR)
 /* Similarly, but for sorrys.  */
-#define sorrycount diagnostic_kind_count (global_dc, DK_SORRY)
+#define sorrycount global_dc->diagnostic_count (DK_SORRY)
 
 /* Returns nonzero if warnings should be emitted.  */
 #define diagnostic_report_warnings_p(DC, LOC)  \
-- 
2.26.3

[pushed 3/4] diagnostics: introduce class diagnostic_option_classifier

[David Malcolm]

This patch gathers the 6 fields in diagnostic_context relating to
keeping track of overriding the severity of warnings, and
pushing/popping those severities, moving them all into a new class
diagnostic_option_classifier.

No functional change intended.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-5168-g38763e2c188fa9.

gcc/ChangeLog:
* diagnostic.cc (diagnostic_context::push_diagnostics): Convert
to...
(diagnostic_option_classifier::push): ...this.
(diagnostic_context::pop_diagnostics): Convert to...
(diagnostic_option_classifier::pop): ...this.
(diagnostic_context::initialize): Move code to...
(diagnostic_option_classifier::init): ...this new function.
(diagnostic_context::finish): Move code to...
(diagnostic_option_classifier::fini): ...this new function.
(diagnostic_context::classify_diagnostic): Convert to...
(diagnostic_option_classifier::classify_diagnostic): ...this.
(diagnostic_context::update_effective_level_from_pragmas): Convert
to...
(diagnostic_option_classifier::update_effective_level_from_pragmas):
...this.
(diagnostic_context::diagnostic_enabled): Update for refactoring.
* diagnostic.h (struct diagnostic_classification_change_t): Move into...
(class diagnostic_option_classifier): ...this new class.
(diagnostic_context::option_unspecified_p): Update for move of
fields into m_option_classifier.
(diagnostic_context::classify_diagnostic): Likewise.
(diagnostic_context::push_diagnostics): Likewise.
(diagnostic_context::pop_diagnostics): Likewise.
(diagnostic_context::update_effective_level_from_pragmas): Delete.
(diagnostic_context::m_classify_diagnostic): Move into class
diagnostic_option_classifier.
(diagnostic_context::m_option_classifier): Likewise.
(diagnostic_context::m_classification_history): Likewise.
(diagnostic_context::m_n_classification_history): Likewise.
(diagnostic_context::m_push_list): Likewise.
(diagnostic_context::m_n_push): Likewise.
(diagnostic_context::m_option_classifier): New.
---
 gcc/diagnostic.cc | 124 ---
 gcc/diagnostic.h  | 131 --
 2 files changed, 163 insertions(+), 92 deletions(-)

diff --git a/gcc/diagnostic.cc b/gcc/diagnostic.cc
index c617b34f02b..addd6606eaa 100644
--- a/gcc/diagnostic.cc
+++ b/gcc/diagnostic.cc
@@ -149,13 +149,63 @@ diagnostic_set_caret_max_width (diagnostic_context 
*context, int value)
   context->m_source_printing.max_width = value;
 }
 
+void
+diagnostic_option_classifier::init (int n_opts)
+{
+  m_n_opts = n_opts;
+  m_classify_diagnostic = XNEWVEC (diagnostic_t, n_opts);
+  for (int i = 0; i < n_opts; i++)
+m_classify_diagnostic[i] = DK_UNSPECIFIED;
+  m_push_list = nullptr;
+  m_n_push = 0;
+}
+
+void
+diagnostic_option_classifier::fini ()
+{
+  XDELETEVEC (m_classify_diagnostic);
+  m_classify_diagnostic = nullptr;
+  free (m_push_list);
+  m_n_push = 0;
+}
+
+/* Save all diagnostic classifications in a stack.  */
+
+void
+diagnostic_option_classifier::push ()
+{
+  m_push_list = (int *) xrealloc (m_push_list, (m_n_push + 1) * sizeof (int));
+  m_push_list[m_n_push ++] = m_n_classification_history;
+}
+
+/* Restore the topmost classification set off the stack.  If the stack
+   is empty, revert to the state based on command line parameters.  */
+
+void
+diagnostic_option_classifier::pop (location_t where)
+{
+  int jump_to;
+
+  if (m_n_push)
+jump_to = m_push_list [-- m_n_push];
+  else
+jump_to = 0;
+
+  const int i = m_n_classification_history;
+  m_classification_history =
+(diagnostic_classification_change_t *) xrealloc (m_classification_history, 
(i + 1)
+* sizeof 
(diagnostic_classification_change_t));
+  m_classification_history[i].location = where;
+  m_classification_history[i].option = jump_to;
+  m_classification_history[i].kind = DK_POP;
+  m_n_classification_history ++;
+}
+
 /* Initialize the diagnostic message outputting machinery.  */
 
 void
 diagnostic_context::initialize (int n_opts)
 {
-  int i;
-
   /* Allocate a basic pretty-printer.  Clients will replace this a
  much more elaborated pretty-printer if they wish.  */
   this->printer = XNEW (pretty_printer);
@@ -165,12 +215,10 @@ diagnostic_context::initialize (int n_opts)
   memset (m_diagnostic_count, 0, sizeof m_diagnostic_count);
   m_warning_as_error_requested = false;
   m_n_opts = n_opts;
-  m_classify_diagnostic = XNEWVEC (diagnostic_t, n_opts);
-  for (i = 0; i < n_opts; i++)
-m_classify_diagnostic[i] = DK_UNSPECIFIED;
+  m_option_classifier.init (n_opts);
   m_source_printing.enabled = false;
   diagnostic_set_caret_max_width (this, pp_line_cutoff (this->printer));
-  for (i = 0; i < rich_location::STA

[pushed 4/4] diagnostics: split out struct diagnostic_source_printing_options

[David Malcolm]

This patch removes almost all use of diagnostic_context from the
source-printing code.

No functional change intended.

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to trunk as r14-5169-g54da47f9459890.

gcc/ChangeLog:
* diagnostic-show-locus.cc (class colorizer): Take just a
pretty_printer rather than a diagnostic_context.
(layout::layout): Make context param a const reference,
and pretty_printer param non-optional.
(layout::m_context): Drop field.
(layout::m_options): New field.
(layout::m_colorize_source_p): Drop field.
(layout::m_show_labels_p): Drop field.
(layout::m_show_line_numbers_p): Drop field.
(layout::print_gap_in_line_numbering): Use m_options.
(layout::calculate_line_spans): Likewise.
(layout::calculate_linenum_width): Likewise.
(layout::calculate_x_offset_display): Likewise.
(layout::print_source_line): Likewise.
(layout::start_annotation_line): Likewise.
(layout::print_annotation_line): Likewise.
(layout::print_line): Likewise.
(gcc_rich_location::add_location_if_nearby): Update for changes to
layout ctor.
(diagnostic_show_locus): Likewise.
(selftest::test_offset_impl): Likewise.
(selftest::test_layout_x_offset_display_utf8): Likewise.
(selftest::test_layout_x_offset_display_tab): Likewise.
(selftest::test_tab_expansion): Likewise.
* diagnostic.h (diagnostic_context::m_source_printing): Move
declaration of struct outside diagnostic_context as...
(struct diagnostic_source_printing_options)... this.
---
 gcc/diagnostic-show-locus.cc | 94 +---
 gcc/diagnostic.h | 88 -
 2 files changed, 88 insertions(+), 94 deletions(-)

diff --git a/gcc/diagnostic-show-locus.cc b/gcc/diagnostic-show-locus.cc
index d7a471426b9..43523572fe5 100644
--- a/gcc/diagnostic-show-locus.cc
+++ b/gcc/diagnostic-show-locus.cc
@@ -83,7 +83,7 @@ struct point_state
 class colorizer
 {
  public:
-  colorizer (diagnostic_context *context,
+  colorizer (pretty_printer *pp,
 diagnostic_t diagnostic_kind);
   ~colorizer ();
 
@@ -113,7 +113,7 @@ class colorizer
   static const int STATE_FIXIT_INSERT  = -2;
   static const int STATE_FIXIT_DELETE  = -3;
 
-  diagnostic_context *m_context;
+  pretty_printer *m_pp;
   diagnostic_t m_diagnostic_kind;
   int m_current_state;
   const char *m_range1;
@@ -365,10 +365,10 @@ struct char_display_policy : public cpp_char_column_policy
 class layout
 {
  public:
-  layout (diagnostic_context *context,
+  layout (const diagnostic_context &context,
  rich_location *richloc,
  diagnostic_t diagnostic_kind,
- pretty_printer *pp = nullptr);
+ pretty_printer *pp);
 
   bool maybe_add_location_range (const location_range *loc_range,
 unsigned original_idx,
@@ -428,15 +428,12 @@ class layout
   move_to_column (int *column, int dest_column, bool add_left_margin);
 
  private:
-  diagnostic_context *m_context;
+  const diagnostic_source_printing_options &m_options;
   pretty_printer *m_pp;
   char_display_policy m_policy;
   location_t m_primary_loc;
   exploc_with_display_col m_exploc;
   colorizer m_colorizer;
-  bool m_colorize_source_p;
-  bool m_show_labels_p;
-  bool m_show_line_numbers_p;
   bool m_diagnostic_path_p;
   auto_vec  m_layout_ranges;
   auto_vec  m_fixit_hints;
@@ -451,9 +448,9 @@ class layout
 /* The constructor for "colorizer".  Lookup and store color codes for the
different kinds of things we might need to print.  */
 
-colorizer::colorizer (diagnostic_context *context,
+colorizer::colorizer (pretty_printer *pp,
  diagnostic_t diagnostic_kind) :
-  m_context (context),
+  m_pp (pp),
   m_diagnostic_kind (diagnostic_kind),
   m_current_state (STATE_NORMAL_TEXT)
 {
@@ -461,7 +458,7 @@ colorizer::colorizer (diagnostic_context *context,
   m_range2 = get_color_by_name ("range2");
   m_fixit_insert = get_color_by_name ("fixit-insert");
   m_fixit_delete = get_color_by_name ("fixit-delete");
-  m_stop_color = colorize_stop (pp_show_color (context->printer));
+  m_stop_color = colorize_stop (pp_show_color (m_pp));
 }
 
 /* The destructor for "colorize".  If colorization is on, print a code to
@@ -497,35 +494,35 @@ colorizer::begin_state (int state)
   break;
 
 case STATE_FIXIT_INSERT:
-  pp_string (m_context->printer, m_fixit_insert);
+  pp_string (m_pp, m_fixit_insert);
   break;
 
 case STATE_FIXIT_DELETE:
-  pp_string (m_context->printer, m_fixit_delete);
+  pp_string (m_pp, m_fixit_delete);
   break;
 
 case 0:
   /* Make range 0 be the same color as the "kind" text
 (error vs warning vs note).  */
   pp_string
-   (m_context->printer,
-colorize_start (pp_show_color (m_context->printer),
+   (m_pp,
+ 

[PATCH/RFC] libdiagnostics: a shared library for emitting diagnostics

[David Malcolm]

It's fairly easy for tools to implement simple diagnostics
via fprintf of 
  FILE:LINE:COLUMN: error: message
to stderr, but as diagnostics get more featureful, using a shared
library makes sense.

This patch kit extends GCC to add a new "libdiagnostics" shared library
on the host, built around GCC's existing diagnostic-handling code, exposed
via a pure C API intended for client code that wants to emit GCC-style
diagnostics. It implements:
- quoting pertinent source code (with a cache)
- underlining points and ranges in the source code, possibly with labels
- emitting fix-it hints
- generating patches from fix-it hints
- SARIF output

The first patch (for GCC) shows libdiagnostic.h (the public header
file), along with examples of simple self-contained programs that
show various uses of the API.

The second patch (for GCC) is the work-in-progress implementation.

The third patch (for binutils) is an experiment at using the API
with gas.

Status: this is a rough prototype.  I'm posting it now to get
feedback, both from GCC developers, and from projects that might make
use of this library (binutils? GNU Cobol? others?).  The header
file has a "TODO" list at the end listing various known unfinished
aspects, and "known unknowns".

Thoughts?


David Malcolm (2):
  libdiagnostics: header and examples
  libdiagnostics: work-in-progress implementation

 gcc/Makefile.in   |  134 +-
 gcc/configure |2 +-
 gcc/configure.ac  |2 +-
 gcc/input.h   |2 +-
 gcc/libdiagnostics.cc | 1124 +
 gcc/libdiagnostics.h  |  544 
 gcc/libdiagnostics.map|   57 +
 .../libdiagnostics.dg/libdiagnostics.exp  |  544 
 .../libdiagnostics.dg/test-error-with-note.c  |   57 +
 gcc/testsuite/libdiagnostics.dg/test-error.c  |   49 +
 .../libdiagnostics.dg/test-fix-it-hint.c  |   48 +
 .../libdiagnostics.dg/test-helpers.h  |   29 +
 .../libdiagnostics.dg/test-labelled-ranges.c  |   52 +
 .../libdiagnostics.dg/test-logical-location.c |   62 +
 .../libdiagnostics.dg/test-metadata.c |   53 +
 .../libdiagnostics.dg/test-multiple-lines.c   |   58 +
 .../test-note-with-fix-it-hint.c  |   51 +
 .../libdiagnostics.dg/test-warning.c  |   52 +
 .../test-write-sarif-to-file.c|   46 +
 .../test-write-text-to-file.c |   47 +
 20 files changed, 3008 insertions(+), 5 deletions(-)
 create mode 100644 gcc/libdiagnostics.cc
 create mode 100644 gcc/libdiagnostics.h
 create mode 100644 gcc/libdiagnostics.map
 create mode 100644 gcc/testsuite/libdiagnostics.dg/libdiagnostics.exp
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-error-with-note.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-error.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-fix-it-hint.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-helpers.h
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-labelled-ranges.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-logical-location.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-metadata.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-multiple-lines.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-note-with-fix-it-hint.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-warning.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-write-sarif-to-file.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-write-text-to-file.c

-- 
2.26.3

[PATCH 1/2] libdiagnostics: header and examples

[David Malcolm]

Here's a work-in-progress patch for GCC that adds a libdiagnostics.h
header describing the public interface, along with various testcases
that show usage examples for the API.  Various aspects of this need
work; posting now for early feedback on overall direction.

How does the interface look?

gcc/ChangeLog:
* libdiagnostics.h: New file.

gcc/testsuite/ChangeLog:
* libdiagnostics.dg/test-error-with-note.c: New test.
* libdiagnostics.dg/test-error.c: New test.
* libdiagnostics.dg/test-fix-it-hint.c: New test.
* libdiagnostics.dg/test-helpers.h: New.
* libdiagnostics.dg/test-labelled-ranges.c: New test.
* libdiagnostics.dg/test-logical-location.c: New test.
* libdiagnostics.dg/test-metadata.c: New test.
* libdiagnostics.dg/test-multiple-lines.c: New test.
* libdiagnostics.dg/test-note-with-fix-it-hint.c: New test.
* libdiagnostics.dg/test-warning.c: New test.
* libdiagnostics.dg/test-write-sarif-to-file.c: New test.
* libdiagnostics.dg/test-write-text-to-file.c: New test.
---
 gcc/libdiagnostics.h  | 544 ++
 .../libdiagnostics.dg/test-error-with-note.c  |  57 ++
 gcc/testsuite/libdiagnostics.dg/test-error.c  |  49 ++
 .../libdiagnostics.dg/test-fix-it-hint.c  |  48 ++
 .../libdiagnostics.dg/test-helpers.h  |  29 +
 .../libdiagnostics.dg/test-labelled-ranges.c  |  52 ++
 .../libdiagnostics.dg/test-logical-location.c |  62 ++
 .../libdiagnostics.dg/test-metadata.c |  53 ++
 .../libdiagnostics.dg/test-multiple-lines.c   |  58 ++
 .../test-note-with-fix-it-hint.c  |  51 ++
 .../libdiagnostics.dg/test-warning.c  |  52 ++
 .../test-write-sarif-to-file.c|  46 ++
 .../test-write-text-to-file.c |  47 ++
 13 files changed, 1148 insertions(+)
 create mode 100644 gcc/libdiagnostics.h
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-error-with-note.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-error.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-fix-it-hint.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-helpers.h
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-labelled-ranges.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-logical-location.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-metadata.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-multiple-lines.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-note-with-fix-it-hint.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-warning.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-write-sarif-to-file.c
 create mode 100644 gcc/testsuite/libdiagnostics.dg/test-write-text-to-file.c

diff --git a/gcc/libdiagnostics.h b/gcc/libdiagnostics.h
new file mode 100644
index 000..672594598fa
--- /dev/null
+++ b/gcc/libdiagnostics.h
@@ -0,0 +1,544 @@
+/* A pure C API for emitting diagnostics.
+   Copyright (C) 2023 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 3, or (at your option)
+any later version.
+
+GCC is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3.  If not see
+.  */
+
+#ifndef LIBDIAGNOSTICS_H
+#define LIBDIAGNOSTICS_H
+
+/* We use FILE * for streams */
+#include 
+
+#ifdef __cplusplus
+extern "C" {
+#endif /* __cplusplus */
+
+/**
+ Macros for attributes.
+ These are all currently empty, and thus for the human reader rather than
+ the compiler.
+ **/
+
+#define LIBDIAGNOSTICS_PARAM_MUST_BE_NON_NULL(ARG_NUM)
+
+#define LIBDIAGNOSTICS_PARAM_CAN_BE_NULL(ARG_NUM)
+
+#define LIBDIAGNOSTICS_PARAM_GCC_FORMAT_STRING(FMT_ARG_NUM, ARGS_ARG_NUM)
+
+
+/**
+ Data structures and types.
+ All structs within the API are opaque.
+ **/
+
+/* An opaque bundle of state for a client of the library.
+   Has zero of more "sinks" to which diagnostics are emitted.
+   Responsibilities:
+   - location-management
+   - caching of source file content
+   - patch generation.  */
+typedef struct diagnostic_manager diagnostic_manager;
+
+/* Types relating to diagnostic output sinks.  */
+
+/* An enum for determining if we should colorize a text output sink.  */
+enum diagnostic_colorize
+{
+  DIAGNOSTIC_C

[PATCH 2/2] libdiagnostics: work-in-progress implementation

[David Malcolm]

c_manager_new_logical_location (diagnostic_manager *diag_mgr,
+enum 
diagnostic_logical_location_kind_t kind,
+const diagnostic_logical_location 
*parent,
+const char *short_name,
+const char *fully_qualified_name,
+const char *decorated_name)
+{
+  gcc_assert (diag_mgr);
+
+  return diag_mgr->new_logical_location (kind,
+parent,
+short_name,
+fully_qualified_name,
+decorated_name);
+}
+
+/* Public entrypoint.  */
+
+void
+diagnostic_manager_begin_group (diagnostic_manager *diag_mgr)
+{
+  gcc_assert (diag_mgr);
+  diag_mgr->begin_group ();
+}
+
+/* Public entrypoint.  */
+
+extern void
+diagnostic_manager_end_group (diagnostic_manager *diag_mgr)
+{
+  gcc_assert (diag_mgr);
+  diag_mgr->end_group ();
+}
+
+/* Public entrypoint.  */
+
+diagnostic *
+diagnostic_begin (diagnostic_manager *diag_mgr,
+ enum diagnostic_level level)
+{
+  gcc_assert (diag_mgr);
+
+  return new diagnostic (*diag_mgr, level);
+}
+
+/* Public entrypoint.  */
+
+void
+diagnostic_set_cwe (diagnostic *diag,
+   unsigned cwe_id)
+{
+  gcc_assert (diag);
+
+  diag->set_cwe (cwe_id);
+}
+
+/* Public entrypoint.  */
+
+void
+diagnostic_add_rule (diagnostic *diag,
+const char *title,
+const char *url)
+{
+  gcc_assert (diag);
+
+  diag->add_rule (title, url);
+}
+
+/* Public entrypoint.  */
+
+void
+diagnostic_set_location (diagnostic *diag,
+diagnostic_location_t loc)
+{
+  gcc_assert (diag);
+  diag->get_manager ().assert_valid_diagnostic_location_t (loc);
+
+  diag->set_location (loc);
+}
+
+/* Public entrypoint.  */
+
+void
+diagnostic_add_location (diagnostic *diag,
+diagnostic_location_t loc)
+{
+  gcc_assert (diag);
+  diag->get_manager ().assert_valid_diagnostic_location_t (loc);
+
+  diag->add_location (loc);
+}
+
+/* Public entrypoint.  */
+
+void
+diagnostic_add_location_with_label (diagnostic *diag,
+   diagnostic_location_t loc,
+   const char *text)
+{
+  gcc_assert (diag);
+  diag->get_manager ().assert_valid_diagnostic_location_t (loc);
+  gcc_assert (text);
+
+  diag->add_location_with_label (loc, text);
+}
+
+/* Public entrypoint.  */
+
+void
+diagnostic_set_logical_location (diagnostic *diag,
+const diagnostic_logical_location *logical_loc)
+{
+  gcc_assert (diag);
+  gcc_assert (logical_loc);
+
+  diag->set_logical_location (logical_loc);
+}
+
+/* Public entrypoint.  */
+
+void
+diagnostic_add_fix_it_hint_insert_before (diagnostic *diag,
+ diagnostic_location_t loc,
+ const char *addition)
+{
+  gcc_assert (diag);
+  diag->get_manager ().assert_valid_diagnostic_location_t (loc);
+  gcc_assert (addition);
+
+  diag->get_manager ().set_line_table_global ();
+  diag->get_rich_location ()->add_fixit_insert_before (loc, addition);
+}
+
+/* Public entrypoint.  */
+
+void
+diagnostic_add_fix_it_hint_insert_after (diagnostic *diag,
+diagnostic_location_t loc,
+const char *addition)
+{
+  gcc_assert (diag);
+  diag->get_manager ().assert_valid_diagnostic_location_t (loc);
+  gcc_assert (addition);
+
+  diag->get_manager ().set_line_table_global ();
+  diag->get_rich_location ()->add_fixit_insert_after (loc, addition);
+}
+
+/* Public entrypoint.  */
+
+void
+diagnostic_add_fix_it_hint_replace (diagnostic *diag,
+   diagnostic_location_t loc,
+   const char *replacement)
+{
+  gcc_assert (diag);
+  diag->get_manager ().assert_valid_diagnostic_location_t (loc);
+  gcc_assert (replacement);
+
+  diag->get_manager ().set_line_table_global ();
+  diag->get_rich_location ()->add_fixit_replace (loc, replacement);
+}
+
+/* Public entrypoint.  */
+
+void
+diagnostic_add_fix_it_hint_delete (diagnostic *diag,
+  diagnostic_location_t loc)
+{
+  gcc_assert (diag);
+  diag->get_manager ().assert_valid_diagnostic_location_t (loc);
+
+  diag->get_manager ().set_line_table_global ();
+  diag->get_rich_location ()->add_fixit_remove (loc);
+}
+
+/* Public entrypoint.  */
+
+void
+diagnostic_finish (diagnostic *diag, const char *gmsgid, ...)
+{
+  gcc_assert (diag);
+
+  if (const char *tool_name
+  = diag->get_manager ().get_client_version_info ()->m_name.get_str ())
+progname = tool_name;
+  else
+progname = "progname"

Re: [wwwdocs] readings.html - "Porting GCC for Dunces" is gone

[David Malcolm]

On Sun, 2019-11-10 at 14:53 +0100, Gerald Pfeifer wrote:
> Hi H-P,
> 
> it appears this download is gone. Do you have an alternate location?
> 
> For now I applied the patch below which disables that link in 
> readings.html.
> 
> Gerald

FWIW archive.org seems to have a copy here:

https://web.archive.org/web/20190214220423/http://ftp.axis.se/pub/users/hp/pgccfd/pgccfd.pdf

[PATCH] analyzer: fix ICE on METHOD_TYPE (PR 93212)

[David Malcolm]

PR analyzer/93212 reports an ICE when attempting to use -fanalyzer
on a C++ source file.  That isn't supported yet, but the fix is
trivial (handling METHOD_TYPE as well as FUNCTION_TYPE).

Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.
Pushed to the dmalcolm/analyzer branch on the GCC git mirror.

gcc/analyzer/ChangeLog:
PR analyzer/93212
* region-model.cc (make_region_for_type): Use
FUNC_OR_METHOD_TYPE_P rather than comparing against FUNCTION_TYPE.
* region-model.h (function_region::function_region): Likewise.
---
 gcc/analyzer/region-model.cc | 2 +-
 gcc/analyzer/region-model.h  | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/gcc/analyzer/region-model.cc b/gcc/analyzer/region-model.cc
index cebb1d7127a..ab52fe7cf31 100644
--- a/gcc/analyzer/region-model.cc
+++ b/gcc/analyzer/region-model.cc
@@ -5951,7 +5951,7 @@ make_region_for_type (region_id parent_rid, tree type)
   if (TREE_CODE (type) == UNION_TYPE)
 return new union_region (parent_rid, type);
 
-  if (TREE_CODE (type) == FUNCTION_TYPE)
+  if (FUNC_OR_METHOD_TYPE_P (type))
 return new function_region (parent_rid, type);
 
   /* If we have a void *, make a new symbolic region.  */
diff --git a/gcc/analyzer/region-model.h b/gcc/analyzer/region-model.h
index b3b33452327..883c4608474 100644
--- a/gcc/analyzer/region-model.h
+++ b/gcc/analyzer/region-model.h
@@ -1210,7 +1210,7 @@ public:
   function_region (region_id parent_rid, tree type)
   : map_region (parent_rid, type)
   {
-gcc_assert (TREE_CODE (type) == FUNCTION_TYPE);
+gcc_assert (FUNC_OR_METHOD_TYPE_P (type));
   }
   function_region (const function_region &other)
   : map_region (other)
-- 
2.21.0

Re: [PATCH 33/49] analyzer: new files: sm.{cc|h}

[David Malcolm]

On Wed, 2019-12-11 at 12:23 -0700, Jeff Law wrote:
> On Fri, 2019-11-15 at 20:23 -0500, David Malcolm wrote:
> > This patch adds a "state_machine" base class for describing
> > API checkers in terms of state machine transitions.  Followup
> > patches use this to add specific API checkers.
> > 
> > gcc/ChangeLog:
> > * analyzer/sm.cc: New file.
> > * analyzer/sm.h: New file.
> > ---
> >  gcc/analyzer/sm.cc | 135
> > 
> >  gcc/analyzer/sm.h  | 160
> > +
> >  2 files changed, 295 insertions(+)
> >  create mode 100644 gcc/analyzer/sm.cc
> >  create mode 100644 gcc/analyzer/sm.h
> > 
> > diff --git a/gcc/analyzer/sm.cc b/gcc/analyzer/sm.cc
> > new file mode 100644
> > index 000..eda9350
> > --- /dev/null
> > +++ b/gcc/analyzer/sm.cc
> > @@ -0,0 +1,135 @@
> > +/* Modeling API uses and misuses via state machines.
> > +   Copyright (C) 2019 Free Software Foundation, Inc.
> > +   Contributed by David Malcolm .
> > +
> > +This file is part of GCC.
> > +
> > +GCC is free software; you can redistribute it and/or modify it
> > +under the terms of the GNU General Public License as published by
> > +the Free Software Foundation; either version 3, or (at your
> > option)
> > +any later version.
> > +
> > +GCC is distributed in the hope that it will be useful, but
> > +WITHOUT ANY WARRANTY; without even the implied warranty of
> > +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > +General Public License for more details.
> > +
> > +You should have received a copy of the GNU General Public License
> > +along with GCC; see the file COPYING3.  If not see
> > +<http://www.gnu.org/licenses/>;;.  */
> > +
> > +#include "config.h"
> > +#include "gcc-plugin.h"
> > +#include "system.h"
> > +#include "coretypes.h"
> > +#include "tree.h"
> > +#include "gimple.h"
> > +#include "analyzer/analyzer.h"
> > +#include "analyzer/sm.h"
> > +
> > +//
> > //
> > 
> > +
> > +/* If STMT is an assignment to zero, return the LHS.  */
> > +
> > +tree
> > +is_zero_assignment (const gimple *stmt)
> > +{
> > +  const gassign *assign_stmt = dyn_cast  (stmt);
> > +  if (!assign_stmt)
> > +return NULL_TREE;
> > +
> > +  enum tree_code op = gimple_assign_rhs_code (assign_stmt);
> > +  if (op != INTEGER_CST)
> > +return NULL_TREE;
> > +
> > +  if (!zerop (gimple_assign_rhs1 (assign_stmt)))
> > +return NULL_TREE;
> > +
> > +  return gimple_assign_lhs (assign_stmt);
> > +}
> "assignment from zero" rather than "assignment to zero" I think.

Fixed.

> I think you'd want to use "integer_zerop" rather than an open-coded
> check.  That'll also allow you to pick up other forms such as
> COMPLEX_CST and VECTOR_CST.

zero_p calls integer_zero_p, and also the equivalents for real and
fixed, so ought to be as flexible as you suggest.

I think what's needed is to replace that op != INTEGER_CST check with
  TREE_CODE_CLASS (op) == tcc_constant
to keep the generality you suggest, whilst rejecting e.g. a binary op
with a zero as arg 1.

I've done that in the following patch.
 
> > +
> > +/* If COND_STMT is a comparison against zero of the form (LHS OP
> > 0),
> > +   return true and write what's being compared to *OUT_LHS and the
> > kind of
> > +   the comparison to *OUT_OP.  */
> > +
> > +bool
> > +is_comparison_against_zero (const gcond *cond_stmt,
> > +   tree *out_lhs, enum tree_code *out_op)
> > +{
> > +  enum tree_code op = gimple_cond_code (cond_stmt);
> > +  tree lhs = gimple_cond_lhs (cond_stmt);
> > +  tree rhs = gimple_cond_rhs (cond_stmt);
> > +  if (!zerop (rhs))
> > +return false;
> > +  // TODO: make it symmetric?
> > +
> > +  switch (op)
> > +{
> > +case NE_EXPR:
> > +case EQ_EXPR:
> > +  *out_lhs = lhs;
> > +  *out_op = op;
> > +  return true;
> > +
> > +default:
> > +  return false;
> > +}
> > +}
> Seems like this might be useful to make generically available.

It turned out that I'm not using this anymore so I've deleted it
in the following version of the patch (for the curious, I think I
stopp

Re: [PATCH 43/49] analyzer: new file: exploded-graph.h

[David Malcolm]

On Wed, 2019-12-11 at 13:04 -0700, Jeff Law wrote:
> On Fri, 2019-11-15 at 20:23 -0500, David Malcolm wrote:
> > This patch adds exploded_graph and related classes, for managing
> > exploring paths through the user's code as a directed graph
> > of  pairs.
> > 
> > gcc/ChangeLog:
> > * analyzer/exploded-graph.h: New file.
> > ---
> >  gcc/analyzer/exploded-graph.h | 754
> > ++
> >  1 file changed, 754 insertions(+)
> >  create mode 100644 gcc/analyzer/exploded-graph.h
> > 
> > diff --git a/gcc/analyzer/exploded-graph.h b/gcc/analyzer/exploded-
> > graph.h
> > new file mode 100644
> > index 000..f97d2b6
> > --- /dev/null
> > +++ b/gcc/analyzer/exploded-graph.h
> > @@ -0,0 +1,754 @@
> > +/* Classes for managing a directed graph of  pairs.
> > +   Copyright (C) 2019 Free Software Foundation, Inc.
> > +   Contributed by David Malcolm .
> > +
> > +This file is part of GCC.
> > +
> > +GCC is free software; you can redistribute it and/or modify it
> > +under the terms of the GNU General Public License as published by
> > +the Free Software Foundation; either version 3, or (at your
> > option)
> > +any later version.
> > +
> > +GCC is distributed in the hope that it will be useful, but
> > +WITHOUT ANY WARRANTY; without even the implied warranty of
> > +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > +General Public License for more details.
> > +
> > +You should have received a copy of the GNU General Public License
> > +along with GCC; see the file COPYING3.  If not see
> > +<http://www.gnu.org/licenses/>;;.  */
> > +
> > +#ifndef GCC_ANALYZER_EXPLODED_GRAPH_H
> > +#define GCC_ANALYZER_EXPLODED_GRAPH_H
> > +
> > +#include "fibonacci_heap.h"
> > +#include "analyzer/analyzer-logging.h"
> > +#include "analyzer/constraint-manager.h"
> > +#include "analyzer/diagnostic-manager.h"
> > +#include "analyzer/program-point.h"
> > +#include "analyzer/program-state.h"
> > +#include "analyzer/shortest-paths.h"
> > +
> > +//
> > //
> NIT.  Is there some reason you don't just use whitespace for these
> kind
> of vertical separators.  THe more I see them, the more they bug me,
> probably because that's not been the style we've used for GCC.

Removed throughout.

> > ///
> > +
> > +/* Concrete implementation of region_model_context, wiring it up
> > to
> > the
> > +   rest of the analysis engine.  */
> > +
> > +class impl_region_model_context : public region_model_context,
> > public log_user
> Multiple inheritance?  Is it really needed?  Can we handle via
> composition instead?

I've reworked things throughout to avoid multiple inheritance.

> > +/* A  pair, used internally by
> > +   exploded_node as its immutable data, and as a key for
> > identifying
> > +   exploded_nodes we've already seen in the graph.  */
> > +
> > +struct point_and_state
> Shouldn't this be a class?

Fixed.

> +
> > +/* Per-program_point data for an exploded_graph.  */
> > +
> > +struct per_program_point_data
> Here too?  THere may be others.  I'd suggest reviewing all your
> "structs" and determine if we're better off calling them
> "class".  I'm
> not going to insist on it though since I think the last discussion in
> this space was to relax the conventions :-)

I went through these; mostly they're a bundle of fields, so making
them classes and adding getters and setters seemed pointless.
I made m_key "const" within struct per_program_point_data and
struct per_call_string_data as they cannot change after construction.


> > +
> > +class exploded_graph : public digraph, public log_user
> Multiple inheritance again?

Fixed.

> Jeff
>

Here's an updated patch (v6) which I've successfully bootstrapped &
regrtested (with the rest of the kit) on x86_64-pc-linux-gnu.

OK for trunk? (assuming the rest of the kit is approved)

Thanks
Dave


Changed in v6:
- updates per Jeff's comments here
https://gcc.gnu.org/ml/gcc-patches/2019-12/msg00813.html
  about considering converting structs to classes:
  - converted point_and_state from a struct to a class (given the
cached hash value)
  - made m_key "const" within struct per_program_point_data and
struct per_call_string_data
  Otherwise, converting the various other structs to classes would
  have

[PATCH] testsuite: add dg-enable-nn-line-numbers (v6)

[David Malcolm]

On Wed, 2020-01-08 at 17:07 -0500, David Malcolm wrote:
[...]
> Here's an alterative patch to the above that replaces the
> "-fdiagnostics-nn-line-numbers" option in earlier versions of the
> analyzer patch kit, by doing it at the DejaGnu level instead.
[...]

> I'm testing this now (but it seems to be a working, drop-in
> replacement
> for the option in the parts of the patch kit I've tested with it).

I spoke too soon; it didn't work with the non-gcc parts of the
testsuite that use prune.exp due to load_lib vs load_gcc_lib Tcl
lib path issues.

Given that this is meant to be used in conjunction with multiline.exp
the simplest fix was to move the code into multiline.exp.

Here's an updated version of the patch which does work;
successfully bootstrapped & regrtested on x86_64-pc-linux-gnu.

OK for trunk?

Thanks
Dave


> Examples of use can be seen in the analyzer test suite:
>   https://gcc.gnu.org/ml/gcc-patches/2020-01/msg00320.html
> (search for -fdiagnostics-nn-line-numbers and dg-begin-multiline-
> output
> there to get the idea)



Changed in v6:
- moved to multiline.exp to avoid having to add a new .exp file (which
  would have to be loaded manually via load_gcc_lib in various places
  before prune.exp due to lack of lib path support in tcl; one of these
  places is libgo.exp which we only mirror)
- clarify a comment to indicate that it doesn't have to be at the start
  of a line

Changed in v5:
- added, replacing the "-fdiagnostics-nn-line-numbers" option

This patch adds support for obscuring the line numbers printed in the
left-hand margin when printing the source code, converting them to "NN",
e.g from:

  7111 |   if (!(flags & 0x0001)) {
   |  ^
   |  |
   |  (1) following 'true' branch...
  7112 |

to:

   NN  |   if (!(flags & 0x0001)) {
   |  ^
   |  |
   |  (1) following 'true' branch...
   NN  |

This is useful in followup patches e.g. when testing how interprocedural
paths are printed using multiline.exp, to avoid depending on precise line
numbers.

This replaces the "-fdiagnostics-nn-line-numbers" option in earlier
versions of the analyzer patch kit, by doing it at the DejaGnu level
instead.

gcc/testsuite/ChangeLog:
* lib/gcc-dg.exp (cleanup-after-saved-dg-test): Reset global
nn_line_numbers_enabled.
* lib/multiline.exp (nn_line_numbers_enabled): New global.
(dg-enable-nn-line-numbers): New proc.
(maybe-handle-nn-line-numbers): New proc.
* lib/prune.exp (prune_gcc_output): Call maybe-handle-nn-line-numbers.
---
 gcc/testsuite/lib/gcc-dg.exp|  2 +
 gcc/testsuite/lib/multiline.exp | 88 +
 gcc/testsuite/lib/prune.exp |  4 ++
 3 files changed, 94 insertions(+)

diff --git a/gcc/testsuite/lib/gcc-dg.exp b/gcc/testsuite/lib/gcc-dg.exp
index e6875de2383..cccd3ce4742 100644
--- a/gcc/testsuite/lib/gcc-dg.exp
+++ b/gcc/testsuite/lib/gcc-dg.exp
@@ -940,6 +940,7 @@ if { [info procs saved-dg-test] == [list] } {
global set_compiler_env_var
global saved_compiler_env_var
global keep_saved_temps_suffixes
+   global nn_line_numbers_enabled
global multiline_expected_outputs
global freeform_regexps
global save_linenr_varnames
@@ -967,6 +968,7 @@ if { [info procs saved-dg-test] == [list] } {
if [info exists testname_with_flags] {
unset testname_with_flags
}
+   set nn_line_numbers_enabled 0
set multiline_expected_outputs []
set freeform_regexps []
 
diff --git a/gcc/testsuite/lib/multiline.exp b/gcc/testsuite/lib/multiline.exp
index e965d8946f7..627ad68817f 100644
--- a/gcc/testsuite/lib/multiline.exp
+++ b/gcc/testsuite/lib/multiline.exp
@@ -60,6 +60,9 @@ set _multiline_last_beginning_line -1
 # This is cleared at the end of each test by gcc-dg.exp's wrapper for dg-test.
 set multiline_expected_outputs []
 
+# Was dg-enable-nn-line-numbers called?
+set nn_line_numbers_enabled 0
+
 
 # Exported functions.
 
@@ -177,6 +180,91 @@ proc handle-multiline-outputs { text } {
 return $text
 }
 
+# DejaGnu directive to enable post-processing the line numbers printed in
+# the left-hand margin when printing the source code, converting them to
+# "NN", e.g from:
+#
+#100 |   if (flag)
+#|  ^
+#|  |
+#|  (1) following 'true' branch...
+#101 | {
+#102 |   foo ();
+#|   ^
+#|   |
+#|   (2) ...to here
+#
+# to:
+#
+# NN |   if (flag)
+#|  ^
+#|  |
+#|  (1) following 'true' branch...
+# NN | {
+# NN |   foo ();
+#   

[PATCH] Add ordered_hash_map (v6)

[David Malcolm]

On Thu, 2020-01-09 at 11:56 +, Jonathan Wakely wrote:
> On 08/01/20 18:47 -0500, David Malcolm wrote:
> > On Wed, 2019-12-04 at 10:59 -0700, Martin Sebor wrote:
> > > On 11/15/19 6:23 PM, David Malcolm wrote:
> > > > This patch adds an ordered_hash_map template, which is similar
> > > > to
> > > > hash_map, but preserves insertion order.
> > > > 
> > > > gcc/ChangeLog:
> > > > * Makefile.in (OBJS): Add ordered-hash-map-tests.o.
> > > > * ordered-hash-map-tests.cc: New file.
> > > > * ordered-hash-map.h: New file.
> > > > * selftest-run-tests.c (selftest::run_tests): Call
> > > > selftest::ordered_hash_map_tests_cc_tests.
> > > > * selftest.h
> > > > (selftest::ordered_hash_map_tests_cc_tests): New
> > > > decl.
> > > > ---
> > > >   gcc/Makefile.in   |   1 +
> > > >   gcc/ordered-hash-map-tests.cc | 247
> > > > ++
> > > >   gcc/ordered-hash-map.h| 184
> > > > +++
> > > >   gcc/selftest-run-tests.c  |   1 +
> > > >   gcc/selftest.h|   1 +
> > > >   5 files changed, 434 insertions(+)
> > > >   create mode 100644 gcc/ordered-hash-map-tests.cc
> > > >   create mode 100644 gcc/ordered-hash-map.h
> > > > 
> > 
> > [...]
> > 
> > > The container defines a copy-constructor but no copy assignment.
> > > Is it safely assignable? (I don't think auto_vec is safely
> > > copyable
> > > or assignable due to PR 90904.  It looks like the copy ctor works
> > > around it below.)
> > 
> > It's not safely assignable; I don't believe I'm using that (I am
> > using
> > the copy ctor).
> > 
> > I can make it private or similar to ensure it's not used.
> 
> Yes, if the compiler-generated assignment isn't safe, please declare
> (but don't define) a private assignment operator, to suppress the
> compiler-generated one.

Done.

> > > I don't think I've made use of the hash_map copy ctor or copy
> > > assignment but if it's anything like other GCC containers I'd
> > > worry about it not doing the right thing, especially for non-
> > > PODs.  I spent too much time chasing down miscompilations and
> > > other problems due to bugs (or design limitations) in these
> > > classes.
> > > 
> > > I'd far prefer to see us use libstdc++ containers in new code
> > > than introduce new ones of our own.  They are better designed
> > > and much better tested than these.  (I realize we're still
> > > hampered by targeting C++ 98.)
> > > 
> > > Martin
> > 
> > [CCing Jonathan for libstdc++ expertise]
> > 
> > Is there such a container in libstdc++?
> > This patch implements a map that preserves insertion order when
> > iterating, without requiring the Key type to be comparable.
> > 
> > As far as I can tell, std::map instead is a map that requires the
> > Key
> > type to be comparable,
> 
> Right.
> 
> > and uses that to implement a red-black tree
> > (rather than via hashing),
> 
> Right.
> 
> > and doesn't preserve insertion ordering.
> 
> std::map has unique keys anyway, so insertion order is irrelevant.
> Iteration is done by key order.
> 
> std::multimap is the non-unique version, and preserves insertion
> order
> for equivalent keys, but only since C++11. Before that it was
> unspecified whether or not a duplicate key was inserted after
> existing
> elements with equivalent keys.
> 
> > I use this ordered_hash_map class later in various places in the
> > analyzer patch kit to ensure more deterministic results, so that
> > results aren't affected by hash values of possibly-changing pointer
> > values.

(My class is similar to Python's OrderedDict, FWIW)

> Nothing in the standard library provides that functionality out of
> the
> box. The C++11 hash containers don't preserve order at all, even
> equivalent elements can be reordered when insertion triggers
> rehashing.

Thanks.  

> You could build something using a sequence container (vector, deque
> or
> list) that preserves a stable order according to insertion order, and
> then manually maintain a separate index of hash values used for
> lookup. That might not make things any simpler (I haven't looked at
> your act

v6 of analyzer patch kit

[David Malcolm]

On Wed, 2020-01-08 at 04:02 -0500, David Malcolm wrote:
> Here's an updated version of the analyzer patch kit.

I've rebased (to r280067) and applied various fixes to the kit; I've
pushed a v6 of the kit to the dmalcolm/analyzer branch of the git
mirror (along with various followups):
https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/dmalcolm/analyzer

I've also annotated the patch Subject lines with a brief review status
line e.g. {approved}, {needs review} etc.

Current review status (numbering is that of v6 of the kit):

Needs review:
  0003-needs-review-testsuite-add-dg-enable-nn-line-numbers.patch
  0005-needs-review-Add-ordered_hash_map.patch
  0006-needs-review-timevar.def-add-TVs-for-analyzer.patch
  0008-needs-review-analyzer-changes-to-configure.ac.patch
  0009-needs-review-analyzer-add-new-files-to-Makefile.in.patch
  0011-needs-review-analyzer-command-line-options.patch
  0012-needs-review-analyzer-logging-support.patch
  0013-needs-review-analyzer-new-file-analyzer-pass.cc-and-.patch
  0014-needs-review-analyzer-new-files-graphviz.-cc-h.patch
  0016-needs-review-analyzer-new-files-supergraph.-cc-h.patch
  0017-needs-review-response-analyzer-new-files-analyzer.-c.patch
  0019-needs-review-analyzer-new-files-constraint-manager.-.patch
  0020-needs-review-analyzer-new-files-region-model.-cc-h.patch
  0021-needs-review-analyzer-new-files-pending-diagnostic.-.patch
  0023-needs-review-analyzer-new-files-sm-malloc.cc-and-sm-.patch
  0027-needs-review-analyzer-new-file-sm-signal.cc.patch
  0032-needs-review-analyzer-new-files-program-state.-cc-h.patch
  0033-needs-review-analyzer-new-file-exploded-graph.h.patch
  0035-needs-review-analyzer-new-files-engine.-cc-h.patch
  0037-needs-review-analyzer-new-files-diagnostic-manager.-.patch
  0038-needs-review-gdbinit.in-add-break-on-saved-diagnosti.patch
  0039-needs-review-analyzer-test-suite.patch

Needs re-review:
  0022-needs-re-review-analyzer-new-files-sm.-cc-h.patch
  0036-needs-re-approval-analyzer-new-files-checker-path.-c.patch

Semi-approved:
  0015-semi-approved-analyzer-new-files-digraph.-cc-h-and-s.patch
  0018-semi-approved-analyzer-new-files-tristate.-cc-h.patch
  0024-semi-approved-analyzer-new-file-sm-file.cc.patch
  0026-semi-approved-analyzer-new-file-sm-sensitive.cc.patch
  0028-semi-approved-analyzer-new-file-sm-taint.cc.patch

Approved:
  0001-approved-analyzer-user-facing-documentation.patch
  0002-approved-analyzer-internal-documentation.patch
  0010-approved-analyzer-new-files-analyzer-selftests.-cc-h.patch
  0025-approved-analyzer-new-file-sm-pattern-test.cc.patch
  0029-approved-analyzer-new-files-analysis-plan.-cc-h.patch
  0030-approved-analyzer-new-files-call-string.-cc-h.patch
  0031-approved-analyzer-new-files-program-point.-cc-h.patch
  0034-approved-analyzer-new-files-state-purge.-cc-h.patch

Can self approve:
  0004-can-self-approve-Add-diagnostic-paths.patch
  0007-obvious-analyzer-add-ChangeLog.patch

I can resend these to the list if that would be helpful.

Thanks
Dave

Re: [PATCH 07/41] Add ordered_hash_map

[David Malcolm]

On Fri, 2020-01-10 at 09:22 -0700, Jeff Law wrote:
> On Wed, 2020-01-08 at 04:02 -0500, David Malcolm wrote:
> > Needs review.  This is used in many places in the analyzer.
> > msebor made some comments about the v1 version of this patch here:
> >   https://gcc.gnu.org/ml/gcc-patches/2019-12/msg00231.html
> > 
> > Changed in v5:
> > - updated copyright years to include 2020
> > 
> > This patch adds an ordered_hash_map template, which is similar to
> > hash_map, but preserves insertion order.
> > 
> > gcc/ChangeLog:
> > * Makefile.in (OBJS): Add ordered-hash-map-tests.o.
> > * ordered-hash-map-tests.cc: New file.
> > * ordered-hash-map.h: New file.
> > * selftest-run-tests.c (selftest::run_tests): Call
> > selftest::ordered_hash_map_tests_cc_tests.
> > * selftest.h (selftest::ordered_hash_map_tests_cc_tests): New
> > decl.
> There's nothing inherent in the data that would preclude us from
> using
> a standard container (ie, nothing with embedded GC based on our
> meeting
> Tuesday).

Correct: this doesn't support our GC, but doesn't need to for the uses
I'm making of it.

>   But there isn't an existing standard container with the
> right properties (based on email between you and Jon).  Right?

Correct; std::map uses Key ordering to build a red-black tree; I'm
using insertion ordering (not Key ordering), to get more deterministic
results in the face of ptr values that can change from under me.  It's
analogous to Python's OrderedDict, fwiw.

> Do you need a private assignment operator?

I added that in:
  https://gcc.gnu.org/ml/gcc-patches/2020-01/msg00518.html
to ensure that we don't erroneously use the compiler-generated one;
that's the latest version of this patch.

Dave

Re: [PATCH 29/41] analyzer: new file: sm-signal.cc

[David Malcolm]

On Fri, 2020-01-10 at 09:01 -0700, Jeff Law wrote:
> On Wed, 2020-01-08 at 04:02 -0500, David Malcolm wrote:
> > Needs review.
> > 
> > Although the comments say "experimental", there are followups
> > (using
> > function_set) which make this much more production-ready than the
> > other sm-*.cc (other than malloc).
> > 
> > Changed in v5:
> > - update ChangeLog path
> > - updated copyright years to include 2020
> > 
> > New in v4; part of:
> >   https://gcc.gnu.org/ml/gcc-patches/2019-12/msg00214.html
> > with various fixups
> > 
> > gcc/analyzer/ChangeLog:
> > * sm-signal.cc: New file.
> OK. 

Thanks.

> Extending this one seems pretty trivial -- it's just checking for
> specific functions we know aren't safe, right?

Indeed, and I've done some followup work on this in the branch:

  "[PATCH 0/4] analyzer: add class function_set and use in various places"
 https://gcc.gnu.org/ml/gcc-patches/2019-12/msg01422.html

(plus I have experimental code in my working copy that can load such
sets on demand from data files, to avoid baking them into the analyzer,
or relying on attributes in old header files)

Dave