https://gcc.gnu.org/bugzilla/show_bug.cgi?id=116792

--- Comment #16 from GCC Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by David Malcolm <dmalc...@gcc.gnu.org>:

https://gcc.gnu.org/g:2334d30cd8feac28831ffef857cd09753a3ca3d3

commit r16-1631-g2334d30cd8feac28831ffef857cd09753a3ca3d3
Author: David Malcolm <dmalc...@redhat.com>
Date:   Mon Jun 23 11:06:43 2025 -0400

    diagnostics: add state diagrams to analyzer experimental-html output
[PR116792]

    This patch adds various support for debugging diagnostic paths and
    events, intended initially for myself to help with debugging -fanalyzer.

    It adds the optional ability for a diagnostic_event to supply a
    description of the predicted state of the program at that point along
    the diagnostic_path.  To isolate the diagnostic subsystem from the
    analyzer, this representation is currently an xml::document with custom
    elements.  The XML representation is similar to the analyzer's internal
    state but can be easier to read - for example, rather than storing the
    contents of memory via byte offsets, it uses fields for structs and
    element indexes for arrays, recursively.

    These states are handled by the HTML and SARIF diagnostic sinks.

    The SARIF sink simply embeds the XML as a string in a property bag of the
    threadFlowLocation object (SARIF v2.1.0 section 3.38).

    For HTML output, the "experimental-html" sink gains a new
    "show-state-diagrams=yes" option i.e.:
      -fdiagnostics-add-output=experimental-html:show-state-diagrams=yes
    which converts the state XML into SVG diagrams visualizing the state of
    memory at each event, inspired by the "ddd" debugger.  These can be seen
    by pressing 'j' and 'k' to single-step forward and backward through
    events, making it *much* easier to debug -fanalyzer.

    An example of output can be seen here:
      https://dmalcolm.fedorapeople.org/gcc/2025-06-23/state-diagram-1.c.html
    showing an issue in a singly-linked list; there are various other
    examples in the parent directory.

    Generating the SVG diagrams requires an invocation of "dot" per event,
    so it noticeable slows down diagnostic emission, hence the opt-in
    command-line flag.  However, I'm already finding bugs in -fanalyzer with
    this that I hadn't seen before.

    Given that the UI is rather clunky and there is lots of room for
    improvement to the visualizations, for now this feature is marked
    as being for GCC developers, not end-users.

    The patch also adds a dot::ast_node class hierarachy to make it easy to
    create GraphViz dot files with the correct escaping, and adds a C++
    wrapper around pex adding some syntactic sugar for invoking
    subprocesses.

    gcc/ChangeLog:
            PR other/116792
            * Makefile.in (ANALYZER_OBJS): Add
            analyzer/ana-state-to-diagnostic-state.o.
            (OBJS): Move graphviz.o to...
            (OBJS-libcommon): ...here.  Add diagnostic-state-to-dot.o and
pex.o.
            * diagnostic-format-html.cc: Include "diagnostic-state.h" and
            "graphviz.h".
            (html_generation_options::html_generation_options): Initialize the
            new flags.
            (HTML_SCRIPT): Add function "get_any_state_diagram".  Use it
            when changing current focus id to update the visibility of the
            pertinent diagram, if any.
            (print_pre_source): New.
            (html_builder::maybe_make_state_diagram): New.
            (html_path_label_writer::html_path_label_writer): Add "path" param.
            Initialize m_path and m_curr_event_id.
            (html_path_label_writer::begin_label): Store current event id.
            (html_path_label_writer::end_label): Attempt to make a state
            diagram and add it if successful.
            (html_path_label_writer::get_element_id): New.
            (html_path_label_writer::m_path): New field.
            (html_path_label_writer::m_curr_event_id): New field.
            (html_builder::make_element_for_diagnostic): Pass path to label
            writer.
            * diagnostic-format-html.h
            (html_generation_options::m_show_state_diagrams): New field.
            (html_generation_options::m_show_state_diagram_xml): New field.
            (html_generation_options::m_show_state_diagram_dot_src): New field.
            * diagnostic-format-sarif.cc: Include "xml.h".
            (populate_thread_flow_location_object): If requested, attempt to
            generate xml state and add it to the proeprty bag as
            "gcc/diagnostic_event/xml_state" in xml source form.
            (sarif_generation_options::sarif_generation_options): Initialize
            m_xml_state.
            * diagnostic-format-sarif.h
            (sarif_generation_options::m_xml_state): New field.
            * diagnostic-path.cc: Define INCLUDE_MAP.  Include "xml.h".
            (diagnostic_event::maybe_make_xml_state): New.
            * diagnostic-path.h (class xml::document): New forward decl.
            (diagnostic_event::maybe_make_xml_state): New vfunc decl.
            * diagnostic-state-to-dot.cc: New file.
            * diagnostic-state.h: New file.
            * digraph.cc: Define INCLUDE_STRING and INCLUDE_VECTOR.
            * doc/analyzer.texi: Document state diagrams in html output.
            (__analyzer_dump_dot): New.
            (__analyzer_dump_xml): New.
            * doc/invoke.texi (sarif): Add "xml-state" key.
            (experimental-html): Add keys "show-state-diagrams",
            "show-state-diagrams-dot-src" and "show-state-diagrams-xml".
            * graphviz.cc: Define INCLUDE_MAP, INCLUDE_STRING, and
            INCLUDE_VECTOR.  Include "xml.h", "xml-printer.h", "pex.h" and
            "selftest.h".
            (graphviz_out::graphviz_out): Extract...
            (dot::writer::writer): ...this.
            (graphviz_out::write_indent): Convert to...
            (dot::writer::write_indent): ...this.
            (graphviz_out::print): Use get_pp.
            (graphviz_out::println): Likewise.
            (graphviz_out::begin_tr): Likewise.
            (graphviz_out::end_tr): Likewise.
            (graphviz_out::begin_td): Likewise.
            (graphviz_out::end_td): Likewise.
            (graphviz_out::begin_trtd): Likewise.
            (graphviz_out::end_tdtr): Likewise.
            (dot::ast_node::dump): New.
            (dot::id::id): New.
            (dot::id::print): New.
            (dot::id::is_identifier_p): New.
            (dot::kv_pair::print): New.
            (dot::attr_list::print): New.
            (dot::stmt_list::print): New.
            (dot::stmt_list::add_edge): New.
            (dot::stmt_list::add_attr): New.
            (dot::graph::print): New.
            (dot::stmt_with_attr_list::set_label): New.
            (dot::node_stmt::print): New.
            (dot::attr_stmt::print): New.
            (dot::kv_stmt::print): New.
            (dot::node_id::print): New.
            (dot::port::print): New.
            (dot::edge_stmt::print): New.
            (dot::subgraph::print): New.
            (dot::make_svg_document_buffer_from_graph): New.
            (dot::make_svg_from_graph): New.
            (selftest:test_ids): New.
            (selftest:test_trivial_graph): New.
            (selftest:test_layout_example): New.
            (selftest:graphviz_cc_tests): New.
            * graphviz.h (xml::node): New forward decl.
            (class graphviz_out): Split out into...
            (class dot::writer): ...this new class
            (struct dot::ast_node): New.
            (struct dot::id): New.
            (struct dot::kv_pair): New.
            (struct dot::attr_list): New.
            (struct dot::stmt_list): New.
            (struct dot::graph): New.
            (struct dot::stmt): New.
            (struct dot::stmt_with_attr_list): New.
            (struct dot::node_stmt): New.
            (struct dot::attr_stmt): New.
            (struct dot::kv_stmt): New.
            (enum class dot::compass_pt): New.
            (struct dot::port): New.
            (struct dot::node_id): New.
            (struct dot::edge_stmt): New.
            (struct dot::subgraph): New.
            (dot::make_svg_from_graph): New.
            * opts-diagnostic.cc (sarif_scheme_handler::make_sink): Add
            "xml-state" flag.
            (html_scheme_handler::make_sink): Add flags "show-state-diagrams",
            "show-state-diagram-dot-src", and "show-state-diagram-xml".
            * pex.cc: New file.
            * pex.h: New file.
            * selftest-run-tests.cc (selftest::run_tests): Call
            graphviz_cc_tests.
            * selftest.h (selftest::graphviz_cc_tests): New decl.
            * xml.cc (xml::node_with_children::add_comment): New.
            (xml::node_with_children::find_child_element): New.
            (xml::element::get_attr): New.
            (xml::comment::write_as_xml): New.
            (selftest::test_printer): Add coverage of find_child_element and
            get_attr.
            (selftest::test_comment): New.
            (selftest::xml_cc_tests): Call test_comment.
            * xml.h: New forward decls.
            (xml::node::dyn_cast_text): Use nullptr.
            (xml::node::dyn_cast_element): New vfunc.
            (xml::node_with_children::add_comment): New decl.
            (xml::node_with_children::find_child_element): New decl.
            (xml::element::dyn_cast_element): New vfunc impl.
            (xml::element::get_attr): New decl.
            (struct xml::comment): New xml::node subclass.

    gcc/analyzer/ChangeLog:
            PR other/116792
            * ana-state-to-diagnostic-state.cc: New file.
            * ana-state-to-diagnostic-state.h: New file.
            * checker-event.cc: Include "xml.h".
            (checker_event::checker_event): Initialize m_path.
            (checker_event::prepare_for_emission): Store the path pointer into
            m_path.
            (checker_event::maybe_make_xml_state): New.
            (function_entry_event::function_entry_event): Add "state" param
            and use it to initialize m_state.
            (superedge_event::get_program_state): New.
            (call_event::get_program_state): New.
            (warning_event::get_program_state): New.
            * checker-event.h (checker_event::get_program_state): New vfunc.
            (checker_event::maybe_make_xml_state): New decl.
            (checker_event::m_path): New field.
            (statement_event::get_program_state): New vfunc impl.
            (function_entry_event::function_entry_event): Add "state" param.
            (function_entry_event::get_program_state): New vfunc impl.
            (function_entry_event::m_state): New field.
            (state_change_event::get_program_state): New vfunc impl.
            (superedge_event::get_program_state): New vfunc decl.
            (warning_event::warning_event): Add "program_state_" param and
            copy it.
            (warning_event::get_program_state): New vfunc decl.
            (warning_event::m_program_state): New field.
            * checker-path.h (checker_path::checker_path): Add ext_state param.
            (checker_path::get_ext_state): New accessor.
            (checker_path::m_ext_state): New field.
            * common.h: Define INCLUDE_MAP and INCLUDE_STRING.
            * diagnostic-manager.cc (saved_diagnostic::operator==): Don't
            deduplicate dump_path_diagnostic instances.
            (diagnostic_manager::emit_saved_diagnostic): Pass ext_state to
            checker_path ctor.
            * engine.cc:
            (impl_region_model_context::on_state_leak): Pass old and new state
            to state_machine::on_leak.
            (exploded_node::on_stmt_pre): Implement __analyzer_dump_xml and
            __analyzer_dump_dot.
            * exploded-graph.h (impl_region_model_context::get_state): New.
            * infinite-recursion.cc
            (recursive_function_entry_event::recursive_function_entry_event):
            Add "dst_state" param and pass to function_entry_event ctor.
            (infinite_recursion_diagnostic::add_function_entry_event): Pass
state
            to event ctor.
            * kf-analyzer.cc: Include "analyzer/program-state.h"
            (dump_path_diagnostic::dump_path_diagnostic): Add "state" param.
            (dump_path_diagnostic::get_final_state): New.
            (dump_path_diagnostic::m_state): New field.
            (kf_analyzer_dump_path::impl_call_pre): Pass state to warning.
            * pending-diagnostic.cc
            (pending_diagnostic::add_function_entry_event): Pass state to
            function_entry_event.
            (pending_diagnostic::add_final_event): Likewise to warning_event.
            * pending-diagnostic.h (pending_diagnostic::get_final_state): New
            vfunc decl.
            * program-state.cc: Include "diagnostic-state.h", "graphviz.h" and
            "analyzer/ana-state-to-diagnostic-state.h".
            (program_state::dump_dot): New.
            * program-state.h: Include "text-art/tree-widget.h" and
            "analyzer/store.h".
            (class xml::document): New forward decl.
            (make_xml): New.
            (dump_xml_to_pp): New.
            (dump_xml_to_file): New.
            (dump_xml): New.
            (dump_dot): New.
            * record-layout.cc (record_layout::record_layout): Make param
            const_tree.
            * record-layout.h (item::item): Likewise.
            (item::m_field): Likewise.
            (record_layout::record_layout): Likewise.
            (record_layout::begin): New.
            (record_layout::end): New.
            * region-model.cc
            (exposure_through_uninit_copy::complain_about_fully_uninit_item):
            Use const_tree.
           
(exposure_through_uninit_copy::complain_about_partially_uninit_item):
            Likewise.
            * region-model.h (region_model_context::get_state): New vfunc.
            (noop_region_model_context::get_state): New.
            (region_model_context_decorator::get_state): New.
            * sm-fd.cc (fd_leak::fd_leak): Add "final_state" param and capture
            it if present.
            (fd_leak::get_final_state): New.
            (fd_leak::m_final_state): New.
            (fd_state_machine::on_open): Pass nullptr for new "final_state"
            param.
            (fd_state_machine::on_creat): Likewise.
            (fd_state_machine::on_socket): Likewise.
            (fd_state_machine::on_accept): Likewise.
            (fd_state_machine::on_leak): Add state params and pass new state
            as final state to fd_leak ctor.
            * sm-file.cc: Include "analyzer/program-state.h".
            (file_leak::file_leak): Add "final_state" param and capture it if
            present.
            (file_leak::get_final_state): New.
            (file_leak::m_final_state): New.
            (fileptr_state_machine::on_leak): Add state params and pass new
            state as final state to fd_leak ctor.
            * sm-malloc.cc: Include
            "analyzer/ana-state-to-diagnostic-state.h".
            (malloc_leak::malloc_leak): Add "final_state" param and use it.
            (malloc_leak::get_final_state): New vfunc impl.
            (malloc_leak::m_final_state): New field.
            (malloc_state_machine::on_leak): Add state params; capture final
            state.
            (malloc_state_machine::add_state_to_xml): New.
            * sm.cc (state_machine::on_leak): Add "old_state" and "new_state"
            params.  Use nullptr.
            (state_machine::add_state_to_xml): New.
            (state_machine::add_global_state_to_xml): New.
            * sm.h (class xml_state): New forward decl.
            (state_machine::on_leak): Add state params.
            (state_machine::add_state_to_xml): New vfunc decl.
            (state_machine::add_global_state_to_xml): New vfunc decl.
            * store.h (bit_range::operator<): New.
            * varargs.cc (va_list_leak::va_list_leak): Add final_state param
            and capture it if non-null.
            (va_list_leak::get_final_state): New.
            (va_list_leak::m_final_state): New.
            (va_list_state_machine::on_leak): Add state params and pass final
            state to va_list_leak ctor.

    gcc/testsuite/ChangeLog:
            PR other/116792
            * g++.dg/analyzer/state-diagram.C: New test.
            * gcc.dg/analyzer/analyzer-decls.h (__analyzer_dump_dot): New
            decl.
            (__analyzer_dump_xml): New decl.
            * gcc.dg/analyzer/state-diagram-1-sarif.py: New test script.
            * gcc.dg/analyzer/state-diagram-1.c: New test.
            * gcc.dg/analyzer/state-diagram-2.c: New test.
            * gcc.dg/analyzer/state-diagram-3.c: New test.
            * gcc.dg/analyzer/state-diagram-4.c: New test.
            * gcc.dg/analyzer/state-diagram-5-html.py: New test script.
            * gcc.dg/analyzer/state-diagram-5-sarif.py: New test script.
            * gcc.dg/analyzer/state-diagram-5.c: New test.
            * gcc.dg/plugin/analyzer_cpython_plugin.cc: Define INCLUDE_STRING.
            * gcc.dg/plugin/analyzer_gil_plugin.cc: Likewise.
            * gcc.dg/plugin/analyzer_kernel_plugin.cc: Likewise.
            * gcc.dg/plugin/analyzer_known_fns_plugin.cc: Likewise.
            * lib/htmltest.py (ns): Add SVG namespace.
            * lib/sarif.py (get_result_by_index): New.
            (get_xml_state): New.

    Signed-off-by: David Malcolm <dmalc...@redhat.com>

Reply via email to