Hi.
We've started discussion the patch with Honza when we started working on
reproducibility of -fprofile-generate/use. The patch replaces pre-allocated
TOP N counters with a dynamical linked list allocation that happens during
profiling. The similar approach is used by Clang and it provides these benefits:
1) addition to linked list is being done atomically, we should not end up
with corrupted profiles
2) we waste a pointer per each key-value-pair, but we reduce memory footprint
for counters that are not used
3) the method provide better stability results, there are collected stats for
GCC
PGO build:
Covered threshold: 0.25
== Stats for /home/marxin/Programming/gcc/objdir/gcc ==
stats for indirect_call:
Total: 9240, total freq: 5999706513, covered freq: 4309848999 (71.83%), missing
freq: 14248293 (0.24%)
Total tuples: 8759 (size before: 9*N=83160, after: 2*N + (2*TUPLE_COUNT)=35998
Histogram:
0 tracked: 6278 (67.94%), >=0.25: 0 (cov. freq: 0 (0.00%))
1 tracked: 1784 (19.31%), >=0.25: 1784 (cov. freq: 2276692405 (37.95%))
2 tracked: 222 (2.40%), >=0.25: 300 (cov. freq: 575634854 (9.59%))
3 tracked: 79 (0.85%), >=0.25: 143 (cov. freq: 220498760 (3.68%))
4 tracked: 78 (0.84%), >=0.25: 135 (cov. freq: 160196399 (2.67%))
5 tracked: 105 (1.14%), >=0.25: 162 (cov. freq: 48169245 (0.80%))
6 tracked: 239 (2.59%), >=0.25: 369 (cov. freq: 115649188 (1.93%))
7 tracked: 135 (1.46%), >=0.25: 210 (cov. freq: 11660200 (0.19%))
8 tracked: 224 (2.42%), >=0.25: 260 (cov. freq: 313294659 (5.22%))
9 tracked: 62 (0.67%), >=0.25: 50 (cov. freq: 222127899 (3.70%))
10 tracked: 2 (0.02%), >=0.25: 3 (cov. freq: 334914 (0.01%))
11 tracked: 2 (0.02%), >=0.25: 1 (cov. freq: 2454696 (0.04%))
12 tracked: 2 (0.02%), >=0.25: 3 (cov. freq: 981658 (0.02%))
13 tracked: 4 (0.04%), >=0.25: 3 (cov. freq: 23617640 (0.39%))
14 tracked: 4 (0.04%), >=0.25: 4 (cov. freq: 9493736 (0.16%))
17 tracked: 2 (0.02%), >=0.25: 3 (cov. freq: 8416719 (0.14%))
20 tracked: 1 (0.01%), >=0.25: 1 (cov. freq: 5041813 (0.08%))
21 tracked: 3 (0.03%), >=0.25: 4 (cov. freq: 23487922 (0.39%))
27 tracked: 1 (0.01%), >=0.25: 1 (cov. freq: 99561333 (1.66%))
28 tracked: 1 (0.01%), >=0.25: 2 (cov. freq: 64012144 (1.07%))
30 tracked: 1 (0.01%), >=0.25: 1 (cov. freq: 114573384 (1.91%))
32 tracked: 11 (0.12%), >=0.25: 8 (cov. freq: 13949431 (0.23%))
stats for topn:
Total: 1519, total freq: 1660520369, covered freq: 767273373 (46.21%), missing
freq: 84286328 (5.08%)
Total tuples: 5365 (size before: 9*N=13671, after: 2*N + (2*TUPLE_COUNT)=13768
Histogram:
0 tracked: 1032 (67.94%), >=0.25: 0 (cov. freq: 0 (0.00%))
1 tracked: 154 (10.14%), >=0.25: 154 (cov. freq: 240913892 (14.51%))
2 tracked: 43 (2.83%), >=0.25: 62 (cov. freq: 28671331 (1.73%))
3 tracked: 38 (2.50%), >=0.25: 56 (cov. freq: 72179614 (4.35%))
4 tracked: 23 (1.51%), >=0.25: 38 (cov. freq: 9598177 (0.58%))
5 tracked: 36 (2.37%), >=0.25: 48 (cov. freq: 89840680 (5.41%))
6 tracked: 13 (0.86%), >=0.25: 17 (cov. freq: 4548625 (0.27%))
7 tracked: 8 (0.53%), >=0.25: 10 (cov. freq: 23456524 (1.41%))
8 tracked: 6 (0.39%), >=0.25: 6 (cov. freq: 357781 (0.02%))
9 tracked: 2 (0.13%), >=0.25: 2 (cov. freq: 266881 (0.02%))
10 tracked: 5 (0.33%), >=0.25: 5 (cov. freq: 68985 (0.00%))
11 tracked: 6 (0.39%), >=0.25: 7 (cov. freq: 6216337 (0.37%))
12 tracked: 1 (0.07%), >=0.25: 1 (cov. freq: 892 (0.00%))
13 tracked: 11 (0.72%), >=0.25: 11 (cov. freq: 70639704 (4.25%))
14 tracked: 3 (0.20%), >=0.25: 1 (cov. freq: 16524 (0.00%))
15 tracked: 1 (0.07%), >=0.25: 0 (cov. freq: 0 (0.00%))
16 tracked: 1 (0.07%), >=0.25: 1 (cov. freq: 106363 (0.01%))
17 tracked: 1 (0.07%), >=0.25: 0 (cov. freq: 0 (0.00%))
18 tracked: 2 (0.13%), >=0.25: 3 (cov. freq: 32033 (0.00%))
20 tracked: 1 (0.07%), >=0.25: 0 (cov. freq: 0 (0.00%))
21 tracked: 3 (0.20%), >=0.25: 2 (cov. freq: 355742 (0.02%))
22 tracked: 1 (0.07%), >=0.25: 1 (cov. freq: 624 (0.00%))
24 tracked: 3 (0.20%), >=0.25: 2 (cov. freq: 1684949 (0.10%))
25 tracked: 1 (0.07%), >=0.25: 1 (cov. freq: 655 (0.00%))
26 tracked: 2 (0.13%), >=0.25: 2 (cov. freq: 72971 (0.00%))
27 tracked: 1 (0.07%), >=0.25: 0 (cov. freq: 0 (0.00%))
28 tracked: 1 (0.07%), >=0.25: 1 (cov. freq: 14053 (0.00%))
30 tracked: 3 (0.20%), >=0.25: 1 (cov. freq: 99046 (0.01%))
31 tracked: 1 (0.07%), >=0.25: 2 (cov. freq: 2055 (0.00%))
32 tracked: 116 (7.64%), >=0.25: 55 (cov. freq: 218128935 (13.14%))
Patch can bootstrap on x86_64-linux-gnu and survives regression tests.
Ready to be installed in next stage1?
Thanks,
Martin
gcc/ChangeLog:
2020-04-03 Martin Liska <mli...@suse.cz>
* coverage.c (get_coverage_counts): Skip sanity check for TOP N counters
as they have variable number of counters.
* gcov-dump.c (main): Add new option -r.
(print_usage): Likewise.
(tag_counters): All new raw format.
* gcov-io.h (struct gcov_kvp): New.
(GCOV_TOPN_VALUES): Remove.
(GCOV_TOPN_VALUES_COUNTERS): Likewise.
(GCOV_TOPN_MEM_COUNTERS): New.
(GCOV_TOPN_DISK_COUNTERS): Likewise.
(GCOV_TOPN_MAXIMUM_TRACKED_VALUES): Likewise.
* ipa-profile.c (ipa_profile_generate_summary): Use
GCOV_TOPN_MAXIMUM_TRACKED_VALUES.
(ipa_profile_write_edge_summary): Likewise.
(ipa_profile_read_edge_summary): Likewise.
(ipa_profile): Remove usage of GCOV_TOPN_VALUES.
* profile.c (sort_hist_values): Sort variable number
of counters.
(compute_value_histograms): Special case for TOP N counters
that have dynamic number of key-value pairs.
* value-prof.c (dump_histogram_value): Dump variable number
of key-value pairs.
(stream_in_histogram_value): Stream in variable number
of key-value pairs for TOP N counter.
(get_nth_most_common_value): Deal with variable number
of key-value pairs.
(dump_ic_profile): Use GCOV_TOPN_MAXIMUM_TRACKED_VALUES
for loop iteration.
(gimple_find_values_to_profile): Set GCOV_TOPN_MEM_COUNTERS
to n_counters.
* doc/gcov-dump.texi: Document new -r option.
libgcc/ChangeLog:
2020-04-03 Martin Liska <mli...@suse.cz>
* libgcov-driver.c (prune_topn_counter): Remove.
(prune_counters): Likewise.
(merge_one_data): Special case TOP N counters
as they have variable length.
(write_top_counters): New.
(write_one_data): Special case TOP N.
(dump_one_gcov): Do not prune TOP N counters.
* libgcov-merge.c (merge_topn_values_set): Remove.
(__gcov_merge_topn): Use gcov_topn_add_value.
* libgcov-profiler.c (__gcov_topn_values_profiler_body):
Likewise here.
* libgcov.h (gcov_counter_add): New.
(gcov_counter_set_if_null): Likewise.
(gcov_topn_add_value): New.
---
gcc/coverage.c | 7 ++-
gcc/doc/gcov-dump.texi | 5 ++
gcc/gcov-dump.c | 14 ++++-
gcc/gcov-io.h | 22 ++++++--
gcc/ipa-profile.c | 11 ++--
gcc/profile.c | 70 +++++++++++++----------
gcc/value-prof.c | 59 ++++++++++++-------
libgcc/libgcov-driver.c | 116 ++++++++++++++++++--------------------
libgcc/libgcov-merge.c | 103 +++++++--------------------------
libgcc/libgcov-profiler.c | 40 +------------
libgcc/libgcov.h | 87 ++++++++++++++++++++++++++++
11 files changed, 285 insertions(+), 249 deletions(-)
