Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-26 Thread Martin Liška
Hello Honza. Thank you for the fix! Martin

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-23 Thread Jan Hubicka
Hi, it seems that there is additional bug in merging which caused the extra 0 counts. The attached patch imprves the stats on gcc dir of profiledbootstrap: stats for indirect_call: total: 9876 invalid: 705 tracked values: 0 values: 7189 times (72.79%) 1 values: 1890 times (19

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-23 Thread Martin Liška
On 1/23/20 1:41 PM, Jan Hubicka wrote: So my bet is that before the patch we had a bogus code. We detected invalid stated with hiving first counter value == -1. Which could be also reached with decrement of all values (0 - 1 == -1). Maybe we would be interested in usage of a huge negative number

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-23 Thread Jan Hubicka
> So my bet is that before the patch we had a bogus code. We detected invalid > stated with hiving first counter value == -1. Which could be also reached > with decrement of all values (0 - 1 == -1). > > Maybe we would be interested in usage of a huge negative number to reflect > invalid state? A

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-23 Thread Martin Liška
On 1/22/20 2:35 PM, Jan Hubicka wrote: On 1/22/20 12:09 PM, Martin Liška wrote: stats for indirect_call:   total: 9210   invalid: 600   tracked values: 0 values: 6280 times (68.19%) 1 values: 1856 times (20.15%) 2 values:  264 times (2.87%) 3 values:

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-22 Thread Jan Hubicka
> On 1/22/20 12:09 PM, Martin Liška wrote: > > stats for indirect_call: > >   total: 9210 > >   invalid: 600 > >   tracked values: > > 0 values: 6280 times (68.19%) > > 1 values: 1856 times (20.15%) > > 2 values:  264 times (2.87%) > > 3 values:  157 times (1.

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-22 Thread Martin Liška
On 1/22/20 12:09 PM, Martin Liška wrote: stats for indirect_call:   total: 9210   invalid: 600   tracked values: 0 values: 6280 times (68.19%) 1 values: 1856 times (20.15%) 2 values:  264 times (2.87%) 3 values:  157 times (1.70%) 4 values:   53 tim

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-22 Thread Martin Liška
On 1/22/20 1:19 PM, Jan Hubicka wrote: I think you can save an instruction in the loop if you drop "&& empty_counter == -1" My motivation was to find a first empty slot. Which means we'll have less holes, but it's probably a micro-optimization. I'm going to install the patch without the "&& e

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-22 Thread Jan Hubicka
> 2020-01-22 Martin Liska > > PR tree-optimization/92924 > * libgcov-profiler.c (__gcov_topn_values_profiler_body): First > try to find an existing value, then find an empty slot > if not found. This looks good, one nit below. > --- > libgcc/libgcov-profiler.c | 46 +++

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-22 Thread Martin Liška
On 1/22/20 12:27 PM, Jan Hubicka wrote: Hi. I've updated the patch based on Honza's notes and installed it as 5f32f9cf13f99f6295591927950aaf98aa8dba91. Thanks, you omitted the patch, but looking at git log I think there is still problem I commented on in the previous mail: Whoops, sorry for t

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-22 Thread Jan Hubicka
> Hi. > > I've updated the patch based on Honza's notes and installed it > as 5f32f9cf13f99f6295591927950aaf98aa8dba91. Thanks, you omitted the patch, but looking at git log I think there is still problem I commented on in the previous mail: for (unsigned i = 0; i < GCOV_TOPN_VALUES; i++)

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-22 Thread Martin Liška
Hi. I've updated the patch based on Honza's notes and installed it as 5f32f9cf13f99f6295591927950aaf98aa8dba91. Just for the record, there are stats for cc1plus for PGO bootstrap: before: == Stats for gcc == stats for indirect_call: total: 9210 invalid: 760 tracked values: 0 values:

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-21 Thread Martin Liška
On 1/21/20 4:27 PM, Jan Hubicka wrote: If distro build takes seriously FDO and reproducibility it will eventually run into the limitation for single-threaded programs and we will need to do reproducible runs for multithread apps which means putting indirect call profiles into thread local storage

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-21 Thread Jan Hubicka
> Ok, after personal discussion with Honza that I had, > we should be more conservative and prune only > a run-time TOP N counters before we merge them. > The patch does that. Can you please Honza test me the patch of Firefox? > > Thanks, > Martin For a record, we ran tests of Firefox. It does no

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-17 Thread Martin Liška
Ok, after personal discussion with Honza that I had, we should be more conservative and prune only a run-time TOP N counters before we merge them. The patch does that. Can you please Honza test me the patch of Firefox? Thanks, Martin >From 7fe1e6a59139ae00cefd1f5edf082d428952203e Mon Sep 17 00:00

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-15 Thread Martin Liška
Hi. So I'm sending first version of the relaxation patch and a demonstration source file that can demonstrate how me merge profile for indirect functions provided on input. There are some demonstrations: 1) rm indir-call.gcda -f ; ./a.out ABCD && gcov-dump -l indir-call.gcda | grep -A2 'indirec

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-10 Thread Jan Hubicka
> > > > counter less than total_count/N (they are not useful anyway). > > > > However it seems that I missed a problem here. With step 2 and 4 the > > merge is not distributive. > > > > I added 2 and 4 trying to work around the fact that we have no > > convenient place to do pruning if mergi

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-10 Thread Martin Liška
On 1/9/20 10:51 AM, Jan Hubicka wrote: On 1/8/20 3:05 PM, Jan Hubicka wrote: I would still preffer invalidation before streaming (which is fully deterministic) and possibly have option Do you mean __gcov_merge_topn? I suggest we do the following: - have non-deterministic and determini

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-09 Thread Jan Hubicka
> On 1/8/20 3:05 PM, Jan Hubicka wrote: > > > > > > > > > > > I would still preffer invalidation before streaming (which is fully > > > > deterministic) and possibly have option > > > > > > Do you mean __gcov_merge_topn? > > > > I suggest we do the following: > > > > - have non-deterministic

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-09 Thread Martin Liška
On 1/8/20 3:05 PM, Jan Hubicka wrote: I would still preffer invalidation before streaming (which is fully deterministic) and possibly have option Do you mean __gcov_merge_topn? I suggest we do the following: - have non-deterministic and deterministic version of TOPN counter and a fl

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-08 Thread Jan Hubicka
> > > > > I would still preffer invalidation before streaming (which is fully > > deterministic) and possibly have option > > Do you mean __gcov_merge_topn? I suggest we do the following: - have non-deterministic and deterministic version of TOPN counter and a flag chosing between determi

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-08 Thread Martin Liška
On 1/8/20 1:24 PM, Jan Hubicka wrote: On 1/8/20 11:35 AM, Jan Hubicka wrote: Hi, Just to explain better what I am worried about. The overall sum of counters in TOPN does not have very good meaning if you have more than N target. Lets for simplicity assume that we have TOPN for N=1 (i.e. old co

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-08 Thread Jan Hubicka
> On 1/8/20 11:35 AM, Jan Hubicka wrote: > > Hi, > > Just to explain better what I am worried about. The overall sum of > > counters in TOPN does not have very good meaning if you have more than N > > target. > > > > Lets for simplicity assume that we have TOPN for N=1 (i.e. old code). It > > gua

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-08 Thread Martin Liška
On 1/8/20 11:35 AM, Jan Hubicka wrote: Hi, Just to explain better what I am worried about. The overall sum of counters in TOPN does not have very good meaning if you have more than N target. Lets for simplicity assume that we have TOPN for N=1 (i.e. old code). It guarantees if target X is taken

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-08 Thread Jan Hubicka
Hi, Just to explain better what I am worried about. The overall sum of counters in TOPN does not have very good meaning if you have more than N target. Lets for simplicity assume that we have TOPN for N=1 (i.e. old code). It guarantees if target X is taken by more than 50% of times, it will win,

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-07 Thread Jan Hubicka
> On 1/7/20 11:08 AM, Jan Hubicka wrote: > > > On 1/6/20 8:03 PM, Jan Hubicka wrote: > > > > > > > OK > > > > > > Actually I am not so sure about this patch - how do we ensure > > > > > > reproducibility in this case? > > > > > ISTM that anyone trying to have reproducible builds shouldn't be using

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-07 Thread Martin Liška
On 1/7/20 11:08 AM, Jan Hubicka wrote: On 1/6/20 8:03 PM, Jan Hubicka wrote: OK Actually I am not so sure about this patch - how do we ensure reproducibility in this case? ISTM that anyone trying to have reproducible builds shouldn't be using PGO based optimizations. OpenSUSE does that. Buil

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-07 Thread Jan Hubicka
> On 1/6/20 8:03 PM, Jan Hubicka wrote: > > > > > OK > > > > Actually I am not so sure about this patch - how do we ensure > > > > reproducibility in this case? > > > ISTM that anyone trying to have reproducible builds shouldn't be using > > > PGO based optimizations. > > > > OpenSUSE does that. B

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-07 Thread Martin Liška
On 1/6/20 8:03 PM, Jan Hubicka wrote: OK Actually I am not so sure about this patch - how do we ensure reproducibility in this case? ISTM that anyone trying to have reproducible builds shouldn't be using PGO based optimizations. OpenSUSE does that. Builds are supposed to be reproducible + PGO

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-06 Thread Jan Hubicka
> > > OK > > Actually I am not so sure about this patch - how do we ensure > > reproducibility in this case? > ISTM that anyone trying to have reproducible builds shouldn't be using > PGO based optimizations. OpenSUSE does that. Builds are supposed to be reproducible + PGO is used for number of co

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-06 Thread Jeff Law
On Mon, 2020-01-06 at 19:23 +0100, Jan Hubicka wrote: > > On Mon, 2020-01-06 at 15:08 +0100, Martin Liška wrote: > > > Hi. > > > > > > As Honza noticed in the PR, we are quite strict about TOP N > > > counter invalidation due to multiple values that can't > > > fit in a counter. We due it in order

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-06 Thread Jan Hubicka
> On Mon, 2020-01-06 at 15:08 +0100, Martin Liška wrote: > > Hi. > > > > As Honza noticed in the PR, we are quite strict about TOP N > > counter invalidation due to multiple values that can't > > fit in a counter. We due it in order to have a reproducible > > builds. I guess we should do a comprom

Re: [PATCH] Relax invalidation of TOP N counters in PGO.

2020-01-06 Thread Jeff Law
On Mon, 2020-01-06 at 15:08 +0100, Martin Liška wrote: > Hi. > > As Honza noticed in the PR, we are quite strict about TOP N > counter invalidation due to multiple values that can't > fit in a counter. We due it in order to have a reproducible > builds. I guess we should do a compromise in between