> > I don't think you kept this logic in the new code? I really apologize for late reply. I missed that you wait for it. > > To be honest, I didn't really follow the logic here. Thinking about the > single-exit case (which the current code is designed to handle), both > the body of the if and the else if logically implement the same update: > > P'(exit) = vf * P(exit) > > I suppose the code in the body of the if is just a more numerically stable way > of doing this?
Here the code just trying to be more robust for broken profiles. I.e. exit probability should correspond to the iteration count determined by header_count/entry_count. If entry_count is non-zero and we can divie, it forcingly set exit probability to it, so the profile will end up being flow consistent even if it was not before. In the remaining case, If entry_count happens to be 0 we only can scale exit_count. The second test should prevent the situation wehre multiplying it by vf will result in value >=1 which will end up predicting the loop as never iterating. It is relatively common that VF > iteration count as predicted by profile, since static profile estimation is not very good on it and it predicts most of loops to iterate approx 3x since that is the expected loop iterations of a random loop in spec. > diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc > index 3de1ea646b4..71787301e6d 100644 > --- a/gcc/tree-vect-loop.cc > +++ b/gcc/tree-vect-loop.cc > @@ -12092,6 +12092,9 @@ scale_profile_for_vect_loop (class loop *loop, edge > exit_e, unsigned vf, bool fl > sensible profile. */ > else if (exit_e->probability < profile_probability::always () / (vf * 2)) > set_edge_probability_and_rescale_others (exit_e, exit_e->probability * > vf); > + else > + gcc_unreachable (); > + > > on top of current trunk, and the unreachable wasn't hit once. I suppose this happens because entry_count needs to be non-zero for maybe_hot_loop predicate to return true, so we do not vectorize in case of this profile breakage. Not sure if it is a good idea - i.e. for auto-fdo 0s on wrong spots are quite common. I will look into it. Honza > > Thanks, > Alex > > > > > Honza > > >- set_edge_probability_and_rescale_others (exit_e, exit_e->probability > > >* vf); > > >- loop->latch->count = single_pred_edge (loop->latch)->count (); > > >- > > >- scale_loop_profile (loop, profile_probability::always () / vf, > > >- get_likely_max_loop_iterations_int (loop)); > > >+ const auto likely_max_niters = get_likely_max_loop_iterations_int > > >(loop); > > >+ scale_loop_profile_hold_exit_counts (loop, > > >+ profile_probability::always () / vf, > > >+ likely_max_niters); > >
