On Tue, Nov 15, 2016 at 1:57 PM, Richard Sandiford <richard.sandif...@arm.com> wrote: > vect_transform_loop has to reduce three iteration counts by > the vectorisation factor: nb_iterations_upper_bound, > nb_iterations_likely_upper_bound and nb_iterations_estimate. > All three are latch execution counts rather than loop body > execution counts. The calculations were taking that into > account for the first two, but not for nb_iterations_estimate. > > This patch updates the way the calculations are done to fix > this and to add a bit more commentary about what is going on. > > Tested on aarch64-linux-gnu and x86_64-linux-gnu. OK to install?
Ok. Richard. > Thanks, > Richard > > > [ This patch is part of the SVE series posted here: > https://gcc.gnu.org/ml/gcc/2016-11/msg00030.html ] > > gcc/ > 2016-11-15 Richard Sandiford <richard.sandif...@arm.com> > Alan Hayward <alan.hayw...@arm.com> > David Sherwood <david.sherw...@arm.com> > > * tree-vect-loop.c (vect_transform_loop): Protect the updates of > all three iteration counts with an any_* test. Use a single update > for each count. Fix the calculation of nb_iterations_estimate. > > diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c > index 1cd9c72..53570f3 100644 > --- a/gcc/tree-vect-loop.c > +++ b/gcc/tree-vect-loop.c > @@ -7043,27 +7043,25 @@ vect_transform_loop (loop_vec_info loop_vinfo) > /* Reduce loop iterations by the vectorization factor. */ > scale_loop_profile (loop, GCOV_COMPUTE_SCALE (1, vf), > expected_iterations / vf); > - if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo)) > - { > - if (loop->nb_iterations_upper_bound != 0) > - loop->nb_iterations_upper_bound = loop->nb_iterations_upper_bound - > 1; > - if (loop->nb_iterations_likely_upper_bound != 0) > - loop->nb_iterations_likely_upper_bound > - = loop->nb_iterations_likely_upper_bound - 1; > - } > - loop->nb_iterations_upper_bound > - = wi::udiv_floor (loop->nb_iterations_upper_bound + 1, vf) - 1; > - loop->nb_iterations_likely_upper_bound > - = wi::udiv_floor (loop->nb_iterations_likely_upper_bound + 1, vf) - 1; > - > + /* The minimum number of iterations performed by the epilogue. This > + is 1 when peeling for gaps because we always need a final scalar > + iteration. */ > + int min_epilogue_iters = LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo) ? 1 : 0; > + /* +1 to convert latch counts to loop iteration counts, > + -min_epilogue_iters to remove iterations that cannot be performed > + by the vector code. */ > + int bias = 1 - min_epilogue_iters; > + /* In these calculations the "- 1" converts loop iteration counts > + back to latch counts. */ > + if (loop->any_upper_bound) > + loop->nb_iterations_upper_bound > + = wi::udiv_floor (loop->nb_iterations_upper_bound + bias, vf) - 1; > + if (loop->any_likely_upper_bound) > + loop->nb_iterations_likely_upper_bound > + = wi::udiv_floor (loop->nb_iterations_likely_upper_bound + bias, vf) - > 1; > if (loop->any_estimate) > - { > - loop->nb_iterations_estimate > - = wi::udiv_floor (loop->nb_iterations_estimate, vf); > - if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo) > - && loop->nb_iterations_estimate != 0) > - loop->nb_iterations_estimate = loop->nb_iterations_estimate - 1; > - } > + loop->nb_iterations_estimate > + = wi::udiv_floor (loop->nb_iterations_estimate + bias, vf) - 1; > > if (dump_enabled_p ()) > { >
