https://gcc.gnu.org/bugzilla/show_bug.cgi?id=128847

            Bug ID: 128847
           Summary: x86: PGO-enabled -funroll-loops is defeated by the
                    default -munroll-only-small-loops
           Product: gcc
           Version: 17.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
          Assignee: unassigned at gcc dot gnu.org
          Reporter: yansendao at 126 dot com
  Target Milestone: ---

Created attachment 66213
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=66213&action=edit
reproducer

With -fprofile-use, GCC 11 enables -funroll-loops and RTL-unrolls medium-sized
hot loops. GCC 13+ enables the same flag internally, but the x86 default
-munroll-only-small-loops still limits unrolling to loops with at most
ix86_cost->small_unroll_ninsns instructions (4 with generic tuning), so such
loops are not unrolled at all.

This was noticed while investigating a performance regression in CPython 3.12:
binascii.b2a_base64() compiled with a PGO+LTO build is ~30% slower with GCC 17
than with GCC 11, and the difference disappears when the module is rebuilt with
-mno-unroll-only-small-loops.

How to reproduce:


/* t.c: the base64 encode loop from CPython Modules/binascii.c */
#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>

static const unsigned char b64[] =
  "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";

size_t enc (const unsigned char *in, size_t n, unsigned char *out)
{
  int bits = 0; unsigned char ch; unsigned int cur = 0; unsigned char *o = out;
  for (; n > 0; n--, in++)
    {
      cur = (cur << 8) | *in;
      bits += 8;
      while (bits >= 6)
        {
          ch = (cur >> (bits - 6)) & 0x3f;
          bits -= 6;
          *o++ = b64[ch];
        }
    }
  if (bits == 2) { *o++ = b64[(cur & 3) << 4]; *o++ = 61; *o++ = 61; }
  else if (bits == 4) { *o++ = b64[(cur & 0xf) << 2]; *o++ = 61; }
  return (size_t) (o - out);
}

int main (void)
{
  size_t n = 1 << 20;
  unsigned char *in = malloc (n), *out = malloc (2 * n + 8);
  for (size_t i = 0; i < n; i++) in[i] = (unsigned char) (i * 131 + 17);
  size_t sum = 0;
  for (int i = 0; i < 200; i++) sum += enc (in, n, out);
  printf ("%zu\n", sum);
  return 0;
}

$ gcc-11 -O2 -fprofile-generate -o t t.c && ./t
$ gcc-11 -O2 -fprofile-use -fprofile-correction -fdump-rtl-loop2_unroll -c t.c
$ grep -E 'Unrolled loop|is too big' t.c.*loop2_unroll
  1 ;; Unrolled loop 1 times, constant # of iterations 80 insns
  2 ;; Unrolled loop 1 times, counting # of iterations in runtime, 52 insns

$ gcc-trunk -O2 -fprofile-use -fprofile-correction -fdump-rtl-loop2_unroll -c
t.c
$ grep -E 'Unrolled loop|is too big' t.c.*loop2_unroll
  6 ;; Not considering loop, is too big

$ gcc-trunk ... -funroll-loops                 # explicit flag
  1 ;; Unrolled loop 1 times, constant # of iterations 74 insns
$ gcc-trunk ... -mno-unroll-only-small-loops
  1 ;; Unrolled loop 1 times, constant # of iterations 74 insns
Expected: a -fprofile-use build should unroll the same loops as GCC 11 did.

Actual: the loop is not unrolled unless -funroll-loops or
-mno-unroll-only-small-loops is passed explicitly, even though -fprofile-use
enabled -funroll-loops internally.

Analysis:

ix86_loop_unroll_adjust() (gcc/config/i386/i386.cc:28177) returns 1 for any
loop larger than ix86_cost->small_unroll_ninsns when the gate is on:


if (ix86_unroll_only_small_loops && !loop->unroll)
  {
    if (loop->ninsns <= ix86_cost->small_unroll_ninsns)
      return MIN (nunroll, ix86_cost->small_unroll_factor);
    else
      return 1;
  }
ix86_override_options_after_change_1() (gcc/config/i386/i386-options.cc:1949)
clears the gate only when -f{,no-}unroll-loops was given on the command line
(OPTS_SET_P (flag_unroll_loops)).

enable_fdo_optimizations() (gcc/opts.cc:2167) enables flag_unroll_loops with
SET_OPTION_IF_UNSET, which sets the value but not the corresponding opts_set
bit, so OPTS_SET_P (flag_unroll_loops) stays false and the x86 hook never
clears the gate. The gate itself was introduced in r13-3932-g8caf155a3d6
("i386: Only enable small loop unrolling in backend [PR 107692]"), which GCC 11
does not have.

Impact (CPython 3.12.15, PGO+LTO, 1 MiB b2a, Hygon C86):


GCC 11 (default)                     1401/1406/1408 us
GCC 17 (default)                     1833/1839/1840 us
GCC 17 + -mno-unroll-only-small-loops 1434/1434/1441 us
Suggested fix: make the implicit flag_unroll_loops enabled by
-fprofile-use/-fauto-profile visible to the x86 hook — either record it in
opts_set in enable_fdo_optimizations(), or let
ix86_override_options_after_change_1() also clear ix86_unroll_only_small_loops
when flag_unroll_loops is enabled by PGO. Alternatively, raise
small_unroll_ninsns under profile feedback instead of disabling the gate.

Reply via email to