[pushed][PR117946][LRA]: When assigning hard reg use biggest mode to check ira_prohibited_class_mode_regs

[Vladimir Makarov]

The following patch fixes

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=117946
The patch was successfully tested and bootstrapped on x86_64, ppc64le, 
aarch64.

commit 6fc3da8fa2af1d4ee154ea803636eabde358b553
Author: Vladimir N. Makarov <vmaka...@redhat.com>
Date:   Tue Dec 10 12:50:27 2024 -0500

    [PR117946][LRA]: When assigning hard reg use biggest mode to check ira_prohibited_class_mode_regs
    
    A pseudo in the PR test case gets hard reg 43 which is x86 r15 (after
    r15, xmm regs go).  The pseudo is of INT_SSE_CLASS and SImode but is
    used in TImode as paradoxical subreg.  r15 in TImode is wrong and does
    not satisfy constraint 'r'.  Therefore LRA creates moves involving the
    pseudo in TImode until the limit of reload insns is achieved.
    Unfortunately x86 hard_regno_mode_ok (as some hooks for other targets)
    says that it is ok to use r15 for TImode pseudo.  Therefore LRA uses
    ira_prohibited_class_mode_regs for such cases but it was checked
    against native pseudo mode.  The patch fixes it by using the biggest
    pseudo mode.
    
    gcc/ChangeLog:
    
            PR rtl-optimization/117946
            * lra-assigns.cc: (find_hard_regno_for_1): Use the biggest mode to
            check ira_prohibited_class_mode_regs.
    
    gcc/testsuite/ChangeLog:
    
            PR rtl-optimization/117946
            * gcc.target/i386/pr117946.c: New.

diff --git a/gcc/lra-assigns.cc b/gcc/lra-assigns.cc
index 0a14bde5e74..405afc06f57 100644
--- a/gcc/lra-assigns.cc
+++ b/gcc/lra-assigns.cc
@@ -629,13 +629,12 @@ find_hard_regno_for_1 (int regno, int *cost, int try_only_hard_regno,
 	hard_regno = ira_class_hard_regs[rclass][i];
       if (! overlaps_hard_reg_set_p (conflict_set,
 				     PSEUDO_REGNO_MODE (regno), hard_regno)
-	  && targetm.hard_regno_mode_ok (hard_regno,
-					 PSEUDO_REGNO_MODE (regno))
+	  && targetm.hard_regno_mode_ok (hard_regno, PSEUDO_REGNO_MODE (regno))
 	  /* We cannot use prohibited_class_mode_regs for all classes
 	     because it is not defined for all classes.  */
 	  && (ira_allocno_class_translate[rclass] != rclass
 	      || ! TEST_HARD_REG_BIT (ira_prohibited_class_mode_regs
-				      [rclass][PSEUDO_REGNO_MODE (regno)],
+				      [rclass][biggest_mode],
 				      hard_regno))
 	  && ! TEST_HARD_REG_BIT (impossible_start_hard_regs, hard_regno)
 	  && (nregs_diff == 0
diff --git a/gcc/testsuite/gcc.target/i386/pr117946.c b/gcc/testsuite/gcc.target/i386/pr117946.c
new file mode 100644
index 00000000000..7304e01d1a7
--- /dev/null
+++ b/gcc/testsuite/gcc.target/i386/pr117946.c
@@ -0,0 +1,16 @@
+/* { dg-do compile  { target { ! ia32 } } } */
+/* { dg-options "-O -favoid-store-forwarding -mavx10.1 -mprefer-avx128 --param=store-forwarding-max-distance=128 -Wno-psabi" } */
+typedef __attribute__((__vector_size__ (64))) _Decimal32 V;
+
+void
+bar (float, float, float, float, float, _Complex, float, float, float,
+     _BitInt(1023), _BitInt (1023), float, float, float, float, float, float,
+     float, float, float, float, float, float, _Decimal64, float, float, float,
+     V, float, _Decimal64);
+
+void
+foo ()
+{
+  bar (0, 0, 0, 0, 0, 0, 0, __builtin_nand64 ("nan"), 0, 0, 0, 0, 0, 0, 0, 0,
+       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, (V){}, 0, 0);
+}