[PATCH] simplify-rtx: Fix VOIDmode operand handling in simplify_subreg [PR108805]

[Uros Bizjak via Gcc-patches]

simplify_subreg can return VOIDmode const_int operand and will
cause ICE in simplify_gen_subreg when this operand is passed to it.

The patch prevents VOIDmode temporary from entering simplify_gen_subreg.
We can't process const_int operand any further, since outermode
is not an integer mode here.

2023-02-16  Uroš Bizjak  <ubiz...@gmail.com>

gcc/ChangeLog:

    PR target/108805
    * simplify_rtx.cc (simplify_context::simplify_subreg): Prevent
    VOIDmode const_int result from simplify_subreg from entering
    simplify_gen_subreg.

gcc/testsuite/ChangeLog:

    PR target/108805
    * gcc.dg/pr108805.c: New test.

Bootstrapped and regression tested on x86_64-linux-gnu {,-m32}.

OK for master and release branches?

Uros.

diff --git a/gcc/simplify-rtx.cc b/gcc/simplify-rtx.cc
index 0a1dd88b0a8..281bc418df0 100644
--- a/gcc/simplify-rtx.cc
+++ b/gcc/simplify-rtx.cc
@@ -7664,7 +7664,7 @@ simplify_context::simplify_subreg (machine_mode 
outermode, rtx op,
                            0).exists (&int_outermode))
     {
       rtx tem = simplify_subreg (int_outermode, op, innermode, byte);
-      if (tem)
+      if (tem && GET_MODE (tem) != VOIDmode)
        return simplify_gen_subreg (outermode, tem, GET_MODE (tem), 0);
     }
 
diff --git a/gcc/testsuite/gcc.dg/pr108805.c b/gcc/testsuite/gcc.dg/pr108805.c
new file mode 100644
index 00000000000..280d3f5c377
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/pr108805.c
@@ -0,0 +1,20 @@
+/* { dg-do compile { target longlong64 } } */
+/* { dg-options "-O" } */
+/* { dg-additional-options "-msse2" { target x86_64-*-* i?86-*-* } } */
+
+typedef __INT8_TYPE__ __attribute__((__vector_size__ (4))) U;
+typedef __INT32_TYPE__ __attribute__((__vector_size__ (4))) V;
+typedef __UINT64_TYPE__ __attribute__((__vector_size__ (8))) W;
+
+int i;
+U h;
+W g;
+
+U
+foo (void)
+{
+  W w = i != g;
+  V v = __builtin_convertvector (i | w >> 2, V);
+  U u = (U) v[0] + h;
+  return u;
+}