Hi,
This patch is authored-by: Zhongjie Guo <[email protected]>
AVX512 vector comparisons produce mask results, but the instructions
operate on vector operands. The instruction cost should therefore be
based on the operand vector mode rather than the mask result mode.
The mismatch is especially visible on AVX512 split-regs targets, where
ix86_vec_cost scales 512-bit vector operations. Using the mask result
type misses that scaling and can make 512-bit vectorization look too
cheap.
For mask-producing comparisons, use the operand vector type recorded on
the SLP children. If no operand type is recorded, derive a related vector
type from the scalar comparison type without changing the vectorizer's
chosen modes. Mask-vs-mask comparisons keep their mask result type and
continue to be costed as mask operations.
Bootstrapped and regtested on a hygon c86-4g-m7 machine.
Is it ok for trunk?
BR,
Kewen
-----
gcc/ChangeLog:
* config/i386/i386.cc (ix86_vector_costs::add_stmt_cost): Use the
comparison operand vector mode for mask-producing vector comparisons.
gcc/testsuite/ChangeLog:
* gcc.target/i386/vect-compare-cost.c: New test.
Signed-off-by: Zhongjie Guo <[email protected]>
---
gcc/config/i386/i386.cc | 21 +++++++++++++++++++
.../gcc.target/i386/vect-compare-cost.c | 14 +++++++++++++
2 files changed, 35 insertions(+)
create mode 100644 gcc/testsuite/gcc.target/i386/vect-compare-cost.c
diff --git a/gcc/config/i386/i386.cc b/gcc/config/i386/i386.cc
index 2945081234b..53d47596fb7 100644
--- a/gcc/config/i386/i386.cc
+++ b/gcc/config/i386/i386.cc
@@ -26497,6 +26497,27 @@ ix86_vector_costs::add_stmt_cost (int count,
vect_cost_for_stmt kind,
default:
if (truth_value_p (subcode))
{
+ if (kind == vector_stmt
+ && vectype
+ && VECTOR_BOOLEAN_TYPE_P (vectype))
+ {
+ tree operand_vectype
+ = SLP_TREE_VECTYPE (SLP_TREE_CHILDREN (node)[0]);
+ if (!operand_vectype)
+ operand_vectype
+ = SLP_TREE_VECTYPE (SLP_TREE_CHILDREN (node)[1]);
+
+ if (!operand_vectype)
+ if (tree cmp_type = vect_comparison_type (stmt_info))
+ if (!VECT_SCALAR_BOOLEAN_TYPE_P (cmp_type))
+ operand_vectype
+ = get_related_vectype_for_scalar_type
+ (m_vinfo->vector_mode, cmp_type);
+
+ if (operand_vectype)
+ mode = TYPE_MODE (operand_vectype);
+ }
+
if (SSE_FLOAT_MODE_SSEMATH_OR_HFBF_P (mode))
/* CMPccS? insructions are cheap, so use sse_op. While they
produce a mask which may need to be turned to 0/1 by and,
diff --git a/gcc/testsuite/gcc.target/i386/vect-compare-cost.c
b/gcc/testsuite/gcc.target/i386/vect-compare-cost.c
new file mode 100644
index 00000000000..5fb04773b4a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/i386/vect-compare-cost.c
@@ -0,0 +1,14 @@
+/* { dg-do compile } */
+/* { dg-options "-O3 -mavx512f -mtune=c86-4g-m7 --param
ix86-vect-compare-costs=1 -fdump-tree-vect-details" } */
+
+int
+find_ll (const unsigned long long *p, unsigned long long x, int n)
+{
+ for (int i = 0; i < n; ++i)
+ if (p[i] == x)
+ return i;
+ return -1;
+}
+
+/* The compare cost should make c86-4g-m7 prefer 32-byte vectors. */
+/* { dg-final { scan-tree-dump "loop vectorized using 32 byte vectors" "vect"
} } */
--
2.34.1
