The following fixes the SLP FMA patterns to preserve reduction
info and the reduction vectorization to consider internal function
call defs for the reduction stmt.
Bootstrap & regtest running on x86_64-unknown-linux-gnu, Andre
verified we're not turning an ICE into a wrong-code bug
(.COMPLEX_MUL now appears in the reduction chain).
Note there's a testcase for the ICE which adds -march=armv8.3-a
and a testcase for correctness which doesn't since I didn't find
any dg effective target verifying armv8.3-a code can run.
2021-06-09 Richard Biener <rguent...@suse.de>
PR tree-optimization/100981
* tree-vect-loop.c (vect_create_epilog_for_reduction): Use
gimple_get_lhs to also handle calls.
* tree-vect-slp-patterns.c (complex_pattern::build): Transfer
reduction info.
* gfortran.dg/vect/pr100981-1.f90: New testcase.
* gfortran.dg/vect/pr100981-2.f90: Likewise.
---
gcc/tree-vect-loop.c | 2 +-
gcc/tree-vect-slp-patterns.c | 5 ++++-
2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c
index ba36348b835..ee79808472c 100644
--- a/gcc/tree-vect-loop.c
+++ b/gcc/tree-vect-loop.c
@@ -5247,7 +5247,7 @@ vect_create_epilog_for_reduction (loop_vec_info
loop_vinfo,
gcc_assert (STMT_VINFO_RELATED_STMT (orig_stmt_info) == stmt_info);
}
- scalar_dest = gimple_assign_lhs (orig_stmt_info->stmt);
+ scalar_dest = gimple_get_lhs (orig_stmt_info->stmt);
scalar_type = TREE_TYPE (scalar_dest);
scalar_results.create (group_size);
new_scalar_dest = vect_create_destination_var (scalar_dest, NULL);
diff --git a/gcc/tree-vect-slp-patterns.c b/gcc/tree-vect-slp-patterns.c
index b25655c9876..2ed49cd9edc 100644
--- a/gcc/tree-vect-slp-patterns.c
+++ b/gcc/tree-vect-slp-patterns.c
@@ -544,6 +544,8 @@ complex_pattern::build (vec_info *vinfo)
{
/* Calculate the location of the statement in NODE to replace. */
stmt_info = SLP_TREE_REPRESENTATIVE (node);
+ stmt_vec_info reduc_def
+ = STMT_VINFO_REDUC_DEF (vect_orig_stmt (stmt_info));
gimple* old_stmt = STMT_VINFO_STMT (stmt_info);
tree lhs_old_stmt = gimple_get_lhs (old_stmt);
tree type = TREE_TYPE (lhs_old_stmt);
@@ -568,9 +570,10 @@ complex_pattern::build (vec_info *vinfo)
= vinfo->add_pattern_stmt (call_stmt, stmt_info);
/* Make sure to mark the representative statement pure_slp and
- relevant. */
+ relevant and transfer reduction info. */
STMT_VINFO_RELEVANT (call_stmt_info) = vect_used_in_scope;
STMT_SLP_TYPE (call_stmt_info) = pure_slp;
+ STMT_VINFO_REDUC_DEF (call_stmt_info) = reduc_def;
gimple_set_bb (call_stmt, gimple_bb (stmt_info->stmt));
STMT_VINFO_VECTYPE (call_stmt_info) = SLP_TREE_VECTYPE (node);
--
2.26.2
