I was looking at the rs6000_builtin_vectorized_function and noticed that it takes an integer that gives the builtin function index. Unfortunately on the powerpc and cell architectures, the builtin function id overlaps with the system builtins, and you need to look at the builtin class before using the builtin index.
This can be fixed in two ways, vectorizable_function in tree-vect-stmts.c can add a check for DECL_BUILT_IN_CLASS (fndecl) == BUILT_IN_NORMAL or the two ports that use the hook (i386, rs6000) can be modified to take a tree instead of the integer. I tend to think the later is preferable, because it would allow a MD scalar builtin to be vectorized. -- Summary: TARGET_VECTORIZE_BUILTIN_VECTORIZED_FUNCTION can misoptimize when MD builtins overlap with standard builtins Product: gcc Version: 4.5.0 Status: UNCONFIRMED Severity: normal Priority: P3 Component: tree-optimization AssignedTo: meissner at gcc dot gnu dot org ReportedBy: meissner at gcc dot gnu dot org GCC build triplet: powerpc64-unknown-linux-gnu GCC host triplet: powerpc64-unknown-linux-gnu GCC target triplet: powerpc64-unknown-linux-gnu http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43544