[Bug middle-end/66313] New: Unsafe factorization of a*b+a*c

[glisse at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66313

            Bug ID: 66313
           Summary: Unsafe factorization of a*b+a*c
           Product: gcc
           Version: 6.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: middle-end
          Assignee: unassigned at gcc dot gnu.org
          Reporter: glisse at gcc dot gnu.org
  Target Milestone: ---

int f(int a, int b, int c){
  return a * b + a * c;
}
int main(){
  return f(0, __INT_MAX__, __INT_MAX__);
}

$ gcc -fsanitize=undefined e.c
$ ./a.out
e.c:2:16: runtime error: signed integer overflow: 2147483647 + 2147483647
cannot be represented in type 'int'

But I thought I was only computing 0+0?

f(-1, __INT_MAX__, 1) yields:
e.c:2:16: runtime error: signed integer overflow: 2147483647 + 1 cannot be
represented in type 'int'
e.c:2:10: runtime error: signed integer overflow: -2147483648 * -1 cannot be
represented in type 'int'

I am not very excited about restricting this transformation to
TYPE_OVERFLOW_WRAPS, but the alternative is to cast to unsigned (and back after
the operations), which isn't so nice either.