[Bug tree-optimization/81110] New: tree-vrp optimize unsigned comparison to signed

[qian.liu at ingenic dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81110

Bug ID: 81110
   Summary: tree-vrp optimize unsigned comparison to signed
   Product: gcc
   Version: 5.2.0
Status: UNCONFIRMED
  Severity: normal
  Priority: P3
 Component: tree-optimization
  Assignee: unassigned at gcc dot gnu.org
  Reporter: qian.liu at ingenic dot com
  Target Milestone: ---
Target: mips-linux-gnu mipsel-linux-gnu

Created attachment 41567
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=41567&action=edit
reduced testcase

The following testcase is miscompiled at -O2 on mips .  An unsigned comparison
operation is optimized to a singed comparion instruciton. When d[0] and d[1]
large enough, will get error result.

unsigned short d[2];
int DiffInt ()
{
  if ((unsigned long)(65536*d[1]+d[0]) < (1<<28)){
return -1;
  }
}

Currently assembly:
li  $3,268435456# 0x1000
slt $2,$2,$3  # error
beq $2,$0,$L4

But expected：
li  $3,268435456# 0x1000
sltu$2,$2,$3 # unsigned 
beq $2,$0,$L4

In tree-vrp dump file test.c.081t.vrp1, unsigned _8 is simplifed to singned _7.
But this is incorrect.

  int _7;
  long unsigned int _8;
  .
  _8 = (long unsigned int) _7;

 -  if (_8 <= 268435455)
 +  if (_7 <= 268435455)

[Bug tree-optimization/81110] tree-vrp optimize unsigned comparison to signed

[qian.liu at ingenic dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81110

--- Comment #1 from Ada Liu  ---
Comment on attachment 41567
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=41567
reduced testcase

>
>unsigned short  d[2];
>__attribute__ ((noinline))int  DiffInt ()
>{
>  if ((unsigned long)(65536*d[1]+d[0]) < (1<<28)){
>return -1;
>  }
>  return 0;
>}
>
>void main()
>{
>  d[0] = 0x;
>  d[1] = 0x;
>  if (DiffInt())
>__builtin_abort();
>  
>}

[Bug tree-optimization/81110] tree-vrp optimize unsigned comparison to signed

[qian.liu at ingenic dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81110

--- Comment #2 from Ada Liu  ---
Comment on attachment 41567
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=41567
reduced testcase

>
>unsigned short  d[2];
>__attribute__ ((noinline))int  DiffInt ()
>{
>  if ((unsigned long)(65536*d[1]+d[0]) < (1<<28)){
>return -1;
>  }
>  return 0;
>}
>
>void main()
>{
>  d[0] = 0x;
>  d[1] = 0x;
>  if (DiffInt())
>__builtin_abort();
>  
>}

[Bug tree-optimization/81110] tree-vrp optimize unsigned comparison to signed

[qian.liu at ingenic dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81110

--- Comment #5 from Ada Liu  ---
(In reply to Andrew Pinski from comment #3)
> 65536*d[1] will overflow if d[1] is 65535 (0x).
> 
> If you did this instead it would be defined:
> 65536u*d[1]
> 
> as that is an unsigned * unsigned short so the unsigned short is implicitly
> converted into unsigned; in the original case, the unsigned short is
> implicitly converted into signed int.

But (65536*d[1]+d[0]) is convered to (unsigned long), should not be treated as
unsigned？

On gcc4.7.2 I can get expected result.