[Bug tree-optimization/122039] New: [13/14/15/16 Regression] fab could in theory go into an infinite loop and is not as effective any more

[pinskia at gcc dot gnu.org via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=122039

            Bug ID: 122039
           Summary: [13/14/15/16 Regression] fab could in theory go into
                    an infinite loop and is not as effective any more
           Product: gcc
           Version: 16.0
            Status: UNCONFIRMED
          Keywords: missed-optimization
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: pinskia at gcc dot gnu.org
          Reporter: pinskia at gcc dot gnu.org
  Target Milestone: ---

The code does:
```
      for (i = gsi_start_bb (bb); !gsi_end_p (i); )
        {
          if (!callee || !fndecl_built_in_p (callee, BUILT_IN_NORMAL))
            {
              gsi_next (&i);
              continue;
            }

          fcode = DECL_FUNCTION_CODE (callee);
.... // fold and maybe change i

          stmt = gsi_stmt (i);
          update_stmt (stmt);
....
          callee = gimple_call_fndecl (stmt);
          if (!callee
              || !fndecl_built_in_p (callee, fcode))
            gsi_next (&i);
        }
```

So if fold_stmt returns true after a change but the builtin stayed the same we
go back and try again. but if the builtin is different we go to the next stmt.
So if fold_stmt was broken and kept on returning true (though that might cause
other issues for say forwprop ) we would hit an infinite loop.
Now this problem was introduced by  r9-2640-g3d78e00879b425.
and it was:
```
          if (!callee
              || DECL_BUILT_IN_CLASS (callee) != BUILT_IN_NORMAL
              || DECL_FUNCTION_CODE (callee) == fcode)
```
Which meant this meant was if this is not a normal builtin or if this is not
the same builtin as before go to the next statement.

This is more of a missed optimization rather than anything else.