[Bug target/71191] aarch64 and others: __atomic_load;arithmetic;__atomic_compare_exchange loops should be able to generate better code with LL/SC-type constructs than a CAS loop

[dhowells at redhat dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71191

--- Comment #6 from dhowells at redhat dot com <dhowells at redhat dot com> ---
There are a couple of ways the problem could be reduced in scope.  Most of the
constructs that the kernel has that fall into this category are conditional
adds/subtracts:

    typedef struct { int counter; } atomic_t;
    bool atomic_inc_unless_negative(atomic_t *v)
    bool atomic_dec_unless_positive(atomic_t *v)
    bool atomic_inc_not_zero(atomic_t *v)
    bool atomic_dec_if_positive(atomic_t *v)
    bool atomic_add_unless(atomic_t *v, int addend, int unless)

all of which conform to a pattern that looks like:

    ({
        TYPE cur = __atomic_load_n(PTR, __ATOMIC_RELAXED);
        TYPE new;
        bool added = true;

        do {
                if (cur COMPARISON COMPAREND) {
                        added = false;
                        break;
                }
                new = cur + ADDEND;
        } while (!__atomic_compare_exchange_n(PTR,
                                              &cur, new,
                                              false,
                                              __ATOMIC_SEQ_CST,
                                              __ATOMIC_RELAXED));
        added;
    })

where PTR is a pointer to the memory variable to be modified, TYPE is the type
of *PTR, ADDEND and COMPAREND are integer values and COMPARISON is a numeric
comparison.  Then there's:

    bool atomic_inc_not_zero_hint(atomic_t *v, int hint)

This is similar to the above, except that hint is used in place of the initial
__atomic_load_n().  Note that if you're doing LL/SC rather than a CAS loop, the
hint is of no interest.

And then there is:

    int __atomic_add_unless(atomic_t *v, int addend, int unless)

which returns the original value of v->counter instead of an indication as to
the success.  I would probably replace this with something like:

    bool atomic_add_unless_return(atomic_t *v, int addend, int unless, int
*_orig)

and return the original value of v->counter through the *_orig so that it
conforms more closely with the pattern above.


So the two ways to reduce the scope of the problem that I'm thinking of are:

 (1) Add very restricted patterns.  Just a single comparison and an add.

 (2) Add extra intrinsics.  I presume there would need to be one per
comparison:

        bool __atomic_fetch_add_if_eq(T *ptr, T addend, T comparend, T *_orig,
                                         int memorder_yes, int memorder_no);
        bool __atomic_fetch_add_if_ne(...);
        bool __atomic_fetch_add_if_lt(...);
        bool __atomic_fetch_add_if_le(...);
        bool __atomic_fetch_add_if_gt(...);
        bool __atomic_fetch_add_if_ge(...);