Muhammad Kamran <[email protected]> writes:
> The pattern expands uaddc sum into:
> cmp openand[4], #1
> adcs operand[0], operand[2], operand[3]
> cset operand[1], cs
>
> Carry chained uaddc expansion results in a rount trip CC -> register -> CC.
> Remove that round-trip as CSET-like materialization does not clobber flags,
> so CC already carries the same bit.
>
> gcc/ChangeLog:
>
> * config/aarch64/aarch64.md
> (uaddc<mode>5): Expansion pattern for uaddc<m>5
> (*aarch64_setc_from_carry_combine_cc_c_<mode>) Post-combine no-op split
> pattern for carry chain elimination in CC_Cmode
> (*aarch64_setc_from_carry_combine_cc_adc_<mode> Post-combine no-op
> split for carry chain elimination in CC_ADCmode
>
> gcc/testsuite/ChangeLog:
>
> * gcc.target/aarch64/uaddcm5.c: test chained uaddc expansion
> ---
> gcc/config/aarch64/aarch64.md | 60 ++++++++++++++++++++++
> gcc/testsuite/gcc.target/aarch64/uaddcm5.c | 26 ++++++++++
> 2 files changed, 86 insertions(+)
> create mode 100644 gcc/testsuite/gcc.target/aarch64/uaddcm5.c
>
> diff --git a/gcc/config/aarch64/aarch64.md b/gcc/config/aarch64/aarch64.md
> index df5f72ab94c..e875bcaec28 100644
> --- a/gcc/config/aarch64/aarch64.md
> +++ b/gcc/config/aarch64/aarch64.md
> @@ -3390,6 +3390,66 @@ (define_insn "*add<mode>3_carryinC"
> [(set_attr "type" "adc_reg")]
> )
>
> +(define_expand "uaddc<mode>5"
> + [(match_operand:GPI 0 "register_operand")
> + (match_operand:GPI 1 "register_operand")
> + (match_operand:GPI 2 "register_operand")
> + (match_operand:GPI 3 "register_operand")
> + (match_operand:GPI 4 "nonmemory_operand")]
> + ""
> +{
> + if (operands[4] == const0_rtx)
> + {
> + emit_insn (gen_add<mode>3_compareC (operands[0], operands[2],
> + operands[3]));
> + rtx cc = gen_rtx_REG (CC_Cmode, CC_REGNUM);
> + rtx pat = gen_rtx_LTU (<MODE>mode, cc, const0_rtx);
> + emit_insn (gen_rtx_SET (operands[1], pat));
> + }
> + else
> + {
> + rtx cin = force_reg (<MODE>mode, operands[4]);
> + rtx cc = gen_rtx_REG (CC_Cmode, CC_REGNUM);
> + rtx adc_cc = gen_rtx_REG (CC_ADCmode, CC_REGNUM);
> + rtx cmp = gen_rtx_COMPARE (CC_Cmode,
> + gen_rtx_PLUS (<MODE>mode, cin, constm1_rtx),
> + cin);
> + emit_insn (gen_rtx_SET (cc, cmp));
> + emit_insn (gen_add<mode>3_carryinC (operands[0], operands[2],
> + operands[3]));
> + rtx pat = gen_rtx_GEU (<MODE>mode, adc_cc, const0_rtx);
> + emit_insn (gen_rtx_SET (operands[1], pat));
> + }
> + DONE;
> +})
> +
> +;; The uaddc expansion results in a rount trip CC -> register -> CC. Remove
> +;; that round-trip as CSET-like materialization does not clobber flags, so CC
> +;; already carries the same bit.
> +(define_insn_and_split "*aarch64_setc_from_carry_combine_cc_c_<mode>"
> + [(set (reg:CC_C CC_REGNUM)
> + (compare:CC_C
> + (neg:GPI
> + (geu:GPI (reg:CC_C CC_REGNUM) (const_int 0)))
> + (ltu:GPI (reg:CC_C CC_REGNUM) (const_int 0))))]
> + ""
> + "#"
> + "&& 1"
> + [(const_int 0)]
> + "emit_note (NOTE_INSN_DELETED); DONE;")
If we want this kind of simplification to be target-specific,
I think we should add a dedicated mechanism for it, rather than using
define_insn_and_split. There are two reasons, one kind of subjective
and one more practical:
(1) The instruction above is never really intended to be an instruction
and is not needed to preserve RTL semantics (since it's a nop in
RTL terms as well as machine terms). It also adds overhead by
taking up insn codes and requiring entries in recog_data etc.
In other words, it seems like we're using define_insn_and_split
because it's already there, rather than because it's what we would
choose if we were starting from scratch.
(2) As a simplification rule, the above is really:
(compare:CC_C (neg:M (geu:M X (const_int 0)))
(ltu:M X (const_int 0)))
-> X
Expressing this as an instruction pattern is less powerful than
using open-coded simplify-rtx.cc rules, again for two reasons:
(a) The symbolic rule above works for all X, including the rhs of a
(reg:CC_C CC_REGNUM) definition. The define_insn_and_split only
works for (reg:CC_C CC_REGNUM) itself.
(b) A simplify-rtx.cc rule would expose the "nopness" straight away,
enabling follow-on optimisations in the same RTL pass.
The instruction pattern only exposes the nopness at the next
split point.
FWIW, I remember writing a patch once to add simplification rules to md
files, with the aim of using them to simplify UNSPECs. Can't remember
whether I ever posted it though, and it would have been on an Arm
machine that I no longer have access to.
That said, the above rule seems like it would work for all CC modes on
all STORE_FLAG_VALUE==1 targets, unless I'm missing something. The rule
below is an inversion by means of a mode change, which could be expressed
using a target hook to say that CC_ADCmode and CC_Cmode are mutual inverses
for all operations that are defined on them (that is, LTU and GEU).
Thanks,
Richard
> +
> +(define_insn_and_split "*aarch64_setc_from_carry_combine_cc_adc_<mode>"
> + [(set (reg:CC_C CC_REGNUM)
> + (compare:CC_C
> + (neg:GPI
> + (ltu:GPI (reg:CC_ADC CC_REGNUM) (const_int 0)))
> + (geu:GPI (reg:CC_ADC CC_REGNUM) (const_int 0))))]
> + ""
> + "#"
> + "&& 1"
> + [(const_int 0)]
> + "emit_note (NOTE_INSN_DELETED); DONE;")
> +
> (define_expand "add<mode>3_carryinV"
> [(parallel
> [(set (reg:CC_V CC_REGNUM)
> diff --git a/gcc/testsuite/gcc.target/aarch64/uaddcm5.c
> b/gcc/testsuite/gcc.target/aarch64/uaddcm5.c
> new file mode 100644
> index 00000000000..39a3508b066
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/uaddcm5.c
> @@ -0,0 +1,26 @@
> +/* { dg-do compile } */
> +/* { dg-options "-O2" } */
> +
> +static unsigned long
> +uaddc (unsigned long x, unsigned long y, unsigned long carry_in, unsigned
> long *carry_out)
> +{
> + unsigned long r;
> + unsigned long c1 = __builtin_add_overflow (x, y, &r);
> + unsigned long c2 = __builtin_add_overflow (r, carry_in, &r);
> + *carry_out = c1 + c2;
> + return r;
> +}
> +
> +void
> +foo (unsigned long *p, unsigned long *q)
> +{
> + unsigned long c;
> + p[0] = uaddc (p[0], q[0], 0, &c);
> + p[1] = uaddc (p[1], q[1], c, &c);
> + p[2] = uaddc (p[2], q[2], c, &c);
> + p[3] = uaddc (p[3], q[3], c, &c);
> +}
> +
> +
> +/* { dg-final { scan-assembler-times "adcs\\t\[xw\]\[0-9\]+, \[xw\]\[0-9\]+,
> \[xw\]\[0-9\]+" 3 } } */
> +/* { dg-final { scan-assembler-not "cset\\t\[xw\]\[0-9\]+, cs+.*" } } */
> \ No newline at end of file
