llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT--> @llvm/pr-subscribers-clang-driver Author: Fateme Hosseini (fhossein-quic) <details> <summary>Changes</summary> Introduce two new passes for the Hexagon HVX floating-point pipeline, targeting v79+ where QFloat (qf16/qf32) is the native HVX FP format. HexagonXQFloatGenerator lowers IEEE-754 HVX floating-point sequences (sf/hf) to native QFloat (qf16/qf32) operations. QFloat instructions are faster and more power-efficient than their IEEE counterparts, with optional accuracy trade-offs. The pass exposes four modes: * Strict IEEE-754 compliant * IEEE-754 compliant (extended dynamic range and subnormal precision, no IEEE-754 overflow handling) * Lossy subnormals * Legacy HexagonPostRAHandleQFP runs after register allocation and corrects the spill/refill paths. QFloat operands carry four extra precision bits that are silently dropped if the value passes through a spill slot or a non-HVX instruction. The pass uses the Register DataFlow Graph (RDF) to walk use-def chains in non-SSA form, inserts qf->IEEE conversions before spills and non-HVX uses, and rewrites saturating instruction opcodes that can absorb IEEE operands directly. Co-authored-by: Sumanth Gundapaneni <sgundapa@<!-- -->quicinc.com> Co-authored-by: Santanu Das <santdas@<!-- -->qti.qualcomm.com> --- Patch is 438.55 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/198902.diff 52 Files Affected: - (modified) clang/include/clang/Options/Options.td (+3) - (modified) clang/lib/Driver/ToolChains/Clang.cpp (+112) - (modified) clang/lib/Driver/ToolChains/Hexagon.cpp (+23-6) - (added) clang/test/Driver/hexagon-hvx-ieee-qfloat.c (+25) - (added) clang/test/Driver/hexagon-hvx-qfloat-backend.c (+43) - (modified) llvm/include/llvm/CodeGen/RDFGraph.h (+2) - (modified) llvm/lib/Target/Hexagon/CMakeLists.txt (+2) - (modified) llvm/lib/Target/Hexagon/Hexagon.h (+4-1) - (modified) llvm/lib/Target/Hexagon/HexagonInstrInfo.cpp (+24) - (modified) llvm/lib/Target/Hexagon/HexagonInstrInfo.h (+3) - (added) llvm/lib/Target/Hexagon/HexagonPostRAHandleQFP.cpp (+1854) - (modified) llvm/lib/Target/Hexagon/HexagonTargetMachine.cpp (+31-1) - (modified) llvm/lib/Target/Hexagon/HexagonTargetMachine.h (+3) - (added) llvm/lib/Target/Hexagon/HexagonXQFloatGenerator.cpp (+2177) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-add-qf.ll (+157) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-assertion1.ll (+84) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-check-free-reg.ll (+110) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-check-qf-instrs.ll (+73) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-compliant-ieee-mul-qf16.ll (+86) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-compliant-ieee-mul-qf32.ll (+136) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-convert-elim.ll (+77) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-corner-case1.ll (+147) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-fix-invalid-opcode.ll (+72) - (removed) llvm/test/CodeGen/Hexagon/autohvx/xqf-fixup-qfp1.ll (-372) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-handle-conv.ll (+180) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-input-rt.ll (+63) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-lossy-mul-qf16.ll (+74) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-lossy-mul-qf32.ll (+109) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-mode-flags.ll (+76) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-multi-conv.ll (+133) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-multidef.ll (+49) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-normalization-assert.ll (+459) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-conv-double.mir (+120) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-conv-double2.ll (+28) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-copy3.ll (+20) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-fakereg.ll (+130) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-handle-crash.ll (+23) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-handle-crash2.mir (+86) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-handle-qf32-mul.ll (+69) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-legacy-mode.ll (+30) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-subreg2.ll (+99) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-subreg3.ll (+45) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-postra-warnings.ll (+60) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-strict-ieee-mul-qf16.ll (+91) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-strictieee-mul-qf32.ll (+123) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-unary-crash.ll (+25) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-v81/xqf-v81-compliant-ieee-mul-qf32.ll (+109) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-v81/xqf-v81-lossy-mul-qf32.ll (+98) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-v81/xqf-v81-strict-mul-qf32.ll (+119) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-v81/xqf-v81-vsub.ll (+164) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-vsub.ll (+130) - (added) llvm/test/CodeGen/Hexagon/autohvx/xqf-warnings.ll (+143) ``````````diff diff --git a/clang/include/clang/Options/Options.td b/clang/include/clang/Options/Options.td index 4fd892e58df86..c82c00c98d3ef 100644 --- a/clang/include/clang/Options/Options.td +++ b/clang/include/clang/Options/Options.td @@ -6986,6 +6986,9 @@ def mhexagon_hvx_length_EQ : Joined<["-"], "mhvx-length=">, def mhexagon_hvx_qfloat : Flag<["-"], "mhvx-qfloat">, Group<m_hexagon_Features_HVX_Group>, HelpText<"Enable Hexagon HVX QFloat instructions">; +def mhexagon_hvx_qfloat_EQ : Joined<["-"], "mhvx-qfloat=">, + Group<m_hexagon_Features_HVX_Group>, + HelpText<"Enable Hexagon HVX QFloat instructions with mode: strict-ieee, ieee, lossy, legacy (v79+)">; def mno_hexagon_hvx_qfloat : Flag<["-"], "mno-hvx-qfloat">, Group<m_hexagon_Features_HVX_Group>, HelpText<"Disable Hexagon HVX QFloat instructions">; diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp index 8a0efd70e6c0d..b942e74d8933f 100644 --- a/clang/lib/Driver/ToolChains/Clang.cpp +++ b/clang/lib/Driver/ToolChains/Clang.cpp @@ -61,6 +61,7 @@ #include "llvm/TargetParser/RISCVISAInfo.h" #include "llvm/TargetParser/RISCVTargetParser.h" #include <cctype> +#include <iterator> using namespace clang::driver; using namespace clang::driver::tools; @@ -2235,6 +2236,115 @@ void Clang::AddX86TargetArgs(const ArgList &Args, } } +static StringRef getOptionName(StringRef Option, const char Delimiter = '=') { + size_t Index = Option.find(Delimiter); + if (Index != StringRef::npos) + Option = Option.substr(0, Index); + return Option; +} + +static void checkAndRemoveLLVMArg(ArgStringList &CmdArgs, StringRef Opt) { + Opt = getOptionName(Opt); + if (CmdArgs.size() < 2) + return; + + for (auto It = std::next(CmdArgs.begin()); It != CmdArgs.end(); ++It) { + StringRef Option = *It; + if (!Option.starts_with(Opt)) + continue; + Option = getOptionName(Option); + if (Option != Opt) + continue; + if (StringRef(*(It - 1)) != "-mllvm") + continue; + + It = CmdArgs.erase(It); + CmdArgs.erase(It - 1); + return; + } +} + +static void pushBackLLVMArg(ArgStringList &CmdArgs, const char *A) { + checkAndRemoveLLVMArg(CmdArgs, A); + CmdArgs.push_back("-mllvm"); + CmdArgs.push_back(A); +} + +static void addQFloatLossyFastMathArgs(ArgStringList &CmdArgs) { + for (auto It = CmdArgs.begin(), Ie = CmdArgs.end(); It != Ie;) { + StringRef Option = *It; + if (Option == "-fmath-errno" || Option == "-ffp-contract=on") { + It = CmdArgs.erase(It); + Ie = CmdArgs.end(); + } else { + ++It; + } + } + + CmdArgs.push_back("-menable-no-infs"); + CmdArgs.push_back("-menable-no-nans"); + CmdArgs.push_back("-fapprox-func"); + CmdArgs.push_back("-funsafe-math-optimizations"); + CmdArgs.push_back("-fno-signed-zeros"); + CmdArgs.push_back("-mreassociate"); + CmdArgs.push_back("-freciprocal-math"); + CmdArgs.push_back("-ffp-contract=fast"); + CmdArgs.push_back("-ffast-math"); + CmdArgs.push_back("-ffinite-math-only"); + CmdArgs.push_back("-D__FAST_MATH__"); + pushBackLLVMArg(CmdArgs, "-fast-math=true"); +} + +static void addQFloatBackendArg(const Driver &D, const ArgList &Args, + ArgStringList &CmdArgs) { + auto HvxVerOpt = toolchains::HexagonToolChain::GetHVXVersion(Args); + bool HasHVX = HvxVerOpt.has_value(); + std::string HvxVer = HasHVX ? *HvxVerOpt : std::string(); + if (Args.hasArg(options::OPT_mhexagon_hvx, options::OPT_mhexagon_hvx_EQ, + options::OPT_mhexagon_hvx_ieee_fp) && + HasHVX) { + unsigned HvxVerNum = 0; + if (StringRef(HvxVer).drop_front(1).getAsInteger(10, HvxVerNum)) + HvxVerNum = 0; + + if (Arg *A = Args.getLastArg(options::OPT_mhexagon_hvx_qfloat, + options::OPT_mhexagon_hvx_qfloat_EQ, + options::OPT_mhexagon_hvx_ieee_fp)) { + if (HvxVerNum >= 79) { + if (A->getOption().matches(options::OPT_mhexagon_hvx_qfloat_EQ)) { + const char *Mode = + llvm::StringSwitch<const char *>(StringRef(A->getValue()).lower()) + .Case("strict-ieee", "-hexagon-qfloat-mode=strict-ieee") + .Case("ieee", "-hexagon-qfloat-mode=ieee") + .Case("lossy", "-hexagon-qfloat-mode=lossy") + .Case("legacy", "-hexagon-qfloat-mode=legacy") + .Default("-hexagon-qfloat-mode=lossy"); + pushBackLLVMArg(CmdArgs, Mode); + if (strcmp(Mode, "-hexagon-qfloat-mode=lossy") == 0) + addQFloatLossyFastMathArgs(CmdArgs); + } else if (A->getOption().matches(options::OPT_mhexagon_hvx_qfloat)) { + pushBackLLVMArg(CmdArgs, "-hexagon-qfloat-mode=lossy"); + addQFloatLossyFastMathArgs(CmdArgs); + } else { + pushBackLLVMArg(CmdArgs, "-hexagon-qfloat-mode=ieee"); + } + } else { + if (Arg *QFloatArg = + Args.getLastArg(options::OPT_mhexagon_hvx_qfloat, + options::OPT_mhexagon_hvx_qfloat_EQ, + options::OPT_mno_hexagon_hvx_qfloat); + QFloatArg && QFloatArg->getOption().matches( + options::OPT_mhexagon_hvx_qfloat_EQ)) { + D.Diag(diag::warn_drv_unsupported_option_part_for_target) + << QFloatArg->getValue() << QFloatArg->getAsString(Args) + << (std::string("HVX ") + HvxVer + + "; falling back to legacy qfloat mode"); + } + } + } + } +} + void Clang::AddHexagonTargetArgs(const ArgList &Args, ArgStringList &CmdArgs) const { CmdArgs.push_back("-mqdsp6-compat"); @@ -2254,6 +2364,8 @@ void Clang::AddHexagonTargetArgs(const ArgList &Args, } CmdArgs.push_back("-mllvm"); CmdArgs.push_back("-machine-sink-split=0"); + + addQFloatBackendArg(getToolChain().getDriver(), Args, CmdArgs); } void Clang::AddLanaiTargetArgs(const ArgList &Args, diff --git a/clang/lib/Driver/ToolChains/Hexagon.cpp b/clang/lib/Driver/ToolChains/Hexagon.cpp index 0e7055797a1f0..d60b0f201f1e0 100644 --- a/clang/lib/Driver/ToolChains/Hexagon.cpp +++ b/clang/lib/Driver/ToolChains/Hexagon.cpp @@ -106,14 +106,14 @@ static void handleHVXTargetFeatures(const Driver &D, const ArgList &Args, // Handle HVX floating point flags. auto checkFlagHvxVersion = - [&](auto FlagOn, auto FlagOff, + [&](auto FlagOn, auto FlagOnWithModes, auto FlagOff, bool CheckMode, unsigned MinVerNum) -> std::optional<StringRef> { // Return an std::optional<StringRef>: // - std::nullopt indicates a verification failure, or that the flag was not // present in Args. // - Otherwise the returned value is that name of the feature to add // to Features. - Arg *A = Args.getLastArg(FlagOn, FlagOff); + Arg *A = Args.getLastArg(FlagOn, FlagOnWithModes, FlagOff); if (!A) return std::nullopt; @@ -130,17 +130,34 @@ static void handleHVXTargetFeatures(const Driver &D, const ArgList &Args, << withMinus(OptName) << ("v" + std::to_string(HvxVerNum)); return std::nullopt; } + + if (CheckMode && A->getOption().matches(FlagOnWithModes)) { + bool ValidMode = + llvm::StringSwitch<bool>(StringRef(A->getValue()).lower()) + .Cases({"strict-ieee", "ieee", "lossy", "legacy"}, true) + .Default(false); + if (!ValidMode) + D.Diag(diag::err_drv_invalid_value) + << A->getAsString(Args) << A->getValue(); + } return makeFeature(OptName, true); }; - if (auto F = checkFlagHvxVersion(options::OPT_mhexagon_hvx_qfloat, - options::OPT_mno_hexagon_hvx_qfloat, 68)) { + if (auto F = checkFlagHvxVersion( + options::OPT_mhexagon_hvx_qfloat, options::OPT_mhexagon_hvx_qfloat_EQ, + options::OPT_mno_hexagon_hvx_qfloat, /*CheckMode=*/true, 68)) { Features.push_back(*F); } - if (auto F = checkFlagHvxVersion(options::OPT_mhexagon_hvx_ieee_fp, - options::OPT_mno_hexagon_hvx_ieee_fp, 68)) { + if (auto F = checkFlagHvxVersion( + options::OPT_mhexagon_hvx_ieee_fp, options::OPT_mhexagon_hvx_ieee_fp, + options::OPT_mno_hexagon_hvx_ieee_fp, /*CheckMode=*/false, 68)) { Features.push_back(*F); } + + // On v79 and above, there is no IEEE hardware. Treat -mhvx-ieee-fp + // as "qfloat mode ieee". + if (HvxVerNum >= 79 && Args.getLastArg(options::OPT_mhexagon_hvx_ieee_fp)) + Features.push_back("+hvx-qfloat"); } // Hexagon target features. diff --git a/clang/test/Driver/hexagon-hvx-ieee-qfloat.c b/clang/test/Driver/hexagon-hvx-ieee-qfloat.c new file mode 100644 index 0000000000000..ee8dc9de25751 --- /dev/null +++ b/clang/test/Driver/hexagon-hvx-ieee-qfloat.c @@ -0,0 +1,25 @@ +// --------------------------------------------------------------------------- +// Tests for the hvx qfloat target feature and backend flag if -mhvx-ieee-fp is +// passed on v79 and above. +// --------------------------------------------------------------------------- + +// Test for v79, the correct backend flag is passed for -mhvx-ieee-fp. +// CHECK-IEEE: "-mllvm" "-hexagon-qfloat-mode=ieee" +// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \ +// RUN: -mhvx-ieee-fp 2>&1 | FileCheck -check-prefix=CHECK-IEEE %s + +// Test for arches lower than v79 does not pass any backend flag. +// CHECK-MODE-NOT: "-mllvm" "-hexagon-qfloat-mode=" +// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv75 -mhvx \ +// RUN: -mhvx-ieee-fp 2>&1 | FileCheck -check-prefix=CHECK-MODE %s + +// Test for v79, the correct qfloat target feature is set when -mhvx-ieee-fp is +// passed. +// CHECK-HVX-QFLOAT-ON: "-target-feature" "+hvx-qfloat" +// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \ +// RUN: -mhvx-ieee-fp 2>&1 | FileCheck -check-prefix=CHECK-HVX-QFLOAT-ON %s + +// Test for arches lower than v79 does not set the qfloat target feature. +// CHECK-FEATURE-NOT: "-target-feature" "+hvx-qfloat" +// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv75 -mhvx \ +// RUN: -mhvx-ieee-fp 2>&1 | FileCheck -check-prefix=CHECK-FEATURE %s diff --git a/clang/test/Driver/hexagon-hvx-qfloat-backend.c b/clang/test/Driver/hexagon-hvx-qfloat-backend.c new file mode 100644 index 0000000000000..ae6866777acff --- /dev/null +++ b/clang/test/Driver/hexagon-hvx-qfloat-backend.c @@ -0,0 +1,43 @@ +// --------------------------------------------------------------------------- +// Tests for the hvx qfloat modes backend flag. +// --------------------------------------------------------------------------- + +// Test for correct backend flag with case-insensitive values. +// CHECK-STRICT-IEEE: "-mllvm" "-hexagon-qfloat-mode=strict-ieee" +// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \ +// RUN: -mhvx-qfloat=strict-ieee 2>&1 | FileCheck -check-prefix=CHECK-STRICT-IEEE %s +// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \ +// RUN: -mhvx-qfloat=sTriCt-Ieee 2>&1 | FileCheck -check-prefix=CHECK-STRICT-IEEE %s + +// CHECK-IEEE: "-mllvm" "-hexagon-qfloat-mode=ieee" +// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \ +// RUN: -mhvx-qfloat=ieee 2>&1 | FileCheck -check-prefix=CHECK-IEEE %s +// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \ +// RUN: -mhvx-qfloat=IEEE 2>&1 | FileCheck -check-prefix=CHECK-IEEE %s + +// CHECK-LOSSY: "-mllvm" "-hexagon-qfloat-mode=lossy" +// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \ +// RUN: -mhvx-qfloat=lossy 2>&1 | FileCheck -check-prefix=CHECK-LOSSY %s +// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \ +// RUN: -mhvx-qfloat=lOSSy 2>&1 | FileCheck -check-prefix=CHECK-LOSSY %s + +// CHECK-LEGACY: "-mllvm" "-hexagon-qfloat-mode=legacy" +// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \ +// RUN: -mhvx-qfloat=legacy 2>&1 | FileCheck -check-prefix=CHECK-LEGACY %s +// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \ +// RUN: -mhvx-qfloat=LEGacy 2>&1 | FileCheck -check-prefix=CHECK-LEGACY %s + +// Test for default mode, if no mode is specified on v79. +// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv79 -mhvx \ +// RUN: -mhvx-qfloat 2>&1 | FileCheck -check-prefix=CHECK-LOSSY %s + +// Test for arches lower than v79 does not pass any backend flag. +// CHECK-MODE-NOT: "-mllvm" "-hexagon-qfloat-mode=" +// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv75 -mhvx \ +// RUN: -mhvx-qfloat 2>&1 | FileCheck -check-prefix=CHECK-MODE %s + +// Test for arches lower than v79 warns that qfloat mode is ignored. +// CHECK-MODE-WARN: warning: ignoring 'ieee' in '-mhvx-qfloat=ieee' option as it is not currently supported for target 'HVX v75' +// CHECK-MODE-WARN-NOT: "-mllvm" "-hexagon-qfloat-mode=" +// RUN: %clang -c %s -### -target hexagon-unknown-elf -mv75 -mhvx \ +// RUN: -mhvx-qfloat=ieee 2>&1 | FileCheck -check-prefix=CHECK-MODE-WARN %s diff --git a/llvm/include/llvm/CodeGen/RDFGraph.h b/llvm/include/llvm/CodeGen/RDFGraph.h index c1ec2ddff14a3..8d96efd8e2f0f 100644 --- a/llvm/include/llvm/CodeGen/RDFGraph.h +++ b/llvm/include/llvm/CodeGen/RDFGraph.h @@ -358,6 +358,8 @@ template <typename T> struct NodeAddr { return !operator==(NA); } + bool operator<(const NodeAddr<T> &NA) const { return Id < NA.Id; } + T Addr = nullptr; NodeId Id = 0; }; diff --git a/llvm/lib/Target/Hexagon/CMakeLists.txt b/llvm/lib/Target/Hexagon/CMakeLists.txt index 38dcc09282330..f98b519fe4974 100644 --- a/llvm/lib/Target/Hexagon/CMakeLists.txt +++ b/llvm/lib/Target/Hexagon/CMakeLists.txt @@ -58,9 +58,11 @@ add_llvm_target(HexagonCodeGen HexagonOptAddrMode.cpp HexagonOptimizeSZextends.cpp HexagonPeephole.cpp + HexagonPostRAHandleQFP.cpp HexagonQFPOptimizer.cpp HexagonRDFOpt.cpp HexagonRegisterInfo.cpp + HexagonXQFloatGenerator.cpp HexagonSelectionDAGInfo.cpp HexagonSplitConst32AndConst64.cpp HexagonSplitDouble.cpp diff --git a/llvm/lib/Target/Hexagon/Hexagon.h b/llvm/lib/Target/Hexagon/Hexagon.h index 1db2326b274dc..e9ee7b7c48e3e 100644 --- a/llvm/lib/Target/Hexagon/Hexagon.h +++ b/llvm/lib/Target/Hexagon/Hexagon.h @@ -69,8 +69,9 @@ void initializeHexagonOptimizeSZextendsPass(PassRegistry &); void initializeHexagonPeepholePass(PassRegistry &); void initializeHexagonSplitConst32AndConst64Pass(PassRegistry &); void initializeHexagonVectorPrintPass(PassRegistry &); - void initializeHexagonQFPOptimizerPass(PassRegistry &); +void initializeHexagonPostRAHandleQFPPass(PassRegistry &); +void initializeHexagonXQFloatGeneratorPass(PassRegistry &); Pass *createHexagonLoopIdiomPass(); Pass *createHexagonVectorLoopCarriedReuseLegacyPass(); @@ -119,6 +120,8 @@ FunctionPass *createHexagonVectorPrint(); FunctionPass *createHexagonVExtract(); FunctionPass *createHexagonExpandCondsets(); FunctionPass *createHexagonQFPOptimizer(); +FunctionPass *createHexagonPostRAHandleQFP(); +FunctionPass *createHexagonXQFloatGenerator(); } // end namespace llvm; diff --git a/llvm/lib/Target/Hexagon/HexagonInstrInfo.cpp b/llvm/lib/Target/Hexagon/HexagonInstrInfo.cpp index 6c95d54bf111b..5e8578a5d407d 100644 --- a/llvm/lib/Target/Hexagon/HexagonInstrInfo.cpp +++ b/llvm/lib/Target/Hexagon/HexagonInstrInfo.cpp @@ -4950,6 +4950,30 @@ bool HexagonInstrInfo::isQFPInstr(MachineInstr *MI) const { return isQFP32Instr(MI) || isQFP16Instr(MI); } +// Return true if the function contains any qf-generating instructions. +bool HexagonInstrInfo::hasQFPInstrs(const MachineFunction &MF) const { + for (const MachineBasicBlock &MBB : MF) + for (const MachineInstr &MI : MBB) + if (isQFPInstr(const_cast<MachineInstr *>(&MI))) + return true; + return false; +} + +// Returns true if A appears before B within the same basic block. +bool HexagonInstrInfo::isMIBefore(const MachineInstr *A, + const MachineInstr *B) const { + if (!A || !B || A->getParent() != B->getParent()) + return false; + + for (const MachineInstr &MI : *A->getParent()) { + if (&MI == A) + return true; + if (&MI == B) + return false; + } + return false; +} + // Addressing mode relations. short HexagonInstrInfo::changeAddrMode_abs_io(short Opc) const { return Opc >= 0 ? Hexagon::changeAddrMode_abs_io(Opc) : Opc; diff --git a/llvm/lib/Target/Hexagon/HexagonInstrInfo.h b/llvm/lib/Target/Hexagon/HexagonInstrInfo.h index 230f5d2228457..1901b260926d2 100644 --- a/llvm/lib/Target/Hexagon/HexagonInstrInfo.h +++ b/llvm/lib/Target/Hexagon/HexagonInstrInfo.h @@ -52,6 +52,9 @@ class HexagonInstrInfo : public HexagonGenInstrInfo { const HexagonRegisterInfo &getRegisterInfo() const { return RegInfo; } + bool isMIBefore(const MachineInstr *A, const MachineInstr *B) const; + bool hasQFPInstrs(const MachineFunction &MF) const; + /// TargetInstrInfo overrides. /// If the specified machine instruction is a direct diff --git a/llvm/lib/Target/Hexagon/HexagonPostRAHandleQFP.cpp b/llvm/lib/Target/Hexagon/HexagonPostRAHandleQFP.cpp new file mode 100644 index 0000000000000..31a97918d3c1f --- /dev/null +++ b/llvm/lib/Target/Hexagon/HexagonPostRAHandleQFP.cpp @@ -0,0 +1,1854 @@ +//===--------------------- HexagonPostRAHandleQFP.cpp -------------------------- +//===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===---------------------------------------------------------------------===// +// For v79 and above, we generate qf operations for HVX which includes vadd, +// vsub and vmpy instructions. These qf operations with qf operands are fast, +// maintain similar accuracy as IEEE and saves power. +// +// However, these qf operands should always be converted back to IEEE format +// when used in non-HVX instructions. This is because of how the qf values +// are stored in memory. qf operands have 4 extra bits. If used in non-HVX +// operations, these bits get dropped resulting in incorrect value being +// used. So, before use in any non-HVX operation we need to convert these +// qf values to IEEE format. +// +// During register allocation, when no more physical registers are available +// the qf operands may be spilled to memory. This instantly causes loss of +// accuracy. This pass prevents that by: +// 1. Inserting qf type to IEEE type conversion instructions before the spill. +// 2. Iterating over the uses of qf def (created before the spill) and +// changing their opcodes to handle IEEE type operands for saturating +// instructions. This is because, the refills will use IEEE type operands, but +// the instructions will still assume qf operands. For non-saturating +// instructions which uses qf, we incorporate a conversion to IEEE before that. +// 3. Iterating over the uses of qf def created by the spill and replacing +// them with appropiate opcode (which uses IEEE operands) for saturating +// instructions. For non-saturating instructions which uses qf, +// we incorporate a conversion to IEEE before that. +// 4. Iterating over the copy instructions and checking their uses, +// inserting conversions from qf to IEEE whenever required. The conversions +// are inserted after their reaching def since there can be multiple defs +// for use in non-SSA form. +// +// To get the use-def chains, we make use of Register DataFlow Graph (RDF), +// since after register allocation SSA form is lost. This can be done during +// spills and fills during Frame Lowering for register allocation. However, +// that was abandoned due to the intermediate state of the code. +// Liveness is preserved in this pass. +// +// NOTE: +// Saturating instructions: Instructions for which transformation involves +// only changing the opcode. Eg. vmpy(qf32, sf) saturates to vmpy(sf, sf) when +// we see that the first operand is now a sf type. +// Non-Saturating instructions: Instructions for which conversion(s) have +// to be inserted. Eg. Vd.f8=Vu.qf16. If the use operand is now hf type, +// we have to insert a conversion qf16 = hf before this instruction. +// +// FIXME tags have been added for potential errors, along with the underlying +// assumption. +// FIXME Implement v81 specific optimizations as below. At the moment, we add +// converts. +// Vd.qf16=Vu.... [truncated] `````````` </details> https://github.com/llvm/llvm-project/pull/198902 _______________________________________________ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
