https://github.com/adams381 updated https://github.com/llvm/llvm-project/pull/213315
>From ca454d34a6fa09b473025b11bd72c8dfecbcbd61 Mon Sep 17 00:00:00 2001 From: Adam Smith <[email protected]> Date: Fri, 31 Jul 2026 09:25:22 -0700 Subject: [PATCH 1/2] [CIR] Lower variadic calls in CallConvLowering for x86_64 CallConvLowering classified each function once from its declared signature and reused that classification at every call site. An argument passed through an ellipsis has no entry in the callee's parameter list, so the classification came up short. rewriteCallSite rejected such a call when the declared parameters needed a rewrite, and otherwise returned early and left it as written, so on x86_64 an ellipsis argument that needed an extension attribute, a register coercion, or a byval slot was emitted unchanged. What the ABI does with an ellipsis argument depends on the whole argument list, since it competes for the same registers as a declared one: the same two-eightbyte record goes in a register pair early in the list and byval once the integer registers are gone. Variadic call sites under the x86_64 driver are now classified from the call's own operand types, recorded in a walk that runs before any signature is rewritten. The declared parameter count reaches llvm::abi::FunctionInfo::create as its NumRequired argument, which is what lets the x86_64 classifier tell a named argument from one passed through the ellipsis. No type that reaches classification today is passed differently for being unnamed, so that boundary changes no output yet. A debug assertion checks that a call site and its callee classify the return and the declared parameters the same way, since the definition and its call sites are now rewritten from separate classifications. prependIndirectCallee rebuilt the callee pointer's pointee without isVarArg, dropping the ellipsis from a lowered indirect call and with it the vector-register count the x86_64 SysV ABI passes in AL. rewriteCallSite's operand-count check also moves ahead of the pass-through early return, so a call left without a classification of its own is reported instead of passed through silently, and it now covers both directions of a mismatch. A callee declared no_proto turns off the verifier's argument-count check entirely, so a call can pass fewer arguments than the callee declares, which left the classification with an entry that has no operand to pair with. That reached an assertion, and in a build without assertions an argument index past the end. A surplus operand on a no_proto callee whose declared parameters were all pass-through took the early return instead and was lowered with an operand the classification never covered, so a program with no variadic call in it is now diagnosed where it previously compiled. The indirect-call walk visits CIRCallOpInterface rather than cir.call, so a variadic indirect cir.try_call is accounted for instead of being left as written while the signature around it is coerced. A non-variadic one is dropped where the walk collects, so it never reaches the ellipsis accounting: rewriting a cir.try_call is unimplemented either way, and this leaves it exactly as the pass found it. Variadic indirect calls that need a rewrite, variadic calls under the non-x86_64 drivers, an ellipsis argument whose ABI coercion the bridge cannot represent, and a call that passes more or fewer arguments than a no_proto callee declares all report NYI. The no_proto shape passes the dialect verifier without being variadic, so its operand count can differ from its declared one in either direction while the ellipsis accounting does not apply to it. ABILowering.rst no longer records variadic support as deferred, and says what x86_64 lowers and what is left. needsRewrite moves from CIRABIRewriteContext.cpp to ABIRewriteContext.h, factored over a new per-argument pass-through predicate, so the pass and the rewriter share one definition. A void return classifies as Ignore and so does a return the ABI drops, such as an empty record, but the two need opposite treatment: void is already its own wire form, while a dropped record return has to be rewritten to one. FunctionClassification now records which it was, at the point the classification is produced, next to the return type it came from. Deriving it at each consumer instead would let a classification be paired with the wrong answer, and reading a dropped return as void means silently skipping the rewrite it needs. The x86_64 classifier already knows the answer, and the two dialect-neutral drivers cannot, so the pass supplies it for them. mlir::abi::test::classify takes a TypeRange, so a caller holding a call's operand types no longer copies them to classify it. Classifying a call's own operands is also what first exposes an integer wider than a register to the ellipsis, so the coverage runs the widths the x86_64 filter admits: a _BitInt below 32 extends, 48 widens to one register, 96 coerces to a register pair whose halves both go through the ellipsis, and 128 stays in its natural type. A wider _BitInt is rejected by the filter and reaches errorNYI. --- clang/docs/CIR/ABILowering.rst | 8 +- .../Transforms/CallConvLoweringPass.cpp | 256 +++++++++++++++--- .../TargetLowering/CIRABIRewriteContext.cpp | 51 ++-- .../call-conv-lowering-x86_64-variadic.c | 133 +++++++++ .../indirect-call-classification-attr.cir | 45 ++- .../abi-lowering/variadic-call-nyi.cir | 30 +- .../abi-lowering/x86_64-indirect-try-call.cir | 62 +++++ .../abi-lowering/x86_64-variadic-call.cir | 209 ++++++++++++++ .../abi-lowering/x86_64-variadic-nyi.cir | 240 ++++++++++++++++ mlir/include/mlir/ABI/ABIRewriteContext.h | 41 +++ .../mlir/ABI/Targets/Test/TestTarget.h | 3 +- mlir/lib/ABI/Targets/Test/TestTarget.cpp | 2 +- 12 files changed, 1005 insertions(+), 75 deletions(-) create mode 100644 clang/test/CIR/CodeGen/call-conv-lowering-x86_64-variadic.c create mode 100644 clang/test/CIR/Transforms/abi-lowering/x86_64-indirect-try-call.cir create mode 100644 clang/test/CIR/Transforms/abi-lowering/x86_64-variadic-call.cir create mode 100644 clang/test/CIR/Transforms/abi-lowering/x86_64-variadic-nyi.cir diff --git a/clang/docs/CIR/ABILowering.rst b/clang/docs/CIR/ABILowering.rst index b708960c48eb4..1b788cade4dc3 100644 --- a/clang/docs/CIR/ABILowering.rst +++ b/clang/docs/CIR/ABILowering.rst @@ -38,8 +38,12 @@ and MLIR integration layer; FIR can adopt the same infrastructure with minimal dialect-specific adaptation (e.g. cdecl when calling C from Fortran). ABI compliance will be validated through differential testing against Classic Clang CodeGen, and performance overhead should remain under 5% compared to a direct, -dialect-specific implementation. Initial scope focuses on fixed-argument -functions; variadic support (varargs) is deferred. +dialect-specific implementation. Variadic calls are lowered on x86_64 by +classifying each call site from its own operand types, since an argument +passed through an ellipsis competes for registers with the declared ones. +An indirect variadic call whose operands already carry their wire form is left +as written. One that needs an ABI rewrite is deferred, as is variadic +lowering for other targets. Background and Context ====================== diff --git a/clang/lib/CIR/Dialect/Transforms/CallConvLoweringPass.cpp b/clang/lib/CIR/Dialect/Transforms/CallConvLoweringPass.cpp index 06dae4c541e39..b9ab1a6368506 100644 --- a/clang/lib/CIR/Dialect/Transforms/CallConvLoweringPass.cpp +++ b/clang/lib/CIR/Dialect/Transforms/CallConvLoweringPass.cpp @@ -321,17 +321,38 @@ convertABIArgInfo(const llvm::abi::ArgInfo &info, MLIRContext *ctx, return ArgClassification::getIgnore(); } +/// The number of leading arguments the classifier must treat as declared +/// parameters, which is meaningful only for a variadic signature: an argument +/// past the ellipsis is unnamed, and the x86_64 rules pass some unnamed types +/// differently. std::nullopt for a non-variadic signature, where every +/// argument is declared. +/// +/// The only x86_64 rule that reads this boundary today sends an unnamed vector +/// wider than 128 bits to memory, and isSupportedType rejects every vector, so +/// no input that currently reaches classification can observe the difference. +static std::optional<unsigned> requiredArgCount(cir::FuncType fnTy) { + if (!fnTy.isVarArg()) + return std::nullopt; + return fnTy.getNumInputs(); +} + /// Classify an x86_64 SysV signature (return type + argument types) using the -/// LLVM ABI library. Shared by the cir.func path and the indirect-call path -/// (which classifies from the callee function pointer's pointee FuncType). -/// Returns std::nullopt and emits an NYI error via \p emitError if the -/// signature uses a type the bridge does not handle yet. +/// LLVM ABI library. Shared by the cir.func path, the variadic-call path and +/// the indirect-call path (the latter classifies from the callee function +/// pointer's pointee FuncType). \p numRequired is the number of leading +/// entries in \p inputs that are declared parameters, set only for a variadic +/// signature: the classifier treats the rest as arguments passed through the +/// ellipsis. Returns std::nullopt and emits an NYI error via \p emitError if +/// the signature uses a type the bridge does not handle yet. static std::optional<FunctionClassification> classifyX86_64Signature( - mlir::Type retCIR, mlir::TypeRange inputs, MLIRContext *ctx, - const DataLayout &dl, mlir::abi::ABITypeMapper &typeMapper, + mlir::Type retCIR, mlir::TypeRange inputs, + std::optional<unsigned> numRequired, MLIRContext *ctx, const DataLayout &dl, + mlir::abi::ABITypeMapper &typeMapper, const llvm::abi::TargetInfo &targetInfo, ModuleOp modOp, llvm::function_ref<mlir::InFlightDiagnostic()> emitError) { assert(retCIR && "signature return type must be non-null"); + assert((!numRequired || *numRequired <= inputs.size()) && + "declared parameters cannot outnumber the classified arguments"); bool voidRet = isa<cir::VoidType>(retCIR); auto reject = [&](mlir::Type t) -> bool { @@ -355,8 +376,8 @@ static std::optional<FunctionClassification> classifyX86_64Signature( for (mlir::Type a : inputs) argAbi.push_back(mapCIRType(a, typeMapper, dl, modOp)); - std::unique_ptr<llvm::abi::FunctionInfo> fi = - llvm::abi::FunctionInfo::create(llvm::CallingConv::C, retAbi, argAbi); + std::unique_ptr<llvm::abi::FunctionInfo> fi = llvm::abi::FunctionInfo::create( + llvm::CallingConv::C, retAbi, argAbi, numRequired); targetInfo.computeInfo(*fi); // convertABIArgInfo returns nullopt when the classifier picks a coercion @@ -369,6 +390,7 @@ static std::optional<FunctionClassification> classifyX86_64Signature( }; FunctionClassification fc; + fc.returnsVoid = voidRet; mlir::Type origRet = voidRet ? mlir::Type() : retCIR; std::optional<ArgClassification> retAc = convertABIArgInfo(fi->getReturnInfo(), ctx, origRet); @@ -400,16 +422,68 @@ classifyX86_64Function(cir::FuncOp func, const DataLayout &dl, ModuleOp modOp) { cir::FuncType fnTy = func.getFunctionType(); return classifyX86_64Signature(fnTy.getReturnType(), fnTy.getInputs(), - func->getContext(), dl, typeMapper, targetInfo, - modOp, [&]() { return func.emitOpError(); }); + requiredArgCount(fnTy), func->getContext(), dl, + typeMapper, targetInfo, modOp, + [&]() { return func.emitOpError(); }); +} + +/// Classify a call that passes arguments through an ellipsis. The callee's +/// own classification covers only its declared parameters, but an ellipsis +/// argument competes for the same argument registers as a declared one, so +/// what the ABI does with it depends on the whole argument list: the same +/// small struct is passed in registers early in the list and in memory once +/// the integer registers are gone. Classifying from the call's operands +/// rather than the callee's signature is what makes that accounting right. +static std::optional<FunctionClassification> classifyX86_64VariadicCall( + cir::CIRCallOpInterface call, cir::FuncType calleeTy, const DataLayout &dl, + mlir::abi::ABITypeMapper &typeMapper, + const llvm::abi::TargetInfo &targetInfo, ModuleOp modOp) { + assert(calleeTy.isVarArg() && + "only a variadic callee can take more operands than it declares"); + Operation *op = call.getOperation(); + return classifyX86_64Signature( + calleeTy.getReturnType(), call.getArgOperands().getTypes(), + requiredArgCount(calleeTy), op->getContext(), dl, typeMapper, targetInfo, + modOp, [&]() { return op->emitOpError(); }); } +#ifndef NDEBUG +/// Whether \p callFc classifies a call's leading arguments and its return +/// exactly as \p calleeFc classifies the callee's declared parameters and +/// return. A function definition and its call sites are rewritten from +/// separate classifications, so the two would silently disagree on the wire +/// format if an ellipsis argument could ever change how a declared parameter +/// is passed. +static bool classifiesSamePrefix(const FunctionClassification &calleeFc, + const FunctionClassification &callFc) { + if (callFc.argInfos.size() < calleeFc.argInfos.size()) + return false; + return calleeFc.returnInfo == callFc.returnInfo && + std::equal(calleeFc.argInfos.begin(), calleeFc.argInfos.end(), + callFc.argInfos.begin()); +} +#endif + struct CallConvLoweringPass : public impl::CallConvLoweringBase<CallConvLoweringPass> { using CallConvLoweringBase::CallConvLoweringBase; void runOnOperation() override; }; +/// Record on \p fc whether \p returnType is CIR's void. The x86_64 classifier +/// answers this itself, but the other two drivers cannot: the test target is +/// dialect-neutral and has no notion of CIR's void, and the +/// classification-attr schema carries no return type at all. Both route +/// through here so a classification always reaches needsRewrite paired with +/// the return type it was built from. +static std::optional<FunctionClassification> +withReturnVoidness(std::optional<FunctionClassification> fc, + mlir::Type returnType) { + if (fc) + fc->returnsVoid = mlir::isa<cir::VoidType>(returnType); + return fc; +} + /// Classify \p func using whichever driver mode is configured. Returns /// std::nullopt and emits an error on the function if classification fails /// (e.g. injection-driver mode but the function is missing the attribute, @@ -428,15 +502,17 @@ classifyFunction(cir::FuncOp func, const DataLayout &dl, << "' (CallConvLowering driver mode 'classification-attr')"; return std::nullopt; } - return mlir::abi::test::parseClassificationAttr( - attr, [&]() { return func.emitOpError(); }); + return withReturnVoidness(mlir::abi::test::parseClassificationAttr( + attr, [&]() { return func.emitOpError(); }), + returnType); } // The x86_64 target is handled directly in runOnOperation (it needs a shared // ABITypeMapper and TargetInfo), so only the test target reaches here. assert(target == cir::CallConvTarget::Test && "classifyFunction only handles the test target"); - return mlir::abi::test::classify(argTypes, returnType, dl); + return withReturnVoidness(mlir::abi::test::classify(argTypes, returnType, dl), + returnType); } /// Find the cir.func declaration matching a direct cir.call / cir.try_call @@ -457,6 +533,31 @@ cir::FuncOp lookupCallee(Operation *callOp, SymbolTable &symbolTable) { return symbolTable.lookup<cir::FuncOp>(callee.getValue()); } +/// The callee pointer of an indirect cir.call / cir.try_call, or nullptr for a +/// direct call. Both ops spell this the same way, but they inherit it from a +/// shared TableGen base rather than a common C++ type, so reaching it needs +/// this dispatch. +mlir::Value indirectCallee(Operation *callOp) { + if (auto call = dyn_cast<cir::CallOp>(callOp)) + return call.isIndirect() ? call.getIndirectCall() : nullptr; + if (auto tryCall = dyn_cast<cir::TryCallOp>(callOp)) + return tryCall.isIndirect() ? tryCall.getIndirectCall() : nullptr; + return nullptr; +} + +/// The signature an indirect call reaches its callee through, or a null type +/// for a direct call. The callee's pointer-to-function shape is asserted +/// rather than verified: the dialect checks operand types against the callee +/// only for a direct call, so IR that breaks it fails here instead of in the +/// verifier. +cir::FuncType indirectCalleeType(Operation *callOp) { + mlir::Value callee = indirectCallee(callOp); + if (!callee) + return {}; + return cast<cir::FuncType>( + cast<cir::PointerType>(callee.getType()).getPointee()); +} + void CallConvLoweringPass::runOnOperation() { ModuleOp moduleOp = getOperation(); MLIRContext *ctx = &getContext(); @@ -519,13 +620,53 @@ void CallConvLoweringPass::runOnOperation() { // cir.call / cir.try_call to each cir.func; the loop below rewrites a // function and all of its call sites together. Indirect or unresolved // callees are skipped here; rewriteCallSite errors on those at the end. + // + // A call that passes arguments through an ellipsis gets its own + // classification, recorded here while every signature is still in its + // original form. The callee's classification covers only its declared + // parameters and cannot describe those extra arguments. llvm::DenseMap<cir::FuncOp, SmallVector<Operation *>> callers; + // Keyed on the call op collected below, looked up once when that same op is + // rewritten. A key must never come from an op created during the rewrite: + // a recycled address could match an unrelated entry. + llvm::DenseMap<Operation *, FunctionClassification> variadicCallSites; moduleOp.walk([&](Operation *op) { - if (!isa<cir::CallOp, cir::TryCallOp>(op)) + auto call = dyn_cast<cir::CIRCallOpInterface>(op); + if (!call) + return; + cir::FuncOp callee = lookupCallee(op, symbolTable); + if (!callee) + return; + callers[callee].push_back(op); + + // Only the x86_64 driver classifies per call site. Under the other + // drivers the classification comes from a fixed per-function source, so + // such a call stays short a classification and rewriteCallSite reports it. + cir::FuncType calleeTy = callee.getFunctionType(); + if (!x86Target || call.getNumArgOperands() <= calleeTy.getNumInputs()) + return; + // A callee declared without a prototype also takes more operands than it + // declares, and the verifier allows it. Those extra arguments are named + // rather than passed through an ellipsis, so the accounting below does not + // describe them. + if (!calleeTy.isVarArg()) { + op->emitOpError() << "extra arguments to a callee without a prototype " + "not yet implemented in CallConvLowering"; + anyFailed = true; + return; + } + std::optional<FunctionClassification> fc = classifyX86_64VariadicCall( + call, calleeTy, dl, *x86TypeMapper, *x86Target, moduleOp); + if (!fc) { + anyFailed = true; return; - if (cir::FuncOp callee = lookupCallee(op, symbolTable)) - callers[callee].push_back(op); + } + variadicCallSites.insert({op, std::move(*fc)}); }); + if (anyFailed) { + signalPassFailure(); + return; + } // Rewrite each function together with every direct call to it. By the // time we move on to function F+1, F's signature and every direct call to @@ -547,7 +688,15 @@ void CallConvLoweringPass::runOnOperation() { return; } for (Operation *callOp : callers.lookup(func)) { - if (failed(rewriteCtx.rewriteCallSite(callOp, fc, builder))) { + const FunctionClassification *callFc = &fc; + if (auto it = variadicCallSites.find(callOp); + it != variadicCallSites.end()) { + callFc = &it->second; + assert(classifiesSamePrefix(fc, *callFc) && + "a call site's declared parameters must be classified the same " + "way as the callee's"); + } + if (failed(rewriteCtx.rewriteCallSite(callOp, *callFc, builder))) { signalPassFailure(); return; } @@ -560,38 +709,71 @@ void CallConvLoweringPass::runOnOperation() { // when an sret rewrite reuses a single-use store's destination as the return // slot it erases that store, which is the operation a live walk has already // cached as the next one to visit. - SmallVector<cir::CallOp> indirectCalls; - moduleOp.walk([&](cir::CallOp c) { - if (c.isIndirect()) - indirectCalls.push_back(c); + SmallVector<cir::CIRCallOpInterface> indirectCalls; + moduleOp.walk([&](cir::CIRCallOpInterface c) { + cir::FuncType calleeTy = indirectCalleeType(c.getOperation()); + if (!calleeTy) + return; + // A cir.try_call is in this walk so that a variadic one reaches the + // ellipsis accounting below. CIRABIRewriteContext cannot rebuild a + // cir.try_call at all, so a non-variadic one has never been rewritten + // here. Keep it out rather than start reporting a gap that has nothing + // to do with the ellipsis. + if (!calleeTy.isVarArg() && isa<cir::TryCallOp>(c.getOperation())) + return; + indirectCalls.push_back(c); }); - for (cir::CallOp c : indirectCalls) { + for (cir::CIRCallOpInterface c : indirectCalls) { // classification-attr mode injects a per-function classification, which // cannot describe a callee resolved at run time. Report it rather than // leave the indirect call unrewritten while direct calls are coerced. if (!classificationAttr.empty()) { - c.emitOpError() << "indirect call cannot be classified in the " - "'classification-attr' driver mode"; + c->emitOpError() << "indirect call cannot be classified in the " + "'classification-attr' driver mode"; signalPassFailure(); return; } - // The CallOp verifier guarantees an indirect callee is a pointer to a - // function type. - auto ptrTy = cast<cir::PointerType>(c.getIndirectCall().getType()); - auto funcTy = cast<cir::FuncType>(ptrTy.getPointee()); - std::optional<FunctionClassification> fc; - if (x86Target) - fc = classifyX86_64Signature(funcTy.getReturnType(), funcTy.getInputs(), - ctx, dl, *x86TypeMapper, *x86Target, - moduleOp, [&]() { return c.emitOpError(); }); - else - fc = mlir::abi::test::classify(funcTy.getInputs(), funcTy.getReturnType(), - dl); + cir::FuncType funcTy = indirectCalleeType(c.getOperation()); + auto classifySignature = + [&](mlir::TypeRange argTypes) -> std::optional<FunctionClassification> { + if (x86Target) + return classifyX86_64Signature(funcTy.getReturnType(), argTypes, + requiredArgCount(funcTy), ctx, dl, + *x86TypeMapper, *x86Target, moduleOp, + [&]() { return c->emitOpError(); }); + return withReturnVoidness( + mlir::abi::test::classify(argTypes, funcTy.getReturnType(), dl), + funcTy.getReturnType()); + }; + + // An argument passed through an ellipsis has no counterpart in the + // pointee's parameter list, so classify the call's own operands to learn + // what the ABI does with it. If nothing in the full list needs a rewrite + // the call already carries its wire form and can stand as written. + // Anything else needs a rewrite the pointee's signature cannot describe, + // since it has no entry for the arguments past the ellipsis. + if (c.getNumArgOperands() > funcTy.getNumInputs()) { + std::optional<FunctionClassification> callFc = + classifySignature(c.getArgOperands().getTypes()); + if (!callFc) { + signalPassFailure(); + return; + } + if (!callFc->needsRewrite()) + continue; + c->emitOpError() << "variadic arguments to an indirect call not yet " + "implemented in CallConvLowering"; + signalPassFailure(); + return; + } + + std::optional<FunctionClassification> fc = + classifySignature(funcTy.getInputs()); if (!fc) { signalPassFailure(); return; } - if (failed(rewriteCtx.rewriteCallSite(c, *fc, builder))) { + if (failed(rewriteCtx.rewriteCallSite(c.getOperation(), *fc, builder))) { signalPassFailure(); return; } diff --git a/clang/lib/CIR/Dialect/Transforms/TargetLowering/CIRABIRewriteContext.cpp b/clang/lib/CIR/Dialect/Transforms/TargetLowering/CIRABIRewriteContext.cpp index e87ae2326d524..1050b5f768f51 100644 --- a/clang/lib/CIR/Dialect/Transforms/TargetLowering/CIRABIRewriteContext.cpp +++ b/clang/lib/CIR/Dialect/Transforms/TargetLowering/CIRABIRewriteContext.cpp @@ -44,19 +44,6 @@ using namespace mlir::abi; namespace { -bool needsRewrite(const FunctionClassification &fc) { - // Direct without coercion is a true pass-through; any other kind (or a - // coerced Direct) means the rewriter must touch the IR. Extend is - // technically attribute-only at the IR level but still counts because the - // attribute attachment changes observable behavior. - if ((fc.returnInfo.kind != ArgKind::Direct) || fc.returnInfo.coercedType) - return true; - for (const ArgClassification &ac : fc.argInfos) - if ((ac.kind != ArgKind::Direct) || ac.coercedType) - return true; - return false; -} - /// Return the coerced RecordType for a Direct classification that should be /// flattened into individual scalar arguments, or a null type if the /// classification does not call for flattening. @@ -817,8 +804,14 @@ static void prependIndirectCallee(cir::CallOp call, // pointer's pointee and takes the call's result from that type, so the // pointee's return type has to track the rewrite: an sret return would // leave a result the call no longer produces, and a coerced return one of - // the wrong type. - auto newPtrTy = cir::PointerType::get(cir::FuncType::get(paramTypes, retTy)); + // the wrong type. The ellipsis has to survive for the same reason: the + // rebuilt pointee is what makes the lowered call variadic, and only a + // variadic call gets the vector-register count that the x86_64 SysV ABI + // passes in AL and that the callee's va_arg reads back. + auto calleeFnTy = cast<cir::FuncType>( + cast<cir::PointerType>(calleePtr.getType()).getPointee()); + auto newPtrTy = cir::PointerType::get( + cir::FuncType::get(paramTypes, retTy, calleeFnTy.isVarArg())); if (calleePtr.getType() != newPtrTy) calleePtr = cir::CastOp::create(builder, call.getLoc(), newPtrTy, cir::CastKind::bitcast, calleePtr); @@ -934,7 +927,7 @@ mlir::LogicalResult CIRABIRewriteContext::rewriteFunctionDefinition( // CIRGlobalValueInterface). cir::FuncOp funcOp = mlir::cast<cir::FuncOp>(funcOpInterface); - if (!needsRewrite(fc)) + if (!fc.needsRewrite()) return mlir::success(); ArrayRef<mlir::Type> oldArgTypes = funcOp.getArgumentTypes(); @@ -1101,7 +1094,26 @@ mlir::LogicalResult CIRABIRewriteContext::rewriteCallSite(mlir::Operation *callOp, const FunctionClassification &fc, mlir::OpBuilder &builder) { - if (!needsRewrite(fc)) + // The classification covers exactly the callee's declared parameters, and + // the rewrite below pairs it with the call's operands one for one. Both + // directions of a mismatch have to be reported before the pass-through early + // return, or a call whose declared parameters happen to be pass-through is + // left as written with its surplus operands never classified. + // + // A surplus operand went through an ellipsis. A shortfall means the callee + // was declared no_proto, which turns off the verifier's argument-count check + // altogether. + unsigned numOperands = + mlir::cast<cir::CIRCallOpInterface>(callOp).getNumArgOperands(); + if (numOperands > fc.argInfos.size()) + return callOp->emitOpError() + << "variadic arguments not yet implemented in CallConvLowering"; + if (numOperands < fc.argInfos.size()) + return callOp->emitOpError() + << "call passes fewer arguments than the callee declares, which is " + "not yet implemented in CallConvLowering"; + + if (!fc.needsRewrite()) return mlir::success(); if (mlir::isa<cir::TryCallOp>(callOp)) @@ -1128,11 +1140,6 @@ CIRABIRewriteContext::rewriteCallSite(mlir::Operation *callOp, // types here for use in updateArgAttrs). SmallVector<mlir::Type> origCallArgTypes; llvm::append_range(origCallArgTypes, argOperands.getTypes()); - if (argOperands.size() > fc.argInfos.size()) - return call.emitOpError() - << "variadic arguments not yet implemented in CallConvLowering"; - assert(fc.argInfos.size() == argOperands.size() && - "call operand count must match classified arg count"); for (auto [idx, ac] : llvm::enumerate(fc.argInfos)) { if (ac.kind == ArgKind::Ignore) continue; diff --git a/clang/test/CIR/CodeGen/call-conv-lowering-x86_64-variadic.c b/clang/test/CIR/CodeGen/call-conv-lowering-x86_64-variadic.c new file mode 100644 index 0000000000000..d3e28e58d2823 --- /dev/null +++ b/clang/test/CIR/CodeGen/call-conv-lowering-x86_64-variadic.c @@ -0,0 +1,133 @@ +// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -fclangir -clangir-enable-call-conv-lowering -emit-cir %s -o %t.cir +// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s +// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -fclangir -clangir-enable-call-conv-lowering -emit-llvm %s -o %t-cir.ll +// RUN: FileCheck --check-prefixes=LLVM,LLVM-CIR --input-file=%t-cir.ll %s +// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -emit-llvm %s -o %t.ll +// RUN: FileCheck --check-prefixes=LLVM,LLVM-OGCG --input-file=%t.ll %s + +typedef struct { int x; int y; } Pair2; +typedef struct { long a; long b; } Pair16; +typedef struct { long a, b, c, d; } Big; +typedef struct { __int128 w; } Wide; +typedef struct { __int128 w; char c; } WideChar; + +int vf(Pair2 p, ...); + +// CIR: cir.func private @vf(!u64i, ...) -> !s32i + +int call_scalar(Pair2 p, int a, double d) { return vf(p, a, d); } + +// CIR-LABEL: cir.func {{.*}}@call_scalar(%arg0: !u64i loc({{.+}}), %arg1: !s32i {llvm.noundef} loc({{.+}}), %arg2: !cir.double {llvm.noundef} loc({{.+}})) -> !s32i +// CIR: cir.call @vf(%{{.+}}, %{{.+}}, %{{.+}}) : (!u64i, !s32i {llvm.noundef}, !cir.double {llvm.noundef}) -> !s32i + +// LLVM-LABEL: i32 @call_scalar(i64 %{{.+}}, i32 noundef %{{.+}}, double noundef %{{.+}}) +// LLVM: call i32 (i64, ...) @vf(i64 %{{.+}}, i32 noundef %{{.+}}, double noundef %{{.+}}) + +// A two-eightbyte record at the ellipsis is flattened into two INTEGER +// registers while registers remain. +int call_small(Pair2 p, Pair16 q) { return vf(p, q); } + +// CIR-LABEL: cir.func {{.*}}@call_small(%arg0: !u64i loc({{.+}}), %arg1: !s64i loc({{.+}}), %arg2: !s64i loc({{.+}})) -> !s32i +// CIR: cir.call @vf(%{{.+}}, %{{.+}}, %{{.+}}) : (!u64i, !s64i, !s64i) -> !s32i + +// LLVM-LABEL: i32 @call_small(i64 %{{.+}}, i64 %{{.+}}, i64 %{{.+}}) +// LLVM: call i32 (i64, ...) @vf(i64 %{{.+}}, i64 %{{.+}}, i64 %{{.+}}) + +// Larger than two eightbytes is MEMORY regardless of register availability. +int call_big(Pair2 p, Big b) { return vf(p, b); } + +// CIR-LABEL: cir.func {{.*}}@call_big(%arg0: !u64i loc({{.+}}), %arg1: !cir.ptr<!rec_Big> {llvm.align = 8 : i64, llvm.byval = !rec_Big, llvm.noalias, llvm.noundef} loc({{.+}})) -> !s32i +// CIR: cir.call @vf(%{{.+}}, %{{.+}}) : (!u64i, !cir.ptr<!rec_Big> {llvm.align = 8 : i64, llvm.byval = !rec_Big, llvm.noalias, llvm.noundef}) -> !s32i + +// LLVM-CIR-LABEL: i32 @call_big(i64 %{{.+}}, ptr noalias noundef byval(%struct.Big) align 8 %{{.+}}) +// LLVM-CIR: call i32 (i64, ...) @vf(i64 %{{.+}}, ptr noalias noundef byval(%struct.Big) align 8 %{{.+}}) +// LLVM-OGCG-LABEL: i32 @call_big(i64 %{{.+}}, ptr noundef byval(%struct.Big) align 8 %{{.+}}) +// LLVM-OGCG: call i32 (i64, ...) @vf(i64 %{{.+}}, ptr noundef byval(%struct.Big) align 8 %{{.+}}) + +// The same Pair16 that went to registers in call_small goes to memory here: +// the named parameter and four longs leave only one INTEGER register, and a +// two-eightbyte record cannot be split across a register and the stack. +int call_exhausted(Pair2 p, long a, long b, long c, long d, Pair16 q) { + return vf(p, a, b, c, d, q); +} + +// CIR-LABEL: cir.func {{.*}}@call_exhausted(%arg0: !u64i loc({{.+}}), %arg1: !s64i {llvm.noundef} loc({{.+}}), %arg2: !s64i {llvm.noundef} loc({{.+}}), %arg3: !s64i {llvm.noundef} loc({{.+}}), %arg4: !s64i {llvm.noundef} loc({{.+}}), %arg5: !cir.ptr<!rec_Pair16> {llvm.align = 8 : i64, llvm.byval = !rec_Pair16, llvm.noalias, llvm.noundef} loc({{.+}})) -> !s32i +// CIR: cir.call @vf(%{{.+}}, %{{.+}}, %{{.+}}, %{{.+}}, %{{.+}}, %{{.+}}) : (!u64i, !s64i {llvm.noundef}, !s64i {llvm.noundef}, !s64i {llvm.noundef}, !s64i {llvm.noundef}, !cir.ptr<!rec_Pair16> {llvm.align = 8 : i64, llvm.byval = !rec_Pair16, llvm.noalias, llvm.noundef}) -> !s32i + +// LLVM-CIR-LABEL: i32 @call_exhausted(i64 %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, ptr noalias noundef byval(%struct.Pair16) align 8 %{{.+}}) +// LLVM-CIR: call i32 (i64, ...) @vf(i64 %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, ptr noalias noundef byval(%struct.Pair16) align 8 %{{.+}}) +// LLVM-OGCG-LABEL: i32 @call_exhausted(i64 %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, ptr noundef byval(%struct.Pair16) align 8 %{{.+}}) +// LLVM-OGCG: call i32 (i64, ...) @vf(i64 %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, ptr noundef byval(%struct.Pair16) align 8 %{{.+}}) + +// A 128-bit integer spans two eightbytes but is still passed whole. +int call_int128(Pair2 p, __int128 w) { return vf(p, w); } + +// CIR-LABEL: cir.func {{.*}}@call_int128(%arg0: !u64i loc({{.+}}), %arg1: !s128i {llvm.noundef} loc({{.+}})) -> !s32i +// CIR: cir.call @vf(%{{.+}}, %{{.+}}) : (!u64i, !s128i {llvm.noundef}) -> !s32i + +// LLVM-LABEL: i32 @call_int128(i64 %{{.+}}, i128 noundef %{{.+}}) +// LLVM: call i32 (i64, ...) @vf(i64 %{{.+}}, i128 noundef %{{.+}}) + +// Wrapping it in a record does not change the class: both eightbytes are +// INTEGER, so the record is coerced back to a bare i128. +int call_wide(Pair2 p, Wide w) { return vf(p, w); } + +// CIR-LABEL: cir.func {{.*}}@call_wide(%arg0: !u64i loc({{.+}}), %arg1: !s128i loc({{.+}})) -> !s32i +// CIR: cir.call @vf(%{{.+}}, %{{.+}}) : (!u64i, !s128i) -> !s32i + +// LLVM-LABEL: i32 @call_wide(i64 %{{.+}}, i128 %{{.+}}) +// LLVM: call i32 (i64, ...) @vf(i64 %{{.+}}, i128 %{{.+}}) + +// One trailing byte pushes the record past two eightbytes, so it goes to +// memory, and the 128-bit member keeps the slot at 16-byte alignment. +int call_wide_char(Pair2 p, WideChar w) { return vf(p, w); } + +// CIR-LABEL: cir.func {{.*}}@call_wide_char(%arg0: !u64i loc({{.+}}), %arg1: !cir.ptr<!rec_WideChar> {llvm.align = 16 : i64, llvm.byval = !rec_WideChar, llvm.noalias, llvm.noundef} loc({{.+}})) -> !s32i +// CIR: cir.call @vf(%{{.+}}, %{{.+}}) : (!u64i, !cir.ptr<!rec_WideChar> {llvm.align = 16 : i64, llvm.byval = !rec_WideChar, llvm.noalias, llvm.noundef}) -> !s32i + +// LLVM-CIR-LABEL: i32 @call_wide_char(i64 %{{.+}}, ptr noalias noundef byval(%struct.WideChar) align 16 %{{.+}}) +// LLVM-CIR: call i32 (i64, ...) @vf(i64 %{{.+}}, ptr noalias noundef byval(%struct.WideChar) align 16 %{{.+}}) +// LLVM-OGCG-LABEL: i32 @call_wide_char(i64 %{{.+}}, ptr noundef byval(%struct.WideChar) align 16 %{{.+}}) +// LLVM-OGCG: call i32 (i64, ...) @vf(i64 %{{.+}}, ptr noundef byval(%struct.WideChar) align 16 %{{.+}}) + +// A _BitInt narrower than a register is extended at the ellipsis, same as a +// declared parameter. +int ell_bitint17(Pair2 p, _BitInt(17) b) { return vf(p, b); } + +// CIR-LABEL: cir.func {{.*}}@ell_bitint17(%arg0: !u64i loc({{.+}}), %arg1: !cir.int<s, 17, bitint> {llvm.signext} loc({{.+}})) -> !s32i +// CIR: cir.call @vf(%{{.+}}, %{{.+}}) : (!u64i, !cir.int<s, 17, bitint> {llvm.signext}) -> !s32i + +// LLVM-CIR-LABEL: i32 @ell_bitint17(i64 %{{.+}}, i17 signext %{{.+}}) +// LLVM-CIR: call i32 (i64, ...) @vf(i64 %{{.+}}, i17 signext %{{.+}}) +// LLVM-OGCG-LABEL: i32 @ell_bitint17(i64 %{{.+}}, i17 noundef signext %{{.+}}) +// LLVM-OGCG: call i32 (i64, ...) @vf(i64 %{{.+}}, i17 noundef signext %{{.+}}) + +// A width between 33 and 63 widens to one register. +int ell_bitint48(Pair2 p, _BitInt(48) b) { return vf(p, b); } + +// CIR-LABEL: cir.func {{.*}}@ell_bitint48(%arg0: !u64i loc({{.+}}), %arg1: !u64i {llvm.noundef} loc({{.+}})) -> !s32i +// CIR: cir.call @vf(%{{.+}}, %{{.+}}) : (!u64i, !u64i {llvm.noundef}) -> !s32i + +// LLVM-LABEL: i32 @ell_bitint48(i64 %{{.+}}, i64 noundef %{{.+}}) +// LLVM: call i32 (i64, ...) @vf(i64 %{{.+}}, i64 noundef %{{.+}}) + +// A width between 65 and 127 coerces to a register pair, and both halves are +// passed through the ellipsis. +int ell_bitint96(Pair2 p, _BitInt(96) b) { return vf(p, b); } + +// CIR-LABEL: cir.func {{.*}}@ell_bitint96(%arg0: !u64i loc({{.+}}), %arg1: !u64i loc({{.+}}), %arg2: !u64i loc({{.+}})) -> !s32i +// CIR: cir.call @vf(%{{.+}}, %{{.+}}, %{{.+}}) : (!u64i, !u64i, !u64i) -> !s32i + +// LLVM-CIR-LABEL: i32 @ell_bitint96(i64 %{{.+}}, i64 %{{.+}}, i64 %{{.+}}) +// LLVM-CIR: call i32 (i64, ...) @vf(i64 %{{.+}}, i64 %{{.+}}, i64 %{{.+}}) +// LLVM-OGCG-LABEL: i32 @ell_bitint96(i64 %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}) +// LLVM-OGCG: call i32 (i64, ...) @vf(i64 %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}) + +// At exactly 128 bits it stays in its natural type. +int ell_bitint128(Pair2 p, _BitInt(128) b) { return vf(p, b); } + +// CIR-LABEL: cir.func {{.*}}@ell_bitint128(%arg0: !u64i loc({{.+}}), %arg1: !s128i_bitint {llvm.noundef} loc({{.+}})) -> !s32i +// CIR: cir.call @vf(%{{.+}}, %{{.+}}) : (!u64i, !s128i_bitint {llvm.noundef}) -> !s32i + +// LLVM-LABEL: i32 @ell_bitint128(i64 %{{.+}}, i128 noundef %{{.+}}) +// LLVM: call i32 (i64, ...) @vf(i64 %{{.+}}, i128 noundef %{{.+}}) diff --git a/clang/test/CIR/Transforms/abi-lowering/indirect-call-classification-attr.cir b/clang/test/CIR/Transforms/abi-lowering/indirect-call-classification-attr.cir index 590e9eb21b1a7..9c709d4ee8042 100644 --- a/clang/test/CIR/Transforms/abi-lowering/indirect-call-classification-attr.cir +++ b/clang/test/CIR/Transforms/abi-lowering/indirect-call-classification-attr.cir @@ -1,5 +1,5 @@ // RUN: not cir-opt %s -cir-call-conv-lowering="classification-attr=test_classify" \ -// RUN: 2>&1 | FileCheck %s +// RUN: -split-input-file 2>&1 | FileCheck %s !s32i = !cir.int<s, 32> @@ -26,3 +26,46 @@ module attributes { } // CHECK: error: 'cir.call' op indirect call cannot be classified in the 'classification-attr' driver mode + +// ----- + +!s8i = !cir.int<s, 8> +!s32i = !cir.int<s, 32> +!void = !cir.void + +#passthrough = { + return = { kind = "direct" }, + args = [ { kind = "direct" }, { kind = "direct" } ] +} + +module attributes { + dlti.dl_spec = #dlti.dl_spec<#dlti.dl_entry<i32, dense<32>: vector<2xi64>>> +} { + + // A variadic indirect cir.try_call reaches the same driver-mode error. It is + // collected so that its ellipsis is accounted for, which a per-function + // injected classification cannot describe either. + cir.func @variadic_try_caller(%fp: !cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...) -> !s32i>>, + %fmt: !cir.ptr<!s8i>, %n: !s32i) -> !s32i + attributes { test_classify = #passthrough } { + %0 = cir.try_call %fp(%fmt, %n) ^bb1, ^bb2 + : (!cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...) -> !s32i>>, !cir.ptr<!s8i>, !s32i) -> !s32i + ^bb1: + cir.br ^bb4(%0 : !s32i) + ^bb2: + %1 = cir.eh.initiate : !cir.eh_token + cir.eh.dispatch %1 : !cir.eh_token [ + catch_all : ^bb3 + ] + ^bb3(%tok : !cir.eh_token): + %ct, %exn = cir.begin_catch %tok : !cir.eh_token -> (!cir.catch_token, !cir.ptr<!void>) + cir.end_catch %ct : !cir.catch_token + %2 = cir.const #cir.int<0> : !s32i + cir.br ^bb4(%2 : !s32i) + ^bb4(%r : !s32i): + cir.return %r : !s32i + } + +} + +// CHECK: error: 'cir.try_call' op indirect call cannot be classified in the 'classification-attr' driver mode diff --git a/clang/test/CIR/Transforms/abi-lowering/variadic-call-nyi.cir b/clang/test/CIR/Transforms/abi-lowering/variadic-call-nyi.cir index 72f72ed73899b..af606c9677993 100644 --- a/clang/test/CIR/Transforms/abi-lowering/variadic-call-nyi.cir +++ b/clang/test/CIR/Transforms/abi-lowering/variadic-call-nyi.cir @@ -1,26 +1,34 @@ // RUN: not cir-opt %s -cir-call-conv-lowering="classification-attr=test_classify" \ // RUN: 2>&1 | FileCheck %s +!s8i = !cir.int<s, 8> !s32i = !cir.int<s, 32> -#ignore_first_arg = { +#one_direct = { return = { kind = "direct" }, - args = [ { kind = "ignore" }, { kind = "direct" } ] + args = [ { kind = "direct" } ] +} + +#two_direct = { + return = { kind = "direct" }, + args = [ { kind = "direct" }, { kind = "direct" } ] } module attributes { - dlti.dl_spec = #dlti.dl_spec<#dlti.dl_entry<i32, dense<32>: vector<2xi64>>> + dlti.dl_spec = #dlti.dl_spec< + #dlti.dl_entry<i8, dense<8>: vector<2xi64>>, + #dlti.dl_entry<i32, dense<32>: vector<2xi64>>> } { - cir.func @callee(%arg0: !s32i, %arg1: !s32i, %arg2: !s32i) -> !s32i - attributes { test_classify = #ignore_first_arg } { - cir.return %arg2 : !s32i - } + cir.func private @variadic(!s8i, ...) -> !s32i + attributes { test_classify = #one_direct } - cir.func @caller(%arg0: !s32i, %arg1: !s32i, %arg2: !s32i) -> !s32i - attributes { test_classify = #ignore_first_arg } { - %0 = cir.call @callee(%arg0, %arg1, %arg2) - : (!s32i, !s32i, !s32i) -> !s32i + // A fixed per-function classification attribute covers the declared + // parameter only, so the argument at the ellipsis is unclassified. The call + // is rejected even though nothing in the declared signature needs a rewrite. + cir.func @caller(%arg0: !s8i, %arg1: !s32i) -> !s32i + attributes { test_classify = #two_direct } { + %0 = cir.call @variadic(%arg0, %arg1) : (!s8i, !s32i) -> !s32i cir.return %0 : !s32i } diff --git a/clang/test/CIR/Transforms/abi-lowering/x86_64-indirect-try-call.cir b/clang/test/CIR/Transforms/abi-lowering/x86_64-indirect-try-call.cir new file mode 100644 index 0000000000000..75e9c9d69c830 --- /dev/null +++ b/clang/test/CIR/Transforms/abi-lowering/x86_64-indirect-try-call.cir @@ -0,0 +1,62 @@ +// RUN: cir-opt %s -cir-call-conv-lowering=target=x86_64 | FileCheck %s + +!s8i = !cir.int<s, 8> +!s32i = !cir.int<s, 32> +!void = !cir.void + +module attributes { + cir.triple = "x86_64-unknown-linux-gnu", + dlti.dl_spec = #dlti.dl_spec< + #dlti.dl_entry<i8, dense<8>: vector<2xi64>>, + #dlti.dl_entry<i32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<i64, dense<64>: vector<2xi64>>> +} { + + // Scalars already carry their wire form, so the call stands as written. + cir.func @pass_through(%arg0: !cir.ptr<!cir.func<(!s32i) -> !s32i>>, + %arg1: !s32i) { + %0 = cir.try_call %arg0(%arg1) ^bb1, ^bb2 + : (!cir.ptr<!cir.func<(!s32i) -> !s32i>>, !s32i) -> !s32i + ^bb1: + cir.br ^bb4 + ^bb2: + %1 = cir.eh.initiate : !cir.eh_token + cir.eh.dispatch %1 : !cir.eh_token [ + catch_all : ^bb3 + ] + ^bb3(%tok : !cir.eh_token): + %ct, %exn = cir.begin_catch %tok : !cir.eh_token -> (!cir.catch_token, !cir.ptr<!void>) + cir.end_catch %ct : !cir.catch_token + cir.br ^bb4 + ^bb4: + cir.return + } + + // CHECK: cir.func @pass_through(%arg0: !cir.ptr<!cir.func<(!s32i) -> !s32i>>, %arg1: !s32i) + // CHECK: %0 = cir.try_call %arg0(%arg1) ^bb1, ^bb2 : (!cir.ptr<!cir.func<(!s32i) -> !s32i>>, !s32i) -> !s32i + + // A variadic pointee is classified even with no ellipsis argument passed, + // and a void return is its own wire form, so this call stands as written + // too. + cir.func @variadic_void(%arg0: !cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...)>>, + %arg1: !cir.ptr<!s8i>) { + cir.try_call %arg0(%arg1) ^bb1, ^bb2 + : (!cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...)>>, !cir.ptr<!s8i>) -> () + ^bb1: + cir.br ^bb4 + ^bb2: + %0 = cir.eh.initiate : !cir.eh_token + cir.eh.dispatch %0 : !cir.eh_token [ + catch_all : ^bb3 + ] + ^bb3(%tok : !cir.eh_token): + %ct, %exn = cir.begin_catch %tok : !cir.eh_token -> (!cir.catch_token, !cir.ptr<!void>) + cir.end_catch %ct : !cir.catch_token + cir.br ^bb4 + ^bb4: + cir.return + } + + // CHECK: cir.func @variadic_void(%arg0: !cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...)>>, %arg1: !cir.ptr<!s8i>) + // CHECK: cir.try_call %arg0(%arg1) ^bb1, ^bb2 : (!cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...)>>, !cir.ptr<!s8i>) -> () +} diff --git a/clang/test/CIR/Transforms/abi-lowering/x86_64-variadic-call.cir b/clang/test/CIR/Transforms/abi-lowering/x86_64-variadic-call.cir new file mode 100644 index 0000000000000..7a531604073fb --- /dev/null +++ b/clang/test/CIR/Transforms/abi-lowering/x86_64-variadic-call.cir @@ -0,0 +1,209 @@ +// RUN: cir-opt %s -cir-call-conv-lowering=target=x86_64 | FileCheck %s +// RUN: cir-opt %s -cir-call-conv-lowering=target=x86_64 -cir-to-llvm -o - 2>/dev/null \ +// RUN: | mlir-translate -mlir-to-llvmir --allow-unregistered-dialect \ +// RUN: | FileCheck %s --check-prefix=LLVM + +!s8i = !cir.int<s, 8> +!s32i = !cir.int<s, 32> +!s64i = !cir.int<s, 64> +!rec_Pair = !cir.struct<"Pair" {!s32i, !s32i}> +!rec_Two = !cir.struct<"Two" {!s64i, !s64i}> +!rec_Big = !cir.struct<"Big" {!s64i, !s64i, !s64i}> +!rec_E0 = !cir.struct<"E0" {}> + +module attributes { + cir.triple = "x86_64-unknown-linux-gnu", + dlti.dl_spec = #dlti.dl_spec< + #dlti.dl_entry<i8, dense<8>: vector<2xi64>>, + #dlti.dl_entry<i32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<i64, dense<64>: vector<2xi64>>, + #dlti.dl_entry<f64, dense<64>: vector<2xi64>>> +} { + + cir.func private @variadic(!cir.ptr<!s8i>, ...) -> !s32i + cir.func private @variadic_pair(!rec_Pair, ...) -> !s32i + cir.func private @variadic_big(!cir.ptr<!s8i>, ...) -> !rec_Big + + // A rewritten variadic declaration keeps its ellipsis. + // CHECK: cir.func private @variadic_pair(!u64i, ...) -> !s32i + // CHECK: cir.func private @variadic_big(!cir.ptr<!rec_Big> {llvm.align = 8 : i64, llvm.dead_on_unwind, llvm.sret = !rec_Big, llvm.writable}, !cir.ptr<!s8i>, ...) + + // Passing nothing through the ellipsis leaves the call at the callee's + // declared parameters. + cir.func @pass_none(%arg0: !cir.ptr<!s8i>) { + %0 = cir.call @variadic(%arg0) : (!cir.ptr<!s8i>) -> !s32i + cir.return + } + + // CHECK: cir.func{{.*}} @pass_none(%arg0: !cir.ptr<!s8i>) + // CHECK: cir.call @variadic(%arg0) : (!cir.ptr<!s8i>) -> !s32i + + cir.func @pass_scalar(%arg0: !cir.ptr<!s8i>, %arg1: !s32i) { + %0 = cir.call @variadic(%arg0, %arg1) : (!cir.ptr<!s8i>, !s32i) -> !s32i + cir.return + } + + // CHECK: cir.func{{.*}} @pass_scalar(%arg0: !cir.ptr<!s8i>, %arg1: !s32i) + // CHECK: cir.call @variadic(%arg0, %arg1) : (!cir.ptr<!s8i>, !s32i) -> !s32i + + // An integer narrower than a word is extended at the ellipsis just as it is + // in a declared parameter position. + cir.func @pass_narrow(%arg0: !cir.ptr<!s8i>, %arg1: !s8i) { + %0 = cir.call @variadic(%arg0, %arg1) : (!cir.ptr<!s8i>, !s8i) -> !s32i + cir.return + } + + // CHECK: cir.func{{.*}} @pass_narrow(%arg0: !cir.ptr<!s8i>, %arg1: !s8i {llvm.signext}) + // CHECK: cir.call @variadic(%arg0, %arg1) : (!cir.ptr<!s8i>, !s8i {llvm.signext}) -> !s32i + + cir.func @pass_double(%arg0: !cir.ptr<!s8i>, %arg1: !cir.double) { + %0 = cir.call @variadic(%arg0, %arg1) : (!cir.ptr<!s8i>, !cir.double) -> !s32i + cir.return + } + + // CHECK: cir.func{{.*}} @pass_double(%arg0: !cir.ptr<!s8i>, %arg1: !cir.double) + // CHECK: cir.call @variadic(%arg0, %arg1) : (!cir.ptr<!s8i>, !cir.double) -> !s32i + + // The format string consumes one INTEGER register, so the two eightbytes of + // this record still fit in registers and are passed flattened. + cir.func @pass_two_early(%arg0: !cir.ptr<!s8i>, %arg1: !rec_Two) { + %0 = cir.call @variadic(%arg0, %arg1) : (!cir.ptr<!s8i>, !rec_Two) -> !s32i + cir.return + } + + // CHECK: cir.func{{.*}} @pass_two_early(%arg0: !cir.ptr<!s8i>, %arg1: !s64i, %arg2: !s64i) + // CHECK: cir.call @variadic(%arg0, %{{.*}}, %{{.*}}) : (!cir.ptr<!s8i>, !s64i, !s64i) -> !s32i + + // The same record after the format string and five ints have used all six + // INTEGER registers: it goes to memory instead, as a byval slot. + cir.func @pass_two_exhausted(%arg0: !cir.ptr<!s8i>, %arg1: !s32i, %arg2: !rec_Two) { + %0 = cir.call @variadic(%arg0, %arg1, %arg1, %arg1, %arg1, %arg1, %arg2) + : (!cir.ptr<!s8i>, !s32i, !s32i, !s32i, !s32i, !s32i, !rec_Two) -> !s32i + cir.return + } + + // CHECK: cir.func{{.*}} @pass_two_exhausted(%arg0: !cir.ptr<!s8i>, %arg1: !s32i, %arg2: !s64i, %arg3: !s64i) + // CHECK: %[[SLOT:.*]] = cir.alloca "byval" align(8) : !cir.ptr<!rec_Two> + // CHECK: cir.call @variadic(%arg0, %arg1, %arg1, %arg1, %arg1, %arg1, %[[SLOT]]) : (!cir.ptr<!s8i>, !s32i, !s32i, !s32i, !s32i, !s32i, !cir.ptr<!rec_Two> {llvm.align = 8 : i64, llvm.byval = !rec_Two, llvm.noalias, llvm.noundef}) -> !s32i + + // A record larger than two eightbytes is memory class no matter how many + // registers are free. + cir.func @pass_big(%arg0: !cir.ptr<!s8i>, %arg1: !rec_Big) { + %0 = cir.call @variadic(%arg0, %arg1) : (!cir.ptr<!s8i>, !rec_Big) -> !s32i + cir.return + } + + // CHECK: cir.func{{.*}} @pass_big(%arg0: !cir.ptr<!s8i>, %arg1: !cir.ptr<!rec_Big> {llvm.align = 8 : i64, llvm.byval = !rec_Big, llvm.noalias, llvm.noundef}) + // CHECK: %[[BSLOT:.*]] = cir.alloca "byval" align(8) : !cir.ptr<!rec_Big> + // CHECK: cir.call @variadic(%arg0, %[[BSLOT]]) : (!cir.ptr<!s8i>, !cir.ptr<!rec_Big> {llvm.align = 8 : i64, llvm.byval = !rec_Big, llvm.noalias, llvm.noundef}) -> !s32i + + // An empty record occupies no register and is dropped, without displacing + // the argument behind it. + cir.func @pass_empty(%arg0: !cir.ptr<!s8i>, %arg1: !rec_E0, %arg2: !s32i) { + %0 = cir.call @variadic(%arg0, %arg1, %arg2) + : (!cir.ptr<!s8i>, !rec_E0, !s32i) -> !s32i + cir.return + } + + // CHECK: cir.func{{.*}} @pass_empty(%arg0: !cir.ptr<!s8i>, %arg1: !s32i) + // CHECK: cir.call @variadic(%arg0, %arg1) : (!cir.ptr<!s8i>, !s32i) -> !s32i + + // A declared parameter is coerced the same way whether or not the call also + // passes ellipsis arguments. + cir.func @pass_declared_coerced(%arg0: !rec_Pair, %arg1: !s32i) { + %0 = cir.call @variadic_pair(%arg0, %arg1) : (!rec_Pair, !s32i) -> !s32i + cir.return + } + + // CHECK: cir.func{{.*}} @pass_declared_coerced(%arg0: !u64i, %arg1: !s32i) + // CHECK: cir.call @variadic_pair(%{{.*}}, %arg1) : (!u64i, !s32i) -> !s32i + + // An sret slot is prepended ahead of the declared parameters, so the + // ellipsis arguments shift right by one. + cir.func @pass_sret_return(%arg0: !cir.ptr<!s8i>, %arg1: !s32i) -> !rec_Big { + %0 = cir.call @variadic_big(%arg0, %arg1) : (!cir.ptr<!s8i>, !s32i) -> !rec_Big + cir.return %0 : !rec_Big + } + + // CHECK: cir.func{{.*}} @pass_sret_return(%arg0: !cir.ptr<!rec_Big> {llvm.align = 8 : i64, llvm.dead_on_unwind, llvm.noalias, llvm.sret = !rec_Big, llvm.writable}, %arg1: !cir.ptr<!s8i>, %arg2: !s32i) + // CHECK: cir.call @variadic_big(%arg0, %arg1, %arg2) : (!cir.ptr<!rec_Big> {llvm.align = 8 : i64, llvm.dead_on_unwind, llvm.sret = !rec_Big, llvm.writable}, !cir.ptr<!s8i>, !s32i) -> () + + // An indirect variadic call whose ellipsis arguments all pass as written + // needs no rewrite, and the callee pointer keeps its ellipsis. + cir.func @indirect_scalar(%arg0: !cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...) -> !s32i>>, + %arg1: !cir.ptr<!s8i>, %arg2: !s32i) { + %0 = cir.call %arg0(%arg1, %arg2) + : (!cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...) -> !s32i>>, !cir.ptr<!s8i>, !s32i) -> !s32i + cir.return + } + + // CHECK: cir.func{{.*}} @indirect_scalar(%arg0: !cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...) -> !s32i>>, %arg1: !cir.ptr<!s8i>, %arg2: !s32i) + // CHECK: cir.call %arg0(%arg1, %arg2) : (!cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...) -> !s32i>>, !cir.ptr<!s8i>, !s32i) -> !s32i + + // A void return is already its own wire form, so it does not by itself make + // the call need a rewrite. + cir.func @indirect_void(%arg0: !cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...)>>, + %arg1: !cir.ptr<!s8i>, %arg2: !s32i) { + cir.call %arg0(%arg1, %arg2) + : (!cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...)>>, !cir.ptr<!s8i>, !s32i) -> () + cir.return + } + + // CHECK: cir.func{{.*}} @indirect_void(%arg0: !cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...)>>, %arg1: !cir.ptr<!s8i>, %arg2: !s32i) + // CHECK: cir.call %arg0(%arg1, %arg2) : (!cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...)>>, !cir.ptr<!s8i>, !s32i) -> () + + // Coercing the declared parameter rebuilds the callee pointer, whose pointee + // has to stay variadic or the lowered call loses its ellipsis. + cir.func @indirect_coerced(%arg0: !cir.ptr<!cir.func<(!rec_Pair, ...) -> !s32i>>, + %arg1: !rec_Pair) { + %0 = cir.call %arg0(%arg1) + : (!cir.ptr<!cir.func<(!rec_Pair, ...) -> !s32i>>, !rec_Pair) -> !s32i + cir.return + } + + // CHECK: cir.func{{.*}} @indirect_coerced(%arg0: !cir.ptr<!cir.func<(!rec_Pair, ...) -> !s32i>>, %arg1: !u64i) + // CHECK: %[[CALLEE:.*]] = cir.cast bitcast %arg0 : !cir.ptr<!cir.func<(!rec_Pair, ...) -> !s32i>> -> !cir.ptr<!cir.func<(!u64i, ...) -> !s32i>> + // CHECK: cir.call %[[CALLEE]](%{{.*}}) : (!cir.ptr<!cir.func<(!u64i, ...) -> !s32i>>, !u64i) -> !s32i +} + +// LLVM: declare i32 @variadic_pair(i64, ...) +// LLVM: declare void @variadic_big(ptr dead_on_unwind writable sret(%struct.Big) align 8, ptr, ...) + +// LLVM: define void @pass_none(ptr %{{.+}}) +// LLVM: call i32 (ptr, ...) @variadic(ptr %{{.+}}) + +// LLVM: define void @pass_scalar(ptr %{{.+}}, i32 %{{.+}}) +// LLVM: call i32 (ptr, ...) @variadic(ptr %{{.+}}, i32 %{{.+}}) + +// LLVM: define void @pass_narrow(ptr %{{.+}}, i8 signext %{{.+}}) +// LLVM: call i32 (ptr, ...) @variadic(ptr %{{.+}}, i8 signext %{{.+}}) + +// LLVM: define void @pass_double(ptr %{{.+}}, double %{{.+}}) +// LLVM: call i32 (ptr, ...) @variadic(ptr %{{.+}}, double %{{.+}}) + +// LLVM: define void @pass_two_early(ptr %{{.+}}, i64 %{{.+}}, i64 %{{.+}}) +// LLVM: call i32 (ptr, ...) @variadic(ptr %{{.+}}, i64 %{{.+}}, i64 %{{.+}}) + +// LLVM: define void @pass_two_exhausted(ptr %{{.+}}, i32 %{{.+}}, i64 %{{.+}}, i64 %{{.+}}) +// LLVM: call i32 (ptr, ...) @variadic(ptr %{{.+}}, i32 %{{.+}}, i32 %{{.+}}, i32 %{{.+}}, i32 %{{.+}}, i32 %{{.+}}, ptr noalias noundef byval(%struct.Two) align 8 %{{.+}}) + +// LLVM: define void @pass_big(ptr %{{.+}}, ptr noalias noundef byval(%struct.Big) align 8 %{{.+}}) +// LLVM: call i32 (ptr, ...) @variadic(ptr %{{.+}}, ptr noalias noundef byval(%struct.Big) align 8 %{{.+}}) + +// LLVM: define void @pass_empty(ptr %{{.+}}, i32 %{{.+}}) +// LLVM: call i32 (ptr, ...) @variadic(ptr %{{.+}}, i32 %{{.+}}) + +// LLVM: define void @pass_declared_coerced(i64 %{{.+}}, i32 %{{.+}}) +// LLVM: call i32 (i64, ...) @variadic_pair(i64 %{{.+}}, i32 %{{.+}}) + +// LLVM: define void @pass_sret_return(ptr dead_on_unwind noalias writable sret(%struct.Big) align 8 %{{.+}}, ptr %{{.+}}, i32 %{{.+}}) +// LLVM: call void (ptr, ptr, ...) @variadic_big(ptr dead_on_unwind writable sret(%struct.Big) align 8 %{{.+}}, ptr %{{.+}}, i32 %{{.+}}) + +// LLVM: define void @indirect_scalar(ptr %{{.+}}, ptr %{{.+}}, i32 %{{.+}}) +// LLVM: call i32 (ptr, ...) %{{.+}}(ptr %{{.+}}, i32 %{{.+}}) + +// LLVM: define void @indirect_void(ptr %{{.+}}, ptr %{{.+}}, i32 %{{.+}}) +// LLVM: call void (ptr, ...) %{{.+}}(ptr %{{.+}}, i32 %{{.+}}) + +// LLVM: define void @indirect_coerced(ptr %{{.+}}, i64 %{{.+}}) +// LLVM: call i32 (i64, ...) %{{.+}}(i64 %{{.+}}) diff --git a/clang/test/CIR/Transforms/abi-lowering/x86_64-variadic-nyi.cir b/clang/test/CIR/Transforms/abi-lowering/x86_64-variadic-nyi.cir new file mode 100644 index 0000000000000..aae2cf94c8125 --- /dev/null +++ b/clang/test/CIR/Transforms/abi-lowering/x86_64-variadic-nyi.cir @@ -0,0 +1,240 @@ +// RUN: not cir-opt %s -cir-call-conv-lowering=target=x86_64 -split-input-file 2>&1 \ +// RUN: | FileCheck %s + +!s8i = !cir.int<s, 8> +!s32i = !cir.int<s, 32> +!rec_Pair = !cir.struct<"Pair" {!s32i, !s32i}> + +module attributes { + cir.triple = "x86_64-unknown-linux-gnu", + dlti.dl_spec = #dlti.dl_spec< + #dlti.dl_entry<i8, dense<8>: vector<2xi64>>, + #dlti.dl_entry<i32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<i64, dense<64>: vector<2xi64>>> +} { + + // A two-eightbyte record at the ellipsis coerces to a register pair the + // pointee's signature cannot describe, so the call is rejected. + cir.func @indirect_record(%arg0: !cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...) -> !s32i>>, + %arg1: !cir.ptr<!s8i>, %arg2: !rec_Pair) { + %0 = cir.call %arg0(%arg1, %arg2) + : (!cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...) -> !s32i>>, !cir.ptr<!s8i>, !rec_Pair) -> !s32i + cir.return + } + + // CHECK: error: 'cir.call' op variadic arguments to an indirect call not yet implemented in CallConvLowering +} + +// ----- + +!s8i = !cir.int<s, 8> +!s32i = !cir.int<s, 32> +!s64i = !cir.int<s, 64> +!rec_Big = !cir.struct<"Big" {!s64i, !s64i, !s64i}> + +module attributes { + cir.triple = "x86_64-unknown-linux-gnu", + dlti.dl_spec = #dlti.dl_spec< + #dlti.dl_entry<i8, dense<8>: vector<2xi64>>, + #dlti.dl_entry<i32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<i64, dense<64>: vector<2xi64>>> +} { + + // A memory-class record needs a byval slot at the ellipsis, which the + // pointee's signature cannot describe either. + cir.func @indirect_big(%arg0: !cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...) -> !s32i>>, + %arg1: !cir.ptr<!s8i>, %arg2: !rec_Big) { + %0 = cir.call %arg0(%arg1, %arg2) + : (!cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...) -> !s32i>>, !cir.ptr<!s8i>, !rec_Big) -> !s32i + cir.return + } + + // CHECK: error: 'cir.call' op variadic arguments to an indirect call not yet implemented in CallConvLowering +} + +// ----- + +!s8i = !cir.int<s, 8> +!s64i = !cir.int<s, 64> +!rec_Big = !cir.struct<"Big" {!s64i, !s64i, !s64i}> + +module attributes { + cir.triple = "x86_64-unknown-linux-gnu", + dlti.dl_spec = #dlti.dl_spec< + #dlti.dl_entry<i8, dense<8>: vector<2xi64>>, + #dlti.dl_entry<i32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<i64, dense<64>: vector<2xi64>>> +} { + + // A void return is its own wire form and so never needs a rewrite on its + // own, but it does not excuse an argument that still needs a byval slot. + cir.func @indirect_big_void(%arg0: !cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...)>>, + %arg1: !cir.ptr<!s8i>, %arg2: !rec_Big) { + cir.call %arg0(%arg1, %arg2) + : (!cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...)>>, !cir.ptr<!s8i>, !rec_Big) -> () + cir.return + } + + // CHECK: error: 'cir.call' op variadic arguments to an indirect call not yet implemented in CallConvLowering +} + +// ----- + +!s8i = !cir.int<s, 8> +!s32i = !cir.int<s, 32> +!rec_FF = !cir.struct<"FF" {!cir.float, !cir.float}> + +module attributes { + cir.triple = "x86_64-unknown-linux-gnu", + dlti.dl_spec = #dlti.dl_spec< + #dlti.dl_entry<i8, dense<8>: vector<2xi64>>, + #dlti.dl_entry<i32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<f32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<i64, dense<64>: vector<2xi64>>> +} { + + cir.func private @variadic(!cir.ptr<!s8i>, ...) -> !s32i + + // Classifying the call's own operands can reach a coercion the bridge cannot + // represent, here the <2 x float> an all-float eightbyte pair coerces to. + cir.func @ellipsis_unrepresentable(%arg0: !cir.ptr<!s8i>) { + %slot = cir.alloca "p" align(4) : !cir.ptr<!rec_FF> + %v = cir.load %slot : !cir.ptr<!rec_FF>, !rec_FF + %0 = cir.call @variadic(%arg0, %v) : (!cir.ptr<!s8i>, !rec_FF) -> !s32i + cir.return + } + + // CHECK: error: 'cir.call' op x86_64 calling-convention lowering not yet implemented for the ABI coercion of type '!cir.struct<"FF" {!cir.float, !cir.float}>' +} + +// ----- + +!s8i = !cir.int<s, 8> +!s32i = !cir.int<s, 32> + +module attributes { + cir.triple = "x86_64-unknown-linux-gnu", + dlti.dl_spec = #dlti.dl_spec< + #dlti.dl_entry<i8, dense<8>: vector<2xi64>>, + #dlti.dl_entry<i32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<i64, dense<64>: vector<2xi64>>> +} { + + cir.func no_proto private @kandr(!cir.ptr<!s8i>) -> !s32i + + // A callee declared without a prototype also takes more operands than it + // declares, but those extra arguments are named rather than passed through + // an ellipsis, so the variadic accounting does not apply to them. + cir.func @extra_args_no_proto(%arg0: !cir.ptr<!s8i>, %arg1: !s32i) { + %0 = cir.call @kandr(%arg0, %arg1) : (!cir.ptr<!s8i>, !s32i) -> !s32i + cir.return + } + + // CHECK: error: 'cir.call' op extra arguments to a callee without a prototype not yet implemented in CallConvLowering +} + +// ----- + +!s8i = !cir.int<s, 8> +!s32i = !cir.int<s, 32> +!s64i = !cir.int<s, 64> +!void = !cir.void +!rec_Big = !cir.struct<"Big" {!s64i, !s64i, !s64i}> + +module attributes { + cir.triple = "x86_64-unknown-linux-gnu", + dlti.dl_spec = #dlti.dl_spec< + #dlti.dl_entry<i8, dense<8>: vector<2xi64>>, + #dlti.dl_entry<i32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<i64, dense<64>: vector<2xi64>>> +} { + + // A cir.try_call reaches the same ellipsis accounting as cir.call, so a + // record needing a byval slot past the ellipsis is rejected there too. + cir.func @indirect_try_call(%arg0: !cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...) -> !s32i>>, + %arg1: !cir.ptr<!s8i>, %arg2: !rec_Big) { + %0 = cir.try_call %arg0(%arg1, %arg2) ^bb1, ^bb2 + : (!cir.ptr<!cir.func<(!cir.ptr<!s8i>, ...) -> !s32i>>, !cir.ptr<!s8i>, !rec_Big) -> !s32i + ^bb1: + cir.br ^bb4 + ^bb2: + %1 = cir.eh.initiate : !cir.eh_token + cir.eh.dispatch %1 : !cir.eh_token [ + catch_all : ^bb3 + ] + ^bb3(%tok : !cir.eh_token): + %ct, %exn = cir.begin_catch %tok : !cir.eh_token -> (!cir.catch_token, !cir.ptr<!void>) + cir.end_catch %ct : !cir.catch_token + cir.br ^bb4 + ^bb4: + cir.return + } + + // CHECK: error: 'cir.try_call' op variadic arguments to an indirect call not yet implemented in CallConvLowering +} + +// ----- + +!s32i = !cir.int<s, 32> +!void = !cir.void +!rec_Pair = !cir.struct<"Pair" {!s32i, !s32i}> + +module attributes { + cir.triple = "x86_64-unknown-linux-gnu", + dlti.dl_spec = #dlti.dl_spec< + #dlti.dl_entry<i8, dense<8>: vector<2xi64>>, + #dlti.dl_entry<i32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<i64, dense<64>: vector<2xi64>>> +} { + + // A variadic pointee is classified even when no argument is passed through + // the ellipsis, so the declared parameter's coercion is what needs the + // rewrite here, and rebuilding a cir.try_call is unimplemented. + cir.func @variadic_try_call_no_ellipsis( + %arg0: !cir.ptr<!cir.func<(!rec_Pair, ...) -> !s32i>>, %arg1: !rec_Pair) { + %0 = cir.try_call %arg0(%arg1) ^bb1, ^bb2 + : (!cir.ptr<!cir.func<(!rec_Pair, ...) -> !s32i>>, !rec_Pair) -> !s32i + ^bb1: + cir.br ^bb4 + ^bb2: + %1 = cir.eh.initiate : !cir.eh_token + cir.eh.dispatch %1 : !cir.eh_token [ + catch_all : ^bb3 + ] + ^bb3(%tok : !cir.eh_token): + %ct, %exn = cir.begin_catch %tok : !cir.eh_token -> (!cir.catch_token, !cir.ptr<!void>) + cir.end_catch %ct : !cir.catch_token + cir.br ^bb4 + ^bb4: + cir.return + } + + // CHECK: error: 'cir.try_call' op TryCallOp not yet implemented in CallConvLowering +} + +// ----- + +!s32i = !cir.int<s, 32> +!rec_Pair = !cir.struct<"Pair" {!s32i, !s32i}> + +module attributes { + cir.triple = "x86_64-unknown-linux-gnu", + dlti.dl_spec = #dlti.dl_spec< + #dlti.dl_entry<i8, dense<8>: vector<2xi64>>, + #dlti.dl_entry<i32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<i64, dense<64>: vector<2xi64>>> +} { + + cir.func no_proto private @kandr_two(!rec_Pair, !rec_Pair) -> !s32i + + // no_proto turns off the verifier's argument-count check in both directions, + // so a call can also pass fewer arguments than the callee declares. The + // classification has an entry per declared parameter with no operand to + // pair it with. + cir.func @too_few_no_proto(%arg0: !rec_Pair) { + %0 = cir.call @kandr_two(%arg0) : (!rec_Pair) -> !s32i + cir.return + } + + // CHECK: error: 'cir.call' op call passes fewer arguments than the callee declares, which is not yet implemented in CallConvLowering +} diff --git a/mlir/include/mlir/ABI/ABIRewriteContext.h b/mlir/include/mlir/ABI/ABIRewriteContext.h index 1982110c5ad6e..8394aeccc75ce 100644 --- a/mlir/include/mlir/ABI/ABIRewriteContext.h +++ b/mlir/include/mlir/ABI/ABIRewriteContext.h @@ -24,6 +24,7 @@ #include "mlir/IR/Types.h" #include "mlir/IR/Value.h" #include "mlir/Interfaces/FunctionInterfaces.h" +#include "llvm/ADT/STLExtras.h" #include "llvm/Support/Alignment.h" namespace mlir { @@ -74,6 +75,21 @@ struct ArgClassification { /// For Indirect: whether the callee gets ownership (byval). bool byVal = false; + /// Whether the value is passed as-is, so a rewriter can leave it alone. + /// Only an uncoerced Direct qualifies. Extend counts as needing a rewrite + /// even though it only adds an attribute, because the attribute changes + /// observable behavior. + bool isPassThrough() const { return kind == ArgKind::Direct && !coercedType; } + + /// Whether two classifications describe the same wire format. Every field + /// participates, so a field added above must be added here as well. + bool operator==(const ArgClassification &other) const { + return kind == other.kind && coercedType == other.coercedType && + indirectAlign == other.indirectAlign && + signExtend == other.signExtend && canFlatten == other.canFlatten && + byVal == other.byVal; + } + static ArgClassification getDirect(Type coerced = nullptr) { ArgClassification c; c.kind = ArgKind::Direct; @@ -109,6 +125,31 @@ struct ArgClassification { struct FunctionClassification { ArgClassification returnInfo; SmallVector<ArgClassification> argInfos; + + /// Whether the classified return type was the source language's void. + /// + /// A void return classifies as Ignore, and so does a return the ABI drops, + /// such as an empty record. The two need opposite treatment: void is + /// already its own wire form, while a dropped record return has to be + /// rewritten to one. returnInfo alone cannot tell them apart, so whoever + /// produces the classification records it here, next to the return type it + /// came from. A consumer that re-derived it from something else could pair + /// a classification with the wrong answer, and reading a dropped return as + /// void means silently skipping the rewrite it needs. + /// + /// Left false when unknown, which costs a needless rewrite rather than a + /// skipped one. + bool returnsVoid = false; + + /// Whether any value in the signature is passed differently from how it is + /// written, so a rewriter has work to do. + bool needsRewrite() const { + if (!returnsVoid && !returnInfo.isPassThrough()) + return true; + return !llvm::all_of(argInfos, [](const ArgClassification &ac) { + return ac.isPassThrough(); + }); + } }; /// ABIRewriteContext is the abstract interface that each dialect diff --git a/mlir/include/mlir/ABI/Targets/Test/TestTarget.h b/mlir/include/mlir/ABI/Targets/Test/TestTarget.h index 4404d47f8df45..8bf949bf3b67f 100644 --- a/mlir/include/mlir/ABI/Targets/Test/TestTarget.h +++ b/mlir/include/mlir/ABI/Targets/Test/TestTarget.h @@ -27,6 +27,7 @@ #include "mlir/ABI/ABIRewriteContext.h" #include "mlir/IR/BuiltinAttributes.h" #include "mlir/IR/Diagnostics.h" +#include "mlir/IR/TypeRange.h" #include "mlir/Interfaces/DataLayoutInterfaces.h" #include "llvm/Support/Error.h" @@ -44,7 +45,7 @@ namespace test { /// \param argTypes Argument types of the function. /// \param returnType Return type of the function. /// \param dl DataLayout used for size and alignment queries. -FunctionClassification classify(ArrayRef<Type> argTypes, Type returnType, +FunctionClassification classify(TypeRange argTypes, Type returnType, const DataLayout &dl); /// Parse a `FunctionClassification` from a plain MLIR DictionaryAttr. diff --git a/mlir/lib/ABI/Targets/Test/TestTarget.cpp b/mlir/lib/ABI/Targets/Test/TestTarget.cpp index 51510b0c18009..0cfdb2eb3fdb0 100644 --- a/mlir/lib/ABI/Targets/Test/TestTarget.cpp +++ b/mlir/lib/ABI/Targets/Test/TestTarget.cpp @@ -108,7 +108,7 @@ ArgClassification classifyOne(Type type, const DataLayout &dl) { } // namespace -FunctionClassification mlir::abi::test::classify(ArrayRef<Type> argTypes, +FunctionClassification mlir::abi::test::classify(TypeRange argTypes, Type returnType, const DataLayout &dl) { FunctionClassification fc; >From 011f23fdd7ab43a5a058d2901b44368832958cdf Mon Sep 17 00:00:00 2001 From: Adam Smith <[email protected]> Date: Sun, 2 Aug 2026 21:32:37 -0700 Subject: [PATCH 2/2] [CIR] Hoist indirect-call accessors onto CIRCallOpInterface CallConvLowering reached an indirect call's callee pointer through a dyn_cast dispatch over cir.call and cir.try_call, because both ops get isIndirect and getIndirectCall stamped in from a shared TableGen class rather than a common C++ base. The pass already walks call sites as CIRCallOpInterface, so it held the interface type and then cast back down to concrete ops to ask one question. Both accessors move from CIR_CallOpBase's extraClassDeclaration onto the interface, where DeclareOpInterfaceMethods generates the per-op declarations. getIndirectCall keeps its existing out-of-line definitions. isIndirect had been defined inline, so its body moves into the class's existing extraClassDefinition. indirectCalleeType now takes the interface, and the dispatch helper is gone. call-conv-lowering-x86_64-variadic.c wildcarded the operands of every checked instruction, so it pinned argument types but not which value reached which argument position: the two eightbytes of a record passed through the ellipsis could arrive at the call in either order and every check still passed. Each function now captures its incoming registers and asserts the path from each one to the argument position it reaches, and the checks pin every byval slot's alignment along with the CIR versus OGCG differences at the byval copy and the coerce reload. --- clang/include/clang/CIR/Dialect/IR/CIROps.td | 5 +- .../clang/CIR/Interfaces/CIROpInterfaces.td | 8 + .../Transforms/CallConvLoweringPass.cpp | 23 +- .../call-conv-lowering-x86_64-variadic.c | 274 ++++++++++++++---- 4 files changed, 235 insertions(+), 75 deletions(-) diff --git a/clang/include/clang/CIR/Dialect/IR/CIROps.td b/clang/include/clang/CIR/Dialect/IR/CIROps.td index 2d908a37070a9..fceeaba2c5484 100644 --- a/clang/include/clang/CIR/Dialect/IR/CIROps.td +++ b/clang/include/clang/CIR/Dialect/IR/CIROps.td @@ -4482,9 +4482,6 @@ class CIR_CallOpBase<string mnemonic, list<Trait> extra_traits = []> mlir::cast<mlir::SymbolRefAttr>(callee)); } - bool isIndirect() { return !getCallee(); } - mlir::Value getIndirectCall(); - /// Returns the `cir.func` being called or null if the call is indirect or /// cannot be resolved. cir::FuncOp resolveCalleeInTable(mlir::SymbolTableCollection &symbolTable); @@ -4501,6 +4498,8 @@ class CIR_CallOpBase<string mnemonic, list<Trait> extra_traits = []> }]; let extraClassDefinition = [{ + bool $cppClass::isIndirect() { return !getCallee(); } + cir::FuncOp $cppClass::resolveCalleeInTable( mlir::SymbolTableCollection &symbolTable) { mlir::FlatSymbolRefAttr callee = getCalleeAttr(); diff --git a/clang/include/clang/CIR/Interfaces/CIROpInterfaces.td b/clang/include/clang/CIR/Interfaces/CIROpInterfaces.td index 89fffa716c8c6..9f0e91c0ac40c 100644 --- a/clang/include/clang/CIR/Interfaces/CIROpInterfaces.td +++ b/clang/include/clang/CIR/Interfaces/CIROpInterfaces.td @@ -35,6 +35,14 @@ let cppNamespace = "::cir" in { "Return the number of operands, accounts for indirect call or " "exception info", "unsigned", "getNumArgOperands", (ins)>, + InterfaceMethod< + "Return whether the callee is a value resolved at run time rather " + "than a symbol named by the op", + "bool", "isIndirect", (ins)>, + InterfaceMethod< + "Return the callee pointer of an indirect call. Only valid when " + "isIndirect() is true", + "mlir::Value", "getIndirectCall", (ins)>, InterfaceMethod<"Return whether the callee is nothrow", "bool", "getNothrow", (ins)>, InterfaceMethod<"Return the side effects of the call operation", diff --git a/clang/lib/CIR/Dialect/Transforms/CallConvLoweringPass.cpp b/clang/lib/CIR/Dialect/Transforms/CallConvLoweringPass.cpp index b9ab1a6368506..2b201f88b8e27 100644 --- a/clang/lib/CIR/Dialect/Transforms/CallConvLoweringPass.cpp +++ b/clang/lib/CIR/Dialect/Transforms/CallConvLoweringPass.cpp @@ -533,29 +533,16 @@ cir::FuncOp lookupCallee(Operation *callOp, SymbolTable &symbolTable) { return symbolTable.lookup<cir::FuncOp>(callee.getValue()); } -/// The callee pointer of an indirect cir.call / cir.try_call, or nullptr for a -/// direct call. Both ops spell this the same way, but they inherit it from a -/// shared TableGen base rather than a common C++ type, so reaching it needs -/// this dispatch. -mlir::Value indirectCallee(Operation *callOp) { - if (auto call = dyn_cast<cir::CallOp>(callOp)) - return call.isIndirect() ? call.getIndirectCall() : nullptr; - if (auto tryCall = dyn_cast<cir::TryCallOp>(callOp)) - return tryCall.isIndirect() ? tryCall.getIndirectCall() : nullptr; - return nullptr; -} - /// The signature an indirect call reaches its callee through, or a null type /// for a direct call. The callee's pointer-to-function shape is asserted /// rather than verified: the dialect checks operand types against the callee /// only for a direct call, so IR that breaks it fails here instead of in the /// verifier. -cir::FuncType indirectCalleeType(Operation *callOp) { - mlir::Value callee = indirectCallee(callOp); - if (!callee) +cir::FuncType indirectCalleeType(cir::CIRCallOpInterface call) { + if (!call.isIndirect()) return {}; return cast<cir::FuncType>( - cast<cir::PointerType>(callee.getType()).getPointee()); + cast<cir::PointerType>(call.getIndirectCall().getType()).getPointee()); } void CallConvLoweringPass::runOnOperation() { @@ -711,7 +698,7 @@ void CallConvLoweringPass::runOnOperation() { // cached as the next one to visit. SmallVector<cir::CIRCallOpInterface> indirectCalls; moduleOp.walk([&](cir::CIRCallOpInterface c) { - cir::FuncType calleeTy = indirectCalleeType(c.getOperation()); + cir::FuncType calleeTy = indirectCalleeType(c); if (!calleeTy) return; // A cir.try_call is in this walk so that a variadic one reaches the @@ -733,7 +720,7 @@ void CallConvLoweringPass::runOnOperation() { signalPassFailure(); return; } - cir::FuncType funcTy = indirectCalleeType(c.getOperation()); + cir::FuncType funcTy = indirectCalleeType(c); auto classifySignature = [&](mlir::TypeRange argTypes) -> std::optional<FunctionClassification> { if (x86Target) diff --git a/clang/test/CIR/CodeGen/call-conv-lowering-x86_64-variadic.c b/clang/test/CIR/CodeGen/call-conv-lowering-x86_64-variadic.c index d3e28e58d2823..69edb45129abd 100644 --- a/clang/test/CIR/CodeGen/call-conv-lowering-x86_64-variadic.c +++ b/clang/test/CIR/CodeGen/call-conv-lowering-x86_64-variadic.c @@ -18,31 +18,74 @@ int vf(Pair2 p, ...); int call_scalar(Pair2 p, int a, double d) { return vf(p, a, d); } // CIR-LABEL: cir.func {{.*}}@call_scalar(%arg0: !u64i loc({{.+}}), %arg1: !s32i {llvm.noundef} loc({{.+}}), %arg2: !cir.double {llvm.noundef} loc({{.+}})) -> !s32i -// CIR: cir.call @vf(%{{.+}}, %{{.+}}, %{{.+}}) : (!u64i, !s32i {llvm.noundef}, !cir.double {llvm.noundef}) -> !s32i - -// LLVM-LABEL: i32 @call_scalar(i64 %{{.+}}, i32 noundef %{{.+}}, double noundef %{{.+}}) -// LLVM: call i32 (i64, ...) @vf(i64 %{{.+}}, i32 noundef %{{.+}}, double noundef %{{.+}}) +// CIR: cir.store %arg1, %[[ASLOT:[0-9]+]] : !s32i, !cir.ptr<!s32i> +// CIR: cir.store %arg2, %[[DSLOT:[0-9]+]] : !cir.double, !cir.ptr<!cir.double> +// CIR: %[[AV:[0-9]+]] = cir.load align(4) %[[ASLOT]] : !cir.ptr<!s32i>, !s32i +// CIR: %[[DV:[0-9]+]] = cir.load align(8) %[[DSLOT]] : !cir.ptr<!cir.double>, !cir.double +// CIR: %[[PV:[0-9]+]] = cir.load %{{[0-9]+}} : !cir.ptr<!u64i>, !u64i +// CIR: cir.call @vf(%[[PV]], %[[AV]], %[[DV]]) : (!u64i, !s32i {llvm.noundef}, !cir.double {llvm.noundef}) -> !s32i + +// LLVM-LABEL: define dso_local i32 @call_scalar( +// LLVM-SAME: i64 %[[P:[0-9a-zA-Z._]+]], i32 noundef %[[A:[0-9a-zA-Z._]+]], double noundef %[[D:[0-9a-zA-Z._]+]]) +// LLVM: store i32 %[[A]], ptr %[[ASLOT:[0-9a-zA-Z._]+]], align 4 +// LLVM: store double %[[D]], ptr %[[DSLOT:[0-9a-zA-Z._]+]], align 8 +// LLVM: %[[AV:[0-9a-zA-Z._]+]] = load i32, ptr %[[ASLOT]], align 4 +// LLVM: %[[DV:[0-9a-zA-Z._]+]] = load double, ptr %[[DSLOT]], align 8 +// LLVM: %[[PV:[0-9a-zA-Z._]+]] = load i64, ptr %{{[0-9a-zA-Z._]+}}, align +// LLVM: call i32 (i64, ...) @vf(i64 %[[PV]], i32 noundef %[[AV]], double noundef %[[DV]]) // A two-eightbyte record at the ellipsis is flattened into two INTEGER // registers while registers remain. int call_small(Pair2 p, Pair16 q) { return vf(p, q); } // CIR-LABEL: cir.func {{.*}}@call_small(%arg0: !u64i loc({{.+}}), %arg1: !s64i loc({{.+}}), %arg2: !s64i loc({{.+}})) -> !s32i -// CIR: cir.call @vf(%{{.+}}, %{{.+}}, %{{.+}}) : (!u64i, !s64i, !s64i) -> !s32i - -// LLVM-LABEL: i32 @call_small(i64 %{{.+}}, i64 %{{.+}}, i64 %{{.+}}) -// LLVM: call i32 (i64, ...) @vf(i64 %{{.+}}, i64 %{{.+}}, i64 %{{.+}}) +// CIR: %[[IN0:[0-9]+]] = cir.get_member %[[IN:[0-9]+]][0] {name = ""} : !cir.ptr<!rec_anon_struct> -> !cir.ptr<!s64i> +// CIR: cir.store %arg1, %[[IN0]] : !s64i, !cir.ptr<!s64i> +// CIR: %[[IN1:[0-9]+]] = cir.get_member %[[IN]][1] {name = ""} : !cir.ptr<!rec_anon_struct> -> !cir.ptr<!s64i> +// CIR: cir.store %arg2, %[[IN1]] : !s64i, !cir.ptr<!s64i> +// CIR: %[[PV:[0-9]+]] = cir.load %{{[0-9]+}} : !cir.ptr<!u64i>, !u64i +// CIR: %[[OUT0:[0-9]+]] = cir.get_member %[[OUT:[0-9]+]][0] {name = ""} : !cir.ptr<!rec_anon_struct> -> !cir.ptr<!s64i> +// CIR: %[[Q0:[0-9]+]] = cir.load %[[OUT0]] : !cir.ptr<!s64i>, !s64i +// CIR: %[[OUT1:[0-9]+]] = cir.get_member %[[OUT]][1] {name = ""} : !cir.ptr<!rec_anon_struct> -> !cir.ptr<!s64i> +// CIR: %[[Q1:[0-9]+]] = cir.load %[[OUT1]] : !cir.ptr<!s64i>, !s64i +// CIR: cir.call @vf(%[[PV]], %[[Q0]], %[[Q1]]) : (!u64i, !s64i, !s64i) -> !s32i + +// LLVM-LABEL: define dso_local i32 @call_small( +// LLVM-SAME: i64 %[[P:[0-9a-zA-Z._]+]], i64 %[[Q0:[0-9a-zA-Z._]+]], i64 %[[Q1:[0-9a-zA-Z._]+]]) +// LLVM: %[[S0:[0-9a-zA-Z._]+]] = getelementptr inbounds nuw { i64, i64 }, ptr %[[SLOT:[0-9a-zA-Z._]+]], i32 0, i32 0 +// LLVM: store i64 %[[Q0]], ptr %[[S0]], align 8 +// LLVM: %[[S1:[0-9a-zA-Z._]+]] = getelementptr inbounds nuw { i64, i64 }, ptr %[[SLOT]], i32 0, i32 1 +// LLVM: store i64 %[[Q1]], ptr %[[S1]], align 8 +// LLVM: %[[PV:[0-9a-zA-Z._]+]] = load i64, ptr %{{[0-9a-zA-Z._]+}}, align +// LLVM: %[[L0:[0-9a-zA-Z._]+]] = getelementptr inbounds nuw { i64, i64 }, ptr %[[REC:[0-9a-zA-Z._]+]], i32 0, i32 0 +// LLVM: %[[A0:[0-9a-zA-Z._]+]] = load i64, ptr %[[L0]], align 8 +// LLVM: %[[L1:[0-9a-zA-Z._]+]] = getelementptr inbounds nuw { i64, i64 }, ptr %[[REC]], i32 0, i32 1 +// LLVM: %[[A1:[0-9a-zA-Z._]+]] = load i64, ptr %[[L1]], align 8 +// LLVM: call i32 (i64, ...) @vf(i64 %[[PV]], i64 %[[A0]], i64 %[[A1]]) // Larger than two eightbytes is MEMORY regardless of register availability. int call_big(Pair2 p, Big b) { return vf(p, b); } // CIR-LABEL: cir.func {{.*}}@call_big(%arg0: !u64i loc({{.+}}), %arg1: !cir.ptr<!rec_Big> {llvm.align = 8 : i64, llvm.byval = !rec_Big, llvm.noalias, llvm.noundef} loc({{.+}})) -> !s32i -// CIR: cir.call @vf(%{{.+}}, %{{.+}}) : (!u64i, !cir.ptr<!rec_Big> {llvm.align = 8 : i64, llvm.byval = !rec_Big, llvm.noalias, llvm.noundef}) -> !s32i - -// LLVM-CIR-LABEL: i32 @call_big(i64 %{{.+}}, ptr noalias noundef byval(%struct.Big) align 8 %{{.+}}) -// LLVM-CIR: call i32 (i64, ...) @vf(i64 %{{.+}}, ptr noalias noundef byval(%struct.Big) align 8 %{{.+}}) -// LLVM-OGCG-LABEL: i32 @call_big(i64 %{{.+}}, ptr noundef byval(%struct.Big) align 8 %{{.+}}) -// LLVM-OGCG: call i32 (i64, ...) @vf(i64 %{{.+}}, ptr noundef byval(%struct.Big) align 8 %{{.+}}) +// CIR: %{{[0-9]+}} = cir.load %arg1 : !cir.ptr<!rec_Big>, !rec_Big +// CIR: %[[PV:[0-9]+]] = cir.load %{{[0-9]+}} : !cir.ptr<!u64i>, !u64i +// CIR-NEXT: %[[COPY:[0-9]+]] = cir.alloca "byval" align(8) : !cir.ptr<!rec_Big> +// CIR-NEXT: cir.store %{{[0-9]+}}, %[[COPY]] : !rec_Big, !cir.ptr<!rec_Big> +// CIR-NEXT: %{{[0-9]+}} = cir.call @vf(%[[PV]], %[[COPY]]) : (!u64i, !cir.ptr<!rec_Big> {llvm.align = 8 : i64, llvm.byval = !rec_Big, llvm.noalias, llvm.noundef}) -> !s32i + +// CIR copies the incoming byval slot before forwarding it. OGCG does not. +// LLVM-CIR-LABEL: define dso_local i32 @call_big( +// LLVM-CIR-SAME: i64 %[[P:[0-9a-zA-Z._]+]], ptr noalias noundef byval(%struct.Big) align 8 %[[B:[0-9a-zA-Z._]+]]) +// LLVM-CIR: %{{[0-9a-zA-Z._]+}} = load %struct.Big, ptr %[[B]], align 8 +// LLVM-CIR: %[[PV:[0-9a-zA-Z._]+]] = load i64, ptr %{{[0-9a-zA-Z._]+}}, align 8 +// LLVM-CIR-NEXT: %[[COPY:[0-9a-zA-Z._]+]] = alloca %struct.Big, i64 1, align 8 +// LLVM-CIR-NEXT: store %struct.Big %{{[0-9a-zA-Z._]+}}, ptr %[[COPY]], align 8 +// LLVM-CIR-NEXT: %{{[0-9a-zA-Z._]+}} = call i32 (i64, ...) @vf(i64 %[[PV]], ptr noalias noundef byval(%struct.Big) align 8 %[[COPY]]) + +// LLVM-OGCG-LABEL: define dso_local i32 @call_big( +// LLVM-OGCG-SAME: i64 %[[P:[0-9a-zA-Z._]+]], ptr noundef byval(%struct.Big) align 8 %[[B:[0-9a-zA-Z._]+]]) +// LLVM-OGCG: %[[PV:[0-9a-zA-Z._]+]] = load i64, ptr %{{[0-9a-zA-Z._]+}}, align 4 +// LLVM-OGCG-NEXT: %{{[0-9a-zA-Z._]+}} = call i32 (i64, ...) @vf(i64 %[[PV]], ptr noundef byval(%struct.Big) align 8 %[[B]]) // The same Pair16 that went to registers in call_small goes to memory here: // the named parameter and four longs leave only one INTEGER register, and a @@ -52,82 +95,205 @@ int call_exhausted(Pair2 p, long a, long b, long c, long d, Pair16 q) { } // CIR-LABEL: cir.func {{.*}}@call_exhausted(%arg0: !u64i loc({{.+}}), %arg1: !s64i {llvm.noundef} loc({{.+}}), %arg2: !s64i {llvm.noundef} loc({{.+}}), %arg3: !s64i {llvm.noundef} loc({{.+}}), %arg4: !s64i {llvm.noundef} loc({{.+}}), %arg5: !cir.ptr<!rec_Pair16> {llvm.align = 8 : i64, llvm.byval = !rec_Pair16, llvm.noalias, llvm.noundef} loc({{.+}})) -> !s32i -// CIR: cir.call @vf(%{{.+}}, %{{.+}}, %{{.+}}, %{{.+}}, %{{.+}}, %{{.+}}) : (!u64i, !s64i {llvm.noundef}, !s64i {llvm.noundef}, !s64i {llvm.noundef}, !s64i {llvm.noundef}, !cir.ptr<!rec_Pair16> {llvm.align = 8 : i64, llvm.byval = !rec_Pair16, llvm.noalias, llvm.noundef}) -> !s32i - -// LLVM-CIR-LABEL: i32 @call_exhausted(i64 %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, ptr noalias noundef byval(%struct.Pair16) align 8 %{{.+}}) -// LLVM-CIR: call i32 (i64, ...) @vf(i64 %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, ptr noalias noundef byval(%struct.Pair16) align 8 %{{.+}}) -// LLVM-OGCG-LABEL: i32 @call_exhausted(i64 %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, ptr noundef byval(%struct.Pair16) align 8 %{{.+}}) -// LLVM-OGCG: call i32 (i64, ...) @vf(i64 %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}, ptr noundef byval(%struct.Pair16) align 8 %{{.+}}) +// CIR: cir.store %arg1, %[[AS:[0-9]+]] : !s64i, !cir.ptr<!s64i> +// CIR: cir.store %arg2, %[[BS:[0-9]+]] : !s64i, !cir.ptr<!s64i> +// CIR: cir.store %arg3, %[[CS:[0-9]+]] : !s64i, !cir.ptr<!s64i> +// CIR: cir.store %arg4, %[[DS:[0-9]+]] : !s64i, !cir.ptr<!s64i> +// CIR: %[[AV:[0-9]+]] = cir.load align(8) %[[AS]] : !cir.ptr<!s64i>, !s64i +// CIR: %[[BV:[0-9]+]] = cir.load align(8) %[[BS]] : !cir.ptr<!s64i>, !s64i +// CIR: %[[CV:[0-9]+]] = cir.load align(8) %[[CS]] : !cir.ptr<!s64i>, !s64i +// CIR: %[[DV:[0-9]+]] = cir.load align(8) %[[DS]] : !cir.ptr<!s64i>, !s64i +// CIR: %[[PV:[0-9]+]] = cir.load %{{[0-9]+}} : !cir.ptr<!u64i>, !u64i +// CIR-NEXT: %[[COPY:[0-9]+]] = cir.alloca "byval" align(8) : !cir.ptr<!rec_Pair16> +// CIR-NEXT: cir.store %{{[0-9]+}}, %[[COPY]] : !rec_Pair16, !cir.ptr<!rec_Pair16> +// CIR-NEXT: %{{[0-9]+}} = cir.call @vf(%[[PV]], %[[AV]], %[[BV]], %[[CV]], %[[DV]], %[[COPY]]) : (!u64i, !s64i {llvm.noundef}, !s64i {llvm.noundef}, !s64i {llvm.noundef}, !s64i {llvm.noundef}, !cir.ptr<!rec_Pair16> {llvm.align = 8 : i64, llvm.byval = !rec_Pair16, llvm.noalias, llvm.noundef}) -> !s32i + +// LLVM-CIR-LABEL: define dso_local i32 @call_exhausted( +// LLVM-CIR-SAME: i64 %[[P:[0-9a-zA-Z._]+]], i64 noundef %[[A:[0-9a-zA-Z._]+]], i64 noundef %[[B:[0-9a-zA-Z._]+]], i64 noundef %[[C:[0-9a-zA-Z._]+]], i64 noundef %[[D:[0-9a-zA-Z._]+]], ptr noalias noundef byval(%struct.Pair16) align 8 %[[Q:[0-9a-zA-Z._]+]]) +// LLVM-OGCG-LABEL: define dso_local i32 @call_exhausted( +// LLVM-OGCG-SAME: i64 %[[P:[0-9a-zA-Z._]+]], i64 noundef %[[A:[0-9a-zA-Z._]+]], i64 noundef %[[B:[0-9a-zA-Z._]+]], i64 noundef %[[C:[0-9a-zA-Z._]+]], i64 noundef %[[D:[0-9a-zA-Z._]+]], ptr noundef byval(%struct.Pair16) align 8 %[[Q:[0-9a-zA-Z._]+]]) +// LLVM: store i64 %[[A]], ptr %[[AS:[0-9a-zA-Z._]+]], align 8 +// LLVM: store i64 %[[B]], ptr %[[BS:[0-9a-zA-Z._]+]], align 8 +// LLVM: store i64 %[[C]], ptr %[[CS:[0-9a-zA-Z._]+]], align 8 +// LLVM: store i64 %[[D]], ptr %[[DS:[0-9a-zA-Z._]+]], align 8 +// LLVM: %[[AV:[0-9a-zA-Z._]+]] = load i64, ptr %[[AS]], align 8 +// LLVM: %[[BV:[0-9a-zA-Z._]+]] = load i64, ptr %[[BS]], align 8 +// LLVM: %[[CV:[0-9a-zA-Z._]+]] = load i64, ptr %[[CS]], align 8 +// LLVM: %[[DV:[0-9a-zA-Z._]+]] = load i64, ptr %[[DS]], align 8 +// LLVM-CIR: %[[PV:[0-9a-zA-Z._]+]] = load i64, ptr %{{[0-9a-zA-Z._]+}}, align 8 +// LLVM-CIR-NEXT: %[[COPY:[0-9a-zA-Z._]+]] = alloca %struct.Pair16, i64 1, align 8 +// LLVM-CIR-NEXT: store %struct.Pair16 %{{[0-9a-zA-Z._]+}}, ptr %[[COPY]], align 8 +// LLVM-CIR-NEXT: %{{[0-9a-zA-Z._]+}} = call i32 (i64, ...) @vf(i64 %[[PV]], i64 noundef %[[AV]], i64 noundef %[[BV]], i64 noundef %[[CV]], i64 noundef %[[DV]], ptr noalias noundef byval(%struct.Pair16) align 8 %[[COPY]]) +// LLVM-OGCG: %[[PV:[0-9a-zA-Z._]+]] = load i64, ptr %{{[0-9a-zA-Z._]+}}, align 4 +// LLVM-OGCG-NEXT: %{{[0-9a-zA-Z._]+}} = call i32 (i64, ...) @vf(i64 %[[PV]], i64 noundef %[[AV]], i64 noundef %[[BV]], i64 noundef %[[CV]], i64 noundef %[[DV]], ptr noundef byval(%struct.Pair16) align 8 %[[Q]]) // A 128-bit integer spans two eightbytes but is still passed whole. int call_int128(Pair2 p, __int128 w) { return vf(p, w); } // CIR-LABEL: cir.func {{.*}}@call_int128(%arg0: !u64i loc({{.+}}), %arg1: !s128i {llvm.noundef} loc({{.+}})) -> !s32i -// CIR: cir.call @vf(%{{.+}}, %{{.+}}) : (!u64i, !s128i {llvm.noundef}) -> !s32i - -// LLVM-LABEL: i32 @call_int128(i64 %{{.+}}, i128 noundef %{{.+}}) -// LLVM: call i32 (i64, ...) @vf(i64 %{{.+}}, i128 noundef %{{.+}}) +// CIR: cir.store %arg1, %[[WSLOT:[0-9]+]] : !s128i, !cir.ptr<!s128i> +// CIR: %[[WV:[0-9]+]] = cir.load align(16) %[[WSLOT]] : !cir.ptr<!s128i>, !s128i +// CIR: %[[PV:[0-9]+]] = cir.load %{{[0-9]+}} : !cir.ptr<!u64i>, !u64i +// CIR: cir.call @vf(%[[PV]], %[[WV]]) : (!u64i, !s128i {llvm.noundef}) -> !s32i + +// LLVM-LABEL: define dso_local i32 @call_int128( +// LLVM-SAME: i64 %[[P:[0-9a-zA-Z._]+]], i128 noundef %[[W:[0-9a-zA-Z._]+]]) +// LLVM: store i128 %[[W]], ptr %[[WSLOT:[0-9a-zA-Z._]+]], align 16 +// LLVM: %[[WV:[0-9a-zA-Z._]+]] = load i128, ptr %[[WSLOT]], align 16 +// LLVM: %[[PV:[0-9a-zA-Z._]+]] = load i64, ptr %{{[0-9a-zA-Z._]+}}, align +// LLVM: call i32 (i64, ...) @vf(i64 %[[PV]], i128 noundef %[[WV]]) // Wrapping it in a record does not change the class: both eightbytes are // INTEGER, so the record is coerced back to a bare i128. int call_wide(Pair2 p, Wide w) { return vf(p, w); } // CIR-LABEL: cir.func {{.*}}@call_wide(%arg0: !u64i loc({{.+}}), %arg1: !s128i loc({{.+}})) -> !s32i -// CIR: cir.call @vf(%{{.+}}, %{{.+}}) : (!u64i, !s128i) -> !s32i - -// LLVM-LABEL: i32 @call_wide(i64 %{{.+}}, i128 %{{.+}}) -// LLVM: call i32 (i64, ...) @vf(i64 %{{.+}}, i128 %{{.+}}) +// CIR: cir.store %arg1, %[[IN:[0-9]+]] : !s128i, !cir.ptr<!s128i> +// CIR: %[[INCAST:[0-9]+]] = cir.cast bitcast %[[IN]] : !cir.ptr<!s128i> -> !cir.ptr<!rec_Wide> +// CIR: %{{[0-9]+}} = cir.load %[[INCAST]] : !cir.ptr<!rec_Wide>, !rec_Wide +// CIR: %[[PV:[0-9]+]] = cir.load %{{[0-9]+}} : !cir.ptr<!u64i>, !u64i +// CIR: cir.store %{{[0-9]+}}, %[[OUT:[0-9]+]] : !rec_Wide, !cir.ptr<!rec_Wide> +// CIR: %[[OUTCAST:[0-9]+]] = cir.cast bitcast %[[OUT]] : !cir.ptr<!rec_Wide> -> !cir.ptr<!s128i> +// CIR: %[[WV:[0-9]+]] = cir.load %[[OUTCAST]] : !cir.ptr<!s128i>, !s128i +// CIR: cir.call @vf(%[[PV]], %[[WV]]) : (!u64i, !s128i) -> !s32i + +// LLVM-LABEL: define dso_local i32 @call_wide( +// LLVM-SAME: i64 %[[P:[0-9a-zA-Z._]+]], i128 %[[W:[0-9a-zA-Z._]+]]) +// LLVM-CIR: store i128 %[[W]], ptr %[[IN:[0-9a-zA-Z._]+]], align 16 +// LLVM-CIR: %{{[0-9a-zA-Z._]+}} = load %struct.Wide, ptr %[[IN]], align 16 +// LLVM-CIR: %[[PV:[0-9a-zA-Z._]+]] = load i64, ptr %{{[0-9a-zA-Z._]+}}, align 8 +// LLVM-CIR: store %struct.Wide %{{[0-9a-zA-Z._]+}}, ptr %[[OUT:[0-9a-zA-Z._]+]], align 16 +// LLVM-CIR-NEXT: %[[WV:[0-9a-zA-Z._]+]] = load i128, ptr %[[OUT]], align 16 + +// LLVM-OGCG: %[[DIVE:[0-9a-zA-Z._]+]] = getelementptr inbounds nuw %struct.Wide, ptr %[[WSLOT:[0-9a-zA-Z._]+]], i32 0, i32 0 +// LLVM-OGCG-NEXT: store i128 %[[W]], ptr %[[DIVE]], align 16 +// LLVM-OGCG: %[[PV:[0-9a-zA-Z._]+]] = load i64, ptr %{{[0-9a-zA-Z._]+}}, align 4 +// LLVM-OGCG-NEXT: %[[DIVE1:[0-9a-zA-Z._]+]] = getelementptr inbounds nuw %struct.Wide, ptr %[[WSLOT]], i32 0, i32 0 +// LLVM-OGCG-NEXT: %[[WV:[0-9a-zA-Z._]+]] = load i128, ptr %[[DIVE1]], align 16 + +// LLVM-NEXT: %{{[0-9a-zA-Z._]+}} = call i32 (i64, ...) @vf(i64 %[[PV]], i128 %[[WV]]) // One trailing byte pushes the record past two eightbytes, so it goes to // memory, and the 128-bit member keeps the slot at 16-byte alignment. int call_wide_char(Pair2 p, WideChar w) { return vf(p, w); } // CIR-LABEL: cir.func {{.*}}@call_wide_char(%arg0: !u64i loc({{.+}}), %arg1: !cir.ptr<!rec_WideChar> {llvm.align = 16 : i64, llvm.byval = !rec_WideChar, llvm.noalias, llvm.noundef} loc({{.+}})) -> !s32i -// CIR: cir.call @vf(%{{.+}}, %{{.+}}) : (!u64i, !cir.ptr<!rec_WideChar> {llvm.align = 16 : i64, llvm.byval = !rec_WideChar, llvm.noalias, llvm.noundef}) -> !s32i - -// LLVM-CIR-LABEL: i32 @call_wide_char(i64 %{{.+}}, ptr noalias noundef byval(%struct.WideChar) align 16 %{{.+}}) -// LLVM-CIR: call i32 (i64, ...) @vf(i64 %{{.+}}, ptr noalias noundef byval(%struct.WideChar) align 16 %{{.+}}) -// LLVM-OGCG-LABEL: i32 @call_wide_char(i64 %{{.+}}, ptr noundef byval(%struct.WideChar) align 16 %{{.+}}) -// LLVM-OGCG: call i32 (i64, ...) @vf(i64 %{{.+}}, ptr noundef byval(%struct.WideChar) align 16 %{{.+}}) +// CIR: %{{[0-9]+}} = cir.load %arg1 : !cir.ptr<!rec_WideChar>, !rec_WideChar +// CIR: %[[PV:[0-9]+]] = cir.load %{{[0-9]+}} : !cir.ptr<!u64i>, !u64i +// CIR-NEXT: %[[COPY:[0-9]+]] = cir.alloca "byval" align(16) : !cir.ptr<!rec_WideChar> +// CIR-NEXT: cir.store %{{[0-9]+}}, %[[COPY]] : !rec_WideChar, !cir.ptr<!rec_WideChar> +// CIR-NEXT: %{{[0-9]+}} = cir.call @vf(%[[PV]], %[[COPY]]) : (!u64i, !cir.ptr<!rec_WideChar> {llvm.align = 16 : i64, llvm.byval = !rec_WideChar, llvm.noalias, llvm.noundef}) -> !s32i + +// LLVM-CIR-LABEL: define dso_local i32 @call_wide_char( +// LLVM-CIR-SAME: i64 %[[P:[0-9a-zA-Z._]+]], ptr noalias noundef byval(%struct.WideChar) align 16 %[[W:[0-9a-zA-Z._]+]]) +// LLVM-CIR: %{{[0-9a-zA-Z._]+}} = load %struct.WideChar, ptr %[[W]], align 16 +// LLVM-CIR: %[[PV:[0-9a-zA-Z._]+]] = load i64, ptr %{{[0-9a-zA-Z._]+}}, align 8 +// LLVM-CIR-NEXT: %[[COPY:[0-9a-zA-Z._]+]] = alloca %struct.WideChar, i64 1, align 16 +// LLVM-CIR-NEXT: store %struct.WideChar %{{[0-9a-zA-Z._]+}}, ptr %[[COPY]], align 16 +// LLVM-CIR-NEXT: %{{[0-9a-zA-Z._]+}} = call i32 (i64, ...) @vf(i64 %[[PV]], ptr noalias noundef byval(%struct.WideChar) align 16 %[[COPY]]) + +// LLVM-OGCG-LABEL: define dso_local i32 @call_wide_char( +// LLVM-OGCG-SAME: i64 %[[P:[0-9a-zA-Z._]+]], ptr noundef byval(%struct.WideChar) align 16 %[[W:[0-9a-zA-Z._]+]]) +// LLVM-OGCG: %[[PV:[0-9a-zA-Z._]+]] = load i64, ptr %{{[0-9a-zA-Z._]+}}, align 4 +// LLVM-OGCG-NEXT: %{{[0-9a-zA-Z._]+}} = call i32 (i64, ...) @vf(i64 %[[PV]], ptr noundef byval(%struct.WideChar) align 16 %[[W]]) // A _BitInt narrower than a register is extended at the ellipsis, same as a // declared parameter. int ell_bitint17(Pair2 p, _BitInt(17) b) { return vf(p, b); } // CIR-LABEL: cir.func {{.*}}@ell_bitint17(%arg0: !u64i loc({{.+}}), %arg1: !cir.int<s, 17, bitint> {llvm.signext} loc({{.+}})) -> !s32i -// CIR: cir.call @vf(%{{.+}}, %{{.+}}) : (!u64i, !cir.int<s, 17, bitint> {llvm.signext}) -> !s32i - -// LLVM-CIR-LABEL: i32 @ell_bitint17(i64 %{{.+}}, i17 signext %{{.+}}) -// LLVM-CIR: call i32 (i64, ...) @vf(i64 %{{.+}}, i17 signext %{{.+}}) -// LLVM-OGCG-LABEL: i32 @ell_bitint17(i64 %{{.+}}, i17 noundef signext %{{.+}}) -// LLVM-OGCG: call i32 (i64, ...) @vf(i64 %{{.+}}, i17 noundef signext %{{.+}}) +// CIR: cir.store %arg1, %[[BSLOT:[0-9]+]] : !cir.int<s, 17, bitint>, !cir.ptr<!cir.int<s, 17, bitint>> +// CIR: %[[BV:[0-9]+]] = cir.load align(4) %[[BSLOT]] : !cir.ptr<!cir.int<s, 17, bitint>>, !cir.int<s, 17, bitint> +// CIR: %[[PV:[0-9]+]] = cir.load %{{[0-9]+}} : !cir.ptr<!u64i>, !u64i +// CIR: cir.call @vf(%[[PV]], %[[BV]]) : (!u64i, !cir.int<s, 17, bitint> {llvm.signext}) -> !s32i + +// LLVM-CIR-LABEL: define dso_local i32 @ell_bitint17( +// LLVM-CIR-SAME: i64 %[[P:[0-9a-zA-Z._]+]], i17 signext %[[B:[0-9a-zA-Z._]+]]) +// LLVM-OGCG-LABEL: define dso_local i32 @ell_bitint17( +// LLVM-OGCG-SAME: i64 %[[P:[0-9a-zA-Z._]+]], i17 noundef signext %[[B:[0-9a-zA-Z._]+]]) +// LLVM: %[[EXT:[0-9a-zA-Z._]+]] = sext i17 %[[B]] to i32 +// LLVM: store i32 %[[EXT]], ptr %[[BSLOT:[0-9a-zA-Z._]+]], align 4 +// LLVM: %[[RE:[0-9a-zA-Z._]+]] = load i32, ptr %[[BSLOT]], align 4 +// LLVM: %[[TR:[0-9a-zA-Z._]+]] = trunc i32 %[[RE]] to i17 +// LLVM: %[[PV:[0-9a-zA-Z._]+]] = load i64, ptr %{{[0-9a-zA-Z._]+}}, align +// LLVM-CIR: call i32 (i64, ...) @vf(i64 %[[PV]], i17 signext %[[TR]]) +// LLVM-OGCG: call i32 (i64, ...) @vf(i64 %[[PV]], i17 noundef signext %[[TR]]) // A width between 33 and 63 widens to one register. int ell_bitint48(Pair2 p, _BitInt(48) b) { return vf(p, b); } // CIR-LABEL: cir.func {{.*}}@ell_bitint48(%arg0: !u64i loc({{.+}}), %arg1: !u64i {llvm.noundef} loc({{.+}})) -> !s32i -// CIR: cir.call @vf(%{{.+}}, %{{.+}}) : (!u64i, !u64i {llvm.noundef}) -> !s32i - -// LLVM-LABEL: i32 @ell_bitint48(i64 %{{.+}}, i64 noundef %{{.+}}) -// LLVM: call i32 (i64, ...) @vf(i64 %{{.+}}, i64 noundef %{{.+}}) +// CIR: %[[BV:[0-9]+]] = cir.load align(8) %{{[0-9]+}} : !cir.ptr<!cir.int<s, 48, bitint>>, !cir.int<s, 48, bitint> +// CIR: %[[PV:[0-9]+]] = cir.load %{{[0-9]+}} : !cir.ptr<!u64i>, !u64i +// CIR: %[[OUTCAST:[0-9]+]] = cir.cast bitcast %[[OUT:[0-9]+]] : !cir.ptr<!u64i> -> !cir.ptr<!cir.int<s, 48, bitint>> +// CIR-NEXT: cir.store %[[BV]], %[[OUTCAST]] : !cir.int<s, 48, bitint>, !cir.ptr<!cir.int<s, 48, bitint>> +// CIR-NEXT: %[[ARG:[0-9]+]] = cir.load %[[OUT]] : !cir.ptr<!u64i>, !u64i +// CIR-NEXT: %{{[0-9]+}} = cir.call @vf(%[[PV]], %[[ARG]]) : (!u64i, !u64i {llvm.noundef}) -> !s32i + +// LLVM-LABEL: define dso_local i32 @ell_bitint48( +// LLVM-SAME: i64 %[[P:[0-9a-zA-Z._]+]], i64 noundef %[[B:[0-9a-zA-Z._]+]]) +// LLVM: %[[TR:[0-9a-zA-Z._]+]] = trunc i64 %{{[0-9a-zA-Z._]+}} to i48 +// LLVM: %[[EXT:[0-9a-zA-Z._]+]] = sext i48 %[[TR]] to i64 +// LLVM: store i64 %[[EXT]], ptr %[[BSLOT:[0-9a-zA-Z._]+]], align 8 +// LLVM: %[[RE:[0-9a-zA-Z._]+]] = load i64, ptr %[[BSLOT]], align 8 +// LLVM: %[[TR2:[0-9a-zA-Z._]+]] = trunc i64 %[[RE]] to i48 +// LLVM: %[[PV:[0-9a-zA-Z._]+]] = load i64, ptr %{{[0-9a-zA-Z._]+}}, align +// LLVM: %[[EXT2:[0-9a-zA-Z._]+]] = sext i48 %[[TR2]] to i64 +// LLVM: store i64 %[[EXT2]], ptr %[[CSLOT:[0-9a-zA-Z._]+]], align 8 +// LLVM: %[[ARG:[0-9a-zA-Z._]+]] = load i64, ptr %[[CSLOT]], align 8 +// LLVM: call i32 (i64, ...) @vf(i64 %[[PV]], i64 noundef %[[ARG]]) // A width between 65 and 127 coerces to a register pair, and both halves are // passed through the ellipsis. int ell_bitint96(Pair2 p, _BitInt(96) b) { return vf(p, b); } // CIR-LABEL: cir.func {{.*}}@ell_bitint96(%arg0: !u64i loc({{.+}}), %arg1: !u64i loc({{.+}}), %arg2: !u64i loc({{.+}})) -> !s32i -// CIR: cir.call @vf(%{{.+}}, %{{.+}}, %{{.+}}) : (!u64i, !u64i, !u64i) -> !s32i - -// LLVM-CIR-LABEL: i32 @ell_bitint96(i64 %{{.+}}, i64 %{{.+}}, i64 %{{.+}}) -// LLVM-CIR: call i32 (i64, ...) @vf(i64 %{{.+}}, i64 %{{.+}}, i64 %{{.+}}) -// LLVM-OGCG-LABEL: i32 @ell_bitint96(i64 %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}) -// LLVM-OGCG: call i32 (i64, ...) @vf(i64 %{{.+}}, i64 noundef %{{.+}}, i64 noundef %{{.+}}) +// CIR: %[[IN0:[0-9]+]] = cir.get_member %[[IN:[0-9]+]][0] {name = ""} : !cir.ptr<!rec_anon_struct1> -> !cir.ptr<!u64i> +// CIR: cir.store %arg1, %[[IN0]] : !u64i, !cir.ptr<!u64i> +// CIR: %[[IN1:[0-9]+]] = cir.get_member %[[IN]][1] {name = ""} : !cir.ptr<!rec_anon_struct1> -> !cir.ptr<!u64i> +// CIR: cir.store %arg2, %[[IN1]] : !u64i, !cir.ptr<!u64i> +// CIR: %[[PV:[0-9]+]] = cir.load %{{[0-9]+}} : !cir.ptr<!u64i>, !u64i +// CIR: %[[OUT0:[0-9]+]] = cir.get_member %[[OUT:[0-9]+]][0] {name = ""} : !cir.ptr<!rec_anon_struct1> -> !cir.ptr<!u64i> +// CIR: %[[B0:[0-9]+]] = cir.load %[[OUT0]] : !cir.ptr<!u64i>, !u64i +// CIR: %[[OUT1:[0-9]+]] = cir.get_member %[[OUT]][1] {name = ""} : !cir.ptr<!rec_anon_struct1> -> !cir.ptr<!u64i> +// CIR: %[[B1:[0-9]+]] = cir.load %[[OUT1]] : !cir.ptr<!u64i>, !u64i +// CIR: cir.call @vf(%[[PV]], %[[B0]], %[[B1]]) : (!u64i, !u64i, !u64i) -> !s32i + +// LLVM-CIR-LABEL: define dso_local i32 @ell_bitint96( +// LLVM-CIR-SAME: i64 %[[P:[0-9a-zA-Z._]+]], i64 %[[B0:[0-9a-zA-Z._]+]], i64 %[[B1:[0-9a-zA-Z._]+]]) +// LLVM-OGCG-LABEL: define dso_local i32 @ell_bitint96( +// LLVM-OGCG-SAME: i64 %[[P:[0-9a-zA-Z._]+]], i64 noundef %[[B0:[0-9a-zA-Z._]+]], i64 noundef %[[B1:[0-9a-zA-Z._]+]]) +// LLVM: %[[S0:[0-9a-zA-Z._]+]] = getelementptr inbounds nuw { i64, i64 }, ptr %[[SLOT:[0-9a-zA-Z._]+]], i32 0, i32 0 +// LLVM: store i64 %[[B0]], ptr %[[S0]], align 8 +// LLVM: %[[S1:[0-9a-zA-Z._]+]] = getelementptr inbounds nuw { i64, i64 }, ptr %[[SLOT]], i32 0, i32 1 +// LLVM: store i64 %[[B1]], ptr %[[S1]], align 8 +// LLVM: %[[TR:[0-9a-zA-Z._]+]] = trunc i128 %{{[0-9a-zA-Z._]+}} to i96 +// LLVM: %[[EXT:[0-9a-zA-Z._]+]] = sext i96 %[[TR]] to i128 +// LLVM: store i128 %[[EXT]], ptr %[[BSLOT:[0-9a-zA-Z._]+]], align 8 +// LLVM: %[[RE:[0-9a-zA-Z._]+]] = load i128, ptr %[[BSLOT]], align 8 +// LLVM: %[[TR2:[0-9a-zA-Z._]+]] = trunc i128 %[[RE]] to i96 +// LLVM: %[[PV:[0-9a-zA-Z._]+]] = load i64, ptr %{{[0-9a-zA-Z._]+}}, align +// LLVM: %[[EXT2:[0-9a-zA-Z._]+]] = sext i96 %[[TR2]] to i128 +// LLVM: store i128 %[[EXT2]], ptr %[[COERCE:[0-9a-zA-Z._]+]], align 8 +// LLVM: %[[C0:[0-9a-zA-Z._]+]] = getelementptr inbounds nuw { i64, i64 }, ptr %[[COERCE]], i32 0, i32 0 +// LLVM: %[[A0:[0-9a-zA-Z._]+]] = load i64, ptr %[[C0]], align 8 +// LLVM: %[[C1:[0-9a-zA-Z._]+]] = getelementptr inbounds nuw { i64, i64 }, ptr %[[COERCE]], i32 0, i32 1 +// LLVM: %[[A1:[0-9a-zA-Z._]+]] = load i64, ptr %[[C1]], align 8 +// LLVM-CIR: call i32 (i64, ...) @vf(i64 %[[PV]], i64 %[[A0]], i64 %[[A1]]) +// LLVM-OGCG: call i32 (i64, ...) @vf(i64 %[[PV]], i64 noundef %[[A0]], i64 noundef %[[A1]]) // At exactly 128 bits it stays in its natural type. int ell_bitint128(Pair2 p, _BitInt(128) b) { return vf(p, b); } // CIR-LABEL: cir.func {{.*}}@ell_bitint128(%arg0: !u64i loc({{.+}}), %arg1: !s128i_bitint {llvm.noundef} loc({{.+}})) -> !s32i -// CIR: cir.call @vf(%{{.+}}, %{{.+}}) : (!u64i, !s128i_bitint {llvm.noundef}) -> !s32i - -// LLVM-LABEL: i32 @ell_bitint128(i64 %{{.+}}, i128 noundef %{{.+}}) -// LLVM: call i32 (i64, ...) @vf(i64 %{{.+}}, i128 noundef %{{.+}}) +// CIR: cir.store %arg1, %[[BSLOT:[0-9]+]] : !s128i_bitint, !cir.ptr<!s128i_bitint> +// CIR: %[[BV:[0-9]+]] = cir.load align(8) %[[BSLOT]] : !cir.ptr<!s128i_bitint>, !s128i_bitint +// CIR: %[[PV:[0-9]+]] = cir.load %{{[0-9]+}} : !cir.ptr<!u64i>, !u64i +// CIR: cir.call @vf(%[[PV]], %[[BV]]) : (!u64i, !s128i_bitint {llvm.noundef}) -> !s32i + +// LLVM-LABEL: define dso_local i32 @ell_bitint128( +// LLVM-SAME: i64 %[[P:[0-9a-zA-Z._]+]], i128 noundef %[[B:[0-9a-zA-Z._]+]]) +// LLVM: store i128 %[[B]], ptr %[[BSLOT:[0-9a-zA-Z._]+]], align 8 +// LLVM: %[[BV:[0-9a-zA-Z._]+]] = load i128, ptr %[[BSLOT]], align 8 +// LLVM: %[[PV:[0-9a-zA-Z._]+]] = load i64, ptr %{{[0-9a-zA-Z._]+}}, align +// LLVM: call i32 (i64, ...) @vf(i64 %[[PV]], i128 noundef %[[BV]]) _______________________________________________ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
