[llvm-branch-commits] [llvm] [ctx_prof] Add support for ICP (PR #105469)
@@ -572,6 +575,89 @@ CallBase &llvm::promoteCallWithIfThenElse(CallBase &CB,
Function *Callee,
return promoteCall(NewInst, Callee);
}
+CallBase *llvm::promoteCallWithIfThenElse(CallBase &CB, Function &Callee,
+ PGOContextualProfile &CtxProf) {
+ assert(CB.isIndirectCall());
+ if (!CtxProf.isFunctionKnown(Callee))
+return nullptr;
+ auto &Caller = *CB.getParent()->getParent();
snehasish wrote:
Can you use `CB->getFunction()`?
https://github.com/llvm/llvm-project/pull/105469
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [ctx_prof] Add support for ICP (PR #105469)
@@ -300,6 +305,11 @@ Value *InstrProfCallsite::getCallee() const {
return nullptr;
}
+void InstrProfCallsite::setCallee(Value *V) {
snehasish wrote:
nit: `Value* Callee` to match the decl.
https://github.com/llvm/llvm-project/pull/105469
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [ctx_prof] Add support for ICP (PR #105469)
@@ -572,6 +575,89 @@ CallBase &llvm::promoteCallWithIfThenElse(CallBase &CB,
Function *Callee,
return promoteCall(NewInst, Callee);
}
+CallBase *llvm::promoteCallWithIfThenElse(CallBase &CB, Function &Callee,
+ PGOContextualProfile &CtxProf) {
+ assert(CB.isIndirectCall());
+ if (!CtxProf.isFunctionKnown(Callee))
snehasish wrote:
Might be useful to have some statistics on how many promoted / dropped?
https://github.com/llvm/llvm-project/pull/105469
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [ctx_prof] Add support for ICP (PR #105469)
@@ -572,6 +575,89 @@ CallBase &llvm::promoteCallWithIfThenElse(CallBase &CB,
Function *Callee,
return promoteCall(NewInst, Callee);
}
+CallBase *llvm::promoteCallWithIfThenElse(CallBase &CB, Function &Callee,
+ PGOContextualProfile &CtxProf) {
+ assert(CB.isIndirectCall());
+ if (!CtxProf.isFunctionKnown(Callee))
+return nullptr;
+ auto &Caller = *CB.getParent()->getParent();
+ auto *CSInstr = CtxProfAnalysis::getCallsiteInstrumentation(CB);
+ if (!CSInstr)
+return nullptr;
+ const auto CSIndex = CSInstr->getIndex()->getZExtValue();
snehasish wrote:
Spell out the type here to make easier for the reader.
https://github.com/llvm/llvm-project/pull/105469
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [ctx_prof] Add support for ICP (PR #105469)
@@ -572,6 +575,89 @@ CallBase &llvm::promoteCallWithIfThenElse(CallBase &CB,
Function *Callee,
return promoteCall(NewInst, Callee);
}
+CallBase *llvm::promoteCallWithIfThenElse(CallBase &CB, Function &Callee,
+ PGOContextualProfile &CtxProf) {
+ assert(CB.isIndirectCall());
+ if (!CtxProf.isFunctionKnown(Callee))
+return nullptr;
+ auto &Caller = *CB.getParent()->getParent();
+ auto *CSInstr = CtxProfAnalysis::getCallsiteInstrumentation(CB);
+ if (!CSInstr)
+return nullptr;
+ const auto CSIndex = CSInstr->getIndex()->getZExtValue();
+
+ CallBase &DirectCall = promoteCall(
+ versionCallSite(CB, &Callee, /*BranchWeights=*/nullptr), &Callee);
+ CSInstr->moveBefore(&CB);
+ const auto NewCSID = CtxProf.allocateNextCallsiteIndex(Caller);
+ auto *NewCSInstr = cast(CSInstr->clone());
+ NewCSInstr->setIndex(NewCSID);
+ NewCSInstr->setCallee(&Callee);
+ NewCSInstr->insertBefore(&DirectCall);
+ auto &DirectBB = *DirectCall.getParent();
+ auto &IndirectBB = *CB.getParent();
+
+ assert((CtxProfAnalysis::getBBInstrumentation(IndirectBB) == nullptr) &&
+ "The ICP direct BB is new, it shouldn't have instrumentation");
+ assert((CtxProfAnalysis::getBBInstrumentation(DirectBB) == nullptr) &&
+ "The ICP indirect BB is new, it shouldn't have instrumentation");
+
+ // Make the 2 new BBs have counters.
+ const uint32_t DirectID = CtxProf.allocateNextCounterIndex(Caller);
+ const uint32_t IndirectID = CtxProf.allocateNextCounterIndex(Caller);
+ const uint32_t NewCountersSize = IndirectID + 1;
snehasish wrote:
Move to L620, closer to where it's used (just outside the lambda)?
https://github.com/llvm/llvm-project/pull/105469
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [ctx_prof] Add support for ICP (PR #105469)
@@ -572,6 +575,89 @@ CallBase &llvm::promoteCallWithIfThenElse(CallBase &CB,
Function *Callee,
return promoteCall(NewInst, Callee);
}
+CallBase *llvm::promoteCallWithIfThenElse(CallBase &CB, Function &Callee,
+ PGOContextualProfile &CtxProf) {
+ assert(CB.isIndirectCall());
+ if (!CtxProf.isFunctionKnown(Callee))
+return nullptr;
+ auto &Caller = *CB.getParent()->getParent();
+ auto *CSInstr = CtxProfAnalysis::getCallsiteInstrumentation(CB);
+ if (!CSInstr)
+return nullptr;
+ const auto CSIndex = CSInstr->getIndex()->getZExtValue();
+
+ CallBase &DirectCall = promoteCall(
+ versionCallSite(CB, &Callee, /*BranchWeights=*/nullptr), &Callee);
+ CSInstr->moveBefore(&CB);
+ const auto NewCSID = CtxProf.allocateNextCallsiteIndex(Caller);
+ auto *NewCSInstr = cast(CSInstr->clone());
+ NewCSInstr->setIndex(NewCSID);
+ NewCSInstr->setCallee(&Callee);
+ NewCSInstr->insertBefore(&DirectCall);
+ auto &DirectBB = *DirectCall.getParent();
+ auto &IndirectBB = *CB.getParent();
+
+ assert((CtxProfAnalysis::getBBInstrumentation(IndirectBB) == nullptr) &&
+ "The ICP direct BB is new, it shouldn't have instrumentation");
+ assert((CtxProfAnalysis::getBBInstrumentation(DirectBB) == nullptr) &&
+ "The ICP indirect BB is new, it shouldn't have instrumentation");
+
+ // Make the 2 new BBs have counters.
snehasish wrote:
nit: `Allocate counters for the new basic blocks`
https://github.com/llvm/llvm-project/pull/105469
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [ctx_prof] Add support for ICP (PR #105469)
@@ -456,3 +463,170 @@ declare void @_ZN5Base35func3Ev(ptr)
// 1 call instruction from the entry block.
EXPECT_EQ(F->front().size(), OrigEntryBBSize + 4);
}
+
+using namespace llvm::ctx_profile;
+
+class ContextManager final {
+ std::vector> Nodes;
+ std::map Roots;
+
+public:
+ ContextNode *createNode(GUID Guid, uint32_t NrCounters, uint32_t NrCallsites,
+ ContextNode *Next = nullptr) {
+auto AllocSize = ContextNode::getAllocSize(NrCounters, NrCallsites);
+auto *Mem = Nodes.emplace_back(std::make_unique(AllocSize)).get();
+std::memset(Mem, 0, AllocSize);
+auto *Ret = new (Mem) ContextNode(Guid, NrCounters, NrCallsites, Next);
+return Ret;
+ }
+};
+
+TEST(CallPromotionUtilsTest, PromoteWithIcmpAndCtxProf) {
+ LLVMContext C;
+ std::unique_ptr M = parseIR(C,
+ R"IR(
+define i32 @testfunc1(ptr %d) !guid !0 {
+ call void @llvm.instrprof.increment(ptr null, i64 0, i32 1, i32 0)
+ call void @llvm.instrprof.callsite(ptr null, i64 0, i32 1, i32 0, ptr %d)
+ %call = call i32 %d()
+ ret i32 %call
+}
+
+define i32 @f1() !guid !1 {
+ call void @llvm.instrprof.increment(ptr null, i64 0, i32 1, i32 0)
+ ret i32 2
+}
+
+define i32 @f2() !guid !2 {
+ call void @llvm.instrprof.increment(ptr null, i64 0, i32 1, i32 0)
+ call void @llvm.instrprof.callsite(ptr null, i64 0, i32 1, i32 0, ptr @f4)
+ %r = call i32 @f4()
+ ret i32 %r
+}
+
+define i32 @testfunc2(ptr %p) !guid !4 {
+ call void @llvm.instrprof.increment(ptr null, i64 0, i32 1, i32 0)
+ call void @llvm.instrprof.callsite(ptr null, i64 0, i32 1, i32 0, ptr
@testfunc1)
+ %r = call i32 @testfunc1(ptr %p)
+ ret i32 %r
+}
+
+declare i32 @f3()
+
+define i32 @f4() !guid !3 {
+ ret i32 3
+}
+
+!0 = !{i64 1000}
+!1 = !{i64 1001}
+!2 = !{i64 1002}
+!3 = !{i64 1004}
+!4 = !{i64 1005}
+)IR");
+
+ const char *Profile = R"(
+[
+{
+ "Guid": 1000,
+ "Counters": [1],
+ "Callsites": [
+[{ "Guid": 1001,
+"Counters": [10]},
+ { "Guid": 1002,
+"Counters": [11],
+"Callsites": [[{"Guid": 1004, "Counters":[13]}]]
+ },
+ { "Guid": 1003,
+"Counters": [12]
+ }]]
+},
+{
+ "Guid": 1005,
+ "Counters": [2],
+ "Callsites": [
+[{ "Guid": 1000,
+"Counters": [1],
+"Callsites": [
+ [{ "Guid": 1001,
+ "Counters": [101]},
+{ "Guid": 1002,
+ "Counters": [102],
+ "Callsites": [[{"Guid": 1004, "Counters":[104]}]]
+},
+{ "Guid": 1003,
+ "Counters": [103]
+}]]}]]}]
+)";
+
+ llvm::unittest::TempFile ProfileFile("ctx_profile", "", "", /*Unique*/ true);
+ {
+std::error_code EC;
+raw_fd_stream Out(ProfileFile.path(), EC);
+ASSERT_FALSE(EC);
+// "False" means no error.
+ASSERT_FALSE(llvm::createCtxProfFromJSON(Profile, Out));
+ }
+
+ ModuleAnalysisManager MAM;
+ MAM.registerPass([&]() { return CtxProfAnalysis(ProfileFile.path()); });
+ MAM.registerPass([&]() { return PassInstrumentationAnalysis(); });
+ auto &CtxProf = MAM.getResult(*M);
+ auto *Caller = M->getFunction("testfunc1");
+ ASSERT_TRUE(!!Caller);
snehasish wrote:
`getFunction` returns Function* or nullptr. Why do you need `!!` here? Can this
be `ASSERT_NE(Caller, nullptr);`? Same for similar usage below.
https://github.com/llvm/llvm-project/pull/105469
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [ctx_prof] Add support for ICP (PR #105469)
@@ -572,6 +575,89 @@ CallBase &llvm::promoteCallWithIfThenElse(CallBase &CB,
Function *Callee,
return promoteCall(NewInst, Callee);
}
+CallBase *llvm::promoteCallWithIfThenElse(CallBase &CB, Function &Callee,
+ PGOContextualProfile &CtxProf) {
+ assert(CB.isIndirectCall());
+ if (!CtxProf.isFunctionKnown(Callee))
+return nullptr;
+ auto &Caller = *CB.getParent()->getParent();
+ auto *CSInstr = CtxProfAnalysis::getCallsiteInstrumentation(CB);
+ if (!CSInstr)
+return nullptr;
+ const auto CSIndex = CSInstr->getIndex()->getZExtValue();
+
+ CallBase &DirectCall = promoteCall(
+ versionCallSite(CB, &Callee, /*BranchWeights=*/nullptr), &Callee);
+ CSInstr->moveBefore(&CB);
+ const auto NewCSID = CtxProf.allocateNextCallsiteIndex(Caller);
+ auto *NewCSInstr = cast(CSInstr->clone());
+ NewCSInstr->setIndex(NewCSID);
+ NewCSInstr->setCallee(&Callee);
+ NewCSInstr->insertBefore(&DirectCall);
+ auto &DirectBB = *DirectCall.getParent();
+ auto &IndirectBB = *CB.getParent();
+
+ assert((CtxProfAnalysis::getBBInstrumentation(IndirectBB) == nullptr) &&
+ "The ICP direct BB is new, it shouldn't have instrumentation");
+ assert((CtxProfAnalysis::getBBInstrumentation(DirectBB) == nullptr) &&
+ "The ICP indirect BB is new, it shouldn't have instrumentation");
+
+ // Make the 2 new BBs have counters.
+ const uint32_t DirectID = CtxProf.allocateNextCounterIndex(Caller);
+ const uint32_t IndirectID = CtxProf.allocateNextCounterIndex(Caller);
+ const uint32_t NewCountersSize = IndirectID + 1;
+ auto *EntryBBIns =
+ CtxProfAnalysis::getBBInstrumentation(Caller.getEntryBlock());
+ auto *DirectBBIns = cast(EntryBBIns->clone());
+ DirectBBIns->setIndex(DirectID);
+ DirectBBIns->insertInto(&DirectBB, DirectBB.getFirstInsertionPt());
+
+ auto *IndirectBBIns = cast(EntryBBIns->clone());
+ IndirectBBIns->setIndex(IndirectID);
+ IndirectBBIns->insertInto(&IndirectBB, IndirectBB.getFirstInsertionPt());
+
+ const GlobalValue::GUID CalleeGUID = AssignGUIDPass::getGUID(Callee);
+
+ auto ProfileUpdater = [&](PGOCtxProfContext &Ctx) {
+assert(Ctx.guid() == AssignGUIDPass::getGUID(Caller));
+assert(NewCountersSize - 2 == Ctx.counters().size());
+// Regardless what next, all the ctx-es belonging to a function must have
snehasish wrote:
I'm not sure what's implied by "Regardless what next", either document specific
concerns or drop to make it clearer?
https://github.com/llvm/llvm-project/pull/105469
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [ctx_prof] Add support for ICP (PR #105469)
@@ -456,3 +463,170 @@ declare void @_ZN5Base35func3Ev(ptr)
// 1 call instruction from the entry block.
EXPECT_EQ(F->front().size(), OrigEntryBBSize + 4);
}
+
+using namespace llvm::ctx_profile;
+
+class ContextManager final {
+ std::vector> Nodes;
+ std::map Roots;
+
+public:
+ ContextNode *createNode(GUID Guid, uint32_t NrCounters, uint32_t NrCallsites,
+ ContextNode *Next = nullptr) {
+auto AllocSize = ContextNode::getAllocSize(NrCounters, NrCallsites);
+auto *Mem = Nodes.emplace_back(std::make_unique(AllocSize)).get();
+std::memset(Mem, 0, AllocSize);
snehasish wrote:
I think you don't need the memset here.
The make_unique above will zero initialize the memory. See the documentation at
https://en.cppreference.com/w/cpp/memory/unique_ptr/make_unique
"2) Constructs an array of the given dynamic size. The array elements are
value-initialized. This overload participates in overload resolution only if T
is an array of unknown bound."
Also most (all?) implementations of placement new will zero out the memory
before constructing the object as far as I remember.
https://github.com/llvm/llvm-project/pull/105469
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [ctx_prof] Add support for ICP (PR #105469)
https://github.com/snehasish edited https://github.com/llvm/llvm-project/pull/105469 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [ctx_prof] Add support for ICP (PR #105469)
@@ -456,3 +463,170 @@ declare void @_ZN5Base35func3Ev(ptr)
// 1 call instruction from the entry block.
EXPECT_EQ(F->front().size(), OrigEntryBBSize + 4);
}
+
+using namespace llvm::ctx_profile;
+
+class ContextManager final {
+ std::vector> Nodes;
+ std::map Roots;
+
+public:
+ ContextNode *createNode(GUID Guid, uint32_t NrCounters, uint32_t NrCallsites,
+ ContextNode *Next = nullptr) {
+auto AllocSize = ContextNode::getAllocSize(NrCounters, NrCallsites);
+auto *Mem = Nodes.emplace_back(std::make_unique(AllocSize)).get();
+std::memset(Mem, 0, AllocSize);
+auto *Ret = new (Mem) ContextNode(Guid, NrCounters, NrCallsites, Next);
+return Ret;
+ }
+};
+
+TEST(CallPromotionUtilsTest, PromoteWithIcmpAndCtxProf) {
+ LLVMContext C;
+ std::unique_ptr M = parseIR(C,
+ R"IR(
+define i32 @testfunc1(ptr %d) !guid !0 {
+ call void @llvm.instrprof.increment(ptr @testfunc1, i64 0, i32 1, i32 0)
+ call void @llvm.instrprof.callsite(ptr @testfunc1, i64 0, i32 1, i32 0, ptr
%d)
+ %call = call i32 %d()
+ ret i32 %call
+}
+
+define i32 @f1() !guid !1 {
+ call void @llvm.instrprof.increment(ptr @f1, i64 0, i32 1, i32 0)
+ ret i32 2
+}
+
+define i32 @f2() !guid !2 {
+ call void @llvm.instrprof.increment(ptr @f2, i64 0, i32 1, i32 0)
+ call void @llvm.instrprof.callsite(ptr @f2, i64 0, i32 1, i32 0, ptr @f4)
+ %r = call i32 @f4()
+ ret i32 %r
+}
+
+define i32 @testfunc2(ptr %p) !guid !4 {
+ call void @llvm.instrprof.increment(ptr @testfunc2, i64 0, i32 1, i32 0)
+ call void @llvm.instrprof.callsite(ptr @testfunc2, i64 0, i32 1, i32 0, ptr
@testfunc1)
+ %r = call i32 @testfunc1(ptr %p)
+ ret i32 %r
+}
+
+declare i32 @f3()
+
+define i32 @f4() !guid !3 {
+ ret i32 3
+}
+
+!0 = !{i64 1000}
+!1 = !{i64 1001}
+!2 = !{i64 1002}
+!3 = !{i64 1004}
+!4 = !{i64 1005}
+)IR");
+
+ const char *Profile = R"(
snehasish wrote:
Can you use `R"json(...)json"` to get these formatted? Some lines look
inconsistent with closing brackets grouped together. Or are you intentionally
saving white-space here?
https://github.com/llvm/llvm-project/pull/105469
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [ctx_prof] Add support for ICP (PR #105469)
https://github.com/snehasish approved this pull request. lgtm, rest are minor nits. https://github.com/llvm/llvm-project/pull/105469 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [ctx_prof] Add support for ICP (PR #105469)
@@ -456,3 +463,170 @@ declare void @_ZN5Base35func3Ev(ptr)
// 1 call instruction from the entry block.
EXPECT_EQ(F->front().size(), OrigEntryBBSize + 4);
}
+
+using namespace llvm::ctx_profile;
+
+class ContextManager final {
+ std::vector> Nodes;
+ std::map Roots;
+
+public:
+ ContextNode *createNode(GUID Guid, uint32_t NrCounters, uint32_t NrCallsites,
+ ContextNode *Next = nullptr) {
+auto AllocSize = ContextNode::getAllocSize(NrCounters, NrCallsites);
+auto *Mem = Nodes.emplace_back(std::make_unique(AllocSize)).get();
+std::memset(Mem, 0, AllocSize);
+auto *Ret = new (Mem) ContextNode(Guid, NrCounters, NrCallsites, Next);
+return Ret;
+ }
+};
+
+TEST(CallPromotionUtilsTest, PromoteWithIcmpAndCtxProf) {
+ LLVMContext C;
+ std::unique_ptr M = parseIR(C,
+ R"IR(
+define i32 @testfunc1(ptr %d) !guid !0 {
+ call void @llvm.instrprof.increment(ptr @testfunc1, i64 0, i32 1, i32 0)
+ call void @llvm.instrprof.callsite(ptr @testfunc1, i64 0, i32 1, i32 0, ptr
%d)
+ %call = call i32 %d()
+ ret i32 %call
+}
+
+define i32 @f1() !guid !1 {
+ call void @llvm.instrprof.increment(ptr @f1, i64 0, i32 1, i32 0)
+ ret i32 2
+}
+
+define i32 @f2() !guid !2 {
+ call void @llvm.instrprof.increment(ptr @f2, i64 0, i32 1, i32 0)
+ call void @llvm.instrprof.callsite(ptr @f2, i64 0, i32 1, i32 0, ptr @f4)
+ %r = call i32 @f4()
+ ret i32 %r
+}
+
+define i32 @testfunc2(ptr %p) !guid !4 {
+ call void @llvm.instrprof.increment(ptr @testfunc2, i64 0, i32 1, i32 0)
+ call void @llvm.instrprof.callsite(ptr @testfunc2, i64 0, i32 1, i32 0, ptr
@testfunc1)
+ %r = call i32 @testfunc1(ptr %p)
+ ret i32 %r
+}
+
+declare i32 @f3()
+
+define i32 @f4() !guid !3 {
+ ret i32 3
+}
+
+!0 = !{i64 1000}
+!1 = !{i64 1001}
+!2 = !{i64 1002}
+!3 = !{i64 1004}
+!4 = !{i64 1005}
+)IR");
+
+ const char *Profile = R"(
+[
+{
+ "Guid": 1000,
+ "Counters": [1],
+ "Callsites": [
+[{ "Guid": 1001,
+"Counters": [10]},
+ { "Guid": 1002,
+"Counters": [11],
+"Callsites": [[{"Guid": 1004, "Counters":[13]}]]
+ },
+ { "Guid": 1003,
+"Counters": [12]
+ }]]
+},
+{
+ "Guid": 1005,
+ "Counters": [2],
+ "Callsites": [
+[{ "Guid": 1000,
+"Counters": [1],
+"Callsites": [
+ [{ "Guid": 1001,
+ "Counters": [101]},
+{ "Guid": 1002,
+ "Counters": [102],
+ "Callsites": [[{"Guid": 1004, "Counters":[104]}]]
+},
+{ "Guid": 1003,
+ "Counters": [103]
+}]]}]]}]
+)";
+
+ llvm::unittest::TempFile ProfileFile("ctx_profile", "", "", /*Unique*/ true);
snehasish wrote:
`/*Unique=*/` for clang-tidy to pick it up.
https://github.com/llvm/llvm-project/pull/105469
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [Metadata] Preserve MD_prof when merging instructions when one is missing. (PR #132433)
https://github.com/snehasish created
https://github.com/llvm/llvm-project/pull/132433
Preserve branch weight metadata when merging instructions if one of the
instructions is missing metadata. This is similar in behaviour to what
we do today for other types of metadata such as mmra, memprof and
callsite metadata.
>From 8677f6470103b6af093f57f79c18c4f81912c5fa Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 21 Mar 2025 17:00:38 +
Subject: [PATCH] [Metadata] Preserve MD_prof when merging instructions when
one is missing.
Preserve branch weight metadata when merging instructions if one of the
instructions is missing metadata. This is similar in behaviour to what
we do today for other types of metadata such as mmra, memprof and
callsite metadata.
---
llvm/lib/Transforms/Utils/Local.cpp | 18 --
...rect-call-branch-weights-preserve-hoist.ll | 62 +++
...irect-call-branch-weights-preserve-sink.ll | 62 +++
3 files changed, 137 insertions(+), 5 deletions(-)
create mode 100644
llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
create mode 100644
llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-sink.ll
diff --git a/llvm/lib/Transforms/Utils/Local.cpp
b/llvm/lib/Transforms/Utils/Local.cpp
index d18adac5fa914..069ebae117b48 100644
--- a/llvm/lib/Transforms/Utils/Local.cpp
+++ b/llvm/lib/Transforms/Utils/Local.cpp
@@ -3355,9 +3355,11 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
case LLVMContext::MD_invariant_group:
// Preserve !invariant.group in K.
break;
- // Keep empty cases for mmra, memprof, and callsite to prevent them from
+ // Keep empty cases for prof, mmra, memprof, and callsite to prevent
them from
// being removed as unknown metadata. The actual merging is handled
// separately below.
+ case LLVMContext::MD_prof:
+[[fallthrough]];
case LLVMContext::MD_mmra:
case LLVMContext::MD_memprof:
case LLVMContext::MD_callsite:
@@ -3386,10 +3388,6 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
if (!AAOnly)
K->setMetadata(Kind, JMD);
break;
- case LLVMContext::MD_prof:
-if (!AAOnly && DoesKMove)
- K->setMetadata(Kind, MDNode::getMergedProfMetadata(KMD, JMD, K, J));
-break;
case LLVMContext::MD_noalias_addrspace:
if (DoesKMove)
K->setMetadata(Kind,
@@ -3436,6 +3434,16 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
K->setMetadata(LLVMContext::MD_callsite,
MDNode::getMergedCallsiteMetadata(KCallSite, JCallSite));
}
+
+ // Merge prof metadata.
+ // Handle separately to support cases where only one instruction has the
+ // metadata.
+ auto JProf = J->getMetadata(LLVMContext::MD_prof);
+ auto KProf = K->getMetadata(LLVMContext::MD_prof);
+ if (!AAOnly && (JProf || KProf)) {
+K->setMetadata(LLVMContext::MD_prof,
+ MDNode::getMergedProfMetadata(KProf, JProf, K, J));
+ }
}
void llvm::combineMetadataForCSE(Instruction *K, const Instruction *J,
diff --git
a/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
b/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
new file mode 100644
index 0..2a0fc3ffc2d4d
--- /dev/null
+++
b/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
@@ -0,0 +1,62 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
UTC_ARGS: --check-globals --version 2
+; RUN: opt < %s -passes='simplifycfg'
-simplifycfg-require-and-preserve-domtree=1 -S | FileCheck %s
--check-prefix=HOIST
+
+; Test case based on C++ code with manualy annotated !prof metadata.
+; This is to test that when calls to 'func1' from 'if.then' block
+; and 'if.else' block are hoisted, the branch_weights are merged and
+; attached to merged call rather than dropped.
+;
+; int func1(int a, int b) ;
+; int func2(int a, int b) ;
+
+; int func(int a, int b, bool c) {
+;int sum= 0;
+;if(c) {
+;sum += func1(a, b);
+;} else {
+;sum += func1(a, b);
+;sum -= func2(a, b);
+;}
+;return sum;
+; }
+define i32 @_Z4funciib(i32 %a, i32 %b, i1 %c) {
+; HOIST-LABEL: define i32 @_Z4funciib
+; HOIST-SAME: (i32 [[A:%.*]], i32 [[B:%.*]], i1 [[C:%.*]]) {
+; HOIST-NEXT: entry:
+; HOIST-NEXT:[[CALL:%.*]] = tail call i32 @_Z5func1ii(i32 [[A]], i32
[[B]]), !prof [[PROF0:![0-9]+]]
+; HOIST-NEXT:br i1 [[C]], label [[IF_END:%.*]], label [[IF_ELSE:%.*]]
+; HOIST: if.else:
+; HOIST-NEXT:[[CALL3:%.*]] = tail call i32 @_Z5func2ii(i32 [[A]], i32
[[B]])
+; HOIST-NEXT:[[SUB:%.*]] = sub i32 [[CALL]], [[CALL3]]
+; HOIST-NEXT:br label [[IF_END]]
+; HOIST: if.end:
+; HOIST-NEXT:[[SUM_0:%.*]] = phi i32 [ [[SUB]], [[IF_ELSE]] ], [ [[CALL]],
[
[llvm-branch-commits] [llvm] [Metadata] Preserve MD_prof when merging instructions when one is missing. (PR #132433)
snehasish wrote: > [!WARNING] > This pull request is not mergeable via GitHub because a downstack PR is > open. Once all requirements are satisfied, merge this PR as a stack href="https://app.graphite.dev/github/pr/llvm/llvm-project/132433?utm_source=stack-comment-downstack-mergeability-warning"; > >on Graphite. > https://graphite.dev/docs/merge-pull-requests";>Learn more * **#132433** https://app.graphite.dev/github/pr/llvm/llvm-project/132433?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> 👈 https://app.graphite.dev/github/pr/llvm/llvm-project/132433?utm_source=stack-comment-view-in-graphite"; target="_blank">(View in Graphite) * **#132106** https://app.graphite.dev/github/pr/llvm/llvm-project/132106?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * `main` This stack of pull requests is managed by https://graphite.dev?utm-source=stack-comment";>Graphite. Learn more about https://stacking.dev/?utm_source=stack-comment";>stacking. https://github.com/llvm/llvm-project/pull/132433 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [Metadata] Preserve MD_prof when merging instructions when one is missing. (PR #132433)
https://github.com/snehasish updated
https://github.com/llvm/llvm-project/pull/132433
>From 5b2f87db7ec72d12e4d8837203c3c7b953649907 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 21 Mar 2025 17:00:38 +
Subject: [PATCH] [Metadata] Preserve MD_prof when merging instructions when
one is missing.
Preserve branch weight metadata when merging instructions if one of the
instructions is missing metadata. This is similar in behaviour to what
we do today for other types of metadata such as mmra, memprof and
callsite metadata.
---
llvm/lib/Transforms/Utils/Local.cpp | 19 --
...rect-call-branch-weights-preserve-hoist.ll | 62 +++
...irect-call-branch-weights-preserve-sink.ll | 62 +++
3 files changed, 137 insertions(+), 6 deletions(-)
create mode 100644
llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
create mode 100644
llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-sink.ll
diff --git a/llvm/lib/Transforms/Utils/Local.cpp
b/llvm/lib/Transforms/Utils/Local.cpp
index d18adac5fa914..a9408df328030 100644
--- a/llvm/lib/Transforms/Utils/Local.cpp
+++ b/llvm/lib/Transforms/Utils/Local.cpp
@@ -3355,9 +3355,10 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
case LLVMContext::MD_invariant_group:
// Preserve !invariant.group in K.
break;
- // Keep empty cases for mmra, memprof, and callsite to prevent them from
- // being removed as unknown metadata. The actual merging is handled
+ // Keep empty cases for prof, mmra, memprof, and callsite to prevent them
+ // from being removed as unknown metadata. The actual merging is handled
// separately below.
+ case LLVMContext::MD_prof:
case LLVMContext::MD_mmra:
case LLVMContext::MD_memprof:
case LLVMContext::MD_callsite:
@@ -3386,10 +3387,6 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
if (!AAOnly)
K->setMetadata(Kind, JMD);
break;
- case LLVMContext::MD_prof:
-if (!AAOnly && DoesKMove)
- K->setMetadata(Kind, MDNode::getMergedProfMetadata(KMD, JMD, K, J));
-break;
case LLVMContext::MD_noalias_addrspace:
if (DoesKMove)
K->setMetadata(Kind,
@@ -3436,6 +3433,16 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
K->setMetadata(LLVMContext::MD_callsite,
MDNode::getMergedCallsiteMetadata(KCallSite, JCallSite));
}
+
+ // Merge prof metadata.
+ // Handle separately to support cases where only one instruction has the
+ // metadata.
+ auto JProf = J->getMetadata(LLVMContext::MD_prof);
+ auto KProf = K->getMetadata(LLVMContext::MD_prof);
+ if (!AAOnly && (JProf || KProf)) {
+K->setMetadata(LLVMContext::MD_prof,
+ MDNode::getMergedProfMetadata(KProf, JProf, K, J));
+ }
}
void llvm::combineMetadataForCSE(Instruction *K, const Instruction *J,
diff --git
a/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
b/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
new file mode 100644
index 0..2a0fc3ffc2d4d
--- /dev/null
+++
b/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
@@ -0,0 +1,62 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
UTC_ARGS: --check-globals --version 2
+; RUN: opt < %s -passes='simplifycfg'
-simplifycfg-require-and-preserve-domtree=1 -S | FileCheck %s
--check-prefix=HOIST
+
+; Test case based on C++ code with manualy annotated !prof metadata.
+; This is to test that when calls to 'func1' from 'if.then' block
+; and 'if.else' block are hoisted, the branch_weights are merged and
+; attached to merged call rather than dropped.
+;
+; int func1(int a, int b) ;
+; int func2(int a, int b) ;
+
+; int func(int a, int b, bool c) {
+;int sum= 0;
+;if(c) {
+;sum += func1(a, b);
+;} else {
+;sum += func1(a, b);
+;sum -= func2(a, b);
+;}
+;return sum;
+; }
+define i32 @_Z4funciib(i32 %a, i32 %b, i1 %c) {
+; HOIST-LABEL: define i32 @_Z4funciib
+; HOIST-SAME: (i32 [[A:%.*]], i32 [[B:%.*]], i1 [[C:%.*]]) {
+; HOIST-NEXT: entry:
+; HOIST-NEXT:[[CALL:%.*]] = tail call i32 @_Z5func1ii(i32 [[A]], i32
[[B]]), !prof [[PROF0:![0-9]+]]
+; HOIST-NEXT:br i1 [[C]], label [[IF_END:%.*]], label [[IF_ELSE:%.*]]
+; HOIST: if.else:
+; HOIST-NEXT:[[CALL3:%.*]] = tail call i32 @_Z5func2ii(i32 [[A]], i32
[[B]])
+; HOIST-NEXT:[[SUB:%.*]] = sub i32 [[CALL]], [[CALL3]]
+; HOIST-NEXT:br label [[IF_END]]
+; HOIST: if.end:
+; HOIST-NEXT:[[SUM_0:%.*]] = phi i32 [ [[SUB]], [[IF_ELSE]] ], [ [[CALL]],
[[ENTRY:%.*]] ]
+; HOIST-NEXT:ret i32 [[SUM_0]]
+;
+entry:
+ br i1 %c, label %if.then, label %if.else
+
+if.then: ; preds = %entry
+ %c
[llvm-branch-commits] [compiler-rt] [llvm] [ctxprof] Track unhandled call targets (PR #131417)
@@ -83,6 +84,20 @@ struct ContextRoot {
// Count the number of entries - regardless if we could take the `Taken`
mutex
::__sanitizer::atomic_uint64_t TotalEntries = {};
+ // Profiles for functions we encounter when collecting a contexutal profile,
+ // that are not associated with a callsite. This is expected to happen for
+ // signal handlers, but it also - problematically - currently happens for
+ // mem{memset|copy|move|set}, which are currently inserted after profile
snehasish wrote:
Maybe just generalize this statement to "it also happens for functions
generated after profile instrumention e.g. mem{set|copy|move}".
https://github.com/llvm/llvm-project/pull/131417
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [compiler-rt] [llvm] [ctxprof] Track unhandled call targets (PR #131417)
@@ -246,22 +246,37 @@ ContextNode *getFlatProfile(FunctionData &Data, GUID Guid,
ContextNode *getUnhandledContext(FunctionData &Data, GUID Guid,
uint32_t NumCounters) {
- // 1) if we are under a root (regardless if this thread is collecting or not
a
+
+ // 1) if we are currently collecting a contextual profile, fetch a
ContextNode
+ // in the `Unhandled` set. We want to do this regardless of
`ProfilingStarted`
+ // to (hopefully) offset the penalty of creating these contexts to before
+ // profiling.
+ //
+ // 2) if we are under a root (regardless if this thread is collecting or not
a
// contextual profile for that root), do not collect a flat profile. We want
// to keep flat profiles only for activations that can't happen under a root,
// to avoid confusing profiles. We can, for example, combine flattened and
// flat profiles meaningfully, as we wouldn't double-count anything.
//
- // 2) to avoid lengthy startup, don't bother with flat profiles until the
+ // 3) to avoid lengthy startup, don't bother with flat profiles until the
snehasish wrote:
"the profiling has started"?
https://github.com/llvm/llvm-project/pull/131417
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [compiler-rt] [llvm] [ctxprof] Track unhandled call targets (PR #131417)
@@ -265,7 +275,16 @@ Error llvm::createCtxProfFromYAML(StringRef Profile,
raw_ostream &Out) {
if (!TopList)
return createStringError(
"Unexpected error converting internal structure to ctx profile");
- Writer.writeContextual(*TopList, DC.TotalRootEntryCount);
+
+ ctx_profile::ContextNode *FirstUnhandled = nullptr;
+ for (const auto &U : DC.Unhandled) {
+SerializableCtxRepresentation Unhandled;
+Unhandled.Guid = U.first;
+Unhandled.Counters.insert(Unhandled.Counters.begin(), U.second.begin(),
snehasish wrote:
Repeatedly inserting in the front is inefficient. How about reserve(size),
append and then reverse the vector?
https://github.com/llvm/llvm-project/pull/131417
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [compiler-rt] [llvm] [ctxprof] Track unhandled call targets (PR #131417)
@@ -246,22 +246,37 @@ ContextNode *getFlatProfile(FunctionData &Data, GUID Guid,
ContextNode *getUnhandledContext(FunctionData &Data, GUID Guid,
uint32_t NumCounters) {
- // 1) if we are under a root (regardless if this thread is collecting or not
a
+
+ // 1) if we are currently collecting a contextual profile, fetch a
ContextNode
+ // in the `Unhandled` set. We want to do this regardless of
`ProfilingStarted`
+ // to (hopefully) offset the penalty of creating these contexts to before
+ // profiling.
+ //
+ // 2) if we are under a root (regardless if this thread is collecting or not
a
// contextual profile for that root), do not collect a flat profile. We want
// to keep flat profiles only for activations that can't happen under a root,
// to avoid confusing profiles. We can, for example, combine flattened and
// flat profiles meaningfully, as we wouldn't double-count anything.
//
- // 2) to avoid lengthy startup, don't bother with flat profiles until the
+ // 3) to avoid lengthy startup, don't bother with flat profiles until the
// profiling started. We would reset them anyway when profiling starts.
// HOWEVER. This does lose profiling for message pumps: those functions are
// entered once and never exit. They should be assumed to be entered before
// profiling starts - because profiling should start after the server is up
// and running (which is equivalent to "message pumps are set up").
- if (IsUnderContext || !__sanitizer::atomic_load_relaxed(&ProfilingStarted))
-return TheScratchContext;
- return markAsScratch(
- onContextEnter(*getFlatProfile(Data, Guid, NumCounters)));
+ ContextRoot *R = __llvm_ctx_profile_current_context_root;
+ if (!R) {
+if (IsUnderContext || !__sanitizer::atomic_load_relaxed(&ProfilingStarted))
+ return TheScratchContext;
+else
+ return markAsScratch(
+ onContextEnter(*getFlatProfile(Data, Guid, NumCounters)));
+ }
+ auto It = R->Unhandled.insert({Guid, nullptr});
snehasish wrote:
`auto& [Iter, Inserted] = R->Unhandled.insert(...)`
So that we can make the first second usage below a bit more readable.
https://github.com/llvm/llvm-project/pull/131417
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [CodeGen][StaticDataSplitter]Support constant pool partitioning (PR #129781)
https://github.com/snehasish edited https://github.com/llvm/llvm-project/pull/129781 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [CodeGen][StaticDataSplitter]Support constant pool partitioning (PR #129781)
@@ -0,0 +1,141 @@ +; RUN: llc -mtriple=aarch64 -enable-split-machine-functions \ snehasish wrote: Can we drop the `-enable-split-machine-functions` flag here and below? https://github.com/llvm/llvm-project/pull/129781 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [CodeGen][StaticDataSplitter]Support constant pool partitioning (PR #129781)
@@ -0,0 +1,141 @@
+; RUN: llc -mtriple=aarch64 -enable-split-machine-functions \
+; RUN: -partition-static-data-sections=true -function-sections=true \
+; RUN: -unique-section-names=false \
+; RUN: %s -o - 2>&1 | FileCheck %s --dump-input=always
+
+; Repeat the RUN command above for big-endian systems.
+; RUN: llc -mtriple=aarch64_be -enable-split-machine-functions \
+; RUN: -partition-static-data-sections=true -function-sections=true \
+; RUN: -unique-section-names=false \
+; RUN: %s -o - 2>&1 | FileCheck %s --dump-input=always
+
+; Tests that constant pool hotness is aggregated across the module. The
+; static-data-splitter processes data from cold_func first, unprofiled_func
+; secondly, and then hot_func. Specifically, tests that
+; - If a constant is accessed by hot functions, all constant pools for this
+; constant (e.g., from an unprofiled function, or cold function) should have
+; `.hot` suffix.
+; - Similarly if a constant is accessed by both cold function and un-profiled
+; function, constant pools for this constant should not have `.unlikely`
suffix.
+
+; CHECK: .section .rodata.cst8.hot,"aM",@progbits,8
+; CHECK: .LCPI0_0:
+; CHECK: .xword 0x3fe5c28f5c28f5c3 // double
0.68005
+; CHECK: .section .rodata.cst8.unlikely,"aM",@progbits,8
+; CHECK: .LCPI0_1:
+; CHECK: .xword 0x3fe5eb851eb851ec // double
0.68505
+; CHECK: .section .rodata.cst8,"aM",@progbits,8
+; CHECK: .LCPI0_2:
+; CHECK: .byte 0 // 0x0
+; CHECK: .byte 4 // 0x4
+; CHECK: .byte 8 // 0x8
+; CHECK: .byte 12 // 0xc
+; CHECK: .byte 255 // 0xff
+; CHECK: .byte 255 // 0xff
+; CHECK: .byte 255 // 0xff
+; CHECK: .byte 255 // 0xff
+
+; CHECK: .section .rodata.cst8,"aM",@progbits,8
+; CHECK: .LCPI1_0:
+; CHECK: .byte 0 // 0x0
+; CHECK: .byte 4 // 0x4
+; CHECK: .byte 8 // 0x8
+; CHECK: .byte 12 // 0xc
+; CHECK: .byte 255 // 0xff
+; CHECK: .byte 255 // 0xff
+; CHECK: .byte 255 // 0xff
+; CHECK: .byte 255 // 0xff
+; CHECK: .section.rodata.cst16.hot,"aM",@progbits,16
+; CHECK: .LCPI1_1:
+; CHECK: .word 442 // 0x1ba
+; CHECK: .word 100 // 0x64
+; CHECK: .word 0 // 0x0
+; CHECK: .word 0 // 0x0
+
+; CHECK: .section.rodata.cst8.hot,"aM",@progbits,8
+; CHECK: .LCPI2_0:
+; CHECK: .xword 0x3fe5c28f5c28f5c3 // double
0.68005
+; CHECK: .section.rodata.cst16.hot,"aM",@progbits,16
+; CHECK: .LCPI2_1:
+; CHECK: .word 442 // 0x1ba
+; CHECK: .word 100 // 0x64
+; CHECK: .word 0 // 0x0
+; CHECK: .word 0 // 0x0
+
+; CHECK:.section .rodata.cst32,"aM",@progbits,32
+; CHECK:.globl val
+
+define i32 @cold_func(double %x, <16 x i8> %a, <16 x i8> %b) !prof !16 {
+ %2 = tail call i32 (...) @func_taking_arbitrary_param(double 6.80e-01)
+ %num = tail call i32 (...) @func_taking_arbitrary_param(double 6.850e-01)
+ %t1 = call <8 x i8> @llvm.aarch64.neon.tbl2.v8i8(<16 x i8> %a, <16 x i8> %b,
<8 x i8> )
+ %t2 = bitcast <8 x i8> %t1 to <2 x i32>
+ %3 = extractelement <2 x i32> %t2, i32 1
+ %sum = add i32 %2, %3
+ %ret = add i32 %sum, %num
+ ret i32 %ret
+}
+
+declare <8 x i8> @llvm.aarch64.neon.tbl2.v8i8(<16 x i8>, <16 x i8>, <8 x i8>)
+declare i32 @func_taking_arbitrary_param(...)
+
+define <4 x i1> @unprofiled_func(<16 x i8> %a, <16 x i8> %b) {
+ %t1 = call <8 x i8> @llvm.aarch64.neon.tbl2.v8i8(<16 x i8> %a, <16 x i8> %b,
<8 x i8> )
+ %t2 = bitcast <8 x i8> %t1 to <4 x i16>
+ %t3 = zext <4 x i16> %t2 to <4 x i32>
+ %cmp = icmp ule <4 x i32> , %t3
snehasish wrote:
Can we use different values for the constant in the unprofiled func and the hot
func? Using <442, 100, 0, 0> for both seems like it could lead to false
positives.
https://github.com/llvm/llvm-project/pull/129781
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [CodeGen][StaticDataSplitter]Support constant pool partitioning (PR #129781)
@@ -0,0 +1,141 @@
+; RUN: llc -mtriple=aarch64 -enable-split-machine-functions \
+; RUN: -partition-static-data-sections=true -function-sections=true \
+; RUN: -unique-section-names=false \
+; RUN: %s -o - 2>&1 | FileCheck %s --dump-input=always
+
+; Repeat the RUN command above for big-endian systems.
+; RUN: llc -mtriple=aarch64_be -enable-split-machine-functions \
+; RUN: -partition-static-data-sections=true -function-sections=true \
+; RUN: -unique-section-names=false \
+; RUN: %s -o - 2>&1 | FileCheck %s --dump-input=always
+
+; Tests that constant pool hotness is aggregated across the module. The
+; static-data-splitter processes data from cold_func first, unprofiled_func
+; secondly, and then hot_func. Specifically, tests that
+; - If a constant is accessed by hot functions, all constant pools for this
+; constant (e.g., from an unprofiled function, or cold function) should have
+; `.hot` suffix.
+; - Similarly if a constant is accessed by both cold function and un-profiled
+; function, constant pools for this constant should not have `.unlikely`
suffix.
+
+; CHECK: .section .rodata.cst8.hot,"aM",@progbits,8
+; CHECK: .LCPI0_0:
+; CHECK: .xword 0x3fe5c28f5c28f5c3 // double
0.68005
+; CHECK: .section .rodata.cst8.unlikely,"aM",@progbits,8
+; CHECK: .LCPI0_1:
+; CHECK: .xword 0x3fe5eb851eb851ec // double
0.68505
+; CHECK: .section .rodata.cst8,"aM",@progbits,8
+; CHECK: .LCPI0_2:
+; CHECK: .byte 0 // 0x0
+; CHECK: .byte 4 // 0x4
+; CHECK: .byte 8 // 0x8
+; CHECK: .byte 12 // 0xc
+; CHECK: .byte 255 // 0xff
+; CHECK: .byte 255 // 0xff
+; CHECK: .byte 255 // 0xff
+; CHECK: .byte 255 // 0xff
+
+; CHECK: .section .rodata.cst8,"aM",@progbits,8
+; CHECK: .LCPI1_0:
+; CHECK: .byte 0 // 0x0
+; CHECK: .byte 4 // 0x4
+; CHECK: .byte 8 // 0x8
+; CHECK: .byte 12 // 0xc
+; CHECK: .byte 255 // 0xff
+; CHECK: .byte 255 // 0xff
+; CHECK: .byte 255 // 0xff
+; CHECK: .byte 255 // 0xff
+; CHECK: .section.rodata.cst16.hot,"aM",@progbits,16
+; CHECK: .LCPI1_1:
+; CHECK: .word 442 // 0x1ba
+; CHECK: .word 100 // 0x64
+; CHECK: .word 0 // 0x0
+; CHECK: .word 0 // 0x0
+
+; CHECK: .section.rodata.cst8.hot,"aM",@progbits,8
+; CHECK: .LCPI2_0:
+; CHECK: .xword 0x3fe5c28f5c28f5c3 // double
0.68005
+; CHECK: .section.rodata.cst16.hot,"aM",@progbits,16
+; CHECK: .LCPI2_1:
+; CHECK: .word 442 // 0x1ba
+; CHECK: .word 100 // 0x64
+; CHECK: .word 0 // 0x0
+; CHECK: .word 0 // 0x0
+
+; CHECK:.section .rodata.cst32,"aM",@progbits,32
+; CHECK:.globl val
+
+define i32 @cold_func(double %x, <16 x i8> %a, <16 x i8> %b) !prof !16 {
+ %2 = tail call i32 (...) @func_taking_arbitrary_param(double 6.80e-01)
+ %num = tail call i32 (...) @func_taking_arbitrary_param(double 6.850e-01)
+ %t1 = call <8 x i8> @llvm.aarch64.neon.tbl2.v8i8(<16 x i8> %a, <16 x i8> %b,
<8 x i8> )
+ %t2 = bitcast <8 x i8> %t1 to <2 x i32>
+ %3 = extractelement <2 x i32> %t2, i32 1
+ %sum = add i32 %2, %3
+ %ret = add i32 %sum, %num
+ ret i32 %ret
+}
+
+declare <8 x i8> @llvm.aarch64.neon.tbl2.v8i8(<16 x i8>, <16 x i8>, <8 x i8>)
+declare i32 @func_taking_arbitrary_param(...)
+
+define <4 x i1> @unprofiled_func(<16 x i8> %a, <16 x i8> %b) {
+ %t1 = call <8 x i8> @llvm.aarch64.neon.tbl2.v8i8(<16 x i8> %a, <16 x i8> %b,
<8 x i8> )
+ %t2 = bitcast <8 x i8> %t1 to <4 x i16>
+ %t3 = zext <4 x i16> %t2 to <4 x i32>
+ %cmp = icmp ule <4 x i32> , %t3
+ ret <4 x i1> %cmp
+}
+
+define <4 x i1> @hot_func(i32 %0, <4 x i32> %a) !prof !17 {
+ %2 = tail call i32 (...) @func_taking_arbitrary_param(double 6.80e-01)
+ %b = icmp ule <4 x i32> %a,
+ ret <4 x i1> %b
+}
+
+@val = unnamed_addr constant i256 1
snehasish wrote:
Should this be used somewhere so that we can check that we correctly assign the
section suffix for 32 bit consts?
https://github.com/llvm/llvm-project/pull/129781
___
llvm-branch-commits mailing list
llvm-branch-co
[llvm-branch-commits] [llvm] [CodeGen][StaticDataSplitter]Support constant pool partitioning (PR #129781)
@@ -0,0 +1,141 @@ +; RUN: llc -mtriple=aarch64 -enable-split-machine-functions \ +; RUN: -partition-static-data-sections=true -function-sections=true \ +; RUN: -unique-section-names=false \ +; RUN: %s -o - 2>&1 | FileCheck %s --dump-input=always + +; Repeat the RUN command above for big-endian systems. +; RUN: llc -mtriple=aarch64_be -enable-split-machine-functions \ +; RUN: -partition-static-data-sections=true -function-sections=true \ +; RUN: -unique-section-names=false \ +; RUN: %s -o - 2>&1 | FileCheck %s --dump-input=always + +; Tests that constant pool hotness is aggregated across the module. The +; static-data-splitter processes data from cold_func first, unprofiled_func +; secondly, and then hot_func. Specifically, tests that +; - If a constant is accessed by hot functions, all constant pools for this +; constant (e.g., from an unprofiled function, or cold function) should have +; `.hot` suffix. +; - Similarly if a constant is accessed by both cold function and un-profiled +; function, constant pools for this constant should not have `.unlikely` suffix. + +; CHECK: .section .rodata.cst8.hot,"aM",@progbits,8 +; CHECK: .LCPI0_0: snehasish wrote: Can some (most) of these be CHECK-NEXT to make the test tighter? https://github.com/llvm/llvm-project/pull/129781 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [CodeGen][StaticDataSplitter]Support constant pool partitioning (PR #129781)
https://github.com/snehasish approved this pull request. lgtm with some suggested changes to the tests. https://github.com/llvm/llvm-project/pull/129781 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [CodeGen][StaticDataSplitter]Support constant pool partitioning (PR #129781)
@@ -0,0 +1,141 @@ +; RUN: llc -mtriple=aarch64 -enable-split-machine-functions \ snehasish wrote: Yeah, it makes sense to rely on a more precise profile due to the fine grained nature of this optimization. Refactoring out the requirement as a condition common to MFS and SDS sounds good. https://github.com/llvm/llvm-project/pull/129781 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [CodeGen][StaticDataSplitter]Support constant pool partitioning (PR #129781)
https://github.com/snehasish approved this pull request. https://github.com/llvm/llvm-project/pull/129781 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [CodeGen][StaticDataSplitter]Support constant pool partitioning (PR #129781)
@@ -0,0 +1,131 @@ +target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128" +target triple = "x86_64-grtev4-linux-gnu" + +; Tests that constant pool hotness is aggregated across the module. The snehasish wrote: I think the comments for the ARM testt applies to this file too. https://github.com/llvm/llvm-project/pull/129781 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [ctxprof] Flatten indirect call info in pre-thinlink compilation (PR #134766)
@@ -414,6 +417,58 @@ void removeInstrumentation(Function &F) {
I.eraseFromParent();
}
+void annotateIndirectCall(
+Module &M, CallBase &CB,
+const DenseMap &FlatProf,
+const InstrProfCallsite &Ins) {
+ auto Idx = Ins.getIndex()->getZExtValue();
+ auto FIt = FlatProf.find(Idx);
+ if (FIt == FlatProf.end())
+return;
+ const auto &Targets = FIt->second;
+ SmallVector Data;
+ uint64_t Sum = 0;
+ for (auto &[Guid, Count] : Targets) {
+Data.push_back({/*.Value=*/Guid, /*.Count=*/Count});
+Sum += Count;
+ }
+ struct InstrProfValueDataGTComparer {
+bool operator()(const InstrProfValueData &A, const InstrProfValueData &B) {
+ return A.Count > B.Count;
+}
+ };
+ llvm::sort(Data, InstrProfValueDataGTComparer());
+ llvm::annotateValueSite(M, CB, Data, Sum,
+ InstrProfValueKind::IPVK_IndirectCallTarget,
+ Data.size());
+ LLVM_DEBUG(dbgs() << "[ctxprof] flat indirect call prof: " << CB
+<< CB.getMetadata(LLVMContext::MD_prof) << "\n");
+}
+
+// We normally return a "Changed" bool, but the calling pass' run assumes
+// something will change - some profile will be added - so this won't add much
+// by returning false when applicable.
+void annotateIndCalls(Module &M, const CtxProfAnalysis::Result &CtxProf) {
snehasish wrote:
nit: s/Ind/Indirect/ to be consistent with the other method?
https://github.com/llvm/llvm-project/pull/134766
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [ctxprof] Flatten indirect call info in pre-thinlink compilation (PR #134766)
@@ -414,6 +417,58 @@ void removeInstrumentation(Function &F) {
I.eraseFromParent();
}
+void annotateIndirectCall(
+Module &M, CallBase &CB,
+const DenseMap &FlatProf,
+const InstrProfCallsite &Ins) {
+ auto Idx = Ins.getIndex()->getZExtValue();
+ auto FIt = FlatProf.find(Idx);
+ if (FIt == FlatProf.end())
+return;
+ const auto &Targets = FIt->second;
+ SmallVector Data;
+ uint64_t Sum = 0;
+ for (auto &[Guid, Count] : Targets) {
+Data.push_back({/*.Value=*/Guid, /*.Count=*/Count});
+Sum += Count;
+ }
+ struct InstrProfValueDataGTComparer {
snehasish wrote:
This could be an inline lambda as an argument to llvm::sort().
https://github.com/llvm/llvm-project/pull/134766
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [ctxprof] Flatten indirect call info in pre-thinlink compilation (PR #134766)
https://github.com/snehasish approved this pull request. lgtm https://github.com/llvm/llvm-project/pull/134766 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [Metadata] Preserve MD_prof when merging instructions when one is missing. (PR #132433)
https://github.com/snehasish updated
https://github.com/llvm/llvm-project/pull/132433
>From 4680029efc3f4e350f7a07033088a9925b9300d7 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 21 Mar 2025 17:00:38 +
Subject: [PATCH] Update tests, apply clang-tidy suggestions
---
llvm/lib/Transforms/Utils/Local.cpp | 19 --
...rect-call-branch-weights-preserve-hoist.ll | 62 ++
...irect-call-branch-weights-preserve-sink.ll | 63 +++
3 files changed, 138 insertions(+), 6 deletions(-)
create mode 100644
llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
create mode 100644
llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-sink.ll
diff --git a/llvm/lib/Transforms/Utils/Local.cpp
b/llvm/lib/Transforms/Utils/Local.cpp
index edec0e7a94422..c136825d47b9c 100644
--- a/llvm/lib/Transforms/Utils/Local.cpp
+++ b/llvm/lib/Transforms/Utils/Local.cpp
@@ -3355,9 +3355,10 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
case LLVMContext::MD_invariant_group:
// Preserve !invariant.group in K.
break;
- // Keep empty cases for mmra, memprof, and callsite to prevent them from
- // being removed as unknown metadata. The actual merging is handled
+ // Keep empty cases for prof, mmra, memprof, and callsite to prevent them
+ // from being removed as unknown metadata. The actual merging is handled
// separately below.
+ case LLVMContext::MD_prof:
case LLVMContext::MD_mmra:
case LLVMContext::MD_memprof:
case LLVMContext::MD_callsite:
@@ -3386,10 +3387,6 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
if (!AAOnly)
K->setMetadata(Kind, JMD);
break;
- case LLVMContext::MD_prof:
-if (!AAOnly && DoesKMove)
- K->setMetadata(Kind, MDNode::getMergedProfMetadata(KMD, JMD, K, J));
-break;
case LLVMContext::MD_noalias_addrspace:
if (DoesKMove)
K->setMetadata(Kind,
@@ -3436,6 +3433,16 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
K->setMetadata(LLVMContext::MD_callsite,
MDNode::getMergedCallsiteMetadata(KCallSite, JCallSite));
}
+
+ // Merge prof metadata.
+ // Handle separately to support cases where only one instruction has the
+ // metadata.
+ auto *JProf = J->getMetadata(LLVMContext::MD_prof);
+ auto *KProf = K->getMetadata(LLVMContext::MD_prof);
+ if (!AAOnly && (JProf || KProf)) {
+K->setMetadata(LLVMContext::MD_prof,
+ MDNode::getMergedProfMetadata(KProf, JProf, K, J));
+ }
}
void llvm::combineMetadataForCSE(Instruction *K, const Instruction *J,
diff --git
a/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
b/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
new file mode 100644
index 0..d6058134f5285
--- /dev/null
+++
b/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
@@ -0,0 +1,62 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
UTC_ARGS: --check-globals --version 2
+; RUN: opt < %s -passes='simplifycfg'
-simplifycfg-require-and-preserve-domtree=1 -S | FileCheck %s
--check-prefix=HOIST
+
+; Test case based on C++ code with manualy annotated !prof metadata.
+; This is to test that when calls to 'func1' from 'if.then' block
+; and 'if.else' block are hoisted, the branch_weights are merged and
+; attached to merged call rather than dropped.
+;
+; int func1(int a, int b) ;
+; int func2(int a, int b) ;
+
+; int func(int a, int b, bool c) {
+;int sum= 0;
+;if(c) {
+;sum += func1(a, b);
+;} else {
+;sum += func1(a, b);
+;sum -= func2(a, b);
+;}
+;return sum;
+; }
+define i32 @_Z4funciib(i32 %a, i32 %b, i1 %c) {
+; HOIST-LABEL: define i32 @_Z4funciib
+; HOIST-SAME: (i32 [[A:%.*]], i32 [[B:%.*]], i1 [[C:%.*]]) {
+; HOIST-NEXT: entry:
+; HOIST-NEXT:[[CALL:%.*]] = tail call i32 @_Z5func1ii(i32 [[A]], i32
[[B]]), !prof [[PROF0:![0-9]+]]
+; HOIST-NEXT:br i1 [[C]], label [[IF_END:%.*]], label [[IF_ELSE:%.*]]
+; HOIST: if.else:
+; HOIST-NEXT:[[CALL3:%.*]] = tail call i32 @_Z5func2ii(i32 [[A]], i32
[[B]])
+; HOIST-NEXT:[[SUB:%.*]] = sub i32 [[CALL]], [[CALL3]]
+; HOIST-NEXT:br label [[IF_END]]
+; HOIST: if.end:
+; HOIST-NEXT:[[SUM_0:%.*]] = phi i32 [ [[SUB]], [[IF_ELSE]] ], [ [[CALL]],
[[ENTRY:%.*]] ]
+; HOIST-NEXT:ret i32 [[SUM_0]]
+;
+entry:
+ br i1 %c, label %if.then, label %if.else
+
+if.then: ; preds = %entry
+ %call = tail call i32 @_Z5func1ii(i32 %a, i32 %b)
+ br label %if.end
+
+if.else: ; preds = %entry
+ %call1 = tail call i32 @_Z5func1ii(i32 %a, i32 %b), !prof !0
+ %call3 = tail call i32 @_Z5func2ii(i32 %a, i32 %b)
+ %
[llvm-branch-commits] [llvm] [Metadata] Preserve MD_prof when merging instructions when one is missing. (PR #132433)
https://github.com/snehasish updated
https://github.com/llvm/llvm-project/pull/132433
>From 4680029efc3f4e350f7a07033088a9925b9300d7 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 21 Mar 2025 17:00:38 +
Subject: [PATCH] Update tests, apply clang-tidy suggestions
---
llvm/lib/Transforms/Utils/Local.cpp | 19 --
...rect-call-branch-weights-preserve-hoist.ll | 62 ++
...irect-call-branch-weights-preserve-sink.ll | 63 +++
3 files changed, 138 insertions(+), 6 deletions(-)
create mode 100644
llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
create mode 100644
llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-sink.ll
diff --git a/llvm/lib/Transforms/Utils/Local.cpp
b/llvm/lib/Transforms/Utils/Local.cpp
index edec0e7a94422..c136825d47b9c 100644
--- a/llvm/lib/Transforms/Utils/Local.cpp
+++ b/llvm/lib/Transforms/Utils/Local.cpp
@@ -3355,9 +3355,10 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
case LLVMContext::MD_invariant_group:
// Preserve !invariant.group in K.
break;
- // Keep empty cases for mmra, memprof, and callsite to prevent them from
- // being removed as unknown metadata. The actual merging is handled
+ // Keep empty cases for prof, mmra, memprof, and callsite to prevent them
+ // from being removed as unknown metadata. The actual merging is handled
// separately below.
+ case LLVMContext::MD_prof:
case LLVMContext::MD_mmra:
case LLVMContext::MD_memprof:
case LLVMContext::MD_callsite:
@@ -3386,10 +3387,6 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
if (!AAOnly)
K->setMetadata(Kind, JMD);
break;
- case LLVMContext::MD_prof:
-if (!AAOnly && DoesKMove)
- K->setMetadata(Kind, MDNode::getMergedProfMetadata(KMD, JMD, K, J));
-break;
case LLVMContext::MD_noalias_addrspace:
if (DoesKMove)
K->setMetadata(Kind,
@@ -3436,6 +3433,16 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
K->setMetadata(LLVMContext::MD_callsite,
MDNode::getMergedCallsiteMetadata(KCallSite, JCallSite));
}
+
+ // Merge prof metadata.
+ // Handle separately to support cases where only one instruction has the
+ // metadata.
+ auto *JProf = J->getMetadata(LLVMContext::MD_prof);
+ auto *KProf = K->getMetadata(LLVMContext::MD_prof);
+ if (!AAOnly && (JProf || KProf)) {
+K->setMetadata(LLVMContext::MD_prof,
+ MDNode::getMergedProfMetadata(KProf, JProf, K, J));
+ }
}
void llvm::combineMetadataForCSE(Instruction *K, const Instruction *J,
diff --git
a/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
b/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
new file mode 100644
index 0..d6058134f5285
--- /dev/null
+++
b/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
@@ -0,0 +1,62 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
UTC_ARGS: --check-globals --version 2
+; RUN: opt < %s -passes='simplifycfg'
-simplifycfg-require-and-preserve-domtree=1 -S | FileCheck %s
--check-prefix=HOIST
+
+; Test case based on C++ code with manualy annotated !prof metadata.
+; This is to test that when calls to 'func1' from 'if.then' block
+; and 'if.else' block are hoisted, the branch_weights are merged and
+; attached to merged call rather than dropped.
+;
+; int func1(int a, int b) ;
+; int func2(int a, int b) ;
+
+; int func(int a, int b, bool c) {
+;int sum= 0;
+;if(c) {
+;sum += func1(a, b);
+;} else {
+;sum += func1(a, b);
+;sum -= func2(a, b);
+;}
+;return sum;
+; }
+define i32 @_Z4funciib(i32 %a, i32 %b, i1 %c) {
+; HOIST-LABEL: define i32 @_Z4funciib
+; HOIST-SAME: (i32 [[A:%.*]], i32 [[B:%.*]], i1 [[C:%.*]]) {
+; HOIST-NEXT: entry:
+; HOIST-NEXT:[[CALL:%.*]] = tail call i32 @_Z5func1ii(i32 [[A]], i32
[[B]]), !prof [[PROF0:![0-9]+]]
+; HOIST-NEXT:br i1 [[C]], label [[IF_END:%.*]], label [[IF_ELSE:%.*]]
+; HOIST: if.else:
+; HOIST-NEXT:[[CALL3:%.*]] = tail call i32 @_Z5func2ii(i32 [[A]], i32
[[B]])
+; HOIST-NEXT:[[SUB:%.*]] = sub i32 [[CALL]], [[CALL3]]
+; HOIST-NEXT:br label [[IF_END]]
+; HOIST: if.end:
+; HOIST-NEXT:[[SUM_0:%.*]] = phi i32 [ [[SUB]], [[IF_ELSE]] ], [ [[CALL]],
[[ENTRY:%.*]] ]
+; HOIST-NEXT:ret i32 [[SUM_0]]
+;
+entry:
+ br i1 %c, label %if.then, label %if.else
+
+if.then: ; preds = %entry
+ %call = tail call i32 @_Z5func1ii(i32 %a, i32 %b)
+ br label %if.end
+
+if.else: ; preds = %entry
+ %call1 = tail call i32 @_Z5func1ii(i32 %a, i32 %b), !prof !0
+ %call3 = tail call i32 @_Z5func2ii(i32 %a, i32 %b)
+ %
[llvm-branch-commits] [llvm] [ctxprof] Flatten indirect call info in pre-thinlink compilation (PR #134766)
https://github.com/snehasish edited https://github.com/llvm/llvm-project/pull/134766 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [CodeGen][StaticDataSplitter]Support constant pool partitioning (PR #129781)
@@ -2769,6 +2769,23 @@ namespace {
} // end anonymous namespace
+StringRef AsmPrinter::getConstantSectionSuffix(const Constant *C) const {
+ SmallString<8> SectionNameSuffix;
+ if (TM.Options.EnableStaticDataPartitioning) {
+if (C && SDPI && PSI) {
+ auto Count = SDPI->getConstantProfileCount(C);
+ if (Count) {
+if (PSI->isHotCount(*Count)) {
+ SectionNameSuffix.append("hot");
+} else if (PSI->isColdCount(*Count) && !SDPI->hasUnknownCount(C)) {
+ SectionNameSuffix.append("unlikely");
+}
+ }
+}
+ }
+ return SectionNameSuffix.str();
snehasish wrote:
This is returning a StringRef whose underlying memory is stack allocated. I
don't think you need a SmallString here, just return std::string or c-string
and convert it at the callsite?
https://github.com/llvm/llvm-project/pull/129781
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [CodeGen][StaticDataSplitter]Support constant pool partitioning (PR #129781)
@@ -2769,6 +2769,23 @@ namespace {
} // end anonymous namespace
+StringRef AsmPrinter::getConstantSectionSuffix(const Constant *C) const {
snehasish wrote:
Consider refactoring a bit to reduce nesting --
```
if(!TM.Options.EnableStaticDataPartitioning || C == nullptr || SDPI == nullptr
|| PSI == nullptr) return "";
auto Count = SDPI->getConstantProfileCount(C);
if(!Count.has_value()) return "";
if (PSI->isHotCount(*Count)) {
return "hot";
} else if (PSI->isColdCount(*Count) && !SDPI->hasUnknownCount(C)) {
return "unlikely";
}
return "";
```
https://github.com/llvm/llvm-project/pull/129781
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [CodeGen][StaticDataSplitter]Support constant pool partitioning (PR #129781)
@@ -206,14 +234,10 @@ void
StaticDataSplitter::annotateStaticDataWithoutProfiles(
for (const auto &MBB : MF) {
for (const MachineInstr &I : MBB) {
for (const MachineOperand &Op : I.operands()) {
-if (!Op.isGlobal())
- continue;
-const GlobalVariable *GV =
-getLocalLinkageGlobalVariable(Op.getGlobal());
-if (!GV || GV->getName().starts_with("llvm.") ||
-!inStaticDataSection(GV, MF.getTarget()))
- continue;
-SDPI->addConstantProfileCount(GV, std::nullopt);
+const Constant *C =
snehasish wrote:
nit: Can be combined with the code below --
```
if(auto *C = getConstant(); C) {
SDPI->addConstantProfileCount(C, std::nullopt);
}
```
Same for `PSIW` etc below.
https://github.com/llvm/llvm-project/pull/129781
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [Metadata] Preserve MD_prof when merging instructions when one is missing. (PR #132433)
@@ -3436,6 +3433,16 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
K->setMetadata(LLVMContext::MD_callsite,
MDNode::getMergedCallsiteMetadata(KCallSite, JCallSite));
}
+
+ // Merge prof metadata.
+ // Handle separately to support cases where only one instruction has the
+ // metadata.
+ auto JProf = J->getMetadata(LLVMContext::MD_prof);
+ auto KProf = K->getMetadata(LLVMContext::MD_prof);
+ if (!AAOnly && (JProf || KProf)) {
snehasish wrote:
Removing the condition was intentional. In the case that `DoesKMove` is true,
the merging of md_prof can be skipped if `J` does not have prof metadata (a
minor optimization). I felt that it was simpler to just perform the merge
regardless. Let me know if you feel otherwise.
https://github.com/llvm/llvm-project/pull/132433
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [Metadata] Preserve MD_prof when merging instructions when one is missing. (PR #132433)
@@ -0,0 +1,62 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
UTC_ARGS: --check-globals --version 2
+; RUN: opt < %s -passes='simplifycfg'
-simplifycfg-require-and-preserve-domtree=1 -S | FileCheck %s
--check-prefix=HOIST
+
+; Test case based on C++ code with manualy annotated !prof metadata.
+; This is to test that when calls to 'func1' from 'if.then' block
+; and 'if.else' block are hoisted, the branch_weights are merged and
+; attached to merged call rather than dropped.
+;
+; int func1(int a, int b) ;
+; int func2(int a, int b) ;
+
+; int func(int a, int b, bool c) {
+;int sum= 0;
+;if(c) {
+;sum += func1(a, b);
+;} else {
+;sum += func1(a, b);
+;sum -= func2(a, b);
+;}
+;return sum;
+; }
+define i32 @_Z4funciib(i32 %a, i32 %b, i1 %c) {
+; HOIST-LABEL: define i32 @_Z4funciib
+; HOIST-SAME: (i32 [[A:%.*]], i32 [[B:%.*]], i1 [[C:%.*]]) {
+; HOIST-NEXT: entry:
+; HOIST-NEXT:[[CALL:%.*]] = tail call i32 @_Z5func1ii(i32 [[A]], i32
[[B]]), !prof [[PROF0:![0-9]+]]
+; HOIST-NEXT:br i1 [[C]], label [[IF_END:%.*]], label [[IF_ELSE:%.*]]
+; HOIST: if.else:
+; HOIST-NEXT:[[CALL3:%.*]] = tail call i32 @_Z5func2ii(i32 [[A]], i32
[[B]])
+; HOIST-NEXT:[[SUB:%.*]] = sub i32 [[CALL]], [[CALL3]]
+; HOIST-NEXT:br label [[IF_END]]
+; HOIST: if.end:
+; HOIST-NEXT:[[SUM_0:%.*]] = phi i32 [ [[SUB]], [[IF_ELSE]] ], [ [[CALL]],
[[ENTRY:%.*]] ]
+; HOIST-NEXT:ret i32 [[SUM_0]]
+;
+entry:
+ br i1 %c, label %if.then, label %if.else
+
+if.then: ; preds = %entry
+ %call = tail call i32 @_Z5func1ii(i32 %a, i32 %b), !prof !0
+ br label %if.end
+
+if.else: ; preds = %entry
+ %call1 = tail call i32 @_Z5func1ii(i32 %a, i32 %b)
+ %call3 = tail call i32 @_Z5func2ii(i32 %a, i32 %b)
+ %sub = sub i32 %call1, %call3
+ br label %if.end
+
+if.end: ; preds = %if.else, %if.then
+ %sum.0 = phi i32 [ %call, %if.then ], [ %sub, %if.else ]
+ ret i32 %sum.0
+}
+
+declare i32 @_Z5func1ii(i32, i32)
+
+declare i32 @_Z5func2ii(i32, i32)
+
+!0 = !{!"branch_weights", i32 10}
+!1 = !{!"branch_weights", i32 90}
snehasish wrote:
Removed.
https://github.com/llvm/llvm-project/pull/132433
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [Metadata] Preserve MD_prof when merging instructions when one is missing. (PR #132433)
https://github.com/snehasish updated
https://github.com/llvm/llvm-project/pull/132433
>From 42a9972571f8c8872e7d71def2236be400428606 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 21 Mar 2025 17:00:38 +
Subject: [PATCH] Update tests, apply clang-tidy suggestions
---
llvm/lib/Transforms/Utils/Local.cpp | 19 --
...rect-call-branch-weights-preserve-hoist.ll | 62 ++
...irect-call-branch-weights-preserve-sink.ll | 63 +++
3 files changed, 138 insertions(+), 6 deletions(-)
create mode 100644
llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
create mode 100644
llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-sink.ll
diff --git a/llvm/lib/Transforms/Utils/Local.cpp
b/llvm/lib/Transforms/Utils/Local.cpp
index edec0e7a94422..c136825d47b9c 100644
--- a/llvm/lib/Transforms/Utils/Local.cpp
+++ b/llvm/lib/Transforms/Utils/Local.cpp
@@ -3355,9 +3355,10 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
case LLVMContext::MD_invariant_group:
// Preserve !invariant.group in K.
break;
- // Keep empty cases for mmra, memprof, and callsite to prevent them from
- // being removed as unknown metadata. The actual merging is handled
+ // Keep empty cases for prof, mmra, memprof, and callsite to prevent them
+ // from being removed as unknown metadata. The actual merging is handled
// separately below.
+ case LLVMContext::MD_prof:
case LLVMContext::MD_mmra:
case LLVMContext::MD_memprof:
case LLVMContext::MD_callsite:
@@ -3386,10 +3387,6 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
if (!AAOnly)
K->setMetadata(Kind, JMD);
break;
- case LLVMContext::MD_prof:
-if (!AAOnly && DoesKMove)
- K->setMetadata(Kind, MDNode::getMergedProfMetadata(KMD, JMD, K, J));
-break;
case LLVMContext::MD_noalias_addrspace:
if (DoesKMove)
K->setMetadata(Kind,
@@ -3436,6 +3433,16 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
K->setMetadata(LLVMContext::MD_callsite,
MDNode::getMergedCallsiteMetadata(KCallSite, JCallSite));
}
+
+ // Merge prof metadata.
+ // Handle separately to support cases where only one instruction has the
+ // metadata.
+ auto *JProf = J->getMetadata(LLVMContext::MD_prof);
+ auto *KProf = K->getMetadata(LLVMContext::MD_prof);
+ if (!AAOnly && (JProf || KProf)) {
+K->setMetadata(LLVMContext::MD_prof,
+ MDNode::getMergedProfMetadata(KProf, JProf, K, J));
+ }
}
void llvm::combineMetadataForCSE(Instruction *K, const Instruction *J,
diff --git
a/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
b/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
new file mode 100644
index 0..d6058134f5285
--- /dev/null
+++
b/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
@@ -0,0 +1,62 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
UTC_ARGS: --check-globals --version 2
+; RUN: opt < %s -passes='simplifycfg'
-simplifycfg-require-and-preserve-domtree=1 -S | FileCheck %s
--check-prefix=HOIST
+
+; Test case based on C++ code with manualy annotated !prof metadata.
+; This is to test that when calls to 'func1' from 'if.then' block
+; and 'if.else' block are hoisted, the branch_weights are merged and
+; attached to merged call rather than dropped.
+;
+; int func1(int a, int b) ;
+; int func2(int a, int b) ;
+
+; int func(int a, int b, bool c) {
+;int sum= 0;
+;if(c) {
+;sum += func1(a, b);
+;} else {
+;sum += func1(a, b);
+;sum -= func2(a, b);
+;}
+;return sum;
+; }
+define i32 @_Z4funciib(i32 %a, i32 %b, i1 %c) {
+; HOIST-LABEL: define i32 @_Z4funciib
+; HOIST-SAME: (i32 [[A:%.*]], i32 [[B:%.*]], i1 [[C:%.*]]) {
+; HOIST-NEXT: entry:
+; HOIST-NEXT:[[CALL:%.*]] = tail call i32 @_Z5func1ii(i32 [[A]], i32
[[B]]), !prof [[PROF0:![0-9]+]]
+; HOIST-NEXT:br i1 [[C]], label [[IF_END:%.*]], label [[IF_ELSE:%.*]]
+; HOIST: if.else:
+; HOIST-NEXT:[[CALL3:%.*]] = tail call i32 @_Z5func2ii(i32 [[A]], i32
[[B]])
+; HOIST-NEXT:[[SUB:%.*]] = sub i32 [[CALL]], [[CALL3]]
+; HOIST-NEXT:br label [[IF_END]]
+; HOIST: if.end:
+; HOIST-NEXT:[[SUM_0:%.*]] = phi i32 [ [[SUB]], [[IF_ELSE]] ], [ [[CALL]],
[[ENTRY:%.*]] ]
+; HOIST-NEXT:ret i32 [[SUM_0]]
+;
+entry:
+ br i1 %c, label %if.then, label %if.else
+
+if.then: ; preds = %entry
+ %call = tail call i32 @_Z5func1ii(i32 %a, i32 %b)
+ br label %if.end
+
+if.else: ; preds = %entry
+ %call1 = tail call i32 @_Z5func1ii(i32 %a, i32 %b), !prof !0
+ %call3 = tail call i32 @_Z5func2ii(i32 %a, i32 %b)
+ %
[llvm-branch-commits] [llvm] [Metadata] Preserve MD_prof when merging instructions when one is missing. (PR #132433)
https://github.com/snehasish updated
https://github.com/llvm/llvm-project/pull/132433
>From 42a9972571f8c8872e7d71def2236be400428606 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 21 Mar 2025 17:00:38 +
Subject: [PATCH] Update tests, apply clang-tidy suggestions
---
llvm/lib/Transforms/Utils/Local.cpp | 19 --
...rect-call-branch-weights-preserve-hoist.ll | 62 ++
...irect-call-branch-weights-preserve-sink.ll | 63 +++
3 files changed, 138 insertions(+), 6 deletions(-)
create mode 100644
llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
create mode 100644
llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-sink.ll
diff --git a/llvm/lib/Transforms/Utils/Local.cpp
b/llvm/lib/Transforms/Utils/Local.cpp
index edec0e7a94422..c136825d47b9c 100644
--- a/llvm/lib/Transforms/Utils/Local.cpp
+++ b/llvm/lib/Transforms/Utils/Local.cpp
@@ -3355,9 +3355,10 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
case LLVMContext::MD_invariant_group:
// Preserve !invariant.group in K.
break;
- // Keep empty cases for mmra, memprof, and callsite to prevent them from
- // being removed as unknown metadata. The actual merging is handled
+ // Keep empty cases for prof, mmra, memprof, and callsite to prevent them
+ // from being removed as unknown metadata. The actual merging is handled
// separately below.
+ case LLVMContext::MD_prof:
case LLVMContext::MD_mmra:
case LLVMContext::MD_memprof:
case LLVMContext::MD_callsite:
@@ -3386,10 +3387,6 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
if (!AAOnly)
K->setMetadata(Kind, JMD);
break;
- case LLVMContext::MD_prof:
-if (!AAOnly && DoesKMove)
- K->setMetadata(Kind, MDNode::getMergedProfMetadata(KMD, JMD, K, J));
-break;
case LLVMContext::MD_noalias_addrspace:
if (DoesKMove)
K->setMetadata(Kind,
@@ -3436,6 +3433,16 @@ static void combineMetadata(Instruction *K, const
Instruction *J,
K->setMetadata(LLVMContext::MD_callsite,
MDNode::getMergedCallsiteMetadata(KCallSite, JCallSite));
}
+
+ // Merge prof metadata.
+ // Handle separately to support cases where only one instruction has the
+ // metadata.
+ auto *JProf = J->getMetadata(LLVMContext::MD_prof);
+ auto *KProf = K->getMetadata(LLVMContext::MD_prof);
+ if (!AAOnly && (JProf || KProf)) {
+K->setMetadata(LLVMContext::MD_prof,
+ MDNode::getMergedProfMetadata(KProf, JProf, K, J));
+ }
}
void llvm::combineMetadataForCSE(Instruction *K, const Instruction *J,
diff --git
a/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
b/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
new file mode 100644
index 0..d6058134f5285
--- /dev/null
+++
b/llvm/test/Transforms/SimplifyCFG/merge-direct-call-branch-weights-preserve-hoist.ll
@@ -0,0 +1,62 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
UTC_ARGS: --check-globals --version 2
+; RUN: opt < %s -passes='simplifycfg'
-simplifycfg-require-and-preserve-domtree=1 -S | FileCheck %s
--check-prefix=HOIST
+
+; Test case based on C++ code with manualy annotated !prof metadata.
+; This is to test that when calls to 'func1' from 'if.then' block
+; and 'if.else' block are hoisted, the branch_weights are merged and
+; attached to merged call rather than dropped.
+;
+; int func1(int a, int b) ;
+; int func2(int a, int b) ;
+
+; int func(int a, int b, bool c) {
+;int sum= 0;
+;if(c) {
+;sum += func1(a, b);
+;} else {
+;sum += func1(a, b);
+;sum -= func2(a, b);
+;}
+;return sum;
+; }
+define i32 @_Z4funciib(i32 %a, i32 %b, i1 %c) {
+; HOIST-LABEL: define i32 @_Z4funciib
+; HOIST-SAME: (i32 [[A:%.*]], i32 [[B:%.*]], i1 [[C:%.*]]) {
+; HOIST-NEXT: entry:
+; HOIST-NEXT:[[CALL:%.*]] = tail call i32 @_Z5func1ii(i32 [[A]], i32
[[B]]), !prof [[PROF0:![0-9]+]]
+; HOIST-NEXT:br i1 [[C]], label [[IF_END:%.*]], label [[IF_ELSE:%.*]]
+; HOIST: if.else:
+; HOIST-NEXT:[[CALL3:%.*]] = tail call i32 @_Z5func2ii(i32 [[A]], i32
[[B]])
+; HOIST-NEXT:[[SUB:%.*]] = sub i32 [[CALL]], [[CALL3]]
+; HOIST-NEXT:br label [[IF_END]]
+; HOIST: if.end:
+; HOIST-NEXT:[[SUM_0:%.*]] = phi i32 [ [[SUB]], [[IF_ELSE]] ], [ [[CALL]],
[[ENTRY:%.*]] ]
+; HOIST-NEXT:ret i32 [[SUM_0]]
+;
+entry:
+ br i1 %c, label %if.then, label %if.else
+
+if.then: ; preds = %entry
+ %call = tail call i32 @_Z5func1ii(i32 %a, i32 %b)
+ br label %if.end
+
+if.else: ; preds = %entry
+ %call1 = tail call i32 @_Z5func1ii(i32 %a, i32 %b), !prof !0
+ %call3 = tail call i32 @_Z5func2ii(i32 %a, i32 %b)
+ %
[llvm-branch-commits] [llvm] [NFC][MemProf] Move IndexedMemProfData to its own header. (PR #140503)
https://github.com/snehasish created
https://github.com/llvm/llvm-project/pull/140503
None
>From 61b636b6367ff9fe41eefce3430ee58551ffedf3 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 16 May 2025 23:41:29 -0700
Subject: [PATCH] [NFC][MemProf] Move IndexedMemProfData to its own header.
---
.../llvm/ProfileData/IndexedMemProfData.h | 70 ++-
.../llvm/ProfileData/InstrProfWriter.h| 2 +-
llvm/include/llvm/ProfileData/MemProf.h | 51 --
.../llvm/ProfileData/MemProfRadixTree.h | 1 +
llvm/include/llvm/ProfileData/MemProfReader.h | 1 +
llvm/lib/Bitcode/Writer/BitcodeWriter.cpp | 1 +
llvm/lib/ProfileData/InstrProfWriter.cpp | 1 -
llvm/lib/ProfileData/MemProf.cpp | 13
llvm/unittests/ProfileData/InstrProfTest.cpp | 1 +
llvm/unittests/ProfileData/MemProfTest.cpp| 1 +
.../Instrumentation/MemProfUseTest.cpp| 1 +
11 files changed, 75 insertions(+), 68 deletions(-)
diff --git a/llvm/include/llvm/ProfileData/IndexedMemProfData.h
b/llvm/include/llvm/ProfileData/IndexedMemProfData.h
index 3c6c329d1c49d..94a16227477cb 100644
--- a/llvm/include/llvm/ProfileData/IndexedMemProfData.h
+++ b/llvm/include/llvm/ProfileData/IndexedMemProfData.h
@@ -6,18 +6,84 @@
//
//===--===//
//
-// MemProf data is serialized in writeMemProf provided in this header file.
+// This file implements IndexedMemProfData, a data structure to hold MemProf
+// in a space optimized format. It also provides utility methods for writing
+// MemProf data.
//
//===--===//
+#ifndef LLVM_PROFILEDATA_INDEXEDMEMPROFDATA_H
+#define LLVM_PROFILEDATA_INDEXEDMEMPROFDATA_H
+
#include "llvm/ProfileData/InstrProf.h"
#include "llvm/ProfileData/MemProf.h"
namespace llvm {
+namespace memprof {
+struct IndexedMemProfData {
+ // A map to hold memprof data per function. The lower 64 bits obtained from
+ // the md5 hash of the function name is used to index into the map.
+ llvm::MapVector Records;
+
+ // A map to hold frame id to frame mappings. The mappings are used to
+ // convert IndexedMemProfRecord to MemProfRecords with frame information
+ // inline.
+ llvm::MapVector Frames;
+
+ // A map to hold call stack id to call stacks.
+ llvm::MapVector> CallStacks;
+
+ FrameId addFrame(const Frame &F) {
+const FrameId Id = hashFrame(F);
+Frames.try_emplace(Id, F);
+return Id;
+ }
+
+ CallStackId addCallStack(ArrayRef CS) {
+CallStackId CSId = hashCallStack(CS);
+CallStacks.try_emplace(CSId, CS);
+return CSId;
+ }
+
+ CallStackId addCallStack(SmallVector &&CS) {
+CallStackId CSId = hashCallStack(CS);
+CallStacks.try_emplace(CSId, std::move(CS));
+return CSId;
+ }
+
+private:
+ // Return a hash value based on the contents of the frame. Here we use a
+ // cryptographic hash function to minimize the chance of hash collisions. We
+ // do persist FrameIds as part of memprof formats up to Version 2, inclusive.
+ // However, the deserializer never calls this function; it uses FrameIds
+ // merely as keys to look up Frames proper.
+ FrameId hashFrame(const Frame &F) const {
+llvm::HashBuilder, llvm::endianness::little>
+HashBuilder;
+HashBuilder.add(F.Function, F.LineOffset, F.Column, F.IsInlineFrame);
+llvm::BLAKE3Result<8> Hash = HashBuilder.final();
+FrameId Id;
+std::memcpy(&Id, Hash.data(), sizeof(Hash));
+return Id;
+ }
+
+ // Compute a CallStackId for a given call stack.
+ CallStackId hashCallStack(ArrayRef CS) const {
+ llvm::HashBuilder, llvm::endianness::little>
+ HashBuilder;
+ for (FrameId F : CS)
+HashBuilder.add(F);
+ llvm::BLAKE3Result<8> Hash = HashBuilder.final();
+ CallStackId CSId;
+ std::memcpy(&CSId, Hash.data(), sizeof(Hash));
+ return CSId;
+}
+};
+} // namespace memprof
// Write the MemProf data to OS.
Error writeMemProf(ProfOStream &OS, memprof::IndexedMemProfData &MemProfData,
memprof::IndexedVersion MemProfVersionRequested,
bool MemProfFullSchema);
-
} // namespace llvm
+#endif
diff --git a/llvm/include/llvm/ProfileData/InstrProfWriter.h
b/llvm/include/llvm/ProfileData/InstrProfWriter.h
index 67d85daa81623..16d2ef3fab3e3 100644
--- a/llvm/include/llvm/ProfileData/InstrProfWriter.h
+++ b/llvm/include/llvm/ProfileData/InstrProfWriter.h
@@ -20,7 +20,7 @@
#include "llvm/IR/GlobalValue.h"
#include "llvm/Object/BuildID.h"
#include "llvm/ProfileData/InstrProf.h"
-#include "llvm/ProfileData/MemProf.h"
+#include "llvm/ProfileData/IndexedMemProfData.h"
#include "llvm/Support/Error.h"
#include
#include
diff --git a/llvm/include/llvm/ProfileData/MemProf.h
b/llvm/include/llvm/ProfileData/MemProf.h
index 215102c131fff..ce5cd5ee4856b 100644
--- a/llvm/include/llvm/ProfileData/MemProf.h
+++ b/llvm/include/llvm/ProfileData/MemProf.h
@@ -842,57 +842
[llvm-branch-commits] [llvm] [NFC][MemProf] Move types shared between Analysis, ProfileData and ModuleSummary (Core) to a separate header (PR #140505)
https://github.com/snehasish created
https://github.com/llvm/llvm-project/pull/140505
None
>From d4413c619035039133f7d3509bbe6c94a1650adf Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Mon, 19 May 2025 00:03:59 -0700
Subject: [PATCH] [NFC][MemProf] Move types shared between Analysis,
ProfileData and ModuleSummary (Core) to a separate header
---
.../include/llvm/Analysis/MemoryProfileInfo.h | 3 +-
llvm/include/llvm/IR/ModuleSummaryIndex.h | 22 +-
llvm/include/llvm/ProfileData/MemProfCommon.h | 44 +++
.../Instrumentation/MemProfiler.cpp | 1 +
4 files changed, 49 insertions(+), 21 deletions(-)
create mode 100644 llvm/include/llvm/ProfileData/MemProfCommon.h
diff --git a/llvm/include/llvm/Analysis/MemoryProfileInfo.h
b/llvm/include/llvm/Analysis/MemoryProfileInfo.h
index 1d98f86f50484..33d59efe8d77e 100644
--- a/llvm/include/llvm/Analysis/MemoryProfileInfo.h
+++ b/llvm/include/llvm/Analysis/MemoryProfileInfo.h
@@ -14,7 +14,8 @@
#define LLVM_ANALYSIS_MEMORYPROFILEINFO_H
#include "llvm/IR/Metadata.h"
-#include "llvm/IR/ModuleSummaryIndex.h"
+#include "llvm/ProfileData/MemProfCommon.h"
+#include "llvm/IR/InstrTypes.h"
#include
namespace llvm {
diff --git a/llvm/include/llvm/IR/ModuleSummaryIndex.h
b/llvm/include/llvm/IR/ModuleSummaryIndex.h
index 65e428a3adea7..77430c5cb5ea1 100644
--- a/llvm/include/llvm/IR/ModuleSummaryIndex.h
+++ b/llvm/include/llvm/IR/ModuleSummaryIndex.h
@@ -27,6 +27,7 @@
#include "llvm/IR/ConstantRange.h"
#include "llvm/IR/GlobalValue.h"
#include "llvm/IR/Module.h"
+#include "llvm/ProfileData/MemProfCommon.h"
#include "llvm/Support/Allocator.h"
#include "llvm/Support/Compiler.h"
#include "llvm/Support/InterleavedRange.h"
@@ -306,13 +307,7 @@ template <> struct DenseMapInfo {
static unsigned getHashValue(ValueInfo I) { return hash_value(I.getRef()); }
};
-// For optional hinted size reporting, holds a pair of the full stack id
-// (pre-trimming, from the full context in the profile), and the associated
-// total profiled size.
-struct ContextTotalSize {
- uint64_t FullStackId;
- uint64_t TotalSize;
-};
+
/// Summary of memprof callsite metadata.
struct CallsiteInfo {
@@ -350,19 +345,6 @@ inline raw_ostream &operator<<(raw_ostream &OS, const
CallsiteInfo &SNI) {
return OS;
}
-// Allocation type assigned to an allocation reached by a given context.
-// More can be added, now this is cold, notcold and hot.
-// Values should be powers of two so that they can be ORed, in particular to
-// track allocations that have different behavior with different calling
-// contexts.
-enum class AllocationType : uint8_t {
- None = 0,
- NotCold = 1,
- Cold = 2,
- Hot = 4,
- All = 7 // This should always be set to the OR of all values.
-};
-
/// Summary of a single MIB in a memprof metadata on allocations.
struct MIBInfo {
// The allocation type for this profiled context.
diff --git a/llvm/include/llvm/ProfileData/MemProfCommon.h
b/llvm/include/llvm/ProfileData/MemProfCommon.h
new file mode 100644
index 0..4097ccb651188
--- /dev/null
+++ b/llvm/include/llvm/ProfileData/MemProfCommon.h
@@ -0,0 +1,44 @@
+//===- MemProfCommon.h - MemProf support *- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM
Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+//
+// This file contains common types used by different parts of the MemProf code.
+//
+//===--===//
+
+#ifndef LLVM_PROFILEDATA_MEMPROFCOMMON_H
+#define LLVM_PROFILEDATA_MEMPROFCOMMON_H
+
+#include
+
+namespace llvm {
+
+// For optional hinted size reporting, holds a pair of the full stack id
+// (pre-trimming, from the full context in the profile), and the associated
+// total profiled size.
+struct ContextTotalSize {
+ uint64_t FullStackId;
+ uint64_t TotalSize;
+};
+
+// Allocation type assigned to an allocation reached by a given context.
+// More can be added, now this is cold, notcold and hot.
+// Values should be powers of two so that they can be ORed, in particular to
+// track allocations that have different behavior with different calling
+// contexts.
+enum class AllocationType : uint8_t {
+ None = 0,
+ NotCold = 1,
+ Cold = 2,
+ Hot = 4,
+ All = 7 // This should always be set to the OR of all values.
+};
+
+} // namespace llvm
+
+#endif // LLVM_PROFILEDATA_MEMPROFCOMMON_H
+
diff --git a/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp
b/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp
index 5982476f3994e..6538311571529 100644
--- a/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp
+++ b/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp
@@ -46,6 +46,7 @@
#include "llvm/Transforms/Utils/ModuleUtils.h"
#include
#include
+#include
[llvm-branch-commits] [llvm] [NFC][MemProf] Add the LLVM license text and minor clean up. (PR #140504)
https://github.com/snehasish created
https://github.com/llvm/llvm-project/pull/140504
None
>From 532d85a11742e5a3994e5efce357f85c0c60a6c7 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 16 May 2025 23:55:43 -0700
Subject: [PATCH] [NFC][MemProf] Add the LLVM license text and minor clean up.
---
llvm/include/llvm/ProfileData/MemProf.h | 20
1 file changed, 16 insertions(+), 4 deletions(-)
diff --git a/llvm/include/llvm/ProfileData/MemProf.h
b/llvm/include/llvm/ProfileData/MemProf.h
index ce5cd5ee4856b..683193aa42747 100644
--- a/llvm/include/llvm/ProfileData/MemProf.h
+++ b/llvm/include/llvm/ProfileData/MemProf.h
@@ -1,5 +1,18 @@
-#ifndef LLVM_PROFILEDATA_MEMPROF_H_
-#define LLVM_PROFILEDATA_MEMPROF_H_
+//===- MemProf.h - MemProf support --*- C++
-*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM
Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+//
+// This file contains common definitions used in the reading and writing of
+// memory profile data.
+//
+//===--===//
+
+#ifndef LLVM_PROFILEDATA_MEMPROF_H
+#define LLVM_PROFILEDATA_MEMPROF_H
#include "llvm/ADT/BitVector.h"
#include "llvm/ADT/MapVector.h"
@@ -844,5 +857,4 @@ struct LineLocation {
using CallEdgeTy = std::pair;
} // namespace memprof
} // namespace llvm
-
-#endif // LLVM_PROFILEDATA_MEMPROF_H_
+#endif // LLVM_PROFILEDATA_MEMPROF_H
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Move IndexedMemProfData to its own header. (PR #140503)
snehasish wrote: > [!WARNING] > This pull request is not mergeable via GitHub because a downstack PR is > open. Once all requirements are satisfied, merge this PR as a stack href="https://app.graphite.dev/github/pr/llvm/llvm-project/140503?utm_source=stack-comment-downstack-mergeability-warning"; > >on Graphite. > https://graphite.dev/docs/merge-pull-requests";>Learn more * **#140505** https://app.graphite.dev/github/pr/llvm/llvm-project/140505?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140504** https://app.graphite.dev/github/pr/llvm/llvm-project/140504?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140503** https://app.graphite.dev/github/pr/llvm/llvm-project/140503?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> 👈 https://app.graphite.dev/github/pr/llvm/llvm-project/140503?utm_source=stack-comment-view-in-graphite"; target="_blank">(View in Graphite) * **#140502** https://app.graphite.dev/github/pr/llvm/llvm-project/140502?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140501** https://app.graphite.dev/github/pr/llvm/llvm-project/140501?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140500** https://app.graphite.dev/github/pr/llvm/llvm-project/140500?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * `main` This stack of pull requests is managed by https://graphite.dev?utm-source=stack-comment";>Graphite. Learn more about https://stacking.dev/?utm_source=stack-comment";>stacking. https://github.com/llvm/llvm-project/pull/140503 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Add the LLVM license text and minor clean up. (PR #140504)
snehasish wrote: > [!WARNING] > This pull request is not mergeable via GitHub because a downstack PR is > open. Once all requirements are satisfied, merge this PR as a stack href="https://app.graphite.dev/github/pr/llvm/llvm-project/140504?utm_source=stack-comment-downstack-mergeability-warning"; > >on Graphite. > https://graphite.dev/docs/merge-pull-requests";>Learn more * **#140505** https://app.graphite.dev/github/pr/llvm/llvm-project/140505?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140504** https://app.graphite.dev/github/pr/llvm/llvm-project/140504?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> 👈 https://app.graphite.dev/github/pr/llvm/llvm-project/140504?utm_source=stack-comment-view-in-graphite"; target="_blank">(View in Graphite) * **#140503** https://app.graphite.dev/github/pr/llvm/llvm-project/140503?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140502** https://app.graphite.dev/github/pr/llvm/llvm-project/140502?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140501** https://app.graphite.dev/github/pr/llvm/llvm-project/140501?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140500** https://app.graphite.dev/github/pr/llvm/llvm-project/140500?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * `main` This stack of pull requests is managed by https://graphite.dev?utm-source=stack-comment";>Graphite. Learn more about https://stacking.dev/?utm_source=stack-comment";>stacking. https://github.com/llvm/llvm-project/pull/140504 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Move types shared between Analysis, ProfileData and ModuleSummary (Core) to a separate header (PR #140505)
snehasish wrote: > [!WARNING] > This pull request is not mergeable via GitHub because a downstack PR is > open. Once all requirements are satisfied, merge this PR as a stack href="https://app.graphite.dev/github/pr/llvm/llvm-project/140505?utm_source=stack-comment-downstack-mergeability-warning"; > >on Graphite. > https://graphite.dev/docs/merge-pull-requests";>Learn more * **#140505** https://app.graphite.dev/github/pr/llvm/llvm-project/140505?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> 👈 https://app.graphite.dev/github/pr/llvm/llvm-project/140505?utm_source=stack-comment-view-in-graphite"; target="_blank">(View in Graphite) * **#140504** https://app.graphite.dev/github/pr/llvm/llvm-project/140504?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140503** https://app.graphite.dev/github/pr/llvm/llvm-project/140503?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140502** https://app.graphite.dev/github/pr/llvm/llvm-project/140502?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140501** https://app.graphite.dev/github/pr/llvm/llvm-project/140501?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140500** https://app.graphite.dev/github/pr/llvm/llvm-project/140500?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * `main` This stack of pull requests is managed by https://graphite.dev?utm-source=stack-comment";>Graphite. Learn more about https://stacking.dev/?utm_source=stack-comment";>stacking. https://github.com/llvm/llvm-project/pull/140505 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Move getGUID out of IndexedMemProfRecord (PR #140502)
snehasish wrote: > [!WARNING] > This pull request is not mergeable via GitHub because a downstack PR is > open. Once all requirements are satisfied, merge this PR as a stack href="https://app.graphite.dev/github/pr/llvm/llvm-project/140502?utm_source=stack-comment-downstack-mergeability-warning"; > >on Graphite. > https://graphite.dev/docs/merge-pull-requests";>Learn more * **#140505** https://app.graphite.dev/github/pr/llvm/llvm-project/140505?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140504** https://app.graphite.dev/github/pr/llvm/llvm-project/140504?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140503** https://app.graphite.dev/github/pr/llvm/llvm-project/140503?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140502** https://app.graphite.dev/github/pr/llvm/llvm-project/140502?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> 👈 https://app.graphite.dev/github/pr/llvm/llvm-project/140502?utm_source=stack-comment-view-in-graphite"; target="_blank">(View in Graphite) * **#140501** https://app.graphite.dev/github/pr/llvm/llvm-project/140501?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140500** https://app.graphite.dev/github/pr/llvm/llvm-project/140500?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * `main` This stack of pull requests is managed by https://graphite.dev?utm-source=stack-comment";>Graphite. Learn more about https://stacking.dev/?utm_source=stack-comment";>stacking. https://github.com/llvm/llvm-project/pull/140502 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Move Radix tree methods to their own header and cpp. (PR #140501)
https://github.com/snehasish edited https://github.com/llvm/llvm-project/pull/140501 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Move getGUID out of IndexedMemProfRecord (PR #140502)
https://github.com/snehasish edited https://github.com/llvm/llvm-project/pull/140502 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Add the LLVM license text and minor clean up. (PR #140504)
https://github.com/snehasish edited https://github.com/llvm/llvm-project/pull/140504 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Move IndexedMemProfData to its own header. (PR #140503)
https://github.com/snehasish edited https://github.com/llvm/llvm-project/pull/140503 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Move Radix tree methods to their own header and cpp. (PR #140501)
https://github.com/snehasish created
https://github.com/llvm/llvm-project/pull/140501
None
>From c8e520c48fe9e64f9e2ac389498d0e27797bf362 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 16 May 2025 18:54:05 -0700
Subject: [PATCH] [NFC][MemProf] Move Radix tree methods to their own header
and cpp.
---
llvm/include/llvm/ProfileData/MemProf.h | 336
.../llvm/ProfileData/MemProfRadixTree.h | 358 ++
llvm/include/llvm/ProfileData/MemProfReader.h | 2 +-
llvm/lib/Bitcode/Writer/BitcodeWriter.cpp | 1 +
llvm/lib/ProfileData/CMakeLists.txt | 1 +
llvm/lib/ProfileData/IndexedMemProfData.cpp | 1 +
llvm/lib/ProfileData/InstrProfReader.cpp | 3 +-
llvm/lib/ProfileData/MemProf.cpp | 235
llvm/lib/ProfileData/MemProfRadixTree.cpp | 253 +
llvm/unittests/ProfileData/InstrProfTest.cpp | 1 +
llvm/unittests/ProfileData/MemProfTest.cpp| 3 +-
11 files changed, 620 insertions(+), 574 deletions(-)
create mode 100644 llvm/include/llvm/ProfileData/MemProfRadixTree.h
create mode 100644 llvm/lib/ProfileData/MemProfRadixTree.cpp
diff --git a/llvm/include/llvm/ProfileData/MemProf.h
b/llvm/include/llvm/ProfileData/MemProf.h
index e713c3807611b..0bc1432f7d198 100644
--- a/llvm/include/llvm/ProfileData/MemProf.h
+++ b/llvm/include/llvm/ProfileData/MemProf.h
@@ -818,133 +818,6 @@ class CallStackLookupTrait {
}
};
-namespace detail {
-// "Dereference" the iterator from DenseMap or OnDiskChainedHashTable. We have
-// to do so in one of two different ways depending on the type of the hash
-// table.
-template
-value_type DerefIterator(IterTy Iter) {
- using deref_type = llvm::remove_cvref_t;
- if constexpr (std::is_same_v)
-return *Iter;
- else
-return Iter->second;
-}
-} // namespace detail
-
-// A function object that returns a frame for a given FrameId.
-template struct FrameIdConverter {
- std::optional LastUnmappedId;
- MapTy ⤅
-
- FrameIdConverter() = delete;
- FrameIdConverter(MapTy &Map) : Map(Map) {}
-
- // Delete the copy constructor and copy assignment operator to avoid a
- // situation where a copy of FrameIdConverter gets an error in LastUnmappedId
- // while the original instance doesn't.
- FrameIdConverter(const FrameIdConverter &) = delete;
- FrameIdConverter &operator=(const FrameIdConverter &) = delete;
-
- Frame operator()(FrameId Id) {
-auto Iter = Map.find(Id);
-if (Iter == Map.end()) {
- LastUnmappedId = Id;
- return Frame();
-}
-return detail::DerefIterator(Iter);
- }
-};
-
-// A function object that returns a call stack for a given CallStackId.
-template struct CallStackIdConverter {
- std::optional LastUnmappedId;
- MapTy ⤅
- llvm::function_ref FrameIdToFrame;
-
- CallStackIdConverter() = delete;
- CallStackIdConverter(MapTy &Map,
- llvm::function_ref FrameIdToFrame)
- : Map(Map), FrameIdToFrame(FrameIdToFrame) {}
-
- // Delete the copy constructor and copy assignment operator to avoid a
- // situation where a copy of CallStackIdConverter gets an error in
- // LastUnmappedId while the original instance doesn't.
- CallStackIdConverter(const CallStackIdConverter &) = delete;
- CallStackIdConverter &operator=(const CallStackIdConverter &) = delete;
-
- std::vector operator()(CallStackId CSId) {
-std::vector Frames;
-auto CSIter = Map.find(CSId);
-if (CSIter == Map.end()) {
- LastUnmappedId = CSId;
-} else {
- llvm::SmallVector CS =
- detail::DerefIterator>(CSIter);
- Frames.reserve(CS.size());
- for (FrameId Id : CS)
-Frames.push_back(FrameIdToFrame(Id));
-}
-return Frames;
- }
-};
-
-// A function object that returns a Frame stored at a given index into the
Frame
-// array in the profile.
-struct LinearFrameIdConverter {
- const unsigned char *FrameBase;
-
- LinearFrameIdConverter() = delete;
- LinearFrameIdConverter(const unsigned char *FrameBase)
- : FrameBase(FrameBase) {}
-
- Frame operator()(LinearFrameId LinearId) {
-uint64_t Offset = static_cast(LinearId) *
Frame::serializedSize();
-return Frame::deserialize(FrameBase + Offset);
- }
-};
-
-// A function object that returns a call stack stored at a given index into the
-// call stack array in the profile.
-struct LinearCallStackIdConverter {
- const unsigned char *CallStackBase;
- llvm::function_ref FrameIdToFrame;
-
- LinearCallStackIdConverter() = delete;
- LinearCallStackIdConverter(
- const unsigned char *CallStackBase,
- llvm::function_ref FrameIdToFrame)
- : CallStackBase(CallStackBase), FrameIdToFrame(FrameIdToFrame) {}
-
- std::vector operator()(LinearCallStackId LinearCSId) {
-std::vector Frames;
-
-const unsigned char *Ptr =
-CallStackBase +
-static_cast(LinearCSId) * sizeof(LinearFrameId);
-uint32_t NumFrames =
-support::endian::readNext(Ptr);
-Frames.reserve(NumFrames
[llvm-branch-commits] [llvm] [NFC][MemProf] Move types shared between Analysis, ProfileData and ModuleSummary (Core) to a separate header (PR #140505)
https://github.com/snehasish updated
https://github.com/llvm/llvm-project/pull/140505
>From 8751ae1be821ebc4803bddd05ddb49a84c0b2773 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Mon, 19 May 2025 00:03:59 -0700
Subject: [PATCH] [NFC][MemProf] Move types shared between Analysis,
ProfileData and ModuleSummary (Core) to a separate header
---
.../include/llvm/Analysis/MemoryProfileInfo.h | 3 +-
llvm/include/llvm/IR/ModuleSummaryIndex.h | 22 +-
llvm/include/llvm/ProfileData/MemProfCommon.h | 43 +++
llvm/lib/ProfileData/CMakeLists.txt | 1 -
.../Instrumentation/MemProfiler.cpp | 1 +
5 files changed, 47 insertions(+), 23 deletions(-)
create mode 100644 llvm/include/llvm/ProfileData/MemProfCommon.h
diff --git a/llvm/include/llvm/Analysis/MemoryProfileInfo.h
b/llvm/include/llvm/Analysis/MemoryProfileInfo.h
index 1d98f86f50484..9fcb81a0a1b4c 100644
--- a/llvm/include/llvm/Analysis/MemoryProfileInfo.h
+++ b/llvm/include/llvm/Analysis/MemoryProfileInfo.h
@@ -13,8 +13,9 @@
#ifndef LLVM_ANALYSIS_MEMORYPROFILEINFO_H
#define LLVM_ANALYSIS_MEMORYPROFILEINFO_H
+#include "llvm/IR/InstrTypes.h"
#include "llvm/IR/Metadata.h"
-#include "llvm/IR/ModuleSummaryIndex.h"
+#include "llvm/ProfileData/MemProfCommon.h"
#include
namespace llvm {
diff --git a/llvm/include/llvm/IR/ModuleSummaryIndex.h
b/llvm/include/llvm/IR/ModuleSummaryIndex.h
index 65e428a3adea7..23f9504b44fab 100644
--- a/llvm/include/llvm/IR/ModuleSummaryIndex.h
+++ b/llvm/include/llvm/IR/ModuleSummaryIndex.h
@@ -27,6 +27,7 @@
#include "llvm/IR/ConstantRange.h"
#include "llvm/IR/GlobalValue.h"
#include "llvm/IR/Module.h"
+#include "llvm/ProfileData/MemProfCommon.h"
#include "llvm/Support/Allocator.h"
#include "llvm/Support/Compiler.h"
#include "llvm/Support/InterleavedRange.h"
@@ -306,14 +307,6 @@ template <> struct DenseMapInfo {
static unsigned getHashValue(ValueInfo I) { return hash_value(I.getRef()); }
};
-// For optional hinted size reporting, holds a pair of the full stack id
-// (pre-trimming, from the full context in the profile), and the associated
-// total profiled size.
-struct ContextTotalSize {
- uint64_t FullStackId;
- uint64_t TotalSize;
-};
-
/// Summary of memprof callsite metadata.
struct CallsiteInfo {
// Actual callee function.
@@ -350,19 +343,6 @@ inline raw_ostream &operator<<(raw_ostream &OS, const
CallsiteInfo &SNI) {
return OS;
}
-// Allocation type assigned to an allocation reached by a given context.
-// More can be added, now this is cold, notcold and hot.
-// Values should be powers of two so that they can be ORed, in particular to
-// track allocations that have different behavior with different calling
-// contexts.
-enum class AllocationType : uint8_t {
- None = 0,
- NotCold = 1,
- Cold = 2,
- Hot = 4,
- All = 7 // This should always be set to the OR of all values.
-};
-
/// Summary of a single MIB in a memprof metadata on allocations.
struct MIBInfo {
// The allocation type for this profiled context.
diff --git a/llvm/include/llvm/ProfileData/MemProfCommon.h
b/llvm/include/llvm/ProfileData/MemProfCommon.h
new file mode 100644
index 0..a638824ec000e
--- /dev/null
+++ b/llvm/include/llvm/ProfileData/MemProfCommon.h
@@ -0,0 +1,43 @@
+//===- MemProfCommon.h - MemProf support *- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM
Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+//
+// This file contains common types used by different parts of the MemProf code.
+//
+//===--===//
+
+#ifndef LLVM_PROFILEDATA_MEMPROFCOMMON_H
+#define LLVM_PROFILEDATA_MEMPROFCOMMON_H
+
+#include
+
+namespace llvm {
+
+// For optional hinted size reporting, holds a pair of the full stack id
+// (pre-trimming, from the full context in the profile), and the associated
+// total profiled size.
+struct ContextTotalSize {
+ uint64_t FullStackId;
+ uint64_t TotalSize;
+};
+
+// Allocation type assigned to an allocation reached by a given context.
+// More can be added, now this is cold, notcold and hot.
+// Values should be powers of two so that they can be ORed, in particular to
+// track allocations that have different behavior with different calling
+// contexts.
+enum class AllocationType : uint8_t {
+ None = 0,
+ NotCold = 1,
+ Cold = 2,
+ Hot = 4,
+ All = 7 // This should always be set to the OR of all values.
+};
+
+} // namespace llvm
+
+#endif // LLVM_PROFILEDATA_MEMPROFCOMMON_H
diff --git a/llvm/lib/ProfileData/CMakeLists.txt
b/llvm/lib/ProfileData/CMakeLists.txt
index ca9ea3205ee1d..de60a655d5bd5 100644
--- a/llvm/lib/ProfileData/CMakeLists.txt
+++ b/llvm/lib/ProfileData/CMakeLists.txt
@@ -26,7 +26,6 @@ add_llvm_component_library(LLV
[llvm-branch-commits] [llvm] [NFC][MemProf] Move getGUID out of IndexedMemProfRecord (PR #140502)
https://github.com/snehasish created
https://github.com/llvm/llvm-project/pull/140502
None
>From 83ff2babe9dc5e71bc01eefb0aa78e6634d25351 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 16 May 2025 20:20:00 -0700
Subject: [PATCH] [NFC][MemProf] Move getGUID out of IndexedMemProfRecord
---
llvm/include/llvm/ProfileData/MemProf.h | 10 ++--
llvm/include/llvm/ProfileData/MemProfYAML.h | 2 +-
llvm/lib/ProfileData/MemProf.cpp | 2 +-
llvm/lib/ProfileData/MemProfReader.cpp| 2 +-
.../Instrumentation/MemProfiler.cpp | 4 +-
llvm/unittests/ProfileData/MemProfTest.cpp| 20
.../Instrumentation/MemProfUseTest.cpp| 48 +--
7 files changed, 44 insertions(+), 44 deletions(-)
diff --git a/llvm/include/llvm/ProfileData/MemProf.h
b/llvm/include/llvm/ProfileData/MemProf.h
index 0bc1432f7d198..215102c131fff 100644
--- a/llvm/include/llvm/ProfileData/MemProf.h
+++ b/llvm/include/llvm/ProfileData/MemProf.h
@@ -472,13 +472,13 @@ struct IndexedMemProfRecord {
// translate CallStackId to call stacks with frames inline.
MemProfRecord toMemProfRecord(
llvm::function_ref(const CallStackId)> Callback)
const;
-
- // Returns the GUID for the function name after canonicalization. For
- // memprof, we remove any .llvm suffix added by LTO. MemProfRecords are
- // mapped to functions using this GUID.
- static GlobalValue::GUID getGUID(const StringRef FunctionName);
};
+// Returns the GUID for the function name after canonicalization. For
+// memprof, we remove any .llvm suffix added by LTO. MemProfRecords are
+// mapped to functions using this GUID.
+GlobalValue::GUID getGUID(const StringRef FunctionName);
+
// Holds call site information with frame contents inline.
struct CallSiteInfo {
// The frames in the call stack
diff --git a/llvm/include/llvm/ProfileData/MemProfYAML.h
b/llvm/include/llvm/ProfileData/MemProfYAML.h
index 08dee253f615a..b642e3098aa0e 100644
--- a/llvm/include/llvm/ProfileData/MemProfYAML.h
+++ b/llvm/include/llvm/ProfileData/MemProfYAML.h
@@ -46,7 +46,7 @@ template <> struct ScalarTraits {
Val = Num;
} else {
// Otherwise, treat the input as a string containing a function name.
- Val = memprof::IndexedMemProfRecord::getGUID(Scalar);
+ Val = memprof::getGUID(Scalar);
}
return StringRef();
}
diff --git a/llvm/lib/ProfileData/MemProf.cpp b/llvm/lib/ProfileData/MemProf.cpp
index a9c5ee09a6daf..795e97bee38f5 100644
--- a/llvm/lib/ProfileData/MemProf.cpp
+++ b/llvm/lib/ProfileData/MemProf.cpp
@@ -343,7 +343,7 @@ MemProfRecord IndexedMemProfRecord::toMemProfRecord(
return Record;
}
-GlobalValue::GUID IndexedMemProfRecord::getGUID(const StringRef FunctionName) {
+GlobalValue::GUID getGUID(const StringRef FunctionName) {
// Canonicalize the function name to drop suffixes such as ".llvm.". Note
// we do not drop any ".__uniq." suffixes, as getCanonicalFnName does not
drop
// those by default. This is by design to differentiate internal linkage
diff --git a/llvm/lib/ProfileData/MemProfReader.cpp
b/llvm/lib/ProfileData/MemProfReader.cpp
index e0f280b9eb2f6..aca534b0a4c98 100644
--- a/llvm/lib/ProfileData/MemProfReader.cpp
+++ b/llvm/lib/ProfileData/MemProfReader.cpp
@@ -570,7 +570,7 @@ Error RawMemProfReader::symbolizeAndFilterStackFrames(
I++) {
const auto &DIFrame = DI.getFrame(I);
const uint64_t Guid =
-IndexedMemProfRecord::getGUID(DIFrame.FunctionName);
+memprof::getGUID(DIFrame.FunctionName);
const Frame F(Guid, DIFrame.Line - DIFrame.StartLine, DIFrame.Column,
// Only the last entry is not an inlined location.
I != NumFrames - 1);
diff --git a/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp
b/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp
index 375ff84f82ed2..5982476f3994e 100644
--- a/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp
+++ b/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp
@@ -865,8 +865,8 @@ memprof::extractCallsFromIR(Module &M, const
TargetLibraryInfo &TLI,
StringRef CallerName = DIL->getSubprogramLinkageName();
assert(!CallerName.empty() &&
"Be sure to enable -fdebug-info-for-profiling");
- uint64_t CallerGUID = IndexedMemProfRecord::getGUID(CallerName);
- uint64_t CalleeGUID = IndexedMemProfRecord::getGUID(CalleeName);
+ uint64_t CallerGUID = memprof::getGUID(CallerName);
+ uint64_t CalleeGUID = memprof::getGUID(CalleeName);
// Pretend that we are calling a function with GUID == 0 if we are
// in the inline stack leading to a heap allocation function.
if (IsAlloc) {
diff --git a/llvm/unittests/ProfileData/MemProfTest.cpp
b/llvm/unittests/ProfileData/MemProfTest.cpp
index a072dee26d9a0..2ae9cd96f0197 100644
--- a/llvm/unittests/ProfileData/MemProfTest.cpp
+++ b/llvm/unittests/ProfileData/Mem
[llvm-branch-commits] [llvm] [NFC][MemProf] Move types shared between Analysis, ProfileData and ModuleSummary (Core) to a separate header (PR #140505)
https://github.com/snehasish edited https://github.com/llvm/llvm-project/pull/140505 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Move Radix tree methods to their own header and cpp. (PR #140501)
snehasish wrote: > [!WARNING] > This pull request is not mergeable via GitHub because a downstack PR is > open. Once all requirements are satisfied, merge this PR as a stack href="https://app.graphite.dev/github/pr/llvm/llvm-project/140501?utm_source=stack-comment-downstack-mergeability-warning"; > >on Graphite. > https://graphite.dev/docs/merge-pull-requests";>Learn more * **#140505** https://app.graphite.dev/github/pr/llvm/llvm-project/140505?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140504** https://app.graphite.dev/github/pr/llvm/llvm-project/140504?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140503** https://app.graphite.dev/github/pr/llvm/llvm-project/140503?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140502** https://app.graphite.dev/github/pr/llvm/llvm-project/140502?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * **#140501** https://app.graphite.dev/github/pr/llvm/llvm-project/140501?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> 👈 https://app.graphite.dev/github/pr/llvm/llvm-project/140501?utm_source=stack-comment-view-in-graphite"; target="_blank">(View in Graphite) * **#140500** https://app.graphite.dev/github/pr/llvm/llvm-project/140500?utm_source=stack-comment-icon"; target="_blank">https://static.graphite.dev/graphite-32x32-black.png"; alt="Graphite" width="10px" height="10px"/> * `main` This stack of pull requests is managed by https://graphite.dev?utm-source=stack-comment";>Graphite. Learn more about https://stacking.dev/?utm_source=stack-comment";>stacking. https://github.com/llvm/llvm-project/pull/140501 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Move types shared between Analysis, ProfileData and ModuleSummary (Core) to a separate header (PR #140505)
@@ -46,6 +46,7 @@ #include "llvm/Transforms/Utils/ModuleUtils.h" #include #include +#include snehasish wrote: Yes, we need it for the `LocHashToCallSites` type but we were including it via ModuleSummaryIndex.h. https://github.com/llvm/llvm-project/pull/140505 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Move types shared between Analysis, ProfileData and ModuleSummary (Core) to a separate header (PR #140505)
https://github.com/snehasish updated
https://github.com/llvm/llvm-project/pull/140505
>From 305e2bdbab27828633afb3d1e2698002f7ccadda Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Mon, 19 May 2025 00:03:59 -0700
Subject: [PATCH] [NFC][MemProf] Move types shared between Analysis,
ProfileData and ModuleSummary (Core) to a separate header
---
.../include/llvm/Analysis/MemoryProfileInfo.h | 3 +-
llvm/include/llvm/IR/ModuleSummaryIndex.h | 22 +-
llvm/include/llvm/ProfileData/MemProfCommon.h | 43 +++
llvm/lib/ProfileData/CMakeLists.txt | 1 -
.../Instrumentation/MemProfiler.cpp | 1 +
5 files changed, 47 insertions(+), 23 deletions(-)
create mode 100644 llvm/include/llvm/ProfileData/MemProfCommon.h
diff --git a/llvm/include/llvm/Analysis/MemoryProfileInfo.h
b/llvm/include/llvm/Analysis/MemoryProfileInfo.h
index 1d98f86f50484..9fcb81a0a1b4c 100644
--- a/llvm/include/llvm/Analysis/MemoryProfileInfo.h
+++ b/llvm/include/llvm/Analysis/MemoryProfileInfo.h
@@ -13,8 +13,9 @@
#ifndef LLVM_ANALYSIS_MEMORYPROFILEINFO_H
#define LLVM_ANALYSIS_MEMORYPROFILEINFO_H
+#include "llvm/IR/InstrTypes.h"
#include "llvm/IR/Metadata.h"
-#include "llvm/IR/ModuleSummaryIndex.h"
+#include "llvm/ProfileData/MemProfCommon.h"
#include
namespace llvm {
diff --git a/llvm/include/llvm/IR/ModuleSummaryIndex.h
b/llvm/include/llvm/IR/ModuleSummaryIndex.h
index 65e428a3adea7..23f9504b44fab 100644
--- a/llvm/include/llvm/IR/ModuleSummaryIndex.h
+++ b/llvm/include/llvm/IR/ModuleSummaryIndex.h
@@ -27,6 +27,7 @@
#include "llvm/IR/ConstantRange.h"
#include "llvm/IR/GlobalValue.h"
#include "llvm/IR/Module.h"
+#include "llvm/ProfileData/MemProfCommon.h"
#include "llvm/Support/Allocator.h"
#include "llvm/Support/Compiler.h"
#include "llvm/Support/InterleavedRange.h"
@@ -306,14 +307,6 @@ template <> struct DenseMapInfo {
static unsigned getHashValue(ValueInfo I) { return hash_value(I.getRef()); }
};
-// For optional hinted size reporting, holds a pair of the full stack id
-// (pre-trimming, from the full context in the profile), and the associated
-// total profiled size.
-struct ContextTotalSize {
- uint64_t FullStackId;
- uint64_t TotalSize;
-};
-
/// Summary of memprof callsite metadata.
struct CallsiteInfo {
// Actual callee function.
@@ -350,19 +343,6 @@ inline raw_ostream &operator<<(raw_ostream &OS, const
CallsiteInfo &SNI) {
return OS;
}
-// Allocation type assigned to an allocation reached by a given context.
-// More can be added, now this is cold, notcold and hot.
-// Values should be powers of two so that they can be ORed, in particular to
-// track allocations that have different behavior with different calling
-// contexts.
-enum class AllocationType : uint8_t {
- None = 0,
- NotCold = 1,
- Cold = 2,
- Hot = 4,
- All = 7 // This should always be set to the OR of all values.
-};
-
/// Summary of a single MIB in a memprof metadata on allocations.
struct MIBInfo {
// The allocation type for this profiled context.
diff --git a/llvm/include/llvm/ProfileData/MemProfCommon.h
b/llvm/include/llvm/ProfileData/MemProfCommon.h
new file mode 100644
index 0..a638824ec000e
--- /dev/null
+++ b/llvm/include/llvm/ProfileData/MemProfCommon.h
@@ -0,0 +1,43 @@
+//===- MemProfCommon.h - MemProf support *- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM
Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+//
+// This file contains common types used by different parts of the MemProf code.
+//
+//===--===//
+
+#ifndef LLVM_PROFILEDATA_MEMPROFCOMMON_H
+#define LLVM_PROFILEDATA_MEMPROFCOMMON_H
+
+#include
+
+namespace llvm {
+
+// For optional hinted size reporting, holds a pair of the full stack id
+// (pre-trimming, from the full context in the profile), and the associated
+// total profiled size.
+struct ContextTotalSize {
+ uint64_t FullStackId;
+ uint64_t TotalSize;
+};
+
+// Allocation type assigned to an allocation reached by a given context.
+// More can be added, now this is cold, notcold and hot.
+// Values should be powers of two so that they can be ORed, in particular to
+// track allocations that have different behavior with different calling
+// contexts.
+enum class AllocationType : uint8_t {
+ None = 0,
+ NotCold = 1,
+ Cold = 2,
+ Hot = 4,
+ All = 7 // This should always be set to the OR of all values.
+};
+
+} // namespace llvm
+
+#endif // LLVM_PROFILEDATA_MEMPROFCOMMON_H
diff --git a/llvm/lib/ProfileData/CMakeLists.txt
b/llvm/lib/ProfileData/CMakeLists.txt
index ca9ea3205ee1d..de60a655d5bd5 100644
--- a/llvm/lib/ProfileData/CMakeLists.txt
+++ b/llvm/lib/ProfileData/CMakeLists.txt
@@ -26,7 +26,6 @@ add_llvm_component_library(LLV
[llvm-branch-commits] [llvm] [NFC][MemProf] Move getGUID out of IndexedMemProfRecord (PR #140502)
https://github.com/snehasish updated
https://github.com/llvm/llvm-project/pull/140502
>From 9028dc98ac740d72c9c6ad02e4503da5e9c02a13 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 16 May 2025 20:20:00 -0700
Subject: [PATCH 1/2] [NFC][MemProf] Move getGUID out of IndexedMemProfRecord
---
llvm/include/llvm/ProfileData/MemProf.h | 10 ++--
llvm/include/llvm/ProfileData/MemProfYAML.h | 2 +-
llvm/lib/ProfileData/MemProf.cpp | 2 +-
llvm/lib/ProfileData/MemProfReader.cpp| 2 +-
.../Instrumentation/MemProfiler.cpp | 4 +-
llvm/unittests/ProfileData/MemProfTest.cpp| 20
.../Instrumentation/MemProfUseTest.cpp| 48 +--
7 files changed, 44 insertions(+), 44 deletions(-)
diff --git a/llvm/include/llvm/ProfileData/MemProf.h
b/llvm/include/llvm/ProfileData/MemProf.h
index 0bc1432f7d198..215102c131fff 100644
--- a/llvm/include/llvm/ProfileData/MemProf.h
+++ b/llvm/include/llvm/ProfileData/MemProf.h
@@ -472,13 +472,13 @@ struct IndexedMemProfRecord {
// translate CallStackId to call stacks with frames inline.
MemProfRecord toMemProfRecord(
llvm::function_ref(const CallStackId)> Callback)
const;
-
- // Returns the GUID for the function name after canonicalization. For
- // memprof, we remove any .llvm suffix added by LTO. MemProfRecords are
- // mapped to functions using this GUID.
- static GlobalValue::GUID getGUID(const StringRef FunctionName);
};
+// Returns the GUID for the function name after canonicalization. For
+// memprof, we remove any .llvm suffix added by LTO. MemProfRecords are
+// mapped to functions using this GUID.
+GlobalValue::GUID getGUID(const StringRef FunctionName);
+
// Holds call site information with frame contents inline.
struct CallSiteInfo {
// The frames in the call stack
diff --git a/llvm/include/llvm/ProfileData/MemProfYAML.h
b/llvm/include/llvm/ProfileData/MemProfYAML.h
index 08dee253f615a..b642e3098aa0e 100644
--- a/llvm/include/llvm/ProfileData/MemProfYAML.h
+++ b/llvm/include/llvm/ProfileData/MemProfYAML.h
@@ -46,7 +46,7 @@ template <> struct ScalarTraits {
Val = Num;
} else {
// Otherwise, treat the input as a string containing a function name.
- Val = memprof::IndexedMemProfRecord::getGUID(Scalar);
+ Val = memprof::getGUID(Scalar);
}
return StringRef();
}
diff --git a/llvm/lib/ProfileData/MemProf.cpp b/llvm/lib/ProfileData/MemProf.cpp
index a9c5ee09a6daf..795e97bee38f5 100644
--- a/llvm/lib/ProfileData/MemProf.cpp
+++ b/llvm/lib/ProfileData/MemProf.cpp
@@ -343,7 +343,7 @@ MemProfRecord IndexedMemProfRecord::toMemProfRecord(
return Record;
}
-GlobalValue::GUID IndexedMemProfRecord::getGUID(const StringRef FunctionName) {
+GlobalValue::GUID getGUID(const StringRef FunctionName) {
// Canonicalize the function name to drop suffixes such as ".llvm.". Note
// we do not drop any ".__uniq." suffixes, as getCanonicalFnName does not
drop
// those by default. This is by design to differentiate internal linkage
diff --git a/llvm/lib/ProfileData/MemProfReader.cpp
b/llvm/lib/ProfileData/MemProfReader.cpp
index e0f280b9eb2f6..aca534b0a4c98 100644
--- a/llvm/lib/ProfileData/MemProfReader.cpp
+++ b/llvm/lib/ProfileData/MemProfReader.cpp
@@ -570,7 +570,7 @@ Error RawMemProfReader::symbolizeAndFilterStackFrames(
I++) {
const auto &DIFrame = DI.getFrame(I);
const uint64_t Guid =
-IndexedMemProfRecord::getGUID(DIFrame.FunctionName);
+memprof::getGUID(DIFrame.FunctionName);
const Frame F(Guid, DIFrame.Line - DIFrame.StartLine, DIFrame.Column,
// Only the last entry is not an inlined location.
I != NumFrames - 1);
diff --git a/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp
b/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp
index 375ff84f82ed2..5982476f3994e 100644
--- a/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp
+++ b/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp
@@ -865,8 +865,8 @@ memprof::extractCallsFromIR(Module &M, const
TargetLibraryInfo &TLI,
StringRef CallerName = DIL->getSubprogramLinkageName();
assert(!CallerName.empty() &&
"Be sure to enable -fdebug-info-for-profiling");
- uint64_t CallerGUID = IndexedMemProfRecord::getGUID(CallerName);
- uint64_t CalleeGUID = IndexedMemProfRecord::getGUID(CalleeName);
+ uint64_t CallerGUID = memprof::getGUID(CallerName);
+ uint64_t CalleeGUID = memprof::getGUID(CalleeName);
// Pretend that we are calling a function with GUID == 0 if we are
// in the inline stack leading to a heap allocation function.
if (IsAlloc) {
diff --git a/llvm/unittests/ProfileData/MemProfTest.cpp
b/llvm/unittests/ProfileData/MemProfTest.cpp
index 26b09698c7ea3..201ee2d7272cf 100644
--- a/llvm/unittests/ProfileData/MemProfTest.cpp
+++ b/llvm/unittests/ProfileData/MemPr
[llvm-branch-commits] [llvm] [NFC][MemProf] Add the LLVM license text and minor clean up. (PR #140504)
https://github.com/snehasish updated
https://github.com/llvm/llvm-project/pull/140504
>From 36aeb3a179b64b9631af7849ed08d8cf7c7564e5 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 16 May 2025 23:55:43 -0700
Subject: [PATCH] [NFC][MemProf] Add the LLVM license text and minor clean up.
---
llvm/include/llvm/ProfileData/MemProf.h | 20
1 file changed, 16 insertions(+), 4 deletions(-)
diff --git a/llvm/include/llvm/ProfileData/MemProf.h
b/llvm/include/llvm/ProfileData/MemProf.h
index ce5cd5ee4856b..683193aa42747 100644
--- a/llvm/include/llvm/ProfileData/MemProf.h
+++ b/llvm/include/llvm/ProfileData/MemProf.h
@@ -1,5 +1,18 @@
-#ifndef LLVM_PROFILEDATA_MEMPROF_H_
-#define LLVM_PROFILEDATA_MEMPROF_H_
+//===- MemProf.h - MemProf support --*- C++
-*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM
Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+//
+// This file contains common definitions used in the reading and writing of
+// memory profile data.
+//
+//===--===//
+
+#ifndef LLVM_PROFILEDATA_MEMPROF_H
+#define LLVM_PROFILEDATA_MEMPROF_H
#include "llvm/ADT/BitVector.h"
#include "llvm/ADT/MapVector.h"
@@ -844,5 +857,4 @@ struct LineLocation {
using CallEdgeTy = std::pair;
} // namespace memprof
} // namespace llvm
-
-#endif // LLVM_PROFILEDATA_MEMPROF_H_
+#endif // LLVM_PROFILEDATA_MEMPROF_H
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Add the LLVM license text and minor clean up. (PR #140504)
https://github.com/snehasish updated
https://github.com/llvm/llvm-project/pull/140504
>From 36aeb3a179b64b9631af7849ed08d8cf7c7564e5 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 16 May 2025 23:55:43 -0700
Subject: [PATCH] [NFC][MemProf] Add the LLVM license text and minor clean up.
---
llvm/include/llvm/ProfileData/MemProf.h | 20
1 file changed, 16 insertions(+), 4 deletions(-)
diff --git a/llvm/include/llvm/ProfileData/MemProf.h
b/llvm/include/llvm/ProfileData/MemProf.h
index ce5cd5ee4856b..683193aa42747 100644
--- a/llvm/include/llvm/ProfileData/MemProf.h
+++ b/llvm/include/llvm/ProfileData/MemProf.h
@@ -1,5 +1,18 @@
-#ifndef LLVM_PROFILEDATA_MEMPROF_H_
-#define LLVM_PROFILEDATA_MEMPROF_H_
+//===- MemProf.h - MemProf support --*- C++
-*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM
Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+//
+// This file contains common definitions used in the reading and writing of
+// memory profile data.
+//
+//===--===//
+
+#ifndef LLVM_PROFILEDATA_MEMPROF_H
+#define LLVM_PROFILEDATA_MEMPROF_H
#include "llvm/ADT/BitVector.h"
#include "llvm/ADT/MapVector.h"
@@ -844,5 +857,4 @@ struct LineLocation {
using CallEdgeTy = std::pair;
} // namespace memprof
} // namespace llvm
-
-#endif // LLVM_PROFILEDATA_MEMPROF_H_
+#endif // LLVM_PROFILEDATA_MEMPROF_H
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Move Radix tree methods to their own header and cpp. (PR #140501)
https://github.com/snehasish updated
https://github.com/llvm/llvm-project/pull/140501
>From 178cc0cac2258349b482f9315694f84e9d3581a5 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 16 May 2025 18:54:05 -0700
Subject: [PATCH 1/2] [NFC][MemProf] Move Radix tree methods to their own
header and cpp.
---
llvm/include/llvm/ProfileData/MemProf.h | 336
.../llvm/ProfileData/MemProfRadixTree.h | 358 ++
llvm/include/llvm/ProfileData/MemProfReader.h | 2 +-
llvm/lib/Bitcode/Writer/BitcodeWriter.cpp | 1 +
llvm/lib/ProfileData/CMakeLists.txt | 1 +
llvm/lib/ProfileData/IndexedMemProfData.cpp | 1 +
llvm/lib/ProfileData/InstrProfReader.cpp | 3 +-
llvm/lib/ProfileData/MemProf.cpp | 235
llvm/lib/ProfileData/MemProfRadixTree.cpp | 253 +
llvm/unittests/ProfileData/InstrProfTest.cpp | 1 +
llvm/unittests/ProfileData/MemProfTest.cpp| 3 +-
11 files changed, 620 insertions(+), 574 deletions(-)
create mode 100644 llvm/include/llvm/ProfileData/MemProfRadixTree.h
create mode 100644 llvm/lib/ProfileData/MemProfRadixTree.cpp
diff --git a/llvm/include/llvm/ProfileData/MemProf.h
b/llvm/include/llvm/ProfileData/MemProf.h
index e713c3807611b..0bc1432f7d198 100644
--- a/llvm/include/llvm/ProfileData/MemProf.h
+++ b/llvm/include/llvm/ProfileData/MemProf.h
@@ -818,133 +818,6 @@ class CallStackLookupTrait {
}
};
-namespace detail {
-// "Dereference" the iterator from DenseMap or OnDiskChainedHashTable. We have
-// to do so in one of two different ways depending on the type of the hash
-// table.
-template
-value_type DerefIterator(IterTy Iter) {
- using deref_type = llvm::remove_cvref_t;
- if constexpr (std::is_same_v)
-return *Iter;
- else
-return Iter->second;
-}
-} // namespace detail
-
-// A function object that returns a frame for a given FrameId.
-template struct FrameIdConverter {
- std::optional LastUnmappedId;
- MapTy ⤅
-
- FrameIdConverter() = delete;
- FrameIdConverter(MapTy &Map) : Map(Map) {}
-
- // Delete the copy constructor and copy assignment operator to avoid a
- // situation where a copy of FrameIdConverter gets an error in LastUnmappedId
- // while the original instance doesn't.
- FrameIdConverter(const FrameIdConverter &) = delete;
- FrameIdConverter &operator=(const FrameIdConverter &) = delete;
-
- Frame operator()(FrameId Id) {
-auto Iter = Map.find(Id);
-if (Iter == Map.end()) {
- LastUnmappedId = Id;
- return Frame();
-}
-return detail::DerefIterator(Iter);
- }
-};
-
-// A function object that returns a call stack for a given CallStackId.
-template struct CallStackIdConverter {
- std::optional LastUnmappedId;
- MapTy ⤅
- llvm::function_ref FrameIdToFrame;
-
- CallStackIdConverter() = delete;
- CallStackIdConverter(MapTy &Map,
- llvm::function_ref FrameIdToFrame)
- : Map(Map), FrameIdToFrame(FrameIdToFrame) {}
-
- // Delete the copy constructor and copy assignment operator to avoid a
- // situation where a copy of CallStackIdConverter gets an error in
- // LastUnmappedId while the original instance doesn't.
- CallStackIdConverter(const CallStackIdConverter &) = delete;
- CallStackIdConverter &operator=(const CallStackIdConverter &) = delete;
-
- std::vector operator()(CallStackId CSId) {
-std::vector Frames;
-auto CSIter = Map.find(CSId);
-if (CSIter == Map.end()) {
- LastUnmappedId = CSId;
-} else {
- llvm::SmallVector CS =
- detail::DerefIterator>(CSIter);
- Frames.reserve(CS.size());
- for (FrameId Id : CS)
-Frames.push_back(FrameIdToFrame(Id));
-}
-return Frames;
- }
-};
-
-// A function object that returns a Frame stored at a given index into the
Frame
-// array in the profile.
-struct LinearFrameIdConverter {
- const unsigned char *FrameBase;
-
- LinearFrameIdConverter() = delete;
- LinearFrameIdConverter(const unsigned char *FrameBase)
- : FrameBase(FrameBase) {}
-
- Frame operator()(LinearFrameId LinearId) {
-uint64_t Offset = static_cast(LinearId) *
Frame::serializedSize();
-return Frame::deserialize(FrameBase + Offset);
- }
-};
-
-// A function object that returns a call stack stored at a given index into the
-// call stack array in the profile.
-struct LinearCallStackIdConverter {
- const unsigned char *CallStackBase;
- llvm::function_ref FrameIdToFrame;
-
- LinearCallStackIdConverter() = delete;
- LinearCallStackIdConverter(
- const unsigned char *CallStackBase,
- llvm::function_ref FrameIdToFrame)
- : CallStackBase(CallStackBase), FrameIdToFrame(FrameIdToFrame) {}
-
- std::vector operator()(LinearCallStackId LinearCSId) {
-std::vector Frames;
-
-const unsigned char *Ptr =
-CallStackBase +
-static_cast(LinearCSId) * sizeof(LinearFrameId);
-uint32_t NumFrames =
-support::endian::readNext(Ptr);
-Frames.reserve(NumFrames);
[llvm-branch-commits] [llvm] [NFC][MemProf] Move types shared between Analysis, ProfileData and ModuleSummary (Core) to a separate header (PR #140505)
https://github.com/snehasish updated
https://github.com/llvm/llvm-project/pull/140505
>From 305e2bdbab27828633afb3d1e2698002f7ccadda Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Mon, 19 May 2025 00:03:59 -0700
Subject: [PATCH] [NFC][MemProf] Move types shared between Analysis,
ProfileData and ModuleSummary (Core) to a separate header
---
.../include/llvm/Analysis/MemoryProfileInfo.h | 3 +-
llvm/include/llvm/IR/ModuleSummaryIndex.h | 22 +-
llvm/include/llvm/ProfileData/MemProfCommon.h | 43 +++
llvm/lib/ProfileData/CMakeLists.txt | 1 -
.../Instrumentation/MemProfiler.cpp | 1 +
5 files changed, 47 insertions(+), 23 deletions(-)
create mode 100644 llvm/include/llvm/ProfileData/MemProfCommon.h
diff --git a/llvm/include/llvm/Analysis/MemoryProfileInfo.h
b/llvm/include/llvm/Analysis/MemoryProfileInfo.h
index 1d98f86f50484..9fcb81a0a1b4c 100644
--- a/llvm/include/llvm/Analysis/MemoryProfileInfo.h
+++ b/llvm/include/llvm/Analysis/MemoryProfileInfo.h
@@ -13,8 +13,9 @@
#ifndef LLVM_ANALYSIS_MEMORYPROFILEINFO_H
#define LLVM_ANALYSIS_MEMORYPROFILEINFO_H
+#include "llvm/IR/InstrTypes.h"
#include "llvm/IR/Metadata.h"
-#include "llvm/IR/ModuleSummaryIndex.h"
+#include "llvm/ProfileData/MemProfCommon.h"
#include
namespace llvm {
diff --git a/llvm/include/llvm/IR/ModuleSummaryIndex.h
b/llvm/include/llvm/IR/ModuleSummaryIndex.h
index 65e428a3adea7..23f9504b44fab 100644
--- a/llvm/include/llvm/IR/ModuleSummaryIndex.h
+++ b/llvm/include/llvm/IR/ModuleSummaryIndex.h
@@ -27,6 +27,7 @@
#include "llvm/IR/ConstantRange.h"
#include "llvm/IR/GlobalValue.h"
#include "llvm/IR/Module.h"
+#include "llvm/ProfileData/MemProfCommon.h"
#include "llvm/Support/Allocator.h"
#include "llvm/Support/Compiler.h"
#include "llvm/Support/InterleavedRange.h"
@@ -306,14 +307,6 @@ template <> struct DenseMapInfo {
static unsigned getHashValue(ValueInfo I) { return hash_value(I.getRef()); }
};
-// For optional hinted size reporting, holds a pair of the full stack id
-// (pre-trimming, from the full context in the profile), and the associated
-// total profiled size.
-struct ContextTotalSize {
- uint64_t FullStackId;
- uint64_t TotalSize;
-};
-
/// Summary of memprof callsite metadata.
struct CallsiteInfo {
// Actual callee function.
@@ -350,19 +343,6 @@ inline raw_ostream &operator<<(raw_ostream &OS, const
CallsiteInfo &SNI) {
return OS;
}
-// Allocation type assigned to an allocation reached by a given context.
-// More can be added, now this is cold, notcold and hot.
-// Values should be powers of two so that they can be ORed, in particular to
-// track allocations that have different behavior with different calling
-// contexts.
-enum class AllocationType : uint8_t {
- None = 0,
- NotCold = 1,
- Cold = 2,
- Hot = 4,
- All = 7 // This should always be set to the OR of all values.
-};
-
/// Summary of a single MIB in a memprof metadata on allocations.
struct MIBInfo {
// The allocation type for this profiled context.
diff --git a/llvm/include/llvm/ProfileData/MemProfCommon.h
b/llvm/include/llvm/ProfileData/MemProfCommon.h
new file mode 100644
index 0..a638824ec000e
--- /dev/null
+++ b/llvm/include/llvm/ProfileData/MemProfCommon.h
@@ -0,0 +1,43 @@
+//===- MemProfCommon.h - MemProf support *- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM
Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+//
+// This file contains common types used by different parts of the MemProf code.
+//
+//===--===//
+
+#ifndef LLVM_PROFILEDATA_MEMPROFCOMMON_H
+#define LLVM_PROFILEDATA_MEMPROFCOMMON_H
+
+#include
+
+namespace llvm {
+
+// For optional hinted size reporting, holds a pair of the full stack id
+// (pre-trimming, from the full context in the profile), and the associated
+// total profiled size.
+struct ContextTotalSize {
+ uint64_t FullStackId;
+ uint64_t TotalSize;
+};
+
+// Allocation type assigned to an allocation reached by a given context.
+// More can be added, now this is cold, notcold and hot.
+// Values should be powers of two so that they can be ORed, in particular to
+// track allocations that have different behavior with different calling
+// contexts.
+enum class AllocationType : uint8_t {
+ None = 0,
+ NotCold = 1,
+ Cold = 2,
+ Hot = 4,
+ All = 7 // This should always be set to the OR of all values.
+};
+
+} // namespace llvm
+
+#endif // LLVM_PROFILEDATA_MEMPROFCOMMON_H
diff --git a/llvm/lib/ProfileData/CMakeLists.txt
b/llvm/lib/ProfileData/CMakeLists.txt
index ca9ea3205ee1d..de60a655d5bd5 100644
--- a/llvm/lib/ProfileData/CMakeLists.txt
+++ b/llvm/lib/ProfileData/CMakeLists.txt
@@ -26,7 +26,6 @@ add_llvm_component_library(LLV
[llvm-branch-commits] [llvm] [NFC][MemProf] Move IndexedMemProfData to its own header. (PR #140503)
https://github.com/snehasish updated
https://github.com/llvm/llvm-project/pull/140503
>From 1f2f80df22f154bc317ae9298cf20c9f91574f55 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 16 May 2025 23:41:29 -0700
Subject: [PATCH 1/2] [NFC][MemProf] Move IndexedMemProfData to its own header.
---
.../llvm/ProfileData/IndexedMemProfData.h | 70 ++-
.../llvm/ProfileData/InstrProfWriter.h| 2 +-
llvm/include/llvm/ProfileData/MemProf.h | 51 --
.../llvm/ProfileData/MemProfRadixTree.h | 1 +
llvm/include/llvm/ProfileData/MemProfReader.h | 1 +
llvm/lib/Bitcode/Writer/BitcodeWriter.cpp | 1 +
llvm/lib/ProfileData/InstrProfWriter.cpp | 1 -
llvm/lib/ProfileData/MemProf.cpp | 13
llvm/unittests/ProfileData/InstrProfTest.cpp | 1 +
llvm/unittests/ProfileData/MemProfTest.cpp| 4 +-
.../Instrumentation/MemProfUseTest.cpp| 1 +
11 files changed, 76 insertions(+), 70 deletions(-)
diff --git a/llvm/include/llvm/ProfileData/IndexedMemProfData.h
b/llvm/include/llvm/ProfileData/IndexedMemProfData.h
index 3c6c329d1c49d..94a16227477cb 100644
--- a/llvm/include/llvm/ProfileData/IndexedMemProfData.h
+++ b/llvm/include/llvm/ProfileData/IndexedMemProfData.h
@@ -6,18 +6,84 @@
//
//===--===//
//
-// MemProf data is serialized in writeMemProf provided in this header file.
+// This file implements IndexedMemProfData, a data structure to hold MemProf
+// in a space optimized format. It also provides utility methods for writing
+// MemProf data.
//
//===--===//
+#ifndef LLVM_PROFILEDATA_INDEXEDMEMPROFDATA_H
+#define LLVM_PROFILEDATA_INDEXEDMEMPROFDATA_H
+
#include "llvm/ProfileData/InstrProf.h"
#include "llvm/ProfileData/MemProf.h"
namespace llvm {
+namespace memprof {
+struct IndexedMemProfData {
+ // A map to hold memprof data per function. The lower 64 bits obtained from
+ // the md5 hash of the function name is used to index into the map.
+ llvm::MapVector Records;
+
+ // A map to hold frame id to frame mappings. The mappings are used to
+ // convert IndexedMemProfRecord to MemProfRecords with frame information
+ // inline.
+ llvm::MapVector Frames;
+
+ // A map to hold call stack id to call stacks.
+ llvm::MapVector> CallStacks;
+
+ FrameId addFrame(const Frame &F) {
+const FrameId Id = hashFrame(F);
+Frames.try_emplace(Id, F);
+return Id;
+ }
+
+ CallStackId addCallStack(ArrayRef CS) {
+CallStackId CSId = hashCallStack(CS);
+CallStacks.try_emplace(CSId, CS);
+return CSId;
+ }
+
+ CallStackId addCallStack(SmallVector &&CS) {
+CallStackId CSId = hashCallStack(CS);
+CallStacks.try_emplace(CSId, std::move(CS));
+return CSId;
+ }
+
+private:
+ // Return a hash value based on the contents of the frame. Here we use a
+ // cryptographic hash function to minimize the chance of hash collisions. We
+ // do persist FrameIds as part of memprof formats up to Version 2, inclusive.
+ // However, the deserializer never calls this function; it uses FrameIds
+ // merely as keys to look up Frames proper.
+ FrameId hashFrame(const Frame &F) const {
+llvm::HashBuilder, llvm::endianness::little>
+HashBuilder;
+HashBuilder.add(F.Function, F.LineOffset, F.Column, F.IsInlineFrame);
+llvm::BLAKE3Result<8> Hash = HashBuilder.final();
+FrameId Id;
+std::memcpy(&Id, Hash.data(), sizeof(Hash));
+return Id;
+ }
+
+ // Compute a CallStackId for a given call stack.
+ CallStackId hashCallStack(ArrayRef CS) const {
+ llvm::HashBuilder, llvm::endianness::little>
+ HashBuilder;
+ for (FrameId F : CS)
+HashBuilder.add(F);
+ llvm::BLAKE3Result<8> Hash = HashBuilder.final();
+ CallStackId CSId;
+ std::memcpy(&CSId, Hash.data(), sizeof(Hash));
+ return CSId;
+}
+};
+} // namespace memprof
// Write the MemProf data to OS.
Error writeMemProf(ProfOStream &OS, memprof::IndexedMemProfData &MemProfData,
memprof::IndexedVersion MemProfVersionRequested,
bool MemProfFullSchema);
-
} // namespace llvm
+#endif
diff --git a/llvm/include/llvm/ProfileData/InstrProfWriter.h
b/llvm/include/llvm/ProfileData/InstrProfWriter.h
index 67d85daa81623..16d2ef3fab3e3 100644
--- a/llvm/include/llvm/ProfileData/InstrProfWriter.h
+++ b/llvm/include/llvm/ProfileData/InstrProfWriter.h
@@ -20,7 +20,7 @@
#include "llvm/IR/GlobalValue.h"
#include "llvm/Object/BuildID.h"
#include "llvm/ProfileData/InstrProf.h"
-#include "llvm/ProfileData/MemProf.h"
+#include "llvm/ProfileData/IndexedMemProfData.h"
#include "llvm/Support/Error.h"
#include
#include
diff --git a/llvm/include/llvm/ProfileData/MemProf.h
b/llvm/include/llvm/ProfileData/MemProf.h
index 215102c131fff..ce5cd5ee4856b 100644
--- a/llvm/include/llvm/ProfileData/MemProf.h
+++ b/llvm/include/llvm/ProfileData/MemProf.h
@@ -842,57 +842,
[llvm-branch-commits] [llvm] [NFC][MemProf] Move IndexedMemProfData to its own header. (PR #140503)
https://github.com/snehasish updated
https://github.com/llvm/llvm-project/pull/140503
>From 1f2f80df22f154bc317ae9298cf20c9f91574f55 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 16 May 2025 23:41:29 -0700
Subject: [PATCH 1/2] [NFC][MemProf] Move IndexedMemProfData to its own header.
---
.../llvm/ProfileData/IndexedMemProfData.h | 70 ++-
.../llvm/ProfileData/InstrProfWriter.h| 2 +-
llvm/include/llvm/ProfileData/MemProf.h | 51 --
.../llvm/ProfileData/MemProfRadixTree.h | 1 +
llvm/include/llvm/ProfileData/MemProfReader.h | 1 +
llvm/lib/Bitcode/Writer/BitcodeWriter.cpp | 1 +
llvm/lib/ProfileData/InstrProfWriter.cpp | 1 -
llvm/lib/ProfileData/MemProf.cpp | 13
llvm/unittests/ProfileData/InstrProfTest.cpp | 1 +
llvm/unittests/ProfileData/MemProfTest.cpp| 4 +-
.../Instrumentation/MemProfUseTest.cpp| 1 +
11 files changed, 76 insertions(+), 70 deletions(-)
diff --git a/llvm/include/llvm/ProfileData/IndexedMemProfData.h
b/llvm/include/llvm/ProfileData/IndexedMemProfData.h
index 3c6c329d1c49d..94a16227477cb 100644
--- a/llvm/include/llvm/ProfileData/IndexedMemProfData.h
+++ b/llvm/include/llvm/ProfileData/IndexedMemProfData.h
@@ -6,18 +6,84 @@
//
//===--===//
//
-// MemProf data is serialized in writeMemProf provided in this header file.
+// This file implements IndexedMemProfData, a data structure to hold MemProf
+// in a space optimized format. It also provides utility methods for writing
+// MemProf data.
//
//===--===//
+#ifndef LLVM_PROFILEDATA_INDEXEDMEMPROFDATA_H
+#define LLVM_PROFILEDATA_INDEXEDMEMPROFDATA_H
+
#include "llvm/ProfileData/InstrProf.h"
#include "llvm/ProfileData/MemProf.h"
namespace llvm {
+namespace memprof {
+struct IndexedMemProfData {
+ // A map to hold memprof data per function. The lower 64 bits obtained from
+ // the md5 hash of the function name is used to index into the map.
+ llvm::MapVector Records;
+
+ // A map to hold frame id to frame mappings. The mappings are used to
+ // convert IndexedMemProfRecord to MemProfRecords with frame information
+ // inline.
+ llvm::MapVector Frames;
+
+ // A map to hold call stack id to call stacks.
+ llvm::MapVector> CallStacks;
+
+ FrameId addFrame(const Frame &F) {
+const FrameId Id = hashFrame(F);
+Frames.try_emplace(Id, F);
+return Id;
+ }
+
+ CallStackId addCallStack(ArrayRef CS) {
+CallStackId CSId = hashCallStack(CS);
+CallStacks.try_emplace(CSId, CS);
+return CSId;
+ }
+
+ CallStackId addCallStack(SmallVector &&CS) {
+CallStackId CSId = hashCallStack(CS);
+CallStacks.try_emplace(CSId, std::move(CS));
+return CSId;
+ }
+
+private:
+ // Return a hash value based on the contents of the frame. Here we use a
+ // cryptographic hash function to minimize the chance of hash collisions. We
+ // do persist FrameIds as part of memprof formats up to Version 2, inclusive.
+ // However, the deserializer never calls this function; it uses FrameIds
+ // merely as keys to look up Frames proper.
+ FrameId hashFrame(const Frame &F) const {
+llvm::HashBuilder, llvm::endianness::little>
+HashBuilder;
+HashBuilder.add(F.Function, F.LineOffset, F.Column, F.IsInlineFrame);
+llvm::BLAKE3Result<8> Hash = HashBuilder.final();
+FrameId Id;
+std::memcpy(&Id, Hash.data(), sizeof(Hash));
+return Id;
+ }
+
+ // Compute a CallStackId for a given call stack.
+ CallStackId hashCallStack(ArrayRef CS) const {
+ llvm::HashBuilder, llvm::endianness::little>
+ HashBuilder;
+ for (FrameId F : CS)
+HashBuilder.add(F);
+ llvm::BLAKE3Result<8> Hash = HashBuilder.final();
+ CallStackId CSId;
+ std::memcpy(&CSId, Hash.data(), sizeof(Hash));
+ return CSId;
+}
+};
+} // namespace memprof
// Write the MemProf data to OS.
Error writeMemProf(ProfOStream &OS, memprof::IndexedMemProfData &MemProfData,
memprof::IndexedVersion MemProfVersionRequested,
bool MemProfFullSchema);
-
} // namespace llvm
+#endif
diff --git a/llvm/include/llvm/ProfileData/InstrProfWriter.h
b/llvm/include/llvm/ProfileData/InstrProfWriter.h
index 67d85daa81623..16d2ef3fab3e3 100644
--- a/llvm/include/llvm/ProfileData/InstrProfWriter.h
+++ b/llvm/include/llvm/ProfileData/InstrProfWriter.h
@@ -20,7 +20,7 @@
#include "llvm/IR/GlobalValue.h"
#include "llvm/Object/BuildID.h"
#include "llvm/ProfileData/InstrProf.h"
-#include "llvm/ProfileData/MemProf.h"
+#include "llvm/ProfileData/IndexedMemProfData.h"
#include "llvm/Support/Error.h"
#include
#include
diff --git a/llvm/include/llvm/ProfileData/MemProf.h
b/llvm/include/llvm/ProfileData/MemProf.h
index 215102c131fff..ce5cd5ee4856b 100644
--- a/llvm/include/llvm/ProfileData/MemProf.h
+++ b/llvm/include/llvm/ProfileData/MemProf.h
@@ -842,57 +842,
[llvm-branch-commits] [llvm] [NFC][MemProf] Move getGUID out of IndexedMemProfRecord (PR #140502)
https://github.com/snehasish updated
https://github.com/llvm/llvm-project/pull/140502
>From 9028dc98ac740d72c9c6ad02e4503da5e9c02a13 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 16 May 2025 20:20:00 -0700
Subject: [PATCH 1/2] [NFC][MemProf] Move getGUID out of IndexedMemProfRecord
---
llvm/include/llvm/ProfileData/MemProf.h | 10 ++--
llvm/include/llvm/ProfileData/MemProfYAML.h | 2 +-
llvm/lib/ProfileData/MemProf.cpp | 2 +-
llvm/lib/ProfileData/MemProfReader.cpp| 2 +-
.../Instrumentation/MemProfiler.cpp | 4 +-
llvm/unittests/ProfileData/MemProfTest.cpp| 20
.../Instrumentation/MemProfUseTest.cpp| 48 +--
7 files changed, 44 insertions(+), 44 deletions(-)
diff --git a/llvm/include/llvm/ProfileData/MemProf.h
b/llvm/include/llvm/ProfileData/MemProf.h
index 0bc1432f7d198..215102c131fff 100644
--- a/llvm/include/llvm/ProfileData/MemProf.h
+++ b/llvm/include/llvm/ProfileData/MemProf.h
@@ -472,13 +472,13 @@ struct IndexedMemProfRecord {
// translate CallStackId to call stacks with frames inline.
MemProfRecord toMemProfRecord(
llvm::function_ref(const CallStackId)> Callback)
const;
-
- // Returns the GUID for the function name after canonicalization. For
- // memprof, we remove any .llvm suffix added by LTO. MemProfRecords are
- // mapped to functions using this GUID.
- static GlobalValue::GUID getGUID(const StringRef FunctionName);
};
+// Returns the GUID for the function name after canonicalization. For
+// memprof, we remove any .llvm suffix added by LTO. MemProfRecords are
+// mapped to functions using this GUID.
+GlobalValue::GUID getGUID(const StringRef FunctionName);
+
// Holds call site information with frame contents inline.
struct CallSiteInfo {
// The frames in the call stack
diff --git a/llvm/include/llvm/ProfileData/MemProfYAML.h
b/llvm/include/llvm/ProfileData/MemProfYAML.h
index 08dee253f615a..b642e3098aa0e 100644
--- a/llvm/include/llvm/ProfileData/MemProfYAML.h
+++ b/llvm/include/llvm/ProfileData/MemProfYAML.h
@@ -46,7 +46,7 @@ template <> struct ScalarTraits {
Val = Num;
} else {
// Otherwise, treat the input as a string containing a function name.
- Val = memprof::IndexedMemProfRecord::getGUID(Scalar);
+ Val = memprof::getGUID(Scalar);
}
return StringRef();
}
diff --git a/llvm/lib/ProfileData/MemProf.cpp b/llvm/lib/ProfileData/MemProf.cpp
index a9c5ee09a6daf..795e97bee38f5 100644
--- a/llvm/lib/ProfileData/MemProf.cpp
+++ b/llvm/lib/ProfileData/MemProf.cpp
@@ -343,7 +343,7 @@ MemProfRecord IndexedMemProfRecord::toMemProfRecord(
return Record;
}
-GlobalValue::GUID IndexedMemProfRecord::getGUID(const StringRef FunctionName) {
+GlobalValue::GUID getGUID(const StringRef FunctionName) {
// Canonicalize the function name to drop suffixes such as ".llvm.". Note
// we do not drop any ".__uniq." suffixes, as getCanonicalFnName does not
drop
// those by default. This is by design to differentiate internal linkage
diff --git a/llvm/lib/ProfileData/MemProfReader.cpp
b/llvm/lib/ProfileData/MemProfReader.cpp
index e0f280b9eb2f6..aca534b0a4c98 100644
--- a/llvm/lib/ProfileData/MemProfReader.cpp
+++ b/llvm/lib/ProfileData/MemProfReader.cpp
@@ -570,7 +570,7 @@ Error RawMemProfReader::symbolizeAndFilterStackFrames(
I++) {
const auto &DIFrame = DI.getFrame(I);
const uint64_t Guid =
-IndexedMemProfRecord::getGUID(DIFrame.FunctionName);
+memprof::getGUID(DIFrame.FunctionName);
const Frame F(Guid, DIFrame.Line - DIFrame.StartLine, DIFrame.Column,
// Only the last entry is not an inlined location.
I != NumFrames - 1);
diff --git a/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp
b/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp
index 375ff84f82ed2..5982476f3994e 100644
--- a/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp
+++ b/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp
@@ -865,8 +865,8 @@ memprof::extractCallsFromIR(Module &M, const
TargetLibraryInfo &TLI,
StringRef CallerName = DIL->getSubprogramLinkageName();
assert(!CallerName.empty() &&
"Be sure to enable -fdebug-info-for-profiling");
- uint64_t CallerGUID = IndexedMemProfRecord::getGUID(CallerName);
- uint64_t CalleeGUID = IndexedMemProfRecord::getGUID(CalleeName);
+ uint64_t CallerGUID = memprof::getGUID(CallerName);
+ uint64_t CalleeGUID = memprof::getGUID(CalleeName);
// Pretend that we are calling a function with GUID == 0 if we are
// in the inline stack leading to a heap allocation function.
if (IsAlloc) {
diff --git a/llvm/unittests/ProfileData/MemProfTest.cpp
b/llvm/unittests/ProfileData/MemProfTest.cpp
index 26b09698c7ea3..201ee2d7272cf 100644
--- a/llvm/unittests/ProfileData/MemProfTest.cpp
+++ b/llvm/unittests/ProfileData/MemPr
[llvm-branch-commits] [llvm] [NFC][MemProf] Move Radix tree methods to their own header and cpp. (PR #140501)
https://github.com/snehasish updated
https://github.com/llvm/llvm-project/pull/140501
>From 178cc0cac2258349b482f9315694f84e9d3581a5 Mon Sep 17 00:00:00 2001
From: Snehasish Kumar
Date: Fri, 16 May 2025 18:54:05 -0700
Subject: [PATCH 1/2] [NFC][MemProf] Move Radix tree methods to their own
header and cpp.
---
llvm/include/llvm/ProfileData/MemProf.h | 336
.../llvm/ProfileData/MemProfRadixTree.h | 358 ++
llvm/include/llvm/ProfileData/MemProfReader.h | 2 +-
llvm/lib/Bitcode/Writer/BitcodeWriter.cpp | 1 +
llvm/lib/ProfileData/CMakeLists.txt | 1 +
llvm/lib/ProfileData/IndexedMemProfData.cpp | 1 +
llvm/lib/ProfileData/InstrProfReader.cpp | 3 +-
llvm/lib/ProfileData/MemProf.cpp | 235
llvm/lib/ProfileData/MemProfRadixTree.cpp | 253 +
llvm/unittests/ProfileData/InstrProfTest.cpp | 1 +
llvm/unittests/ProfileData/MemProfTest.cpp| 3 +-
11 files changed, 620 insertions(+), 574 deletions(-)
create mode 100644 llvm/include/llvm/ProfileData/MemProfRadixTree.h
create mode 100644 llvm/lib/ProfileData/MemProfRadixTree.cpp
diff --git a/llvm/include/llvm/ProfileData/MemProf.h
b/llvm/include/llvm/ProfileData/MemProf.h
index e713c3807611b..0bc1432f7d198 100644
--- a/llvm/include/llvm/ProfileData/MemProf.h
+++ b/llvm/include/llvm/ProfileData/MemProf.h
@@ -818,133 +818,6 @@ class CallStackLookupTrait {
}
};
-namespace detail {
-// "Dereference" the iterator from DenseMap or OnDiskChainedHashTable. We have
-// to do so in one of two different ways depending on the type of the hash
-// table.
-template
-value_type DerefIterator(IterTy Iter) {
- using deref_type = llvm::remove_cvref_t;
- if constexpr (std::is_same_v)
-return *Iter;
- else
-return Iter->second;
-}
-} // namespace detail
-
-// A function object that returns a frame for a given FrameId.
-template struct FrameIdConverter {
- std::optional LastUnmappedId;
- MapTy ⤅
-
- FrameIdConverter() = delete;
- FrameIdConverter(MapTy &Map) : Map(Map) {}
-
- // Delete the copy constructor and copy assignment operator to avoid a
- // situation where a copy of FrameIdConverter gets an error in LastUnmappedId
- // while the original instance doesn't.
- FrameIdConverter(const FrameIdConverter &) = delete;
- FrameIdConverter &operator=(const FrameIdConverter &) = delete;
-
- Frame operator()(FrameId Id) {
-auto Iter = Map.find(Id);
-if (Iter == Map.end()) {
- LastUnmappedId = Id;
- return Frame();
-}
-return detail::DerefIterator(Iter);
- }
-};
-
-// A function object that returns a call stack for a given CallStackId.
-template struct CallStackIdConverter {
- std::optional LastUnmappedId;
- MapTy ⤅
- llvm::function_ref FrameIdToFrame;
-
- CallStackIdConverter() = delete;
- CallStackIdConverter(MapTy &Map,
- llvm::function_ref FrameIdToFrame)
- : Map(Map), FrameIdToFrame(FrameIdToFrame) {}
-
- // Delete the copy constructor and copy assignment operator to avoid a
- // situation where a copy of CallStackIdConverter gets an error in
- // LastUnmappedId while the original instance doesn't.
- CallStackIdConverter(const CallStackIdConverter &) = delete;
- CallStackIdConverter &operator=(const CallStackIdConverter &) = delete;
-
- std::vector operator()(CallStackId CSId) {
-std::vector Frames;
-auto CSIter = Map.find(CSId);
-if (CSIter == Map.end()) {
- LastUnmappedId = CSId;
-} else {
- llvm::SmallVector CS =
- detail::DerefIterator>(CSIter);
- Frames.reserve(CS.size());
- for (FrameId Id : CS)
-Frames.push_back(FrameIdToFrame(Id));
-}
-return Frames;
- }
-};
-
-// A function object that returns a Frame stored at a given index into the
Frame
-// array in the profile.
-struct LinearFrameIdConverter {
- const unsigned char *FrameBase;
-
- LinearFrameIdConverter() = delete;
- LinearFrameIdConverter(const unsigned char *FrameBase)
- : FrameBase(FrameBase) {}
-
- Frame operator()(LinearFrameId LinearId) {
-uint64_t Offset = static_cast(LinearId) *
Frame::serializedSize();
-return Frame::deserialize(FrameBase + Offset);
- }
-};
-
-// A function object that returns a call stack stored at a given index into the
-// call stack array in the profile.
-struct LinearCallStackIdConverter {
- const unsigned char *CallStackBase;
- llvm::function_ref FrameIdToFrame;
-
- LinearCallStackIdConverter() = delete;
- LinearCallStackIdConverter(
- const unsigned char *CallStackBase,
- llvm::function_ref FrameIdToFrame)
- : CallStackBase(CallStackBase), FrameIdToFrame(FrameIdToFrame) {}
-
- std::vector operator()(LinearCallStackId LinearCSId) {
-std::vector Frames;
-
-const unsigned char *Ptr =
-CallStackBase +
-static_cast(LinearCSId) * sizeof(LinearFrameId);
-uint32_t NumFrames =
-support::endian::readNext(Ptr);
-Frames.reserve(NumFrames);
[llvm-branch-commits] [llvm] [NFC][MemProf] Move types shared between Analysis, ProfileData and ModuleSummary (Core) to a separate header (PR #140505)
snehasish wrote: ### Merge activity * **May 19, 7:09 PM EDT**: A user started a stack merge that includes this pull request via [Graphite](https://app.graphite.dev/github/pr/llvm/llvm-project/140505). https://github.com/llvm/llvm-project/pull/140505 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Add the LLVM license text and minor clean up. (PR #140504)
snehasish wrote: ### Merge activity * **May 19, 7:09 PM EDT**: A user started a stack merge that includes this pull request via [Graphite](https://app.graphite.dev/github/pr/llvm/llvm-project/140504). https://github.com/llvm/llvm-project/pull/140504 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Move IndexedMemProfData to its own header. (PR #140503)
snehasish wrote: ### Merge activity * **May 19, 7:09 PM EDT**: A user started a stack merge that includes this pull request via [Graphite](https://app.graphite.dev/github/pr/llvm/llvm-project/140503). https://github.com/llvm/llvm-project/pull/140503 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Move getGUID out of IndexedMemProfRecord (PR #140502)
snehasish wrote: ### Merge activity * **May 19, 7:09 PM EDT**: A user started a stack merge that includes this pull request via [Graphite](https://app.graphite.dev/github/pr/llvm/llvm-project/140502). https://github.com/llvm/llvm-project/pull/140502 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [NFC][MemProf] Move Radix tree methods to their own header and cpp. (PR #140501)
snehasish wrote: ### Merge activity * **May 19, 7:09 PM EDT**: A user started a stack merge that includes this pull request via [Graphite](https://app.graphite.dev/github/pr/llvm/llvm-project/140501). https://github.com/llvm/llvm-project/pull/140501 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
