[clang] Fix undefined lld::wasm::link symbol while building clangInterpreter for wasm (PR #113446)
https://github.com/anutosh491 created https://github.com/llvm/llvm-project/pull/113446 While building llvm (clang, lld) for wasm using emscripten (recipe hosted on emscripten-forge https://github.com/emscripten-forge/recipes/tree/main/recipes/recipes_emscripten/llvm) I ended up with this error ``` │ │ wasm-ld: error: ../../../../lib/libclangInterpreter.a(Wasm.cpp.o): undefined symbol: lld::wasm::link(llvm::ArrayRef, llvm::raw_ostream&, llvm: │ │ :raw_ostream&, bool, bool) ``` This is due to the link function here https://github.com/llvm/llvm-project/blob/a4819bd46d8baebc3aaa8b38f78065de33593199/clang/lib/Interpreter/Wasm.cpp#L25-L30 This was added through this PR (https://github.com/llvm/llvm-project/pull/86402) as an attempt to support running clang-repl and executing C++ code interactively inside a Javascript engine using WebAssembly when built with Emscripten. The definition for link is present in lldwasm and when building for the emscripten platform we should be linking against it. >From a10d5c42b8685295a5092917750d18a59441bad5 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Wed, 23 Oct 2024 16:58:57 +0530 Subject: [PATCH] Fix undefined lld::wasm::link symbol while building clangInterpreter for wasm --- clang/lib/Interpreter/CMakeLists.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/clang/lib/Interpreter/CMakeLists.txt b/clang/lib/Interpreter/CMakeLists.txt index 2cc7c59b61d318..d5ffe78251d253 100644 --- a/clang/lib/Interpreter/CMakeLists.txt +++ b/clang/lib/Interpreter/CMakeLists.txt @@ -14,6 +14,7 @@ set(LLVM_LINK_COMPONENTS if (EMSCRIPTEN AND "lld" IN_LIST LLVM_ENABLE_PROJECTS) set(WASM_SRC Wasm.cpp) + set(WASM_LINK lldWasm) endif() add_clang_library(clangInterpreter @@ -44,6 +45,7 @@ add_clang_library(clangInterpreter clangParse clangSema clangSerialization + ${WASM_LINK} ) if ((MINGW OR CYGWIN) AND BUILD_SHARED_LIBS) ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] Fix undefined lld::wasm::link symbol while building clangInterpreter for wasm (PR #113446)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/113446 >From a19fb1d00dfd9a3be9463396d476ef7415391a0f Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Wed, 23 Oct 2024 16:58:57 +0530 Subject: [PATCH] Fix undefined lld::wasm::link symbol while building clangInterpreter for wasm --- clang/lib/Interpreter/CMakeLists.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/clang/lib/Interpreter/CMakeLists.txt b/clang/lib/Interpreter/CMakeLists.txt index 2cc7c59b61d318..d5ffe78251d253 100644 --- a/clang/lib/Interpreter/CMakeLists.txt +++ b/clang/lib/Interpreter/CMakeLists.txt @@ -14,6 +14,7 @@ set(LLVM_LINK_COMPONENTS if (EMSCRIPTEN AND "lld" IN_LIST LLVM_ENABLE_PROJECTS) set(WASM_SRC Wasm.cpp) + set(WASM_LINK lldWasm) endif() add_clang_library(clangInterpreter @@ -44,6 +45,7 @@ add_clang_library(clangInterpreter clangParse clangSema clangSerialization + ${WASM_LINK} ) if ((MINGW OR CYGWIN) AND BUILD_SHARED_LIBS) ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Fix undefined lld::wasm::link symbol while building clangInterpreter for wasm (PR #113446)
https://github.com/anutosh491 edited https://github.com/llvm/llvm-project/pull/113446 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] Fix undefined lld::wasm::link symbol while building clangInterpreter for wasm (PR #113446)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/113446 >From 921d6d4f2acdd85d95bb8c5904177b0d39b5393b Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Wed, 23 Oct 2024 16:58:57 +0530 Subject: [PATCH] Fix undefined lld::wasm::link symbol while building clangInterpreter for wasm --- clang/lib/Interpreter/CMakeLists.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/clang/lib/Interpreter/CMakeLists.txt b/clang/lib/Interpreter/CMakeLists.txt index 2cc7c59b61d318..d5ffe78251d253 100644 --- a/clang/lib/Interpreter/CMakeLists.txt +++ b/clang/lib/Interpreter/CMakeLists.txt @@ -14,6 +14,7 @@ set(LLVM_LINK_COMPONENTS if (EMSCRIPTEN AND "lld" IN_LIST LLVM_ENABLE_PROJECTS) set(WASM_SRC Wasm.cpp) + set(WASM_LINK lldWasm) endif() add_clang_library(clangInterpreter @@ -44,6 +45,7 @@ add_clang_library(clangInterpreter clangParse clangSema clangSerialization + ${WASM_LINK} ) if ((MINGW OR CYGWIN) AND BUILD_SHARED_LIBS) ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Fix undefined lld::wasm::link symbol while building clangInterpreter for wasm (PR #113446)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/113446 >From d61f58f66839652ec4c1101521c2a7297cffacff Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Wed, 23 Oct 2024 16:58:57 +0530 Subject: [PATCH] Fix undefined lld::wasm::link symbol while building clangInterpreter for wasm --- clang/lib/Interpreter/CMakeLists.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/clang/lib/Interpreter/CMakeLists.txt b/clang/lib/Interpreter/CMakeLists.txt index 2cc7c59b61d318..d5ffe78251d253 100644 --- a/clang/lib/Interpreter/CMakeLists.txt +++ b/clang/lib/Interpreter/CMakeLists.txt @@ -14,6 +14,7 @@ set(LLVM_LINK_COMPONENTS if (EMSCRIPTEN AND "lld" IN_LIST LLVM_ENABLE_PROJECTS) set(WASM_SRC Wasm.cpp) + set(WASM_LINK lldWasm) endif() add_clang_library(clangInterpreter @@ -44,6 +45,7 @@ add_clang_library(clangInterpreter clangParse clangSema clangSerialization + ${WASM_LINK} ) if ((MINGW OR CYGWIN) AND BUILD_SHARED_LIBS) ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Fix generation of wasm binaries while running clang-repl in browser (PR #117978)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/117978 >From 987f77db9d45dee264c60f434652131438784f6f Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Thu, 28 Nov 2024 14:02:00 +0530 Subject: [PATCH 1/4] Fix generation of wasm binaries while running clang-repl in browser --- clang/lib/Interpreter/CMakeLists.txt | 2 + clang/lib/Interpreter/Interpreter.cpp | 1 + clang/lib/Interpreter/Wasm.cpp| 57 --- 3 files changed, 45 insertions(+), 15 deletions(-) diff --git a/clang/lib/Interpreter/CMakeLists.txt b/clang/lib/Interpreter/CMakeLists.txt index df7ea82e0dada5..bf70cdfbee01e1 100644 --- a/clang/lib/Interpreter/CMakeLists.txt +++ b/clang/lib/Interpreter/CMakeLists.txt @@ -16,6 +16,7 @@ set(LLVM_LINK_COMPONENTS if (EMSCRIPTEN AND "lld" IN_LIST LLVM_ENABLE_PROJECTS) set(WASM_SRC Wasm.cpp) set(WASM_LINK lldWasm) + set(COMMON_LINK lldCommon) endif() add_clang_library(clangInterpreter @@ -47,6 +48,7 @@ add_clang_library(clangInterpreter clangSema clangSerialization ${WASM_LINK} + ${COMMON_LINK} ) if ((MINGW OR CYGWIN) AND BUILD_SHARED_LIBS) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 5dc67f6375098f..887b494ff98f19 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -201,6 +201,7 @@ IncrementalCompilerBuilder::CreateCpp() { Argv.push_back("-target"); Argv.push_back("wasm32-unknown-emscripten"); Argv.push_back("-shared"); + Argv.push_back("-fvisibility=default"); #endif Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end()); diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index 79efbaa03982d0..0fd6ad509c2938 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -23,6 +23,31 @@ #include namespace lld { +enum Flavor { + Invalid, + Gnu, // -flavor gnu + MinGW, // -flavor gnu MinGW + WinLink, // -flavor link + Darwin, // -flavor darwin + Wasm,// -flavor wasm +}; + +using Driver = bool (*)(llvm::ArrayRef, llvm::raw_ostream &, +llvm::raw_ostream &, bool, bool); + +struct DriverDef { + Flavor f; + Driver d; +}; + +struct Result { +int retCode; +bool canRunAgain; +}; + +Result lldMain(llvm::ArrayRef args, llvm::raw_ostream &stdoutOS, +llvm::raw_ostream &stderrOS, llvm::ArrayRef drivers); + namespace wasm { bool link(llvm::ArrayRef args, llvm::raw_ostream &stdoutOS, llvm::raw_ostream &stderrOS, bool exitEarly, bool disableOutput); @@ -51,45 +76,47 @@ llvm::Error WasmIncrementalExecutor::addModule(PartialTranslationUnit &PTU) { llvm::TargetMachine *TargetMachine = Target->createTargetMachine( PTU.TheModule->getTargetTriple(), "", "", TO, llvm::Reloc::Model::PIC_); PTU.TheModule->setDataLayout(TargetMachine->createDataLayout()); - std::string OutputFileName = PTU.TheModule->getName().str() + ".wasm"; + std::string ObjectFileName = PTU.TheModule->getName().str() + ".o"; // For the wasm object + std::string BinaryFileName = PTU.TheModule->getName().str() + ".wasm"; // For the wasm binary std::error_code Error; - llvm::raw_fd_ostream OutputFile(llvm::StringRef(OutputFileName), Error); + llvm::raw_fd_ostream ObjectFileOutput(llvm::StringRef(ObjectFileName), Error); llvm::legacy::PassManager PM; - if (TargetMachine->addPassesToEmitFile(PM, OutputFile, nullptr, + if (TargetMachine->addPassesToEmitFile(PM, ObjectFileOutput, nullptr, llvm::CodeGenFileType::ObjectFile)) { return llvm::make_error( "Wasm backend cannot produce object.", llvm::inconvertibleErrorCode()); } if (!PM.run(*PTU.TheModule)) { - return llvm::make_error("Failed to emit Wasm object.", llvm::inconvertibleErrorCode()); } - OutputFile.close(); + ObjectFileOutput.close(); std::vector LinkerArgs = {"wasm-ld", "-shared", "--import-memory", - "--no-entry", - "--export-all", "--experimental-pic", "--stack-first", "--allow-undefined", - OutputFileName.c_str(), + ObjectFileName.c_str(), "-o", - OutputFileName.c_str()}; - int Result = - lld::wasm::link(LinkerArgs, llvm::outs(), llvm::errs(), false, false); - if (!Result) + BinaryFileName.c_str()}; + + const lld::DriverDef WasmDriver = {lld::Flavor::Wasm, &lld::wasm::link}; + std::vector WasmDriverArg
[clang] [clang-repl] Fix generation of wasm binaries while running clang-repl in browser (PR #117978)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/117978 >From 987f77db9d45dee264c60f434652131438784f6f Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Thu, 28 Nov 2024 14:02:00 +0530 Subject: [PATCH 1/4] Fix generation of wasm binaries while running clang-repl in browser --- clang/lib/Interpreter/CMakeLists.txt | 2 + clang/lib/Interpreter/Interpreter.cpp | 1 + clang/lib/Interpreter/Wasm.cpp| 57 --- 3 files changed, 45 insertions(+), 15 deletions(-) diff --git a/clang/lib/Interpreter/CMakeLists.txt b/clang/lib/Interpreter/CMakeLists.txt index df7ea82e0dada5..bf70cdfbee01e1 100644 --- a/clang/lib/Interpreter/CMakeLists.txt +++ b/clang/lib/Interpreter/CMakeLists.txt @@ -16,6 +16,7 @@ set(LLVM_LINK_COMPONENTS if (EMSCRIPTEN AND "lld" IN_LIST LLVM_ENABLE_PROJECTS) set(WASM_SRC Wasm.cpp) set(WASM_LINK lldWasm) + set(COMMON_LINK lldCommon) endif() add_clang_library(clangInterpreter @@ -47,6 +48,7 @@ add_clang_library(clangInterpreter clangSema clangSerialization ${WASM_LINK} + ${COMMON_LINK} ) if ((MINGW OR CYGWIN) AND BUILD_SHARED_LIBS) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 5dc67f6375098f..887b494ff98f19 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -201,6 +201,7 @@ IncrementalCompilerBuilder::CreateCpp() { Argv.push_back("-target"); Argv.push_back("wasm32-unknown-emscripten"); Argv.push_back("-shared"); + Argv.push_back("-fvisibility=default"); #endif Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end()); diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index 79efbaa03982d0..0fd6ad509c2938 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -23,6 +23,31 @@ #include namespace lld { +enum Flavor { + Invalid, + Gnu, // -flavor gnu + MinGW, // -flavor gnu MinGW + WinLink, // -flavor link + Darwin, // -flavor darwin + Wasm,// -flavor wasm +}; + +using Driver = bool (*)(llvm::ArrayRef, llvm::raw_ostream &, +llvm::raw_ostream &, bool, bool); + +struct DriverDef { + Flavor f; + Driver d; +}; + +struct Result { +int retCode; +bool canRunAgain; +}; + +Result lldMain(llvm::ArrayRef args, llvm::raw_ostream &stdoutOS, +llvm::raw_ostream &stderrOS, llvm::ArrayRef drivers); + namespace wasm { bool link(llvm::ArrayRef args, llvm::raw_ostream &stdoutOS, llvm::raw_ostream &stderrOS, bool exitEarly, bool disableOutput); @@ -51,45 +76,47 @@ llvm::Error WasmIncrementalExecutor::addModule(PartialTranslationUnit &PTU) { llvm::TargetMachine *TargetMachine = Target->createTargetMachine( PTU.TheModule->getTargetTriple(), "", "", TO, llvm::Reloc::Model::PIC_); PTU.TheModule->setDataLayout(TargetMachine->createDataLayout()); - std::string OutputFileName = PTU.TheModule->getName().str() + ".wasm"; + std::string ObjectFileName = PTU.TheModule->getName().str() + ".o"; // For the wasm object + std::string BinaryFileName = PTU.TheModule->getName().str() + ".wasm"; // For the wasm binary std::error_code Error; - llvm::raw_fd_ostream OutputFile(llvm::StringRef(OutputFileName), Error); + llvm::raw_fd_ostream ObjectFileOutput(llvm::StringRef(ObjectFileName), Error); llvm::legacy::PassManager PM; - if (TargetMachine->addPassesToEmitFile(PM, OutputFile, nullptr, + if (TargetMachine->addPassesToEmitFile(PM, ObjectFileOutput, nullptr, llvm::CodeGenFileType::ObjectFile)) { return llvm::make_error( "Wasm backend cannot produce object.", llvm::inconvertibleErrorCode()); } if (!PM.run(*PTU.TheModule)) { - return llvm::make_error("Failed to emit Wasm object.", llvm::inconvertibleErrorCode()); } - OutputFile.close(); + ObjectFileOutput.close(); std::vector LinkerArgs = {"wasm-ld", "-shared", "--import-memory", - "--no-entry", - "--export-all", "--experimental-pic", "--stack-first", "--allow-undefined", - OutputFileName.c_str(), + ObjectFileName.c_str(), "-o", - OutputFileName.c_str()}; - int Result = - lld::wasm::link(LinkerArgs, llvm::outs(), llvm::errs(), false, false); - if (!Result) + BinaryFileName.c_str()}; + + const lld::DriverDef WasmDriver = {lld::Flavor::Wasm, &lld::wasm::link}; + std::vector WasmDriverArg
[clang] [clang-repl] Fix generation of wasm binaries while running clang-repl in browser (PR #117978)
anutosh491 wrote: Hey @vgvassilev thanks for the reviews. Addressed them. Once merged, I shall do a cherry pick of the required commit ! https://github.com/llvm/llvm-project/pull/117978 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Fix generation of wasm binaries while running clang-repl in browser (PR #117978)
anutosh491 wrote: /cherry-pick a174aa1e416c4e27945f5a8c646b119126dc8441 https://github.com/llvm/llvm-project/pull/117978 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Remove redundant shared flag while running clang-repl in browser (PR #118107)
https://github.com/anutosh491 created https://github.com/llvm/llvm-project/pull/118107 While running clang-repl in the browser, we would be interested in this cc1 command ` "" -cc1 -triple wasm32-unknown-emscripten -emit-obj -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name "<<< inputs >>>" -mrelocation-model static -mframe-pointer=none -ffp-contract=on -fno-rounding-math -mconstructor-aliases -target-cpu generic -debugger-tuning=gdb -fdebug-compilation-dir=/ -v -fcoverage-compilation-dir=/ -resource-dir /lib/clang/19 -internal-isystem /include/wasm32-emscripten/c++/v1 -internal-isystem /include/c++/v1 -internal-isystem /lib/clang/19/include -internal-isystem /include/wasm32-emscripten -internal-isystem /include -std=c++17 -fdeprecated-macro -ferror-limit 19 -fvisibility=default -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fcxx-exceptions -fexceptions -fincremental-extensions -o "<<< inputs >>>.o" -x c++ "<<< inputs >>>" ` As can be seen shared is anyway overwritten by static which is also what would be provided by default. Hence we can get rid of the shared flag here. >From 594b58c80cc0e3a27a3f4c6b92bd8e54a3118ad5 Mon Sep 17 00:00:00 2001 From: Anutosh Bhat Date: Fri, 29 Nov 2024 21:46:25 +0530 Subject: [PATCH] Remove redundant shared flag while running clang-repl in browser --- clang/lib/Interpreter/Interpreter.cpp | 1 - 1 file changed, 1 deletion(-) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 887b494ff98f19..fa4c1439c92612 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -200,7 +200,6 @@ IncrementalCompilerBuilder::CreateCpp() { #ifdef __EMSCRIPTEN__ Argv.push_back("-target"); Argv.push_back("wasm32-unknown-emscripten"); - Argv.push_back("-shared"); Argv.push_back("-fvisibility=default"); #endif Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end()); ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Remove redundant shared flag while running clang-repl in browser (PR #118107)
https://github.com/anutosh491 edited https://github.com/llvm/llvm-project/pull/118107 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Fix generation of wasm binaries while running clang-repl in browser (PR #117978)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/117978 >From 987f77db9d45dee264c60f434652131438784f6f Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Thu, 28 Nov 2024 14:02:00 +0530 Subject: [PATCH 1/3] Fix generation of wasm binaries while running clang-repl in browser --- clang/lib/Interpreter/CMakeLists.txt | 2 + clang/lib/Interpreter/Interpreter.cpp | 1 + clang/lib/Interpreter/Wasm.cpp| 57 --- 3 files changed, 45 insertions(+), 15 deletions(-) diff --git a/clang/lib/Interpreter/CMakeLists.txt b/clang/lib/Interpreter/CMakeLists.txt index df7ea82e0dada5..bf70cdfbee01e1 100644 --- a/clang/lib/Interpreter/CMakeLists.txt +++ b/clang/lib/Interpreter/CMakeLists.txt @@ -16,6 +16,7 @@ set(LLVM_LINK_COMPONENTS if (EMSCRIPTEN AND "lld" IN_LIST LLVM_ENABLE_PROJECTS) set(WASM_SRC Wasm.cpp) set(WASM_LINK lldWasm) + set(COMMON_LINK lldCommon) endif() add_clang_library(clangInterpreter @@ -47,6 +48,7 @@ add_clang_library(clangInterpreter clangSema clangSerialization ${WASM_LINK} + ${COMMON_LINK} ) if ((MINGW OR CYGWIN) AND BUILD_SHARED_LIBS) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 5dc67f6375098f..887b494ff98f19 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -201,6 +201,7 @@ IncrementalCompilerBuilder::CreateCpp() { Argv.push_back("-target"); Argv.push_back("wasm32-unknown-emscripten"); Argv.push_back("-shared"); + Argv.push_back("-fvisibility=default"); #endif Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end()); diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index 79efbaa03982d0..0fd6ad509c2938 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -23,6 +23,31 @@ #include namespace lld { +enum Flavor { + Invalid, + Gnu, // -flavor gnu + MinGW, // -flavor gnu MinGW + WinLink, // -flavor link + Darwin, // -flavor darwin + Wasm,// -flavor wasm +}; + +using Driver = bool (*)(llvm::ArrayRef, llvm::raw_ostream &, +llvm::raw_ostream &, bool, bool); + +struct DriverDef { + Flavor f; + Driver d; +}; + +struct Result { +int retCode; +bool canRunAgain; +}; + +Result lldMain(llvm::ArrayRef args, llvm::raw_ostream &stdoutOS, +llvm::raw_ostream &stderrOS, llvm::ArrayRef drivers); + namespace wasm { bool link(llvm::ArrayRef args, llvm::raw_ostream &stdoutOS, llvm::raw_ostream &stderrOS, bool exitEarly, bool disableOutput); @@ -51,45 +76,47 @@ llvm::Error WasmIncrementalExecutor::addModule(PartialTranslationUnit &PTU) { llvm::TargetMachine *TargetMachine = Target->createTargetMachine( PTU.TheModule->getTargetTriple(), "", "", TO, llvm::Reloc::Model::PIC_); PTU.TheModule->setDataLayout(TargetMachine->createDataLayout()); - std::string OutputFileName = PTU.TheModule->getName().str() + ".wasm"; + std::string ObjectFileName = PTU.TheModule->getName().str() + ".o"; // For the wasm object + std::string BinaryFileName = PTU.TheModule->getName().str() + ".wasm"; // For the wasm binary std::error_code Error; - llvm::raw_fd_ostream OutputFile(llvm::StringRef(OutputFileName), Error); + llvm::raw_fd_ostream ObjectFileOutput(llvm::StringRef(ObjectFileName), Error); llvm::legacy::PassManager PM; - if (TargetMachine->addPassesToEmitFile(PM, OutputFile, nullptr, + if (TargetMachine->addPassesToEmitFile(PM, ObjectFileOutput, nullptr, llvm::CodeGenFileType::ObjectFile)) { return llvm::make_error( "Wasm backend cannot produce object.", llvm::inconvertibleErrorCode()); } if (!PM.run(*PTU.TheModule)) { - return llvm::make_error("Failed to emit Wasm object.", llvm::inconvertibleErrorCode()); } - OutputFile.close(); + ObjectFileOutput.close(); std::vector LinkerArgs = {"wasm-ld", "-shared", "--import-memory", - "--no-entry", - "--export-all", "--experimental-pic", "--stack-first", "--allow-undefined", - OutputFileName.c_str(), + ObjectFileName.c_str(), "-o", - OutputFileName.c_str()}; - int Result = - lld::wasm::link(LinkerArgs, llvm::outs(), llvm::errs(), false, false); - if (!Result) + BinaryFileName.c_str()}; + + const lld::DriverDef WasmDriver = {lld::Flavor::Wasm, &lld::wasm::link}; + std::vector WasmDriverArg
[clang] [clang-repl] Fix generation of wasm binaries while running clang-repl in browser (PR #117978)
@@ -109,12 +139,12 @@ llvm::Error WasmIncrementalExecutor::runCtors() const { return llvm::Error::success(); } -llvm::Error WasmIncrementalExecutor::cleanUp() const { anutosh491 wrote: I think we would need it. I see a build error otherwise ``` │ │ /home/runner/work/recipes/recipes/output/bld/rattler-build_llvm_1732028457/work/clang/lib/Interpreter/Wasm.cpp:111:38: error: out-of-line definition of 'cl │ │ eanUp' does not match any declaration in 'clang::WasmIncrementalExecutor' │ │ 111 | llvm::Error WasmIncrementalExecutor::cleanUp() const { │ │ | ^~~ │ │ /home/runner/work/recipes/recipes/output/bld/rattler-build_llvm_1732028457/work/clang/lib/Interpreter/Wasm.h:31:15: note: member declaration does not match │ │ because it is not const qualified │ │31 | llvm::Error cleanUp() override; │ │ | ^ ``` CleanUp is defined like this https://github.com/llvm/llvm-project/blob/b2d3cb1e7502a5ddb7d6688dce5cf51573f6189c/clang/lib/Interpreter/Wasm.h#L31 https://github.com/llvm/llvm-project/pull/117978 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] Fix generation of wasm binaries while running clang-repl in browser (PR #117978)
https://github.com/anutosh491 created https://github.com/llvm/llvm-project/pull/117978 None >From 987f77db9d45dee264c60f434652131438784f6f Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Thu, 28 Nov 2024 14:02:00 +0530 Subject: [PATCH] Fix generation of wasm binaries while running clang-repl in browser --- clang/lib/Interpreter/CMakeLists.txt | 2 + clang/lib/Interpreter/Interpreter.cpp | 1 + clang/lib/Interpreter/Wasm.cpp| 57 --- 3 files changed, 45 insertions(+), 15 deletions(-) diff --git a/clang/lib/Interpreter/CMakeLists.txt b/clang/lib/Interpreter/CMakeLists.txt index df7ea82e0dada5..bf70cdfbee01e1 100644 --- a/clang/lib/Interpreter/CMakeLists.txt +++ b/clang/lib/Interpreter/CMakeLists.txt @@ -16,6 +16,7 @@ set(LLVM_LINK_COMPONENTS if (EMSCRIPTEN AND "lld" IN_LIST LLVM_ENABLE_PROJECTS) set(WASM_SRC Wasm.cpp) set(WASM_LINK lldWasm) + set(COMMON_LINK lldCommon) endif() add_clang_library(clangInterpreter @@ -47,6 +48,7 @@ add_clang_library(clangInterpreter clangSema clangSerialization ${WASM_LINK} + ${COMMON_LINK} ) if ((MINGW OR CYGWIN) AND BUILD_SHARED_LIBS) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 5dc67f6375098f..887b494ff98f19 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -201,6 +201,7 @@ IncrementalCompilerBuilder::CreateCpp() { Argv.push_back("-target"); Argv.push_back("wasm32-unknown-emscripten"); Argv.push_back("-shared"); + Argv.push_back("-fvisibility=default"); #endif Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end()); diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index 79efbaa03982d0..0fd6ad509c2938 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -23,6 +23,31 @@ #include namespace lld { +enum Flavor { + Invalid, + Gnu, // -flavor gnu + MinGW, // -flavor gnu MinGW + WinLink, // -flavor link + Darwin, // -flavor darwin + Wasm,// -flavor wasm +}; + +using Driver = bool (*)(llvm::ArrayRef, llvm::raw_ostream &, +llvm::raw_ostream &, bool, bool); + +struct DriverDef { + Flavor f; + Driver d; +}; + +struct Result { +int retCode; +bool canRunAgain; +}; + +Result lldMain(llvm::ArrayRef args, llvm::raw_ostream &stdoutOS, +llvm::raw_ostream &stderrOS, llvm::ArrayRef drivers); + namespace wasm { bool link(llvm::ArrayRef args, llvm::raw_ostream &stdoutOS, llvm::raw_ostream &stderrOS, bool exitEarly, bool disableOutput); @@ -51,45 +76,47 @@ llvm::Error WasmIncrementalExecutor::addModule(PartialTranslationUnit &PTU) { llvm::TargetMachine *TargetMachine = Target->createTargetMachine( PTU.TheModule->getTargetTriple(), "", "", TO, llvm::Reloc::Model::PIC_); PTU.TheModule->setDataLayout(TargetMachine->createDataLayout()); - std::string OutputFileName = PTU.TheModule->getName().str() + ".wasm"; + std::string ObjectFileName = PTU.TheModule->getName().str() + ".o"; // For the wasm object + std::string BinaryFileName = PTU.TheModule->getName().str() + ".wasm"; // For the wasm binary std::error_code Error; - llvm::raw_fd_ostream OutputFile(llvm::StringRef(OutputFileName), Error); + llvm::raw_fd_ostream ObjectFileOutput(llvm::StringRef(ObjectFileName), Error); llvm::legacy::PassManager PM; - if (TargetMachine->addPassesToEmitFile(PM, OutputFile, nullptr, + if (TargetMachine->addPassesToEmitFile(PM, ObjectFileOutput, nullptr, llvm::CodeGenFileType::ObjectFile)) { return llvm::make_error( "Wasm backend cannot produce object.", llvm::inconvertibleErrorCode()); } if (!PM.run(*PTU.TheModule)) { - return llvm::make_error("Failed to emit Wasm object.", llvm::inconvertibleErrorCode()); } - OutputFile.close(); + ObjectFileOutput.close(); std::vector LinkerArgs = {"wasm-ld", "-shared", "--import-memory", - "--no-entry", - "--export-all", "--experimental-pic", "--stack-first", "--allow-undefined", - OutputFileName.c_str(), + ObjectFileName.c_str(), "-o", - OutputFileName.c_str()}; - int Result = - lld::wasm::link(LinkerArgs, llvm::outs(), llvm::errs(), false, false); - if (!Result) + BinaryFileName.c_str()}; + + const lld::DriverDef WasmDriver = {lld::Flavor::Wasm, &lld::wasm::link}; + std::vector WasmDriverA
[clang] [clang-repl] Fix generation of wasm binaries while running clang-repl in browser (PR #117978)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/117978 >From 987f77db9d45dee264c60f434652131438784f6f Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Thu, 28 Nov 2024 14:02:00 +0530 Subject: [PATCH 1/2] Fix generation of wasm binaries while running clang-repl in browser --- clang/lib/Interpreter/CMakeLists.txt | 2 + clang/lib/Interpreter/Interpreter.cpp | 1 + clang/lib/Interpreter/Wasm.cpp| 57 --- 3 files changed, 45 insertions(+), 15 deletions(-) diff --git a/clang/lib/Interpreter/CMakeLists.txt b/clang/lib/Interpreter/CMakeLists.txt index df7ea82e0dada5..bf70cdfbee01e1 100644 --- a/clang/lib/Interpreter/CMakeLists.txt +++ b/clang/lib/Interpreter/CMakeLists.txt @@ -16,6 +16,7 @@ set(LLVM_LINK_COMPONENTS if (EMSCRIPTEN AND "lld" IN_LIST LLVM_ENABLE_PROJECTS) set(WASM_SRC Wasm.cpp) set(WASM_LINK lldWasm) + set(COMMON_LINK lldCommon) endif() add_clang_library(clangInterpreter @@ -47,6 +48,7 @@ add_clang_library(clangInterpreter clangSema clangSerialization ${WASM_LINK} + ${COMMON_LINK} ) if ((MINGW OR CYGWIN) AND BUILD_SHARED_LIBS) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 5dc67f6375098f..887b494ff98f19 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -201,6 +201,7 @@ IncrementalCompilerBuilder::CreateCpp() { Argv.push_back("-target"); Argv.push_back("wasm32-unknown-emscripten"); Argv.push_back("-shared"); + Argv.push_back("-fvisibility=default"); #endif Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end()); diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index 79efbaa03982d0..0fd6ad509c2938 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -23,6 +23,31 @@ #include namespace lld { +enum Flavor { + Invalid, + Gnu, // -flavor gnu + MinGW, // -flavor gnu MinGW + WinLink, // -flavor link + Darwin, // -flavor darwin + Wasm,// -flavor wasm +}; + +using Driver = bool (*)(llvm::ArrayRef, llvm::raw_ostream &, +llvm::raw_ostream &, bool, bool); + +struct DriverDef { + Flavor f; + Driver d; +}; + +struct Result { +int retCode; +bool canRunAgain; +}; + +Result lldMain(llvm::ArrayRef args, llvm::raw_ostream &stdoutOS, +llvm::raw_ostream &stderrOS, llvm::ArrayRef drivers); + namespace wasm { bool link(llvm::ArrayRef args, llvm::raw_ostream &stdoutOS, llvm::raw_ostream &stderrOS, bool exitEarly, bool disableOutput); @@ -51,45 +76,47 @@ llvm::Error WasmIncrementalExecutor::addModule(PartialTranslationUnit &PTU) { llvm::TargetMachine *TargetMachine = Target->createTargetMachine( PTU.TheModule->getTargetTriple(), "", "", TO, llvm::Reloc::Model::PIC_); PTU.TheModule->setDataLayout(TargetMachine->createDataLayout()); - std::string OutputFileName = PTU.TheModule->getName().str() + ".wasm"; + std::string ObjectFileName = PTU.TheModule->getName().str() + ".o"; // For the wasm object + std::string BinaryFileName = PTU.TheModule->getName().str() + ".wasm"; // For the wasm binary std::error_code Error; - llvm::raw_fd_ostream OutputFile(llvm::StringRef(OutputFileName), Error); + llvm::raw_fd_ostream ObjectFileOutput(llvm::StringRef(ObjectFileName), Error); llvm::legacy::PassManager PM; - if (TargetMachine->addPassesToEmitFile(PM, OutputFile, nullptr, + if (TargetMachine->addPassesToEmitFile(PM, ObjectFileOutput, nullptr, llvm::CodeGenFileType::ObjectFile)) { return llvm::make_error( "Wasm backend cannot produce object.", llvm::inconvertibleErrorCode()); } if (!PM.run(*PTU.TheModule)) { - return llvm::make_error("Failed to emit Wasm object.", llvm::inconvertibleErrorCode()); } - OutputFile.close(); + ObjectFileOutput.close(); std::vector LinkerArgs = {"wasm-ld", "-shared", "--import-memory", - "--no-entry", - "--export-all", "--experimental-pic", "--stack-first", "--allow-undefined", - OutputFileName.c_str(), + ObjectFileName.c_str(), "-o", - OutputFileName.c_str()}; - int Result = - lld::wasm::link(LinkerArgs, llvm::outs(), llvm::errs(), false, false); - if (!Result) + BinaryFileName.c_str()}; + + const lld::DriverDef WasmDriver = {lld::Flavor::Wasm, &lld::wasm::link}; + std::vector WasmDriverArg
[clang] [clang-repl] Fix generation of wasm binaries while running clang-repl in browser (PR #117978)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/117978 >From 987f77db9d45dee264c60f434652131438784f6f Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Thu, 28 Nov 2024 14:02:00 +0530 Subject: [PATCH 1/2] Fix generation of wasm binaries while running clang-repl in browser --- clang/lib/Interpreter/CMakeLists.txt | 2 + clang/lib/Interpreter/Interpreter.cpp | 1 + clang/lib/Interpreter/Wasm.cpp| 57 --- 3 files changed, 45 insertions(+), 15 deletions(-) diff --git a/clang/lib/Interpreter/CMakeLists.txt b/clang/lib/Interpreter/CMakeLists.txt index df7ea82e0dada5..bf70cdfbee01e1 100644 --- a/clang/lib/Interpreter/CMakeLists.txt +++ b/clang/lib/Interpreter/CMakeLists.txt @@ -16,6 +16,7 @@ set(LLVM_LINK_COMPONENTS if (EMSCRIPTEN AND "lld" IN_LIST LLVM_ENABLE_PROJECTS) set(WASM_SRC Wasm.cpp) set(WASM_LINK lldWasm) + set(COMMON_LINK lldCommon) endif() add_clang_library(clangInterpreter @@ -47,6 +48,7 @@ add_clang_library(clangInterpreter clangSema clangSerialization ${WASM_LINK} + ${COMMON_LINK} ) if ((MINGW OR CYGWIN) AND BUILD_SHARED_LIBS) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 5dc67f6375098f..887b494ff98f19 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -201,6 +201,7 @@ IncrementalCompilerBuilder::CreateCpp() { Argv.push_back("-target"); Argv.push_back("wasm32-unknown-emscripten"); Argv.push_back("-shared"); + Argv.push_back("-fvisibility=default"); #endif Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end()); diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index 79efbaa03982d0..0fd6ad509c2938 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -23,6 +23,31 @@ #include namespace lld { +enum Flavor { + Invalid, + Gnu, // -flavor gnu + MinGW, // -flavor gnu MinGW + WinLink, // -flavor link + Darwin, // -flavor darwin + Wasm,// -flavor wasm +}; + +using Driver = bool (*)(llvm::ArrayRef, llvm::raw_ostream &, +llvm::raw_ostream &, bool, bool); + +struct DriverDef { + Flavor f; + Driver d; +}; + +struct Result { +int retCode; +bool canRunAgain; +}; + +Result lldMain(llvm::ArrayRef args, llvm::raw_ostream &stdoutOS, +llvm::raw_ostream &stderrOS, llvm::ArrayRef drivers); + namespace wasm { bool link(llvm::ArrayRef args, llvm::raw_ostream &stdoutOS, llvm::raw_ostream &stderrOS, bool exitEarly, bool disableOutput); @@ -51,45 +76,47 @@ llvm::Error WasmIncrementalExecutor::addModule(PartialTranslationUnit &PTU) { llvm::TargetMachine *TargetMachine = Target->createTargetMachine( PTU.TheModule->getTargetTriple(), "", "", TO, llvm::Reloc::Model::PIC_); PTU.TheModule->setDataLayout(TargetMachine->createDataLayout()); - std::string OutputFileName = PTU.TheModule->getName().str() + ".wasm"; + std::string ObjectFileName = PTU.TheModule->getName().str() + ".o"; // For the wasm object + std::string BinaryFileName = PTU.TheModule->getName().str() + ".wasm"; // For the wasm binary std::error_code Error; - llvm::raw_fd_ostream OutputFile(llvm::StringRef(OutputFileName), Error); + llvm::raw_fd_ostream ObjectFileOutput(llvm::StringRef(ObjectFileName), Error); llvm::legacy::PassManager PM; - if (TargetMachine->addPassesToEmitFile(PM, OutputFile, nullptr, + if (TargetMachine->addPassesToEmitFile(PM, ObjectFileOutput, nullptr, llvm::CodeGenFileType::ObjectFile)) { return llvm::make_error( "Wasm backend cannot produce object.", llvm::inconvertibleErrorCode()); } if (!PM.run(*PTU.TheModule)) { - return llvm::make_error("Failed to emit Wasm object.", llvm::inconvertibleErrorCode()); } - OutputFile.close(); + ObjectFileOutput.close(); std::vector LinkerArgs = {"wasm-ld", "-shared", "--import-memory", - "--no-entry", - "--export-all", "--experimental-pic", "--stack-first", "--allow-undefined", - OutputFileName.c_str(), + ObjectFileName.c_str(), "-o", - OutputFileName.c_str()}; - int Result = - lld::wasm::link(LinkerArgs, llvm::outs(), llvm::errs(), false, false); - if (!Result) + BinaryFileName.c_str()}; + + const lld::DriverDef WasmDriver = {lld::Flavor::Wasm, &lld::wasm::link}; + std::vector WasmDriverArg
[clang] [clang-repl] Fix generation of wasm binaries while running clang-repl in browser (PR #117978)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/117978 >From 987f77db9d45dee264c60f434652131438784f6f Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Thu, 28 Nov 2024 14:02:00 +0530 Subject: [PATCH 1/2] Fix generation of wasm binaries while running clang-repl in browser --- clang/lib/Interpreter/CMakeLists.txt | 2 + clang/lib/Interpreter/Interpreter.cpp | 1 + clang/lib/Interpreter/Wasm.cpp| 57 --- 3 files changed, 45 insertions(+), 15 deletions(-) diff --git a/clang/lib/Interpreter/CMakeLists.txt b/clang/lib/Interpreter/CMakeLists.txt index df7ea82e0dada5..bf70cdfbee01e1 100644 --- a/clang/lib/Interpreter/CMakeLists.txt +++ b/clang/lib/Interpreter/CMakeLists.txt @@ -16,6 +16,7 @@ set(LLVM_LINK_COMPONENTS if (EMSCRIPTEN AND "lld" IN_LIST LLVM_ENABLE_PROJECTS) set(WASM_SRC Wasm.cpp) set(WASM_LINK lldWasm) + set(COMMON_LINK lldCommon) endif() add_clang_library(clangInterpreter @@ -47,6 +48,7 @@ add_clang_library(clangInterpreter clangSema clangSerialization ${WASM_LINK} + ${COMMON_LINK} ) if ((MINGW OR CYGWIN) AND BUILD_SHARED_LIBS) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 5dc67f6375098f..887b494ff98f19 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -201,6 +201,7 @@ IncrementalCompilerBuilder::CreateCpp() { Argv.push_back("-target"); Argv.push_back("wasm32-unknown-emscripten"); Argv.push_back("-shared"); + Argv.push_back("-fvisibility=default"); #endif Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end()); diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index 79efbaa03982d0..0fd6ad509c2938 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -23,6 +23,31 @@ #include namespace lld { +enum Flavor { + Invalid, + Gnu, // -flavor gnu + MinGW, // -flavor gnu MinGW + WinLink, // -flavor link + Darwin, // -flavor darwin + Wasm,// -flavor wasm +}; + +using Driver = bool (*)(llvm::ArrayRef, llvm::raw_ostream &, +llvm::raw_ostream &, bool, bool); + +struct DriverDef { + Flavor f; + Driver d; +}; + +struct Result { +int retCode; +bool canRunAgain; +}; + +Result lldMain(llvm::ArrayRef args, llvm::raw_ostream &stdoutOS, +llvm::raw_ostream &stderrOS, llvm::ArrayRef drivers); + namespace wasm { bool link(llvm::ArrayRef args, llvm::raw_ostream &stdoutOS, llvm::raw_ostream &stderrOS, bool exitEarly, bool disableOutput); @@ -51,45 +76,47 @@ llvm::Error WasmIncrementalExecutor::addModule(PartialTranslationUnit &PTU) { llvm::TargetMachine *TargetMachine = Target->createTargetMachine( PTU.TheModule->getTargetTriple(), "", "", TO, llvm::Reloc::Model::PIC_); PTU.TheModule->setDataLayout(TargetMachine->createDataLayout()); - std::string OutputFileName = PTU.TheModule->getName().str() + ".wasm"; + std::string ObjectFileName = PTU.TheModule->getName().str() + ".o"; // For the wasm object + std::string BinaryFileName = PTU.TheModule->getName().str() + ".wasm"; // For the wasm binary std::error_code Error; - llvm::raw_fd_ostream OutputFile(llvm::StringRef(OutputFileName), Error); + llvm::raw_fd_ostream ObjectFileOutput(llvm::StringRef(ObjectFileName), Error); llvm::legacy::PassManager PM; - if (TargetMachine->addPassesToEmitFile(PM, OutputFile, nullptr, + if (TargetMachine->addPassesToEmitFile(PM, ObjectFileOutput, nullptr, llvm::CodeGenFileType::ObjectFile)) { return llvm::make_error( "Wasm backend cannot produce object.", llvm::inconvertibleErrorCode()); } if (!PM.run(*PTU.TheModule)) { - return llvm::make_error("Failed to emit Wasm object.", llvm::inconvertibleErrorCode()); } - OutputFile.close(); + ObjectFileOutput.close(); std::vector LinkerArgs = {"wasm-ld", "-shared", "--import-memory", - "--no-entry", - "--export-all", "--experimental-pic", "--stack-first", "--allow-undefined", - OutputFileName.c_str(), + ObjectFileName.c_str(), "-o", - OutputFileName.c_str()}; - int Result = - lld::wasm::link(LinkerArgs, llvm::outs(), llvm::errs(), false, false); - if (!Result) + BinaryFileName.c_str()}; + + const lld::DriverDef WasmDriver = {lld::Flavor::Wasm, &lld::wasm::link}; + std::vector WasmDriverArg
[clang] [clang-repl] Fix generation of wasm binaries while running clang-repl in browser (PR #117978)
https://github.com/anutosh491 edited https://github.com/llvm/llvm-project/pull/117978 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Use default visibility for symbols while building CompilerInstance against emscripten (PR #116779)
https://github.com/anutosh491 closed https://github.com/llvm/llvm-project/pull/116779 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Use default visibility for symbols while building CompilerInstance against emscripten (PR #116779)
anutosh491 wrote: Closing as superseded by https://github.com/llvm/llvm-project/pull/116779 https://github.com/llvm/llvm-project/pull/116779 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Improve flags responsible for generating shared wasm binaries (PR #116735)
@@ -72,13 +72,13 @@ llvm::Error WasmIncrementalExecutor::addModule(PartialTranslationUnit &PTU) { OutputFile.close(); std::vector LinkerArgs = {"wasm-ld", - "-pie", + "-shared", "--import-memory", "--no-entry", "--export-all", "--experimental-pic", - "--no-export-dynamic", anutosh491 wrote: This change has been made cause `-shared` by default invokes `--export-dynamic`. Not a breaking change that harms us (but surely we would want `shared` instead of `pie`) https://github.com/llvm/llvm-project/pull/116735 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Improve flags responsible for generating shared wasm binaries (PR #116735)
https://github.com/anutosh491 created https://github.com/llvm/llvm-project/pull/116735 There are a couple changes in this PR that help getting clang-repl to run in the browser. Using a jupyterlite instance for the example pasted below 1) Updating flags responsible for generating shared wasm binaries that need to be dynamically loaded Most Importantly as can be seen in the changes `shared` and `allow-undefined` are crucial.  2) While exiting we encounter this.  Now as can be seen here https://github.com/llvm/llvm-project/blob/cd418030de7ae75750bc4e48d1238baf03c675e5/clang/lib/Interpreter/Interpreter.cpp#L421-L430 We call cleanUP in the destructor. Now cleanUP through IncrementalExecutor tries to deinitialize the JIT which wasn't even intialized as runCtors in wasm.cpp is a no-op https://github.com/llvm/llvm-project/blob/cd418030de7ae75750bc4e48d1238baf03c675e5/clang/lib/Interpreter/IncrementalExecutor.cpp#L94-L101 https://github.com/llvm/llvm-project/blob/cd418030de7ae75750bc4e48d1238baf03c675e5/clang/lib/Interpreter/Wasm.cpp#L107-L109 >From 07371ceff88bbfce990f9bf284bd64b4bbfd851a Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Tue, 19 Nov 2024 08:37:36 +0530 Subject: [PATCH] Improve flags responsible for generating shared wasm binaries --- clang/lib/Interpreter/IncrementalExecutor.h | 2 +- clang/lib/Interpreter/Wasm.cpp | 11 +-- clang/lib/Interpreter/Wasm.h| 1 + 3 files changed, 11 insertions(+), 3 deletions(-) diff --git a/clang/lib/Interpreter/IncrementalExecutor.h b/clang/lib/Interpreter/IncrementalExecutor.h index 7954cde36588bd..dbd61f0b8b1ebb 100644 --- a/clang/lib/Interpreter/IncrementalExecutor.h +++ b/clang/lib/Interpreter/IncrementalExecutor.h @@ -56,7 +56,7 @@ class IncrementalExecutor { virtual llvm::Error addModule(PartialTranslationUnit &PTU); virtual llvm::Error removeModule(PartialTranslationUnit &PTU); virtual llvm::Error runCtors() const; - llvm::Error cleanUp(); + virtual llvm::Error cleanUp(); llvm::Expected getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const; diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index 1001410aa0f279..6f00bac8c0bc46 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -72,13 +72,14 @@ llvm::Error WasmIncrementalExecutor::addModule(PartialTranslationUnit &PTU) { OutputFile.close(); std::vector LinkerArgs = {"wasm-ld", - "-pie", + "-shared", "--import-memory", "--no-entry", "--export-all", "--experimental-pic", - "--no-export-dynamic", + "--export-dynamic", "--stack-first", + "--allow-undefined", OutputFileName.c_str(), "-o", OutputFileName.c_str()}; @@ -109,6 +110,12 @@ llvm::Error WasmIncrementalExecutor::runCtors() const { return llvm::Error::success(); } +llvm::Error WasmIncrementalExecutor::cleanUp() const { + // Can't call cleanUp through IncrementalExecutor as it + // tries to deinitialize JIT which hasn't been initialized + return llvm::Error::success(); +} + WasmIncrementalExecutor::~WasmIncrementalExecutor() = default; } // namespace clang diff --git a/clang/lib/Interpreter/Wasm.h b/clang/lib/Interpreter/Wasm.h index b1fd88024f14d7..4632613326d39b 100644 --- a/clang/lib/Interpreter/Wasm.h +++ b/clang/lib/Interpreter/Wasm.h @@ -28,6 +28,7 @@ class WasmIncrementalExecutor : public IncrementalExecutor { llvm::Error addModule(PartialTranslationUnit &PTU) override; llvm::Error removeModule(PartialTranslationUnit &PTU) override; llvm::Error runCtors() const override; + llvm::Error cleanUp() override; ~WasmIncrementalExecutor() override; }; ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Use default visibility for symbols while building CompilerInstance against emscripten (PR #116779)
anutosh491 wrote: I just realized that cleanUp shouldn't use const. While building against emscripten, I go this error ``` │ │ /home/runner/work/recipes/recipes/output/bld/rattler-build_llvm_1732028457/work/clang/lib/Interpreter/Wasm.cpp:111:38: error: out-of-line definition of 'cl │ │ eanUp' does not match any declaration in 'clang::WasmIncrementalExecutor' │ │ 111 | llvm::Error WasmIncrementalExecutor::cleanUp() const { │ │ | ^~~ │ │ /home/runner/work/recipes/recipes/output/bld/rattler-build_llvm_1732028457/work/clang/lib/Interpreter/Wasm.h:31:15: note: member declaration does not match │ │ because it is not const qualified │ │31 | llvm::Error cleanUp() override; │ │ | ^ ``` Fixed this through the latest commit. https://github.com/llvm/llvm-project/pull/116779 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Use default visibility for symbols while building CompilerInstance against emscripten (PR #116779)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/116779 >From 616ebd6b6487eeaa302e6d33de3a371e6c9843f5 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Tue, 19 Nov 2024 15:38:37 +0530 Subject: [PATCH 1/2] Use default visibility for symbols while building CompilerInstance against emscripten --- clang/lib/Interpreter/Interpreter.cpp | 1 + clang/lib/Interpreter/Wasm.cpp| 1 - 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 73ad766e655180..999271aae7491d 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -197,6 +197,7 @@ IncrementalCompilerBuilder::CreateCpp() { Argv.push_back("-target"); Argv.push_back("wasm32-unknown-emscripten"); Argv.push_back("-shared"); + Argv.push_back("-fvisibility=default") #endif Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end()); diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index 79efbaa03982d0..6d4cc478dd6a85 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -75,7 +75,6 @@ llvm::Error WasmIncrementalExecutor::addModule(PartialTranslationUnit &PTU) { "-shared", "--import-memory", "--no-entry", - "--export-all", "--experimental-pic", "--stack-first", "--allow-undefined", >From fedc699c66ba973798bd452b5a6607d412e4cb75 Mon Sep 17 00:00:00 2001 From: Anutosh Bhat Date: Tue, 19 Nov 2024 17:27:59 +0530 Subject: [PATCH 2/2] Update Interpreter.cpp --- clang/lib/Interpreter/Interpreter.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 999271aae7491d..00111daab01e37 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -197,7 +197,7 @@ IncrementalCompilerBuilder::CreateCpp() { Argv.push_back("-target"); Argv.push_back("wasm32-unknown-emscripten"); Argv.push_back("-shared"); - Argv.push_back("-fvisibility=default") + Argv.push_back("-fvisibility=default"); #endif Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end()); ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Use default visibility for symbols while building CompilerInstance against emscripten (PR #116779)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/116779 >From 616ebd6b6487eeaa302e6d33de3a371e6c9843f5 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Tue, 19 Nov 2024 15:38:37 +0530 Subject: [PATCH 1/2] Use default visibility for symbols while building CompilerInstance against emscripten --- clang/lib/Interpreter/Interpreter.cpp | 1 + clang/lib/Interpreter/Wasm.cpp| 1 - 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 73ad766e655180..999271aae7491d 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -197,6 +197,7 @@ IncrementalCompilerBuilder::CreateCpp() { Argv.push_back("-target"); Argv.push_back("wasm32-unknown-emscripten"); Argv.push_back("-shared"); + Argv.push_back("-fvisibility=default") #endif Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end()); diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index 79efbaa03982d0..6d4cc478dd6a85 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -75,7 +75,6 @@ llvm::Error WasmIncrementalExecutor::addModule(PartialTranslationUnit &PTU) { "-shared", "--import-memory", "--no-entry", - "--export-all", "--experimental-pic", "--stack-first", "--allow-undefined", >From fedc699c66ba973798bd452b5a6607d412e4cb75 Mon Sep 17 00:00:00 2001 From: Anutosh Bhat Date: Tue, 19 Nov 2024 17:27:59 +0530 Subject: [PATCH 2/2] Update Interpreter.cpp --- clang/lib/Interpreter/Interpreter.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 999271aae7491d..00111daab01e37 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -197,7 +197,7 @@ IncrementalCompilerBuilder::CreateCpp() { Argv.push_back("-target"); Argv.push_back("wasm32-unknown-emscripten"); Argv.push_back("-shared"); - Argv.push_back("-fvisibility=default") + Argv.push_back("-fvisibility=default"); #endif Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end()); ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Use default visibility for symbols while building CompilerInstance against emscripten (PR #116779)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/116779 >From 616ebd6b6487eeaa302e6d33de3a371e6c9843f5 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Tue, 19 Nov 2024 15:38:37 +0530 Subject: [PATCH 1/3] Use default visibility for symbols while building CompilerInstance against emscripten --- clang/lib/Interpreter/Interpreter.cpp | 1 + clang/lib/Interpreter/Wasm.cpp| 1 - 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 73ad766e655180..999271aae7491d 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -197,6 +197,7 @@ IncrementalCompilerBuilder::CreateCpp() { Argv.push_back("-target"); Argv.push_back("wasm32-unknown-emscripten"); Argv.push_back("-shared"); + Argv.push_back("-fvisibility=default") #endif Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end()); diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index 79efbaa03982d0..6d4cc478dd6a85 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -75,7 +75,6 @@ llvm::Error WasmIncrementalExecutor::addModule(PartialTranslationUnit &PTU) { "-shared", "--import-memory", "--no-entry", - "--export-all", "--experimental-pic", "--stack-first", "--allow-undefined", >From fedc699c66ba973798bd452b5a6607d412e4cb75 Mon Sep 17 00:00:00 2001 From: Anutosh Bhat Date: Tue, 19 Nov 2024 17:27:59 +0530 Subject: [PATCH 2/3] Update Interpreter.cpp --- clang/lib/Interpreter/Interpreter.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 999271aae7491d..00111daab01e37 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -197,7 +197,7 @@ IncrementalCompilerBuilder::CreateCpp() { Argv.push_back("-target"); Argv.push_back("wasm32-unknown-emscripten"); Argv.push_back("-shared"); - Argv.push_back("-fvisibility=default") + Argv.push_back("-fvisibility=default"); #endif Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end()); >From 4f05817d03f7bf5a9ec201c8de551810dca176f5 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Tue, 19 Nov 2024 21:28:22 +0530 Subject: [PATCH 3/3] fix cleanUp --- clang/lib/Interpreter/Wasm.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index 6d4cc478dd6a85..8d4bc20bcfe54f 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -108,7 +108,7 @@ llvm::Error WasmIncrementalExecutor::runCtors() const { return llvm::Error::success(); } -llvm::Error WasmIncrementalExecutor::cleanUp() const { +llvm::Error WasmIncrementalExecutor::cleanUp() { // Can't call cleanUp through IncrementalExecutor as it // tries to deinitialize JIT which hasn't been initialized return llvm::Error::success(); ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Use default visibility for symbols while building CompilerInstance against emscripten (PR #116779)
https://github.com/anutosh491 created https://github.com/llvm/llvm-project/pull/116779 When running clang-repl in the browser we end up having something like the following ` "" -cc1 -triple wasm32-unknown-emscripten .. -main-file-name "<<< inputs >>>" -fvisibility=hidden -o "<<< inputs >>>.o" -x c++ "<<< inputs >>>" ` Due to the default introduced through this commit (https://github.com/llvm/llvm-project/commit/e3d71e14d756d78f9e2538f2e530aa7c051541cd#diff-b5496baaf5c83e1ffa1a26d0815843b8d3224aba84366cbb6aeecf703808c803R2083) That being said, when we generated the shared libraries to be loaded on top of the main module while running clang-repl in the browser, we have to surpass the above through `--export-all` https://github.com/llvm/llvm-project/blob/b3e2b1a7eb258a7c9c55691d08342eface083499/clang/lib/Interpreter/Wasm.cpp#L78 This is because obviously `incr_module_XX.wasm` might want to access symbols out of file from `incr_module_XX-1.wasm` to `incr_mdoule_0.wasm` But this also exports some problematic things like `__dso_handle` that causes conflicts across modules.  A better way to deal with this is to pass `-fvisibility=default` to the `CompilerInstance`. >From 616ebd6b6487eeaa302e6d33de3a371e6c9843f5 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Tue, 19 Nov 2024 15:38:37 +0530 Subject: [PATCH] Use default visibility for symbols while building CompilerInstance against emscripten --- clang/lib/Interpreter/Interpreter.cpp | 1 + clang/lib/Interpreter/Wasm.cpp| 1 - 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 73ad766e655180..999271aae7491d 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -197,6 +197,7 @@ IncrementalCompilerBuilder::CreateCpp() { Argv.push_back("-target"); Argv.push_back("wasm32-unknown-emscripten"); Argv.push_back("-shared"); + Argv.push_back("-fvisibility=default") #endif Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end()); diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index 79efbaa03982d0..6d4cc478dd6a85 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -75,7 +75,6 @@ llvm::Error WasmIncrementalExecutor::addModule(PartialTranslationUnit &PTU) { "-shared", "--import-memory", "--no-entry", - "--export-all", "--experimental-pic", "--stack-first", "--allow-undefined", ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Remove redundant shared flag while running clang-repl in browser (PR #118107)
anutosh491 wrote: cc @vgvassilev Now that 19.1.5 is out. We can run clang-repl in the browser. And this flag is just adding redundancy and doesn't play a role. Small change. Should be ready ! https://github.com/llvm/llvm-project/pull/118107 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Implement value printing of custom types (PR #84769)
anutosh491 wrote: Hey @vgvassilev Just curious to know as to what work is left here and if any help is needed for the same ? https://github.com/llvm/llvm-project/pull/84769 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Define LLVM_ABI and CLANG_ABI for __EMSCRIPTEN__ builds (PR #131578)
anutosh491 wrote: Hey, Thanks for the review. The CI is green now ! https://github.com/llvm/llvm-project/pull/131578 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Define LLVM_ABI and CLANG_ABI for __EMSCRIPTEN__ builds (PR #131578)
anutosh491 wrote: /cherry-pick [e57cd10](https://github.com/llvm/llvm-project/commit/e57cd100ca297cf81854e35cccbf703e4aad) https://github.com/llvm/llvm-project/pull/131578 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Define LLVM_ABI and CLANG_ABI for __EMSCRIPTEN__ builds (PR #131578)
https://github.com/anutosh491 edited https://github.com/llvm/llvm-project/pull/131578 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Address error recovery fixing infinite loop while parsing (PR #127569)
anutosh491 wrote: Hi @vgvassilev, Would be nice to know your thoughts here https://github.com/llvm/llvm-project/pull/127569 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Define LLVM_ABI and CLANG_ABI for __EMSCRIPTEN__ builds (PR #131578)
anutosh491 wrote: Hey @serge-sans-paille One final request here would be if you could add the milestone (**LLVM 20.X Release** I think) So that this is picked up in the next release (20.0.2) and we wouldn't need to use the patch then ! https://github.com/llvm/llvm-project/pull/131578 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [Clang-repl] Implementation for removeModule for wasm use case (PR #131558)
anutosh491 wrote: Has some debug logs for now (shall be removed once ready) Also the issue tracking this on emscripten is https://github.com/emscripten-core/emscripten/issues/23793 https://github.com/llvm/llvm-project/pull/131558 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [Clang-repl] Implementation for removeModule for wasm use case (PR #131558)
https://github.com/anutosh491 created https://github.com/llvm/llvm-project/pull/131558 Raising as draft for now. This PR implements the removeModule function that can be put to use while running clang-repl in the browser. The implementation works as we would like it to. But due to some issues in how the dlopen-dlclose pair works for the wasm case (as compared to the native case) this has some shortcomings. Check the following video to understand why I say so https://github.com/user-attachments/assets/a120e767-79c0-4c41-bbb0-f086e6af883b >From 56d3e301d01cb8ab8f44751d7684378f38b0661e Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Fri, 28 Feb 2025 12:11:28 +0530 Subject: [PATCH] Implementation for removeModule --- clang/lib/Interpreter/Wasm.cpp | 37 -- clang/lib/Interpreter/Wasm.h | 4 2 files changed, 39 insertions(+), 2 deletions(-) diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index aa10b160ccf84..ed746beefd0f5 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -125,12 +125,45 @@ llvm::Error WasmIncrementalExecutor::addModule(PartialTranslationUnit &PTU) { "Failed to load incremental module", llvm::inconvertibleErrorCode()); } + LoadedModules[BinaryFileName] = LoadedLibModule; + return llvm::Error::success(); } llvm::Error WasmIncrementalExecutor::removeModule(PartialTranslationUnit &PTU) { - return llvm::make_error("Not implemented yet", - llvm::inconvertibleErrorCode()); + std::string BinaryFileName = PTU.TheModule->getName().str() + ".wasm"; + + llvm::errs() << "[DEBUG] Entering removeModule() for: " << BinaryFileName << "\n"; + llvm::errs() << "[DEBUG] Current Loaded Modules:\n"; + for (const auto &entry : LoadedModules) { +llvm::errs() << " -> " << entry.first << "\n"; + } + + auto It = LoadedModules.find(BinaryFileName); + if (It == LoadedModules.end()) { +llvm::errs() << "[ERROR] Module not found in LoadedModules!\n"; +return llvm::make_error( +"Module not found in loaded modules", llvm::inconvertibleErrorCode()); + } + + llvm::errs() << "[DEBUG] Unloading module: " << BinaryFileName << "\n"; + // Attempt to unload the module + + if (dlclose(It->second) != 0) { +llvm::errs() << "Failed to unload module: " << dlerror() << '\n'; +return llvm::make_error( +"Failed to unload module", llvm::inconvertibleErrorCode()); + } + + // Remove the module from tracking + LoadedModules.erase(It); + + llvm::errs() << "[DEBUG] Remaining Loaded Modules:\n"; + for (const auto &entry : LoadedModules) { +llvm::errs() << " -> " << entry.first << "\n"; + } + + return llvm::Error::success(); } llvm::Error WasmIncrementalExecutor::runCtors() const { diff --git a/clang/lib/Interpreter/Wasm.h b/clang/lib/Interpreter/Wasm.h index 4632613326d39..7bf597bbf0b0c 100644 --- a/clang/lib/Interpreter/Wasm.h +++ b/clang/lib/Interpreter/Wasm.h @@ -31,6 +31,10 @@ class WasmIncrementalExecutor : public IncrementalExecutor { llvm::Error cleanUp() override; ~WasmIncrementalExecutor() override; + +private: + // Tracks loaded modules and their handles + std::unordered_map LoadedModules; }; } // namespace clang ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Define LLVM_ABI and CLANG_ABI for __EMSCRIPTEN__ builds (PR #131578)
https://github.com/anutosh491 created https://github.com/llvm/llvm-project/pull/131578 While building llvm (clang, lld) against emscripten we see this [error](https://github.com/emscripten-forge/recipes/actions/runs/13803029307/job/38608794602#step:9:1715) ``` │ │ In file included from $SRC_DIR/llvm/lib/Frontend/OpenACC/ACC.cpp:9: │ │ $SRC_DIR/build/include/llvm/Frontend/OpenACC/ACC.h.inc:192:1: error: unknown type name 'LLVM_ABI' │ │ 192 | LLVM_ABI Directive getOpenACCDirectiveKind(llvm::StringRef Str); │ │ | ^ │ │ $SRC_DIR/build/include/llvm/Frontend/OpenACC/ACC.h.inc:192:19: error: expected ';' after top level declarator │ │ 192 | LLVM_ABI Directive getOpenACCDirectiveKind(llvm::StringRef Str); │ │ | ^ ``` Now this was happening because we weren't defining LLVM_ABI correctly when building against emscripten. If you see [llvm/Support/Compiler.h](https://github.com/llvm/llvm-project/blob/main/llvm/include/llvm/Support/Compiler.h#L206-L210), the condition only checked for the platform __WASM__ . Now Emscripten targets WebAssembly but doesn't imply the platform by default so the check isn't complete to define LLVM_ABI. >From ed4c4946bdcce63eda9b2ef9072210197202c3a6 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Mon, 17 Mar 2025 13:22:49 +0530 Subject: [PATCH] Define LLVM_ABI and CLANG_ABI for __EMSCRIPTEN__ builds --- clang/include/clang/Support/Compiler.h | 2 +- llvm/include/llvm/Support/Compiler.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/clang/include/clang/Support/Compiler.h b/clang/include/clang/Support/Compiler.h index 13582b899dc2a..5a74f8e3b6723 100644 --- a/clang/include/clang/Support/Compiler.h +++ b/clang/include/clang/Support/Compiler.h @@ -54,7 +54,7 @@ #define CLANG_ABI LLVM_ATTRIBUTE_VISIBILITY_DEFAULT #define CLANG_TEMPLATE_ABI LLVM_ATTRIBUTE_VISIBILITY_DEFAULT #define CLANG_EXPORT_TEMPLATE -#elif defined(__MACH__) || defined(__WASM__) +#elif defined(__MACH__) || defined(__WASM__) || defined(__EMSCRIPTEN__) #define CLANG_ABI LLVM_ATTRIBUTE_VISIBILITY_DEFAULT #define CLANG_TEMPLATE_ABI #define CLANG_EXPORT_TEMPLATE diff --git a/llvm/include/llvm/Support/Compiler.h b/llvm/include/llvm/Support/Compiler.h index f9c57b89f1f03..dc8b5389069eb 100644 --- a/llvm/include/llvm/Support/Compiler.h +++ b/llvm/include/llvm/Support/Compiler.h @@ -203,7 +203,7 @@ #define LLVM_TEMPLATE_ABI LLVM_ATTRIBUTE_VISIBILITY_DEFAULT #define LLVM_EXPORT_TEMPLATE #define LLVM_ABI_EXPORT LLVM_ATTRIBUTE_VISIBILITY_DEFAULT -#elif defined(__MACH__) || defined(__WASM__) +#elif defined(__MACH__) || defined(__WASM__) || defined(__EMSCRIPTEN__) #define LLVM_ABI LLVM_ATTRIBUTE_VISIBILITY_DEFAULT #define LLVM_TEMPLATE_ABI #define LLVM_EXPORT_TEMPLATE ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Define LLVM_ABI and CLANG_ABI for __EMSCRIPTEN__ builds (PR #131578)
https://github.com/anutosh491 ready_for_review https://github.com/llvm/llvm-project/pull/131578 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Define LLVM_ABI and CLANG_ABI for __EMSCRIPTEN__ builds (PR #131578)
anutosh491 wrote: cc @serge-sans-paille for review. https://github.com/llvm/llvm-project/pull/131578 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Define LLVM_ABI and CLANG_ABI for __EMSCRIPTEN__ builds (PR #131578)
https://github.com/anutosh491 edited https://github.com/llvm/llvm-project/pull/131578 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Implement LoadDynamicLibrary for clang-repl wasm use cases (PR #133037)
https://github.com/anutosh491 edited https://github.com/llvm/llvm-project/pull/133037 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Implement LoadDynamicLibrary for clang-repl wasm use cases (PR #133037)
https://github.com/anutosh491 edited https://github.com/llvm/llvm-project/pull/133037 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Implement LoadDynamicLibrary for clang-repl wasm use cases (PR #133037)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/133037 >From dfe49e826705a5e9371e17e66e40c31602beea8e Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Wed, 26 Mar 2025 10:33:37 +0530 Subject: [PATCH 1/2] Implement LoadDynamicLibrary for clang-repl wasm use cases --- clang/lib/Interpreter/Interpreter.cpp | 10 ++ 1 file changed, 10 insertions(+) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index fa4c1439c9261..b4c68f340abc0 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -18,6 +18,7 @@ #include "llvm/Support/VirtualFileSystem.h" #ifdef __EMSCRIPTEN__ #include "Wasm.h" +#include #endif // __EMSCRIPTEN__ #include "clang/AST/ASTConsumer.h" @@ -711,6 +712,14 @@ llvm::Error Interpreter::Undo(unsigned N) { } llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { +#ifdef __EMSCRIPTEN__ + void* handle = dlopen(name, RTLD_NOW | RTLD_GLOBAL); + if (!handle) { +llvm::errs() << dlerror() << '\n'; +return llvm::make_error( +"Failed to load dynamic library", llvm::inconvertibleErrorCode()); + } +#else auto EE = getExecutionEngine(); if (!EE) return EE.takeError(); @@ -722,6 +731,7 @@ llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { EE->getMainJITDylib().addGenerator(std::move(*DLSG)); else return DLSG.takeError(); +#endif return llvm::Error::success(); } >From cb274780d84c5355c4aede7b8fe7501dc300bc02 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Wed, 26 Mar 2025 11:14:06 +0530 Subject: [PATCH 2/2] formatting --- clang/lib/Interpreter/Interpreter.cpp | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index b4c68f340abc0..f8c8d0a425659 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -713,11 +713,11 @@ llvm::Error Interpreter::Undo(unsigned N) { llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { #ifdef __EMSCRIPTEN__ - void* handle = dlopen(name, RTLD_NOW | RTLD_GLOBAL); + void *handle = dlopen(name, RTLD_NOW | RTLD_GLOBAL); if (!handle) { llvm::errs() << dlerror() << '\n'; -return llvm::make_error( -"Failed to load dynamic library", llvm::inconvertibleErrorCode()); +return llvm::make_error("Failed to load dynamic library", + llvm::inconvertibleErrorCode()); } #else auto EE = getExecutionEngine(); ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Implement LoadDynamicLibrary for clang-repl wasm use cases (PR #133037)
anutosh491 wrote: For anyone concerned, Such a feature would be tested out in downstream projects like [cppinterop](https://github.com/compiler-research/CppInterOp) and [xeus-cpp](https://github.com/compiler-research/xeus-cpp) as currently llvm doesn't have a concrete structure to test clang-repl in the browser. https://github.com/llvm/llvm-project/pull/133037 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Implement LoadDynamicLibrary for clang-repl wasm use cases (PR #133037)
https://github.com/anutosh491 created https://github.com/llvm/llvm-project/pull/133037 None >From dfe49e826705a5e9371e17e66e40c31602beea8e Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Wed, 26 Mar 2025 10:33:37 +0530 Subject: [PATCH] Implement LoadDynamicLibrary for clang-repl wasm use cases --- clang/lib/Interpreter/Interpreter.cpp | 10 ++ 1 file changed, 10 insertions(+) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index fa4c1439c9261..b4c68f340abc0 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -18,6 +18,7 @@ #include "llvm/Support/VirtualFileSystem.h" #ifdef __EMSCRIPTEN__ #include "Wasm.h" +#include #endif // __EMSCRIPTEN__ #include "clang/AST/ASTConsumer.h" @@ -711,6 +712,14 @@ llvm::Error Interpreter::Undo(unsigned N) { } llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { +#ifdef __EMSCRIPTEN__ + void* handle = dlopen(name, RTLD_NOW | RTLD_GLOBAL); + if (!handle) { +llvm::errs() << dlerror() << '\n'; +return llvm::make_error( +"Failed to load dynamic library", llvm::inconvertibleErrorCode()); + } +#else auto EE = getExecutionEngine(); if (!EE) return EE.takeError(); @@ -722,6 +731,7 @@ llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { EE->getMainJITDylib().addGenerator(std::move(*DLSG)); else return DLSG.takeError(); +#endif return llvm::Error::success(); } ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Implement LoadDynamicLibrary for clang-repl wasm use cases (PR #133037)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/133037 >From a7ccd0e0b261c303639d3880fc51b0ed99e17363 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Wed, 26 Mar 2025 10:33:37 +0530 Subject: [PATCH 1/3] Implement LoadDynamicLibrary for clang-repl wasm use cases --- clang/lib/Interpreter/Interpreter.cpp | 10 ++ 1 file changed, 10 insertions(+) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index fa4c1439c9261..f8c8d0a425659 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -18,6 +18,7 @@ #include "llvm/Support/VirtualFileSystem.h" #ifdef __EMSCRIPTEN__ #include "Wasm.h" +#include #endif // __EMSCRIPTEN__ #include "clang/AST/ASTConsumer.h" @@ -711,6 +712,14 @@ llvm::Error Interpreter::Undo(unsigned N) { } llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { +#ifdef __EMSCRIPTEN__ + void *handle = dlopen(name, RTLD_NOW | RTLD_GLOBAL); + if (!handle) { +llvm::errs() << dlerror() << '\n'; +return llvm::make_error("Failed to load dynamic library", + llvm::inconvertibleErrorCode()); + } +#else auto EE = getExecutionEngine(); if (!EE) return EE.takeError(); @@ -722,6 +731,7 @@ llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { EE->getMainJITDylib().addGenerator(std::move(*DLSG)); else return DLSG.takeError(); +#endif return llvm::Error::success(); } >From bc41b71965e1e33a32d1a54fd79148d1b6890dce Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Thu, 27 Mar 2025 12:34:23 +0530 Subject: [PATCH 2/3] Override getSymbolAddress --- clang/lib/Interpreter/IncrementalExecutor.h | 2 +- clang/lib/Interpreter/Wasm.cpp | 13 + clang/lib/Interpreter/Wasm.h| 2 ++ 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/clang/lib/Interpreter/IncrementalExecutor.h b/clang/lib/Interpreter/IncrementalExecutor.h index dbd61f0b8b1eb..71d71bc3883e2 100644 --- a/clang/lib/Interpreter/IncrementalExecutor.h +++ b/clang/lib/Interpreter/IncrementalExecutor.h @@ -57,7 +57,7 @@ class IncrementalExecutor { virtual llvm::Error removeModule(PartialTranslationUnit &PTU); virtual llvm::Error runCtors() const; virtual llvm::Error cleanUp(); - llvm::Expected + virtual llvm::Expected getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const; llvm::orc::LLJIT &GetExecutionEngine() { return *Jit; } diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index f7cb7598c77f8..0543a3504c9a2 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -144,6 +144,19 @@ llvm::Error WasmIncrementalExecutor::cleanUp() { return llvm::Error::success(); } +llvm::Expected +WasmIncrementalExecutor::getSymbolAddress(llvm::StringRef Name, + SymbolNameKind NameKind) const { + void *Sym = dlsym(RTLD_DEFAULT, Name.str().c_str()); + if (!Sym) { +return llvm::make_error("dlsym failed for symbol: " + + Name.str(), + llvm::inconvertibleErrorCode()); + } + + return llvm::orc::ExecutorAddr::fromPtr(Sym); +} + WasmIncrementalExecutor::~WasmIncrementalExecutor() = default; } // namespace clang diff --git a/clang/lib/Interpreter/Wasm.h b/clang/lib/Interpreter/Wasm.h index 4632613326d39..beafb01be7d8a 100644 --- a/clang/lib/Interpreter/Wasm.h +++ b/clang/lib/Interpreter/Wasm.h @@ -29,6 +29,8 @@ class WasmIncrementalExecutor : public IncrementalExecutor { llvm::Error removeModule(PartialTranslationUnit &PTU) override; llvm::Error runCtors() const override; llvm::Error cleanUp() override; + llvm::Expected + getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const; ~WasmIncrementalExecutor() override; }; >From 60d4622e23902bf7c5fd1040c9f82f9b03e134ce Mon Sep 17 00:00:00 2001 From: Anutosh Bhat Date: Thu, 27 Mar 2025 14:05:35 +0530 Subject: [PATCH 3/3] Update Wasm.h --- clang/lib/Interpreter/Wasm.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/lib/Interpreter/Wasm.h b/clang/lib/Interpreter/Wasm.h index beafb01be7d8a..d27455ec42c02 100644 --- a/clang/lib/Interpreter/Wasm.h +++ b/clang/lib/Interpreter/Wasm.h @@ -30,7 +30,7 @@ class WasmIncrementalExecutor : public IncrementalExecutor { llvm::Error runCtors() const override; llvm::Error cleanUp() override; llvm::Expected - getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const; + getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const override;; ~WasmIncrementalExecutor() override; }; ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Implement LoadDynamicLibrary for clang-repl wasm use cases (PR #133037)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/133037 >From a7ccd0e0b261c303639d3880fc51b0ed99e17363 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Wed, 26 Mar 2025 10:33:37 +0530 Subject: [PATCH 1/2] Implement LoadDynamicLibrary for clang-repl wasm use cases --- clang/lib/Interpreter/Interpreter.cpp | 10 ++ 1 file changed, 10 insertions(+) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index fa4c1439c9261..f8c8d0a425659 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -18,6 +18,7 @@ #include "llvm/Support/VirtualFileSystem.h" #ifdef __EMSCRIPTEN__ #include "Wasm.h" +#include #endif // __EMSCRIPTEN__ #include "clang/AST/ASTConsumer.h" @@ -711,6 +712,14 @@ llvm::Error Interpreter::Undo(unsigned N) { } llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { +#ifdef __EMSCRIPTEN__ + void *handle = dlopen(name, RTLD_NOW | RTLD_GLOBAL); + if (!handle) { +llvm::errs() << dlerror() << '\n'; +return llvm::make_error("Failed to load dynamic library", + llvm::inconvertibleErrorCode()); + } +#else auto EE = getExecutionEngine(); if (!EE) return EE.takeError(); @@ -722,6 +731,7 @@ llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { EE->getMainJITDylib().addGenerator(std::move(*DLSG)); else return DLSG.takeError(); +#endif return llvm::Error::success(); } >From bc41b71965e1e33a32d1a54fd79148d1b6890dce Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Thu, 27 Mar 2025 12:34:23 +0530 Subject: [PATCH 2/2] Override getSymbolAddress --- clang/lib/Interpreter/IncrementalExecutor.h | 2 +- clang/lib/Interpreter/Wasm.cpp | 13 + clang/lib/Interpreter/Wasm.h| 2 ++ 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/clang/lib/Interpreter/IncrementalExecutor.h b/clang/lib/Interpreter/IncrementalExecutor.h index dbd61f0b8b1eb..71d71bc3883e2 100644 --- a/clang/lib/Interpreter/IncrementalExecutor.h +++ b/clang/lib/Interpreter/IncrementalExecutor.h @@ -57,7 +57,7 @@ class IncrementalExecutor { virtual llvm::Error removeModule(PartialTranslationUnit &PTU); virtual llvm::Error runCtors() const; virtual llvm::Error cleanUp(); - llvm::Expected + virtual llvm::Expected getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const; llvm::orc::LLJIT &GetExecutionEngine() { return *Jit; } diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index f7cb7598c77f8..0543a3504c9a2 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -144,6 +144,19 @@ llvm::Error WasmIncrementalExecutor::cleanUp() { return llvm::Error::success(); } +llvm::Expected +WasmIncrementalExecutor::getSymbolAddress(llvm::StringRef Name, + SymbolNameKind NameKind) const { + void *Sym = dlsym(RTLD_DEFAULT, Name.str().c_str()); + if (!Sym) { +return llvm::make_error("dlsym failed for symbol: " + + Name.str(), + llvm::inconvertibleErrorCode()); + } + + return llvm::orc::ExecutorAddr::fromPtr(Sym); +} + WasmIncrementalExecutor::~WasmIncrementalExecutor() = default; } // namespace clang diff --git a/clang/lib/Interpreter/Wasm.h b/clang/lib/Interpreter/Wasm.h index 4632613326d39..beafb01be7d8a 100644 --- a/clang/lib/Interpreter/Wasm.h +++ b/clang/lib/Interpreter/Wasm.h @@ -29,6 +29,8 @@ class WasmIncrementalExecutor : public IncrementalExecutor { llvm::Error removeModule(PartialTranslationUnit &PTU) override; llvm::Error runCtors() const override; llvm::Error cleanUp() override; + llvm::Expected + getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const; ~WasmIncrementalExecutor() override; }; ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Implement LoadDynamicLibrary for clang-repl wasm use cases (PR #133037)
@@ -711,6 +712,14 @@ llvm::Error Interpreter::Undo(unsigned N) { } llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { +#ifdef __EMSCRIPTEN__ + void *handle = dlopen(name, RTLD_NOW | RTLD_GLOBAL); + if (!handle) { +llvm::errs() << dlerror() << '\n'; +return llvm::make_error("Failed to load dynamic library", + llvm::inconvertibleErrorCode()); + } +#else anutosh491 wrote: Additionally, path resolution isn't a concern in the wasm case for a few reasons (which is why a simple combination of dlopen + dlsym should do the job isn't it ?) 1) All dynamic libraries are preloaded into MEMFS at known paths during compilation or initialization (e.g., via Emscripten’s --preload-file flag). This means the library’s location is fully deterministic at runtime. 2) No concept of system-level search paths like LD_LIBRARY_PATH in the browser — we can’t rely on an OS-level dynamic loader. Instead, we always load by the full path (e.g., dlopen("/libsymengine.so")), which resolves directly in the virtual MEMFS. 3) Because users control the preload path, and because dlopen in Emscripten expects an exact match in the in-memory filesystem, we don’t need infrastructure for path lookup, search path prioritization, or canonicalization. A simple, direct path-based dlopen is all that’s required. https://github.com/llvm/llvm-project/pull/133037 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Implement LoadDynamicLibrary for clang-repl wasm use cases (PR #133037)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/133037 >From dfe49e826705a5e9371e17e66e40c31602beea8e Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Wed, 26 Mar 2025 10:33:37 +0530 Subject: [PATCH 1/4] Implement LoadDynamicLibrary for clang-repl wasm use cases --- clang/lib/Interpreter/Interpreter.cpp | 10 ++ 1 file changed, 10 insertions(+) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index fa4c1439c9261..b4c68f340abc0 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -18,6 +18,7 @@ #include "llvm/Support/VirtualFileSystem.h" #ifdef __EMSCRIPTEN__ #include "Wasm.h" +#include #endif // __EMSCRIPTEN__ #include "clang/AST/ASTConsumer.h" @@ -711,6 +712,14 @@ llvm::Error Interpreter::Undo(unsigned N) { } llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { +#ifdef __EMSCRIPTEN__ + void* handle = dlopen(name, RTLD_NOW | RTLD_GLOBAL); + if (!handle) { +llvm::errs() << dlerror() << '\n'; +return llvm::make_error( +"Failed to load dynamic library", llvm::inconvertibleErrorCode()); + } +#else auto EE = getExecutionEngine(); if (!EE) return EE.takeError(); @@ -722,6 +731,7 @@ llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { EE->getMainJITDylib().addGenerator(std::move(*DLSG)); else return DLSG.takeError(); +#endif return llvm::Error::success(); } >From cb274780d84c5355c4aede7b8fe7501dc300bc02 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Wed, 26 Mar 2025 11:14:06 +0530 Subject: [PATCH 2/4] formatting --- clang/lib/Interpreter/Interpreter.cpp | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index b4c68f340abc0..f8c8d0a425659 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -713,11 +713,11 @@ llvm::Error Interpreter::Undo(unsigned N) { llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { #ifdef __EMSCRIPTEN__ - void* handle = dlopen(name, RTLD_NOW | RTLD_GLOBAL); + void *handle = dlopen(name, RTLD_NOW | RTLD_GLOBAL); if (!handle) { llvm::errs() << dlerror() << '\n'; -return llvm::make_error( -"Failed to load dynamic library", llvm::inconvertibleErrorCode()); +return llvm::make_error("Failed to load dynamic library", + llvm::inconvertibleErrorCode()); } #else auto EE = getExecutionEngine(); >From 0911ef1d634603718d5418694a02b0e9665d4552 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Thu, 27 Mar 2025 12:34:23 +0530 Subject: [PATCH 3/4] Override getSymbolAddress --- clang/lib/Interpreter/IncrementalExecutor.h | 2 +- clang/lib/Interpreter/Wasm.cpp | 13 + clang/lib/Interpreter/Wasm.h| 2 ++ 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/clang/lib/Interpreter/IncrementalExecutor.h b/clang/lib/Interpreter/IncrementalExecutor.h index dbd61f0b8b1eb..71d71bc3883e2 100644 --- a/clang/lib/Interpreter/IncrementalExecutor.h +++ b/clang/lib/Interpreter/IncrementalExecutor.h @@ -57,7 +57,7 @@ class IncrementalExecutor { virtual llvm::Error removeModule(PartialTranslationUnit &PTU); virtual llvm::Error runCtors() const; virtual llvm::Error cleanUp(); - llvm::Expected + virtual llvm::Expected getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const; llvm::orc::LLJIT &GetExecutionEngine() { return *Jit; } diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index f7cb7598c77f8..c91fe950d428d 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -144,6 +144,19 @@ llvm::Error WasmIncrementalExecutor::cleanUp() { return llvm::Error::success(); } +llvm::Expected +WasmIncrementalExecutor::getSymbolAddress(llvm::StringRef Name, + SymbolNameKind NameKind) const { + void* Sym = dlsym(RTLD_DEFAULT, Name.str().c_str()); + if (!Sym) { +return llvm::make_error( +"dlsym failed for symbol: " + Name.str(), +llvm::inconvertibleErrorCode()); + } + + return llvm::orc::ExecutorAddr::fromPtr(Sym); +} + WasmIncrementalExecutor::~WasmIncrementalExecutor() = default; } // namespace clang diff --git a/clang/lib/Interpreter/Wasm.h b/clang/lib/Interpreter/Wasm.h index 4632613326d39..beafb01be7d8a 100644 --- a/clang/lib/Interpreter/Wasm.h +++ b/clang/lib/Interpreter/Wasm.h @@ -29,6 +29,8 @@ class WasmIncrementalExecutor : public IncrementalExecutor { llvm::Error removeModule(PartialTranslationUnit &PTU) override; llvm::Error runCtors() const override; llvm::Error cleanUp() override; + llvm::Expected + getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const; ~WasmIncrementalExecutor() override; }; >From b717060a8a0cf85e2ca218d7c303
[clang] [clang-repl] Implement LoadDynamicLibrary for clang-repl wasm use cases (PR #133037)
@@ -711,6 +712,14 @@ llvm::Error Interpreter::Undo(unsigned N) { } llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { +#ifdef __EMSCRIPTEN__ + void *handle = dlopen(name, RTLD_NOW | RTLD_GLOBAL); + if (!handle) { +llvm::errs() << dlerror() << '\n'; +return llvm::make_error("Failed to load dynamic library", + llvm::inconvertibleErrorCode()); + } +#else anutosh491 wrote: Hi, thanks for the suggestion. I did some surface level introspection on `DynamicLibrarySearchGenerator::Load` and I see it calls dlopen internally but I think the handle is wrapped in a DynamicLibrarySearchGenerator, which lets the ORC JIT symbol resolution system look inside the .so for symbols during later getSymbolAddress(...) calls. So even if DynamicLibrarySearchGenerator::Load(...) works under WASM (which it might), it: i) Creates a JITDylib search generator ii) Which requires an ORC execution engine iii) Which you are not using in the wasm path (we're using WasmIncrementalExecutor instead) So as per what I see DynamicLibrarySearchGenerator infrastructure creates symbol generators for ORC JIT Dylibs, which isn’t used in the wasm execution model. So while the underlying dlopen(...) mechanism is similar, DynamicLibrarySearchGenerator wouldn’t integrate meaningfully unless the wasm executor also moved to using ORC JIT infrastructure, which it currently doesn’t. https://github.com/llvm/llvm-project/pull/133037 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Implement LoadDynamicLibrary for clang-repl wasm use cases (PR #133037)
https://github.com/anutosh491 edited https://github.com/llvm/llvm-project/pull/133037 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Implement LoadDynamicLibrary for clang-repl wasm use cases (PR #133037)
@@ -711,6 +712,14 @@ llvm::Error Interpreter::Undo(unsigned N) { } llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { +#ifdef __EMSCRIPTEN__ + void *handle = dlopen(name, RTLD_NOW | RTLD_GLOBAL); + if (!handle) { +llvm::errs() << dlerror() << '\n'; +return llvm::make_error("Failed to load dynamic library", + llvm::inconvertibleErrorCode()); + } +#else anutosh491 wrote: Check how we can use the changes here in cppinterop to get dynamic library loaded and symbol addresses fetched at runtime ! https://github.com/compiler-research/CppInterOp/compare/main...anutosh491:CppInterOp:shared_libs Everything boils down to preload as we know where in the MEMFS our shared lib exists ! https://github.com/llvm/llvm-project/pull/133037 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Implement LoadDynamicLibrary for clang-repl wasm use cases (PR #133037)
https://github.com/anutosh491 edited https://github.com/llvm/llvm-project/pull/133037 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Handle frontend options for clang-repl before calling executeAction (PR #132670)
anutosh491 wrote: Hey @serge-sans-paille Sorry for missing your last reply completely. Thanks for going through it in the first place :) I've added a small test to get us started https://github.com/llvm/llvm-project/pull/132670/files#diff-339061dce5c23e378165cd07b0511a10e5f921803cb8d7ac76a220cdb3eb54a8 We would get the following as a whole ``` (base) anutosh491@Anutoshs-MacBook-Air bin % ./clang-repl --Xcc -Xclang --Xcc -version LLVM (http://llvm.org/): LLVM version 21.0.0git Optimized build. clang-repl: Version displayed ``` But I am just checking the presence of the string "Version displayed" cause the version number and the output through help is subject to change in the future. https://github.com/llvm/llvm-project/pull/132670 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Implement LoadDynamicLibrary for clang-repl wasm use cases (PR #133037)
anutosh491 wrote: > We really need to figure out how to add in-tree tests. Yupp sadly I was only able to test this through cppinterop and xeus-cpp as I've commented above (https://github.com/llvm/llvm-project/pull/133037#discussion_r2015819533) where it works as expected :\ I shall take out some time in the near future to ideate on this. https://github.com/llvm/llvm-project/pull/133037 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Implement LoadDynamicLibrary for clang-repl wasm use cases (PR #133037)
anutosh491 wrote: > I am wondering if we can add such CI for llvm, too... maybe you can discuss > that at the llvm discord's infrastructure channel... Absolutely, I shall do that and get back to you on this (I think there are people at llvm looking into llvm+emscripten use cases and should support something like this) Shall keep you updated. In the meantime would be great to have your views on the above ... and possibly if you could give me the green flag here :) I think the [reasoning](https://github.com/llvm/llvm-project/pull/133037#discussion_r2015816316) sits well in a wasm context and would expand the interpreter's functionality to put any 3rd party/custom shared lib to use (issue description shows the interpreter loading **symengine** at runtime) . We also have a decent framework now through downstream projects to test something like the above. https://github.com/llvm/llvm-project/pull/133037 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Implement LoadDynamicLibrary for clang-repl wasm use cases (PR #133037)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/133037 >From a7ccd0e0b261c303639d3880fc51b0ed99e17363 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Wed, 26 Mar 2025 10:33:37 +0530 Subject: [PATCH 1/4] Implement LoadDynamicLibrary for clang-repl wasm use cases --- clang/lib/Interpreter/Interpreter.cpp | 10 ++ 1 file changed, 10 insertions(+) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index fa4c1439c9261..f8c8d0a425659 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -18,6 +18,7 @@ #include "llvm/Support/VirtualFileSystem.h" #ifdef __EMSCRIPTEN__ #include "Wasm.h" +#include #endif // __EMSCRIPTEN__ #include "clang/AST/ASTConsumer.h" @@ -711,6 +712,14 @@ llvm::Error Interpreter::Undo(unsigned N) { } llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { +#ifdef __EMSCRIPTEN__ + void *handle = dlopen(name, RTLD_NOW | RTLD_GLOBAL); + if (!handle) { +llvm::errs() << dlerror() << '\n'; +return llvm::make_error("Failed to load dynamic library", + llvm::inconvertibleErrorCode()); + } +#else auto EE = getExecutionEngine(); if (!EE) return EE.takeError(); @@ -722,6 +731,7 @@ llvm::Error Interpreter::LoadDynamicLibrary(const char *name) { EE->getMainJITDylib().addGenerator(std::move(*DLSG)); else return DLSG.takeError(); +#endif return llvm::Error::success(); } >From bc41b71965e1e33a32d1a54fd79148d1b6890dce Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Thu, 27 Mar 2025 12:34:23 +0530 Subject: [PATCH 2/4] Override getSymbolAddress --- clang/lib/Interpreter/IncrementalExecutor.h | 2 +- clang/lib/Interpreter/Wasm.cpp | 13 + clang/lib/Interpreter/Wasm.h| 2 ++ 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/clang/lib/Interpreter/IncrementalExecutor.h b/clang/lib/Interpreter/IncrementalExecutor.h index dbd61f0b8b1eb..71d71bc3883e2 100644 --- a/clang/lib/Interpreter/IncrementalExecutor.h +++ b/clang/lib/Interpreter/IncrementalExecutor.h @@ -57,7 +57,7 @@ class IncrementalExecutor { virtual llvm::Error removeModule(PartialTranslationUnit &PTU); virtual llvm::Error runCtors() const; virtual llvm::Error cleanUp(); - llvm::Expected + virtual llvm::Expected getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const; llvm::orc::LLJIT &GetExecutionEngine() { return *Jit; } diff --git a/clang/lib/Interpreter/Wasm.cpp b/clang/lib/Interpreter/Wasm.cpp index f7cb7598c77f8..0543a3504c9a2 100644 --- a/clang/lib/Interpreter/Wasm.cpp +++ b/clang/lib/Interpreter/Wasm.cpp @@ -144,6 +144,19 @@ llvm::Error WasmIncrementalExecutor::cleanUp() { return llvm::Error::success(); } +llvm::Expected +WasmIncrementalExecutor::getSymbolAddress(llvm::StringRef Name, + SymbolNameKind NameKind) const { + void *Sym = dlsym(RTLD_DEFAULT, Name.str().c_str()); + if (!Sym) { +return llvm::make_error("dlsym failed for symbol: " + + Name.str(), + llvm::inconvertibleErrorCode()); + } + + return llvm::orc::ExecutorAddr::fromPtr(Sym); +} + WasmIncrementalExecutor::~WasmIncrementalExecutor() = default; } // namespace clang diff --git a/clang/lib/Interpreter/Wasm.h b/clang/lib/Interpreter/Wasm.h index 4632613326d39..beafb01be7d8a 100644 --- a/clang/lib/Interpreter/Wasm.h +++ b/clang/lib/Interpreter/Wasm.h @@ -29,6 +29,8 @@ class WasmIncrementalExecutor : public IncrementalExecutor { llvm::Error removeModule(PartialTranslationUnit &PTU) override; llvm::Error runCtors() const override; llvm::Error cleanUp() override; + llvm::Expected + getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const; ~WasmIncrementalExecutor() override; }; >From 60d4622e23902bf7c5fd1040c9f82f9b03e134ce Mon Sep 17 00:00:00 2001 From: Anutosh Bhat Date: Thu, 27 Mar 2025 14:05:35 +0530 Subject: [PATCH 3/4] Update Wasm.h --- clang/lib/Interpreter/Wasm.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/lib/Interpreter/Wasm.h b/clang/lib/Interpreter/Wasm.h index beafb01be7d8a..d27455ec42c02 100644 --- a/clang/lib/Interpreter/Wasm.h +++ b/clang/lib/Interpreter/Wasm.h @@ -30,7 +30,7 @@ class WasmIncrementalExecutor : public IncrementalExecutor { llvm::Error runCtors() const override; llvm::Error cleanUp() override; llvm::Expected - getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const; + getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const override;; ~WasmIncrementalExecutor() override; }; >From 50370179c93d5f524fd38e093248d6e74db8155c Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Tue, 1 Apr 2025 10:40:57 +0530 Subject: [PATCH 4/4] fix clang format --- clang/lib/Interpreter/Wasm.h | 3 ++
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
@@ -451,13 +451,44 @@ const char *const Runtimes = R"( )"; llvm::Expected> -Interpreter::create(std::unique_ptr CI) { +Interpreter::create(std::unique_ptr CI, +std::unique_ptr DeviceCI) { llvm::Error Err = llvm::Error::success(); auto Interp = std::unique_ptr(new Interpreter(std::move(CI), Err)); if (Err) return std::move(Err); + if (DeviceCI) { +// auto DeviceLLVMCtx = std::make_unique(); +// auto DeviceTSCtx = +// std::make_unique(std::move(DeviceLLVMCtx)); + +// llvm::Error DeviceErr = llvm::Error::success(); +// llvm::ErrorAsOutParameter EAO(&DeviceErr); + +// auto DeviceAct = std::make_unique( +// *DeviceCI, *DeviceTSCtx->getContext(), DeviceErr, *Interp); + +// if (DeviceErr) +// return std::move(DeviceErr); + +// DeviceCI->ExecuteAction(*DeviceAct); +DeviceCI->ExecuteAction(*Interp->Act); anutosh491 wrote: Yes, what we have currently should do the job ! Thanks for the reply. https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
@@ -451,13 +451,44 @@ const char *const Runtimes = R"( )"; llvm::Expected> -Interpreter::create(std::unique_ptr CI) { +Interpreter::create(std::unique_ptr CI, +std::unique_ptr DeviceCI) { llvm::Error Err = llvm::Error::success(); auto Interp = std::unique_ptr(new Interpreter(std::move(CI), Err)); if (Err) return std::move(Err); + if (DeviceCI) { +// auto DeviceLLVMCtx = std::make_unique(); +// auto DeviceTSCtx = +// std::make_unique(std::move(DeviceLLVMCtx)); + +// llvm::Error DeviceErr = llvm::Error::success(); +// llvm::ErrorAsOutParameter EAO(&DeviceErr); + +// auto DeviceAct = std::make_unique( +// *DeviceCI, *DeviceTSCtx->getContext(), DeviceErr, *Interp); + +// if (DeviceErr) +// return std::move(DeviceErr); + +// DeviceCI->ExecuteAction(*DeviceAct); +DeviceCI->ExecuteAction(*Interp->Act); anutosh491 wrote: Well sadly I can't test the whole thing end to end being on macos, but at least this block should end up being platform independent is what I think (Using DeviceACT or ACT should work similarly) https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
@@ -451,13 +451,44 @@ const char *const Runtimes = R"( )"; llvm::Expected> -Interpreter::create(std::unique_ptr CI) { +Interpreter::create(std::unique_ptr CI, +std::unique_ptr DeviceCI) { llvm::Error Err = llvm::Error::success(); auto Interp = std::unique_ptr(new Interpreter(std::move(CI), Err)); if (Err) return std::move(Err); + if (DeviceCI) { +// auto DeviceLLVMCtx = std::make_unique(); +// auto DeviceTSCtx = +// std::make_unique(std::move(DeviceLLVMCtx)); + +// llvm::Error DeviceErr = llvm::Error::success(); +// llvm::ErrorAsOutParameter EAO(&DeviceErr); + +// auto DeviceAct = std::make_unique( +// *DeviceCI, *DeviceTSCtx->getContext(), DeviceErr, *Interp); + +// if (DeviceErr) +// return std::move(DeviceErr); + +// DeviceCI->ExecuteAction(*DeviceAct); +DeviceCI->ExecuteAction(*Interp->Act); anutosh491 wrote: Afk right now, shall address the above soon ! https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
https://github.com/anutosh491 edited https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clan-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/136404 >From 82a2be2dd0780bd3d57f4a22d61a4b0752696934 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Fri, 18 Apr 2025 18:45:00 +0530 Subject: [PATCH 1/5] Fix cuda flag with clang-repl --- clang/include/clang/Interpreter/Interpreter.h | 3 +- clang/lib/Interpreter/Interpreter.cpp | 43 +++ 2 files changed, 27 insertions(+), 19 deletions(-) diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index b1b63aedf86ab..20d22c4437a97 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -129,7 +129,8 @@ class Interpreter { public: virtual ~Interpreter(); static llvm::Expected> - create(std::unique_ptr CI); + create(std::unique_ptr CI, +std::unique_ptr DeviceCI = nullptr); static llvm::Expected> createWithCUDA(std::unique_ptr CI, std::unique_ptr DCI); diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index f8c8d0a425659..2002854daa2a0 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -451,20 +451,40 @@ const char *const Runtimes = R"( )"; llvm::Expected> -Interpreter::create(std::unique_ptr CI) { +Interpreter::create(std::unique_ptr CI, +std::unique_ptr DeviceCI) { llvm::Error Err = llvm::Error::success(); auto Interp = std::unique_ptr(new Interpreter(std::move(CI), Err)); if (Err) return std::move(Err); - // Add runtime code and set a marker to hide it from user code. Undo will not - // go through that. + // CUDA-specific logic (only if CUDA mode) + if (DeviceCI) { +// Ensure we initialize Sema on the device CI +// DeviceCI->createSema(Interp->Act->getTranslationUnitKind(), nullptr); +DeviceCI->ExecuteAction(*Interp->Act); + +// Use a custom in-memory VFS for fatbin writing +llvm::IntrusiveRefCntPtr IMVFS = +std::make_unique(); + +// Create the CUDA device parser +auto DeviceParser = std::make_unique( +std::move(DeviceCI), *Interp->getCompilerInstance(), IMVFS, Err, +Interp->PTUs); + +if (Err) + return std::move(Err); + +Interp->DeviceParser = std::move(DeviceParser); + } + + // Add runtime code and set a marker to hide it from user code. auto PTU = Interp->Parse(Runtimes); if (!PTU) return PTU.takeError(); Interp->markUserCodeStart(); - Interp->ValuePrintingInfo.resize(4); return std::move(Interp); } @@ -481,20 +501,7 @@ Interpreter::createWithCUDA(std::unique_ptr CI, OverlayVFS->pushOverlay(IMVFS); CI->createFileManager(OverlayVFS); - auto Interp = Interpreter::create(std::move(CI)); - if (auto E = Interp.takeError()) -return std::move(E); - - llvm::Error Err = llvm::Error::success(); - auto DeviceParser = std::make_unique( - std::move(DCI), *(*Interp)->getCompilerInstance(), IMVFS, Err, - (*Interp)->PTUs); - if (Err) -return std::move(Err); - - (*Interp)->DeviceParser = std::move(DeviceParser); - - return Interp; + return Interpreter::create(std::move(CI), std::move(DCI)); } const CompilerInstance *Interpreter::getCompilerInstance() const { >From ae0605f4c2b4c991074d0f91f046b9460ff4eac1 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Fri, 18 Apr 2025 19:33:00 +0530 Subject: [PATCH 2/5] execute action on deviceCI --- clang/lib/Interpreter/Interpreter.cpp | 23 ++- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 2002854daa2a0..b2c33480107d8 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -459,17 +459,30 @@ Interpreter::create(std::unique_ptr CI, if (Err) return std::move(Err); - // CUDA-specific logic (only if CUDA mode) if (DeviceCI) { -// Ensure we initialize Sema on the device CI -// DeviceCI->createSema(Interp->Act->getTranslationUnitKind(), nullptr); +// Create a fresh LLVM context for the CUDA device interpreter +// auto DeviceLLVMCtx = std::make_unique(); +// auto DeviceTSCtx = +// std::make_unique(std::move(DeviceLLVMCtx)); + +// llvm::Error DeviceErr = llvm::Error::success(); +// llvm::ErrorAsOutParameter EAO(&DeviceErr); + +// // Create an IncrementalAction for the device CompilerInstance +// auto DeviceAct = std::make_unique( +// *DeviceCI, *DeviceTSCtx->getContext(), DeviceErr, *Interp); + +// if (DeviceErr) +// return std::move(DeviceErr); + +// Execute the device-side action (this will create Sema etc.) DeviceCI->ExecuteAction(*Interp->Act); -// Use a custom in-memory VFS for fatbin writing +// Set up a virtual file system to emit fatbin to memory llvm::IntrusiveRefCntPtr IMVFS = std::make_unique();
[clang] [clan-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
anutosh491 wrote: cc @AaronBallman @vgvassilev This fixes this discussion y'all had https://github.com/llvm/llvm-project/pull/107737#discussion_r1806913203 Unfortunately I am on an Apple m1 so I can just justify seeing this on my side ``` (base) anutosh491@Anutoshs-MacBook-Air bin % ./clang-repl --cuda clang-repl: Invalid CUDA architecture ``` https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clan-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
@@ -451,13 +451,44 @@ const char *const Runtimes = R"( )"; llvm::Expected> -Interpreter::create(std::unique_ptr CI) { +Interpreter::create(std::unique_ptr CI, +std::unique_ptr DeviceCI) { llvm::Error Err = llvm::Error::success(); auto Interp = std::unique_ptr(new Interpreter(std::move(CI), Err)); if (Err) return std::move(Err); + if (DeviceCI) { +// auto DeviceLLVMCtx = std::make_unique(); +// auto DeviceTSCtx = +// std::make_unique(std::move(DeviceLLVMCtx)); + +// llvm::Error DeviceErr = llvm::Error::success(); +// llvm::ErrorAsOutParameter EAO(&DeviceErr); + +// auto DeviceAct = std::make_unique( +// *DeviceCI, *DeviceTSCtx->getContext(), DeviceErr, *Interp); + +// if (DeviceErr) +// return std::move(DeviceErr); + +// DeviceCI->ExecuteAction(*DeviceAct); +DeviceCI->ExecuteAction(*Interp->Act); anutosh491 wrote: Here, I am not 100% sure how tightly coupled is an `Action` with a `CompilerInstance`. So I thought maybe we would need a new "DeviceAct" here rather than using the Host CI's Action. So I wrote that and commented it out ! If that's not required, I'll get rid of whatever we don't want. https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clan-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/136404 >From 1b18e96882590825075b8f8e5094fdcb5225d349 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Fri, 18 Apr 2025 18:45:00 +0530 Subject: [PATCH] Fix cuda flag with clang-repl --- clang/include/clang/Interpreter/Interpreter.h | 3 +- clang/lib/Interpreter/DeviceOffload.cpp | 3 +- clang/lib/Interpreter/Interpreter.cpp | 48 +-- 3 files changed, 37 insertions(+), 17 deletions(-) diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index b1b63aedf86ab..7425797c55297 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -129,7 +129,8 @@ class Interpreter { public: virtual ~Interpreter(); static llvm::Expected> - create(std::unique_ptr CI); + create(std::unique_ptr CI, + std::unique_ptr DeviceCI = nullptr); static llvm::Expected> createWithCUDA(std::unique_ptr CI, std::unique_ptr DCI); diff --git a/clang/lib/Interpreter/DeviceOffload.cpp b/clang/lib/Interpreter/DeviceOffload.cpp index 1999d63d1aa04..9a7be006250a0 100644 --- a/clang/lib/Interpreter/DeviceOffload.cpp +++ b/clang/lib/Interpreter/DeviceOffload.cpp @@ -34,14 +34,15 @@ IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( TargetOpts(HostInstance.getTargetOpts()) { if (Err) return; - DeviceCI = std::move(DeviceInstance); StringRef Arch = TargetOpts.CPU; if (!Arch.starts_with("sm_") || Arch.substr(3).getAsInteger(10, SMVersion)) { +DeviceInstance.release(); Err = llvm::joinErrors(std::move(Err), llvm::make_error( "Invalid CUDA architecture", llvm::inconvertibleErrorCode())); return; } + DeviceCI = std::move(DeviceInstance); } llvm::Expected diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index f8c8d0a425659..049cc00cd198f 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -451,13 +451,44 @@ const char *const Runtimes = R"( )"; llvm::Expected> -Interpreter::create(std::unique_ptr CI) { +Interpreter::create(std::unique_ptr CI, +std::unique_ptr DeviceCI) { llvm::Error Err = llvm::Error::success(); auto Interp = std::unique_ptr(new Interpreter(std::move(CI), Err)); if (Err) return std::move(Err); + if (DeviceCI) { +// auto DeviceLLVMCtx = std::make_unique(); +// auto DeviceTSCtx = +// std::make_unique(std::move(DeviceLLVMCtx)); + +// llvm::Error DeviceErr = llvm::Error::success(); +// llvm::ErrorAsOutParameter EAO(&DeviceErr); + +// auto DeviceAct = std::make_unique( +// *DeviceCI, *DeviceTSCtx->getContext(), DeviceErr, *Interp); + +// if (DeviceErr) +// return std::move(DeviceErr); + +// DeviceCI->ExecuteAction(*DeviceAct); +DeviceCI->ExecuteAction(*Interp->Act); + +llvm::IntrusiveRefCntPtr IMVFS = +std::make_unique(); + +auto DeviceParser = std::make_unique( +std::move(DeviceCI), *Interp->getCompilerInstance(), IMVFS, Err, +Interp->PTUs); + +if (Err) + return std::move(Err); + +Interp->DeviceParser = std::move(DeviceParser); + } + // Add runtime code and set a marker to hide it from user code. Undo will not // go through that. auto PTU = Interp->Parse(Runtimes); @@ -481,20 +512,7 @@ Interpreter::createWithCUDA(std::unique_ptr CI, OverlayVFS->pushOverlay(IMVFS); CI->createFileManager(OverlayVFS); - auto Interp = Interpreter::create(std::move(CI)); - if (auto E = Interp.takeError()) -return std::move(E); - - llvm::Error Err = llvm::Error::success(); - auto DeviceParser = std::make_unique( - std::move(DCI), *(*Interp)->getCompilerInstance(), IMVFS, Err, - (*Interp)->PTUs); - if (Err) -return std::move(Err); - - (*Interp)->DeviceParser = std::move(DeviceParser); - - return Interp; + return Interpreter::create(std::move(CI), std::move(DCI)); } const CompilerInstance *Interpreter::getCompilerInstance() const { ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
https://github.com/anutosh491 edited https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clan-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
https://github.com/anutosh491 created https://github.com/llvm/llvm-project/pull/136404 `clang-repl --cuda` was previously crashing with a segmentation fault, instead of reporting a clean error ``` (base) anutosh491@Anutoshs-MacBook-Air bin % ./clang-repl --cuda #0 0x000111da4fbc llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/opt/local/libexec/llvm-20/lib/libLLVM.dylib+0x150fbc) #1 0x000111da31dc llvm::sys::RunSignalHandlers() (/opt/local/libexec/llvm-20/lib/libLLVM.dylib+0x14f1dc) #2 0x000111da5628 SignalHandler(int) (/opt/local/libexec/llvm-20/lib/libLLVM.dylib+0x151628) #3 0x00019b242de4 (/usr/lib/system/libsystem_platform.dylib+0x180482de4) #4 0x000107f638d0 clang::IncrementalCUDADeviceParser::IncrementalCUDADeviceParser(std::__1::unique_ptr>, clang::CompilerInstance&, llvm::IntrusiveRefCntPtr, llvm::Error&, std::__1::list> const&) (/opt/local/libexec/llvm-20/lib/libclang-cpp.dylib+0x216b8d0) #5 0x000107f638d0 clang::IncrementalCUDADeviceParser::IncrementalCUDADeviceParser(std::__1::unique_ptr>, clang::CompilerInstance&, llvm::IntrusiveRefCntPtr, llvm::Error&, std::__1::list> const&) (/opt/local/libexec/llvm-20/lib/libclang-cpp.dylib+0x216b8d0) #6 0x000107f6bac8 clang::Interpreter::createWithCUDA(std::__1::unique_ptr>, std::__1::unique_ptr>) (/opt/local/libexec/llvm-20/lib/libclang-cpp.dylib+0x2173ac8) #7 0x00010206f8a8 main (/opt/local/libexec/llvm-20/bin/clang-repl+0x138a8) #8 0x00019ae8c274 Segmentation fault: 11 ``` The underlying issue was that the `DeviceCompilerInstance` (used for device-side CUDA compilation) was never initialized with a `Sema`, which is required before constructing the `IncrementalCUDADeviceParser`. https://github.com/llvm/llvm-project/blob/89687e6f383b742a3c6542dc673a84d9f82d02de/clang/lib/Interpreter/DeviceOffload.cpp#L32 https://github.com/llvm/llvm-project/blob/89687e6f383b742a3c6542dc673a84d9f82d02de/clang/lib/Interpreter/IncrementalParser.cpp#L31 Unlike the host-side `CompilerInstance` which runs `ExecuteAction` inside the Interpreter constructor (thereby setting up Sema), the device-side CI was passed into the parser uninitialized, leading to an assertion or crash when accessing its internals. To fix this, I refactored the `Interpreter::create` method to include an optional `DeviceCI` parameter. If provided, we know we need to take care of this instance too. Only then do we construct the `IncrementalCUDADeviceParser`. >From 82a2be2dd0780bd3d57f4a22d61a4b0752696934 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Fri, 18 Apr 2025 18:45:00 +0530 Subject: [PATCH 1/4] Fix cuda flag with clang-repl --- clang/include/clang/Interpreter/Interpreter.h | 3 +- clang/lib/Interpreter/Interpreter.cpp | 43 +++ 2 files changed, 27 insertions(+), 19 deletions(-) diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index b1b63aedf86ab..20d22c4437a97 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -129,7 +129,8 @@ class Interpreter { public: virtual ~Interpreter(); static llvm::Expected> - create(std::unique_ptr CI); + create(std::unique_ptr CI, +std::unique_ptr DeviceCI = nullptr); static llvm::Expected> createWithCUDA(std::unique_ptr CI, std::unique_ptr DCI); diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index f8c8d0a425659..2002854daa2a0 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -451,20 +451,40 @@ const char *const Runtimes = R"( )"; llvm::Expected> -Interpreter::create(std::unique_ptr CI) { +Interpreter::create(std::unique_ptr CI, +std::unique_ptr DeviceCI) { llvm::Error Err = llvm::Error::success(); auto Interp = std::unique_ptr(new Interpreter(std::move(CI), Err)); if (Err) return std::move(Err); - // Add runtime code and set a marker to hide it from user code. Undo will not - // go through that. + // CUDA-specific logic (only if CUDA mode) + if (DeviceCI) { +// Ensure we initialize Sema on the device CI +// DeviceCI->createSema(Interp->Act->getTranslationUnitKind(), nullptr); +DeviceCI->ExecuteAction(*Interp->Act); + +// Use a custom in-memory VFS for fatbin writing +llvm::IntrusiveRefCntPtr IMVFS = +std::make_unique(); + +// Create the CUDA device parser +auto DeviceParser = std::make_unique( +std::move(DeviceCI), *Interp->getCompilerInstance(), IMVFS, Err, +Interp->PTUs); + +if (Err) + return std::move(Err); + +Interp->DeviceParser = std::move(DeviceParser); + } + + // Add runtime code and set a marker to hide it from user code. auto PTU = Interp->Parse(Runtimes); if (!PTU) return PTU.takeError(); Interp->markUserCodeStart(); - Interp->ValuePrintingInfo.resize(4); return std::
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/136404 >From 87ca1f6992d1413d1d2b2e0d230b4f41e1979fef Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Fri, 18 Apr 2025 18:45:00 +0530 Subject: [PATCH 1/2] Fix cuda flag with clang-repl --- clang/include/clang/Interpreter/Interpreter.h | 10 ++- clang/lib/Interpreter/DeviceOffload.cpp | 39 +-- clang/lib/Interpreter/DeviceOffload.h | 4 +- clang/lib/Interpreter/Interpreter.cpp | 65 ++- 4 files changed, 75 insertions(+), 43 deletions(-) diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index b1b63aedf86ab..59089cf639f96 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -95,6 +95,9 @@ class Interpreter { // An optional parser for CUDA offloading std::unique_ptr DeviceParser; + // An optional action for CUDA offloading + std::unique_ptr DeviceAct; + /// List containing information about each incrementally parsed piece of code. std::list PTUs; @@ -175,10 +178,11 @@ class Interpreter { llvm::Expected ExtractValueFromExpr(Expr *E); llvm::Expected CompileDtorCall(CXXRecordDecl *CXXRD); - CodeGenerator *getCodeGen() const; - std::unique_ptr GenModule(); + CodeGenerator *getCodeGen(IncrementalAction *Action = nullptr) const; + std::unique_ptr GenModule(IncrementalAction *Action = nullptr); PartialTranslationUnit &RegisterPTU(TranslationUnitDecl *TU, - std::unique_ptr M = {}); + std::unique_ptr M = {}, + IncrementalAction *Action = nullptr); // A cache for the compiled destructors used to for de-allocation of managed // clang::Values. diff --git a/clang/lib/Interpreter/DeviceOffload.cpp b/clang/lib/Interpreter/DeviceOffload.cpp index 1999d63d1aa04..d9b00787f038d 100644 --- a/clang/lib/Interpreter/DeviceOffload.cpp +++ b/clang/lib/Interpreter/DeviceOffload.cpp @@ -28,20 +28,21 @@ IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr FS, -llvm::Error &Err, const std::list &PTUs) +llvm::Error &Err, std::list &PTUs) : IncrementalParser(*DeviceInstance, Err), PTUs(PTUs), VFS(FS), CodeGenOpts(HostInstance.getCodeGenOpts()), - TargetOpts(HostInstance.getTargetOpts()) { + TargetOpts(DeviceInstance->getTargetOpts()) { if (Err) return; - DeviceCI = std::move(DeviceInstance); StringRef Arch = TargetOpts.CPU; if (!Arch.starts_with("sm_") || Arch.substr(3).getAsInteger(10, SMVersion)) { +DeviceInstance.release(); Err = llvm::joinErrors(std::move(Err), llvm::make_error( "Invalid CUDA architecture", llvm::inconvertibleErrorCode())); return; } + DeviceCI = std::move(DeviceInstance); } llvm::Expected @@ -50,25 +51,6 @@ IncrementalCUDADeviceParser::Parse(llvm::StringRef Input) { if (!PTU) return PTU.takeError(); - auto PTX = GeneratePTX(); - if (!PTX) -return PTX.takeError(); - - auto Err = GenerateFatbinary(); - if (Err) -return std::move(Err); - - std::string FatbinFileName = - "/incr_module_" + std::to_string(PTUs.size()) + ".fatbin"; - VFS->addFile(FatbinFileName, 0, - llvm::MemoryBuffer::getMemBuffer( - llvm::StringRef(FatbinContent.data(), FatbinContent.size()), - "", false)); - - CodeGenOpts.CudaGpuBinaryFileName = FatbinFileName; - - FatbinContent.clear(); - return PTU; } @@ -172,6 +154,19 @@ llvm::Error IncrementalCUDADeviceParser::GenerateFatbinary() { FatbinContent.append(PTXCode.begin(), PTXCode.end()); + auto &PTU = PTUs.back(); + + std::string FatbinFileName = "/" + PTU.TheModule->getName().str() + ".fatbin"; + + VFS->addFile(FatbinFileName, 0, + llvm::MemoryBuffer::getMemBuffer( + llvm::StringRef(FatbinContent.data(), FatbinContent.size()), + "", false)); + + CodeGenOpts.CudaGpuBinaryFileName = FatbinFileName; + + FatbinContent.clear(); + return llvm::Error::success(); } diff --git a/clang/lib/Interpreter/DeviceOffload.h b/clang/lib/Interpreter/DeviceOffload.h index b9a1acab004c3..23d89046c09e1 100644 --- a/clang/lib/Interpreter/DeviceOffload.h +++ b/clang/lib/Interpreter/DeviceOffload.h @@ -24,14 +24,14 @@ class CodeGenOptions; class TargetOptions; class IncrementalCUDADeviceParser : public IncrementalParser { - const std::list &PTUs; + std::list &PTUs; public: IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr VFS, - llvm::Error &Err, const std::list &PTUs); + llvm::Error &Err, std::l
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/136404 >From 6c64e64c4a3b56f50808cae244b29da1231525f1 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Fri, 18 Apr 2025 18:45:00 +0530 Subject: [PATCH] Fix cuda flag with clang-repl --- clang/include/clang/Interpreter/Interpreter.h | 13 ++-- clang/lib/Interpreter/DeviceOffload.cpp | 39 ++-- clang/lib/Interpreter/DeviceOffload.h | 4 +- clang/lib/Interpreter/Interpreter.cpp | 59 ++- 4 files changed, 72 insertions(+), 43 deletions(-) diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index b1b63aedf86ab..56213f88b9e30 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -41,6 +41,7 @@ class CXXRecordDecl; class Decl; class IncrementalExecutor; class IncrementalParser; +class IncrementalCUDADeviceParser; /// Create a pre-configured \c CompilerInstance for incremental processing. class IncrementalCompilerBuilder { @@ -93,7 +94,10 @@ class Interpreter { std::unique_ptr IncrExecutor; // An optional parser for CUDA offloading - std::unique_ptr DeviceParser; + std::unique_ptr DeviceParser; + + // An optional action for CUDA offloading + std::unique_ptr DeviceAct; /// List containing information about each incrementally parsed piece of code. std::list PTUs; @@ -175,10 +179,11 @@ class Interpreter { llvm::Expected ExtractValueFromExpr(Expr *E); llvm::Expected CompileDtorCall(CXXRecordDecl *CXXRD); - CodeGenerator *getCodeGen() const; - std::unique_ptr GenModule(); + CodeGenerator *getCodeGen(IncrementalAction *Action = nullptr) const; + std::unique_ptr GenModule(IncrementalAction *Action = nullptr); PartialTranslationUnit &RegisterPTU(TranslationUnitDecl *TU, - std::unique_ptr M = {}); + std::unique_ptr M = {}, + IncrementalAction *Action = nullptr); // A cache for the compiled destructors used to for de-allocation of managed // clang::Values. diff --git a/clang/lib/Interpreter/DeviceOffload.cpp b/clang/lib/Interpreter/DeviceOffload.cpp index 1999d63d1aa04..d9b00787f038d 100644 --- a/clang/lib/Interpreter/DeviceOffload.cpp +++ b/clang/lib/Interpreter/DeviceOffload.cpp @@ -28,20 +28,21 @@ IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr FS, -llvm::Error &Err, const std::list &PTUs) +llvm::Error &Err, std::list &PTUs) : IncrementalParser(*DeviceInstance, Err), PTUs(PTUs), VFS(FS), CodeGenOpts(HostInstance.getCodeGenOpts()), - TargetOpts(HostInstance.getTargetOpts()) { + TargetOpts(DeviceInstance->getTargetOpts()) { if (Err) return; - DeviceCI = std::move(DeviceInstance); StringRef Arch = TargetOpts.CPU; if (!Arch.starts_with("sm_") || Arch.substr(3).getAsInteger(10, SMVersion)) { +DeviceInstance.release(); Err = llvm::joinErrors(std::move(Err), llvm::make_error( "Invalid CUDA architecture", llvm::inconvertibleErrorCode())); return; } + DeviceCI = std::move(DeviceInstance); } llvm::Expected @@ -50,25 +51,6 @@ IncrementalCUDADeviceParser::Parse(llvm::StringRef Input) { if (!PTU) return PTU.takeError(); - auto PTX = GeneratePTX(); - if (!PTX) -return PTX.takeError(); - - auto Err = GenerateFatbinary(); - if (Err) -return std::move(Err); - - std::string FatbinFileName = - "/incr_module_" + std::to_string(PTUs.size()) + ".fatbin"; - VFS->addFile(FatbinFileName, 0, - llvm::MemoryBuffer::getMemBuffer( - llvm::StringRef(FatbinContent.data(), FatbinContent.size()), - "", false)); - - CodeGenOpts.CudaGpuBinaryFileName = FatbinFileName; - - FatbinContent.clear(); - return PTU; } @@ -172,6 +154,19 @@ llvm::Error IncrementalCUDADeviceParser::GenerateFatbinary() { FatbinContent.append(PTXCode.begin(), PTXCode.end()); + auto &PTU = PTUs.back(); + + std::string FatbinFileName = "/" + PTU.TheModule->getName().str() + ".fatbin"; + + VFS->addFile(FatbinFileName, 0, + llvm::MemoryBuffer::getMemBuffer( + llvm::StringRef(FatbinContent.data(), FatbinContent.size()), + "", false)); + + CodeGenOpts.CudaGpuBinaryFileName = FatbinFileName; + + FatbinContent.clear(); + return llvm::Error::success(); } diff --git a/clang/lib/Interpreter/DeviceOffload.h b/clang/lib/Interpreter/DeviceOffload.h index b9a1acab004c3..23d89046c09e1 100644 --- a/clang/lib/Interpreter/DeviceOffload.h +++ b/clang/lib/Interpreter/DeviceOffload.h @@ -24,14 +24,14 @@ class CodeGenOptions; class TargetOptions; class IncrementalCUDADevi
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
anutosh491 wrote: Making a note of the major changes below ! https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/136404 >From 1f7205615f8d11c1b58e2a2760f85663f97767c5 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Fri, 18 Apr 2025 18:45:00 +0530 Subject: [PATCH 1/4] Fix cuda flag with clang-repl --- clang/include/clang/Interpreter/Interpreter.h | 13 ++- clang/lib/Interpreter/DeviceOffload.cpp | 43 + clang/lib/Interpreter/DeviceOffload.h | 4 +- clang/lib/Interpreter/Interpreter.cpp | 89 --- 4 files changed, 88 insertions(+), 61 deletions(-) diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index b1b63aedf86ab..1b228e0917d02 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -95,6 +95,9 @@ class Interpreter { // An optional parser for CUDA offloading std::unique_ptr DeviceParser; + // An optional action for CUDA offloading + std::unique_ptr DeviceAct; + /// List containing information about each incrementally parsed piece of code. std::list PTUs; @@ -129,7 +132,8 @@ class Interpreter { public: virtual ~Interpreter(); static llvm::Expected> - create(std::unique_ptr CI); + create(std::unique_ptr CI, + std::unique_ptr DeviceCI = nullptr); static llvm::Expected> createWithCUDA(std::unique_ptr CI, std::unique_ptr DCI); @@ -175,10 +179,11 @@ class Interpreter { llvm::Expected ExtractValueFromExpr(Expr *E); llvm::Expected CompileDtorCall(CXXRecordDecl *CXXRD); - CodeGenerator *getCodeGen() const; - std::unique_ptr GenModule(); + CodeGenerator *getCodeGen(IncrementalAction *Action = nullptr) const; + std::unique_ptr GenModule(IncrementalAction *Action = nullptr); PartialTranslationUnit &RegisterPTU(TranslationUnitDecl *TU, - std::unique_ptr M = {}); + std::unique_ptr M = {}, + IncrementalAction *Action = nullptr); // A cache for the compiled destructors used to for de-allocation of managed // clang::Values. diff --git a/clang/lib/Interpreter/DeviceOffload.cpp b/clang/lib/Interpreter/DeviceOffload.cpp index 1999d63d1aa04..6977d7fa674ab 100644 --- a/clang/lib/Interpreter/DeviceOffload.cpp +++ b/clang/lib/Interpreter/DeviceOffload.cpp @@ -28,20 +28,21 @@ IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr FS, -llvm::Error &Err, const std::list &PTUs) +llvm::Error &Err, std::list &PTUs) : IncrementalParser(*DeviceInstance, Err), PTUs(PTUs), VFS(FS), CodeGenOpts(HostInstance.getCodeGenOpts()), - TargetOpts(HostInstance.getTargetOpts()) { + TargetOpts(DeviceInstance->getTargetOpts()) { if (Err) return; - DeviceCI = std::move(DeviceInstance); StringRef Arch = TargetOpts.CPU; if (!Arch.starts_with("sm_") || Arch.substr(3).getAsInteger(10, SMVersion)) { +DeviceInstance.release(); Err = llvm::joinErrors(std::move(Err), llvm::make_error( "Invalid CUDA architecture", llvm::inconvertibleErrorCode())); return; } + DeviceCI = std::move(DeviceInstance); } llvm::Expected @@ -50,25 +51,6 @@ IncrementalCUDADeviceParser::Parse(llvm::StringRef Input) { if (!PTU) return PTU.takeError(); - auto PTX = GeneratePTX(); - if (!PTX) -return PTX.takeError(); - - auto Err = GenerateFatbinary(); - if (Err) -return std::move(Err); - - std::string FatbinFileName = - "/incr_module_" + std::to_string(PTUs.size()) + ".fatbin"; - VFS->addFile(FatbinFileName, 0, - llvm::MemoryBuffer::getMemBuffer( - llvm::StringRef(FatbinContent.data(), FatbinContent.size()), - "", false)); - - CodeGenOpts.CudaGpuBinaryFileName = FatbinFileName; - - FatbinContent.clear(); - return PTU; } @@ -78,9 +60,11 @@ llvm::Expected IncrementalCUDADeviceParser::GeneratePTX() { const llvm::Target *Target = llvm::TargetRegistry::lookupTarget( PTU.TheModule->getTargetTriple(), Error); - if (!Target) + if (!Target) { return llvm::make_error(std::move(Error), std::error_code()); + } + llvm::TargetOptions TO = llvm::TargetOptions(); llvm::TargetMachine *TargetMachine = Target->createTargetMachine( PTU.TheModule->getTargetTriple(), TargetOpts.CPU, "", TO, @@ -172,6 +156,19 @@ llvm::Error IncrementalCUDADeviceParser::GenerateFatbinary() { FatbinContent.append(PTXCode.begin(), PTXCode.end()); + auto &PTU = PTUs.back(); + + std::string FatbinFileName = "/" + PTU.TheModule->getName().str() + ".fatbin"; + + VFS->addFile(FatbinFileName, 0, + llvm::MemoryBuffer::getMemBuffer( + llvm
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
https://github.com/anutosh491 edited https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
https://github.com/anutosh491 edited https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
@@ -760,8 +787,10 @@ std::unique_ptr Interpreter::GenModule() { return nullptr; } -CodeGenerator *Interpreter::getCodeGen() const { - FrontendAction *WrappedAct = Act->getWrapped(); +CodeGenerator *Interpreter::getCodeGen(IncrementalAction *Action) const { + if (!Action) +Action = Act.get(); + FrontendAction *WrappedAct = Action->getWrapped(); anutosh491 wrote: Yupp. Basically this comment https://github.com/llvm/llvm-project/pull/136404#discussion_r2058443908 https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/136404 >From 6c64e64c4a3b56f50808cae244b29da1231525f1 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Fri, 18 Apr 2025 18:45:00 +0530 Subject: [PATCH 1/3] Fix cuda flag with clang-repl --- clang/include/clang/Interpreter/Interpreter.h | 13 ++-- clang/lib/Interpreter/DeviceOffload.cpp | 39 ++-- clang/lib/Interpreter/DeviceOffload.h | 4 +- clang/lib/Interpreter/Interpreter.cpp | 59 ++- 4 files changed, 72 insertions(+), 43 deletions(-) diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index b1b63aedf86ab..56213f88b9e30 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -41,6 +41,7 @@ class CXXRecordDecl; class Decl; class IncrementalExecutor; class IncrementalParser; +class IncrementalCUDADeviceParser; /// Create a pre-configured \c CompilerInstance for incremental processing. class IncrementalCompilerBuilder { @@ -93,7 +94,10 @@ class Interpreter { std::unique_ptr IncrExecutor; // An optional parser for CUDA offloading - std::unique_ptr DeviceParser; + std::unique_ptr DeviceParser; + + // An optional action for CUDA offloading + std::unique_ptr DeviceAct; /// List containing information about each incrementally parsed piece of code. std::list PTUs; @@ -175,10 +179,11 @@ class Interpreter { llvm::Expected ExtractValueFromExpr(Expr *E); llvm::Expected CompileDtorCall(CXXRecordDecl *CXXRD); - CodeGenerator *getCodeGen() const; - std::unique_ptr GenModule(); + CodeGenerator *getCodeGen(IncrementalAction *Action = nullptr) const; + std::unique_ptr GenModule(IncrementalAction *Action = nullptr); PartialTranslationUnit &RegisterPTU(TranslationUnitDecl *TU, - std::unique_ptr M = {}); + std::unique_ptr M = {}, + IncrementalAction *Action = nullptr); // A cache for the compiled destructors used to for de-allocation of managed // clang::Values. diff --git a/clang/lib/Interpreter/DeviceOffload.cpp b/clang/lib/Interpreter/DeviceOffload.cpp index 1999d63d1aa04..d9b00787f038d 100644 --- a/clang/lib/Interpreter/DeviceOffload.cpp +++ b/clang/lib/Interpreter/DeviceOffload.cpp @@ -28,20 +28,21 @@ IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr FS, -llvm::Error &Err, const std::list &PTUs) +llvm::Error &Err, std::list &PTUs) : IncrementalParser(*DeviceInstance, Err), PTUs(PTUs), VFS(FS), CodeGenOpts(HostInstance.getCodeGenOpts()), - TargetOpts(HostInstance.getTargetOpts()) { + TargetOpts(DeviceInstance->getTargetOpts()) { if (Err) return; - DeviceCI = std::move(DeviceInstance); StringRef Arch = TargetOpts.CPU; if (!Arch.starts_with("sm_") || Arch.substr(3).getAsInteger(10, SMVersion)) { +DeviceInstance.release(); Err = llvm::joinErrors(std::move(Err), llvm::make_error( "Invalid CUDA architecture", llvm::inconvertibleErrorCode())); return; } + DeviceCI = std::move(DeviceInstance); } llvm::Expected @@ -50,25 +51,6 @@ IncrementalCUDADeviceParser::Parse(llvm::StringRef Input) { if (!PTU) return PTU.takeError(); - auto PTX = GeneratePTX(); - if (!PTX) -return PTX.takeError(); - - auto Err = GenerateFatbinary(); - if (Err) -return std::move(Err); - - std::string FatbinFileName = - "/incr_module_" + std::to_string(PTUs.size()) + ".fatbin"; - VFS->addFile(FatbinFileName, 0, - llvm::MemoryBuffer::getMemBuffer( - llvm::StringRef(FatbinContent.data(), FatbinContent.size()), - "", false)); - - CodeGenOpts.CudaGpuBinaryFileName = FatbinFileName; - - FatbinContent.clear(); - return PTU; } @@ -172,6 +154,19 @@ llvm::Error IncrementalCUDADeviceParser::GenerateFatbinary() { FatbinContent.append(PTXCode.begin(), PTXCode.end()); + auto &PTU = PTUs.back(); + + std::string FatbinFileName = "/" + PTU.TheModule->getName().str() + ".fatbin"; + + VFS->addFile(FatbinFileName, 0, + llvm::MemoryBuffer::getMemBuffer( + llvm::StringRef(FatbinContent.data(), FatbinContent.size()), + "", false)); + + CodeGenOpts.CudaGpuBinaryFileName = FatbinFileName; + + FatbinContent.clear(); + return llvm::Error::success(); } diff --git a/clang/lib/Interpreter/DeviceOffload.h b/clang/lib/Interpreter/DeviceOffload.h index b9a1acab004c3..23d89046c09e1 100644 --- a/clang/lib/Interpreter/DeviceOffload.h +++ b/clang/lib/Interpreter/DeviceOffload.h @@ -24,14 +24,14 @@ class CodeGenOptions; class TargetOptions; class IncrementalCUDA
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
https://github.com/anutosh491 edited https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
@@ -50,25 +50,6 @@ IncrementalCUDADeviceParser::Parse(llvm::StringRef Input) { if (!PTU) return PTU.takeError(); - auto PTX = GeneratePTX(); anutosh491 wrote: Yes absolutely ! Sorry for overlooking this. Made the change. https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/136404 >From 1f7205615f8d11c1b58e2a2760f85663f97767c5 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Fri, 18 Apr 2025 18:45:00 +0530 Subject: [PATCH 1/2] Fix cuda flag with clang-repl --- clang/include/clang/Interpreter/Interpreter.h | 13 ++- clang/lib/Interpreter/DeviceOffload.cpp | 43 + clang/lib/Interpreter/DeviceOffload.h | 4 +- clang/lib/Interpreter/Interpreter.cpp | 89 --- 4 files changed, 88 insertions(+), 61 deletions(-) diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index b1b63aedf86ab..1b228e0917d02 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -95,6 +95,9 @@ class Interpreter { // An optional parser for CUDA offloading std::unique_ptr DeviceParser; + // An optional action for CUDA offloading + std::unique_ptr DeviceAct; + /// List containing information about each incrementally parsed piece of code. std::list PTUs; @@ -129,7 +132,8 @@ class Interpreter { public: virtual ~Interpreter(); static llvm::Expected> - create(std::unique_ptr CI); + create(std::unique_ptr CI, + std::unique_ptr DeviceCI = nullptr); static llvm::Expected> createWithCUDA(std::unique_ptr CI, std::unique_ptr DCI); @@ -175,10 +179,11 @@ class Interpreter { llvm::Expected ExtractValueFromExpr(Expr *E); llvm::Expected CompileDtorCall(CXXRecordDecl *CXXRD); - CodeGenerator *getCodeGen() const; - std::unique_ptr GenModule(); + CodeGenerator *getCodeGen(IncrementalAction *Action = nullptr) const; + std::unique_ptr GenModule(IncrementalAction *Action = nullptr); PartialTranslationUnit &RegisterPTU(TranslationUnitDecl *TU, - std::unique_ptr M = {}); + std::unique_ptr M = {}, + IncrementalAction *Action = nullptr); // A cache for the compiled destructors used to for de-allocation of managed // clang::Values. diff --git a/clang/lib/Interpreter/DeviceOffload.cpp b/clang/lib/Interpreter/DeviceOffload.cpp index 1999d63d1aa04..6977d7fa674ab 100644 --- a/clang/lib/Interpreter/DeviceOffload.cpp +++ b/clang/lib/Interpreter/DeviceOffload.cpp @@ -28,20 +28,21 @@ IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr FS, -llvm::Error &Err, const std::list &PTUs) +llvm::Error &Err, std::list &PTUs) : IncrementalParser(*DeviceInstance, Err), PTUs(PTUs), VFS(FS), CodeGenOpts(HostInstance.getCodeGenOpts()), - TargetOpts(HostInstance.getTargetOpts()) { + TargetOpts(DeviceInstance->getTargetOpts()) { if (Err) return; - DeviceCI = std::move(DeviceInstance); StringRef Arch = TargetOpts.CPU; if (!Arch.starts_with("sm_") || Arch.substr(3).getAsInteger(10, SMVersion)) { +DeviceInstance.release(); Err = llvm::joinErrors(std::move(Err), llvm::make_error( "Invalid CUDA architecture", llvm::inconvertibleErrorCode())); return; } + DeviceCI = std::move(DeviceInstance); } llvm::Expected @@ -50,25 +51,6 @@ IncrementalCUDADeviceParser::Parse(llvm::StringRef Input) { if (!PTU) return PTU.takeError(); - auto PTX = GeneratePTX(); - if (!PTX) -return PTX.takeError(); - - auto Err = GenerateFatbinary(); - if (Err) -return std::move(Err); - - std::string FatbinFileName = - "/incr_module_" + std::to_string(PTUs.size()) + ".fatbin"; - VFS->addFile(FatbinFileName, 0, - llvm::MemoryBuffer::getMemBuffer( - llvm::StringRef(FatbinContent.data(), FatbinContent.size()), - "", false)); - - CodeGenOpts.CudaGpuBinaryFileName = FatbinFileName; - - FatbinContent.clear(); - return PTU; } @@ -78,9 +60,11 @@ llvm::Expected IncrementalCUDADeviceParser::GeneratePTX() { const llvm::Target *Target = llvm::TargetRegistry::lookupTarget( PTU.TheModule->getTargetTriple(), Error); - if (!Target) + if (!Target) { return llvm::make_error(std::move(Error), std::error_code()); + } + llvm::TargetOptions TO = llvm::TargetOptions(); llvm::TargetMachine *TargetMachine = Target->createTargetMachine( PTU.TheModule->getTargetTriple(), TargetOpts.CPU, "", TO, @@ -172,6 +156,19 @@ llvm::Error IncrementalCUDADeviceParser::GenerateFatbinary() { FatbinContent.append(PTXCode.begin(), PTXCode.end()); + auto &PTU = PTUs.back(); + + std::string FatbinFileName = "/" + PTU.TheModule->getName().str() + ".fatbin"; + + VFS->addFile(FatbinFileName, 0, + llvm::MemoryBuffer::getMemBuffer( + llvm
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/136404 >From 1f7205615f8d11c1b58e2a2760f85663f97767c5 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Fri, 18 Apr 2025 18:45:00 +0530 Subject: [PATCH 1/3] Fix cuda flag with clang-repl --- clang/include/clang/Interpreter/Interpreter.h | 13 ++- clang/lib/Interpreter/DeviceOffload.cpp | 43 + clang/lib/Interpreter/DeviceOffload.h | 4 +- clang/lib/Interpreter/Interpreter.cpp | 89 --- 4 files changed, 88 insertions(+), 61 deletions(-) diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index b1b63aedf86ab..1b228e0917d02 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -95,6 +95,9 @@ class Interpreter { // An optional parser for CUDA offloading std::unique_ptr DeviceParser; + // An optional action for CUDA offloading + std::unique_ptr DeviceAct; + /// List containing information about each incrementally parsed piece of code. std::list PTUs; @@ -129,7 +132,8 @@ class Interpreter { public: virtual ~Interpreter(); static llvm::Expected> - create(std::unique_ptr CI); + create(std::unique_ptr CI, + std::unique_ptr DeviceCI = nullptr); static llvm::Expected> createWithCUDA(std::unique_ptr CI, std::unique_ptr DCI); @@ -175,10 +179,11 @@ class Interpreter { llvm::Expected ExtractValueFromExpr(Expr *E); llvm::Expected CompileDtorCall(CXXRecordDecl *CXXRD); - CodeGenerator *getCodeGen() const; - std::unique_ptr GenModule(); + CodeGenerator *getCodeGen(IncrementalAction *Action = nullptr) const; + std::unique_ptr GenModule(IncrementalAction *Action = nullptr); PartialTranslationUnit &RegisterPTU(TranslationUnitDecl *TU, - std::unique_ptr M = {}); + std::unique_ptr M = {}, + IncrementalAction *Action = nullptr); // A cache for the compiled destructors used to for de-allocation of managed // clang::Values. diff --git a/clang/lib/Interpreter/DeviceOffload.cpp b/clang/lib/Interpreter/DeviceOffload.cpp index 1999d63d1aa04..6977d7fa674ab 100644 --- a/clang/lib/Interpreter/DeviceOffload.cpp +++ b/clang/lib/Interpreter/DeviceOffload.cpp @@ -28,20 +28,21 @@ IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr FS, -llvm::Error &Err, const std::list &PTUs) +llvm::Error &Err, std::list &PTUs) : IncrementalParser(*DeviceInstance, Err), PTUs(PTUs), VFS(FS), CodeGenOpts(HostInstance.getCodeGenOpts()), - TargetOpts(HostInstance.getTargetOpts()) { + TargetOpts(DeviceInstance->getTargetOpts()) { if (Err) return; - DeviceCI = std::move(DeviceInstance); StringRef Arch = TargetOpts.CPU; if (!Arch.starts_with("sm_") || Arch.substr(3).getAsInteger(10, SMVersion)) { +DeviceInstance.release(); Err = llvm::joinErrors(std::move(Err), llvm::make_error( "Invalid CUDA architecture", llvm::inconvertibleErrorCode())); return; } + DeviceCI = std::move(DeviceInstance); } llvm::Expected @@ -50,25 +51,6 @@ IncrementalCUDADeviceParser::Parse(llvm::StringRef Input) { if (!PTU) return PTU.takeError(); - auto PTX = GeneratePTX(); - if (!PTX) -return PTX.takeError(); - - auto Err = GenerateFatbinary(); - if (Err) -return std::move(Err); - - std::string FatbinFileName = - "/incr_module_" + std::to_string(PTUs.size()) + ".fatbin"; - VFS->addFile(FatbinFileName, 0, - llvm::MemoryBuffer::getMemBuffer( - llvm::StringRef(FatbinContent.data(), FatbinContent.size()), - "", false)); - - CodeGenOpts.CudaGpuBinaryFileName = FatbinFileName; - - FatbinContent.clear(); - return PTU; } @@ -78,9 +60,11 @@ llvm::Expected IncrementalCUDADeviceParser::GeneratePTX() { const llvm::Target *Target = llvm::TargetRegistry::lookupTarget( PTU.TheModule->getTargetTriple(), Error); - if (!Target) + if (!Target) { return llvm::make_error(std::move(Error), std::error_code()); + } + llvm::TargetOptions TO = llvm::TargetOptions(); llvm::TargetMachine *TargetMachine = Target->createTargetMachine( PTU.TheModule->getTargetTriple(), TargetOpts.CPU, "", TO, @@ -172,6 +156,19 @@ llvm::Error IncrementalCUDADeviceParser::GenerateFatbinary() { FatbinContent.append(PTXCode.begin(), PTXCode.end()); + auto &PTU = PTUs.back(); + + std::string FatbinFileName = "/" + PTU.TheModule->getName().str() + ".fatbin"; + + VFS->addFile(FatbinFileName, 0, + llvm::MemoryBuffer::getMemBuffer( + llvm
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
@@ -760,8 +787,10 @@ std::unique_ptr Interpreter::GenModule() { return nullptr; } -CodeGenerator *Interpreter::getCodeGen() const { - FrontendAction *WrappedAct = Act->getWrapped(); +CodeGenerator *Interpreter::getCodeGen(IncrementalAction *Action) const { + if (!Action) +Action = Act.get(); + FrontendAction *WrappedAct = Action->getWrapped(); anutosh491 wrote: Talking about changes with respect to codegen. So errors like these ``` ./bin/clang-repl --cuda module flag identifiers must be unique (or of 'require' type) !"wchar_size" module flag identifiers must be unique (or of 'require' type) !"nvvm-reflect-ftz" module flag identifiers must be unique (or of 'require' type) !"frame-pointer" fatal error: error in backend: Broken module found, compilation aborted! ``` or even something like this ``` error: Added modules have incompatible data layouts: e-p6:32:32-i64:64-i128:128-v16:16-v32:32-n16:32:64 (module) vs e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128 (jit) ``` Occur because we use the `Act` (the host IncrementalAction) to come up with the Module every time as of now. But think about it when we are registering a PTU through registerPTU ``` if (!M) M = GenModule(); ``` And we have this block. We shouldn't be generating the same Module for both the hostPTU (parsed through the incremental parser) and the devicePTU (parsed through the incrementalCudaParser) . For the first case the Module should revolve around the triple `x86_64-unknown-linux-gnu` and for the device case the module should be framed around the target triple `nvptx64-nvidia-cuda`. This is taken care of based on which `IncrementalAction` we use. We should use Act for hostPTU and DeviceACT for PTU. Hence the changes here (as one trickles into the other) ``` CodeGenerator *getCodeGen(IncrementalAction *Action = nullptr) const; std::unique_ptr GenModule(IncrementalAction *Action = nullptr); PartialTranslationUnit &RegisterPTU(TranslationUnitDecl *TU, std::unique_ptr M = {}, IncrementalAction *Action = nullptr); ``` So by default we use Act but we can take of it when required. 1) So while regiserting a devicePTU we use this https://github.com/llvm/llvm-project/pull/136404/files#diff-b8484f1fc5b057f146ed5d9b6e2cd47c3f6f5ae879c7a0eee44f0a272581a88cR580 2) While registering a hostptu we use this https://github.com/llvm/llvm-project/pull/136404/files#diff-b8484f1fc5b057f146ed5d9b6e2cd47c3f6f5ae879c7a0eee44f0a272581a88cR600 https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
@@ -560,6 +576,16 @@ Interpreter::Parse(llvm::StringRef Code) { llvm::Expected DeviceTU = DeviceParser->Parse(Code); if (auto E = DeviceTU.takeError()) return std::move(E); + +RegisterPTU(*DeviceTU, nullptr, DeviceAct.get()); + +llvm::Expected PTX = DeviceParser->GeneratePTX(); +if (!PTX) + return PTX.takeError(); + +llvm::Error Err = DeviceParser->GenerateFatbinary(); +if (Err) + return std::move(Err); } anutosh491 wrote: Changes with respect to parsing if `DeviceParser` is non nullptr If we currently look at how generatePTX is framed (check [lines](https://github.com/llvm/llvm-project/pull/136404/files#diff-facf198ec2120aeb2be299cac501daf8e32793547be89a06139bcb988ad532c7R58-R59)) We operate on PTUs from the list of PTUS. But currently we add no "devicePTUs" (produced by the deviceparser) as such in our list of PTUs. So `auto &PTU = PTUs.back();` would i) either return a nullptr ii) or point to a host PTU Both of which are wrong. The generatePTX function is meant to work on "devicePTUs" (produced by the deviceparser) as such who's modules have the target triple `nvptx64-nvidia-cuda` rather than that provided by the host (possibly `x86_64-unknown-linux-gnu`) So first we i) Parse the code ii) Register the PTU just like we do for the host case iii) Then we operate on it (using `GeneratePTX` and `GenerateFatbinary`) https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/136404 >From 87ca1f6992d1413d1d2b2e0d230b4f41e1979fef Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Fri, 18 Apr 2025 18:45:00 +0530 Subject: [PATCH] Fix cuda flag with clang-repl --- clang/include/clang/Interpreter/Interpreter.h | 10 ++- clang/lib/Interpreter/DeviceOffload.cpp | 39 +-- clang/lib/Interpreter/DeviceOffload.h | 4 +- clang/lib/Interpreter/Interpreter.cpp | 65 ++- 4 files changed, 75 insertions(+), 43 deletions(-) diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index b1b63aedf86ab..59089cf639f96 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -95,6 +95,9 @@ class Interpreter { // An optional parser for CUDA offloading std::unique_ptr DeviceParser; + // An optional action for CUDA offloading + std::unique_ptr DeviceAct; + /// List containing information about each incrementally parsed piece of code. std::list PTUs; @@ -175,10 +178,11 @@ class Interpreter { llvm::Expected ExtractValueFromExpr(Expr *E); llvm::Expected CompileDtorCall(CXXRecordDecl *CXXRD); - CodeGenerator *getCodeGen() const; - std::unique_ptr GenModule(); + CodeGenerator *getCodeGen(IncrementalAction *Action = nullptr) const; + std::unique_ptr GenModule(IncrementalAction *Action = nullptr); PartialTranslationUnit &RegisterPTU(TranslationUnitDecl *TU, - std::unique_ptr M = {}); + std::unique_ptr M = {}, + IncrementalAction *Action = nullptr); // A cache for the compiled destructors used to for de-allocation of managed // clang::Values. diff --git a/clang/lib/Interpreter/DeviceOffload.cpp b/clang/lib/Interpreter/DeviceOffload.cpp index 1999d63d1aa04..d9b00787f038d 100644 --- a/clang/lib/Interpreter/DeviceOffload.cpp +++ b/clang/lib/Interpreter/DeviceOffload.cpp @@ -28,20 +28,21 @@ IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr FS, -llvm::Error &Err, const std::list &PTUs) +llvm::Error &Err, std::list &PTUs) : IncrementalParser(*DeviceInstance, Err), PTUs(PTUs), VFS(FS), CodeGenOpts(HostInstance.getCodeGenOpts()), - TargetOpts(HostInstance.getTargetOpts()) { + TargetOpts(DeviceInstance->getTargetOpts()) { if (Err) return; - DeviceCI = std::move(DeviceInstance); StringRef Arch = TargetOpts.CPU; if (!Arch.starts_with("sm_") || Arch.substr(3).getAsInteger(10, SMVersion)) { +DeviceInstance.release(); Err = llvm::joinErrors(std::move(Err), llvm::make_error( "Invalid CUDA architecture", llvm::inconvertibleErrorCode())); return; } + DeviceCI = std::move(DeviceInstance); } llvm::Expected @@ -50,25 +51,6 @@ IncrementalCUDADeviceParser::Parse(llvm::StringRef Input) { if (!PTU) return PTU.takeError(); - auto PTX = GeneratePTX(); - if (!PTX) -return PTX.takeError(); - - auto Err = GenerateFatbinary(); - if (Err) -return std::move(Err); - - std::string FatbinFileName = - "/incr_module_" + std::to_string(PTUs.size()) + ".fatbin"; - VFS->addFile(FatbinFileName, 0, - llvm::MemoryBuffer::getMemBuffer( - llvm::StringRef(FatbinContent.data(), FatbinContent.size()), - "", false)); - - CodeGenOpts.CudaGpuBinaryFileName = FatbinFileName; - - FatbinContent.clear(); - return PTU; } @@ -172,6 +154,19 @@ llvm::Error IncrementalCUDADeviceParser::GenerateFatbinary() { FatbinContent.append(PTXCode.begin(), PTXCode.end()); + auto &PTU = PTUs.back(); + + std::string FatbinFileName = "/" + PTU.TheModule->getName().str() + ".fatbin"; + + VFS->addFile(FatbinFileName, 0, + llvm::MemoryBuffer::getMemBuffer( + llvm::StringRef(FatbinContent.data(), FatbinContent.size()), + "", false)); + + CodeGenOpts.CudaGpuBinaryFileName = FatbinFileName; + + FatbinContent.clear(); + return llvm::Error::success(); } diff --git a/clang/lib/Interpreter/DeviceOffload.h b/clang/lib/Interpreter/DeviceOffload.h index b9a1acab004c3..23d89046c09e1 100644 --- a/clang/lib/Interpreter/DeviceOffload.h +++ b/clang/lib/Interpreter/DeviceOffload.h @@ -24,14 +24,14 @@ class CodeGenOptions; class TargetOptions; class IncrementalCUDADeviceParser : public IncrementalParser { - const std::list &PTUs; + std::list &PTUs; public: IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr VFS, - llvm::Error &Err, const std::list &PTUs); + llvm::Error &Err, std::list
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
@@ -28,20 +28,21 @@ IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr FS, -llvm::Error &Err, const std::list &PTUs) +llvm::Error &Err, std::list &PTUs) : IncrementalParser(*DeviceInstance, Err), PTUs(PTUs), VFS(FS), CodeGenOpts(HostInstance.getCodeGenOpts()), - TargetOpts(HostInstance.getTargetOpts()) { + TargetOpts(DeviceInstance->getTargetOpts()) { anutosh491 wrote: `TargetOpts` should be fetched from `DeviceInstance` rather than `HostInstance` This can be seen even through the cc1 command generated through createCI (skipping some flags) ``` // FOR THE DEVICE (Skipping lot of flags just keeping the main ones) "/Users/anutosh491/work/llvm-project/build2/bin/clang-repl" -cc1 -triple nvptx64-nvidia-cuda -aux-triple arm64-apple-darwin24.3.0 ... -aux-target-cpu apple-m1 -target-cpu sm_35 -target-feature .. --offload-new-driver -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fcxx-exceptions -fexceptions -fcolor-diagnostics -fincremental-extensions -cuid=6c8ca225f780e660 -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o "/var/folders/m1/cdn74f917994jd99d_2cpf44gn/T/<<< inputs >>>-sm_35-14d471.s" -x cuda "<<< inputs >>>" "ptxas" -m64 -O0 -v --gpu-name sm_35 --output-file "<<< inputs >>>-cuda-nvptx64-nvidia-cuda-sm_35.o" "/var/folders/m1/cdn74f917994jd99d_2cpf44gn/T/<<< inputs >>>-sm_35-14d471.s" ``` As seen `-target-cpu sm_35` so the check below with regards to the `Arch` doesn't even make sense if we use it from the host side. `if (!Arch.starts_with("sm_") || Arch.substr(3).getAsInteger(10, SMVersion)) {` https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
@@ -28,20 +28,21 @@ IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr FS, -llvm::Error &Err, const std::list &PTUs) anutosh491 wrote: Thanks for pointing this out. I was probably testing something when i took it out ! https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
@@ -28,20 +28,21 @@ IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr FS, -llvm::Error &Err, const std::list &PTUs) +llvm::Error &Err, std::list &PTUs) : IncrementalParser(*DeviceInstance, Err), PTUs(PTUs), VFS(FS), CodeGenOpts(HostInstance.getCodeGenOpts()), - TargetOpts(HostInstance.getTargetOpts()) { + TargetOpts(DeviceInstance->getTargetOpts()) { if (Err) return; - DeviceCI = std::move(DeviceInstance); StringRef Arch = TargetOpts.CPU; if (!Arch.starts_with("sm_") || Arch.substr(3).getAsInteger(10, SMVersion)) { +DeviceInstance.release(); anutosh491 wrote: Yupp. Got rid of it ! https://github.com/llvm/llvm-project/pull/136404 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] : Fix clang-repl crash with --cuda flag (PR #136404)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/136404 >From 6c64e64c4a3b56f50808cae244b29da1231525f1 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Fri, 18 Apr 2025 18:45:00 +0530 Subject: [PATCH 1/2] Fix cuda flag with clang-repl --- clang/include/clang/Interpreter/Interpreter.h | 13 ++-- clang/lib/Interpreter/DeviceOffload.cpp | 39 ++-- clang/lib/Interpreter/DeviceOffload.h | 4 +- clang/lib/Interpreter/Interpreter.cpp | 59 ++- 4 files changed, 72 insertions(+), 43 deletions(-) diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index b1b63aedf86ab..56213f88b9e30 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -41,6 +41,7 @@ class CXXRecordDecl; class Decl; class IncrementalExecutor; class IncrementalParser; +class IncrementalCUDADeviceParser; /// Create a pre-configured \c CompilerInstance for incremental processing. class IncrementalCompilerBuilder { @@ -93,7 +94,10 @@ class Interpreter { std::unique_ptr IncrExecutor; // An optional parser for CUDA offloading - std::unique_ptr DeviceParser; + std::unique_ptr DeviceParser; + + // An optional action for CUDA offloading + std::unique_ptr DeviceAct; /// List containing information about each incrementally parsed piece of code. std::list PTUs; @@ -175,10 +179,11 @@ class Interpreter { llvm::Expected ExtractValueFromExpr(Expr *E); llvm::Expected CompileDtorCall(CXXRecordDecl *CXXRD); - CodeGenerator *getCodeGen() const; - std::unique_ptr GenModule(); + CodeGenerator *getCodeGen(IncrementalAction *Action = nullptr) const; + std::unique_ptr GenModule(IncrementalAction *Action = nullptr); PartialTranslationUnit &RegisterPTU(TranslationUnitDecl *TU, - std::unique_ptr M = {}); + std::unique_ptr M = {}, + IncrementalAction *Action = nullptr); // A cache for the compiled destructors used to for de-allocation of managed // clang::Values. diff --git a/clang/lib/Interpreter/DeviceOffload.cpp b/clang/lib/Interpreter/DeviceOffload.cpp index 1999d63d1aa04..d9b00787f038d 100644 --- a/clang/lib/Interpreter/DeviceOffload.cpp +++ b/clang/lib/Interpreter/DeviceOffload.cpp @@ -28,20 +28,21 @@ IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr FS, -llvm::Error &Err, const std::list &PTUs) +llvm::Error &Err, std::list &PTUs) : IncrementalParser(*DeviceInstance, Err), PTUs(PTUs), VFS(FS), CodeGenOpts(HostInstance.getCodeGenOpts()), - TargetOpts(HostInstance.getTargetOpts()) { + TargetOpts(DeviceInstance->getTargetOpts()) { if (Err) return; - DeviceCI = std::move(DeviceInstance); StringRef Arch = TargetOpts.CPU; if (!Arch.starts_with("sm_") || Arch.substr(3).getAsInteger(10, SMVersion)) { +DeviceInstance.release(); Err = llvm::joinErrors(std::move(Err), llvm::make_error( "Invalid CUDA architecture", llvm::inconvertibleErrorCode())); return; } + DeviceCI = std::move(DeviceInstance); } llvm::Expected @@ -50,25 +51,6 @@ IncrementalCUDADeviceParser::Parse(llvm::StringRef Input) { if (!PTU) return PTU.takeError(); - auto PTX = GeneratePTX(); - if (!PTX) -return PTX.takeError(); - - auto Err = GenerateFatbinary(); - if (Err) -return std::move(Err); - - std::string FatbinFileName = - "/incr_module_" + std::to_string(PTUs.size()) + ".fatbin"; - VFS->addFile(FatbinFileName, 0, - llvm::MemoryBuffer::getMemBuffer( - llvm::StringRef(FatbinContent.data(), FatbinContent.size()), - "", false)); - - CodeGenOpts.CudaGpuBinaryFileName = FatbinFileName; - - FatbinContent.clear(); - return PTU; } @@ -172,6 +154,19 @@ llvm::Error IncrementalCUDADeviceParser::GenerateFatbinary() { FatbinContent.append(PTXCode.begin(), PTXCode.end()); + auto &PTU = PTUs.back(); + + std::string FatbinFileName = "/" + PTU.TheModule->getName().str() + ".fatbin"; + + VFS->addFile(FatbinFileName, 0, + llvm::MemoryBuffer::getMemBuffer( + llvm::StringRef(FatbinContent.data(), FatbinContent.size()), + "", false)); + + CodeGenOpts.CudaGpuBinaryFileName = FatbinFileName; + + FatbinContent.clear(); + return llvm::Error::success(); } diff --git a/clang/lib/Interpreter/DeviceOffload.h b/clang/lib/Interpreter/DeviceOffload.h index b9a1acab004c3..23d89046c09e1 100644 --- a/clang/lib/Interpreter/DeviceOffload.h +++ b/clang/lib/Interpreter/DeviceOffload.h @@ -24,14 +24,14 @@ class CodeGenOptions; class TargetOptions; class IncrementalCUDA
[clang] [clang-repl] Delegate CodeGen related operations for PTU to IncrementalParser (PR #137458)
https://github.com/anutosh491 created https://github.com/llvm/llvm-project/pull/137458 Read discussion https://github.com/llvm/llvm-project/pull/136404#discussion_r2059149768 and the following comments for context >From 1d773dea825579e43485b99d57931c6fa19f1768 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Sat, 26 Apr 2025 18:56:38 +0530 Subject: [PATCH] Delegate CodeGen related operations for PTU to IncrementalParser --- clang/include/clang/Interpreter/Interpreter.h | 11 - clang/lib/Interpreter/CMakeLists.txt | 1 + clang/lib/Interpreter/DeviceOffload.cpp | 6 +- clang/lib/Interpreter/DeviceOffload.h | 6 +- clang/lib/Interpreter/IncrementalAction.cpp | 110 ++ clang/lib/Interpreter/IncrementalAction.h | 68 ++ clang/lib/Interpreter/IncrementalParser.cpp | 68 +- clang/lib/Interpreter/IncrementalParser.h | 31 ++- clang/lib/Interpreter/Interpreter.cpp | 198 ++ .../Interpreter/InterpreterValuePrinter.cpp | 2 +- 10 files changed, 294 insertions(+), 207 deletions(-) create mode 100644 clang/lib/Interpreter/IncrementalAction.cpp create mode 100644 clang/lib/Interpreter/IncrementalAction.h diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index 56213f88b9e30..145ed409b7807 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -36,7 +36,6 @@ class ThreadSafeContext; namespace clang { class CompilerInstance; -class CodeGenerator; class CXXRecordDecl; class Decl; class IncrementalExecutor; @@ -109,10 +108,6 @@ class Interpreter { // printing happens, it's in an invalid state. Value LastValue; - /// When CodeGen is created the first llvm::Module gets cached in many places - /// and we must keep it alive. - std::unique_ptr CachedInCodeGenModule; - /// Compiler instance performing the incremental compilation. std::unique_ptr CI; @@ -179,12 +174,6 @@ class Interpreter { llvm::Expected ExtractValueFromExpr(Expr *E); llvm::Expected CompileDtorCall(CXXRecordDecl *CXXRD); - CodeGenerator *getCodeGen(IncrementalAction *Action = nullptr) const; - std::unique_ptr GenModule(IncrementalAction *Action = nullptr); - PartialTranslationUnit &RegisterPTU(TranslationUnitDecl *TU, - std::unique_ptr M = {}, - IncrementalAction *Action = nullptr); - // A cache for the compiled destructors used to for de-allocation of managed // clang::Values. llvm::DenseMap Dtors; diff --git a/clang/lib/Interpreter/CMakeLists.txt b/clang/lib/Interpreter/CMakeLists.txt index bf70cdfbee01e..79b8369371db6 100644 --- a/clang/lib/Interpreter/CMakeLists.txt +++ b/clang/lib/Interpreter/CMakeLists.txt @@ -22,6 +22,7 @@ endif() add_clang_library(clangInterpreter DeviceOffload.cpp CodeCompletion.cpp + IncrementalAction.cpp IncrementalExecutor.cpp IncrementalParser.cpp Interpreter.cpp diff --git a/clang/lib/Interpreter/DeviceOffload.cpp b/clang/lib/Interpreter/DeviceOffload.cpp index 7d0125403ea52..5415697d24daf 100644 --- a/clang/lib/Interpreter/DeviceOffload.cpp +++ b/clang/lib/Interpreter/DeviceOffload.cpp @@ -26,10 +26,10 @@ namespace clang { IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, -CompilerInstance &HostInstance, +CompilerInstance &HostInstance, IncrementalAction *DeviceAct, llvm::IntrusiveRefCntPtr FS, -llvm::Error &Err, const std::list &PTUs) -: IncrementalParser(*DeviceInstance, Err), PTUs(PTUs), VFS(FS), +llvm::Error &Err, std::list &PTUs) +: IncrementalParser(*DeviceInstance, DeviceAct, Err, PTUs), VFS(FS), CodeGenOpts(HostInstance.getCodeGenOpts()), TargetOpts(DeviceInstance->getTargetOpts()) { if (Err) diff --git a/clang/lib/Interpreter/DeviceOffload.h b/clang/lib/Interpreter/DeviceOffload.h index 43645033c4840..e4cfd26aa4871 100644 --- a/clang/lib/Interpreter/DeviceOffload.h +++ b/clang/lib/Interpreter/DeviceOffload.h @@ -22,16 +22,16 @@ struct PartialTranslationUnit; class CompilerInstance; class CodeGenOptions; class TargetOptions; +class IncrementalAction; class IncrementalCUDADeviceParser : public IncrementalParser { - const std::list &PTUs; public: IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, - CompilerInstance &HostInstance, + CompilerInstance &HostInstance, IncrementalAction *DeviceAct, llvm::IntrusiveRefCntPtr VFS, - llvm::Error &Err, const std::list &PTUs); + llvm::Error &Err, std::list &PTUs); // Generate PTX for the last PTU. llvm::Expected GeneratePTX(); diff --git a/clang/lib/Interpreter/IncrementalAction.cpp b/clang/lib/Interpreter/IncrementalAction.cpp new file mode 100644 index 0..683925859c645 --- /dev/null +++ b/clang/lib/Interpreter/IncrementalAction.cpp @@ -0,0 +1,110 @@ +//===--- Increm
[clang] [clang-repl] Delegate CodeGen related operations for PTU to IncrementalParser (PR #137458)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/137458 >From 1d773dea825579e43485b99d57931c6fa19f1768 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Sat, 26 Apr 2025 18:56:38 +0530 Subject: [PATCH 1/2] Delegate CodeGen related operations for PTU to IncrementalParser --- clang/include/clang/Interpreter/Interpreter.h | 11 - clang/lib/Interpreter/CMakeLists.txt | 1 + clang/lib/Interpreter/DeviceOffload.cpp | 6 +- clang/lib/Interpreter/DeviceOffload.h | 6 +- clang/lib/Interpreter/IncrementalAction.cpp | 110 ++ clang/lib/Interpreter/IncrementalAction.h | 68 ++ clang/lib/Interpreter/IncrementalParser.cpp | 68 +- clang/lib/Interpreter/IncrementalParser.h | 31 ++- clang/lib/Interpreter/Interpreter.cpp | 198 ++ .../Interpreter/InterpreterValuePrinter.cpp | 2 +- 10 files changed, 294 insertions(+), 207 deletions(-) create mode 100644 clang/lib/Interpreter/IncrementalAction.cpp create mode 100644 clang/lib/Interpreter/IncrementalAction.h diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index 56213f88b9e30..145ed409b7807 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -36,7 +36,6 @@ class ThreadSafeContext; namespace clang { class CompilerInstance; -class CodeGenerator; class CXXRecordDecl; class Decl; class IncrementalExecutor; @@ -109,10 +108,6 @@ class Interpreter { // printing happens, it's in an invalid state. Value LastValue; - /// When CodeGen is created the first llvm::Module gets cached in many places - /// and we must keep it alive. - std::unique_ptr CachedInCodeGenModule; - /// Compiler instance performing the incremental compilation. std::unique_ptr CI; @@ -179,12 +174,6 @@ class Interpreter { llvm::Expected ExtractValueFromExpr(Expr *E); llvm::Expected CompileDtorCall(CXXRecordDecl *CXXRD); - CodeGenerator *getCodeGen(IncrementalAction *Action = nullptr) const; - std::unique_ptr GenModule(IncrementalAction *Action = nullptr); - PartialTranslationUnit &RegisterPTU(TranslationUnitDecl *TU, - std::unique_ptr M = {}, - IncrementalAction *Action = nullptr); - // A cache for the compiled destructors used to for de-allocation of managed // clang::Values. llvm::DenseMap Dtors; diff --git a/clang/lib/Interpreter/CMakeLists.txt b/clang/lib/Interpreter/CMakeLists.txt index bf70cdfbee01e..79b8369371db6 100644 --- a/clang/lib/Interpreter/CMakeLists.txt +++ b/clang/lib/Interpreter/CMakeLists.txt @@ -22,6 +22,7 @@ endif() add_clang_library(clangInterpreter DeviceOffload.cpp CodeCompletion.cpp + IncrementalAction.cpp IncrementalExecutor.cpp IncrementalParser.cpp Interpreter.cpp diff --git a/clang/lib/Interpreter/DeviceOffload.cpp b/clang/lib/Interpreter/DeviceOffload.cpp index 7d0125403ea52..5415697d24daf 100644 --- a/clang/lib/Interpreter/DeviceOffload.cpp +++ b/clang/lib/Interpreter/DeviceOffload.cpp @@ -26,10 +26,10 @@ namespace clang { IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, -CompilerInstance &HostInstance, +CompilerInstance &HostInstance, IncrementalAction *DeviceAct, llvm::IntrusiveRefCntPtr FS, -llvm::Error &Err, const std::list &PTUs) -: IncrementalParser(*DeviceInstance, Err), PTUs(PTUs), VFS(FS), +llvm::Error &Err, std::list &PTUs) +: IncrementalParser(*DeviceInstance, DeviceAct, Err, PTUs), VFS(FS), CodeGenOpts(HostInstance.getCodeGenOpts()), TargetOpts(DeviceInstance->getTargetOpts()) { if (Err) diff --git a/clang/lib/Interpreter/DeviceOffload.h b/clang/lib/Interpreter/DeviceOffload.h index 43645033c4840..e4cfd26aa4871 100644 --- a/clang/lib/Interpreter/DeviceOffload.h +++ b/clang/lib/Interpreter/DeviceOffload.h @@ -22,16 +22,16 @@ struct PartialTranslationUnit; class CompilerInstance; class CodeGenOptions; class TargetOptions; +class IncrementalAction; class IncrementalCUDADeviceParser : public IncrementalParser { - const std::list &PTUs; public: IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, - CompilerInstance &HostInstance, + CompilerInstance &HostInstance, IncrementalAction *DeviceAct, llvm::IntrusiveRefCntPtr VFS, - llvm::Error &Err, const std::list &PTUs); + llvm::Error &Err, std::list &PTUs); // Generate PTX for the last PTU. llvm::Expected GeneratePTX(); diff --git a/clang/lib/Interpreter/IncrementalAction.cpp b/clang/lib/Interpreter/IncrementalAction.cpp new file mode 100644 index 0..683925859c645 --- /dev/null +++ b/clang/lib/Interpreter/IncrementalAction.cpp @@ -0,0 +1,110 @@ +//===--- IncrementalAction.h - Incremental Frontend Action -*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 w
[clang] [clang-repl] Delegate CodeGen related operations for PTU to IncrementalParser (PR #137458)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/137458 >From 1d773dea825579e43485b99d57931c6fa19f1768 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Sat, 26 Apr 2025 18:56:38 +0530 Subject: [PATCH 1/2] Delegate CodeGen related operations for PTU to IncrementalParser --- clang/include/clang/Interpreter/Interpreter.h | 11 - clang/lib/Interpreter/CMakeLists.txt | 1 + clang/lib/Interpreter/DeviceOffload.cpp | 6 +- clang/lib/Interpreter/DeviceOffload.h | 6 +- clang/lib/Interpreter/IncrementalAction.cpp | 110 ++ clang/lib/Interpreter/IncrementalAction.h | 68 ++ clang/lib/Interpreter/IncrementalParser.cpp | 68 +- clang/lib/Interpreter/IncrementalParser.h | 31 ++- clang/lib/Interpreter/Interpreter.cpp | 198 ++ .../Interpreter/InterpreterValuePrinter.cpp | 2 +- 10 files changed, 294 insertions(+), 207 deletions(-) create mode 100644 clang/lib/Interpreter/IncrementalAction.cpp create mode 100644 clang/lib/Interpreter/IncrementalAction.h diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index 56213f88b9e30..145ed409b7807 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -36,7 +36,6 @@ class ThreadSafeContext; namespace clang { class CompilerInstance; -class CodeGenerator; class CXXRecordDecl; class Decl; class IncrementalExecutor; @@ -109,10 +108,6 @@ class Interpreter { // printing happens, it's in an invalid state. Value LastValue; - /// When CodeGen is created the first llvm::Module gets cached in many places - /// and we must keep it alive. - std::unique_ptr CachedInCodeGenModule; - /// Compiler instance performing the incremental compilation. std::unique_ptr CI; @@ -179,12 +174,6 @@ class Interpreter { llvm::Expected ExtractValueFromExpr(Expr *E); llvm::Expected CompileDtorCall(CXXRecordDecl *CXXRD); - CodeGenerator *getCodeGen(IncrementalAction *Action = nullptr) const; - std::unique_ptr GenModule(IncrementalAction *Action = nullptr); - PartialTranslationUnit &RegisterPTU(TranslationUnitDecl *TU, - std::unique_ptr M = {}, - IncrementalAction *Action = nullptr); - // A cache for the compiled destructors used to for de-allocation of managed // clang::Values. llvm::DenseMap Dtors; diff --git a/clang/lib/Interpreter/CMakeLists.txt b/clang/lib/Interpreter/CMakeLists.txt index bf70cdfbee01e..79b8369371db6 100644 --- a/clang/lib/Interpreter/CMakeLists.txt +++ b/clang/lib/Interpreter/CMakeLists.txt @@ -22,6 +22,7 @@ endif() add_clang_library(clangInterpreter DeviceOffload.cpp CodeCompletion.cpp + IncrementalAction.cpp IncrementalExecutor.cpp IncrementalParser.cpp Interpreter.cpp diff --git a/clang/lib/Interpreter/DeviceOffload.cpp b/clang/lib/Interpreter/DeviceOffload.cpp index 7d0125403ea52..5415697d24daf 100644 --- a/clang/lib/Interpreter/DeviceOffload.cpp +++ b/clang/lib/Interpreter/DeviceOffload.cpp @@ -26,10 +26,10 @@ namespace clang { IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, -CompilerInstance &HostInstance, +CompilerInstance &HostInstance, IncrementalAction *DeviceAct, llvm::IntrusiveRefCntPtr FS, -llvm::Error &Err, const std::list &PTUs) -: IncrementalParser(*DeviceInstance, Err), PTUs(PTUs), VFS(FS), +llvm::Error &Err, std::list &PTUs) +: IncrementalParser(*DeviceInstance, DeviceAct, Err, PTUs), VFS(FS), CodeGenOpts(HostInstance.getCodeGenOpts()), TargetOpts(DeviceInstance->getTargetOpts()) { if (Err) diff --git a/clang/lib/Interpreter/DeviceOffload.h b/clang/lib/Interpreter/DeviceOffload.h index 43645033c4840..e4cfd26aa4871 100644 --- a/clang/lib/Interpreter/DeviceOffload.h +++ b/clang/lib/Interpreter/DeviceOffload.h @@ -22,16 +22,16 @@ struct PartialTranslationUnit; class CompilerInstance; class CodeGenOptions; class TargetOptions; +class IncrementalAction; class IncrementalCUDADeviceParser : public IncrementalParser { - const std::list &PTUs; public: IncrementalCUDADeviceParser( std::unique_ptr DeviceInstance, - CompilerInstance &HostInstance, + CompilerInstance &HostInstance, IncrementalAction *DeviceAct, llvm::IntrusiveRefCntPtr VFS, - llvm::Error &Err, const std::list &PTUs); + llvm::Error &Err, std::list &PTUs); // Generate PTX for the last PTU. llvm::Expected GeneratePTX(); diff --git a/clang/lib/Interpreter/IncrementalAction.cpp b/clang/lib/Interpreter/IncrementalAction.cpp new file mode 100644 index 0..683925859c645 --- /dev/null +++ b/clang/lib/Interpreter/IncrementalAction.cpp @@ -0,0 +1,110 @@ +//===--- IncrementalAction.h - Incremental Frontend Action -*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 w
[clang] [clang-repl] Delegate CodeGen related operations for PTU to IncrementalParser (PR #137458)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/137458 Rate limit · GitHub body { background-color: #f6f8fa; color: #24292e; font-family: -apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,Arial,sans-serif,Apple Color Emoji,Segoe UI Emoji,Segoe UI Symbol; font-size: 14px; line-height: 1.5; margin: 0; } .container { margin: 50px auto; max-width: 600px; text-align: center; padding: 0 24px; } a { color: #0366d6; text-decoration: none; } a:hover { text-decoration: underline; } h1 { line-height: 60px; font-size: 48px; font-weight: 300; margin: 0px; text-shadow: 0 1px 0 #fff; } p { color: rgba(0, 0, 0, 0.5); margin: 20px 0 40px; } ul { list-style: none; margin: 25px 0; padding: 0; } li { display: table-cell; font-weight: bold; width: 1%; } .logo { display: inline-block; margin-top: 35px; } .logo-img-2x { display: none; } @media only screen and (-webkit-min-device-pixel-ratio: 2), only screen and ( min--moz-device-pixel-ratio: 2), only screen and ( -o-min-device-pixel-ratio: 2/1), only screen and (min-device-pixel-ratio: 2), only screen and (min-resolution: 192dpi), only screen and (min-resolution: 2dppx) { .logo-img-1x { display: none; } .logo-img-2x { display: inline-block; } } #suggestions { margin-top: 35px; color: #ccc; } #suggestions a { color: #66; font-weight: 200; font-size: 14px; margin: 0 10px; } Whoa there! You have exceeded a secondary rate limit. Please wait a few minutes before you try again; in some cases this may take up to an hour. https://support.github.com/contact";>Contact Support — https://githubstatus.com";>GitHub Status — https://twitter.com/githubstatus";>@githubstatus ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Fix destructor for interpreter for the cuda negation case (PR #138091)
anutosh491 wrote: /cherry-pick https://github.com/llvm/llvm-project/commit/529b6fcb00aabbed17365e5fb3abbc2ae127c967 https://github.com/llvm/llvm-project/pull/138091 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Fix destructor for interpreter for the cuda negation case (PR #138091)
anutosh491 wrote: The cuda tests for the negation case should now work (not segfault because libcudart.so is not found but simply get skipped). ``` 1: Test command: /build/anutosh491/CppInterOp/build1/unittests/CppInterOp/CppInterOpTests/unittests/bin/Release/CppInterOpTests 1: Working Directory: /build/anutosh491/CppInterOp/build1/unittests/CppInterOp 1: Environment variables: 1: CPLUS_INCLUDE_PATH=/build/anutosh491/CppInterOp/build1/etc 1: Test timeout computed to be: 2400 1: Running main() from /build/anutosh491/CppInterOp/build1/unittests/googletest-prefix/src/googletest/googletest/src/gtest_main.cc 1: [==] Running 3 tests from 1 test suite. 1: [--] Global test environment set-up. 1: [--] 3 tests from CUDATest 1: [ RUN ] CUDATest.Sanity 1: Failed load libcudart.so runtime:libcudart1.so: cannot open shared object file: No such file or directory 1: Interpreter creation failed 1: /build/anutosh491/CppInterOp/unittests/CppInterOp/CUDATest.cpp:59: Skipped 1: Skipping CUDA tests as CUDA SDK not found 1: 1: [ SKIPPED ] CUDATest.Sanity (1078 ms) 1: [ RUN ] CUDATest.CUDAH 1: /build/anutosh491/CppInterOp/unittests/CppInterOp/CUDATest.cpp:68: Skipped 1: Skipping CUDA tests as CUDA SDK not found 1: 1: [ SKIPPED ] CUDATest.CUDAH (0 ms) 1: [ RUN ] CUDATest.CUDARuntime 1: /build/anutosh491/CppInterOp/unittests/CppInterOp/CUDATest.cpp:80: Skipped 1: Skipping CUDA tests as CUDA runtime not found 1: 1: [ SKIPPED ] CUDATest.CUDARuntime (0 ms) 1: [--] 3 tests from CUDATest (1078 ms total) 1: 1: [--] Global test environment tear-down 1: [==] 3 tests from 1 test suite ran. (1078 ms total) 1: [ PASSED ] 0 tests. 1: [ SKIPPED ] 3 tests, listed below: 1: [ SKIPPED ] CUDATest.Sanity 1: [ SKIPPED ] CUDATest.CUDAH 1: [ SKIPPED ] CUDATest.CUDARuntime 1/1 Test #1: cppinterop-CppInterOpTests ... Passed1.10 sec ``` And they were anyways working for systems having cuda support ``` 1: Test command: /build/anutosh491/CppInterOp/build1/unittests/CppInterOp/CppInterOpTests/unittests/bin/Release/CppInterOpTests 1: Working Directory: /build/anutosh491/CppInterOp/build1/unittests/CppInterOp 1: Environment variables: 1: CPLUS_INCLUDE_PATH=/build/anutosh491/CppInterOp/build1/etc 1: Test timeout computed to be: 2400 1: Running main() from /build/anutosh491/CppInterOp/build1/unittests/googletest-prefix/src/googletest/googletest/src/gtest_main.cc 1: [==] Running 3 tests from 1 test suite. 1: [--] Global test environment set-up. 1: [--] 3 tests from CUDATest 1: [ RUN ] CUDATest.Sanity 1: [ OK ] CUDATest.Sanity (2177 ms) 1: [ RUN ] CUDATest.CUDAH 1: [ OK ] CUDATest.CUDAH (1016 ms) 1: [ RUN ] CUDATest.CUDARuntime 1: In file included from <<< inputs >>>:1: 1: input_line_1:1:1: warning: expression result unused [-Wunused-value] 1: 1 | (bool)cudaGetLastError() 1: | ^ ~~ 1: /build/anutosh491/CppInterOp/unittests/CppInterOp/CUDATest.cpp:80: Skipped 1: Skipping CUDA tests as CUDA runtime not found 1: 1: [ SKIPPED ] CUDATest.CUDARuntime (1068 ms) 1: [--] 3 tests from CUDATest (4263 ms total) 1: 1: [--] Global test environment tear-down 1: [==] 3 tests from 1 test suite ran. (4263 ms total) 1: [ PASSED ] 2 tests. 1: [ SKIPPED ] 1 test, listed below: 1: [ SKIPPED ] CUDATest.CUDARuntime 1/1 Test #1: cppinterop-CppInterOpTests ... Passed4.32 sec ``` https://github.com/llvm/llvm-project/pull/138091 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Fix destructor for interpreter for the cuda negation case (PR #138091)
https://github.com/anutosh491 created https://github.com/llvm/llvm-project/pull/138091 Check this error for more context (https://github.com/compiler-research/CppInterOp/actions/runs/14749797085/job/41407625681?pr=491#step:10:531) This fails with ``` * thread #1, name = 'CppInterOpTests', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0x55500356d6d3) * frame #0: 0x7fffee41cfe3 libclangCppInterOp.so.21.0gitclang::PragmaNamespace::~PragmaNamespace() + 99 frame #1: 0x7fffee435666 libclangCppInterOp.so.21.0gitclang::Preprocessor::~Preprocessor() + 3830 frame #2: 0x7fffee20917a libclangCppInterOp.so.21.0gitstd::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() + 58 frame #3: 0x7fffee224796 libclangCppInterOp.so.21.0gitclang::CompilerInstance::~CompilerInstance() + 838 frame #4: 0x7fffee22494d libclangCppInterOp.so.21.0gitclang::CompilerInstance::~CompilerInstance() + 13 frame #5: 0x7fffed95ec62 libclangCppInterOp.so.21.0gitclang::IncrementalCUDADeviceParser::~IncrementalCUDADeviceParser() + 98 frame #6: 0x7fffed9551b6 libclangCppInterOp.so.21.0gitclang::Interpreter::~Interpreter() + 102 frame #7: 0x7fffed95598d libclangCppInterOp.so.21.0gitclang::Interpreter::~Interpreter() + 13 frame #8: 0x7fffed9181e7 libclangCppInterOp.so.21.0gitcompat::createClangInterpreter(std::vector>&) + 2919 ``` Problem : 1) The destructor currently handles no clearance for the DeviceParser and the DeviceAct. We currently only have this https://github.com/llvm/llvm-project/blob/976493822443c52a71ed3c67aaca9a555b20c55d/clang/lib/Interpreter/Interpreter.cpp#L416-L419 2) The ownership for DeviceCI currently is present in IncrementalCudaDeviceParser. But this should be similar to how the combination for hostCI, hostAction and hostParser are managed by the Interpreter. As on master the DeviceAct and DeviceParser are managed by the Interpreter but not DeviceCI. This is problematic because : IncrementalParser holds a Sema& which points into the DeviceCI. On master, DeviceCI is destroyed before the base class ~IncrementalParser() runs, causing Parser::reset() to access a dangling Sema (and as Sema holds a reference to Preprocessor which owns PragmaNamespace) we see this ``` * frame #0: 0x7fffee41cfe3 libclangCppInterOp.so.21.0gitclang::PragmaNamespace::~PragmaNamespace() + 99 frame #1: 0x7fffee435666 libclangCppInterOp.so.21.0gitclang::Preprocessor::~Preprocessor() + 3830 ``` >From 8783ea05d2d7f4504978427764da7e78c088aa39 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Thu, 1 May 2025 12:16:06 +0530 Subject: [PATCH] Fix destructor for interpreter for the cuda negation case --- clang/include/clang/Interpreter/Interpreter.h | 3 +++ clang/lib/Interpreter/DeviceOffload.cpp | 7 +++ clang/lib/Interpreter/DeviceOffload.h | 3 +-- clang/lib/Interpreter/Interpreter.cpp | 8 +++- 4 files changed, 14 insertions(+), 7 deletions(-) diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index 56213f88b9e30..f8663e3193a18 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -116,6 +116,9 @@ class Interpreter { /// Compiler instance performing the incremental compilation. std::unique_ptr CI; + /// An optional compiler instance for CUDA offloading + std::unique_ptr DeviceCI; + protected: // Derived classes can use an extended interface of the Interpreter. Interpreter(std::unique_ptr Instance, llvm::Error &Err, diff --git a/clang/lib/Interpreter/DeviceOffload.cpp b/clang/lib/Interpreter/DeviceOffload.cpp index 7d0125403ea52..8ab6a61993972 100644 --- a/clang/lib/Interpreter/DeviceOffload.cpp +++ b/clang/lib/Interpreter/DeviceOffload.cpp @@ -25,13 +25,13 @@ namespace clang { IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( -std::unique_ptr DeviceInstance, +CompilerInstance &DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr FS, llvm::Error &Err, const std::list &PTUs) -: IncrementalParser(*DeviceInstance, Err), PTUs(PTUs), VFS(FS), +: IncrementalParser(DeviceInstance, Err), PTUs(PTUs), VFS(FS), CodeGenOpts(HostInstance.getCodeGenOpts()), - TargetOpts(DeviceInstance->getTargetOpts()) { + TargetOpts(DeviceInstance.getTargetOpts()) { if (Err) return; StringRef Arch = TargetOpts.CPU; @@ -41,7 +41,6 @@ IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( llvm::inconvertibleErrorCode())); return; } - DeviceCI = std::move(DeviceInstance); } llvm::Expected IncrementalCUDADeviceParser::GeneratePTX() { diff --git a/clang/lib/Interpreter/DeviceOffload.h b/clang/lib/Interpreter/DeviceOffload.h index 43645033c4840..c1cb4cd23a4b5 100644 --- a/clang/lib/Interpreter/DeviceOffload.h +++ b/clang/l
[clang] [clang-repl] Fix destructor for interpreter for the cuda negation case (PR #138091)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/138091 >From 8783ea05d2d7f4504978427764da7e78c088aa39 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Thu, 1 May 2025 12:16:06 +0530 Subject: [PATCH 1/2] Fix destructor for interpreter for the cuda negation case --- clang/include/clang/Interpreter/Interpreter.h | 3 +++ clang/lib/Interpreter/DeviceOffload.cpp | 7 +++ clang/lib/Interpreter/DeviceOffload.h | 3 +-- clang/lib/Interpreter/Interpreter.cpp | 8 +++- 4 files changed, 14 insertions(+), 7 deletions(-) diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index 56213f88b9e30..f8663e3193a18 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -116,6 +116,9 @@ class Interpreter { /// Compiler instance performing the incremental compilation. std::unique_ptr CI; + /// An optional compiler instance for CUDA offloading + std::unique_ptr DeviceCI; + protected: // Derived classes can use an extended interface of the Interpreter. Interpreter(std::unique_ptr Instance, llvm::Error &Err, diff --git a/clang/lib/Interpreter/DeviceOffload.cpp b/clang/lib/Interpreter/DeviceOffload.cpp index 7d0125403ea52..8ab6a61993972 100644 --- a/clang/lib/Interpreter/DeviceOffload.cpp +++ b/clang/lib/Interpreter/DeviceOffload.cpp @@ -25,13 +25,13 @@ namespace clang { IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( -std::unique_ptr DeviceInstance, +CompilerInstance &DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr FS, llvm::Error &Err, const std::list &PTUs) -: IncrementalParser(*DeviceInstance, Err), PTUs(PTUs), VFS(FS), +: IncrementalParser(DeviceInstance, Err), PTUs(PTUs), VFS(FS), CodeGenOpts(HostInstance.getCodeGenOpts()), - TargetOpts(DeviceInstance->getTargetOpts()) { + TargetOpts(DeviceInstance.getTargetOpts()) { if (Err) return; StringRef Arch = TargetOpts.CPU; @@ -41,7 +41,6 @@ IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( llvm::inconvertibleErrorCode())); return; } - DeviceCI = std::move(DeviceInstance); } llvm::Expected IncrementalCUDADeviceParser::GeneratePTX() { diff --git a/clang/lib/Interpreter/DeviceOffload.h b/clang/lib/Interpreter/DeviceOffload.h index 43645033c4840..c1cb4cd23a4b5 100644 --- a/clang/lib/Interpreter/DeviceOffload.h +++ b/clang/lib/Interpreter/DeviceOffload.h @@ -28,7 +28,7 @@ class IncrementalCUDADeviceParser : public IncrementalParser { public: IncrementalCUDADeviceParser( - std::unique_ptr DeviceInstance, + CompilerInstance &DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr VFS, llvm::Error &Err, const std::list &PTUs); @@ -42,7 +42,6 @@ class IncrementalCUDADeviceParser : public IncrementalParser { ~IncrementalCUDADeviceParser(); protected: - std::unique_ptr DeviceCI; int SMVersion; llvm::SmallString<1024> PTXCode; llvm::SmallVector FatbinContent; diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 80487eb4ef74a..4ca12f6858c9d 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -416,6 +416,10 @@ Interpreter::Interpreter(std::unique_ptr Instance, Interpreter::~Interpreter() { IncrParser.reset(); Act->FinalizeAction(); + if (DeviceParser) +DeviceParser.reset(); + if (DeviceAct) +DeviceAct->FinalizeAction(); if (IncrExecutor) { if (llvm::Error Err = IncrExecutor->cleanUp()) llvm::report_fatal_error( @@ -501,8 +505,10 @@ Interpreter::createWithCUDA(std::unique_ptr CI, DCI->ExecuteAction(*Interp->DeviceAct); + Interp->DeviceCI = std::move(DCI); + auto DeviceParser = std::make_unique( - std::move(DCI), *Interp->getCompilerInstance(), IMVFS, Err, Interp->PTUs); +*Interp->DeviceCI, *Interp->getCompilerInstance(), IMVFS, Err, Interp->PTUs); if (Err) return std::move(Err); >From 5df3bc6d7c28c728d14b3d484441b00e5074544e Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Thu, 1 May 2025 12:39:08 +0530 Subject: [PATCH 2/2] formatting --- clang/lib/Interpreter/DeviceOffload.cpp | 3 +-- clang/lib/Interpreter/DeviceOffload.h | 3 +-- clang/lib/Interpreter/Interpreter.cpp | 3 ++- 3 files changed, 4 insertions(+), 5 deletions(-) diff --git a/clang/lib/Interpreter/DeviceOffload.cpp b/clang/lib/Interpreter/DeviceOffload.cpp index 8ab6a61993972..05625ddedb72f 100644 --- a/clang/lib/Interpreter/DeviceOffload.cpp +++ b/clang/lib/Interpreter/DeviceOffload.cpp @@ -25,8 +25,7 @@ namespace clang { IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( -CompilerInstance &DeviceInstance, -CompilerInstance &HostInstance, +CompilerInstance &DeviceInstance, CompilerInstance &HostInstance, llvm::In
[clang] [clang-repl] Fix destructor for interpreter for the cuda negation case (PR #138091)
https://github.com/anutosh491 updated https://github.com/llvm/llvm-project/pull/138091 >From aba1800393455fd99feeb9017a4ead4fd582 Mon Sep 17 00:00:00 2001 From: anutosh491 Date: Thu, 1 May 2025 12:16:06 +0530 Subject: [PATCH] Fix destructor for interpreter for the cuda negation case --- clang/include/clang/Interpreter/Interpreter.h | 3 +++ clang/lib/Interpreter/DeviceOffload.cpp | 8 +++- clang/lib/Interpreter/DeviceOffload.h | 4 +--- clang/lib/Interpreter/Interpreter.cpp | 9 - 4 files changed, 15 insertions(+), 9 deletions(-) diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h index 56213f88b9e30..f8663e3193a18 100644 --- a/clang/include/clang/Interpreter/Interpreter.h +++ b/clang/include/clang/Interpreter/Interpreter.h @@ -116,6 +116,9 @@ class Interpreter { /// Compiler instance performing the incremental compilation. std::unique_ptr CI; + /// An optional compiler instance for CUDA offloading + std::unique_ptr DeviceCI; + protected: // Derived classes can use an extended interface of the Interpreter. Interpreter(std::unique_ptr Instance, llvm::Error &Err, diff --git a/clang/lib/Interpreter/DeviceOffload.cpp b/clang/lib/Interpreter/DeviceOffload.cpp index 7d0125403ea52..05625ddedb72f 100644 --- a/clang/lib/Interpreter/DeviceOffload.cpp +++ b/clang/lib/Interpreter/DeviceOffload.cpp @@ -25,13 +25,12 @@ namespace clang { IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( -std::unique_ptr DeviceInstance, -CompilerInstance &HostInstance, +CompilerInstance &DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr FS, llvm::Error &Err, const std::list &PTUs) -: IncrementalParser(*DeviceInstance, Err), PTUs(PTUs), VFS(FS), +: IncrementalParser(DeviceInstance, Err), PTUs(PTUs), VFS(FS), CodeGenOpts(HostInstance.getCodeGenOpts()), - TargetOpts(DeviceInstance->getTargetOpts()) { + TargetOpts(DeviceInstance.getTargetOpts()) { if (Err) return; StringRef Arch = TargetOpts.CPU; @@ -41,7 +40,6 @@ IncrementalCUDADeviceParser::IncrementalCUDADeviceParser( llvm::inconvertibleErrorCode())); return; } - DeviceCI = std::move(DeviceInstance); } llvm::Expected IncrementalCUDADeviceParser::GeneratePTX() { diff --git a/clang/lib/Interpreter/DeviceOffload.h b/clang/lib/Interpreter/DeviceOffload.h index 43645033c4840..0b903e31c6799 100644 --- a/clang/lib/Interpreter/DeviceOffload.h +++ b/clang/lib/Interpreter/DeviceOffload.h @@ -28,8 +28,7 @@ class IncrementalCUDADeviceParser : public IncrementalParser { public: IncrementalCUDADeviceParser( - std::unique_ptr DeviceInstance, - CompilerInstance &HostInstance, + CompilerInstance &DeviceInstance, CompilerInstance &HostInstance, llvm::IntrusiveRefCntPtr VFS, llvm::Error &Err, const std::list &PTUs); @@ -42,7 +41,6 @@ class IncrementalCUDADeviceParser : public IncrementalParser { ~IncrementalCUDADeviceParser(); protected: - std::unique_ptr DeviceCI; int SMVersion; llvm::SmallString<1024> PTXCode; llvm::SmallVector FatbinContent; diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp index 80487eb4ef74a..c04b2099a4b9a 100644 --- a/clang/lib/Interpreter/Interpreter.cpp +++ b/clang/lib/Interpreter/Interpreter.cpp @@ -416,6 +416,10 @@ Interpreter::Interpreter(std::unique_ptr Instance, Interpreter::~Interpreter() { IncrParser.reset(); Act->FinalizeAction(); + if (DeviceParser) +DeviceParser.reset(); + if (DeviceAct) +DeviceAct->FinalizeAction(); if (IncrExecutor) { if (llvm::Error Err = IncrExecutor->cleanUp()) llvm::report_fatal_error( @@ -501,8 +505,11 @@ Interpreter::createWithCUDA(std::unique_ptr CI, DCI->ExecuteAction(*Interp->DeviceAct); + Interp->DeviceCI = std::move(DCI); + auto DeviceParser = std::make_unique( - std::move(DCI), *Interp->getCompilerInstance(), IMVFS, Err, Interp->PTUs); + *Interp->DeviceCI, *Interp->getCompilerInstance(), IMVFS, Err, + Interp->PTUs); if (Err) return std::move(Err); ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Fix destructor for interpreter for the cuda negation case (PR #138091)
anutosh491 wrote: /cherry-pick https://github.com/llvm/llvm-project/commit/529b6fcb00aabbed17365e5fb3abbc2ae127c967 https://github.com/llvm/llvm-project/pull/138091 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-repl] Fix destructor for interpreter for the cuda negation case (PR #138091)
anutosh491 wrote: Deleting the branch and cherry picking seems problematic (https://github.com/llvm/llvm-project/pull/138460#issuecomment-2853718367) hence I've restored it and tried cherry picking again ! https://github.com/llvm/llvm-project/pull/138091 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang] Add support for Debian 14 Forky and Debian 15 Duke (PR #138460)
anutosh491 wrote: I did the same and worked pretty smoothly. Thank you :) https://github.com/llvm/llvm-project/pull/138460 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang] Add support for Debian 14 Forky and Debian 15 Duke (PR #138460)
anutosh491 wrote: > Someone probably ought to open an issue for this caveat. I don't know who would be responsible for looking into this just yet :| Can we raise something and tag someone responsible for this ? https://github.com/llvm/llvm-project/pull/138460 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang] Add support for Debian 14 Forky and Debian 15 Duke (PR #138460)
anutosh491 wrote: > Seems like the cherry-pick only works if the branch of the PR still exists. 🤔 Thanks for pointing this out. I faced a similar issue when I deleted the branch and tried cherry picking ! Now that I've restored the branch, should I try cherry picking again ? https://github.com/llvm/llvm-project/pull/138460 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits