[llvm-branch-commits] [libcxx] [libc++][modules] Adds module testing. (PR #76246)
https://github.com/mordante edited https://github.com/llvm/llvm-project/pull/76246 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [clang] [TySan] A Type Sanitizer (Clang) (PR #76260)
https://github.com/fhahn created https://github.com/llvm/llvm-project/pull/76260 This patch introduces the runtime components of a type sanitizer: a sanitizer for type-based aliasing violations. C/C++ have type-based aliasing rules, and LLVM's optimizer can exploit these given TBAA metadata added by Clang. Roughly, a pointer of given type cannot be used to access an object of a different type (with, of course, certain exceptions). Unfortunately, there's a lot of code in the wild that violates these rules (e.g. for type punning), and such code often must be built with -fno-strict-aliasing. Performance is often sacrificed as a result. Part of the problem is the difficulty of finding TBAA violations. Hopefully, this sanitizer will help. The Clang changes seems mostly formulaic, the one specific change being that when the TBAA sanitizer is enabled, TBAA is always generated, even at -O0. Clang's TBAA representation currently has a problem representing unions, as demonstrated by the one XFAIL'd test in the runtime patch. We'll update the TBAA representation to fix this, and at the same time, update the sanitizer. Based on https://reviews.llvm.org/D32199. ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [clang] [compiler-rt] [TySan] A Type Sanitizer (Runtime Library) (PR #76261)
https://github.com/fhahn created https://github.com/llvm/llvm-project/pull/76261 This patch introduces the runtime components of a type sanitizer: a sanitizer for type-based aliasing violations. C/C++ have type-based aliasing rules, and LLVM's optimizer can exploit these given TBAA metadata added by Clang. Roughly, a pointer of given type cannot be used to access an object of a different type (with, of course, certain exceptions). Unfortunately, there's a lot of code in the wild that violates these rules (e.g. for type punning), and such code often must be built with -fno-strict-aliasing. Performance is often sacrificed as a result. Part of the problem is the difficulty of finding TBAA violations. Hopefully, this sanitizer will help. For each TBAA type-access descriptor, encoded in LLVM's IR using metadata, the corresponding instrumentation pass generates descriptor tables. Thus, for each type (and access descriptor), we have a unique pointer representation. Excepting anonymous-namespace types, these tables are comdat, so the pointer values should be unique across the program. The descriptors refer to other descriptors to form a type aliasing tree (just like LLVM's TBAA metadata does). The instrumentation handles the "fast path" (where the types match exactly and no partial-overlaps are detected), and defers to the runtime to handle all of the more-complicated cases. The runtime, of course, is also responsible for reporting errors when those are detected. The runtime uses essentially the same shadow memory region as tsan, and we use 8 bytes of shadow memory, the size of the pointer to the type descriptor, for every byte of accessed data in the program. The value 0 is used to represent an unknown type. The value -1 is used to represent an interior byte (a byte that is part of a type, but not the first byte). The instrumentation first checks for an exact match between the type of the current access and the type for that address recorded in the shadow memory. If it matches, it then checks the shadow for the remainder of the bytes in the type to make sure that they're all -1. If not, we call the runtime. If the exact match fails, we next check if the value is 0 (i.e. unknown). If it is, then we check the shadow for the remainder of the byes in the type (to make sure they're all 0). If they're not, we call the runtime. We then set the shadow for the access address and set the shadow for the remaining bytes in the type to -1 (i.e. marking them as interior bytes). If the type indicated by the shadow memory for the access address is neither an exact match nor 0, we call the runtime. The instrumentation pass inserts calls to the memset intrinsic to set the memory updated by memset, memcpy, and memmove, as well as allocas/byval (and for lifetime.start/end) to reset the shadow memory to reflect that the type is now unknown. The runtime intercepts memset, memcpy, etc. to perform the same function for the library calls. The runtime essentially repeats these checks, but uses the full TBAA algorithm, just as the compiler does, to determine when two types are permitted to alias. In a situation where access overlap has occurred and aliasing is not permitted, an error is generated. Clang's TBAA representation currently has a problem representing unions, as demonstrated by the one XFAIL'd test. We'll update the TBAA representation to fix this, and at the same time, update the sanitizer. As a note, this implementation does not use the compressed shadow-memory scheme discussed previously (http://lists.llvm.org/pipermail/llvm-dev/2017-April/111766.html). That scheme would not handle the struct-path (i.e. structure offset) information that our TBAA represents. I expect we'll want to further work on compressing the shadow-memory representation, but I think it makes sense to do that as follow-up work. (This includes build fixes for Linux from Mingjie Xu) Based on https://reviews.llvm.org/D32197. ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [clang] [TySan] A Type Sanitizer (Clang) (PR #76260)
llvmbot wrote:
@llvm/pr-subscribers-clang
@llvm/pr-subscribers-clang-driver
Author: Florian Hahn (fhahn)
Changes
This patch introduces the runtime components of a type sanitizer: a sanitizer
for type-based aliasing violations.
C/C++ have type-based aliasing rules, and LLVM's optimizer can exploit these
given TBAA metadata added by Clang. Roughly, a pointer of given type cannot be
used to access an object of a different type (with, of course, certain
exceptions). Unfortunately, there's a lot of code in the wild that violates
these rules (e.g. for type punning), and such code often must be built with
-fno-strict-aliasing. Performance is often sacrificed as a result. Part of the
problem is the difficulty of finding TBAA violations. Hopefully, this sanitizer
will help.
The Clang changes seems mostly formulaic, the one specific change being that
when the TBAA sanitizer is enabled, TBAA is always generated, even at -O0.
Clang's TBAA representation currently has a problem representing unions, as
demonstrated by the one XFAIL'd test in the runtime patch. We'll update the
TBAA representation to fix this, and at the same time, update the sanitizer.
Based on https://reviews.llvm.org/D32199.
---
Patch is 22.30 KiB, truncated to 20.00 KiB below, full version:
https://github.com/llvm/llvm-project/pull/76260.diff
17 Files Affected:
- (modified) clang/include/clang/Basic/Features.def (+1)
- (modified) clang/include/clang/Basic/Sanitizers.def (+3)
- (modified) clang/include/clang/Driver/SanitizerArgs.h (+1)
- (modified) clang/lib/CodeGen/BackendUtil.cpp (+6)
- (modified) clang/lib/CodeGen/CGDecl.cpp (+2-1)
- (modified) clang/lib/CodeGen/CGDeclCXX.cpp (+4)
- (modified) clang/lib/CodeGen/CodeGenFunction.cpp (+2)
- (modified) clang/lib/CodeGen/CodeGenModule.cpp (+7-5)
- (modified) clang/lib/CodeGen/CodeGenTBAA.cpp (+4-2)
- (modified) clang/lib/CodeGen/SanitizerMetadata.cpp (+34-10)
- (modified) clang/lib/CodeGen/SanitizerMetadata.h (+7-6)
- (modified) clang/lib/Driver/SanitizerArgs.cpp (+10-5)
- (modified) clang/lib/Driver/ToolChains/CommonArgs.cpp (+5-1)
- (modified) clang/lib/Driver/ToolChains/Darwin.cpp (+5)
- (modified) clang/lib/Driver/ToolChains/Linux.cpp (+2)
- (added) clang/test/CodeGen/sanitize-type-attr.cpp (+74)
- (modified) clang/test/Driver/sanitizer-ld.c (+23)
``diff
diff --git a/clang/include/clang/Basic/Features.def
b/clang/include/clang/Basic/Features.def
index 06efac0cf1abd7..6a1bc04963af68 100644
--- a/clang/include/clang/Basic/Features.def
+++ b/clang/include/clang/Basic/Features.def
@@ -98,6 +98,7 @@ FEATURE(nullability_nullable_result, true)
FEATURE(memory_sanitizer,
LangOpts.Sanitize.hasOneOf(SanitizerKind::Memory |
SanitizerKind::KernelMemory))
+FEATURE(type_sanitizer, LangOpts.Sanitize.has(SanitizerKind::Type))
FEATURE(thread_sanitizer, LangOpts.Sanitize.has(SanitizerKind::Thread))
FEATURE(dataflow_sanitizer, LangOpts.Sanitize.has(SanitizerKind::DataFlow))
FEATURE(scudo, LangOpts.Sanitize.hasOneOf(SanitizerKind::Scudo))
diff --git a/clang/include/clang/Basic/Sanitizers.def
b/clang/include/clang/Basic/Sanitizers.def
index c2137e3f61f645..a0f54d9e86327a 100644
--- a/clang/include/clang/Basic/Sanitizers.def
+++ b/clang/include/clang/Basic/Sanitizers.def
@@ -73,6 +73,9 @@ SANITIZER("fuzzer", Fuzzer)
// libFuzzer-required instrumentation, no linking.
SANITIZER("fuzzer-no-link", FuzzerNoLink)
+// TypeSanitizer
+SANITIZER("type", Type)
+
// ThreadSanitizer
SANITIZER("thread", Thread)
diff --git a/clang/include/clang/Driver/SanitizerArgs.h
b/clang/include/clang/Driver/SanitizerArgs.h
index 07070ec4fc0653..52b482a0e8a1a9 100644
--- a/clang/include/clang/Driver/SanitizerArgs.h
+++ b/clang/include/clang/Driver/SanitizerArgs.h
@@ -86,6 +86,7 @@ class SanitizerArgs {
bool needsHwasanAliasesRt() const {
return needsHwasanRt() && HwasanUseAliases;
}
+ bool needsTysanRt() const { return Sanitizers.has(SanitizerKind::Type); }
bool needsTsanRt() const { return Sanitizers.has(SanitizerKind::Thread); }
bool needsMsanRt() const { return Sanitizers.has(SanitizerKind::Memory); }
bool needsFuzzer() const { return Sanitizers.has(SanitizerKind::Fuzzer); }
diff --git a/clang/lib/CodeGen/BackendUtil.cpp
b/clang/lib/CodeGen/BackendUtil.cpp
index 480410db1021b7..d7c233b4da3ff5 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -79,6 +79,7 @@
#include "llvm/Transforms/Instrumentation/SanitizerBinaryMetadata.h"
#include "llvm/Transforms/Instrumentation/SanitizerCoverage.h"
#include "llvm/Transforms/Instrumentation/ThreadSanitizer.h"
+#include "llvm/Transforms/Instrumentation/TypeSanitizer.h"
#include "llvm/Transforms/ObjCARC.h"
#include "llvm/Transforms/Scalar/EarlyCSE.h"
#include "llvm/Transforms/Scalar/GVN.h"
@@ -685,6 +686,11 @@ static void addSanitizers(const Triple &TargetTriple,
MPM.addPass(createModuleToFunctionPassAdaptor(ThreadSanitize
[llvm-branch-commits] [llvm] [compiler-rt] [clang] [TySan] A Type Sanitizer (Runtime Library) (PR #76261)
llvmbot wrote: @llvm/pr-subscribers-compiler-rt-sanitizer @llvm/pr-subscribers-clang Author: Florian Hahn (fhahn) Changes This patch introduces the runtime components of a type sanitizer: a sanitizer for type-based aliasing violations. C/C++ have type-based aliasing rules, and LLVM's optimizer can exploit these given TBAA metadata added by Clang. Roughly, a pointer of given type cannot be used to access an object of a different type (with, of course, certain exceptions). Unfortunately, there's a lot of code in the wild that violates these rules (e.g. for type punning), and such code often must be built with -fno-strict-aliasing. Performance is often sacrificed as a result. Part of the problem is the difficulty of finding TBAA violations. Hopefully, this sanitizer will help. For each TBAA type-access descriptor, encoded in LLVM's IR using metadata, the corresponding instrumentation pass generates descriptor tables. Thus, for each type (and access descriptor), we have a unique pointer representation. Excepting anonymous-namespace types, these tables are comdat, so the pointer values should be unique across the program. The descriptors refer to other descriptors to form a type aliasing tree (just like LLVM's TBAA metadata does). The instrumentation handles the "fast path" (where the types match exactly and no partial-overlaps are detected), and defers to the runtime to handle all of the more-complicated cases. The runtime, of course, is also responsible for reporting errors when those are detected. The runtime uses essentially the same shadow memory region as tsan, and we use 8 bytes of shadow memory, the size of the pointer to the type descriptor, for every byte of accessed data in the program. The value 0 is used to represent an unknown type. The value -1 is used to represent an interior byte (a byte that is part of a type, but not the first byte). The instrumentation first checks for an exact match between the type of the current access and the type for that address recorded in the shadow memory. If it matches, it then checks the shadow for the remainder of the bytes in the type to make sure that they're all -1. If not, we call the runtime. If the exact match fails, we next check if the value is 0 (i.e. unknown). If it is, then we check the shadow for the remainder of the byes in the type (to make sure they're all 0). If they're not, we call the runtime. We then set the shadow for the access address and set the shadow for the remaining bytes in the type to -1 (i.e. marking them as interior bytes). If the type indicated by the shadow memory for the access address is neither an exact match nor 0, we call the runtime. The instrumentation pass inserts calls to the memset intrinsic to set the memory updated by memset, memcpy, and memmove, as well as allocas/byval (and for lifetime.start/end) to reset the shadow memory to reflect that the type is now unknown. The runtime intercepts memset, memcpy, etc. to perform the same function for the library calls. The runtime essentially repeats these checks, but uses the full TBAA algorithm, just as the compiler does, to determine when two types are permitted to alias. In a situation where access overlap has occurred and aliasing is not permitted, an error is generated. Clang's TBAA representation currently has a problem representing unions, as demonstrated by the one XFAIL'd test. We'll update the TBAA representation to fix this, and at the same time, update the sanitizer. As a note, this implementation does not use the compressed shadow-memory scheme discussed previously (http://lists.llvm.org/pipermail/llvm-dev/2017-April/111766.html). That scheme would not handle the struct-path (i.e. structure offset) information that our TBAA represents. I expect we'll want to further work on compressing the shadow-memory representation, but I think it makes sense to do that as follow-up work. (This includes build fixes for Linux from Mingjie Xu) Based on https://reviews.llvm.org/D32197. --- Patch is 53.06 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/76261.diff 28 Files Affected: - (modified) clang/runtime/CMakeLists.txt (+1-1) - (modified) compiler-rt/cmake/Modules/AllSupportedArchDefs.cmake (+1) - (modified) compiler-rt/cmake/config-ix.cmake (+14-1) - (added) compiler-rt/lib/tysan/CMakeLists.txt (+64) - (added) compiler-rt/lib/tysan/lit.cfg (+35) - (added) compiler-rt/lib/tysan/lit.site.cfg.in (+12) - (added) compiler-rt/lib/tysan/tysan.cpp (+339) - (added) compiler-rt/lib/tysan/tysan.h (+79) - (added) compiler-rt/lib/tysan/tysan.syms.extra (+2) - (added) compiler-rt/lib/tysan/tysan_flags.inc (+17) - (added) compiler-rt/lib/tysan/tysan_interceptors.cpp (+250) - (added) compiler-rt/lib/tysan/tysan_platform.h (+93) - (added) compiler-rt/lib/tysan/weak_symbols.txt () - (added) compiler-rt/test/tysan/CMakeLists.txt (+32) - (added) compiler-rt/test/tysan/an
[llvm-branch-commits] [llvm] [compiler-rt] [clang] [TySan] A Type Sanitizer (Runtime Library) (PR #76261)
github-actions[bot] wrote: :warning: Python code formatter, darker found issues in your code. :warning: You can test this locally with the following command: ``bash darker --check --diff -r fbaff4033ba060d1478c924ba736be407b4edb80...00e6e657f700e84501cf46a786ccd9389e208ff2 compiler-rt/test/tysan/lit.cfg.py `` View the diff from darker here. ``diff --- lit.cfg.py 2023-12-22 19:18:34.00 + +++ lit.cfg.py 2023-12-22 19:20:58.101017 + @@ -7,133 +7,155 @@ import lit.formats # Get shlex.quote if available (added in 3.3), and fall back to pipes.quote if # it's not available. try: - import shlex - sh_quote = shlex.quote +import shlex + +sh_quote = shlex.quote except: - import pipes - sh_quote = pipes.quote +import pipes + +sh_quote = pipes.quote + def get_required_attr(config, attr_name): - attr_value = getattr(config, attr_name, None) - if attr_value == None: -lit_config.fatal( - "No attribute %r in test configuration! You may need to run " - "tests from your build directory or add this attribute " - "to lit.site.cfg.py " % attr_name) - return attr_value +attr_value = getattr(config, attr_name, None) +if attr_value == None: +lit_config.fatal( +"No attribute %r in test configuration! You may need to run " +"tests from your build directory or add this attribute " +"to lit.site.cfg.py " % attr_name +) +return attr_value + def push_dynamic_library_lookup_path(config, new_path): - if platform.system() == 'Windows': -dynamic_library_lookup_var = 'PATH' - elif platform.system() == 'Darwin': -dynamic_library_lookup_var = 'DYLD_LIBRARY_PATH' - else: -dynamic_library_lookup_var = 'LD_LIBRARY_PATH' +if platform.system() == "Windows": +dynamic_library_lookup_var = "PATH" +elif platform.system() == "Darwin": +dynamic_library_lookup_var = "DYLD_LIBRARY_PATH" +else: +dynamic_library_lookup_var = "LD_LIBRARY_PATH" - new_ld_library_path = os.path.pathsep.join( -(new_path, config.environment.get(dynamic_library_lookup_var, ''))) - config.environment[dynamic_library_lookup_var] = new_ld_library_path +new_ld_library_path = os.path.pathsep.join( +(new_path, config.environment.get(dynamic_library_lookup_var, "")) +) +config.environment[dynamic_library_lookup_var] = new_ld_library_path - if platform.system() == 'FreeBSD': -dynamic_library_lookup_var = 'LD_32_LIBRARY_PATH' -new_ld_32_library_path = os.path.pathsep.join( - (new_path, config.environment.get(dynamic_library_lookup_var, ''))) -config.environment[dynamic_library_lookup_var] = new_ld_32_library_path +if platform.system() == "FreeBSD": +dynamic_library_lookup_var = "LD_32_LIBRARY_PATH" +new_ld_32_library_path = os.path.pathsep.join( +(new_path, config.environment.get(dynamic_library_lookup_var, "")) +) +config.environment[dynamic_library_lookup_var] = new_ld_32_library_path - if platform.system() == 'SunOS': -dynamic_library_lookup_var = 'LD_LIBRARY_PATH_32' -new_ld_library_path_32 = os.path.pathsep.join( - (new_path, config.environment.get(dynamic_library_lookup_var, ''))) -config.environment[dynamic_library_lookup_var] = new_ld_library_path_32 +if platform.system() == "SunOS": +dynamic_library_lookup_var = "LD_LIBRARY_PATH_32" +new_ld_library_path_32 = os.path.pathsep.join( +(new_path, config.environment.get(dynamic_library_lookup_var, "")) +) +config.environment[dynamic_library_lookup_var] = new_ld_library_path_32 -dynamic_library_lookup_var = 'LD_LIBRARY_PATH_64' -new_ld_library_path_64 = os.path.pathsep.join( - (new_path, config.environment.get(dynamic_library_lookup_var, ''))) -config.environment[dynamic_library_lookup_var] = new_ld_library_path_64 +dynamic_library_lookup_var = "LD_LIBRARY_PATH_64" +new_ld_library_path_64 = os.path.pathsep.join( +(new_path, config.environment.get(dynamic_library_lookup_var, "")) +) +config.environment[dynamic_library_lookup_var] = new_ld_library_path_64 + # Setup config name. -config.name = 'TypeSanitizer' + config.name_suffix +config.name = "TypeSanitizer" + config.name_suffix # Platform-specific default TYSAN_OPTIONS for lit tests. default_tysan_opts = list(config.default_sanitizer_opts) # On Darwin, leak checking is not enabled by default. Enable on macOS # tests to prevent regressions -if config.host_os == 'Darwin' and config.apple_platform == 'osx': - default_tysan_opts += ['detect_leaks=1'] +if config.host_os == "Darwin" and config.apple_platform == "osx": +default_tysan_opts += ["detect_leaks=1"] -default_tysan_opts_str = ':'.join(default_tysan_opts) +default_tysan_opts_str = ":".join(default_tysan_opts) if default_tysan_opts_str: - config.environment['TYSAN_OPTIONS'
[llvm-branch-commits] [llvm] [clang] [TySan] A Type Sanitizer (Clang) (PR #76260)
github-actions[bot] wrote: :warning: C/C++ code formatter, clang-format found issues in your code. :warning: You can test this locally with the following command: ``bash git-clang-format --diff 58839969d139a2e492ac217a5b49ac345ed73497 4ee0decc0c0a3e7aad9fc5705720752de8c15b04 -- clang/test/CodeGen/sanitize-type-attr.cpp clang/include/clang/Driver/SanitizerArgs.h clang/lib/CodeGen/BackendUtil.cpp clang/lib/CodeGen/CGDecl.cpp clang/lib/CodeGen/CGDeclCXX.cpp clang/lib/CodeGen/CodeGenFunction.cpp clang/lib/CodeGen/CodeGenModule.cpp clang/lib/CodeGen/CodeGenTBAA.cpp clang/lib/CodeGen/SanitizerMetadata.cpp clang/lib/CodeGen/SanitizerMetadata.h clang/lib/Driver/SanitizerArgs.cpp clang/lib/Driver/ToolChains/CommonArgs.cpp clang/lib/Driver/ToolChains/Darwin.cpp clang/lib/Driver/ToolChains/Linux.cpp clang/test/Driver/sanitizer-ld.c `` View the diff from clang-format here. ``diff diff --git a/clang/lib/Driver/SanitizerArgs.cpp b/clang/lib/Driver/SanitizerArgs.cpp index 6865451b8b..802f46735d 100644 --- a/clang/lib/Driver/SanitizerArgs.cpp +++ b/clang/lib/Driver/SanitizerArgs.cpp @@ -550,10 +550,10 @@ SanitizerArgs::SanitizerArgs(const ToolChain &TC, SanitizerKind::Leak | SanitizerKind::Thread | SanitizerKind::Memory | SanitizerKind::KernelAddress | SanitizerKind::Scudo | SanitizerKind::SafeStack), - std::make_pair(SanitizerKind::MemTag, - SanitizerKind::Address | SanitizerKind::KernelAddress | - SanitizerKind::HWAddress | - SanitizerKind::KernelHWAddress), + std::make_pair(SanitizerKind::MemTag, SanitizerKind::Address | +SanitizerKind::KernelAddress | +SanitizerKind::HWAddress | + SanitizerKind::KernelHWAddress), std::make_pair(SanitizerKind::KCFI, SanitizerKind::Function)}; // Enable toolchain specific default sanitizers if not explicitly disabled. SanitizerMask Default = TC.getDefaultSanitizers() & ~AllRemove; `` https://github.com/llvm/llvm-project/pull/76260 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [clang] [compiler-rt] [TySan] A Type Sanitizer (Runtime Library) (PR #76261)
https://github.com/fhahn edited https://github.com/llvm/llvm-project/pull/76261 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] [clang] [TySan] A Type Sanitizer (Clang) (PR #76260)
https://github.com/fhahn edited https://github.com/llvm/llvm-project/pull/76260 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [libcxx] [libc++][modules] Increase clang-tidy version used. (PR #76268)
https://github.com/mordante created
https://github.com/llvm/llvm-project/pull/76268
As suggested in #71438 we should use
export import std;
in the std.compat module.
Testing this locally failed when building with the clang-tidy-17 plugin. The
std module was considered corrupt in the test
libcxx/test/libcxx/module_std_compat.gen.py
however the test
libcxx/test/libcxx/module_std.gen.py
passed. Both test generated identical std.pcm files. Using the clang-tidy-18
plugin solves the issue.
>From 082ae9517251ef253ca475a8bcdcdb5519806eee Mon Sep 17 00:00:00 2001
From: Mark de Wever
Date: Fri, 22 Dec 2023 21:43:57 +0100
Subject: [PATCH] [libc++][modules] Increase clang-tidy version used.
As suggested in #71438 we should use
export import std;
in the std.compat module.
Testing this locally failed when building with the clang-tidy-17
plugin. The std module was considered corrupt in the test
libcxx/test/libcxx/module_std_compat.gen.py
however the test
libcxx/test/libcxx/module_std.gen.py
passed. Both test generated identical std.pcm files. Using the
clang-tidy-18 plugin solves the issue.
---
libcxx/test/tools/clang_tidy_checks/CMakeLists.txt | 4 ++--
libcxx/utils/libcxx/test/features.py | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/libcxx/test/tools/clang_tidy_checks/CMakeLists.txt
b/libcxx/test/tools/clang_tidy_checks/CMakeLists.txt
index 260e90f45f577c..978e7095216522 100644
--- a/libcxx/test/tools/clang_tidy_checks/CMakeLists.txt
+++ b/libcxx/test/tools/clang_tidy_checks/CMakeLists.txt
@@ -5,9 +5,9 @@
set(LLVM_DIR_SAVE ${LLVM_DIR})
set(Clang_DIR_SAVE ${Clang_DIR})
-find_package(Clang 17)
+find_package(Clang 18)
if (NOT Clang_FOUND)
- find_package(Clang 18)
+ find_package(Clang 17)
endif()
set(SOURCES
diff --git a/libcxx/utils/libcxx/test/features.py
b/libcxx/utils/libcxx/test/features.py
index 58c33e5bb37475..1ada090a9abff7 100644
--- a/libcxx/utils/libcxx/test/features.py
+++ b/libcxx/utils/libcxx/test/features.py
@@ -31,8 +31,8 @@ def _getSuitableClangTidy(cfg):
return None
# TODO MODULES require ToT due module specific fixes.
-if runScriptExitCode(cfg, ['clang-tidy-17 --version']) == 0:
- return 'clang-tidy-17'
+if runScriptExitCode(cfg, ['clang-tidy-18 --version']) == 0:
+ return 'clang-tidy-18'
# TODO This should be the last stable release.
# LLVM RELEASE bump to latest stable version
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [libcxx] [libc++][modules] Increase clang-tidy version used. (PR #76268)
llvmbot wrote:
@llvm/pr-subscribers-libcxx
Author: Mark de Wever (mordante)
Changes
As suggested in #71438 we should use
export import std;
in the std.compat module.
Testing this locally failed when building with the clang-tidy-17 plugin. The
std module was considered corrupt in the test
libcxx/test/libcxx/module_std_compat.gen.py
however the test
libcxx/test/libcxx/module_std.gen.py
passed. Both test generated identical std.pcm files. Using the clang-tidy-18
plugin solves the issue.
---
Full diff: https://github.com/llvm/llvm-project/pull/76268.diff
2 Files Affected:
- (modified) libcxx/test/tools/clang_tidy_checks/CMakeLists.txt (+2-2)
- (modified) libcxx/utils/libcxx/test/features.py (+2-2)
``diff
diff --git a/libcxx/test/tools/clang_tidy_checks/CMakeLists.txt
b/libcxx/test/tools/clang_tidy_checks/CMakeLists.txt
index 260e90f45f577c..978e7095216522 100644
--- a/libcxx/test/tools/clang_tidy_checks/CMakeLists.txt
+++ b/libcxx/test/tools/clang_tidy_checks/CMakeLists.txt
@@ -5,9 +5,9 @@
set(LLVM_DIR_SAVE ${LLVM_DIR})
set(Clang_DIR_SAVE ${Clang_DIR})
-find_package(Clang 17)
+find_package(Clang 18)
if (NOT Clang_FOUND)
- find_package(Clang 18)
+ find_package(Clang 17)
endif()
set(SOURCES
diff --git a/libcxx/utils/libcxx/test/features.py
b/libcxx/utils/libcxx/test/features.py
index 58c33e5bb37475..1ada090a9abff7 100644
--- a/libcxx/utils/libcxx/test/features.py
+++ b/libcxx/utils/libcxx/test/features.py
@@ -31,8 +31,8 @@ def _getSuitableClangTidy(cfg):
return None
# TODO MODULES require ToT due module specific fixes.
-if runScriptExitCode(cfg, ['clang-tidy-17 --version']) == 0:
- return 'clang-tidy-17'
+if runScriptExitCode(cfg, ['clang-tidy-18 --version']) == 0:
+ return 'clang-tidy-18'
# TODO This should be the last stable release.
# LLVM RELEASE bump to latest stable version
``
https://github.com/llvm/llvm-project/pull/76268
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [libcxx] [libc++][modules] Increase clang-tidy version used. (PR #76268)
github-actions[bot] wrote:
:warning: Python code formatter, darker found issues in your code. :warning:
You can test this locally with the following command:
``bash
darker --check --diff -r
10c2d9a35aafe8e21835a6274eddb780baaa6f2c...082ae9517251ef253ca475a8bcdcdb5519806eee
libcxx/utils/libcxx/test/features.py
``
View the diff from darker here.
``diff
--- features.py 2023-12-22 20:43:57.00 +
+++ features.py 2023-12-22 20:53:12.875248 +
@@ -29,12 +29,12 @@
# If we didn't build the libcxx-tidy plugin via CMake, we can't run
the clang-tidy tests.
if runScriptExitCode(cfg, ["stat
%{test-tools}/clang_tidy_checks/libcxx-tidy.plugin"]) != 0:
return None
# TODO MODULES require ToT due module specific fixes.
-if runScriptExitCode(cfg, ['clang-tidy-18 --version']) == 0:
- return 'clang-tidy-18'
+if runScriptExitCode(cfg, ["clang-tidy-18 --version"]) == 0:
+return "clang-tidy-18"
# TODO This should be the last stable release.
# LLVM RELEASE bump to latest stable version
if runScriptExitCode(cfg, ["clang-tidy-16 --version"]) == 0:
return "clang-tidy-16"
``
https://github.com/llvm/llvm-project/pull/76268
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [clang] 4c72931 - Revert "[Sema] Fix crash on invalid code with parenthesized aggregate initialization (#76232)"
Author: Vitaly Buka
Date: 2023-12-22T14:23:46-08:00
New Revision: 4c7293181d3139e16647c3c6daeeb3ae7b344816
URL:
https://github.com/llvm/llvm-project/commit/4c7293181d3139e16647c3c6daeeb3ae7b344816
DIFF:
https://github.com/llvm/llvm-project/commit/4c7293181d3139e16647c3c6daeeb3ae7b344816.diff
LOG: Revert "[Sema] Fix crash on invalid code with parenthesized aggregate
initialization (#76232)"
This reverts commit 86dc6e15f22610bbb53eb4efda0a178ecefc933a.
Added:
Modified:
clang/lib/Sema/SemaInit.cpp
Removed:
clang/test/SemaCXX/crash-GH76228.cpp
diff --git a/clang/lib/Sema/SemaInit.cpp b/clang/lib/Sema/SemaInit.cpp
index cc9db5ded1149a..61d244f3bb9798 100644
--- a/clang/lib/Sema/SemaInit.cpp
+++ b/clang/lib/Sema/SemaInit.cpp
@@ -5512,14 +5512,6 @@ static void TryOrBuildParenListInitialization(
} else if (auto *RT = Entity.getType()->getAs()) {
bool IsUnion = RT->isUnionType();
const CXXRecordDecl *RD = cast(RT->getDecl());
-if (RD->isInvalidDecl()) {
- // Exit early to avoid confusion when processing members.
- // We do the same for braced list initialization in
- // `CheckStructUnionTypes`.
- Sequence.SetFailed(
- clang::InitializationSequence::FK_ParenthesizedListInitFailed);
- return;
-}
if (!IsUnion) {
for (const CXXBaseSpecifier &Base : RD->bases()) {
diff --git a/clang/test/SemaCXX/crash-GH76228.cpp
b/clang/test/SemaCXX/crash-GH76228.cpp
deleted file mode 100644
index 33a9395823127e..00
--- a/clang/test/SemaCXX/crash-GH76228.cpp
+++ /dev/null
@@ -1,28 +0,0 @@
-// RUN: %clang_cc1 -std=c++20 -verify %s
-// Check we don't crash on incomplete members and bases when handling
parenthesized initialization.
-class incomplete; // expected-note@-0 3 {{forward declaration of
'incomplete'}}
-struct foo {
- int a;
- incomplete b;
- // expected-error@-1 {{incomplete type}}
-};
-foo a1(0);
-
-struct one_int {
-int a;
-};
-struct bar : one_int, incomplete {};
-// expected-error@-1 {{incomplete type}}
-bar a2(0);
-
-incomplete a3[3](1,2,3);
-// expected-error@-1 {{incomplete type}}
-
-struct qux : foo {
-};
-qux a4(0);
-
-struct fred {
-foo a[3];
-};
-fred a5(0);
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [libcxx] [libc++][modules] Adds module testing. (PR #76246)
https://github.com/hawkinsw edited https://github.com/llvm/llvm-project/pull/76246 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [libcxx] [libc++][modules] Adds module testing. (PR #76246)
@@ -131,13 +138,62 @@ def parseScript(test, preamble):
script += preamble
script += scriptInTest
+has_std_module = False
+has_std_compat_module = False
+for module in modules:
+if module == "std":
+has_std_module = True
+elif module == "std.compat":
+has_std_compat_module = True
+else:
+ script.insert(0, f"echo \"The module '{module}' is not valid, use
'std' or 'std.compat'\"")
+ script.insert(1, "false");
+ return script
+
+if modules:
+# This flag is needed for both modules.
+moduleCompileFlags.append("-fprebuilt-module-path=%T")
+
+# Building the modules needs to happen before the other script commands
+# are executed. Therefore the commands are added to the front of the
+# list.
+if has_std_compat_module:
+script.insert(
+0,
+"%dbg(MODULE std.compat) %{cxx} %{flags} %{compile_flags} "
+"-Wno-reserved-module-identifier
-Wno-reserved-user-defined-literal "
+"--precompile -o %T/std.compat.pcm -c
%{module}/std.compat.cppm",
+)
+moduleCompileFlags.append("%T/std.compat.pcm")
+
+# Make sure the std module is added before std.compat.
+# Libc++'s std.compat module will depend on its std module.
+# It is not known whether the compiler expects the modules in the order
+# of their dependencies. However it's trivial to provide them in that
+# order.
+if has_std_module:
+script.insert(
+0,
+"%dbg(MODULE std) %{cxx} %{flags} %{compile_flags} "
+"-Wno-reserved-module-identifier
-Wno-reserved-user-defined-literal "
+"--precompile -o %T/std.pcm -c %{module}/std.cppm",
+)
+moduleCompileFlags.append("%T/std.pcm")
+
# Add compile flags specified with ADDITIONAL_COMPILE_FLAGS.
substitutions = [
(s, x + " " + " ".join(additionalCompileFlags))
if s == "%{compile_flags}"
else (s, x)
for (s, x) in substitutions
]
+# In order to use modules additional compilation flags are required.
hawkinsw wrote:
```suggestion
# In order to use modules, additional compilation flags are required.
```
https://github.com/llvm/llvm-project/pull/76246
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [libcxx] [libc++][modules] Adds module testing. (PR #76246)
@@ -223,6 +280,12 @@ class CxxStandardLibraryTest(lit.formats.FileBasedTest):
allows adding special compilation flags without having to use a
.sh.cpp test, which would be more powerful but perhaps overkill.
+// MODULE: std std.compat
+
+ This directive will build the required C++23 standard library
+ modules and add the provide the additional compiler flags in
+ %{module_flags}. (Libc++ offers these modules in C++20 as an
+ extenstion.)
hawkinsw wrote:
```suggestion
extension.)
```
https://github.com/llvm/llvm-project/pull/76246
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [libcxx] [libc++][modules] Adds module testing. (PR #76246)
@@ -223,6 +280,12 @@ class CxxStandardLibraryTest(lit.formats.FileBasedTest): allows adding special compilation flags without having to use a .sh.cpp test, which would be more powerful but perhaps overkill. +// MODULE: std std.compat + + This directive will build the required C++23 standard library + modules and add the provide the additional compiler flags in hawkinsw wrote: ```suggestion modules and add the additional compiler flags in ``` https://github.com/llvm/llvm-project/pull/76246 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [libcxx] [libc++][modules] Adds module testing. (PR #76246)
@@ -131,13 +138,62 @@ def parseScript(test, preamble):
script += preamble
script += scriptInTest
+has_std_module = False
+has_std_compat_module = False
+for module in modules:
+if module == "std":
+has_std_module = True
+elif module == "std.compat":
+has_std_compat_module = True
+else:
+ script.insert(0, f"echo \"The module '{module}' is not valid, use
'std' or 'std.compat'\"")
+ script.insert(1, "false");
+ return script
+
+if modules:
+# This flag is needed for both modules.
+moduleCompileFlags.append("-fprebuilt-module-path=%T")
+
+# Building the modules needs to happen before the other script commands
+# are executed. Therefore the commands are added to the front of the
+# list.
+if has_std_compat_module:
+script.insert(
+0,
+"%dbg(MODULE std.compat) %{cxx} %{flags} %{compile_flags} "
+"-Wno-reserved-module-identifier
-Wno-reserved-user-defined-literal "
+"--precompile -o %T/std.compat.pcm -c
%{module}/std.compat.cppm",
+)
+moduleCompileFlags.append("%T/std.compat.pcm")
+
+# Make sure the std module is added before std.compat.
+# Libc++'s std.compat module will depend on its std module.
+# It is not known whether the compiler expects the modules in the order
+# of their dependencies. However it's trivial to provide them in that
+# order.
+if has_std_module:
+script.insert(
+0,
+"%dbg(MODULE std) %{cxx} %{flags} %{compile_flags} "
+"-Wno-reserved-module-identifier
-Wno-reserved-user-defined-literal "
+"--precompile -o %T/std.pcm -c %{module}/std.cppm",
+)
+moduleCompileFlags.append("%T/std.pcm")
+
# Add compile flags specified with ADDITIONAL_COMPILE_FLAGS.
substitutions = [
(s, x + " " + " ".join(additionalCompileFlags))
if s == "%{compile_flags}"
else (s, x)
for (s, x) in substitutions
]
+# In order to use modules additional compilation flags are required.
+# Adding these to the %{compile_flags} gives a chicken and egg issue:
+# - the modules need to be build with the same compilation flags as the
hawkinsw wrote:
```suggestion
# - the modules need to be built with the same compilation flags as the
```
https://github.com/llvm/llvm-project/pull/76246
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [libcxx] [libc++][modules] Adds module testing. (PR #76246)
https://github.com/hawkinsw commented: If you are okay with the suggestions that I made for some typos in the documentation, I will preemptively incorporate it into my documentation PR. I really like how you integrated module testing so nicely. I hope my comments are helpful! https://github.com/llvm/llvm-project/pull/76246 ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [compiler-rt] [sanitizers] Optimize locking StackDepotBase for fork (PR #76280)
https://github.com/vitalybuka created https://github.com/llvm/llvm-project/pull/76280 Locking StackDepotBase fully is very expensive, as 2^20 buckets needs to be locked. Not locking, and unclocking only buckets, needed to be unlocked to avoid deadlocks, increases a chance of data race, when same item can be inserted into table twice. However this is just a small additional memory usage by forked process. ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [compiler-rt] [sanitizers] Optimize locking StackDepotBase for fork (PR #76280)
llvmbot wrote:
@llvm/pr-subscribers-compiler-rt-sanitizer
Author: Vitaly Buka (vitalybuka)
Changes
Locking StackDepotBase fully is very expensive, as 2^20 buckets needs to
be locked. Not locking, and unclocking only buckets, needed to be
unlocked to avoid deadlocks, increases a chance of data race, when same
item can be inserted into table twice. However this is just a small
additional memory usage by forked process.
---
Full diff: https://github.com/llvm/llvm-project/pull/76280.diff
4 Files Affected:
- (modified) compiler-rt/lib/dfsan/dfsan_chained_origin_depot.cpp (+5-1)
- (modified) compiler-rt/lib/msan/msan_chained_origin_depot.cpp (+7-2)
- (modified) compiler-rt/lib/sanitizer_common/sanitizer_stackdepot.cpp (+12-2)
- (modified) compiler-rt/lib/sanitizer_common/sanitizer_stackdepotbase.h (+2-1)
``diff
diff --git a/compiler-rt/lib/dfsan/dfsan_chained_origin_depot.cpp
b/compiler-rt/lib/dfsan/dfsan_chained_origin_depot.cpp
index 6644bd6a7c6c0c..d078265258ca14 100644
--- a/compiler-rt/lib/dfsan/dfsan_chained_origin_depot.cpp
+++ b/compiler-rt/lib/dfsan/dfsan_chained_origin_depot.cpp
@@ -19,9 +19,13 @@ static ChainedOriginDepot chainedOriginDepot;
ChainedOriginDepot* GetChainedOriginDepot() { return &chainedOriginDepot; }
-void ChainedOriginDepotLockBeforeFork() { chainedOriginDepot.LockAll(); }
+void ChainedOriginDepotLockBeforeFork() {
+ // TODO: Consider same optimization as `StackDepotLockBeforeFork`.
+ chainedOriginDepot.LockAll();
+}
void ChainedOriginDepotUnlockAfterFork(bool fork_child) {
+ // TODO: Consider same optimization as `StackDepotUnlockAfterFork`.
chainedOriginDepot.UnlockAll();
}
diff --git a/compiler-rt/lib/msan/msan_chained_origin_depot.cpp
b/compiler-rt/lib/msan/msan_chained_origin_depot.cpp
index c3bd54141e6c38..66c80708669373 100644
--- a/compiler-rt/lib/msan/msan_chained_origin_depot.cpp
+++ b/compiler-rt/lib/msan/msan_chained_origin_depot.cpp
@@ -31,10 +31,15 @@ u32 ChainedOriginDepotGet(u32 id, u32 *other) {
return chainedOriginDepot.Get(id, other);
}
-void ChainedOriginDepotBeforeFork() { chainedOriginDepot.LockAll(); }
+void ChainedOriginDepotBeforeFork() {
+ // Don't `chainedOriginDepot.LockAll()`, see `StackDepotLockBeforeFork`.
+}
void ChainedOriginDepotAfterFork(bool fork_child) {
- chainedOriginDepot.UnlockAll();
+ // See `StackDepotUnlockAfterFork`.
+ if (fork_child) {
+chainedOriginDepot.UnlockAll();
+ }
}
} // namespace __msan
diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_stackdepot.cpp
b/compiler-rt/lib/sanitizer_common/sanitizer_stackdepot.cpp
index ce21f3c178bce0..8d134ca5a41aee 100644
--- a/compiler-rt/lib/sanitizer_common/sanitizer_stackdepot.cpp
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_stackdepot.cpp
@@ -216,7 +216,13 @@ StackTrace StackDepotGet(u32 id) {
}
void StackDepotLockBeforeFork() {
- theDepot.LockAll();
+ // Do not `theDepot.LockAll()`. It's very slow, but not rely needed. The
+ // parent process will neither lock nor unlock. Child process risks to be
+ // deadlocked on already locked buckets. To avoid deadlock we will unlock
+ // every locked buckets. This may affect consistency of the hash table, but
+ // the only possible issue is a few items inserted by parent process will be
+ // not found by child, and the child may insert them again, wasting some
space
+ // in `stackStore`.
compress_thread.LockAndStop();
stackStore.LockAll();
}
@@ -224,7 +230,11 @@ void StackDepotLockBeforeFork() {
void StackDepotUnlockAfterFork(bool fork_child) {
stackStore.UnlockAll();
compress_thread.Unlock();
- theDepot.UnlockAll();
+ if (fork_child) {
+// Only child process needs to unlock to avoid deadlock. See
+// `StackDepotLockAll`.
+theDepot.UnlockAll();
+ }
}
void StackDepotPrintAll() {
diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_stackdepotbase.h
b/compiler-rt/lib/sanitizer_common/sanitizer_stackdepotbase.h
index 96d1ddc87fd032..3440a8f628e971 100644
--- a/compiler-rt/lib/sanitizer_common/sanitizer_stackdepotbase.h
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_stackdepotbase.h
@@ -171,7 +171,8 @@ void StackDepotBase::UnlockAll() {
for (int i = 0; i < kTabSize; ++i) {
atomic_uint32_t *p = &tab[i];
uptr s = atomic_load(p, memory_order_relaxed);
-unlock(p, s & kUnlockMask);
+if (s & kLockMask)
+ unlock(p, s & kUnlockMask);
}
}
``
https://github.com/llvm/llvm-project/pull/76280
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
