[clang] [llvm] [clang][deps] Cache `VFS::getRealPath()` (PR #68645)

2024-04-10 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 updated https://github.com/llvm/llvm-project/pull/68645 >From fce5325720bcc945baed5923e00d09d84daf58e6 Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Mon, 9 Oct 2023 10:14:17 -0700 Subject: [PATCH 1/6] [clang] Move lookup filename into function --- .../Depend

[clang] [llvm] [clang][deps] Cache `VFS::getRealPath()` (PR #68645)

2024-04-10 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 updated https://github.com/llvm/llvm-project/pull/68645 >From fce5325720bcc945baed5923e00d09d84daf58e6 Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Mon, 9 Oct 2023 10:14:17 -0700 Subject: [PATCH 1/7] [clang] Move lookup filename into function --- .../Depend

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-10 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: > Not sure @jansvoboda11 perhaps if we want to cherry pick > [b768a8c](https://github.com/llvm/llvm-project/commit/b768a8c1db85b9e84fd8b356570a3a8fbe37acf6) > on `release/18.x`? Or should we land just a simple PR with just the function > change above? I can try pulling b76

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-10 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: I'd like to see a unit test specific to `DependencyScanningFilesystem`, similar to what I have here: https://github.com/llvm/llvm-project/pull/68645. https://github.com/llvm/llvm-project/pull/88152 ___ cfe-commits mailing list cfe-

[clang] [llvm] [llvm][clang] Trace VFS calls (PR #88326)

2024-04-10 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 created https://github.com/llvm/llvm-project/pull/88326 None >From 540321e84dbd3c5687cfcc60e9deec79d790896e Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Wed, 10 Apr 2024 16:03:19 -0700 Subject: [PATCH] [llvm][clang] Trace VFS calls --- clang/include/clang/

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-10 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 approved this pull request. LGTM. Would be nice to land Ben's change separately. https://github.com/llvm/llvm-project/pull/88152 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mail

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-10 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: Seems like some tests failed on Linux. https://github.com/llvm/llvm-project/pull/88152 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [clang][deps] Cache `VFS::getRealPath()` (PR #68645)

2024-04-10 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 updated https://github.com/llvm/llvm-project/pull/68645 >From fce5325720bcc945baed5923e00d09d84daf58e6 Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Mon, 9 Oct 2023 10:14:17 -0700 Subject: [PATCH 1/8] [clang] Move lookup filename into function --- .../Depend

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-10 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 edited https://github.com/llvm/llvm-project/pull/88152 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-10 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 requested changes to this pull request. https://github.com/llvm/llvm-project/pull/88152 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-10 Thread Jan Svoboda via cfe-commits
@@ -0,0 +1,51 @@ +//===- DependencyScanningFilesystemTest.cpp ---===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apa

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-10 Thread Jan Svoboda via cfe-commits
@@ -270,6 +270,12 @@ DependencyScanningWorkerFilesystem::status(const Twine &Path) { return Result->getStatus(); } +bool +DependencyScanningWorkerFilesystem::exists(const Twine &Path) { + llvm::ErrorOr Status = status(Path); jansvoboda11 wrote: Would be g

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-11 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: @aganea Ah, got it. Unfortunately, caching stat failures for all directories doesn't work for modules. Clang is supposed to create the modules cache directory if one doesn't exist. But if we first cache its non-existence, Clang will never see it again, even after Clang itse

[clang] [clang-tools-extra] [clang][modules] Do not resolve `HeaderFileInfo` externally in `ASTWriter` (PR #87848)

2024-04-11 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 updated https://github.com/llvm/llvm-project/pull/87848 >From ee56548604be9473f33cd809c901886f37a3d8e9 Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Fri, 5 Apr 2024 15:12:39 -0700 Subject: [PATCH 1/4] [clang][modules] Do not resolve `HeaderFileInfo` externall

[clang] [clang-tools-extra] [clang][modules] Do not resolve `HeaderFileInfo` externally in `ASTWriter` (PR #87848)

2024-04-11 Thread Jan Svoboda via cfe-commits
@@ -409,7 +409,7 @@ class SymbolCollector::HeaderFileURICache { // Framework headers are spelled as , not // "path/FrameworkName.framework/Headers/Foo.h". auto &HS = PP->getHeaderSearchInfo(); -if (const auto *HFI = HS.getExistingFileInfo(*FE, /*WantExternal*/ f

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-11 Thread Jan Svoboda via cfe-commits
@@ -270,6 +270,12 @@ DependencyScanningWorkerFilesystem::status(const Twine &Path) { return Result->getStatus(); } +bool +DependencyScanningWorkerFilesystem::exists(const Twine &Path) { + llvm::ErrorOr Status = status(Path); jansvoboda11 wrote: The one fr

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-11 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 edited https://github.com/llvm/llvm-project/pull/88152 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [clang][deps] Cache `VFS::getRealPath()` (PR #68645)

2024-04-11 Thread Jan Svoboda via cfe-commits
@@ -230,6 +251,26 @@ class DependencyScanningFilesystemLocalCache { assert(InsertedEntry == &Entry && "entry already present"); return *InsertedEntry; } + + /// Returns real path associated with the filename or nullptr if none is + /// found. + const CachedRealPath

[clang] [llvm] [clang][deps] Cache `VFS::getRealPath()` (PR #68645)

2024-04-11 Thread Jan Svoboda via cfe-commits
@@ -226,9 +247,28 @@ class DependencyScanningFilesystemLocalCache { insertEntryForFilename(StringRef Filename, const CachedFileSystemEntry &Entry) { assert(llvm::sys::path::is_absolute_gnu(Filename)); -const auto *InsertedEntry = Cache.insert(

[clang] [llvm] [clang][deps] Cache `VFS::getRealPath()` (PR #68645)

2024-04-11 Thread Jan Svoboda via cfe-commits
@@ -130,11 +130,11 @@ DependencyScanningFilesystemSharedCache::CacheShard:: getOrEmplaceEntryForFilename(StringRef Filename, llvm::ErrorOr Stat) { std::lock_guard LockGuard(CacheLock); - auto Insertion = EntriesByFilename.insert({Filename

[clang] [llvm] [clang][deps] Cache `VFS::getRealPath()` (PR #68645)

2024-04-11 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 updated https://github.com/llvm/llvm-project/pull/68645 >From fce5325720bcc945baed5923e00d09d84daf58e6 Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Mon, 9 Oct 2023 10:14:17 -0700 Subject: [PATCH 01/14] [clang] Move lookup filename into function --- .../Depe

[clang] [clang-tools-extra] [clang][modules] Do not resolve `HeaderFileInfo` externally in `ASTWriter` (PR #87848)

2024-04-11 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 updated https://github.com/llvm/llvm-project/pull/87848 >From ee56548604be9473f33cd809c901886f37a3d8e9 Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Fri, 5 Apr 2024 15:12:39 -0700 Subject: [PATCH 1/5] [clang][modules] Do not resolve `HeaderFileInfo` externall

[clang] [clang-tools-extra] [clang][modules] Do not resolve `HeaderFileInfo` externally in `ASTWriter` (PR #87848)

2024-04-11 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 closed https://github.com/llvm/llvm-project/pull/87848 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [clangd] Don't ignore external HFI in `SymbolCollector` (PR #88446)

2024-04-11 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 created https://github.com/llvm/llvm-project/pull/88446 The `FileEntry` corresponds to a `FileID` containing the `SourceLocation` of a `NamedDecl` which (I think) might've been deserialized from a PCM file. Considering external `HeaderFileInfo` here is most like

[clang] [clang-tools-extra] [clang][modules] Do not resolve `HeaderFileInfo` externally in `ASTWriter` (PR #87848)

2024-04-11 Thread Jan Svoboda via cfe-commits
@@ -409,7 +409,7 @@ class SymbolCollector::HeaderFileURICache { // Framework headers are spelled as , not // "path/FrameworkName.framework/Headers/Foo.h". auto &HS = PP->getHeaderSearchInfo(); -if (const auto *HFI = HS.getExistingFileInfo(*FE, /*WantExternal*/ f

[clang] [clang/DependencyScanning/ModuleDepCollector] Refactor part of `makeCommonInvocationForModuleBuild` into its own function, NFC (PR #88447)

2024-04-11 Thread Jan Svoboda via cfe-commits
@@ -308,6 +308,9 @@ class ModuleDepCollector final : public DependencyCollector { ModuleDeps &Deps); }; +/// Resets some options that introduce dependencies unnecessarily. +void removeUnnecessaryDependencies(CompilerInvocation &CI, bool ForMod

[clang] [clang/DependencyScanning/ModuleDepCollector] Refactor part of `makeCommonInvocationForModuleBuild` into its own function, NFC (PR #88447)

2024-04-11 Thread Jan Svoboda via cfe-commits
@@ -154,6 +154,20 @@ void ModuleDepCollector::addOutputPaths(CowCompilerInvocation &CI, } } +void dependencies::removeUnnecessaryDependencies(CompilerInvocation &CI, + bool ForModuleBuild) { + if (CI.getFrontendOpts().Program

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-11 Thread Jan Svoboda via cfe-commits
@@ -0,0 +1,51 @@ +//===- DependencyScanningFilesystemTest.cpp ---===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apa

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-11 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: Please apply the code formatting suggestions. https://github.com/llvm/llvm-project/pull/88152 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [clangd] Don't ignore external HFI in `SymbolCollector` (PR #88446)

2024-04-11 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 closed https://github.com/llvm/llvm-project/pull/88446 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang/DependencyScanning/ModuleDepCollector] Refactor part of `makeCommonInvocationForModuleBuild` into its own function, NFC (PR #88447)

2024-04-12 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 approved this pull request. LGTM. I assume you'll call `resetBenignCodeGenOptions()` from `ModuleDepCollector::applyDiscoveredDependencies()` in a follow-up, non-NFC patch, right? (So that it applies to the primary invocation, i.e. the actual PCH.) Maybe leaving

[clang] fe59cb2 - [clang][deps] NFCI: Extract `FilenameForLookup` into new function

2024-04-12 Thread Jan Svoboda via cfe-commits
Author: Jan Svoboda Date: 2024-04-12T10:32:58-07:00 New Revision: fe59cb256289d9f2f6ebe75571389f3cc90bbdec URL: https://github.com/llvm/llvm-project/commit/fe59cb256289d9f2f6ebe75571389f3cc90bbdec DIFF: https://github.com/llvm/llvm-project/commit/fe59cb256289d9f2f6ebe75571389f3cc90bbdec.diff L

[clang] c11976f - [clang][deps] Create separate unittest directory

2024-04-12 Thread Jan Svoboda via cfe-commits
Author: Jan Svoboda Date: 2024-04-12T10:32:58-07:00 New Revision: c11976f525f3b9b9dc6080f0b23d2ef1ec5fe8fd URL: https://github.com/llvm/llvm-project/commit/c11976f525f3b9b9dc6080f0b23d2ef1ec5fe8fd DIFF: https://github.com/llvm/llvm-project/commit/c11976f525f3b9b9dc6080f0b23d2ef1ec5fe8fd.diff L

[clang] [clang][deps] Cache `VFS::getRealPath()` (PR #68645)

2024-04-12 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 updated https://github.com/llvm/llvm-project/pull/68645 >From 3b89f001adf027b2128c72c7b907b41717ce1e4c Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Mon, 9 Oct 2023 10:14:22 -0700 Subject: [PATCH 01/13] [clang][deps] Cache `VFS::getRealPath()` --- .../Depend

[clang] [clang][deps] Cache `VFS::getRealPath()` (PR #68645)

2024-04-12 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 closed https://github.com/llvm/llvm-project/pull/68645 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-scan-deps] Fix contention when updating `TrackingStatistic`s in hot code paths in `FileManager`. (PR #88427)

2024-04-12 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: I'd like for @vsapsai to chime in. As an alternative approach: could we turn these into member variables, make them non-atomic and take care to update the stats of the superior `FileManager` whenever an inferior `FileManager` goes out of scope? (e.g. after implicitly build

[clang] a9111d4 - [clang] Remove unused variable in test

2024-04-12 Thread Jan Svoboda via cfe-commits
Author: Jan Svoboda Date: 2024-04-12T12:37:11-07:00 New Revision: a9111d4a26514e169ca57c11e6533ca7c5408832 URL: https://github.com/llvm/llvm-project/commit/a9111d4a26514e169ca57c11e6533ca7c5408832 DIFF: https://github.com/llvm/llvm-project/commit/a9111d4a26514e169ca57c11e6533ca7c5408832.diff L

[clang] [llvm] [llvm][clang] Trace VFS calls (PR #88326)

2024-04-12 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 updated https://github.com/llvm/llvm-project/pull/88326 >From b395e907fcb3895941b58e0d00ef69f14b07c4ae Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Fri, 12 Apr 2024 10:47:13 -0700 Subject: [PATCH] [llvm][support] Implement tracing virtual file system --- ..

[clang] [llvm] [llvm][support] Implement tracing virtual file system (PR #88326)

2024-04-12 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 edited https://github.com/llvm/llvm-project/pull/88326 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [llvm][support] Implement tracing virtual file system (PR #88326)

2024-04-12 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 edited https://github.com/llvm/llvm-project/pull/88326 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [llvm][support] Implement tracing virtual file system (PR #88326)

2024-04-12 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 ready_for_review https://github.com/llvm/llvm-project/pull/88326 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [llvm][support] Implement tracing virtual file system (PR #88326)

2024-04-12 Thread Jan Svoboda via cfe-commits
@@ -1125,6 +1125,54 @@ class YAMLVFSWriter { void write(llvm::raw_ostream &OS); }; +/// File system that tracks the number of calls to the underlying file system. +/// This is particularly useful when wrapped around \c RealFileSystem to add +/// lightweight tracking of expen

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-12 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: I assume clang-format was overly eager with this one, as these are just whitespace changes. Can we undo that? https://github.com/llvm/llvm-project/pull/88152 ___ cfe-commits mailing list cfe-commi

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-12 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: Clang test now looks good to me. Might be nice to drop `InstrumentingInMemoryFilesystem` in favor of the existing `InstrumentingFilesystem` (that I added just moments ago, sorry!) wrapped around a normal `InMemoryFileSystem`, but I'm happy to do that myself in a follow-up

[clang] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-12 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 approved this pull request. LGTM, thanks a lot! https://github.com/llvm/llvm-project/pull/88152 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-12 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 closed https://github.com/llvm/llvm-project/pull/88152 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [llvm][support] Implement tracing virtual file system (PR #88326)

2024-04-12 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 updated https://github.com/llvm/llvm-project/pull/88326 >From 665fd1a3fab4730851d14bc879c34341cbfd6a5e Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Fri, 12 Apr 2024 10:47:13 -0700 Subject: [PATCH] [llvm][support] Implement tracing virtual file system --- ..

[clang] [llvm] [llvm][support] Implement tracing virtual file system (PR #88326)

2024-04-12 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 updated https://github.com/llvm/llvm-project/pull/88326 >From 44791e807564e9c6eb489eccdefb1c0f2647a38a Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Fri, 12 Apr 2024 10:47:13 -0700 Subject: [PATCH] [llvm][support] Implement tracing virtual file system --- ..

[clang] [llvm] [llvm][support] Implement tracing virtual file system (PR #88326)

2024-04-12 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 updated https://github.com/llvm/llvm-project/pull/88326 >From 0afe6a50e4faf95769a17a0d2aaf79b272e991b4 Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Fri, 12 Apr 2024 10:47:13 -0700 Subject: [PATCH] [llvm][support] Implement tracing virtual file system --- ..

[clang] Pass LangOpts from CompilerInstance to DependencyScanningWorker (PR #93753)

2024-06-05 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: > > You can have a project that has both C and C++ implementation files that > > end up including the same header files from the C standard library. One can > > be compiled under C11 (without separator support), the other under C++14 > > (with separator support). > > Thank

[clang] Revert "Pass LangOpts from CompilerInstance to DependencyScanningWork… (PR #94488)

2024-06-05 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 approved this pull request. https://github.com/llvm/llvm-project/pull/94488 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] Revert "Pass LangOpts from CompilerInstance to DependencyScanningWork… (PR #94488)

2024-06-05 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 closed https://github.com/llvm/llvm-project/pull/94488 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Serialization] No transitive identifier change (PR #92085)

2024-06-12 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 approved this pull request. LGTM, but I think it would be better to split out the `LazyIdentifierInfoPtr` change into a separate commit. https://github.com/llvm/llvm-project/pull/92085 ___ cfe-commits mailing list cfe-c

[clang] [clang][modules] HeaderSearch::MarkFileModuleHeader sets textual headers' HeaderFileInfo non-external when it shouldn't (PR #89005)

2024-06-14 Thread Jan Svoboda via cfe-commits
@@ -1313,11 +1313,19 @@ OptionalFileEntryRef HeaderSearch::LookupSubframeworkHeader( // File Info Management. //===--===// +static bool moduleMembershipNeedsMerge(const HeaderFileInfo *HFI, +

[clang] [clang][modules] HeaderSearch::MarkFileModuleHeader sets textual headers' HeaderFileInfo non-external when it shouldn't (PR #89005)

2024-06-14 Thread Jan Svoboda via cfe-commits
@@ -308,5 +308,73 @@ TEST_F(HeaderSearchTest, HeaderMapFrameworkLookup) { EXPECT_EQ(Search.getIncludeNameForHeader(FE), "Foo/Foo.h"); } +TEST_F(HeaderSearchTest, HeaderFileInfoMerge) { + auto AddHeader = [&](std::string HeaderPath) -> FileEntryRef { +VFS->addFile(Header

[clang] [clang][modules] HeaderSearch::MarkFileModuleHeader sets textual headers' HeaderFileInfo non-external when it shouldn't (PR #89005)

2024-06-14 Thread Jan Svoboda via cfe-commits
@@ -1313,11 +1313,19 @@ OptionalFileEntryRef HeaderSearch::LookupSubframeworkHeader( // File Info Management. //===--===// +static bool moduleMembershipNeedsMerge(const HeaderFileInfo *HFI, +

[clang] [clang][modules] HeaderSearch::MarkFileModuleHeader sets textual headers' HeaderFileInfo non-external when it shouldn't (PR #89005)

2024-06-14 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 edited https://github.com/llvm/llvm-project/pull/89005 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang][modules] HeaderSearch::MarkFileModuleHeader sets textual headers' HeaderFileInfo non-external when it shouldn't (PR #89005)

2024-06-14 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 approved this pull request. LGTM! https://github.com/llvm/llvm-project/pull/89005 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang][modules] HeaderSearch::MarkFileModuleHeader sets textual headers' HeaderFileInfo non-external when it shouldn't (PR #89005)

2024-06-14 Thread Jan Svoboda via cfe-commits
@@ -84,7 +84,9 @@ struct HeaderFileInfo { LLVM_PREFERRED_TYPE(bool) unsigned isModuleHeader : 1; - /// Whether this header is a `textual header` in a module. + /// Whether this header is a `textual header` in a module. If a header is + /// textual in one module and norm

[clang] [Serialization] No transitive identifier change (PR #92085)

2024-06-03 Thread Jan Svoboda via cfe-commits
@@ -3896,7 +3903,7 @@ void ASTWriter::WriteIdentifierTable(Preprocessor &PP, // Write out identifiers if either the ID is local or the identifier has // changed since it was loaded. - if (ID >= FirstIdentID || !Chain || !II->isFromAST() || + if (isLocalId

[clang] [Serialization] No transitive identifier change (PR #92085)

2024-06-03 Thread Jan Svoboda via cfe-commits
@@ -918,7 +918,7 @@ ASTSelectorLookupTrait::ReadKey(const unsigned char* d, unsigned) { SelectorTable &SelTable = Reader.getContext().Selectors; unsigned N = endian::readNext(d); const IdentifierInfo *FirstII = Reader.getLocalIdentifier( - F, endian::readNext(d)); +

[clang] [serialization] no transitive decl change (PR #92083)

2024-06-03 Thread Jan Svoboda via cfe-commits
@@ -111,6 +109,28 @@ void *Decl::operator new(std::size_t Size, const ASTContext &Ctx, return ::operator new(Size + Extra, Ctx); } +GlobalDeclID Decl::getGlobalID() const { + if (!isFromASTFile()) +return GlobalDeclID(); + uint64_t ID = *((const uint64_t *)this - 1);

[clang] Pass LangOpts from CompilerInstance to DependencyScanningWorker (PR #93753)

2024-06-03 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: I don't think this is correct. If you concurrently scan the same file under two language standards with the same scanning service, it becomes non-deterministic which one gets cached in the filesystem cache. For subsequent FS queries the cache might return wrong results, ign

[clang] Pass LangOpts from CompilerInstance to DependencyScanningWorker (PR #93753)

2024-06-03 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: > Thanks for the comments @jansvoboda11 . I am new to all these different > moving parts and want to understand better. I have a few questions. > > > If you concurrently scan the same file under two language standards with > > the same scanning service, it becomes non-deter

[clang] [Modules] No transitive source location change (PR #86912)

2024-04-15 Thread Jan Svoboda via cfe-commits
@@ -149,14 +157,44 @@ class SourceLocationSequence::State { operator SourceLocationSequence *() { return &Seq; } }; -inline uint64_t SourceLocationEncoding::encode(SourceLocation Loc, - SourceLocationSequence *Seq) { - return Se

[clang] [clang][deps] Support single-file mode for all formats (PR #88764)

2024-04-15 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 created https://github.com/llvm/llvm-project/pull/88764 The `clang-scan-deps` tool can be used for fast scanning of batches of compilation commands passed in via the `-compilation-database` option. This gets awkward in our tests where we have to resort to using

[clang] [clang][deps] Add `-o` flag to specify output path (PR #88767)

2024-04-15 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 created https://github.com/llvm/llvm-project/pull/88767 This makes it possible to pass "-o /dev/null" to `clang-scan-deps` and skip some potentially expensive work, making timings less noisy. Also removes the need for stream redirection. >From cab6c7b0da73836b3

[clang] [clang][deps] Add `-o` flag to specify output path (PR #88767)

2024-04-15 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 updated https://github.com/llvm/llvm-project/pull/88767 >From cab6c7b0da73836b36201be9a72295811b015367 Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Mon, 15 Apr 2024 10:53:18 -0700 Subject: [PATCH 1/2] [clang][deps] Add `-o` flag to specify output path This m

[clang] [clang][deps] Support single-file mode for all formats (PR #88764)

2024-04-15 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 edited https://github.com/llvm/llvm-project/pull/88764 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang][deps] Support single-file mode for all formats (PR #88764)

2024-04-15 Thread Jan Svoboda via cfe-commits
@@ -694,38 +687,28 @@ static std::string getModuleCachePath(ArrayRef Args) { return std::string(Path); } -// getCompilationDataBase - If -compilation-database is set, load the -// compilation database from the specified file. Otherwise if the we're -// generating P1689 form

[clang] [clang/DependencyScanning/ModuleDepCollector] Refactor part of `makeCommonInvocationForModuleBuild` into its own function (PR #88447)

2024-04-15 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 approved this pull request. Thanks! Still LGTM. https://github.com/llvm/llvm-project/pull/88447 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang][deps] Support single-file mode for all formats (PR #88764)

2024-04-15 Thread Jan Svoboda via cfe-commits
@@ -1,23 +1,10 @@ // RUN: rm -rf %t // RUN: split-file %s %t -//--- missing_tu.json.in -[{ - "directory": "DIR", - "command": "clang -fsyntax-only DIR/missing_tu.c", - "file": "DIR/missing_tu.c" -}] -//--- missing_header.json.in -[{ - "directory": "DIR", - "command": "cla

[clang] [Modules] No transitive source location change (PR #86912)

2024-04-15 Thread Jan Svoboda via cfe-commits
@@ -2220,33 +2221,40 @@ class ASTReader return Sema::AlignPackInfo::getFromRawEncoding(Raw); } + using RawLocEncoding = SourceLocationEncoding::RawLocEncoding; + /// Read a source location from raw form and return it in its /// originating module file's source loc

[clang] [Modules] No transitive source location change (PR #86912)

2024-04-15 Thread Jan Svoboda via cfe-commits
@@ -2220,33 +2221,40 @@ class ASTReader return Sema::AlignPackInfo::getFromRawEncoding(Raw); } + using RawLocEncoding = SourceLocationEncoding::RawLocEncoding; + /// Read a source location from raw form and return it in its /// originating module file's source loc

[clang] [clang][deps] Support single-file mode for all formats (PR #88764)

2024-04-15 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 updated https://github.com/llvm/llvm-project/pull/88764 >From 0c6f6a7794b5bd3264632f54bc3bf56f1dfa118e Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Mon, 15 Apr 2024 10:27:44 -0700 Subject: [PATCH 1/2] [clang][deps] Support single-file mode for all formats --

[clang] [clang][deps] Support single-file mode for all formats (PR #88764)

2024-04-15 Thread Jan Svoboda via cfe-commits
@@ -1,23 +1,10 @@ // RUN: rm -rf %t // RUN: split-file %s %t -//--- missing_tu.json.in -[{ - "directory": "DIR", - "command": "clang -fsyntax-only DIR/missing_tu.c", - "file": "DIR/missing_tu.c" -}] -//--- missing_header.json.in -[{ - "directory": "DIR", - "command": "cla

[clang] [clang][deps] Only bypass scanning VFS for the module cache (PR #88800)

2024-04-15 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 created https://github.com/llvm/llvm-project/pull/88800 The scanning VFS doesn't cache stat failures of paths with no extension. This was originally implemented to avoid caching the non-existence of the modules cache directory that the modular scanner will event

[clang] [clang][deps] Only bypass scanning VFS for the module cache (PR #88800)

2024-04-16 Thread Jan Svoboda via cfe-commits
@@ -201,11 +201,8 @@ const CachedRealPath &DependencyScanningFilesystemSharedCache::CacheShard:: return *StoredRealPath; } -static bool shouldCacheStatFailures(StringRef Filename) { - StringRef Ext = llvm::sys::path::extension(Filename); - if (Ext.empty()) -return fal

[clang] [clang][deps] Only bypass scanning VFS for the module cache (PR #88800)

2024-04-16 Thread Jan Svoboda via cfe-commits
@@ -362,7 +357,7 @@ DependencyScanningWorkerFilesystem::openFileForRead(const Twine &Path) { SmallString<256> OwnedFilename; StringRef Filename = Path.toStringRef(OwnedFilename); - if (Filename.ends_with(".pcm")) + if (shouldBypass(Filename)) jansvoboda

[clang] [clang][deps] Only bypass scanning VFS for the module cache (PR #88800)

2024-04-16 Thread Jan Svoboda via cfe-commits
@@ -201,11 +201,8 @@ const CachedRealPath &DependencyScanningFilesystemSharedCache::CacheShard:: return *StoredRealPath; } -static bool shouldCacheStatFailures(StringRef Filename) { - StringRef Ext = llvm::sys::path::extension(Filename); - if (Ext.empty()) -return fal

[clang] [clang][modules] Only compute affecting module maps with implicit search (PR #87849)

2024-04-16 Thread Jan Svoboda via cfe-commits
@@ -161,8 +161,13 @@ static TypeCode getTypeCodeForTypeClass(Type::TypeClass id) { namespace { -std::set GetAffectingModuleMaps(const Preprocessor &PP, - Module *RootModule) { +std::optional> +GetAffectingModuleMaps(const Pre

[clang] [clang][modules] HeaderSearch::MarkFileModuleHeader creates extra HeaderFileInfo, breaks PCM reuse (PR #89005)

2024-04-16 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: As a test, maybe you could probe the resulting PCM with `-module-file-info`. https://github.com/llvm/llvm-project/pull/89005 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe

[clang] [clang][deps] Support single-file mode for all formats (PR #88764)

2024-04-16 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 closed https://github.com/llvm/llvm-project/pull/88764 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang][deps] Add `-o` flag to specify output path (PR #88767)

2024-04-16 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 closed https://github.com/llvm/llvm-project/pull/88767 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang][modules] HeaderSearch::MarkFileModuleHeader creates extra HeaderFileInfo, breaks PCM reuse (PR #89005)

2024-04-17 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: > getExistingFileInfo(, /*WantExternal=*/false) Until [recently](https://github.com/llvm/llvm-project/pull/87848) that function still consulted `ExternalSource` for `HeaderFileInfo` that `IsValid && !External && !Resolved`. Maybe you want to try the new `getExistingLocalFi

[clang] [clang][modules] HeaderSearch::MarkFileModuleHeader creates extra HeaderFileInfo, breaks PCM reuse (PR #89005)

2024-04-17 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: Also note that `ASTWriter` uses this logic in couple of places to avoid serializing unrelated headers: ```c++ if (!HFI || (HFI->isModuleHeader && !HFI->isCompilingModuleHeader)) continue; ``` Since textual headers from other modules have `isModuleHeader=false` and `isCom

[clang] [libclang] Compute the right spelling location (PR #72400)

2024-04-18 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: The change itself looks good, but I think we should have a test and an entry in release notes. https://github.com/llvm/llvm-project/pull/72400 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/m

[clang] [Rewrite] Fix offset computation in RemoveText (PR #73827)

2024-04-18 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: Sorry, I'm not familiar with the rewriter. Maybe @PiotrZSL would be able to take look? https://github.com/llvm/llvm-project/pull/73827 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/l

[clang] [Modules] No transitive source location change (PR #86912)

2024-04-18 Thread Jan Svoboda via cfe-commits
@@ -2221,33 +,45 @@ class ASTReader return Sema::AlignPackInfo::getFromRawEncoding(Raw); } + using RawLocEncoding = SourceLocationEncoding::RawLocEncoding; + /// Read a source location from raw form and return it in its /// originating module file's source loc

[clang] [Modules] No transitive source location change (PR #86912)

2024-04-18 Thread Jan Svoboda via cfe-commits
@@ -4082,14 +4069,14 @@ void ASTReader::ReadModuleOffsetMap(ModuleFile &F) const { : ModuleMgr.lookupByFileName(Name)); if (!OM) { std::string Msg = - "SourceLocation remap refers to unknown module, cannot find "; + "cannot

[clang] [Modules] No transitive source location change (PR #86912)

2024-04-18 Thread Jan Svoboda via cfe-commits
@@ -5574,10 +5577,34 @@ void ASTWriter::AddFileID(FileID FID, RecordDataImpl &Record) { Record.push_back(getAdjustedFileID(FID).getOpaqueValue()); } +SourceLocationEncoding::RawLocEncoding +ASTWriter::getRawSourceLocationEncoding(SourceLocation Loc, LocSeq *Seq) { + unsign

[clang] [clang-scan-deps] Fix contention when updating `TrackingStatistic`s in hot code paths in `FileManager`. (PR #88427)

2024-04-18 Thread Jan Svoboda via cfe-commits
@@ -1293,6 +1293,10 @@ compileModuleImpl(CompilerInstance &ImportingInstance, SourceLocation ImportLoc, diag::remark_module_build_done) << ModuleName; + // Propagate the statistics to the parent FileManager. + if (FrontendOpts

[clang] [Modules] No transitive source location change (PR #86912)

2024-04-18 Thread Jan Svoboda via cfe-commits
@@ -2220,33 +2221,40 @@ class ASTReader return Sema::AlignPackInfo::getFromRawEncoding(Raw); } + using RawLocEncoding = SourceLocationEncoding::RawLocEncoding; + /// Read a source location from raw form and return it in its /// originating module file's source loc

[clang] [libclang] Compute the right spelling location (PR #72400)

2024-04-19 Thread Jan Svoboda via cfe-commits
@@ -1292,6 +1292,31 @@ void func() {} EXPECT_EQ(attrCount, 1); } +TEST_F(LibclangParseTest, clang_getSpellingLocation) { + std::string fileName = "main.c"; + WriteFile(fileName, "#define X(value) int x = value;\nX(42)\n"); + + ClangTU = clang_parseTranslationUnit(Index, f

[clang] [clang][modules] Only avoid pruning module maps when asked to (PR #89428)

2024-04-19 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 created https://github.com/llvm/llvm-project/pull/89428 Pruning non-affecting module maps is useful even when passing module maps explicitly via `-fmodule-map-file=`. For this situation, this patch reinstates the behavior we had prior to #87849. For the situatio

[clang] [clang][modules] Only avoid pruning module maps when asked to (PR #89428)

2024-04-19 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 updated https://github.com/llvm/llvm-project/pull/89428 >From baa15bb0f35e3f9845407c6b843b82c3a466369f Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Fri, 19 Apr 2024 10:28:33 -0700 Subject: [PATCH 1/2] [clang][modules] Only avoid pruning module maps when aske

[clang] [clang][modules] Only avoid pruning module maps when asked to (PR #89428)

2024-04-19 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 updated https://github.com/llvm/llvm-project/pull/89428 >From baa15bb0f35e3f9845407c6b843b82c3a466369f Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Fri, 19 Apr 2024 10:28:33 -0700 Subject: [PATCH] [clang][modules] Only avoid pruning module maps when asked to

[clang] [clang][modules] Allow module map files with textual headers be non-affecting (PR #89441)

2024-04-19 Thread Jan Svoboda via cfe-commits
https://github.com/jansvoboda11 created https://github.com/llvm/llvm-project/pull/89441 Depends on #89428. >From baa15bb0f35e3f9845407c6b843b82c3a466369f Mon Sep 17 00:00:00 2001 From: Jan Svoboda Date: Fri, 19 Apr 2024 10:28:33 -0700 Subject: [PATCH 1/2] [clang][modules] Only avoid pruning mo

[clang] [clang][modules] HeaderSearch::MarkFileModuleHeader creates extra HeaderFileInfo, breaks PCM reuse (PR #89005)

2024-04-19 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: https://github.com/llvm/llvm-project/pull/89441 might be of interest too. https://github.com/llvm/llvm-project/pull/89005 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-co

[clang] [clang][modules] Allow module map files with textual headers be non-affecting (PR #89441)

2024-04-19 Thread Jan Svoboda via cfe-commits
jansvoboda11 wrote: (Only commit c68b6468347e0a6141ff312139a82a3a8be9ce2e is relevant here, the other one is from #89428.) https://github.com/llvm/llvm-project/pull/89441 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/c

<    4   5   6   7   8   9   10   11   12   13   >