[PATCH] D80296: [clangd] Don't traverse the AST within uninteresting files during indexing

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added a comment.

Correct me if I'm wrong, but this might lead to a file never being updated in 
the index, even after edit, right?
Let's say you have file a.cpp that includes a.h. a.h has
namespace foo {

  #define EXPAND(foo) whatever
  #include "b.h"

}
all nodes from b.h will be inside the namespace node in a.h. If a.h did not 
change, b.h will never be indexed.

It's a corner case, of course, so may the speed gain is worth the cost, but 
don't we have inline #includes like this even in clang codebase?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80296/new/

https://reviews.llvm.org/D80296



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80525: [clangd] Fix crash-bug in preamble indexing when using modules.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
adamcz added a reviewer: sammccall.
Herald added subscribers: cfe-commits, usaxena95, kadircet, arphaman, jkorous, 
MaskRay, ilya-biryukov.
Herald added a project: clang.

When preamble contains #undef, indexing code finds the matching #define
and uses that during indexing. However, it would only look for local
definitions. If the macro was defined in a module, MacroInfo
would be nullptr and clangd would crash.

This change fixes the crash by looking for definition in module when
that happens. The indexing result is then exactly the same whether
modules are used or not.

The indexing of macros happens for preamble only, so then #undef must be
in the preamble, which is why we need two .h files in a test.

Note that clangd is currently not ready for module support, but this
brings us one step closer.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D80525

Files:
  clang-tools-extra/clangd/test/Inputs/symbols-modules/bar.h
  clang-tools-extra/clangd/test/Inputs/symbols-modules/compile_commands.json
  clang-tools-extra/clangd/test/Inputs/symbols-modules/foo.h
  clang-tools-extra/clangd/test/Inputs/symbols-modules/module.map
  clang-tools-extra/clangd/test/symbols-modules.test
  clang/include/clang/Lex/Preprocessor.h
  clang/lib/Index/IndexingAction.cpp

Index: clang/lib/Index/IndexingAction.cpp
===
--- clang/lib/Index/IndexingAction.cpp
+++ clang/lib/Index/IndexingAction.cpp
@@ -147,14 +147,23 @@
   Unit.visitLocalTopLevelDecls(&IndexCtx, topLevelDeclVisitor);
 }
 
-static void indexPreprocessorMacros(const Preprocessor &PP,
+static void indexPreprocessorMacros(Preprocessor &PP,
 IndexDataConsumer &DataConsumer) {
-  for (const auto &M : PP.macros())
-if (MacroDirective *MD = M.second.getLatest())
+  for (const auto &M : PP.macros()) {
+if (MacroDirective *MD = M.second.getLatest()) {
+  auto *MI = MD->getMacroInfo();
+  if (!MI) {
+// It is possible for MI to be nullptr if we our translation unit had
+// only #undef of a macro defined in a module. In that case, dig deeper
+// and use the definition from the module to yield the same indexing
+// result whether we are using modules or not.
+MI = PP.getMacroInfoForLatestDefinition(M.first);
+  }
   DataConsumer.handleMacroOccurrence(
-  M.first, MD->getMacroInfo(),
-  static_cast(index::SymbolRole::Definition),
+  M.first, MI, static_cast(index::SymbolRole::Definition),
   MD->getLocation());
+}
+  }
 }
 
 void index::indexASTUnit(ASTUnit &Unit, IndexDataConsumer &DataConsumer,
Index: clang/include/clang/Lex/Preprocessor.h
===
--- clang/include/clang/Lex/Preprocessor.h
+++ clang/include/clang/Lex/Preprocessor.h
@@ -1115,6 +1115,30 @@
 return nullptr;
   }
 
+  // Returns MacroInfo for the latest definition of II. It could possibly be
+  // imported from a module (thus not accesible via getMacroInfo()), whether
+  // it's active or not.
+  MacroInfo *getMacroInfoForLatestDefinition(const IdentifierInfo *II) {
+if (!II->hasMacroDefinition())
+  return nullptr;
+if (auto *MI = getMacroInfo(II))
+  return MI;
+MacroState &S = CurSubmoduleState->Macros[II];
+auto ActiveModuleMacros = S.getActiveModuleMacros(*this, II);
+for (auto MM = ActiveModuleMacros.rbegin(); MM != ActiveModuleMacros.rend();
+ MM++) {
+  if (auto *MI = (*MM)->getMacroInfo())
+return MI;
+}
+auto OverridenMacros = S.getOverriddenMacros();
+for (auto MM = OverridenMacros.rbegin(); MM != OverridenMacros.rend();
+ MM++) {
+  if (auto *MI = (*MM)->getMacroInfo())
+return MI;
+}
+return nullptr;
+  }
+
   /// Given an identifier, return the latest non-imported macro
   /// directive for that identifier.
   ///
Index: clang-tools-extra/clangd/test/symbols-modules.test
===
--- /dev/null
+++ clang-tools-extra/clangd/test/symbols-modules.test
@@ -0,0 +1,44 @@
+# Verify that we do not crash and correctly find the definition of a macro that
+# was imported from a module and then #undef-ed in preamble.
+#
+# Copy over the test files and fix the paths.
+# RUN: rm -rf %t
+# RUN: cp -r %S/Inputs/symbols-modules %t
+# RUN: sed -i -e "s|DIRECTORY|%t|" %t/compile_commands.json
+#
+# Fix the paths in this file as well. We will pass this copy to clangd.
+# RUN: sed -e "s|DIRECTORY|%/t|g" %s > %t.test.1
+#
+# RUN: clangd -lit-test < %t.test.1 | FileCheck %s
+{"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":123,"rootPath":"clangd","capabilities":{"workspace":{"symbol":{"symbolKind":{"valueSet": [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26],"trace":"off"}}
+---
+{"jsonrpc":"2.0","method":"textDocument/didOpen","pa

[PATCH] D80296: [clangd] Don't traverse the AST within uninteresting files during indexing

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz marked an inline comment as done.
adamcz added a comment.

I have no objections to landing this as-is, even if it represents a regression 
for some evil codebases.

A possible solution is to iterate through all the includes in a given file 
(SourceManager knows that already) and if any of them are outside of preamble, 
do not use this optimization on that file. Should be easy and, hopefully, fast. 
Most files are well-behaved and have includes in preamble only, thus the 
optimization would still kick in most of the time.




Comment at: clang-tools-extra/clangd/unittests/IndexActionTests.cpp:22
 using ::testing::ElementsAre;
+using testing::EndsWith;
 using ::testing::Not;

sammccall wrote:
> kadircet wrote:
> > nit `using ::testing::EndsWith`
> > 
> > was this `add using` tweak ?
> Yes, it was.
> I feel a bit silly as I'm well aware of the behaviour here and have argued 
> it's the right thing, and wrote `testing::EndsWith` inline, extracted it, and 
> didn't check the output (as it was offscreen).
> 
> @adamcz FYI
> If we see this a lot maybe we should consider something like "if all existing 
> usings in the block are globally qualified...". Current behavior is 
> definitely defensible though.
Agreed. I'm starting to think we should have some sort of config file that 
defines style for the codebase (beyond clang-format config, things like 
always-use-::-for-using or never-add-using-for-::std) and possibly guess the 
values, when missing, during background indexing. It could simplify code edits 
a lot.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80296/new/

https://reviews.llvm.org/D80296



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80525: [clangd] Fix crash-bug in preamble indexing when using modules.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 266240.
adamcz marked 3 inline comments as done.
adamcz added a comment.

Addressed review comments.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80525/new/

https://reviews.llvm.org/D80525

Files:
  clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
  clang-tools-extra/clangd/unittests/TestTU.cpp
  clang-tools-extra/clangd/unittests/TestTU.h
  clang/include/clang/Lex/Preprocessor.h
  clang/lib/Index/IndexingAction.cpp

Index: clang/lib/Index/IndexingAction.cpp
===
--- clang/lib/Index/IndexingAction.cpp
+++ clang/lib/Index/IndexingAction.cpp
@@ -147,14 +147,24 @@
   Unit.visitLocalTopLevelDecls(&IndexCtx, topLevelDeclVisitor);
 }
 
-static void indexPreprocessorMacros(const Preprocessor &PP,
+static void indexPreprocessorMacros(Preprocessor &PP,
 IndexDataConsumer &DataConsumer) {
-  for (const auto &M : PP.macros())
-if (MacroDirective *MD = M.second.getLatest())
+  for (const auto &M : PP.macros()) {
+if (MacroDirective *MD = M.second.getLatest()) {
+  auto *MI = MD->getMacroInfo();
+  if (!MI) {
+// It is possible for MI to be nullptr if we our translation unit had
+// only #undef of a macro defined in a module. In that case, dig deeper
+// and use the definition from the module to yield the same indexing
+// result whether we are using modules or not.
+MI = PP.getMacroInfoFromModules(M.first);
+  }
+
   DataConsumer.handleMacroOccurrence(
-  M.first, MD->getMacroInfo(),
-  static_cast(index::SymbolRole::Definition),
+  M.first, MI, static_cast(index::SymbolRole::Definition),
   MD->getLocation());
+}
+  }
 }
 
 void index::indexASTUnit(ASTUnit &Unit, IndexDataConsumer &DataConsumer,
Index: clang/include/clang/Lex/Preprocessor.h
===
--- clang/include/clang/Lex/Preprocessor.h
+++ clang/include/clang/Lex/Preprocessor.h
@@ -1115,6 +1115,28 @@
 return nullptr;
   }
 
+  // Returns MacroInfo for the latest definition of II within a module, nullptr
+  // if this macro was never defined in a module. The return MacroInfo may refer
+  // to definition that was overridden locally.
+  MacroInfo *getMacroInfoFromModules(const IdentifierInfo *II) {
+if (!II->hasMacroDefinition())
+  return nullptr;
+MacroState &S = CurSubmoduleState->Macros[II];
+auto ActiveModuleMacros = S.getActiveModuleMacros(*this, II);
+for (auto MM = ActiveModuleMacros.rbegin(); MM != ActiveModuleMacros.rend();
+ MM++) {
+  if (auto *MI = (*MM)->getMacroInfo())
+return MI;
+}
+auto OverriddenMacros = S.getOverriddenMacros();
+for (auto MM = OverriddenMacros.rbegin(); MM != OverriddenMacros.rend();
+ MM++) {
+  if (auto *MI = (*MM)->getMacroInfo())
+return MI;
+}
+return nullptr;
+  }
+
   /// Given an identifier, return the latest non-imported macro
   /// directive for that identifier.
   ///
Index: clang-tools-extra/clangd/unittests/TestTU.h
===
--- clang-tools-extra/clangd/unittests/TestTU.h
+++ clang-tools-extra/clangd/unittests/TestTU.h
@@ -65,6 +65,10 @@
   // Simulate a header guard of the header (using an #import directive).
   bool ImplicitHeaderGuard = true;
 
+  // Whether to use overlay the TestFS over the real filesystem. This is
+  // required for use of modules.
+  bool OverlayRealFileSystem = false;
+
   // By default, build() will report Error diagnostics as GTest errors.
   // Suppress this behavior by adding an 'error-ok' comment to the code.
   ParsedAST build() const;
Index: clang-tools-extra/clangd/unittests/TestTU.cpp
===
--- clang-tools-extra/clangd/unittests/TestTU.cpp
+++ clang-tools-extra/clangd/unittests/TestTU.cpp
@@ -54,7 +54,14 @@
   Inputs.CompileCommand.Filename = FullFilename;
   Inputs.CompileCommand.Directory = testRoot();
   Inputs.Contents = Code;
-  Inputs.FS = buildTestFS(Files);
+  if (OverlayRealFileSystem) {
+IntrusiveRefCntPtr OverlayFileSystem =
+new llvm::vfs::OverlayFileSystem(llvm::vfs::getRealFileSystem());
+OverlayFileSystem->pushOverlay(buildTestFS(Files));
+Inputs.FS = OverlayFileSystem;
+  } else {
+Inputs.FS = buildTestFS(Files);
+  }
   Inputs.Opts = ParseOptions();
   Inputs.Opts.BuildRecoveryAST = true;
   Inputs.Opts.PreserveRecoveryASTType = true;
Index: clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
===
--- clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
+++ clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
@@ -1490,6 +1490,29 @@
   EXPECT_THAT(Symbols, Contains(QName("operator delete")));
 }
 
+TEST_F(

[PATCH] D77656: [clangd] Fix a crash bug in AddUsing tweak around template handling.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 255694.
adamcz added a comment.

Removed unnecessary change.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77656/new/

https://reviews.llvm.org/D77656

Files:
  clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
  clang-tools-extra/clangd/unittests/TweakTests.cpp


Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -2436,6 +2436,7 @@
 #define NS(name) one::two::name
 namespace one {
 void oo() {}
+template class tt {};
 namespace two {
 enum ee {};
 void ff() {}
@@ -2458,6 +2459,10 @@
   EXPECT_UNAVAILABLE(Header +
  "void fun() { o^n^e^:^:^t^w^o^:^:^c^c^:^:^s^t inst; }");
   EXPECT_UNAVAILABLE(Header + "void fun() { N^S(c^c) inst; }");
+  // This used to crash. Ideally we would support this case, but for now we 
just
+  // test that we don't crash.
+  EXPECT_UNAVAILABLE(Header +
+ "template using foo = one::tt;");
 }
 
 TEST_F(AddUsingTest, Apply) {
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -206,9 +206,12 @@
 Name = D->getDecl()->getName();
   } else if (auto *T = Node->ASTNode.get()) {
 if (auto E = T->getAs()) {
-  QualifierToRemove = E.getQualifierLoc();
-  Name =
-  E.getType().getUnqualifiedType().getBaseTypeIdentifier()->getName();
+  if (auto *BaseTypeIdentifier =
+  E.getType().getUnqualifiedType().getBaseTypeIdentifier()) {
+Name =
+
E.getType().getUnqualifiedType().getBaseTypeIdentifier()->getName();
+QualifierToRemove = E.getQualifierLoc();
+  }
 }
   }
 


Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -2436,6 +2436,7 @@
 #define NS(name) one::two::name
 namespace one {
 void oo() {}
+template class tt {};
 namespace two {
 enum ee {};
 void ff() {}
@@ -2458,6 +2459,10 @@
   EXPECT_UNAVAILABLE(Header +
  "void fun() { o^n^e^:^:^t^w^o^:^:^c^c^:^:^s^t inst; }");
   EXPECT_UNAVAILABLE(Header + "void fun() { N^S(c^c) inst; }");
+  // This used to crash. Ideally we would support this case, but for now we just
+  // test that we don't crash.
+  EXPECT_UNAVAILABLE(Header +
+ "template using foo = one::tt;");
 }
 
 TEST_F(AddUsingTest, Apply) {
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -206,9 +206,12 @@
 Name = D->getDecl()->getName();
   } else if (auto *T = Node->ASTNode.get()) {
 if (auto E = T->getAs()) {
-  QualifierToRemove = E.getQualifierLoc();
-  Name =
-  E.getType().getUnqualifiedType().getBaseTypeIdentifier()->getName();
+  if (auto *BaseTypeIdentifier =
+  E.getType().getUnqualifiedType().getBaseTypeIdentifier()) {
+Name =
+E.getType().getUnqualifiedType().getBaseTypeIdentifier()->getName();
+QualifierToRemove = E.getQualifierLoc();
+  }
 }
   }
 
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D77656: [clangd] Fix a crash bug in AddUsing tweak around template handling.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
adamcz added a reviewer: sammccall.
Herald added subscribers: cfe-commits, usaxena95, kadircet, arphaman, jkorous, 
MaskRay, ilya-biryukov.
Herald added a project: clang.
adamcz updated this revision to Diff 255694.
adamcz added a comment.

Removed unnecessary change.


The crash happened on cases like:
template using one = two::three;
because we tried to call getName() on getBaseTypeIdentifier(), which can
be nullptr.

Ideally we would support this use case as well, but for now not crashing
will do.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D77656

Files:
  clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
  clang-tools-extra/clangd/unittests/TweakTests.cpp


Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -2436,6 +2436,7 @@
 #define NS(name) one::two::name
 namespace one {
 void oo() {}
+template class tt {};
 namespace two {
 enum ee {};
 void ff() {}
@@ -2458,6 +2459,10 @@
   EXPECT_UNAVAILABLE(Header +
  "void fun() { o^n^e^:^:^t^w^o^:^:^c^c^:^:^s^t inst; }");
   EXPECT_UNAVAILABLE(Header + "void fun() { N^S(c^c) inst; }");
+  // This used to crash. Ideally we would support this case, but for now we 
just
+  // test that we don't crash.
+  EXPECT_UNAVAILABLE(Header +
+ "template using foo = one::tt;");
 }
 
 TEST_F(AddUsingTest, Apply) {
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -206,9 +206,12 @@
 Name = D->getDecl()->getName();
   } else if (auto *T = Node->ASTNode.get()) {
 if (auto E = T->getAs()) {
-  QualifierToRemove = E.getQualifierLoc();
-  Name =
-  E.getType().getUnqualifiedType().getBaseTypeIdentifier()->getName();
+  if (auto *BaseTypeIdentifier =
+  E.getType().getUnqualifiedType().getBaseTypeIdentifier()) {
+Name =
+
E.getType().getUnqualifiedType().getBaseTypeIdentifier()->getName();
+QualifierToRemove = E.getQualifierLoc();
+  }
 }
   }
 


Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -2436,6 +2436,7 @@
 #define NS(name) one::two::name
 namespace one {
 void oo() {}
+template class tt {};
 namespace two {
 enum ee {};
 void ff() {}
@@ -2458,6 +2459,10 @@
   EXPECT_UNAVAILABLE(Header +
  "void fun() { o^n^e^:^:^t^w^o^:^:^c^c^:^:^s^t inst; }");
   EXPECT_UNAVAILABLE(Header + "void fun() { N^S(c^c) inst; }");
+  // This used to crash. Ideally we would support this case, but for now we just
+  // test that we don't crash.
+  EXPECT_UNAVAILABLE(Header +
+ "template using foo = one::tt;");
 }
 
 TEST_F(AddUsingTest, Apply) {
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -206,9 +206,12 @@
 Name = D->getDecl()->getName();
   } else if (auto *T = Node->ASTNode.get()) {
 if (auto E = T->getAs()) {
-  QualifierToRemove = E.getQualifierLoc();
-  Name =
-  E.getType().getUnqualifiedType().getBaseTypeIdentifier()->getName();
+  if (auto *BaseTypeIdentifier =
+  E.getType().getUnqualifiedType().getBaseTypeIdentifier()) {
+Name =
+E.getType().getUnqualifiedType().getBaseTypeIdentifier()->getName();
+QualifierToRemove = E.getQualifierLoc();
+  }
 }
   }
 
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D77656: [clangd] Fix a crash bug in AddUsing tweak around template handling.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz marked an inline comment as done.
adamcz added a comment.

Can you submit this? Thanks!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77656/new/

https://reviews.llvm.org/D77656



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D77656: [clangd] Fix a crash bug in AddUsing tweak around template handling.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 255950.
adamcz added a comment.

review comment


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77656/new/

https://reviews.llvm.org/D77656

Files:
  clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
  clang-tools-extra/clangd/unittests/TweakTests.cpp


Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -2436,6 +2436,7 @@
 #define NS(name) one::two::name
 namespace one {
 void oo() {}
+template class tt {};
 namespace two {
 enum ee {};
 void ff() {}
@@ -2458,6 +2459,10 @@
   EXPECT_UNAVAILABLE(Header +
  "void fun() { o^n^e^:^:^t^w^o^:^:^c^c^:^:^s^t inst; }");
   EXPECT_UNAVAILABLE(Header + "void fun() { N^S(c^c) inst; }");
+  // This used to crash. Ideally we would support this case, but for now we 
just
+  // test that we don't crash.
+  EXPECT_UNAVAILABLE(Header +
+ "template using foo = one::tt;");
 }
 
 TEST_F(AddUsingTest, Apply) {
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -206,9 +206,11 @@
 Name = D->getDecl()->getName();
   } else if (auto *T = Node->ASTNode.get()) {
 if (auto E = T->getAs()) {
-  QualifierToRemove = E.getQualifierLoc();
-  Name =
-  E.getType().getUnqualifiedType().getBaseTypeIdentifier()->getName();
+  if (auto *BaseTypeIdentifier =
+  E.getType().getUnqualifiedType().getBaseTypeIdentifier()) {
+Name = BaseTypeIdentifier->getName();
+QualifierToRemove = E.getQualifierLoc();
+  }
 }
   }
 


Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -2436,6 +2436,7 @@
 #define NS(name) one::two::name
 namespace one {
 void oo() {}
+template class tt {};
 namespace two {
 enum ee {};
 void ff() {}
@@ -2458,6 +2459,10 @@
   EXPECT_UNAVAILABLE(Header +
  "void fun() { o^n^e^:^:^t^w^o^:^:^c^c^:^:^s^t inst; }");
   EXPECT_UNAVAILABLE(Header + "void fun() { N^S(c^c) inst; }");
+  // This used to crash. Ideally we would support this case, but for now we just
+  // test that we don't crash.
+  EXPECT_UNAVAILABLE(Header +
+ "template using foo = one::tt;");
 }
 
 TEST_F(AddUsingTest, Apply) {
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -206,9 +206,11 @@
 Name = D->getDecl()->getName();
   } else if (auto *T = Node->ASTNode.get()) {
 if (auto E = T->getAs()) {
-  QualifierToRemove = E.getQualifierLoc();
-  Name =
-  E.getType().getUnqualifiedType().getBaseTypeIdentifier()->getName();
+  if (auto *BaseTypeIdentifier =
+  E.getType().getUnqualifiedType().getBaseTypeIdentifier()) {
+Name = BaseTypeIdentifier->getName();
+QualifierToRemove = E.getQualifierLoc();
+  }
 }
   }
 
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D81169: [clangd] Improve hover on arguments to function call

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
adamcz added a reviewer: kadircet.
Herald added subscribers: cfe-commits, usaxena95, arphaman, jkorous, MaskRay, 
ilya-biryukov.
Herald added a project: clang.

In cases like:

  foo(a, ^b);

We now additionally show the name and type of the parameter to foo that
corresponds that "b" is passed as.

The name should help with understanding what it's used for and type can
be useful to find out if call to foo() can mutate variable "b" or not
(i.e. if it is pass by value, reference, const reference, etc).


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D81169

Files:
  clang-tools-extra/clangd/Hover.cpp
  clang-tools-extra/clangd/Hover.h
  clang-tools-extra/clangd/unittests/HoverTests.cpp

Index: clang-tools-extra/clangd/unittests/HoverTests.cpp
===
--- clang-tools-extra/clangd/unittests/HoverTests.cpp
+++ clang-tools-extra/clangd/unittests/HoverTests.cpp
@@ -696,6 +696,51 @@
  HI.Parameters->back().Name = "v";
  HI.AccessSpecifier = "public";
}},
+  {// Extra info for function call.
+   R"cpp(
+  void fun(int arg_a, int &arg_b) {};
+  void code() {
+int a = 1, b = 2;
+fun(a, [[^b]]);
+  }
+  )cpp",
+   [](HoverInfo &HI) {
+ HI.Name = "b";
+ HI.Kind = index::SymbolKind::Variable;
+ HI.NamespaceScope = "";
+ HI.Definition = "int b = 2";
+ HI.LocalScope = "code::";
+ HI.Value = "2";
+ HI.Type = "int";
+ HI.CalleeArgInfo.emplace();
+ HI.CalleeArgInfo->Name = "arg_b";
+ HI.CalleeArgInfo->Type = "int &";
+   }},
+  {// Extra info for method call.
+   R"cpp(
+  class C {
+   public:
+void fun(int arg_a = 3, int arg_b = 4) {}
+  };
+  void code() {
+int a = 1, b = 2;
+C c;
+c.fun([[^a]], b);
+  }
+  )cpp",
+   [](HoverInfo &HI) {
+ HI.Name = "a";
+ HI.Kind = index::SymbolKind::Variable;
+ HI.NamespaceScope = "";
+ HI.Definition = "int a = 1";
+ HI.LocalScope = "code::";
+ HI.Value = "1";
+ HI.Type = "int";
+ HI.CalleeArgInfo.emplace();
+ HI.CalleeArgInfo->Name = "arg_a";
+ HI.CalleeArgInfo->Type = "int";
+ HI.CalleeArgInfo->Default = "3";
+   }},
   };
   for (const auto &Case : Cases) {
 SCOPED_TRACE(Case.Code);
@@ -729,6 +774,7 @@
 EXPECT_EQ(H->Size, Expected.Size);
 EXPECT_EQ(H->Offset, Expected.Offset);
 EXPECT_EQ(H->AccessSpecifier, Expected.AccessSpecifier);
+EXPECT_EQ(H->CalleeArgInfo, Expected.CalleeArgInfo);
   }
 }
 
@@ -2022,6 +2068,29 @@
 
 // In namespace ns1
 private: union foo {})",
+  },
+  {
+  [](HoverInfo &HI) {
+HI.Kind = index::SymbolKind::Variable;
+HI.Name = "foo";
+HI.Definition = "int foo = 3";
+HI.LocalScope = "test::Bar::";
+HI.Value = "3";
+HI.Type = "int";
+HI.CalleeArgInfo.emplace();
+HI.CalleeArgInfo->Name = "arg_a";
+HI.CalleeArgInfo->Type = "int";
+HI.CalleeArgInfo->Default = "7";
+  },
+  R"(variable foo
+
+Type: int
+Value = 3
+
+// In test::Bar
+int foo = 3
+
+Passed as int arg_a = 7)",
   }};
 
   for (const auto &C : Cases) {
Index: clang-tools-extra/clangd/Hover.h
===
--- clang-tools-extra/clangd/Hover.h
+++ clang-tools-extra/clangd/Hover.h
@@ -77,6 +77,9 @@
   llvm::Optional Size;
   /// Contains the offset of fields within the enclosing class.
   llvm::Optional Offset;
+  // Set when symbol is inside function call. Contains information extracted
+  // from the callee definition about the argument this is passed as.
+  llvm::Optional CalleeArgInfo;
 
   /// Produce a user-readable information.
   markup::Document present() const;
Index: clang-tools-extra/clangd/Hover.cpp
===
--- clang-tools-extra/clangd/Hover.cpp
+++ clang-tools-extra/clangd/Hover.cpp
@@ -289,6 +289,26 @@
 : PVD->getDefaultArg();
 }
 
+void fillParam(const ParmVarDecl *PVD, HoverInfo::Param &Out,
+   const PrintingPolicy &Policy) {
+  if (!PVD->getType().isNull()) {
+Out.Type = printType(PVD->getType(), Policy);
+  } else {
+std::string Param;
+llvm::raw_string_ostream OS(Param);
+PVD->dump(OS);
+OS.flush();
+elog("Got param with null type: {0}", Param);
+  }
+  if (!PVD->getName().empty())
+Out.Name = PVD->getNameAsString();
+  if (const Expr *DefArg = getDefaultArg(PVD)) {
+Out.Default.emplace();
+llvm::raw_string_ostream OS(*Out.Default);
+DefArg->printPretty(OS, nullptr, Policy);
+  }
+}
+
 // Populates Type, ReturnType, and Parameters f

[PATCH] D81169: [clangd] Improve hover on arguments to function call

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 268780.
adamcz marked 11 inline comments as done.
adamcz added a comment.

addressed review comments, some UI questions remain open for now.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D81169/new/

https://reviews.llvm.org/D81169

Files:
  clang-tools-extra/clangd/Hover.cpp
  clang-tools-extra/clangd/Hover.h
  clang-tools-extra/clangd/unittests/HoverTests.cpp

Index: clang-tools-extra/clangd/unittests/HoverTests.cpp
===
--- clang-tools-extra/clangd/unittests/HoverTests.cpp
+++ clang-tools-extra/clangd/unittests/HoverTests.cpp
@@ -696,6 +696,51 @@
  HI.Parameters->back().Name = "v";
  HI.AccessSpecifier = "public";
}},
+  {// Extra info for function call.
+   R"cpp(
+  void fun(int arg_a, int &arg_b) {};
+  void code() {
+int a = 1, b = 2;
+fun(a, [[^b]]);
+  }
+  )cpp",
+   [](HoverInfo &HI) {
+ HI.Name = "b";
+ HI.Kind = index::SymbolKind::Variable;
+ HI.NamespaceScope = "";
+ HI.Definition = "int b = 2";
+ HI.LocalScope = "code::";
+ HI.Value = "2";
+ HI.Type = "int";
+ HI.CalleeArgInfo.emplace();
+ HI.CalleeArgInfo->Name = "arg_b";
+ HI.CalleeArgInfo->Type = "int &";
+   }},
+  {// Extra info for method call.
+   R"cpp(
+  class C {
+   public:
+void fun(int arg_a = 3, int arg_b = 4) {}
+  };
+  void code() {
+int a = 1, b = 2;
+C c;
+c.fun([[^a]], b);
+  }
+  )cpp",
+   [](HoverInfo &HI) {
+ HI.Name = "a";
+ HI.Kind = index::SymbolKind::Variable;
+ HI.NamespaceScope = "";
+ HI.Definition = "int a = 1";
+ HI.LocalScope = "code::";
+ HI.Value = "1";
+ HI.Type = "int";
+ HI.CalleeArgInfo.emplace();
+ HI.CalleeArgInfo->Name = "arg_a";
+ HI.CalleeArgInfo->Type = "int";
+ HI.CalleeArgInfo->Default = "3";
+   }},
   };
   for (const auto &Case : Cases) {
 SCOPED_TRACE(Case.Code);
@@ -729,6 +774,7 @@
 EXPECT_EQ(H->Size, Expected.Size);
 EXPECT_EQ(H->Offset, Expected.Offset);
 EXPECT_EQ(H->AccessSpecifier, Expected.AccessSpecifier);
+EXPECT_EQ(H->CalleeArgInfo, Expected.CalleeArgInfo);
   }
 }
 
@@ -2022,6 +2068,29 @@
 
 // In namespace ns1
 private: union foo {})",
+  },
+  {
+  [](HoverInfo &HI) {
+HI.Kind = index::SymbolKind::Variable;
+HI.Name = "foo";
+HI.Definition = "int foo = 3";
+HI.LocalScope = "test::Bar::";
+HI.Value = "3";
+HI.Type = "int";
+HI.CalleeArgInfo.emplace();
+HI.CalleeArgInfo->Name = "arg_a";
+HI.CalleeArgInfo->Type = "int";
+HI.CalleeArgInfo->Default = "7";
+  },
+  R"(variable foo
+
+Type: int
+Value = 3
+
+Passed as int arg_a = 7
+
+// In test::Bar
+int foo = 3)",
   }};
 
   for (const auto &C : Cases) {
Index: clang-tools-extra/clangd/Hover.h
===
--- clang-tools-extra/clangd/Hover.h
+++ clang-tools-extra/clangd/Hover.h
@@ -77,6 +77,9 @@
   llvm::Optional Size;
   /// Contains the offset of fields within the enclosing class.
   llvm::Optional Offset;
+  // Set when symbol is inside function call. Contains information extracted
+  // from the callee definition about the argument this is passed as.
+  llvm::Optional CalleeArgInfo;
 
   /// Produce a user-readable information.
   markup::Document present() const;
Index: clang-tools-extra/clangd/Hover.cpp
===
--- clang-tools-extra/clangd/Hover.cpp
+++ clang-tools-extra/clangd/Hover.cpp
@@ -289,30 +289,35 @@
 : PVD->getDefaultArg();
 }
 
+HoverInfo::Param toHoverInfoParam(const ParmVarDecl *PVD,
+  const PrintingPolicy &Policy) {
+  HoverInfo::Param Out;
+  if (!PVD->getType().isNull()) {
+Out.Type = printType(PVD->getType(), Policy);
+  } else {
+std::string Param;
+llvm::raw_string_ostream OS(Param);
+PVD->dump(OS);
+OS.flush();
+elog("Got param with null type: {0}", Param);
+  }
+  if (!PVD->getName().empty())
+Out.Name = PVD->getNameAsString();
+  if (const Expr *DefArg = getDefaultArg(PVD)) {
+Out.Default.emplace();
+llvm::raw_string_ostream OS(*Out.Default);
+DefArg->printPretty(OS, nullptr, Policy);
+  }
+  return Out;
+}
+
 // Populates Type, ReturnType, and Parameters for function-like decls.
 void fillFunctionTypeAndParams(HoverInfo &HI, const Decl *D,
const FunctionDecl *FD,
const PrintingPolicy &Policy) {
   HI.Parameters.emplace();
  

[PATCH] D81169: [clangd] Improve hover on arguments to function call

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/Hover.cpp:525
 
+  if (const auto *Var = dyn_cast(D)) {
+// Check if we are inside CallExpr

kadircet wrote:
> umm, is this a fixme or a leftover? I suppose it is from a previous version 
> of the patch.
Yep, previous version, sorry about that.



Comment at: clang-tools-extra/clangd/Hover.cpp:713
+  }
+  if (I >= CE->getNumArgs())
+return;

kadircet wrote:
> i don't think this is necessary, we are baling out if `FD->getNumParams() <= 
> I` anyways.
It's possible for FD->getNumParams() to be higher than CE->getNumArgs(), for 
example due to default arguments. If we couldn't find selected node inside 
CallExpr, we shouldn't be trying to match it to some "random" argument in 
FunctionDecl.

It's a bit irrelevant now, after addressing other review comments ;-)



Comment at: clang-tools-extra/clangd/Hover.cpp:716
+  // Extract matching argument from function declaration.
+  if (const FunctionDecl *FD = CE->getDirectCallee()) {
+if (FD->isOverloadedOperator() || FD->isVariadic() ||

kadircet wrote:
> nit: put this before arg index finding logic and do an early exit:
> 
> ```
> if(!CE) return;
> auto *FD = CE->getDirectCallee();
> if(!FD || FD->isOverloaded || variadic..) return;
> ```
> 
> after doing this you can also change the arg index finding logic to:
> ```
> auto *SelectedArg = N->outerImplicit().ASTNode.get();
> if(!SelectedParam) return; // you can also move this above other exit 
> conditions.
> for (unsigned I = 0; I < min(CE->getNumArgs, FD->getNumParams())...) { // I 
> suppose these two can be different in presence of default params ?
>   if (CE->getArg(I) == SelectedArg) {
>   // do magic;
>   break;
>   }
> }
> ``
Yea, that's nicer, thanks.

(I opted for "continue" rather than "break" to reduce indentation).



Comment at: clang-tools-extra/clangd/Hover.cpp:717
+  if (const FunctionDecl *FD = CE->getDirectCallee()) {
+if (FD->isOverloadedOperator() || FD->isVariadic() ||
+FD->getNumParams() <= I)

kadircet wrote:
> i can see the reason for variadics, but why bail out on overloaded operators? 
> also can we have some tests for these two?
I could totally see us supporting variadic, I'm just not sure how useful that 
will be, so I didn't do it right away.

As for the operators, I don't think we should trigger on those. There are two 
cases:
- things that do not look like a function call: operator+, operator<<, etc. 
When you hover over a variable it's not immediately obvious what the "passed 
to" would be referring to. Something like:
foo(a+b);
if I see "a passed as const int&" I might think this is about passing it to 
foo() (at least of first glance).
At the same time, the value of this is very low for operators. You know what 
their signature is already.

-operator() - this is much more like a function call. Maybe it would make sense 
to support it. It gets tricky with matching CallExpr args to FD params though. 
I'll leave a todo for now and worry about this in a separate change, if don't 
mind.jj



Comment at: clang-tools-extra/clangd/Hover.cpp:897
+llvm::raw_string_ostream OS(Buffer);
+OS << "Passed as " << *CalleeArgInfo;
+Output.addParagraph().appendText(OS.str());

kadircet wrote:
> i bet there will always be some people getting confused no matter what we say 
> in here (especially since we are not just printing the type, and I totally 
> find printing param name useful especially in case of literals). But to align 
> with other types of information we provide maybe say:
> 
> `Substitutes: ` ?
I am very much open to suggestions for phrasing, I'm not too happy about 
"Passed as". 

However, I am not a fan of Substitutes. If I saw that, I would have no idea 
what it's referring to. Since there is no visible connection between this hover 
card and the function call, without knowing what I'm looking at I would not 
make this connection. 

Might be just me. I am really as far from being a UI expert as possible. I'll 
ask around to see if people would get the Substitute: and get back to you on 
this one.



Comment at: clang-tools-extra/clangd/unittests/HoverTests.cpp:728
+C c;
+c.fun([[^a]], b);
+  }

kadircet wrote:
> can you also add a test for a literal ?
Currently there is no hover at all for literals. The reason was that it 
wouldn't be useful. This information, however, might be, especially in cases 
like:
printThis(that, true, false, true);
knowing what these refer to could be useful.

Do you think this is good enough reason to enable hover on literals, with just 
this information? I'm thinking yes, but I'm not sure.



Comment at: clang-tools-extra/clangd/unittests/HoverTests.cpp:2093
+
+Passed as int arg_a = 7)",
   }};

[PATCH] D81958: [clangd] Add library to semantically strip flags by name.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/CompileCommands.cpp:256
+  case Option::RemainingArgsClass:
+return {1, 0};
+  case Option::RemainingArgsJoinedClass:

nit: could you replace 1 with some constant value with descriptive name?  
It's not immediately clear if this is some significant value (e.g. some flag 
number or something) or just, as it's the case, just a very large number.



Comment at: clang-tools-extra/clangd/CompileCommands.cpp:301
+
+// Collect sets of aliases, so we can treet -foo and -foo= as synonyms.
+// Conceptually a double-linked list: PrevAlias[I] -> I -> NextAlias[I].

typo: treet



Comment at: clang-tools-extra/clangd/CompileCommands.cpp:401
+
+  // Examine args list to determine if we're in GCC, CL-compatible, or cc1 
mode.
+  DriverMode MainMode = DM_GCC;

nit: this function is already quite long and this part seems like a re-usable 
and self-contained piece. Could we extra this into some GetDriverMode() helper?



Comment at: clang-tools-extra/clangd/CompileCommands.cpp:431
+continue; // current arg doesn't match the prefix string
+  bool PrefixMatch = Arg.size() > R.Text.size();
+  unsigned ArgCount = PrefixMatch ? R.PrefixArgs : R.ExactArgs;

Correct me if I'm wrong, but this is relying on the fact that Rules are sorted, 
right? Or, to be more specific, on the fact that -foo comes before -foobar.

Consider two rules in config file, one to remove -include, another to remove 
-include-pch, in that order. -include will do a prefix match on Arg == 
-include-pch and attempt to remove exactly one arg (-include is 
JoinedOrSeparate, which has 1 for PrefixArgs), when in reality that was 
"--include-pch foo.pch", an exact match on a different option.

So a config like:
Remove: [-include, -include-pch]
and command line:
[-include-pch, foo.pch]
should be [], but ends up being [foo.pch]

It looks like Options.inc is sorted in the correct way (include-pch will always 
be before -include). I don't know if that's guaranteed, but it looks to be the 
case. However, since you are adding these options to Rules on each strip() 
call, you end up depending on the order of strip() calls. That seems like a bug.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D81958/new/

https://reviews.llvm.org/D81958



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D81169: [clangd] Improve hover on arguments to function call

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 275073.
adamcz added a comment.

Final review comment + rebase on top of HEAD


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D81169/new/

https://reviews.llvm.org/D81169

Files:
  clang-tools-extra/clangd/Hover.cpp
  clang-tools-extra/clangd/Hover.h
  clang-tools-extra/clangd/unittests/HoverTests.cpp

Index: clang-tools-extra/clangd/unittests/HoverTests.cpp
===
--- clang-tools-extra/clangd/unittests/HoverTests.cpp
+++ clang-tools-extra/clangd/unittests/HoverTests.cpp
@@ -26,6 +26,8 @@
 namespace clangd {
 namespace {
 
+using PassMode = HoverInfo::PassType::PassMode;
+
 TEST(Hover, Structured) {
   struct {
 const char *const Code;
@@ -719,6 +721,57 @@
  // Bindings are in theory public members of an anonymous struct.
  HI.AccessSpecifier = "public";
}},
+  {// Extra info for function call.
+   R"cpp(
+  void fun(int arg_a, int &arg_b) {};
+  void code() {
+int a = 1, b = 2;
+fun(a, [[^b]]);
+  }
+  )cpp",
+   [](HoverInfo &HI) {
+ HI.Name = "b";
+ HI.Kind = index::SymbolKind::Variable;
+ HI.NamespaceScope = "";
+ HI.Definition = "int b = 2";
+ HI.LocalScope = "code::";
+ HI.Value = "2";
+ HI.Type = "int";
+ HI.CalleeArgInfo.emplace();
+ HI.CalleeArgInfo->Name = "arg_b";
+ HI.CalleeArgInfo->Type = "int &";
+ HI.CallPassType.emplace();
+ HI.CallPassType->PassBy = PassMode::Ref;
+ HI.CallPassType->Converted = false;
+   }},
+  {// Extra info for method call.
+   R"cpp(
+  class C {
+   public:
+void fun(int arg_a = 3, int arg_b = 4) {}
+  };
+  void code() {
+int a = 1, b = 2;
+C c;
+c.fun([[^a]], b);
+  }
+  )cpp",
+   [](HoverInfo &HI) {
+ HI.Name = "a";
+ HI.Kind = index::SymbolKind::Variable;
+ HI.NamespaceScope = "";
+ HI.Definition = "int a = 1";
+ HI.LocalScope = "code::";
+ HI.Value = "1";
+ HI.Type = "int";
+ HI.CalleeArgInfo.emplace();
+ HI.CalleeArgInfo->Name = "arg_a";
+ HI.CalleeArgInfo->Type = "int";
+ HI.CalleeArgInfo->Default = "3";
+ HI.CallPassType.emplace();
+ HI.CallPassType->PassBy = PassMode::Value;
+ HI.CallPassType->Converted = false;
+   }},
   };
   for (const auto &Case : Cases) {
 SCOPED_TRACE(Case.Code);
@@ -752,6 +805,85 @@
 EXPECT_EQ(H->Size, Expected.Size);
 EXPECT_EQ(H->Offset, Expected.Offset);
 EXPECT_EQ(H->AccessSpecifier, Expected.AccessSpecifier);
+EXPECT_EQ(H->CalleeArgInfo, Expected.CalleeArgInfo);
+EXPECT_EQ(H->CallPassType, Expected.CallPassType);
+  }
+}
+
+TEST(Hover, CallPassType) {
+  const llvm::StringRef CodePrefix = R"cpp(
+class Base {};
+class Derived : public Base {};
+class CustomClass {
+ public:
+  CustomClass() {}
+  CustomClass(const Base &x) {}
+  CustomClass(int &x) {}
+  CustomClass(float x) {}
+};
+
+void int_by_ref(int &x) {}
+void int_by_const_ref(const int &x) {}
+void int_by_value(int x) {}
+void base_by_ref(Base &x) {}
+void base_by_const_ref(const Base &x) {}
+void base_by_value(Base x) {}
+void float_by_value(float x) {}
+void custom_by_value(CustomClass x) {}
+
+void fun() {
+  int int_x;
+  int &int_ref = int_x;
+  const int &int_const_ref = int_x;
+  Base base;
+  const Base &base_const_ref = base;
+  Derived derived;
+  float float_x;
+)cpp";
+  const llvm::StringRef CodeSuffix = "}";
+
+  struct {
+const char *const Code;
+HoverInfo::PassType::PassMode PassBy;
+bool Converted;
+  } Tests[] = {
+  // Integer tests
+  {"int_by_value([[^int_x]]);", PassMode::Value, false},
+  {"int_by_ref([[^int_x]]);", PassMode::Ref, false},
+  {"int_by_const_ref([[^int_x]]);", PassMode::ConstRef, false},
+  {"int_by_value([[^int_ref]]);", PassMode::Value, false},
+  {"int_by_const_ref([[^int_ref]]);", PassMode::ConstRef, false},
+  {"int_by_const_ref([[^int_ref]]);", PassMode::ConstRef, false},
+  {"int_by_const_ref([[^int_const_ref]]);", PassMode::ConstRef, false},
+  // Custom class tests
+  {"base_by_ref([[^base]]);", PassMode::Ref, false},
+  {"base_by_const_ref([[^base]]);", PassMode::ConstRef, false},
+  {"base_by_const_ref([[^base_const_ref]]);", PassMode::ConstRef, false},
+  {"base_by_value([[^base]]);", PassMode::Value, false},
+  {"base_by_value([[^base_const_ref]]);", PassMode::Value, false},
+  {"base_by_ref([[^derived]]);", PassMode::Ref, false},
+  {"base_by_const_ref([[^derived]]);", PassMode::ConstRef, false},
+  {"base_by_value([[^derived]]);", PassMode::Value, false},
+  // Converted tests
+  {"float_by_value([[^int_x]]);", PassMode::Value, true},
+  {"floa

[PATCH] D81169: [clangd] Improve hover on arguments to function call

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 275104.
adamcz marked an inline comment as done.
adamcz added a comment.

removed some more {}


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D81169/new/

https://reviews.llvm.org/D81169

Files:
  clang-tools-extra/clangd/Hover.cpp
  clang-tools-extra/clangd/Hover.h
  clang-tools-extra/clangd/unittests/HoverTests.cpp

Index: clang-tools-extra/clangd/unittests/HoverTests.cpp
===
--- clang-tools-extra/clangd/unittests/HoverTests.cpp
+++ clang-tools-extra/clangd/unittests/HoverTests.cpp
@@ -26,6 +26,8 @@
 namespace clangd {
 namespace {
 
+using PassMode = HoverInfo::PassType::PassMode;
+
 TEST(Hover, Structured) {
   struct {
 const char *const Code;
@@ -719,6 +721,57 @@
  // Bindings are in theory public members of an anonymous struct.
  HI.AccessSpecifier = "public";
}},
+  {// Extra info for function call.
+   R"cpp(
+  void fun(int arg_a, int &arg_b) {};
+  void code() {
+int a = 1, b = 2;
+fun(a, [[^b]]);
+  }
+  )cpp",
+   [](HoverInfo &HI) {
+ HI.Name = "b";
+ HI.Kind = index::SymbolKind::Variable;
+ HI.NamespaceScope = "";
+ HI.Definition = "int b = 2";
+ HI.LocalScope = "code::";
+ HI.Value = "2";
+ HI.Type = "int";
+ HI.CalleeArgInfo.emplace();
+ HI.CalleeArgInfo->Name = "arg_b";
+ HI.CalleeArgInfo->Type = "int &";
+ HI.CallPassType.emplace();
+ HI.CallPassType->PassBy = PassMode::Ref;
+ HI.CallPassType->Converted = false;
+   }},
+  {// Extra info for method call.
+   R"cpp(
+  class C {
+   public:
+void fun(int arg_a = 3, int arg_b = 4) {}
+  };
+  void code() {
+int a = 1, b = 2;
+C c;
+c.fun([[^a]], b);
+  }
+  )cpp",
+   [](HoverInfo &HI) {
+ HI.Name = "a";
+ HI.Kind = index::SymbolKind::Variable;
+ HI.NamespaceScope = "";
+ HI.Definition = "int a = 1";
+ HI.LocalScope = "code::";
+ HI.Value = "1";
+ HI.Type = "int";
+ HI.CalleeArgInfo.emplace();
+ HI.CalleeArgInfo->Name = "arg_a";
+ HI.CalleeArgInfo->Type = "int";
+ HI.CalleeArgInfo->Default = "3";
+ HI.CallPassType.emplace();
+ HI.CallPassType->PassBy = PassMode::Value;
+ HI.CallPassType->Converted = false;
+   }},
   };
   for (const auto &Case : Cases) {
 SCOPED_TRACE(Case.Code);
@@ -752,6 +805,85 @@
 EXPECT_EQ(H->Size, Expected.Size);
 EXPECT_EQ(H->Offset, Expected.Offset);
 EXPECT_EQ(H->AccessSpecifier, Expected.AccessSpecifier);
+EXPECT_EQ(H->CalleeArgInfo, Expected.CalleeArgInfo);
+EXPECT_EQ(H->CallPassType, Expected.CallPassType);
+  }
+}
+
+TEST(Hover, CallPassType) {
+  const llvm::StringRef CodePrefix = R"cpp(
+class Base {};
+class Derived : public Base {};
+class CustomClass {
+ public:
+  CustomClass() {}
+  CustomClass(const Base &x) {}
+  CustomClass(int &x) {}
+  CustomClass(float x) {}
+};
+
+void int_by_ref(int &x) {}
+void int_by_const_ref(const int &x) {}
+void int_by_value(int x) {}
+void base_by_ref(Base &x) {}
+void base_by_const_ref(const Base &x) {}
+void base_by_value(Base x) {}
+void float_by_value(float x) {}
+void custom_by_value(CustomClass x) {}
+
+void fun() {
+  int int_x;
+  int &int_ref = int_x;
+  const int &int_const_ref = int_x;
+  Base base;
+  const Base &base_const_ref = base;
+  Derived derived;
+  float float_x;
+)cpp";
+  const llvm::StringRef CodeSuffix = "}";
+
+  struct {
+const char *const Code;
+HoverInfo::PassType::PassMode PassBy;
+bool Converted;
+  } Tests[] = {
+  // Integer tests
+  {"int_by_value([[^int_x]]);", PassMode::Value, false},
+  {"int_by_ref([[^int_x]]);", PassMode::Ref, false},
+  {"int_by_const_ref([[^int_x]]);", PassMode::ConstRef, false},
+  {"int_by_value([[^int_ref]]);", PassMode::Value, false},
+  {"int_by_const_ref([[^int_ref]]);", PassMode::ConstRef, false},
+  {"int_by_const_ref([[^int_ref]]);", PassMode::ConstRef, false},
+  {"int_by_const_ref([[^int_const_ref]]);", PassMode::ConstRef, false},
+  // Custom class tests
+  {"base_by_ref([[^base]]);", PassMode::Ref, false},
+  {"base_by_const_ref([[^base]]);", PassMode::ConstRef, false},
+  {"base_by_const_ref([[^base_const_ref]]);", PassMode::ConstRef, false},
+  {"base_by_value([[^base]]);", PassMode::Value, false},
+  {"base_by_value([[^base_const_ref]]);", PassMode::Value, false},
+  {"base_by_ref([[^derived]]);", PassMode::Ref, false},
+  {"base_by_const_ref([[^derived]]);", PassMode::ConstRef, false},
+  {"base_by_value([[^derived]]);", PassMode::Value, false},
+  // Converted tests
+  {"float_by_value([[^int_x]]);", PassMode::Value, tru

[PATCH] D81169: [clangd] Improve hover on arguments to function call

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 273742.
adamcz marked 2 inline comments as done.
adamcz added a comment.

Formatting changed to spell out [const] reference and hide type unless 
conversion happens.
Defining what the "correct" behavior is on various cases is very difficult.
This is my best shot at incorporating previous requests and comments.

Also addressed more review comments.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D81169/new/

https://reviews.llvm.org/D81169

Files:
  clang-tools-extra/clangd/Hover.cpp
  clang-tools-extra/clangd/Hover.h
  clang-tools-extra/clangd/unittests/HoverTests.cpp

Index: clang-tools-extra/clangd/unittests/HoverTests.cpp
===
--- clang-tools-extra/clangd/unittests/HoverTests.cpp
+++ clang-tools-extra/clangd/unittests/HoverTests.cpp
@@ -708,6 +708,59 @@
  HI.Definition = "X x";
  HI.Type = "struct X";
}},
+  {// Extra info for function call.
+   R"cpp(
+  void fun(int arg_a, int &arg_b) {};
+  void code() {
+int a = 1, b = 2;
+fun(a, [[^b]]);
+  }
+  )cpp",
+   [](HoverInfo &HI) {
+ HI.Name = "b";
+ HI.Kind = index::SymbolKind::Variable;
+ HI.NamespaceScope = "";
+ HI.Definition = "int b = 2";
+ HI.LocalScope = "code::";
+ HI.Value = "2";
+ HI.Type = "int";
+ HI.CalleeArgInfo.emplace();
+ HI.CalleeArgInfo->Name = "arg_b";
+ HI.CalleeArgInfo->Type = "int &";
+ HI.CallPassType.emplace();
+ HI.CallPassType->Reference = true;
+ HI.CallPassType->Const = false;
+ HI.CallPassType->Converted = false;
+   }},
+  {// Extra info for method call.
+   R"cpp(
+  class C {
+   public:
+void fun(int arg_a = 3, int arg_b = 4) {}
+  };
+  void code() {
+int a = 1, b = 2;
+C c;
+c.fun([[^a]], b);
+  }
+  )cpp",
+   [](HoverInfo &HI) {
+ HI.Name = "a";
+ HI.Kind = index::SymbolKind::Variable;
+ HI.NamespaceScope = "";
+ HI.Definition = "int a = 1";
+ HI.LocalScope = "code::";
+ HI.Value = "1";
+ HI.Type = "int";
+ HI.CalleeArgInfo.emplace();
+ HI.CalleeArgInfo->Name = "arg_a";
+ HI.CalleeArgInfo->Type = "int";
+ HI.CalleeArgInfo->Default = "3";
+ HI.CallPassType.emplace();
+ HI.CallPassType->Reference = false;
+ HI.CallPassType->Const = false;
+ HI.CallPassType->Converted = false;
+   }},
   };
   for (const auto &Case : Cases) {
 SCOPED_TRACE(Case.Code);
@@ -741,6 +794,87 @@
 EXPECT_EQ(H->Size, Expected.Size);
 EXPECT_EQ(H->Offset, Expected.Offset);
 EXPECT_EQ(H->AccessSpecifier, Expected.AccessSpecifier);
+EXPECT_EQ(H->CalleeArgInfo, Expected.CalleeArgInfo);
+EXPECT_EQ(H->CallPassType, Expected.CallPassType);
+  }
+}
+
+TEST(Hover, CallPassType) {
+  const llvm::StringRef CodePrefix = R"cpp(
+class Base {};
+class Derived : public Base {};
+class CustomClass {
+ public:
+  CustomClass() {}
+  CustomClass(const Base &x) {}
+  CustomClass(int &x) {}
+  CustomClass(float x) {}
+};
+
+void int_by_ref(int &x) {}
+void int_by_const_ref(const int &x) {}
+void int_by_value(int x) {}
+void base_by_ref(Base &x) {}
+void base_by_const_ref(const Base &x) {}
+void base_by_value(Base x) {}
+void float_by_value(float x) {}
+void custom_by_value(CustomClass x) {}
+
+void fun() {
+  int int_x;
+  int &int_ref = int_x;
+  const int &int_const_ref = int_x;
+  Base base;
+  const Base &base_const_ref = base;
+  Derived derived;
+  float float_x;
+)cpp";
+  const llvm::StringRef CodeSuffix = "}";
+
+  struct {
+const char *const Code;
+bool Reference;
+bool Const;
+bool Converted;
+  } Tests[] = {
+  // Integer tests
+  {"int_by_value([[^int_x]]);", false, false, false},
+  {"int_by_ref([[^int_x]]);", true, false, false},
+  {"int_by_const_ref([[^int_x]]);", true, true, false},
+  {"int_by_value([[^int_ref]]);", false, false, false},
+  {"int_by_const_ref([[^int_ref]]);", true, true, false},
+  {"int_by_const_ref([[^int_ref]]);", true, true, false},
+  {"int_by_const_ref([[^int_const_ref]]);", true, true, false},
+  // Custom class tests
+  {"base_by_ref([[^base]]);", true, false, false},
+  {"base_by_const_ref([[^base]]);", true, true, false},
+  {"base_by_const_ref([[^base_const_ref]]);", true, true, false},
+  {"base_by_value([[^base]]);", false, false, false},
+  {"base_by_value([[^base_const_ref]]);", false, false, false},
+  {"base_by_ref([[^derived]]);", true, false, false},
+  {"base_by_const_ref([[^derived]]);", true, true, false},
+  {"base_by_value([[^derived]]);", false, false, false},
+  // Converted tests
+  {"float_by_value([[^int_x]]);",

[PATCH] D81169: [clangd] Improve hover on arguments to function call

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 274147.
adamcz marked 8 inline comments as done.
adamcz added a comment.

Addressed review comments, also added some "const" here and there.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D81169/new/

https://reviews.llvm.org/D81169

Files:
  clang-tools-extra/clangd/Hover.cpp
  clang-tools-extra/clangd/Hover.h
  clang-tools-extra/clangd/unittests/HoverTests.cpp

Index: clang-tools-extra/clangd/unittests/HoverTests.cpp
===
--- clang-tools-extra/clangd/unittests/HoverTests.cpp
+++ clang-tools-extra/clangd/unittests/HoverTests.cpp
@@ -26,6 +26,8 @@
 namespace clangd {
 namespace {
 
+using PassMode = HoverInfo::PassType::PassMode;
+
 TEST(Hover, Structured) {
   struct {
 const char *const Code;
@@ -708,6 +710,57 @@
  HI.Definition = "X x";
  HI.Type = "struct X";
}},
+  {// Extra info for function call.
+   R"cpp(
+  void fun(int arg_a, int &arg_b) {};
+  void code() {
+int a = 1, b = 2;
+fun(a, [[^b]]);
+  }
+  )cpp",
+   [](HoverInfo &HI) {
+ HI.Name = "b";
+ HI.Kind = index::SymbolKind::Variable;
+ HI.NamespaceScope = "";
+ HI.Definition = "int b = 2";
+ HI.LocalScope = "code::";
+ HI.Value = "2";
+ HI.Type = "int";
+ HI.CalleeArgInfo.emplace();
+ HI.CalleeArgInfo->Name = "arg_b";
+ HI.CalleeArgInfo->Type = "int &";
+ HI.CallPassType.emplace();
+ HI.CallPassType->PassBy = PassMode::Ref;
+ HI.CallPassType->Converted = false;
+   }},
+  {// Extra info for method call.
+   R"cpp(
+  class C {
+   public:
+void fun(int arg_a = 3, int arg_b = 4) {}
+  };
+  void code() {
+int a = 1, b = 2;
+C c;
+c.fun([[^a]], b);
+  }
+  )cpp",
+   [](HoverInfo &HI) {
+ HI.Name = "a";
+ HI.Kind = index::SymbolKind::Variable;
+ HI.NamespaceScope = "";
+ HI.Definition = "int a = 1";
+ HI.LocalScope = "code::";
+ HI.Value = "1";
+ HI.Type = "int";
+ HI.CalleeArgInfo.emplace();
+ HI.CalleeArgInfo->Name = "arg_a";
+ HI.CalleeArgInfo->Type = "int";
+ HI.CalleeArgInfo->Default = "3";
+ HI.CallPassType.emplace();
+ HI.CallPassType->PassBy = PassMode::Value;
+ HI.CallPassType->Converted = false;
+   }},
   };
   for (const auto &Case : Cases) {
 SCOPED_TRACE(Case.Code);
@@ -741,6 +794,85 @@
 EXPECT_EQ(H->Size, Expected.Size);
 EXPECT_EQ(H->Offset, Expected.Offset);
 EXPECT_EQ(H->AccessSpecifier, Expected.AccessSpecifier);
+EXPECT_EQ(H->CalleeArgInfo, Expected.CalleeArgInfo);
+EXPECT_EQ(H->CallPassType, Expected.CallPassType);
+  }
+}
+
+TEST(Hover, CallPassType) {
+  const llvm::StringRef CodePrefix = R"cpp(
+class Base {};
+class Derived : public Base {};
+class CustomClass {
+ public:
+  CustomClass() {}
+  CustomClass(const Base &x) {}
+  CustomClass(int &x) {}
+  CustomClass(float x) {}
+};
+
+void int_by_ref(int &x) {}
+void int_by_const_ref(const int &x) {}
+void int_by_value(int x) {}
+void base_by_ref(Base &x) {}
+void base_by_const_ref(const Base &x) {}
+void base_by_value(Base x) {}
+void float_by_value(float x) {}
+void custom_by_value(CustomClass x) {}
+
+void fun() {
+  int int_x;
+  int &int_ref = int_x;
+  const int &int_const_ref = int_x;
+  Base base;
+  const Base &base_const_ref = base;
+  Derived derived;
+  float float_x;
+)cpp";
+  const llvm::StringRef CodeSuffix = "}";
+
+  struct {
+const char *const Code;
+HoverInfo::PassType::PassMode PassBy;
+bool Converted;
+  } Tests[] = {
+  // Integer tests
+  {"int_by_value([[^int_x]]);", PassMode::Value, false},
+  {"int_by_ref([[^int_x]]);", PassMode::Ref, false},
+  {"int_by_const_ref([[^int_x]]);", PassMode::ConstRef, false},
+  {"int_by_value([[^int_ref]]);", PassMode::Value, false},
+  {"int_by_const_ref([[^int_ref]]);", PassMode::ConstRef, false},
+  {"int_by_const_ref([[^int_ref]]);", PassMode::ConstRef, false},
+  {"int_by_const_ref([[^int_const_ref]]);", PassMode::ConstRef, false},
+  // Custom class tests
+  {"base_by_ref([[^base]]);", PassMode::Ref, false},
+  {"base_by_const_ref([[^base]]);", PassMode::ConstRef, false},
+  {"base_by_const_ref([[^base_const_ref]]);", PassMode::ConstRef, false},
+  {"base_by_value([[^base]]);", PassMode::Value, false},
+  {"base_by_value([[^base_const_ref]]);", PassMode::Value, false},
+  {"base_by_ref([[^derived]]);", PassMode::Ref, false},
+  {"base_by_const_ref([[^derived]]);", PassMode::ConstRef, false},
+  {"base_by_value([[^derived]]);", PassMode::Value, false},
+  // Converted tests
+  {"float_by_value([[^int_x]]);", PassMode::Value, true},
+

[PATCH] D81169: [clangd] Improve hover on arguments to function call

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/Hover.cpp:721
+// Extract matching argument from function declaration.
+if (const ParmVarDecl *PVD = FD->getParamDecl(I)) {
+  HI.CalleeArgInfo.emplace(toHoverInfoParam(PVD, Policy));

kadircet wrote:
> nit: braces are usually omitted when "then" part has a single expression. 
> (applies to multiple other places in the patch)
Ah, yes, that. I keep doing that because I am personally strongly opposed to 
this convention. I respect the rules, of course, and avoid {} in those cases, 
but I think my personal feelings about it make it hard for me to avoid mistakes 
like this ;-). Sorry.

Fixed.



Comment at: clang-tools-extra/clangd/Hover.cpp:735
+if (E->getType().isConstQualified()) {
+  PassType.Const = true;
+}

kadircet wrote:
> isn't this same as `PVD->getType().isConstQualified()`? maybe set it in there?
> 
> I think it is more explicit that way, because in theory the `Expr` above is 
> always the `DeclRefExpr` referencing the current parameter, which is quite 
> subtle to figure out while looking at this. Or am I missing something?
It's different. Consider:
class C {
 public:
  C(int &x) {}
};
void foo(const C &c) {}

int y;
foo(y);

PVD is const-qualified (and it's type is const C&), but y is int and passed by 
non-const reference to the C c-tor, which might mutate y, even though the 
resulting instance of C will be const.



Comment at: clang-tools-extra/clangd/Hover.cpp:760
+break;
+  default:
+PassType.Reference = false;

kadircet wrote:
> *sigh* wish we didn't have more than 60 types of implicit casts in c++.
> 
> I can see how it is hard to list everything in here and even if you did it 
> would still be really hard to reason about them.
> I can't help myself but wonder though, was there a reasoning when choosing 
> what to leave out? Because that would help me a lot, and possibly future 
> readers. (I guess I am also fine with "the list looked comprehensive enough", 
> as we can always iterate over)
> 
> I am a little bit hesitant about the fail policy though(both in here and in 
> other branches), as I can't prove to myself that being passed as a reference 
> would be false in all the other cases for example. And showing incorrect 
> information in here could really end up confusing the user. Unless there's a 
> clear reasoning behind this failure mode, I would suggest dropping 
> `CalleeArgInfo` completely to be on the safe side. That way we can improve 
> the handling for important cases later on, without surfacing possibly wrong 
> results. WDYT?
I basically went through the list of all CK_ values and looked for ones that 
preserve the ref bit. There aren't many. Basically any type conversion other 
than casting to base must drop the ref bit (because it's a different type then) 
and then there's few that are explicitly about copying data. So in the end the 
default is to drop the reference bit and there's only a handful that preserve 
it (DerivedToBase, Decay and NoOp, basically).

I do agree with you that this is fragile, likely to be slightly wrong and too 
complicated. I can't think of a better solution though.

I prefer to not drop the whole CalleeArgInfo / hover thing in those cases 
because, even if you ignore the whole mutable/non-mutable bit of information, I 
still find the name of the argument it's passed as to be very valuable).

I like the idea of now showing the ref/const-ref/value bit in cases where we 
fear it might not be correct, but I do not know how to display this to the user 
without confusing them (i.e. if there's no mention of ref/const-ref, does that 
mean it's by-value or that we couldn't tell?). Note that I intentionally 
avoided showing the words "by value" to the user, since it will confuse 
everyone when used on pointers.

This is essentially why I was hesitant to extract the reference bit and why my 
initial change had only the type, to let user figure it out themselves. It's 
going to be quite hard to get this right.




Comment at: clang-tools-extra/clangd/Hover.h:86
+// implicit constructor call.
+bool Reference = true;
+// True if that reference is false. Never set for non-reference pass.

kadircet wrote:
> why is this true by default ?
Passing by reference is the default. It takes extra nodes in the AST to turn it 
to pass by value. Basically the defaults are what we see on the very inner node 
and then we adjust this as we go up the tree. Does that make sense?



Comment at: clang-tools-extra/clangd/Hover.h:87
+bool Reference = true;
+// True if that reference is false. Never set for non-reference pass.
+bool Const = false;

kadircet wrote:
> s/false/const/ ?
> 
> what would you say to having an enum instead ?
> ```
> enum PassBy {
>   Value,
>Ref,
>ConstRef

[PATCH] D80525: [clangd] Fix crash-bug in preamble indexing when using modules.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 274482.
adamcz added a comment.

Changed to just ignore undefs without matching def


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80525/new/

https://reviews.llvm.org/D80525

Files:
  clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
  clang-tools-extra/clangd/unittests/TestFS.h
  clang-tools-extra/clangd/unittests/TestTU.cpp
  clang-tools-extra/clangd/unittests/TestTU.h
  clang/lib/Index/IndexingAction.cpp

Index: clang/lib/Index/IndexingAction.cpp
===
--- clang/lib/Index/IndexingAction.cpp
+++ clang/lib/Index/IndexingAction.cpp
@@ -165,11 +165,20 @@
 static void indexPreprocessorMacros(const Preprocessor &PP,
 IndexDataConsumer &DataConsumer) {
   for (const auto &M : PP.macros())
-if (MacroDirective *MD = M.second.getLatest())
+if (MacroDirective *MD = M.second.getLatest()) {
+  auto *MI = MD->getMacroInfo();
+  // When using modules, it may happen that we find #undef of a macro that
+  // was defined in another module. In such case, MI may be nullptr, since
+  // we only look for macro definitions in the current TU. In that case,
+  // there is nothing to index.
+  if (!MI)
+continue;
+
   DataConsumer.handleMacroOccurrence(
   M.first, MD->getMacroInfo(),
   static_cast(index::SymbolRole::Definition),
   MD->getLocation());
+}
 }
 
 void index::indexASTUnit(ASTUnit &Unit, IndexDataConsumer &DataConsumer,
Index: clang-tools-extra/clangd/unittests/TestTU.h
===
--- clang-tools-extra/clangd/unittests/TestTU.h
+++ clang-tools-extra/clangd/unittests/TestTU.h
@@ -66,6 +66,10 @@
   // Simulate a header guard of the header (using an #import directive).
   bool ImplicitHeaderGuard = true;
 
+  // Whether to use overlay the TestFS over the real filesystem. This is
+  // required for use of modules.
+  bool OverlayRealFileSystem = false;
+
   // By default, build() will report Error diagnostics as GTest errors.
   // Suppress this behavior by adding an 'error-ok' comment to the code.
   ParsedAST build() const;
Index: clang-tools-extra/clangd/unittests/TestTU.cpp
===
--- clang-tools-extra/clangd/unittests/TestTU.cpp
+++ clang-tools-extra/clangd/unittests/TestTU.cpp
@@ -54,6 +54,8 @@
   Inputs.CompileCommand.Filename = FullFilename;
   Inputs.CompileCommand.Directory = testRoot();
   Inputs.Contents = Code;
+  if (OverlayRealFileSystem)
+FS.OverlayRealFileSystem = true;
   Inputs.TFS = &FS;
   Inputs.Opts = ParseOptions();
   Inputs.Opts.BuildRecoveryAST = true;
Index: clang-tools-extra/clangd/unittests/TestFS.h
===
--- clang-tools-extra/clangd/unittests/TestFS.h
+++ clang-tools-extra/clangd/unittests/TestFS.h
@@ -35,12 +35,21 @@
 public:
   IntrusiveRefCntPtr
   view(llvm::NoneType) const override {
-return buildTestFS(Files, Timestamps);
+auto MemFS = buildTestFS(Files, Timestamps);
+if (!OverlayRealFileSystem)
+  return MemFS;
+llvm::IntrusiveRefCntPtr OverlayFileSystem =
+new llvm::vfs::OverlayFileSystem(llvm::vfs::getRealFileSystem());
+OverlayFileSystem->pushOverlay(MemFS);
+return OverlayFileSystem;
   }
 
   // If relative paths are used, they are resolved with testPath().
   llvm::StringMap Files;
   llvm::StringMap Timestamps;
+  // If true, real file system will be used as fallback for the in-memory one.
+  // This is useful for testing module support.
+  bool OverlayRealFileSystem = false;
 };
 
 // A Compilation database that returns a fixed set of compile flags.
Index: clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
===
--- clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
+++ clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
@@ -1504,6 +1504,30 @@
   EXPECT_THAT(Refs, Contains(Pair(findSymbol(Symbols, "PUNCT").ID, _)));
 }
 
+// Regression test for a crash-bug we used to have.
+TEST_F(SymbolCollectorTest, UndefOfModuleMacro) {
+  auto TU = TestTU::withCode(R"cpp(#include "/bar.h")cpp");
+  TU.AdditionalFiles["/bar.h"] = R"cpp(
+#include "/foo.h"
+#undef X
+)cpp";
+  TU.AdditionalFiles["/foo.h"] = "#define X 1";
+  TU.AdditionalFiles["/module.map"] = R"cpp(
+module foo {
+ header "/foo.h"
+ export *
+   }
+   )cpp";
+  TU.ExtraArgs.push_back("-fmodules");
+  TU.ExtraArgs.push_back("-fmodule-map-file=/module.map");
+  TU.OverlayRealFileSystem = true;
+
+  TU.build();
+  // We mostly care about not crashing, but verify that we didn't insert garbage
+  // about X too.
+  EXPECT_THAT(TU.headerSymbols(), Not(Contains(QName("X";
+}
+
 } // namespace
 } // namespace clangd
 } // namespace clang

[PATCH] D80525: [clangd] Fix crash-bug in preamble indexing when using modules.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added a comment.

Resurrecting this, with the ignore #undef when #define is in a module approach. 
Still worth having a test, especially that it's the first test we have for 
modules.

Note that the test code changed since last version due to introduction of 
ThreasafeFS since my original change.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80525/new/

https://reviews.llvm.org/D80525



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D78454: [clangd] Highlight related control flow.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/XRefs.cpp:628
+return FD->getBody();
+  if (const auto *FD = N.get())
+return FD->getBody();

You are checking for FunctionDecl twice



Comment at: clang-tools-extra/clangd/XRefs.cpp:654
+  // Types of control-flow statements we might highlight.
+  enum Target {
+Break = 1,

What about "goto"? It's more difficult to figure out if it breaks out of the 
loop or just jumps inside it, but it might still be worth highlighting it, even 
unconditionally (inside a loop only, not function).



Comment at: clang-tools-extra/clangd/XRefs.cpp:761
+  auto CaseBefore = std::prev(CaseAfter);
+  // ... rewind CaseBefore to the first in a `case A: case B: ...` sequence.
+  while (CaseBefore != Cases.begin() &&

That's a nice feature, but there's no test for it. Can you add it?



Comment at: clang-tools-extra/clangd/unittests/XRefsTests.cpp:119
 
+TEST(HighlightsTest, ControlFlow) {
+  const char *Tests[] = {

Can you add a test for macros? I think highlighting like ASSIGN_OR_RETURN(int 
x, foo()); would be great, but even if we don't support that, just having a 
test that we don't crash would be valuable, IMHO.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D78454/new/

https://reviews.llvm.org/D78454



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85532: Correctly set CompilingPCH in PrecompilePreambleAction.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
Herald added subscribers: cfe-commits, usaxena95, kadircet, arphaman, jkorous, 
mgorny.
Herald added a project: clang.
adamcz requested review of this revision.
Herald added a subscriber: ilya-biryukov.

This fixes a crash bug in clangd when used with modules. ASTWriter would
end up writing references to submodules into the PCH file, but upon
reading the submodules would not exists and
HeaderFileInfoTrait::ReadData would crash.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85532

Files:
  clang-tools-extra/clangd/unittests/CMakeLists.txt
  clang-tools-extra/clangd/unittests/ModulesTests.cpp


Index: clang-tools-extra/clangd/unittests/ModulesTests.cpp
===
--- /dev/null
+++ clang-tools-extra/clangd/unittests/ModulesTests.cpp
@@ -0,0 +1,45 @@
+//===-- ModulesTests.cpp  ---*- C++ 
-*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM 
Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+
+#include "Annotations.h"
+#include "TestFS.h"
+#include "TestTU.h"
+#include "gmock/gmock.h"
+#include "gtest/gtest.h"
+
+#include 
+#include 
+
+namespace clang {
+namespace clangd {
+namespace {
+
+TEST(Modules, TextualIncludeInPreamble) {
+  TestTU TU = TestTU::withCode(R"cpp(
+#include "Textual.h"
+
+void foo() {}
+)cpp");
+  TU.ExtraArgs.push_back("-fmodules");
+  TU.ExtraArgs.push_back("-fmodule-name=M");
+  TU.ExtraArgs.push_back("-fmodule-map-file=m.modulemap");
+  TU.AdditionalFiles["Textual.h"] = "void foo();";
+  TU.AdditionalFiles["m.modulemap"] = R"modulemap(
+module M {
+  module Textual {
+textual header "Textual.h"
+  }
+}
+)modulemap";
+  // Test that we do not crash.
+  TU.index();
+}
+
+} // namespace
+} // namespace clangd
+} // namespace clang
Index: clang-tools-extra/clangd/unittests/CMakeLists.txt
===
--- clang-tools-extra/clangd/unittests/CMakeLists.txt
+++ clang-tools-extra/clangd/unittests/CMakeLists.txt
@@ -63,6 +63,7 @@
   IndexTests.cpp
   JSONTransportTests.cpp
   LSPClient.cpp
+  ModulesTests.cpp
   ParsedASTTests.cpp
   PathMappingTests.cpp
   PreambleTests.cpp


Index: clang-tools-extra/clangd/unittests/ModulesTests.cpp
===
--- /dev/null
+++ clang-tools-extra/clangd/unittests/ModulesTests.cpp
@@ -0,0 +1,45 @@
+//===-- ModulesTests.cpp  ---*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+
+#include "Annotations.h"
+#include "TestFS.h"
+#include "TestTU.h"
+#include "gmock/gmock.h"
+#include "gtest/gtest.h"
+
+#include 
+#include 
+
+namespace clang {
+namespace clangd {
+namespace {
+
+TEST(Modules, TextualIncludeInPreamble) {
+  TestTU TU = TestTU::withCode(R"cpp(
+#include "Textual.h"
+
+void foo() {}
+)cpp");
+  TU.ExtraArgs.push_back("-fmodules");
+  TU.ExtraArgs.push_back("-fmodule-name=M");
+  TU.ExtraArgs.push_back("-fmodule-map-file=m.modulemap");
+  TU.AdditionalFiles["Textual.h"] = "void foo();";
+  TU.AdditionalFiles["m.modulemap"] = R"modulemap(
+module M {
+  module Textual {
+textual header "Textual.h"
+  }
+}
+)modulemap";
+  // Test that we do not crash.
+  TU.index();
+}
+
+} // namespace
+} // namespace clangd
+} // namespace clang
Index: clang-tools-extra/clangd/unittests/CMakeLists.txt
===
--- clang-tools-extra/clangd/unittests/CMakeLists.txt
+++ clang-tools-extra/clangd/unittests/CMakeLists.txt
@@ -63,6 +63,7 @@
   IndexTests.cpp
   JSONTransportTests.cpp
   LSPClient.cpp
+  ModulesTests.cpp
   ParsedASTTests.cpp
   PathMappingTests.cpp
   PreambleTests.cpp
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85532: Correctly set CompilingPCH in PrecompilePreambleAction.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 283922.
adamcz added a comment.

This time the fix is included ;-)


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85532/new/

https://reviews.llvm.org/D85532

Files:
  clang-tools-extra/clangd/unittests/CMakeLists.txt
  clang-tools-extra/clangd/unittests/ModulesTests.cpp
  clang/lib/Frontend/PrecompiledPreamble.cpp


Index: clang/lib/Frontend/PrecompiledPreamble.cpp
===
--- clang/lib/Frontend/PrecompiledPreamble.cpp
+++ clang/lib/Frontend/PrecompiledPreamble.cpp
@@ -208,6 +208,11 @@
 Callbacks.AfterPCHEmitted(Writer);
   }
 
+  bool BeginSourceFileAction(CompilerInstance &CI) override {
+CI.getLangOpts().CompilingPCH = true;
+return ASTFrontendAction::BeginSourceFileAction(CI);
+  }
+
   bool shouldEraseOutputFiles() override { return !hasEmittedPreamblePCH(); }
   bool hasCodeCompletionSupport() const override { return false; }
   bool hasASTFileSupport() const override { return false; }
Index: clang-tools-extra/clangd/unittests/ModulesTests.cpp
===
--- /dev/null
+++ clang-tools-extra/clangd/unittests/ModulesTests.cpp
@@ -0,0 +1,45 @@
+//===-- ModulesTests.cpp  ---*- C++ 
-*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM 
Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+
+#include "Annotations.h"
+#include "TestFS.h"
+#include "TestTU.h"
+#include "gmock/gmock.h"
+#include "gtest/gtest.h"
+
+#include 
+#include 
+
+namespace clang {
+namespace clangd {
+namespace {
+
+TEST(Modules, TextualIncludeInPreamble) {
+  TestTU TU = TestTU::withCode(R"cpp(
+#include "Textual.h"
+
+void foo() {}
+)cpp");
+  TU.ExtraArgs.push_back("-fmodules");
+  TU.ExtraArgs.push_back("-fmodule-name=M");
+  TU.ExtraArgs.push_back("-fmodule-map-file=m.modulemap");
+  TU.AdditionalFiles["Textual.h"] = "void foo();";
+  TU.AdditionalFiles["m.modulemap"] = R"modulemap(
+module M {
+  module Textual {
+textual header "Textual.h"
+  }
+}
+)modulemap";
+  // Test that we do not crash.
+  TU.index();
+}
+
+} // namespace
+} // namespace clangd
+} // namespace clang
Index: clang-tools-extra/clangd/unittests/CMakeLists.txt
===
--- clang-tools-extra/clangd/unittests/CMakeLists.txt
+++ clang-tools-extra/clangd/unittests/CMakeLists.txt
@@ -63,6 +63,7 @@
   IndexTests.cpp
   JSONTransportTests.cpp
   LSPClient.cpp
+  ModulesTests.cpp
   ParsedASTTests.cpp
   PathMappingTests.cpp
   PreambleTests.cpp


Index: clang/lib/Frontend/PrecompiledPreamble.cpp
===
--- clang/lib/Frontend/PrecompiledPreamble.cpp
+++ clang/lib/Frontend/PrecompiledPreamble.cpp
@@ -208,6 +208,11 @@
 Callbacks.AfterPCHEmitted(Writer);
   }
 
+  bool BeginSourceFileAction(CompilerInstance &CI) override {
+CI.getLangOpts().CompilingPCH = true;
+return ASTFrontendAction::BeginSourceFileAction(CI);
+  }
+
   bool shouldEraseOutputFiles() override { return !hasEmittedPreamblePCH(); }
   bool hasCodeCompletionSupport() const override { return false; }
   bool hasASTFileSupport() const override { return false; }
Index: clang-tools-extra/clangd/unittests/ModulesTests.cpp
===
--- /dev/null
+++ clang-tools-extra/clangd/unittests/ModulesTests.cpp
@@ -0,0 +1,45 @@
+//===-- ModulesTests.cpp  ---*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+
+#include "Annotations.h"
+#include "TestFS.h"
+#include "TestTU.h"
+#include "gmock/gmock.h"
+#include "gtest/gtest.h"
+
+#include 
+#include 
+
+namespace clang {
+namespace clangd {
+namespace {
+
+TEST(Modules, TextualIncludeInPreamble) {
+  TestTU TU = TestTU::withCode(R"cpp(
+#include "Textual.h"
+
+void foo() {}
+)cpp");
+  TU.ExtraArgs.push_back("-fmodules");
+  TU.ExtraArgs.push_back("-fmodule-name=M");
+  TU.ExtraArgs.push_back("-fmodule-map-file=m.modulemap");
+  TU.AdditionalFiles["Textual.h"] = "void foo();";
+  TU.AdditionalFiles["m.modulemap"] = R"modulemap(
+module M {
+  module Textual {
+textual header "Textual.h"
+  }
+}
+)modulemap";
+  // Test that we do not crash.
+  TU.index();
+}
+
+} // namespace
+} // namespace clangd
+} // namespace clang
Index: clang-tools-extra/clangd/unittests/CMakeLists.txt
===
--- c

[PATCH] D85532: Correctly set CompilingPCH in PrecompilePreambleAction.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 284329.
adamcz marked 3 inline comments as done.
adamcz added a comment.

addressed review comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85532/new/

https://reviews.llvm.org/D85532

Files:
  clang-tools-extra/clangd/unittests/CMakeLists.txt
  clang-tools-extra/clangd/unittests/ModulesTests.cpp
  clang/lib/Frontend/PrecompiledPreamble.cpp
  clang/unittests/Frontend/ASTUnitTest.cpp

Index: clang/unittests/Frontend/ASTUnitTest.cpp
===
--- clang/unittests/Frontend/ASTUnitTest.cpp
+++ clang/unittests/Frontend/ASTUnitTest.cpp
@@ -13,6 +13,7 @@
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Frontend/PCHContainerOperations.h"
+#include "clang/Lex/HeaderSearch.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/ToolOutputFile.h"
@@ -111,4 +112,41 @@
 llvm::MemoryBuffer::MemoryBuffer_MMap);
 }
 
+TEST_F(ASTUnitTest, ModuleTextualHeader) {
+  llvm::IntrusiveRefCntPtr InMemoryFs =
+  new llvm::vfs::InMemoryFileSystem();
+  InMemoryFs->addFile("test.cpp", 0, llvm::MemoryBuffer::getMemBuffer(R"cpp(
+  #include "Textual.h"
+  void foo() {}
+)cpp"));
+  InMemoryFs->addFile("m.modulemap", 0, llvm::MemoryBuffer::getMemBuffer(R"cpp(
+  module M {
+module Textual {
+  textual header "Textual.h"
+}
+  }
+)cpp"));
+  InMemoryFs->addFile("Textual.h", 0, llvm::MemoryBuffer::getMemBuffer(R"cpp(
+  void foo();
+)cpp"));
+
+  const char *Args[] = {"clang", "test.cpp", "-fmodule-map-file=m.modulemap",
+"-fmodule-name=M"};
+  Diags = CompilerInstance::createDiagnostics(new DiagnosticOptions());
+  CInvok = createInvocationFromCommandLine(Args, Diags);
+  ASSERT_TRUE(CInvok);
+
+  FileManager *FileMgr = new FileManager(FileSystemOptions(), InMemoryFs);
+  PCHContainerOps = std::make_shared();
+
+  auto AU = ASTUnit::LoadFromCompilerInvocation(
+  CInvok, PCHContainerOps, Diags, FileMgr, false, CaptureDiagsKind::None, 1,
+  TU_Complete, false, false, false);
+  auto File = AU->getFileManager().getFileRef("Textual.h", false, false);
+  assert(File);
+  // Verify that we do not crash here.
+  EXPECT_TRUE(AU->getPreprocessor().getHeaderSearchInfo().getExistingFileInfo(
+  &File->getFileEntry()));
+}
+
 } // anonymous namespace
Index: clang/lib/Frontend/PrecompiledPreamble.cpp
===
--- clang/lib/Frontend/PrecompiledPreamble.cpp
+++ clang/lib/Frontend/PrecompiledPreamble.cpp
@@ -208,6 +208,11 @@
 Callbacks.AfterPCHEmitted(Writer);
   }
 
+  bool BeginSourceFileAction(CompilerInstance &CI) override {
+assert(CI.getLangOpts().CompilingPCH);
+return ASTFrontendAction::BeginSourceFileAction(CI);
+  }
+
   bool shouldEraseOutputFiles() override { return !hasEmittedPreamblePCH(); }
   bool hasCodeCompletionSupport() const override { return false; }
   bool hasASTFileSupport() const override { return false; }
@@ -396,6 +401,8 @@
   auto PreambleDepCollector = std::make_shared();
   Clang->addDependencyCollector(PreambleDepCollector);
 
+  Clang->getLangOpts().CompilingPCH = true;
+
   // Remap the main source file to the preamble buffer.
   StringRef MainFilePath = FrontendOpts.Inputs[0].getFile();
   auto PreambleInputBuffer = llvm::MemoryBuffer::getMemBufferCopy(
Index: clang-tools-extra/clangd/unittests/ModulesTests.cpp
===
--- /dev/null
+++ clang-tools-extra/clangd/unittests/ModulesTests.cpp
@@ -0,0 +1,44 @@
+//===-- ModulesTests.cpp  ---*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+
+#include "Annotations.h"
+#include "TestFS.h"
+#include "TestTU.h"
+#include "gmock/gmock.h"
+#include "gtest/gtest.h"
+
+#include 
+#include 
+
+namespace clang {
+namespace clangd {
+namespace {
+
+TEST(Modules, TextualIncludeInPreamble) {
+  TestTU TU = TestTU::withCode(R"cpp(
+#include "Textual.h"
+
+void foo() {}
+)cpp");
+  TU.ExtraArgs.push_back("-fmodule-name=M");
+  TU.ExtraArgs.push_back("-fmodule-map-file=m.modulemap");
+  TU.AdditionalFiles["Textual.h"] = "void foo();";
+  TU.AdditionalFiles["m.modulemap"] = R"modulemap(
+module M {
+  module Textual {
+textual header "Textual.h"
+  }
+}
+)modulemap";
+  // Test that we do not crash.
+  TU.index();
+}
+
+} // namespace
+} // namespace clangd
+} // namespace clang
Index: clang-tools-extra/clangd/unittests/CMakeLists.txt

[PATCH] D85532: Correctly set CompilingPCH in PrecompilePreambleAction.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/unittests/ModulesTests.cpp:28
+)cpp");
+  TU.ExtraArgs.push_back("-fmodules");
+  TU.ExtraArgs.push_back("-fmodule-name=M");

kadircet wrote:
> this one is not needed right?
I suppose not.



Comment at: clang/lib/Frontend/PrecompiledPreamble.cpp:421
   StoreInMemory ? &Storage.asMemory().Data : nullptr, Callbacks));
   Callbacks.BeforeExecute(*Clang);
   if (!Act->BeginSourceFile(*Clang.get(), Clang->getFrontendOpts().Inputs[0]))

kadircet wrote:
> anything operating on CompilerInstance before and including the callbacks 
> invoked here will see a broken LangOtps. Why not set it here (and maybe 
> assert in the override, or just not do anything at all)
I think we should at least assert() on this, so others don't waste as much time 
as I did looking for bugs like this. PrecompilePreambleAction is not usable 
with CompilingPCH = false.

I'm a bit conflicted on setting CompilingPCH outside of 
PrecompilePreambleAction, it makes it seem like you can change it somehow and 
it makes that Action class not self-contained (it requires that external bit 
that sets the options), but with assert it should be fine and it's essentially 
an internal detail of PrecompilePreamble class anyway.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85532/new/

https://reviews.llvm.org/D85532

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80525: [clangd] Fix crash-bug in preamble indexing when using modules.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 284360.
adamcz marked 2 inline comments as done.
adamcz added a comment.

final(?) review comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80525/new/

https://reviews.llvm.org/D80525

Files:
  clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
  clang-tools-extra/clangd/unittests/TestFS.h
  clang-tools-extra/clangd/unittests/TestTU.cpp
  clang-tools-extra/clangd/unittests/TestTU.h
  clang/lib/Index/IndexingAction.cpp

Index: clang/lib/Index/IndexingAction.cpp
===
--- clang/lib/Index/IndexingAction.cpp
+++ clang/lib/Index/IndexingAction.cpp
@@ -165,11 +165,20 @@
 static void indexPreprocessorMacros(const Preprocessor &PP,
 IndexDataConsumer &DataConsumer) {
   for (const auto &M : PP.macros())
-if (MacroDirective *MD = M.second.getLatest())
+if (MacroDirective *MD = M.second.getLatest()) {
+  auto *MI = MD->getMacroInfo();
+  // When using modules, it may happen that we find #undef of a macro that
+  // was defined in another module. In such case, MI may be nullptr, since
+  // we only look for macro definitions in the current TU. In that case,
+  // there is nothing to index.
+  if (!MI)
+continue;
+
   DataConsumer.handleMacroOccurrence(
   M.first, MD->getMacroInfo(),
   static_cast(index::SymbolRole::Definition),
   MD->getLocation());
+}
 }
 
 void index::indexASTUnit(ASTUnit &Unit, IndexDataConsumer &DataConsumer,
Index: clang-tools-extra/clangd/unittests/TestTU.h
===
--- clang-tools-extra/clangd/unittests/TestTU.h
+++ clang-tools-extra/clangd/unittests/TestTU.h
@@ -66,6 +66,14 @@
   // Simulate a header guard of the header (using an #import directive).
   bool ImplicitHeaderGuard = true;
 
+  // Whether to use overlay the TestFS over the real filesystem. This is
+  // required for use of modules.
+  // FIXME: Intercept the write of implicitly build module to disk and update
+  // TestFS with that content to avoid using real file system in tests.
+  // Please avoid using this for things other than implicit modules. The plan is
+  // to eliminate this option some day.
+  bool OverlayRealFileSystemForModules = false;
+
   // By default, build() will report Error diagnostics as GTest errors.
   // Suppress this behavior by adding an 'error-ok' comment to the code.
   ParsedAST build() const;
Index: clang-tools-extra/clangd/unittests/TestTU.cpp
===
--- clang-tools-extra/clangd/unittests/TestTU.cpp
+++ clang-tools-extra/clangd/unittests/TestTU.cpp
@@ -54,6 +54,8 @@
   Inputs.CompileCommand.Filename = FullFilename;
   Inputs.CompileCommand.Directory = testRoot();
   Inputs.Contents = Code;
+  if (OverlayRealFileSystemForModules)
+FS.OverlayRealFileSystem = true;
   Inputs.TFS = &FS;
   Inputs.Opts = ParseOptions();
   Inputs.Opts.BuildRecoveryAST = true;
Index: clang-tools-extra/clangd/unittests/TestFS.h
===
--- clang-tools-extra/clangd/unittests/TestFS.h
+++ clang-tools-extra/clangd/unittests/TestFS.h
@@ -34,12 +34,21 @@
 class MockFS : public ThreadsafeFS {
 public:
   IntrusiveRefCntPtr viewImpl() const override {
-return buildTestFS(Files, Timestamps);
+auto MemFS = buildTestFS(Files, Timestamps);
+if (!OverlayRealFileSystem)
+  return MemFS;
+llvm::IntrusiveRefCntPtr OverlayFileSystem =
+new llvm::vfs::OverlayFileSystem(llvm::vfs::getRealFileSystem());
+OverlayFileSystem->pushOverlay(MemFS);
+return OverlayFileSystem;
   }
 
   // If relative paths are used, they are resolved with testPath().
   llvm::StringMap Files;
   llvm::StringMap Timestamps;
+  // If true, real file system will be used as fallback for the in-memory one.
+  // This is useful for testing module support.
+  bool OverlayRealFileSystem = false;
 };
 
 // A Compilation database that returns a fixed set of compile flags.
Index: clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
===
--- clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
+++ clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
@@ -1520,6 +1520,31 @@
   EXPECT_THAT(Symbols,
   UnorderedElementsAre(AllOf(QName("X"), ForCodeCompletion(true;
 }
+
+// Regression test for a crash-bug we used to have.
+TEST_F(SymbolCollectorTest, UndefOfModuleMacro) {
+  auto TU = TestTU::withCode(R"cpp(#include "bar.h")cpp");
+  TU.AdditionalFiles["bar.h"] = R"cpp(
+#include "foo.h"
+#undef X
+)cpp";
+  TU.AdditionalFiles["foo.h"] = "#define X 1";
+  TU.AdditionalFiles["module.map"] = R"cpp(
+module foo {
+ header "foo.h"
+ export *
+   }
+   )cpp";
+  TU.ExtraArgs.push_b

[PATCH] D80525: [clangd] Fix crash-bug in preamble indexing when using modules.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp:1523
+  TU.ExtraArgs.push_back("-fmodule-map-file=/module.map");
+  TU.OverlayRealFileSystem = true;
+

sammccall wrote:
> I'm a bit torn on this - this is obviously a bit messy (writing modules to 
> the real filesystem and exposing the whole FS to our test).
> It's not a huge thing and we could slap a FIXME on it, though you clearly 
> have to plumb it through a few layers.
> 
> I think the alternative is an explicit module build:
>  - subclass `clang::GenerateHeaderModuleAction`to override the output so it 
> ends up in a buffer
>  - add a method to TestTU like buildHeaderModule() that returns a 
> `std::string` containing the module content
>  - This test would have two TestTU instances, and the second looks something 
> like:
> ```
> TU2.AdditionalFiles["foo.pcm"] = TU1.buildHeaderModule();
> TU2.ExtraArgs = {"-fprebuild-module-path=.", "-std=c++20", "-fmodules"};
> TU2.HeaderCode = "import foo;\n#undef X";
> ```
> 
> I guess whether this is actually better probably depends on whether we're 
> likely to go in this direction (explicitly building and managing a module 
> cache ourselves) for modules. If we're likely to let clang implicitly build 
> modules and maybe just put the physical cache storage behind an abstraction, 
> maybe it's best to overlay the FS for now and rip this out later. If we're 
> going to do explicit module builds and schedule them ourselves, then starting 
> to experiment in that direction is useful, and there won't be an obvious time 
> to rip out the OverlayRealFileSystem feature.
> 
> I think all of which is to say feel free to land it in this form, or 
> experiment further.
> I think the OverlayRealFileSystem should maybe be `ExposeModuleCacheDir` or 
> something with a FIXME to come up with a better solution, though. It's not a 
> feature of TesttU we should be using for other purposes (without similar 
> careful consideration)
I think we should support implicit modules in general. It's a useful feature. 
This means we should be able to test it. While it might not matter for this 
particular fix, we need some implicit modules tests to prevent regressions.

We should also have explicit modules tests, of course. All of that is coming 
soon :-)

Ultimately, I think a proper solution is to either:
a. Support writing to disk in VFS, then implement that in InMemoryFS.
b. Create abstraction around writing implicitly compiled module to disk, 
intercept that in test and update the memory FS.

Both seem acceptable to me, although I haven't looked at details yet. I think 
for now, we should just accept the sad fact that some tests will write to disk. 
It should be easy to remove it later.

I added a FIXME and renamed to OverlayRealFilesystemForModules to make it clear 
it should not be used for anything else. Does that sound good to you?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80525/new/

https://reviews.llvm.org/D80525

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85532: Correctly set CompilingPCH in PrecompilePreambleAction.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 284365.
adamcz marked an inline comment as done.
adamcz added a comment.

more review comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85532/new/

https://reviews.llvm.org/D85532

Files:
  clang-tools-extra/clangd/unittests/CMakeLists.txt
  clang-tools-extra/clangd/unittests/ModulesTests.cpp
  clang/lib/Frontend/PrecompiledPreamble.cpp
  clang/unittests/Frontend/ASTUnitTest.cpp

Index: clang/unittests/Frontend/ASTUnitTest.cpp
===
--- clang/unittests/Frontend/ASTUnitTest.cpp
+++ clang/unittests/Frontend/ASTUnitTest.cpp
@@ -13,6 +13,7 @@
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Frontend/PCHContainerOperations.h"
+#include "clang/Lex/HeaderSearch.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/ToolOutputFile.h"
@@ -111,4 +112,42 @@
 llvm::MemoryBuffer::MemoryBuffer_MMap);
 }
 
+TEST_F(ASTUnitTest, ModuleTextualHeader) {
+  llvm::IntrusiveRefCntPtr InMemoryFs =
+  new llvm::vfs::InMemoryFileSystem();
+  InMemoryFs->addFile("test.cpp", 0, llvm::MemoryBuffer::getMemBuffer(R"cpp(
+  #include "Textual.h"
+  void foo() {}
+)cpp"));
+  InMemoryFs->addFile("m.modulemap", 0, llvm::MemoryBuffer::getMemBuffer(R"cpp(
+  module M {
+module Textual {
+  textual header "Textual.h"
+}
+  }
+)cpp"));
+  InMemoryFs->addFile("Textual.h", 0, llvm::MemoryBuffer::getMemBuffer(R"cpp(
+  void foo();
+)cpp"));
+
+  const char *Args[] = {"clang", "test.cpp", "-fmodule-map-file=m.modulemap",
+"-fmodule-name=M"};
+  Diags = CompilerInstance::createDiagnostics(new DiagnosticOptions());
+  CInvok = createInvocationFromCommandLine(Args, Diags);
+  ASSERT_TRUE(CInvok);
+
+  FileManager *FileMgr = new FileManager(FileSystemOptions(), InMemoryFs);
+  PCHContainerOps = std::make_shared();
+
+  auto AU = ASTUnit::LoadFromCompilerInvocation(
+  CInvok, PCHContainerOps, Diags, FileMgr, false, CaptureDiagsKind::None, 1,
+  TU_Complete, false, false, false);
+  ASSERT_TRUE(AU);
+  auto File = AU->getFileManager().getFileRef("Textual.h", false, false);
+  assert(File);
+  // Verify that we do not crash here.
+  EXPECT_TRUE(AU->getPreprocessor().getHeaderSearchInfo().getExistingFileInfo(
+  &File->getFileEntry()));
+}
+
 } // anonymous namespace
Index: clang/lib/Frontend/PrecompiledPreamble.cpp
===
--- clang/lib/Frontend/PrecompiledPreamble.cpp
+++ clang/lib/Frontend/PrecompiledPreamble.cpp
@@ -208,6 +208,11 @@
 Callbacks.AfterPCHEmitted(Writer);
   }
 
+  bool BeginSourceFileAction(CompilerInstance &CI) override {
+assert(CI.getLangOpts().CompilingPCH);
+return ASTFrontendAction::BeginSourceFileAction(CI);
+  }
+
   bool shouldEraseOutputFiles() override { return !hasEmittedPreamblePCH(); }
   bool hasCodeCompletionSupport() const override { return false; }
   bool hasASTFileSupport() const override { return false; }
@@ -396,6 +401,8 @@
   auto PreambleDepCollector = std::make_shared();
   Clang->addDependencyCollector(PreambleDepCollector);
 
+  Clang->getLangOpts().CompilingPCH = true;
+
   // Remap the main source file to the preamble buffer.
   StringRef MainFilePath = FrontendOpts.Inputs[0].getFile();
   auto PreambleInputBuffer = llvm::MemoryBuffer::getMemBufferCopy(
Index: clang-tools-extra/clangd/unittests/ModulesTests.cpp
===
--- /dev/null
+++ clang-tools-extra/clangd/unittests/ModulesTests.cpp
@@ -0,0 +1,44 @@
+//===-- ModulesTests.cpp  ---*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+
+#include "Annotations.h"
+#include "TestFS.h"
+#include "TestTU.h"
+#include "gmock/gmock.h"
+#include "gtest/gtest.h"
+
+#include 
+#include 
+
+namespace clang {
+namespace clangd {
+namespace {
+
+TEST(Modules, TextualIncludeInPreamble) {
+  TestTU TU = TestTU::withCode(R"cpp(
+#include "Textual.h"
+
+void foo() {}
+)cpp");
+  TU.ExtraArgs.push_back("-fmodule-name=M");
+  TU.ExtraArgs.push_back("-fmodule-map-file=m.modulemap");
+  TU.AdditionalFiles["Textual.h"] = "void foo();";
+  TU.AdditionalFiles["m.modulemap"] = R"modulemap(
+module M {
+  module Textual {
+textual header "Textual.h"
+  }
+}
+)modulemap";
+  // Test that we do not crash.
+  TU.index();
+}
+
+} // namespace
+} // namespace clangd
+} // namespace clang
Index: clang-tools-extra/clangd/unittests/CMakeLists.txt
=

[PATCH] D85532: Correctly set CompilingPCH in PrecompilePreambleAction.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang/unittests/Frontend/ASTUnitTest.cpp:146
+  auto File = AU->getFileManager().getFileRef("Textual.h", false, false);
+  assert(File);
+  // Verify that we do not crash here.

kadircet wrote:
> ASSERT_TRUE again
You can 't ASSERT_TRUE() on llvm::Expected, since operator bool() is not const. 
No idea why, but that's why it's assert()


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85532/new/

https://reviews.llvm.org/D85532

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85532: Correctly set CompilingPCH in PrecompilePreambleAction.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang/unittests/Frontend/ASTUnitTest.cpp:146
+  auto File = AU->getFileManager().getFileRef("Textual.h", false, false);
+  assert(File);
+  // Verify that we do not crash here.

adamcz wrote:
> kadircet wrote:
> > ASSERT_TRUE again
> You can 't ASSERT_TRUE() on llvm::Expected, since operator bool() is not 
> const. No idea why, but that's why it's assert()
Nevermind, I do know why - because it changes the "checked" status, that makes 
sense. I guess we could make that mutable, but that seems out of scope for this 
change.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85532/new/

https://reviews.llvm.org/D85532

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80525: [clangd] Fix crash-bug in preamble indexing when using modules.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 284376.
adamcz marked 2 inline comments as done.
adamcz added a comment.

var rename, comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80525/new/

https://reviews.llvm.org/D80525

Files:
  clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
  clang-tools-extra/clangd/unittests/TestFS.h
  clang-tools-extra/clangd/unittests/TestTU.cpp
  clang-tools-extra/clangd/unittests/TestTU.h
  clang/lib/Index/IndexingAction.cpp

Index: clang/lib/Index/IndexingAction.cpp
===
--- clang/lib/Index/IndexingAction.cpp
+++ clang/lib/Index/IndexingAction.cpp
@@ -165,11 +165,20 @@
 static void indexPreprocessorMacros(const Preprocessor &PP,
 IndexDataConsumer &DataConsumer) {
   for (const auto &M : PP.macros())
-if (MacroDirective *MD = M.second.getLatest())
+if (MacroDirective *MD = M.second.getLatest()) {
+  auto *MI = MD->getMacroInfo();
+  // When using modules, it may happen that we find #undef of a macro that
+  // was defined in another module. In such case, MI may be nullptr, since
+  // we only look for macro definitions in the current TU. In that case,
+  // there is nothing to index.
+  if (!MI)
+continue;
+
   DataConsumer.handleMacroOccurrence(
   M.first, MD->getMacroInfo(),
   static_cast(index::SymbolRole::Definition),
   MD->getLocation());
+}
 }
 
 void index::indexASTUnit(ASTUnit &Unit, IndexDataConsumer &DataConsumer,
Index: clang-tools-extra/clangd/unittests/TestTU.h
===
--- clang-tools-extra/clangd/unittests/TestTU.h
+++ clang-tools-extra/clangd/unittests/TestTU.h
@@ -66,6 +66,16 @@
   // Simulate a header guard of the header (using an #import directive).
   bool ImplicitHeaderGuard = true;
 
+  // Whether to use overlay the TestFS over the real filesystem. This is
+  // required for use of implicit modules.where the module file is written to
+  // disk and later read back.
+  // FIXME: Change the way reading/writing modules work to allow us to keep them
+  // in memory across multiple clang invocations, at least in tests, to
+  // eliminate the need for real file system here.
+  // Please avoid using this for things other than implicit modules. The plan is
+  // to eliminate this option some day.
+  bool OverlayRealFileSystemForModules = false;
+
   // By default, build() will report Error diagnostics as GTest errors.
   // Suppress this behavior by adding an 'error-ok' comment to the code.
   ParsedAST build() const;
Index: clang-tools-extra/clangd/unittests/TestTU.cpp
===
--- clang-tools-extra/clangd/unittests/TestTU.cpp
+++ clang-tools-extra/clangd/unittests/TestTU.cpp
@@ -54,6 +54,8 @@
   Inputs.CompileCommand.Filename = FullFilename;
   Inputs.CompileCommand.Directory = testRoot();
   Inputs.Contents = Code;
+  if (OverlayRealFileSystemForModules)
+FS.OverlayRealFileSystemForModules = true;
   Inputs.TFS = &FS;
   Inputs.Opts = ParseOptions();
   Inputs.Opts.BuildRecoveryAST = true;
Index: clang-tools-extra/clangd/unittests/TestFS.h
===
--- clang-tools-extra/clangd/unittests/TestFS.h
+++ clang-tools-extra/clangd/unittests/TestFS.h
@@ -34,12 +34,21 @@
 class MockFS : public ThreadsafeFS {
 public:
   IntrusiveRefCntPtr viewImpl() const override {
-return buildTestFS(Files, Timestamps);
+auto MemFS = buildTestFS(Files, Timestamps);
+if (!OverlayRealFileSystemForModules)
+  return MemFS;
+llvm::IntrusiveRefCntPtr OverlayFileSystem =
+new llvm::vfs::OverlayFileSystem(llvm::vfs::getRealFileSystem());
+OverlayFileSystem->pushOverlay(MemFS);
+return OverlayFileSystem;
   }
 
   // If relative paths are used, they are resolved with testPath().
   llvm::StringMap Files;
   llvm::StringMap Timestamps;
+  // If true, real file system will be used as fallback for the in-memory one.
+  // This is useful for testing module support.
+  bool OverlayRealFileSystemForModules = false;
 };
 
 // A Compilation database that returns a fixed set of compile flags.
Index: clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
===
--- clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
+++ clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
@@ -1520,6 +1520,31 @@
   EXPECT_THAT(Symbols,
   UnorderedElementsAre(AllOf(QName("X"), ForCodeCompletion(true;
 }
+
+// Regression test for a crash-bug we used to have.
+TEST_F(SymbolCollectorTest, UndefOfModuleMacro) {
+  auto TU = TestTU::withCode(R"cpp(#include "bar.h")cpp");
+  TU.AdditionalFiles["bar.h"] = R"cpp(
+#include "foo.h"
+#undef X
+)cpp";
+  TU.AdditionalFiles["foo.h"

[PATCH] D80525: [clangd] Fix crash-bug in preamble indexing when using modules.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/unittests/TestTU.h:71
+  // required for use of modules.
+  // FIXME: Intercept the write of implicitly build module to disk and update
+  // TestFS with that content to avoid using real file system in tests.

sammccall wrote:
> nit: I'm not sure the (V)FS should be involved at all in the final test setup.
> The way I see it, the module storage is basically a cache from (module name + 
> validity-stuff) ==> bytes, and *reads* could be abstracted just like writes.
> For tests this could just be an in-memory map of course, but I think even in 
> production we'd benefit a bit from not having to express this map through the 
> filesystem. (E.g the way it's versioned doesn't have much to do with the way 
> versioned filesystems are)
> 
> Only actionable thing for now is to maybe tweak this comment slightly, but 
> wanted to mention it.
Updated comment, is this better?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80525/new/

https://reviews.llvm.org/D80525

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85532: Correctly set CompilingPCH in PrecompilePreambleAction.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 284390.
adamcz added a comment.

assert() -> ASSERT_TRUE


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85532/new/

https://reviews.llvm.org/D85532

Files:
  clang-tools-extra/clangd/unittests/CMakeLists.txt
  clang-tools-extra/clangd/unittests/ModulesTests.cpp
  clang/lib/Frontend/PrecompiledPreamble.cpp
  clang/unittests/Frontend/ASTUnitTest.cpp

Index: clang/unittests/Frontend/ASTUnitTest.cpp
===
--- clang/unittests/Frontend/ASTUnitTest.cpp
+++ clang/unittests/Frontend/ASTUnitTest.cpp
@@ -13,6 +13,7 @@
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Frontend/PCHContainerOperations.h"
+#include "clang/Lex/HeaderSearch.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/ToolOutputFile.h"
@@ -111,4 +112,42 @@
 llvm::MemoryBuffer::MemoryBuffer_MMap);
 }
 
+TEST_F(ASTUnitTest, ModuleTextualHeader) {
+  llvm::IntrusiveRefCntPtr InMemoryFs =
+  new llvm::vfs::InMemoryFileSystem();
+  InMemoryFs->addFile("test.cpp", 0, llvm::MemoryBuffer::getMemBuffer(R"cpp(
+  #include "Textual.h"
+  void foo() {}
+)cpp"));
+  InMemoryFs->addFile("m.modulemap", 0, llvm::MemoryBuffer::getMemBuffer(R"cpp(
+  module M {
+module Textual {
+  textual header "Textual.h"
+}
+  }
+)cpp"));
+  InMemoryFs->addFile("Textual.h", 0, llvm::MemoryBuffer::getMemBuffer(R"cpp(
+  void foo();
+)cpp"));
+
+  const char *Args[] = {"clang", "test.cpp", "-fmodule-map-file=m.modulemap",
+"-fmodule-name=M"};
+  Diags = CompilerInstance::createDiagnostics(new DiagnosticOptions());
+  CInvok = createInvocationFromCommandLine(Args, Diags);
+  ASSERT_TRUE(CInvok);
+
+  FileManager *FileMgr = new FileManager(FileSystemOptions(), InMemoryFs);
+  PCHContainerOps = std::make_shared();
+
+  auto AU = ASTUnit::LoadFromCompilerInvocation(
+  CInvok, PCHContainerOps, Diags, FileMgr, false, CaptureDiagsKind::None, 1,
+  TU_Complete, false, false, false);
+  ASSERT_TRUE(AU);
+  auto File = AU->getFileManager().getFileRef("Textual.h", false, false);
+  ASSERT_TRUE(bool(File));
+  // Verify that we do not crash here.
+  EXPECT_TRUE(AU->getPreprocessor().getHeaderSearchInfo().getExistingFileInfo(
+  &File->getFileEntry()));
+}
+
 } // anonymous namespace
Index: clang/lib/Frontend/PrecompiledPreamble.cpp
===
--- clang/lib/Frontend/PrecompiledPreamble.cpp
+++ clang/lib/Frontend/PrecompiledPreamble.cpp
@@ -208,6 +208,11 @@
 Callbacks.AfterPCHEmitted(Writer);
   }
 
+  bool BeginSourceFileAction(CompilerInstance &CI) override {
+assert(CI.getLangOpts().CompilingPCH);
+return ASTFrontendAction::BeginSourceFileAction(CI);
+  }
+
   bool shouldEraseOutputFiles() override { return !hasEmittedPreamblePCH(); }
   bool hasCodeCompletionSupport() const override { return false; }
   bool hasASTFileSupport() const override { return false; }
@@ -396,6 +401,8 @@
   auto PreambleDepCollector = std::make_shared();
   Clang->addDependencyCollector(PreambleDepCollector);
 
+  Clang->getLangOpts().CompilingPCH = true;
+
   // Remap the main source file to the preamble buffer.
   StringRef MainFilePath = FrontendOpts.Inputs[0].getFile();
   auto PreambleInputBuffer = llvm::MemoryBuffer::getMemBufferCopy(
Index: clang-tools-extra/clangd/unittests/ModulesTests.cpp
===
--- /dev/null
+++ clang-tools-extra/clangd/unittests/ModulesTests.cpp
@@ -0,0 +1,44 @@
+//===-- ModulesTests.cpp  ---*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+
+#include "Annotations.h"
+#include "TestFS.h"
+#include "TestTU.h"
+#include "gmock/gmock.h"
+#include "gtest/gtest.h"
+
+#include 
+#include 
+
+namespace clang {
+namespace clangd {
+namespace {
+
+TEST(Modules, TextualIncludeInPreamble) {
+  TestTU TU = TestTU::withCode(R"cpp(
+#include "Textual.h"
+
+void foo() {}
+)cpp");
+  TU.ExtraArgs.push_back("-fmodule-name=M");
+  TU.ExtraArgs.push_back("-fmodule-map-file=m.modulemap");
+  TU.AdditionalFiles["Textual.h"] = "void foo();";
+  TU.AdditionalFiles["m.modulemap"] = R"modulemap(
+module M {
+  module Textual {
+textual header "Textual.h"
+  }
+}
+)modulemap";
+  // Test that we do not crash.
+  TU.index();
+}
+
+} // namespace
+} // namespace clangd
+} // namespace clang
Index: clang-tools-extra/clangd/unittests/CMakeLists.txt

[PATCH] D85532: Correctly set CompilingPCH in PrecompilePreambleAction.

[Adam Czachorowski via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rGe2d61ae57333: Correctly set CompilingPCH in 
PrecompilePreambleAction. (authored by adamcz).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85532/new/

https://reviews.llvm.org/D85532

Files:
  clang-tools-extra/clangd/unittests/CMakeLists.txt
  clang-tools-extra/clangd/unittests/ModulesTests.cpp
  clang/lib/Frontend/PrecompiledPreamble.cpp
  clang/unittests/Frontend/ASTUnitTest.cpp

Index: clang/unittests/Frontend/ASTUnitTest.cpp
===
--- clang/unittests/Frontend/ASTUnitTest.cpp
+++ clang/unittests/Frontend/ASTUnitTest.cpp
@@ -13,6 +13,7 @@
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Frontend/PCHContainerOperations.h"
+#include "clang/Lex/HeaderSearch.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/ToolOutputFile.h"
@@ -111,4 +112,42 @@
 llvm::MemoryBuffer::MemoryBuffer_MMap);
 }
 
+TEST_F(ASTUnitTest, ModuleTextualHeader) {
+  llvm::IntrusiveRefCntPtr InMemoryFs =
+  new llvm::vfs::InMemoryFileSystem();
+  InMemoryFs->addFile("test.cpp", 0, llvm::MemoryBuffer::getMemBuffer(R"cpp(
+  #include "Textual.h"
+  void foo() {}
+)cpp"));
+  InMemoryFs->addFile("m.modulemap", 0, llvm::MemoryBuffer::getMemBuffer(R"cpp(
+  module M {
+module Textual {
+  textual header "Textual.h"
+}
+  }
+)cpp"));
+  InMemoryFs->addFile("Textual.h", 0, llvm::MemoryBuffer::getMemBuffer(R"cpp(
+  void foo();
+)cpp"));
+
+  const char *Args[] = {"clang", "test.cpp", "-fmodule-map-file=m.modulemap",
+"-fmodule-name=M"};
+  Diags = CompilerInstance::createDiagnostics(new DiagnosticOptions());
+  CInvok = createInvocationFromCommandLine(Args, Diags);
+  ASSERT_TRUE(CInvok);
+
+  FileManager *FileMgr = new FileManager(FileSystemOptions(), InMemoryFs);
+  PCHContainerOps = std::make_shared();
+
+  auto AU = ASTUnit::LoadFromCompilerInvocation(
+  CInvok, PCHContainerOps, Diags, FileMgr, false, CaptureDiagsKind::None, 1,
+  TU_Complete, false, false, false);
+  ASSERT_TRUE(AU);
+  auto File = AU->getFileManager().getFileRef("Textual.h", false, false);
+  ASSERT_TRUE(bool(File));
+  // Verify that we do not crash here.
+  EXPECT_TRUE(AU->getPreprocessor().getHeaderSearchInfo().getExistingFileInfo(
+  &File->getFileEntry()));
+}
+
 } // anonymous namespace
Index: clang/lib/Frontend/PrecompiledPreamble.cpp
===
--- clang/lib/Frontend/PrecompiledPreamble.cpp
+++ clang/lib/Frontend/PrecompiledPreamble.cpp
@@ -208,6 +208,11 @@
 Callbacks.AfterPCHEmitted(Writer);
   }
 
+  bool BeginSourceFileAction(CompilerInstance &CI) override {
+assert(CI.getLangOpts().CompilingPCH);
+return ASTFrontendAction::BeginSourceFileAction(CI);
+  }
+
   bool shouldEraseOutputFiles() override { return !hasEmittedPreamblePCH(); }
   bool hasCodeCompletionSupport() const override { return false; }
   bool hasASTFileSupport() const override { return false; }
@@ -396,6 +401,8 @@
   auto PreambleDepCollector = std::make_shared();
   Clang->addDependencyCollector(PreambleDepCollector);
 
+  Clang->getLangOpts().CompilingPCH = true;
+
   // Remap the main source file to the preamble buffer.
   StringRef MainFilePath = FrontendOpts.Inputs[0].getFile();
   auto PreambleInputBuffer = llvm::MemoryBuffer::getMemBufferCopy(
Index: clang-tools-extra/clangd/unittests/ModulesTests.cpp
===
--- /dev/null
+++ clang-tools-extra/clangd/unittests/ModulesTests.cpp
@@ -0,0 +1,44 @@
+//===-- ModulesTests.cpp  ---*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+
+#include "Annotations.h"
+#include "TestFS.h"
+#include "TestTU.h"
+#include "gmock/gmock.h"
+#include "gtest/gtest.h"
+
+#include 
+#include 
+
+namespace clang {
+namespace clangd {
+namespace {
+
+TEST(Modules, TextualIncludeInPreamble) {
+  TestTU TU = TestTU::withCode(R"cpp(
+#include "Textual.h"
+
+void foo() {}
+)cpp");
+  TU.ExtraArgs.push_back("-fmodule-name=M");
+  TU.ExtraArgs.push_back("-fmodule-map-file=m.modulemap");
+  TU.AdditionalFiles["Textual.h"] = "void foo();";
+  TU.AdditionalFiles["m.modulemap"] = R"modulemap(
+module M {
+  module Textual {
+textual header "Textual.h"
+  }
+}
+)modulemap";
+  // Test that we do not crash.
+  TU.index();
+}
+
+} // namespace
+} // namespace cla

[PATCH] D80525: [clangd] Fix crash-bug in preamble indexing when using modules.

[Adam Czachorowski via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG4061d9e42cff: [clangd] Fix crash-bug in preamble indexing 
when using modules. (authored by adamcz).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80525/new/

https://reviews.llvm.org/D80525

Files:
  clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
  clang-tools-extra/clangd/unittests/TestFS.h
  clang-tools-extra/clangd/unittests/TestTU.cpp
  clang-tools-extra/clangd/unittests/TestTU.h
  clang/lib/Index/IndexingAction.cpp

Index: clang/lib/Index/IndexingAction.cpp
===
--- clang/lib/Index/IndexingAction.cpp
+++ clang/lib/Index/IndexingAction.cpp
@@ -165,11 +165,20 @@
 static void indexPreprocessorMacros(const Preprocessor &PP,
 IndexDataConsumer &DataConsumer) {
   for (const auto &M : PP.macros())
-if (MacroDirective *MD = M.second.getLatest())
+if (MacroDirective *MD = M.second.getLatest()) {
+  auto *MI = MD->getMacroInfo();
+  // When using modules, it may happen that we find #undef of a macro that
+  // was defined in another module. In such case, MI may be nullptr, since
+  // we only look for macro definitions in the current TU. In that case,
+  // there is nothing to index.
+  if (!MI)
+continue;
+
   DataConsumer.handleMacroOccurrence(
   M.first, MD->getMacroInfo(),
   static_cast(index::SymbolRole::Definition),
   MD->getLocation());
+}
 }
 
 void index::indexASTUnit(ASTUnit &Unit, IndexDataConsumer &DataConsumer,
Index: clang-tools-extra/clangd/unittests/TestTU.h
===
--- clang-tools-extra/clangd/unittests/TestTU.h
+++ clang-tools-extra/clangd/unittests/TestTU.h
@@ -66,6 +66,16 @@
   // Simulate a header guard of the header (using an #import directive).
   bool ImplicitHeaderGuard = true;
 
+  // Whether to use overlay the TestFS over the real filesystem. This is
+  // required for use of implicit modules.where the module file is written to
+  // disk and later read back.
+  // FIXME: Change the way reading/writing modules work to allow us to keep them
+  // in memory across multiple clang invocations, at least in tests, to
+  // eliminate the need for real file system here.
+  // Please avoid using this for things other than implicit modules. The plan is
+  // to eliminate this option some day.
+  bool OverlayRealFileSystemForModules = false;
+
   // By default, build() will report Error diagnostics as GTest errors.
   // Suppress this behavior by adding an 'error-ok' comment to the code.
   ParsedAST build() const;
Index: clang-tools-extra/clangd/unittests/TestTU.cpp
===
--- clang-tools-extra/clangd/unittests/TestTU.cpp
+++ clang-tools-extra/clangd/unittests/TestTU.cpp
@@ -54,6 +54,8 @@
   Inputs.CompileCommand.Filename = FullFilename;
   Inputs.CompileCommand.Directory = testRoot();
   Inputs.Contents = Code;
+  if (OverlayRealFileSystemForModules)
+FS.OverlayRealFileSystemForModules = true;
   Inputs.TFS = &FS;
   Inputs.Opts = ParseOptions();
   Inputs.Opts.BuildRecoveryAST = true;
Index: clang-tools-extra/clangd/unittests/TestFS.h
===
--- clang-tools-extra/clangd/unittests/TestFS.h
+++ clang-tools-extra/clangd/unittests/TestFS.h
@@ -34,12 +34,21 @@
 class MockFS : public ThreadsafeFS {
 public:
   IntrusiveRefCntPtr viewImpl() const override {
-return buildTestFS(Files, Timestamps);
+auto MemFS = buildTestFS(Files, Timestamps);
+if (!OverlayRealFileSystemForModules)
+  return MemFS;
+llvm::IntrusiveRefCntPtr OverlayFileSystem =
+new llvm::vfs::OverlayFileSystem(llvm::vfs::getRealFileSystem());
+OverlayFileSystem->pushOverlay(MemFS);
+return OverlayFileSystem;
   }
 
   // If relative paths are used, they are resolved with testPath().
   llvm::StringMap Files;
   llvm::StringMap Timestamps;
+  // If true, real file system will be used as fallback for the in-memory one.
+  // This is useful for testing module support.
+  bool OverlayRealFileSystemForModules = false;
 };
 
 // A Compilation database that returns a fixed set of compile flags.
Index: clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
===
--- clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
+++ clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
@@ -1520,6 +1520,31 @@
   EXPECT_THAT(Symbols,
   UnorderedElementsAre(AllOf(QName("X"), ForCodeCompletion(true;
 }
+
+// Regression test for a crash-bug we used to have.
+TEST_F(SymbolCollectorTest, UndefOfModuleMacro) {
+  auto TU = TestTU::withCode(R"cpp(#include "bar.h")cpp");
+  TU.AdditionalFiles["bar.h"] = R"cpp(
+#include "f

[PATCH] D85753: [clangd] Discard diagnostics from another SourceManager.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
Herald added subscribers: cfe-commits, usaxena95, kadircet, arphaman, jkorous.
Herald added a project: clang.
adamcz requested review of this revision.
Herald added subscribers: MaskRay, ilya-biryukov.

This can happen when building implicit modules, as demonstrated in test.
The CompilerInstance uses the same StoredDiags, but different
SourceManager. This used to crash clangd when it tried to relocate the
diagnostic to the main file, which, according to SourceManager from the
diagnostic, is a fake  file.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85753

Files:
  clang-tools-extra/clangd/Diagnostics.cpp
  clang-tools-extra/clangd/Diagnostics.h
  clang-tools-extra/clangd/unittests/ModulesTests.cpp


Index: clang-tools-extra/clangd/unittests/ModulesTests.cpp
===
--- clang-tools-extra/clangd/unittests/ModulesTests.cpp
+++ clang-tools-extra/clangd/unittests/ModulesTests.cpp
@@ -39,6 +39,34 @@
   TU.index();
 }
 
+TEST(Modules, Diagnostic) {
+  // Produce a diagnostic while building an implicit module. Use
+  // -fmodules-strick-decluse, but any non-silenced diagnostic will do.
+  TestTU TU = TestTU::withCode(R"cpp(
+/*error-ok*/
+#include "modular.h"
+
+void bar() {}
+)cpp");
+  TU.OverlayRealFileSystemForModules = true;
+  TU.ExtraArgs.push_back("-fmodule-map-file=" + testPath("m.modulemap"));
+  TU.ExtraArgs.push_back("-fmodules");
+  TU.ExtraArgs.push_back("-fimplicit-modules");
+  TU.ExtraArgs.push_back("-fmodules-strict-decluse");
+  TU.AdditionalFiles["modular.h"] = R"cpp(
+#include "non-modular.h"
+  )cpp";
+  TU.AdditionalFiles["non-modular.h"] = "";
+  TU.AdditionalFiles["m.modulemap"] = R"modulemap(
+module M {
+  header "modular.h"
+}
+)modulemap";
+
+  // Test that we do not crash.
+  TU.build();
+}
+
 } // namespace
 } // namespace clangd
 } // namespace clang
Index: clang-tools-extra/clangd/Diagnostics.h
===
--- clang-tools-extra/clangd/Diagnostics.h
+++ clang-tools-extra/clangd/Diagnostics.h
@@ -122,7 +122,8 @@
   // The ClangTidyContext populates Source and Name for clang-tidy diagnostics.
   std::vector take(const clang::tidy::ClangTidyContext *Tidy = nullptr);
 
-  void BeginSourceFile(const LangOptions &Opts, const Preprocessor *) override;
+  void BeginSourceFile(const LangOptions &Opts,
+   const Preprocessor *PP) override;
   void EndSourceFile() override;
   void HandleDiagnostic(DiagnosticsEngine::Level DiagLevel,
 const clang::Diagnostic &Info) override;
@@ -148,6 +149,7 @@
   llvm::Optional LastDiag;
   llvm::Optional LastDiagLoc; // Valid only when LastDiag is 
set.
   bool LastDiagOriginallyError = false;  // Valid only when LastDiag is 
set.
+  SourceManager *OrigSrcMgr = nullptr;
 
   llvm::DenseSet> IncludedErrorLocations;
   bool LastPrimaryDiagnosticWasSuppressed = false;
Index: clang-tools-extra/clangd/Diagnostics.cpp
===
--- clang-tools-extra/clangd/Diagnostics.cpp
+++ clang-tools-extra/clangd/Diagnostics.cpp
@@ -518,13 +518,17 @@
 }
 
 void StoreDiags::BeginSourceFile(const LangOptions &Opts,
- const Preprocessor *) {
+ const Preprocessor *PP) {
   LangOpts = Opts;
+  if (PP) {
+OrigSrcMgr = &PP->getSourceManager();
+  }
 }
 
 void StoreDiags::EndSourceFile() {
   flushLastDiag();
   LangOpts = None;
+  OrigSrcMgr = nullptr;
 }
 
 /// Sanitizes a piece for presenting it in a synthesized fix message. Ensures
@@ -560,6 +564,12 @@
 
 void StoreDiags::HandleDiagnostic(DiagnosticsEngine::Level DiagLevel,
   const clang::Diagnostic &Info) {
+  // If the diagnostic was generated for a different SourceManager, skip it.
+  // This can happen when using implicit modules.
+  if (OrigSrcMgr && Info.hasSourceManager() &&
+  OrigSrcMgr != &Info.getSourceManager())
+return;
+
   DiagnosticConsumer::HandleDiagnostic(DiagLevel, Info);
   bool OriginallyError =
   Info.getDiags()->getDiagnosticIDs()->isDefaultMappingAsError(


Index: clang-tools-extra/clangd/unittests/ModulesTests.cpp
===
--- clang-tools-extra/clangd/unittests/ModulesTests.cpp
+++ clang-tools-extra/clangd/unittests/ModulesTests.cpp
@@ -39,6 +39,34 @@
   TU.index();
 }
 
+TEST(Modules, Diagnostic) {
+  // Produce a diagnostic while building an implicit module. Use
+  // -fmodules-strick-decluse, but any non-silenced diagnostic will do.
+  TestTU TU = TestTU::withCode(R"cpp(
+/*error-ok*/
+#include "modular.h"
+
+void bar() {}
+)cpp");
+  TU.OverlayRealFileSystemForModules = true;
+  TU.ExtraArgs.push_back("-fmodule-map-file=" + testPath("m.modulemap"));
+  TU.ExtraArgs.push_back("-fmodules");
+  TU.ExtraArgs.push_back("-fimplic

[PATCH] D85753: [clangd] Discard diagnostics from another SourceManager.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 284811.
adamcz added a comment.

log dropped dianostic


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85753/new/

https://reviews.llvm.org/D85753

Files:
  clang-tools-extra/clangd/Diagnostics.cpp
  clang-tools-extra/clangd/Diagnostics.h
  clang-tools-extra/clangd/unittests/ModulesTests.cpp


Index: clang-tools-extra/clangd/unittests/ModulesTests.cpp
===
--- clang-tools-extra/clangd/unittests/ModulesTests.cpp
+++ clang-tools-extra/clangd/unittests/ModulesTests.cpp
@@ -39,6 +39,34 @@
   TU.index();
 }
 
+TEST(Modules, Diagnostic) {
+  // Produce a diagnostic while building an implicit module. Use
+  // -fmodules-strick-decluse, but any non-silenced diagnostic will do.
+  TestTU TU = TestTU::withCode(R"cpp(
+/*error-ok*/
+#include "modular.h"
+
+void bar() {}
+)cpp");
+  TU.OverlayRealFileSystemForModules = true;
+  TU.ExtraArgs.push_back("-fmodule-map-file=" + testPath("m.modulemap"));
+  TU.ExtraArgs.push_back("-fmodules");
+  TU.ExtraArgs.push_back("-fimplicit-modules");
+  TU.ExtraArgs.push_back("-fmodules-strict-decluse");
+  TU.AdditionalFiles["modular.h"] = R"cpp(
+#include "non-modular.h"
+  )cpp";
+  TU.AdditionalFiles["non-modular.h"] = "";
+  TU.AdditionalFiles["m.modulemap"] = R"modulemap(
+module M {
+  header "modular.h"
+}
+)modulemap";
+
+  // Test that we do not crash.
+  TU.build();
+}
+
 } // namespace
 } // namespace clangd
 } // namespace clang
Index: clang-tools-extra/clangd/Diagnostics.h
===
--- clang-tools-extra/clangd/Diagnostics.h
+++ clang-tools-extra/clangd/Diagnostics.h
@@ -122,7 +122,8 @@
   // The ClangTidyContext populates Source and Name for clang-tidy diagnostics.
   std::vector take(const clang::tidy::ClangTidyContext *Tidy = nullptr);
 
-  void BeginSourceFile(const LangOptions &Opts, const Preprocessor *) override;
+  void BeginSourceFile(const LangOptions &Opts,
+   const Preprocessor *PP) override;
   void EndSourceFile() override;
   void HandleDiagnostic(DiagnosticsEngine::Level DiagLevel,
 const clang::Diagnostic &Info) override;
@@ -148,6 +149,7 @@
   llvm::Optional LastDiag;
   llvm::Optional LastDiagLoc; // Valid only when LastDiag is 
set.
   bool LastDiagOriginallyError = false;  // Valid only when LastDiag is 
set.
+  SourceManager *OrigSrcMgr = nullptr;
 
   llvm::DenseSet> IncludedErrorLocations;
   bool LastPrimaryDiagnosticWasSuppressed = false;
Index: clang-tools-extra/clangd/Diagnostics.cpp
===
--- clang-tools-extra/clangd/Diagnostics.cpp
+++ clang-tools-extra/clangd/Diagnostics.cpp
@@ -518,13 +518,17 @@
 }
 
 void StoreDiags::BeginSourceFile(const LangOptions &Opts,
- const Preprocessor *) {
+ const Preprocessor *PP) {
   LangOpts = Opts;
+  if (PP) {
+OrigSrcMgr = &PP->getSourceManager();
+  }
 }
 
 void StoreDiags::EndSourceFile() {
   flushLastDiag();
   LangOpts = None;
+  OrigSrcMgr = nullptr;
 }
 
 /// Sanitizes a piece for presenting it in a synthesized fix message. Ensures
@@ -560,6 +564,14 @@
 
 void StoreDiags::HandleDiagnostic(DiagnosticsEngine::Level DiagLevel,
   const clang::Diagnostic &Info) {
+  // If the diagnostic was generated for a different SourceManager, skip it.
+  // This can happen when using implicit modules.
+  if (OrigSrcMgr && Info.hasSourceManager() &&
+  OrigSrcMgr != &Info.getSourceManager()) {
+IgnoreDiagnostics::log(DiagLevel, Info);
+return;
+  }
+
   DiagnosticConsumer::HandleDiagnostic(DiagLevel, Info);
   bool OriginallyError =
   Info.getDiags()->getDiagnosticIDs()->isDefaultMappingAsError(


Index: clang-tools-extra/clangd/unittests/ModulesTests.cpp
===
--- clang-tools-extra/clangd/unittests/ModulesTests.cpp
+++ clang-tools-extra/clangd/unittests/ModulesTests.cpp
@@ -39,6 +39,34 @@
   TU.index();
 }
 
+TEST(Modules, Diagnostic) {
+  // Produce a diagnostic while building an implicit module. Use
+  // -fmodules-strick-decluse, but any non-silenced diagnostic will do.
+  TestTU TU = TestTU::withCode(R"cpp(
+/*error-ok*/
+#include "modular.h"
+
+void bar() {}
+)cpp");
+  TU.OverlayRealFileSystemForModules = true;
+  TU.ExtraArgs.push_back("-fmodule-map-file=" + testPath("m.modulemap"));
+  TU.ExtraArgs.push_back("-fmodules");
+  TU.ExtraArgs.push_back("-fimplicit-modules");
+  TU.ExtraArgs.push_back("-fmodules-strict-decluse");
+  TU.AdditionalFiles["modular.h"] = R"cpp(
+#include "non-modular.h"
+  )cpp";
+  TU.AdditionalFiles["non-modular.h"] = "";
+  TU.AdditionalFiles["m.modulemap"] = R"modulemap(
+module M {
+  header "modular.h"
+}
+)modulemap";
+
+  // Test tha

[PATCH] D85753: [clangd] Discard diagnostics from another SourceManager.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added a reviewer: sammccall.
adamcz added a comment.

Hey Sam. What's your opinion on this?

The options we have are:

1. Drop the diagnostics, like this change is doing
2. Relocate the diagnostic to the beginning of the "real" main file, like we do 
with SourceManager-less diagnostics
3. Try to find the place where we import the module and relocate the diagnostic 
there

Right now this change is just an example of approach 1.  Not necessarily meant 
to be submitted as-is, since it silently drops possibly valuable diagnostics, 
although it does prevent a crash.

Option 3 sounds best, but it's very tricky. At the point HandleDiagnostic() is 
called, the module is not imported yet, so we would have to either delay the 
relocation until later or do some tricks with Preprocessor object (not sure how 
that would work).

Let me know what you think about this.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85753/new/

https://reviews.llvm.org/D85753

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85753: [clangd] Discard diagnostics from another SourceManager.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/Diagnostics.cpp:209
   // Update diag to point at include inside main file.
   D.File = SM.getFileEntryForID(SM.getMainFileID())->getName().str();
   D.Range = std::move(R);

kadircet wrote:
> can't we rather record the mainfilename once and make use of it here?
> i believe the rest of the logic operates on the local sourcelocations/files 
> associated with the sourcemanager inside the diagnostic in process.
getMainFileRange() doesn't.

We can easily get main file from the original source manager or record it in 
BeginSourceFile. The thing is that it's not easy to find a place in the main 
file to put this diagnostic at. We can, of course, put it at the top of the 
file, but normally we try to find the #include location in the main file. 
That's done in the getMainFileRange() above and it mixes the location of the 
diagnostic (which is for the new SourceManager) and the main file buffer (which 
is for the OrigSrcMgr).

I'm a bit worried that if we put a diagnostic relating to #include at the top 
of the file, where another #include may be, it will become very confusing.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85753/new/

https://reviews.llvm.org/D85753

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85753: [clangd] Discard diagnostics from another SourceManager.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/Diagnostics.cpp:209
   // Update diag to point at include inside main file.
   D.File = SM.getFileEntryForID(SM.getMainFileID())->getName().str();
   D.Range = std::move(R);

kadircet wrote:
> adamcz wrote:
> > kadircet wrote:
> > > can't we rather record the mainfilename once and make use of it here?
> > > i believe the rest of the logic operates on the local 
> > > sourcelocations/files associated with the sourcemanager inside the 
> > > diagnostic in process.
> > getMainFileRange() doesn't.
> > 
> > We can easily get main file from the original source manager or record it 
> > in BeginSourceFile. The thing is that it's not easy to find a place in the 
> > main file to put this diagnostic at. We can, of course, put it at the top 
> > of the file, but normally we try to find the #include location in the main 
> > file. That's done in the getMainFileRange() above and it mixes the location 
> > of the diagnostic (which is for the new SourceManager) and the main file 
> > buffer (which is for the OrigSrcMgr).
> > 
> > I'm a bit worried that if we put a diagnostic relating to #include at the 
> > top of the file, where another #include may be, it will become very 
> > confusing.
> > getMainFileRange() doesn't.
> 
> ah right i forgot about that bit :/
> 
> > I'm a bit worried that if we put a diagnostic relating to #include at the 
> > top of the file, where another #include may be, it will become very 
> > confusing.
> 
> I totally agree.
> 
> A crazy idea, we can register a callback to PP on `BeginSourceFile` to track 
> file changes, than whenever we see a diagnostic from a different 
> sourcemanager, we can map it back to last position a filechange was 
> triggered. Ofc, this goes with the assumption that we see a filechanged event 
> on `#include "modular.h"` line, but I guess we should at least get an 
> includedirective callback or something.
Right, that's a good idea, thanks. Sam suggested something like that in 
parallel, during in-person discussion too. I'll look into that in the near 
future.

There's the other issue of only seeing this diagnostics when a module is built, 
but not when a module was loaded from cache, so we will not be consistent 
unless we serialize them somehow.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85753/new/

https://reviews.llvm.org/D85753

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80525: [clangd] Fix crash-bug in preamble indexing when using modules.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added a comment.

This is quite strange. I can't reproduce this with gcc 7.5.0, 9.3.0 or 10.0.1. 
Which version are you using?

There are bugs around clangd + modules + diagnostics. I'm fixing one in 
https://reviews.llvm.org/D85753 (not in it's final form yet), but it doesn't 
seem to be the same issue (can't be sure, but I think this is using the right 
SourceManager). This test is not supposed to generate any diagnostics, so 
that's the really surprising bit. Are you building from clean HEAD, or do you 
have any local modifications that could generate a extra diagnostics in this 
test?

The other option is that the failure is caused by module_cache_path. This test, 
unlike other clangd tests, needs real file system (to write the .pcm file and 
then read it back) and perhaps that doesn't work in your jenkins setup.

Are there any special options or things unique to your setup I could try out to 
reproduce this?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80525/new/

https://reviews.llvm.org/D80525

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80525: [clangd] Fix crash-bug in preamble indexing when using modules.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added a comment.

In D80525#2215755 , @bjope wrote:

> In D80525#2215641 , @adamcz wrote:
>
>> This is quite strange. I can't reproduce this with gcc 7.5.0, 9.3.0 or 
>> 10.0.1. Which version are you using?
>>
>> There are bugs around clangd + modules + diagnostics. I'm fixing one in 
>> https://reviews.llvm.org/D85753 (not in it's final form yet), but it doesn't 
>> seem to be the same issue (can't be sure, but I think this is using the 
>> right SourceManager). This test is not supposed to generate any diagnostics, 
>> so that's the really surprising bit. Are you building from clean HEAD, or do 
>> you have any local modifications that could generate a extra diagnostics in 
>> this test?
>>
>> The other option is that the failure is caused by module_cache_path. This 
>> test, unlike other clangd tests, needs real file system (to write the .pcm 
>> file and then read it back) and perhaps that doesn't work in your jenkins 
>> setup.
>>
>> Are there any special options or things unique to your setup I could try out 
>> to reproduce this?
>
> We actually have had problems reproducing it outside jenkins ourselves as 
> well. That is kind of why we were a bit clueless about what to do and what 
> might be causing it.
>
> Are those `VFS: failed to set CWD to /clangd-test: No such file or directory` 
> outputs expected?

Yes, they are harmless. InMemoryFS uses fake paths like /clangd-test, but the 
real file system can't follow that. The real file system is supposed to be used 
for module cache only.

If you can't reproduce this locally, but can on Jenkins that makes me think 
it's definitely something with filesystem. I suppose running strace on jenkins 
to log all FS access is not easy?

Maybe try to figure out what the effective ModuleCachePath is and whether that 
makes sense (i.e. is accessible) in your Jenkins setup? I can't think of a way 
to make clangd test log it, but you can just:

  llvm::errs() << ModuleCachePath << "\n"; llvm::errs().flush();

as last line of setModuleCachePath() in clang/lib/clang/Lex/HeaderSearch.h 
(around line 325) to see what path it's using.  There's probably a much smarter 
way to do this, but I don't know it.

This is still a random guess that it's something to do with file system.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80525/new/

https://reviews.llvm.org/D80525

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80525: [clangd] Fix crash-bug in preamble indexing when using modules.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added a comment.

Looks like it's an issue with not clearing the module cache. I'll revert this 
change to fix the build, then re-submit with proper fix.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80525/new/

https://reviews.llvm.org/D80525

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85907: [clangd] Revert "[clangd] Fix crash-bug in preamble indexing when using modules."

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
adamcz added a reviewer: sammccall.
Herald added subscribers: cfe-commits, usaxena95, kadircet, arphaman, jkorous.
Herald added a project: clang.
adamcz requested review of this revision.
Herald added subscribers: MaskRay, ilya-biryukov.

This reverts commit 4061d9e42cff621462931ac7df9666806c77a237 
.
Tests are failing in some configuration, likely due to not cleaning up
module cache path before running the test.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85907

Files:
  clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
  clang-tools-extra/clangd/unittests/TestFS.h
  clang-tools-extra/clangd/unittests/TestTU.cpp
  clang-tools-extra/clangd/unittests/TestTU.h
  clang/lib/Index/IndexingAction.cpp

Index: clang/lib/Index/IndexingAction.cpp
===
--- clang/lib/Index/IndexingAction.cpp
+++ clang/lib/Index/IndexingAction.cpp
@@ -165,20 +165,11 @@
 static void indexPreprocessorMacros(const Preprocessor &PP,
 IndexDataConsumer &DataConsumer) {
   for (const auto &M : PP.macros())
-if (MacroDirective *MD = M.second.getLatest()) {
-  auto *MI = MD->getMacroInfo();
-  // When using modules, it may happen that we find #undef of a macro that
-  // was defined in another module. In such case, MI may be nullptr, since
-  // we only look for macro definitions in the current TU. In that case,
-  // there is nothing to index.
-  if (!MI)
-continue;
-
+if (MacroDirective *MD = M.second.getLatest())
   DataConsumer.handleMacroOccurrence(
   M.first, MD->getMacroInfo(),
   static_cast(index::SymbolRole::Definition),
   MD->getLocation());
-}
 }
 
 void index::indexASTUnit(ASTUnit &Unit, IndexDataConsumer &DataConsumer,
Index: clang-tools-extra/clangd/unittests/TestTU.h
===
--- clang-tools-extra/clangd/unittests/TestTU.h
+++ clang-tools-extra/clangd/unittests/TestTU.h
@@ -66,16 +66,6 @@
   // Simulate a header guard of the header (using an #import directive).
   bool ImplicitHeaderGuard = true;
 
-  // Whether to use overlay the TestFS over the real filesystem. This is
-  // required for use of implicit modules.where the module file is written to
-  // disk and later read back.
-  // FIXME: Change the way reading/writing modules work to allow us to keep them
-  // in memory across multiple clang invocations, at least in tests, to
-  // eliminate the need for real file system here.
-  // Please avoid using this for things other than implicit modules. The plan is
-  // to eliminate this option some day.
-  bool OverlayRealFileSystemForModules = false;
-
   // By default, build() will report Error diagnostics as GTest errors.
   // Suppress this behavior by adding an 'error-ok' comment to the code.
   ParsedAST build() const;
Index: clang-tools-extra/clangd/unittests/TestTU.cpp
===
--- clang-tools-extra/clangd/unittests/TestTU.cpp
+++ clang-tools-extra/clangd/unittests/TestTU.cpp
@@ -54,8 +54,6 @@
   Inputs.CompileCommand.Filename = FullFilename;
   Inputs.CompileCommand.Directory = testRoot();
   Inputs.Contents = Code;
-  if (OverlayRealFileSystemForModules)
-FS.OverlayRealFileSystemForModules = true;
   Inputs.TFS = &FS;
   Inputs.Opts = ParseOptions();
   Inputs.Opts.BuildRecoveryAST = true;
Index: clang-tools-extra/clangd/unittests/TestFS.h
===
--- clang-tools-extra/clangd/unittests/TestFS.h
+++ clang-tools-extra/clangd/unittests/TestFS.h
@@ -34,21 +34,12 @@
 class MockFS : public ThreadsafeFS {
 public:
   IntrusiveRefCntPtr viewImpl() const override {
-auto MemFS = buildTestFS(Files, Timestamps);
-if (!OverlayRealFileSystemForModules)
-  return MemFS;
-llvm::IntrusiveRefCntPtr OverlayFileSystem =
-new llvm::vfs::OverlayFileSystem(llvm::vfs::getRealFileSystem());
-OverlayFileSystem->pushOverlay(MemFS);
-return OverlayFileSystem;
+return buildTestFS(Files, Timestamps);
   }
 
   // If relative paths are used, they are resolved with testPath().
   llvm::StringMap Files;
   llvm::StringMap Timestamps;
-  // If true, real file system will be used as fallback for the in-memory one.
-  // This is useful for testing module support.
-  bool OverlayRealFileSystemForModules = false;
 };
 
 // A Compilation database that returns a fixed set of compile flags.
Index: clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
===
--- clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
+++ clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
@@ -1610,31 +1610,6 @@
   EXPECT_THAT(Symbols,
   UnorderedElementsAre(AllOf(QName("X

[PATCH] D85907: [clangd] Revert "[clangd] Fix crash-bug in preamble indexing when using modules."

[Adam Czachorowski via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rG73f0772c0baf: [clangd] Revert "[clangd] Fix crash-bug 
in preamble indexing when using modules. (authored by adamcz).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85907/new/

https://reviews.llvm.org/D85907

Files:
  clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
  clang-tools-extra/clangd/unittests/TestFS.h
  clang-tools-extra/clangd/unittests/TestTU.cpp
  clang-tools-extra/clangd/unittests/TestTU.h
  clang/lib/Index/IndexingAction.cpp

Index: clang/lib/Index/IndexingAction.cpp
===
--- clang/lib/Index/IndexingAction.cpp
+++ clang/lib/Index/IndexingAction.cpp
@@ -165,20 +165,11 @@
 static void indexPreprocessorMacros(const Preprocessor &PP,
 IndexDataConsumer &DataConsumer) {
   for (const auto &M : PP.macros())
-if (MacroDirective *MD = M.second.getLatest()) {
-  auto *MI = MD->getMacroInfo();
-  // When using modules, it may happen that we find #undef of a macro that
-  // was defined in another module. In such case, MI may be nullptr, since
-  // we only look for macro definitions in the current TU. In that case,
-  // there is nothing to index.
-  if (!MI)
-continue;
-
+if (MacroDirective *MD = M.second.getLatest())
   DataConsumer.handleMacroOccurrence(
   M.first, MD->getMacroInfo(),
   static_cast(index::SymbolRole::Definition),
   MD->getLocation());
-}
 }
 
 void index::indexASTUnit(ASTUnit &Unit, IndexDataConsumer &DataConsumer,
Index: clang-tools-extra/clangd/unittests/TestTU.h
===
--- clang-tools-extra/clangd/unittests/TestTU.h
+++ clang-tools-extra/clangd/unittests/TestTU.h
@@ -66,16 +66,6 @@
   // Simulate a header guard of the header (using an #import directive).
   bool ImplicitHeaderGuard = true;
 
-  // Whether to use overlay the TestFS over the real filesystem. This is
-  // required for use of implicit modules.where the module file is written to
-  // disk and later read back.
-  // FIXME: Change the way reading/writing modules work to allow us to keep them
-  // in memory across multiple clang invocations, at least in tests, to
-  // eliminate the need for real file system here.
-  // Please avoid using this for things other than implicit modules. The plan is
-  // to eliminate this option some day.
-  bool OverlayRealFileSystemForModules = false;
-
   // By default, build() will report Error diagnostics as GTest errors.
   // Suppress this behavior by adding an 'error-ok' comment to the code.
   ParsedAST build() const;
Index: clang-tools-extra/clangd/unittests/TestTU.cpp
===
--- clang-tools-extra/clangd/unittests/TestTU.cpp
+++ clang-tools-extra/clangd/unittests/TestTU.cpp
@@ -54,8 +54,6 @@
   Inputs.CompileCommand.Filename = FullFilename;
   Inputs.CompileCommand.Directory = testRoot();
   Inputs.Contents = Code;
-  if (OverlayRealFileSystemForModules)
-FS.OverlayRealFileSystemForModules = true;
   Inputs.TFS = &FS;
   Inputs.Opts = ParseOptions();
   Inputs.Opts.BuildRecoveryAST = true;
Index: clang-tools-extra/clangd/unittests/TestFS.h
===
--- clang-tools-extra/clangd/unittests/TestFS.h
+++ clang-tools-extra/clangd/unittests/TestFS.h
@@ -34,21 +34,12 @@
 class MockFS : public ThreadsafeFS {
 public:
   IntrusiveRefCntPtr viewImpl() const override {
-auto MemFS = buildTestFS(Files, Timestamps);
-if (!OverlayRealFileSystemForModules)
-  return MemFS;
-llvm::IntrusiveRefCntPtr OverlayFileSystem =
-new llvm::vfs::OverlayFileSystem(llvm::vfs::getRealFileSystem());
-OverlayFileSystem->pushOverlay(MemFS);
-return OverlayFileSystem;
+return buildTestFS(Files, Timestamps);
   }
 
   // If relative paths are used, they are resolved with testPath().
   llvm::StringMap Files;
   llvm::StringMap Timestamps;
-  // If true, real file system will be used as fallback for the in-memory one.
-  // This is useful for testing module support.
-  bool OverlayRealFileSystemForModules = false;
 };
 
 // A Compilation database that returns a fixed set of compile flags.
Index: clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
===
--- clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
+++ clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
@@ -1610,31 +1610,6 @@
   EXPECT_THAT(Symbols,
   UnorderedElementsAre(AllOf(QName("X"), ForCodeCompletion(true;
 }
-
-// Regression test for a crash-bug we used to have.
-TEST_F(SymbolCollectorTest, UndefOfModuleMacro) {
-  auto TU = TestTU::withCode(R"cpp(#includ

[PATCH] D85883: [clangd] Add ClangdServer::customAction() extension point

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added a comment.

LGTM


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85883/new/

https://reviews.llvm.org/D85883

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85923: [clangd] Fix crash-bug in preamble indexing when using modules.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
adamcz added a reviewer: sammccall.
Herald added subscribers: cfe-commits, usaxena95, kadircet, arphaman, jkorous.
Herald added a project: clang.
adamcz requested review of this revision.
Herald added subscribers: MaskRay, ilya-biryukov.

When preamble contains #undef, indexing code finds the matching #define
and uses that during indexing. However, it would only look for local
definitions. If the macro was defined in a module, MacroInfo
would be nullptr and clangd would crash.

This change makes clangd ignore any #undef without a matching #define
inside the same TU.

The indexing of macros happens for preamble only, so then #undef must be
in the preamble, which is why we need two .h files in a test.

Note that clangd is currently not ready for module support, but this
brings us one step closer.

This was previously attempted in
4061d9e42cff621462931ac7df9666806c77a237 
, but had 
to be reverted due to
broken test. This version fixes that test-only bug by setting a custom module
cache path to avoid re-use of modules across test invocations.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85923

Files:
  clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp
  clang-tools-extra/clangd/unittests/TestFS.h
  clang-tools-extra/clangd/unittests/TestTU.cpp
  clang-tools-extra/clangd/unittests/TestTU.h
  clang/lib/Index/IndexingAction.cpp

Index: clang/lib/Index/IndexingAction.cpp
===
--- clang/lib/Index/IndexingAction.cpp
+++ clang/lib/Index/IndexingAction.cpp
@@ -165,11 +165,20 @@
 static void indexPreprocessorMacros(const Preprocessor &PP,
 IndexDataConsumer &DataConsumer) {
   for (const auto &M : PP.macros())
-if (MacroDirective *MD = M.second.getLatest())
+if (MacroDirective *MD = M.second.getLatest()) {
+  auto *MI = MD->getMacroInfo();
+  // When using modules, it may happen that we find #undef of a macro that
+  // was defined in another module. In such case, MI may be nullptr, since
+  // we only look for macro definitions in the current TU. In that case,
+  // there is nothing to index.
+  if (!MI)
+continue;
+
   DataConsumer.handleMacroOccurrence(
   M.first, MD->getMacroInfo(),
   static_cast(index::SymbolRole::Definition),
   MD->getLocation());
+}
 }
 
 void index::indexASTUnit(ASTUnit &Unit, IndexDataConsumer &DataConsumer,
Index: clang-tools-extra/clangd/unittests/TestTU.h
===
--- clang-tools-extra/clangd/unittests/TestTU.h
+++ clang-tools-extra/clangd/unittests/TestTU.h
@@ -66,6 +66,16 @@
   // Simulate a header guard of the header (using an #import directive).
   bool ImplicitHeaderGuard = true;
 
+  // Whether to use overlay the TestFS over the real filesystem. This is
+  // required for use of implicit modules.where the module file is written to
+  // disk and later read back.
+  // FIXME: Change the way reading/writing modules work to allow us to keep them
+  // in memory across multiple clang invocations, at least in tests, to
+  // eliminate the need for real file system here.
+  // Please avoid using this for things other than implicit modules. The plan is
+  // to eliminate this option some day.
+  bool OverlayRealFileSystemForModules = false;
+
   // By default, build() will report Error diagnostics as GTest errors.
   // Suppress this behavior by adding an 'error-ok' comment to the code.
   ParsedAST build() const;
Index: clang-tools-extra/clangd/unittests/TestTU.cpp
===
--- clang-tools-extra/clangd/unittests/TestTU.cpp
+++ clang-tools-extra/clangd/unittests/TestTU.cpp
@@ -16,6 +16,7 @@
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Frontend/Utils.h"
+#include "llvm/ADT/ScopeExit.h"
 
 namespace clang {
 namespace clangd {
@@ -54,6 +55,8 @@
   Inputs.CompileCommand.Filename = FullFilename;
   Inputs.CompileCommand.Directory = testRoot();
   Inputs.Contents = Code;
+  if (OverlayRealFileSystemForModules)
+FS.OverlayRealFileSystemForModules = true;
   Inputs.TFS = &FS;
   Inputs.Opts = ParseOptions();
   Inputs.Opts.BuildRecoveryAST = true;
@@ -66,12 +69,31 @@
   return Inputs;
 }
 
+void initializeModuleCache(CompilerInvocation &CI) {
+  llvm::SmallString<128> ModuleCachePath;
+  ASSERT_FALSE(
+  llvm::sys::fs::createUniqueDirectory("module-cache", ModuleCachePath));
+  CI.getHeaderSearchOpts().ModuleCachePath = ModuleCachePath.c_str();
+  llvm::errs() << "MC: " << ModuleCachePath << "\n";
+  llvm::errs().flush();
+}
+
+void deleteModuleCache(const std::string ModuleCachePath) {
+  if (!ModuleCachePath.empty()) {
+ASSERT_FALSE(llvm::sys::fs::remove_directories(ModuleCachePath));
+  }
+}
+
 std::shared_ptr T

[PATCH] D85753: [clangd] Discard diagnostics from another SourceManager.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/Diagnostics.cpp:568
+  // If the diagnostic was generated for a different SourceManager, skip it.
+  // This can happen when using implicit modules.
+  if (OrigSrcMgr && Info.hasSourceManager() &&

sammccall wrote:
> sammccall wrote:
> > maybe a FIXME: errors from implicitly built modules should be surfaced 
> > somehow (but then must also be surfaced when the module was cached)
> Can we be a bit more specific: this happens when an #include causes a module 
> to be implicitly built, using a separate SourceManager.
It's not just #include, could be "import". But yes, we can be more specific. 
Done.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85753/new/

https://reviews.llvm.org/D85753

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85753: [clangd] Discard diagnostics from another SourceManager.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 285961.
adamcz marked 3 inline comments as done.
adamcz added a comment.

addressed review comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85753/new/

https://reviews.llvm.org/D85753

Files:
  clang-tools-extra/clangd/Diagnostics.cpp
  clang-tools-extra/clangd/Diagnostics.h
  clang-tools-extra/clangd/unittests/ModulesTests.cpp

Index: clang-tools-extra/clangd/unittests/ModulesTests.cpp
===
--- clang-tools-extra/clangd/unittests/ModulesTests.cpp
+++ clang-tools-extra/clangd/unittests/ModulesTests.cpp
@@ -39,6 +39,34 @@
   TU.index();
 }
 
+TEST(Modules, Diagnostic) {
+  // Produce a diagnostic while building an implicit module. Use
+  // -fmodules-strict-decluse, but any non-silenced diagnostic will do.
+  TestTU TU = TestTU::withCode(R"cpp(
+/*error-ok*/
+#include "modular.h"
+
+void bar() {}
+)cpp");
+  TU.OverlayRealFileSystemForModules = true;
+  TU.ExtraArgs.push_back("-fmodule-map-file=" + testPath("m.modulemap"));
+  TU.ExtraArgs.push_back("-fmodules");
+  TU.ExtraArgs.push_back("-fimplicit-modules");
+  TU.ExtraArgs.push_back("-fmodules-strict-decluse");
+  TU.AdditionalFiles["modular.h"] = R"cpp(
+#include "non-modular.h"
+  )cpp";
+  TU.AdditionalFiles["non-modular.h"] = "";
+  TU.AdditionalFiles["m.modulemap"] = R"modulemap(
+module M {
+  header "modular.h"
+}
+)modulemap";
+
+  // Test that we do not crash.
+  TU.build();
+}
+
 } // namespace
 } // namespace clangd
 } // namespace clang
Index: clang-tools-extra/clangd/Diagnostics.h
===
--- clang-tools-extra/clangd/Diagnostics.h
+++ clang-tools-extra/clangd/Diagnostics.h
@@ -122,7 +122,8 @@
   // The ClangTidyContext populates Source and Name for clang-tidy diagnostics.
   std::vector take(const clang::tidy::ClangTidyContext *Tidy = nullptr);
 
-  void BeginSourceFile(const LangOptions &Opts, const Preprocessor *) override;
+  void BeginSourceFile(const LangOptions &Opts,
+   const Preprocessor *PP) override;
   void EndSourceFile() override;
   void HandleDiagnostic(DiagnosticsEngine::Level DiagLevel,
 const clang::Diagnostic &Info) override;
@@ -148,6 +149,7 @@
   llvm::Optional LastDiag;
   llvm::Optional LastDiagLoc; // Valid only when LastDiag is set.
   bool LastDiagOriginallyError = false;  // Valid only when LastDiag is set.
+  SourceManager *OrigSrcMgr = nullptr;
 
   llvm::DenseSet> IncludedErrorLocations;
   bool LastPrimaryDiagnosticWasSuppressed = false;
Index: clang-tools-extra/clangd/Diagnostics.cpp
===
--- clang-tools-extra/clangd/Diagnostics.cpp
+++ clang-tools-extra/clangd/Diagnostics.cpp
@@ -518,13 +518,17 @@
 }
 
 void StoreDiags::BeginSourceFile(const LangOptions &Opts,
- const Preprocessor *) {
+ const Preprocessor *PP) {
   LangOpts = Opts;
+  if (PP) {
+OrigSrcMgr = &PP->getSourceManager();
+  }
 }
 
 void StoreDiags::EndSourceFile() {
   flushLastDiag();
   LangOpts = None;
+  OrigSrcMgr = nullptr;
 }
 
 /// Sanitizes a piece for presenting it in a synthesized fix message. Ensures
@@ -560,6 +564,16 @@
 
 void StoreDiags::HandleDiagnostic(DiagnosticsEngine::Level DiagLevel,
   const clang::Diagnostic &Info) {
+  // If the diagnostic was generated for a different SourceManager, skip it.
+  // This happens when a module is imported and needs to be implicitly built.
+  // The compilation of that module will use the same StoreDiags, but different
+  // SourceManager.
+  if (OrigSrcMgr && Info.hasSourceManager() &&
+  OrigSrcMgr != &Info.getSourceManager()) {
+IgnoreDiagnostics::log(DiagLevel, Info);
+return;
+  }
+
   DiagnosticConsumer::HandleDiagnostic(DiagLevel, Info);
   bool OriginallyError =
   Info.getDiags()->getDiagnosticIDs()->isDefaultMappingAsError(
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D86069: [clang] When loading preamble from AST file, re-export modules in Sema.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
Herald added subscribers: cfe-commits, usaxena95, kadircet, arphaman, jkorous.
Herald added a project: clang.
adamcz requested review of this revision.

This addresses a FIXME in ASTReader.

Modules were already re-exported for Preprocessor, but not for Sema.
The result was that, with -fmodules-local-submodule-visibility, all AST
nodes belonging to a module that was loaded in a premable where not
accesible from the main part of the file and a diagnostic recommending
importing those modules would be generated.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D86069

Files:
  clang-tools-extra/clangd/unittests/ModulesTests.cpp
  clang/include/clang/Sema/Sema.h
  clang/lib/Serialization/ASTReader.cpp
  clang/test/PCH/Inputs/modules/Foo.h
  clang/test/PCH/preamble-modules.cpp

Index: clang/test/PCH/preamble-modules.cpp
===
--- /dev/null
+++ clang/test/PCH/preamble-modules.cpp
@@ -0,0 +1,15 @@
+// Check that modules included in the preamble remain visible to the rest of the
+// file.
+
+// RUN: rm -rf %t.mcp
+// RUN: %clang_cc1 -emit-pch -o %t.pch %s -fmodules -fmodule-map-file=%S/Inputs/modules/module.modulemap -fmodules-local-submodule-visibility -fmodules-cache-path=%t.mcp
+// RUN: %clang_cc1 -include-pch %t.pch %s -fmodules -fmodule-map-file=%S/Inputs/modules/module.modulemap -fmodules-local-submodule-visibility -fmodules-cache-path=%t.mcp
+
+#ifndef MAIN_FILE
+#define MAIN_FILE
+// Premable section.
+#include "Inputs/modules/Foo.h"
+#else
+// Main section.
+MyType foo;
+#endif
Index: clang/test/PCH/Inputs/modules/Foo.h
===
--- clang/test/PCH/Inputs/modules/Foo.h
+++ clang/test/PCH/Inputs/modules/Foo.h
@@ -1 +1,3 @@
 void make_foo(void);
+
+typedef int MyType;
Index: clang/lib/Serialization/ASTReader.cpp
===
--- clang/lib/Serialization/ASTReader.cpp
+++ clang/lib/Serialization/ASTReader.cpp
@@ -4987,10 +4987,8 @@
 /*ImportLoc=*/Import.ImportLoc);
   if (Import.ImportLoc.isValid())
 PP.makeModuleVisible(Imported, Import.ImportLoc);
-  // FIXME: should we tell Sema to make the module visible too?
 }
   }
-  ImportedModules.clear();
 }
 
 void ASTReader::finalizeForWriting() {
@@ -7942,6 +7940,15 @@
   SemaObj->FpPragmaStack.CurrentPragmaLocation = FpPragmaCurrentLocation;
 }
   }
+
+  // For non-modular AST files, restore visiblity of modules.
+  for (auto &Import : ImportedModules) {
+if (Import.ImportLoc.isInvalid())
+  continue;
+if (Module *Imported = getSubmodule(Import.ID)) {
+  SemaObj->makeModuleVisible(Imported, Import.ImportLoc);
+}
+  }
 }
 
 IdentifierInfo *ASTReader::get(StringRef Name) {
Index: clang/include/clang/Sema/Sema.h
===
--- clang/include/clang/Sema/Sema.h
+++ clang/include/clang/Sema/Sema.h
@@ -1912,6 +1912,12 @@
 
   bool isModuleVisible(const Module *M, bool ModulePrivate = false);
 
+  // When loading a non-modular PCH files, this is used to restore module
+  // visibility.
+  void makeModuleVisible(Module *Mod, SourceLocation ImportLoc) {
+VisibleModules.setVisible(Mod, ImportLoc);
+  }
+
   /// Determine whether a declaration is visible to name lookup.
   bool isVisible(const NamedDecl *D) {
 return D->isUnconditionallyVisible() || isVisibleSlow(D);
Index: clang-tools-extra/clangd/unittests/ModulesTests.cpp
===
--- clang-tools-extra/clangd/unittests/ModulesTests.cpp
+++ clang-tools-extra/clangd/unittests/ModulesTests.cpp
@@ -26,7 +26,7 @@
 void foo() {}
 )cpp");
   TU.ExtraArgs.push_back("-fmodule-name=M");
-  TU.ExtraArgs.push_back("-fmodule-map-file=m.modulemap");
+  TU.ExtraArgs.push_back("-fmodule-map-file=" + testPath("m.modulemap"));
   TU.AdditionalFiles["Textual.h"] = "void foo();";
   TU.AdditionalFiles["m.modulemap"] = R"modulemap(
 module M {
@@ -39,6 +39,31 @@
   TU.index();
 }
 
+// Verify that visibility of AST nodes belonging to modules, but loaded from
+// preamble PCH, is restored.
+TEST(Modules, PreambleBuildVisibility) {
+  TestTU TU = TestTU::withCode(R"cpp(
+#include "module.h"
+
+foo x;
+)cpp");
+  TU.OverlayRealFileSystemForModules = true;
+  TU.ExtraArgs.push_back("-fmodules");
+  TU.ExtraArgs.push_back("-fmodules-strict-decluse");
+  TU.ExtraArgs.push_back("-Xclang");
+  TU.ExtraArgs.push_back("-fmodules-local-submodule-visibility");
+  TU.ExtraArgs.push_back("-fmodule-map-file=" + testPath("m.modulemap"));
+  TU.AdditionalFiles["module.h"] = R"cpp(
+typedef int foo;
+)cpp";
+  TU.AdditionalFiles["m.modulemap"] = R"modulemap(
+module M {
+  header "module.h"
+}
+)modulemap";
+  EXPECT_TRUE(TU.build().getDiagnostics().empty());
+}
+
 } // namespace
 } // namespace clangd
 } // namespace 

[PATCH] D86077: [clangd] Add a way for exporting memory usage metrics

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/TUScheduler.cpp:172
+for (const auto &Elem : LRU)
+  TotalBytes += Elem.second->getUsedBytes();
+return TotalBytes;

Any idea how expensive this is? I suppose TUScheduler::update() is rare enough 
that it's not a big deal?

I ask because of bad experience with estimating memory usage inside critical 
path on another project ;-)

Maybe we can add a span around this, so we can see if it's expensive in traces? 
Or maybe that will turn out to be more expensive than memory estimation? 



Comment at: clang-tools-extra/clangd/TUScheduler.cpp:1299
+
+  // Update memory usage metrics. Note that these are just estimates.
+  size_t ASTCacheBytes = IdleASTs->getUsedBytes();

Isn't FD->Worker->update() above async (with correct options)? If so, this is 
still reading the old data, right?



Comment at: clang-tools-extra/clangd/TUScheduler.cpp:1311
+  }
+  trace::recordMemoryUsage(PreambleBytes, "preambles");
   return NewFile;

Would it make sense to collect the number of preambles and ASTs cached at this 
point as well? Exported under a different metric, but at the same time, to be 
able to join the data together?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D86077/new/

https://reviews.llvm.org/D86077

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D86077: [clangd] Add a way for exporting memory usage metrics

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/TUScheduler.cpp:168
 
+  size_t getUsedBytes() {
+size_t TotalBytes = 0;

nit: This is the same name as getUsedBytes(Key K). Maybe rename to 
getTotalUsedBytes()?



Comment at: clang-tools-extra/clangd/TUScheduler.cpp:172
+for (const auto &Elem : LRU)
+  TotalBytes += Elem.second->getUsedBytes();
+return TotalBytes;

kadircet wrote:
> adamcz wrote:
> > Any idea how expensive this is? I suppose TUScheduler::update() is rare 
> > enough that it's not a big deal?
> > 
> > I ask because of bad experience with estimating memory usage inside 
> > critical path on another project ;-)
> > 
> > Maybe we can add a span around this, so we can see if it's expensive in 
> > traces? Or maybe that will turn out to be more expensive than memory 
> > estimation? 
> > TUScheduler::update() is rare enough
> 
> actually it can potentially be triggered after every keystroke. I suppose it 
> might make sense to move this out of the main thread completely. I suppose 
> `runWithPreamble`s action might be a better place to record, which is still 
> quite often btw (every signature help and code completion request goes 
> through it), but runs on a separate thread.
> 
> i would expect `that will turn out to be more expensive than memory 
> estimation` to be the case, will do some checks though.
> 
> Most of the calculations happening in there is just addition of some members. 
> the source manager makes me afraid a little bit tho, as it seems to be going 
> over all the buffers.
+1 to moving call to this to runWithPreamble or something like that. If a 
request gets cancelled, for example, there's no need to update the memory usage.



Comment at: clang-tools-extra/clangd/TUScheduler.cpp:1299
+
+  // Update memory usage metrics. Note that these are just estimates.
+  size_t ASTCacheBytes = IdleASTs->getUsedBytes();

kadircet wrote:
> adamcz wrote:
> > Isn't FD->Worker->update() above async (with correct options)? If so, this 
> > is still reading the old data, right?
> yes that's true. i was aiming for this to be an estimate, rather than an 
> accurate view.
I think this deserves a comment.



Comment at: clang-tools-extra/clangd/TUScheduler.cpp:1309
+// fileStats results include ast cache sizes too, subtract them.
+PreambleBytes -= ASTCacheBytes;
+  }

So technically this is incorrect.
IdleASTs might contain AST for a file that is no longer tracked (e.g. had 
removeDocument() called on it). ASTCacheBytes will include it, but 
PreambleBytes will not, since it only asks for size of the tracked files. It's 
probably fine, considering how rare this will be and that these are all 
estimates, but I would recommend a comment, in case someone sees PreambleBytes 
be negative and wonders what's going on.



Comment at: clang-tools-extra/clangd/TUScheduler.cpp:1311
+  }
+  trace::recordMemoryUsage(PreambleBytes, "preambles");
   return NewFile;

kadircet wrote:
> adamcz wrote:
> > Would it make sense to collect the number of preambles and ASTs cached at 
> > this point as well? Exported under a different metric, but at the same 
> > time, to be able to join the data together?
> yeah that definitely makes sense, but I would rather do that on a separate 
> patch.
Sounds good.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D86077/new/

https://reviews.llvm.org/D86077

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D86069: [clang] When loading preamble from AST file, re-export modules in Sema.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang/test/PCH/preamble-modules.cpp:8
+
+#ifndef MAIN_FILE
+#define MAIN_FILE

sammccall wrote:
> what are these ifdefs about? you never seem to define this symbol
> 
> (Possible you're copying from a test that is doing something tricky like 
> being run in two modes or including itself? Or did you mean to give different 
> flags for the include vs emit pch?)
Oh, I stole this trick from Kadir. You compile this file twice. First one is to 
generate a PCH for preamble. MAIN_FILE is not set, so it only reads the 
preamble section. The second one has -include-pch. That means MAIN_FILE is now 
set (from the preamble PCH file), so we only compile the main section.

It's a creative solution to a problem I wish I didn't have ;-)
Is there a better way to do this, without splitting the file in two? 
preamble_bytes seems fragile.



Comment at: clang/test/PCH/preamble-modules.cpp:14
+// Main section.
+MyType foo;
+#endif

sammccall wrote:
> This seems like currently it's never getting parsed as MAIN_FILE is never 
> defined.
It's parsed. Without this fix, there's a warning generated here. See comment 
above for explanation.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D86069/new/

https://reviews.llvm.org/D86069

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D79456: [clangd] Complete filenames after < / ".

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/CodeComplete.cpp:1922
+  Line.consume_front("include") || Line.consume_front("import") ||
+  Line.consume_front("import_next");
+  Line = Line.ltrim();

Did you mean include_next?

Also, this will never trigger, because you already consumed import before, so 
you'd need "#importimport_next" ;-) Reverse the order.

You should also return false if the whole || thing is not true.



Comment at: clang-tools-extra/clangd/unittests/CodeCompleteTests.cpp:2846
+  "#include \"foo.h\"^",
+  "#error <^",
+  };

Add test for "#<^" here, since it should be false, but I think it will be true 
right now.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D79456/new/

https://reviews.llvm.org/D79456



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D79488: [clangd] Do not offer "Add using" tweak in header files.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
adamcz added a reviewer: sammccall.
Herald added subscribers: cfe-commits, usaxena95, kadircet, arphaman, jkorous, 
MaskRay, ilya-biryukov.
Herald added a project: clang.
sammccall accepted this revision.
This revision is now accepted and ready to land.
adamcz updated this revision to Diff 262364.
adamcz marked 2 inline comments as done.
adamcz added a comment.

Addressed review comments




Comment at: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp:181
+  const auto FileName = SM.getFileEntryForID(SM.getMainFileID())->getName();
+  if (FileName.endswith(".h") || FileName.endswith(".hpp")) {
+return false;

can you use isHeaderFile instead? (in SourceCode.h)



Comment at: clang-tools-extra/clangd/unittests/TweakTests.cpp:2471
+  EXPECT_UNAVAILABLE(Header + "void fun() { one::two::f^f(); }");
+  FileName = "test.hpp";
+  EXPECT_UNAVAILABLE(Header + "void fun() { one::two::f^f(); }");

no need for this extra test if using isHeaderFile



Comment at: clang-tools-extra/clangd/unittests/TweakTests.cpp:2471
+  EXPECT_UNAVAILABLE(Header + "void fun() { one::two::f^f(); }");
+  FileName = "test.hpp";
+  EXPECT_UNAVAILABLE(Header + "void fun() { one::two::f^f(); }");

sammccall wrote:
> no need for this extra test if using isHeaderFile
I'd argue there is a need for this test - if only to check that we are, in 
fact, using isHeaderFile() or something equivalent :-)

I can delete it if you want, but I think it's better this way.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D79488

Files:
  clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
  clang-tools-extra/clangd/unittests/TweakTests.cpp


Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -2463,6 +2463,13 @@
   // test that we don't crash.
   EXPECT_UNAVAILABLE(Header +
  "template using foo = one::tt;");
+
+  // Check that we do not trigger in header files.
+  FileName = "test.h";
+  ExtraArgs.push_back("-xc++-header"); // .h file is treated a C by default.
+  EXPECT_UNAVAILABLE(Header + "void fun() { one::two::f^f(); }");
+  FileName = "test.hpp";
+  EXPECT_UNAVAILABLE(Header + "void fun() { one::two::f^f(); }");
 }
 
 TEST_F(AddUsingTest, Apply) {
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -175,6 +175,12 @@
 
 bool AddUsing::prepare(const Selection &Inputs) {
   auto &SM = Inputs.AST->getSourceManager();
+
+  // Do not suggest "using" in header files. That way madness lies.
+  if (isHeaderFile(SM.getFileEntryForID(SM.getMainFileID())->getName(),
+   Inputs.AST->getLangOpts()))
+return false;
+
   auto *Node = Inputs.ASTSelection.commonAncestor();
   if (Node == nullptr)
 return false;


Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -2463,6 +2463,13 @@
   // test that we don't crash.
   EXPECT_UNAVAILABLE(Header +
  "template using foo = one::tt;");
+
+  // Check that we do not trigger in header files.
+  FileName = "test.h";
+  ExtraArgs.push_back("-xc++-header"); // .h file is treated a C by default.
+  EXPECT_UNAVAILABLE(Header + "void fun() { one::two::f^f(); }");
+  FileName = "test.hpp";
+  EXPECT_UNAVAILABLE(Header + "void fun() { one::two::f^f(); }");
 }
 
 TEST_F(AddUsingTest, Apply) {
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -175,6 +175,12 @@
 
 bool AddUsing::prepare(const Selection &Inputs) {
   auto &SM = Inputs.AST->getSourceManager();
+
+  // Do not suggest "using" in header files. That way madness lies.
+  if (isHeaderFile(SM.getFileEntryForID(SM.getMainFileID())->getName(),
+   Inputs.AST->getLangOpts()))
+return false;
+
   auto *Node = Inputs.ASTSelection.commonAncestor();
   if (Node == nullptr)
 return false;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D79488: [clangd] Do not offer "Add using" tweak in header files.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/unittests/TweakTests.cpp:2471
+  EXPECT_UNAVAILABLE(Header + "void fun() { one::two::f^f(); }");
+  FileName = "test.hpp";
+  EXPECT_UNAVAILABLE(Header + "void fun() { one::two::f^f(); }");

sammccall wrote:
> no need for this extra test if using isHeaderFile
I'd argue there is a need for this test - if only to check that we are, in 
fact, using isHeaderFile() or something equivalent :-)

I can delete it if you want, but I think it's better this way.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D79488/new/

https://reviews.llvm.org/D79488



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D79488: [clangd] Do not offer "Add using" tweak in header files.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 262364.
adamcz marked 2 inline comments as done.
adamcz added a comment.

Addressed review comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D79488/new/

https://reviews.llvm.org/D79488

Files:
  clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
  clang-tools-extra/clangd/unittests/TweakTests.cpp


Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -2463,6 +2463,13 @@
   // test that we don't crash.
   EXPECT_UNAVAILABLE(Header +
  "template using foo = one::tt;");
+
+  // Check that we do not trigger in header files.
+  FileName = "test.h";
+  ExtraArgs.push_back("-xc++-header"); // .h file is treated a C by default.
+  EXPECT_UNAVAILABLE(Header + "void fun() { one::two::f^f(); }");
+  FileName = "test.hpp";
+  EXPECT_UNAVAILABLE(Header + "void fun() { one::two::f^f(); }");
 }
 
 TEST_F(AddUsingTest, Apply) {
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -175,6 +175,12 @@
 
 bool AddUsing::prepare(const Selection &Inputs) {
   auto &SM = Inputs.AST->getSourceManager();
+
+  // Do not suggest "using" in header files. That way madness lies.
+  if (isHeaderFile(SM.getFileEntryForID(SM.getMainFileID())->getName(),
+   Inputs.AST->getLangOpts()))
+return false;
+
   auto *Node = Inputs.ASTSelection.commonAncestor();
   if (Node == nullptr)
 return false;


Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -2463,6 +2463,13 @@
   // test that we don't crash.
   EXPECT_UNAVAILABLE(Header +
  "template using foo = one::tt;");
+
+  // Check that we do not trigger in header files.
+  FileName = "test.h";
+  ExtraArgs.push_back("-xc++-header"); // .h file is treated a C by default.
+  EXPECT_UNAVAILABLE(Header + "void fun() { one::two::f^f(); }");
+  FileName = "test.hpp";
+  EXPECT_UNAVAILABLE(Header + "void fun() { one::two::f^f(); }");
 }
 
 TEST_F(AddUsingTest, Apply) {
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -175,6 +175,12 @@
 
 bool AddUsing::prepare(const Selection &Inputs) {
   auto &SM = Inputs.AST->getSourceManager();
+
+  // Do not suggest "using" in header files. That way madness lies.
+  if (isHeaderFile(SM.getFileEntryForID(SM.getMainFileID())->getName(),
+   Inputs.AST->getLangOpts()))
+return false;
+
   auto *Node = Inputs.ASTSelection.commonAncestor();
   if (Node == nullptr)
 return false;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D79496: [clangd] Fix AddUsing tweak for out-of-line functions.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
adamcz added a reviewer: hokein.
Herald added subscribers: cfe-commits, usaxena95, kadircet, arphaman, jkorous, 
MaskRay, ilya-biryukov.
Herald added a project: clang.

We used getEnclosingNamespaceContext(), which calls getParent() rather
than getLexicalParent(), so we would end up adding the "using" line in
places that do not affect the cursor location, or just return an error
when declaration was in another file.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D79496

Files:
  clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
  clang-tools-extra/clangd/unittests/TweakTests.cpp


Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -2663,6 +2663,23 @@
 
 void fun() {
   CALL(ff);
+})cpp"},
+// Parent namespace != lexical parent namespace
+{R"cpp(
+#include "test.hpp"
+namespace foo { void fun(); }
+
+void foo::fun() {
+  one::two::f^f();
+})cpp",
+ R"cpp(
+#include "test.hpp"
+using one::two::ff;
+
+namespace foo { void fun(); }
+
+void foo::fun() {
+  ff();
 })cpp"}};
   llvm::StringMap EditedFiles;
   for (const auto &Case : Cases) {
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -141,10 +141,12 @@
   }
 
   // No relevant "using" statements. Try the nearest namespace level.
-  const auto *NS = Inputs.ASTSelection.commonAncestor()
-   ->getDeclContext()
-   .getEnclosingNamespaceContext();
-  if (auto *ND = dyn_cast(NS)) {
+  const DeclContext *ParentDeclCtx =
+  &Inputs.ASTSelection.commonAncestor()->getDeclContext();
+  while (ParentDeclCtx && !ParentDeclCtx->isFileContext()) {
+ParentDeclCtx = ParentDeclCtx->getLexicalParent();
+  }
+  if (auto *ND = llvm::dyn_cast_or_null(ParentDeclCtx)) {
 auto Toks = Inputs.AST->getTokens().expandedTokens(ND->getSourceRange());
 const auto *Tok = llvm::find_if(Toks, [](const syntax::Token &Tok) {
   return Tok.kind() == tok::l_brace;


Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -2663,6 +2663,23 @@
 
 void fun() {
   CALL(ff);
+})cpp"},
+// Parent namespace != lexical parent namespace
+{R"cpp(
+#include "test.hpp"
+namespace foo { void fun(); }
+
+void foo::fun() {
+  one::two::f^f();
+})cpp",
+ R"cpp(
+#include "test.hpp"
+using one::two::ff;
+
+namespace foo { void fun(); }
+
+void foo::fun() {
+  ff();
 })cpp"}};
   llvm::StringMap EditedFiles;
   for (const auto &Case : Cases) {
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -141,10 +141,12 @@
   }
 
   // No relevant "using" statements. Try the nearest namespace level.
-  const auto *NS = Inputs.ASTSelection.commonAncestor()
-   ->getDeclContext()
-   .getEnclosingNamespaceContext();
-  if (auto *ND = dyn_cast(NS)) {
+  const DeclContext *ParentDeclCtx =
+  &Inputs.ASTSelection.commonAncestor()->getDeclContext();
+  while (ParentDeclCtx && !ParentDeclCtx->isFileContext()) {
+ParentDeclCtx = ParentDeclCtx->getLexicalParent();
+  }
+  if (auto *ND = llvm::dyn_cast_or_null(ParentDeclCtx)) {
 auto Toks = Inputs.AST->getTokens().expandedTokens(ND->getSourceRange());
 const auto *Tok = llvm::find_if(Toks, [](const syntax::Token &Tok) {
   return Tok.kind() == tok::l_brace;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D79496: [clangd] Fix AddUsing tweak for out-of-line functions.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz marked an inline comment as done.
adamcz added inline comments.



Comment at: clang-tools-extra/clangd/unittests/TweakTests.cpp:2673
+void foo::fun() {
+  one::two::f^f();
+})cpp",

hokein wrote:
> IIUC, before this patch, the using was added inside the above namespace 
> `foo`, the code was still valid after applying the tweak.
> 
> There is no enclosing namespace for `foo::fun()`, so we will insert the using 
> at the first top-level decl. It seems that we will break the code if the 
> `one::two::ff` decl is defined in the main file (not the `test.hpp`), e.g.
> 
> ```
> // using one::two::ff will be inserted here, which is not valid
> namespace one {
> namespace two {
> void ff();
> }
> }
> 
> namespace foo { void fun(); }
> void foo::fun() {...}
> ```
> 
> this case was working before the patch, and now is broken after this patch. 
> It is not a common case, probably ok for now.
You are correct that this will not work. It is not a regression, however, since 
it was broken before too. Prior to this patch, the using would be inserted 
inside the namespace foo { }, so while the using statement would compile, it 
wouldn't affect the out-of-line definition of the function, meaning removing 
the qualifier would not work.

The insertion point logic will have to change a little when we add the feature 
to remove all other full qualifiers for that name. I think it might be best to 
address these corner cases at that point, but I can also do it here if you 
prefer.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D79496/new/

https://reviews.llvm.org/D79496



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D79582: [clangd] Fix crash in AddUsing tweak due to non-identifier DeclName

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
adamcz added a reviewer: hokein.
Herald added subscribers: cfe-commits, usaxena95, kadircet, arphaman, jkorous, 
MaskRay, ilya-biryukov.
Herald added a project: clang.

Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D79582

Files:
  clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
  clang-tools-extra/clangd/unittests/TweakTests.cpp


Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -2444,6 +2444,7 @@
 public:
   struct st {};
   static void mm() {}
+  cc operator|(const cc& x) const { return x; }
 };
 }
 })cpp";
@@ -2463,6 +2464,9 @@
   // test that we don't crash.
   EXPECT_UNAVAILABLE(Header +
  "template using foo = one::tt;");
+  // Test that we don't crash or misbehave on unnamed DeclRefExpr.
+  EXPECT_UNAVAILABLE(Header +
+ "void fun() { one::two::cc() ^| one::two::cc(); }");
 
   // Check that we do not trigger in header files.
   FileName = "test.h";
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -210,8 +210,11 @@
 return false;
 
   if (auto *D = Node->ASTNode.get()) {
-QualifierToRemove = D->getQualifierLoc();
-Name = D->getDecl()->getName();
+if (D->getDecl()->getDeclName().getNameKind() ==
+DeclarationName::Identifier) {
+  QualifierToRemove = D->getQualifierLoc();
+  Name = D->getDecl()->getName();
+}
   } else if (auto *T = Node->ASTNode.get()) {
 if (auto E = T->getAs()) {
   if (auto *BaseTypeIdentifier =


Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -2444,6 +2444,7 @@
 public:
   struct st {};
   static void mm() {}
+  cc operator|(const cc& x) const { return x; }
 };
 }
 })cpp";
@@ -2463,6 +2464,9 @@
   // test that we don't crash.
   EXPECT_UNAVAILABLE(Header +
  "template using foo = one::tt;");
+  // Test that we don't crash or misbehave on unnamed DeclRefExpr.
+  EXPECT_UNAVAILABLE(Header +
+ "void fun() { one::two::cc() ^| one::two::cc(); }");
 
   // Check that we do not trigger in header files.
   FileName = "test.h";
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -210,8 +210,11 @@
 return false;
 
   if (auto *D = Node->ASTNode.get()) {
-QualifierToRemove = D->getQualifierLoc();
-Name = D->getDecl()->getName();
+if (D->getDecl()->getDeclName().getNameKind() ==
+DeclarationName::Identifier) {
+  QualifierToRemove = D->getQualifierLoc();
+  Name = D->getDecl()->getName();
+}
   } else if (auto *T = Node->ASTNode.get()) {
 if (auto E = T->getAs()) {
   if (auto *BaseTypeIdentifier =
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D79582: [clangd] Fix crash in AddUsing tweak due to non-identifier DeclName

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 262857.
adamcz marked an inline comment as done.
adamcz added a comment.

Addressed review comment


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D79582/new/

https://reviews.llvm.org/D79582

Files:
  clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
  clang-tools-extra/clangd/unittests/TweakTests.cpp


Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -2444,6 +2444,7 @@
 public:
   struct st {};
   static void mm() {}
+  cc operator|(const cc& x) const { return x; }
 };
 }
 })cpp";
@@ -2463,6 +2464,9 @@
   // test that we don't crash.
   EXPECT_UNAVAILABLE(Header +
  "template using foo = one::tt;");
+  // Test that we don't crash or misbehave on unnamed DeclRefExpr.
+  EXPECT_UNAVAILABLE(Header +
+ "void fun() { one::two::cc() ^| one::two::cc(); }");
 
   // Check that we do not trigger in header files.
   FileName = "test.h";
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -210,8 +210,10 @@
 return false;
 
   if (auto *D = Node->ASTNode.get()) {
-QualifierToRemove = D->getQualifierLoc();
-Name = D->getDecl()->getName();
+if (auto *II = D->getDecl()->getIdentifier()) {
+  QualifierToRemove = D->getQualifierLoc();
+  Name = II->getName();
+}
   } else if (auto *T = Node->ASTNode.get()) {
 if (auto E = T->getAs()) {
   if (auto *BaseTypeIdentifier =


Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -2444,6 +2444,7 @@
 public:
   struct st {};
   static void mm() {}
+  cc operator|(const cc& x) const { return x; }
 };
 }
 })cpp";
@@ -2463,6 +2464,9 @@
   // test that we don't crash.
   EXPECT_UNAVAILABLE(Header +
  "template using foo = one::tt;");
+  // Test that we don't crash or misbehave on unnamed DeclRefExpr.
+  EXPECT_UNAVAILABLE(Header +
+ "void fun() { one::two::cc() ^| one::two::cc(); }");
 
   // Check that we do not trigger in header files.
   FileName = "test.h";
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -210,8 +210,10 @@
 return false;
 
   if (auto *D = Node->ASTNode.get()) {
-QualifierToRemove = D->getQualifierLoc();
-Name = D->getDecl()->getName();
+if (auto *II = D->getDecl()->getIdentifier()) {
+  QualifierToRemove = D->getQualifierLoc();
+  Name = II->getName();
+}
   } else if (auto *T = Node->ASTNode.get()) {
 if (auto E = T->getAs()) {
   if (auto *BaseTypeIdentifier =
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D78454: [clangd] Highlight related control flow.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz accepted this revision.
adamcz added inline comments.
This revision is now accepted and ready to land.



Comment at: clang-tools-extra/clangd/XRefs.cpp:907
+// Highlight the loop keyword itself.
+Result.push_back(P->ASTNode.getSourceRange().getBegin());
+break;

nit: in a do {...} while(...) case, would we want to highlight both keywords?

(doesn't seem very important, feel free to ignore)



Comment at: clang-tools-extra/clangd/XRefs.cpp:980
+auto Refs = findRefs({Decls.begin(), Decls.end()}, AST);
+// FIXME: we may get multiple
+// DocumentHighlights with the same location

nit: weird line wrap here?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D78454/new/

https://reviews.llvm.org/D78454



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D121824: [clang] Do not crash on arrow operator on dependent type.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 418219.
adamcz marked an inline comment as done.
adamcz added a comment.

added a comment


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D121824/new/

https://reviews.llvm.org/D121824

Files:
  clang/lib/Sema/TreeTransform.h
  clang/test/SemaCXX/arrow-operator.cpp


Index: clang/test/SemaCXX/arrow-operator.cpp
===
--- clang/test/SemaCXX/arrow-operator.cpp
+++ clang/test/SemaCXX/arrow-operator.cpp
@@ -65,3 +65,51 @@
 }
 
 } // namespace arrow_suggest
+
+namespace no_crash_dependent_type {
+
+template 
+struct A {
+  void call();
+  A *operator->();
+};
+
+template 
+void foo() {
+  // The "requires an initializer" error seems unnecessary.
+  A &x = blah[7]; // expected-error {{use of undeclared identifier 
'blah'}} \
+// expected-error {{requires an initializer}}
+  // x is dependent.
+  x->call();
+}
+
+void test() {
+  foo(); // expected-note {{requested here}}
+}
+
+} // namespace no_crash_dependent_type
+
+namespace clangd_issue_1073_no_crash_dependent_type {
+
+template  struct Ptr {
+  T *operator->();
+};
+
+struct Struct {
+  int len;
+};
+
+template 
+struct TemplateStruct {
+  Ptr val(); // expected-note {{declared here}}
+};
+
+template 
+void templateFunc(const TemplateStruct &ts) {
+  Ptr ptr = ts.val(); // expected-error {{function is not marked 
const}}
+  auto foo = ptr->len;
+}
+
+template void templateFunc<0>(const TemplateStruct<0> &); // expected-note 
{{requested here}}
+
+} // namespace clangd_issue_1073_no_crash_dependent_type
Index: clang/lib/Sema/TreeTransform.h
===
--- clang/lib/Sema/TreeTransform.h
+++ clang/lib/Sema/TreeTransform.h
@@ -14757,6 +14757,10 @@
   return getSema().CreateBuiltinArraySubscriptExpr(
   First, Callee->getBeginLoc(), Second, OpLoc);
   } else if (Op == OO_Arrow) {
+// It is possible that the type refers to a RecoveryExpr created earlier
+// in the tree transformation.
+if (First->getType()->isDependentType())
+  return ExprError();
 // -> is never a builtin operation.
 return SemaRef.BuildOverloadedArrowExpr(nullptr, First, OpLoc);
   } else if (Second == nullptr || isPostIncDec) {


Index: clang/test/SemaCXX/arrow-operator.cpp
===
--- clang/test/SemaCXX/arrow-operator.cpp
+++ clang/test/SemaCXX/arrow-operator.cpp
@@ -65,3 +65,51 @@
 }
 
 } // namespace arrow_suggest
+
+namespace no_crash_dependent_type {
+
+template 
+struct A {
+  void call();
+  A *operator->();
+};
+
+template 
+void foo() {
+  // The "requires an initializer" error seems unnecessary.
+  A &x = blah[7]; // expected-error {{use of undeclared identifier 'blah'}} \
+// expected-error {{requires an initializer}}
+  // x is dependent.
+  x->call();
+}
+
+void test() {
+  foo(); // expected-note {{requested here}}
+}
+
+} // namespace no_crash_dependent_type
+
+namespace clangd_issue_1073_no_crash_dependent_type {
+
+template  struct Ptr {
+  T *operator->();
+};
+
+struct Struct {
+  int len;
+};
+
+template 
+struct TemplateStruct {
+  Ptr val(); // expected-note {{declared here}}
+};
+
+template 
+void templateFunc(const TemplateStruct &ts) {
+  Ptr ptr = ts.val(); // expected-error {{function is not marked const}}
+  auto foo = ptr->len;
+}
+
+template void templateFunc<0>(const TemplateStruct<0> &); // expected-note {{requested here}}
+
+} // namespace clangd_issue_1073_no_crash_dependent_type
Index: clang/lib/Sema/TreeTransform.h
===
--- clang/lib/Sema/TreeTransform.h
+++ clang/lib/Sema/TreeTransform.h
@@ -14757,6 +14757,10 @@
   return getSema().CreateBuiltinArraySubscriptExpr(
   First, Callee->getBeginLoc(), Second, OpLoc);
   } else if (Op == OO_Arrow) {
+// It is possible that the type refers to a RecoveryExpr created earlier
+// in the tree transformation.
+if (First->getType()->isDependentType())
+  return ExprError();
 // -> is never a builtin operation.
 return SemaRef.BuildOverloadedArrowExpr(nullptr, First, OpLoc);
   } else if (Second == nullptr || isPostIncDec) {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D121824: [clang] Do not crash on arrow operator on dependent type.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added a comment.

Thanks for all the comments!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D121824/new/

https://reviews.llvm.org/D121824

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D121824: [clang] Do not crash on arrow operator on dependent type.

[Adam Czachorowski via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rG7e459126185f: [clang] Do not crash on arrow operator on 
dependent type. (authored by adamcz).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D121824/new/

https://reviews.llvm.org/D121824

Files:
  clang/lib/Sema/TreeTransform.h
  clang/test/SemaCXX/arrow-operator.cpp


Index: clang/test/SemaCXX/arrow-operator.cpp
===
--- clang/test/SemaCXX/arrow-operator.cpp
+++ clang/test/SemaCXX/arrow-operator.cpp
@@ -65,3 +65,51 @@
 }
 
 } // namespace arrow_suggest
+
+namespace no_crash_dependent_type {
+
+template 
+struct A {
+  void call();
+  A *operator->();
+};
+
+template 
+void foo() {
+  // The "requires an initializer" error seems unnecessary.
+  A &x = blah[7]; // expected-error {{use of undeclared identifier 
'blah'}} \
+// expected-error {{requires an initializer}}
+  // x is dependent.
+  x->call();
+}
+
+void test() {
+  foo(); // expected-note {{requested here}}
+}
+
+} // namespace no_crash_dependent_type
+
+namespace clangd_issue_1073_no_crash_dependent_type {
+
+template  struct Ptr {
+  T *operator->();
+};
+
+struct Struct {
+  int len;
+};
+
+template 
+struct TemplateStruct {
+  Ptr val(); // expected-note {{declared here}}
+};
+
+template 
+void templateFunc(const TemplateStruct &ts) {
+  Ptr ptr = ts.val(); // expected-error {{function is not marked 
const}}
+  auto foo = ptr->len;
+}
+
+template void templateFunc<0>(const TemplateStruct<0> &); // expected-note 
{{requested here}}
+
+} // namespace clangd_issue_1073_no_crash_dependent_type
Index: clang/lib/Sema/TreeTransform.h
===
--- clang/lib/Sema/TreeTransform.h
+++ clang/lib/Sema/TreeTransform.h
@@ -14757,6 +14757,10 @@
   return getSema().CreateBuiltinArraySubscriptExpr(
   First, Callee->getBeginLoc(), Second, OpLoc);
   } else if (Op == OO_Arrow) {
+// It is possible that the type refers to a RecoveryExpr created earlier
+// in the tree transformation.
+if (First->getType()->isDependentType())
+  return ExprError();
 // -> is never a builtin operation.
 return SemaRef.BuildOverloadedArrowExpr(nullptr, First, OpLoc);
   } else if (Second == nullptr || isPostIncDec) {


Index: clang/test/SemaCXX/arrow-operator.cpp
===
--- clang/test/SemaCXX/arrow-operator.cpp
+++ clang/test/SemaCXX/arrow-operator.cpp
@@ -65,3 +65,51 @@
 }
 
 } // namespace arrow_suggest
+
+namespace no_crash_dependent_type {
+
+template 
+struct A {
+  void call();
+  A *operator->();
+};
+
+template 
+void foo() {
+  // The "requires an initializer" error seems unnecessary.
+  A &x = blah[7]; // expected-error {{use of undeclared identifier 'blah'}} \
+// expected-error {{requires an initializer}}
+  // x is dependent.
+  x->call();
+}
+
+void test() {
+  foo(); // expected-note {{requested here}}
+}
+
+} // namespace no_crash_dependent_type
+
+namespace clangd_issue_1073_no_crash_dependent_type {
+
+template  struct Ptr {
+  T *operator->();
+};
+
+struct Struct {
+  int len;
+};
+
+template 
+struct TemplateStruct {
+  Ptr val(); // expected-note {{declared here}}
+};
+
+template 
+void templateFunc(const TemplateStruct &ts) {
+  Ptr ptr = ts.val(); // expected-error {{function is not marked const}}
+  auto foo = ptr->len;
+}
+
+template void templateFunc<0>(const TemplateStruct<0> &); // expected-note {{requested here}}
+
+} // namespace clangd_issue_1073_no_crash_dependent_type
Index: clang/lib/Sema/TreeTransform.h
===
--- clang/lib/Sema/TreeTransform.h
+++ clang/lib/Sema/TreeTransform.h
@@ -14757,6 +14757,10 @@
   return getSema().CreateBuiltinArraySubscriptExpr(
   First, Callee->getBeginLoc(), Second, OpLoc);
   } else if (Op == OO_Arrow) {
+// It is possible that the type refers to a RecoveryExpr created earlier
+// in the tree transformation.
+if (First->getType()->isDependentType())
+  return ExprError();
 // -> is never a builtin operation.
 return SemaRef.BuildOverloadedArrowExpr(nullptr, First, OpLoc);
   } else if (Second == nullptr || isPostIncDec) {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D122894: [clangd] Record IO precentage for first preamble build of the instance

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added a comment.

Thanks, I was just about to fix that!




Comment at: clang-tools-extra/clangd/TUScheduler.cpp:115
  bool IsFirstPreamble) {
-  static llvm::once_flag OnceFlag;
-  llvm::call_once(OnceFlag, [&] {
+  auto RecordWithLabel = [&Stats](llvm::StringRef Label) {
 PreambleBuildFilesystemLatency.record(Stats.FileSystemTime, "first_build");

You're not using Label anywhere. Replace first_build below with Label.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D122894/new/

https://reviews.llvm.org/D122894

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D130636: [clangd] Upgrade vlog() to log() for preamble build stats

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
adamcz added a reviewer: kadircet.
Herald added subscribers: usaxena95, arphaman.
Herald added a project: All.
adamcz requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

This is very useful information for better understanding performance and
preamble builds don't happen that often, so it's not that spammy.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D130636

Files:
  clang-tools-extra/clangd/Preamble.cpp


Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -552,7 +552,7 @@
   }
 
   if (BuiltPreamble) {
-vlog("Built preamble of size {0} for file {1} version {2} in {3} seconds",
+log("Built preamble of size {0} for file {1} version {2} in {3} seconds",
  BuiltPreamble->getSize(), FileName, Inputs.Version,
  PreambleTimer.getTime());
 std::vector Diags = PreambleDiagnostics.take();


Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -552,7 +552,7 @@
   }
 
   if (BuiltPreamble) {
-vlog("Built preamble of size {0} for file {1} version {2} in {3} seconds",
+log("Built preamble of size {0} for file {1} version {2} in {3} seconds",
  BuiltPreamble->getSize(), FileName, Inputs.Version,
  PreambleTimer.getTime());
 std::vector Diags = PreambleDiagnostics.take();
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D130636: [clangd] Upgrade vlog() to log() for preamble build stats

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/Preamble.cpp:555
   if (BuiltPreamble) {
-vlog("Built preamble of size {0} for file {1} version {2} in {3} seconds",
+log("Built preamble of size {0} for file {1} version {2} in {3} seconds",
  BuiltPreamble->getSize(), FileName, Inputs.Version,

kadircet wrote:
> what about also moving this log into TUScheduler.cpp::reportPreamble, in 
> which we can also log whether this is a "first preamble" or not?
I considered that, but this isn't structured log or something you'd process 
automatically and it's already clear from context in that log file if it's 
first build or rebuild. I think it's simpler to keep the log line here, to 
match the error line below.

If you have a use case for this information let me know and I'll update this 
change, otherwise I'll just go with the simple solution.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D130636/new/

https://reviews.llvm.org/D130636

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D130636: [clangd] Upgrade vlog() to log() for preamble build stats

[Adam Czachorowski via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rG6b2fed3ab419: [clangd] Upgrade vlog() to log() for preamble 
build stats (authored by adamcz).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D130636/new/

https://reviews.llvm.org/D130636

Files:
  clang-tools-extra/clangd/Preamble.cpp


Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -552,7 +552,7 @@
   }
 
   if (BuiltPreamble) {
-vlog("Built preamble of size {0} for file {1} version {2} in {3} seconds",
+log("Built preamble of size {0} for file {1} version {2} in {3} seconds",
  BuiltPreamble->getSize(), FileName, Inputs.Version,
  PreambleTimer.getTime());
 std::vector Diags = PreambleDiagnostics.take();


Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -552,7 +552,7 @@
   }
 
   if (BuiltPreamble) {
-vlog("Built preamble of size {0} for file {1} version {2} in {3} seconds",
+log("Built preamble of size {0} for file {1} version {2} in {3} seconds",
  BuiltPreamble->getSize(), FileName, Inputs.Version,
  PreambleTimer.getTime());
 std::vector Diags = PreambleDiagnostics.take();
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D121712: [clangd] Track time spent in filesystem ops during preamble builds

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
adamcz added a reviewer: kadircet.
Herald added subscribers: usaxena95, arphaman, javed.absar.
Herald added a project: All.
adamcz requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

In some deployments, for example when running on FUSE or using some
network-based VFS implementation, the filesystem operations might add up
to a significant fraction of preamble build time. This change allows us
to track time spent in FS operations to better understand the problem.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D121712

Files:
  clang-tools-extra/clangd/FS.cpp
  clang-tools-extra/clangd/FS.h
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/TUScheduler.cpp
  clang-tools-extra/clangd/tool/Check.cpp
  clang-tools-extra/clangd/unittests/CodeCompleteTests.cpp
  clang-tools-extra/clangd/unittests/FileIndexTests.cpp
  clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
  clang-tools-extra/clangd/unittests/PreambleTests.cpp
  clang-tools-extra/clangd/unittests/TestTU.cpp

Index: clang-tools-extra/clangd/unittests/TestTU.cpp
===
--- clang-tools-extra/clangd/unittests/TestTU.cpp
+++ clang-tools-extra/clangd/unittests/TestTU.cpp
@@ -101,7 +101,8 @@
   auto ModuleCacheDeleter = llvm::make_scope_exit(
   std::bind(deleteModuleCache, CI->getHeaderSearchOpts().ModuleCachePath));
   return clang::clangd::buildPreamble(testPath(Filename), *CI, Inputs,
-  /*StoreInMemory=*/true, PreambleCallback);
+  /*StoreInMemory=*/true, /*Stats=*/nullptr,
+  PreambleCallback);
 }
 
 ParsedAST TestTU::build() const {
@@ -117,6 +118,7 @@
 
   auto Preamble = clang::clangd::buildPreamble(testPath(Filename), *CI, Inputs,
/*StoreInMemory=*/true,
+   /*Stats=*/nullptr,
/*PreambleCallback=*/nullptr);
   auto AST = ParsedAST::build(testPath(Filename), Inputs, std::move(CI),
   Diags.take(), Preamble);
Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -172,8 +172,9 @@
   TU.AdditionalFiles["b.h"] = "";
   TU.AdditionalFiles["c.h"] = "";
   auto PI = TU.inputs(FS);
-  auto BaselinePreamble = buildPreamble(
-  TU.Filename, *buildCompilerInvocation(PI, Diags), PI, true, nullptr);
+  auto BaselinePreamble =
+  buildPreamble(TU.Filename, *buildCompilerInvocation(PI, Diags), PI, true,
+/*Stats=*/nullptr, nullptr);
   // We drop c.h from modified and add a new header. Since the latter is patched
   // we should only get a.h in preamble includes.
   TU.Code = R"cpp(
Index: clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
===
--- clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
+++ clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
@@ -494,8 +494,8 @@
   MockFS FS;
   auto Inputs = TU.inputs(FS);
   auto CI = buildCompilerInvocation(Inputs, Diags);
-  auto EmptyPreamble =
-  buildPreamble(testPath("foo.cpp"), *CI, Inputs, true, nullptr);
+  auto EmptyPreamble = buildPreamble(testPath("foo.cpp"), *CI, Inputs, true,
+ /*Stats=*/nullptr, nullptr);
   ASSERT_TRUE(EmptyPreamble);
   EXPECT_THAT(EmptyPreamble->Includes.MainFileIncludes, IsEmpty());
 
@@ -537,8 +537,8 @@
   MockFS FS;
   auto Inputs = TU.inputs(FS);
   auto CI = buildCompilerInvocation(Inputs, Diags);
-  auto BaselinePreamble =
-  buildPreamble(testPath("foo.cpp"), *CI, Inputs, true, nullptr);
+  auto BaselinePreamble = buildPreamble(testPath("foo.cpp"), *CI, Inputs, true,
+/*Stats=*/nullptr, nullptr);
   ASSERT_TRUE(BaselinePreamble);
   EXPECT_THAT(BaselinePreamble->Includes.MainFileIncludes,
   ElementsAre(testing::Field(&Inclusion::Written, "")));
Index: clang-tools-extra/clangd/unittests/FileIndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/FileIndexTests.cpp
+++ clang-tools-extra/clangd/unittests/FileIndexTests.cpp
@@ -307,6 +307,7 @@
   bool IndexUpdated = false;
   buildPreamble(FooCpp, *CI, PI,
 /*StoreInMemory=*/true,
+/*Stats=*/nullptr,
 [&](ASTContext &Ctx, Preprocessor &PP,
 const CanonicalIncludes &CanonIncludes) {
   EXPECT_FALSE(IndexUpdated)
Index: clang-tools-extra/clangd/unittests/CodeCompleteTests.cpp

[PATCH] D121824: [clang] Do not crash on arrow operator on dependent type.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
adamcz added a reviewer: hokein.
Herald added a project: All.
adamcz requested review of this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.

Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D121824

Files:
  clang/lib/Sema/TreeTransform.h
  clang/test/SemaCXX/arrow-operator.cpp


Index: clang/test/SemaCXX/arrow-operator.cpp
===
--- clang/test/SemaCXX/arrow-operator.cpp
+++ clang/test/SemaCXX/arrow-operator.cpp
@@ -65,3 +65,25 @@
 }
 
 } // namespace arrow_suggest
+
+namespace no_crash_dependent_type {
+
+template
+struct A {
+  void call();
+  A* operator->();
+};
+
+template 
+void foo() {
+  // x is dependent.
+  A& x = blah[7];  // expected-error {{use of undeclared identifier 
'blah'}} \
+// expected-error {{declaration of reference variable 
'x' requires an initializer}}
+  x->call();
+}
+
+void test() {
+  foo();  // expected-note {{in instantiation}}
+}
+
+} // namespace no_crash_dependent_type
Index: clang/lib/Sema/TreeTransform.h
===
--- clang/lib/Sema/TreeTransform.h
+++ clang/lib/Sema/TreeTransform.h
@@ -14706,6 +14706,8 @@
   First, Callee->getBeginLoc(), Second, OpLoc);
   } else if (Op == OO_Arrow) {
 // -> is never a builtin operation.
+if (First->getType()->isDependentType())
+  return ExprError();
 return SemaRef.BuildOverloadedArrowExpr(nullptr, First, OpLoc);
   } else if (Second == nullptr || isPostIncDec) {
 if (!First->getType()->isOverloadableType() ||


Index: clang/test/SemaCXX/arrow-operator.cpp
===
--- clang/test/SemaCXX/arrow-operator.cpp
+++ clang/test/SemaCXX/arrow-operator.cpp
@@ -65,3 +65,25 @@
 }
 
 } // namespace arrow_suggest
+
+namespace no_crash_dependent_type {
+
+template
+struct A {
+  void call();
+  A* operator->();
+};
+
+template 
+void foo() {
+  // x is dependent.
+  A& x = blah[7];  // expected-error {{use of undeclared identifier 'blah'}} \
+// expected-error {{declaration of reference variable 'x' requires an initializer}}
+  x->call();
+}
+
+void test() {
+  foo();  // expected-note {{in instantiation}}
+}
+
+} // namespace no_crash_dependent_type
Index: clang/lib/Sema/TreeTransform.h
===
--- clang/lib/Sema/TreeTransform.h
+++ clang/lib/Sema/TreeTransform.h
@@ -14706,6 +14706,8 @@
   First, Callee->getBeginLoc(), Second, OpLoc);
   } else if (Op == OO_Arrow) {
 // -> is never a builtin operation.
+if (First->getType()->isDependentType())
+  return ExprError();
 return SemaRef.BuildOverloadedArrowExpr(nullptr, First, OpLoc);
   } else if (Second == nullptr || isPostIncDec) {
 if (!First->getType()->isOverloadableType() ||
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D121712: [clangd] Track time spent in filesystem ops during preamble builds

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 415919.
adamcz marked 12 inline comments as done.
adamcz added a comment.

addressed review comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D121712/new/

https://reviews.llvm.org/D121712

Files:
  clang-tools-extra/clangd/FS.cpp
  clang-tools-extra/clangd/FS.h
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/TUScheduler.cpp
  clang-tools-extra/clangd/tool/Check.cpp
  clang-tools-extra/clangd/unittests/CodeCompleteTests.cpp
  clang-tools-extra/clangd/unittests/FileIndexTests.cpp
  clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
  clang-tools-extra/clangd/unittests/PreambleTests.cpp
  clang-tools-extra/clangd/unittests/TestTU.cpp

Index: clang-tools-extra/clangd/unittests/TestTU.cpp
===
--- clang-tools-extra/clangd/unittests/TestTU.cpp
+++ clang-tools-extra/clangd/unittests/TestTU.cpp
@@ -101,7 +101,8 @@
   auto ModuleCacheDeleter = llvm::make_scope_exit(
   std::bind(deleteModuleCache, CI->getHeaderSearchOpts().ModuleCachePath));
   return clang::clangd::buildPreamble(testPath(Filename), *CI, Inputs,
-  /*StoreInMemory=*/true, PreambleCallback);
+  /*StoreInMemory=*/true, /*Stats=*/nullptr,
+  PreambleCallback);
 }
 
 ParsedAST TestTU::build() const {
@@ -117,6 +118,7 @@
 
   auto Preamble = clang::clangd::buildPreamble(testPath(Filename), *CI, Inputs,
/*StoreInMemory=*/true,
+   /*Stats=*/nullptr,
/*PreambleCallback=*/nullptr);
   auto AST = ParsedAST::build(testPath(Filename), Inputs, std::move(CI),
   Diags.take(), Preamble);
Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -172,8 +172,9 @@
   TU.AdditionalFiles["b.h"] = "";
   TU.AdditionalFiles["c.h"] = "";
   auto PI = TU.inputs(FS);
-  auto BaselinePreamble = buildPreamble(
-  TU.Filename, *buildCompilerInvocation(PI, Diags), PI, true, nullptr);
+  auto BaselinePreamble =
+  buildPreamble(TU.Filename, *buildCompilerInvocation(PI, Diags), PI, true,
+/*Stats=*/nullptr, nullptr);
   // We drop c.h from modified and add a new header. Since the latter is patched
   // we should only get a.h in preamble includes.
   TU.Code = R"cpp(
Index: clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
===
--- clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
+++ clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
@@ -494,8 +494,8 @@
   MockFS FS;
   auto Inputs = TU.inputs(FS);
   auto CI = buildCompilerInvocation(Inputs, Diags);
-  auto EmptyPreamble =
-  buildPreamble(testPath("foo.cpp"), *CI, Inputs, true, nullptr);
+  auto EmptyPreamble = buildPreamble(testPath("foo.cpp"), *CI, Inputs, true,
+ /*Stats=*/nullptr, nullptr);
   ASSERT_TRUE(EmptyPreamble);
   EXPECT_THAT(EmptyPreamble->Includes.MainFileIncludes, IsEmpty());
 
@@ -537,8 +537,8 @@
   MockFS FS;
   auto Inputs = TU.inputs(FS);
   auto CI = buildCompilerInvocation(Inputs, Diags);
-  auto BaselinePreamble =
-  buildPreamble(testPath("foo.cpp"), *CI, Inputs, true, nullptr);
+  auto BaselinePreamble = buildPreamble(testPath("foo.cpp"), *CI, Inputs, true,
+/*Stats=*/nullptr, nullptr);
   ASSERT_TRUE(BaselinePreamble);
   EXPECT_THAT(BaselinePreamble->Includes.MainFileIncludes,
   ElementsAre(testing::Field(&Inclusion::Written, "")));
Index: clang-tools-extra/clangd/unittests/FileIndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/FileIndexTests.cpp
+++ clang-tools-extra/clangd/unittests/FileIndexTests.cpp
@@ -307,6 +307,7 @@
   bool IndexUpdated = false;
   buildPreamble(FooCpp, *CI, PI,
 /*StoreInMemory=*/true,
+/*Stats=*/nullptr,
 [&](ASTContext &Ctx, Preprocessor &PP,
 const CanonicalIncludes &CanonIncludes) {
   EXPECT_FALSE(IndexUpdated)
Index: clang-tools-extra/clangd/unittests/CodeCompleteTests.cpp
===
--- clang-tools-extra/clangd/unittests/CodeCompleteTests.cpp
+++ clang-tools-extra/clangd/unittests/CodeCompleteTests.cpp
@@ -121,8 +121,9 @@
 ADD_FAILURE() << "Couldn't build CompilerInvocation";
 return {};
   }
-  auto Preamble = buildPreamble(testPath(TU.Filename), *CI, Inputs,
-  

[PATCH] D121712: [clangd] Track time spent in filesystem ops during preamble builds

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added a comment.

In D121712#3383944 , @Trass3r wrote:

> Hmm just a few curious questions from the sidelines.
> Why a "custom system" instead of something -ftime-trace based?

I'm not sure if I understand. -ftime-trace is a one-off. I want to monitor 
these values across multiple clangd instances, over time.

> How much overhead does this introduce, esp. for normal use-cases?

On my machine it takes about 3.5 milliseconds to do 100k startTimer(); 
stopTimer(); calls in a loop. Building preamble for SemaOverload.cpp (random 
file I checked) takes about ~2200 such calls. Basically the impact of this 
should not be noticeable.

> What's the gain? The information is very coarse and general-purpose system 
> profiling tools should give you much better information regarding file system 
> perf.

The idea is to use this to monitor large deployments of clangd on many machines 
over long period of time. One question to answer is whether some sort of 
prefetching of header files (with parallel I/O) would be beneficial. That 
depends on what fraction of the time we spend waiting on the serial file reads 
and how warm the caches are, on average.
It will also help us catch regressions, like one we had recently (that went 
unnoticed for quite a while) where adding some -fmodule-map-file flags to do a 
layering check caused a lot of stat() calls, which can be quite expensive.

Does that make sense? Does it answer your questions?




Comment at: clang-tools-extra/clangd/FS.h:71
 
+/// Used for tracking time spent in FS operations. Like llvm::Timer, but only
+/// tracks wall time, which is much cheaper.

kadircet wrote:
> kadircet wrote:
> > I don't think mentioning `FS` here is worth it. Can we rather emphasize on 
> > this being an accumulating timer?
> what about making all of this an implementation detail of `Preamble.cpp` ? we 
> can lift it up once we have other places that we want to make use of.
Before I make the change, let me clarify: are you suggesting moving the whole 
TimedFS into Preamble.cpp?

Definitely possible.



Comment at: clang-tools-extra/clangd/FS.h:73
+/// tracks wall time, which is much cheaper.
+/// Since this is a generic timer, We may want to move this to support/ if we
+/// find a use case outside of FS time tracking.

kadircet wrote:
> maybe put a `FIXME:`
IMHO FIXME should be actionable. This is not - we shouldn't do it unless 
something changes. It's just a note that if someone ever wonders "could I just 
move this to support/ and re-use it", the answer is yes.

Does that make sense?



Comment at: clang-tools-extra/clangd/FS.h:82
+  void stopTime();
+  // Return total time, in seconds.
+  double getTime();

kadircet wrote:
> what about returning an `int` with `ms` granularity ?
We could, but why? llvm::Timer returns WallTime as double so this is somewhat 
consistent with it. Not a strong argument, of course, since that consistency 
doesn't really matter, but I'm not sure what benefit using int here offers?



Comment at: clang-tools-extra/clangd/FS.h:86
+private:
+  std::chrono::steady_clock::duration TotalTime;
+  std::chrono::steady_clock::time_point StartTime;

kadircet wrote:
> what about just storing double/int ?
Again, why?

(I'm not strictly against it, just trying to understand why you're asking for 
this)



Comment at: clang-tools-extra/clangd/FS.h:91
+IntrusiveRefCntPtr
+getTimeTrackingFS(std::shared_ptr Timer,
+  IntrusiveRefCntPtr FS);

kadircet wrote:
> kadircet wrote:
> > i feel like this deserves the `FS` specific comment mentioned above. maybe 
> > something like:
> > ```
> > This will record all time spent on IO operations in \p Timer.
> > ```
> i don't think we ever want concurrent access to Timer, i.e. startTime should 
> never be called without a matching call to stopTime first. passing it in as a 
> shared_ptr somehow gives the feeling that it might be shared across multiple 
> objects, which might do whatever they want with the object.
> maybe just pass in a reference ?
It's not about concurrency, it's about live time. This timer needs to have the 
same lifetime as the entire VFS, which is also ref-counted.

Alternative would be for the TimerFS to own this timer and expose it's own 
getTime() method. That means TimerFS must now be visible outside of FS.cpp, but 
if we're moving it to Preamble.cpp per your other comment that's fine.



Comment at: clang-tools-extra/clangd/Preamble.cpp:385
   auto BuiltPreamble = PrecompiledPreamble::Build(
-  CI, ContentsBuffer.get(), Bounds, *PreambleDiagsEngine,
-  StatCache->getProducingFS(VFS),
+  CI, ContentsBuffer.get(), Bounds, *PreambleDiagsEngine, TimedFS,
   std::make_shared(), StoreInMemory, CapturedInfo);

kadircet wrote:
> it might be worth leaving some

[PATCH] D121824: [clang] Do not crash on arrow operator on dependent type.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 416561.
adamcz added a comment.

changed to marking VarDecl as invalid if it's ref type and initializer is 
invalid


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D121824/new/

https://reviews.llvm.org/D121824

Files:
  clang/lib/Sema/SemaDecl.cpp
  clang/test/AST/ast-dump-invalid-initialized.cpp
  clang/test/SemaCXX/arrow-operator.cpp


Index: clang/test/SemaCXX/arrow-operator.cpp
===
--- clang/test/SemaCXX/arrow-operator.cpp
+++ clang/test/SemaCXX/arrow-operator.cpp
@@ -65,3 +65,24 @@
 }
 
 } // namespace arrow_suggest
+
+namespace no_crash_dependent_type {
+
+template
+struct A {
+  void call();
+  A* operator->();
+};
+
+template 
+void foo() {
+  // x is dependent.
+  A& x = blah[7];  // expected-error {{use of undeclared identifier 
'blah'}}
+  x->call();
+}
+
+void test() {
+  foo();
+}
+
+} // namespace no_crash_dependent_type
Index: clang/test/AST/ast-dump-invalid-initialized.cpp
===
--- clang/test/AST/ast-dump-invalid-initialized.cpp
+++ clang/test/AST/ast-dump-invalid-initialized.cpp
@@ -24,4 +24,6 @@
   auto b4 = A(1);
   // CHECK: `-VarDecl {{.*}} invalid b5 'auto'
   auto b5 = A{1};
-}
\ No newline at end of file
+  // CHECK: `-VarDecl {{.*}} invalid b6 'A &'
+  A& b6 = garbage[1];
+}
Index: clang/lib/Sema/SemaDecl.cpp
===
--- clang/lib/Sema/SemaDecl.cpp
+++ clang/lib/Sema/SemaDecl.cpp
@@ -12794,6 +12794,12 @@
 return;
   }
 
+  // Reference type without initializer doesn't work.
+  if (VD->getType()->isReferenceType()) {
+VD->setInvalidDecl();
+return;
+  }
+
   QualType Ty = VD->getType();
   if (Ty->isDependentType()) return;
 


Index: clang/test/SemaCXX/arrow-operator.cpp
===
--- clang/test/SemaCXX/arrow-operator.cpp
+++ clang/test/SemaCXX/arrow-operator.cpp
@@ -65,3 +65,24 @@
 }
 
 } // namespace arrow_suggest
+
+namespace no_crash_dependent_type {
+
+template
+struct A {
+  void call();
+  A* operator->();
+};
+
+template 
+void foo() {
+  // x is dependent.
+  A& x = blah[7];  // expected-error {{use of undeclared identifier 'blah'}}
+  x->call();
+}
+
+void test() {
+  foo();
+}
+
+} // namespace no_crash_dependent_type
Index: clang/test/AST/ast-dump-invalid-initialized.cpp
===
--- clang/test/AST/ast-dump-invalid-initialized.cpp
+++ clang/test/AST/ast-dump-invalid-initialized.cpp
@@ -24,4 +24,6 @@
   auto b4 = A(1);
   // CHECK: `-VarDecl {{.*}} invalid b5 'auto'
   auto b5 = A{1};
-}
\ No newline at end of file
+  // CHECK: `-VarDecl {{.*}} invalid b6 'A &'
+  A& b6 = garbage[1];
+}
Index: clang/lib/Sema/SemaDecl.cpp
===
--- clang/lib/Sema/SemaDecl.cpp
+++ clang/lib/Sema/SemaDecl.cpp
@@ -12794,6 +12794,12 @@
 return;
   }
 
+  // Reference type without initializer doesn't work.
+  if (VD->getType()->isReferenceType()) {
+VD->setInvalidDecl();
+return;
+  }
+
   QualType Ty = VD->getType();
   if (Ty->isDependentType()) return;
 
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D121824: [clang] Do not crash on arrow operator on dependent type.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added a comment.

Thanks!

I updated the change. Let me know if this is what you had in mind.
I kept the original test too, can't hurt right?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D121824/new/

https://reviews.llvm.org/D121824

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D121712: [clangd] Track time spent in filesystem ops during preamble builds

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 416964.
adamcz marked 3 inline comments as done.
adamcz added a comment.

more review comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D121712/new/

https://reviews.llvm.org/D121712

Files:
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/TUScheduler.cpp

Index: clang-tools-extra/clangd/TUScheduler.cpp
===
--- clang-tools-extra/clangd/TUScheduler.cpp
+++ clang-tools-extra/clangd/TUScheduler.cpp
@@ -98,6 +98,33 @@
 using std::chrono::steady_clock;
 
 namespace {
+// Tracks latency (in seconds) of FS operations done during a preamble build.
+// build_type allows to split by expected VFS cache state (cold on first
+// preamble, somewhat warm after that when building first preamble for new file,
+// likely ~everything cached on preamble rebuild.
+constexpr trace::Metric
+PreambleBuildFilesystemLatency("preamble_fs_latency",
+   trace::Metric::Distribution, "build_type");
+// Tracks latency of FS operations done during a preamble build as a ratio of
+// preamble build time. build_type is same as above.
+constexpr trace::Metric PreambleBuildFilesystemLatencyRatio(
+"preamble_fs_latency_ratio", trace::Metric::Distribution, "build_type");
+
+void reportPreambleBuild(const PreambleBuildStats &Stats,
+ bool IsFirstPreamble) {
+  static llvm::once_flag OnceFlag;
+  llvm::call_once(OnceFlag, [&] {
+PreambleBuildFilesystemLatency.record(Stats.FileSystemTime, "first_build");
+  });
+
+  const std::string Label =
+  IsFirstPreamble ? "first_build_for_file" : "rebuild";
+  PreambleBuildFilesystemLatency.record(Stats.FileSystemTime, Label);
+  if (Stats.TotalBuildTime > 0) // Avoid division by zero.
+PreambleBuildFilesystemLatencyRatio.record(
+Stats.FileSystemTime / Stats.TotalBuildTime, Label);
+}
+
 class ASTWorker;
 } // namespace
 
@@ -975,13 +1002,19 @@
 crashDumpParseInputs(llvm::errs(), Inputs);
   });
 
+  PreambleBuildStats Stats;
+  bool IsFirstPreamble = !LatestBuild;
   LatestBuild = clang::clangd::buildPreamble(
   FileName, *Req.CI, Inputs, StoreInMemory,
   [this, Version(Inputs.Version)](ASTContext &Ctx, Preprocessor &PP,
   const CanonicalIncludes &CanonIncludes) {
 Callbacks.onPreambleAST(FileName, Version, Ctx, PP, CanonIncludes);
-  });
-  if (LatestBuild && isReliable(LatestBuild->CompileCommand))
+  },
+  &Stats);
+  if (!LatestBuild)
+return;
+  reportPreambleBuild(Stats, IsFirstPreamble);
+  if (isReliable(LatestBuild->CompileCommand))
 HeaderIncluders.update(FileName, LatestBuild->Includes.allHeaders());
 }
 
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -75,13 +75,25 @@
 using PreambleParsedCallback = std::function;
 
+/// Timings and statistics from the premble build. Unlike PreambleData, these
+/// do not need to be stored for later, but can be useful for logging, metrics,
+/// etc.
+struct PreambleBuildStats {
+  /// Total wall time it took to build preamble, in seconds.
+  double TotalBuildTime;
+  /// Time spent in filesystem operations during the build, in seconds.
+  double FileSystemTime;
+};
+
 /// Build a preamble for the new inputs unless an old one can be reused.
 /// If \p PreambleCallback is set, it will be run on top of the AST while
 /// building the preamble.
+/// If Stats is not non-null, build statistics will be exported there.
 std::shared_ptr
 buildPreamble(PathRef FileName, CompilerInvocation CI,
   const ParseInputs &Inputs, bool StoreInMemory,
-  PreambleParsedCallback PreambleCallback);
+  PreambleParsedCallback PreambleCallback,
+  PreambleBuildStats *Stats = nullptr);
 
 /// Returns true if \p Preamble is reusable for \p Inputs. Note that it will
 /// return true when some missing headers are now available.
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -312,12 +312,98 @@
   return FE && *FE == SM.getFileEntryForID(SM.getMainFileID());
 }
 
+/// Accumulating wall time timer. Similar to llvm::Timer, but much cheaper,
+/// it only tracks wall time.
+/// Since this is a generic timer, We may want to move this to support/ if we
+/// find a use case outside of FS time tracking.
+class WallTimer {
+public:
+  WallTimer() : TotalTime(std::chrono::steady_clock::duration::zero()) {}
+  /// [Re-]Start the timer.
+  void startTimer() { StartTime = std::chrono::steady_clock::now(); }
+  /// Stop the timer and update total time.
+  void stopTimer() {
+TotalTime += std::chr

[PATCH] D121712: [clangd] Track time spent in filesystem ops during preamble builds

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/FS.h:91
+IntrusiveRefCntPtr
+getTimeTrackingFS(std::shared_ptr Timer,
+  IntrusiveRefCntPtr FS);

kadircet wrote:
> adamcz wrote:
> > kadircet wrote:
> > > kadircet wrote:
> > > > i feel like this deserves the `FS` specific comment mentioned above. 
> > > > maybe something like:
> > > > ```
> > > > This will record all time spent on IO operations in \p Timer.
> > > > ```
> > > i don't think we ever want concurrent access to Timer, i.e. startTime 
> > > should never be called without a matching call to stopTime first. passing 
> > > it in as a shared_ptr somehow gives the feeling that it might be shared 
> > > across multiple objects, which might do whatever they want with the 
> > > object.
> > > maybe just pass in a reference ?
> > It's not about concurrency, it's about live time. This timer needs to have 
> > the same lifetime as the entire VFS, which is also ref-counted.
> > 
> > Alternative would be for the TimerFS to own this timer and expose it's own 
> > getTime() method. That means TimerFS must now be visible outside of FS.cpp, 
> > but if we're moving it to Preamble.cpp per your other comment that's fine.
> > It's not about concurrency, it's about live time. This timer needs to have 
> > the same lifetime as the entire VFS, which is also ref-counted.
> 
> Right, I've noticed that as I was going through the rest of the review, but 
> forgot to delete these as it was split into two days. sorry for the churn.
> 
> > Alternative would be for the TimerFS to own this timer and expose it's own 
> > getTime() method. That means TimerFS must now be visible outside of FS.cpp, 
> > but if we're moving it to Preamble.cpp per your other comment that's fine.
> 
> yes, i think that would be a nice simplification.
> > It's not about concurrency, it's about live time. This timer needs to have 
> > the same lifetime as the entire VFS, which is also ref-counted.
> 
> Right, I've noticed that as I was going through the rest of the review, but 
> forgot to delete these as it was split into two days. sorry for the churn.
> 
> > Alternative would be for the TimerFS to own this timer and expose it's own 
> > getTime() method. That means TimerFS must now be visible outside of FS.cpp, 
> > but if we're moving it to Preamble.cpp per your other comment that's fine.
> 
> yes, i think that would be a nice simplification.

Done.



Comment at: clang-tools-extra/clangd/Preamble.cpp:389
+  Stats != nullptr ? getTimeTrackingFS(FSTimer, StatCacheFS) : StatCacheFS;
+  FSTimer->stopTimer();
+

kadircet wrote:
> is this call intentional ?
Oops, no, it's not.




Comment at: clang-tools-extra/clangd/Preamble.h:95
   const ParseInputs &Inputs, bool StoreInMemory,
+  PreambleBuildStats *Stats,
   PreambleParsedCallback PreambleCallback);

kadircet wrote:
> adamcz wrote:
> > kadircet wrote:
> > > nit: maybe make this the last parameter and default to `nullptr` to get 
> > > rid of changes in tests.
> > I'd rather not, unless you insist. Besides not having to modify tests 
> > (which I already did anyway), what's the benefit of having it be default? 
> > Do you think it's more readable?
> right, i think it's more readable, and moreover it will reduce the need for 
> typing that parameter more times in the future (mostly in tests again).
> at the very least, what's the reason for not inserting it at the last 
> position but rather before PreambleCallback?
OK, I made it default to nullptr.

The logic behind it not being last was that it's usual (though not a hard rule) 
for callbacks to be the last argument, probably to make passing lambdas look 
nicer. Not really important.



Comment at: clang-tools-extra/clangd/TUScheduler.cpp:1008
+  PreambleBuildStats Stats;
+  const bool IsFirstPreamble = !LatestBuild;
   LatestBuild = clang::clangd::buildPreamble(

kadircet wrote:
> adamcz wrote:
> > kadircet wrote:
> > > nit: drop the const per style.
> > Hmm...is that actually documented somewhere? There's definitely many cases 
> > of "const bool" in LLVM codebase. I think the "const" improves readability.
> > is that actually documented somewhere
> 
> nothing that I can find either.
> 
> > There's definitely many cases of "const bool" in LLVM codebase. I think the 
> > "const" improves readability.
> 
> yes, but I think the majority is still not having "const" in front of bool. 
> it's at least the convention in clangd.
> I also agree that the `const` improves readability, but for a local variable 
> it carries much less importance and being consistent with the majority of the 
> cases here is a lot more important.
> because seeing occurrences of both locals with const and non-const will 
> eventually make the reasoning hard and each case surprising.
> 
> if you think there's enough value to having

[PATCH] D121712: [clangd] Track time spent in filesystem ops during preamble builds

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 417006.
adamcz marked an inline comment as done.
adamcz added a comment.

fix comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D121712/new/

https://reviews.llvm.org/D121712

Files:
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/TUScheduler.cpp

Index: clang-tools-extra/clangd/TUScheduler.cpp
===
--- clang-tools-extra/clangd/TUScheduler.cpp
+++ clang-tools-extra/clangd/TUScheduler.cpp
@@ -98,6 +98,33 @@
 using std::chrono::steady_clock;
 
 namespace {
+// Tracks latency (in seconds) of FS operations done during a preamble build.
+// build_type allows to split by expected VFS cache state (cold on first
+// preamble, somewhat warm after that when building first preamble for new file,
+// likely ~everything cached on preamble rebuild.
+constexpr trace::Metric
+PreambleBuildFilesystemLatency("preamble_fs_latency",
+   trace::Metric::Distribution, "build_type");
+// Tracks latency of FS operations done during a preamble build as a ratio of
+// preamble build time. build_type is same as above.
+constexpr trace::Metric PreambleBuildFilesystemLatencyRatio(
+"preamble_fs_latency_ratio", trace::Metric::Distribution, "build_type");
+
+void reportPreambleBuild(const PreambleBuildStats &Stats,
+ bool IsFirstPreamble) {
+  static llvm::once_flag OnceFlag;
+  llvm::call_once(OnceFlag, [&] {
+PreambleBuildFilesystemLatency.record(Stats.FileSystemTime, "first_build");
+  });
+
+  const std::string Label =
+  IsFirstPreamble ? "first_build_for_file" : "rebuild";
+  PreambleBuildFilesystemLatency.record(Stats.FileSystemTime, Label);
+  if (Stats.TotalBuildTime > 0) // Avoid division by zero.
+PreambleBuildFilesystemLatencyRatio.record(
+Stats.FileSystemTime / Stats.TotalBuildTime, Label);
+}
+
 class ASTWorker;
 } // namespace
 
@@ -975,13 +1002,19 @@
 crashDumpParseInputs(llvm::errs(), Inputs);
   });
 
+  PreambleBuildStats Stats;
+  bool IsFirstPreamble = !LatestBuild;
   LatestBuild = clang::clangd::buildPreamble(
   FileName, *Req.CI, Inputs, StoreInMemory,
   [this, Version(Inputs.Version)](ASTContext &Ctx, Preprocessor &PP,
   const CanonicalIncludes &CanonIncludes) {
 Callbacks.onPreambleAST(FileName, Version, Ctx, PP, CanonIncludes);
-  });
-  if (LatestBuild && isReliable(LatestBuild->CompileCommand))
+  },
+  &Stats);
+  if (!LatestBuild)
+return;
+  reportPreambleBuild(Stats, IsFirstPreamble);
+  if (isReliable(LatestBuild->CompileCommand))
 HeaderIncluders.update(FileName, LatestBuild->Includes.allHeaders());
 }
 
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -75,13 +75,25 @@
 using PreambleParsedCallback = std::function;
 
+/// Timings and statistics from the premble build. Unlike PreambleData, these
+/// do not need to be stored for later, but can be useful for logging, metrics,
+/// etc.
+struct PreambleBuildStats {
+  /// Total wall time it took to build preamble, in seconds.
+  double TotalBuildTime;
+  /// Time spent in filesystem operations during the build, in seconds.
+  double FileSystemTime;
+};
+
 /// Build a preamble for the new inputs unless an old one can be reused.
 /// If \p PreambleCallback is set, it will be run on top of the AST while
 /// building the preamble.
+/// If Stats is not non-null, build statistics will be exported there.
 std::shared_ptr
 buildPreamble(PathRef FileName, CompilerInvocation CI,
   const ParseInputs &Inputs, bool StoreInMemory,
-  PreambleParsedCallback PreambleCallback);
+  PreambleParsedCallback PreambleCallback,
+  PreambleBuildStats *Stats = nullptr);
 
 /// Returns true if \p Preamble is reusable for \p Inputs. Note that it will
 /// return true when some missing headers are now available.
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -312,12 +312,98 @@
   return FE && *FE == SM.getFileEntryForID(SM.getMainFileID());
 }
 
+// Accumulating wall time timer. Similar to llvm::Timer, but much cheaper,
+// it only tracks wall time.
+// Since this is a generic timer, We may want to move this to support/ if we
+// find a use case outside of FS time tracking.
+class WallTimer {
+public:
+  WallTimer() : TotalTime(std::chrono::steady_clock::duration::zero()) {}
+  // [Re-]Start the timer.
+  void startTimer() { StartTime = std::chrono::steady_clock::now(); }
+  // Stop the timer and update total time.
+  void stopTimer() {
+TotalTime += std::chrono::steady_cl

[PATCH] D121712: [clangd] Track time spent in filesystem ops during preamble builds

[Adam Czachorowski via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rG6009d0d5801d: [clangd] Track time spent in filesystem ops 
during preamble builds (authored by adamcz).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D121712/new/

https://reviews.llvm.org/D121712

Files:
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/TUScheduler.cpp

Index: clang-tools-extra/clangd/TUScheduler.cpp
===
--- clang-tools-extra/clangd/TUScheduler.cpp
+++ clang-tools-extra/clangd/TUScheduler.cpp
@@ -98,6 +98,33 @@
 using std::chrono::steady_clock;
 
 namespace {
+// Tracks latency (in seconds) of FS operations done during a preamble build.
+// build_type allows to split by expected VFS cache state (cold on first
+// preamble, somewhat warm after that when building first preamble for new file,
+// likely ~everything cached on preamble rebuild.
+constexpr trace::Metric
+PreambleBuildFilesystemLatency("preamble_fs_latency",
+   trace::Metric::Distribution, "build_type");
+// Tracks latency of FS operations done during a preamble build as a ratio of
+// preamble build time. build_type is same as above.
+constexpr trace::Metric PreambleBuildFilesystemLatencyRatio(
+"preamble_fs_latency_ratio", trace::Metric::Distribution, "build_type");
+
+void reportPreambleBuild(const PreambleBuildStats &Stats,
+ bool IsFirstPreamble) {
+  static llvm::once_flag OnceFlag;
+  llvm::call_once(OnceFlag, [&] {
+PreambleBuildFilesystemLatency.record(Stats.FileSystemTime, "first_build");
+  });
+
+  const std::string Label =
+  IsFirstPreamble ? "first_build_for_file" : "rebuild";
+  PreambleBuildFilesystemLatency.record(Stats.FileSystemTime, Label);
+  if (Stats.TotalBuildTime > 0) // Avoid division by zero.
+PreambleBuildFilesystemLatencyRatio.record(
+Stats.FileSystemTime / Stats.TotalBuildTime, Label);
+}
+
 class ASTWorker;
 } // namespace
 
@@ -975,13 +1002,19 @@
 crashDumpParseInputs(llvm::errs(), Inputs);
   });
 
+  PreambleBuildStats Stats;
+  bool IsFirstPreamble = !LatestBuild;
   LatestBuild = clang::clangd::buildPreamble(
   FileName, *Req.CI, Inputs, StoreInMemory,
   [this, Version(Inputs.Version)](ASTContext &Ctx, Preprocessor &PP,
   const CanonicalIncludes &CanonIncludes) {
 Callbacks.onPreambleAST(FileName, Version, Ctx, PP, CanonIncludes);
-  });
-  if (LatestBuild && isReliable(LatestBuild->CompileCommand))
+  },
+  &Stats);
+  if (!LatestBuild)
+return;
+  reportPreambleBuild(Stats, IsFirstPreamble);
+  if (isReliable(LatestBuild->CompileCommand))
 HeaderIncluders.update(FileName, LatestBuild->Includes.allHeaders());
 }
 
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -75,13 +75,25 @@
 using PreambleParsedCallback = std::function;
 
+/// Timings and statistics from the premble build. Unlike PreambleData, these
+/// do not need to be stored for later, but can be useful for logging, metrics,
+/// etc.
+struct PreambleBuildStats {
+  /// Total wall time it took to build preamble, in seconds.
+  double TotalBuildTime;
+  /// Time spent in filesystem operations during the build, in seconds.
+  double FileSystemTime;
+};
+
 /// Build a preamble for the new inputs unless an old one can be reused.
 /// If \p PreambleCallback is set, it will be run on top of the AST while
 /// building the preamble.
+/// If Stats is not non-null, build statistics will be exported there.
 std::shared_ptr
 buildPreamble(PathRef FileName, CompilerInvocation CI,
   const ParseInputs &Inputs, bool StoreInMemory,
-  PreambleParsedCallback PreambleCallback);
+  PreambleParsedCallback PreambleCallback,
+  PreambleBuildStats *Stats = nullptr);
 
 /// Returns true if \p Preamble is reusable for \p Inputs. Note that it will
 /// return true when some missing headers are now available.
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -312,12 +312,98 @@
   return FE && *FE == SM.getFileEntryForID(SM.getMainFileID());
 }
 
+// Accumulating wall time timer. Similar to llvm::Timer, but much cheaper,
+// it only tracks wall time.
+// Since this is a generic timer, We may want to move this to support/ if we
+// find a use case outside of FS time tracking.
+class WallTimer {
+public:
+  WallTimer() : TotalTime(std::chrono::steady_clock::duration::zero()) {}
+  // [Re-]Start the timer.
+  void startTimer() { StartTime = std::chrono::s

[PATCH] D122102: [clangd] Introduce "add subclass" tweak

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added a comment.

Thanks, this looks really good! I always wanted a code action like this.

I always envisioned this as a code action on a class to implement missing 
virtual methods though. The idea would be:

  class Foo : public B^ar { };

expands into:
class Foo : public Bar {

  int baz() overrides { return Bar::baz(); }

};

This has several advantages:

- no need to rename (the user already provided the name)
- no need to move the code (it adds the code in appropriate place)
- in situations like this change, where you implement a subclass of Tweak, 
there's no need to navigate to the Tweak.h header file (which could be outside 
of your project, maybe even read-only, or could just #include[_next] something 
else) and you don't need to wait for the AST build for the header file when 
you're just going to move it to another file anyway.

Have you considered an approach like this? Obviously it's a bit more 
complicated, but I think it would make this tweak even better. The way it's 
triggered now has it's use cases too, of course. I wonder if we could share the 
code between two use cases like this.

Ideally I'd like to see this code action show up as code completion option as 
well. Something like:

  class Foo : public Bar {^

could result in "add virtual methods" as #1 option, greatly increasing 
discoverability, but that's a whole different problem and not specific to this 
tweak (PopulateSwitch would be great candidate for this too).




Comment at: clang-tools-extra/clangd/refactor/tweaks/AddSubclass.cpp:1
+//===--- ExpandAutoType.cpp --*- 
C++-*-===//
+//

Wrong file name in the comment.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D122102/new/

https://reviews.llvm.org/D122102

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D121824: [clang] Do not crash on arrow operator on dependent type.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 417331.
adamcz added a comment.

Reverted to previous version + new test


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D121824/new/

https://reviews.llvm.org/D121824

Files:
  clang/lib/Sema/TreeTransform.h
  clang/test/SemaCXX/arrow-operator.cpp


Index: clang/test/SemaCXX/arrow-operator.cpp
===
--- clang/test/SemaCXX/arrow-operator.cpp
+++ clang/test/SemaCXX/arrow-operator.cpp
@@ -65,3 +65,51 @@
 }
 
 } // namespace arrow_suggest
+
+namespace no_crash_dependent_type {
+
+template
+struct A {
+  void call();
+  A* operator->();
+};
+
+template 
+void foo() {
+  // The "requires an initializer" error seems unnecessary.
+  A& x = blah[7];  // expected-error {{use of undeclared identifier 
'blah'}} \
+// expected-error {{requires an initializer}}
+  // x is dependent.
+  x->call();
+}
+
+void test() {
+  foo();  // expected-note {{requested here}}
+}
+
+} // namespace no_crash_dependent_type
+
+namespace clangd_issue_1073_no_crash_dependent_type {
+
+template  struct Ptr {
+  T *operator->();
+};
+
+struct Struct {
+  int len;
+};
+
+template 
+struct TemplateStruct {
+  Ptr val();  // expected-note {{declared here}}
+};
+
+template 
+void templateFunc(const TemplateStruct &ts) {
+  Ptr ptr = ts.val(); // expected-error {{function is not marked 
const}}
+  auto foo = ptr->len;
+}
+
+template void templateFunc<0>(const TemplateStruct<0> &);  // expected-note 
{{requested here}}
+
+} // namespace clangd_issue_1073_no_crash_dependent_type
Index: clang/lib/Sema/TreeTransform.h
===
--- clang/lib/Sema/TreeTransform.h
+++ clang/lib/Sema/TreeTransform.h
@@ -14705,6 +14705,8 @@
   return getSema().CreateBuiltinArraySubscriptExpr(
   First, Callee->getBeginLoc(), Second, OpLoc);
   } else if (Op == OO_Arrow) {
+if (First->getType()->isDependentType())
+  return ExprError();
 // -> is never a builtin operation.
 return SemaRef.BuildOverloadedArrowExpr(nullptr, First, OpLoc);
   } else if (Second == nullptr || isPostIncDec) {


Index: clang/test/SemaCXX/arrow-operator.cpp
===
--- clang/test/SemaCXX/arrow-operator.cpp
+++ clang/test/SemaCXX/arrow-operator.cpp
@@ -65,3 +65,51 @@
 }
 
 } // namespace arrow_suggest
+
+namespace no_crash_dependent_type {
+
+template
+struct A {
+  void call();
+  A* operator->();
+};
+
+template 
+void foo() {
+  // The "requires an initializer" error seems unnecessary.
+  A& x = blah[7];  // expected-error {{use of undeclared identifier 'blah'}} \
+// expected-error {{requires an initializer}}
+  // x is dependent.
+  x->call();
+}
+
+void test() {
+  foo();  // expected-note {{requested here}}
+}
+
+} // namespace no_crash_dependent_type
+
+namespace clangd_issue_1073_no_crash_dependent_type {
+
+template  struct Ptr {
+  T *operator->();
+};
+
+struct Struct {
+  int len;
+};
+
+template 
+struct TemplateStruct {
+  Ptr val();  // expected-note {{declared here}}
+};
+
+template 
+void templateFunc(const TemplateStruct &ts) {
+  Ptr ptr = ts.val(); // expected-error {{function is not marked const}}
+  auto foo = ptr->len;
+}
+
+template void templateFunc<0>(const TemplateStruct<0> &);  // expected-note {{requested here}}
+
+} // namespace clangd_issue_1073_no_crash_dependent_type
Index: clang/lib/Sema/TreeTransform.h
===
--- clang/lib/Sema/TreeTransform.h
+++ clang/lib/Sema/TreeTransform.h
@@ -14705,6 +14705,8 @@
   return getSema().CreateBuiltinArraySubscriptExpr(
   First, Callee->getBeginLoc(), Second, OpLoc);
   } else if (Op == OO_Arrow) {
+if (First->getType()->isDependentType())
+  return ExprError();
 // -> is never a builtin operation.
 return SemaRef.BuildOverloadedArrowExpr(nullptr, First, OpLoc);
   } else if (Second == nullptr || isPostIncDec) {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D121824: [clang] Do not crash on arrow operator on dependent type.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added a comment.

In D121824#3399208 , @sammccall wrote:

> I reduced something very similar recently as 
> https://github.com/clangd/clangd/issues/1073
>
> This patch does not fix it, but looks closely related, want to take a look?

Very similar. My original fix would've fixed that too. I'm reverting this 
change to that version and adding this as a test.

Based on my conversation with Sam I'm also reverting the 
mark-ref-VarDecl-as-invalid-when-on-invalid-init, as I no longer believe this 
is appropriate. The idea is that, init or not, we know the type of VarDecl so 
it's not invalid (unless it's auto or something).

PTAL


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D121824/new/

https://reviews.llvm.org/D121824

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D123672: [clangd] Export preamble AST and serialized size as metrics

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/Preamble.cpp:500
 
+  if (Stats != nullptr) {
+Stats->TotalBuildTime = PreambleTimer.getTime();

This is a significant change. You are now exporting this information for failed 
preamble builds. Do you think these are interesting? It might make the data a 
bit confusing (i.e. lots of small "built in <1s, did not read any files) when 
something is really wrong with preamble.

I think we should either only export on successful preamble OR add a boolean 
"success" dimension to the metric.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D123672/new/

https://reviews.llvm.org/D123672

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D124176: [clangd] Add beforeExecute() callback to FeatureModules.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
adamcz added a reviewer: kadircet.
Herald added subscribers: usaxena95, arphaman.
Herald added a project: All.
adamcz requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

It runs immediatelly before FrontendAction::Execute() with a mutable
CompilerInstance, allowing FeatureModules to register callbacks, remap
files, etc.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D124176

Files:
  clang-tools-extra/clangd/FeatureModule.h
  clang-tools-extra/clangd/ParsedAST.cpp
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/unittests/FeatureModulesTests.cpp

Index: clang-tools-extra/clangd/unittests/FeatureModulesTests.cpp
===
--- clang-tools-extra/clangd/unittests/FeatureModulesTests.cpp
+++ clang-tools-extra/clangd/unittests/FeatureModulesTests.cpp
@@ -12,6 +12,7 @@
 #include "TestTU.h"
 #include "refactor/Tweak.h"
 #include "support/Logger.h"
+#include "clang/Lex/PreprocessorOptions.h"
 #include "llvm/Support/Error.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
@@ -85,6 +86,41 @@
   }
 }
 
+TEST(FeatureModulesTest, BeforeExecute) {
+  struct BeforeExecuteModule final : public FeatureModule {
+struct Listener : public FeatureModule::ASTListener {
+  void beforeExecute(CompilerInstance &CI) override {
+CI.getPreprocessor().SetSuppressIncludeNotFoundError(true);
+  }
+};
+std::unique_ptr astListeners() override {
+  return std::make_unique();
+};
+  };
+  FeatureModuleSet FMS;
+  FMS.add(std::make_unique());
+
+  TestTU TU = TestTU::withCode(R"cpp(
+/*error-ok*/
+#include "not_found.h"
+
+void foo() {
+  #include "not_found_not_preamble.h"
+}
+  )cpp");
+
+  {
+auto AST = TU.build();
+EXPECT_THAT(*AST.getDiagnostics(), testing::Not(testing::IsEmpty()));
+  }
+
+  TU.FeatureModules = &FMS;
+  {
+auto AST = TU.build();
+EXPECT_THAT(*AST.getDiagnostics(), testing::IsEmpty());
+  }
+}
+
 } // namespace
 } // namespace clangd
 } // namespace clang
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -64,9 +64,12 @@
 
 class CppFilePreambleCallbacks : public PreambleCallbacks {
 public:
-  CppFilePreambleCallbacks(PathRef File, PreambleParsedCallback ParsedCallback,
-   PreambleBuildStats *Stats)
-  : File(File), ParsedCallback(ParsedCallback), Stats(Stats) {}
+  CppFilePreambleCallbacks(
+  PathRef File, PreambleParsedCallback ParsedCallback,
+  PreambleBuildStats *Stats,
+  std::function BeforeExecuteCallback)
+  : File(File), ParsedCallback(ParsedCallback), Stats(Stats),
+BeforeExecuteCallback(BeforeExecuteCallback) {}
 
   IncludeStructure takeIncludes() { return std::move(Includes); }
 
@@ -111,6 +114,8 @@
   }
 
   void BeforeExecute(CompilerInstance &CI) override {
+if (BeforeExecuteCallback)
+  BeforeExecuteCallback(CI);
 CanonIncludes.addSystemHeadersMapping(CI.getLangOpts());
 LangOpts = &CI.getLangOpts();
 SourceMgr = &CI.getSourceManager();
@@ -156,6 +161,7 @@
   const clang::LangOptions *LangOpts = nullptr;
   const SourceManager *SourceMgr = nullptr;
   PreambleBuildStats *Stats;
+  std::function BeforeExecuteCallback;
 };
 
 // Represents directives other than includes, where basic textual information is
@@ -477,7 +483,12 @@
   // to read back. We rely on dynamic index for the comments instead.
   CI.getPreprocessorOpts().WriteCommentListToPCH = false;
 
-  CppFilePreambleCallbacks CapturedInfo(FileName, PreambleCallback, Stats);
+  CppFilePreambleCallbacks CapturedInfo(FileName, PreambleCallback, Stats,
+[&ASTListeners](CompilerInstance &CI) {
+  for (const auto &L : ASTListeners) {
+L->beforeExecute(CI);
+  }
+});
   auto VFS = Inputs.TFS->view(Inputs.CompileCommand.Directory);
   llvm::SmallString<32> AbsFileName(FileName);
   VFS->makeAbsolute(AbsFileName);
@@ -716,5 +727,6 @@
   }
   return Loc;
 }
+
 } // namespace clangd
 } // namespace clang
Index: clang-tools-extra/clangd/ParsedAST.cpp
===
--- clang-tools-extra/clangd/ParsedAST.cpp
+++ clang-tools-extra/clangd/ParsedAST.cpp
@@ -550,6 +550,9 @@
   // Collect tokens of the main file.
   syntax::TokenCollector CollectTokens(Clang->getPreprocessor());
 
+  for (const auto &L : ASTListeners)
+L->beforeExecute(*Clang);
+
   if (llvm::Error Err = Action->Execute())
 log("Execute() failed when building AST for {0}: {1}", MainInput.getFile(),
 toString(std::move(Err)));

[PATCH] D124177: [Frontend] Simplify PrecompiledPreamble::PCHStorage. NFC

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz accepted this revision.
adamcz added inline comments.
This revision is now accepted and ready to land.



Comment at: clang/lib/Frontend/PrecompiledPreamble.cpp:373
+  }
+  llvm::StringRef filePath() const { return File->getFilePath(); }
+  llvm::StringRef memoryContents() const { return *Memory; }

Maybe add asserts here and below to check that Kind is indeed File or Memory?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D124177/new/

https://reviews.llvm.org/D124177

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D124176: [clangd] Add beforeExecute() callback to FeatureModules.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 424215.
adamcz marked 5 inline comments as done.
adamcz added a comment.

addressed review comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D124176/new/

https://reviews.llvm.org/D124176

Files:
  clang-tools-extra/clangd/FeatureModule.h
  clang-tools-extra/clangd/ParsedAST.cpp
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/unittests/FeatureModulesTests.cpp

Index: clang-tools-extra/clangd/unittests/FeatureModulesTests.cpp
===
--- clang-tools-extra/clangd/unittests/FeatureModulesTests.cpp
+++ clang-tools-extra/clangd/unittests/FeatureModulesTests.cpp
@@ -12,6 +12,7 @@
 #include "TestTU.h"
 #include "refactor/Tweak.h"
 #include "support/Logger.h"
+#include "clang/Lex/PreprocessorOptions.h"
 #include "llvm/Support/Error.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
@@ -85,6 +86,41 @@
   }
 }
 
+TEST(FeatureModulesTest, BeforeExecute) {
+  struct BeforeExecuteModule final : public FeatureModule {
+struct Listener : public FeatureModule::ASTListener {
+  void beforeExecute(CompilerInstance &CI) override {
+CI.getPreprocessor().SetSuppressIncludeNotFoundError(true);
+  }
+};
+std::unique_ptr astListeners() override {
+  return std::make_unique();
+};
+  };
+  FeatureModuleSet FMS;
+  FMS.add(std::make_unique());
+
+  TestTU TU = TestTU::withCode(R"cpp(
+/*error-ok*/
+#include "not_found.h"
+
+void foo() {
+  #include "not_found_not_preamble.h"
+}
+  )cpp");
+
+  {
+auto AST = TU.build();
+EXPECT_THAT(*AST.getDiagnostics(), testing::Not(testing::IsEmpty()));
+  }
+
+  TU.FeatureModules = &FMS;
+  {
+auto AST = TU.build();
+EXPECT_THAT(*AST.getDiagnostics(), testing::IsEmpty());
+  }
+}
+
 } // namespace
 } // namespace clangd
 } // namespace clang
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -64,9 +64,12 @@
 
 class CppFilePreambleCallbacks : public PreambleCallbacks {
 public:
-  CppFilePreambleCallbacks(PathRef File, PreambleParsedCallback ParsedCallback,
-   PreambleBuildStats *Stats)
-  : File(File), ParsedCallback(ParsedCallback), Stats(Stats) {}
+  CppFilePreambleCallbacks(
+  PathRef File, PreambleParsedCallback ParsedCallback,
+  PreambleBuildStats *Stats,
+  std::function BeforeExecuteCallback)
+  : File(File), ParsedCallback(ParsedCallback), Stats(Stats),
+BeforeExecuteCallback(std::move(BeforeExecuteCallback)) {}
 
   IncludeStructure takeIncludes() { return std::move(Includes); }
 
@@ -115,6 +118,8 @@
 LangOpts = &CI.getLangOpts();
 SourceMgr = &CI.getSourceManager();
 Includes.collect(CI);
+if (BeforeExecuteCallback)
+  BeforeExecuteCallback(CI);
   }
 
   std::unique_ptr createPPCallbacks() override {
@@ -156,6 +161,7 @@
   const clang::LangOptions *LangOpts = nullptr;
   const SourceManager *SourceMgr = nullptr;
   PreambleBuildStats *Stats;
+  std::function BeforeExecuteCallback;
 };
 
 // Represents directives other than includes, where basic textual information is
@@ -477,7 +483,11 @@
   // to read back. We rely on dynamic index for the comments instead.
   CI.getPreprocessorOpts().WriteCommentListToPCH = false;
 
-  CppFilePreambleCallbacks CapturedInfo(FileName, PreambleCallback, Stats);
+  CppFilePreambleCallbacks CapturedInfo(FileName, PreambleCallback, Stats,
+[&ASTListeners](CompilerInstance &CI) {
+  for (const auto &L : ASTListeners)
+L->beforeExecute(CI);
+});
   auto VFS = Inputs.TFS->view(Inputs.CompileCommand.Directory);
   llvm::SmallString<32> AbsFileName(FileName);
   VFS->makeAbsolute(AbsFileName);
@@ -716,5 +726,6 @@
   }
   return Loc;
 }
+
 } // namespace clangd
 } // namespace clang
Index: clang-tools-extra/clangd/ParsedAST.cpp
===
--- clang-tools-extra/clangd/ParsedAST.cpp
+++ clang-tools-extra/clangd/ParsedAST.cpp
@@ -550,6 +550,12 @@
   // Collect tokens of the main file.
   syntax::TokenCollector CollectTokens(Clang->getPreprocessor());
 
+  // To remain consistent with preamble builds, these callbacks must be called
+  // exactly here, after preprocessor is initialized and BeginSourceFile() was
+  // called already.
+  for (const auto &L : ASTListeners)
+L->beforeExecute(*Clang);
+
   if (llvm::Error Err = Action->Execute())
 log("Execute() failed when building AST for {0}: {1}", MainInput.getFile(),
 toString(std::move(Err)));
Index: clang-tools-extra/clangd/FeatureModule.h
===
--- clang-tools

[PATCH] D124176: [clangd] Add beforeExecute() callback to FeatureModules.

[Adam Czachorowski via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rGad46aaede6e4: [clangd] Add beforeExecute() callback to 
FeatureModules. (authored by adamcz).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D124176/new/

https://reviews.llvm.org/D124176

Files:
  clang-tools-extra/clangd/FeatureModule.h
  clang-tools-extra/clangd/ParsedAST.cpp
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/unittests/FeatureModulesTests.cpp

Index: clang-tools-extra/clangd/unittests/FeatureModulesTests.cpp
===
--- clang-tools-extra/clangd/unittests/FeatureModulesTests.cpp
+++ clang-tools-extra/clangd/unittests/FeatureModulesTests.cpp
@@ -12,6 +12,7 @@
 #include "TestTU.h"
 #include "refactor/Tweak.h"
 #include "support/Logger.h"
+#include "clang/Lex/PreprocessorOptions.h"
 #include "llvm/Support/Error.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
@@ -85,6 +86,41 @@
   }
 }
 
+TEST(FeatureModulesTest, BeforeExecute) {
+  struct BeforeExecuteModule final : public FeatureModule {
+struct Listener : public FeatureModule::ASTListener {
+  void beforeExecute(CompilerInstance &CI) override {
+CI.getPreprocessor().SetSuppressIncludeNotFoundError(true);
+  }
+};
+std::unique_ptr astListeners() override {
+  return std::make_unique();
+};
+  };
+  FeatureModuleSet FMS;
+  FMS.add(std::make_unique());
+
+  TestTU TU = TestTU::withCode(R"cpp(
+/*error-ok*/
+#include "not_found.h"
+
+void foo() {
+  #include "not_found_not_preamble.h"
+}
+  )cpp");
+
+  {
+auto AST = TU.build();
+EXPECT_THAT(*AST.getDiagnostics(), testing::Not(testing::IsEmpty()));
+  }
+
+  TU.FeatureModules = &FMS;
+  {
+auto AST = TU.build();
+EXPECT_THAT(*AST.getDiagnostics(), testing::IsEmpty());
+  }
+}
+
 } // namespace
 } // namespace clangd
 } // namespace clang
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -64,9 +64,12 @@
 
 class CppFilePreambleCallbacks : public PreambleCallbacks {
 public:
-  CppFilePreambleCallbacks(PathRef File, PreambleParsedCallback ParsedCallback,
-   PreambleBuildStats *Stats)
-  : File(File), ParsedCallback(ParsedCallback), Stats(Stats) {}
+  CppFilePreambleCallbacks(
+  PathRef File, PreambleParsedCallback ParsedCallback,
+  PreambleBuildStats *Stats,
+  std::function BeforeExecuteCallback)
+  : File(File), ParsedCallback(ParsedCallback), Stats(Stats),
+BeforeExecuteCallback(std::move(BeforeExecuteCallback)) {}
 
   IncludeStructure takeIncludes() { return std::move(Includes); }
 
@@ -115,6 +118,8 @@
 LangOpts = &CI.getLangOpts();
 SourceMgr = &CI.getSourceManager();
 Includes.collect(CI);
+if (BeforeExecuteCallback)
+  BeforeExecuteCallback(CI);
   }
 
   std::unique_ptr createPPCallbacks() override {
@@ -156,6 +161,7 @@
   const clang::LangOptions *LangOpts = nullptr;
   const SourceManager *SourceMgr = nullptr;
   PreambleBuildStats *Stats;
+  std::function BeforeExecuteCallback;
 };
 
 // Represents directives other than includes, where basic textual information is
@@ -477,7 +483,11 @@
   // to read back. We rely on dynamic index for the comments instead.
   CI.getPreprocessorOpts().WriteCommentListToPCH = false;
 
-  CppFilePreambleCallbacks CapturedInfo(FileName, PreambleCallback, Stats);
+  CppFilePreambleCallbacks CapturedInfo(FileName, PreambleCallback, Stats,
+[&ASTListeners](CompilerInstance &CI) {
+  for (const auto &L : ASTListeners)
+L->beforeExecute(CI);
+});
   auto VFS = Inputs.TFS->view(Inputs.CompileCommand.Directory);
   llvm::SmallString<32> AbsFileName(FileName);
   VFS->makeAbsolute(AbsFileName);
@@ -716,5 +726,6 @@
   }
   return Loc;
 }
+
 } // namespace clangd
 } // namespace clang
Index: clang-tools-extra/clangd/ParsedAST.cpp
===
--- clang-tools-extra/clangd/ParsedAST.cpp
+++ clang-tools-extra/clangd/ParsedAST.cpp
@@ -550,6 +550,12 @@
   // Collect tokens of the main file.
   syntax::TokenCollector CollectTokens(Clang->getPreprocessor());
 
+  // To remain consistent with preamble builds, these callbacks must be called
+  // exactly here, after preprocessor is initialized and BeginSourceFile() was
+  // called already.
+  for (const auto &L : ASTListeners)
+L->beforeExecute(*Clang);
+
   if (llvm::Error Err = Action->Execute())
 log("Execute() failed when building AST for {0}: {1}", MainInput.getFile(),
 toString(std::move(Err)));
Index: clang-tools-extra/c

[PATCH] D129499: [clang] Do not crash on "requires" after a fatal error occurred.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
adamcz added a reviewer: ilya-biryukov.
Herald added a project: All.
adamcz requested review of this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.

The code would assume that SubstExpr() cannot fail on concept
specialization. This is incorret - we give up on some things after fatal
error occurred, since there's no value in doing futher work that the
user will not see anyway. In this case, this lead to crash.

The fatal error is simulated in tests with -ferror-limit=1, but this
could happen in other cases too.

Fixes https://github.com/llvm/llvm-project/issues/55401


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D129499

Files:
  clang/lib/Sema/SemaExprCXX.cpp
  clang/test/SemaCXX/concept-fatal-error.cpp


Index: clang/test/SemaCXX/concept-fatal-error.cpp
===
--- /dev/null
+++ clang/test/SemaCXX/concept-fatal-error.cpp
@@ -0,0 +1,11 @@
+// RUN: not %clang_cc1 -fsyntax-only -std=c++20 -ferror-limit 1 %s 2>&1 | 
FileCheck %s
+
+template 
+concept f = requires { 42; };
+struct h {
+  // The missing semicolon will trigger an error and -ferror-limit=1 will make 
it fatal
+  // We test that we do not crash in such cases (#55401)
+  int i = requires { { i } f } // expected-error {{expected ';' at end of 
declaration list}}
+
+  // CHECK: fatal error: too many errors emitted, stopping now
+};
Index: clang/lib/Sema/SemaExprCXX.cpp
===
--- clang/lib/Sema/SemaExprCXX.cpp
+++ clang/lib/Sema/SemaExprCXX.cpp
@@ -8971,14 +8971,14 @@
 cast(TPL->getParam(0))->getTypeConstraint()
 ->getImmediatelyDeclaredConstraint();
 ExprResult Constraint = SubstExpr(IDC, MLTAL);
-assert(!Constraint.isInvalid() &&
-   "Substitution cannot fail as it is simply putting a type template "
-   "argument into a concept specialization expression's parameter.");
-
-SubstitutedConstraintExpr =
-cast(Constraint.get());
-if (!SubstitutedConstraintExpr->isSatisfied())
-  Status = concepts::ExprRequirement::SS_ConstraintsNotSatisfied;
+if (Constraint.isInvalid()) {
+  Status = concepts::ExprRequirement::SS_ExprSubstitutionFailure;
+} else {
+  SubstitutedConstraintExpr =
+  cast(Constraint.get());
+  if (!SubstitutedConstraintExpr->isSatisfied())
+Status = concepts::ExprRequirement::SS_ConstraintsNotSatisfied;
+}
   }
   return new (Context) concepts::ExprRequirement(E, IsSimple, NoexceptLoc,
  ReturnTypeRequirement, Status,


Index: clang/test/SemaCXX/concept-fatal-error.cpp
===
--- /dev/null
+++ clang/test/SemaCXX/concept-fatal-error.cpp
@@ -0,0 +1,11 @@
+// RUN: not %clang_cc1 -fsyntax-only -std=c++20 -ferror-limit 1 %s 2>&1 | FileCheck %s
+
+template 
+concept f = requires { 42; };
+struct h {
+  // The missing semicolon will trigger an error and -ferror-limit=1 will make it fatal
+  // We test that we do not crash in such cases (#55401)
+  int i = requires { { i } f } // expected-error {{expected ';' at end of declaration list}}
+
+  // CHECK: fatal error: too many errors emitted, stopping now
+};
Index: clang/lib/Sema/SemaExprCXX.cpp
===
--- clang/lib/Sema/SemaExprCXX.cpp
+++ clang/lib/Sema/SemaExprCXX.cpp
@@ -8971,14 +8971,14 @@
 cast(TPL->getParam(0))->getTypeConstraint()
 ->getImmediatelyDeclaredConstraint();
 ExprResult Constraint = SubstExpr(IDC, MLTAL);
-assert(!Constraint.isInvalid() &&
-   "Substitution cannot fail as it is simply putting a type template "
-   "argument into a concept specialization expression's parameter.");
-
-SubstitutedConstraintExpr =
-cast(Constraint.get());
-if (!SubstitutedConstraintExpr->isSatisfied())
-  Status = concepts::ExprRequirement::SS_ConstraintsNotSatisfied;
+if (Constraint.isInvalid()) {
+  Status = concepts::ExprRequirement::SS_ExprSubstitutionFailure;
+} else {
+  SubstitutedConstraintExpr =
+  cast(Constraint.get());
+  if (!SubstitutedConstraintExpr->isSatisfied())
+Status = concepts::ExprRequirement::SS_ConstraintsNotSatisfied;
+}
   }
   return new (Context) concepts::ExprRequirement(E, IsSimple, NoexceptLoc,
  ReturnTypeRequirement, Status,
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D129499: [clang] Do not crash on "requires" after a fatal error occurred.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 444315.
adamcz added a comment.

improved the test


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D129499/new/

https://reviews.llvm.org/D129499

Files:
  clang/lib/Sema/SemaExprCXX.cpp
  clang/test/SemaCXX/concept-fatal-error.cpp


Index: clang/test/SemaCXX/concept-fatal-error.cpp
===
--- /dev/null
+++ clang/test/SemaCXX/concept-fatal-error.cpp
@@ -0,0 +1,10 @@
+// RUN: not %clang_cc1 -fsyntax-only -std=c++20 -ferror-limit 1 -verify %s
+
+template 
+concept f = requires { 42; };
+struct h {
+  // The missing semicolon will trigger an error and -ferror-limit=1 will make 
it fatal
+  // We test that we do not crash in such cases (#55401)
+  int i = requires { { i } f } // expected-error {{expected ';' at end of 
declaration list}}
+   // expected-error@* {{too many errros emitted}}
+};
Index: clang/lib/Sema/SemaExprCXX.cpp
===
--- clang/lib/Sema/SemaExprCXX.cpp
+++ clang/lib/Sema/SemaExprCXX.cpp
@@ -9006,14 +9006,14 @@
 cast(TPL->getParam(0))->getTypeConstraint()
 ->getImmediatelyDeclaredConstraint();
 ExprResult Constraint = SubstExpr(IDC, MLTAL);
-assert(!Constraint.isInvalid() &&
-   "Substitution cannot fail as it is simply putting a type template "
-   "argument into a concept specialization expression's parameter.");
-
-SubstitutedConstraintExpr =
-cast(Constraint.get());
-if (!SubstitutedConstraintExpr->isSatisfied())
-  Status = concepts::ExprRequirement::SS_ConstraintsNotSatisfied;
+if (Constraint.isInvalid()) {
+  Status = concepts::ExprRequirement::SS_ExprSubstitutionFailure;
+} else {
+  SubstitutedConstraintExpr =
+  cast(Constraint.get());
+  if (!SubstitutedConstraintExpr->isSatisfied())
+Status = concepts::ExprRequirement::SS_ConstraintsNotSatisfied;
+}
   }
   return new (Context) concepts::ExprRequirement(E, IsSimple, NoexceptLoc,
  ReturnTypeRequirement, Status,


Index: clang/test/SemaCXX/concept-fatal-error.cpp
===
--- /dev/null
+++ clang/test/SemaCXX/concept-fatal-error.cpp
@@ -0,0 +1,10 @@
+// RUN: not %clang_cc1 -fsyntax-only -std=c++20 -ferror-limit 1 -verify %s
+
+template 
+concept f = requires { 42; };
+struct h {
+  // The missing semicolon will trigger an error and -ferror-limit=1 will make it fatal
+  // We test that we do not crash in such cases (#55401)
+  int i = requires { { i } f } // expected-error {{expected ';' at end of declaration list}}
+   // expected-error@* {{too many errros emitted}}
+};
Index: clang/lib/Sema/SemaExprCXX.cpp
===
--- clang/lib/Sema/SemaExprCXX.cpp
+++ clang/lib/Sema/SemaExprCXX.cpp
@@ -9006,14 +9006,14 @@
 cast(TPL->getParam(0))->getTypeConstraint()
 ->getImmediatelyDeclaredConstraint();
 ExprResult Constraint = SubstExpr(IDC, MLTAL);
-assert(!Constraint.isInvalid() &&
-   "Substitution cannot fail as it is simply putting a type template "
-   "argument into a concept specialization expression's parameter.");
-
-SubstitutedConstraintExpr =
-cast(Constraint.get());
-if (!SubstitutedConstraintExpr->isSatisfied())
-  Status = concepts::ExprRequirement::SS_ConstraintsNotSatisfied;
+if (Constraint.isInvalid()) {
+  Status = concepts::ExprRequirement::SS_ExprSubstitutionFailure;
+} else {
+  SubstitutedConstraintExpr =
+  cast(Constraint.get());
+  if (!SubstitutedConstraintExpr->isSatisfied())
+Status = concepts::ExprRequirement::SS_ConstraintsNotSatisfied;
+}
   }
   return new (Context) concepts::ExprRequirement(E, IsSimple, NoexceptLoc,
  ReturnTypeRequirement, Status,
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D129499: [clang] Do not crash on "requires" after a fatal error occurred.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang/test/SemaCXX/concept-fatal-error.cpp:8
+  // We test that we do not crash in such cases (#55401)
+  int i = requires { { i } f } // expected-error {{expected ';' at end of 
declaration list}}
+

ilya-biryukov wrote:
> We need want to pass `-verify` to clang for these directives to have an 
> effect.
> The fatal error can be matched by `// expected-error@* {{too...}}` (maybe 
> that's obvious, but it took me some time to figure it out first time I needed 
> it)
> And there is no need to run `FileCheck` after that!
Ah, thanks! I knew there was a way to match fatal errors, I just didn't know 
what it was.

No clue what happened to -verify, I swear it was there at some point ;-)


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D129499/new/

https://reviews.llvm.org/D129499

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D129499: [clang] Do not crash on "requires" after a fatal error occurred.

[Adam Czachorowski via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rGcab3cfd013cf: [clang] Do not crash on "requires" 
after a fatal error occurred. (authored by adamcz).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D129499/new/

https://reviews.llvm.org/D129499

Files:
  clang/lib/Sema/SemaExprCXX.cpp
  clang/test/SemaCXX/concept-fatal-error.cpp


Index: clang/test/SemaCXX/concept-fatal-error.cpp
===
--- /dev/null
+++ clang/test/SemaCXX/concept-fatal-error.cpp
@@ -0,0 +1,10 @@
+// RUN: not %clang_cc1 -fsyntax-only -std=c++20 -ferror-limit 1 -verify %s
+
+template 
+concept f = requires { 42; };
+struct h {
+  // The missing semicolon will trigger an error and -ferror-limit=1 will make 
it fatal
+  // We test that we do not crash in such cases (#55401)
+  int i = requires { { i } f } // expected-error {{expected ';' at end of 
declaration list}}
+   // expected-error@* {{too many errros emitted}}
+};
Index: clang/lib/Sema/SemaExprCXX.cpp
===
--- clang/lib/Sema/SemaExprCXX.cpp
+++ clang/lib/Sema/SemaExprCXX.cpp
@@ -9006,14 +9006,14 @@
 cast(TPL->getParam(0))->getTypeConstraint()
 ->getImmediatelyDeclaredConstraint();
 ExprResult Constraint = SubstExpr(IDC, MLTAL);
-assert(!Constraint.isInvalid() &&
-   "Substitution cannot fail as it is simply putting a type template "
-   "argument into a concept specialization expression's parameter.");
-
-SubstitutedConstraintExpr =
-cast(Constraint.get());
-if (!SubstitutedConstraintExpr->isSatisfied())
-  Status = concepts::ExprRequirement::SS_ConstraintsNotSatisfied;
+if (Constraint.isInvalid()) {
+  Status = concepts::ExprRequirement::SS_ExprSubstitutionFailure;
+} else {
+  SubstitutedConstraintExpr =
+  cast(Constraint.get());
+  if (!SubstitutedConstraintExpr->isSatisfied())
+Status = concepts::ExprRequirement::SS_ConstraintsNotSatisfied;
+}
   }
   return new (Context) concepts::ExprRequirement(E, IsSimple, NoexceptLoc,
  ReturnTypeRequirement, Status,


Index: clang/test/SemaCXX/concept-fatal-error.cpp
===
--- /dev/null
+++ clang/test/SemaCXX/concept-fatal-error.cpp
@@ -0,0 +1,10 @@
+// RUN: not %clang_cc1 -fsyntax-only -std=c++20 -ferror-limit 1 -verify %s
+
+template 
+concept f = requires { 42; };
+struct h {
+  // The missing semicolon will trigger an error and -ferror-limit=1 will make it fatal
+  // We test that we do not crash in such cases (#55401)
+  int i = requires { { i } f } // expected-error {{expected ';' at end of declaration list}}
+   // expected-error@* {{too many errros emitted}}
+};
Index: clang/lib/Sema/SemaExprCXX.cpp
===
--- clang/lib/Sema/SemaExprCXX.cpp
+++ clang/lib/Sema/SemaExprCXX.cpp
@@ -9006,14 +9006,14 @@
 cast(TPL->getParam(0))->getTypeConstraint()
 ->getImmediatelyDeclaredConstraint();
 ExprResult Constraint = SubstExpr(IDC, MLTAL);
-assert(!Constraint.isInvalid() &&
-   "Substitution cannot fail as it is simply putting a type template "
-   "argument into a concept specialization expression's parameter.");
-
-SubstitutedConstraintExpr =
-cast(Constraint.get());
-if (!SubstitutedConstraintExpr->isSatisfied())
-  Status = concepts::ExprRequirement::SS_ConstraintsNotSatisfied;
+if (Constraint.isInvalid()) {
+  Status = concepts::ExprRequirement::SS_ExprSubstitutionFailure;
+} else {
+  SubstitutedConstraintExpr =
+  cast(Constraint.get());
+  if (!SubstitutedConstraintExpr->isSatisfied())
+Status = concepts::ExprRequirement::SS_ConstraintsNotSatisfied;
+}
   }
   return new (Context) concepts::ExprRequirement(E, IsSimple, NoexceptLoc,
  ReturnTypeRequirement, Status,
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D83814: [clangd] Add Random Forest runtime for code completion.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/unittests/DecisionForestTests.cpp:5
+
+namespace clangd {
+namespace clangd {

This is supposed to be "namespace clang", right?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D83814/new/

https://reviews.llvm.org/D83814

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D83814: [clangd] Add Random Forest runtime for code completion.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added a comment.

Looks good to me overall, some minor style comments included ;-)

Do we expect this to be a generic model code generator that gets reused for 
other things? If not, maybe we can hardcode more (like the namespace, class 
name, etc), but if you think there's other use cases for this then this LGTM.




Comment at: clang-tools-extra/clangd/quality/CompletionModelCodegen.py:39
+'''Returns the header guard for the generated header.'''
+return "GENERATED_DECISION_FOREST_MODEL_{}_H".format(filename.upper())
+

Why GENERATED_DECISON_FOREST_MODEL instead of output_dir, to be consistent with 
header guards for other files? Doesn't matter much for generated code, but if 
someone opens this in vim they'll see warnings.



Comment at: clang-tools-extra/clangd/quality/CompletionModelCodegen.py:57
+Control falls through if condition is evaluated to false."""
+return "{label}: if(E.{feature} >= {encoded} /*{threshold}*/) goto 
{true_label};".format(
+label=label,

nit: add space after if for readability (also below)



Comment at: clang-tools-extra/clangd/quality/CompletionModelCodegen.py:93
+def tree(t, tree_num: int, node_num: int):
+"""Returns code for inferencing a Decision Tree.
+

Please extend the comment to mention the second return value (size of the tree)



Comment at: clang-tools-extra/clangd/quality/CompletionModelCodegen.py:105
+"""
+label = "t{tree}_n{node}".format(tree=tree_num, node=node_num)
+code = []

This is a good place to use an Python's f-string.

Also in few places below.



Comment at: clang-tools-extra/clangd/quality/CompletionModelCodegen.py:126
+
+return code + false_code + true_code, 1 + false_size+true_size
+

style nit: be consistent with spaces around +



Comment at: clang-tools-extra/clangd/quality/CompletionModelCodegen.py:177
+
+code += "  friend float Evaluate(const {}&);\n".format(cpp_class.name)
+code += "};\n"

Is there a reason to make this a friend free-function instead of static method 
on the Example class? The problem is that you now end up with 
clang::clangd::Evaluate, so if we every re-use this code gen for another model 
we'll have a name collision.



Comment at: clang-tools-extra/clangd/quality/CompletionModelCodegen.py:271
+parser = argparse.ArgumentParser('DecisionForestCodegen')
+parser.add_argument('--filename', help='output file name.')
+parser.add_argument('--output_dir', help='output directory')

nit: be consistent about putting a "." at the end of the help text or not.



Comment at: clang-tools-extra/clangd/unittests/model/CategoricalFeature.h:1
+namespace ns1 {
+namespace ns2 {

Can we rename this directory? quality/model makes some sense (although it would 
be better to include something about code completion there), but 
unittests/model is not very descriptive - what model?

How about unittests/decision_forest_model/ or something like that? Or go with 
the Input/TEST_NAME pattern.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D83814/new/

https://reviews.llvm.org/D83814

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D87673: [clangd] Don't use zlib when it's unavailable.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz accepted this revision.
adamcz added inline comments.
This revision is now accepted and ready to land.



Comment at: clang-tools-extra/clangd/index/Serialization.cpp:209
 Uncompressed = UncompressedStorage;
+  } else {
+return error("Compressed string table, but zlib is unavailable");

nit: skip { } to stay consistent with local style


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D87673/new/

https://reviews.llvm.org/D87673

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D87710: [clangd] Actually parse Index section of the YAML file.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
adamcz added a reviewer: sammccall.
Herald added subscribers: cfe-commits, usaxena95, kadircet, arphaman, jkorous.
Herald added a project: clang.
adamcz requested review of this revision.
Herald added subscribers: MaskRay, ilya-biryukov.

This fixes a bug in dbf486c0de92c76df77c1a1f815cf16533ecbb3a 
, which
introduced the Index section of the config, but did not register the
parse method, so it didn't work in a YAML file (but did in a test).


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D87710

Files:
  clang-tools-extra/clangd/ConfigYAML.cpp
  clang-tools-extra/clangd/unittests/ConfigYAMLTests.cpp


Index: clang-tools-extra/clangd/unittests/ConfigYAMLTests.cpp
===
--- clang-tools-extra/clangd/unittests/ConfigYAMLTests.cpp
+++ clang-tools-extra/clangd/unittests/ConfigYAMLTests.cpp
@@ -47,16 +47,21 @@
   Add: |
 b
 az
+---
+Index:
+  Background: Skip
   )yaml";
   auto Results = Fragment::parseYAML(YAML, "config.yaml", Diags.callback());
   EXPECT_THAT(Diags.Diagnostics, IsEmpty());
-  ASSERT_EQ(Results.size(), 2u);
-  EXPECT_FALSE(Results.front().If.HasUnrecognizedCondition);
-  EXPECT_THAT(Results.front().If.PathMatch, ElementsAre(Val("abc")));
-  EXPECT_THAT(Results.front().CompileFlags.Add,
-  ElementsAre(Val("foo"), Val("bar")));
+  ASSERT_EQ(Results.size(), 3u);
+  EXPECT_FALSE(Results[0].If.HasUnrecognizedCondition);
+  EXPECT_THAT(Results[0].If.PathMatch, ElementsAre(Val("abc")));
+  EXPECT_THAT(Results[0].CompileFlags.Add, ElementsAre(Val("foo"), 
Val("bar")));
+
+  EXPECT_THAT(Results[1].CompileFlags.Add, ElementsAre(Val("b\naz\n")));
 
-  EXPECT_THAT(Results.back().CompileFlags.Add, ElementsAre(Val("b\naz\n")));
+  ASSERT_TRUE(Results[2].Index.Background);
+  EXPECT_EQ("Skip", *Results[2].Index.Background.getValue());
 }
 
 TEST(ParseYAML, Locations) {
Index: clang-tools-extra/clangd/ConfigYAML.cpp
===
--- clang-tools-extra/clangd/ConfigYAML.cpp
+++ clang-tools-extra/clangd/ConfigYAML.cpp
@@ -38,6 +38,7 @@
 DictParser Dict("Config", this);
 Dict.handle("If", [&](Node &N) { parse(F.If, N); });
 Dict.handle("CompileFlags", [&](Node &N) { parse(F.CompileFlags, N); });
+Dict.handle("Index", [&](Node &N) { parse(F.Index, N); });
 Dict.parse(N);
 return !(N.failed() || HadError);
   }


Index: clang-tools-extra/clangd/unittests/ConfigYAMLTests.cpp
===
--- clang-tools-extra/clangd/unittests/ConfigYAMLTests.cpp
+++ clang-tools-extra/clangd/unittests/ConfigYAMLTests.cpp
@@ -47,16 +47,21 @@
   Add: |
 b
 az
+---
+Index:
+  Background: Skip
   )yaml";
   auto Results = Fragment::parseYAML(YAML, "config.yaml", Diags.callback());
   EXPECT_THAT(Diags.Diagnostics, IsEmpty());
-  ASSERT_EQ(Results.size(), 2u);
-  EXPECT_FALSE(Results.front().If.HasUnrecognizedCondition);
-  EXPECT_THAT(Results.front().If.PathMatch, ElementsAre(Val("abc")));
-  EXPECT_THAT(Results.front().CompileFlags.Add,
-  ElementsAre(Val("foo"), Val("bar")));
+  ASSERT_EQ(Results.size(), 3u);
+  EXPECT_FALSE(Results[0].If.HasUnrecognizedCondition);
+  EXPECT_THAT(Results[0].If.PathMatch, ElementsAre(Val("abc")));
+  EXPECT_THAT(Results[0].CompileFlags.Add, ElementsAre(Val("foo"), Val("bar")));
+
+  EXPECT_THAT(Results[1].CompileFlags.Add, ElementsAre(Val("b\naz\n")));
 
-  EXPECT_THAT(Results.back().CompileFlags.Add, ElementsAre(Val("b\naz\n")));
+  ASSERT_TRUE(Results[2].Index.Background);
+  EXPECT_EQ("Skip", *Results[2].Index.Background.getValue());
 }
 
 TEST(ParseYAML, Locations) {
Index: clang-tools-extra/clangd/ConfigYAML.cpp
===
--- clang-tools-extra/clangd/ConfigYAML.cpp
+++ clang-tools-extra/clangd/ConfigYAML.cpp
@@ -38,6 +38,7 @@
 DictParser Dict("Config", this);
 Dict.handle("If", [&](Node &N) { parse(F.If, N); });
 Dict.handle("CompileFlags", [&](Node &N) { parse(F.CompileFlags, N); });
+Dict.handle("Index", [&](Node &N) { parse(F.Index, N); });
 Dict.parse(N);
 return !(N.failed() || HadError);
   }
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D87710: [clangd] Actually parse Index section of the YAML file.

[Adam Czachorowski via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rG7029e5d4ca20: [clangd] Actually parse Index section of the 
YAML file. (authored by adamcz).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D87710/new/

https://reviews.llvm.org/D87710

Files:
  clang-tools-extra/clangd/ConfigYAML.cpp
  clang-tools-extra/clangd/unittests/ConfigYAMLTests.cpp


Index: clang-tools-extra/clangd/unittests/ConfigYAMLTests.cpp
===
--- clang-tools-extra/clangd/unittests/ConfigYAMLTests.cpp
+++ clang-tools-extra/clangd/unittests/ConfigYAMLTests.cpp
@@ -47,16 +47,21 @@
   Add: |
 b
 az
+---
+Index:
+  Background: Skip
   )yaml";
   auto Results = Fragment::parseYAML(YAML, "config.yaml", Diags.callback());
   EXPECT_THAT(Diags.Diagnostics, IsEmpty());
-  ASSERT_EQ(Results.size(), 2u);
-  EXPECT_FALSE(Results.front().If.HasUnrecognizedCondition);
-  EXPECT_THAT(Results.front().If.PathMatch, ElementsAre(Val("abc")));
-  EXPECT_THAT(Results.front().CompileFlags.Add,
-  ElementsAre(Val("foo"), Val("bar")));
+  ASSERT_EQ(Results.size(), 3u);
+  EXPECT_FALSE(Results[0].If.HasUnrecognizedCondition);
+  EXPECT_THAT(Results[0].If.PathMatch, ElementsAre(Val("abc")));
+  EXPECT_THAT(Results[0].CompileFlags.Add, ElementsAre(Val("foo"), 
Val("bar")));
+
+  EXPECT_THAT(Results[1].CompileFlags.Add, ElementsAre(Val("b\naz\n")));
 
-  EXPECT_THAT(Results.back().CompileFlags.Add, ElementsAre(Val("b\naz\n")));
+  ASSERT_TRUE(Results[2].Index.Background);
+  EXPECT_EQ("Skip", *Results[2].Index.Background.getValue());
 }
 
 TEST(ParseYAML, Locations) {
Index: clang-tools-extra/clangd/ConfigYAML.cpp
===
--- clang-tools-extra/clangd/ConfigYAML.cpp
+++ clang-tools-extra/clangd/ConfigYAML.cpp
@@ -38,6 +38,7 @@
 DictParser Dict("Config", this);
 Dict.handle("If", [&](Node &N) { parse(F.If, N); });
 Dict.handle("CompileFlags", [&](Node &N) { parse(F.CompileFlags, N); });
+Dict.handle("Index", [&](Node &N) { parse(F.Index, N); });
 Dict.parse(N);
 return !(N.failed() || HadError);
   }


Index: clang-tools-extra/clangd/unittests/ConfigYAMLTests.cpp
===
--- clang-tools-extra/clangd/unittests/ConfigYAMLTests.cpp
+++ clang-tools-extra/clangd/unittests/ConfigYAMLTests.cpp
@@ -47,16 +47,21 @@
   Add: |
 b
 az
+---
+Index:
+  Background: Skip
   )yaml";
   auto Results = Fragment::parseYAML(YAML, "config.yaml", Diags.callback());
   EXPECT_THAT(Diags.Diagnostics, IsEmpty());
-  ASSERT_EQ(Results.size(), 2u);
-  EXPECT_FALSE(Results.front().If.HasUnrecognizedCondition);
-  EXPECT_THAT(Results.front().If.PathMatch, ElementsAre(Val("abc")));
-  EXPECT_THAT(Results.front().CompileFlags.Add,
-  ElementsAre(Val("foo"), Val("bar")));
+  ASSERT_EQ(Results.size(), 3u);
+  EXPECT_FALSE(Results[0].If.HasUnrecognizedCondition);
+  EXPECT_THAT(Results[0].If.PathMatch, ElementsAre(Val("abc")));
+  EXPECT_THAT(Results[0].CompileFlags.Add, ElementsAre(Val("foo"), Val("bar")));
+
+  EXPECT_THAT(Results[1].CompileFlags.Add, ElementsAre(Val("b\naz\n")));
 
-  EXPECT_THAT(Results.back().CompileFlags.Add, ElementsAre(Val("b\naz\n")));
+  ASSERT_TRUE(Results[2].Index.Background);
+  EXPECT_EQ("Skip", *Results[2].Index.Background.getValue());
 }
 
 TEST(ParseYAML, Locations) {
Index: clang-tools-extra/clangd/ConfigYAML.cpp
===
--- clang-tools-extra/clangd/ConfigYAML.cpp
+++ clang-tools-extra/clangd/ConfigYAML.cpp
@@ -38,6 +38,7 @@
 DictParser Dict("Config", this);
 Dict.handle("If", [&](Node &N) { parse(F.If, N); });
 Dict.handle("CompileFlags", [&](Node &N) { parse(F.CompileFlags, N); });
+Dict.handle("Index", [&](Node &N) { parse(F.Index, N); });
 Dict.parse(N);
 return !(N.failed() || HadError);
   }
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D87775: [clangd] Add option for disabling AddUsing tweak on some namespaces.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz created this revision.
adamcz added a reviewer: sammccall.
Herald added subscribers: cfe-commits, usaxena95, kadircet, arphaman, jkorous.
Herald added a project: clang.
adamcz requested review of this revision.
Herald added subscribers: MaskRay, ilya-biryukov.

For style guides forbid "using" declarations for namespaces like "std".
With this new config option, AddUsing can be selectively disabled on
those.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D87775

Files:
  clang-tools-extra/clangd/Config.h
  clang-tools-extra/clangd/ConfigCompile.cpp
  clang-tools-extra/clangd/ConfigFragment.h
  clang-tools-extra/clangd/ConfigYAML.cpp
  clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
  clang-tools-extra/clangd/unittests/TweakTests.cpp

Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "Annotations.h"
+#include "Config.h"
 #include "SourceCode.h"
 #include "TestFS.h"
 #include "TestTU.h"
@@ -2471,8 +2472,14 @@
 
 TWEAK_TEST(AddUsing);
 TEST_F(AddUsingTest, Prepare) {
+  Config Cfg;
+  Cfg.Style.FullyQualifiedNamespaces.push_back("ban");
+  WithContextValue WithConfig(Config::Key, std::move(Cfg));
+
   const std::string Header = R"cpp(
 #define NS(name) one::two::name
+namespace ban { void foo() {} }
+namespace banana { void foo() {} }
 namespace one {
 void oo() {}
 template class tt {};
@@ -2506,6 +2513,10 @@
   // Test that we don't crash or misbehave on unnamed DeclRefExpr.
   EXPECT_UNAVAILABLE(Header +
  "void fun() { one::two::cc() ^| one::two::cc(); }");
+  // Do not offer code action when operating on a banned namespace.
+  EXPECT_UNAVAILABLE(Header + "void fun() { ban::fo^o(); }");
+  EXPECT_UNAVAILABLE(Header + "void fun() { ::ban::fo^o(); }");
+  EXPECT_AVAILABLE(Header + "void fun() { banana::fo^o(); }");
 
   // Check that we do not trigger in header files.
   FileName = "test.h";
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "AST.h"
+#include "Config.h"
 #include "FindTarget.h"
 #include "refactor/Tweak.h"
 #include "support/Logger.h"
@@ -190,6 +191,29 @@
   return Out;
 }
 
+bool isNamespaceForbidden(const Tweak::Selection &Inputs,
+  const NestedNameSpecifier &Namespace) {
+  std::string Buf;
+  llvm::raw_string_ostream NamespaceStream(Buf);
+  Namespace.print(NamespaceStream,
+  Inputs.AST->getASTContext().getPrintingPolicy());
+  NamespaceStream << "::";
+  StringRef NamespaceStr = NamespaceStream.str();
+  NamespaceStr.consume_front("::");
+
+  for (StringRef Banned : Config::current().Style.FullyQualifiedNamespaces) {
+// We want to match regardless of leading ::, so we remove it from both
+// NamespaceStr and Banned. We append it at the end of both to make sure
+// that we do not prefix-match ::foo to ::foobar.
+Banned.consume_front("::");
+Banned.consume_back("::");
+if (NamespaceStr.startswith(Banned.str() + "::"))
+  return true;
+  }
+
+  return false;
+}
+
 bool AddUsing::prepare(const Selection &Inputs) {
   auto &SM = Inputs.AST->getSourceManager();
 
@@ -248,6 +272,9 @@
 return false;
   }
 
+  if (isNamespaceForbidden(Inputs, *QualifierToRemove.getNestedNameSpecifier()))
+return false;
+
   // Macros are difficult. We only want to offer code action when what's spelled
   // under the cursor is a namespace qualifier. If it's a macro that expands to
   // a qualifier, user would not know what code action will actually change.
Index: clang-tools-extra/clangd/ConfigYAML.cpp
===
--- clang-tools-extra/clangd/ConfigYAML.cpp
+++ clang-tools-extra/clangd/ConfigYAML.cpp
@@ -38,6 +38,7 @@
 DictParser Dict("Config", this);
 Dict.handle("If", [&](Node &N) { parse(F.If, N); });
 Dict.handle("CompileFlags", [&](Node &N) { parse(F.CompileFlags, N); });
+Dict.handle("Style", [&](Node &N) { parse(F.Style, N); });
 Dict.parse(N);
 return !(N.failed() || HadError);
   }
@@ -71,6 +72,15 @@
 Dict.parse(N);
   }
 
+  void parse(Fragment::StyleBlock &F, Node &N) {
+DictParser Dict("Style", this);
+Dict.handle("FullyQualifiedNamespaces", [&](Node &N) {
+  if (auto Values = scalarValues(N))
+F.FullyQualifiedNamespaces = std::move(*Values);
+});
+Dict.parse(N);
+  }
+
   void parse(Fragment::IndexBlock &F, Node &N) {
 DictParser Dict("Index", this);
 Dict.handle("Bac

[PATCH] D87775: [clangd] Add option for disabling AddUsing tweak on some namespaces.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/ConfigFragment.h:167
+  struct StyleBlock {
+// List of namespaces that should not appear in "using" declarations.
+std::vector> FullyQualifiedNamespaces;

sammccall wrote:
> Can we describe this positively first?
> 
> Namespaces whose members should be fully-qualified, rather than via using 
> declarations or directives.
> Affects availability of the "add using declaration" tweak.
I copied the comment from the other place and added the bit about tweak. WDYT?



Comment at: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp:197
+  std::string Buf;
+  llvm::raw_string_ostream NamespaceStream(Buf);
+  Namespace.print(NamespaceStream,

sammccall wrote:
> validate it's a namespace, and then call printNamespaceScope (AST.h)?
> 
> This handles the right spelling of anonymous/inline NSes.
It's already guaranteed to be a namespace, that's checked before calling this 
function.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D87775/new/

https://reviews.llvm.org/D87775

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D87775: [clangd] Add option for disabling AddUsing tweak on some namespaces.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 292521.
adamcz marked 5 inline comments as done.
adamcz added a comment.

addressed review comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D87775/new/

https://reviews.llvm.org/D87775

Files:
  clang-tools-extra/clangd/Config.h
  clang-tools-extra/clangd/ConfigCompile.cpp
  clang-tools-extra/clangd/ConfigFragment.h
  clang-tools-extra/clangd/ConfigYAML.cpp
  clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
  clang-tools-extra/clangd/unittests/TweakTests.cpp

Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "Annotations.h"
+#include "Config.h"
 #include "SourceCode.h"
 #include "TestFS.h"
 #include "TestTU.h"
@@ -2471,8 +2472,14 @@
 
 TWEAK_TEST(AddUsing);
 TEST_F(AddUsingTest, Prepare) {
+  Config Cfg;
+  Cfg.Style.FullyQualifiedNamespaces.push_back("ban");
+  WithContextValue WithConfig(Config::Key, std::move(Cfg));
+
   const std::string Header = R"cpp(
 #define NS(name) one::two::name
+namespace ban { void foo() {} }
+namespace banana { void foo() {} }
 namespace one {
 void oo() {}
 template class tt {};
@@ -2506,6 +2513,10 @@
   // Test that we don't crash or misbehave on unnamed DeclRefExpr.
   EXPECT_UNAVAILABLE(Header +
  "void fun() { one::two::cc() ^| one::two::cc(); }");
+  // Do not offer code action when operating on a banned namespace.
+  EXPECT_UNAVAILABLE(Header + "void fun() { ban::fo^o(); }");
+  EXPECT_UNAVAILABLE(Header + "void fun() { ::ban::fo^o(); }");
+  EXPECT_AVAILABLE(Header + "void fun() { banana::fo^o(); }");
 
   // Check that we do not trigger in header files.
   FileName = "test.h";
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "AST.h"
+#include "Config.h"
 #include "FindTarget.h"
 #include "refactor/Tweak.h"
 #include "support/Logger.h"
@@ -190,6 +191,21 @@
   return Out;
 }
 
+bool isNamespaceForbidden(const Tweak::Selection &Inputs,
+  const NestedNameSpecifier &Namespace) {
+  std::string NamespaceStr = printNamespaceScope(*Namespace.getAsNamespace());
+
+  for (StringRef Banned : Config::current().Style.FullyQualifiedNamespaces) {
+StringRef PrefixMatch = NamespaceStr;
+PrefixMatch.consume_front("::");
+PrefixMatch.consume_front(Banned);
+if (PrefixMatch.consume_front("::"))
+  return true;
+  }
+
+  return false;
+}
+
 bool AddUsing::prepare(const Selection &Inputs) {
   auto &SM = Inputs.AST->getSourceManager();
 
@@ -248,6 +264,9 @@
 return false;
   }
 
+  if (isNamespaceForbidden(Inputs, *QualifierToRemove.getNestedNameSpecifier()))
+return false;
+
   // Macros are difficult. We only want to offer code action when what's spelled
   // under the cursor is a namespace qualifier. If it's a macro that expands to
   // a qualifier, user would not know what code action will actually change.
Index: clang-tools-extra/clangd/ConfigYAML.cpp
===
--- clang-tools-extra/clangd/ConfigYAML.cpp
+++ clang-tools-extra/clangd/ConfigYAML.cpp
@@ -38,6 +38,7 @@
 DictParser Dict("Config", this);
 Dict.handle("If", [&](Node &N) { parse(F.If, N); });
 Dict.handle("CompileFlags", [&](Node &N) { parse(F.CompileFlags, N); });
+Dict.handle("Style", [&](Node &N) { parse(F.Style, N); });
 Dict.parse(N);
 return !(N.failed() || HadError);
   }
@@ -71,6 +72,15 @@
 Dict.parse(N);
   }
 
+  void parse(Fragment::StyleBlock &F, Node &N) {
+DictParser Dict("Style", this);
+Dict.handle("FullyQualifiedNamespaces", [&](Node &N) {
+  if (auto Values = scalarValues(N))
+F.FullyQualifiedNamespaces = std::move(*Values);
+});
+Dict.parse(N);
+  }
+
   void parse(Fragment::IndexBlock &F, Node &N) {
 DictParser Dict("Index", this);
 Dict.handle("Background",
Index: clang-tools-extra/clangd/ConfigFragment.h
===
--- clang-tools-extra/clangd/ConfigFragment.h
+++ clang-tools-extra/clangd/ConfigFragment.h
@@ -161,6 +161,16 @@
 llvm::Optional> Background;
   };
   IndexBlock Index;
+
+  // Describes the style of the codebase, beyond formatting.
+  struct StyleBlock {
+// Namespaces that should always be fully qualified, meaning no "using"
+// declarations, always spell out the whole name (with or without leading
+// ::). All nested namespaces are affected a

[PATCH] D87775: [clangd] Add option for disabling AddUsing tweak on some namespaces.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz updated this revision to Diff 292786.
adamcz marked 2 inline comments as done.
adamcz added a comment.

addressed review comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D87775/new/

https://reviews.llvm.org/D87775

Files:
  clang-tools-extra/clangd/Config.h
  clang-tools-extra/clangd/ConfigCompile.cpp
  clang-tools-extra/clangd/ConfigFragment.h
  clang-tools-extra/clangd/ConfigYAML.cpp
  clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
  clang-tools-extra/clangd/unittests/TweakTests.cpp

Index: clang-tools-extra/clangd/unittests/TweakTests.cpp
===
--- clang-tools-extra/clangd/unittests/TweakTests.cpp
+++ clang-tools-extra/clangd/unittests/TweakTests.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "Annotations.h"
+#include "Config.h"
 #include "SourceCode.h"
 #include "TestFS.h"
 #include "TestTU.h"
@@ -2471,8 +2472,14 @@
 
 TWEAK_TEST(AddUsing);
 TEST_F(AddUsingTest, Prepare) {
+  Config Cfg;
+  Cfg.Style.FullyQualifiedNamespaces.push_back("ban");
+  WithContextValue WithConfig(Config::Key, std::move(Cfg));
+
   const std::string Header = R"cpp(
 #define NS(name) one::two::name
+namespace ban { void foo() {} }
+namespace banana { void foo() {} }
 namespace one {
 void oo() {}
 template class tt {};
@@ -2506,6 +2513,10 @@
   // Test that we don't crash or misbehave on unnamed DeclRefExpr.
   EXPECT_UNAVAILABLE(Header +
  "void fun() { one::two::cc() ^| one::two::cc(); }");
+  // Do not offer code action when operating on a banned namespace.
+  EXPECT_UNAVAILABLE(Header + "void fun() { ban::fo^o(); }");
+  EXPECT_UNAVAILABLE(Header + "void fun() { ::ban::fo^o(); }");
+  EXPECT_AVAILABLE(Header + "void fun() { banana::fo^o(); }");
 
   // Check that we do not trigger in header files.
   FileName = "test.h";
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "AST.h"
+#include "Config.h"
 #include "FindTarget.h"
 #include "refactor/Tweak.h"
 #include "support/Logger.h"
@@ -190,6 +191,19 @@
   return Out;
 }
 
+bool isNamespaceForbidden(const Tweak::Selection &Inputs,
+  const NestedNameSpecifier &Namespace) {
+  std::string NamespaceStr = printNamespaceScope(*Namespace.getAsNamespace());
+
+  for (StringRef Banned : Config::current().Style.FullyQualifiedNamespaces) {
+StringRef PrefixMatch = NamespaceStr;
+if (PrefixMatch.consume_front(Banned) && PrefixMatch.consume_front("::"))
+  return true;
+  }
+
+  return false;
+}
+
 bool AddUsing::prepare(const Selection &Inputs) {
   auto &SM = Inputs.AST->getSourceManager();
 
@@ -248,6 +262,9 @@
 return false;
   }
 
+  if (isNamespaceForbidden(Inputs, *QualifierToRemove.getNestedNameSpecifier()))
+return false;
+
   // Macros are difficult. We only want to offer code action when what's spelled
   // under the cursor is a namespace qualifier. If it's a macro that expands to
   // a qualifier, user would not know what code action will actually change.
Index: clang-tools-extra/clangd/ConfigYAML.cpp
===
--- clang-tools-extra/clangd/ConfigYAML.cpp
+++ clang-tools-extra/clangd/ConfigYAML.cpp
@@ -38,6 +38,7 @@
 DictParser Dict("Config", this);
 Dict.handle("If", [&](Node &N) { parse(F.If, N); });
 Dict.handle("CompileFlags", [&](Node &N) { parse(F.CompileFlags, N); });
+Dict.handle("Style", [&](Node &N) { parse(F.Style, N); });
 Dict.parse(N);
 return !(N.failed() || HadError);
   }
@@ -71,6 +72,15 @@
 Dict.parse(N);
   }
 
+  void parse(Fragment::StyleBlock &F, Node &N) {
+DictParser Dict("Style", this);
+Dict.handle("FullyQualifiedNamespaces", [&](Node &N) {
+  if (auto Values = scalarValues(N))
+F.FullyQualifiedNamespaces = std::move(*Values);
+});
+Dict.parse(N);
+  }
+
   void parse(Fragment::IndexBlock &F, Node &N) {
 DictParser Dict("Index", this);
 Dict.handle("Background",
Index: clang-tools-extra/clangd/ConfigFragment.h
===
--- clang-tools-extra/clangd/ConfigFragment.h
+++ clang-tools-extra/clangd/ConfigFragment.h
@@ -161,6 +161,16 @@
 llvm::Optional> Background;
   };
   IndexBlock Index;
+
+  // Describes the style of the codebase, beyond formatting.
+  struct StyleBlock {
+// Namespaces that should always be fully qualified, meaning no "using"
+// declarations, always spell out the whole name (with or without leading
+// ::). All nested namespaces are affected as well.
+// Affects availability of t

[PATCH] D87775: [clangd] Add option for disabling AddUsing tweak on some namespaces.

[Adam Czachorowski via Phabricator via cfe-commits]

adamcz added inline comments.



Comment at: clang-tools-extra/clangd/Config.h:70
+// declarations, always spell out the whole name (with or without leading
+// ::). All nested namespaces are affected as well.
+std::vector FullyQualifiedNamespaces;

sammccall wrote:
> should just be "without leading ::" now I think?
What I was trying to say here is that both ::foo::bar and foo::bar are allowed, 
but:
  using foo;
  bar
is not. Basically I'm trying to explain that what we mean by fully qualified is 
the no-using-shortcuts, not always-start-with-::

This isn't about what's stored in the vector, but rather what this options 
means.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D87775/new/

https://reviews.llvm.org/D87775

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits