[PATCH] D52696: Update ifunc attribute support documentation

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin accepted this revision.
DmitryPolukhin added a comment.
This revision is now accepted and ready to land.

LGTM


https://reviews.llvm.org/D52696



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D63482: [clang-tidy] Fix the YAML created for checks like modernize-pass-by-value

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

@mgehre @yvvan it seems that the issue still not fixed and '\n' gets duplicated 
in the replacements. Are you going to fix this issue or I should create a patch 
to fix it?
Before this change '\n' was actually processed correctly `ReplacementText: 
'#include \n\n'` is actually replacement text with two new lines in 
standard YAML deserialiser.


Repository:
  rL LLVM

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D63482/new/

https://reviews.llvm.org/D63482



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80301: [yaml][clang-tidy] Fix new line YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin created this revision.
DmitryPolukhin added reviewers: gribozavr, mgehre, yvvan.
DmitryPolukhin added projects: clang-tools-extra, clang.
Herald added subscribers: hiraditya, xazax.hun.
Herald added a project: LLVM.
DmitryPolukhin edited the summary of this revision.

Move new line duplication logic to YMAL string
serialization/deserilization level instead of Replacement that was
introduced in D63482 . D63482 
 led to duplicated
new lines if you apply replacements with clang-apply-replacements. New
line duplication happened only during serialisation and there was no
opposite transformation in deserialization.

Test Plan: check-all


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D80301

Files:
  clang/include/clang/Tooling/ReplacementsYaml.h
  llvm/lib/Support/YAMLTraits.cpp
  llvm/unittests/Support/YAMLIOTest.cpp


Index: llvm/unittests/Support/YAMLIOTest.cpp
===
--- llvm/unittests/Support/YAMLIOTest.cpp
+++ llvm/unittests/Support/YAMLIOTest.cpp
@@ -285,7 +285,7 @@
 YOut << Original;
   }
   auto Expected = "---\n"
-  "str1:'a multiline string\n"
+  "str1:'a multiline string\n\n"
   "foobarbaz'\n"
   "str2:'another one\r"
   "foobarbaz'\n"
Index: llvm/lib/Support/YAMLTraits.cpp
===
--- llvm/lib/Support/YAMLTraits.cpp
+++ llvm/lib/Support/YAMLTraits.cpp
@@ -887,13 +887,32 @@
 }
 
 void ScalarTraits::output(const std::string &Val, void *,
- raw_ostream &Out) {
-  Out << Val;
+   raw_ostream &Out) {
+  size_t currentPos = 0;
+  size_t lineBreakPos = Val.find('\n');
+  while (lineBreakPos != std::string::npos) {
+Out << StringRef(&Val[currentPos], lineBreakPos - currentPos + 1);
+Out << '\n';
+currentPos = lineBreakPos + 1;
+lineBreakPos = Val.find('\n', currentPos);
+  }
+  Out << StringRef(&Val[currentPos], Val.size() - currentPos);
 }
 
 StringRef ScalarTraits::input(StringRef Scalar, void *,
- std::string &Val) {
-  Val = Scalar.str();
+   std::string &Val) {
+  Val.clear();
+  size_t currentPos = 0;
+  size_t lineBreakPos = Scalar.find('\n');
+  while (lineBreakPos != std::string::npos) {
+// '\n\n' convert to '\n' and don't copy single '\n'.
+if (currentPos + 1 < Scalar.size() && Scalar[lineBreakPos + 1] == '\n')
+  ++lineBreakPos;
+Val += Scalar.substr(currentPos, lineBreakPos);
+currentPos = lineBreakPos + 1;
+lineBreakPos = Scalar.find('\n', currentPos);
+  }
+  Val += Scalar.substr(currentPos, Scalar.size() - currentPos);
   return StringRef();
 }
 
Index: clang/include/clang/Tooling/ReplacementsYaml.h
===
--- clang/include/clang/Tooling/ReplacementsYaml.h
+++ clang/include/clang/Tooling/ReplacementsYaml.h
@@ -35,13 +35,7 @@
 
 NormalizedReplacement(const IO &, const clang::tooling::Replacement &R)
 : FilePath(R.getFilePath()), Offset(R.getOffset()),
-  Length(R.getLength()), ReplacementText(R.getReplacementText()) {
-  size_t lineBreakPos = ReplacementText.find('\n');
-  while (lineBreakPos != std::string::npos) {
-ReplacementText.replace(lineBreakPos, 1, "\n\n");
-lineBreakPos = ReplacementText.find('\n', lineBreakPos + 2);
-  }
-}
+  Length(R.getLength()), ReplacementText(R.getReplacementText()) {}
 
 clang::tooling::Replacement denormalize(const IO &) {
   return clang::tooling::Replacement(FilePath, Offset, Length,


Index: llvm/unittests/Support/YAMLIOTest.cpp
===
--- llvm/unittests/Support/YAMLIOTest.cpp
+++ llvm/unittests/Support/YAMLIOTest.cpp
@@ -285,7 +285,7 @@
 YOut << Original;
   }
   auto Expected = "---\n"
-  "str1:'a multiline string\n"
+  "str1:'a multiline string\n\n"
   "foobarbaz'\n"
   "str2:'another one\r"
   "foobarbaz'\n"
Index: llvm/lib/Support/YAMLTraits.cpp
===
--- llvm/lib/Support/YAMLTraits.cpp
+++ llvm/lib/Support/YAMLTraits.cpp
@@ -887,13 +887,32 @@
 }
 
 void ScalarTraits::output(const std::string &Val, void *,
- raw_ostream &Out) {
-  Out << Val;
+   raw_ostream &Out) {
+  size_t currentPos = 0;
+  size_t lineBreakPos = Val.find('\n');
+  while (lineBreakPos != std::string::npos) {
+Out << StringRef(&Val[currentPos], lineBreakPos - currentPos + 1);
+Out << '\n';
+currentPos = lineBre

[PATCH] D63482: [clang-tidy] Fix the YAML created for checks like modernize-pass-by-value

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

Sent D80301  for review.


Repository:
  rL LLVM

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D63482/new/

https://reviews.llvm.org/D63482



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80301: [yaml][clang-tidy] Fix new line YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 265292.
DmitryPolukhin added a comment.

Fix clang-tidy warnings


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301

Files:
  clang/include/clang/Tooling/ReplacementsYaml.h
  llvm/lib/Support/YAMLTraits.cpp
  llvm/unittests/Support/YAMLIOTest.cpp


Index: llvm/unittests/Support/YAMLIOTest.cpp
===
--- llvm/unittests/Support/YAMLIOTest.cpp
+++ llvm/unittests/Support/YAMLIOTest.cpp
@@ -275,7 +275,7 @@
 TEST(YAMLIO, MultilineStrings) {
   WithStringField Original;
   Original.str1 = "a multiline string\nfoobarbaz";
-  Original.str2 = "another one\rfoobarbaz";
+  Original.str2 = "another one\rfoobarbaz\n";
   Original.str3 = "a one-line string";
 
   std::string Serialized;
@@ -285,10 +285,10 @@
 YOut << Original;
   }
   auto Expected = "---\n"
-  "str1:'a multiline string\n"
+  "str1:'a multiline string\n\n"
   "foobarbaz'\n"
   "str2:'another one\r"
-  "foobarbaz'\n"
+  "foobarbaz\n\n'\n"
   "str3:a one-line string\n"
   "...\n";
   ASSERT_EQ(Serialized, Expected);
Index: llvm/lib/Support/YAMLTraits.cpp
===
--- llvm/lib/Support/YAMLTraits.cpp
+++ llvm/lib/Support/YAMLTraits.cpp
@@ -887,13 +887,32 @@
 }
 
 void ScalarTraits::output(const std::string &Val, void *,
- raw_ostream &Out) {
-  Out << Val;
+   raw_ostream &Out) {
+  size_t CurrentPos = 0;
+  size_t LineBreakPos = Val.find('\n');
+  while (LineBreakPos != std::string::npos) {
+Out << StringRef(&Val[CurrentPos], LineBreakPos - CurrentPos + 1);
+Out << '\n';
+CurrentPos = LineBreakPos + 1;
+LineBreakPos = Val.find('\n', CurrentPos);
+  }
+  Out << StringRef(&Val[CurrentPos], Val.size() - CurrentPos);
 }
 
 StringRef ScalarTraits::input(StringRef Scalar, void *,
- std::string &Val) {
-  Val = Scalar.str();
+   std::string &Val) {
+  Val.clear();
+  size_t CurrentPos = 0;
+  size_t LineBreakPos = Scalar.find('\n');
+  while (LineBreakPos != std::string::npos) {
+// '\n\n' convert to '\n' and don't copy single '\n'.
+if (CurrentPos + 1 < Scalar.size() && Scalar[LineBreakPos + 1] == '\n')
+  ++LineBreakPos;
+Val += Scalar.substr(CurrentPos, LineBreakPos);
+CurrentPos = LineBreakPos + 1;
+LineBreakPos = Scalar.find('\n', CurrentPos);
+  }
+  Val += Scalar.substr(CurrentPos, Scalar.size() - CurrentPos);
   return StringRef();
 }
 
Index: clang/include/clang/Tooling/ReplacementsYaml.h
===
--- clang/include/clang/Tooling/ReplacementsYaml.h
+++ clang/include/clang/Tooling/ReplacementsYaml.h
@@ -35,13 +35,7 @@
 
 NormalizedReplacement(const IO &, const clang::tooling::Replacement &R)
 : FilePath(R.getFilePath()), Offset(R.getOffset()),
-  Length(R.getLength()), ReplacementText(R.getReplacementText()) {
-  size_t lineBreakPos = ReplacementText.find('\n');
-  while (lineBreakPos != std::string::npos) {
-ReplacementText.replace(lineBreakPos, 1, "\n\n");
-lineBreakPos = ReplacementText.find('\n', lineBreakPos + 2);
-  }
-}
+  Length(R.getLength()), ReplacementText(R.getReplacementText()) {}
 
 clang::tooling::Replacement denormalize(const IO &) {
   return clang::tooling::Replacement(FilePath, Offset, Length,


Index: llvm/unittests/Support/YAMLIOTest.cpp
===
--- llvm/unittests/Support/YAMLIOTest.cpp
+++ llvm/unittests/Support/YAMLIOTest.cpp
@@ -275,7 +275,7 @@
 TEST(YAMLIO, MultilineStrings) {
   WithStringField Original;
   Original.str1 = "a multiline string\nfoobarbaz";
-  Original.str2 = "another one\rfoobarbaz";
+  Original.str2 = "another one\rfoobarbaz\n";
   Original.str3 = "a one-line string";
 
   std::string Serialized;
@@ -285,10 +285,10 @@
 YOut << Original;
   }
   auto Expected = "---\n"
-  "str1:'a multiline string\n"
+  "str1:'a multiline string\n\n"
   "foobarbaz'\n"
   "str2:'another one\r"
-  "foobarbaz'\n"
+  "foobarbaz\n\n'\n"
   "str3:a one-line string\n"
   "...\n";
   ASSERT_EQ(Serialized, Expected);
Index: llvm/lib/Support/YAMLTraits.cpp
===
--- llvm/lib/Support/YAMLTraits.cpp
+++ llvm/lib/Support/YAMLTraits.cpp
@@ -887,13 +887,32 @@
 }
 
 void ScalarTraits::output(const std::string &Val

[PATCH] D80301: [yaml][clang-tidy] Fix new line YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 265329.
DmitryPolukhin added a comment.

Fixed second string after new line + more test case


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301

Files:
  clang/include/clang/Tooling/ReplacementsYaml.h
  llvm/lib/Support/YAMLTraits.cpp
  llvm/unittests/Support/YAMLIOTest.cpp


Index: llvm/unittests/Support/YAMLIOTest.cpp
===
--- llvm/unittests/Support/YAMLIOTest.cpp
+++ llvm/unittests/Support/YAMLIOTest.cpp
@@ -274,8 +274,8 @@
 
 TEST(YAMLIO, MultilineStrings) {
   WithStringField Original;
-  Original.str1 = "a multiline string\nfoobarbaz";
-  Original.str2 = "another one\rfoobarbaz";
+  Original.str1 = "a\n\nmultiline\nstring\nfoobarbaz";
+  Original.str2 = "another one\rfoobarbaz\n";
   Original.str3 = "a one-line string";
 
   std::string Serialized;
@@ -285,10 +285,10 @@
 YOut << Original;
   }
   auto Expected = "---\n"
-  "str1:'a multiline string\n"
+  "str1:'a\n\n\n\nmultiline\n\nstring\n\n"
   "foobarbaz'\n"
   "str2:'another one\r"
-  "foobarbaz'\n"
+  "foobarbaz\n\n'\n"
   "str3:a one-line string\n"
   "...\n";
   ASSERT_EQ(Serialized, Expected);
Index: llvm/lib/Support/YAMLTraits.cpp
===
--- llvm/lib/Support/YAMLTraits.cpp
+++ llvm/lib/Support/YAMLTraits.cpp
@@ -887,13 +887,32 @@
 }
 
 void ScalarTraits::output(const std::string &Val, void *,
- raw_ostream &Out) {
-  Out << Val;
+   raw_ostream &Out) {
+  size_t CurrentPos = 0;
+  size_t LineBreakPos = Val.find('\n');
+  while (LineBreakPos != std::string::npos) {
+Out << StringRef(&Val[CurrentPos], LineBreakPos - CurrentPos + 1);
+Out << '\n';
+CurrentPos = LineBreakPos + 1;
+LineBreakPos = Val.find('\n', CurrentPos);
+  }
+  Out << StringRef(&Val[CurrentPos], Val.size() - CurrentPos);
 }
 
 StringRef ScalarTraits::input(StringRef Scalar, void *,
- std::string &Val) {
-  Val = Scalar.str();
+   std::string &Val) {
+  Val.clear();
+  size_t CurrentPos = 0;
+  size_t LineBreakPos = Scalar.find('\n');
+  while (LineBreakPos != std::string::npos) {
+// '\n\n' convert to '\n' and don't copy single '\n'.
+if (CurrentPos + 1 < Scalar.size() && Scalar[LineBreakPos + 1] == '\n')
+  ++LineBreakPos;
+Val += Scalar.substr(CurrentPos, LineBreakPos - CurrentPos);
+CurrentPos = LineBreakPos + 1;
+LineBreakPos = Scalar.find('\n', CurrentPos);
+  }
+  Val += Scalar.substr(CurrentPos, Scalar.size() - CurrentPos);
   return StringRef();
 }
 
Index: clang/include/clang/Tooling/ReplacementsYaml.h
===
--- clang/include/clang/Tooling/ReplacementsYaml.h
+++ clang/include/clang/Tooling/ReplacementsYaml.h
@@ -35,13 +35,7 @@
 
 NormalizedReplacement(const IO &, const clang::tooling::Replacement &R)
 : FilePath(R.getFilePath()), Offset(R.getOffset()),
-  Length(R.getLength()), ReplacementText(R.getReplacementText()) {
-  size_t lineBreakPos = ReplacementText.find('\n');
-  while (lineBreakPos != std::string::npos) {
-ReplacementText.replace(lineBreakPos, 1, "\n\n");
-lineBreakPos = ReplacementText.find('\n', lineBreakPos + 2);
-  }
-}
+  Length(R.getLength()), ReplacementText(R.getReplacementText()) {}
 
 clang::tooling::Replacement denormalize(const IO &) {
   return clang::tooling::Replacement(FilePath, Offset, Length,


Index: llvm/unittests/Support/YAMLIOTest.cpp
===
--- llvm/unittests/Support/YAMLIOTest.cpp
+++ llvm/unittests/Support/YAMLIOTest.cpp
@@ -274,8 +274,8 @@
 
 TEST(YAMLIO, MultilineStrings) {
   WithStringField Original;
-  Original.str1 = "a multiline string\nfoobarbaz";
-  Original.str2 = "another one\rfoobarbaz";
+  Original.str1 = "a\n\nmultiline\nstring\nfoobarbaz";
+  Original.str2 = "another one\rfoobarbaz\n";
   Original.str3 = "a one-line string";
 
   std::string Serialized;
@@ -285,10 +285,10 @@
 YOut << Original;
   }
   auto Expected = "---\n"
-  "str1:'a multiline string\n"
+  "str1:'a\n\n\n\nmultiline\n\nstring\n\n"
   "foobarbaz'\n"
   "str2:'another one\r"
-  "foobarbaz'\n"
+  "foobarbaz\n\n'\n"
   "str3:a one-line string\n"
   "...\n";
   ASSERT_EQ(Serialized, Expected);
Index: llvm/lib/Support/YAMLTraits.cpp
===

[PATCH] D80301: [yaml][clang-tidy] Fix new line YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added reviewers: Eugene.Zelenko, thegameg.
DmitryPolukhin marked an inline comment as done.
DmitryPolukhin added subscribers: thegameg, Eugene.Zelenko.
DmitryPolukhin added a comment.

+ @gribozavr, @Eugene.Zelenko, @thegameg who touched/reviewed this code, please 
take a look.




Comment at: llvm/lib/Support/YAMLTraits.cpp:904
+   std::string &Val) {
+  Val.clear();
+  size_t CurrentPos = 0;

mgehre wrote:
> I wonder whether using StringRef::split() would lead to an easier 
> implementation 
> (https://llvm.org/doxygen/classllvm_1_1StringRef.html#af0284e4c41c0e09c0bc4767bc77a899d)
I'm not sure that it will be easier to read or more efficient 
(`StringRef::split` will require additional vector).


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D75184: [clang-tidy] Optional inheritance of file configs from parent directories

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added inline comments.



Comment at: clang-tools-extra/clang-tidy/ClangTidyOptions.cpp:116-121
+  unsigned Priority = 0;
   for (ClangTidyModuleRegistry::iterator I = ClangTidyModuleRegistry::begin(),
  E = ClangTidyModuleRegistry::end();
I != E; ++I)
-Options = Options.mergeWith(I->instantiate()->getModuleOptions());
+Options =
+Options.mergeWith(I->instantiate()->getModuleOptions(), ++Priority);

njames93 wrote:
> Is there a reason for incrementing the priority on each successive iteration, 
> Seems like a bug that will lead to the later registered modules having higher 
> priority for their options.
It seems that you are right and it is just a bug. Modules are registered as 
static object in different translation units so there is no guarantee about 
their order. But in general I didn't expect conflicting options here. If you 
have a diff, I'll be happy to stamp it; if not, I'll create the diff.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D75184/new/

https://reviews.llvm.org/D75184

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D84850: [clang-tidy] Fix module options being registered with different priorities

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin accepted this revision.
DmitryPolukhin added a comment.
This revision is now accepted and ready to land.

Thank you for identifying and fixing!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84850/new/

https://reviews.llvm.org/D84850

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D75184: [clang-tidy] Optional inheritance of file configs from parent directories

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

@alexfh friend ping, please take a look.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D75184/new/

https://reviews.llvm.org/D75184



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D75184: [clang-tidy] Optional inheritance of file configs from parent directories

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

@alexfh thank you for review!




Comment at: clang-tools-extra/clang-tidy/ClangTidyOptions.h:101-104
+ClangTidyValue(const char *Value) : Value(Value), Priority(0) {}
+ClangTidyValue(const std::string &Value) : Value(Value), Priority(0) {}
+ClangTidyValue(const std::string &Value, unsigned Priority)
+: Value(Value), Priority(Priority) {}

alexfh wrote:
> Maybe just `ClangTidyValue(StringRef Value, unsigned Priority = 0)`?
Reduced number of c-tors to 2 with default value. We need `ClangTidyValue(const 
char *Value)` for supporting lots of constructs like 
`Opts["cert-dcl16-c.NewSuffixes"] = "L;LL;LU;LLU";` without changing lots of 
code without good reason.



Comment at: clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp:41
+  in the parent directory (if any exists) will be taken and current config file
+  will be applied on top of the parent one. If any configuration options have
+  a corresponding command-line option, command-line option takes precedence.

alexfh wrote:
> Does the new logic related to local and global options deserve a separate 
> mention here?
I think adding global vs local here will only add more confusion and general 
statement about precedence still true. As far as I can see there are no 
оreferences to global options in user documentation. 


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D75184/new/

https://reviews.llvm.org/D75184



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D75184: [clang-tidy] Optional inheritance of file configs from parent directories

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 257336.
DmitryPolukhin marked 6 inline comments as done.
DmitryPolukhin added a comment.

Comments resolved + rebase


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D75184/new/

https://reviews.llvm.org/D75184

Files:
  clang-tools-extra/clang-tidy/ClangTidy.cpp
  clang-tools-extra/clang-tidy/ClangTidyCheck.cpp
  clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.cpp
  clang-tools-extra/clang-tidy/ClangTidyOptions.cpp
  clang-tools-extra/clang-tidy/ClangTidyOptions.h
  clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp
  clang-tools-extra/docs/clang-tidy/index.rst
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/3/.clang-tidy
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/.clang-tidy
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/44/.clang-tidy
  clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
  clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp

Index: clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp
===
--- clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp
+++ clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp
@@ -87,7 +87,7 @@
   ExtraArgsBefore: ['arg-before3', 'arg-before4']
   )");
   ASSERT_TRUE(!!Options2);
-  ClangTidyOptions Options = Options1->mergeWith(*Options2);
+  ClangTidyOptions Options = Options1->mergeWith(*Options2, 0);
   EXPECT_EQ("check1,check2,check3,check4", *Options.Checks);
   EXPECT_EQ("filter2", *Options.HeaderFilterRegex);
   EXPECT_EQ("user2", *Options.User);
Index: clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
===
--- clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
+++ clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
@@ -7,6 +7,26 @@
 // RUN: clang-tidy -dump-config %S/Inputs/config-files/2/- -- | FileCheck %s -check-prefix=CHECK-CHILD2
 // CHECK-CHILD2: Checks: {{.*}}from-parent
 // CHECK-CHILD2: HeaderFilterRegex: parent
+// RUN: clang-tidy -dump-config %S/Inputs/config-files/3/- -- | FileCheck %s -check-prefix=CHECK-CHILD3
+// CHECK-CHILD3: Checks: {{.*}}from-parent,from-child3
+// CHECK-CHILD3: HeaderFilterRegex: child3
 // RUN: clang-tidy -dump-config -checks='from-command-line' -header-filter='from command line' %S/Inputs/config-files/- -- | FileCheck %s -check-prefix=CHECK-COMMAND-LINE
 // CHECK-COMMAND-LINE: Checks: {{.*}}from-parent,from-command-line
 // CHECK-COMMAND-LINE: HeaderFilterRegex: from command line
+
+// For this test we have to use names of the real checks because otherwise values are ignored.
+// RUN: clang-tidy -dump-config %S/Inputs/config-files/4/44/- -- | FileCheck %s -check-prefix=CHECK-CHILD4
+// CHECK-CHILD4: Checks: {{.*}}modernize-loop-convert,modernize-use-using,llvm-qualified-auto
+// CHECK-CHILD4: - key: llvm-qualified-auto.AddConstToQualified
+// CHECK-CHILD4-NEXT: value: '1
+// CHECK-CHILD4: - key: modernize-loop-convert.MaxCopySize
+// CHECK-CHILD4-NEXT: value: '20'
+// CHECK-CHILD4: - key: modernize-loop-convert.MinConfidence
+// CHECK-CHILD4-NEXT: value: reasonable
+// CHECK-CHILD4: - key: modernize-use-using.IgnoreMacros
+// CHECK-CHILD4-NEXT: value: '0'
+
+// RUN: clang-tidy --explain-config %S/Inputs/config-files/4/44/- -- | FileCheck %s -check-prefix=CHECK-EXPLAIN
+// CHECK-EXPLAIN: 'llvm-qualified-auto' is enabled in the {{.*}}/Inputs/config-files/4/44/.clang-tidy.
+// CHECK-EXPLAIN: 'modernize-loop-convert' is enabled in the {{.*}}/Inputs/config-files/4/.clang-tidy.
+// CHECK-EXPLAIN: 'modernize-use-using' is enabled in the {{.*}}/Inputs/config-files/4/.clang-tidy.
Index: clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/44/.clang-tidy
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/44/.clang-tidy
@@ -0,0 +1,9 @@
+InheritParentConfig: true
+Checks: 'llvm-qualified-auto'
+CheckOptions:
+  - key: modernize-loop-convert.MaxCopySize
+value:   '20'
+  - key: llvm-qualified-auto.AddConstToQualified
+value:   '1'
+  - key: IgnoreMacros
+value:   '0'
Index: clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/.clang-tidy
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/.clang-tidy
@@ -0,0 +1,8 @@
+Checks: '-*,modernize-loop-convert,modernize-use-using'
+CheckOptions:
+  - key: modernize-loop-convert.MaxCopySize
+value:   '10'
+  - key: modernize-loop-convert.MinConfidence
+value:   reasonable
+  - key: modernize-use-using.IgnoreMa

[PATCH] D75184: [clang-tidy] Optional inheritance of file configs from parent directories

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 257338.
DmitryPolukhin added a comment.

Remove clang-format errors in diff


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D75184/new/

https://reviews.llvm.org/D75184

Files:
  clang-tools-extra/clang-tidy/ClangTidy.cpp
  clang-tools-extra/clang-tidy/ClangTidyCheck.cpp
  clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.cpp
  clang-tools-extra/clang-tidy/ClangTidyOptions.cpp
  clang-tools-extra/clang-tidy/ClangTidyOptions.h
  clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp
  clang-tools-extra/docs/clang-tidy/index.rst
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/3/.clang-tidy
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/.clang-tidy
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/44/.clang-tidy
  clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
  clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp

Index: clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp
===
--- clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp
+++ clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp
@@ -87,7 +87,7 @@
   ExtraArgsBefore: ['arg-before3', 'arg-before4']
   )");
   ASSERT_TRUE(!!Options2);
-  ClangTidyOptions Options = Options1->mergeWith(*Options2);
+  ClangTidyOptions Options = Options1->mergeWith(*Options2, 0);
   EXPECT_EQ("check1,check2,check3,check4", *Options.Checks);
   EXPECT_EQ("filter2", *Options.HeaderFilterRegex);
   EXPECT_EQ("user2", *Options.User);
Index: clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
===
--- clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
+++ clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
@@ -7,6 +7,26 @@
 // RUN: clang-tidy -dump-config %S/Inputs/config-files/2/- -- | FileCheck %s -check-prefix=CHECK-CHILD2
 // CHECK-CHILD2: Checks: {{.*}}from-parent
 // CHECK-CHILD2: HeaderFilterRegex: parent
+// RUN: clang-tidy -dump-config %S/Inputs/config-files/3/- -- | FileCheck %s -check-prefix=CHECK-CHILD3
+// CHECK-CHILD3: Checks: {{.*}}from-parent,from-child3
+// CHECK-CHILD3: HeaderFilterRegex: child3
 // RUN: clang-tidy -dump-config -checks='from-command-line' -header-filter='from command line' %S/Inputs/config-files/- -- | FileCheck %s -check-prefix=CHECK-COMMAND-LINE
 // CHECK-COMMAND-LINE: Checks: {{.*}}from-parent,from-command-line
 // CHECK-COMMAND-LINE: HeaderFilterRegex: from command line
+
+// For this test we have to use names of the real checks because otherwise values are ignored.
+// RUN: clang-tidy -dump-config %S/Inputs/config-files/4/44/- -- | FileCheck %s -check-prefix=CHECK-CHILD4
+// CHECK-CHILD4: Checks: {{.*}}modernize-loop-convert,modernize-use-using,llvm-qualified-auto
+// CHECK-CHILD4: - key: llvm-qualified-auto.AddConstToQualified
+// CHECK-CHILD4-NEXT: value: '1
+// CHECK-CHILD4: - key: modernize-loop-convert.MaxCopySize
+// CHECK-CHILD4-NEXT: value: '20'
+// CHECK-CHILD4: - key: modernize-loop-convert.MinConfidence
+// CHECK-CHILD4-NEXT: value: reasonable
+// CHECK-CHILD4: - key: modernize-use-using.IgnoreMacros
+// CHECK-CHILD4-NEXT: value: '0'
+
+// RUN: clang-tidy --explain-config %S/Inputs/config-files/4/44/- -- | FileCheck %s -check-prefix=CHECK-EXPLAIN
+// CHECK-EXPLAIN: 'llvm-qualified-auto' is enabled in the {{.*}}/Inputs/config-files/4/44/.clang-tidy.
+// CHECK-EXPLAIN: 'modernize-loop-convert' is enabled in the {{.*}}/Inputs/config-files/4/.clang-tidy.
+// CHECK-EXPLAIN: 'modernize-use-using' is enabled in the {{.*}}/Inputs/config-files/4/.clang-tidy.
Index: clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/44/.clang-tidy
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/44/.clang-tidy
@@ -0,0 +1,9 @@
+InheritParentConfig: true
+Checks: 'llvm-qualified-auto'
+CheckOptions:
+  - key: modernize-loop-convert.MaxCopySize
+value:   '20'
+  - key: llvm-qualified-auto.AddConstToQualified
+value:   '1'
+  - key: IgnoreMacros
+value:   '0'
Index: clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/.clang-tidy
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/.clang-tidy
@@ -0,0 +1,8 @@
+Checks: '-*,modernize-loop-convert,modernize-use-using'
+CheckOptions:
+  - key: modernize-loop-convert.MaxCopySize
+value:   '10'
+  - key: modernize-loop-convert.MinConfidence
+value:   reasonable
+  - key: modernize-use-using.IgnoreMacros
+value:   1
Index: clang

[PATCH] D75184: [clang-tidy] Optional inheritance of file configs from parent directories

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 257344.
DmitryPolukhin added a comment.

One more try to remove useless lint issues


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D75184/new/

https://reviews.llvm.org/D75184

Files:
  clang-tools-extra/clang-tidy/ClangTidy.cpp
  clang-tools-extra/clang-tidy/ClangTidyCheck.cpp
  clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.cpp
  clang-tools-extra/clang-tidy/ClangTidyOptions.cpp
  clang-tools-extra/clang-tidy/ClangTidyOptions.h
  clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp
  clang-tools-extra/docs/clang-tidy/index.rst
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/3/.clang-tidy
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/.clang-tidy
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/44/.clang-tidy
  clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
  clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp

Index: clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp
===
--- clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp
+++ clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp
@@ -87,7 +87,7 @@
   ExtraArgsBefore: ['arg-before3', 'arg-before4']
   )");
   ASSERT_TRUE(!!Options2);
-  ClangTidyOptions Options = Options1->mergeWith(*Options2);
+  ClangTidyOptions Options = Options1->mergeWith(*Options2, 0);
   EXPECT_EQ("check1,check2,check3,check4", *Options.Checks);
   EXPECT_EQ("filter2", *Options.HeaderFilterRegex);
   EXPECT_EQ("user2", *Options.User);
Index: clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
===
--- clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
+++ clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
@@ -7,6 +7,26 @@
 // RUN: clang-tidy -dump-config %S/Inputs/config-files/2/- -- | FileCheck %s -check-prefix=CHECK-CHILD2
 // CHECK-CHILD2: Checks: {{.*}}from-parent
 // CHECK-CHILD2: HeaderFilterRegex: parent
+// RUN: clang-tidy -dump-config %S/Inputs/config-files/3/- -- | FileCheck %s -check-prefix=CHECK-CHILD3
+// CHECK-CHILD3: Checks: {{.*}}from-parent,from-child3
+// CHECK-CHILD3: HeaderFilterRegex: child3
 // RUN: clang-tidy -dump-config -checks='from-command-line' -header-filter='from command line' %S/Inputs/config-files/- -- | FileCheck %s -check-prefix=CHECK-COMMAND-LINE
 // CHECK-COMMAND-LINE: Checks: {{.*}}from-parent,from-command-line
 // CHECK-COMMAND-LINE: HeaderFilterRegex: from command line
+
+// For this test we have to use names of the real checks because otherwise values are ignored.
+// RUN: clang-tidy -dump-config %S/Inputs/config-files/4/44/- -- | FileCheck %s -check-prefix=CHECK-CHILD4
+// CHECK-CHILD4: Checks: {{.*}}modernize-loop-convert,modernize-use-using,llvm-qualified-auto
+// CHECK-CHILD4: - key: llvm-qualified-auto.AddConstToQualified
+// CHECK-CHILD4-NEXT: value: '1
+// CHECK-CHILD4: - key: modernize-loop-convert.MaxCopySize
+// CHECK-CHILD4-NEXT: value: '20'
+// CHECK-CHILD4: - key: modernize-loop-convert.MinConfidence
+// CHECK-CHILD4-NEXT: value: reasonable
+// CHECK-CHILD4: - key: modernize-use-using.IgnoreMacros
+// CHECK-CHILD4-NEXT: value: '0'
+
+// RUN: clang-tidy --explain-config %S/Inputs/config-files/4/44/- -- | FileCheck %s -check-prefix=CHECK-EXPLAIN
+// CHECK-EXPLAIN: 'llvm-qualified-auto' is enabled in the {{.*}}/Inputs/config-files/4/44/.clang-tidy.
+// CHECK-EXPLAIN: 'modernize-loop-convert' is enabled in the {{.*}}/Inputs/config-files/4/.clang-tidy.
+// CHECK-EXPLAIN: 'modernize-use-using' is enabled in the {{.*}}/Inputs/config-files/4/.clang-tidy.
Index: clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/44/.clang-tidy
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/44/.clang-tidy
@@ -0,0 +1,9 @@
+InheritParentConfig: true
+Checks: 'llvm-qualified-auto'
+CheckOptions:
+  - key: modernize-loop-convert.MaxCopySize
+value:   '20'
+  - key: llvm-qualified-auto.AddConstToQualified
+value:   '1'
+  - key: IgnoreMacros
+value:   '0'
Index: clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/.clang-tidy
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/.clang-tidy
@@ -0,0 +1,8 @@
+Checks: '-*,modernize-loop-convert,modernize-use-using'
+CheckOptions:
+  - key: modernize-loop-convert.MaxCopySize
+value:   '10'
+  - key: modernize-loop-convert.MinConfidence
+value:   reasonable
+  - key: modernize-use-using.IgnoreMacros
+value:   1
Inde

[PATCH] D75184: [clang-tidy] Optional inheritance of file configs from parent directories

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 257405.
DmitryPolukhin added a comment.

And one more time :(


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D75184/new/

https://reviews.llvm.org/D75184

Files:
  clang-tools-extra/clang-tidy/ClangTidy.cpp
  clang-tools-extra/clang-tidy/ClangTidyCheck.cpp
  clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.cpp
  clang-tools-extra/clang-tidy/ClangTidyOptions.cpp
  clang-tools-extra/clang-tidy/ClangTidyOptions.h
  clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp
  clang-tools-extra/docs/clang-tidy/index.rst
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/3/.clang-tidy
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/.clang-tidy
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/44/.clang-tidy
  clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
  clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp

Index: clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp
===
--- clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp
+++ clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp
@@ -87,7 +87,7 @@
   ExtraArgsBefore: ['arg-before3', 'arg-before4']
   )");
   ASSERT_TRUE(!!Options2);
-  ClangTidyOptions Options = Options1->mergeWith(*Options2);
+  ClangTidyOptions Options = Options1->mergeWith(*Options2, 0);
   EXPECT_EQ("check1,check2,check3,check4", *Options.Checks);
   EXPECT_EQ("filter2", *Options.HeaderFilterRegex);
   EXPECT_EQ("user2", *Options.User);
Index: clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
===
--- clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
+++ clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
@@ -7,6 +7,26 @@
 // RUN: clang-tidy -dump-config %S/Inputs/config-files/2/- -- | FileCheck %s -check-prefix=CHECK-CHILD2
 // CHECK-CHILD2: Checks: {{.*}}from-parent
 // CHECK-CHILD2: HeaderFilterRegex: parent
+// RUN: clang-tidy -dump-config %S/Inputs/config-files/3/- -- | FileCheck %s -check-prefix=CHECK-CHILD3
+// CHECK-CHILD3: Checks: {{.*}}from-parent,from-child3
+// CHECK-CHILD3: HeaderFilterRegex: child3
 // RUN: clang-tidy -dump-config -checks='from-command-line' -header-filter='from command line' %S/Inputs/config-files/- -- | FileCheck %s -check-prefix=CHECK-COMMAND-LINE
 // CHECK-COMMAND-LINE: Checks: {{.*}}from-parent,from-command-line
 // CHECK-COMMAND-LINE: HeaderFilterRegex: from command line
+
+// For this test we have to use names of the real checks because otherwise values are ignored.
+// RUN: clang-tidy -dump-config %S/Inputs/config-files/4/44/- -- | FileCheck %s -check-prefix=CHECK-CHILD4
+// CHECK-CHILD4: Checks: {{.*}}modernize-loop-convert,modernize-use-using,llvm-qualified-auto
+// CHECK-CHILD4: - key: llvm-qualified-auto.AddConstToQualified
+// CHECK-CHILD4-NEXT: value: '1
+// CHECK-CHILD4: - key: modernize-loop-convert.MaxCopySize
+// CHECK-CHILD4-NEXT: value: '20'
+// CHECK-CHILD4: - key: modernize-loop-convert.MinConfidence
+// CHECK-CHILD4-NEXT: value: reasonable
+// CHECK-CHILD4: - key: modernize-use-using.IgnoreMacros
+// CHECK-CHILD4-NEXT: value: '0'
+
+// RUN: clang-tidy --explain-config %S/Inputs/config-files/4/44/- -- | FileCheck %s -check-prefix=CHECK-EXPLAIN
+// CHECK-EXPLAIN: 'llvm-qualified-auto' is enabled in the {{.*}}/Inputs/config-files/4/44/.clang-tidy.
+// CHECK-EXPLAIN: 'modernize-loop-convert' is enabled in the {{.*}}/Inputs/config-files/4/.clang-tidy.
+// CHECK-EXPLAIN: 'modernize-use-using' is enabled in the {{.*}}/Inputs/config-files/4/.clang-tidy.
Index: clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/44/.clang-tidy
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/44/.clang-tidy
@@ -0,0 +1,9 @@
+InheritParentConfig: true
+Checks: 'llvm-qualified-auto'
+CheckOptions:
+  - key: modernize-loop-convert.MaxCopySize
+value:   '20'
+  - key: llvm-qualified-auto.AddConstToQualified
+value:   '1'
+  - key: IgnoreMacros
+value:   '0'
Index: clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/.clang-tidy
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/.clang-tidy
@@ -0,0 +1,8 @@
+Checks: '-*,modernize-loop-convert,modernize-use-using'
+CheckOptions:
+  - key: modernize-loop-convert.MaxCopySize
+value:   '10'
+  - key: modernize-loop-convert.MinConfidence
+value:   reasonable
+  - key: modernize-use-using.IgnoreMacros
+value:   1
Index: clang-tools-extra/t

[PATCH] D76594: [clang][AST] Support AST files larger than 512M

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 257633.
DmitryPolukhin added a comment.

Split BitOffset in DeclOffset in high/low parts; rebase


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D76594/new/

https://reviews.llvm.org/D76594

Files:
  clang/include/clang/Serialization/ASTBitCodes.h
  clang/include/clang/Serialization/ASTReader.h
  clang/include/clang/Serialization/ASTWriter.h
  clang/include/clang/Serialization/ModuleFile.h
  clang/lib/Serialization/ASTReader.cpp
  clang/lib/Serialization/ASTReaderDecl.cpp
  clang/lib/Serialization/ASTWriter.cpp
  clang/lib/Serialization/ASTWriterDecl.cpp

Index: clang/lib/Serialization/ASTWriterDecl.cpp
===
--- clang/lib/Serialization/ASTWriterDecl.cpp
+++ clang/lib/Serialization/ASTWriterDecl.cpp
@@ -2422,12 +2422,12 @@
   SourceLocation Loc = D->getLocation();
   unsigned Index = ID - FirstDeclID;
   if (DeclOffsets.size() == Index)
-DeclOffsets.push_back(DeclOffset(Loc, Offset));
+DeclOffsets.emplace_back(Loc, Offset);
   else if (DeclOffsets.size() < Index) {
 // FIXME: Can/should this happen?
 DeclOffsets.resize(Index+1);
 DeclOffsets[Index].setLocation(Loc);
-DeclOffsets[Index].BitOffset = Offset;
+DeclOffsets[Index].setBitOffset(Offset);
   } else {
 llvm_unreachable("declarations should be emitted in ID order");
   }
Index: clang/lib/Serialization/ASTWriter.cpp
===
--- clang/lib/Serialization/ASTWriter.cpp
+++ clang/lib/Serialization/ASTWriter.cpp
@@ -1893,6 +1893,7 @@
   // Write out the source location entry table. We skip the first
   // entry, which is always the same dummy entry.
   std::vector SLocEntryOffsets;
+  uint64_t SLocEntryOffsetsBase = Stream.GetCurrentBitNo();
   RecordData PreloadSLocs;
   SLocEntryOffsets.reserve(SourceMgr.local_sloc_entry_size() - 1);
   for (unsigned I = 1, N = SourceMgr.local_sloc_entry_size();
@@ -1903,7 +1904,9 @@
 assert(&SourceMgr.getSLocEntry(FID) == SLoc);
 
 // Record the offset of this source-location entry.
-SLocEntryOffsets.push_back(Stream.GetCurrentBitNo());
+uint64_t Offset = Stream.GetCurrentBitNo() - SLocEntryOffsetsBase;
+assert((Offset >> 32) == 0 && "SLocEntry offset too large");
+SLocEntryOffsets.push_back(Offset);
 
 // Figure out which record code to use.
 unsigned Code;
@@ -2011,12 +2014,14 @@
   Abbrev->Add(BitCodeAbbrevOp(SOURCE_LOCATION_OFFSETS));
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 16)); // # of slocs
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 16)); // total size
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 32)); // base offset
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); // offsets
   unsigned SLocOffsetsAbbrev = Stream.EmitAbbrev(std::move(Abbrev));
   {
 RecordData::value_type Record[] = {
 SOURCE_LOCATION_OFFSETS, SLocEntryOffsets.size(),
-SourceMgr.getNextLocalOffset() - 1 /* skip dummy */};
+SourceMgr.getNextLocalOffset() - 1 /* skip dummy */,
+SLocEntryOffsetsBase};
 Stream.EmitRecordWithBlob(SLocOffsetsAbbrev, Record,
   bytes(SLocEntryOffsets));
   }
@@ -2093,9 +2098,11 @@
 /// Writes the block containing the serialized form of the
 /// preprocessor.
 void ASTWriter::WritePreprocessor(const Preprocessor &PP, bool IsModule) {
+  uint64_t MacroOffsetsBase = Stream.GetCurrentBitNo();
+
   PreprocessingRecord *PPRec = PP.getPreprocessingRecord();
   if (PPRec)
-WritePreprocessorDetail(*PPRec);
+WritePreprocessorDetail(*PPRec, MacroOffsetsBase);
 
   RecordData Record;
   RecordData ModuleMacroRecord;
@@ -2156,7 +2163,8 @@
   // identifier they belong to.
   for (const IdentifierInfo *Name : MacroIdentifiers) {
 MacroDirective *MD = PP.getLocalMacroDirectiveHistory(Name);
-auto StartOffset = Stream.GetCurrentBitNo();
+uint64_t StartOffset = Stream.GetCurrentBitNo() - MacroOffsetsBase;
+assert((StartOffset >> 32) == 0 && "Macro identifiers offset too large");
 
 // Emit the macro directives in reverse source order.
 for (; MD; MD = MD->getPrevious()) {
@@ -2229,14 +2237,12 @@
 
 // Record the local offset of this macro.
 unsigned Index = ID - FirstMacroID;
-if (Index == MacroOffsets.size())
-  MacroOffsets.push_back(Stream.GetCurrentBitNo());
-else {
-  if (Index > MacroOffsets.size())
-MacroOffsets.resize(Index + 1);
+if (Index >= MacroOffsets.size())
+  MacroOffsets.resize(Index + 1);
 
-  MacroOffsets[Index] = Stream.GetCurrentBitNo();
-}
+uint64_t Offset = Stream.GetCurrentBitNo() - MacroOffsetsBase;
+assert((Offset >> 32) == 0 && "Macro offset too large");
+MacroOffsets[Index] = Offset;
 
 AddIdentifierRef(Name, Record);
 AddSourceLocation(MI->getDefinitionLoc(), Record);
@@ -2287,17 +2293,20 @@
   Abbrev->Add(BitCodeAbbrevOp(MACRO_OFFSET)

[PATCH] D76594: [clang][AST] Support AST files larger than 512M

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 257632.
DmitryPolukhin added a comment.

Split BitOffset in DeclOffset in high/low parts; rebase


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D76594/new/

https://reviews.llvm.org/D76594

Files:
  clang/include/clang/Serialization/ASTBitCodes.h
  clang/include/clang/Serialization/ASTReader.h
  clang/include/clang/Serialization/ASTWriter.h
  clang/include/clang/Serialization/ModuleFile.h
  clang/lib/Serialization/ASTReader.cpp
  clang/lib/Serialization/ASTReaderDecl.cpp
  clang/lib/Serialization/ASTWriter.cpp
  clang/lib/Serialization/ASTWriterDecl.cpp

Index: clang/lib/Serialization/ASTWriterDecl.cpp
===
--- clang/lib/Serialization/ASTWriterDecl.cpp
+++ clang/lib/Serialization/ASTWriterDecl.cpp
@@ -2422,12 +2422,12 @@
   SourceLocation Loc = D->getLocation();
   unsigned Index = ID - FirstDeclID;
   if (DeclOffsets.size() == Index)
-DeclOffsets.push_back(DeclOffset(Loc, Offset));
+DeclOffsets.emplace_back(Loc, Offset);
   else if (DeclOffsets.size() < Index) {
 // FIXME: Can/should this happen?
 DeclOffsets.resize(Index+1);
 DeclOffsets[Index].setLocation(Loc);
-DeclOffsets[Index].BitOffset = Offset;
+DeclOffsets[Index].setBitOffset(Offset);
   } else {
 llvm_unreachable("declarations should be emitted in ID order");
   }
Index: clang/lib/Serialization/ASTWriter.cpp
===
--- clang/lib/Serialization/ASTWriter.cpp
+++ clang/lib/Serialization/ASTWriter.cpp
@@ -1893,6 +1893,7 @@
   // Write out the source location entry table. We skip the first
   // entry, which is always the same dummy entry.
   std::vector SLocEntryOffsets;
+  uint64_t SLocEntryOffsetsBase = Stream.GetCurrentBitNo();
   RecordData PreloadSLocs;
   SLocEntryOffsets.reserve(SourceMgr.local_sloc_entry_size() - 1);
   for (unsigned I = 1, N = SourceMgr.local_sloc_entry_size();
@@ -1903,7 +1904,9 @@
 assert(&SourceMgr.getSLocEntry(FID) == SLoc);
 
 // Record the offset of this source-location entry.
-SLocEntryOffsets.push_back(Stream.GetCurrentBitNo());
+uint64_t Offset = Stream.GetCurrentBitNo() - SLocEntryOffsetsBase;
+assert((Offset >> 32) == 0 && "SLocEntry offset too large");
+SLocEntryOffsets.push_back(Offset);
 
 // Figure out which record code to use.
 unsigned Code;
@@ -2011,12 +2014,14 @@
   Abbrev->Add(BitCodeAbbrevOp(SOURCE_LOCATION_OFFSETS));
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 16)); // # of slocs
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 16)); // total size
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 32)); // base offset
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); // offsets
   unsigned SLocOffsetsAbbrev = Stream.EmitAbbrev(std::move(Abbrev));
   {
 RecordData::value_type Record[] = {
 SOURCE_LOCATION_OFFSETS, SLocEntryOffsets.size(),
-SourceMgr.getNextLocalOffset() - 1 /* skip dummy */};
+SourceMgr.getNextLocalOffset() - 1 /* skip dummy */,
+SLocEntryOffsetsBase};
 Stream.EmitRecordWithBlob(SLocOffsetsAbbrev, Record,
   bytes(SLocEntryOffsets));
   }
@@ -2093,9 +2098,11 @@
 /// Writes the block containing the serialized form of the
 /// preprocessor.
 void ASTWriter::WritePreprocessor(const Preprocessor &PP, bool IsModule) {
+  uint64_t MacroOffsetsBase = Stream.GetCurrentBitNo();
+
   PreprocessingRecord *PPRec = PP.getPreprocessingRecord();
   if (PPRec)
-WritePreprocessorDetail(*PPRec);
+WritePreprocessorDetail(*PPRec, MacroOffsetsBase);
 
   RecordData Record;
   RecordData ModuleMacroRecord;
@@ -2156,7 +2163,8 @@
   // identifier they belong to.
   for (const IdentifierInfo *Name : MacroIdentifiers) {
 MacroDirective *MD = PP.getLocalMacroDirectiveHistory(Name);
-auto StartOffset = Stream.GetCurrentBitNo();
+uint64_t StartOffset = Stream.GetCurrentBitNo() - MacroOffsetsBase;
+assert((StartOffset >> 32) == 0 && "Macro identifiers offset too large");
 
 // Emit the macro directives in reverse source order.
 for (; MD; MD = MD->getPrevious()) {
@@ -2229,14 +2237,12 @@
 
 // Record the local offset of this macro.
 unsigned Index = ID - FirstMacroID;
-if (Index == MacroOffsets.size())
-  MacroOffsets.push_back(Stream.GetCurrentBitNo());
-else {
-  if (Index > MacroOffsets.size())
-MacroOffsets.resize(Index + 1);
+if (Index >= MacroOffsets.size())
+  MacroOffsets.resize(Index + 1);
 
-  MacroOffsets[Index] = Stream.GetCurrentBitNo();
-}
+uint64_t Offset = Stream.GetCurrentBitNo() - MacroOffsetsBase;
+assert((Offset >> 32) == 0 && "Macro offset too large");
+MacroOffsets[Index] = Offset;
 
 AddIdentifierRef(Name, Record);
 AddSourceLocation(MI->getDefinitionLoc(), Record);
@@ -2287,17 +2293,20 @@
   Abbrev->Add(BitCodeAbbrevOp(MACRO_OFFSET)

[PATCH] D76594: [clang][AST] Support AST files larger than 512M

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 257635.
DmitryPolukhin added a comment.

Fix nit


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D76594/new/

https://reviews.llvm.org/D76594

Files:
  clang/include/clang/Serialization/ASTBitCodes.h
  clang/include/clang/Serialization/ASTReader.h
  clang/include/clang/Serialization/ASTWriter.h
  clang/include/clang/Serialization/ModuleFile.h
  clang/lib/Serialization/ASTReader.cpp
  clang/lib/Serialization/ASTReaderDecl.cpp
  clang/lib/Serialization/ASTWriter.cpp
  clang/lib/Serialization/ASTWriterDecl.cpp

Index: clang/lib/Serialization/ASTWriterDecl.cpp
===
--- clang/lib/Serialization/ASTWriterDecl.cpp
+++ clang/lib/Serialization/ASTWriterDecl.cpp
@@ -2422,12 +2422,12 @@
   SourceLocation Loc = D->getLocation();
   unsigned Index = ID - FirstDeclID;
   if (DeclOffsets.size() == Index)
-DeclOffsets.push_back(DeclOffset(Loc, Offset));
+DeclOffsets.emplace_back(Loc, Offset);
   else if (DeclOffsets.size() < Index) {
 // FIXME: Can/should this happen?
 DeclOffsets.resize(Index+1);
 DeclOffsets[Index].setLocation(Loc);
-DeclOffsets[Index].BitOffset = Offset;
+DeclOffsets[Index].setBitOffset(Offset);
   } else {
 llvm_unreachable("declarations should be emitted in ID order");
   }
Index: clang/lib/Serialization/ASTWriter.cpp
===
--- clang/lib/Serialization/ASTWriter.cpp
+++ clang/lib/Serialization/ASTWriter.cpp
@@ -1893,6 +1893,7 @@
   // Write out the source location entry table. We skip the first
   // entry, which is always the same dummy entry.
   std::vector SLocEntryOffsets;
+  uint64_t SLocEntryOffsetsBase = Stream.GetCurrentBitNo();
   RecordData PreloadSLocs;
   SLocEntryOffsets.reserve(SourceMgr.local_sloc_entry_size() - 1);
   for (unsigned I = 1, N = SourceMgr.local_sloc_entry_size();
@@ -1903,7 +1904,9 @@
 assert(&SourceMgr.getSLocEntry(FID) == SLoc);
 
 // Record the offset of this source-location entry.
-SLocEntryOffsets.push_back(Stream.GetCurrentBitNo());
+uint64_t Offset = Stream.GetCurrentBitNo() - SLocEntryOffsetsBase;
+assert((Offset >> 32) == 0 && "SLocEntry offset too large");
+SLocEntryOffsets.push_back(Offset);
 
 // Figure out which record code to use.
 unsigned Code;
@@ -2011,12 +2014,14 @@
   Abbrev->Add(BitCodeAbbrevOp(SOURCE_LOCATION_OFFSETS));
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 16)); // # of slocs
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 16)); // total size
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 32)); // base offset
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); // offsets
   unsigned SLocOffsetsAbbrev = Stream.EmitAbbrev(std::move(Abbrev));
   {
 RecordData::value_type Record[] = {
 SOURCE_LOCATION_OFFSETS, SLocEntryOffsets.size(),
-SourceMgr.getNextLocalOffset() - 1 /* skip dummy */};
+SourceMgr.getNextLocalOffset() - 1 /* skip dummy */,
+SLocEntryOffsetsBase};
 Stream.EmitRecordWithBlob(SLocOffsetsAbbrev, Record,
   bytes(SLocEntryOffsets));
   }
@@ -2093,9 +2098,11 @@
 /// Writes the block containing the serialized form of the
 /// preprocessor.
 void ASTWriter::WritePreprocessor(const Preprocessor &PP, bool IsModule) {
+  uint64_t MacroOffsetsBase = Stream.GetCurrentBitNo();
+
   PreprocessingRecord *PPRec = PP.getPreprocessingRecord();
   if (PPRec)
-WritePreprocessorDetail(*PPRec);
+WritePreprocessorDetail(*PPRec, MacroOffsetsBase);
 
   RecordData Record;
   RecordData ModuleMacroRecord;
@@ -2156,7 +2163,8 @@
   // identifier they belong to.
   for (const IdentifierInfo *Name : MacroIdentifiers) {
 MacroDirective *MD = PP.getLocalMacroDirectiveHistory(Name);
-auto StartOffset = Stream.GetCurrentBitNo();
+uint64_t StartOffset = Stream.GetCurrentBitNo() - MacroOffsetsBase;
+assert((StartOffset >> 32) == 0 && "Macro identifiers offset too large");
 
 // Emit the macro directives in reverse source order.
 for (; MD; MD = MD->getPrevious()) {
@@ -2229,14 +2237,12 @@
 
 // Record the local offset of this macro.
 unsigned Index = ID - FirstMacroID;
-if (Index == MacroOffsets.size())
-  MacroOffsets.push_back(Stream.GetCurrentBitNo());
-else {
-  if (Index > MacroOffsets.size())
-MacroOffsets.resize(Index + 1);
+if (Index >= MacroOffsets.size())
+  MacroOffsets.resize(Index + 1);
 
-  MacroOffsets[Index] = Stream.GetCurrentBitNo();
-}
+uint64_t Offset = Stream.GetCurrentBitNo() - MacroOffsetsBase;
+assert((Offset >> 32) == 0 && "Macro offset too large");
+MacroOffsets[Index] = Offset;
 
 AddIdentifierRef(Name, Record);
 AddSourceLocation(MI->getDefinitionLoc(), Record);
@@ -2287,17 +2293,20 @@
   Abbrev->Add(BitCodeAbbrevOp(MACRO_OFFSET));
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevO

[PATCH] D76594: [clang][AST] Support AST files larger than 512M

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin marked an inline comment as done.
DmitryPolukhin added a comment.

@sammccall thank you for review!
I'll wait for one more day for additional feedback if any, and land this diff.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D76594/new/

https://reviews.llvm.org/D76594



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D75184: [clang-tidy] Optional inheritance of file configs from parent directories

[Dmitry Polukhin via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rGcb1ee34e9d32: [clang-tidy] Optional inheritance of file 
configs from parent directories  (authored by DmitryPolukhin).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D75184/new/

https://reviews.llvm.org/D75184

Files:
  clang-tools-extra/clang-tidy/ClangTidy.cpp
  clang-tools-extra/clang-tidy/ClangTidyCheck.cpp
  clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.cpp
  clang-tools-extra/clang-tidy/ClangTidyOptions.cpp
  clang-tools-extra/clang-tidy/ClangTidyOptions.h
  clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp
  clang-tools-extra/docs/clang-tidy/index.rst
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/3/.clang-tidy
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/.clang-tidy
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/44/.clang-tidy
  clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
  clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp

Index: clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp
===
--- clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp
+++ clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp
@@ -87,7 +87,7 @@
   ExtraArgsBefore: ['arg-before3', 'arg-before4']
   )");
   ASSERT_TRUE(!!Options2);
-  ClangTidyOptions Options = Options1->mergeWith(*Options2);
+  ClangTidyOptions Options = Options1->mergeWith(*Options2, 0);
   EXPECT_EQ("check1,check2,check3,check4", *Options.Checks);
   EXPECT_EQ("filter2", *Options.HeaderFilterRegex);
   EXPECT_EQ("user2", *Options.User);
Index: clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
===
--- clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
+++ clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp
@@ -7,6 +7,26 @@
 // RUN: clang-tidy -dump-config %S/Inputs/config-files/2/- -- | FileCheck %s -check-prefix=CHECK-CHILD2
 // CHECK-CHILD2: Checks: {{.*}}from-parent
 // CHECK-CHILD2: HeaderFilterRegex: parent
+// RUN: clang-tidy -dump-config %S/Inputs/config-files/3/- -- | FileCheck %s -check-prefix=CHECK-CHILD3
+// CHECK-CHILD3: Checks: {{.*}}from-parent,from-child3
+// CHECK-CHILD3: HeaderFilterRegex: child3
 // RUN: clang-tidy -dump-config -checks='from-command-line' -header-filter='from command line' %S/Inputs/config-files/- -- | FileCheck %s -check-prefix=CHECK-COMMAND-LINE
 // CHECK-COMMAND-LINE: Checks: {{.*}}from-parent,from-command-line
 // CHECK-COMMAND-LINE: HeaderFilterRegex: from command line
+
+// For this test we have to use names of the real checks because otherwise values are ignored.
+// RUN: clang-tidy -dump-config %S/Inputs/config-files/4/44/- -- | FileCheck %s -check-prefix=CHECK-CHILD4
+// CHECK-CHILD4: Checks: {{.*}}modernize-loop-convert,modernize-use-using,llvm-qualified-auto
+// CHECK-CHILD4: - key: llvm-qualified-auto.AddConstToQualified
+// CHECK-CHILD4-NEXT: value: '1
+// CHECK-CHILD4: - key: modernize-loop-convert.MaxCopySize
+// CHECK-CHILD4-NEXT: value: '20'
+// CHECK-CHILD4: - key: modernize-loop-convert.MinConfidence
+// CHECK-CHILD4-NEXT: value: reasonable
+// CHECK-CHILD4: - key: modernize-use-using.IgnoreMacros
+// CHECK-CHILD4-NEXT: value: '0'
+
+// RUN: clang-tidy --explain-config %S/Inputs/config-files/4/44/- -- | FileCheck %s -check-prefix=CHECK-EXPLAIN
+// CHECK-EXPLAIN: 'llvm-qualified-auto' is enabled in the {{.*}}/Inputs/config-files/4/44/.clang-tidy.
+// CHECK-EXPLAIN: 'modernize-loop-convert' is enabled in the {{.*}}/Inputs/config-files/4/.clang-tidy.
+// CHECK-EXPLAIN: 'modernize-use-using' is enabled in the {{.*}}/Inputs/config-files/4/.clang-tidy.
Index: clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/44/.clang-tidy
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/44/.clang-tidy
@@ -0,0 +1,9 @@
+InheritParentConfig: true
+Checks: 'llvm-qualified-auto'
+CheckOptions:
+  - key: modernize-loop-convert.MaxCopySize
+value:   '20'
+  - key: llvm-qualified-auto.AddConstToQualified
+value:   '1'
+  - key: IgnoreMacros
+value:   '0'
Index: clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/.clang-tidy
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-files/4/.clang-tidy
@@ -0,0 +1,8 @@
+Checks: '-*,modernize-loop-convert,modernize-use-using'
+CheckOptions:
+  - key: modernize-loop-convert.MaxCopySize
+value:   '10'
+  - key: modernize-loop-convert.MinConfidence
+value:   reasonable

[PATCH] D75184: [clang-tidy] Optional inheritance of file configs from parent directories

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

@thakis sorry, I didn't notice the issue because of massive breakage with diff 
landed just be bore mine and also cmake issues on Windows bots. Thank you for 
trying to fix it.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D75184/new/

https://reviews.llvm.org/D75184



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D75184: [clang-tidy] Optional inheritance of file configs from parent directories

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

@thakis, I don't see this bot on LLVM http://lab.llvm.org:8011/console
Windows bots there still fail due to cmake issues. The issue is very real and 
thank you for pointing out but how should I find the bot?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D75184/new/

https://reviews.llvm.org/D75184



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80301: [yaml][clang-tidy] Fix new line YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 268583.
DmitryPolukhin added a comment.

Apply suggested changes with string split


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301

Files:
  clang/include/clang/Tooling/ReplacementsYaml.h
  llvm/lib/Support/YAMLTraits.cpp
  llvm/unittests/Support/YAMLIOTest.cpp


Index: llvm/unittests/Support/YAMLIOTest.cpp
===
--- llvm/unittests/Support/YAMLIOTest.cpp
+++ llvm/unittests/Support/YAMLIOTest.cpp
@@ -274,8 +274,8 @@
 
 TEST(YAMLIO, MultilineStrings) {
   WithStringField Original;
-  Original.str1 = "a multiline string\nfoobarbaz";
-  Original.str2 = "another one\rfoobarbaz";
+  Original.str1 = "a\n\nmultiline\nstring\nfoobarbaz";
+  Original.str2 = "another one\rfoobarbaz\n";
   Original.str3 = "a one-line string";
 
   std::string Serialized;
@@ -285,10 +285,10 @@
 YOut << Original;
   }
   auto Expected = "---\n"
-  "str1:'a multiline string\n"
+  "str1:'a\n\n\n\nmultiline\n\nstring\n\n"
   "foobarbaz'\n"
   "str2:'another one\r"
-  "foobarbaz'\n"
+  "foobarbaz\n\n'\n"
   "str3:a one-line string\n"
   "...\n";
   ASSERT_EQ(Serialized, Expected);
Index: llvm/lib/Support/YAMLTraits.cpp
===
--- llvm/lib/Support/YAMLTraits.cpp
+++ llvm/lib/Support/YAMLTraits.cpp
@@ -876,13 +876,33 @@
 }
 
 void ScalarTraits::output(const std::string &Val, void *,
- raw_ostream &Out) {
-  Out << Val;
+   raw_ostream &Out) {
+  SmallVector Lines;
+  StringRef(Val).split(Lines, '\n');
+  bool First = true;
+  for (StringRef Line : Lines) {
+if (First)
+  First = false;
+else
+  Out << "\n\n";
+Out << Line;
+  }
 }
 
 StringRef ScalarTraits::input(StringRef Scalar, void *,
- std::string &Val) {
-  Val = Scalar.str();
+   std::string &Val) {
+  Val.clear();
+  size_t CurrentPos = 0;
+  size_t LineBreakPos = Scalar.find('\n');
+  while (LineBreakPos != std::string::npos) {
+// '\n\n' convert to '\n' and don't copy single '\n'.
+if (CurrentPos + 1 < Scalar.size() && Scalar[LineBreakPos + 1] == '\n')
+  ++LineBreakPos;
+Val += Scalar.substr(CurrentPos, LineBreakPos - CurrentPos);
+CurrentPos = LineBreakPos + 1;
+LineBreakPos = Scalar.find('\n', CurrentPos);
+  }
+  Val += Scalar.substr(CurrentPos, Scalar.size() - CurrentPos);
   return StringRef();
 }
 
Index: clang/include/clang/Tooling/ReplacementsYaml.h
===
--- clang/include/clang/Tooling/ReplacementsYaml.h
+++ clang/include/clang/Tooling/ReplacementsYaml.h
@@ -35,13 +35,7 @@
 
 NormalizedReplacement(const IO &, const clang::tooling::Replacement &R)
 : FilePath(R.getFilePath()), Offset(R.getOffset()),
-  Length(R.getLength()), ReplacementText(R.getReplacementText()) {
-  size_t lineBreakPos = ReplacementText.find('\n');
-  while (lineBreakPos != std::string::npos) {
-ReplacementText.replace(lineBreakPos, 1, "\n\n");
-lineBreakPos = ReplacementText.find('\n', lineBreakPos + 2);
-  }
-}
+  Length(R.getLength()), ReplacementText(R.getReplacementText()) {}
 
 clang::tooling::Replacement denormalize(const IO &) {
   return clang::tooling::Replacement(FilePath, Offset, Length,


Index: llvm/unittests/Support/YAMLIOTest.cpp
===
--- llvm/unittests/Support/YAMLIOTest.cpp
+++ llvm/unittests/Support/YAMLIOTest.cpp
@@ -274,8 +274,8 @@
 
 TEST(YAMLIO, MultilineStrings) {
   WithStringField Original;
-  Original.str1 = "a multiline string\nfoobarbaz";
-  Original.str2 = "another one\rfoobarbaz";
+  Original.str1 = "a\n\nmultiline\nstring\nfoobarbaz";
+  Original.str2 = "another one\rfoobarbaz\n";
   Original.str3 = "a one-line string";
 
   std::string Serialized;
@@ -285,10 +285,10 @@
 YOut << Original;
   }
   auto Expected = "---\n"
-  "str1:'a multiline string\n"
+  "str1:'a\n\n\n\nmultiline\n\nstring\n\n"
   "foobarbaz'\n"
   "str2:'another one\r"
-  "foobarbaz'\n"
+  "foobarbaz\n\n'\n"
   "str3:a one-line string\n"
   "...\n";
   ASSERT_EQ(Serialized, Expected);
Index: llvm/lib/Support/YAMLTraits.cpp
===
--- llvm/lib/Support/YAMLTraits.cpp
+++ llvm/lib/Support/YAMLTraits.cpp
@@ -876,13 +876,33 @@
 }
 
 void ScalarTraits::output(con

[PATCH] D80301: [yaml][clang-tidy] Fix new line YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added inline comments.



Comment at: llvm/lib/Support/YAMLTraits.cpp:894
+   std::string &Val) {
+  Val.clear();
+  size_t CurrentPos = 0;

njames93 wrote:
> If you want to do the same here...
> ```
>   SmallVector Lines;
>   Scalar.split(Lines, "\n\n");
>   Val = llvm::join(Lines, "\n");
>   return StringRef();```
Personally I don't like approach with string split because it is potentially 
less efficient due to memory allocations in case of multiple newlines that can 
be avoided. BUT I would like to move the needle and fix this real bug in clang 
ASAP. So I'm ready to rewrite it to something that (1) works and (2) moves 
review forward. So thank you for review and please take another look.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80301: [yaml][clang-tidy] Fix new line YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 268736.
DmitryPolukhin marked an inline comment as done.
DmitryPolukhin added a comment.

Rewrite input string split too


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301

Files:
  clang/include/clang/Tooling/ReplacementsYaml.h
  llvm/lib/Support/YAMLTraits.cpp
  llvm/unittests/Support/YAMLIOTest.cpp

Index: llvm/unittests/Support/YAMLIOTest.cpp
===
--- llvm/unittests/Support/YAMLIOTest.cpp
+++ llvm/unittests/Support/YAMLIOTest.cpp
@@ -274,8 +274,8 @@
 
 TEST(YAMLIO, MultilineStrings) {
   WithStringField Original;
-  Original.str1 = "a multiline string\nfoobarbaz";
-  Original.str2 = "another one\rfoobarbaz";
+  Original.str1 = "a\n\nmultiline\nstring\nfoobarbaz";
+  Original.str2 = "another one\rfoobarbaz\n";
   Original.str3 = "a one-line string";
 
   std::string Serialized;
@@ -285,10 +285,10 @@
 YOut << Original;
   }
   auto Expected = "---\n"
-  "str1:'a multiline string\n"
+  "str1:'a\n\n\n\nmultiline\n\nstring\n\n"
   "foobarbaz'\n"
   "str2:'another one\r"
-  "foobarbaz'\n"
+  "foobarbaz\n\n'\n"
   "str3:a one-line string\n"
   "...\n";
   ASSERT_EQ(Serialized, Expected);
@@ -305,6 +305,25 @@
   EXPECT_EQ(Original.str1, Deserialized.str1);
   EXPECT_EQ(Original.str2, Deserialized.str2);
   EXPECT_EQ(Original.str3, Deserialized.str3);
+
+  // Check deserialization of single '\n' that should be ignored.
+  Serialized = "---\n"
+   "str1:'\na\n\n\n\nmultiline\n\nstring\n\n"
+   "foobarbaz\n'\n"
+   "str2:'another\n one\r"
+   "foobarbaz\n\n'\n"
+   "str3:a one-line string\n"
+   "...\n";
+  {
+Input YIn(Serialized);
+YIn >> Deserialized;
+ASSERT_FALSE(YIn.error())
+<< "Parsing error occurred during deserialization. Serialized string:\n"
+<< Serialized;
+  }
+  EXPECT_EQ(Original.str1, Deserialized.str1);
+  EXPECT_EQ(Original.str2, Deserialized.str2);
+  EXPECT_EQ(Original.str3, Deserialized.str3);
 }
 
 TEST(YAMLIO, NoQuotesForTab) {
Index: llvm/lib/Support/YAMLTraits.cpp
===
--- llvm/lib/Support/YAMLTraits.cpp
+++ llvm/lib/Support/YAMLTraits.cpp
@@ -876,13 +876,35 @@
 }
 
 void ScalarTraits::output(const std::string &Val, void *,
- raw_ostream &Out) {
-  Out << Val;
+   raw_ostream &Out) {
+  SmallVector Lines;
+  StringRef(Val).split(Lines, '\n');
+  bool First = true;
+  for (StringRef Line : Lines) {
+if (First)
+  First = false;
+else
+  Out << "\n\n";
+Out << Line;
+  }
 }
 
 StringRef ScalarTraits::input(StringRef Scalar, void *,
- std::string &Val) {
-  Val = Scalar.str();
+   std::string &Val) {
+  Val.clear();
+  SmallVector Lines;
+  Scalar.split(Lines, '\n');
+  size_t C = Lines.size();
+  for (size_t I = 0; I < C; ++I) {
+Val += Lines[I];
+// Next empty line means that it was '\n\n' that should convert to '\n'
+// single '\n' should be skiped. +2 here for the right hadling single '\n'
+// at the end of line.
+if (I + 2 < C && Lines[I + 1].empty()) {
+  Val += '\n';
+  ++I;
+}
+  }
   return StringRef();
 }
 
Index: clang/include/clang/Tooling/ReplacementsYaml.h
===
--- clang/include/clang/Tooling/ReplacementsYaml.h
+++ clang/include/clang/Tooling/ReplacementsYaml.h
@@ -35,13 +35,7 @@
 
 NormalizedReplacement(const IO &, const clang::tooling::Replacement &R)
 : FilePath(R.getFilePath()), Offset(R.getOffset()),
-  Length(R.getLength()), ReplacementText(R.getReplacementText()) {
-  size_t lineBreakPos = ReplacementText.find('\n');
-  while (lineBreakPos != std::string::npos) {
-ReplacementText.replace(lineBreakPos, 1, "\n\n");
-lineBreakPos = ReplacementText.find('\n', lineBreakPos + 2);
-  }
-}
+  Length(R.getLength()), ReplacementText(R.getReplacementText()) {}
 
 clang::tooling::Replacement denormalize(const IO &) {
   return clang::tooling::Replacement(FilePath, Offset, Length,
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80301: [yaml][clang-tidy] Fix new line YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 268737.
DmitryPolukhin marked 2 inline comments as done.
DmitryPolukhin added a comment.

Fix spelling


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301

Files:
  clang/include/clang/Tooling/ReplacementsYaml.h
  llvm/lib/Support/YAMLTraits.cpp
  llvm/unittests/Support/YAMLIOTest.cpp

Index: llvm/unittests/Support/YAMLIOTest.cpp
===
--- llvm/unittests/Support/YAMLIOTest.cpp
+++ llvm/unittests/Support/YAMLIOTest.cpp
@@ -274,8 +274,8 @@
 
 TEST(YAMLIO, MultilineStrings) {
   WithStringField Original;
-  Original.str1 = "a multiline string\nfoobarbaz";
-  Original.str2 = "another one\rfoobarbaz";
+  Original.str1 = "a\n\nmultiline\nstring\nfoobarbaz";
+  Original.str2 = "another one\rfoobarbaz\n";
   Original.str3 = "a one-line string";
 
   std::string Serialized;
@@ -285,10 +285,10 @@
 YOut << Original;
   }
   auto Expected = "---\n"
-  "str1:'a multiline string\n"
+  "str1:'a\n\n\n\nmultiline\n\nstring\n\n"
   "foobarbaz'\n"
   "str2:'another one\r"
-  "foobarbaz'\n"
+  "foobarbaz\n\n'\n"
   "str3:a one-line string\n"
   "...\n";
   ASSERT_EQ(Serialized, Expected);
@@ -305,6 +305,25 @@
   EXPECT_EQ(Original.str1, Deserialized.str1);
   EXPECT_EQ(Original.str2, Deserialized.str2);
   EXPECT_EQ(Original.str3, Deserialized.str3);
+
+  // Check deserialization of single '\n' that should be ignored.
+  Serialized = "---\n"
+   "str1:'\na\n\n\n\nmultiline\n\nstring\n\n"
+   "foobarbaz\n'\n"
+   "str2:'another\n one\r"
+   "foobarbaz\n\n'\n"
+   "str3:a one-line string\n"
+   "...\n";
+  {
+Input YIn(Serialized);
+YIn >> Deserialized;
+ASSERT_FALSE(YIn.error())
+<< "Parsing error occurred during deserialization. Serialized string:\n"
+<< Serialized;
+  }
+  EXPECT_EQ(Original.str1, Deserialized.str1);
+  EXPECT_EQ(Original.str2, Deserialized.str2);
+  EXPECT_EQ(Original.str3, Deserialized.str3);
 }
 
 TEST(YAMLIO, NoQuotesForTab) {
Index: llvm/lib/Support/YAMLTraits.cpp
===
--- llvm/lib/Support/YAMLTraits.cpp
+++ llvm/lib/Support/YAMLTraits.cpp
@@ -876,13 +876,35 @@
 }
 
 void ScalarTraits::output(const std::string &Val, void *,
- raw_ostream &Out) {
-  Out << Val;
+   raw_ostream &Out) {
+  SmallVector Lines;
+  StringRef(Val).split(Lines, '\n');
+  bool First = true;
+  for (StringRef Line : Lines) {
+if (First)
+  First = false;
+else
+  Out << "\n\n";
+Out << Line;
+  }
 }
 
 StringRef ScalarTraits::input(StringRef Scalar, void *,
- std::string &Val) {
-  Val = Scalar.str();
+   std::string &Val) {
+  Val.clear();
+  SmallVector Lines;
+  Scalar.split(Lines, '\n');
+  size_t C = Lines.size();
+  for (size_t I = 0; I < C; ++I) {
+Val += Lines[I];
+// Next empty line means that it was '\n\n' that should convert to '\n'.
+// Single '\n' should be skipped. +2 here for the right handling
+// single '\n' at the end of line.
+if (I + 2 < C && Lines[I + 1].empty()) {
+  Val += '\n';
+  ++I;
+}
+  }
   return StringRef();
 }
 
Index: clang/include/clang/Tooling/ReplacementsYaml.h
===
--- clang/include/clang/Tooling/ReplacementsYaml.h
+++ clang/include/clang/Tooling/ReplacementsYaml.h
@@ -35,13 +35,7 @@
 
 NormalizedReplacement(const IO &, const clang::tooling::Replacement &R)
 : FilePath(R.getFilePath()), Offset(R.getOffset()),
-  Length(R.getLength()), ReplacementText(R.getReplacementText()) {
-  size_t lineBreakPos = ReplacementText.find('\n');
-  while (lineBreakPos != std::string::npos) {
-ReplacementText.replace(lineBreakPos, 1, "\n\n");
-lineBreakPos = ReplacementText.find('\n', lineBreakPos + 2);
-  }
-}
+  Length(R.getLength()), ReplacementText(R.getReplacementText()) {}
 
 clang::tooling::Replacement denormalize(const IO &) {
   return clang::tooling::Replacement(FilePath, Offset, Length,
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80301: [yaml][clang-tidy] Fix new line YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 270206.
DmitryPolukhin added a comment.

Fix single new line handling, it should be replace with space


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301

Files:
  clang/include/clang/Tooling/ReplacementsYaml.h
  llvm/lib/Support/YAMLTraits.cpp
  llvm/unittests/Support/YAMLIOTest.cpp

Index: llvm/unittests/Support/YAMLIOTest.cpp
===
--- llvm/unittests/Support/YAMLIOTest.cpp
+++ llvm/unittests/Support/YAMLIOTest.cpp
@@ -274,8 +274,8 @@
 
 TEST(YAMLIO, MultilineStrings) {
   WithStringField Original;
-  Original.str1 = "a multiline string\nfoobarbaz";
-  Original.str2 = "another one\rfoobarbaz";
+  Original.str1 = "a\n\nmultiline\nstring\nfoobarbaz ";
+  Original.str2 = "another one\rfoobarbaz\n";
   Original.str3 = "a one-line string";
 
   std::string Serialized;
@@ -285,10 +285,10 @@
 YOut << Original;
   }
   auto Expected = "---\n"
-  "str1:'a multiline string\n"
-  "foobarbaz'\n"
+  "str1:'a\n\n\n\nmultiline\n\nstring\n\n"
+  "foobarbaz '\n"
   "str2:'another one\r"
-  "foobarbaz'\n"
+  "foobarbaz\n\n'\n"
   "str3:a one-line string\n"
   "...\n";
   ASSERT_EQ(Serialized, Expected);
@@ -305,6 +305,25 @@
   EXPECT_EQ(Original.str1, Deserialized.str1);
   EXPECT_EQ(Original.str2, Deserialized.str2);
   EXPECT_EQ(Original.str3, Deserialized.str3);
+
+  // Check deserialization of single '\n' that should be converted to space.
+  Serialized = "---\n"
+   "str1:'a\n\n\n\nmultiline\n\nstring\n\n"
+   "foobarbaz\n'\n"
+   "str2:'another\none\r"
+   "foobarbaz\n\n'\n"
+   "str3:a one-line string\n"
+   "...\n";
+  {
+Input YIn(Serialized);
+YIn >> Deserialized;
+ASSERT_FALSE(YIn.error())
+<< "Parsing error occurred during deserialization. Serialized string:\n"
+<< Serialized;
+  }
+  EXPECT_EQ(Original.str1, Deserialized.str1);
+  EXPECT_EQ(Original.str2, Deserialized.str2);
+  EXPECT_EQ(Original.str3, Deserialized.str3);
 }
 
 TEST(YAMLIO, NoQuotesForTab) {
Index: llvm/lib/Support/YAMLTraits.cpp
===
--- llvm/lib/Support/YAMLTraits.cpp
+++ llvm/lib/Support/YAMLTraits.cpp
@@ -876,13 +876,38 @@
 }
 
 void ScalarTraits::output(const std::string &Val, void *,
- raw_ostream &Out) {
-  Out << Val;
+   raw_ostream &Out) {
+  SmallVector Lines;
+  StringRef(Val).split(Lines, '\n');
+  bool First = true;
+  for (StringRef Line : Lines) {
+if (First)
+  First = false;
+else
+  Out << "\n\n";
+Out << Line;
+  }
 }
 
 StringRef ScalarTraits::input(StringRef Scalar, void *,
- std::string &Val) {
-  Val = Scalar.str();
+   std::string &Val) {
+  Val.clear();
+  SmallVector Lines;
+  Scalar.split(Lines, '\n');
+  size_t C = Lines.size();
+  for (size_t I = 0; I < C; ++I) {
+Val += Lines[I];
+// Next empty line means that it was '\n\n' that should convert to '\n'.
+// +2 here because we need separator only if more elements will be added.
+if (I + 2 < C && Lines[I + 1].empty()) {
+  Val += '\n';
+  ++I;
+} else if ((I + 1 < C && !Lines[I + 1].empty()) ||
+   (I + 2 == C && Lines[I + 1].empty())) {
+  // Single '\n' should be converted to space ' '.
+  Val += ' ';
+}
+  }
   return StringRef();
 }
 
Index: clang/include/clang/Tooling/ReplacementsYaml.h
===
--- clang/include/clang/Tooling/ReplacementsYaml.h
+++ clang/include/clang/Tooling/ReplacementsYaml.h
@@ -35,13 +35,7 @@
 
 NormalizedReplacement(const IO &, const clang::tooling::Replacement &R)
 : FilePath(R.getFilePath()), Offset(R.getOffset()),
-  Length(R.getLength()), ReplacementText(R.getReplacementText()) {
-  size_t lineBreakPos = ReplacementText.find('\n');
-  while (lineBreakPos != std::string::npos) {
-ReplacementText.replace(lineBreakPos, 1, "\n\n");
-lineBreakPos = ReplacementText.find('\n', lineBreakPos + 2);
-  }
-}
+  Length(R.getLength()), ReplacementText(R.getReplacementText()) {}
 
 clang::tooling::Replacement denormalize(const IO &) {
   return clang::tooling::Replacement(FilePath, Offset, Length,
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80301: [yaml][clang-tidy] Fix new line YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 270351.
DmitryPolukhin added a comment.

Rebase


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301

Files:
  clang/include/clang/Tooling/ReplacementsYaml.h
  llvm/lib/Support/YAMLTraits.cpp
  llvm/unittests/Support/YAMLIOTest.cpp

Index: llvm/unittests/Support/YAMLIOTest.cpp
===
--- llvm/unittests/Support/YAMLIOTest.cpp
+++ llvm/unittests/Support/YAMLIOTest.cpp
@@ -274,8 +274,8 @@
 
 TEST(YAMLIO, MultilineStrings) {
   WithStringField Original;
-  Original.str1 = "a multiline string\nfoobarbaz";
-  Original.str2 = "another one\rfoobarbaz";
+  Original.str1 = "a\n\nmultiline\nstring\nfoobarbaz ";
+  Original.str2 = "another one\rfoobarbaz\n";
   Original.str3 = "a one-line string";
 
   std::string Serialized;
@@ -285,10 +285,10 @@
 YOut << Original;
   }
   auto Expected = "---\n"
-  "str1:'a multiline string\n"
-  "foobarbaz'\n"
+  "str1:'a\n\n\n\nmultiline\n\nstring\n\n"
+  "foobarbaz '\n"
   "str2:'another one\r"
-  "foobarbaz'\n"
+  "foobarbaz\n\n'\n"
   "str3:a one-line string\n"
   "...\n";
   ASSERT_EQ(Serialized, Expected);
@@ -305,6 +305,25 @@
   EXPECT_EQ(Original.str1, Deserialized.str1);
   EXPECT_EQ(Original.str2, Deserialized.str2);
   EXPECT_EQ(Original.str3, Deserialized.str3);
+
+  // Check deserialization of single '\n' that should be converted to space.
+  Serialized = "---\n"
+   "str1:'a\n\n\n\nmultiline\n\nstring\n\n"
+   "foobarbaz\n'\n"
+   "str2:'another\none\r"
+   "foobarbaz\n\n'\n"
+   "str3:a one-line string\n"
+   "...\n";
+  {
+Input YIn(Serialized);
+YIn >> Deserialized;
+ASSERT_FALSE(YIn.error())
+<< "Parsing error occurred during deserialization. Serialized string:\n"
+<< Serialized;
+  }
+  EXPECT_EQ(Original.str1, Deserialized.str1);
+  EXPECT_EQ(Original.str2, Deserialized.str2);
+  EXPECT_EQ(Original.str3, Deserialized.str3);
 }
 
 TEST(YAMLIO, NoQuotesForTab) {
Index: llvm/lib/Support/YAMLTraits.cpp
===
--- llvm/lib/Support/YAMLTraits.cpp
+++ llvm/lib/Support/YAMLTraits.cpp
@@ -876,13 +876,38 @@
 }
 
 void ScalarTraits::output(const std::string &Val, void *,
- raw_ostream &Out) {
-  Out << Val;
+   raw_ostream &Out) {
+  SmallVector Lines;
+  StringRef(Val).split(Lines, '\n');
+  bool First = true;
+  for (StringRef Line : Lines) {
+if (First)
+  First = false;
+else
+  Out << "\n\n";
+Out << Line;
+  }
 }
 
 StringRef ScalarTraits::input(StringRef Scalar, void *,
- std::string &Val) {
-  Val = Scalar.str();
+   std::string &Val) {
+  Val.clear();
+  SmallVector Lines;
+  Scalar.split(Lines, '\n');
+  size_t C = Lines.size();
+  for (size_t I = 0; I < C; ++I) {
+Val += Lines[I];
+// Next empty line means that it was '\n\n' that should convert to '\n'.
+// +2 here because we need separator only if more elements will be added.
+if (I + 2 < C && Lines[I + 1].empty()) {
+  Val += '\n';
+  ++I;
+} else if ((I + 1 < C && !Lines[I + 1].empty()) ||
+   (I + 2 == C && Lines[I + 1].empty())) {
+  // Single '\n' should be converted to space ' '.
+  Val += ' ';
+}
+  }
   return StringRef();
 }
 
Index: clang/include/clang/Tooling/ReplacementsYaml.h
===
--- clang/include/clang/Tooling/ReplacementsYaml.h
+++ clang/include/clang/Tooling/ReplacementsYaml.h
@@ -35,13 +35,7 @@
 
 NormalizedReplacement(const IO &, const clang::tooling::Replacement &R)
 : FilePath(R.getFilePath()), Offset(R.getOffset()),
-  Length(R.getLength()), ReplacementText(R.getReplacementText()) {
-  size_t lineBreakPos = ReplacementText.find('\n');
-  while (lineBreakPos != std::string::npos) {
-ReplacementText.replace(lineBreakPos, 1, "\n\n");
-lineBreakPos = ReplacementText.find('\n', lineBreakPos + 2);
-  }
-}
+  Length(R.getLength()), ReplacementText(R.getReplacementText()) {}
 
 clang::tooling::Replacement denormalize(const IO &) {
   return clang::tooling::Replacement(FilePath, Offset, Length,
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D79285: [clang-tidy] Add diagnostics level to YAML output

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a reviewer: aaron.ballman.
DmitryPolukhin added a subscriber: aaron.ballman.
DmitryPolukhin added a comment.

@alexfh, @gribozavr2, @aaron.ballman - friendly ping, please take a look to 
this diff. It is just trivial implementation of TODO in the code.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D79285/new/

https://reviews.llvm.org/D79285



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D79285: [clang-tidy] Add diagnostics level to YAML output

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin marked an inline comment as done.
DmitryPolukhin added inline comments.



Comment at: clang/include/clang/Tooling/DiagnosticsYaml.h:109
+  static void enumeration(IO &IO, clang::tooling::Diagnostic::Level &Value) {
+IO.enumCase(Value, "Warning", clang::tooling::Diagnostic::Warning);
+IO.enumCase(Value, "Error", clang::tooling::Diagnostic::Error);

aaron.ballman wrote:
> Do we have to handle notes as well?
clang::tooling::Diagnostic::Level supports only 2 levels error and warning:
https://github.com/llvm/llvm-project/blob/master/clang/include/clang/Tooling/Core/Diagnostic.h#L65


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D79285/new/

https://reviews.llvm.org/D79285



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D79285: [clang-tidy] Add diagnostics level to YAML output

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

@aaron.ballman thank you for review and quick response!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D79285/new/

https://reviews.llvm.org/D79285



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D79285: [clang-tidy] Add diagnostics level to YAML output

[Dmitry Polukhin via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rGc98c94d85f85: [clang-tidy] Add diagnostics level to YAML 
output (authored by DmitryPolukhin).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D79285/new/

https://reviews.llvm.org/D79285

Files:
  clang-tools-extra/test/clang-tidy/infrastructure/export-diagnostics.cpp
  clang/include/clang/Tooling/DiagnosticsYaml.h
  clang/unittests/Tooling/DiagnosticsYamlTest.cpp

Index: clang/unittests/Tooling/DiagnosticsYamlTest.cpp
===
--- clang/unittests/Tooling/DiagnosticsYamlTest.cpp
+++ clang/unittests/Tooling/DiagnosticsYamlTest.cpp
@@ -65,6 +65,8 @@
 "  Offset:  100\n"
 "  Length:  12\n"
 "  ReplacementText: 'replacement #1'\n"
+"Level:   Warning\n"
+"BuildDirectory:  'path/to/build/directory'\n"
 "  - DiagnosticName:  'diagnostic#2'\n"
 "DiagnosticMessage:\n"
 "  Message: 'message #2'\n"
@@ -75,6 +77,8 @@
 "  Offset:  62\n"
 "  Length:  2\n"
 "  ReplacementText: 'replacement #2'\n"
+"Level:   Warning\n"
+"BuildDirectory:  'path/to/build/directory'\n"
 "Ranges:\n"
 "  - FilePath:'path/to/source.cpp'\n"
 "FileOffset:  10\n"
@@ -94,6 +98,8 @@
 "FilePath:'path/to/note2.cpp'\n"
 "FileOffset:  99\n"
 "Replacements:[]\n"
+"Level:   Warning\n"
+"BuildDirectory:  'path/to/build/directory'\n"
 "...\n";
 
 TEST(DiagnosticsYamlTest, serializesDiagnostics) {
Index: clang/include/clang/Tooling/DiagnosticsYaml.h
===
--- clang/include/clang/Tooling/DiagnosticsYaml.h
+++ clang/include/clang/Tooling/DiagnosticsYaml.h
@@ -77,7 +77,6 @@
 
 std::string DiagnosticName;
 clang::tooling::DiagnosticMessage Message;
-llvm::StringMap Fix;
 SmallVector Notes;
 clang::tooling::Diagnostic::Level DiagLevel;
 std::string BuildDirectory;
@@ -90,9 +89,9 @@
 Io.mapRequired("DiagnosticName", Keys->DiagnosticName);
 Io.mapRequired("DiagnosticMessage", Keys->Message);
 Io.mapOptional("Notes", Keys->Notes);
+Io.mapOptional("Level", Keys->DiagLevel);
+Io.mapOptional("BuildDirectory", Keys->BuildDirectory);
 Io.mapOptional("Ranges", Keys->Ranges);
-
-// FIXME: Export properly all the different fields.
   }
 };
 
@@ -104,6 +103,14 @@
 Io.mapRequired("Diagnostics", Doc.Diagnostics);
   }
 };
+
+template <> struct ScalarEnumerationTraits {
+  static void enumeration(IO &IO, clang::tooling::Diagnostic::Level &Value) {
+IO.enumCase(Value, "Warning", clang::tooling::Diagnostic::Warning);
+IO.enumCase(Value, "Error", clang::tooling::Diagnostic::Error);
+  }
+};
+
 } // end namespace yaml
 } // end namespace llvm
 
Index: clang-tools-extra/test/clang-tidy/infrastructure/export-diagnostics.cpp
===
--- clang-tools-extra/test/clang-tidy/infrastructure/export-diagnostics.cpp
+++ clang-tools-extra/test/clang-tidy/infrastructure/export-diagnostics.cpp
@@ -1,15 +1,17 @@
 // RUN: grep -Ev "// *[A-Z-]+:" %s > %t-input.cpp
-// RUN: clang-tidy %t-input.cpp -checks='-*,google-explicit-constructor,clang-diagnostic-missing-prototypes' -export-fixes=%t.yaml -- -Wmissing-prototypes > %t.msg 2>&1
+// RUN: not clang-tidy %t-input.cpp -checks='-*,google-explicit-constructor,clang-diagnostic-missing-prototypes' -export-fixes=%t.yaml -- -Wmissing-prototypes > %t.msg 2>&1
 // RUN: FileCheck -input-file=%t.msg -check-prefix=CHECK-MESSAGES %s -implicit-check-not='{{warning|error|note}}:'
 // RUN: FileCheck -input-file=%t.yaml -check-prefix=CHECK-YAML %s
 #define X(n) void n ## n() {}
 X(f)
+int a[-1];
 
 // CHECK-MESSAGES: -input.cpp:2:1: warning: no previous prototype for function 'ff' [clang-diagnostic-missing-prototypes]
 // CHECK-MESSAGES: -input.cpp:1:19: note: expanded from macro 'X'
 // CHECK-MESSAGES: {{^}}note: expanded from here{{$}}
 // CHECK-MESSAGES: -input.cpp:2:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
 // CHECK-MESSAGES: -input.cpp:1:14: note: expanded from macro 'X'
+// CHECK-MESSAGES: -input.cpp:3:7: error: 'a' declared as an array with a negative size [clang-diagnostic-error]
 
 // CHECK-YAML: ---
 // CHECK-YAML-NEXT: MainSourceFile:  '{{.*}}-input.cpp'
@@ -42,4 +44,18 @@
 // CHECK-YAML-NEXT: FilePath:'{{.*}}-input.cpp'
 // CHECK-YAML-NEXT: FileOffset:  13
 // CHECK-YAML-NEXT: Replacements:[]
+// CHECK-YAML-NEXT: Level:   Warning
+// CHECK-YAML-NEXT: BuildDirectory:  '{{.*}}'
+// CHECK-YAML-NEXT:   - DiagnosticName:  clang-diagnostic-error
+// CHECK-YAML-NEXT: Diagn

[PATCH] D75184: [clang-tidy] Optional inheritance of file configs from parent directories

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

In D75184#2093089 , @njames93 wrote:

> As an afterthought, Would you think it a good idea to extend this logic to 
> the `ConfigOptionsProvider` That way you can use something along the lines of:
>
>   clang-tidy --config='{InheritParentConfig: true, CheckOptions: [ overriding/complimenting .clang-tidy files>]}'
>
>
> clang-tidy would then go to read the .clang-tidy configuration files in much 
> the same was as is done here. This would let you set custom configuration 
> options on the command line on top of whats in configuration files.
>  Obviously is `InheritParentConfig` is unset or `false` then we wouldn't go 
> looking for .clang-tidy files.


It sounds logical. I would even extend it a little bit to allow case 
`--config=@filepath` where filepath file will be read and expanded as the 
highest priority config. @njames93 are you going to work on this?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D75184/new/

https://reviews.llvm.org/D75184



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D81949: [clang-tidy] Extend InheritParentConfig to CommandLineConfig

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin accepted this revision.
DmitryPolukhin added a comment.
This revision is now accepted and ready to land.

LGTM


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D81949/new/

https://reviews.llvm.org/D81949



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80301: [yaml][clang-tidy] Fix new line YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 271232.
DmitryPolukhin added a comment.

Rebase


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301

Files:
  clang/include/clang/Tooling/ReplacementsYaml.h
  llvm/lib/Support/YAMLTraits.cpp
  llvm/unittests/Support/YAMLIOTest.cpp

Index: llvm/unittests/Support/YAMLIOTest.cpp
===
--- llvm/unittests/Support/YAMLIOTest.cpp
+++ llvm/unittests/Support/YAMLIOTest.cpp
@@ -274,8 +274,8 @@
 
 TEST(YAMLIO, MultilineStrings) {
   WithStringField Original;
-  Original.str1 = "a multiline string\nfoobarbaz";
-  Original.str2 = "another one\rfoobarbaz";
+  Original.str1 = "a\n\nmultiline\nstring\nfoobarbaz ";
+  Original.str2 = "another one\rfoobarbaz\n";
   Original.str3 = "a one-line string";
 
   std::string Serialized;
@@ -285,10 +285,10 @@
 YOut << Original;
   }
   auto Expected = "---\n"
-  "str1:'a multiline string\n"
-  "foobarbaz'\n"
+  "str1:'a\n\n\n\nmultiline\n\nstring\n\n"
+  "foobarbaz '\n"
   "str2:'another one\r"
-  "foobarbaz'\n"
+  "foobarbaz\n\n'\n"
   "str3:a one-line string\n"
   "...\n";
   ASSERT_EQ(Serialized, Expected);
@@ -305,6 +305,25 @@
   EXPECT_EQ(Original.str1, Deserialized.str1);
   EXPECT_EQ(Original.str2, Deserialized.str2);
   EXPECT_EQ(Original.str3, Deserialized.str3);
+
+  // Check deserialization of single '\n' that should be converted to space.
+  Serialized = "---\n"
+   "str1:'a\n\n\n\nmultiline\n\nstring\n\n"
+   "foobarbaz\n'\n"
+   "str2:'another\none\r"
+   "foobarbaz\n\n'\n"
+   "str3:a one-line string\n"
+   "...\n";
+  {
+Input YIn(Serialized);
+YIn >> Deserialized;
+ASSERT_FALSE(YIn.error())
+<< "Parsing error occurred during deserialization. Serialized string:\n"
+<< Serialized;
+  }
+  EXPECT_EQ(Original.str1, Deserialized.str1);
+  EXPECT_EQ(Original.str2, Deserialized.str2);
+  EXPECT_EQ(Original.str3, Deserialized.str3);
 }
 
 TEST(YAMLIO, NoQuotesForTab) {
Index: llvm/lib/Support/YAMLTraits.cpp
===
--- llvm/lib/Support/YAMLTraits.cpp
+++ llvm/lib/Support/YAMLTraits.cpp
@@ -878,13 +878,38 @@
 }
 
 void ScalarTraits::output(const std::string &Val, void *,
- raw_ostream &Out) {
-  Out << Val;
+   raw_ostream &Out) {
+  SmallVector Lines;
+  StringRef(Val).split(Lines, '\n');
+  bool First = true;
+  for (StringRef Line : Lines) {
+if (First)
+  First = false;
+else
+  Out << "\n\n";
+Out << Line;
+  }
 }
 
 StringRef ScalarTraits::input(StringRef Scalar, void *,
- std::string &Val) {
-  Val = Scalar.str();
+   std::string &Val) {
+  Val.clear();
+  SmallVector Lines;
+  Scalar.split(Lines, '\n');
+  size_t C = Lines.size();
+  for (size_t I = 0; I < C; ++I) {
+Val += Lines[I];
+// Next empty line means that it was '\n\n' that should convert to '\n'.
+// +2 here because we need separator only if more elements will be added.
+if (I + 2 < C && Lines[I + 1].empty()) {
+  Val += '\n';
+  ++I;
+} else if ((I + 1 < C && !Lines[I + 1].empty()) ||
+   (I + 2 == C && Lines[I + 1].empty())) {
+  // Single '\n' should be converted to space ' '.
+  Val += ' ';
+}
+  }
   return StringRef();
 }
 
Index: clang/include/clang/Tooling/ReplacementsYaml.h
===
--- clang/include/clang/Tooling/ReplacementsYaml.h
+++ clang/include/clang/Tooling/ReplacementsYaml.h
@@ -35,13 +35,7 @@
 
 NormalizedReplacement(const IO &, const clang::tooling::Replacement &R)
 : FilePath(R.getFilePath()), Offset(R.getOffset()),
-  Length(R.getLength()), ReplacementText(R.getReplacementText()) {
-  size_t lineBreakPos = ReplacementText.find('\n');
-  while (lineBreakPos != std::string::npos) {
-ReplacementText.replace(lineBreakPos, 1, "\n\n");
-lineBreakPos = ReplacementText.find('\n', lineBreakPos + 2);
-  }
-}
+  Length(R.getLength()), ReplacementText(R.getReplacementText()) {}
 
 clang::tooling::Replacement denormalize(const IO &) {
   return clang::tooling::Replacement(FilePath, Offset, Length,
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80301: [yaml][clang-tidy] Fix multiline YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

@njames93 and @aaron.ballman - please take a look to this diff. Multiline 
replacements in YAML are broken and cannot be applied correctly.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80301: [yaml][clang-tidy] Fix multiline YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG9e7fddbd36f5: [yaml][clang-tidy] Fix multiline YAML 
serialization (authored by DmitryPolukhin).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301

Files:
  clang/include/clang/Tooling/ReplacementsYaml.h
  clang/unittests/Tooling/ReplacementsYamlTest.cpp
  llvm/include/llvm/Support/YAMLTraits.h
  llvm/lib/Support/YAMLTraits.cpp
  llvm/test/Transforms/LowerMatrixIntrinsics/remarks-shared-subtrees.ll
  llvm/unittests/Support/YAMLIOTest.cpp

Index: llvm/unittests/Support/YAMLIOTest.cpp
===
--- llvm/unittests/Support/YAMLIOTest.cpp
+++ llvm/unittests/Support/YAMLIOTest.cpp
@@ -285,10 +285,8 @@
 YOut << Original;
   }
   auto Expected = "---\n"
-  "str1:'a multiline string\n"
-  "foobarbaz'\n"
-  "str2:'another one\r"
-  "foobarbaz'\n"
+  "str1:\"a multiline string\\nfoobarbaz\"\n"
+  "str2:\"another one\\rfoobarbaz\"\n"
   "str3:a one-line string\n"
   "...\n";
   ASSERT_EQ(Serialized, Expected);
Index: llvm/test/Transforms/LowerMatrixIntrinsics/remarks-shared-subtrees.ll
===
--- llvm/test/Transforms/LowerMatrixIntrinsics/remarks-shared-subtrees.ll
+++ llvm/test/Transforms/LowerMatrixIntrinsics/remarks-shared-subtrees.ll
@@ -18,8 +18,7 @@
 ; YAML-NEXT:- String:  ' loads, '
 ; YAML-NEXT:- NumComputeOps:   '0'
 ; YAML-NEXT:- String:  ' compute ops'
-; YAML-NEXT:- String:  ',
-; YAML-NEXT:  additionally '
+; YAML-NEXT:- String:  ",\nadditionally "
 ; YAML-NEXT:- NumStores:   '0'
 ; YAML-NEXT:- String:  ' stores, '
 ; YAML-NEXT:- NumLoads:'4'
@@ -47,8 +46,7 @@
 ; YAML-NEXT:- String:  ' loads, '
 ; YAML-NEXT:- NumComputeOps:   '120'
 ; YAML-NEXT:- String:  ' compute ops'
-; YAML-NEXT:- String:  ',
-; YAML-NEXT:  additionally '
+; YAML-NEXT:- String:  ",\nadditionally "
 ; YAML-NEXT:- NumStores:   '0'
 ; YAML-NEXT:- String:  ' stores, '
 ; YAML-NEXT:- NumLoads:'4'
Index: llvm/lib/Support/YAMLTraits.cpp
===
--- llvm/lib/Support/YAMLTraits.cpp
+++ llvm/lib/Support/YAMLTraits.cpp
@@ -878,12 +878,12 @@
 }
 
 void ScalarTraits::output(const std::string &Val, void *,
- raw_ostream &Out) {
+   raw_ostream &Out) {
   Out << Val;
 }
 
 StringRef ScalarTraits::input(StringRef Scalar, void *,
- std::string &Val) {
+   std::string &Val) {
   Val = Scalar.str();
   return StringRef();
 }
Index: llvm/include/llvm/Support/YAMLTraits.h
===
--- llvm/include/llvm/Support/YAMLTraits.h
+++ llvm/include/llvm/Support/YAMLTraits.h
@@ -649,24 +649,25 @@
 inline QuotingType needsQuotes(StringRef S) {
   if (S.empty())
 return QuotingType::Single;
+
+  QuotingType MaxQuotingNeeded = QuotingType::None;
   if (isSpace(static_cast(S.front())) ||
   isSpace(static_cast(S.back(
-return QuotingType::Single;
+MaxQuotingNeeded = QuotingType::Single;
   if (isNull(S))
-return QuotingType::Single;
+MaxQuotingNeeded = QuotingType::Single;
   if (isBool(S))
-return QuotingType::Single;
+MaxQuotingNeeded = QuotingType::Single;
   if (isNumeric(S))
-return QuotingType::Single;
+MaxQuotingNeeded = QuotingType::Single;
 
   // 7.3.3 Plain Style
   // Plain scalars must not begin with most indicators, as this would cause
   // ambiguity with other YAML constructs.
   static constexpr char Indicators[] = R"(-?:\,[]{}#&*!|>'"%@`)";
   if (S.find_first_of(Indicators) == 0)
-return QuotingType::Single;
+MaxQuotingNeeded = QuotingType::Single;
 
-  QuotingType MaxQuotingNeeded = QuotingType::None;
   for (unsigned char C : S) {
 // Alphanum is safe.
 if (isAlnum(C))
@@ -684,11 +685,11 @@
 case 0x9:
   continue;
 // LF(0xA) and CR(0xD) may delimit values and so require at least single
-// quotes.
+// quotes. LLVM YAML parser cannot handle single quoted multiline so use
+// double quoting to produce valid YAML.
 case 0xA:
 case 0xD:
-  MaxQuotingNeeded = QuotingType::Single;
-  continue;
+  return QuotingType::Double;
 // DEL (0x7F) are excluded from the allowed character range.
 case 0x7F:
   return QuotingType::Double;
Index: clang/unittests/Tooling/ReplacementsYamlTest.cpp
=

[PATCH] D80301: [yaml][clang-tidy] Fix multiline YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

@aaron.ballman - thank you for the review!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80301: [yaml][clang-tidy] Fix new line YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

@njames93 - friendly ping, could you please take another look.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80301: [yaml][clang-tidy] Fix new line YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

It looks like there is no support for the proposed solution so I found 
alternative solution that might be even better. We can use double quotation `"` 
for multiline strings. It solves problem because in case of double quotation 
LLVM escapes new line like `\n` so there is no need to double newlines. Escaped 
newlines can be parsed correctly by other YAML parsers like pyyaml. BTW, LLVM 
YAML reading also has issue with not removing leading spaces for multiline 
strings so multiline strings serialised by pyyaml with single quotation cannot 
be parsed correctly by clang-apply-replacements. With double quotation it seems 
to work fine in both directions.

What do you think about using double quotation for multiline strings?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D80301: [yaml][clang-tidy] Fix multiline YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 272952.
DmitryPolukhin added a comment.

Use double quotation for multiline strings. It solves problems with internal 
newlines because now they are escaped. Also double quotation solves the problem 
with leading whitespace after newline. In case of single quotation YAML parsers 
should remove leading whitespace according to specification. In case of double 
quotation these leading are internal space and they are preserved. There is no 
way to instruct YAML parsers to preserve leading whitespaces after newline so 
double quotation is the only viable option that solves all problems at once.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301

Files:
  clang/include/clang/Tooling/ReplacementsYaml.h
  clang/unittests/Tooling/ReplacementsYamlTest.cpp
  llvm/include/llvm/Support/YAMLTraits.h
  llvm/lib/Support/YAMLTraits.cpp
  llvm/unittests/Support/YAMLIOTest.cpp

Index: llvm/unittests/Support/YAMLIOTest.cpp
===
--- llvm/unittests/Support/YAMLIOTest.cpp
+++ llvm/unittests/Support/YAMLIOTest.cpp
@@ -285,10 +285,8 @@
 YOut << Original;
   }
   auto Expected = "---\n"
-  "str1:'a multiline string\n"
-  "foobarbaz'\n"
-  "str2:'another one\r"
-  "foobarbaz'\n"
+  "str1:\"a multiline string\\nfoobarbaz\"\n"
+  "str2:\"another one\\rfoobarbaz\"\n"
   "str3:a one-line string\n"
   "...\n";
   ASSERT_EQ(Serialized, Expected);
Index: llvm/lib/Support/YAMLTraits.cpp
===
--- llvm/lib/Support/YAMLTraits.cpp
+++ llvm/lib/Support/YAMLTraits.cpp
@@ -878,12 +878,12 @@
 }
 
 void ScalarTraits::output(const std::string &Val, void *,
- raw_ostream &Out) {
+   raw_ostream &Out) {
   Out << Val;
 }
 
 StringRef ScalarTraits::input(StringRef Scalar, void *,
- std::string &Val) {
+   std::string &Val) {
   Val = Scalar.str();
   return StringRef();
 }
Index: llvm/include/llvm/Support/YAMLTraits.h
===
--- llvm/include/llvm/Support/YAMLTraits.h
+++ llvm/include/llvm/Support/YAMLTraits.h
@@ -649,24 +649,25 @@
 inline QuotingType needsQuotes(StringRef S) {
   if (S.empty())
 return QuotingType::Single;
+
+  QuotingType MaxQuotingNeeded = QuotingType::None;
   if (isSpace(static_cast(S.front())) ||
   isSpace(static_cast(S.back(
-return QuotingType::Single;
+MaxQuotingNeeded = QuotingType::Single;
   if (isNull(S))
-return QuotingType::Single;
+MaxQuotingNeeded = QuotingType::Single;
   if (isBool(S))
-return QuotingType::Single;
+MaxQuotingNeeded = QuotingType::Single;
   if (isNumeric(S))
-return QuotingType::Single;
+MaxQuotingNeeded = QuotingType::Single;
 
   // 7.3.3 Plain Style
   // Plain scalars must not begin with most indicators, as this would cause
   // ambiguity with other YAML constructs.
   static constexpr char Indicators[] = R"(-?:\,[]{}#&*!|>'"%@`)";
   if (S.find_first_of(Indicators) == 0)
-return QuotingType::Single;
+MaxQuotingNeeded = QuotingType::Single;
 
-  QuotingType MaxQuotingNeeded = QuotingType::None;
   for (unsigned char C : S) {
 // Alphanum is safe.
 if (isAlnum(C))
@@ -684,11 +685,11 @@
 case 0x9:
   continue;
 // LF(0xA) and CR(0xD) may delimit values and so require at least single
-// quotes.
+// quotes. LLVM YAML parser cannot handle single quoted multiline so use
+// double quoting to produce valid YAML.
 case 0xA:
 case 0xD:
-  MaxQuotingNeeded = QuotingType::Single;
-  continue;
+  return QuotingType::Double;
 // DEL (0x7F) are excluded from the allowed character range.
 case 0x7F:
   return QuotingType::Double;
Index: clang/unittests/Tooling/ReplacementsYamlTest.cpp
===
--- clang/unittests/Tooling/ReplacementsYamlTest.cpp
+++ clang/unittests/Tooling/ReplacementsYamlTest.cpp
@@ -65,7 +65,7 @@
"  - FilePath:'/path/to/file1.h'\n"
"Offset:  0\n"
"Length:  0\n"
-   "ReplacementText: '#include \n\n'\n"
+   "ReplacementText: \"#include \\n\"\n"
"...\n",
YamlContentStream.str().c_str());
 }
Index: clang/include/clang/Tooling/ReplacementsYaml.h
===
--- clang/include/clang/Tooling/ReplacementsYaml.h
+++ clang/include/clang/Tooling/ReplacementsYaml.h
@@ -35,13 +35,7 @@
 

[PATCH] D80301: [yaml][clang-tidy] Fix multiline YAML serialization

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 272996.
DmitryPolukhin added a comment.

Fix test


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D80301/new/

https://reviews.llvm.org/D80301

Files:
  clang/include/clang/Tooling/ReplacementsYaml.h
  clang/unittests/Tooling/ReplacementsYamlTest.cpp
  llvm/include/llvm/Support/YAMLTraits.h
  llvm/lib/Support/YAMLTraits.cpp
  llvm/test/Transforms/LowerMatrixIntrinsics/remarks-shared-subtrees.ll
  llvm/unittests/Support/YAMLIOTest.cpp

Index: llvm/unittests/Support/YAMLIOTest.cpp
===
--- llvm/unittests/Support/YAMLIOTest.cpp
+++ llvm/unittests/Support/YAMLIOTest.cpp
@@ -285,10 +285,8 @@
 YOut << Original;
   }
   auto Expected = "---\n"
-  "str1:'a multiline string\n"
-  "foobarbaz'\n"
-  "str2:'another one\r"
-  "foobarbaz'\n"
+  "str1:\"a multiline string\\nfoobarbaz\"\n"
+  "str2:\"another one\\rfoobarbaz\"\n"
   "str3:a one-line string\n"
   "...\n";
   ASSERT_EQ(Serialized, Expected);
Index: llvm/test/Transforms/LowerMatrixIntrinsics/remarks-shared-subtrees.ll
===
--- llvm/test/Transforms/LowerMatrixIntrinsics/remarks-shared-subtrees.ll
+++ llvm/test/Transforms/LowerMatrixIntrinsics/remarks-shared-subtrees.ll
@@ -18,8 +18,7 @@
 ; YAML-NEXT:- String:  ' loads, '
 ; YAML-NEXT:- NumComputeOps:   '0'
 ; YAML-NEXT:- String:  ' compute ops'
-; YAML-NEXT:- String:  ',
-; YAML-NEXT:  additionally '
+; YAML-NEXT:- String:  ",\nadditionally "
 ; YAML-NEXT:- NumStores:   '0'
 ; YAML-NEXT:- String:  ' stores, '
 ; YAML-NEXT:- NumLoads:'4'
@@ -47,8 +46,7 @@
 ; YAML-NEXT:- String:  ' loads, '
 ; YAML-NEXT:- NumComputeOps:   '120'
 ; YAML-NEXT:- String:  ' compute ops'
-; YAML-NEXT:- String:  ',
-; YAML-NEXT:  additionally '
+; YAML-NEXT:- String:  ",\nadditionally "
 ; YAML-NEXT:- NumStores:   '0'
 ; YAML-NEXT:- String:  ' stores, '
 ; YAML-NEXT:- NumLoads:'4'
Index: llvm/lib/Support/YAMLTraits.cpp
===
--- llvm/lib/Support/YAMLTraits.cpp
+++ llvm/lib/Support/YAMLTraits.cpp
@@ -878,12 +878,12 @@
 }
 
 void ScalarTraits::output(const std::string &Val, void *,
- raw_ostream &Out) {
+   raw_ostream &Out) {
   Out << Val;
 }
 
 StringRef ScalarTraits::input(StringRef Scalar, void *,
- std::string &Val) {
+   std::string &Val) {
   Val = Scalar.str();
   return StringRef();
 }
Index: llvm/include/llvm/Support/YAMLTraits.h
===
--- llvm/include/llvm/Support/YAMLTraits.h
+++ llvm/include/llvm/Support/YAMLTraits.h
@@ -649,24 +649,25 @@
 inline QuotingType needsQuotes(StringRef S) {
   if (S.empty())
 return QuotingType::Single;
+
+  QuotingType MaxQuotingNeeded = QuotingType::None;
   if (isSpace(static_cast(S.front())) ||
   isSpace(static_cast(S.back(
-return QuotingType::Single;
+MaxQuotingNeeded = QuotingType::Single;
   if (isNull(S))
-return QuotingType::Single;
+MaxQuotingNeeded = QuotingType::Single;
   if (isBool(S))
-return QuotingType::Single;
+MaxQuotingNeeded = QuotingType::Single;
   if (isNumeric(S))
-return QuotingType::Single;
+MaxQuotingNeeded = QuotingType::Single;
 
   // 7.3.3 Plain Style
   // Plain scalars must not begin with most indicators, as this would cause
   // ambiguity with other YAML constructs.
   static constexpr char Indicators[] = R"(-?:\,[]{}#&*!|>'"%@`)";
   if (S.find_first_of(Indicators) == 0)
-return QuotingType::Single;
+MaxQuotingNeeded = QuotingType::Single;
 
-  QuotingType MaxQuotingNeeded = QuotingType::None;
   for (unsigned char C : S) {
 // Alphanum is safe.
 if (isAlnum(C))
@@ -684,11 +685,11 @@
 case 0x9:
   continue;
 // LF(0xA) and CR(0xD) may delimit values and so require at least single
-// quotes.
+// quotes. LLVM YAML parser cannot handle single quoted multiline so use
+// double quoting to produce valid YAML.
 case 0xA:
 case 0xD:
-  MaxQuotingNeeded = QuotingType::Single;
-  continue;
+  return QuotingType::Double;
 // DEL (0x7F) are excluded from the allowed character range.
 case 0x7F:
   return QuotingType::Double;
Index: clang/unittests/Tooling/ReplacementsYamlTest.cpp
===
--- clang/unittests/Tooling/ReplacementsYamlTest.cpp
+++ clang/unittest

[PATCH] D76594: [clang][AST] Support AST files larger than 512M

[Dmitry Polukhin via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG30d5946db95f: [clang][AST] Support AST files larger than 
512M (authored by DmitryPolukhin).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D76594/new/

https://reviews.llvm.org/D76594

Files:
  clang/include/clang/Serialization/ASTBitCodes.h
  clang/include/clang/Serialization/ASTReader.h
  clang/include/clang/Serialization/ASTWriter.h
  clang/include/clang/Serialization/ModuleFile.h
  clang/lib/Serialization/ASTReader.cpp
  clang/lib/Serialization/ASTReaderDecl.cpp
  clang/lib/Serialization/ASTWriter.cpp
  clang/lib/Serialization/ASTWriterDecl.cpp

Index: clang/lib/Serialization/ASTWriterDecl.cpp
===
--- clang/lib/Serialization/ASTWriterDecl.cpp
+++ clang/lib/Serialization/ASTWriterDecl.cpp
@@ -2434,12 +2434,12 @@
   SourceLocation Loc = D->getLocation();
   unsigned Index = ID - FirstDeclID;
   if (DeclOffsets.size() == Index)
-DeclOffsets.push_back(DeclOffset(Loc, Offset));
+DeclOffsets.emplace_back(Loc, Offset);
   else if (DeclOffsets.size() < Index) {
 // FIXME: Can/should this happen?
 DeclOffsets.resize(Index+1);
 DeclOffsets[Index].setLocation(Loc);
-DeclOffsets[Index].BitOffset = Offset;
+DeclOffsets[Index].setBitOffset(Offset);
   } else {
 llvm_unreachable("declarations should be emitted in ID order");
   }
Index: clang/lib/Serialization/ASTWriter.cpp
===
--- clang/lib/Serialization/ASTWriter.cpp
+++ clang/lib/Serialization/ASTWriter.cpp
@@ -1893,6 +1893,7 @@
   // Write out the source location entry table. We skip the first
   // entry, which is always the same dummy entry.
   std::vector SLocEntryOffsets;
+  uint64_t SLocEntryOffsetsBase = Stream.GetCurrentBitNo();
   RecordData PreloadSLocs;
   SLocEntryOffsets.reserve(SourceMgr.local_sloc_entry_size() - 1);
   for (unsigned I = 1, N = SourceMgr.local_sloc_entry_size();
@@ -1903,7 +1904,9 @@
 assert(&SourceMgr.getSLocEntry(FID) == SLoc);
 
 // Record the offset of this source-location entry.
-SLocEntryOffsets.push_back(Stream.GetCurrentBitNo());
+uint64_t Offset = Stream.GetCurrentBitNo() - SLocEntryOffsetsBase;
+assert((Offset >> 32) == 0 && "SLocEntry offset too large");
+SLocEntryOffsets.push_back(Offset);
 
 // Figure out which record code to use.
 unsigned Code;
@@ -2011,12 +2014,14 @@
   Abbrev->Add(BitCodeAbbrevOp(SOURCE_LOCATION_OFFSETS));
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 16)); // # of slocs
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 16)); // total size
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 32)); // base offset
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); // offsets
   unsigned SLocOffsetsAbbrev = Stream.EmitAbbrev(std::move(Abbrev));
   {
 RecordData::value_type Record[] = {
 SOURCE_LOCATION_OFFSETS, SLocEntryOffsets.size(),
-SourceMgr.getNextLocalOffset() - 1 /* skip dummy */};
+SourceMgr.getNextLocalOffset() - 1 /* skip dummy */,
+SLocEntryOffsetsBase};
 Stream.EmitRecordWithBlob(SLocOffsetsAbbrev, Record,
   bytes(SLocEntryOffsets));
   }
@@ -2093,9 +2098,11 @@
 /// Writes the block containing the serialized form of the
 /// preprocessor.
 void ASTWriter::WritePreprocessor(const Preprocessor &PP, bool IsModule) {
+  uint64_t MacroOffsetsBase = Stream.GetCurrentBitNo();
+
   PreprocessingRecord *PPRec = PP.getPreprocessingRecord();
   if (PPRec)
-WritePreprocessorDetail(*PPRec);
+WritePreprocessorDetail(*PPRec, MacroOffsetsBase);
 
   RecordData Record;
   RecordData ModuleMacroRecord;
@@ -2156,7 +2163,8 @@
   // identifier they belong to.
   for (const IdentifierInfo *Name : MacroIdentifiers) {
 MacroDirective *MD = PP.getLocalMacroDirectiveHistory(Name);
-auto StartOffset = Stream.GetCurrentBitNo();
+uint64_t StartOffset = Stream.GetCurrentBitNo() - MacroOffsetsBase;
+assert((StartOffset >> 32) == 0 && "Macro identifiers offset too large");
 
 // Emit the macro directives in reverse source order.
 for (; MD; MD = MD->getPrevious()) {
@@ -2229,14 +2237,12 @@
 
 // Record the local offset of this macro.
 unsigned Index = ID - FirstMacroID;
-if (Index == MacroOffsets.size())
-  MacroOffsets.push_back(Stream.GetCurrentBitNo());
-else {
-  if (Index > MacroOffsets.size())
-MacroOffsets.resize(Index + 1);
+if (Index >= MacroOffsets.size())
+  MacroOffsets.resize(Index + 1);
 
-  MacroOffsets[Index] = Stream.GetCurrentBitNo();
-}
+uint64_t Offset = Stream.GetCurrentBitNo() - MacroOffsetsBase;
+assert((Offset >> 32) == 0 && "Macro offset too large");
+MacroOffsets[Index] = Offset;
 
 AddIdentifierRef(Name, Record);
 AddSourceLocation(MI->getDefinitionLoc(), Record);
@@ -2287,17 +2293,20 @@
 

[PATCH] D76594: [clang][AST] Support AST files larger than 512M

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

In D76594#1986846 , @martong wrote:

> I am not sure, but maybe this patch causes an undefined behavior?
>  
> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/40472/steps/check-clang%20ubsan/logs/stdio
>
>   
> /b/sanitizer-x86_64-linux-fast/build/llvm-project/clang/lib/Serialization/ASTReader.cpp:6327:28:
>  runtime error: load of misaligned address 0x7feac40a55bc for type 'const 
> uint64_t' (aka 'const unsigned long'), which requires 8 byte alignment
>   0x7feac40a55bc: note: pointer points here
> 00 00 00 00 a3 c7 01 00  00 00 00 00 0c c8 01 00  00 00 00 00 29 c8 01 00 
>  00 00 00 00 7a cc 01 00
> ^ 
>   #0 0x3be2fe4 in clang::ASTReader::TypeCursorForIndex(unsigned int) 
> /b/sanitizer-x86_64-linux-fast/build/llvm-project/clang/lib/Serialization/ASTReader.cpp:6327:28
>   #1 0x3be30a0 in clang::ASTReader::readTypeRecord(unsigned int) 
> /b/sanitizer-x86_64-linux-fast/build/llvm-project/clang/lib/Serialization/ASTReader.cpp:6348:24
>   #2 0x3bd3d4a in clang::ASTReader::GetType(unsigned int) 
> /b/sanitizer-x86_64-linux-fast/build/llvm-project/clang/lib/Serialization/ASTReader.cpp:6985:26
>   ...
>
>
> (I am in the blamelist too of that build, so that's why I am sniffing.)


Temporary reverted in a8f85da9f538a400dfea00e4954e403bf5f3269c 



Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D76594/new/

https://reviews.llvm.org/D76594



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D76594: [clang][AST] Support AST files larger than 512M

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 258088.
DmitryPolukhin added a comment.

Split TypeOffsets to low/high parts too


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D76594/new/

https://reviews.llvm.org/D76594

Files:
  clang/include/clang/Serialization/ASTBitCodes.h
  clang/include/clang/Serialization/ASTReader.h
  clang/include/clang/Serialization/ASTWriter.h
  clang/include/clang/Serialization/ModuleFile.h
  clang/lib/Serialization/ASTReader.cpp
  clang/lib/Serialization/ASTReaderDecl.cpp
  clang/lib/Serialization/ASTWriter.cpp
  clang/lib/Serialization/ASTWriterDecl.cpp

Index: clang/lib/Serialization/ASTWriterDecl.cpp
===
--- clang/lib/Serialization/ASTWriterDecl.cpp
+++ clang/lib/Serialization/ASTWriterDecl.cpp
@@ -2434,12 +2434,12 @@
   SourceLocation Loc = D->getLocation();
   unsigned Index = ID - FirstDeclID;
   if (DeclOffsets.size() == Index)
-DeclOffsets.push_back(DeclOffset(Loc, Offset));
+DeclOffsets.emplace_back(Loc, Offset);
   else if (DeclOffsets.size() < Index) {
 // FIXME: Can/should this happen?
 DeclOffsets.resize(Index+1);
 DeclOffsets[Index].setLocation(Loc);
-DeclOffsets[Index].BitOffset = Offset;
+DeclOffsets[Index].setBitOffset(Offset);
   } else {
 llvm_unreachable("declarations should be emitted in ID order");
   }
Index: clang/lib/Serialization/ASTWriter.cpp
===
--- clang/lib/Serialization/ASTWriter.cpp
+++ clang/lib/Serialization/ASTWriter.cpp
@@ -1893,6 +1893,7 @@
   // Write out the source location entry table. We skip the first
   // entry, which is always the same dummy entry.
   std::vector SLocEntryOffsets;
+  uint64_t SLocEntryOffsetsBase = Stream.GetCurrentBitNo();
   RecordData PreloadSLocs;
   SLocEntryOffsets.reserve(SourceMgr.local_sloc_entry_size() - 1);
   for (unsigned I = 1, N = SourceMgr.local_sloc_entry_size();
@@ -1903,7 +1904,9 @@
 assert(&SourceMgr.getSLocEntry(FID) == SLoc);
 
 // Record the offset of this source-location entry.
-SLocEntryOffsets.push_back(Stream.GetCurrentBitNo());
+uint64_t Offset = Stream.GetCurrentBitNo() - SLocEntryOffsetsBase;
+assert((Offset >> 32) == 0 && "SLocEntry offset too large");
+SLocEntryOffsets.push_back(Offset);
 
 // Figure out which record code to use.
 unsigned Code;
@@ -2011,12 +2014,14 @@
   Abbrev->Add(BitCodeAbbrevOp(SOURCE_LOCATION_OFFSETS));
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 16)); // # of slocs
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 16)); // total size
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 32)); // base offset
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); // offsets
   unsigned SLocOffsetsAbbrev = Stream.EmitAbbrev(std::move(Abbrev));
   {
 RecordData::value_type Record[] = {
 SOURCE_LOCATION_OFFSETS, SLocEntryOffsets.size(),
-SourceMgr.getNextLocalOffset() - 1 /* skip dummy */};
+SourceMgr.getNextLocalOffset() - 1 /* skip dummy */,
+SLocEntryOffsetsBase};
 Stream.EmitRecordWithBlob(SLocOffsetsAbbrev, Record,
   bytes(SLocEntryOffsets));
   }
@@ -2093,9 +2098,11 @@
 /// Writes the block containing the serialized form of the
 /// preprocessor.
 void ASTWriter::WritePreprocessor(const Preprocessor &PP, bool IsModule) {
+  uint64_t MacroOffsetsBase = Stream.GetCurrentBitNo();
+
   PreprocessingRecord *PPRec = PP.getPreprocessingRecord();
   if (PPRec)
-WritePreprocessorDetail(*PPRec);
+WritePreprocessorDetail(*PPRec, MacroOffsetsBase);
 
   RecordData Record;
   RecordData ModuleMacroRecord;
@@ -2156,7 +2163,8 @@
   // identifier they belong to.
   for (const IdentifierInfo *Name : MacroIdentifiers) {
 MacroDirective *MD = PP.getLocalMacroDirectiveHistory(Name);
-auto StartOffset = Stream.GetCurrentBitNo();
+uint64_t StartOffset = Stream.GetCurrentBitNo() - MacroOffsetsBase;
+assert((StartOffset >> 32) == 0 && "Macro identifiers offset too large");
 
 // Emit the macro directives in reverse source order.
 for (; MD; MD = MD->getPrevious()) {
@@ -2229,14 +2237,12 @@
 
 // Record the local offset of this macro.
 unsigned Index = ID - FirstMacroID;
-if (Index == MacroOffsets.size())
-  MacroOffsets.push_back(Stream.GetCurrentBitNo());
-else {
-  if (Index > MacroOffsets.size())
-MacroOffsets.resize(Index + 1);
+if (Index >= MacroOffsets.size())
+  MacroOffsets.resize(Index + 1);
 
-  MacroOffsets[Index] = Stream.GetCurrentBitNo();
-}
+uint64_t Offset = Stream.GetCurrentBitNo() - MacroOffsetsBase;
+assert((Offset >> 32) == 0 && "Macro offset too large");
+MacroOffsets[Index] = Offset;
 
 AddIdentifierRef(Name, Record);
 AddSourceLocation(MI->getDefinitionLoc(), Record);
@@ -2287,17 +2293,20 @@
   Abbrev->Add(BitCodeAbbrevOp(MACRO_OFFSET));
   Abbrev->Ad

[PATCH] D76594: [clang][AST] Support AST files larger than 512M

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added inline comments.



Comment at: clang/include/clang/Serialization/ASTBitCodes.h:228
+  /// value in the padding affects AST hash.
+  uint32_t BitOffsetLow = 0;
+  uint32_t BitOffsetHigh = 0;

sammccall wrote:
> consider making this a class and these members private
I kept members public because they public for all other similar struct in the 
file. I think initial idea to keep them public was to have POD struct but it is 
actually not the case due to user defined constructor.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D76594/new/

https://reviews.llvm.org/D76594



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D76594: [clang][AST] Support AST files larger than 512M

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 258192.
DmitryPolukhin marked 3 inline comments as done.
DmitryPolukhin added a comment.

Comments resolved


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D76594/new/

https://reviews.llvm.org/D76594

Files:
  clang/include/clang/Serialization/ASTBitCodes.h
  clang/include/clang/Serialization/ASTReader.h
  clang/include/clang/Serialization/ASTWriter.h
  clang/include/clang/Serialization/ModuleFile.h
  clang/lib/Serialization/ASTReader.cpp
  clang/lib/Serialization/ASTReaderDecl.cpp
  clang/lib/Serialization/ASTWriter.cpp
  clang/lib/Serialization/ASTWriterDecl.cpp

Index: clang/lib/Serialization/ASTWriterDecl.cpp
===
--- clang/lib/Serialization/ASTWriterDecl.cpp
+++ clang/lib/Serialization/ASTWriterDecl.cpp
@@ -2434,12 +2434,12 @@
   SourceLocation Loc = D->getLocation();
   unsigned Index = ID - FirstDeclID;
   if (DeclOffsets.size() == Index)
-DeclOffsets.push_back(DeclOffset(Loc, Offset));
+DeclOffsets.emplace_back(Loc, Offset);
   else if (DeclOffsets.size() < Index) {
 // FIXME: Can/should this happen?
 DeclOffsets.resize(Index+1);
 DeclOffsets[Index].setLocation(Loc);
-DeclOffsets[Index].BitOffset = Offset;
+DeclOffsets[Index].setBitOffset(Offset);
   } else {
 llvm_unreachable("declarations should be emitted in ID order");
   }
Index: clang/lib/Serialization/ASTWriter.cpp
===
--- clang/lib/Serialization/ASTWriter.cpp
+++ clang/lib/Serialization/ASTWriter.cpp
@@ -1893,6 +1893,7 @@
   // Write out the source location entry table. We skip the first
   // entry, which is always the same dummy entry.
   std::vector SLocEntryOffsets;
+  uint64_t SLocEntryOffsetsBase = Stream.GetCurrentBitNo();
   RecordData PreloadSLocs;
   SLocEntryOffsets.reserve(SourceMgr.local_sloc_entry_size() - 1);
   for (unsigned I = 1, N = SourceMgr.local_sloc_entry_size();
@@ -1903,7 +1904,9 @@
 assert(&SourceMgr.getSLocEntry(FID) == SLoc);
 
 // Record the offset of this source-location entry.
-SLocEntryOffsets.push_back(Stream.GetCurrentBitNo());
+uint64_t Offset = Stream.GetCurrentBitNo() - SLocEntryOffsetsBase;
+assert((Offset >> 32) == 0 && "SLocEntry offset too large");
+SLocEntryOffsets.push_back(Offset);
 
 // Figure out which record code to use.
 unsigned Code;
@@ -2011,12 +2014,14 @@
   Abbrev->Add(BitCodeAbbrevOp(SOURCE_LOCATION_OFFSETS));
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 16)); // # of slocs
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 16)); // total size
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 32)); // base offset
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); // offsets
   unsigned SLocOffsetsAbbrev = Stream.EmitAbbrev(std::move(Abbrev));
   {
 RecordData::value_type Record[] = {
 SOURCE_LOCATION_OFFSETS, SLocEntryOffsets.size(),
-SourceMgr.getNextLocalOffset() - 1 /* skip dummy */};
+SourceMgr.getNextLocalOffset() - 1 /* skip dummy */,
+SLocEntryOffsetsBase};
 Stream.EmitRecordWithBlob(SLocOffsetsAbbrev, Record,
   bytes(SLocEntryOffsets));
   }
@@ -2093,9 +2098,11 @@
 /// Writes the block containing the serialized form of the
 /// preprocessor.
 void ASTWriter::WritePreprocessor(const Preprocessor &PP, bool IsModule) {
+  uint64_t MacroOffsetsBase = Stream.GetCurrentBitNo();
+
   PreprocessingRecord *PPRec = PP.getPreprocessingRecord();
   if (PPRec)
-WritePreprocessorDetail(*PPRec);
+WritePreprocessorDetail(*PPRec, MacroOffsetsBase);
 
   RecordData Record;
   RecordData ModuleMacroRecord;
@@ -2156,7 +2163,8 @@
   // identifier they belong to.
   for (const IdentifierInfo *Name : MacroIdentifiers) {
 MacroDirective *MD = PP.getLocalMacroDirectiveHistory(Name);
-auto StartOffset = Stream.GetCurrentBitNo();
+uint64_t StartOffset = Stream.GetCurrentBitNo() - MacroOffsetsBase;
+assert((StartOffset >> 32) == 0 && "Macro identifiers offset too large");
 
 // Emit the macro directives in reverse source order.
 for (; MD; MD = MD->getPrevious()) {
@@ -2229,14 +2237,12 @@
 
 // Record the local offset of this macro.
 unsigned Index = ID - FirstMacroID;
-if (Index == MacroOffsets.size())
-  MacroOffsets.push_back(Stream.GetCurrentBitNo());
-else {
-  if (Index > MacroOffsets.size())
-MacroOffsets.resize(Index + 1);
+if (Index >= MacroOffsets.size())
+  MacroOffsets.resize(Index + 1);
 
-  MacroOffsets[Index] = Stream.GetCurrentBitNo();
-}
+uint64_t Offset = Stream.GetCurrentBitNo() - MacroOffsetsBase;
+assert((Offset >> 32) == 0 && "Macro offset too large");
+MacroOffsets[Index] = Offset;
 
 AddIdentifierRef(Name, Record);
 AddSourceLocation(MI->getDefinitionLoc(), Record);
@@ -2287,17 +2293,20 @@
   Abbrev->Add(BitCodeAbbrevOp(MA

[PATCH] D76594: [clang][AST] Support AST files larger than 512M

[Dmitry Polukhin via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rGa7afb211dc46: [clang][AST] Support AST files larger than 
512M (authored by DmitryPolukhin).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D76594/new/

https://reviews.llvm.org/D76594

Files:
  clang/include/clang/Serialization/ASTBitCodes.h
  clang/include/clang/Serialization/ASTReader.h
  clang/include/clang/Serialization/ASTWriter.h
  clang/include/clang/Serialization/ModuleFile.h
  clang/lib/Serialization/ASTReader.cpp
  clang/lib/Serialization/ASTReaderDecl.cpp
  clang/lib/Serialization/ASTWriter.cpp
  clang/lib/Serialization/ASTWriterDecl.cpp

Index: clang/lib/Serialization/ASTWriterDecl.cpp
===
--- clang/lib/Serialization/ASTWriterDecl.cpp
+++ clang/lib/Serialization/ASTWriterDecl.cpp
@@ -2434,12 +2434,12 @@
   SourceLocation Loc = D->getLocation();
   unsigned Index = ID - FirstDeclID;
   if (DeclOffsets.size() == Index)
-DeclOffsets.push_back(DeclOffset(Loc, Offset));
+DeclOffsets.emplace_back(Loc, Offset);
   else if (DeclOffsets.size() < Index) {
 // FIXME: Can/should this happen?
 DeclOffsets.resize(Index+1);
 DeclOffsets[Index].setLocation(Loc);
-DeclOffsets[Index].BitOffset = Offset;
+DeclOffsets[Index].setBitOffset(Offset);
   } else {
 llvm_unreachable("declarations should be emitted in ID order");
   }
Index: clang/lib/Serialization/ASTWriter.cpp
===
--- clang/lib/Serialization/ASTWriter.cpp
+++ clang/lib/Serialization/ASTWriter.cpp
@@ -1893,6 +1893,7 @@
   // Write out the source location entry table. We skip the first
   // entry, which is always the same dummy entry.
   std::vector SLocEntryOffsets;
+  uint64_t SLocEntryOffsetsBase = Stream.GetCurrentBitNo();
   RecordData PreloadSLocs;
   SLocEntryOffsets.reserve(SourceMgr.local_sloc_entry_size() - 1);
   for (unsigned I = 1, N = SourceMgr.local_sloc_entry_size();
@@ -1903,7 +1904,9 @@
 assert(&SourceMgr.getSLocEntry(FID) == SLoc);
 
 // Record the offset of this source-location entry.
-SLocEntryOffsets.push_back(Stream.GetCurrentBitNo());
+uint64_t Offset = Stream.GetCurrentBitNo() - SLocEntryOffsetsBase;
+assert((Offset >> 32) == 0 && "SLocEntry offset too large");
+SLocEntryOffsets.push_back(Offset);
 
 // Figure out which record code to use.
 unsigned Code;
@@ -2011,12 +2014,14 @@
   Abbrev->Add(BitCodeAbbrevOp(SOURCE_LOCATION_OFFSETS));
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 16)); // # of slocs
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 16)); // total size
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 32)); // base offset
   Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); // offsets
   unsigned SLocOffsetsAbbrev = Stream.EmitAbbrev(std::move(Abbrev));
   {
 RecordData::value_type Record[] = {
 SOURCE_LOCATION_OFFSETS, SLocEntryOffsets.size(),
-SourceMgr.getNextLocalOffset() - 1 /* skip dummy */};
+SourceMgr.getNextLocalOffset() - 1 /* skip dummy */,
+SLocEntryOffsetsBase};
 Stream.EmitRecordWithBlob(SLocOffsetsAbbrev, Record,
   bytes(SLocEntryOffsets));
   }
@@ -2093,9 +2098,11 @@
 /// Writes the block containing the serialized form of the
 /// preprocessor.
 void ASTWriter::WritePreprocessor(const Preprocessor &PP, bool IsModule) {
+  uint64_t MacroOffsetsBase = Stream.GetCurrentBitNo();
+
   PreprocessingRecord *PPRec = PP.getPreprocessingRecord();
   if (PPRec)
-WritePreprocessorDetail(*PPRec);
+WritePreprocessorDetail(*PPRec, MacroOffsetsBase);
 
   RecordData Record;
   RecordData ModuleMacroRecord;
@@ -2156,7 +2163,8 @@
   // identifier they belong to.
   for (const IdentifierInfo *Name : MacroIdentifiers) {
 MacroDirective *MD = PP.getLocalMacroDirectiveHistory(Name);
-auto StartOffset = Stream.GetCurrentBitNo();
+uint64_t StartOffset = Stream.GetCurrentBitNo() - MacroOffsetsBase;
+assert((StartOffset >> 32) == 0 && "Macro identifiers offset too large");
 
 // Emit the macro directives in reverse source order.
 for (; MD; MD = MD->getPrevious()) {
@@ -2229,14 +2237,12 @@
 
 // Record the local offset of this macro.
 unsigned Index = ID - FirstMacroID;
-if (Index == MacroOffsets.size())
-  MacroOffsets.push_back(Stream.GetCurrentBitNo());
-else {
-  if (Index > MacroOffsets.size())
-MacroOffsets.resize(Index + 1);
+if (Index >= MacroOffsets.size())
+  MacroOffsets.resize(Index + 1);
 
-  MacroOffsets[Index] = Stream.GetCurrentBitNo();
-}
+uint64_t Offset = Stream.GetCurrentBitNo() - MacroOffsetsBase;
+assert((Offset >> 32) == 0 && "Macro offset too large");
+MacroOffsets[Index] = Offset;
 
 AddIdentifierRef(Name, Record);
 AddSourceLocation(MI->getDefinitionLoc(), Record);
@@ -2287,17 +2293,20 @@
 

[PATCH] D79285: [clang-tidy] Add diagnostics level to YAML output

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin created this revision.
DmitryPolukhin added reviewers: alexfh, gribozavr2.
DmitryPolukhin added projects: clang-tools-extra, clang.
Herald added a subscriber: xazax.hun.

Also added BuildDirectory for completness and removed unused `Fix`.

Test Plan: check-all


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D79285

Files:
  clang-tools-extra/test/clang-tidy/infrastructure/export-diagnostics.cpp
  clang/include/clang/Tooling/DiagnosticsYaml.h
  clang/unittests/Tooling/DiagnosticsYamlTest.cpp

Index: clang/unittests/Tooling/DiagnosticsYamlTest.cpp
===
--- clang/unittests/Tooling/DiagnosticsYamlTest.cpp
+++ clang/unittests/Tooling/DiagnosticsYamlTest.cpp
@@ -65,6 +65,8 @@
 "  Offset:  100\n"
 "  Length:  12\n"
 "  ReplacementText: 'replacement #1'\n"
+"Level:   Warning\n"
+"BuildDirectory:  'path/to/build/directory'\n"
 "  - DiagnosticName:  'diagnostic#2'\n"
 "DiagnosticMessage:\n"
 "  Message: 'message #2'\n"
@@ -75,6 +77,8 @@
 "  Offset:  62\n"
 "  Length:  2\n"
 "  ReplacementText: 'replacement #2'\n"
+"Level:   Warning\n"
+"BuildDirectory:  'path/to/build/directory'\n"
 "Ranges:\n"
 "  - FilePath:'path/to/source.cpp'\n"
 "FileOffset:  10\n"
@@ -94,6 +98,8 @@
 "FilePath:'path/to/note2.cpp'\n"
 "FileOffset:  99\n"
 "Replacements:[]\n"
+"Level:   Warning\n"
+"BuildDirectory:  'path/to/build/directory'\n"
 "...\n";
 
 TEST(DiagnosticsYamlTest, serializesDiagnostics) {
Index: clang/include/clang/Tooling/DiagnosticsYaml.h
===
--- clang/include/clang/Tooling/DiagnosticsYaml.h
+++ clang/include/clang/Tooling/DiagnosticsYaml.h
@@ -77,7 +77,6 @@
 
 std::string DiagnosticName;
 clang::tooling::DiagnosticMessage Message;
-llvm::StringMap Fix;
 SmallVector Notes;
 clang::tooling::Diagnostic::Level DiagLevel;
 std::string BuildDirectory;
@@ -90,9 +89,9 @@
 Io.mapRequired("DiagnosticName", Keys->DiagnosticName);
 Io.mapRequired("DiagnosticMessage", Keys->Message);
 Io.mapOptional("Notes", Keys->Notes);
+Io.mapOptional("Level", Keys->DiagLevel);
+Io.mapOptional("BuildDirectory", Keys->BuildDirectory);
 Io.mapOptional("Ranges", Keys->Ranges);
-
-// FIXME: Export properly all the different fields.
   }
 };
 
@@ -104,6 +103,14 @@
 Io.mapRequired("Diagnostics", Doc.Diagnostics);
   }
 };
+
+template <> struct ScalarEnumerationTraits {
+  static void enumeration(IO &IO, clang::tooling::Diagnostic::Level &Value) {
+IO.enumCase(Value, "Warning", clang::tooling::Diagnostic::Warning);
+IO.enumCase(Value, "Error", clang::tooling::Diagnostic::Error);
+  }
+};
+
 } // end namespace yaml
 } // end namespace llvm
 
Index: clang-tools-extra/test/clang-tidy/infrastructure/export-diagnostics.cpp
===
--- clang-tools-extra/test/clang-tidy/infrastructure/export-diagnostics.cpp
+++ clang-tools-extra/test/clang-tidy/infrastructure/export-diagnostics.cpp
@@ -1,15 +1,17 @@
 // RUN: grep -Ev "// *[A-Z-]+:" %s > %t-input.cpp
-// RUN: clang-tidy %t-input.cpp -checks='-*,google-explicit-constructor,clang-diagnostic-missing-prototypes' -export-fixes=%t.yaml -- -Wmissing-prototypes > %t.msg 2>&1
+// RUN: not clang-tidy %t-input.cpp -checks='-*,google-explicit-constructor,clang-diagnostic-missing-prototypes' -export-fixes=%t.yaml -- -Wmissing-prototypes > %t.msg 2>&1
 // RUN: FileCheck -input-file=%t.msg -check-prefix=CHECK-MESSAGES %s -implicit-check-not='{{warning|error|note}}:'
 // RUN: FileCheck -input-file=%t.yaml -check-prefix=CHECK-YAML %s
 #define X(n) void n ## n() {}
 X(f)
+int a[-1];
 
 // CHECK-MESSAGES: -input.cpp:2:1: warning: no previous prototype for function 'ff' [clang-diagnostic-missing-prototypes]
 // CHECK-MESSAGES: -input.cpp:1:19: note: expanded from macro 'X'
 // CHECK-MESSAGES: {{^}}note: expanded from here{{$}}
 // CHECK-MESSAGES: -input.cpp:2:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
 // CHECK-MESSAGES: -input.cpp:1:14: note: expanded from macro 'X'
+// CHECK-MESSAGES: -input.cpp:3:7: error: 'a' declared as an array with a negative size [clang-diagnostic-error]
 
 // CHECK-YAML: ---
 // CHECK-YAML-NEXT: MainSourceFile:  '{{.*}}-input.cpp'
@@ -42,4 +44,18 @@
 // CHECK-YAML-NEXT: FilePath:'{{.*}}-input.cpp'
 // CHECK-YAML-NEXT: FileOffset:  13
 // CHECK-YAML-NEXT: Replacements:[]
+// CHECK-YAML-NEXT: Level:   Warning
+// CHECK-YAML-NEXT: BuildDirectory:  '{{.*}}'
+// CHECK-YAML-NEXT:   - DiagnosticName:  clang-diagnostic-error
+// CHE

[PATCH] D79285: [clang-tidy] Add diagnostics level to YAML output

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

@alexfh and @gribozavr2 friendly ping could you please take a look


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D79285/new/

https://reviews.llvm.org/D79285



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D133948: [clang][C++20] Fix clang/clangd assert/crash after compilation errors

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin created this revision.
DmitryPolukhin added reviewers: Tyker, rsmith, aaron.ballman.
DmitryPolukhin added a project: clang.
Herald added subscribers: kadircet, arphaman.
Herald added a project: All.
DmitryPolukhin requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.

After compilation errors, expression a transformation result may not be usable.
It triggers an assert in RemoveNestedImmediateInvocation and SIGSEGV in case of
builds without asserts. This issue significantly affects clangd because source
may not be valid during typing. Tests cases that I attached was reduce from huge
C++ translation unit.

Test Plan: check-clang


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D133948

Files:
  clang/lib/Sema/SemaExpr.cpp
  clang/test/SemaCXX/remove-nested-immediate-invocation-crash.cpp


Index: clang/test/SemaCXX/remove-nested-immediate-invocation-crash.cpp
===
--- /dev/null
+++ clang/test/SemaCXX/remove-nested-immediate-invocation-crash.cpp
@@ -0,0 +1,11 @@
+// RUN: not %clang_cc1 -fsyntax-only -verify -std=gnu++20 -ferror-limit 19 %s
+// Creduced test case for the crash in RemoveNestedImmediateInvocation after 
compliation errros.
+
+a, class b {   template < typename c>  
   consteval b(c
+} template  using d = b;
+auto e(d<>) -> int:;
+}
+f
+}
+g() {
+auto h = "":(::i(e(h))
Index: clang/lib/Sema/SemaExpr.cpp
===
--- clang/lib/Sema/SemaExpr.cpp
+++ clang/lib/Sema/SemaExpr.cpp
@@ -17624,9 +17624,13 @@
 Transformer.AllowSkippingFirstCXXConstructExpr = false;
 
   ExprResult Res = Transformer.TransformExpr(It->getPointer()->getSubExpr());
-  assert(Res.isUsable());
-  Res = SemaRef.MaybeCreateExprWithCleanups(Res);
-  It->getPointer()->setSubExpr(Res.get());
+  // The result may not be usable in case of previous compilation errors.
+  // In this case evaluation of the expression may result in crash so just
+  // don't do anything further with the result.
+  if(Res.isUsable()) {
+Res = SemaRef.MaybeCreateExprWithCleanups(Res);
+It->getPointer()->setSubExpr(Res.get());
+  }
 }
 
 static void


Index: clang/test/SemaCXX/remove-nested-immediate-invocation-crash.cpp
===
--- /dev/null
+++ clang/test/SemaCXX/remove-nested-immediate-invocation-crash.cpp
@@ -0,0 +1,11 @@
+// RUN: not %clang_cc1 -fsyntax-only -verify -std=gnu++20 -ferror-limit 19 %s
+// Creduced test case for the crash in RemoveNestedImmediateInvocation after compliation errros.
+
+a, class b {   template < typename c> consteval b(c
+} template  using d = b;
+auto e(d<>) -> int:;
+}
+f
+}
+g() {
+auto h = "":(::i(e(h))
Index: clang/lib/Sema/SemaExpr.cpp
===
--- clang/lib/Sema/SemaExpr.cpp
+++ clang/lib/Sema/SemaExpr.cpp
@@ -17624,9 +17624,13 @@
 Transformer.AllowSkippingFirstCXXConstructExpr = false;
 
   ExprResult Res = Transformer.TransformExpr(It->getPointer()->getSubExpr());
-  assert(Res.isUsable());
-  Res = SemaRef.MaybeCreateExprWithCleanups(Res);
-  It->getPointer()->setSubExpr(Res.get());
+  // The result may not be usable in case of previous compilation errors.
+  // In this case evaluation of the expression may result in crash so just
+  // don't do anything further with the result.
+  if(Res.isUsable()) {
+Res = SemaRef.MaybeCreateExprWithCleanups(Res);
+It->getPointer()->setSubExpr(Res.get());
+  }
 }
 
 static void
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D133948: [clang][C++20] Fix clang/clangd assert/crash after compilation errors

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 460437.
DmitryPolukhin added a comment.

clang-format


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D133948/new/

https://reviews.llvm.org/D133948

Files:
  clang/lib/Sema/SemaExpr.cpp
  clang/test/SemaCXX/remove-nested-immediate-invocation-crash.cpp


Index: clang/test/SemaCXX/remove-nested-immediate-invocation-crash.cpp
===
--- /dev/null
+++ clang/test/SemaCXX/remove-nested-immediate-invocation-crash.cpp
@@ -0,0 +1,11 @@
+// RUN: not %clang_cc1 -fsyntax-only -verify -std=gnu++20 -ferror-limit 19 %s
+// Creduced test case for the crash in RemoveNestedImmediateInvocation after 
compliation errros.
+
+a, class b {   template < typename c>  
   consteval b(c
+} template  using d = b;
+auto e(d<>) -> int:;
+}
+f
+}
+g() {
+auto h = "":(::i(e(h))
Index: clang/lib/Sema/SemaExpr.cpp
===
--- clang/lib/Sema/SemaExpr.cpp
+++ clang/lib/Sema/SemaExpr.cpp
@@ -17624,9 +17624,13 @@
 Transformer.AllowSkippingFirstCXXConstructExpr = false;
 
   ExprResult Res = Transformer.TransformExpr(It->getPointer()->getSubExpr());
-  assert(Res.isUsable());
-  Res = SemaRef.MaybeCreateExprWithCleanups(Res);
-  It->getPointer()->setSubExpr(Res.get());
+  // The result may not be usable in case of previous compilation errors.
+  // In this case evaluation of the expression may result in crash so just
+  // don't do anything further with the result.
+  if (Res.isUsable()) {
+Res = SemaRef.MaybeCreateExprWithCleanups(Res);
+It->getPointer()->setSubExpr(Res.get());
+  }
 }
 
 static void


Index: clang/test/SemaCXX/remove-nested-immediate-invocation-crash.cpp
===
--- /dev/null
+++ clang/test/SemaCXX/remove-nested-immediate-invocation-crash.cpp
@@ -0,0 +1,11 @@
+// RUN: not %clang_cc1 -fsyntax-only -verify -std=gnu++20 -ferror-limit 19 %s
+// Creduced test case for the crash in RemoveNestedImmediateInvocation after compliation errros.
+
+a, class b {   template < typename c> consteval b(c
+} template  using d = b;
+auto e(d<>) -> int:;
+}
+f
+}
+g() {
+auto h = "":(::i(e(h))
Index: clang/lib/Sema/SemaExpr.cpp
===
--- clang/lib/Sema/SemaExpr.cpp
+++ clang/lib/Sema/SemaExpr.cpp
@@ -17624,9 +17624,13 @@
 Transformer.AllowSkippingFirstCXXConstructExpr = false;
 
   ExprResult Res = Transformer.TransformExpr(It->getPointer()->getSubExpr());
-  assert(Res.isUsable());
-  Res = SemaRef.MaybeCreateExprWithCleanups(Res);
-  It->getPointer()->setSubExpr(Res.get());
+  // The result may not be usable in case of previous compilation errors.
+  // In this case evaluation of the expression may result in crash so just
+  // don't do anything further with the result.
+  if (Res.isUsable()) {
+Res = SemaRef.MaybeCreateExprWithCleanups(Res);
+It->getPointer()->setSubExpr(Res.get());
+  }
 }
 
 static void
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D118755: [clangd] Crash in __memcmp_avx2_movbe

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

@ivanmurashko

Overall looks good to me but please re-upload patch on top of some stable 
revision where failed test passes (it seems unrelated to your changes).

`struct Include` is a light weight structure so having a copy seems to be the 
simplest solution.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D118755/new/

https://reviews.llvm.org/D118755

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89936: [clang-tidy] adding "--config-file=" to specify custom config file.

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

Sorry, I don't think it's worth reverting and resubmitting the same changes in 
such a case. Commit attribution is correct and there is link to this code 
review.
After all, most clang-tidy users will never read commit messages and will only 
use `--help` for discovering this feature.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89936/new/

https://reviews.llvm.org/D89936

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D90835: [RFC][clang-tidy] Ignore diagnostics due to macro expansion from not-interested headers

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin created this revision.
DmitryPolukhin added reviewers: alexfh, njames93, thakis.
DmitryPolukhin added projects: clang, clang-tools-extra.
Herald added subscribers: kbarton, xazax.hun, nemanjai.
DmitryPolukhin requested review of this revision.

This diff is an attempt to workaround a common problem with clang-tidy 
deployment.
There are headers that might not be compatible with your project style guide 
but you
have to use macro from that headers so clang-tidy could report lots of style 
issues
that you cannot fix or is some cases even annotate with NOLINT. Solution 
proposed
in this diff is to avoid reporting diagnostics in macro expansion of the macro 
defined
in headers that doesn't match the desired `--header-filter`.

Just one real life example of this issue from a very popular gflags library.
This library requires using macro in use code like this:

  DEFINE_bool(some_bool_flag,
      "the default value should go here, not the description",
      false);

But the macro has outdated version of compiler assert that uses C-style arrays 
.
Therefore use code becomes incompatible with clang-tidy check 
`modernize-avoid-c-arrays`.
Another example of problematic is googletest/googlemock with lots of macro that 
you cannot avoid.

This diff implements new behavior by default but it might be also possible to
enable it behind some configuration flag.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D90835

Files:
  clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.cpp
  clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.h
  clang-tools-extra/clang-tidy/cppcoreguidelines/ProTypeVarargCheck.cpp
  clang-tools-extra/test/clang-tidy/infrastructure/macros.cpp
  clang-tools-extra/test/clang-tidy/infrastructure/macros.h
  clang-tools-extra/test/clang-tidy/infrastructure/macros_filter.h

Index: clang-tools-extra/test/clang-tidy/infrastructure/macros_filter.h
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/macros_filter.h
@@ -0,0 +1,3 @@
+#define HEADER_FILTER_MACRO_DIAG_IN_ARG(a) a
+#define HEADER_FILTER_MACRO_DIAG_IN_BODY int var_B1[10]
+#define HEADER_FILTER_MACRO_DIAG_IN_BODY2 int var_B2[10]
Index: clang-tools-extra/test/clang-tidy/infrastructure/macros.h
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/macros.h
@@ -0,0 +1,9 @@
+#define HEADER_MACRO_DIAG_IN_ARG(a) a
+#define HEADER_MACRO_DIAG_IN_BODY int var_C1[10]
+#define HEADER_MACRO_DIAG_IN_BODY2 int var_C2[10]
+
+#define DEFINE_bool(name, val)\
+  namespace fLB { \
+  typedef char FLAG_##name##_value_is_not_a_bool[(sizeof(val) != sizeof(bool)) ? -1 : 1]; \
+  }   \
+  bool FLAG_##name = val
Index: clang-tools-extra/test/clang-tidy/infrastructure/macros.cpp
===
--- clang-tools-extra/test/clang-tidy/infrastructure/macros.cpp
+++ clang-tools-extra/test/clang-tidy/infrastructure/macros.cpp
@@ -1,7 +1,41 @@
-// RUN: clang-tidy -checks='-*,google-explicit-constructor' %s -- | FileCheck %s
+// RUN: clang-tidy -checks='-*,google-explicit-constructor,modernize-avoid-c-arrays' --header-filter='macros_filter.h' %s -- | FileCheck %s
+
+#include "macros.h"
+#include "macros_filter.h"
 
 #define Q(name) class name { name(int i); }
 
 Q(A);
 // CHECK: :[[@LINE-1]]:3: warning: single-argument constructors must be marked explicit
-// CHECK: :3:30: note: expanded from macro 'Q'
+// CHECK: :[[@LINE-4]]:30: note: expanded from macro 'Q'
+
+#define MAIN_MACRO_DIAG_IN_ARG(a) a
+MAIN_MACRO_DIAG_IN_ARG(int var_A[10]);
+// CHECK: :[[@LINE-1]]:24: warning: do not declare C-style arrays, use std::array<> instead
+
+#define MAIN_MACRO_DIAG_IN_BODY int var_A1[10]
+MAIN_MACRO_DIAG_IN_BODY;
+// CHECK: :[[@LINE-1]]:1: warning: do not declare C-style arrays, use std::array<> instead
+// CHECK: :[[@LINE-3]]:33: note: expanded from macro 'MAIN_MACRO_DIAG_IN_BODY'
+
+HEADER_FILTER_MACRO_DIAG_IN_ARG(int var_B[10]);
+// CHECK: :[[@LINE-1]]:33: warning: do not declare C-style arrays, use std::array<> instead
+
+HEADER_FILTER_MACRO_DIAG_IN_BODY;
+// CHECK: :[[@LINE-1]]:1: warning: do not declare C-style arrays, use std::array<> instead
+// CHECK: note: expanded from macro 'HEADER_FILTER_MACRO_DIAG_IN_BODY'
+
+#define MAIN_MACRO_WRAPPER HEADER_FILTER_MACRO_DIAG_IN_BODY2
+MAIN_MACRO_WRAPPER;
+// CHECK: :[[@LINE-1]]:1: warning: do not declare C-style arrays, use std::array<> instead
+// CHECK: note: expanded from macro 'MAIN_MACRO_WRAPPER'
+// CHECK: note: expanded from macro 'HEADER_FILTER_MACRO_DIA

[PATCH] D90835: [RFC][clang-tidy] Ignore diagnostics due to macro expansion from not-interested headers

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 303121.
DmitryPolukhin added a comment.

Fix clang-tidy warning


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D90835/new/

https://reviews.llvm.org/D90835

Files:
  clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.cpp
  clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.h
  clang-tools-extra/clang-tidy/cppcoreguidelines/ProTypeVarargCheck.cpp
  clang-tools-extra/test/clang-tidy/infrastructure/macros.cpp
  clang-tools-extra/test/clang-tidy/infrastructure/macros.h
  clang-tools-extra/test/clang-tidy/infrastructure/macros_filter.h

Index: clang-tools-extra/test/clang-tidy/infrastructure/macros_filter.h
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/macros_filter.h
@@ -0,0 +1,3 @@
+#define HEADER_FILTER_MACRO_DIAG_IN_ARG(a) a
+#define HEADER_FILTER_MACRO_DIAG_IN_BODY int var_B1[10]
+#define HEADER_FILTER_MACRO_DIAG_IN_BODY2 int var_B2[10]
Index: clang-tools-extra/test/clang-tidy/infrastructure/macros.h
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/macros.h
@@ -0,0 +1,9 @@
+#define HEADER_MACRO_DIAG_IN_ARG(a) a
+#define HEADER_MACRO_DIAG_IN_BODY int var_C1[10]
+#define HEADER_MACRO_DIAG_IN_BODY2 int var_C2[10]
+
+#define DEFINE_bool(name, val)\
+  namespace fLB { \
+  typedef char FLAG_##name##_value_is_not_a_bool[(sizeof(val) != sizeof(bool)) ? -1 : 1]; \
+  }   \
+  bool FLAG_##name = val
Index: clang-tools-extra/test/clang-tidy/infrastructure/macros.cpp
===
--- clang-tools-extra/test/clang-tidy/infrastructure/macros.cpp
+++ clang-tools-extra/test/clang-tidy/infrastructure/macros.cpp
@@ -1,7 +1,41 @@
-// RUN: clang-tidy -checks='-*,google-explicit-constructor' %s -- | FileCheck %s
+// RUN: clang-tidy -checks='-*,google-explicit-constructor,modernize-avoid-c-arrays' --header-filter='macros_filter.h' %s -- | FileCheck %s
+
+#include "macros.h"
+#include "macros_filter.h"
 
 #define Q(name) class name { name(int i); }
 
 Q(A);
 // CHECK: :[[@LINE-1]]:3: warning: single-argument constructors must be marked explicit
-// CHECK: :3:30: note: expanded from macro 'Q'
+// CHECK: :[[@LINE-4]]:30: note: expanded from macro 'Q'
+
+#define MAIN_MACRO_DIAG_IN_ARG(a) a
+MAIN_MACRO_DIAG_IN_ARG(int var_A[10]);
+// CHECK: :[[@LINE-1]]:24: warning: do not declare C-style arrays, use std::array<> instead
+
+#define MAIN_MACRO_DIAG_IN_BODY int var_A1[10]
+MAIN_MACRO_DIAG_IN_BODY;
+// CHECK: :[[@LINE-1]]:1: warning: do not declare C-style arrays, use std::array<> instead
+// CHECK: :[[@LINE-3]]:33: note: expanded from macro 'MAIN_MACRO_DIAG_IN_BODY'
+
+HEADER_FILTER_MACRO_DIAG_IN_ARG(int var_B[10]);
+// CHECK: :[[@LINE-1]]:33: warning: do not declare C-style arrays, use std::array<> instead
+
+HEADER_FILTER_MACRO_DIAG_IN_BODY;
+// CHECK: :[[@LINE-1]]:1: warning: do not declare C-style arrays, use std::array<> instead
+// CHECK: note: expanded from macro 'HEADER_FILTER_MACRO_DIAG_IN_BODY'
+
+#define MAIN_MACRO_WRAPPER HEADER_FILTER_MACRO_DIAG_IN_BODY2
+MAIN_MACRO_WRAPPER;
+// CHECK: :[[@LINE-1]]:1: warning: do not declare C-style arrays, use std::array<> instead
+// CHECK: note: expanded from macro 'MAIN_MACRO_WRAPPER'
+// CHECK: note: expanded from macro 'HEADER_FILTER_MACRO_DIAG_IN_BODY2'
+
+HEADER_MACRO_DIAG_IN_ARG(int var_C[10]);
+HEADER_MACRO_DIAG_IN_BODY;
+
+#define MAIN_MACRO_WRAPPER2 HEADER_MACRO_DIAG_IN_BODY2
+MAIN_MACRO_WRAPPER2;
+
+DEFINE_bool(test, false);
+// CHECK-NOT: warning:
Index: clang-tools-extra/clang-tidy/cppcoreguidelines/ProTypeVarargCheck.cpp
===
--- clang-tools-extra/clang-tidy/cppcoreguidelines/ProTypeVarargCheck.cpp
+++ clang-tools-extra/clang-tidy/cppcoreguidelines/ProTypeVarargCheck.cpp
@@ -115,7 +115,10 @@
   }
 
   if (const auto *Matched = Result.Nodes.getNodeAs("va_use")) {
-diag(Matched->getExprLoc(),
+// va_arg is a macro defined in system header but diags from such macro
+// are not always reported so use expansion location instead to always
+// report
+diag(Result.SourceManager->getExpansionLoc(Matched->getExprLoc()),
  "do not use va_arg to define c-style vararg functions; "
  "use variadic templates instead");
   }
Index: clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.h
===
--- clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.h
+++ clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.h
@@ -177,6 +177,10 @@
 DiagEngine->getDiagnosticIDs()->getDesc

[PATCH] D90835: [RFC][clang-tidy] Ignore diagnostics due to macro expansion from not-interested headers

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 306046.
DmitryPolukhin marked 2 inline comments as done.
DmitryPolukhin added a comment.

- addressed comments in the diff
- stop ignoring system macro


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D90835/new/

https://reviews.llvm.org/D90835

Files:
  clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.cpp
  clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.h
  clang-tools-extra/test/clang-tidy/infrastructure/macros.cpp
  clang-tools-extra/test/clang-tidy/infrastructure/macros.h
  clang-tools-extra/test/clang-tidy/infrastructure/macros_filter.h

Index: clang-tools-extra/test/clang-tidy/infrastructure/macros_filter.h
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/macros_filter.h
@@ -0,0 +1,3 @@
+#define HEADER_FILTER_MACRO_DIAG_IN_ARG(a) a
+#define HEADER_FILTER_MACRO_DIAG_IN_BODY int var_B1[10]
+#define HEADER_FILTER_MACRO_DIAG_IN_BODY2 int var_B2[10]
Index: clang-tools-extra/test/clang-tidy/infrastructure/macros.h
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/macros.h
@@ -0,0 +1,9 @@
+#define HEADER_MACRO_DIAG_IN_ARG(a) a
+#define HEADER_MACRO_DIAG_IN_BODY int var_C1[10]
+#define HEADER_MACRO_DIAG_IN_BODY2 int var_C2[10]
+
+#define DEFINE_bool(name, val)\
+  namespace fLB { \
+  typedef char FLAG_##name##_value_is_not_a_bool[(sizeof(val) != sizeof(bool)) ? -1 : 1]; \
+  }   \
+  bool FLAG_##name = val
Index: clang-tools-extra/test/clang-tidy/infrastructure/macros.cpp
===
--- clang-tools-extra/test/clang-tidy/infrastructure/macros.cpp
+++ clang-tools-extra/test/clang-tidy/infrastructure/macros.cpp
@@ -1,7 +1,41 @@
-// RUN: clang-tidy -checks='-*,google-explicit-constructor' %s -- | FileCheck %s
+// RUN: clang-tidy -checks='-*,google-explicit-constructor,modernize-avoid-c-arrays' --header-filter='macros_filter.h' %s -- | FileCheck %s
+
+#include "macros.h"
+#include "macros_filter.h"
 
 #define Q(name) class name { name(int i); }
 
 Q(A);
 // CHECK: :[[@LINE-1]]:3: warning: single-argument constructors must be marked explicit
-// CHECK: :3:30: note: expanded from macro 'Q'
+// CHECK: :[[@LINE-4]]:30: note: expanded from macro 'Q'
+
+#define MAIN_MACRO_DIAG_IN_ARG(a) a
+MAIN_MACRO_DIAG_IN_ARG(int var_A[10]);
+// CHECK: :[[@LINE-1]]:24: warning: do not declare C-style arrays, use std::array<> instead
+
+#define MAIN_MACRO_DIAG_IN_BODY int var_A1[10]
+MAIN_MACRO_DIAG_IN_BODY;
+// CHECK: :[[@LINE-1]]:1: warning: do not declare C-style arrays, use std::array<> instead
+// CHECK: :[[@LINE-3]]:33: note: expanded from macro 'MAIN_MACRO_DIAG_IN_BODY'
+
+HEADER_FILTER_MACRO_DIAG_IN_ARG(int var_B[10]);
+// CHECK: :[[@LINE-1]]:33: warning: do not declare C-style arrays, use std::array<> instead
+
+HEADER_FILTER_MACRO_DIAG_IN_BODY;
+// CHECK: :[[@LINE-1]]:1: warning: do not declare C-style arrays, use std::array<> instead
+// CHECK: note: expanded from macro 'HEADER_FILTER_MACRO_DIAG_IN_BODY'
+
+#define MAIN_MACRO_WRAPPER HEADER_FILTER_MACRO_DIAG_IN_BODY2
+MAIN_MACRO_WRAPPER;
+// CHECK: :[[@LINE-1]]:1: warning: do not declare C-style arrays, use std::array<> instead
+// CHECK: note: expanded from macro 'MAIN_MACRO_WRAPPER'
+// CHECK: note: expanded from macro 'HEADER_FILTER_MACRO_DIAG_IN_BODY2'
+
+HEADER_MACRO_DIAG_IN_ARG(int var_C[10]);
+HEADER_MACRO_DIAG_IN_BODY;
+
+#define MAIN_MACRO_WRAPPER2 HEADER_MACRO_DIAG_IN_BODY2
+MAIN_MACRO_WRAPPER2;
+
+DEFINE_bool(test, false);
+// CHECK-NOT: warning:
Index: clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.h
===
--- clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.h
+++ clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.h
@@ -177,6 +177,10 @@
 DiagEngine->getDiagnosticIDs()->getDescription(DiagnosticID)));
   }
 
+  /// Returns the \c HeaderFilter constructed for the options set in the
+  /// context.
+  llvm::Regex *getHeaderFilter() const;
+
 private:
   // Writes to Stats.
   friend class ClangTidyDiagnosticConsumer;
@@ -189,6 +193,8 @@
   class CachedGlobList;
   std::unique_ptr CheckFilter;
   std::unique_ptr WarningAsErrorFilter;
+  // Cache for compiled regex with HeaderFilter.
+  mutable std::unique_ptr HeaderFilter;
 
   LangOptions LangOpts;
 
@@ -243,10 +249,6 @@
   void removeIncompatibleErrors();
   void removeDuplicatedDiagnosticsOfAliasCheckers();
 
-  /// Returns the \c HeaderFilter constructed for the options set in the
-  /// context.
-  llvm::Regex *getHeaderFilter();
-
   /// Updates \c LastErrorRelatesToUse

[PATCH] D90835: [RFC][clang-tidy] Ignore diagnostics due to macro expansion from not-interested headers

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

@njames93 thank you for the feedback!

Still waiting for more feedback and especially high level one about this 
feature in general. Does it look useful? Should it be behind a flag i.e. 
changing default behaviour is too dangerous?




Comment at: clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.cpp:249-251
+  if (!HeaderFilter)
+HeaderFilter =
+std::make_unique(*getOptions().HeaderFilterRegex);

njames93 wrote:
> This should also check if the Optional has a value
This code was moved from `ClangTidyDiagnosticConsumer::getHeaderFilter` but you 
are right, it makes sense to add support for missing value.



Comment at: clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.cpp:349-351
+  // Skip macro from system headers.
+  if (!*Context.getOptions().SystemHeaders && SM.isInSystemHeader(SpellingLoc))
+return true;

njames93 wrote:
> This looks suspicious, in clang-tidy land `SystemHeaders` is always set, 
> however outside clang-tidy it may not be.
> Perhaps `getValueOr(false)` should be used to prevent any asserts.
> 
> Also can a test be added to show this respecting the SystemHeaders setting.
Running this code on bigger codebase I found undesired side-effect that it is 
better to report issues on system macro like NULL such issues usually from 
external code, not from macro itself.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D90835/new/

https://reviews.llvm.org/D90835

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89886: [clang-tidy] Fix redefinition of module in the same module.modulemap file

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin created this revision.
DmitryPolukhin added reviewers: alexfh, gribozavr, klimek.
DmitryPolukhin added a project: clang-tools-extra.
Herald added subscribers: kbarton, xazax.hun, nemanjai.
Herald added a project: clang.
DmitryPolukhin requested review of this revision.

In memory VFS cannot handle aceesssing the same file with different paths.
This diff just stops using VFS for modulemap files.

Fixes PR47839


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D89886

Files:
  clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp


Index: clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
===
--- clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
+++ clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
@@ -13,13 +13,20 @@
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Serialization/ASTReader.h"
 
+#define DEBUG_TYPE "clang-tidy"
+
 namespace clang {
 namespace tooling {
 
 class ExpandModularHeadersPPCallbacks::FileRecorder {
 public:
   /// Records that a given file entry is needed for replaying callbacks.
-  void addNecessaryFile(const FileEntry *File) { FilesToRecord.insert(File); }
+  void addNecessaryFile(const FileEntry *File) {
+// Don't record module.modulemap files because it breaks same file
+// detection.
+if (!File->getName().endswith("module.modulemap"))
+  FilesToRecord.insert(File);
+  }
 
   /// Records content for a file and adds it to the FileSystem.
   void recordFileContent(const FileEntry *File,
@@ -44,8 +51,8 @@
   /// `FilesToRecord` should be empty.
   void checkAllFilesRecorded() {
 for (auto FileEntry : FilesToRecord)
-  llvm::errs() << "Did not record contents for input file: "
-   << FileEntry->getName() << "\n";
+  LLVM_DEBUG(llvm::dbgs() << "Did not record contents for input file: "
+  << FileEntry->getName() << "\n");
   }
 
 private:


Index: clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
===
--- clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
+++ clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
@@ -13,13 +13,20 @@
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Serialization/ASTReader.h"
 
+#define DEBUG_TYPE "clang-tidy"
+
 namespace clang {
 namespace tooling {
 
 class ExpandModularHeadersPPCallbacks::FileRecorder {
 public:
   /// Records that a given file entry is needed for replaying callbacks.
-  void addNecessaryFile(const FileEntry *File) { FilesToRecord.insert(File); }
+  void addNecessaryFile(const FileEntry *File) {
+// Don't record module.modulemap files because it breaks same file
+// detection.
+if (!File->getName().endswith("module.modulemap"))
+  FilesToRecord.insert(File);
+  }
 
   /// Records content for a file and adds it to the FileSystem.
   void recordFileContent(const FileEntry *File,
@@ -44,8 +51,8 @@
   /// `FilesToRecord` should be empty.
   void checkAllFilesRecorded() {
 for (auto FileEntry : FilesToRecord)
-  llvm::errs() << "Did not record contents for input file: "
-   << FileEntry->getName() << "\n";
+  LLVM_DEBUG(llvm::dbgs() << "Did not record contents for input file: "
+  << FileEntry->getName() << "\n");
   }
 
 private:
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89886: [clang-tidy] Fix redefinition of module in the same module.modulemap file

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 299697.
DmitryPolukhin added a comment.

Linting diff


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89886/new/

https://reviews.llvm.org/D89886

Files:
  clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp


Index: clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
===
--- clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
+++ clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
@@ -13,13 +13,20 @@
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Serialization/ASTReader.h"
 
+#define DEBUG_TYPE "clang-tidy"
+
 namespace clang {
 namespace tooling {
 
 class ExpandModularHeadersPPCallbacks::FileRecorder {
 public:
   /// Records that a given file entry is needed for replaying callbacks.
-  void addNecessaryFile(const FileEntry *File) { FilesToRecord.insert(File); }
+  void addNecessaryFile(const FileEntry *File) {
+// Don't record module.modulemap files because it breaks same file
+// detection.
+if (!File->getName().endswith("module.modulemap"))
+  FilesToRecord.insert(File);
+  }
 
   /// Records content for a file and adds it to the FileSystem.
   void recordFileContent(const FileEntry *File,
@@ -44,8 +51,8 @@
   /// `FilesToRecord` should be empty.
   void checkAllFilesRecorded() {
 for (auto FileEntry : FilesToRecord)
-  llvm::errs() << "Did not record contents for input file: "
-   << FileEntry->getName() << "\n";
+  LLVM_DEBUG(llvm::dbgs() << "Did not record contents for input file: "
+  << FileEntry->getName() << "\n");
   }
 
 private:


Index: clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
===
--- clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
+++ clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
@@ -13,13 +13,20 @@
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Serialization/ASTReader.h"
 
+#define DEBUG_TYPE "clang-tidy"
+
 namespace clang {
 namespace tooling {
 
 class ExpandModularHeadersPPCallbacks::FileRecorder {
 public:
   /// Records that a given file entry is needed for replaying callbacks.
-  void addNecessaryFile(const FileEntry *File) { FilesToRecord.insert(File); }
+  void addNecessaryFile(const FileEntry *File) {
+// Don't record module.modulemap files because it breaks same file
+// detection.
+if (!File->getName().endswith("module.modulemap"))
+  FilesToRecord.insert(File);
+  }
 
   /// Records content for a file and adds it to the FileSystem.
   void recordFileContent(const FileEntry *File,
@@ -44,8 +51,8 @@
   /// `FilesToRecord` should be empty.
   void checkAllFilesRecorded() {
 for (auto FileEntry : FilesToRecord)
-  llvm::errs() << "Did not record contents for input file: "
-   << FileEntry->getName() << "\n";
+  LLVM_DEBUG(llvm::dbgs() << "Did not record contents for input file: "
+  << FileEntry->getName() << "\n");
   }
 
 private:
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89886: [clang-tidy] Fix redefinition of module in the same module.modulemap file

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 299698.
DmitryPolukhin added a comment.

And one more time linting


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89886/new/

https://reviews.llvm.org/D89886

Files:
  clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp


Index: clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
===
--- clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
+++ clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
@@ -13,13 +13,20 @@
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Serialization/ASTReader.h"
 
+#define DEBUG_TYPE "clang-tidy"
+
 namespace clang {
 namespace tooling {
 
 class ExpandModularHeadersPPCallbacks::FileRecorder {
 public:
   /// Records that a given file entry is needed for replaying callbacks.
-  void addNecessaryFile(const FileEntry *File) { FilesToRecord.insert(File); }
+  void addNecessaryFile(const FileEntry *File) {
+// Don't record module.modulemap files because it breaks same file
+// detection.
+if (!File->getName().endswith("module.modulemap"))
+  FilesToRecord.insert(File);
+  }
 
   /// Records content for a file and adds it to the FileSystem.
   void recordFileContent(const FileEntry *File,
@@ -44,8 +51,8 @@
   /// `FilesToRecord` should be empty.
   void checkAllFilesRecorded() {
 for (auto FileEntry : FilesToRecord)
-  llvm::errs() << "Did not record contents for input file: "
-   << FileEntry->getName() << "\n";
+  LLVM_DEBUG(llvm::dbgs() << "Did not record contents for input file: "
+  << FileEntry->getName() << "\n");
   }
 
 private:


Index: clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
===
--- clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
+++ clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
@@ -13,13 +13,20 @@
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Serialization/ASTReader.h"
 
+#define DEBUG_TYPE "clang-tidy"
+
 namespace clang {
 namespace tooling {
 
 class ExpandModularHeadersPPCallbacks::FileRecorder {
 public:
   /// Records that a given file entry is needed for replaying callbacks.
-  void addNecessaryFile(const FileEntry *File) { FilesToRecord.insert(File); }
+  void addNecessaryFile(const FileEntry *File) {
+// Don't record module.modulemap files because it breaks same file
+// detection.
+if (!File->getName().endswith("module.modulemap"))
+  FilesToRecord.insert(File);
+  }
 
   /// Records content for a file and adds it to the FileSystem.
   void recordFileContent(const FileEntry *File,
@@ -44,8 +51,8 @@
   /// `FilesToRecord` should be empty.
   void checkAllFilesRecorded() {
 for (auto FileEntry : FilesToRecord)
-  llvm::errs() << "Did not record contents for input file: "
-   << FileEntry->getName() << "\n";
+  LLVM_DEBUG(llvm::dbgs() << "Did not record contents for input file: "
+  << FileEntry->getName() << "\n");
   }
 
 private:
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89886: [clang-tidy] Fix redefinition of module in the same module.modulemap file

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 299977.
DmitryPolukhin added a comment.

Added all module map file names


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89886/new/

https://reviews.llvm.org/D89886

Files:
  clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp


Index: clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
===
--- clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
+++ clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
@@ -13,13 +13,22 @@
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Serialization/ASTReader.h"
 
+#define DEBUG_TYPE "clang-tidy"
+
 namespace clang {
 namespace tooling {
 
 class ExpandModularHeadersPPCallbacks::FileRecorder {
 public:
   /// Records that a given file entry is needed for replaying callbacks.
-  void addNecessaryFile(const FileEntry *File) { FilesToRecord.insert(File); }
+  void addNecessaryFile(const FileEntry *File) {
+// Don't record modulemap files because it breaks same file detection.
+if (!(File->getName().endswith("module.modulemap") ||
+  File->getName().endswith("module.private.modulemap") ||
+  File->getName().endswith("module.map") ||
+  File->getName().endswith("module_private.map")))
+  FilesToRecord.insert(File);
+  }
 
   /// Records content for a file and adds it to the FileSystem.
   void recordFileContent(const FileEntry *File,
@@ -44,8 +53,8 @@
   /// `FilesToRecord` should be empty.
   void checkAllFilesRecorded() {
 for (auto FileEntry : FilesToRecord)
-  llvm::errs() << "Did not record contents for input file: "
-   << FileEntry->getName() << "\n";
+  LLVM_DEBUG(llvm::dbgs() << "Did not record contents for input file: "
+  << FileEntry->getName() << "\n");
   }
 
 private:


Index: clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
===
--- clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
+++ clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
@@ -13,13 +13,22 @@
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Serialization/ASTReader.h"
 
+#define DEBUG_TYPE "clang-tidy"
+
 namespace clang {
 namespace tooling {
 
 class ExpandModularHeadersPPCallbacks::FileRecorder {
 public:
   /// Records that a given file entry is needed for replaying callbacks.
-  void addNecessaryFile(const FileEntry *File) { FilesToRecord.insert(File); }
+  void addNecessaryFile(const FileEntry *File) {
+// Don't record modulemap files because it breaks same file detection.
+if (!(File->getName().endswith("module.modulemap") ||
+  File->getName().endswith("module.private.modulemap") ||
+  File->getName().endswith("module.map") ||
+  File->getName().endswith("module_private.map")))
+  FilesToRecord.insert(File);
+  }
 
   /// Records content for a file and adds it to the FileSystem.
   void recordFileContent(const FileEntry *File,
@@ -44,8 +53,8 @@
   /// `FilesToRecord` should be empty.
   void checkAllFilesRecorded() {
 for (auto FileEntry : FilesToRecord)
-  llvm::errs() << "Did not record contents for input file: "
-   << FileEntry->getName() << "\n";
+  LLVM_DEBUG(llvm::dbgs() << "Did not record contents for input file: "
+  << FileEntry->getName() << "\n");
   }
 
 private:
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89886: [clang-tidy] Fix redefinition of module in the same module.modulemap file

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin marked an inline comment as done.
DmitryPolukhin added a comment.

In D89886#2346851 , @alexfh wrote:

> Ah, btw, any chance of adding a test for this?

Oh, I was not able to create small reproducer that without including large 
Apple Frameworks with modules :( My hypothesis that it is side effect of module 
cache that triggers module load before it is referenced from sources. I tested 
it on reproducer from PR47839 + my real internal example.




Comment at: clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp:27
+// detection.
+if (!File->getName().endswith("module.modulemap"))
+  FilesToRecord.insert(File);

alexfh wrote:
> Looking at the relevant code I find special file names like "module.map", 
> "module_private.map", "module.private.modulemap". Is this problem relevant 
> for any of those?
I think it is relevant for these files too, added them all.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89886/new/

https://reviews.llvm.org/D89886

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89886: [clang-tidy] Fix redefinition of module in the same module.modulemap file

[Dmitry Polukhin via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
DmitryPolukhin marked an inline comment as done.
Closed by commit rG55a2deed075b: [clang-tidy] Fix redefinition of module in the 
same module.modulemap file (authored by DmitryPolukhin).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89886/new/

https://reviews.llvm.org/D89886

Files:
  clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp


Index: clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
===
--- clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
+++ clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
@@ -13,13 +13,22 @@
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Serialization/ASTReader.h"
 
+#define DEBUG_TYPE "clang-tidy"
+
 namespace clang {
 namespace tooling {
 
 class ExpandModularHeadersPPCallbacks::FileRecorder {
 public:
   /// Records that a given file entry is needed for replaying callbacks.
-  void addNecessaryFile(const FileEntry *File) { FilesToRecord.insert(File); }
+  void addNecessaryFile(const FileEntry *File) {
+// Don't record modulemap files because it breaks same file detection.
+if (!(File->getName().endswith("module.modulemap") ||
+  File->getName().endswith("module.private.modulemap") ||
+  File->getName().endswith("module.map") ||
+  File->getName().endswith("module_private.map")))
+  FilesToRecord.insert(File);
+  }
 
   /// Records content for a file and adds it to the FileSystem.
   void recordFileContent(const FileEntry *File,
@@ -44,8 +53,8 @@
   /// `FilesToRecord` should be empty.
   void checkAllFilesRecorded() {
 for (auto FileEntry : FilesToRecord)
-  llvm::errs() << "Did not record contents for input file: "
-   << FileEntry->getName() << "\n";
+  LLVM_DEBUG(llvm::dbgs() << "Did not record contents for input file: "
+  << FileEntry->getName() << "\n");
   }
 
 private:


Index: clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
===
--- clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
+++ clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
@@ -13,13 +13,22 @@
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Serialization/ASTReader.h"
 
+#define DEBUG_TYPE "clang-tidy"
+
 namespace clang {
 namespace tooling {
 
 class ExpandModularHeadersPPCallbacks::FileRecorder {
 public:
   /// Records that a given file entry is needed for replaying callbacks.
-  void addNecessaryFile(const FileEntry *File) { FilesToRecord.insert(File); }
+  void addNecessaryFile(const FileEntry *File) {
+// Don't record modulemap files because it breaks same file detection.
+if (!(File->getName().endswith("module.modulemap") ||
+  File->getName().endswith("module.private.modulemap") ||
+  File->getName().endswith("module.map") ||
+  File->getName().endswith("module_private.map")))
+  FilesToRecord.insert(File);
+  }
 
   /// Records content for a file and adds it to the FileSystem.
   void recordFileContent(const FileEntry *File,
@@ -44,8 +53,8 @@
   /// `FilesToRecord` should be empty.
   void checkAllFilesRecorded() {
 for (auto FileEntry : FilesToRecord)
-  llvm::errs() << "Did not record contents for input file: "
-   << FileEntry->getName() << "\n";
+  LLVM_DEBUG(llvm::dbgs() << "Did not record contents for input file: "
+  << FileEntry->getName() << "\n");
   }
 
 private:
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89936: clang-tidy: adding "--clang-tidy-config=" to specify custom config file

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

I'm not sure that we need additional option to read configuration from file 
but, if we do need, I think this diff needs some improvements + test for new 
option.




Comment at: clang-tools-extra/clang-tidy/ClangTidyOptions.cpp:324
+llvm::sys::fs::is_symlink_file(Twine(AbsoluteFilePath), IsLink);
+if (!(IsFile || IsLink)) {
+  std::string Msg;

Is it actually required to check absolute path, link it or not, etc.? Why not 
just try reading file with provided filename and report error if it fails?



Comment at: clang-tools-extra/clang-tidy/ClangTidyOptions.cpp:408
+FileOptionsProvider::tryReadConfigFile(StringRef Path, bool IsFile) {
+  // llvm::outs() << "tryReadConfigFile IsFile<" <<
+  // OverrideOptions.ClangTidyConfig << ">\n";

It seems like some debug prints.



Comment at: clang-tools-extra/clang-tidy/ClangTidyOptions.h:68
+  /// Clang-tidy-config
+  llvm::Optional ClangTidyConfig;
+

I'm not sure that we need it here. I would reuse code path for `--config` 
options as much as possible and implement new option as a simple wrapper that 
reads content of the file and interpret it as `--config` option. Moreover I 
think it should not be possible to specify both options in command line.



Comment at: clang-tools-extra/clang-tidy/ClangTidyOptions.h:233
   llvm::Optional tryReadConfigFile(llvm::StringRef Directory);
+  llvm::Optional tryReadConfigFile(llvm::StringRef Path,
+  bool IsFile);

It looks like the second argument was added only for overload resolution. But I 
think it is better to rename this function. After all it is not "try" anymore 
because it reports fatal error in case of errors.



Comment at: clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp:76
 
+static cl::opt ClangTidyConfig("clang-tidy-config", cl::desc(R"(
+Specify full path of .clang-tidy config file.

I would call it something like `--config-file`


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89936/new/

https://reviews.llvm.org/D89936

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89936: [clang-tidy] adding "--clang-tidy-config=" to specify custom config file

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

@Hiralo, it looks like you uploaded wrong diff because your previous revision 
is shown as "base" version. Base revision should be clang-tidy sources without 
your changes.

Please add Lit test for the new option, see 
https://llvm.org/docs/TestingGuide.html for more details and existing 
clang-tidy tests.




Comment at: clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp:330
   if (!Config.empty()) {
 if (llvm::ErrorOr ParsedConfig =
 parseConfiguration(Config)) {

I think you can make this option much simpler if you just read file content and 
use it or `Config` here. No changes in 
clang-tools-extra/clang-tidy/ClangTidyOptions.h/.cpp will be required.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89936/new/

https://reviews.llvm.org/D89936

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89936: [clang-tidy] adding "--clang-tidy-config=" to specify custom config file

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

Yes, it is what I meant as a simpler solution. But please add Lit test even for 
such trivial option.




Comment at: clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp:337
+
+Config.assign((*Text)->getBuffer());
+  }

I suggest creating new local variable with text of the config from `Config` or 
`ConfigFile` file content i.e. avoid modifying `Config` itself.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89936/new/

https://reviews.llvm.org/D89936

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89936: [clang-tidy] adding "--clang-tidy-config=" to specify custom config file

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added inline comments.



Comment at: clang-tools-extra/test/clang-tidy/infrastructure/config-file.cpp:2
+// REQUIRES: shell
+// RUN: mkdir -p %T/config-file/
+// RUN: cp %s %T/read-file-config/test.cpp

I think it is better to use some file in `%S/Inputs` like in 
https://github.com/llvm/llvm-project/blob/master/clang-tools-extra/test/clang-tidy/infrastructure/config-files.cpp


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89936/new/

https://reviews.llvm.org/D89936

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89936: [clang-tidy] adding "--clang-tidy-config=" to specify custom config file

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

I think this diff looks very close to what we need. I hope it will be the last 
iteration.

Please also update title and description with the new name of the option, etc.




Comment at: clang-tools-extra/test/clang-tidy/infrastructure/config-file.cpp:2
+// REQUIRES: shell
+// RUN: clang-tidy --config-file=%S/Inputs/config-file/config-file 
%T/read-file-config/test.cpp | grep "warning: 
.*\[hicpp-uppercase-literal-suffix\]$"
+

Please use `FileCheck` as all other tests do instead of `grep` to make test 
more portable. Also with `FileCheck`  it should be no need in `REQUIRES: shell`.

And I think it is better to use `-dump-config` instead of checking specific 
check on a source file. This way you will tests only functionality that you 
implemented, without dependency some check behavior. Also it could be just 
another tests case in clang-tidy/infrastructure/config-files.cpp.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89936/new/

https://reviews.llvm.org/D89936

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89936: [clang-tidy] adding "--config-file=" to specify custom config file.

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added inline comments.



Comment at: clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp:337
+
+Config.assign((*Text)->getBuffer());
+  }

DmitryPolukhin wrote:
> I suggest creating new local variable with text of the config from `Config` 
> or `ConfigFile` file content i.e. avoid modifying `Config` itself.
It doesn't compile and using local variable instead of changing command line 
option will make this code easier to read and understand:
```
clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp:334:12: error: no matching 
member function for call to 'assign'
Config.assign((*Text)->getBuffer());
~~~^~
```
Could you please fix build and please upload diff with `arc diff` as @njames93 
suggested so buildbot can test your changes?


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89936/new/

https://reviews.llvm.org/D89936

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89936: [clang-tidy] adding "--config-file=" to specify custom config file.

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

In D89936#2361580 , @njames93 wrote:

> It's also be nice if --config-file would also support being passed a 
> directory. If a directory was passed it would append ".clang-tidy" to the 
> path and load that file, WDYT?

It might be misleading what "InheritParentConfig: true" in the file means in 
this case. Current implementation that it will use directory structure of the 
source file and I think it is right approach. I would prefer to keep things 
simple and straightforward: `--config-file` should be file and it is simple 
helper for `--config` for the cases when you cannot use shell power to read 
file content to the command line or you exceed limit on command line length.




Comment at: clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp:320-324
+if (!Checks.empty()) {
+  llvm::errs() << "Error: --config-file and --checks are mutually "
+  "exclusive. Specify only one.\n";
+  return nullptr;
+}

njames93 wrote:
> I disagree with this check here, `Config` is not mutually exclusive with 
> `Checks`, `Checks` gets applied atop of `Config`. So the same should happen 
> when using `ConfigFile` with `Checks`
+1


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89936/new/

https://reviews.llvm.org/D89936

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89936: [clang-tidy] adding "--config-file=" to specify custom config file.

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added inline comments.



Comment at: clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp:320-324
+if (!Checks.empty()) {
+  llvm::errs() << "Error: --config-file and --checks are mutually "
+  "exclusive. Specify only one.\n";
+  return nullptr;
+}

DmitryPolukhin wrote:
> njames93 wrote:
> > I disagree with this check here, `Config` is not mutually exclusive with 
> > `Checks`, `Checks` gets applied atop of `Config`. So the same should happen 
> > when using `ConfigFile` with `Checks`
> +1
Clarify: +1 to @njames93 that we don't need this check.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89936/new/

https://reviews.llvm.org/D89936

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89936: [clang-tidy] adding "--config-file=" to specify custom config file.

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added inline comments.



Comment at: clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp:320-324
+if (!Checks.empty()) {
+  llvm::errs() << "Error: --config-file and --checks are mutually "
+  "exclusive. Specify only one.\n";
+  return nullptr;
+}

Hiralo wrote:
> DmitryPolukhin wrote:
> > DmitryPolukhin wrote:
> > > njames93 wrote:
> > > > I disagree with this check here, `Config` is not mutually exclusive 
> > > > with `Checks`, `Checks` gets applied atop of `Config`. So the same 
> > > > should happen when using `ConfigFile` with `Checks`
> > > +1
> > Clarify: +1 to @njames93 that we don't need this check.
> > I disagree with this check here, `Config` is not mutually exclusive with 
> > `Checks`, `Checks` gets applied atop of `Config`. So the same should happen 
> > when using `ConfigFile` with `Checks`
> 
> ok, will remove these two checks :)
I think we need to keep the first check. There is no reason to specify both 
`--config` and `--config-file` or give on of them precedence.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89936/new/

https://reviews.llvm.org/D89936

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89936: [clang-tidy] adding "--config-file=" to specify custom config file.

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin accepted this revision.
DmitryPolukhin added a comment.
This revision is now accepted and ready to land.

Looks good to me.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89936/new/

https://reviews.llvm.org/D89936

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89936: [clang-tidy] adding "--config-file=" to specify custom config file.

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

In D89936#2370163 , @Hiralo wrote:

> When this patch will be merged and available in master?

@Hiralo I can push this changes to master for you. Please let me know if you 
need it.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89936/new/

https://reviews.llvm.org/D89936

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89936: [clang-tidy] adding "--config-file=" to specify custom config file.

[Dmitry Polukhin via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rGd6a468d622b2: [clang-tidy] adding 
"--config-file=" to specify custom config file. 
(authored by Hiralo, committed by DmitryPolukhin).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89936/new/

https://reviews.llvm.org/D89936

Files:
  clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-file/config-file
  clang-tools-extra/test/clang-tidy/infrastructure/config-file.cpp


Index: clang-tools-extra/test/clang-tidy/infrastructure/config-file.cpp
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/config-file.cpp
@@ -0,0 +1,2 @@
+// RUN: clang-tidy -config-file=%S/Inputs/config-file/config-file -dump-config 
-- | FileCheck %s -check-prefix=CHECK-BASE
+// CHECK-BASE: Checks: {{.*}}hicpp-uppercase-literal-suffix
Index: 
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-file/config-file
===
--- /dev/null
+++ 
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-file/config-file
@@ -0,0 +1 @@
+Checks: "-*,hicpp-uppercase-literal-suffix"
Index: clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp
===
--- clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp
+++ clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp
@@ -168,6 +168,16 @@
 )"),
cl::init(""), cl::cat(ClangTidyCategory));
 
+static cl::opt ConfigFile("config-file", cl::desc(R"(
+Specify the path of .clang-tidy or custom config file:
+ e.g. --config-file=/some/path/myTidyConfigFile
+This option internally works exactly the same way as
+ --config option after reading specified config file.
+Use either --config-file or --config, not both.
+)"),
+   cl::init(""),
+   cl::cat(ClangTidyCategory));
+
 static cl::opt DumpConfig("dump-config", cl::desc(R"(
 Dumps configuration in the YAML format to
 stdout. This option can be used along with a
@@ -302,19 +312,41 @@
   if (UseColor.getNumOccurrences() > 0)
 OverrideOptions.UseColor = UseColor;
 
-  if (!Config.empty()) {
-if (llvm::ErrorOr ParsedConfig =
-parseConfiguration(Config)) {
+  auto LoadConfig = [&](StringRef Configuration)
+  -> std::unique_ptr {
+llvm::ErrorOr ParsedConfig =
+parseConfiguration(Configuration);
+if (ParsedConfig)
   return std::make_unique(
   GlobalOptions,
   ClangTidyOptions::getDefaults().mergeWith(DefaultOptions, 0),
   *ParsedConfig, OverrideOptions, std::move(FS));
-} else {
-  llvm::errs() << "Error: invalid configuration specified.\n"
-   << ParsedConfig.getError().message() << "\n";
+llvm::errs() << "Error: invalid configuration specified.\n"
+ << ParsedConfig.getError().message() << "\n";
+return nullptr;
+  };
+
+  if (ConfigFile.getNumOccurrences() > 0) {
+if (Config.getNumOccurrences() > 0) {
+  llvm::errs() << "Error: --config-file and --config are "
+  "mutually exclusive. Specify only one.\n";
   return nullptr;
 }
+
+llvm::ErrorOr> Text =
+llvm::MemoryBuffer::getFile(ConfigFile.c_str());
+if (std::error_code EC = Text.getError()) {
+  llvm::errs() << "Error: can't read config-file '" << ConfigFile
+   << "': " << EC.message() << "\n";
+  return nullptr;
+}
+
+return LoadConfig((*Text)->getBuffer());
   }
+
+  if (Config.getNumOccurrences() > 0)
+return LoadConfig(Config);
+
   return std::make_unique(GlobalOptions, DefaultOptions,
 OverrideOptions, 
std::move(FS));
 }


Index: clang-tools-extra/test/clang-tidy/infrastructure/config-file.cpp
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/config-file.cpp
@@ -0,0 +1,2 @@
+// RUN: clang-tidy -config-file=%S/Inputs/config-file/config-file -dump-config -- | FileCheck %s -check-prefix=CHECK-BASE
+// CHECK-BASE: Checks: {{.*}}hicpp-uppercase-literal-suffix
Index: clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-file/config-file
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/Inputs/config-file/config-file
@@ -0,0 +1 @@
+Checks: "-*,hicpp-uppercase-literal-suffix"
Index: clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp
===
--- clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp
+++ clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp
@@ -168,6 +168,16 @@
 )"),

[PATCH] D89936: [clang-tidy] adding "--config-file=" to specify custom config file.

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

> Hello @@DmitryPolukhin ,
>
> When I submitted latest via 'arc diff' my commit-message was...
> Can you please help to have following commit message? Below commit message is 
> more clear and helpful.
> Sorry for inconvenience caused!

It is not possible to update commit message afterwords. I received commit 
message from `arc patch` your diff 
(https://llvm.org/docs/Phabricator.html#committing-someone-s-change-from-phabricator).
 The only thing that I did manually was removing extra spaces and second link 
to the same codereview. I think `arc patch`  ignores upload messages and uses 
diff description from Phabricator. Upload message just describes what you have 
changed in comparison with previous upload so it is expected behaviour.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89936/new/

https://reviews.llvm.org/D89936

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102906: [clang-tidy] Remark was added to clang tooling Diagnostic

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added inline comments.



Comment at: clang/include/clang/Tooling/Core/Diagnostic.h:72
+Error = DiagnosticsEngine::Error,
+Remark = DiagnosticsEngine::Remark
   };

nit, I would move remark first to have list by increasing severity. It looks 
like this enum values is not used outside of the sources so I hope it is safe.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102906/new/

https://reviews.llvm.org/D102906

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D103142: [clang][clangd] Use reverse header map lookup in suggestPathToFileForDiagnostics

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin created this revision.
DmitryPolukhin added reviewers: dexonsmith, bruno, rsmith.
DmitryPolukhin added a project: clang.
Herald added subscribers: usaxena95, ormris, kadircet, arphaman.
DmitryPolukhin requested review of this revision.
Herald added subscribers: MaskRay, ilya-biryukov.

suggestPathToFileForDiagnostics is actively used in clangd for converting
an absolute path to a header file to a header name as it should be spelled
in the sources. Current approach converts absolute path to relative path.
This diff implements missing logic that makes a reverse lookup from the
relative path to the key in the header map that should be used in the sources.

Test Plan: check-clang


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D103142

Files:
  clang/include/clang/Lex/HeaderMap.h
  clang/lib/Lex/HeaderMap.cpp
  clang/lib/Lex/HeaderSearch.cpp
  clang/unittests/Lex/HeaderMapTest.cpp
  clang/unittests/Lex/HeaderMapTestUtils.h
  clang/unittests/Lex/HeaderSearchTest.cpp

Index: clang/unittests/Lex/HeaderSearchTest.cpp
===
--- clang/unittests/Lex/HeaderSearchTest.cpp
+++ clang/unittests/Lex/HeaderSearchTest.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "clang/Lex/HeaderSearch.h"
+#include "HeaderMapTestUtils.h"
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/DiagnosticOptions.h"
 #include "clang/Basic/FileManager.h"
@@ -45,6 +46,21 @@
 Search.AddSearchPath(DL, /*isAngled=*/false);
   }
 
+  void addHeaderMap(llvm::StringRef Filename,
+std::unique_ptr Buf) {
+VFS->addFile(Filename, 0, std::move(Buf), /*User=*/None, /*Group=*/None,
+ llvm::sys::fs::file_type::regular_file);
+auto FE = FileMgr.getFile(Filename, true);
+assert(FE);
+
+// Test class supports only one HMap at a time.
+assert(!HMap);
+HMap = HeaderMap::Create(*FE, FileMgr);
+auto DL =
+DirectoryLookup(HMap.get(), SrcMgr::C_User, /*isFramework=*/false);
+Search.AddSearchPath(DL, /*isAngled=*/false);
+  }
+
   IntrusiveRefCntPtr VFS;
   FileSystemOptions FileMgrOpts;
   FileManager FileMgr;
@@ -55,6 +71,7 @@
   std::shared_ptr TargetOpts;
   IntrusiveRefCntPtr Target;
   HeaderSearch Search;
+  std::unique_ptr HMap;
 };
 
 TEST_F(HeaderSearchTest, NoSearchDir) {
@@ -136,5 +153,31 @@
 "y/z/t.h");
 }
 
+// Helper struct with null terminator character to make MemoryBuffer happy.
+template 
+struct NullTerminatedFile : public FileTy {
+  PaddingTy Padding = 0;
+};
+
+TEST_F(HeaderSearchTest, HeaderMapReverseLookup) {
+  typedef NullTerminatedFile, char> FileTy;
+  FileTy File;
+  File.init();
+
+  test::HMapFileMockMaker Maker(File);
+  auto a = Maker.addString("d.h");
+  auto b = Maker.addString("b/");
+  auto c = Maker.addString("c.h");
+  Maker.addBucket("d.h", a, b, c);
+
+  addHeaderMap("/x/y/z.hmap", File.getBuffer());
+  addSearchDir("/a");
+
+  EXPECT_EQ(Search.suggestPathToFileForDiagnostics("/a/b/c.h",
+   /*WorkingDir=*/"",
+   /*MainFile=*/""),
+"d.h");
+}
+
 } // namespace
 } // namespace clang
Index: clang/unittests/Lex/HeaderMapTestUtils.h
===
--- /dev/null
+++ clang/unittests/Lex/HeaderMapTestUtils.h
@@ -0,0 +1,100 @@
+//===- unittests/Lex/HeaderMapTestUtils.h - HeaderMap utils ---===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+
+#ifndef LLVM_CLANG_UNITTESTS_LEX_HEADERMAPTESTUTILS_H
+#define LLVM_CLANG_UNITTESTS_LEX_HEADERMAPTESTUTILS_H
+
+#include "clang/Basic/CharInfo.h"
+#include "clang/Lex/HeaderMap.h"
+#include "clang/Lex/HeaderMapTypes.h"
+#include "llvm/Support/SwapByteOrder.h"
+#include 
+
+namespace clang {
+namespace test {
+
+// Lay out a header file for testing.
+template  struct HMapFileMock {
+  HMapHeader Header;
+  HMapBucket Buckets[NumBuckets];
+  unsigned char Bytes[NumBytes];
+
+  void init() {
+memset(this, 0, sizeof(HMapFileMock));
+Header.Magic = HMAP_HeaderMagicNumber;
+Header.Version = HMAP_HeaderVersion;
+Header.NumBuckets = NumBuckets;
+Header.StringsOffset = sizeof(Header) + sizeof(Buckets);
+  }
+
+  void swapBytes() {
+using llvm::sys::getSwappedBytes;
+Header.Magic = getSwappedBytes(Header.Magic);
+Header.Version = getSwappedBytes(Header.Version);
+Header.NumBuckets = getSwappedBytes(Header.NumBuckets);
+Header.StringsOffset = getSwappedBytes(Header.StringsOffset);
+  }
+
+  std::unique_ptr getBuffer() {
+return llvm::MemoryBuffer::getMemBuffer(
+StringRef(reinterpret_cast(this), sizeof(H

[PATCH] D103142: [clang][clangd] Use reverse header map lookup in suggestPathToFileForDiagnostics

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 347862.
DmitryPolukhin added a comment.

Fix clang-format issue


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D103142/new/

https://reviews.llvm.org/D103142

Files:
  clang/include/clang/Lex/HeaderMap.h
  clang/lib/Lex/HeaderMap.cpp
  clang/lib/Lex/HeaderSearch.cpp
  clang/unittests/Lex/HeaderMapTest.cpp
  clang/unittests/Lex/HeaderMapTestUtils.h
  clang/unittests/Lex/HeaderSearchTest.cpp

Index: clang/unittests/Lex/HeaderSearchTest.cpp
===
--- clang/unittests/Lex/HeaderSearchTest.cpp
+++ clang/unittests/Lex/HeaderSearchTest.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "clang/Lex/HeaderSearch.h"
+#include "HeaderMapTestUtils.h"
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/DiagnosticOptions.h"
 #include "clang/Basic/FileManager.h"
@@ -45,6 +46,21 @@
 Search.AddSearchPath(DL, /*isAngled=*/false);
   }
 
+  void addHeaderMap(llvm::StringRef Filename,
+std::unique_ptr Buf) {
+VFS->addFile(Filename, 0, std::move(Buf), /*User=*/None, /*Group=*/None,
+ llvm::sys::fs::file_type::regular_file);
+auto FE = FileMgr.getFile(Filename, true);
+assert(FE);
+
+// Test class supports only one HMap at a time.
+assert(!HMap);
+HMap = HeaderMap::Create(*FE, FileMgr);
+auto DL =
+DirectoryLookup(HMap.get(), SrcMgr::C_User, /*isFramework=*/false);
+Search.AddSearchPath(DL, /*isAngled=*/false);
+  }
+
   IntrusiveRefCntPtr VFS;
   FileSystemOptions FileMgrOpts;
   FileManager FileMgr;
@@ -55,6 +71,7 @@
   std::shared_ptr TargetOpts;
   IntrusiveRefCntPtr Target;
   HeaderSearch Search;
+  std::unique_ptr HMap;
 };
 
 TEST_F(HeaderSearchTest, NoSearchDir) {
@@ -136,5 +153,31 @@
 "y/z/t.h");
 }
 
+// Helper struct with null terminator character to make MemoryBuffer happy.
+template 
+struct NullTerminatedFile : public FileTy {
+  PaddingTy Padding = 0;
+};
+
+TEST_F(HeaderSearchTest, HeaderMapReverseLookup) {
+  typedef NullTerminatedFile, char> FileTy;
+  FileTy File;
+  File.init();
+
+  test::HMapFileMockMaker Maker(File);
+  auto a = Maker.addString("d.h");
+  auto b = Maker.addString("b/");
+  auto c = Maker.addString("c.h");
+  Maker.addBucket("d.h", a, b, c);
+
+  addHeaderMap("/x/y/z.hmap", File.getBuffer());
+  addSearchDir("/a");
+
+  EXPECT_EQ(Search.suggestPathToFileForDiagnostics("/a/b/c.h",
+   /*WorkingDir=*/"",
+   /*MainFile=*/""),
+"d.h");
+}
+
 } // namespace
 } // namespace clang
Index: clang/unittests/Lex/HeaderMapTestUtils.h
===
--- /dev/null
+++ clang/unittests/Lex/HeaderMapTestUtils.h
@@ -0,0 +1,100 @@
+//===- unittests/Lex/HeaderMapTestUtils.h - HeaderMap utils ---===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+
+#ifndef LLVM_CLANG_UNITTESTS_LEX_HEADERMAPTESTUTILS_H
+#define LLVM_CLANG_UNITTESTS_LEX_HEADERMAPTESTUTILS_H
+
+#include "clang/Basic/CharInfo.h"
+#include "clang/Lex/HeaderMap.h"
+#include "clang/Lex/HeaderMapTypes.h"
+#include "llvm/Support/SwapByteOrder.h"
+#include 
+
+namespace clang {
+namespace test {
+
+// Lay out a header file for testing.
+template  struct HMapFileMock {
+  HMapHeader Header;
+  HMapBucket Buckets[NumBuckets];
+  unsigned char Bytes[NumBytes];
+
+  void init() {
+memset(this, 0, sizeof(HMapFileMock));
+Header.Magic = HMAP_HeaderMagicNumber;
+Header.Version = HMAP_HeaderVersion;
+Header.NumBuckets = NumBuckets;
+Header.StringsOffset = sizeof(Header) + sizeof(Buckets);
+  }
+
+  void swapBytes() {
+using llvm::sys::getSwappedBytes;
+Header.Magic = getSwappedBytes(Header.Magic);
+Header.Version = getSwappedBytes(Header.Version);
+Header.NumBuckets = getSwappedBytes(Header.NumBuckets);
+Header.StringsOffset = getSwappedBytes(Header.StringsOffset);
+  }
+
+  std::unique_ptr getBuffer() {
+return llvm::MemoryBuffer::getMemBuffer(
+StringRef(reinterpret_cast(this), sizeof(HMapFileMock)),
+"header",
+/* RequresNullTerminator */ false);
+  }
+};
+
+template  struct HMapFileMockMaker {
+  FileTy &File;
+  unsigned SI = 1;
+  unsigned BI = 0;
+  HMapFileMockMaker(FileTy &File) : File(File) {}
+
+  unsigned addString(StringRef S) {
+assert(SI + S.size() + 1 <= sizeof(File.Bytes));
+std::copy(S.begin(), S.end(), File.Bytes + SI);
+auto OldSI = SI;
+SI += S.size() + 1;
+return OldSI;
+  }
+
+  void addBucket(StringRef Str, unsigned Key, unsigned Prefix,
+

[PATCH] D103142: [clang][clangd] Use reverse header map lookup in suggestPathToFileForDiagnostics

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 347864.
DmitryPolukhin added a comment.

Fix linter


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D103142/new/

https://reviews.llvm.org/D103142

Files:
  clang/include/clang/Lex/HeaderMap.h
  clang/lib/Lex/HeaderMap.cpp
  clang/lib/Lex/HeaderSearch.cpp
  clang/unittests/Lex/HeaderMapTest.cpp
  clang/unittests/Lex/HeaderMapTestUtils.h
  clang/unittests/Lex/HeaderSearchTest.cpp

Index: clang/unittests/Lex/HeaderSearchTest.cpp
===
--- clang/unittests/Lex/HeaderSearchTest.cpp
+++ clang/unittests/Lex/HeaderSearchTest.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "clang/Lex/HeaderSearch.h"
+#include "HeaderMapTestUtils.h"
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/DiagnosticOptions.h"
 #include "clang/Basic/FileManager.h"
@@ -45,6 +46,21 @@
 Search.AddSearchPath(DL, /*isAngled=*/false);
   }
 
+  void addHeaderMap(llvm::StringRef Filename,
+std::unique_ptr Buf) {
+VFS->addFile(Filename, 0, std::move(Buf), /*User=*/None, /*Group=*/None,
+ llvm::sys::fs::file_type::regular_file);
+auto FE = FileMgr.getFile(Filename, true);
+assert(FE);
+
+// Test class supports only one HMap at a time.
+assert(!HMap);
+HMap = HeaderMap::Create(*FE, FileMgr);
+auto DL =
+DirectoryLookup(HMap.get(), SrcMgr::C_User, /*isFramework=*/false);
+Search.AddSearchPath(DL, /*isAngled=*/false);
+  }
+
   IntrusiveRefCntPtr VFS;
   FileSystemOptions FileMgrOpts;
   FileManager FileMgr;
@@ -55,6 +71,7 @@
   std::shared_ptr TargetOpts;
   IntrusiveRefCntPtr Target;
   HeaderSearch Search;
+  std::unique_ptr HMap;
 };
 
 TEST_F(HeaderSearchTest, NoSearchDir) {
@@ -136,5 +153,31 @@
 "y/z/t.h");
 }
 
+// Helper struct with null terminator character to make MemoryBuffer happy.
+template 
+struct NullTerminatedFile : public FileTy {
+  PaddingTy Padding = 0;
+};
+
+TEST_F(HeaderSearchTest, HeaderMapReverseLookup) {
+  typedef NullTerminatedFile, char> FileTy;
+  FileTy File;
+  File.init();
+
+  test::HMapFileMockMaker Maker(File);
+  auto a = Maker.addString("d.h");
+  auto b = Maker.addString("b/");
+  auto c = Maker.addString("c.h");
+  Maker.addBucket("d.h", a, b, c);
+
+  addHeaderMap("/x/y/z.hmap", File.getBuffer());
+  addSearchDir("/a");
+
+  EXPECT_EQ(Search.suggestPathToFileForDiagnostics("/a/b/c.h",
+   /*WorkingDir=*/"",
+   /*MainFile=*/""),
+"d.h");
+}
+
 } // namespace
 } // namespace clang
Index: clang/unittests/Lex/HeaderMapTestUtils.h
===
--- /dev/null
+++ clang/unittests/Lex/HeaderMapTestUtils.h
@@ -0,0 +1,100 @@
+//===- unittests/Lex/HeaderMapTestUtils.h - HeaderMap utils ---===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+
+#ifndef LLVM_CLANG_UNITTESTS_LEX_HEADERMAPTESTUTILS_H
+#define LLVM_CLANG_UNITTESTS_LEX_HEADERMAPTESTUTILS_H
+
+#include "clang/Basic/CharInfo.h"
+#include "clang/Lex/HeaderMap.h"
+#include "clang/Lex/HeaderMapTypes.h"
+#include "llvm/Support/SwapByteOrder.h"
+#include 
+
+namespace clang {
+namespace test {
+
+// Lay out a header file for testing.
+template  struct HMapFileMock {
+  HMapHeader Header;
+  HMapBucket Buckets[NumBuckets];
+  unsigned char Bytes[NumBytes];
+
+  void init() {
+memset(this, 0, sizeof(HMapFileMock));
+Header.Magic = HMAP_HeaderMagicNumber;
+Header.Version = HMAP_HeaderVersion;
+Header.NumBuckets = NumBuckets;
+Header.StringsOffset = sizeof(Header) + sizeof(Buckets);
+  }
+
+  void swapBytes() {
+using llvm::sys::getSwappedBytes;
+Header.Magic = getSwappedBytes(Header.Magic);
+Header.Version = getSwappedBytes(Header.Version);
+Header.NumBuckets = getSwappedBytes(Header.NumBuckets);
+Header.StringsOffset = getSwappedBytes(Header.StringsOffset);
+  }
+
+  std::unique_ptr getBuffer() {
+return llvm::MemoryBuffer::getMemBuffer(
+StringRef(reinterpret_cast(this), sizeof(HMapFileMock)),
+"header",
+/* RequresNullTerminator */ false);
+  }
+};
+
+template  struct HMapFileMockMaker {
+  FileTy &File;
+  unsigned SI = 1;
+  unsigned BI = 0;
+  HMapFileMockMaker(FileTy &File) : File(File) {}
+
+  unsigned addString(StringRef S) {
+assert(SI + S.size() + 1 <= sizeof(File.Bytes));
+std::copy(S.begin(), S.end(), File.Bytes + SI);
+auto OldSI = SI;
+SI += S.size() + 1;
+return OldSI;
+  }
+
+  void addBucket(StringRef Str, unsigned Key, unsigned Prefix,
+

[PATCH] D90835: [clang-tidy] Ignore diagnostics due to macro expansion from not-interested headers

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 347968.
DmitryPolukhin added a comment.
Herald added a subscriber: dexonsmith.

Added test for system like object macro


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D90835/new/

https://reviews.llvm.org/D90835

Files:
  clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.cpp
  clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.h
  clang-tools-extra/clang-tidy/ClangTidyOptions.cpp
  clang-tools-extra/clang-tidy/ClangTidyOptions.h
  clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp
  clang-tools-extra/test/clang-tidy/infrastructure/Inputs/macros/macros.h
  clang-tools-extra/test/clang-tidy/infrastructure/Inputs/macros/macros_filter.h
  
clang-tools-extra/test/clang-tidy/infrastructure/Inputs/macros/system/sysmacros.h
  clang-tools-extra/test/clang-tidy/infrastructure/macros.cpp
  clang/include/clang/Basic/SourceManager.h

Index: clang/include/clang/Basic/SourceManager.h
===
--- clang/include/clang/Basic/SourceManager.h
+++ clang/include/clang/Basic/SourceManager.h
@@ -397,6 +397,11 @@
getExpansionLocStart() != getExpansionLocEnd();
   }
 
+  bool isObjectMacroExpansion() const {
+return getExpansionLocStart().isValid() &&
+   getExpansionLocStart() == getExpansionLocEnd();
+  }
+
   /// Return a ExpansionInfo for an expansion.
   ///
   /// Start and End specify the expansion range (where the macro is
Index: clang-tools-extra/test/clang-tidy/infrastructure/macros.cpp
===
--- clang-tools-extra/test/clang-tidy/infrastructure/macros.cpp
+++ clang-tools-extra/test/clang-tidy/infrastructure/macros.cpp
@@ -1,7 +1,50 @@
-// RUN: clang-tidy -checks='-*,google-explicit-constructor' %s -- | FileCheck %s
+// RUN: clang-tidy -checks='-*,google-explicit-constructor,modernize-avoid-c-arrays' --header-filter='macros_filter.h' --filter-macro-from-headers %s -- -isystem %S/Inputs/macros/system -I %S/Inputs/macros | FileCheck %s
+// RUN: clang-tidy --config='{"Checks": "-*,google-explicit-constructor,modernize-avoid-c-arrays", "HeaderFilterRegex": "macros_filter.h", "FilterMacroFromHeaders": true}' %s -- -isystem %S/Inputs/macros/system -I %S/Inputs/macros | FileCheck %s
+
+#include "macros.h"
+#include "macros_filter.h"
+#include 
 
 #define Q(name) class name { name(int i); }
 
 Q(A);
 // CHECK: :[[@LINE-1]]:3: warning: single-argument constructors must be marked explicit
-// CHECK: :3:30: note: expanded from macro 'Q'
+// CHECK: :[[@LINE-4]]:30: note: expanded from macro 'Q'
+
+#define MAIN_MACRO_DIAG_IN_ARG(a) a
+MAIN_MACRO_DIAG_IN_ARG(int var_A[10]);
+// CHECK: :[[@LINE-1]]:24: warning: do not declare C-style arrays, use std::array<> instead
+
+#define MAIN_MACRO_DIAG_IN_BODY int var_A1[10]
+MAIN_MACRO_DIAG_IN_BODY;
+// CHECK: :[[@LINE-1]]:1: warning: do not declare C-style arrays, use std::array<> instead
+// CHECK: :[[@LINE-3]]:33: note: expanded from macro 'MAIN_MACRO_DIAG_IN_BODY'
+
+HEADER_FILTER_MACRO_DIAG_IN_ARG(int var_B[10]);
+// CHECK: :[[@LINE-1]]:33: warning: do not declare C-style arrays, use std::array<> instead
+
+HEADER_FILTER_MACRO_DIAG_IN_BODY;
+// CHECK: :[[@LINE-1]]:1: warning: do not declare C-style arrays, use std::array<> instead
+// CHECK: note: expanded from macro 'HEADER_FILTER_MACRO_DIAG_IN_BODY'
+
+#define MAIN_MACRO_WRAPPER HEADER_FILTER_MACRO_DIAG_IN_BODY2
+MAIN_MACRO_WRAPPER;
+// CHECK: :[[@LINE-1]]:1: warning: do not declare C-style arrays, use std::array<> instead
+// CHECK: note: expanded from macro 'MAIN_MACRO_WRAPPER'
+// CHECK: note: expanded from macro 'HEADER_FILTER_MACRO_DIAG_IN_BODY2'
+
+// CHECK-NOT: warning:
+HEADER_MACRO_DIAG_IN_ARG(int var_C[10]);
+HEADER_MACRO_DIAG_IN_BODY;
+
+#define MAIN_MACRO_WRAPPER2 HEADER_MACRO_DIAG_IN_BODY2
+MAIN_MACRO_WRAPPER2;
+
+SYS_HEADER_MACRO_FUNCTION(int var_D[10]);
+
+SYS_HEADER_MACRO_OBJECT var_D1[10];
+// CHECK: :[[@LINE-1]]:1: warning: do not declare C-style arrays, use std::array<> instead
+// CHECK: note: expanded from macro 'SYS_HEADER_MACRO_OBJECT'
+
+DEFINE_bool(test, false);
+// CHECK-NOT: warning:
Index: clang-tools-extra/test/clang-tidy/infrastructure/Inputs/macros/system/sysmacros.h
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/Inputs/macros/system/sysmacros.h
@@ -0,0 +1,2 @@
+#define SYS_HEADER_MACRO_FUNCTION(a) a
+#define SYS_HEADER_MACRO_OBJECT int
Index: clang-tools-extra/test/clang-tidy/infrastructure/Inputs/macros/macros_filter.h
===
--- /dev/null
+++ clang-tools-extra/test/clang-tidy/infrastructure/Inputs/macros/macros_filter.h
@@ -0,0 +1,3 @@
+#define HEADER_FILTER_MACRO_DIAG_IN_ARG(a) a
+#define HEADER_FILTER_MACRO_DIAG_IN_BODY int var_B1[10]
+#define HEADER_FILTER_MACRO_DIAG_IN_BODY2 int var_B2[10]
Index: clang-tools-e

[PATCH] D90835: [clang-tidy] Ignore diagnostics due to macro expansion from not-interested headers

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

@alexfh, @njames93 and @thakis please take a look! I added all tests cases and 
put new logic behind a flag to make it as safe as possible.
Issue with diagnostics from macro expansion from third-party headers is the one 
of the biggest problem with deployment that we have and it cannot be properly 
fixed with wrappers around clang-tidy.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D90835/new/

https://reviews.llvm.org/D90835

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D103229: [clang] NFC: split HeaderMapTest to have re-usable header map implementation for testing

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin created this revision.
DmitryPolukhin added reviewers: dexonsmith, bruno.
DmitryPolukhin added a project: clang.
DmitryPolukhin requested review of this revision.

NFC changes required for https://reviews.llvm.org/D103142

Test Plan: check-clang


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D103229

Files:
  clang/unittests/Lex/HeaderMapTest.cpp
  clang/unittests/Lex/HeaderMapTestUtils.h

Index: clang/unittests/Lex/HeaderMapTestUtils.h
===
--- /dev/null
+++ clang/unittests/Lex/HeaderMapTestUtils.h
@@ -0,0 +1,100 @@
+//===- unittests/Lex/HeaderMapTestUtils.h - HeaderMap utils ---===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+
+#ifndef LLVM_CLANG_UNITTESTS_LEX_HEADERMAPTESTUTILS_H
+#define LLVM_CLANG_UNITTESTS_LEX_HEADERMAPTESTUTILS_H
+
+#include "clang/Basic/CharInfo.h"
+#include "clang/Lex/HeaderMap.h"
+#include "clang/Lex/HeaderMapTypes.h"
+#include "llvm/Support/SwapByteOrder.h"
+#include 
+
+namespace clang {
+namespace test {
+
+// Lay out a header file for testing.
+template  struct HMapFileMock {
+  HMapHeader Header;
+  HMapBucket Buckets[NumBuckets];
+  unsigned char Bytes[NumBytes];
+
+  void init() {
+memset(this, 0, sizeof(HMapFileMock));
+Header.Magic = HMAP_HeaderMagicNumber;
+Header.Version = HMAP_HeaderVersion;
+Header.NumBuckets = NumBuckets;
+Header.StringsOffset = sizeof(Header) + sizeof(Buckets);
+  }
+
+  void swapBytes() {
+using llvm::sys::getSwappedBytes;
+Header.Magic = getSwappedBytes(Header.Magic);
+Header.Version = getSwappedBytes(Header.Version);
+Header.NumBuckets = getSwappedBytes(Header.NumBuckets);
+Header.StringsOffset = getSwappedBytes(Header.StringsOffset);
+  }
+
+  std::unique_ptr getBuffer() {
+return llvm::MemoryBuffer::getMemBuffer(
+StringRef(reinterpret_cast(this), sizeof(HMapFileMock)),
+"header",
+/* RequresNullTerminator */ false);
+  }
+};
+
+template  struct HMapFileMockMaker {
+  FileTy &File;
+  unsigned SI = 1;
+  unsigned BI = 0;
+  HMapFileMockMaker(FileTy &File) : File(File) {}
+
+  unsigned addString(StringRef S) {
+assert(SI + S.size() + 1 <= sizeof(File.Bytes));
+std::copy(S.begin(), S.end(), File.Bytes + SI);
+auto OldSI = SI;
+SI += S.size() + 1;
+return OldSI;
+  }
+
+  void addBucket(StringRef Str, unsigned Key, unsigned Prefix,
+ unsigned Suffix) {
+addBucket(getHash(Str), Key, Prefix, Suffix);
+  }
+
+  void addBucket(unsigned Hash, unsigned Key, unsigned Prefix,
+ unsigned Suffix) {
+assert(!(File.Header.NumBuckets & (File.Header.NumBuckets - 1)));
+unsigned I = Hash & (File.Header.NumBuckets - 1);
+do {
+  if (!File.Buckets[I].Key) {
+File.Buckets[I].Key = Key;
+File.Buckets[I].Prefix = Prefix;
+File.Buckets[I].Suffix = Suffix;
+++File.Header.NumEntries;
+return;
+  }
+  ++I;
+  I &= File.Header.NumBuckets - 1;
+} while (I != (Hash & (File.Header.NumBuckets - 1)));
+llvm_unreachable("no empty buckets");
+  }
+
+  // The header map hash function.
+  static unsigned getHash(StringRef Str) {
+unsigned Result = 0;
+for (char C : Str)
+  Result += toLowercase(C) * 13;
+return Result;
+  }
+};
+
+} // namespace test
+} // namespace clang
+
+#endif
Index: clang/unittests/Lex/HeaderMapTest.cpp
===
--- clang/unittests/Lex/HeaderMapTest.cpp
+++ clang/unittests/Lex/HeaderMapTest.cpp
@@ -6,89 +6,17 @@
 //
 //===--===//
 
-#include "clang/Basic/CharInfo.h"
-#include "clang/Lex/HeaderMap.h"
-#include "clang/Lex/HeaderMapTypes.h"
+#include "HeaderMapTestUtils.h"
 #include "llvm/ADT/SmallString.h"
-#include "llvm/Support/SwapByteOrder.h"
 #include "gtest/gtest.h"
-#include 
 #include 
 
 using namespace clang;
 using namespace llvm;
+using namespace clang::test;
 
 namespace {
 
-// Lay out a header file for testing.
-template  struct MapFile {
-  HMapHeader Header;
-  HMapBucket Buckets[NumBuckets];
-  unsigned char Bytes[NumBytes];
-
-  void init() {
-memset(this, 0, sizeof(MapFile));
-Header.Magic = HMAP_HeaderMagicNumber;
-Header.Version = HMAP_HeaderVersion;
-Header.NumBuckets = NumBuckets;
-Header.StringsOffset = sizeof(Header) + sizeof(Buckets);
-  }
-
-  void swapBytes() {
-using llvm::sys::getSwappedBytes;
-Header.Magic = getSwappedBytes(Header.Magic);
-Header.Version = getSwappedBytes(Header.Version);
-Header.NumBuckets = getSwappedBytes(Header.NumBuckets);
-Header.StringsOffset = getSwappedBytes(Header.StringsOffset);
-  }
-
-  std::

[PATCH] D103142: [clang][clangd] Use reverse header map lookup in suggestPathToFileForDiagnostics

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 348183.
DmitryPolukhin marked 2 inline comments as done.
DmitryPolukhin added a comment.

Split changes into NFC https://reviews.llvm.org/D103229 and the rest


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D103142/new/

https://reviews.llvm.org/D103142

Files:
  clang/include/clang/Lex/HeaderMap.h
  clang/lib/Lex/HeaderMap.cpp
  clang/lib/Lex/HeaderSearch.cpp
  clang/unittests/Lex/HeaderSearchTest.cpp

Index: clang/unittests/Lex/HeaderSearchTest.cpp
===
--- clang/unittests/Lex/HeaderSearchTest.cpp
+++ clang/unittests/Lex/HeaderSearchTest.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "clang/Lex/HeaderSearch.h"
+#include "HeaderMapTestUtils.h"
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/DiagnosticOptions.h"
 #include "clang/Basic/FileManager.h"
@@ -45,6 +46,21 @@
 Search.AddSearchPath(DL, /*isAngled=*/false);
   }
 
+  void addHeaderMap(llvm::StringRef Filename,
+std::unique_ptr Buf) {
+VFS->addFile(Filename, 0, std::move(Buf), /*User=*/None, /*Group=*/None,
+ llvm::sys::fs::file_type::regular_file);
+auto FE = FileMgr.getFile(Filename, true);
+assert(FE);
+
+// Test class supports only one HMap at a time.
+assert(!HMap);
+HMap = HeaderMap::Create(*FE, FileMgr);
+auto DL =
+DirectoryLookup(HMap.get(), SrcMgr::C_User, /*isFramework=*/false);
+Search.AddSearchPath(DL, /*isAngled=*/false);
+  }
+
   IntrusiveRefCntPtr VFS;
   FileSystemOptions FileMgrOpts;
   FileManager FileMgr;
@@ -55,6 +71,7 @@
   std::shared_ptr TargetOpts;
   IntrusiveRefCntPtr Target;
   HeaderSearch Search;
+  std::unique_ptr HMap;
 };
 
 TEST_F(HeaderSearchTest, NoSearchDir) {
@@ -136,5 +153,31 @@
 "y/z/t.h");
 }
 
+// Helper struct with null terminator character to make MemoryBuffer happy.
+template 
+struct NullTerminatedFile : public FileTy {
+  PaddingTy Padding = 0;
+};
+
+TEST_F(HeaderSearchTest, HeaderMapReverseLookup) {
+  typedef NullTerminatedFile, char> FileTy;
+  FileTy File;
+  File.init();
+
+  test::HMapFileMockMaker Maker(File);
+  auto a = Maker.addString("d.h");
+  auto b = Maker.addString("b/");
+  auto c = Maker.addString("c.h");
+  Maker.addBucket("d.h", a, b, c);
+
+  addHeaderMap("/x/y/z.hmap", File.getBuffer());
+  addSearchDir("/a");
+
+  EXPECT_EQ(Search.suggestPathToFileForDiagnostics("/a/b/c.h",
+   /*WorkingDir=*/"",
+   /*MainFile=*/""),
+"d.h");
+}
+
 } // namespace
 } // namespace clang
Index: clang/lib/Lex/HeaderSearch.cpp
===
--- clang/lib/Lex/HeaderSearch.cpp
+++ clang/lib/Lex/HeaderSearch.cpp
@@ -1834,7 +1834,7 @@
   };
 
   for (unsigned I = 0; I != SearchDirs.size(); ++I) {
-// FIXME: Support this search within frameworks and header maps.
+// FIXME: Support this search within frameworks.
 if (!SearchDirs[I].isNormalDir())
   continue;
 
@@ -1848,6 +1848,18 @@
   if (!BestPrefixLength && CheckDir(path::parent_path(MainFile)) && IsSystem)
 *IsSystem = false;
 
-
-  return path::convert_to_slash(File.drop_front(BestPrefixLength));
+  // Try resolving resulting filaname via reverse search in header maps,
+  // key from header name is user prefered name for the include file.
+  StringRef Filename = File.drop_front(BestPrefixLength);
+  for (unsigned I = 0; I != SearchDirs.size(); ++I) {
+if (SearchDirs[I].isHeaderMap()) {
+  StringRef SpelledFilename =
+  SearchDirs[I].getHeaderMap()->reverseLookupFilename(Filename);
+  if (!SpelledFilename.empty()) {
+Filename = SpelledFilename;
+break;
+  }
+}
+  }
+  return path::convert_to_slash(Filename);
 }
Index: clang/lib/Lex/HeaderMap.cpp
===
--- clang/lib/Lex/HeaderMap.cpp
+++ clang/lib/Lex/HeaderMap.cpp
@@ -240,3 +240,26 @@
 return StringRef(DestPath.begin(), DestPath.size());
   }
 }
+
+StringRef HeaderMapImpl::reverseLookupFilename(StringRef DestPath) const {
+  if (ReverseMap.empty()) {
+const HMapHeader &Hdr = getHeader();
+unsigned NumBuckets = getEndianAdjustedWord(Hdr.NumBuckets);
+for (unsigned i = 0; i != NumBuckets; ++i) {
+  HMapBucket B = getBucket(i);
+  if (B.Key == HMAP_EmptyBucketKey)
+continue;
+
+  Optional Key = getString(B.Key);
+  Optional Prefix = getString(B.Prefix);
+  Optional Suffix = getString(B.Suffix);
+  if (Key && Prefix && Suffix) {
+SmallVector Buf;
+Buf.append(Prefix->begin(), Prefix->end());
+Buf.append(Suffix->begin(), Suffix->end());
+ReverseMap[StringRef(Buf.begin(), Buf.size())] = *Key;
+  }
+}
+  }
+  return Revers

[PATCH] D103142: [clang][clangd] Use reverse header map lookup in suggestPathToFileForDiagnostics

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a subscriber: bkramer.
DmitryPolukhin added a comment.

In D103142#2783665 , @dexonsmith 
wrote:

> Naively, this sounds like it could be a non-trivial tax on build times. But 
> it looks like it's only called in Clang from `Sema::diagnoseMissingImport`, 
> which only happens on error anyway.

Yes, in Clang `suggestPathToFileForDiagnostics` is used only in 
`Sema::diagnoseMissingImport`. In addition to clangd this function is also used 
in clang-include-fixer but new logic should also work there too (add @bkramer 
author of clang-include-fixer). This diff builds reverse index lazy only when 
it is required so it shouldn't normal build speed.




Comment at: clang/lib/Lex/HeaderSearch.cpp:1855
+  for (unsigned I = 0; I != SearchDirs.size(); ++I) {
+if (SearchDirs[I].isHeaderMap()) {
+  StringRef SpelledFilename =

bruno wrote:
> Can we save some dir scanning time by adding this logic to the previous loop? 
> Shouldn't get hard to read if you early `continue` for each failed condition.
It seems that unfortunately no, because we need the previous loop to convert 
absolute path into relative and this loop to convert relative path to key in 
header map. Normal header lookup also happens as two lookups: first lookup uses 
header map to convert key to relative path and then another lookup of the 
relative path.



Comment at: clang/lib/Lex/HeaderSearch.cpp:1855
+  for (unsigned I = 0; I != SearchDirs.size(); ++I) {
+if (SearchDirs[I].isHeaderMap()) {
+  StringRef SpelledFilename =

DmitryPolukhin wrote:
> bruno wrote:
> > Can we save some dir scanning time by adding this logic to the previous 
> > loop? Shouldn't get hard to read if you early `continue` for each failed 
> > condition.
> It seems that unfortunately no, because we need the previous loop to convert 
> absolute path into relative and this loop to convert relative path to key in 
> header map. Normal header lookup also happens as two lookups: first lookup 
> uses header map to convert key to relative path and then another lookup of 
> the relative path.
It seems that unfortunately no, because we need the previous loop to convert 
absolute path into relative and this loop to convert relative path to key in 
header map. Normal header lookup also happens as two lookups: first lookup uses 
header map to convert key to relative path and then another lookup of the 
relative path.



Comment at: clang/lib/Lex/HeaderSearch.cpp:1859
+  if (!SpelledFilename.empty()) {
+Filename = SpelledFilename;
+break;

bruno wrote:
> I guess one of the rationale behind this change is that whatever is consuming 
> your diagnostics is expecting the hmap key entry as an actionable path they 
> could use.
> 
> I wonder whether other consumers of `suggestPathToFileForDiagnostics` expect 
> an actual mapped value or just don't care. If the former, this might be 
> better under some flag.
> 
> @dexonsmith @vsapsai @JDevlieghere @arphaman how does this relate with what 
> you expect out of `suggestPathToFileForDiagnostics`? (If at all).
I think it is safe to change default behaviour without introducing new 
flag/option because I don't expect that anyone really needs value from header 
map search in the source.



Comment at: clang/unittests/Lex/HeaderMapTest.cpp:9
 
-#include "clang/Basic/CharInfo.h"
-#include "clang/Lex/HeaderMap.h"
-#include "clang/Lex/HeaderMapTypes.h"
+#include "HeaderMapTestUtils.h"
 #include "llvm/ADT/SmallString.h"

dexonsmith wrote:
> Splitting this out seems like a great idea, but please split it out to a 
> separate prep commit that's NFC.
Done, https://reviews.llvm.org/D103229


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D103142/new/

https://reviews.llvm.org/D103142

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D103142: [clang][clangd] Use reverse header map lookup in suggestPathToFileForDiagnostics

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 348264.
DmitryPolukhin added a comment.

Fix forgotten comment


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D103142/new/

https://reviews.llvm.org/D103142

Files:
  clang/include/clang/Lex/HeaderMap.h
  clang/lib/Lex/HeaderMap.cpp
  clang/lib/Lex/HeaderSearch.cpp
  clang/unittests/Lex/HeaderSearchTest.cpp

Index: clang/unittests/Lex/HeaderSearchTest.cpp
===
--- clang/unittests/Lex/HeaderSearchTest.cpp
+++ clang/unittests/Lex/HeaderSearchTest.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "clang/Lex/HeaderSearch.h"
+#include "HeaderMapTestUtils.h"
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/DiagnosticOptions.h"
 #include "clang/Basic/FileManager.h"
@@ -45,6 +46,21 @@
 Search.AddSearchPath(DL, /*isAngled=*/false);
   }
 
+  void addHeaderMap(llvm::StringRef Filename,
+std::unique_ptr Buf) {
+VFS->addFile(Filename, 0, std::move(Buf), /*User=*/None, /*Group=*/None,
+ llvm::sys::fs::file_type::regular_file);
+auto FE = FileMgr.getFile(Filename, true);
+assert(FE);
+
+// Test class supports only one HMap at a time.
+assert(!HMap);
+HMap = HeaderMap::Create(*FE, FileMgr);
+auto DL =
+DirectoryLookup(HMap.get(), SrcMgr::C_User, /*isFramework=*/false);
+Search.AddSearchPath(DL, /*isAngled=*/false);
+  }
+
   IntrusiveRefCntPtr VFS;
   FileSystemOptions FileMgrOpts;
   FileManager FileMgr;
@@ -55,6 +71,7 @@
   std::shared_ptr TargetOpts;
   IntrusiveRefCntPtr Target;
   HeaderSearch Search;
+  std::unique_ptr HMap;
 };
 
 TEST_F(HeaderSearchTest, NoSearchDir) {
@@ -136,5 +153,31 @@
 "y/z/t.h");
 }
 
+// Helper struct with null terminator character to make MemoryBuffer happy.
+template 
+struct NullTerminatedFile : public FileTy {
+  PaddingTy Padding = 0;
+};
+
+TEST_F(HeaderSearchTest, HeaderMapReverseLookup) {
+  typedef NullTerminatedFile, char> FileTy;
+  FileTy File;
+  File.init();
+
+  test::HMapFileMockMaker Maker(File);
+  auto a = Maker.addString("d.h");
+  auto b = Maker.addString("b/");
+  auto c = Maker.addString("c.h");
+  Maker.addBucket("d.h", a, b, c);
+
+  addHeaderMap("/x/y/z.hmap", File.getBuffer());
+  addSearchDir("/a");
+
+  EXPECT_EQ(Search.suggestPathToFileForDiagnostics("/a/b/c.h",
+   /*WorkingDir=*/"",
+   /*MainFile=*/""),
+"d.h");
+}
+
 } // namespace
 } // namespace clang
Index: clang/lib/Lex/HeaderSearch.cpp
===
--- clang/lib/Lex/HeaderSearch.cpp
+++ clang/lib/Lex/HeaderSearch.cpp
@@ -1834,7 +1834,7 @@
   };
 
   for (unsigned I = 0; I != SearchDirs.size(); ++I) {
-// FIXME: Support this search within frameworks and header maps.
+// FIXME: Support this search within frameworks.
 if (!SearchDirs[I].isNormalDir())
   continue;
 
@@ -1848,6 +1848,18 @@
   if (!BestPrefixLength && CheckDir(path::parent_path(MainFile)) && IsSystem)
 *IsSystem = false;
 
-
-  return path::convert_to_slash(File.drop_front(BestPrefixLength));
+  // Try resolving resulting filename via reverse search in header maps,
+  // key from header name is user prefered name for the include file.
+  StringRef Filename = File.drop_front(BestPrefixLength);
+  for (unsigned I = 0; I != SearchDirs.size(); ++I) {
+if (SearchDirs[I].isHeaderMap()) {
+  StringRef SpelledFilename =
+  SearchDirs[I].getHeaderMap()->reverseLookupFilename(Filename);
+  if (!SpelledFilename.empty()) {
+Filename = SpelledFilename;
+break;
+  }
+}
+  }
+  return path::convert_to_slash(Filename);
 }
Index: clang/lib/Lex/HeaderMap.cpp
===
--- clang/lib/Lex/HeaderMap.cpp
+++ clang/lib/Lex/HeaderMap.cpp
@@ -240,3 +240,26 @@
 return StringRef(DestPath.begin(), DestPath.size());
   }
 }
+
+StringRef HeaderMapImpl::reverseLookupFilename(StringRef DestPath) const {
+  if (ReverseMap.empty()) {
+const HMapHeader &Hdr = getHeader();
+unsigned NumBuckets = getEndianAdjustedWord(Hdr.NumBuckets);
+for (unsigned i = 0; i != NumBuckets; ++i) {
+  HMapBucket B = getBucket(i);
+  if (B.Key == HMAP_EmptyBucketKey)
+continue;
+
+  Optional Key = getString(B.Key);
+  Optional Prefix = getString(B.Prefix);
+  Optional Suffix = getString(B.Suffix);
+  if (Key && Prefix && Suffix) {
+SmallVector Buf;
+Buf.append(Prefix->begin(), Prefix->end());
+Buf.append(Suffix->begin(), Suffix->end());
+ReverseMap[StringRef(Buf.begin(), Buf.size())] = *Key;
+  }
+}
+  }
+  return ReverseMap.lookup(DestPath);
+}
Index: clang/include/clang/Lex/HeaderMap.h
===

[PATCH] D103229: [clang] NFC: split HeaderMapTest to have re-usable header map implementation for testing

[Dmitry Polukhin via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG37b530a2ea8b: [clang] NFC: split HeaderMapTest to have 
re-usable header map implementation… (authored by DmitryPolukhin).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D103229/new/

https://reviews.llvm.org/D103229

Files:
  clang/unittests/Lex/HeaderMapTest.cpp
  clang/unittests/Lex/HeaderMapTestUtils.h

Index: clang/unittests/Lex/HeaderMapTestUtils.h
===
--- /dev/null
+++ clang/unittests/Lex/HeaderMapTestUtils.h
@@ -0,0 +1,100 @@
+//===- unittests/Lex/HeaderMapTestUtils.h - HeaderMap utils ---===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+
+#ifndef LLVM_CLANG_UNITTESTS_LEX_HEADERMAPTESTUTILS_H
+#define LLVM_CLANG_UNITTESTS_LEX_HEADERMAPTESTUTILS_H
+
+#include "clang/Basic/CharInfo.h"
+#include "clang/Lex/HeaderMap.h"
+#include "clang/Lex/HeaderMapTypes.h"
+#include "llvm/Support/SwapByteOrder.h"
+#include 
+
+namespace clang {
+namespace test {
+
+// Lay out a header file for testing.
+template  struct HMapFileMock {
+  HMapHeader Header;
+  HMapBucket Buckets[NumBuckets];
+  unsigned char Bytes[NumBytes];
+
+  void init() {
+memset(this, 0, sizeof(HMapFileMock));
+Header.Magic = HMAP_HeaderMagicNumber;
+Header.Version = HMAP_HeaderVersion;
+Header.NumBuckets = NumBuckets;
+Header.StringsOffset = sizeof(Header) + sizeof(Buckets);
+  }
+
+  void swapBytes() {
+using llvm::sys::getSwappedBytes;
+Header.Magic = getSwappedBytes(Header.Magic);
+Header.Version = getSwappedBytes(Header.Version);
+Header.NumBuckets = getSwappedBytes(Header.NumBuckets);
+Header.StringsOffset = getSwappedBytes(Header.StringsOffset);
+  }
+
+  std::unique_ptr getBuffer() {
+return llvm::MemoryBuffer::getMemBuffer(
+StringRef(reinterpret_cast(this), sizeof(HMapFileMock)),
+"header",
+/* RequresNullTerminator */ false);
+  }
+};
+
+template  struct HMapFileMockMaker {
+  FileTy &File;
+  unsigned SI = 1;
+  unsigned BI = 0;
+  HMapFileMockMaker(FileTy &File) : File(File) {}
+
+  unsigned addString(StringRef S) {
+assert(SI + S.size() + 1 <= sizeof(File.Bytes));
+std::copy(S.begin(), S.end(), File.Bytes + SI);
+auto OldSI = SI;
+SI += S.size() + 1;
+return OldSI;
+  }
+
+  void addBucket(StringRef Str, unsigned Key, unsigned Prefix,
+ unsigned Suffix) {
+addBucket(getHash(Str), Key, Prefix, Suffix);
+  }
+
+  void addBucket(unsigned Hash, unsigned Key, unsigned Prefix,
+ unsigned Suffix) {
+assert(!(File.Header.NumBuckets & (File.Header.NumBuckets - 1)));
+unsigned I = Hash & (File.Header.NumBuckets - 1);
+do {
+  if (!File.Buckets[I].Key) {
+File.Buckets[I].Key = Key;
+File.Buckets[I].Prefix = Prefix;
+File.Buckets[I].Suffix = Suffix;
+++File.Header.NumEntries;
+return;
+  }
+  ++I;
+  I &= File.Header.NumBuckets - 1;
+} while (I != (Hash & (File.Header.NumBuckets - 1)));
+llvm_unreachable("no empty buckets");
+  }
+
+  // The header map hash function.
+  static unsigned getHash(StringRef Str) {
+unsigned Result = 0;
+for (char C : Str)
+  Result += toLowercase(C) * 13;
+return Result;
+  }
+};
+
+} // namespace test
+} // namespace clang
+
+#endif
Index: clang/unittests/Lex/HeaderMapTest.cpp
===
--- clang/unittests/Lex/HeaderMapTest.cpp
+++ clang/unittests/Lex/HeaderMapTest.cpp
@@ -6,89 +6,17 @@
 //
 //===--===//
 
-#include "clang/Basic/CharInfo.h"
-#include "clang/Lex/HeaderMap.h"
-#include "clang/Lex/HeaderMapTypes.h"
+#include "HeaderMapTestUtils.h"
 #include "llvm/ADT/SmallString.h"
-#include "llvm/Support/SwapByteOrder.h"
 #include "gtest/gtest.h"
-#include 
 #include 
 
 using namespace clang;
 using namespace llvm;
+using namespace clang::test;
 
 namespace {
 
-// Lay out a header file for testing.
-template  struct MapFile {
-  HMapHeader Header;
-  HMapBucket Buckets[NumBuckets];
-  unsigned char Bytes[NumBytes];
-
-  void init() {
-memset(this, 0, sizeof(MapFile));
-Header.Magic = HMAP_HeaderMagicNumber;
-Header.Version = HMAP_HeaderVersion;
-Header.NumBuckets = NumBuckets;
-Header.StringsOffset = sizeof(Header) + sizeof(Buckets);
-  }
-
-  void swapBytes() {
-using llvm::sys::getSwappedBytes;
-Header.Magic = getSwappedBytes(Header.Magic);
-Header.Version = getSwappedBytes(Header.Version);
-Header.NumBuckets = getSwappedBytes(Header.NumBuckets);
-Header.StringsOffset = getSwappedBytes(Header.StringsOff

[PATCH] D103142: [clang][clangd] Use reverse header map lookup in suggestPathToFileForDiagnostics

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 348900.
DmitryPolukhin marked 2 inline comments as done.
DmitryPolukhin added a comment.

Resolved comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D103142/new/

https://reviews.llvm.org/D103142

Files:
  clang/include/clang/Lex/HeaderMap.h
  clang/lib/Lex/HeaderMap.cpp
  clang/lib/Lex/HeaderSearch.cpp
  clang/unittests/Lex/HeaderSearchTest.cpp

Index: clang/unittests/Lex/HeaderSearchTest.cpp
===
--- clang/unittests/Lex/HeaderSearchTest.cpp
+++ clang/unittests/Lex/HeaderSearchTest.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "clang/Lex/HeaderSearch.h"
+#include "HeaderMapTestUtils.h"
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/DiagnosticOptions.h"
 #include "clang/Basic/FileManager.h"
@@ -45,6 +46,21 @@
 Search.AddSearchPath(DL, /*isAngled=*/false);
   }
 
+  void addHeaderMap(llvm::StringRef Filename,
+std::unique_ptr Buf) {
+VFS->addFile(Filename, 0, std::move(Buf), /*User=*/None, /*Group=*/None,
+ llvm::sys::fs::file_type::regular_file);
+auto FE = FileMgr.getFile(Filename, true);
+assert(FE);
+
+// Test class supports only one HMap at a time.
+assert(!HMap);
+HMap = HeaderMap::Create(*FE, FileMgr);
+auto DL =
+DirectoryLookup(HMap.get(), SrcMgr::C_User, /*isFramework=*/false);
+Search.AddSearchPath(DL, /*isAngled=*/false);
+  }
+
   IntrusiveRefCntPtr VFS;
   FileSystemOptions FileMgrOpts;
   FileManager FileMgr;
@@ -55,6 +71,7 @@
   std::shared_ptr TargetOpts;
   IntrusiveRefCntPtr Target;
   HeaderSearch Search;
+  std::unique_ptr HMap;
 };
 
 TEST_F(HeaderSearchTest, NoSearchDir) {
@@ -136,5 +153,31 @@
 "y/z/t.h");
 }
 
+// Helper struct with null terminator character to make MemoryBuffer happy.
+template 
+struct NullTerminatedFile : public FileTy {
+  PaddingTy Padding = 0;
+};
+
+TEST_F(HeaderSearchTest, HeaderMapReverseLookup) {
+  typedef NullTerminatedFile, char> FileTy;
+  FileTy File;
+  File.init();
+
+  test::HMapFileMockMaker Maker(File);
+  auto a = Maker.addString("d.h");
+  auto b = Maker.addString("b/");
+  auto c = Maker.addString("c.h");
+  Maker.addBucket("d.h", a, b, c);
+
+  addHeaderMap("/x/y/z.hmap", File.getBuffer());
+  addSearchDir("/a");
+
+  EXPECT_EQ(Search.suggestPathToFileForDiagnostics("/a/b/c.h",
+   /*WorkingDir=*/"",
+   /*MainFile=*/""),
+"d.h");
+}
+
 } // namespace
 } // namespace clang
Index: clang/lib/Lex/HeaderSearch.cpp
===
--- clang/lib/Lex/HeaderSearch.cpp
+++ clang/lib/Lex/HeaderSearch.cpp
@@ -1834,7 +1834,7 @@
   };
 
   for (unsigned I = 0; I != SearchDirs.size(); ++I) {
-// FIXME: Support this search within frameworks and header maps.
+// FIXME: Support this search within frameworks.
 if (!SearchDirs[I].isNormalDir())
   continue;
 
@@ -1848,6 +1848,19 @@
   if (!BestPrefixLength && CheckDir(path::parent_path(MainFile)) && IsSystem)
 *IsSystem = false;
 
+  // Try resolving resulting filename via reverse search in header maps,
+  // key from header name is user prefered name for the include file.
+  StringRef Filename = File.drop_front(BestPrefixLength);
+  for (unsigned I = 0; I != SearchDirs.size(); ++I) {
+if (!SearchDirs[I].isHeaderMap())
+  continue;
 
-  return path::convert_to_slash(File.drop_front(BestPrefixLength));
+StringRef SpelledFilename =
+SearchDirs[I].getHeaderMap()->reverseLookupFilename(Filename);
+if (!SpelledFilename.empty()) {
+  Filename = SpelledFilename;
+  break;
+}
+  }
+  return path::convert_to_slash(Filename);
 }
Index: clang/lib/Lex/HeaderMap.cpp
===
--- clang/lib/Lex/HeaderMap.cpp
+++ clang/lib/Lex/HeaderMap.cpp
@@ -240,3 +240,27 @@
 return StringRef(DestPath.begin(), DestPath.size());
   }
 }
+
+StringRef HeaderMapImpl::reverseLookupFilename(StringRef DestPath) const {
+  if (!ReverseMap.empty())
+return ReverseMap.lookup(DestPath);
+
+  const HMapHeader &Hdr = getHeader();
+  unsigned NumBuckets = getEndianAdjustedWord(Hdr.NumBuckets);
+  for (unsigned i = 0; i != NumBuckets; ++i) {
+HMapBucket B = getBucket(i);
+if (B.Key == HMAP_EmptyBucketKey)
+  continue;
+
+Optional Key = getString(B.Key);
+Optional Prefix = getString(B.Prefix);
+Optional Suffix = getString(B.Suffix);
+if (Key && Prefix && Suffix) {
+  SmallVector Buf;
+  Buf.append(Prefix->begin(), Prefix->end());
+  Buf.append(Suffix->begin(), Suffix->end());
+  ReverseMap[StringRef(Buf.begin(), Buf.size())] = *Key;
+}
+  }
+  return ReverseMap.lookup(DestPath);
+}
Index: clang/include/cl

[PATCH] D103142: [clang][clangd] Use reverse header map lookup in suggestPathToFileForDiagnostics

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added inline comments.



Comment at: clang/lib/Lex/HeaderMap.cpp:245
+StringRef HeaderMapImpl::reverseLookupFilename(StringRef DestPath) const {
+  if (ReverseMap.empty()) {
+const HMapHeader &Hdr = getHeader();

bruno wrote:
> Please rewrite this to early return in case this isn't empty. 
Done, but it causes a bit of code duplication due to second lookup. Please let 
me know if you thought about rewriting it somehow else.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D103142/new/

https://reviews.llvm.org/D103142

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D103142: [clang][clangd] Use reverse header map lookup in suggestPathToFileForDiagnostics

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 348965.
DmitryPolukhin added a comment.

Rebase + try build on Windows again


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D103142/new/

https://reviews.llvm.org/D103142

Files:
  clang/include/clang/Lex/HeaderMap.h
  clang/lib/Lex/HeaderMap.cpp
  clang/lib/Lex/HeaderSearch.cpp
  clang/unittests/Lex/HeaderSearchTest.cpp

Index: clang/unittests/Lex/HeaderSearchTest.cpp
===
--- clang/unittests/Lex/HeaderSearchTest.cpp
+++ clang/unittests/Lex/HeaderSearchTest.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "clang/Lex/HeaderSearch.h"
+#include "HeaderMapTestUtils.h"
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/DiagnosticOptions.h"
 #include "clang/Basic/FileManager.h"
@@ -45,6 +46,21 @@
 Search.AddSearchPath(DL, /*isAngled=*/false);
   }
 
+  void addHeaderMap(llvm::StringRef Filename,
+std::unique_ptr Buf) {
+VFS->addFile(Filename, 0, std::move(Buf), /*User=*/None, /*Group=*/None,
+ llvm::sys::fs::file_type::regular_file);
+auto FE = FileMgr.getFile(Filename, true);
+assert(FE);
+
+// Test class supports only one HMap at a time.
+assert(!HMap);
+HMap = HeaderMap::Create(*FE, FileMgr);
+auto DL =
+DirectoryLookup(HMap.get(), SrcMgr::C_User, /*isFramework=*/false);
+Search.AddSearchPath(DL, /*isAngled=*/false);
+  }
+
   IntrusiveRefCntPtr VFS;
   FileSystemOptions FileMgrOpts;
   FileManager FileMgr;
@@ -55,6 +71,7 @@
   std::shared_ptr TargetOpts;
   IntrusiveRefCntPtr Target;
   HeaderSearch Search;
+  std::unique_ptr HMap;
 };
 
 TEST_F(HeaderSearchTest, NoSearchDir) {
@@ -136,5 +153,31 @@
 "y/z/t.h");
 }
 
+// Helper struct with null terminator character to make MemoryBuffer happy.
+template 
+struct NullTerminatedFile : public FileTy {
+  PaddingTy Padding = 0;
+};
+
+TEST_F(HeaderSearchTest, HeaderMapReverseLookup) {
+  typedef NullTerminatedFile, char> FileTy;
+  FileTy File;
+  File.init();
+
+  test::HMapFileMockMaker Maker(File);
+  auto a = Maker.addString("d.h");
+  auto b = Maker.addString("b/");
+  auto c = Maker.addString("c.h");
+  Maker.addBucket("d.h", a, b, c);
+
+  addHeaderMap("/x/y/z.hmap", File.getBuffer());
+  addSearchDir("/a");
+
+  EXPECT_EQ(Search.suggestPathToFileForDiagnostics("/a/b/c.h",
+   /*WorkingDir=*/"",
+   /*MainFile=*/""),
+"d.h");
+}
+
 } // namespace
 } // namespace clang
Index: clang/lib/Lex/HeaderSearch.cpp
===
--- clang/lib/Lex/HeaderSearch.cpp
+++ clang/lib/Lex/HeaderSearch.cpp
@@ -1834,7 +1834,7 @@
   };
 
   for (unsigned I = 0; I != SearchDirs.size(); ++I) {
-// FIXME: Support this search within frameworks and header maps.
+// FIXME: Support this search within frameworks.
 if (!SearchDirs[I].isNormalDir())
   continue;
 
@@ -1848,6 +1848,19 @@
   if (!BestPrefixLength && CheckDir(path::parent_path(MainFile)) && IsSystem)
 *IsSystem = false;
 
+  // Try resolving resulting filename via reverse search in header maps,
+  // key from header name is user prefered name for the include file.
+  StringRef Filename = File.drop_front(BestPrefixLength);
+  for (unsigned I = 0; I != SearchDirs.size(); ++I) {
+if (!SearchDirs[I].isHeaderMap())
+  continue;
 
-  return path::convert_to_slash(File.drop_front(BestPrefixLength));
+StringRef SpelledFilename =
+SearchDirs[I].getHeaderMap()->reverseLookupFilename(Filename);
+if (!SpelledFilename.empty()) {
+  Filename = SpelledFilename;
+  break;
+}
+  }
+  return path::convert_to_slash(Filename);
 }
Index: clang/lib/Lex/HeaderMap.cpp
===
--- clang/lib/Lex/HeaderMap.cpp
+++ clang/lib/Lex/HeaderMap.cpp
@@ -240,3 +240,27 @@
 return StringRef(DestPath.begin(), DestPath.size());
   }
 }
+
+StringRef HeaderMapImpl::reverseLookupFilename(StringRef DestPath) const {
+  if (!ReverseMap.empty())
+return ReverseMap.lookup(DestPath);
+
+  const HMapHeader &Hdr = getHeader();
+  unsigned NumBuckets = getEndianAdjustedWord(Hdr.NumBuckets);
+  for (unsigned i = 0; i != NumBuckets; ++i) {
+HMapBucket B = getBucket(i);
+if (B.Key == HMAP_EmptyBucketKey)
+  continue;
+
+Optional Key = getString(B.Key);
+Optional Prefix = getString(B.Prefix);
+Optional Suffix = getString(B.Suffix);
+if (Key && Prefix && Suffix) {
+  SmallVector Buf;
+  Buf.append(Prefix->begin(), Prefix->end());
+  Buf.append(Suffix->begin(), Suffix->end());
+  ReverseMap[StringRef(Buf.begin(), Buf.size())] = *Key;
+}
+  }
+  return ReverseMap.lookup(DestPath);
+}
Index: clang/include/clang/Lex/HeaderMap.h
===

[PATCH] D103142: [clang][clangd] Use reverse header map lookup in suggestPathToFileForDiagnostics

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 349199.
DmitryPolukhin added a comment.

Updated HeaderMapImpl::reverseLookupFilename according to the review comment


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D103142/new/

https://reviews.llvm.org/D103142

Files:
  clang/include/clang/Lex/HeaderMap.h
  clang/lib/Lex/HeaderMap.cpp
  clang/lib/Lex/HeaderSearch.cpp
  clang/unittests/Lex/HeaderSearchTest.cpp

Index: clang/unittests/Lex/HeaderSearchTest.cpp
===
--- clang/unittests/Lex/HeaderSearchTest.cpp
+++ clang/unittests/Lex/HeaderSearchTest.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "clang/Lex/HeaderSearch.h"
+#include "HeaderMapTestUtils.h"
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/DiagnosticOptions.h"
 #include "clang/Basic/FileManager.h"
@@ -45,6 +46,21 @@
 Search.AddSearchPath(DL, /*isAngled=*/false);
   }
 
+  void addHeaderMap(llvm::StringRef Filename,
+std::unique_ptr Buf) {
+VFS->addFile(Filename, 0, std::move(Buf), /*User=*/None, /*Group=*/None,
+ llvm::sys::fs::file_type::regular_file);
+auto FE = FileMgr.getFile(Filename, true);
+assert(FE);
+
+// Test class supports only one HMap at a time.
+assert(!HMap);
+HMap = HeaderMap::Create(*FE, FileMgr);
+auto DL =
+DirectoryLookup(HMap.get(), SrcMgr::C_User, /*isFramework=*/false);
+Search.AddSearchPath(DL, /*isAngled=*/false);
+  }
+
   IntrusiveRefCntPtr VFS;
   FileSystemOptions FileMgrOpts;
   FileManager FileMgr;
@@ -55,6 +71,7 @@
   std::shared_ptr TargetOpts;
   IntrusiveRefCntPtr Target;
   HeaderSearch Search;
+  std::unique_ptr HMap;
 };
 
 TEST_F(HeaderSearchTest, NoSearchDir) {
@@ -136,5 +153,31 @@
 "y/z/t.h");
 }
 
+// Helper struct with null terminator character to make MemoryBuffer happy.
+template 
+struct NullTerminatedFile : public FileTy {
+  PaddingTy Padding = 0;
+};
+
+TEST_F(HeaderSearchTest, HeaderMapReverseLookup) {
+  typedef NullTerminatedFile, char> FileTy;
+  FileTy File;
+  File.init();
+
+  test::HMapFileMockMaker Maker(File);
+  auto a = Maker.addString("d.h");
+  auto b = Maker.addString("b/");
+  auto c = Maker.addString("c.h");
+  Maker.addBucket("d.h", a, b, c);
+
+  addHeaderMap("/x/y/z.hmap", File.getBuffer());
+  addSearchDir("/a");
+
+  EXPECT_EQ(Search.suggestPathToFileForDiagnostics("/a/b/c.h",
+   /*WorkingDir=*/"",
+   /*MainFile=*/""),
+"d.h");
+}
+
 } // namespace
 } // namespace clang
Index: clang/lib/Lex/HeaderSearch.cpp
===
--- clang/lib/Lex/HeaderSearch.cpp
+++ clang/lib/Lex/HeaderSearch.cpp
@@ -1834,7 +1834,7 @@
   };
 
   for (unsigned I = 0; I != SearchDirs.size(); ++I) {
-// FIXME: Support this search within frameworks and header maps.
+// FIXME: Support this search within frameworks.
 if (!SearchDirs[I].isNormalDir())
   continue;
 
@@ -1848,6 +1848,19 @@
   if (!BestPrefixLength && CheckDir(path::parent_path(MainFile)) && IsSystem)
 *IsSystem = false;
 
+  // Try resolving resulting filename via reverse search in header maps,
+  // key from header name is user prefered name for the include file.
+  StringRef Filename = File.drop_front(BestPrefixLength);
+  for (unsigned I = 0; I != SearchDirs.size(); ++I) {
+if (!SearchDirs[I].isHeaderMap())
+  continue;
 
-  return path::convert_to_slash(File.drop_front(BestPrefixLength));
+StringRef SpelledFilename =
+SearchDirs[I].getHeaderMap()->reverseLookupFilename(Filename);
+if (!SpelledFilename.empty()) {
+  Filename = SpelledFilename;
+  break;
+}
+  }
+  return path::convert_to_slash(Filename);
 }
Index: clang/lib/Lex/HeaderMap.cpp
===
--- clang/lib/Lex/HeaderMap.cpp
+++ clang/lib/Lex/HeaderMap.cpp
@@ -240,3 +240,32 @@
 return StringRef(DestPath.begin(), DestPath.size());
   }
 }
+
+StringRef HeaderMapImpl::reverseLookupFilename(StringRef DestPath) const {
+  if (!ReverseMap.empty())
+return ReverseMap.lookup(DestPath);
+
+  const HMapHeader &Hdr = getHeader();
+  unsigned NumBuckets = getEndianAdjustedWord(Hdr.NumBuckets);
+  StringRef RetKey;
+  for (unsigned i = 0; i != NumBuckets; ++i) {
+HMapBucket B = getBucket(i);
+if (B.Key == HMAP_EmptyBucketKey)
+  continue;
+
+Optional Key = getString(B.Key);
+Optional Prefix = getString(B.Prefix);
+Optional Suffix = getString(B.Suffix);
+if (LLVM_LIKELY(Key && Prefix && Suffix)) {
+  SmallVector Buf;
+  Buf.append(Prefix->begin(), Prefix->end());
+  Buf.append(Suffix->begin(), Suffix->end());
+  StringRef Value(Buf.begin(), Buf.size());
+  ReverseMap[Value] = *Key;
+
+  if (

[PATCH] D103142: [clang][clangd] Use reverse header map lookup in suggestPathToFileForDiagnostics

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added inline comments.



Comment at: clang/lib/Lex/HeaderMap.cpp:265
+  }
+  return ReverseMap.lookup(DestPath);
+}

bruno wrote:
> How about something along the lines of:
> 
> ```
> ...
> StringRef Key;
> for (unsigned i = 0; i != NumBuckets; ++i) {
> HMapBucket B = getBucket(i);
> if (B.Key == HMAP_EmptyBucketKey)
>   continue;
> 
> Key = getString(B.Key);
> Optional Prefix = getString(B.Prefix);
> Optional Suffix = getString(B.Suffix);
> if (!Key.empty() && Prefix && Suffix) {
>   SmallVector Buf;
>   Buf.append(Prefix->begin(), Prefix->end());
>   Buf.append(Suffix->begin(), Suffix->end());
>   ReverseMap[StringRef(Buf.begin(), Buf.size())] = Key;
>   break;
> }
> }
> assert(!Key.empty() && "expected valid key");
> return Key;
> ```
> 
> While proposing this change I've noticed that it would keep looking for other 
> buckets even in face of a valid result, so I've added a `break`. Was that 
> intentional? 
Yes, keep iterating over all key-value pairs in the header map is important to 
add all elements to the map for subsequent lookups. If we stop on the current 
match, next lookup will not even try to populate the reverse lookup map 
(because the map is not empty) but not all elements are in the map due to early 
return from previous lookup, so subsequent lookup may not find the match even 
if it exists in the header map. We also cannot expect that DestPath is present 
in the given header map, so we cannot assert if it is missing. This function 
should return empty string as an indication that DestPath is not found.

Updated implementation with the review suggestion, please take another look.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D103142/new/

https://reviews.llvm.org/D103142

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D103142: [clang][clangd] Use reverse header map lookup in suggestPathToFileForDiagnostics

[Dmitry Polukhin via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG178ad93e3f1f: [clang][clangd] Use reverse header map lookup 
in suggestPathToFileForDiagnostics (authored by DmitryPolukhin).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D103142/new/

https://reviews.llvm.org/D103142

Files:
  clang/include/clang/Lex/HeaderMap.h
  clang/lib/Lex/HeaderMap.cpp
  clang/lib/Lex/HeaderSearch.cpp
  clang/unittests/Lex/HeaderSearchTest.cpp

Index: clang/unittests/Lex/HeaderSearchTest.cpp
===
--- clang/unittests/Lex/HeaderSearchTest.cpp
+++ clang/unittests/Lex/HeaderSearchTest.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "clang/Lex/HeaderSearch.h"
+#include "HeaderMapTestUtils.h"
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/DiagnosticOptions.h"
 #include "clang/Basic/FileManager.h"
@@ -45,6 +46,21 @@
 Search.AddSearchPath(DL, /*isAngled=*/false);
   }
 
+  void addHeaderMap(llvm::StringRef Filename,
+std::unique_ptr Buf) {
+VFS->addFile(Filename, 0, std::move(Buf), /*User=*/None, /*Group=*/None,
+ llvm::sys::fs::file_type::regular_file);
+auto FE = FileMgr.getFile(Filename, true);
+assert(FE);
+
+// Test class supports only one HMap at a time.
+assert(!HMap);
+HMap = HeaderMap::Create(*FE, FileMgr);
+auto DL =
+DirectoryLookup(HMap.get(), SrcMgr::C_User, /*isFramework=*/false);
+Search.AddSearchPath(DL, /*isAngled=*/false);
+  }
+
   IntrusiveRefCntPtr VFS;
   FileSystemOptions FileMgrOpts;
   FileManager FileMgr;
@@ -55,6 +71,7 @@
   std::shared_ptr TargetOpts;
   IntrusiveRefCntPtr Target;
   HeaderSearch Search;
+  std::unique_ptr HMap;
 };
 
 TEST_F(HeaderSearchTest, NoSearchDir) {
@@ -136,5 +153,31 @@
 "y/z/t.h");
 }
 
+// Helper struct with null terminator character to make MemoryBuffer happy.
+template 
+struct NullTerminatedFile : public FileTy {
+  PaddingTy Padding = 0;
+};
+
+TEST_F(HeaderSearchTest, HeaderMapReverseLookup) {
+  typedef NullTerminatedFile, char> FileTy;
+  FileTy File;
+  File.init();
+
+  test::HMapFileMockMaker Maker(File);
+  auto a = Maker.addString("d.h");
+  auto b = Maker.addString("b/");
+  auto c = Maker.addString("c.h");
+  Maker.addBucket("d.h", a, b, c);
+
+  addHeaderMap("/x/y/z.hmap", File.getBuffer());
+  addSearchDir("/a");
+
+  EXPECT_EQ(Search.suggestPathToFileForDiagnostics("/a/b/c.h",
+   /*WorkingDir=*/"",
+   /*MainFile=*/""),
+"d.h");
+}
+
 } // namespace
 } // namespace clang
Index: clang/lib/Lex/HeaderSearch.cpp
===
--- clang/lib/Lex/HeaderSearch.cpp
+++ clang/lib/Lex/HeaderSearch.cpp
@@ -1834,7 +1834,7 @@
   };
 
   for (unsigned I = 0; I != SearchDirs.size(); ++I) {
-// FIXME: Support this search within frameworks and header maps.
+// FIXME: Support this search within frameworks.
 if (!SearchDirs[I].isNormalDir())
   continue;
 
@@ -1848,6 +1848,19 @@
   if (!BestPrefixLength && CheckDir(path::parent_path(MainFile)) && IsSystem)
 *IsSystem = false;
 
+  // Try resolving resulting filename via reverse search in header maps,
+  // key from header name is user prefered name for the include file.
+  StringRef Filename = File.drop_front(BestPrefixLength);
+  for (unsigned I = 0; I != SearchDirs.size(); ++I) {
+if (!SearchDirs[I].isHeaderMap())
+  continue;
 
-  return path::convert_to_slash(File.drop_front(BestPrefixLength));
+StringRef SpelledFilename =
+SearchDirs[I].getHeaderMap()->reverseLookupFilename(Filename);
+if (!SpelledFilename.empty()) {
+  Filename = SpelledFilename;
+  break;
+}
+  }
+  return path::convert_to_slash(Filename);
 }
Index: clang/lib/Lex/HeaderMap.cpp
===
--- clang/lib/Lex/HeaderMap.cpp
+++ clang/lib/Lex/HeaderMap.cpp
@@ -240,3 +240,32 @@
 return StringRef(DestPath.begin(), DestPath.size());
   }
 }
+
+StringRef HeaderMapImpl::reverseLookupFilename(StringRef DestPath) const {
+  if (!ReverseMap.empty())
+return ReverseMap.lookup(DestPath);
+
+  const HMapHeader &Hdr = getHeader();
+  unsigned NumBuckets = getEndianAdjustedWord(Hdr.NumBuckets);
+  StringRef RetKey;
+  for (unsigned i = 0; i != NumBuckets; ++i) {
+HMapBucket B = getBucket(i);
+if (B.Key == HMAP_EmptyBucketKey)
+  continue;
+
+Optional Key = getString(B.Key);
+Optional Prefix = getString(B.Prefix);
+Optional Suffix = getString(B.Suffix);
+if (LLVM_LIKELY(Key && Prefix && Suffix)) {
+  SmallVector Buf;
+  Buf.append(Prefix->begin(), Prefix->end());
+  Buf.append(Suffix->begin(), Suffix->end());
+  StringRef Value(Buf.begin(), Buf.

[PATCH] D103142: [clang][clangd] Use reverse header map lookup in suggestPathToFileForDiagnostics

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

@bruno and @dexonsmith thank you for the review!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D103142/new/

https://reviews.llvm.org/D103142

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D76594: [clang][AST] Support AST files larger than 512M

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added inline comments.



Comment at: clang/lib/Serialization/ASTWriter.cpp:1908
+uint64_t Offset = Stream.GetCurrentBitNo() - SLocEntryOffsetsBase;
+assert((Offset >> 32) == 0 && "SLocEntry offset too large");
+SLocEntryOffsets.push_back(Offset);

JackKemp99 wrote:
> clangd crashes here:
> {F15446669}
@JackKemp99 It means that you have a source file that has number of source 
location that exceeds 32-bit offset. Do you have a reproducer that you can 
share?

Is it something Boost related? There is a discussion about too many source 
location with Boost 
https://lists.llvm.org/pipermail/cfe-dev/2021-February/067599.html Your example 
may not exceed amount in of SourceLocation IDs but exceeds lower bound on 
offsets during preamble generation.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D76594/new/

https://reviews.llvm.org/D76594

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D61989: [clang-tidy] enable modernize-concat-nested-namespaces on header files

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin added a comment.

@shixiao do you have plans to keep working on this diff? If not, do you have 
any objections if I send for review a similar diff that will eliminate the 
check `Sources.isInMainFile` and will rely on generic header filtration 
mechanism in clang-tidy?

We have faced the same issue on our codebase and would like to make this check 
works on headers too.

In D61989#2367012 , @njames93 wrote:

> Tbh clang-tidy already handles disabling warnings for header files, so some 
> of the code here isn't needed

+1 to use generic header filtration mechanism and avoid adding new option for 
this particular check. I tried to find the reason why `Sources.isInMainFile` 
was added but it looks like it was not even discussed during initial codereview 
D52136 

Everyone please speak up if there is a good reason to limit this check to the 
main file only.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D61989/new/

https://reviews.llvm.org/D61989

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D103722: [clang] NFC: test for undefined behaviour in RawComment::getFormattedText()

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin created this revision.
DmitryPolukhin added reviewers: teemperor, obruns, bruno.
DmitryPolukhin requested review of this revision.
Herald added a project: clang.

This diff adds testcase for the issue fixed in https://reviews.llvm.org/D77468
but regression test was not added in the diff. On Clang 9 it caused
crash in cland during code completion.

Test Plan: check-clang-unit


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D103722

Files:
  clang/unittests/AST/CommentTextTest.cpp


Index: clang/unittests/AST/CommentTextTest.cpp
===
--- clang/unittests/AST/CommentTextTest.cpp
+++ clang/unittests/AST/CommentTextTest.cpp
@@ -124,4 +124,11 @@
   // clang-format on
 }
 
+TEST_F(CommentTextTest, EmptyFormattedText) {
+  // Test that empty formatted text doesn't cause crash.
+  auto ExpectedOutput = "";
+  auto Formatted = formatComment("//!<");
+  EXPECT_EQ(ExpectedOutput, Formatted);
+}
+
 } // namespace clang


Index: clang/unittests/AST/CommentTextTest.cpp
===
--- clang/unittests/AST/CommentTextTest.cpp
+++ clang/unittests/AST/CommentTextTest.cpp
@@ -124,4 +124,11 @@
   // clang-format on
 }
 
+TEST_F(CommentTextTest, EmptyFormattedText) {
+  // Test that empty formatted text doesn't cause crash.
+  auto ExpectedOutput = "";
+  auto Formatted = formatComment("//!<");
+  EXPECT_EQ(ExpectedOutput, Formatted);
+}
+
 } // namespace clang
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D103722: [clang] NFC: test for undefined behaviour in RawComment::getFormattedText()

[Dmitry Polukhin via Phabricator via cfe-commits]

DmitryPolukhin updated this revision to Diff 350205.
DmitryPolukhin added a comment.

Fix clang-tidy warning


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D103722/new/

https://reviews.llvm.org/D103722

Files:
  clang/unittests/AST/CommentTextTest.cpp


Index: clang/unittests/AST/CommentTextTest.cpp
===
--- clang/unittests/AST/CommentTextTest.cpp
+++ clang/unittests/AST/CommentTextTest.cpp
@@ -124,4 +124,11 @@
   // clang-format on
 }
 
+TEST_F(CommentTextTest, EmptyFormattedText) {
+  // Test that empty formatted text doesn't cause crash.
+  const char *ExpectedOutput = "";
+  auto Formatted = formatComment("//!<");
+  EXPECT_EQ(ExpectedOutput, Formatted);
+}
+
 } // namespace clang


Index: clang/unittests/AST/CommentTextTest.cpp
===
--- clang/unittests/AST/CommentTextTest.cpp
+++ clang/unittests/AST/CommentTextTest.cpp
@@ -124,4 +124,11 @@
   // clang-format on
 }
 
+TEST_F(CommentTextTest, EmptyFormattedText) {
+  // Test that empty formatted text doesn't cause crash.
+  const char *ExpectedOutput = "";
+  auto Formatted = formatComment("//!<");
+  EXPECT_EQ(ExpectedOutput, Formatted);
+}
+
 } // namespace clang
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D103722: [clang] NFC: test for undefined behaviour in RawComment::getFormattedText()

[Dmitry Polukhin via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rGaa0d7179bbb3: [clang] NFC: test for undefined behaviour in 
RawComment::getFormattedText() (authored by DmitryPolukhin).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D103722/new/

https://reviews.llvm.org/D103722

Files:
  clang/unittests/AST/CommentTextTest.cpp


Index: clang/unittests/AST/CommentTextTest.cpp
===
--- clang/unittests/AST/CommentTextTest.cpp
+++ clang/unittests/AST/CommentTextTest.cpp
@@ -124,4 +124,11 @@
   // clang-format on
 }
 
+TEST_F(CommentTextTest, EmptyFormattedText) {
+  // Test that empty formatted text doesn't cause crash.
+  const char *ExpectedOutput = "";
+  auto Formatted = formatComment("//!<");
+  EXPECT_EQ(ExpectedOutput, Formatted);
+}
+
 } // namespace clang


Index: clang/unittests/AST/CommentTextTest.cpp
===
--- clang/unittests/AST/CommentTextTest.cpp
+++ clang/unittests/AST/CommentTextTest.cpp
@@ -124,4 +124,11 @@
   // clang-format on
 }
 
+TEST_F(CommentTextTest, EmptyFormattedText) {
+  // Test that empty formatted text doesn't cause crash.
+  const char *ExpectedOutput = "";
+  auto Formatted = formatComment("//!<");
+  EXPECT_EQ(ExpectedOutput, Formatted);
+}
+
 } // namespace clang
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits