[PATCH] D83660: [analyzer] Fix a crash for dereferencing an empty llvm::Optional variable in SMTConstraintManager.h.

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: NoQ, dcoughlin.
OikawaKirie added a project: clang.
Herald added subscribers: cfe-commits, ASDenysPetrov, Charusso, dkrupp, 
donat.nagy, Szelethus, mikhail.ramalho, a.sidorin, szepet, baloghadamsoftware, 
xazax.hun.

The first crash reported in the bug report 44338 
.

> Condition `!isSat.hasValue() || isNotSat.getValue()` here should be 
> `!isNotSat.hasValue() || isNotSat.getValue()`.
>  `getValue` here crashed when we used the static analyzer to analyze 
> postgresql-12.0.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D83660

Files:
  clang/include/clang/StaticAnalyzer/Core/PathSensitive/SMTConstraintManager.h


Index: 
clang/include/clang/StaticAnalyzer/Core/PathSensitive/SMTConstraintManager.h
===
--- clang/include/clang/StaticAnalyzer/Core/PathSensitive/SMTConstraintManager.h
+++ clang/include/clang/StaticAnalyzer/Core/PathSensitive/SMTConstraintManager.h
@@ -147,7 +147,7 @@
   Solver->addConstraint(NotExp);
 
   Optional isNotSat = Solver->check();
-  if (!isSat.hasValue() || isNotSat.getValue())
+  if (!isNotSat.hasValue() || isNotSat.getValue())
 return nullptr;
 
   // This is the only solution, store it


Index: clang/include/clang/StaticAnalyzer/Core/PathSensitive/SMTConstraintManager.h
===
--- clang/include/clang/StaticAnalyzer/Core/PathSensitive/SMTConstraintManager.h
+++ clang/include/clang/StaticAnalyzer/Core/PathSensitive/SMTConstraintManager.h
@@ -147,7 +147,7 @@
   Solver->addConstraint(NotExp);
 
   Optional isNotSat = Solver->check();
-  if (!isSat.hasValue() || isNotSat.getValue())
+  if (!isNotSat.hasValue() || isNotSat.getValue())
 return nullptr;
 
   // This is the only solution, store it
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D83660: [analyzer] Fix a crash for dereferencing an empty llvm::Optional variable in SMTConstraintManager.h.

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

> do you accidentally have a test case to reproduce the crash

@NoQ I am now working with the reporter of this bug to make a simple test case 
to trigger the crash.

> looks like a good opportunity for `SmartPtr` checker and `llvm::Optional`!

@vsavchenko It seems not difficult to write a checker specifically for 
`llvm::Optional`. However, the root reason for this crash is an assertion 
failure. It seems to be better to write a checker to report assertion failures.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D83660/new/

https://reviews.llvm.org/D83660



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D83660: [analyzer] Fix a crash for dereferencing an empty llvm::Optional variable in SMTConstraintManager.h.

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

> Might in the future I would spend some time on it - we will see.

@steakhal My boss always asks me about how to improve the performance for SMT 
solver based constraint solving, on both the engine side and the SMT solver 
side. If there is anything that our research group can do, you are free to 
contact us.

> BTW nice catch & fix.

Thank you. This crash was discovered by one of our team members when we were 
testing our CSA based tool. If the patch gets merged in the future, I'd like to 
use his name and email for the commit.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D83660/new/

https://reviews.llvm.org/D83660



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85105: [doxygen] Fix bad doxygen results for BugReporterVisitors.h

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: NoQ, vsavchenko.
OikawaKirie added a project: clang.
Herald added subscribers: cfe-commits, Charusso.
OikawaKirie requested review of this revision.

Because of `{@code x}` will trigger a Doxygen bug. And as far as I am 
thinking, the bug may be matching the close brace with the open brace of the 
namespace declaration (`namespace clang {` or `namespace ento {`). With this 
patch, we can bypass the problem and successfully generate documents for the 
classes in BugReporterVisitors.h.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85105

Files:
  clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h


Index: clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
===
--- clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
+++ clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
@@ -53,7 +53,7 @@
   /// Note that this function does *not* get run on the very last node
   /// of the report, as the PathDiagnosticPiece associated with the
   /// last node should be unique.
-  /// Use {@code getEndPath} to customize the note associated with the report
+  /// Use `getEndPath` to customize the note associated with the report
   /// end instead.
   ///
   /// The last parameter can be used to register a new visitor with the given


Index: clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
===
--- clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
+++ clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
@@ -53,7 +53,7 @@
   /// Note that this function does *not* get run on the very last node
   /// of the report, as the PathDiagnosticPiece associated with the
   /// last node should be unique.
-  /// Use {@code getEndPath} to customize the note associated with the report
+  /// Use `getEndPath` to customize the note associated with the report
   /// end instead.
   ///
   /// The last parameter can be used to register a new visitor with the given
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85105: [doxygen] Fix bad doxygen results for BugReporterVisitors.h

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 284952.
OikawaKirie marked an inline comment as done.
OikawaKirie added a comment.

Now it has a link to the `getEndPath`.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85105/new/

https://reviews.llvm.org/D85105

Files:
  clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h


Index: clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
===
--- clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
+++ clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
@@ -53,7 +53,7 @@
   /// Note that this function does *not* get run on the very last node
   /// of the report, as the PathDiagnosticPiece associated with the
   /// last node should be unique.
-  /// Use {@code getEndPath} to customize the note associated with the report
+  /// Use @ref getEndPath to customize the note associated with the report
   /// end instead.
   ///
   /// The last parameter can be used to register a new visitor with the given


Index: clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
===
--- clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
+++ clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
@@ -53,7 +53,7 @@
   /// Note that this function does *not* get run on the very last node
   /// of the report, as the PathDiagnosticPiece associated with the
   /// last node should be unique.
-  /// Use {@code getEndPath} to customize the note associated with the report
+  /// Use @ref getEndPath to customize the note associated with the report
   /// end instead.
   ///
   /// The last parameter can be used to register a new visitor with the given
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85105: [doxygen] Fix bad doxygen results for BugReporterVisitors.h

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 284991.
OikawaKirie added a comment.

Ok, I got it. I will pay attention to this in the future submits.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85105/new/

https://reviews.llvm.org/D85105

Files:
  clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h


Index: clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
===
--- clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
+++ clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
@@ -53,7 +53,7 @@
   /// Note that this function does *not* get run on the very last node
   /// of the report, as the PathDiagnosticPiece associated with the
   /// last node should be unique.
-  /// Use {@code getEndPath} to customize the note associated with the report
+  /// Use \ref getEndPath to customize the note associated with the report
   /// end instead.
   ///
   /// The last parameter can be used to register a new visitor with the given


Index: clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
===
--- clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
+++ clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h
@@ -53,7 +53,7 @@
   /// Note that this function does *not* get run on the very last node
   /// of the report, as the PathDiagnosticPiece associated with the
   /// last node should be unique.
-  /// Use {@code getEndPath} to customize the note associated with the report
+  /// Use \ref getEndPath to customize the note associated with the report
   /// end instead.
   ///
   /// The last parameter can be used to register a new visitor with the given
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.
Herald added a project: All.

Thanks, @keith.

I agree with @keith to commit this patch without using on-demand-parsing 
through cc1.
As this patch has nothing to do with the target triple issue we found.

In the current version, I use the PCH file to load the external TU.
And it seems to work fine on my system and on the Windows CI.

IMO, maybe we can just leave a FIXME or something else in the test case and 
commit this patch to fix the original problem we want to fix.
(of course, re-submit to rerun the test case on Linux)
What do you think? @steakhal


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 416861.
OikawaKirie edited the summary of this revision.
OikawaKirie added a comment.

- Add FIXME in test case.
- Add discourse topic link in summary.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

Files:
  clang/docs/analyzer/user-docs/CrossTranslationUnit.rst
  clang/include/clang/Basic/DiagnosticCrossTUKinds.td
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-import.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/Inputs/ctu-other.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.ast-dump.txt
  
clang/test/Analysis/Inputs/plist-macros-with-expansion-ctu.c.externalDefMap.txt
  clang/test/Analysis/ctu-inherited-default-ctor.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/func-mapping-test.cpp
  clang/unittests/CrossTU/CrossTranslationUnitTest.cpp

Index: clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
===
--- clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
+++ clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
@@ -61,7 +61,7 @@
 ASSERT_FALSE(llvm::sys::fs::createTemporaryFile("index", "txt", IndexFD,
 IndexFileName));
 llvm::ToolOutputFile IndexFile(IndexFileName, IndexFD);
-IndexFile.os() << "c:@F@f#I# " << ASTFileName << "\n";
+IndexFile.os() << "9:c:@F@f#I# " << ASTFileName << "\n";
 IndexFile.os().flush();
 EXPECT_TRUE(llvm::sys::fs::exists(IndexFileName));
 
Index: clang/test/Analysis/func-mapping-test.cpp
===
--- clang/test/Analysis/func-mapping-test.cpp
+++ clang/test/Analysis/func-mapping-test.cpp
@@ -3,10 +3,10 @@
 int f(int) {
   return 0;
 }
-// CHECK-DAG: c:@F@f#I#
+// CHECK-DAG: 9:c:@F@f#I#
 
 extern const int x = 5;
-// CHECK-DAG: c:@x
+// CHECK-DAG: 4:c:@x
 
 // Non-const variables should not be collected.
 int y = 5;
@@ -18,29 +18,29 @@
   int a;
 };
 extern S const s = {.a = 2};
-// CHECK-DAG: c:@s
+// CHECK-DAG: 4:c:@s
 
 struct SF {
   const int a;
 };
 SF sf = {.a = 2};
-// CHECK-DAG: c:@sf
+// CHECK-DAG: 5:c:@sf
 
 struct SStatic {
   static const int a = 4;
 };
 const int SStatic::a;
-// CHECK-DAG: c:@S@SStatic@a
+// CHECK-DAG: 14:c:@S@SStatic@a
 
 extern int const arr[5] = { 0, 1 };
-// CHECK-DAG: c:@arr
+// CHECK-DAG: 6:c:@arr
 
 union U {
   const int a;
   const unsigned int b;
 };
 U u = {.a = 6};
-// CHECK-DAG: c:@u
+// CHECK-DAG: 4:c:@u
 
 // No USR can be generated for this.
 // Check for no crash in this case.
@@ -48,3 +48,15 @@
   float uf;
   const int ui;
 };
+
+void f(int (*)(char));
+void f(bool (*)(char));
+
+struct G {
+  G() {
+f([](char) -> int { return 42; });
+// CHECK-DAG: 41:c:@S@G@F@G#@Sa@F@operator int (*)(char)#1
+f([](char) -> bool { return true; });
+// CHECK-DAG: 42:c:@S@G@F@G#@Sa@F@operator bool (*)(char)#1
+  }
+};
Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,41 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: echo '41:c:@S@G@F@G#@Sa@F@operator void (*)(int)#1 %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: echo '38:c:@S@G@F@G#@Sa@F@operator void (*)()#1 %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: echo '14:c:@F@importee# %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: %clang_cc1 -emit-pch %/S/Inputs/ctu-lookup-name-with-space.cpp -o %t/importee.ast
+
+// RUN: cd %t
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config display-ctu-progress=true \
+// RUN:   -verify %s 2>&1 | FileCheck %s
+
+// CHECK: CTU loaded AST file
+
+// FIXME: In this test case, we cannot use the on-demand-parsing approach to
+//load the external TU.
+//
+//In the Darwin system, the target triple is determined by the driver,
+//rather than using the default one like other systems. However, when
+//using bare `clang -cc1`, the adjustment is not done, which cannot
+//match the one loaded with on-demand-parsing (adjusted triple).
+//We bypass this problem by loading AST files, whose target triple is
+//also unadjusted when generated via `clang -cc1 -emit-pch`.
+//
+//Refer to: https://discourse.llvm.org/t/60762
+//
+//This is also the reason why the test case of D75665 (introducing
+//the on-demand-parsing feature) is enabled only on Linux.
+
+void importee();
+
+void trigger() {
+  // Call an external function to trigger the parsing process of CTU index.
+  // Refer to file Inputs/c

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG9f90254286dc: [analyzer][ctu] Fix wrong 'multiple 
definitions' errors caused by space… (authored by OikawaKirie).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

Files:
  clang/docs/analyzer/user-docs/CrossTranslationUnit.rst
  clang/include/clang/Basic/DiagnosticCrossTUKinds.td
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-import.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/Inputs/ctu-other.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.ast-dump.txt
  
clang/test/Analysis/Inputs/plist-macros-with-expansion-ctu.c.externalDefMap.txt
  clang/test/Analysis/ctu-inherited-default-ctor.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/func-mapping-test.cpp
  clang/unittests/CrossTU/CrossTranslationUnitTest.cpp

Index: clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
===
--- clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
+++ clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
@@ -61,7 +61,7 @@
 ASSERT_FALSE(llvm::sys::fs::createTemporaryFile("index", "txt", IndexFD,
 IndexFileName));
 llvm::ToolOutputFile IndexFile(IndexFileName, IndexFD);
-IndexFile.os() << "c:@F@f#I# " << ASTFileName << "\n";
+IndexFile.os() << "9:c:@F@f#I# " << ASTFileName << "\n";
 IndexFile.os().flush();
 EXPECT_TRUE(llvm::sys::fs::exists(IndexFileName));
 
Index: clang/test/Analysis/func-mapping-test.cpp
===
--- clang/test/Analysis/func-mapping-test.cpp
+++ clang/test/Analysis/func-mapping-test.cpp
@@ -3,10 +3,10 @@
 int f(int) {
   return 0;
 }
-// CHECK-DAG: c:@F@f#I#
+// CHECK-DAG: 9:c:@F@f#I#
 
 extern const int x = 5;
-// CHECK-DAG: c:@x
+// CHECK-DAG: 4:c:@x
 
 // Non-const variables should not be collected.
 int y = 5;
@@ -18,29 +18,29 @@
   int a;
 };
 extern S const s = {.a = 2};
-// CHECK-DAG: c:@s
+// CHECK-DAG: 4:c:@s
 
 struct SF {
   const int a;
 };
 SF sf = {.a = 2};
-// CHECK-DAG: c:@sf
+// CHECK-DAG: 5:c:@sf
 
 struct SStatic {
   static const int a = 4;
 };
 const int SStatic::a;
-// CHECK-DAG: c:@S@SStatic@a
+// CHECK-DAG: 14:c:@S@SStatic@a
 
 extern int const arr[5] = { 0, 1 };
-// CHECK-DAG: c:@arr
+// CHECK-DAG: 6:c:@arr
 
 union U {
   const int a;
   const unsigned int b;
 };
 U u = {.a = 6};
-// CHECK-DAG: c:@u
+// CHECK-DAG: 4:c:@u
 
 // No USR can be generated for this.
 // Check for no crash in this case.
@@ -48,3 +48,15 @@
   float uf;
   const int ui;
 };
+
+void f(int (*)(char));
+void f(bool (*)(char));
+
+struct G {
+  G() {
+f([](char) -> int { return 42; });
+// CHECK-DAG: 41:c:@S@G@F@G#@Sa@F@operator int (*)(char)#1
+f([](char) -> bool { return true; });
+// CHECK-DAG: 42:c:@S@G@F@G#@Sa@F@operator bool (*)(char)#1
+  }
+};
Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,41 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: echo '41:c:@S@G@F@G#@Sa@F@operator void (*)(int)#1 %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: echo '38:c:@S@G@F@G#@Sa@F@operator void (*)()#1 %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: echo '14:c:@F@importee# %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: %clang_cc1 -emit-pch %/S/Inputs/ctu-lookup-name-with-space.cpp -o %t/importee.ast
+
+// RUN: cd %t
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config display-ctu-progress=true \
+// RUN:   -verify %s 2>&1 | FileCheck %s
+
+// CHECK: CTU loaded AST file
+
+// FIXME: In this test case, we cannot use the on-demand-parsing approach to
+//load the external TU.
+//
+//In the Darwin system, the target triple is determined by the driver,
+//rather than using the default one like other systems. However, when
+//using bare `clang -cc1`, the adjustment is not done, which cannot
+//match the one loaded with on-demand-parsing (adjusted triple).
+//We bypass this problem by loading AST files, whose target triple is
+//also unadjusted when generated via `clang -cc1 -emit-pch`.
+//
+//Refer to: https://discourse.llvm.org/t/60762
+//
+//This is also the reason why the test case of D75665 (introducing
+//the on-demand-parsing feature) is enabled only on Linux.
+
+void importee();
+
+void trigger() {
+  // Call an external function to trigger the parsin

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

I think I have found out the reason for the problem, and it proved my guesses.

When executing the test case of the static analyzer, we usually use 
`%clang_analyze_cc1` as the entry, which is `%clang_cc1 -analyze`.  And we 
usually do not set a target triple, as it is not required by the analyzer. 
However, things are complicated in Darwin Unix.

In Darwin, the default target triple is `ARCH-apple-darwinXX.XX.XX`, where 
`ARCH` is the architecture (e.g. `x86_64`) and `XX.XX.XX` is the version of the 
Darwin system. And the default target triple will be then adjusted to 
`ARCH-apple-SYSTEMXX.XX.XX` by the 
`Driver::Darwin::ComputeEffectiveClangTriple` in driver related code, where 
`SYSTEM` can be `watchos`, `tvos`, `ios` and `macosx`.

In the clang driver, the adjusted target triple will be passed to the new cc1 
process; whereas in tooling and `ASTUnit::LoadFromCommandLine`, the adjusted 
target triple will be used to generate cc1 arguments to create 
`CompilerInvocation`. But when executing bare `clang -cc1`, if the target 
triple argument is not provided, it remains the default 
`ARCH-apple-darwinXX.XX.XX`. And this is the reason for the conflict.

The CTU on-demand-parsing mechanism uses `ASTUnit::LoadFromCommandLine` to load 
external ASTs. The tool `clang-check` uses clang tooling to parse the entry 
file. Therefore, both target triples are the adjusted ones, which can be 
matched. And so is the driver (`clang --analyze ...`). But not the bare 
`%clang_cc1`, its target triple is the default one.

---

Let's have a look at the simple example, suppose externalDefMap.txt and 
invocations.yaml are generated correctly.

input.cc:

  void bar();
  void foo() { bar(); }

importee.cc:

  void bar() { }



---

Using driver:

  clang -v --analyze input.cc -Xanalyzer -analyzer-config -Xanalyzer 
experimental-enable-naive-ctu-analysis=true,ctu-dir=.

Output: OK, adjusted to adjusted

  (in-process)
  /path/to/clang-15 -cc1 -triple x86_64-apple-macosx10.15.0 ...



---

Using clang-check:

  clang-check -analyze input.cc -- -v -Xanalyzer -analyzer-config -Xanalyzer 
experimental-enable-naive-ctu-analysis=true,ctu-dir=.

Output: OK, adjusted to adjusted

  clang Invocation:
   "/path/to/clang-tool" "-cc1" "-triple" "x86_64-apple-macosx10.15.0"



---

Using cc1:

  clang -cc1 -v -analyze input.cc  -analyzer-checker=core -analyzer-config 
experimental-enable-naive-ctu-analysis=true,ctu-dir=.

Output: ERROR, default to adjusted

  warning: imported AST from 'importee.cc' had been generated for a different 
target, current: x86_64-apple-darwin19.6.0, imported: 
x86_64-apple-macosx10.15.0 [-Wctu]



---

What do you think is a better way to fix this problem? @gamesh411 @steakhal 
@martong 
Using `clang-check` to run the test case seems to be a good way to overcome the 
problem, but the problem still exists.
However, IMO it is not a good idea to make clang cc1 to adjust the default 
target triple manually.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102614: [index] Add support for type of pointers to class members

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

ping.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102614/new/

https://reviews.llvm.org/D102614

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D116924: [clang-extdef-mapping] Allow clang-extdef-mapping tool to output customized filename for each index entry

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

ping


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D116924/new/

https://reviews.llvm.org/D116924

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92160: [clang] Fix wrong FDs are used for files with same name in Tooling

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D92160#3485400 , @Kale wrote:

> In D92160#2449507 , @dexonsmith 
> wrote:
>
>> But it's possible we don't need this. If it's safe for us to update the 
>> tests and make `FileManager::getFileRef` always canonicalize to an absolute 
>> path, that would definitely be cleaner. `FileManager::makeAbsolute` can use 
>> whatever the FS's CWD is at the time of query... nice and clean.
>
> I've tried to fix this bug through this way but found that "making 
> `FileManager::getFileRef` always canonicalize to an absolute path" was pretty 
> hard, because there are many test cases using states stored in FileManager 
> and assuming that they are all in relative form. Besides, the assumption that 
> "the CWD won't change" indeed is correct by design and works fine for single 
> compiler execution. We might not change that unless for a strong necessity.
>
> So I personally believed that this bug is caused by libtooling's incorrect 
> use of `FileManger`. I plan to fix it by implementing a class like 
> `LibtoolingFileManager`, or `AbsoluteFileManager`, which extends the original 
> `FileManager` and using abs path as key to store the status of seen files, 
> but only used for libtooling.
>
> I will try to upload a patch later to verify the possibility.

Thanks a lot for taking over this bug. Please remember to close this patch if 
your new patch can solve this problem.
I will forward all the comments from the original author (Hao Zhang) if he has 
any suggestions for your new patch.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D92160/new/

https://reviews.llvm.org/D92160

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D129737: [analyzer] Fixing SVal::getType returns Null Type for NonLoc::ConcreteInt in boolean type

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: vsavchenko, NoQ, ASDenysPetrov.
OikawaKirie added a project: clang.
Herald added subscribers: steakhal, manas, martong, dkrupp, donat.nagy, 
Szelethus, mikhail.ramalho, a.sidorin, szepet, baloghadamsoftware, xazax.hun.
Herald added a project: All.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

In method `TypeRetrievingVisitor::VisitConcreteInt`, 
`ASTContext::getIntTypeForBitwidth` is used to get the type for `ConcreteInt`s.
However, the getter in ASTContext cannot handle the boolean type with the bit 
width of 1, which will make method `SVal::getType` return a Null `Type`.
In this patch, a check for this case is added to fix this problem by returning 
the bool type directly when the bit width is 1.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D129737

Files:
  clang/lib/StaticAnalyzer/Core/SVals.cpp
  clang/unittests/StaticAnalyzer/SValTest.cpp


Index: clang/unittests/StaticAnalyzer/SValTest.cpp
===
--- clang/unittests/StaticAnalyzer/SValTest.cpp
+++ clang/unittests/StaticAnalyzer/SValTest.cpp
@@ -161,6 +161,7 @@
 void foo() {
   int x = 42;
   int *y = nullptr;
+  bool z = true;
 })") {
   SVal X = getByName("x");
   ASSERT_FALSE(X.getType(Context).isNull());
@@ -170,6 +171,10 @@
   ASSERT_FALSE(Y.getType(Context).isNull());
   expectSameSignAndBitWidth(Context.getUIntPtrType(), Y.getType(Context),
 Context);
+
+  SVal Z = getByName("z");
+  ASSERT_FALSE(Z.getType(Context).isNull());
+  EXPECT_EQ(Context.BoolTy, Z.getType(Context));
 }
 
 SVAL_TEST(GetLocAsIntType, R"(
Index: clang/lib/StaticAnalyzer/Core/SVals.cpp
===
--- clang/lib/StaticAnalyzer/Core/SVals.cpp
+++ clang/lib/StaticAnalyzer/Core/SVals.cpp
@@ -136,6 +136,8 @@
   }
   template  QualType VisitConcreteInt(ConcreteInt CI) {
 const llvm::APSInt &Value = CI.getValue();
+if (1 == Value.getBitWidth())
+  return Context.BoolTy;
 return Context.getIntTypeForBitwidth(Value.getBitWidth(), 
Value.isSigned());
   }
   QualType VisitLocConcreteInt(loc::ConcreteInt CI) {


Index: clang/unittests/StaticAnalyzer/SValTest.cpp
===
--- clang/unittests/StaticAnalyzer/SValTest.cpp
+++ clang/unittests/StaticAnalyzer/SValTest.cpp
@@ -161,6 +161,7 @@
 void foo() {
   int x = 42;
   int *y = nullptr;
+  bool z = true;
 })") {
   SVal X = getByName("x");
   ASSERT_FALSE(X.getType(Context).isNull());
@@ -170,6 +171,10 @@
   ASSERT_FALSE(Y.getType(Context).isNull());
   expectSameSignAndBitWidth(Context.getUIntPtrType(), Y.getType(Context),
 Context);
+
+  SVal Z = getByName("z");
+  ASSERT_FALSE(Z.getType(Context).isNull());
+  EXPECT_EQ(Context.BoolTy, Z.getType(Context));
 }
 
 SVAL_TEST(GetLocAsIntType, R"(
Index: clang/lib/StaticAnalyzer/Core/SVals.cpp
===
--- clang/lib/StaticAnalyzer/Core/SVals.cpp
+++ clang/lib/StaticAnalyzer/Core/SVals.cpp
@@ -136,6 +136,8 @@
   }
   template  QualType VisitConcreteInt(ConcreteInt CI) {
 const llvm::APSInt &Value = CI.getValue();
+if (1 == Value.getBitWidth())
+  return Context.BoolTy;
 return Context.getIntTypeForBitwidth(Value.getBitWidth(), Value.isSigned());
   }
   QualType VisitLocConcreteInt(loc::ConcreteInt CI) {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D129737: [analyzer] Fixing SVal::getType returns Null Type for NonLoc::ConcreteInt in boolean type

[Ella Ma via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG32fe1a4be95c: [analyzer] Fixing SVal::getType returns Null 
Type for NonLoc::ConcreteInt in… (authored by OikawaKirie).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D129737/new/

https://reviews.llvm.org/D129737

Files:
  clang/lib/StaticAnalyzer/Core/SVals.cpp
  clang/unittests/StaticAnalyzer/SValTest.cpp


Index: clang/unittests/StaticAnalyzer/SValTest.cpp
===
--- clang/unittests/StaticAnalyzer/SValTest.cpp
+++ clang/unittests/StaticAnalyzer/SValTest.cpp
@@ -161,6 +161,7 @@
 void foo() {
   int x = 42;
   int *y = nullptr;
+  bool z = true;
 })") {
   SVal X = getByName("x");
   ASSERT_FALSE(X.getType(Context).isNull());
@@ -170,6 +171,10 @@
   ASSERT_FALSE(Y.getType(Context).isNull());
   expectSameSignAndBitWidth(Context.getUIntPtrType(), Y.getType(Context),
 Context);
+
+  SVal Z = getByName("z");
+  ASSERT_FALSE(Z.getType(Context).isNull());
+  EXPECT_EQ(Context.BoolTy, Z.getType(Context));
 }
 
 SVAL_TEST(GetLocAsIntType, R"(
Index: clang/lib/StaticAnalyzer/Core/SVals.cpp
===
--- clang/lib/StaticAnalyzer/Core/SVals.cpp
+++ clang/lib/StaticAnalyzer/Core/SVals.cpp
@@ -136,6 +136,8 @@
   }
   template  QualType VisitConcreteInt(ConcreteInt CI) {
 const llvm::APSInt &Value = CI.getValue();
+if (1 == Value.getBitWidth())
+  return Context.BoolTy;
 return Context.getIntTypeForBitwidth(Value.getBitWidth(), 
Value.isSigned());
   }
   QualType VisitLocConcreteInt(loc::ConcreteInt CI) {


Index: clang/unittests/StaticAnalyzer/SValTest.cpp
===
--- clang/unittests/StaticAnalyzer/SValTest.cpp
+++ clang/unittests/StaticAnalyzer/SValTest.cpp
@@ -161,6 +161,7 @@
 void foo() {
   int x = 42;
   int *y = nullptr;
+  bool z = true;
 })") {
   SVal X = getByName("x");
   ASSERT_FALSE(X.getType(Context).isNull());
@@ -170,6 +171,10 @@
   ASSERT_FALSE(Y.getType(Context).isNull());
   expectSameSignAndBitWidth(Context.getUIntPtrType(), Y.getType(Context),
 Context);
+
+  SVal Z = getByName("z");
+  ASSERT_FALSE(Z.getType(Context).isNull());
+  EXPECT_EQ(Context.BoolTy, Z.getType(Context));
 }
 
 SVAL_TEST(GetLocAsIntType, R"(
Index: clang/lib/StaticAnalyzer/Core/SVals.cpp
===
--- clang/lib/StaticAnalyzer/Core/SVals.cpp
+++ clang/lib/StaticAnalyzer/Core/SVals.cpp
@@ -136,6 +136,8 @@
   }
   template  QualType VisitConcreteInt(ConcreteInt CI) {
 const llvm::APSInt &Value = CI.getValue();
+if (1 == Value.getBitWidth())
+  return Context.BoolTy;
 return Context.getIntTypeForBitwidth(Value.getBitWidth(), Value.isSigned());
   }
   QualType VisitLocConcreteInt(loc::ConcreteInt CI) {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85105: [doxygen] Fix bad doxygen results for BugReporterVisitors.h

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

@vsavchenko @NoQ 
The patch has been updated as required. Is there anything I need to do with the 
patch?




Comment at: 
clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h:56
   /// last node should be unique.
-  /// Use {@code getEndPath} to customize the note associated with the report
+  /// Use `getEndPath` to customize the note associated with the report
   /// end instead.

vsavchenko wrote:
> Maybe it is better to create a reference to this function using the [[ 
> https://www.doxygen.nl/manual/commands.html#cmdref | \ref ]] command?
Yes, using `\ref` looks better. I will submit it again. To be similar to the 
original code, I will use `@ref getEndPath` instead.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85105/new/

https://reviews.llvm.org/D85105

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D83660: [analyzer] Fix a crash for dereferencing an empty llvm::Optional variable in SMTConstraintManager.h.

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

We have tried to trigger the crash with the original project where the crash 
was encountered. But the problem is we cannot trigger the crash with the 
project, and we have lost all the previous records about this crash. Besides, 
both I and the bug reporter himself are now working on our own research right 
now. We have very limited time working on this patch in recent months. We are 
now planning to just make a simple regression test case to trigger the crash 
during our spire time. Sorry for the delay.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D83660/new/

https://reviews.llvm.org/D83660

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D83660: [analyzer] Fix a crash for dereferencing an empty llvm::Optional variable in SMTConstraintManager.h.

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

After reviewing the code of this snippet, I think it would be very difficult to 
make a regression test case for the crash, as far as what I know about Z3 and 
SMT solvers.

First of all, all calls to `Solver->check()` will return `true` for sat, 
`false` for unsat, and empty for a timeout.
On line 132, the manager invokes Z3 for solving the constraints under the 
current state.
On line 137, the manager invokes Z3 for getting a model (a valid result) 
satisfying the constraints.
On line 141, the manager adds another constraint to exclude the model gotten 
from the model.
On line 149, the manager invokes Z3 for solving the excluded constraint.
In summary, the manager will first solver the constraint for a result of the 
queried symbolic variable. If there is a valid value, it excludes the value and 
solves again to check whether it is the only valid result.

To trigger the crash, we need to construct a group of constraints that are sat. 
Then, we need to exclude a valid value for a symbolic variable and make the 
constraints lead to a **timeout** (rather than an unsat). Simple linear 
constraints have very few chances to lead to a timeout. I tried to create a 
group of constraints from 
https://stackoverflow.com/questions/20536435/z3-what-might-be-the-reason-for-timeout,
 which are a group of non-linear unsat constraints that can trigger a timeout 
(under a timeout of 10 seconds). However, I have not successfully made one, as 
it has too many things to do with mathematics. And my SMT solver colleagues 
also think it is quite difficult to make one.

As far as I am thinking, it is also very tricky to trigger a constraint solver 
timeout. Since it can be impacted by which version of the constraint solver is 
used, how much time is set for the timeout, how fast your computer runs, and so 
on. Chances are that even if I reproduced the crash with a test case, the same 
test case may not work on your computer.

Is it possible to hack the SMT solver creator to make a mock solver for 
triggering the problem?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D83660/new/

https://reviews.llvm.org/D83660

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92160: [clang] Fix wrong FDs are used for files with same name in Tooling

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D92160#2443405 , @dexonsmith wrote:

> 



In D92160#2443413 , @dexonsmith wrote:

> 

Replies on my own:

> Can you tell me more about the scenario this happens?

You can first manually try the regression test case in this patch.
As the `%t/b/config.h` is empty, there should not be any reading operations to 
the file. However, as the length of `%t/a/config.h` is wrongly used for 
`%t/b/config.h` because of the bug, the file does be read, and some warnings 
about reading `'\0'` characters are generated during compilation.

> Why is the FileManager being reused after the working directory change?

It seems to be the problem of ClangTool. All the input source files are handled 
one by one with the same FileManager instance from the ClangTool::Files. (See 
the comments in the file `clang/lib/Tooling/Tooling.cpp`)

> I'm also wondering about an alternate solution that optimizes for the common 
> case, such as the following:
>
> Don't specifically track relative paths / absolute paths.
> Add an API, something like: dropRelativePaths(), which drops all 
> {File,Directory}Entry{,Ref} that are somehow based on relative paths. This 
> can forward to a similar function the stat cache if necessary.
> Update clients to call that if the working directory changes (you'd want to 
> do it only if it actually changed, not always).
> This would keep FileManager from having to growing complexity to track 
> working directory changes.
>
> I realize this may not be clear; what I'm suggesting is that 
> dropRelativePaths() would visit all entries, check if they're relative, and 
> drop them if so.

I am not quite clear with the fixes in this patch, I will forward the replies 
to these questions from the original author Hao Zhang  to 
you when he replies to me.

Thanks




Comment at: clang/lib/Tooling/Tooling.cpp:545
   LLVM_DEBUG({ llvm::dbgs() << "Processing: " << File << ".\n"; });
   ToolInvocation Invocation(std::move(CommandLine), Action, Files.get(),
 PCHContainerOps);

The CWD changes for each input file, but the same FileManager instance is used.

And relative paths are used to index the file entries.

When there are two files in different CWDs having the same relative path, they 
will be mixed up.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D92160/new/

https://reviews.llvm.org/D92160

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92160: [clang] Fix wrong FDs are used for files with same name in Tooling

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

Replies from the original author Hao Zhang 
(Sorry for the wrong email address in the previous reply.)

--

Thanks for your reply.

I noticed this problem when I was using clang-check to analyze a project which 
had a `compile_commands.json`. A simplified test case is provided in the patch.

As far as I know this problem only happens in `clang/lib/Tooling/Tooling.cpp`. 
(as @OikawaKirie has explained)

However, it is more likely a bug in `FileManager`, since `FileManager` is not 
aware that whether the current working directory has been changed or not.

One solution could be to add an API, such as 
`notifyCurrentWorkingDirectoryChange`. Clients call if necessary. (Similar to 
the 3rd one in your alternative solution). Now the question is what to do in 
`notifyCurrentWorkingDirectoryChange`:

- In my solution, I use a pretty straightforward approach, which is to have an 
individual cache (for `FileEntry` and any other related things) for each 
working directory, and switch caches between working directories.



- In your alternative solution, if I standerstand it correctly, 
`dropRelativePaths` is called in `notifyCurrentWorkingDirectoryChange`. All 
those `FileEntry` which are based on relative paths are dropped from the cache. 
I think this is a good idea, especially for keeping `FileManager` simple. 
However I'm not sure how many files are based on relative paths. If 
`dropRelativePaths` drops too many files and the working directory is switched 
back to a previous one (this could happen when analyzing with a 
`compile_commands.json`), it might result in many cache misses, thus involving 
more system calls to open files.

There was an another solution I have ever tried, without things like 
`notifyCurrentWorkingDirectoryChange`. The solution is still straightforward, 
which is to use absolute path (instead of filename) to query/store a 
`FileEntry` from/to the cache. Actually I have tried this before but it ended 
up with lots of test cases failed. I don't know where I did wrong. If you think 
this approach is okay, I will continue working on this, and it might take some 
time.

I'm still not sure which solution is more suitable, or if there is a better 
one. Any suggestions are welcome!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D92160/new/

https://reviews.llvm.org/D92160

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92160: [clang] Fix wrong FDs are used for files with same name in Tooling

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

Replies from the original author Hao Zhang 

--

> ...split the data structures between relative and absolute paths. The 
> existing data structures would only store absolute paths, but there are new 
> ones for relative paths that can be cached and swapped out.

I like the idea that using one cache for absolute paths and multiple caches for 
relative paths. Better than my approach.

> Another point is that FileManager takes the working directory at construction 
> time via FileSystemOptions. There's a FIXME to remove that argument, but (as 
> you've seen) the assumption is baked in that the CWD doesn't change. I think 
> the current patch may not completely fix it, since you can't modify 
> FileSystemOptions.

It's the point. It is the assumption that the CWD won't change leads to the 
problem. I agree with you that `FileSystemOptions` should be removed and grab 
the CWD directly from the VFS.

> Another option here is to assume the VFS never changes directories (no need 
> to store the CWD) ... Then if ClangTool wants to change directories, it would 
> be expected to maintain an appropriate set of VFS layers.

IIUC, does this mean that clients should create new `Filesystem`s when they 
change directories? It looks to me that it might conflict with the design of 
`Filesystem` since it already provides an API to change the CWD.

Another option (which I mentioned in the previous post) is to use absolute 
paths to query the cache, something like this:

  llvm::ErrorOr
  FileManager::getFile(StringRef Filename, bool openFile, bool CacheFailure) {
// Froce using the absolute path to query the cache
llvm::SmallString<128> AbsPath(Filename);
makeAbsolutePath(AbsPath);
auto Result = getFileRef(AbsPath.str(), openFile, CacheFailure);
if (Result)
  return &Result->getFileEntry();
return llvm::errorToErrorCode(Result.takeError());
  }

It is more likely a defensive workaround to this problem. It might not be a 
good solution to the real problem (the assumption that the CWD won't change). 
The above code resulted in failures of some test cases, I haven't looked 
closely into why they failed. In my point of view, `Filename` should only be 
used as the key to query the cache. Logically, if I change it into the absolute 
path, those test cases should pass as usual.

Anyway, your approach (remove `FileSystemOptions`, one cache for absolute 
paths, multiple caches for relative paths) looks good to me.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D92160/new/

https://reviews.llvm.org/D92160

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92160: [clang] Fix wrong FDs are used for files with same name in Tooling

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

Replies from the original author Hao Zhang 

---

Sorry for replying late.

> I think it would be good to understand why the tests failed. Are some clients 
> relying on the name matching the query? If so, why?

I'm currently working on this. Maybe try making `FileManager::getFileRef` 
always canonicalize to an absolute path, as you mentioned.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D92160/new/

https://reviews.llvm.org/D92160

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92634: [Analyzer] Diagnose signed integer overflow

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

I think it could be better to implement this check with a checker on 
`PreStmt` and so on. And IMO, checkers have enough 
functionalities to report these problems.

Besides, the return value should be the exact value computed from the two 
integers, even unknown, rather than undefined. As the developers may overflow 
an integer on purpose.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D92634/new/

https://reviews.llvm.org/D92634

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D91410: [llvm][clang][mlir] Add checks for the return values from Target::createXXX to prevent protential null deref

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: serge-sans-paille, ahmedcharles, niravd, echristo, 
maskray0, pcc, tejohnson, espindola, courbet, andreadb, epastor, enderby, 
lhames, whchung, ftynse.
OikawaKirie added projects: LLVM, clang, MLIR.
Herald added subscribers: llvm-commits, cfe-commits, rdzhabarov, tatianashp, 
msifontes, jurahul, Kayjukh, grosul1, Joonsoo, liufengdb, aartbik, lucyrfox, 
mgester, arpith-jacob, csigg, antiagainst, shauheen, rriddle, mehdi_amini, 
rupprecht, steven_wu, gbedwell, hiraditya.
Herald added a reviewer: jhenderson.
Herald added a reviewer: MaskRay.
OikawaKirie requested review of this revision.
Herald added subscribers: stephenneuendorffer, nicolasvasilache.
Herald added a reviewer: herhut.

All these potential null pointer dereferences are reported by my static 
analyzer for null smart pointer dereferences, which has a different 
implementation from `alpha.cplusplus.SmartPtr`.

The checked pointers in this patch are initialized by Target::createXXX 
functions. When the creator function pointer is not correctly set, a null 
pointer will be returned, or the creator function may originally return a null 
pointer.

Some of them may not make sense as they may be checked before entering the 
function, but I fixed them all in this patch. I submit this fix because 1) 
similar checks are found in some other places in the LLVM codebase for the same 
return value of the function; and, 2) some of the pointers are dereferenced 
before they are checked, which may definitely trigger a null pointer 
dereference if the return value is nullptr.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D91410

Files:
  clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp
  clang/tools/driver/cc1as_main.cpp
  llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
  llvm/lib/CodeGen/AsmPrinter/AsmPrinterInlineAsm.cpp
  llvm/lib/CodeGen/LLVMTargetMachine.cpp
  llvm/lib/CodeGen/ParallelCG.cpp
  llvm/lib/LTO/LTOBackend.cpp
  llvm/lib/LTO/LTOCodeGenerator.cpp
  llvm/lib/LTO/LTOModule.cpp
  llvm/lib/LTO/ThinLTOCodeGenerator.cpp
  llvm/lib/Target/AArch64/AArch64AsmPrinter.cpp
  llvm/tools/llvm-exegesis/lib/LlvmState.cpp
  llvm/tools/llvm-exegesis/llvm-exegesis.cpp
  llvm/tools/llvm-mc/llvm-mc.cpp
  llvm/tools/llvm-mca/llvm-mca.cpp
  llvm/tools/llvm-ml/llvm-ml.cpp
  llvm/tools/llvm-objdump/MachODump.cpp
  llvm/tools/llvm-rtdyld/llvm-rtdyld.cpp
  mlir/lib/Conversion/GPUCommon/ConvertKernelFuncToBlob.cpp
  mlir/lib/ExecutionEngine/ExecutionEngine.cpp

Index: mlir/lib/ExecutionEngine/ExecutionEngine.cpp
===
--- mlir/lib/ExecutionEngine/ExecutionEngine.cpp
+++ mlir/lib/ExecutionEngine/ExecutionEngine.cpp
@@ -131,6 +131,10 @@
 
   std::unique_ptr machine(target->createTargetMachine(
   targetTriple, cpu, features.getString(), {}, {}));
+  if (!machine) {
+errs() << "Unable to create target machine\n";
+return true;
+  }
   llvmModule->setDataLayout(machine->createDataLayout());
   llvmModule->setTargetTriple(targetTriple);
   return false;
Index: mlir/lib/Conversion/GPUCommon/ConvertKernelFuncToBlob.cpp
===
--- mlir/lib/Conversion/GPUCommon/ConvertKernelFuncToBlob.cpp
+++ mlir/lib/Conversion/GPUCommon/ConvertKernelFuncToBlob.cpp
@@ -131,6 +131,10 @@
 }
 targetMachine.reset(target->createTargetMachine(triple.str(), targetChip,
 features, {}, {}));
+if (targetMachine == nullptr) {
+  emitError(loc, "connot initialize target machine");
+  return {};
+}
   }
 
   llvmModule.setDataLayout(targetMachine->createDataLayout());
Index: llvm/tools/llvm-rtdyld/llvm-rtdyld.cpp
===
--- llvm/tools/llvm-rtdyld/llvm-rtdyld.cpp
+++ llvm/tools/llvm-rtdyld/llvm-rtdyld.cpp
@@ -764,6 +764,8 @@
 ErrorAndExit("Unable to create disassembler!");
 
   std::unique_ptr MII(TheTarget->createMCInstrInfo());
+  if (!MII)
+ErrorAndExit("Unable to create target instruction info!");
 
   std::unique_ptr InstPrinter(
   TheTarget->createMCInstPrinter(Triple(TripleName), 0, *MAI, *MII, *MRI));
Index: llvm/tools/llvm-objdump/MachODump.cpp
===
--- llvm/tools/llvm-objdump/MachODump.cpp
+++ llvm/tools/llvm-objdump/MachODump.cpp
@@ -7200,10 +7200,32 @@
   else
 MachOMCPU = MCPU;
 
+#define CHECK_TARGET_INFO_CREATION(NAME)  \
+  do {\
+if (!NAME) {  \
+  WithColor::error(errs(), "llvm-objdump")\
+  << "couldn't initialize disassembler for target " << TripleName \
+  << '\n';\
+  return; 

[PATCH] D91540: [preprocessor] Assertions on the inferrable null pointers in Preprocessor befere dereference

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: dnsampaio, rsmith, lattner, ilya-biryukov.
OikawaKirie added a project: clang.
Herald added a subscriber: cfe-commits.
OikawaKirie requested review of this revision.

This patch asserts on some smart pointers that can be statically inferred to be 
null pointers. For a smart pointer, when its nullability is checked, and the 
null branch can reach a dereference, we think it is a potential null pointer 
dereference. In this patch, I added an assertion before each reported 
dereference.

- clang/lib/Lex/PPDirectives.cpp:441, inferred from the loop body: e.g. line 
481, check and continue
- clang/lib/Lex/PPDirectives.cpp:1385, inferred from line 911: check `CurLexer`
- clang/lib/Lex/PPDirectives.cpp:1723, inferred from line 911: check `CurLexer`
- clang/lib/Lex/PPDirectives.cpp:2540, inferred from line 911: check `CurLexer`
- clang/lib/Lex/PPLexerChange.cpp:320, inferred from line 311: check `CurLexer`
- clang/lib/Lex/Preprocessor.cpp:1226, inferred from recomputeCurLexerKind line 
383: check `CurLexer`

Besides, the analyzer also reports the dereference in function 
SkipTokensWhileUsingPCH and Lex. I think these two reported dereference cannot 
actually happen, but I also add the asserts to make sure the value is correct.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D91540

Files:
  clang/lib/Lex/PPDirectives.cpp
  clang/lib/Lex/PPLexerChange.cpp
  clang/lib/Lex/Preprocessor.cpp


Index: clang/lib/Lex/Preprocessor.cpp
===
--- clang/lib/Lex/Preprocessor.cpp
+++ clang/lib/Lex/Preprocessor.cpp
@@ -637,9 +637,12 @@
 (CurLexer && CurLexer->getFileID() == getPredefinesFileID());
 switch (CurLexerKind) {
 case CLK_Lexer:
+  assert(CurLexer && "CurLexer is empty while CurLexerKind is CLK_Lexer.");
   CurLexer->Lex(Tok);
  break;
 case CLK_TokenLexer:
+  assert(CurTokenLexer &&
+  "CurTokenLexer is empty while CurLexerKind is CLK_TokenLexer.");
   CurTokenLexer->Lex(Tok);
   break;
 case CLK_CachingLexer:
@@ -895,9 +898,12 @@
   do {
 switch (CurLexerKind) {
 case CLK_Lexer:
+  assert(CurLexer && "CurLexer is empty while CurLexerKind is CLK_Lexer.");
   ReturnedToken = CurLexer->Lex(Result);
   break;
 case CLK_TokenLexer:
+  assert(CurTokenLexer &&
+  "CurTokenLexer is empty while CurLexerKind is CLK_TokenLexer.");
   ReturnedToken = CurTokenLexer->Lex(Result);
   break;
 case CLK_CachingLexer:
@@ -1217,6 +1223,7 @@
   assert(TheModuleLoader.HadFatalFailure &&
  "This should be an early exit only to a fatal error");
   Result.setKind(tok::eof);
+  assert(CurLexer);
   CurLexer->cutOffLexing();
   EnterTokens(Suffix);
   return true;
Index: clang/lib/Lex/PPLexerChange.cpp
===
--- clang/lib/Lex/PPLexerChange.cpp
+++ clang/lib/Lex/PPLexerChange.cpp
@@ -317,6 +317,7 @@
 Module *M = LeaveSubmodule(/*ForPragma*/true);
 
 Result.startToken();
+assert(CurLexer && "Got EOF but no current lexer set!");
 const char *EndPos = getCurLexerEndPos();
 CurLexer->BufferPtr = EndPos;
 CurLexer->FormTokenWithChars(Result, EndPos, tok::annot_module_end);
Index: clang/lib/Lex/PPDirectives.cpp
===
--- clang/lib/Lex/PPDirectives.cpp
+++ clang/lib/Lex/PPDirectives.cpp
@@ -438,6 +438,7 @@
   }
   SourceLocation endLoc;
   while (true) {
+assert(CurLexer);
 CurLexer->Lex(Tok);
 
 if (Tok.is(tok::code_completion)) {
@@ -1381,6 +1382,8 @@
 ///
 void Preprocessor::HandleUserDiagnosticDirective(Token &Tok,
  bool isWarning) {
+  assert(CurLexer);
+
   // Read the rest of the line raw.  We do this because we don't want macros
   // to be expanded and we don't require that the tokens be valid preprocessing
   // tokens.  For example, this is allowed: "#warning `   'foo".  GCC does
@@ -1717,6 +1720,7 @@
"This should be an early exit only to a fatal error");
 TheModuleLoader.HadFatalFailure = true;
 IncludeTok.setKind(tok::eof);
+assert(CurLexer);
 CurLexer->cutOffLexing();
 return;
   }
@@ -2533,6 +2537,7 @@
   // Ensure we consume the rest of the macro body if errors occur.
   auto _ = llvm::make_scope_exit([&]() {
 // The flag indicates if we are still waiting for 'eod'.
+assert(CurLexer);
 if (CurLexer->ParsingPreprocessorDirective)
   DiscardUntilEndOfDirective();
   });


Index: clang/lib/Lex/Preprocessor.cpp
===
--- clang/lib/Lex/Preprocessor.cpp
+++ clang/lib/Lex/Preprocessor.cpp
@@ -637,9 +637,12 @@
 (CurLexer && CurLexer->getFileID() == getPredefinesFileID());
 switch (CurLexerKind) {
 case CLK_Lexer:
+  assert(CurLexer &

[PATCH] D91410: [llvm][clang][mlir] Add checks for the return values from Target::createXXX to prevent protential null deref

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D91410#2400018 , @tejohnson wrote:

> 



> ... the new checking is a mix of assert and fatal errors. Is that intended?

No. The added checks are based on the checks on other calls to the 
`Target::createXXX` functions in this file or other related files. If there are 
any fatal errors nearby (e.g. llvm/lib/LTO/ThinLTOCodeGenerator.cpp:581 vs 
569), the check will be a  fatal error; and if there are any assertions (e.g. 
llvm/lib/CodeGen/LLVMTargetMachine.cpp:43,45,52 vs 60) or the calls are never 
checked (e.g. llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp:300), the added check 
will be an assertion.

> If these are not likely due to user input issues, then perhaps they should 
> all be assert so that they are compiled out in release compilers?

Since all these problems are reported by my static analyzer, I do not really 
know whether these checked pointers will actually be null when the code is 
executed. And I did not try to dynamically execute the program to check the 
problems either. But chances are that if the creator callbacks are not properly 
set or the called creator functions returns nullptr, the problem will happen. 
In my opinion, these problems may only happen during development. Therefore, I 
believe asserts can be sufficient to diagnose the problems.

If you think it would be better to use assertions instead of fatal errors, I 
will make an update on all `llvm/lib/xxx` files (or maybe all files). But 
before that, I'd prefer waiting for the replies from other reviewers on the 
remaining parts of the patch and making an update for all the suggestions.

Thank you.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D91410/new/

https://reviews.llvm.org/D91410

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D91844: [llvm][clang] Add checks for the smart pointers with the possibility to be null

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: aaron.ballman, avl, mehdi_amini, ilya-biryukov, 
tejohnson, jansvoboda11.
OikawaKirie added projects: LLVM, clang.
Herald added subscribers: llvm-commits, cfe-commits, dexonsmith, steven_wu, 
hiraditya.
OikawaKirie requested review of this revision.

All these potential null pointer dereferences are reported by my static 
analyzer for null smart pointer dereferences, which has a different 
implementation from `alpha.cplusplus.SmartPtr`.

The checked pointers are:

- The return value of `createArgument` in file 
clang/utils/TableGen/ClangAttrEmitter.cpp. Although there are a lot of checks 
in the function, nullptr is still allowed to be returned. As a recursive 
function it is, I added checks to all the places where the function is called.
- The local variable `Unit` in function `DWARFLinker::loadClangModule` in file 
llvm/lib/DWARFLinker/DWARFLinker.cpp. If the variable is not set in the loop 
below its definition, it will trigger a null pointer dereference after the loop.
- The local variable `Index` in function `ThinLTOCodeGenerator::run` in file 
llvm/lib/LTO/ThinLTOCodeGenerator.cpp. When function 
`ThinLTOCodeGenerator::linkCombinedIndex` returns nullptr, the pointer `Index` 
will be null and be dereferenced below.
- The parameter variable `Buffer` in function `InMemoryFileSystem::addFile` in 
file llvm/lib/Support/VirtualFileSystem.cpp. The assertion in this function 
(`assert(!(HardLinkTarget && Buffer))`) only checks whether these two 
parameters can both be non-null. But It can be inferred that both pointers can 
be null together. A null `Buffer` pointer can be dereferenced without a check.
- The return value of function `ModuleLazyLoaderCache::operator` in file 
llvm/tools/llvm-link/llvm-link.cpp. According to the bug report of my static 
analyzer, the std::function variable `ModuleLazyLoaderCache::createLazyModule` 
points to function `loadFile`, which may return nullptr when error. And the 
pointer is returned as a reference without a check to the return value.
- The local variable `Ret` in function `MarshallingKindInfo::create` in file 
`llvm/utils/TableGen/OptParserEmitter.cpp`. If not all MarshallingKind's are 
handled, variable `Ret` will be kept as nullptr.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D91844

Files:
  clang/utils/TableGen/ClangAttrEmitter.cpp
  llvm/lib/DWARFLinker/DWARFLinker.cpp
  llvm/lib/LTO/ThinLTOCodeGenerator.cpp
  llvm/lib/Support/VirtualFileSystem.cpp
  llvm/tools/llvm-link/llvm-link.cpp
  llvm/utils/TableGen/OptParserEmitter.cpp

Index: llvm/utils/TableGen/OptParserEmitter.cpp
===
--- llvm/utils/TableGen/OptParserEmitter.cpp
+++ llvm/utils/TableGen/OptParserEmitter.cpp
@@ -183,6 +183,7 @@
 Ret = std::make_unique(
 R, *R.getValueAsDef("NegOption"));
   }
+  assert(Ret && "Unknown MarshallingInfoKind.");
 
   Ret->ShouldAlwaysEmit = R.getValueAsBit("ShouldAlwaysEmit");
   Ret->KeyPath = R.getValueAsString("KeyPath");
Index: llvm/tools/llvm-link/llvm-link.cpp
===
--- llvm/tools/llvm-link/llvm-link.cpp
+++ llvm/tools/llvm-link/llvm-link.cpp
@@ -243,8 +243,10 @@
 Module &ModuleLazyLoaderCache::operator()(const char *argv0,
   const std::string &Identifier) {
   auto &Module = ModuleMap[Identifier];
-  if (!Module)
+  if (!Module) {
 Module = createLazyModule(argv0, Identifier);
+assert(Module && "Failed to create lazy module!");
+  }
   return *Module;
 }
 } // anonymous namespace
Index: llvm/lib/Support/VirtualFileSystem.cpp
===
--- llvm/lib/Support/VirtualFileSystem.cpp
+++ llvm/lib/Support/VirtualFileSystem.cpp
@@ -732,6 +732,7 @@
 if (HardLinkTarget)
   Child.reset(new detail::InMemoryHardLink(P.str(), *HardLinkTarget));
 else {
+  assert(Buffer && "Non-HardLink without a buffer?");
   // Create a new file or directory.
   Status Stat(P.str(), getNextVirtualUniqueID(),
   llvm::sys::toTimePoint(ModificationTime), ResolvedUser,
@@ -771,6 +772,7 @@
 return false;
 
   // Return false only if the new file is different from the existing one.
+  assert(Buffer);
   if (auto Link = dyn_cast(Node)) {
 return Link->getResolvedFile().getBuffer()->getBuffer() ==
Buffer->getBuffer();
Index: llvm/lib/LTO/ThinLTOCodeGenerator.cpp
===
--- llvm/lib/LTO/ThinLTOCodeGenerator.cpp
+++ llvm/lib/LTO/ThinLTOCodeGenerator.cpp
@@ -959,6 +959,7 @@
 
   // Sequential linking phase
   auto Index = linkCombinedIndex();
+  assert(Index);
 
   // Save temps: index.
   if (!SaveTempsDir.empty()) {
Index: llvm/lib/DWARFLinker/DWARFLinker.cpp

[PATCH] D91410: [llvm][clang][mlir] Add checks for the return values from Target::createXXX to prevent protential null deref

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 306590.
OikawaKirie added a comment.

1. Replace fatal errors with assertions.

>> In D91410#2400018 , @tejohnson 
>> wrote:
>
> If these are not likely due to user input issues, then perhaps they should 
> all be assert so that they are compiled out in release compilers?



2. Fix clang-format errors in llvm/tools/llvm-objdump/MachODump.cpp


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D91410/new/

https://reviews.llvm.org/D91410

Files:
  clang/tools/clang-fuzzer/handle-llvm/handle_llvm.cpp
  clang/tools/driver/cc1as_main.cpp
  llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
  llvm/lib/CodeGen/AsmPrinter/AsmPrinterInlineAsm.cpp
  llvm/lib/CodeGen/LLVMTargetMachine.cpp
  llvm/lib/CodeGen/ParallelCG.cpp
  llvm/lib/LTO/LTOBackend.cpp
  llvm/lib/LTO/LTOCodeGenerator.cpp
  llvm/lib/LTO/LTOModule.cpp
  llvm/lib/LTO/ThinLTOCodeGenerator.cpp
  llvm/lib/Target/AArch64/AArch64AsmPrinter.cpp
  llvm/tools/llvm-exegesis/lib/LlvmState.cpp
  llvm/tools/llvm-exegesis/llvm-exegesis.cpp
  llvm/tools/llvm-mc/llvm-mc.cpp
  llvm/tools/llvm-mca/llvm-mca.cpp
  llvm/tools/llvm-ml/llvm-ml.cpp
  llvm/tools/llvm-objdump/MachODump.cpp
  llvm/tools/llvm-rtdyld/llvm-rtdyld.cpp
  mlir/lib/Conversion/GPUCommon/ConvertKernelFuncToBlob.cpp
  mlir/lib/ExecutionEngine/ExecutionEngine.cpp

Index: mlir/lib/ExecutionEngine/ExecutionEngine.cpp
===
--- mlir/lib/ExecutionEngine/ExecutionEngine.cpp
+++ mlir/lib/ExecutionEngine/ExecutionEngine.cpp
@@ -130,6 +130,10 @@
 
   std::unique_ptr machine(target->createTargetMachine(
   targetTriple, cpu, features.getString(), {}, {}));
+  if (!machine) {
+errs() << "Unable to create target machine\n";
+return true;
+  }
   llvmModule->setDataLayout(machine->createDataLayout());
   llvmModule->setTargetTriple(targetTriple);
   return false;
Index: mlir/lib/Conversion/GPUCommon/ConvertKernelFuncToBlob.cpp
===
--- mlir/lib/Conversion/GPUCommon/ConvertKernelFuncToBlob.cpp
+++ mlir/lib/Conversion/GPUCommon/ConvertKernelFuncToBlob.cpp
@@ -130,6 +130,10 @@
 }
 targetMachine.reset(target->createTargetMachine(triple.str(), targetChip,
 features, {}, {}));
+if (targetMachine == nullptr) {
+  emitError(loc, "connot initialize target machine");
+  return {};
+}
   }
 
   llvmModule.setDataLayout(targetMachine->createDataLayout());
Index: llvm/tools/llvm-rtdyld/llvm-rtdyld.cpp
===
--- llvm/tools/llvm-rtdyld/llvm-rtdyld.cpp
+++ llvm/tools/llvm-rtdyld/llvm-rtdyld.cpp
@@ -764,6 +764,8 @@
 ErrorAndExit("Unable to create disassembler!");
 
   std::unique_ptr MII(TheTarget->createMCInstrInfo());
+  if (!MII)
+ErrorAndExit("Unable to create target instruction info!");
 
   std::unique_ptr InstPrinter(
   TheTarget->createMCInstPrinter(Triple(TripleName), 0, *MAI, *MII, *MRI));
Index: llvm/tools/llvm-objdump/MachODump.cpp
===
--- llvm/tools/llvm-objdump/MachODump.cpp
+++ llvm/tools/llvm-objdump/MachODump.cpp
@@ -7200,10 +7200,32 @@
   else
 MachOMCPU = MCPU;
 
+#define CHECK_TARGET_INFO_CREATION(NAME)   \
+  do { \
+if (!NAME) {   \
+  WithColor::error(errs(), "llvm-objdump") \
+  << "couldn't initialize disassembler for target " << TripleName  \
+  << '\n'; \
+  return;  \
+}  \
+  } while (false)
+#define CHECK_THUMB_TARGET_INFO_CREATION(NAME) \
+  do { \
+if (!NAME) {   \
+  WithColor::error(errs(), "llvm-objdump") \
+  << "couldn't initialize disassembler for target " << ThumbTripleName \
+  << '\n'; \
+  return;  \
+}  \
+  } while (false)
+
   std::unique_ptr InstrInfo(TheTarget->createMCInstrInfo());
+  CHECK_TARGET_INFO_CREATION(InstrInfo);
   std::unique_ptr ThumbInstrInfo;
-  if (ThumbTarget)
+  if (ThumbTarget) {
 ThumbInstrInfo.reset(ThumbTarget->createMCInstrInfo());
+CHECK_THUMB_TARGET_INFO_CREATION(ThumbInstrInfo);
+  }
 
   

[PATCH] D91844: [llvm][clang] Add checks for the smart pointers with the possibility to be null

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D91844#2408897 , @dexonsmith wrote:

> Is it possible to split these up into separate patches for unrelated code?

Since these are reported by one static scan, and these reported places cannot 
be categorized with others, I choose to submit them in one patch for simplicity 
and avoiding spam. If it is necessary to separate them one by one, I will close 
this review and start a new one for each of them.

Or, maybe you are thinking of just separating the patch of clang with llvm? If 
so, I will start a new review just for the patch of clang and leave the patches 
of llvm here.




Comment at: clang/utils/TableGen/ClangAttrEmitter.cpp:1346-1353
   if (!Ptr) {
 // Search in reverse order so that the most-derived type is handled first.
 ArrayRef> Bases = Search->getSuperClasses();
 for (const auto &Base : llvm::reverse(Bases)) {
   if ((Ptr = createArgument(Arg, Attr, Base.first)))
 break;
 }

dexonsmith wrote:
> Can we just add a single assertion here? It looks to me like every caller 
> wants a valid return.
Ok, I will add an assertion here (below line 1353) in the new submits, and 
remove all other assertions I added in this file together with the checks on 
this pointer after the assertion (line 1355 and 1358).


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D91844/new/

https://reviews.llvm.org/D91844

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92160: [clang] Fix wrong FDs are used for files with same name in Tooling

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: dexonsmith, harlanhaskins, rsmith, arphaman, 
MaskRay, bkramer, ilya-biryukov.
OikawaKirie added a project: clang.
Herald added a subscriber: cfe-commits.
OikawaKirie requested review of this revision.

ClangTool will make FileManager mix up two header files with the same relative 
path in different absolute paths.

As the cache of previously opened FileEntry in FileManager is indexed by the 
file name, when relative paths are used as the index, wrong FileEntry may be 
used for the file with the same relative path. With ClangTool, as the current 
working directory will change when parsing multiple files, files with the same 
relative paths but different absolute paths will be mixed up by the FileManager.

Submit on behalf of Hao Zhang , I will forward the reviews and 
his replies.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D92160

Files:
  clang/include/clang/Basic/FileManager.h
  clang/lib/Basic/FileManager.cpp
  clang/lib/Tooling/Tooling.cpp
  
clang/test/Tooling/multiple-source-include-different-header-with-same-relative-path.cpp

Index: clang/test/Tooling/multiple-source-include-different-header-with-same-relative-path.cpp
===
--- /dev/null
+++ clang/test/Tooling/multiple-source-include-different-header-with-same-relative-path.cpp
@@ -0,0 +1,42 @@
+// This test case presents a circumstance that the information related one file
+// is misused by another.
+//
+// The path tree of the test directory is as follors.
+//   .
+//   ├── a
+//   │   ├── a.c
+//   │   └── config.h
+//   ├── b
+//   │   ├── b.c
+//   │   └── config.h
+//   └── compile_commands.json
+//
+// Both source files (a/a.c and b/b.c) includes the config.h file of their own
+// directory with `#include "config.h"`. However, the two config.h files are
+// different. File a/config.h is longer than b/config.h. Both a/a.c and b/b.c
+// are compiled in their own directory, which is recorded in the compilation
+// database.
+//
+// When using ClangTool to parse these two source files one by one, since the
+// file name of both header files are the same, the FileManager will confuse
+// with them and using the file entry of a/config.h for b/config.h. And the
+// wrong file length will lead to a buffer overflow when reading the file.
+//
+// In this test case, to avoid buffer overflow, we use the leading '\0' in an
+// empty buffer to trigger the problem. We set a/config.h as an empty line
+// comment, and leave b/config.h empty. Firstly, a/config.h is read and cached,
+// then when reading b/config.h, if the size of a/config.h is used, the first
+// two chars are read and the first one must be a '\0'.
+
+// RUN: rm -rf %t
+// RUN: mkdir -p %t/a
+// RUN: mkdir -p %t/b
+// RUN: echo '#include "config.h"' > %t/a/a.c
+// RUN: echo '#include "config.h"' > %t/b/b.c
+// RUN: echo '//' > %t/a/config.h
+// RUN: echo ''   > %t/b/config.h
+// RUN: echo '[{"arguments": ["cc", "-c", "-o", "a.o", "a.c"], "directory": "%t/a", "file": "a.c"}, {"arguments": ["cc", "-c", "-o", "b.o", "b.c"], "directory": "%t/b", "file": "b.c"}]' > %t/compile_commands.json
+
+// The following two test RUNs should have no output.
+// RUN: cd %t && clang-check a/a.c b/b.c 2>&1 | count 0
+// RUN: cd %t && clang-check b/b.c a/a.c 2>&1 | count 0
Index: clang/lib/Tooling/Tooling.cpp
===
--- clang/lib/Tooling/Tooling.cpp
+++ clang/lib/Tooling/Tooling.cpp
@@ -513,7 +513,7 @@
   CompileCommand.Directory))
 llvm::report_fatal_error("Cannot chdir into \"" +
  Twine(CompileCommand.Directory) + "\"!");
-
+  Files->notifyCurrentWorkingDirectoryChange();
   // Now fill the in-memory VFS with the relative file mappings so it will
   // have the correct relative paths. We never remove mappings but that
   // should be fine.
@@ -560,6 +560,7 @@
 OverlayFileSystem->setCurrentWorkingDirectory(InitialWorkingDir))
   llvm::errs() << "Error when trying to restore working dir: "
<< EC.message() << "\n";
+Files->notifyCurrentWorkingDirectoryChange();
   }
   return ProcessingFailed ? 1 : (FileSkipped ? 2 : 0);
 }
Index: clang/lib/Basic/FileManager.cpp
===
--- clang/lib/Basic/FileManager.cpp
+++ clang/lib/Basic/FileManager.cpp
@@ -50,12 +50,13 @@
 
 FileManager::FileManager(const FileSystemOptions &FSO,
  IntrusiveRefCntPtr FS)
-: FS(std::move(FS)), FileSystemOpts(FSO), SeenDirEntries(64),
-  SeenFileEntries(64), NextFileUID(0) {
+: FS(std::move(FS)), FileSystemOpts(FSO) {
   // If the caller doesn't provide a virtual file system, just grab the real
   // file system.
   if (!this->FS)
 this->FS = llvm::vfs::getRealFileSystem();
+

[PATCH] D92639: [analyzer] Add control flow arrows to the analyzer's HTML reports

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

It is really a good idea!

The operations that would not leave an event in the report are now clearly 
printed.

But there are three arrows that confuse me in the example report: the 
assignment `x = 0` (x -> 0 -> x), the function call `dereference(x)` (x -> 
dereference), and the return statement `return *x` (int -> *x). I know the 
arrow is based on the evaluation order of the engine. But from the view of a 
user, I think these arrows are confusing to some extent.

For the first two, I think it would be better to point just the statement 
(maybe a `CFGElement`) without inner arrows (x -> 0 -> x and x -> dereference), 
or point to the location of the operator itself rather than the BeginLoc (e.g. 
x -> 0 -> =). For the third one, an arrow from the function name to the first 
`CFGElement` looks good to me. And an arrow from the returned expr to the 
return type or to a special mark (e.g. ⬅️) can also be added, together with 
function calls (an arrow from the callstmt to a special mark, e.g. ➡️).

By the way, what do you think about adding arrows for data flows of specific 
symbolic values in the future?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D92639/new/

https://reviews.llvm.org/D92639

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D89987: [analyzer] [NFC] Rename SymbolRef to SymExprRef

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

Since `SymbolRef` is just a `const SymExpr *` in the current codebase, I'd 
prefer using `const SymExpr *` directly, just like how `MemRegion` is used, 
which would be clearer than both `SymbolRef` and `SymExprRef` as far as I am 
thinking.

Different from `ProgramStateRef` which is an alias to `IntrusiveRefCntPtr`, or 
`StoreRef` which is a wrapper object, an alias to a `const SymExpr *` makes no 
sense to me.

And this is also where I have been confused for a long while.

Or on the opposite, do we also need a similar alias for a `const MemRegion *`, 
maybe say `MemRegionRef`? After all, it is shorter. :-)


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D89987/new/

https://reviews.llvm.org/D89987

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102614: [index] Add support for type of pointers to class members

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added a reviewer: akyrtzi.
OikawaKirie added a project: clang.
Herald added a subscriber: arphaman.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

Required in D102159  that we should add 
support for the unhandled items instead of workaround them.
In this patch, support is added for indexing the type of pointers to class 
members. Such usages are found in MySQL.
The format is `::*`.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D102614

Files:
  clang/lib/Index/USRGeneration.cpp
  clang/test/Index/USR/MemberFunctionPtr.cpp


Index: clang/test/Index/USR/MemberFunctionPtr.cpp
===
--- /dev/null
+++ clang/test/Index/USR/MemberFunctionPtr.cpp
@@ -0,0 +1,33 @@
+// RUN: c-index-test core -print-source-symbols -- %s | FileCheck %s
+
+struct C {
+  int X;
+  void f(char);
+};
+
+void f(int C::*) {}
+// CHECK: function/C | f | c:@F@f#$@S@C::*I#
+void f(void (C::*)(char)) {}
+// CHECK: function/C | f | c:@F@f#$@S@C::*Fv(#C)#
+
+typedef int C::*Xtd;
+void ftd(Xtd) {}
+// CHECK: function/C | ftd | c:@F@ftd#$@S@C::*I#
+typedef void (C::*Ftd)(char);
+void ftd(Ftd) {}
+// CHECK: function/C | ftd | c:@F@ftd#$@S@C::*Fv(#C)#
+
+using Xus = int C::*;
+void fus(Xus) {}
+// CHECK: function/C | fus | c:@F@fus#$@S@C::*I#
+using Fus = void (C::*)(char);
+void fus(Fus) {}
+// CHECK: function/C | fus | c:@F@fus#$@S@C::*Fv(#C)#
+
+template  struct S;
+template  struct S {
+  static const bool V = true;
+  // CHECK: static-property/C++ | V | c:@SP>2#T#T@S>#t0.1::*t0.0@V
+  void f() {}
+  // CHECK: instance-method/C++ | f | c:@SP>2#T#T@S>#t0.1::*t0.0@F@f#
+};
Index: clang/lib/Index/USRGeneration.cpp
===
--- clang/lib/Index/USRGeneration.cpp
+++ clang/lib/Index/USRGeneration.cpp
@@ -893,6 +893,12 @@
   T = AT->getElementType();
   continue;
 }
+if (const MemberPointerType *MPT = T->getAs()) {
+  VisitType(QualType(MPT->getClass(), 0));
+  Out << "::*";
+  T = MPT->getPointeeType();
+  continue;
+}
 
 // Unhandled type.
 Out << ' ';


Index: clang/test/Index/USR/MemberFunctionPtr.cpp
===
--- /dev/null
+++ clang/test/Index/USR/MemberFunctionPtr.cpp
@@ -0,0 +1,33 @@
+// RUN: c-index-test core -print-source-symbols -- %s | FileCheck %s
+
+struct C {
+  int X;
+  void f(char);
+};
+
+void f(int C::*) {}
+// CHECK: function/C | f | c:@F@f#$@S@C::*I#
+void f(void (C::*)(char)) {}
+// CHECK: function/C | f | c:@F@f#$@S@C::*Fv(#C)#
+
+typedef int C::*Xtd;
+void ftd(Xtd) {}
+// CHECK: function/C | ftd | c:@F@ftd#$@S@C::*I#
+typedef void (C::*Ftd)(char);
+void ftd(Ftd) {}
+// CHECK: function/C | ftd | c:@F@ftd#$@S@C::*Fv(#C)#
+
+using Xus = int C::*;
+void fus(Xus) {}
+// CHECK: function/C | fus | c:@F@fus#$@S@C::*I#
+using Fus = void (C::*)(char);
+void fus(Fus) {}
+// CHECK: function/C | fus | c:@F@fus#$@S@C::*Fv(#C)#
+
+template  struct S;
+template  struct S {
+  static const bool V = true;
+  // CHECK: static-property/C++ | V | c:@SP>2#T#T@S>#t0.1::*t0.0@V
+  void f() {}
+  // CHECK: instance-method/C++ | f | c:@SP>2#T#T@S>#t0.1::*t0.0@F@f#
+};
Index: clang/lib/Index/USRGeneration.cpp
===
--- clang/lib/Index/USRGeneration.cpp
+++ clang/lib/Index/USRGeneration.cpp
@@ -893,6 +893,12 @@
   T = AT->getElementType();
   continue;
 }
+if (const MemberPointerType *MPT = T->getAs()) {
+  VisitType(QualType(MPT->getClass(), 0));
+  Out << "::*";
+  T = MPT->getPointeeType();
+  continue;
+}
 
 // Unhandled type.
 Out << ' ';
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: gamesh411, martong, hgabii.
OikawaKirie added a project: clang.
Herald added subscribers: steakhal, ASDenysPetrov, dkrupp, donat.nagy, 
Szelethus, arphaman, mikhail.ramalho, a.sidorin, rnkovacs, szepet, 
baloghadamsoftware, xazax.hun.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

This error was found when analyzing MySQL with CTU enabled.

When there are space characters in the lookup name, the current delimiter 
searching strategy will make the file path wrongly parsed.
And when two lookup names have the same prefix before their first space 
characters, a 'multiple definitions' error will be wrongly reported.

e.g. The lookup names for the two lambda exprs in the test case are 
`c:@S@G@F@G#@Sa@F@operator int (*)(char)#1` and `c:@S@G@F@G#@Sa@F@operator bool 
(*)(char)#1` respectively. And their prefixes are both 
`c:@S@G@F@G#@Sa@F@operator` when using the first space character as the 
delimiter.

This patch solves the problem by replacing `find` with `rfind` for the 
delimiter, as the chance of file paths having space characters seems to be far 
less than the lookup name.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D102669

Files:
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp


Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,29 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: cp %s %t/trigger.cpp
+// RUN: cp %S/Inputs/ctu-lookup-name-with-space.cpp %t/importee.cpp
+// RUN: echo '"%t/importee.cpp" : ["g++", "-c", "%t/importee.cpp"]' > 
%t/invocations.yaml
+// RUN: echo '[{"directory": "%t", "command": "g++ -c %t/importee.cpp", 
"file": "%t/importee.cpp"}, {"directory": "%t", "command": "g++ -c 
%t/trigger.cpp", "file": "%t/trigger.cpp"}]' > %t/compile_commands.json
+
+// RUN: %clang_extdef_map -p %t %t/importee.cpp > %t/externalDefMap.txt
+
+// RUN: cd %t && %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   trigger.cpp 2>&1 | FileCheck importee.cpp
+
+// RUN: cd %t && %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   -verify trigger.cpp
+
+int importee(int);
+
+void trigger() {
+  importee(0); // expected-warn...@importee.cpp:13 {{Division by zero}}
+}
Index: clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,14 @@
+void f(int (*)(char));
+void f(bool (*)(char));
+
+struct G {
+  G() {
+// CHECK-NOT: error: multiple definitions are found for the same key in 
index
+f([](char) -> int { return 42; });
+f([](char) -> bool { return true; });
+  }
+};
+
+int importee(int X) {
+  return 1 / X;
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -157,7 +157,7 @@
   unsigned LineNo = 1;
   while (std::getline(ExternalMapFile, Line)) {
 StringRef LineRef{Line};
-const size_t Delimiter = LineRef.find(' ');
+const size_t Delimiter = LineRef.rfind(' ');
 if (Delimiter > 0 && Delimiter != std::string::npos) {
   StringRef LookupName = LineRef.substr(0, Delimiter);
 


Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,29 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: cp %s %t/trigger.cpp
+// RUN: cp %S/Inputs/ctu-lookup-name-with-space.cpp %t/importee.cpp
+// RUN: echo '"%t/importee.cpp" : ["g++", "-c", "%t/importee.cpp"]' > %t/invocations.yaml
+// RUN: echo '[{"directory": "%t", "command": "g++ -c %t/importee.cpp", "file": "%t/importee.cpp"}, {"directory": "%t", "command": "g++ -c %t/trigger.cpp", "file": "%t/trigger.cpp"}]' > %t/compile_commands.json
+
+// RUN: %clang_extdef_map -p %t %t/importee.cpp > %t/externalDefMap.txt
+
+// RUN: cd %t && %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// R

[PATCH] D102159: [index][analyzer][ctu] Eliminate white spaces in the CTU lookup name.

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie abandoned this revision.
OikawaKirie added a comment.

It seems impossible and not so reasonable to eliminate all white space 
characters in the USR as mentioned in the test case of revision D102669 
.
This patch is split to revision D102669  to 
fix the wrongly parsed CTU index file and revision D102614 
 to handle the member pointer type mentioned 
here.
Please continue with these two following revisions and I will close this one.

Thanks to all reviewers.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102159/new/

https://reviews.llvm.org/D102159

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie marked an inline comment as done.
OikawaKirie added a comment.

In D102669#2765233 , @steakhal wrote:

> I'm not really familiar with the extdefmap part, but I'm surprised that we 
> are using spaces as separators.
> Shouldn't we consider using a different character?

I prefer the idea of changing the delimiter character, but it may lead to 
modifying a lot of test cases.
I think we'd better make this change in another revision in the future if we do 
want to change it.




Comment at: clang/test/Analysis/ctu-lookup-name-with-space.cpp:7
+// RUN: echo '"%t/importee.cpp" : ["g++", "-c", "%t/importee.cpp"]' > 
%t/invocations.yaml
+// RUN: echo '[{"directory": "%t", "command": "g++ -c %t/importee.cpp", 
"file": "%t/importee.cpp"}, {"directory": "%t", "command": "g++ -c 
%t/trigger.cpp", "file": "%t/trigger.cpp"}]' > %t/compile_commands.json
+

steakhal wrote:
> Probably splitting this up into multiple lines would result in a more 
> readable solution.
> 
> Something along these lines should work:
> ```
> cat >%t/compile_commands.json < line 1
> line 2
> ...
> EOL
> ```
The suggestion is great, however I cannot find a way to write the `RUN` 
commands.
Could you please tell me how to write the commands in this way? It is also 
useful to help me merging the test case into one file.



Comment at: clang/test/Analysis/ctu-lookup-name-with-space.cpp:11-23
+// RUN: cd %t && %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   trigger.cpp 2>&1 | FileCheck importee.cpp
+

steakhal wrote:
> Why do you need two separate invocations? Couldn't you just merge these?
> I've seen cases where `-verify` was used in conjunction with `FileCheck`.
I forgot the `--allow-empty` argument during writing this test case. I will 
merge them in an update.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102149: [analyzer][ctu] Allow loading invocation list from a compilation database automatically detected from the ctu-dir

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D102149#2765402 , @martong wrote:

> Thank you @OikawaKirie for working on this many CTU related patches! I am 
> going to find time for a thorough review and going to pursue @gamesh411 as 
> well to do the same! On the other hand, it would be really useful if you 
> could build a "Stack" from these patches, I mean could you please set which 
> patch depends from which other (see "Edit related revisions")?

To make it easier to review, all these recently submitted patches are based on 
the `main` branch. Their code does not depend on each other.

Logically, they can be separated into two branches: invocation list and ctu 
index.
The invocation list related patches target on constructing the invocation list 
in ASTLoader from a compilation database. And the problems of using invocation 
lists are also fixed.
Whereas the ctu index related patches mainly focus on the delimiter character 
in the ctu index, which is generated by the `clang-extdef-mapping` tool.

If the "stack" means the logical relations, I will update that later. 
Otherwise, it is OK to start with any of the patches.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102149/new/

https://reviews.llvm.org/D102149

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102062: [analyzer][ctu] Append ctu-dir to ctu-invocation-list for non-absolute paths

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie abandoned this revision.
OikawaKirie added a comment.

In D102062#2768569 , @martong wrote:

> First of all, thank you for the patch!
> We had a meeting with my colleges (@steakhal, @gamesh411) and we agreed in 
> the following.
>
> We'd like to keep the current behavior because this way the behavior is 
> similar that we got used to with any other command line tools.

Ok, if you all agree with using `ctu-invocation-list` with a path related to 
CWD rather than `ctu-dir`, I will drop this revision here.
Although, I still believe that it would be better to make the behavior similar 
to the ctu index file, which is related to the `ctu-dir` (see the comments in 
the code).

Thanks for your suggestions on this revision.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102062/new/

https://reviews.llvm.org/D102062

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102149: [analyzer][ctu] Allow loading invocation list from a compilation database automatically detected from the ctu-dir

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D102149#2768541 , @martong wrote:

> Thank you for the patch!
>
> Though, the idea is nice, there is a serious technical obstacle here: we 
> cannot use the clangTooling lib as a dependency of the CTU lib because that 
> would introduce a circular dependency. Actually, this was the reason for 
> introducing the invocation list yaml file; we could not use the compilation 
> database implementation from tooling 
> (https://reviews.llvm.org/D75665?id=260238#inline-722328).

According to my recent experiences on using clang-check, converting the 
compilation database to an invocation list is also an acceptable solution.
Currently, a better solution may be adding a tool to handle the conversion, 
just as what has been mentioned in the revision you presented. Although, it 
would be a very simple python script such as:

  a = dict()
  for i in json.load(open("/path/to/compile_commands.json")):
  a[os.path.abspath(os.path.join(i['directory'], i['file']))] = \
  (shlex.split(i['command']) if 'command' in i else i['arguments']) + \
  ['-Xclang', '-working-directory=' + i['directory']]
  print(yaml.dump(a));

If you agree with this idea, I will follow the way to create a tool for the 
conversion and drop this revision.

Actually, I am writing a makefile to schedule the ctu-index generation as well 
as analyzing the code with the clang-check tool. And the python script snippet 
presented above is just a part of my makefile.
Maybe I can add my makefile to as a part of the analyzer.




Comment at: clang/lib/CrossTU/CrossTranslationUnit.cpp:711-712
+
+  List.emplace_back("-Xclang");
+  List.emplace_back("-working-directory=" + CC.Directory);
+}

The approach of converting a compilation database to an invocation list is just 
simply adding the `-working-directory` argument for the `"directory"` entry for 
each compile command object. Therefore, a script doing the conversion can make 
it simpler and easier to use.

The main reason why this patch is submitted is I previously do not know the 
`-working-directory` argument. Without this argument, it would be very 
difficult and complex to convert a compilation database to an invocation list.

If you agree that my approach of converting the database (presented in the for 
loop) is OK, I will continue with making a tool for the conversion.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102149/new/

https://reviews.llvm.org/D102149

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102614: [index] Add support for type of pointers to class members

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added inline comments.



Comment at: clang/lib/Index/USRGeneration.cpp:897
+if (const MemberPointerType *MPT = T->getAs()) {
+  VisitType(QualType(MPT->getClass(), 0));
+  Out << "::*";

akyrtzi wrote:
> A bit better to do `VisitTagDecl(MPT->getClass())`, what do you think?
This was my first version of the patch, which was 
`VisitTagDecl(MPT->getMostRecentCXXRecordDecl())`. However, it cannot handle 
the `TemplateTypeParmType` type, which is presented in the last two CHECKs of 
the test case. Call to method `MemberPointerType::getMostRecentCXXRecordDecl` 
will dereference a nullptr in the function (`getClass()->getAsCXXRecordDecl()` 
returns nullptr) . To fully handle all circumstances, I finally choose to use 
the `VisitType` method.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102614/new/

https://reviews.llvm.org/D102614

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 347327.
OikawaKirie added a comment.
Herald added a subscriber: manas.

In D101763#2768549 , @martong wrote:

> First of all, thank you for the patch!
> We had a meeting with my colleges (@steakhal, @gamesh411) and we agreed in 
> the following. This is indeed an issue and the fix is okay. About the test, 
> we'd like to ask if you could create a small test lib with its own 'open' 
> function that would simply call the system's open and would print 'open'. And 
> then with LD_PREOLOAD you could use that lib in the lit test. (If that 
> approach is not working for some reason then we may just mark the test with 
> XFAIL.)

Updated as suggested based on a previous submit 
https://reviews.llvm.org/D101763?id=343346. An empty invocation list is used to 
trigger a parsing failure, and name `invocations.yaml` is used as the file name 
for logging and checking.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp

Index: clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp
@@ -0,0 +1,56 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// RUN: %host_cxx %s -fPIC -shared -o %t/mock_open.so
+
+// RUN: echo "void bar(); void foo() { bar(); bar(); }" > %t/trigger.c
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file": "%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+
+// Add an empty invocation list to make the on-demand parsing fail and load it again.
+// RUN: echo '' > %t/invocations.yaml
+
+// RUN: cd %t && \
+// RUN: LD_PRELOAD=%t/mock_open.so \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %t/trigger.c | FileCheck %s
+
+// REQUIRES: shell, system-linux
+
+// CHECK: {{Opening file invocations.yaml: 1}}
+// CHECK-NOT: {{Opening file invocations.yaml: 2}}
+
+#define _GNU_SOURCE 1
+#include 
+#include 
+
+#include 
+#include 
+#include 
+using namespace std;
+
+extern "C" int open(const char *name, int flag, ...) {
+  // Log how many times the invocation list is opened.
+  if ("invocations.yaml" == string(name)) {
+static unsigned N = 0;
+cout << "Opening file invocations.yaml: " << ++N << endl;
+  }
+
+  // The original open function will be called to open the files.
+  using open_t = int (*)(const char *, int, mode_t);
+  static open_t o_open = nullptr;
+  if (!o_open)
+o_open = reinterpret_cast(dlsym(RTLD_NEXT, "open"));
+  assert(o_open && "Cannot find function `open'.");
+
+  va_list vl;
+  va_start(vl, flag);
+  auto mode = va_arg(vl, mode_t);
+  va_end(vl);
+  return o_open(name, flag, mode);
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -92,6 +92,10 @@
 
   std::string message(int Condition) const override {
 switch (static_cast(Condition)) {
+case index_error_code::success:
+  // There should not be a success error. Jump to unreachable directly.
+  // Add this case to make the compiler stop complaining.
+  break;
 case index_error_code::unspecified:
   return "An unknown error has occurred.";
 case index_error_code::missing_index_file:
@@ -667,12 +671,15 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult = index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +687,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvo

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 347441.
OikawaKirie added a comment.

Re-submit the patch to re-build the code.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp

Index: clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp
@@ -0,0 +1,56 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// RUN: %host_cxx %s -fPIC -shared -o %t/mock_open.so
+
+// RUN: echo "void bar(); void foo() { bar(); bar(); }" > %t/trigger.c
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file": "%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+
+// Add an empty invocation list to make the on-demand parsing fail and load it again.
+// RUN: echo '' > %t/invocations.yaml
+
+// RUN: cd %t && \
+// RUN: LD_PRELOAD=%t/mock_open.so \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %t/trigger.c | FileCheck %s
+
+// REQUIRES: shell, system-linux
+
+// CHECK: {{Opening file invocations.yaml: 1}}
+// CHECK-NOT: {{Opening file invocations.yaml: 2}}
+
+#define _GNU_SOURCE 1
+#include 
+#include 
+
+#include 
+#include 
+#include 
+using namespace std;
+
+extern "C" int open(const char *name, int flag, ...) {
+  // Log how many times the invocation list is opened.
+  if ("invocations.yaml" == string(name)) {
+static unsigned N = 0;
+cout << "Opening file invocations.yaml: " << ++N << endl;
+  }
+
+  // The original open function will be called to open the files.
+  using open_t = int (*)(const char *, int, mode_t);
+  static open_t o_open = nullptr;
+  if (!o_open)
+o_open = reinterpret_cast(dlsym(RTLD_NEXT, "open"));
+  assert(o_open && "Cannot find function `open'.");
+
+  va_list vl;
+  va_start(vl, flag);
+  auto mode = va_arg(vl, mode_t);
+  va_end(vl);
+  return o_open(name, flag, mode);
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -92,6 +92,10 @@
 
   std::string message(int Condition) const override {
 switch (static_cast(Condition)) {
+case index_error_code::success:
+  // There should not be a success error. Jump to unreachable directly.
+  // Add this case to make the compiler stop complaining.
+  break;
 case index_error_code::unspecified:
   return "An unknown error has occurred.";
 case index_error_code::missing_index_file:
@@ -667,12 +671,15 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult = index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +687,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](const IndexError &E) { PreviousParsingResult = E.getCode(); });
+return llvm::make_error(PreviousParsingResult);
+  }
 
   InvocationList = *ExpectedInvocationList;
 
Index: clang/include/clang/CrossTU/CrossTranslationUnit.h
===
--- clang/include/clang/CrossTU/CrossTranslationUnit.h
+++ clang/include/clang/CrossTU/CrossTranslationUnit.h
@@ -38,6 +38,7 @@
 namespace cross_tu {
 
 enum class index_error_code {
+  success = 0,
   unspecified = 1,
   missing_index_file,
   invalid_index_format,
@@ -253,6 +254,7 @@
 /// In case of on-deman

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 347446.
OikawaKirie added a comment.

Re-submit the patch to re-build the code.

Sorry for the spam of submitting an incomplete diff which triggers a patch 
apply failure.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp

Index: clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp
@@ -0,0 +1,56 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// RUN: %host_cxx %s -fPIC -shared -o %t/mock_open.so
+
+// RUN: echo "void bar(); void foo() { bar(); bar(); }" > %t/trigger.c
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file": "%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+
+// Add an empty invocation list to make the on-demand parsing fail and load it again.
+// RUN: echo '' > %t/invocations.yaml
+
+// RUN: cd %t && \
+// RUN: LD_PRELOAD=%t/mock_open.so \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %t/trigger.c | FileCheck %s
+
+// REQUIRES: shell, system-linux
+
+// CHECK: {{Opening file invocations.yaml: 1}}
+// CHECK-NOT: {{Opening file invocations.yaml: 2}}
+
+#define _GNU_SOURCE 1
+#include 
+#include 
+
+#include 
+#include 
+#include 
+using namespace std;
+
+extern "C" int open(const char *name, int flag, ...) {
+  // Log how many times the invocation list is opened.
+  if ("invocations.yaml" == string(name)) {
+static unsigned N = 0;
+cout << "Opening file invocations.yaml: " << ++N << endl;
+  }
+
+  // The original open function will be called to open the files.
+  using open_t = int (*)(const char *, int, mode_t);
+  static open_t o_open = nullptr;
+  if (!o_open)
+o_open = reinterpret_cast(dlsym(RTLD_NEXT, "open"));
+  assert(o_open && "Cannot find function `open'.");
+
+  va_list vl;
+  va_start(vl, flag);
+  auto mode = va_arg(vl, mode_t);
+  va_end(vl);
+  return o_open(name, flag, mode);
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -92,6 +92,10 @@
 
   std::string message(int Condition) const override {
 switch (static_cast(Condition)) {
+case index_error_code::success:
+  // There should not be a success error. Jump to unreachable directly.
+  // Add this case to make the compiler stop complaining.
+  break;
 case index_error_code::unspecified:
   return "An unknown error has occurred.";
 case index_error_code::missing_index_file:
@@ -667,12 +671,15 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult = index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +687,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](const IndexError &E) { PreviousParsingResult = E.getCode(); });
+return llvm::make_error(PreviousParsingResult);
+  }
 
   InvocationList = *ExpectedInvocationList;
 
Index: clang/include/clang/CrossTU/CrossTranslationUnit.h
===
--- clang/include/clang/CrossTU/CrossTranslationUnit.h
+++ clang/include/clang/CrossTU/CrossTranslationUnit.h
@@ -38,6 +38,7 @@
 namespace cross_tu {
 
 enum class index_error_code {
+  success = 0,
   unspecified = 1,
   m

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D101763#2777821 , @steakhal wrote:

> It seems like everything passes. Yeey, good job!
> Shall I commit this tomorrow on your behalf?

I do not have commit access to the code base. It would be appreciated if you 
could commit it on my behalf (Ella Ma ).
Thanks to all reviewers again for your suggestions.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102614: [index] Add support for type of pointers to class members

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 347581.
OikawaKirie added a comment.

Update the test case to avoid a crash in the Windows version of the 
`c-index-test` tool.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102614/new/

https://reviews.llvm.org/D102614

Files:
  clang/lib/Index/USRGeneration.cpp
  clang/test/Index/USR/MemberFunctionPtr.cpp


Index: clang/test/Index/USR/MemberFunctionPtr.cpp
===
--- /dev/null
+++ clang/test/Index/USR/MemberFunctionPtr.cpp
@@ -0,0 +1,33 @@
+// RUN: c-index-test -index-file %s | FileCheck %s
+
+struct C {
+  int X;
+  void f(char);
+};
+
+void f(int C::*) {}
+// CHECK: name: f | USR: c:@F@f#$@S@C::*I#
+void f(void (C::*)(char)) {}
+// CHECK: name: f | USR: c:@F@f#$@S@C::*Fv(#C)#
+
+typedef int C::*Xtd;
+void ftd(Xtd) {}
+// CHECK: name: ftd | USR: c:@F@ftd#$@S@C::*I#
+typedef void (C::*Ftd)(char);
+void ftd(Ftd) {}
+// CHECK: name: ftd | USR: c:@F@ftd#$@S@C::*Fv(#C)#
+
+using Xus = int C::*;
+void fus(Xus) {}
+// CHECK: name: fus | USR: c:@F@fus#$@S@C::*I#
+using Fus = void (C::*)(char);
+void fus(Fus) {}
+// CHECK: name: fus | USR: c:@F@fus#$@S@C::*Fv(#C)#
+
+template  struct S;
+template  struct S {
+  static const bool V = true;
+  // CHECK: name: V | USR: c:@SP>2#T#T@S>#t0.1::*t0.0@V
+  void f() {}
+  // CHECK: name: f | USR: c:@SP>2#T#T@S>#t0.1::*t0.0@F@f#
+};
Index: clang/lib/Index/USRGeneration.cpp
===
--- clang/lib/Index/USRGeneration.cpp
+++ clang/lib/Index/USRGeneration.cpp
@@ -893,6 +893,12 @@
   T = AT->getElementType();
   continue;
 }
+if (const MemberPointerType *MPT = T->getAs()) {
+  VisitType(QualType(MPT->getClass(), 0));
+  Out << "::*";
+  T = MPT->getPointeeType();
+  continue;
+}
 
 // Unhandled type.
 Out << ' ';


Index: clang/test/Index/USR/MemberFunctionPtr.cpp
===
--- /dev/null
+++ clang/test/Index/USR/MemberFunctionPtr.cpp
@@ -0,0 +1,33 @@
+// RUN: c-index-test -index-file %s | FileCheck %s
+
+struct C {
+  int X;
+  void f(char);
+};
+
+void f(int C::*) {}
+// CHECK: name: f | USR: c:@F@f#$@S@C::*I#
+void f(void (C::*)(char)) {}
+// CHECK: name: f | USR: c:@F@f#$@S@C::*Fv(#C)#
+
+typedef int C::*Xtd;
+void ftd(Xtd) {}
+// CHECK: name: ftd | USR: c:@F@ftd#$@S@C::*I#
+typedef void (C::*Ftd)(char);
+void ftd(Ftd) {}
+// CHECK: name: ftd | USR: c:@F@ftd#$@S@C::*Fv(#C)#
+
+using Xus = int C::*;
+void fus(Xus) {}
+// CHECK: name: fus | USR: c:@F@fus#$@S@C::*I#
+using Fus = void (C::*)(char);
+void fus(Fus) {}
+// CHECK: name: fus | USR: c:@F@fus#$@S@C::*Fv(#C)#
+
+template  struct S;
+template  struct S {
+  static const bool V = true;
+  // CHECK: name: V | USR: c:@SP>2#T#T@S>#t0.1::*t0.0@V
+  void f() {}
+  // CHECK: name: f | USR: c:@SP>2#T#T@S>#t0.1::*t0.0@F@f#
+};
Index: clang/lib/Index/USRGeneration.cpp
===
--- clang/lib/Index/USRGeneration.cpp
+++ clang/lib/Index/USRGeneration.cpp
@@ -893,6 +893,12 @@
   T = AT->getElementType();
   continue;
 }
+if (const MemberPointerType *MPT = T->getAs()) {
+  VisitType(QualType(MPT->getClass(), 0));
+  Out << "::*";
+  T = MPT->getPointeeType();
+  continue;
+}
 
 // Unhandled type.
 Out << ' ';
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97251: [llvm] Add assertions for the smart pointers with the possibility to be null in ClangAttrEmitter

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: dexonsmith, aaron.ballman, pcc.
OikawaKirie added a project: clang.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

Split from D91844 .

The return value of createArgument in file 
clang/utils/TableGen/ClangAttrEmitter.cpp. Although there are a lot of checks 
in the function, nullptr is still possible to be returned.
In this patch, I added an assertion to confirm that.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D97251

Files:
  clang/utils/TableGen/ClangAttrEmitter.cpp


Index: clang/utils/TableGen/ClangAttrEmitter.cpp
===
--- clang/utils/TableGen/ClangAttrEmitter.cpp
+++ clang/utils/TableGen/ClangAttrEmitter.cpp
@@ -1361,6 +1361,8 @@
 }
   }
 
+  assert(Ptr && "Cannot create argument.");
+
   if (Ptr && Arg.getValueAsBit("Optional"))
 Ptr->setOptional(true);
 


Index: clang/utils/TableGen/ClangAttrEmitter.cpp
===
--- clang/utils/TableGen/ClangAttrEmitter.cpp
+++ clang/utils/TableGen/ClangAttrEmitter.cpp
@@ -1361,6 +1361,8 @@
 }
   }
 
+  assert(Ptr && "Cannot create argument.");
+
   if (Ptr && Arg.getValueAsBit("Optional"))
 Ptr->setOptional(true);
 
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97251: [llvm] Add assertions for the smart pointers with the possibility to be null in ClangAttrEmitter

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

As a recursive function it is, the original patch in D91844 
 was adding assertions where the function is 
called other places in the file.

To confirm it will not lead to a crash here, I re-build my code base with clang 
and clang-tools-extra enabled. Everything seems to be good.

What do you think about the assertion? Should it be located just here or where 
the function is called?

Refer to D91844  for more details about the 
previous version.




Comment at: clang/utils/TableGen/ClangAttrEmitter.cpp:1364
 
+  assert(Ptr && "Cannot create argument.");
+

Just as what has been suggested by @dexonsmith in D91844, I add the assertion 
here. However, as a recursive function it is (line 1359), I still concern 
whether the assertion here will lead to a crash.

What do you think?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97251/new/

https://reviews.llvm.org/D97251

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D91844: [llvm][clang] Add checks for the smart pointers with the possibility to be null

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie abandoned this revision.
OikawaKirie added a comment.

Split to

- D97251  
(clang/utils/TableGen/ClangAttrEmitter.cpp)
- D97185  
(llvm/lib/DWARFLinker/DWARFLinker.cpp)
- D97254  
(llvm/lib/LTO/ThinLTOCodeGenerator.cpp)
- D97255  
(llvm/lib/Support/VirtualFileSystem.cpp)
- D97258  (llvm/tools/llvm-link/llvm-link.cpp)

And the last one (llvm/utils/TableGen/OptParserEmitter.cpp) has been fixed by 
others.

Please review the new revisions, and this revision will be closed.

Thanks to all reviewers and sorry for the delay.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D91844/new/

https://reviews.llvm.org/D91844

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97265: [clang] Allow clang-check to customize output file name

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: hokein, sammccall, kbobyrev, alexfh.
OikawaKirie added a project: clang.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

Required by https://stackoverflow.com/questions/58073606

As the output argument is stripped out in the `clang-check` tool, it seems 
impossible for `clang-check` users to customize the output file name, even with 
`-extra-args` and `-extra-arg-before`.

This patch adds the `-output` argument to allow users to adjust the output 
name. And if the argument is not set, the original strip output adjuster will 
remove the output arguments.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D97265

Files:
  clang/test/Tooling/clang-check-reset-o.cpp
  clang/tools/clang-check/ClangCheck.cpp


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -76,7 +76,9 @@
 Analyze("analyze",
 cl::desc(Options.getOptionHelpText(options::OPT_analyze)),
 cl::cat(ClangCheckCategory));
-
+static cl::opt
+Output("output", cl::desc(Options.getOptionHelpText(options::OPT_o)),
+   cl::cat(ClangCheckCategory));
 static cl::opt
 Fixit("fixit", cl::desc(Options.getOptionHelpText(options::OPT_fixit)),
   cl::cat(ClangCheckCategory));
@@ -206,7 +208,18 @@
 
   // Clear adjusters because -fsyntax-only is inserted by the default chain.
   Tool.clearArgumentsAdjusters();
-  Tool.appendArgumentsAdjuster(getClangStripOutputAdjuster());
+
+  // Reset output path if is provided by user.
+  Tool.appendArgumentsAdjuster(
+  [&](const CommandLineArguments &Args, StringRef File) {
+auto Ret = getClangStripOutputAdjuster()(Args, File);
+if (!Output.empty()) {
+  Ret.emplace_back("-o");
+  Ret.emplace_back(Output);
+}
+return Ret;
+  });
+
   Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
 
   // Running the analyzer requires --analyze. Other modes can work with the
Index: clang/test/Tooling/clang-check-reset-o.cpp
===
--- /dev/null
+++ clang/test/Tooling/clang-check-reset-o.cpp
@@ -0,0 +1,11 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+// RUN: echo '[{"directory":".","command":"clang++ -c %t/test.cpp -o foo 
-ofoo","file":"%t/test.cpp"}]' | sed -e 's/\\/\//g' > %t/compile_commands.json
+// RUN: cp "%s" "%t/test.cpp"
+// RUN: not clang-check -p "%t" "%t/test.cpp" -analyze -output=qwerty 
-extra-arg=-v 2>&1|FileCheck %s
+// FIXME: Make the above easier.
+
+// CHECK: Invocation
+// CHECK: {{qwerty}}
+// CHECK: C++ requires
+invalid;


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -76,7 +76,9 @@
 Analyze("analyze",
 cl::desc(Options.getOptionHelpText(options::OPT_analyze)),
 cl::cat(ClangCheckCategory));
-
+static cl::opt
+Output("output", cl::desc(Options.getOptionHelpText(options::OPT_o)),
+   cl::cat(ClangCheckCategory));
 static cl::opt
 Fixit("fixit", cl::desc(Options.getOptionHelpText(options::OPT_fixit)),
   cl::cat(ClangCheckCategory));
@@ -206,7 +208,18 @@
 
   // Clear adjusters because -fsyntax-only is inserted by the default chain.
   Tool.clearArgumentsAdjusters();
-  Tool.appendArgumentsAdjuster(getClangStripOutputAdjuster());
+
+  // Reset output path if is provided by user.
+  Tool.appendArgumentsAdjuster(
+  [&](const CommandLineArguments &Args, StringRef File) {
+auto Ret = getClangStripOutputAdjuster()(Args, File);
+if (!Output.empty()) {
+  Ret.emplace_back("-o");
+  Ret.emplace_back(Output);
+}
+return Ret;
+  });
+
   Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
 
   // Running the analyzer requires --analyze. Other modes can work with the
Index: clang/test/Tooling/clang-check-reset-o.cpp
===
--- /dev/null
+++ clang/test/Tooling/clang-check-reset-o.cpp
@@ -0,0 +1,11 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+// RUN: echo '[{"directory":".","command":"clang++ -c %t/test.cpp -o foo -ofoo","file":"%t/test.cpp"}]' | sed -e 's/\\/\//g' > %t/compile_commands.json
+// RUN: cp "%s" "%t/test.cpp"
+// RUN: not clang-check -p "%t" "%t/test.cpp" -analyze -output=qwerty -extra-arg=-v 2>&1|FileCheck %s
+// FIXME: Make the above easier.
+
+// CHECK: Invocation
+// CHECK: {{qwerty}}
+// CHECK: C++ requires
+invalid;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97251: [llvm] Add assertions for the smart pointers with the possibility to be null in ClangAttrEmitter

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie requested review of this revision.
OikawaKirie added a comment.

@aaron.ballman

My only concern is the recursive call to this function on line 1359. If you 
think it is also OK for the recursive call on line 1359, I will update the 
patch as you mentioned above.

BTW, how can I test and debug this tblgen backend?




Comment at: clang/utils/TableGen/ClangAttrEmitter.cpp:1359
 for (const auto &Base : llvm::reverse(Bases)) {
   if ((Ptr = createArgument(Arg, Attr, Base.first)))
 break;

The recursive call is here.

The for loop here seems to allow a failed attempt on its bases. Although the 
top-level callers expect the return value of this function is always non-null, 
what about the call here?



Comment at: clang/utils/TableGen/ClangAttrEmitter.cpp:1364
 
+  assert(Ptr && "Cannot create argument.");
+

aaron.ballman wrote:
> aaron.ballman wrote:
> > OikawaKirie wrote:
> > > Just as what has been suggested by @dexonsmith in D91844, I add the 
> > > assertion here. However, as a recursive function it is (line 1359), I 
> > > still concern whether the assertion here will lead to a crash.
> > > 
> > > What do you think?
> > The intent with this function is that it never returns `nullptr` unless the 
> > programmer messed up their call of the function (passed in the wrong kind 
> > of `Record` kind of problem). All of the callers assume this returns 
> > non-null, so I think it's reasonable to assert that the value is non-null 
> > by this point.
> 
However, there is a recursive call to this function on line 1359, and it seems 
that `nullptr` is allowed to be returned for the call. I am just worrying about 
whether the assertion here will lead to a crash in the recursive calls.

IMO, a better way is to wrap this function and assert the return value of this 
function in the wrapper function.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97251/new/

https://reviews.llvm.org/D97251

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97265: [clang] Allow clang-check to customize output file name

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D97265#2581653 , @sammccall wrote:

> 



> It looks like either we're in syntax-only mode (and -o will be ignored 
> regardless) or we're in analyzer mode, and want -o.
> It seems tempting to just stop stripping it in this case, but then we'll end 
> up with -o foo.o -o foo.html, which is no good.
> So the logic of this patch seems sound.

The strip output adjuster is invoked before appending the new `-o` option (line 
215). Therefore, there will be only one `-o` option left if the new `-output` 
option is set when running `clang-check`.

> I wonder about the CLI though, I think the user will have to pass four flags 
> with different styles to use it!
>
> `clang-check -analyze ... -extra-arg=-Xanalyzer 
> -extra-arg=-analyzer-output=html -output=html_output `
>
> (Maybe -analyze should imply -Xanalyzer, and analyzer-output should be 
> promoted to a real flag without the legacy plist default...)

You are right. It is better if we can add the `clang-check` version of 
`-Xanalyzer` options. However, the analyzer options can only determine the 
format of the output, rather than the name of the output. Therefore, we still 
need another option to set the output name.

My original idea is to add a generic way for the `clang-check` users to 
customize the output name. Although there is currently only one user, the 
static analyzer, there may be other users of this option in the future. 
Therefore, I thought that it would be better to add just a generic output 
option here.

According to your suggestions, it seems to be a good idea to use `-Xanalyzer` 
to set the output name, and the users can have a good experience when using 
clang-check to analyze their code. However, the disadvantage of this method is 
that the argument parsing procedure will become more complex. IMO, we can also 
add a `clang-check` option to set the analyzer output format together with the 
analyzer output name, and leave other analyzer options that are not commonly 
used to the `-Xanalyzer` option (refer to the second inline comment).




Comment at: clang/test/Tooling/clang-check-reset-o.cpp:10
+// CHECK: {{qwerty}}
+// CHECK: C++ requires
+invalid;

sammccall wrote:
> it looks like you're checking for the analysis finding in stdout of 
> clang-check, rather than in the generated html file.
> So i'm not sure this test verifies it actually works!
Different kinds of output formats can be either a directory or a file named 
after the `-o` option. It is inconvenient to check the exact output file or 
directory.

This test case is created by imitating the 
`clang/test/Tooling/clang-check-strip-o.cpp` test case, which only checks 
whether the cc1 arguments are correctly set.





Comment at: clang/tools/clang-check/ClangCheck.cpp:80
+static cl::opt
+Output("output", cl::desc(Options.getOptionHelpText(options::OPT_o)),
+   cl::cat(ClangCheckCategory));

sammccall wrote:
> I think this should have a name with clear semantics like 
> -analyzer-output-file (and only be used in analyzer mode).
> 
> While it does just append -o to the command-line, I think it will confuse 
> users to treat it as a generic output flag.
> I think this should have a name with clear semantics like 
> -analyzer-output-file (and only be used in analyzer mode).
> While it does just append -o to the command-line, I think it will confuse 
> users to treat it as a generic output flag.

My original idea is to add a generic output flag to the `clang-check` tool, as 
it seems impossible for tool users to customize the output name. However, it 
will probably confuse users with the useless `-o` options set via `-extra-arg` 
or appended after `--` and with other `-fsyntax-only` actions, just as what you 
mentioned.

So, which you think is better? All with `-Xanalyzer` option
```
$ clang-check -analyze -Xanalyzer -analyzer-output-name=html_output -Xanalyzer 
-analyzer-output=html -Xanalyzer -analyzer-other-options=value input1.cc 
input2.cc ...
```
OR `-analyzer-output-name` without `-Xanalyzer` option
```
$  clang-check -analyze -analyzer-output-name=html_output -Xanalyzer 
-analyzer-output=html -Xanalyzer -analyzer-other-options=value input1.cc 
input2.cc ...
```
OR even more aggressively, only non-commonly used options with `-Xanalyzer` 
option
```
$  clang-check -analyze -analyzer-output-name=html_output -analyzer-output=html 
-Xanalyzer -analyzer-other-options=value input1.cc input2.cc ...
```



Comment at: clang/tools/clang-check/ClangCheck.cpp:213-221
+  Tool.appendArgumentsAdjuster(
+  [&](const CommandLineArguments &Args, StringRef File) {
+auto Ret = getClangStripOutputAdjuster()(Args, File);
+if (!Output.empty()) {
+  Ret.emplace_back("-o");
+  Ret.emplace_back(Output);
+}

If we use the `-analyzer-output-name` option, we need to check whether 
`

[PATCH] D97265: [clang] Allow clang-check to customize analyzer output file or dir name

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 328396.
OikawaKirie retitled this revision from "[clang] Allow clang-check to customize 
output file name" to "[clang] Allow clang-check to customize analyzer output 
file or dir name".
OikawaKirie edited the summary of this revision.
OikawaKirie added a comment.
Herald added subscribers: steakhal, ASDenysPetrov, dkrupp, donat.nagy, 
Szelethus, a.sidorin, baloghadamsoftware.

1. Update option name from `-output` to `-analyzer-output-path`. As it can 
either be a file name or a directory name, the word `path` may not be accurate. 
Please give me your suggestions on this option name.
2. Add more verifications on the output files, rather than verifying the value 
of the `-o` option. For simplicity, the regression test case only checks the 
plist format.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97265/new/

https://reviews.llvm.org/D97265

Files:
  clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
  clang/tools/clang-check/ClangCheck.cpp


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -76,7 +76,10 @@
 Analyze("analyze",
 cl::desc(Options.getOptionHelpText(options::OPT_analyze)),
 cl::cat(ClangCheckCategory));
-
+static cl::opt
+AnalyzerOutput("analyzer-output-path",
+   cl::desc(Options.getOptionHelpText(options::OPT_o)),
+   cl::cat(ClangCheckCategory));
 static cl::opt
 Fixit("fixit", cl::desc(Options.getOptionHelpText(options::OPT_fixit)),
   cl::cat(ClangCheckCategory));
@@ -206,7 +209,19 @@
 
   // Clear adjusters because -fsyntax-only is inserted by the default chain.
   Tool.clearArgumentsAdjusters();
-  Tool.appendArgumentsAdjuster(getClangStripOutputAdjuster());
+
+  // Reset output path if is provided by user.
+  Tool.appendArgumentsAdjuster(
+  Analyze ? [&](const CommandLineArguments &Args, StringRef File) {
+  auto Ret = getClangStripOutputAdjuster()(Args, File);
+  if (!AnalyzerOutput.empty()) {
+Ret.emplace_back("-o");
+Ret.emplace_back(AnalyzerOutput);
+  }
+  return Ret;
+}
+  : getClangStripOutputAdjuster());
+
   Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
 
   // Running the analyzer requires --analyze. Other modes can work with the
Index: clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
===
--- /dev/null
+++ clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
@@ -0,0 +1,15 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+// RUN: cd %t
+// RUN: echo '[{"directory":".","command":"clang++ -c %t/test.cpp -o foo 
-ofoo","file":"%t/test.cpp"}]' | sed -e 's/\\/\//g' > %t/compile_commands.json
+// RUN: cp "%s" "%t/test.cpp"
+// RUN: clang-check -p "%t" "%t/test.cpp" -analyze 
-analyzer-output-path=qwerty -extra-arg=-v -extra-arg=-Xclang 
-extra-arg=-verify 2>&1 | cat - qwerty | FileCheck %s
+// FIXME: Make the above easier.
+
+// CHECK: Invocation
+// CHECK: {{qwerty}}
+// CHECK: DOCTYPE plist
+// CHECK: Division by zero
+int f() {
+  return 1 / 0; // expected-warning {{Division by zero}}
+}


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -76,7 +76,10 @@
 Analyze("analyze",
 cl::desc(Options.getOptionHelpText(options::OPT_analyze)),
 cl::cat(ClangCheckCategory));
-
+static cl::opt
+AnalyzerOutput("analyzer-output-path",
+   cl::desc(Options.getOptionHelpText(options::OPT_o)),
+   cl::cat(ClangCheckCategory));
 static cl::opt
 Fixit("fixit", cl::desc(Options.getOptionHelpText(options::OPT_fixit)),
   cl::cat(ClangCheckCategory));
@@ -206,7 +209,19 @@
 
   // Clear adjusters because -fsyntax-only is inserted by the default chain.
   Tool.clearArgumentsAdjusters();
-  Tool.appendArgumentsAdjuster(getClangStripOutputAdjuster());
+
+  // Reset output path if is provided by user.
+  Tool.appendArgumentsAdjuster(
+  Analyze ? [&](const CommandLineArguments &Args, StringRef File) {
+  auto Ret = getClangStripOutputAdjuster()(Args, File);
+  if (!AnalyzerOutput.empty()) {
+Ret.emplace_back("-o");
+Ret.emplace_back(AnalyzerOutput);
+  }
+  return Ret;
+}
+  : getClangStripOutputAdjuster());
+
   Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
 
   // Running the analyzer requires --analyze. Other modes can work with the
Index: clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
==

[PATCH] D97265: [clang] Allow clang-check to customize analyzer output file or dir name

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 328927.
OikawaKirie added a comment.

Replace concatenating the output from clang-check and plist with separated 
checks to workaround the test failure on Windows.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97265/new/

https://reviews.llvm.org/D97265

Files:
  clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
  clang/tools/clang-check/ClangCheck.cpp


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -76,7 +76,10 @@
 Analyze("analyze",
 cl::desc(Options.getOptionHelpText(options::OPT_analyze)),
 cl::cat(ClangCheckCategory));
-
+static cl::opt
+AnalyzerOutput("analyzer-output-path",
+   cl::desc(Options.getOptionHelpText(options::OPT_o)),
+   cl::cat(ClangCheckCategory));
 static cl::opt
 Fixit("fixit", cl::desc(Options.getOptionHelpText(options::OPT_fixit)),
   cl::cat(ClangCheckCategory));
@@ -206,7 +209,19 @@
 
   // Clear adjusters because -fsyntax-only is inserted by the default chain.
   Tool.clearArgumentsAdjusters();
-  Tool.appendArgumentsAdjuster(getClangStripOutputAdjuster());
+
+  // Reset output path if is provided by user.
+  Tool.appendArgumentsAdjuster(
+  Analyze ? [&](const CommandLineArguments &Args, StringRef File) {
+  auto Ret = getClangStripOutputAdjuster()(Args, File);
+  if (!AnalyzerOutput.empty()) {
+Ret.emplace_back("-o");
+Ret.emplace_back(AnalyzerOutput);
+  }
+  return Ret;
+}
+  : getClangStripOutputAdjuster());
+
   Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
 
   // Running the analyzer requires --analyze. Other modes can work with the
Index: clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
===
--- /dev/null
+++ clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
@@ -0,0 +1,14 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+// RUN: cd %t
+// RUN: echo '[{"directory":".","command":"clang++ -c %t/test.cpp -o foo 
-ofoo","file":"%t/test.cpp"}]' | sed -e 's/\\/\//g' > %t/compile_commands.json
+// RUN: cp "%s" "%t/test.cpp"
+// RUN: echo '// CHECK: {{qwerty}}' > %t/cclog-check
+// RUN: clang-check -p "%t" "%t/test.cpp" -analyze 
-analyzer-output-path=%t/qwerty -extra-arg=-v -extra-arg=-Xclang 
-extra-arg=-verify 2>&1 | FileCheck %t/cclog-check
+// RUN: FileCheck %s --input-file=%t/qwerty
+
+// CHECK: DOCTYPE plist
+// CHECK: Division by zero
+int f() {
+  return 1 / 0; // expected-warning {{Division by zero}}
+}


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -76,7 +76,10 @@
 Analyze("analyze",
 cl::desc(Options.getOptionHelpText(options::OPT_analyze)),
 cl::cat(ClangCheckCategory));
-
+static cl::opt
+AnalyzerOutput("analyzer-output-path",
+   cl::desc(Options.getOptionHelpText(options::OPT_o)),
+   cl::cat(ClangCheckCategory));
 static cl::opt
 Fixit("fixit", cl::desc(Options.getOptionHelpText(options::OPT_fixit)),
   cl::cat(ClangCheckCategory));
@@ -206,7 +209,19 @@
 
   // Clear adjusters because -fsyntax-only is inserted by the default chain.
   Tool.clearArgumentsAdjusters();
-  Tool.appendArgumentsAdjuster(getClangStripOutputAdjuster());
+
+  // Reset output path if is provided by user.
+  Tool.appendArgumentsAdjuster(
+  Analyze ? [&](const CommandLineArguments &Args, StringRef File) {
+  auto Ret = getClangStripOutputAdjuster()(Args, File);
+  if (!AnalyzerOutput.empty()) {
+Ret.emplace_back("-o");
+Ret.emplace_back(AnalyzerOutput);
+  }
+  return Ret;
+}
+  : getClangStripOutputAdjuster());
+
   Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
 
   // Running the analyzer requires --analyze. Other modes can work with the
Index: clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
===
--- /dev/null
+++ clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
@@ -0,0 +1,14 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+// RUN: cd %t
+// RUN: echo '[{"directory":".","command":"clang++ -c %t/test.cpp -o foo -ofoo","file":"%t/test.cpp"}]' | sed -e 's/\\/\//g' > %t/compile_commands.json
+// RUN: cp "%s" "%t/test.cpp"
+// RUN: echo '// CHECK: {{qwerty}}' > %t/cclog-check
+// RUN: clang-check -p "%t" "%t/test.cpp" -analyze -analyzer-output-path=%t/qwerty -extra-arg=-v -extra-arg=-Xclang -extra-arg=-verify 2

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 390653.
OikawaKirie marked an inline comment as done.
OikawaKirie edited the summary of this revision.
OikawaKirie added a comment.
Herald added a subscriber: manas.

It has been a long period since the last discussion, I hope you can still 
remember this bug. And apologize for the delay.

Updated as required, the lookup name generator 
`CrossTranslationUnitContext::getLookupName` and parser `parseCrossTUIndex` are 
modified.
And assertions are added before accessing the CTU index mapping for input 
lookup names to be searched.

Corresponding test cases and documents are also updated.

Please let me know if there are other files to be updated.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

Files:
  clang/docs/analyzer/user-docs/CrossTranslationUnit.rst
  clang/include/clang/Basic/DiagnosticCrossTUKinds.td
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-import.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/Inputs/ctu-other.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.ast-dump.txt
  
clang/test/Analysis/Inputs/plist-macros-with-expansion-ctu.c.externalDefMap.txt
  clang/test/Analysis/ctu-inherited-default-ctor.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/func-mapping-test.cpp
  clang/unittests/CrossTU/CrossTranslationUnitTest.cpp

Index: clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
===
--- clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
+++ clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
@@ -61,7 +61,7 @@
 ASSERT_FALSE(llvm::sys::fs::createTemporaryFile("index", "txt", IndexFD,
 IndexFileName));
 llvm::ToolOutputFile IndexFile(IndexFileName, IndexFD);
-IndexFile.os() << "c:@F@f#I# " << ASTFileName << "\n";
+IndexFile.os() << "9:c:@F@f#I# " << ASTFileName << "\n";
 IndexFile.os().flush();
 EXPECT_TRUE(llvm::sys::fs::exists(IndexFileName));
 
@@ -154,9 +154,9 @@
 
 TEST(CrossTranslationUnit, IndexFormatCanBeParsed) {
   llvm::StringMap Index;
-  Index["a"] = "/b/f1";
-  Index["c"] = "/d/f2";
-  Index["e"] = "/f/f3";
+  Index["1:a"] = "/b/f1";
+  Index["1:c"] = "/d/f2";
+  Index["1:e"] = "/f/f3";
   std::string IndexText = createCrossTUIndexString(Index);
 
   int IndexFD;
Index: clang/test/Analysis/func-mapping-test.cpp
===
--- clang/test/Analysis/func-mapping-test.cpp
+++ clang/test/Analysis/func-mapping-test.cpp
@@ -3,10 +3,10 @@
 int f(int) {
   return 0;
 }
-// CHECK-DAG: c:@F@f#I#
+// CHECK-DAG: 9:c:@F@f#I#
 
 extern const int x = 5;
-// CHECK-DAG: c:@x
+// CHECK-DAG: 4:c:@x
 
 // Non-const variables should not be collected.
 int y = 5;
@@ -18,29 +18,29 @@
   int a;
 };
 extern S const s = {.a = 2};
-// CHECK-DAG: c:@s
+// CHECK-DAG: 4:c:@s
 
 struct SF {
   const int a;
 };
 SF sf = {.a = 2};
-// CHECK-DAG: c:@sf
+// CHECK-DAG: 5:c:@sf
 
 struct SStatic {
   static const int a = 4;
 };
 const int SStatic::a;
-// CHECK-DAG: c:@S@SStatic@a
+// CHECK-DAG: 14:c:@S@SStatic@a
 
 extern int const arr[5] = { 0, 1 };
-// CHECK-DAG: c:@arr
+// CHECK-DAG: 6:c:@arr
 
 union U {
   const int a;
   const unsigned int b;
 };
 U u = {.a = 6};
-// CHECK-DAG: c:@u
+// CHECK-DAG: 4:c:@u
 
 // No USR can be generated for this.
 // Check for no crash in this case.
Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,22 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: cp %s %t/trigger.cpp
+// RUN: cp %S/Inputs/ctu-lookup-name-with-space.cpp %t/importee.cpp
+// RUN: echo '"%t/importee.cpp" : ["g++", "-c", "%t/importee.cpp"]' > %t/invocations.yaml
+// RUN: echo '[{"directory": "%t", "command": "g++ -c %t/importee.cpp", "file": "%t/importee.cpp"}, {"directory": "%t", "command": "g++ -c %t/trigger.cpp", "file": "%t/trigger.cpp"}]' > %t/compile_commands.json
+
+// RUN: %clang_extdef_map -p %t %t/importee.cpp > %t/externalDefMap.txt
+
+// RUN: cd %t && %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   -verify trigger.cpp 2>&1 | FileCheck --allow-empty importee.cpp
+
+int importee(int);
+
+void trigger() {
+  importee(0); // expected-warn...@importee.cpp:13 {{Division by zero}}
+}
Index: clang/test/Analysis/ctu-inherited-default-ctor.cpp
===
--- clang/test/Analysis/ctu-inherited-default-ctor.cpp
+++ clang/test/Analysis/ctu-in

[PATCH] D102149: [analyzer][ctu] Allow loading invocation list from a compilation database automatically detected from the ctu-dir

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie abandoned this revision.
OikawaKirie added a comment.
Herald added a subscriber: manas.

It seems that this issue has better solutions outside the CTU module, I will 
drop this revision as mentioned above.
Sorry for the delay.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102149/new/

https://reviews.llvm.org/D102149

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 391968.
OikawaKirie added a comment.

1. Fix formatting bugs
2. Update lookup name format as `: ` in all 
comments and documents
3. Add the new test case as a part of 
`clang/test/Analysis/func-mapping-test.cpp` to verify the lookup name
4. Change the macros for asserting the lookup name format to an `NDEBUG` 
wrapped function


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

Files:
  clang/docs/analyzer/user-docs/CrossTranslationUnit.rst
  clang/include/clang/Basic/DiagnosticCrossTUKinds.td
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-import.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/Inputs/ctu-other.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.ast-dump.txt
  
clang/test/Analysis/Inputs/plist-macros-with-expansion-ctu.c.externalDefMap.txt
  clang/test/Analysis/ctu-inherited-default-ctor.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/func-mapping-test.cpp
  clang/unittests/CrossTU/CrossTranslationUnitTest.cpp

Index: clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
===
--- clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
+++ clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
@@ -61,7 +61,7 @@
 ASSERT_FALSE(llvm::sys::fs::createTemporaryFile("index", "txt", IndexFD,
 IndexFileName));
 llvm::ToolOutputFile IndexFile(IndexFileName, IndexFD);
-IndexFile.os() << "c:@F@f#I# " << ASTFileName << "\n";
+IndexFile.os() << "9:c:@F@f#I# " << ASTFileName << "\n";
 IndexFile.os().flush();
 EXPECT_TRUE(llvm::sys::fs::exists(IndexFileName));
 
@@ -154,9 +154,9 @@
 
 TEST(CrossTranslationUnit, IndexFormatCanBeParsed) {
   llvm::StringMap Index;
-  Index["a"] = "/b/f1";
-  Index["c"] = "/d/f2";
-  Index["e"] = "/f/f3";
+  Index["1:a"] = "/b/f1";
+  Index["1:c"] = "/d/f2";
+  Index["1:e"] = "/f/f3";
   std::string IndexText = createCrossTUIndexString(Index);
 
   int IndexFD;
Index: clang/test/Analysis/func-mapping-test.cpp
===
--- clang/test/Analysis/func-mapping-test.cpp
+++ clang/test/Analysis/func-mapping-test.cpp
@@ -3,10 +3,10 @@
 int f(int) {
   return 0;
 }
-// CHECK-DAG: c:@F@f#I#
+// CHECK-DAG: 9:c:@F@f#I#
 
 extern const int x = 5;
-// CHECK-DAG: c:@x
+// CHECK-DAG: 4:c:@x
 
 // Non-const variables should not be collected.
 int y = 5;
@@ -18,29 +18,29 @@
   int a;
 };
 extern S const s = {.a = 2};
-// CHECK-DAG: c:@s
+// CHECK-DAG: 4:c:@s
 
 struct SF {
   const int a;
 };
 SF sf = {.a = 2};
-// CHECK-DAG: c:@sf
+// CHECK-DAG: 5:c:@sf
 
 struct SStatic {
   static const int a = 4;
 };
 const int SStatic::a;
-// CHECK-DAG: c:@S@SStatic@a
+// CHECK-DAG: 14:c:@S@SStatic@a
 
 extern int const arr[5] = { 0, 1 };
-// CHECK-DAG: c:@arr
+// CHECK-DAG: 6:c:@arr
 
 union U {
   const int a;
   const unsigned int b;
 };
 U u = {.a = 6};
-// CHECK-DAG: c:@u
+// CHECK-DAG: 4:c:@u
 
 // No USR can be generated for this.
 // Check for no crash in this case.
@@ -48,3 +48,15 @@
   float uf;
   const int ui;
 };
+
+void f(int (*)(char));
+void f(bool (*)(char));
+
+struct G {
+  G() {
+f([](char) -> int { return 42; });
+// CHECK-DAG: 41:c:@S@G@F@G#@Sa@F@operator int (*)(char)#1
+f([](char) -> bool { return true; });
+// CHECK-DAG: 42:c:@S@G@F@G#@Sa@F@operator bool (*)(char)#1
+  }
+};
Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,22 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: cp %s %t/trigger.cpp
+// RUN: cp %S/Inputs/ctu-lookup-name-with-space.cpp %t/importee.cpp
+// RUN: echo '"%t/importee.cpp" : ["g++", "-c", "%t/importee.cpp"]' > %t/invocations.yaml
+// RUN: echo '[{"directory": "%t", "command": "g++ -c %t/importee.cpp", "file": "%t/importee.cpp"}, {"directory": "%t", "command": "g++ -c %t/trigger.cpp", "file": "%t/trigger.cpp"}]' > %t/compile_commands.json
+
+// RUN: %clang_extdef_map -p %t %t/importee.cpp > %t/externalDefMap.txt
+
+// RUN: cd %t && %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   -verify trigger.cpp 2>&1 | FileCheck --allow-empty importee.cpp
+
+int importee(int);
+
+void trigger() {
+  importee(0); // expected-warn...@importee.cpp:13 {{Division by zero}}
+}
Index: clang/test/Analysis/ctu-inherited-default-ctor.cpp
===
--- cla

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added inline comments.



Comment at: clang/lib/CrossTU/CrossTranslationUnit.cpp:154-172
+  // Find the length delimiter.
+  const size_t LengthDelimiter = LineRef.find(':');
+  if (StringRef::npos == LengthDelimiter)
+return false;
+
+  // Parse the length of LookupName as USRLength.
+  size_t USRLength = 0;

steakhal wrote:
> 
The source of lookup name of the function being imported is function 
`CrossTranslationUnitContext::getLookupName`. Keeping the length in the mapping 
can avoid parsing the lookup name during importing.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102614: [index] Add support for type of pointers to class members

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

ping

What do you think about this patch? Can it be landed now? Or I should debug the 
crash in the Windows version detected with the previous version of my patch.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102614/new/

https://reviews.llvm.org/D102614

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added inline comments.



Comment at: clang/lib/CrossTU/CrossTranslationUnit.cpp:154-172
+  // Find the length delimiter.
+  const size_t LengthDelimiter = LineRef.find(':');
+  if (StringRef::npos == LengthDelimiter)
+return false;
+
+  // Parse the length of LookupName as USRLength.
+  size_t USRLength = 0;

steakhal wrote:
> OikawaKirie wrote:
> > steakhal wrote:
> > > 
> > The source of lookup name of the function being imported is function 
> > `CrossTranslationUnitContext::getLookupName`. Keeping the length in the 
> > mapping can avoid parsing the lookup name during importing.
> Okay; you can copy the original StringRef to have that. But by consuming it 
> on one path makes the code much more readable.
> 
The `getAsInterger` call can also check whether the content before the first 
colon is an integer. Therefore, a sub-string operation is required here.



Comment at: clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp:7-8
+// CHECK-NOT: multiple definitions are found for the same key in index
+f([](char) -> int { return 42; });
+f([](char) -> bool { return true; });
+  }

steakhal wrote:
> I would rather put these into the `importee()`
The lambda exprs will not be included in the CTU index file if they are 
declared in a normal function.



Comment at: clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp:13
+int importee(int X) {
+  return 1 / X;
+}

steakhal wrote:
> Why do you need to have a div by zero warning?
I am not sure whether we should test if an external function can be correctly 
imported. Hence, I wrote a div-by-zero bug here. A call to function 
`clang_analyzer_warnIfReached` is also OK here.

As the imported lambda expr will not be called, I think I can only test whether 
CTU works via another normal function.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added inline comments.



Comment at: clang/lib/CrossTU/CrossTranslationUnit.cpp:154-172
+  // Find the length delimiter.
+  const size_t LengthDelimiter = LineRef.find(':');
+  if (StringRef::npos == LengthDelimiter)
+return false;
+
+  // Parse the length of LookupName as USRLength.
+  size_t USRLength = 0;

steakhal wrote:
> OikawaKirie wrote:
> > steakhal wrote:
> > > OikawaKirie wrote:
> > > > steakhal wrote:
> > > > > 
> > > > The source of lookup name of the function being imported is function 
> > > > `CrossTranslationUnitContext::getLookupName`. Keeping the length in the 
> > > > mapping can avoid parsing the lookup name during importing.
> > > Okay; you can copy the original StringRef to have that. But by consuming 
> > > it on one path makes the code much more readable.
> > > 
> > The `getAsInterger` call can also check whether the content before the 
> > first colon is an integer. Therefore, a sub-string operation is required 
> > here.
> I don't doubt that your proposed way of doing this works and is efficient.
> What I say is that I think there is room for improvement in the 
> non-functional aspects, in the readability. However, it's not really a 
> blocking issue, more of a personal taste.
I know what you are considering, it is clearer and more readable by consuming 
the length, then the USR. However, to correctly separate the USR and file path, 
the length of `USR-Length` is also required, which makes it impossible to just 
*consume* the length at the beginning.

Another way of solving this problem is to re-create the string with the 
USR-Length and the USR after parsing, but I think it is not a good solution.

BTW, is it necessary to assert the `USR-Length` to be greater than zero? I 
think it is more reasonable to report *invalid format* rather than assert the 
value, as it can be provided by the user.



Comment at: clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp:13
+int importee(int X) {
+  return 1 / X;
+}

steakhal wrote:
> OikawaKirie wrote:
> > steakhal wrote:
> > > Why do you need to have a div by zero warning?
> > I am not sure whether we should test if an external function can be 
> > correctly imported. Hence, I wrote a div-by-zero bug here. A call to 
> > function `clang_analyzer_warnIfReached` is also OK here.
> > 
> > As the imported lambda expr will not be called, I think I can only test 
> > whether CTU works via another normal function.
> AFAIK importing a function and import-related stuff are orthogonal to 
> actually emitting bug reports produced by the analyzer. That being said, if 
> the `importee()` would have an empty body, the observable behavior would 
> remain the same. And this is what I propose now.
Sorry, but I am not quite clear about your suggestions on this function.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D155445: [analyzer][docs] Add CSA release notes

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie accepted this revision.
OikawaKirie added a comment.

LGTM for my part. Thx.

Since I am not very familiar with other changes, I have no detailed suggestions 
for the order.




Comment at: clang/docs/ReleaseNotes.rst:922-923
+- The ``CStringChecker`` will invalidate less if the copy operation is
+  inferable to be bounded. For example, if the argument of ``strcpy`` is known
+  to be of certain length and that is in-bounds.
+

The lengths of both src and dst buffers need to be known.



Comment at: clang/docs/ReleaseNotes.rst:937
+
+  Similarly, functions like ``strsep`` now won't invalidate the source buffer,
+  because it can never overflow.

I think this may be a typo here, as we do not invalidate the source buffer 
originally.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D155445/new/

https://reviews.llvm.org/D155445

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 536578.
OikawaKirie marked 12 inline comments as done and an inline comment as not done.
OikawaKirie added a comment.

1. Function and variable names: functions: lower camel, variables: upper camel, 
lambdas: upper camel
2. std::function -> llvm::function_ref
3. update test case verification directions by following others in this file.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D152435/new/

https://reviews.llvm.org/D152435

Files:
  clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp
  clang/test/Analysis/Inputs/system-header-simulator.h
  clang/test/Analysis/issue-55019.cpp
  clang/test/Analysis/pr22954.c

Index: clang/test/Analysis/pr22954.c
===
--- clang/test/Analysis/pr22954.c
+++ clang/test/Analysis/pr22954.c
@@ -556,12 +556,13 @@
   x263.s2 = strdup("hello");
   char input[] = {'a', 'b', 'c', 'd'};
   memcpy(x263.s1, input, *(len + n));
-  clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}
+  clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}\
+  expected-warning{{Potential leak of memory pointed to by 'x263.s2'}}
   clang_analyzer_eval(x263.s1[1] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[2] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[3] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s2 == 0); // expected-warning{{UNKNOWN}}
-  return 0; // expected-warning{{Potential leak of memory pointed to by 'x263.s2'}}
+  return 0;
 }
 
 
Index: clang/test/Analysis/issue-55019.cpp
===
--- /dev/null
+++ clang/test/Analysis/issue-55019.cpp
@@ -0,0 +1,89 @@
+// Refer issue 55019 for more details.
+// A supplemental test case of pr22954.c for other functions modeled in
+// the CStringChecker.
+
+// RUN: %clang_analyze_cc1 %s -verify \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-checker=unix \
+// RUN:   -analyzer-checker=debug.ExprInspection
+
+#include "Inputs/system-header-simulator.h"
+#include "Inputs/system-header-simulator-cxx.h"
+
+void *malloc(size_t);
+void free(void *);
+
+struct mystruct {
+  void *ptr;
+  char arr[4];
+};
+
+void clang_analyzer_dump(const void *);
+
+// CStringChecker::memsetAux
+void fmemset() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memset(x.arr, 0, sizeof(x.arr));
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalCopyCommon
+void fmemcpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memcpy(x.arr, "hi", 2);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStrcpyCommon
+void fstrcpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  strcpy(x.arr, "hi");
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+void fstrncpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  strncpy(x.arr, "hi", sizeof(x.arr));
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStrsep
+void fstrsep() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  char *p = x.arr;
+  (void)strsep(&p, "x");
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStdCopyCommon
+void fstdcopy() {
+  mystruct x;
+  x.ptr = new char;
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+
+  const char *p = "x";
+  std::copy(p, p + 1, x.arr);
+
+  // FIXME: As we currently cannot know whether the copy overflows, the checker
+  // invalidates the entire `x` object. When the copy size through iterators
+  // can be correctly modeled, we can then update the verify direction from
+  // SymRegion to HeapSymRegion as this std::copy call never overflows and
+  // hence the pointer `x.ptr` shall not be invalidated.
+  clang_analyzer_dump(x.ptr);   // expected-warning {{SymRegion}}
+  delete static_cast(x.ptr); // no-leak-warning
+}
Index: clang/test/Analysis/Inputs/system-header-simulator.h
===
--- clang/test/Analysis/Inputs/system-header-simulator.h
+++ clang/test/Analysis/Inputs/system-header-simulator.h
@@ -63,7 +63,9 @@
 
 char *strcpy(char *restrict, const char *restrict);
 char *strncpy(char *dst, const char *src, size_t n);
+char *strsep(char **stringp, const char *delim);
 void *memcp

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added inline comments.



Comment at: clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp:1054
+SVal SizeV, QualType SizeTy) {
+  return InvalidateBufferAux(
+  C, S, BufE, BufV,

steakhal wrote:
> OikawaKirie wrote:
> > steakhal wrote:
> > > Shouldn't we assert that `SizeV` is some known value or at least smaller 
> > > than (or equal to) the extent of the buffer?
> > The calls in `CStringChecker::evalCopyCommon` and 
> > `CStringChecker::evalStrcpyCommon` seem to have chances able to pass an 
> > unknown value. But I am not very sure about this.
> > 
> > If the `SizeV` is sure to be smaller or equal to the extent of the buffer, 
> > the `IsFirstBufInBound` check seems useless.
> > Maybe I need to dig deeper into the callers.
> Indeed.
Function `CStringChecker::CheckBufferAccess` is responsible to check whether 
the size overflows.
However, when `Filter.CheckCStringOutOfBounds` is set to false, the function 
will skip the overflow checking.
This makes that we cannot assert SizeV is always inbound, and the check in 
`IsFirstBufInBound` seems to be necessary.
(`evalMemset` -> `CheckBufferAccess`(skip overflow checking) -> `memsetAux` -> 
`InvalidateDestinationBufferBySize` with function call of `memset(x.arr, 'a', 
42)`)

Besides, if the size argument is originally an unknown value expression (e.g. 
`(unsigned char) 1.0`), the unknown value can be passed to this function.
(`CStringChecker::evalCopyCommon` -> `invalidateDestinationBufferBySize`, with 
function call of `memcpy(x.arr, "hi", (unsigned char) 1.0)`)

Hence, we cannot make such an assertion here. : (


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D152435/new/

https://reviews.llvm.org/D152435

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 536662.
OikawaKirie added a comment.

Sorry for the formatting error... :(


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D152435/new/

https://reviews.llvm.org/D152435

Files:
  clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp
  clang/test/Analysis/Inputs/system-header-simulator.h
  clang/test/Analysis/issue-55019.cpp
  clang/test/Analysis/pr22954.c

Index: clang/test/Analysis/pr22954.c
===
--- clang/test/Analysis/pr22954.c
+++ clang/test/Analysis/pr22954.c
@@ -556,12 +556,13 @@
   x263.s2 = strdup("hello");
   char input[] = {'a', 'b', 'c', 'd'};
   memcpy(x263.s1, input, *(len + n));
-  clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}
+  clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}\
+  expected-warning{{Potential leak of memory pointed to by 'x263.s2'}}
   clang_analyzer_eval(x263.s1[1] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[2] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[3] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s2 == 0); // expected-warning{{UNKNOWN}}
-  return 0; // expected-warning{{Potential leak of memory pointed to by 'x263.s2'}}
+  return 0;
 }
 
 
Index: clang/test/Analysis/issue-55019.cpp
===
--- /dev/null
+++ clang/test/Analysis/issue-55019.cpp
@@ -0,0 +1,89 @@
+// Refer issue 55019 for more details.
+// A supplemental test case of pr22954.c for other functions modeled in
+// the CStringChecker.
+
+// RUN: %clang_analyze_cc1 %s -verify \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-checker=unix \
+// RUN:   -analyzer-checker=debug.ExprInspection
+
+#include "Inputs/system-header-simulator.h"
+#include "Inputs/system-header-simulator-cxx.h"
+
+void *malloc(size_t);
+void free(void *);
+
+struct mystruct {
+  void *ptr;
+  char arr[4];
+};
+
+void clang_analyzer_dump(const void *);
+
+// CStringChecker::memsetAux
+void fmemset() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memset(x.arr, 0, sizeof(x.arr));
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalCopyCommon
+void fmemcpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memcpy(x.arr, "hi", 2);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStrcpyCommon
+void fstrcpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  strcpy(x.arr, "hi");
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+void fstrncpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  strncpy(x.arr, "hi", sizeof(x.arr));
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStrsep
+void fstrsep() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  char *p = x.arr;
+  (void)strsep(&p, "x");
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStdCopyCommon
+void fstdcopy() {
+  mystruct x;
+  x.ptr = new char;
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+
+  const char *p = "x";
+  std::copy(p, p + 1, x.arr);
+
+  // FIXME: As we currently cannot know whether the copy overflows, the checker
+  // invalidates the entire `x` object. When the copy size through iterators
+  // can be correctly modeled, we can then update the verify direction from
+  // SymRegion to HeapSymRegion as this std::copy call never overflows and
+  // hence the pointer `x.ptr` shall not be invalidated.
+  clang_analyzer_dump(x.ptr);   // expected-warning {{SymRegion}}
+  delete static_cast(x.ptr); // no-leak-warning
+}
Index: clang/test/Analysis/Inputs/system-header-simulator.h
===
--- clang/test/Analysis/Inputs/system-header-simulator.h
+++ clang/test/Analysis/Inputs/system-header-simulator.h
@@ -63,7 +63,9 @@
 
 char *strcpy(char *restrict, const char *restrict);
 char *strncpy(char *dst, const char *src, size_t n);
+char *strsep(char **stringp, const char *delim);
 void *memcpy(void *dst, const void *src, size_t n);
+void *memset(void *s, int c, size_t n);
 
 typedef unsigned long __darwin_pthread_key_t;
 typedef __darwin_pthread_key_t pthread_key_t;
Index: clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp
=

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

I will submit it again later to update the code as you suggested.

Besides,

> Build Status
> Buildable 242737  
> Build 376801: pre-merge checkslibcxx CI FAILED

What does this mean? Was this patch correctly compiled and checked in the 
previous build? How can I avoid this failure?

Thx


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D152435/new/

https://reviews.llvm.org/D152435

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rG1bd2d335b649: [analyzer][CStringChecker] Adjust the 
invalidation operation on the super… (authored by OikawaKirie).

Changed prior to commit:
  https://reviews.llvm.org/D152435?vs=536662&id=536683#toc

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D152435/new/

https://reviews.llvm.org/D152435

Files:
  clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp
  clang/test/Analysis/Inputs/system-header-simulator.h
  clang/test/Analysis/issue-55019.cpp
  clang/test/Analysis/pr22954.c

Index: clang/test/Analysis/pr22954.c
===
--- clang/test/Analysis/pr22954.c
+++ clang/test/Analysis/pr22954.c
@@ -556,12 +556,13 @@
   x263.s2 = strdup("hello");
   char input[] = {'a', 'b', 'c', 'd'};
   memcpy(x263.s1, input, *(len + n));
-  clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}
+  clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}\
+  expected-warning{{Potential leak of memory pointed to by 'x263.s2'}}
   clang_analyzer_eval(x263.s1[1] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[2] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[3] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s2 == 0); // expected-warning{{UNKNOWN}}
-  return 0; // expected-warning{{Potential leak of memory pointed to by 'x263.s2'}}
+  return 0;
 }
 
 
Index: clang/test/Analysis/issue-55019.cpp
===
--- /dev/null
+++ clang/test/Analysis/issue-55019.cpp
@@ -0,0 +1,89 @@
+// Refer issue 55019 for more details.
+// A supplemental test case of pr22954.c for other functions modeled in
+// the CStringChecker.
+
+// RUN: %clang_analyze_cc1 %s -verify \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-checker=unix \
+// RUN:   -analyzer-checker=debug.ExprInspection
+
+#include "Inputs/system-header-simulator.h"
+#include "Inputs/system-header-simulator-cxx.h"
+
+void *malloc(size_t);
+void free(void *);
+
+struct mystruct {
+  void *ptr;
+  char arr[4];
+};
+
+void clang_analyzer_dump(const void *);
+
+// CStringChecker::memsetAux
+void fmemset() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memset(x.arr, 0, sizeof(x.arr));
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalCopyCommon
+void fmemcpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memcpy(x.arr, "hi", 2);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStrcpyCommon
+void fstrcpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  strcpy(x.arr, "hi");
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+void fstrncpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  strncpy(x.arr, "hi", sizeof(x.arr));
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStrsep
+void fstrsep() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  char *p = x.arr;
+  (void)strsep(&p, "x");
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStdCopyCommon
+void fstdcopy() {
+  mystruct x;
+  x.ptr = new char;
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+
+  const char *p = "x";
+  std::copy(p, p + 1, x.arr);
+
+  // FIXME: As we currently cannot know whether the copy overflows, the checker
+  // invalidates the entire `x` object. When the copy size through iterators
+  // can be correctly modeled, we can then update the verify direction from
+  // SymRegion to HeapSymRegion as this std::copy call never overflows and
+  // hence the pointer `x.ptr` shall not be invalidated.
+  clang_analyzer_dump(x.ptr);   // expected-warning {{SymRegion}}
+  delete static_cast(x.ptr); // no-leak-warning
+}
Index: clang/test/Analysis/Inputs/system-header-simulator.h
===
--- clang/test/Analysis/Inputs/system-header-simulator.h
+++ clang/test/Analysis/Inputs/system-header-simulator.h
@@ -63,7 +63,9 @@
 
 char *strcpy(char *restrict, const char *restrict);
 char *strncpy(char *dst, const char *src, size_t n);
+char *strsep(char **stringp, const char *delim);
 vo

[PATCH] D155445: [analyzer][docs] Add CSA release notes

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

The key idea of my commit 1bd2d335b649 
:

- For string APIs that will not provide the copy length (`strcpy`), we will use 
the buffer decl and literal length to infer whether it overflows. If the copy 
operation does not overflow, we will now only invalidate the buffer string 
being copied to.
- For string APIs that never overflow (`strsep`), we will always invalidate the 
target buffer only.
- For those that we cannot correctly handle now (`std::copy`), we will also 
invalidate the base region and make all pointers in the base region escape.

Hence,
For `strcpy`s, we infer through buffer size and string literals.
For `strsep`, we believe it never overflows through its functionality 
specification. It is also an inference.

Whereas for `memcpy` where the copy length is given in arguments, the 
non-inferring circumstances, it was implemented previously in patch D12571 
, not a part of my changes.




Comment at: clang/docs/ReleaseNotes.rst:920-922
+- The ``CStringChecker`` will invalidate less if the copy operation is bounded.
+  (`1bd2d335b649 `_)
+  (`#55019 `_)

One tiny change to the abstraction.
The ``CStringChecker`` will invalidate less if the copy operation is 
**inferable to be** bounded.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D155445/new/

https://reviews.llvm.org/D155445

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

BTW, what does the `Done` checkbox mean in the code comments?




Comment at: clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp:286-287
+  SVal V,
+  std::function);
 

steakhal wrote:
> I'd highly suggest making this a template taking the functor that way.
> Given that we modify these functions, we should make them comply with the 
> lower camel case naming convention.
> And their variables with upper camel case.
> I'd highly suggest making this a template taking the functor that way.

Any examples?
Do you mean
```
template 
static ProgramStateRef
InvalidateBufferAux(CheckerContext &C, ProgramStateRef state, const Expr *Ex,
SVal V, std::function CallBack);
```



Comment at: clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp:1054
+SVal SizeV, QualType SizeTy) {
+  return InvalidateBufferAux(
+  C, S, BufE, BufV,

steakhal wrote:
> Shouldn't we assert that `SizeV` is some known value or at least smaller than 
> (or equal to) the extent of the buffer?
The calls in `CStringChecker::evalCopyCommon` and 
`CStringChecker::evalStrcpyCommon` seem to have chances able to pass an unknown 
value. But I am not very sure about this.

If the `SizeV` is sure to be smaller or equal to the extent of the buffer, the 
`IsFirstBufInBound` check seems useless.
Maybe I need to dig deeper into the callers.



Comment at: clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp:1075-1077
+  [](RegionAndSymbolInvalidationTraits &, const MemRegion *R) {
+return MemRegion::FieldRegionKind == R->getKind();
+  });

steakhal wrote:
> The lambdas inline in the call look a bit messy. I would prefer declaring it 
> and passing it as an argument separately.
> This applies to the rest of the lambdas as well.
> And their variables with upper camel case.

What about the variable names for the lambdas? Lower camel or upper camel?
I cannot find the rules for named lambdas from the guidance 
.



Comment at: clang/test/Analysis/issue-55019.cpp:80-87
+  std::copy(p, p + 1, x.arr);
+
+  // FIXME: As we currently cannot know whether the copy overflows, the checker
+  // invalidates the entire `x` object. When the copy size through iterators
+  // can be correctly modeled, we can then update the verify direction from
+  // SymRegion to HeapSymRegion as this std::copy call never overflows and
+  // hence the pointer `x.ptr` shall not be invalidated.

> So, you say that It should be `HeapSymRegion` instead because `x.arr` 
> shouldn't be invalidated because the `std::copy` won't modify it. Right?

Yes, but just in this test case.

The size of `x.arr` is 4 and the copy length is 1, which means the call to 
`std::copy` here never overflows.
Therefore, we should not invalidate the super region `x` and keep `x.ptr` still 
pointing to the original `HeapSymRegion` unchanged.
However, the copy length computed through iterators is not modeled currently in 
`CStringChecker::evalStdCopyCommon`, where 
`InvalidateDestinationBufferAlwaysEscapeSuperRegion` is called.
When it is modeled in the future, the function should call 
`InvalidateDestinationBufferBySize` instead to make the invalidation of the 
super region determined by the copy length to report potential leaks.



Comment at: clang/test/Analysis/pr22954.c:560
   clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}
+  // expected-warning@-1{{Potential leak of memory pointed to by 'x263.s2'}}
   clang_analyzer_eval(x263.s1[1] == 0); // expected-warning{{UNKNOWN}}

steakhal wrote:
> Ah, the leak should have been reported at the end of the full-expression of 
> `memcpy`.
> If we have this expect line here it could mislead the reader that it has 
> anything to do with the `eval` call before.
> 
> In fact, any statement after `memcpy` would have this diagnostic.
> I have seen cases where we have a `next_line()` opaque call after such places 
> to anchor the leak diagnostic.
> I think we can do the same here.
> Its not advised to have different, but similar looking hunks out of which 
> some are the consequence of the semantic change we made and also have others 
> where they are just refactors.

Do you mean I only need to update the verification direction here as you 
suggested above and leave others unchanged?


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D152435/new/

https://reviews.llvm.org/D152435

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: pgousseau, NoQ, steakhal, balazske, xazax.hun.
OikawaKirie added a project: clang.
Herald added subscribers: manas, ASDenysPetrov, martong, dkrupp, donat.nagy, 
Szelethus, mikhail.ramalho, a.sidorin, szepet, baloghadamsoftware.
Herald added a project: All.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

Fixing GitHub issue: https://github.com/llvm/llvm-project/issues/55019
Following the previous fix https://reviews.llvm.org/D12571 on issue 
https://github.com/llvm/llvm-project/issues/23328

The two issues report false memory leaks after calling string-copy APIs with a 
buffer field in an object as the destination.
The buffer invalidation incorrectly drops the assignment to a heap memory block 
when no overflow problems happen.
And the pointer of the dropped assignment is declared in the same object of the 
destination buffer.

The previous fix only considers the `memcpy` functions whose copy length is 
available from arguments.
In this issue, the copy length is inferable from the buffer declaration and 
string literals being copied.
Therefore, I have adjusted the previous fix to reuse the copy length computed 
before.

Besides, for APIs that never overflow (strsep) or we never know whether they 
can overflow (std::copy),
new flags have been introduced to inform CStringChecker::InvalidateBuffer 
whether or not to
invalidate the super region that encompasses the destination buffer.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D152435

Files:
  clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp
  clang/test/Analysis/Inputs/system-header-simulator.h
  clang/test/Analysis/issue-55019.c
  clang/test/Analysis/issue-55019.cpp
  clang/test/Analysis/pr22954.c

Index: clang/test/Analysis/pr22954.c
===
--- clang/test/Analysis/pr22954.c
+++ clang/test/Analysis/pr22954.c
@@ -557,11 +557,12 @@
   char input[] = {'a', 'b', 'c', 'd'};
   memcpy(x263.s1, input, *(len + n));
   clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}
+  // expected-warning@-1{{Potential leak of memory pointed to by 'x263.s2'}}
   clang_analyzer_eval(x263.s1[1] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[2] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[3] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s2 == 0); // expected-warning{{UNKNOWN}}
-  return 0; // expected-warning{{Potential leak of memory pointed to by 'x263.s2'}}
+  return 0;
 }
 
 
Index: clang/test/Analysis/issue-55019.cpp
===
--- /dev/null
+++ clang/test/Analysis/issue-55019.cpp
@@ -0,0 +1,29 @@
+// Refer issue 55019 for more details.
+
+// RUN: %clang_analyze_cc1 %s -verify \
+// RUN:   -analyzer-checker=core,debug.ExprInspection
+
+#include "Inputs/system-header-simulator-cxx.h"
+
+struct mystruct {
+  char *ptr;
+  char arr[4];
+};
+
+void clang_analyzer_dump(const void *);
+
+// CStringChecker::evalStdCopyCommon
+void fstdcopy() {
+  mystruct x;
+  x.ptr = new char;
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+
+  const char *p = "x";
+  std::copy(p, p + 1, x.arr);
+
+  // FIXME: As we cannot know whether the copy overflows, we will invalidate the
+  // entire object. Modify the verify direction from SymRegion to HeapSymRegion
+  // when the size if modeled in CStringChecker.
+  clang_analyzer_dump(x.ptr); // expected-warning {{SymRegion}}
+  delete x.ptr;   // no-warning
+}
Index: clang/test/Analysis/issue-55019.c
===
--- /dev/null
+++ clang/test/Analysis/issue-55019.c
@@ -0,0 +1,69 @@
+// Refer issue 55019 for more details.
+
+// RUN: %clang_analyze_cc1 %s -verify -Wno-strict-prototypes \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-checker=unix \
+// RUN:   -analyzer-checker=debug.ExprInspection
+
+#include "Inputs/system-header-simulator.h"
+void *malloc(size_t);
+void free(void *);
+
+#define SIZE 4
+
+struct mystruct {
+  void *ptr;
+  char arr[SIZE];
+};
+
+void clang_analyzer_dump(const void *);
+
+// CStringChecker::memsetAux
+void fmemset() {
+  struct mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memset(x.arr, 0, SIZE);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-warning
+}
+
+// CStringChecker::evalCopyCommon
+void fmemcpy() {
+  struct mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memcpy(x.arr, "hi", 2);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-warning
+}
+
+// CStringChecker::evalStrcpyCommon
+void fstrcpy() {
+  struct mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.p

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 532372.
OikawaKirie edited the summary of this revision.
OikawaKirie added a comment.

Update the implementation of InvalidateBuffer in a multi-entrance-with-callback 
manner.
Update test cases as suggested.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D152435/new/

https://reviews.llvm.org/D152435

Files:
  clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp
  clang/test/Analysis/Inputs/system-header-simulator.h
  clang/test/Analysis/issue-55019.cpp
  clang/test/Analysis/pr22954.c

Index: clang/test/Analysis/pr22954.c
===
--- clang/test/Analysis/pr22954.c
+++ clang/test/Analysis/pr22954.c
@@ -557,11 +557,12 @@
   char input[] = {'a', 'b', 'c', 'd'};
   memcpy(x263.s1, input, *(len + n));
   clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}
+  // expected-warning@-1{{Potential leak of memory pointed to by 'x263.s2'}}
   clang_analyzer_eval(x263.s1[1] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[2] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[3] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s2 == 0); // expected-warning{{UNKNOWN}}
-  return 0; // expected-warning{{Potential leak of memory pointed to by 'x263.s2'}}
+  return 0;
 }
 
 
Index: clang/test/Analysis/issue-55019.cpp
===
--- /dev/null
+++ clang/test/Analysis/issue-55019.cpp
@@ -0,0 +1,89 @@
+// Refer issue 55019 for more details.
+// A supplemental test case of pr22954.c for other functions modeled in
+// the CStringChecker.
+
+// RUN: %clang_analyze_cc1 %s -verify \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-checker=unix \
+// RUN:   -analyzer-checker=debug.ExprInspection
+
+#include "Inputs/system-header-simulator.h"
+#include "Inputs/system-header-simulator-cxx.h"
+
+void *malloc(size_t);
+void free(void *);
+
+struct mystruct {
+  void *ptr;
+  char arr[4];
+};
+
+void clang_analyzer_dump(const void *);
+
+// CStringChecker::memsetAux
+void fmemset() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memset(x.arr, 0, sizeof(x.arr));
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalCopyCommon
+void fmemcpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memcpy(x.arr, "hi", 2);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStrcpyCommon
+void fstrcpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  strcpy(x.arr, "hi");
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+void fstrncpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  strncpy(x.arr, "hi", sizeof(x.arr));
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStrsep
+void fstrsep() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  char *p = x.arr;
+  (void)strsep(&p, "x");
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStdCopyCommon
+void fstdcopy() {
+  mystruct x;
+  x.ptr = new char;
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+
+  const char *p = "x";
+  std::copy(p, p + 1, x.arr);
+
+  // FIXME: As we currently cannot know whether the copy overflows, the checker
+  // invalidates the entire `x` object. When the copy size through iterators
+  // can be correctly modeled, we can then update the verify direction from
+  // SymRegion to HeapSymRegion as this std::copy call never overflows and
+  // hence the pointer `x.ptr` shall not be invalidated.
+  clang_analyzer_dump(x.ptr);   // expected-warning {{SymRegion}}
+  delete static_cast(x.ptr); // no-leak-warning
+}
Index: clang/test/Analysis/Inputs/system-header-simulator.h
===
--- clang/test/Analysis/Inputs/system-header-simulator.h
+++ clang/test/Analysis/Inputs/system-header-simulator.h
@@ -63,7 +63,9 @@
 
 char *strcpy(char *restrict, const char *restrict);
 char *strncpy(char *dst, const char *src, size_t n);
+char *strsep(char **stringp, const char *delim);
 void *memcpy(void *dst, const void *src, size_t n);
+void *memset(void *s, int c, size_t n);
 
 typedef unsigned long __darwin_pthread_key_t;
 typedef __darwin_pthread_key_t pthread_key_t;
Index: clang/lib/StaticAnalyzer/Checkers/CStringC

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added inline comments.



Comment at: clang/test/Analysis/issue-55019.cpp:13-14
+
+void *malloc(size_t);
+void free(void *);
+

> Ah, I see that it's for c function declarations. If that's the case, have you 
> considered adding the malloc and free declarations to that header?

What about doing this in another patch and updating all test cases that use 
malloc and free? Maybe other libc APIs as well?
The test case malloc-three-arg.c declares malloc and in a different signature 
and also includes this header. Simply doing so in this patch will lead to other 
conflicts.



Comment at: clang/test/Analysis/pr22954.c:581
   clang_analyzer_eval(m27.s3[i] == 1); // expected-warning{{UNKNOWN}}\
   expected-warning{{Potential leak of memory pointed to by 'm27.s4'}}
   return 0;

Do I need to update all other leak expectation directions in this file as well? 
Such as this one here.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D152435/new/

https://reviews.llvm.org/D152435

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97265: [clang] Allow clang-check to customize analyzer output file or dir name

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

ping @sammccall again

Do I need to add other reviewers if you are busy doing your job?


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97265/new/

https://reviews.llvm.org/D97265

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D83660: [analyzer] Fix a crash for dereferencing an empty llvm::Optional variable in SMTConstraintManager.h.

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D83660#2675064 , @mikhail.ramalho 
wrote:

> Indeed it looks like a copy & paste error, I'm surprised no one found it 
> earlier.
>
> Regarding the tests, we used to have `make check-clang-analysis-z3` (or 
> something similar) that would run only the analyzer's tests, but using Z3 as 
> the constraint solver. It looks like this change broke it: 
> https://reviews.llvm.org/D62445

I add `VERBOSE=1` during execution, and command 
`/path/to/llvm-project/build/./bin/llvm-lit -sv --param USE_Z3_SOLVER=0 
/path/to/llvm-project/clang/test/Analysis/z3` is used to execute lit for 
testing.
And this test case is not executed.

Should the execution requirements be changed to make it run if z3 is enabled? 
Or just keep it as it is now?

If there are no other suggestions for this patch, I'd like to see it landed 
ASAP. I think it is a far too long period for a fix of a copy & paste error.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D83660/new/

https://reviews.llvm.org/D83660

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97265: [clang] Allow clang-check to customize analyzer output file or dir name

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

If this patch is OK, could you please commit it on my behalf (Ella Ma 
)?
Thx.
And I will continue working on the `-Xanalyzer` option for clang-check, as it 
is too complex when adding analyzer options via the `-extra-arg` option.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97265/new/

https://reviews.llvm.org/D97265

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: gamesh411, balazske, martong, xazax.hun.
OikawaKirie added a project: clang.
Herald added subscribers: steakhal, ASDenysPetrov, dkrupp, donat.nagy, 
Szelethus, mikhail.ramalho, a.sidorin, rnkovacs, szepet, baloghadamsoftware.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

During CTU, the *on-demand parsing* will read and parse the invocation list to 
know how to compile the file being imported. However, it seems that the 
invocation list will be parsed again if a previous parsing has failed. Then, 
parse again and fail again. This patch tries to overcome the problem by storing 
the error code during the first parsing, and re-create the stored error during 
the later parsings.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp


Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -667,12 +667,16 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  else if (index_error_code::no_error != InvocationListParsingError)
+return llvm::make_error(InvocationListParsingError);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+InvocationListParsingError =
+index_error_code::invocation_list_file_not_found;
+return llvm::make_error(InvocationListParsingError);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer 
"
   "should not be nullptr.");
@@ -680,8 +684,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](IndexError &E) { InvocationListParsingError = E.getCode(); });
+return llvm::make_error(InvocationListParsingError);
+  }
 
   InvocationList = *ExpectedInvocationList;
 
Index: clang/include/clang/CrossTU/CrossTranslationUnit.h
===
--- clang/include/clang/CrossTU/CrossTranslationUnit.h
+++ clang/include/clang/CrossTU/CrossTranslationUnit.h
@@ -38,6 +38,7 @@
 namespace cross_tu {
 
 enum class index_error_code {
+  no_error = 0,
   unspecified = 1,
   missing_index_file,
   invalid_index_format,
@@ -253,6 +254,7 @@
 /// In case of on-demand parsing, the invocations for parsing the source
 /// files is stored.
 llvm::Optional InvocationList;
+index_error_code InvocationListParsingError = index_error_code::no_error;
   };
 
   /// Maintain number of AST loads and check for reaching the load limit.


Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -667,12 +667,16 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  else if (index_error_code::no_error != InvocationListParsingError)
+return llvm::make_error(InvocationListParsingError);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+InvocationListParsingError =
+index_error_code::invocation_list_file_not_found;
+return llvm::make_error(InvocationListParsingError);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +684,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](IndexError &E) { InvocationListParsingError = E.getCode(); });
+return llvm::make_error(InvocationListParsingError);
+  }
 
   InvocationList = *ExpectedIn

[PATCH] D97265: [clang] Allow clang-check to customize analyzer output file or dir name

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

Ping @sammccall again for landing this patch on my behalf.
Thanks.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97265/new/

https://reviews.llvm.org/D97265

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 343293.
OikawaKirie added a comment.

Add a regression test case by mocking the `open` function. When this function 
is called with the file name of the invocation list, the mocked `open` function 
will reject the open operation and dump a log. And we will then check how many 
times are the invocation list opened. (Thanks to the ideas by steakhal in 
another patch of mocking a library function :-).)

Identifiers renamed as suggested.

Unfortunately, we cannot store and copy a `llvm::Error` object. Therefore, the 
new version still uses the error code to store the previous parsing results.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/multiple-invocation-list-parsing.cpp

Index: clang/test/Analysis/multiple-invocation-list-parsing.cpp
===
--- /dev/null
+++ clang/test/Analysis/multiple-invocation-list-parsing.cpp
@@ -0,0 +1,59 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// Compile the mocked `open` function.
+// RUN: %host_cxx %s -fPIC -shared -o %t/mock_open.so
+
+// RUN: echo "void bar(); void foo() { bar(); bar(); }" > %t/importer.c
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file":"%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+
+// Run the test code.
+// RUN: LD_PRELOAD=%t/mock_open.so \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=%t \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %t/importer.c | FileCheck %s
+
+// REQUIRES: shell, system-linux
+
+// Check the log for the second open of the invocation list.
+// CHECK: {{Mocking file invocations.yaml: 1}}
+// CHECK-NOT: {{Mocking file invocations.yaml: 2}}
+
+#define _GNU_SOURCE 1
+#include 
+#include 
+
+#include 
+#include 
+#include 
+using namespace std;
+
+extern "C" int open(const char *name, int flag, ...) {
+  // If the opened function is the invocation list, reject opening and return
+  // an error.
+  // Log how many times the open operation have been rejected.
+  string sname(name);
+  if ("invocations.yaml" == sname) {
+static unsigned N = 0;
+cout << "Mocking file invocations.yaml: " << ++N << endl;
+return -1;
+  }
+
+  // For other files, the original open function will be called.
+  using open_t = int (*)(const char *, int, mode_t);
+  static open_t o_open = nullptr;
+  if (!o_open)
+o_open = reinterpret_cast(dlsym(RTLD_NEXT, "open"));
+  assert(o_open && "Cannot find function `open'.");
+
+  va_list vl;
+  va_start(vl, flag);
+  auto mode = va_arg(vl, mode_t);
+  va_end(vl);
+  return o_open(name, flag, mode);
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -667,12 +667,16 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  else if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult =
+index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +684,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](const IndexError &E) { PreviousParsingResult = E.getCode(); });
+return llvm::make_error(PreviousParsingResult);
+  }
 
   InvocationList = *ExpectedInvocationList;
 
Index: clang/include/clang/CrossTU/CrossTranslationUnit.h
===
--- clang/include/clang/CrossTU/CrossTranslationUnit.h
+++ clang/include/clang/CrossTU/CrossTranslationUnit.h
@@ -38,6 +38,7 @@
 namespace cross_tu {
 
 enum class index_error_code 

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 343346.
OikawaKirie added a comment.

In D101763#2741536 , @steakhal wrote:

> Overall, it looks promising. But I don't quite get this test.
> There is no invocation yaml in the temp directory. So, you are probably not 
> testing the right thing.
> You wanted to test if the invocation yaml exists, and could be opened **but 
> the parsing fails**.
> You should demonstrate that when a parsing error happens, the error code has 
> recoded and it won't try to reparse the invocation yaml again and again.

The main idea of the test case is that the mocked `open` function will dump a 
log every time when the invocation list file is opened to be read. If a second 
open operation is detected in the log, it means the list is parsed again.

As the behavior of missing the invocation list file or failing to parse it are 
the same (both returns the error), the previous version checks whether function 
`open` will be called again after a file-not-found error.
In this update, I use an empty invocation list to really trigger an 
`invocation_list_empty` error and check whether the invocation list file will 
be read again via function `open`.
Compared with the previous version, opening the invocation list will not be 
manually failed, and a real parsing error is triggered instead.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/multiple-invocation-list-parsing.cpp

Index: clang/test/Analysis/multiple-invocation-list-parsing.cpp
===
--- /dev/null
+++ clang/test/Analysis/multiple-invocation-list-parsing.cpp
@@ -0,0 +1,56 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// Compile the mocked `open` function.
+// RUN: %host_cxx %s -fPIC -shared -o %t/mock_open.so
+
+// RUN: echo "void bar(); void foo() { bar(); bar(); }" > %t/importer.c
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file":"%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+// RUN: echo "" > %t/invocations.yaml
+
+// Run the test code.
+// RUN: LD_PRELOAD=%t/mock_open.so \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=%t \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %t/importer.c | FileCheck %s
+
+// REQUIRES: shell, system-linux
+
+// Check the log for the second open of the invocation list.
+// CHECK: {{Mocking file invocations.yaml: 1}}
+// CHECK-NOT: {{Mocking file invocations.yaml: 2}}
+
+#define _GNU_SOURCE 1
+#include 
+#include 
+
+#include 
+#include 
+#include 
+using namespace std;
+
+extern "C" int open(const char *name, int flag, ...) {
+  // Log how many times the invocation list is opened.
+  if ("invocations.yaml" == string(name)) {
+static unsigned N = 0;
+cout << "Mocking file invocations.yaml: " << ++N << endl;
+  }
+
+  // The original open function will be called to open the files.
+  using open_t = int (*)(const char *, int, mode_t);
+  static open_t o_open = nullptr;
+  if (!o_open)
+o_open = reinterpret_cast(dlsym(RTLD_NEXT, "open"));
+  assert(o_open && "Cannot find function `open'.");
+
+  va_list vl;
+  va_start(vl, flag);
+  auto mode = va_arg(vl, mode_t);
+  va_end(vl);
+  return o_open(name, flag, mode);
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -667,12 +667,15 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  else if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult = index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +683,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for ne

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 343582.
OikawaKirie added a comment.

- Update the test case as suggested.
- Add a case branch for the `index_error_code::success` error code in function 
`IndexErrorCategory::message` to silent the compiler warnings.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/multiple-invocation-list-parsing.c

Index: clang/test/Analysis/multiple-invocation-list-parsing.c
===
--- /dev/null
+++ clang/test/Analysis/multiple-invocation-list-parsing.c
@@ -0,0 +1,29 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file":"%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+// RUN: echo "" > %t/invocations.yaml
+
+// RUN: strace -e trace=openat \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=%t \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %s 2>&1 | grep 'invocations\.yaml' | wc -l | FileCheck %s
+
+// REQUIRES: shell, system-linux
+
+// Check the log for the second open of the invocation list.
+// CHECK: {{1}}
+
+void bar();
+
+void foo() {
+  // Import twice to see whether the second import will open the invocation
+  // list again.
+  bar();
+  bar();
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -92,6 +92,10 @@
 
   std::string message(int Condition) const override {
 switch (static_cast(Condition)) {
+case index_error_code::success:
+  // There should not be a success error. Jump to unreachable directly.
+  // Add this case to make the compiler stop complaining.
+  break;
 case index_error_code::unspecified:
   return "An unknown error has occurred.";
 case index_error_code::missing_index_file:
@@ -667,12 +671,15 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  else if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult = index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +687,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](const IndexError &E) { PreviousParsingResult = E.getCode(); });
+return llvm::make_error(PreviousParsingResult);
+  }
 
   InvocationList = *ExpectedInvocationList;
 
Index: clang/include/clang/CrossTU/CrossTranslationUnit.h
===
--- clang/include/clang/CrossTU/CrossTranslationUnit.h
+++ clang/include/clang/CrossTU/CrossTranslationUnit.h
@@ -38,6 +38,7 @@
 namespace cross_tu {
 
 enum class index_error_code {
+  success = 0,
   unspecified = 1,
   missing_index_file,
   invalid_index_format,
@@ -253,6 +254,7 @@
 /// In case of on-demand parsing, the invocations for parsing the source
 /// files is stored.
 llvm::Optional InvocationList;
+index_error_code PreviousParsingResult = index_error_code::success;
   };
 
   /// Maintain number of AST loads and check for reaching the load limit.
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 343585.
OikawaKirie added a comment.

Replace `wc -l` with `count`.

Sorry for the spam.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/multiple-invocation-list-parsing.c

Index: clang/test/Analysis/multiple-invocation-list-parsing.c
===
--- /dev/null
+++ clang/test/Analysis/multiple-invocation-list-parsing.c
@@ -0,0 +1,26 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file":"%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+// RUN: echo "" > %t/invocations.yaml
+
+// RUN: strace -e trace=openat \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=%t \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %s 2>&1 | grep 'invocations\.yaml' | count 1
+
+// REQUIRES: shell, system-linux
+
+void bar();
+
+void foo() {
+  // Import twice to see whether the second import will open the invocation
+  // list again.
+  bar();
+  bar();
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -92,6 +92,10 @@
 
   std::string message(int Condition) const override {
 switch (static_cast(Condition)) {
+case index_error_code::success:
+  // There should not be a success error. Jump to unreachable directly.
+  // Add this case to make the compiler stop complaining.
+  break;
 case index_error_code::unspecified:
   return "An unknown error has occurred.";
 case index_error_code::missing_index_file:
@@ -667,12 +671,15 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  else if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult = index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +687,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](const IndexError &E) { PreviousParsingResult = E.getCode(); });
+return llvm::make_error(PreviousParsingResult);
+  }
 
   InvocationList = *ExpectedInvocationList;
 
Index: clang/include/clang/CrossTU/CrossTranslationUnit.h
===
--- clang/include/clang/CrossTU/CrossTranslationUnit.h
+++ clang/include/clang/CrossTU/CrossTranslationUnit.h
@@ -38,6 +38,7 @@
 namespace cross_tu {
 
 enum class index_error_code {
+  success = 0,
   unspecified = 1,
   missing_index_file,
   invalid_index_format,
@@ -253,6 +254,7 @@
 /// In case of on-demand parsing, the invocations for parsing the source
 /// files is stored.
 llvm::Optional InvocationList;
+index_error_code PreviousParsingResult = index_error_code::success;
   };
 
   /// Maintain number of AST loads and check for reaching the load limit.
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 343626.
OikawaKirie added a comment.

In D101763#2743984 , @steakhal wrote:

> By inspecting the output that is piped to the `count`, I have a suspicion:
>
>   openat(AT_FDCWD, "invocations.yaml", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No 
> such file or directory)

According to the output of the failed test, it seems that CTU is not working 
during the test case execution (`Expected 1 lines, got 0.`). Otherwise, it 
should be matched by grep.
There seems to be something wrong with the test case, although it works well in 
my environment.

> My guess is that `ctu-invocation-list` is not using the `ctu-dir`. It should 
> be a different bug though.
> To workaround this issue, simply use an absolute path for the invocation yaml.

It seems that you are correct, I will fix this problem later.
I can just `cd` to the `ctu-dir` before running CSA as a workaround. Just as 
the author of on-demand parsing did (refer to 
`clang/test/Analysis/ctu-on-demand-parsing.c`).

> `// CHECK: openat(AT_FDCWD, "{{[^"]+}}invocations.yaml", O_RDONLY|O_CLOEXEC) 
> = {{[0-9]+}}`

I am not quite sure whether we need to verify the flags (`AT_FDCWD`, `O_RDONLY` 
and `O_CLOEXEC`) as well. Therefore, I removed them in this update.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/multiple-invocation-list-parsing.c

Index: clang/test/Analysis/multiple-invocation-list-parsing.c
===
--- /dev/null
+++ clang/test/Analysis/multiple-invocation-list-parsing.c
@@ -0,0 +1,30 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file":"%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+// RUN: echo "" > %t/invocations.yaml
+
+// RUN: cd %t && \
+// RUN: strace -e trace=openat \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %s 2>&1 | grep 'invocations\.yaml' | FileCheck %s
+
+// REQUIRES: shell, system-linux
+
+void bar();
+
+void foo() {
+  // It finds the invocation yaml, then it tries to parse it which expected to fail.
+  bar();
+  // CHECK: openat({{[^,]+}}, "{{[^"]*/?}}invocations.yaml", {{[^)]+}}) = {{[0-9]+}}
+
+  // We should remember that the parsing failed, we shouldn't try to reopen and parse the yaml again.
+  bar();
+  // CHECK-NOT: openat({{[^,]+}}, "{{[^"]*/?}}invocations.yaml", {{[^)]+}}) = {{[0-9]+}}
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -92,6 +92,10 @@
 
   std::string message(int Condition) const override {
 switch (static_cast(Condition)) {
+case index_error_code::success:
+  // There should not be a success error. Jump to unreachable directly.
+  // Add this case to make the compiler stop complaining.
+  break;
 case index_error_code::unspecified:
   return "An unknown error has occurred.";
 case index_error_code::missing_index_file:
@@ -667,12 +671,15 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  else if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult = index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +687,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](const IndexError &E) { PreviousParsingResult = E.getCode(); });
+return llvm::make_error(PreviousParsingResult);
+  }
 
   InvocationList = *ExpectedInvoca

[PATCH] D102062: [analyzer][ctu] Append ctu-dir to ctu-invocation-list for non-absolute paths

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: gamesh411, steakhal, martong, balazske.
OikawaKirie added a project: clang.
Herald added subscribers: ASDenysPetrov, dkrupp, donat.nagy, Szelethus, 
mikhail.ramalho, a.sidorin, rnkovacs, szepet, baloghadamsoftware, xazax.hun.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

This patch fixes the problem mentioned in D101763#2743984 
.
When the `ctu-invocation-list` is not an absolute path and the current working 
directory is not the `ctu-dir`, the invocation list will fail to be loaded.
In this patch, the `ctu-dir` is appended to the `ctu-invocation-list` if the 
`ctu-invocation-list` is not an absolute path.
And the original test cases for on-demand parsing are reused to test this 
feature.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D102062

Files:
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/ctu-on-demand-parsing.c
  clang/test/Analysis/ctu-on-demand-parsing.cpp


Index: clang/test/Analysis/ctu-on-demand-parsing.cpp
===
--- clang/test/Analysis/ctu-on-demand-parsing.cpp
+++ clang/test/Analysis/ctu-on-demand-parsing.cpp
@@ -11,20 +11,20 @@
 //
 // RUN: echo '{"%t/Inputs/ctu-chain.cpp": ["g++", "%t/Inputs/ctu-chain.cpp"], 
"%t/Inputs/ctu-other.cpp": ["g++", "%t/Inputs/ctu-other.cpp"]}' | sed -e 
's/\\//g' > %t/invocations.yaml
 //
-// RUN: cd "%t" && %clang_extdef_map Inputs/ctu-chain.cpp Inputs/ctu-other.cpp 
> externalDefMap.txt
+// RUN: %clang_extdef_map -p %t %t/Inputs/ctu-chain.cpp 
%t/Inputs/ctu-other.cpp > %t/externalDefMap.txt
 //
-// RUN: cd "%t" && %clang_analyze_cc1 \
+// RUN: %clang_analyze_cc1 \
 // RUN:   -analyzer-checker=core,debug.ExprInspection \
 // RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
-// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-dir=%t \
 // RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
-// RUN:   -verify ctu-on-demand-parsing.cpp
-// RUN: cd "%t" && %clang_analyze_cc1 \
+// RUN:   -verify %t/ctu-on-demand-parsing.cpp
+// RUN: %clang_analyze_cc1 \
 // RUN:   -analyzer-checker=core,debug.ExprInspection \
 // RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
-// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-dir=%t \
 // RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
-// RUN:   -analyzer-config display-ctu-progress=true ctu-on-demand-parsing.cpp 
2>&1 | FileCheck %t/ctu-on-demand-parsing.cpp
+// RUN:   -analyzer-config display-ctu-progress=true 
%t/ctu-on-demand-parsing.cpp 2>&1 | FileCheck %t/ctu-on-demand-parsing.cpp
 //
 // CHECK: CTU loaded AST file: {{.*}}ctu-other.cpp
 // CHECK: CTU loaded AST file: {{.*}}ctu-chain.cpp
Index: clang/test/Analysis/ctu-on-demand-parsing.c
===
--- clang/test/Analysis/ctu-on-demand-parsing.c
+++ clang/test/Analysis/ctu-on-demand-parsing.c
@@ -7,16 +7,16 @@
 // compile_commands.json is only needed for extdef_mapping, not for the 
analysis itself.
 // RUN: echo '[{"directory":"%t","command":"gcc -std=c89 -Wno-visibility 
ctu-other.c","file":"ctu-other.c"}]' | sed -e 's/\\//g' > 
%t/compile_commands.json
 //
-// RUN: echo '"%t/ctu-other.c": ["gcc", "-std=c89", "-Wno-visibility", 
"ctu-other.c"]' | sed -e 's/\\//g' > %t/invocations.yaml
+// RUN: echo '"%t/ctu-other.c": ["gcc", "-std=c89", "-Wno-visibility", 
"%t/ctu-other.c"]' | sed -e 's/\\//g' > %t/invocations.yaml
 //
-// RUN: cd "%t" && %clang_extdef_map "%t/ctu-other.c" > externalDefMap.txt
+// RUN: %clang_extdef_map -p %t "%t/ctu-other.c" > %t/externalDefMap.txt
 //
-// RUN: cd "%t" && %clang_cc1 -fsyntax-only -std=c89 -analyze \
+// RUN: %clang_cc1 -fsyntax-only -std=c89 -analyze \
 // RUN:   -analyzer-checker=core,debug.ExprInspection \
 // RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
-// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-dir=%t \
 // RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
-// RUN:   -verify ctu-on-demand-parsing.c
+// RUN:   -verify %t/ctu-on-demand-parsing.c
 //
 // FIXME: Path handling should work on all platforms.
 // REQUIRES: system-linux
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -668,8 +668,14 @@
   if (InvocationList)
 return llvm::Error::success();
 
+  SmallString<256> InvocationListFileAbsolutePath = CTUDir;
+  if (llvm::sys::path::is_absolute(InvocationListFilePath))
+InvocationListFileAbsolutePath = InvocationListFilePath;
+  else
+llvm::sys::path::append(InvocationListFileAbsolutePath,
+InvocationListFilePath);
   llvm::ErrorOr> FileConte

[PATCH] D102062: [analyzer][ctu] Append ctu-dir to ctu-invocation-list for non-absolute paths

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D102062#2744267 , @steakhal wrote:

> On second thought the current behavior is reasonable.

The behavior of all CTU affairs are related to the `ctu-dir` rather than CWD, 
such as loading the external function map file and looking for the AST dumps 
and source files to be imported.
If a relative path is provided for these files, the `ctu-dir` will be added to 
the path when trying to open the file (see the comments in the code).
Therefore, IMO the behavior of `ctu-invocation-list` and its contents should 
also be relative to the `ctu-dir` unless an absolute path is provided.

> We call clang from a command line, so I think it's fair to expect that 
> relative paths are resolved using the CWD.

I do not know how other people use CSA, but I prefer using CSA via 
`clang-check`.
It works with the relative path to the "directory" option in the compilation 
database of each source file, which could make the CWD not unique.

> AFAIK if one does not supply the `ctu-invocation-list`, then it would be 
> substituted by `ctu-dir/invocations.yaml` anyway.

But we cannot currently distinguish the difference between setting 
`ctu-invocation-list` to `invocations.yaml` and not setting it. As the default 
value of `ctu-invocation-list` is `invocations.yaml` rather than an empty 
string.
Therefore, adding the `ctu-dir` to a relative `ctu-invocation-list` path by 
default seems to be a better choice.




Comment at: clang/lib/CrossTU/CrossTranslationUnit.cpp:470
   else
 llvm::sys::path::append(IndexFile, IndexName);
 

Loading the external function map.



Comment at: clang/lib/CrossTU/CrossTranslationUnit.cpp:516
 Path = CTUDir;
 llvm::sys::path::append(Path, PathStyle, Identifier);
   }

Loading the ASTUnit from external AST dumps or source files.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102062/new/

https://reviews.llvm.org/D102062

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D101763#2741629 , @steakhal wrote:

> Okay, so you 'just' want an indication for the given open call. What about 
> using `strace`?
> `strace -e trace=openat %clang_cc1 ... 2>&1 | grep '"invocations.yaml"' | 
> FileCheck %s`

I do not know why the test case always fails on the build server, it runs 
perfectly on my computer. : (
Do any reviewers have any ideas?


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 343818.
OikawaKirie added a comment.

Monitor function `open` together with `openat` to fix the failure in the test 
case.
As some distros actually call function `open` rather than forward to function 
`openat`, both functions should be monitored by `strace`.

But I am still worrying about whether function `open` will call `openat` and 
vice versa. If this circumstance is possible, the test case may still fail.
Although, I have not found any pieces of evidence for such invocations.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.c

Index: clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.c
===
--- /dev/null
+++ clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.c
@@ -0,0 +1,30 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file":"%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+// RUN: echo "" > %t/invocations.yaml
+
+// RUN: cd %t && \
+// RUN: strace -e trace=open,openat \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %s 2>&1 | grep 'invocations\.yaml' | FileCheck %s
+
+// REQUIRES: shell, system-linux
+
+void bar();
+
+void foo() {
+  // It finds the invocation yaml, then it tries to parse it which expected to fail.
+  bar();
+  // CHECK: open{{(at)?}}({{[^"]*}}"{{[^"]*/?}}invocations.yaml", {{[^)]+}}) = {{[0-9]+}}
+
+  // We should remember that the parsing failed, we shouldn't try to reopen and parse the yaml again.
+  bar();
+  // CHECK-NOT: open{{(at)?}}({{[^"]*}}"{{[^"]*/?}}invocations.yaml", {{[^)]+}}) = {{[0-9]+}}
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -92,6 +92,10 @@
 
   std::string message(int Condition) const override {
 switch (static_cast(Condition)) {
+case index_error_code::success:
+  // There should not be a success error. Jump to unreachable directly.
+  // Add this case to make the compiler stop complaining.
+  break;
 case index_error_code::unspecified:
   return "An unknown error has occurred.";
 case index_error_code::missing_index_file:
@@ -667,12 +671,15 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  else if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult = index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +687,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](const IndexError &E) { PreviousParsingResult = E.getCode(); });
+return llvm::make_error(PreviousParsingResult);
+  }
 
   InvocationList = *ExpectedInvocationList;
 
Index: clang/include/clang/CrossTU/CrossTranslationUnit.h
===
--- clang/include/clang/CrossTU/CrossTranslationUnit.h
+++ clang/include/clang/CrossTU/CrossTranslationUnit.h
@@ -38,6 +38,7 @@
 namespace cross_tu {
 
 enum class index_error_code {
+  success = 0,
   unspecified = 1,
   missing_index_file,
   invalid_index_format,
@@ -253,6 +254,7 @@
 /// In case of on-demand parsing, the invocations for parsing the source
 /// files is stored.
 llvm::Optional InvocationList;
+index_error_code PreviousParsingResult = index_error_code::success;
   };
 
   /// Maintain number of AST loads and c

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

> FileCheck error: '' is empty.

It fails again on the build bot, another dead end. Maybe I need a Debian server 
to reproduce the problem encountered in the test case.

In D101763#2745918 , @steakhal wrote:

> In D101763#2745802 , @OikawaKirie 
> wrote:
>
>> I do not know why the test case always fails on the build server, it runs 
>> perfectly on my computer. : (
>
> I've locally run your patch and seemed good to me, I regret not having a look 
> at the actual build bot.

I have tried the test case on a Ubuntu 20.04 server with kernel 5.4.0 and gcc 
9.3.0, as well as a CentOS 7 server with kernel 3.10.0 and gcc 7.1.0. Both of 
them work well.

> In D101763#2745900 , @OikawaKirie 
> wrote:
>
>> Monitor function `open` together with `openat` to fix the failure in the 
>> test case.
>> As some distros actually call function `open` rather than forward to 
>> function `openat`, both functions should be monitored by `strace`.
>>
>> But I am still worrying about whether function `open` will call `openat` and 
>> vice versa. If this circumstance is possible, the test case may still fail.
>> Although, I have not found any pieces of evidence for such invocations.
>
> Uh, that's a fair point. Now, I'm also worried xD
> To be fair, I don't have any better idea testing this without overwhelming 
> effort.
> If anything goes wrong, we can simply revert this change. This is 'just' a 
> performance fix.

I will try to reproduce and fix the problem of the test case next week. If I 
cannot find other ways to overcome the problem by the next weekend, we can have 
a try on landing this patch.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102149: [analyzer][ctu] Allow loading invocation list from a compilation database automatically detected from the ctu-dir

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: steakhal, gamesh411, martong, balazske.
OikawaKirie added a project: clang.
Herald added subscribers: ASDenysPetrov, dkrupp, donat.nagy, Szelethus, 
mikhail.ramalho, a.sidorin, rnkovacs, szepet, baloghadamsoftware, xazax.hun, 
mgorny.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

It seems to be simpler and more convenient to reuse the compilation database 
generated for ClangTool when using clang-check to run the analyzer.
In this patch, when the invocation list file is failed to be loaded, the 
ASTLoader will try to find a compilation database from the `ctu-dir` and 
convert the detected database to an invocation list.

For each compile command objects in the database:

- the "directory" entry is converted to option `-working-directory` appended to 
the invocation commands;
- the "file" entry is updated to an absolute path based on the "directory" 
entry as the index;
- and the "command" entry is converted to the invocation commands.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D102149

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CMakeLists.txt
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/ctu-on-demand-parsing.c
  clang/test/Analysis/ctu-on-demand-parsing.cpp

Index: clang/test/Analysis/ctu-on-demand-parsing.cpp
===
--- clang/test/Analysis/ctu-on-demand-parsing.cpp
+++ clang/test/Analysis/ctu-on-demand-parsing.cpp
@@ -13,6 +13,7 @@
 //
 // RUN: cd "%t" && %clang_extdef_map Inputs/ctu-chain.cpp Inputs/ctu-other.cpp > externalDefMap.txt
 //
+// Run with invocation list.
 // RUN: cd "%t" && %clang_analyze_cc1 \
 // RUN:   -analyzer-checker=core,debug.ExprInspection \
 // RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
@@ -26,6 +27,19 @@
 // RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
 // RUN:   -analyzer-config display-ctu-progress=true ctu-on-demand-parsing.cpp 2>&1 | FileCheck %t/ctu-on-demand-parsing.cpp
 //
+// Run with compilation database.
+// RUN: rm %t/invocations.yaml
+// RUN: cd "%t" && %clang_analyze_cc1 \
+// RUN:   -analyzer-checker=core,debug.ExprInspection \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -verify ctu-on-demand-parsing.cpp
+// RUN: cd "%t" && %clang_analyze_cc1 \
+// RUN:   -analyzer-checker=core,debug.ExprInspection \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config display-ctu-progress=true ctu-on-demand-parsing.cpp 2>&1 | FileCheck %t/ctu-on-demand-parsing.cpp
+//
 // CHECK: CTU loaded AST file: {{.*}}ctu-other.cpp
 // CHECK: CTU loaded AST file: {{.*}}ctu-chain.cpp
 //
Index: clang/test/Analysis/ctu-on-demand-parsing.c
===
--- clang/test/Analysis/ctu-on-demand-parsing.c
+++ clang/test/Analysis/ctu-on-demand-parsing.c
@@ -11,6 +11,7 @@
 //
 // RUN: cd "%t" && %clang_extdef_map "%t/ctu-other.c" > externalDefMap.txt
 //
+// Run with invocation list.
 // RUN: cd "%t" && %clang_cc1 -fsyntax-only -std=c89 -analyze \
 // RUN:   -analyzer-checker=core,debug.ExprInspection \
 // RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
@@ -18,6 +19,14 @@
 // RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
 // RUN:   -verify ctu-on-demand-parsing.c
 //
+// Run with compilation database.
+// RUN: rm %t/invocations.yaml
+// RUN: cd "%t" && %clang_cc1 -fsyntax-only -std=c89 -analyze \
+// RUN:   -analyzer-checker=core,debug.ExprInspection \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -verify ctu-on-demand-parsing.c
+//
 // FIXME: Path handling should work on all platforms.
 // REQUIRES: system-linux
 
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -19,6 +19,8 @@
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/TextDiagnosticPrinter.h"
 #include "clang/Index/USRGeneration.h"
+#include "clang/Tooling/CompilationDatabase.h"
+#include "clang/Tooling/Tooling.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/ADT/Statistic.h"
 #include "llvm/ADT/Triple.h"
@@ -119,14 +121,14 @@
 case index_error_code::invocation_list_ambiguous:
   return "Invocation list file contains multiple references to the same "
  "source file.";
-case index_error_code::invocation_list_file_not_found:
-  return "Invocation list file is not found.";
 case index_error_code::invocation_list_empty:
   return "Invocation list file is empty.";
 case index_error_code::invocation_li

[PATCH] D102159: [index][analyzer][ctu] Eliminate white spaces in the CTU lookup name.

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: akyrtzi, martong, steakhal.
OikawaKirie added a project: clang.
Herald added subscribers: ASDenysPetrov, dkrupp, donat.nagy, Szelethus, 
arphaman, mikhail.ramalho, a.sidorin, rnkovacs, szepet, baloghadamsoftware, 
xazax.hun.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

In the analyzer, the CTU definition index file requires there are no white 
spaces in the CTU lookup name, aka. the USR of a function decl or a global 
variable decl. Otherwise, the index file will be wrongly parsed.
However, it is difficult for the analyzer to know whether a white space `' '` 
is the separator between the index string and the file path, or it is just a 
part of the index string or the path. It means that using either the first or 
the last white space as the separator could not be a good idea. As it is valid 
for a file path to have white spaces, a better choice seems to be eliminating 
all the white spaces in the index string.
In this patch, the white space `' '` for an unsupported type is replaced with a 
question mark `'?'`, which solves the problem as well as makes the index more 
reasonable as far as I am thinking.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D102159

Files:
  clang/lib/Index/USRGeneration.cpp
  clang/test/Index/unsupported.cpp


Index: clang/test/Index/unsupported.cpp
===
--- /dev/null
+++ clang/test/Index/unsupported.cpp
@@ -0,0 +1,10 @@
+// RUN: %clang_extdef_map %s -- | FileCheck %s
+
+struct X {
+  bool f(char *);
+};
+
+typedef bool (X::*XFP)(char *);
+
+// CHECK-NOT: {{ [^ ]+ }}
+void f(XFP xfp) {}
Index: clang/lib/Index/USRGeneration.cpp
===
--- clang/lib/Index/USRGeneration.cpp
+++ clang/lib/Index/USRGeneration.cpp
@@ -895,7 +895,7 @@
 }
 
 // Unhandled type.
-Out << ' ';
+Out << '?';
 break;
   } while (true);
 }


Index: clang/test/Index/unsupported.cpp
===
--- /dev/null
+++ clang/test/Index/unsupported.cpp
@@ -0,0 +1,10 @@
+// RUN: %clang_extdef_map %s -- | FileCheck %s
+
+struct X {
+  bool f(char *);
+};
+
+typedef bool (X::*XFP)(char *);
+
+// CHECK-NOT: {{ [^ ]+ }}
+void f(XFP xfp) {}
Index: clang/lib/Index/USRGeneration.cpp
===
--- clang/lib/Index/USRGeneration.cpp
+++ clang/lib/Index/USRGeneration.cpp
@@ -895,7 +895,7 @@
 }
 
 // Unhandled type.
-Out << ' ';
+Out << '?';
 break;
   } while (true);
 }
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102159: [index][analyzer][ctu] Eliminate white spaces in the CTU lookup name.

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D102159#2749033 , @akyrtzi wrote:

> Maybe we could also handle this kind of type instead of leaving it 
> 'unhandled'? What `Type` is it?

The member function pointer type, see the test case.

Although it would be perfect to handle this kind of type, I mean the white 
spaces should still be removed from the USR.
Currently, the white space character is used as the separator between the index 
string and the file path in the output of `clang-extdef-mapping`.
And it is difficult to determine when the index string with white space 
characters ends when parsing the output of `clang-extdef-mapping`.
Therefore, IMO the white space character had better not be used in the index 
string.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102159/new/

https://reviews.llvm.org/D102159

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D115716: [Analyzer][BugReporter] Replace the example bug report with the one used to generate PathDiagnostic

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 397207.
OikawaKirie edited the summary of this revision.
OikawaKirie added a comment.

In D115716#3217786 , @Szelethus wrote:

> First off, your patch is great, and I'm pretty sure we want it!
>
> I remember working around here, and I either have never quite understood what 
> makes `exampleReport` an example, or have long forgotten. Can we not just 
> rename it to `chosenReport`, or simply `report`?

Thanks a lot.

New updates:

1. add a new `BugReporter::generateDiagnosticForConsumerMap` method to handle 
the job before adding notes
2. add a `BugReport` field to `DiagnosticForConsumerMapTy`
3. move `findReportInEquivalenceClass` calls to 
`generateDiagnosticForConsumerMap` methods
4. the `exampleReport` variable in method `findReportInEquivalenceClass` is 
unchanged, others are removed as they are no longer required


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D115716/new/

https://reviews.llvm.org/D115716

Files:
  clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporter.h
  clang/lib/StaticAnalyzer/Core/BugReporter.cpp
  clang/test/Analysis/Inputs/expected-plists/malloc-plist.c.plist
  clang/test/Analysis/keychainAPI.m
  clang/test/Analysis/malloc-plist.c
  clang/test/Analysis/stream.c

Index: clang/test/Analysis/stream.c
===
--- clang/test/Analysis/stream.c
+++ clang/test/Analysis/stream.c
@@ -261,9 +261,7 @@
   if (!F1)
 return;
   if (Test == 1) {
-return; // no warning
+return; // expected-warning {{Opened stream never closed. Potential resource leak}}
   }
   rewind(F1);
-} // expected-warning {{Opened stream never closed. Potential resource leak}}
-// FIXME: This warning should be placed at the `return` above.
-// See https://reviews.llvm.org/D83120 about details.
+} // no warning
Index: clang/test/Analysis/malloc-plist.c
===
--- clang/test/Analysis/malloc-plist.c
+++ clang/test/Analysis/malloc-plist.c
@@ -135,8 +135,8 @@
 static void function_with_leak3(int y) {
 char *x = (char*)malloc(12);
 if (y)
-y++;
-}//expected-warning{{Potential leak}}
+  y++; // expected-warning{{Potential leak}}
+}
 void use_function_with_leak3(int y) {
 function_with_leak3(y);
 }
Index: clang/test/Analysis/keychainAPI.m
===
--- clang/test/Analysis/keychainAPI.m
+++ clang/test/Analysis/keychainAPI.m
@@ -402,10 +402,10 @@
 OSStatus st = 0;
 void *outData = my_AllocateReturn(&st); 
 if (x) {
-  consumeChar(*(char*)outData); // expected-warning{{Allocated data is not released:}}
+  consumeChar(*(char *)outData);
   return;
 } else {
-  consumeChar(*(char*)outData);
+  consumeChar(*(char *)outData); // expected-warning{{Allocated data is not released:}}
 }
 return;
 }
Index: clang/test/Analysis/Inputs/expected-plists/malloc-plist.c.plist
===
--- clang/test/Analysis/Inputs/expected-plists/malloc-plist.c.plist
+++ clang/test/Analysis/Inputs/expected-plists/malloc-plist.c.plist
@@ -3764,12 +3764,12 @@
  
   
line138
-   col9
+   col7
file0
   
   
line138
-   col9
+   col7
file0
   
  
@@ -3781,7 +3781,7 @@
  location
  
   line138
-  col9
+  col7
   file0
  
  depth1
@@ -3803,7 +3803,7 @@
   location
   
line138
-   col9
+   col7
file0
   
   ExecutedLines
Index: clang/lib/StaticAnalyzer/Core/BugReporter.cpp
===
--- clang/lib/StaticAnalyzer/Core/BugReporter.cpp
+++ clang/lib/StaticAnalyzer/Core/BugReporter.cpp
@@ -244,6 +244,9 @@
   std::unique_ptr
   generate(const PathDiagnosticConsumer *PDC) const;
 
+  /// Get the bug report used to generate the PathDiagnostic.
+  const PathSensitiveBugReport *getBugReport() const { return R; }
+
 private:
   void updateStackPiecesWithMessage(PathDiagnosticPieceRef P,
 const CallWithEntryStack &CallStack) const;
@@ -271,8 +274,6 @@
   PathDiagnosticLocation
   ExecutionContinues(llvm::raw_string_ostream &os,
  const PathDiagnosticConstruct &C) const;
-
-  const PathSensitiveBugReport *getBugReport() const { return R; }
 };
 
 } // namespace
@@ -2889,6 +2890,7 @@
 (*Out)[PC] = std::move(PD);
   }
 }
+Out->Report = PDB->getBugReport();
   }
 
   return Out;
@@ -3060,24 +3062,28 @@
   return exampleReport;
 }
 
-void BugReporter::FlushReport(BugReportEquivClass& EQ) {
-  SmallVector bugReports;
-  BugReport *report = findReportInEquivalenceClass(EQ, bugReports);
-  if (!report)
-return;
-
+std::unique_ptr
+BugReporter::generateDiagnosticForConsumerMap(BugR

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 397289.
OikawaKirie added a comment.

Replace on-demand-parsing with loading AST file for the new test case.
Tested on Linux and MacOS(x86).
If it can also pass the CI test on Windows, I think we can have another try on 
landing this patch.

Besides, as mentioned above, to trigger the problem of target triple on MacOS, 
we can simply remove the requirement of Linux system for the two test cases of 
on-demand-parsing, i.e. `clang/test/Analysis/ctu-on-demand-parsing.c` and 
`clang/test/Analysis/ctu-on-demand-parsing.cpp`.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

Files:
  clang/docs/analyzer/user-docs/CrossTranslationUnit.rst
  clang/include/clang/Basic/DiagnosticCrossTUKinds.td
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-import.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/Inputs/ctu-other.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.ast-dump.txt
  
clang/test/Analysis/Inputs/plist-macros-with-expansion-ctu.c.externalDefMap.txt
  clang/test/Analysis/ctu-inherited-default-ctor.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/func-mapping-test.cpp
  clang/unittests/CrossTU/CrossTranslationUnitTest.cpp

Index: clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
===
--- clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
+++ clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
@@ -61,7 +61,7 @@
 ASSERT_FALSE(llvm::sys::fs::createTemporaryFile("index", "txt", IndexFD,
 IndexFileName));
 llvm::ToolOutputFile IndexFile(IndexFileName, IndexFD);
-IndexFile.os() << "c:@F@f#I# " << ASTFileName << "\n";
+IndexFile.os() << "9:c:@F@f#I# " << ASTFileName << "\n";
 IndexFile.os().flush();
 EXPECT_TRUE(llvm::sys::fs::exists(IndexFileName));
 
Index: clang/test/Analysis/func-mapping-test.cpp
===
--- clang/test/Analysis/func-mapping-test.cpp
+++ clang/test/Analysis/func-mapping-test.cpp
@@ -3,10 +3,10 @@
 int f(int) {
   return 0;
 }
-// CHECK-DAG: c:@F@f#I#
+// CHECK-DAG: 9:c:@F@f#I#
 
 extern const int x = 5;
-// CHECK-DAG: c:@x
+// CHECK-DAG: 4:c:@x
 
 // Non-const variables should not be collected.
 int y = 5;
@@ -18,29 +18,29 @@
   int a;
 };
 extern S const s = {.a = 2};
-// CHECK-DAG: c:@s
+// CHECK-DAG: 4:c:@s
 
 struct SF {
   const int a;
 };
 SF sf = {.a = 2};
-// CHECK-DAG: c:@sf
+// CHECK-DAG: 5:c:@sf
 
 struct SStatic {
   static const int a = 4;
 };
 const int SStatic::a;
-// CHECK-DAG: c:@S@SStatic@a
+// CHECK-DAG: 14:c:@S@SStatic@a
 
 extern int const arr[5] = { 0, 1 };
-// CHECK-DAG: c:@arr
+// CHECK-DAG: 6:c:@arr
 
 union U {
   const int a;
   const unsigned int b;
 };
 U u = {.a = 6};
-// CHECK-DAG: c:@u
+// CHECK-DAG: 4:c:@u
 
 // No USR can be generated for this.
 // Check for no crash in this case.
@@ -48,3 +48,15 @@
   float uf;
   const int ui;
 };
+
+void f(int (*)(char));
+void f(bool (*)(char));
+
+struct G {
+  G() {
+f([](char) -> int { return 42; });
+// CHECK-DAG: 41:c:@S@G@F@G#@Sa@F@operator int (*)(char)#1
+f([](char) -> bool { return true; });
+// CHECK-DAG: 42:c:@S@G@F@G#@Sa@F@operator bool (*)(char)#1
+  }
+};
Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,22 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: %clang_extdef_map %S/Inputs/ctu-lookup-name-with-space.cpp -- | \
+// RUN:   sed 's:%/S/Inputs/ctu-lookup-name-with-space.cpp:%/t/importee.ast:g' > %t/externalDefMap.txt
+
+// RUN: cd %t
+// RUN: %clang_cc1 -emit-pch %/S/Inputs/ctu-lookup-name-with-space.cpp -o %t/importee.ast
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -verify %s
+
+void importee();
+
+void trigger() {
+  // Call an external function to trigger the parsing process of CTU index.
+  // Refer to file Inputs/ctu-lookup-name-with-space.cpp for more details.
+
+  importee(); // expected-no-diagnostics
+}
Index: clang/test/Analysis/ctu-inherited-default-ctor.cpp
===
--- clang/test/Analysis/ctu-inherited-default-ctor.cpp
+++ clang/test/Analysis/ctu-inherited-default-ctor.cpp
@@ -4,7 +4,7 @@
 // RUN: %clang_cc1 -std=c++14 -triple x86_64-pc-linux-gnu \
 // RUN:   -emit-pch -o %t/ctudir/ctu-inherited-default-ctor-other.cpp.ast \
 // RUN:%S/Inputs/ctu-inherited-default-ctor-other.cpp
-// RUN: echo "c:@N@clang@S@DeclContextLookupResult@SingleElementDummyList 

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 397767.
OikawaKirie added a comment.

To make it work on Windows, Linux, and Mac OS, using `echo` to create the 
external function map, and using AST file for CTU analysis.
Tested on Windows, Linux, and Mac OS under x64.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

Files:
  clang/docs/analyzer/user-docs/CrossTranslationUnit.rst
  clang/include/clang/Basic/DiagnosticCrossTUKinds.td
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-import.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/Inputs/ctu-other.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.ast-dump.txt
  
clang/test/Analysis/Inputs/plist-macros-with-expansion-ctu.c.externalDefMap.txt
  clang/test/Analysis/ctu-inherited-default-ctor.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/func-mapping-test.cpp
  clang/unittests/CrossTU/CrossTranslationUnitTest.cpp

Index: clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
===
--- clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
+++ clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
@@ -61,7 +61,7 @@
 ASSERT_FALSE(llvm::sys::fs::createTemporaryFile("index", "txt", IndexFD,
 IndexFileName));
 llvm::ToolOutputFile IndexFile(IndexFileName, IndexFD);
-IndexFile.os() << "c:@F@f#I# " << ASTFileName << "\n";
+IndexFile.os() << "9:c:@F@f#I# " << ASTFileName << "\n";
 IndexFile.os().flush();
 EXPECT_TRUE(llvm::sys::fs::exists(IndexFileName));
 
Index: clang/test/Analysis/func-mapping-test.cpp
===
--- clang/test/Analysis/func-mapping-test.cpp
+++ clang/test/Analysis/func-mapping-test.cpp
@@ -3,10 +3,10 @@
 int f(int) {
   return 0;
 }
-// CHECK-DAG: c:@F@f#I#
+// CHECK-DAG: 9:c:@F@f#I#
 
 extern const int x = 5;
-// CHECK-DAG: c:@x
+// CHECK-DAG: 4:c:@x
 
 // Non-const variables should not be collected.
 int y = 5;
@@ -18,29 +18,29 @@
   int a;
 };
 extern S const s = {.a = 2};
-// CHECK-DAG: c:@s
+// CHECK-DAG: 4:c:@s
 
 struct SF {
   const int a;
 };
 SF sf = {.a = 2};
-// CHECK-DAG: c:@sf
+// CHECK-DAG: 5:c:@sf
 
 struct SStatic {
   static const int a = 4;
 };
 const int SStatic::a;
-// CHECK-DAG: c:@S@SStatic@a
+// CHECK-DAG: 14:c:@S@SStatic@a
 
 extern int const arr[5] = { 0, 1 };
-// CHECK-DAG: c:@arr
+// CHECK-DAG: 6:c:@arr
 
 union U {
   const int a;
   const unsigned int b;
 };
 U u = {.a = 6};
-// CHECK-DAG: c:@u
+// CHECK-DAG: 4:c:@u
 
 // No USR can be generated for this.
 // Check for no crash in this case.
@@ -48,3 +48,15 @@
   float uf;
   const int ui;
 };
+
+void f(int (*)(char));
+void f(bool (*)(char));
+
+struct G {
+  G() {
+f([](char) -> int { return 42; });
+// CHECK-DAG: 41:c:@S@G@F@G#@Sa@F@operator int (*)(char)#1
+f([](char) -> bool { return true; });
+// CHECK-DAG: 42:c:@S@G@F@G#@Sa@F@operator bool (*)(char)#1
+  }
+};
Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,26 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: echo '41:c:@S@G@F@G#@Sa@F@operator void (*)(int)#1 %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: echo '38:c:@S@G@F@G#@Sa@F@operator void (*)()#1 %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: echo '14:c:@F@importee# %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: %clang_cc1 -emit-pch %/S/Inputs/ctu-lookup-name-with-space.cpp -o %t/importee.ast
+
+// RUN: cd %t
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config display-ctu-progress=true \
+// RUN:   -verify %s 2>&1 | FileCheck %s
+
+// CHECK: CTU loaded AST file
+
+void importee();
+
+void trigger() {
+  // Call an external function to trigger the parsing process of CTU index.
+  // Refer to file Inputs/ctu-lookup-name-with-space.cpp for more details.
+
+  importee(); // expected-no-diagnostics
+}
Index: clang/test/Analysis/ctu-inherited-default-ctor.cpp
===
--- clang/test/Analysis/ctu-inherited-default-ctor.cpp
+++ clang/test/Analysis/ctu-inherited-default-ctor.cpp
@@ -4,7 +4,7 @@
 // RUN: %clang_cc1 -std=c++14 -triple x86_64-pc-linux-gnu \
 // RUN:   -emit-pch -o %t/ctudir/ctu-inherited-default-ctor-other.cpp.ast \
 // RUN:%S/Inputs/ctu-inherited-default-ctor-other.cpp
-// RUN: echo "c:@N@clang@S@DeclContextLookupResult@SingleElementDummyList ctu-inherited-default-ctor-other.cpp.ast" \
+// RUN: echo "59:c:@N@clang@S@DeclContextLookupResu

[PATCH] D116924: [clang-extdef-mapping] Allow clang-extdef-mapping tool to output customized filename for each index entry

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: martong, NoQ, steakhal.
OikawaKirie added a project: clang.
Herald added subscribers: arphaman, rnkovacs.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

The clang-extdef-mapping tool can only output the real file name of the input 
source file. When analyzing with AST file based CTU analysis, the file name 
should be adjusted to the corresponding AST file, where `.ast` will be appended 
to the file name, and sometimes the path will even be changed. It is very 
inconvenient to adjust the file path with such a separated step, especially on 
Windows, where utility tools such as `sed` are not available.

This patch uses `Regex::sub` function to adjust the output file name for each 
index entry with the pattern provided by the user.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D116924

Files:
  clang/test/Analysis/func-mapping-path.c
  clang/tools/clang-extdef-mapping/ClangExtDefMapGen.cpp

Index: clang/tools/clang-extdef-mapping/ClangExtDefMapGen.cpp
===
--- clang/tools/clang-extdef-mapping/ClangExtDefMapGen.cpp
+++ clang/tools/clang-extdef-mapping/ClangExtDefMapGen.cpp
@@ -30,11 +30,45 @@
 using namespace clang::tooling;
 
 static cl::OptionCategory ClangExtDefMapGenCategory("clang-extdefmapgen options");
+static cl::opt OutputPathAdjusterMatcher(
+"m", cl::desc("Match the part of output path to be replaced with regex"),
+cl::cat(ClangExtDefMapGenCategory));
+static cl::opt OutputPathAdjusterNewPath(
+"r", cl::desc("The string which matched pattern will be replaced to"),
+cl::cat(ClangExtDefMapGenCategory));
+
+namespace {
+class OutputPathAdjuster {
+  Regex Matcher;
+  std::string NewPath;
+
+public:
+  std::string adjust(StringRef Path) const {
+return Matcher.sub(NewPath, Path);
+  }
+
+  bool resetMatcher(StringRef MatcherStr, StringRef NewPathStr);
+};
+
+bool OutputPathAdjuster::resetMatcher(StringRef MatcherStr,
+  StringRef NewPathStr) {
+  std::string ErrMsg;
+  Regex MatcherRegex(MatcherStr);
+  if (!MatcherRegex.isValid(ErrMsg)) {
+errs() << "Invalid adjuster template: " << ErrMsg << '\n';
+return false;
+  }
+
+  Matcher = std::move(MatcherRegex);
+  NewPath = NewPathStr.str();
+  return true;
+}
+} // namespace
 
 class MapExtDefNamesConsumer : public ASTConsumer {
 public:
-  MapExtDefNamesConsumer(ASTContext &Context)
-  : Ctx(Context), SM(Context.getSourceManager()) {}
+  MapExtDefNamesConsumer(const OutputPathAdjuster *Adjuster)
+  : Adjuster(Adjuster) {}
 
   ~MapExtDefNamesConsumer() {
 // Flush results to standard output.
@@ -49,10 +83,9 @@
   void handleDecl(const Decl *D);
   void addIfInMain(const DeclaratorDecl *DD, SourceLocation defStart);
 
-  ASTContext &Ctx;
-  SourceManager &SM;
   llvm::StringMap Index;
   std::string CurrentFileName;
+  const OutputPathAdjuster *Adjuster = nullptr;
 };
 
 void MapExtDefNamesConsumer::handleDecl(const Decl *D) {
@@ -64,7 +97,7 @@
   if (const Stmt *Body = FD->getBody())
 addIfInMain(FD, Body->getBeginLoc());
   } else if (const auto *VD = dyn_cast(D)) {
-if (cross_tu::containsConst(VD, Ctx) && VD->hasInit())
+if (cross_tu::containsConst(VD, VD->getASTContext()) && VD->hasInit())
   if (const Expr *Init = VD->getInit())
 addIfInMain(VD, Init->getBeginLoc());
   }
@@ -82,9 +115,12 @@
 return;
   assert(!LookupName->empty() && "Lookup name should be non-empty.");
 
+  SourceManager &SM = DD->getASTContext().getSourceManager();
   if (CurrentFileName.empty()) {
 CurrentFileName = std::string(
 SM.getFileEntryForID(SM.getMainFileID())->tryGetRealPathName());
+if (Adjuster)
+  CurrentFileName = Adjuster->adjust(CurrentFileName);
 if (CurrentFileName.empty())
   CurrentFileName = "invalid_file";
   }
@@ -101,11 +137,15 @@
   }
 }
 
-class MapExtDefNamesAction : public ASTFrontendAction {
-protected:
-  std::unique_ptr CreateASTConsumer(CompilerInstance &CI,
- llvm::StringRef) override {
-return std::make_unique(CI.getASTContext());
+class MapExtDefNamesConsumerFactory {
+  const OutputPathAdjuster *Adjuster = nullptr;
+
+public:
+  MapExtDefNamesConsumerFactory(const OutputPathAdjuster *Adjuster)
+  : Adjuster(Adjuster) {}
+
+  std::unique_ptr newASTConsumer() {
+return std::make_unique(Adjuster);
   }
 };
 
@@ -118,7 +158,8 @@
 
   const char *Overview = "\nThis tool collects the USR name and location "
  "of external definitions in the source files "
- "(excluding headers).\n";
+ "(excluding headers).\n\nReset output path with -m "
+ " -r .\n";
   auto ExpectedParser = CommonOptionsParser::create(
   argc, argv, ClangExtDefMapGenCategory, cl::ZeroOrMore, Overview);
   

[PATCH] D116329: [clang-check] Adjust argument adjusters for clang-check to strip options blocking the static analyzer

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 399841.
OikawaKirie added a comment.

Updated as suggested.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D116329/new/

https://reviews.llvm.org/D116329

Files:
  clang/test/Tooling/clang-check-analyze-save-temps.cpp
  clang/tools/clang-check/ClangCheck.cpp


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -208,27 +208,37 @@
   ClangTool Tool(OptionsParser.getCompilations(),
  OptionsParser.getSourcePathList());
 
-  // Clear adjusters because -fsyntax-only is inserted by the default chain.
-  Tool.clearArgumentsAdjusters();
-
-  // Reset output path if is provided by user.
-  Tool.appendArgumentsAdjuster(
-  Analyze ? [&](const CommandLineArguments &Args, StringRef File) {
-  auto Ret = getClangStripOutputAdjuster()(Args, File);
-  if (!AnalyzerOutput.empty()) {
-Ret.emplace_back("-o");
-Ret.emplace_back(AnalyzerOutput);
-  }
-  return Ret;
-}
-  : getClangStripOutputAdjuster());
-
-  Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
-
-  // Running the analyzer requires --analyze. Other modes can work with the
-  // -fsyntax-only option.
-  Tool.appendArgumentsAdjuster(getInsertArgumentAdjuster(
-  Analyze ? "--analyze" : "-fsyntax-only", ArgumentInsertPosition::BEGIN));
+  if (Analyze) {
+// Set output path if is provided by user.
+//
+// As the original -o options have been removed by default via the
+// strip-output adjuster, we only need to add the analyzer -o options here
+// when it is provided by users.
+if (!AnalyzerOutput.empty())
+  Tool.appendArgumentsAdjuster(
+  getInsertArgumentAdjuster(CommandLineArguments{"-o", AnalyzerOutput},
+ArgumentInsertPosition::END));
+
+// Running the analyzer requires --analyze. Other modes can work with the
+// -fsyntax-only option.
+//
+// The syntax-only adjuster is installed by default.
+// Good: It also strips options that trigger extra output, like 
-save-temps.
+// Bad:  We don't want the -fsyntax-only when executing the static 
analyzer.
+//
+// To enable the static analyzer, we first strip all -fsyntax-only options
+// and then add an --analyze option to the front.
+Tool.appendArgumentsAdjuster(
+[&](const CommandLineArguments &Args, StringRef /*unused*/) {
+  CommandLineArguments AdjustedArgs;
+  for (const std::string &Arg : Args)
+if (Arg != "-fsyntax-only")
+  AdjustedArgs.emplace_back(Arg);
+  return AdjustedArgs;
+});
+Tool.appendArgumentsAdjuster(
+getInsertArgumentAdjuster("--analyze", ArgumentInsertPosition::BEGIN));
+  }
 
   ClangCheckActionFactory CheckFactory;
   std::unique_ptr FrontendFactory;
Index: clang/test/Tooling/clang-check-analyze-save-temps.cpp
===
--- /dev/null
+++ clang/test/Tooling/clang-check-analyze-save-temps.cpp
@@ -0,0 +1,19 @@
+// Check whether output generation options (like -save-temps) will not affect
+// the execution of the analyzer.
+
+// RUN: clang-check -analyze %s -- -save-temps -c -Xclang -verify
+
+// Check whether redundant -fsyntax-only options will affect the execution of
+// the analyzer.
+
+// RUN: clang-check -analyze %s -- \
+// RUN:   -fsyntax-only -c -fsyntax-only -Xclang -verify 2>&1 | \
+// RUN:   FileCheck %s --allow-empty
+
+// CHECK-NOT: argument unused during compilation: '--analyze'
+
+void a(int *x) {
+  if (x) {
+  }
+  *x = 47; // expected-warning {{Dereference of null pointer}}
+}


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -208,27 +208,37 @@
   ClangTool Tool(OptionsParser.getCompilations(),
  OptionsParser.getSourcePathList());
 
-  // Clear adjusters because -fsyntax-only is inserted by the default chain.
-  Tool.clearArgumentsAdjusters();
-
-  // Reset output path if is provided by user.
-  Tool.appendArgumentsAdjuster(
-  Analyze ? [&](const CommandLineArguments &Args, StringRef File) {
-  auto Ret = getClangStripOutputAdjuster()(Args, File);
-  if (!AnalyzerOutput.empty()) {
-Ret.emplace_back("-o");
-Ret.emplace_back(AnalyzerOutput);
-  }
-  return Ret;
-}
-  : getClangStripOutputAdjuster());
-
-  Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
-
-  // Running the analyzer requires --analyze. Other modes can work with the
-  // -fsyntax

[PATCH] D116329: [clang-check] Adjust argument adjusters for clang-check to strip options blocking the static analyzer

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

Could you please commit it on my behalf (Ella Ma )?
Thanks a lot.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D116329/new/

https://reviews.llvm.org/D116329

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D116329: [clang-check] Adjust argument adjusters for clang-check to strip options blocking the static analyzer

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

It seems that it is not committed on my behalf. Maybe you have forgotten to add 
me as the author?
Sad : (


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D116329/new/

https://reviews.llvm.org/D116329

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added inline comments.



Comment at: clang/lib/CrossTU/CrossTranslationUnit.cpp:154-172
+  // Find the length delimiter.
+  const size_t LengthDelimiter = LineRef.find(':');
+  if (StringRef::npos == LengthDelimiter)
+return false;
+
+  // Parse the length of LookupName as USRLength.
+  size_t USRLength = 0;

steakhal wrote:
> OikawaKirie wrote:
> > steakhal wrote:
> > > OikawaKirie wrote:
> > > > steakhal wrote:
> > > > > OikawaKirie wrote:
> > > > > > steakhal wrote:
> > > > > > > 
> > > > > > The source of lookup name of the function being imported is 
> > > > > > function `CrossTranslationUnitContext::getLookupName`. Keeping the 
> > > > > > length in the mapping can avoid parsing the lookup name during 
> > > > > > importing.
> > > > > Okay; you can copy the original StringRef to have that. But by 
> > > > > consuming it on one path makes the code much more readable.
> > > > > 
> > > > The `getAsInterger` call can also check whether the content before the 
> > > > first colon is an integer. Therefore, a sub-string operation is 
> > > > required here.
> > > I don't doubt that your proposed way of doing this works and is efficient.
> > > What I say is that I think there is room for improvement in the 
> > > non-functional aspects, in the readability. However, it's not really a 
> > > blocking issue, more of a personal taste.
> > I know what you are considering, it is clearer and more readable by 
> > consuming the length, then the USR. However, to correctly separate the USR 
> > and file path, the length of `USR-Length` is also required, which makes it 
> > impossible to just *consume* the length at the beginning.
> > 
> > Another way of solving this problem is to re-create the string with the 
> > USR-Length and the USR after parsing, but I think it is not a good solution.
> > 
> > BTW, is it necessary to assert the `USR-Length` to be greater than zero? I 
> > think it is more reasonable to report *invalid format* rather than assert 
> > the value, as it can be provided by the user.
> I think what causes the misunderstanding is the definition of //consume// in 
> the context of `StringRef`.
> ```lang=C++
> const StringRef Copy = Line;
> Line.consumeInteger(...); // Line advances forward by the number of 
> characters that were parsed as an integral value.
> // Copy still refers to the unmodified, original characters.
> // I can use it however I want.
> 
> // `Line` is a suffix of `Copy`, and the `.end()` should be the same, only 
> `.begin()` should differ.
> ```
> 
> I hope that caused the miscommunication.
> 
> ---
> > BTW, is it necessary to assert the USR-Length to be greater than zero? I 
> > think it is more reasonable to report *invalid format* rather than assert 
> > the value, as it can be provided by the user.
> Yeah, sure!
I think I have figured out what have been misunderstood.

In the current patch, I just modify function 
`CrossTranslationUnitContext::getLookupName` by adding a length at the 
beginning. Therefore, the lookup name for the CTU query will have the length 
part. And for the sake of simplicity and efficiency, the length together with 
the USR is stored in the mapping as the key.

To correctly parse the `:` part, I cannot just consume the 
`` at the beginning. Otherwise, I cannot know the length of 
`:` part, which makes it impossible to parse the entire 
`:` part, even though the original `Line` is copied.

I will update the approach of adding the length part. Since the length is only 
used during parsing the CTU index, I will modify function 
`createCrossTUIndexString` to add the length and revert the changes to function 
`CrossTranslationUnitContext::getLookupName` to keep the lookup name for CTU 
query unchanged.



Comment at: clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp:12-14
+int importee(int X) {
+  return 1 / X;
+}

steakhal wrote:
> Also fixup the return type in the declaration within the main TU. Also add 
> the `// expected-no-diagnostics` comment to the primary TU.
Yes, you are right.
I was misled by myself.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 393064.
OikawaKirie added a comment.

1. Revert function `CrossTranslationUnitContext::getLookupName`
2. Add length when dumping the CTU index mapping via function 
`createCrossTUIndexString`
3. Remove the assertions during CTU map query process
4. Make function `parseCrossTUIndexItem` more readable


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

Files:
  clang/docs/analyzer/user-docs/CrossTranslationUnit.rst
  clang/include/clang/Basic/DiagnosticCrossTUKinds.td
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-import.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/Inputs/ctu-other.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.ast-dump.txt
  
clang/test/Analysis/Inputs/plist-macros-with-expansion-ctu.c.externalDefMap.txt
  clang/test/Analysis/ctu-inherited-default-ctor.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/func-mapping-test.cpp
  clang/unittests/CrossTU/CrossTranslationUnitTest.cpp

Index: clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
===
--- clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
+++ clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
@@ -61,7 +61,7 @@
 ASSERT_FALSE(llvm::sys::fs::createTemporaryFile("index", "txt", IndexFD,
 IndexFileName));
 llvm::ToolOutputFile IndexFile(IndexFileName, IndexFD);
-IndexFile.os() << "c:@F@f#I# " << ASTFileName << "\n";
+IndexFile.os() << "9:c:@F@f#I# " << ASTFileName << "\n";
 IndexFile.os().flush();
 EXPECT_TRUE(llvm::sys::fs::exists(IndexFileName));
 
Index: clang/test/Analysis/func-mapping-test.cpp
===
--- clang/test/Analysis/func-mapping-test.cpp
+++ clang/test/Analysis/func-mapping-test.cpp
@@ -3,10 +3,10 @@
 int f(int) {
   return 0;
 }
-// CHECK-DAG: c:@F@f#I#
+// CHECK-DAG: 9:c:@F@f#I#
 
 extern const int x = 5;
-// CHECK-DAG: c:@x
+// CHECK-DAG: 4:c:@x
 
 // Non-const variables should not be collected.
 int y = 5;
@@ -18,29 +18,29 @@
   int a;
 };
 extern S const s = {.a = 2};
-// CHECK-DAG: c:@s
+// CHECK-DAG: 4:c:@s
 
 struct SF {
   const int a;
 };
 SF sf = {.a = 2};
-// CHECK-DAG: c:@sf
+// CHECK-DAG: 5:c:@sf
 
 struct SStatic {
   static const int a = 4;
 };
 const int SStatic::a;
-// CHECK-DAG: c:@S@SStatic@a
+// CHECK-DAG: 14:c:@S@SStatic@a
 
 extern int const arr[5] = { 0, 1 };
-// CHECK-DAG: c:@arr
+// CHECK-DAG: 6:c:@arr
 
 union U {
   const int a;
   const unsigned int b;
 };
 U u = {.a = 6};
-// CHECK-DAG: c:@u
+// CHECK-DAG: 4:c:@u
 
 // No USR can be generated for this.
 // Check for no crash in this case.
@@ -48,3 +48,15 @@
   float uf;
   const int ui;
 };
+
+void f(int (*)(char));
+void f(bool (*)(char));
+
+struct G {
+  G() {
+f([](char) -> int { return 42; });
+// CHECK-DAG: 41:c:@S@G@F@G#@Sa@F@operator int (*)(char)#1
+f([](char) -> bool { return true; });
+// CHECK-DAG: 42:c:@S@G@F@G#@Sa@F@operator bool (*)(char)#1
+  }
+};
Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,25 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: cp %s %t/trigger.cpp
+// RUN: cp %S/Inputs/ctu-lookup-name-with-space.cpp %t/importee.cpp
+// RUN: echo '"%t/importee.cpp" : ["g++", "-c", "%t/importee.cpp"]' > %t/invocations.yaml
+// RUN: echo '[{"directory": "%t", "command": "g++ -c %t/importee.cpp", "file": "%t/importee.cpp"}, {"directory": "%t", "command": "g++ -c %t/trigger.cpp", "file": "%t/trigger.cpp"}]' > %t/compile_commands.json
+
+// RUN: %clang_extdef_map -p %t %t/importee.cpp > %t/externalDefMap.txt
+
+// RUN: cd %t && %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core,debug.ExprInspection \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   -verify trigger.cpp
+
+void importee();
+
+void trigger() {
+  // Call an external function to trigger the parsing process of CTU index.
+  // Refer to file Inputs/ctu-lookup-name-with-space.cpp for more details.
+
+  importee(); // expected-no-diagnostics
+}
Index: clang/test/Analysis/ctu-inherited-default-ctor.cpp
===
--- clang/test/Analysis/ctu-inherited-default-ctor.cpp
+++ clang/test/Analysis/ctu-inherited-default-ctor.cpp
@@ -4,7 +4,7 @@
 // RUN: %clang_cc1 -std=c++14 -triple x86_64-pc-linux-gnu \
 // RUN:   -emit-pch -o %t/ctudir/ctu-inherited-default-ctor-other.cpp.ast \
 // RUN:%S/Inputs/ctu-inherited-default-ctor-other.cpp
-/

[PATCH] D115716: [Analyzer][BugReporter] Replace the example bug report with the one used to generate PathDiagnostic

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: NoQ, steakhal, vsavchenko, ASDenysPetrov.
OikawaKirie added a project: clang.
Herald added subscribers: manas, martong, dkrupp, donat.nagy, Szelethus, 
mikhail.ramalho, a.sidorin, szepet, baloghadamsoftware, xazax.hun.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

In function `BugReporter::FlushReport`, function 
`PathSensitiveBugReporter::generatePathDiagnostics` will re-select one report 
from the equivalence class to construct the `PathDiagnostic`. It makes the 
report used to generate `PathDiagnostic` may be different from the one used in 
function `BugReporter::FlushReport`. Changes to the bug report via a 
`BugReporterVisitor` may not be reflected when generating report notes or 
fix-hints.

This patch tries to fix this problem by resetting the pointer to the example 
report selected in function `BugReporter::FlushReport` with the one used to 
generate `PathDiagnostic`.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D115716

Files:
  clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporter.h
  clang/lib/StaticAnalyzer/Core/BugReporter.cpp
  clang/test/Analysis/keychainAPI.m
  clang/test/Analysis/malloc-plist.c
  clang/test/Analysis/stream.c

Index: clang/test/Analysis/stream.c
===
--- clang/test/Analysis/stream.c
+++ clang/test/Analysis/stream.c
@@ -261,9 +261,7 @@
   if (!F1)
 return;
   if (Test == 1) {
-return; // no warning
+return; // expected-warning {{Opened stream never closed. Potential resource leak}}
   }
   rewind(F1);
-} // expected-warning {{Opened stream never closed. Potential resource leak}}
-// FIXME: This warning should be placed at the `return` above.
-// See https://reviews.llvm.org/D83120 about details.
+} // no warning
Index: clang/test/Analysis/malloc-plist.c
===
--- clang/test/Analysis/malloc-plist.c
+++ clang/test/Analysis/malloc-plist.c
@@ -135,8 +135,8 @@
 static void function_with_leak3(int y) {
 char *x = (char*)malloc(12);
 if (y)
-y++;
-}//expected-warning{{Potential leak}}
+y++;//expected-warning{{Potential leak}}
+}
 void use_function_with_leak3(int y) {
 function_with_leak3(y);
 }
Index: clang/test/Analysis/keychainAPI.m
===
--- clang/test/Analysis/keychainAPI.m
+++ clang/test/Analysis/keychainAPI.m
@@ -402,10 +402,10 @@
 OSStatus st = 0;
 void *outData = my_AllocateReturn(&st); 
 if (x) {
-  consumeChar(*(char*)outData); // expected-warning{{Allocated data is not released:}}
+  consumeChar(*(char*)outData);
   return;
 } else {
-  consumeChar(*(char*)outData);
+  consumeChar(*(char*)outData); // expected-warning{{Allocated data is not released:}}
 }
 return;
 }
Index: clang/lib/StaticAnalyzer/Core/BugReporter.cpp
===
--- clang/lib/StaticAnalyzer/Core/BugReporter.cpp
+++ clang/lib/StaticAnalyzer/Core/BugReporter.cpp
@@ -244,6 +244,9 @@
   std::unique_ptr
   generate(const PathDiagnosticConsumer *PDC) const;
 
+  /// Get the bug report used to generate the PathDiagnostic.
+  const PathSensitiveBugReport *getBugReport() const { return R; }
+
 private:
   void updateStackPiecesWithMessage(PathDiagnosticPieceRef P,
 const CallWithEntryStack &CallStack) const;
@@ -271,8 +274,6 @@
   PathDiagnosticLocation
   ExecutionContinues(llvm::raw_string_ostream &os,
  const PathDiagnosticConstruct &C) const;
-
-  const PathSensitiveBugReport *getBugReport() const { return R; }
 };
 
 } // namespace
@@ -2874,6 +2875,7 @@
 
 std::unique_ptr
 PathSensitiveBugReporter::generatePathDiagnostics(
+const BugReport *&exampleReport,
 ArrayRef consumers,
 ArrayRef &bugReports) {
   assert(!bugReports.empty());
@@ -2889,6 +2891,7 @@
 (*Out)[PC] = std::move(PD);
   }
 }
+exampleReport = PDB->getBugReport();
   }
 
   return Out;
@@ -3062,7 +3065,7 @@
 
 void BugReporter::FlushReport(BugReportEquivClass& EQ) {
   SmallVector bugReports;
-  BugReport *report = findReportInEquivalenceClass(EQ, bugReports);
+  const BugReport *report = findReportInEquivalenceClass(EQ, bugReports);
   if (!report)
 return;
 
@@ -3200,7 +3203,8 @@
 
 std::unique_ptr
 BugReporter::generateDiagnosticForConsumerMap(
-BugReport *exampleReport, ArrayRef consumers,
+const BugReport *&exampleReport,
+ArrayRef consumers,
 ArrayRef bugReports) {
   auto *basicReport = cast(exampleReport);
   auto Out = std::make_unique();
@@ -3272,11 +3276,10 @@
   }
 }
 
-
-
 std::unique_ptr
 PathSensitiveBugReporter::generateDiagnosticForConsumerMap(
-BugReport *exampleReport, ArrayRef consumers,
+const BugReport *&exampleReport,
+ArrayRef consumers,
 Arr

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

When running my test case `ctu-lookup-name-with-space.cpp` on Windows, 
`llvm-lit` reports `'cp': command not found`. And this is the reason why it 
fails on Windows.
And when I remove the `cp`s and replace them with original file names, clang 
reports `YAML:1:293: error: Unrecognized escape code`, it seems that the static 
analyzer only reads compilation database in YAML format on Windows.
Should I disable this test case on Windows? Or is there any other approaches to 
make it work on Windows?


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits