[PATCH] D80822: Run syntax tree tests in many language modes
hlopko accepted this revision. hlopko added a comment. This revision is now accepted and ready to land. LGTM, thanks! Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D80822/new/ https://reviews.llvm.org/D80822 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D81000: Renamed Lang_C to Lang_C99, Lang_CXX to Lang_CXX03, and 2a to 20
hlopko accepted this revision. hlopko added a comment. This revision is now accepted and ready to land. Looks good, thanks! Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D81000/new/ https://reviews.llvm.org/D81000 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D81019: Syntax tree: ignore implicit expressions at the top level of statements
hlopko accepted this revision. hlopko added a comment. This revision is now accepted and ready to land. LGTM :) Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D81019/new/ https://reviews.llvm.org/D81019 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D81040: Split syntax tree tests into more granular ones
hlopko accepted this revision. hlopko added a comment. Thanks for working on this! :) Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:2066 +int func3b(int *); +int func4(int a, float b); +int func4a(int, float); func4 -> func4a func4a -> func4b? Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D81040/new/ https://reviews.llvm.org/D81040 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D84623: Remove HAVE_VCS_VERSION_INC, not needed
hlopko created this revision.
hlopko added a reviewer: gribozavr2.
Herald added subscribers: llvm-commits, lldb-commits, cfe-commits, mgorny.
Herald added a reviewer: MaskRay.
Herald added projects: clang, LLDB, LLVM.
This preprocessor define was meant to be used to conditionally include
headers with VCS information (for example VCSRevision.inc). However, the
define was always set, and it was the content of the header that was
conditionally generated. Therefore HAVE_VCS_VERSION_INC should be
cleaned up.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D84623
Files:
clang/lib/Basic/CMakeLists.txt
clang/lib/Basic/Version.cpp
lld/Common/CMakeLists.txt
lld/Common/Version.cpp
lldb/source/CMakeLists.txt
lldb/source/lldb.cpp
llvm/utils/gn/secondary/clang/lib/Basic/BUILD.gn
llvm/utils/gn/secondary/lld/Common/BUILD.gn
Index: llvm/utils/gn/secondary/lld/Common/BUILD.gn
===
--- llvm/utils/gn/secondary/lld/Common/BUILD.gn
+++ llvm/utils/gn/secondary/lld/Common/BUILD.gn
@@ -42,5 +42,4 @@
"Timer.cpp",
"Version.cpp",
]
- defines = [ "HAVE_VCS_VERSION_INC" ] # For Version.cpp
}
Index: llvm/utils/gn/secondary/clang/lib/Basic/BUILD.gn
===
--- llvm/utils/gn/secondary/clang/lib/Basic/BUILD.gn
+++ llvm/utils/gn/secondary/clang/lib/Basic/BUILD.gn
@@ -102,5 +102,4 @@
"XRayInstr.cpp",
"XRayLists.cpp",
]
- defines = [ "HAVE_VCS_VERSION_INC" ] # For Version.cpp
}
Index: lldb/source/lldb.cpp
===
--- lldb/source/lldb.cpp
+++ lldb/source/lldb.cpp
@@ -13,9 +13,7 @@
#include "clang/Basic/Version.h"
-#ifdef HAVE_VCS_VERSION_INC
#include "VCSVersion.inc"
-#endif
static const char *GetLLDBRevision() {
#ifdef LLDB_REVISION
Index: lldb/source/CMakeLists.txt
===
--- lldb/source/CMakeLists.txt
+++ lldb/source/CMakeLists.txt
@@ -34,9 +34,6 @@
PROPERTIES GENERATED TRUE
HEADER_FILE_ONLY TRUE)
-set_property(SOURCE lldb.cpp APPEND PROPERTY
- COMPILE_DEFINITIONS "HAVE_VCS_VERSION_INC")
-
list(APPEND lldbBase_SOURCES ${version_inc})
if(LLDB_VERSION_STRING)
Index: lld/Common/Version.cpp
===
--- lld/Common/Version.cpp
+++ lld/Common/Version.cpp
@@ -12,9 +12,7 @@
#include "lld/Common/Version.h"
-#ifdef HAVE_VCS_VERSION_INC
#include "VCSVersion.inc"
-#endif
// Returns a version string, e.g.:
// lld 9.0.0 (https://github.com/llvm/llvm-project.git 9efdd7ac5e914d3c9fa1ef)
Index: lld/Common/CMakeLists.txt
===
--- lld/Common/CMakeLists.txt
+++ lld/Common/CMakeLists.txt
@@ -20,9 +20,6 @@
PROPERTIES GENERATED TRUE
HEADER_FILE_ONLY TRUE)
-set_property(SOURCE Version.cpp APPEND PROPERTY
- COMPILE_DEFINITIONS "HAVE_VCS_VERSION_INC")
-
add_lld_library(lldCommon
Args.cpp
DWARF.cpp
Index: clang/lib/Basic/Version.cpp
===
--- clang/lib/Basic/Version.cpp
+++ clang/lib/Basic/Version.cpp
@@ -17,9 +17,7 @@
#include
#include
-#ifdef HAVE_VCS_VERSION_INC
#include "VCSVersion.inc"
-#endif
namespace clang {
Index: clang/lib/Basic/CMakeLists.txt
===
--- clang/lib/Basic/CMakeLists.txt
+++ clang/lib/Basic/CMakeLists.txt
@@ -33,9 +33,6 @@
PROPERTIES GENERATED TRUE
HEADER_FILE_ONLY TRUE)
-set_property(SOURCE Version.cpp APPEND PROPERTY
- COMPILE_DEFINITIONS "HAVE_VCS_VERSION_INC")
-
add_clang_library(clangBasic
Attributes.cpp
Builtins.cpp
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D77209: [Syntax] Add mapping from spelled to expanded tokens for TokenBuffer
hlopko updated this revision to Diff 255326.
hlopko marked 4 inline comments as done.
hlopko added a comment.
Adressing comments
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D77209/new/
https://reviews.llvm.org/D77209
Files:
clang/include/clang/Tooling/Syntax/Tokens.h
clang/lib/Tooling/Syntax/Tokens.cpp
clang/unittests/Tooling/Syntax/TokensTest.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1215,6 +1215,9 @@
| `-}
`-}
)txt");
+}
+
+TEST_F(SyntaxTreeTest, ModifiableNodes) {
// All nodes can be mutated.
expectTreeDumpEqual(
R"cpp(
Index: clang/unittests/Tooling/Syntax/TokensTest.cpp
===
--- clang/unittests/Tooling/Syntax/TokensTest.cpp
+++ clang/unittests/Tooling/Syntax/TokensTest.cpp
@@ -57,6 +57,7 @@
using ::testing::Contains;
using ::testing::ElementsAre;
using ::testing::Field;
+using ::testing::IsEmpty;
using ::testing::Matcher;
using ::testing::Not;
using ::testing::Pointee;
@@ -185,10 +186,14 @@
template
llvm::ArrayRef findSubrange(llvm::ArrayRef Subrange,
llvm::ArrayRef Range, Eq F) {
-for (auto Begin = Range.begin(); Begin < Range.end(); ++Begin) {
+assert(Subrange.size() >= 1);
+if (Range.size() < Subrange.size())
+ return llvm::makeArrayRef(Range.end(), Range.end());
+for (auto Begin = Range.begin(), Last = Range.end() - Subrange.size();
+ Begin <= Last; ++Begin) {
auto It = Begin;
- for (auto ItSub = Subrange.begin();
- ItSub != Subrange.end() && It != Range.end(); ++ItSub, ++It) {
+ for (auto ItSub = Subrange.begin(); ItSub != Subrange.end();
+ ++ItSub, ++It) {
if (!F(*ItSub, *It))
goto continue_outer;
}
@@ -889,4 +894,111 @@
ASSERT_EQ(Code.points().size(), 8u);
}
+TEST_F(TokenBufferTest, ExpandedBySpelled) {
+ recordTokens(R"cpp(
+a1 a2 a3 b1 b2
+ )cpp");
+ // Sanity check: expanded and spelled tokens are stored separately.
+ EXPECT_THAT(findExpanded("a1 a2"), Not(SameRange(findSpelled("a1 a2";
+ // Searching for subranges of expanded tokens should give the corresponding
+ // spelled ones.
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1 a2 a3 b1 b2")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1 a2 a3")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("b1 b2")),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+
+ // Test search on simple macro expansions.
+ recordTokens(R"cpp(
+#define A a1 a2 a3
+#define B b1 b2
+
+A split B
+ )cpp");
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("A split B")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 split b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("A split").drop_back()),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("split B").drop_front()),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+
+ // Ranges not fully covering macro expansions should fail.
+ recordTokens(R"cpp(
+#define ID(x) x
+
+ID(a)
+ )cpp");
+ // Spelled don't cover entire mapping (missing ID token) -> empty result
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("( a )")), IsEmpty());
+ // Spelled don't cover entire mapping (missing ) token) -> empty result
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( a")), IsEmpty());
+
+ // Recursive macro invocations.
+ recordTokens(R"cpp(
+#define ID(x) x
+#define B b1 b2
+
+ID(ID(ID(a1) a2 a3)) split ID(B)
+ )cpp");
+
+ EXPECT_THAT(
+ Buffer.expandedForSpelled(findSpelled("ID ( ID ( ID ( a1 ) a2 a3 ) )")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( B )")),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(
+ findSpelled("ID ( ID ( ID ( a1 ) a2 a3 ) ) split ID ( B )")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 split b1 b2";
+ // FIXME: these should succeed, but we do not support macro arguments yet.
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1")), IsEmpty());
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( a1 ) a2")),
+ IsEmpty());
+
+ // Empty macro expansions.
+ recordTokens(R"cpp(
+#define EMPTY
+#define ID(X) X
+
+EMPTY EMPTY ID(1 2 3) EMPTY EMPTY split1
+EMPTY EMPTY ID(4 5 6) split2
+ID(7 8 9) EMPTY EMPTY
+ )cpp");
+ //
[PATCH] D77209: [Syntax] Add mapping from spelled to expanded tokens for TokenBuffer
hlopko marked an inline comment as done. hlopko added a comment. Adressing comments. Comment at: clang/lib/Tooling/Syntax/Tokens.cpp:264 + auto *FrontMapping = mappingStartingBeforeSpelled(File, &Spelled.front()); + unsigned SpelledFrontI = &Spelled.front() - File.SpelledTokens.data(); + unsigned ExpandedBegin; gribozavr2 wrote: > Or assert that SpelledFrontI is less than File.SpelledTokens.size(). I think the assert I added is good enough? Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D77209/new/ https://reviews.llvm.org/D77209 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D77209: [Syntax] Add mapping from spelled to expanded tokens for TokenBuffer
hlopko updated this revision to Diff 255609.
hlopko marked 6 inline comments as done.
hlopko added a comment.
Adressing comments
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D77209/new/
https://reviews.llvm.org/D77209
Files:
clang/include/clang/Tooling/Syntax/Tokens.h
clang/lib/Tooling/Syntax/Tokens.cpp
clang/unittests/Tooling/Syntax/TokensTest.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1215,6 +1215,9 @@
| `-}
`-}
)txt");
+}
+
+TEST_F(SyntaxTreeTest, ModifiableNodes) {
// All nodes can be mutated.
expectTreeDumpEqual(
R"cpp(
Index: clang/unittests/Tooling/Syntax/TokensTest.cpp
===
--- clang/unittests/Tooling/Syntax/TokensTest.cpp
+++ clang/unittests/Tooling/Syntax/TokensTest.cpp
@@ -57,6 +57,7 @@
using ::testing::Contains;
using ::testing::ElementsAre;
using ::testing::Field;
+using ::testing::IsEmpty;
using ::testing::Matcher;
using ::testing::Not;
using ::testing::Pointee;
@@ -185,10 +186,14 @@
template
llvm::ArrayRef findSubrange(llvm::ArrayRef Subrange,
llvm::ArrayRef Range, Eq F) {
-for (auto Begin = Range.begin(); Begin < Range.end(); ++Begin) {
+assert(Subrange.size() >= 1);
+if (Range.size() < Subrange.size())
+ return llvm::makeArrayRef(Range.end(), Range.end());
+for (auto Begin = Range.begin(), Last = Range.end() - Subrange.size();
+ Begin <= Last; ++Begin) {
auto It = Begin;
- for (auto ItSub = Subrange.begin();
- ItSub != Subrange.end() && It != Range.end(); ++ItSub, ++It) {
+ for (auto ItSub = Subrange.begin(); ItSub != Subrange.end();
+ ++ItSub, ++It) {
if (!F(*ItSub, *It))
goto continue_outer;
}
@@ -889,4 +894,111 @@
ASSERT_EQ(Code.points().size(), 8u);
}
+TEST_F(TokenBufferTest, ExpandedBySpelled) {
+ recordTokens(R"cpp(
+a1 a2 a3 b1 b2
+ )cpp");
+ // Sanity check: expanded and spelled tokens are stored separately.
+ EXPECT_THAT(findExpanded("a1 a2"), Not(SameRange(findSpelled("a1 a2";
+ // Searching for subranges of expanded tokens should give the corresponding
+ // spelled ones.
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1 a2 a3 b1 b2")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1 a2 a3")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("b1 b2")),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+
+ // Test search on simple macro expansions.
+ recordTokens(R"cpp(
+#define A a1 a2 a3
+#define B b1 b2
+
+A split B
+ )cpp");
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("A split B")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 split b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("A split").drop_back()),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("split B").drop_front()),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+
+ // Ranges not fully covering macro expansions should fail.
+ recordTokens(R"cpp(
+#define ID(x) x
+
+ID(a)
+ )cpp");
+ // Spelled don't cover entire mapping (missing ID token) -> empty result
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("( a )")), IsEmpty());
+ // Spelled don't cover entire mapping (missing ) token) -> empty result
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( a")), IsEmpty());
+
+ // Recursive macro invocations.
+ recordTokens(R"cpp(
+#define ID(x) x
+#define B b1 b2
+
+ID(ID(ID(a1) a2 a3)) split ID(B)
+ )cpp");
+
+ EXPECT_THAT(
+ Buffer.expandedForSpelled(findSpelled("ID ( ID ( ID ( a1 ) a2 a3 ) )")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( B )")),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(
+ findSpelled("ID ( ID ( ID ( a1 ) a2 a3 ) ) split ID ( B )")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 split b1 b2";
+ // FIXME: these should succeed, but we do not support macro arguments yet.
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1")), IsEmpty());
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( a1 ) a2")),
+ IsEmpty());
+
+ // Empty macro expansions.
+ recordTokens(R"cpp(
+#define EMPTY
+#define ID(X) X
+
+EMPTY EMPTY ID(1 2 3) EMPTY EMPTY split1
+EMPTY EMPTY ID(4 5 6) split2
+ID(7 8 9) EMPTY EMPTY
+ )cpp");
+ //
[PATCH] D77209: [Syntax] Add mapping from spelled to expanded tokens for TokenBuffer
hlopko added inline comments.
Comment at: clang/lib/Tooling/Syntax/Tokens.cpp:264
+ auto *FrontMapping = mappingStartingBeforeSpelled(File, &Spelled.front());
+ unsigned SpelledFrontI = &Spelled.front() - File.SpelledTokens.data();
+ unsigned ExpandedBegin;
gribozavr2 wrote:
> hlopko wrote:
> > gribozavr2 wrote:
> > > Or assert that SpelledFrontI is less than File.SpelledTokens.size().
> > I think the assert I added is good enough?
> The assertion that I suggested is stronger, because it would prevent an
> out-of-bounds read from even happening, and would not rely on
> `isBeforeInTranslationUnit` returning false on garbage data.
Great points! Done.
Comment at: clang/lib/Tooling/Syntax/Tokens.cpp:602
+for (auto &pair : Result.Files) {
+ auto &mappings = pair.second.Mappings;
+ assert(std::is_sorted(
gribozavr2 wrote:
> We'd get an "unused variable" warning here when assertions are disabled.
> Please wrap the whole loop in `#ifndef NDEBUG`.
Done, added ifndef also above for T1/T2 variables.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D77209/new/
https://reviews.llvm.org/D77209
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D77209: [Syntax] Add mapping from spelled to expanded tokens for TokenBuffer
hlopko updated this revision to Diff 255610.
hlopko added a comment.
Rebase
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D77209/new/
https://reviews.llvm.org/D77209
Files:
clang/include/clang/Tooling/Syntax/Tokens.h
clang/lib/Tooling/Syntax/Tokens.cpp
clang/unittests/Tooling/Syntax/TokensTest.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1215,6 +1215,9 @@
| `-}
`-}
)txt");
+}
+
+TEST_F(SyntaxTreeTest, ModifiableNodes) {
// All nodes can be mutated.
expectTreeDumpEqual(
R"cpp(
Index: clang/unittests/Tooling/Syntax/TokensTest.cpp
===
--- clang/unittests/Tooling/Syntax/TokensTest.cpp
+++ clang/unittests/Tooling/Syntax/TokensTest.cpp
@@ -57,6 +57,7 @@
using ::testing::Contains;
using ::testing::ElementsAre;
using ::testing::Field;
+using ::testing::IsEmpty;
using ::testing::Matcher;
using ::testing::Not;
using ::testing::Pointee;
@@ -185,10 +186,14 @@
template
llvm::ArrayRef findSubrange(llvm::ArrayRef Subrange,
llvm::ArrayRef Range, Eq F) {
-for (auto Begin = Range.begin(); Begin < Range.end(); ++Begin) {
+assert(Subrange.size() >= 1);
+if (Range.size() < Subrange.size())
+ return llvm::makeArrayRef(Range.end(), Range.end());
+for (auto Begin = Range.begin(), Last = Range.end() - Subrange.size();
+ Begin <= Last; ++Begin) {
auto It = Begin;
- for (auto ItSub = Subrange.begin();
- ItSub != Subrange.end() && It != Range.end(); ++ItSub, ++It) {
+ for (auto ItSub = Subrange.begin(); ItSub != Subrange.end();
+ ++ItSub, ++It) {
if (!F(*ItSub, *It))
goto continue_outer;
}
@@ -889,4 +894,111 @@
ASSERT_EQ(Code.points().size(), 8u);
}
+TEST_F(TokenBufferTest, ExpandedBySpelled) {
+ recordTokens(R"cpp(
+a1 a2 a3 b1 b2
+ )cpp");
+ // Sanity check: expanded and spelled tokens are stored separately.
+ EXPECT_THAT(findExpanded("a1 a2"), Not(SameRange(findSpelled("a1 a2";
+ // Searching for subranges of expanded tokens should give the corresponding
+ // spelled ones.
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1 a2 a3 b1 b2")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1 a2 a3")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("b1 b2")),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+
+ // Test search on simple macro expansions.
+ recordTokens(R"cpp(
+#define A a1 a2 a3
+#define B b1 b2
+
+A split B
+ )cpp");
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("A split B")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 split b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("A split").drop_back()),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("split B").drop_front()),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+
+ // Ranges not fully covering macro expansions should fail.
+ recordTokens(R"cpp(
+#define ID(x) x
+
+ID(a)
+ )cpp");
+ // Spelled don't cover entire mapping (missing ID token) -> empty result
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("( a )")), IsEmpty());
+ // Spelled don't cover entire mapping (missing ) token) -> empty result
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( a")), IsEmpty());
+
+ // Recursive macro invocations.
+ recordTokens(R"cpp(
+#define ID(x) x
+#define B b1 b2
+
+ID(ID(ID(a1) a2 a3)) split ID(B)
+ )cpp");
+
+ EXPECT_THAT(
+ Buffer.expandedForSpelled(findSpelled("ID ( ID ( ID ( a1 ) a2 a3 ) )")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( B )")),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(
+ findSpelled("ID ( ID ( ID ( a1 ) a2 a3 ) ) split ID ( B )")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 split b1 b2";
+ // FIXME: these should succeed, but we do not support macro arguments yet.
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1")), IsEmpty());
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( a1 ) a2")),
+ IsEmpty());
+
+ // Empty macro expansions.
+ recordTokens(R"cpp(
+#define EMPTY
+#define ID(X) X
+
+EMPTY EMPTY ID(1 2 3) EMPTY EMPTY split1
+EMPTY EMPTY ID(4 5 6) split2
+ID(7 8 9) EMPTY EMPTY
+ )cpp");
+ // Covered by empty expansions on one of both of the si
[PATCH] D77209: [Syntax] Add mapping from spelled to expanded tokens for TokenBuffer
hlopko updated this revision to Diff 255654.
hlopko added a comment.
Assert the Spelled subrange more effectively
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D77209/new/
https://reviews.llvm.org/D77209
Files:
clang/include/clang/Tooling/Syntax/Tokens.h
clang/lib/Tooling/Syntax/Tokens.cpp
clang/unittests/Tooling/Syntax/TokensTest.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1215,6 +1215,9 @@
| `-}
`-}
)txt");
+}
+
+TEST_F(SyntaxTreeTest, ModifiableNodes) {
// All nodes can be mutated.
expectTreeDumpEqual(
R"cpp(
Index: clang/unittests/Tooling/Syntax/TokensTest.cpp
===
--- clang/unittests/Tooling/Syntax/TokensTest.cpp
+++ clang/unittests/Tooling/Syntax/TokensTest.cpp
@@ -57,6 +57,7 @@
using ::testing::Contains;
using ::testing::ElementsAre;
using ::testing::Field;
+using ::testing::IsEmpty;
using ::testing::Matcher;
using ::testing::Not;
using ::testing::Pointee;
@@ -185,10 +186,14 @@
template
llvm::ArrayRef findSubrange(llvm::ArrayRef Subrange,
llvm::ArrayRef Range, Eq F) {
-for (auto Begin = Range.begin(); Begin < Range.end(); ++Begin) {
+assert(Subrange.size() >= 1);
+if (Range.size() < Subrange.size())
+ return llvm::makeArrayRef(Range.end(), Range.end());
+for (auto Begin = Range.begin(), Last = Range.end() - Subrange.size();
+ Begin <= Last; ++Begin) {
auto It = Begin;
- for (auto ItSub = Subrange.begin();
- ItSub != Subrange.end() && It != Range.end(); ++ItSub, ++It) {
+ for (auto ItSub = Subrange.begin(); ItSub != Subrange.end();
+ ++ItSub, ++It) {
if (!F(*ItSub, *It))
goto continue_outer;
}
@@ -889,4 +894,111 @@
ASSERT_EQ(Code.points().size(), 8u);
}
+TEST_F(TokenBufferTest, ExpandedBySpelled) {
+ recordTokens(R"cpp(
+a1 a2 a3 b1 b2
+ )cpp");
+ // Sanity check: expanded and spelled tokens are stored separately.
+ EXPECT_THAT(findExpanded("a1 a2"), Not(SameRange(findSpelled("a1 a2";
+ // Searching for subranges of expanded tokens should give the corresponding
+ // spelled ones.
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1 a2 a3 b1 b2")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1 a2 a3")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("b1 b2")),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+
+ // Test search on simple macro expansions.
+ recordTokens(R"cpp(
+#define A a1 a2 a3
+#define B b1 b2
+
+A split B
+ )cpp");
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("A split B")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 split b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("A split").drop_back()),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("split B").drop_front()),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+
+ // Ranges not fully covering macro expansions should fail.
+ recordTokens(R"cpp(
+#define ID(x) x
+
+ID(a)
+ )cpp");
+ // Spelled don't cover entire mapping (missing ID token) -> empty result
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("( a )")), IsEmpty());
+ // Spelled don't cover entire mapping (missing ) token) -> empty result
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( a")), IsEmpty());
+
+ // Recursive macro invocations.
+ recordTokens(R"cpp(
+#define ID(x) x
+#define B b1 b2
+
+ID(ID(ID(a1) a2 a3)) split ID(B)
+ )cpp");
+
+ EXPECT_THAT(
+ Buffer.expandedForSpelled(findSpelled("ID ( ID ( ID ( a1 ) a2 a3 ) )")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( B )")),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(
+ findSpelled("ID ( ID ( ID ( a1 ) a2 a3 ) ) split ID ( B )")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 split b1 b2";
+ // FIXME: these should succeed, but we do not support macro arguments yet.
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1")), IsEmpty());
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( a1 ) a2")),
+ IsEmpty());
+
+ // Empty macro expansions.
+ recordTokens(R"cpp(
+#define EMPTY
+#define ID(X) X
+
+EMPTY EMPTY ID(1 2 3) EMPTY EMPTY split1
+EMPTY EMPTY ID(4 5 6) split2
+ID(7 8 9) EMPTY EMPTY
+ )cpp");
+ // Covered by emp
[PATCH] D81107: Make syntax tree test print the line number when it fails
hlopko accepted this revision. hlopko added a comment. Thanks! Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D81107/new/ https://reviews.llvm.org/D81107 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D81150: Use libClangTesting in the unittest for AST matchers
hlopko accepted this revision. hlopko added a comment. This revision is now accepted and ready to land. Nice! :) Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D81150/new/ https://reviews.llvm.org/D81150 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D84623: Remove HAVE_VCS_VERSION_INC, not needed
hlopko added a comment. Herald added a subscriber: JDevlieghere. Friendly ping, this change will remove HAVE_VCS_VERSION_INC define. If you disagree, please complain :) Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D84623/new/ https://reviews.llvm.org/D84623 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D88446: docs: add documentation describing API Notes
hlopko accepted this revision. hlopko added a comment. This revision is now accepted and ready to land. Thanks for pushing this forward! Current proposal seems to be only dealing with annotating ObjC for Swift, but API notes seem like a feature that would be useful for other languages as well. I briefly chatted with gribozavr and we concluded that if the need arises, the current design should be extensible to e.g. C++ (one of the bigger problems being how to name C++ function/overload sets). Comment at: clang/docs/APINotes.rst:187-190 + - "N"onnull (``_Nonnull``) + - "O"ptional (``_Nullable``) + - "U"nspecified (``_Null_unspecified``) + - "S"calar (deprecated) compnerd wrote: > rsmith wrote: > > Is it important that these are single letters? Spelling out the name in > > full (as you do for other enumerated values like `MethodKind` and > > `PropertyKind`) would seem a little more readable. (We could accept the > > single-letter forms as aliases.) > I don't think that they be single letters is important. As long as we have > the compatibility aliases, I think it should be fine to support the fully > spelt out versions, as the compatibility is needed for existing APINotes. +1, this is the only attribute that is abbreviated. Comment at: clang/docs/APINotes.rst:216 +Due to a compiler bug, 'NullabilityOfRet' may change nullability of the +parameters as well (rdar://30544062). Avoid using it and instead use +'ResultType' and specify the return type along with a nullability Could we get more context from the Apple bug tracker? Depending on the details it might be reasonable to drop NullabilityOfRet attribute completely, or more precisely document when it should be used. Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D88446/new/ https://reviews.llvm.org/D88446 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76497: [Syntax] Test both the default and windows target platforms in unittests
hlopko updated this revision to Diff 251649.
hlopko added a comment.
.equals -> == :)
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76497/new/
https://reviews.llvm.org/D76497
Files:
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -47,7 +47,9 @@
class SyntaxTreeTest : public ::testing::Test {
protected:
// Build a syntax tree for the code.
- syntax::TranslationUnit *buildTree(llvm::StringRef Code) {
+ syntax::TranslationUnit *
+ buildTree(llvm::StringRef Code,
+const std::string &Target = "x86_64-pc-linux-gnu") {
// FIXME: this code is almost the identical to the one in TokensTest. Share
//it.
class BuildSyntaxTree : public ASTConsumer {
@@ -98,9 +100,10 @@
if (!Diags->getClient())
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
-std::vector Args = {"syntax-test", "-std=c++11",
- "-fno-delayed-template-parsing",
- "-fsyntax-only", FileName};
+std::vector Args = {
+"syntax-test", "-target", Target.c_str(),
+FileName, "-fsyntax-only", "-std=c++17",
+};
Invocation = createInvocationFromCommandLine(Args, Diags, FS);
assert(Invocation);
Invocation->getFrontendOpts().DisableFree = false;
@@ -121,14 +124,30 @@
return Root;
}
- void expectTreeDumpEqual(StringRef code, StringRef tree) {
-SCOPED_TRACE(code);
-
-auto *Root = buildTree(code);
-std::string Expected = tree.trim().str();
-std::string Actual =
-std::string(llvm::StringRef(Root->dump(*Arena)).trim());
-EXPECT_EQ(Expected, Actual) << "the resulting dump is:\n" << Actual;
+ void expectTreeDumpEqual(StringRef Code, StringRef Tree,
+ bool RunWithDelayedTemplateParsing = true) {
+SCOPED_TRACE(Code);
+
+std::string Expected = Tree.trim().str();
+
+// We want to run the test with -fdelayed-template-parsing enabled and
+// disabled, therefore we use these representative targets that differ in
+// the default value.
+// We are not passing -fdelayed-template-parsing directly but we are using
+// the `-target` to improve coverage and discover differences in behavior
+// early.
+for (const std::string Target :
+ {"x86_64-pc-linux-gnu", "x86_64-pc-win32-msvc"}) {
+ if (!RunWithDelayedTemplateParsing &&
+ Target == "x86_64-pc-win32-msvc") {
+continue;
+ }
+ auto *Root = buildTree(Code, Target);
+ std::string Actual = std::string(StringRef(Root->dump(*Arena)).trim());
+ EXPECT_EQ(Expected, Actual)
+ << "for target " << Target << " the resulting dump is:\n"
+ << Actual;
+}
}
// Adds a file to the test VFS.
@@ -794,7 +813,10 @@
`-CompoundStatement
|-{
`-}
-)txt");
+)txt",
+ // FIXME: Make this test work on windows by generating the expected
Syntax
+ // tree when -fdelayed-template-parsing is active.
+ /*RunWithDelayedTemplateParsing=*/false);
}
TEST_F(SyntaxTreeTest, NestedTemplates) {
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -47,7 +47,9 @@
class SyntaxTreeTest : public ::testing::Test {
protected:
// Build a syntax tree for the code.
- syntax::TranslationUnit *buildTree(llvm::StringRef Code) {
+ syntax::TranslationUnit *
+ buildTree(llvm::StringRef Code,
+const std::string &Target = "x86_64-pc-linux-gnu") {
// FIXME: this code is almost the identical to the one in TokensTest. Share
//it.
class BuildSyntaxTree : public ASTConsumer {
@@ -98,9 +100,10 @@
if (!Diags->getClient())
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
-std::vector Args = {"syntax-test", "-std=c++11",
- "-fno-delayed-template-parsing",
- "-fsyntax-only", FileName};
+std::vector Args = {
+"syntax-test", "-target", Target.c_str(),
+FileName, "-fsyntax-only", "-std=c++17",
+};
Invocation = createInvocationFromCommandLine(Args, Diags, FS);
assert(Invocation);
Invocation->getFrontendOpts().DisableFree = false;
@@ -121,14 +124,30 @@
return Root;
}
- void expectTreeDumpEqual(StringRef code, StringRef tree) {
-SCOPED_TRACE(code);
-
-auto *Root = buildTree(code);
-std::string Expected = tree.trim().str();
-std::string Actual =
-std::string(llvm::StringRef(Root->dump(*Arena)).trim());
-EXPECT_EQ(E
[PATCH] D76497: [Syntax] Test both the default and windows target platforms in unittests
hlopko updated this revision to Diff 251646.
hlopko added a comment.
Polish
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76497/new/
https://reviews.llvm.org/D76497
Files:
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -47,7 +47,9 @@
class SyntaxTreeTest : public ::testing::Test {
protected:
// Build a syntax tree for the code.
- syntax::TranslationUnit *buildTree(llvm::StringRef Code) {
+ syntax::TranslationUnit *
+ buildTree(llvm::StringRef Code,
+const std::string &Target = "x86_64-pc-linux-gnu") {
// FIXME: this code is almost the identical to the one in TokensTest. Share
//it.
class BuildSyntaxTree : public ASTConsumer {
@@ -98,9 +100,10 @@
if (!Diags->getClient())
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
-std::vector Args = {"syntax-test", "-std=c++11",
- "-fno-delayed-template-parsing",
- "-fsyntax-only", FileName};
+std::vector Args = {
+"syntax-test", "-target", Target.c_str(),
+FileName, "-fsyntax-only", "-std=c++17",
+};
Invocation = createInvocationFromCommandLine(Args, Diags, FS);
assert(Invocation);
Invocation->getFrontendOpts().DisableFree = false;
@@ -121,14 +124,30 @@
return Root;
}
- void expectTreeDumpEqual(StringRef code, StringRef tree) {
-SCOPED_TRACE(code);
-
-auto *Root = buildTree(code);
-std::string Expected = tree.trim().str();
-std::string Actual =
-std::string(llvm::StringRef(Root->dump(*Arena)).trim());
-EXPECT_EQ(Expected, Actual) << "the resulting dump is:\n" << Actual;
+ void expectTreeDumpEqual(StringRef Code, StringRef Tree,
+ bool RunWithDelayedTemplateParsing = true) {
+SCOPED_TRACE(Code);
+
+std::string Expected = Tree.trim().str();
+
+// We want to run the test with -fdelayed-template-parsing enabled and
+// disabled, therefore we use these representative targets that differ in
+// the default value.
+// We are not passing -fdelayed-template-parsing directly but we are using
+// the `-target` to improve coverage and discover differences in behavior
+// early.
+for (const std::string Target :
+ {"x86_64-pc-linux-gnu", "x86_64-pc-win32-msvc"}) {
+ if (!RunWithDelayedTemplateParsing &&
+ Target.equals("x86_64-pc-win32-msvc")) {
+continue;
+ }
+ auto *Root = buildTree(Code, Target);
+ std::string Actual = std::string(StringRef(Root->dump(*Arena)).trim());
+ EXPECT_EQ(Expected, Actual)
+ << "for target " << Target << " the resulting dump is:\n"
+ << Actual;
+}
}
// Adds a file to the test VFS.
@@ -794,7 +813,10 @@
`-CompoundStatement
|-{
`-}
-)txt");
+)txt",
+ // FIXME: Make this test work on windows by generating the expected
Syntax
+ // tree when -fdelayed-template-parsing is active.
+ /*RunWithDelayedTemplateParsing=*/false);
}
TEST_F(SyntaxTreeTest, NestedTemplates) {
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -47,7 +47,9 @@
class SyntaxTreeTest : public ::testing::Test {
protected:
// Build a syntax tree for the code.
- syntax::TranslationUnit *buildTree(llvm::StringRef Code) {
+ syntax::TranslationUnit *
+ buildTree(llvm::StringRef Code,
+const std::string &Target = "x86_64-pc-linux-gnu") {
// FIXME: this code is almost the identical to the one in TokensTest. Share
//it.
class BuildSyntaxTree : public ASTConsumer {
@@ -98,9 +100,10 @@
if (!Diags->getClient())
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
-std::vector Args = {"syntax-test", "-std=c++11",
- "-fno-delayed-template-parsing",
- "-fsyntax-only", FileName};
+std::vector Args = {
+"syntax-test", "-target", Target.c_str(),
+FileName, "-fsyntax-only", "-std=c++17",
+};
Invocation = createInvocationFromCommandLine(Args, Diags, FS);
assert(Invocation);
Invocation->getFrontendOpts().DisableFree = false;
@@ -121,14 +124,30 @@
return Root;
}
- void expectTreeDumpEqual(StringRef code, StringRef tree) {
-SCOPED_TRACE(code);
-
-auto *Root = buildTree(code);
-std::string Expected = tree.trim().str();
-std::string Actual =
-std::string(llvm::StringRef(Root->dump(*Arena)).trim());
-EXPECT_EQ(Expect
[PATCH] D76355: [Syntax] Build mapping from AST to syntax tree nodes
hlopko updated this revision to Diff 251968.
hlopko added a comment.
Reformat
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76355/new/
https://reviews.llvm.org/D76355
Files:
clang/include/clang/Tooling/Syntax/Tree.h
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Tree.cpp
Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -66,14 +66,20 @@
bool syntax::Tree::classof(const Node *N) { return N->kind() > NodeKind::Leaf; }
void syntax::Tree::prependChildLowLevel(Node *Child, NodeRole Role) {
- assert(Child->Parent == nullptr);
- assert(Child->NextSibling == nullptr);
assert(Child->role() == NodeRole::Detached);
assert(Role != NodeRole::Detached);
+ Child->Role = static_cast(Role);
+ prependChildLowLevel(Child);
+}
+
+void syntax::Tree::prependChildLowLevel(Node *Child) {
+ assert(Child->Parent == nullptr);
+ assert(Child->NextSibling == nullptr);
+ assert(Child->role() != NodeRole::Detached);
+
Child->Parent = this;
Child->NextSibling = this->FirstChild;
- Child->Role = static_cast(Role);
this->FirstChild = Child;
}
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -25,6 +25,8 @@
#include "clang/Tooling/Syntax/Tokens.h"
#include "clang/Tooling/Syntax/Tree.h"
#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/PointerUnion.h"
#include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/ScopeExit.h"
#include "llvm/ADT/SmallVector.h"
@@ -34,6 +36,7 @@
#include "llvm/Support/FormatVariadic.h"
#include "llvm/Support/MemoryBuffer.h"
#include "llvm/Support/raw_ostream.h"
+#include
#include
using namespace clang;
@@ -145,6 +148,31 @@
return SourceRange(Start, End);
}
+namespace {
+/// All AST hierarchy roots that can be represented as pointers.
+using ASTPtr = llvm::PointerUnion;
+/// Maintains a mapping from AST to syntax tree nodes. This class will get more
+/// complicated as we support more kinds of AST nodes, e.g. TypeLocs.
+/// FIXME: expose this as public API.
+class ASTToSyntaxMapping {
+public:
+ void add(ASTPtr From, syntax::Tree *To) {
+assert(To != nullptr);
+assert(!From.isNull());
+
+bool Added = Nodes.insert({From, To}).second;
+(void)Added;
+assert(Added && "mapping added twice");
+ }
+
+ syntax::Tree *find(ASTPtr P) const { return Nodes.lookup(P); }
+
+private:
+ // Keys are either Stmt* or Decl*.
+ llvm::DenseMap Nodes;
+};
+} // namespace
+
/// A helper class for constructing the syntax tree while traversing a clang
/// AST.
///
@@ -172,7 +200,18 @@
/// Populate children for \p New node, assuming it covers tokens from \p
/// Range.
- void foldNode(llvm::ArrayRef Range, syntax::Tree *New);
+ void foldNode(llvm::ArrayRef Range, syntax::Tree *New,
+ASTPtr From) {
+assert(New);
+Pending.foldChildren(Arena, Range, New);
+if (From)
+ Mapping.add(From, New);
+ }
+ void foldNode(llvm::ArrayRef Range, syntax::Tree *New,
+TypeLoc L) {
+// FIXME: add mapping for TypeLocs
+foldNode(Range, New, nullptr);
+ }
/// Must be called with the range of each `DeclaratorDecl`. Ensures the
/// corresponding declarator nodes are covered by `SimpleDeclaration`.
@@ -195,8 +234,10 @@
/// Set role for \p T.
void markChildToken(const syntax::Token *T, NodeRole R);
- /// Set role for the node that spans exactly \p Range.
- void markChild(llvm::ArrayRef Range, NodeRole R);
+ /// Set role for \p N.
+ void markChild(syntax::Node *N, NodeRole R);
+ /// Set role for the syntax node matching \p N.
+ void markChild(ASTPtr N, NodeRole R);
/// Set role for the delayed node that spans exactly \p Range.
void markDelayedChild(llvm::ArrayRef Range, NodeRole R);
/// Set role for the node that may or may not be delayed. Node must span
@@ -290,6 +331,11 @@
return Tokens;
}
+ void setRole(syntax::Node *N, NodeRole R) {
+assert(N->role() == NodeRole::Detached);
+N->Role = static_cast(R);
+ }
+
/// A collection of trees covering the input tokens.
/// When created, each tree corresponds to a single token in the file.
/// Clients call 'foldChildren' to attach one or more subtrees to a parent
@@ -306,7 +352,7 @@
auto *L = new (A.allocator()) syntax::Leaf(&T);
L->Original = true;
L->CanModify = A.tokenBuffer().spelledForExpanded(T).hasValue();
-Trees.insert(Trees.end(), {&T, NodeAndRole{L}});
+Trees.insert(Trees.end(), {&T, L});
}
}
@@ -338,7 +384,9 @@
assert((std::next(It) == Trees.end() ||
std::next(It)->first == Range.end()) &&
"no child with the specified range");
- I
[PATCH] D76355: [Syntax] Build mapping from AST to syntax tree nodes
hlopko added inline comments.
Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:570
+ auto *N = new (allocator()) syntax::SimpleDeclarator;
+ Builder.foldNode(Builder.getRange(R.getBegin(), R.getEnd()), N, D);
+ Builder.markChild(N, syntax::NodeRole::SimpleDeclaration_declarator);
gribozavr2 wrote:
> It might make sense to add a helper `Builder.getRange(SourceRange)` and
> simplify these calls to `Builder.getRange(something.getBegin(),
> something.getEnd())` throughout the patch.
Done and refactored all the calls that made sense to use the SourceRange
method. I didn't modify call like Builder.getRange(L.getLBracketLoc(),
L.getRBracketLoc()).
Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:975
+ const syntax::Token *TemplateKW,
+ syntax::SimpleDeclaration *InnerDeclaration) {
assert(!ExternKW || ExternKW->kind() == tok::kw_extern);
gribozavr2 wrote:
> Add a `Decl *From` parameter and pass it through to `Builder.foldNode()`
> below?
Done, but to get rid of all nullptr parents in BuildTree.cpp we'd have to
implement support for Types in the AST mapping. Let's not do that in this patch.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76355/new/
https://reviews.llvm.org/D76355
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76355: [Syntax] Build mapping from AST to syntax tree nodes
hlopko updated this revision to Diff 251993.
hlopko marked 6 inline comments as done.
hlopko added a comment.
Address comments
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76355/new/
https://reviews.llvm.org/D76355
Files:
clang/include/clang/Tooling/Syntax/Tree.h
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Mutations.cpp
clang/lib/Tooling/Syntax/Tree.cpp
Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -58,22 +58,33 @@
syntax::Node::Node(NodeKind Kind)
: Parent(nullptr), NextSibling(nullptr), Kind(static_cast(Kind)),
- Role(static_cast(NodeRole::Detached)), Original(false),
- CanModify(false) {}
+ Role(0), Original(false), CanModify(false) {
+ this->SetRole(NodeRole::Detached);
+}
bool syntax::Node::isDetached() const { return role() == NodeRole::Detached; }
+void syntax::Node::SetRole(NodeRole NR) {
+ this->Role = static_cast(NR);
+}
+
bool syntax::Tree::classof(const Node *N) { return N->kind() > NodeKind::Leaf; }
void syntax::Tree::prependChildLowLevel(Node *Child, NodeRole Role) {
- assert(Child->Parent == nullptr);
- assert(Child->NextSibling == nullptr);
assert(Child->role() == NodeRole::Detached);
assert(Role != NodeRole::Detached);
+ Child->SetRole(Role);
+ prependChildLowLevel(Child);
+}
+
+void syntax::Tree::prependChildLowLevel(Node *Child) {
+ assert(Child->Parent == nullptr);
+ assert(Child->NextSibling == nullptr);
+ assert(Child->role() != NodeRole::Detached);
+
Child->Parent = this;
Child->NextSibling = this->FirstChild;
- Child->Role = static_cast(Role);
this->FirstChild = Child;
}
@@ -94,7 +105,7 @@
N != End;) {
auto *Next = N->NextSibling;
-N->Role = static_cast(NodeRole::Detached);
+N->SetRole(NodeRole::Detached);
N->Parent = nullptr;
N->NextSibling = nullptr;
if (N->Original)
Index: clang/lib/Tooling/Syntax/Mutations.cpp
===
--- clang/lib/Tooling/Syntax/Mutations.cpp
+++ clang/lib/Tooling/Syntax/Mutations.cpp
@@ -35,7 +35,7 @@
assert(!New->isDetached());
assert(Role != NodeRole::Detached);
-New->Role = static_cast(Role);
+New->SetRole(Role);
auto *P = Anchor->parent();
P->replaceChildRangeLowLevel(Anchor, Anchor, New);
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -25,6 +25,8 @@
#include "clang/Tooling/Syntax/Tokens.h"
#include "clang/Tooling/Syntax/Tree.h"
#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/PointerUnion.h"
#include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/ScopeExit.h"
#include "llvm/ADT/SmallVector.h"
@@ -34,6 +36,7 @@
#include "llvm/Support/FormatVariadic.h"
#include "llvm/Support/MemoryBuffer.h"
#include "llvm/Support/raw_ostream.h"
+#include
#include
using namespace clang;
@@ -145,6 +148,30 @@
return SourceRange(Start, End);
}
+namespace {
+/// All AST hierarchy roots that can be represented as pointers.
+using ASTPtr = llvm::PointerUnion;
+/// Maintains a mapping from AST to syntax tree nodes. This class will get more
+/// complicated as we support more kinds of AST nodes, e.g. TypeLocs.
+/// FIXME: expose this as public API.
+class ASTToSyntaxMapping {
+public:
+ void add(ASTPtr From, syntax::Tree *To) {
+assert(To != nullptr);
+assert(!From.isNull());
+
+bool Added = Nodes.insert({From, To}).second;
+(void)Added;
+assert(Added && "mapping added twice");
+ }
+
+ syntax::Tree *find(ASTPtr P) const { return Nodes.lookup(P); }
+
+private:
+ llvm::DenseMap Nodes;
+};
+} // namespace
+
/// A helper class for constructing the syntax tree while traversing a clang
/// AST.
///
@@ -172,7 +199,18 @@
/// Populate children for \p New node, assuming it covers tokens from \p
/// Range.
- void foldNode(llvm::ArrayRef Range, syntax::Tree *New);
+ void foldNode(llvm::ArrayRef Range, syntax::Tree *New,
+ASTPtr From) {
+assert(New);
+Pending.foldChildren(Arena, Range, New);
+if (From)
+ Mapping.add(From, New);
+ }
+ void foldNode(llvm::ArrayRef Range, syntax::Tree *New,
+TypeLoc L) {
+// FIXME: add mapping for TypeLocs
+foldNode(Range, New, nullptr);
+ }
/// Must be called with the range of each `DeclaratorDecl`. Ensures the
/// corresponding declarator nodes are covered by `SimpleDeclaration`.
@@ -195,8 +233,10 @@
/// Set role for \p T.
void markChildToken(const syntax::Token *T, NodeRole R);
- /// Set role for the node that spans exactly \p Range.
- void markChild(llvm::ArrayRef Range, NodeRole R);
+ /// Set role for \p N.
+ void markChild(syntax::Nod
[PATCH] D76355: [Syntax] Build mapping from AST to syntax tree nodes
hlopko added a comment. PTAL. Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:173 +private: + // Keys are either Stmt* or Decl*. + llvm::DenseMap Nodes; gribozavr2 wrote: > The comment is not needed anymore. Clang tidy complains when I remove it :/ Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D76355/new/ https://reviews.llvm.org/D76355 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76355: [Syntax] Build mapping from AST to syntax tree nodes
hlopko updated this revision to Diff 252045.
hlopko marked 5 inline comments as done.
hlopko added a comment.
Address comments
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76355/new/
https://reviews.llvm.org/D76355
Files:
clang/include/clang/Tooling/Syntax/Tree.h
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Mutations.cpp
clang/lib/Tooling/Syntax/Tree.cpp
Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -58,22 +58,33 @@
syntax::Node::Node(NodeKind Kind)
: Parent(nullptr), NextSibling(nullptr), Kind(static_cast(Kind)),
- Role(static_cast(NodeRole::Detached)), Original(false),
- CanModify(false) {}
+ Role(0), Original(false), CanModify(false) {
+ this->setRole(NodeRole::Detached);
+}
bool syntax::Node::isDetached() const { return role() == NodeRole::Detached; }
+void syntax::Node::setRole(NodeRole NR) {
+ this->Role = static_cast(NR);
+}
+
bool syntax::Tree::classof(const Node *N) { return N->kind() > NodeKind::Leaf; }
void syntax::Tree::prependChildLowLevel(Node *Child, NodeRole Role) {
- assert(Child->Parent == nullptr);
- assert(Child->NextSibling == nullptr);
assert(Child->role() == NodeRole::Detached);
assert(Role != NodeRole::Detached);
+ Child->setRole(Role);
+ prependChildLowLevel(Child);
+}
+
+void syntax::Tree::prependChildLowLevel(Node *Child) {
+ assert(Child->Parent == nullptr);
+ assert(Child->NextSibling == nullptr);
+ assert(Child->role() != NodeRole::Detached);
+
Child->Parent = this;
Child->NextSibling = this->FirstChild;
- Child->Role = static_cast(Role);
this->FirstChild = Child;
}
@@ -94,7 +105,7 @@
N != End;) {
auto *Next = N->NextSibling;
-N->Role = static_cast(NodeRole::Detached);
+N->setRole(NodeRole::Detached);
N->Parent = nullptr;
N->NextSibling = nullptr;
if (N->Original)
Index: clang/lib/Tooling/Syntax/Mutations.cpp
===
--- clang/lib/Tooling/Syntax/Mutations.cpp
+++ clang/lib/Tooling/Syntax/Mutations.cpp
@@ -35,7 +35,7 @@
assert(!New->isDetached());
assert(Role != NodeRole::Detached);
-New->Role = static_cast(Role);
+New->setRole(Role);
auto *P = Anchor->parent();
P->replaceChildRangeLowLevel(Anchor, Anchor, New);
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -25,6 +25,8 @@
#include "clang/Tooling/Syntax/Tokens.h"
#include "clang/Tooling/Syntax/Tree.h"
#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/PointerUnion.h"
#include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/ScopeExit.h"
#include "llvm/ADT/SmallVector.h"
@@ -34,6 +36,7 @@
#include "llvm/Support/FormatVariadic.h"
#include "llvm/Support/MemoryBuffer.h"
#include "llvm/Support/raw_ostream.h"
+#include
#include
using namespace clang;
@@ -145,6 +148,30 @@
return SourceRange(Start, End);
}
+namespace {
+/// All AST hierarchy roots that can be represented as pointers.
+using ASTPtr = llvm::PointerUnion;
+/// Maintains a mapping from AST to syntax tree nodes. This class will get more
+/// complicated as we support more kinds of AST nodes, e.g. TypeLocs.
+/// FIXME: expose this as public API.
+class ASTToSyntaxMapping {
+public:
+ void add(ASTPtr From, syntax::Tree *To) {
+assert(To != nullptr);
+assert(!From.isNull());
+
+bool Added = Nodes.insert({From, To}).second;
+(void)Added;
+assert(Added && "mapping added twice");
+ }
+
+ syntax::Tree *find(ASTPtr P) const { return Nodes.lookup(P); }
+
+private:
+ llvm::DenseMap Nodes;
+};
+} // namespace
+
/// A helper class for constructing the syntax tree while traversing a clang
/// AST.
///
@@ -172,7 +199,18 @@
/// Populate children for \p New node, assuming it covers tokens from \p
/// Range.
- void foldNode(llvm::ArrayRef Range, syntax::Tree *New);
+ void foldNode(llvm::ArrayRef Range, syntax::Tree *New,
+ASTPtr From) {
+assert(New);
+Pending.foldChildren(Arena, Range, New);
+if (From)
+ Mapping.add(From, New);
+ }
+ void foldNode(llvm::ArrayRef Range, syntax::Tree *New,
+TypeLoc L) {
+// FIXME: add mapping for TypeLocs
+foldNode(Range, New, nullptr);
+ }
/// Must be called with the range of each `DeclaratorDecl`. Ensures the
/// corresponding declarator nodes are covered by `SimpleDeclaration`.
@@ -195,8 +233,10 @@
/// Set role for \p T.
void markChildToken(const syntax::Token *T, NodeRole R);
- /// Set role for the node that spans exactly \p Range.
- void markChild(llvm::ArrayRef Range, NodeRole R);
+ /// Set role for \p N.
+ void markChild(syntax::Nod
[PATCH] D76922: [Syntax] Remove delayed folding from tree building.
hlopko created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
hlopko updated this revision to Diff 253108.
hlopko added a comment.
hlopko edited the summary of this revision.
Reformat
This patch removes delayed folding and replaces it with forward peeking.
Delayed folding was previously used as a solution to the problem that
declaration doesn't have a representation in the AST. For example following
code:
int a,b;
is expressed in the AST as:
TranslationUnitDecl
|-...
|-VarDecl `int a`
`-VarDecl `int b`
And in the syntax tree we need:
*: TranslationUnit
`-SimpleDeclaration
|-int
|-SimpleDeclarator
| `-a
|-,
|-SimpleDeclarator
| `-b
|-;
So in words, we need to create SimpleDeclaration to be a parent of
SimpleDeclarator nodes. Previously we used delayed folding to make sure
SimpleDeclarations will be
eventually created. And in case multiple declarators requested declaration
creation, declaration range was extended to cover all declarators.
This design started to be hard to reason about, so we decided to replace it with
forward peeking. The last declarator node in the chain is responsible for
creating
SimpleDeclaration for the whole chain. Range of the declaration corresponds to
the source range of the declarator node. Declarator decides whether its the last
one by peeking to the next AST node (see `isResponsibleForCreatingDeclaration`).
This patch does following:
- Removed delayed folding logic
- Tweaks Token.dumpForTests
- Moves getQualifiedNameStart inside BuildTreeVisitor
- Extracts BuildTreeVisitor.ProcessDeclaratorAndDeclaration
- Renames Builder.getDeclRange to Builder.getDeclarationRange and uses the
method in all places.
- Adds a bunch of tests
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D76922
Files:
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Tokens.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -224,6 +224,59 @@
)txt");
}
+TEST_F(SyntaxTreeTest, SimpleVariable) {
+ expectTreeDumpEqual(
+ R"cpp(
+int a;
+int b = 42;
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | `-a
+| `-;
+`-SimpleDeclaration
+ |-int
+ |-SimpleDeclarator
+ | |-b
+ | |-=
+ | `-UnknownExpression
+ | `-42
+ `-;
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, SimpleFunction) {
+ expectTreeDumpEqual(
+ R"cpp(
+void foo(int a, int b) {}
+)cpp",
+ R"txt(
+*: TranslationUnit
+`-SimpleDeclaration
+ |-void
+ |-SimpleDeclarator
+ | |-foo
+ | `-ParametersAndQualifiers
+ | |-(
+ | |-SimpleDeclaration
+ | | |-int
+ | | `-SimpleDeclarator
+ | | `-a
+ | |-,
+ | |-SimpleDeclaration
+ | | |-int
+ | | `-SimpleDeclarator
+ | | `-b
+ | `-)
+ `-CompoundStatement
+|-{
+`-}
+)txt");
+}
+
TEST_F(SyntaxTreeTest, If) {
expectTreeDumpEqual(
R"cpp(
@@ -541,20 +594,32 @@
TEST_F(SyntaxTreeTest, MultipleDeclaratorsGrouping) {
expectTreeDumpEqual(
R"cpp(
- int *a, b;
+ int *a, b; int *c, d;
)cpp",
R"txt(
*: TranslationUnit
+|-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | |-*
+| | `-a
+| |-,
+| |-SimpleDeclarator
+| | `-b
+| `-;
`-SimpleDeclaration
|-int
|-SimpleDeclarator
| |-*
- | `-a
+ | `-c
|-,
|-SimpleDeclarator
- | `-b
+ | `-d
`-;
)txt");
+}
+
+TEST_F(SyntaxTreeTest, MultipleDeclaratorsGroupingTypedef) {
expectTreeDumpEqual(
R"cpp(
typedef int *a, b;
@@ -1800,7 +1865,7 @@
}
TEST_F(SyntaxTreeTest, SynthesizedNodes) {
- buildTree("");
+ buildTree(";");
auto *C = syntax::createPunctuation(*Arena, tok::comma);
ASSERT_NE(C, nullptr);
Index: clang/lib/Tooling/Syntax/Tokens.cpp
===
--- clang/lib/Tooling/Syntax/Tokens.cpp
+++ clang/lib/Tooling/Syntax/Tokens.cpp
@@ -645,8 +645,8 @@
}
std::string syntax::Token::dumpForTests(const SourceManager &SM) const {
- return std::string(
- llvm::formatv("{0} {1}", tok::getTokenName(kind()), text(SM)));
+ return std::string(llvm::formatv("Token(`{0}`, {1}, length = {2})", text(SM),
+ tok::getTokenName(kind()), length()));
}
std::string TokenBuffer::dumpForTests() const {
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -44,15 +44,6 @@
LLVM_ATTRIBUTE_UNUSED
static bool isImplicitExpr(clang::Expr *E) { return E->IgnoreImplicit() != E; }
-static SourceLocation getQualifiedNameStart(DeclaratorDecl *D) {
- auto DN = D->getDeclName();
- bool IsAnonymous = DN.isIdentifier() &&
[PATCH] D76922: [Syntax] Remove delayed folding from tree building.
hlopko updated this revision to Diff 253108.
hlopko added a comment.
Reformat
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76922/new/
https://reviews.llvm.org/D76922
Files:
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Tokens.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -224,6 +224,59 @@
)txt");
}
+TEST_F(SyntaxTreeTest, SimpleVariable) {
+ expectTreeDumpEqual(
+ R"cpp(
+int a;
+int b = 42;
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | `-a
+| `-;
+`-SimpleDeclaration
+ |-int
+ |-SimpleDeclarator
+ | |-b
+ | |-=
+ | `-UnknownExpression
+ | `-42
+ `-;
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, SimpleFunction) {
+ expectTreeDumpEqual(
+ R"cpp(
+void foo(int a, int b) {}
+)cpp",
+ R"txt(
+*: TranslationUnit
+`-SimpleDeclaration
+ |-void
+ |-SimpleDeclarator
+ | |-foo
+ | `-ParametersAndQualifiers
+ | |-(
+ | |-SimpleDeclaration
+ | | |-int
+ | | `-SimpleDeclarator
+ | | `-a
+ | |-,
+ | |-SimpleDeclaration
+ | | |-int
+ | | `-SimpleDeclarator
+ | | `-b
+ | `-)
+ `-CompoundStatement
+|-{
+`-}
+)txt");
+}
+
TEST_F(SyntaxTreeTest, If) {
expectTreeDumpEqual(
R"cpp(
@@ -541,20 +594,32 @@
TEST_F(SyntaxTreeTest, MultipleDeclaratorsGrouping) {
expectTreeDumpEqual(
R"cpp(
- int *a, b;
+ int *a, b; int *c, d;
)cpp",
R"txt(
*: TranslationUnit
+|-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | |-*
+| | `-a
+| |-,
+| |-SimpleDeclarator
+| | `-b
+| `-;
`-SimpleDeclaration
|-int
|-SimpleDeclarator
| |-*
- | `-a
+ | `-c
|-,
|-SimpleDeclarator
- | `-b
+ | `-d
`-;
)txt");
+}
+
+TEST_F(SyntaxTreeTest, MultipleDeclaratorsGroupingTypedef) {
expectTreeDumpEqual(
R"cpp(
typedef int *a, b;
@@ -1800,7 +1865,7 @@
}
TEST_F(SyntaxTreeTest, SynthesizedNodes) {
- buildTree("");
+ buildTree(";");
auto *C = syntax::createPunctuation(*Arena, tok::comma);
ASSERT_NE(C, nullptr);
Index: clang/lib/Tooling/Syntax/Tokens.cpp
===
--- clang/lib/Tooling/Syntax/Tokens.cpp
+++ clang/lib/Tooling/Syntax/Tokens.cpp
@@ -645,8 +645,8 @@
}
std::string syntax::Token::dumpForTests(const SourceManager &SM) const {
- return std::string(
- llvm::formatv("{0} {1}", tok::getTokenName(kind()), text(SM)));
+ return std::string(llvm::formatv("Token(`{0}`, {1}, length = {2})", text(SM),
+ tok::getTokenName(kind()), length()));
}
std::string TokenBuffer::dumpForTests() const {
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -44,15 +44,6 @@
LLVM_ATTRIBUTE_UNUSED
static bool isImplicitExpr(clang::Expr *E) { return E->IgnoreImplicit() != E; }
-static SourceLocation getQualifiedNameStart(DeclaratorDecl *D) {
- auto DN = D->getDeclName();
- bool IsAnonymous = DN.isIdentifier() && !DN.getAsIdentifierInfo();
- if (IsAnonymous)
-return SourceLocation();
- return D->getQualifierLoc() ? D->getQualifierLoc().getBeginLoc()
- : D->getLocation();
-}
-
namespace {
/// Get start location of the Declarator from the TypeLoc.
/// E.g.:
@@ -212,10 +203,6 @@
foldNode(Range, New, nullptr);
}
- /// Must be called with the range of each `DeclaratorDecl`. Ensures the
- /// corresponding declarator nodes are covered by `SimpleDeclaration`.
- void noticeDeclRange(llvm::ArrayRef Range);
-
/// Notifies that we should not consume trailing semicolon when computing
/// token range of \p D.
void noticeDeclWithoutSemicolon(Decl *D);
@@ -237,11 +224,6 @@
void markChild(syntax::Node *N, NodeRole R);
/// Set role for the syntax node matching \p N.
void markChild(ASTPtr N, NodeRole R);
- /// Set role for the delayed node that spans exactly \p Range.
- void markDelayedChild(llvm::ArrayRef Range, NodeRole R);
- /// Set role for the node that may or may not be delayed. Node must span
- /// exactly \p Range.
- void markMaybeDelayedChild(llvm::ArrayRef Range, NodeRole R);
/// Finish building the tree and consume the root node.
syntax::TranslationUnit *finalize() && {
@@ -285,18 +267,54 @@
return maybeAppendSemicolon(Tokens, D);
}
- llvm::ArrayRef getDeclRange(const Decl *D) const {
+ template
+ bool isResponsibleForCreatingDeclaration(const T *D) const {
+static_assert((std::is_base_of::value ||
+ std::is_base_of::value),
+ "only DeclaratorDecl and TypedefNameDecl are supported.");
[PATCH] D76922: [Syntax] Remove delayed folding from tree building.
hlopko updated this revision to Diff 253123.
hlopko added a comment.
Rebase
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76922/new/
https://reviews.llvm.org/D76922
Files:
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Tokens.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -224,6 +224,59 @@
)txt");
}
+TEST_F(SyntaxTreeTest, SimpleVariable) {
+ expectTreeDumpEqual(
+ R"cpp(
+int a;
+int b = 42;
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | `-a
+| `-;
+`-SimpleDeclaration
+ |-int
+ |-SimpleDeclarator
+ | |-b
+ | |-=
+ | `-UnknownExpression
+ | `-42
+ `-;
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, SimpleFunction) {
+ expectTreeDumpEqual(
+ R"cpp(
+void foo(int a, int b) {}
+)cpp",
+ R"txt(
+*: TranslationUnit
+`-SimpleDeclaration
+ |-void
+ |-SimpleDeclarator
+ | |-foo
+ | `-ParametersAndQualifiers
+ | |-(
+ | |-SimpleDeclaration
+ | | |-int
+ | | `-SimpleDeclarator
+ | | `-a
+ | |-,
+ | |-SimpleDeclaration
+ | | |-int
+ | | `-SimpleDeclarator
+ | | `-b
+ | `-)
+ `-CompoundStatement
+|-{
+`-}
+)txt");
+}
+
TEST_F(SyntaxTreeTest, If) {
expectTreeDumpEqual(
R"cpp(
@@ -541,20 +594,32 @@
TEST_F(SyntaxTreeTest, MultipleDeclaratorsGrouping) {
expectTreeDumpEqual(
R"cpp(
- int *a, b;
+ int *a, b; int *c, d;
)cpp",
R"txt(
*: TranslationUnit
+|-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | |-*
+| | `-a
+| |-,
+| |-SimpleDeclarator
+| | `-b
+| `-;
`-SimpleDeclaration
|-int
|-SimpleDeclarator
| |-*
- | `-a
+ | `-c
|-,
|-SimpleDeclarator
- | `-b
+ | `-d
`-;
)txt");
+}
+
+TEST_F(SyntaxTreeTest, MultipleDeclaratorsGroupingTypedef) {
expectTreeDumpEqual(
R"cpp(
typedef int *a, b;
Index: clang/lib/Tooling/Syntax/Tokens.cpp
===
--- clang/lib/Tooling/Syntax/Tokens.cpp
+++ clang/lib/Tooling/Syntax/Tokens.cpp
@@ -645,8 +645,8 @@
}
std::string syntax::Token::dumpForTests(const SourceManager &SM) const {
- return std::string(
- llvm::formatv("{0} {1}", tok::getTokenName(kind()), text(SM)));
+ return std::string(llvm::formatv("Token(`{0}`, {1}, length = {2})", text(SM),
+ tok::getTokenName(kind()), length()));
}
std::string TokenBuffer::dumpForTests() const {
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -44,15 +44,6 @@
LLVM_ATTRIBUTE_UNUSED
static bool isImplicitExpr(clang::Expr *E) { return E->IgnoreImplicit() != E; }
-static SourceLocation getQualifiedNameStart(DeclaratorDecl *D) {
- auto DN = D->getDeclName();
- bool IsAnonymous = DN.isIdentifier() && !DN.getAsIdentifierInfo();
- if (IsAnonymous)
-return SourceLocation();
- return D->getQualifierLoc() ? D->getQualifierLoc().getBeginLoc()
- : D->getLocation();
-}
-
namespace {
/// Get start location of the Declarator from the TypeLoc.
/// E.g.:
@@ -212,10 +203,6 @@
foldNode(Range, New, nullptr);
}
- /// Must be called with the range of each `DeclaratorDecl`. Ensures the
- /// corresponding declarator nodes are covered by `SimpleDeclaration`.
- void noticeDeclRange(llvm::ArrayRef Range);
-
/// Notifies that we should not consume trailing semicolon when computing
/// token range of \p D.
void noticeDeclWithoutSemicolon(Decl *D);
@@ -237,11 +224,6 @@
void markChild(syntax::Node *N, NodeRole R);
/// Set role for the syntax node matching \p N.
void markChild(ASTPtr N, NodeRole R);
- /// Set role for the delayed node that spans exactly \p Range.
- void markDelayedChild(llvm::ArrayRef Range, NodeRole R);
- /// Set role for the node that may or may not be delayed. Node must span
- /// exactly \p Range.
- void markMaybeDelayedChild(llvm::ArrayRef Range, NodeRole R);
/// Finish building the tree and consume the root node.
syntax::TranslationUnit *finalize() && {
@@ -285,7 +267,43 @@
return maybeAppendSemicolon(Tokens, D);
}
- llvm::ArrayRef getDeclRange(const Decl *D) const {
+ /// Returns true if \p D is the last declarator in a chain and is thus
+ /// reponsible for creating SimpleDeclaration for the whole chain.
+ template
+ bool isResponsibleForCreatingDeclaration(const T *D) const {
+static_assert((std::is_base_of::value ||
+ std::is_base_of::value),
+ "only DeclaratorDecl and TypedefNameDecl are supported.");
+
+const auto *TD = llvm::dyn_cast(D);
+if (TD
[PATCH] D76922: [Syntax] Remove delayed folding from tree building.
hlopko updated this revision to Diff 253122.
hlopko added a comment.
Polish
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76922/new/
https://reviews.llvm.org/D76922
Files:
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Tokens.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -224,6 +224,59 @@
)txt");
}
+TEST_F(SyntaxTreeTest, SimpleVariable) {
+ expectTreeDumpEqual(
+ R"cpp(
+int a;
+int b = 42;
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | `-a
+| `-;
+`-SimpleDeclaration
+ |-int
+ |-SimpleDeclarator
+ | |-b
+ | |-=
+ | `-UnknownExpression
+ | `-42
+ `-;
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, SimpleFunction) {
+ expectTreeDumpEqual(
+ R"cpp(
+void foo(int a, int b) {}
+)cpp",
+ R"txt(
+*: TranslationUnit
+`-SimpleDeclaration
+ |-void
+ |-SimpleDeclarator
+ | |-foo
+ | `-ParametersAndQualifiers
+ | |-(
+ | |-SimpleDeclaration
+ | | |-int
+ | | `-SimpleDeclarator
+ | | `-a
+ | |-,
+ | |-SimpleDeclaration
+ | | |-int
+ | | `-SimpleDeclarator
+ | | `-b
+ | `-)
+ `-CompoundStatement
+|-{
+`-}
+)txt");
+}
+
TEST_F(SyntaxTreeTest, If) {
expectTreeDumpEqual(
R"cpp(
@@ -541,20 +594,32 @@
TEST_F(SyntaxTreeTest, MultipleDeclaratorsGrouping) {
expectTreeDumpEqual(
R"cpp(
- int *a, b;
+ int *a, b; int *c, d;
)cpp",
R"txt(
*: TranslationUnit
+|-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | |-*
+| | `-a
+| |-,
+| |-SimpleDeclarator
+| | `-b
+| `-;
`-SimpleDeclaration
|-int
|-SimpleDeclarator
| |-*
- | `-a
+ | `-c
|-,
|-SimpleDeclarator
- | `-b
+ | `-d
`-;
)txt");
+}
+
+TEST_F(SyntaxTreeTest, MultipleDeclaratorsGroupingTypedef) {
expectTreeDumpEqual(
R"cpp(
typedef int *a, b;
Index: clang/lib/Tooling/Syntax/Tokens.cpp
===
--- clang/lib/Tooling/Syntax/Tokens.cpp
+++ clang/lib/Tooling/Syntax/Tokens.cpp
@@ -645,8 +645,8 @@
}
std::string syntax::Token::dumpForTests(const SourceManager &SM) const {
- return std::string(
- llvm::formatv("{0} {1}", tok::getTokenName(kind()), text(SM)));
+ return std::string(llvm::formatv("Token(`{0}`, {1}, length = {2})", text(SM),
+ tok::getTokenName(kind()), length()));
}
std::string TokenBuffer::dumpForTests() const {
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -44,15 +44,6 @@
LLVM_ATTRIBUTE_UNUSED
static bool isImplicitExpr(clang::Expr *E) { return E->IgnoreImplicit() != E; }
-static SourceLocation getQualifiedNameStart(DeclaratorDecl *D) {
- auto DN = D->getDeclName();
- bool IsAnonymous = DN.isIdentifier() && !DN.getAsIdentifierInfo();
- if (IsAnonymous)
-return SourceLocation();
- return D->getQualifierLoc() ? D->getQualifierLoc().getBeginLoc()
- : D->getLocation();
-}
-
namespace {
/// Get start location of the Declarator from the TypeLoc.
/// E.g.:
@@ -212,10 +203,6 @@
foldNode(Range, New, nullptr);
}
- /// Must be called with the range of each `DeclaratorDecl`. Ensures the
- /// corresponding declarator nodes are covered by `SimpleDeclaration`.
- void noticeDeclRange(llvm::ArrayRef Range);
-
/// Notifies that we should not consume trailing semicolon when computing
/// token range of \p D.
void noticeDeclWithoutSemicolon(Decl *D);
@@ -237,11 +224,6 @@
void markChild(syntax::Node *N, NodeRole R);
/// Set role for the syntax node matching \p N.
void markChild(ASTPtr N, NodeRole R);
- /// Set role for the delayed node that spans exactly \p Range.
- void markDelayedChild(llvm::ArrayRef Range, NodeRole R);
- /// Set role for the node that may or may not be delayed. Node must span
- /// exactly \p Range.
- void markMaybeDelayedChild(llvm::ArrayRef Range, NodeRole R);
/// Finish building the tree and consume the root node.
syntax::TranslationUnit *finalize() && {
@@ -285,7 +267,43 @@
return maybeAppendSemicolon(Tokens, D);
}
- llvm::ArrayRef getDeclRange(const Decl *D) const {
+ /// Returns true if \p D is the last declarator in a chain and is thus
+ /// reponsible for creating SimpleDeclaration for the whole chain.
+ template
+ bool isResponsibleForCreatingDeclaration(const T *D) const {
+static_assert((std::is_base_of::value ||
+ std::is_base_of::value),
+ "only DeclaratorDecl and TypedefNameDecl are supported.");
+
+const auto *TD = llvm::dyn_cast(D);
+if (TD
[PATCH] D76922: [Syntax] Remove delayed folding from tree building.
hlopko updated this revision to Diff 253517.
hlopko added a comment.
Rebase
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76922/new/
https://reviews.llvm.org/D76922
Files:
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Tokens.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -224,6 +224,59 @@
)txt");
}
+TEST_F(SyntaxTreeTest, SimpleVariable) {
+ expectTreeDumpEqual(
+ R"cpp(
+int a;
+int b = 42;
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | `-a
+| `-;
+`-SimpleDeclaration
+ |-int
+ |-SimpleDeclarator
+ | |-b
+ | |-=
+ | `-UnknownExpression
+ | `-42
+ `-;
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, SimpleFunction) {
+ expectTreeDumpEqual(
+ R"cpp(
+void foo(int a, int b) {}
+)cpp",
+ R"txt(
+*: TranslationUnit
+`-SimpleDeclaration
+ |-void
+ |-SimpleDeclarator
+ | |-foo
+ | `-ParametersAndQualifiers
+ | |-(
+ | |-SimpleDeclaration
+ | | |-int
+ | | `-SimpleDeclarator
+ | | `-a
+ | |-,
+ | |-SimpleDeclaration
+ | | |-int
+ | | `-SimpleDeclarator
+ | | `-b
+ | `-)
+ `-CompoundStatement
+|-{
+`-}
+)txt");
+}
+
TEST_F(SyntaxTreeTest, If) {
expectTreeDumpEqual(
R"cpp(
@@ -541,20 +594,32 @@
TEST_F(SyntaxTreeTest, MultipleDeclaratorsGrouping) {
expectTreeDumpEqual(
R"cpp(
- int *a, b;
+ int *a, b; int *c, d;
)cpp",
R"txt(
*: TranslationUnit
+|-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | |-*
+| | `-a
+| |-,
+| |-SimpleDeclarator
+| | `-b
+| `-;
`-SimpleDeclaration
|-int
|-SimpleDeclarator
| |-*
- | `-a
+ | `-c
|-,
|-SimpleDeclarator
- | `-b
+ | `-d
`-;
)txt");
+}
+
+TEST_F(SyntaxTreeTest, MultipleDeclaratorsGroupingTypedef) {
expectTreeDumpEqual(
R"cpp(
typedef int *a, b;
Index: clang/lib/Tooling/Syntax/Tokens.cpp
===
--- clang/lib/Tooling/Syntax/Tokens.cpp
+++ clang/lib/Tooling/Syntax/Tokens.cpp
@@ -710,8 +710,8 @@
}
std::string syntax::Token::dumpForTests(const SourceManager &SM) const {
- return std::string(
- llvm::formatv("{0} {1}", tok::getTokenName(kind()), text(SM)));
+ return std::string(llvm::formatv("Token(`{0}`, {1}, length = {2})", text(SM),
+ tok::getTokenName(kind()), length()));
}
std::string TokenBuffer::dumpForTests() const {
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -44,15 +44,6 @@
LLVM_ATTRIBUTE_UNUSED
static bool isImplicitExpr(clang::Expr *E) { return E->IgnoreImplicit() != E; }
-static SourceLocation getQualifiedNameStart(DeclaratorDecl *D) {
- auto DN = D->getDeclName();
- bool IsAnonymous = DN.isIdentifier() && !DN.getAsIdentifierInfo();
- if (IsAnonymous)
-return SourceLocation();
- return D->getQualifierLoc() ? D->getQualifierLoc().getBeginLoc()
- : D->getLocation();
-}
-
namespace {
/// Get start location of the Declarator from the TypeLoc.
/// E.g.:
@@ -212,10 +203,6 @@
foldNode(Range, New, nullptr);
}
- /// Must be called with the range of each `DeclaratorDecl`. Ensures the
- /// corresponding declarator nodes are covered by `SimpleDeclaration`.
- void noticeDeclRange(llvm::ArrayRef Range);
-
/// Notifies that we should not consume trailing semicolon when computing
/// token range of \p D.
void noticeDeclWithoutSemicolon(Decl *D);
@@ -237,11 +224,6 @@
void markChild(syntax::Node *N, NodeRole R);
/// Set role for the syntax node matching \p N.
void markChild(ASTPtr N, NodeRole R);
- /// Set role for the delayed node that spans exactly \p Range.
- void markDelayedChild(llvm::ArrayRef Range, NodeRole R);
- /// Set role for the node that may or may not be delayed. Node must span
- /// exactly \p Range.
- void markMaybeDelayedChild(llvm::ArrayRef Range, NodeRole R);
/// Finish building the tree and consume the root node.
syntax::TranslationUnit *finalize() && {
@@ -285,7 +267,43 @@
return maybeAppendSemicolon(Tokens, D);
}
- llvm::ArrayRef getDeclRange(const Decl *D) const {
+ /// Returns true if \p D is the last declarator in a chain and is thus
+ /// reponsible for creating SimpleDeclaration for the whole chain.
+ template
+ bool isResponsibleForCreatingDeclaration(const T *D) const {
+static_assert((std::is_base_of::value ||
+ std::is_base_of::value),
+ "only DeclaratorDecl and TypedefNameDecl are supported.");
+
+const auto *TD = llvm::dyn_cast(D);
+if (TD
[PATCH] D77053: [Syntax] A tool to dump syntax tree and token buffer
hlopko created this revision.
Herald added subscribers: cfe-commits, mgorny.
Herald added a project: clang.
hlopko updated this revision to Diff 253561.
hlopko added a comment.
hlopko updated this revision to Diff 253562.
hlopko added a reviewer: gribozavr2.
hlopko edited the summary of this revision.
Cleanup
hlopko added a comment.
Cleanup
Taking over of https://reviews.llvm.org/D70788.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D77053
Files:
clang/test/CMakeLists.txt
clang/test/clang-syntax/no_args.cpp
clang/test/clang-syntax/syntax_hello_world.cpp
clang/test/clang-syntax/syntax_no_file_arg.cpp
clang/test/clang-syntax/tokens_hello_world.cpp
clang/test/clang-syntax/tokens_no_file_arg.cpp
clang/tools/CMakeLists.txt
clang/tools/clang-syntax/CMakeLists.txt
clang/tools/clang-syntax/SyntaxMain.cpp
Index: clang/tools/clang-syntax/SyntaxMain.cpp
===
--- /dev/null
+++ clang/tools/clang-syntax/SyntaxMain.cpp
@@ -0,0 +1,88 @@
+//===- SyntaxMain.cpp -*- C++ -*-=//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+#include "clang/Frontend/CompilerInstance.h"
+#include "clang/Tooling/Execution.h"
+#include "clang/Tooling/Syntax/BuildTree.h"
+#include "clang/Tooling/Syntax/Tokens.h"
+#include "clang/Tooling/Syntax/Tree.h"
+#include "clang/Tooling/Tooling.h"
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/Support/Signals.h"
+#include "llvm/Support/raw_ostream.h"
+#include
+
+using namespace clang;
+
+namespace {
+
+llvm::cl::OptionCategory ClangSyntaxOptions("clang-syntax common options");
+
+llvm::cl::opt DumpTokens("dump-tokens",
+ llvm::cl::desc("dump the preprocessed tokens"),
+ llvm::cl::init(false),
+ llvm::cl::cat(ClangSyntaxOptions));
+llvm::cl::opt DumpSyntax("dump-syntax",
+ llvm::cl::desc("dump the syntax tree"),
+ llvm::cl::init(false),
+ llvm::cl::cat(ClangSyntaxOptions));
+
+class BuildSyntaxTree : public ASTFrontendAction {
+public:
+ std::unique_ptr CreateASTConsumer(CompilerInstance &CI,
+ StringRef InFile) override {
+class Consumer : public ASTConsumer {
+public:
+ Consumer(CompilerInstance &CI) : Collector(CI.getPreprocessor()) {}
+
+ void HandleTranslationUnit(ASTContext &AST) override {
+syntax::Arena A(AST.getSourceManager(), AST.getLangOpts(),
+std::move(Collector).consume());
+auto *TU = syntax::buildSyntaxTree(A, *AST.getTranslationUnitDecl());
+if (DumpTokens)
+ llvm::outs() << A.tokenBuffer().dumpForTests();
+if (DumpSyntax)
+ llvm::outs() << TU->dump(A);
+ }
+
+private:
+ syntax::TokenCollector Collector;
+};
+return std::make_unique(CI);
+ }
+};
+
+class Factory : public tooling::FrontendActionFactory {
+ std::unique_ptr create() override {
+return std::make_unique();
+ }
+};
+
+} // namespace
+
+int main(int argc, const char **argv) {
+ llvm::sys::PrintStackTraceOnErrorSignal(argv[0]);
+
+ auto Executor = clang::tooling::createExecutorFromCommandLineArgs(
+ argc, argv, ClangSyntaxOptions,
+ "Build syntax trees for the specified files");
+ if (!Executor) {
+llvm::errs() << llvm::toString(Executor.takeError()) << "\n";
+return 1;
+ }
+
+ if (!DumpTokens && !DumpSyntax) {
+llvm::errs()
+<< "Please specify at least one of -dump-tree or -dump-tokens\n";
+return 1;
+ }
+ // Collect symbols found in each translation unit, merging as we go.
+ auto Err = Executor->get()->execute(std::make_unique());
+ if (Err)
+llvm::errs() << llvm::toString(std::move(Err)) << "\n";
+ return 0;
+}
Index: clang/tools/clang-syntax/CMakeLists.txt
===
--- /dev/null
+++ clang/tools/clang-syntax/CMakeLists.txt
@@ -0,0 +1,16 @@
+set(LLVM_LINK_COMPONENTS Support)
+
+add_clang_tool(clang-syntax
+ SyntaxMain.cpp
+ )
+
+target_link_libraries(clang-syntax
+ PRIVATE
+ clangAST
+ clangBasic
+ clangFrontend
+ clangLex
+ clangTooling
+ clangToolingCore
+ clangToolingSyntax
+)
Index: clang/tools/CMakeLists.txt
===
--- clang/tools/CMakeLists.txt
+++ clang/tools/CMakeLists.txt
@@ -18,6 +18,7 @@
if(UNIX)
add_clang_subdirectory(clang-shlib)
endif()
+add_clang_subdirectory(clang-syntax)
if(CLANG_ENABLE_ARCMT)
add_clang_subdirectory(arcmt-test)
Index: clang/test/clang-syntax/tokens_no_file_arg.cpp
===
[PATCH] D77053: [Syntax] A tool to dump syntax tree and token buffer
hlopko updated this revision to Diff 253562.
hlopko added a comment.
Cleanup
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D77053/new/
https://reviews.llvm.org/D77053
Files:
clang/test/CMakeLists.txt
clang/test/clang-syntax/no_args.cpp
clang/test/clang-syntax/syntax_hello_world.cpp
clang/test/clang-syntax/syntax_no_file_arg.cpp
clang/test/clang-syntax/tokens_hello_world.cpp
clang/test/clang-syntax/tokens_no_file_arg.cpp
clang/tools/CMakeLists.txt
clang/tools/clang-syntax/CMakeLists.txt
clang/tools/clang-syntax/SyntaxMain.cpp
Index: clang/tools/clang-syntax/SyntaxMain.cpp
===
--- /dev/null
+++ clang/tools/clang-syntax/SyntaxMain.cpp
@@ -0,0 +1,88 @@
+//===- SyntaxMain.cpp -*- C++ -*-=//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+#include "clang/Frontend/CompilerInstance.h"
+#include "clang/Tooling/Execution.h"
+#include "clang/Tooling/Syntax/BuildTree.h"
+#include "clang/Tooling/Syntax/Tokens.h"
+#include "clang/Tooling/Syntax/Tree.h"
+#include "clang/Tooling/Tooling.h"
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/Support/Signals.h"
+#include "llvm/Support/raw_ostream.h"
+#include
+
+using namespace clang;
+
+namespace {
+
+llvm::cl::OptionCategory ClangSyntaxOptions("clang-syntax common options");
+
+llvm::cl::opt DumpTokens("dump-tokens",
+ llvm::cl::desc("dump the preprocessed tokens"),
+ llvm::cl::init(false),
+ llvm::cl::cat(ClangSyntaxOptions));
+llvm::cl::opt DumpSyntax("dump-syntax",
+ llvm::cl::desc("dump the syntax tree"),
+ llvm::cl::init(false),
+ llvm::cl::cat(ClangSyntaxOptions));
+
+class BuildSyntaxTree : public ASTFrontendAction {
+public:
+ std::unique_ptr CreateASTConsumer(CompilerInstance &CI,
+ StringRef InFile) override {
+class Consumer : public ASTConsumer {
+public:
+ Consumer(CompilerInstance &CI) : Collector(CI.getPreprocessor()) {}
+
+ void HandleTranslationUnit(ASTContext &AST) override {
+syntax::Arena A(AST.getSourceManager(), AST.getLangOpts(),
+std::move(Collector).consume());
+auto *TU = syntax::buildSyntaxTree(A, *AST.getTranslationUnitDecl());
+if (DumpTokens)
+ llvm::outs() << A.tokenBuffer().dumpForTests();
+if (DumpSyntax)
+ llvm::outs() << TU->dump(A);
+ }
+
+private:
+ syntax::TokenCollector Collector;
+};
+return std::make_unique(CI);
+ }
+};
+
+class Factory : public tooling::FrontendActionFactory {
+ std::unique_ptr create() override {
+return std::make_unique();
+ }
+};
+
+} // namespace
+
+int main(int argc, const char **argv) {
+ llvm::sys::PrintStackTraceOnErrorSignal(argv[0]);
+
+ auto Executor = clang::tooling::createExecutorFromCommandLineArgs(
+ argc, argv, ClangSyntaxOptions,
+ "Build syntax trees for the specified files");
+ if (!Executor) {
+llvm::errs() << llvm::toString(Executor.takeError()) << "\n";
+return 1;
+ }
+
+ if (!DumpTokens && !DumpSyntax) {
+llvm::errs()
+<< "Please specify at least one of -dump-tree or -dump-tokens\n";
+return 1;
+ }
+ // Collect symbols found in each translation unit, merging as we go.
+ auto Err = Executor->get()->execute(std::make_unique());
+ if (Err)
+llvm::errs() << llvm::toString(std::move(Err)) << "\n";
+ return 0;
+}
Index: clang/tools/clang-syntax/CMakeLists.txt
===
--- /dev/null
+++ clang/tools/clang-syntax/CMakeLists.txt
@@ -0,0 +1,16 @@
+set(LLVM_LINK_COMPONENTS Support)
+
+add_clang_tool(clang-syntax
+ SyntaxMain.cpp
+ )
+
+target_link_libraries(clang-syntax
+ PRIVATE
+ clangAST
+ clangBasic
+ clangFrontend
+ clangLex
+ clangTooling
+ clangToolingCore
+ clangToolingSyntax
+)
Index: clang/tools/CMakeLists.txt
===
--- clang/tools/CMakeLists.txt
+++ clang/tools/CMakeLists.txt
@@ -18,6 +18,7 @@
if(UNIX)
add_clang_subdirectory(clang-shlib)
endif()
+add_clang_subdirectory(clang-syntax)
if(CLANG_ENABLE_ARCMT)
add_clang_subdirectory(arcmt-test)
Index: clang/test/clang-syntax/tokens_no_file_arg.cpp
===
--- /dev/null
+++ clang/test/clang-syntax/tokens_no_file_arg.cpp
@@ -0,0 +1,10 @@
+// RUN: rm -rf %t.dir
+// RUN: mkdir -p %t.dir
+//
+// RUN: clang-syntax -dump-tokens 2> %t.dir/tokens_no_fil
[PATCH] D77053: [Syntax] A tool to dump syntax tree and token buffer
hlopko updated this revision to Diff 253561.
hlopko added a comment.
Cleanup
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D77053/new/
https://reviews.llvm.org/D77053
Files:
clang/test/CMakeLists.txt
clang/test/clang-syntax/no_args.cpp
clang/test/clang-syntax/syntax_hello_world.cpp
clang/test/clang-syntax/syntax_no_file_arg.cpp
clang/test/clang-syntax/tokens_hello_world.cpp
clang/test/clang-syntax/tokens_no_file_arg.cpp
clang/tools/CMakeLists.txt
clang/tools/clang-syntax/CMakeLists.txt
clang/tools/clang-syntax/SyntaxMain.cpp
Index: clang/tools/clang-syntax/SyntaxMain.cpp
===
--- /dev/null
+++ clang/tools/clang-syntax/SyntaxMain.cpp
@@ -0,0 +1,88 @@
+//===- SyntaxMain.cpp -*- C++ -*-=//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+#include "clang/Frontend/CompilerInstance.h"
+#include "clang/Tooling/Execution.h"
+#include "clang/Tooling/Syntax/BuildTree.h"
+#include "clang/Tooling/Syntax/Tokens.h"
+#include "clang/Tooling/Syntax/Tree.h"
+#include "clang/Tooling/Tooling.h"
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/Support/Signals.h"
+#include "llvm/Support/raw_ostream.h"
+#include
+
+using namespace clang;
+
+namespace {
+
+llvm::cl::OptionCategory ClangSyntaxOptions("clang-syntax common options");
+
+llvm::cl::opt DumpTokens("dump-tokens",
+ llvm::cl::desc("dump the preprocessed tokens"),
+ llvm::cl::init(false),
+ llvm::cl::cat(ClangSyntaxOptions));
+llvm::cl::opt DumpSyntax("dump-syntax",
+ llvm::cl::desc("dump the syntax tree"),
+ llvm::cl::init(false),
+ llvm::cl::cat(ClangSyntaxOptions));
+
+class BuildSyntaxTree : public ASTFrontendAction {
+public:
+ std::unique_ptr CreateASTConsumer(CompilerInstance &CI,
+ StringRef InFile) override {
+class Consumer : public ASTConsumer {
+public:
+ Consumer(CompilerInstance &CI) : Collector(CI.getPreprocessor()) {}
+
+ void HandleTranslationUnit(ASTContext &AST) override {
+syntax::Arena A(AST.getSourceManager(), AST.getLangOpts(),
+std::move(Collector).consume());
+auto *TU = syntax::buildSyntaxTree(A, *AST.getTranslationUnitDecl());
+if (DumpTokens)
+ llvm::outs() << A.tokenBuffer().dumpForTests();
+if (DumpSyntax)
+ llvm::outs() << TU->dump(A);
+ }
+
+private:
+ syntax::TokenCollector Collector;
+};
+return std::make_unique(CI);
+ }
+};
+
+class Factory : public tooling::FrontendActionFactory {
+ std::unique_ptr create() override {
+return std::make_unique();
+ }
+};
+
+} // namespace
+
+int main(int argc, const char **argv) {
+ llvm::sys::PrintStackTraceOnErrorSignal(argv[0]);
+
+ auto Executor = clang::tooling::createExecutorFromCommandLineArgs(
+ argc, argv, ClangSyntaxOptions,
+ "Build syntax trees for the specified files");
+ if (!Executor) {
+llvm::errs() << llvm::toString(Executor.takeError()) << "\n";
+return 1;
+ }
+
+ if (!DumpTokens && !DumpSyntax) {
+llvm::errs()
+<< "Please specify at least one of -dump-tree or -dump-tokens\n";
+return 1;
+ }
+ // Collect symbols found in each translation unit, merging as we go.
+ auto Err = Executor->get()->execute(std::make_unique());
+ if (Err)
+llvm::errs() << llvm::toString(std::move(Err)) << "\n";
+ return 0;
+}
Index: clang/tools/clang-syntax/CMakeLists.txt
===
--- /dev/null
+++ clang/tools/clang-syntax/CMakeLists.txt
@@ -0,0 +1,16 @@
+set(LLVM_LINK_COMPONENTS Support)
+
+add_clang_tool(clang-syntax
+ SyntaxMain.cpp
+ )
+
+target_link_libraries(clang-syntax
+ PRIVATE
+ clangAST
+ clangBasic
+ clangFrontend
+ clangLex
+ clangTooling
+ clangToolingCore
+ clangToolingSyntax
+)
Index: clang/tools/CMakeLists.txt
===
--- clang/tools/CMakeLists.txt
+++ clang/tools/CMakeLists.txt
@@ -18,6 +18,7 @@
if(UNIX)
add_clang_subdirectory(clang-shlib)
endif()
+add_clang_subdirectory(clang-syntax)
if(CLANG_ENABLE_ARCMT)
add_clang_subdirectory(arcmt-test)
Index: clang/test/clang-syntax/tokens_no_file_arg.cpp
===
--- /dev/null
+++ clang/test/clang-syntax/tokens_no_file_arg.cpp
@@ -0,0 +1,10 @@
+// RUN: rm -rf %t.dir
+// RUN: mkdir -p %t.dir
+//
+// RUN: clang-syntax -dump-tokens 2> %t.dir/tokens_no_fil
[PATCH] D77053: [Syntax] A tool to dump syntax tree and token buffer
hlopko abandoned this revision. hlopko added a comment. Talked to gribozavr2 offline and decided to abandon this patch. If in the future we'll have a need for a tool to dump syntax trees, we'll probably implement it in clang behind hidden options, and we'll provide a json output format as well. Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D77053/new/ https://reviews.llvm.org/D77053 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76922: [Syntax] Remove delayed folding from tree building.
hlopko updated this revision to Diff 253787.
hlopko marked an inline comment as done.
hlopko added a comment.
Remove unnecessary cast
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76922/new/
https://reviews.llvm.org/D76922
Files:
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Tokens.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -224,6 +224,59 @@
)txt");
}
+TEST_F(SyntaxTreeTest, SimpleVariable) {
+ expectTreeDumpEqual(
+ R"cpp(
+int a;
+int b = 42;
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | `-a
+| `-;
+`-SimpleDeclaration
+ |-int
+ |-SimpleDeclarator
+ | |-b
+ | |-=
+ | `-UnknownExpression
+ | `-42
+ `-;
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, SimpleFunction) {
+ expectTreeDumpEqual(
+ R"cpp(
+void foo(int a, int b) {}
+)cpp",
+ R"txt(
+*: TranslationUnit
+`-SimpleDeclaration
+ |-void
+ |-SimpleDeclarator
+ | |-foo
+ | `-ParametersAndQualifiers
+ | |-(
+ | |-SimpleDeclaration
+ | | |-int
+ | | `-SimpleDeclarator
+ | | `-a
+ | |-,
+ | |-SimpleDeclaration
+ | | |-int
+ | | `-SimpleDeclarator
+ | | `-b
+ | `-)
+ `-CompoundStatement
+|-{
+`-}
+)txt");
+}
+
TEST_F(SyntaxTreeTest, If) {
expectTreeDumpEqual(
R"cpp(
@@ -541,20 +594,32 @@
TEST_F(SyntaxTreeTest, MultipleDeclaratorsGrouping) {
expectTreeDumpEqual(
R"cpp(
- int *a, b;
+ int *a, b; int *c, d;
)cpp",
R"txt(
*: TranslationUnit
+|-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | |-*
+| | `-a
+| |-,
+| |-SimpleDeclarator
+| | `-b
+| `-;
`-SimpleDeclaration
|-int
|-SimpleDeclarator
| |-*
- | `-a
+ | `-c
|-,
|-SimpleDeclarator
- | `-b
+ | `-d
`-;
)txt");
+}
+
+TEST_F(SyntaxTreeTest, MultipleDeclaratorsGroupingTypedef) {
expectTreeDumpEqual(
R"cpp(
typedef int *a, b;
Index: clang/lib/Tooling/Syntax/Tokens.cpp
===
--- clang/lib/Tooling/Syntax/Tokens.cpp
+++ clang/lib/Tooling/Syntax/Tokens.cpp
@@ -710,8 +710,8 @@
}
std::string syntax::Token::dumpForTests(const SourceManager &SM) const {
- return std::string(
- llvm::formatv("{0} {1}", tok::getTokenName(kind()), text(SM)));
+ return std::string(llvm::formatv("Token(`{0}`, {1}, length = {2})", text(SM),
+ tok::getTokenName(kind()), length()));
}
std::string TokenBuffer::dumpForTests() const {
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -44,15 +44,6 @@
LLVM_ATTRIBUTE_UNUSED
static bool isImplicitExpr(clang::Expr *E) { return E->IgnoreImplicit() != E; }
-static SourceLocation getQualifiedNameStart(DeclaratorDecl *D) {
- auto DN = D->getDeclName();
- bool IsAnonymous = DN.isIdentifier() && !DN.getAsIdentifierInfo();
- if (IsAnonymous)
-return SourceLocation();
- return D->getQualifierLoc() ? D->getQualifierLoc().getBeginLoc()
- : D->getLocation();
-}
-
namespace {
/// Get start location of the Declarator from the TypeLoc.
/// E.g.:
@@ -142,8 +133,9 @@
End = Name;
}
if (Initializer.isValid()) {
-assert(SM.isBeforeInTranslationUnit(End, Initializer.getEnd()));
-End = Initializer.getEnd();
+auto InitializerEnd = Initializer.getEnd();
+assert(SM.isBeforeInTranslationUnit(End, InitializerEnd) || End == InitializerEnd);
+End = InitializerEnd;
}
return SourceRange(Start, End);
}
@@ -212,10 +204,6 @@
foldNode(Range, New, nullptr);
}
- /// Must be called with the range of each `DeclaratorDecl`. Ensures the
- /// corresponding declarator nodes are covered by `SimpleDeclaration`.
- void noticeDeclRange(llvm::ArrayRef Range);
-
/// Notifies that we should not consume trailing semicolon when computing
/// token range of \p D.
void noticeDeclWithoutSemicolon(Decl *D);
@@ -237,11 +225,6 @@
void markChild(syntax::Node *N, NodeRole R);
/// Set role for the syntax node matching \p N.
void markChild(ASTPtr N, NodeRole R);
- /// Set role for the delayed node that spans exactly \p Range.
- void markDelayedChild(llvm::ArrayRef Range, NodeRole R);
- /// Set role for the node that may or may not be delayed. Node must span
- /// exactly \p Range.
- void markMaybeDelayedChild(llvm::ArrayRef Range, NodeRole R);
/// Finish building the tree and consume the root node.
syntax::TranslationUnit *finalize() && {
@@ -285,7 +268,38 @@
return maybeAppendSemicolon(Tokens, D);
}
- llvm::ArrayRef getDeclRange(const Decl *D) const {
[PATCH] D76922: [Syntax] Remove delayed folding from tree building.
hlopko added a comment.
I propose adding those tests in a separate patch to keep this one focused.
Those tests are currently failing, because:
- we assume initializer, if present, is extending the declarator range, but
`struct P {} p;` has initializer ending where declarator ended already.
- UnknownExpressions finds its way inside SimpleDeclarator for p in `struct P
{} p, *pp`;
- typedefs also fail, unable to find the end of declarator properly. I didn't
investigate further.
Fixing those seems worthy of a separate patch.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76922/new/
https://reviews.llvm.org/D76922
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D77209: [Syntax] Add mapping from spelled to expanded tokens for TokenBuffer
hlopko created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
Same restrictions apply as in the other direction: macro arguments are
not supported yet, only full macro expansions can be mapped.
Taking over from https://reviews.llvm.org/D72581.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D77209
Files:
clang/include/clang/Tooling/Syntax/Tokens.h
clang/lib/Tooling/Syntax/Tokens.cpp
clang/unittests/Tooling/Syntax/TokensTest.cpp
Index: clang/unittests/Tooling/Syntax/TokensTest.cpp
===
--- clang/unittests/Tooling/Syntax/TokensTest.cpp
+++ clang/unittests/Tooling/Syntax/TokensTest.cpp
@@ -57,6 +57,7 @@
using ::testing::Contains;
using ::testing::ElementsAre;
using ::testing::Field;
+using ::testing::IsEmpty;
using ::testing::Matcher;
using ::testing::Not;
using ::testing::Pointee;
@@ -185,10 +186,14 @@
template
llvm::ArrayRef findSubrange(llvm::ArrayRef Subrange,
llvm::ArrayRef Range, Eq F) {
-for (auto Begin = Range.begin(); Begin < Range.end(); ++Begin) {
+assert(Subrange.size() >= 1);
+if (Range.size() < Subrange.size())
+ return llvm::makeArrayRef(Range.end(), Range.end());
+for (auto Begin = Range.begin(), Last = Range.end() - Subrange.size();
+ Begin <= Last; ++Begin) {
auto It = Begin;
- for (auto ItSub = Subrange.begin();
- ItSub != Subrange.end() && It != Range.end(); ++ItSub, ++It) {
+ for (auto ItSub = Subrange.begin(); ItSub != Subrange.end();
+ ++ItSub, ++It) {
if (!F(*ItSub, *It))
goto continue_outer;
}
@@ -889,4 +894,100 @@
ASSERT_EQ(Code.points().size(), 8u);
}
+TEST_F(TokenBufferTest, ExpandedBySpelled) {
+ recordTokens(R"cpp(
+a1 a2 a3 b1 b2
+ )cpp");
+ // Sanity check: expanded and spelled tokens are stored separately.
+ EXPECT_THAT(findExpanded("a1 a2"), Not(SameRange(findSpelled("a1 a2";
+ // Searching for subranges of expanded tokens should give the corresponding
+ // spelled ones.
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1 a2 a3 b1 b2")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1 a2 a3")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("b1 b2")),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+
+ // Test search on simple macro expansions.
+ recordTokens(R"cpp(
+#define A a1 a2 a3
+#define B b1 b2
+
+A split B
+ )cpp");
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("A split B")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 split b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("A split").drop_back()),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("split B").drop_front()),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+
+ // Recursive macro invocations.
+ recordTokens(R"cpp(
+#define ID(x) x
+#define B b1 b2
+
+ID(ID(ID(a1) a2 a3)) split ID(B)
+ )cpp");
+
+ EXPECT_THAT(
+ Buffer.expandedForSpelled(findSpelled("ID ( ID ( ID ( a1 ) a2 a3 ) )")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( B )")),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(
+ findSpelled("ID ( ID ( ID ( a1 ) a2 a3 ) ) split ID ( B )")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 split b1 b2";
+ // FIXME: these should succeed, but we do not support macro arguments yet.
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1")), IsEmpty());
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( a1 ) a2")),
+ IsEmpty());
+
+ // Empty macro expansions.
+ recordTokens(R"cpp(
+#define EMPTY
+#define ID(X) X
+
+EMPTY EMPTY ID(1 2 3) EMPTY EMPTY split1
+EMPTY EMPTY ID(4 5 6) split2
+ID(7 8 9) EMPTY EMPTY
+ )cpp");
+ // Covered by empty expansions on one of both of the sides.
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( 1 2 3 )")),
+ ElementsAre(SameRange(findExpanded("1 2 3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( 4 5 6 )")),
+ ElementsAre(SameRange(findExpanded("4 5 6";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( 7 8 9 )")),
+ ElementsAre(SameRange(findExpanded("7 8 9";
+ // Including the empty macro expansions on the side.
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("EMPTY ID ( 1 2 3 )")),
+ ElementsAre(SameRange(findExpanded("1 2 3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( 1 2 3 ) EMPTY")),
+ ElementsAre(SameRange(findExpanded("1
[PATCH] D72581: [Syntax] Add mapping from spelled to expanded tokens for TokenBuffer
hlopko marked 5 inline comments as done.
hlopko added a comment.
I'm submitting this patch at https://reviews.llvm.org/D77209 (with Ilya's
permission). Let's continue the review there.
Comment at: clang/include/clang/Tooling/Syntax/Tokens.h:228
+ /// multiple times and this function will return multiple results in those
+ /// cases. This happens when \p Spelled is inside a macro argument.
+ ///
sammccall wrote:
> Nit: move the FIXME up here? Documenting this justifies the signature, but
> currently it *never* happens.
Done.
Comment at: clang/include/clang/Tooling/Syntax/Tokens.h:248
+ llvm::SmallVector, 1>
+ expandedForSpelled(llvm::ArrayRef Spelled) const;
+
sammccall wrote:
> out of curiosity, do you have a motivating use for this function?
>
> I think it's clear enough that it will be useful, completeness dictates it
> should be here, and use cases aren't always the best way to think about
> designing these libraries. But it'd help me understand some of the details
> better.
Will ask Ilya offline.
Comment at: clang/lib/Tooling/Syntax/Tokens.cpp:198
+TokenBuffer::expandedForSpelled(llvm::ArrayRef Spelled) const {
+ assert(!Spelled.empty());
+ assert(Spelled.front().location().isFileID());
sammccall wrote:
> This is a little surprising (vs returning `{}`).
>
> It seems plausible that you'll map file offsets -> spelled tokens -> expanded
> tokens, and that the middle step might "accidentally" be empty. Caller can
> always check, but why require them to here?
Done.
Comment at: clang/lib/Tooling/Syntax/Tokens.cpp:205
+
+ auto &File = It->second;
+ assert(File.SpelledTokens.data() <= Spelled.data());
sammccall wrote:
> nit: mind spelling this type out? it's important, not particularly verbose,
> and long-lived
Done.
Comment at: clang/lib/Tooling/Syntax/Tokens.cpp:209
+
+ auto *FrontMapping = mappingStartingBeforeSpelled(File, &Spelled.front());
+ unsigned SpelledFrontI = &Spelled.front() - File.SpelledTokens.data();
sammccall wrote:
> This section could use some comments about how the index calculations relate
> to high-level concepts.
>
> AIUI the branches are: spelled token precedes all PP usage, spelled token is
> transformed by PP (subcases: macro args and other PP usage), spelled token
> follows PP usage.
>
> The next section probably only needs to comment about the purpose of the
> divergence (+1/End to refer to one-past the region of interest).
Done.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D72581/new/
https://reviews.llvm.org/D72581
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D77209: [Syntax] Add mapping from spelled to expanded tokens for TokenBuffer
hlopko updated this revision to Diff 254188.
hlopko added a comment.
Adding comments.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D77209/new/
https://reviews.llvm.org/D77209
Files:
clang/include/clang/Tooling/Syntax/Tokens.h
clang/lib/Tooling/Syntax/Tokens.cpp
clang/unittests/Tooling/Syntax/TokensTest.cpp
Index: clang/unittests/Tooling/Syntax/TokensTest.cpp
===
--- clang/unittests/Tooling/Syntax/TokensTest.cpp
+++ clang/unittests/Tooling/Syntax/TokensTest.cpp
@@ -57,6 +57,7 @@
using ::testing::Contains;
using ::testing::ElementsAre;
using ::testing::Field;
+using ::testing::IsEmpty;
using ::testing::Matcher;
using ::testing::Not;
using ::testing::Pointee;
@@ -185,10 +186,14 @@
template
llvm::ArrayRef findSubrange(llvm::ArrayRef Subrange,
llvm::ArrayRef Range, Eq F) {
-for (auto Begin = Range.begin(); Begin < Range.end(); ++Begin) {
+assert(Subrange.size() >= 1);
+if (Range.size() < Subrange.size())
+ return llvm::makeArrayRef(Range.end(), Range.end());
+for (auto Begin = Range.begin(), Last = Range.end() - Subrange.size();
+ Begin <= Last; ++Begin) {
auto It = Begin;
- for (auto ItSub = Subrange.begin();
- ItSub != Subrange.end() && It != Range.end(); ++ItSub, ++It) {
+ for (auto ItSub = Subrange.begin(); ItSub != Subrange.end();
+ ++ItSub, ++It) {
if (!F(*ItSub, *It))
goto continue_outer;
}
@@ -889,4 +894,111 @@
ASSERT_EQ(Code.points().size(), 8u);
}
+TEST_F(TokenBufferTest, ExpandedBySpelled) {
+ recordTokens(R"cpp(
+a1 a2 a3 b1 b2
+ )cpp");
+ // Sanity check: expanded and spelled tokens are stored separately.
+ EXPECT_THAT(findExpanded("a1 a2"), Not(SameRange(findSpelled("a1 a2";
+ // Searching for subranges of expanded tokens should give the corresponding
+ // spelled ones.
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1 a2 a3 b1 b2")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1 a2 a3")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("b1 b2")),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+
+ // Test search on simple macro expansions.
+ recordTokens(R"cpp(
+#define A a1 a2 a3
+#define B b1 b2
+
+A split B
+ )cpp");
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("A split B")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 split b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("A split").drop_back()),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("split B").drop_front()),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+
+ // Ranges not fully covering macro expansions should fail.
+ recordTokens(R"cpp(
+#define ID(x) x
+
+ID(a)
+ )cpp");
+ // Spelled don't cover entire mapping (missing ID token) -> empty result
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("( a )")), IsEmpty());
+ // Spelled don't cover entire mapping (missing ) token) -> empty result
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( a")), IsEmpty());
+
+ // Recursive macro invocations.
+ recordTokens(R"cpp(
+#define ID(x) x
+#define B b1 b2
+
+ID(ID(ID(a1) a2 a3)) split ID(B)
+ )cpp");
+
+ EXPECT_THAT(
+ Buffer.expandedForSpelled(findSpelled("ID ( ID ( ID ( a1 ) a2 a3 ) )")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( B )")),
+ ElementsAre(SameRange(findExpanded("b1 b2";
+ EXPECT_THAT(Buffer.expandedForSpelled(
+ findSpelled("ID ( ID ( ID ( a1 ) a2 a3 ) ) split ID ( B )")),
+ ElementsAre(SameRange(findExpanded("a1 a2 a3 split b1 b2";
+ // FIXME: these should succeed, but we do not support macro arguments yet.
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("a1")), IsEmpty());
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( a1 ) a2")),
+ IsEmpty());
+
+ // Empty macro expansions.
+ recordTokens(R"cpp(
+#define EMPTY
+#define ID(X) X
+
+EMPTY EMPTY ID(1 2 3) EMPTY EMPTY split1
+EMPTY EMPTY ID(4 5 6) split2
+ID(7 8 9) EMPTY EMPTY
+ )cpp");
+ // Covered by empty expansions on one of both of the sides.
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( 1 2 3 )")),
+ ElementsAre(SameRange(findExpanded("1 2 3";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( 4 5 6 )")),
+ ElementsAre(SameRange(findExpanded("4 5 6";
+ EXPECT_THAT(Buffer.expandedForSpelled(findSpelled("ID ( 7 8 9 )")),
+ ElementsAre(SameRange(findExpanded("7 8 9";
+ // Including the em
[PATCH] D76001: [AST] Respect shouldTraversePostOrder when traversing type locs
hlopko created this revision.
hlopko added a reviewer: gribozavr2.
hlopko added a project: clang.
Herald added a subscriber: cfe-commits.
Copy of https://reviews.llvm.org/D72072, submitting with ilya-biryukov's
permission.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D76001
Files:
clang/include/clang/AST/RecursiveASTVisitor.h
Index: clang/include/clang/AST/RecursiveASTVisitor.h
===
--- clang/include/clang/AST/RecursiveASTVisitor.h
+++ clang/include/clang/AST/RecursiveASTVisitor.h
@@ -1127,10 +1127,17 @@
#define DEF_TRAVERSE_TYPELOC(TYPE, CODE)
\
template
\
bool RecursiveASTVisitor::Traverse##TYPE##Loc(TYPE##Loc TL) {
\
-if (getDerived().shouldWalkTypesOfTypeLocs())
\
- TRY_TO(WalkUpFrom##TYPE(const_cast(TL.getTypePtr(;
\
-TRY_TO(WalkUpFrom##TYPE##Loc(TL));
\
+if (!getDerived().shouldTraversePostOrder()) {
\
+ TRY_TO(WalkUpFrom##TYPE##Loc(TL));
\
+ if (getDerived().shouldWalkTypesOfTypeLocs())
\
+TRY_TO(WalkUpFrom##TYPE(const_cast(TL.getTypePtr(;
\
+}
\
{ CODE; }
\
+if (getDerived().shouldTraversePostOrder()) {
\
+ TRY_TO(WalkUpFrom##TYPE##Loc(TL));
\
+ if (getDerived().shouldWalkTypesOfTypeLocs())
\
+TRY_TO(WalkUpFrom##TYPE(const_cast(TL.getTypePtr(;
\
+}
\
return true;
\
}
@@ -1199,22 +1206,22 @@
DEF_TRAVERSE_TYPELOC(ConstantArrayType, {
TRY_TO(TraverseTypeLoc(TL.getElementLoc()));
- return TraverseArrayTypeLocHelper(TL);
+ TRY_TO(TraverseArrayTypeLocHelper(TL));
})
DEF_TRAVERSE_TYPELOC(IncompleteArrayType, {
TRY_TO(TraverseTypeLoc(TL.getElementLoc()));
- return TraverseArrayTypeLocHelper(TL);
+ TRY_TO(TraverseArrayTypeLocHelper(TL));
})
DEF_TRAVERSE_TYPELOC(VariableArrayType, {
TRY_TO(TraverseTypeLoc(TL.getElementLoc()));
- return TraverseArrayTypeLocHelper(TL);
+ TRY_TO(TraverseArrayTypeLocHelper(TL));
})
DEF_TRAVERSE_TYPELOC(DependentSizedArrayType, {
TRY_TO(TraverseTypeLoc(TL.getElementLoc()));
- return TraverseArrayTypeLocHelper(TL);
+ TRY_TO(TraverseArrayTypeLocHelper(TL));
})
DEF_TRAVERSE_TYPELOC(DependentAddressSpaceType, {
Index: clang/include/clang/AST/RecursiveASTVisitor.h
===
--- clang/include/clang/AST/RecursiveASTVisitor.h
+++ clang/include/clang/AST/RecursiveASTVisitor.h
@@ -1127,10 +1127,17 @@
#define DEF_TRAVERSE_TYPELOC(TYPE, CODE) \
template \
bool RecursiveASTVisitor::Traverse##TYPE##Loc(TYPE##Loc TL) { \
-if (getDerived().shouldWalkTypesOfTypeLocs()) \
- TRY_TO(WalkUpFrom##TYPE(const_cast(TL.getTypePtr(; \
-TRY_TO(WalkUpFrom##TYPE##Loc(TL)); \
+if (!getDerived().shouldTraversePostOrder()) { \
+ TRY_TO(WalkUpFrom##TYPE##Loc(TL)); \
+ if (getDerived().shouldWalkTypesOfTypeLocs())\
+TRY_TO(WalkUpFrom##TYPE(const_cast(TL.getTypePtr(; \
+} \
{ CODE; } \
+if (getDerived().shouldTraversePostOrder()) { \
+ TRY_TO(WalkUpFrom##TYPE##Loc(TL)); \
+ if (getDerived().shouldWalkTypesOfTypeLocs())\
+TRY_TO(WalkUpFrom##TYPE(const_cast(TL.getTypePtr(; \
+} \
return true; \
}
@@ -1199,22 +1206,22 @@
DEF_TRAVERSE_TYPELOC(ConstantArrayType, {
TRY_TO(TraverseTypeLoc(TL.getElementLoc()));
- return TraverseArrayTypeLocHelper(TL);
+ TRY_TO(TraverseArrayTypeLocHelper(TL));
})
DEF_TRAVERSE_TYPELOC(IncompleteArrayType, {
TRY_TO(TraverseTypeLoc(TL.getElementLoc()));
- return TraverseArrayTypeLocHelper(TL);
+ TRY_TO(TraverseArrayTypeLocHelper(TL));
})
DEF_TRAVERSE_TYPELOC(VariableArrayType, {
TRY_TO(TraverseTyp
[PATCH] D76061: [Sema] Fix location of star ('*') inside MemberPointerTypeLoc
hlopko created this revision.
hlopko added a reviewer: gribozavr.
hlopko added a project: clang.
Herald added a subscriber: cfe-commits.
Copy of https://reviews.llvm.org/D72073?id=235842, submitting with
ilya-biryukov's permission.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D76061
Files:
clang/include/clang/Sema/DeclSpec.h
clang/lib/Parse/ParseDecl.cpp
clang/lib/Sema/SemaType.cpp
clang/unittests/AST/DeclTest.cpp
Index: clang/unittests/AST/DeclTest.cpp
===
--- clang/unittests/AST/DeclTest.cpp
+++ clang/unittests/AST/DeclTest.cpp
@@ -107,3 +107,20 @@
ASSERT_TRUE(0 == MangleF.compare("\x01" "foo"));
ASSERT_TRUE(0 == MangleG.compare("goo"));
}
+
+TEST(Decl, MemberPointerStarLoc) {
+ StringRef Code = R"(
+struct X {};
+int X::* a;
+ )";
+ auto AST = tooling::buildASTFromCodeWithArgs(Code, {});
+ SourceManager &SM = AST->getSourceManager();
+
+ SmallVector Decls;
+ AST->findFileRegionDecls(SM.getMainFileID(), Code.find('X'), 0, Decls);
+ ASSERT_TRUE(Decls.size() == 2);
+ VarDecl *D = cast(Decls[1]);
+ auto TL =
D->getTypeSourceInfo()->getTypeLoc().castAs();
+ auto StarLoc = TL.getStarLoc().printToString(SM);
+ ASSERT_EQ(StarLoc, "input.cc:3:12");
+}
Index: clang/lib/Sema/SemaType.cpp
===
--- clang/lib/Sema/SemaType.cpp
+++ clang/lib/Sema/SemaType.cpp
@@ -5840,7 +5840,7 @@
}
// Finally fill in MemberPointerLocInfo fields.
- TL.setStarLoc(Chunk.Loc);
+ TL.setStarLoc(SourceLocation::getFromRawEncoding(Chunk.Mem.StarLoc));
TL.setClassTInfo(ClsTInfo);
}
void VisitLValueReferenceTypeLoc(LValueReferenceTypeLoc TL) {
Index: clang/lib/Parse/ParseDecl.cpp
===
--- clang/lib/Parse/ParseDecl.cpp
+++ clang/lib/Parse/ParseDecl.cpp
@@ -5643,8 +5643,8 @@
return;
}
- SourceLocation Loc = ConsumeToken();
- D.SetRangeEnd(Loc);
+ SourceLocation StarLoc = ConsumeToken();
+ D.SetRangeEnd(StarLoc);
DeclSpec DS(AttrFactory);
ParseTypeQualifierListOpt(DS);
D.ExtendWithDeclSpec(DS);
@@ -5655,7 +5655,7 @@
// Sema will have to catch (syntactically invalid) pointers into global
// scope. It has to catch pointers into namespace scope anyway.
D.AddTypeInfo(DeclaratorChunk::getMemberPointer(
-SS, DS.getTypeQualifiers(), DS.getEndLoc()),
+SS, DS.getTypeQualifiers(), StarLoc, DS.getEndLoc()),
std::move(DS.getAttributes()),
/* Don't replace range end. */ SourceLocation());
return;
Index: clang/include/clang/Sema/DeclSpec.h
===
--- clang/include/clang/Sema/DeclSpec.h
+++ clang/include/clang/Sema/DeclSpec.h
@@ -1518,6 +1518,8 @@
struct MemberPointerTypeInfo {
/// The type qualifiers: const/volatile/restrict/__unaligned/_Atomic.
unsigned TypeQuals : 5;
+/// Location of the '*' token.
+unsigned StarLoc;
// CXXScopeSpec has a constructor, so it can't be a direct member.
// So we need some pointer-aligned storage and a bit of trickery.
alignas(CXXScopeSpec) char ScopeMem[sizeof(CXXScopeSpec)];
@@ -1660,11 +1662,13 @@
static DeclaratorChunk getMemberPointer(const CXXScopeSpec &SS,
unsigned TypeQuals,
- SourceLocation Loc) {
+ SourceLocation StarLoc,
+ SourceLocation EndLoc) {
DeclaratorChunk I;
I.Kind = MemberPointer;
I.Loc = SS.getBeginLoc();
-I.EndLoc= Loc;
+I.EndLoc= EndLoc;
+I.Mem.StarLoc = StarLoc.getRawEncoding();
I.Mem.TypeQuals = TypeQuals;
new (I.Mem.ScopeMem) CXXScopeSpec(SS);
return I;
Index: clang/unittests/AST/DeclTest.cpp
===
--- clang/unittests/AST/DeclTest.cpp
+++ clang/unittests/AST/DeclTest.cpp
@@ -107,3 +107,20 @@
ASSERT_TRUE(0 == MangleF.compare("\x01" "foo"));
ASSERT_TRUE(0 == MangleG.compare("goo"));
}
+
+TEST(Decl, MemberPointerStarLoc) {
+ StringRef Code = R"(
+struct X {};
+int X::* a;
+ )";
+ auto AST = tooling::buildASTFromCodeWithArgs(Code, {});
+ SourceManager &SM = AST->getSourceManager();
+
+ SmallVector Decls;
+ AST->findFileRegionDecls(SM.getMainFileID(), Code.find('X'), 0, Decls);
+ ASSERT_TRUE(Decls.size() == 2);
+ VarDecl *D = cast(Decls[1]);
+ auto TL = D->getTypeSourceInfo()->getTypeLoc().castAs();
+ auto StarLoc = TL.getStarLoc().printToString(SM);
+ ASSERT_EQ(StarLoc, "input.cc:3:12");
+}
Index: clang/lib/Sema/SemaType.cpp
===
--- clang/lib/Sema/SemaTyp
[PATCH] D76061: [Sema] Fix location of star ('*') inside MemberPointerTypeLoc
hlopko updated this revision to Diff 250024.
hlopko added a comment.
Herald added a subscriber: mgorny.
[Sema] Fix location of star ('*') inside MemberPointerTypeLoc
Copy of https://reviews.llvm.org/D72073?id=235842, submitting with
ilya-biryukov's permission.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76061/new/
https://reviews.llvm.org/D76061
Files:
clang/include/clang/Sema/DeclSpec.h
clang/lib/Parse/ParseDecl.cpp
clang/lib/Sema/SemaType.cpp
clang/unittests/AST/CMakeLists.txt
clang/unittests/AST/SourceLocationTest.cpp
Index: clang/unittests/AST/SourceLocationTest.cpp
===
--- clang/unittests/AST/SourceLocationTest.cpp
+++ clang/unittests/AST/SourceLocationTest.cpp
@@ -21,6 +21,7 @@
#include "clang/ASTMatchers/ASTMatchers.h"
#include "clang/Tooling/Tooling.h"
#include "gtest/gtest.h"
+#include "llvm/Testing/Support/Annotations.h"
namespace clang {
namespace ast_matchers {
@@ -843,4 +844,25 @@
}
} // end namespace ast_matchers
+
+namespace decls {
+
+TEST(Decl, MemberPointerStarLoc) {
+ llvm::Annotations Example(R"cpp(
+struct X {};
+int X::$star^* a;
+ )cpp");
+ auto AST = tooling::buildASTFromCode(Example.code());
+ SourceManager &SM = AST->getSourceManager();
+
+ SmallVector Decls;
+ AST->findFileRegionDecls(SM.getMainFileID(), Example.point("star"), 0, Decls);
+ ASSERT_TRUE(Decls.size() == 2);
+
+ VarDecl *D = cast(Decls[1]);
+ auto TL = D->getTypeSourceInfo()->getTypeLoc().castAs();
+ ASSERT_EQ(SM.getFileOffset(TL.getStarLoc()), Example.point("star"));
+}
+
+} // end namespace decls
} // end namespace clang
Index: clang/unittests/AST/CMakeLists.txt
===
--- clang/unittests/AST/CMakeLists.txt
+++ clang/unittests/AST/CMakeLists.txt
@@ -42,4 +42,5 @@
clangFrontend
clangSerialization
clangTooling
+ LLVMTestingSupport
)
Index: clang/lib/Sema/SemaType.cpp
===
--- clang/lib/Sema/SemaType.cpp
+++ clang/lib/Sema/SemaType.cpp
@@ -5840,7 +5840,7 @@
}
// Finally fill in MemberPointerLocInfo fields.
- TL.setStarLoc(Chunk.Loc);
+ TL.setStarLoc(SourceLocation::getFromRawEncoding(Chunk.Mem.StarLoc));
TL.setClassTInfo(ClsTInfo);
}
void VisitLValueReferenceTypeLoc(LValueReferenceTypeLoc TL) {
Index: clang/lib/Parse/ParseDecl.cpp
===
--- clang/lib/Parse/ParseDecl.cpp
+++ clang/lib/Parse/ParseDecl.cpp
@@ -5643,8 +5643,8 @@
return;
}
- SourceLocation Loc = ConsumeToken();
- D.SetRangeEnd(Loc);
+ SourceLocation StarLoc = ConsumeToken();
+ D.SetRangeEnd(StarLoc);
DeclSpec DS(AttrFactory);
ParseTypeQualifierListOpt(DS);
D.ExtendWithDeclSpec(DS);
@@ -5655,7 +5655,7 @@
// Sema will have to catch (syntactically invalid) pointers into global
// scope. It has to catch pointers into namespace scope anyway.
D.AddTypeInfo(DeclaratorChunk::getMemberPointer(
-SS, DS.getTypeQualifiers(), DS.getEndLoc()),
+SS, DS.getTypeQualifiers(), StarLoc, DS.getEndLoc()),
std::move(DS.getAttributes()),
/* Don't replace range end. */ SourceLocation());
return;
Index: clang/include/clang/Sema/DeclSpec.h
===
--- clang/include/clang/Sema/DeclSpec.h
+++ clang/include/clang/Sema/DeclSpec.h
@@ -1518,6 +1518,8 @@
struct MemberPointerTypeInfo {
/// The type qualifiers: const/volatile/restrict/__unaligned/_Atomic.
unsigned TypeQuals : 5;
+/// Location of the '*' token.
+unsigned StarLoc;
// CXXScopeSpec has a constructor, so it can't be a direct member.
// So we need some pointer-aligned storage and a bit of trickery.
alignas(CXXScopeSpec) char ScopeMem[sizeof(CXXScopeSpec)];
@@ -1660,11 +1662,13 @@
static DeclaratorChunk getMemberPointer(const CXXScopeSpec &SS,
unsigned TypeQuals,
- SourceLocation Loc) {
+ SourceLocation StarLoc,
+ SourceLocation EndLoc) {
DeclaratorChunk I;
I.Kind = MemberPointer;
I.Loc = SS.getBeginLoc();
-I.EndLoc= Loc;
+I.EndLoc= EndLoc;
+I.Mem.StarLoc = StarLoc.getRawEncoding();
I.Mem.TypeQuals = TypeQuals;
new (I.Mem.ScopeMem) CXXScopeSpec(SS);
return I;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76061: [Sema] Fix location of star ('*') inside MemberPointerTypeLoc
hlopko updated this revision to Diff 250142.
hlopko added a comment.
Reformat
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76061/new/
https://reviews.llvm.org/D76061
Files:
clang/include/clang/Sema/DeclSpec.h
clang/lib/Parse/ParseDecl.cpp
clang/lib/Sema/SemaType.cpp
clang/unittests/AST/CMakeLists.txt
clang/unittests/AST/SourceLocationTest.cpp
Index: clang/unittests/AST/SourceLocationTest.cpp
===
--- clang/unittests/AST/SourceLocationTest.cpp
+++ clang/unittests/AST/SourceLocationTest.cpp
@@ -15,11 +15,12 @@
//
//===--===//
-#include "clang/AST/ASTContext.h"
#include "MatchVerifier.h"
+#include "clang/AST/ASTContext.h"
#include "clang/ASTMatchers/ASTMatchFinder.h"
#include "clang/ASTMatchers/ASTMatchers.h"
#include "clang/Tooling/Tooling.h"
+#include "llvm/Testing/Support/Annotations.h"
#include "gtest/gtest.h"
namespace clang {
@@ -842,5 +843,21 @@
Language::Lang_CXX11));
}
+TEST(Decl, MemberPointerStarLoc) {
+ llvm::Annotations Example(R"cpp(
+struct X {};
+int X::$star^* a;
+ )cpp");
+ auto AST = tooling::buildASTFromCode(Example.code());
+ SourceManager &SM = AST->getSourceManager();
+ auto &Ctx = AST->getASTContext();
+
+ auto *D = selectFirst("vd", match(varDecl().bind("vd"), Ctx));
+
+ ASSERT_TRUE(D != nullptr);
+ auto TL = D->getTypeSourceInfo()->getTypeLoc().castAs();
+ ASSERT_EQ(SM.getFileOffset(TL.getStarLoc()), Example.point("star"));
+}
+
} // end namespace ast_matchers
} // end namespace clang
Index: clang/unittests/AST/CMakeLists.txt
===
--- clang/unittests/AST/CMakeLists.txt
+++ clang/unittests/AST/CMakeLists.txt
@@ -42,4 +42,5 @@
clangFrontend
clangSerialization
clangTooling
+ LLVMTestingSupport
)
Index: clang/lib/Sema/SemaType.cpp
===
--- clang/lib/Sema/SemaType.cpp
+++ clang/lib/Sema/SemaType.cpp
@@ -5840,7 +5840,7 @@
}
// Finally fill in MemberPointerLocInfo fields.
- TL.setStarLoc(Chunk.Loc);
+ TL.setStarLoc(SourceLocation::getFromRawEncoding(Chunk.Mem.StarLoc));
TL.setClassTInfo(ClsTInfo);
}
void VisitLValueReferenceTypeLoc(LValueReferenceTypeLoc TL) {
Index: clang/lib/Parse/ParseDecl.cpp
===
--- clang/lib/Parse/ParseDecl.cpp
+++ clang/lib/Parse/ParseDecl.cpp
@@ -5643,8 +5643,8 @@
return;
}
- SourceLocation Loc = ConsumeToken();
- D.SetRangeEnd(Loc);
+ SourceLocation StarLoc = ConsumeToken();
+ D.SetRangeEnd(StarLoc);
DeclSpec DS(AttrFactory);
ParseTypeQualifierListOpt(DS);
D.ExtendWithDeclSpec(DS);
@@ -5655,7 +5655,7 @@
// Sema will have to catch (syntactically invalid) pointers into global
// scope. It has to catch pointers into namespace scope anyway.
D.AddTypeInfo(DeclaratorChunk::getMemberPointer(
-SS, DS.getTypeQualifiers(), DS.getEndLoc()),
+SS, DS.getTypeQualifiers(), StarLoc, DS.getEndLoc()),
std::move(DS.getAttributes()),
/* Don't replace range end. */ SourceLocation());
return;
Index: clang/include/clang/Sema/DeclSpec.h
===
--- clang/include/clang/Sema/DeclSpec.h
+++ clang/include/clang/Sema/DeclSpec.h
@@ -1518,6 +1518,8 @@
struct MemberPointerTypeInfo {
/// The type qualifiers: const/volatile/restrict/__unaligned/_Atomic.
unsigned TypeQuals : 5;
+/// Location of the '*' token.
+unsigned StarLoc;
// CXXScopeSpec has a constructor, so it can't be a direct member.
// So we need some pointer-aligned storage and a bit of trickery.
alignas(CXXScopeSpec) char ScopeMem[sizeof(CXXScopeSpec)];
@@ -1660,11 +1662,13 @@
static DeclaratorChunk getMemberPointer(const CXXScopeSpec &SS,
unsigned TypeQuals,
- SourceLocation Loc) {
+ SourceLocation StarLoc,
+ SourceLocation EndLoc) {
DeclaratorChunk I;
I.Kind = MemberPointer;
I.Loc = SS.getBeginLoc();
-I.EndLoc= Loc;
+I.EndLoc= EndLoc;
+I.Mem.StarLoc = StarLoc.getRawEncoding();
I.Mem.TypeQuals = TypeQuals;
new (I.Mem.ScopeMem) CXXScopeSpec(SS);
return I;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76120: Refactor SourceLocationTest to `using namespace`
hlopko updated this revision to Diff 250147.
hlopko added a comment.
Reformat
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76120/new/
https://reviews.llvm.org/D76120
Files:
clang/unittests/AST/SourceLocationTest.cpp
Index: clang/unittests/AST/SourceLocationTest.cpp
===
--- clang/unittests/AST/SourceLocationTest.cpp
+++ clang/unittests/AST/SourceLocationTest.cpp
@@ -23,8 +23,8 @@
#include "llvm/Testing/Support/Annotations.h"
#include "gtest/gtest.h"
-namespace clang {
-namespace ast_matchers {
+using namespace clang;
+using namespace clang::ast_matchers;
// FIXME: Pull the *Verifier tests into their own test file.
@@ -646,7 +646,7 @@
EXPECT_TRUE(Verifier.match(
"void f() noexcept(false);\n",
functionDecl(),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
}
class FunctionDeclParametersRangeVerifier : public RangeVerifier
{
@@ -773,7 +773,7 @@
"void f() noexcept(false);\n"
"};\n",
functionDecl(),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
}
class ExceptionSpecRangeVerifier : public RangeVerifier {
@@ -816,19 +816,20 @@
std::vector Args;
Args.push_back("-fms-extensions");
EXPECT_TRUE(Verifier.match("void f() throw(...);\n", loc(functionType()),
- Args, Language::Lang_CXX));
+ Args, clang::ast_matchers::Language::Lang_CXX));
Verifier.expectRange(1, 10, 1, 10);
EXPECT_TRUE(Verifier.match("void f() noexcept;\n", loc(functionType()),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
Verifier.expectRange(1, 10, 1, 24);
EXPECT_TRUE(Verifier.match("void f() noexcept(false);\n",
loc(functionType()),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
Verifier.expectRange(1, 10, 1, 32);
EXPECT_TRUE(Verifier.match("void f() noexcept(noexcept(1+1));\n",
- loc(functionType()), Language::Lang_CXX11));
+ loc(functionType()),
+ clang::ast_matchers::Language::Lang_CXX11));
ParmVarExceptionSpecRangeVerifier Verifier2;
Verifier2.expectRange(1, 25, 1, 31);
@@ -840,7 +841,7 @@
EXPECT_TRUE(Verifier2.match("void g(void (*fp)(void) noexcept(true));\n",
parmVarDecl(hasType(pointerType(pointee(
parenType(innerType(functionType())),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
}
TEST(Decl, MemberPointerStarLoc) {
@@ -859,6 +860,3 @@
auto TL =
D->getTypeSourceInfo()->getTypeLoc().castAs();
ASSERT_EQ(SM.getFileOffset(TL.getStarLoc()), Example.point("star"));
}
-
-} // end namespace ast_matchers
-} // end namespace clang
Index: clang/unittests/AST/SourceLocationTest.cpp
===
--- clang/unittests/AST/SourceLocationTest.cpp
+++ clang/unittests/AST/SourceLocationTest.cpp
@@ -23,8 +23,8 @@
#include "llvm/Testing/Support/Annotations.h"
#include "gtest/gtest.h"
-namespace clang {
-namespace ast_matchers {
+using namespace clang;
+using namespace clang::ast_matchers;
// FIXME: Pull the *Verifier tests into their own test file.
@@ -646,7 +646,7 @@
EXPECT_TRUE(Verifier.match(
"void f() noexcept(false);\n",
functionDecl(),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
}
class FunctionDeclParametersRangeVerifier : public RangeVerifier {
@@ -773,7 +773,7 @@
"void f() noexcept(false);\n"
"};\n",
functionDecl(),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
}
class ExceptionSpecRangeVerifier : public RangeVerifier {
@@ -816,19 +816,20 @@
std::vector Args;
Args.push_back("-fms-extensions");
EXPECT_TRUE(Verifier.match("void f() throw(...);\n", loc(functionType()),
- Args, Language::Lang_CXX));
+ Args, clang::ast_matchers::Language::Lang_CXX));
Verifier.expectRange(1, 10, 1, 10);
EXPECT_TRUE(Verifier.match("void f() noexcept;\n", loc(functionType()),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
Verifier.expectRange(1, 10, 1, 24);
EXPECT_TRUE(Verifier.match("void f() noexcept(false);\n", loc(functionType()),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
Verifier.expectRange(1, 10, 1, 32);
EXPECT_TRUE(Verifier.match("void f() noexcept(noexcept(1+1));\n",
-
[PATCH] D76120: Refactor SourceLocationTest to `using namespace`
hlopko created this revision.
hlopko added a reviewer: gribozavr.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
hlopko updated this revision to Diff 250147.
hlopko added a comment.
Reformat
Only for the readability reasons.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D76120
Files:
clang/unittests/AST/SourceLocationTest.cpp
Index: clang/unittests/AST/SourceLocationTest.cpp
===
--- clang/unittests/AST/SourceLocationTest.cpp
+++ clang/unittests/AST/SourceLocationTest.cpp
@@ -23,8 +23,8 @@
#include "llvm/Testing/Support/Annotations.h"
#include "gtest/gtest.h"
-namespace clang {
-namespace ast_matchers {
+using namespace clang;
+using namespace clang::ast_matchers;
// FIXME: Pull the *Verifier tests into their own test file.
@@ -646,7 +646,7 @@
EXPECT_TRUE(Verifier.match(
"void f() noexcept(false);\n",
functionDecl(),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
}
class FunctionDeclParametersRangeVerifier : public RangeVerifier
{
@@ -773,7 +773,7 @@
"void f() noexcept(false);\n"
"};\n",
functionDecl(),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
}
class ExceptionSpecRangeVerifier : public RangeVerifier {
@@ -816,19 +816,20 @@
std::vector Args;
Args.push_back("-fms-extensions");
EXPECT_TRUE(Verifier.match("void f() throw(...);\n", loc(functionType()),
- Args, Language::Lang_CXX));
+ Args, clang::ast_matchers::Language::Lang_CXX));
Verifier.expectRange(1, 10, 1, 10);
EXPECT_TRUE(Verifier.match("void f() noexcept;\n", loc(functionType()),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
Verifier.expectRange(1, 10, 1, 24);
EXPECT_TRUE(Verifier.match("void f() noexcept(false);\n",
loc(functionType()),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
Verifier.expectRange(1, 10, 1, 32);
EXPECT_TRUE(Verifier.match("void f() noexcept(noexcept(1+1));\n",
- loc(functionType()), Language::Lang_CXX11));
+ loc(functionType()),
+ clang::ast_matchers::Language::Lang_CXX11));
ParmVarExceptionSpecRangeVerifier Verifier2;
Verifier2.expectRange(1, 25, 1, 31);
@@ -840,7 +841,7 @@
EXPECT_TRUE(Verifier2.match("void g(void (*fp)(void) noexcept(true));\n",
parmVarDecl(hasType(pointerType(pointee(
parenType(innerType(functionType())),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
}
TEST(Decl, MemberPointerStarLoc) {
@@ -859,6 +860,3 @@
auto TL =
D->getTypeSourceInfo()->getTypeLoc().castAs();
ASSERT_EQ(SM.getFileOffset(TL.getStarLoc()), Example.point("star"));
}
-
-} // end namespace ast_matchers
-} // end namespace clang
Index: clang/unittests/AST/SourceLocationTest.cpp
===
--- clang/unittests/AST/SourceLocationTest.cpp
+++ clang/unittests/AST/SourceLocationTest.cpp
@@ -23,8 +23,8 @@
#include "llvm/Testing/Support/Annotations.h"
#include "gtest/gtest.h"
-namespace clang {
-namespace ast_matchers {
+using namespace clang;
+using namespace clang::ast_matchers;
// FIXME: Pull the *Verifier tests into their own test file.
@@ -646,7 +646,7 @@
EXPECT_TRUE(Verifier.match(
"void f() noexcept(false);\n",
functionDecl(),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
}
class FunctionDeclParametersRangeVerifier : public RangeVerifier {
@@ -773,7 +773,7 @@
"void f() noexcept(false);\n"
"};\n",
functionDecl(),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
}
class ExceptionSpecRangeVerifier : public RangeVerifier {
@@ -816,19 +816,20 @@
std::vector Args;
Args.push_back("-fms-extensions");
EXPECT_TRUE(Verifier.match("void f() throw(...);\n", loc(functionType()),
- Args, Language::Lang_CXX));
+ Args, clang::ast_matchers::Language::Lang_CXX));
Verifier.expectRange(1, 10, 1, 10);
EXPECT_TRUE(Verifier.match("void f() noexcept;\n", loc(functionType()),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
Verifier.expectRange(1, 10, 1, 24);
EXPECT_TRUE(Verifier.match("void f() noexcept(false);\n", loc(functionType()),
- Language::Lang_CXX11));
+ clang::ast_matchers::Language::Lang_CXX11));
Verifier.expec
[PATCH] D76061: [Sema] Fix location of star ('*') inside MemberPointerTypeLoc
hlopko updated this revision to Diff 250141.
hlopko added a comment.
Refactor the test to use matchers.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76061/new/
https://reviews.llvm.org/D76061
Files:
clang/include/clang/Sema/DeclSpec.h
clang/lib/Parse/ParseDecl.cpp
clang/lib/Sema/SemaType.cpp
clang/unittests/AST/CMakeLists.txt
clang/unittests/AST/SourceLocationTest.cpp
Index: clang/unittests/AST/SourceLocationTest.cpp
===
--- clang/unittests/AST/SourceLocationTest.cpp
+++ clang/unittests/AST/SourceLocationTest.cpp
@@ -21,6 +21,7 @@
#include "clang/ASTMatchers/ASTMatchers.h"
#include "clang/Tooling/Tooling.h"
#include "gtest/gtest.h"
+#include "llvm/Testing/Support/Annotations.h"
namespace clang {
namespace ast_matchers {
@@ -842,5 +843,23 @@
Language::Lang_CXX11));
}
+TEST(Decl, MemberPointerStarLoc) {
+ llvm::Annotations Example(R"cpp(
+struct X {};
+int X::$star^* a;
+ )cpp");
+ auto AST = tooling::buildASTFromCode(Example.code());
+ SourceManager &SM = AST->getSourceManager();
+ auto &Ctx = AST->getASTContext();
+
+ auto *D = selectFirst(
+ "vd",
+ match(varDecl().bind("vd"), Ctx));
+
+ ASSERT_TRUE(D != nullptr);
+ auto TL = D->getTypeSourceInfo()->getTypeLoc().castAs();
+ ASSERT_EQ(SM.getFileOffset(TL.getStarLoc()), Example.point("star"));
+}
+
} // end namespace ast_matchers
} // end namespace clang
Index: clang/unittests/AST/CMakeLists.txt
===
--- clang/unittests/AST/CMakeLists.txt
+++ clang/unittests/AST/CMakeLists.txt
@@ -42,4 +42,5 @@
clangFrontend
clangSerialization
clangTooling
+ LLVMTestingSupport
)
Index: clang/lib/Sema/SemaType.cpp
===
--- clang/lib/Sema/SemaType.cpp
+++ clang/lib/Sema/SemaType.cpp
@@ -5840,7 +5840,7 @@
}
// Finally fill in MemberPointerLocInfo fields.
- TL.setStarLoc(Chunk.Loc);
+ TL.setStarLoc(SourceLocation::getFromRawEncoding(Chunk.Mem.StarLoc));
TL.setClassTInfo(ClsTInfo);
}
void VisitLValueReferenceTypeLoc(LValueReferenceTypeLoc TL) {
Index: clang/lib/Parse/ParseDecl.cpp
===
--- clang/lib/Parse/ParseDecl.cpp
+++ clang/lib/Parse/ParseDecl.cpp
@@ -5643,8 +5643,8 @@
return;
}
- SourceLocation Loc = ConsumeToken();
- D.SetRangeEnd(Loc);
+ SourceLocation StarLoc = ConsumeToken();
+ D.SetRangeEnd(StarLoc);
DeclSpec DS(AttrFactory);
ParseTypeQualifierListOpt(DS);
D.ExtendWithDeclSpec(DS);
@@ -5655,7 +5655,7 @@
// Sema will have to catch (syntactically invalid) pointers into global
// scope. It has to catch pointers into namespace scope anyway.
D.AddTypeInfo(DeclaratorChunk::getMemberPointer(
-SS, DS.getTypeQualifiers(), DS.getEndLoc()),
+SS, DS.getTypeQualifiers(), StarLoc, DS.getEndLoc()),
std::move(DS.getAttributes()),
/* Don't replace range end. */ SourceLocation());
return;
Index: clang/include/clang/Sema/DeclSpec.h
===
--- clang/include/clang/Sema/DeclSpec.h
+++ clang/include/clang/Sema/DeclSpec.h
@@ -1518,6 +1518,8 @@
struct MemberPointerTypeInfo {
/// The type qualifiers: const/volatile/restrict/__unaligned/_Atomic.
unsigned TypeQuals : 5;
+/// Location of the '*' token.
+unsigned StarLoc;
// CXXScopeSpec has a constructor, so it can't be a direct member.
// So we need some pointer-aligned storage and a bit of trickery.
alignas(CXXScopeSpec) char ScopeMem[sizeof(CXXScopeSpec)];
@@ -1660,11 +1662,13 @@
static DeclaratorChunk getMemberPointer(const CXXScopeSpec &SS,
unsigned TypeQuals,
- SourceLocation Loc) {
+ SourceLocation StarLoc,
+ SourceLocation EndLoc) {
DeclaratorChunk I;
I.Kind = MemberPointer;
I.Loc = SS.getBeginLoc();
-I.EndLoc= Loc;
+I.EndLoc= EndLoc;
+I.Mem.StarLoc = StarLoc.getRawEncoding();
I.Mem.TypeQuals = TypeQuals;
new (I.Mem.ScopeMem) CXXScopeSpec(SS);
return I;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76061: [Sema] Fix location of star ('*') inside MemberPointerTypeLoc
hlopko updated this revision to Diff 250143.
hlopko added a comment.
Rearrange
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76061/new/
https://reviews.llvm.org/D76061
Files:
clang/include/clang/Sema/DeclSpec.h
clang/lib/Parse/ParseDecl.cpp
clang/lib/Sema/SemaType.cpp
clang/unittests/AST/CMakeLists.txt
clang/unittests/AST/SourceLocationTest.cpp
Index: clang/unittests/AST/SourceLocationTest.cpp
===
--- clang/unittests/AST/SourceLocationTest.cpp
+++ clang/unittests/AST/SourceLocationTest.cpp
@@ -15,11 +15,12 @@
//
//===--===//
-#include "clang/AST/ASTContext.h"
#include "MatchVerifier.h"
+#include "clang/AST/ASTContext.h"
#include "clang/ASTMatchers/ASTMatchFinder.h"
#include "clang/ASTMatchers/ASTMatchers.h"
#include "clang/Tooling/Tooling.h"
+#include "llvm/Testing/Support/Annotations.h"
#include "gtest/gtest.h"
namespace clang {
@@ -842,5 +843,22 @@
Language::Lang_CXX11));
}
+TEST(Decl, MemberPointerStarLoc) {
+ llvm::Annotations Example(R"cpp(
+struct X {};
+int X::$star^* a;
+ )cpp");
+
+ auto AST = tooling::buildASTFromCode(Example.code());
+ SourceManager &SM = AST->getSourceManager();
+ auto &Ctx = AST->getASTContext();
+
+ auto *D = selectFirst("vd", match(varDecl().bind("vd"), Ctx));
+ ASSERT_TRUE(D != nullptr);
+
+ auto TL = D->getTypeSourceInfo()->getTypeLoc().castAs();
+ ASSERT_EQ(SM.getFileOffset(TL.getStarLoc()), Example.point("star"));
+}
+
} // end namespace ast_matchers
} // end namespace clang
Index: clang/unittests/AST/CMakeLists.txt
===
--- clang/unittests/AST/CMakeLists.txt
+++ clang/unittests/AST/CMakeLists.txt
@@ -42,4 +42,5 @@
clangFrontend
clangSerialization
clangTooling
+ LLVMTestingSupport
)
Index: clang/lib/Sema/SemaType.cpp
===
--- clang/lib/Sema/SemaType.cpp
+++ clang/lib/Sema/SemaType.cpp
@@ -5840,7 +5840,7 @@
}
// Finally fill in MemberPointerLocInfo fields.
- TL.setStarLoc(Chunk.Loc);
+ TL.setStarLoc(SourceLocation::getFromRawEncoding(Chunk.Mem.StarLoc));
TL.setClassTInfo(ClsTInfo);
}
void VisitLValueReferenceTypeLoc(LValueReferenceTypeLoc TL) {
Index: clang/lib/Parse/ParseDecl.cpp
===
--- clang/lib/Parse/ParseDecl.cpp
+++ clang/lib/Parse/ParseDecl.cpp
@@ -5643,8 +5643,8 @@
return;
}
- SourceLocation Loc = ConsumeToken();
- D.SetRangeEnd(Loc);
+ SourceLocation StarLoc = ConsumeToken();
+ D.SetRangeEnd(StarLoc);
DeclSpec DS(AttrFactory);
ParseTypeQualifierListOpt(DS);
D.ExtendWithDeclSpec(DS);
@@ -5655,7 +5655,7 @@
// Sema will have to catch (syntactically invalid) pointers into global
// scope. It has to catch pointers into namespace scope anyway.
D.AddTypeInfo(DeclaratorChunk::getMemberPointer(
-SS, DS.getTypeQualifiers(), DS.getEndLoc()),
+SS, DS.getTypeQualifiers(), StarLoc, DS.getEndLoc()),
std::move(DS.getAttributes()),
/* Don't replace range end. */ SourceLocation());
return;
Index: clang/include/clang/Sema/DeclSpec.h
===
--- clang/include/clang/Sema/DeclSpec.h
+++ clang/include/clang/Sema/DeclSpec.h
@@ -1518,6 +1518,8 @@
struct MemberPointerTypeInfo {
/// The type qualifiers: const/volatile/restrict/__unaligned/_Atomic.
unsigned TypeQuals : 5;
+/// Location of the '*' token.
+unsigned StarLoc;
// CXXScopeSpec has a constructor, so it can't be a direct member.
// So we need some pointer-aligned storage and a bit of trickery.
alignas(CXXScopeSpec) char ScopeMem[sizeof(CXXScopeSpec)];
@@ -1660,11 +1662,13 @@
static DeclaratorChunk getMemberPointer(const CXXScopeSpec &SS,
unsigned TypeQuals,
- SourceLocation Loc) {
+ SourceLocation StarLoc,
+ SourceLocation EndLoc) {
DeclaratorChunk I;
I.Kind = MemberPointer;
I.Loc = SS.getBeginLoc();
-I.EndLoc= Loc;
+I.EndLoc= EndLoc;
+I.Mem.StarLoc = StarLoc.getRawEncoding();
I.Mem.TypeQuals = TypeQuals;
new (I.Mem.ScopeMem) CXXScopeSpec(SS);
return I;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76120: Refactor SourceLocationTest to `using namespace`
hlopko updated this revision to Diff 250152.
hlopko added a comment.
Wrap the test with an anonymous namespace, remove fully qualified name to
Lang_*.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76120/new/
https://reviews.llvm.org/D76120
Files:
clang/unittests/AST/SourceLocationTest.cpp
Index: clang/unittests/AST/SourceLocationTest.cpp
===
--- clang/unittests/AST/SourceLocationTest.cpp
+++ clang/unittests/AST/SourceLocationTest.cpp
@@ -23,8 +23,10 @@
#include "llvm/Testing/Support/Annotations.h"
#include "gtest/gtest.h"
-namespace clang {
-namespace ast_matchers {
+using namespace clang;
+using namespace clang::ast_matchers;
+
+namespace {
// FIXME: Pull the *Verifier tests into their own test file.
@@ -643,10 +645,8 @@
TEST(FunctionDecl, FunctionDeclWithNoExceptSpecification) {
RangeVerifier Verifier;
Verifier.expectRange(1, 1, 1, 24);
- EXPECT_TRUE(Verifier.match(
- "void f() noexcept(false);\n",
- functionDecl(),
- Language::Lang_CXX11));
+ EXPECT_TRUE(Verifier.match("void f() noexcept(false);\n", functionDecl(),
+ Lang_CXX11));
}
class FunctionDeclParametersRangeVerifier : public RangeVerifier
{
@@ -768,12 +768,10 @@
TEST(CXXMethodDecl, CXXMethodDeclWithNoExceptSpecification) {
RangeVerifier Verifier;
Verifier.expectRange(2, 1, 2, 24);
- EXPECT_TRUE(Verifier.match(
- "class A {\n"
- "void f() noexcept(false);\n"
- "};\n",
- functionDecl(),
- Language::Lang_CXX11));
+ EXPECT_TRUE(Verifier.match("class A {\n"
+ "void f() noexcept(false);\n"
+ "};\n",
+ functionDecl(), Lang_CXX11));
}
class ExceptionSpecRangeVerifier : public RangeVerifier {
@@ -816,19 +814,19 @@
std::vector Args;
Args.push_back("-fms-extensions");
EXPECT_TRUE(Verifier.match("void f() throw(...);\n", loc(functionType()),
- Args, Language::Lang_CXX));
+ Args, Lang_CXX));
Verifier.expectRange(1, 10, 1, 10);
- EXPECT_TRUE(Verifier.match("void f() noexcept;\n", loc(functionType()),
- Language::Lang_CXX11));
+ EXPECT_TRUE(
+ Verifier.match("void f() noexcept;\n", loc(functionType()), Lang_CXX11));
Verifier.expectRange(1, 10, 1, 24);
EXPECT_TRUE(Verifier.match("void f() noexcept(false);\n",
loc(functionType()),
- Language::Lang_CXX11));
+ Lang_CXX11));
Verifier.expectRange(1, 10, 1, 32);
EXPECT_TRUE(Verifier.match("void f() noexcept(noexcept(1+1));\n",
- loc(functionType()), Language::Lang_CXX11));
+ loc(functionType()), Lang_CXX11));
ParmVarExceptionSpecRangeVerifier Verifier2;
Verifier2.expectRange(1, 25, 1, 31);
@@ -840,7 +838,7 @@
EXPECT_TRUE(Verifier2.match("void g(void (*fp)(void) noexcept(true));\n",
parmVarDecl(hasType(pointerType(pointee(
parenType(innerType(functionType())),
- Language::Lang_CXX11));
+ Lang_CXX11));
}
TEST(Decl, MemberPointerStarLoc) {
@@ -860,5 +858,4 @@
ASSERT_EQ(SM.getFileOffset(TL.getStarLoc()), Example.point("star"));
}
-} // end namespace ast_matchers
-} // end namespace clang
+} // end namespace
Index: clang/unittests/AST/SourceLocationTest.cpp
===
--- clang/unittests/AST/SourceLocationTest.cpp
+++ clang/unittests/AST/SourceLocationTest.cpp
@@ -23,8 +23,10 @@
#include "llvm/Testing/Support/Annotations.h"
#include "gtest/gtest.h"
-namespace clang {
-namespace ast_matchers {
+using namespace clang;
+using namespace clang::ast_matchers;
+
+namespace {
// FIXME: Pull the *Verifier tests into their own test file.
@@ -643,10 +645,8 @@
TEST(FunctionDecl, FunctionDeclWithNoExceptSpecification) {
RangeVerifier Verifier;
Verifier.expectRange(1, 1, 1, 24);
- EXPECT_TRUE(Verifier.match(
- "void f() noexcept(false);\n",
- functionDecl(),
- Language::Lang_CXX11));
+ EXPECT_TRUE(Verifier.match("void f() noexcept(false);\n", functionDecl(),
+ Lang_CXX11));
}
class FunctionDeclParametersRangeVerifier : public RangeVerifier {
@@ -768,12 +768,10 @@
TEST(CXXMethodDecl, CXXMethodDeclWithNoExceptSpecification) {
RangeVerifier Verifier;
Verifier.expectRange(2, 1, 2, 24);
- EXPECT_TRUE(Verifier.match(
- "class A {\n"
- "void f() noexcept(false);\n"
- "};\n",
- functionDecl(),
- Language::Lang_CXX11));
+ EXPECT_TRUE(Verifier.match("class A {\n"
+ "void f() noexcept(false);\n"
+ "};\n",
+ functionDecl(), Lang_CX
[PATCH] D76121: Modernize DeclTest
hlopko created this revision.
hlopko added a reviewer: gribozavr.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
This patch removes a call to the old ASTUnit::findFileRegionDecls and
replaces it with ast matchers.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D76121
Files:
clang/unittests/AST/DeclTest.cpp
Index: clang/unittests/AST/DeclTest.cpp
===
--- clang/unittests/AST/DeclTest.cpp
+++ clang/unittests/AST/DeclTest.cpp
@@ -13,10 +13,11 @@
#include "clang/AST/ASTContext.h"
#include "clang/AST/Mangle.h"
#include "clang/ASTMatchers/ASTMatchFinder.h"
+#include "clang/ASTMatchers/ASTMatchers.h"
#include "clang/Basic/LLVM.h"
#include "clang/Tooling/Tooling.h"
-#include "gtest/gtest.h"
#include "llvm/IR/DataLayout.h"
+#include "gtest/gtest.h"
using namespace clang::ast_matchers;
using namespace clang::tooling;
@@ -76,14 +77,9 @@
assert(Ctx.getTargetInfo().getDataLayout().getGlobalPrefix() &&
"Expected target to have a global prefix");
DiagnosticsEngine &Diags = AST->getDiagnostics();
- SourceManager &SM = AST->getSourceManager();
- FileID MainFileID = SM.getMainFileID();
- // Find the method decls within the AST.
- SmallVector Decls;
- AST->findFileRegionDecls(MainFileID, Code.find('{'), 0, Decls);
- ASSERT_TRUE(Decls.size() == 1);
- CXXRecordDecl *DeclS = cast(Decls[0]);
+ auto *DeclS =
+ selectFirst("d", match(cxxRecordDecl().bind("d"), Ctx));
NamedDecl *DeclF = *DeclS->method_begin();
NamedDecl *DeclG = *(++DeclS->method_begin());
Index: clang/unittests/AST/DeclTest.cpp
===
--- clang/unittests/AST/DeclTest.cpp
+++ clang/unittests/AST/DeclTest.cpp
@@ -13,10 +13,11 @@
#include "clang/AST/ASTContext.h"
#include "clang/AST/Mangle.h"
#include "clang/ASTMatchers/ASTMatchFinder.h"
+#include "clang/ASTMatchers/ASTMatchers.h"
#include "clang/Basic/LLVM.h"
#include "clang/Tooling/Tooling.h"
-#include "gtest/gtest.h"
#include "llvm/IR/DataLayout.h"
+#include "gtest/gtest.h"
using namespace clang::ast_matchers;
using namespace clang::tooling;
@@ -76,14 +77,9 @@
assert(Ctx.getTargetInfo().getDataLayout().getGlobalPrefix() &&
"Expected target to have a global prefix");
DiagnosticsEngine &Diags = AST->getDiagnostics();
- SourceManager &SM = AST->getSourceManager();
- FileID MainFileID = SM.getMainFileID();
- // Find the method decls within the AST.
- SmallVector Decls;
- AST->findFileRegionDecls(MainFileID, Code.find('{'), 0, Decls);
- ASSERT_TRUE(Decls.size() == 1);
- CXXRecordDecl *DeclS = cast(Decls[0]);
+ auto *DeclS =
+ selectFirst("d", match(cxxRecordDecl().bind("d"), Ctx));
NamedDecl *DeclF = *DeclS->method_begin();
NamedDecl *DeclG = *(++DeclS->method_begin());
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76120: Refactor SourceLocationTest to `using namespace`
hlopko updated this revision to Diff 250158.
hlopko added a comment.
Rebase
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76120/new/
https://reviews.llvm.org/D76120
Files:
clang/unittests/AST/DeclTest.cpp
Index: clang/unittests/AST/DeclTest.cpp
===
--- clang/unittests/AST/DeclTest.cpp
+++ clang/unittests/AST/DeclTest.cpp
@@ -13,6 +13,7 @@
#include "clang/AST/ASTContext.h"
#include "clang/AST/Mangle.h"
#include "clang/ASTMatchers/ASTMatchFinder.h"
+#include "clang/ASTMatchers/ASTMatchers.h"
#include "clang/Basic/LLVM.h"
#include "clang/Basic/TargetInfo.h"
#include "clang/Tooling/Tooling.h"
@@ -77,14 +78,9 @@
assert(Ctx.getTargetInfo().getDataLayout().getGlobalPrefix() &&
"Expected target to have a global prefix");
DiagnosticsEngine &Diags = AST->getDiagnostics();
- SourceManager &SM = AST->getSourceManager();
- FileID MainFileID = SM.getMainFileID();
- // Find the method decls within the AST.
- SmallVector Decls;
- AST->findFileRegionDecls(MainFileID, Code.find('{'), 0, Decls);
- ASSERT_TRUE(Decls.size() == 1);
- CXXRecordDecl *DeclS = cast(Decls[0]);
+ auto *DeclS =
+ selectFirst("d", match(cxxRecordDecl().bind("d"), Ctx));
NamedDecl *DeclF = *DeclS->method_begin();
NamedDecl *DeclG = *(++DeclS->method_begin());
Index: clang/unittests/AST/DeclTest.cpp
===
--- clang/unittests/AST/DeclTest.cpp
+++ clang/unittests/AST/DeclTest.cpp
@@ -13,6 +13,7 @@
#include "clang/AST/ASTContext.h"
#include "clang/AST/Mangle.h"
#include "clang/ASTMatchers/ASTMatchFinder.h"
+#include "clang/ASTMatchers/ASTMatchers.h"
#include "clang/Basic/LLVM.h"
#include "clang/Basic/TargetInfo.h"
#include "clang/Tooling/Tooling.h"
@@ -77,14 +78,9 @@
assert(Ctx.getTargetInfo().getDataLayout().getGlobalPrefix() &&
"Expected target to have a global prefix");
DiagnosticsEngine &Diags = AST->getDiagnostics();
- SourceManager &SM = AST->getSourceManager();
- FileID MainFileID = SM.getMainFileID();
- // Find the method decls within the AST.
- SmallVector Decls;
- AST->findFileRegionDecls(MainFileID, Code.find('{'), 0, Decls);
- ASSERT_TRUE(Decls.size() == 1);
- CXXRecordDecl *DeclS = cast(Decls[0]);
+ auto *DeclS =
+ selectFirst("d", match(cxxRecordDecl().bind("d"), Ctx));
NamedDecl *DeclF = *DeclS->method_begin();
NamedDecl *DeclG = *(++DeclS->method_begin());
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76121: Modernize DeclTest
hlopko updated this revision to Diff 250159.
hlopko added a comment.
Rebase
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76121/new/
https://reviews.llvm.org/D76121
Files:
clang/unittests/AST/DeclTest.cpp
Index: clang/unittests/AST/DeclTest.cpp
===
--- clang/unittests/AST/DeclTest.cpp
+++ clang/unittests/AST/DeclTest.cpp
@@ -13,6 +13,7 @@
#include "clang/AST/ASTContext.h"
#include "clang/AST/Mangle.h"
#include "clang/ASTMatchers/ASTMatchFinder.h"
+#include "clang/ASTMatchers/ASTMatchers.h"
#include "clang/Basic/LLVM.h"
#include "clang/Basic/TargetInfo.h"
#include "clang/Tooling/Tooling.h"
@@ -77,14 +78,9 @@
assert(Ctx.getTargetInfo().getDataLayout().getGlobalPrefix() &&
"Expected target to have a global prefix");
DiagnosticsEngine &Diags = AST->getDiagnostics();
- SourceManager &SM = AST->getSourceManager();
- FileID MainFileID = SM.getMainFileID();
- // Find the method decls within the AST.
- SmallVector Decls;
- AST->findFileRegionDecls(MainFileID, Code.find('{'), 0, Decls);
- ASSERT_TRUE(Decls.size() == 1);
- CXXRecordDecl *DeclS = cast(Decls[0]);
+ auto *DeclS =
+ selectFirst("d", match(cxxRecordDecl().bind("d"), Ctx));
NamedDecl *DeclF = *DeclS->method_begin();
NamedDecl *DeclG = *(++DeclS->method_begin());
Index: clang/unittests/AST/DeclTest.cpp
===
--- clang/unittests/AST/DeclTest.cpp
+++ clang/unittests/AST/DeclTest.cpp
@@ -13,6 +13,7 @@
#include "clang/AST/ASTContext.h"
#include "clang/AST/Mangle.h"
#include "clang/ASTMatchers/ASTMatchFinder.h"
+#include "clang/ASTMatchers/ASTMatchers.h"
#include "clang/Basic/LLVM.h"
#include "clang/Basic/TargetInfo.h"
#include "clang/Tooling/Tooling.h"
@@ -77,14 +78,9 @@
assert(Ctx.getTargetInfo().getDataLayout().getGlobalPrefix() &&
"Expected target to have a global prefix");
DiagnosticsEngine &Diags = AST->getDiagnostics();
- SourceManager &SM = AST->getSourceManager();
- FileID MainFileID = SM.getMainFileID();
- // Find the method decls within the AST.
- SmallVector Decls;
- AST->findFileRegionDecls(MainFileID, Code.find('{'), 0, Decls);
- ASSERT_TRUE(Decls.size() == 1);
- CXXRecordDecl *DeclS = cast(Decls[0]);
+ auto *DeclS =
+ selectFirst("d", match(cxxRecordDecl().bind("d"), Ctx));
NamedDecl *DeclF = *DeclS->method_begin();
NamedDecl *DeclG = *(++DeclS->method_begin());
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76061: [Sema] Fix location of star ('*') inside MemberPointerTypeLoc
hlopko updated this revision to Diff 250161.
hlopko added a comment.
Rebase
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76061/new/
https://reviews.llvm.org/D76061
Files:
clang/include/clang/Sema/DeclSpec.h
clang/lib/Parse/ParseDecl.cpp
clang/lib/Sema/SemaType.cpp
clang/unittests/AST/CMakeLists.txt
clang/unittests/AST/SourceLocationTest.cpp
Index: clang/unittests/AST/SourceLocationTest.cpp
===
--- clang/unittests/AST/SourceLocationTest.cpp
+++ clang/unittests/AST/SourceLocationTest.cpp
@@ -15,11 +15,12 @@
//
//===--===//
-#include "clang/AST/ASTContext.h"
#include "MatchVerifier.h"
+#include "clang/AST/ASTContext.h"
#include "clang/ASTMatchers/ASTMatchFinder.h"
#include "clang/ASTMatchers/ASTMatchers.h"
#include "clang/Tooling/Tooling.h"
+#include "llvm/Testing/Support/Annotations.h"
#include "gtest/gtest.h"
namespace clang {
@@ -842,5 +843,23 @@
Language::Lang_CXX11));
}
+TEST(Decl, MemberPointerStarLoc) {
+ llvm::Annotations Example(R"cpp(
+struct X {};
+int X::$star^* a;
+ )cpp");
+
+ auto AST = tooling::buildASTFromCode(Example.code());
+ SourceManager &SM = AST->getSourceManager();
+ auto &Ctx = AST->getASTContext();
+
+ auto *VD = selectFirst("vd", match(varDecl().bind("vd"), Ctx));
+ ASSERT_TRUE(VD != nullptr);
+
+ auto TL =
+ VD->getTypeSourceInfo()->getTypeLoc().castAs();
+ ASSERT_EQ(SM.getFileOffset(TL.getStarLoc()), Example.point("star"));
+}
+
} // end namespace ast_matchers
} // end namespace clang
Index: clang/unittests/AST/CMakeLists.txt
===
--- clang/unittests/AST/CMakeLists.txt
+++ clang/unittests/AST/CMakeLists.txt
@@ -42,4 +42,5 @@
clangFrontend
clangSerialization
clangTooling
+ LLVMTestingSupport
)
Index: clang/lib/Sema/SemaType.cpp
===
--- clang/lib/Sema/SemaType.cpp
+++ clang/lib/Sema/SemaType.cpp
@@ -5840,7 +5840,7 @@
}
// Finally fill in MemberPointerLocInfo fields.
- TL.setStarLoc(Chunk.Loc);
+ TL.setStarLoc(SourceLocation::getFromRawEncoding(Chunk.Mem.StarLoc));
TL.setClassTInfo(ClsTInfo);
}
void VisitLValueReferenceTypeLoc(LValueReferenceTypeLoc TL) {
Index: clang/lib/Parse/ParseDecl.cpp
===
--- clang/lib/Parse/ParseDecl.cpp
+++ clang/lib/Parse/ParseDecl.cpp
@@ -5643,8 +5643,8 @@
return;
}
- SourceLocation Loc = ConsumeToken();
- D.SetRangeEnd(Loc);
+ SourceLocation StarLoc = ConsumeToken();
+ D.SetRangeEnd(StarLoc);
DeclSpec DS(AttrFactory);
ParseTypeQualifierListOpt(DS);
D.ExtendWithDeclSpec(DS);
@@ -5655,7 +5655,7 @@
// Sema will have to catch (syntactically invalid) pointers into global
// scope. It has to catch pointers into namespace scope anyway.
D.AddTypeInfo(DeclaratorChunk::getMemberPointer(
-SS, DS.getTypeQualifiers(), DS.getEndLoc()),
+SS, DS.getTypeQualifiers(), StarLoc, DS.getEndLoc()),
std::move(DS.getAttributes()),
/* Don't replace range end. */ SourceLocation());
return;
Index: clang/include/clang/Sema/DeclSpec.h
===
--- clang/include/clang/Sema/DeclSpec.h
+++ clang/include/clang/Sema/DeclSpec.h
@@ -1518,6 +1518,8 @@
struct MemberPointerTypeInfo {
/// The type qualifiers: const/volatile/restrict/__unaligned/_Atomic.
unsigned TypeQuals : 5;
+/// Location of the '*' token.
+unsigned StarLoc;
// CXXScopeSpec has a constructor, so it can't be a direct member.
// So we need some pointer-aligned storage and a bit of trickery.
alignas(CXXScopeSpec) char ScopeMem[sizeof(CXXScopeSpec)];
@@ -1660,11 +1662,13 @@
static DeclaratorChunk getMemberPointer(const CXXScopeSpec &SS,
unsigned TypeQuals,
- SourceLocation Loc) {
+ SourceLocation StarLoc,
+ SourceLocation EndLoc) {
DeclaratorChunk I;
I.Kind = MemberPointer;
I.Loc = SS.getBeginLoc();
-I.EndLoc= Loc;
+I.EndLoc = EndLoc;
+I.Mem.StarLoc = StarLoc.getRawEncoding();
I.Mem.TypeQuals = TypeQuals;
new (I.Mem.ScopeMem) CXXScopeSpec(SS);
return I;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76061: [Sema] Fix location of star ('*') inside MemberPointerTypeLoc
hlopko updated this revision to Diff 250160.
hlopko added a comment.
Rename D->VD, rebase master.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76061/new/
https://reviews.llvm.org/D76061
Files:
clang/include/clang/Sema/DeclSpec.h
clang/lib/Parse/ParseDecl.cpp
clang/lib/Sema/SemaType.cpp
clang/unittests/AST/CMakeLists.txt
clang/unittests/AST/SourceLocationTest.cpp
Index: clang/unittests/AST/SourceLocationTest.cpp
===
--- clang/unittests/AST/SourceLocationTest.cpp
+++ clang/unittests/AST/SourceLocationTest.cpp
@@ -15,11 +15,12 @@
//
//===--===//
-#include "clang/AST/ASTContext.h"
#include "MatchVerifier.h"
+#include "clang/AST/ASTContext.h"
#include "clang/ASTMatchers/ASTMatchFinder.h"
#include "clang/ASTMatchers/ASTMatchers.h"
#include "clang/Tooling/Tooling.h"
+#include "llvm/Testing/Support/Annotations.h"
#include "gtest/gtest.h"
namespace clang {
@@ -842,5 +843,23 @@
Language::Lang_CXX11));
}
+TEST(Decl, MemberPointerStarLoc) {
+ llvm::Annotations Example(R"cpp(
+struct X {};
+int X::$star^* a;
+ )cpp");
+
+ auto AST = tooling::buildASTFromCode(Example.code());
+ SourceManager &SM = AST->getSourceManager();
+ auto &Ctx = AST->getASTContext();
+
+ auto *VD = selectFirst("vd", match(varDecl().bind("vd"), Ctx));
+ ASSERT_TRUE(VD != nullptr);
+
+ auto TL =
+ VD->getTypeSourceInfo()->getTypeLoc().castAs();
+ ASSERT_EQ(SM.getFileOffset(TL.getStarLoc()), Example.point("star"));
+}
+
} // end namespace ast_matchers
} // end namespace clang
Index: clang/unittests/AST/CMakeLists.txt
===
--- clang/unittests/AST/CMakeLists.txt
+++ clang/unittests/AST/CMakeLists.txt
@@ -42,4 +42,5 @@
clangFrontend
clangSerialization
clangTooling
+ LLVMTestingSupport
)
Index: clang/lib/Sema/SemaType.cpp
===
--- clang/lib/Sema/SemaType.cpp
+++ clang/lib/Sema/SemaType.cpp
@@ -5840,7 +5840,7 @@
}
// Finally fill in MemberPointerLocInfo fields.
- TL.setStarLoc(Chunk.Loc);
+ TL.setStarLoc(SourceLocation::getFromRawEncoding(Chunk.Mem.StarLoc));
TL.setClassTInfo(ClsTInfo);
}
void VisitLValueReferenceTypeLoc(LValueReferenceTypeLoc TL) {
Index: clang/lib/Parse/ParseDecl.cpp
===
--- clang/lib/Parse/ParseDecl.cpp
+++ clang/lib/Parse/ParseDecl.cpp
@@ -5643,8 +5643,8 @@
return;
}
- SourceLocation Loc = ConsumeToken();
- D.SetRangeEnd(Loc);
+ SourceLocation StarLoc = ConsumeToken();
+ D.SetRangeEnd(StarLoc);
DeclSpec DS(AttrFactory);
ParseTypeQualifierListOpt(DS);
D.ExtendWithDeclSpec(DS);
@@ -5655,7 +5655,7 @@
// Sema will have to catch (syntactically invalid) pointers into global
// scope. It has to catch pointers into namespace scope anyway.
D.AddTypeInfo(DeclaratorChunk::getMemberPointer(
-SS, DS.getTypeQualifiers(), DS.getEndLoc()),
+SS, DS.getTypeQualifiers(), StarLoc, DS.getEndLoc()),
std::move(DS.getAttributes()),
/* Don't replace range end. */ SourceLocation());
return;
Index: clang/include/clang/Sema/DeclSpec.h
===
--- clang/include/clang/Sema/DeclSpec.h
+++ clang/include/clang/Sema/DeclSpec.h
@@ -1518,6 +1518,8 @@
struct MemberPointerTypeInfo {
/// The type qualifiers: const/volatile/restrict/__unaligned/_Atomic.
unsigned TypeQuals : 5;
+/// Location of the '*' token.
+unsigned StarLoc;
// CXXScopeSpec has a constructor, so it can't be a direct member.
// So we need some pointer-aligned storage and a bit of trickery.
alignas(CXXScopeSpec) char ScopeMem[sizeof(CXXScopeSpec)];
@@ -1660,11 +1662,13 @@
static DeclaratorChunk getMemberPointer(const CXXScopeSpec &SS,
unsigned TypeQuals,
- SourceLocation Loc) {
+ SourceLocation StarLoc,
+ SourceLocation EndLoc) {
DeclaratorChunk I;
I.Kind = MemberPointer;
I.Loc = SS.getBeginLoc();
-I.EndLoc= Loc;
+I.EndLoc = EndLoc;
+I.Mem.StarLoc = StarLoc.getRawEncoding();
I.Mem.TypeQuals = TypeQuals;
new (I.Mem.ScopeMem) CXXScopeSpec(SS);
return I;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76120: Refactor SourceLocationTest to `using namespace`
hlopko updated this revision to Diff 250163.
hlopko added a comment.
Remove extra blank line
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76120/new/
https://reviews.llvm.org/D76120
Files:
clang/unittests/AST/SourceLocationTest.cpp
Index: clang/unittests/AST/SourceLocationTest.cpp
===
--- clang/unittests/AST/SourceLocationTest.cpp
+++ clang/unittests/AST/SourceLocationTest.cpp
@@ -22,8 +22,10 @@
#include "clang/Tooling/Tooling.h"
#include "gtest/gtest.h"
-namespace clang {
-namespace ast_matchers {
+using namespace clang;
+using namespace clang::ast_matchers;
+
+namespace {
// FIXME: Pull the *Verifier tests into their own test file.
@@ -642,10 +644,8 @@
TEST(FunctionDecl, FunctionDeclWithNoExceptSpecification) {
RangeVerifier Verifier;
Verifier.expectRange(1, 1, 1, 24);
- EXPECT_TRUE(Verifier.match(
- "void f() noexcept(false);\n",
- functionDecl(),
- Language::Lang_CXX11));
+ EXPECT_TRUE(Verifier.match("void f() noexcept(false);\n", functionDecl(),
+ Lang_CXX11));
}
class FunctionDeclParametersRangeVerifier : public RangeVerifier
{
@@ -767,12 +767,10 @@
TEST(CXXMethodDecl, CXXMethodDeclWithNoExceptSpecification) {
RangeVerifier Verifier;
Verifier.expectRange(2, 1, 2, 24);
- EXPECT_TRUE(Verifier.match(
- "class A {\n"
- "void f() noexcept(false);\n"
- "};\n",
- functionDecl(),
- Language::Lang_CXX11));
+ EXPECT_TRUE(Verifier.match("class A {\n"
+ "void f() noexcept(false);\n"
+ "};\n",
+ functionDecl(), Lang_CXX11));
}
class ExceptionSpecRangeVerifier : public RangeVerifier {
@@ -815,19 +813,19 @@
std::vector Args;
Args.push_back("-fms-extensions");
EXPECT_TRUE(Verifier.match("void f() throw(...);\n", loc(functionType()),
- Args, Language::Lang_CXX));
+ Args, Lang_CXX));
Verifier.expectRange(1, 10, 1, 10);
- EXPECT_TRUE(Verifier.match("void f() noexcept;\n", loc(functionType()),
- Language::Lang_CXX11));
+ EXPECT_TRUE(
+ Verifier.match("void f() noexcept;\n", loc(functionType()), Lang_CXX11));
Verifier.expectRange(1, 10, 1, 24);
EXPECT_TRUE(Verifier.match("void f() noexcept(false);\n",
loc(functionType()),
- Language::Lang_CXX11));
+ Lang_CXX11));
Verifier.expectRange(1, 10, 1, 32);
EXPECT_TRUE(Verifier.match("void f() noexcept(noexcept(1+1));\n",
- loc(functionType()), Language::Lang_CXX11));
+ loc(functionType()), Lang_CXX11));
ParmVarExceptionSpecRangeVerifier Verifier2;
Verifier2.expectRange(1, 25, 1, 31);
@@ -839,8 +837,7 @@
EXPECT_TRUE(Verifier2.match("void g(void (*fp)(void) noexcept(true));\n",
parmVarDecl(hasType(pointerType(pointee(
parenType(innerType(functionType())),
- Language::Lang_CXX11));
+ Lang_CXX11));
}
-} // end namespace ast_matchers
-} // end namespace clang
+} // end namespace
Index: clang/unittests/AST/SourceLocationTest.cpp
===
--- clang/unittests/AST/SourceLocationTest.cpp
+++ clang/unittests/AST/SourceLocationTest.cpp
@@ -22,8 +22,10 @@
#include "clang/Tooling/Tooling.h"
#include "gtest/gtest.h"
-namespace clang {
-namespace ast_matchers {
+using namespace clang;
+using namespace clang::ast_matchers;
+
+namespace {
// FIXME: Pull the *Verifier tests into their own test file.
@@ -642,10 +644,8 @@
TEST(FunctionDecl, FunctionDeclWithNoExceptSpecification) {
RangeVerifier Verifier;
Verifier.expectRange(1, 1, 1, 24);
- EXPECT_TRUE(Verifier.match(
- "void f() noexcept(false);\n",
- functionDecl(),
- Language::Lang_CXX11));
+ EXPECT_TRUE(Verifier.match("void f() noexcept(false);\n", functionDecl(),
+ Lang_CXX11));
}
class FunctionDeclParametersRangeVerifier : public RangeVerifier {
@@ -767,12 +767,10 @@
TEST(CXXMethodDecl, CXXMethodDeclWithNoExceptSpecification) {
RangeVerifier Verifier;
Verifier.expectRange(2, 1, 2, 24);
- EXPECT_TRUE(Verifier.match(
- "class A {\n"
- "void f() noexcept(false);\n"
- "};\n",
- functionDecl(),
- Language::Lang_CXX11));
+ EXPECT_TRUE(Verifier.match("class A {\n"
+ "void f() noexcept(false);\n"
+ "};\n",
+ functionDecl(), Lang_CXX11));
}
class ExceptionSpecRangeVerifier : public RangeVerifier {
@@ -815,19 +813,19 @@
std::vector Args;
Args.push_back("-fms-extensions");
EXPECT_TRUE(Verifier.match("void f() throw(...);\n", loc(fu
[PATCH] D76120: Refactor SourceLocationTest to `using namespace`
hlopko updated this revision to Diff 250162.
hlopko added a comment.
Rebase
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76120/new/
https://reviews.llvm.org/D76120
Files:
clang/unittests/AST/SourceLocationTest.cpp
Index: clang/unittests/AST/SourceLocationTest.cpp
===
--- clang/unittests/AST/SourceLocationTest.cpp
+++ clang/unittests/AST/SourceLocationTest.cpp
@@ -22,8 +22,10 @@
#include "clang/Tooling/Tooling.h"
#include "gtest/gtest.h"
-namespace clang {
-namespace ast_matchers {
+using namespace clang;
+using namespace clang::ast_matchers;
+
+namespace {
// FIXME: Pull the *Verifier tests into their own test file.
@@ -642,10 +644,8 @@
TEST(FunctionDecl, FunctionDeclWithNoExceptSpecification) {
RangeVerifier Verifier;
Verifier.expectRange(1, 1, 1, 24);
- EXPECT_TRUE(Verifier.match(
- "void f() noexcept(false);\n",
- functionDecl(),
- Language::Lang_CXX11));
+ EXPECT_TRUE(Verifier.match("void f() noexcept(false);\n", functionDecl(),
+ Lang_CXX11));
}
class FunctionDeclParametersRangeVerifier : public RangeVerifier
{
@@ -767,12 +767,10 @@
TEST(CXXMethodDecl, CXXMethodDeclWithNoExceptSpecification) {
RangeVerifier Verifier;
Verifier.expectRange(2, 1, 2, 24);
- EXPECT_TRUE(Verifier.match(
- "class A {\n"
- "void f() noexcept(false);\n"
- "};\n",
- functionDecl(),
- Language::Lang_CXX11));
+ EXPECT_TRUE(Verifier.match("class A {\n"
+ "void f() noexcept(false);\n"
+ "};\n",
+ functionDecl(), Lang_CXX11));
}
class ExceptionSpecRangeVerifier : public RangeVerifier {
@@ -815,19 +813,19 @@
std::vector Args;
Args.push_back("-fms-extensions");
EXPECT_TRUE(Verifier.match("void f() throw(...);\n", loc(functionType()),
- Args, Language::Lang_CXX));
+ Args, Lang_CXX));
Verifier.expectRange(1, 10, 1, 10);
- EXPECT_TRUE(Verifier.match("void f() noexcept;\n", loc(functionType()),
- Language::Lang_CXX11));
+ EXPECT_TRUE(
+ Verifier.match("void f() noexcept;\n", loc(functionType()), Lang_CXX11));
Verifier.expectRange(1, 10, 1, 24);
EXPECT_TRUE(Verifier.match("void f() noexcept(false);\n",
loc(functionType()),
- Language::Lang_CXX11));
+ Lang_CXX11));
Verifier.expectRange(1, 10, 1, 32);
EXPECT_TRUE(Verifier.match("void f() noexcept(noexcept(1+1));\n",
- loc(functionType()), Language::Lang_CXX11));
+ loc(functionType()), Lang_CXX11));
ParmVarExceptionSpecRangeVerifier Verifier2;
Verifier2.expectRange(1, 25, 1, 31);
@@ -839,8 +837,8 @@
EXPECT_TRUE(Verifier2.match("void g(void (*fp)(void) noexcept(true));\n",
parmVarDecl(hasType(pointerType(pointee(
parenType(innerType(functionType())),
- Language::Lang_CXX11));
+ Lang_CXX11));
}
-} // end namespace ast_matchers
-} // end namespace clang
+
+} // end namespace
Index: clang/unittests/AST/SourceLocationTest.cpp
===
--- clang/unittests/AST/SourceLocationTest.cpp
+++ clang/unittests/AST/SourceLocationTest.cpp
@@ -22,8 +22,10 @@
#include "clang/Tooling/Tooling.h"
#include "gtest/gtest.h"
-namespace clang {
-namespace ast_matchers {
+using namespace clang;
+using namespace clang::ast_matchers;
+
+namespace {
// FIXME: Pull the *Verifier tests into their own test file.
@@ -642,10 +644,8 @@
TEST(FunctionDecl, FunctionDeclWithNoExceptSpecification) {
RangeVerifier Verifier;
Verifier.expectRange(1, 1, 1, 24);
- EXPECT_TRUE(Verifier.match(
- "void f() noexcept(false);\n",
- functionDecl(),
- Language::Lang_CXX11));
+ EXPECT_TRUE(Verifier.match("void f() noexcept(false);\n", functionDecl(),
+ Lang_CXX11));
}
class FunctionDeclParametersRangeVerifier : public RangeVerifier {
@@ -767,12 +767,10 @@
TEST(CXXMethodDecl, CXXMethodDeclWithNoExceptSpecification) {
RangeVerifier Verifier;
Verifier.expectRange(2, 1, 2, 24);
- EXPECT_TRUE(Verifier.match(
- "class A {\n"
- "void f() noexcept(false);\n"
- "};\n",
- functionDecl(),
- Language::Lang_CXX11));
+ EXPECT_TRUE(Verifier.match("class A {\n"
+ "void f() noexcept(false);\n"
+ "};\n",
+ functionDecl(), Lang_CXX11));
}
class ExceptionSpecRangeVerifier : public RangeVerifier {
@@ -815,19 +813,19 @@
std::vector Args;
Args.push_back("-fms-extensions");
EXPECT_TRUE(Verifier.match("void f() throw(...);\n", loc(functionType()),
[PATCH] D72089: [Syntax] Build declarator nodes
hlopko added a comment.
I'm taking over of this patch (with a kind permission from Ilya :) I've replied
to comments here, but let's continue the review over at
https://reviews.llvm.org/D76220. Thanks :)
Comment at: clang/include/clang/Tooling/Syntax/Nodes.h:395
}
+ /// FIXME: use custom iterator instead of 'vector'.
+ std::vector declarators();
gribozavr2 wrote:
> s/iterator/container/ ?
>
> Also unclear why a custom one should be used.
>
> I also think it should be an implementation comment (in the .cc file).
Waiting for reply from Ilya offline, in the meantime I'm keeping the comment as
is.
Comment at: clang/include/clang/Tooling/Syntax/Nodes.h:491
+/// Array size specified inside a declarator.
+/// E.g. `[10]` in `int a[10]`.
+class ArraySubscript final : public Tree {
gribozavr2 wrote:
> Also `[static 10]` in `void f(int xs[static 10]);`
Added example into the comment + added a test verifying it's handled correctly.
Comment at: clang/include/clang/Tooling/Syntax/Nodes.h:500
+ syntax::Expression *sizeExpression();
+ syntax::Leaf *rbracket();
+};
gribozavr2 wrote:
> + TODO: add an accessor for the "static" keyword.
Added a TODO.
Comment at: clang/include/clang/Tooling/Syntax/Nodes.h:503
+
+/// Trailing return type inside parameter list, starting from the arrow token.
+/// E.g. `-> int***`.
gribozavr2 wrote:
> s/inside parameter list/after the parameter list/ ?
>
> s/starting from/starting with/ or "including the arrow token" to emphasize
> that the arrow is included.
Done.
Comment at: clang/include/clang/Tooling/Syntax/Nodes.h:512
+ syntax::Leaf *arrow();
+ syntax::SimpleDeclarator *declarator();
+};
gribozavr2 wrote:
> + TODO: add accessors for specifiers.
>
> Or we could add a syntax node for the "type-id" construct -- it seems like it
> will be useful in other places that require exactly one type, like the
> argument of sizeof, type argument of static_cast etc. (Also could be a TODO,
> this patch is pretty long.)
Added a TODO (also mentioning type-id node)
Comment at: clang/include/clang/Tooling/Syntax/Nodes.h:515
+
+/// Parameter list for a function type.
+class ParametersAndQualifiers final : public Tree {
gribozavr2 wrote:
> Would be great to show an example of what sort of qualifiers we're talking
> about here (trailing "const", "volatile", "&", "&&", right?) What about
> "noexcept"? Would be also good to say that "override" does not belong here.
Added a comment (and added tests where there was no coverage):
+/// E.g.:
+/// `(volatile int a)` in `int foo(volatile int a);`
+/// `(int&& a)` in `int foo(int&& a);`
+/// `() -> int` in `auto foo() -> int;`
+/// `() const` in `int foo() const;`
+/// `() noexcept` in `int foo() noexcept;`
+/// `() throw()` in `int foo() throw();`
+///
+/// (!) override doesn't belong here.
Comment at: clang/include/clang/Tooling/Syntax/Nodes.h:524
+ /// FIXME: use custom iterator instead of 'vector'.
+ std::vector parameters();
+ syntax::Leaf *rparen();
gribozavr2 wrote:
> Similarly, I think it should be an implementation comment.
Will handle once I hear from Ilya.
Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:53
+namespace {
+struct GetStartLoc : TypeLocVisitor {
+ SourceLocation VisitParenTypeLoc(ParenTypeLoc T) {
gribozavr2 wrote:
> A brief comment about why this is necessary (and why it works) would be
> appreciated. Just remind the reader that type locs are stored inside-out, and
> that the start location in the source order would be on the innermost node.
Added:
53 /// Get start location of the Decl from the TypeLoc.
54 /// E.g.:
55 /// loc of `(` in `int (a)`
56 /// loc of `*` in `int *(a)`
57 /// loc of the first `(` in `int (*a)(int)`
[PATCH] D76220: Build declarator nodes
hlopko created this revision.
hlopko added a reviewer: gribozavr2.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
Copy of https://reviews.llvm.org/D72089 with Ilya's permission. See
https://reviews.llvm.org/D72089 for the first batch of comments.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D76220
Files:
clang/include/clang/Tooling/Syntax/Nodes.h
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Nodes.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -174,17 +174,21 @@
*: TranslationUnit
|-SimpleDeclaration
| |-int
-| |-main
-| |-(
-| |-)
+| |-SimpleDeclarator
+| | |-main
+| | `-ParametersAndQualifiers
+| | |-(
+| | `-)
| `-CompoundStatement
| |-{
| `-}
`-SimpleDeclaration
|-void
- |-foo
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-foo
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
`-}
@@ -201,9 +205,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-int
- |-main
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-main
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-IfStatement
@@ -246,9 +252,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-ForStatement
@@ -268,18 +276,21 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-DeclarationStatement
| |-SimpleDeclaration
| | |-int
-| | |-a
-| | |-=
-| | `-UnknownExpression
-| | `-10
+| | `-SimpleDeclarator
+| | |-a
+| | |-=
+| | `-UnknownExpression
+| | `-10
| `-;
`-}
)txt"},
@@ -287,9 +298,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-EmptyStatement
@@ -309,9 +322,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-SwitchStatement
@@ -345,9 +360,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-WhileStatement
@@ -375,9 +392,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-int
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-ReturnStatement
@@ -398,26 +417,31 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-DeclarationStatement
| |-SimpleDeclaration
| | |-int
-| | |-a
-| | |-[
-| | |-UnknownExpression
-| | | `-3
-| | `-]
+| | `-SimpleDeclarator
+| | |-a
+| | `-ArraySubscript
+| | |-[
+| | |-UnknownExpression
+| | | `-3
+| | `-]
| `-;
|-RangeBasedForStatement
| |-for
| |-(
| |-SimpleDeclaration
| | |-int
-| | |-x
+| | |-SimpleDeclarator
+| | | `-x
| | `-:
| |-UnknownExpression
| | `-a
@@ -433,9 +457,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-main
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-main
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-UnknownStatement
@@ -460,9 +486,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-ExpressionStatement
@@ -500,10 +528,12 @@
*: TranslationUnit
`-SimpleDeclaration
|-int
- |-*
- |-a
+ |-SimpleDeclarator
+ | |-*
+ | `-a
|-,
- |-b
+ |-SimpleDeclarator
+ | `-b
`-;
)txt"},
{R"cpp(
@@ -514,10 +544,12 @@
`-SimpleDeclaration
|-typedef
|-int
- |-*
- |-a
+ |-SimpleDeclarator
+ | |-*
+ | `-a
|-,
- |-b
+ |-SimpleDeclarator
+ | `-b
`-;
)txt"},
// Multiple declarators inside a statement.
@@ -531,27 +563,33 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-foo
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-foo
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-DeclarationStatement
| |-SimpleDeclaration
| | |-int
-
[PATCH] D76220: [Syntax] Build declarator nodes
hlopko updated this revision to Diff 250574.
hlopko marked 8 inline comments as done.
hlopko added a comment.
Resolving comments.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76220/new/
https://reviews.llvm.org/D76220
Files:
clang/include/clang/Tooling/Syntax/Nodes.h
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Nodes.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -174,17 +174,21 @@
*: TranslationUnit
|-SimpleDeclaration
| |-int
-| |-main
-| |-(
-| |-)
+| |-SimpleDeclarator
+| | |-main
+| | `-ParametersAndQualifiers
+| | |-(
+| | `-)
| `-CompoundStatement
| |-{
| `-}
`-SimpleDeclaration
|-void
- |-foo
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-foo
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
`-}
@@ -201,9 +205,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-int
- |-main
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-main
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-IfStatement
@@ -246,9 +252,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-ForStatement
@@ -268,18 +276,21 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-DeclarationStatement
| |-SimpleDeclaration
| | |-int
-| | |-a
-| | |-=
-| | `-UnknownExpression
-| | `-10
+| | `-SimpleDeclarator
+| | |-a
+| | |-=
+| | `-UnknownExpression
+| | `-10
| `-;
`-}
)txt"},
@@ -287,9 +298,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-EmptyStatement
@@ -309,9 +322,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-SwitchStatement
@@ -345,9 +360,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-WhileStatement
@@ -375,9 +392,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-int
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-ReturnStatement
@@ -398,26 +417,31 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-DeclarationStatement
| |-SimpleDeclaration
| | |-int
-| | |-a
-| | |-[
-| | |-UnknownExpression
-| | | `-3
-| | `-]
+| | `-SimpleDeclarator
+| | |-a
+| | `-ArraySubscript
+| | |-[
+| | |-UnknownExpression
+| | | `-3
+| | `-]
| `-;
|-RangeBasedForStatement
| |-for
| |-(
| |-SimpleDeclaration
| | |-int
-| | |-x
+| | |-SimpleDeclarator
+| | | `-x
| | `-:
| |-UnknownExpression
| | `-a
@@ -433,9 +457,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-main
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-main
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-UnknownStatement
@@ -460,9 +486,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-ExpressionStatement
@@ -500,10 +528,12 @@
*: TranslationUnit
`-SimpleDeclaration
|-int
- |-*
- |-a
+ |-SimpleDeclarator
+ | |-*
+ | `-a
|-,
- |-b
+ |-SimpleDeclarator
+ | `-b
`-;
)txt"},
{R"cpp(
@@ -514,10 +544,12 @@
`-SimpleDeclaration
|-typedef
|-int
- |-*
- |-a
+ |-SimpleDeclarator
+ | |-*
+ | `-a
|-,
- |-b
+ |-SimpleDeclarator
+ | `-b
`-;
)txt"},
// Multiple declarators inside a statement.
@@ -531,27 +563,33 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-foo
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-foo
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-DeclarationStatement
| |-SimpleDeclaration
| | |-int
-| | |-*
-| | |-a
+| | |-SimpleDeclarator
+| | | |-*
+| |
[PATCH] D76220: [Syntax] Build declarator nodes
hlopko added inline comments.
Comment at: clang/include/clang/Tooling/Syntax/Nodes.h:522
+/// `(volatile int a)` in `int foo(volatile int a);`
+/// `(int&& a)` in `int foo(int&& a);`
+/// `() -> int` in `auto foo() -> int;`
gribozavr2 wrote:
> I meant:
>
> `int foo() volatile;`
> `int foo() &&;`
>
> Of course parameters can be cv and ref-qualified, but that's not the
> interesting part here. What's interesting is function qualifiers.
Rewrote the comment, added tests.
Comment at: clang/include/clang/Tooling/Syntax/Nodes.h:526
+/// `() noexcept` in `int foo() noexcept;`
+/// `() throw()` in `int foo() throw();`
+///
gribozavr2 wrote:
> Would you mind adding these examples to tests?
We already have them (see "Exception specification in parameter lists" and
"Trailing return type in parameter lists").
Comment at: clang/include/clang/Tooling/Syntax/Nodes.h:544
+/// E.g. `X::*` in `int X::* a = 0;`
+class MemberPointer final : public Tree {
+public:
gribozavr2 wrote:
> Seems a bit weird that we have a separate node for member pointers, array
> subscripts, but not for regular pointers.
>
> I think the rationale behind it is that since we are not building a tree
> structure out of declarators, the only token under a regular pointer
> declarator would be a star, so creating that node would not be very helpful.
Yeah I agree, and even though not very helpful, it looks more polished to me,
so I'll add upload a separate patch with that.
Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:61
+///
+/// (!) TypeLocs are stored inside out (in the example above `*volatile` is
+/// the TypeLoc returned by `Decl.getTypeSourceInfo()`, and `*const` is
gribozavr2 wrote:
> I don't think LLVM does `(!)` type of ASCII art.
>
> """
> It is non-trivial to get the start location because TypeLocs are stored
> inside out. In the example above, ...
> """
Ack, I saw some usages of (!) so was using it too :) Removed in the whole file.
Updated the comment.
Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:106
+if (T.getTypePtr()->hasTrailingReturn())
+ return SourceLocation(); // avoid recursing into the suffix of
declarator.
+return VisitTypeLoc(T);
gribozavr2 wrote:
> I suspect there might be an issue here with nested function declarators
> because we will avoid recursing not only into the suffix (which I think means
> the trailing return type), but also into parameters.
>
> If it was not necessary to recurse into parameters, why would the return
> statement below do that?
Noted, will address in a separate patch.
Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:1282
+struct X {};
+int X::* a;
+)cpp",
gribozavr2 wrote:
> Please also add tests with qualifiers.
>
> `const int X::* b;`
> `const int X::* const c;`
>
> Also for member functions:
>
> `int (X::*d)(int param);`
These 2 tests break asserts, will fix in a separate patch.
const int X::* const c;
int (X::*d)(int param);
Added a test for `const int X::* b;` into this one.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76220/new/
https://reviews.llvm.org/D76220
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76220: [Syntax] Build declarator nodes
hlopko updated this revision to Diff 250591.
hlopko added a comment.
Fix clang tidy warning.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76220/new/
https://reviews.llvm.org/D76220
Files:
clang/include/clang/Tooling/Syntax/Nodes.h
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Nodes.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -174,17 +174,21 @@
*: TranslationUnit
|-SimpleDeclaration
| |-int
-| |-main
-| |-(
-| |-)
+| |-SimpleDeclarator
+| | |-main
+| | `-ParametersAndQualifiers
+| | |-(
+| | `-)
| `-CompoundStatement
| |-{
| `-}
`-SimpleDeclaration
|-void
- |-foo
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-foo
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
`-}
@@ -201,9 +205,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-int
- |-main
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-main
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-IfStatement
@@ -246,9 +252,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-ForStatement
@@ -268,18 +276,21 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-DeclarationStatement
| |-SimpleDeclaration
| | |-int
-| | |-a
-| | |-=
-| | `-UnknownExpression
-| | `-10
+| | `-SimpleDeclarator
+| | |-a
+| | |-=
+| | `-UnknownExpression
+| | `-10
| `-;
`-}
)txt"},
@@ -287,9 +298,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-EmptyStatement
@@ -309,9 +322,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-SwitchStatement
@@ -345,9 +360,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-WhileStatement
@@ -375,9 +392,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-int
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-ReturnStatement
@@ -398,26 +417,31 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-DeclarationStatement
| |-SimpleDeclaration
| | |-int
-| | |-a
-| | |-[
-| | |-UnknownExpression
-| | | `-3
-| | `-]
+| | `-SimpleDeclarator
+| | |-a
+| | `-ArraySubscript
+| | |-[
+| | |-UnknownExpression
+| | | `-3
+| | `-]
| `-;
|-RangeBasedForStatement
| |-for
| |-(
| |-SimpleDeclaration
| | |-int
-| | |-x
+| | |-SimpleDeclarator
+| | | `-x
| | `-:
| |-UnknownExpression
| | `-a
@@ -433,9 +457,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-main
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-main
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-UnknownStatement
@@ -460,9 +486,11 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-test
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-test
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-ExpressionStatement
@@ -500,10 +528,12 @@
*: TranslationUnit
`-SimpleDeclaration
|-int
- |-*
- |-a
+ |-SimpleDeclarator
+ | |-*
+ | `-a
|-,
- |-b
+ |-SimpleDeclarator
+ | `-b
`-;
)txt"},
{R"cpp(
@@ -514,10 +544,12 @@
`-SimpleDeclaration
|-typedef
|-int
- |-*
- |-a
+ |-SimpleDeclarator
+ | |-*
+ | `-a
|-,
- |-b
+ |-SimpleDeclarator
+ | `-b
`-;
)txt"},
// Multiple declarators inside a statement.
@@ -531,27 +563,33 @@
*: TranslationUnit
`-SimpleDeclaration
|-void
- |-foo
- |-(
- |-)
+ |-SimpleDeclarator
+ | |-foo
+ | `-ParametersAndQualifiers
+ | |-(
+ | `-)
`-CompoundStatement
|-{
|-DeclarationStatement
| |-SimpleDeclaration
| | |-int
-| | |-*
-| | |-a
+| | |-SimpleDeclarator
+| | | |-*
+| | | `-a
| | |-,
-| | `-b
+
[PATCH] D76346: Build template declaration nodes
hlopko created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
hlopko added a reviewer: gribozavr2.
Copy of https://reviews.llvm.org/D72334, submitting with Ilya's permission.
Handles template declaration of all kinds.
Also builds template declaration nodes for specializations and explicit
instantiations of classes.
Some missing things will be addressed in the follow-up patches:
specializations of functions and variables,
template parameters.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D76346
Files:
clang/include/clang/Tooling/Syntax/Nodes.h
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Nodes.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -678,6 +678,212 @@
`-;
)txt"},
{R"cpp(
+template struct cls {};
+template int var = 10;
+template int fun() {}
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-cls
+| |-{
+| |-}
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | |-var
+| | |-=
+| | `-UnknownExpression
+| | `-10
+| `-;
+`-TemplateDeclaration
+ |-template
+ |-<
+ |-UnknownDeclaration
+ | |-class
+ | `-T
+ |->
+ `-SimpleDeclaration
+|-int
+|-SimpleDeclarator
+| |-fun
+| `-ParametersAndQualifiers
+| |-(
+| `-)
+`-CompoundStatement
+ |-{
+ `-}
+)txt"},
+ {R"cpp(
+template
+struct X {
+ template
+ U foo();
+};
+)cpp",
+ R"txt(
+*: TranslationUnit
+`-TemplateDeclaration
+ |-template
+ |-<
+ |-UnknownDeclaration
+ | |-class
+ | `-T
+ |->
+ `-SimpleDeclaration
+|-struct
+|-X
+|-{
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-U
+| |->
+| `-SimpleDeclaration
+| |-U
+| |-SimpleDeclarator
+| | |-foo
+| | `-ParametersAndQualifiers
+| | |-(
+| | `-)
+| `-;
+|-}
+`-;
+)txt"},
+ {R"cpp(
+template struct X {};
+template struct X {};
+template <> struct X {};
+
+template struct X;
+extern template struct X;
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-}
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-<
+| |-T
+| |-*
+| |->
+| |-{
+| |-}
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-<
+| |-int
+| |->
+| |-{
+| |-}
+| `-;
+|-ExplicitTemplateInstantiation
+| |-template
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-<
+| |-double
+| |->
+| `-;
+`-ExplicitTemplateInstantiation
+ |-extern
+ |-template
+ `-SimpleDeclaration
+|-struct
+|-X
+|-<
+|-float
+|->
+`-;
+)txt"},
+ {R"cpp(
+template struct X { struct Y; };
+template struct X::Y {};
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-struct
+| | |-Y
+| | `-;
+| |-}
+| `-;
+`-TemplateDeclaration
+ |-template
+ |-<
+ |-UnknownDeclaration
+ | |-class
+ | `-T
+ |->
+ `-SimpleDeclaration
+|-struct
+|-X
+|-<
+|-T
+|->
+|-::
+|-Y
+|-{
+|-}
+`-;
+ )txt"},
+ {R"cpp(
namespace ns {}
using namespace ::ns;
)cpp",
@@ -726,7 +932,7 @@
)cpp",
R"txt(
*: TranslationUnit
-`-UnknownDeclaration
+`-TemplateDeclaration
|-template
|-<
|-UnknownDeclaration
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -58,6 +58,10 @@
return OS << "LinkageSpecificationDeclaration";
case NodeKind::SimpleDeclaration:
return OS << "SimpleDeclaration";
+ case NodeKind::TemplateDeclaration:
+return OS << "TemplateDeclaration";
+ case NodeKind::ExplicitTemplateInstantiation:
+return OS << "ExplicitTemplateInstantiation";
case NodeKind::NamespaceDefinition:
return OS << "NamespaceDefinition";
case NodeKind::NamespaceAliasDefinition:
@@ -118,6 +122,12 @@
return OS << "StaticAssertDeclaration_message";
case syntax::NodeRole::SimpleDeclaration_declarator:
return OS <
[PATCH] D72334: [Syntax] Build nodes for template declarations.
hlopko added a comment.
Let's continue the review at https://reviews.llvm.org/D76346.
Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:190
+ // Set role for the node that may or may not be delayed. Node must span
+ // exactly \p Range.
+ void markMaybeDelayedChild(llvm::ArrayRef Range, NodeRole R);
gribozavr2 wrote:
> Three slashes for docs.
Done.
Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:930
+Builder.markChildToken(TemplateKW, syntax::NodeRole::IntroducerKeyword);
+Builder.markMaybeDelayedChild(
+TemplatedDeclaration,
gribozavr2 wrote:
> Why is this range maybe-delayed?
E.g because `template struct cls {};` is doesn't need delaying, but
template int var = 10;` does (SimpleDeclaration is processed before
Declarator). I'll try to come up with a more fitting design after I submit this
and https://reviews.llvm.org/D72446.
Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:738
+ U foo();
+};
+)cpp",
gribozavr2 wrote:
> Could you also add an out-of-line definition of X::foo? It will have two
> template parameter lists, which is a special case.
Example:
template
struct X {
template
U foo();
};
**template **
template
U X::foo() {}
Discussed offline, RAV doesn't visit TemplateParameters and therefore we're
not creating a node for the first template parameter (in bold). We could
overcome this in the BuildTree.cpp, but more principled solution would be to
teach RAV to visit TemplateParameters. In a separate patch.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D72334/new/
https://reviews.llvm.org/D72334
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76355: [Syntax] Build mapping from AST to syntax tree nodes
hlopko created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
hlopko edited the summary of this revision.
hlopko added a parent revision: D76346: [Syntax] Build template declaration
nodes.
Copy of https://reviews.llvm.org/D72446, submitting with Ilya's permission.
Only used to assign roles to child nodes for now. This is more efficient
than doing range-based queries.
In the future, will be exposed in the public API of syntax trees.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D76355
Files:
clang/include/clang/Tooling/Syntax/Tree.h
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Tree.cpp
Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -66,14 +66,20 @@
bool syntax::Tree::classof(const Node *N) { return N->kind() > NodeKind::Leaf; }
void syntax::Tree::prependChildLowLevel(Node *Child, NodeRole Role) {
- assert(Child->Parent == nullptr);
- assert(Child->NextSibling == nullptr);
assert(Child->role() == NodeRole::Detached);
assert(Role != NodeRole::Detached);
+ Child->Role = static_cast(Role);
+ prependChildLowLevel(Child);
+}
+
+void syntax::Tree::prependChildLowLevel(Node *Child) {
+ assert(Child->Parent == nullptr);
+ assert(Child->NextSibling == nullptr);
+ assert(Child->role() != NodeRole::Detached);
+
Child->Parent = this;
Child->NextSibling = this->FirstChild;
- Child->Role = static_cast(Role);
this->FirstChild = Child;
}
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -25,6 +25,8 @@
#include "clang/Tooling/Syntax/Tokens.h"
#include "clang/Tooling/Syntax/Tree.h"
#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/PointerUnion.h"
#include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/ScopeExit.h"
#include "llvm/ADT/SmallVector.h"
@@ -34,6 +36,7 @@
#include "llvm/Support/FormatVariadic.h"
#include "llvm/Support/MemoryBuffer.h"
#include "llvm/Support/raw_ostream.h"
+#include
#include
using namespace clang;
@@ -145,6 +148,32 @@
return SourceRange(Start, End);
}
+namespace {
+/// All AST hierarchy roots that can be represented as pointers.
+using ASTPtr = llvm::PointerUnion;
+/// Maintains a mapping from AST to syntax tree nodes. This class will get more
+/// complicated as we support more kinds of AST nodes, e.g. TypeLocs.
+/// FIXME: expose this as public API.
+class ASTToSyntaxMapping {
+public:
+ void add(ASTPtr From, syntax::Tree *To) {
+assert(To != nullptr);
+
+bool Added = Nodes.insert({From.getOpaqueValue(), To}).second;
+(void)Added;
+assert(Added && "mapping added twice");
+ }
+
+ syntax::Tree *find(ASTPtr P) const {
+return Nodes.lookup(P.getOpaqueValue());
+ }
+
+private:
+ // Keys are either Stmt* or Decl*.
+ llvm::DenseMap Nodes;
+};
+} // namespace
+
/// A helper class for constructing the syntax tree while traversing a clang
/// AST.
///
@@ -172,7 +201,18 @@
/// Populate children for \p New node, assuming it covers tokens from \p
/// Range.
- void foldNode(llvm::ArrayRef Range, syntax::Tree *New);
+ void foldNode(llvm::ArrayRef Range, syntax::Tree *New,
+ASTPtr From) {
+assert(New);
+Pending.foldChildren(Arena, Range, New);
+if (From)
+ Mapping.add(From, New);
+ }
+ void foldNode(llvm::ArrayRef Range, syntax::Tree *New,
+TypeLoc L) {
+// FIXME: add mapping for TypeLocs
+foldNode(Range, New, nullptr);
+ }
/// Must be called with the range of each `DeclaratorDecl`. Ensures the
/// corresponding declarator nodes are covered by `SimpleDeclaration`.
@@ -195,8 +235,10 @@
/// Set role for \p T.
void markChildToken(const syntax::Token *T, NodeRole R);
- /// Set role for the node that spans exactly \p Range.
- void markChild(llvm::ArrayRef Range, NodeRole R);
+ /// Set role for \p N.
+ void markChild(syntax::Node *N, NodeRole R);
+ /// Set role for the syntax node matching \p N.
+ void markChild(ASTPtr N, NodeRole R);
/// Set role for the delayed node that spans exactly \p Range.
void markDelayedChild(llvm::ArrayRef Range, NodeRole R);
/// Set role for the node that may or may not be delayed. Node must span
@@ -290,6 +332,11 @@
return Tokens;
}
+ void setRole(syntax::Node *N, NodeRole R) {
+assert(N->role() == NodeRole::Detached);
+N->Role = static_cast(R);
+ }
+
/// A collection of trees covering the input tokens.
/// When created, each tree corresponds to a single token in the file.
/// Clients call 'foldChildren' to attach one or more subtrees to a parent
@@ -306,7 +353,7 @@
auto *L = new (A.allocator()) syntax::Leaf(&T);
L->Original = true;
L->CanModify = A.toke
[PATCH] D76366: [Syntax] Split syntax tests
hlopko created this revision.
hlopko added a reviewer: gribozavr2.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
hlopko added a parent revision: D76346: [Syntax] Build template declaration
nodes.
This patch split Basic test into multple individual tests to allow simpler
filtering and clearer signal into what's broken when it's broken.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D76366
Files:
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -120,6 +120,16 @@
return Root;
}
+ void expectTreeDumpEqual(StringRef code, StringRef tree) {
+SCOPED_TRACE(code);
+
+auto *Root = buildTree(code);
+std::string Expected = tree.trim().str();
+std::string Actual =
+std::string(llvm::StringRef(Root->dump(*Arena)).trim());
+EXPECT_EQ(Expected, Actual) << "the resulting dump is:\n" << Actual;
+ }
+
// Adds a file to the test VFS.
void addFile(llvm::StringRef Path, llvm::StringRef Contents) {
if (!FS->addFile(Path, time_t(),
@@ -163,14 +173,13 @@
std::unique_ptr Arena;
};
-TEST_F(SyntaxTreeTest, Basic) {
- std::pair Cases[] = {
- {
- R"cpp(
+TEST_F(SyntaxTreeTest, Simple) {
+ expectTreeDumpEqual(
+ R"cpp(
int main() {}
void foo() {}
)cpp",
- R"txt(
+ R"txt(
*: TranslationUnit
|-SimpleDeclaration
| |-int
@@ -192,16 +201,18 @@
`-CompoundStatement
|-{
`-}
-)txt"},
- // if.
- {
- R"cpp(
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, If) {
+ expectTreeDumpEqual(
+ R"cpp(
int main() {
if (true) {}
if (true) {} else if (false) {}
}
)cpp",
- R"txt(
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-int
@@ -241,14 +252,17 @@
| |-{
| `-}
`-}
-)txt"},
- // for.
- {R"cpp(
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, For) {
+ expectTreeDumpEqual(
+ R"cpp(
void test() {
for (;;) {}
}
)cpp",
- R"txt(
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-void
@@ -269,10 +283,18 @@
| |-{
| `-}
`-}
-)txt"},
- // declaration statement.
- {"void test() { int a = 10; }",
- R"txt(
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, RangeBasedFor) {
+ expectTreeDumpEqual(
+ R"cpp(
+void test() {
+ int a[3];
+ for (int x : a) ;
+}
+ )cpp",
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-void
@@ -288,13 +310,32 @@
| | |-int
| | `-SimpleDeclarator
| | |-a
-| | |-=
-| | `-UnknownExpression
-| | `-10
+| | `-ArraySubscript
+| | |-[
+| | |-UnknownExpression
+| | | `-3
+| | `-]
| `-;
+|-RangeBasedForStatement
+| |-for
+| |-(
+| |-SimpleDeclaration
+| | |-int
+| | |-SimpleDeclarator
+| | | `-x
+| | `-:
+| |-UnknownExpression
+| | `-a
+| |-)
+| `-EmptyStatement
+| `-;
`-}
-)txt"},
- {"void test() { ; }", R"txt(
+ )txt");
+}
+
+TEST_F(SyntaxTreeTest, DeclarationStatement) {
+ expectTreeDumpEqual("void test() { int a = 10; }",
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-void
@@ -305,12 +346,22 @@
| `-)
`-CompoundStatement
|-{
-|-EmptyStatement
+|-DeclarationStatement
+| |-SimpleDeclaration
+| | |-int
+| | `-SimpleDeclarator
+| | |-a
+| | |-=
+| | `-UnknownExpression
+| | `-10
| `-;
`-}
-)txt"},
- // switch, case and default.
- {R"cpp(
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, Switch) {
+ expectTreeDumpEqual(
+ R"cpp(
void test() {
switch (true) {
case 0:
@@ -318,7 +369,7 @@
}
}
)cpp",
- R"txt(
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-void
@@ -349,14 +400,17 @@
| | `-;
| `-}
`-}
-)txt"},
- // while.
- {R"cpp(
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, While) {
+ expectTreeDumpEqual(
+ R"cpp(
void test() {
while (true) { continue; break; }
}
)cpp",
- R"txt(
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-void
@@ -383,77 +437,15 @@
| | `-;
| `-}
`-}
-)txt"},
- // return.
- {R"cpp(
-int test() { return 1; }
- )cpp",
- R"txt(
-*: TranslationUnit
-`-SimpleDeclaration
- |-int
- |-SimpleDeclarator
- | |-test
- | `-ParametersAndQualifiers
- | |-(
- | `-)
- `-CompoundStatement
-|-{
-|-ReturnStatement
-| |-return
-| |-UnknownExpression
-| | `-1
-| `-;
-`-}
-)txt"},
- // Range-based for.
- {R"cpp(
-void test() {
- int a[3];
- for (int x : a) ;
+)txt");
}
- )cpp",
- R"txt(
-*: TranslationUnit
-`-SimpleDeclaration
- |-void
- |-SimpleDeclarator
- | |-test
- | `-Para
[PATCH] D76346: [Syntax] Build template declaration nodes
hlopko updated this revision to Diff 251095.
hlopko added a comment.
Clang tidy
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76346/new/
https://reviews.llvm.org/D76346
Files:
clang/include/clang/Tooling/Syntax/Nodes.h
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Nodes.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -678,6 +678,212 @@
`-;
)txt"},
{R"cpp(
+template struct cls {};
+template int var = 10;
+template int fun() {}
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-cls
+| |-{
+| |-}
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | |-var
+| | |-=
+| | `-UnknownExpression
+| | `-10
+| `-;
+`-TemplateDeclaration
+ |-template
+ |-<
+ |-UnknownDeclaration
+ | |-class
+ | `-T
+ |->
+ `-SimpleDeclaration
+|-int
+|-SimpleDeclarator
+| |-fun
+| `-ParametersAndQualifiers
+| |-(
+| `-)
+`-CompoundStatement
+ |-{
+ `-}
+)txt"},
+ {R"cpp(
+template
+struct X {
+ template
+ U foo();
+};
+)cpp",
+ R"txt(
+*: TranslationUnit
+`-TemplateDeclaration
+ |-template
+ |-<
+ |-UnknownDeclaration
+ | |-class
+ | `-T
+ |->
+ `-SimpleDeclaration
+|-struct
+|-X
+|-{
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-U
+| |->
+| `-SimpleDeclaration
+| |-U
+| |-SimpleDeclarator
+| | |-foo
+| | `-ParametersAndQualifiers
+| | |-(
+| | `-)
+| `-;
+|-}
+`-;
+)txt"},
+ {R"cpp(
+template struct X {};
+template struct X {};
+template <> struct X {};
+
+template struct X;
+extern template struct X;
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-}
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-<
+| |-T
+| |-*
+| |->
+| |-{
+| |-}
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-<
+| |-int
+| |->
+| |-{
+| |-}
+| `-;
+|-ExplicitTemplateInstantiation
+| |-template
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-<
+| |-double
+| |->
+| `-;
+`-ExplicitTemplateInstantiation
+ |-extern
+ |-template
+ `-SimpleDeclaration
+|-struct
+|-X
+|-<
+|-float
+|->
+`-;
+)txt"},
+ {R"cpp(
+template struct X { struct Y; };
+template struct X::Y {};
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-struct
+| | |-Y
+| | `-;
+| |-}
+| `-;
+`-TemplateDeclaration
+ |-template
+ |-<
+ |-UnknownDeclaration
+ | |-class
+ | `-T
+ |->
+ `-SimpleDeclaration
+|-struct
+|-X
+|-<
+|-T
+|->
+|-::
+|-Y
+|-{
+|-}
+`-;
+ )txt"},
+ {R"cpp(
namespace ns {}
using namespace ::ns;
)cpp",
@@ -726,7 +932,7 @@
)cpp",
R"txt(
*: TranslationUnit
-`-UnknownDeclaration
+`-TemplateDeclaration
|-template
|-<
|-UnknownDeclaration
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -58,6 +58,10 @@
return OS << "LinkageSpecificationDeclaration";
case NodeKind::SimpleDeclaration:
return OS << "SimpleDeclaration";
+ case NodeKind::TemplateDeclaration:
+return OS << "TemplateDeclaration";
+ case NodeKind::ExplicitTemplateInstantiation:
+return OS << "ExplicitTemplateInstantiation";
case NodeKind::NamespaceDefinition:
return OS << "NamespaceDefinition";
case NodeKind::NamespaceAliasDefinition:
@@ -118,6 +122,12 @@
return OS << "StaticAssertDeclaration_message";
case syntax::NodeRole::SimpleDeclaration_declarator:
return OS << "SimpleDeclaration_declarator";
+ case syntax::NodeRole::TemplateDeclaration_declaration:
+return OS << "TemplateDeclaration_declaration";
+ case syntax::NodeRole::ExplicitTemplateInstantiation_externKeyword:
+return OS << "ExplicitTemplateInstantiation_externKeyword";
+ case syntax::NodeRole::ExplicitTemplateInstantiation_decla
[PATCH] D76366: [Syntax] Split syntax tests
hlopko updated this revision to Diff 251102.
hlopko marked 5 inline comments as done.
hlopko added a comment.
Renaming some tests.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76366/new/
https://reviews.llvm.org/D76366
Files:
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -120,6 +120,16 @@
return Root;
}
+ void expectTreeDumpEqual(StringRef code, StringRef tree) {
+SCOPED_TRACE(code);
+
+auto *Root = buildTree(code);
+std::string Expected = tree.trim().str();
+std::string Actual =
+std::string(llvm::StringRef(Root->dump(*Arena)).trim());
+EXPECT_EQ(Expected, Actual) << "the resulting dump is:\n" << Actual;
+ }
+
// Adds a file to the test VFS.
void addFile(llvm::StringRef Path, llvm::StringRef Contents) {
if (!FS->addFile(Path, time_t(),
@@ -163,14 +173,13 @@
std::unique_ptr Arena;
};
-TEST_F(SyntaxTreeTest, Basic) {
- std::pair Cases[] = {
- {
- R"cpp(
+TEST_F(SyntaxTreeTest, Simple) {
+ expectTreeDumpEqual(
+ R"cpp(
int main() {}
void foo() {}
)cpp",
- R"txt(
+ R"txt(
*: TranslationUnit
|-SimpleDeclaration
| |-int
@@ -192,16 +201,18 @@
`-CompoundStatement
|-{
`-}
-)txt"},
- // if.
- {
- R"cpp(
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, If) {
+ expectTreeDumpEqual(
+ R"cpp(
int main() {
if (true) {}
if (true) {} else if (false) {}
}
)cpp",
- R"txt(
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-int
@@ -241,14 +252,17 @@
| |-{
| `-}
`-}
-)txt"},
- // for.
- {R"cpp(
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, For) {
+ expectTreeDumpEqual(
+ R"cpp(
void test() {
for (;;) {}
}
)cpp",
- R"txt(
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-void
@@ -269,10 +283,18 @@
| |-{
| `-}
`-}
-)txt"},
- // declaration statement.
- {"void test() { int a = 10; }",
- R"txt(
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, RangeBasedFor) {
+ expectTreeDumpEqual(
+ R"cpp(
+void test() {
+ int a[3];
+ for (int x : a) ;
+}
+ )cpp",
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-void
@@ -288,13 +310,32 @@
| | |-int
| | `-SimpleDeclarator
| | |-a
-| | |-=
-| | `-UnknownExpression
-| | `-10
+| | `-ArraySubscript
+| | |-[
+| | |-UnknownExpression
+| | | `-3
+| | `-]
| `-;
+|-RangeBasedForStatement
+| |-for
+| |-(
+| |-SimpleDeclaration
+| | |-int
+| | |-SimpleDeclarator
+| | | `-x
+| | `-:
+| |-UnknownExpression
+| | `-a
+| |-)
+| `-EmptyStatement
+| `-;
`-}
-)txt"},
- {"void test() { ; }", R"txt(
+ )txt");
+}
+
+TEST_F(SyntaxTreeTest, DeclarationStatement) {
+ expectTreeDumpEqual("void test() { int a = 10; }",
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-void
@@ -305,12 +346,22 @@
| `-)
`-CompoundStatement
|-{
-|-EmptyStatement
+|-DeclarationStatement
+| |-SimpleDeclaration
+| | |-int
+| | `-SimpleDeclarator
+| | |-a
+| | |-=
+| | `-UnknownExpression
+| | `-10
| `-;
`-}
-)txt"},
- // switch, case and default.
- {R"cpp(
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, Switch) {
+ expectTreeDumpEqual(
+ R"cpp(
void test() {
switch (true) {
case 0:
@@ -318,7 +369,7 @@
}
}
)cpp",
- R"txt(
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-void
@@ -349,14 +400,17 @@
| | `-;
| `-}
`-}
-)txt"},
- // while.
- {R"cpp(
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, While) {
+ expectTreeDumpEqual(
+ R"cpp(
void test() {
while (true) { continue; break; }
}
)cpp",
- R"txt(
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-void
@@ -383,77 +437,15 @@
| | `-;
| `-}
`-}
-)txt"},
- // return.
- {R"cpp(
-int test() { return 1; }
- )cpp",
- R"txt(
-*: TranslationUnit
-`-SimpleDeclaration
- |-int
- |-SimpleDeclarator
- | |-test
- | `-ParametersAndQualifiers
- | |-(
- | `-)
- `-CompoundStatement
-|-{
-|-ReturnStatement
-| |-return
-| |-UnknownExpression
-| | `-1
-| `-;
-`-}
-)txt"},
- // Range-based for.
- {R"cpp(
-void test() {
- int a[3];
- for (int x : a) ;
+)txt");
}
- )cpp",
- R"txt(
-*: TranslationUnit
-`-SimpleDeclaration
- |-void
- |-SimpleDeclarator
- | |-test
- | `-ParametersAndQualifiers
- | |-(
- | `-)
- `-CompoundStatement
-|-{
-|-DeclarationStatement
-| |-SimpleDeclaration
-| | |-int
-| | `-SimpleDecla
[PATCH] D76366: [Syntax] Split syntax tests
hlopko added a comment. Addressed comments. Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D76366/new/ https://reviews.llvm.org/D76366 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76418: [Syntax] Build template declaration nodes
hlopko created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
Rollforward of
https://reviews.llvm.org/rGdd12826808f9079e164b82e64b0697a077379241 after
temporarily adding -fno-delayed-template-parsing to the TreeTest.
Original summary:
> Copy of https://reviews.llvm.org/D72334, submitting with Ilya's permission.
>
> Handles template declaration of all kinds.
>
> Also builds template declaration nodes for specializations and explicit
> instantiations of classes.
>
> Some missing things will be addressed in the follow-up patches:
>
> - specializations of functions and variables,
> - template parameters.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D76418
Files:
clang/include/clang/Tooling/Syntax/Nodes.h
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Nodes.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -99,6 +99,7 @@
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
std::vector Args = {"syntax-test", "-std=c++11",
+ "-fno-delayed-template-parsing",
"-fsyntax-only", FileName};
Invocation = createInvocationFromCommandLine(Args, Diags, FS);
assert(Invocation);
@@ -678,6 +679,212 @@
`-;
)txt"},
{R"cpp(
+template struct cls {};
+template int var = 10;
+template int fun() {}
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-cls
+| |-{
+| |-}
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | |-var
+| | |-=
+| | `-UnknownExpression
+| | `-10
+| `-;
+`-TemplateDeclaration
+ |-template
+ |-<
+ |-UnknownDeclaration
+ | |-class
+ | `-T
+ |->
+ `-SimpleDeclaration
+|-int
+|-SimpleDeclarator
+| |-fun
+| `-ParametersAndQualifiers
+| |-(
+| `-)
+`-CompoundStatement
+ |-{
+ `-}
+)txt"},
+ {R"cpp(
+template
+struct X {
+ template
+ U foo();
+};
+)cpp",
+ R"txt(
+*: TranslationUnit
+`-TemplateDeclaration
+ |-template
+ |-<
+ |-UnknownDeclaration
+ | |-class
+ | `-T
+ |->
+ `-SimpleDeclaration
+|-struct
+|-X
+|-{
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-U
+| |->
+| `-SimpleDeclaration
+| |-U
+| |-SimpleDeclarator
+| | |-foo
+| | `-ParametersAndQualifiers
+| | |-(
+| | `-)
+| `-;
+|-}
+`-;
+)txt"},
+ {R"cpp(
+template struct X {};
+template struct X {};
+template <> struct X {};
+
+template struct X;
+extern template struct X;
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-}
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-<
+| |-T
+| |-*
+| |->
+| |-{
+| |-}
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-<
+| |-int
+| |->
+| |-{
+| |-}
+| `-;
+|-ExplicitTemplateInstantiation
+| |-template
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-<
+| |-double
+| |->
+| `-;
+`-ExplicitTemplateInstantiation
+ |-extern
+ |-template
+ `-SimpleDeclaration
+|-struct
+|-X
+|-<
+|-float
+|->
+`-;
+)txt"},
+ {R"cpp(
+template struct X { struct Y; };
+template struct X::Y {};
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-struct
+| | |-Y
+| | `-;
+| |-}
+| `-;
+`-TemplateDeclaration
+ |-template
+ |-<
+ |-UnknownDeclaration
+ | |-class
+ | `-T
+ |->
+ `-SimpleDeclaration
+|-struct
+|-X
+|-<
+|-T
+|->
+|-::
+|-Y
+|-{
+|-}
+`-;
+ )txt"},
+ {R"cpp(
namespace ns {}
using namespace ::ns;
)cpp",
@@ -726,7 +933,7 @@
)cpp",
R"txt(
*: TranslationUnit
-`-UnknownDeclaration
+`-TemplateDeclaration
|-template
|-<
|-UnknownDeclaration
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -58,6 +58,10 @@
return OS << "Linka
[PATCH] D76422: Testing presubmits
hlopko created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
ignore me
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D76422
Files:
clang/include/clang/Tooling/Syntax/Nodes.h
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Nodes.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -99,6 +99,7 @@
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
std::vector Args = {"syntax-test", "-std=c++11",
+ "-fno-delayed-template-parsing",
"-fsyntax-only", FileName};
Invocation = createInvocationFromCommandLine(Args, Diags, FS);
assert(Invocation);
@@ -678,6 +679,212 @@
`-;
)txt"},
{R"cpp(
+template struct cls {};
+template int var = 10;
+template int fun() {}
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-cls
+| |-{
+| |-}
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | |-var
+| | |-=
+| | `-UnknownExpression
+| | `-10
+| `-;
+`-TemplateDeclaration
+ |-template
+ |-<
+ |-UnknownDeclaration
+ | |-class
+ | `-T
+ |->
+ `-SimpleDeclaration
+|-int
+|-SimpleDeclarator
+| |-fun
+| `-ParametersAndQualifiers
+| |-(
+| `-)
+`-CompoundStatement
+ |-{
+ `-}
+)txt"},
+ {R"cpp(
+template
+struct X {
+ template
+ U foo();
+};
+)cpp",
+ R"txt(
+*: TranslationUnit
+`-TemplateDeclaration
+ |-template
+ |-<
+ |-UnknownDeclaration
+ | |-class
+ | `-T
+ |->
+ `-SimpleDeclaration
+|-struct
+|-X
+|-{
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-U
+| |->
+| `-SimpleDeclaration
+| |-U
+| |-SimpleDeclarator
+| | |-foo
+| | `-ParametersAndQualifiers
+| | |-(
+| | `-)
+| `-;
+|-}
+`-;
+)txt"},
+ {R"cpp(
+template struct X {};
+template struct X {};
+template <> struct X {};
+
+template struct X;
+extern template struct X;
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-}
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-<
+| |-T
+| |-*
+| |->
+| |-{
+| |-}
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-<
+| |-int
+| |->
+| |-{
+| |-}
+| `-;
+|-ExplicitTemplateInstantiation
+| |-template
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-<
+| |-double
+| |->
+| `-;
+`-ExplicitTemplateInstantiation
+ |-extern
+ |-template
+ `-SimpleDeclaration
+|-struct
+|-X
+|-<
+|-float
+|->
+`-;
+)txt"},
+ {R"cpp(
+template struct X { struct Y; };
+template struct X::Y {};
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-struct
+| | |-Y
+| | `-;
+| |-}
+| `-;
+`-TemplateDeclaration
+ |-template
+ |-<
+ |-UnknownDeclaration
+ | |-class
+ | `-T
+ |->
+ `-SimpleDeclaration
+|-struct
+|-X
+|-<
+|-T
+|->
+|-::
+|-Y
+|-{
+|-}
+`-;
+ )txt"},
+ {R"cpp(
namespace ns {}
using namespace ::ns;
)cpp",
@@ -726,7 +933,7 @@
)cpp",
R"txt(
*: TranslationUnit
-`-UnknownDeclaration
+`-TemplateDeclaration
|-template
|-<
|-UnknownDeclaration
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -58,6 +58,10 @@
return OS << "LinkageSpecificationDeclaration";
case NodeKind::SimpleDeclaration:
return OS << "SimpleDeclaration";
+ case NodeKind::TemplateDeclaration:
+return OS << "TemplateDeclaration";
+ case NodeKind::ExplicitTemplateInstantiation:
+return OS << "ExplicitTemplateInstantiation";
case NodeKind::NamespaceDefinition:
return OS << "NamespaceDefinition";
case NodeKind::NamespaceAliasDefinition:
@@ -118,6 +122,12 @@
return OS << "StaticAssertDeclaration_message";
case syntax::NodeRole::SimpleDeclaration_declarator:
[PATCH] D76422: Testing presubmits
hlopko updated this revision to Diff 251387.
hlopko added a comment.
Removing fix
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76422/new/
https://reviews.llvm.org/D76422
Files:
clang/include/clang/Tooling/Syntax/Nodes.h
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Nodes.cpp
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -678,6 +678,212 @@
`-;
)txt"},
{R"cpp(
+template struct cls {};
+template int var = 10;
+template int fun() {}
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-cls
+| |-{
+| |-}
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | |-var
+| | |-=
+| | `-UnknownExpression
+| | `-10
+| `-;
+`-TemplateDeclaration
+ |-template
+ |-<
+ |-UnknownDeclaration
+ | |-class
+ | `-T
+ |->
+ `-SimpleDeclaration
+|-int
+|-SimpleDeclarator
+| |-fun
+| `-ParametersAndQualifiers
+| |-(
+| `-)
+`-CompoundStatement
+ |-{
+ `-}
+)txt"},
+ {R"cpp(
+template
+struct X {
+ template
+ U foo();
+};
+)cpp",
+ R"txt(
+*: TranslationUnit
+`-TemplateDeclaration
+ |-template
+ |-<
+ |-UnknownDeclaration
+ | |-class
+ | `-T
+ |->
+ `-SimpleDeclaration
+|-struct
+|-X
+|-{
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-U
+| |->
+| `-SimpleDeclaration
+| |-U
+| |-SimpleDeclarator
+| | |-foo
+| | `-ParametersAndQualifiers
+| | |-(
+| | `-)
+| `-;
+|-}
+`-;
+)txt"},
+ {R"cpp(
+template struct X {};
+template struct X {};
+template <> struct X {};
+
+template struct X;
+extern template struct X;
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-}
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-<
+| |-T
+| |-*
+| |->
+| |-{
+| |-}
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-<
+| |-int
+| |->
+| |-{
+| |-}
+| `-;
+|-ExplicitTemplateInstantiation
+| |-template
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-<
+| |-double
+| |->
+| `-;
+`-ExplicitTemplateInstantiation
+ |-extern
+ |-template
+ `-SimpleDeclaration
+|-struct
+|-X
+|-<
+|-float
+|->
+`-;
+)txt"},
+ {R"cpp(
+template struct X { struct Y; };
+template struct X::Y {};
+)cpp",
+ R"txt(
+*: TranslationUnit
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-class
+| | `-T
+| |->
+| `-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-struct
+| | |-Y
+| | `-;
+| |-}
+| `-;
+`-TemplateDeclaration
+ |-template
+ |-<
+ |-UnknownDeclaration
+ | |-class
+ | `-T
+ |->
+ `-SimpleDeclaration
+|-struct
+|-X
+|-<
+|-T
+|->
+|-::
+|-Y
+|-{
+|-}
+`-;
+ )txt"},
+ {R"cpp(
namespace ns {}
using namespace ::ns;
)cpp",
@@ -726,7 +932,7 @@
)cpp",
R"txt(
*: TranslationUnit
-`-UnknownDeclaration
+`-TemplateDeclaration
|-template
|-<
|-UnknownDeclaration
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -58,6 +58,10 @@
return OS << "LinkageSpecificationDeclaration";
case NodeKind::SimpleDeclaration:
return OS << "SimpleDeclaration";
+ case NodeKind::TemplateDeclaration:
+return OS << "TemplateDeclaration";
+ case NodeKind::ExplicitTemplateInstantiation:
+return OS << "ExplicitTemplateInstantiation";
case NodeKind::NamespaceDefinition:
return OS << "NamespaceDefinition";
case NodeKind::NamespaceAliasDefinition:
@@ -118,6 +122,12 @@
return OS << "StaticAssertDeclaration_message";
case syntax::NodeRole::SimpleDeclaration_declarator:
return OS << "SimpleDeclaration_declarator";
+ case syntax::NodeRole::TemplateDeclaration_declaration:
+return OS << "TemplateDeclaration_declaration";
+ case syntax::NodeRole::ExplicitTemplateInstantiation_externKeyword:
+return OS << "ExplicitTemplateInstantiation_externKeyword";
+ case syntax::NodeRole::ExplicitTemplateInstantiation_dec
[PATCH] D76433: [Syntax] Make TreeTest independent from the host platform.
hlopko created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
hlopko added a parent revision: D76366: [Syntax] Split syntax tests.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D76433
Files:
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -98,8 +98,12 @@
if (!Diags->getClient())
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
-std::vector Args = {"syntax-test", "-std=c++11",
- "-fsyntax-only", FileName};
+std::vector Args = {
+ "syntax-test", "-std=c++11",
+ // Hard-setting target so the unit test behavior doesn't depend on the
+ // host.
+ "-target", "x86_64-unknown-unknown",
+ "-fsyntax-only", FileName};
Invocation = createInvocationFromCommandLine(Args, Diags, FS);
assert(Invocation);
Invocation->getFrontendOpts().DisableFree = false;
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -98,8 +98,12 @@
if (!Diags->getClient())
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
-std::vector Args = {"syntax-test", "-std=c++11",
- "-fsyntax-only", FileName};
+std::vector Args = {
+ "syntax-test", "-std=c++11",
+ // Hard-setting target so the unit test behavior doesn't depend on the
+ // host.
+ "-target", "x86_64-unknown-unknown",
+ "-fsyntax-only", FileName};
Invocation = createInvocationFromCommandLine(Args, Diags, FS);
assert(Invocation);
Invocation->getFrontendOpts().DisableFree = false;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76433: [Syntax] Make TreeTest independent from the host platform.
hlopko updated this revision to Diff 251399.
hlopko added a comment.
Reformat
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76433/new/
https://reviews.llvm.org/D76433
Files:
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -99,6 +99,9 @@
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
std::vector Args = {"syntax-test", "-std=c++11",
+ // Hard-setting target so the unit test
+ // behavior doesn't depend on the host.
+ "-target", "x86_64-unknown-unknown",
"-fsyntax-only", FileName};
Invocation = createInvocationFromCommandLine(Args, Diags, FS);
assert(Invocation);
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -99,6 +99,9 @@
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
std::vector Args = {"syntax-test", "-std=c++11",
+ // Hard-setting target so the unit test
+ // behavior doesn't depend on the host.
+ "-target", "x86_64-unknown-unknown",
"-fsyntax-only", FileName};
Invocation = createInvocationFromCommandLine(Args, Diags, FS);
assert(Invocation);
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76366: [Syntax] Split syntax tests
hlopko updated this revision to Diff 251571.
hlopko added a comment.
Rebase
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76366/new/
https://reviews.llvm.org/D76366
Files:
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -121,6 +121,16 @@
return Root;
}
+ void expectTreeDumpEqual(StringRef code, StringRef tree) {
+SCOPED_TRACE(code);
+
+auto *Root = buildTree(code);
+std::string Expected = tree.trim().str();
+std::string Actual =
+std::string(llvm::StringRef(Root->dump(*Arena)).trim());
+EXPECT_EQ(Expected, Actual) << "the resulting dump is:\n" << Actual;
+ }
+
// Adds a file to the test VFS.
void addFile(llvm::StringRef Path, llvm::StringRef Contents) {
if (!FS->addFile(Path, time_t(),
@@ -164,14 +174,13 @@
std::unique_ptr Arena;
};
-TEST_F(SyntaxTreeTest, Basic) {
- std::pair Cases[] = {
- {
- R"cpp(
+TEST_F(SyntaxTreeTest, Simple) {
+ expectTreeDumpEqual(
+ R"cpp(
int main() {}
void foo() {}
)cpp",
- R"txt(
+ R"txt(
*: TranslationUnit
|-SimpleDeclaration
| |-int
@@ -193,16 +202,18 @@
`-CompoundStatement
|-{
`-}
-)txt"},
- // if.
- {
- R"cpp(
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, If) {
+ expectTreeDumpEqual(
+ R"cpp(
int main() {
if (true) {}
if (true) {} else if (false) {}
}
)cpp",
- R"txt(
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-int
@@ -242,14 +253,17 @@
| |-{
| `-}
`-}
-)txt"},
- // for.
- {R"cpp(
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, For) {
+ expectTreeDumpEqual(
+ R"cpp(
void test() {
for (;;) {}
}
)cpp",
- R"txt(
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-void
@@ -270,10 +284,18 @@
| |-{
| `-}
`-}
-)txt"},
- // declaration statement.
- {"void test() { int a = 10; }",
- R"txt(
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, RangeBasedFor) {
+ expectTreeDumpEqual(
+ R"cpp(
+void test() {
+ int a[3];
+ for (int x : a) ;
+}
+ )cpp",
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-void
@@ -289,13 +311,32 @@
| | |-int
| | `-SimpleDeclarator
| | |-a
-| | |-=
-| | `-UnknownExpression
-| | `-10
+| | `-ArraySubscript
+| | |-[
+| | |-UnknownExpression
+| | | `-3
+| | `-]
| `-;
+|-RangeBasedForStatement
+| |-for
+| |-(
+| |-SimpleDeclaration
+| | |-int
+| | |-SimpleDeclarator
+| | | `-x
+| | `-:
+| |-UnknownExpression
+| | `-a
+| |-)
+| `-EmptyStatement
+| `-;
`-}
-)txt"},
- {"void test() { ; }", R"txt(
+ )txt");
+}
+
+TEST_F(SyntaxTreeTest, DeclarationStatement) {
+ expectTreeDumpEqual("void test() { int a = 10; }",
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-void
@@ -306,12 +347,22 @@
| `-)
`-CompoundStatement
|-{
-|-EmptyStatement
+|-DeclarationStatement
+| |-SimpleDeclaration
+| | |-int
+| | `-SimpleDeclarator
+| | |-a
+| | |-=
+| | `-UnknownExpression
+| | `-10
| `-;
`-}
-)txt"},
- // switch, case and default.
- {R"cpp(
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, Switch) {
+ expectTreeDumpEqual(
+ R"cpp(
void test() {
switch (true) {
case 0:
@@ -319,7 +370,7 @@
}
}
)cpp",
- R"txt(
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-void
@@ -350,14 +401,17 @@
| | `-;
| `-}
`-}
-)txt"},
- // while.
- {R"cpp(
+)txt");
+}
+
+TEST_F(SyntaxTreeTest, While) {
+ expectTreeDumpEqual(
+ R"cpp(
void test() {
while (true) { continue; break; }
}
)cpp",
- R"txt(
+ R"txt(
*: TranslationUnit
`-SimpleDeclaration
|-void
@@ -384,77 +438,15 @@
| | `-;
| `-}
`-}
-)txt"},
- // return.
- {R"cpp(
-int test() { return 1; }
- )cpp",
- R"txt(
-*: TranslationUnit
-`-SimpleDeclaration
- |-int
- |-SimpleDeclarator
- | |-test
- | `-ParametersAndQualifiers
- | |-(
- | `-)
- `-CompoundStatement
-|-{
-|-ReturnStatement
-| |-return
-| |-UnknownExpression
-| | `-1
-| `-;
-`-}
-)txt"},
- // Range-based for.
- {R"cpp(
-void test() {
- int a[3];
- for (int x : a) ;
+)txt");
}
- )cpp",
- R"txt(
-*: TranslationUnit
-`-SimpleDeclaration
- |-void
- |-SimpleDeclarator
- | |-test
- | `-ParametersAndQualifiers
- | |-(
- | `-)
- `-CompoundStatement
-|-{
-|-DeclarationStatement
-| |-SimpleDeclaration
-| | |-int
-| | `-SimpleDeclarator
-| | |-a
-| | `-ArraySubscript
-|
[PATCH] D76433: [Syntax] Test both the default and windows target platforms in unittests
hlopko updated this revision to Diff 251576.
hlopko added a comment.
Refactored to run tests on both default and windows platforms.
Opting out Templates test for now since it doesn't work with
-fdelayed-template-parsing yet.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76433/new/
https://reviews.llvm.org/D76433
Files:
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -47,7 +47,7 @@
class SyntaxTreeTest : public ::testing::Test {
protected:
// Build a syntax tree for the code.
- syntax::TranslationUnit *buildTree(llvm::StringRef Code) {
+ syntax::TranslationUnit *buildTree(llvm::StringRef Code, StringRef Target) {
// FIXME: this code is almost the identical to the one in TokensTest. Share
//it.
class BuildSyntaxTree : public ASTConsumer {
@@ -98,8 +98,9 @@
if (!Diags->getClient())
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
-std::vector Args = {"syntax-test", "-std=c++11",
- "-fsyntax-only", FileName};
+std::vector Args = {"-target", Target.data(),
+ "-fsyntax-only", "-std=c++11",
+ "syntax-test", FileName};
Invocation = createInvocationFromCommandLine(Args, Diags, FS);
assert(Invocation);
Invocation->getFrontendOpts().DisableFree = false;
@@ -120,14 +121,23 @@
return Root;
}
- void expectTreeDumpEqual(StringRef code, StringRef tree) {
-SCOPED_TRACE(code);
+ void expectTreeDumpEqual(StringRef Code, StringRef Tree,
+ bool SkipWindows = false) {
+SCOPED_TRACE(Code);
-auto *Root = buildTree(code);
-std::string Expected = tree.trim().str();
-std::string Actual =
-std::string(llvm::StringRef(Root->dump(*Arena)).trim());
-EXPECT_EQ(Expected, Actual) << "the resulting dump is:\n" << Actual;
+std::string Expected = Tree.trim().str();
+
+for (const StringRef Target :
+ {"x86_64-unknown-unknown", "x86_64-pc-win32"}) {
+ if (SkipWindows && Target.equals("x86_64-pc-win32")) {
+continue;
+ }
+ auto *Root = buildTree(Code, Target);
+ std::string Actual = std::string(StringRef(Root->dump(*Arena)).trim());
+ EXPECT_EQ(Expected, Actual)
+ << "for target " << Target << " the resulting dump is:\n"
+ << Actual;
+}
}
// Adds a file to the test VFS.
@@ -793,7 +803,10 @@
`-CompoundStatement
|-{
`-}
-)txt");
+)txt",
+ // FIXME: Make this test work on windows by generating the expected
Syntax
+ // tree when -fdelayed-template-parsing is active.
+ /*SkipWindows*/ true);
}
TEST_F(SyntaxTreeTest, NestedTemplates) {
@@ -1739,7 +1752,7 @@
auto CheckTransformation = [this](std::string Input, std::string Expected,
Transformation Transform) -> void {
llvm::Annotations Source(Input);
-auto *Root = buildTree(Source.code());
+auto *Root = buildTree(Source.code(), "x86_64-unknown-unknown");
Transform(Source, Root);
@@ -1777,7 +1790,7 @@
}
TEST_F(SyntaxTreeTest, SynthesizedNodes) {
- buildTree("");
+ buildTree("", "x86_64-unknown-unknown");
auto *C = syntax::createPunctuation(*Arena, tok::comma);
ASSERT_NE(C, nullptr);
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -47,7 +47,7 @@
class SyntaxTreeTest : public ::testing::Test {
protected:
// Build a syntax tree for the code.
- syntax::TranslationUnit *buildTree(llvm::StringRef Code) {
+ syntax::TranslationUnit *buildTree(llvm::StringRef Code, StringRef Target) {
// FIXME: this code is almost the identical to the one in TokensTest. Share
//it.
class BuildSyntaxTree : public ASTConsumer {
@@ -98,8 +98,9 @@
if (!Diags->getClient())
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
-std::vector Args = {"syntax-test", "-std=c++11",
- "-fsyntax-only", FileName};
+std::vector Args = {"-target", Target.data(),
+ "-fsyntax-only", "-std=c++11",
+ "syntax-test", FileName};
Invocation = createInvocationFromCommandLine(Args, Diags, FS);
assert(Invocation);
Invocation->getFrontendOpts().DisableFree = false;
@@ -120,14 +121,23 @@
return Root;
}
- void expectTreeDumpEqual(StringRef code, StringRef tree) {
-SCOPED_TRACE(code);
+ void expectTreeDumpEqual(StringRef Code, StringR
[PATCH] D76433: [Syntax] Test both the default and windows target platforms in unittests
hlopko updated this revision to Diff 251585.
hlopko added a comment.
Rebase
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76433/new/
https://reviews.llvm.org/D76433
Files:
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -47,7 +47,7 @@
class SyntaxTreeTest : public ::testing::Test {
protected:
// Build a syntax tree for the code.
- syntax::TranslationUnit *buildTree(llvm::StringRef Code) {
+ syntax::TranslationUnit *buildTree(llvm::StringRef Code, StringRef Target) {
// FIXME: this code is almost the identical to the one in TokensTest. Share
//it.
class BuildSyntaxTree : public ASTConsumer {
@@ -98,9 +98,9 @@
if (!Diags->getClient())
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
-std::vector Args = {"syntax-test", "-std=c++11",
- "-fno-delayed-template-parsing",
- "-fsyntax-only", FileName};
+std::vector Args = {"-target", Target.data(),
+ "-fsyntax-only", "-std=c++17",
+ "syntax-test", FileName};
Invocation = createInvocationFromCommandLine(Args, Diags, FS);
assert(Invocation);
Invocation->getFrontendOpts().DisableFree = false;
@@ -121,14 +121,29 @@
return Root;
}
- void expectTreeDumpEqual(StringRef code, StringRef tree) {
-SCOPED_TRACE(code);
-
-auto *Root = buildTree(code);
-std::string Expected = tree.trim().str();
-std::string Actual =
-std::string(llvm::StringRef(Root->dump(*Arena)).trim());
-EXPECT_EQ(Expected, Actual) << "the resulting dump is:\n" << Actual;
+ void expectTreeDumpEqual(StringRef Code, StringRef Tree,
+ bool RunWithDelayedTemplateParsing = true) {
+SCOPED_TRACE(Code);
+
+std::string Expected = Tree.trim().str();
+
+// We want to run the test with -fdelayed-template-parsing enabled and
+// disabled, therefore we use these representative targets that differ in
+// the default value.
+// We are not passing -fdelayed-template-parsing directly but we are using
+// the `-target` to improve coverage and discover differences in behavior
+// early.
+for (const StringRef Target :
+ {"x86_64-unknown-unknown", "x86_64-pc-win32"}) {
+ if (!RunWithDelayedTemplateParsing && Target.equals("x86_64-pc-win32")) {
+continue;
+ }
+ auto *Root = buildTree(Code, Target);
+ std::string Actual = std::string(StringRef(Root->dump(*Arena)).trim());
+ EXPECT_EQ(Expected, Actual)
+ << "for target " << Target << " the resulting dump is:\n"
+ << Actual;
+}
}
// Adds a file to the test VFS.
@@ -794,7 +809,10 @@
`-CompoundStatement
|-{
`-}
-)txt");
+)txt",
+ // FIXME: Make this test work on windows by generating the expected
Syntax
+ // tree when -fdelayed-template-parsing is active.
+ /*RunWithDelayedTemplateParsing=*/true);
}
TEST_F(SyntaxTreeTest, NestedTemplates) {
@@ -1740,7 +1758,7 @@
auto CheckTransformation = [this](std::string Input, std::string Expected,
Transformation Transform) -> void {
llvm::Annotations Source(Input);
-auto *Root = buildTree(Source.code());
+auto *Root = buildTree(Source.code(), "x86_64-unknown-unknown");
Transform(Source, Root);
@@ -1778,7 +1796,7 @@
}
TEST_F(SyntaxTreeTest, SynthesizedNodes) {
- buildTree("");
+ buildTree("", "x86_64-unknown-unknown");
auto *C = syntax::createPunctuation(*Arena, tok::comma);
ASSERT_NE(C, nullptr);
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -47,7 +47,7 @@
class SyntaxTreeTest : public ::testing::Test {
protected:
// Build a syntax tree for the code.
- syntax::TranslationUnit *buildTree(llvm::StringRef Code) {
+ syntax::TranslationUnit *buildTree(llvm::StringRef Code, StringRef Target) {
// FIXME: this code is almost the identical to the one in TokensTest. Share
//it.
class BuildSyntaxTree : public ASTConsumer {
@@ -98,9 +98,9 @@
if (!Diags->getClient())
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
-std::vector Args = {"syntax-test", "-std=c++11",
- "-fno-delayed-template-parsing",
- "-fsyntax-only", FileName};
+std::vector Args = {"-target", Target.data(),
+ "-fsyntax-only", "-std=c++1
[PATCH] D76433: [Syntax] Test both the default and windows target platforms in unittests
hlopko updated this revision to Diff 251582.
hlopko marked 3 inline comments as done.
hlopko added a comment.
Address comments.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76433/new/
https://reviews.llvm.org/D76433
Files:
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -47,7 +47,7 @@
class SyntaxTreeTest : public ::testing::Test {
protected:
// Build a syntax tree for the code.
- syntax::TranslationUnit *buildTree(llvm::StringRef Code) {
+ syntax::TranslationUnit *buildTree(llvm::StringRef Code, StringRef Target) {
// FIXME: this code is almost the identical to the one in TokensTest. Share
//it.
class BuildSyntaxTree : public ASTConsumer {
@@ -98,8 +98,9 @@
if (!Diags->getClient())
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
-std::vector Args = {"syntax-test", "-std=c++11",
- "-fsyntax-only", FileName};
+std::vector Args = {"-target", Target.data(),
+ "-fsyntax-only", "-std=c++17",
+ "syntax-test", FileName};
Invocation = createInvocationFromCommandLine(Args, Diags, FS);
assert(Invocation);
Invocation->getFrontendOpts().DisableFree = false;
@@ -120,14 +121,29 @@
return Root;
}
- void expectTreeDumpEqual(StringRef code, StringRef tree) {
-SCOPED_TRACE(code);
-
-auto *Root = buildTree(code);
-std::string Expected = tree.trim().str();
-std::string Actual =
-std::string(llvm::StringRef(Root->dump(*Arena)).trim());
-EXPECT_EQ(Expected, Actual) << "the resulting dump is:\n" << Actual;
+ void expectTreeDumpEqual(StringRef Code, StringRef Tree,
+ bool RunWithDelayedTemplateParsing = true) {
+SCOPED_TRACE(Code);
+
+std::string Expected = Tree.trim().str();
+
+// We want to run the test with -fdelayed-template-parsing enabled and
+// disabled, therefore we use these representative targets that differ in
+// the default value.
+// We are not passing -fdelayed-template-parsing directly but we are using
+// the `-target` to improve coverage and discover differences in behavior
+// early.
+for (const StringRef Target :
+ {"x86_64-unknown-unknown", "x86_64-pc-win32"}) {
+ if (!RunWithDelayedTemplateParsing && Target.equals("x86_64-pc-win32")) {
+continue;
+ }
+ auto *Root = buildTree(Code, Target);
+ std::string Actual = std::string(StringRef(Root->dump(*Arena)).trim());
+ EXPECT_EQ(Expected, Actual)
+ << "for target " << Target << " the resulting dump is:\n"
+ << Actual;
+}
}
// Adds a file to the test VFS.
@@ -793,7 +809,10 @@
`-CompoundStatement
|-{
`-}
-)txt");
+)txt",
+ // FIXME: Make this test work on windows by generating the expected
Syntax
+ // tree when -fdelayed-template-parsing is active.
+ /*RunWithDelayedTemplateParsing=*/true);
}
TEST_F(SyntaxTreeTest, NestedTemplates) {
@@ -1739,7 +1758,7 @@
auto CheckTransformation = [this](std::string Input, std::string Expected,
Transformation Transform) -> void {
llvm::Annotations Source(Input);
-auto *Root = buildTree(Source.code());
+auto *Root = buildTree(Source.code(), "x86_64-unknown-unknown");
Transform(Source, Root);
@@ -1777,7 +1796,7 @@
}
TEST_F(SyntaxTreeTest, SynthesizedNodes) {
- buildTree("");
+ buildTree("", "x86_64-unknown-unknown");
auto *C = syntax::createPunctuation(*Arena, tok::comma);
ASSERT_NE(C, nullptr);
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -47,7 +47,7 @@
class SyntaxTreeTest : public ::testing::Test {
protected:
// Build a syntax tree for the code.
- syntax::TranslationUnit *buildTree(llvm::StringRef Code) {
+ syntax::TranslationUnit *buildTree(llvm::StringRef Code, StringRef Target) {
// FIXME: this code is almost the identical to the one in TokensTest. Share
//it.
class BuildSyntaxTree : public ASTConsumer {
@@ -98,8 +98,9 @@
if (!Diags->getClient())
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
-std::vector Args = {"syntax-test", "-std=c++11",
- "-fsyntax-only", FileName};
+std::vector Args = {"-target", Target.data(),
+ "-fsyntax-only", "-std=c++17",
+ "syntax-test", FileName};
Invocation = cre
[PATCH] D72446: [Syntax] Build mapping from AST to syntax tree nodes
hlopko added a comment.
Taking over the patch in https://reviews.llvm.org/D76355.
Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:147
+ void add(ASTPtr From, syntax::Tree *To) {
+assert(To != nullptr);
+
gribozavr2 wrote:
> Also assert that From is not null either?
Done.
Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:160
+ // Keys are either Stmt* or Decl*.
+ llvm::DenseMap Nodes;
+};
gribozavr2 wrote:
> I think it is possible to use a PointerUnion as a DenseMap key (there's a
> DenseMapInfo for it).
Done.
Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:491
/// FIXME: storing the end tokens is redundant.
/// FIXME: the key of a map is redundant, it is also stored in
NodeForRange.
+std::map Trees;
gribozavr2 wrote:
> I don't understand the first fixme (maybe it is stale).
>
> The second fixme is not applicable after this patch.
>
Removed both FIXMEs now, asking Ilya offline for clarification.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D72446/new/
https://reviews.llvm.org/D72446
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D76355: [Syntax] Build mapping from AST to syntax tree nodes
hlopko updated this revision to Diff 251603.
hlopko added a comment.
Addressing comments.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D76355/new/
https://reviews.llvm.org/D76355
Files:
clang/include/clang/Tooling/Syntax/Tree.h
clang/lib/Tooling/Syntax/BuildTree.cpp
clang/lib/Tooling/Syntax/Tree.cpp
Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -66,14 +66,20 @@
bool syntax::Tree::classof(const Node *N) { return N->kind() > NodeKind::Leaf; }
void syntax::Tree::prependChildLowLevel(Node *Child, NodeRole Role) {
- assert(Child->Parent == nullptr);
- assert(Child->NextSibling == nullptr);
assert(Child->role() == NodeRole::Detached);
assert(Role != NodeRole::Detached);
+ Child->Role = static_cast(Role);
+ prependChildLowLevel(Child);
+}
+
+void syntax::Tree::prependChildLowLevel(Node *Child) {
+ assert(Child->Parent == nullptr);
+ assert(Child->NextSibling == nullptr);
+ assert(Child->role() != NodeRole::Detached);
+
Child->Parent = this;
Child->NextSibling = this->FirstChild;
- Child->Role = static_cast(Role);
this->FirstChild = Child;
}
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -25,6 +25,8 @@
#include "clang/Tooling/Syntax/Tokens.h"
#include "clang/Tooling/Syntax/Tree.h"
#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/PointerUnion.h"
#include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/ScopeExit.h"
#include "llvm/ADT/SmallVector.h"
@@ -34,6 +36,7 @@
#include "llvm/Support/FormatVariadic.h"
#include "llvm/Support/MemoryBuffer.h"
#include "llvm/Support/raw_ostream.h"
+#include
#include
using namespace clang;
@@ -145,6 +148,33 @@
return SourceRange(Start, End);
}
+namespace {
+/// All AST hierarchy roots that can be represented as pointers.
+using ASTPtr = llvm::PointerUnion;
+/// Maintains a mapping from AST to syntax tree nodes. This class will get more
+/// complicated as we support more kinds of AST nodes, e.g. TypeLocs.
+/// FIXME: expose this as public API.
+class ASTToSyntaxMapping {
+public:
+ void add(ASTPtr From, syntax::Tree *To) {
+assert(To != nullptr);
+assert(!From.isNull());
+
+bool Added = Nodes.insert({From, To}).second;
+(void)Added;
+assert(Added && "mapping added twice");
+ }
+
+ syntax::Tree *find(ASTPtr P) const {
+return Nodes.lookup(P);
+ }
+
+private:
+ // Keys are either Stmt* or Decl*.
+ llvm::DenseMap Nodes;
+};
+} // namespace
+
/// A helper class for constructing the syntax tree while traversing a clang
/// AST.
///
@@ -172,7 +202,18 @@
/// Populate children for \p New node, assuming it covers tokens from \p
/// Range.
- void foldNode(llvm::ArrayRef Range, syntax::Tree *New);
+ void foldNode(llvm::ArrayRef Range, syntax::Tree *New,
+ASTPtr From) {
+assert(New);
+Pending.foldChildren(Arena, Range, New);
+if (From)
+ Mapping.add(From, New);
+ }
+ void foldNode(llvm::ArrayRef Range, syntax::Tree *New,
+TypeLoc L) {
+// FIXME: add mapping for TypeLocs
+foldNode(Range, New, nullptr);
+ }
/// Must be called with the range of each `DeclaratorDecl`. Ensures the
/// corresponding declarator nodes are covered by `SimpleDeclaration`.
@@ -195,8 +236,10 @@
/// Set role for \p T.
void markChildToken(const syntax::Token *T, NodeRole R);
- /// Set role for the node that spans exactly \p Range.
- void markChild(llvm::ArrayRef Range, NodeRole R);
+ /// Set role for \p N.
+ void markChild(syntax::Node *N, NodeRole R);
+ /// Set role for the syntax node matching \p N.
+ void markChild(ASTPtr N, NodeRole R);
/// Set role for the delayed node that spans exactly \p Range.
void markDelayedChild(llvm::ArrayRef Range, NodeRole R);
/// Set role for the node that may or may not be delayed. Node must span
@@ -290,6 +333,11 @@
return Tokens;
}
+ void setRole(syntax::Node *N, NodeRole R) {
+assert(N->role() == NodeRole::Detached);
+N->Role = static_cast(R);
+ }
+
/// A collection of trees covering the input tokens.
/// When created, each tree corresponds to a single token in the file.
/// Clients call 'foldChildren' to attach one or more subtrees to a parent
@@ -306,7 +354,7 @@
auto *L = new (A.allocator()) syntax::Leaf(&T);
L->Original = true;
L->CanModify = A.tokenBuffer().spelledForExpanded(T).hasValue();
-Trees.insert(Trees.end(), {&T, NodeAndRole{L}});
+Trees.insert(Trees.end(), {&T, L});
}
}
@@ -338,7 +386,9 @@
assert((std::next(It) == Trees.end() ||
std::next(It)->first == Range.end()) &&
"no child with the specifi
[PATCH] D76497: [Syntax] Test both the default and windows target platforms in unittests
hlopko created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
This increases the coverage for things that differ between Linux and Windows,
such as -fdelayed-template-parsing. This would have prevented the rollback of
https://reviews.llvm.org/D76346.
While at it, update -std=c++11 to c++17 for the test.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D76497
Files:
clang/unittests/Tooling/Syntax/TreeTest.cpp
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -47,7 +47,8 @@
class SyntaxTreeTest : public ::testing::Test {
protected:
// Build a syntax tree for the code.
- syntax::TranslationUnit *buildTree(llvm::StringRef Code) {
+ syntax::TranslationUnit *buildTree(llvm::StringRef Code,
+ StringRef Target = "x86_64-pc-linux-gnu")
{
// FIXME: this code is almost the identical to the one in TokensTest. Share
//it.
class BuildSyntaxTree : public ASTConsumer {
@@ -98,9 +99,10 @@
if (!Diags->getClient())
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
-std::vector Args = {"syntax-test", "-std=c++11",
- "-fno-delayed-template-parsing",
- "-fsyntax-only", FileName};
+std::vector Args = {
+"syntax-test", FileName, "-fsyntax-only",
+"-std=c++17", "-target", Target.data(),
+};
Invocation = createInvocationFromCommandLine(Args, Diags, FS);
assert(Invocation);
Invocation->getFrontendOpts().DisableFree = false;
@@ -121,14 +123,30 @@
return Root;
}
- void expectTreeDumpEqual(StringRef code, StringRef tree) {
-SCOPED_TRACE(code);
-
-auto *Root = buildTree(code);
-std::string Expected = tree.trim().str();
-std::string Actual =
-std::string(llvm::StringRef(Root->dump(*Arena)).trim());
-EXPECT_EQ(Expected, Actual) << "the resulting dump is:\n" << Actual;
+ void expectTreeDumpEqual(StringRef Code, StringRef Tree,
+ bool RunWithDelayedTemplateParsing = true) {
+SCOPED_TRACE(Code);
+
+std::string Expected = Tree.trim().str();
+
+// We want to run the test with -fdelayed-template-parsing enabled and
+// disabled, therefore we use these representative targets that differ in
+// the default value.
+// We are not passing -fdelayed-template-parsing directly but we are using
+// the `-target` to improve coverage and discover differences in behavior
+// early.
+for (const StringRef Target :
+ {"x86_64-pc-linux-gnu", "x86_64-pc-win32-msvc"}) {
+ if (!RunWithDelayedTemplateParsing &&
+ Target.equals("x86_64-pc-win32-msvc")) {
+continue;
+ }
+ auto *Root = buildTree(Code, Target);
+ std::string Actual = std::string(StringRef(Root->dump(*Arena)).trim());
+ EXPECT_EQ(Expected, Actual)
+ << "for target " << Target << " the resulting dump is:\n"
+ << Actual;
+}
}
// Adds a file to the test VFS.
@@ -794,7 +812,10 @@
`-CompoundStatement
|-{
`-}
-)txt");
+)txt",
+ // FIXME: Make this test work on windows by generating the expected
Syntax
+ // tree when -fdelayed-template-parsing is active.
+ /*RunWithDelayedTemplateParsing=*/false);
}
TEST_F(SyntaxTreeTest, NestedTemplates) {
Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -47,7 +47,8 @@
class SyntaxTreeTest : public ::testing::Test {
protected:
// Build a syntax tree for the code.
- syntax::TranslationUnit *buildTree(llvm::StringRef Code) {
+ syntax::TranslationUnit *buildTree(llvm::StringRef Code,
+ StringRef Target = "x86_64-pc-linux-gnu") {
// FIXME: this code is almost the identical to the one in TokensTest. Share
//it.
class BuildSyntaxTree : public ASTConsumer {
@@ -98,9 +99,10 @@
if (!Diags->getClient())
Diags->setClient(new IgnoringDiagConsumer);
// Prepare to run a compiler.
-std::vector Args = {"syntax-test", "-std=c++11",
- "-fno-delayed-template-parsing",
- "-fsyntax-only", FileName};
+std::vector Args = {
+"syntax-test", FileName, "-fsyntax-only",
+"-std=c++17", "-target", Target.data(),
+};
Invocation = createInvocationFromCommandLine(Args, Diags, FS);
assert(Invocation);
Invocation->getFrontendOpts().DisableFree = false;
@@ -121,14 +123,30 @@
return Root;
}
- void expectTreeDumpEqual(StringRef code, StringRef tree) {
-SCOPED_TRAC
