Re: [PATCH] D50415: [clangd] extend the publishDiagnostics response to send back fixits to the client directly as well (if requested)

[Sam McCall via cfe-commits]

Couple of thoughts. (Technically I'm out on leave so will let Jan/Ilya
review implementation and happy with whatever you decide)

Enabling
- negotiating LSP extensions is probably better done in the "capabilities"
message exchange than as a command-line flag. Generally, we want this
extension on if the *client* is aware of it. Roughly, the client
capabilities are owned by the client, and the flags are owned by the *user*.
- for simplicity, we could always enable this, unless we really think the
message size is a problem, or are worried about conflicts with future LSP
versions

Naming
- elsewhere in clangd we settled on calling a "Fix" what clang calls a
"FixItHint". The latter is long/awkward/jargon, and often gets shortened to
"FixIt" which isn't obviously a noun. The former mostly has its plain
English meaning. I'd prefer "fix" in the protocol/flags, for the same
reasons.
- obviously feel free to give these any name you prefer in your UI!

On Wed, Aug 8, 2018, 00:53 Alex Lorenz via Phabricator <
revi...@reviews.llvm.org> wrote:

> arphaman created this revision.
> arphaman added reviewers: jkorous, sammccall, ilya-biryukov.
> Herald added subscribers: dexonsmith, MaskRay, ioeric.
>
> This change extends the 'textDocument/publishDiagnostics' notification
> sent from Clangd to the client. The extension can be enabled using the
> '-fixit-usage=embed-in-diagnostic' argument. When it's enabled, Clangd
> sends out the fixits associated with the appropriate diagnostic in the body
> of the 'publicDiagnostics' notification.
>
>
> Repository:
>   rCTE Clang Tools Extra
>
> https://reviews.llvm.org/D50415
>
> Files:
>   clangd/ClangdLSPServer.cpp
>   clangd/ClangdLSPServer.h
>   clangd/Diagnostics.h
>   clangd/fuzzer/ClangdFuzzer.cpp
>   clangd/tool/ClangdMain.cpp
>   test/clangd/fixits-embed-in-diagnostic.test
>
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r332378 - [clangd] Extract scoring/ranking logic, and shave yaks.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue May 15 10:43:27 2018
New Revision: 332378

URL: http://llvm.org/viewvc/llvm-project?rev=332378&view=rev
Log:
[clangd] Extract scoring/ranking logic, and shave yaks.

Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
 - awkward to test. The mechanisms (extracting info from index/sema) can be
   unit-tested well, the policy (scoring) should be quantitatively measured.
   Neither was easily possible, and debugging was hard.
   The intermediate signal struct makes this easier.
 - hard to reuse. This is a bug in workspaceSymbols: it just presents the
   results in the index order, which is not sorted in practice, it needs to rank
   them!
   Also, index implementations care about scoring (both query-dependent and
   independent) in order to truncate result lists appropriately.

The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).

Reviewers: ilya-biryukov

Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits

Differential Revision: https://reviews.llvm.org/D46524

Added:
clang-tools-extra/trunk/clangd/Quality.cpp
clang-tools-extra/trunk/clangd/Quality.h
clang-tools-extra/trunk/unittests/clangd/QualityTests.cpp
clang-tools-extra/trunk/unittests/clangd/TestTU.cpp
clang-tools-extra/trunk/unittests/clangd/TestTU.h
Modified:
clang-tools-extra/trunk/clangd/CMakeLists.txt
clang-tools-extra/trunk/clangd/CodeComplete.cpp
clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt
clang-tools-extra/trunk/unittests/clangd/ClangdUnitTests.cpp
clang-tools-extra/trunk/unittests/clangd/FileIndexTests.cpp
clang-tools-extra/trunk/unittests/clangd/TestFS.cpp
clang-tools-extra/trunk/unittests/clangd/XRefsTests.cpp

Modified: clang-tools-extra/trunk/clangd/CMakeLists.txt
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/CMakeLists.txt?rev=332378&r1=332377&r2=332378&view=diff
==
--- clang-tools-extra/trunk/clangd/CMakeLists.txt (original)
+++ clang-tools-extra/trunk/clangd/CMakeLists.txt Tue May 15 10:43:27 2018
@@ -28,6 +28,7 @@ add_clang_library(clangDaemon
   Logger.cpp
   Protocol.cpp
   ProtocolHandlers.cpp
+  Quality.cpp
   SourceCode.cpp
   Threading.cpp
   Trace.cpp

Modified: clang-tools-extra/trunk/clangd/CodeComplete.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/CodeComplete.cpp?rev=332378&r1=332377&r2=332378&view=diff
==
--- clang-tools-extra/trunk/clangd/CodeComplete.cpp (original)
+++ clang-tools-extra/trunk/clangd/CodeComplete.cpp Tue May 15 10:43:27 2018
@@ -20,6 +20,7 @@
 #include "FuzzyMatch.h"
 #include "Headers.h"
 #include "Logger.h"
+#include "Quality.h"
 #include "SourceCode.h"
 #include "Trace.h"
 #include "URI.h"
@@ -34,6 +35,9 @@
 #include "llvm/Support/Format.h"
 #include 
 
+// We log detailed candidate here if you run with -debug-only=codecomplete.
+#define DEBUG_TYPE "codecomplete"
+
 namespace clang {
 namespace clangd {
 namespace {
@@ -192,35 +196,6 @@ getOptionalParameters(const CodeCompleti
   return Result;
 }
 
-// Produces an integer that sorts in the same order as F.
-// That is: a < b <==> encodeFloat(a) < encodeFloat(b).
-uint32_t encodeFloat(float F) {
-  static_assert(std::numeric_limits::is_iec559, "");
-  static_assert(sizeof(float) == sizeof(uint32_t), "");
-  constexpr uint32_t TopBit = ~(~uint32_t{0} >> 1);
-
-  // Get the bits of the float. Endianness is the same as for integers.
-  uint32_t U;
-  memcpy(&U, &F, sizeof(float));
-  // IEEE 754 floats compare like sign-magnitude integers.
-  if (U & TopBit)// Negative float.
-return 0 - U;// Map onto the low half of integers, order reversed.
-  return U + TopBit; // Positive floats map onto the high half of integers.
-}
-
-// Returns a string that sorts in the same order as (-Score, Name), for LSP.
-std::string sortText(float Score, llvm::StringRef Name) {
-  // We convert -Score to an integer, and hex-encode for readability.
-  // Example: [0.5, "foo"] -> "4100foo"
-  std::string S;
-  llvm::raw_string_ostream OS(S);
-  write_hex(OS, encodeFloat(-Score), llvm::HexPrintStyle::Lower,
-/*Width=*/2 * sizeof(Score));
-  OS << Name;
-  OS.flush();
-  return S;
-}
-
 /// Creates a `HeaderFile` from \p Header which can be either a URI or a 
literal
 /// include.
 static llvm::Expected toHeaderFile(StringRef Header,
@@ -251,33 +226,6 @@ struct CompletionCandidate {
   const CodeCompletionResult *SemaResult = nullptr;
   const Symbol *IndexResult = nullptr;
 
-  // Computes the "symbol quality" score for this completion. Higher is better.
-  float score() const {
-float Score = 1;
-if (IndexResult)
-  Score *= quality(*IndexResult);
-if (SemaResult) {
-  // For now we just use the 

r333514 - Fix -Wunused in NDEBUG introduced by HIP r333484

[Sam McCall via cfe-commits]

Author: sammccall
Date: Wed May 30 01:03:43 2018
New Revision: 333514

URL: http://llvm.org/viewvc/llvm-project?rev=333514&view=rev
Log:
Fix -Wunused in NDEBUG introduced by HIP r333484

Modified:
cfe/trunk/lib/Driver/ToolChains/HIP.cpp

Modified: cfe/trunk/lib/Driver/ToolChains/HIP.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/ToolChains/HIP.cpp?rev=333514&r1=333513&r2=333514&view=diff
==
--- cfe/trunk/lib/Driver/ToolChains/HIP.cpp (original)
+++ cfe/trunk/lib/Driver/ToolChains/HIP.cpp Wed May 30 01:03:43 2018
@@ -186,9 +186,7 @@ void AMDGCN::Linker::ConstructJob(Compil
const ArgList &Args,
const char *LinkingOutput) const {
 
-  const auto &TC =
-  static_cast(getToolChain());
-  assert(TC.getTriple().getArch() == llvm::Triple::amdgcn &&
+  assert(getToolChain().getTriple().getArch() == llvm::Triple::amdgcn &&
  "Unsupported target");
 
   std::string SubArchName = JA.getOffloadingArch();
@@ -224,6 +222,7 @@ void HIPToolChain::addClangTargetOptions
 
   StringRef GpuArch = DriverArgs.getLastArgValue(options::OPT_march_EQ);
   assert(!GpuArch.empty() && "Must have an explicit GPU arch.");
+  (void) GpuArch;
   assert(DeviceOffloadingKind == Action::OFK_HIP &&
  "Only HIP offloading kinds are supported for GPUs.");
 


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r314532 - Small clangd cleanups, NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Fri Sep 29 09:41:23 2017
New Revision: 314532

URL: http://llvm.org/viewvc/llvm-project?rev=314532&view=rev
Log:
Small clangd cleanups, NFC

 - remove old ASTUnit includes
 - fix typo (regiterCallbackHandlers)

Modified:
clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.h
clang-tools-extra/trunk/clangd/ClangdUnit.h
clang-tools-extra/trunk/clangd/ProtocolHandlers.cpp
clang-tools-extra/trunk/clangd/ProtocolHandlers.h

Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=314532&r1=314531&r2=314532&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp Fri Sep 29 09:41:23 2017
@@ -256,7 +256,7 @@ void ClangdLSPServer::run(std::istream &
   // Set up JSONRPCDispatcher.
   LSPProtocolCallbacks Callbacks(*this);
   JSONRPCDispatcher Dispatcher(llvm::make_unique(Out));
-  regiterCallbackHandlers(Dispatcher, Out, Callbacks);
+  registerCallbackHandlers(Dispatcher, Out, Callbacks);
 
   // Run the Language Server loop.
   runLanguageServerLoop(In, Out, Dispatcher, IsDone);

Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=314532&r1=314531&r2=314532&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Fri Sep 29 09:41:23 2017
@@ -9,7 +9,6 @@
 
 #include "ClangdServer.h"
 #include "clang/Format/Format.h"
-#include "clang/Frontend/ASTUnit.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Tooling/CompilationDatabase.h"

Modified: clang-tools-extra/trunk/clangd/ClangdServer.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.h?rev=314532&r1=314531&r2=314532&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.h (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.h Fri Sep 29 09:41:23 2017
@@ -13,7 +13,6 @@
 #include "ClangdUnitStore.h"
 #include "DraftStore.h"
 #include "GlobalCompilationDatabase.h"
-#include "clang/Frontend/ASTUnit.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "clang/Tooling/Core/Replacement.h"
 #include "llvm/ADT/IntrusiveRefCntPtr.h"

Modified: clang-tools-extra/trunk/clangd/ClangdUnit.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnit.h?rev=314532&r1=314531&r2=314532&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdUnit.h (original)
+++ clang-tools-extra/trunk/clangd/ClangdUnit.h Fri Sep 29 09:41:23 2017
@@ -12,7 +12,7 @@
 
 #include "Path.h"
 #include "Protocol.h"
-#include "clang/Frontend/ASTUnit.h"
+#include "clang/Frontend/FrontendAction.h"
 #include "clang/Frontend/PrecompiledPreamble.h"
 #include "clang/Serialization/ASTBitCodes.h"
 #include "clang/Tooling/CompilationDatabase.h"
@@ -27,7 +27,6 @@ class raw_ostream;
 }
 
 namespace clang {
-class ASTUnit;
 class PCHContainerOperations;
 
 namespace vfs {

Modified: clang-tools-extra/trunk/clangd/ProtocolHandlers.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ProtocolHandlers.cpp?rev=314532&r1=314531&r2=314532&view=diff
==
--- clang-tools-extra/trunk/clangd/ProtocolHandlers.cpp (original)
+++ clang-tools-extra/trunk/clangd/ProtocolHandlers.cpp Fri Sep 29 09:41:23 2017
@@ -228,7 +228,7 @@ private:
 
 } // namespace
 
-void clangd::regiterCallbackHandlers(JSONRPCDispatcher &Dispatcher,
+void clangd::registerCallbackHandlers(JSONRPCDispatcher &Dispatcher,
  JSONOutput &Out,
  ProtocolCallbacks &Callbacks) {
   Dispatcher.registerHandler(

Modified: clang-tools-extra/trunk/clangd/ProtocolHandlers.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ProtocolHandlers.h?rev=314532&r1=314531&r2=314532&view=diff
==
--- clang-tools-extra/trunk/clangd/ProtocolHandlers.h (original)
+++ clang-tools-extra/trunk/clangd/ProtocolHandlers.h Fri Sep 29 09:41:23 2017
@@ -53,7 +53,7 @@ public:
 JSONOutput &Out) = 0;  

 };
 
-void regiterCallbackHandlers(JSONRPCDispatcher &Dispatcher, JSONOutput &Out,
+void registerCallbackHandlers(JSONRPCDispatcher &Dispatcher, JSONOutput &O

[clang-tools-extra] r314587 - [clangd] simplify ClangdLSPServer by private-inheriting callback interfaces. NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Sat Sep 30 03:08:52 2017
New Revision: 314587

URL: http://llvm.org/viewvc/llvm-project?rev=314587&view=rev
Log:
[clangd] simplify ClangdLSPServer by private-inheriting callback interfaces. NFC

Summary:
There doesn't seem to be any real separation between the current three objects.
Feel free to reject this if you find the current style valuable, though.

(Mostly I'm just looking around for cleanups to help me understand the code).

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D38414

Modified:
clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
clang-tools-extra/trunk/clangd/ClangdLSPServer.h

Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=314587&r1=314586&r2=314587&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp Sat Sep 30 03:08:52 2017
@@ -9,7 +9,6 @@
 
 #include "ClangdLSPServer.h"
 #include "JSONRPCDispatcher.h"
-#include "ProtocolHandlers.h"
 
 using namespace clang::clangd;
 using namespace clang;
@@ -38,50 +37,8 @@ replacementsToEdits(StringRef Code,
 
 } // namespace
 
-ClangdLSPServer::LSPDiagnosticsConsumer::LSPDiagnosticsConsumer(
-ClangdLSPServer &Server)
-: Server(Server) {}
-
-void ClangdLSPServer::LSPDiagnosticsConsumer::onDiagnosticsReady(
-PathRef File, Tagged> Diagnostics) {
-  Server.consumeDiagnostics(File, Diagnostics.Value);
-}
-
-class ClangdLSPServer::LSPProtocolCallbacks : public ProtocolCallbacks {
-public:
-  LSPProtocolCallbacks(ClangdLSPServer &LangServer) : LangServer(LangServer) {}
-
-  void onInitialize(StringRef ID, InitializeParams IP,
-JSONOutput &Out) override;
-  void onShutdown(JSONOutput &Out) override;
-  void onDocumentDidOpen(DidOpenTextDocumentParams Params,
- JSONOutput &Out) override;
-  void onDocumentDidChange(DidChangeTextDocumentParams Params,
-   JSONOutput &Out) override;
-  void onDocumentDidClose(DidCloseTextDocumentParams Params,
-  JSONOutput &Out) override;
-  void onDocumentOnTypeFormatting(DocumentOnTypeFormattingParams Params,
-  StringRef ID, JSONOutput &Out) override;
-  void onDocumentRangeFormatting(DocumentRangeFormattingParams Params,
- StringRef ID, JSONOutput &Out) override;
-  void onDocumentFormatting(DocumentFormattingParams Params, StringRef ID,
-JSONOutput &Out) override;
-  void onCodeAction(CodeActionParams Params, StringRef ID,
-JSONOutput &Out) override;
-  void onCompletion(TextDocumentPositionParams Params, StringRef ID,
-JSONOutput &Out) override;
-  void onGoToDefinition(TextDocumentPositionParams Params, StringRef ID,
-JSONOutput &Out) override;
-  void onSwitchSourceHeader(TextDocumentIdentifier Params, StringRef ID,
-JSONOutput &Out) override;  
-
-private:
-  ClangdLSPServer &LangServer;
-};
-
-void ClangdLSPServer::LSPProtocolCallbacks::onInitialize(StringRef ID,
- InitializeParams IP,
- JSONOutput &Out) {
+void ClangdLSPServer::onInitialize(StringRef ID, InitializeParams IP,
+   JSONOutput &Out) {
   Out.writeMessage(
   R"({"jsonrpc":"2.0","id":)" + ID +
   R"(,"result":{"capabilities":{
@@ -94,79 +51,74 @@ void ClangdLSPServer::LSPProtocolCallbac
   "definitionProvider": true
 }}})");
   if (IP.rootUri && !IP.rootUri->file.empty())
-LangServer.Server.setRootPath(IP.rootUri->file);
+Server.setRootPath(IP.rootUri->file);
   else if (IP.rootPath && !IP.rootPath->empty())
-LangServer.Server.setRootPath(*IP.rootPath);
+Server.setRootPath(*IP.rootPath);
 }
 
-void ClangdLSPServer::LSPProtocolCallbacks::onShutdown(JSONOutput &Out) {
-  LangServer.IsDone = true;
-}
+void ClangdLSPServer::onShutdown(JSONOutput &Out) { IsDone = true; }
 
-void ClangdLSPServer::LSPProtocolCallbacks::onDocumentDidOpen(
-DidOpenTextDocumentParams Params, JSONOutput &Out) {
+void ClangdLSPServer::onDocumentDidOpen(DidOpenTextDocumentParams Params,
+JSONOutput &Out) {
   if (Params.metadata && !Params.metadata->extraFlags.empty())
-LangServer.CDB.setExtraFlagsForFile(Params.textDocument.uri.file,
-
std::move(Params.metadata->extraFlags));
-  LangServer.Server.addDocument(Params.textDocument.uri.file,
-Params.textDocument.text);
+CDB.setExtraFlagsForFile(Params.textDocument.uri.file,
+  

Re: [PATCH] D38464: [clangd] less boilerplate in RPC dispatch

[Sam McCall via cfe-commits]

Interesting - this is pretty primitive, and still fairly tightly coupled to
JSON-RPC.
I can't easily tell from the code how the ORC RPC functionality - would it
be easy to use with JSON-RPC, does it make sense to use serialization only,
does it have opinions about threading models? And really, what would we
expect to gain vs using the current more naive code (it looks more
complicated, probably having more complex requirements).

On Mon, Oct 9, 2017 at 11:16 PM, David Blaikie  wrote:

> hey Lang (& folks here) any chance there's some overlap between the RPC
> functionality here and the RPC functionality in ORC that could be
> deduplicated/refactored?
>
> On Fri, Oct 6, 2017 at 5:30 AM Ilya Biryukov via Phabricator via
> cfe-commits  wrote:
>
>> ilya-biryukov accepted this revision.
>> ilya-biryukov added a comment.
>> This revision is now accepted and ready to land.
>>
>> LGTM.
>> Note there's a new LSP method handler added upstream
>> (`textDocument/signatureHelp`), we should add it to this change before
>> submitting.
>>
>>
>>
>> 
>> Comment at: clangd/ClangdLSPServer.h:47
>>// Implement ProtocolCallbacks.
>> -  void onInitialize(StringRef ID, InitializeParams IP,
>> -JSONOutput &Out) override;
>> -  void onShutdown(JSONOutput &Out) override;
>> -  void onDocumentDidOpen(DidOpenTextDocumentParams Params,
>> - JSONOutput &Out) override;
>> -  void onDocumentDidChange(DidChangeTextDocumentParams Params,
>> -   JSONOutput &Out) override;
>> -  void onDocumentDidClose(DidCloseTextDocumentParams Params,
>> -  JSONOutput &Out) override;
>> -  void onDocumentOnTypeFormatting(DocumentOnTypeFormattingParams Params,
>> -  StringRef ID, JSONOutput &Out)
>> override;
>> -  void onDocumentRangeFormatting(DocumentRangeFormattingParams Params,
>> - StringRef ID, JSONOutput &Out) override;
>> -  void onDocumentFormatting(DocumentFormattingParams Params, StringRef
>> ID,
>> -JSONOutput &Out) override;
>> -  void onCodeAction(CodeActionParams Params, StringRef ID,
>> -JSONOutput &Out) override;
>> -  void onCompletion(TextDocumentPositionParams Params, StringRef ID,
>> -JSONOutput &Out) override;
>> -  void onGoToDefinition(TextDocumentPositionParams Params, StringRef ID,
>> -JSONOutput &Out) override;
>> -  void onSwitchSourceHeader(TextDocumentIdentifier Params, StringRef ID,
>> -JSONOutput &Out) override;
>> +  void onInitialize(Ctx &C, InitializeParams &Params) override;
>> +  void onShutdown(Ctx &C, NoParams &Params) override;
>> 
>> sammccall wrote:
>> > ilya-biryukov wrote:
>> > > ilya-biryukov wrote:
>> > > > Maybe there's a way to have a typed return value instead of `Ctx`?
>> > > > This would give an interface that's harder to misuse: one can't
>> forget to call `reply` or call it twice.
>> > > >
>> > > > Here's on design that comes to mind.
>> > > > Notification handlers could have `void` return, normal requests can
>> return `Optional` or `Optional` (with result json).
>> > > > `Optional` is be used to indicate whether error occurred while
>> processing the request.
>> > > >
>> > > After putting more thought into it, `Ctx`-based API seems to have an
>> advantage: it will allow us to easily implement async responses.
>> > > E.g., we can run code completion on a background thread and continue
>> processing other requests. When completion is ready, we will simply call
>> `Ctx.reply` to return results to the language client.
>> > >
>> > > To make that possible, can we allow moving `RequestContext` and pass
>> it by-value instead of by-ref?
>> > Yeah I thought about returning a value... it certainly reads more
>> nicely, but I don't think we're ready to do a good job in this patch:
>> >  - return value should be an object ready to be serialized (rather than
>> a JSON string) - I don't want to bring that in scope here, but it might
>> affect the details of the API
>> >  - there's several cases we know about (return object, no reply, error
>> reply) and some we're not sure about (async as you mention - any multiple
>> responses)? I think this needs some design, and I don't yet know the
>> project well enough to drive it.
>> >
>> > I've switched to passing Ctx by value as you suggest (though it's
>> certainly cheap enough to copy, too).
>> Yeah, copy is also fine there performance-wise.
>>
>> I do think move-only interface fits slightly better. We can check a whole
>> bunch of invariants if `Ctx` is move-only (i.e., that request wasn't
>> dropped without response or `reply` was not called twice).
>>
>>
>> 
>> Comment at: clangd/ClangdLSPServer.h:48
>> +  void onInitialize(Ctx &C, InitializeParams &Params) override;
>> +  void onShutdown(Ctx &C, NoParams &Params) override;
>> +  void onDocumentDid

[clang-tools-extra] r315577 - [clangd] less boilerplate in RPC dispatch

[Sam McCall via cfe-commits]

Author: sammccall
Date: Thu Oct 12 06:29:58 2017
New Revision: 315577

URL: http://llvm.org/viewvc/llvm-project?rev=315577&view=rev
Log:
[clangd] less boilerplate in RPC dispatch

Summary:
Make the ProtocolHandlers glue between JSONRPCDispatcher and
ClangdLSPServer generic.
Eliminate small differences between methods, de-emphasize the unimportant
distinction between notifications and methods.

ClangdLSPServer is no longer responsible for producing a complete
JSON-RPC response, just the JSON of the result object. (In future, we
should move that JSON serialization out, too).
Handler methods now take a context object that we may hang more
functionality off in the future.

Added documentation to ProtocolHandlers.

Reviewers: ilya-biryukov, bkramer

Reviewed By: ilya-biryukov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D38464

Modified:
clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
clang-tools-extra/trunk/clangd/ClangdLSPServer.h
clang-tools-extra/trunk/clangd/JSONRPCDispatcher.cpp
clang-tools-extra/trunk/clangd/JSONRPCDispatcher.h
clang-tools-extra/trunk/clangd/Protocol.cpp
clang-tools-extra/trunk/clangd/Protocol.h
clang-tools-extra/trunk/clangd/ProtocolHandlers.cpp
clang-tools-extra/trunk/clangd/ProtocolHandlers.h
clang-tools-extra/trunk/test/clangd/did-change-watch-files.test
clang-tools-extra/trunk/test/clangd/fixits.test

Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=315577&r1=315576&r2=315577&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp Thu Oct 12 06:29:58 2017
@@ -37,11 +37,9 @@ replacementsToEdits(StringRef Code,
 
 } // namespace
 
-void ClangdLSPServer::onInitialize(StringRef ID, InitializeParams IP,
-   JSONOutput &Out) {
-  Out.writeMessage(
-  R"({"jsonrpc":"2.0","id":)" + ID +
-  R"(,"result":{"capabilities":{
+void ClangdLSPServer::onInitialize(Ctx C, InitializeParams &Params) {
+  C.reply(
+  R"({"capabilities":{
   "textDocumentSync": 1,
   "documentFormattingProvider": true,
   "documentRangeFormattingProvider": true,
@@ -50,73 +48,68 @@ void ClangdLSPServer::onInitialize(Strin
   "completionProvider": {"resolveProvider": false, 
"triggerCharacters": [".",">",":"]},
   "signatureHelpProvider": {"triggerCharacters": ["(",","]},
   "definitionProvider": true
-}}})");
-  if (IP.rootUri && !IP.rootUri->file.empty())
-Server.setRootPath(IP.rootUri->file);
-  else if (IP.rootPath && !IP.rootPath->empty())
-Server.setRootPath(*IP.rootPath);
+}})");
+  if (Params.rootUri && !Params.rootUri->file.empty())
+Server.setRootPath(Params.rootUri->file);
+  else if (Params.rootPath && !Params.rootPath->empty())
+Server.setRootPath(*Params.rootPath);
 }
 
-void ClangdLSPServer::onShutdown(JSONOutput &Out) { IsDone = true; }
+void ClangdLSPServer::onShutdown(Ctx C, ShutdownParams &Params) {
+  IsDone = true;
+}
 
-void ClangdLSPServer::onDocumentDidOpen(DidOpenTextDocumentParams Params,
-JSONOutput &Out) {
+void ClangdLSPServer::onDocumentDidOpen(Ctx C,
+DidOpenTextDocumentParams &Params) {
   if (Params.metadata && !Params.metadata->extraFlags.empty())
 CDB.setExtraFlagsForFile(Params.textDocument.uri.file,
  std::move(Params.metadata->extraFlags));
   Server.addDocument(Params.textDocument.uri.file, Params.textDocument.text);
 }
 
-void ClangdLSPServer::onDocumentDidChange(DidChangeTextDocumentParams Params,
-  JSONOutput &Out) {
+void ClangdLSPServer::onDocumentDidChange(Ctx C,
+  DidChangeTextDocumentParams &Params) 
{
   // We only support full syncing right now.
   Server.addDocument(Params.textDocument.uri.file,
  Params.contentChanges[0].text);
 }
 
-void ClangdLSPServer::onFileEvent(const DidChangeWatchedFilesParams &Params) {
+void ClangdLSPServer::onFileEvent(Ctx C, DidChangeWatchedFilesParams &Params) {
   Server.onFileEvent(Params);
 }
 
-void ClangdLSPServer::onDocumentDidClose(DidCloseTextDocumentParams Params,
- JSONOutput &Out) {
+void ClangdLSPServer::onDocumentDidClose(Ctx C,
+ DidCloseTextDocumentParams &Params) {
   Server.removeDocument(Params.textDocument.uri.file);
 }
 
 void ClangdLSPServer::onDocumentOnTypeFormatting(
-DocumentOnTypeFormattingParams Params, StringRef ID, JSONOutput &Out) {
+Ctx C, DocumentOnTypeFormattingParams &Params) {
   auto File = Params.textDocument.uri.file;
   std::string Code = Server.getDocument(

[clang-tools-extra] r341076 - [clangd] Run SignatureHelp using an up-to-date preamble, waiting if needed.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Thu Aug 30 08:07:34 2018
New Revision: 341076

URL: http://llvm.org/viewvc/llvm-project?rev=341076&view=rev
Log:
[clangd] Run SignatureHelp using an up-to-date preamble, waiting if needed.

Summary:
After code completion inserts a header, running signature help using the old
preamble will usually fail. So we add support for consistent preamble reads.

Reviewers: ilya-biryukov

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, 
cfe-commits

Differential Revision: https://reviews.llvm.org/D51438

Modified:
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/TUScheduler.cpp
clang-tools-extra/trunk/clangd/TUScheduler.h
clang-tools-extra/trunk/unittests/clangd/TUSchedulerTests.cpp

Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=341076&r1=341075&r2=341076&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Thu Aug 30 08:07:34 2018
@@ -230,7 +230,8 @@ TaskHandle ClangdServer::codeComplete(Pa
 // is called as soon as results are available.
   };
 
-  WorkScheduler.runWithPreamble("CodeComplete", File,
+  // We use a potentially-stale preamble because latency is critical here.
+  WorkScheduler.runWithPreamble("CodeComplete", File, TUScheduler::Stale,
 Bind(Task, File.str(), std::move(CB)));
   return TH;
 }
@@ -252,7 +253,11 @@ void ClangdServer::signatureHelp(PathRef
  IP->Contents, Pos, FS, PCHs, Index));
   };
 
-  WorkScheduler.runWithPreamble("SignatureHelp", File,
+  // Unlike code completion, we wait for an up-to-date preamble here.
+  // Signature help is often triggered after code completion. If the code
+  // completion inserted a header to make the symbol available, then using
+  // the old preamble would yield useless results.
+  WorkScheduler.runWithPreamble("SignatureHelp", File, TUScheduler::Consistent,
 Bind(Action, File.str(), std::move(CB)));
 }
 

Modified: clang-tools-extra/trunk/clangd/TUScheduler.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/TUScheduler.cpp?rev=341076&r1=341075&r2=341076&view=diff
==
--- clang-tools-extra/trunk/clangd/TUScheduler.cpp (original)
+++ clang-tools-extra/trunk/clangd/TUScheduler.cpp Thu Aug 30 08:07:34 2018
@@ -183,6 +183,10 @@ public:
   bool blockUntilIdle(Deadline Timeout) const;
 
   std::shared_ptr getPossiblyStalePreamble() const;
+  /// Obtain a preamble reflecting all updates so far. Threadsafe.
+  /// It may be delivered immediately, or later on the worker thread.
+  void getCurrentPreamble(
+  llvm::unique_function)>);
   /// Wait for the first build of preamble to finish. Preamble itself can be
   /// accessed via getPossibleStalePreamble(). Note that this function will
   /// return after an unsuccessful build of the preamble too, i.e. result of
@@ -464,6 +468,34 @@ ASTWorker::getPossiblyStalePreamble() co
   return LastBuiltPreamble;
 }
 
+void ASTWorker::getCurrentPreamble(
+llvm::unique_function)> Callback) 
{
+  // We could just call startTask() to throw the read on the queue, knowing
+  // it will run after any updates. But we know this task is cheap, so to
+  // improve latency we cheat: insert it on the queue after the last update.
+  std::unique_lock Lock(Mutex);
+  auto LastUpdate =
+  std::find_if(Requests.rbegin(), Requests.rend(),
+   [](const Request &R) { return R.UpdateType.hasValue(); });
+  // If there were no writes in the queue, the preamble is ready now.
+  if (LastUpdate == Requests.rend()) {
+Lock.unlock();
+return Callback(getPossiblyStalePreamble());
+  }
+  assert(!RunSync && "Running synchronously, but queue is non-empty!");
+  Requests.insert(LastUpdate.base(),
+  Request{Bind(
+  [this](decltype(Callback) Callback) {
+Callback(getPossiblyStalePreamble());
+  },
+  std::move(Callback)),
+  "GetPreamble", steady_clock::now(),
+  Context::current().clone(),
+  /*UpdateType=*/llvm::None});
+  Lock.unlock();
+  RequestsCV.notify_all();
+}
+
 void ASTWorker::waitForFirstPreamble() const {
   PreambleWasBuilt.wait();
 }
@@ -711,7 +743,7 @@ void TUScheduler::runWithAST(
 }
 
 void TUScheduler::runWithPreamble(
-llvm::StringRef Name, PathRef File,
+llvm::StringRef Name, PathRef File, PreambleConsistency Consistency,
 llvm::unique_function)> Action) {
   auto It = Files.find(File);
   if (It == Files.end()) {
@@ -731,22 +763,40 @@ void TUScheduler::r

[clang-tools-extra] r341211 - [clangd] Flatten out Symbol::Details. It was ill-conceived, sorry.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Fri Aug 31 06:55:01 2018
New Revision: 341211

URL: http://llvm.org/viewvc/llvm-project?rev=341211&view=rev
Log:
[clangd] Flatten out Symbol::Details. It was ill-conceived, sorry.

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51504

Modified:
clang-tools-extra/trunk/clangd/CodeComplete.cpp

clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
clang-tools-extra/trunk/clangd/index/Index.cpp
clang-tools-extra/trunk/clangd/index/Index.h
clang-tools-extra/trunk/clangd/index/Merge.cpp
clang-tools-extra/trunk/clangd/index/Merge.h
clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp
clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp
clang-tools-extra/trunk/clangd/index/SymbolYAML.h
clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp
clang-tools-extra/trunk/unittests/clangd/FileIndexTests.cpp
clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp
clang-tools-extra/trunk/unittests/clangd/SymbolCollectorTests.cpp

Modified: clang-tools-extra/trunk/clangd/CodeComplete.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/CodeComplete.cpp?rev=341211&r1=341210&r2=341211&view=diff
==
--- clang-tools-extra/trunk/clangd/CodeComplete.cpp (original)
+++ clang-tools-extra/trunk/clangd/CodeComplete.cpp Fri Aug 31 06:55:01 2018
@@ -285,8 +285,7 @@ struct CompletionCandidate {
   }
 
   llvm::Optional headerToInsertIfNotPresent() const {
-if (!IndexResult || !IndexResult->Detail ||
-IndexResult->Detail->IncludeHeader.empty())
+if (!IndexResult || IndexResult->IncludeHeader.empty())
   return llvm::None;
 if (SemaResult && SemaResult->Declaration) {
   // Avoid inserting new #include if the declaration is found in the 
current
@@ -296,7 +295,7 @@ struct CompletionCandidate {
 if (SM.isInMainFile(SM.getExpansionLoc(RD->getBeginLoc(
   return llvm::None;
 }
-return IndexResult->Detail->IncludeHeader;
+return IndexResult->IncludeHeader;
   }
 
   using Bundle = llvm::SmallVector;
@@ -382,7 +381,7 @@ struct CodeCompletionBuilder {
 log("Failed to generate include insertion edits for adding header "
 "(FileURI='{0}', IncludeHeader='{1}') into {2}",
 C.IndexResult->CanonicalDeclaration.FileURI,
-C.IndexResult->Detail->IncludeHeader, FileName);
+C.IndexResult->IncludeHeader, FileName);
 }
   }
 
@@ -397,12 +396,11 @@ struct CodeCompletionBuilder {
 } else if (C.IndexResult) {
   S.Signature = C.IndexResult->Signature;
   S.SnippetSuffix = C.IndexResult->CompletionSnippetSuffix;
-  if (auto *D = C.IndexResult->Detail)
-S.ReturnType = D->ReturnType;
+  S.ReturnType = C.IndexResult->ReturnType;
 }
 if (ExtractDocumentation && Completion.Documentation.empty()) {
-  if (C.IndexResult && C.IndexResult->Detail)
-Completion.Documentation = C.IndexResult->Detail->Documentation;
+  if (C.IndexResult)
+Completion.Documentation = C.IndexResult->Documentation;
   else if (C.SemaResult)
 Completion.Documentation = getDocComment(ASTCtx, *C.SemaResult,
  /*CommentsFromHeader=*/false);
@@ -846,9 +844,8 @@ public:
 IndexRequest.IDs.insert(*S.IDForDoc);
   }
   Index->lookup(IndexRequest, [&](const Symbol &S) {
-if (!S.Detail || S.Detail->Documentation.empty())
-  return;
-FetchedDocs[S.ID] = S.Detail->Documentation;
+if (!S.Documentation.empty())
+  FetchedDocs[S.ID] = S.Documentation;
   });
   log("SigHelp: requested docs for {0} symbols from the index, got {1} "
   "symbols with non-empty docs in the response",

Modified: 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp?rev=341211&r1=341210&r2=341211&view=diff
==
--- 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
 (original)
+++ 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
 Fri Aug 31 06:55:01 2018
@@ -160,17 +160,14 @@ public:
 
   SymbolSlab mergeResults() override {
 SymbolSlab::Builder UniqueSymbols;
-llvm::BumpPtrAllocator Arena;
-Symbol::Details Scratch;
 Executor.getToolResults()->forEachResult(
 [&](llvm::StringRef Key, llvm::StringRef Value) {
-  Arena.Reset();
-  llvm::yaml::Input Yin(Value, &Arena);
-  auto Sym = clang::clangd::SymbolFromYAML(Yin, Arena);
+  llvm::yaml::Input Yin(Value);
+  auto Sym = clang:

[clang-tools-extra] r341318 - [clangd] Factor out the data-swapping functionality from MemIndex/DexIndex.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Sep  3 07:37:43 2018
New Revision: 341318

URL: http://llvm.org/viewvc/llvm-project?rev=341318&view=rev
Log:
[clangd] Factor out the data-swapping functionality from MemIndex/DexIndex.

Summary:
This is now handled by a wrapper class SwapIndex, so MemIndex/DexIndex can be
immutable and focus on their job.

Old and busted:
 I have a MemIndex, which holds a shared_ptr>, which keeps the
 symbol slab alive. I update by calling build(shared_ptr>).

New hotness: I have a SwapIndex, which holds a unique_ptr, which
 holds a MemIndex, which holds a shared_ptr, which keeps backing
 data alive.
 I update by building a new MemIndex and calling SwapIndex::reset().

Reviewers: kbobyrev, ioeric

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, mgrang, arphaman, 
kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51422

Modified:
clang-tools-extra/trunk/clangd/index/FileIndex.cpp
clang-tools-extra/trunk/clangd/index/FileIndex.h
clang-tools-extra/trunk/clangd/index/Index.cpp
clang-tools-extra/trunk/clangd/index/Index.h
clang-tools-extra/trunk/clangd/index/MemIndex.cpp
clang-tools-extra/trunk/clangd/index/MemIndex.h
clang-tools-extra/trunk/clangd/index/dex/DexIndex.cpp
clang-tools-extra/trunk/clangd/index/dex/DexIndex.h
clang-tools-extra/trunk/unittests/clangd/DexIndexTests.cpp
clang-tools-extra/trunk/unittests/clangd/FileIndexTests.cpp
clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp
clang-tools-extra/trunk/unittests/clangd/TestIndex.cpp
clang-tools-extra/trunk/unittests/clangd/TestIndex.h

Modified: clang-tools-extra/trunk/clangd/index/FileIndex.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/FileIndex.cpp?rev=341318&r1=341317&r2=341318&view=diff
==
--- clang-tools-extra/trunk/clangd/index/FileIndex.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/FileIndex.cpp Mon Sep  3 07:37:43 2018
@@ -71,7 +71,9 @@ indexAST(ASTContext &AST, std::shared_pt
 }
 
 FileIndex::FileIndex(std::vector URISchemes)
-: URISchemes(std::move(URISchemes)) {}
+: URISchemes(std::move(URISchemes)) {
+  reset(FSymbols.buildMemIndex());
+}
 
 void FileSymbols::update(PathRef Path, std::unique_ptr Slab,
  std::unique_ptr Occurrences) {
@@ -86,52 +88,32 @@ void FileSymbols::update(PathRef Path, s
 FileToOccurrenceSlabs[Path] = std::move(Occurrences);
 }
 
-std::shared_ptr> FileSymbols::allSymbols() {
-  // The snapshot manages life time of symbol slabs and provides pointers of 
all
-  // symbols in all slabs.
-  struct Snapshot {
-std::vector Pointers;
-std::vector> KeepAlive;
-  };
-  auto Snap = std::make_shared();
+std::unique_ptr FileSymbols::buildMemIndex() {
+  std::vector> Slabs;
+  std::vector> OccurrenceSlabs;
   {
 std::lock_guard Lock(Mutex);
-
-for (const auto &FileAndSlab : FileToSlabs) {
-  Snap->KeepAlive.push_back(FileAndSlab.second);
-  for (const auto &Iter : *FileAndSlab.second)
-Snap->Pointers.push_back(&Iter);
-}
+for (const auto &FileAndSlab : FileToSlabs)
+  Slabs.push_back(FileAndSlab.second);
+for (const auto &FileAndOccurrenceSlab : FileToOccurrenceSlabs)
+  OccurrenceSlabs.push_back(FileAndOccurrenceSlab.second);
   }
-  auto *Pointers = &Snap->Pointers;
-  // Use aliasing constructor to keep the snapshot alive along with the
-  // pointers.
-  return {std::move(Snap), Pointers};
-}
-
-std::shared_ptr FileSymbols::allOccurrences() const {
-  // The snapshot manages life time of symbol occurrence slabs and provides
-  // pointers to all occurrences in all occurrence slabs.
-  struct Snapshot {
-MemIndex::OccurrenceMap Occurrences; // ID => {Occurrence}
-std::vector> KeepAlive;
-  };
-
-  auto Snap = std::make_shared();
-  {
-std::lock_guard Lock(Mutex);
-
-for (const auto &FileAndSlab : FileToOccurrenceSlabs) {
-  Snap->KeepAlive.push_back(FileAndSlab.second);
-  for (const auto &IDAndOccurrences : *FileAndSlab.second) {
-auto &Occurrences = Snap->Occurrences[IDAndOccurrences.first];
-for (const auto &Occurrence : IDAndOccurrences.second)
-  Occurrences.push_back(&Occurrence);
-  }
+  std::vector AllSymbols;
+  for (const auto &Slab : Slabs)
+for (const auto &Sym : *Slab)
+  AllSymbols.push_back(&Sym);
+  MemIndex::OccurrenceMap AllOccurrences;
+  for (const auto &OccurrenceSlab : OccurrenceSlabs)
+for (const auto &Sym : *OccurrenceSlab) {
+  auto &Entry = AllOccurrences[Sym.first];
+  for (const auto &Occ : Sym.second)
+Entry.push_back(&Occ);
 }
-  }
 
-  return {std::move(Snap), &Snap->Occurrences};
+  // Index must keep the slabs alive.
+  return llvm::make_unique(
+  llvm::make_pointee_range(AllSymbols), std::move(AllOccurrences),
+  std::make_pair(std::move(Slabs), std::move(OccurrenceSlabs)));
 }
 
 void FileIndex::upda

[clang-tools-extra] r341321 - [clangd] Fix ambiguous make_unique with c++17. NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Sep  3 08:23:01 2018
New Revision: 341321

URL: http://llvm.org/viewvc/llvm-project?rev=341321&view=rev
Log:
[clangd] Fix ambiguous make_unique with c++17. NFC

Modified:
clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp

Modified: clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp?rev=341321&r1=341320&r2=341321&view=diff
==
--- clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp (original)
+++ clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp Mon Sep  3 08:23:01 
2018
@@ -56,11 +56,11 @@ TEST(SwapIndexTest, OldIndexRecycled) {
   auto Token = std::make_shared();
   std::weak_ptr WeakToken = Token;
 
-  SwapIndex S(make_unique(SymbolSlab(), MemIndex::OccurrenceMap(),
-std::move(Token)));
-  EXPECT_FALSE(WeakToken.expired()); // Current MemIndex keeps it alive.
-  S.reset(make_unique());  // Now the MemIndex is destroyed.
-  EXPECT_TRUE(WeakToken.expired());  // So the token is too.
+  SwapIndex S(llvm::make_unique(
+  SymbolSlab(), MemIndex::OccurrenceMap(), std::move(Token)));
+  EXPECT_FALSE(WeakToken.expired()); // Current MemIndex keeps it alive.
+  S.reset(llvm::make_unique()); // Now the MemIndex is destroyed.
+  EXPECT_TRUE(WeakToken.expired());  // So the token is too.
 }
 
 TEST(MemIndexTest, MemIndexDeduplicate) {


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r341325 - [clangd] Some nitpicking around the new split (preamble/main) dynamic index

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Sep  3 09:37:59 2018
New Revision: 341325

URL: http://llvm.org/viewvc/llvm-project?rev=341325&view=rev
Log:
[clangd] Some nitpicking around the new split (preamble/main) dynamic index

Summary:
- DynamicIndex doesn't implement ParsingCallbacks, to make its role clearer.
  ParsingCallbacks is a separate object owned by the receiving TUScheduler.
  (I tried to get rid of the "index-like-object that doesn't implement index"
  but it was too messy).
- Clarified(?) docs around DynamicIndex - fewer details up front, more details
  inside.
- Exposed dynamic index from ClangdServer for memory monitoring and more
  direct testing of its contents (actual tests not added here, wanted to get
  this out for review)
- Removed a redundant and sligthly confusing filename param in a callback

Reviewers: ilya-biryukov

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, 
cfe-commits

Differential Revision: https://reviews.llvm.org/D51221

Modified:
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.h
clang-tools-extra/trunk/clangd/ClangdUnit.cpp
clang-tools-extra/trunk/clangd/ClangdUnit.h
clang-tools-extra/trunk/clangd/CodeComplete.cpp
clang-tools-extra/trunk/clangd/CodeComplete.h
clang-tools-extra/trunk/clangd/TUScheduler.cpp
clang-tools-extra/trunk/clangd/TUScheduler.h
clang-tools-extra/trunk/unittests/clangd/FileIndexTests.cpp
clang-tools-extra/trunk/unittests/clangd/TUSchedulerTests.cpp

Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=341325&r1=341324&r2=341325&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Mon Sep  3 09:37:59 2018
@@ -71,34 +71,57 @@ public:
 };
 } // namespace
 
-/// Manages dynamic index for open files. Each file might contribute two sets
-/// of symbols to the dynamic index: symbols from the preamble and symbols
-/// from the file itself. Those have different lifetimes and we merge results
-/// from both
-class ClangdServer::DynamicIndex : public ParsingCallbacks {
+/// The dynamic index tracks symbols visible in open files.
+/// For boring reasons, it doesn't implement SymbolIndex directly - use 
index().
+class ClangdServer::DynamicIndex {
 public:
   DynamicIndex(std::vector URISchemes)
   : PreambleIdx(URISchemes), MainFileIdx(URISchemes),
 MergedIndex(mergeIndex(&MainFileIdx, &PreambleIdx)) {}
 
-  SymbolIndex &index() const { return *MergedIndex; }
+  const SymbolIndex &index() const { return *MergedIndex; }
 
-  void onPreambleAST(PathRef Path, ASTContext &Ctx,
- std::shared_ptr PP) override {
-PreambleIdx.update(Path, &Ctx, PP, /*TopLevelDecls=*/llvm::None);
-  }
-
-  void onMainAST(PathRef Path, ParsedAST &AST) override {
-
-MainFileIdx.update(Path, &AST.getASTContext(), AST.getPreprocessorPtr(),
-   AST.getLocalTopLevelDecls());
-  }
+  // Returns callbacks that can be used to update the index with new ASTs.
+  // Index() presents a merged view of the supplied main-file and preamble 
ASTs.
+  std::unique_ptr makeUpdateCallbacks() {
+struct CB : public ParsingCallbacks {
+  CB(ClangdServer::DynamicIndex *This) : This(This) {}
+  DynamicIndex *This;
+
+  void onPreambleAST(PathRef Path, ASTContext &Ctx,
+ std::shared_ptr PP) override {
+This->PreambleIdx.update(Path, &Ctx, std::move(PP));
+  }
+
+  void onMainAST(PathRef Path, ParsedAST &AST) override {
+This->MainFileIdx.update(Path, &AST.getASTContext(),
+ AST.getPreprocessorPtr(),
+ AST.getLocalTopLevelDecls());
+  }
+};
+return llvm::make_unique(this);
+  };
 
 private:
+  // Contains information from each file's preamble only.
+  // These are large, but update fairly infrequently (preambles are stable).
+  // Missing information:
+  //  - symbol occurrences (these are always "from the main file")
+  //  - definition locations in the main file
+  //
+  // FIXME: Because the preambles for different TUs have large overlap and
+  // FileIndex doesn't deduplicate, this uses lots of extra RAM.
+  // The biggest obstacle in fixing this: the obvious approach of partitioning
+  // by declaring file (rather than main file) fails if headers provide
+  // different symbols based on preprocessor state.
   FileIndex PreambleIdx;
+  // Contains information from each file's main AST.
+  // These are updated frequently (on file change), but are relatively small.
+  // Mostly contains:
+  //  - occurrences of symbols declared in the preamble and referenced from 
main
+  //  - symbols declared both in the main file and the preamble
+  // (Note that symbols *only* in the main 

[clang-tools-extra] r341337 - [clangd] Fix index-twice regression from r341242

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Sep  3 13:26:26 2018
New Revision: 341337

URL: http://llvm.org/viewvc/llvm-project?rev=341337&view=rev
Log:
[clangd] Fix index-twice regression from r341242

Modified:
clang-tools-extra/trunk/clangd/index/FileIndex.cpp

Modified: clang-tools-extra/trunk/clangd/index/FileIndex.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/FileIndex.cpp?rev=341337&r1=341336&r2=341337&view=diff
==
--- clang-tools-extra/trunk/clangd/index/FileIndex.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/FileIndex.cpp Mon Sep  3 13:26:26 2018
@@ -125,7 +125,6 @@ void FileIndex::update(PathRef Path, AST
 assert(PP);
 auto Slab = llvm::make_unique();
 auto OccurrenceSlab = llvm::make_unique();
-auto IndexResults = indexAST(*AST, PP, TopLevelDecls, URISchemes);
 std::tie(*Slab, *OccurrenceSlab) =
 indexAST(*AST, PP, TopLevelDecls, URISchemes);
 FSymbols.update(Path, std::move(Slab), std::move(OccurrenceSlab));


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r341368 - [clangd] SymbolOccurrences -> Refs and cleanup

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Sep  4 07:39:56 2018
New Revision: 341368

URL: http://llvm.org/viewvc/llvm-project?rev=341368&view=rev
Log:
[clangd] SymbolOccurrences -> Refs and cleanup

Summary:
A few things that I noticed while merging the SwapIndex patch:
 - SymbolOccurrences and particularly SymbolOccurrenceSlab are unwieldy names,
   and these names appear *a lot*. Ref, RefSlab, etc seem clear enough
   and read/format much better.
 - The asymmetry between SymbolSlab and RefSlab (build() vs freeze()) is
   confusing and irritating, and doesn't even save much code.
   Avoiding RefSlab::Builder was my idea, but it was a bad one; add it.
 - DenseMap> seems like a reasonable compromise for
   constructing MemIndex - and means many less wasted allocations than the
   current DenseMap> for FileIndex, and none for
   slabs.
 - RefSlab::find() is not actually used for anything, so we can throw
   away the DenseMap and keep the representation much more compact.
 - A few naming/consistency fixes: e.g. Slabs,Refs -> Symbols,Refs.

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, mgrang, arphaman, kadircet, 
cfe-commits

Differential Revision: https://reviews.llvm.org/D51605

Modified:
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/index/FileIndex.cpp
clang-tools-extra/trunk/clangd/index/FileIndex.h
clang-tools-extra/trunk/clangd/index/Index.cpp
clang-tools-extra/trunk/clangd/index/Index.h
clang-tools-extra/trunk/clangd/index/MemIndex.cpp
clang-tools-extra/trunk/clangd/index/MemIndex.h
clang-tools-extra/trunk/clangd/index/Merge.cpp
clang-tools-extra/trunk/clangd/index/Merge.h
clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp
clang-tools-extra/trunk/clangd/index/SymbolCollector.h
clang-tools-extra/trunk/clangd/index/dex/DexIndex.cpp
clang-tools-extra/trunk/clangd/index/dex/DexIndex.h
clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp
clang-tools-extra/trunk/unittests/clangd/FileIndexTests.cpp
clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp
clang-tools-extra/trunk/unittests/clangd/SymbolCollectorTests.cpp
clang-tools-extra/trunk/unittests/clangd/TestTU.cpp

Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=341368&r1=341367&r2=341368&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Tue Sep  4 07:39:56 2018
@@ -106,7 +106,7 @@ private:
   // Contains information from each file's preamble only.
   // These are large, but update fairly infrequently (preambles are stable).
   // Missing information:
-  //  - symbol occurrences (these are always "from the main file")
+  //  - symbol refs (these are always "from the main file")
   //  - definition locations in the main file
   //
   // FIXME: Because the preambles for different TUs have large overlap and
@@ -118,7 +118,7 @@ private:
   // Contains information from each file's main AST.
   // These are updated frequently (on file change), but are relatively small.
   // Mostly contains:
-  //  - occurrences of symbols declared in the preamble and referenced from 
main
+  //  - refs to symbols declared in the preamble and referenced from main
   //  - symbols declared both in the main file and the preamble
   // (Note that symbols *only* in the main file are not indexed).
   FileIndex MainFileIdx;

Modified: clang-tools-extra/trunk/clangd/index/FileIndex.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/FileIndex.cpp?rev=341368&r1=341367&r2=341368&view=diff
==
--- clang-tools-extra/trunk/clangd/index/FileIndex.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/FileIndex.cpp Tue Sep  4 07:39:56 2018
@@ -16,7 +16,7 @@
 namespace clang {
 namespace clangd {
 
-std::pair
+std::pair
 indexAST(ASTContext &AST, std::shared_ptr PP,
  llvm::Optional> TopLevelDecls,
  llvm::ArrayRef URISchemes) {
@@ -45,12 +45,12 @@ indexAST(ASTContext &AST, std::shared_pt
 DeclsToIndex.assign(AST.getTranslationUnitDecl()->decls().begin(),
 AST.getTranslationUnitDecl()->decls().end());
 
-  // We only collect occurrences when indexing main AST.
+  // We only collect refs when indexing main AST.
   // FIXME: this is a hacky way to detect whether we are indexing preamble AST
   // or main AST, we should make it explicitly.
   bool IsIndexMainAST = TopLevelDecls.hasValue();
   if (IsIndexMainAST)
-CollectorOpts.OccurrenceFilter = AllOccurrenceKinds;
+CollectorOpts.RefFilter = RefKind::All;
 
   SymbolCollector Collector(std::move(CollectorOpts));
   Collector.setPreprocessor(PP);
@@ -61,13 +6

[clang-tools-extra] r341375 - [clangd] Define a compact binary serialization fomat for symbol slab/index.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Sep  4 09:16:50 2018
New Revision: 341375

URL: http://llvm.org/viewvc/llvm-project?rev=341375&view=rev
Log:
[clangd] Define a compact binary serialization fomat for symbol slab/index.

Summary:
This is intended to replace the current YAML format for general use.
It's ~10x more compact than YAML, and ~40% more compact than gzipped YAML:
  llvmidx.riff = 20M, llvmidx.yaml = 272M, llvmidx.yaml.gz = 32M
It's also simpler/faster to read and write.

The format is a RIFF container (chunks of (type, size, data)) with:
 - a compressed string table
 - simple binary encoding of symbols (with varints for compactness)
It can be extended to include occurrences, Dex posting lists, etc.

There's no rich backwards-compatibility scheme, but a version number is included
so we can detect incompatible files and do ad-hoc back-compat.

Alternatives considered:
 - compressed YAML or JSON: bulky and slow to load
 - llvm bitstream: confusing model and libraries are hard to use. My attempt
   produced slightly larger files, and the code was longer and slower.
 - protobuf or similar: would be really nice (esp for back-compat) but the
   dependency is a big hassle
 - ad-hoc binary format without a container: it seems clear we're going
   to add posting lists and occurrences here, and that they will benefit
   from sharing a string table. The container makes it easy to debug
   these pieces in isolation, and make them optional.

Reviewers: ioeric

Subscribers: mgorny, ilya-biryukov, MaskRay, jkorous, mgrang, arphaman, 
kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51585

Added:
clang-tools-extra/trunk/clangd/RIFF.cpp
clang-tools-extra/trunk/clangd/RIFF.h
clang-tools-extra/trunk/clangd/index/Serialization.cpp
clang-tools-extra/trunk/clangd/index/Serialization.h
clang-tools-extra/trunk/unittests/clangd/RIFFTests.cpp
clang-tools-extra/trunk/unittests/clangd/SerializationTests.cpp
Modified:
clang-tools-extra/trunk/clangd/CMakeLists.txt

clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
clang-tools-extra/trunk/clangd/index/Index.cpp
clang-tools-extra/trunk/clangd/index/Index.h
clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp
clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt
clang-tools-extra/trunk/unittests/clangd/SymbolCollectorTests.cpp

Modified: clang-tools-extra/trunk/clangd/CMakeLists.txt
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/CMakeLists.txt?rev=341375&r1=341374&r2=341375&view=diff
==
--- clang-tools-extra/trunk/clangd/CMakeLists.txt (original)
+++ clang-tools-extra/trunk/clangd/CMakeLists.txt Tue Sep  4 09:16:50 2018
@@ -29,6 +29,7 @@ add_clang_library(clangDaemon
   Protocol.cpp
   ProtocolHandlers.cpp
   Quality.cpp
+  RIFF.cpp
   SourceCode.cpp
   Threading.cpp
   Trace.cpp
@@ -41,6 +42,7 @@ add_clang_library(clangDaemon
   index/Index.cpp
   index/MemIndex.cpp
   index/Merge.cpp
+  index/Serialization.cpp
   index/SymbolCollector.cpp
   index/SymbolYAML.cpp
 

Added: clang-tools-extra/trunk/clangd/RIFF.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/RIFF.cpp?rev=341375&view=auto
==
--- clang-tools-extra/trunk/clangd/RIFF.cpp (added)
+++ clang-tools-extra/trunk/clangd/RIFF.cpp Tue Sep  4 09:16:50 2018
@@ -0,0 +1,88 @@
+//===--- RIFF.cpp - Binary container file format 
--===//
+//
+// The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===--===//
+
+#include "RIFF.h"
+#include "llvm/Support/Endian.h"
+
+using namespace llvm;
+namespace clang {
+namespace clangd {
+namespace riff {
+
+static Error makeError(const char *Msg) {
+  return createStringError(inconvertibleErrorCode(), Msg);
+}
+
+Expected readChunk(StringRef &Stream) {
+  if (Stream.size() < 8)
+return makeError("incomplete chunk header");
+  Chunk C;
+  std::copy(Stream.begin(), Stream.begin() + 4, C.ID.begin());
+  Stream = Stream.drop_front(4);
+  uint32_t Len = support::endian::read32le(Stream.take_front(4).begin());
+  Stream = Stream.drop_front(4);
+  if (Stream.size() < Len)
+return makeError("truncated chunk");
+  C.Data = Stream.take_front(Len);
+  Stream = Stream.drop_front(Len);
+  if (Len % 2 & !Stream.empty()) { // Skip padding byte.
+if (Stream.front())
+  return makeError("nonzero padding byte");
+Stream = Stream.drop_front();
+  }
+  return C;
+};
+
+raw_ostream &operator<<(raw_ostream &OS, const Chunk &C) {
+  OS.write(C.ID.begin(), C.ID.size());
+  char Size[4];
+  llvm::support::endian::write32le(Size, C.Dat

[clang-tools-extra] r341376 - [clangd] Load static index asynchronously, add tracing.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Sep  4 09:19:40 2018
New Revision: 341376

URL: http://llvm.org/viewvc/llvm-project?rev=341376&view=rev
Log:
[clangd] Load static index asynchronously, add tracing.

Summary:
Like D51475 but simplified based on recent patches.
While here, clarify that loadIndex() takes a filename, not file content.

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51638

Modified:
clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp
clang-tools-extra/trunk/clangd/index/SymbolYAML.h
clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp

Modified: clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp?rev=341376&r1=341375&r2=341376&view=diff
==
--- clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp Tue Sep  4 09:19:40 2018
@@ -8,6 +8,7 @@
 
//===--===//
 
 #include "SymbolYAML.h"
+#include "../Trace.h"
 #include "Index.h"
 #include "Serialization.h"
 #include "dex/DexIndex.h"
@@ -183,25 +184,31 @@ std::string SymbolToYAML(Symbol Sym) {
   return OS.str();
 }
 
-std::unique_ptr loadIndex(llvm::StringRef SymbolFile,
+std::unique_ptr loadIndex(llvm::StringRef SymbolFilename,
bool UseDex) {
-  auto Buffer = llvm::MemoryBuffer::getFile(SymbolFile);
+  trace::Span OverallTracer("LoadIndex");
+  auto Buffer = llvm::MemoryBuffer::getFile(SymbolFilename);
   if (!Buffer) {
-llvm::errs() << "Can't open " << SymbolFile << "\n";
+llvm::errs() << "Can't open " << SymbolFilename << "\n";
 return nullptr;
   }
   StringRef Data = Buffer->get()->getBuffer();
 
   llvm::Optional Slab;
   if (Data.startswith("RIFF")) { // Magic for binary index file.
+trace::Span Tracer("ParseRIFF");
 if (auto RIFF = readIndexFile(Data))
   Slab = std::move(RIFF->Symbols);
 else
   llvm::errs() << "Bad RIFF: " << llvm::toString(RIFF.takeError()) << "\n";
   } else {
+trace::Span Tracer("ParseYAML");
 Slab = symbolsFromYAML(Data);
   }
 
+  if (!Slab)
+return nullptr;
+  trace::Span Tracer("BuildIndex");
   return UseDex ? dex::DexIndex::build(std::move(*Slab))
 : MemIndex::build(std::move(*Slab), RefSlab());
 }

Modified: clang-tools-extra/trunk/clangd/index/SymbolYAML.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/SymbolYAML.h?rev=341376&r1=341375&r2=341376&view=diff
==
--- clang-tools-extra/trunk/clangd/index/SymbolYAML.h (original)
+++ clang-tools-extra/trunk/clangd/index/SymbolYAML.h Tue Sep  4 09:19:40 2018
@@ -44,7 +44,7 @@ void SymbolsToYAML(const SymbolSlab &Sym
 // Build an in-memory static index for global symbols from a symbol file.
 // The size of global symbols should be relatively small, so that all symbols
 // can be managed in memory.
-std::unique_ptr loadIndex(llvm::StringRef SymbolFile,
+std::unique_ptr loadIndex(llvm::StringRef SymbolFilename,
bool UseDex = true);
 
 } // namespace clangd

Modified: clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp?rev=341376&r1=341375&r2=341376&view=diff
==
--- clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp (original)
+++ clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp Tue Sep  4 09:19:40 2018
@@ -281,9 +281,15 @@ int main(int argc, char *argv[]) {
   Opts.BuildDynamicSymbolIndex = EnableIndex;
   std::unique_ptr StaticIdx;
   if (EnableIndex && !YamlSymbolFile.empty()) {
-StaticIdx = loadIndex(YamlSymbolFile, UseDex);
-Opts.StaticIndex = StaticIdx.get();
+// Load the index asynchronously. Meanwhile SwapIndex returns no results.
+SwapIndex *Placeholder;
+StaticIdx.reset(Placeholder = new 
SwapIndex(llvm::make_unique()));
+runAsync([Placeholder] {
+  if (auto Idx = loadIndex(YamlSymbolFile))
+Placeholder->reset(std::move(Idx));
+});
   }
+  Opts.StaticIndex = StaticIdx.get();
   Opts.AsyncThreadsCount = WorkerThreadsCount;
 
   clangd::CodeCompleteOptions CCOpts;


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r341450 - clang-format: Fix formatting C++ namespaces with preceding 'inline' or 'export' specifier

[Sam McCall via cfe-commits]

Author: sammccall
Date: Wed Sep  5 00:44:02 2018
New Revision: 341450

URL: http://llvm.org/viewvc/llvm-project?rev=341450&view=rev
Log:
clang-format: Fix formatting C++ namespaces with preceding 'inline' or 'export' 
specifier

This fixes formatting namespaces with preceding 'inline' and 'export' (Modules 
TS) specifiers.

This change fixes namespaces not being identified as such with preceding 
'inline' or 'export' specifiers.

Motivation: I was experimenting with the Modules TS (-fmodules-ts) and found it 
would be useful if clang-format would correctly format 'export namespace'. 
While making the changes, I noticed that similar issues still exist with 
'inline namespace', and addressed them as well.

Patch by Marco Elver!

Reviewers: klimek, djasper, owenpan, sammccall

Reviewed By: owenpan, sammccall

Subscribers: owenpan, cfe-commits

Differential Revision: https://reviews.llvm.org/D51036

Modified:
cfe/trunk/lib/Format/Format.cpp
cfe/trunk/lib/Format/FormatToken.h
cfe/trunk/lib/Format/NamespaceEndCommentsFixer.cpp
cfe/trunk/lib/Format/TokenAnnotator.h
cfe/trunk/lib/Format/UnwrappedLineFormatter.cpp
cfe/trunk/lib/Format/UnwrappedLineParser.cpp
cfe/trunk/unittests/Format/FormatTest.cpp

Modified: cfe/trunk/lib/Format/Format.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Format/Format.cpp?rev=341450&r1=341449&r2=341450&view=diff
==
--- cfe/trunk/lib/Format/Format.cpp (original)
+++ cfe/trunk/lib/Format/Format.cpp Wed Sep  5 00:44:02 2018
@@ -1309,8 +1309,7 @@ private:
 std::set DeletedLines;
 for (unsigned i = 0, e = AnnotatedLines.size(); i != e; ++i) {
   auto &Line = *AnnotatedLines[i];
-  if (Line.startsWith(tok::kw_namespace) ||
-  Line.startsWith(tok::kw_inline, tok::kw_namespace)) {
+  if (Line.startsWithNamespace()) {
 checkEmptyNamespace(AnnotatedLines, i, i, DeletedLines);
   }
 }
@@ -1347,9 +1346,7 @@ private:
   if (AnnotatedLines[CurrentLine]->startsWith(tok::r_brace))
 break;
 
-  if (AnnotatedLines[CurrentLine]->startsWith(tok::kw_namespace) ||
-  AnnotatedLines[CurrentLine]->startsWith(tok::kw_inline,
-  tok::kw_namespace)) {
+  if (AnnotatedLines[CurrentLine]->startsWithNamespace()) {
 if (!checkEmptyNamespace(AnnotatedLines, CurrentLine, NewLine,
  DeletedLines))
   return false;

Modified: cfe/trunk/lib/Format/FormatToken.h
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Format/FormatToken.h?rev=341450&r1=341449&r2=341450&view=diff
==
--- cfe/trunk/lib/Format/FormatToken.h (original)
+++ cfe/trunk/lib/Format/FormatToken.h Wed Sep  5 00:44:02 2018
@@ -520,8 +520,8 @@ struct FormatToken {
 const FormatToken *NamespaceTok = this;
 if (is(tok::comment))
   NamespaceTok = NamespaceTok->getNextNonComment();
-// Detect "(inline)? namespace" in the beginning of a line.
-if (NamespaceTok && NamespaceTok->is(tok::kw_inline))
+// Detect "(inline|export)? namespace" in the beginning of a line.
+if (NamespaceTok && NamespaceTok->isOneOf(tok::kw_inline, tok::kw_export))
   NamespaceTok = NamespaceTok->getNextNonComment();
 return NamespaceTok && NamespaceTok->is(tok::kw_namespace) ? NamespaceTok
: nullptr;

Modified: cfe/trunk/lib/Format/NamespaceEndCommentsFixer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Format/NamespaceEndCommentsFixer.cpp?rev=341450&r1=341449&r2=341450&view=diff
==
--- cfe/trunk/lib/Format/NamespaceEndCommentsFixer.cpp (original)
+++ cfe/trunk/lib/Format/NamespaceEndCommentsFixer.cpp Wed Sep  5 00:44:02 2018
@@ -125,12 +125,7 @@ getNamespaceToken(const AnnotatedLine *L
 if (StartLineIndex > 0)
   NamespaceTok = AnnotatedLines[StartLineIndex - 1]->First;
   }
-  // Detect "(inline)? namespace" in the beginning of a line.
-  if (NamespaceTok->is(tok::kw_inline))
-NamespaceTok = NamespaceTok->getNextNonComment();
-  if (!NamespaceTok || NamespaceTok->isNot(tok::kw_namespace))
-return nullptr;
-  return NamespaceTok;
+  return NamespaceTok->getNamespaceToken();
 }
 
 NamespaceEndCommentsFixer::NamespaceEndCommentsFixer(const Environment &Env,

Modified: cfe/trunk/lib/Format/TokenAnnotator.h
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Format/TokenAnnotator.h?rev=341450&r1=341449&r2=341450&view=diff
==
--- cfe/trunk/lib/Format/TokenAnnotator.h (original)
+++ cfe/trunk/lib/Format/TokenAnnotator.h Wed Sep  5 00:44:02 2018
@@ -105,6 +105,13 @@ public:
 return !Last->isOneOf(tok::semi, tok::comment);
   }
 
+  /// \c true if this 

[clang-tools-extra] r341451 - [clangd] Fix buildbot failures on older compilers from r341375

[Sam McCall via cfe-commits]

Author: sammccall
Date: Wed Sep  5 00:52:49 2018
New Revision: 341451

URL: http://llvm.org/viewvc/llvm-project?rev=341451&view=rev
Log:
[clangd] Fix buildbot failures on older compilers from r341375

Modified:
clang-tools-extra/trunk/clangd/RIFF.cpp
clang-tools-extra/trunk/clangd/index/Serialization.cpp

Modified: clang-tools-extra/trunk/clangd/RIFF.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/RIFF.cpp?rev=341451&r1=341450&r2=341451&view=diff
==
--- clang-tools-extra/trunk/clangd/RIFF.cpp (original)
+++ clang-tools-extra/trunk/clangd/RIFF.cpp Wed Sep  5 00:52:49 2018
@@ -36,11 +36,11 @@ Expected readChunk(StringRef &Str
   return makeError("nonzero padding byte");
 Stream = Stream.drop_front();
   }
-  return C;
+  return std::move(C);
 };
 
 raw_ostream &operator<<(raw_ostream &OS, const Chunk &C) {
-  OS.write(C.ID.begin(), C.ID.size());
+  OS.write(C.ID.data(), C.ID.size());
   char Size[4];
   llvm::support::endian::write32le(Size, C.Data.size());
   OS.write(Size, sizeof(Size));
@@ -65,7 +65,7 @@ llvm::Expected readFile(llvm::Stri
   F.Chunks.push_back(*Chunk);
 } else
   return Chunk.takeError();
-  return F;
+  return std::move(F);
 }
 
 raw_ostream &operator<<(raw_ostream &OS, const File &F) {
@@ -77,7 +77,7 @@ raw_ostream &operator<<(raw_ostream &OS,
   char Size[4];
   llvm::support::endian::write32le(Size, DataLen);
   OS.write(Size, sizeof(Size));
-  OS.write(F.Type.begin(), F.Type.size());
+  OS.write(F.Type.data(), F.Type.size());
   for (const auto &C : F.Chunks)
 OS << C;
   return OS;

Modified: clang-tools-extra/trunk/clangd/index/Serialization.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/Serialization.cpp?rev=341451&r1=341450&r2=341451&view=diff
==
--- clang-tools-extra/trunk/clangd/index/Serialization.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/Serialization.cpp Wed Sep  5 00:52:49 
2018
@@ -151,7 +151,7 @@ Expected readStringTable(
 Table.Strings.push_back(Saver.save(consume(Rest, Len)));
 Rest = Rest.drop_front();
   }
-  return Table;
+  return std::move(Table);
 }
 
 // SYMBOL ENCODING
@@ -272,7 +272,7 @@ Expected readSymbol(StringRef &D
   }
 
 #undef READ_STRING
-  return Sym;
+  return std::move(Sym);
 }
 
 } // namespace
@@ -322,7 +322,7 @@ Expected readIndexFile(Stri
 return Sym.takeError();
 Result.Symbols = std::move(Symbols).build();
   }
-  return Result;
+  return std::move(Result);
 }
 
 raw_ostream &operator<<(raw_ostream &OS, const IndexFileOut &Data) {


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [clang-tools-extra] r341375 - [clangd] Define a compact binary serialization fomat for symbol slab/index.

[Sam McCall via cfe-commits]

>^
> ../tools/clang/tools/extra/clangd/index/Serialization.cpp:325:10: note:
> in instantiation of function template specialization
> 'llvm::Expected::Expected
> &>' requested here
>return Result;
>   ^
> ../tools/clang/tools/extra/clangd/index/Index.h:324:26: note: copy
> constructor of 'SymbolSlab' is implicitly deleted because field 'Arena'
> has a deleted copy constructor
>llvm::BumpPtrAllocator Arena; // Owns Symbol data that the Symbols do
> not.
>   ^
> ../include/llvm/Support/Allocator.h:157:3: note: copy constructor is
> implicitly deleted because 'BumpPtrAllocatorImpl 4096, 4096>' has a user-declared move constructor
>BumpPtrAllocatorImpl(BumpPtrAllocatorImpl &&Old)
>^
> 2 errors generated.
>
> Several buildbots fail the same way.
>
> /Mikael
>
> On 09/04/2018 06:16 PM, Sam McCall via cfe-commits wrote:
> > Author: sammccall
> > Date: Tue Sep  4 09:16:50 2018
> > New Revision: 341375
> >
> > URL: http://llvm.org/viewvc/llvm-project?rev=341375&view=rev
> > Log:
> > [clangd] Define a compact binary serialization fomat for symbol
> slab/index.
> >
> > Summary:
> > This is intended to replace the current YAML format for general use.
> > It's ~10x more compact than YAML, and ~40% more compact than gzipped
> YAML:
> >llvmidx.riff = 20M, llvmidx.yaml = 272M, llvmidx.yaml.gz = 32M
> > It's also simpler/faster to read and write.
> >
> > The format is a RIFF container (chunks of (type, size, data)) with:
> >   - a compressed string table
> >   - simple binary encoding of symbols (with varints for compactness)
> > It can be extended to include occurrences, Dex posting lists, etc.
> >
> > There's no rich backwards-compatibility scheme, but a version number is
> included
> > so we can detect incompatible files and do ad-hoc back-compat.
> >
> > Alternatives considered:
> >   - compressed YAML or JSON: bulky and slow to load
> >   - llvm bitstream: confusing model and libraries are hard to use. My
> attempt
> > produced slightly larger files, and the code was longer and slower.
> >   - protobuf or similar: would be really nice (esp for back-compat) but
> the
> > dependency is a big hassle
> >   - ad-hoc binary format without a container: it seems clear we're going
> > to add posting lists and occurrences here, and that they will benefit
> > from sharing a string table. The container makes it easy to debug
> > these pieces in isolation, and make them optional.
> >
> > Reviewers: ioeric
> >
> > Subscribers: mgorny, ilya-biryukov, MaskRay, jkorous, mgrang, arphaman,
> kadircet, cfe-commits
> >
> > Differential Revision: https://reviews.llvm.org/D51585
> >
> > Added:
> >  clang-tools-extra/trunk/clangd/RIFF.cpp
> >  clang-tools-extra/trunk/clangd/RIFF.h
> >  clang-tools-extra/trunk/clangd/index/Serialization.cpp
> >  clang-tools-extra/trunk/clangd/index/Serialization.h
> >  clang-tools-extra/trunk/unittests/clangd/RIFFTests.cpp
> >  clang-tools-extra/trunk/unittests/clangd/SerializationTests.cpp
> > Modified:
> >  clang-tools-extra/trunk/clangd/CMakeLists.txt
> >
> clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
> >  clang-tools-extra/trunk/clangd/index/Index.cpp
> >  clang-tools-extra/trunk/clangd/index/Index.h
> >  clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp
> >  clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
> >  clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt
> >  clang-tools-extra/trunk/unittests/clangd/SymbolCollectorTests.cpp
> >
> > Modified: clang-tools-extra/trunk/clangd/CMakeLists.txt
> > URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/CMakeLists.txt?rev=341375&r1=341374&r2=341375&view=diff
> >
> ==
> > --- clang-tools-extra/trunk/clangd/CMakeLists.txt (original)
> > +++ clang-tools-extra/trunk/clangd/CMakeLists.txt Tue Sep  4 09:16:50
> 2018
> > @@ -29,6 +29,7 @@ add_clang_library(clangDaemon
> > Protocol.cpp
> > ProtocolHandlers.cpp
> > Quality.cpp
> > +  RIFF.cpp
> > SourceCode.cpp
> > Threading.cpp
> > Trace.cpp
> > @@ -41,6 +42,7 @@ add_clang_library(clangDaemon
> > index/Index.cpp
> > index/MemIndex.cpp
> > index/Merge.cpp
> > +  index/Seria

[clang-tools-extra] r341458 - [clangd] Implement findReferences function

[Sam McCall via cfe-commits]

Author: sammccall
Date: Wed Sep  5 03:33:36 2018
New Revision: 341458

URL: http://llvm.org/viewvc/llvm-project?rev=341458&view=rev
Log:
[clangd] Implement findReferences function

clangd will use findReferences to provide LSP's reference feature.

Modified:
clang-tools-extra/trunk/clangd/XRefs.cpp
clang-tools-extra/trunk/clangd/XRefs.h
clang-tools-extra/trunk/unittests/clangd/TestTU.cpp
clang-tools-extra/trunk/unittests/clangd/XRefsTests.cpp

Modified: clang-tools-extra/trunk/clangd/XRefs.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/XRefs.cpp?rev=341458&r1=341457&r2=341458&view=diff
==
--- clang-tools-extra/trunk/clangd/XRefs.cpp (original)
+++ clang-tools-extra/trunk/clangd/XRefs.cpp Wed Sep  5 03:33:36 2018
@@ -174,30 +174,27 @@ IdentifiedSymbol getSymbolAtPosition(Par
   return {DeclMacrosFinder.takeDecls(), DeclMacrosFinder.takeMacroInfos()};
 }
 
-llvm::Optional
-makeLocation(ParsedAST &AST, const SourceRange &ValSourceRange) {
+Range getTokenRange(ParsedAST &AST, SourceLocation TokLoc) {
   const SourceManager &SourceMgr = AST.getASTContext().getSourceManager();
-  const LangOptions &LangOpts = AST.getASTContext().getLangOpts();
-  SourceLocation LocStart = ValSourceRange.getBegin();
+  SourceLocation LocEnd = Lexer::getLocForEndOfToken(
+  TokLoc, 0, SourceMgr, AST.getASTContext().getLangOpts());
+  return {sourceLocToPosition(SourceMgr, TokLoc),
+  sourceLocToPosition(SourceMgr, LocEnd)};
+}
 
-  const FileEntry *F =
-  SourceMgr.getFileEntryForID(SourceMgr.getFileID(LocStart));
+llvm::Optional makeLocation(ParsedAST &AST, SourceLocation TokLoc) {
+  const SourceManager &SourceMgr = AST.getASTContext().getSourceManager();
+  const FileEntry *F = 
SourceMgr.getFileEntryForID(SourceMgr.getFileID(TokLoc));
   if (!F)
 return llvm::None;
-  SourceLocation LocEnd = Lexer::getLocForEndOfToken(ValSourceRange.getEnd(), 
0,
- SourceMgr, LangOpts);
-  Position Begin = sourceLocToPosition(SourceMgr, LocStart);
-  Position End = sourceLocToPosition(SourceMgr, LocEnd);
-  Range R = {Begin, End};
-  Location L;
-
   auto FilePath = getRealPath(F, SourceMgr);
   if (!FilePath) {
 log("failed to get path!");
 return llvm::None;
   }
+  Location L;
   L.uri = URIForFile(*FilePath);
-  L.range = R;
+  L.range = getTokenRange(AST, TokLoc);
   return L;
 }
 
@@ -223,7 +220,7 @@ std::vector findDefinitions(Pa
 
   for (auto Item : Symbols.Macros) {
 auto Loc = Item.Info->getDefinitionLoc();
-auto L = makeLocation(AST, SourceRange(Loc, Loc));
+auto L = makeLocation(AST, Loc);
 if (L)
   Result.push_back(*L);
   }
@@ -266,7 +263,7 @@ std::vector findDefinitions(Pa
 
 auto &Candidate = ResultCandidates[Key];
 auto Loc = findNameLoc(D);
-auto L = makeLocation(AST, SourceRange(Loc, Loc));
+auto L = makeLocation(AST, Loc);
 // The declaration in the identified symbols is a definition if possible
 // otherwise it is declaration.
 bool IsDef = getDefinition(D) == D;
@@ -316,24 +313,36 @@ std::vector findDefinitions(Pa
 
 namespace {
 
-/// Finds document highlights that a given list of declarations refers to.
-class DocumentHighlightsFinder : public index::IndexDataConsumer {
-  std::vector &Decls;
-  std::vector DocumentHighlights;
-  const ASTContext *
-
+/// Collects references to symbols within the main file.
+class ReferenceFinder : public index::IndexDataConsumer {
 public:
-  DocumentHighlightsFinder(ASTContext &AST, Preprocessor &PP,
-   std::vector &Decls)
-  : Decls(Decls), AST(AST) {}
-  std::vector takeHighlights() {
-// Don't keep the same highlight multiple times.
-// This can happen when nodes in the AST are visited twice.
-std::sort(DocumentHighlights.begin(), DocumentHighlights.end());
-auto Last =
-std::unique(DocumentHighlights.begin(), DocumentHighlights.end());
-DocumentHighlights.erase(Last, DocumentHighlights.end());
-return std::move(DocumentHighlights);
+  struct Reference {
+const Decl *Target;
+SourceLocation Loc;
+index::SymbolRoleSet Role;
+  };
+
+  ReferenceFinder(ASTContext &AST, Preprocessor &PP,
+  const std::vector &TargetDecls)
+  : AST(AST) {
+for (const Decl *D : TargetDecls)
+  Targets.insert(D);
+  }
+
+  std::vector take() && {
+std::sort(References.begin(), References.end(),
+  [](const Reference &L, const Reference &R) {
+return std::tie(L.Loc, L.Target, L.Role) <
+   std::tie(R.Loc, R.Target, R.Role);
+  });
+// We sometimes see duplicates when parts of the AST get traversed twice.
+References.erase(std::unique(References.begin(), References.end(),
+ [](const Reference &L, const Reference &R) {
+   r

[clang-tools-extra] r341459 - [clangd] Avoid enum class+enumValN to avoid GCC bug(?), and use consistent style.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Wed Sep  5 03:39:58 2018
New Revision: 341459

URL: http://llvm.org/viewvc/llvm-project?rev=341459&view=rev
Log:
[clangd] Avoid enum class+enumValN to avoid GCC bug(?), and use consistent 
style.

Modified:

clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp

Modified: 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp?rev=341459&r1=341458&r2=341459&view=diff
==
--- 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
 (original)
+++ 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
 Wed Sep  5 03:39:58 2018
@@ -60,7 +60,7 @@ static llvm::cl::opt MergeOnTheFly
 "MapReduce."),
 llvm::cl::init(true), llvm::cl::Hidden);
 
-enum class Format { YAML, Binary };
+enum Format { YAML, Binary };
 static llvm::cl::opt
 Format("format", llvm::cl::desc("Format of the index to be written"),
llvm::cl::values(

Modified: clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp?rev=341459&r1=341458&r2=341459&view=diff
==
--- clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp (original)
+++ clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp Wed Sep  5 03:39:58 2018
@@ -36,12 +36,6 @@ static llvm::cl::opt
llvm::cl::desc("Use experimental Dex static index."),
llvm::cl::init(true), llvm::cl::Hidden);
 
-namespace {
-
-enum class PCHStorageFlag { Disk, Memory };
-
-} // namespace
-
 static llvm::cl::opt CompileCommandsDir(
 "compile-commands-dir",
 llvm::cl::desc("Specify a path to look for compile_commands.json. If path "
@@ -54,10 +48,7 @@ static llvm::cl::opt
llvm::cl::init(getDefaultAsyncThreadsCount()));
 
 // FIXME: also support "plain" style where signatures are always omitted.
-enum CompletionStyleFlag {
-  Detailed,
-  Bundled,
-};
+enum CompletionStyleFlag { Detailed, Bundled };
 static llvm::cl::opt CompletionStyle(
 "completion-style",
 llvm::cl::desc("Granularity of code completion suggestions"),
@@ -106,6 +97,7 @@ static llvm::cl::opt Test(
 "Intended to simplify lit tests."),
 llvm::cl::init(false), llvm::cl::Hidden);
 
+enum PCHStorageFlag { Disk, Memory };
 static llvm::cl::opt PCHStorage(
 "pch-storage",
 llvm::cl::desc("Storing PCHs in memory increases memory usages, but may "
@@ -167,7 +159,6 @@ static llvm::cl::opt YamlSymbolFil
 llvm::cl::init(""), llvm::cl::Hidden);
 
 enum CompileArgsFrom { LSPCompileArgs, FilesystemCompileArgs };
-
 static llvm::cl::opt CompileArgsFrom(
 "compile_args_from", llvm::cl::desc("The source of compile commands"),
 llvm::cl::values(clEnumValN(LSPCompileArgs, "lsp",


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r341462 - [clangd] Add xrefs LSP boilerplate implementation.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Wed Sep  5 04:53:07 2018
New Revision: 341462

URL: http://llvm.org/viewvc/llvm-project?rev=341462&view=rev
Log:
[clangd] Add xrefs LSP boilerplate implementation.

Reviewers: ilya-biryukov, ioeric

Subscribers: MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D50896

Added:
clang-tools-extra/trunk/test/clangd/references.test
Modified:
clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
clang-tools-extra/trunk/clangd/ClangdLSPServer.h
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.h
clang-tools-extra/trunk/clangd/Protocol.cpp
clang-tools-extra/trunk/clangd/Protocol.h
clang-tools-extra/trunk/clangd/ProtocolHandlers.cpp
clang-tools-extra/trunk/clangd/ProtocolHandlers.h
clang-tools-extra/trunk/clangd/XRefs.cpp
clang-tools-extra/trunk/test/clangd/initialize-params-invalid.test
clang-tools-extra/trunk/test/clangd/initialize-params.test

Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=341462&r1=341461&r2=341462&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp Wed Sep  5 04:53:07 2018
@@ -129,6 +129,7 @@ void ClangdLSPServer::onInitialize(Initi
 {"renameProvider", true},
 {"documentSymbolProvider", true},
 {"workspaceSymbolProvider", true},
+{"referencesProvider", true},
 {"executeCommandProvider",
  json::Object{
  {"commands", 
{ExecuteCommandParams::CLANGD_APPLY_FIX_COMMAND}},
@@ -449,6 +450,17 @@ void ClangdLSPServer::onChangeConfigurat
   applyConfiguration(Params.settings);
 }
 
+void ClangdLSPServer::onReference(ReferenceParams &Params) {
+  Server.findReferences(Params.textDocument.uri.file(), Params.position,
+[](llvm::Expected> Locations) {
+  if (!Locations)
+return replyError(
+ErrorCode::InternalError,
+llvm::toString(Locations.takeError()));
+  reply(llvm::json::Array(*Locations));
+});
+}
+
 ClangdLSPServer::ClangdLSPServer(JSONOutput &Out,
  const clangd::CodeCompleteOptions &CCOpts,
  llvm::Optional CompileCommandsDir,

Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.h?rev=341462&r1=341461&r2=341462&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdLSPServer.h (original)
+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.h Wed Sep  5 04:53:07 2018
@@ -67,6 +67,7 @@ private:
   void onCompletion(TextDocumentPositionParams &Params) override;
   void onSignatureHelp(TextDocumentPositionParams &Params) override;
   void onGoToDefinition(TextDocumentPositionParams &Params) override;
+  void onReference(ReferenceParams &Params) override;
   void onSwitchSourceHeader(TextDocumentIdentifier &Params) override;
   void onDocumentHighlight(TextDocumentPositionParams &Params) override;
   void onFileEvent(DidChangeWatchedFilesParams &Params) override;

Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=341462&r1=341461&r2=341462&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Wed Sep  5 04:53:07 2018
@@ -560,6 +560,18 @@ void ClangdServer::documentSymbols(
Bind(Action, std::move(CB)));
 }
 
+void ClangdServer::findReferences(PathRef File, Position Pos,
+  Callback> CB) {
+  auto Action = [Pos, this](Callback> CB,
+llvm::Expected InpAST) {
+if (!InpAST)
+  return CB(InpAST.takeError());
+CB(clangd::findReferences(InpAST->AST, Pos, Index));
+  };
+
+  WorkScheduler.runWithAST("References", File, Bind(Action, std::move(CB)));
+}
+
 std::vector>
 ClangdServer::getUsedBytesPerFile() const {
   return WorkScheduler.getUsedBytesPerFile();

Modified: clang-tools-extra/trunk/clangd/ClangdServer.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.h?rev=341462&r1=341461&r2=341462&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.h (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.h Wed Sep  5 04:53:07 2018
@@ -

[clang-tools-extra] r341466 - [clangd] Fix references.test assertions

[Sam McCall via cfe-commits]

Author: sammccall
Date: Wed Sep  5 06:17:51 2018
New Revision: 341466

URL: http://llvm.org/viewvc/llvm-project?rev=341466&view=rev
Log:
[clangd] Fix references.test assertions

Modified:
clang-tools-extra/trunk/test/clangd/references.test

Modified: clang-tools-extra/trunk/test/clangd/references.test
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/clangd/references.test?rev=341466&r1=341465&r2=341466&view=diff
==
--- clang-tools-extra/trunk/test/clangd/references.test (original)
+++ clang-tools-extra/trunk/test/clangd/references.test Wed Sep  5 06:17:51 2018
@@ -18,7 +18,7 @@
 # CHECK-NEXT:  "line": 0
 # CHECK-NEXT:}
 # CHECK-NEXT:  },
-# CHECK-NEXT:  "uri": "test:///main.cpp"
+# CHECK-NEXT:  "uri": "{{.*}}/main.cpp"
 # CHECK-NEXT:},
 # CHECK-NEXT:{
 # CHECK-NEXT:  "range": {
@@ -31,8 +31,8 @@
 # CHECK-NEXT:  "line": 0
 # CHECK-NEXT:}
 # CHECK-NEXT:  },
-# CHECK-NEXT:  "uri": "test:///main.cpp"
-# CHECK-NEXT:},
+# CHECK-NEXT:  "uri": "{{.*}}/main.cpp"
+# CHECK-NEXT:}
 # CHECK-NEXT:  ]
 ---
 {"jsonrpc":"2.0","id":3,"method":"shutdown"}


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r341465 - [clangd] make zlib compression optional for binary format

[Sam McCall via cfe-commits]

Author: sammccall
Date: Wed Sep  5 06:17:47 2018
New Revision: 341465

URL: http://llvm.org/viewvc/llvm-project?rev=341465&view=rev
Log:
[clangd] make zlib compression optional for binary format

Modified:
clang-tools-extra/trunk/clangd/index/Serialization.cpp

Modified: clang-tools-extra/trunk/clangd/index/Serialization.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/Serialization.cpp?rev=341465&r1=341464&r2=341465&view=diff
==
--- clang-tools-extra/trunk/clangd/index/Serialization.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/Serialization.cpp Wed Sep  5 06:17:47 
2018
@@ -86,7 +86,7 @@ uint32_t consumeVar(StringRef &Data) {
 // We store each string once, and refer to them by index.
 //
 // The string table's format is:
-//   - UncompressedSize : uint32
+//   - UncompressedSize : uint32 (or 0 for no compression)
 //   - CompressedData   : byte[CompressedSize]
 //
 // CompressedData is a zlib-compressed byte[UncompressedSize].
@@ -102,6 +102,11 @@ class StringTableOut {
   DenseMap, unsigned> Index;
 
 public:
+  StringTableOut() {
+// Ensure there's at least one string in the table.
+// Table size zero is reserved to indicate no compression.
+Unique.insert("");
+  }
   // Add a string to the table. Overwrites S if an identical string exists.
   void intern(StringRef &S) { S = *Unique.insert(S).first; };
   // Finalize the table and write it to OS. No more strings may be added.
@@ -116,10 +121,15 @@ public:
   RawTable.append(S);
   RawTable.push_back(0);
 }
-SmallString<1> Compressed;
-cantFail(zlib::compress(RawTable, Compressed));
-write32(RawTable.size(), OS);
-OS << Compressed;
+if (zlib::isAvailable()) {
+  SmallString<1> Compressed;
+  cantFail(zlib::compress(RawTable, Compressed));
+  write32(RawTable.size(), OS);
+  OS << Compressed;
+} else {
+  write32(0, OS); // No compression.
+  OS << RawTable;
+}
   }
   // Get the ID of an string, which must be interned. Table must be finalized.
   unsigned index(StringRef S) const {
@@ -138,9 +148,17 @@ Expected readStringTable(
   if (Data.size() < 4)
 return makeError("Bad string table: not enough metadata");
   size_t UncompressedSize = consume32(Data);
-  SmallString<1> Uncompressed;
-  if (Error E = llvm::zlib::uncompress(Data, Uncompressed, UncompressedSize))
-return std::move(E);
+
+  StringRef Uncompressed;
+  SmallString<1> UncompressedStorage;
+  if (UncompressedSize == 0) // No compression
+Uncompressed = Data;
+  else {
+if (Error E =
+llvm::zlib::uncompress(Data, UncompressedStorage, 
UncompressedSize))
+  return std::move(E);
+Uncompressed = UncompressedStorage;
+  }
 
   StringTableIn Table;
   StringSaver Saver(Table.Arena);
@@ -285,9 +303,9 @@ Expected readSymbol(StringRef &D
 //   - symb: symbols
 
 // The current versioning scheme is simple - non-current versions are rejected.
-// This allows arbitrary format changes, which invalidate stored data.
-// Later we may want to support some backward compatibility.
-constexpr static uint32_t Version = 1;
+// If you make a breaking change, bump this version number to invalidate stored
+// data. Later we may want to support some backward compatibility.
+constexpr static uint32_t Version = 2;
 
 Expected readIndexFile(StringRef Data) {
   auto RIFF = riff::readFile(Data);


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r341467 - [clangd] Fix type/variable name conflict on some compilers

[Sam McCall via cfe-commits]

Author: sammccall
Date: Wed Sep  5 06:22:11 2018
New Revision: 341467

URL: http://llvm.org/viewvc/llvm-project?rev=341467&view=rev
Log:
[clangd] Fix type/variable name conflict on some compilers

Modified:

clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp

Modified: 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp?rev=341467&r1=341466&r2=341467&view=diff
==
--- 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
 (original)
+++ 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
 Wed Sep  5 06:22:11 2018
@@ -60,13 +60,12 @@ static llvm::cl::opt MergeOnTheFly
 "MapReduce."),
 llvm::cl::init(true), llvm::cl::Hidden);
 
-enum Format { YAML, Binary };
-static llvm::cl::opt
-Format("format", llvm::cl::desc("Format of the index to be written"),
-   llvm::cl::values(
-   clEnumValN(Format::YAML, "yaml", "human-readable YAML format"),
-   clEnumValN(Format::Binary, "binary", "binary RIFF format")),
-   llvm::cl::init(Format::YAML));
+enum IndexFormat { YAML, Binary };
+static llvm::cl::opt Format(
+"format", llvm::cl::desc("Format of the index to be written"),
+llvm::cl::values(clEnumValN(YAML, "yaml", "human-readable YAML format"),
+ clEnumValN(Binary, "binary", "binary RIFF format")),
+llvm::cl::init(YAML));
 
 /// Responsible for aggregating symbols from each processed file and producing
 /// the final results. All methods in this class must be thread-safe,
@@ -273,10 +272,10 @@ int main(int argc, const char **argv) {
   auto UniqueSymbols = Consumer->mergeResults();
   // Output phase: emit result symbols.
   switch (clang::clangd::Format) {
-  case clang::clangd::Format::YAML:
+  case clang::clangd::IndexFormat::YAML:
 SymbolsToYAML(UniqueSymbols, llvm::outs());
 break;
-  case clang::clangd::Format::Binary: {
+  case clang::clangd::IndexFormat::Binary: {
 clang::clangd::IndexFileOut Out;
 Out.Symbols = &UniqueSymbols;
 llvm::outs() << Out;


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [clang-tools-extra] r341459 - [clangd] Avoid enum class+enumValN to avoid GCC bug(?), and use consistent style.

[Sam McCall via cfe-commits]

It turned out to be a different bug: the problem was referring to
`Format::YAML` in the initializer for a variable also named `Format`.
This is legal but old versions of GCC get this wrong.
As usual with buildbot failures, I was throwing things at the wall to see
what sticks.

Regarding style - either would work, there were 2x enum and 2x enum class.
My reasons for leaning towards enum here is that for this command-line flag
pattern
 - it mainly leads to repeating the type name in a context where the type
is obvious
 - there's minimal risk/consequence to a namespace conflict as we're in an
anonymous namespace in a CC file
 - there's often not a second good name (need one for the flag + one for
the enum), so it's a bad name that ends up repeated
But if you feel strongly about it, feel free to flip it.

On Thu, Sep 6, 2018 at 11:45 AM Ilya Biryukov  wrote:

> Any pointers to the GCC bug/breakage mentioned?
>
> On Thu, Sep 6, 2018 at 11:44 AM Ilya Biryukov 
> wrote:
>
>> +1 for consistent style, but why not use enum class everywhere instead?
>>
>> On Wed, Sep 5, 2018 at 12:41 PM Sam McCall via cfe-commits <
>> cfe-commits@lists.llvm.org> wrote:
>>
>>> Author: sammccall
>>> Date: Wed Sep  5 03:39:58 2018
>>> New Revision: 341459
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=341459&view=rev
>>> Log:
>>> [clangd] Avoid enum class+enumValN to avoid GCC bug(?), and use
>>> consistent style.
>>>
>>> Modified:
>>>
>>> clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
>>> clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
>>>
>>> Modified:
>>> clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp?rev=341459&r1=341458&r2=341459&view=diff
>>>
>>> ==
>>> ---
>>> clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
>>> (original)
>>> +++
>>> clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
>>> Wed Sep  5 03:39:58 2018
>>> @@ -60,7 +60,7 @@ static llvm::cl::opt MergeOnTheFly
>>>  "MapReduce."),
>>>  llvm::cl::init(true), llvm::cl::Hidden);
>>>
>>> -enum class Format { YAML, Binary };
>>> +enum Format { YAML, Binary };
>>>  static llvm::cl::opt
>>>  Format("format", llvm::cl::desc("Format of the index to be
>>> written"),
>>> llvm::cl::values(
>>>
>>> Modified: clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp?rev=341459&r1=341458&r2=341459&view=diff
>>>
>>> ==
>>> --- clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp (original)
>>> +++ clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp Wed Sep  5
>>> 03:39:58 2018
>>> @@ -36,12 +36,6 @@ static llvm::cl::opt
>>> llvm::cl::desc("Use experimental Dex static index."),
>>> llvm::cl::init(true), llvm::cl::Hidden);
>>>
>>> -namespace {
>>> -
>>> -enum class PCHStorageFlag { Disk, Memory };
>>> -
>>> -} // namespace
>>> -
>>>  static llvm::cl::opt CompileCommandsDir(
>>>  "compile-commands-dir",
>>>  llvm::cl::desc("Specify a path to look for compile_commands.json.
>>> If path "
>>> @@ -54,10 +48,7 @@ static llvm::cl::opt
>>> llvm::cl::init(getDefaultAsyncThreadsCount()));
>>>
>>>  // FIXME: also support "plain" style where signatures are always
>>> omitted.
>>> -enum CompletionStyleFlag {
>>> -  Detailed,
>>> -  Bundled,
>>> -};
>>> +enum CompletionStyleFlag { Detailed, Bundled };
>>>  static llvm::cl::opt CompletionStyle(
>>>  "completion-style",
>>>  llvm::cl::desc("Granularity of code completion suggestions"),
>>> @@ -106,6 +97,7 @@ static llvm::cl::opt Test(
>>>  "Intended to simplify lit tests."),
>>>  llvm::cl::init(false), llvm::cl::Hidden);
>>>
>>> +enum PCHStorageFlag { Disk, Memory };
>

Re: [clang-tools-extra] r341459 - [clangd] Avoid enum class+enumValN to avoid GCC bug(?), and use consistent style.

[Sam McCall via cfe-commits]

On Thu, Sep 6, 2018 at 2:26 PM Ilya Biryukov  wrote:

> I would generally vouch for strongly typed enums, because there're nicer
> in many aspects (no implicit integer conversions, no enumerators thrown
> into the namespaces).
>
I mostly (generally!) agree.
In particular they have strong advantages when being exposed as part of a
widely-visible API.
The disadvantages are mostly verbosity (in cases where it doesn't aid
readability) and it being hard to use them in bitfields.


> With regards to naming conventions, PCHStorage::Memory or
> CompletionStyle::Bundled look pretty neat to me, the usual alternative for
> enums is coming up with prefixes, e.g. PS_Memory CS_Bundled.
>
It's shorter, but it's hard to remember which prefix to use (without the
> prefix, it's also hard to keep all enum values in your head)
>
I don't think this is the case here (within the scope of the main file).


> while the type name is evident from signature help or completion results
> and completing EnumType:: yields full list of enumerators right away.
>
(this is true of both scoped and unscoped enums)

The "Flag" suffix in the enum names seems redundant, though, I have removed
> it from the examples on purpose.
>
Yes, agree with this.


> WDYT?
>
I think plain enum is better for this case for the reasons above, but I
don't feel strongly, feel free to change it.


>
> PS BTW, we definitely need to make enumerator completions work in more
> cases than switch-case at some point.
>
>
>
>
> On Thu, Sep 6, 2018 at 2:11 PM Sam McCall  wrote:
>
>> It turned out to be a different bug: the problem was referring to
>> `Format::YAML` in the initializer for a variable also named `Format`.
>> This is legal but old versions of GCC get this wrong.
>> As usual with buildbot failures, I was throwing things at the wall to see
>> what sticks.
>>
>> Regarding style - either would work, there were 2x enum and 2x enum class.
>> My reasons for leaning towards enum here is that for this command-line
>> flag pattern
>>  - it mainly leads to repeating the type name in a context where the type
>> is obvious
>>  - there's minimal risk/consequence to a namespace conflict as we're in
>> an anonymous namespace in a CC file
>>  - there's often not a second good name (need one for the flag + one for
>> the enum), so it's a bad name that ends up repeated
>> But if you feel strongly about it, feel free to flip it.
>>
>> On Thu, Sep 6, 2018 at 11:45 AM Ilya Biryukov 
>> wrote:
>>
>>> Any pointers to the GCC bug/breakage mentioned?
>>>
>>> On Thu, Sep 6, 2018 at 11:44 AM Ilya Biryukov 
>>> wrote:
>>>
>>>> +1 for consistent style, but why not use enum class everywhere instead?
>>>>
>>>> On Wed, Sep 5, 2018 at 12:41 PM Sam McCall via cfe-commits <
>>>> cfe-commits@lists.llvm.org> wrote:
>>>>
>>>>> Author: sammccall
>>>>> Date: Wed Sep  5 03:39:58 2018
>>>>> New Revision: 341459
>>>>>
>>>>> URL: http://llvm.org/viewvc/llvm-project?rev=341459&view=rev
>>>>> Log:
>>>>> [clangd] Avoid enum class+enumValN to avoid GCC bug(?), and use
>>>>> consistent style.
>>>>>
>>>>> Modified:
>>>>>
>>>>> clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
>>>>> clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
>>>>>
>>>>> Modified:
>>>>> clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
>>>>> URL:
>>>>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp?rev=341459&r1=341458&r2=341459&view=diff
>>>>>
>>>>> ==
>>>>> ---
>>>>> clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
>>>>> (original)
>>>>> +++
>>>>> clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
>>>>> Wed Sep  5 03:39:58 2018
>>>>> @@ -60,7 +60,7 @@ static llvm::cl::opt MergeOnTheFly
>>>>>  "MapReduce."),
>>>>>  llvm::cl::init(true), llvm::cl::Hidden);
>>>>>
>>>>> -enum class Format { YAML, Binary };
>>>>> +enum Format { YAML, Binary };
>>>>>  static llvm:

r341549 - Revert "[DebugInfo] Generate debug information for labels. (Fix PR37395)"

[Sam McCall via cfe-commits]

Author: sammccall
Date: Thu Sep  6 07:27:40 2018
New Revision: 341549

URL: http://llvm.org/viewvc/llvm-project?rev=341549&view=rev
Log:
Revert "[DebugInfo] Generate debug information for labels. (Fix PR37395)"

This reverts commit r341519, which generates debug info that causes
backend crashes. (with -split-dwarf-file)

Details in https://reviews.llvm.org/D50495

Removed:
cfe/trunk/test/CodeGen/debug-label-inline.c
cfe/trunk/test/CodeGen/debug-label.c
Modified:
cfe/trunk/lib/CodeGen/CGDebugInfo.cpp
cfe/trunk/lib/CodeGen/CGDebugInfo.h
cfe/trunk/lib/CodeGen/CGStmt.cpp

Modified: cfe/trunk/lib/CodeGen/CGDebugInfo.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGDebugInfo.cpp?rev=341549&r1=341548&r2=341549&view=diff
==
--- cfe/trunk/lib/CodeGen/CGDebugInfo.cpp (original)
+++ cfe/trunk/lib/CodeGen/CGDebugInfo.cpp Thu Sep  6 07:27:40 2018
@@ -3769,32 +3769,6 @@ CGDebugInfo::EmitDeclareOfAutoVariable(c
   return EmitDeclare(VD, Storage, llvm::None, Builder);
 }
 
-void CGDebugInfo::EmitLabel(const LabelDecl *D, CGBuilderTy &Builder) {
-  assert(DebugKind >= codegenoptions::LimitedDebugInfo);
-  assert(!LexicalBlockStack.empty() && "Region stack mismatch, stack empty!");
-
-  if (D->hasAttr())
-return;
-
-  auto *Scope = cast(LexicalBlockStack.back());
-  llvm::DIFile *Unit = getOrCreateFile(D->getLocation());
-
-  // Get location information.
-  unsigned Line = getLineNumber(D->getLocation());
-  unsigned Column = getColumnNumber(D->getLocation());
-
-  StringRef Name = D->getName();
-
-  // Create the descriptor for the label.
-  auto *L =
-  DBuilder.createLabel(Scope, Name, Unit, Line, 
CGM.getLangOpts().Optimize);
-
-  // Insert an llvm.dbg.label into the current block.
-  DBuilder.insertLabel(L,
-   llvm::DebugLoc::get(Line, Column, Scope, CurInlinedAt),
-   Builder.GetInsertBlock());
-}
-
 llvm::DIType *CGDebugInfo::CreateSelfType(const QualType &QualTy,
   llvm::DIType *Ty) {
   llvm::DIType *CachedTy = getTypeOrNull(QualTy);

Modified: cfe/trunk/lib/CodeGen/CGDebugInfo.h
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGDebugInfo.h?rev=341549&r1=341548&r2=341549&view=diff
==
--- cfe/trunk/lib/CodeGen/CGDebugInfo.h (original)
+++ cfe/trunk/lib/CodeGen/CGDebugInfo.h Thu Sep  6 07:27:40 2018
@@ -415,9 +415,6 @@ public:
llvm::Value *AI,
CGBuilderTy &Builder);
 
-  /// Emit call to \c llvm.dbg.label for an label.
-  void EmitLabel(const LabelDecl *D, CGBuilderTy &Builder);
-
   /// Emit call to \c llvm.dbg.declare for an imported variable
   /// declaration in a block.
   void EmitDeclareOfBlockDeclRefVariable(

Modified: cfe/trunk/lib/CodeGen/CGStmt.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGStmt.cpp?rev=341549&r1=341548&r2=341549&view=diff
==
--- cfe/trunk/lib/CodeGen/CGStmt.cpp (original)
+++ cfe/trunk/lib/CodeGen/CGStmt.cpp Thu Sep  6 07:27:40 2018
@@ -531,16 +531,6 @@ void CodeGenFunction::EmitLabel(const La
   }
 
   EmitBlock(Dest.getBlock());
-
-  // Emit debug info for labels.
-  if (CGDebugInfo *DI = getDebugInfo()) {
-if (CGM.getCodeGenOpts().getDebugInfo() >=
-codegenoptions::LimitedDebugInfo) {
-  DI->setLocation(D->getLocation());
-  DI->EmitLabel(D, Builder);
-}
-  }
-
   incrementProfileCounter(D->getStmt());
 }
 

Removed: cfe/trunk/test/CodeGen/debug-label-inline.c
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/debug-label-inline.c?rev=341548&view=auto
==
--- cfe/trunk/test/CodeGen/debug-label-inline.c (original)
+++ cfe/trunk/test/CodeGen/debug-label-inline.c (removed)
@@ -1,28 +0,0 @@
-// This test will test the correctness of generating DILabel and
-// llvm.dbg.label when the label is in inlined functions.
-//
-// RUN: %clang_cc1 -O2 %s -o - -emit-llvm -debug-info-kind=limited | FileCheck 
%s
-inline int f1(int a, int b) {
-  int sum;
-
-top:
-  sum = a + b;
-  return sum;
-}
-
-extern int ga, gb;
-
-int f2(void) {
-  int result;
-
-  result = f1(ga, gb);
-  // CHECK: call void @llvm.dbg.label(metadata [[LABEL_METADATA:!.*]]), !dbg 
[[LABEL_LOCATION:!.*]]
-
-  return result;
-}
-
-// CHECK: distinct !DISubprogram(name: "f1", {{.*}}, retainedNodes: 
[[ELEMENTS:!.*]])
-// CHECK: [[ELEMENTS]] = !{{{.*}}, [[LABEL_METADATA]]}
-// CHECK: [[LABEL_METADATA]] = !DILabel({{.*}}, name: "top", {{.*}}, line: 8)
-// CHECK: [[INLINEDAT:!.*]] = distinct !DILocation(line: 18,
-// CHECK: [[LABEL_LOCATION]] = !DILocation(line: 8, {{.*}}, inlinedAt: 
[[INLINEDAT]])

Removed: cfe/trunk/test/CodeGen/

[clang-tools-extra] r341797 - [clangd] Fix async index loading (from r341376).

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Sep 10 03:00:47 2018
New Revision: 341797

URL: http://llvm.org/viewvc/llvm-project?rev=341797&view=rev
Log:
[clangd] Fix async index loading (from r341376).

Summary:
This wasn't actually async (due to std::future destructor blocking).
If it were, we would have clean shutdown issues if main returned
and destroyed Placeholder before the thread is done with it.

We could attempt to avoid any blocking by using shared_ptr or weak_ptr tricks so
the thread can detect Placeholder's destruction, but there are other potential
issues (e.g. loadIndex does tracing, and we'll destroy the tracer...)
Instead, once LSPServer::run returns, we wait for the index to finish loading
before exiting. Performance is not critical in this situation.

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51674

Modified:
clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp

Modified: clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp?rev=341797&r1=341796&r2=341797&view=diff
==
--- clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp (original)
+++ clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp Mon Sep 10 03:00:47 2018
@@ -270,14 +270,17 @@ int main(int argc, char *argv[]) {
 Opts.ResourceDir = ResourceDir;
   Opts.BuildDynamicSymbolIndex = EnableIndex;
   std::unique_ptr StaticIdx;
+  std::future AsyncIndexLoad; // Block exit while loading the index.
   if (EnableIndex && !YamlSymbolFile.empty()) {
 // Load the index asynchronously. Meanwhile SwapIndex returns no results.
 SwapIndex *Placeholder;
 StaticIdx.reset(Placeholder = new 
SwapIndex(llvm::make_unique()));
-runAsync([Placeholder, &Opts] {
+AsyncIndexLoad = runAsync([Placeholder, &Opts] {
   if (auto Idx = loadIndex(YamlSymbolFile, Opts.URISchemes, UseDex))
 Placeholder->reset(std::move(Idx));
 });
+if (RunSynchronously)
+  AsyncIndexLoad.wait();
   }
   Opts.StaticIndex = StaticIdx.get();
   Opts.AsyncThreadsCount = WorkerThreadsCount;


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [PATCH] D51747: [clangd] Implement deprecation diagnostics with lower severity.

[Sam McCall via cfe-commits]

On Tue, Sep 11, 2018 at 3:43 PM Ilya Biryukov via Phabricator <
revi...@reviews.llvm.org> wrote:

> ilya-biryukov added a comment.
>
> In https://reviews.llvm.org/D51747#1229066, @sammccall wrote:
>
> > A few thoughts here:
> >
> > - does CDB describe user or project preferences? unclear.
>
>
> Agree, it's a mix, defaults are from the project but users can add extra
> flags.
>
> > - "show this warning for code I build" is a higher bar than "show this
> warning for code I edit". So CDB probably enables too few warnings.
> > - Some warnings play well with -Werror (like uninit warnings), some
> don't (like deprecated). -Werror projects often disable interesting
> warnings.
>
> Agreed, editors are different from build.
>
> > I'm not sure we should strictly follow the CDB, but the bar to override
> it should probably be high.
>
> WDYT in the long term about a more general mechanism (to allow users
> override compiler or warning flags at the clangd level?
> So that even if clangd is opinionated about the default warnings it
> enables, users have an option to override according to their preferences.
>
Yeah, I can see making "extra clang flags" a clangd flag (at some point we
really need .clangd config file or something...)

The scary thing about the extra flags is how they interact with driver mode
(clang-cl vs clang), but maybe that's the user's problem.


> In https://reviews.llvm.org/D51747#1230420, @kadircet wrote:
>
> > if user wants to see all diagnostics as a list suddenly they will get
> deprecations in that list as well :(.
>
>
> Yeah, some level of noise is probably inevitable.
>
>
> Repository:
>   rCTE Clang Tools Extra
>
> https://reviews.llvm.org/D51747
>
>
>
>
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r342130 - [clangd] Simplify cancellation public API

[Sam McCall via cfe-commits]

Author: sammccall
Date: Thu Sep 13 04:47:48 2018
New Revision: 342130

URL: http://llvm.org/viewvc/llvm-project?rev=342130&view=rev
Log:
[clangd] Simplify cancellation public API

Summary:
Task is no longer exposed:
 - task cancellation is hidden as a std::function
 - task creation returns the new context directly
 - checking is via free function only, with no way to avoid the context lookup
The implementation is essentially the same, but a bit terser as it's hidden.

isCancelled() is now safe to use outside any task (it returns false).
This will leave us free to sprinkle cancellation in e.g. TUScheduler without
needing elaborate test setup, and lets callers that don't cancel "just work".

Updated the docs to describe the new expected use pattern.
One thing I noticed: there's nothing async-specific about the cancellation.
Async tasks can be cancelled from any thread (typically the one that created
them), sync tasks can be cancelled from any *other* thread in the same way.
So the docs now refer to "long-running" tasks instead of async ones.

Updated usage in code complete, without any structural changes.
I didn't update all the names of the helpers in ClangdLSPServer (these will
likely be moved to JSONRPCDispatcher anyway).

Reviewers: ilya-biryukov, kadircet

Subscribers: ioeric, MaskRay, jkorous, arphaman, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D51996

Modified:
clang-tools-extra/trunk/clangd/Cancellation.cpp
clang-tools-extra/trunk/clangd/Cancellation.h
clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
clang-tools-extra/trunk/clangd/ClangdLSPServer.h
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.h
clang-tools-extra/trunk/unittests/clangd/CancellationTests.cpp

Modified: clang-tools-extra/trunk/clangd/Cancellation.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/Cancellation.cpp?rev=342130&r1=342129&r2=342130&view=diff
==
--- clang-tools-extra/trunk/clangd/Cancellation.cpp (original)
+++ clang-tools-extra/trunk/clangd/Cancellation.cpp Thu Sep 13 04:47:48 2018
@@ -13,21 +13,21 @@
 namespace clang {
 namespace clangd {
 
-namespace {
-static Key TaskKey;
-} // namespace
-
 char CancelledError::ID = 0;
+static Key>> FlagKey;
 
-const Task &getCurrentTask() {
-  const auto TH = Context::current().getExisting(TaskKey);
-  assert(TH && "Fetched a nullptr for TaskHandle from context.");
-  return *TH;
+std::pair cancelableTask() {
+  auto Flag = std::make_shared>();
+  return {
+  Context::current().derive(FlagKey, Flag),
+  [Flag] { *Flag = true; },
+  };
 }
 
-Context setCurrentTask(ConstTaskHandle TH) {
-  assert(TH && "Trying to stash a nullptr as TaskHandle into context.");
-  return Context::current().derive(TaskKey, std::move(TH));
+bool isCancelled() {
+  if (auto *Flag = Context::current().get(FlagKey))
+return **Flag;
+  return false; // Not in scope of a task.
 }
 
 } // namespace clangd

Modified: clang-tools-extra/trunk/clangd/Cancellation.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/Cancellation.h?rev=342130&r1=342129&r2=342130&view=diff
==
--- clang-tools-extra/trunk/clangd/Cancellation.h (original)
+++ clang-tools-extra/trunk/clangd/Cancellation.h Thu Sep 13 04:47:48 2018
@@ -6,124 +6,82 @@
 // License. See LICENSE.TXT for details.
 //
 
//===--===//
-// Cancellation mechanism for async tasks. Roughly all the clients of this code
-// can be classified into three categories:
-// 1. The code that creates and schedules async tasks, e.g. TUScheduler.
-// 2. The callers of the async method that can cancel some of the running 
tasks,
-// e.g. `ClangdLSPServer`
-// 3. The code running inside the async task itself, i.e. code completion or
-// find definition implementation that run clang, etc.
-//
-// For (1), the guideline is to accept a callback for the result of async
-// operation and return a `TaskHandle` to allow cancelling the request.
-//
-//  TaskHandle someAsyncMethod(Runnable T,
-//  function)> Callback) {
-//   auto TH = Task::createHandle();
-//   WithContext ContextWithCancellationToken(TH);
-//   auto run = [](){
-// Callback(T());
+// Cancellation mechanism for long-running tasks.
+//
+// This manages interactions between:
+//
+// 1. Client code that starts some long-running work, and maybe cancels later.
+//
+//   std::pair Task = cancelableTask();
+//   {
+// WithContext Cancelable(std::move(Task.first));
+// Expected
+// deepThoughtAsync([](int answer){ errs() << answer; });
+//   }
+//   // ...some time later...
+//   if (User.fellAsleep())
+// Task.second();
+//
+//  (This example has an asynchronous computation, but synchronous examples
+//  work similarly - the Canceler should 

[clang-tools-extra] r342135 - [clangd] Allow all LSP methods to signal cancellation via $/cancelRequest

[Sam McCall via cfe-commits]

Author: sammccall
Date: Thu Sep 13 05:58:36 2018
New Revision: 342135

URL: http://llvm.org/viewvc/llvm-project?rev=342135&view=rev
Log:
[clangd] Allow all LSP methods to signal cancellation via $/cancelRequest

Summary:
The cancelable scopes are managed by JSONRPCDispatcher so that all Handlers
run in cancelable contexts.
(Previously ClangdServer did this, for code completion only).

Cancellation request processing is therefore also in JSONRPCDispatcher.
(Previously it was in ClangdLSPServer).

This doesn't actually make any new commands *respect* cancellation - they'd
need to check isCancelled() and bail out. But it opens the door to doing
this incrementally, and putting such logic in common machinery like TUScheduler.

I also rewrote the ClangdServer class/threading comments because I wanted to
add to it and I got carried away.

Reviewers: ilya-biryukov, kadircet

Subscribers: ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D52004

Modified:
clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
clang-tools-extra/trunk/clangd/ClangdLSPServer.h
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.h
clang-tools-extra/trunk/clangd/JSONRPCDispatcher.cpp
clang-tools-extra/trunk/clangd/JSONRPCDispatcher.h
clang-tools-extra/trunk/clangd/Protocol.cpp
clang-tools-extra/trunk/clangd/Protocol.h
clang-tools-extra/trunk/clangd/ProtocolHandlers.cpp
clang-tools-extra/trunk/clangd/ProtocolHandlers.h

Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=342135&r1=342134&r2=342135&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp Thu Sep 13 05:58:36 2018
@@ -8,7 +8,6 @@
 
//===--===//
 
 #include "ClangdLSPServer.h"
-#include "Cancellation.h"
 #include "Diagnostics.h"
 #include "JSONRPCDispatcher.h"
 #include "SourceCode.h"
@@ -71,11 +70,6 @@ SymbolKindBitset defaultSymbolKinds() {
   return Defaults;
 }
 
-std::string NormalizeRequestID(const json::Value &ID) {
-  auto NormalizedID = parseNumberOrString(&ID);
-  assert(NormalizedID && "Was not able to parse request id.");
-  return std::move(*NormalizedID);
-}
 } // namespace
 
 void ClangdLSPServer::onInitialize(InitializeParams &Params) {
@@ -347,21 +341,16 @@ void ClangdLSPServer::onCodeAction(CodeA
 }
 
 void ClangdLSPServer::onCompletion(TextDocumentPositionParams &Params) {
-  CreateSpaceForTaskHandle();
-  Canceler Cancel = Server.codeComplete(
-  Params.textDocument.uri.file(), Params.position, CCOpts,
-  [this](llvm::Expected List) {
-auto _ = llvm::make_scope_exit([this]() { CleanupTaskHandle(); });
-
-if (!List)
-  return replyError(List.takeError());
-CompletionList LSPList;
-LSPList.isIncomplete = List->HasMore;
-for (const auto &R : List->Completions)
-  LSPList.items.push_back(R.render(CCOpts));
-return reply(std::move(LSPList));
-  });
-  StoreTaskHandle(std::move(Cancel));
+  Server.codeComplete(Params.textDocument.uri.file(), Params.position, CCOpts,
+  [this](llvm::Expected List) {
+if (!List)
+  return replyError(List.takeError());
+CompletionList LSPList;
+LSPList.isIncomplete = List->HasMore;
+for (const auto &R : List->Completions)
+  LSPList.items.push_back(R.render(CCOpts));
+return reply(std::move(LSPList));
+  });
 }
 
 void ClangdLSPServer::onSignatureHelp(TextDocumentPositionParams &Params) {
@@ -629,48 +618,3 @@ GlobalCompilationDatabase &ClangdLSPServ
 return *CachingCDB;
   return *CDB;
 }
-
-void ClangdLSPServer::onCancelRequest(CancelParams &Params) {
-  std::lock_guard Lock(TaskHandlesMutex);
-  const auto &It = TaskHandles.find(Params.ID);
-  if (It == TaskHandles.end())
-return;
-  It->second();
-  TaskHandles.erase(It);
-}
-
-void ClangdLSPServer::CleanupTaskHandle() {
-  const json::Value *ID = getRequestId();
-  if (!ID)
-return;
-  std::string NormalizedID = NormalizeRequestID(*ID);
-  std::lock_guard Lock(TaskHandlesMutex);
-  TaskHandles.erase(NormalizedID);
-}
-
-void ClangdLSPServer::CreateSpaceForTaskHandle() {
-  const json::Value *ID = getRequestId();
-  if (!ID)
-return;
-  std::string NormalizedID = NormalizeRequestID(*ID);
-  std::lock_guard Lock(TaskHandlesMutex);
-  if (!TaskHandles.insert({NormalizedID, nullptr}).second)
-elog("Creation of space for task handle: {0} failed.", NormalizedID);
-}
-
-void ClangdLSPServer::StoreTaskHandle(Canceler TH) {
-  const json::Value *

r342228 - [Tooling] JSONCompilationDatabasePlugin infers compile commands for missing files

[Sam McCall via cfe-commits]

Author: sammccall
Date: Fri Sep 14 05:24:09 2018
New Revision: 342228

URL: http://llvm.org/viewvc/llvm-project?rev=342228&view=rev
Log:
[Tooling] JSONCompilationDatabasePlugin infers compile commands for missing 
files

Summary:
See the existing InterpolatingCompilationDatabase for details on how this works.
We've been using this in clangd for a while, the heuristics seem to work well.

Reviewers: bkramer

Subscribers: ilya-biryukov, ioeric, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51729

Modified:
cfe/trunk/lib/Tooling/JSONCompilationDatabase.cpp
cfe/trunk/test/Tooling/auto-detect-from-source.cpp

Modified: cfe/trunk/lib/Tooling/JSONCompilationDatabase.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/JSONCompilationDatabase.cpp?rev=342228&r1=342227&r2=342228&view=diff
==
--- cfe/trunk/lib/Tooling/JSONCompilationDatabase.cpp (original)
+++ cfe/trunk/lib/Tooling/JSONCompilationDatabase.cpp Fri Sep 14 05:24:09 2018
@@ -157,13 +157,16 @@ std::vector unescapeCommand
   return parser.parse();
 }
 
+// This plugin locates a nearby compile_command.json file, and also infers
+// compile commands for files not present in the database.
 class JSONCompilationDatabasePlugin : public CompilationDatabasePlugin {
   std::unique_ptr
   loadFromDirectory(StringRef Directory, std::string &ErrorMessage) override {
 SmallString<1024> JSONDatabasePath(Directory);
 llvm::sys::path::append(JSONDatabasePath, "compile_commands.json");
-return JSONCompilationDatabase::loadFromFile(
+auto Base = JSONCompilationDatabase::loadFromFile(
 JSONDatabasePath, ErrorMessage, JSONCommandLineSyntax::AutoDetect);
+return Base ? inferMissingCompileCommands(std::move(Base)) : nullptr;
   }
 };
 

Modified: cfe/trunk/test/Tooling/auto-detect-from-source.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Tooling/auto-detect-from-source.cpp?rev=342228&r1=342227&r2=342228&view=diff
==
--- cfe/trunk/test/Tooling/auto-detect-from-source.cpp (original)
+++ cfe/trunk/test/Tooling/auto-detect-from-source.cpp Fri Sep 14 05:24:09 2018
@@ -1,8 +1,12 @@
 // RUN: rm -rf %t
 // RUN: mkdir %t
-// RUN: echo "[{\"directory\":\".\",\"command\":\"clang++ -c 
%/t/test.cpp\",\"file\":\"%/t/test.cpp\"}]" | sed -e 's/\\//g' > 
%t/compile_commands.json
+// RUN: echo "[{\"directory\":\".\",\"command\":\"clang++ -DSECRET=XYZZY -c 
%/t/test.cpp\",\"file\":\"%/t/test.cpp\"}]" | sed -e 's/\\//g' > 
%t/compile_commands.json
 // RUN: cp "%s" "%t/test.cpp"
 // RUN: not clang-check "%t/test.cpp" 2>&1 | FileCheck %s
 
-// CHECK: C++ requires
-invalid;
+// CHECK: XYZZY
+SECRET;
+
+// Copy to a different file, and rely on the command being inferred.
+// RUN: cp "%s" "%t/other.cpp"
+// RUN: not clang-check "%t/other.cpp" 2>&1 | FileCheck %s


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r342229 - [clangd] Don't double-infer compile commands after r342228

[Sam McCall via cfe-commits]

Author: sammccall
Date: Fri Sep 14 05:32:08 2018
New Revision: 342229

URL: http://llvm.org/viewvc/llvm-project?rev=342229&view=rev
Log:
[clangd] Don't double-infer compile commands after r342228

Modified:
clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.cpp

Modified: clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.cpp?rev=342229&r1=342228&r2=342229&view=diff
==
--- clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.cpp (original)
+++ clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.cpp Fri Sep 14 
05:32:08 2018
@@ -95,8 +95,6 @@ DirectoryBasedGlobalCompilationDatabase:
 return CachedIt->second.get();
   std::string Error = "";
   auto CDB = tooling::CompilationDatabase::loadFromDirectory(Dir, Error);
-  if (CDB)
-CDB = tooling::inferMissingCompileCommands(std::move(CDB));
   auto Result = CDB.get();
   CompilationDatabases.insert(std::make_pair(Dir, std::move(CDB)));
   return Result;


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r342230 - [clangd] Don't override the preamble while completing inside it, it doesn't work.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Fri Sep 14 05:36:06 2018
New Revision: 342230

URL: http://llvm.org/viewvc/llvm-project?rev=342230&view=rev
Log:
[clangd] Don't override the preamble while completing inside it, it doesn't 
work.

Summary:
To stay fast, enable single-file-mode instead. This is fine since completions
in the preamble are simple.

The net effect for now is to suppress the spurious TopLevel completions when
completing inside the preamble.
Once Sema has include directive completion, this will be more important.

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52071

Modified:
clang-tools-extra/trunk/clangd/CodeComplete.cpp
clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp

Modified: clang-tools-extra/trunk/clangd/CodeComplete.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/CodeComplete.cpp?rev=342230&r1=342229&r2=342230&view=diff
==
--- clang-tools-extra/trunk/clangd/CodeComplete.cpp (original)
+++ clang-tools-extra/trunk/clangd/CodeComplete.cpp Fri Sep 14 05:36:06 2018
@@ -40,6 +40,7 @@
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/FrontendActions.h"
 #include "clang/Index/USRGeneration.h"
+#include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Sema/CodeCompleteConsumer.h"
 #include "clang/Sema/Sema.h"
 #include "clang/Tooling/Core/Replacement.h"
@@ -1053,11 +1054,19 @@ bool semaCodeComplete(std::unique_ptrgetLangOpts(), ContentsBuffer.get(), 0).Size >
+  *Offset;
   // NOTE: we must call BeginSourceFile after prepareCompilerInstance. 
Otherwise
   // the remapped buffers do not get freed.
   auto Clang = prepareCompilerInstance(
-  std::move(CI), Input.Preamble, std::move(ContentsBuffer),
-  std::move(Input.PCHs), std::move(Input.VFS), DummyDiagsConsumer);
+  std::move(CI), CompletingInPreamble ? nullptr : Input.Preamble,
+  std::move(ContentsBuffer), std::move(Input.PCHs), std::move(Input.VFS),
+  DummyDiagsConsumer);
+  Clang->getPreprocessorOpts().SingleFileParseMode = CompletingInPreamble;
   Clang->setCodeCompletionConsumer(Consumer.release());
 
   SyntaxOnlyAction Action;

Modified: clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp?rev=342230&r1=342229&r2=342230&view=diff
==
--- clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp (original)
+++ clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp Fri Sep 14 
05:36:06 2018
@@ -657,6 +657,22 @@ TEST(CompletionTest, IndexSuppressesPrea
   UnorderedElementsAre(Named("local"), Named("preamble")));
 }
 
+// This verifies that we get normal preprocessor completions in the preamble.
+// This is a regression test for an old bug: if we override the preamble and
+// try to complete inside it, clang kicks our completion point just outside the
+// preamble, resulting in always getting top-level completions.
+TEST(CompletionTest, CompletionInPreamble) {
+  EXPECT_THAT(completions(R"cpp(
+#ifnd^ef FOO_H_
+#define BAR_H_
+#include 
+int foo() {}
+#endif
+)cpp")
+  .Completions,
+  ElementsAre(Named("ifndef")));
+};
+
 TEST(CompletionTest, DynamicIndexMultiFile) {
   MockFSProvider FS;
   MockCompilationDatabase CDB;


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r342232 - [VFS] vfs::directory_iterator yields path and file type instead of full Status

[Sam McCall via cfe-commits]

Author: sammccall
Date: Fri Sep 14 05:47:38 2018
New Revision: 342232

URL: http://llvm.org/viewvc/llvm-project?rev=342232&view=rev
Log:
[VFS] vfs::directory_iterator yields path and file type instead of full Status

Summary:
Most callers I can find are using only `getName()`. Type is used by the
recursive iterator.

Now we don't have to call stat() on every listed file (on most platforms).
Exceptions are e.g. Solaris where readdir() doesn't include type information.
On those platforms we'll still stat() - see D51918.

The result is significantly faster (stat() can be slow).
My motivation: this may allow us to improve clang IO on large TUs with long
include search paths. Caching readdir() results may allow us to skip many stat()
and open() operations on nonexistent files.

Reviewers: bkramer

Subscribers: fedor.sergeev, cfe-commits

Differential Revision: https://reviews.llvm.org/D51921

Modified:
cfe/trunk/include/clang/Basic/VirtualFileSystem.h
cfe/trunk/lib/Basic/VirtualFileSystem.cpp
cfe/trunk/lib/Driver/ToolChains/BareMetal.cpp
cfe/trunk/lib/Driver/ToolChains/Gnu.cpp
cfe/trunk/lib/Frontend/CompilerInstance.cpp
cfe/trunk/lib/Frontend/FrontendAction.cpp
cfe/trunk/lib/Lex/HeaderSearch.cpp
cfe/trunk/lib/Lex/ModuleMap.cpp
cfe/trunk/lib/Lex/PPLexerChange.cpp
cfe/trunk/unittests/Basic/VirtualFileSystemTest.cpp

Modified: cfe/trunk/include/clang/Basic/VirtualFileSystem.h
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/VirtualFileSystem.h?rev=342232&r1=342231&r2=342232&view=diff
==
--- cfe/trunk/include/clang/Basic/VirtualFileSystem.h (original)
+++ cfe/trunk/include/clang/Basic/VirtualFileSystem.h Fri Sep 14 05:47:38 2018
@@ -126,6 +126,21 @@ public:
   virtual std::error_code close() = 0;
 };
 
+/// A member of a directory, yielded by a directory_iterator.
+/// Only information available on most platforms is included.
+class directory_entry {
+  std::string Path;
+  llvm::sys::fs::file_type Type;
+
+public:
+  directory_entry() = default;
+  directory_entry(std::string Path, llvm::sys::fs::file_type Type)
+  : Path(std::move(Path)), Type(Type) {}
+
+  llvm::StringRef path() const { return Path; }
+  llvm::sys::fs::file_type type() const { return Type; }
+};
+
 namespace detail {
 
 /// An interface for virtual file systems to provide an iterator over the
@@ -134,10 +149,10 @@ struct DirIterImpl {
   virtual ~DirIterImpl();
 
   /// Sets \c CurrentEntry to the next entry in the directory on success,
-  /// or returns a system-defined \c error_code.
+  /// to directory_entry() at end,  or returns a system-defined \c error_code.
   virtual std::error_code increment() = 0;
 
-  Status CurrentEntry;
+  directory_entry CurrentEntry;
 };
 
 } // namespace detail
@@ -151,7 +166,7 @@ public:
   directory_iterator(std::shared_ptr I)
   : Impl(std::move(I)) {
 assert(Impl.get() != nullptr && "requires non-null implementation");
-if (!Impl->CurrentEntry.isStatusKnown())
+if (Impl->CurrentEntry.path().empty())
   Impl.reset(); // Normalize the end iterator to Impl == nullptr.
   }
 
@@ -162,17 +177,17 @@ public:
   directory_iterator &increment(std::error_code &EC) {
 assert(Impl && "attempting to increment past end");
 EC = Impl->increment();
-if (!Impl->CurrentEntry.isStatusKnown())
+if (Impl->CurrentEntry.path().empty())
   Impl.reset(); // Normalize the end iterator to Impl == nullptr.
 return *this;
   }
 
-  const Status &operator*() const { return Impl->CurrentEntry; }
-  const Status *operator->() const { return &Impl->CurrentEntry; }
+  const directory_entry &operator*() const { return Impl->CurrentEntry; }
+  const directory_entry *operator->() const { return &Impl->CurrentEntry; }
 
   bool operator==(const directory_iterator &RHS) const {
 if (Impl && RHS.Impl)
-  return Impl->CurrentEntry.equivalent(RHS.Impl->CurrentEntry);
+  return Impl->CurrentEntry.path() == RHS.Impl->CurrentEntry.path();
 return !Impl && !RHS.Impl;
   }
   bool operator!=(const directory_iterator &RHS) const {
@@ -201,8 +216,8 @@ public:
   /// Equivalent to operator++, with an error code.
   recursive_directory_iterator &increment(std::error_code &EC);
 
-  const Status &operator*() const { return *State->top(); }
-  const Status *operator->() const { return &*State->top(); }
+  const directory_entry &operator*() const { return *State->top(); }
+  const directory_entry *operator->() const { return &*State->top(); }
 
   bool operator==(const recursive_directory_iterator &Other) const {
 return State == Other.State; // identity

Modified: cfe/trunk/lib/Basic/VirtualFileSystem.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/VirtualFileSystem.cpp?rev=342232&r1=342231&r2=342232&view=diff
==
--- cfe/trunk/lib/Basic/VirtualFileSystem.cpp (origin

[clang-tools-extra] r342261 - [clangd] Work around compiler macro expansion bugs(?) in completion tests

[Sam McCall via cfe-commits]

Author: sammccall
Date: Fri Sep 14 11:49:16 2018
New Revision: 342261

URL: http://llvm.org/viewvc/llvm-project?rev=342261&view=rev
Log:
[clangd] Work around compiler macro expansion bugs(?) in completion tests

Modified:
clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp

Modified: clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp?rev=342261&r1=342260&r2=342261&view=diff
==
--- clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp (original)
+++ clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp Fri Sep 14 
11:49:16 2018
@@ -662,15 +662,15 @@ TEST(CompletionTest, IndexSuppressesPrea
 // try to complete inside it, clang kicks our completion point just outside the
 // preamble, resulting in always getting top-level completions.
 TEST(CompletionTest, CompletionInPreamble) {
-  EXPECT_THAT(completions(R"cpp(
+  auto Results = completions(R"cpp(
 #ifnd^ef FOO_H_
 #define BAR_H_
 #include 
 int foo() {}
 #endif
 )cpp")
-  .Completions,
-  ElementsAre(Named("ifndef")));
+ .Completions;
+  EXPECT_THAT(Results, ElementsAre(Named("ifndef")));
 };
 
 TEST(CompletionTest, DynamicIndexMultiFile) {


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r337527 - [clangd] FuzzyMatch exposes an API for its word segmentation. NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Fri Jul 20 01:01:37 2018
New Revision: 337527

URL: http://llvm.org/viewvc/llvm-project?rev=337527&view=rev
Log:
[clangd] FuzzyMatch exposes an API for its word segmentation. NFC

Summary: This is intended to be used for indexing, e.g. in D49417

Reviewers: ioeric, omtcyfz

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D49540

Modified:
clang-tools-extra/trunk/clangd/FuzzyMatch.cpp
clang-tools-extra/trunk/clangd/FuzzyMatch.h
clang-tools-extra/trunk/unittests/clangd/FuzzyMatchTests.cpp

Modified: clang-tools-extra/trunk/clangd/FuzzyMatch.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/FuzzyMatch.cpp?rev=337527&r1=337526&r2=337527&view=diff
==
--- clang-tools-extra/trunk/clangd/FuzzyMatch.cpp (original)
+++ clang-tools-extra/trunk/clangd/FuzzyMatch.cpp Fri Jul 20 01:01:37 2018
@@ -87,8 +87,8 @@ FuzzyMatcher::FuzzyMatcher(StringRef Pat
 for (int W = 0; W < P; ++W)
   for (Action A : {Miss, Match})
 Scores[P][W][A] = {AwfulScore, Miss};
-  if (PatN > 0)
-calculateRoles(Pat, PatRole, PatTypeSet, PatN);
+  PatTypeSet =
+  calculateRoles(StringRef(Pat, PatN), makeMutableArrayRef(PatRole, PatN));
 }
 
 Optional FuzzyMatcher::match(StringRef Word) {
@@ -110,25 +110,6 @@ Optional FuzzyMatcher::match(Stri
   return Score;
 }
 
-// Segmentation of words and patterns.
-// A name like "fooBar_baz" consists of several parts foo, bar, baz.
-// Aligning segmentation of word and pattern improves the fuzzy-match.
-// For example: [lol] matches "LaughingOutLoud" better than "LionPopulation"
-//
-// First we classify each character into types (uppercase, lowercase, etc).
-// Then we look at the sequence: e.g. [upper, lower] is the start of a segment.
-
-// We only distinguish the types of characters that affect segmentation.
-// It's not obvious how to segment digits, we treat them as lowercase letters.
-// As we don't decode UTF-8, we treat bytes over 127 as lowercase too.
-// This means we require exact (case-sensitive) match.
-enum FuzzyMatcher::CharType : unsigned char {
-  Empty = 0,   // Before-the-start and after-the-end (and control chars).
-  Lower = 1,   // Lowercase letters, digits, and non-ASCII bytes.
-  Upper = 2,   // Uppercase letters.
-  Punctuation = 3, // ASCII punctuation (including Space)
-};
-
 // We get CharTypes from a lookup table. Each is 2 bits, 4 fit in each byte.
 // The top 6 bits of the char select the byte, the bottom 2 select the offset.
 // e.g. 'q' = 010100 01 = byte 28 (55), bits 3-2 (01) -> Lower.
@@ -147,17 +128,6 @@ constexpr static uint8_t CharTypes[] = {
 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55,
 };
 
-// Each character's Role is the Head or Tail of a segment, or a Separator.
-// e.g. XMLHttpRequest_Async
-//  +--+---+-- +
-//  ^Head   ^Tail ^Separator
-enum FuzzyMatcher::CharRole : unsigned char {
-  Unknown = 0,   // Stray control characters or impossible states.
-  Tail = 1,  // Part of a word segment, but not the first character.
-  Head = 2,  // The first character of a word segment.
-  Separator = 3, // Punctuation characters that separate word segments.
-};
-
 // The Role can be determined from the Type of a character and its neighbors:
 //
 //   Example  | Chars | Type | Role
@@ -183,26 +153,28 @@ constexpr static uint8_t CharRoles[] = {
 template  static T packedLookup(const uint8_t *Data, int I) {
   return static_cast((Data[I >> 2] >> ((I & 3) * 2)) & 3);
 }
-void FuzzyMatcher::calculateRoles(const char *Text, CharRole *Out, int 
&TypeSet,
-  int N) {
-  assert(N > 0);
+CharTypeSet calculateRoles(StringRef Text, MutableArrayRef Roles) {
+  assert(Text.size() == Roles.size());
+  if (Text.size() == 0)
+return 0;
   CharType Type = packedLookup(CharTypes, Text[0]);
-  TypeSet = 1 << Type;
+  CharTypeSet TypeSet = 1 << Type;
   // Types holds a sliding window of (Prev, Curr, Next) types.
   // Initial value is (Empty, Empty, type of Text[0]).
   int Types = Type;
   // Rotate slides in the type of the next character.
   auto Rotate = [&](CharType T) { Types = ((Types << 2) | T) & 0x3f; };
-  for (int I = 0; I < N - 1; ++I) {
+  for (unsigned I = 0; I < Text.size() - 1; ++I) {
 // For each character, rotate in the next, and look up the role.
 Type = packedLookup(CharTypes, Text[I + 1]);
 TypeSet |= 1 << Type;
 Rotate(Type);
-*Out++ = packedLookup(CharRoles, Types);
+Roles[I] = packedLookup(CharRoles, Types);
   }
   // For the last character, the "next character" is Empty.
   Rotate(Empty);
-  *Out++ = packedLookup(CharRoles, Types);
+  Roles[Text.size() - 1] = packedLookup(CharRoles, Types);
+  return TypeSet;
 }
 
 // Sets up the data structures matching Word.
@@ -228,7 +200,8 @@ bool FuzzyMatcher::init(StringRef NewWor
   // FIXME: some wor

r337682 - [Tooling] Use UniqueStringSaver. NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Jul 23 04:25:25 2018
New Revision: 337682

URL: http://llvm.org/viewvc/llvm-project?rev=337682&view=rev
Log:
[Tooling] Use UniqueStringSaver. NFC

Modified:
cfe/trunk/include/clang/Tooling/Execution.h
cfe/trunk/lib/Tooling/Execution.cpp

Modified: cfe/trunk/include/clang/Tooling/Execution.h
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Tooling/Execution.h?rev=337682&r1=337681&r2=337682&view=diff
==
--- cfe/trunk/include/clang/Tooling/Execution.h (original)
+++ cfe/trunk/include/clang/Tooling/Execution.h Mon Jul 23 04:25:25 2018
@@ -57,7 +57,7 @@ public:
 /// set of different results, or a large set of duplicated results.
 class InMemoryToolResults : public ToolResults {
 public:
-  InMemoryToolResults() : StringsPool(Arena) {}
+  InMemoryToolResults() : Strings(Arena) {}
   void addResult(StringRef Key, StringRef Value) override;
   std::vector>
   AllKVResults() override;
@@ -66,8 +66,7 @@ public:
 
 private:
   llvm::BumpPtrAllocator Arena;
-  llvm::StringSaver StringsPool;
-  llvm::DenseSet Strings;
+  llvm::UniqueStringSaver Strings;
 
   std::vector> KVResults;
 };

Modified: cfe/trunk/lib/Tooling/Execution.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Tooling/Execution.cpp?rev=337682&r1=337681&r2=337682&view=diff
==
--- cfe/trunk/lib/Tooling/Execution.cpp (original)
+++ cfe/trunk/lib/Tooling/Execution.cpp Mon Jul 23 04:25:25 2018
@@ -21,16 +21,7 @@ static llvm::cl::opt
  llvm::cl::init("standalone"));
 
 void InMemoryToolResults::addResult(StringRef Key, StringRef Value) {
-  auto Intern = [&](StringRef &V) {
-auto R = Strings.insert(V);
-if (R.second) { // A new entry, create a new string copy.
-  *R.first = StringsPool.save(V);
-}
-V = *R.first;
-  };
-  Intern(Key);
-  Intern(Value);
-  KVResults.push_back({Key, Value});
+  KVResults.push_back({Strings.save(Key), Strings.save(Value)});
 }
 
 std::vector>


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [PATCH] D49657: [clangd] Make SymbolLocation => bool conversion explicitly.

[Sam McCall via cfe-commits]

Or make operator bool explicit

On Mon, Jul 23, 2018, 13:55 Ilya Biryukov via Phabricator <
revi...@reviews.llvm.org> wrote:

> ilya-biryukov accepted this revision.
> ilya-biryukov added a comment.
> This revision is now accepted and ready to land.
>
> Scary
> Definitely LGTM!
>
>
>
> 
> Comment at: clangd/index/Index.h:45
>
> -  operator bool() const { return !FileURI.empty(); }
> +  explicit operator bool() const { return !FileURI.empty(); }
> +  bool operator==(const SymbolLocation& Loc) const {
> 
> Maybe we should go even further and replace this with an `isValid` method?
> Conversion ops are confusing.
>
>
> Repository:
>   rCTE Clang Tools Extra
>
> https://reviews.llvm.org/D49657
>
>
>
>
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r337834 - [VFS] Cleanups to VFS interfaces.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Jul 24 09:00:55 2018
New Revision: 337834

URL: http://llvm.org/viewvc/llvm-project?rev=337834&view=rev
Log:
[VFS] Cleanups to VFS interfaces.

Summary:
- add comments clarifying semantics
- Status::copyWithNewName(Status, Name) --> instance method
- Status::copyWithNewName(fs::file_status, Name) --> constructor (it's not a 
copy)
- File::getName() -> getRealPath(), reflecting its actual behavior/function
  and stop returning status().getName() in the base class (callers can do this
  fallback if they want to, it complicates the contracts).

This is mostly NFC, but the behavior of File::getName() affects FileManager's
FileEntry::tryGetRealPathName(), which now fails in more cases:
 - non-real file cases
 - real-file cases where the underlying vfs::File was opened in a way that
   doesn't call realpath().
(In these cases we don't know a distinct real name, so in principle it seems OK)

Reviewers: klimek

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D49724

Modified:
cfe/trunk/include/clang/Basic/VirtualFileSystem.h
cfe/trunk/lib/Basic/FileManager.cpp
cfe/trunk/lib/Basic/VirtualFileSystem.cpp

Modified: cfe/trunk/include/clang/Basic/VirtualFileSystem.h
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/VirtualFileSystem.h?rev=337834&r1=337833&r2=337834&view=diff
==
--- cfe/trunk/include/clang/Basic/VirtualFileSystem.h (original)
+++ cfe/trunk/include/clang/Basic/VirtualFileSystem.h Tue Jul 24 09:00:55 2018
@@ -45,7 +45,8 @@ class MemoryBuffer;
 namespace clang {
 namespace vfs {
 
-/// The result of a \p status operation.
+/// File information from a \p File::status() operation.
+/// This is roughly equivalent to a `struct stat` plus a file path.
 class Status {
   std::string Name;
   llvm::sys::fs::UniqueID UID;
@@ -66,13 +67,14 @@ public:
  llvm::sys::TimePoint<> MTime, uint32_t User, uint32_t Group,
  uint64_t Size, llvm::sys::fs::file_type Type,
  llvm::sys::fs::perms Perms);
+  Status(const llvm::sys::fs::file_status &FSStatus, StringRef Name);
 
-  /// Get a copy of a Status with a different name.
-  static Status copyWithNewName(const Status &In, StringRef NewName);
-  static Status copyWithNewName(const llvm::sys::fs::file_status &In,
-StringRef NewName);
+  /// Get a copy of a this Status with a different name.
+  Status copyWithNewName(StringRef NewName);
 
   /// Returns the name that should be used for this file or directory.
+  /// This is usually the path that the file was opened as, without resolving
+  /// relative paths or symlinks.
   StringRef getName() const { return Name; }
 
   /// @name Status interface from llvm::sys::fs
@@ -107,15 +109,16 @@ public:
   virtual ~File();
 
   /// Get the status of the file.
+  /// This may access the filesystem (e.g. `stat()`), or return a cached value.
   virtual llvm::ErrorOr status() = 0;
 
-  /// Get the name of the file
-  virtual llvm::ErrorOr getName() {
-if (auto Status = status())
-  return Status->getName().str();
-else
-  return Status.getError();
-  }
+  /// Get the "real name" of the file, if available.
+  /// This should be absolute, and typically has symlinks resolved.
+  ///
+  /// Only some VFS implementations provide this, and only sometimes.
+  /// FIXME: these maybe-available semantics are not very useful. It would be
+  /// nice if this was more consistent with FileSystem::getRealPath().
+  virtual llvm::Optional getRealPath() { return llvm::None; }
 
   /// Get the contents of the file as a \p MemoryBuffer.
   virtual llvm::ErrorOr>

Modified: cfe/trunk/lib/Basic/FileManager.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/FileManager.cpp?rev=337834&r1=337833&r2=337834&view=diff
==
--- cfe/trunk/lib/Basic/FileManager.cpp (original)
+++ cfe/trunk/lib/Basic/FileManager.cpp Tue Jul 24 09:00:55 2018
@@ -316,7 +316,7 @@ const FileEntry *FileManager::getFile(St
   UFE.File = std::move(F);
   UFE.IsValid = true;
   if (UFE.File)
-if (auto RealPathName = UFE.File->getName())
+if (auto RealPathName = UFE.File->getRealPath())
   UFE.RealPathName = *RealPathName;
   return &UFE;
 }

Modified: cfe/trunk/lib/Basic/VirtualFileSystem.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/VirtualFileSystem.cpp?rev=337834&r1=337833&r2=337834&view=diff
==
--- cfe/trunk/lib/Basic/VirtualFileSystem.cpp (original)
+++ cfe/trunk/lib/Basic/VirtualFileSystem.cpp Tue Jul 24 09:00:55 2018
@@ -73,16 +73,14 @@ Status::Status(StringRef Name, UniqueID
 : Name(Name), UID(UID), MTime(MTime), User(User), Group(Group), Size(Size),
   Type(Type), Perms(Perms) {}
 
-Status Status::copyWithNewName(const Status &In, StringRef NewName) {

[clang-tools-extra] r324334 - [clangd] Cut input-mirror.test down to size. NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Feb  6 02:51:22 2018
New Revision: 324334

URL: http://llvm.org/viewvc/llvm-project?rev=324334&view=rev
Log:
[clangd] Cut input-mirror.test down to size. NFC

Modified:
clang-tools-extra/trunk/test/clangd/input-mirror.test

Modified: clang-tools-extra/trunk/test/clangd/input-mirror.test
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/clangd/input-mirror.test?rev=324334&r1=324333&r2=324334&view=diff
==
--- clang-tools-extra/trunk/test/clangd/input-mirror.test (original)
+++ clang-tools-extra/trunk/test/clangd/input-mirror.test Tue Feb  6 02:51:22 
2018
@@ -6,152 +6,9 @@
 Content-Length: 125
 
 
{"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":123,"rootPath":"clangd","capabilities":{},"trace":"off"}}
-
 Content-Length: 172
 
 
{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"test:///main.cpp","languageId":"cpp","version":1,"text":"int
 main() {\nint a;\na;\n}\n"}}}
-
-Content-Length: 148
-
-{"jsonrpc":"2.0","id":1,"method":"textDocument/definition","params":{"textDocument":{"uri":"test:///main.cpp"},"position":{"line":2,"character":0}}}
-# Go to local variable
-
-Content-Length: 148
-
-{"jsonrpc":"2.0","id":1,"method":"textDocument/definition","params":{"textDocument":{"uri":"test:///main.cpp"},"position":{"line":2,"character":1}}}
-# Go to local variable, end of token
-
-Content-Length: 214
-
-{"jsonrpc":"2.0","method":"textDocument/didChange","params":{"textDocument":{"uri":"test:///main.cpp","version":2},"contentChanges":[{"text":"struct
 Foo {\nint x;\n};\nint main() {\n  Foo bar = { x : 1 };\n}\n"}]}}
-
-Content-Length: 149
-
-{"jsonrpc":"2.0","id":1,"method":"textDocument/definition","params":{"textDocument":{"uri":"test:///main.cpp"},"position":{"line":4,"character":14}}}
-# Go to field, GNU old-style field designator 
-
-Content-Length: 215
-
-{"jsonrpc":"2.0","method":"textDocument/didChange","params":{"textDocument":{"uri":"test:///main.cpp","version":3},"contentChanges":[{"text":"struct
 Foo {\nint x;\n};\nint main() {\n  Foo baz = { .x = 2 };\n}\n"}]}}
-
-Content-Length: 149
-
-{"jsonrpc":"2.0","id":1,"method":"textDocument/definition","params":{"textDocument":{"uri":"test:///main.cpp"},"position":{"line":4,"character":15}}}
-# Go to field, field designator 
-
-Content-Length: 187
-
-{"jsonrpc":"2.0","method":"textDocument/didChange","params":{"textDocument":{"uri":"test:///main.cpp","version":4},"contentChanges":[{"text":"int
 main() {\n   main();\n   return 0;\n}"}]}}
-
-Content-Length: 148
-
-{"jsonrpc":"2.0","id":1,"method":"textDocument/definition","params":{"textDocument":{"uri":"test:///main.cpp"},"position":{"line":1,"character":3}}}
-# Go to function declaration, function call 
-
-Content-Length: 208
-
-{"jsonrpc":"2.0","method":"textDocument/didChange","params":{"textDocument":{"uri":"test:///main.cpp","version":5},"contentChanges":[{"text":"struct
 Foo {\n};\nint main() {\n   Foo bar;\n   return 0;\n}\n"}]}}
-
-Content-Length: 148
-
-{"jsonrpc":"2.0","id":1,"method":"textDocument/definition","params":{"textDocument":{"uri":"test:///main.cpp"},"position":{"line":3,"character":3}}}
-# Go to struct declaration, new struct instance 
-
-Content-Length: 231
-
-{"jsonrpc":"2.0","method":"textDocument/didChange","params":{"textDocument":{"uri":"test:///main.cpp","version":5},"contentChanges":[{"text":"namespace
 n1 {\nstruct Foo {\n};\n}\nint main() {\n   n1::Foo bar;\n   return 
0;\n}\n"}]}}
-
-Content-Length: 148
-
-{"jsonrpc":"2.0","id":1,"method":"textDocument/definition","params":{"textDocument":{"uri":"test:///main.cpp"},"position":{"line":5,"character":4}}}
-# Go to struct declaration, new struct instance, qualified name 
-
-Content-Length: 215
-
-{"jsonrpc":"2.0","method":"textDocument/didChange","params":{"textDocument":{"uri":"test:///main.cpp","version":6},"contentChanges":[{"text":"struct
 Foo {\n  int x;\n};\nint main() {\n   Foo bar;\n   bar.x;\n}\n"}]}}
-
-Content-Length: 148
-
-{"jsonrpc":"2.0","id":1,"method":"textDocument/definition","params":{"textDocument":{"uri":"test:///main.cpp"},"position":{"line":5,"character":7}}}
-# Go to field declaration, field reference 
-
-Content-Length: 220
-
-{"jsonrpc":"2.0","method":"textDocument/didChange","params":{"textDocument":{"uri":"test:///main.cpp","version":7},"contentChanges":[{"text":"struct
 Foo {\n  void x();\n};\nint main() {\n   Foo bar;\n   bar.x();\n}\n"}]}}
-
-Content-Length: 148
-
-{"jsonrpc":"2.0","id":1,"method":"textDocument/definition","params":{"textDocument":{"uri":"test:///main.cpp"},"position":{"line":5,"character":7}}}
-# Go to method declaration, method call 
-
-Content-Length: 240
-
-{"jsonrpc":"2.0","method":"textDocument/didChange","params":{"textDocument":{"uri":"test:///main.cpp","version":7},"contentChanges":[{"text":"struct
 Foo {\n};\ntypedef Foo TypedefFoo;\nint main() {\n   TypedefFoo bar;\n   
return 0;\n}\n"}]}}
-
-Conte

[clang-tools-extra] r324351 - [clangd] Don't try pthread, just use thread_local. Reverts r323949.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Feb  6 06:25:02 2018
New Revision: 324351

URL: http://llvm.org/viewvc/llvm-project?rev=324351&view=rev
Log:
[clangd] Don't try pthread, just use thread_local. Reverts r323949.

The pthread solution here breaks standalone builds, which don't have the
relevant cmake magic for feature-detection.

The original reason for trying pthread was fear of libgcc without
support for thread_local (e.g. on the clang-x86_64-linux-selfhost-modules bot).
However the earliest supported GCC is 4.8, and this has __cxa_thread_atexit.

This will probably break that bot, it's not running a supported GCC and needs
to be upgraded. I'll try to find out how to do this.

Modified:
clang-tools-extra/trunk/clangd/Context.cpp

Modified: clang-tools-extra/trunk/clangd/Context.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/Context.cpp?rev=324351&r1=324350&r2=324351&view=diff
==
--- clang-tools-extra/trunk/clangd/Context.cpp (original)
+++ clang-tools-extra/trunk/clangd/Context.cpp Tue Feb  6 06:25:02 2018
@@ -8,49 +8,8 @@
 //===-===//
 
 #include "Context.h"
-#include "llvm/Config/config.h"
 #include 
 
-// The thread-local Context is scoped in a function to avoid init-order issues.
-// It's created by currentContext() when first needed.
-
-#ifdef HAVE_PTHREAD_GETSPECIFIC
-// We'd love to use thread_local everywhere.
-// It requires support from the runtime: __cxa_thread_atexit.
-// Rather than detect this, we use the pthread API where available.
-#include 
-#include 
-static clang::clangd::Context ¤tContext() {
-  using clang::clangd::Context;
-  static pthread_key_t CtxKey;
-
-  // Once (across threads), set up pthread TLS for Context, and its destructor.
-  static int Dummy = [] { // Create key only once, for all threads.
-if (auto Err = pthread_key_create(&CtxKey, /*destructor=*/+[](void *Ctx) {
-  delete reinterpret_cast(Ctx);
-}))
-  llvm_unreachable(strerror(Err));
-return 0;
-  }();
-  (void)Dummy;
-
-  // Now grab the current context from TLS, and create it if it doesn't exist.
-  void *Ctx = pthread_getspecific(CtxKey);
-  if (!Ctx) {
-Ctx = new Context();
-if (auto Err = pthread_setspecific(CtxKey, Ctx))
-  llvm_unreachable(strerror(Err));
-  }
-  return *reinterpret_cast(Ctx);
-}
-#else
-// Only supported platform without pthread is windows, and thread_local works.
-static clang::clangd::Context ¤tContext() {
-  static thread_local auto C = clang::clangd::Context::empty();
-  return C;
-}
-#endif
-
 namespace clang {
 namespace clangd {
 
@@ -61,6 +20,11 @@ Context::Context(std::shared_ptrhttp://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r324735 - [clangd] Collect definitions when indexing.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Fri Feb  9 06:42:01 2018
New Revision: 324735

URL: http://llvm.org/viewvc/llvm-project?rev=324735&view=rev
Log:
[clangd] Collect definitions when indexing.

Within a TU:
 - as now, collect a declaration from the first occurrence of a symbol
   (taking clang's canonical declaration)
 - when we first see a definition occurrence, copy the symbol and add it
Across TUs/sources:
 - mergeSymbol in Merge.h is responsible for combining matching Symbols.
   This covers dynamic/static merges and cross-TU merges in the static index.
 - it prefers declarations from Symbols that have a definition.
 - GlobalSymbolBuilderMain is modified to use mergeSymbol as a reduce step.
Random cleanups (can be pulled out):
 - SymbolFromYAML -> SymbolsFromYAML, new singular SymbolFromYAML added
 - avoid uninit'd SymbolLocations. Add an idiomatic way to check "absent".
 - CanonicalDeclaration (as well as Definition) are mapped as optional in YAML.
 - added operator<< for Symbol & SymbolLocation, for debugging

Reviewers: ioeric, hokein

Subscribers: klimek, ilya-biryukov, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D42942

Modified:

clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
clang-tools-extra/trunk/clangd/index/Index.cpp
clang-tools-extra/trunk/clangd/index/Index.h
clang-tools-extra/trunk/clangd/index/Merge.cpp
clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp
clang-tools-extra/trunk/clangd/index/SymbolCollector.h
clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp
clang-tools-extra/trunk/clangd/index/SymbolYAML.h
clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp
clang-tools-extra/trunk/unittests/clangd/SymbolCollectorTests.cpp

Modified: 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp?rev=324735&r1=324734&r2=324735&view=diff
==
--- 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
 (original)
+++ 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
 Fri Feb  9 06:42:01 2018
@@ -14,9 +14,11 @@
 //===-===//
 
 #include "index/Index.h"
+#include "index/Merge.h"
 #include "index/SymbolCollector.h"
 #include "index/SymbolYAML.h"
 #include "clang/Frontend/FrontendActions.h"
+#include "clang/Frontend/CompilerInstance.h"
 #include "clang/Index/IndexDataConsumer.h"
 #include "clang/Index/IndexingAction.h"
 #include "clang/Tooling/CommonOptionsParser.h"
@@ -34,6 +36,7 @@ using clang::clangd::SymbolSlab;
 
 namespace clang {
 namespace clangd {
+namespace {
 
 static llvm::cl::opt AssumedHeaderDir(
 "assume-header-dir",
@@ -91,6 +94,25 @@ public:
   tooling::ExecutionContext *Ctx;
 };
 
+// Combine occurrences of the same symbol across translation units.
+SymbolSlab mergeSymbols(tooling::ToolResults *Results) {
+  SymbolSlab::Builder UniqueSymbols;
+  llvm::BumpPtrAllocator Arena;
+  Symbol::Details Scratch;
+  Results->forEachResult([&](llvm::StringRef Key, llvm::StringRef Value) {
+Arena.Reset();
+auto Sym = clang::clangd::SymbolFromYAML(Value, Arena);
+clang::clangd::SymbolID ID;
+Key >> ID;
+if (const auto *Existing = UniqueSymbols.find(ID))
+  UniqueSymbols.insert(mergeSymbol(*Existing, Sym, &Scratch));
+else
+  UniqueSymbols.insert(Sym);
+  });
+  return std::move(UniqueSymbols).build();
+}
+
+} // namespace
 } // namespace clangd
 } // namespace clang
 
@@ -115,6 +137,7 @@ int main(int argc, const char **argv) {
 return 1;
   }
 
+  // Map phase: emit symbols found in each translation unit.
   auto Err = Executor->get()->execute(
   llvm::make_unique(
   Executor->get()->getExecutionContext()));
@@ -122,22 +145,11 @@ int main(int argc, const char **argv) {
 llvm::errs() << llvm::toString(std::move(Err)) << "\n";
   }
 
-  // Deduplicate the result by key and keep the longest value.
-  // FIXME(ioeric): Merge occurrences, rather than just dropping all but one.
-  // Definitions and forward declarations have the same key and may both have
-  // information. Usage count will need to be aggregated across occurrences,
-  // too.
-  llvm::StringMap UniqueSymbols;
-  Executor->get()->getToolResults()->forEachResult(
-  [&UniqueSymbols](llvm::StringRef Key, llvm::StringRef Value) {
-auto Ret = UniqueSymbols.try_emplace(Key, Value);
-if (!Ret.second) {
-  // If key already exists, keep the longest value.
-  llvm::StringRef &V = Ret.first->second;
-  V = V.size() < Value.size() ? Value : V;
-}
-  });
-  for (const auto &Sym : UniqueSymbols)
-llvm::outs() << Sym.second;
+  // Reduc

[clang-tools-extra] r324990 - [clangd] Stop exposing Futures from ClangdServer operations.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Feb 13 00:59:23 2018
New Revision: 324990

URL: http://llvm.org/viewvc/llvm-project?rev=324990&view=rev
Log:
[clangd] Stop exposing Futures from ClangdServer operations.

Summary:
LSP has asynchronous semantics, being able to block on an async operation
completing is unneccesary and leads to tighter coupling with the threading.

In practice only tests depend on this, so we add a general-purpose "block until
idle" function to the scheduler which will work for all operations.

To get this working, fix a latent condition-variable bug in ASTWorker, and make
AsyncTaskRunner const-correct.

Reviewers: ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D43127

Modified:
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.h
clang-tools-extra/trunk/clangd/TUScheduler.cpp
clang-tools-extra/trunk/clangd/TUScheduler.h
clang-tools-extra/trunk/clangd/Threading.cpp
clang-tools-extra/trunk/clangd/Threading.h
clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp
clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp
clang-tools-extra/trunk/unittests/clangd/ThreadingTests.cpp

Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=324990&r1=324989&r2=324990&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Tue Feb 13 00:59:23 2018
@@ -139,11 +139,11 @@ void ClangdServer::setRootPath(PathRef R
 this->RootPath = NewRootPath;
 }
 
-std::future ClangdServer::addDocument(PathRef File, StringRef Contents) {
+void ClangdServer::addDocument(PathRef File, StringRef Contents) {
   DocVersion Version = DraftMgr.updateDraft(File, Contents);
   auto TaggedFS = FSProvider.getTaggedFileSystem(File);
-  return scheduleReparseAndDiags(File, VersionedDraft{Version, Contents.str()},
- std::move(TaggedFS));
+  scheduleReparseAndDiags(File, VersionedDraft{Version, Contents.str()},
+  std::move(TaggedFS));
 }
 
 void ClangdServer::removeDocument(PathRef File) {
@@ -152,7 +152,7 @@ void ClangdServer::removeDocument(PathRe
   WorkScheduler.remove(File);
 }
 
-std::future ClangdServer::forceReparse(PathRef File) {
+void ClangdServer::forceReparse(PathRef File) {
   auto FileContents = DraftMgr.getDraft(File);
   assert(FileContents.Draft &&
  "forceReparse() was called for non-added document");
@@ -162,8 +162,7 @@ std::future ClangdServer::forceRep
   CompileArgs.invalidate(File);
 
   auto TaggedFS = FSProvider.getTaggedFileSystem(File);
-  return scheduleReparseAndDiags(File, std::move(FileContents),
- std::move(TaggedFS));
+  scheduleReparseAndDiags(File, std::move(FileContents), std::move(TaggedFS));
 }
 
 void ClangdServer::codeComplete(
@@ -463,7 +462,7 @@ ClangdServer::findDocumentHighlights(Pat
   return blockingRunWithAST(WorkScheduler, File, Action);
 }
 
-std::future ClangdServer::scheduleReparseAndDiags(
+void ClangdServer::scheduleReparseAndDiags(
 PathRef File, VersionedDraft Contents,
 Tagged> TaggedFS) {
   tooling::CompileCommand Command = CompileArgs.getCompileCommand(File);
@@ -474,12 +473,7 @@ std::future ClangdServer::schedule
   Path FileStr = File.str();
   VFSTag Tag = std::move(TaggedFS.Tag);
 
-  std::promise DonePromise;
-  std::future DoneFuture = DonePromise.get_future();
-
-  auto Callback = [this, Version, FileStr, Tag](std::promise DonePromise,
-OptDiags Diags) {
-auto Guard = llvm::make_scope_exit([&]() { DonePromise.set_value(); });
+  auto Callback = [this, Version, FileStr, Tag](OptDiags Diags) {
 if (!Diags)
   return; // A new reparse was requested before this one completed.
 
@@ -503,8 +497,7 @@ std::future ClangdServer::schedule
ParseInputs{std::move(Command),
std::move(TaggedFS.Value),
std::move(*Contents.Draft)},
-   BindWithForward(Callback, std::move(DonePromise)));
-  return DoneFuture;
+   std::move(Callback));
 }
 
 void ClangdServer::onFileEvent(const DidChangeWatchedFilesParams &Params) {
@@ -516,3 +509,8 @@ std::vector
 ClangdServer::getUsedBytesPerFile() const {
   return WorkScheduler.getUsedBytesPerFile();
 }
+
+LLVM_NODISCARD bool
+ClangdServer::blockUntilIdleForTest(llvm::Optional TimeoutSeconds) {
+  return WorkScheduler.blockUntilIdle(timeoutSeconds(TimeoutSeconds));
+}

Modified: clang-tools-extra/trunk/clangd/ClangdServer.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.h?rev=324990&r1=324989&r2=324990&view=diff
===

[clang-tools-extra] r325097 - [clangd] Configure clangd tracing with CLANGD_TRACE env instead of -trace flag

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Feb 13 19:20:07 2018
New Revision: 325097

URL: http://llvm.org/viewvc/llvm-project?rev=325097&view=rev
Log:
[clangd] Configure clangd tracing with CLANGD_TRACE env instead of -trace flag

Modified:
clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
clang-tools-extra/trunk/test/clangd/trace.test

Modified: clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp?rev=325097&r1=325096&r2=325097&view=diff
==
--- clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp (original)
+++ clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp Tue Feb 13 19:20:07 2018
@@ -17,6 +17,7 @@
 #include "llvm/Support/Path.h"
 #include "llvm/Support/Program.h"
 #include "llvm/Support/raw_ostream.h"
+#include 
 #include 
 #include 
 #include 
@@ -123,12 +124,6 @@ static llvm::cl::opt InputMirrorFi
 "Mirror all LSP input to the specified file. Useful for debugging."),
 llvm::cl::init(""), llvm::cl::Hidden);
 
-static llvm::cl::opt TraceFile(
-"trace",
-llvm::cl::desc(
-"Trace internal events and timestamps in chrome://tracing JSON 
format"),
-llvm::cl::init(""), llvm::cl::Hidden);
-
 static llvm::cl::opt EnableIndexBasedCompletion(
 "enable-index-based-completion",
 llvm::cl::desc(
@@ -176,15 +171,18 @@ int main(int argc, char *argv[]) {
 }
   }
 
-  // Setup tracing facilities.
+  // Setup tracing facilities if CLANGD_TRACE is set. In practice enabling a
+  // trace flag in your editor's config is annoying, launching with
+  // `CLANGD_TRACE=trace.json vim` is easier.
   llvm::Optional TraceStream;
   std::unique_ptr Tracer;
-  if (!TraceFile.empty()) {
+  if (auto *TraceFile = getenv("CLANGD_TRACE")) {
 std::error_code EC;
 TraceStream.emplace(TraceFile, /*ref*/ EC, llvm::sys::fs::F_RW);
 if (EC) {
-  TraceFile.reset();
-  llvm::errs() << "Error while opening trace file: " << EC.message();
+  TraceStream.reset();
+  llvm::errs() << "Error while opening trace file " << TraceFile << ": "
+   << EC.message();
 } else {
   Tracer = trace::createJSONTracer(*TraceStream, PrettyPrint);
 }

Modified: clang-tools-extra/trunk/test/clangd/trace.test
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/clangd/trace.test?rev=325097&r1=325096&r2=325097&view=diff
==
--- clang-tools-extra/trunk/test/clangd/trace.test (original)
+++ clang-tools-extra/trunk/test/clangd/trace.test Tue Feb 13 19:20:07 2018
@@ -1,4 +1,4 @@
-# RUN: clangd -lit-test -trace %t < %s && FileCheck %s < %t
+# RUN: CLANGD_TRACE=%t clangd -lit-test < %s && FileCheck %s < %t
 
{"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":123,"rootPath":"clangd","capabilities":{},"trace":"off"}}
 ---
 
{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"test:///foo.c","languageId":"c","version":1,"text":"void
 main() {}"}}}


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [clang-tools-extra] r325097 - [clangd] Configure clangd tracing with CLANGD_TRACE env instead of -trace flag

[Sam McCall via cfe-commits]

On Wed, Feb 14, 2018 at 10:45 AM Ilya Biryukov  wrote:

> Personally, I'm not a big fan of environment variables. There are harder
> to discover than command-line flags and I still have to change editor
> config often to switch between different builds of clangd.
> I know it may sound weird, but maybe we could have both the flag and the
> environment variables? (flag taking the precedence if both are specified)
>
Do you use tracing often? Can you describe your workflow?

(Less concerned about discoverability - this was always a hidden flag
anyway)


> On Wed, Feb 14, 2018 at 4:22 AM Sam McCall via cfe-commits <
> cfe-commits@lists.llvm.org> wrote:
>
>> Author: sammccall
>> Date: Tue Feb 13 19:20:07 2018
>> New Revision: 325097
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=325097&view=rev
>> Log:
>> [clangd] Configure clangd tracing with CLANGD_TRACE env instead of -trace
>> flag
>>
>> Modified:
>> clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
>> clang-tools-extra/trunk/test/clangd/trace.test
>>
>> Modified: clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp?rev=325097&r1=325096&r2=325097&view=diff
>>
>> ==
>> --- clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp (original)
>> +++ clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp Tue Feb 13
>> 19:20:07 2018
>> @@ -17,6 +17,7 @@
>>  #include "llvm/Support/Path.h"
>>  #include "llvm/Support/Program.h"
>>  #include "llvm/Support/raw_ostream.h"
>> +#include 
>>  #include 
>>  #include 
>>  #include 
>> @@ -123,12 +124,6 @@ static llvm::cl::opt InputMirrorFi
>>  "Mirror all LSP input to the specified file. Useful for
>> debugging."),
>>  llvm::cl::init(""), llvm::cl::Hidden);
>>
>> -static llvm::cl::opt TraceFile(
>> -"trace",
>> -llvm::cl::desc(
>> -"Trace internal events and timestamps in chrome://tracing JSON
>> format"),
>> -llvm::cl::init(""), llvm::cl::Hidden);
>> -
>>  static llvm::cl::opt EnableIndexBasedCompletion(
>>  "enable-index-based-completion",
>>  llvm::cl::desc(
>> @@ -176,15 +171,18 @@ int main(int argc, char *argv[]) {
>>  }
>>}
>>
>> -  // Setup tracing facilities.
>> +  // Setup tracing facilities if CLANGD_TRACE is set. In practice
>> enabling a
>> +  // trace flag in your editor's config is annoying, launching with
>> +  // `CLANGD_TRACE=trace.json vim` is easier.
>>llvm::Optional TraceStream;
>>std::unique_ptr Tracer;
>> -  if (!TraceFile.empty()) {
>> +  if (auto *TraceFile = getenv("CLANGD_TRACE")) {
>>  std::error_code EC;
>>  TraceStream.emplace(TraceFile, /*ref*/ EC, llvm::sys::fs::F_RW);
>>  if (EC) {
>> -  TraceFile.reset();
>> -  llvm::errs() << "Error while opening trace file: " << EC.message();
>> +  TraceStream.reset();
>> +  llvm::errs() << "Error while opening trace file " << TraceFile <<
>> ": "
>> +   << EC.message();
>>  } else {
>>Tracer = trace::createJSONTracer(*TraceStream, PrettyPrint);
>>  }
>>
>> Modified: clang-tools-extra/trunk/test/clangd/trace.test
>> URL:
>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/clangd/trace.test?rev=325097&r1=325096&r2=325097&view=diff
>>
>> ==
>> --- clang-tools-extra/trunk/test/clangd/trace.test (original)
>> +++ clang-tools-extra/trunk/test/clangd/trace.test Tue Feb 13 19:20:07
>> 2018
>> @@ -1,4 +1,4 @@
>> -# RUN: clangd -lit-test -trace %t < %s && FileCheck %s < %t
>> +# RUN: CLANGD_TRACE=%t clangd -lit-test < %s && FileCheck %s < %t
>>
>>  
>> {"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":123,"rootPath":"clangd","capabilities":{},"trace":"off"}}
>>  ---
>>  
>> {"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"test:///foo.c","languageId":"c","version":1,"text":"void
>> main() {}"}}}
>>
>>
>> ___
>> cfe-commits mailing list
>> cfe-commits@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
>>
>
>
> --
> Regards,
> Ilya Biryukov
>
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r325220 - [clangd] Fix tracing now that spans lifetimes can overlap on a thread.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Thu Feb 15 00:40:54 2018
New Revision: 325220

URL: http://llvm.org/viewvc/llvm-project?rev=325220&view=rev
Log:
[clangd] Fix tracing now that spans lifetimes can overlap on a thread.

Summary:
The chrome trace viewer requires events within a thread to strictly nest.
So we need to record the lifetime of the Span objects, not the contexts.

But we still want to show the relationship between spans where a context crosses
threads, so do this with flow events (i.e. arrows).

Before: https://photos.app.goo.gl/q4Dd9u9xtelaXk1v1
After: https://photos.app.goo.gl/5RNLmAMLZR3unvY83

(This could stand some further improvement, in particular I think we want a
container span whenever we schedule work on a thread. But that's another patch)

Reviewers: ioeric

Subscribers: klimek, ilya-biryukov, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D43272

Modified:
clang-tools-extra/trunk/clangd/Trace.cpp
clang-tools-extra/trunk/clangd/Trace.h
clang-tools-extra/trunk/test/clangd/trace.test
clang-tools-extra/trunk/unittests/clangd/TraceTests.cpp

Modified: clang-tools-extra/trunk/clangd/Trace.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/Trace.cpp?rev=325220&r1=325219&r2=325220&view=diff
==
--- clang-tools-extra/trunk/clangd/Trace.cpp (original)
+++ clang-tools-extra/trunk/clangd/Trace.cpp Thu Feb 15 00:40:54 2018
@@ -16,6 +16,7 @@
 #include "llvm/Support/FormatProviders.h"
 #include "llvm/Support/FormatVariadic.h"
 #include "llvm/Support/Threading.h"
+#include 
 #include 
 
 namespace clang {
@@ -46,40 +47,103 @@ public:
 Out.flush();
   }
 
+  // We stash a Span object in the context. It will record the start/end,
+  // and this also allows us to look up the parent Span's information.
   Context beginSpan(llvm::StringRef Name, json::obj *Args) override {
-jsonEvent("B", json::obj{{"name", Name}});
-return Context::current().derive(make_scope_exit([this, Args] {
-  jsonEvent("E", json::obj{{"args", std::move(*Args)}});
-}));
+return Context::current().derive(SpanKey,
+ make_unique(this, Name, Args));
+  }
+
+  // Trace viewer requires each thread to properly stack events.
+  // So we need to mark only duration that the span was active on the thread.
+  // (Hopefully any off-thread activity will be connected by a flow event).
+  // Record the end time here, but don't write the event: Args aren't ready 
yet.
+  void endSpan() override {
+Context::current().getExisting(SpanKey)->markEnded();
   }
 
   void instant(llvm::StringRef Name, json::obj &&Args) override {
+captureThreadMetadata();
 jsonEvent("i", json::obj{{"name", Name}, {"args", std::move(Args)}});
   }
 
   // Record an event on the current thread. ph, pid, tid, ts are set.
   // Contents must be a list of the other JSON key/values.
-  void jsonEvent(StringRef Phase, json::obj &&Contents) {
-uint64_t TID = get_threadid();
-std::lock_guard Lock(Mu);
-// If we haven't already, emit metadata describing this thread.
-if (ThreadsWithMD.insert(TID).second) {
-  SmallString<32> Name;
-  get_thread_name(Name);
-  if (!Name.empty()) {
-rawEvent("M", json::obj{
-  {"tid", TID},
-  {"name", "thread_name"},
-  {"args", json::obj{{"name", Name}}},
-  });
-  }
-}
-Contents["ts"] = timestamp();
+  void jsonEvent(StringRef Phase, json::obj &&Contents,
+ uint64_t TID = get_threadid(),
+ double Timestamp = 0) {
+Contents["ts"] = Timestamp ? Timestamp : timestamp();
 Contents["tid"] = TID;
+std::lock_guard Lock(Mu);
 rawEvent(Phase, std::move(Contents));
   }
 
 private:
+  class JSONSpan {
+  public:
+JSONSpan(JSONTracer *Tracer, llvm::StringRef Name, json::obj *Args)
+: StartTime(Tracer->timestamp()), EndTime(0), Name(Name),
+  TID(get_threadid()), Tracer(Tracer), Args(Args) {
+  // ~JSONSpan() may run in a different thread, so we need to capture now.
+  Tracer->captureThreadMetadata();
+
+  // We don't record begin events here (and end events in the destructor)
+  // because B/E pairs have to appear in the right order, which is awkward.
+  // Instead we send the complete (X) event in the destructor.
+
+  // If our parent was on a different thread, add an arrow to this span.
+  auto *Parent = Context::current().get(SpanKey);
+  if (Parent && *Parent && (*Parent)->TID != TID) {
+// If the parent span ended already, then show this as "following" it.
+// Otherwise show us as "parallel".
+double OriginTime = (*Parent)->EndTime;
+if (!OriginTime)
+  OriginTime = (*Parent)->StartTime;
+
+auto FlowID = nextID();
+Tracer->jsonEvent("s",
+ 

[clang-tools-extra] r325239 - [clangd] Fix make_unique ambiguity, NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Thu Feb 15 06:16:17 2018
New Revision: 325239

URL: http://llvm.org/viewvc/llvm-project?rev=325239&view=rev
Log:
[clangd] Fix make_unique ambiguity, NFC

Modified:
clang-tools-extra/trunk/clangd/Trace.cpp

Modified: clang-tools-extra/trunk/clangd/Trace.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/Trace.cpp?rev=325239&r1=325238&r2=325239&view=diff
==
--- clang-tools-extra/trunk/clangd/Trace.cpp (original)
+++ clang-tools-extra/trunk/clangd/Trace.cpp Thu Feb 15 06:16:17 2018
@@ -50,8 +50,8 @@ public:
   // We stash a Span object in the context. It will record the start/end,
   // and this also allows us to look up the parent Span's information.
   Context beginSpan(llvm::StringRef Name, json::obj *Args) override {
-return Context::current().derive(SpanKey,
- make_unique(this, Name, Args));
+return Context::current().derive(
+SpanKey, llvm::make_unique(this, Name, Args));
   }
 
   // Trace viewer requires each thread to properly stack events.


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r325257 - [clangd] Use explicit env in test to make lit+windows happy. NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Thu Feb 15 08:20:33 2018
New Revision: 325257

URL: http://llvm.org/viewvc/llvm-project?rev=325257&view=rev
Log:
[clangd] Use explicit env in test to make lit+windows happy. NFC

Modified:
clang-tools-extra/trunk/test/clangd/trace.test

Modified: clang-tools-extra/trunk/test/clangd/trace.test
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/clangd/trace.test?rev=325257&r1=325256&r2=325257&view=diff
==
--- clang-tools-extra/trunk/test/clangd/trace.test (original)
+++ clang-tools-extra/trunk/test/clangd/trace.test Thu Feb 15 08:20:33 2018
@@ -1,4 +1,4 @@
-# RUN: CLANGD_TRACE=%t clangd -lit-test < %s && FileCheck %s < %t
+# RUN: env CLANGD_TRACE=%t clangd -lit-test < %s && FileCheck %s < %t
 
{"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":123,"rootPath":"clangd","capabilities":{},"trace":"off"}}
 ---
 
{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"test:///foo.c","languageId":"c","version":1,"text":"void
 main() {}"}}}


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r325326 - [clangd] TestFS cleanup: getVirtualBlahBlah -> testPath/testRoot. Remove SmallString micro-opt for more ergonomic tests. NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Fri Feb 16 01:41:43 2018
New Revision: 325326

URL: http://llvm.org/viewvc/llvm-project?rev=325326&view=rev
Log:
[clangd] TestFS cleanup: getVirtualBlahBlah -> testPath/testRoot. Remove 
SmallString micro-opt for more ergonomic tests. NFC

Modified:
clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp
clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp
clang-tools-extra/trunk/unittests/clangd/SymbolCollectorTests.cpp
clang-tools-extra/trunk/unittests/clangd/TUSchedulerTests.cpp
clang-tools-extra/trunk/unittests/clangd/TestFS.cpp
clang-tools-extra/trunk/unittests/clangd/TestFS.h
clang-tools-extra/trunk/unittests/clangd/URITests.cpp
clang-tools-extra/trunk/unittests/clangd/XRefsTests.cpp

Modified: clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp?rev=325326&r1=325325&r2=325326&view=diff
==
--- clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp (original)
+++ clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp Fri Feb 16 
01:41:43 2018
@@ -114,13 +114,10 @@ protected:
 ClangdServer Server(CDB, DiagConsumer, FS, getDefaultAsyncThreadsCount(),
 /*StorePreamblesInMemory=*/true);
 for (const auto &FileWithContents : ExtraFiles)
-  FS.Files[getVirtualTestFilePath(FileWithContents.first)] =
-  FileWithContents.second;
-
-auto SourceFilename = getVirtualTestFilePath(SourceFileRelPath);
+  FS.Files[testPath(FileWithContents.first)] = FileWithContents.second;
 
+auto SourceFilename = testPath(SourceFileRelPath);
 FS.ExpectedFile = SourceFilename;
-
 Server.addDocument(SourceFilename, SourceContents);
 auto Result = dumpASTWithoutMemoryLocs(Server, SourceFilename);
 EXPECT_TRUE(Server.blockUntilIdleForTest()) << "Waiting for diagnostics";
@@ -176,10 +173,9 @@ TEST_F(ClangdVFSTest, Reparse) {
 int b = a;
 )cpp";
 
-  auto FooCpp = getVirtualTestFilePath("foo.cpp");
-  auto FooH = getVirtualTestFilePath("foo.h");
+  auto FooCpp = testPath("foo.cpp");
 
-  FS.Files[FooH] = "int a;";
+  FS.Files[testPath("foo.h")] = "int a;";
   FS.Files[FooCpp] = SourceContents;
   FS.ExpectedFile = FooCpp;
 
@@ -215,8 +211,8 @@ TEST_F(ClangdVFSTest, ReparseOnHeaderCha
 int b = a;
 )cpp";
 
-  auto FooCpp = getVirtualTestFilePath("foo.cpp");
-  auto FooH = getVirtualTestFilePath("foo.h");
+  auto FooCpp = testPath("foo.cpp");
+  auto FooH = testPath("foo.h");
 
   FS.Files[FooH] = "int a;";
   FS.Files[FooCpp] = SourceContents;
@@ -252,7 +248,7 @@ TEST_F(ClangdVFSTest, CheckVersions) {
   /*AsyncThreadsCount=*/0,
   /*StorePreamblesInMemory=*/true);
 
-  auto FooCpp = getVirtualTestFilePath("foo.cpp");
+  auto FooCpp = testPath("foo.cpp");
   const auto SourceContents = "int a;";
   FS.Files[FooCpp] = SourceContents;
   FS.ExpectedFile = FooCpp;
@@ -309,7 +305,7 @@ TEST_F(ClangdVFSTest, SearchLibDir) {
   llvm::sys::path::append(StringPath, IncludeDir, "string");
   FS.Files[StringPath] = "class mock_string {};";
 
-  auto FooCpp = getVirtualTestFilePath("foo.cpp");
+  auto FooCpp = testPath("foo.cpp");
   const auto SourceContents = R"cpp(
 #include 
 mock_string x;
@@ -340,7 +336,7 @@ TEST_F(ClangdVFSTest, ForceReparseCompil
 
   // No need to sync reparses, because reparses are performed on the calling
   // thread.
-  auto FooCpp = getVirtualTestFilePath("foo.cpp");
+  auto FooCpp = testPath("foo.cpp");
   const auto SourceContents1 = R"cpp(
 template 
 struct foo { T x; };
@@ -388,13 +384,13 @@ TEST_F(ClangdVFSTest, MemoryUsage) {
 
   // No need to sync reparses, because reparses are performed on the calling
   // thread.
-  Path FooCpp = getVirtualTestFilePath("foo.cpp").str();
+  Path FooCpp = testPath("foo.cpp");
   const auto SourceContents = R"cpp(
 struct Something {
   int method();
 };
 )cpp";
-  Path BarCpp = getVirtualTestFilePath("bar.cpp").str();
+  Path BarCpp = testPath("bar.cpp");
 
   FS.Files[FooCpp] = "";
   FS.Files[BarCpp] = "";
@@ -423,11 +419,11 @@ TEST_F(ClangdVFSTest, InvalidCompileComm
   /*AsyncThreadsCount=*/0,
   /*StorePreamblesInMemory=*/true);
 
-  auto FooCpp = getVirtualTestFilePath("foo.cpp");
+  auto FooCpp = testPath("foo.cpp");
   // clang cannot create CompilerInvocation if we pass two files in the
   // CompileCommand. We pass the file in ExtraFlags once and CDB adds another
   // one in getCompileCommand().
-  CDB.ExtraClangFlags.push_back(FooCpp.str());
+  CDB.ExtraClangFlags.push_back(FooCpp);
 
   // Clang can't parse command args in that case, but we shouldn't crash.
   Server.addDocument(FooCpp, "int main() {}");
@@ -478,16 +474,13 @@ int d;
   unsigned MaxLineForFileRequests = 7;
   unsigned MaxColumnForFileRequests = 10;
 
-  std::vector> FilePaths;
-  FilePaths.reserve(FilesCo

[clang-tools-extra] r325357 - [clangd] Include timestamps in log messages.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Fri Feb 16 08:41:42 2018
New Revision: 325357

URL: http://llvm.org/viewvc/llvm-project?rev=325357&view=rev
Log:
[clangd] Include timestamps in log messages.

Modified:
clang-tools-extra/trunk/clangd/JSONRPCDispatcher.cpp

Modified: clang-tools-extra/trunk/clangd/JSONRPCDispatcher.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/JSONRPCDispatcher.cpp?rev=325357&r1=325356&r2=325357&view=diff
==
--- clang-tools-extra/trunk/clangd/JSONRPCDispatcher.cpp (original)
+++ clang-tools-extra/trunk/clangd/JSONRPCDispatcher.cpp Fri Feb 16 08:41:42 
2018
@@ -13,6 +13,7 @@
 #include "Trace.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringExtras.h"
+#include "llvm/Support/Chrono.h"
 #include "llvm/Support/SourceMgr.h"
 #include 
 
@@ -60,20 +61,19 @@ void JSONOutput::writeMessage(const json
 OS << Message;
   OS.flush();
 
-  std::lock_guard Guard(StreamMutex);
-  // Log without headers.
-  Logs << "--> " << S << '\n';
-  Logs.flush();
-
-  // Emit message with header.
-  Outs << "Content-Length: " << S.size() << "\r\n\r\n" << S;
-  Outs.flush();
+  {
+std::lock_guard Guard(StreamMutex);
+Outs << "Content-Length: " << S.size() << "\r\n\r\n" << S;
+Outs.flush();
+  }
+  log(llvm::Twine("--> ") + S);
 }
 
 void JSONOutput::log(const Twine &Message) {
+  llvm::sys::TimePoint<> Timestamp = std::chrono::system_clock::now();
   trace::log(Message);
   std::lock_guard Guard(StreamMutex);
-  Logs << Message << '\n';
+  Logs << llvm::formatv("[{0:%H:%M:%S.%L}] {1}\n", Timestamp, Message);
   Logs.flush();
 }
 


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r325476 - [clangd] Fix use-after-free in SymbolYAML: strings are owned by yaml::Input!

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Feb 19 01:31:26 2018
New Revision: 325476

URL: http://llvm.org/viewvc/llvm-project?rev=325476&view=rev
Log:
[clangd] Fix use-after-free in SymbolYAML: strings are owned by yaml::Input!

Summary:
There are a few implementation options here - alternatives are either both
awkward and inefficient, or really inefficient.
This is at least potentially a hot path when used as a reducer for common
symbols.

(Also fix an unused-var that sneaked in)

Reviewers: ioeric

Subscribers: klimek, ilya-biryukov, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D43381

Modified:
clang-tools-extra/trunk/clangd/XRefs.cpp

clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp
clang-tools-extra/trunk/clangd/index/SymbolYAML.h

Modified: clang-tools-extra/trunk/clangd/XRefs.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/XRefs.cpp?rev=325476&r1=325475&r2=325476&view=diff
==
--- clang-tools-extra/trunk/clangd/XRefs.cpp (original)
+++ clang-tools-extra/trunk/clangd/XRefs.cpp Mon Feb 19 01:31:26 2018
@@ -360,9 +360,8 @@ static std::string NamedDeclQualifiedNam
 static llvm::Optional getScopeName(const Decl *D) {
   const DeclContext *DC = D->getDeclContext();
 
-  if (const TranslationUnitDecl *TUD = dyn_cast(DC))
+  if (isa(DC))
 return std::string("global namespace");
-
   if (const TypeDecl *TD = dyn_cast(DC))
 return TypeDeclToString(TD);
   else if (const NamespaceDecl *ND = dyn_cast(DC))

Modified: 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp?rev=325476&r1=325475&r2=325476&view=diff
==
--- 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
 (original)
+++ 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
 Mon Feb 19 01:31:26 2018
@@ -20,7 +20,6 @@
 #include "index/SymbolYAML.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/FrontendActions.h"
-#include "clang/Frontend/CompilerInstance.h"
 #include "clang/Index/IndexDataConsumer.h"
 #include "clang/Index/IndexingAction.h"
 #include "clang/Tooling/CommonOptionsParser.h"
@@ -31,6 +30,7 @@
 #include "llvm/Support/Path.h"
 #include "llvm/Support/Signals.h"
 #include "llvm/Support/ThreadPool.h"
+#include "llvm/Support/YAMLTraits.h"
 
 using namespace llvm;
 using namespace clang::tooling;
@@ -117,7 +117,8 @@ SymbolSlab mergeSymbols(tooling::ToolRes
   Symbol::Details Scratch;
   Results->forEachResult([&](llvm::StringRef Key, llvm::StringRef Value) {
 Arena.Reset();
-auto Sym = clang::clangd::SymbolFromYAML(Value, Arena);
+llvm::yaml::Input Yin(Value, &Arena);
+auto Sym = clang::clangd::SymbolFromYAML(Yin, Arena);
 clang::clangd::SymbolID ID;
 Key >> ID;
 if (const auto *Existing = UniqueSymbols.find(ID))

Modified: clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp?rev=325476&r1=325475&r2=325476&view=diff
==
--- clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp Mon Feb 19 01:31:26 2018
@@ -12,7 +12,6 @@
 #include "llvm/ADT/Optional.h"
 #include "llvm/Support/Errc.h"
 #include "llvm/Support/MemoryBuffer.h"
-#include "llvm/Support/YAMLTraits.h"
 #include "llvm/Support/raw_ostream.h"
 
 LLVM_YAML_IS_DOCUMENT_LIST_VECTOR(clang::clangd::Symbol)
@@ -176,11 +175,11 @@ SymbolSlab SymbolsFromYAML(llvm::StringR
   return std::move(Syms).build();
 }
 
-Symbol SymbolFromYAML(llvm::StringRef YAMLContent,
-  llvm::BumpPtrAllocator &Arena) {
-  llvm::yaml::Input Yin(YAMLContent, &Arena);
+Symbol SymbolFromYAML(llvm::yaml::Input &Input, llvm::BumpPtrAllocator &Arena) 
{
+  // We could grab Arena out of Input, but it'd be a huge hazard for callers.
+  assert(Input.getContext() == &Arena);
   Symbol S;
-  Yin >> S;
+  Input >> S;
   return S;
 }
 

Modified: clang-tools-extra/trunk/clangd/index/SymbolYAML.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/SymbolYAML.h?rev=325476&r1=325475&r2=325476&view=diff
==
--- clang-tools-extra/trunk/clangd/index/SymbolYAML.h (original)
+++ clang-tools-extra/trunk/clangd/index/SymbolYAML.h Mon Feb 19 01:31:26 2018
@@ -20,6 +20,7 @@
 
 #include "Index.h"
 #include "llvm/Support/Error.h"
+#include "llvm/Support/YAMLTraits.h"
 #include "llvm/Support/raw_ostream.h"
 
 namespace clang {
@@ -28,9 +29,10 @

[clang-tools-extra] r325477 - [clangd] Add "clangd.trace" VSCode setting to enable tracing.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Feb 19 01:43:46 2018
New Revision: 325477

URL: http://llvm.org/viewvc/llvm-project?rev=325477&view=rev
Log:
[clangd] Add "clangd.trace" VSCode setting to enable tracing.

Summary:
Setting the CLANGD_TRACE environment variable directly is awkward with VSCode's
"reload from the command palette" workflow.

Reviewers: ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D43385

Modified:
clang-tools-extra/trunk/clangd/clients/clangd-vscode/package.json
clang-tools-extra/trunk/clangd/clients/clangd-vscode/src/extension.ts

Modified: clang-tools-extra/trunk/clangd/clients/clangd-vscode/package.json
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/clients/clangd-vscode/package.json?rev=325477&r1=325476&r2=325477&view=diff
==
--- clang-tools-extra/trunk/clangd/clients/clangd-vscode/package.json (original)
+++ clang-tools-extra/trunk/clangd/clients/clangd-vscode/package.json Mon Feb 
19 01:43:46 2018
@@ -43,8 +43,8 @@
 "@types/mocha": "^2.2.32"
 },
 "repository": {
-  "type": "svn",
-  "url": 
"http://llvm.org/svn/llvm-project/clang-tools-extra/trunk/clangd/clients/clangd-vscode/";
+"type": "svn",
+"url": 
"http://llvm.org/svn/llvm-project/clang-tools-extra/trunk/clangd/clients/clangd-vscode/";
 },
 "contributes": {
 "configuration": {
@@ -68,6 +68,10 @@
 "type": "boolean",
 "default": true,
 "description": "Whether or not to send file events to 
clangd (File created, changed or deleted). This can be disabled for performance 
consideration."
+},
+"clangd.trace": {
+"type": "string",
+"description": "Names a file that clangd should log a 
performance trace to, in chrome trace-viewer JSON format."
 }
 }
 }

Modified: clang-tools-extra/trunk/clangd/clients/clangd-vscode/src/extension.ts
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/clients/clangd-vscode/src/extension.ts?rev=325477&r1=325476&r2=325477&view=diff
==
--- clang-tools-extra/trunk/clangd/clients/clangd-vscode/src/extension.ts 
(original)
+++ clang-tools-extra/trunk/clangd/clients/clangd-vscode/src/extension.ts Mon 
Feb 19 01:43:46 2018
@@ -16,11 +16,16 @@ function getConfig(option: string, de
  *  your extension is activated the very first time the command is executed
  */
 export function activate(context: vscode.ExtensionContext) {
-const clangdPath = getConfig('path');
-const clangdArgs = getConfig('arguments');
 const syncFileEvents = getConfig('syncFileEvents', true);
 
-const serverOptions: vscodelc.ServerOptions = { command: clangdPath, args: 
clangdArgs };
+const clangd: vscodelc.Executable = {
+command: getConfig('path'),
+args: getConfig('arguments')
+};
+const traceFile = getConfig('trace');
+if (traceFile != null)
+clangd.options = {env: {CLANGD_TRACE: traceFile}};
+const serverOptions: vscodelc.ServerOptions = clangd;
 
 const filePattern: string = '**/*.{' +
   ['cpp', 'c', 'cc', 'cxx', 'c++', 'm', 'mm', 'h', 'hh', 'hpp', 'hxx', 
'inc'].join() + '}';


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r325480 - [clangd] Tracing: name worker threads, and enforce naming scheduled async tasks

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Feb 19 01:56:28 2018
New Revision: 325480

URL: http://llvm.org/viewvc/llvm-project?rev=325480&view=rev
Log:
[clangd] Tracing: name worker threads, and enforce naming scheduled async tasks

Summary:
This has a bit of a blast radius, but I think there's enough value in "forcing"
us to give names to these async tasks for debugging. Guessing about what
multithreaded code is doing is so unfun...

The "file" param attached to the tasks may seem to be redundant with the thread
names, but note that thread names are truncated to 15 chars on linux!
We'll be lucky to get the whole basename...

Reviewers: ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D43388

Modified:
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/TUScheduler.cpp
clang-tools-extra/trunk/clangd/TUScheduler.h
clang-tools-extra/trunk/clangd/Threading.cpp
clang-tools-extra/trunk/clangd/Threading.h
clang-tools-extra/trunk/clangd/Trace.cpp
clang-tools-extra/trunk/clangd/Trace.h
clang-tools-extra/trunk/unittests/clangd/TUSchedulerTests.cpp
clang-tools-extra/trunk/unittests/clangd/ThreadingTests.cpp

Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=325480&r1=325479&r2=325480&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Mon Feb 19 01:56:28 2018
@@ -179,9 +179,10 @@ void ClangdServer::codeComplete(
 Callback(make_tagged(std::move(Result), std::move(TaggedFS.Tag)));
   };
 
-  WorkScheduler.runWithPreamble(File, BindWithForward(Task, 
std::move(Contents),
-  File.str(),
-  std::move(Callback)));
+  WorkScheduler.runWithPreamble("CodeComplete", File,
+BindWithForward(Task, std::move(Contents),
+File.str(),
+std::move(Callback)));
 }
 
 void ClangdServer::signatureHelp(
@@ -222,7 +223,8 @@ void ClangdServer::signatureHelp(
   };
 
   WorkScheduler.runWithPreamble(
-  File, BindWithForward(Action, File.str(), std::move(Callback)));
+  "SignatureHelp", File,
+  BindWithForward(Action, File.str(), std::move(Callback)));
 }
 
 llvm::Expected
@@ -307,7 +309,7 @@ void ClangdServer::rename(
   };
 
   WorkScheduler.runWithAST(
-  File,
+  "Rename", File,
   BindWithForward(Action, File.str(), NewName.str(), std::move(Callback)));
 }
 
@@ -376,7 +378,8 @@ void ClangdServer::dumpAST(PathRef File,
 Callback(Result);
   };
 
-  WorkScheduler.runWithAST(File, BindWithForward(Action, std::move(Callback)));
+  WorkScheduler.runWithAST("DumpAST", File,
+   BindWithForward(Action, std::move(Callback)));
 }
 
 void ClangdServer::findDefinitions(
@@ -392,7 +395,8 @@ void ClangdServer::findDefinitions(
 Callback(make_tagged(std::move(Result), TaggedFS.Tag));
   };
 
-  WorkScheduler.runWithAST(File, BindWithForward(Action, std::move(Callback)));
+  WorkScheduler.runWithAST("Definitions", File,
+   BindWithForward(Action, std::move(Callback)));
 }
 
 llvm::Optional ClangdServer::switchSourceHeader(PathRef Path) {
@@ -488,7 +492,8 @@ void ClangdServer::findDocumentHighlight
 Callback(make_tagged(std::move(Result), TaggedFS.Tag));
   };
 
-  WorkScheduler.runWithAST(File, BindWithForward(Action, std::move(Callback)));
+  WorkScheduler.runWithAST("Highlights", File,
+   BindWithForward(Action, std::move(Callback)));
 }
 
 void ClangdServer::findHover(
@@ -511,7 +516,8 @@ void ClangdServer::findHover(
 Callback(make_tagged(std::move(Result), TaggedFS.Tag));
   };
 
-  WorkScheduler.runWithAST(File, BindWithForward(Action, std::move(Callback)));
+  WorkScheduler.runWithAST("Hover", File,
+   BindWithForward(Action, std::move(Callback)));
 }
 
 void ClangdServer::scheduleReparseAndDiags(

Modified: clang-tools-extra/trunk/clangd/TUScheduler.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/TUScheduler.cpp?rev=325480&r1=325479&r2=325480&view=diff
==
--- clang-tools-extra/trunk/clangd/TUScheduler.cpp (original)
+++ clang-tools-extra/trunk/clangd/TUScheduler.cpp Mon Feb 19 01:56:28 2018
@@ -44,8 +44,10 @@
 
 #include "TUScheduler.h"
 #include "Logger.h"
+#include "Trace.h"
 #include "clang/Frontend/PCHContainerOperations.h"
 #include "llvm/Support/Errc.h"
+#include "llvm/Support/Path.h"
 #include 
 #include 
 #include 
@@ -67,7 +69,8 @@ class ASTWorkerHandle;
 /// worker.
 class ASTWorker {
   friend c

[clang-tools-extra] r325493 - [clangd] Invert return value of fuzzyFind() (fix MemIndex's return value)

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Feb 19 05:04:41 2018
New Revision: 325493

URL: http://llvm.org/viewvc/llvm-project?rev=325493&view=rev
Log:
[clangd] Invert return value of fuzzyFind() (fix MemIndex's return value)

Have had way too many bugs by converting between "isComplete" and
"isIncomplete". LSP is immovable, so use isIncomplete everywhere.

Modified:
clang-tools-extra/trunk/clangd/CodeComplete.cpp
clang-tools-extra/trunk/clangd/index/Index.h
clang-tools-extra/trunk/clangd/index/Merge.cpp
clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp
clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp

Modified: clang-tools-extra/trunk/clangd/CodeComplete.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/CodeComplete.cpp?rev=325493&r1=325492&r2=325493&view=diff
==
--- clang-tools-extra/trunk/clangd/CodeComplete.cpp (original)
+++ clang-tools-extra/trunk/clangd/CodeComplete.cpp Mon Feb 19 05:04:41 2018
@@ -919,8 +919,9 @@ private:
   Req.Query,
   llvm::join(Req.Scopes.begin(), Req.Scopes.end(), ",")));
 // Run the query against the index.
-Incomplete |= !Opts.Index->fuzzyFind(
-Req, [&](const Symbol &Sym) { ResultsBuilder.insert(Sym); });
+if (Opts.Index->fuzzyFind(
+Req, [&](const Symbol &Sym) { ResultsBuilder.insert(Sym); }))
+  Incomplete = true;
 return std::move(ResultsBuilder).build();
   }
 
@@ -978,7 +979,8 @@ private:
 NSema += bool(SemaResult);
 NIndex += bool(IndexResult);
 NBoth += SemaResult && IndexResult;
-Incomplete |= Candidates.push({C, Scores});
+if (Candidates.push({C, Scores}))
+  Incomplete = true;
   }
 
   CompletionItem toCompletionItem(const CompletionCandidate &Candidate,

Modified: clang-tools-extra/trunk/clangd/index/Index.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/Index.h?rev=325493&r1=325492&r2=325493&view=diff
==
--- clang-tools-extra/trunk/clangd/index/Index.h (original)
+++ clang-tools-extra/trunk/clangd/index/Index.h Mon Feb 19 05:04:41 2018
@@ -255,8 +255,7 @@ public:
   /// each matched symbol before returning.
   /// If returned Symbols are used outside Callback, they must be deep-copied!
   ///
-  /// Returns true if the result list is complete, false if it was truncated 
due
-  /// to MaxCandidateCount
+  /// Returns true if there may be more results (limited by MaxCandidateCount).
   virtual bool
   fuzzyFind(const FuzzyFindRequest &Req,
 llvm::function_ref Callback) const = 0;

Modified: clang-tools-extra/trunk/clangd/index/Merge.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/Merge.cpp?rev=325493&r1=325492&r2=325493&view=diff
==
--- clang-tools-extra/trunk/clangd/index/Merge.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/Merge.cpp Mon Feb 19 05:04:41 2018
@@ -49,7 +49,7 @@ class MergedIndex : public SymbolIndex {
  for (const Symbol &S : Dyn)
if (!SeenDynamicSymbols.count(S.ID))
  Callback(S);
- return !More; // returning true indicates the result is complete.
+ return More;
   }
 
 private:

Modified: clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp?rev=325493&r1=325492&r2=325493&view=diff
==
--- clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp (original)
+++ clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp Mon Feb 19 
05:04:41 2018
@@ -698,7 +698,7 @@ public:
   fuzzyFind(const FuzzyFindRequest &Req,
 llvm::function_ref Callback) const override {
 Requests.push_back(Req);
-return false;
+return true;
   }
 
   const std::vector allRequests() const { return Requests; }

Modified: clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp?rev=325493&r1=325492&r2=325493&view=diff
==
--- clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp (original)
+++ clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp Mon Feb 19 05:04:41 
2018
@@ -90,12 +90,15 @@ generateNumSymbols(int Begin, int End,
 }
 
 std::vector match(const SymbolIndex &I,
-   const FuzzyFindRequest &Req) {
+   const FuzzyFindRequest &Req,
+   bool *Incomplete = nullptr) {
   std::vector Matches;
-  I.fuzzyFind(Req, [&](const Symbol &Sym) {
+  bool IsIncomplete = I.fuzzyFind(Req, [&](const Symbol &Sym) {
 

r325553 - [Sema] Fix -Wunused-variable

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Feb 19 23:21:56 2018
New Revision: 325553

URL: http://llvm.org/viewvc/llvm-project?rev=325553&view=rev
Log:
[Sema] Fix -Wunused-variable

Modified:
cfe/trunk/lib/Sema/AnalysisBasedWarnings.cpp

Modified: cfe/trunk/lib/Sema/AnalysisBasedWarnings.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/AnalysisBasedWarnings.cpp?rev=325553&r1=325552&r2=325553&view=diff
==
--- cfe/trunk/lib/Sema/AnalysisBasedWarnings.cpp (original)
+++ cfe/trunk/lib/Sema/AnalysisBasedWarnings.cpp Mon Feb 19 23:21:56 2018
@@ -366,7 +366,6 @@ static void checkThrowInNonThrowingFunc(
 return;
   if (BodyCFG->getExit().pred_empty())
 return;
-  SourceLocation OpLoc;
   visitReachableThrows(BodyCFG, [&](const CXXThrowExpr *Throw, CFGBlock 
&Block) {
 if (throwEscapes(S, Throw, Block, BodyCFG))
   EmitDiagForCXXThrowInNonThrowingFunc(S, Throw->getThrowLoc(), FD);


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r325574 - [clangd] Dump stack on crash

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Feb 20 03:46:39 2018
New Revision: 325574

URL: http://llvm.org/viewvc/llvm-project?rev=325574&view=rev
Log:
[clangd] Dump stack on crash

Modified:
clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp

Modified: clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp?rev=325574&r1=325573&r2=325574&view=diff
==
--- clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp (original)
+++ clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp Tue Feb 20 03:46:39 2018
@@ -135,6 +135,7 @@ static llvm::cl::opt YamlSymbolFil
 llvm::cl::init(""), llvm::cl::Hidden);
 
 int main(int argc, char *argv[]) {
+  llvm::sys::PrintStackTraceOnErrorSignal(argv[0]);
   llvm::cl::ParseCommandLineOptions(argc, argv, "clangd");
   if (Test) {
 RunSynchronously = true;


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r325774 - [clangd] Allow embedders some control over when diagnostics are generated.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Thu Feb 22 05:11:12 2018
New Revision: 325774

URL: http://llvm.org/viewvc/llvm-project?rev=325774&view=rev
Log:
[clangd] Allow embedders some control over when diagnostics are generated.

Summary:
Through the C++ API, we support for a given snapshot version:
 - Yes: make sure we generate diagnostics for exactly this version
 - Auto: generate eventually-consistent diagnostics for at least this version
 - No: don't generate diagnostics for this version
Eventually auto should be debounced for better UX.

Through LSP, we force diagnostics for initial load (bypassing future debouncing)
and all updates follow the "auto" policy.

This is complicated to implement under the CancellationFlag design, so
rewrote that part to just inspect the queue instead.

It turns out we never pass None to the diagnostics callback, so remove Optional
from the signature. The questionable behavior of not invoking the callback at
all if CppFile::rebuild fails is not changed.

Reviewers: ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D43518

Modified:
clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.h
clang-tools-extra/trunk/clangd/TUScheduler.cpp
clang-tools-extra/trunk/clangd/TUScheduler.h
clang-tools-extra/trunk/clangd/Threading.cpp
clang-tools-extra/trunk/clangd/Threading.h
clang-tools-extra/trunk/unittests/clangd/TUSchedulerTests.cpp

Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=325774&r1=325773&r2=325774&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp Thu Feb 22 05:11:12 2018
@@ -141,7 +141,8 @@ void ClangdLSPServer::onDocumentDidOpen(
   if (Params.metadata && !Params.metadata->extraFlags.empty())
 CDB.setExtraFlagsForFile(Params.textDocument.uri.file(),
  std::move(Params.metadata->extraFlags));
-  Server.addDocument(Params.textDocument.uri.file(), Params.textDocument.text);
+  Server.addDocument(Params.textDocument.uri.file(), Params.textDocument.text,
+ WantDiagnostics::Yes);
 }
 
 void ClangdLSPServer::onDocumentDidChange(DidChangeTextDocumentParams &Params) 
{
@@ -150,7 +151,7 @@ void ClangdLSPServer::onDocumentDidChang
   "can only apply one change at a time");
   // We only support full syncing right now.
   Server.addDocument(Params.textDocument.uri.file(),
- Params.contentChanges[0].text);
+ Params.contentChanges[0].text, WantDiagnostics::Auto);
 }
 
 void ClangdLSPServer::onFileEvent(DidChangeWatchedFilesParams &Params) {

Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=325774&r1=325773&r2=325774&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Thu Feb 22 05:11:12 2018
@@ -110,11 +110,12 @@ void ClangdServer::setRootPath(PathRef R
 this->RootPath = NewRootPath;
 }
 
-void ClangdServer::addDocument(PathRef File, StringRef Contents) {
+void ClangdServer::addDocument(PathRef File, StringRef Contents,
+   WantDiagnostics WantDiags) {
   DocVersion Version = DraftMgr.updateDraft(File, Contents);
   auto TaggedFS = FSProvider.getTaggedFileSystem(File);
   scheduleReparseAndDiags(File, VersionedDraft{Version, Contents.str()},
-  std::move(TaggedFS));
+  WantDiags, std::move(TaggedFS));
 }
 
 void ClangdServer::removeDocument(PathRef File) {
@@ -133,7 +134,8 @@ void ClangdServer::forceReparse(PathRef
   CompileArgs.invalidate(File);
 
   auto TaggedFS = FSProvider.getTaggedFileSystem(File);
-  scheduleReparseAndDiags(File, std::move(FileContents), std::move(TaggedFS));
+  scheduleReparseAndDiags(File, std::move(FileContents), WantDiagnostics::Yes,
+  std::move(TaggedFS));
 }
 
 void ClangdServer::codeComplete(
@@ -519,20 +521,16 @@ void ClangdServer::findHover(
 }
 
 void ClangdServer::scheduleReparseAndDiags(
-PathRef File, VersionedDraft Contents,
+PathRef File, VersionedDraft Contents, WantDiagnostics WantDiags,
 Tagged> TaggedFS) {
   tooling::CompileCommand Command = CompileArgs.getCompileCommand(File);
 
-  using OptDiags = llvm::Optional>;
-
   DocVersion Version = Contents.Version;
   Path FileStr = File.str();
   VFSTag Tag = std::move(TaggedFS.Tag);
 
-  auto Callback = [this, Version, FileStr, Tag](OptDiags Diags) {
-if (!Diags)
-  return; //

[clang-tools-extra] r325801 - [clangd] fix test use-after-free from r325774

[Sam McCall via cfe-commits]

Author: sammccall
Date: Thu Feb 22 07:33:33 2018
New Revision: 325801

URL: http://llvm.org/viewvc/llvm-project?rev=325801&view=rev
Log:
[clangd] fix test use-after-free from r325774

Modified:
clang-tools-extra/trunk/unittests/clangd/TUSchedulerTests.cpp

Modified: clang-tools-extra/trunk/unittests/clangd/TUSchedulerTests.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/TUSchedulerTests.cpp?rev=325801&r1=325800&r2=325801&view=diff
==
--- clang-tools-extra/trunk/unittests/clangd/TUSchedulerTests.cpp (original)
+++ clang-tools-extra/trunk/unittests/clangd/TUSchedulerTests.cpp Thu Feb 22 
07:33:33 2018
@@ -91,14 +91,13 @@ TEST_F(TUSchedulerTests, MissingFiles) {
 TEST_F(TUSchedulerTests, WantDiagnostics) {
   std::atomic CallbackCount(0);
   {
+// To avoid a racy test, don't allow tasks to actualy run on the worker
+// thread until we've scheduled them all.
+Notification Ready;
 TUScheduler S(getDefaultAsyncThreadsCount(),
   /*StorePreamblesInMemory=*/true,
   /*ASTParsedCallback=*/nullptr);
 auto Path = testPath("foo.cpp");
-
-// To avoid a racy test, don't allow tasks to actualy run on the worker
-// thread until we've scheduled them all.
-Notification Ready;
 S.update(Path, getInputs(Path, ""), WantDiagnostics::Yes,
  [&](std::vector) { Ready.wait(); });
 


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r325868 - [clangd] BindWithForward -> Bind. NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Thu Feb 22 23:54:17 2018
New Revision: 325868

URL: http://llvm.org/viewvc/llvm-project?rev=325868&view=rev
Log:
[clangd] BindWithForward -> Bind. NFC

Modified:
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/Function.h
clang-tools-extra/trunk/clangd/TUScheduler.cpp
clang-tools-extra/trunk/unittests/clangd/SyncAPI.cpp

Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=325868&r1=325867&r2=325868&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Thu Feb 22 23:54:17 2018
@@ -181,10 +181,9 @@ void ClangdServer::codeComplete(
 Callback(make_tagged(std::move(Result), std::move(TaggedFS.Tag)));
   };
 
-  WorkScheduler.runWithPreamble("CodeComplete", File,
-BindWithForward(Task, std::move(Contents),
-File.str(),
-std::move(Callback)));
+  WorkScheduler.runWithPreamble(
+  "CodeComplete", File,
+  Bind(Task, std::move(Contents), File.str(), std::move(Callback)));
 }
 
 void ClangdServer::signatureHelp(
@@ -224,9 +223,8 @@ void ClangdServer::signatureHelp(
 TaggedFS.Tag));
   };
 
-  WorkScheduler.runWithPreamble(
-  "SignatureHelp", File,
-  BindWithForward(Action, File.str(), std::move(Callback)));
+  WorkScheduler.runWithPreamble("SignatureHelp", File,
+Bind(Action, File.str(), std::move(Callback)));
 }
 
 llvm::Expected
@@ -312,7 +310,7 @@ void ClangdServer::rename(
 
   WorkScheduler.runWithAST(
   "Rename", File,
-  BindWithForward(Action, File.str(), NewName.str(), std::move(Callback)));
+  Bind(Action, File.str(), NewName.str(), std::move(Callback)));
 }
 
 Expected
@@ -378,8 +376,7 @@ void ClangdServer::dumpAST(PathRef File,
 Callback(Result);
   };
 
-  WorkScheduler.runWithAST("DumpAST", File,
-   BindWithForward(Action, std::move(Callback)));
+  WorkScheduler.runWithAST("DumpAST", File, Bind(Action, std::move(Callback)));
 }
 
 void ClangdServer::findDefinitions(
@@ -396,7 +393,7 @@ void ClangdServer::findDefinitions(
   };
 
   WorkScheduler.runWithAST("Definitions", File,
-   BindWithForward(Action, std::move(Callback)));
+   Bind(Action, std::move(Callback)));
 }
 
 llvm::Optional ClangdServer::switchSourceHeader(PathRef Path) {
@@ -493,7 +490,7 @@ void ClangdServer::findDocumentHighlight
   };
 
   WorkScheduler.runWithAST("Highlights", File,
-   BindWithForward(Action, std::move(Callback)));
+   Bind(Action, std::move(Callback)));
 }
 
 void ClangdServer::findHover(
@@ -516,8 +513,7 @@ void ClangdServer::findHover(
 Callback(make_tagged(std::move(Result), TaggedFS.Tag));
   };
 
-  WorkScheduler.runWithAST("Hover", File,
-   BindWithForward(Action, std::move(Callback)));
+  WorkScheduler.runWithAST("Hover", File, Bind(Action, std::move(Callback)));
 }
 
 void ClangdServer::scheduleReparseAndDiags(

Modified: clang-tools-extra/trunk/clangd/Function.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/Function.h?rev=325868&r1=325867&r2=325868&view=diff
==
--- clang-tools-extra/trunk/clangd/Function.h (original)
+++ clang-tools-extra/trunk/clangd/Function.h Thu Feb 22 23:54:17 2018
@@ -117,7 +117,7 @@ public:
  std::forward(Rest)...)) {
 
 #ifndef NDEBUG
-assert(!WasCalled && "Can only call result of BindWithForward once.");
+assert(!WasCalled && "Can only call result of Bind once.");
 WasCalled = true;
 #endif
 return CallImpl(llvm::index_sequence_for(),
@@ -132,7 +132,7 @@ public:
 /// The returned object must be called no more than once, as \p As are
 /// std::forwarded'ed (therefore can be moved) into \p F during the call.
 template 
-ForwardBinder BindWithForward(Func F, Args &&... As) {
+ForwardBinder Bind(Func F, Args &&... As) {
   return ForwardBinder(
   std::make_tuple(std::forward(F), std::forward(As)...));
 }

Modified: clang-tools-extra/trunk/clangd/TUScheduler.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/TUScheduler.cpp?rev=325868&r1=325867&r2=325868&view=diff
==
--- clang-tools-extra/trunk/clangd/TUScheduler.cpp (original)
+++ clang-tools-extra/trunk/clangd/TUScheduler.cpp Thu Feb 22 23:54:17 2018
@@ -218,7 +218,7 @@ void ASTWorker::update(
   OnUpdated(std::move(*Diags));
   };
 
-  startTask("Update", BindWithForward(Task, std::move(OnUpdated)), WantDiags);
+ 

[clang-tools-extra] r320972 - [clangd] in VSCode client, filter extensions properly and only accept file: URIs

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Dec 18 03:29:45 2017
New Revision: 320972

URL: http://llvm.org/viewvc/llvm-project?rev=320972&view=rev
Log:
[clangd] in VSCode client, filter extensions properly and only accept file: URIs

Summary:
The filtering wasn't previously working as intended - the string list is
interpreted as a list of editor modes, not file extensions.
(It happens to mostly work as "c" and "cpp" are the names of modes, but we're
missing objective-c)

The file: restriction is new - clangd needs to be able to convert URI<->path
in order to determine how to build.

Reviewers: hokein

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41343

Modified:
clang-tools-extra/trunk/clangd/clients/clangd-vscode/src/extension.ts

Modified: clang-tools-extra/trunk/clangd/clients/clangd-vscode/src/extension.ts
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/clients/clangd-vscode/src/extension.ts?rev=320972&r1=320971&r2=320972&view=diff
==
--- clang-tools-extra/trunk/clangd/clients/clangd-vscode/src/extension.ts 
(original)
+++ clang-tools-extra/trunk/clangd/clients/clangd-vscode/src/extension.ts Mon 
Dec 18 03:29:45 2017
@@ -22,11 +22,11 @@ export function activate(context: vscode
 
 const serverOptions: vscodelc.ServerOptions = { command: clangdPath, args: 
clangdArgs };
 
-const cppFileExtensions: string[] = ['cpp', 'c', 'cc', 'cxx', 'c++', 'm', 
'mm', 'h', 'hh', 'hpp', 'hxx', 'inc'];
-const cppFileExtensionsPattern = cppFileExtensions.join();
+const filePattern: string = '**/*.{' +
+  ['cpp', 'c', 'cc', 'cxx', 'c++', 'm', 'mm', 'h', 'hh', 'hpp', 'hxx', 
'inc'].join() + '}';
 const clientOptions: vscodelc.LanguageClientOptions = {
 // Register the server for C/C++ files
-documentSelector: cppFileExtensions,
+documentSelector: [{scheme: 'file', pattern: filePattern}],
 uriConverters: {
 // FIXME: by default the URI sent over the protocol will be 
percent encoded (see rfc3986#section-2.1)
 //the "workaround" below disables temporarily the encoding 
until decoding
@@ -35,7 +35,7 @@ export function activate(context: vscode
 protocol2Code: (uri: string) : vscode.Uri => vscode.Uri.parse(uri)
 },
 synchronize: !syncFileEvents ? undefined : {
-fileEvents: vscode.workspace.createFileSystemWatcher('**/*.{' + 
cppFileExtensionsPattern + '}')
+fileEvents: vscode.workspace.createFileSystemWatcher(filePattern)
 }
 };
 


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r321065 - [clangd] Add unit tests for signature help. SigHelp/CodeComplete lit tests are smoke only.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Dec 19 02:29:27 2017
New Revision: 321065

URL: http://llvm.org/viewvc/llvm-project?rev=321065&view=rev
Log:
[clangd] Add unit tests for signature help. SigHelp/CodeComplete lit tests are 
smoke only.

Modified:
clang-tools-extra/trunk/test/clangd/completion.test
clang-tools-extra/trunk/test/clangd/signature-help.test
clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp

Modified: clang-tools-extra/trunk/test/clangd/completion.test
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/clangd/completion.test?rev=321065&r1=321064&r2=321065&view=diff
==
--- clang-tools-extra/trunk/test/clangd/completion.test (original)
+++ clang-tools-extra/trunk/test/clangd/completion.test Tue Dec 19 02:29:27 2017
@@ -6,13 +6,13 @@ Content-Length: 125
 
 
{"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":123,"rootPath":"clangd","capabilities":{},"trace":"off"}}
 
-Content-Length: 246
+Content-Length: 186
 
-{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"file:///main.cpp","languageId":"cpp","version":1,"text":"struct
 fake { int a, bb, ccc; int f(int i, const float f) const; };\nint main() {\n  
fake f;\n  f.\n}\n"}}}
+{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"file:///main.cpp","languageId":"cpp","version":1,"text":"struct
 S { int a; };\nint main() {\nS().\n}"}}}
 
 Content-Length: 148
 
-{"jsonrpc":"2.0","id":1,"method":"textDocument/completion","params":{"textDocument":{"uri":"file:///main.cpp"},"position":{"line":3,"character":5}}}
+{"jsonrpc":"2.0","id":1,"method":"textDocument/completion","params":{"textDocument":{"uri":"file:///main.cpp"},"position":{"line":2,"character":4}}}
 #  CHECK:  "id": 1
 # CHECK-NEXT:  "jsonrpc": "2.0",
 # CHECK-NEXT:  "result": {
@@ -27,157 +27,31 @@ Content-Length: 148
 # CHECK-NEXT:  "label": "a",
 # CHECK-NEXT:  "sortText": "{{.*}}a"
 # CHECK-NEXT:},
-# CHECK-NEXT:{
-# CHECK-NEXT:  "detail": "int",
-# CHECK-NEXT:  "filterText": "bb",
-# CHECK-NEXT:  "insertText": "bb",
-# CHECK-NEXT:  "insertTextFormat": 1,
-# CHECK-NEXT:  "kind": 5,
-# CHECK-NEXT:  "label": "bb",
-# CHECK-NEXT:  "sortText": "{{.*}}bb"
-# CHECK-NEXT:},
-# CHECK-NEXT:{
-# CHECK-NEXT:  "detail": "int",
-# CHECK-NEXT:  "filterText": "ccc",
-# CHECK-NEXT:  "insertText": "ccc",
-# CHECK-NEXT:  "insertTextFormat": 1,
-# CHECK-NEXT:  "kind": 5,
-# CHECK-NEXT:  "label": "ccc",
-# CHECK-NEXT:  "sortText": "{{.*}}ccc"
-# CHECK-NEXT:},
-# CHECK-NEXT:{
-# CHECK-NEXT:  "detail": "int",
-# CHECK-NEXT:  "filterText": "f",
-# CHECK-NEXT:  "insertText": "f",
-# CHECK-NEXT:  "insertTextFormat": 1,
-# CHECK-NEXT:  "kind": 2,
-# CHECK-NEXT:  "label": "f(int i, const float f) const",
-# CHECK-NEXT:  "sortText": "{{.*}}f"
-# CHECK-NEXT:},
-# CHECK-NEXT:{
-# CHECK-NEXT:  "filterText": "fake",
-# CHECK-NEXT:  "insertText": "fake",
-# CHECK-NEXT:  "insertTextFormat": 1,
-# CHECK-NEXT:  "kind": 7,
-# CHECK-NEXT:  "label": "fake::",
-# CHECK-NEXT:  "sortText": "{{.*}}fake"
-# CHECK-NEXT:},
-# FIXME: Why do buildbots show different operator=s here?
-#  CHECK:{
-#  CHECK:  "detail": "void",
-# CHECK-NEXT:  "filterText": "~fake",
-# CHECK-NEXT:  "insertText": "~fake",
-# CHECK-NEXT:  "insertTextFormat": 1,
-# CHECK-NEXT:  "kind": 4,
-# CHECK-NEXT:  "label": "~fake()",
-# CHECK-NEXT:  "sortText": "{{.*}}~fake"
-# CHECK-NEXT:}
-# CHECK-NEXT:  ]
-Content-Length: 148
-
-{"jsonrpc":"2.0","id":2,"method":"textDocument/completion","params":{"textDocument":{"uri":"file:///main.cpp"},"position":{"line":3,"character":5}}}
-#  CHECK:  "id": 2
-# CHECK-NEXT:  "jsonrpc": "2.0",
-# CHECK-NEXT:  "result": {
-# CHECK-NEXT:"isIncomplete": false,
-# CHECK-NEXT:"items": [
-# CHECK-NEXT:{
-# CHECK-NEXT:  "detail": "int",
-# CHECK-NEXT:  "filterText": "a",
-# CHECK-NEXT:  "insertText": "a",
-# CHECK-NEXT:  "insertTextFormat": 1,
-# CHECK-NEXT:  "kind": 5,
-# CHECK-NEXT:  "label": "a",
-# CHECK-NEXT:  "sortText": "{{.*}}"
-# CHECK-NEXT:},
-# CHECK-NEXT:{
-# CHECK-NEXT:  "detail": "int",
-# CHECK-NEXT:  "filterText": "bb",
-# CHECK-NEXT:  "insertText": "bb",
-# CHECK-NEXT:  "insertTextFormat": 1,
-# CHECK-NEXT:  "kind": 5,
-# CHECK-NEXT:  "label": "bb",
-# CHECK-NEXT:  "sortText": "{{.*}}"
-# CHECK-NEXT:},
-# CHECK-NEXT:{
-# CHECK-NEXT:  "detail": "int",
-# CHECK-NEXT:  "filterText": "ccc",
-# CHECK-NEXT:  "insertText": "ccc",
-# CHECK-NEXT:  "insertTextFormat": 1,
-# CHECK-NEXT:  "kind": 5,
-# CHECK-NEXT:  "label": "ccc",
-# CHECK-NEXT:  "sortText": "{{.*}}"
-# CHECK-NEXT:},
-# CHECK-NEXT:{
-# CHECK-NEXT:  "detail": "int

[clang-tools-extra] r321073 - [clangd] Expose offset <-> LSP position functions, and fix bugs

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Dec 19 04:23:48 2017
New Revision: 321073

URL: http://llvm.org/viewvc/llvm-project?rev=321073&view=rev
Log:
[clangd] Expose offset <-> LSP position functions, and fix bugs

Summary:
- Moved these functions to SourceCode.h
- added unit tests
- fix off by one in positionToOffset: Offset - 1 in final calculation was wrong
- fixed formatOnType which had an equal and opposite off-by-one
- positionToOffset and offsetToPosition both consistently clamp to beginning/end
  of file when input is out of range
- gave variables more descriptive names
- removed windows line ending fixmes where there is nothing to fix
- elaborated on UTF-8 fixmes

This will conflict with Eric's D41281, but in a pretty easy-to-resolve way.

Reviewers: ioeric

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41351

Added:
clang-tools-extra/trunk/clangd/SourceCode.cpp
clang-tools-extra/trunk/clangd/SourceCode.h
clang-tools-extra/trunk/unittests/clangd/SourceCodeTests.cpp
Modified:
clang-tools-extra/trunk/clangd/CMakeLists.txt
clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.h
clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt
clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp

Modified: clang-tools-extra/trunk/clangd/CMakeLists.txt
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/CMakeLists.txt?rev=321073&r1=321072&r2=321073&view=diff
==
--- clang-tools-extra/trunk/clangd/CMakeLists.txt (original)
+++ clang-tools-extra/trunk/clangd/CMakeLists.txt Tue Dec 19 04:23:48 2017
@@ -18,6 +18,7 @@ add_clang_library(clangDaemon
   Logger.cpp
   Protocol.cpp
   ProtocolHandlers.cpp
+  SourceCode.cpp
   Trace.cpp
   index/FileIndex.cpp
   index/Index.cpp

Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=321073&r1=321072&r2=321073&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp Tue Dec 19 04:23:48 2017
@@ -9,6 +9,7 @@
 
 #include "ClangdLSPServer.h"
 #include "JSONRPCDispatcher.h"
+#include "SourceCode.h"
 #include "llvm/Support/FormatVariadic.h"
 
 using namespace clang::clangd;

Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=321073&r1=321072&r2=321073&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Tue Dec 19 04:23:48 2017
@@ -8,6 +8,7 @@
 //===---===//
 
 #include "ClangdServer.h"
+#include "SourceCode.h"
 #include "clang/Format/Format.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/CompilerInvocation.h"
@@ -57,29 +58,6 @@ public:
 
 } // namespace
 
-size_t clangd::positionToOffset(StringRef Code, Position P) {
-  size_t Offset = 0;
-  for (int I = 0; I != P.line; ++I) {
-// FIXME: \r\n
-// FIXME: UTF-8
-size_t F = Code.find('\n', Offset);
-if (F == StringRef::npos)
-  return 0; // FIXME: Is this reasonable?
-Offset = F + 1;
-  }
-  return (Offset == 0 ? 0 : (Offset - 1)) + P.character;
-}
-
-/// Turn an offset in Code into a [line, column] pair.
-Position clangd::offsetToPosition(StringRef Code, size_t Offset) {
-  StringRef JustBefore = Code.substr(0, Offset);
-  // FIXME: \r\n
-  // FIXME: UTF-8
-  int Lines = JustBefore.count('\n');
-  int Cols = JustBefore.size() - JustBefore.rfind('\n') - 1;
-  return {Lines, Cols};
-}
-
 Tagged>
 RealFileSystemProvider::getTaggedFileSystem(PathRef File) {
   return make_tagged(vfs::getRealFileSystem(), VFSTag());
@@ -337,7 +315,7 @@ ClangdServer::formatOnType(StringRef Cod
   size_t PreviousLBracePos = StringRef(Code).find_last_of('{', CursorPos);
   if (PreviousLBracePos == StringRef::npos)
 PreviousLBracePos = CursorPos;
-  size_t Len = 1 + CursorPos - PreviousLBracePos;
+  size_t Len = CursorPos - PreviousLBracePos;
 
   return formatCode(Code, File, {tooling::Range(PreviousLBracePos, Len)});
 }

Modified: clang-tools-extra/trunk/clangd/ClangdServer.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.h?rev=321073&r1=321072&r2=321073&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.h (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.h Tue Dec 19 04:23:48 2017
@@ -35,12 +35,6 @@ class PCHContainerOperations;
 
 na

[clang-tools-extra] r321086 - [clangd] Fix warnings/compiler pickiness after r321083

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Dec 19 09:05:00 2017
New Revision: 321086

URL: http://llvm.org/viewvc/llvm-project?rev=321086&view=rev
Log:
[clangd] Fix warnings/compiler pickiness after r321083

Modified:
clang-tools-extra/trunk/clangd/CodeComplete.cpp
clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp

Modified: clang-tools-extra/trunk/clangd/CodeComplete.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/CodeComplete.cpp?rev=321086&r1=321085&r2=321086&view=diff
==
--- clang-tools-extra/trunk/clangd/CodeComplete.cpp (original)
+++ clang-tools-extra/trunk/clangd/CodeComplete.cpp Tue Dec 19 09:05:00 2017
@@ -784,7 +784,7 @@ SpecifiedScope extraCompletionScope(Sema
 DeclContext *DC = S.computeDeclContext(SS);
 if (auto *NS = llvm::dyn_cast(DC)) {
   Info.Resolved = NS->getQualifiedNameAsString();
-} else if (auto *TU = llvm::dyn_cast(DC)) {
+} else if (llvm::dyn_cast(DC) != nullptr) {
   Info.Resolved = "::";
   // Sema does not include the suffix "::" in the range of SS, so we add
   // it back here.

Modified: clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp?rev=321086&r1=321085&r2=321086&view=diff
==
--- clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp (original)
+++ clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp Tue Dec 19 
09:05:00 2017
@@ -488,7 +488,7 @@ std::unique_ptr simpleIndex
   auto S = std::shared_ptr>(std::move(Snap),
 &Snap->Pointers);
   I->build(std::move(S));
-  return I;
+  return std::move(I);
 }
 
 TEST(CompletionTest, NoIndex) {


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r321087 - [clangd] Split findDefs/highlights into XRefs, from ClangdUnit. NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Dec 19 09:06:07 2017
New Revision: 321087

URL: http://llvm.org/viewvc/llvm-project?rev=321087&view=rev
Log:
[clangd] Split findDefs/highlights into XRefs, from ClangdUnit. NFC

Going to add unit tests in the next patch. (Haha!) But seriously there's
some work to do first - need to extract the markers-in-source-code
parser from CodeComplete test and make it more flexible, to allow
annotated ranges etc.

Added:
clang-tools-extra/trunk/clangd/XRefs.cpp
clang-tools-extra/trunk/clangd/XRefs.h
Modified:
clang-tools-extra/trunk/clangd/CMakeLists.txt
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/ClangdUnit.cpp
clang-tools-extra/trunk/clangd/ClangdUnit.h

Modified: clang-tools-extra/trunk/clangd/CMakeLists.txt
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/CMakeLists.txt?rev=321087&r1=321086&r2=321087&view=diff
==
--- clang-tools-extra/trunk/clangd/CMakeLists.txt (original)
+++ clang-tools-extra/trunk/clangd/CMakeLists.txt Tue Dec 19 09:06:07 2017
@@ -20,6 +20,7 @@ add_clang_library(clangDaemon
   ProtocolHandlers.cpp
   SourceCode.cpp
   Trace.cpp
+  XRefs.cpp
   index/FileIndex.cpp
   index/Index.cpp
   index/MemIndex.cpp

Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=321087&r1=321086&r2=321087&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Tue Dec 19 09:06:07 2017
@@ -8,7 +8,9 @@
 //===---===//
 
 #include "ClangdServer.h"
+#include "CodeComplete.h"
 #include "SourceCode.h"
+#include "XRefs.h"
 #include "clang/Format/Format.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/CompilerInvocation.h"

Modified: clang-tools-extra/trunk/clangd/ClangdUnit.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnit.cpp?rev=321087&r1=321086&r2=321087&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdUnit.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdUnit.cpp Tue Dec 19 09:06:07 2017
@@ -267,268 +267,15 @@ ParsedAST::Build(const Context &Ctx,
 namespace {
 
 SourceLocation getMacroArgExpandedLocation(const SourceManager &Mgr,
-   const FileEntry *FE,
-   unsigned Offset) {
-  SourceLocation FileLoc = Mgr.translateFileLineCol(FE, 1, 1);
-  return Mgr.getMacroArgExpandedLocation(FileLoc.getLocWithOffset(Offset));
-}
-
-SourceLocation getMacroArgExpandedLocation(const SourceManager &Mgr,
const FileEntry *FE, Position Pos) {
   SourceLocation InputLoc =
   Mgr.translateFileLineCol(FE, Pos.line + 1, Pos.character + 1);
   return Mgr.getMacroArgExpandedLocation(InputLoc);
 }
 
-/// Finds declarations locations that a given source location refers to.
-class DeclarationAndMacrosFinder : public index::IndexDataConsumer {
-  std::vector Decls;
-  std::vector MacroInfos;
-  const SourceLocation &SearchedLocation;
-  const ASTContext *
-  Preprocessor &PP;
-
-public:
-  DeclarationAndMacrosFinder(raw_ostream &OS,
- const SourceLocation &SearchedLocation,
- ASTContext &AST, Preprocessor &PP)
-  : SearchedLocation(SearchedLocation), AST(AST), PP(PP) {}
-
-  std::vector takeDecls() {
-// Don't keep the same declaration multiple times.
-// This can happen when nodes in the AST are visited twice.
-std::sort(Decls.begin(), Decls.end());
-auto Last = std::unique(Decls.begin(), Decls.end());
-Decls.erase(Last, Decls.end());
-return std::move(Decls);
-  }
-
-  std::vector takeMacroInfos() {
-// Don't keep the same Macro info multiple times.
-std::sort(MacroInfos.begin(), MacroInfos.end());
-auto Last = std::unique(MacroInfos.begin(), MacroInfos.end());
-MacroInfos.erase(Last, MacroInfos.end());
-return std::move(MacroInfos);
-  }
-
-  bool
-  handleDeclOccurence(const Decl *D, index::SymbolRoleSet Roles,
-  ArrayRef Relations, FileID FID,
-  unsigned Offset,
-  index::IndexDataConsumer::ASTNodeInfo ASTNode) override {
-if (isSearchedLocation(FID, Offset))
-  Decls.push_back(D);
-return true;
-  }
-
-private:
-  bool isSearchedLocation(FileID FID, unsigned Offset) const {
-const SourceManager &SourceMgr = AST.getSourceManager();
-return SourceMgr.getFileOffset(SearchedLocation) == Offset &&
-   SourceMgr.getFileID(SearchedLocation) == FID;
-  }
-
-  void finish() override {
-// 

[clang-tools-extra] r321161 - [clangd] Add debug printers for basic protocol types. NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Wed Dec 20 02:26:53 2017
New Revision: 321161

URL: http://llvm.org/viewvc/llvm-project?rev=321161&view=rev
Log:
[clangd] Add debug printers for basic protocol types. NFC

Modified:
clang-tools-extra/trunk/clangd/Protocol.cpp
clang-tools-extra/trunk/clangd/Protocol.h
clang-tools-extra/trunk/unittests/clangd/SourceCodeTests.cpp

Modified: clang-tools-extra/trunk/clangd/Protocol.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/Protocol.cpp?rev=321161&r1=321160&r2=321161&view=diff
==
--- clang-tools-extra/trunk/clangd/Protocol.cpp (original)
+++ clang-tools-extra/trunk/clangd/Protocol.cpp Wed Dec 20 02:26:53 2017
@@ -59,6 +59,10 @@ bool fromJSON(const json::Expr &E, URI &
 
 json::Expr toJSON(const URI &U) { return U.uri; }
 
+llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, const URI &U) {
+  return OS << U.uri;
+}
+
 bool fromJSON(const json::Expr &Params, TextDocumentIdentifier &R) {
   json::ObjectMapper O(Params);
   return O && O.map("uri", R.uri);
@@ -76,6 +80,10 @@ json::Expr toJSON(const Position &P) {
   };
 }
 
+llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, const Position &P) {
+  return OS << P.line << ':' << P.character;
+}
+
 bool fromJSON(const json::Expr &Params, Range &R) {
   json::ObjectMapper O(Params);
   return O && O.map("start", R.start) && O.map("end", R.end);
@@ -88,6 +96,10 @@ json::Expr toJSON(const Range &P) {
   };
 }
 
+llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, const Range &R) {
+  return OS << R.start << '-' << R.end;
+}
+
 json::Expr toJSON(const Location &P) {
   return json::obj{
   {"uri", P.uri},
@@ -95,6 +107,10 @@ json::Expr toJSON(const Location &P) {
   };
 }
 
+llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, const Location &L) {
+  return OS << L.range << '@' << L.uri;
+}
+
 bool fromJSON(const json::Expr &Params, TextDocumentItem &R) {
   json::ObjectMapper O(Params);
   return O && O.map("uri", R.uri) && O.map("languageId", R.languageId) &&

Modified: clang-tools-extra/trunk/clangd/Protocol.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/Protocol.h?rev=321161&r1=321160&r2=321161&view=diff
==
--- clang-tools-extra/trunk/clangd/Protocol.h (original)
+++ clang-tools-extra/trunk/clangd/Protocol.h Wed Dec 20 02:26:53 2017
@@ -16,6 +16,9 @@
 // Each struct has a toJSON and fromJSON function, that converts between
 // the struct and a JSON representation. (See JSONExpr.h)
 //
+// Some structs also have operator<< serialization. This is for debugging and
+// tests, and is not generally machine-readable.
+//
 
//===--===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_PROTOCOL_H
@@ -65,6 +68,7 @@ struct URI {
 };
 json::Expr toJSON(const URI &U);
 bool fromJSON(const json::Expr &, URI &);
+llvm::raw_ostream &operator<<(llvm::raw_ostream &, const URI &);
 
 struct TextDocumentIdentifier {
   /// The text document's URI.
@@ -90,6 +94,7 @@ struct Position {
 };
 bool fromJSON(const json::Expr &, Position &);
 json::Expr toJSON(const Position &);
+llvm::raw_ostream &operator<<(llvm::raw_ostream &, const Position &);
 
 struct Range {
   /// The range's start position.
@@ -107,6 +112,7 @@ struct Range {
 };
 bool fromJSON(const json::Expr &, Range &);
 json::Expr toJSON(const Range &);
+llvm::raw_ostream &operator<<(llvm::raw_ostream &, const Range &);
 
 struct Location {
   /// The text document's URI.
@@ -126,6 +132,7 @@ struct Location {
   }
 };
 json::Expr toJSON(const Location &);
+llvm::raw_ostream &operator<<(llvm::raw_ostream &, const Location &);
 
 struct Metadata {
   std::vector extraFlags;

Modified: clang-tools-extra/trunk/unittests/clangd/SourceCodeTests.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/SourceCodeTests.cpp?rev=321161&r1=321160&r2=321161&view=diff
==
--- clang-tools-extra/trunk/unittests/clangd/SourceCodeTests.cpp (original)
+++ clang-tools-extra/trunk/unittests/clangd/SourceCodeTests.cpp Wed Dec 20 
02:26:53 2017
@@ -13,10 +13,6 @@
 
 namespace clang{
 namespace clangd {
-void PrintTo(const Position &P, std::ostream *O) {
-  llvm::raw_os_ostream OS(*O);
-  OS << toJSON(P);
-}
 namespace {
 
 MATCHER_P2(Pos, Line, Col, "") {


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r321184 - [clangd] Switch xrefs and documenthighlight to annotated-code unit tests. NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Wed Dec 20 08:06:05 2017
New Revision: 321184

URL: http://llvm.org/viewvc/llvm-project?rev=321184&view=rev
Log:
[clangd] Switch xrefs and documenthighlight to annotated-code unit tests. NFC

Summary:
The goal here is again to make it easier to read and write the tests.

I've extracted `parseTextMarker` from CodeCompleteTests into an `Annotations`
class, adding features to it:
  - as well as points `^s` it allows ranges `[[...]]`
  - multiple points and ranges are supported
  - points and ranges may be named: `$name^` and `$name[[...]]`

These features are used for the xrefs tests. This also paves the way for
replacing the lit diagnostics.test with more readable unit tests, using named
ranges.

Alternative considered: `TestSelectionRange` in clang-refactor/TestSupport
Main problems were:
 - delimiting the end of ranges is awkward, requiring counting
 - comment syntax is long and at least as cryptic for most cases
 - no separate syntax for point vs range, which keeps xrefs tests concise
 - Still need to convert to Position everywhere
 - Still need helpers for common case of expecting exactly one point/range

(I'll probably promote the extra `PrintTo`s from some of the core Protocol types
into `operator<<` in `Protocol.h` itself in a separate, prior patch...)

Reviewers: ioeric

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41432

Added:
clang-tools-extra/trunk/test/clangd/xrefs.test
clang-tools-extra/trunk/unittests/clangd/Annotations.cpp
clang-tools-extra/trunk/unittests/clangd/Annotations.h
clang-tools-extra/trunk/unittests/clangd/XRefsTests.cpp
Removed:
clang-tools-extra/trunk/test/clangd/definitions.test
clang-tools-extra/trunk/test/clangd/documenthighlight.test
Modified:
clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt
clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp
clang-tools-extra/trunk/unittests/clangd/Matchers.h

Removed: clang-tools-extra/trunk/test/clangd/definitions.test
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/clangd/definitions.test?rev=321183&view=auto
==
--- clang-tools-extra/trunk/test/clangd/definitions.test (original)
+++ clang-tools-extra/trunk/test/clangd/definitions.test (removed)
@@ -1,421 +0,0 @@
-# RUN: clangd -pretty -run-synchronously < %s | FileCheck -strict-whitespace %s
-# It is absolutely vital that this file has CRLF line endings.
-#
-Content-Length: 125
-
-{"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":123,"rootPath":"clangd","capabilities":{},"trace":"off"}}
-
-Content-Length: 172
-
-{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"file:///main.cpp","languageId":"cpp","version":1,"text":"int
 main() {\nint a;\na;\n}\n"}}}
-
-Content-Length: 148
-
-{"jsonrpc":"2.0","id":1,"method":"textDocument/definition","params":{"textDocument":{"uri":"file:///main.cpp"},"position":{"line":2,"character":0}}}
-# Go to local variable
-#  CHECK:  "id": 1,
-# CHECK-NEXT:  "jsonrpc": "2.0",
-# CHECK-NEXT:  "result": [
-# CHECK-NEXT:{
-# CHECK-NEXT:  "range": {
-# CHECK-NEXT:"end": {
-# CHECK-NEXT:  "character": 5,
-# CHECK-NEXT:  "line": 1
-# CHECK-NEXT:},
-# CHECK-NEXT:"start": {
-# CHECK-NEXT:  "character": 0,
-# CHECK-NEXT:  "line": 1
-# CHECK-NEXT:}
-# CHECK-NEXT:  },
-# CHECK-NEXT:  "uri": "file:///{{([A-Z]:/)?}}main.cpp"
-# CHECK-NEXT:}
-# CHECK-NEXT:  ]
-Content-Length: 148
-
-{"jsonrpc":"2.0","id":1,"method":"textDocument/definition","params":{"textDocument":{"uri":"file:///main.cpp"},"position":{"line":2,"character":1}}}
-# Go to local variable, end of token
-#  CHECK:  "id": 1,
-# CHECK-NEXT:  "jsonrpc": "2.0",
-# CHECK-NEXT:  "result": [
-# CHECK-NEXT:{
-# CHECK-NEXT:  "range": {
-# CHECK-NEXT:"end": {
-# CHECK-NEXT:  "character": 5,
-# CHECK-NEXT:  "line": 1
-# CHECK-NEXT:},
-# CHECK-NEXT:"start": {
-# CHECK-NEXT:  "character": 0,
-# CHECK-NEXT:  "line": 1
-# CHECK-NEXT:}
-# CHECK-NEXT:  },
-# CHECK-NEXT:  "uri": "file:///{{([A-Z]:/)?}}main.cpp"
-# CHECK-NEXT:}
-# CHECK-NEXT:  ]
-Content-Length: 214
-
-{"jsonrpc":"2.0","method":"textDocument/didChange","params":{"textDocument":{"uri":"file:///main.cpp","version":2},"contentChanges":[{"text":"struct
 Foo {\nint x;\n};\nint main() {\n  Foo bar = { x : 1 };\n}\n"}]}}
-
-Content-Length: 149
-
-{"jsonrpc":"2.0","id":1,"method":"textDocument/definition","params":{"textDocument":{"uri":"file:///main.cpp"},"position":{"line":4,"character":14}}}
-# Go to field, GNU old-style field designator 
-#  CHECK:  "id": 1,
-# CHECK-NEXT:  "jsonrpc": "2.0",
-# CHECK-NEXT:  "result": [
-# CHECK-NEXT:{
-# CHECK-NEXT:  "range": {
-# CHECK-NEXT:"end": {
-# CHECK-NEX

[clang-tools-extra] r321272 - [clangd] Index symbols share storage within a slab.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Thu Dec 21 06:58:44 2017
New Revision: 321272

URL: http://llvm.org/viewvc/llvm-project?rev=321272&view=rev
Log:
[clangd] Index symbols share storage within a slab.

Summary:
Symbols are not self-contained - it's only safe to hand them out if you
guarantee the lifetime of the underlying data.

Before this lands, I'm going to measure the before/after memory usage of the
LLVM index loaded into memory in a single slab.

Reviewers: hokein

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41483

Modified:
clang-tools-extra/trunk/clangd/index/Index.cpp
clang-tools-extra/trunk/clangd/index/Index.h
clang-tools-extra/trunk/unittests/clangd/FileIndexTests.cpp
clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp
clang-tools-extra/trunk/unittests/clangd/SymbolCollectorTests.cpp

Modified: clang-tools-extra/trunk/clangd/index/Index.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/Index.cpp?rev=321272&r1=321271&r2=321272&view=diff
==
--- clang-tools-extra/trunk/clangd/index/Index.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/Index.cpp Thu Dec 21 06:58:44 2017
@@ -38,9 +38,17 @@ SymbolSlab::const_iterator SymbolSlab::f
 
 void SymbolSlab::freeze() { Frozen = true; }
 
-void SymbolSlab::insert(Symbol S) {
+void SymbolSlab::insert(const Symbol &S) {
   assert(!Frozen && "Can't insert a symbol after the slab has been frozen!");
-  Symbols[S.ID] = std::move(S);
+  auto ItInserted = Symbols.try_emplace(S.ID, S);
+  if (!ItInserted.second)
+return;
+  auto &Sym = ItInserted.first->second;
+
+  // We inserted a new symbol, so copy the underlying data.
+  intern(Sym.Name);
+  intern(Sym.Scope);
+  intern(Sym.CanonicalDeclaration.FilePath);
 }
 
 } // namespace clangd

Modified: clang-tools-extra/trunk/clangd/index/Index.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/Index.h?rev=321272&r1=321271&r2=321272&view=diff
==
--- clang-tools-extra/trunk/clangd/index/Index.h (original)
+++ clang-tools-extra/trunk/clangd/index/Index.h Thu Dec 21 06:58:44 2017
@@ -15,6 +15,7 @@
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/Hashing.h"
 #include "llvm/ADT/StringExtras.h"
+#include "llvm/ADT/StringSet.h"
 #include 
 #include 
 
@@ -23,7 +24,7 @@ namespace clangd {
 
 struct SymbolLocation {
   // The absolute path of the source file where a symbol occurs.
-  std::string FilePath;
+  llvm::StringRef FilePath;
   // The 0-based offset to the first character of the symbol from the beginning
   // of the source file.
   unsigned StartOffset;
@@ -71,16 +72,19 @@ void operator>>(llvm::StringRef HexStr,
 
 // The class presents a C++ symbol, e.g. class, function.
 //
-// FIXME: instead of having own copy fields for each symbol, we can share
-// storage from SymbolSlab.
+// WARNING: Symbols do not own much of their underlying data - typically 
strings
+// are owned by a SymbolSlab. They should be treated as non-owning references.
+// Copies are shallow.
+// When adding new unowned data fields to Symbol, remember to update
+// SymbolSlab::insert to copy them to the slab's storage.
 struct Symbol {
   // The ID of the symbol.
   SymbolID ID;
   // The unqualified name of the symbol, e.g. "bar" (for "n1::n2::bar").
-  std::string Name;
+  llvm::StringRef Name;
   // The scope (e.g. namespace) of the symbol, e.g. "n1::n2" (for
   // "n1::n2::bar").
-  std::string Scope;
+  llvm::StringRef Scope;
   // The symbol information, like symbol kind.
   index::SymbolInfo SymInfo;
   // The location of the canonical declaration of the symbol.
@@ -100,9 +104,6 @@ struct Symbol {
 
 // A symbol container that stores a set of symbols. The container will maintain
 // the lifetime of the symbols.
-//
-// FIXME: Use a space-efficient implementation, a lot of Symbol fields could
-// share the same storage.
 class SymbolSlab {
 public:
   using const_iterator = llvm::DenseMap::const_iterator;
@@ -117,11 +118,21 @@ public:
   // operation is irreversible.
   void freeze();
 
-  void insert(Symbol S);
+  // Adds the symbol to this slab.
+  // This is a deep copy: underlying strings will be owned by the slab.
+  void insert(const Symbol& S);
 
 private:
+  // Replaces S with a reference to the same string, owned by this slab.
+  void intern(llvm::StringRef &S) {
+S = S.empty() ? llvm::StringRef() : Strings.insert(S).first->getKey();
+  }
+
   bool Frozen = false;
 
+  // Intern table for strings. Not StringPool as we don't refcount, just 
insert.
+  // We use BumpPtrAllocator to avoid lots of tiny allocations for nodes.
+  llvm::StringSet Strings;
   llvm::DenseMap Symbols;
 };
 

Modified: clang-tools-extra/trunk/unittests/clangd/FileIndexTests.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/FileIndexT

[clang-tools-extra] r321302 - [clangd] Don't re-hash SymbolID in maps, just use the SHA1 data

[Sam McCall via cfe-commits]

Author: sammccall
Date: Thu Dec 21 12:11:46 2017
New Revision: 321302

URL: http://llvm.org/viewvc/llvm-project?rev=321302&view=rev
Log:
[clangd] Don't re-hash SymbolID in maps, just use the SHA1 data

Modified:
clang-tools-extra/trunk/clangd/index/Index.cpp
clang-tools-extra/trunk/clangd/index/Index.h

Modified: clang-tools-extra/trunk/clangd/index/Index.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/Index.cpp?rev=321302&r1=321301&r2=321302&view=diff
==
--- clang-tools-extra/trunk/clangd/index/Index.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/Index.cpp Thu Dec 21 12:11:46 2017
@@ -24,7 +24,7 @@ llvm::raw_ostream &operator<<(llvm::raw_
 
 void operator>>(llvm::StringRef Str, SymbolID &ID) {
   std::string HexString = fromHex(Str);
-  assert(HexString.size() == 20);
+  assert(HexString.size() == ID.HashValue.size());
   std::copy(HexString.begin(), HexString.end(), ID.HashValue.begin());
 }
 

Modified: clang-tools-extra/trunk/clangd/index/Index.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/Index.h?rev=321302&r1=321301&r2=321302&view=diff
==
--- clang-tools-extra/trunk/clangd/index/Index.h (original)
+++ clang-tools-extra/trunk/clangd/index/Index.h Thu Dec 21 12:11:46 2017
@@ -52,7 +52,9 @@ public:
 
 private:
   friend llvm::hash_code hash_value(const SymbolID &ID) {
-return hash_value(ArrayRef(ID.HashValue));
+// We already have a good hash, just return the first bytes.
+static_assert(sizeof(size_t) <= 20, "size_t longer than SHA1!");
+return *reinterpret_cast(ID.HashValue.data());
   }
   friend llvm::raw_ostream &operator<<(llvm::raw_ostream &OS,
const SymbolID &ID);


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r321346 - Fix unused variable warning in SemaTemplate. NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Thu Dec 21 23:09:51 2017
New Revision: 321346

URL: http://llvm.org/viewvc/llvm-project?rev=321346&view=rev
Log:
Fix unused variable warning in SemaTemplate. NFC

Modified:
cfe/trunk/lib/Sema/SemaTemplate.cpp

Modified: cfe/trunk/lib/Sema/SemaTemplate.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/SemaTemplate.cpp?rev=321346&r1=321345&r2=321346&view=diff
==
--- cfe/trunk/lib/Sema/SemaTemplate.cpp (original)
+++ cfe/trunk/lib/Sema/SemaTemplate.cpp Thu Dec 21 23:09:51 2017
@@ -950,7 +950,7 @@ Decl *Sema::ActOnNonTypeTemplateParamete
 if (DS.getStorageClassSpec() != DeclSpec::SCS_unspecified)
   EmitDiag(DS.getStorageClassSpecLoc());
 
-if (DeclSpec::TSCS TSCS = DS.getThreadStorageClassSpec())
+if (DS.getThreadStorageClassSpec() != TSCS_unspecified)
   EmitDiag(DS.getThreadStorageClassSpecLoc());
 
 // [dcl.inline]p1: 


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r321348 - [clangd] Improve packing of Symbol struct. NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Fri Dec 22 00:12:39 2017
New Revision: 321348

URL: http://llvm.org/viewvc/llvm-project?rev=321348&view=rev
Log:
[clangd] Improve packing of Symbol struct. NFC

Modified:
clang-tools-extra/trunk/clangd/index/Index.h
clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp

Modified: clang-tools-extra/trunk/clangd/index/Index.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/Index.h?rev=321348&r1=321347&r2=321348&view=diff
==
--- clang-tools-extra/trunk/clangd/index/Index.h (original)
+++ clang-tools-extra/trunk/clangd/index/Index.h Fri Dec 22 00:12:39 2017
@@ -82,13 +82,13 @@ void operator>>(llvm::StringRef HexStr,
 struct Symbol {
   // The ID of the symbol.
   SymbolID ID;
+  // The symbol information, like symbol kind.
+  index::SymbolInfo SymInfo;
   // The unqualified name of the symbol, e.g. "bar" (for "n1::n2::bar").
   llvm::StringRef Name;
   // The scope (e.g. namespace) of the symbol, e.g. "n1::n2" (for
   // "n1::n2::bar").
   llvm::StringRef Scope;
-  // The symbol information, like symbol kind.
-  index::SymbolInfo SymInfo;
   // The location of the canonical declaration of the symbol.
   //
   // A C++ symbol could have multiple declarations and one definition (e.g.

Modified: clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp?rev=321348&r1=321347&r2=321348&view=diff
==
--- clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp Fri Dec 22 
00:12:39 2017
@@ -101,8 +101,14 @@ bool SymbolCollector::handleDeclOccurenc
SM.getFileOffset(D->getLocEnd())};
 std::string QName = ND->getQualifiedNameAsString();
 auto ScopeAndName = splitQualifiedName(QName);
-Symbols.insert({std::move(ID), ScopeAndName.second, ScopeAndName.first,
-index::getSymbolInfo(D), std::move(Location)});
+
+Symbol S;
+S.ID = std::move(ID);
+S.Scope = ScopeAndName.first;
+S.Name = ScopeAndName.second;
+S.SymInfo = index::getSymbolInfo(D);
+S.CanonicalDeclaration = Location;
+Symbols.insert(S);
   }
 
   return true;


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [clang-tools-extra] r321286 - [clangd] Fix use after free.

[Sam McCall via cfe-commits]

Thanks Ben!

This was my r321272 - in hindsight I really should have run the tests with
asan for that type of change.

On Thu, Dec 21, 2017 at 6:51 PM, Benjamin Kramer via cfe-commits <
cfe-commits@lists.llvm.org> wrote:

> Author: d0k
> Date: Thu Dec 21 09:51:35 2017
> New Revision: 321286
>
> URL: http://llvm.org/viewvc/llvm-project?rev=321286&view=rev
> Log:
> [clangd] Fix use after free.
>
> Found by asan.
>
> Modified:
> clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp
>
> Modified: clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp
> URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/
> trunk/clangd/index/SymbolCollector.cpp?rev=321286&r1=321285&r2=321286&
> view=diff
> 
> ==
> --- clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp (original)
> +++ clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp Thu Dec 21
> 09:51:35 2017
> @@ -95,9 +95,10 @@ bool SymbolCollector::handleDeclOccurenc
>return true;
>
>  auto &SM = ND->getASTContext().getSourceManager();
> -SymbolLocation Location = {
> -makeAbsolutePath(SM, SM.getFilename(D->getLocation())),
> -SM.getFileOffset(D->getLocStart()),
> SM.getFileOffset(D->getLocEnd())};
> +std::string FilePath =
> +makeAbsolutePath(SM, SM.getFilename(D->getLocation()));
> +SymbolLocation Location = {FilePath, SM.getFileOffset(D->
> getLocStart()),
> +   SM.getFileOffset(D->getLocEnd())};
>  std::string QName = ND->getQualifiedNameAsString();
>  auto ScopeAndName = splitQualifiedName(QName);
>  Symbols.insert({std::move(ID), ScopeAndName.second,
> ScopeAndName.first,
>
>
> ___
> cfe-commits mailing list
> cfe-commits@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
>
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r321350 - [clangd] Simplify GlobalCompilationDatabase, cache missing GCDs

[Sam McCall via cfe-commits]

Author: sammccall
Date: Fri Dec 22 01:47:34 2017
New Revision: 321350

URL: http://llvm.org/viewvc/llvm-project?rev=321350&view=rev
Log:
[clangd] Simplify GlobalCompilationDatabase, cache missing GCDs

Modified:
clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.cpp
clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.h

Modified: clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.cpp?rev=321350&r1=321349&r2=321350&view=diff
==
--- clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.cpp (original)
+++ clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.cpp Fri Dec 22 
01:47:34 2017
@@ -31,12 +31,15 @@ DirectoryBasedGlobalCompilationDatabase:
 
 llvm::Optional
 DirectoryBasedGlobalCompilationDatabase::getCompileCommand(PathRef File) const 
{
-  if (auto CDB = getCompilationDatabase(File)) {
+  if (auto CDB = getCDBForFile(File)) {
 auto Candidates = CDB->getCompileCommands(File);
 if (!Candidates.empty()) {
   addExtraFlags(File, Candidates.front());
   return std::move(Candidates.front());
 }
+  } else {
+log(Context::empty(), // FIXME(ibiryukov): pass a proper Context here.
+"Failed to find compilation database for " + Twine(File));
   }
   return llvm::None;
 }
@@ -71,59 +74,32 @@ void DirectoryBasedGlobalCompilationData
 }
 
 tooling::CompilationDatabase *
-DirectoryBasedGlobalCompilationDatabase::tryLoadDatabaseFromPath(
-PathRef File) const {
-
-  namespace path = llvm::sys::path;
-  auto CachedIt = CompilationDatabases.find(File);
-
-  assert((path::is_absolute(File, path::Style::posix) ||
-  path::is_absolute(File, path::Style::windows)) &&
- "path must be absolute");
-
+DirectoryBasedGlobalCompilationDatabase::getCDBInDirLocked(PathRef Dir) const {
+  // FIXME(ibiryukov): Invalidate cached compilation databases on changes
+  auto CachedIt = CompilationDatabases.find(Dir);
   if (CachedIt != CompilationDatabases.end())
 return CachedIt->second.get();
   std::string Error = "";
-  auto CDB = tooling::CompilationDatabase::loadFromDirectory(File, Error);
-  if (CDB) {
-auto Result = CDB.get();
-CompilationDatabases.insert(std::make_pair(File, std::move(CDB)));
-return Result;
-  }
-
-  return nullptr;
+  auto CDB = tooling::CompilationDatabase::loadFromDirectory(Dir, Error);
+  auto Result = CDB.get();
+  CompilationDatabases.insert(std::make_pair(Dir, std::move(CDB)));
+  return Result;
 }
 
 tooling::CompilationDatabase *
-DirectoryBasedGlobalCompilationDatabase::getCompilationDatabase(
-PathRef File) const {
-  std::lock_guard Lock(Mutex);
-
+DirectoryBasedGlobalCompilationDatabase::getCDBForFile(PathRef File) const {
   namespace path = llvm::sys::path;
-  if (CompileCommandsDir.hasValue()) {
-tooling::CompilationDatabase *ReturnValue =
-tryLoadDatabaseFromPath(CompileCommandsDir.getValue());
-if (ReturnValue == nullptr) {
-  // FIXME(ibiryukov): pass a proper Context here.
-  log(Context::empty(), "Failed to find compilation database for " +
-Twine(File) + "in overriden directory " +
-CompileCommandsDir.getValue());
-}
-return ReturnValue;
-  }
+  assert((path::is_absolute(File, path::Style::posix) ||
+  path::is_absolute(File, path::Style::windows)) &&
+ "path must be absolute");
 
+  std::lock_guard Lock(Mutex);
+  if (CompileCommandsDir)
+return getCDBInDirLocked(*CompileCommandsDir);
   for (auto Path = path::parent_path(File); !Path.empty();
-   Path = path::parent_path(Path)) {
-auto CDB = tryLoadDatabaseFromPath(Path);
-if (!CDB)
-  continue;
-// FIXME(ibiryukov): Invalidate cached compilation databases on changes
-return CDB;
-  }
-
-  // FIXME(ibiryukov): pass a proper Context here.
-  log(Context::empty(),
-  "Failed to find compilation database for " + Twine(File));
+   Path = path::parent_path(Path))
+if (auto CDB = getCDBInDirLocked(Path))
+  return CDB;
   return nullptr;
 }
 

Modified: clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.h?rev=321350&r1=321349&r2=321350&view=diff
==
--- clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.h (original)
+++ clang-tools-extra/trunk/clangd/GlobalCompilationDatabase.h Fri Dec 22 
01:47:34 2017
@@ -65,8 +65,8 @@ public:
   void setExtraFlagsForFile(PathRef File, std::vector ExtraFlags);
 
 private:
-  tooling::CompilationDatabase *getCompilationDatabase(PathRef File) const;
-  tooling::CompilationDatabase *tryLoadDatabaseFromPath(PathRef File) const;
+  tooling::CompilationDatabase *getCDBForFile(PathRef File) const;
+  too

r321411 - [Index] Reduce size of SymbolInfo struct.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Sat Dec 23 11:31:24 2017
New Revision: 321411

URL: http://llvm.org/viewvc/llvm-project?rev=321411&view=rev
Log:
[Index] Reduce size of SymbolInfo struct.

Summary:
This is currently 16 bytes, the patch reduces it to 4.
(Building with clang on linux x84, I guess others are similar)

The only subfield that might need a bigger type is SymbolPropertySet,
I've moved it to the end of the struct so if it grows, SymbolInfo will
only be 8 bytes.

With a full index of namespace-scope symbols from the LLVM project (200k)
loaded into clangd, this saves ~2MB of RAM.

Reviewers: akyrtzi

Subscribers: ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41514

Modified:
cfe/trunk/include/clang/Index/IndexSymbol.h
cfe/trunk/lib/Index/IndexSymbol.cpp
cfe/trunk/tools/libclang/CXIndexDataConsumer.cpp

Modified: cfe/trunk/include/clang/Index/IndexSymbol.h
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Index/IndexSymbol.h?rev=321411&r1=321410&r2=321411&view=diff
==
--- cfe/trunk/include/clang/Index/IndexSymbol.h (original)
+++ cfe/trunk/include/clang/Index/IndexSymbol.h Sat Dec 23 11:31:24 2017
@@ -56,7 +56,7 @@ enum class SymbolKind : uint8_t {
   Using,
 };
 
-enum class SymbolLanguage {
+enum class SymbolLanguage : uint8_t {
   C,
   ObjC,
   CXX,
@@ -64,7 +64,7 @@ enum class SymbolLanguage {
 };
 
 /// Language specific sub-kinds.
-enum class SymbolSubKind {
+enum class SymbolSubKind : uint8_t {
   None,
   CXXCopyConstructor,
   CXXMoveConstructor,
@@ -74,8 +74,9 @@ enum class SymbolSubKind {
   UsingValue,
 };
 
+typedef uint8_t SymbolPropertySet;
 /// Set of properties that provide additional info about a symbol.
-enum class SymbolProperty : uint8_t {
+enum class SymbolProperty : SymbolPropertySet {
   Generic   = 1 << 0,
   TemplatePartialSpecialization = 1 << 1,
   TemplateSpecialization= 1 << 2,
@@ -86,7 +87,6 @@ enum class SymbolProperty : uint8_t {
   Local = 1 << 7,
 };
 static const unsigned SymbolPropertyBitNum = 8;
-typedef unsigned SymbolPropertySet;
 
 /// Set of roles that are attributed to symbol occurrences.
 enum class SymbolRole : uint32_t {
@@ -127,8 +127,8 @@ struct SymbolRelation {
 struct SymbolInfo {
   SymbolKind Kind;
   SymbolSubKind SubKind;
-  SymbolPropertySet Properties;
   SymbolLanguage Lang;
+  SymbolPropertySet Properties;
 };
 
 SymbolInfo getSymbolInfo(const Decl *D);

Modified: cfe/trunk/lib/Index/IndexSymbol.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Index/IndexSymbol.cpp?rev=321411&r1=321410&r2=321411&view=diff
==
--- cfe/trunk/lib/Index/IndexSymbol.cpp (original)
+++ cfe/trunk/lib/Index/IndexSymbol.cpp Sat Dec 23 11:31:24 2017
@@ -42,10 +42,10 @@ static bool isUnitTest(const ObjCMethodD
 
 static void checkForIBOutlets(const Decl *D, SymbolPropertySet &PropSet) {
   if (D->hasAttr()) {
-PropSet |= (unsigned)SymbolProperty::IBAnnotated;
+PropSet |= (SymbolPropertySet)SymbolProperty::IBAnnotated;
   } else if (D->hasAttr()) {
-PropSet |= (unsigned)SymbolProperty::IBAnnotated;
-PropSet |= (unsigned)SymbolProperty::IBOutletCollection;
+PropSet |= (SymbolPropertySet)SymbolProperty::IBAnnotated;
+PropSet |= (SymbolPropertySet)SymbolProperty::IBOutletCollection;
   }
 }
 
@@ -93,7 +93,7 @@ SymbolInfo index::getSymbolInfo(const De
   Info.Lang = SymbolLanguage::C;
 
   if (isFunctionLocalSymbol(D)) {
-Info.Properties |= (unsigned)SymbolProperty::Local;
+Info.Properties |= (SymbolPropertySet)SymbolProperty::Local;
   }
 
   if (const TagDecl *TD = dyn_cast(D)) {
@@ -118,17 +118,19 @@ SymbolInfo index::getSymbolInfo(const De
   if (!CXXRec->isCLike()) {
 Info.Lang = SymbolLanguage::CXX;
 if (CXXRec->getDescribedClassTemplate()) {
-  Info.Properties |= (unsigned)SymbolProperty::Generic;
+  Info.Properties |= (SymbolPropertySet)SymbolProperty::Generic;
 }
   }
 }
 
 if (isa(D)) {
-  Info.Properties |= (unsigned)SymbolProperty::Generic;
-  Info.Properties |= 
(unsigned)SymbolProperty::TemplatePartialSpecialization;
+  Info.Properties |= (SymbolPropertySet)SymbolProperty::Generic;
+  Info.Properties |=
+  (SymbolPropertySet)SymbolProperty::TemplatePartialSpecialization;
 } else if (isa(D)) {
-  Info.Properties |= (unsigned)SymbolProperty::Generic;
-  Info.Properties |= (unsigned)SymbolProperty::TemplateSpecialization;
+  Info.Properties |= (SymbolPropertySet)SymbolProperty::Generic;
+  Info.Properties |=
+  (SymbolPropertySet)SymbolProperty::TemplateSpecialization;
 }
 
   } else if (auto *VD = dyn_cast(D)) {
@@ -142,15 +144,17 @@ SymbolInfo index::getSymbolInfo(const De
 
 if (isa(D)) {
   Info.Lang = SymbolLanguage::CXX;
-  Info.Properties |= (un

[clang-tools-extra] r321412 - [clangd] Use Builder for symbol slabs, and use sorted-vector for storage

[Sam McCall via cfe-commits]

Author: sammccall
Date: Sat Dec 23 11:38:03 2017
New Revision: 321412

URL: http://llvm.org/viewvc/llvm-project?rev=321412&view=rev
Log:
[clangd] Use Builder for symbol slabs, and use sorted-vector for storage

Summary:
This improves a few things:
 - the insert -> freeze -> read sequence is now enforced/communicated by the
   type system
 - SymbolSlab::const_iterator iterates over symbols, not over id-symbol pairs
 - we avoid permanently storing a second copy of the IDs, and the
   string map's hashtable

The slab size is now down to 21.8MB for the LLVM project.
Of this only 2.7MB is strings, the rest is #symbols * `sizeof(Symbol)`.
`sizeof(Symbol)` is currently 96, which seems too big - I think
SymbolInfo isn't efficiently packed. That's a topic for another patch!

Also added simple API to see the memory usage/#symbols of a slab, since
it seems likely we will continue to care about this.

Reviewers: ilya-biryukov

Subscribers: klimek, mgrang, cfe-commits

Differential Revision: https://reviews.llvm.org/D41506

Modified:

clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
clang-tools-extra/trunk/clangd/index/FileIndex.cpp
clang-tools-extra/trunk/clangd/index/Index.cpp
clang-tools-extra/trunk/clangd/index/Index.h
clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp
clang-tools-extra/trunk/clangd/index/SymbolCollector.h
clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp
clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp
clang-tools-extra/trunk/unittests/clangd/FileIndexTests.cpp
clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp
clang-tools-extra/trunk/unittests/clangd/SymbolCollectorTests.cpp

Modified: 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp?rev=321412&r1=321411&r2=321412&view=diff
==
--- 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
 (original)
+++ 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
 Sat Dec 23 11:38:03 2017
@@ -80,16 +80,16 @@ int main(int argc, const char **argv) {
 
   // Found compilation database, we iterate all TUs from database to get all
   // symbols, and then merge them into a single SymbolSlab.
-  SymbolSlab GlobalSymbols;
+  SymbolSlab::Builder GlobalSymbols;
   std::mutex SymbolMutex;
   auto AddSymbols = [&](const SymbolSlab& NewSymbols) {
 // Synchronize set accesses.
 std::unique_lock LockGuard(SymbolMutex);
-for (auto It : NewSymbols) {
+for (auto Sym : NewSymbols) {
   // FIXME: Better handling the overlap symbols, currently we overwrite it
   // with the latest one, but we always want to good declarations (class
   // definitions, instead of forward declarations).
-  GlobalSymbols.insert(It.second);
+  GlobalSymbols.insert(Sym);
 }
   };
 
@@ -105,6 +105,6 @@ int main(int argc, const char **argv) {
 }
   }
 
-  llvm::outs() << SymbolToYAML(GlobalSymbols);
+  llvm::outs() << SymbolToYAML(std::move(GlobalSymbols).build());
   return 0;
 }

Modified: clang-tools-extra/trunk/clangd/index/FileIndex.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/FileIndex.cpp?rev=321412&r1=321411&r2=321412&view=diff
==
--- clang-tools-extra/trunk/clangd/index/FileIndex.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/FileIndex.cpp Sat Dec 23 11:38:03 2017
@@ -54,7 +54,7 @@ std::shared_ptrKeepAlive.push_back(FileAndSlab.second);
   for (const auto &Iter : *FileAndSlab.second)
-Snap->Pointers.push_back(&Iter.second);
+Snap->Pointers.push_back(&Iter);
 }
   }
   auto *Pointers = &Snap->Pointers;

Modified: clang-tools-extra/trunk/clangd/index/Index.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/Index.cpp?rev=321412&r1=321411&r2=321412&view=diff
==
--- clang-tools-extra/trunk/clangd/index/Index.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/Index.cpp Sat Dec 23 11:38:03 2017
@@ -13,16 +13,17 @@
 
 namespace clang {
 namespace clangd {
+using namespace llvm;
 
-SymbolID::SymbolID(llvm::StringRef USR)
-: HashValue(llvm::SHA1::hash(arrayRefFromStringRef(USR))) {}
+SymbolID::SymbolID(StringRef USR)
+: HashValue(SHA1::hash(arrayRefFromStringRef(USR))) {}
 
-llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, const SymbolID &ID) {
-  OS << toHex(llvm::toStringRef(ID.HashValue));
+raw_ostream &operator<<(raw_ostream &OS, const SymbolID &ID) {
+  OS << toHex(toStringRef(ID.HashValue));
   return OS;
 }
 
-void operator>>(llvm::StringRef Str, SymbolID &ID) {
+void operator>>(StringRef Str, SymbolID 

r321639 - [Driver] Fix unused variables and test-writing-into-workdir after r321621

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Jan  2 01:35:10 2018
New Revision: 321639

URL: http://llvm.org/viewvc/llvm-project?rev=321639&view=rev
Log:
[Driver] Fix unused variables and test-writing-into-workdir after r321621

Modified:
cfe/trunk/lib/Driver/Driver.cpp
cfe/trunk/test/Driver/config-file.c

Modified: cfe/trunk/lib/Driver/Driver.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/Driver.cpp?rev=321639&r1=321638&r2=321639&view=diff
==
--- cfe/trunk/lib/Driver/Driver.cpp (original)
+++ cfe/trunk/lib/Driver/Driver.cpp Tue Jan  2 01:35:10 2018
@@ -682,7 +682,7 @@ bool Driver::loadConfigFile() {
   CfgDir.append(
   CLOptions->getLastArgValue(options::OPT_config_system_dir_EQ));
   if (!CfgDir.empty()) {
-if (std::error_code EC = llvm::sys::fs::make_absolute(CfgDir))
+if (llvm::sys::fs::make_absolute(CfgDir).value() != 0)
   SystemConfigDir.clear();
 else
   SystemConfigDir = std::string(CfgDir.begin(), CfgDir.end());
@@ -693,7 +693,7 @@ bool Driver::loadConfigFile() {
   CfgDir.append(
   CLOptions->getLastArgValue(options::OPT_config_user_dir_EQ));
   if (!CfgDir.empty()) {
-if (std::error_code EC = llvm::sys::fs::make_absolute(CfgDir))
+if (llvm::sys::fs::make_absolute(CfgDir).value() != 0)
   UserConfigDir.clear();
 else
   UserConfigDir = std::string(CfgDir.begin(), CfgDir.end());

Modified: cfe/trunk/test/Driver/config-file.c
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Driver/config-file.c?rev=321639&r1=321638&r2=321639&view=diff
==
--- cfe/trunk/test/Driver/config-file.c (original)
+++ cfe/trunk/test/Driver/config-file.c Tue Jan  2 01:35:10 2018
@@ -1,6 +1,6 @@
 //--- Config file search directories
 //
-// RUN: %clang --config-system-dir=%S/Inputs/config 
--config-user-dir=%S/Inputs/config2 -v 2>&1 | FileCheck %s -check-prefix 
CHECK-DIRS
+// RUN: %clang --config-system-dir=%S/Inputs/config 
--config-user-dir=%S/Inputs/config2 -o /dev/null -v 2>&1 | FileCheck %s 
-check-prefix CHECK-DIRS
 // CHECK-DIRS: System configuration file directory: {{.*}}/Inputs/config
 // CHECK-DIRS: User configuration file directory: {{.*}}/Inputs/config2
 
@@ -15,7 +15,7 @@
 
 //--- Config file (full path) in output of -v
 //
-// RUN: %clang --config %S/Inputs/config-1.cfg -c %s -v 2>&1 | FileCheck %s 
-check-prefix CHECK-V
+// RUN: %clang --config %S/Inputs/config-1.cfg -c %s -o /dev/null -v 2>&1 | 
FileCheck %s -check-prefix CHECK-V
 // CHECK-V: Configuration file: {{.*}}Inputs{{.}}config-1.cfg
 // CHECK-V: -Werror
 // CHECK-V: -std=c99
@@ -31,7 +31,7 @@
 
 //--- Config file in output of -v
 //
-// RUN: %clang --config-system-dir=%S/Inputs --config-user-dir= --config 
config-1.cfg -c %s -v 2>&1 | FileCheck %s -check-prefix CHECK-V2
+// RUN: %clang --config-system-dir=%S/Inputs --config-user-dir= --config 
config-1.cfg -c %s -o /dev/null -v 2>&1 | FileCheck %s -check-prefix CHECK-V2
 // CHECK-V2: Configuration file: {{.*}}Inputs{{.}}config-1.cfg
 // CHECK-V2: -Werror
 // CHECK-V2: -std=c99
@@ -61,12 +61,12 @@
 
 //--- Unused options in config file do not produce warnings
 //
-// RUN: %clang --config %S/Inputs/config-4.cfg -c %s -v 2>&1 | FileCheck %s 
-check-prefix CHECK-UNUSED
+// RUN: %clang --config %S/Inputs/config-4.cfg -c %s -o /dev/null -v 2>&1 | 
FileCheck %s -check-prefix CHECK-UNUSED
 // CHECK-UNUSED-NOT: argument unused during compilation:
 
 
 //--- User directory is searched first.
 //
-// RUN: %clang --config-system-dir=%S/Inputs/config 
--config-user-dir=%S/Inputs/config2 --config config-4 -c %s -v 2>&1 | FileCheck 
%s -check-prefix CHECK-PRECEDENCE
+// RUN: %clang --config-system-dir=%S/Inputs/config 
--config-user-dir=%S/Inputs/config2 --config config-4 -c %s -o /dev/null -v 
2>&1 | FileCheck %s -check-prefix CHECK-PRECEDENCE
 // CHECK-PRECEDENCE: Configuration file: 
{{.*}}Inputs{{.}}config2{{.}}config-4.cfg
 // CHECK-PRECEDENCE: -Wall


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r321997 - Avoid assumption that lit tests are writable. NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Jan  8 07:05:01 2018
New Revision: 321997

URL: http://llvm.org/viewvc/llvm-project?rev=321997&view=rev
Log:
Avoid assumption that lit tests are writable. NFC

Modified:
cfe/trunk/test/ARCMT/releases-driver.m
cfe/trunk/test/ARCMT/releases-driver.m.result
cfe/trunk/test/ARCMT/with-arc-mode-modify.m
cfe/trunk/test/ARCMT/with-arc-mode-modify.m.result
cfe/trunk/test/PCH/verify_pch.m
cfe/trunk/test/VFS/real-path-found-first.m

Modified: cfe/trunk/test/ARCMT/releases-driver.m
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/ARCMT/releases-driver.m?rev=321997&r1=321996&r2=321997&view=diff
==
--- cfe/trunk/test/ARCMT/releases-driver.m (original)
+++ cfe/trunk/test/ARCMT/releases-driver.m Mon Jan  8 07:05:01 2018
@@ -1,5 +1,5 @@
 // RUN: %clang_cc1 -fblocks -fsyntax-only -fobjc-arc -x objective-c %s.result
-// RUN: cp %s %t
+// RUN: cat %s > %t
 // RUN: %clang_cc1 -arcmt-modify -triple x86_64-apple-macosx10.6 -x 
objective-c %t
 // RUN: diff %t %s.result
 // RUN: rm %t

Modified: cfe/trunk/test/ARCMT/releases-driver.m.result
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/ARCMT/releases-driver.m.result?rev=321997&r1=321996&r2=321997&view=diff
==
--- cfe/trunk/test/ARCMT/releases-driver.m.result (original)
+++ cfe/trunk/test/ARCMT/releases-driver.m.result Mon Jan  8 07:05:01 2018
@@ -1,5 +1,5 @@
 // RUN: %clang_cc1 -fblocks -fsyntax-only -fobjc-arc -x objective-c %s.result
-// RUN: cp %s %t
+// RUN: cat %s > %t
 // RUN: %clang_cc1 -arcmt-modify -triple x86_64-apple-macosx10.6 -x 
objective-c %t
 // RUN: diff %t %s.result
 // RUN: rm %t

Modified: cfe/trunk/test/ARCMT/with-arc-mode-modify.m
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/ARCMT/with-arc-mode-modify.m?rev=321997&r1=321996&r2=321997&view=diff
==
--- cfe/trunk/test/ARCMT/with-arc-mode-modify.m (original)
+++ cfe/trunk/test/ARCMT/with-arc-mode-modify.m Mon Jan  8 07:05:01 2018
@@ -1,5 +1,5 @@
 // RUN: %clang_cc1 -fsyntax-only -fobjc-arc -x objective-c %s.result
-// RUN: cp %s %t
+// RUN: cat %s > %t
 // RUN: %clang_cc1 -arcmt-modify -fsyntax-only -fobjc-arc -x objective-c %t
 // RUN: diff %t %s.result
 // RUN: rm %t

Modified: cfe/trunk/test/ARCMT/with-arc-mode-modify.m.result
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/ARCMT/with-arc-mode-modify.m.result?rev=321997&r1=321996&r2=321997&view=diff
==
--- cfe/trunk/test/ARCMT/with-arc-mode-modify.m.result (original)
+++ cfe/trunk/test/ARCMT/with-arc-mode-modify.m.result Mon Jan  8 07:05:01 2018
@@ -1,5 +1,5 @@
 // RUN: %clang_cc1 -fsyntax-only -fobjc-arc -x objective-c %s.result
-// RUN: cp %s %t
+// RUN: cat %s > %t
 // RUN: %clang_cc1 -arcmt-modify -fsyntax-only -fobjc-arc -x objective-c %t
 // RUN: diff %t %s.result
 // RUN: rm %t

Modified: cfe/trunk/test/PCH/verify_pch.m
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/PCH/verify_pch.m?rev=321997&r1=321996&r2=321997&view=diff
==
--- cfe/trunk/test/PCH/verify_pch.m (original)
+++ cfe/trunk/test/PCH/verify_pch.m Mon Jan  8 07:05:01 2018
@@ -2,7 +2,7 @@
 // RUN: rm -rf %t
 // RUN: mkdir -p %t/usr/include
 // RUN: echo '// empty' > %t/usr/include/sys_header.h
-// RUN: cp %s %t.h
+// RUN: cat %s > %t.h
 //
 // Precompile
 // RUN: %clang_cc1 -isystem %t/usr/include -x objective-c-header -emit-pch -o 
%t.pch %t.h

Modified: cfe/trunk/test/VFS/real-path-found-first.m
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/VFS/real-path-found-first.m?rev=321997&r1=321996&r2=321997&view=diff
==
--- cfe/trunk/test/VFS/real-path-found-first.m (original)
+++ cfe/trunk/test/VFS/real-path-found-first.m Mon Jan  8 07:05:01 2018
@@ -7,7 +7,7 @@
 // REQUIRES: shell
 // RUN: rm -rf %t %t-cache %t.pch
 // RUN: mkdir -p %t/SomeFramework.framework/Modules
-// RUN: cp %S/Inputs/some_frame_module.map 
%t/SomeFramework.framework/Modules/module.modulemap
+// RUN: cat %S/Inputs/some_frame_module.map > 
%t/SomeFramework.framework/Modules/module.modulemap
 // RUN: sed -e "s:INPUT_DIR:%S/Inputs:g" -e "s:OUT_DIR:%t:g" 
%S/Inputs/vfsoverlay.yaml > %t.yaml
 
 // Build


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r322001 - Avoid assumption that lit tests are writable. NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Jan  8 07:49:40 2018
New Revision: 322001

URL: http://llvm.org/viewvc/llvm-project?rev=322001&view=rev
Log:
Avoid assumption that lit tests are writable. NFC

Modified:
clang-tools-extra/trunk/test/clang-apply-replacements/crlf.cpp
clang-tools-extra/trunk/test/clang-move/move-function.cpp
clang-tools-extra/trunk/test/clang-move/no-move-macro-helpers.cpp

Modified: clang-tools-extra/trunk/test/clang-apply-replacements/crlf.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/clang-apply-replacements/crlf.cpp?rev=322001&r1=322000&r2=322001&view=diff
==
--- clang-tools-extra/trunk/test/clang-apply-replacements/crlf.cpp (original)
+++ clang-tools-extra/trunk/test/clang-apply-replacements/crlf.cpp Mon Jan  8 
07:49:40 2018
@@ -1,5 +1,5 @@
 // RUN: mkdir -p %T/Inputs/crlf
-// RUN: cp %S/Inputs/crlf/crlf.cpp %T/Inputs/crlf/crlf.cpp
+// RUN: cat %S/Inputs/crlf/crlf.cpp > %T/Inputs/crlf/crlf.cpp
 // RUN: sed "s#\$(path)#%/T/Inputs/crlf#" %S/Inputs/crlf/file1.yaml > 
%T/Inputs/crlf/file1.yaml
 // RUN: clang-apply-replacements %T/Inputs/crlf
 // RUN: diff %T/Inputs/crlf/crlf.cpp %S/Inputs/crlf/crlf.cpp.expected

Modified: clang-tools-extra/trunk/test/clang-move/move-function.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/clang-move/move-function.cpp?rev=322001&r1=322000&r2=322001&view=diff
==
--- clang-tools-extra/trunk/test/clang-move/move-function.cpp (original)
+++ clang-tools-extra/trunk/test/clang-move/move-function.cpp Mon Jan  8 
07:49:40 2018
@@ -1,5 +1,6 @@
 // RUN: mkdir -p %T/move-function
-// RUN: cp %S/Inputs/function_test*  %T/move-function
+// RUN: cat %S/Inputs/function_test.h > %T/move-function/function_test.h
+// RUN: cat %S/Inputs/function_test.cpp > %T/move-function/function_test.cpp
 // RUN: cd %T/move-function
 // RUN: clang-move -names="g" -new_header=%T/move-function/new_function_test.h 
-old_header=../move-function/function_test.h %T/move-function/function_test.cpp 
--
 // RUN: FileCheck -input-file=%T/move-function/new_function_test.h 
-check-prefix=CHECK-NEW-TEST-H-CASE1 %s
@@ -39,12 +40,14 @@
 // CHECK-NEW-TEST-CPP-CASE3: {{[[:space:]]+}}
 // CHECK-NEW-TEST-CPP-CASE3: void f() {}
 //
-// RUN: cp %S/Inputs/function_test*  %T/move-function
+// RUN: cat %S/Inputs/function_test.h > %T/move-function/function_test.h
+// RUN: cat %S/Inputs/function_test.cpp > %T/move-function/function_test.cpp
 // RUN: clang-move -names="A::f" 
-new_header=%T/move-function/new_function_test.h 
-new_cc=%T/move-function/new_function_test.cpp 
-old_header=../move-function/function_test.h 
-old_cc=../move-function/function_test.cpp %T/move-function/function_test.cpp 
-dump_result -- | FileCheck %s -check-prefix=CHECK-EMPTY
 //
 // CHECK-EMPTY: [{{[[:space:]]*}}]
 //
-// RUN: cp %S/Inputs/function_test*  %T/move-function
+// RUN: cat %S/Inputs/function_test.h > %T/move-function/function_test.h
+// RUN: cat %S/Inputs/function_test.cpp > %T/move-function/function_test.cpp
 // RUN: clang-move -names="f,A" 
-new_header=%T/move-function/new_function_test.h 
-new_cc=%T/move-function/new_function_test.cpp 
-old_header=../move-function/function_test.h 
-old_cc=../move-function/function_test.cpp %T/move-function/function_test.cpp --
 // RUN: FileCheck -input-file=%T/move-function/new_function_test.h 
-check-prefix=CHECK-NEW-TEST-H-CASE4 %s
 // RUN: FileCheck -input-file=%T/move-function/new_function_test.cpp 
-check-prefix=CHECK-NEW-TEST-CPP-CASE4 %s

Modified: clang-tools-extra/trunk/test/clang-move/no-move-macro-helpers.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/clang-move/no-move-macro-helpers.cpp?rev=322001&r1=322000&r2=322001&view=diff
==
--- clang-tools-extra/trunk/test/clang-move/no-move-macro-helpers.cpp (original)
+++ clang-tools-extra/trunk/test/clang-move/no-move-macro-helpers.cpp Mon Jan  
8 07:49:40 2018
@@ -1,6 +1,6 @@
 // RUN: mkdir -p %T/no-move-macro-helper
-// RUN: cp %S/Inputs/macro_helper_test.h  
%T/no-move-macro-helper/macro_helper_test.h
-// RUN: cp %S/Inputs/macro_helper_test.cpp 
%T/no-move-macro-helper/macro_helper_test.cpp
+// RUN: cat %S/Inputs/macro_helper_test.h > 
%T/no-move-macro-helper/macro_helper_test.h
+// RUN: cat %S/Inputs/macro_helper_test.cpp > 
%T/no-move-macro-helper/macro_helper_test.cpp
 // RUN: cd %T/no-move-macro-helper
 //
 // 
-
@@ -24,8 +24,8 @@
 // 
-
 // Test moving all.
 // 
-
-// RUN: cp %S/Inputs/macro_helper_test.h  
%T/no-move-macro-helper/macro_helper_test.h
-// RUN: cp %S/Inputs/macro_helper_test.cpp 

Re: r321997 - Avoid assumption that lit tests are writable. NFC

[Sam McCall via cfe-commits]

Hi David (and the list this time!),

If X is readonly, then after `cp X Y`, Y is also readonly. The `cat`
version doesn't propagate permissions.

The environment for lit tests isn't really spelled out, but relying on the
input files being +w doesn't seem obviously reasonable.
Google's internal runner puts them on a readonly filesystem, and has hacks
to deal with tests like this.
These hacks are causing me some pain, so I'm experimenting with removing
them. (Can give more details internally).

I don't have a great plan for keeping this from regressing - obviously only
the stuff that llvm-lit enforces can govern what gets checked in upstream.
That said, there are other cases (like writing outside of %T) where
google's runner is stricter, and fixing these upstream seems to be mostly
working.

Cheers, Sam

On Mon, Jan 8, 2018 at 5:05 PM, David Blaikie  wrote:

> I'm sure it's something obvious I don't understand here, but maybe someone
> else doesn't either & could benefit from it:
>
> What exactly does this change do? In what important way is "cp X Y"
> different from "cat X > Y"?
>
> On Mon, Jan 8, 2018 at 7:06 AM Sam McCall via cfe-commits <
> cfe-commits@lists.llvm.org> wrote:
>
>> Author: sammccall
>> Date: Mon Jan  8 07:05:01 2018
>> New Revision: 321997
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=321997&view=rev
>> Log:
>> Avoid assumption that lit tests are writable. NFC
>>
>> Modified:
>> cfe/trunk/test/ARCMT/releases-driver.m
>> cfe/trunk/test/ARCMT/releases-driver.m.result
>> cfe/trunk/test/ARCMT/with-arc-mode-modify.m
>> cfe/trunk/test/ARCMT/with-arc-mode-modify.m.result
>> cfe/trunk/test/PCH/verify_pch.m
>> cfe/trunk/test/VFS/real-path-found-first.m
>>
>> Modified: cfe/trunk/test/ARCMT/releases-driver.m
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/ARCMT/rel
>> eases-driver.m?rev=321997&r1=321996&r2=321997&view=diff
>> 
>> ==
>> --- cfe/trunk/test/ARCMT/releases-driver.m (original)
>> +++ cfe/trunk/test/ARCMT/releases-driver.m Mon Jan  8 07:05:01 2018
>> @@ -1,5 +1,5 @@
>>  // RUN: %clang_cc1 -fblocks -fsyntax-only -fobjc-arc -x objective-c
>> %s.result
>> -// RUN: cp %s %t
>> +// RUN: cat %s > %t
>>  // RUN: %clang_cc1 -arcmt-modify -triple x86_64-apple-macosx10.6 -x
>> objective-c %t
>>  // RUN: diff %t %s.result
>>  // RUN: rm %t
>>
>> Modified: cfe/trunk/test/ARCMT/releases-driver.m.result
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/ARCMT/rel
>> eases-driver.m.result?rev=321997&r1=321996&r2=321997&view=diff
>> 
>> ==
>> --- cfe/trunk/test/ARCMT/releases-driver.m.result (original)
>> +++ cfe/trunk/test/ARCMT/releases-driver.m.result Mon Jan  8 07:05:01
>> 2018
>> @@ -1,5 +1,5 @@
>>  // RUN: %clang_cc1 -fblocks -fsyntax-only -fobjc-arc -x objective-c
>> %s.result
>> -// RUN: cp %s %t
>> +// RUN: cat %s > %t
>>  // RUN: %clang_cc1 -arcmt-modify -triple x86_64-apple-macosx10.6 -x
>> objective-c %t
>>  // RUN: diff %t %s.result
>>  // RUN: rm %t
>>
>> Modified: cfe/trunk/test/ARCMT/with-arc-mode-modify.m
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/ARCMT/wit
>> h-arc-mode-modify.m?rev=321997&r1=321996&r2=321997&view=diff
>> 
>> ==
>> --- cfe/trunk/test/ARCMT/with-arc-mode-modify.m (original)
>> +++ cfe/trunk/test/ARCMT/with-arc-mode-modify.m Mon Jan  8 07:05:01 2018
>> @@ -1,5 +1,5 @@
>>  // RUN: %clang_cc1 -fsyntax-only -fobjc-arc -x objective-c %s.result
>> -// RUN: cp %s %t
>> +// RUN: cat %s > %t
>>  // RUN: %clang_cc1 -arcmt-modify -fsyntax-only -fobjc-arc -x objective-c
>> %t
>>  // RUN: diff %t %s.result
>>  // RUN: rm %t
>>
>> Modified: cfe/trunk/test/ARCMT/with-arc-mode-modify.m.result
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/ARCMT/wit
>> h-arc-mode-modify.m.result?rev=321997&r1=321996&r2=321997&view=diff
>> 
>> ==
>> --- cfe/trunk/test/ARCMT/with-arc-mode-modify.m.result (original)
>> +++ cfe/trunk/test/ARCMT/with-arc-mode-modify.m.result Mon Jan  8
>> 07:05:01 2018
>> @@ -1,5 +1,5 @@
>>  // RUN: %clang_cc1 -fsyntax-only -fobjc-arc -x objective-c %s.result
>> -// RUN

r322065 - Avoid assumption that lit tests are writable (in a couple more places). NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Jan  9 01:32:53 2018
New Revision: 322065

URL: http://llvm.org/viewvc/llvm-project?rev=322065&view=rev
Log:
Avoid assumption that lit tests are writable (in a couple more places). NFC

Modified:
cfe/trunk/test/Modules/modify-module.m
cfe/trunk/test/PCH/modified-header-crash.c

Modified: cfe/trunk/test/Modules/modify-module.m
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/modify-module.m?rev=322065&r1=322064&r2=322065&view=diff
==
--- cfe/trunk/test/Modules/modify-module.m (original)
+++ cfe/trunk/test/Modules/modify-module.m Tue Jan  9 01:32:53 2018
@@ -3,9 +3,9 @@
 
 // RUN: rm -rf %t
 // RUN: mkdir -p %t/include
-// RUN: cp %S/Inputs/Modified/A.h %t/include
-// RUN: cp %S/Inputs/Modified/B.h %t/include
-// RUN: cp %S/Inputs/Modified/module.map %t/include
+// RUN: cat %S/Inputs/Modified/A.h > %t/include/A.h
+// RUN: cat %S/Inputs/Modified/B.h > %t/include/B.h
+// RUN: cat %S/Inputs/Modified/module.map > %t/include/module.map
 // RUN: %clang_cc1 -fdisable-module-hash -fmodules-cache-path=%t/cache 
-fmodules -fimplicit-module-maps -I %t/include %s -verify
 // RUN: echo '' >> %t/include/B.h
 // RUN: %clang_cc1 -fdisable-module-hash -fmodules-cache-path=%t/cache 
-fmodules -fimplicit-module-maps -I %t/include %s -verify

Modified: cfe/trunk/test/PCH/modified-header-crash.c
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/PCH/modified-header-crash.c?rev=322065&r1=322064&r2=322065&view=diff
==
--- cfe/trunk/test/PCH/modified-header-crash.c (original)
+++ cfe/trunk/test/PCH/modified-header-crash.c Tue Jan  9 01:32:53 2018
@@ -1,6 +1,6 @@
 // Don't crash.
 
-// RUN: cp %S/modified-header-crash.h %t.h
+// RUN: cat %S/modified-header-crash.h > %t.h
 // RUN: %clang_cc1 -DCAKE -x c-header %t.h -emit-pch -o %t
 // RUN: echo 'int foobar;' >> %t.h
 // RUN: not %clang_cc1 %s -include-pch %t -fsyntax-only


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r326211 - [clangd] Remove codecomplete override content API. Long live addDocument!

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Feb 27 09:15:50 2018
New Revision: 326211

URL: http://llvm.org/viewvc/llvm-project?rev=326211&view=rev
Log:
[clangd] Remove codecomplete override content API. Long live addDocument!

Modified:
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.h
clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp
clang-tools-extra/trunk/unittests/clangd/SyncAPI.cpp
clang-tools-extra/trunk/unittests/clangd/SyncAPI.h

Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=326211&r1=326210&r2=326211&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Tue Feb 27 09:15:50 2018
@@ -141,7 +141,6 @@ void ClangdServer::forceReparse(PathRef
 void ClangdServer::codeComplete(
 PathRef File, Position Pos, const clangd::CodeCompleteOptions &Opts,
 UniqueFunction)> Callback,
-llvm::Optional OverridenContents,
 IntrusiveRefCntPtr *UsedFS) {
   using CallbackType = UniqueFunction)>;
 
@@ -154,14 +153,9 @@ void ClangdServer::codeComplete(
   if (!CodeCompleteOpts.Index) // Respect overridden index.
 CodeCompleteOpts.Index = Index;
 
-  std::string Contents;
-  if (OverridenContents) {
-Contents = OverridenContents->str();
-  } else {
-VersionedDraft Latest = DraftMgr.getDraft(File);
-assert(Latest.Draft && "codeComplete called for non-added document");
-Contents = *Latest.Draft;
-  }
+  VersionedDraft Latest = DraftMgr.getDraft(File);
+  // FIXME(sammccall): return error for consistency?
+  assert(Latest.Draft && "codeComplete called for non-added document");
 
   // Copy PCHs to avoid accessing this->PCHs concurrently
   std::shared_ptr PCHs = this->PCHs;
@@ -183,34 +177,27 @@ void ClangdServer::codeComplete(
 
   WorkScheduler.runWithPreamble(
   "CodeComplete", File,
-  Bind(Task, std::move(Contents), File.str(), std::move(Callback)));
+  Bind(Task, std::move(*Latest.Draft), File.str(), std::move(Callback)));
 }
 
 void ClangdServer::signatureHelp(
 PathRef File, Position Pos,
 UniqueFunction>)> Callback,
-llvm::Optional OverridenContents,
 IntrusiveRefCntPtr *UsedFS) {
   auto TaggedFS = FSProvider.getTaggedFileSystem(File);
   if (UsedFS)
 *UsedFS = TaggedFS.Value;
 
-  std::string Contents;
-  if (OverridenContents) {
-Contents = OverridenContents->str();
-  } else {
-VersionedDraft Latest = DraftMgr.getDraft(File);
-if (!Latest.Draft)
-  return Callback(llvm::make_error(
-  "signatureHelp is called for non-added document",
-  llvm::errc::invalid_argument));
-Contents = std::move(*Latest.Draft);
-  }
+  VersionedDraft Latest = DraftMgr.getDraft(File);
+  if (!Latest.Draft)
+return Callback(llvm::make_error(
+"signatureHelp is called for non-added document",
+llvm::errc::invalid_argument));
 
   auto PCHs = this->PCHs;
-  auto Action = [Contents, Pos, TaggedFS,
- PCHs](Path File, decltype(Callback) Callback,
-   llvm::Expected IP) {
+  auto Action = [Pos, TaggedFS, PCHs](std::string Contents, Path File,
+  decltype(Callback) Callback,
+  llvm::Expected IP) {
 if (!IP)
   return Callback(IP.takeError());
 
@@ -223,8 +210,9 @@ void ClangdServer::signatureHelp(
 TaggedFS.Tag));
   };
 
-  WorkScheduler.runWithPreamble("SignatureHelp", File,
-Bind(Action, File.str(), std::move(Callback)));
+  WorkScheduler.runWithPreamble(
+  "SignatureHelp", File,
+  Bind(Action, std::move(*Latest.Draft), File.str(), std::move(Callback)));
 }
 
 llvm::Expected

Modified: clang-tools-extra/trunk/clangd/ClangdServer.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.h?rev=326211&r1=326210&r2=326211&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.h (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.h Tue Feb 27 09:15:50 2018
@@ -171,10 +171,7 @@ public:
   /// Run code completion for \p File at \p Pos.
   /// Request is processed asynchronously.
   ///
-  /// If \p OverridenContents is not None, they will used only for code
-  /// completion, i.e. no diagnostics update will be scheduled and a draft for
-  /// \p File will not be updated. If \p OverridenContents is None, contents of
-  /// the current draft for \p File will be used. If \p UsedFS is non-null, it
+  /// The current draft for \p File will be used. If \p UsedFS is non-null, it
   /// will be overwritten by vfs::FileSystem used for completion.
   ///
   /// This method should only be called for currently tracked files. However, 
it
@@ -185,

[clang-tools-extra] r326546 - [clangd] Debounce streams of updates.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Fri Mar  2 00:56:37 2018
New Revision: 326546

URL: http://llvm.org/viewvc/llvm-project?rev=326546&view=rev
Log:
[clangd] Debounce streams of updates.

Summary:
Don't actually start building ASTs for new revisions until either:
- 500ms have passed since the last revision, or
- we actually need the revision for something (or to unblock the queue)

In practice, this avoids the "first keystroke results in diagnostics" problem.
This is kind of awkward to test, and the test is pretty bad.
It can be observed nicely by capturing a trace, though.

Reviewers: hokein, ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D43648

Modified:
clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.h
clang-tools-extra/trunk/clangd/TUScheduler.cpp
clang-tools-extra/trunk/clangd/TUScheduler.h
clang-tools-extra/trunk/clangd/Threading.cpp
clang-tools-extra/trunk/clangd/Threading.h
clang-tools-extra/trunk/unittests/clangd/TUSchedulerTests.cpp

Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=326546&r1=326545&r2=326546&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp Fri Mar  2 00:56:37 2018
@@ -405,7 +405,7 @@ ClangdLSPServer::ClangdLSPServer(JSONOut
 : Out(Out), CDB(std::move(CompileCommandsDir)), CCOpts(CCOpts),
   Server(CDB, /*DiagConsumer=*/*this, FSProvider, AsyncThreadsCount,
  StorePreamblesInMemory, BuildDynamicSymbolIndex, StaticIdx,
- ResourceDir) {}
+ ResourceDir, /*UpdateDebounce=*/std::chrono::milliseconds(500)) {}
 
 bool ClangdLSPServer::run(std::istream &In, JSONStreamStyle InputStyle) {
   assert(!IsDone && "Run was called before");

Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=326546&r1=326545&r2=326546&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Fri Mar  2 00:56:37 2018
@@ -76,7 +76,8 @@ ClangdServer::ClangdServer(GlobalCompila
unsigned AsyncThreadsCount,
bool StorePreamblesInMemory,
bool BuildDynamicSymbolIndex, SymbolIndex 
*StaticIdx,
-   llvm::Optional ResourceDir)
+   llvm::Optional ResourceDir,
+   std::chrono::steady_clock::duration UpdateDebounce)
 : CompileArgs(CDB,
   ResourceDir ? ResourceDir->str() : getStandardResourceDir()),
   DiagConsumer(DiagConsumer), FSProvider(FSProvider),
@@ -91,7 +92,8 @@ ClangdServer::ClangdServer(GlobalCompila
 FileIdx
 ? [this](PathRef Path,
  ParsedAST *AST) { FileIdx->update(Path, AST); 
}
-: ASTParsedCallback()) {
+: ASTParsedCallback(),
+UpdateDebounce) {
   if (FileIdx && StaticIdx) {
 MergedIndex = mergeIndex(FileIdx.get(), StaticIdx);
 Index = MergedIndex.get();

Modified: clang-tools-extra/trunk/clangd/ClangdServer.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.h?rev=326546&r1=326545&r2=326546&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.h (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.h Fri Mar  2 00:56:37 2018
@@ -125,6 +125,8 @@ public:
   /// \p DiagConsumer. Note that a callback to \p DiagConsumer happens on a
   /// worker thread. Therefore, instances of \p DiagConsumer must properly
   /// synchronize access to shared state.
+  /// UpdateDebounce determines how long to wait after a new version of the 
file
+  /// before starting to compute diagnostics.
   ///
   /// \p StorePreamblesInMemory defines whether the Preambles generated by
   /// clangd are stored in-memory or on disk.
@@ -135,13 +137,17 @@ public:
   ///
   /// If \p StaticIdx is set, ClangdServer uses the index for global code
   /// completion.
+  /// FIXME(sammccall): pull out an options struct.
   ClangdServer(GlobalCompilationDatabase &CDB,
DiagnosticsConsumer &DiagConsumer,
FileSystemProvider &FSProvider, unsigned AsyncThreadsCount,
bool StorePreamblesInMemory,
bool BuildDynamicSymbolIndex = false,
SymbolIndex *StaticIdx = nullptr,
-   llvm::Optional

[clang-tools-extra] r326719 - [clangd] Extract ClangdServer::Options struct.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Mar  5 09:28:54 2018
New Revision: 326719

URL: http://llvm.org/viewvc/llvm-project?rev=326719&view=rev
Log:
[clangd] Extract ClangdServer::Options struct.

Summary:
This subsumes most of the params to ClangdServer and ClangdLSPServer.
Adjacent changes:
 - tests use a consistent set of options, except when testing specific options
 - tests that previously used synchronous mode for convenience no longer do
 - added a runAddDocument helper to SyncAPIs to mitigate the extra code
 - rearranged main a bit to follow the structure of the options

Reviewers: ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D44088

Modified:
clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
clang-tools-extra/trunk/clangd/ClangdLSPServer.h
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.h
clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp
clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp
clang-tools-extra/trunk/unittests/clangd/SyncAPI.cpp
clang-tools-extra/trunk/unittests/clangd/SyncAPI.h
clang-tools-extra/trunk/unittests/clangd/XRefsTests.cpp

Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=326719&r1=326718&r2=326719&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp Mon Mar  5 09:28:54 2018
@@ -395,17 +395,12 @@ void ClangdLSPServer::onChangeConfigurat
   }
 }
 
-ClangdLSPServer::ClangdLSPServer(JSONOutput &Out, unsigned AsyncThreadsCount,
- bool StorePreamblesInMemory,
+ClangdLSPServer::ClangdLSPServer(JSONOutput &Out,
  const clangd::CodeCompleteOptions &CCOpts,
- llvm::Optional ResourceDir,
  llvm::Optional CompileCommandsDir,
- bool BuildDynamicSymbolIndex,
- SymbolIndex *StaticIdx)
+ const ClangdServer::Options &Opts)
 : Out(Out), CDB(std::move(CompileCommandsDir)), CCOpts(CCOpts),
-  Server(CDB, /*DiagConsumer=*/*this, FSProvider, AsyncThreadsCount,
- StorePreamblesInMemory, BuildDynamicSymbolIndex, StaticIdx,
- ResourceDir, /*UpdateDebounce=*/std::chrono::milliseconds(500)) {}
+  Server(CDB, FSProvider, /*DiagConsumer=*/*this, Opts) {}
 
 bool ClangdLSPServer::run(std::istream &In, JSONStreamStyle InputStyle) {
   assert(!IsDone && "Run was called before");

Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.h?rev=326719&r1=326718&r2=326719&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdLSPServer.h (original)
+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.h Mon Mar  5 09:28:54 2018
@@ -31,13 +31,9 @@ public:
   /// If \p CompileCommandsDir has a value, compile_commands.json will be
   /// loaded only from \p CompileCommandsDir. Otherwise, clangd will look
   /// for compile_commands.json in all parent directories of each file.
-  ClangdLSPServer(JSONOutput &Out, unsigned AsyncThreadsCount,
-  bool StorePreamblesInMemory,
-  const clangd::CodeCompleteOptions &CCOpts,
-  llvm::Optional ResourceDir,
+  ClangdLSPServer(JSONOutput &Out, const clangd::CodeCompleteOptions &CCOpts,
   llvm::Optional CompileCommandsDir,
-  bool BuildDynamicSymbolIndex,
-  SymbolIndex *StaticIdx = nullptr);
+  const ClangdServer::Options &Opts);
 
   /// Run LSP server loop, receiving input for it from \p In. \p In must be
   /// opened in binary mode. Output will be written using Out variable passed 
to

Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=326719&r1=326718&r2=326719&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Mon Mar  5 09:28:54 2018
@@ -70,37 +70,41 @@ RealFileSystemProvider::getTaggedFileSys
   return make_tagged(vfs::getRealFileSystem(), VFSTag());
 }
 
+ClangdServer::Options ClangdServer::optsForTest() {
+  ClangdServer::Options Opts;
+  Opts.UpdateDebounce = std::chrono::steady_clock::duration::zero(); // Faster!
+  Opts.StorePreamblesInMemory = true;
+  Opts.AsyncThreadsCount = 4; // Consistent!
+  return Opts;
+}

[clang-tools-extra] r326721 - [clangd] Fix unintentionally loose fuzzy matching, and the tests masking it.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Mar  5 09:34:33 2018
New Revision: 326721

URL: http://llvm.org/viewvc/llvm-project?rev=326721&view=rev
Log:
[clangd] Fix unintentionally loose fuzzy matching, and the tests masking it.

Summary:
The intent was that [ar] doesn't match "FooBar"; the first character must match
a Head character (hard requirement, not just a low score).
This matches VSCode, and was "tested" but the tests were defective.

The tests expected matches("FooBar") to fail for lack of a match. But instead
it fails because the string should be annotated - matches("FooB[ar]").
This patch makes matches("FooBar") ignore annotations, as was intended.

Fixing the code to reject weak matches for the first char causes problems:
-  [bre] no longer matches "HTMLBRElement".
   We allow matching against an uppercase char even if we don't think it's head.
   Only do this if there's at least one lowercase, to avoid triggering on MACROS
-  [print] no longer matches "sprintf".
   This is hard to fix without false positives (e.g. [int] vs "sprintf"])
   This patch leaves this case broken. A future patch will add a dictionary
   providing custom segmentation to common names from the standard library.

Fixed a couple of index tests that indirectly relied on broken fuzzy matching.
Added const in a couple of missing places for consistency with new code.

Subscribers: klimek, ilya-biryukov, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D44003

Modified:
clang-tools-extra/trunk/clangd/FuzzyMatch.cpp
clang-tools-extra/trunk/clangd/FuzzyMatch.h
clang-tools-extra/trunk/unittests/clangd/FuzzyMatchTests.cpp
clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp

Modified: clang-tools-extra/trunk/clangd/FuzzyMatch.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/FuzzyMatch.cpp?rev=326721&r1=326720&r2=326721&view=diff
==
--- clang-tools-extra/trunk/clangd/FuzzyMatch.cpp (original)
+++ clang-tools-extra/trunk/clangd/FuzzyMatch.cpp Mon Mar  5 09:34:33 2018
@@ -33,6 +33,8 @@
 //Legal if the characters match.
 //  - Moving down (consuming a pattern character) is never legal.
 //Never legal: all pattern characters must match something.
+// Characters are matched case-insensitively.
+// The first pattern character may only match the start of a word segment.
 //
 // The scoring is based on heuristics:
 //  - when matching a character, apply a bonus or penalty depending on the
@@ -74,13 +76,11 @@ static bool isAwful(int S) { return S <
 static constexpr int PerfectBonus = 3; // Perfect per-pattern-char score.
 
 FuzzyMatcher::FuzzyMatcher(StringRef Pattern)
-: PatN(std::min(MaxPat, Pattern.size())), CaseSensitive(false),
+: PatN(std::min(MaxPat, Pattern.size())),
   ScoreScale(PatN ? float{1} / (PerfectBonus * PatN) : 0), WordN(0) {
   std::copy(Pattern.begin(), Pattern.begin() + PatN, Pat);
-  for (int I = 0; I < PatN; ++I) {
+  for (int I = 0; I < PatN; ++I)
 LowPat[I] = lower(Pat[I]);
-CaseSensitive |= LowPat[I] != Pat[I];
-  }
   Scores[0][0][Miss] = {0, Miss};
   Scores[0][0][Match] = {AwfulScore, Miss};
   for (int P = 0; P <= PatN; ++P)
@@ -88,7 +88,7 @@ FuzzyMatcher::FuzzyMatcher(StringRef Pat
   for (Action A : {Miss, Match})
 Scores[P][W][A] = {AwfulScore, Miss};
   if (PatN > 0)
-calculateRoles(Pat, PatRole, PatN);
+calculateRoles(Pat, PatRole, PatTypeSet, PatN);
 }
 
 Optional FuzzyMatcher::match(StringRef Word) {
@@ -177,16 +177,21 @@ constexpr static uint8_t CharRoles[] = {
 template  static T packedLookup(const uint8_t *Data, int I) {
   return static_cast((Data[I >> 2] >> ((I & 3) * 2)) & 3);
 }
-void FuzzyMatcher::calculateRoles(const char *Text, CharRole *Out, int N) {
+void FuzzyMatcher::calculateRoles(const char *Text, CharRole *Out, int 
&TypeSet,
+  int N) {
   assert(N > 0);
+  CharType Type = packedLookup(CharTypes, Text[0]);
+  TypeSet = 1 << Type;
   // Types holds a sliding window of (Prev, Curr, Next) types.
   // Initial value is (Empty, Empty, type of Text[0]).
-  int Types = packedLookup(CharTypes, Text[0]);
+  int Types = Type;
   // Rotate slides in the type of the next character.
   auto Rotate = [&](CharType T) { Types = ((Types << 2) | T) & 0x3f; };
   for (int I = 0; I < N - 1; ++I) {
 // For each character, rotate in the next, and look up the role.
-Rotate(packedLookup(CharTypes, Text[I + 1]));
+Type = packedLookup(CharTypes, Text[I + 1]);
+TypeSet |= 1 << Type;
+Rotate(Type);
 *Out++ = packedLookup(CharRoles, Types);
   }
   // For the last character, the "next character" is Empty.
@@ -214,7 +219,10 @@ bool FuzzyMatcher::init(StringRef NewWor
   ++P;
   }
 
-  calculateRoles(Word, WordRole, WordN);
+  // FIXME: some words are hard to tokenize algorithmically.
+  // e.g. vsprintf is V S Print F, and should match [pri] but not [int].
+  // We cou

[clang-tools-extra] r326798 - [clangd] Address missed comments from D44003

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Mar  6 06:30:07 2018
New Revision: 326798

URL: http://llvm.org/viewvc/llvm-project?rev=326798&view=rev
Log:
[clangd] Address missed comments from D44003

Modified:
clang-tools-extra/trunk/unittests/clangd/FuzzyMatchTests.cpp

Modified: clang-tools-extra/trunk/unittests/clangd/FuzzyMatchTests.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/FuzzyMatchTests.cpp?rev=326798&r1=326797&r2=326798&view=diff
==
--- clang-tools-extra/trunk/unittests/clangd/FuzzyMatchTests.cpp (original)
+++ clang-tools-extra/trunk/unittests/clangd/FuzzyMatchTests.cpp Tue Mar  6 
06:30:07 2018
@@ -30,14 +30,16 @@ struct ExpectedMatch {
   bool accepts(StringRef ActualAnnotated) const {
 return !Annotated || ActualAnnotated == *Annotated;
   }
-  std::string Word;
 
   friend raw_ostream &operator<<(raw_ostream &OS, const ExpectedMatch &M) {
-return OS << "'" << M.Word;
+OS << "'" << M.Word;
 if (M.Annotated)
   OS << "' as " << *M.Annotated;
+return OS;
   }
 
+  std::string Word;
+
 private:
   Optional Annotated;
 };


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r327115 - [clang-tidy] fix header guard

[Sam McCall via cfe-commits]

Author: sammccall
Date: Fri Mar  9 03:47:37 2018
New Revision: 327115

URL: http://llvm.org/viewvc/llvm-project?rev=327115&view=rev
Log:
[clang-tidy] fix header guard

Modified:
clang-tools-extra/trunk/clang-tidy/abseil/StringFindStartswithCheck.h

Modified: clang-tools-extra/trunk/clang-tidy/abseil/StringFindStartswithCheck.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clang-tidy/abseil/StringFindStartswithCheck.h?rev=327115&r1=327114&r2=327115&view=diff
==
--- clang-tools-extra/trunk/clang-tidy/abseil/StringFindStartswithCheck.h 
(original)
+++ clang-tools-extra/trunk/clang-tidy/abseil/StringFindStartswithCheck.h Fri 
Mar  9 03:47:37 2018
@@ -7,8 +7,8 @@
 //
 
//===--===//
 
-#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANG_TIDY_ABSEIL_STRING_FIND_STARTSWITH_H
-#define LLVM_CLANG_TOOLS_EXTRA_CLANG_TIDY_ABSEIL_STRING_FIND_STARTSWITH_H_
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANG_TIDY_ABSEIL_STRINGFINDSTARTSWITHCHECK_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANG_TIDY_ABSEIL_STRINGFINDSTARTSWITHCHECK_H
 
 #include "../ClangTidy.h"
 #include "../utils/IncludeInserter.h"
@@ -45,4 +45,4 @@ private:
 } // namespace tidy
 } // namespace clang
 
-#endif // LLVM_CLANG_TOOLS_EXTRA_CLANG_TIDY_ABSEIL_STRING_FIND_STARTSWITH_H_
+#endif // LLVM_CLANG_TOOLS_EXTRA_CLANG_TIDY_ABSEIL_STRINGFINDSTARTSWITHCHECK_H


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r327127 - [clangd] Don't index template specializations.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Fri Mar  9 05:25:29 2018
New Revision: 327127

URL: http://llvm.org/viewvc/llvm-project?rev=327127&view=rev
Log:
[clangd] Don't index template specializations.

Summary:
These have different USRs than the underlying entity, but are not typically
interesting in their own right and can be numerous (e.g. generated traits).

Reviewers: ioeric

Subscribers: klimek, ilya-biryukov, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D44298

Modified:
clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp
clang-tools-extra/trunk/unittests/clangd/SymbolCollectorTests.cpp

Modified: clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp?rev=327127&r1=327126&r2=327127&view=diff
==
--- clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp Fri Mar  9 
05:25:29 2018
@@ -115,10 +115,16 @@ bool shouldFilterDecl(const NamedDecl *N
   //   * enum constants in unscoped enum decl (e.g. "red" in "enum {red};")
   auto InTopLevelScope = hasDeclContext(
   anyOf(namespaceDecl(), translationUnitDecl(), linkageSpecDecl()));
+  // Don't index template specializations.
+  auto IsSpecialization =
+  anyOf(functionDecl(isExplicitTemplateSpecialization()),
+cxxRecordDecl(isExplicitTemplateSpecialization()),
+varDecl(isExplicitTemplateSpecialization()));
   if (match(decl(allOf(unless(isExpansionInMainFile()),
anyOf(InTopLevelScope,
  hasDeclContext(enumDecl(InTopLevelScope,
- unless(isScoped())),
+ unless(isScoped(),
+   unless(IsSpecialization))),
 *ND, *ASTCtx)
   .empty())
 return true;

Modified: clang-tools-extra/trunk/unittests/clangd/SymbolCollectorTests.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/SymbolCollectorTests.cpp?rev=327127&r1=327126&r2=327127&view=diff
==
--- clang-tools-extra/trunk/unittests/clangd/SymbolCollectorTests.cpp (original)
+++ clang-tools-extra/trunk/unittests/clangd/SymbolCollectorTests.cpp Fri Mar  
9 05:25:29 2018
@@ -198,6 +198,19 @@ TEST_F(SymbolCollectorTest, CollectSymbo
QName("foo::bar::v2"), QName("foo::baz")}));
 }
 
+TEST_F(SymbolCollectorTest, Template) {
+  Annotations Header(R"(
+// Template is indexed, specialization and instantiation is not.
+template  struct [[Tmpl]] {T x = 0};
+template <> struct Tmpl {};
+extern template struct Tmpl;
+template struct Tmpl;
+  )");
+  runSymbolCollector(Header.code(), /*Main=*/"");
+  EXPECT_THAT(Symbols, UnorderedElementsAreArray({AllOf(
+   QName("Tmpl"), DeclRange(Header.offsetRange()))}));
+}
+
 TEST_F(SymbolCollectorTest, Locations) {
   Annotations Header(R"cpp(
 // Declared in header, defined in main.


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r327275 - [clangd] Collect the number of files referencing a symbol in the static index.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Mar 12 07:49:09 2018
New Revision: 327275

URL: http://llvm.org/viewvc/llvm-project?rev=327275&view=rev
Log:
[clangd] Collect the number of files referencing a symbol in the static index.

Summary:
This is an important ranking signal.
It's off for the dynamic index for now. Correspondingly, tell the index
infrastructure only to report declarations for the dynamic index.

Reviewers: ioeric, hokein

Subscribers: klimek, ilya-biryukov, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D44315

Modified:

clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
clang-tools-extra/trunk/clangd/index/FileIndex.cpp
clang-tools-extra/trunk/clangd/index/Index.h
clang-tools-extra/trunk/clangd/index/Merge.cpp
clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp
clang-tools-extra/trunk/clangd/index/SymbolCollector.h
clang-tools-extra/trunk/clangd/index/SymbolYAML.cpp
clang-tools-extra/trunk/unittests/clangd/IndexTests.cpp
clang-tools-extra/trunk/unittests/clangd/SymbolCollectorTests.cpp

Modified: 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp?rev=327275&r1=327274&r2=327275&view=diff
==
--- 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
 (original)
+++ 
clang-tools-extra/trunk/clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp
 Mon Mar 12 07:49:09 2018
@@ -99,6 +99,7 @@ public:
 auto CollectorOpts = SymbolCollector::Options();
 CollectorOpts.FallbackDir = AssumedHeaderDir;
 CollectorOpts.CollectIncludePath = true;
+CollectorOpts.CountReferences = true;
 auto Includes = llvm::make_unique();
 addSystemHeadersMapping(Includes.get());
 CollectorOpts.Includes = Includes.get();

Modified: clang-tools-extra/trunk/clangd/index/FileIndex.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/FileIndex.cpp?rev=327275&r1=327274&r2=327275&view=diff
==
--- clang-tools-extra/trunk/clangd/index/FileIndex.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/FileIndex.cpp Mon Mar 12 07:49:09 2018
@@ -26,12 +26,14 @@ std::unique_ptr indexAST(AST
   // AST at this point, but we also need preprocessor callbacks (e.g.
   // CommentHandler for IWYU pragma) to canonicalize includes.
   CollectorOpts.CollectIncludePath = false;
+  CollectorOpts.CountReferences = false;
 
   auto Collector = std::make_shared(std::move(CollectorOpts));
   Collector->setPreprocessor(std::move(PP));
   index::IndexingOptions IndexOpts;
+  // We only need declarations, because we don't count references.
   IndexOpts.SystemSymbolFilter =
-  index::IndexingOptions::SystemSymbolFilterKind::All;
+  index::IndexingOptions::SystemSymbolFilterKind::DeclarationsOnly;
   IndexOpts.IndexFunctionLocals = false;
 
   index::indexTopLevelDecls(Ctx, Decls, Collector, IndexOpts);

Modified: clang-tools-extra/trunk/clangd/index/Index.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/Index.h?rev=327275&r1=327274&r2=327275&view=diff
==
--- clang-tools-extra/trunk/clangd/index/Index.h (original)
+++ clang-tools-extra/trunk/clangd/index/Index.h Mon Mar 12 07:49:09 2018
@@ -131,6 +131,9 @@ struct Symbol {
   //   * For non-inline functions, the canonical declaration typically appears
   // in the ".h" file corresponding to the definition.
   SymbolLocation CanonicalDeclaration;
+  // The number of translation units that reference this symbol from their main
+  // file. This number is only meaningful if aggregated in an index.
+  unsigned References = 0;
 
   /// A brief description of the symbol that can be displayed in the completion
   /// candidate list. For example, "Foo(X x, Y y) const" is a labal for a

Modified: clang-tools-extra/trunk/clangd/index/Merge.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/index/Merge.cpp?rev=327275&r1=327274&r2=327275&view=diff
==
--- clang-tools-extra/trunk/clangd/index/Merge.cpp (original)
+++ clang-tools-extra/trunk/clangd/index/Merge.cpp Mon Mar 12 07:49:09 2018
@@ -73,6 +73,7 @@ mergeSymbol(const Symbol &L, const Symbo
 S.Definition = O.Definition;
   if (!S.CanonicalDeclaration)
 S.CanonicalDeclaration = O.CanonicalDeclaration;
+  S.References += O.References;
   if (S.CompletionLabel == "")
 S.CompletionLabel = O.CompletionLabel;
   if (S.CompletionFilterText == "")

Modified: clang-tools-extra/trunk/clangd/index/SymbolCollector.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/t

[clang-tools-extra] r327344 - [clangd] Remove Tagged and some related APIs from ClangdServer.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Mar 12 16:22:35 2018
New Revision: 327344

URL: http://llvm.org/viewvc/llvm-project?rev=327344&view=rev
Log:
[clangd] Remove Tagged and some related APIs from ClangdServer.

Context can do what Tagged was intended to support (snapshot filesystems),
and less intrusively.
getTaggedFileSystem() no longer needs a filename.

Cleanups while here:
 - code-complete now returns errors as Expected, like other functions
 - added an alias Callback for the usual callback function type

Modified:
clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
clang-tools-extra/trunk/clangd/ClangdLSPServer.h
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.h
clang-tools-extra/trunk/clangd/Function.h
clang-tools-extra/trunk/clangd/TUScheduler.h
clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp
clang-tools-extra/trunk/unittests/clangd/CodeCompleteTests.cpp
clang-tools-extra/trunk/unittests/clangd/HeadersTests.cpp
clang-tools-extra/trunk/unittests/clangd/SyncAPI.cpp
clang-tools-extra/trunk/unittests/clangd/SyncAPI.h
clang-tools-extra/trunk/unittests/clangd/TestFS.cpp
clang-tools-extra/trunk/unittests/clangd/TestFS.h
clang-tools-extra/trunk/unittests/clangd/XRefsTests.cpp

Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=327344&r1=327343&r2=327344&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp Mon Mar 12 16:22:35 2018
@@ -330,28 +330,33 @@ void ClangdLSPServer::onCodeAction(CodeA
 
 void ClangdLSPServer::onCompletion(TextDocumentPositionParams &Params) {
   Server.codeComplete(Params.textDocument.uri.file(), Params.position, CCOpts,
-  [](Tagged List) { reply(List.Value); });
+  [](llvm::Expected List) {
+if (!List)
+  return replyError(ErrorCode::InvalidParams,
+llvm::toString(List.takeError()));
+reply(*List);
+  });
 }
 
 void ClangdLSPServer::onSignatureHelp(TextDocumentPositionParams &Params) {
   Server.signatureHelp(Params.textDocument.uri.file(), Params.position,
-   [](llvm::Expected> SignatureHelp) 
{
+   [](llvm::Expected SignatureHelp) {
  if (!SignatureHelp)
return replyError(
ErrorCode::InvalidParams,
llvm::toString(SignatureHelp.takeError()));
- reply(SignatureHelp->Value);
+ reply(*SignatureHelp);
});
 }
 
 void ClangdLSPServer::onGoToDefinition(TextDocumentPositionParams &Params) {
   Server.findDefinitions(
   Params.textDocument.uri.file(), Params.position,
-  [](llvm::Expected>> Items) {
+  [](llvm::Expected> Items) {
 if (!Items)
   return replyError(ErrorCode::InvalidParams,
 llvm::toString(Items.takeError()));
-reply(json::ary(Items->Value));
+reply(json::ary(*Items));
   });
 }
 
@@ -363,24 +368,24 @@ void ClangdLSPServer::onSwitchSourceHead
 void ClangdLSPServer::onDocumentHighlight(TextDocumentPositionParams &Params) {
   Server.findDocumentHighlights(
   Params.textDocument.uri.file(), Params.position,
-  [](llvm::Expected>> Highlights) {
+  [](llvm::Expected> Highlights) {
 if (!Highlights)
   return replyError(ErrorCode::InternalError,
 llvm::toString(Highlights.takeError()));
-reply(json::ary(Highlights->Value));
+reply(json::ary(*Highlights));
   });
 }
 
 void ClangdLSPServer::onHover(TextDocumentPositionParams &Params) {
   Server.findHover(Params.textDocument.uri.file(), Params.position,
-   [](llvm::Expected> H) {
+   [](llvm::Expected H) {
  if (!H) {
replyError(ErrorCode::InternalError,
   llvm::toString(H.takeError()));
return;
  }
 
- reply(H->Value);
+ reply(*H);
});
 }
 
@@ -437,12 +442,12 @@ std::vector ClangdLSPServer::getFix
   return FixItsIter->second;
 }
 
-void ClangdLSPServer::onDiagnosticsReady(
-PathRef File, Tagged> Diagnostics) {
+void ClangdLSPServer::onDiagnosticsReady(PathRef File,
+ std::vector Diagnostics) {
   json::ary DiagnosticsJSON;
 
   DiagnosticToReplacementMap LocalFixIts; // Temporary storage
-  for (auto &Diag : Diagnostics.Value) {
+  for (auto &Diag : Diagnostics) {
 to

Re: [PATCH] D44462: [clangd] Don't use DraftMgr in implementation of forceReparse.

[Sam McCall via cfe-commits]

On Wed, Mar 14, 2018, 19:39 Simon Marchi via Phabricator <
revi...@reviews.llvm.org> wrote:

> simark added inline comments.
>
>
> 
> Comment at: clangd/TUScheduler.h:69
> +  /// FIXME: remove the callback from this function
> +  void updateCompileCommand(PathRef File, tooling::CompileCommand
> NewCommand,
> +IntrusiveRefCntPtr FS,
> 
> ilya-biryukov wrote:
> > sammccall wrote:
> > > simark wrote:
> > > > sammccall wrote:
> > > > > (summarizing offline discussion)
> > > > >
> > > > > this is so close to `update`, it'd be nice if we could just call
> `update` instead.
> > > > >
> > > > > For that we need the contents, so forceReparse needs contents,
> so... can forceReparse just be addDocument(skipCache=true) or something?
> > > > >
> > > > > Requiring content to be passed doesn't seem like a big burden in
> practice, and makes it clear that clangdserver is never responsible for
> maintaining copies of the content on the callers behalf (and
> clangdlspserver is).
> > > > >
> > > > > reparseOpenFiles needs to move to clangdlspserver, but this seems
> consistent with the design. (so I think we can drop getTrackedFiles?)
> > > > I also thought it would be nice to have only one method `update`.
> What about if the `Contents` member of `ParseInputs` is optional?  When it
> is not instantiated (such as when calling `forceReparse`), it would mean to
> re-use the previously sent source.
> > > That would also work. If we can get away with just requiring the
> contents to always be passed in, I think it's simpler to understand.
> > > That would also work. If we can get away with just requiring the
> contents to always be passed in, I think it's simpler to understand.
> > Also makes the implementation of `update` a bit simpler, because it
> doesn't have to care about missing files.
> > And `ParseInputs` is used in other places too, all of them need to be
> updated or we need two versions of `ParseInputs`: with optional and
> required contents.
> > Requiring contents for `forceReparse` seems easier. And when we got
> automated tracking of changes to compilation database, this will go away
> Do you mean `forceReparse` will go away?  Won't it still be required for
> when the user manually changes the compilation database path?
>
Tracking changes to files and compile commands means revisiting the
abstractions around build systems and file systems. We'll need some way to
deal with that case, but it probably shouldn't require lspserver to
explicitly tell clangdserver to rebuild.

While on this subject though, I never understood why the "move compilation
database" extension was added instead of just restarting clangd - what's
usefully preserved?

>
>
> Repository:
>   rL LLVM
>
> https://reviews.llvm.org/D44462
>
>
>
>
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r333547 - Revert "Update NRVO logic to support early return"

[Sam McCall via cfe-commits]

Author: sammccall
Date: Wed May 30 07:14:58 2018
New Revision: 333547

URL: http://llvm.org/viewvc/llvm-project?rev=333547&view=rev
Log:
Revert "Update NRVO logic to support early return"

This reverts commit r333500, which causes stage2 compiler crashes.

Removed:
cfe/trunk/test/CodeGenCXX/nrvo-noopt.cpp
cfe/trunk/test/SemaCXX/nrvo-ast.cpp
Modified:
cfe/trunk/include/clang/AST/Decl.h
cfe/trunk/include/clang/Sema/Scope.h
cfe/trunk/lib/Sema/Scope.cpp
cfe/trunk/lib/Sema/SemaDecl.cpp
cfe/trunk/lib/Sema/SemaExpr.cpp
cfe/trunk/lib/Sema/SemaStmt.cpp
cfe/trunk/lib/Serialization/ASTReaderDecl.cpp
cfe/trunk/lib/Serialization/ASTWriterDecl.cpp
cfe/trunk/test/CodeGenCXX/nrvo.cpp

Modified: cfe/trunk/include/clang/AST/Decl.h
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/AST/Decl.h?rev=333547&r1=333546&r2=333547&view=diff
==
--- cfe/trunk/include/clang/AST/Decl.h (original)
+++ cfe/trunk/include/clang/AST/Decl.h Wed May 30 07:14:58 2018
@@ -879,12 +879,6 @@ protected:
 DAK_Normal
   };
 
-  enum NRVOMode {
-NRVO_Candidate,
-NRVO_Disabled,
-NRVO_Enabled,
-  };
-
   class ParmVarDeclBitfields {
 friend class ASTDeclReader;
 friend class ParmVarDecl;
@@ -937,7 +931,7 @@ protected:
 /// Whether this local variable could be allocated in the return
 /// slot of its function, enabling the named return value optimization
 /// (NRVO).
-unsigned NRVOMode : 2;
+unsigned NRVOVariable : 1;
 
 /// Whether this variable is the for-range-declaration in a C++0x
 /// for-range statement.
@@ -1325,20 +1319,12 @@ public:
   /// return slot when returning from the function. Within the function body,
   /// each return that returns the NRVO object will have this variable as its
   /// NRVO candidate.
-  NRVOMode getNRVOMode() const {
-if (isa(this))
-  return NRVO_Disabled;
-return static_cast(NonParmVarDeclBits.NRVOMode);
-  }
-  bool isNRVOCandidate() const {
-return isa(this) ? false : NonParmVarDeclBits.NRVOMode == 
NRVO_Candidate;
-  }
   bool isNRVOVariable() const {
-return isa(this) ? false : NonParmVarDeclBits.NRVOMode == 
NRVO_Enabled;
+return isa(this) ? false : NonParmVarDeclBits.NRVOVariable;
   }
   void setNRVOVariable(bool NRVO) {
 assert(!isa(this));
-NonParmVarDeclBits.NRVOMode = NRVO ? NRVO_Enabled : NRVO_Disabled;
+NonParmVarDeclBits.NRVOVariable = NRVO;
   }
 
   /// Determine whether this variable is the for-range-declaration in

Modified: cfe/trunk/include/clang/Sema/Scope.h
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Sema/Scope.h?rev=333547&r1=333546&r2=333547&view=diff
==
--- cfe/trunk/include/clang/Sema/Scope.h (original)
+++ cfe/trunk/include/clang/Sema/Scope.h Wed May 30 07:14:58 2018
@@ -201,6 +201,10 @@ private:
   /// Used to determine if errors occurred in this scope.
   DiagnosticErrorTrap ErrorTrap;
 
+  /// A lattice consisting of undefined, a single NRVO candidate variable in
+  /// this scope, or over-defined. The bit is true when over-defined.
+  llvm::PointerIntPair NRVO;
+
   void setFlags(Scope *Parent, unsigned F);
 
 public:
@@ -462,7 +466,23 @@ public:
   UsingDirectives.end());
   }
 
-  void setNRVOCandidate(VarDecl *Candidate);
+  void addNRVOCandidate(VarDecl *VD) {
+if (NRVO.getInt())
+  return;
+if (NRVO.getPointer() == nullptr) {
+  NRVO.setPointer(VD);
+  return;
+}
+if (NRVO.getPointer() != VD)
+  setNoNRVO();
+  }
+
+  void setNoNRVO() {
+NRVO.setInt(true);
+NRVO.setPointer(nullptr);
+  }
+
+  void mergeNRVOIntoParent();
 
   /// Init - This is used by the parser to implement scope caching.
   void Init(Scope *parent, unsigned flags);

Modified: cfe/trunk/lib/Sema/Scope.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/Scope.cpp?rev=333547&r1=333546&r2=333547&view=diff
==
--- cfe/trunk/lib/Sema/Scope.cpp (original)
+++ cfe/trunk/lib/Sema/Scope.cpp Wed May 30 07:14:58 2018
@@ -92,6 +92,7 @@ void Scope::Init(Scope *parent, unsigned
   UsingDirectives.clear();
   Entity = nullptr;
   ErrorTrap.reset();
+  NRVO.setPointerAndInt(nullptr, 0);
 }
 
 bool Scope::containedInPrototypeScope() const {
@@ -118,15 +119,19 @@ void Scope::AddFlags(unsigned FlagsToSet
   Flags |= FlagsToSet;
 }
 
-void Scope::setNRVOCandidate(VarDecl *Candidate) {
-  for (Decl *D : DeclsInScope) {
-VarDecl *VD = dyn_cast(D);
-if (VD && VD != Candidate && VD->isNRVOCandidate())
-  VD->setNRVOVariable(false);
+void Scope::mergeNRVOIntoParent() {
+  if (VarDecl *Candidate = NRVO.getPointer()) {
+if (isDeclScope(Candidate))
+  Candidate->setNRVOVariable(true);
   }
 
-  if (Scope *parent = getParent())
-parent->setNRVOCan

[clang-tools-extra] r333881 - [clangd] Hover should return null when not hovering over anything.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Jun  4 03:37:16 2018
New Revision: 333881

URL: http://llvm.org/viewvc/llvm-project?rev=333881&view=rev
Log:
[clangd] Hover should return null when not hovering over anything.

Summary: Also made JSON serialize Optional to simplify this.

Reviewers: ioeric

Subscribers: klimek, ilya-biryukov, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47701

Modified:
clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.h
clang-tools-extra/trunk/clangd/JSONExpr.h
clang-tools-extra/trunk/clangd/XRefs.cpp
clang-tools-extra/trunk/clangd/XRefs.h
clang-tools-extra/trunk/test/clangd/hover.test
clang-tools-extra/trunk/unittests/clangd/JSONExprTests.cpp
clang-tools-extra/trunk/unittests/clangd/XRefsTests.cpp

Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=333881&r1=333880&r2=333881&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp Mon Jun  4 03:37:16 2018
@@ -365,7 +365,7 @@ void ClangdLSPServer::onDocumentHighligh
 
 void ClangdLSPServer::onHover(TextDocumentPositionParams &Params) {
   Server.findHover(Params.textDocument.uri.file(), Params.position,
-   [](llvm::Expected H) {
+   [](llvm::Expected> H) {
  if (!H) {
replyError(ErrorCode::InternalError,
   llvm::toString(H.takeError()));

Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=333881&r1=333880&r2=333881&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Mon Jun  4 03:37:16 2018
@@ -403,9 +403,10 @@ void ClangdServer::findDocumentHighlight
   WorkScheduler.runWithAST("Highlights", File, Bind(Action, std::move(CB)));
 }
 
-void ClangdServer::findHover(PathRef File, Position Pos, Callback CB) {
+void ClangdServer::findHover(PathRef File, Position Pos,
+ Callback> CB) {
   auto FS = FSProvider.getFileSystem();
-  auto Action = [Pos, FS](Callback CB,
+  auto Action = [Pos, FS](Callback> CB,
   llvm::Expected InpAST) {
 if (!InpAST)
   return CB(InpAST.takeError());

Modified: clang-tools-extra/trunk/clangd/ClangdServer.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.h?rev=333881&r1=333880&r2=333881&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.h (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.h Mon Jun  4 03:37:16 2018
@@ -158,7 +158,8 @@ public:
   Callback> CB);
 
   /// Get code hover for a given position.
-  void findHover(PathRef File, Position Pos, Callback CB);
+  void findHover(PathRef File, Position Pos,
+ Callback> CB);
 
   /// Retrieve the top symbols from the workspace matching a query.
   void workspaceSymbols(StringRef Query, int Limit,

Modified: clang-tools-extra/trunk/clangd/JSONExpr.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/JSONExpr.h?rev=333881&r1=333880&r2=333881&view=diff
==
--- clang-tools-extra/trunk/clangd/JSONExpr.h (original)
+++ clang-tools-extra/trunk/clangd/JSONExpr.h Mon Jun  4 03:37:16 2018
@@ -22,6 +22,8 @@
 namespace clang {
 namespace clangd {
 namespace json {
+class Expr;
+template  Expr toJSON(const llvm::Optional &Opt);
 
 // An Expr is an JSON value of unknown type.
 // They can be copied, but should generally be moved.
@@ -516,6 +518,11 @@ bool fromJSON(const json::Expr &E, std::
   return false;
 }
 
+template 
+json::Expr toJSON(const llvm::Optional& Opt) {
+  return Opt ? json::Expr(*Opt) : json::Expr(nullptr);
+}
+
 // Helper for mapping JSON objects onto protocol structs.
 // See file header for example.
 class ObjectMapper {

Modified: clang-tools-extra/trunk/clangd/XRefs.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/XRefs.cpp?rev=333881&r1=333880&r2=333881&view=diff
==
--- clang-tools-extra/trunk/clangd/XRefs.cpp (original)
+++ clang-tools-extra/trunk/clangd/XRefs.cpp Mon Jun  4 03:37:16 2018
@@ -526,7 +526,7 @@ static Hover getHoverContents(StringRef
   return H;
 }
 
-Hover getHover(ParsedAST &AST, Position Pos) {
+Optional getHover(ParsedAST &AST, Position Pos) 

[clang-tools-extra] r333897 - [clangd] Fix inverted test (--gtest_filter strikes again...)

[Sam McCall via cfe-commits]

Author: sammccall
Date: Mon Jun  4 06:28:17 2018
New Revision: 333897

URL: http://llvm.org/viewvc/llvm-project?rev=333897&view=rev
Log:
[clangd] Fix inverted test (--gtest_filter strikes again...)

Modified:
clang-tools-extra/trunk/unittests/clangd/XRefsTests.cpp

Modified: clang-tools-extra/trunk/unittests/clangd/XRefsTests.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/XRefsTests.cpp?rev=333897&r1=333896&r2=333897&view=diff
==
--- clang-tools-extra/trunk/unittests/clangd/XRefsTests.cpp (original)
+++ clang-tools-extra/trunk/unittests/clangd/XRefsTests.cpp Mon Jun  4 06:28:17 
2018
@@ -643,7 +643,7 @@ TEST(Hover, All) {
 Annotations T(Test.Input);
 auto AST = TestTU::withCode(T.code()).build();
 if (auto H = getHover(AST, T.point())) {
-  EXPECT_EQ("", Test.ExpectedHover) << Test.Input;
+  EXPECT_NE("", Test.ExpectedHover) << Test.Input;
   EXPECT_EQ(H->contents.value, Test.ExpectedHover) << Test.Input;
 } else
   EXPECT_EQ("", Test.ExpectedHover) << Test.Input;


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] r333993 - [clangd] Rewrite JSON dispatcher loop using C IO (FILE*) instead of std::istream.

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Jun  5 02:34:46 2018
New Revision: 333993

URL: http://llvm.org/viewvc/llvm-project?rev=333993&view=rev
Log:
[clangd] Rewrite JSON dispatcher loop using C IO (FILE*) instead of 
std::istream.

Summary:
The EINTR loop around getline was added to fix an issue with mac gdb, but seems
to loop infinitely in rare cases on linux where the parent editor exits (most
reports with VSCode).
I can't work out how to fix this in a portable way with std::istream, but the
C APIs have clearer contracts and LLVM has a RetryAfterSignal function for use
with them which seems battle-tested.

While here, clean up some inconsistency around \n in log messages (now
add it only after JSON payloads), and reduce the scope of the
long-message handling which was only really added to fight fuzzers.

Reviewers: malaperle, ilya-biryukov

Subscribers: klimek, ioeric, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47643

Modified:
clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
clang-tools-extra/trunk/clangd/ClangdLSPServer.h
clang-tools-extra/trunk/clangd/JSONRPCDispatcher.cpp
clang-tools-extra/trunk/clangd/JSONRPCDispatcher.h
clang-tools-extra/trunk/clangd/tool/ClangdMain.cpp
clang-tools-extra/trunk/test/clangd/too_large.test

Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=333993&r1=333992&r2=333993&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp Tue Jun  5 02:34:46 2018
@@ -396,7 +396,7 @@ ClangdLSPServer::ClangdLSPServer(JSONOut
   SupportedSymbolKinds(defaultSymbolKinds()),
   Server(CDB, FSProvider, /*DiagConsumer=*/*this, Opts) {}
 
-bool ClangdLSPServer::run(std::istream &In, JSONStreamStyle InputStyle) {
+bool ClangdLSPServer::run(std::FILE *In, JSONStreamStyle InputStyle) {
   assert(!IsDone && "Run was called before");
 
   // Set up JSONRPCDispatcher.

Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.h?rev=333993&r1=333992&r2=333993&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdLSPServer.h (original)
+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.h Tue Jun  5 02:34:46 2018
@@ -42,8 +42,8 @@ public:
   /// class constructor. This method must not be executed more than once for
   /// each instance of ClangdLSPServer.
   ///
-  /// \return Wether we received a 'shutdown' request before an 'exit' request
-  bool run(std::istream &In,
+  /// \return Whether we received a 'shutdown' request before an 'exit' 
request.
+  bool run(std::FILE *In,
JSONStreamStyle InputStyle = JSONStreamStyle::Standard);
 
 private:

Modified: clang-tools-extra/trunk/clangd/JSONRPCDispatcher.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/JSONRPCDispatcher.cpp?rev=333993&r1=333992&r2=333993&view=diff
==
--- clang-tools-extra/trunk/clangd/JSONRPCDispatcher.cpp (original)
+++ clang-tools-extra/trunk/clangd/JSONRPCDispatcher.cpp Tue Jun  5 02:34:46 
2018
@@ -14,6 +14,7 @@
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/Support/Chrono.h"
+#include "llvm/Support/Errno.h"
 #include "llvm/Support/SourceMgr.h"
 #include 
 
@@ -66,7 +67,7 @@ void JSONOutput::writeMessage(const json
 Outs << "Content-Length: " << S.size() << "\r\n\r\n" << S;
 Outs.flush();
   }
-  log(llvm::Twine("--> ") + S);
+  log(llvm::Twine("--> ") + S + "\n");
 }
 
 void JSONOutput::log(const Twine &Message) {
@@ -180,27 +181,43 @@ bool JSONRPCDispatcher::call(const json:
   return true;
 }
 
-static llvm::Optional readStandardMessage(std::istream &In,
+// Tries to read a line up to and including \n.
+// If failing, feof() or ferror() will be set.
+static bool readLine(std::FILE *In, std::string &Out) {
+  static constexpr int BufSize = 1024;
+  size_t Size = 0;
+  Out.clear();
+  for (;;) {
+Out.resize(Size + BufSize);
+// Handle EINTR which is sent when a debugger attaches on some platforms.
+if (!llvm::sys::RetryAfterSignal(nullptr, ::fgets, &Out[Size], BufSize, 
In))
+  return false;
+clearerr(In);
+// If the line contained null bytes, anything after it (including \n) will
+// be ignored. Fortunately this is not a legal header or JSON.
+size_t Read = std::strlen(&Out[Size]);
+if (Read > 0 && Out[Size + Read - 1] == '\n') {
+  Out.resize(Size + Read);
+  return true;
+}
+Size += Read;
+  }
+}
+
+// Returns None when:
+//  - ferror() or feof() are set.
+//  - Content-Length is missing or empty (protocol error)
+static llvm::Optional readStandar

[clang-tools-extra] r333994 - [clang-tidy] fix broken test (no compile command) from r331763

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Jun  5 02:42:06 2018
New Revision: 333994

URL: http://llvm.org/viewvc/llvm-project?rev=333994&view=rev
Log:
[clang-tidy] fix broken test (no compile command) from r331763

Modified:

clang-tools-extra/trunk/test/clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp

clang-tools-extra/trunk/test/clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp

Modified: 
clang-tools-extra/trunk/test/clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp?rev=333994&r1=333993&r2=333994&view=diff
==
--- 
clang-tools-extra/trunk/test/clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp
 (original)
+++ 
clang-tools-extra/trunk/test/clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp
 Tue Jun  5 02:42:06 2018
@@ -1,4 +1,4 @@
-// RUN: clang-tidy -enable-check-profile 
-checks='-*,readability-function-size' %s 2>&1 | FileCheck --match-full-lines 
-implicit-check-not='{{warning:|error:}}' %s
+// RUN: clang-tidy -enable-check-profile 
-checks='-*,readability-function-size' %s -- 2>&1 | FileCheck 
--match-full-lines -implicit-check-not='{{warning:|error:}}' %s
 
 // CHECK: 
===-===
 // CHECK-NEXT: {{.*}}  --- Name ---

Modified: 
clang-tools-extra/trunk/test/clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp?rev=333994&r1=333993&r2=333994&view=diff
==
--- 
clang-tools-extra/trunk/test/clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp
 (original)
+++ 
clang-tools-extra/trunk/test/clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp
 Tue Jun  5 02:42:06 2018
@@ -1,4 +1,4 @@
-// RUN: clang-tidy -enable-check-profile 
-checks='-*,readability-function-size' %s %s 2>&1 | FileCheck 
--match-full-lines -implicit-check-not='{{warning:|error:}}' %s
+// RUN: clang-tidy -enable-check-profile 
-checks='-*,readability-function-size' %s %s -- 2>&1 | FileCheck 
--match-full-lines -implicit-check-not='{{warning:|error:}}' %s
 
 // CHECK: 
===-===
 // CHECK-NEXT: {{.*}}  --- Name ---


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [clang-tools-extra] r333994 - [clang-tidy] fix broken test (no compile command) from r331763

[Sam McCall via cfe-commits]

Hi Roman,

Definitely something different in our configs, but it wasn't just me - a
couple of colleagues have mentioned those tests as being broken for a while!

The problem is the lack of compilation database, so I can imagine it can
depend on filesystem layout, e.g. whether you have a separate build/source
directory. (Potentially the CDB from llvm itself may be picked up or not)

I can't tell from your lit output whether clang-tidy or FileCheck failed,
what does this command return for you?

bin/clang-tidy -enable-check-profile -checks='-*,readability-function-size'
../src/llvm/tools/clang/tools/extra/test/clang-tid
y/clang-tidy-enable-check-profile-one-tu.cpp --

(adjust path as needed)

On Tue, Jun 5, 2018 at 12:30 PM Roman Lebedev  wrote:

> This is strange.
>
> First, i'm pretty sure the test worked for me before.
> Second, this commit actually *breaks* those two tests for me:
>
> $ ninja check-clang-tools
> [0/1] Running the Clang extra tools' regression tests
> FAIL: Clang Tools ::
> clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp (123 of 867)
>  TEST 'Clang Tools ::
> clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp' FAILED
> 
> Script:
> --
> : 'RUN: at line 1';   clang-tidy -enable-check-profile
> -checks='-*,readability-function-size'
>
> /build/clang-tools-extra/test/clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp
> -- 2>&1 | FileCheck --match-full-lines
> -implicit-check-not='{{warning:|error:}}' /build/clang
> -tools-extra/test/clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp
> --
> Exit Code: 2
>
> Command Output (stderr):
> --
> FileCheck error: '-' is empty.
> FileCheck command line:  FileCheck --match-full-lines
> -implicit-check-not={{warning:|error:}}
>
> /build/clang-tools-extra/test/clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp
>
> --
>
> 
> FAIL: Clang Tools ::
> clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp (125 of 867)
>  TEST 'Clang Tools ::
> clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp' FAILED
> 
> Script:
> --
> : 'RUN: at line 1';   clang-tidy -enable-check-profile
> -checks='-*,readability-function-size'
>
> /build/clang-tools-extra/test/clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp
>
> /build/clang-tools-extra/test/clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp
> -- 2>&1 |
> FileCheck --match-full-lines -implicit-check-not='{{warning:|error:}}'
>
> /build/clang-tools-extra/test/clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp
> --
> Exit Code: 2
>
> Command Output (stderr):
> --
> FileCheck error: '-' is empty.
> FileCheck command line:  FileCheck --match-full-lines
> -implicit-check-not={{warning:|error:}}
>
> /build/clang-tools-extra/test/clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp
>
> --
>
> Roman.
>
> On Tue, Jun 5, 2018 at 12:42 PM, Sam McCall via cfe-commits
>  wrote:
> > Author: sammccall
> > Date: Tue Jun  5 02:42:06 2018
> > New Revision: 333994
> >
> > URL: http://llvm.org/viewvc/llvm-project?rev=333994&view=rev
> > Log:
> > [clang-tidy] fix broken test (no compile command) from r331763
> >
> > Modified:
> >
>  
> clang-tools-extra/trunk/test/clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp
> >
>  
> clang-tools-extra/trunk/test/clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp
> >
> > Modified:
> clang-tools-extra/trunk/test/clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp
> > URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp?rev=333994&r1=333993&r2=333994&view=diff
> >
> ==
> > ---
> clang-tools-extra/trunk/test/clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp
> (original)
> > +++
> clang-tools-extra/trunk/test/clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp
> Tue Jun  5 02:42:06 2018
> > @@ -1,4 +1,4 @@
> > -// RUN: clang-tidy -enable-check-profile
> -checks='-*,readability-function-size' %s 2>&1 | FileCheck
> --match-full-lines -implicit-check-not='{{warning:|error:}}' %s
> > +// RUN: clang-tidy -enable-check-profile
> -checks='-*,readability-function-size' %s -- 2>&1 | FileCheck
> --match-full-lines -implicit-check-not='{{warning:|error:}}' %s
> >
> >  // CHECK:
> ===---

Re: [clang-tools-extra] r333994 - [clang-tidy] fix broken test (no compile command) from r331763

[Sam McCall via cfe-commits]

Ah, I've been there. No worries!

On Tue, Jun 5, 2018 at 1:08 PM Roman Lebedev  wrote:

> Ah, hm, yes, that was a further local problem in
> https://reviews.llvm.org/D46602
> And i didn't double-check on master initially, because that causes all
> the checks to be rebuilt :/
>
> So thanks for the fix!
>
> Roman.
>
>
> On Tue, Jun 5, 2018 at 1:40 PM, Sam McCall  wrote:
> > Hi Roman,
> >
> > Definitely something different in our configs, but it wasn't just me - a
> > couple of colleagues have mentioned those tests as being broken for a
> while!
> >
> > The problem is the lack of compilation database, so I can imagine it can
> > depend on filesystem layout, e.g. whether you have a separate
> build/source
> > directory. (Potentially the CDB from llvm itself may be picked up or not)
> >
> > I can't tell from your lit output whether clang-tidy or FileCheck failed,
> > what does this command return for you?
> >
> > bin/clang-tidy -enable-check-profile
> -checks='-*,readability-function-size'
> > ../src/llvm/tools/clang/tools/extra/test/clang-tid
> > y/clang-tidy-enable-check-profile-one-tu.cpp --
> >
> > (adjust path as needed)
> >
> > On Tue, Jun 5, 2018 at 12:30 PM Roman Lebedev 
> wrote:
> >>
> >> This is strange.
> >>
> >> First, i'm pretty sure the test worked for me before.
> >> Second, this commit actually *breaks* those two tests for me:
> >>
> >> $ ninja check-clang-tools
> >> [0/1] Running the Clang extra tools' regression tests
> >> FAIL: Clang Tools ::
> >> clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp (123 of 867)
> >>  TEST 'Clang Tools ::
> >> clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp' FAILED
> >> 
> >> Script:
> >> --
> >> : 'RUN: at line 1';   clang-tidy -enable-check-profile
> >> -checks='-*,readability-function-size'
> >>
> >>
> /build/clang-tools-extra/test/clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp
> >> -- 2>&1 | FileCheck --match-full-lines
> >> -implicit-check-not='{{warning:|error:}}' /build/clang
> >> -tools-extra/test/clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp
> >> --
> >> Exit Code: 2
> >>
> >> Command Output (stderr):
> >> --
> >> FileCheck error: '-' is empty.
> >> FileCheck command line:  FileCheck --match-full-lines
> >> -implicit-check-not={{warning:|error:}}
> >>
> >>
> /build/clang-tools-extra/test/clang-tidy/clang-tidy-enable-check-profile-one-tu.cpp
> >>
> >> --
> >>
> >> 
> >> FAIL: Clang Tools ::
> >> clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp (125 of 867)
> >>  TEST 'Clang Tools ::
> >> clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp' FAILED
> >> 
> >> Script:
> >> --
> >> : 'RUN: at line 1';   clang-tidy -enable-check-profile
> >> -checks='-*,readability-function-size'
> >>
> >>
> /build/clang-tools-extra/test/clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp
> >>
> >>
> /build/clang-tools-extra/test/clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp
> >> -- 2>&1 |
> >> FileCheck --match-full-lines -implicit-check-not='{{warning:|error:}}'
> >>
> >>
> /build/clang-tools-extra/test/clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp
> >> --
> >> Exit Code: 2
> >>
> >> Command Output (stderr):
> >> --
> >> FileCheck error: '-' is empty.
> >> FileCheck command line:  FileCheck --match-full-lines
> >> -implicit-check-not={{warning:|error:}}
> >>
> >>
> /build/clang-tools-extra/test/clang-tidy/clang-tidy-enable-check-profile-two-tu.cpp
> >>
> >> --
> >>
> >> Roman.
> >>
> >> On Tue, Jun 5, 2018 at 12:42 PM, Sam McCall via cfe-commits
> >>  wrote:
> >> > Author: sammccall
> >> > Date: Tue Jun  5 02:42:06 2018
> >> > New Revision: 333994
> >> >
> >> > URL: http://llvm.org/viewvc/llvm-project?rev=333994&view=rev
> >> > Log:
> >> > [clang-tidy] fix broken test (no compile command) from r331763
> >> >
> >> > Modified:
> >> >
> >> >
> cl

[clang-tools-extra] r334014 - [clangd] Test tweaks (consistency, shorter, doc). NFC

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Jun  5 05:22:43 2018
New Revision: 334014

URL: http://llvm.org/viewvc/llvm-project?rev=334014&view=rev
Log:
[clangd] Test tweaks (consistency, shorter, doc). NFC

Modified:
clang-tools-extra/trunk/clangd/Quality.h
clang-tools-extra/trunk/unittests/clangd/QualityTests.cpp
clang-tools-extra/trunk/unittests/clangd/TestTU.h

Modified: clang-tools-extra/trunk/clangd/Quality.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/Quality.h?rev=334014&r1=334013&r2=334014&view=diff
==
--- clang-tools-extra/trunk/clangd/Quality.h (original)
+++ clang-tools-extra/trunk/clangd/Quality.h Tue Jun  5 05:22:43 2018
@@ -61,11 +61,10 @@ llvm::raw_ostream &operator<<(llvm::raw_
 
 /// Attributes of a symbol-query pair that affect how much we like it.
 struct SymbolRelevanceSignals {
-  // 0-1 fuzzy-match score for unqualified name. Must be explicitly assigned.
+  /// 0-1 fuzzy-match score for unqualified name. Must be explicitly assigned.
   float NameMatch = 1;
   bool Forbidden = false; // Unavailable (e.g const) or inaccessible (private).
-  /// Proximity between the best declaration and the query location. [0-1] 
score
-  /// where 1 is closest
+  /// Proximity between best declaration and the query. [0-1], 1 is closest.
   float ProximityScore = 0;
 
   void merge(const CodeCompletionResult &SemaResult);

Modified: clang-tools-extra/trunk/unittests/clangd/QualityTests.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/clangd/QualityTests.cpp?rev=334014&r1=334013&r2=334014&view=diff
==
--- clang-tools-extra/trunk/unittests/clangd/QualityTests.cpp (original)
+++ clang-tools-extra/trunk/unittests/clangd/QualityTests.cpp Tue Jun  5 
05:22:43 2018
@@ -60,44 +60,34 @@ TEST(QualityTests, SymbolQualitySignalEx
 TEST(QualityTests, SymbolRelevanceSignalExtraction) {
   TestTU Test;
   Test.HeaderCode = R"cpp(
-int test_func_in_header();
-int test_func_in_header_and_cpp();
+int header();
+int header_main();
 )cpp";
   Test.Code = R"cpp(
-int ::test_func_in_header_and_cpp() {
-}
-int test_func_in_cpp();
+int ::header_main() {}
+int main();
 
 [[deprecated]]
-int test_deprecated() { return 0; }
+int deprecated() { return 0; }
   )cpp";
   auto AST = Test.build();
 
-  SymbolRelevanceSignals Deprecated;
-  Deprecated.merge(CodeCompletionResult(&findDecl(AST, "test_deprecated"),
-/*Priority=*/42, nullptr, false,
-/*Accessible=*/false));
-  EXPECT_EQ(Deprecated.NameMatch, SymbolRelevanceSignals().NameMatch);
-  EXPECT_TRUE(Deprecated.Forbidden);
-
-  // Test proximity scores.
-  SymbolRelevanceSignals FuncInCpp;
-  FuncInCpp.merge(CodeCompletionResult(&findDecl(AST, "test_func_in_cpp"),
-   CCP_Declaration));
-  /// Decls in the current file should get a proximity score of 1.0.
-  EXPECT_FLOAT_EQ(FuncInCpp.ProximityScore, 1.0);
-
-  SymbolRelevanceSignals FuncInHeader;
-  FuncInHeader.merge(CodeCompletionResult(&findDecl(AST, 
"test_func_in_header"),
-  CCP_Declaration));
-  /// Decls outside current file currently don't get a proximity score boost.
-  EXPECT_FLOAT_EQ(FuncInHeader.ProximityScore, 0.0);
-
-  SymbolRelevanceSignals FuncInHeaderAndCpp;
-  FuncInHeaderAndCpp.merge(CodeCompletionResult(
-  &findDecl(AST, "test_func_in_header_and_cpp"), CCP_Declaration));
-  /// Decls in both header **and** the main file get the same boost.
-  EXPECT_FLOAT_EQ(FuncInHeaderAndCpp.ProximityScore, 1.0);
+  SymbolRelevanceSignals Relevance;
+  Relevance.merge(CodeCompletionResult(&findDecl(AST, "deprecated"),
+   /*Priority=*/42, nullptr, false,
+   /*Accessible=*/false));
+  EXPECT_EQ(Relevance.NameMatch, SymbolRelevanceSignals().NameMatch);
+  EXPECT_TRUE(Relevance.Forbidden);
+
+  Relevance = {};
+  Relevance.merge(CodeCompletionResult(&findDecl(AST, "main"), 42));
+  EXPECT_FLOAT_EQ(Relevance.ProximityScore, 1.0) << "Decl in current file";
+  Relevance = {};
+  Relevance.merge(CodeCompletionResult(&findDecl(AST, "header"), 42));
+  EXPECT_FLOAT_EQ(Relevance.ProximityScore, 0.0) << "Decl from header";
+  Relevance = {};
+  Relevance.merge(CodeCompletionResult(&findDecl(AST, "header_main"), 42));
+  EXPECT_FLOAT_EQ(Relevance.ProximityScore, 1.0) << "Current file and header";
 }
 
 // Do the signals move the scores in the direction we expect?
@@ -136,7 +126,7 @@ TEST(QualityTests, SymbolRelevanceSignal
 
   SymbolRelevanceSignals WithProximity;
   WithProximity.ProximityScore = 0.2f;
-  EXPECT_LT(Default.evaluate(), WithProximity.evaluate());
+  EXPECT_GT(WithProximity.evaluate(), Default.evaluate());
 }
 
 TEST(QualityTests, Sor

[clang-tools-extra] r334026 - [clangd] Boost code completion results that are narrowly scoped (local, members)

[Sam McCall via cfe-commits]

Author: sammccall
Date: Tue Jun  5 09:30:25 2018
New Revision: 334026

URL: http://llvm.org/viewvc/llvm-project?rev=334026&view=rev
Log:
[clangd] Boost code completion results that are narrowly scoped (local, members)

Summary:
This signal is considered a relevance rather than a quality signal because it's
dependent on the query (the fact that it's completion, and implicitly the query
context).

This is part of the effort to reduce reliance on Sema priority, so we can have
consistent ranking between Index and Sema results.

Reviewers: ioeric

Subscribers: klimek, ilya-biryukov, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47762

Modified:
clang-tools-extra/trunk/clangd/ClangdUnit.cpp
clang-tools-extra/trunk/clangd/ClangdUnit.h
clang-tools-extra/trunk/clangd/CodeComplete.cpp
clang-tools-extra/trunk/clangd/Quality.cpp
clang-tools-extra/trunk/clangd/Quality.h
clang-tools-extra/trunk/unittests/clangd/QualityTests.cpp
clang-tools-extra/trunk/unittests/clangd/TestTU.cpp
clang-tools-extra/trunk/unittests/clangd/TestTU.h

Modified: clang-tools-extra/trunk/clangd/ClangdUnit.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnit.cpp?rev=334026&r1=334025&r2=334026&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdUnit.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdUnit.cpp Tue Jun  5 09:30:25 2018
@@ -51,11 +51,11 @@ template  std::size_t getUsedBy
 
 class DeclTrackingASTConsumer : public ASTConsumer {
 public:
-  DeclTrackingASTConsumer(std::vector &TopLevelDecls)
+  DeclTrackingASTConsumer(std::vector &TopLevelDecls)
   : TopLevelDecls(TopLevelDecls) {}
 
   bool HandleTopLevelDecl(DeclGroupRef DG) override {
-for (const Decl *D : DG) {
+for (Decl *D : DG) {
   // ObjCMethodDecl are not actually top-level decls.
   if (isa(D))
 continue;
@@ -66,14 +66,12 @@ public:
   }
 
 private:
-  std::vector &TopLevelDecls;
+  std::vector &TopLevelDecls;
 };
 
 class ClangdFrontendAction : public SyntaxOnlyAction {
 public:
-  std::vector takeTopLevelDecls() {
-return std::move(TopLevelDecls);
-  }
+  std::vector takeTopLevelDecls() { return std::move(TopLevelDecls); }
 
 protected:
   std::unique_ptr CreateASTConsumer(CompilerInstance &CI,
@@ -82,7 +80,7 @@ protected:
   }
 
 private:
-  std::vector TopLevelDecls;
+  std::vector TopLevelDecls;
 };
 
 class CppFilePreambleCallbacks : public PreambleCallbacks {
@@ -174,7 +172,7 @@ ParsedAST::Build(std::unique_ptr ParsedDecls = Action->takeTopLevelDecls();
+  std::vector ParsedDecls = Action->takeTopLevelDecls();
   std::vector Diags = ASTDiags.take();
   // Add diagnostics from the preamble, if any.
   if (Preamble)
@@ -210,7 +208,7 @@ const Preprocessor &ParsedAST::getPrepro
   return Clang->getPreprocessor();
 }
 
-ArrayRef ParsedAST::getLocalTopLevelDecls() {
+ArrayRef ParsedAST::getLocalTopLevelDecls() {
   return LocalTopLevelDecls;
 }
 
@@ -261,7 +259,7 @@ PreambleData::PreambleData(PrecompiledPr
 ParsedAST::ParsedAST(std::shared_ptr Preamble,
  std::unique_ptr Clang,
  std::unique_ptr Action,
- std::vector LocalTopLevelDecls,
+ std::vector LocalTopLevelDecls,
  std::vector Diags, std::vector 
Inclusions)
 : Preamble(std::move(Preamble)), Clang(std::move(Clang)),
   Action(std::move(Action)), Diags(std::move(Diags)),

Modified: clang-tools-extra/trunk/clangd/ClangdUnit.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdUnit.h?rev=334026&r1=334025&r2=334026&view=diff
==
--- clang-tools-extra/trunk/clangd/ClangdUnit.h (original)
+++ clang-tools-extra/trunk/clangd/ClangdUnit.h Tue Jun  5 09:30:25 2018
@@ -91,7 +91,8 @@ public:
 
   /// This function returns top-level decls present in the main file of the 
AST.
   /// The result does not include the decls that come from the preamble.
-  ArrayRef getLocalTopLevelDecls();
+  /// (These should be const, but RecursiveASTVisitor requires Decl*).
+  ArrayRef getLocalTopLevelDecls();
 
   const std::vector &getDiagnostics() const;
 
@@ -104,8 +105,8 @@ private:
   ParsedAST(std::shared_ptr Preamble,
 std::unique_ptr Clang,
 std::unique_ptr Action,
-std::vector LocalTopLevelDecls,
-std::vector Diags, std::vector Inclusions);
+std::vector LocalTopLevelDecls, std::vector Diags,
+std::vector Inclusions);
 
   // In-memory preambles must outlive the AST, it is important that this member
   // goes before Clang and Action.
@@ -122,7 +123,7 @@ private:
   std::vector Diags;
   // Top-level decls inside the current file. Not that this does not include
   // top-level decls from the preamble.
-  std::vector LocalTopLevelDecls;
+  std::vector L