[PATCH] D77385: [clangd] Add index inspection helper tool

[Mark Nauwelaerts via Phabricator via cfe-commits]

mnauw created this revision.
mnauw added a reviewer: sammccall.
mnauw added a project: clang-tools-extra.
Herald added subscribers: usaxena95, kadircet, arphaman, jkorous, MaskRay, 
ilya-biryukov, mgorny.
Herald added a project: clang.

Add a standalone executable that can read indexed data and output in requested 
format.

Typical use is to read RIFF data to dump to YAML for inspection of indexed 
data.  The YAML (de)serialization has also been extended to aid in this regard.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D77385

Files:
  clang-tools-extra/clangd/CMakeLists.txt
  clang-tools-extra/clangd/index-dump/CMakeLists.txt
  clang-tools-extra/clangd/index-dump/IndexDumpMain.cpp
  clang-tools-extra/clangd/index/YAMLSerialization.cpp
  clang-tools-extra/clangd/test/CMakeLists.txt

Index: clang-tools-extra/clangd/test/CMakeLists.txt
===
--- clang-tools-extra/clangd/test/CMakeLists.txt
+++ clang-tools-extra/clangd/test/CMakeLists.txt
@@ -13,6 +13,7 @@
   ClangdTests
   # No tests for these, but we should still make sure they build.
   clangd-indexer
+  clangd-index-dump
   dexp
   )
 
Index: clang-tools-extra/clangd/index/YAMLSerialization.cpp
===
--- clang-tools-extra/clangd/index/YAMLSerialization.cpp
+++ clang-tools-extra/clangd/index/YAMLSerialization.cpp
@@ -41,6 +41,8 @@
   llvm::Optional Symbol;
   llvm::Optional Refs;
   llvm::Optional Relation;
+  llvm::Optional Source;
+  llvm::Optional Cmd;
 };
 // A class helps YAML to serialize the 32-bit encoded position (Line&Column),
 // as YAMLIO can't directly map bitfields.
@@ -53,6 +55,9 @@
 namespace llvm {
 namespace yaml {
 
+using clang::clangd::FileDigest;
+using clang::clangd::IncludeGraph;
+using clang::clangd::IncludeGraphNode;
 using clang::clangd::Ref;
 using clang::clangd::RefKind;
 using clang::clangd::Relation;
@@ -65,6 +70,7 @@
 using clang::index::SymbolKind;
 using clang::index::SymbolLanguage;
 using clang::index::SymbolRole;
+using clang::tooling::CompileCommand;
 
 // Helper to (de)serialize the SymbolID. We serialize it as a hex string.
 struct NormalizedSymbolID {
@@ -308,6 +314,76 @@
   }
 };
 
+struct NormalizedSourceFlag {
+  NormalizedSourceFlag(IO &) {}
+  NormalizedSourceFlag(IO &, IncludeGraphNode::SourceFlag O) {
+Flag = static_cast(O);
+  }
+
+  IncludeGraphNode::SourceFlag denormalize(IO &) {
+return static_cast(Flag);
+  }
+
+  uint8_t Flag = 0;
+};
+
+struct NormalizedFileDigest {
+  NormalizedFileDigest(IO &) {}
+  NormalizedFileDigest(IO &, const FileDigest &Digest) {
+HexString = llvm::toHex(Digest);
+  }
+
+  static FileDigest fromRaw(llvm::StringRef Raw) {
+FileDigest Digest;
+assert(Raw.size() == sizeof(Digest));
+memcpy(Digest.data(), Raw.data(), Raw.size());
+return Digest;
+  }
+
+  static llvm::Expected fromStr(llvm::StringRef Str) {
+const int RawSize = sizeof(FileDigest);
+if (Str.size() != RawSize * 2)
+  return llvm::createStringError(llvm::inconvertibleErrorCode(),
+ "Bad ID length");
+for (char C : Str)
+  if (!llvm::isHexDigit(C))
+return llvm::createStringError(llvm::inconvertibleErrorCode(),
+   "Bad hex ID");
+return fromRaw(llvm::fromHex(Str));
+  }
+
+  FileDigest denormalize(IO &I) {
+auto Digest = fromStr(HexString);
+if (!Digest) {
+  I.setError(llvm::toString(Digest.takeError()));
+  return FileDigest();
+}
+return *Digest;
+  }
+
+  std::string HexString;
+};
+
+template <> struct MappingTraits {
+  static void mapping(IO &IO, IncludeGraphNode &Node) {
+IO.mapRequired("URI", Node.URI);
+MappingNormalization
+NSourceFlag(IO, Node.Flags);
+IO.mapRequired("Flags", NSourceFlag->Flag);
+MappingNormalization NDigest(IO,
+   Node.Digest);
+IO.mapRequired("Digest", NDigest->HexString);
+IO.mapRequired("DirectIncludes", Node.DirectIncludes);
+  }
+};
+
+template <> struct MappingTraits {
+  static void mapping(IO &IO, CompileCommand &Cmd) {
+IO.mapRequired("Directory", Cmd.Directory);
+IO.mapRequired("CommandLine", Cmd.CommandLine);
+  }
+};
+
 template <> struct MappingTraits {
   static void mapping(IO &IO, VariantEntry &Variant) {
 if (IO.mapTag("!Symbol", Variant.Symbol.hasValue())) {
@@ -322,6 +398,14 @@
   if (!IO.outputting())
 Variant.Relation.emplace();
   MappingTraits::mapping(IO, *Variant.Relation);
+} else if (IO.mapTag("!Source", Variant.Source.hasValue())) {
+  if (!IO.outputting())
+Variant.Source.emplace();
+  MappingTraits::mapping(IO, *Variant.Source);
+} else if (IO.mapTag("!Cmd", Variant.Cmd.hasValue())) {
+  if (!IO.outputting())
+Variant.Cmd.emplace();
+  MappingTraits::mapping(IO, *Variant.Cmd);
 }
   }
 };
@@ -35

[PATCH] D77385: [clangd] Add index inspection helper tool

[Mark Nauwelaerts via Phabricator via cfe-commits]

mnauw added a comment.

Thanks for having a look at this patch.  My use-case/goal would be to have a 
way to run/do a command that reads some index and dumps it (in e.g. YAML) to 
(e.g.) stdout. That way it can be inspected/viewed all at once by plain viewer. 
 And not only symbols, refs, but also recorded paths (that form the 
include-graph).  The latter in particular was/is useful when investigating some 
not-entirely-consistent symlink-resolution issues.  The goal is not necessarily 
a separate binary, so if it's ok to extend dexp to achieve the former (e.g. by 
adding option(s)) then I can also update to patch that way accordingly.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77385/new/

https://reviews.llvm.org/D77385



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D77385: [clangd] Add index export to dexp

[Mark Nauwelaerts via Phabricator via cfe-commits]

mnauw updated this revision to Diff 255019.
mnauw retitled this revision from "[clangd] Add index inspection helper tool" 
to "[clangd] Add index export to dexp".
mnauw edited the summary of this revision.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77385/new/

https://reviews.llvm.org/D77385

Files:
  clang-tools-extra/clangd/index/YAMLSerialization.cpp
  clang-tools-extra/clangd/index/dex/dexp/Dexp.cpp

Index: clang-tools-extra/clangd/index/dex/dexp/Dexp.cpp
===
--- clang-tools-extra/clangd/index/dex/dexp/Dexp.cpp
+++ clang-tools-extra/clangd/index/dex/dexp/Dexp.cpp
@@ -25,6 +25,10 @@
 namespace clangd {
 namespace {
 
+static llvm::cl::opt
+Export("export", llvm::cl::desc("Export index non-interactively"),
+   llvm::cl::init(false));
+
 llvm::cl::opt IndexPath("index-path",
  llvm::cl::desc("Path to the index"),
  llvm::cl::Positional, llvm::cl::Required);
@@ -252,6 +256,44 @@
   }
 };
 
+class ExportImpl {
+  llvm::cl::opt Format{
+  "format", llvm::cl::desc("Format of index export"),
+  llvm::cl::values(
+  clEnumValN(IndexFileFormat::YAML, "yaml",
+ "human-readable YAML format"),
+  clEnumValN(IndexFileFormat::RIFF, "binary", "binary RIFF format")),
+  llvm::cl::init(IndexFileFormat::YAML)};
+
+public:
+  int dump() {
+using namespace clang::clangd;
+// Read input file (as specified in global option)
+auto Buffer = llvm::MemoryBuffer::getFile(IndexPath);
+if (!Buffer) {
+  llvm::errs() << llvm::formatv("Can't open {0}", IndexPath) << "\n";
+  return 1;
+}
+
+// Auto-detects input format when parsing
+auto IndexIn = clang::clangd::readIndexFile(Buffer->get()->getBuffer());
+if (!IndexIn) {
+  llvm::errs() << llvm::toString(IndexIn.takeError()) << "\n";
+  return 1;
+}
+
+// Dump output file
+clang::clangd::IndexFileOut IndexOut(IndexIn.get());
+IndexOut.Format = Format;
+llvm::outs() << IndexOut;
+return 0;
+  }
+};
+
+class ExportCmd : public ExportImpl, public Command {
+  void run() override { dump(); }
+};
+
 struct {
   const char *Name;
   const char *Description;
@@ -262,6 +304,7 @@
  std::make_unique},
 {"refs", "Find references by ID or qualified name",
  std::make_unique},
+{"export", "Export index", std::make_unique},
 };
 
 std::unique_ptr openIndex(llvm::StringRef Index) {
@@ -275,10 +318,20 @@
 int main(int argc, const char *argv[]) {
   using namespace clang::clangd;
 
+  // Make Export command option(s) available on command line.
+  // That allows for convenient (piping/redirecting) a dump non-interactively
+  // without passing through REPL.
+  auto NonInteractiveExport = std::make_unique();
   llvm::cl::ParseCommandLineOptions(argc, argv, Overview);
-  llvm::cl::ResetCommandLineParser(); // We reuse it for REPL commands.
   llvm::sys::PrintStackTraceOnErrorSignal(argv[0]);
 
+  if (Export)
+return NonInteractiveExport->dump();
+
+  // Back to normal
+  llvm::cl::ResetCommandLineParser(); // We reuse it for REPL commands.
+  NonInteractiveExport.reset();
+
   std::unique_ptr Index;
   reportTime("Dex build", [&]() {
 Index = openIndex(IndexPath);
Index: clang-tools-extra/clangd/index/YAMLSerialization.cpp
===
--- clang-tools-extra/clangd/index/YAMLSerialization.cpp
+++ clang-tools-extra/clangd/index/YAMLSerialization.cpp
@@ -41,6 +41,8 @@
   llvm::Optional Symbol;
   llvm::Optional Refs;
   llvm::Optional Relation;
+  llvm::Optional Source;
+  llvm::Optional Cmd;
 };
 // A class helps YAML to serialize the 32-bit encoded position (Line&Column),
 // as YAMLIO can't directly map bitfields.
@@ -53,6 +55,9 @@
 namespace llvm {
 namespace yaml {
 
+using clang::clangd::FileDigest;
+using clang::clangd::IncludeGraph;
+using clang::clangd::IncludeGraphNode;
 using clang::clangd::Ref;
 using clang::clangd::RefKind;
 using clang::clangd::Relation;
@@ -65,6 +70,7 @@
 using clang::index::SymbolKind;
 using clang::index::SymbolLanguage;
 using clang::index::SymbolRole;
+using clang::tooling::CompileCommand;
 
 // Helper to (de)serialize the SymbolID. We serialize it as a hex string.
 struct NormalizedSymbolID {
@@ -308,6 +314,76 @@
   }
 };
 
+struct NormalizedSourceFlag {
+  NormalizedSourceFlag(IO &) {}
+  NormalizedSourceFlag(IO &, IncludeGraphNode::SourceFlag O) {
+Flag = static_cast(O);
+  }
+
+  IncludeGraphNode::SourceFlag denormalize(IO &) {
+return static_cast(Flag);
+  }
+
+  uint8_t Flag = 0;
+};
+
+struct NormalizedFileDigest {
+  NormalizedFileDigest(IO &) {}
+  NormalizedFileDigest(IO &, const FileDigest &Digest) {
+HexString = llvm::toHex(Digest);
+  }
+
+  static FileDigest fromRaw(llvm::StringRef Raw) {
+FileDigest Digest;
+assert(Raw.size() == sizeof(Digest));
+memcpy(Digest.data(), Raw.data(), 

[PATCH] D77385: [clangd] Add index export to dexp

[Mark Nauwelaerts via Phabricator via cfe-commits]

mnauw updated this revision to Diff 256557.
mnauw edited the summary of this revision.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77385/new/

https://reviews.llvm.org/D77385

Files:
  clang-tools-extra/clangd/index/dex/dexp/Dexp.cpp


Index: clang-tools-extra/clangd/index/dex/dexp/Dexp.cpp
===
--- clang-tools-extra/clangd/index/dex/dexp/Dexp.cpp
+++ clang-tools-extra/clangd/index/dex/dexp/Dexp.cpp
@@ -259,6 +259,56 @@
   }
 };
 
+class Export : public Command {
+  llvm::cl::opt Format{
+  "format",
+  llvm::cl::desc("Format of index export"),
+  llvm::cl::values(
+  clEnumValN(IndexFileFormat::YAML, "yaml",
+ "human-readable YAML format"),
+  clEnumValN(IndexFileFormat::RIFF, "binary", "binary RIFF format")),
+  llvm::cl::init(IndexFileFormat::YAML),
+  };
+  llvm::cl::opt OutputFile{
+  "output-file",
+  llvm::cl::Positional,
+  llvm::cl::Required,
+  llvm::cl::desc("Output file for export"),
+  };
+
+public:
+  void run() {
+using namespace clang::clangd;
+// Read input file (as specified in global option)
+auto Buffer = llvm::MemoryBuffer::getFile(IndexPath);
+if (!Buffer) {
+  llvm::errs() << llvm::formatv("Can't open {0}", IndexPath) << "\n";
+  return;
+}
+
+// Auto-detects input format when parsing
+auto IndexIn = clang::clangd::readIndexFile(Buffer->get()->getBuffer());
+if (!IndexIn) {
+  llvm::errs() << llvm::toString(IndexIn.takeError()) << "\n";
+  return;
+}
+
+// Prepare output file
+std::error_code EC;
+llvm::raw_fd_ostream OutputStream(OutputFile, EC);
+if (EC) {
+  llvm::errs() << llvm::formatv("Can't open {0} for writing", OutputFile)
+   << "\n";
+  return;
+}
+
+// Export
+clang::clangd::IndexFileOut IndexOut(IndexIn.get());
+IndexOut.Format = Format;
+OutputStream << IndexOut;
+  }
+};
+
 struct {
   const char *Name;
   const char *Description;
@@ -269,6 +319,7 @@
  std::make_unique},
 {"refs", "Find references by ID or qualified name",
  std::make_unique},
+{"export", "Export index", std::make_unique},
 };
 
 std::unique_ptr openIndex(llvm::StringRef Index) {


Index: clang-tools-extra/clangd/index/dex/dexp/Dexp.cpp
===
--- clang-tools-extra/clangd/index/dex/dexp/Dexp.cpp
+++ clang-tools-extra/clangd/index/dex/dexp/Dexp.cpp
@@ -259,6 +259,56 @@
   }
 };
 
+class Export : public Command {
+  llvm::cl::opt Format{
+  "format",
+  llvm::cl::desc("Format of index export"),
+  llvm::cl::values(
+  clEnumValN(IndexFileFormat::YAML, "yaml",
+ "human-readable YAML format"),
+  clEnumValN(IndexFileFormat::RIFF, "binary", "binary RIFF format")),
+  llvm::cl::init(IndexFileFormat::YAML),
+  };
+  llvm::cl::opt OutputFile{
+  "output-file",
+  llvm::cl::Positional,
+  llvm::cl::Required,
+  llvm::cl::desc("Output file for export"),
+  };
+
+public:
+  void run() {
+using namespace clang::clangd;
+// Read input file (as specified in global option)
+auto Buffer = llvm::MemoryBuffer::getFile(IndexPath);
+if (!Buffer) {
+  llvm::errs() << llvm::formatv("Can't open {0}", IndexPath) << "\n";
+  return;
+}
+
+// Auto-detects input format when parsing
+auto IndexIn = clang::clangd::readIndexFile(Buffer->get()->getBuffer());
+if (!IndexIn) {
+  llvm::errs() << llvm::toString(IndexIn.takeError()) << "\n";
+  return;
+}
+
+// Prepare output file
+std::error_code EC;
+llvm::raw_fd_ostream OutputStream(OutputFile, EC);
+if (EC) {
+  llvm::errs() << llvm::formatv("Can't open {0} for writing", OutputFile)
+   << "\n";
+  return;
+}
+
+// Export
+clang::clangd::IndexFileOut IndexOut(IndexIn.get());
+IndexOut.Format = Format;
+OutputStream << IndexOut;
+  }
+};
+
 struct {
   const char *Name;
   const char *Description;
@@ -269,6 +319,7 @@
  std::make_unique},
 {"refs", "Find references by ID or qualified name",
  std::make_unique},
+{"export", "Export index", std::make_unique},
 };
 
 std::unique_ptr openIndex(llvm::StringRef Index) {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D77385: [clangd] Add index export to dexp

[Mark Nauwelaerts via Phabricator via cfe-commits]

mnauw added a comment.

As suggested, I will provide the YAMLSerialization changes in a separate patch. 
 Regarding test for that, I was thinking of some running some "simple/mock YAML 
data" through some load and save cycle(s).  I presume a separate YAML file (in 
e.g. test/Inputs) is best for that?


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77385/new/

https://reviews.llvm.org/D77385



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D77938: [clangd] Extend YAML Serialization

[Mark Nauwelaerts via Phabricator via cfe-commits]

mnauw added a comment.

This was previously part of D77385 .  Rather 
than a separate file (which seems a bit tricky), the test YAML has simply been 
included in the unit test itself.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77938/new/

https://reviews.llvm.org/D77938



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D77938: [clangd] Extend YAML Serialization

[Mark Nauwelaerts via Phabricator via cfe-commits]

mnauw created this revision.
mnauw added a reviewer: sammccall.
mnauw added a project: clang-tools-extra.
Herald added subscribers: cfe-commits, usaxena95, kadircet, arphaman, jkorous, 
MaskRay, ilya-biryukov, mgorny.
Herald added a project: clang.
mnauw added a comment.

This was previously part of D77385 .  Rather 
than a separate file (which seems a bit tricky), the test YAML has simply been 
included in the unit test itself.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D77938

Files:
  clang-tools-extra/clangd/index/YAMLSerialization.cpp
  clang-tools-extra/clangd/unittests/CMakeLists.txt
  clang-tools-extra/clangd/unittests/YAMLTests.cpp

Index: clang-tools-extra/clangd/unittests/YAMLTests.cpp
===
--- /dev/null
+++ clang-tools-extra/clangd/unittests/YAMLTests.cpp
@@ -0,0 +1,244 @@
+//===-- YAMLTests.cpp - YAML container unit tests -===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+
+#include "index/Serialization.h"
+#include "gmock/gmock.h"
+#include "gtest/gtest.h"
+
+namespace clang {
+namespace clangd {
+namespace {
+
+std::string YAMLData =
+R"|(--- !Symbol
+ID:  13C6B898A37B5E70
+Name:changedFilesTask
+Scope:   'clang::clangd::BackgroundIndex::'
+SymInfo:
+  Kind:InstanceMethod
+  Lang:Cpp
+CanonicalDeclaration:
+  FileURI: 'file:///llvm-project/clang-tools-extra/clangd/index/Background.h'
+  Start:
+Line:197
+Column:  2
+  End:
+Line:197
+Column:  18
+Definition:
+  FileURI: 'file:///llvm-project/clang-tools-extra/clangd/index/Background.cpp'
+  Start:
+Line:166
+Column:  39
+  End:
+Line:166
+Column:  55
+Origin:  4
+Flags:   8
+Signature:   ''
+TemplateSpecializationArgs: ''
+CompletionSnippetSuffix: ''
+Documentation:   ''
+ReturnType:  ''
+Type:''
+...
+--- !Refs
+ID:  FA89F28EEE2EDA4F
+References:
+  - Kind:4
+Location:
+  FileURI: 'file:///llvm-project/clang-tools-extra/clangd/index/Background.cpp'
+  Start:
+Line:408
+Column:  40
+  End:
+Line:408
+Column:  42
+  - Kind:4
+Location:
+  FileURI: 'file:///llvm-project/clang-tools-extra/clangd/index/Background.cpp'
+  Start:
+Line:409
+Column:  13
+  End:
+Line:409
+Column:  15
+  - Kind:4
+Location:
+  FileURI: 'file:///llvm-project/clang-tools-extra/clangd/index/Background.cpp'
+  Start:
+Line:412
+Column:  30
+  End:
+Line:412
+Column:  32
+  - Kind:4
+Location:
+  FileURI: 'file:///llvm-project/clang-tools-extra/clangd/index/Background.cpp'
+  Start:
+Line:415
+Column:  8
+  End:
+Line:415
+Column:  10
+  - Kind:4
+Location:
+  FileURI: 'file:///llvm-project/clang-tools-extra/clangd/index/Background.cpp'
+  Start:
+Line:428
+Column:  24
+  End:
+Line:428
+Column:  26
+  - Kind:4
+Location:
+  FileURI: 'file:///llvm-project/clang-tools-extra/clangd/index/Background.cpp'
+  Start:
+Line:429
+Column:  24
+  End:
+Line:429
+Column:  26
+  - Kind:4
+Location:
+  FileURI: 'file:///llvm-project/clang-tools-extra/clangd/index/Background.cpp'
+  Start:
+Line:460
+Column:  18
+  End:
+Line:460
+Column:  20
+  - Kind:4
+Location:
+  FileURI: 'file:///llvm-project/clang-tools-extra/clangd/index/Background.cpp'
+  Start:
+Line:461
+Column:  64
+  End:
+Line:461
+Column:  66
+  - Kind:4
+Location:
+  FileURI: 'file:///llvm-project/clang-tools-extra/clangd/index/Background.cpp'
+  Start:
+Line:463
+Column:  24
+  End:
+Line:463
+Column:  26
+  - Kind:4
+Location:
+  FileURI: 'file:///llvm-project/clang-tools-extra/clangd/index/Background.cpp'
+  Start

[PATCH] D77938: [clangd] Extend YAML Serialization

[Mark Nauwelaerts via Phabricator via cfe-commits]

mnauw updated this revision to Diff 258292.
mnauw added a comment.

As suggested, existing (YAML) Serialization test has been extended.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77938/new/

https://reviews.llvm.org/D77938

Files:
  clang-tools-extra/clangd/index/YAMLSerialization.cpp
  clang-tools-extra/clangd/unittests/SerializationTests.cpp

Index: clang-tools-extra/clangd/unittests/SerializationTests.cpp
===
--- clang-tools-extra/clangd/unittests/SerializationTests.cpp
+++ clang-tools-extra/clangd/unittests/SerializationTests.cpp
@@ -16,6 +16,7 @@
 
 using ::testing::_;
 using ::testing::AllOf;
+using ::testing::ElementsAre;
 using ::testing::Pair;
 using ::testing::UnorderedElementsAre;
 using ::testing::UnorderedElementsAreArray;
@@ -91,6 +92,20 @@
 Object:
   ID:  6512AEC512EA3A2D
 ...
+--- !Cmd
+Directory:   'testdir'
+CommandLine:
+  - 'cmd1'
+  - 'cmd2'
+...
+--- !Source
+URI: 'file:///path/source1.cpp'
+Flags:   1
+Digest:  EED8F5EAF25C453C
+DirectIncludes:
+  - 'file:///path/inc1.h'
+  - 'file:///path/inc2.h'
+...
 )";
 
 MATCHER_P(ID, I, "") { return arg.ID == cantFail(SymbolID::fromStr(I)); }
@@ -155,6 +170,21 @@
   EXPECT_THAT(
   *ParsedYAML->Relations,
   UnorderedElementsAre(Relation{Base, RelationKind::BaseOf, Derived}));
+
+  ASSERT_TRUE(bool(ParsedYAML->Cmd));
+  auto &Cmd = *ParsedYAML->Cmd;
+  ASSERT_EQ(Cmd.Directory, "testdir");
+  EXPECT_THAT(Cmd.CommandLine, ElementsAre("cmd1", "cmd2"));
+
+  ASSERT_TRUE(bool(ParsedYAML->Sources));
+  const auto *URI = "file:///path/source1.cpp";
+  ASSERT_TRUE(ParsedYAML->Sources->count(URI));
+  auto IGNDeserialized = ParsedYAML->Sources->lookup(URI);
+  EXPECT_EQ(llvm::toHex(IGNDeserialized.Digest), "EED8F5EAF25C453C");
+  EXPECT_THAT(IGNDeserialized.DirectIncludes,
+  ElementsAre("file:///path/inc1.h", "file:///path/inc2.h"));
+  EXPECT_EQ(IGNDeserialized.URI, URI);
+  EXPECT_EQ(IGNDeserialized.Flags, IncludeGraphNode::SourceFlag(1));
 }
 
 std::vector YAMLFromSymbols(const SymbolSlab &Slab) {
Index: clang-tools-extra/clangd/index/YAMLSerialization.cpp
===
--- clang-tools-extra/clangd/index/YAMLSerialization.cpp
+++ clang-tools-extra/clangd/index/YAMLSerialization.cpp
@@ -41,6 +41,8 @@
   llvm::Optional Symbol;
   llvm::Optional Refs;
   llvm::Optional Relation;
+  llvm::Optional Source;
+  llvm::Optional Cmd;
 };
 // A class helps YAML to serialize the 32-bit encoded position (Line&Column),
 // as YAMLIO can't directly map bitfields.
@@ -49,10 +51,16 @@
   uint32_t Column;
 };
 
+// avoid ODR violation of specialization for non-owned CompileCommand
+struct CompileCommandYAML : clang::tooling::CompileCommand {};
+
 } // namespace
 namespace llvm {
 namespace yaml {
 
+using clang::clangd::FileDigest;
+using clang::clangd::IncludeGraph;
+using clang::clangd::IncludeGraphNode;
 using clang::clangd::Ref;
 using clang::clangd::RefKind;
 using clang::clangd::Relation;
@@ -65,6 +73,7 @@
 using clang::index::SymbolKind;
 using clang::index::SymbolLanguage;
 using clang::index::SymbolRole;
+using clang::tooling::CompileCommand;
 
 // Helper to (de)serialize the SymbolID. We serialize it as a hex string.
 struct NormalizedSymbolID {
@@ -308,6 +317,59 @@
   }
 };
 
+struct NormalizedSourceFlag {
+  NormalizedSourceFlag(IO &) {}
+  NormalizedSourceFlag(IO &, IncludeGraphNode::SourceFlag O) {
+Flag = static_cast(O);
+  }
+
+  IncludeGraphNode::SourceFlag denormalize(IO &) {
+return static_cast(Flag);
+  }
+
+  uint8_t Flag = 0;
+};
+
+struct NormalizedFileDigest {
+  NormalizedFileDigest(IO &) {}
+  NormalizedFileDigest(IO &, const FileDigest &Digest) {
+HexString = llvm::toHex(Digest);
+  }
+
+  FileDigest denormalize(IO &I) {
+FileDigest Digest;
+if (HexString.size() == Digest.size() * 2 &&
+llvm::all_of(HexString, llvm::isHexDigit)) {
+  memcpy(Digest.data(), llvm::fromHex(HexString).data(), Digest.size());
+} else {
+  I.setError(std::string("Bad hex file digest: ") + HexString);
+}
+return Digest;
+  }
+
+  std::string HexString;
+};
+
+template <> struct MappingTraits {
+  static void mapping(IO &IO, IncludeGraphNode &Node) {
+IO.mapRequired("URI", Node.URI);
+MappingNormalization
+NSourceFlag(IO, Node.Flags);
+IO.mapRequired("Flags", NSourceFlag->Flag);
+MappingNormalization NDigest(IO,
+   Node.Digest);
+IO.mapRequired("Digest", NDigest->HexString);
+IO.mapRequired("DirectIncludes", Node.DirectIncludes);
+  }
+};
+
+template <> struct MappingTraits {
+  static void mapping(IO &IO, CompileCommandYAML &Cmd) {
+IO.mapRequired("Directory", Cmd.Directory);
+IO.mapRequired("CommandLine", Cmd.CommandLine);
+  }
+};
+
 template <> struct MappingTraits {
   static void mapping(IO &IO, VariantEntry &Variant

[PATCH] D77938: [clangd] Extend YAML Serialization

[Mark Nauwelaerts via Phabricator via cfe-commits]

mnauw added a comment.

I do not have commit access, so it would be helpful that you land this (and the 
other) ;-)


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77938/new/

https://reviews.llvm.org/D77938



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D90534: [clang-format] Add new option PenaltyIndentedWhitespace

[Mark Nauwelaerts via Phabricator via cfe-commits]

mnauw added a comment.

Thanks.  However, I do not have commit access, so someone may have to arrange 
for that  ...


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D90534/new/

https://reviews.llvm.org/D90534

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D90533: [clang-format] Always consider option PenaltyBreakBeforeFirstCallParameter

[Mark Nauwelaerts via Phabricator via cfe-commits]

mnauw abandoned this revision.
mnauw added a comment.

Looks like PenaltyBreakBeforeFirstCallParameter not being considered so well 
has been addressed somewhat differently in D90246 
, and so that does the job as well.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D90533/new/

https://reviews.llvm.org/D90533

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D90534: [clang-format] Add new option PenaltyIndentedWhitespace

[Mark Nauwelaerts via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rG1e4d6d1c1ff3: [clang-format] Add new option 
PenaltyIndentedWhitespace (authored by mnauw).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D90534/new/

https://reviews.llvm.org/D90534

Files:
  clang/docs/ClangFormatStyleOptions.rst
  clang/include/clang/Format/Format.h
  clang/lib/Format/ContinuationIndenter.cpp
  clang/lib/Format/Format.cpp
  clang/unittests/Format/FormatTest.cpp

Index: clang/unittests/Format/FormatTest.cpp
===
--- clang/unittests/Format/FormatTest.cpp
+++ clang/unittests/Format/FormatTest.cpp
@@ -17214,6 +17214,28 @@
Style);
 }
 
+TEST_F(FormatTest, PenaltyIndentedWhitespace) {
+  verifyFormat("Constructor()\n"
+   ": aa(aa), aa(\n"
+   "  (aa, "
+   "aat))");
+  verifyFormat("Constructor()\n"
+   ": a(aa), "
+   "aa(aa)");
+
+  FormatStyle StyleWithWhitespacePenalty = getLLVMStyle();
+  StyleWithWhitespacePenalty.PenaltyIndentedWhitespace = 5;
+  verifyFormat("Constructor()\n"
+   ": aa(aa),\n"
+   "  aa(\n"
+   "  (aa, aat))",
+   StyleWithWhitespacePenalty);
+  verifyFormat("Constructor()\n"
+   ": a(aa), "
+   "aa(aa)",
+   StyleWithWhitespacePenalty);
+}
+
 TEST_F(FormatTest, LLVMDefaultStyle) {
   FormatStyle Style = getLLVMStyle();
   verifyFormat("extern \"C\" {\n"
Index: clang/lib/Format/Format.cpp
===
--- clang/lib/Format/Format.cpp
+++ clang/lib/Format/Format.cpp
@@ -590,6 +590,8 @@
 IO.mapOptional("PenaltyExcessCharacter", Style.PenaltyExcessCharacter);
 IO.mapOptional("PenaltyReturnTypeOnItsOwnLine",
Style.PenaltyReturnTypeOnItsOwnLine);
+IO.mapOptional("PenaltyIndentedWhitespace",
+   Style.PenaltyIndentedWhitespace);
 IO.mapOptional("PointerAlignment", Style.PointerAlignment);
 IO.mapOptional("RawStringFormats", Style.RawStringFormats);
 IO.mapOptional("ReflowComments", Style.ReflowComments);
@@ -968,6 +970,7 @@
   LLVMStyle.PenaltyReturnTypeOnItsOwnLine = 60;
   LLVMStyle.PenaltyBreakBeforeFirstCallParameter = 19;
   LLVMStyle.PenaltyBreakTemplateDeclaration = prec::Relational;
+  LLVMStyle.PenaltyIndentedWhitespace = 0;
 
   LLVMStyle.DisableFormat = false;
   LLVMStyle.SortIncludes = true;
Index: clang/lib/Format/ContinuationIndenter.cpp
===
--- clang/lib/Format/ContinuationIndenter.cpp
+++ clang/lib/Format/ContinuationIndenter.cpp
@@ -785,6 +785,22 @@
 
   State.Column = getNewLineColumn(State);
 
+  // Add Penalty proportional to amount of whitespace away from FirstColumn
+  // This tends to penalize several lines that are far-right indented,
+  // and prefers a line-break prior to such a block, e.g:
+  //
+  // Constructor() :
+  //   member(value), long_member(
+  //  looong_call(param_1, param_2, param_3))
+  // would then become
+  // Constructor() :
+  //   member(value),
+  //   long_member(
+  //   looong_call(param_1, param_2, param_3))
+  if (State.Column > State.FirstIndent)
+Penalty +=
+Style.PenaltyIndentedWhitespace * (State.Column - State.FirstIndent);
+
   // Indent nested blocks relative to this column, unless in a very specific
   // JavaScript special case where:
   //
Index: clang/include/clang/Format/Format.h
===
--- clang/include/clang/Format/Format.h
+++ clang/include/clang/Format/Format.h
@@ -1913,6 +1913,10 @@
   /// line.
   unsigned PenaltyReturnTypeOnItsOwnLine;
 
+  /// Penalty for each character of whitespace indentation
+  /// (counted relative to leading non-whitespace column).
+  unsigned PenaltyIndentedWhitespace;
+
   /// The ``&`` and ``*`` alignment style.
   enum PointerAlignmentStyle {
 /// Align pointer to the left.
Index: clang/docs/ClangFormatStyleOptions.rst
===
--- clang/docs/ClangFormatStyleOptions.rst
+++ clang/docs/ClangFormatStyleOptions.rst
@@ -2301,6 +2301,10 @@
 **PenaltyExcessCharacter** (``unsigned``)
   The penalty for each character outside of the column limit.
 
+**PenaltyIndentedWhitespace** (``unsigned``)
+  Penalty for each character of whitespace indentation
+  (counted relative to leading n

[PATCH] D90533: [clang-format] Always consider option PenaltyBreakBeforeFirstCallParameter

[Mark Nauwelaerts via Phabricator via cfe-commits]

mnauw created this revision.
mnauw added a reviewer: sammccall.
mnauw added a project: clang-format.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
mnauw requested review of this revision.

If AlignAfterOpenBracket is set to BAS_DontAlign, then it turns out that 
PenaltyBreakBeforeFirstCallParameter is not considered (due to hard-coded 
shortcut, so to speak).

This patch attempts to preserve the current (specially crafted) behavior, while 
considering PenaltyBreakBeforeFirstCallParameter when really specified.  There 
may also be other ways to go about this to allow the user to actually set and 
use PenaltyBreakBeforeFirstCallParameter.  Failing all such options, it might 
otherwise be documented that PenaltyBreakBeforeFirstCallParameter has no effect 
in BAS_DontAlign.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D90533

Files:
  clang/lib/Format/Format.cpp
  clang/lib/Format/TokenAnnotator.cpp
  clang/unittests/Format/FormatTest.cpp

Index: clang/unittests/Format/FormatTest.cpp
===
--- clang/unittests/Format/FormatTest.cpp
+++ clang/unittests/Format/FormatTest.cpp
@@ -4478,6 +4478,7 @@
   Style.TabWidth = 4;
   Style.UseTab = FormatStyle::UT_Always;
   Style.AlignAfterOpenBracket = FormatStyle::BAS_DontAlign;
+  Style.PenaltyBreakBeforeFirstCallParameter = 0;
   Style.AlignOperands = FormatStyle::OAS_DontAlign;
   EXPECT_EQ("return someVeryVeryLongConditionThatBarelyFitsOnALine\n"
 "\t&& (someOtherLongishConditionPart1\n"
@@ -4628,6 +4629,7 @@
Style);
 
   Style.AlignAfterOpenBracket = FormatStyle::BAS_DontAlign;
+  Style.PenaltyBreakBeforeFirstCallParameter = 0;
   verifyFormat("return (a > b\n"
"// comment1\n"
"// comment2\n"
@@ -6028,6 +6030,7 @@
   " a));");
   FormatStyle Style = getLLVMStyle();
   Style.AlignAfterOpenBracket = FormatStyle::BAS_DontAlign;
+  Style.PenaltyBreakBeforeFirstCallParameter = 0;
   verifyFormat("void aa(\n"
"aaa , a aaa) {}",
Style);
@@ -6106,11 +6109,13 @@
"  bb);",
Style);
   Style.AlignAfterOpenBracket = FormatStyle::BAS_DontAlign;
+  Style.PenaltyBreakBeforeFirstCallParameter = 0;
   Style.AlignOperands = FormatStyle::OAS_Align;
   verifyFormat("int a = f(aa &&\n"
"  bb);",
Style);
   Style.AlignAfterOpenBracket = FormatStyle::BAS_DontAlign;
+  Style.PenaltyBreakBeforeFirstCallParameter = 0;
   Style.AlignOperands = FormatStyle::OAS_DontAlign;
   verifyFormat("int a = f(aa &&\n"
"bb);",
@@ -7460,6 +7465,7 @@
 TEST_F(FormatTest, WrapsTemplateParameters) {
   FormatStyle Style = getLLVMStyle();
   Style.AlignAfterOpenBracket = FormatStyle::BAS_DontAlign;
+  Style.PenaltyBreakBeforeFirstCallParameter = 0;
   Style.BreakBeforeBinaryOperators = FormatStyle::BOS_None;
   verifyFormat(
   "template  struct q {};\n"
@@ -7468,6 +7474,7 @@
   "y;",
   Style);
   Style.AlignAfterOpenBracket = FormatStyle::BAS_DontAlign;
+  Style.PenaltyBreakBeforeFirstCallParameter = 0;
   Style.BreakBeforeBinaryOperators = FormatStyle::BOS_All;
   verifyFormat(
   "template  struct r {};\n"
Index: clang/lib/Format/TokenAnnotator.cpp
===
--- clang/lib/Format/TokenAnnotator.cpp
+++ clang/lib/Format/TokenAnnotator.cpp
@@ -2740,7 +2740,7 @@
 return 100;
   if (Left.opensScope()) {
 if (Style.AlignAfterOpenBracket == FormatStyle::BAS_DontAlign)
-  return 0;
+  return Style.PenaltyBreakBeforeFirstCallParameter;
 if (Left.is(tok::l_brace) && !Style.Cpp11BracedListStyle)
   return 19;
 return Left.ParameterCount > 1 ? Style.PenaltyBreakBeforeFirstCallParameter
Index: clang/lib/Format/Format.cpp
===
--- clang/lib/Format/Format.cpp
+++ clang/lib/Format/Format.cpp
@@ -579,6 +579,13 @@
 IO.mapOptional("ObjCSpaceBeforeProtocolList",
Style.ObjCSpaceBeforeProtocolList);
 IO.mapOptional("PenaltyBreakAssignment", Style.PenaltyBreakAssignment);
+// If AlignAfterBracket is DontAlign,
+// adjust default PenaltyBreakBeforeFirstCallParameter to favor Newline.
+// Always accept any explicitly specified value though.
+if (!IO.outputting() &&
+Style.AlignAfterOpenBracket == FormatStyle::BAS_DontAlign) {
+  Style.PenaltyBreakBeforeFirstCallParameter = 0;
+}
 IO.mapOptional("PenaltyBreakBeforeFirstCallParameter",
Style.PenaltyBreakBeforeFirstCallParameter);
 IO.mapOptional("PenaltyBreakComment", Style.PenaltyBreakComment);

[PATCH] D90534: [clang-format] Add new option PenaltyIndentedWhitespace

[Mark Nauwelaerts via Phabricator via cfe-commits]

mnauw created this revision.
mnauw added a reviewer: sammccall.
mnauw added a project: clang-format.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
mnauw requested review of this revision.

As the example in the comment within the patch shows, the intention of 
yet-another-penalty is to discourage some "far-right-cascading" indentation 
cases.  Of course, mileage and taste may vary, so the default does not affect 
current behavior in any way.  Likewise, the option may well be named otherwise 
(etc).

I realize the patch is likely incomplete (and requires additional test and 
documentation changes), but it would first have to be considered useful and 
acceptable in the first place.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D90534

Files:
  clang/include/clang/Format/Format.h
  clang/lib/Format/ContinuationIndenter.cpp
  clang/lib/Format/Format.cpp


Index: clang/lib/Format/Format.cpp
===
--- clang/lib/Format/Format.cpp
+++ clang/lib/Format/Format.cpp
@@ -597,6 +597,8 @@
 IO.mapOptional("PenaltyExcessCharacter", Style.PenaltyExcessCharacter);
 IO.mapOptional("PenaltyReturnTypeOnItsOwnLine",
Style.PenaltyReturnTypeOnItsOwnLine);
+IO.mapOptional("PenaltyIndentedWhitespace",
+   Style.PenaltyIndentedWhitespace);
 IO.mapOptional("PointerAlignment", Style.PointerAlignment);
 IO.mapOptional("RawStringFormats", Style.RawStringFormats);
 IO.mapOptional("ReflowComments", Style.ReflowComments);
@@ -975,6 +977,7 @@
   LLVMStyle.PenaltyReturnTypeOnItsOwnLine = 60;
   LLVMStyle.PenaltyBreakBeforeFirstCallParameter = 19;
   LLVMStyle.PenaltyBreakTemplateDeclaration = prec::Relational;
+  LLVMStyle.PenaltyIndentedWhitespace = 0;
 
   LLVMStyle.DisableFormat = false;
   LLVMStyle.SortIncludes = true;
Index: clang/lib/Format/ContinuationIndenter.cpp
===
--- clang/lib/Format/ContinuationIndenter.cpp
+++ clang/lib/Format/ContinuationIndenter.cpp
@@ -783,6 +783,22 @@
 
   State.Column = getNewLineColumn(State);
 
+  // Add Penalty proportional to amount of whitespace away from FirstColumn
+  // This tends to penalize several lines that are far-right indented,
+  // and prefers a line-break prior to such a block, e.g:
+  //
+  // Constructor() :
+  //   member(value), long_member(
+  //  looong_call(param_1, param_2, param_3))
+  // would then become
+  // Constructor() :
+  //   member(value),
+  //   long_member(
+  //   looong_call(param_1, param_2, param_3))
+  if (State.Column > State.FirstIndent)
+Penalty +=
+Style.PenaltyIndentedWhitespace * (State.Column - State.FirstIndent);
+
   // Indent nested blocks relative to this column, unless in a very specific
   // JavaScript special case where:
   //
Index: clang/include/clang/Format/Format.h
===
--- clang/include/clang/Format/Format.h
+++ clang/include/clang/Format/Format.h
@@ -1913,6 +1913,9 @@
   /// line.
   unsigned PenaltyReturnTypeOnItsOwnLine;
 
+  /// Penalty for whitespace indentation
+  unsigned PenaltyIndentedWhitespace;
+
   /// The ``&`` and ``*`` alignment style.
   enum PointerAlignmentStyle {
 /// Align pointer to the left.


Index: clang/lib/Format/Format.cpp
===
--- clang/lib/Format/Format.cpp
+++ clang/lib/Format/Format.cpp
@@ -597,6 +597,8 @@
 IO.mapOptional("PenaltyExcessCharacter", Style.PenaltyExcessCharacter);
 IO.mapOptional("PenaltyReturnTypeOnItsOwnLine",
Style.PenaltyReturnTypeOnItsOwnLine);
+IO.mapOptional("PenaltyIndentedWhitespace",
+   Style.PenaltyIndentedWhitespace);
 IO.mapOptional("PointerAlignment", Style.PointerAlignment);
 IO.mapOptional("RawStringFormats", Style.RawStringFormats);
 IO.mapOptional("ReflowComments", Style.ReflowComments);
@@ -975,6 +977,7 @@
   LLVMStyle.PenaltyReturnTypeOnItsOwnLine = 60;
   LLVMStyle.PenaltyBreakBeforeFirstCallParameter = 19;
   LLVMStyle.PenaltyBreakTemplateDeclaration = prec::Relational;
+  LLVMStyle.PenaltyIndentedWhitespace = 0;
 
   LLVMStyle.DisableFormat = false;
   LLVMStyle.SortIncludes = true;
Index: clang/lib/Format/ContinuationIndenter.cpp
===
--- clang/lib/Format/ContinuationIndenter.cpp
+++ clang/lib/Format/ContinuationIndenter.cpp
@@ -783,6 +783,22 @@
 
   State.Column = getNewLineColumn(State);
 
+  // Add Penalty proportional to amount of whitespace away from FirstColumn
+  // This tends to penalize several lines that are far-right indented,
+  // and prefers a line-break prior to such a block, e.g:
+  //
+  // Constructor() :
+  //   member(value), long_member(
+  //  looong_

[PATCH] D90533: [clang-format] Always consider option PenaltyBreakBeforeFirstCallParameter

[Mark Nauwelaerts via Phabricator via cfe-commits]

mnauw added inline comments.



Comment at: clang/unittests/Format/FormatTest.cpp:4481
   Style.AlignAfterOpenBracket = FormatStyle::BAS_DontAlign;
+  Style.PenaltyBreakBeforeFirstCallParameter = 0;
   Style.AlignOperands = FormatStyle::OAS_DontAlign;

MyDeveloperDay wrote:
> Why do you need these? what is the default value of this as set in the LLVM 
> style?
The default value in the LLVM style is 19.  However, the fragment of code in 
TokenAnnotator.cpp basically hard-codes this to 0 (in case of BAS_DontAlign; 
see that part).  Since that part is now changed to really use 
PenaltyBreakBeforeFirstCallParameter style option, the latter is set to 0 in 
these cases to retain the original behavior.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D90533/new/

https://reviews.llvm.org/D90533

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D90534: [clang-format] Add new option PenaltyIndentedWhitespace

[Mark Nauwelaerts via Phabricator via cfe-commits]

mnauw updated this revision to Diff 302144.
mnauw added a comment.

Patch updated to include unit test and documentation changes (and to use full 
diff).


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D90534/new/

https://reviews.llvm.org/D90534

Files:
  clang/docs/ClangFormatStyleOptions.rst
  clang/include/clang/Format/Format.h
  clang/lib/Format/ContinuationIndenter.cpp
  clang/lib/Format/Format.cpp
  clang/unittests/Format/FormatTest.cpp

Index: clang/unittests/Format/FormatTest.cpp
===
--- clang/unittests/Format/FormatTest.cpp
+++ clang/unittests/Format/FormatTest.cpp
@@ -17167,6 +17167,28 @@
Style);
 }
 
+TEST_F(FormatTest, PenaltyIndentedWhitespace) {
+  verifyFormat("Constructor()\n"
+   ": aa(aa), aa(\n"
+   "  (aa, "
+   "aat))");
+  verifyFormat("Constructor()\n"
+   ": a(aa), "
+   "aa(aa)");
+
+  FormatStyle StyleWithWhitespacePenalty = getLLVMStyle();
+  StyleWithWhitespacePenalty.PenaltyIndentedWhitespace = 5;
+  verifyFormat("Constructor()\n"
+   ": aa(aa),\n"
+   "  aa(\n"
+   "  (aa, aat))",
+   StyleWithWhitespacePenalty);
+  verifyFormat("Constructor()\n"
+   ": a(aa), "
+   "aa(aa)",
+   StyleWithWhitespacePenalty);
+}
+
 TEST_F(FormatTest, LLVMDefaultStyle) {
   FormatStyle Style = getLLVMStyle();
   verifyFormat("extern \"C\" {\n"
Index: clang/lib/Format/Format.cpp
===
--- clang/lib/Format/Format.cpp
+++ clang/lib/Format/Format.cpp
@@ -597,6 +597,8 @@
 IO.mapOptional("PenaltyExcessCharacter", Style.PenaltyExcessCharacter);
 IO.mapOptional("PenaltyReturnTypeOnItsOwnLine",
Style.PenaltyReturnTypeOnItsOwnLine);
+IO.mapOptional("PenaltyIndentedWhitespace",
+   Style.PenaltyIndentedWhitespace);
 IO.mapOptional("PointerAlignment", Style.PointerAlignment);
 IO.mapOptional("RawStringFormats", Style.RawStringFormats);
 IO.mapOptional("ReflowComments", Style.ReflowComments);
@@ -975,6 +977,7 @@
   LLVMStyle.PenaltyReturnTypeOnItsOwnLine = 60;
   LLVMStyle.PenaltyBreakBeforeFirstCallParameter = 19;
   LLVMStyle.PenaltyBreakTemplateDeclaration = prec::Relational;
+  LLVMStyle.PenaltyIndentedWhitespace = 0;
 
   LLVMStyle.DisableFormat = false;
   LLVMStyle.SortIncludes = true;
Index: clang/lib/Format/ContinuationIndenter.cpp
===
--- clang/lib/Format/ContinuationIndenter.cpp
+++ clang/lib/Format/ContinuationIndenter.cpp
@@ -783,6 +783,22 @@
 
   State.Column = getNewLineColumn(State);
 
+  // Add Penalty proportional to amount of whitespace away from FirstColumn
+  // This tends to penalize several lines that are far-right indented,
+  // and prefers a line-break prior to such a block, e.g:
+  //
+  // Constructor() :
+  //   member(value), long_member(
+  //  looong_call(param_1, param_2, param_3))
+  // would then become
+  // Constructor() :
+  //   member(value),
+  //   long_member(
+  //   looong_call(param_1, param_2, param_3))
+  if (State.Column > State.FirstIndent)
+Penalty +=
+Style.PenaltyIndentedWhitespace * (State.Column - State.FirstIndent);
+
   // Indent nested blocks relative to this column, unless in a very specific
   // JavaScript special case where:
   //
Index: clang/include/clang/Format/Format.h
===
--- clang/include/clang/Format/Format.h
+++ clang/include/clang/Format/Format.h
@@ -1913,6 +1913,10 @@
   /// line.
   unsigned PenaltyReturnTypeOnItsOwnLine;
 
+  /// Penalty for each character of whitespace indentation
+  /// (counted relative to leading non-whitespace column).
+  unsigned PenaltyIndentedWhitespace;
+
   /// The ``&`` and ``*`` alignment style.
   enum PointerAlignmentStyle {
 /// Align pointer to the left.
Index: clang/docs/ClangFormatStyleOptions.rst
===
--- clang/docs/ClangFormatStyleOptions.rst
+++ clang/docs/ClangFormatStyleOptions.rst
@@ -2301,6 +2301,10 @@
 **PenaltyExcessCharacter** (``unsigned``)
   The penalty for each character outside of the column limit.
 
+**PenaltyIndentedWhitespace** (``unsigned``)
+  Penalty for each character of whitespace indentation
+  (counted relative to leading non-whitespace column).
+
 **PenaltyReturnTypeOnItsOwnLine** (``unsigned``)
   Penalty