sammccall created this revision.
sammccall added a reviewer: adamcz.
Herald added subscribers: kadircet, arphaman.
Herald added a project: All.
sammccall requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.
SymbolSlab::Builder has an arena to store strings of owned symbols, and
deduplicates them. build() copies all the strings and deduplicates them again!
This is potentially useful: we may have overwritten a symbol and
rendered some strings unreachable.
However in practice this is not the case. When testing on a variety of
files in LLVM (e.g. SemaExpr.cpp), the strings for the full preamble
index are 3MB and shrink by 0.4% (12KB). For comparison the serializde
preamble is >50MB.
There are also hundreds of smaller slabs (file sharding) that do not shrink at
all.
CPU time spent on this is significant (something like 3-5% of buildPreamble).
We're better off not bothering.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D135231
Files:
clang-tools-extra/clangd/index/Symbol.cpp
Index: clang-tools-extra/clangd/index/Symbol.cpp
===================================================================
--- clang-tools-extra/clangd/index/Symbol.cpp
+++ clang-tools-extra/clangd/index/Symbol.cpp
@@ -61,12 +61,9 @@
SortedSymbols.push_back(std::move(Entry.second));
llvm::sort(SortedSymbols,
[](const Symbol &L, const Symbol &R) { return L.ID < R.ID; });
- // We may have unused strings from overwritten symbols. Build a new arena.
- llvm::BumpPtrAllocator NewArena;
- llvm::UniqueStringSaver Strings(NewArena);
- for (auto &S : SortedSymbols)
- own(S, Strings);
- return SymbolSlab(std::move(NewArena), std::move(SortedSymbols));
+ // We may have unused strings from overwritten symbols.
+ // In practice, these are extremely small, it's not worth compacting.
+ return SymbolSlab(std::move(Arena), std::move(SortedSymbols));
}
llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, const SymbolSlab &Slab) {
Index: clang-tools-extra/clangd/index/Symbol.cpp
===================================================================
--- clang-tools-extra/clangd/index/Symbol.cpp
+++ clang-tools-extra/clangd/index/Symbol.cpp
@@ -61,12 +61,9 @@
SortedSymbols.push_back(std::move(Entry.second));
llvm::sort(SortedSymbols,
[](const Symbol &L, const Symbol &R) { return L.ID < R.ID; });
- // We may have unused strings from overwritten symbols. Build a new arena.
- llvm::BumpPtrAllocator NewArena;
- llvm::UniqueStringSaver Strings(NewArena);
- for (auto &S : SortedSymbols)
- own(S, Strings);
- return SymbolSlab(std::move(NewArena), std::move(SortedSymbols));
+ // We may have unused strings from overwritten symbols.
+ // In practice, these are extremely small, it's not worth compacting.
+ return SymbolSlab(std::move(Arena), std::move(SortedSymbols));
}
llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, const SymbolSlab &Slab) {
_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
