mikemccand commented on code in PR #14765:
URL: https://github.com/apache/lucene/pull/14765#discussion_r2137796082
##########
lucene/core/src/java/org/apache/lucene/util/hnsw/OnHeapHnswGraph.java:
##########
@@ -32,6 +32,15 @@
*/
public final class OnHeapHnswGraph extends HnswGraph implements Accountable {
+ private static final long RAM_BYTES_USED =
+ 4L * Integer.BYTES // all int fields
+ + 1 // field: noGrowth
+ + RamUsageEstimator.NUM_BYTES_OBJECT_REF
Review Comment:
Is this for the `graph` pointer we hold?
##########
lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java:
##########
@@ -444,7 +444,6 @@ void finish() throws IOException {
// see: https://github.com/apache/lucene/issues/14214
// connectComponents();
frozen = true;
- hnsw.finishBuild();
Review Comment:
Hmm, why remove this?
##########
lucene/core/src/java/org/apache/lucene/util/hnsw/OnHeapHnswGraph.java:
##########
@@ -158,21 +168,30 @@ public void addNode(int level, int node) {
size.incrementAndGet();
}
if (level == 0) {
- graph[node][level] = new NeighborArray(nsize0, true);
+ graph[node][level] =
+ new NeighborArray(
+ nsize0,
+ true,
+ l -> {
+ long bytesUsed = graphRamBytesUsed;
+ graphRamBytesUsed = bytesUsed + l;
+ assert l > 0;
Review Comment:
Why both asserts? Maybe move this `assert` up to the top of the lambda?
##########
lucene/core/src/java/org/apache/lucene/util/hnsw/OnHeapHnswGraph.java:
##########
@@ -32,6 +32,15 @@
*/
public final class OnHeapHnswGraph extends HnswGraph implements Accountable {
+ private static final long RAM_BYTES_USED =
+ 4L * Integer.BYTES // all int fields
+ + 1 // field: noGrowth
+ + RamUsageEstimator.NUM_BYTES_OBJECT_REF
+ + RamUsageEstimator.NUM_BYTES_OBJECT_HEADER
+ + 2 * Integer.BYTES // field: entryNode
+ + 3L * (Integer.BYTES + RamUsageEstimator.NUM_BYTES_OBJECT_HEADER)
// 3 AtomicInteger
Review Comment:
Maybe as Claude or Q or CoPiolot or ChatGPT or Gemini or so to generate this
big long `RAM_BYTES_USED`? It's so error proned and time consuming for we
humans ... maybe genai can do it one-time quickly / more reliably for this PR.
##########
lucene/core/src/java/org/apache/lucene/util/hnsw/OnHeapHnswGraph.java:
##########
@@ -32,6 +32,15 @@
*/
public final class OnHeapHnswGraph extends HnswGraph implements Accountable {
+ private static final long RAM_BYTES_USED =
+ 4L * Integer.BYTES // all int fields
+ + 1 // field: noGrowth
+ + RamUsageEstimator.NUM_BYTES_OBJECT_REF
+ + RamUsageEstimator.NUM_BYTES_OBJECT_HEADER
+ + 2 * Integer.BYTES // field: entryNode
+ + 3L * (Integer.BYTES + RamUsageEstimator.NUM_BYTES_OBJECT_HEADER)
// 3 AtomicInteger
Review Comment:
Also add in `RamUsageEstimator.NUM_BYTES_OBJECT_REF` for the pointer to
these `AtomicInteger`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
