[GitHub] [lucene] MarcusSorealheis commented on a diff in pull request #12254: add ConcurrentOnHeapHnswGraph and Builder

2023-05-01 Thread via GitHub
MarcusSorealheis commented on code in PR #12254: URL: https://github.com/apache/lucene/pull/12254#discussion_r1182114316 ## lucene/core/src/java/org/apache/lucene/util/hnsw/ConcurrentOnHeapHnswGraph.java: ## @@ -0,0 +1,280 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [lucene] MarcusSorealheis commented on a diff in pull request #12254: add ConcurrentOnHeapHnswGraph and Builder

2023-05-01 Thread via GitHub
MarcusSorealheis commented on code in PR #12254: URL: https://github.com/apache/lucene/pull/12254#discussion_r1182114316 ## lucene/core/src/java/org/apache/lucene/util/hnsw/ConcurrentOnHeapHnswGraph.java: ## @@ -0,0 +1,280 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [lucene] MarcusSorealheis commented on a diff in pull request #12254: add ConcurrentOnHeapHnswGraph and Builder

2023-05-01 Thread via GitHub
MarcusSorealheis commented on code in PR #12254: URL: https://github.com/apache/lucene/pull/12254#discussion_r1182114316 ## lucene/core/src/java/org/apache/lucene/util/hnsw/ConcurrentOnHeapHnswGraph.java: ## @@ -0,0 +1,280 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [lucene] jbellis commented on pull request #12254: add ConcurrentOnHeapHnswGraph and Builder

2023-05-01 Thread via GitHub
jbellis commented on PR #12254: URL: https://github.com/apache/lucene/pull/12254#issuecomment-1530167402 thanks again. and you're right, the ram usage needed a rewrite, it is passing now with the seed you found. -- This is an automated message from the Apache Git Service. To respon

[GitHub] [lucene] tang-hi commented on pull request #12254: add ConcurrentOnHeapHnswGraph and Builder

2023-05-01 Thread via GitHub
tang-hi commented on PR #12254: URL: https://github.com/apache/lucene/pull/12254#issuecomment-1530005788 > Thanks! For me it is working: > > ``` > $ ./gradlew -p lucene/core test --tests "*Hnsw*" > ... > :lucene:core:test (SUCCESS): 75 test(s) > ``` > > Note that add

[GitHub] [lucene] jbellis commented on pull request #12254: add ConcurrentOnHeapHnswGraph and Builder

2023-05-01 Thread via GitHub
jbellis commented on PR #12254: URL: https://github.com/apache/lucene/pull/12254#issuecomment-1529956854 Thanks! For me it is working: ``` $ ./gradlew -p lucene/core test --tests "*Hnsw*" ... :lucene:core:test (SUCCESS): 75 test(s) ``` Note that adding these args do

[GitHub] [lucene] tang-hi commented on pull request #12254: add ConcurrentOnHeapHnswGraph and Builder

2023-05-01 Thread via GitHub
tang-hi commented on PR #12254: URL: https://github.com/apache/lucene/pull/12254#issuecomment-1529914311 > I've made a couple optimizations and merged to main. > > With the optimizations it's still about 38% slower than the non-concurrent code (including the HashMap optimization for t

[GitHub] [lucene] jbellis commented on pull request #12254: add ConcurrentOnHeapHnswGraph and Builder

2023-05-01 Thread via GitHub
jbellis commented on PR #12254: URL: https://github.com/apache/lucene/pull/12254#issuecomment-1529909165 Replaced parallel stream with ExecutorService. Only ThreadPoolExecutor is supported, since it knows how many thread it has available. After a quick look that is the only type I see Luc

[GitHub] [lucene] jbellis commented on pull request #12254: add ConcurrentOnHeapHnswGraph and Builder

2023-05-01 Thread via GitHub
jbellis commented on PR #12254: URL: https://github.com/apache/lucene/pull/12254#issuecomment-1529796265 I've made a couple optimizations and merged to main. With the optimizations it's still about 38% slower than the non-concurrent code (including the HashMap optimization for the non