[GitHub] [lucene] jpountz commented on a diff in pull request #780: LUCENE-10496: avoid unnecessary attempts to evaluate skipping doc if index sort and search sort are in opposite direction

2022-05-10 Thread GitBox
jpountz commented on code in PR #780: URL: https://github.com/apache/lucene/pull/780#discussion_r868900109 ## lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java: ## @@ -42,6 +42,8 @@ * but in this case you must override both of these methods. */

[GitHub] [lucene] jpountz commented on a diff in pull request #864: LUCENE-10555: avoid repeated NumericLeafComparator#setScorer calls

2022-05-10 Thread GitBox
jpountz commented on code in PR #864: URL: https://github.com/apache/lucene/pull/864#discussion_r868918283 ## lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java: ## @@ -165,10 +166,11 @@ public void copy(int slot, int doc) throws IOException {

[GitHub] [lucene] mocobeta commented on a diff in pull request #872: LUCENE-10527 Use 2*maxConn for last layer in HNSW

2022-05-10 Thread GitBox
mocobeta commented on code in PR #872: URL: https://github.com/apache/lucene/pull/872#discussion_r868923104 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java: ## @@ -43,7 +43,8 @@ public final class HnswGraphBuilder { /** Random seed for level generati

[GitHub] [lucene] jpountz commented on a diff in pull request #864: LUCENE-10555: avoid repeated NumericLeafComparator#setScorer calls

2022-05-10 Thread GitBox
jpountz commented on code in PR #864: URL: https://github.com/apache/lucene/pull/864#discussion_r868918283 ## lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java: ## @@ -165,10 +166,11 @@ public void copy(int slot, int doc) throws IOException {

[GitHub] [lucene] mocobeta commented on a diff in pull request #872: LUCENE-10527 Use 2*maxConn for last layer in HNSW

2022-05-10 Thread GitBox
mocobeta commented on code in PR #872: URL: https://github.com/apache/lucene/pull/872#discussion_r868923104 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java: ## @@ -43,7 +43,8 @@ public final class HnswGraphBuilder { /** Random seed for level generati

[GitHub] [lucene] mocobeta commented on a diff in pull request #872: LUCENE-10527 Use 2*maxConn for last layer in HNSW

2022-05-10 Thread GitBox
mocobeta commented on code in PR #872: URL: https://github.com/apache/lucene/pull/872#discussion_r868929217 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java: ## @@ -43,7 +43,8 @@ public final class HnswGraphBuilder { /** Random seed for level generati

[GitHub] [lucene] mocobeta commented on a diff in pull request #872: LUCENE-10527 Use 2*maxConn for last layer in HNSW

2022-05-10 Thread GitBox
mocobeta commented on code in PR #872: URL: https://github.com/apache/lucene/pull/872#discussion_r868929217 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java: ## @@ -43,7 +43,8 @@ public final class HnswGraphBuilder { /** Random seed for level generati

[GitHub] [lucene] jpountz opened a new pull request, #876: LUCENE-9356: Change test to detect mismatched checksums instead of byte flips.

2022-05-10 Thread GitBox
jpountz opened a new pull request, #876: URL: https://github.com/apache/lucene/pull/876 This makes the test more robust and gives a good sense of whether file formats are implementing `checkIntegrity` correctly. -- This is an automated message from the Apache Git Service. To respond to

[jira] [Commented] (LUCENE-9356) Add tests for corruptions caused by byte flips

2022-05-10 Thread Adrien Grand (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-9356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534216#comment-17534216 ] Adrien Grand commented on LUCENE-9356: -- This test does try to account for CRC32 col

[GitHub] [lucene] wjp719 commented on a diff in pull request #864: LUCENE-10555: avoid NumericLeafComparator#iteratorCost repeated initialization when NumericLeafComparator#setScorer is called

2022-05-10 Thread GitBox
wjp719 commented on code in PR #864: URL: https://github.com/apache/lucene/pull/864#discussion_r868964284 ## lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java: ## @@ -165,10 +166,11 @@ public void copy(int slot, int doc) throws IOException {

[GitHub] [lucene] wjp719 commented on pull request #864: LUCENE-10555: avoid NumericLeafComparator#iteratorCost repeated initialization when NumericLeafComparator#setScorer is called

2022-05-10 Thread GitBox
wjp719 commented on PR #864: URL: https://github.com/apache/lucene/pull/864#issuecomment-1122089465 @jpountz Hi, I have add the change entry. please review again, thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [lucene] jpountz merged pull request #864: LUCENE-10555: avoid NumericLeafComparator#iteratorCost repeated initialization when NumericLeafComparator#setScorer is called

2022-05-10 Thread GitBox
jpountz merged PR #864: URL: https://github.com/apache/lucene/pull/864 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.

[jira] [Commented] (LUCENE-10555) avoid repeated NumericLeafComparator setScorer calls

2022-05-10 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534290#comment-17534290 ] ASF subversion and git services commented on LUCENE-10555: -- Co

[GitHub] [lucene] jpountz merged pull request #863: LUCENE-10554: fix bkd test case logic error and java doc error

2022-05-10 Thread GitBox
jpountz merged PR #863: URL: https://github.com/apache/lucene/pull/863 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.

[GitHub] [lucene] rmuir commented on pull request #876: LUCENE-9356: Change test to detect mismatched checksums instead of byte flips.

2022-05-10 Thread GitBox
rmuir commented on PR #876: URL: https://github.com/apache/lucene/pull/876#issuecomment-1122257235 do we really need the linefiledocs? If the test has rare failures, we'll want to reproduce them. practically, linefiledocs just prevents reproducibility (uses a different file in CI th

[GitHub] [lucene] gsmiller commented on a diff in pull request #779: LUCENE-10488: Optimize Facets#getTopDims in IntTaxonomyFacets

2022-05-10 Thread GitBox
gsmiller commented on code in PR #779: URL: https://github.com/apache/lucene/pull/779#discussion_r868497732 ## lucene/facet/src/java/org/apache/lucene/facet/taxonomy/TaxonomyFacets.java: ## @@ -109,7 +109,23 @@ public boolean childrenLoaded() { * @lucene.experimental */

[jira] [Created] (LUCENE-10565) Can we "warm" SSDV ordinal maps on index reopen?

2022-05-10 Thread Greg Miller (Jira)
Greg Miller created LUCENE-10565: Summary: Can we "warm" SSDV ordinal maps on index reopen? Key: LUCENE-10565 URL: https://issues.apache.org/jira/browse/LUCENE-10565 Project: Lucene - Core Is

[jira] [Commented] (LUCENE-10555) avoid repeated NumericLeafComparator setScorer calls

2022-05-10 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534311#comment-17534311 ] ASF subversion and git services commented on LUCENE-10555: -- Co

[GitHub] [lucene] jpountz commented on pull request #875: LUCENE-10560: Speed up OrdinalMap construction a bit.

2022-05-10 Thread GitBox
jpountz commented on PR #875: URL: https://github.com/apache/lucene/pull/875#issuecomment-1122324472 As a data point, I ran the current version of the benchmark at https://github.com/mikemccand/luceneutil/pull/170 and got the following numbers: ``` Before: id: 12556.30082 msec

[GitHub] [lucene] dweiss commented on a diff in pull request #875: LUCENE-10560: Speed up OrdinalMap construction a bit.

2022-05-10 Thread GitBox
dweiss commented on code in PR #875: URL: https://github.com/apache/lucene/pull/875#discussion_r869191445 ## lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java: ## @@ -48,10 +49,69 @@ public class OrdinalMap implements Accountable { // need it // TODO: use more e

[GitHub] [lucene] rmuir commented on a diff in pull request #875: LUCENE-10560: Speed up OrdinalMap construction a bit.

2022-05-10 Thread GitBox
rmuir commented on code in PR #875: URL: https://github.com/apache/lucene/pull/875#discussion_r869220064 ## lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java: ## @@ -48,10 +49,69 @@ public class OrdinalMap implements Accountable { // need it // TODO: use more ef

[GitHub] [lucene] mayya-sharipova commented on a diff in pull request #872: LUCENE-10527 Use 2*maxConn for last layer in HNSW

2022-05-10 Thread GitBox
mayya-sharipova commented on code in PR #872: URL: https://github.com/apache/lucene/pull/872#discussion_r869226440 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java: ## @@ -43,7 +43,8 @@ public final class HnswGraphBuilder { /** Random seed for level g

[GitHub] [lucene] rmuir commented on pull request #874: LUCENE-10471 Increse max dims for vectors to 2048

2022-05-10 Thread GitBox
rmuir commented on PR #874: URL: https://github.com/apache/lucene/pull/874#issuecomment-1122395334 My concerns are on the JIRA issue, I don't want them to be forgotten. https://issues.apache.org/jira/browse/LUCENE-10471 I don't know how we can say "we will not recommend further increa

[GitHub] [lucene] dweiss commented on a diff in pull request #875: LUCENE-10560: Speed up OrdinalMap construction a bit.

2022-05-10 Thread GitBox
dweiss commented on code in PR #875: URL: https://github.com/apache/lucene/pull/875#discussion_r869363854 ## lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java: ## @@ -48,10 +49,69 @@ public class OrdinalMap implements Accountable { // need it // TODO: use more e

[GitHub] [lucene] jpountz commented on a diff in pull request #875: LUCENE-10560: Speed up OrdinalMap construction a bit.

2022-05-10 Thread GitBox
jpountz commented on code in PR #875: URL: https://github.com/apache/lucene/pull/875#discussion_r869377332 ## lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java: ## @@ -48,10 +49,69 @@ public class OrdinalMap implements Accountable { // need it // TODO: use more

[GitHub] [lucene] dweiss commented on a diff in pull request #875: LUCENE-10560: Speed up OrdinalMap construction a bit.

2022-05-10 Thread GitBox
dweiss commented on code in PR #875: URL: https://github.com/apache/lucene/pull/875#discussion_r869397640 ## lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java: ## @@ -48,10 +49,69 @@ public class OrdinalMap implements Accountable { // need it // TODO: use more e

[GitHub] [lucene] jpountz commented on a diff in pull request #875: LUCENE-10560: Speed up OrdinalMap construction a bit.

2022-05-10 Thread GitBox
jpountz commented on code in PR #875: URL: https://github.com/apache/lucene/pull/875#discussion_r869429459 ## lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java: ## @@ -48,10 +49,69 @@ public class OrdinalMap implements Accountable { // need it // TODO: use more

[GitHub] [lucene] rmuir commented on a diff in pull request #875: LUCENE-10560: Speed up OrdinalMap construction a bit.

2022-05-10 Thread GitBox
rmuir commented on code in PR #875: URL: https://github.com/apache/lucene/pull/875#discussion_r869459393 ## lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java: ## @@ -48,10 +49,69 @@ public class OrdinalMap implements Accountable { // need it // TODO: use more ef

[GitHub] [lucene] gsmiller commented on pull request #777: LUCENE-10488: Optimize Facets#getTopDims in ConcurrentSortedSetDocValuesFacetCounts

2022-05-10 Thread GitBox
gsmiller commented on PR #777: URL: https://github.com/apache/lucene/pull/777#issuecomment-1122652678 > just an observation, this is a large amount of code changes for performance change that may be in the noise? I'm a bit confused. I don't think we actually have benchmarks for this f

[GitHub] [lucene] Yuti-G commented on pull request #777: LUCENE-10488: Optimize Facets#getTopDims in ConcurrentSortedSetDocValuesFacetCounts

2022-05-10 Thread GitBox
Yuti-G commented on PR #777: URL: https://github.com/apache/lucene/pull/777#issuecomment-1122671065 Hi @rmuir and @gsmiller, I haven't added `getTopDims` to benchmarks yet, but I plan to add it once all the PRs in LUCENE-10488 are merged. Thanks! -- This is an automated message from the A

[GitHub] [lucene] rmuir commented on pull request #777: LUCENE-10488: Optimize Facets#getTopDims in ConcurrentSortedSetDocValuesFacetCounts

2022-05-10 Thread GitBox
rmuir commented on PR #777: URL: https://github.com/apache/lucene/pull/777#issuecomment-1122687335 Thanks both of you for the explanation, sorry for the confusion! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [lucene] mayya-sharipova merged pull request #870: LUCENE-10502: Refactor hnswVectors format

2022-05-10 Thread GitBox
mayya-sharipova merged PR #870: URL: https://github.com/apache/lucene/pull/870 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene

[GitHub] [lucene] mayya-sharipova commented on pull request #870: LUCENE-10502: Refactor hnswVectors format

2022-05-10 Thread GitBox
mayya-sharipova commented on PR #870: URL: https://github.com/apache/lucene/pull/870#issuecomment-1122736714 @msokolov In the interest of time, I will be merging this PR to the feature branch `vectors-disi-direct`. There will an opportunity to provide more feedback once @LuXugang opens a ne

[GitHub] [lucene] mayya-sharipova commented on pull request #870: LUCENE-10502: Refactor hnswVectors format

2022-05-10 Thread GitBox
mayya-sharipova commented on PR #870: URL: https://github.com/apache/lucene/pull/870#issuecomment-1122737290 @LuXugang Please feel free to open a new PR to merge vectors-disi-direct feature branch to the main branch when you have time. -- This is an automated message from the Apache Git S

[jira] [Commented] (LUCENE-10502) Use IndexedDISI to store docIds and DirectMonotonicWriter/Reader to handle ordToDoc

2022-05-10 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534529#comment-17534529 ] ASF subversion and git services commented on LUCENE-10502: -- Co

[GitHub] [lucene] LuXugang opened a new pull request, #877: LUCENE-10502: Use IndexedDISI to store docIds and DirectMonotonicWriter/Reader to handle ordToDoc

2022-05-10 Thread GitBox
LuXugang opened a new pull request, #877: URL: https://github.com/apache/lucene/pull/877 follow up of https://github.com/apache/lucene/pull/792 and https://github.com/apache/lucene/pull/870 -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [lucene] LuXugang commented on pull request #870: LUCENE-10502: Refactor hnswVectors format

2022-05-10 Thread GitBox
LuXugang commented on PR #870: URL: https://github.com/apache/lucene/pull/870#issuecomment-1122770417 Thanks @mayya-sharipova , let's move to https://github.com/apache/lucene/pull/877 to continue this change. -- This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [lucene] gsmiller commented on pull request #843: LUCENE-10538: TopN is not being used in getTopChildren in RangeFacetCounts

2022-05-10 Thread GitBox
gsmiller commented on PR #843: URL: https://github.com/apache/lucene/pull/843#issuecomment-1122824451 Thanks @Yuti-G ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [lucene] jtibshirani commented on a diff in pull request #872: LUCENE-10527 Use 2*maxConn for last layer in HNSW

2022-05-10 Thread GitBox
jtibshirani commented on code in PR #872: URL: https://github.com/apache/lucene/pull/872#discussion_r869686535 ## lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java: ## @@ -43,7 +43,8 @@ public final class HnswGraphBuilder { /** Random seed for level gener

[GitHub] [lucene] vigyasharma commented on a diff in pull request #633: LUCENE-10216: Use MergeScheduler and MergePolicy to run addIndexes(CodecReader[]) merges.

2022-05-10 Thread GitBox
vigyasharma commented on code in PR #633: URL: https://github.com/apache/lucene/pull/633#discussion_r869769905 ## lucene/test-framework/src/java/org/apache/lucene/tests/index/MockRandomMergePolicy.java: ## @@ -86,6 +86,20 @@ public MergeSpecification findMerges( return merg

[jira] [Created] (LUCENE-10566) Create an @Concurrent test group for concurrency related tests

2022-05-10 Thread Vigya Sharma (Jira)
Vigya Sharma created LUCENE-10566: - Summary: Create an @Concurrent test group for concurrency related tests Key: LUCENE-10566 URL: https://issues.apache.org/jira/browse/LUCENE-10566 Project: Lucene -

[GitHub] [lucene] wjp719 commented on pull request #780: LUCENE-10496: avoid unnecessary attempts to evaluate skipping doc if index sort and search sort are in opposite direction

2022-05-10 Thread GitBox
wjp719 commented on PR #780: URL: https://github.com/apache/lucene/pull/780#issuecomment-1123104829 @jpountz Hi, I modify the logic. The skip interval is only changed when `updateCounter` is bigger than 256, and the speed of double skip interval is 3 times slower than the speed of divide s

[GitHub] [lucene] wjp719 opened a new pull request, #878: LUCENE-10555: fix NumericComparator#iteratorCost initial logic error

2022-05-10 Thread GitBox
wjp719 opened a new pull request, #878: URL: https://github.com/apache/lucene/pull/878 #864 judges if `NumericComparator#iteratorCost` is -1 or not to determine whether to reset `NumericComparator#iteratorCost` in `NumericComparator#setScorer`, but `NumericComparator#iteratorCost` is set a

[GitHub] [lucene] wjp719 commented on pull request #878: LUCENE-10555: fix NumericComparator#iteratorCost initial logic error

2022-05-10 Thread GitBox
wjp719 commented on PR #878: URL: https://github.com/apache/lucene/pull/878#issuecomment-1123114722 @jpountz I see your commit [8476ac1f6ac5258052f93e85278f70770fe3dfd5](https://github.com/apache/lucene/commit/8476ac1f6ac5258052f93e85278f70770fe3dfd5) to solve tests error, maybe is cause b

[GitHub] [lucene] Yuti-G commented on a diff in pull request #779: LUCENE-10488: Optimize Facets#getTopDims in IntTaxonomyFacets

2022-05-10 Thread GitBox
Yuti-G commented on code in PR #779: URL: https://github.com/apache/lucene/pull/779#discussion_r869834570 ## lucene/facet/src/java/org/apache/lucene/facet/taxonomy/IntTaxonomyFacets.java: ## @@ -169,18 +176,54 @@ public FacetResult getTopChildren(int topN, String dim, String...

[GitHub] [lucene] wormday opened a new pull request, #879: fix errors in DocStats comments

2022-05-10 Thread GitBox
wormday opened a new pull request, #879: URL: https://github.com/apache/lucene/pull/879 Comments for two fields of DocStats are reversed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [lucene] Yuti-G commented on pull request #806: LUCENE-10488: Optimize Facets#getTopDims in FloatTaxonomyFacets

2022-05-10 Thread GitBox
Yuti-G commented on PR #806: URL: https://github.com/apache/lucene/pull/806#issuecomment-1123167787 Thanks! Please see the latest commit for the update. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [lucene] jpountz merged pull request #878: LUCENE-10555: fix NumericComparator#iteratorCost initial logic error

2022-05-10 Thread GitBox
jpountz merged PR #878: URL: https://github.com/apache/lucene/pull/878 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.

[jira] [Commented] (LUCENE-10555) avoid repeated NumericLeafComparator setScorer calls

2022-05-10 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534699#comment-17534699 ] ASF subversion and git services commented on LUCENE-10555: -- Co

[jira] [Commented] (LUCENE-10555) avoid repeated NumericLeafComparator setScorer calls

2022-05-10 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/LUCENE-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534702#comment-17534702 ] ASF subversion and git services commented on LUCENE-10555: -- Co

[GitHub] [lucene] jpountz commented on a diff in pull request #780: LUCENE-10496: avoid unnecessary attempts to evaluate skipping doc if index sort and search sort are in opposite direction

2022-05-10 Thread GitBox
jpountz commented on code in PR #780: URL: https://github.com/apache/lucene/pull/780#discussion_r869932548 ## lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java: ## @@ -269,11 +276,23 @@ public PointValues.Relation compare(byte[] minPackedValue, by