[PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
ChrisHegarty opened a new pull request, #13119: URL: https://github.com/apache/lucene/pull/13119 This commit updates the FieldInfosFormat translation of vector similarity functions to be independent of the `VectorSimilartyFunction` enum. The `VectorSimilartyFunction` enum lives outsid

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
ChrisHegarty commented on PR #13119: URL: https://github.com/apache/lucene/pull/13119#issuecomment-1953858737 This PR is a prerequisite for future work to make the similarity function symbolic and lookup-able, see https://github.com/apache/lucene/pull/13076#issuecomment-1930363479. -- Th

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
ChrisHegarty commented on code in PR #13119: URL: https://github.com/apache/lucene/pull/13119#discussion_r1495556754 ## lucene/core/src/java/org/apache/lucene/codecs/lucene94/Lucene94FieldInfosFormat.java: ## @@ -111,6 +112,8 @@ * 0: EUCLIDEAN distance. ({@link Vector

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
tteofili commented on code in PR #13119: URL: https://github.com/apache/lucene/pull/13119#discussion_r1495629501 ## lucene/core/src/java/org/apache/lucene/codecs/lucene94/Lucene94FieldInfosFormat.java: ## @@ -111,6 +112,8 @@ * 0: EUCLIDEAN distance. ({@link VectorSimi

[I] org.apache.lucene.search.TestReqOptSumScorer.testFilterRandomRareOpt fails intermittently [lucene]

2024-02-20 Thread via GitHub
ChrisHegarty opened a new issue, #13120: URL: https://github.com/apache/lucene/issues/13120 ### Description ``` org.apache.lucene.search.TestReqOptSumScorer > testFilterRandomRareOpt FAILED java.lang.NullPointerException: Cannot invoke "org.apache.lucene.search.Scorer.twoPha

Re: [I] org.apache.lucene.search.TestReqOptSumScorer.testFilterRandomRareOpt fails intermittently [lucene]

2024-02-20 Thread via GitHub
ChrisHegarty commented on issue #13120: URL: https://github.com/apache/lucene/issues/13120#issuecomment-1954031628 I believe that this is a test bug. If you look further up in `doCheckMaxScores`, (in the previous for loop) there is an explicit check for `s2 == null`, which appears to be cor

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
uschindler commented on PR #13119: URL: https://github.com/apache/lucene/pull/13119#issuecomment-1954105604 > Index format wise, I think the index corruption can occur when reading a Lucene 9.8.0 index with Lucene 9.7.0, as the format would allow that, but I am not sure this is an expected

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
ChrisHegarty commented on PR #13119: URL: https://github.com/apache/lucene/pull/13119#issuecomment-1954132655 > Hi, as stated in the other issue: I am not really happy to have that enum at all! The similarity/distance functions should be pluggable using `NamedSPILoader`. To implement that,

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
ChrisHegarty commented on PR #13119: URL: https://github.com/apache/lucene/pull/13119#issuecomment-1954136856 > > Index format wise, I think the index corruption can occur when reading a Lucene 9.8.0 index with Lucene 9.7.0, as the format would allow that, but I am not sure this is an expec

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
uschindler commented on PR #13119: URL: https://github.com/apache/lucene/pull/13119#issuecomment-1954160317 > > Hi, as stated in the other issue: I am not really happy to have that enum at all! The similarity/distance functions should be pluggable using `NamedSPILoader`. To implement that,

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
ChrisHegarty commented on PR #13119: URL: https://github.com/apache/lucene/pull/13119#issuecomment-1954278599 I see now that we have a similar dependency in `Lucene99HnswVectorsReader`. I'll update in a similar way. -- This is an automated message from the Apache Git Service. To respond t

Re: [I] org.apache.lucene.search.TestFloatVectorSimilarityQuery.testVectorsAboveSimilarity fails intermittently [lucene]

2024-02-20 Thread via GitHub
benwtrent closed issue #12955: org.apache.lucene.search.TestFloatVectorSimilarityQuery.testVectorsAboveSimilarity fails intermittently URL: https://github.com/apache/lucene/issues/12955 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[PR] SOLR-17168: Add netty-transport-classes-epoll [lucene-solr]

2024-02-20 Thread via GitHub
colvinco opened a new pull request, #2684: URL: https://github.com/apache/lucene-solr/pull/2684 I've checked that the jar appears where I expect it to (solrj-lib and the WEB-INF/lib). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] SOLR-17168: Add netty-transport-classes-epoll [lucene-solr]

2024-02-20 Thread via GitHub
janhoy merged PR #2684: URL: https://github.com/apache/lucene-solr/pull/2684 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.a

Re: [PR] Short circuit queued flush check when flush on update is disabled [lucene]

2024-02-20 Thread via GitHub
jpountz commented on PR #13115: URL: https://github.com/apache/lucene/pull/13115#issuecomment-1954568783 FYI I played with this change and [IndexGeoNames](https://github.com/mikemccand/luceneutil/blob/master/src/main/perf/IndexGeoNames.java) which has been good at detecting contention in th

Re: [PR] Support getMaxScore of DisjunctionSumScorer for non top level scoring clause [lucene]

2024-02-20 Thread via GitHub
jpountz commented on PR #13066: URL: https://github.com/apache/lucene/pull/13066#issuecomment-1954572850 This indeed! It's a bit manual, but it's what powers nightly benchmarks and what we run to check the performance impact of the changes that we merge. -- This is an automated message fr

Re: [PR] Added queued flush check optimization to changes.txt [lucene]

2024-02-20 Thread via GitHub
jpountz merged PR #13118: URL: https://github.com/apache/lucene/pull/13118 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apa

Re: [PR] Added queued flush check optimization to changes.txt [lucene]

2024-02-20 Thread via GitHub
mikemccand commented on PR #13118: URL: https://github.com/apache/lucene/pull/13118#issuecomment-1954617499 Thank you @CaptainDredge and @jpountz! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] Add mult-leaf optimizations for diversify children collector [lucene]

2024-02-20 Thread via GitHub
benwtrent opened a new pull request, #13121: URL: https://github.com/apache/lucene/pull/13121 This adds multi-leaf optimizations for diversified children collector. This means as children vectors are collected within a block join, we can share information between leaves to speed up vector s

Re: [I] Better wire up HNSW concurrent merge config [lucene]

2024-02-20 Thread via GitHub
benwtrent commented on issue #12740: URL: https://github.com/apache/lucene/issues/12740#issuecomment-1954964162 @jpountz @zhaih Looking at @dweiss old proposal I am thinking we should add a `TaskExecutor` to `IndexWriter`. The only thing thats weird is that with `ConcurrentMergeScheduler`,

Re: [PR] Enable parent field in sorted bwc tests [lucene]

2024-02-20 Thread via GitHub
jpountz commented on PR #13067: URL: https://github.com/apache/lucene/pull/13067#issuecomment-1955162575 FYI I had not paid attention to the timing but this got merged after I cut the branch for 9.10, so I just bumped the version in this change to 9.11. -- This is an automated message fro

Re: [PR] Removing thread sleep calls from TestIndexWriter.testThreadInterruptDeadlock and TestDirectoryReader.testStressTryIncRef [lucene]

2024-02-20 Thread via GitHub
github-actions[bot] commented on PR #13037: URL: https://github.com/apache/lucene/pull/13037#issuecomment-1955472251 This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the d...@lucene.apache.org list. Thank you for your contributi

Re: [PR] Reduce ArrayUtil#grow in decompress [lucene]

2024-02-20 Thread via GitHub
vigyasharma commented on PR #12996: URL: https://github.com/apache/lucene/pull/12996#issuecomment-1955921730 @easyice Let's add a changes.txt entry for this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] Reduce ArrayUtil#grow in decompress [lucene]

2024-02-20 Thread via GitHub
easyice commented on PR #12996: URL: https://github.com/apache/lucene/pull/12996#issuecomment-1955973968 Thank you for reviewing! @vigyasharma I have added the CHANGES entry under Lucene 9.11.0 -- This is an automated message from the Apache Git Service. To respond to the message