Re: [PR] Port over gradle setting generator from Solr [lucene]

2024-04-30 Thread via GitHub
uschindler commented on PR #12131: URL: https://github.com/apache/lucene/pull/12131#issuecomment-2084544680 Hi, I agree with Dawid. I also don't like the non-runtime settings enforcing you to put it into the per project properties file. We should not discuss this here, maybe a bett

Re: [PR] Convert more classes to record classes [lucene]

2024-04-30 Thread via GitHub
shubhamvishu commented on code in PR #13328: URL: https://github.com/apache/lucene/pull/13328#discussion_r1584249124 ## lucene/core/src/java/org/apache/lucene/codecs/TermStats.java: ## @@ -16,24 +16,10 @@ */ package org.apache.lucene.codecs; -import org.apache.lucene.index.

Re: [I] Support writes with previous major lucene versions [lucene]

2024-04-30 Thread via GitHub
itiyama commented on issue #12391: URL: https://github.com/apache/lucene/issues/12391#issuecomment-2084574670 @msokolov @mikemccand What is your recommendation on this for minor version upgrades? Lucene does not support writes on the old codec format for minor versions as well, but I can po

Re: [PR] Convert more classes to record classes [lucene]

2024-04-30 Thread via GitHub
uschindler commented on code in PR #13328: URL: https://github.com/apache/lucene/pull/13328#discussion_r1584262673 ## lucene/core/src/java/org/apache/lucene/codecs/TermStats.java: ## @@ -16,24 +16,10 @@ */ package org.apache.lucene.codecs; -import org.apache.lucene.index.Te

Re: [PR] Convert more classes to record classes [lucene]

2024-04-30 Thread via GitHub
uschindler commented on code in PR #13328: URL: https://github.com/apache/lucene/pull/13328#discussion_r1584264324 ## lucene/core/src/java/org/apache/lucene/codecs/TermStats.java: ## @@ -16,24 +16,10 @@ */ package org.apache.lucene.codecs; -import org.apache.lucene.index.Te

Re: [PR] Convert more classes to record classes [lucene]

2024-04-30 Thread via GitHub
uschindler commented on code in PR #13328: URL: https://github.com/apache/lucene/pull/13328#discussion_r1584267265 ## lucene/core/src/java/org/apache/lucene/codecs/TermStats.java: ## @@ -16,24 +16,10 @@ */ package org.apache.lucene.codecs; -import org.apache.lucene.index.Te

Re: [PR] Convert more classes to record classes [lucene]

2024-04-30 Thread via GitHub
uschindler commented on code in PR #13328: URL: https://github.com/apache/lucene/pull/13328#discussion_r1584276533 ## lucene/core/src/java/org/apache/lucene/codecs/TermStats.java: ## @@ -16,24 +16,10 @@ */ package org.apache.lucene.codecs; -import org.apache.lucene.index.Te

Re: [PR] Fix numDeletesToMerge for unchanged segments [lucene]

2024-04-30 Thread via GitHub
s1monw commented on code in PR #13324: URL: https://github.com/apache/lucene/pull/13324#discussion_r1584305983 ## lucene/core/src/java/org/apache/lucene/index/IndexWriter.java: ## @@ -6128,7 +6128,7 @@ public final int numDeletesToMerge(SegmentCommitInfo info) throws IOExceptio

Re: [PR] Convert more classes to record classes [lucene]

2024-04-30 Thread via GitHub
uschindler commented on code in PR #13328: URL: https://github.com/apache/lucene/pull/13328#discussion_r1584314019 ## lucene/core/src/java/org/apache/lucene/codecs/TermStats.java: ## @@ -16,24 +16,10 @@ */ package org.apache.lucene.codecs; -import org.apache.lucene.index.Te

Re: [PR] Convert more classes to record classes [lucene]

2024-04-30 Thread via GitHub
uschindler commented on code in PR #13328: URL: https://github.com/apache/lucene/pull/13328#discussion_r1584332062 ## lucene/core/src/java/org/apache/lucene/codecs/TermStats.java: ## @@ -16,24 +16,10 @@ */ package org.apache.lucene.codecs; -import org.apache.lucene.index.Te

Re: [PR] Convert more classes to record classes [lucene]

2024-04-30 Thread via GitHub
shubhamvishu commented on code in PR #13328: URL: https://github.com/apache/lucene/pull/13328#discussion_r1584391143 ## lucene/core/src/java/org/apache/lucene/codecs/TermStats.java: ## @@ -16,24 +16,10 @@ */ package org.apache.lucene.codecs; -import org.apache.lucene.index.

[PR] Temporarily exclude record ctors and methods from javadoc checks [lucene]

2024-04-30 Thread via GitHub
shubhamvishu opened a new pull request, #13329: URL: https://github.com/apache/lucene/pull/13329 ### Description In #13328 we got to know `renderSiteJavadoc` task fails for javadocs checks are not working as expected with records ctor and methods(synthetic ones). As pointed by @uschi

Re: [PR] Convert more classes to record classes [lucene]

2024-04-30 Thread via GitHub
shubhamvishu commented on code in PR #13328: URL: https://github.com/apache/lucene/pull/13328#discussion_r1584467051 ## lucene/core/src/java/org/apache/lucene/codecs/TermStats.java: ## @@ -16,24 +16,10 @@ */ package org.apache.lucene.codecs; -import org.apache.lucene.index.

Re: [PR] Performance improvements to use read lock to access LRUQueryCache [lucene]

2024-04-30 Thread via GitHub
boicehuang commented on code in PR #13306: URL: https://github.com/apache/lucene/pull/13306#discussion_r1584475381 ## lucene/core/src/java/org/apache/lucene/search/LRUQueryCache.java: ## @@ -628,7 +632,7 @@ private class LeafCache implements Accountable { LeafCache(Object

Re: [PR] Performance improvements to use read lock to access LRUQueryCache [lucene]

2024-04-30 Thread via GitHub
boicehuang commented on code in PR #13306: URL: https://github.com/apache/lucene/pull/13306#discussion_r1584480939 ## lucene/core/src/java/org/apache/lucene/search/LRUQueryCache.java: ## @@ -628,7 +632,7 @@ private class LeafCache implements Accountable { LeafCache(Object

Re: [I] Examine performance of individual data accessor methods of MemorySegmentIndexInput [lucene]

2024-04-30 Thread via GitHub
uschindler commented on issue #13325: URL: https://github.com/apache/lucene/issues/13325#issuecomment-2084925574 We have the confimation: https://github.com/dacapobench/dacapobench/issues/264#issuecomment-2084907716 I think we can close this issue. Anyways we should talk to th

Re: [I] Examine performance of individual data accessor methods of MemorySegmentIndexInput [lucene]

2024-04-30 Thread via GitHub
uschindler closed issue #13325: Examine performance of individual data accessor methods of MemorySegmentIndexInput URL: https://github.com/apache/lucene/issues/13325 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Examine performance of individual data accessor methods of MemorySegmentIndexInput [lucene]

2024-04-30 Thread via GitHub
ChrisHegarty commented on issue #13325: URL: https://github.com/apache/lucene/issues/13325#issuecomment-2084934317 Thanks for jumping on this so quickly @uschindler. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Temporarily exclude record ctors and methods from javadoc checks [lucene]

2024-04-30 Thread via GitHub
uschindler commented on PR #13329: URL: https://github.com/apache/lucene/pull/13329#issuecomment-2084935782 I am working on a solution, but also the enum ctors are not even marked as "mandated" elements. My idea was to replace the whole method (also for the enums by this: ```ja

Re: [PR] Temporarily exclude record ctors and methods from javadoc checks [lucene]

2024-04-30 Thread via GitHub
uschindler commented on PR #13329: URL: https://github.com/apache/lucene/pull/13329#issuecomment-2084939587 Maybe it works for records, because they are "newer". I will add a fake record for testing. -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] Temporarily exclude record ctors and methods from javadoc checks [lucene]

2024-04-30 Thread via GitHub
uschindler commented on PR #13329: URL: https://github.com/apache/lucene/pull/13329#issuecomment-2084983288 With the above code it was able to detect the default constructor of records, but it did not detect the record component accessor methods (same issue like with enums). -- This is a

Re: [PR] Performance improvements to use read lock to access LRUQueryCache [lucene]

2024-04-30 Thread via GitHub
boicehuang commented on code in PR #13306: URL: https://github.com/apache/lucene/pull/13306#discussion_r1584480939 ## lucene/core/src/java/org/apache/lucene/search/LRUQueryCache.java: ## @@ -628,7 +632,7 @@ private class LeafCache implements Accountable { LeafCache(Object

Re: [I] Support for building materialized views using Lucene formats [lucene]

2024-04-30 Thread via GitHub
msokolov commented on issue #13188: URL: https://github.com/apache/lucene/issues/13188#issuecomment-2085177581 My main concern was to ensure this exciting effort didn't get blocked by the need to do major changes to existing indexing workloads. It sounds like the plan here is less intrusive

Re: [I] Support writes with previous major lucene versions [lucene]

2024-04-30 Thread via GitHub
msokolov commented on issue #12391: URL: https://github.com/apache/lucene/issues/12391#issuecomment-2085191890 I thought "OpenSearch attempted to solve mixed cluster issue by updating primary shard copies to keep using older codec until all replica copies are on latest software" was solving

Re: [PR] Reduce memory usage of field maps in FieldInfos and BlockTree TermsReader. [lucene]

2024-04-30 Thread via GitHub
jpountz commented on code in PR #13327: URL: https://github.com/apache/lucene/pull/13327#discussion_r1584722315 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/blocktree/Lucene90BlockTreeTermsReader.java: ## @@ -113,7 +114,8 @@ public final class Lucene90BlockTreeTerms

[PR] Remove unused "implements Accountable". [lucene]

2024-04-30 Thread via GitHub
jpountz opened a new pull request, #13330: URL: https://github.com/apache/lucene/pull/13330 A number of file formats no longer implement `Accountable`. This allows removing this interface on a number of internal classes as well. -- This is an automated message from the Apache Git Service.

Re: [I] Examine performance of individual data accessor methods of MemorySegmentIndexInput [lucene]

2024-04-30 Thread via GitHub
uschindler commented on issue #13325: URL: https://github.com/apache/lucene/issues/13325#issuecomment-2085320446 They identified that deoptimizations are indeed the cause of all this. When you trigger a thread-local handshake by closing an IndexInput, all code in other threads which runs at

Re: [PR] Temporarily exclude record ctors and methods from javadoc checks [lucene]

2024-04-30 Thread via GitHub
uschindler commented on PR #13329: URL: https://github.com/apache/lucene/pull/13329#issuecomment-2085324323 I have a working solution already which works with records. I just want to figure out how to ideally check for missing `@param` on record components. I will commit to this PR.

Re: [PR] Temporarily exclude record ctors and methods from javadoc checks [lucene]

2024-04-30 Thread via GitHub
uschindler commented on PR #13329: URL: https://github.com/apache/lucene/pull/13329#issuecomment-2085327836 Here is my code that works: ```java /** * Return true if the method is synthetic enum (values/valueOf) or record accessor method. * According to the doctree doc

Re: [PR] Temporarily exclude record ctors and methods from javadoc checks [lucene]

2024-04-30 Thread via GitHub
shubhamvishu commented on PR #13329: URL: https://github.com/apache/lucene/pull/13329#issuecomment-2085404981 Nicethis is great! I think this takes care of record ctors along with synthetic methods due to `ExecutableElement.getEnclosingElement().getEnclosedElements()` returning c

Re: [PR] Port over gradle setting generator from Solr [lucene]

2024-04-30 Thread via GitHub
dsmiley commented on PR #12131: URL: https://github.com/apache/lucene/pull/12131#issuecomment-2085413668 Of course I agree that Gradle could be improved in the ways mentioned here. RE why is this bothering me today: Where I work, I have a Gradle project and within it, a copy of Solr,

[PR] Make segment/field attribute updates thread-safe. [lucene]

2024-04-30 Thread via GitHub
jpountz opened a new pull request, #13331: URL: https://github.com/apache/lucene/pull/13331 Because of concurrent merging (#13124), multiple threads may be updating (different) attributes concurrently, so we need to make reads and writes to attributes thread-safe. -- This is an automated

Re: [I] Support for building materialized views using Lucene formats [lucene]

2024-04-30 Thread via GitHub
jpountz commented on issue #13188: URL: https://github.com/apache/lucene/issues/13188#issuecomment-2085447524 At first sight I don't like the fact that this seems to plug in a whole new way of doing things. Either you don't use a star tree index and you do things the usual way with filters

Re: [PR] Temporarily exclude record ctors and methods from javadoc checks [lucene]

2024-04-30 Thread via GitHub
uschindler commented on PR #13329: URL: https://github.com/apache/lucene/pull/13329#issuecomment-2085449503 > Nice! This is neat. Thank you Uwe! > > I think this takes care of record ctors along with synthetic methods due to `ExecutableElement.getEnclosingElement().getEnclosedElements

Re: [PR] Reduce memory usage of field maps in FieldInfos and BlockTree TermsReader. [lucene]

2024-04-30 Thread via GitHub
bruno-roustant commented on PR #13327: URL: https://github.com/apache/lucene/pull/13327#issuecomment-2085548260 Very interesting, I didn't know this history of the FieldInfos. I ended up analyzing the FieldInfos after we saw the time and memory usage for the byNumber array when there

Re: [PR] Temporarily exclude record ctors and methods from javadoc checks [lucene]

2024-04-30 Thread via GitHub
uschindler commented on PR #13329: URL: https://github.com/apache/lucene/pull/13329#issuecomment-2085565104 I created a new PR #13332 implementing the linter correctly: - exclude default ctors - exclude record accessor methods - check in level=METHOD that all record components have a

Re: [PR] Temporarily exclude record ctors and methods from javadoc checks [lucene]

2024-04-30 Thread via GitHub
uschindler closed pull request #13329: Temporarily exclude record ctors and methods from javadoc checks URL: https://github.com/apache/lucene/pull/13329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Temporarily exclude record ctors and methods from javadoc checks [lucene]

2024-04-30 Thread via GitHub
uschindler commented on PR #13329: URL: https://github.com/apache/lucene/pull/13329#issuecomment-2085565818 I am closing this in favour of #13332. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Reduce memory usage of field maps in FieldInfos and BlockTree TermsReader. [lucene]

2024-04-30 Thread via GitHub
dsmiley commented on code in PR #13327: URL: https://github.com/apache/lucene/pull/13327#discussion_r1584952051 ## lucene/core/src/java/org/apache/lucene/index/FieldInfos.java: ## @@ -81,30 +81,13 @@ public FieldInfos(FieldInfo[] infos) { String softDeletesField = null;

Re: [PR] Improve MissingDoclet linter to check records correctly [lucene]

2024-04-30 Thread via GitHub
uschindler commented on PR #13332: URL: https://github.com/apache/lucene/pull/13332#issuecomment-2085570888 We have no tests for the linter, but I validated that the code complains correctly on missing javadocs for components and excludes component accessors. -- This is an automated messa

Re: [PR] Make segment/field attribute updates thread-safe. [lucene]

2024-04-30 Thread via GitHub
benwtrent commented on code in PR #13331: URL: https://github.com/apache/lucene/pull/13331#discussion_r1584982931 ## lucene/core/src/java/org/apache/lucene/index/FieldInfo.java: ## @@ -628,12 +630,17 @@ public String getAttribute(String key) { * If the value of the attribute

Re: [PR] Make segment/field attribute updates thread-safe. [lucene]

2024-04-30 Thread via GitHub
jpountz commented on code in PR #13331: URL: https://github.com/apache/lucene/pull/13331#discussion_r1584993348 ## lucene/core/src/java/org/apache/lucene/index/FieldInfo.java: ## @@ -628,12 +630,17 @@ public String getAttribute(String key) { * If the value of the attributes

Re: [PR] Convert more classes to record classes [lucene]

2024-04-30 Thread via GitHub
uschindler commented on code in PR #13328: URL: https://github.com/apache/lucene/pull/13328#discussion_r1585004047 ## lucene/core/src/java/org/apache/lucene/codecs/TermStats.java: ## @@ -16,24 +16,10 @@ */ package org.apache.lucene.codecs; -import org.apache.lucene.index.Te

Re: [PR] Reduce memory usage of field maps in FieldInfos and BlockTree TermsReader. [lucene]

2024-04-30 Thread via GitHub
bruno-roustant commented on code in PR #13327: URL: https://github.com/apache/lucene/pull/13327#discussion_r1585022423 ## lucene/core/src/java/org/apache/lucene/index/FieldInfos.java: ## @@ -156,15 +139,17 @@ public FieldInfos(FieldInfo[] infos) { this.softDeletesField = so

Re: [PR] Improve MissingDoclet linter to check records correctly [lucene]

2024-04-30 Thread via GitHub
uschindler commented on PR #13332: URL: https://github.com/apache/lucene/pull/13332#issuecomment-2085671536 I will merge this later this evening so we can proceed with #13328 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Reduce memory usage of field maps in FieldInfos and BlockTree TermsReader. [lucene]

2024-04-30 Thread via GitHub
bruno-roustant commented on code in PR #13327: URL: https://github.com/apache/lucene/pull/13327#discussion_r1585048996 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/blocktree/Lucene90BlockTreeTermsReader.java: ## @@ -113,7 +114,8 @@ public final class Lucene90BlockTr

Re: [PR] Reduce memory usage of field maps in FieldInfos and BlockTree TermsReader. [lucene]

2024-04-30 Thread via GitHub
bruno-roustant commented on code in PR #13327: URL: https://github.com/apache/lucene/pull/13327#discussion_r1585064742 ## lucene/core/src/java/org/apache/lucene/codecs/lucene90/blocktree/Lucene90BlockTreeTermsReader.java: ## @@ -113,7 +114,8 @@ public final class Lucene90BlockTr

Re: [PR] Remove unused "implements Accountable". [lucene]

2024-04-30 Thread via GitHub
benwtrent commented on code in PR #13330: URL: https://github.com/apache/lucene/pull/13330#discussion_r1585037924 ## lucene/codecs/src/java/org/apache/lucene/codecs/blockterms/VariableGapTermsIndexReader.java: ## @@ -168,22 +165,6 @@ public FieldIndexEnum getFieldEnum(FieldInfo

Re: [PR] Improve MissingDoclet linter to check records correctly [lucene]

2024-04-30 Thread via GitHub
uschindler merged PR #13332: URL: https://github.com/apache/lucene/pull/13332 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.

Re: [PR] Convert more classes to record classes [lucene]

2024-04-30 Thread via GitHub
uschindler commented on PR #13328: URL: https://github.com/apache/lucene/pull/13328#issuecomment-2085780025 Hi, you can merge main into your branch and hopefully checks pass. For some packages like the "codecs" one the new code requires you to also document all record components with `@para

Re: [PR] Convert more classes to record classes [lucene]

2024-04-30 Thread via GitHub
uschindler commented on code in PR #13328: URL: https://github.com/apache/lucene/pull/13328#discussion_r1585133751 ## lucene/core/src/java/org/apache/lucene/codecs/TermStats.java: ## @@ -16,24 +16,10 @@ */ package org.apache.lucene.codecs; -import org.apache.lucene.index.Te

Re: [PR] Convert more classes to record classes [lucene]

2024-04-30 Thread via GitHub
uschindler commented on code in PR #13328: URL: https://github.com/apache/lucene/pull/13328#discussion_r1585147534 ## lucene/core/src/java/org/apache/lucene/util/TermAndVector.java: ## @@ -24,23 +24,7 @@ * * @lucene.experimental */ -public class TermAndVector { - - privat

Re: [PR] Convert more classes to record classes [lucene]

2024-04-30 Thread via GitHub
uschindler commented on code in PR #13328: URL: https://github.com/apache/lucene/pull/13328#discussion_r1585152280 ## lucene/core/src/java/org/apache/lucene/util/packed/PackedInts.java: ## @@ -171,14 +171,7 @@ public final float overheadRatio(int bitsPerValue) { } /** Si

Re: [PR] Convert more classes to record classes [lucene]

2024-04-30 Thread via GitHub
uschindler commented on code in PR #13328: URL: https://github.com/apache/lucene/pull/13328#discussion_r1585154771 ## lucene/core/src/java/org/apache/lucene/search/CollectionStatistics.java: ## @@ -116,7 +105,8 @@ public CollectionStatistics( * * @return field's name, no

Re: [I] Support for building materialized views using Lucene formats [lucene]

2024-04-30 Thread via GitHub
bharath-techie commented on issue #13188: URL: https://github.com/apache/lucene/issues/13188#issuecomment-2086176227 Thanks for the inputs @jpountz . Let me spend some more time on this. But this is a topic which was thought of as well and one idea was to do query abstraction / planni

Re: [I] Support writes with previous major lucene versions [lucene]

2024-04-30 Thread via GitHub
msokolov commented on issue #12391: URL: https://github.com/apache/lucene/issues/12391#issuecomment-2086294536 > Since the writer logic is available in backward codecs for testing, I can still go ahead and override the codec write methods to work with old writers. But the path is not te

Re: [I] Support writes with previous major lucene versions [lucene]

2024-04-30 Thread via GitHub
itiyama commented on issue #12391: URL: https://github.com/apache/lucene/issues/12391#issuecomment-2087338286 Lucene does support reading N-1 index versions at the same time, and it should theoretically be feasible to enable writing for N-1 versions, though not concurrently. The software bo

Re: [PR] Performance improvements to use read lock to access LRUQueryCache [lucene]

2024-04-30 Thread via GitHub
boicehuang commented on code in PR #13306: URL: https://github.com/apache/lucene/pull/13306#discussion_r1582091772 ## lucene/core/src/java/org/apache/lucene/search/LRUQueryCache.java: ## @@ -265,7 +269,6 @@ boolean requiresEviction() { } CacheAndCount get(Query key, Inde

Re: [PR] Disjunction as CompetitiveIterator for numeric dynamic pruning [lucene]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #13221: URL: https://github.com/apache/lucene/pull/13221#issuecomment-2087758114 This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the d...@lucene.apache.org list. Thank you for your contributi