[jira] [Updated] (SOLR-12823) remove clusterstate.json in Lucene/Solr 9.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Høydahl updated SOLR-12823: --- Summary: remove clusterstate.json in Lucene/Solr 9.0 (was: remove clusterstate.json in Lucene/Solr 8.0) > remove clusterstate.json in Lucene/Solr 9.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > Time Spent: 3h 50m > Remaining Estimate: 0h > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 8.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-12823) remove clusterstate.json in Lucene/Solr 9.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Høydahl updated SOLR-12823: --- Description: clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove that in 9.0 It stays empty unless you explicitly ask to create the collection with the old "stateFormat" and there is no reason for one to create a collection with the old stateFormat. We should also remove the "stateFormat" argument in create collection We should also remove MIGRATESTATEVERSION as well was: clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove that in 8.0 It stays empty unless you explicitly ask to create the collection with the old "stateFormat" and there is no reason for one to create a collection with the old stateFormat. We should also remove the "stateFormat" argument in create collection We should also remove MIGRATESTATEVERSION as well > remove clusterstate.json in Lucene/Solr 9.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > Time Spent: 3h 50m > Remaining Estimate: 0h > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 9.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14467) inconsistent server errors combining relatedness() with allBuckets:true
[
https://issues.apache.org/jira/browse/SOLR-14467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125612#comment-17125612
]
Lucene/Solr QA commented on SOLR-14467:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 1m 8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 1m 4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 1m 4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate ref guide {color} |
{color:green} 1m 4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 45m
16s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 49m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-14467 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13004759/SOLR-14467.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns validaterefguide |
| uname | Linux lucene1-us-west 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 08a13ce5894 |
| ant | version: Apache Ant(TM) version 1.10.5 compiled on March 28 2019 |
| Default Java | LTS |
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/760/testReport/ |
| modules | C: solr/core solr/solr-ref-guide U: solr |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/760/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> inconsistent server errors combining relatedness() with allBuckets:true
> ---
>
> Key: SOLR-14467
> URL: https://issues.apache.org/jira/browse/SOLR-14467
> Project: Solr
> Issue Type: Bug
> Components: Facet Module
>Reporter: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-14467.patch, SOLR-14467.patch, SOLR-14467.patch,
> SOLR-14467.patch, SOLR-14467_allBuckets_refine.patch, SOLR-14467_test.patch,
> SOLR-14467_test.patch, beast.log.txt, beast2.log.txt
>
>
> While working on randomized testing for SOLR-13132 i discovered a variety of
> different ways that JSON Faceting's "allBuckets" option can fail when
> combined with the "relatedness()" function.
> I haven't found a trivial way to manual reproduce this, but i have been able
> to trigger the failures with a trivial patch to {{TestCloudJSONFacetSKG}}
> which i will attach.
> Based on the nature of the failures it looks like it may have something to do
> with multiple segments of different sizes, and or resizing the SlotAccs ?
> The relatedness() function doesn't have much (any?) existing tests in place
> that leverage "allBuckets" so this is probably a bug that has always existed
> -- it's possible it may be excessively cumbersome to fix and we might
> nee/wnat to just document that incompatibility and add some code to try and
> detect if the user combines these options and if so fail with a 400 error?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14518) Add support for partitioned unique agg to JSON facets
[ https://issues.apache.org/jira/browse/SOLR-14518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125661#comment-17125661 ] Mikhail Khludnev commented on SOLR-14518: - [~dan2097] It sounds like a way to go, but I've already made wrong suggestion in the comment above. So, beware. > Add support for partitioned unique agg to JSON facets > - > > Key: SOLR-14518 > URL: https://issues.apache.org/jira/browse/SOLR-14518 > Project: Solr > Issue Type: New Feature > Components: Facet Module >Reporter: Joel Bernstein >Priority: Major > > There are scenarios where documents are partitioned across shards based on > the same field that the *unique* agg is applied to with JSON facets. In this > scenario exact unique counts can be calculated by simply sending the bucket > level unique counts to the aggregator where they can be summed. Suggested > syntax is to add a boolean flag to the unique aggregation function: > *unique*(partitioned_field, true). > The *true* value turns on the "partitioned" unique logic. The default is > false. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14521) Establish the Solr project (resolution)
[ https://issues.apache.org/jira/browse/SOLR-14521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Høydahl updated SOLR-14521: --- Attachment: Skjermbilde 2020-06-04 kl. 11.23.53.png > Establish the Solr project (resolution) > --- > > Key: SOLR-14521 > URL: https://issues.apache.org/jira/browse/SOLR-14521 > Project: Solr > Issue Type: Sub-task >Reporter: Jan Høydahl >Priority: Major > Attachments: Skjermbilde 2020-06-04 kl. 11.23.53.png > > > By submitting a board resolution to the next board meeting. See > [https://cwiki.apache.org/confluence/display/LUCENE/Solr+TLP+needed+changes#SolrTLPneededchanges-resolutionDraftboardresolution] > for the draft resolution -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14521) Establish the Solr project (resolution)
[ https://issues.apache.org/jira/browse/SOLR-14521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Høydahl updated SOLR-14521: --- Description: By submitting a board resolution to the next board meeting. See [https://cwiki.apache.org/confluence/display/LUCENE/Solr+TLP+needed+changes#SolrTLPneededchanges-resolutionDraftboardresolution] for the draft resolution. Once the resolution draft is approved (by vote or lazy consensus), PMC chair can add it to Board meeting agenda using Whimsy [https://whimsy.apache.org/board/agenda/2020-06-17/] and clicking the "Add Item" button and then "Establish Project" button. It's just a form with checkboxes for PMC members. The first PMC member selected becomes the chair. The next meeting is June 17th, when Lucene quarterly report is also due. !Skjermbilde 2020-06-04 kl. 11.23.53.png! was:By submitting a board resolution to the next board meeting. See [https://cwiki.apache.org/confluence/display/LUCENE/Solr+TLP+needed+changes#SolrTLPneededchanges-resolutionDraftboardresolution] for the draft resolution > Establish the Solr project (resolution) > --- > > Key: SOLR-14521 > URL: https://issues.apache.org/jira/browse/SOLR-14521 > Project: Solr > Issue Type: Sub-task >Reporter: Jan Høydahl >Priority: Major > Attachments: Skjermbilde 2020-06-04 kl. 11.23.53.png > > > By submitting a board resolution to the next board meeting. See > [https://cwiki.apache.org/confluence/display/LUCENE/Solr+TLP+needed+changes#SolrTLPneededchanges-resolutionDraftboardresolution] > for the draft resolution. > Once the resolution draft is approved (by vote or lazy consensus), PMC chair > can add it to Board meeting agenda using Whimsy > [https://whimsy.apache.org/board/agenda/2020-06-17/] and clicking the "Add > Item" button and then "Establish Project" button. It's just a form with > checkboxes for PMC members. The first PMC member selected becomes the chair. > The next meeting is June 17th, when Lucene quarterly report is also due. > !Skjermbilde 2020-06-04 kl. 11.23.53.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9391) Upgrade to HPPC 0.8.2
[ https://issues.apache.org/jira/browse/LUCENE-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125762#comment-17125762 ] Dawid Weiss commented on LUCENE-9391: - I think this is applicable to Solr, not Lucene? Care to provide a patch? > Upgrade to HPPC 0.8.2 > - > > Key: LUCENE-9391 > URL: https://issues.apache.org/jira/browse/LUCENE-9391 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Haoyu Zhai >Priority: Minor > > HPPC 0.8.2 is out and exposes an Accountable-like interface using to estimate > the memory usage. > [https://issues.carrot2.org/secure/ReleaseNote.jspa?projectId=10070&version=13522&styleName=Text] > We should upgrade to that if any of components using hppc need to estimate > memory better. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] markharwood commented on pull request #1496: Bugfix for FuzzyQuery false negative when prefix length == search term length
markharwood commented on pull request #1496: URL: https://github.com/apache/lucene-solr/pull/1496#issuecomment-638749347 Closing - addressed instead by https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=45611d0 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] markharwood closed pull request #1496: Bugfix for FuzzyQuery false negative when prefix length == search term length
markharwood closed pull request #1496: URL: https://github.com/apache/lucene-solr/pull/1496 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson closed pull request #1492: SOLR-11934: Visit Solr logging, it's too noisy.
ErickErickson closed pull request #1492: URL: https://github.com/apache/lucene-solr/pull/1492 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on pull request #1492: SOLR-11934: Visit Solr logging, it's too noisy.
ErickErickson commented on pull request #1492: URL: https://github.com/apache/lucene-solr/pull/1492#issuecomment-638769279 Forgot to close it, thanks for pointing it out. Erick > On Jun 3, 2020, at 4:41 PM, Mike Drob wrote: > > > @ErickErickson There's no changes here. Stale PR? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub, or unsubscribe. > This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] markharwood commented on pull request #1541: RegExp - add case insensitive matching option
markharwood commented on pull request #1541: URL: https://github.com/apache/lucene-solr/pull/1541#issuecomment-638776739 I updated the constructor on this @jpountz - good to go? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14533) Fix or suppress warnings in solr/handler/admin
[ https://issues.apache.org/jira/browse/SOLR-14533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125820#comment-17125820 ] ASF subversion and git services commented on SOLR-14533: Commit bab4fccba22707e254807c8152921da143fb8706 in lucene-solr's branch refs/heads/master from Erick Erickson [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=bab4fcc ] SOLR-14533: Fix or suppress warnings in solr/handler/admin > Fix or suppress warnings in solr/handler/admin > -- > > Key: SOLR-14533 > URL: https://issues.apache.org/jira/browse/SOLR-14533 > Project: Solr > Issue Type: Sub-task >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-14533.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14533) Fix or suppress warnings in solr/handler/admin
[ https://issues.apache.org/jira/browse/SOLR-14533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125821#comment-17125821 ] ASF subversion and git services commented on SOLR-14533: Commit 94916a6a8a1cf38cc03767f3f2f4e2d38a4ad271 in lucene-solr's branch refs/heads/branch_8x from Erick Erickson [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=94916a6 ] SOLR-14533: Fix or suppress warnings in solr/handler/admin > Fix or suppress warnings in solr/handler/admin > -- > > Key: SOLR-14533 > URL: https://issues.apache.org/jira/browse/SOLR-14533 > Project: Solr > Issue Type: Sub-task >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-14533.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14533) Fix or suppress warnings in solr/handler/admin
[ https://issues.apache.org/jira/browse/SOLR-14533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson updated SOLR-14533: -- Attachment: SOLR-14533.patch Status: Open (was: Open) Changed a couple of things: 1> it's rarely (never?) the right thing to do to throw an exception from a test method, JUnit will deal with that. 2> AutoCloseable.close(). We crossed wires a bit. And no big deal at all, just for future reference. I looked at the JavaDocs for AutoCloseable.close(), and it's _strongly_ advised to throw a concrete exception rather than Exception, even though that's the method signature for AutoCloseable.close(). I'd already changed the method signature for some classes that implement AutoCloseable.close (SolrMetricProducer is case in point) to throw an IOException rather than Exception. That rippled through a couple of try-with-resources blocks and needed adding a catch (IOException) block. This is just in case you run into this in future so you won't scratch your head. 3> <2> notwithstanding, I had to make a few changes for the 8x backport because the AutoCloseable interface hadn't been added to one of the metrics classes, but I'm not going to put up a separate patch for that... Attached is a patch with the rest of the warnings removed. Thanks again! > Fix or suppress warnings in solr/handler/admin > -- > > Key: SOLR-14533 > URL: https://issues.apache.org/jira/browse/SOLR-14533 > Project: Solr > Issue Type: Sub-task >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-14533.patch, SOLR-14533.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14533) Fix or suppress warnings in solr/handler/admin
[ https://issues.apache.org/jira/browse/SOLR-14533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson resolved SOLR-14533. --- Fix Version/s: 8.6 Resolution: Fixed Thanks Andras! > Fix or suppress warnings in solr/handler/admin > -- > > Key: SOLR-14533 > URL: https://issues.apache.org/jira/browse/SOLR-14533 > Project: Solr > Issue Type: Sub-task >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Fix For: 8.6 > > Attachments: SOLR-14533.patch, SOLR-14533.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14426) forbidden api error during precommit DateMathFunction
[
https://issues.apache.org/jira/browse/SOLR-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Erickson resolved SOLR-14426.
---
Resolution: Fixed
I believe all the package-private classes are now either static nested classes
or in their own file and this issue won't happen again.
Since nobody objected, I'll close it again. We can reopen if necessary.
> forbidden api error during precommit DateMathFunction
> -
>
> Key: SOLR-14426
> URL: https://issues.apache.org/jira/browse/SOLR-14426
> Project: Solr
> Issue Type: Bug
> Components: Build
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Fix For: master (9.0)
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When running `./gradlew precommit` I'll occasionally see
> {code}
> * What went wrong:
> Execution failed for task ':solr:contrib:analytics:forbiddenApisMain'.
> > de.thetaphi.forbiddenapis.ForbiddenApiException: Check for forbidden API
> > calls failed while scanning class
> > 'org.apache.solr.analytics.function.mapping.DateMathFunction'
> > (DateMathFunction.java): java.lang.ClassNotFoundException:
> > org.apache.solr.analytics.function.mapping.DateMathValueFunction (while
> > looking up details about referenced class
> > 'org.apache.solr.analytics.function.mapping.DateMathValueFunction')
> {code}
> `./gradlew clean` fixes this, but I don't understand what or why this
> happens. Feels like a gradle issue?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14532) Add iml file to gitignore
[
https://issues.apache.org/jira/browse/SOLR-14532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125852#comment-17125852
]
Erick Erickson commented on SOLR-14532:
---
[~asalamon74] I don't see this happening. If I do the following:
- git clean -dxf
- ./gradlew idea
I do see a file SolrEOEFork.iml, but it's not lagged by Git as being part of
the repo. In fact if I do an "ant idea" I see a ton of new iml files scattered
all over the source tree, not only in {{dev-tools}}, none of which are flagged
by git.
How do reproduce this?
> Add iml file to gitignore
> -
>
> Key: SOLR-14532
> URL: https://issues.apache.org/jira/browse/SOLR-14532
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Andras Salamon
>Priority: Trivial
> Attachments: SOLR-14532.patch
>
>
> If I execute {{gradlew idea}} in my {{lucene-solr-upstream}} directory, it
> will create three files in the root directory:
> {noformat}
> lucene-solr-upstream.iml
> lucene-solr-upstream.ipr
> lucene-solr-upstream.iws
> {noformat}
> Git will ignore the {{ipr}} and the {{iws}} file, but it lists the iml file
> as a new file. We should also ignore that one.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14535) Fix or suppress warnings in apache/solr/handler/component
Erick Erickson created SOLR-14535: - Summary: Fix or suppress warnings in apache/solr/handler/component Key: SOLR-14535 URL: https://issues.apache.org/jira/browse/SOLR-14535 Project: Solr Issue Type: Sub-task Reporter: Erick Erickson Assignee: Erick Erickson -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14535) Fix or suppress warnings in apache/solr/handler/component, sql and loader
[ https://issues.apache.org/jira/browse/SOLR-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson updated SOLR-14535: -- Summary: Fix or suppress warnings in apache/solr/handler/component, sql and loader (was: Fix or suppress warnings in apache/solr/handler/component) > Fix or suppress warnings in apache/solr/handler/component, sql and loader > - > > Key: SOLR-14535 > URL: https://issues.apache.org/jira/browse/SOLR-14535 > Project: Solr > Issue Type: Sub-task >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13458) Make Jetty timeouts configurable system wide
[ https://issues.apache.org/jira/browse/SOLR-13458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125908#comment-17125908 ] Gus Heck commented on SOLR-13458: - Not at the time, that particular work got abandoned by the customer soon after I wrote this and I didn't dig further, but I'm actually once again bumping into timeouts (in a cluster with many billions of docs) so I may soon. > Make Jetty timeouts configurable system wide > > > Key: SOLR-13458 > URL: https://issues.apache.org/jira/browse/SOLR-13458 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Priority: Major > > Our jetty container has several timeouts associated with it, and at least one > of these is regularly getting in my way (the idle timeout after 120 sec). I > tried setting a system property, with no effect and I've tried altering a > jetty.xml found at solr-install/solr/server/etc/jetty.xml on all (50) > machines and rebooting all servers only to have an exception with the old 120 > sec timeout still show up. This ticket proposes that these values are by > nature "Global System Timeouts" and should be made configurable in solr.xml > (which may be difficult because they will be needed early in the boot > sequence). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] msokolov commented on pull request #1543: LUCENE-9378: Disable compression on binary values whose length is less than 32.
msokolov commented on pull request #1543: URL: https://github.com/apache/lucene-solr/pull/1543#issuecomment-638852369 Our internal benchmarks show slowdowns even as we increase the threshold here, although we can nearly recover (-9% QPS) the previous query-time performance disabling below 128 bytes. I'm beginning to wonder what the case is for compressing BDV at all? Do we see large BinaryDocValues fields that are not, or only rarely, decoded at query time? I think if such fields participate in the search in any significant way (ie for all hits), we are going to see these slowdowns. Maybe the bytes threshold is really just a proxy for binary doc values used as stored fields? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14536) Fix or suppress warnings in apache/solr/common
Erick Erickson created SOLR-14536: - Summary: Fix or suppress warnings in apache/solr/common Key: SOLR-14536 URL: https://issues.apache.org/jira/browse/SOLR-14536 Project: Solr Issue Type: Sub-task Reporter: Erick Erickson Assignee: Erick Erickson -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14532) Add iml file to gitignore
[
https://issues.apache.org/jira/browse/SOLR-14532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125973#comment-17125973
]
Dawid Weiss commented on SOLR-14532:
If you're using gradle then I'd suggest importing the project into IntelliJ as
a gradle project (not using the plugin to generate project files). This results
in directory-based layout (.gradle) rather than file-based layout of IntelliJ
files.
> Add iml file to gitignore
> -
>
> Key: SOLR-14532
> URL: https://issues.apache.org/jira/browse/SOLR-14532
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Andras Salamon
>Priority: Trivial
> Attachments: SOLR-14532.patch
>
>
> If I execute {{gradlew idea}} in my {{lucene-solr-upstream}} directory, it
> will create three files in the root directory:
> {noformat}
> lucene-solr-upstream.iml
> lucene-solr-upstream.ipr
> lucene-solr-upstream.iws
> {noformat}
> Git will ignore the {{ipr}} and the {{iws}} file, but it lists the iml file
> as a new file. We should also ignore that one.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14532) Add iml file to gitignore
[
https://issues.apache.org/jira/browse/SOLR-14532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125980#comment-17125980
]
Andras Salamon commented on SOLR-14532:
---
Here are my reproduction steps:
{noformat}
$ git clone g...@github.com:apache/lucene-solr.git
$ cd lucene-solr
$ git status
# it shows that we have no new files which is OK
$ ./gradlew idea
git status
On branch master
Your branch is up to date with 'origin/master'.
Untracked files:
(use "git add ..." to include in what will be committed)
lucene-solr.iml
nothing added to commit but untracked files present (use "git add" to
track){noformat}
So git shows this file as an untracked file. The other two files
(lucene-solr.ipr, lucene-solr.iws) are ignored so they don't listed as
untracked files.
> Add iml file to gitignore
> -
>
> Key: SOLR-14532
> URL: https://issues.apache.org/jira/browse/SOLR-14532
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Andras Salamon
>Priority: Trivial
> Attachments: SOLR-14532.patch
>
>
> If I execute {{gradlew idea}} in my {{lucene-solr-upstream}} directory, it
> will create three files in the root directory:
> {noformat}
> lucene-solr-upstream.iml
> lucene-solr-upstream.ipr
> lucene-solr-upstream.iws
> {noformat}
> Git will ignore the {{ipr}} and the {{iws}} file, but it lists the iml file
> as a new file. We should also ignore that one.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14523) Enhance gradle logging calls validation: eliminate getMessage()
[
https://issues.apache.org/jira/browse/SOLR-14523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125988#comment-17125988
]
Andras Salamon commented on SOLR-14523:
---
[~erickerickson] I followed your description it was working correctly. Thx.
I'll start on the fix.
> Enhance gradle logging calls validation: eliminate getMessage()
> ---
>
> Key: SOLR-14523
> URL: https://issues.apache.org/jira/browse/SOLR-14523
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Andras Salamon
>Assignee: Erick Erickson
>Priority: Minor
> Attachments: validation.patch
>
>
> SOLR-14280 fixed a logging problem in SolrConfig by removing a few
> getMessage() calls. We could enhance this solution by modifying gradle's
> logging calls validation and forbid getMessage() calls during logging. We
> should check the existing code and eliminate such calls.
> It is possible to suppress the warning using {{//logok}}.
> [~erickerickson] [~gerlowskija]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9389) Enhance gradle logging calls validation: eliminate getMessage()
[
https://issues.apache.org/jira/browse/LUCENE-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125990#comment-17125990
]
Andras Salamon commented on LUCENE-9389:
LUCENE-9382 turned off log validation for Lucene now this is only called for
Solr. So even if I apply the patch provided in SOLR-14523 it will not list
Lucene problems.
It is still useful to fix the Lucene log problems but without the checker they
will come back eventually.
> Enhance gradle logging calls validation: eliminate getMessage()
> ---
>
> Key: LUCENE-9389
> URL: https://issues.apache.org/jira/browse/LUCENE-9389
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Andras Salamon
>Assignee: Erick Erickson
>Priority: Minor
>
> SOLR-14280 fixed a logging problem in SolrConfig by removing a few
> getMessage() calls. We could enhance this solution by modifying gradle's
> logging calls validation and forbid getMessage() calls during logging. We
> should check the existing code and eliminate such calls.
> It is possible to suppress the warning using {{//logok}}.
> [~erickerickson] [~gerlowskija]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on pull request #1543: LUCENE-9378: Disable compression on binary values whose length is less than 32.
jpountz commented on pull request #1543: URL: https://github.com/apache/lucene-solr/pull/1543#issuecomment-638908545 Hi @msokolov I'm looking into improving the encoding of lengths, which is the next bottleneck for binary doc values. We are using binary doc values to run regex queries on a high cardinality field. A ngram index helps find good candidates and binary doc values are then used for verification. Field values are typically files, URLs, ... which can have significant redundancy. I'm ok with making compression less aggressive though I think that it would be a pity to disable it entirely and never take advantage of redundancy. You mentioned slowdowns, but this actually depends on the query, e.g. I'm seeing an almost 2x speedup when sorting a `MatchAllDocsQuery` on `wikimedium10m`. Let me give a try at making compression less aggressive/slow? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9391) Upgrade to HPPC 0.8.2
[ https://issues.apache.org/jira/browse/LUCENE-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126013#comment-17126013 ] Michael McCandless commented on LUCENE-9391: Let's also upgrade Lucene facet module? Not sure if there are any other Lucene modules consuming HPPC? > Upgrade to HPPC 0.8.2 > - > > Key: LUCENE-9391 > URL: https://issues.apache.org/jira/browse/LUCENE-9391 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Haoyu Zhai >Priority: Minor > > HPPC 0.8.2 is out and exposes an Accountable-like interface using to estimate > the memory usage. > [https://issues.carrot2.org/secure/ReleaseNote.jspa?projectId=10070&version=13522&styleName=Text] > We should upgrade to that if any of components using hppc need to estimate > memory better. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14419) Query DLS {"param":"ref"}
[
https://issues.apache.org/jira/browse/SOLR-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126045#comment-17126045
]
ASF subversion and git services commented on SOLR-14419:
Commit 7c55ba954763d2e4d554992542a814fb438576b3 in lucene-solr's branch
refs/heads/master from Mikhail Khludnev
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=7c55ba9 ]
SOLR-14419: Ref Guide update for {ref:param}
> Query DLS {"param":"ref"}
> -
>
> Key: SOLR-14419
> URL: https://issues.apache.org/jira/browse/SOLR-14419
> Project: Solr
> Issue Type: Improvement
> Components: JSON Request API
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Major
> Fix For: 8.6
>
> Attachments: SOLR-14419-refguide.patch, SOLR-14419.patch,
> SOLR-14419.patch, SOLR-14419.patch
>
>
> What we can do with plain params:
> {{q=\{!parent which=$prnts}...&prnts=type:parent}}
> obviously I want to have something like this in Query DSL:
> {code}
> { "query": { "parents":{ "which":{"param":"prnts"}, "query":"..."}}
> "params": {
> "prnts":"type:parent"
>}
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14419) Query DLS {"param":"ref"}
[
https://issues.apache.org/jira/browse/SOLR-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126055#comment-17126055
]
ASF subversion and git services commented on SOLR-14419:
Commit 3bbb8e3d4168c3d84658438e10fadf98da08d006 in lucene-solr's branch
refs/heads/branch_8x from Mikhail Khludnev
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=3bbb8e3 ]
SOLR-14419: Ref Guide update for {ref:param}
> Query DLS {"param":"ref"}
> -
>
> Key: SOLR-14419
> URL: https://issues.apache.org/jira/browse/SOLR-14419
> Project: Solr
> Issue Type: Improvement
> Components: JSON Request API
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Major
> Fix For: 8.6
>
> Attachments: SOLR-14419-refguide.patch, SOLR-14419.patch,
> SOLR-14419.patch, SOLR-14419.patch
>
>
> What we can do with plain params:
> {{q=\{!parent which=$prnts}...&prnts=type:parent}}
> obviously I want to have something like this in Query DSL:
> {code}
> { "query": { "parents":{ "which":{"param":"prnts"}, "query":"..."}}
> "params": {
> "prnts":"type:parent"
>}
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] NazerkeBS commented on a change in pull request #1527: SOLR-14384 Stack SolrRequestInfo
NazerkeBS commented on a change in pull request #1527:
URL: https://github.com/apache/lucene-solr/pull/1527#discussion_r435401394
##
File path: solr/core/src/java/org/apache/solr/request/SolrRequestInfo.java
##
@@ -52,35 +56,60 @@
private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
public static SolrRequestInfo getRequestInfo() {
-return threadLocal.get();
+Deque stack = threadLocal.get();
+if (stack.isEmpty()) return null;
+return stack.peek();
}
+ /** Adds the SolrRequestInfo onto the stack provided that the stack is not
reached MAX_STACK_SIZE */
public static void setRequestInfo(SolrRequestInfo info) {
-// TODO: temporary sanity check... this can be changed to just an assert
in the future
-SolrRequestInfo prev = threadLocal.get();
-if (prev != null) {
- log.error("Previous SolrRequestInfo was not closed! req={}",
prev.req.getOriginalParams());
- log.error("prev == info : {}", prev.req == info.req, new
RuntimeException());
+Deque stack = threadLocal.get();
+if (info == null) {
+ throw new IllegalArgumentException("SolrRequestInfo is null");
+} else {
+ if (stack.size() <= MAX_STACK_SIZE) {
+stack.push(info);
+ } else {
+assert true : "SolrRequestInfo Stack is full";
+log.error("SolrRequestInfo Stack is full");
+ }
}
-assert prev == null;
-
-threadLocal.set(info);
}
+ /** Removes the most recent SolrRequestInfo from the stack */
public static void clearRequestInfo() {
-try {
- SolrRequestInfo info = threadLocal.get();
- if (info != null && info.closeHooks != null) {
-for (Closeable hook : info.closeHooks) {
- try {
-hook.close();
- } catch (Exception e) {
-SolrException.log(log, "Exception during close hook", e);
- }
+Deque stack = threadLocal.get();
+if (stack.isEmpty()) {
+ log.error("clearRequestInfo called too many times");
+} else {
+ SolrRequestInfo info = stack.pop();
+ closeHooks(info);
+}
+ }
+
+ /**
+ * This reset method is more of a protection mechanism as
+ * we expect it to be empty by now because all "set" calls need to be
balanced with a "clear".
+ */
+ public static void reset() {
+Deque stack = threadLocal.get();
+boolean isEmpty = stack.isEmpty();
+while (!stack.isEmpty()) {
+ SolrRequestInfo info = stack.pop();
+ closeHooks(info);
+}
+assert isEmpty : "SolrRequestInfo Stack should have been cleared.";
+ }
+
+ private static void closeHooks(SolrRequestInfo info) {
+if (info != null && info.closeHooks != null) {
Review comment:
it can be null; when getRequestInfo is called, in case of the
stack.isEmpty(), it returns null;
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] NazerkeBS commented on a change in pull request #1527: SOLR-14384 Stack SolrRequestInfo
NazerkeBS commented on a change in pull request #1527:
URL: https://github.com/apache/lucene-solr/pull/1527#discussion_r435401394
##
File path: solr/core/src/java/org/apache/solr/request/SolrRequestInfo.java
##
@@ -52,35 +56,60 @@
private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
public static SolrRequestInfo getRequestInfo() {
-return threadLocal.get();
+Deque stack = threadLocal.get();
+if (stack.isEmpty()) return null;
+return stack.peek();
}
+ /** Adds the SolrRequestInfo onto the stack provided that the stack is not
reached MAX_STACK_SIZE */
public static void setRequestInfo(SolrRequestInfo info) {
-// TODO: temporary sanity check... this can be changed to just an assert
in the future
-SolrRequestInfo prev = threadLocal.get();
-if (prev != null) {
- log.error("Previous SolrRequestInfo was not closed! req={}",
prev.req.getOriginalParams());
- log.error("prev == info : {}", prev.req == info.req, new
RuntimeException());
+Deque stack = threadLocal.get();
+if (info == null) {
+ throw new IllegalArgumentException("SolrRequestInfo is null");
+} else {
+ if (stack.size() <= MAX_STACK_SIZE) {
+stack.push(info);
+ } else {
+assert true : "SolrRequestInfo Stack is full";
+log.error("SolrRequestInfo Stack is full");
+ }
}
-assert prev == null;
-
-threadLocal.set(info);
}
+ /** Removes the most recent SolrRequestInfo from the stack */
public static void clearRequestInfo() {
-try {
- SolrRequestInfo info = threadLocal.get();
- if (info != null && info.closeHooks != null) {
-for (Closeable hook : info.closeHooks) {
- try {
-hook.close();
- } catch (Exception e) {
-SolrException.log(log, "Exception during close hook", e);
- }
+Deque stack = threadLocal.get();
+if (stack.isEmpty()) {
+ log.error("clearRequestInfo called too many times");
+} else {
+ SolrRequestInfo info = stack.pop();
+ closeHooks(info);
+}
+ }
+
+ /**
+ * This reset method is more of a protection mechanism as
+ * we expect it to be empty by now because all "set" calls need to be
balanced with a "clear".
+ */
+ public static void reset() {
+Deque stack = threadLocal.get();
+boolean isEmpty = stack.isEmpty();
+while (!stack.isEmpty()) {
+ SolrRequestInfo info = stack.pop();
+ closeHooks(info);
+}
+assert isEmpty : "SolrRequestInfo Stack should have been cleared.";
+ }
+
+ private static void closeHooks(SolrRequestInfo info) {
+if (info != null && info.closeHooks != null) {
Review comment:
it can be null; when getRequestInfo is called, if the stack.isEmpty(),
it returns null;
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1527: SOLR-14384 Stack SolrRequestInfo
dsmiley commented on a change in pull request #1527:
URL: https://github.com/apache/lucene-solr/pull/1527#discussion_r435405488
##
File path: solr/core/src/java/org/apache/solr/request/SolrRequestInfo.java
##
@@ -52,35 +56,60 @@
private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
public static SolrRequestInfo getRequestInfo() {
-return threadLocal.get();
+Deque stack = threadLocal.get();
+if (stack.isEmpty()) return null;
+return stack.peek();
}
+ /** Adds the SolrRequestInfo onto the stack provided that the stack is not
reached MAX_STACK_SIZE */
public static void setRequestInfo(SolrRequestInfo info) {
-// TODO: temporary sanity check... this can be changed to just an assert
in the future
-SolrRequestInfo prev = threadLocal.get();
-if (prev != null) {
- log.error("Previous SolrRequestInfo was not closed! req={}",
prev.req.getOriginalParams());
- log.error("prev == info : {}", prev.req == info.req, new
RuntimeException());
+Deque stack = threadLocal.get();
+if (info == null) {
+ throw new IllegalArgumentException("SolrRequestInfo is null");
+} else {
+ if (stack.size() <= MAX_STACK_SIZE) {
+stack.push(info);
+ } else {
+assert true : "SolrRequestInfo Stack is full";
+log.error("SolrRequestInfo Stack is full");
+ }
}
-assert prev == null;
-
-threadLocal.set(info);
}
+ /** Removes the most recent SolrRequestInfo from the stack */
public static void clearRequestInfo() {
-try {
- SolrRequestInfo info = threadLocal.get();
- if (info != null && info.closeHooks != null) {
-for (Closeable hook : info.closeHooks) {
- try {
-hook.close();
- } catch (Exception e) {
-SolrException.log(log, "Exception during close hook", e);
- }
+Deque stack = threadLocal.get();
+if (stack.isEmpty()) {
+ log.error("clearRequestInfo called too many times");
+} else {
+ SolrRequestInfo info = stack.pop();
+ closeHooks(info);
+}
+ }
+
+ /**
+ * This reset method is more of a protection mechanism as
+ * we expect it to be empty by now because all "set" calls need to be
balanced with a "clear".
+ */
+ public static void reset() {
+Deque stack = threadLocal.get();
+boolean isEmpty = stack.isEmpty();
+while (!stack.isEmpty()) {
+ SolrRequestInfo info = stack.pop();
+ closeHooks(info);
+}
+assert isEmpty : "SolrRequestInfo Stack should have been cleared.";
+ }
+
+ private static void closeHooks(SolrRequestInfo info) {
+if (info != null && info.closeHooks != null) {
Review comment:
The callers of this method will not pass null though; right?
##
File path:
solr/core/src/java/org/apache/solr/client/solrj/embedded/EmbeddedSolrServer.java
##
@@ -271,7 +273,9 @@ public void writeResults(ResultContext ctx, JavaBinCodec
codec) throws IOExcepti
throw new SolrServerException(ex);
} finally {
if (req != null) req.close();
- SolrRequestInfo.clearRequestInfo();
+ if (mustClearSolrRequestInfo) {
Review comment:
Hmm; might it simply be enough to call clearRequestInfo if req != null,
thus no new boolean?
##
File path:
solr/core/src/java/org/apache/solr/client/solrj/embedded/EmbeddedSolrServer.java
##
@@ -71,6 +71,7 @@
protected final String coreName;
private final SolrRequestParsers _parser;
private final RequestWriterSupplier supplier;
+ private boolean mustClearSolrRequestInfo = false;
Review comment:
This change is highly suspicious. Why is this boolean now a field of
EmbeddedSolrServer? It doesn't seem like *state* of this object and so I don't
think it belongs here. It seems like a transient status in the course of work
within the method that sets/clears the requestInfo.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14537) Improve performance of ExportWriter
Andrzej Bialecki created SOLR-14537:
---
Summary: Improve performance of ExportWriter
Key: SOLR-14537
URL: https://issues.apache.org/jira/browse/SOLR-14537
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: Export Writer
Reporter: Andrzej Bialecki
Retrieving, sorting and writing out documents in {{ExportWriter}} are three
aspects of the /export handler that can be further optimized.
SOLR-14470 introduced some level of caching in {{StringValue}}. Further options
for caching and speedups should be explored.
Currently the sort/retrieve and write operations are done sequentially, but
they could be parallelized, considering that they block on different channels -
the first is index reading & CPU bound, the other is bound by the receiving end
because it uses blocking IO. The sorting and retrieving of values could be done
in parallel with the operation of writing out the current batch of results.
One possible approach here would be to use "double buffering" where one
buffered batch that is ready (already sorted and retrieved) is being written
out, while the other batch is being prepared in a background thread, and when
both are done the buffers are swapped. This wouldn't complicate the current
code too much but it should instantly give up to 2x higher throughput.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14408) Refactor MoreLikeThisHandler Implementation
[ https://issues.apache.org/jira/browse/SOLR-14408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126088#comment-17126088 ] Nazerke Seidan commented on SOLR-14408: --- [~alessandro.benedetti], wondering if you have taken a look at the PR. > Refactor MoreLikeThisHandler Implementation > --- > > Key: SOLR-14408 > URL: https://issues.apache.org/jira/browse/SOLR-14408 > Project: Solr > Issue Type: Improvement > Components: MoreLikeThis >Reporter: Nazerke Seidan >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > The main goal of this refactoring is for readability and accessibility of > MoreLikeThisHandler class. Current MoreLikeThisHandler class consists of two > static subclasses and accessing them later in MoreLikeThisComponent. I > propose to have them as separate public classes. > cc: [~abenedetti], as you have had the recent commit for MLT, what do you > think about this? Anyway, the code is ready for review. > > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14467) inconsistent server errors combining relatedness() with allBuckets:true
[
https://issues.apache.org/jira/browse/SOLR-14467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126126#comment-17126126
]
Michael Gibney commented on SOLR-14467:
---
Yes, this looks good to me! Integrating the "implied" concept more generally
and not as an explicit special case makes sense.
> inconsistent server errors combining relatedness() with allBuckets:true
> ---
>
> Key: SOLR-14467
> URL: https://issues.apache.org/jira/browse/SOLR-14467
> Project: Solr
> Issue Type: Bug
> Components: Facet Module
>Reporter: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-14467.patch, SOLR-14467.patch, SOLR-14467.patch,
> SOLR-14467.patch, SOLR-14467_allBuckets_refine.patch, SOLR-14467_test.patch,
> SOLR-14467_test.patch, beast.log.txt, beast2.log.txt
>
>
> While working on randomized testing for SOLR-13132 i discovered a variety of
> different ways that JSON Faceting's "allBuckets" option can fail when
> combined with the "relatedness()" function.
> I haven't found a trivial way to manual reproduce this, but i have been able
> to trigger the failures with a trivial patch to {{TestCloudJSONFacetSKG}}
> which i will attach.
> Based on the nature of the failures it looks like it may have something to do
> with multiple segments of different sizes, and or resizing the SlotAccs ?
> The relatedness() function doesn't have much (any?) existing tests in place
> that leverage "allBuckets" so this is probably a bug that has always existed
> -- it's possible it may be excessively cumbersome to fix and we might
> nee/wnat to just document that incompatibility and add some code to try and
> detect if the user combines these options and if so fail with a 400 error?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9390) Kuromoji tokenizer discards tokens if they start with a punctuation character
[ https://issues.apache.org/jira/browse/LUCENE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126141#comment-17126141 ] Jim Ferenczi commented on LUCENE-9390: -- > I usually set the "discardPunctuation" flag to False to avoid such subtle >situation. I thought that discardPunctuation set to false was relevant only in the context of the JapaneseNumberFilter. The `icu` tokenizer removes the punctuation for instance so I am not sure it should be the default. *_（株）_*is kind of special since the parenthesis are required so it shouldn't need a breaking change to preserve this term in the Japanese tokenizer ? > Kuromoji tokenizer discards tokens if they start with a punctuation character > - > > Key: LUCENE-9390 > URL: https://issues.apache.org/jira/browse/LUCENE-9390 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Jim Ferenczi >Priority: Minor > > This issue was first raised in Elasticsearch > [here|https://github.com/elastic/elasticsearch/issues/57614] > The unidic dictionary that is used by the Kuromoji tokenizer contains entries > that mix punctuations and other characters. For instance the following entry: > _（株）,1285,1285,3690,名詞,一般,*,*,*,*,（株）,カブシキガイシャ,カブシキガイシャ_ > can be found in the Noun.csv file. > Today, tokens that start with punctuations are automatically removed by > default (discardPunctuation is true). I think the code was written this way > because we expect punctuations to be separated from normal tokens but there > are exceptions in the original dictionary. Maybe we should check the entire > token when discarding punctuations ? > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13458) Make Jetty timeouts configurable system wide
[ https://issues.apache.org/jira/browse/SOLR-13458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126148#comment-17126148 ] Alexander Zhideev commented on SOLR-13458: -- Thanks for your response. Similar is happening here - collections grow by many millions per day and total volume is over 3 Billion docs. So far no luck on fixing the timeouts that come and go. One approach which seems to sort of work is by using commitWithinMs along with also allowing some time for cool down between queries by not sending requests to Solr and then retrying on Timeout exception. It literally throws 1 timeout after 2 mins then after a few minutes of cool down period same query runs for 4-5 mins and returns just fine with no issue > Make Jetty timeouts configurable system wide > > > Key: SOLR-13458 > URL: https://issues.apache.org/jira/browse/SOLR-13458 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Priority: Major > > Our jetty container has several timeouts associated with it, and at least one > of these is regularly getting in my way (the idle timeout after 120 sec). I > tried setting a system property, with no effect and I've tried altering a > jetty.xml found at solr-install/solr/server/etc/jetty.xml on all (50) > machines and rebooting all servers only to have an exception with the old 120 > sec timeout still show up. This ticket proposes that these values are by > nature "Global System Timeouts" and should be made configurable in solr.xml > (which may be difficult because they will be needed early in the boot > sequence). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9389) Enhance gradle logging calls validation: eliminate getMessage()
[
https://issues.apache.org/jira/browse/LUCENE-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126152#comment-17126152
]
Erick Erickson commented on LUCENE-9389:
The only actual logging calls in Lucene are in the Luke module, so the Lucene
folks decided against keeping the check so we just don't need to worry about
it...
> Enhance gradle logging calls validation: eliminate getMessage()
> ---
>
> Key: LUCENE-9389
> URL: https://issues.apache.org/jira/browse/LUCENE-9389
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Andras Salamon
>Assignee: Erick Erickson
>Priority: Minor
>
> SOLR-14280 fixed a logging problem in SolrConfig by removing a few
> getMessage() calls. We could enhance this solution by modifying gradle's
> logging calls validation and forbid getMessage() calls during logging. We
> should check the existing code and eliminate such calls.
> It is possible to suppress the warning using {{//logok}}.
> [~erickerickson] [~gerlowskija]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9392) Change the visibility of o.a.l.f.FacetsConfig.DELIM_CHAR to public
Ankur created LUCENE-9392: - Summary: Change the visibility of o.a.l.f.FacetsConfig.DELIM_CHAR to public Key: LUCENE-9392 URL: https://issues.apache.org/jira/browse/LUCENE-9392 Project: Lucene - Core Issue Type: Improvement Affects Versions: 8.5.2 Reporter: Ankur FacetsConfig.DELIM_CHAR is marked as private. An application that wants to use this delimiter (in a unit test for example) is forced to re-declare it in the application code. This can break the application if DELIM_CHAR is changed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9392) Change the visibility of o.a.l.f.FacetsConfig.DELIM_CHAR to public
[ https://issues.apache.org/jira/browse/LUCENE-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur updated LUCENE-9392: -- Description: FacetsConfig.DELIM_CHAR is marked as private. An application that wants to use this delimiter (in a unit test for example) is forced to re-declare it in the application code. This can break the application if tetshe value of DELIM_CHAR is changed in FacetsConfig (was: FacetsConfig.DELIM_CHAR is marked as private. An application that wants to use this delimiter (in a unit test for example) is forced to re-declare it in the application code. This can break the application if DELIM_CHAR is changed.) > Change the visibility of o.a.l.f.FacetsConfig.DELIM_CHAR to public > --- > > Key: LUCENE-9392 > URL: https://issues.apache.org/jira/browse/LUCENE-9392 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 8.5.2 >Reporter: Ankur >Priority: Minor > > FacetsConfig.DELIM_CHAR is marked as private. An application that wants to > use this delimiter (in a unit test for example) is forced to re-declare it in > the application code. This can break the application if tetshe value of > DELIM_CHAR is changed in FacetsConfig -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] NazerkeBS commented on a change in pull request #1527: SOLR-14384 Stack SolrRequestInfo
NazerkeBS commented on a change in pull request #1527:
URL: https://github.com/apache/lucene-solr/pull/1527#discussion_r435544642
##
File path:
solr/core/src/java/org/apache/solr/client/solrj/embedded/EmbeddedSolrServer.java
##
@@ -271,7 +273,9 @@ public void writeResults(ResultContext ctx, JavaBinCodec
codec) throws IOExcepti
throw new SolrServerException(ex);
} finally {
if (req != null) req.close();
- SolrRequestInfo.clearRequestInfo();
+ if (mustClearSolrRequestInfo) {
Review comment:
actually that's enough
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] NazerkeBS commented on a change in pull request #1527: SOLR-14384 Stack SolrRequestInfo
NazerkeBS commented on a change in pull request #1527:
URL: https://github.com/apache/lucene-solr/pull/1527#discussion_r435545076
##
File path: solr/core/src/java/org/apache/solr/request/SolrRequestInfo.java
##
@@ -52,35 +56,60 @@
private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
public static SolrRequestInfo getRequestInfo() {
-return threadLocal.get();
+Deque stack = threadLocal.get();
+if (stack.isEmpty()) return null;
+return stack.peek();
}
+ /** Adds the SolrRequestInfo onto the stack provided that the stack is not
reached MAX_STACK_SIZE */
public static void setRequestInfo(SolrRequestInfo info) {
-// TODO: temporary sanity check... this can be changed to just an assert
in the future
-SolrRequestInfo prev = threadLocal.get();
-if (prev != null) {
- log.error("Previous SolrRequestInfo was not closed! req={}",
prev.req.getOriginalParams());
- log.error("prev == info : {}", prev.req == info.req, new
RuntimeException());
+Deque stack = threadLocal.get();
+if (info == null) {
+ throw new IllegalArgumentException("SolrRequestInfo is null");
+} else {
+ if (stack.size() <= MAX_STACK_SIZE) {
+stack.push(info);
+ } else {
+assert true : "SolrRequestInfo Stack is full";
+log.error("SolrRequestInfo Stack is full");
+ }
}
-assert prev == null;
-
-threadLocal.set(info);
}
+ /** Removes the most recent SolrRequestInfo from the stack */
public static void clearRequestInfo() {
-try {
- SolrRequestInfo info = threadLocal.get();
- if (info != null && info.closeHooks != null) {
-for (Closeable hook : info.closeHooks) {
- try {
-hook.close();
- } catch (Exception e) {
-SolrException.log(log, "Exception during close hook", e);
- }
+Deque stack = threadLocal.get();
+if (stack.isEmpty()) {
+ log.error("clearRequestInfo called too many times");
+} else {
+ SolrRequestInfo info = stack.pop();
+ closeHooks(info);
+}
+ }
+
+ /**
+ * This reset method is more of a protection mechanism as
+ * we expect it to be empty by now because all "set" calls need to be
balanced with a "clear".
+ */
+ public static void reset() {
+Deque stack = threadLocal.get();
+boolean isEmpty = stack.isEmpty();
+while (!stack.isEmpty()) {
+ SolrRequestInfo info = stack.pop();
+ closeHooks(info);
+}
+assert isEmpty : "SolrRequestInfo Stack should have been cleared.";
+ }
+
+ private static void closeHooks(SolrRequestInfo info) {
+if (info != null && info.closeHooks != null) {
Review comment:
in fact, it's true; as it is a private method and called inside
SolrRequestInfo
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob merged pull request #1548: SOLR-14524: Harden MultiThreadedOCPTest testFillWorkQueue()
madrob merged pull request #1548: URL: https://github.com/apache/lucene-solr/pull/1548 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14524) Harden MultiThreadedOCPTest
[
https://issues.apache.org/jira/browse/SOLR-14524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126202#comment-17126202
]
ASF subversion and git services commented on SOLR-14524:
Commit dec692252874a5b60dfb71d8c38643a285975b22 in lucene-solr's branch
refs/heads/master from murblanc
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=dec6922 ]
SOLR-14524: Harden MultiThreadedOCPTest testFillWorkQueue() (#1548)
Make MultiThreadedOCPTest.testFillWorkQueue() less vulnerable to timing issues
Co-authored-by: Ilan Ginzburg
> Harden MultiThreadedOCPTest
> ---
>
> Key: SOLR-14524
> URL: https://issues.apache.org/jira/browse/SOLR-14524
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
>Affects Versions: master (9.0)
>Reporter: Ilan Ginzburg
>Priority: Minor
> Labels: test
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> {{MultiThreadedOCPTest.test()}} fails occasionally in Jenkins because of
> timing of tasks enqueue to the Collection API queue.
> This test in {{testFillWorkQueue()}} enqueues a large number of tasks (115,
> more than the 100 Collection API parallel executors) to the Collection API
> queue for a collection COLL_A, then observes a short delay and enqueues a
> task for another collection COLL_B.
> It verifies that the COLL_B task (that does not require the same lock as the
> COLL_A tasks) completes before the third COLL_A task.
> Test failures happen because when enqueues are slowed down enough, the first
> 3 tasks on COLL_A complete even before the COLL_B task gets enqueued!
> In one sample failed Jenkins test execution, the COLL_B task enqueue happened
> 1275ms after the enqueue of the first COLL_A, leaving plenty of time for a
> few (and possibly all) COLL_A tasks to complete.
> Fix will be along the lines of:
> * Make the “blocking” COLL_A task longer to execute (currently 1 second) to
> compensate for slow enqueues.
> * Verify the COLL_B task (a 1ms task) finishes before the long running
> COLL_A task does. This would be a good indication that even though the
> collection queue was filled with tasks waiting for a busy lock, a non
> competing task was picked and executed right away.
> * Delay the enqueue of the COLL_B task to the end of processing of the first
> COLL_A task. This would guarantee that COLL_B is enqueued once at least some
> COLL_A tasks started processing at the Overseer. Possibly also verify that
> the long running task of COLL_A didn't finish execution yet when the COLL_B
> task is enqueued...
> * It might be possible to set a (very) long duration for the slow task of
> COLL_A (to be less vulnerable to execution delays) without requiring the test
> to wait for that task to complete, but only wait for the COLL_B task to
> complete (so the test doesn't run for too long).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14524) Harden MultiThreadedOCPTest
[
https://issues.apache.org/jira/browse/SOLR-14524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Drob resolved SOLR-14524.
--
Fix Version/s: master (9.0)
Assignee: Mike Drob
Resolution: Fixed
Thanks for the fix, [~murblanc]!
> Harden MultiThreadedOCPTest
> ---
>
> Key: SOLR-14524
> URL: https://issues.apache.org/jira/browse/SOLR-14524
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
>Affects Versions: master (9.0)
>Reporter: Ilan Ginzburg
>Assignee: Mike Drob
>Priority: Minor
> Labels: test
> Fix For: master (9.0)
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> {{MultiThreadedOCPTest.test()}} fails occasionally in Jenkins because of
> timing of tasks enqueue to the Collection API queue.
> This test in {{testFillWorkQueue()}} enqueues a large number of tasks (115,
> more than the 100 Collection API parallel executors) to the Collection API
> queue for a collection COLL_A, then observes a short delay and enqueues a
> task for another collection COLL_B.
> It verifies that the COLL_B task (that does not require the same lock as the
> COLL_A tasks) completes before the third COLL_A task.
> Test failures happen because when enqueues are slowed down enough, the first
> 3 tasks on COLL_A complete even before the COLL_B task gets enqueued!
> In one sample failed Jenkins test execution, the COLL_B task enqueue happened
> 1275ms after the enqueue of the first COLL_A, leaving plenty of time for a
> few (and possibly all) COLL_A tasks to complete.
> Fix will be along the lines of:
> * Make the “blocking” COLL_A task longer to execute (currently 1 second) to
> compensate for slow enqueues.
> * Verify the COLL_B task (a 1ms task) finishes before the long running
> COLL_A task does. This would be a good indication that even though the
> collection queue was filled with tasks waiting for a busy lock, a non
> competing task was picked and executed right away.
> * Delay the enqueue of the COLL_B task to the end of processing of the first
> COLL_A task. This would guarantee that COLL_B is enqueued once at least some
> COLL_A tasks started processing at the Overseer. Possibly also verify that
> the long running task of COLL_A didn't finish execution yet when the COLL_B
> task is enqueued...
> * It might be possible to set a (very) long duration for the slow task of
> COLL_A (to be less vulnerable to execution delays) without requiring the test
> to wait for that task to complete, but only wait for the COLL_B task to
> complete (so the test doesn't run for too long).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14535) Fix or suppress warnings in apache/solr/handler/component, sql and loader
[ https://issues.apache.org/jira/browse/SOLR-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126208#comment-17126208 ] ASF subversion and git services commented on SOLR-14535: Commit bf7db012ec31e6ef553b88abd2c2f75dc0298615 in lucene-solr's branch refs/heads/branch_8x from Erick Erickson [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=bf7db01 ] SOLR-14535: Fix or suppress warnings in apache/solr/handler/component, sql and loader > Fix or suppress warnings in apache/solr/handler/component, sql and loader > - > > Key: SOLR-14535 > URL: https://issues.apache.org/jira/browse/SOLR-14535 > Project: Solr > Issue Type: Sub-task >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14535) Fix or suppress warnings in apache/solr/handler/component, sql and loader
[ https://issues.apache.org/jira/browse/SOLR-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126210#comment-17126210 ] ASF subversion and git services commented on SOLR-14535: Commit 0c4d8fb116f7dec83a04d27df550640dcec4eb8d in lucene-solr's branch refs/heads/master from Erick Erickson [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=0c4d8fb ] SOLR-14535: Fix or suppress warnings in apache/solr/handler/component, sql and loader > Fix or suppress warnings in apache/solr/handler/component, sql and loader > - > > Key: SOLR-14535 > URL: https://issues.apache.org/jira/browse/SOLR-14535 > Project: Solr > Issue Type: Sub-task >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14535) Fix or suppress warnings in apache/solr/handler/component, sql and loader
[ https://issues.apache.org/jira/browse/SOLR-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson resolved SOLR-14535. --- Fix Version/s: 8.6 Resolution: Fixed > Fix or suppress warnings in apache/solr/handler/component, sql and loader > - > > Key: SOLR-14535 > URL: https://issues.apache.org/jira/browse/SOLR-14535 > Project: Solr > Issue Type: Sub-task >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Fix For: 8.6 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14524) Harden MultiThreadedOCPTest
[
https://issues.apache.org/jira/browse/SOLR-14524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126221#comment-17126221
]
Ilan Ginzburg commented on SOLR-14524:
--
Thanks for the review and for merging it [~mdrob].
> Harden MultiThreadedOCPTest
> ---
>
> Key: SOLR-14524
> URL: https://issues.apache.org/jira/browse/SOLR-14524
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
>Affects Versions: master (9.0)
>Reporter: Ilan Ginzburg
>Assignee: Mike Drob
>Priority: Minor
> Labels: test
> Fix For: master (9.0)
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> {{MultiThreadedOCPTest.test()}} fails occasionally in Jenkins because of
> timing of tasks enqueue to the Collection API queue.
> This test in {{testFillWorkQueue()}} enqueues a large number of tasks (115,
> more than the 100 Collection API parallel executors) to the Collection API
> queue for a collection COLL_A, then observes a short delay and enqueues a
> task for another collection COLL_B.
> It verifies that the COLL_B task (that does not require the same lock as the
> COLL_A tasks) completes before the third COLL_A task.
> Test failures happen because when enqueues are slowed down enough, the first
> 3 tasks on COLL_A complete even before the COLL_B task gets enqueued!
> In one sample failed Jenkins test execution, the COLL_B task enqueue happened
> 1275ms after the enqueue of the first COLL_A, leaving plenty of time for a
> few (and possibly all) COLL_A tasks to complete.
> Fix will be along the lines of:
> * Make the “blocking” COLL_A task longer to execute (currently 1 second) to
> compensate for slow enqueues.
> * Verify the COLL_B task (a 1ms task) finishes before the long running
> COLL_A task does. This would be a good indication that even though the
> collection queue was filled with tasks waiting for a busy lock, a non
> competing task was picked and executed right away.
> * Delay the enqueue of the COLL_B task to the end of processing of the first
> COLL_A task. This would guarantee that COLL_B is enqueued once at least some
> COLL_A tasks started processing at the Overseer. Possibly also verify that
> the long running task of COLL_A didn't finish execution yet when the COLL_B
> task is enqueued...
> * It might be possible to set a (very) long duration for the slow task of
> COLL_A (to be less vulnerable to execution delays) without requiring the test
> to wait for that task to complete, but only wait for the COLL_B task to
> complete (so the test doesn't run for too long).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14538) Fix or suppress remaining warnings in apache/solr/handler
Erick Erickson created SOLR-14538: - Summary: Fix or suppress remaining warnings in apache/solr/handler Key: SOLR-14538 URL: https://issues.apache.org/jira/browse/SOLR-14538 Project: Solr Issue Type: Sub-task Reporter: Erick Erickson Assignee: Erick Erickson -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9392) Change the visibility of o.a.l.f.FacetsConfig.DELIM_CHAR to public
[ https://issues.apache.org/jira/browse/LUCENE-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126235#comment-17126235 ] Michael McCandless commented on LUCENE-9392: +1 > Change the visibility of o.a.l.f.FacetsConfig.DELIM_CHAR to public > --- > > Key: LUCENE-9392 > URL: https://issues.apache.org/jira/browse/LUCENE-9392 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 8.5.2 >Reporter: Ankur >Priority: Minor > > FacetsConfig.DELIM_CHAR is marked as private. An application that wants to > use this delimiter (in a unit test for example) is forced to re-declare it in > the application code. This can break the application if tetshe value of > DELIM_CHAR is changed in FacetsConfig -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9390) Kuromoji tokenizer discards tokens if they start with a punctuation character
[ https://issues.apache.org/jira/browse/LUCENE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126237#comment-17126237 ] Michael McCandless commented on LUCENE-9390: Not sure if it is related, but LUCENE-9100 is another struggle with Kuromoji and punctuation. > Kuromoji tokenizer discards tokens if they start with a punctuation character > - > > Key: LUCENE-9390 > URL: https://issues.apache.org/jira/browse/LUCENE-9390 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Jim Ferenczi >Priority: Minor > > This issue was first raised in Elasticsearch > [here|https://github.com/elastic/elasticsearch/issues/57614] > The unidic dictionary that is used by the Kuromoji tokenizer contains entries > that mix punctuations and other characters. For instance the following entry: > _（株）,1285,1285,3690,名詞,一般,*,*,*,*,（株）,カブシキガイシャ,カブシキガイシャ_ > can be found in the Noun.csv file. > Today, tokens that start with punctuations are automatically removed by > default (discardPunctuation is true). I think the code was written this way > because we expect punctuations to be separated from normal tokens but there > are exceptions in the original dictionary. Maybe we should check the entire > token when discarding punctuations ? > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9393) FunctionScoreQuery shouldn’t use TOP_DOCS for creating inner weight
Tomas Eduardo Fernandez Lobbe created LUCENE-9393:
-
Summary: FunctionScoreQuery shouldn’t use TOP_DOCS for creating
inner weight
Key: LUCENE-9393
URL: https://issues.apache.org/jira/browse/LUCENE-9393
Project: Lucene - Core
Issue Type: Improvement
Reporter: Tomas Eduardo Fernandez Lobbe
{{FunctionScoreQuery.createWeight}} creates the weight of the inner query using
the {{scoreMode}} from it’s input parameter, however, FunctionScoreQuery can’t
really use WAND algorithm, and the Scorer used will ignore calls to set
competitive scores. FunctionScoreQuery should just turn {{TOP_DOCS}} to
{{COMPLETE}} before creating the inner query's weight.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8962) Can we merge small segments during refresh, for faster searching?
[
https://issues.apache.org/jira/browse/LUCENE-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126253#comment-17126253
]
Nhat Nguyen commented on LUCENE-8962:
-
> For some reason it's not linked above, and I'm not sure how to remedy that
I guess the title of the PR needs to be in the format of "LUCENE-:
description".
> Can we merge small segments during refresh, for faster searching?
> -
>
> Key: LUCENE-8962
> URL: https://issues.apache.org/jira/browse/LUCENE-8962
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Reporter: Michael McCandless
>Priority: Major
> Fix For: 8.6
>
> Attachments: LUCENE-8962_demo.png, failed-tests.patch
>
> Time Spent: 10h
> Remaining Estimate: 0h
>
> With near-real-time search we ask {{IndexWriter}} to write all in-memory
> segments to disk and open an {{IndexReader}} to search them, and this is
> typically a quick operation.
> However, when you use many threads for concurrent indexing, {{IndexWriter}}
> will accumulate write many small segments during {{refresh}} and this then
> adds search-time cost as searching must visit all of these tiny segments.
> The merge policy would normally quickly coalesce these small segments if
> given a little time ... so, could we somehow improve {{IndexWriter'}}s
> refresh to optionally kick off merge policy to merge segments below some
> threshold before opening the near-real-time reader? It'd be a bit tricky
> because while we are waiting for merges, indexing may continue, and new
> segments may be flushed, but those new segments shouldn't be included in the
> point-in-time segments returned by refresh ...
> One could almost do this on top of Lucene today, with a custom merge policy,
> and some hackity logic to have the merge policy target small segments just
> written by refresh, but it's tricky to then open a near-real-time reader,
> excluding newly flushed but including newly merged segments since the refresh
> originally finished ...
> I'm not yet sure how best to solve this, so I wanted to open an issue for
> discussion!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on pull request #1552: LUCENE-8962
mikemccand commented on pull request #1552: URL: https://github.com/apache/lucene-solr/pull/1552#issuecomment-639161416 I'll beast this PR on my 128-core AMD Ryzen box :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on a change in pull request #1552: LUCENE-8962
mikemccand commented on a change in pull request #1552:
URL: https://github.com/apache/lucene-solr/pull/1552#discussion_r435592460
##
File path: lucene/CHANGES.txt
##
@@ -376,6 +376,8 @@ Improvements
* LUCENE-9253: KoreanTokenizer now supports custom dictionaries(system,
unknown). (Namgyu Kim)
+* LUCENE-8962: Add ability to selectively merge on commit (Michael Froh)
Review comment:
Maybe `Add IndexWriter merge-on-commit feature to selectively merge
small segments on commit, subject to a configurable timeout, to improve search
performance by reducing the number of small segments for searching`?
##
File path: lucene/core/src/java/org/apache/lucene/index/IndexWriterConfig.java
##
@@ -459,6 +463,31 @@ public IndexWriterConfig setCommitOnClose(boolean
commitOnClose) {
return this;
}
+ /**
+ * Expert: sets the amount of time to wait for merges returned by
MergePolicy.findFullFlushMerges(...).
+ * If this time is reached, we proceed with the commit based on segments
merged up to that point.
+ * The merges are not cancelled, and may still run to completion independent
of the commit.
Review comment:
Maybe for the last sentence: `The merges are not cancelled, and will
still run to completion independent of the commit like normal segment merges`?
Maybe also state that this setting has no effect unless the `MergePolicy`
actually returns merges from `findFullFlushMerges`, which the default merge
policy does not?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe opened a new pull request #1553: LUCENE-9393: FunctionScoreQuery turns TOP_DOCS to COMPLETE in inner weights
tflobbe opened a new pull request #1553: URL: https://github.com/apache/lucene-solr/pull/1553 FunctionScoreQuery can't really use WAND algorithm even if TOP_DOCS score mode is requested. This commit makes the inner weight created use COMPLETE in this case. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe commented on a change in pull request #1553: LUCENE-9393: FunctionScoreQuery turns TOP_DOCS to COMPLETE in inner weights
tflobbe commented on a change in pull request #1553:
URL: https://github.com/apache/lucene-solr/pull/1553#discussion_r435604487
##
File path:
lucene/queries/src/test/org/apache/lucene/queries/function/TestFunctionScoreQuery.java
##
@@ -244,4 +247,33 @@ public void testAccessToValueSource() throws Exception {
}
+ public void testScoreMode() throws Exception {
+// Value Source doesn't need scores
+assertInnerScoreMode(ScoreMode.COMPLETE_NO_SCORES, ScoreMode.COMPLETE,
DoubleValuesSource.fromDoubleField("foo"));
Review comment:
While this test makes the current code pass, I'm trying to understand
why it is correct. Why do we pass a COMPLETE_NO_SCORES to the underlying weight
when the function's ValueSource doesn't need scores?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13132) Improve JSON "terms" facet performance when sorted by relatedness
[
https://issues.apache.org/jira/browse/SOLR-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126314#comment-17126314
]
Michael Gibney commented on SOLR-13132:
---
I pushed a new commit (18c3d21f44b382b5e35ba7551ebf20abb31fdb76) to the branch
for PR 751 that reverts the five preceding commits (which were attempting to
pursue a different approach to addressing SOLR-14467).
> Improve JSON "terms" facet performance when sorted by relatedness
> --
>
> Key: SOLR-13132
> URL: https://issues.apache.org/jira/browse/SOLR-13132
> Project: Solr
> Issue Type: Improvement
> Components: Facet Module
>Affects Versions: 7.4, master (9.0)
>Reporter: Michael Gibney
>Priority: Major
> Attachments: SOLR-13132-with-cache-01.patch,
> SOLR-13132-with-cache.patch, SOLR-13132.patch, SOLR-13132_testSweep.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> When sorting buckets by {{relatedness}}, JSON "terms" facet must calculate
> {{relatedness}} for every term.
> The current implementation uses a standard uninverted approach (either
> {{docValues}} or {{UnInvertedField}}) to get facet counts over the domain
> base docSet, and then uses that initial pass as a pre-filter for a
> second-pass, inverted approach of fetching docSets for each relevant term
> (i.e., {{count > minCount}}?) and calculating intersection size of those sets
> with the domain base docSet.
> Over high-cardinality fields, the overhead of per-term docSet creation and
> set intersection operations increases request latency to the point where
> relatedness sort may not be usable in practice (for my use case, even after
> applying the patch for SOLR-13108, for a field with ~220k unique terms per
> core, QTime for high-cardinality domain docSets were, e.g.: cardinality
> 1816684=9000ms, cardinality 5032902=18000ms).
> The attached patch brings the above example QTimes down to a manageable
> ~300ms and ~250ms respectively. The approach calculates uninverted facet
> counts over domain base, foreground, and background docSets in parallel in a
> single pass. This allows us to take advantage of the efficiencies built into
> the standard uninverted {{FacetFieldProcessorByArray[DV|UIF]}}), and avoids
> the per-term docSet creation and set intersection overhead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
