[jira] [Created] (LUCENE-9425) Too many open files issue in Jbosseap 7.1 Server
Archana A M created LUCENE-9425: --- Summary: Too many open files issue in Jbosseap 7.1 Server Key: LUCENE-9425 URL: https://issues.apache.org/jira/browse/LUCENE-9425 Project: Lucene - Core Issue Type: Bug Affects Versions: 6.5.1 Reporter: Archana A M Hi Team, We are getting "too many open files" in server while trying to access apache Lucene cache. Could someone please suggest the recommended open file limit while using apache Lucene in the application. Please find the relevant details below Lucene version - 6.5.1 current ulimit soft limit - 1024 current ulimit hard limit - 4096 Server - Jbosseap 7.1 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9425) Too many open files issue in Jbosseap 7.1 Server
[ https://issues.apache.org/jira/browse/LUCENE-9425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Archana A M updated LUCENE-9425: Priority: Blocker (was: Critical) > Too many open files issue in Jbosseap 7.1 Server > > > Key: LUCENE-9425 > URL: https://issues.apache.org/jira/browse/LUCENE-9425 > Project: Lucene - Core > Issue Type: Bug >Affects Versions: 6.5.1 >Reporter: Archana A M >Priority: Blocker > > Hi Team, > We are getting "too many open files" in server while trying to access apache > Lucene cache. > Could someone please suggest the recommended open file limit while using > apache Lucene in the application. > Please find the relevant details below > Lucene version - 6.5.1 > current ulimit soft limit - 1024 > current ulimit hard limit - 4096 > Server - Jbosseap 7.1 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13939) Extract any non-gradle related patches (deprecations, URL fixes, etc.) from gradle effort
[ https://issues.apache.org/jira/browse/SOLR-13939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157210#comment-17157210 ] Dawid Weiss commented on SOLR-13939: Hi Erick. This annotation (and others from randomizedtesting) is inherited by subclasses but is not cumulative. So you could have one annotation on the top class (which would be most convenient) but you can't "add or remove" filters in subclasses - the redefinition in a subclass is complete. The best way would be for all of those custom thread filters to be removed... The whole point of thread leak detection is to signal that something is wrong... once you let threads slip between test cases you can no longer be sure of the consistency of behavior. > Extract any non-gradle related patches (deprecations, URL fixes, etc.) from > gradle effort > - > > Key: SOLR-13939 > URL: https://issues.apache.org/jira/browse/SOLR-13939 > Project: Solr > Issue Type: Sub-task >Reporter: Dawid Weiss >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-13939.patch, eoe_merged.patch > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14575) Solr restore is failing when basic authentication is enabled
[
https://issues.apache.org/jira/browse/SOLR-14575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157273#comment-17157273
]
Jan Høydahl commented on SOLR-14575:
[~ykonathala] the issue is still not reproduced. You can help us reproducing by
downloading a fresh version of Solr (e.g. 8.6.0), and trying to reproduce on a
minimal collection with minimal security.json config. If you are able to
provide a step by step reproduction instruction from scratch, then others may
be able to debug and see what is happening.
Have you tried with forwardCredentials=false?
> Solr restore is failing when basic authentication is enabled
>
>
> Key: SOLR-14575
> URL: https://issues.apache.org/jira/browse/SOLR-14575
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Backup/Restore
>Affects Versions: 8.2
>Reporter: Yaswanth
>Priority: Blocker
>
> Hi Team,
> I was testing backup / restore for solrcloud and its failing exactly when I
> am trying to restore a successfully backed up collection.
> I am using solr 8.2 with basic authentication enabled and then creating a 2
> replica collection. When I run the backup like
> curl -u xxx:xxx -k
> '[https://x.x.x.x:8080/solr/admin/collections?action=BACKUP&name=test&collection=test&location=/solrdatabkup'|https://x.x.x.x:8080/solr/admin/collections?action=BACKUP&name=test&collection=test&location=/solrdatabkup%27]
> it worked fine and I do see a folder was created with the collection name
> under /solrdatabackup
> But now when I deleted the existing collection and then try running restore
> api like
> curl -u xxx:xxx -k
> '[https://x.x.x.x:8080/solr/admin/collections?action=RESTORE&name=test&collection=test&location=/solrdatabkup'|https://x.x.x.x:8080/solr/admin/collections?action=BACKUP&name=test&collection=test&location=/solrdatabkup%27]
> its throwing an error like
> {
> "responseHeader":{
> "status":500,
> "QTime":457},
> "Operation restore caused
> *exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> ADDREPLICA failed to create replica",*
> "exception":{
> "msg":"ADDREPLICA failed to create replica",
> "rspCode":500},
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"ADDREPLICA failed to create replica",
> "trace":"org.apache.solr.common.SolrException: ADDREPLICA failed to create
> replica\n\tat
> org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:53)\n\tat
>
> org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:280)\n\tat
>
> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:252)\n\tat
>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
>
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:820)\n\tat
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:786)\n\tat
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:546)\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)\n\tat
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)\n\tat
>
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHan
[jira] [Created] (LUCENE-9426) UnifiedHighlighter does not handle SpanNotQuery correctly.
Christoph Goller created LUCENE-9426: Summary: UnifiedHighlighter does not handle SpanNotQuery correctly. Key: LUCENE-9426 URL: https://issues.apache.org/jira/browse/LUCENE-9426 Project: Lucene - Core Issue Type: Bug Affects Versions: 8.5.1 Environment: I tested with 8.5.1, but other versions are probably also affected. Reporter: Christoph Goller If UnifiedHighlighter uses MemoryIndexOffsetStrategy, it does not treat SpanNotQuery correctly. Since UnifiedHighlighter uses actual search in order to determine which locations to highlight, it should be consistent with search and only highlight locations in a document that really match the query. However, it does not for SpanNotQuery. For the query spanNot(spanNear([content:100, content:dollars], 1, true), content:thousand, 0, 0) it produces A 100 fucking dollars wasn't enough to fix it. ... We need 100 thousand dollars to buy the house -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9426) UnifiedHighlighter does not handle SpanNotQuery correctly.
[ https://issues.apache.org/jira/browse/LUCENE-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157281#comment-17157281 ] Christoph Goller commented on LUCENE-9426: -- Analysis: With PostingsOffsetStrategy highlighting for SpanNotQuery works correctly. With MemoryIndexOffsetStrategy UnifiedHighligher creates an In-Memory Index of the document that must be highlighted. However, it does not use the tokenstream produced by the indexAnalyzer. Instead it aplies a FilteringTokenFilter throwing away all tokens that do not occur in the query. I guess this is done for efficiency reasons. The filter is based on an automaton that is built by MultiTermHighlighting. MultiTermHighlighting is based on the Visitor concept and it ignores all subqueries that have BooleanClause.Occur.MUST_NOT. While this may be correct for a Boolean NOT-query, it is not correct for a SpanNotQuery. In the above example we need the SpanNot token. Otherwise the query logic is corrupted. As a fix I recommend to add all tokens form the query even if they have BooleanClause.Occur.MUST_NOT. Still the index remains small, but query logic will be correct. > UnifiedHighlighter does not handle SpanNotQuery correctly. > -- > > Key: LUCENE-9426 > URL: https://issues.apache.org/jira/browse/LUCENE-9426 > Project: Lucene - Core > Issue Type: Bug >Affects Versions: 8.5.1 > Environment: I tested with 8.5.1, but other versions are probably > also affected. >Reporter: Christoph Goller >Priority: Major > Labels: easyfix > > If UnifiedHighlighter uses MemoryIndexOffsetStrategy, it does not treat > SpanNotQuery correctly. > Since UnifiedHighlighter uses actual search in order to determine which > locations to highlight, it should be consistent with search and only > highlight locations in a document that really match the query. However, it > does not for SpanNotQuery. > For the query spanNot(spanNear([content:100, content:dollars], 1, true), > content:thousand, 0, 0) > it produces > A 100 fucking dollars wasn't enough to fix it. ... We need > 100 thousand dollars to buy the house -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-9426) UnifiedHighlighter does not handle SpanNotQuery correctly.
[ https://issues.apache.org/jira/browse/LUCENE-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157281#comment-17157281 ] Christoph Goller edited comment on LUCENE-9426 at 7/14/20, 11:02 AM: - Analysis: With PostingsOffsetStrategy highlighting for SpanNotQuery works correctly. With MemoryIndexOffsetStrategy UnifiedHighligher creates an In-Memory Index of the document that must be highlighted. However, it does not use the tokenstream produced by the indexAnalyzer. Instead it aplies a FilteringTokenFilter throwing away all tokens that do not occur in the query. I guess this is done for efficiency reasons. The filter is based on an automaton that is built by MultiTermHighlighting. MultiTermHighlighting is based on the Visitor concept and it ignores all subqueries that have BooleanClause.Occur.MUST_NOT. While this may be correct for a Boolean NOT-query, it is not correct for a SpanNotQuery. In the above example we need the SpanNot token. Otherwise the query logic is corrupted. As a fix I recommend to add all tokens form the query even if they have BooleanClause.Occur.MUST_NOT. Still the index remains small, but query logic will be correct. I attatch a unit test that demonstrates the problem. was (Author: gol...@detego-software.de): Analysis: With PostingsOffsetStrategy highlighting for SpanNotQuery works correctly. With MemoryIndexOffsetStrategy UnifiedHighligher creates an In-Memory Index of the document that must be highlighted. However, it does not use the tokenstream produced by the indexAnalyzer. Instead it aplies a FilteringTokenFilter throwing away all tokens that do not occur in the query. I guess this is done for efficiency reasons. The filter is based on an automaton that is built by MultiTermHighlighting. MultiTermHighlighting is based on the Visitor concept and it ignores all subqueries that have BooleanClause.Occur.MUST_NOT. While this may be correct for a Boolean NOT-query, it is not correct for a SpanNotQuery. In the above example we need the SpanNot token. Otherwise the query logic is corrupted. As a fix I recommend to add all tokens form the query even if they have BooleanClause.Occur.MUST_NOT. Still the index remains small, but query logic will be correct. > UnifiedHighlighter does not handle SpanNotQuery correctly. > -- > > Key: LUCENE-9426 > URL: https://issues.apache.org/jira/browse/LUCENE-9426 > Project: Lucene - Core > Issue Type: Bug >Affects Versions: 8.5.1 > Environment: I tested with 8.5.1, but other versions are probably > also affected. >Reporter: Christoph Goller >Priority: Major > Labels: easyfix > > If UnifiedHighlighter uses MemoryIndexOffsetStrategy, it does not treat > SpanNotQuery correctly. > Since UnifiedHighlighter uses actual search in order to determine which > locations to highlight, it should be consistent with search and only > highlight locations in a document that really match the query. However, it > does not for SpanNotQuery. > For the query spanNot(spanNear([content:100, content:dollars], 1, true), > content:thousand, 0, 0) > it produces > A 100 fucking dollars wasn't enough to fix it. ... We need > 100 thousand dollars to buy the house -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9426) UnifiedHighlighter does not handle SpanNotQuery correctly.
[ https://issues.apache.org/jira/browse/LUCENE-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christoph Goller updated LUCENE-9426: - Attachment: TestUnifiedHighlighter.java Status: Open (was: Open) > UnifiedHighlighter does not handle SpanNotQuery correctly. > -- > > Key: LUCENE-9426 > URL: https://issues.apache.org/jira/browse/LUCENE-9426 > Project: Lucene - Core > Issue Type: Bug >Affects Versions: 8.5.1 > Environment: I tested with 8.5.1, but other versions are probably > also affected. >Reporter: Christoph Goller >Priority: Major > Labels: easyfix > Attachments: TestUnifiedHighlighter.java > > > If UnifiedHighlighter uses MemoryIndexOffsetStrategy, it does not treat > SpanNotQuery correctly. > Since UnifiedHighlighter uses actual search in order to determine which > locations to highlight, it should be consistent with search and only > highlight locations in a document that really match the query. However, it > does not for SpanNotQuery. > For the query spanNot(spanNear([content:100, content:dollars], 1, true), > content:thousand, 0, 0) > it produces > A 100 fucking dollars wasn't enough to fix it. ... We need > 100 thousand dollars to buy the house -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14637) Update CloudSolrClient examples
[ https://issues.apache.org/jira/browse/SOLR-14637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157290#comment-17157290 ] Andras Salamon commented on SOLR-14637: --- Good idea [~epugh], let's keep the samples simple. I just copied the example code from javadoc, but setting the timeout is not really required. It could be deleted from all three example method. Can you also please check the spelling of my name? It's Salamon not Salaman. Thanks. > Update CloudSolrClient examples > --- > > Key: SOLR-14637 > URL: https://issues.apache.org/jira/browse/SOLR-14637 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Andras Salamon >Assignee: David Eric Pugh >Priority: Minor > Attachments: SOLR-14637.patch, SOLR-14637.patch > > Time Spent: 10m > Remaining Estimate: 0h > > CloudSolrClient.Builder() is deprecated ( SOLR-11629 ) but in the > documentation we still use this constructor. I think it would be better to > use a non-deprecated constructor in the examples. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9425) Too many open files issue in Jbosseap 7.1 Server
[ https://issues.apache.org/jira/browse/LUCENE-9425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Woodward resolved LUCENE-9425. --- Resolution: Not A Problem Please ask questions like this on the lucene user mailing list, we reserve JIRA for bugs and enhancements. > Too many open files issue in Jbosseap 7.1 Server > > > Key: LUCENE-9425 > URL: https://issues.apache.org/jira/browse/LUCENE-9425 > Project: Lucene - Core > Issue Type: Bug >Affects Versions: 6.5.1 >Reporter: Archana A M >Priority: Blocker > > Hi Team, > We are getting "too many open files" in server while trying to access apache > Lucene cache. > Could someone please suggest the recommended open file limit while using > apache Lucene in the application. > Please find the relevant details below > Lucene version - 6.5.1 > current ulimit soft limit - 1024 > current ulimit hard limit - 4096 > Server - Jbosseap 7.1 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13939) Extract any non-gradle related patches (deprecations, URL fixes, etc.) from gradle effort
[ https://issues.apache.org/jira/browse/SOLR-13939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157322#comment-17157322 ] Erick Erickson commented on SOLR-13939: --- Thanks Dawid: I totally agree that having the threads actually not leak is preferable, but we don't always have control in which case they're just noise that distracts from things we actually can fix. I've been wondering about how we test to see if leak problems are fixed in, say, HDFS rather than just accumulate forever. For instance, there's a specific case for a problem with Java9 that will shortly be obsolete when the minimum version is 11... There's also a specific exception for log4j2 etc. But I'll leave that discussion for another time. > Extract any non-gradle related patches (deprecations, URL fixes, etc.) from > gradle effort > - > > Key: SOLR-13939 > URL: https://issues.apache.org/jira/browse/SOLR-13939 > Project: Solr > Issue Type: Sub-task >Reporter: Dawid Weiss >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-13939.patch, eoe_merged.patch > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14648) Creating TLOG with pure multiple PULL replica, leading to 0 doc count
Sayan Das created SOLR-14648: Summary: Creating TLOG with pure multiple PULL replica, leading to 0 doc count Key: SOLR-14648 URL: https://issues.apache.org/jira/browse/SOLR-14648 Project: Solr Issue Type: Bug Security Level: Public (Default Security Level. Issues are Public) Components: SolrCloud Affects Versions: 8.3.1 Reporter: Sayan Das With only PULL replica whenever we create a new TLOG as leader fresh replication happens, resulting in flushing the older indexes from existing PULL replicas Steps to replicate: # Create 1 NRT or 1 TLOG replica as leader with multiple PULL replicas # Index few documents and let it replicate in all the replicas # Delete all the TLOG/NRT replica leaving PULL types # Create a new TLOG/NRT as leader, once recovery completes it replaces all the older indexes In ideal scenario it should have replicated from any one of the PULL replicas that has latest indexes after that TLOG/NRT replica should be registered as leader -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14649) Package manager should use SHA512, not SHA1
Ishan Chattopadhyaya created SOLR-14649: --- Summary: Package manager should use SHA512, not SHA1 Key: SOLR-14649 URL: https://issues.apache.org/jira/browse/SOLR-14649 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Reporter: Ishan Chattopadhyaya Due to a massive oversight, we only support SHA1 based package signing. We should immediately switch to SHA512. Post that, all existing packages must be re-signed with SHA512. I'll propose this for a 8.6.1 breakfix release. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14244) Remove ReplicaInfo
[
https://issues.apache.org/jira/browse/SOLR-14244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157345#comment-17157345
]
ASF subversion and git services commented on SOLR-14244:
Commit a0488c1cf1be4f02f3b7b449345f506ace64 in lucene-solr's branch
refs/heads/master from Andrzej Bialecki
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a0488c1 ]
SOLR-14244: Remove ReplicaInfo.
> Remove ReplicaInfo
> --
>
> Key: SOLR-14244
> URL: https://issues.apache.org/jira/browse/SOLR-14244
> Project: Solr
> Issue Type: Improvement
> Components: SolrCloud
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
> Fix For: master (9.0)
>
>
> SolrCloud uses {{Replica}} and {{ReplicaInfo}} beans more or less
> interchangeably and rather inconsistently across the code base. They seem to
> mean exactly the same thing.
> We should get rid of one or the other.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14244) Remove ReplicaInfo
[
https://issues.apache.org/jira/browse/SOLR-14244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrzej Bialecki resolved SOLR-14244.
-
Resolution: Fixed
> Remove ReplicaInfo
> --
>
> Key: SOLR-14244
> URL: https://issues.apache.org/jira/browse/SOLR-14244
> Project: Solr
> Issue Type: Improvement
> Components: SolrCloud
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
> Fix For: master (9.0)
>
>
> SolrCloud uses {{Replica}} and {{ReplicaInfo}} beans more or less
> interchangeably and rather inconsistently across the code base. They seem to
> mean exactly the same thing.
> We should get rid of one or the other.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12847) Cut over implementation of maxShardsPerNode to a collection policy
[ https://issues.apache.org/jira/browse/SOLR-12847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157368#comment-17157368 ] Noble Paul commented on SOLR-12847: --- I'm not sure we should have this. The policy calculations are slow and screwed up. Until we fix that , this should be a bad decision > Cut over implementation of maxShardsPerNode to a collection policy > -- > > Key: SOLR-12847 > URL: https://issues.apache.org/jira/browse/SOLR-12847 > Project: Solr > Issue Type: Bug > Components: AutoScaling, SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Andrzej Bialecki >Priority: Major > Fix For: master (9.0) > > Time Spent: 40m > Remaining Estimate: 0h > > We've back and forth over handling maxShardsPerNode with autoscaling policies > (see SOLR-11005 for history). Now that we've reimplemented support for > creating collections with maxShardsPerNode when autoscaling policy is > enabled, we should re-look at how it is implemented. > I propose that we fold maxShardsPerNode (if specified) to a collection level > policy that overrides the corresponding default in cluster policy (see > SOLR-12845). We'll need to ensure that if maxShardsPerNode is specified then > the user sees neither violations nor corresponding suggestions because of the > default cluster policy. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14649) Package manager should use SHA512, not SHA1
[ https://issues.apache.org/jira/browse/SOLR-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Chattopadhyaya updated SOLR-14649: Security: Public (was: Private (Security Issue)) > Package manager should use SHA512, not SHA1 > --- > > Key: SOLR-14649 > URL: https://issues.apache.org/jira/browse/SOLR-14649 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Ishan Chattopadhyaya >Priority: Major > > Due to a massive oversight, we only support SHA1 based package signing. We > should immediately switch to SHA512. Post that, all existing packages must be > re-signed with SHA512. > I'll propose this for a 8.6.1 breakfix release. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14649) Package manager should use SHA512, not SHA1
[ https://issues.apache.org/jira/browse/SOLR-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157388#comment-17157388 ] Ishan Chattopadhyaya commented on SOLR-14649: - This is not an immediate danger, because: # Breaking SHA1 is still prohibitively expensive: https://wccftech.com/google-cracked-sha1/ # Not many packages are out there However, because adoption is still low, it is the right time to fix this. > Package manager should use SHA512, not SHA1 > --- > > Key: SOLR-14649 > URL: https://issues.apache.org/jira/browse/SOLR-14649 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Ishan Chattopadhyaya >Priority: Major > > Due to a massive oversight, we only support SHA1 based package signing. We > should immediately switch to SHA512. Post that, all existing packages must be > re-signed with SHA512. > I'll propose this for a 8.6.1 breakfix release. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14516) NPE during Realtime GET
[
https://issues.apache.org/jira/browse/SOLR-14516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157392#comment-17157392
]
David Smiley commented on SOLR-14516:
-
I love that this change was simple enough that it didn't actually involve
changes to RTG or other inner complex beasts. It seems that there is/was a
gotcha on
\{{org.apache.solr.schema.FieldType#write(org.apache.solr.response.TextResponseWriter,
java.lang.String, org.apache.lucene.index.IndexableField)}} such that
implementer/FieldType should be really careful about using 'f' – that it should
most likely call toExternal on it.
Can you please add javadocs to this write method about this so plugin authors
have a hope of doing this correctly?
> NPE during Realtime GET
> ---
>
> Key: SOLR-14516
> URL: https://issues.apache.org/jira/browse/SOLR-14516
> Project: Solr
> Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Attachments: SOLR-14516.patch, SOLR-14516.patch
>
>
> The exact reason is unknown. But the following is the stacktrace
>
> o.a.s.s.HttpSolrCall null:java.lang.NullPointerException\n\tat
> org.apache.solr.common.util.JsonTextWriter.writeStr(JsonTextWriter.java:83)\n\tat
> org.apache.solr.schema.StrField.write(StrField.java:101)\n\tat
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:124)\n\tat
>
> org.apache.solr.response.JSONWriter.writeSolrDocument(JSONWriter.java:106)\n\tat
>
> org.apache.solr.response.TextResponseWriter.writeSolrDocumentList(TextResponseWriter.java:170)\n\tat
>
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:147)\n\tat
>
> org.apache.solr.common.util.JsonTextWriter.writeNamedListAsMapWithDups(JsonTextWriter.java:386)\n\tat
>
> org.apache.solr.common.util.JsonTextWriter.writeNamedList(JsonTextWriter.java:292)\n\tat
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14649) Package manager should use SHA512, not SHA1
[ https://issues.apache.org/jira/browse/SOLR-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157405#comment-17157405 ] David Smiley commented on SOLR-14649: - To me, it doesn't seem that this new/experimental system justifies causing a bug-fix release for this security bug that I'm guessing is hard to exploit in practice. > Package manager should use SHA512, not SHA1 > --- > > Key: SOLR-14649 > URL: https://issues.apache.org/jira/browse/SOLR-14649 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Ishan Chattopadhyaya >Priority: Major > > Due to a massive oversight, we only support SHA1 based package signing. We > should immediately switch to SHA512. Post that, all existing packages must be > re-signed with SHA512. > I'll propose this for a 8.6.1 breakfix release. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9426) UnifiedHighlighter does not handle SpanNotQuery correctly.
[
https://issues.apache.org/jira/browse/LUCENE-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157410#comment-17157410
]
David Smiley commented on LUCENE-9426:
--
{quote}As a fix I recommend to add all tokens form the query even if they have
BooleanClause.Occur.MUST_NOT. Still the index remains small, but query logic
will be correct.
{quote}
Makes sense to me. Or alternatively, just disable this optimization if _any_
NOT is found if it's too difficult to do what you suggest.
Note that if you put offsets in your main index for this field, you'll not be
susceptible to this problem because there will be no re-analysis + MemoryIndex.
> UnifiedHighlighter does not handle SpanNotQuery correctly.
> --
>
> Key: LUCENE-9426
> URL: https://issues.apache.org/jira/browse/LUCENE-9426
> Project: Lucene - Core
> Issue Type: Bug
>Affects Versions: 8.5.1
> Environment: I tested with 8.5.1, but other versions are probably
> also affected.
>Reporter: Christoph Goller
>Priority: Major
> Labels: easyfix
> Attachments: TestUnifiedHighlighter.java
>
>
> If UnifiedHighlighter uses MemoryIndexOffsetStrategy, it does not treat
> SpanNotQuery correctly.
> Since UnifiedHighlighter uses actual search in order to determine which
> locations to highlight, it should be consistent with search and only
> highlight locations in a document that really match the query. However, it
> does not for SpanNotQuery.
> For the query spanNot(spanNear([content:100, content:dollars], 1, true),
> content:thousand, 0, 0)
> it produces
> A 100 fucking dollars wasn't enough to fix it. ... We need
> 100 thousand dollars to buy the house
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9426) UnifiedHighlighter does not handle SpanNotQuery correctly.
[ https://issues.apache.org/jira/browse/LUCENE-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley updated LUCENE-9426: - Component/s: modules/highlighter > UnifiedHighlighter does not handle SpanNotQuery correctly. > -- > > Key: LUCENE-9426 > URL: https://issues.apache.org/jira/browse/LUCENE-9426 > Project: Lucene - Core > Issue Type: Bug > Components: modules/highlighter >Affects Versions: 8.5.1 > Environment: I tested with 8.5.1, but other versions are probably > also affected. >Reporter: Christoph Goller >Priority: Major > Labels: easyfix > Attachments: TestUnifiedHighlighter.java > > > If UnifiedHighlighter uses MemoryIndexOffsetStrategy, it does not treat > SpanNotQuery correctly. > Since UnifiedHighlighter uses actual search in order to determine which > locations to highlight, it should be consistent with search and only > highlight locations in a document that really match the query. However, it > does not for SpanNotQuery. > For the query spanNot(spanNear([content:100, content:dollars], 1, true), > content:thousand, 0, 0) > it produces > A 100 fucking dollars wasn't enough to fix it. ... We need > 100 thousand dollars to buy the house -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9426) UnifiedHighlighter ANALYSIS mode does not accurately highlight SpanNotQuery or MUST_NOT
[ https://issues.apache.org/jira/browse/LUCENE-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley updated LUCENE-9426: - Summary: UnifiedHighlighter ANALYSIS mode does not accurately highlight SpanNotQuery or MUST_NOT (was: UnifiedHighlighter ANALYSIS mode does not accurately highlight SpanNotQuery or NOT) > UnifiedHighlighter ANALYSIS mode does not accurately highlight SpanNotQuery > or MUST_NOT > --- > > Key: LUCENE-9426 > URL: https://issues.apache.org/jira/browse/LUCENE-9426 > Project: Lucene - Core > Issue Type: Bug > Components: modules/highlighter >Affects Versions: 8.5.1 > Environment: I tested with 8.5.1, but other versions are probably > also affected. >Reporter: Christoph Goller >Priority: Major > Labels: easyfix > Attachments: TestUnifiedHighlighter.java > > > If UnifiedHighlighter uses MemoryIndexOffsetStrategy, it does not treat > SpanNotQuery correctly. > Since UnifiedHighlighter uses actual search in order to determine which > locations to highlight, it should be consistent with search and only > highlight locations in a document that really match the query. However, it > does not for SpanNotQuery. > For the query spanNot(spanNear([content:100, content:dollars], 1, true), > content:thousand, 0, 0) > it produces > A 100 fucking dollars wasn't enough to fix it. ... We need > 100 thousand dollars to buy the house -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9426) UnifiedHighlighter ANALYSIS mode does not accurately highlight SpanNotQuery or NOT
[ https://issues.apache.org/jira/browse/LUCENE-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley updated LUCENE-9426: - Summary: UnifiedHighlighter ANALYSIS mode does not accurately highlight SpanNotQuery or NOT (was: UnifiedHighlighter does not handle SpanNotQuery correctly.) > UnifiedHighlighter ANALYSIS mode does not accurately highlight SpanNotQuery > or NOT > -- > > Key: LUCENE-9426 > URL: https://issues.apache.org/jira/browse/LUCENE-9426 > Project: Lucene - Core > Issue Type: Bug > Components: modules/highlighter >Affects Versions: 8.5.1 > Environment: I tested with 8.5.1, but other versions are probably > also affected. >Reporter: Christoph Goller >Priority: Major > Labels: easyfix > Attachments: TestUnifiedHighlighter.java > > > If UnifiedHighlighter uses MemoryIndexOffsetStrategy, it does not treat > SpanNotQuery correctly. > Since UnifiedHighlighter uses actual search in order to determine which > locations to highlight, it should be consistent with search and only > highlight locations in a document that really match the query. However, it > does not for SpanNotQuery. > For the query spanNot(spanNear([content:100, content:dollars], 1, true), > content:thousand, 0, 0) > it produces > A 100 fucking dollars wasn't enough to fix it. ... We need > 100 thousand dollars to buy the house -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14637) Update CloudSolrClient examples
[ https://issues.apache.org/jira/browse/SOLR-14637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157416#comment-17157416 ] ASF subversion and git services commented on SOLR-14637: Commit 1d5a0ad8a358378abf00314d0b698221e9737584 in lucene-solr's branch refs/heads/master from Eric Pugh [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=1d5a0ad ] SOLR-14637 update CloudSolrClient examples to remove deprecated .Builder() method (#1670) * update CloudSolrClient examples to remove deprecated .Builder() method * remove extra method lines that arent specific to what we are explaining. > Update CloudSolrClient examples > --- > > Key: SOLR-14637 > URL: https://issues.apache.org/jira/browse/SOLR-14637 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Andras Salamon >Assignee: David Eric Pugh >Priority: Minor > Attachments: SOLR-14637.patch, SOLR-14637.patch > > Time Spent: 10m > Remaining Estimate: 0h > > CloudSolrClient.Builder() is deprecated ( SOLR-11629 ) but in the > documentation we still use this constructor. I think it would be better to > use a non-deprecated constructor in the examples. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] epugh merged pull request #1670: SOLR-14637 update CloudSolrClient examples to remove deprecated .Builder() method
epugh merged pull request #1670: URL: https://github.com/apache/lucene-solr/pull/1670 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14637) Update CloudSolrClient examples
[ https://issues.apache.org/jira/browse/SOLR-14637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157420#comment-17157420 ] ASF subversion and git services commented on SOLR-14637: Commit 99a12b58d32f7feaba2260fecd4fd5c8245a23c5 in lucene-solr's branch refs/heads/branch_8x from Eric Pugh [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=99a12b5 ] SOLR-14637 update CloudSolrClient examples to remove deprecated .Builder() method (#1670) * update CloudSolrClient examples to remove deprecated .Builder() method * remove extra method lines that arent specific to what we are explaining. > Update CloudSolrClient examples > --- > > Key: SOLR-14637 > URL: https://issues.apache.org/jira/browse/SOLR-14637 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Andras Salamon >Assignee: David Eric Pugh >Priority: Minor > Attachments: SOLR-14637.patch, SOLR-14637.patch > > Time Spent: 20m > Remaining Estimate: 0h > > CloudSolrClient.Builder() is deprecated ( SOLR-11629 ) but in the > documentation we still use this constructor. I think it would be better to > use a non-deprecated constructor in the examples. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14648) Creating TLOG with pure multiple PULL replica, leading to 0 doc count
[ https://issues.apache.org/jira/browse/SOLR-14648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157424#comment-17157424 ] Erick Erickson commented on SOLR-14648: --- First, I agree that nuking all the existing PULL replica indexes is A Bad Thing in the scenario you point out. That said I don't think picking a PULL replica to grab the index from automatically is a good solution since there's no guarantee that any PULL replica is up to date. Consider this scenario: PULL replica1 is offline the TLOG and other PULL replicas get lots of udpates. For some unfathomable reason, everything except replica1 is taken down. Now a new TLOG replica is created and it gets the index from replica1 which isn't up to date at all. This is just the most egregious scenario. In a more realistic scenario, the PULL replicas may or may not have gotten the latest index changes when the TLOG replica goes down, so how would it be possible to choose among them? In fact there's no guarantee at all that _any_ of the PULL replicas remaining have the latest updates. I'm thinking of something along the lines of creating a new TLOG replica in your scenario failing. More generally failing if there are no active TLOG replicas (leaders) in the existing collection. We'd need a way to fix this, perhaps a way to "promote" a PULL replica to a TLOG replica. There'd still be the possibility of losing documents, but just like FORCELEADER we can document this and let people decide whether it's worth the risk or they should just reindex everything. We should not risk data inconsistency without somehow making sure that the users understand the risk. > Creating TLOG with pure multiple PULL replica, leading to 0 doc count > - > > Key: SOLR-14648 > URL: https://issues.apache.org/jira/browse/SOLR-14648 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 8.3.1 >Reporter: Sayan Das >Priority: Major > > With only PULL replica whenever we create a new TLOG as leader fresh > replication happens, resulting in flushing the older indexes from existing > PULL replicas > Steps to replicate: > # Create 1 NRT or 1 TLOG replica as leader with multiple PULL replicas > # Index few documents and let it replicate in all the replicas > # Delete all the TLOG/NRT replica leaving PULL types > # Create a new TLOG/NRT as leader, once recovery completes it replaces all > the older indexes > In ideal scenario it should have replicated from any one of the PULL replicas > that has latest indexes after that TLOG/NRT replica should be registered as > leader -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14637) Update CloudSolrClient examples
[ https://issues.apache.org/jira/browse/SOLR-14637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Eric Pugh updated SOLR-14637: --- Fix Version/s: 8.7 Resolution: Fixed Status: Resolved (was: Patch Available) Git commit 1d5a0ad8a358378abf00314d0b698221e9737584 > Update CloudSolrClient examples > --- > > Key: SOLR-14637 > URL: https://issues.apache.org/jira/browse/SOLR-14637 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Andras Salamon >Assignee: David Eric Pugh >Priority: Minor > Fix For: 8.7 > > Attachments: SOLR-14637.patch, SOLR-14637.patch > > Time Spent: 20m > Remaining Estimate: 0h > > CloudSolrClient.Builder() is deprecated ( SOLR-11629 ) but in the > documentation we still use this constructor. I think it would be better to > use a non-deprecated constructor in the examples. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14649) Package manager should use SHA512, not SHA1
[ https://issues.apache.org/jira/browse/SOLR-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157423#comment-17157423 ] Ishan Chattopadhyaya commented on SOLR-14649: - I just wanted to have this go out asap so that package authors can immediately make a switch, before it is late (and many people have started using this already). I'm specially concerned about Yasa adoption and DIH adoption. There's SOLR-14593 as well, which could go into such a release. However, I don't mind waiting till 8.7 as well. > Package manager should use SHA512, not SHA1 > --- > > Key: SOLR-14649 > URL: https://issues.apache.org/jira/browse/SOLR-14649 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Ishan Chattopadhyaya >Priority: Major > > Due to a massive oversight, we only support SHA1 based package signing. We > should immediately switch to SHA512. Post that, all existing packages must be > re-signed with SHA512. > I'll propose this for a 8.6.1 breakfix release. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14648) Creating TLOG with pure multiple PULL replica, leading to 0 doc count
[ https://issues.apache.org/jira/browse/SOLR-14648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157456#comment-17157456 ] Ishan Chattopadhyaya commented on SOLR-14648: - Agree, Erick. This will result in a data loss. I propose we fix this as: When a TLOG replica comes up and sees there are no other TLOG/NRT replicas to replicate from, either # TLOG replica doesn't become a leader, since it is behind the PULL replicas. This will stop the scary situation where it becomes leader with empty index. # Or, when the ADDREPLICA command contains some special expert level flag, it replicas from the PULL replica (with the full knowledge that there will be a data loss situation). WDYT, [~sayan.das], [~erickerickson]? > Creating TLOG with pure multiple PULL replica, leading to 0 doc count > - > > Key: SOLR-14648 > URL: https://issues.apache.org/jira/browse/SOLR-14648 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 8.3.1 >Reporter: Sayan Das >Priority: Major > > With only PULL replica whenever we create a new TLOG as leader fresh > replication happens, resulting in flushing the older indexes from existing > PULL replicas > Steps to replicate: > # Create 1 NRT or 1 TLOG replica as leader with multiple PULL replicas > # Index few documents and let it replicate in all the replicas > # Delete all the TLOG/NRT replica leaving PULL types > # Create a new TLOG/NRT as leader, once recovery completes it replaces all > the older indexes > In ideal scenario it should have replicated from any one of the PULL replicas > that has latest indexes after that TLOG/NRT replica should be registered as > leader -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14648) Creating TLOG with pure multiple PULL replica, leading to 0 doc count
[ https://issues.apache.org/jira/browse/SOLR-14648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157456#comment-17157456 ] Ishan Chattopadhyaya edited comment on SOLR-14648 at 7/14/20, 3:19 PM: --- Agree, Erick. This will result in a data loss. I propose we fix this as: When a TLOG replica comes up and sees there are no other TLOG/NRT replicas to replicate from, either # TLOG replica doesn't become a leader, since it is behind the PULL replicas. This will stop the scary situation where it becomes leader with empty index. # Or, when the ADDREPLICA command (for creating that TLOG replica) contains some special expert level flag, it replicates from the PULL replica (with the full knowledge that there will be a data loss situation). WDYT, [~sayan.das], [~erickerickson]? was (Author: ichattopadhyaya): Agree, Erick. This will result in a data loss. I propose we fix this as: When a TLOG replica comes up and sees there are no other TLOG/NRT replicas to replicate from, either # TLOG replica doesn't become a leader, since it is behind the PULL replicas. This will stop the scary situation where it becomes leader with empty index. # Or, when the ADDREPLICA command contains some special expert level flag, it replicas from the PULL replica (with the full knowledge that there will be a data loss situation). WDYT, [~sayan.das], [~erickerickson]? > Creating TLOG with pure multiple PULL replica, leading to 0 doc count > - > > Key: SOLR-14648 > URL: https://issues.apache.org/jira/browse/SOLR-14648 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 8.3.1 >Reporter: Sayan Das >Priority: Major > > With only PULL replica whenever we create a new TLOG as leader fresh > replication happens, resulting in flushing the older indexes from existing > PULL replicas > Steps to replicate: > # Create 1 NRT or 1 TLOG replica as leader with multiple PULL replicas > # Index few documents and let it replicate in all the replicas > # Delete all the TLOG/NRT replica leaving PULL types > # Create a new TLOG/NRT as leader, once recovery completes it replaces all > the older indexes > In ideal scenario it should have replicated from any one of the PULL replicas > that has latest indexes after that TLOG/NRT replica should be registered as > leader -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1669: SOLR-14151 Make schema components load from packages
dsmiley commented on a change in pull request #1669:
URL: https://github.com/apache/lucene-solr/pull/1669#discussion_r454404343
##
File path: solr/core/src/java/org/apache/solr/core/SolrResourceLoader.java
##
@@ -198,6 +206,11 @@ synchronized void reloadLuceneSPI() {
TokenizerFactory.reloadTokenizers(this.classLoader);
}
+ public SolrCore getCore(){
Review comment:
This is not used?
##
File path: solr/core/src/java/org/apache/solr/pkg/PackageListeners.java
##
@@ -96,15 +97,42 @@ private synchronized void
invokeListeners(PackageLoader.Package pkg) {
public interface Listener {
-/**Name of the package or null to loisten to all package changes
+/**Name of the package or null to listen to all package changes
*/
String packageName();
PluginInfo pluginInfo();
-void changed(PackageLoader.Package pkg);
+void changed(PackageLoader.Package pkg, Ctx ctx);
PackageLoader.Package.Version getPackageVersion();
+class Ctx {
+ private Map runLater;
+
+ /** If there are multiple packages to be updated and there are multiple
listeners,
+ * This is executed after all of the {@link
Listener#changed(PackageLoader.Package, Ctx)}
+ * calls are invoked. The name is a unique identifier that can be used
by consumers to avoid duplicate
+ * If no deduplication is required, use null
+ * runnable objects
+ */
+ public void runLater(String name, Runnable runnable ) {
+if(runLater == null) runLater = new LinkedHashMap<>();
+if(name == null) {
+ name = runnable.getClass().getSimpleName() + "@" +
runnable.hashCode();
+}
+runLater.put(name, runnable);
+ }
+ private void runLaterTasks(){
+if(runLater == null) return;
+new Thread(() -> runLater.forEach((s, runnable) -> {
Review comment:
Could use find a Solr based ExecutorService for this instead? It sets
up MDC and we ensure it gets shut down.
##
File path: solr/core/src/java/org/apache/solr/handler/SchemaHandler.java
##
@@ -202,6 +202,38 @@ private void handleGET(SolrQueryRequest req,
SolrQueryResponse rsp) {
}
}
+ /**If a plugin is loaded from a package, the version of the package being
used should be added
+ * to the response
+ *
+ */
+ @SuppressWarnings("rawtypes")
+ private void insertPackageInfo(Object o, SolrQueryRequest req) {
+if(!req.getParams().getBool("meta",false)) return;
Review comment:
nitpick: auto-format this code to be consistent with spaces
##
File path: solr/core/src/java/org/apache/solr/handler/StreamHandler.java
##
@@ -158,8 +158,8 @@ public Class getClazz() {
}
@Override
-protected void initNewInstance(PackageLoader.Package.Version newest) {
- clazz = newest.getLoader().findClass(pluginInfo.className,
Expressible.class);
+protected Object initNewInstance(PackageLoader.Package.Version newest) {
+ return clazz = newest.getLoader().findClass(pluginInfo.className,
Expressible.class);
Review comment:
This code here returns a class and not an instance. This seems wrong?
##
File path: solr/core/src/java/org/apache/solr/core/SolrResourceLoader.java
##
@@ -752,6 +765,9 @@ public void close() throws IOException {
}
+ public CoreContainer getCoreContainer(){
Review comment:
this is not used?
##
File path: solr/core/src/java/org/apache/solr/core/SolrClassLoader.java
##
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core;
+
+
+/**A generic interface to load plugin classes
Review comment:
nit: see my longer comment on formatting javadoc. This case right here
really gets to me.
##
File path: solr/core/src/java/org/apache/solr/pkg/PackageListeners.java
##
@@ -96,15 +97,42 @@ private synchronized void
invokeListeners(PackageLoader.Package pkg) {
public interface Listener {
-/**Name of the package or null to loisten to all package changes
+/**Name of the package or null to listen to all package changes
*/
String packageName();
PluginInfo p
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1669: SOLR-14151 Make schema components load from packages
dsmiley commented on a change in pull request #1669:
URL: https://github.com/apache/lucene-solr/pull/1669#discussion_r454450712
##
File path: solr/core/src/java/org/apache/solr/schema/IndexSchema.java
##
@@ -188,6 +190,7 @@ public IndexSchema(String name, InputSource is, Version
luceneVersion, SolrResou
protected IndexSchema(Version luceneVersion, SolrResourceLoader loader,
Properties substitutableProperties) {
this.luceneVersion = Objects.requireNonNull(luceneVersion);
this.loader = loader;
+this.solrClassLoader = loader.getCore() == null? loader:
loader.getCore().getSchemaPluginsLoader();
Review comment:
Continuing my idea... Imagine a "ZkConfigSetResourceProvider" and
"FileSystemConfigSetResourceProvider" pair of classes implementing an interface
ConfigSetResourceProvider that is only for serving a resource (InputStream),
not a class. Just method openResource ideally. Then imagine a SRL that
demands a ConfigSetResourceProvider, and a ClassLoader (or SolrClassLoader as
you prefer). See where this is going?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14537) Improve performance of ExportWriter
[
https://issues.apache.org/jira/browse/SOLR-14537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrzej Bialecki resolved SOLR-14537.
-
Resolution: Fixed
> Improve performance of ExportWriter
> ---
>
> Key: SOLR-14537
> URL: https://issues.apache.org/jira/browse/SOLR-14537
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Export Writer
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
> Fix For: 8.7
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Retrieving, sorting and writing out documents in {{ExportWriter}} are three
> aspects of the /export handler that can be further optimized.
> SOLR-14470 introduced some level of caching in {{StringValue}}. Further
> options for caching and speedups should be explored.
> Currently the sort/retrieve and write operations are done sequentially, but
> they could be parallelized, considering that they block on different channels
> - the first is index reading & CPU bound, the other is bound by the receiving
> end because it uses blocking IO. The sorting and retrieving of values could
> be done in parallel with the operation of writing out the current batch of
> results.
> One possible approach here would be to use "double buffering" where one
> buffered batch that is ready (already sorted and retrieved) is being written
> out, while the other batch is being prepared in a background thread, and when
> both are done the buffers are swapped. This wouldn't complicate the current
> code too much but it should instantly give up to 2x higher throughput.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-12815) Implement maxOps limit for IndexSizeTrigger
[
https://issues.apache.org/jira/browse/SOLR-12815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrzej Bialecki resolved SOLR-12815.
-
Resolution: Fixed
> Implement maxOps limit for IndexSizeTrigger
> ---
>
> Key: SOLR-12815
> URL: https://issues.apache.org/jira/browse/SOLR-12815
> Project: Solr
> Issue Type: Sub-task
> Components: AutoScaling
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
> Fix For: 7.6
>
>
> {{IndexSizeTrigger}} can currently generate unlimited number of requested
> operations (such as split shard). Combined with the avalanche effect in
> SOLR-12730 this may cause a massive spike in the cluster load, and some of
> these operations may fail.
> It's probably better to put an arbitrary limit on the maximum number of
> generated ops requested in one trigger event. This will delay some of the
> needed changes (thus leading to thresholds being exceeded to a larger degree)
> but it will increase the likelihood that all operations succeed.
> A similar limit already exists in {{SearchRateTrigger}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-12815) Implement maxOps limit for IndexSizeTrigger
[
https://issues.apache.org/jira/browse/SOLR-12815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrzej Bialecki updated SOLR-12815:
Fix Version/s: 7.6
> Implement maxOps limit for IndexSizeTrigger
> ---
>
> Key: SOLR-12815
> URL: https://issues.apache.org/jira/browse/SOLR-12815
> Project: Solr
> Issue Type: Sub-task
> Components: AutoScaling
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
> Fix For: 7.6
>
>
> {{IndexSizeTrigger}} can currently generate unlimited number of requested
> operations (such as split shard). Combined with the avalanche effect in
> SOLR-12730 this may cause a massive spike in the cluster load, and some of
> these operations may fail.
> It's probably better to put an arbitrary limit on the maximum number of
> generated ops requested in one trigger event. This will delay some of the
> needed changes (thus leading to thresholds being exceeded to a larger degree)
> but it will increase the likelihood that all operations succeed.
> A similar limit already exists in {{SearchRateTrigger}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1669: SOLR-14151 Make schema components load from packages
dsmiley commented on a change in pull request #1669:
URL: https://github.com/apache/lucene-solr/pull/1669#discussion_r454456488
##
File path: solr/core/src/java/org/apache/solr/schema/IndexSchema.java
##
@@ -188,6 +190,7 @@ public IndexSchema(String name, InputSource is, Version
luceneVersion, SolrResou
protected IndexSchema(Version luceneVersion, SolrResourceLoader loader,
Properties substitutableProperties) {
this.luceneVersion = Objects.requireNonNull(luceneVersion);
this.loader = loader;
+this.solrClassLoader = loader.getCore() == null? loader:
loader.getCore().getSchemaPluginsLoader();
Review comment:
With that idea, there would only need to be one SRL class (no
subclasses), and it'd be easy to create new instances based on those two
primary components.
I'm sure there is some tech debt entanglements in SRL relating to tracking
instancePath (there's a TODO I added in there, initLibs() to remove that one)
and harder are waitingForCore, infoMBeans, waitingForResources, and of course
managedResourceRegistry. If those get moved off somehow, then I hope the
picture I propose becomes more clear.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Assigned] (SOLR-14647) Importing Gradle Projects into Eclipse pollutes checkout
[ https://issues.apache.org/jira/browse/SOLR-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Dyer reassigned SOLR-14647: - Assignee: James Dyer > Importing Gradle Projects into Eclipse pollutes checkout > > > Key: SOLR-14647 > URL: https://issues.apache.org/jira/browse/SOLR-14647 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Affects Versions: master (9.0) >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Attachments: SOLR-14647.patch > > > Switch to master branch, then open Eclipse IDE and select "file > import > > existing gradle project" to import lucene/solr. Afterwards, "git status" > shows unstaged ".project", ".classpath", ".settings" and "bin" > files/directories. > Adjust the .gitignore file to correctly filter these out. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14647) Importing Gradle Projects into Eclipse pollutes checkout
[ https://issues.apache.org/jira/browse/SOLR-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157473#comment-17157473 ] ASF subversion and git services commented on SOLR-14647: Commit e5007c15ee7f266639fff4f209aa56626b6987ab in lucene-solr's branch refs/heads/master from James Dyer [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e5007c1 ] SOLR-14647 - fix .gitignore for eclipse users > Importing Gradle Projects into Eclipse pollutes checkout > > > Key: SOLR-14647 > URL: https://issues.apache.org/jira/browse/SOLR-14647 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Affects Versions: master (9.0) >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Attachments: SOLR-14647.patch > > > Switch to master branch, then open Eclipse IDE and select "file > import > > existing gradle project" to import lucene/solr. Afterwards, "git status" > shows unstaged ".project", ".classpath", ".settings" and "bin" > files/directories. > Adjust the .gitignore file to correctly filter these out. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14647) Importing Gradle Projects into Eclipse pollutes checkout
[ https://issues.apache.org/jira/browse/SOLR-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Dyer resolved SOLR-14647. --- Resolution: Fixed > Importing Gradle Projects into Eclipse pollutes checkout > > > Key: SOLR-14647 > URL: https://issues.apache.org/jira/browse/SOLR-14647 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Affects Versions: master (9.0) >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Attachments: SOLR-14647.patch > > > Switch to master branch, then open Eclipse IDE and select "file > import > > existing gradle project" to import lucene/solr. Afterwards, "git status" > shows unstaged ".project", ".classpath", ".settings" and "bin" > files/directories. > Adjust the .gitignore file to correctly filter these out. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14647) Importing Gradle Projects into Eclipse pollutes checkout
[ https://issues.apache.org/jira/browse/SOLR-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Dyer updated SOLR-14647: -- Fix Version/s: master (9.0) > Importing Gradle Projects into Eclipse pollutes checkout > > > Key: SOLR-14647 > URL: https://issues.apache.org/jira/browse/SOLR-14647 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Affects Versions: master (9.0) >Reporter: James Dyer >Assignee: James Dyer >Priority: Minor > Fix For: master (9.0) > > Attachments: SOLR-14647.patch > > > Switch to master branch, then open Eclipse IDE and select "file > import > > existing gradle project" to import lucene/solr. Afterwards, "git status" > shows unstaged ".project", ".classpath", ".settings" and "bin" > files/directories. > Adjust the .gitignore file to correctly filter these out. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12847) Cut over implementation of maxShardsPerNode to a collection policy
[
https://issues.apache.org/jira/browse/SOLR-12847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157484#comment-17157484
]
Andrzej Bialecki commented on SOLR-12847:
-
This is for 9.0 only. And {{maxShardsPerNode}} was already broken anyway - it
didn't have any impact on the placements, it was just artificially limiting the
average number of replicas / nodes.
> Cut over implementation of maxShardsPerNode to a collection policy
> --
>
> Key: SOLR-12847
> URL: https://issues.apache.org/jira/browse/SOLR-12847
> Project: Solr
> Issue Type: Bug
> Components: AutoScaling, SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Andrzej Bialecki
>Priority: Major
> Fix For: master (9.0)
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> We've back and forth over handling maxShardsPerNode with autoscaling policies
> (see SOLR-11005 for history). Now that we've reimplemented support for
> creating collections with maxShardsPerNode when autoscaling policy is
> enabled, we should re-look at how it is implemented.
> I propose that we fold maxShardsPerNode (if specified) to a collection level
> policy that overrides the corresponding default in cluster policy (see
> SOLR-12845). We'll need to ensure that if maxShardsPerNode is specified then
> the user sees neither violations nor corresponding suggestions because of the
> default cluster policy.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9427) Unified highlighter can fail to highlight fuzzy query
[ https://issues.apache.org/jira/browse/LUCENE-9427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julie Tibshirani updated LUCENE-9427: - Description: If a fuzzy query corresponds to an exact match (for example it has with maxEdits: 0), then the unified highlighter doesn't produce highlights for the matching terms. I think this is due to the fact that when visiting a fuzzy query, the exact terms are now consumed separately from automata. The unified highlighter doesn't account for the terms and misses highlighting them. was: If a fuzzy query corresponds to an exact match (for example it has with maxEdits: 0), then the unified highlighter doesn't produce highlights for the matching terms. I think this is due to the fact that when visiting a fuzzy query, the exact terms are now consumed separately from automata. The unified highlighter doesn't account for the terms and misses highlighting them. > Unified highlighter can fail to highlight fuzzy query > - > > Key: LUCENE-9427 > URL: https://issues.apache.org/jira/browse/LUCENE-9427 > Project: Lucene - Core > Issue Type: Bug >Reporter: Julie Tibshirani >Priority: Major > > If a fuzzy query corresponds to an exact match (for example it has with > maxEdits: 0), then the unified highlighter doesn't produce highlights for the > matching terms. > I think this is due to the fact that when visiting a fuzzy query, the exact > terms are now consumed separately from automata. The unified highlighter > doesn't account for the terms and misses highlighting them. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9427) Unified highlighter can fail to highlight fuzzy query
Julie Tibshirani created LUCENE-9427: Summary: Unified highlighter can fail to highlight fuzzy query Key: LUCENE-9427 URL: https://issues.apache.org/jira/browse/LUCENE-9427 Project: Lucene - Core Issue Type: Bug Reporter: Julie Tibshirani If a fuzzy query corresponds to an exact match (for example it has with maxEdits: 0), then the unified highlighter doesn't produce highlights for the matching terms. I think this is due to the fact that when visiting a fuzzy query, the exact terms are now consumed separately from automata. The unified highlighter doesn't account for the terms and misses highlighting them. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14636) Provide a reference implementation for SolrCloud that is stable and fast.
[
https://issues.apache.org/jira/browse/SOLR-14636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157546#comment-17157546
]
Mark Robert Miller commented on SOLR-14636:
---
Almost to enabling ignored tests. I have to fix a leader election issue with
collection delete, been trying to put it off, but i'll do it today or tomorrow
and current tests should be getting pretty close to solid.
> Provide a reference implementation for SolrCloud that is stable and fast.
> -
>
> Key: SOLR-14636
> URL: https://issues.apache.org/jira/browse/SOLR-14636
> Project: Solr
> Issue Type: Task
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Robert Miller
>Assignee: Mark Robert Miller
>Priority: Major
>
> SolrCloud powers critical infrastructure and needs the ability to run quickly
> with stability. This reference implementation will allow for this.
> *location*: [https://github.com/apache/lucene-solr/tree/reference_impl]
> *status*: alpha
> *speed*: ludicrous
> *tests***:
> * *core*: near {color:#00875a}*solid*{color} with
> *{color:#de350b}ignores{color}*
> * *solrj*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *test-framework*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/analysis-extras*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/analytics*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/clustering*: near {color:#00875a}*solid*{color} with
> *{color:#de350b}ignores{color}*
> * *contrib/dataimporthandler*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/dataimporthandler-extras*: near {color:#00875a}*solid*{color}
> with *{color:#de350b}ignores{color}*
> * *contrib/extraction*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/jaegertracer-configurator*: near {color:#00875a}*solid*{color}
> with {color:#de350b}*ignores*{color}
> * *contrib/langid*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/prometheus-exporter*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/velocity*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> _* Running tests quickly and efficiently with strict policing will more
> frequently find bugs and requires a period of hardening._
> _** Non Nightly currently, Nightly comes last._
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jtibshirani opened a new pull request #1671: [LUCENE-9427] Ensure unified highlighter considers all terms in fuzzy query.
jtibshirani opened a new pull request #1671: URL: https://github.com/apache/lucene-solr/pull/1671 This fixes a regression where if a fuzzy query corresponds to an exact match (for example it has maxEdits: 0), then the unified highlighter doesn't produce highlights for the matching terms. The proposed fix is to always pass back an automaton when visiting a fuzzy query. This seems to match the previous behavior before the regression was introduced. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jtibshirani commented on pull request #1671: [LUCENE-9427] Ensure unified highlighter considers all terms in fuzzy query.
jtibshirani commented on pull request #1671: URL: https://github.com/apache/lucene-solr/pull/1671#issuecomment-658324084 It looks like this behavior changed during the `QueryVisitor` refactor: https://github.com/apache/lucene-solr/pull/581/files#diff-449ea2758bc22783ca3f819096e1b638R97. Previously we always extracted an automaton from fuzzy queries. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14648) Creating TLOG with pure multiple PULL replica, leading to 0 doc count
[ https://issues.apache.org/jira/browse/SOLR-14648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157579#comment-17157579 ] Erick Erickson commented on SOLR-14648: --- (2) seems safer, assuming that the TLOG replica never gets created without the special flag. Maybe the expert flag would be the PULL replica to grab the index from? (1) seems trickier. It'd have to be generalized to _never_ become the leader, even if the cluster was restarted afterwards. Having a TLOG replica successfully created seems like it'd have more places to go wrong. Also, how would the cluster get out of that state? You'd have to do something like FORCELEADER which would have some of the same problems... > Creating TLOG with pure multiple PULL replica, leading to 0 doc count > - > > Key: SOLR-14648 > URL: https://issues.apache.org/jira/browse/SOLR-14648 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 8.3.1 >Reporter: Sayan Das >Priority: Major > > With only PULL replica whenever we create a new TLOG as leader fresh > replication happens, resulting in flushing the older indexes from existing > PULL replicas > Steps to replicate: > # Create 1 NRT or 1 TLOG replica as leader with multiple PULL replicas > # Index few documents and let it replicate in all the replicas > # Delete all the TLOG/NRT replica leaving PULL types > # Create a new TLOG/NRT as leader, once recovery completes it replaces all > the older indexes > In ideal scenario it should have replicated from any one of the PULL replicas > that has latest indexes after that TLOG/NRT replica should be registered as > leader -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14648) Creating TLOG with pure multiple PULL replica, leading to 0 doc count
[ https://issues.apache.org/jira/browse/SOLR-14648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157579#comment-17157579 ] Erick Erickson edited comment on SOLR-14648 at 7/14/20, 7:01 PM: - (2) seems safer, assuming that the TLOG replica never gets created without the special flag. Maybe the expert flag would be the PULL replica to grab the index from? (1) seems trickier. It'd have to be generalized to _never_ become the leader, even if the cluster was restarted afterwards. Having a TLOG replica successfully created seems like it'd have more places to go wrong. Also, how would the cluster get out of that state? You'd have to do something like FORCELEADER which would have some of the same problems... H, or would the ability to "promote" a replica from PULL to TLOG work? was (Author: erickerickson): (2) seems safer, assuming that the TLOG replica never gets created without the special flag. Maybe the expert flag would be the PULL replica to grab the index from? (1) seems trickier. It'd have to be generalized to _never_ become the leader, even if the cluster was restarted afterwards. Having a TLOG replica successfully created seems like it'd have more places to go wrong. Also, how would the cluster get out of that state? You'd have to do something like FORCELEADER which would have some of the same problems... > Creating TLOG with pure multiple PULL replica, leading to 0 doc count > - > > Key: SOLR-14648 > URL: https://issues.apache.org/jira/browse/SOLR-14648 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 8.3.1 >Reporter: Sayan Das >Priority: Major > > With only PULL replica whenever we create a new TLOG as leader fresh > replication happens, resulting in flushing the older indexes from existing > PULL replicas > Steps to replicate: > # Create 1 NRT or 1 TLOG replica as leader with multiple PULL replicas > # Index few documents and let it replicate in all the replicas > # Delete all the TLOG/NRT replica leaving PULL types > # Create a new TLOG/NRT as leader, once recovery completes it replaces all > the older indexes > In ideal scenario it should have replicated from any one of the PULL replicas > that has latest indexes after that TLOG/NRT replica should be registered as > leader -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14649) Package manager should use SHA512, not SHA1
[ https://issues.apache.org/jira/browse/SOLR-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157621#comment-17157621 ] Jan Høydahl commented on SOLR-14649: Deprecate SHA1 but support it in parallell with SHA512 for 8.x. > Package manager should use SHA512, not SHA1 > --- > > Key: SOLR-14649 > URL: https://issues.apache.org/jira/browse/SOLR-14649 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Ishan Chattopadhyaya >Priority: Major > > Due to a massive oversight, we only support SHA1 based package signing. We > should immediately switch to SHA512. Post that, all existing packages must be > re-signed with SHA512. > I'll propose this for a 8.6.1 breakfix release. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14649) Package manager should use SHA512, not SHA1
[ https://issues.apache.org/jira/browse/SOLR-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157628#comment-17157628 ] Ishan Chattopadhyaya commented on SOLR-14649: - Problem with that approach is that an attacker can still spoof a SHA1 signature, and there's no benefit to switching to SHA512 then. Support for SHA1 should be disabled completely. > Package manager should use SHA512, not SHA1 > --- > > Key: SOLR-14649 > URL: https://issues.apache.org/jira/browse/SOLR-14649 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Ishan Chattopadhyaya >Priority: Major > > Due to a massive oversight, we only support SHA1 based package signing. We > should immediately switch to SHA512. Post that, all existing packages must be > re-signed with SHA512. > I'll propose this for a 8.6.1 breakfix release. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14649) Package manager should use SHA512, not SHA1
[ https://issues.apache.org/jira/browse/SOLR-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157633#comment-17157633 ] Ishan Chattopadhyaya commented on SOLR-14649: - [~rcmuir ], Any suggestions please on whether it is safe to use SHA1 for packages (installed via package manager) in 8x and moving to SHA512 only with 9x? > Package manager should use SHA512, not SHA1 > --- > > Key: SOLR-14649 > URL: https://issues.apache.org/jira/browse/SOLR-14649 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Ishan Chattopadhyaya >Priority: Major > > Due to a massive oversight, we only support SHA1 based package signing. We > should immediately switch to SHA512. Post that, all existing packages must be > re-signed with SHA512. > I'll propose this for a 8.6.1 breakfix release. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14583) Spell suggestion is returned even if hits are non-zero when spellcheck.maxResultsForSuggest=0
[
https://issues.apache.org/jira/browse/SOLR-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157676#comment-17157676
]
James Dyer commented on SOLR-14583:
---
[~munendrasn] I looked at this patch, but do not agree there is a bug. Your
new test case passes without changes to SpellCheckComponent, if we remove
"onlyMorePopular=true".
"maxResultsForSuggest" was designed as a better replacement for
"onlyMorePopular", which possibly should be removed entirely.
> Spell suggestion is returned even if hits are non-zero when
> spellcheck.maxResultsForSuggest=0
> -
>
> Key: SOLR-14583
> URL: https://issues.apache.org/jira/browse/SOLR-14583
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Munendra S N
>Assignee: Munendra S N
>Priority: Major
> Attachments: SOLR-14583.patch
>
>
> SOLR-4280 added to support fractional support for
> spellcheck.maxResultsForSuggest. After SOLR-4280,
> {{spellcheck.maxResultsForSuggest=0}} is treated same as not specify the
> {{spellcheck.maxResultsForSuggest}} parameter. This can cause spell
> suggestions to be returned even when hits are non-zero and greater than
> {{spellcheck.maxResultsForSuggest}} (i.e, greater than 0)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14649) Package manager should use SHA512, not SHA1
[ https://issues.apache.org/jira/browse/SOLR-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157691#comment-17157691 ] Robert Muir commented on SOLR-14649: Hi [~ichattopadhyaya], I don't know the big picture on how this package manager works. How is this hashing used? How are the packages signed? > Package manager should use SHA512, not SHA1 > --- > > Key: SOLR-14649 > URL: https://issues.apache.org/jira/browse/SOLR-14649 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Ishan Chattopadhyaya >Priority: Major > > Due to a massive oversight, we only support SHA1 based package signing. We > should immediately switch to SHA512. Post that, all existing packages must be > re-signed with SHA512. > I'll propose this for a 8.6.1 breakfix release. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14636) Provide a reference implementation for SolrCloud that is stable and fast.
[
https://issues.apache.org/jira/browse/SOLR-14636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157727#comment-17157727
]
Mark Robert Miller commented on SOLR-14636:
---
Almost to stability. The last time I was this close I woke up one morning and
wiped it with a new ubuntu install. I had a lot of other great stuff I didn't
go after, but may in time.
> Provide a reference implementation for SolrCloud that is stable and fast.
> -
>
> Key: SOLR-14636
> URL: https://issues.apache.org/jira/browse/SOLR-14636
> Project: Solr
> Issue Type: Task
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Robert Miller
>Assignee: Mark Robert Miller
>Priority: Major
>
> SolrCloud powers critical infrastructure and needs the ability to run quickly
> with stability. This reference implementation will allow for this.
> *location*: [https://github.com/apache/lucene-solr/tree/reference_impl]
> *status*: alpha
> *speed*: ludicrous
> *tests***:
> * *core*: near {color:#00875a}*solid*{color} with
> *{color:#de350b}ignores{color}*
> * *solrj*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *test-framework*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/analysis-extras*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/analytics*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/clustering*: near {color:#00875a}*solid*{color} with
> *{color:#de350b}ignores{color}*
> * *contrib/dataimporthandler*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/dataimporthandler-extras*: near {color:#00875a}*solid*{color}
> with *{color:#de350b}ignores{color}*
> * *contrib/extraction*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/jaegertracer-configurator*: near {color:#00875a}*solid*{color}
> with {color:#de350b}*ignores*{color}
> * *contrib/langid*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/prometheus-exporter*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/velocity*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> _* Running tests quickly and efficiently with strict policing will more
> frequently find bugs and requires a period of hardening._
> _** Non Nightly currently, Nightly comes last._
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14649) Package manager should use SHA512, not SHA1
[ https://issues.apache.org/jira/browse/SOLR-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157734#comment-17157734 ] Robert Muir commented on SOLR-14649: ok looking at the code here, its signing with SHA1WithRSA. but there's also a lot of other stuff going on here, maybe more problems get found in the future. In general, I feel inventing any package manager is asking for problems. But for sure, if you can't fix bugs going forwards and also balance back compat it will not really work out. So I think its worth making sure you can do that, e.g. create a new format that is more secure, and maybe have some flags (disabled by default ideally) that a user can enable if they want to support older insecure formats. Good error messages are important. > Package manager should use SHA512, not SHA1 > --- > > Key: SOLR-14649 > URL: https://issues.apache.org/jira/browse/SOLR-14649 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Ishan Chattopadhyaya >Priority: Major > > Due to a massive oversight, we only support SHA1 based package signing. We > should immediately switch to SHA512. Post that, all existing packages must be > re-signed with SHA512. > I'll propose this for a 8.6.1 breakfix release. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul commented on a change in pull request #1669: SOLR-14151 Make schema components load from packages
noblepaul commented on a change in pull request #1669:
URL: https://github.com/apache/lucene-solr/pull/1669#discussion_r454700567
##
File path: solr/core/src/java/org/apache/solr/core/SolrResourceLoader.java
##
@@ -752,6 +765,9 @@ public void close() throws IOException {
}
+ public CoreContainer getCoreContainer(){
Review comment:
This is necessary for #1666
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul commented on a change in pull request #1669: SOLR-14151 Make schema components load from packages
noblepaul commented on a change in pull request #1669:
URL: https://github.com/apache/lucene-solr/pull/1669#discussion_r454700983
##
File path: solr/core/src/java/org/apache/solr/handler/StreamHandler.java
##
@@ -158,8 +158,8 @@ public Class getClazz() {
}
@Override
-protected void initNewInstance(PackageLoader.Package.Version newest) {
- clazz = newest.getLoader().findClass(pluginInfo.className,
Expressible.class);
+protected Object initNewInstance(PackageLoader.Package.Version newest) {
+ return clazz = newest.getLoader().findClass(pluginInfo.className,
Expressible.class);
Review comment:
Actually no. it can be anything what the listener wants it to be. In
this case it's a class
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul commented on a change in pull request #1669: SOLR-14151 Make schema components load from packages
noblepaul commented on a change in pull request #1669:
URL: https://github.com/apache/lucene-solr/pull/1669#discussion_r454701342
##
File path: solr/core/src/java/org/apache/solr/pkg/PackageListeners.java
##
@@ -96,15 +97,42 @@ private synchronized void
invokeListeners(PackageLoader.Package pkg) {
public interface Listener {
-/**Name of the package or null to loisten to all package changes
+/**Name of the package or null to listen to all package changes
*/
String packageName();
PluginInfo pluginInfo();
-void changed(PackageLoader.Package pkg);
+void changed(PackageLoader.Package pkg, Ctx ctx);
PackageLoader.Package.Version getPackageVersion();
+class Ctx {
+ private Map runLater;
+
+ /** If there are multiple packages to be updated and there are multiple
listeners,
+ * This is executed after all of the {@link
Listener#changed(PackageLoader.Package, Ctx)}
+ * calls are invoked. The name is a unique identifier that can be used
by consumers to avoid duplicate
+ * If no deduplication is required, use null
+ * runnable objects
+ */
+ public void runLater(String name, Runnable runnable ) {
+if(runLater == null) runLater = new LinkedHashMap<>();
+if(name == null) {
+ name = runnable.getClass().getSimpleName() + "@" +
runnable.hashCode();
+}
+runLater.put(name, runnable);
+ }
+ private void runLaterTasks(){
+if(runLater == null) return;
+new Thread(() -> runLater.forEach((s, runnable) -> {
Review comment:
True. This needs to be done more gracefully.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul commented on a change in pull request #1669: SOLR-14151 Make schema components load from packages
noblepaul commented on a change in pull request #1669: URL: https://github.com/apache/lucene-solr/pull/1669#discussion_r454702702 ## File path: solr/core/src/java/org/apache/solr/core/SolrCore.java ## @@ -191,6 +191,11 @@ private String name; private String logid; // used to show what name is set + /** + * A unique id to differentiate multiple instances of the same core + * If we reload a core, the name remains same , but the id will be new + */ + public final UUID uniqueId = UUID.randomUUID(); Review comment: `AtomicLong` is mutable. We need an immutable object This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul commented on a change in pull request #1669: SOLR-14151 Make schema components load from packages
noblepaul commented on a change in pull request #1669:
URL: https://github.com/apache/lucene-solr/pull/1669#discussion_r454702783
##
File path: solr/core/src/java/org/apache/solr/core/SolrResourceLoader.java
##
@@ -198,6 +206,11 @@ synchronized void reloadLuceneSPI() {
TokenizerFactory.reloadTokenizers(this.classLoader);
}
+ public SolrCore getCore(){
Review comment:
This is used in #1666
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul commented on a change in pull request #1669: SOLR-14151 Make schema components load from packages
noblepaul commented on a change in pull request #1669:
URL: https://github.com/apache/lucene-solr/pull/1669#discussion_r454704926
##
File path: solr/core/src/java/org/apache/solr/schema/IndexSchema.java
##
@@ -188,6 +190,7 @@ public IndexSchema(String name, InputSource is, Version
luceneVersion, SolrResou
protected IndexSchema(Version luceneVersion, SolrResourceLoader loader,
Properties substitutableProperties) {
this.luceneVersion = Objects.requireNonNull(luceneVersion);
this.loader = loader;
+this.solrClassLoader = loader.getCore() == null? loader:
loader.getCore().getSchemaPluginsLoader();
Review comment:
> What if SchemaPluginsLoader was an SRL itself, and delegated the
resource-loading methods to the "real" SRL?
Well, technically it's possible. The current SRL is a mess. At some point in
the future we may end up making it clean and usable. Today it's not. We should
clearly differentiate between places where we need to load resources and places
where we need to load classes. A Minimal interface should be enough for loading
classes. SRL is a heavy concrete class.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Reopened] (SOLR-12847) Cut over implementation of maxShardsPerNode to a collection policy
[ https://issues.apache.org/jira/browse/SOLR-12847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul reopened SOLR-12847: --- > Cut over implementation of maxShardsPerNode to a collection policy > -- > > Key: SOLR-12847 > URL: https://issues.apache.org/jira/browse/SOLR-12847 > Project: Solr > Issue Type: Bug > Components: AutoScaling, SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Andrzej Bialecki >Priority: Major > Fix For: master (9.0) > > Time Spent: 40m > Remaining Estimate: 0h > > We've back and forth over handling maxShardsPerNode with autoscaling policies > (see SOLR-11005 for history). Now that we've reimplemented support for > creating collections with maxShardsPerNode when autoscaling policy is > enabled, we should re-look at how it is implemented. > I propose that we fold maxShardsPerNode (if specified) to a collection level > policy that overrides the corresponding default in cluster policy (see > SOLR-12845). We'll need to ensure that if maxShardsPerNode is specified then > the user sees neither violations nor corresponding suggestions because of the > default cluster policy. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12847) Cut over implementation of maxShardsPerNode to a collection policy
[ https://issues.apache.org/jira/browse/SOLR-12847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157771#comment-17157771 ] Noble Paul commented on SOLR-12847: --- Until we have a cleaner interface for replica placement we should avoid using it. If maxShardsPerNode is broken please remove it. Do not use the current placement startegies. > Cut over implementation of maxShardsPerNode to a collection policy > -- > > Key: SOLR-12847 > URL: https://issues.apache.org/jira/browse/SOLR-12847 > Project: Solr > Issue Type: Bug > Components: AutoScaling, SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Andrzej Bialecki >Priority: Major > Fix For: master (9.0) > > Time Spent: 40m > Remaining Estimate: 0h > > We've back and forth over handling maxShardsPerNode with autoscaling policies > (see SOLR-11005 for history). Now that we've reimplemented support for > creating collections with maxShardsPerNode when autoscaling policy is > enabled, we should re-look at how it is implemented. > I propose that we fold maxShardsPerNode (if specified) to a collection level > policy that overrides the corresponding default in cluster policy (see > SOLR-12845). We'll need to ensure that if maxShardsPerNode is specified then > the user sees neither violations nor corresponding suggestions because of the > default cluster policy. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14648) Creating TLOG with pure multiple PULL replica, leading to 0 doc count
[ https://issues.apache.org/jira/browse/SOLR-14648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157773#comment-17157773 ] Tomas Eduardo Fernandez Lobbe commented on SOLR-14648: -- bq. When a TLOG replica comes up and sees there are no other TLOG/NRT replicas to replicate from, either +1. I think #1 is very important (don't nuke a running cluster), #2 is desired (it'll help users get unstuck once they fell into #1) bq. H, or would the ability to "promote" a replica from PULL to TLOG work? No other "replica type change" is currently supported. And still, this would be an operation that causes data loss. We should make sure it's not used under regular circumstances, etc. I think this would be tricky. > Creating TLOG with pure multiple PULL replica, leading to 0 doc count > - > > Key: SOLR-14648 > URL: https://issues.apache.org/jira/browse/SOLR-14648 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 8.3.1 >Reporter: Sayan Das >Priority: Major > > With only PULL replica whenever we create a new TLOG as leader fresh > replication happens, resulting in flushing the older indexes from existing > PULL replicas > Steps to replicate: > # Create 1 NRT or 1 TLOG replica as leader with multiple PULL replicas > # Index few documents and let it replicate in all the replicas > # Delete all the TLOG/NRT replica leaving PULL types > # Create a new TLOG/NRT as leader, once recovery completes it replaces all > the older indexes > In ideal scenario it should have replicated from any one of the PULL replicas > that has latest indexes after that TLOG/NRT replica should be registered as > leader -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-8146) Allowing SolrJ CloudSolrClient to have preferred replica for query/read
[
https://issues.apache.org/jira/browse/SOLR-8146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomas Eduardo Fernandez Lobbe resolved SOLR-8146.
-
Resolution: Duplicate
Yes, lets close it as duplicate of SOLR-12217
> Allowing SolrJ CloudSolrClient to have preferred replica for query/read
> ---
>
> Key: SOLR-8146
> URL: https://issues.apache.org/jira/browse/SOLR-8146
> Project: Solr
> Issue Type: New Feature
> Components: clients - java
>Affects Versions: 5.3
>Reporter: Arcadius Ahouansou
>Priority: Major
> Attachments: SOLR-8146.patch, SOLR-8146.patch, SOLR-8146.patch,
> SOLR-8146.patch
>
>
> h2. Backgrouds
> Currently, the CloudSolrClient randomly picks a replica to query.
> This is done by shuffling the list of live URLs to query then, picking the
> first item from the list.
> This ticket is to allow more flexibility and control to some extend which
> URLs will be picked up for queries.
> Note that this is for queries only and would not affect update/delete/admin
> operations.
> h2. Implementation
> The current patch uses regex pattern and moves to the top of the list of URLs
> only those matching the given regex specified by the system property
> {code}solr.preferredQueryNodePattern{code}
> Initially, I thought it may be good to have Solr nodes tagged with a string
> pattern (snitch?) and use that pattern for matching the URLs.
> Any comment, recommendation or feedback would be appreciated.
> h2. Use Cases
> There are many cases where the ability to choose the node where queries go
> can be very handy:
> h3. Special node for manual user queries and analytics
> One may have a SolrCLoud cluster where every node host the same set of
> collections with:
> - multiple large SolrCLoud nodes (L) used for production apps and
> - have 1 small node (S) in the same cluster with less ram/cpu used only for
> manual user queries, data export and other production issue investigation.
> This ticket would allow to configure the applications using SolrJ to query
> only the (L) nodes
> This use case is similar to the one described in SOLR-5501 raised by [~manuel
> lenormand]
> h3. Minimizing network traffic
>
> For simplicity, let's say that we have a SolrSloud cluster deployed on 2 (or
> N) separate racks: rack1 and rack2.
> On each rack, we have a set of SolrCloud VMs as well as a couple of client
> VMs querying solr using SolrJ.
> All solr nodes are identical and have the same number of collections.
> What we would like to achieve is:
> - clients on rack1 will by preference query only SolrCloud nodes on rack1,
> and
> - clients on rack2 will by preference query only SolrCloud nodes on rack2.
> - Cross-rack read will happen if and only if one of the racks has no
> available Solr node to serve a request.
> In other words, we want read operations to be local to a rack whenever
> possible.
> Note that write/update/delete/admin operations should not be affected.
> Note that in our use case, we have a cross DC deployment. So, replace
> rack1/rack2 by DC1/DC2
> Any comment would be very appreciated.
> Thanks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-5501) Ability to work with cold replicas
[ https://issues.apache.org/jira/browse/SOLR-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomas Eduardo Fernandez Lobbe resolved SOLR-5501. - Fix Version/s: (was: 6.0) (was: 4.9) 7.4 8.0 Resolution: Duplicate > Ability to work with cold replicas > -- > > Key: SOLR-5501 > URL: https://issues.apache.org/jira/browse/SOLR-5501 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 4.5.1 >Reporter: Manuel Lenormand >Priority: Major > Labels: performance > Fix For: 8.0, 7.4 > > Attachments: 5501.patch, cloud_screenshot.png > > > Following this conversation from the mailing list: > http://lucene.472066.n3.nabble.com/Proposal-for-new-feature-cold-replicas-brainstorming-td4097501.html > Should give the ability to use replicas mainly as backup cores and not for > handling high qps rate. > This way you would avoid using the caching ressources (solr and OS) used when > routing a query to a replica. > With many replicas it's harder hitting the solr cache (same query may hit > another replica) and having many replicas on the same instance would cause a > useless competition on the OS memory for caching. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-8146) Allowing SolrJ CloudSolrClient to have preferred replica for query/read
[
https://issues.apache.org/jira/browse/SOLR-8146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomas Eduardo Fernandez Lobbe updated SOLR-8146:
Fix Version/s: 7.4
8.0
> Allowing SolrJ CloudSolrClient to have preferred replica for query/read
> ---
>
> Key: SOLR-8146
> URL: https://issues.apache.org/jira/browse/SOLR-8146
> Project: Solr
> Issue Type: New Feature
> Components: clients - java
>Affects Versions: 5.3
>Reporter: Arcadius Ahouansou
>Priority: Major
> Fix For: 7.4, 8.0
>
> Attachments: SOLR-8146.patch, SOLR-8146.patch, SOLR-8146.patch,
> SOLR-8146.patch
>
>
> h2. Backgrouds
> Currently, the CloudSolrClient randomly picks a replica to query.
> This is done by shuffling the list of live URLs to query then, picking the
> first item from the list.
> This ticket is to allow more flexibility and control to some extend which
> URLs will be picked up for queries.
> Note that this is for queries only and would not affect update/delete/admin
> operations.
> h2. Implementation
> The current patch uses regex pattern and moves to the top of the list of URLs
> only those matching the given regex specified by the system property
> {code}solr.preferredQueryNodePattern{code}
> Initially, I thought it may be good to have Solr nodes tagged with a string
> pattern (snitch?) and use that pattern for matching the URLs.
> Any comment, recommendation or feedback would be appreciated.
> h2. Use Cases
> There are many cases where the ability to choose the node where queries go
> can be very handy:
> h3. Special node for manual user queries and analytics
> One may have a SolrCLoud cluster where every node host the same set of
> collections with:
> - multiple large SolrCLoud nodes (L) used for production apps and
> - have 1 small node (S) in the same cluster with less ram/cpu used only for
> manual user queries, data export and other production issue investigation.
> This ticket would allow to configure the applications using SolrJ to query
> only the (L) nodes
> This use case is similar to the one described in SOLR-5501 raised by [~manuel
> lenormand]
> h3. Minimizing network traffic
>
> For simplicity, let's say that we have a SolrSloud cluster deployed on 2 (or
> N) separate racks: rack1 and rack2.
> On each rack, we have a set of SolrCloud VMs as well as a couple of client
> VMs querying solr using SolrJ.
> All solr nodes are identical and have the same number of collections.
> What we would like to achieve is:
> - clients on rack1 will by preference query only SolrCloud nodes on rack1,
> and
> - clients on rack2 will by preference query only SolrCloud nodes on rack2.
> - Cross-rack read will happen if and only if one of the racks has no
> available Solr node to serve a request.
> In other words, we want read operations to be local to a rack whenever
> possible.
> Note that write/update/delete/admin operations should not be affected.
> Note that in our use case, we have a cross DC deployment. So, replace
> rack1/rack2 by DC1/DC2
> Any comment would be very appreciated.
> Thanks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe closed pull request #147: SOLR-8146 decouple building url list from CloudSolrClient to separate class for…
tflobbe closed pull request #147: URL: https://github.com/apache/lucene-solr/pull/147 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe commented on pull request #147: SOLR-8146 decouple building url list from CloudSolrClient to separate class for…
tflobbe commented on pull request #147: URL: https://github.com/apache/lucene-solr/pull/147#issuecomment-658484617 Thanks for putting up this PR @susheelks. While this PR wasn't merged, the functionality was added by some other commits This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1669: SOLR-14151 Make schema components load from packages
dsmiley commented on a change in pull request #1669: URL: https://github.com/apache/lucene-solr/pull/1669#discussion_r454756640 ## File path: solr/core/src/java/org/apache/solr/core/SolrCore.java ## @@ -191,6 +191,11 @@ private String name; private String logid; // used to show what name is set + /** + * A unique id to differentiate multiple instances of the same core + * If we reload a core, the name remains same , but the id will be new + */ + public final UUID uniqueId = UUID.randomUUID(); Review comment: The primitive long instance field would be final (immutable). A static final AtomicLong (which is mutable) would only be used to generate a new core UID. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1669: SOLR-14151 Make schema components load from packages
dsmiley commented on a change in pull request #1669:
URL: https://github.com/apache/lucene-solr/pull/1669#discussion_r454763137
##
File path: solr/core/src/java/org/apache/solr/schema/IndexSchema.java
##
@@ -188,6 +190,7 @@ public IndexSchema(String name, InputSource is, Version
luceneVersion, SolrResou
protected IndexSchema(Version luceneVersion, SolrResourceLoader loader,
Properties substitutableProperties) {
this.luceneVersion = Objects.requireNonNull(luceneVersion);
this.loader = loader;
+this.solrClassLoader = loader.getCore() == null? loader:
loader.getCore().getSchemaPluginsLoader();
Review comment:
In the design I propose here, all the debt I mentioned in SRL except
instancePath/getConfigDir could move to the new SolrClassLoader because they
are class-loading related and not resource-loading related.
InstancePath/getConfigDir is debt I could erase quickly. Then I think we're in
good shape to move forward with a debt-free SRL that is a simple combination of
it's two components. If that sounds nice to you, I could work on a POC
immediately.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1669: SOLR-14151 Make schema components load from packages
dsmiley commented on a change in pull request #1669:
URL: https://github.com/apache/lucene-solr/pull/1669#discussion_r454765026
##
File path: solr/core/src/java/org/apache/solr/core/CoreContainer.java
##
@@ -1588,20 +1590,32 @@ private CoreDescriptor
reloadCoreDescriptor(CoreDescriptor oldDesc) {
return ret;
}
+ /**
+ * reloads a core
+ * refer {@link CoreContainer#reload(String, UUID)} for details
+ */
+ public void reload(String name) {
+reload(name, null);
+ }
/**
* Recreates a SolrCore.
* While the new core is loading, requests will continue to be dispatched to
* and processed by the old core
*
* @param name the name of the SolrCore to reload
+ * @param coreId The unique Id of the core
Review comment:
plus mention it's optional, and then if it's specified and the coreId
doesn't match, then the reload is a no-op. Maybe rename coreId to
ifCoreIdMatches to reflect not only what it is but how it's used.
##
File path: solr/core/src/java/org/apache/solr/core/ConfigSet.java
##
@@ -30,15 +32,18 @@
private final SolrConfig solrconfig;
- private final IndexSchema indexSchema;
+ /**Provide a Schema object on demand
+ * The first Boolean is to signify a a forcefetch
+ */
+ private final Function indexSchema;
Review comment:
I think using the generic interface Function here is pushing the bounds
of readability/intuitiveness. Lets just provide an interface/class instead.
Perhaps get() and getForce() with no boolean arg needed.
##
File path: solr/core/src/java/org/apache/solr/pkg/MultiPackageListener.java
##
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.pkg;
+
+import org.apache.lucene.analysis.util.ResourceLoaderAware;
+import org.apache.solr.common.MapWriter;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.core.PluginInfo;
+import org.apache.solr.core.SolrClassLoader;
+import org.apache.solr.core.SolrResourceLoader;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.function.Function;
+/**
+ * A {@link SolrClassLoader} that is designed to listen to a set of packages.
+ * This class would register a listener each package that is loaded through
this
+ * if any of those packages are updated , the onReload runnable is executed
+ * */
+public class MultiPackageListener implements SolrClassLoader ,
PackageListeners.Listener {
Review comment:
I find this confusing... it could just be the name. Something that
implements SolrClassLoader should probably have a name that makes that aspect
more pronounced and less so the listener aspect.
##
File path: solr/core/src/java/org/apache/solr/handler/SchemaHandler.java
##
@@ -202,6 +202,38 @@ private void handleGET(SolrQueryRequest req,
SolrQueryResponse rsp) {
}
}
+ /**If a plugin is loaded from a package, the version of the package being
used should be added
Review comment:
nit: another example of oddly formatted javadoc.
##
File path:
solr/core/src/java/org/apache/solr/util/plugin/AbstractPluginLoader.java
##
@@ -86,7 +86,7 @@ public AbstractPluginLoader(String type, Class
pluginClassType)
* @param node - the XML node defining this plugin
*/
@SuppressWarnings("unchecked")
- protected T create( SolrResourceLoader loader, String name, String
className, Node node ) throws Exception
+ protected T create(SolrClassLoader loader, String name, String className,
Node node ) throws Exception
Review comment:
BTW I do like these sorts of changes where you've found SRL used where
the caller really just needs a class loader.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscrib
[GitHub] [lucene-solr] noblepaul commented on pull request #1666: SOLR-14155: Load all other SolrCore plugins from packages
noblepaul commented on pull request #1666: URL: https://github.com/apache/lucene-solr/pull/1666#issuecomment-658539949 The method for loading package specific classes is moved to SRL This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9428) merge index failed with checksum failed (hardware problem?)
AllenL created LUCENE-9428:
--
Summary: merge index failed with checksum failed (hardware
problem?)
Key: LUCENE-9428
URL: https://issues.apache.org/jira/browse/LUCENE-9428
Project: Lucene - Core
Issue Type: Bug
Environment: lucene version:5.5.4
jdk version :jdk1.8-1.8.0_231-fcs
Reporter: AllenL
Recently, a procedure using ElasticSearch appeared merge Index Failed with the
following exception information
{code:java}
[2020-07-03 13:37:34,113][ERROR][index.engine ] [Deathbird]
[st-sess][4] failed to merge[2020-07-03 13:37:34,113][ERROR][index.engine
] [Deathbird] [st-sess][4] failed to
mergeorg.apache.lucene.index.CorruptIndexException: checksum failed (hardware
problem?) : expected=31f090d9 actual=d9697caa
(resource=BufferedChecksumIndexInput(MMapIndexInput(path="/var/lib/elasticsearch/17412c54-f974-11e9-9eef-80615f029e06/nodes/0/indices/st-sess/4/index/_3jm_Lucene50_0.tim")))
at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:334) at
org.apache.lucene.codecs.CodecUtil.checksumEntireFile(CodecUtil.java:451) at
org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.checkIntegrity(BlockTreeTermsReader.java:333)
at
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.checkIntegrity(PerFieldPostingsFormat.java:317)
at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:96) at
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193) at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95) at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4086) at
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3666) at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
at
org.elasticsearch.index.engine.ElasticsearchConcurrentMergeScheduler.doMerge(ElasticsearchConcurrentMergeScheduler.java:94)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)[2020-07-03
13:37:34,203][WARN ][index.engine ] [Deathbird] [st-sess][4]
failed engine [merge failed]org.apache.lucene.index.MergePolicy$MergeException:
org.apache.lucene.index.CorruptIndexException: checksum failed (hardware
problem?) : expected=31f090d9 actual=d9697caa
(resource=BufferedChecksumIndexInput(MMapIndexInput(path="/var/lib/elasticsearch/17412c54-f974-11e9-9eef-80615f029e06/nodes/0/indices/st-sess/4/index/_3jm_Lucene50_0.tim")))
at
org.elasticsearch.index.engine.InternalEngine$EngineMergeScheduler$1.doRun(InternalEngine.java:1237)
at
org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){code}
The exception shows that it may be a hardware problem. Try to check the
hardware and find no exception. Check the command as follows：
# check device /dev/sda, /dev/sdb; but finds no hardware errors
using command: smartctl --xall /dev/sdx
# check message log /var/log/messages, no hardware problem happend
# The system has a state detection script, i get the system load recorded is
normal, IOwait is very low
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9428) merge index failed with checksum failed (hardware problem?)
[
https://issues.apache.org/jira/browse/LUCENE-9428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
AllenL updated LUCENE-9428:
---
Description:
Recently, a procedure using ElasticSearch appeared merge Index Failed with the
following exception information
{code:java}
[2020-07-03 13:37:34,113][ERROR][index.engine ] [Deathbird]
[st-sess][4] failed to merge[2020-07-03 13:37:34,113][ERROR][index.engine
] [Deathbird] [st-sess][4] failed to
mergeorg.apache.lucene.index.CorruptIndexException: checksum failed (hardware
problem?) : expected=31f090d9 actual=d9697caa
(resource=BufferedChecksumIndexInput(MMapIndexInput(path="/var/lib/elasticsearch/17412c54-f974-11e9-9eef-80615f029e06/nodes/0/indices/st-sess/4/index/_3jm_Lucene50_0.tim")))
at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:334) at
org.apache.lucene.codecs.CodecUtil.checksumEntireFile(CodecUtil.java:451) at
org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.checkIntegrity(BlockTreeTermsReader.java:333)
at
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.checkIntegrity(PerFieldPostingsFormat.java:317)
at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:96) at
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193) at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95) at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4086) at
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3666) at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
at
org.elasticsearch.index.engine.ElasticsearchConcurrentMergeScheduler.doMerge(ElasticsearchConcurrentMergeScheduler.java:94)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)[2020-07-03
13:37:34,203][WARN ][index.engine ] [Deathbird] [st-sess][4]
failed engine [merge failed]org.apache.lucene.index.MergePolicy$MergeException:
org.apache.lucene.index.CorruptIndexException: checksum failed (hardware
problem?) : expected=31f090d9 actual=d9697caa
(resource=BufferedChecksumIndexInput(MMapIndexInput(path="/var/lib/elasticsearch/shterm-17412c54-f974-11e9-9eef-80615f029e06/nodes/0/indices/st-sess/4/index/_3jm_Lucene50_0.tim")))
at
org.elasticsearch.index.engine.InternalEngine$EngineMergeScheduler$1.doRun(InternalEngine.java:1237)
at
org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){code}
The exception shows that it may be a hardware problem. Try to check the
hardware and find no exception. Check the command as follows：
# check device /dev/sda, /dev/sdb; but finds no hardware errors
using command: smartctl --xall /dev/sdx
# check message log /var/log/messages, no hardware problem happend
# The system has a state detection script, i get the system load recorded is
normal, IOwait is very low
was:
Recently, a procedure using ElasticSearch appeared merge Index Failed with the
following exception information
{code:java}
[2020-07-03 13:37:34,113][ERROR][index.engine ] [Deathbird]
[st-sess][4] failed to merge[2020-07-03 13:37:34,113][ERROR][index.engine
] [Deathbird] [st-sess][4] failed to
mergeorg.apache.lucene.index.CorruptIndexException: checksum failed (hardware
problem?) : expected=31f090d9 actual=d9697caa
(resource=BufferedChecksumIndexInput(MMapIndexInput(path="/var/lib/elasticsearch/17412c54-f974-11e9-9eef-80615f029e06/nodes/0/indices/st-sess/4/index/_3jm_Lucene50_0.tim")))
at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:334) at
org.apache.lucene.codecs.CodecUtil.checksumEntireFile(CodecUtil.java:451) at
org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.checkIntegrity(BlockTreeTermsReader.java:333)
at
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.checkIntegrity(PerFieldPostingsFormat.java:317)
at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:96) at
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193) at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95) at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4086) at
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3666) at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
at
org.elasticsearch.index.engine.ElasticsearchConcurrentMergeScheduler.doMerge(ElasticsearchConcurrentMergeScheduler.java:94)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)[2020-07-03
13:37:34,203][WARN ][index.engine ] [Deathbird] [st-se
[jira] [Updated] (LUCENE-9428) merge index failed with checksum failed (hardware problem?)
[
https://issues.apache.org/jira/browse/LUCENE-9428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
AllenL updated LUCENE-9428:
---
Description:
Recently, a procedure using ElasticSearch appeared merge Index Failed with the
following exception information
{code:java}
[2020-07-03 13:37:34,113][ERROR][index.engine ] [Deathbird]
[st-sess][4] failed to merge
[2020-07-03 13:37:34,113][ERROR][index.engine ] [Deathbird]
[st-sess][4] failed to mergeorg.apache.lucene.index.CorruptIndexException:
checksum failed (hardware problem?) : expected=31f090d9 actual=d9697caa
(resource=BufferedChecksumIndexInput(MMapIndexInput(path="/var/lib/elasticsearch/17412c54-f974-11e9-9eef-80615f029e06/nodes/0/indices/st-sess/4/index/_3jm_Lucene50_0.tim")))
at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:334) at
org.apache.lucene.codecs.CodecUtil.checksumEntireFile(CodecUtil.java:451)
at
org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.checkIntegrity(BlockTreeTermsReader.java:333)
at
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.checkIntegrity(PerFieldPostingsFormat.java:317)
at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:96)
at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4086)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3666)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
at
org.elasticsearch.index.engine.ElasticsearchConcurrentMergeScheduler.doMerge(ElasticsearchConcurrentMergeScheduler.java:94)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
[2020-07-03 13:37:34,203][WARN ][index.engine ] [Deathbird]
[st-sess][4] failed engine [merge
failed]org.apache.lucene.index.MergePolicy$MergeException:
org.apache.lucene.index.CorruptIndexException: checksum failed (hardware
problem?) : expected=31f090d9 actual=d9697caa
(resource=BufferedChecksumIndexInput(MMapIndexInput(path="/var/lib/elasticsearch/shterm-17412c54-f974-11e9-9eef-80615f029e06/nodes/0/indices/st-sess/4/index/_3jm_Lucene50_0.tim")))
at
org.elasticsearch.index.engine.InternalEngine$EngineMergeScheduler$1.doRun(InternalEngine.java:1237)
at
org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){code}
The exception shows that it may be a hardware problem. Try to check the
hardware and find no exception. Check the command as follows：
# check device /dev/sda, /dev/sdb; but finds no hardware errors
using command: smartctl --xall /dev/sdx
# check message log /var/log/messages, no hardware problem happend
# The system has a state detection script, i get the system load recorded is
normal, IOwait is very low
was:
Recently, a procedure using ElasticSearch appeared merge Index Failed with the
following exception information
{code:java}
[2020-07-03 13:37:34,113][ERROR][index.engine ] [Deathbird]
[st-sess][4] failed to merge[2020-07-03 13:37:34,113][ERROR][index.engine
] [Deathbird] [st-sess][4] failed to
mergeorg.apache.lucene.index.CorruptIndexException: checksum failed (hardware
problem?) : expected=31f090d9 actual=d9697caa
(resource=BufferedChecksumIndexInput(MMapIndexInput(path="/var/lib/elasticsearch/17412c54-f974-11e9-9eef-80615f029e06/nodes/0/indices/st-sess/4/index/_3jm_Lucene50_0.tim")))
at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:334) at
org.apache.lucene.codecs.CodecUtil.checksumEntireFile(CodecUtil.java:451) at
org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.checkIntegrity(BlockTreeTermsReader.java:333)
at
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.checkIntegrity(PerFieldPostingsFormat.java:317)
at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:96) at
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193) at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95) at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4086) at
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3666) at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
at
org.elasticsearch.index.engine.ElasticsearchConcurrentMergeScheduler.doMerge(ElasticsearchConcurrentMergeScheduler.java:94)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)[2020-07-03
13:37:34,203][WARN ][index.engine ] [Deathb
[jira] [Commented] (LUCENE-9428) merge index failed with checksum failed (hardware problem?)
[
https://issues.apache.org/jira/browse/LUCENE-9428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157942#comment-17157942
]
AllenL commented on LUCENE-9428:
When will this exception occur
> merge index failed with checksum failed (hardware problem?)
> ---
>
> Key: LUCENE-9428
> URL: https://issues.apache.org/jira/browse/LUCENE-9428
> Project: Lucene - Core
> Issue Type: Bug
> Environment: lucene version:5.5.4
> jdk version :jdk1.8-1.8.0_231-fcs
>Reporter: AllenL
>Priority: Major
>
> Recently, a procedure using ElasticSearch appeared merge Index Failed with
> the following exception information
>
> {code:java}
> [2020-07-03 13:37:34,113][ERROR][index.engine ] [Deathbird]
> [st-sess][4] failed to merge
> [2020-07-03 13:37:34,113][ERROR][index.engine ] [Deathbird]
> [st-sess][4] failed to mergeorg.apache.lucene.index.CorruptIndexException:
> checksum failed (hardware problem?) : expected=31f090d9 actual=d9697caa
> (resource=BufferedChecksumIndexInput(MMapIndexInput(path="/var/lib/elasticsearch/17412c54-f974-11e9-9eef-80615f029e06/nodes/0/indices/st-sess/4/index/_3jm_Lucene50_0.tim")))
>
> at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:334) at
> org.apache.lucene.codecs.CodecUtil.checksumEntireFile(CodecUtil.java:451)
> at
> org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.checkIntegrity(BlockTreeTermsReader.java:333)
>
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.checkIntegrity(PerFieldPostingsFormat.java:317)
>
> at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:96)
> at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4086)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3666)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
>
> at
> org.elasticsearch.index.engine.ElasticsearchConcurrentMergeScheduler.doMerge(ElasticsearchConcurrentMergeScheduler.java:94)
>
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
> [2020-07-03 13:37:34,203][WARN ][index.engine ] [Deathbird]
> [st-sess][4] failed engine [merge
> failed]org.apache.lucene.index.MergePolicy$MergeException:
> org.apache.lucene.index.CorruptIndexException: checksum failed (hardware
> problem?) : expected=31f090d9 actual=d9697caa
> (resource=BufferedChecksumIndexInput(MMapIndexInput(path="/var/lib/elasticsearch/shterm-17412c54-f974-11e9-9eef-80615f029e06/nodes/0/indices/st-sess/4/index/_3jm_Lucene50_0.tim")))
>
> at
> org.elasticsearch.index.engine.InternalEngine$EngineMergeScheduler$1.doRun(InternalEngine.java:1237)
>
> at
> org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>
> at java.lang.Thread.run(Thread.java:748){code}
>
> The exception shows that it may be a hardware problem. Try to check the
> hardware and find no exception. Check the command as follows：
> # check device /dev/sda, /dev/sdb; but finds no hardware errors
> using command: smartctl --xall /dev/sdx
> # check message log /var/log/messages, no hardware problem happend
> # The system has a state detection script, i get the system load recorded is
> normal, IOwait is very low
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14579) Remove SolrJ 'Utils' generic map functions
[ https://issues.apache.org/jira/browse/SOLR-14579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul updated SOLR-14579: -- Fix Version/s: master (9.0) > Remove SolrJ 'Utils' generic map functions > -- > > Key: SOLR-14579 > URL: https://issues.apache.org/jira/browse/SOLR-14579 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: master (9.0) >Reporter: Megan Carey >Assignee: Noble Paul >Priority: Minor > Fix For: master (9.0) > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Remove the map functions like `NEW_HASHMAP_FUN` from the Utils class in solrj > module to reduce warnings and improve code quality. > [https://github.com/apache/lucene-solr/blob/master/solr/solrj/src/java/org/apache/solr/common/util/Utils.java#L92] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9428) merge index failed with checksum failed (hardware problem?)
[
https://issues.apache.org/jira/browse/LUCENE-9428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrien Grand resolved LUCENE-9428.
--
Resolution: Invalid
When seeing this error message, chances that this is due to a bug in Lucene are
very small (which is exactly why we added checksums to index files, so that we
could distinguish bugs in Lucene from hardware issues or bugs beneath Lucene:
JDK, kernel). Even if your disks don't report an error, there is always a
possibility of a silent corruption. I would suggest checking your RAM and
making sure that you are on an up-to-date kernel and JDK too.
> merge index failed with checksum failed (hardware problem?)
> ---
>
> Key: LUCENE-9428
> URL: https://issues.apache.org/jira/browse/LUCENE-9428
> Project: Lucene - Core
> Issue Type: Bug
> Environment: lucene version:5.5.4
> jdk version :jdk1.8-1.8.0_231-fcs
>Reporter: AllenL
>Priority: Major
>
> Recently, a procedure using ElasticSearch appeared merge Index Failed with
> the following exception information
>
> {code:java}
> [2020-07-03 13:37:34,113][ERROR][index.engine ] [Deathbird]
> [st-sess][4] failed to merge
> [2020-07-03 13:37:34,113][ERROR][index.engine ] [Deathbird]
> [st-sess][4] failed to mergeorg.apache.lucene.index.CorruptIndexException:
> checksum failed (hardware problem?) : expected=31f090d9 actual=d9697caa
> (resource=BufferedChecksumIndexInput(MMapIndexInput(path="/var/lib/elasticsearch/17412c54-f974-11e9-9eef-80615f029e06/nodes/0/indices/st-sess/4/index/_3jm_Lucene50_0.tim")))
>
> at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:334) at
> org.apache.lucene.codecs.CodecUtil.checksumEntireFile(CodecUtil.java:451)
> at
> org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.checkIntegrity(BlockTreeTermsReader.java:333)
>
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.checkIntegrity(PerFieldPostingsFormat.java:317)
>
> at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:96)
> at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4086)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3666)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
>
> at
> org.elasticsearch.index.engine.ElasticsearchConcurrentMergeScheduler.doMerge(ElasticsearchConcurrentMergeScheduler.java:94)
>
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
> [2020-07-03 13:37:34,203][WARN ][index.engine ] [Deathbird]
> [st-sess][4] failed engine [merge
> failed]org.apache.lucene.index.MergePolicy$MergeException:
> org.apache.lucene.index.CorruptIndexException: checksum failed (hardware
> problem?) : expected=31f090d9 actual=d9697caa
> (resource=BufferedChecksumIndexInput(MMapIndexInput(path="/var/lib/elasticsearch/shterm-17412c54-f974-11e9-9eef-80615f029e06/nodes/0/indices/st-sess/4/index/_3jm_Lucene50_0.tim")))
>
> at
> org.elasticsearch.index.engine.InternalEngine$EngineMergeScheduler$1.doRun(InternalEngine.java:1237)
>
> at
> org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>
> at java.lang.Thread.run(Thread.java:748){code}
>
> The exception shows that it may be a hardware problem. Try to check the
> hardware and find no exception. Check the command as follows：
> # check device /dev/sda, /dev/sdb; but finds no hardware errors
> using command: smartctl --xall /dev/sdx
> # check message log /var/log/messages, no hardware problem happend
> # The system has a state detection script, i get the system load recorded is
> normal, IOwait is very low
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9428) merge index failed with checksum failed (hardware problem?)
[
https://issues.apache.org/jira/browse/LUCENE-9428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157948#comment-17157948
]
Adrien Grand commented on LUCENE-9428:
--
[~Lai_Ding] This exception occurs when the content of a file differs from the
data that had been written. Unfortunately this is something that is very
expensive to check, so Lucene doesn't keep verifying checksums continuously, it
only does it at merge time since files need to be fully read anyway.
> merge index failed with checksum failed (hardware problem?)
> ---
>
> Key: LUCENE-9428
> URL: https://issues.apache.org/jira/browse/LUCENE-9428
> Project: Lucene - Core
> Issue Type: Bug
> Environment: lucene version:5.5.4
> jdk version :jdk1.8-1.8.0_231-fcs
>Reporter: AllenL
>Priority: Major
>
> Recently, a procedure using ElasticSearch appeared merge Index Failed with
> the following exception information
>
> {code:java}
> [2020-07-03 13:37:34,113][ERROR][index.engine ] [Deathbird]
> [st-sess][4] failed to merge
> [2020-07-03 13:37:34,113][ERROR][index.engine ] [Deathbird]
> [st-sess][4] failed to mergeorg.apache.lucene.index.CorruptIndexException:
> checksum failed (hardware problem?) : expected=31f090d9 actual=d9697caa
> (resource=BufferedChecksumIndexInput(MMapIndexInput(path="/var/lib/elasticsearch/17412c54-f974-11e9-9eef-80615f029e06/nodes/0/indices/st-sess/4/index/_3jm_Lucene50_0.tim")))
>
> at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:334) at
> org.apache.lucene.codecs.CodecUtil.checksumEntireFile(CodecUtil.java:451)
> at
> org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.checkIntegrity(BlockTreeTermsReader.java:333)
>
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.checkIntegrity(PerFieldPostingsFormat.java:317)
>
> at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:96)
> at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4086)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3666)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
>
> at
> org.elasticsearch.index.engine.ElasticsearchConcurrentMergeScheduler.doMerge(ElasticsearchConcurrentMergeScheduler.java:94)
>
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
> [2020-07-03 13:37:34,203][WARN ][index.engine ] [Deathbird]
> [st-sess][4] failed engine [merge
> failed]org.apache.lucene.index.MergePolicy$MergeException:
> org.apache.lucene.index.CorruptIndexException: checksum failed (hardware
> problem?) : expected=31f090d9 actual=d9697caa
> (resource=BufferedChecksumIndexInput(MMapIndexInput(path="/var/lib/elasticsearch/shterm-17412c54-f974-11e9-9eef-80615f029e06/nodes/0/indices/st-sess/4/index/_3jm_Lucene50_0.tim")))
>
> at
> org.elasticsearch.index.engine.InternalEngine$EngineMergeScheduler$1.doRun(InternalEngine.java:1237)
>
> at
> org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>
> at java.lang.Thread.run(Thread.java:748){code}
>
> The exception shows that it may be a hardware problem. Try to check the
> hardware and find no exception. Check the command as follows：
> # check device /dev/sda, /dev/sdb; but finds no hardware errors
> using command: smartctl --xall /dev/sdx
> # check message log /var/log/messages, no hardware problem happend
> # The system has a state detection script, i get the system load recorded is
> normal, IOwait is very low
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
