[jira] [Commented] (SOLR-14463) solr admin ui: zkstatus: For input string: "null" with zk 3.6.x

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105243#comment-17105243
 ] 

ASF subversion and git services commented on SOLR-14463:


Commit 6971244134b57e82e922b3bcbfb24b3462bace20 in lucene-solr's branch 
refs/heads/master from Jan Høydahl
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6971244 ]

SOLR-14463: Solr Admin ZkStatus page now works with ZK 3.6 (#1499)



> solr admin ui: zkstatus: For input string: "null" with zk 3.6.x
> ---
>
> Key: SOLR-14463
> URL: https://issues.apache.org/jira/browse/SOLR-14463
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 8.5.1
>Reporter: Bernd Wahlen
>Assignee: Jan Høydahl
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When i select in solr 8.5.1 webinterface: Cloud - ZK Status i got
> For input string: "null"
> This happens after upgrading the leader to zookeeper 3.6.1 (upgrading the 
> followers before was not a problem). From my view configuration is running, 
> just the status web page doesn't.
> I remember that there was similar problem before with solr/zk version 
> incompatibilities. From zookeeper documentation 3.6.1 should be fully 
> compatible with 3.5 clients.
> Annoyingly there is no official zk 3.6.1 docker container available, but i 
> think it is easy to reproduce.
> and from solr.log file:
> {code:java}
> 2020-05-07 15:58:33.194 ERROR (qtp1940055334-231) [   ] 
> o.a.s.h.RequestHandlerBase java.lang.NumberFormatException: For input string: 
> "null"
> at 
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Integer.parseInt(Integer.java:652)
> at java.base/java.lang.Integer.parseInt(Integer.java:770)
> at 
> org.apache.solr.handler.admin.ZookeeperStatusHandler.getZkStatus(ZookeeperStatusHandler.java:116)
> at 
> org.apache.solr.handler.admin.ZookeeperStatusHandler.handleRequestBody(ZookeeperStatusHandler.java:78)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
> at 
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)
> at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1596)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)
> at 
> org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177)
> at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at 
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at org.eclipse.jetty.server.Server.handle(Server.j

[jira] [Commented] (SOLR-14463) solr admin ui: zkstatus: For input string: "null" with zk 3.6.x

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105246#comment-17105246
 ] 

ASF subversion and git services commented on SOLR-14463:


Commit 4aa6145b9ae69a656312ec1914e4f0f93b5d8c2e in lucene-solr's branch 
refs/heads/branch_8x from Jan Høydahl
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4aa6145 ]

SOLR-14463: Solr Admin ZkStatus page now works with ZK 3.6 (#1499)

(cherry picked from commit 6971244134b57e82e922b3bcbfb24b3462bace20)


> solr admin ui: zkstatus: For input string: "null" with zk 3.6.x
> ---
>
> Key: SOLR-14463
> URL: https://issues.apache.org/jira/browse/SOLR-14463
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 8.5.1
>Reporter: Bernd Wahlen
>Assignee: Jan Høydahl
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When i select in solr 8.5.1 webinterface: Cloud - ZK Status i got
> For input string: "null"
> This happens after upgrading the leader to zookeeper 3.6.1 (upgrading the 
> followers before was not a problem). From my view configuration is running, 
> just the status web page doesn't.
> I remember that there was similar problem before with solr/zk version 
> incompatibilities. From zookeeper documentation 3.6.1 should be fully 
> compatible with 3.5 clients.
> Annoyingly there is no official zk 3.6.1 docker container available, but i 
> think it is easy to reproduce.
> and from solr.log file:
> {code:java}
> 2020-05-07 15:58:33.194 ERROR (qtp1940055334-231) [   ] 
> o.a.s.h.RequestHandlerBase java.lang.NumberFormatException: For input string: 
> "null"
> at 
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Integer.parseInt(Integer.java:652)
> at java.base/java.lang.Integer.parseInt(Integer.java:770)
> at 
> org.apache.solr.handler.admin.ZookeeperStatusHandler.getZkStatus(ZookeeperStatusHandler.java:116)
> at 
> org.apache.solr.handler.admin.ZookeeperStatusHandler.handleRequestBody(ZookeeperStatusHandler.java:78)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
> at 
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)
> at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1596)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)
> at 
> org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177)
> at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at 
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrappe

[jira] [Resolved] (SOLR-14463) solr admin ui: zkstatus: For input string: "null" with zk 3.6.x

[Jira]

 [ 
https://issues.apache.org/jira/browse/SOLR-14463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl resolved SOLR-14463.

Resolution: Fixed

> solr admin ui: zkstatus: For input string: "null" with zk 3.6.x
> ---
>
> Key: SOLR-14463
> URL: https://issues.apache.org/jira/browse/SOLR-14463
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 8.5.1
>Reporter: Bernd Wahlen
>Assignee: Jan Høydahl
>Priority: Major
> Fix For: 8.6
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When i select in solr 8.5.1 webinterface: Cloud - ZK Status i got
> For input string: "null"
> This happens after upgrading the leader to zookeeper 3.6.1 (upgrading the 
> followers before was not a problem). From my view configuration is running, 
> just the status web page doesn't.
> I remember that there was similar problem before with solr/zk version 
> incompatibilities. From zookeeper documentation 3.6.1 should be fully 
> compatible with 3.5 clients.
> Annoyingly there is no official zk 3.6.1 docker container available, but i 
> think it is easy to reproduce.
> and from solr.log file:
> {code:java}
> 2020-05-07 15:58:33.194 ERROR (qtp1940055334-231) [   ] 
> o.a.s.h.RequestHandlerBase java.lang.NumberFormatException: For input string: 
> "null"
> at 
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Integer.parseInt(Integer.java:652)
> at java.base/java.lang.Integer.parseInt(Integer.java:770)
> at 
> org.apache.solr.handler.admin.ZookeeperStatusHandler.getZkStatus(ZookeeperStatusHandler.java:116)
> at 
> org.apache.solr.handler.admin.ZookeeperStatusHandler.handleRequestBody(ZookeeperStatusHandler.java:78)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
> at 
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)
> at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1596)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)
> at 
> org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177)
> at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at 
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at org.eclipse.jetty.server.Server.handle(Server.java:500)
> at 
> org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
> at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
> at 
> org.eclipse.jet

[jira] [Updated] (SOLR-14463) solr admin ui: zkstatus: For input string: "null" with zk 3.6.x

[Jira]

 [ 
https://issues.apache.org/jira/browse/SOLR-14463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated SOLR-14463:
---
Fix Version/s: 8.6

> solr admin ui: zkstatus: For input string: "null" with zk 3.6.x
> ---
>
> Key: SOLR-14463
> URL: https://issues.apache.org/jira/browse/SOLR-14463
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 8.5.1
>Reporter: Bernd Wahlen
>Assignee: Jan Høydahl
>Priority: Major
> Fix For: 8.6
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When i select in solr 8.5.1 webinterface: Cloud - ZK Status i got
> For input string: "null"
> This happens after upgrading the leader to zookeeper 3.6.1 (upgrading the 
> followers before was not a problem). From my view configuration is running, 
> just the status web page doesn't.
> I remember that there was similar problem before with solr/zk version 
> incompatibilities. From zookeeper documentation 3.6.1 should be fully 
> compatible with 3.5 clients.
> Annoyingly there is no official zk 3.6.1 docker container available, but i 
> think it is easy to reproduce.
> and from solr.log file:
> {code:java}
> 2020-05-07 15:58:33.194 ERROR (qtp1940055334-231) [   ] 
> o.a.s.h.RequestHandlerBase java.lang.NumberFormatException: For input string: 
> "null"
> at 
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Integer.parseInt(Integer.java:652)
> at java.base/java.lang.Integer.parseInt(Integer.java:770)
> at 
> org.apache.solr.handler.admin.ZookeeperStatusHandler.getZkStatus(ZookeeperStatusHandler.java:116)
> at 
> org.apache.solr.handler.admin.ZookeeperStatusHandler.handleRequestBody(ZookeeperStatusHandler.java:78)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
> at 
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)
> at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1596)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)
> at 
> org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177)
> at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at 
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at org.eclipse.jetty.server.Server.handle(Server.java:500)
> at 
> org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
> at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
> at 
> org.eclipse.jett

[jira] [Updated] (SOLR-14347) Autoscaling placement wrong when concurrent replica placements are calculated

[Andrzej Bialecki (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-14347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrzej Bialecki updated SOLR-14347:

Priority: Critical  (was: Major)

> Autoscaling placement wrong when concurrent replica placements are calculated
> -
>
> Key: SOLR-14347
> URL: https://issues.apache.org/jira/browse/SOLR-14347
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling
>Affects Versions: 8.5
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Critical
> Fix For: 8.6
>
> Attachments: SOLR-14347.patch
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Steps to reproduce:
>  * create a cluster of a few nodes (tested with 7 nodes)
>  * define per-collection policies that distribute replicas exclusively on 
> different nodes per policy
>  * concurrently create a few collections, each using a different policy
>  * resulting replica placement will be seriously wrong, causing many policy 
> violations
> Running the same scenario but instead creating collections sequentially 
> results in no violations.
> I suspect this is caused by incorrect locking level for all collection 
> operations (as defined in {{CollectionParams.CollectionAction}}) that create 
> new replica placements - i.e. CREATE, ADDREPLICA, MOVEREPLICA, DELETENODE, 
> REPLACENODE, SPLITSHARD, RESTORE, REINDEXCOLLECTION. All of these operations 
> use the policy engine to create new replica placements, and as a result they 
> change the cluster state. However, currently these operations are locked (in 
> {{OverseerCollectionMessageHandler.lockTask}} ) using 
> {{LockLevel.COLLECTION}}. In practice this means that the lock is held only 
> for the particular collection that is being modified.
> A straightforward fix for this issue is to change the locking level to 
> CLUSTER (and I confirm this fixes the scenario described above). However, 
> this effectively serializes all collection operations listed above, which 
> will result in general slow-down of all collection operations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Reopened] (SOLR-14347) Autoscaling placement wrong when concurrent replica placements are calculated

[Andrzej Bialecki (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-14347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrzej Bialecki reopened SOLR-14347:
-

> Autoscaling placement wrong when concurrent replica placements are calculated
> -
>
> Key: SOLR-14347
> URL: https://issues.apache.org/jira/browse/SOLR-14347
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling
>Affects Versions: 8.5
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
> Fix For: 8.6
>
> Attachments: SOLR-14347.patch
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Steps to reproduce:
>  * create a cluster of a few nodes (tested with 7 nodes)
>  * define per-collection policies that distribute replicas exclusively on 
> different nodes per policy
>  * concurrently create a few collections, each using a different policy
>  * resulting replica placement will be seriously wrong, causing many policy 
> violations
> Running the same scenario but instead creating collections sequentially 
> results in no violations.
> I suspect this is caused by incorrect locking level for all collection 
> operations (as defined in {{CollectionParams.CollectionAction}}) that create 
> new replica placements - i.e. CREATE, ADDREPLICA, MOVEREPLICA, DELETENODE, 
> REPLACENODE, SPLITSHARD, RESTORE, REINDEXCOLLECTION. All of these operations 
> use the policy engine to create new replica placements, and as a result they 
> change the cluster state. However, currently these operations are locked (in 
> {{OverseerCollectionMessageHandler.lockTask}} ) using 
> {{LockLevel.COLLECTION}}. In practice this means that the lock is held only 
> for the particular collection that is being modified.
> A straightforward fix for this issue is to change the locking level to 
> CLUSTER (and I confirm this fixes the scenario described above). However, 
> this effectively serializes all collection operations listed above, which 
> will result in general slow-down of all collection operations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14347) Autoscaling placement wrong when concurrent replica placements are calculated

[Andrzej Bialecki (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105256#comment-17105256
 ] 

Andrzej Bialecki commented on SOLR-14347:
-

Escalating to critical - this needs to be fixed before 8.6.

> Autoscaling placement wrong when concurrent replica placements are calculated
> -
>
> Key: SOLR-14347
> URL: https://issues.apache.org/jira/browse/SOLR-14347
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling
>Affects Versions: 8.5
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Critical
> Fix For: 8.6
>
> Attachments: SOLR-14347.patch
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Steps to reproduce:
>  * create a cluster of a few nodes (tested with 7 nodes)
>  * define per-collection policies that distribute replicas exclusively on 
> different nodes per policy
>  * concurrently create a few collections, each using a different policy
>  * resulting replica placement will be seriously wrong, causing many policy 
> violations
> Running the same scenario but instead creating collections sequentially 
> results in no violations.
> I suspect this is caused by incorrect locking level for all collection 
> operations (as defined in {{CollectionParams.CollectionAction}}) that create 
> new replica placements - i.e. CREATE, ADDREPLICA, MOVEREPLICA, DELETENODE, 
> REPLACENODE, SPLITSHARD, RESTORE, REINDEXCOLLECTION. All of these operations 
> use the policy engine to create new replica placements, and as a result they 
> change the cluster state. However, currently these operations are locked (in 
> {{OverseerCollectionMessageHandler.lockTask}} ) using 
> {{LockLevel.COLLECTION}}. In practice this means that the lock is held only 
> for the particular collection that is being modified.
> A straightforward fix for this issue is to change the locking level to 
> CLUSTER (and I confirm this fixes the scenario described above). However, 
> this effectively serializes all collection operations listed above, which 
> will result in general slow-down of all collection operations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14347) Autoscaling placement wrong when concurrent replica placements are calculated

[Andrzej Bialecki (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105255#comment-17105255
 ] 

Andrzej Bialecki commented on SOLR-14347:
-

Hmm, I think you are right ... I don't know how I missed this :( I focused on 
the fact that per-collection policies caused side-effects in 
{{Session.expandedClauses}} and missed the fact that we're dropping the 
previous state of the matrix that contains changes from previous ops, which are 
not yet persisted.

At this point I'm not sure how to fix this properly - the changes we're making 
here still need to be pushed back to the original Session, so a Session.copy() 
won't work either.

> Autoscaling placement wrong when concurrent replica placements are calculated
> -
>
> Key: SOLR-14347
> URL: https://issues.apache.org/jira/browse/SOLR-14347
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling
>Affects Versions: 8.5
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
> Fix For: 8.6
>
> Attachments: SOLR-14347.patch
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Steps to reproduce:
>  * create a cluster of a few nodes (tested with 7 nodes)
>  * define per-collection policies that distribute replicas exclusively on 
> different nodes per policy
>  * concurrently create a few collections, each using a different policy
>  * resulting replica placement will be seriously wrong, causing many policy 
> violations
> Running the same scenario but instead creating collections sequentially 
> results in no violations.
> I suspect this is caused by incorrect locking level for all collection 
> operations (as defined in {{CollectionParams.CollectionAction}}) that create 
> new replica placements - i.e. CREATE, ADDREPLICA, MOVEREPLICA, DELETENODE, 
> REPLACENODE, SPLITSHARD, RESTORE, REINDEXCOLLECTION. All of these operations 
> use the policy engine to create new replica placements, and as a result they 
> change the cluster state. However, currently these operations are locked (in 
> {{OverseerCollectionMessageHandler.lockTask}} ) using 
> {{LockLevel.COLLECTION}}. In practice this means that the lock is held only 
> for the particular collection that is being modified.
> A straightforward fix for this issue is to change the locking level to 
> CLUSTER (and I confirm this fixes the scenario described above). However, 
> this effectively serializes all collection operations listed above, which 
> will result in general slow-down of all collection operations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9369) Deprecate grouping by ValueSource

[Alan Woodward (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105293#comment-17105293
 ] 

Alan Woodward commented on LUCENE-9369:
---

I opened a PR here: https://github.com/apache/lucene-solr/pull/1513

There are some open questions on this:
1) Testing Solr's version of the ValueSourceGroupSelector - currently 
BaseGroupSelectorTestCase is in the grouping module's test class module, which 
is not visible to Solr tests.  We have two options here, firstly to export the 
grouping tests as a test jar and add this as a dependency to Solr; or 
alternatively, to move BaseGroupSelectorTestCase to the lucene-test-framework 
module, which will require adding a dependency on the grouping module.  I think 
either are fine?
2) This is removing functionality, in that you can no longer group by the 
string value of a ValueSource.  We don't actually have any VS implementations 
that produce arbitrary strings - we have constants (no use for grouping!), or 
we have docvalues (equivalent to the TermGroupSelector) - but that doesn't 
necessarily mean that somebody hasn't implemented one elsewhere.  If this looks 
like it will cause a problem, we could investigate adding a BinaryValuesSource, 
by analogy with Long/DoubleValuesSource, and adding a group selector that uses 
this.  I can see this would be useful for normalizing values after indexing, 
for example.

> Deprecate grouping by ValueSource
> -
>
> Key: LUCENE-9369
> URL: https://issues.apache.org/jira/browse/LUCENE-9369
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
>
> LUCENE-7889 allows grouping by DoubleValuesSource and LongValuesSource.  We 
> can now deprecate the ValueSourceGroupingSelector, and in 9x remove the 
> grouping module's dependency on the queries module.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13289) Support for BlockMax WAND

[David Smiley (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105399#comment-17105399
 ] 

David Smiley commented on SOLR-13289:
-

I suggest {{minExactFound}} instead because it has the word "Found" which is 
perhaps the most significant portion of "numFound".

> Support for BlockMax WAND
> -
>
> Key: SOLR-13289
> URL: https://issues.apache.org/jira/browse/SOLR-13289
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ishan Chattopadhyaya
>Assignee: Tomas Eduardo Fernandez Lobbe
>Priority: Major
> Attachments: SOLR-13289.patch, SOLR-13289.patch
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> LUCENE-8135 introduced BlockMax WAND as a major speed improvement. Need to 
> expose this via Solr. When enabled, the numFound returned will not be exact.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0

[Jira]

[ 
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105402#comment-17105402
 ] 

Jan Høydahl commented on SOLR-12823:


Any plan to remove clusterstate.json for Solr 9? And perhaps let Overseer fail 
if it is non-empty, i.e. if not all collections are migrated to format 2?

> remove clusterstate.json in Lucene/Solr 8.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
>  Issue Type: Task
>Reporter: Varun Thacker
>Priority: Major
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove 
> that in 8.0
> It stays empty unless you explicitly ask to create the collection with the 
> old "stateFormat" and there is no reason for one to create a collection with 
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (SOLR-14474) Fix auxilliary class warnings in Solr core

[Erick Erickson (Jira)]

Erick Erickson created SOLR-14474:
-

 Summary: Fix auxilliary class warnings in Solr core
 Key: SOLR-14474
 URL: https://issues.apache.org/jira/browse/SOLR-14474
 Project: Solr
  Issue Type: Sub-task
Reporter: Erick Erickson
Assignee: Erick Erickson


We have quite a number of situations where multiple classes are declared in a 
single source file, which is a poor practice. I ran across a bunch of these in 
solr/core, and [~mdrob] fixed some of these in SOLR-14426. [~dsmiley] looked at 
those and thought that it would have been better to just move a particular 
class to its own file. And [~uschindler] do you have any comments?

I have a fork with a _bunch_ of changes to get warnings out that include moving 
more than a few classes into static inner classes, including the one Mike did. 
I do NOT intend to commit this, it's too big/sprawling, but it does serve to 
show a variety of situations. See: 
https://github.com/ErickErickson/lucene-solr/tree/jira/SOLR-10810 for how ugly 
it all looks. I intend to break this wodge down into smaller tasks and start 
over now that I have a clue as to the scope. And do ignore the generics changes 
as well as the consequences of upgrading apache commons CLI, those need to be 
their own JIRA.

What I'd like to do is agree on some guidelines for when to move classes to 
their own file and when to move them to static inner classes.

Some things I saw, reference the fork for the changes (again, I won't check 
that in).

1> DocValuesAcc has no fewer than 9 classes that could be moved inside the main 
class. But they all become "static abstract". And take 
"DoubleSortedNumericDVAcc" in that class, It gets extended over in 4 other 
files. How would all that get resolved? How many of them would people recommend 
moving into their own files? Do we want to proliferate all those? And so on 
with all the other plethora of classes in org.apache.solr.search.facet.

This is particularly thorny because the choices would be about a zillion new 
classes or about a zillion edits.

Does the idea of abstract .vs. concrete classes make any difference? IOW, if we 
change an abstract class to a nested class, then maybe we just have to change 
the class(es) that extend it?

2> StatsComponent.StatsInfo probably should be its own file?

3> FloatCmp, LongCmp, DoubleCmp all declare classes with "Comp" rather than 
"Cmp". Those files should just be renamed.

4> JSONResponseWriter. ???

5> FacetRangeProcessor seems like it needs its own class

6> FacetRequestSorted seems like it needs its own class

7> FacetModule

So what I'd like going forward is to agree on some guidelines to resolve 
whether to move a class to its own file or make it nested (probably static). 
Not hard-and-fast rules, just something to cut down on the rework due to 
objections.

And what about backporting to 8x? My suggestion is to backport what's 
easy/doesn't break back-compat in order to make keeping the two branches in 
sync.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14426) forbidden api error during precommit DateMathFunction

[Erick Erickson (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105428#comment-17105428
 ] 

Erick Erickson commented on SOLR-14426:
---

Also, I don't see any reason this couldn't  be backported to 8x?

I'm going to start a bit of a discussion on SOLR-10778 about how to approach 
this in general rather than have to debate one-by-one. It's a sub-task 
https://issues.apache.org/jira/browse/SOLR-14474.

Let's get some kind of sense of how to approach all this before having more 
rework requests.

> forbidden api error during precommit DateMathFunction
> -
>
> Key: SOLR-14426
> URL: https://issues.apache.org/jira/browse/SOLR-14426
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Build
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> When running `./gradlew precommit` I'll occasionally see
> {code}
> * What went wrong:
> Execution failed for task ':solr:contrib:analytics:forbiddenApisMain'.
> > de.thetaphi.forbiddenapis.ForbiddenApiException: Check for forbidden API 
> > calls failed while scanning class 
> > 'org.apache.solr.analytics.function.mapping.DateMathFunction' 
> > (DateMathFunction.java): java.lang.ClassNotFoundException: 
> > org.apache.solr.analytics.function.mapping.DateMathValueFunction (while 
> > looking up details about referenced class 
> > 'org.apache.solr.analytics.function.mapping.DateMathValueFunction')
> {code}
> `./gradlew clean` fixes this, but I don't understand what or why this 
> happens. Feels like a gradle issue?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (SOLR-14475) Fix deprecation warnings resulting from upgrading commons cli to 1.4

[Erick Erickson (Jira)]

Erick Erickson created SOLR-14475:
-

 Summary: Fix deprecation warnings resulting from upgrading commons 
cli to 1.4
 Key: SOLR-14475
 URL: https://issues.apache.org/jira/browse/SOLR-14475
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Erick Erickson
Assignee: Erick Erickson






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14474) Fix auxilliary class warnings in Solr core

[David Smiley (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105452#comment-17105452
 ] 

David Smiley commented on SOLR-14474:
-

This reads like a dev list post; lets just discuss this on the dev list?

> Fix auxilliary class warnings in Solr core
> --
>
> Key: SOLR-14474
> URL: https://issues.apache.org/jira/browse/SOLR-14474
> Project: Solr
>  Issue Type: Sub-task
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> We have quite a number of situations where multiple classes are declared in a 
> single source file, which is a poor practice. I ran across a bunch of these 
> in solr/core, and [~mdrob] fixed some of these in SOLR-14426. [~dsmiley] 
> looked at those and thought that it would have been better to just move a 
> particular class to its own file. And [~uschindler] do you have any comments?
> I have a fork with a _bunch_ of changes to get warnings out that include 
> moving more than a few classes into static inner classes, including the one 
> Mike did. I do NOT intend to commit this, it's too big/sprawling, but it does 
> serve to show a variety of situations. See: 
> https://github.com/ErickErickson/lucene-solr/tree/jira/SOLR-10810 for how 
> ugly it all looks. I intend to break this wodge down into smaller tasks and 
> start over now that I have a clue as to the scope. And do ignore the generics 
> changes as well as the consequences of upgrading apache commons CLI, those 
> need to be their own JIRA.
> What I'd like to do is agree on some guidelines for when to move classes to 
> their own file and when to move them to static inner classes.
> Some things I saw, reference the fork for the changes (again, I won't check 
> that in).
> 1> DocValuesAcc has no fewer than 9 classes that could be moved inside the 
> main class. But they all become "static abstract". And take 
> "DoubleSortedNumericDVAcc" in that class, It gets extended over in 4 other 
> files. How would all that get resolved? How many of them would people 
> recommend moving into their own files? Do we want to proliferate all those? 
> And so on with all the other plethora of classes in 
> org.apache.solr.search.facet.
> This is particularly thorny because the choices would be about a zillion new 
> classes or about a zillion edits.
> Does the idea of abstract .vs. concrete classes make any difference? IOW, if 
> we change an abstract class to a nested class, then maybe we just have to 
> change the class(es) that extend it?
> 2> StatsComponent.StatsInfo probably should be its own file?
> 3> FloatCmp, LongCmp, DoubleCmp all declare classes with "Comp" rather than 
> "Cmp". Those files should just be renamed.
> 4> JSONResponseWriter. ???
> 5> FacetRangeProcessor seems like it needs its own class
> 6> FacetRequestSorted seems like it needs its own class
> 7> FacetModule
> So what I'd like going forward is to agree on some guidelines to resolve 
> whether to move a class to its own file or make it nested (probably static). 
> Not hard-and-fast rules, just something to cut down on the rework due to 
> objections.
> And what about backporting to 8x? My suggestion is to backport what's 
> easy/doesn't break back-compat in order to make keeping the two branches in 
> sync.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9321) Port documentation task to gradle

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105471#comment-17105471
 ] 

Dawid Weiss commented on LUCENE-9321:
-

I updated Uwe's PR with a few cleanups, adding another task that produces 
relative links and is pretty much ready for "site" consumption. Please take a 
look. 

> Port documentation task to gradle
> -
>
> Key: LUCENE-9321
> URL: https://issues.apache.org/jira/browse/LUCENE-9321
> Project: Lucene - Core
>  Issue Type: Sub-task
>  Components: general/build
>Reporter: Tomoko Uchida
>Assignee: Uwe Schindler
>Priority: Major
> Fix For: master (9.0)
>
> Attachments: screenshot-1.png
>
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> This is a placeholder issue for porting ant "documentation" task to gradle. 
> The generated documents should be able to be published on lucene.apache.org 
> web site on "as-is" basis.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14474) Fix auxilliary class warnings in Solr core

[Erick Erickson (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105477#comment-17105477
 ] 

Erick Erickson commented on SOLR-14474:
---

No. I want this to be recorded permanently for reference.

> Fix auxilliary class warnings in Solr core
> --
>
> Key: SOLR-14474
> URL: https://issues.apache.org/jira/browse/SOLR-14474
> Project: Solr
>  Issue Type: Sub-task
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> We have quite a number of situations where multiple classes are declared in a 
> single source file, which is a poor practice. I ran across a bunch of these 
> in solr/core, and [~mdrob] fixed some of these in SOLR-14426. [~dsmiley] 
> looked at those and thought that it would have been better to just move a 
> particular class to its own file. And [~uschindler] do you have any comments?
> I have a fork with a _bunch_ of changes to get warnings out that include 
> moving more than a few classes into static inner classes, including the one 
> Mike did. I do NOT intend to commit this, it's too big/sprawling, but it does 
> serve to show a variety of situations. See: 
> https://github.com/ErickErickson/lucene-solr/tree/jira/SOLR-10810 for how 
> ugly it all looks. I intend to break this wodge down into smaller tasks and 
> start over now that I have a clue as to the scope. And do ignore the generics 
> changes as well as the consequences of upgrading apache commons CLI, those 
> need to be their own JIRA.
> What I'd like to do is agree on some guidelines for when to move classes to 
> their own file and when to move them to static inner classes.
> Some things I saw, reference the fork for the changes (again, I won't check 
> that in).
> 1> DocValuesAcc has no fewer than 9 classes that could be moved inside the 
> main class. But they all become "static abstract". And take 
> "DoubleSortedNumericDVAcc" in that class, It gets extended over in 4 other 
> files. How would all that get resolved? How many of them would people 
> recommend moving into their own files? Do we want to proliferate all those? 
> And so on with all the other plethora of classes in 
> org.apache.solr.search.facet.
> This is particularly thorny because the choices would be about a zillion new 
> classes or about a zillion edits.
> Does the idea of abstract .vs. concrete classes make any difference? IOW, if 
> we change an abstract class to a nested class, then maybe we just have to 
> change the class(es) that extend it?
> 2> StatsComponent.StatsInfo probably should be its own file?
> 3> FloatCmp, LongCmp, DoubleCmp all declare classes with "Comp" rather than 
> "Cmp". Those files should just be renamed.
> 4> JSONResponseWriter. ???
> 5> FacetRangeProcessor seems like it needs its own class
> 6> FacetRequestSorted seems like it needs its own class
> 7> FacetModule
> So what I'd like going forward is to agree on some guidelines to resolve 
> whether to move a class to its own file or make it nested (probably static). 
> Not hard-and-fast rules, just something to cut down on the rework due to 
> objections.
> And what about backporting to 8x? My suggestion is to backport what's 
> easy/doesn't break back-compat in order to make keeping the two branches in 
> sync.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14384) Stack SolrRequestInfo

[Nazerke Seidan (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105501#comment-17105501
 ] 

Nazerke Seidan commented on SOLR-14384:
---

[~mkhl] , I am working on this issue by introducing suggested stack idea 
([~dsmiley] ).  I noticed SolrRequestInfoSuspender extends SolrRequestInfo and 
removes the threadlocal later on. But SolrRequestInfoSuspender is no longer 
needed now as all SolrRequestInfos are stacked. What do you think if we remove 
SolrRequestInfoSuspender? 

> Stack SolrRequestInfo
> -
>
> Key: SOLR-14384
> URL: https://issues.apache.org/jira/browse/SOLR-14384
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mikhail Khludnev
>Priority: Minor
>
> Sometimes SolrRequestInfo need to be suspended or overridden. [~dsmiley] 
> suggests to introduce stacking for it. See linked issues for the context and 
> discussion.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14347) Autoscaling placement wrong when concurrent replica placements are calculated

[Ilan Ginzburg (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105510#comment-17105510
 ] 

Ilan Ginzburg commented on SOLR-14347:
--

I didn't look in detail at how Sessions are used, but can we use a copy of the 
cached Session rather than a new one built from ZK and once all computation is 
done, copy over the new cluster state from the computed Session back to the 
original one, then return the original one?

If you think it makes sense I can look more in detail and submit a PR.

> Autoscaling placement wrong when concurrent replica placements are calculated
> -
>
> Key: SOLR-14347
> URL: https://issues.apache.org/jira/browse/SOLR-14347
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling
>Affects Versions: 8.5
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Critical
> Fix For: 8.6
>
> Attachments: SOLR-14347.patch
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Steps to reproduce:
>  * create a cluster of a few nodes (tested with 7 nodes)
>  * define per-collection policies that distribute replicas exclusively on 
> different nodes per policy
>  * concurrently create a few collections, each using a different policy
>  * resulting replica placement will be seriously wrong, causing many policy 
> violations
> Running the same scenario but instead creating collections sequentially 
> results in no violations.
> I suspect this is caused by incorrect locking level for all collection 
> operations (as defined in {{CollectionParams.CollectionAction}}) that create 
> new replica placements - i.e. CREATE, ADDREPLICA, MOVEREPLICA, DELETENODE, 
> REPLACENODE, SPLITSHARD, RESTORE, REINDEXCOLLECTION. All of these operations 
> use the policy engine to create new replica placements, and as a result they 
> change the cluster state. However, currently these operations are locked (in 
> {{OverseerCollectionMessageHandler.lockTask}} ) using 
> {{LockLevel.COLLECTION}}. In practice this means that the lock is held only 
> for the particular collection that is being modified.
> A straightforward fix for this issue is to change the locking level to 
> CLUSTER (and I confirm this fixes the scenario described above). However, 
> this effectively serializes all collection operations listed above, which 
> will result in general slow-down of all collection operations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0

[Jira]

[ 
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105542#comment-17105542
 ] 

Jan Høydahl commented on SOLR-12823:


[~murblanc] volunteered on mailing list to try a PR. I have not looked into 
this in detail. You should consult 
[https://lucene.apache.org/solr/guide/8_5/cluster-node-management.html#migratestateformat]
 and the code of course to understand this move. But this could be an approach:
 * In 8.6 do at least this
 ## print a big WARNING if clusterstate.json is used for a collection, telling 
user to migrate
 ## OR auto migrate all collections on overseer startup? Is it safe?
 * In 9.0 (master), do at least this:
 ## Add migration note to RefGuide, telling users to run MIGRATESTATEFORMAT for 
all collections before upgrading
 ## remove parsing of and support for clusterstate.json in Overseer
 ## Perhaps add a check in Overseer and if the old znode exists and is 
non-empty, print an ERROR and abort?
 ## If clusterstatus.json exists and is empty, remove it and print a log

> remove clusterstate.json in Lucene/Solr 8.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
>  Issue Type: Task
>Reporter: Varun Thacker
>Priority: Major
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove 
> that in 8.0
> It stays empty unless you explicitly ask to create the collection with the 
> old "stateFormat" and there is no reason for one to create a collection with 
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14474) Fix auxilliary class warnings in Solr core

[David Smiley (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1710#comment-1710
 ] 

David Smiley commented on SOLR-14474:
-

You speak of "a zillion new classes" but I think you meant new *source files* 
(the classes already exist, we're just moving them), unless I'm 
misunderstanding you.  Secondly, clearly a zillion is made up but I don't think 
we're really talking about so many source files; like maybe 10.

RE DocValuesAcc:  I think these classes should be inside DocValuesAcc because 
they are subclasses of DocValuesAcc; there's a tight coupling.  Additionally, 
you might consider renaming them to remove the redundant suffixes of "DVAcc" 
because references to these classes will need to refer to the containing class 
already.

RE StatsInfo: I agree it should be its own standalone class because it is not 
tightly coupled to StatsComponent. *If* hypothetically only StatsComponent 
created these, then I'd argue the reverse.

RE FloatCmp etc.:  Weird... these should be static constant anonymous singleton 
instances on the parent interface.

RE ArrayOfNameTypeValueJSONWriter inside JSONResponseWriter.java:  should 
become inner class of JSONResponseWriter.java because it's only used by it.

RE NaNFloatWriter inside JSONResponseWriter.java: only has two subclasses 
externally so I think put as inner class of JSONResponseWriter.

RE FacetRangeProcessor: move to its own file; it's huge.

RE FacetParser: move to its own file with it's sub-parsers

 

My suggested guideline for how to refactor a class that is both top-level (not 
inside another class), package-private, and that is NOT in its own source file 
– a bit of a problem for compilers.  The outcome is either to move to its own 
source file or to move it inside the scope of the public class it's currently 
located near.
 * Is this class only referenced by the source file it's currently in?  Easy; 
move it inside the public class there.
 * Is the class tightly associated with the public class in the same source 
file?  _Probably_ move it inside that public class.
 * Is it referenced by >= 5 other source files?  _Probably_ move it to its own 
source file.
 * Is it really long (200+ lines of code)?  _Probably_ move it to its own 
source file.
 * Otherwise, no guidance.  Develop your own opinion if possible and ask for 
peer-review.

> Fix auxilliary class warnings in Solr core
> --
>
> Key: SOLR-14474
> URL: https://issues.apache.org/jira/browse/SOLR-14474
> Project: Solr
>  Issue Type: Sub-task
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> We have quite a number of situations where multiple classes are declared in a 
> single source file, which is a poor practice. I ran across a bunch of these 
> in solr/core, and [~mdrob] fixed some of these in SOLR-14426. [~dsmiley] 
> looked at those and thought that it would have been better to just move a 
> particular class to its own file. And [~uschindler] do you have any comments?
> I have a fork with a _bunch_ of changes to get warnings out that include 
> moving more than a few classes into static inner classes, including the one 
> Mike did. I do NOT intend to commit this, it's too big/sprawling, but it does 
> serve to show a variety of situations. See: 
> https://github.com/ErickErickson/lucene-solr/tree/jira/SOLR-10810 for how 
> ugly it all looks. I intend to break this wodge down into smaller tasks and 
> start over now that I have a clue as to the scope. And do ignore the generics 
> changes as well as the consequences of upgrading apache commons CLI, those 
> need to be their own JIRA.
> What I'd like to do is agree on some guidelines for when to move classes to 
> their own file and when to move them to static inner classes.
> Some things I saw, reference the fork for the changes (again, I won't check 
> that in).
> 1> DocValuesAcc has no fewer than 9 classes that could be moved inside the 
> main class. But they all become "static abstract". And take 
> "DoubleSortedNumericDVAcc" in that class, It gets extended over in 4 other 
> files. How would all that get resolved? How many of them would people 
> recommend moving into their own files? Do we want to proliferate all those? 
> And so on with all the other plethora of classes in 
> org.apache.solr.search.facet.
> This is particularly thorny because the choices would be about a zillion new 
> classes or about a zillion edits.
> Does the idea of abstract .vs. concrete classes make any difference? IOW, if 
> we change an abstract class to a nested class, then maybe we just have to 
> change the class(es) that extend it?
> 2> StatsComponent.StatsInfo probably should be its own file?
> 3> FloatCmp, LongCmp, DoubleCmp all declare classes with "Comp" rather than 
> "Cmp". Those

[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0

[Ilan Ginzburg (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105565#comment-17105565
 ] 

Ilan Ginzburg commented on SOLR-12823:
--

Will give it a try, starting with the 9.0 (master) changes.

If successful, will then implement the 1st option for 8_x (2nd one feels risky 
and too intrusive).

> remove clusterstate.json in Lucene/Solr 8.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
>  Issue Type: Task
>Reporter: Varun Thacker
>Priority: Major
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove 
> that in 8.0
> It stays empty unless you explicitly ask to create the collection with the 
> old "stateFormat" and there is no reason for one to create a collection with 
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9350) Don't cache automata on FuzzyQuery

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105587#comment-17105587
 ] 

ASF subversion and git services commented on LUCENE-9350:
-

Commit 05ba52bd21c780692c367c7ea316192047fec4cb in lucene-solr's branch 
refs/heads/branch_8_5 from Alan Woodward
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=05ba52b ]

LUCENE-9350: Don't hold references to large automata on FuzzyQuery (#1467)

LUCENE-9068 moved fuzzy automata construction into FuzzyQuery itself. However,
this has the nasty side-effect of blowing up query caches that expect queries 
to be
fairly small. This commit restores the previous behaviour of caching the large 
automata
on an AttributeSource shared between segments, while making the construction a
bit clearer by factoring it out into a package-private FuzzyAutomatonBuilder.


> Don't cache automata on FuzzyQuery
> --
>
> Key: LUCENE-9350
> URL: https://issues.apache.org/jira/browse/LUCENE-9350
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 8.6
>
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> LUCENE-9068 moved construction of FuzzyQuery's automaton directly onto the 
> query itself.  However, as SOLR-14428 demonstrates, this ends up blowing up 
> query caches that assume query objects are fairly small when calculating 
> their memory usage.  We should move automaton construction back into 
> FuzzyTermsEnum, while keeping as much of the nice refactoring of LUCENE-9068 
> as possible.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9350) Don't cache automata on FuzzyQuery

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105589#comment-17105589
 ] 

ASF subversion and git services commented on LUCENE-9350:
-

Commit 094784f6124a35f189808595ff8b9c4dfacacd1e in lucene-solr's branch 
refs/heads/branch_8_5 from Alan Woodward
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=094784f ]

LUCENE-9350: Add changes entry


> Don't cache automata on FuzzyQuery
> --
>
> Key: LUCENE-9350
> URL: https://issues.apache.org/jira/browse/LUCENE-9350
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 8.6
>
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> LUCENE-9068 moved construction of FuzzyQuery's automaton directly onto the 
> query itself.  However, as SOLR-14428 demonstrates, this ends up blowing up 
> query caches that assume query objects are fairly small when calculating 
> their memory usage.  We should move automaton construction back into 
> FuzzyTermsEnum, while keeping as much of the nice refactoring of LUCENE-9068 
> as possible.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9068) Build FuzzyQuery automata up-front

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105588#comment-17105588
 ] 

ASF subversion and git services commented on LUCENE-9068:
-

Commit 05ba52bd21c780692c367c7ea316192047fec4cb in lucene-solr's branch 
refs/heads/branch_8_5 from Alan Woodward
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=05ba52b ]

LUCENE-9350: Don't hold references to large automata on FuzzyQuery (#1467)

LUCENE-9068 moved fuzzy automata construction into FuzzyQuery itself. However,
this has the nasty side-effect of blowing up query caches that expect queries 
to be
fairly small. This commit restores the previous behaviour of caching the large 
automata
on an AttributeSource shared between segments, while making the construction a
bit clearer by factoring it out into a package-private FuzzyAutomatonBuilder.


> Build FuzzyQuery automata up-front
> --
>
> Key: LUCENE-9068
> URL: https://issues.apache.org/jira/browse/LUCENE-9068
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 8.5
>
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> FuzzyQuery builds a set of levenshtein automata (one for each possible edit 
> distance) at rewrite time, and passes them between different TermsEnum 
> invocations using an attribute source.  This seems a bit needlessly 
> complicated, and also means that things like visiting a query end up building 
> the automata again.  We should instead build the automata at query 
> construction time, which is how AutomatonQuery does it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13190) Fuzzy search treated as server error instead of client error when terms are too complex

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-13190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105590#comment-17105590
 ] 

ASF subversion and git services commented on SOLR-13190:


Commit 677ee799feaa1bf3b0fe728d046afa0faa25eb93 in lucene-solr's branch 
refs/heads/branch_8_5 from Mike Drob
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=677ee79 ]

SOLR-13190 Backport unit test


> Fuzzy search treated as server error instead of client error when terms are 
> too complex
> ---
>
> Key: SOLR-13190
> URL: https://issues.apache.org/jira/browse/SOLR-13190
> Project: Solr
>  Issue Type: Bug
>  Components: search
>Affects Versions: 5.5, 7.7, 8.0
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> We've seen a fuzzy search end up breaking the automaton and getting reported 
> as a server error. This usage should be improved by
> 1) reporting as a client error, because it's similar to something like too 
> many boolean clauses queries in how an operator should deal with it
> 2) report what field is causing the error, since that currently must be 
> deduced from adjacent query logs and can be difficult if there are multiple 
> terms in the search
> This trigger was added to defend against adversarial regex but somehow hits 
> fuzzy terms as well, I don't understand enough about the automaton mechanisms 
> to really know how to approach a fix there, but improving the operability is 
> a good first step.
> relevant stack trace:
> {noformat}
> org.apache.lucene.util.automaton.TooComplexToDeterminizeException: 
> Determinizing automaton with 13632 states and 21348 transitions would result 
> in more than 1 states.
>   at 
> org.apache.lucene.util.automaton.Operations.determinize(Operations.java:746)
>   at 
> org.apache.lucene.util.automaton.RunAutomaton.(RunAutomaton.java:69)
>   at 
> org.apache.lucene.util.automaton.ByteRunAutomaton.(ByteRunAutomaton.java:32)
>   at 
> org.apache.lucene.util.automaton.CompiledAutomaton.(CompiledAutomaton.java:247)
>   at 
> org.apache.lucene.util.automaton.CompiledAutomaton.(CompiledAutomaton.java:133)
>   at 
> org.apache.lucene.search.FuzzyTermsEnum.(FuzzyTermsEnum.java:143)
>   at org.apache.lucene.search.FuzzyQuery.getTermsEnum(FuzzyQuery.java:154)
>   at 
> org.apache.lucene.search.MultiTermQuery$RewriteMethod.getTermsEnum(MultiTermQuery.java:78)
>   at 
> org.apache.lucene.search.TermCollectingRewrite.collectTerms(TermCollectingRewrite.java:58)
>   at 
> org.apache.lucene.search.TopTermsRewrite.rewrite(TopTermsRewrite.java:67)
>   at 
> org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:310)
>   at 
> org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:667)
>   at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:442)
>   at 
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:200)
>   at 
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1604)
>   at 
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1420)
>   at 
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:567)
>   at 
> org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1435)
>   at 
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:374)
>   at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
>   at org.apache.solr.core.SolrCore.execute(SolrCore.java:2559)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14465) Reproducing seed in FuzzySearchTest

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105591#comment-17105591
 ] 

ASF subversion and git services commented on SOLR-14465:


Commit 6739c494cca24cee21ef43e2bc893b761f132fcf in lucene-solr's branch 
refs/heads/branch_8_5 from Mike Drob
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6739c49 ]

SOLR-14465: Solr query handling code catches FuzzyTermsException

This reverts commit 7ea7ed72aca556f957a5de55911c852124db8715.


> Reproducing seed in FuzzySearchTest
> ---
>
> Key: SOLR-14465
> URL: https://issues.apache.org/jira/browse/SOLR-14465
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Affects Versions: master (9.0)
>Reporter: Erick Erickson
>Assignee: Mike Drob
>Priority: Major
> Fix For: master (9.0)
>
>
> reproduce with: ant test -Dtestcase=FuzzySearchTest 
> -Dtests.method=testTooComplex -Dtests.seed=B6A1F20B2D413B4 
> -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true 
> -Dtests.locale=yue-Hant-HK -Dtests.timezone=America/Dawson 
> -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>  
> [~romseygeek] [~mdrob]  My guess is that changes in LUCENE-9350/LUCENE-9068 
> changed the exception returned, but haven't looked very closely.
>  [junit4] FAILURE 3.01s | FuzzySearchTest.testTooComplex <<<
>  [junit4] > Throwable #1: junit.framework.AssertionFailedError: Unexpected 
> exception type, expected RemoteSolrException but got 
> org.apache.solr.client.solrj.SolrServerException: No live SolrServers 
> available to handle this request:[http://127.0.0.1:51202/solr/c1]
>  [junit4] > at 
> __randomizedtesting.SeedInfo.seed([B6A1F20B2D413B4:DDC85504AD0720FE]:0)
>  [junit4] > at 
> org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2751)
>  [junit4] > at 
> org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2739)
>  [junit4] > at 
> org.apache.solr.search.FuzzySearchTest.testTooComplex(FuzzySearchTest.java:64)
>  [junit4] > at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>  [junit4] > at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>  [junit4] > at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  [junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>  [junit4] > at java.base/java.lang.Thread.run(Thread.java:834)
>  [junit4] > Caused by: org.apache.solr.client.solrj.SolrServerException: No 
> live SolrServers available to handle this 
> request:[http://127.0.0.1:51202/solr/c1]
>  [junit4] > at 
> org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:345)
>  [junit4] > at 
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1147)
>  [junit4] > at 
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:910)
>  [junit4] > at 
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:842)
>  [junit4] > at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:207)
>  [junit4] > at 
> org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1004)
>  [junit4] > at 
> org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1019)
>  [junit4] > at 
> org.apache.solr.search.FuzzySearchTest.lambda$testTooComplex$0(FuzzySearchTest.java:64)
>  [junit4] > at 
> org.apache.lucene.util.LuceneTestCase._expectThrows(LuceneTestCase.java:2869)
>  [junit4] > at 
> org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2744)
>  [junit4] > ... 41 more
>  [junit4] > Caused by: 
> org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: 
> Error from server at http://127.0.0.1:51202/solr/c1: Term too complex: 
> headquarters(在日米海軍横須賀基地司令部庁舎/旧横須賀鎮守府会議所・横須賀海軍艦船部)
>  [junit4] > at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:665)
>  [junit4] > at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:265)
>  [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14423) static caches in StreamHandler ought to move to CoreContainer lifecycle

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105692#comment-17105692
 ] 

ASF subversion and git services commented on SOLR-14423:


Commit 4680e9245f26e8ba99400dfbbdc0002c6b2886fe in lucene-solr's branch 
refs/heads/master from Andrzej Bialecki
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4680e92 ]

SOLR-14423: Move static SolrClientCache from StreamHandler to CoreContainer for 
wider reuse and better life-cycle management.


> static caches in StreamHandler ought to move to CoreContainer lifecycle
> ---
>
> Key: SOLR-14423
> URL: https://issues.apache.org/jira/browse/SOLR-14423
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: David Smiley
>Assignee: Andrzej Bialecki
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> StreamHandler (at "/stream") has several statically declared caches.  I think 
> this is problematic, such as in testing wherein multiple nodes could be in 
> the same JVM.  One of them is more serious -- SolrClientCache which is 
> closed/cleared via a SolrCore close hook.  That's bad for performance but 
> also dangerous since another core might want to use one of these clients!
> CC [~jbernste]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9355) missing releases from testbackwardscompatibility

[Mike Drob (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105699#comment-17105699
 ] 

Mike Drob commented on LUCENE-9355:
---

[~noble.paul] - can you please take this up? I believe the lack of back compat 
indices will be blocking an 8.5.2 release

> missing releases from testbackwardscompatibility
> 
>
> Key: LUCENE-9355
> URL: https://issues.apache.org/jira/browse/LUCENE-9355
> Project: Lucene - Core
>  Issue Type: Test
>Reporter: Mike Drob
>Priority: Major
>
> I'm not sure what needs to be added for the 7.7.3 release, but can you take a 
> look at it [~noble] or figure out who to ask for help?
> {noformat}
>[smoker]   confirm all releases have coverage in TestBackwardsCompatibility
>[smoker] find all past Lucene releases...
>[smoker] run TestBackwardsCompatibility..
>[smoker] Releases that don't seem to be tested:
>[smoker]   7.7.3
>[smoker] Traceback (most recent call last):
>[smoker]   File 
> "/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
>  line 1487, in 
>[smoker] main()
>[smoker]   File 
> "/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
>  line 1413, in main
>[smoker] downloadOnly=c.download_only)
>[smoker]   File 
> "/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
>  line 1465, in smokeTest
>[smoker] unpackAndVerify(java, 'lucene', tmpDir, 'lucene-%s-src.tgz' % 
> version, gitRevision, version, testArgs, baseURL)
>[smoker]   File 
> "/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
>  line 566, in unpackAndVerify
>[smoker] verifyUnpacked(java, project, artifact, unpackPath, 
> gitRevision, version, testArgs, tmpDir, baseURL)
>[smoker]   File 
> "/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
>  line 752, in verifyUnpacked
>[smoker] confirmAllReleasesAreTestedForBackCompat(version, unpackPath)
>[smoker]   File 
> "/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
>  line 1388, in confirmAllReleasesAreTestedForBackCompat
>[smoker] raise RuntimeError('some releases are not tested by 
> TestBackwardsCompatibility?')
>[smoker] RuntimeError: some releases are not tested by 
> TestBackwardsCompatibility?
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-14465) Reproducing seed in FuzzySearchTest

[Mike Drob (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-14465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated SOLR-14465:
-
Fix Version/s: 8.5.2
   8.6

> Reproducing seed in FuzzySearchTest
> ---
>
> Key: SOLR-14465
> URL: https://issues.apache.org/jira/browse/SOLR-14465
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Affects Versions: master (9.0)
>Reporter: Erick Erickson
>Assignee: Mike Drob
>Priority: Major
> Fix For: master (9.0), 8.6, 8.5.2
>
>
> reproduce with: ant test -Dtestcase=FuzzySearchTest 
> -Dtests.method=testTooComplex -Dtests.seed=B6A1F20B2D413B4 
> -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true 
> -Dtests.locale=yue-Hant-HK -Dtests.timezone=America/Dawson 
> -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>  
> [~romseygeek] [~mdrob]  My guess is that changes in LUCENE-9350/LUCENE-9068 
> changed the exception returned, but haven't looked very closely.
>  [junit4] FAILURE 3.01s | FuzzySearchTest.testTooComplex <<<
>  [junit4] > Throwable #1: junit.framework.AssertionFailedError: Unexpected 
> exception type, expected RemoteSolrException but got 
> org.apache.solr.client.solrj.SolrServerException: No live SolrServers 
> available to handle this request:[http://127.0.0.1:51202/solr/c1]
>  [junit4] > at 
> __randomizedtesting.SeedInfo.seed([B6A1F20B2D413B4:DDC85504AD0720FE]:0)
>  [junit4] > at 
> org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2751)
>  [junit4] > at 
> org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2739)
>  [junit4] > at 
> org.apache.solr.search.FuzzySearchTest.testTooComplex(FuzzySearchTest.java:64)
>  [junit4] > at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>  [junit4] > at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>  [junit4] > at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  [junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>  [junit4] > at java.base/java.lang.Thread.run(Thread.java:834)
>  [junit4] > Caused by: org.apache.solr.client.solrj.SolrServerException: No 
> live SolrServers available to handle this 
> request:[http://127.0.0.1:51202/solr/c1]
>  [junit4] > at 
> org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:345)
>  [junit4] > at 
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1147)
>  [junit4] > at 
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:910)
>  [junit4] > at 
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:842)
>  [junit4] > at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:207)
>  [junit4] > at 
> org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1004)
>  [junit4] > at 
> org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1019)
>  [junit4] > at 
> org.apache.solr.search.FuzzySearchTest.lambda$testTooComplex$0(FuzzySearchTest.java:64)
>  [junit4] > at 
> org.apache.lucene.util.LuceneTestCase._expectThrows(LuceneTestCase.java:2869)
>  [junit4] > at 
> org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2744)
>  [junit4] > ... 41 more
>  [junit4] > Caused by: 
> org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: 
> Error from server at http://127.0.0.1:51202/solr/c1: Term too complex: 
> headquarters(在日米海軍横須賀基地司令部庁舎/旧横須賀鎮守府会議所・横須賀海軍艦船部)
>  [junit4] > at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:665)
>  [junit4] > at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:265)
>  [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (LUCENE-9350) Don't cache automata on FuzzyQuery

[Mike Drob (Jira)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-9350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated LUCENE-9350:
--
Fix Version/s: 8.5.2

> Don't cache automata on FuzzyQuery
> --
>
> Key: LUCENE-9350
> URL: https://issues.apache.org/jira/browse/LUCENE-9350
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 8.6, 8.5.2
>
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> LUCENE-9068 moved construction of FuzzyQuery's automaton directly onto the 
> query itself.  However, as SOLR-14428 demonstrates, this ends up blowing up 
> query caches that assume query objects are fairly small when calculating 
> their memory usage.  We should move automaton construction back into 
> FuzzyTermsEnum, while keeping as much of the nice refactoring of LUCENE-9068 
> as possible.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-13190) Fuzzy search treated as server error instead of client error when terms are too complex

[Mike Drob (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-13190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated SOLR-13190:
-
Fix Version/s: 8.5.2
   8.6

> Fuzzy search treated as server error instead of client error when terms are 
> too complex
> ---
>
> Key: SOLR-13190
> URL: https://issues.apache.org/jira/browse/SOLR-13190
> Project: Solr
>  Issue Type: Bug
>  Components: search
>Affects Versions: 5.5, 7.7, 8.0
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Fix For: master (9.0), 8.6, 8.5.2
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> We've seen a fuzzy search end up breaking the automaton and getting reported 
> as a server error. This usage should be improved by
> 1) reporting as a client error, because it's similar to something like too 
> many boolean clauses queries in how an operator should deal with it
> 2) report what field is causing the error, since that currently must be 
> deduced from adjacent query logs and can be difficult if there are multiple 
> terms in the search
> This trigger was added to defend against adversarial regex but somehow hits 
> fuzzy terms as well, I don't understand enough about the automaton mechanisms 
> to really know how to approach a fix there, but improving the operability is 
> a good first step.
> relevant stack trace:
> {noformat}
> org.apache.lucene.util.automaton.TooComplexToDeterminizeException: 
> Determinizing automaton with 13632 states and 21348 transitions would result 
> in more than 1 states.
>   at 
> org.apache.lucene.util.automaton.Operations.determinize(Operations.java:746)
>   at 
> org.apache.lucene.util.automaton.RunAutomaton.(RunAutomaton.java:69)
>   at 
> org.apache.lucene.util.automaton.ByteRunAutomaton.(ByteRunAutomaton.java:32)
>   at 
> org.apache.lucene.util.automaton.CompiledAutomaton.(CompiledAutomaton.java:247)
>   at 
> org.apache.lucene.util.automaton.CompiledAutomaton.(CompiledAutomaton.java:133)
>   at 
> org.apache.lucene.search.FuzzyTermsEnum.(FuzzyTermsEnum.java:143)
>   at org.apache.lucene.search.FuzzyQuery.getTermsEnum(FuzzyQuery.java:154)
>   at 
> org.apache.lucene.search.MultiTermQuery$RewriteMethod.getTermsEnum(MultiTermQuery.java:78)
>   at 
> org.apache.lucene.search.TermCollectingRewrite.collectTerms(TermCollectingRewrite.java:58)
>   at 
> org.apache.lucene.search.TopTermsRewrite.rewrite(TopTermsRewrite.java:67)
>   at 
> org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:310)
>   at 
> org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:667)
>   at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:442)
>   at 
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:200)
>   at 
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1604)
>   at 
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1420)
>   at 
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:567)
>   at 
> org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1435)
>   at 
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:374)
>   at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
>   at org.apache.solr.core.SolrCore.execute(SolrCore.java:2559)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14105) Http2SolrClient SSL not working in branch_8x

[Tim (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105732#comment-17105732
 ] 

Tim commented on SOLR-14105:


This is causing issues in production Solr SSL setups that use multi domain 
certificates (for various deployment reasons). Any workarounds (other than 
generating self-signed certs and not using proper valid CA signed certs)?

> Http2SolrClient SSL not working in branch_8x
> 
>
> Key: SOLR-14105
> URL: https://issues.apache.org/jira/browse/SOLR-14105
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 8.5
>Reporter: Jan Høydahl
>Assignee: Kevin Risden
>Priority: Major
> Attachments: SOLR-14105.patch
>
>
> In branch_8x we upgraded to Jetty 9.4.24. This causes the following 
> exceptions when attempting to start server with SSL:
> {noformat}
> 2019-12-17 14:46:16.646 ERROR (main) [   ] o.a.s.c.SolrCore 
> null:org.apache.solr.common.SolrException: Error instantiating 
> shardHandlerFactory class [HttpShardHandlerFactory]: 
> java.lang.UnsupportedOperationException: X509ExtendedKeyManager only 
> supported on Server
>   at 
> org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:56)
>   at org.apache.solr.core.CoreContainer.load(CoreContainer.java:633)
> ...
> Caused by: java.lang.RuntimeException: 
> java.lang.UnsupportedOperationException: X509ExtendedKeyManager only 
> supported on Server
>   at 
> org.apache.solr.client.solrj.impl.Http2SolrClient.createHttpClient(Http2SolrClient.java:224)
>   at 
> org.apache.solr.client.solrj.impl.Http2SolrClient.(Http2SolrClient.java:154)
>   at 
> org.apache.solr.client.solrj.impl.Http2SolrClient$Builder.build(Http2SolrClient.java:833)
>   at 
> org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:321)
>   at 
> org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:51)
>   ... 50 more
> Caused by: java.lang.UnsupportedOperationException: X509ExtendedKeyManager 
> only supported on Server
>   at 
> org.eclipse.jetty.util.ssl.SslContextFactory.newSniX509ExtendedKeyManager(SslContextFactory.java:1273)
>   at 
> org.eclipse.jetty.util.ssl.SslContextFactory.getKeyManagers(SslContextFactory.java:1255)
>   at 
> org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:374)
>   at 
> org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:245)
>  {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14105) Http2SolrClient SSL not working in branch_8x

[Akhmad Amirov (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105752#comment-17105752
 ] 

Akhmad Amirov commented on SOLR-14105:
--

I created self-signed with 1 DNS entry and it still throwing 
"...X509ExtendedKeyManager only supported on Server..." I deployed 9.4.25 and 
it still says "...X509ExtendedKeyManager only supported on Server...". So, I am 
not sure how does it work and have anyone at all using TLS enabled Solr. 

> Http2SolrClient SSL not working in branch_8x
> 
>
> Key: SOLR-14105
> URL: https://issues.apache.org/jira/browse/SOLR-14105
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 8.5
>Reporter: Jan Høydahl
>Assignee: Kevin Risden
>Priority: Major
> Attachments: SOLR-14105.patch
>
>
> In branch_8x we upgraded to Jetty 9.4.24. This causes the following 
> exceptions when attempting to start server with SSL:
> {noformat}
> 2019-12-17 14:46:16.646 ERROR (main) [   ] o.a.s.c.SolrCore 
> null:org.apache.solr.common.SolrException: Error instantiating 
> shardHandlerFactory class [HttpShardHandlerFactory]: 
> java.lang.UnsupportedOperationException: X509ExtendedKeyManager only 
> supported on Server
>   at 
> org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:56)
>   at org.apache.solr.core.CoreContainer.load(CoreContainer.java:633)
> ...
> Caused by: java.lang.RuntimeException: 
> java.lang.UnsupportedOperationException: X509ExtendedKeyManager only 
> supported on Server
>   at 
> org.apache.solr.client.solrj.impl.Http2SolrClient.createHttpClient(Http2SolrClient.java:224)
>   at 
> org.apache.solr.client.solrj.impl.Http2SolrClient.(Http2SolrClient.java:154)
>   at 
> org.apache.solr.client.solrj.impl.Http2SolrClient$Builder.build(Http2SolrClient.java:833)
>   at 
> org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:321)
>   at 
> org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:51)
>   ... 50 more
> Caused by: java.lang.UnsupportedOperationException: X509ExtendedKeyManager 
> only supported on Server
>   at 
> org.eclipse.jetty.util.ssl.SslContextFactory.newSniX509ExtendedKeyManager(SslContextFactory.java:1273)
>   at 
> org.eclipse.jetty.util.ssl.SslContextFactory.getKeyManagers(SslContextFactory.java:1255)
>   at 
> org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:374)
>   at 
> org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:245)
>  {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9355) missing releases from testbackwardscompatibility

[Mike Drob (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105756#comment-17105756
 ] 

Mike Drob commented on LUCENE-9355:
---

Also, we're missing entries for 7.7.3 in {{dev-tools/doap/lucene.rdf}} and 
{{dev-tools/doap/solr.rdf}}

> missing releases from testbackwardscompatibility
> 
>
> Key: LUCENE-9355
> URL: https://issues.apache.org/jira/browse/LUCENE-9355
> Project: Lucene - Core
>  Issue Type: Test
>Reporter: Mike Drob
>Priority: Major
>
> I'm not sure what needs to be added for the 7.7.3 release, but can you take a 
> look at it [~noble] or figure out who to ask for help?
> {noformat}
>[smoker]   confirm all releases have coverage in TestBackwardsCompatibility
>[smoker] find all past Lucene releases...
>[smoker] run TestBackwardsCompatibility..
>[smoker] Releases that don't seem to be tested:
>[smoker]   7.7.3
>[smoker] Traceback (most recent call last):
>[smoker]   File 
> "/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
>  line 1487, in 
>[smoker] main()
>[smoker]   File 
> "/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
>  line 1413, in main
>[smoker] downloadOnly=c.download_only)
>[smoker]   File 
> "/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
>  line 1465, in smokeTest
>[smoker] unpackAndVerify(java, 'lucene', tmpDir, 'lucene-%s-src.tgz' % 
> version, gitRevision, version, testArgs, baseURL)
>[smoker]   File 
> "/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
>  line 566, in unpackAndVerify
>[smoker] verifyUnpacked(java, project, artifact, unpackPath, 
> gitRevision, version, testArgs, tmpDir, baseURL)
>[smoker]   File 
> "/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
>  line 752, in verifyUnpacked
>[smoker] confirmAllReleasesAreTestedForBackCompat(version, unpackPath)
>[smoker]   File 
> "/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
>  line 1388, in confirmAllReleasesAreTestedForBackCompat
>[smoker] raise RuntimeError('some releases are not tested by 
> TestBackwardsCompatibility?')
>[smoker] RuntimeError: some releases are not tested by 
> TestBackwardsCompatibility?
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0

[Ilan Ginzburg (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105764#comment-17105764
 ] 

Ilan Ginzburg commented on SOLR-12823:
--

Need to remove the "legacyCloud" option when removing clusterstate.json and 
have the cluster behave as if it were false.
[~erickerickson] do you agree?

> remove clusterstate.json in Lucene/Solr 8.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
>  Issue Type: Task
>Reporter: Varun Thacker
>Priority: Major
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove 
> that in 8.0
> It stays empty unless you explicitly ask to create the collection with the 
> old "stateFormat" and there is no reason for one to create a collection with 
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14456) Compressed requests fail in SolrCloud when the request is routed internally by the serving solr node

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105770#comment-17105770
 ] 

ASF subversion and git services commented on SOLR-14456:


Commit adddab9d1466675cd79fd06c37592000a56841d2 in lucene-solr's branch 
refs/heads/master from Samuel García Martínez
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=adddab9d ]

SOLR-14456: Fix Content-Type header forwarding on compressed requests (#1480)

Co-authored-by: Samuel García Martínez 

> Compressed requests fail in SolrCloud when the request is routed internally 
> by the serving solr node
> 
>
> Key: SOLR-14456
> URL: https://issues.apache.org/jira/browse/SOLR-14456
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.7.2
> Environment: Solr version: 7.7.2
> Solr cloud enabled
> Cluster topology: 6 nodes, 1 single collection, 10 shards and 3 replicas. 1 
> HTTP LB using Round Robin over all nodes
> All cluster nodes have gzip enabled for all paths, all HTTP verbs and all 
> MIME types.
> Solr client: HttpSolrClient targeting the HTTP LB
> h3.  
>Reporter: Samuel García Martínez
>Assignee: Houston Putman
>Priority: Major
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> h3. Solr cluster setup
>  * Solr version: 7.7.2
>  * Solr cloud enabled
>  * Cluster topology: 6 nodes, 1 single collection, 10 shards and 3 replicas. 
> 1 HTTP LB using Round Robin over all nodes
>  * All cluster nodes have gzip enabled for all paths, all HTTP verbs and all 
> MIME types.
>  * Solr client: HttpSolrClient targeting the HTTP LB
> h3. Problem description
> When the Solr node that receives the request has to forward it
> to a Solr Node that can actually perform the query, the response headers are 
> added incorrectly to the response, causing any HTTP client to fail, whether 
> it's a SolrClient or a basic HTTP client implementation with any other SDK.
> To simplify the case, let's try to start from the following repro scenario:
>  * Start one node with cloud mode and port 8983
>  * Create one single collection (1 shard, 1 replica)
>  * Start another node with port 8984 and the previusly started zk (-z 
> localhost:9983)
>  * Start a java application and query the cluster using the node on port 8984 
> (the one that doesn't host the collection)
> So, then something like this happens:
>  * The application queries node:8984 with compression enabled 
> ("Accept-Encoding: gzip")
> and wt=javabin
>  * Node:8984 can't perform the query and creates a http request behind the 
> scenes to node:8983
>  * Node:8983 returns a gzipped response with "Content-Encoding: gzip" and 
> "Content-Type:
> application/octet-stream"
> Node:8984 adds the "Content-Encoding: gzip" header as character stream to the 
> response
> (it should be forwarded as "Content-Encoding" header, not character encoding)
>  * HttpSolrClient receives a "Content-Type: 
> application/octet-stream;charset=gzip", causing
> an exception.
>  * HttpSolrClient tries to quietly close the connection, but since the stream 
> is broken,
> the Utils.consumeFully fails to actually consume the entity (it throws 
> another exception in
> GzipDecompressingEntity#getContent() with "not in GZIP format")
> The exception thrown by HttpSolrClient is:
> {code:java}
> java.nio.charset.UnsupportedCharsetException: gzip
>  at java.nio.charset.Charset.forName(Charset.java:531)
>  at org.apache.http.entity.ContentType.create(ContentType.java:271)
>  at org.apache.http.entity.ContentType.create(ContentType.java:261)
>  at org.apache.http.entity.ContentType.parse(ContentType.java:319)
>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:591)
>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
>  at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
>  at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1015)
>  at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1031)
>  at 
> org.apache.solr.client.solrj.SolrClient$$FastClassBySpringCGLIB$$7fcf36a0.invoke()
>  at 
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218){code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14474) Fix auxilliary class warnings in Solr core

[Erick Erickson (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105772#comment-17105772
 ] 

Erick Erickson commented on SOLR-14474:
---

Thanks for the detailed response. Yeah, "zillion" is indefinite. If you take a 
quick look at the PR I referenced, you'll see all the changes in the facet 
code. That's going to be gnarly. I need to think a bit about how to make it a 
smaller set of edits, what I did for SOLR-10810 was mindless. Helps to get a 
sense of the problem though.

Your guidelines seem like a great place build from, that's certainly along the 
lines of what I was thinking it's nice to see I wasn't totally off base.

> Fix auxilliary class warnings in Solr core
> --
>
> Key: SOLR-14474
> URL: https://issues.apache.org/jira/browse/SOLR-14474
> Project: Solr
>  Issue Type: Sub-task
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> We have quite a number of situations where multiple classes are declared in a 
> single source file, which is a poor practice. I ran across a bunch of these 
> in solr/core, and [~mdrob] fixed some of these in SOLR-14426. [~dsmiley] 
> looked at those and thought that it would have been better to just move a 
> particular class to its own file. And [~uschindler] do you have any comments?
> I have a fork with a _bunch_ of changes to get warnings out that include 
> moving more than a few classes into static inner classes, including the one 
> Mike did. I do NOT intend to commit this, it's too big/sprawling, but it does 
> serve to show a variety of situations. See: 
> https://github.com/ErickErickson/lucene-solr/tree/jira/SOLR-10810 for how 
> ugly it all looks. I intend to break this wodge down into smaller tasks and 
> start over now that I have a clue as to the scope. And do ignore the generics 
> changes as well as the consequences of upgrading apache commons CLI, those 
> need to be their own JIRA.
> What I'd like to do is agree on some guidelines for when to move classes to 
> their own file and when to move them to static inner classes.
> Some things I saw, reference the fork for the changes (again, I won't check 
> that in).
> 1> DocValuesAcc has no fewer than 9 classes that could be moved inside the 
> main class. But they all become "static abstract". And take 
> "DoubleSortedNumericDVAcc" in that class, It gets extended over in 4 other 
> files. How would all that get resolved? How many of them would people 
> recommend moving into their own files? Do we want to proliferate all those? 
> And so on with all the other plethora of classes in 
> org.apache.solr.search.facet.
> This is particularly thorny because the choices would be about a zillion new 
> classes or about a zillion edits.
> Does the idea of abstract .vs. concrete classes make any difference? IOW, if 
> we change an abstract class to a nested class, then maybe we just have to 
> change the class(es) that extend it?
> 2> StatsComponent.StatsInfo probably should be its own file?
> 3> FloatCmp, LongCmp, DoubleCmp all declare classes with "Comp" rather than 
> "Cmp". Those files should just be renamed.
> 4> JSONResponseWriter. ???
> 5> FacetRangeProcessor seems like it needs its own class
> 6> FacetRequestSorted seems like it needs its own class
> 7> FacetModule
> So what I'd like going forward is to agree on some guidelines to resolve 
> whether to move a class to its own file or make it nested (probably static). 
> Not hard-and-fast rules, just something to cut down on the rework due to 
> objections.
> And what about backporting to 8x? My suggestion is to backport what's 
> easy/doesn't break back-compat in order to make keeping the two branches in 
> sync.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-11934) Visit Solr logging, it's too noisy.

[Joel Bernstein (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-11934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105777#comment-17105777
 ] 

Joel Bernstein commented on SOLR-11934:
---

Just took a look and seems fine. The postlogs tool will still pickup new 
searchers as it was looking at the new searcher line that you left in. 

One thing that would be nice would be to tie new searchers to a collection as 
well as the core. Right now when doing log analytics you can't include 
newSearcher events with collection level reports. 

> Visit Solr logging, it's too noisy.
> ---
>
> Key: SOLR-11934
> URL: https://issues.apache.org/jira/browse/SOLR-11934
> Project: Solr
>  Issue Type: Improvement
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Fix For: 8.6
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I think we have way too much INFO level logging. Or, perhaps more correctly, 
> Solr logging needs to be examined and messages logged at an appropriate level.
> We log every update at an INFO level for instance. But I think we log LIR at 
> INFO as well. As a sysadmin I don't care to have my logs polluted with a 
> message for every update, but if I'm trying to keep my system healthy I want 
> to see LIR messages and try to understand why.
> Plus, in large installations logging at INFO level is creating a _LOT_ of 
> files.
> What I want to discuss on this JIRA is
> 1> What kinds of messages do we want log at WARN, INFO, DEBUG, and TRACE 
> levels?
> 2> Who's the audience at each level? For a running system that's functioning, 
> sysops folks would really like WARN messages that mean something need 
> attention for instance. If I'm troubleshooting should I turn on INFO? DEBUG? 
> TRACE?
> So let's say we get some kind of agreement as to the above. Then I propose 
> three things
> 1> Someone (and probably me but all help gratefully accepted) needs to go 
> through our logging and assign appropriate levels. This will take quite a 
> while, I intend to work on it in small chunks.
> 2> Actually answer whether unnecessary objects are created when something 
> like log.info("whatever {}", someObjectOrMethodCall); is invoked. Is this 
> independent on the logging implementation used? The SLF4J and log4j seem a 
> bit contradictory.
> 3> Maybe regularize log, logger, LOG as variable names, but that's a nit.
> As a tactical approach, I suggest we tag each LoggerFactory.getLogger in 
> files we work on with //SOLR-(whatever number is assigned when I create 
> this). We can remove them all later, but since I expect to approach this 
> piecemeal it'd be nice to keep track of which files have been done already.
> Finally, I really really really don't want to do this all at once. There are 
> 5-6 thousand log messages. Even at 1,000 a week that's 6 weeks, even starting 
> now it would probably span the 7.3 release.
> This will probably be an umbrella issue so we can keep all the commits 
> straight and people can volunteer to "fix the files in core" as a separate 
> piece of work (hint).
> There are several existing JIRAs about logging in general, let's link them in 
> here as well.
> Let the discussion begin!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-11934) Visit Solr logging, it's too noisy.

[Joel Bernstein (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-11934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105777#comment-17105777
 ] 

Joel Bernstein edited comment on SOLR-11934 at 5/12/20, 10:01 PM:
--

Just took a look and seems fine. The postlogs tool will still pickup new 
searchers as it is looking at the new searcher line that you left in. 

One thing that would be nice would be to tie new searchers to a collection as 
well as the core. Right now when doing log analytics you can't include 
newSearcher events with collection level reports. 


was (Author: joel.bernstein):
Just took a look and seems fine. The postlogs tool will still pickup new 
searchers as it was looking at the new searcher line that you left in. 

One thing that would be nice would be to tie new searchers to a collection as 
well as the core. Right now when doing log analytics you can't include 
newSearcher events with collection level reports. 

> Visit Solr logging, it's too noisy.
> ---
>
> Key: SOLR-11934
> URL: https://issues.apache.org/jira/browse/SOLR-11934
> Project: Solr
>  Issue Type: Improvement
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Fix For: 8.6
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I think we have way too much INFO level logging. Or, perhaps more correctly, 
> Solr logging needs to be examined and messages logged at an appropriate level.
> We log every update at an INFO level for instance. But I think we log LIR at 
> INFO as well. As a sysadmin I don't care to have my logs polluted with a 
> message for every update, but if I'm trying to keep my system healthy I want 
> to see LIR messages and try to understand why.
> Plus, in large installations logging at INFO level is creating a _LOT_ of 
> files.
> What I want to discuss on this JIRA is
> 1> What kinds of messages do we want log at WARN, INFO, DEBUG, and TRACE 
> levels?
> 2> Who's the audience at each level? For a running system that's functioning, 
> sysops folks would really like WARN messages that mean something need 
> attention for instance. If I'm troubleshooting should I turn on INFO? DEBUG? 
> TRACE?
> So let's say we get some kind of agreement as to the above. Then I propose 
> three things
> 1> Someone (and probably me but all help gratefully accepted) needs to go 
> through our logging and assign appropriate levels. This will take quite a 
> while, I intend to work on it in small chunks.
> 2> Actually answer whether unnecessary objects are created when something 
> like log.info("whatever {}", someObjectOrMethodCall); is invoked. Is this 
> independent on the logging implementation used? The SLF4J and log4j seem a 
> bit contradictory.
> 3> Maybe regularize log, logger, LOG as variable names, but that's a nit.
> As a tactical approach, I suggest we tag each LoggerFactory.getLogger in 
> files we work on with //SOLR-(whatever number is assigned when I create 
> this). We can remove them all later, but since I expect to approach this 
> piecemeal it'd be nice to keep track of which files have been done already.
> Finally, I really really really don't want to do this all at once. There are 
> 5-6 thousand log messages. Even at 1,000 a week that's 6 weeks, even starting 
> now it would probably span the 7.3 release.
> This will probably be an umbrella issue so we can keep all the commits 
> straight and people can volunteer to "fix the files in core" as a separate 
> piece of work (hint).
> There are several existing JIRAs about logging in general, let's link them in 
> here as well.
> Let the discussion begin!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14105) Http2SolrClient SSL not working in branch_8x

[Tim (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105784#comment-17105784
 ] 

Tim commented on SOLR-14105:


The issue seems to have to do with 
[https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/handler/component/HttpShardHandlerFactory.java#L316]
  which in turn uses uses client jetty's SslContextFactory.Client inside 
Http2SolrClient, and latest jetty SslContextFactory.Client no longer seems to 
allow using multi domain certs. Perhaps it should be replaced with 
SslContextFactory.Server instead?

> Http2SolrClient SSL not working in branch_8x
> 
>
> Key: SOLR-14105
> URL: https://issues.apache.org/jira/browse/SOLR-14105
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 8.5
>Reporter: Jan Høydahl
>Assignee: Kevin Risden
>Priority: Major
> Attachments: SOLR-14105.patch
>
>
> In branch_8x we upgraded to Jetty 9.4.24. This causes the following 
> exceptions when attempting to start server with SSL:
> {noformat}
> 2019-12-17 14:46:16.646 ERROR (main) [   ] o.a.s.c.SolrCore 
> null:org.apache.solr.common.SolrException: Error instantiating 
> shardHandlerFactory class [HttpShardHandlerFactory]: 
> java.lang.UnsupportedOperationException: X509ExtendedKeyManager only 
> supported on Server
>   at 
> org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:56)
>   at org.apache.solr.core.CoreContainer.load(CoreContainer.java:633)
> ...
> Caused by: java.lang.RuntimeException: 
> java.lang.UnsupportedOperationException: X509ExtendedKeyManager only 
> supported on Server
>   at 
> org.apache.solr.client.solrj.impl.Http2SolrClient.createHttpClient(Http2SolrClient.java:224)
>   at 
> org.apache.solr.client.solrj.impl.Http2SolrClient.(Http2SolrClient.java:154)
>   at 
> org.apache.solr.client.solrj.impl.Http2SolrClient$Builder.build(Http2SolrClient.java:833)
>   at 
> org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:321)
>   at 
> org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:51)
>   ... 50 more
> Caused by: java.lang.UnsupportedOperationException: X509ExtendedKeyManager 
> only supported on Server
>   at 
> org.eclipse.jetty.util.ssl.SslContextFactory.newSniX509ExtendedKeyManager(SslContextFactory.java:1273)
>   at 
> org.eclipse.jetty.util.ssl.SslContextFactory.getKeyManagers(SslContextFactory.java:1255)
>   at 
> org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:374)
>   at 
> org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:245)
>  {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (SOLR-14476) Add percentiles aggregation to stats, facet, facet2D and timeseries Streaming Expressions

[Joel Bernstein (Jira)]

Joel Bernstein created SOLR-14476:
-

 Summary: Add percentiles aggregation to stats, facet, facet2D and 
timeseries Streaming Expressions
 Key: SOLR-14476
 URL: https://issues.apache.org/jira/browse/SOLR-14476
 Project: Solr
  Issue Type: New Feature
  Security Level: Public (Default Security Level. Issues are Public)
  Components: streaming expressions
Reporter: Joel Bernstein


This ticket will add the *per* (percentile) aggregation to the stats, facet, 
facet2D and timeseries Streaming Expression. Syntax:

 
{code:java}
facet(logs, buckets="collection_s", per(qtime_i, 50)) {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-14476) Add percentiles aggregation to stats, facet, facet2D and timeseries Streaming Expressions

[Joel Bernstein (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-14476:
--
Description: 
This ticket will add the *per* (percentile) aggregation to the stats, facet, 
facet2D and timeseries Streaming Expression. Syntax:

 
{code:java}
facet(logs, buckets="collection_s", per(qtime_i, 50)) {code}
 

In a separate ticket percentile aggregations will be added to *rollup*, 
*hashRollup* and *nodes* Streaming Expressions.

  was:
This ticket will add the *per* (percentile) aggregation to the stats, facet, 
facet2D and timeseries Streaming Expression. Syntax:

 
{code:java}
facet(logs, buckets="collection_s", per(qtime_i, 50)) {code}


> Add percentiles aggregation to stats, facet, facet2D and timeseries Streaming 
> Expressions
> -
>
> Key: SOLR-14476
> URL: https://issues.apache.org/jira/browse/SOLR-14476
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>Priority: Major
>
> This ticket will add the *per* (percentile) aggregation to the stats, facet, 
> facet2D and timeseries Streaming Expression. Syntax:
>  
> {code:java}
> facet(logs, buckets="collection_s", per(qtime_i, 50)) {code}
>  
> In a separate ticket percentile aggregations will be added to *rollup*, 
> *hashRollup* and *nodes* Streaming Expressions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-14476) Add percentiles aggregation to stats, facet, facet2D and timeseries Streaming Expressions

[Joel Bernstein (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-14476:
--
Description: 
This ticket will add the *per* (percentile) aggregation to the *stats*, 
*facet*, *facet2D* and *timeseries* Streaming Expression. Syntax:

 
{code:java}
facet(logs, buckets="collection_s", per(qtime_i, 50)) {code}
 

In a separate ticket percentile aggregations will be added to *rollup*, 
*hashRollup* and *nodes* Streaming Expressions.

  was:
This ticket will add the *per* (percentile) aggregation to the stats, facet, 
facet2D and timeseries Streaming Expression. Syntax:

 
{code:java}
facet(logs, buckets="collection_s", per(qtime_i, 50)) {code}
 

In a separate ticket percentile aggregations will be added to *rollup*, 
*hashRollup* and *nodes* Streaming Expressions.


> Add percentiles aggregation to stats, facet, facet2D and timeseries Streaming 
> Expressions
> -
>
> Key: SOLR-14476
> URL: https://issues.apache.org/jira/browse/SOLR-14476
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>Priority: Major
>
> This ticket will add the *per* (percentile) aggregation to the *stats*, 
> *facet*, *facet2D* and *timeseries* Streaming Expression. Syntax:
>  
> {code:java}
> facet(logs, buckets="collection_s", per(qtime_i, 50)) {code}
>  
> In a separate ticket percentile aggregations will be added to *rollup*, 
> *hashRollup* and *nodes* Streaming Expressions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13132) Improve JSON "terms" facet performance when sorted by relatedness

[Chris M. Hostetter (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105821#comment-17105821
 ] 

Chris M. Hostetter commented on SOLR-13132:
---

Michael: I think we should pull the SOLR-13108 related changes (not specific to 
sweeping) out of the PR/branch and tackle those seperately, along with their 
own doc updates and some specific whitebox tests of the hueristic logic.  FWIW: 
I've also added some nocommits to that code with some initial 
thoughts/questions that we should preserve when moving it to it's own 
branch/PR/patch.



> Improve JSON "terms" facet performance when sorted by relatedness 
> --
>
> Key: SOLR-13132
> URL: https://issues.apache.org/jira/browse/SOLR-13132
> Project: Solr
>  Issue Type: Improvement
>  Components: Facet Module
>Affects Versions: 7.4, master (9.0)
>Reporter: Michael Gibney
>Priority: Major
> Attachments: SOLR-13132-with-cache-01.patch, 
> SOLR-13132-with-cache.patch, SOLR-13132.patch, SOLR-13132_testSweep.patch
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> When sorting buckets by {{relatedness}}, JSON "terms" facet must calculate 
> {{relatedness}} for every term. 
> The current implementation uses a standard uninverted approach (either 
> {{docValues}} or {{UnInvertedField}}) to get facet counts over the domain 
> base docSet, and then uses that initial pass as a pre-filter for a 
> second-pass, inverted approach of fetching docSets for each relevant term 
> (i.e., {{count > minCount}}?) and calculating intersection size of those sets 
> with the domain base docSet.
> Over high-cardinality fields, the overhead of per-term docSet creation and 
> set intersection operations increases request latency to the point where 
> relatedness sort may not be usable in practice (for my use case, even after 
> applying the patch for SOLR-13108, for a field with ~220k unique terms per 
> core, QTime for high-cardinality domain docSets were, e.g.: cardinality 
> 1816684=9000ms, cardinality 5032902=18000ms).
> The attached patch brings the above example QTimes down to a manageable 
> ~300ms and ~250ms respectively. The approach calculates uninverted facet 
> counts over domain base, foreground, and background docSets in parallel in a 
> single pass. This allows us to take advantage of the efficiencies built into 
> the standard uninverted {{FacetFieldProcessorByArray[DV|UIF]}}), and avoids 
> the per-term docSet creation and set intersection overhead.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0

[Ilan Ginzburg (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105826#comment-17105826
 ] 

Ilan Ginzburg commented on SOLR-12823:
--

[~janhoy], [~ab], the autoscaling simulation framework 
(org.apache.solr.cloud.autoscaling.sim) blocks clean removal of stateFormat=1 
collections and associated management of /clusterstate.json in the Overseer...

It currently only supports the old state format.

Having it create "stateFormat=2" collections (quotes because I'm removing the 
stateFormat concept altogether) having each its own state.json shouldn't be a 
big deal, but I'm not familiar with how it's used to understand the 
consequences of the absence of a single state ZK file.


Also, the concept of a global config "version" (mapping to a znode version) no 
longer exists with per collection state.json and unfortunately the sim code 
seems to rely on it quite a bit.
I need some guidance on the best way to deal with this here, thanks.

> remove clusterstate.json in Lucene/Solr 8.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
>  Issue Type: Task
>Reporter: Varun Thacker
>Priority: Major
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove 
> that in 8.0
> It stays empty unless you explicitly ask to create the collection with the 
> old "stateFormat" and there is no reason for one to create a collection with 
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (SOLR-14477) relatedness() values can be wrong when using 'prefix'

[Chris M. Hostetter (Jira)]

Chris M. Hostetter created SOLR-14477:
-

 Summary: relatedness() values can be wrong when using 'prefix'
 Key: SOLR-14477
 URL: https://issues.apache.org/jira/browse/SOLR-14477
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Chris M. Hostetter


Another {{relatedness()}} bug found in json facet's while working on increased 
test coverage for SOLR-13132.
if the {{prefix}} option is used when doing a terms facet, then the 
{{relatedess()}} calculations can be wrong in some situations -- most notably 
when using {{limit:-1}} but i'm pretty sure the bug also impacts the code paths 
where the (first) {{sort}} (or {{prelim_sort}} is computed against the  
{{relatedness()}} values.

Real world impacts of this bug should be relatively low since i can't really 
think of any practical usecases for using  {{relatedness()}} in conjunction 
with {{prefix}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-14477) relatedness() values can be wrong when using 'prefix'

[Chris M. Hostetter (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-14477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris M. Hostetter updated SOLR-14477:
--
Attachment: SOLR-14477.patch
  Assignee: Chris M. Hostetter
Status: Open  (was: Open)

patch with test case and fix to make test pass (but i'm not convinced there 
aren't still problematic code paths not covered by these test changes)

> relatedness() values can be wrong when using 'prefix'
> -
>
> Key: SOLR-14477
> URL: https://issues.apache.org/jira/browse/SOLR-14477
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Chris M. Hostetter
>Assignee: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-14477.patch
>
>
> Another {{relatedness()}} bug found in json facet's while working on 
> increased test coverage for SOLR-13132.
> if the {{prefix}} option is used when doing a terms facet, then the 
> {{relatedess()}} calculations can be wrong in some situations -- most notably 
> when using {{limit:-1}} but i'm pretty sure the bug also impacts the code 
> paths where the (first) {{sort}} (or {{prelim_sort}} is computed against the  
> {{relatedness()}} values.
> Real world impacts of this bug should be relatively low since i can't really 
> think of any practical usecases for using  {{relatedness()}} in conjunction 
> with {{prefix}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14105) Http2SolrClient SSL not working in branch_8x

[Tim (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105849#comment-17105849
 ] 

Tim commented on SOLR-14105:


as requested by Jetty developers, opened issue 
https://github.com/eclipse/jetty.project/issues/4874

> Http2SolrClient SSL not working in branch_8x
> 
>
> Key: SOLR-14105
> URL: https://issues.apache.org/jira/browse/SOLR-14105
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 8.5
>Reporter: Jan Høydahl
>Assignee: Kevin Risden
>Priority: Major
> Attachments: SOLR-14105.patch
>
>
> In branch_8x we upgraded to Jetty 9.4.24. This causes the following 
> exceptions when attempting to start server with SSL:
> {noformat}
> 2019-12-17 14:46:16.646 ERROR (main) [   ] o.a.s.c.SolrCore 
> null:org.apache.solr.common.SolrException: Error instantiating 
> shardHandlerFactory class [HttpShardHandlerFactory]: 
> java.lang.UnsupportedOperationException: X509ExtendedKeyManager only 
> supported on Server
>   at 
> org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:56)
>   at org.apache.solr.core.CoreContainer.load(CoreContainer.java:633)
> ...
> Caused by: java.lang.RuntimeException: 
> java.lang.UnsupportedOperationException: X509ExtendedKeyManager only 
> supported on Server
>   at 
> org.apache.solr.client.solrj.impl.Http2SolrClient.createHttpClient(Http2SolrClient.java:224)
>   at 
> org.apache.solr.client.solrj.impl.Http2SolrClient.(Http2SolrClient.java:154)
>   at 
> org.apache.solr.client.solrj.impl.Http2SolrClient$Builder.build(Http2SolrClient.java:833)
>   at 
> org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:321)
>   at 
> org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:51)
>   ... 50 more
> Caused by: java.lang.UnsupportedOperationException: X509ExtendedKeyManager 
> only supported on Server
>   at 
> org.eclipse.jetty.util.ssl.SslContextFactory.newSniX509ExtendedKeyManager(SslContextFactory.java:1273)
>   at 
> org.eclipse.jetty.util.ssl.SslContextFactory.getKeyManagers(SslContextFactory.java:1255)
>   at 
> org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:374)
>   at 
> org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:245)
>  {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13132) Improve JSON "terms" facet performance when sorted by relatedness

[Michael Gibney (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105856#comment-17105856
 ] 

Michael Gibney commented on SOLR-13132:
---

Hey Hoss, those lastest nocommit comments all make sense to me. I've been 
trying to wrap up some of the allBuckets stuff related to SOLR-14467. Fixing 
that issue would make the tests for this issue more robust (and avoid having to 
work around allBuckets issues), and I think it's close at this point. Just 
wrestling with some ordering issues, and blacklisting the {{enum/stream}} 
{{FacetMethod}} when {{allBuckets=true}} (it seems that 
{{FacetFieldProcessorByEnumTermsStream}} doesn't support {{allBuckets}}?). Not 
100% confident on that, still gathering my thoughts ...

> Improve JSON "terms" facet performance when sorted by relatedness 
> --
>
> Key: SOLR-13132
> URL: https://issues.apache.org/jira/browse/SOLR-13132
> Project: Solr
>  Issue Type: Improvement
>  Components: Facet Module
>Affects Versions: 7.4, master (9.0)
>Reporter: Michael Gibney
>Priority: Major
> Attachments: SOLR-13132-with-cache-01.patch, 
> SOLR-13132-with-cache.patch, SOLR-13132.patch, SOLR-13132_testSweep.patch
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> When sorting buckets by {{relatedness}}, JSON "terms" facet must calculate 
> {{relatedness}} for every term. 
> The current implementation uses a standard uninverted approach (either 
> {{docValues}} or {{UnInvertedField}}) to get facet counts over the domain 
> base docSet, and then uses that initial pass as a pre-filter for a 
> second-pass, inverted approach of fetching docSets for each relevant term 
> (i.e., {{count > minCount}}?) and calculating intersection size of those sets 
> with the domain base docSet.
> Over high-cardinality fields, the overhead of per-term docSet creation and 
> set intersection operations increases request latency to the point where 
> relatedness sort may not be usable in practice (for my use case, even after 
> applying the patch for SOLR-13108, for a field with ~220k unique terms per 
> core, QTime for high-cardinality domain docSets were, e.g.: cardinality 
> 1816684=9000ms, cardinality 5032902=18000ms).
> The attached patch brings the above example QTimes down to a manageable 
> ~300ms and ~250ms respectively. The approach calculates uninverted facet 
> counts over domain base, foreground, and background docSets in parallel in a 
> single pass. This allows us to take advantage of the efficiencies built into 
> the standard uninverted {{FacetFieldProcessorByArray[DV|UIF]}}), and avoids 
> the per-term docSet creation and set intersection overhead.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-11489) Create collections as _foo and auto-create an alias foo->_foo

[David Smiley (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105893#comment-17105893
 ] 

David Smiley commented on SOLR-11489:
-

Maybe obsoleted by SOLR-13262?  Not the same but reduces the pain point here.

> Create collections as _foo and auto-create an alias foo->_foo
> -
>
> Key: SOLR-11489
> URL: https://issues.apache.org/jira/browse/SOLR-11489
> Project: Solr
>  Issue Type: Improvement
>Reporter: Erick Erickson
>Priority: Major
>
> Spin-off from SOLR-11488. Currently if a collection and an alias have the 
> same name we don't test how they're resolved. So an innocent change to the 
> code could easily change the behavior (what happens when you have a 
> collection old, and an alias old->new and delete collection "old"? Have we 
> even defined what _should_ happen?).
> See the discussion at SOLR-11488.
> An alternative proposal to SOLR-11488 (thanks Varun for pointing it out) is 
> when creating a collection "foo", actually name it _foo and create an alias 
> foo->_foo. Also don't allow the user to create an alias that begins with an 
> underscore (and maybe the same for collections? An alias _foo->__foo starts 
> to get weird).
> The result would be we'd never have a collection and an alias with the same 
> name, which would go a long way to prevent issues going forward.
> This requires we consider the name in state.json to be an implementation 
> detail, but the user won't notice. Potential here for the list of aliases to 
> be quite large.
> Of course the user could still reference the collection directly as _foo if 
> they insisted.
> Establishing this JIRA for discussion of the alternatives.
> Assigning to myself to keep it from getting lost, feel free to take it over 
> if you'd like.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-11934) Visit Solr logging, it's too noisy.

[David Smiley (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-11934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105900#comment-17105900
 ] 

David Smiley commented on SOLR-11934:
-

{quote}tie new searchers to a collection as well as the core
{quote}
Can you please elaborate on that?  Logs have MDC with collection, core etc.  
That said there are some bugs where some logs are missing MDC but I've been 
addressing that lately; I'm nearly done.

> Visit Solr logging, it's too noisy.
> ---
>
> Key: SOLR-11934
> URL: https://issues.apache.org/jira/browse/SOLR-11934
> Project: Solr
>  Issue Type: Improvement
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Fix For: 8.6
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I think we have way too much INFO level logging. Or, perhaps more correctly, 
> Solr logging needs to be examined and messages logged at an appropriate level.
> We log every update at an INFO level for instance. But I think we log LIR at 
> INFO as well. As a sysadmin I don't care to have my logs polluted with a 
> message for every update, but if I'm trying to keep my system healthy I want 
> to see LIR messages and try to understand why.
> Plus, in large installations logging at INFO level is creating a _LOT_ of 
> files.
> What I want to discuss on this JIRA is
> 1> What kinds of messages do we want log at WARN, INFO, DEBUG, and TRACE 
> levels?
> 2> Who's the audience at each level? For a running system that's functioning, 
> sysops folks would really like WARN messages that mean something need 
> attention for instance. If I'm troubleshooting should I turn on INFO? DEBUG? 
> TRACE?
> So let's say we get some kind of agreement as to the above. Then I propose 
> three things
> 1> Someone (and probably me but all help gratefully accepted) needs to go 
> through our logging and assign appropriate levels. This will take quite a 
> while, I intend to work on it in small chunks.
> 2> Actually answer whether unnecessary objects are created when something 
> like log.info("whatever {}", someObjectOrMethodCall); is invoked. Is this 
> independent on the logging implementation used? The SLF4J and log4j seem a 
> bit contradictory.
> 3> Maybe regularize log, logger, LOG as variable names, but that's a nit.
> As a tactical approach, I suggest we tag each LoggerFactory.getLogger in 
> files we work on with //SOLR-(whatever number is assigned when I create 
> this). We can remove them all later, but since I expect to approach this 
> piecemeal it'd be nice to keep track of which files have been done already.
> Finally, I really really really don't want to do this all at once. There are 
> 5-6 thousand log messages. Even at 1,000 a week that's 6 weeks, even starting 
> now it would probably span the 7.3 release.
> This will probably be an umbrella issue so we can keep all the commits 
> straight and people can volunteer to "fix the files in core" as a separate 
> piece of work (hint).
> There are several existing JIRAs about logging in general, let's link them in 
> here as well.
> Let the discussion begin!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-11489) Create collections as _foo and auto-create an alias foo->_foo

[Andrzej Bialecki (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105997#comment-17105997
 ] 

Andrzej Bialecki commented on SOLR-11489:
-

[~dsmiley] Collection RENAME works precisely because we allow aliases to have 
the same names as collections. I looked into implementing a true RENAME and 
it's so messy and disruptive that this is the lesser evil.

> Create collections as _foo and auto-create an alias foo->_foo
> -
>
> Key: SOLR-11489
> URL: https://issues.apache.org/jira/browse/SOLR-11489
> Project: Solr
>  Issue Type: Improvement
>Reporter: Erick Erickson
>Priority: Major
>
> Spin-off from SOLR-11488. Currently if a collection and an alias have the 
> same name we don't test how they're resolved. So an innocent change to the 
> code could easily change the behavior (what happens when you have a 
> collection old, and an alias old->new and delete collection "old"? Have we 
> even defined what _should_ happen?).
> See the discussion at SOLR-11488.
> An alternative proposal to SOLR-11488 (thanks Varun for pointing it out) is 
> when creating a collection "foo", actually name it _foo and create an alias 
> foo->_foo. Also don't allow the user to create an alias that begins with an 
> underscore (and maybe the same for collections? An alias _foo->__foo starts 
> to get weird).
> The result would be we'd never have a collection and an alias with the same 
> name, which would go a long way to prevent issues going forward.
> This requires we consider the name in state.json to be an implementation 
> detail, but the user won't notice. Potential here for the list of aliases to 
> be quite large.
> Of course the user could still reference the collection directly as _foo if 
> they insisted.
> Establishing this JIRA for discussion of the alternatives.
> Assigning to myself to keep it from getting lost, feel free to take it over 
> if you'd like.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13289) Support for BlockMax WAND

[Tomas Eduardo Fernandez Lobbe (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17106021#comment-17106021
 ] 

Tomas Eduardo Fernandez Lobbe commented on SOLR-13289:
--

[~dsmiley], I'm still debating myself on whether this is the best solution for 
the bug you spotted: 
https://github.com/tflobbe/lucene-solr-1/commit/9beaf4131e53f04651b9e6d9afba60192f348c73,
 please take a look. 
bq. I suggest minExactFound instead because it has the word "Found" which is 
perhaps the most significant portion of "numFound".
I understand it's because of {{numFound}}, but It just sounds very weird to me 
that the input parameter for the query carries the word "found".

> Support for BlockMax WAND
> -
>
> Key: SOLR-13289
> URL: https://issues.apache.org/jira/browse/SOLR-13289
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ishan Chattopadhyaya
>Assignee: Tomas Eduardo Fernandez Lobbe
>Priority: Major
> Attachments: SOLR-13289.patch, SOLR-13289.patch
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> LUCENE-8135 introduced BlockMax WAND as a major speed improvement. Need to 
> expose this via Solr. When enabled, the numFound returned will not be exact.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org