date:20200304

janhoy commented on issue #1309: SOLR-13942: /api/cluster/zk/* to fetch raw ZK 
data 
URL: https://github.com/apache/lucene-solr/pull/1309#issuecomment-594389360
 
 
   Wrong PR linked in https://issues.apache.org/jira/browse/SOLR-13942, this 
should make it show up


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051016#comment-17051016
 ] 

Jan Høydahl commented on SOLR-13942:


Linked up [GitHub Pull Request 
#1309|https://github.com/apache/lucene-solr/pull/1309] which was created March 
3rd 2010 11:19 UTC with an empty PR description, commit hash 
4498b9c360609abd813f57532189be16a6b9d823 and commit message "initial commit 
with test" and then immediately merged to master. The lack of jira-reference in 
commit message is probably the reason why gitbot did not catch it for comment.

I'm concerned for tech reasons but more so for community reasons. We have CTR 
but not commit-then-forgive. Code must both be high quality an undisputed for 
CTR to work.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051038#comment-17051038
 ] 

Ishan Chattopadhyaya commented on SOLR-13942:
-

bq. The lack of jira-reference in commit message is probably the reason why 
gitbot did not catch it for comment.
You are undermining your credibility by levelling unfounded allegations. Please 
stop and apologize for such irresponsible speculation.

bq. We have CTR but not commit-then-forgive. Code must both be high quality an 
undisputed for CTR to work.
What are you even talking about, "commit-then-forgive"?

Commit: Code was merged to master and branch_8x yesterday. I resolved the issue 
yesterday.
Review: We all are free to review now and suggest changes.

Why is this not CTR in your opinion?

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9257) FSTLoadMode should not be BlockTree specific as it is used more generally in index package

2020-03-04 Thread Bruno Roustant (Jira)



[ 
https://issues.apache.org/jira/browse/LUCENE-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051043#comment-17051043
 ] 

Bruno Roustant commented on LUCENE-9257:


Removing FSTLoadMode enum is tempting, but there are several cases handled with 
FSTLoadMode.AUTO:
- force on-heap for ID field.
- force on-heap for SegmentReadState opened from a writer.
- off-heap only for IndexInput instanceof ByteBufferIndexInput.

So your benchmarks showed very small overhead for indexing. What about ID 
field? Also are there important use-cases with IndexInput not instanceof 
ByteBufferIndexInput?

I'm not comfortable with always loading the FST off-heap.
But maybe remove the FSTLoadMode enum and always let the logic behind 
FSTLoadMode.AUTO?
Or replace FSTLoadMode by a simple boolean forceOnHeap (otherwise it is auto 
determined with the above logic)?

> FSTLoadMode should not be BlockTree specific as it is used more generally in 
> index package
> --
>
> Key: LUCENE-9257
> URL: https://issues.apache.org/jira/browse/LUCENE-9257
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Bruno Roustant
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> FSTLoadMode and its associate attribute key (static String) are currently 
> defined in BlockTreeTermsReader, but they are actually used outside of 
> BlockTree in the general "index" package.
> CheckIndex and ReadersAndUpdates are using these enum and attribute key to 
> drive the FST load mode through the SegmentReader which is not specific to a 
> postings format. They have an unnecessary dependency to BlockTreeTermsReader.
> We could move FSTLoadMode out of BlockTreeTermsReader, to make it a public 
> enum of the "index" package. That way CheckIndex and ReadersAndUpdates do not 
> import anymore BlockTreeTermsReader.
> This would also allow other postings formats to use the same enum (e.g. 
> LUCENE-9254)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-9257) FSTLoadMode should not be BlockTree specific as it is used more generally in index package

2020-03-04 Thread Bruno Roustant (Jira)



[ 
https://issues.apache.org/jira/browse/LUCENE-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051043#comment-17051043
 ] 

Bruno Roustant edited comment on LUCENE-9257 at 3/4/20, 9:36 AM:
-

Removing FSTLoadMode enum is tempting, but there are several cases handled with 
FSTLoadMode.AUTO:
 - force on-heap for ID field.
 - force on-heap for SegmentReadState opened from a writer.
 - off-heap only for IndexInput instanceof ByteBufferIndexInput.

So your benchmarks showed very small overhead for indexing. What about ID 
field? Also are there important use-cases with IndexInput not instanceof 
ByteBufferIndexInput?

I'm not comfortable with always loading the FST off-heap.
 But maybe remove the FSTLoadMode enum and always let the logic behind 
FSTLoadMode.AUTO?
 Or replace FSTLoadMode by a simple boolean fstOnHeap (otherwise it is auto 
determined with the above logic)?


was (Author: broustant):
Removing FSTLoadMode enum is tempting, but there are several cases handled with 
FSTLoadMode.AUTO:
- force on-heap for ID field.
- force on-heap for SegmentReadState opened from a writer.
- off-heap only for IndexInput instanceof ByteBufferIndexInput.

So your benchmarks showed very small overhead for indexing. What about ID 
field? Also are there important use-cases with IndexInput not instanceof 
ByteBufferIndexInput?

I'm not comfortable with always loading the FST off-heap.
But maybe remove the FSTLoadMode enum and always let the logic behind 
FSTLoadMode.AUTO?
Or replace FSTLoadMode by a simple boolean forceOnHeap (otherwise it is auto 
determined with the above logic)?

> FSTLoadMode should not be BlockTree specific as it is used more generally in 
> index package
> --
>
> Key: LUCENE-9257
> URL: https://issues.apache.org/jira/browse/LUCENE-9257
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Bruno Roustant
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> FSTLoadMode and its associate attribute key (static String) are currently 
> defined in BlockTreeTermsReader, but they are actually used outside of 
> BlockTree in the general "index" package.
> CheckIndex and ReadersAndUpdates are using these enum and attribute key to 
> drive the FST load mode through the SegmentReader which is not specific to a 
> postings format. They have an unnecessary dependency to BlockTreeTermsReader.
> We could move FSTLoadMode out of BlockTreeTermsReader, to make it a public 
> enum of the "index" package. That way CheckIndex and ReadersAndUpdates do not 
> import anymore BlockTreeTermsReader.
> This would also allow other postings formats to use the same enum (e.g. 
> LUCENE-9254)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051038#comment-17051038
 ] 

Ishan Chattopadhyaya edited comment on SOLR-13942 at 3/4/20, 11:49 AM:
---

bq. The lack of jira-reference in commit message is probably the reason why 
gitbot did not catch it for comment.
You are undermining your credibility by levelling unfounded allegations. Please 
stop and apologize for such irresponsible speculation. [0]

bq. We have CTR but not commit-then-forgive. Code must both be high quality an 
undisputed for CTR to work.
What are you even talking about, "commit-then-forgive"?

Commit: Code was merged to master and branch_8x yesterday. I resolved the issue 
yesterday.
Review: We all are free to review now and suggest changes.

Why is this not CTR in your opinion?

[0] - 
https://github.com/apache/lucene-solr/commit/bc6fa3b65060b17a88013a0378f4a9d285067d82


was (Author: ichattopadhyaya):
bq. The lack of jira-reference in commit message is probably the reason why 
gitbot did not catch it for comment.
You are undermining your credibility by levelling unfounded allegations. Please 
stop and apologize for such irresponsible speculation.

bq. We have CTR but not commit-then-forgive. Code must both be high quality an 
undisputed for CTR to work.
What are you even talking about, "commit-then-forgive"?

Commit: Code was merged to master and branch_8x yesterday. I resolved the issue 
yesterday.
Review: We all are free to review now and suggest changes.

Why is this not CTR in your opinion?

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051164#comment-17051164
 ] 

Jan Høydahl commented on SOLR-13942:


{quote}You are undermining your credibility by levelling unfounded allegations. 
Please stop and apologize for such irresponsible speculation.[0]
{quote}
No need to draw your guns, my comment was primarily to record the commit in 
this JIRA, since gitbot failed to. My theory of why gitbot failed to comment 
was obviously wrong, given that the *merge* message is perfect, with issue ID 
and all. I do apologize for quoting the commit message at all, I can see how 
that just adds fuel to the fire, when it really is a non.issue. So I guess 
gitbot must have had a hickup then!
{quote}Why is this not CTR in your opinion?
{quote}
This feature received push-back and was still disputed at the time of merge. 
Had nobody voiced concerns then it would be CTR. There was a comment here "I 
would like to merge it soon" on the evening March 2nd, I voiced concern right 
after and the merge happened noon March 3rd. That is a strange timeline, it is 
not consensus, not even even lazy consensus. The fact that this happens a day 
or two before the 8.5 RC0 does not make it any better, that's another reason 
why it is important to revert now.

I'm flagging this as Blocker for the 8.5 release for now.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



 [ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated SOLR-13942:
---
Priority: Blocker  (was: Major)

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Resolved] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



 [ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ishan Chattopadhyaya resolved SOLR-13942.
-
Resolution: Fixed

All technical concerns have been answered. There was +1 from me after Noble 
expressed intention to merge. The work has been done, and there are no vetoes. 
I see no reason to keep this unresolved. Thanks [~noble.paul].

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-9909) Nuke one of DefaultSolrThreadFactory and SolrjNamedThreadFactory

2020-03-04 Thread Andras Salamon (Jira)



 [ 
https://issues.apache.org/jira/browse/SOLR-9909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Salamon updated SOLR-9909:
-
Attachment: SOLR-9909-02.patch

> Nuke one of DefaultSolrThreadFactory and SolrjNamedThreadFactory
> 
>
> Key: SOLR-9909
> URL: https://issues.apache.org/jira/browse/SOLR-9909
> Project: Solr
>  Issue Type: Task
>Reporter: Shalin Shekhar Mangar
>Priority: Trivial
> Fix For: 6.7, 7.0
>
> Attachments: SOLR-9909-01.patch, SOLR-9909-02.patch
>
>
> DefaultSolrThreadFactory and SolrjNamedThreadFactory have exactly the same 
> code. Let's remove one of them.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051197#comment-17051197
 ] 

Jan Høydahl commented on SOLR-13942:


-1  Here is a veto, if it was not already clear from the "please revert" 
comments above

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051198#comment-17051198
 ] 

Ishan Chattopadhyaya commented on SOLR-13942:
-

Please state your technical reason for the veto.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051199#comment-17051199
 ] 

Jan Høydahl commented on SOLR-13942:


Part of this discussion is done in Slack #solr-dev as well: 
https://the-asf.slack.com/archives/CEKUCUNE9/p1583269433014300

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051198#comment-17051198
 ] 

Ishan Chattopadhyaya edited comment on SOLR-13942 at 3/4/20, 12:58 PM:
---

Please state your technical reason for the veto. Please don't veto because 
you're not convinced or there is no consensus etc. reasons. I think enough has 
been done to convince everyone.


was (Author: ichattopadhyaya):
Please state your technical reason for the veto.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (SOLR-14303) Deprecate /admin/zookeeper

Ishan Chattopadhyaya created SOLR-14303:
---

 Summary: Deprecate /admin/zookeeper
 Key: SOLR-14303
 URL: https://issues.apache.org/jira/browse/SOLR-14303
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Ishan Chattopadhyaya


This is a spinoff from SOLR-13942 to deprecate /admin/zookeeper and cutover the 
admin UI to the new API proposed there (/cluster/zk).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051207#comment-17051207
 ] 

Ishan Chattopadhyaya commented on SOLR-13942:
-

bq. Deprecate /admin/zookeeper, introduce a clean API, migrate UI to this new 
endpoint or a better alternative and remove /admin/zookeeper in 9.0
[~shalin], I've opened SOLR-14303 for the next steps, as per your proposal.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1303: LUCENE-9114: Improve ValueSourceScorer's Default Cost Implementation

dsmiley commented on a change in pull request #1303: LUCENE-9114: Improve 
ValueSourceScorer's Default Cost Implementation
URL: https://github.com/apache/lucene-solr/pull/1303#discussion_r387663374
 
 

 ##
 File path: 
lucene/queries/src/java/org/apache/lucene/queries/function/FunctionValues.java
 ##
 @@ -151,6 +164,11 @@ public ValueSourceScorer getScorer(Weight weight, 
LeafReaderContext readerContex
   public boolean matches(int doc) {
 return true;
   }
+  @Override
+  public float costEvaluationFunction() {
+// Match everything
 
 Review comment:
   Nitpick comment: the point isn't that we're matching everything, the point 
is that matches() merely returns a constant and is thus virtually free.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1303: LUCENE-9114: Improve ValueSourceScorer's Default Cost Implementation

dsmiley commented on a change in pull request #1303: LUCENE-9114: Improve 
ValueSourceScorer's Default Cost Implementation
URL: https://github.com/apache/lucene-solr/pull/1303#discussion_r387660244
 
 

 ##
 File path: 
lucene/queries/src/java/org/apache/lucene/queries/function/ValueSourceScorer.java
 ##
 @@ -94,4 +97,17 @@ public float getMaxScore(int upTo) throws IOException {
 return Float.POSITIVE_INFINITY;
   }
 
+  /**
+   * Cost evaluation function which defines the cost of access for the 
TwoPhaseIterator for this class
+   * This method should be overridden for specifying custom cost methods. Used 
by {@link TwoPhaseIterator#matchCost()}
+   * for the instance owned by this class
+   *
+   * @return cost of access
+   *
+   * @lucene.experimental
+   */
+  protected float costEvaluationFunction() {
 
 Review comment:
   Can we call this simply matchCost now since, after all, the TPI.matchCost 
just calls it now with no mutable override?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1303: LUCENE-9114: Improve ValueSourceScorer's Default Cost Implementation

dsmiley commented on a change in pull request #1303: LUCENE-9114: Improve 
ValueSourceScorer's Default Cost Implementation
URL: https://github.com/apache/lucene-solr/pull/1303#discussion_r387662567
 
 

 ##
 File path: 
lucene/queries/src/java/org/apache/lucene/queries/function/FunctionValues.java
 ##
 @@ -41,6 +41,9 @@
 //   want the Query carrying around big objects
 public abstract class FunctionValues {
 
+  // Default cost for FunctionValues -- ideally should be overriden by 
concrete implementations
+  public static final int DEFAULT_COST = 100;
 
 Review comment:
   This is fine with me but FWIW I wouldn't even bother defining it.  It has no 
value set aside like this; I doubt any user code would want to refer to it.  If 
we want to document what the default cost is, we should say so in cost()'s 
javadoc.  I know some devs like to make static constants for everything but IMO 
it's sometimes wasted ceremony.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051223#comment-17051223
 ] 

Jan Høydahl commented on SOLR-13942:


{quote}Please state your technical reason for the veto
{quote}
These are summarized nicely by [Jason's comment|#comment-17050254].
 * Where is the decision that all data in zk strictly public? What about 
security.json?
 * We have been trying to abstract away zk. So this API must be 
tagged/documented clearly as internal.
 * The proposal adds redundant API surface. Has there been done any analysis of 
whether it is possible to cut over Admin UI from /admin/zookeeper to 
/api/cluster/zk? The ZookeeperInfoHandler seems to do quite a lot more logic 
than just shuffling bits from ZK to the client.

It is easy to add but hard and time consuming to take the community 
responsibility to cut over to the new and get rid of old APIs, thus we add to 
our tech debt for every release.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Reopened] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



 [ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl reopened SOLR-13942:


> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8962) Can we merge small segments during refresh, for faster searching?

2020-03-04 Thread Michael Sokolov (Jira)



[ 
https://issues.apache.org/jira/browse/LUCENE-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051229#comment-17051229
 ] 

Michael Sokolov commented on LUCENE-8962:
-

I think I'll test and push [~msfroh]'s test fix since it seems to fail 
frequently, while we investigate this new failure. It looks as if we are not 
properly handling the case when an IOException is thrown at some inopportune 
time? 

> Can we merge small segments during refresh, for faster searching?
> -
>
> Key: LUCENE-8962
> URL: https://issues.apache.org/jira/browse/LUCENE-8962
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/index
>Reporter: Michael McCandless
>Priority: Major
> Fix For: 8.5
>
> Attachments: LUCENE-8962_demo.png
>
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>
> With near-real-time search we ask {{IndexWriter}} to write all in-memory 
> segments to disk and open an {{IndexReader}} to search them, and this is 
> typically a quick operation.
> However, when you use many threads for concurrent indexing, {{IndexWriter}} 
> will accumulate write many small segments during {{refresh}} and this then 
> adds search-time cost as searching must visit all of these tiny segments.
> The merge policy would normally quickly coalesce these small segments if 
> given a little time ... so, could we somehow improve {{IndexWriter'}}s 
> refresh to optionally kick off merge policy to merge segments below some 
> threshold before opening the near-real-time reader?  It'd be a bit tricky 
> because while we are waiting for merges, indexing may continue, and new 
> segments may be flushed, but those new segments shouldn't be included in the 
> point-in-time segments returned by refresh ...
> One could almost do this on top of Lucene today, with a custom merge policy, 
> and some hackity logic to have the merge policy target small segments just 
> written by refresh, but it's tricky to then open a near-real-time reader, 
> excluding newly flushed but including newly merged segments since the refresh 
> originally finished ...
> I'm not yet sure how best to solve this, so I wanted to open an issue for 
> discussion!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051232#comment-17051232
 ] 

Ishan Chattopadhyaya commented on SOLR-13942:
-

bq. Where is the decision that all data in zk strictly public? What about 
security.json?
All ZK data was public (read-only) till now anyway (via the older API). When 
there is a security.json, this endpoint can be protected using adequate 
authc/authz configurations. There is no change to our policy, nor is there a 
security vulnerability due to this issue.

bq.We have been trying to abstract away zk. So this API must be 
tagged/documented clearly as internal.
We've been trying to abstract away ZK, but not the data stored in ZK. It is 
useful for monitoring/debugging, in a spirit similar to how debug/explain works 
for a query. I agree this API should be tagged as experimental/internal etc.

bq.The proposal adds redundant API surface. Has there been done any 
analysis of whether it is possible to cut over Admin UI from /admin/zookeeper 
to /api/cluster/zk? The ZookeeperInfoHandler seems to do quite a lot more logic 
than just shuffling bits from ZK to the client.
ZookeeperInfoHandler does a lot of things that should ideally be done by the 
admin UI code. There is no technical reason why the UI won't be able to consume 
this new endpoint (of course, with necessary changes to the UI).

Now, please withdraw your veto and let us move forward.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



 [ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated SOLR-13942:
---
Component/s: v2 API

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



 [ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated SOLR-13942:
---
Issue Type: New Feature  (was: Bug)

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051232#comment-17051232
 ] 

Ishan Chattopadhyaya edited comment on SOLR-13942 at 3/4/20, 1:43 PM:
--

bq. Where is the decision that all data in zk strictly public? What about 
security.json?
All ZK data was public (read-only) till now anyway (via the older API). When 
there is a security.json, this endpoint can be protected using adequate 
authc/authz configurations. There is no change to our policy, nor is there a 
security vulnerability due to this issue.

bq.We have been trying to abstract away zk. So this API must be 
tagged/documented clearly as internal.
We've been trying to abstract away ZK, but not the data stored in ZK. It is 
useful for monitoring/debugging, in a spirit similar to how debug/explain works 
for a query. I agree this API should be tagged as experimental/internal etc.

bq.The proposal adds redundant API surface. Has there been done any 
analysis of whether it is possible to cut over Admin UI from /admin/zookeeper 
to /api/cluster/zk? The ZookeeperInfoHandler seems to do quite a lot more logic 
than just shuffling bits from ZK to the client.
ZookeeperInfoHandler does a lot of things that should ideally be done by the 
admin UI code. There is no technical reason why the UI won't be able to consume 
this new endpoint (of course, with necessary changes to the UI).

I see bunch of questions (most of which have been answered above already), but 
no technical grounds on why this issue *must be veto'ed*. While I'm happy to 
answer any of your questions, please withdraw your veto and let us move forward.


was (Author: ichattopadhyaya):
bq. Where is the decision that all data in zk strictly public? What about 
security.json?
All ZK data was public (read-only) till now anyway (via the older API). When 
there is a security.json, this endpoint can be protected using adequate 
authc/authz configurations. There is no change to our policy, nor is there a 
security vulnerability due to this issue.

bq.We have been trying to abstract away zk. So this API must be 
tagged/documented clearly as internal.
We've been trying to abstract away ZK, but not the data stored in ZK. It is 
useful for monitoring/debugging, in a spirit similar to how debug/explain works 
for a query. I agree this API should be tagged as experimental/internal etc.

bq.The proposal adds redundant API surface. Has there been done any 
analysis of whether it is possible to cut over Admin UI from /admin/zookeeper 
to /api/cluster/zk? The ZookeeperInfoHandler seems to do quite a lot more logic 
than just shuffling bits from ZK to the client.
ZookeeperInfoHandler does a lot of things that should ideally be done by the 
admin UI code. There is no technical reason why the UI won't be able to consume 
this new endpoint (of course, with necessary changes to the UI).

Now, please withdraw your veto and let us move forward.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051232#comment-17051232
 ] 

Ishan Chattopadhyaya edited comment on SOLR-13942 at 3/4/20, 1:45 PM:
--

bq. Where is the decision that all data in zk strictly public? What about 
security.json?
All ZK data was public (read-only) till now anyway (via the older API). When 
there is a security.json, this endpoint can be protected using adequate 
authc/authz configurations. There is no change to our policy, nor is there a 
security vulnerability due to this issue.

bq.We have been trying to abstract away zk. So this API must be 
tagged/documented clearly as internal.
We've been trying to abstract away ZK, but not the data stored in ZK. It is 
useful for monitoring/debugging (Shalin has listed a very nice list of items 
that are useful here), in a spirit similar to how debug/explain works for a 
query. I agree this API should be tagged as experimental/internal etc.

bq.The proposal adds redundant API surface. Has there been done any 
analysis of whether it is possible to cut over Admin UI from /admin/zookeeper 
to /api/cluster/zk? The ZookeeperInfoHandler seems to do quite a lot more logic 
than just shuffling bits from ZK to the client.
ZookeeperInfoHandler does a lot of things that should ideally be done by the 
admin UI code. There is no technical reason why the UI won't be able to consume 
this new endpoint (of course, with necessary changes to the UI).

I see bunch of questions (most of which have been answered above already), but 
no technical grounds on why this issue *must be veto'ed*. While I'm happy to 
answer any of your questions, please withdraw your veto and let us move forward.


was (Author: ichattopadhyaya):
bq. Where is the decision that all data in zk strictly public? What about 
security.json?
All ZK data was public (read-only) till now anyway (via the older API). When 
there is a security.json, this endpoint can be protected using adequate 
authc/authz configurations. There is no change to our policy, nor is there a 
security vulnerability due to this issue.

bq.We have been trying to abstract away zk. So this API must be 
tagged/documented clearly as internal.
We've been trying to abstract away ZK, but not the data stored in ZK. It is 
useful for monitoring/debugging, in a spirit similar to how debug/explain works 
for a query. I agree this API should be tagged as experimental/internal etc.

bq.The proposal adds redundant API surface. Has there been done any 
analysis of whether it is possible to cut over Admin UI from /admin/zookeeper 
to /api/cluster/zk? The ZookeeperInfoHandler seems to do quite a lot more logic 
than just shuffling bits from ZK to the client.
ZookeeperInfoHandler does a lot of things that should ideally be done by the 
admin UI code. There is no technical reason why the UI won't be able to consume 
this new endpoint (of course, with necessary changes to the UI).

I see bunch of questions (most of which have been answered above already), but 
no technical grounds on why this issue *must be veto'ed*. While I'm happy to 
answer any of your questions, please withdraw your veto and let us move forward.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-14291) OldAnalyticsRequestConverter should support fields names with dots

2020-03-04 Thread Anatolii Siuniaev (Jira)



 [ 
https://issues.apache.org/jira/browse/SOLR-14291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anatolii Siuniaev updated SOLR-14291:
-
Attachment: SOLR-14291.patch

> OldAnalyticsRequestConverter should support fields names with dots
> --
>
> Key: SOLR-14291
> URL: https://issues.apache.org/jira/browse/SOLR-14291
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search, SearchComponents - other
>Reporter: Anatolii Siuniaev
>Priority: Trivial
> Attachments: SOLR-14291.patch, SOLR-14291.patch
>
>
> If you send a query with range facets using old olap-style syntax (see pdf 
> [here|https://issues.apache.org/jira/browse/SOLR-5302]), 
> OldAnalyticsRequestConverter just silently (no exception thrown) omits 
> parameters like
> {code:java}
> olap..rangefacet..start
> {code}
> in case if __ has dots inside (for instance field name is 
> _Project.Value_). And thus no range facets are returned in response.  
> Probably the same happens in case of field faceting. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051236#comment-17051236
 ] 

Andrzej Bialecki commented on SOLR-13942:
-

[~noble.paul] and [~ichattopadhyaya] - I understand Jan's concern here, because 
you've just added a new, powerful and totally undocumented API to Solr just a 
moment before the release, and when he pushed back you largely ignored his 
concerns instead of trying to find a better solution.

So, in CTR workflow, if there are valid concerns and the original committer 
ignores them, IMHO a veto is a justified way to draw his attention and force 
the discussion, in order to arrive at a better solution.

Having said that, I also accept that there is at least a temporary need in 8x 
for accessing the full content of Solr state data (wherever it might be, in ZK 
or in a DB or elsewhere) without actually accessing the ZK native API, so I see 
the merit for adding an API like this.

The API that was added has no documentation whatsoever, and only rudimentary 
testing. I don't buy the argument that since it's an expert API it should be 
intentionally undocumented - this smells of security through obscurity, and 
builds a tech debt in terms of the missing docs for public APIs. Just from this 
point of view the work here is NOT done - we need the documentation and more 
comprehensive tests.

Here's my strawman proposal how to proceed:
 * revert (or don't revert) but address Jan's concerns before the 8.5 release, 
plus:
 * mark the API as experimental, both in the docs and in the API output
 * add RefGuide documentation
 * add more tests - this is a powerful new API and it has just one basic test? 
come on.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14291) OldAnalyticsRequestConverter should support fields names with dots

2020-03-04 Thread Anatolii Siuniaev (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051235#comment-17051235
 ] 

Anatolii Siuniaev commented on SOLR-14291:
--

[~houston] I added updated patch. Previous patch revealed another subtle bug - 
another regex which is supposed to match "e|end" param matched "hardend" param 
also. I fixed it. 

> OldAnalyticsRequestConverter should support fields names with dots
> --
>
> Key: SOLR-14291
> URL: https://issues.apache.org/jira/browse/SOLR-14291
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search, SearchComponents - other
>Reporter: Anatolii Siuniaev
>Priority: Trivial
> Attachments: SOLR-14291.patch, SOLR-14291.patch
>
>
> If you send a query with range facets using old olap-style syntax (see pdf 
> [here|https://issues.apache.org/jira/browse/SOLR-5302]), 
> OldAnalyticsRequestConverter just silently (no exception thrown) omits 
> parameters like
> {code:java}
> olap..rangefacet..start
> {code}
> in case if __ has dots inside (for instance field name is 
> _Project.Value_). And thus no range facets are returned in response.  
> Probably the same happens in case of field faceting. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-14291) OldAnalyticsRequestConverter should support fields names with dots

2020-03-04 Thread Anatolii Siuniaev (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051235#comment-17051235
 ] 

Anatolii Siuniaev edited comment on SOLR-14291 at 3/4/20, 1:52 PM:
---

[~houston] I added updated patch. Previous patch revealed another subtle bug - 
another regex which is supposed to match "e|end" param matched "hardend" param 
also. I fixed it. 

cc [~mkhl] [~janhoy]


was (Author: anatolii_siuniaev):
[~houston] I added updated patch. Previous patch revealed another subtle bug - 
another regex which is supposed to match "e|end" param matched "hardend" param 
also. I fixed it. 

> OldAnalyticsRequestConverter should support fields names with dots
> --
>
> Key: SOLR-14291
> URL: https://issues.apache.org/jira/browse/SOLR-14291
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search, SearchComponents - other
>Reporter: Anatolii Siuniaev
>Priority: Trivial
> Attachments: SOLR-14291.patch, SOLR-14291.patch
>
>
> If you send a query with range facets using old olap-style syntax (see pdf 
> [here|https://issues.apache.org/jira/browse/SOLR-5302]), 
> OldAnalyticsRequestConverter just silently (no exception thrown) omits 
> parameters like
> {code:java}
> olap..rangefacet..start
> {code}
> in case if __ has dots inside (for instance field name is 
> _Project.Value_). And thus no range facets are returned in response.  
> Probably the same happens in case of field faceting. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] msokolov commented on a change in pull request #1313: LUCENE-8962: Split test case

msokolov commented on a change in pull request #1313: LUCENE-8962: Split test 
case
URL: https://github.com/apache/lucene-solr/pull/1313#discussion_r387682345
 
 

 ##
 File path: 
lucene/core/src/test/org/apache/lucene/index/TestIndexWriterMergePolicy.java
 ##
 @@ -298,63 +320,44 @@ public void testMergeOnCommit() throws IOException, 
InterruptedException {
 DirectoryReader firstReader = DirectoryReader.open(firstWriter);
 assertEquals(5, firstReader.leaves().size());
 firstReader.close();
-firstWriter.close();
-
-MergePolicy mergeOnCommitPolicy = new LogDocMergePolicy() {
-  @Override
-  public MergeSpecification findFullFlushMerges(MergeTrigger mergeTrigger, 
SegmentInfos segmentInfos, MergeContext mergeContext) {
-// Optimize down to a single segment on commit
-if (mergeTrigger == MergeTrigger.COMMIT && segmentInfos.size() > 1) {
-  List nonMergingSegments = new ArrayList<>();
-  for (SegmentCommitInfo sci : segmentInfos) {
-if (mergeContext.getMergingSegments().contains(sci) == false) {
-  nonMergingSegments.add(sci);
-}
-  }
-  if (nonMergingSegments.size() > 1) {
-MergeSpecification mergeSpecification = new MergeSpecification();
-mergeSpecification.add(new OneMerge(nonMergingSegments));
-return mergeSpecification;
-  }
-}
-return null;
-  }
-};
+firstWriter.close(); // When this writer closes, it does not merge on 
commit.
 
-AtomicInteger abandonedMerges = new AtomicInteger(0);
 IndexWriterConfig iwc = newIndexWriterConfig(new MockAnalyzer(random()))
-.setMergePolicy(mergeOnCommitPolicy)
-.setIndexWriterEvents(new IndexWriterEvents() {
-  @Override
-  public void beginMergeOnCommit() {
-
-  }
-
-  @Override
-  public void finishMergeOnCommit() {
+.setMergePolicy(MERGE_ON_COMMIT_POLICY);
 
-  }
-
-  @Override
-  public void abandonedMergesOnCommit(int abandonedCount) {
-abandonedMerges.incrementAndGet();
-  }
-});
 IndexWriter writerWithMergePolicy = new IndexWriter(dir, iwc);
-
-writerWithMergePolicy.commit();
+writerWithMergePolicy.commit(); // No changes. Commit doesn't trigger a 
merge.
 
 DirectoryReader unmergedReader = 
DirectoryReader.open(writerWithMergePolicy);
-assertEquals(5, unmergedReader.leaves().size()); // Don't merge unless 
there's a change
+assertEquals(5, unmergedReader.leaves().size());
 unmergedReader.close();
 
 TestIndexWriter.addDoc(writerWithMergePolicy);
-writerWithMergePolicy.commit();
+writerWithMergePolicy.commit(); // Doc added, do merge on commit.
+assertEquals(1, writerWithMergePolicy.getSegmentCount()); //
 
 DirectoryReader mergedReader = DirectoryReader.open(writerWithMergePolicy);
-assertEquals(1, mergedReader.leaves().size()); // Now we merge on commit
+assertEquals(1, mergedReader.leaves().size());
 mergedReader.close();
 
+try (IndexReader reader = writerWithMergePolicy.getReader()) {
+  IndexSearcher searcher = new IndexSearcher(reader);
+  assertEquals(6, reader.numDocs());
+  assertEquals(6, searcher.count(new MatchAllDocsQuery()));
+}
+
+writerWithMergePolicy.close();
+dir.close();
+  }
+
+   // Test that when we have multiple indexing threads merging on commit, we 
never throw an exception.
+  @Nightly
 
 Review comment:
   I ran this test a few times, and noticed it takes 2-3 minutes to complete. 
Was it this slow before? I don't remember it taking so long in the previous 
version. Perhaps it's because it's Nightly now so the constants are made larger?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051241#comment-17051241
 ] 

Andrzej Bialecki commented on SOLR-13942:
-

One additional thought: if we're trying to abstract our ZK usage, perhaps we 
should not be naming this new API {{/api/cluster/zk}} (apart from the fact that 
it looks like another word that starts with cluster and ends with k ;) )? 
Perhaps use {{/api/cluster/distribState}} or something like that?

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051247#comment-17051247
 ] 

Ishan Chattopadhyaya commented on SOLR-13942:
-

bq. because you've just added a new, powerful and totally undocumented API to 
Solr just a moment before the release, and when he pushed back you largely 
ignored his concerns instead of trying to find a better solution.
This isn't any more powerful than an already existing API; just cleaner. All of 
his concerns were addressed. Please let me know if we've overlooked or missed 
any of his specific concerns that we haven't addressed yet.

bq. So, in CTR workflow, if there are valid concerns and the original committer 
ignores them, IMHO a veto is a justified way to draw his attention and force 
the discussion, in order to arrive at a better solution.
I still don't see a technical reason for the veto, but a few questions (which 
Jason asked in the past and summarized again and answered).
https://www.apache.org/foundation/voting.html#Veto

{quote}
Here's my strawman proposal how to proceed:
revert (or don't revert) but address Jan's concerns before the 8.5 release, 
plus:
mark the API as experimental, both in the docs and in the API output
add RefGuide documentation
add more tests - this is a powerful new API and it has just one basic test? 
come on.
{quote}
I agree with doing all this, but this can be done without the veto/revert. Btw, 
I don't see how this is a "powerful new API", and a unit test to validate 
whether ZK's data is actually being returned is insufficient.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051248#comment-17051248
 ] 

Ishan Chattopadhyaya commented on SOLR-13942:
-

bq. apart from the fact that it looks like another word that starts with 
cluster and ends with k
Hats off to you! :-D

How about {{/api/cluster/debug}}, whereby all the ZK (or etcd in future) info 
will be shown as if it is for debugging purposes?

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-9909) Nuke one of DefaultSolrThreadFactory and SolrjNamedThreadFactory

2020-03-04 Thread Lucene/Solr QA (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-9909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051249#comment-17051249
 ] 

Lucene/Solr QA commented on SOLR-9909:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 19 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
35s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
43s{color} | {color:red} prometheus-exporter in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
21s{color} | {color:red} core in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
14s{color} | {color:red} test-framework in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m 43s{color} 
| {color:red} prometheus-exporter in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m 21s{color} 
| {color:red} core in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m 14s{color} 
| {color:red} test-framework in the patch failed. {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} | 
{color:green}  0m  2s{color} | {color:green} Release audit (RAT) rat-sources 
passed {color} |
| {color:red}-1{color} | {color:red} Release audit (RAT) {color} | {color:red}  
0m  3s{color} | {color:red} prometheus-exporter in the patch failed. {color} |
| {color:red}-1{color} | {color:red} Release audit (RAT) {color} | {color:red}  
0m 21s{color} | {color:red} core in the patch failed. {color} |
| {color:red}-1{color} | {color:red} Release audit (RAT) {color} | {color:red}  
0m 14s{color} | {color:red} test-framework in the patch failed. {color} |
| {color:red}-1{color} | {color:red} Check forbidden APIs {color} | {color:red} 
 0m  2s{color} | {color:red} Check forbidden APIs check-forbidden-apis failed 
{color} |
| {color:red}-1{color} | {color:red} Validate source patterns {color} | 
{color:red}  0m  2s{color} | {color:red} Check forbidden APIs 
check-forbidden-apis failed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m  
3s{color} | {color:green} tools in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  0m 15s{color} 
| {color:red} prometheus-exporter in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  0m 16s{color} 
| {color:red} core in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  0m 15s{color} 
| {color:red} test-framework in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}  5m  5s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-9909 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12995608/SOLR-9909-02.patch |
| Optional Tests |  compile  javac  unit  ratsources  checkforbiddenapis  
validatesourcepatterns  |
| uname | Linux lucene1-us-west 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
 |
| git revision | master / 46bda6bea0f |
| ant | version: Apache Ant(TM) version 1.10.5 compiled on March 28 2019 |
| Default Java | LTS |
| compile | 
https://builds.apache.org/job/PreCommit-SOLR-Build/695/artifact/out/patch-compile-solr_contrib_prometheus-exporter.txt
 |
| compile | 
https://builds.apache.org/job/PreCommit-SOLR-Build/695/artifact/out/patch-compile-solr_core.txt
 |
| compile | 
https://builds.apache.org/job/PreCommit-SOLR-Build/695/artifact/out/patch-compile-solr_test-framework.txt
 |
| javac | 
https://builds.apache.org/job/PreCommit-SOLR-Build/695/artifact/out/patch-compile-solr_contrib_prometheus-exporter.txt
 |
| javac | 
https://builds.apache.org/job/PreCommit-SOLR-Build/695/artifact/out/patch-compile-solr_core.txt
 |
| javac | 
https://builds.apache.org/job/PreCommit-SOLR-Build/695/artifact/out/patch-compile-solr_test-framework.txt
 |
| Release audit (RAT) | 
https://builds.apache.org/job/PreCommit-SOLR-Build/695/artifact/out/patch-compile-solr_contrib_prometheus-exporter.txt
 |
| Release audit (RAT) | 
https://builds.apache.org/job/PreCommit-SOLR-Build/695/artifact/

[jira] [Comment Edited] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051248#comment-17051248
 ] 

Ishan Chattopadhyaya edited comment on SOLR-13942 at 3/4/20, 2:21 PM:
--

bq. apart from the fact that it looks like another word that starts with 
cluster and ends with k
Hats off to you! :-D

How about {{/api/cluster/debug}} (or /api/cluster/debug?type=zookeeper), 
whereby all the ZK (or etcd in future) info will be shown as if it is for 
debugging purposes?


was (Author: ichattopadhyaya):
bq. apart from the fact that it looks like another word that starts with 
cluster and ends with k
Hats off to you! :-D

How about {{/api/cluster/debug}}, whereby all the ZK (or etcd in future) info 
will be shown as if it is for debugging purposes?

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051265#comment-17051265
 ] 

Andrzej Bialecki commented on SOLR-13942:
-

{quote}This isn't any more powerful than an already existing API; just cleaner.
{quote}
You're missing the point: it's a separate API. It needs to be at least 
documented so that the ops people are made aware of it, know it exists and know 
how to secure it.
{quote}a unit test to validate whether ZK's data is actually being returned is 
insufficient
{quote}
Do you think that this is a sufficient test of a new API?
{code:java}
@Test
public void testZkread() throws Exception {
  URL baseUrl = cluster.getJettySolrRunner(0).getBaseUrl();
  try(  HttpSolrClient client = new 
HttpSolrClient.Builder(baseUrl.toString()).build()){
Object o = Utils.executeGET(client.getHttpClient(), 
baseUrl.toString().replace("/solr", "/api"+ "/cluster/zk/security.json"), 
Utils.JSONCONSUMER );

// THIS IS THE TEST:
assertNotNull(o);

  }
}{code}
 

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14291) OldAnalyticsRequestConverter should support fields names with dots

2020-03-04 Thread Lucene/Solr QA (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051267#comment-17051267
 ] 

Lucene/Solr QA commented on SOLR-14291:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} | 
{color:green}  1m 37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} | 
{color:green}  1m 37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} | 
{color:green}  1m 37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  1m 
55s{color} | {color:green} analytics in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}  9m 11s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-14291 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12995614/SOLR-14291.patch |
| Optional Tests |  compile  javac  unit  ratsources  checkforbiddenapis  
validatesourcepatterns  |
| uname | Linux lucene2-us-west.apache.org 4.4.0-170-generic #199-Ubuntu SMP 
Thu Nov 14 01:45:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
 |
| git revision | master / 46bda6b |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 |
| Default Java | LTS |
|  Test Results | 
https://builds.apache.org/job/PreCommit-SOLR-Build/696/testReport/ |
| modules | C: solr/contrib/analytics U: solr/contrib/analytics |
| Console output | 
https://builds.apache.org/job/PreCommit-SOLR-Build/696/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> OldAnalyticsRequestConverter should support fields names with dots
> --
>
> Key: SOLR-14291
> URL: https://issues.apache.org/jira/browse/SOLR-14291
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search, SearchComponents - other
>Reporter: Anatolii Siuniaev
>Priority: Trivial
> Attachments: SOLR-14291.patch, SOLR-14291.patch
>
>
> If you send a query with range facets using old olap-style syntax (see pdf 
> [here|https://issues.apache.org/jira/browse/SOLR-5302]), 
> OldAnalyticsRequestConverter just silently (no exception thrown) omits 
> parameters like
> {code:java}
> olap..rangefacet..start
> {code}
> in case if __ has dots inside (for instance field name is 
> _Project.Value_). And thus no range facets are returned in response.  
> Probably the same happens in case of field faceting. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051269#comment-17051269
 ] 

Ishan Chattopadhyaya commented on SOLR-13942:
-

bq. Do you think that this is a sufficient test of a new API?
Ah, no! Need some meaningful asserts to validate some of the data against what 
exists in ZK. I assumed, mistakenly, that the entire ZookeeperStatusHandlerTest 
suite was added.

bq. You're missing the point: it's a separate API. It needs to be at least 
documented so that the ops people are made aware of it, know it exists and know 
how to secure it.
Sure, I'm +1 to documenting it in ref guide. But, the veto wasn't on these 
grounds, right?

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051269#comment-17051269
 ] 

Ishan Chattopadhyaya edited comment on SOLR-13942 at 3/4/20, 2:30 PM:
--

bq. Do you think that this is a sufficient test of a new API?
Ah, no! Need some meaningful asserts to validate some of the data against what 
exists in ZK. I assumed, mistakenly, that the entire ZookeeperStatusHandlerTest 
suite was added.

bq. You're missing the point: it's a separate API. It needs to be at least 
documented so that the ops people are made aware of it, know it exists and know 
how to secure it.
Sure, I'm +1 to documenting it in ref guide. But, the veto wasn't on these 
grounds, right? Or if it was, then couldn't a ref guide change been added as a 
follow up commit, without need for a veto?


was (Author: ichattopadhyaya):
bq. Do you think that this is a sufficient test of a new API?
Ah, no! Need some meaningful asserts to validate some of the data against what 
exists in ZK. I assumed, mistakenly, that the entire ZookeeperStatusHandlerTest 
suite was added.

bq. You're missing the point: it's a separate API. It needs to be at least 
documented so that the ops people are made aware of it, know it exists and know 
how to secure it.
Sure, I'm +1 to documenting it in ref guide. But, the veto wasn't on these 
grounds, right?

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051275#comment-17051275
 ] 

Andrzej Bialecki commented on SOLR-13942:
-

{quote}But, the veto wasn't on these grounds, right?
{quote}
I'll let Jan comment on that in detail - but from my POV the work here is NOT 
done for the reasons I listed above, and if it takes veto to complete it before 
the release, so be it.

Jan's concern about the UI is also valid - now the ops can't block the old API 
because the UI still uses it, so they need to open both the old one and the new 
one - which in fact does increase the public API surface to be secured.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9257) FSTLoadMode should not be BlockTree specific as it is used more generally in index package

2020-03-04 Thread Michael McCandless (Jira)



[ 
https://issues.apache.org/jira/browse/LUCENE-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051295#comment-17051295
 ] 

Michael McCandless commented on LUCENE-9257:


{quote}What about removing FSTLoadMode and always loading the FST off-heap? 
I've been running benchmarks with OffheapFSTStore and the overhead is very 
small for end-to-end indexing.
{quote}
+1

> FSTLoadMode should not be BlockTree specific as it is used more generally in 
> index package
> --
>
> Key: LUCENE-9257
> URL: https://issues.apache.org/jira/browse/LUCENE-9257
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Bruno Roustant
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> FSTLoadMode and its associate attribute key (static String) are currently 
> defined in BlockTreeTermsReader, but they are actually used outside of 
> BlockTree in the general "index" package.
> CheckIndex and ReadersAndUpdates are using these enum and attribute key to 
> drive the FST load mode through the SegmentReader which is not specific to a 
> postings format. They have an unnecessary dependency to BlockTreeTermsReader.
> We could move FSTLoadMode out of BlockTreeTermsReader, to make it a public 
> enum of the "index" package. That way CheckIndex and ReadersAndUpdates do not 
> import anymore BlockTreeTermsReader.
> This would also allow other postings formats to use the same enum (e.g. 
> LUCENE-9254)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-9909) Nuke one of DefaultSolrThreadFactory and SolrjNamedThreadFactory

2020-03-04 Thread Andras Salamon (Jira)



 [ 
https://issues.apache.org/jira/browse/SOLR-9909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Salamon updated SOLR-9909:
-
Attachment: SOLR-9909-03.patch

> Nuke one of DefaultSolrThreadFactory and SolrjNamedThreadFactory
> 
>
> Key: SOLR-9909
> URL: https://issues.apache.org/jira/browse/SOLR-9909
> Project: Solr
>  Issue Type: Task
>Reporter: Shalin Shekhar Mangar
>Priority: Trivial
> Fix For: 6.7, 7.0
>
> Attachments: SOLR-9909-01.patch, SOLR-9909-02.patch, 
> SOLR-9909-03.patch
>
>
> DefaultSolrThreadFactory and SolrjNamedThreadFactory have exactly the same 
> code. Let's remove one of them.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051308#comment-17051308
 ] 

Ishan Chattopadhyaya commented on SOLR-13942:
-

Okay, thanks for your suggestions, [~ab].
Requesting Jan to kindly state the technical grounds for the veto, before 
proceeding with any of the pending work (e.g. ref guides update etc.).
Don't want to waste time here until I understand the exact reasons for the 
veto, straight from the horse's mouth.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9259) NGramFilter use wrong argument name for preserve option

2020-03-04 Thread Tomoko Uchida (Jira)



[ 
https://issues.apache.org/jira/browse/LUCENE-9259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051311#comment-17051311
 ] 

Tomoko Uchida commented on LUCENE-9259:
---

The added tests and ref-guide examples also look fine to me. I'm planning to 
commit the patch to the master and branch_8x in shortly.

[~Paul Pazderski] Can I ask your email address for crediting?

 

> NGramFilter use wrong argument name for preserve option
> ---
>
> Key: LUCENE-9259
> URL: https://issues.apache.org/jira/browse/LUCENE-9259
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/analysis
>Affects Versions: 7.4, 8.0
>Reporter: Paul Pazderski
>Priority: Minor
> Attachments: LUCENE-9259.patch
>
>
> LUCENE-7960 added the possibility to preserve the original term when using 
> NGram filters. The documentation says to enable it with 'preserveOriginal' 
> and it works for EdgeNGram filter. But NGram filter requires the initial 
> planned option 'keepShortTerms' to enable this feature.
> This inconsistency is confusing. I'll provide a patch with a possible fix.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14232) Add shareSchema leak protections

2020-03-04 Thread David Smiley (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051325#comment-17051325
 ] 

David Smiley commented on SOLR-14232:
-

[~noble.paul] lets discuss the imperfections of shareSchema here and not 
SOLR-14040

bq. This can cause ClassCastException if the per core component is sharing an 
object with schema

Did did you see this and can you try to explain more specifically how this can 
happen?  Your explanation wasn't bad but I don't quite understand enough to, 
for example, write a test showing it. I'm trying to ascertain how avoidable the 
problem is.  If triggering it requires a schema component that implements 
SolrCoreAware then that doesn't really count as it's avoidable.

> Add shareSchema leak protections
> 
>
> Key: SOLR-14232
> URL: https://issues.apache.org/jira/browse/SOLR-14232
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>Reporter: David Smiley
>Priority: Major
>
> The shareSchema option in solr.xml allows cores to share a common 
> IndexSchema, assuming the underlying schema is literally the same (from the 
> same configSet). However this sharing has no protections to prevent an 
> IndexSchema from accidentally referencing the SolrCore and its settings. The 
> effect might be nondeterministic behavior depending on which core loaded the 
> schema first, or the effect might be a memory leak preventing a closed 
> SolrCore from GC'ing, or maybe an error. Example:
>  * IndexSchema could theoretically do property expansion using the core's 
> props, such as solr.core.name, silly as that may be.
>  * IndexSchema uses the same SolrResourceLoader for the core, which in turn 
> tracks infoMBeans and other things that can refer to the core. It should 
> probably have it's own SolrResourceLoader but it's not trivial; there are 
> complications with life-cycle of ResourceLoaderAware tracking etc.
>  * If anything in IndexSchema is SolrCoreAware, this isn't going to work!
>  ** SchemaSimilarityFactory is SolrCoreAware, though I think it could be 
> reduced to being SchemaAware and work.
>  ** ExternalFileField is currently SchemaAware it grabs the 
> SolrResourceLoader to call getDataDir which is bad.  FYI In a separate PR I'm 
> removing getDataDir from SRL.
>  ** Should probably fail if anything is detected.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data

2020-03-04 Thread Jason Gerlowski (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051333#comment-17051333
 ] 

Jason Gerlowski commented on SOLR-13942:


I'm gonna zoom in to my "Solr vs external tool" objection for now.  I've got 
other concerns still, but I think focusing in is the best way to get to the 
bottom here.

I agree that ZK data is useful for debugging Solr at times.  I agree that 
making access to this data simple is a Good Thing for Solr admins in the 
trenches.  I agree that admins need some sort of ops tool to look at this data.

But I think we can also agree, not all ops tools belong in Solr.

e.g. Metrics visualizations are important for admins managing Solr.  But it 
would be a mistake to start adding Admin UI screens with involved graphs of 
Solr's metrics.  There's already great tools out there for this job, like 
Grafana, with whole teams around them.  Those tools are going to see continuous 
improvement, attention, etc. that our reinvented-wheel just wouldn't get.  Ops 
folks have to bite the bullet on an additional container or two, but they get a 
much better tool as a result.

I think ZK-proxy APIs fall into a similar category.  There are good, solid 
ZooKeeper clients out there, with teams whose main job is to support and keep 
them up to date.  We shouldn't double-down on being a ZK-read-proxy any more 
than we should decide to start implementing Grafana-like graphs in our UI.  
There's just a better tool for that job, and those tools are going to age much 
better than anything we write here, without adding maintenance burden to the 
project. 

Two arguments against an external client mentioned above were that external 
clients required wider ZK-exposure, and a desire to avoid additional 
processes/containers/ops-overhead.

In terms of ZK firewall exposure, there are HTTP-based ZK clients that offer 
the exact indirection we're getting with these Solr APIs now.  Look at 
zooNavigator, or zk-web, or browze (written by a Lucene committer btw!).  More 
recently published and probably the smartest option, Solr admins could look at 
putting an nginx proxy in front of ZooKeeper's new AdminServer jetty instance.

In terms of saving a container or two - I mean, I understand that everyone is 
fighting cost.  But the HTTP-based ZK clients above are (AFAICT) light, 
stateless, glorified proxies.Some support ZK's ACL mechanism, so they're 
known to work with whatever ZK security clients have setup.  Some are 
dockerized and ready to deploy.  They can be started or stopped at need to save 
hardware cost, if that's a concern.  Overall it doesn't seem like that 
burdensome of an ask for Solr admins who want to do the sort of advanced 
monitoring you're describing and can't get what they need from Solr's metrics, 
overseerstatus API, etc.

So neither of those "This is why this must be in Solr itself" counterarguments 
seem super compelling to me.  But I'm open to seeing other sides to this.  Can 
either of you guys point to something I'm missing here?

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051335#comment-17051335
 ] 

Jan Høydahl commented on SOLR-13942:


Why do you want to hold up a release for such a new feature that is clearly not 
ready?

This should not be so hard, just revert and file a new PR, adding some of us 
annoying bikeshedders as reviewers :) Then it's business as usual from there 
on. Except you may have to bribe Alan to hold up the release of course, which 
is still better than forcing him due to the blocker. Consider my veto lifted 
the moment the revert is done.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051346#comment-17051346
 ] 

Ishan Chattopadhyaya commented on SOLR-13942:
-

bq. Why do you want to hold up a release for such a new feature that is clearly 
not ready?
No one is proposing that. Tomorrow is just the branch cutting, and docs/tests 
etc. can go after that as well.

bq. Except you may have to bribe Alan to hold up the release of course, which 
is still better than forcing him due to the blocker.
I doubt that will be the case, since the release is quite a few days away. But, 
now it all rests in your hands.

bq. This should not be so hard, just revert and file a new PR, adding some of 
us annoying bikeshedders as reviewers. 
Until we know what are the technical grounds for the veto, what is the point of 
filing another PR? If it is just to add refguide documents, then we can do so 
even now (without the veto or the revert).

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14284) Document that you can add a new stream function via add-expressible

2020-03-04 Thread David Eric Pugh (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051357#comment-17051357
 ] 

David Eric Pugh commented on SOLR-14284:


Hey [~jbernste] or [~erickerickson] would you be able to review and commit 
this?  Hoping to get this in before 8.5.   Giving a talk next week showing this 
off ;-)

> Document that you can add a new stream function via add-expressible
> ---
>
> Key: SOLR-14284
> URL: https://issues.apache.org/jira/browse/SOLR-14284
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Affects Versions: 8.5
>Reporter: David Eric Pugh
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I confirmed that in Solr 8.5 you will be able to dynamically add a Stream 
> function (assuming the Jar is in the path) via the configset api:
> curl -X POST -H 'Content-type:application/json'  -d '{
>   "add-expressible": {
> "name": "dog",
> "class": "org.apache.solr.handler.CatStream"
>   }
> }' http://localhost:8983/solr/gettingstarted/config



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14294) Typo in response to ?action=XX on StreamHandler

2020-03-04 Thread David Eric Pugh (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051356#comment-17051356
 ] 

David Eric Pugh commented on SOLR-14294:


Hey [~jbernste] or [~erickerickson] would you be able to review and commit 
this?  Hoping to get this in before 8.5.   Giving a talk next week showing this 
off ;-)

> Typo in response to ?action=XX on StreamHandler
> ---
>
> Key: SOLR-14294
> URL: https://issues.apache.org/jira/browse/SOLR-14294
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Affects Versions: 8.4.1
>Reporter: David Eric Pugh
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The messages back when interacting with Streaming API have typo in the word 
> Daemon, they are spelled "Deamon".



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (LUCENE-9261) Point to downloads.apache.org instead of www.apache.org/dist

Jan Høydahl created LUCENE-9261:
---

 Summary: Point to downloads.apache.org instead of 
www.apache.org/dist
 Key: LUCENE-9261
 URL: https://issues.apache.org/jira/browse/LUCENE-9261
 Project: Lucene - Core
  Issue Type: Task
  Components: general/website
Reporter: Jan Høydahl


Ref e-mail from INFRA today.
{quote}As of March 2020, we are deprecating www.apache.org/dist/ in favor of
[https://downloads.apache.org/] for backup downloads as well as signature
 and checksum verification. The primary driver has been splitting up web
 site visits and downloads to gain better control and offer a better
 service for both downloads and web site visits.
{quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data

2020-03-04 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051369#comment-17051369
 ] 

Ishan Chattopadhyaya commented on SOLR-13942:
-

Jason, I think you lack some perspective from the point of view of operations 
engineers who run Solr. Here's my perspective. 

1. In past 4+ years of consulting, I've encountered several clients who have 
brought me on to help resolve a production issue. Often, these calls are over 
conference calls, so I don't have SSH access to their instances (sometimes I 
did). I am at their mercy to browse or capture screenshots of the Solr admin UI 
in order to get a fair idea of the ZK data. Here's a recent example, as 
[~munendrasn] can testify: recently I had to help out with a situation where 
overseer wasn't getting elected and OVERSEERSTATUS API wasn't working. I needed 
a way for the client to be able to dump the entire ZK data quickly and pass it 
to me for further analysis. (In this case, I made a recommendation without 
having access to the ZK data, and still saved the day.) Asking such clients, in 
times of such crisis, to install clients or fight with our ZK client etc. is 
unreasonable because there maybe policy restrictions on their part which will 
make this process lengthy.

2. Most often, Solr is just part of a very large distributed system comprising 
of several components and microservices. Expecting dev-ops to install and 
maintain additional tools for Solr is unreasonable. Also, since we treat ZK as 
an implementation detail of Solr, it is unreasonable to expect dev-ops to now 
start setting up proxies etc. for ZK as a way of monitoring Solr. Solr should 
be able to let expert users peek into the data that Solr puts into ZK. As 
you've rightly identified, cost of maintaining additional tools is a factor. 
Another factor is the complexity of monitoring such additional tools. Imagine, 
the situation when there's a crisis and an outage and the nginx proxy isn't 
working (and there was no alerting setup to warn that it has gone down). Having 
Solr let you keep into Solr's own internal state data reduces moving parts 
needed to debug Solr problems.

Hope this helps.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13794) Delete solr/core/src/test-files/solr/configsets/_default



[ 
https://issues.apache.org/jira/browse/SOLR-13794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051370#comment-17051370
 ] 

ASF subversion and git services commented on SOLR-13794:


Commit 76f74d1217666dc2391a4c4afc9f728e105c99dc in lucene-solr's branch 
refs/heads/branch_8x from Alan Woodward
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=76f74d1 ]

SOLR-13794: Don't look for removed _default configset in addVersion.py


> Delete solr/core/src/test-files/solr/configsets/_default
> 
>
> Key: SOLR-13794
> URL: https://issues.apache.org/jira/browse/SOLR-13794
> Project: Solr
>  Issue Type: Test
>Reporter: Chris M. Hostetter
>Assignee: Chris M. Hostetter
>Priority: Major
> Fix For: master (9.0), 8.5
>
> Attachments: SOLR-13794.patch, SOLR-13794.patch, 
> SOLR-13794_code_only.patch, SOLR-13794_code_only.patch
>
>
> For as long as we've had a {{_default}} configset in solr, we've also had a 
> copy of that default in {{core/src/test-files/}} - as well as a unit test 
> that confirms they are identical.
> It's never really been clear to me *why* we have this duplication, instead of 
> just having the test-framework take the necessary steps to ensure that 
> {{server/solr/configsets/_default}} is properly used when running tests.
> I'd like to propose we eliminate the duplication since it only ever seems to 
> cause problems (notably spurious test failures when people modify the 
> {{_default}} configset w/o remembering that they need to make identical edits 
> to the {{test-files}} clone) and instead have {{SolrTestCase}} set the 
> (already existing & supported) {{solr.default.confdir}} system property to 
> point to the (already existing) {{ExternalPaths.DEFAULT_CONFIGSET}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051369#comment-17051369
 ] 

Ishan Chattopadhyaya edited comment on SOLR-13942 at 3/4/20, 4:13 PM:
--

Jason, I think you lack some perspective from the point of view of experts who 
intend to resolve problems and also operations engineers who run & maintain 
Solr. Here's my perspective. 

1. In past 4+ years of consulting, I've encountered several clients who have 
brought me on to help resolve a production issue. Often, these calls are over 
conference calls, so I don't have SSH access to their instances (sometimes I 
did). I am at their mercy to browse or capture screenshots of the Solr admin UI 
in order to get a fair idea of the ZK data. Here's a recent example, as 
[~munendrasn] can testify: recently I had to help out with a situation where 
overseer wasn't getting elected and OVERSEERSTATUS API wasn't working. I needed 
a way for the client to be able to dump the entire ZK data quickly and pass it 
to me for further analysis. (In this case, I made a recommendation without 
having access to the ZK data, and still saved the day.) Asking such clients, in 
times of such crisis, to install clients or fight with our ZK client etc. is 
unreasonable because there maybe policy restrictions on their part which will 
make this process lengthy.

2. Most often, Solr is just part of a very large distributed system comprising 
of several components and microservices. Expecting dev-ops to install and 
maintain additional tools for Solr is unreasonable. Also, since we treat ZK as 
an implementation detail of Solr, it is unreasonable to expect dev-ops to now 
start setting up proxies etc. for ZK as a way of monitoring Solr. Solr should 
be able to let expert users peek into the data that Solr puts into ZK. As 
you've rightly identified, cost of maintaining additional tools is a factor. 
Another factor is the complexity of monitoring such additional tools. Imagine, 
the situation when there's a crisis and an outage and the nginx proxy isn't 
working (and there was no alerting setup to warn that it has gone down). Having 
Solr let you keep into Solr's own internal state data reduces moving parts 
needed to debug Solr problems.

Hope this helps.


was (Author: ichattopadhyaya):
Jason, I think you lack some perspective from the point of view of operations 
engineers who run Solr. Here's my perspective. 

1. In past 4+ years of consulting, I've encountered several clients who have 
brought me on to help resolve a production issue. Often, these calls are over 
conference calls, so I don't have SSH access to their instances (sometimes I 
did). I am at their mercy to browse or capture screenshots of the Solr admin UI 
in order to get a fair idea of the ZK data. Here's a recent example, as 
[~munendrasn] can testify: recently I had to help out with a situation where 
overseer wasn't getting elected and OVERSEERSTATUS API wasn't working. I needed 
a way for the client to be able to dump the entire ZK data quickly and pass it 
to me for further analysis. (In this case, I made a recommendation without 
having access to the ZK data, and still saved the day.) Asking such clients, in 
times of such crisis, to install clients or fight with our ZK client etc. is 
unreasonable because there maybe policy restrictions on their part which will 
make this process lengthy.

2. Most often, Solr is just part of a very large distributed system comprising 
of several components and microservices. Expecting dev-ops to install and 
maintain additional tools for Solr is unreasonable. Also, since we treat ZK as 
an implementation detail of Solr, it is unreasonable to expect dev-ops to now 
start setting up proxies etc. for ZK as a way of monitoring Solr. Solr should 
be able to let expert users peek into the data that Solr puts into ZK. As 
you've rightly identified, cost of maintaining additional tools is a factor. 
Another factor is the complexity of monitoring such additional tools. Imagine, 
the situation when there's a crisis and an outage and the nginx proxy isn't 
working (and there was no alerting setup to warn that it has gone down). Having 
Solr let you keep into Solr's own internal state data reduces moving parts 
needed to debug Solr problems.

Hope this helps.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/

[GitHub] [lucene-site] janhoy opened a new pull request #16: LUCENE-9261: Change download links to new downloads.apache.org

janhoy opened a new pull request #16: LUCENE-9261: Change download links to new 
downloads.apache.org
URL: https://github.com/apache/lucene-site/pull/16
 
 
   See https://issues.apache.org/jira/browse/LUCENE-9261


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9257) FSTLoadMode should not be BlockTree specific as it is used more generally in index package

2020-03-04 Thread Bruno Roustant (Jira)



[ 
https://issues.apache.org/jira/browse/LUCENE-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051373#comment-17051373
 ] 

Bruno Roustant commented on LUCENE-9257:


Ok, well, if there is consensus then I'll propose another PR soon to remove 
FSTLoadMode enum and always load the FST off-heap.

> FSTLoadMode should not be BlockTree specific as it is used more generally in 
> index package
> --
>
> Key: LUCENE-9257
> URL: https://issues.apache.org/jira/browse/LUCENE-9257
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Bruno Roustant
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> FSTLoadMode and its associate attribute key (static String) are currently 
> defined in BlockTreeTermsReader, but they are actually used outside of 
> BlockTree in the general "index" package.
> CheckIndex and ReadersAndUpdates are using these enum and attribute key to 
> drive the FST load mode through the SegmentReader which is not specific to a 
> postings format. They have an unnecessary dependency to BlockTreeTermsReader.
> We could move FSTLoadMode out of BlockTreeTermsReader, to make it a public 
> enum of the "index" package. That way CheckIndex and ReadersAndUpdates do not 
> import anymore BlockTreeTermsReader.
> This would also allow other postings formats to use the same enum (e.g. 
> LUCENE-9254)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9261) Point to downloads.apache.org instead of www.apache.org/dist



[ 
https://issues.apache.org/jira/browse/LUCENE-9261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051375#comment-17051375
 ] 

Jan Høydahl commented on LUCENE-9261:
-

Here's a PR for the web site [https://github.com/apache/lucene-site/pull/16]

 

> Point to downloads.apache.org instead of www.apache.org/dist
> 
>
> Key: LUCENE-9261
> URL: https://issues.apache.org/jira/browse/LUCENE-9261
> Project: Lucene - Core
>  Issue Type: Task
>  Components: general/website
>Reporter: Jan Høydahl
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ref e-mail from INFRA today.
> {quote}As of March 2020, we are deprecating www.apache.org/dist/ in favor of
> [https://downloads.apache.org/] for backup downloads as well as signature
>  and checksum verification. The primary driver has been splitting up web
>  site visits and downloads to gain better control and offer a better
>  service for both downloads and web site visits.
> {quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Assigned] (LUCENE-9261) Point to downloads.apache.org instead of www.apache.org/dist



 [ 
https://issues.apache.org/jira/browse/LUCENE-9261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl reassigned LUCENE-9261:
---

Assignee: Jan Høydahl

> Point to downloads.apache.org instead of www.apache.org/dist
> 
>
> Key: LUCENE-9261
> URL: https://issues.apache.org/jira/browse/LUCENE-9261
> Project: Lucene - Core
>  Issue Type: Task
>  Components: general/website
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ref e-mail from INFRA today.
> {quote}As of March 2020, we are deprecating www.apache.org/dist/ in favor of
> [https://downloads.apache.org/] for backup downloads as well as signature
>  and checksum verification. The primary driver has been splitting up web
>  site visits and downloads to gain better control and offer a better
>  service for both downloads and web site visits.
> {quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051369#comment-17051369
 ] 

Ishan Chattopadhyaya edited comment on SOLR-13942 at 3/4/20, 4:16 PM:
--

Jason, I think you lack some perspective from the point of view of experts who 
intend to resolve problems and also operations engineers who run & maintain 
Solr. Here's my perspective. 

1. In past 4+ years of consulting, I've encountered several clients who have 
brought me on to help resolve a production issue. Often, these calls are over 
conference calls, so I don't have SSH access to their instances (sometimes I 
did). I am at their mercy to browse or capture screenshots of the Solr admin UI 
in order to get a fair idea of the ZK data. Here's a recent example, as 
[~munendrasn] can testify: recently I had to help out with a situation where 
overseer wasn't getting elected and OVERSEERSTATUS API wasn't working. I needed 
a way for the client to be able to dump the entire ZK data quickly and pass it 
to me for further analysis. (In this case, I made a recommendation without 
having access to the ZK data, and still saved the day.) Asking such clients, in 
times of such crisis, to install clients or fight with our ZK client etc. is 
unreasonable because there maybe policy restrictions on their part which will 
make this process lengthy.

2. Most often, Solr is just part of a very large distributed system comprising 
of several components and microservices. Expecting dev-ops to install and 
maintain additional tools for Solr is unreasonable. Also, since we treat ZK as 
an implementation detail of Solr, it is unreasonable to expect dev-ops to now 
start setting up proxies etc. for ZK as a way of monitoring Solr. Solr should 
be able to let expert users peek into the data that Solr puts into ZK. As 
you've rightly identified, cost of maintaining additional tools is a factor. 
Another factor is the complexity of monitoring such additional tools. Imagine, 
the situation when there's a crisis and an outage and the nginx proxy isn't 
working (and there was no alerting setup to warn that it has gone down). Having 
Solr let you peek into Solr's own internal state data reduces moving parts 
needed to debug Solr problems.

Hope this helps.


was (Author: ichattopadhyaya):
Jason, I think you lack some perspective from the point of view of experts who 
intend to resolve problems and also operations engineers who run & maintain 
Solr. Here's my perspective. 

1. In past 4+ years of consulting, I've encountered several clients who have 
brought me on to help resolve a production issue. Often, these calls are over 
conference calls, so I don't have SSH access to their instances (sometimes I 
did). I am at their mercy to browse or capture screenshots of the Solr admin UI 
in order to get a fair idea of the ZK data. Here's a recent example, as 
[~munendrasn] can testify: recently I had to help out with a situation where 
overseer wasn't getting elected and OVERSEERSTATUS API wasn't working. I needed 
a way for the client to be able to dump the entire ZK data quickly and pass it 
to me for further analysis. (In this case, I made a recommendation without 
having access to the ZK data, and still saved the day.) Asking such clients, in 
times of such crisis, to install clients or fight with our ZK client etc. is 
unreasonable because there maybe policy restrictions on their part which will 
make this process lengthy.

2. Most often, Solr is just part of a very large distributed system comprising 
of several components and microservices. Expecting dev-ops to install and 
maintain additional tools for Solr is unreasonable. Also, since we treat ZK as 
an implementation detail of Solr, it is unreasonable to expect dev-ops to now 
start setting up proxies etc. for ZK as a way of monitoring Solr. Solr should 
be able to let expert users peek into the data that Solr puts into ZK. As 
you've rightly identified, cost of maintaining additional tools is a factor. 
Another factor is the complexity of monitoring such additional tools. Imagine, 
the situation when there's a crisis and an outage and the nginx proxy isn't 
working (and there was no alerting setup to warn that it has gone down). Having 
Solr let you keep into Solr's own internal state data reduces moving parts 
needed to debug Solr problems.

Hope this helps.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {c

[GitHub] [lucene-site] janhoy merged pull request #15: Conditionally show security warning block only if news last 60 days.

janhoy merged pull request #15: Conditionally show security warning block only 
if news last 60 days.
URL: https://github.com/apache/lucene-site/pull/15
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-site] janhoy opened a new pull request #17: Put #15 in production

janhoy opened a new pull request #17: Put #15 in production
URL: https://github.com/apache/lucene-site/pull/17
 
 
   Hide security warning box after 60 days.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data

[
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051369#comment-17051369
]

Ishan Chattopadhyaya edited comment on SOLR-13942 at 3/4/20, 4:24 PM:
--

Jason, I think you lack some perspective from the point of view of experts who
intend to resolve problems and also operations engineers who run & maintain
Solr. Here's my perspective.

1. In past 4+ years of consulting, I've encountered several clients who have
brought me on to help resolve a production issue. Often, these calls are over
conference calls, so I don't have SSH access to their instances (sometimes I
did). I am at their mercy to browse or capture screenshots of the Solr admin UI
in order to get a fair idea of the ZK data. Here's a recent example, as
[~munendrasn] can testify: recently I had to help out with a situation where
overseer wasn't getting elected and OVERSEERSTATUS API wasn't working. I needed
a way for the client to be able to dump the entire ZK data quickly and pass it
to me for further analysis. (In this case, I made a recommendation without
having access to the ZK data, and still saved the day.) Asking such clients, in
times of such crisis, to install clients or fight with our ZK client etc. is
unreasonable because there maybe policy restrictions on their part which will
make this process lengthy.

2. Most often, Solr is just part of a very large distributed system comprising
of several components and microservices. Expecting dev-ops to install and
maintain additional tools for Solr is unreasonable. Also, since we treat ZK as
an implementation detail of Solr, it is unreasonable to expect dev-ops to now
start setting up proxies etc. for ZK as a way of monitoring Solr. Solr should
be able to let expert users peek into the data that Solr puts into ZK. As
you've rightly identified, cost of maintaining additional tools is a factor.
Another factor is the complexity of monitoring such additional tools. Imagine,
the situation when there's a crisis and an outage and the nginx proxy isn't
working (and there was no alerting setup to warn that it has gone down). Having
Solr let you peek into Solr's own internal state data reduces moving parts
needed to debug Solr problems.

Hope this helps.

p.s.: [~erickerickson], [~dsmiley], [~markrmil...@gmail.com], some perspective
from you would help us understand if this issue solves a real problem (which
Noble and I seem to think it does). Your war stories will far outweigh whatever
I've seen, I'm sure.

was (Author: ichattopadhyaya):
Jason, I think you lack some perspective from the point of view of experts who
intend to resolve problems and also operations engineers who run & maintain
Solr. Here's my perspective.

Hope this helps.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
> Issue Type: New Feature

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data

2020-03-04 Thread Erick Erickson (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051382#comment-17051382
 ] 

Erick Erickson commented on SOLR-13942:
---

Can’t reply right now to the JIRA, and don’t know if it’s relevant. But.

bin/solr ZK cp -r / some_local_dir

Best,
Erick



> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14284) Document that you can add a new stream function via add-expressible

2020-03-04 Thread Erick Erickson (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051384#comment-17051384
 ] 

Erick Erickson commented on SOLR-14284:
---

I’m traveling with limited internet access for the next week, so I won’t be 
able to...

> Document that you can add a new stream function via add-expressible
> ---
>
> Key: SOLR-14284
> URL: https://issues.apache.org/jira/browse/SOLR-14284
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Affects Versions: 8.5
>Reporter: David Eric Pugh
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I confirmed that in Solr 8.5 you will be able to dynamically add a Stream 
> function (assuming the Jar is in the path) via the configset api:
> curl -X POST -H 'Content-type:application/json'  -d '{
>   "add-expressible": {
> "name": "dog",
> "class": "org.apache.solr.handler.CatStream"
>   }
> }' http://localhost:8983/solr/gettingstarted/config



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14283) Fix NPE in SolrTestCaseJ4 preventing it from working on external projects

2020-03-04 Thread Alan Woodward (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051387#comment-17051387
 ] 

Alan Woodward commented on SOLR-14283:
--

Can this be closed?

> Fix NPE in SolrTestCaseJ4 preventing it from working on external projects
> -
>
> Key: SOLR-14283
> URL: https://issues.apache.org/jira/browse/SOLR-14283
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Affects Versions: 8.5
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
> Attachments: SOLR-14283.patch
>
>
> Though it's goals were laudable, SOLR-14217 ran afoul of an untested 
> requirement that the SolrTestCaseJ4 class continue to function even if  
> ExternalPaths#SOURCE_HOME has been set to null by the logic in 
> org.apache.solr.util.ExternalPaths#determineSourceHome
> The result is that in any almost all usages of the solr test framework 
> outside of Solr that rely on SolrTestCaseJ4 or it's sub-classes including 
> SolrCloudTestCase the following exception would be thrown:
> {code}
> java.lang.NullPointerException
>   at __randomizedtesting.SeedInfo.seed([7A1D202DDAEA1C07]:0)
>   at 
> java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
>   at 
> java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
>   at java.base/java.util.Properties.put(Properties.java:1337)
>   at java.base/java.util.Properties.setProperty(Properties.java:225)
>   at java.base/java.lang.System.setProperty(System.java:895)
>   at 
> org.apache.solr.SolrTestCaseJ4.setupTestCases(SolrTestCaseJ4.java:284)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
>   at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>   at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>   at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>   at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>   at 
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
>   at 
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>   at 
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>   at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>   at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>   at 
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>   at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>   at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>   at 
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
>   at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> This ticket will fix the issue and provides a test so that we don't break 
> nearly every user of the test framework other than ourselves in the future.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051388#comment-17051388
 ] 

Ishan Chattopadhyaya edited comment on SOLR-13942 at 3/4/20, 4:28 PM:
--

bq. bin/solr ZK cp -r / some_local_dir
Often times Solr and ZK are both firewalled inside a production environment, 
from where it is difficult to get files copied and attached into emails easily. 
However, the Solr ports are accessible (via VPN, firewall whitelists etc.), and 
hence it is possible for devs to grab output of an HTTP call and attach in an 
email or paste on slack. This proposed endpoint will help there.


was (Author: ichattopadhyaya):
bq. bin/solr ZK cp -r / some_local_dir
Often times Solr and ZK are both firewalled inside a production environment, 
from where it is difficult to get files copied and attached into emails easily. 
However, the Solr ports are accessible (via VPN etc.), and hence it is possible 
for devs to grab output of an HTTP call and attach in an email or paste on 
slack. This proposed endpoint will help there.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051388#comment-17051388
 ] 

Ishan Chattopadhyaya commented on SOLR-13942:
-

bq. bin/solr ZK cp -r / some_local_dir
Often times Solr and ZK are both firewalled inside a production environment, 
from where it is difficult to get files copied and attached into emails easily. 
However, the Solr ports are accessible (via VPN etc.), and hence it is possible 
for devs to grab output of an HTTP call and attach in an email or paste on 
slack. This proposed endpoint will help there.

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14040) solr.xml shareSchema does not work in SolrCloud

2020-03-04 Thread Alan Woodward (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051390#comment-17051390
 ] 

Alan Woodward commented on SOLR-14040:
--

Does this need more work before it can be resolved?

> solr.xml shareSchema does not work in SolrCloud
> ---
>
> Key: SOLR-14040
> URL: https://issues.apache.org/jira/browse/SOLR-14040
> Project: Solr
>  Issue Type: Improvement
>  Components: Schema and Analysis
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> solr.xml has a shareSchema boolean option that can be toggled from the 
> default of false to true in order to share IndexSchema objects within the 
> Solr node.  This is silently ignored in SolrCloud mode.  The pertinent code 
> is {{org.apache.solr.core.ConfigSetService#createConfigSetService}} which 
> creates a CloudConfigSetService that is not related to the SchemaCaching 
> class.  This may not be a big deal in SolrCloud which tends not to deal well 
> with many cores per node but I'm working on changing that.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9259) NGramFilter use wrong argument name for preserve option

2020-03-04 Thread Paul Pazderski (Jira)



[ 
https://issues.apache.org/jira/browse/LUCENE-9259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051397#comment-17051397
 ] 

Paul Pazderski commented on LUCENE-9259:


You can use [p...@ppazderski.de|mailto:p...@ppazderski.de] if you like.

> NGramFilter use wrong argument name for preserve option
> ---
>
> Key: LUCENE-9259
> URL: https://issues.apache.org/jira/browse/LUCENE-9259
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/analysis
>Affects Versions: 7.4, 8.0
>Reporter: Paul Pazderski
>Priority: Minor
> Attachments: LUCENE-9259.patch
>
>
> LUCENE-7960 added the possibility to preserve the original term when using 
> NGram filters. The documentation says to enable it with 'preserveOriginal' 
> and it works for EdgeNGram filter. But NGram filter requires the initial 
> planned option 'keepShortTerms' to enable this feature.
> This inconsistency is confusing. I'll provide a patch with a possible fix.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data

2020-03-04 Thread Erick Erickson (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051396#comment-17051396
 ] 

Erick Erickson commented on SOLR-13942:
---

Oh, looked there I can comment on JIRAs.

 

I should have mentioned that I could go on and on about how difficult it is to 
troubleshoot Solr with clients, or, for that matter, operations people. 
Anything we can do to facilitate this is welcome.

We’ve been guilty all along of thinking like developers who have access to 
_everything_. And it’s not just when trying to consult I’ve the phone. There 
are lots of organizations out there that.have their developers get stuff 
working then throw it over the wall to their operations people. I can’t count 
the number of times I feel like saying “find all the Solr log directories on 
all your machines, plus the GC logs plus the ZK logs (oh, where are they?) and 
zip them all up and send them to me” A day or two later they’ve tracked them 
all down, zipped them up and the resulting 100G zip file overwhelms me. 
Assuming they’ve gotten to them in time to capture the problem.

 

I’ve worked with clients with at least several hundred physical nodes. Every 
client at that scale I’ve worked with has invested considerable time and energy 
creating custom tools to help them because Solr doesn’t have much support 
there. Joel’s work with streaming to identify query response and I have an SIP 
out about indexing log files with Solr-specific faceting are two very small 
examples.

Oh, and we need thread dumps of all those hundreds of machines too sometimes.

 

And I’ve completely skipped security restrictions. Government, financial 
institutions, health insurance applications, etc. may or may not even allow 
screen sharing. To make it worse, operations folks in some of these places 
_also_ may not be able to see the raw data.

All this to drive home the fact that we’ve frankly done a poor job of making 
troubleshooting easier...

 



> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] atris commented on a change in pull request #1303: LUCENE-9114: Improve ValueSourceScorer's Default Cost Implementation

atris commented on a change in pull request #1303: LUCENE-9114: Improve 
ValueSourceScorer's Default Cost Implementation
URL: https://github.com/apache/lucene-solr/pull/1303#discussion_r387794239
 
 

 ##
 File path: 
lucene/queries/src/java/org/apache/lucene/queries/function/FunctionValues.java
 ##
 @@ -41,6 +41,9 @@
 //   want the Query carrying around big objects
 public abstract class FunctionValues {
 
+  // Default cost for FunctionValues -- ideally should be overriden by 
concrete implementations
+  public static final int DEFAULT_COST = 100;
 
 Review comment:
   Fixed, thanks


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] atris commented on a change in pull request #1303: LUCENE-9114: Improve ValueSourceScorer's Default Cost Implementation

atris commented on a change in pull request #1303: LUCENE-9114: Improve 
ValueSourceScorer's Default Cost Implementation
URL: https://github.com/apache/lucene-solr/pull/1303#discussion_r387794662
 
 

 ##
 File path: 
lucene/queries/src/java/org/apache/lucene/queries/function/FunctionValues.java
 ##
 @@ -151,6 +164,11 @@ public ValueSourceScorer getScorer(Weight weight, 
LeafReaderContext readerContex
   public boolean matches(int doc) {
 return true;
   }
+  @Override
+  public float costEvaluationFunction() {
+// Match everything
 
 Review comment:
   Removed the comment, thanks


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] atris commented on issue #1303: LUCENE-9114: Improve ValueSourceScorer's Default Cost Implementation

atris commented on issue #1303: LUCENE-9114: Improve ValueSourceScorer's 
Default Cost Implementation
URL: https://github.com/apache/lucene-solr/pull/1303#issuecomment-594654890
 
 
   @dsmiley Updated, please see and let me know your thoughts and comments. I 
have not added CHANGES.txt entry to avoid conflicts, will do so just before 
committing.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (SOLR-14304) CDCR is failing for dynamic field as single valued are sent to target collection with multiple values

Jayesh Shende created SOLR-14304:


 Summary: CDCR is failing for dynamic field as single valued are 
sent to target collection with multiple values
 Key: SOLR-14304
 URL: https://issues.apache.org/jira/browse/SOLR-14304
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: CDCR, SolrCloud
Affects Versions: 8.1.1
 Environment: Server OS : AWS Linux 

Java :

openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)
Reporter: Jayesh Shende


During CDCR, after syncing data directories from source to target collection's 
replicas and then starting Target and Source cluster, when I start the CDCR 
replication using /cdcr?action=START on Source collection,

On Source collection, I am getting following error on Solr admin logging 
section:
|BaseCloudSolrClient|Request to collection [search1] failed due to (400) 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://IP:PORT/solr/search1_shard1_replica_n1: ERROR: 
[doc=A_someID_s] multiple values encountered for non multiValued field 
StartDate_dt: [2012-04-12T00:00:01.000Z, 
2012-04-12T00:00:01.000Z], retry=0 commError=false errorCode=400|

 

and following warning:
|CdcrReplicator|Failed to forward update request to target: search1|

 

Also on Target Collection getting following ERROR :
|ERROR 
false|x:search1_shard1_replica_n1|RequestHandlerBase|org.apache.solr.common.SolrException:
 ERROR: [doc=A_someID_s] multiple values encountered for non multiValued field 
StartDate_dt: [2012-04-12T00:00:01.000Z, 2012-04-12T00:00:01.000Z]|

 

The field I am using is :

 





 

 Other CDCR monitoring actions and STAR, STOP and BOOTSTRAP are working fine.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] mikemccand commented on issue #1274: LUCENE-9164: Prevent IW from closing gracefully if threads are still modifying

mikemccand commented on issue #1274: LUCENE-9164: Prevent IW from closing 
gracefully if threads are still modifying
URL: https://github.com/apache/lucene-solr/pull/1274#issuecomment-594683279
 
 
   > the change proposed here might be the better solution after all #1215
   
   Yeah, +1 -- that change is very simple.  I do think it's still a workaround 
for true simplification for IW (I agree this is long overdue and also very 
challenging!!) since `AlreadyClosedException` really should never be thrown and 
caught by a tragic handler.  But it is a very simple solution to the issue.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-14304) CDCR is failing for dynamic field as single valued are sent to target collection with multiple values



 [ 
https://issues.apache.org/jira/browse/SOLR-14304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jayesh Shende updated SOLR-14304:
-
Environment: 
Server OS : AWS Linux 

Java :

openjdk version "1.8.0_232"
 OpenJDK Runtime Environment (build 1.8.0_232-b09)
 OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)

 

Solr Cloud with external zookeeper having 1 collection "search1"  with 2 shards 
with no extra replicas.

  was:
Server OS : AWS Linux 

Java :

openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)


> CDCR is failing for dynamic field as single valued are sent to target 
> collection with multiple values
> -
>
> Key: SOLR-14304
> URL: https://issues.apache.org/jira/browse/SOLR-14304
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: CDCR, SolrCloud
>Affects Versions: 8.1.1
> Environment: Server OS : AWS Linux 
> Java :
> openjdk version "1.8.0_232"
>  OpenJDK Runtime Environment (build 1.8.0_232-b09)
>  OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)
>  
> Solr Cloud with external zookeeper having 1 collection "search1"  with 2 
> shards with no extra replicas.
>Reporter: Jayesh Shende
>Priority: Major
>
> During CDCR, after syncing data directories from source to target 
> collection's replicas and then starting Target and Source cluster, when I 
> start the CDCR replication using /cdcr?action=START on Source collection,
> On Source collection, I am getting following error on Solr admin logging 
> section:
> |BaseCloudSolrClient|Request to collection [search1] failed due to (400) 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at http://IP:PORT/solr/search1_shard1_replica_n1: ERROR: 
> [doc=A_someID_s] multiple values encountered for non multiValued field 
> StartDate_dt: [2012-04-12T00:00:01.000Z, 
> 2012-04-12T00:00:01.000Z], retry=0 commError=false errorCode=400|
>  
> and following warning:
> |CdcrReplicator|Failed to forward update request to target: search1|
>  
> Also on Target Collection getting following ERROR :
> |ERROR 
> false|x:search1_shard1_replica_n1|RequestHandlerBase|org.apache.solr.common.SolrException:
>  ERROR: [doc=A_someID_s] multiple values encountered for non multiValued 
> field StartDate_dt: [2012-04-12T00:00:01.000Z, 
> 2012-04-12T00:00:01.000Z]|
>  
> The field I am using is :
>  
>  multiValued="false" omitNorms="true"/>
>  positionIncrementGap="0"/>
>  
>  Other CDCR monitoring actions and STAR, STOP and BOOTSTRAP are working fine.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-14304) CDCR is failing for dynamic field as single valued are sent to target collection with multiple values



 [ 
https://issues.apache.org/jira/browse/SOLR-14304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jayesh Shende updated SOLR-14304:
-
Priority: Critical  (was: Major)

> CDCR is failing for dynamic field as single valued are sent to target 
> collection with multiple values
> -
>
> Key: SOLR-14304
> URL: https://issues.apache.org/jira/browse/SOLR-14304
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: CDCR, SolrCloud
>Affects Versions: 8.1.1
> Environment: Server OS : AWS Linux 
> Java :
> openjdk version "1.8.0_232"
>  OpenJDK Runtime Environment (build 1.8.0_232-b09)
>  OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)
>  
> Solr Cloud with external zookeeper having 1 collection "search1"  with 2 
> shards with no extra replicas.
>Reporter: Jayesh Shende
>Priority: Critical
>
> During CDCR, after syncing data directories from source to target 
> collection's replicas and then starting Target and Source cluster, when I 
> start the CDCR replication using /cdcr?action=START on Source collection,
> On Source collection, I am getting following error on Solr admin logging 
> section:
> |BaseCloudSolrClient|Request to collection [search1] failed due to (400) 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at http://IP:PORT/solr/search1_shard1_replica_n1: ERROR: 
> [doc=A_someID_s] multiple values encountered for non multiValued field 
> StartDate_dt: [2012-04-12T00:00:01.000Z, 
> 2012-04-12T00:00:01.000Z], retry=0 commError=false errorCode=400|
>  
> and following warning:
> |CdcrReplicator|Failed to forward update request to target: search1|
>  
> Also on Target Collection getting following ERROR :
> |ERROR 
> false|x:search1_shard1_replica_n1|RequestHandlerBase|org.apache.solr.common.SolrException:
>  ERROR: [doc=A_someID_s] multiple values encountered for non multiValued 
> field StartDate_dt: [2012-04-12T00:00:01.000Z, 
> 2012-04-12T00:00:01.000Z]|
>  
> The field I am using is :
>  
>  multiValued="false" omitNorms="true"/>
>  positionIncrementGap="0"/>
>  
>  Other CDCR monitoring actions and STAR, STOP and BOOTSTRAP are working fine.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-7436) Solr stops printing stacktraces in log and output



[ 
https://issues.apache.org/jira/browse/SOLR-7436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051451#comment-17051451
 ] 

Chris M. Hostetter commented on SOLR-7436:
--

this appears to have regressed since 8.1? see SOLR-14302

> Solr stops printing stacktraces in log and output
> -
>
> Key: SOLR-7436
> URL: https://issues.apache.org/jira/browse/SOLR-7436
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.1
> Environment: Local 5.1
>Reporter: Markus Jelsma
>Assignee: Jan Høydahl
>Priority: Major
> Fix For: 6.3, 7.0
>
> Attachments: solr-8983-console.log
>
>
> After a short while, Solr suddenly stops printing stacktraces in the log and 
> output. 
> {code}
> 251043 [qtp1121454968-17] INFO  org.apache.solr.core.SolrCore.Request  [   
> suggests] - [suggests] webapp=/solr path=/select 
> params={q=*:*&fq={!collapse+field%3Dquery_digest}&fq={!collapse+field%3Dresult_digest}}
>  status=500 QTime=3 
> 251043 [qtp1121454968-17] ERROR org.apache.solr.servlet.SolrDispatchFilter  [ 
>   suggests] - null:java.lang.NullPointerException
> at 
> org.apache.solr.search.CollapsingQParserPlugin$IntScoreCollector.finish(CollapsingQParserPlugin.java:743)
> at 
> org.apache.solr.search.CollapsingQParserPlugin$IntScoreCollector.finish(CollapsingQParserPlugin.java:780)
> at 
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:203)
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1660)
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1479)
> at 
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:556)
> at 
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:518)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:222)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:829)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:446)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
> at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
> at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
> at org.eclipse.jetty.server.Server.handle(Server.java:368)
> at 
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
> at 
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
> at 
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
> at 
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
> at 
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
> at 
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
> at 
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
> at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
> at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
> at

[jira] [Commented] (SOLR-13394) Change default GC from CMS to G1



[ 
https://issues.apache.org/jira/browse/SOLR-13394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051453#comment-17051453
 ] 

Chris M. Hostetter commented on SOLR-13394:
---

this commit appears to have removed {{-XX:-OmitStackTraceInFastThrow}} (which 
isn't really a "GC" tuning option even though it was apparently lumped in with 
them) which causes a regression of SOLR-7436 ... now tracked in SOLR-14302

> Change default GC from CMS to G1
> 
>
> Key: SOLR-13394
> URL: https://issues.apache.org/jira/browse/SOLR-13394
> Project: Solr
>  Issue Type: Improvement
>Reporter: Ishan Chattopadhyaya
>Assignee: Ishan Chattopadhyaya
>Priority: Major
> Fix For: 8.1
>
> Attachments: SOLR-13394.patch, SOLR-13394.patch
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> CMS has been deprecated in new versions of Java 
> (http://openjdk.java.net/jeps/291). This issue is to switch Solr default from 
> CMS to G1.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14302) Solr stops printing stacktraces in log and output due to OmitStackTraceInFastThrow - regression of SOLR-7436



[ 
https://issues.apache.org/jira/browse/SOLR-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051458#comment-17051458
 ] 

Chris M. Hostetter commented on SOLR-14302:
---

[~ichattopadhyaya]: you removed this in SOLR-13394  , i assume unintentionally? 
... any objection to adding it back?

> Solr stops printing stacktraces in log and output due to 
> OmitStackTraceInFastThrow - regression of SOLR-7436 
> -
>
> Key: SOLR-14302
> URL: https://issues.apache.org/jira/browse/SOLR-14302
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 8.1
>Reporter: Chris M. Hostetter
>Priority: Major
>
> I recently saw a person ask a question about an Exception in their logs that 
> didn't have a stack trace even though it certainly seemed like it should.
> I was immediately suspicious that they may have tweaked their solr options to 
> override the {{-XX:-OmitStackTraceInFastThrow}} that was added to bin/solr by 
> SOLR-7436, but then i discovered it's gone now - removed in SOLR-13394 w/o 
> any discussion/consideration (and possibly unintentionally w/o understanding 
> it's purpose?)
> We should almost certainly restore this by default.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-14304) CDCR is failing for dynamic field as single valued are sent to target collection with multiple values



 [ 
https://issues.apache.org/jira/browse/SOLR-14304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jayesh Shende updated SOLR-14304:
-
Description: 
During Uni-Directional CDCR, after syncing data directories from source to 
target collection's replicas and then starting Target and Source cluster, when 
I start the CDCR replication using /cdcr?action=START on Source collection,

On Source collection, I am getting following error on Solr admin logging 
section:
|BaseCloudSolrClient|Request to collection [search1] failed due to (400) 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at 
[http://IP:PORT/solr/search1_shard1_replica_n1:|http://ip:PORT/solr/search1_shard1_replica_n1:]
 ERROR: [doc=A_someID_s] multiple values encountered for non multiValued field 
StartDate_dt: [2012-04-12T00:00:01.000Z, 2012-04-12T00:00:01.000Z|#8203; 
2012-04-12T00:00:01.000Z], retry=0 commError=false errorCode=400|

 

and following warning:
|CdcrReplicator|Failed to forward update request to target: search1|

 

Also on Target Collection getting following ERROR :
|ERROR 
false|x:search1_shard1_replica_n1|RequestHandlerBase|org.apache.solr.common.SolrException:
 ERROR: [doc=A_someID_s] multiple values encountered for non multiValued field 
StartDate_dt: [2012-04-12T00:00:01.000Z, 2012-04-12T00:00:01.000Z|#8203; 
2012-04-12T00:00:01.000Z]|

 

The field I am using is :

 





 

 Other CDCR monitoring actions and STAR, STOP and BOOTSTRAP are working fine.

 

  was:
During CDCR, after syncing data directories from source to target collection's 
replicas and then starting Target and Source cluster, when I start the CDCR 
replication using /cdcr?action=START on Source collection,

On Source collection, I am getting following error on Solr admin logging 
section:
|BaseCloudSolrClient|Request to collection [search1] failed due to (400) 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://IP:PORT/solr/search1_shard1_replica_n1: ERROR: 
[doc=A_someID_s] multiple values encountered for non multiValued field 
StartDate_dt: [2012-04-12T00:00:01.000Z, 
2012-04-12T00:00:01.000Z], retry=0 commError=false errorCode=400|

 

and following warning:
|CdcrReplicator|Failed to forward update request to target: search1|

 

Also on Target Collection getting following ERROR :
|ERROR 
false|x:search1_shard1_replica_n1|RequestHandlerBase|org.apache.solr.common.SolrException:
 ERROR: [doc=A_someID_s] multiple values encountered for non multiValued field 
StartDate_dt: [2012-04-12T00:00:01.000Z, 2012-04-12T00:00:01.000Z]|

 

The field I am using is :

 





 

 Other CDCR monitoring actions and STAR, STOP and BOOTSTRAP are working fine.

 


> CDCR is failing for dynamic field as single valued are sent to target 
> collection with multiple values
> -
>
> Key: SOLR-14304
> URL: https://issues.apache.org/jira/browse/SOLR-14304
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: CDCR, SolrCloud
>Affects Versions: 8.1.1
> Environment: Server OS : AWS Linux 
> Java :
> openjdk version "1.8.0_232"
>  OpenJDK Runtime Environment (build 1.8.0_232-b09)
>  OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)
>  
> Solr Cloud with external zookeeper having 1 collection "search1"  with 2 
> shards with no extra replicas.
>Reporter: Jayesh Shende
>Priority: Critical
>
> During Uni-Directional CDCR, after syncing data directories from source to 
> target collection's replicas and then starting Target and Source cluster, 
> when I start the CDCR replication using /cdcr?action=START on Source 
> collection,
> On Source collection, I am getting following error on Solr admin logging 
> section:
> |BaseCloudSolrClient|Request to collection [search1] failed due to (400) 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at 
> [http://IP:PORT/solr/search1_shard1_replica_n1:|http://ip:PORT/solr/search1_shard1_replica_n1:]
>  ERROR: [doc=A_someID_s] multiple values encountered for non multiValued 
> field StartDate_dt: [2012-04-12T00:00:01.000Z, 
> 2012-04-12T00:00:01.000Z|#8203; 2012-04-12T00:00:01.000Z], retry=0 
> commError=false errorCode=400|
>  
> and following warning:
> |CdcrReplicator|Failed to forward update request to target: search1|
>  
> Also on Target Collection getting following ERROR :
> |ERROR 
> false|x:search1_shard1_replica_n1|RequestHandlerBase|org.apache.solr.common.SolrException:
>  ERROR: [doc=A_someID_s] multiple values encountered for non multiValued 
> field StartDate_dt: [2012-04-12T00:00:01.000Z, 
> 2012-04-12T00:00:01.000Z|#8203; 2012-04-12T00:00:01.000Z]|
>  
> The field I

[jira] [Commented] (SOLR-13132) Improve JSON "terms" facet performance when sorted by relatedness



[ 
https://issues.apache.org/jira/browse/SOLR-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051469#comment-17051469
 ] 

Chris M. Hostetter commented on SOLR-13132:
---

Hey Michael,

I spent a little time looking at the PR – not an in depth review, just poking 
around, to try to get a an overall sense of the idea...

*TL:DR:* Definitely some bugs, needs tests, i'd like to help but it's a bit 
overwhelming to try and review as a monolithic PR

The first thing that jumped out at me is that there are no tests, or test 
modifications – that seemes concerning.

With something like this, where we're trying to optimize the behavior through 
caching, it's tempting to assume that as long as the change doesn't break 
existing tests that's "good enough" because we're not adding new 
functionality/APIs – but there's a few problems with that assumption:
 # There is actually a functionality/API change, in the form of the new 
config/hueristics/params controlling when/if to take advantage of the cache 
(and in this specific case of this PR: hueristics on when/if to do a single 
sweep). those options/hueristics need tested to ensure they behave as expected 
(which can typically be done using white box inspection of cache metrics, using 
things like the /metrics API)
 # existing tests that were written w/o any idea that caching might be invovled 
in the underlying code rarely follow usage patterns that would typically 
trigger cache hits – ie: a test that does the exact same query (or facet) twice 
in a row isn't likely to be written unless/untill you know there may/should be 
a cache hit involved.

Because all these changes are esentially a No-Op (w/o an explicit 
"termFacetCache" configured and/or an explicit query option) then even if 
existing tests would just happen to trigger the cache (or sweep) logic that 
won't matter w/o test configuration changes.

I did a quick and dirty hack, demonstrated by this little patch below, to try 
and *force* the "termFacetCache" to be enabled for all tests (and all term 
faceting, regardless of how small the count was) and was suprised to see a 
*LOT* of test errors/failures (I got 63 just in solr core)...
{noformat}
diff --git a/solr/core/src/java/org/apache/solr/request/TermFacetCache.java 
b/solr/core/src/java/org/apache/solr/request/TermFacetCache.java
index 3f22449b380..b818d2723d5 100644
--- a/solr/core/src/java/org/apache/solr/request/TermFacetCache.java
+++ b/solr/core/src/java/org/apache/solr/request/TermFacetCache.java
@@ -28,7 +28,7 @@ import org.apache.solr.search.QueryResultKey;
 public class TermFacetCache {
 
   public static final String NAME = "termFacetCache";
-  public static final int DEFAULT_THRESHOLD = 5000;
+  public static final int DEFAULT_THRESHOLD = 1; // nocommit: HACK to force 
all tests to use cache // 5000;
 
 
   public static final class FacetCacheKey {
diff --git a/solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java 
b/solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.jav
index ae47aed273b..8767a8f4db2 100644
--- a/solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java
+++ b/solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java
@@ -282,7 +282,8 @@ public class SolrIndexSearcher extends IndexSearcher 
implements Closeable, SolrI
   if (documentCache != null) clist.add(documentCache);
 
   if (solrConfig.userCacheConfigs.isEmpty()) {
-cacheMap = NO_GENERIC_CACHES;
+// nocommit: HACK to force TermFacetCache in all testing...
+cacheMap = new HashMap<>(); // nocommit: cacheMap = NO_GENERIC_CACHES;
   } else {
 cacheMap = new HashMap<>(solrConfig.userCacheConfigs.size());
 for (Map.Entry e : 
solrConfig.userCacheConfigs.entrySet()) {
@@ -294,6 +295,22 @@ public class SolrIndexSearcher extends IndexSearcher 
implements Closeable, SolrI
 }
   }
 
+  { // nocommit: HACK to force TermFacetCache in all testing...
+final Map nocommitArgs = new HashMap<>();
+nocommitArgs.put("name",org.apache.solr.request.TermFacetCache.NAME);
+nocommitArgs.put("size","200");
+nocommitArgs.put("initialSize","200");
+nocommitArgs.put("autowarmCount","200");
+
+final SolrCache nocommitCache =
+  (new CacheConfig(CaffeineCache.class,
+   nocommitArgs,
+   new 
org.apache.solr.request.TermFacetCacheRegenerator())).newInstance();
+
+cacheMap.put(nocommitCache.name(), nocommitCache);
+clist.add(nocommitCache);
+  }
+
   cacheList = clist.toArray(new SolrCache[clist.size()]);
 } else {
   this.filterCache = null;
{noformat}
I didn't analyze the results in depth, but at a glamce the failures seem to 
fall into a few categories:
 * term facet constraints out of order and/or unexpected counts
 ** suggesting that the cache i

[jira] [Commented] (SOLR-9909) Nuke one of DefaultSolrThreadFactory and SolrjNamedThreadFactory

2020-03-04 Thread Lucene/Solr QA (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-9909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051471#comment-17051471
 ] 

Lucene/Solr QA commented on SOLR-9909:
--

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 18 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} | 
{color:green}  0m 28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} | 
{color:green}  0m  4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} | 
{color:green}  0m  4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m  
4s{color} | {color:green} tools in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
23s{color} | {color:green} prometheus-exporter in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 78m  
3s{color} | {color:green} core in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
27s{color} | {color:green} test-framework in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 88m 30s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-9909 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12995628/SOLR-9909-03.patch |
| Optional Tests |  compile  javac  unit  ratsources  checkforbiddenapis  
validatesourcepatterns  |
| uname | Linux lucene2-us-west.apache.org 4.4.0-170-generic #199-Ubuntu SMP 
Thu Nov 14 01:45:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
 |
| git revision | master / 46bda6b |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 |
| Default Java | LTS |
|  Test Results | 
https://builds.apache.org/job/PreCommit-SOLR-Build/697/testReport/ |
| modules | C: lucene/tools solr/contrib/prometheus-exporter solr/core 
solr/test-framework U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-SOLR-Build/697/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Nuke one of DefaultSolrThreadFactory and SolrjNamedThreadFactory
> 
>
> Key: SOLR-9909
> URL: https://issues.apache.org/jira/browse/SOLR-9909
> Project: Solr
>  Issue Type: Task
>Reporter: Shalin Shekhar Mangar
>Priority: Trivial
> Fix For: 6.7, 7.0
>
> Attachments: SOLR-9909-01.patch, SOLR-9909-02.patch, 
> SOLR-9909-03.patch
>
>
> DefaultSolrThreadFactory and SolrjNamedThreadFactory have exactly the same 
> code. Let's remove one of them.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14302) Solr stops printing stacktraces in log and output due to OmitStackTraceInFastThrow - regression of SOLR-7436



[ 
https://issues.apache.org/jira/browse/SOLR-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051473#comment-17051473
 ] 

Ishan Chattopadhyaya commented on SOLR-14302:
-

Ouch, just realized that I had omitted it. I think I mistook it for some CMS 
specific parameter as it was under GC_TUNE.
No objection to bringing it back (so long as it doesn't go back to GC_TUNE 
again. Thanks Hoss!

> Solr stops printing stacktraces in log and output due to 
> OmitStackTraceInFastThrow - regression of SOLR-7436 
> -
>
> Key: SOLR-14302
> URL: https://issues.apache.org/jira/browse/SOLR-14302
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 8.1
>Reporter: Chris M. Hostetter
>Priority: Major
>
> I recently saw a person ask a question about an Exception in their logs that 
> didn't have a stack trace even though it certainly seemed like it should.
> I was immediately suspicious that they may have tweaked their solr options to 
> override the {{-XX:-OmitStackTraceInFastThrow}} that was added to bin/solr by 
> SOLR-7436, but then i discovered it's gone now - removed in SOLR-13394 w/o 
> any discussion/consideration (and possibly unintentionally w/o understanding 
> it's purpose?)
> We should almost certainly restore this by default.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data

2020-03-04 Thread Tomas Eduardo Fernandez Lobbe (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051501#comment-17051501
 ] 

Tomas Eduardo Fernandez Lobbe commented on SOLR-13942:
--

bq. You understand the difference between exposing zookeeper and exposing data 
in zookeeper, right? 
I do. We discussed making ZooKeeper an implementation detail, which means, we 
don’t want to expose any of those. Did you see the abstractions ab recently 
added? that’s the way we’ve decided we wanted to go in the past. This is just a 
step in the opposite direction.

bq. Data in Zookeeper is public data. It is not supposed to be fixed. Everyone 
knows this. 
This comment doesn’t make a ton of sense to me. What do you mean with “public 
data” and how does that relate with “it’s not supposed to be fixed”? Once you 
expose this via an API, it’s part of the API, you understand that, right? 
people will start building on top of it, and that means we either have to start 
supporting backwards compatibility of every ZooKeeper internal or we just 
disregard compatibility, and make it a mess for those users that are using the 
API. An API is a contract that needs to be respected, this includes the 
responses.

Regarding [~shalin] comments. Everything that we think is useful for debugging 
(those are good points), we should have an API that returns them. My point is 
that that API needs to have a defined format that will then be respected over 
versions. Let me give you some examples:
bq. Contents of overseer queue (mildly useful, available in /admin/zookeeper)
I’d add this to the OVERSEERSTATUS API, maybe with a parameter to not overload 
all requests
bq. Overseer election queue and current leader (former is available in 
/admin/zookeeper and latter in overseer status)
Same, this belongs to overseer status.
bq. Solr.xml (no API regardless of whether it is in ZK or filesystem)
Isn’t this static? it’s set on startup, either pushed to Zk or to the local 
node. I’m not sure we should have an API for this, but maybe I’m missing 
something.
bq. Leader election queue and current leader for each shard (available in 
/admin/zookeeper)
current leader is in CLUSTERSTATUS, election queue could be added too?
bq. Shard terms for each shard/replica (not available in any API)
This is somewhat recent. Maybe we need an API for this? I’m not sure
bq. GC logs (no API)
Is this in ZooKeeper?

bq. The overseerstatus API cannot be hit if there is no overseer so there's 
that too.
Maybe we can fix this? Some information would not be available, but it makes 
sense given that there is no OVERSEER?

Shalin, I think the best is to add the necessary information to the existing 
APIs where it fit, or if there is a need for a new API, it needs to have the 
right abstraction level? Returning ZooKeeper data is just hurting us, either 
our ability to evolve (if we decide to have compatibility of data in ZooKeeper) 
or hurt our users and their ability to upgrade Solr versions.

bq. I realize how miserable we make the life of ops guys. 
I’ve managed Solr clusters as both, dev and ops. I understand the pain of 
operating clusters. I also understand the pain of upgrading clusters. 

Ishan, you are disregarding comments because you assume people doesn’t have 
your experience? what’s with that? I’ve managed Solr clusters for long too, I 
understand the pains. I also understand how quick and easy solutions hurt the 
product in the long run. Many times I had to argue with my team/manager about 
things that helped us in some way but don’t belong to Solr because it’s not the 
right thing for it in the long term, maybe that’s something that you should 
consider.

bq. Expecting dev-ops to install and maintain additional tools for Solr is 
unreasonable. 
I don’t think so. Some things just don’t belong in Solr. 

bq. Solr should be able to let expert users peek into the data that Solr puts 
into ZK.
Disagree. This goes against the “hide ZooKeeper” strategy everyone agreed on 
recently.

bq. Having Solr let you peek into Solr's own internal state data reduces moving 
parts needed to debug Solr problems.
Agree, but with the right APIs. 

TL;DR;
I still think this needs to be reverted under the technical reason: "This new 
API makes adds exposure to internal metadata that can't be guaranteed to be 
compatible over versions. By adding this API we are agreeing to either maintain 
compatibility which will make development more complicated, or, if we decide 
not not have compatibility, we hurt users building on top of it, and make 
upgrades more difficult to them."

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>

[GitHub] [lucene-solr] dsmiley commented on issue #1303: LUCENE-9114: Improve ValueSourceScorer's Default Cost Implementation

dsmiley commented on issue #1303: LUCENE-9114: Improve ValueSourceScorer's 
Default Cost Implementation
URL: https://github.com/apache/lucene-solr/pull/1303#issuecomment-594728584
 
 
   Okay; add to "Improvements" section.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13132) Improve JSON "terms" facet performance when sorted by relatedness

2020-03-04 Thread Michael Gibney (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051505#comment-17051505
 ] 

Michael Gibney commented on SOLR-13132:
---

Thanks so much for the review and feedback, [~hossman]! This has been on the 
back burner for me for over a year, but I'm eager to pay some attention to it, 
especially now that I'm able to incorporate (and hopefully address) this 
initial feedback.

# I will separate out the facet cache as an independent PR associated with 
SOLR-13807. It is generally useful independent of any sweep/relatedness work, 
and has such a significant impact on the relatedness work that it might be 
reasonable to treat it as a dependency of this issue.
# I will write tests along the lines of what you've suggested, which will 
hopefully clarify and exercise some of what's going on (you're of course 
correct about the fact that all the caching stuff is a no-op without 
configuring a {{termFacetCache}}, so it is impossible to "rely on existing 
tests"). Among the points I hope to revisit/clarify with testing: regarding 
{{QueryResultKey}}, I believe the change I introduced only treats the main 
query as equivalent to filters for the case where no sort is specified, which 
should actually be ok, I think ({{queryResultsCache}} should always have a sort 
specified? -- in any case that's what I thought a year ago :-)).

I have some questions about exactly how to present the facet cache PR (whether 
to bother making it compatible with SimpleFacets in addition to JSON Facets, 
for one), but I'll ask those in a more deliberate way over at SOLR-13807.

Thanks again for the feedback, and I plan to follow up soon.

> Improve JSON "terms" facet performance when sorted by relatedness 
> --
>
> Key: SOLR-13132
> URL: https://issues.apache.org/jira/browse/SOLR-13132
> Project: Solr
>  Issue Type: Improvement
>  Components: Facet Module
>Affects Versions: 7.4, master (9.0)
>Reporter: Michael Gibney
>Priority: Major
> Attachments: SOLR-13132-with-cache-01.patch, 
> SOLR-13132-with-cache.patch, SOLR-13132.patch
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> When sorting buckets by {{relatedness}}, JSON "terms" facet must calculate 
> {{relatedness}} for every term. 
> The current implementation uses a standard uninverted approach (either 
> {{docValues}} or {{UnInvertedField}}) to get facet counts over the domain 
> base docSet, and then uses that initial pass as a pre-filter for a 
> second-pass, inverted approach of fetching docSets for each relevant term 
> (i.e., {{count > minCount}}?) and calculating intersection size of those sets 
> with the domain base docSet.
> Over high-cardinality fields, the overhead of per-term docSet creation and 
> set intersection operations increases request latency to the point where 
> relatedness sort may not be usable in practice (for my use case, even after 
> applying the patch for SOLR-13108, for a field with ~220k unique terms per 
> core, QTime for high-cardinality domain docSets were, e.g.: cardinality 
> 1816684=9000ms, cardinality 5032902=18000ms).
> The attached patch brings the above example QTimes down to a manageable 
> ~300ms and ~250ms respectively. The approach calculates uninverted facet 
> counts over domain base, foreground, and background docSets in parallel in a 
> single pass. This allows us to take advantage of the efficiencies built into 
> the standard uninverted {{FacetFieldProcessorByArray[DV|UIF]}}), and avoids 
> the per-term docSet creation and set intersection overhead.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13942) /api/cluster/zk/* to fetch raw ZK data

2020-03-04 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051516#comment-17051516
 ] 

Ishan Chattopadhyaya commented on SOLR-13942:
-

bq. Ishan, you are disregarding comments because you assume people doesn’t have 
your experience? what’s with that?
Gosh! How do you come to that conclusion, now? And which comments did I 
disregard? I think I disagree with many comments here, but I haven't 
disregarded anything.

bq. This new API makes adds exposure to internal metadata that can't be 
guaranteed to be compatible over versions. By adding this API we are agreeing 
to either maintain compatibility which will make development more complicated, 
or, if we decide not not have compatibility, we hurt users building on top of 
it, and make upgrades more difficult to them.
Lets have it as an internal API. We needn't guarantee backward compatability. 
It is obvious that this is an API that just helps retrieve ZK data, and if the 
structure of the data itself changes, then relying on that data isn't feasible. 
Think of it as a debug API; when a query is issued with debug/explain, then 
internal details are revealed (that may change in subsequent releases).

bq. ... it’s not the right thing for it in the long term
I'm not against improving all other APIs, but who will work on them to make 
sure all possible info is available via those APIs? And how long do you suppose 
we should wait till that is done? And, how is this API any hindrance to 
improving other APIs? And how about removing this endpoint when those other 
APIs are perfect?

bq. Disagree. This goes against the “hide ZooKeeper” strategy everyone agreed 
on recently.
Having this endpoint is very much important to "hide ZooKeeper" (from public 
exposure). No one argued that the data stored in ZK should be hidden away; Solr 
keeps some internal data somewhere (ZK), and expert users must be able to 
access that (in the raw form, if possible).

> /api/cluster/zk/* to fetch raw ZK data
> --
>
> Key: SOLR-13942
> URL: https://issues.apache.org/jira/browse/SOLR-13942
> Project: Solr
>  Issue Type: New Feature
>  Components: v2 API
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> example
> download the {{state.json}} of
> {code}
> GET http://localhost:8983/api/cluster/zk/collections/gettingstarted/state.json
> {code}
> get a list of all children under {{/live_nodes}}
> {code}
> GET http://localhost:8983/api/cluster/zk/live_nodes
> {code}
> If the requested path is a node with children show the list of child nodes 
> and their meta data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-11746) numeric fields need better error handling for prefix/wildcard syntax -- consider uniform support for "foo:* == foo:[* TO *]"

2020-03-04 Thread Cassandra Targett (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051517#comment-17051517
 ] 

Cassandra Targett commented on SOLR-11746:
--

[~tomoko], please check the version of Asciidoctor you are using. That error is 
because of a different way an older version constructs in-page anchor 
references. You should be on 1.5.7 or higher (preferably 2.0+).

This reminded me that I neglected to update the Ref Guide README when I updated 
the versions in SOLR-12786 (you can find the correct versions in 
{{dev-tools/scripts/jenkins.build.ref.guide.sh}} since that's what the build 
uses)...I'll fix the README now.

> numeric fields need better error handling for prefix/wildcard syntax -- 
> consider uniform support for "foo:* == foo:[* TO *]"
> 
>
> Key: SOLR-11746
> URL: https://issues.apache.org/jira/browse/SOLR-11746
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 7.0
>Reporter: Chris M. Hostetter
>Assignee: Houston Putman
>Priority: Major
> Fix For: master (9.0), 8.5
>
> Attachments: SOLR-11746.patch, SOLR-11746.patch, SOLR-11746.patch, 
> SOLR-11746.patch, SOLR-11746.patch, SOLR-11746.patch, SOLR-11746.patch, 
> SOLR-11746.patch, SOLR-11746.patch, SOLR-11746.patch, SOLR-11746.patch
>
>
> On the solr-user mailing list, Torsten Krah pointed out that with Trie 
> numeric fields, query syntax such as {{foo_d:\*}} has been functionality 
> equivilent to {{foo_d:\[\* TO \*]}} and asked why this was not also supported 
> for Point based numeric fields.
> The fact that this type of syntax works (for {{indexed="true"}} Trie fields) 
> appears to have been an (untested, undocumented) fluke of Trie fields given 
> that they use indexed terms for the (encoded) numeric terms and inherit the 
> default implementation of {{FieldType.getPrefixQuery}} which produces a 
> prefix query against the {{""}} (empty string) term.  
> (Note that this syntax has aparently _*never*_ worked for Trie fields with 
> {{indexed="false" docValues="true"}} )
> In general, we should assess the behavior users attempt a prefix/wildcard 
> syntax query against numeric fields, as currently the behavior is largely 
> non-sensical:  prefix/wildcard syntax frequently match no docs w/o any sort 
> of error, and the aformentioned {{numeric_field:*}} behaves inconsistently 
> between points/trie fields and between indexed/docValued trie fields.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14223) PublicKeyHandler consumes a lot of entropy during tests



[ 
https://issues.apache.org/jira/browse/SOLR-14223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051522#comment-17051522
 ] 

ASF subversion and git services commented on SOLR-14223:


Commit dd9b9f2f7f26ecaec3e3bc1e8a0ddfc0f3e72aac in lucene-solr's branch 
refs/heads/master from Mike Drob
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=dd9b9f2 ]

SOLR-14223 Fix tests for windows


> PublicKeyHandler consumes a lot of entropy during tests
> ---
>
> Key: SOLR-14223
> URL: https://issues.apache.org/jira/browse/SOLR-14223
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 7.4, 8.0
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> After the changes in SOLR-12354 to eagerly create a {{PublicKeyHandler}} for 
> the CoreContainer, the creation of the underlying {{RSAKeyPair}} uses 
> {{SecureRandom}} to generate primes. This eats up a lot of system entropy and 
> can slow down tests significantly (I observed it adding 10s to an individual 
> test).
> Similar to what we do for SSL config for tests, we can swap in a non blocking 
> implementation of SecureRandom for the key pair generation to allow multiple 
> tests to run better in parallel. Primality testing with BigInteger is also 
> slow, so I'm not sure how much total speedup we can get here, maybe it's 
> worth checking if there are faster implementations out there in other 
> libraries.
> In production cases, this also blocks creation of all cores. We should only 
> create the Handler if necessary, i.e. if the existing authn/z tell us that 
> they won't support internode requests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-11746) numeric fields need better error handling for prefix/wildcard syntax -- consider uniform support for "foo:* == foo:[* TO *]"

2020-03-04 Thread Cassandra Targett (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051526#comment-17051526
 ] 

Cassandra Targett commented on SOLR-11746:
--

bq. This reminded me that I neglected to update the Ref Guide README 

Scratch that, I did update it. I was looking at an old branch.

However, if you previously had all the pre-reqs already installed and upgraded 
to the versions mentioned in the README, it would not update the asciidoctor 
version [1]. To upgrade Asciidoctor, use:

{code}
gem upgrade asciidoctor
{code}

[1] The reason for this is because the asciidoctor gem is a dependency of 
jekyll-asciidoc so new installs of the pre-reqs get asciidoctor as part of the 
install of jekyll-asciidoc. When you upgrade jekyll-asciidoc, however, it does 
not automatically update it's dependency, it only validates that it is already 
installed.

> numeric fields need better error handling for prefix/wildcard syntax -- 
> consider uniform support for "foo:* == foo:[* TO *]"
> 
>
> Key: SOLR-11746
> URL: https://issues.apache.org/jira/browse/SOLR-11746
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 7.0
>Reporter: Chris M. Hostetter
>Assignee: Houston Putman
>Priority: Major
> Fix For: master (9.0), 8.5
>
> Attachments: SOLR-11746.patch, SOLR-11746.patch, SOLR-11746.patch, 
> SOLR-11746.patch, SOLR-11746.patch, SOLR-11746.patch, SOLR-11746.patch, 
> SOLR-11746.patch, SOLR-11746.patch, SOLR-11746.patch, SOLR-11746.patch
>
>
> On the solr-user mailing list, Torsten Krah pointed out that with Trie 
> numeric fields, query syntax such as {{foo_d:\*}} has been functionality 
> equivilent to {{foo_d:\[\* TO \*]}} and asked why this was not also supported 
> for Point based numeric fields.
> The fact that this type of syntax works (for {{indexed="true"}} Trie fields) 
> appears to have been an (untested, undocumented) fluke of Trie fields given 
> that they use indexed terms for the (encoded) numeric terms and inherit the 
> default implementation of {{FieldType.getPrefixQuery}} which produces a 
> prefix query against the {{""}} (empty string) term.  
> (Note that this syntax has aparently _*never*_ worked for Trie fields with 
> {{indexed="false" docValues="true"}} )
> In general, we should assess the behavior users attempt a prefix/wildcard 
> syntax query against numeric fields, as currently the behavior is largely 
> non-sensical:  prefix/wildcard syntax frequently match no docs w/o any sort 
> of error, and the aformentioned {{numeric_field:*}} behaves inconsistently 
> between points/trie fields and between indexed/docValued trie fields.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-11746) numeric fields need better error handling for prefix/wildcard syntax -- consider uniform support for "foo:* == foo:[* TO *]"

2020-03-04 Thread Cassandra Targett (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051526#comment-17051526
 ] 

Cassandra Targett edited comment on SOLR-11746 at 3/4/20, 6:54 PM:
---

bq. This reminded me that I neglected to update the Ref Guide README 

Scratch that, I did update it. I was looking at an old branch.

However, if you previously had all the pre-reqs already installed and upgraded 
to the versions mentioned in the README, it would not update the asciidoctor 
version [1]. To upgrade Asciidoctor, use:

{code}
gem upgrade asciidoctor
{code}

[1] The reason for this is because the asciidoctor gem is a dependency of 
jekyll-asciidoc so new installs of the pre-reqs get asciidoctor as part of the 
install of jekyll-asciidoc. When you upgrade jekyll-asciidoc, however, it does 
not automatically update it's dependency, it only validates that it is already 
installed.

Edit: Or, you could use the gradle build: {{./gradlew buildSite}} which 
downloads the dependencies even if you have them already.


was (Author: ctargett):
bq. This reminded me that I neglected to update the Ref Guide README 

Scratch that, I did update it. I was looking at an old branch.

However, if you previously had all the pre-reqs already installed and upgraded 
to the versions mentioned in the README, it would not update the asciidoctor 
version [1]. To upgrade Asciidoctor, use:

{code}
gem upgrade asciidoctor
{code}

[1] The reason for this is because the asciidoctor gem is a dependency of 
jekyll-asciidoc so new installs of the pre-reqs get asciidoctor as part of the 
install of jekyll-asciidoc. When you upgrade jekyll-asciidoc, however, it does 
not automatically update it's dependency, it only validates that it is already 
installed.

> numeric fields need better error handling for prefix/wildcard syntax -- 
> consider uniform support for "foo:* == foo:[* TO *]"
> 
>
> Key: SOLR-11746
> URL: https://issues.apache.org/jira/browse/SOLR-11746
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 7.0
>Reporter: Chris M. Hostetter
>Assignee: Houston Putman
>Priority: Major
> Fix For: master (9.0), 8.5
>
> Attachments: SOLR-11746.patch, SOLR-11746.patch, SOLR-11746.patch, 
> SOLR-11746.patch, SOLR-11746.patch, SOLR-11746.patch, SOLR-11746.patch, 
> SOLR-11746.patch, SOLR-11746.patch, SOLR-11746.patch, SOLR-11746.patch
>
>
> On the solr-user mailing list, Torsten Krah pointed out that with Trie 
> numeric fields, query syntax such as {{foo_d:\*}} has been functionality 
> equivilent to {{foo_d:\[\* TO \*]}} and asked why this was not also supported 
> for Point based numeric fields.
> The fact that this type of syntax works (for {{indexed="true"}} Trie fields) 
> appears to have been an (untested, undocumented) fluke of Trie fields given 
> that they use indexed terms for the (encoded) numeric terms and inherit the 
> default implementation of {{FieldType.getPrefixQuery}} which produces a 
> prefix query against the {{""}} (empty string) term.  
> (Note that this syntax has aparently _*never*_ worked for Trie fields with 
> {{indexed="false" docValues="true"}} )
> In general, we should assess the behavior users attempt a prefix/wildcard 
> syntax query against numeric fields, as currently the behavior is largely 
> non-sensical:  prefix/wildcard syntax frequently match no docs w/o any sort 
> of error, and the aformentioned {{numeric_field:*}} behaves inconsistently 
> between points/trie fields and between indexed/docValued trie fields.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9249) Verify contents of LICENSE and NOTICE files

2020-03-04 Thread Mike Drob (Jira)



[ 
https://issues.apache.org/jira/browse/LUCENE-9249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051531#comment-17051531
 ] 

Mike Drob commented on LUCENE-9249:
---

As best as I can tell the simplified rules for this are:
 * Contents of LICENSE and NOTICE apply to what they are distributed with 
(source, convenience binary) rather than the contents of the repo.
 * LICENSE contains the full text of the ASLv2, and also the full text of other 
licenses that apply. This may be code copied into our source tree, or through 
dependencies bundled in the binary.
 * Anything developed at the ASF goes in NOTICE under "This project includes 
software developed at the ASF (c) [dates]"
 * Weak copyleft licenses (Category B) usually have a NOTICE requirement

> Verify contents of LICENSE and NOTICE files
> ---
>
> Key: LUCENE-9249
> URL: https://issues.apache.org/jira/browse/LUCENE-9249
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Mike Drob
>Priority: Critical
>
> Based on question in Slack, it sounds like somebody needs to take a look at 
> our LICENSE and NOTICE files for both Lucene and Solr to make sure that we 
> are including the right things (and only what is necessary).
> For reference: 
> http://www.apache.org/dev/licensing-howto.html#overview-of-files



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14254) Index backcompat break between 8.3.1 and 8.4.1

2020-03-04 Thread David Smiley (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051532#comment-17051532
 ] 

David Smiley commented on SOLR-14254:
-

I'll add a PR here to improve the ref guide guidance on tagger to show the 
FST50 as a performance trade-off tip and not a must-have.  I'm not sure if that 
"resolves" this issue or not; there is a back-compat break and the best course 
of action is documentation here.

> Index backcompat break between 8.3.1 and 8.4.1
> --
>
> Key: SOLR-14254
> URL: https://issues.apache.org/jira/browse/SOLR-14254
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Jason Gerlowski
>Priority: Major
>
> I believe I found a backcompat break between 8.4.1 and 8.3.1.
> I encountered this when a Solr 8.3.1 cluster was upgraded to 8.4.1.  On 8.4. 
> nodes, several collections had cores fail to come up with 
> {{CorruptIndexException}}:
> {code}
> 2020-02-10 20:58:26.136 ERROR 
> (coreContainerWorkExecutor-2-thread-1-processing-n:192.168.1.194:8983_solr) [ 
>   ] o.a.s.c.CoreContainer Error waiting for SolrCore to be loaded on startup 
> => org.apache.sol
> r.common.SolrException: Unable to create core 
> [testbackcompat_shard1_replica_n1]
> at 
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1313)
> org.apache.solr.common.SolrException: Unable to create core 
> [testbackcompat_shard1_replica_n1]
> at 
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1313)
>  ~[?:?]
> at 
> org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:788) 
> ~[?:?]
> at 
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:202)
>  ~[metrics-core-4.0.5.jar:4.0.5]
> at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
> at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
>  ~[?:?]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>  ~[?:?]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>  ~[?:?]
> at java.lang.Thread.run(Thread.java:834) [?:?]
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.(SolrCore.java:1072) ~[?:?]
> at org.apache.solr.core.SolrCore.(SolrCore.java:901) ~[?:?]
> at 
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1292)
>  ~[?:?]
> ... 7 more
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2182) 
> ~[?:?]
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:2302) 
> ~[?:?]
> at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:1132) 
> ~[?:?]
> at org.apache.solr.core.SolrCore.(SolrCore.java:1013) ~[?:?]
> at org.apache.solr.core.SolrCore.(SolrCore.java:901) ~[?:?]
> at 
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1292)
>  ~[?:?]
> ... 7 more
> Caused by: org.apache.lucene.index.CorruptIndexException: codec mismatch: 
> actual codec=Lucene50PostingsWriterDoc vs expected 
> codec=Lucene84PostingsWriterDoc 
> (resource=MMapIndexInput(path="/Users/jasongerlowski/run/solrdata/data/testbackcompat_shard1_replica_n1/data/index/_0_FST50_0.doc"))
> at 
> org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:208) 
> ~[?:?]
> at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:198) 
> ~[?:?]
> at 
> org.apache.lucene.codecs.CodecUtil.checkIndexHeader(CodecUtil.java:255) ~[?:?]
> at 
> org.apache.lucene.codecs.lucene84.Lucene84PostingsReader.(Lucene84PostingsReader.java:82)
>  ~[?:?]
> at 
> org.apache.lucene.codecs.memory.FSTPostingsFormat.fieldsProducer(FSTPostingsFormat.java:66)
>  ~[?:?]
> at 
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:315)
>  ~[?:?]
> at 
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:395)
>  ~[?:?]
> at 
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:114)
>  ~[?:?]
> at 
> org.apache.lucene.index.SegmentReader.(SegmentReader.java:84) ~[?:?]
> at 
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:177)
>  ~[?:?]
> at 
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:219)
>  ~[?:?]
> at 
> org.apache.lucene

[GitHub] [lucene-site] uschindler commented on issue #15: Conditionally show security warning block only if news last 60 days.