date:20191030

CaoManhDat commented on a change in pull request #951: SOLR-13844: Remove 
replica recovery terms with the replica term
URL: https://github.com/apache/lucene-solr/pull/951#discussion_r340544645
 
 

 ##
 File path: solr/core/src/java/org/apache/solr/cloud/ZkShardTerms.java
 ##
 @@ -524,15 +529,23 @@ Terms ensureHighestTermsAreNotZero() {
 }
 
 /**
- * Return a new {@link Terms} in which term of {@code coreNodeName} is 
removed
+ * Return a new {@link Terms} in which terms for the {@code coreNodeName} 
are removed
  * @param coreNodeName of the replica
  * @return null if term of {@code coreNodeName} is already not exist
  */
 Terms removeTerm(String coreNodeName) {
-  if (!values.containsKey(coreNodeName)) return null;
+  if (!values.containsKey(recoveringTerm(coreNodeName)) && 
!values.containsKey(coreNodeName)) {
+return null;
+  }
 
   HashMap newValues = new HashMap<>(values);
-  newValues.remove(coreNodeName);
+  if (values.containsKey(coreNodeName)) {
 
 Review comment:
   doesn't require a check here. Just blindly remove will be enough


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] CaoManhDat commented on issue #951: SOLR-13844: Remove replica recovery terms with the replica term

CaoManhDat commented on issue #951: SOLR-13844: Remove replica recovery terms 
with the replica term
URL: https://github.com/apache/lucene-solr/pull/951#issuecomment-547842478
 
 
   Just minor change otherwise LGTM!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9028) Public method for MultiTermIntervalSource

2019-10-30 Thread Lucene/Solr QA (Jira)



[ 
https://issues.apache.org/jira/browse/LUCENE-9028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16962920#comment-16962920
 ] 

Lucene/Solr QA commented on LUCENE-9028:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} | 
{color:green}  0m 20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} | 
{color:green}  0m 20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} | 
{color:green}  0m 20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
20s{color} | {color:green} queries in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}  4m 36s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | LUCENE-9028 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12984305/LUCENE-9028.patch |
| Optional Tests |  compile  javac  unit  ratsources  checkforbiddenapis  
validatesourcepatterns  |
| uname | Linux lucene2-us-west.apache.org 4.4.0-112-generic #135-Ubuntu SMP 
Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-LUCENE-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
 |
| git revision | master / 22b6817 |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 |
| Default Java | LTS |
|  Test Results | 
https://builds.apache.org/job/PreCommit-LUCENE-Build/216/testReport/ |
| modules | C: lucene/queries U: lucene/queries |
| Console output | 
https://builds.apache.org/job/PreCommit-LUCENE-Build/216/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Public method for MultiTermIntervalSource
> -
>
> Key: LUCENE-9028
> URL: https://issues.apache.org/jira/browse/LUCENE-9028
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Major
> Attachments: LUCENE-9028.patch
>
>
> Right now we have prefix and widlcard for multiterm Intervals. Sometimes it's 
> necessary to provide terms set in a more generic way as automaton.
> {code:java}
> Intervals.multiterm(CompiledAutomaton automaton, int maxExpansions, String 
> pattern)
> {code}
> As a benefit we can handle it more efficient rather than or over terms. 
> What do you think? 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-13882) Collections API COLSTATUS does not check live_nodes when reporting replica's status



 [ 
https://issues.apache.org/jira/browse/SOLR-13882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson updated SOLR-13882:
--
Summary: Collections API COLSTATUS does not check live_nodes when reporting 
replica's status  (was: Collections API COLSTATUS does not check live_nodes)

> Collections API COLSTATUS does not check live_nodes when reporting replica's 
> status
> ---
>
> Key: SOLR-13882
> URL: https://issues.apache.org/jira/browse/SOLR-13882
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Priority: Major
>
> The COLSTATUS command will report all replicas as "active" even when the node 
> is not in live_nodes.
> To reproduce:
>  * Start two Solr instances
>  * create a collection with replicas on both
>  * issue a "kill -9" to one of the Solr instances.
>  * issue the COLSTATUS command
> Result:
> {code}
> {
>  "responseHeader":{
>"status":0,
>"QTime":7},
>"gettingstarted":{
>  "stateFormat":2,
>  "znodeVersion":15,
>  "properties":{
> "autoAddReplicas":"false",
>  "maxShardsPerNode":"-1",
> "nrtReplicas":"2",
> "pullReplicas":"0",
> "replicationFactor":"2",
> "router":\{"name":"compositeId"},
> "tlogReplicas":"0"},
> "activeShards":2,
> "inactiveShards":0,
> "schemaNonCompliant":["(NONE)"],
> "shards":{
>"shard1":{
>"state":"active",
>"range":"8000-",
>"replicas":{
>"total":2,
> #  "active":2,
> #  "down":0,
>"recovering":0,
>   "recovery_failed":0},
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (SOLR-13882) Collections API COLSTATUS does not check live_nodes

Erick Erickson created SOLR-13882:
-

 Summary: Collections API COLSTATUS does not check live_nodes
 Key: SOLR-13882
 URL: https://issues.apache.org/jira/browse/SOLR-13882
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Erick Erickson


The COLSTATUS command will report all replicas as "active" even when the node 
is not in live_nodes.

To reproduce:
 * Start two Solr instances
 * create a collection with replicas on both
 * issue a "kill -9" to one of the Solr instances.
 * issue the COLSTATUS command

Result:
{code}
{
 "responseHeader":{
   "status":0,
   "QTime":7},
   "gettingstarted":{
 "stateFormat":2,
 "znodeVersion":15,
 "properties":{
"autoAddReplicas":"false",
 "maxShardsPerNode":"-1",
"nrtReplicas":"2",
"pullReplicas":"0",
"replicationFactor":"2",
"router":\{"name":"compositeId"},
"tlogReplicas":"0"},
"activeShards":2,
"inactiveShards":0,
"schemaNonCompliant":["(NONE)"],
"shards":{
   "shard1":{
   "state":"active",
   "range":"8000-",
   "replicas":{
   "total":2,
#  "active":2,
#  "down":0,
   "recovering":0,
  "recovery_failed":0},
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13882) Collections API COLSTATUS does not check live_nodes when reporting replica's status



[ 
https://issues.apache.org/jira/browse/SOLR-13882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16962984#comment-16962984
 ] 

Erick Erickson commented on SOLR-13882:
---

Actually, I didn't check specifically whether COLSTATUS checks live_nodes or 
not, but that's what I'd bet. At any rate, when a node is forcefully killed, 
the command shows all replicas as "active".

> Collections API COLSTATUS does not check live_nodes when reporting replica's 
> status
> ---
>
> Key: SOLR-13882
> URL: https://issues.apache.org/jira/browse/SOLR-13882
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Priority: Major
>
> The COLSTATUS command will report all replicas as "active" even when the node 
> is not in live_nodes.
> To reproduce:
>  * Start two Solr instances
>  * create a collection with replicas on both
>  * issue a "kill -9" to one of the Solr instances.
>  * issue the COLSTATUS command
> Result:
> {code}
> {
>  "responseHeader":{
>"status":0,
>"QTime":7},
>"gettingstarted":{
>  "stateFormat":2,
>  "znodeVersion":15,
>  "properties":{
> "autoAddReplicas":"false",
>  "maxShardsPerNode":"-1",
> "nrtReplicas":"2",
> "pullReplicas":"0",
> "replicationFactor":"2",
> "router":\{"name":"compositeId"},
> "tlogReplicas":"0"},
> "activeShards":2,
> "inactiveShards":0,
> "schemaNonCompliant":["(NONE)"],
> "shards":{
>"shard1":{
>"state":"active",
>"range":"8000-",
>"replicas":{
>"total":2,
> #  "active":2,
> #  "down":0,
>"recovering":0,
>   "recovery_failed":0},
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-13882) Collections API COLSTATUS does not check live_nodes when reporting replica's status



 [ 
https://issues.apache.org/jira/browse/SOLR-13882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson updated SOLR-13882:
--
Attachment: SOLR-13882.patch
Status: Open  (was: Open)

This fixes the problem, I only tested with a single manual test. Needs a test 
though, perhaps in {code}CollectionsAPISolrJTest.testColStatus{code}?

I won't pursue this ATM, so anyone who wants to pick it up and add a test, 
please do.

NOTE: the replicas will be reported as "active" for some time after the node is 
killed while waiting for Zookeeper to remove the entry from live_nodes.

> Collections API COLSTATUS does not check live_nodes when reporting replica's 
> status
> ---
>
> Key: SOLR-13882
> URL: https://issues.apache.org/jira/browse/SOLR-13882
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Priority: Major
> Attachments: SOLR-13882.patch
>
>
> The COLSTATUS command will report all replicas as "active" even when the node 
> is not in live_nodes.
> To reproduce:
>  * Start two Solr instances
>  * create a collection with replicas on both
>  * issue a "kill -9" to one of the Solr instances.
>  * issue the COLSTATUS command
> Result:
> {code}
> {
>  "responseHeader":{
>"status":0,
>"QTime":7},
>"gettingstarted":{
>  "stateFormat":2,
>  "znodeVersion":15,
>  "properties":{
> "autoAddReplicas":"false",
>  "maxShardsPerNode":"-1",
> "nrtReplicas":"2",
> "pullReplicas":"0",
> "replicationFactor":"2",
> "router":\{"name":"compositeId"},
> "tlogReplicas":"0"},
> "activeShards":2,
> "inactiveShards":0,
> "schemaNonCompliant":["(NONE)"],
> "shards":{
>"shard1":{
>"state":"active",
>"range":"8000-",
>"replicas":{
>"total":2,
> #  "active":2,
> #  "down":0,
>"recovering":0,
>   "recovery_failed":0},
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-13882) Collections API COLSTATUS does not check live_nodes when reporting replica's status



[ 
https://issues.apache.org/jira/browse/SOLR-13882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16962997#comment-16962997
 ] 

Erick Erickson edited comment on SOLR-13882 at 10/30/19 12:59 PM:
--

This fixes the problem, I only tested with a single manual test. Needs a test 
though, perhaps in {code}CollectionsAPISolrJTest.testColStatus{code}?

I won't pursue this ATM, so anyone who wants to pick it up and add a test, 
please do.

NOTE: the replicas will be reported as "active" for some time after the node is 
killed while waiting for Zookeeper to remove the entry from live_nodes.

[~ab]Doe reporting this state as "down" break anything you know of?

For discussion: {code}Replica.isActive() and Replica.getState(){code} are 
trappy. It's perfectly reasonable to think "If the replica says it's active, it 
must be". I've spent time debugging this, here's another case where it's an 
issue, I've seen this mentioned more than a few times on the dev and user's 
list.

I know it'd be changes to a number of places in the code, and there are some 
legitimate places where {code}Replica.isActive(){code} _should_ return "active" 
when the node is _not_ in live_nodes. That said, what do people think about:

- changing at least these two methods to return "down" if the replica's node 
isn't in live_nodes

- creating some "expert" level method to return what {{getState()}} does now 
(i.e return "active" if the state is "active" but the node isn't in live_nodes) 
for those cases that need to make a distinction


was (Author: erickerickson):
This fixes the problem, I only tested with a single manual test. Needs a test 
though, perhaps in {code}CollectionsAPISolrJTest.testColStatus{code}?

I won't pursue this ATM, so anyone who wants to pick it up and add a test, 
please do.

NOTE: the replicas will be reported as "active" for some time after the node is 
killed while waiting for Zookeeper to remove the entry from live_nodes.

> Collections API COLSTATUS does not check live_nodes when reporting replica's 
> status
> ---
>
> Key: SOLR-13882
> URL: https://issues.apache.org/jira/browse/SOLR-13882
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Priority: Major
> Attachments: SOLR-13882.patch
>
>
> The COLSTATUS command will report all replicas as "active" even when the node 
> is not in live_nodes.
> To reproduce:
>  * Start two Solr instances
>  * create a collection with replicas on both
>  * issue a "kill -9" to one of the Solr instances.
>  * issue the COLSTATUS command
> Result:
> {code}
> {
>  "responseHeader":{
>"status":0,
>"QTime":7},
>"gettingstarted":{
>  "stateFormat":2,
>  "znodeVersion":15,
>  "properties":{
> "autoAddReplicas":"false",
>  "maxShardsPerNode":"-1",
> "nrtReplicas":"2",
> "pullReplicas":"0",
> "replicationFactor":"2",
> "router":\{"name":"compositeId"},
> "tlogReplicas":"0"},
> "activeShards":2,
> "inactiveShards":0,
> "schemaNonCompliant":["(NONE)"],
> "shards":{
>"shard1":{
>"state":"active",
>"range":"8000-",
>"replicas":{
>"total":2,
> #  "active":2,
> #  "down":0,
>"recovering":0,
>   "recovery_failed":0},
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9013) Migrate existing site content (except javadoc, refguide)



[ 
https://issues.apache.org/jira/browse/LUCENE-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963019#comment-16963019
 ] 

Jan Høydahl commented on LUCENE-9013:
-

[~uschindler], [~dweiss], or others: Any concerns in force-pushing to the 
lucene-site repo which is practically empty and very few will have a checkout 
of it? Should we force-push to GitHub or gitbox, and will force-pushing mess up 
the syncing between them?

> Migrate existing site content (except javadoc, refguide)
> 
>
> Key: LUCENE-9013
> URL: https://issues.apache.org/jira/browse/LUCENE-9013
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Priority: Major
> Attachments: Screen Shot 2019-10-29 at 12.29.49 PM.png
>
>
> Resulting in a successful local build



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9013) Migrate existing site content (except javadoc, refguide)



[ 
https://issues.apache.org/jira/browse/LUCENE-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963040#comment-16963040
 ] 

Jan Høydahl commented on LUCENE-9013:
-

Chatted with infra:
{quote}[Jan Høydahl|https://app.slack.com/team/UEA43LR36][2:54 
PM|https://the-asf.slack.com/archives/CBX4TSBQ8/p1572443646484700]
Hi, we have a new non-prod git repo ‘lucene-site’ for the Lucene site 
migration, with one commit. In 
https://issues.apache.org/jira/browse/LUCENE-9013 we plan to force-push the svn 
history into this repo. Is that safe without involvement from INFRA,a nd should 
we push to gitbox or github?
 
[Daniel Gruno|https://app.slack.com/team/UBY0LUX0D][2:55 
PM|https://the-asf.slack.com/archives/CBX4TSBQ8/p1572443717485400]
[@Jan Høydahl|https://the-asf.slack.com/team/UEA43LR36] I would push that to 
gitbox if it's a very large payload at first
 
[Jan Høydahl|https://app.slack.com/team/UEA43LR36][2:56 
PM|https://the-asf.slack.com/archives/CBX4TSBQ8/p1572443801486600]
The branch we want to push is rewriting history (overwriting the single commit 
currently there). Will that sync nicely over to github?
 
[Daniel Gruno|https://app.slack.com/team/UBY0LUX0D] [2:57 
PM|https://the-asf.slack.com/archives/CBX4TSBQ8/p1572443831486800]
it should, yes
[Jan Høydahl|https://app.slack.com/team/UEA43LR36][2:58 
PM|https://the-asf.slack.com/archives/CBX4TSBQ8/p1572443902487900]Ok. Hmm, 
there are 1394 commits we’ll push. Hope that won’t mean 1394 emails?? 
!https://a.slack-edge.com/production-standard-emoji-assets/10.2/apple-medium/1f...@2x.png!
 
[Daniel Gruno|https://app.slack.com/team/UBY0LUX0D] [2:59 
PM|https://the-asf.slack.com/archives/CBX4TSBQ8/p1572443958488100]I dunno 
¯\_(ツ)_/¯
 
[2:59 PM|https://the-asf.slack.com/archives/CBX4TSBQ8/p1572443976488500]
we'll see? 
!https://a.slack-edge.com/production-standard-emoji-assets/10.2/apple-medium/1f...@2x.png!
 I think there's a hardcoded max of 50
[3:00 PM|https://the-asf.slack.com/archives/CBX4TSBQ8/p1572444030488900]
actually, I might be able to change that to a smaller number temporarily
[3:00 PM|https://the-asf.slack.com/archives/CBX4TSBQ8/p1572444058489100]
I've set it to 10 max now
new messages
[3:01 PM|https://the-asf.slack.com/archives/CBX4TSBQ8/p1572444075489400]
so if you push it all in one go, you'll get max 10 emails
 {quote}
Will push soon 

> Migrate existing site content (except javadoc, refguide)
> 
>
> Key: LUCENE-9013
> URL: https://issues.apache.org/jira/browse/LUCENE-9013
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Priority: Major
> Attachments: Screen Shot 2019-10-29 at 12.29.49 PM.png
>
>
> Resulting in a successful local build



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9013) Migrate existing site content (except javadoc, refguide)

2019-10-30 Thread Dawid Weiss (Jira)



[ 
https://issues.apache.org/jira/browse/LUCENE-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963047#comment-16963047
 ] 

Dawid Weiss commented on LUCENE-9013:
-

Since that repo ([https://github.com/apache/lucene-site]) is empty I don't see 
a problem. Although it could as well be imported as a different branch and 
merged in entirety on top of the existing single commit? Doesn't matter to me.

> Migrate existing site content (except javadoc, refguide)
> 
>
> Key: LUCENE-9013
> URL: https://issues.apache.org/jira/browse/LUCENE-9013
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Priority: Major
> Attachments: Screen Shot 2019-10-29 at 12.29.49 PM.png
>
>
> Resulting in a successful local build



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13506) Upgrade carrot2-guava-*.jar

2019-10-30 Thread DW (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963055#comment-16963055
 ] 

DW commented on SOLR-13506:
---

Hello Dawid, do you know when it will happen?

> Upgrade carrot2-guava-*.jar 
> 
>
> Key: SOLR-13506
> URL: https://issues.apache.org/jira/browse/SOLR-13506
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - Clustering
>Affects Versions: 7.7.1, 8.0, 8.1
>Reporter: DW
>Assignee: Dawid Weiss
>Priority: Major
>
> The Solr package contains /contrib/clustering/lib/carrot2-guava-18.0.jar.
> [cpe:/a:google:guava:18.0|https://web.nvd.nist.gov/view/vuln/search-results?adv_search=true&cves=on&cpe_version=cpe%3A%2Fa%3Agoogle%3Aguava%3A18.0]
>  has know security vulnerabilities. 
> Can you please upgrade the library or remove if not needed.
> Thanks.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9013) Migrate existing site content (except javadoc, refguide)



[ 
https://issues.apache.org/jira/browse/LUCENE-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963060#comment-16963060
 ] 

Jan Høydahl commented on LUCENE-9013:
-

I force pushed to gitbox. See logs in 
[https://gist.github.com/janhoy/0920531abf264c4c1e534ecb4c8f5f19]

INFRA had to disable force push protection for master, and also tuned down 
number of commit emails sent from 50 to 10 :) 

Hopefully it will sync well to GitHub 

> Migrate existing site content (except javadoc, refguide)
> 
>
> Key: LUCENE-9013
> URL: https://issues.apache.org/jira/browse/LUCENE-9013
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Priority: Major
> Attachments: Screen Shot 2019-10-29 at 12.29.49 PM.png
>
>
> Resulting in a successful local build



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13506) Upgrade carrot2-guava-*.jar

2019-10-30 Thread Dawid Weiss (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963063#comment-16963063
 ] 

Dawid Weiss commented on SOLR-13506:


I hope sooner than later... We have been working on a beta release with the new 
API and it doesn't have guava dependency. I don't think the vulnerabilities of 
the original guava are exploitable if those affected classes are not really 
used.

> Upgrade carrot2-guava-*.jar 
> 
>
> Key: SOLR-13506
> URL: https://issues.apache.org/jira/browse/SOLR-13506
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - Clustering
>Affects Versions: 7.7.1, 8.0, 8.1
>Reporter: DW
>Assignee: Dawid Weiss
>Priority: Major
>
> The Solr package contains /contrib/clustering/lib/carrot2-guava-18.0.jar.
> [cpe:/a:google:guava:18.0|https://web.nvd.nist.gov/view/vuln/search-results?adv_search=true&cves=on&cpe_version=cpe%3A%2Fa%3Agoogle%3Aguava%3A18.0]
>  has know security vulnerabilities. 
> Can you please upgrade the library or remove if not needed.
> Thanks.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Assigned] (LUCENE-9013) Migrate existing site content (except javadoc, refguide)



 [ 
https://issues.apache.org/jira/browse/LUCENE-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl reassigned LUCENE-9013:
---

Assignee: Jan Høydahl

> Migrate existing site content (except javadoc, refguide)
> 
>
> Key: LUCENE-9013
> URL: https://issues.apache.org/jira/browse/LUCENE-9013
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: Screen Shot 2019-10-29 at 12.29.49 PM.png
>
>
> Resulting in a successful local build



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9013) Migrate existing site content (except javadoc, refguide)



[ 
https://issues.apache.org/jira/browse/LUCENE-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963065#comment-16963065
 ] 

Jan Høydahl commented on LUCENE-9013:
-

Synced to GitHub too. Will close this sub task now and you can move on to 
creating a PR for the pelican stuff [~adamwalz].

[~dweiss] thanks for chiming in. Needed help from INFRA to enable push to 
master, and they also needed to do a manual force-push to GitHub mirror since 
the auto sync barfed :) 

> Migrate existing site content (except javadoc, refguide)
> 
>
> Key: LUCENE-9013
> URL: https://issues.apache.org/jira/browse/LUCENE-9013
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Priority: Major
> Attachments: Screen Shot 2019-10-29 at 12.29.49 PM.png
>
>
> Resulting in a successful local build



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-9013) Migrate existing site content (except javadoc, refguide)



 [ 
https://issues.apache.org/jira/browse/LUCENE-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl resolved LUCENE-9013.
-
Resolution: Fixed

> Migrate existing site content (except javadoc, refguide)
> 
>
> Key: LUCENE-9013
> URL: https://issues.apache.org/jira/browse/LUCENE-9013
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: Screen Shot 2019-10-29 at 12.29.49 PM.png
>
>
> Resulting in a successful local build



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8987) Move Lucene web site from svn to git



[ 
https://issues.apache.org/jira/browse/LUCENE-8987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963067#comment-16963067
 ] 

Jan Høydahl commented on LUCENE-8987:
-

SVN history imported into new branch, thanks [~adamwalz]. One sub task closed, 
five to go :) 

> Move Lucene web site from svn to git
> 
>
> Key: LUCENE-8987
> URL: https://issues.apache.org/jira/browse/LUCENE-8987
> Project: Lucene - Core
>  Issue Type: Task
>  Components: general/website
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: lucene-site-repo.png
>
>
> INFRA just enabled [a new way of configuring website 
> build|https://s.apache.org/asfyaml] from a git branch, [see dev list 
> email|https://lists.apache.org/thread.html/b6f7e40bece5e83e27072ecc634a7815980c90240bc0a2ccb417f1fd@%3Cdev.lucene.apache.org%3E].
>  It allows for automatic builds of both staging and production site, much 
> like the old CMS. We can choose to auto publish the html content of an 
> {{output/}} folder, or to have a bot build the site using 
> [Pelican|https://github.com/getpelican/pelican] from a {{content/}} folder.
> The goal of this issue is to explore how this can be done for 
> [http://lucene.apache.org|http://lucene.apache.org/] by, by creating a new 
> git repo {{lucene-site}}, copy over the site from svn, see if it can be 
> "Pelicanized" easily and then test staging. Benefits are that more people 
> will be able to edit the web site and we can take PRs from the public (with 
> GitHub preview of pages).
> Non-goals:
>  * Create a new web site or a new graphic design
>  * Change from Markdown to Asciidoc



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9013) Migrate existing site content (except javadoc, refguide)



[ 
https://issues.apache.org/jira/browse/LUCENE-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963071#comment-16963071
 ] 

Uwe Schindler commented on LUCENE-9013:
---

As Infra said: Always force-push to Gitbox/ASF. No more info that I have, I 
just remember when we changed SVN->Git for main repo, force pushes worked. We 
just disabled it after initial setup of repo.

> Migrate existing site content (except javadoc, refguide)
> 
>
> Key: LUCENE-9013
> URL: https://issues.apache.org/jira/browse/LUCENE-9013
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: Screen Shot 2019-10-29 at 12.29.49 PM.png
>
>
> Resulting in a successful local build



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-site] adamwalz opened a new pull request #1: LUCENE-9012 Setup minimal site with working Pelican build

adamwalz opened a new pull request #1: LUCENE-9012 Setup minimal site with 
working Pelican build
URL: https://github.com/apache/lucene-site/pull/1
 
 
   This PR contains a Pelican-ized version of lucene.apache.org for all 
sub-projects except for solr. The TLP page, Lucene Core, PyLucene, and Open 
Relevance all have a similar structure whereas the solr pages are pretty 
different. It would make sense for the solr pages to be reviewed in a different 
PR.
   
   Since Pelican is new for this repo I have tried to keep commits logically 
separated to make it easier to follow the transition, as well as preserve the 
ability to `git blame` line by line to see when content was originally added. 
For this reason it would be best to rebase this PR rather than squash and 
merge. Rebasing will keep a history of renamed files from `git mv` whereas 
squashing will lose that history.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9012) Setup minimal site with working Pelican build

2019-10-30 Thread Adam Walz (Jira)



[ 
https://issues.apache.org/jira/browse/LUCENE-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963072#comment-16963072
 ] 

Adam Walz commented on LUCENE-9012:
---

[~janhoy] Now that the cms version history is merged into github, this PR is 
ready to be reviewed [https://github.com/apache/lucene-site/pull/1]

> Setup minimal site with working Pelican build
> -
>
> Key: LUCENE-9012
> URL: https://issues.apache.org/jira/browse/LUCENE-9012
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-8987) Move Lucene web site from svn to git



[ 
https://issues.apache.org/jira/browse/LUCENE-8987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16962044#comment-16962044
 ] 

Uwe Schindler edited comment on LUCENE-8987 at 10/30/19 2:49 PM:
-

Hi,
I was talking last week with Infra on Apachecon:
- We should only move our website (the CMS) to pelican and GIT.
- We should *not* move or commit any javadocs or other non-CMS generated 
content (refguide) to the CMS Git. This does not scale and is also not wanted. 
The Git repository should be used to manage the markdown sources and the 
resulting static HTML pages after staging, not content that's "one-time write" 
like refguide or javadocs. Those should be Subversion.
- Infra fully supports Subversion for static webpages and will forever do. One 
example is e.g. https://dist.apache.org and https://www.apache.org/dist/, which 
is stored in SVN.
- It's not yet clearly defined how the whole thing would be administered/setup: 
E.g., it could be "Alias" directive in the HTTP Server config or similar. We 
should in all cases open a new Issue with INFRA to set this up.

We should open a new INFRA issue for the one-time-write javadocs/refguide pages 
and its integration external to pelican.


was (Author: thetaphi):
Hi,
I was talking last week with Infra on Apachecon:
- We should only move our website (the CMS) to pelican and GIT.
- We should *not* move or commit any javadocs or other non-CMS generated 
content (refguide) to the CMS Git. This does not scale and is also not wanted. 
The Git repository should be used to manage the markdown sources and the 
resulting static HTML pages after staging, not content that's "one-time write" 
like refguide or javadocs. Those should be Subversion.
- Infra fully supports Subversion for static webpages and will forever do. One 
example is e.g. dist.apache.org which is stored in SVN.
- It's not yet clearly defined how the whole thing would be administered/setup: 
E.g., it could be "Alias" directive in the HTTP Server config or similar. We 
should in all cases open a new Issue with INFRA to set this up.

We should open a new INFRA issue for the one-time-write javadocs/refguide pages 
and its integration external to pelican.

> Move Lucene web site from svn to git
> 
>
> Key: LUCENE-8987
> URL: https://issues.apache.org/jira/browse/LUCENE-8987
> Project: Lucene - Core
>  Issue Type: Task
>  Components: general/website
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: lucene-site-repo.png
>
>
> INFRA just enabled [a new way of configuring website 
> build|https://s.apache.org/asfyaml] from a git branch, [see dev list 
> email|https://lists.apache.org/thread.html/b6f7e40bece5e83e27072ecc634a7815980c90240bc0a2ccb417f1fd@%3Cdev.lucene.apache.org%3E].
>  It allows for automatic builds of both staging and production site, much 
> like the old CMS. We can choose to auto publish the html content of an 
> {{output/}} folder, or to have a bot build the site using 
> [Pelican|https://github.com/getpelican/pelican] from a {{content/}} folder.
> The goal of this issue is to explore how this can be done for 
> [http://lucene.apache.org|http://lucene.apache.org/] by, by creating a new 
> git repo {{lucene-site}}, copy over the site from svn, see if it can be 
> "Pelicanized" easily and then test staging. Benefits are that more people 
> will be able to edit the web site and we can take PRs from the public (with 
> GitHub preview of pages).
> Non-goals:
>  * Create a new web site or a new graphic design
>  * Change from Markdown to Asciidoc



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-site] adamwalz commented on issue #1: LUCENE-9012 Setup minimal site with working Pelican build

adamwalz commented on issue #1: LUCENE-9012 Setup minimal site with working 
Pelican build
URL: https://github.com/apache/lucene-site/pull/1#issuecomment-547945666
 
 
   One reason for the massive number of changed files (711) is that I broke up 
the news files into one file per news item. This allows for limiting the number 
of news items in the TLP page to the most recent N without having to delete old 
events from the markdown.
   
   ```
   {% for article in articles[:15] %}
   {{ article.content }}
   {{ endfor }}
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] yonik opened a new pull request #983: SOLR-13101: many updates, including concurrent update request handling

2019-10-30 Thread ASF subversion and git services (Jira)

yonik opened a new pull request #983: SOLR-13101: many updates, including 
concurrent update request handling
URL: https://github.com/apache/lucene-solr/pull/983
 
 
   Here's an update to the shared storage branch that includes, among many 
other changes, support for concurrent update requests.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9027) SIMD-based decoding of postings lists



[ 
https://issues.apache.org/jira/browse/LUCENE-9027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963084#comment-16963084
 ] 

ASF subversion and git services commented on LUCENE-9027:
-

Commit f6c8940a1d23be4f0f0d1a2800938e6cb5eb2b55 in lucene-solr's branch 
refs/heads/simd from Adrien Grand
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=f6c8940 ]

LUCENE-9027: Use SIMD instructions to decode postings.


> SIMD-based decoding of postings lists
> -
>
> Key: LUCENE-9027
> URL: https://issues.apache.org/jira/browse/LUCENE-9027
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> [~rcmuir] has been mentioning the idea for quite some time that we might be 
> able to write the decoding logic in such a way that Java would use SIMD 
> instructions. More recently [~paul.masurel] wrote a [blog 
> post|https://fulmicoton.com/posts/bitpacking/] that raises the point that 
> Lucene could still do decode multiple ints at once in a single instruction by 
> packing two ints in a long and we've had some discussions about what we could 
> try in Lucene to speed up the decoding of postings. This made me want to look 
> a bit deeper at what we could do.
> Our current decoding logic reads data in a byte[] and decodes packed integers 
> from it. Unfortunately it doesn't make use of SIMD instructions and looks 
> like 
> [this|https://github.com/jpountz/decode-128-ints-benchmark/blob/master/src/main/java/jpountz/NaiveByteDecoder.java].
> I confirmed by looking at the generated assembly that if I take an array of 
> integers and shift them all by the same number of bits then Java will use 
> SIMD instructions to shift multiple integers at once. This led me to writing 
> this 
> [implementation|https://github.com/jpountz/decode-128-ints-benchmark/blob/master/src/main/java/jpountz/SimpleSIMDDecoder.java]
>  that tries as much as possible to shift long sequences of ints by the same 
> number of bits to speed up decoding. It is indeed faster than the current 
> logic we have, up to about 2x faster for some numbers of bits per value.
> Currently the best 
> [implementation|https://github.com/jpountz/decode-128-ints-benchmark/blob/master/src/main/java/jpountz/SIMDDecoder.java]
>  I've been able to come up with combines the above idea with the idea that 
> Paul mentioned in his blog that consists of emulating SIMD from Java by 
> packing multiple integers into a long: 2 ints, 4 shorts or 8 bytes. It is a 
> bit harder to read but gives another speedup on top of the above 
> implementation.
> I have a [JMH 
> benchmark|https://github.com/jpountz/decode-128-ints-benchmark/] available in 
> case someone would like to play with this and maybe even come up with an even 
> faster implementation. It is 2-2.5x faster than our current implementation 
> for most numbers of bits per value. I'm copying results here:
> {noformat}
>  * `readLongs` just reads 2*bitsPerValue longs from the ByteBuffer, it serves 
> as
>a baseline.
>  * `decodeNaiveFromBytes` reads a byte[] and decodes from it. This is what the
>current Lucene codec does.
>  * `decodeNaiveFromLongs` decodes from longs on the fly.
>  * `decodeSimpleSIMD` is a simple implementation that relies on how Java
>recognizes some patterns and uses SIMD instructions.
>  * `decodeSIMD` is a more complex implementation that both relies on the C2
>compiler to generate SIMD instructions and encodes 8 bytes, 4 shorts or
>2 ints in a long in order to decompress multiple values at once.
> Benchmark   (bitsPerValue)  (byteOrder)   
> Mode  Cnt   Score   Error   Units
> PackedIntsDecodeBenchmark.decodeNaiveFromBytes   1   LE  
> thrpt5  12.912 ± 0.393  ops/us
> PackedIntsDecodeBenchmark.decodeNaiveFromBytes   1   BE  
> thrpt5  12.862 ± 0.395  ops/us
> PackedIntsDecodeBenchmark.decodeNaiveFromBytes   2   LE  
> thrpt5  13.040 ± 1.162  ops/us
> PackedIntsDecodeBenchmark.decodeNaiveFromBytes   2   BE  
> thrpt5  13.027 ± 0.270  ops/us
> PackedIntsDecodeBenchmark.decodeNaiveFromBytes   3   LE  
> thrpt5  12.409 ± 0.637  ops/us
> PackedIntsDecodeBenchmark.decodeNaiveFromBytes   3   BE  
> thrpt5  12.268 ± 0.947  ops/us
> PackedIntsDecodeBenchmark.decodeNaiveFromBytes   4   LE  
> thrpt5  14.177 ± 2.263  ops/us
> PackedIntsDecodeBenchmark.decodeNaiveFromBytes   4   BE  
> thrpt5  11.457 ± 0.150  ops/us
> PackedIntsDecodeBenchmark.decodeNaiveFromBytes   5   LE  
> thrpt5  10.988 ± 1.179  ops/us
> PackedIntsDecodeBenchmark.de

[jira] [Commented] (LUCENE-9027) SIMD-based decoding of postings lists

2019-10-30 Thread Adrien Grand (Jira)



[ 
https://issues.apache.org/jira/browse/LUCENE-9027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963116#comment-16963116
 ] 

Adrien Grand commented on LUCENE-9027:
--

The fact that throughput is better for lower bits per value made me want to try 
out PFOR delta to try getting lower numbers of bits per value when possible. I 
implemented it with a maximum of 3 exceptions per block. It made the .doc file 
9.4% smaller and the .pos file 8.7% smaller.

{noformat}
TaskQPS baseline  StdDev   QPS patch  StdDev
Pct diff
  Fuzzy2   71.64  (6.3%)   70.99  (7.9%)   
-0.9% ( -14% -   14%)
  IntNRQ  105.31  (5.9%)  104.79  (4.8%)   
-0.5% ( -10% -   10%)
  Fuzzy1  130.67  (9.6%)  130.58  (8.8%)   
-0.1% ( -16% -   20%)
Term 1397.31  (3.4%) 1400.02  (3.1%)
0.2% (  -6% -6%)
 AndHighHigh   34.23  (2.7%)   34.67  (3.5%)
1.3% (  -4% -7%)
  OrHighHigh   11.64  (2.8%)   11.93  (2.5%)
2.5% (  -2% -8%)
   OrHighMed   72.36  (3.0%)   76.48  (3.0%)
5.7% (   0% -   12%)
  AndHighMed   59.88  (3.1%)   63.68  (3.9%)
6.4% (   0% -   13%)
Wildcard   50.18  (8.4%)   53.41  (8.0%)
6.4% (  -9% -   24%)
SpanNear   23.99  (2.9%)   25.56  (1.1%)
6.5% (   2% -   10%)
 AndHighOrMedMed   34.24  (2.1%)   36.89  (2.2%)
7.8% (   3% -   12%)
IntervalsOrdered4.39  (6.0%)4.76  (5.3%)
8.3% (  -2% -   20%)
AndMedOrHighHigh   30.16  (2.7%)   32.76  (2.6%)
8.6% (   3% -   14%)
  Phrase   69.68  (1.3%)   75.76  (1.4%)
8.7% (   5% -   11%)
 Prefix3  170.85 (14.3%)  194.70 (10.9%)   
14.0% (  -9% -   45%)
SloppyPhrase   18.07  (3.8%)   20.92  (4.1%)   
15.8% (   7% -   24%)
{noformat}

> SIMD-based decoding of postings lists
> -
>
> Key: LUCENE-9027
> URL: https://issues.apache.org/jira/browse/LUCENE-9027
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> [~rcmuir] has been mentioning the idea for quite some time that we might be 
> able to write the decoding logic in such a way that Java would use SIMD 
> instructions. More recently [~paul.masurel] wrote a [blog 
> post|https://fulmicoton.com/posts/bitpacking/] that raises the point that 
> Lucene could still do decode multiple ints at once in a single instruction by 
> packing two ints in a long and we've had some discussions about what we could 
> try in Lucene to speed up the decoding of postings. This made me want to look 
> a bit deeper at what we could do.
> Our current decoding logic reads data in a byte[] and decodes packed integers 
> from it. Unfortunately it doesn't make use of SIMD instructions and looks 
> like 
> [this|https://github.com/jpountz/decode-128-ints-benchmark/blob/master/src/main/java/jpountz/NaiveByteDecoder.java].
> I confirmed by looking at the generated assembly that if I take an array of 
> integers and shift them all by the same number of bits then Java will use 
> SIMD instructions to shift multiple integers at once. This led me to writing 
> this 
> [implementation|https://github.com/jpountz/decode-128-ints-benchmark/blob/master/src/main/java/jpountz/SimpleSIMDDecoder.java]
>  that tries as much as possible to shift long sequences of ints by the same 
> number of bits to speed up decoding. It is indeed faster than the current 
> logic we have, up to about 2x faster for some numbers of bits per value.
> Currently the best 
> [implementation|https://github.com/jpountz/decode-128-ints-benchmark/blob/master/src/main/java/jpountz/SIMDDecoder.java]
>  I've been able to come up with combines the above idea with the idea that 
> Paul mentioned in his blog that consists of emulating SIMD from Java by 
> packing multiple integers into a long: 2 ints, 4 shorts or 8 bytes. It is a 
> bit harder to read but gives another speedup on top of the above 
> implementation.
> I have a [JMH 
> benchmark|https://github.com/jpountz/decode-128-ints-benchmark/] available in 
> case someone would like to play with this and maybe even come up with an even 
> faster implementation. It is 2-2.5x faster than our current implementation 
> for most numbers of bits per value. I'm copying results here:
> {noformat}
>  * `readLongs` just reads 2*bitsPerValue longs from the ByteBuffer, it serves 
> as
>a baseline.
>  * `decodeNaiveFromByte

[GitHub] [lucene-site] janhoy commented on issue #1: LUCENE-9012 Setup minimal site with working Pelican build

janhoy commented on issue #1: LUCENE-9012 Setup minimal site with working 
Pelican build
URL: https://github.com/apache/lucene-site/pull/1#issuecomment-547969380
 
 
   I always wanted to split those looong news pages, glad to see it happen!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-13844) Remove recovering shard term with corresponding core shard term.

2019-10-30 Thread Houston Putman (Jira)



 [ 
https://issues.apache.org/jira/browse/SOLR-13844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Houston Putman updated SOLR-13844:
--
Fix Version/s: 8.4

> Remove recovering shard term with corresponding core shard term.
> 
>
> Key: SOLR-13844
> URL: https://issues.apache.org/jira/browse/SOLR-13844
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: master (9.0)
>Reporter: Houston Putman
>Priority: Minor
> Fix For: master (9.0), 8.4
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Currently if a recovering replica (solr 7.3+) is deleted, the term for that 
> core in the shard's terms in ZK is removed. However the {{_recovering}} 
> term is not removed as well. This can create clutter and confusion in the 
> shard terms ZK node.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #951: SOLR-13844: Remove replica recovery terms with the replica term

HoustonPutman commented on a change in pull request #951: SOLR-13844: Remove 
replica recovery terms with the replica term
URL: https://github.com/apache/lucene-solr/pull/951#discussion_r340698571
 
 

 ##
 File path: solr/core/src/java/org/apache/solr/cloud/ZkShardTerms.java
 ##
 @@ -524,15 +529,23 @@ Terms ensureHighestTermsAreNotZero() {
 }
 
 /**
- * Return a new {@link Terms} in which term of {@code coreNodeName} is 
removed
+ * Return a new {@link Terms} in which terms for the {@code coreNodeName} 
are removed
  * @param coreNodeName of the replica
  * @return null if term of {@code coreNodeName} is already not exist
  */
 Terms removeTerm(String coreNodeName) {
-  if (!values.containsKey(coreNodeName)) return null;
+  if (!values.containsKey(recoveringTerm(coreNodeName)) && 
!values.containsKey(coreNodeName)) {
+return null;
+  }
 
   HashMap newValues = new HashMap<>(values);
-  newValues.remove(coreNodeName);
+  if (values.containsKey(coreNodeName)) {
 
 Review comment:
   Ahh good to know. Took out both checks.
   
   Thanks for taking a look!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-site] janhoy commented on issue #1: LUCENE-9012 Setup minimal site with working Pelican build

janhoy commented on issue #1: LUCENE-9012 Setup minimal site with working 
Pelican build
URL: https://github.com/apache/lucene-site/pull/1#issuecomment-547984534
 
 
   @adamwalz Can you add a README.md with basic instructions to build the site 
locally (including what tools to install)? Then I'll try it here and merge once 
it's working. No need to describe the .asf.yml stuff in README yet


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] HoustonPutman opened a new pull request #984: SOLR-12217: Support shards.preference for individual shard requests

HoustonPutman opened a new pull request #984: SOLR-12217: Support
shards.preference for individual shard requests
URL: https://github.com/apache/lucene-solr/pull/984

# Description

Use shard preference rules when choosing replicas to send individual shard
requests. This includes requests from SolrJ as well as requests sent via
streaming expressions.

# Solution
How routing preferences are determined:

For Streaming Expressions, it will use the preferences from the following
sources, in order of preference (defaults last):
1. The params given in the stream source
1. The URL params sent in the streaming request
1. The `DEFAULT_SHARD_PREFERENCES` set in cluster properties

For SolrJ, it will use the preferences from the params of the request it is
sending.

# Tests

Current tests pass.

**TODO** Add tests for routing functionality

# Checklist

Please review the following and check all that apply:

- [ ] I have reviewed the guidelines for [How to
Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms
to the standards described there to the best of my ability.
- [ ] I have created a Jira issue and added the issue ID to my pull request
title.
- [ ] I am authorized to contribute this code to the ASF and have removed
any code I do not have a license to distribute.
- [ ] I have given Solr maintainers
[access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork)
to contribute to my PR branch. (optional but recommended)
- [ ] I have developed this patch against the `master` branch.
- [ ] I have run `ant precommit` and the appropriate test suite.
- [ ] I have added tests for my changes.
- [ ] I have added documentation for the [Ref
Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide)
(for Solr changes only).

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-12217) Add support for shards.preference in single shard cases

2019-10-30 Thread Houston Putman (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963183#comment-16963183
 ] 

Houston Putman commented on SOLR-12217:
---

Did a first-pass for this, pretty straightforward. Still need to add some 
routing-specific tests, and documentation to the ref-guide.

> Add support for shards.preference in single shard cases
> ---
>
> Key: SOLR-12217
> URL: https://issues.apache.org/jira/browse/SOLR-12217
> Project: Solr
>  Issue Type: New Feature
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> SOLR-11982 Added support for {{shards.preference}}, a way to define the 
> sorting of replicas within a shard by preference (replica types/location). 
> This only works on multi-shard cases. We should add support for the case of 
> single shards when using CloudSolrClient



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] mkhludnev opened a new pull request #985: LUCENE-9031: test unified highlighter on intervals queries

mkhludnev opened a new pull request #985: LUCENE-9031: test unified highlighter 
on intervals queries
URL: https://github.com/apache/lucene-solr/pull/985
 
 
   # Description
   
   Fix UnsupportedOpExc on highlighting Interval queries. 
   
   # Solution
   
   Caching and returning Interval queries. 
   
   # Tests
   
   Cruel harm to TestUnifiedHighlighting 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9031) UnsupportedOperationException on highlighting Interval Query



[ 
https://issues.apache.org/jira/browse/LUCENE-9031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963239#comment-16963239
 ] 

Mikhail Khludnev commented on LUCENE-9031:
--

done 

[https://github.com/apache/lucene-solr/pull/985/files]

> UnsupportedOperationException on highlighting Interval Query
> 
>
> Key: LUCENE-9031
> URL: https://issues.apache.org/jira/browse/LUCENE-9031
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/queries
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Major
> Fix For: 8.4
>
> Attachments: LUCENE-9031.patch, LUCENE-9031.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When UnifiedHighlighter highlights Interval Query it encounters 
> UnsupportedOperationException. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13880) Collection creation fails with coreNodeName core_nodeX does not exist in shard

2019-10-30 Thread Tomas Eduardo Fernandez Lobbe (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963245#comment-16963245
 ] 

Tomas Eduardo Fernandez Lobbe commented on SOLR-13880:
--

This has been happening in tests where a collection with the same name is 
created/removed rapidly

> Collection creation fails with coreNodeName core_nodeX does not exist in shard
> --
>
> Key: SOLR-13880
> URL: https://issues.apache.org/jira/browse/SOLR-13880
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Affects Versions: master (9.0)
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Minor
>
> I've seen this when running tests locally. When issuing a collection 
> creation, the call fails with:
> {noformat}
>   [junit4]   2> 94288 ERROR (qtp149989-237) [n:127.0.0.1:63117_solr 
> c:pull_replica_test_create_delete s:shard1 r:core_node9 
> x:pull_replica_test_create_delete_shard1_replica_p6 ] 
> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error 
> CREATEing SolrCore 'pull_replica_test_create_delete_shard1_replica_p6': 
> Unable to create core [pull_replica_test_create_delete_shard1_replica_p6] 
> Caused by: coreNodeName core_node9 does not exist in shard shard1, ignore the 
> exception if the replica was deleted
>[junit4]   2>  at 
> org.apache.solr.core.CoreContainer.create(CoreContainer.java:1209)
>[junit4]   2>  at 
> org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
>[junit4]   2>  at 
> org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
>[junit4]   2>  at 
> org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
>[junit4]   2>  at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
>[junit4]   2>  at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:198)
>[junit4]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:843)
>[junit4]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:809)
>[junit4]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:562)
>[junit4]   2>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:424)
>[junit4]   2>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
>[junit4]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
>[junit4]   2>  at 
> org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:167)
>[junit4]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
>[junit4]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
>[junit4]   2>  at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
>[junit4]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
>[junit4]   2>  at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>[junit4]   2>  at 
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>[junit4]   2>  at 
> org.eclipse.jetty.server.Server.handle(Server.java:505)
>[junit4]   2>  at 
> org.eclipse.jetty.server.HttpChannel.handle(HttpChann

[jira] [Updated] (SOLR-13880) Collection creation fails with coreNodeName core_nodeX does not exist in shard

2019-10-30 Thread Tomas Eduardo Fernandez Lobbe (Jira)



 [ 
https://issues.apache.org/jira/browse/SOLR-13880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomas Eduardo Fernandez Lobbe updated SOLR-13880:
-
Attachment: TestPullReplica-45-2.log

> Collection creation fails with coreNodeName core_nodeX does not exist in shard
> --
>
> Key: SOLR-13880
> URL: https://issues.apache.org/jira/browse/SOLR-13880
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Affects Versions: master (9.0)
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Minor
> Attachments: TestPullReplica-45-2.log
>
>
> I've seen this when running tests locally. When issuing a collection 
> creation, the call fails with:
> {noformat}
>   [junit4]   2> 94288 ERROR (qtp149989-237) [n:127.0.0.1:63117_solr 
> c:pull_replica_test_create_delete s:shard1 r:core_node9 
> x:pull_replica_test_create_delete_shard1_replica_p6 ] 
> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error 
> CREATEing SolrCore 'pull_replica_test_create_delete_shard1_replica_p6': 
> Unable to create core [pull_replica_test_create_delete_shard1_replica_p6] 
> Caused by: coreNodeName core_node9 does not exist in shard shard1, ignore the 
> exception if the replica was deleted
>[junit4]   2>  at 
> org.apache.solr.core.CoreContainer.create(CoreContainer.java:1209)
>[junit4]   2>  at 
> org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
>[junit4]   2>  at 
> org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
>[junit4]   2>  at 
> org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
>[junit4]   2>  at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
>[junit4]   2>  at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:198)
>[junit4]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:843)
>[junit4]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:809)
>[junit4]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:562)
>[junit4]   2>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:424)
>[junit4]   2>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
>[junit4]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
>[junit4]   2>  at 
> org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:167)
>[junit4]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
>[junit4]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
>[junit4]   2>  at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
>[junit4]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
>[junit4]   2>  at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>[junit4]   2>  at 
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>[junit4]   2>  at 
> org.eclipse.jetty.server.Server.handle(Server.java:505)
>[junit4]   2>  at 
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
>[junit4]   2>  at 
> org.eclipse.jetty.serve

[GitHub] [lucene-site] adamwalz commented on issue #1: LUCENE-9012 Setup minimal site with working Pelican build

adamwalz commented on issue #1: LUCENE-9012 Setup minimal site with working 
Pelican build
URL: https://github.com/apache/lucene-site/pull/1#issuecomment-548027707
 
 
   @janhoy Yes I've added a README as well as a `requirements.txt` file 
containing the list of python dependencies.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13880) Collection creation fails with coreNodeName core_nodeX does not exist in shard

2019-10-30 Thread Tomas Eduardo Fernandez Lobbe (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963269#comment-16963269
 ] 

Tomas Eduardo Fernandez Lobbe commented on SOLR-13880:
--

This can be related to what Erick describes in SOLR-13709, this test will 
create/reload/delete/create/reload/delete (since it's ran twice). I'm going to 
isolate that behavior into it's own test case

> Collection creation fails with coreNodeName core_nodeX does not exist in shard
> --
>
> Key: SOLR-13880
> URL: https://issues.apache.org/jira/browse/SOLR-13880
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Affects Versions: master (9.0)
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Minor
> Attachments: TestPullReplica-45-2.log
>
>
> I've seen this when running tests locally. When issuing a collection 
> creation, the call fails with:
> {noformat}
>   [junit4]   2> 94288 ERROR (qtp149989-237) [n:127.0.0.1:63117_solr 
> c:pull_replica_test_create_delete s:shard1 r:core_node9 
> x:pull_replica_test_create_delete_shard1_replica_p6 ] 
> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error 
> CREATEing SolrCore 'pull_replica_test_create_delete_shard1_replica_p6': 
> Unable to create core [pull_replica_test_create_delete_shard1_replica_p6] 
> Caused by: coreNodeName core_node9 does not exist in shard shard1, ignore the 
> exception if the replica was deleted
>[junit4]   2>  at 
> org.apache.solr.core.CoreContainer.create(CoreContainer.java:1209)
>[junit4]   2>  at 
> org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
>[junit4]   2>  at 
> org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
>[junit4]   2>  at 
> org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
>[junit4]   2>  at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
>[junit4]   2>  at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:198)
>[junit4]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:843)
>[junit4]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:809)
>[junit4]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:562)
>[junit4]   2>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:424)
>[junit4]   2>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
>[junit4]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
>[junit4]   2>  at 
> org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:167)
>[junit4]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
>[junit4]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
>[junit4]   2>  at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
>[junit4]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
>[junit4]   2>  at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>[junit4]   2>  at 
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>[junit4]   2

[GitHub] [lucene-site] janhoy commented on a change in pull request #1: LUCENE-9012 Setup minimal site with working Pelican build

janhoy commented on a change in pull request #1: LUCENE-9012 Setup minimal site 
with working Pelican build
URL: https://github.com/apache/lucene-site/pull/1#discussion_r340768317
 
 

 ##
 File path: Makefile
 ##
 @@ -0,0 +1,82 @@
+PY?=python3
+PELICAN?=pelican
+PELICANOPTS=
+
+BASEDIR=$(CURDIR)
+INPUTDIR=$(BASEDIR)/content
+OUTPUTDIR=$(BASEDIR)/output
+CONFFILE=$(BASEDIR)/pelicanconf.py
+PUBLISHCONF=$(BASEDIR)/publishconf.py
+
+GITHUB_PAGES_BRANCH=master
 
 Review comment:
   Remove GH-pages stuff?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-site] janhoy commented on a change in pull request #1: LUCENE-9012 Setup minimal site with working Pelican build

janhoy commented on a change in pull request #1: LUCENE-9012 Setup minimal site 
with working Pelican build
URL: https://github.com/apache/lucene-site/pull/1#discussion_r340769158
 
 

 ##
 File path: README.md
 ##
 @@ -0,0 +1,54 @@
+# Web site for Apache Lucene and Solr
+
+## Building the site
+
+### Installing Pelican
+
+This site uses [Pelican][1] for static html generation. Pelican requires
+Python 2.7.x and 3.5+ and can be installed with pip.
 
 Review comment:
   Perhaps we can just document for Python3, since Py2 is soon gone and much of 
our other dev scripts already require Python3. So perhaps `pip3 install 
pelican` is better?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-site] janhoy commented on issue #1: LUCENE-9012 Setup minimal site with working Pelican build

janhoy commented on issue #1: LUCENE-9012 Setup minimal site with working 
Pelican build
URL: https://github.com/apache/lucene-site/pull/1#issuecomment-548045491
 
 
   Minor: Perhaps below the news on main page show a text "For older news 
articles, see " and link to the news page?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-site] adamwalz commented on issue #1: LUCENE-9012 Setup minimal site with working Pelican build

adamwalz commented on issue #1: LUCENE-9012 Setup minimal site with working 
Pelican build
URL: https://github.com/apache/lucene-site/pull/1#issuecomment-548052856
 
 
   Agreed on all points. I'd like to rebase this and iterate on issues in 
followup PRs. I'm expecting to find more Pelican idiosyncrasies and differences 
from the current production site as we go along.
   
   The next two PRs which could also be large would be
   1. The solr pages
   2. DRYing this out a little. There's a lot of duplicate code
   
   After that changes such as dead links, build documentation, and unnecessary 
settings can be made in much smaller more reviewable PRs.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-site] janhoy merged pull request #1: LUCENE-9012 Setup minimal site with working Pelican build

janhoy merged pull request #1: LUCENE-9012 Setup minimal site with working 
Pelican build
URL: https://github.com/apache/lucene-site/pull/1
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9012) Setup minimal site with working Pelican build



[ 
https://issues.apache.org/jira/browse/LUCENE-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963315#comment-16963315
 ] 

Jan Høydahl commented on LUCENE-9012:
-

Merged PR 1, please see [https://github.com/apache/lucene-site] now. The site 
(except Solr) can now be built and viewed locally following the steps in README.

Next is solr parts, and then general cleanup (dead links, restructure as needed 
etc). We can do that as followup PRs in this same JIRA.

> Setup minimal site with working Pelican build
> -
>
> Key: LUCENE-9012
> URL: https://issues.apache.org/jira/browse/LUCENE-9012
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Assigned] (LUCENE-9012) Setup minimal site with working Pelican build



 [ 
https://issues.apache.org/jira/browse/LUCENE-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl reassigned LUCENE-9012:
---

Assignee: Jan Høydahl

> Setup minimal site with working Pelican build
> -
>
> Key: LUCENE-9012
> URL: https://issues.apache.org/jira/browse/LUCENE-9012
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (LUCENE-9033) Update Release docs an scripts with new site instructions

Jan Høydahl created LUCENE-9033:
---

 Summary: Update Release docs an scripts with new site instructions
 Key: LUCENE-9033
 URL: https://issues.apache.org/jira/browse/LUCENE-9033
 Project: Lucene - Core
  Issue Type: Sub-task
  Components: general/tools
Reporter: Jan Høydahl


* releaseWizard.py
 * ReleaseTODO page
 * addBackcompatIndexes.py
 * archive-solr-ref-guide.sh
 * createPatch.py
 * publish-solr-ref-guide.sh
 * solr-ref-gudie/src/meta-docs/publish.adoc

There may be others



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13871) JettySolrRunner should stop trying to re-use ports on re-start

2019-10-30 Thread Tomas Eduardo Fernandez Lobbe (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963319#comment-16963319
 ] 

Tomas Eduardo Fernandez Lobbe commented on SOLR-13871:
--

I guess the problem with SolrCloud tests would be that, if you have a replica 
in a node with port A, and you restart, but it starts with port B, then that 
replica is essentially gone, unless you have a way to go and change the node 
for it. Is that what you mean? I don't know how many tests will be doing this, 
but for sure there are some

> JettySolrRunner should stop trying to re-use ports on re-start
> --
>
> Key: SOLR-13871
> URL: https://issues.apache.org/jira/browse/SOLR-13871
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Chris M. Hostetter
>Priority: Major
>
> JettySolrRunner currently has special logic in it's {{start()}} method that 
> will cause it (by default) to try to rebind to the port it was previously 
> assigned if it's restarting and configured with port '0'
> ie: the first time it starts the OS assigns the port, after that it tries to 
> re-use that same port.
> This is a bad idea in general, and leads to (very slow) BindException 
> failures in a lot of jenkins tests where nodes are restarted.
> Example...
> {noformat}
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ShardSplitTest 
> -Dtests.method=testSplitWithChaosMonkey -Dtests.seed=9097C20E8E9ACC68 -Dtes
> ts.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
> -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test
> -data/enwiki.random.lines.txt -Dtests.locale=so-SO -Dtests.timezone=Etc/GMT+9 
> -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>[junit4] ERROR   81.2s J1 | ShardSplitTest.testSplitWithChaosMonkey <<<
>[junit4]> Throwable #1: java.net.BindException: Address already in use
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([9097C20E8E9ACC68:1BB011DFCF9C67EC]:0)
>[junit4]>at java.base/sun.nio.ch.Net.bind0(Native Method)
>[junit4]>at java.base/sun.nio.ch.Net.bind(Net.java:461)
>[junit4]>at java.base/sun.nio.ch.Net.bind(Net.java:453)
>[junit4]>at 
> java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
>[junit4]>at 
> java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:80)
>[junit4]>at 
> org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
>[junit4]>at 
> org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:308)
>[junit4]>at 
> org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
>[junit4]>at 
> org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
>[junit4]>at 
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
>[junit4]>at 
> org.eclipse.jetty.server.Server.doStart(Server.java:396)
>[junit4]>at 
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
>[junit4]>at 
> org.apache.solr.client.solrj.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:569)
>[junit4]>at 
> org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:508)
>[junit4]>at 
> org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:476)
>[junit4]>at 
> org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:499)
> {noformat}
> Ideally JettySolrRunner's default behavior should b to just trust it's config 
> – binding to a random port (even on restart, even if diff from it's previous 
> port) if configured with '0'.
> Callers – including tests – should eliminate any assumptions in their code 
> that the port will be consistent for the life of a JettySolrRunner (ie: 
> accept that the URL may change anytime stop/start is called)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9020) Find a way to publish Solr RefGuide without checking into git



[ 
https://issues.apache.org/jira/browse/LUCENE-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963322#comment-16963322
 ] 

Jan Høydahl commented on LUCENE-9020:
-

Uwe talked to Infra and they suggest we continue to push guide and javadocs to 
svn. But I don't know to what location and how to integrate it into the new 
Pelican site. [~uschindler], should we open a Jira with INFRA in order to get 
instructions on exactly how to proceed?

> Find a way to publish Solr RefGuide without checking into git
> -
>
> Key: LUCENE-9020
> URL: https://issues.apache.org/jira/browse/LUCENE-9020
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Priority: Major
>
> Currently we check in all versions of RefGuide (hundreds of small html files) 
> into svn to publish as part of the site. With new site we should find a 
> smoother way to do this.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Resolved] (SOLR-13783) NamedList.toString() ought to be consistent with AbstractMap

2019-10-30 Thread Tomas Eduardo Fernandez Lobbe (Jira)



 [ 
https://issues.apache.org/jira/browse/SOLR-13783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomas Eduardo Fernandez Lobbe resolved SOLR-13783.
--
Resolution: Fixed

> NamedList.toString() ought to be consistent with AbstractMap
> 
>
> Key: SOLR-13783
> URL: https://issues.apache.org/jira/browse/SOLR-13783
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: David Smiley
>Priority: Minor
>  Labels: newdev
> Fix For: master (9.0)
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> NamedList.toString() does not put a space after the joining commas, whereas 
> AbstractMap does.  I think it ought to be consistent.  This can matter if you 
> write tests that toString() a piece of a SolrResponse that varies in its use 
> of EmbeddedSolrServer versus other SolrClients that serialize the response.  
> Some custom SearchComponents or whatever might prefer to use a Map and that 
> should ultimately be consistent.  I know the assumption is a little brittle 
> but still.
> I think 9.0 without a back-port is safe.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9020) Find a way to publish Solr RefGuide without checking into git



[ 
https://issues.apache.org/jira/browse/LUCENE-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963328#comment-16963328
 ] 

Uwe Schindler commented on LUCENE-9020:
---

I was about to ask the same. I would suggest to open issue. I will do that 
tomorrow morning asking them how to proceed. 

I talked with [~ke4qqq] so I will mention him there, too.

> Find a way to publish Solr RefGuide without checking into git
> -
>
> Key: LUCENE-9020
> URL: https://issues.apache.org/jira/browse/LUCENE-9020
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Priority: Major
>
> Currently we check in all versions of RefGuide (hundreds of small html files) 
> into svn to publish as part of the site. With new site we should find a 
> smoother way to do this.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9020) Find a way to publish Solr RefGuide without checking into git



[ 
https://issues.apache.org/jira/browse/LUCENE-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963331#comment-16963331
 ] 

Uwe Schindler commented on LUCENE-9020:
---

The same also affects Javadocs (the current site has approx 16 gigs of HTML in 
the production folder, referenced by extpath.txt.

> Find a way to publish Solr RefGuide without checking into git
> -
>
> Key: LUCENE-9020
> URL: https://issues.apache.org/jira/browse/LUCENE-9020
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Priority: Major
>
> Currently we check in all versions of RefGuide (hundreds of small html files) 
> into svn to publish as part of the site. With new site we should find a 
> smoother way to do this.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (LUCENE-9034) Officially publish the new site

Jan Høydahl created LUCENE-9034:
---

 Summary: Officially publish the new site
 Key: LUCENE-9034
 URL: https://issues.apache.org/jira/browse/LUCENE-9034
 Project: Lucene - Core
  Issue Type: Sub-task
  Components: general/website
Reporter: Jan Høydahl


Publishing the web site means creating a publish branch and adding the right 
magic instructions to {{.asf.yml}} etc. This will then publish the new site and 
disable old CMS.

Before we do that we should
 # Make sure all docs and release tools are updated for new site publishing 
instructions
 # Create a PR with latest changes in old CMS site since the export. This will 
be the changes done during 8.3.0 release and possibly some news entries related 
to security issues etc.

After publishing we should ask INFRA to make old site svn read-only (and 
perhaps do a commit that replaces svn content with a README.txt), so it is 
obvious for everyone that we have migrated.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13871) JettySolrRunner should stop trying to re-use ports on re-start

2019-10-30 Thread Chris M. Hostetter (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963337#comment-16963337
 ] 

Chris M. Hostetter commented on SOLR-13871:
---

bq.  ...then that replica is essentially gone,...

well, that's the crux of the concern i mentioned about assumptions in SolrCloud 
code.

Once upon a time i thought the point of  'genericCoreNodeNames' was to to make 
it possible for you shutdown a node, and start it up on a completely new 
host/port and those replicas would be recognized exactly the same as they were 
before, w/o any problems, regardless of what the old port was ... but i think i 
asked about this a while back and people who know more then me about SolrCloud 
and routing and replica status said "no, it's not that smart"

My goal here is that *IF* we _can_ easily make SolrCloud that smart -- in a new 
jira that would block this one -- then we can/should fix JettySolrRunner to 
stop jumping through it's current hoops on restart, just pick a new random 
port, and let SolrCloud figure out that those old replicas have come back on a 
new port -- and then fix any test code that breaks when we do that (ie: keeping 
an HttpSolrClient around even after the jetty has been restarted)

If we *CAN'T* easily fix SolrCloud do that, then we should consider other test 
workarounds (like doing proxy port blocking where possible) to eliminate as 
much as possible the *need* to make JettySolrRunner try to forcibly re-use the 
same port as much as possible.

(Another "workaround" i've considered, but haven't had a chance to look into 
yet, is to catch the BindException in tests where we know we've tried to 
"restart" a jetty port, and instead throw an AssumptionFailedException -- so at 
least we get a SKIP instead of a missleading FAIL when the problem is just that 
the test couldn't get the port it needs from the OS.

> JettySolrRunner should stop trying to re-use ports on re-start
> --
>
> Key: SOLR-13871
> URL: https://issues.apache.org/jira/browse/SOLR-13871
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Chris M. Hostetter
>Priority: Major
>
> JettySolrRunner currently has special logic in it's {{start()}} method that 
> will cause it (by default) to try to rebind to the port it was previously 
> assigned if it's restarting and configured with port '0'
> ie: the first time it starts the OS assigns the port, after that it tries to 
> re-use that same port.
> This is a bad idea in general, and leads to (very slow) BindException 
> failures in a lot of jenkins tests where nodes are restarted.
> Example...
> {noformat}
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ShardSplitTest 
> -Dtests.method=testSplitWithChaosMonkey -Dtests.seed=9097C20E8E9ACC68 -Dtes
> ts.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
> -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test
> -data/enwiki.random.lines.txt -Dtests.locale=so-SO -Dtests.timezone=Etc/GMT+9 
> -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>[junit4] ERROR   81.2s J1 | ShardSplitTest.testSplitWithChaosMonkey <<<
>[junit4]> Throwable #1: java.net.BindException: Address already in use
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([9097C20E8E9ACC68:1BB011DFCF9C67EC]:0)
>[junit4]>at java.base/sun.nio.ch.Net.bind0(Native Method)
>[junit4]>at java.base/sun.nio.ch.Net.bind(Net.java:461)
>[junit4]>at java.base/sun.nio.ch.Net.bind(Net.java:453)
>[junit4]>at 
> java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
>[junit4]>at 
> java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:80)
>[junit4]>at 
> org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
>[junit4]>at 
> org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:308)
>[junit4]>at 
> org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
>[junit4]>at 
> org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
>[junit4]>at 
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
>[junit4]>at 
> org.eclipse.jetty.server.Server.doStart(Server.java:396)
>[junit4]>at 
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
>[junit4]>at 
> org.apache.solr.client.solrj.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:569)
>[junit4]>at 
> org.apache.solr.clie

[GitHub] [lucene-solr] tflobbe opened a new pull request #986: SOLR-13880: Add test that creates/reloads/deletes collections

tflobbe opened a new pull request #986: SOLR-13880: Add test that 
creates/reloads/deletes collections
URL: https://github.com/apache/lucene-solr/pull/986
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13880) Collection creation fails with coreNodeName core_nodeX does not exist in shard

2019-10-30 Thread Tomas Eduardo Fernandez Lobbe (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963336#comment-16963336
 ] 

Tomas Eduardo Fernandez Lobbe commented on SOLR-13880:
--

Created a PR with the test, however, I haven't been able to reproduce the 
failure yet

> Collection creation fails with coreNodeName core_nodeX does not exist in shard
> --
>
> Key: SOLR-13880
> URL: https://issues.apache.org/jira/browse/SOLR-13880
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Affects Versions: master (9.0)
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Minor
> Attachments: TestPullReplica-45-2.log
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I've seen this when running tests locally. When issuing a collection 
> creation, the call fails with:
> {noformat}
>   [junit4]   2> 94288 ERROR (qtp149989-237) [n:127.0.0.1:63117_solr 
> c:pull_replica_test_create_delete s:shard1 r:core_node9 
> x:pull_replica_test_create_delete_shard1_replica_p6 ] 
> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error 
> CREATEing SolrCore 'pull_replica_test_create_delete_shard1_replica_p6': 
> Unable to create core [pull_replica_test_create_delete_shard1_replica_p6] 
> Caused by: coreNodeName core_node9 does not exist in shard shard1, ignore the 
> exception if the replica was deleted
>[junit4]   2>  at 
> org.apache.solr.core.CoreContainer.create(CoreContainer.java:1209)
>[junit4]   2>  at 
> org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
>[junit4]   2>  at 
> org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
>[junit4]   2>  at 
> org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
>[junit4]   2>  at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
>[junit4]   2>  at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:198)
>[junit4]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:843)
>[junit4]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:809)
>[junit4]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:562)
>[junit4]   2>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:424)
>[junit4]   2>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
>[junit4]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
>[junit4]   2>  at 
> org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:167)
>[junit4]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
>[junit4]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
>[junit4]   2>  at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
>[junit4]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
>[junit4]   2>  at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>[junit4]   2>  at 
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)
>[junit4]   2>  at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>[junit4]   2>  at 
> org.eclipse.jetty.server.Server.handle(Server.ja

[jira] [Created] (SOLR-13884) Concurrent collection creation leads to unbalanced cluster

Yonik Seeley created SOLR-13884:
---

 Summary: Concurrent collection creation leads to unbalanced cluster
 Key: SOLR-13884
 URL: https://issues.apache.org/jira/browse/SOLR-13884
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Yonik Seeley


When multiple collection creations are done concurrently, the cluster can end 
up very unbalanced, with many (or most) replicas going to a small set of nodes.

This was observed on both 8.2 and master.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13875) Check why LBSolrClient thinks it has to sort on _docid_

2019-10-30 Thread Christine Poerschke (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963341#comment-16963341
 ] 

Christine Poerschke commented on SOLR-13875:


SOLR-5718 is some history here, the {{__docid__}} sort was added there in Feb 
2014 -- 
https://github.com/apache/lucene-solr/commit/0b023e52369906f44c9677d16440895a5a122838
 -- and I recall little else about it.





> Check why LBSolrClient thinks it has to sort on _docid_
> ---
>
> Key: SOLR-13875
> URL: https://issues.apache.org/jira/browse/SOLR-13875
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Priority: Major
>
> Related to SOLR-13874. LBSolrClient issues a query that sorts on \_docid\_. 
> This can take over 12 seconds on a 1B doc replica. Is the sorting necessary 
> at all? If it's removed, what are the effects on performance? Of we change 
> SOLR-13874, we can close this.
> Frankly I suspect that no matter what, there have to be over 1B comparisons 
> done and that's what's taking the time, but I wanted to call this out 
> explicitly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8422) Add Matches iteration to interval queries



 [ 
https://issues.apache.org/jira/browse/LUCENE-8422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated LUCENE-8422:
-
Attachment: image-2019-10-30-12-08-40-145.png

> Add Matches iteration to interval queries
> -
>
> Key: LUCENE-8422
> URL: https://issues.apache.org/jira/browse/LUCENE-8422
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 7.5
>
> Attachments: LUCENE-8422.patch, LUCENE-8422.patch, LUCENE-8422.patch, 
> image-2019-10-30-12-08-40-145.png
>
>
> Follow up to LUCENE-8404, we can now add Matches iteration to interval 
> queries in the sandbox.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13884) Concurrent collection creation leads to unbalanced cluster

2019-10-30 Thread David Smiley (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963345#comment-16963345
 ] 

David Smiley commented on SOLR-13884:
-

Supposedly SOLR-10822 PolicyHelper.SessionWrapper purports to fix this scenario 
but apparently it doesn't actually do so.  CC [~shalin]

> Concurrent collection creation leads to unbalanced cluster
> --
>
> Key: SOLR-13884
> URL: https://issues.apache.org/jira/browse/SOLR-13884
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
>Priority: Major
>
> When multiple collection creations are done concurrently, the cluster can end 
> up very unbalanced, with many (or most) replicas going to a small set of 
> nodes.
> This was observed on both 8.2 and master.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8422) Add Matches iteration to interval queries



[ 
https://issues.apache.org/jira/browse/LUCENE-8422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963344#comment-16963344
 ] 

Mikhail Khludnev commented on LUCENE-8422:
--

!image-2019-10-30-12-08-40-145.png!

What worries me a little is absence of coverage in that caching code, even 
after running Intervals package tests.   

 

> Add Matches iteration to interval queries
> -
>
> Key: LUCENE-8422
> URL: https://issues.apache.org/jira/browse/LUCENE-8422
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 7.5
>
> Attachments: LUCENE-8422.patch, LUCENE-8422.patch, LUCENE-8422.patch, 
> image-2019-10-30-12-08-40-145.png
>
>
> Follow up to LUCENE-8404, we can now add Matches iteration to interval 
> queries in the sandbox.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13884) Concurrent collection creation leads to unbalanced cluster



[ 
https://issues.apache.org/jira/browse/SOLR-13884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963346#comment-16963346
 ] 

Yonik Seeley commented on SOLR-13884:
-

As [~dsmiley] noted, it seems like this issue should have been fixed by 
SOLR-10822

> Concurrent collection creation leads to unbalanced cluster
> --
>
> Key: SOLR-13884
> URL: https://issues.apache.org/jira/browse/SOLR-13884
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
>Priority: Major
>
> When multiple collection creations are done concurrently, the cluster can end 
> up very unbalanced, with many (or most) replicas going to a small set of 
> nodes.
> This was observed on both 8.2 and master.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Issue Comment Deleted] (SOLR-13884) Concurrent collection creation leads to unbalanced cluster



 [ 
https://issues.apache.org/jira/browse/SOLR-13884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yonik Seeley updated SOLR-13884:

Comment: was deleted

(was: As [~dsmiley] noted, it seems like this issue should have been fixed by 
SOLR-10822)

> Concurrent collection creation leads to unbalanced cluster
> --
>
> Key: SOLR-13884
> URL: https://issues.apache.org/jira/browse/SOLR-13884
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
>Priority: Major
>
> When multiple collection creations are done concurrently, the cluster can end 
> up very unbalanced, with many (or most) replicas going to a small set of 
> nodes.
> This was observed on both 8.2 and master.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Assigned] (LUCENE-9014) Design a new solution for publishing JavaDoc



 [ 
https://issues.apache.org/jira/browse/LUCENE-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl reassigned LUCENE-9014:
---

Assignee: Jan Høydahl

> Design a new solution for publishing JavaDoc
> 
>
> Key: LUCENE-9014
> URL: https://issues.apache.org/jira/browse/LUCENE-9014
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
>
> Currently we check in to svn folders with hundreds of html files for JavaDoc. 
> Need a better way.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9014) Design a new solution for publishing JavaDoc



[ 
https://issues.apache.org/jira/browse/LUCENE-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963347#comment-16963347
 ] 

Jan Høydahl commented on LUCENE-9014:
-

Our javadocs looks quite plain and stock.

Comparing [http://lucene.apache.org/solr/8_2_0/solr-core/index.html]  with 
[https://www.javadoc.io/doc/org.apache.solr/solr-core/8.2.0/index.html] I like 
the javadocs.io version better as it allows you to navigate between versions, 
download javadocs jar, move between modules (solr-core, solr-cell etc) from the 
javadoc page itself etc. I can attempt a patch that generates links to 
javadoc.io instead of actually placing the javadocs html files for upload.

> Design a new solution for publishing JavaDoc
> 
>
> Key: LUCENE-9014
> URL: https://issues.apache.org/jira/browse/LUCENE-9014
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Priority: Major
>
> Currently we check in to svn folders with hundreds of html files for JavaDoc. 
> Need a better way.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] yonik opened a new pull request #987: SOLR-13884: add ConcurrentCreateCollectionTest test

yonik opened a new pull request #987: SOLR-13884: add 
ConcurrentCreateCollectionTest test
URL: https://github.com/apache/lucene-solr/pull/987
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13884) Concurrent collection creation leads to unbalanced cluster



[ 
https://issues.apache.org/jira/browse/SOLR-13884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963353#comment-16963353
 ] 

Yonik Seeley commented on SOLR-13884:
-

See #987 for a test.  When I run 20 concurrent single replica collection 
creations, I sometimes see very skewed placement over 2 nodes.  Sometimes 7 and 
13 or more.

Here's the weird thing though... when I switch to doing them serially, I often 
still get bad results.
To try, change
{code}
final int nThreads = 20;
final int createsPerThread = 1;
{code}
to
{code}
final int nThreads = 1;
final int createsPerThread = 20;
{code}

It helps if I add a sleep after every create, but then I still normally end up 
with 11 and 9 instead of 10 replicas on each of the 2 nodes.

> Concurrent collection creation leads to unbalanced cluster
> --
>
> Key: SOLR-13884
> URL: https://issues.apache.org/jira/browse/SOLR-13884
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When multiple collection creations are done concurrently, the cluster can end 
> up very unbalanced, with many (or most) replicas going to a small set of 
> nodes.
> This was observed on both 8.2 and master.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Issue Comment Deleted] (LUCENE-8422) Add Matches iteration to interval queries



 [ 
https://issues.apache.org/jira/browse/LUCENE-8422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated LUCENE-8422:
-
Comment: was deleted

(was: !image-2019-10-30-12-08-40-145.png!

What worries me a little is absence of coverage in that caching code, even 
after running Intervals package tests.   

 )

> Add Matches iteration to interval queries
> -
>
> Key: LUCENE-8422
> URL: https://issues.apache.org/jira/browse/LUCENE-8422
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 7.5
>
> Attachments: LUCENE-8422.patch, LUCENE-8422.patch, LUCENE-8422.patch, 
> image-2019-10-30-12-08-40-145.png
>
>
> Follow up to LUCENE-8404, we can now add Matches iteration to interval 
> queries in the sandbox.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] dsmiley commented on issue #987: SOLR-13884: add ConcurrentCreateCollectionTest test

dsmiley commented on issue #987: SOLR-13884: add ConcurrentCreateCollectionTest 
test
URL: https://github.com/apache/lucene-solr/pull/987#issuecomment-548085272
 
 
   Your test does not use a collection autoscaling policy.  Ideally it would do 
so, I think, to show that the policy can result in a violation that is 
avoidable (if it balanced in the first place).  
   
   I was wondering if the 
org.apache.solr.cloud.api.collections.Assign.AssignStrategyFactory chooses the 
"LEGACY" strategy or "POLICY" in a default scenario as your test currently 
would do.   It chooses LEGACY, which I verified in a debugger.  The 
Policy.Session stuff that supposedly fixes this scenario is, I believe, only 
used by the collections that have a policy.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13884) Concurrent collection creation leads to unbalanced cluster



[ 
https://issues.apache.org/jira/browse/SOLR-13884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963368#comment-16963368
 ] 

Yonik Seeley commented on SOLR-13884:
-

This may not be a concurrency issue
I just tried sequential with sleeping for 10 seconds after every create and I 
got a 5/15 (5 on one node, 15 on another)!
Is the test making some sort of incorrect assumption?

> Concurrent collection creation leads to unbalanced cluster
> --
>
> Key: SOLR-13884
> URL: https://issues.apache.org/jira/browse/SOLR-13884
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When multiple collection creations are done concurrently, the cluster can end 
> up very unbalanced, with many (or most) replicas going to a small set of 
> nodes.
> This was observed on both 8.2 and master.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (LUCENE-9035) Increase doc snippet to attempt to overflow buffers at intervals.CachingMatchesIterator

Mikhail Khludnev created LUCENE-9035:


 Summary: Increase doc snippet to attempt to overflow buffers at 
intervals.CachingMatchesIterator
 Key: LUCENE-9035
 URL: https://issues.apache.org/jira/browse/LUCENE-9035
 Project: Lucene - Core
  Issue Type: Sub-task
Reporter: Mikhail Khludnev


It seems like TestIntervals.testNestedMaxGaps() is the most promising to do so. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (LUCENE-9031) UnsupportedOperationException on highlighting Interval Query



 [ 
https://issues.apache.org/jira/browse/LUCENE-9031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated LUCENE-9031:
-
Attachment: LUCENE-9031.patch

> UnsupportedOperationException on highlighting Interval Query
> 
>
> Key: LUCENE-9031
> URL: https://issues.apache.org/jira/browse/LUCENE-9031
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/queries
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Major
> Fix For: 8.4
>
> Attachments: LUCENE-9031.patch, LUCENE-9031.patch, LUCENE-9031.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When UnifiedHighlighter highlights Interval Query it encounters 
> UnsupportedOperationException. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] Pr0methean commented on a change in pull request #978: SOLR-13207: Handle misuse of < operator

Pr0methean commented on a change in pull request #978: SOLR-13207: Handle 
misuse of < operator
URL: https://github.com/apache/lucene-solr/pull/978#discussion_r340846394
 
 

 ##
 File path: solr/core/src/test/org/apache/solr/util/SolrPluginUtilsTest.java
 ##
 @@ -338,6 +339,18 @@ public void testMinShouldMatchCalculator() {
 
   }
 
+  @Test
+  public void testMinShouldMatchBadQueries() {
+try {
+  calcMSM(1, "1<");
 
 Review comment:
   Done.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] yonik commented on issue #987: SOLR-13884: add ConcurrentCreateCollectionTest test

yonik commented on issue #987: SOLR-13884: add ConcurrentCreateCollectionTest 
test
URL: https://github.com/apache/lucene-solr/pull/987#issuecomment-548102915
 
 
   Yeah, I was trying to start as simple as possible and AFAIK, defaults 
should chose notes with fewer cores.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] asfgit merged pull request #983: SOLR-13101: many updates, including concurrent update request handling

asfgit merged pull request #983: SOLR-13101: many updates, including concurrent 
update request handling
URL: https://github.com/apache/lucene-solr/pull/983
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13867) Make Solrcloud stable and performant and capable of having passing tests.

2019-10-30 Thread Mark Miller (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963435#comment-16963435
 ] 

Mark Miller commented on SOLR-13867:


The ship is full of holes and sinking as we continue to stack crap on it.

Focusing on core or not will not address the dreadful situation we are actually 
in. We have so much duct tape and bad workaround built into the very core of 
the system - on top of 0 day bugs from when the system was first propped up, 
along with a million little bugs due to bad object publishing for multiple 
threads and bad multi-threaded dev in general.

I thought I could show others a system without these fundamental issues - a 
system with tests that take mere seconds, SolrCloud or not, a system build on 
solid parts and with most of the silly concurrency bugs fixed.

But SolrCloud is a multi headed hydra. Everyone is building their own head or 
working on heads they hardly fully understand.

This thing is going to collapse under it's own weight and it cannot be repaired 
above the very core of system - tests and base SolrCloud. We need to dump our 
zk code, dump the overseer code, and dump a lot of bugs and bad ideas.

People are dumping code on faster than I know how to deal with though and few 
if anyone cases about this. They are happy building out their own private hydra 
head somewhere on a sand foundation.

Good luck, I can't fix it.

> Make Solrcloud stable and performant and capable of having passing tests.
> -
>
> Key: SOLR-13867
> URL: https://issues.apache.org/jira/browse/SOLR-13867
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
> Fix For: master (9.0)
>
>
> After spending a bit of time away from SolrCloud after being deeply involved 
> in trying to stabilize it and it's tests, I came back in 2018 and went deep 
> into the system with the Starburst upgrade.
> What I found surprised me, though I guess it should not have. The system is 
> slow, often silly, super buggy, not good at connection reuse or thread safety 
> or efficient Zookeeper communication or efficient startup and shutdown.
> Often, the things we do to make tests pass make things worse because you 
> can't do things reasonably without some major code work and so we fight for 
> tests passes, not correctness.
> Twice now, I've seen the system in the shape it was supposed to take. FAST. 
> Not bug free, but 100X more solid at least and much, much, much, much faster.
> The current system is sick and actually getting worse under it's weight as 
> more is shoveled on top. Even since 1.5 years ago, the problems are worse, 
> not better. Tests will never pass. Yes, our tests where in pretty bad shape. 
> But you can put them in the best shape possible and it won't matter. The 
> system will still fail tests.
> Sadly, I'm smart enough to know what has to be done, but not smart enough to 
> keep my work around after addressing most of the problems twice.
> Non the less, it's time to fix SolrCloud. It's not supposed to be this way. 
> I've twice spent a week or two in a state with super fast SolrCloud. Super 
> fast build system. Developmenet is actually fun. You actually have a chance. 
> I'm talking tests you have never seen take under 45-60 seconds taking 5.  
> Consistently. A different world.
> I spent a lot of time after starburst making tests pass for me. Then a lot of 
> time on a better build system that can help us improve development and good 
> practices around the project. And then a lot of time making tests faster. 
> These are important steps, but little itty bitty baby steps without 
> addressing the core rot that is growing. We don't find a problem and fully 
> understand what is up and craft a careful solution. We find something that we 
> can toss into the grand canyon, listen to it bounce around for a while, and 
> if nobody screams, we move on to the next thing. That's not necessarily 
> anyone's choice, there is little else you can do until the system is fixed. 
> When that happens we can start making smart changes instead of just shoving 
> around the mess.
> Twice I have made the current system fast. What happens first? Nothing works. 
> The system doesn't know how to be fast. It doesn't have the thread safety or 
> proper logic to be fast. And that is not a place I want to be.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Resolved] (SOLR-13867) Make Solrcloud stable and performant and capable of having passing tests.

2019-10-30 Thread Mark Miller (Jira)



 [ 
https://issues.apache.org/jira/browse/SOLR-13867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller resolved SOLR-13867.

Resolution: Won't Fix

> Make Solrcloud stable and performant and capable of having passing tests.
> -
>
> Key: SOLR-13867
> URL: https://issues.apache.org/jira/browse/SOLR-13867
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
> Fix For: master (9.0)
>
>
> After spending a bit of time away from SolrCloud after being deeply involved 
> in trying to stabilize it and it's tests, I came back in 2018 and went deep 
> into the system with the Starburst upgrade.
> What I found surprised me, though I guess it should not have. The system is 
> slow, often silly, super buggy, not good at connection reuse or thread safety 
> or efficient Zookeeper communication or efficient startup and shutdown.
> Often, the things we do to make tests pass make things worse because you 
> can't do things reasonably without some major code work and so we fight for 
> tests passes, not correctness.
> Twice now, I've seen the system in the shape it was supposed to take. FAST. 
> Not bug free, but 100X more solid at least and much, much, much, much faster.
> The current system is sick and actually getting worse under it's weight as 
> more is shoveled on top. Even since 1.5 years ago, the problems are worse, 
> not better. Tests will never pass. Yes, our tests where in pretty bad shape. 
> But you can put them in the best shape possible and it won't matter. The 
> system will still fail tests.
> Sadly, I'm smart enough to know what has to be done, but not smart enough to 
> keep my work around after addressing most of the problems twice.
> Non the less, it's time to fix SolrCloud. It's not supposed to be this way. 
> I've twice spent a week or two in a state with super fast SolrCloud. Super 
> fast build system. Developmenet is actually fun. You actually have a chance. 
> I'm talking tests you have never seen take under 45-60 seconds taking 5.  
> Consistently. A different world.
> I spent a lot of time after starburst making tests pass for me. Then a lot of 
> time on a better build system that can help us improve development and good 
> practices around the project. And then a lot of time making tests faster. 
> These are important steps, but little itty bitty baby steps without 
> addressing the core rot that is growing. We don't find a problem and fully 
> understand what is up and craft a careful solution. We find something that we 
> can toss into the grand canyon, listen to it bounce around for a while, and 
> if nobody screams, we move on to the next thing. That's not necessarily 
> anyone's choice, there is little else you can do until the system is fixed. 
> When that happens we can start making smart changes instead of just shoving 
> around the mess.
> Twice I have made the current system fast. What happens first? Nothing works. 
> The system doesn't know how to be fast. It doesn't have the thread safety or 
> proper logic to be fast. And that is not a place I want to be.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] tflobbe commented on a change in pull request #978: SOLR-13207: Handle query errors in calculateMinShouldMatch

tflobbe commented on a change in pull request #978: SOLR-13207: Handle query 
errors in calculateMinShouldMatch
URL: https://github.com/apache/lucene-solr/pull/978#discussion_r340868404
 
 

 ##
 File path: lucene/tools/src/groovy/install-markdown-filter.groovy
 ##
 @@ -28,7 +28,7 @@ import com.vladsch.flexmark.html.HtmlRenderer;
 import com.vladsch.flexmark.parser.Parser;
 import com.vladsch.flexmark.parser.ParserEmulationProfile;
 import com.vladsch.flexmark.util.html.Escaping;
-import com.vladsch.flexmark.util.options.MutableDataSet;
+import com.vladsch.flexmark.util.data.MutableDataSet;
 
 Review comment:
   Is this intentional? looks like the precommit fails because of this


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13101) Shared storage support in SolrCloud



[ 
https://issues.apache.org/jira/browse/SOLR-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963450#comment-16963450
 ] 

Yonik Seeley commented on SOLR-13101:
-

The branch https://github.com/apache/lucene-solr/tree/jira/SOLR-13101 has been 
brought up to date by this PR:
https://github.com/apache/lucene-solr/pull/983
None of the commits mentioned this JIRA, which is why I think this issue wasn't 
automatically updates.

> Shared storage support in SolrCloud
> ---
>
> Key: SOLR-13101
> URL: https://issues.apache.org/jira/browse/SOLR-13101
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Reporter: Yonik Seeley
>Priority: Major
>  Time Spent: 5h
>  Remaining Estimate: 0h
>
> Solr should have first-class support for shared storage (blob/object stores 
> like S3, google cloud storage, etc. and shared filesystems like HDFS, NFS, 
> etc).
> The key component will likely be a new replica type for shared storage.  It 
> would have many of the benefits of the current "pull" replicas (not indexing 
> on all replicas, all shards identical with no shards getting out-of-sync, 
> etc), but would have additional benefits:
>  - Any shard could become leader (the blob store always has the index)
>  - Better elasticity scaling down
>- durability not linked to number of replcias.. a single replica could be 
> common for write workloads
>- could drop to 0 replicas for a shard when not needed (blob store always 
> has index)
>  - Allow for higher performance write workloads by skipping the transaction 
> log
>- don't pay for what you don't need
>- a commit will be necessary to flush to stable storage (blob store)
>  - A lot of the complexity and failure modes go away
> An additional component a Directory implementation that will work well with 
> blob stores.  We probably want one that treats local disk as a cache since 
> the latency to remote storage is so large.  I think there are still some 
> "locking" issues to be solved here (ensuring that more than one writer to the 
> same index won't corrupt it).  This should probably be pulled out into a 
> different JIRA issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] Pr0methean commented on a change in pull request #978: SOLR-13207: Handle query errors in calculateMinShouldMatch

Pr0methean commented on a change in pull request #978: SOLR-13207: Handle query 
errors in calculateMinShouldMatch
URL: https://github.com/apache/lucene-solr/pull/978#discussion_r340881597
 
 

 ##
 File path: lucene/tools/src/groovy/install-markdown-filter.groovy
 ##
 @@ -28,7 +28,7 @@ import com.vladsch.flexmark.html.HtmlRenderer;
 import com.vladsch.flexmark.parser.Parser;
 import com.vladsch.flexmark.parser.ParserEmulationProfile;
 import com.vladsch.flexmark.util.html.Escaping;
-import com.vladsch.flexmark.util.options.MutableDataSet;
+import com.vladsch.flexmark.util.data.MutableDataSet;
 
 Review comment:
   Reverted.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (LUCENE-9014) Design a new solution for publishing JavaDoc



 [ 
https://issues.apache.org/jira/browse/LUCENE-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated LUCENE-9014:

Attachment: LUCENE-9014.patch

> Design a new solution for publishing JavaDoc
> 
>
> Key: LUCENE-9014
> URL: https://issues.apache.org/jira/browse/LUCENE-9014
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: LUCENE-9014.patch
>
>
> Currently we check in to svn folders with hundreds of html files for JavaDoc. 
> Need a better way.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9014) Design a new solution for publishing JavaDoc



[ 
https://issues.apache.org/jira/browse/LUCENE-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963473#comment-16963473
 ] 

Jan Høydahl commented on LUCENE-9014:
-

Added a simple patch to the index.xsl files that generate index.html for the 
documentation, so that the links will point to javadoc.io instead of assuming 
the javadoc folders are pushed to the lucene site. The other part that needs to 
be done is simply to update release docs and scripts to make sure we do NOT 
copy those *13.615* javadoc html files to our website during a release. This 
will likely make the release process faster too!

> Design a new solution for publishing JavaDoc
> 
>
> Key: LUCENE-9014
> URL: https://issues.apache.org/jira/browse/LUCENE-9014
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: LUCENE-9014.patch
>
>
> Currently we check in to svn folders with hundreds of html files for JavaDoc. 
> Need a better way.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13101) Shared storage support in SolrCloud

2019-10-30 Thread Noble Paul (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963477#comment-16963477
 ] 

Noble Paul commented on SOLR-13101:
---

Is it possible to give an example of how one would use this new feature?

like workflow, APIs, public interfaces, new additions to ZK etc?

> Shared storage support in SolrCloud
> ---
>
> Key: SOLR-13101
> URL: https://issues.apache.org/jira/browse/SOLR-13101
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Reporter: Yonik Seeley
>Priority: Major
>  Time Spent: 5h
>  Remaining Estimate: 0h
>
> Solr should have first-class support for shared storage (blob/object stores 
> like S3, google cloud storage, etc. and shared filesystems like HDFS, NFS, 
> etc).
> The key component will likely be a new replica type for shared storage.  It 
> would have many of the benefits of the current "pull" replicas (not indexing 
> on all replicas, all shards identical with no shards getting out-of-sync, 
> etc), but would have additional benefits:
>  - Any shard could become leader (the blob store always has the index)
>  - Better elasticity scaling down
>- durability not linked to number of replcias.. a single replica could be 
> common for write workloads
>- could drop to 0 replicas for a shard when not needed (blob store always 
> has index)
>  - Allow for higher performance write workloads by skipping the transaction 
> log
>- don't pay for what you don't need
>- a commit will be necessary to flush to stable storage (blob store)
>  - A lot of the complexity and failure modes go away
> An additional component a Directory implementation that will work well with 
> blob stores.  We probably want one that treats local disk as a cache since 
> the latency to remote storage is so large.  I think there are still some 
> "locking" issues to be solved here (ensuring that more than one writer to the 
> same index won't corrupt it).  This should probably be pulled out into a 
> different JIRA issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (LUCENE-9014) Design a new solution for publishing JavaDoc



 [ 
https://issues.apache.org/jira/browse/LUCENE-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated LUCENE-9014:

Attachment: LUCENE-9014.patch

> Design a new solution for publishing JavaDoc
> 
>
> Key: LUCENE-9014
> URL: https://issues.apache.org/jira/browse/LUCENE-9014
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: LUCENE-9014.patch, LUCENE-9014.patch
>
>
> Currently we check in to svn folders with hundreds of html files for JavaDoc. 
> Need a better way.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9014) Design a new solution for publishing JavaDoc



[ 
https://issues.apache.org/jira/browse/LUCENE-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963486#comment-16963486
 ] 

Uwe Schindler commented on LUCENE-9014:
---

Sorry, I am against relying on an non-ASF controlled service for publishing the 
documentation of Apache Lucene. The index.html is something that is archived in 
our artifacts. The links in it should be PERSISTENT, which may not be the case.

If we want to make the publishing of Javadocs easier, IMHO we should use 
another webserver which can server files from ZIP/JAR files. My favourite is 
Jetty - it can use a JAR/ZIP file as source for static file delivery (it's a 
handler). I'd ask Infra, if this is possible. ZIP the build/docs folder and 
place it in SVN. The delivery of the files is then

The javadoc.io version navigation is just a farme on top of the other 
framesets. That's easy to do as part of the CMS on our side, too.

> Design a new solution for publishing JavaDoc
> 
>
> Key: LUCENE-9014
> URL: https://issues.apache.org/jira/browse/LUCENE-9014
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: LUCENE-9014.patch, LUCENE-9014.patch
>
>
> Currently we check in to svn folders with hundreds of html files for JavaDoc. 
> Need a better way.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-9014) Design a new solution for publishing JavaDoc



[ 
https://issues.apache.org/jira/browse/LUCENE-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963486#comment-16963486
 ] 

Uwe Schindler edited comment on LUCENE-9014 at 10/30/19 10:35 PM:
--

Sorry, I am against relying on an non-ASF controlled service for publishing the 
documentation of Apache Lucene. The index.html is something that is archived in 
our artifacts. The links in it should be PERSISTENT, which may not be the case.

If we want to make the publishing of Javadocs easier, IMHO we should use 
another webserver which can serve files from inside ZIP/JAR files. My favourite 
is Jetty - it can use a JAR/ZIP file as source for static file delivery (it's a 
handler). I'd ask Infra, if this is possible. ZIP the build/docs folder and 
place it in SVN. The delivery of the files is then

The javadoc.io version navigation is just a farme on top of the other 
framesets. That's easy to do as part of the CMS on our side, too.


was (Author: thetaphi):
Sorry, I am against relying on an non-ASF controlled service for publishing the 
documentation of Apache Lucene. The index.html is something that is archived in 
our artifacts. The links in it should be PERSISTENT, which may not be the case.

If we want to make the publishing of Javadocs easier, IMHO we should use 
another webserver which can server files from ZIP/JAR files. My favourite is 
Jetty - it can use a JAR/ZIP file as source for static file delivery (it's a 
handler). I'd ask Infra, if this is possible. ZIP the build/docs folder and 
place it in SVN. The delivery of the files is then

The javadoc.io version navigation is just a farme on top of the other 
framesets. That's easy to do as part of the CMS on our side, too.

> Design a new solution for publishing JavaDoc
> 
>
> Key: LUCENE-9014
> URL: https://issues.apache.org/jira/browse/LUCENE-9014
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: LUCENE-9014.patch, LUCENE-9014.patch
>
>
> Currently we check in to svn folders with hundreds of html files for JavaDoc. 
> Need a better way.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-9014) Design a new solution for publishing JavaDoc



[ 
https://issues.apache.org/jira/browse/LUCENE-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963486#comment-16963486
 ] 

Uwe Schindler edited comment on LUCENE-9014 at 10/30/19 10:36 PM:
--

Sorry, I am against relying on an non-ASF controlled service for publishing the 
documentation of Apache Lucene. The index.html is something that is archived in 
our artifacts. The links in it should be PERSISTENT, which may not be the case.

If we want to make the publishing of Javadocs easier, IMHO we should use 
another webserver which can serve files from inside ZIP/JAR files. My favourite 
is Jetty - it can use a JAR/ZIP file as source for static file delivery (it's a 
handler). I'd ask Infra, if this is possible. ZIP the build/docs folder and 
place it in SVN. The delivery of the files is then just a simple Jetty Handler 
configuration. As far as I remeber, also NGINX can do this (with a plugin): 
https://github.com/youzee/nginx-unzip-module

The javadoc.io version navigation is just a farme on top of the other 
framesets. That's easy to do as part of the CMS on our side, too.


was (Author: thetaphi):
Sorry, I am against relying on an non-ASF controlled service for publishing the 
documentation of Apache Lucene. The index.html is something that is archived in 
our artifacts. The links in it should be PERSISTENT, which may not be the case.

If we want to make the publishing of Javadocs easier, IMHO we should use 
another webserver which can serve files from inside ZIP/JAR files. My favourite 
is Jetty - it can use a JAR/ZIP file as source for static file delivery (it's a 
handler). I'd ask Infra, if this is possible. ZIP the build/docs folder and 
place it in SVN. The delivery of the files is then

The javadoc.io version navigation is just a farme on top of the other 
framesets. That's easy to do as part of the CMS on our side, too.

> Design a new solution for publishing JavaDoc
> 
>
> Key: LUCENE-9014
> URL: https://issues.apache.org/jira/browse/LUCENE-9014
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: LUCENE-9014.patch, LUCENE-9014.patch
>
>
> Currently we check in to svn folders with hundreds of html files for JavaDoc. 
> Need a better way.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] tflobbe commented on a change in pull request #984: SOLR-12217: Support shards.preference for individual shard requests

tflobbe commented on a change in pull request #984: SOLR-12217: Support 
shards.preference for individual shard requests
URL: https://github.com/apache/lucene-solr/pull/984#discussion_r340880125
 
 

 ##
 File path: 
solr/solrj/src/java/org/apache/solr/client/solrj/impl/BaseCloudSolrClient.java
 ##
 @@ -1109,11 +1125,12 @@ public RouteException(ErrorCode errorCode, 
NamedList throwables, Map<
 }
   }
 
-  // Shuffle the leaders, if any(none if !sendToLeaders)
+  // Sort the leader replicas, if any, according to the request 
preferences(none if !sendToLeaders)
+  replicaListTransformer.transform(replicas);
 
 Review comment:
   `theUrlList`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] tflobbe commented on a change in pull request #984: SOLR-12217: Support shards.preference for individual shard requests

tflobbe commented on a change in pull request #984: SOLR-12217: Support 
shards.preference for individual shard requests
URL: https://github.com/apache/lucene-solr/pull/984#discussion_r340877793
 
 

 ##
 File path: 
solr/solrj/src/java/org/apache/solr/client/solrj/impl/BaseCloudSolrClient.java
 ##
 @@ -651,6 +659,13 @@ protected RouteException 
getRouteException(SolrException.ErrorCode serverError,
   }
 }
   }
+
+  // Sort the non-leader replicas according to the request parameters
+  replicaListTransformer.transform(urls);
 
 Review comment:
   Is this necessary for updates? are you trying to keep consistency only or do 
you have some use case in mind?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] tflobbe commented on a change in pull request #984: SOLR-12217: Support shards.preference for individual shard requests

tflobbe commented on a change in pull request #984: SOLR-12217: Support 
shards.preference for individual shard requests
URL: https://github.com/apache/lucene-solr/pull/984#discussion_r340880694
 
 

 ##
 File path: 
solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/SignificantTermsStream.java
 ##
 @@ -282,6 +281,7 @@ public Explanation toExplanation(StreamFactory factory) 
throws IOException {
   }
 
   public Tuple read() throws IOException {
+
 
 Review comment:
   Maybe remove this file from the PR


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9028) Public method for MultiTermIntervalSource



[ 
https://issues.apache.org/jira/browse/LUCENE-9028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963503#comment-16963503
 ] 

Mikhail Khludnev commented on LUCENE-9028:
--

I think it may go into. There are minor questions: do we need the second method 
defaulting 128? Should it accept Automaton or CompiledAutomaton? 

> Public method for MultiTermIntervalSource
> -
>
> Key: LUCENE-9028
> URL: https://issues.apache.org/jira/browse/LUCENE-9028
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Major
> Attachments: LUCENE-9028.patch
>
>
> Right now we have prefix and widlcard for multiterm Intervals. Sometimes it's 
> necessary to provide terms set in a more generic way as automaton.
> {code:java}
> Intervals.multiterm(CompiledAutomaton automaton, int maxExpansions, String 
> pattern)
> {code}
> As a benefit we can handle it more efficient rather than or over terms. 
> What do you think? 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9014) Design a new solution for publishing JavaDoc