[GitHub] [lucene-solr] dweiss commented on a change in pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-12-04 Thread GitBox


dweiss commented on a change in pull request #2052:
URL: https://github.com/apache/lucene-solr/pull/2052#discussion_r535921190



##
File path: lucene/misc/native/build.gradle
##
@@ -51,11 +47,7 @@ tasks.withType(CppCompile).configureEach {
   // is present.
   systemIncludes.from file("${javaHome}/include")
 
-  for (def path : [

Review comment:
   "Soften the blow", you say? :) Even the cover of the album this song is 
from kind of reflects the nature of open source, doesn't it? 
   https://www.youtube.com/watch?v=G1pfrCOQWHY





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] vonbox opened a new pull request #2118: SOLR-15031: Prevent null being wrapped in a QueryValueSource

2020-12-04 Thread GitBox


vonbox opened a new pull request #2118:
URL: https://github.com/apache/lucene-solr/pull/2118


   # Description
   
   When parsing a sub query in a function query, 
FunctionQParser#parseValueSource does not check if the produced query object is 
null. When it is, it just wraps a null in a QueryValueSource object. This is a 
cause for NPE's in code consuming that object. Parsed queries can be null, for 
example when the query string only contains stopwords, so we need handle that 
condition.
   
   # Solution
   
   Added a null check to FunctionQParser#parseValueSource
   
   # Tests
   
   The problem can be reproduced via:
   1. Start solr with the techproducts example collection: solr start -e 
techproducts
   2. Add a stopword to 
SOLR_DIR/example/techproducts/solr/techproducts/conf/stopwords.txt, for example 
"at"
   3. Execute a function query: 
http://localhost:8983/solr/techproducts/select?fieldquery={!field%20f=features%20v=%27%22at%22%27}&q={!func}%20if($fieldquery,1,0)
   
   After applying this PR the NPE is gone.
   
   # Checklist
   
   Please review the following and check all that apply:
   
   - [x] I have reviewed the guidelines for [How to 
Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms 
to the standards described there to the best of my ability.
   - [x] I have created a Jira issue and added the issue ID to my pull request 
title.
   - [x] I have given Solr maintainers 
[access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork)
 to contribute to my PR branch. (optional but recommended)
   - [x] I have developed this patch against the `master` branch.
   - [x] I have run `./gradlew check`.
   - [x] I have added tests for my changes.
   - [x] I have added documentation for the [Ref 
Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) 
(for Solr changes only).
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-15031) NPE caused by FunctionQParser returning a null ValueSource

2020-12-04 Thread Pieter (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-15031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17243908#comment-17243908
 ] 

Pieter commented on SOLR-15031:
---

PR fixing this has been created.

> NPE caused by FunctionQParser returning a null ValueSource
> --
>
> Key: SOLR-15031
> URL: https://issues.apache.org/jira/browse/SOLR-15031
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pieter
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When parsing a sub query in a function query, 
> {{FunctionQParser#parseValueSource}} does not check if the produced query 
> object is null. When it is, it just wraps a null in a {{QueryValueSource}} 
> object. This is a cause for NPE's in code consuming that object. Parsed 
> queries can be null, for example when the query string only contains 
> stopwords, so we need handle that condition.
> h3. Steps to reproduce the issue
> # Start solr with the techproducts example collection: {{solr start -e 
> techproducts}}
> # Add a stopword to 
> SOLR_DIR/example/techproducts/solr/techproducts/conf/stopwords.txt, for 
> example "at"
> # Execute a function query: 
> {code}http://localhost:8983/solr/techproducts/select?fieldquery={!field%20f=features%20v=%27%22at%22%27}&q={!func}%20if($fieldquery,1,0){code}
> The following stacktrace is produced:
> {code}
> 2020-12-03 13:35:38.868 INFO  (qtp2095677157-21) [   x:techproducts] 
> o.a.s.c.S.Request [techproducts]  webapp=/solr path=/select 
> params={q={!func}+if($fieldquery,1,0)&fieldquery={!field+f%3Dfeatures+v%3D'"at"'}}
>  status=500 QTime=34
> 2020-12-03 13:35:38.872 ERROR (qtp2095677157-21) [   x:techproducts] 
> o.a.s.s.HttpSolrCall null:java.lang.NullPointerException
> at 
> org.apache.lucene.queries.function.valuesource.QueryValueSource.hashCode(QueryValueSource.java:63)
> at 
> org.apache.lucene.queries.function.valuesource.IfFunction.hashCode(IfFunction.java:129)
> at 
> org.apache.lucene.queries.function.FunctionQuery.hashCode(FunctionQuery.java:176)
> at 
> org.apache.solr.search.QueryResultKey.(QueryResultKey.java:53)
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1341)
> at 
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:580)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Comment Edited] (SOLR-15031) NPE caused by FunctionQParser returning a null ValueSource

2020-12-04 Thread Pieter (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-15031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17243908#comment-17243908
 ] 

Pieter edited comment on SOLR-15031 at 12/4/20, 10:01 AM:
--

[PR|https://github.com/apache/lucene-solr/pull/2118] fixing this has been 
created.


was (Author: boxtelp):
PR fixing this has been created.

> NPE caused by FunctionQParser returning a null ValueSource
> --
>
> Key: SOLR-15031
> URL: https://issues.apache.org/jira/browse/SOLR-15031
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pieter
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When parsing a sub query in a function query, 
> {{FunctionQParser#parseValueSource}} does not check if the produced query 
> object is null. When it is, it just wraps a null in a {{QueryValueSource}} 
> object. This is a cause for NPE's in code consuming that object. Parsed 
> queries can be null, for example when the query string only contains 
> stopwords, so we need handle that condition.
> h3. Steps to reproduce the issue
> # Start solr with the techproducts example collection: {{solr start -e 
> techproducts}}
> # Add a stopword to 
> SOLR_DIR/example/techproducts/solr/techproducts/conf/stopwords.txt, for 
> example "at"
> # Execute a function query: 
> {code}http://localhost:8983/solr/techproducts/select?fieldquery={!field%20f=features%20v=%27%22at%22%27}&q={!func}%20if($fieldquery,1,0){code}
> The following stacktrace is produced:
> {code}
> 2020-12-03 13:35:38.868 INFO  (qtp2095677157-21) [   x:techproducts] 
> o.a.s.c.S.Request [techproducts]  webapp=/solr path=/select 
> params={q={!func}+if($fieldquery,1,0)&fieldquery={!field+f%3Dfeatures+v%3D'"at"'}}
>  status=500 QTime=34
> 2020-12-03 13:35:38.872 ERROR (qtp2095677157-21) [   x:techproducts] 
> o.a.s.s.HttpSolrCall null:java.lang.NullPointerException
> at 
> org.apache.lucene.queries.function.valuesource.QueryValueSource.hashCode(QueryValueSource.java:63)
> at 
> org.apache.lucene.queries.function.valuesource.IfFunction.hashCode(IfFunction.java:129)
> at 
> org.apache.lucene.queries.function.FunctionQuery.hashCode(FunctionQuery.java:176)
> at 
> org.apache.solr.search.QueryResultKey.(QueryResultKey.java:53)
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1341)
> at 
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:580)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-15031) NPE caused by FunctionQParser returning a null ValueSource

2020-12-04 Thread Pieter (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-15031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17243910#comment-17243910
 ] 

Pieter commented on SOLR-15031:
---

FYI; the reproduction query is a bit silly, but this is just a simplified 
version of our real life queries. We use the function queries in a sort clause, 
to pull additional features like freshness in the ranking and to sort special 
documents on top, etc.

> NPE caused by FunctionQParser returning a null ValueSource
> --
>
> Key: SOLR-15031
> URL: https://issues.apache.org/jira/browse/SOLR-15031
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pieter
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When parsing a sub query in a function query, 
> {{FunctionQParser#parseValueSource}} does not check if the produced query 
> object is null. When it is, it just wraps a null in a {{QueryValueSource}} 
> object. This is a cause for NPE's in code consuming that object. Parsed 
> queries can be null, for example when the query string only contains 
> stopwords, so we need handle that condition.
> h3. Steps to reproduce the issue
> # Start solr with the techproducts example collection: {{solr start -e 
> techproducts}}
> # Add a stopword to 
> SOLR_DIR/example/techproducts/solr/techproducts/conf/stopwords.txt, for 
> example "at"
> # Execute a function query: 
> {code}http://localhost:8983/solr/techproducts/select?fieldquery={!field%20f=features%20v=%27%22at%22%27}&q={!func}%20if($fieldquery,1,0){code}
> The following stacktrace is produced:
> {code}
> 2020-12-03 13:35:38.868 INFO  (qtp2095677157-21) [   x:techproducts] 
> o.a.s.c.S.Request [techproducts]  webapp=/solr path=/select 
> params={q={!func}+if($fieldquery,1,0)&fieldquery={!field+f%3Dfeatures+v%3D'"at"'}}
>  status=500 QTime=34
> 2020-12-03 13:35:38.872 ERROR (qtp2095677157-21) [   x:techproducts] 
> o.a.s.s.HttpSolrCall null:java.lang.NullPointerException
> at 
> org.apache.lucene.queries.function.valuesource.QueryValueSource.hashCode(QueryValueSource.java:63)
> at 
> org.apache.lucene.queries.function.valuesource.IfFunction.hashCode(IfFunction.java:129)
> at 
> org.apache.lucene.queries.function.FunctionQuery.hashCode(FunctionQuery.java:176)
> at 
> org.apache.solr.search.QueryResultKey.(QueryResultKey.java:53)
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1341)
> at 
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:580)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-15031) NPE caused by FunctionQParser returning a null ValueSource

2020-12-04 Thread Pieter (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-15031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pieter updated SOLR-15031:
--
Description: 
When parsing a sub query in a function query, 
{{FunctionQParser#parseValueSource}} does not check if the produced query 
object is null. When it is, it just wraps a null in a {{QueryValueSource}} 
object. This is a cause for NPE's in code consuming that object. Parsed queries 
can be null, for example when the query string only contains stopwords, so we 
need handle that condition.
h3. Steps to reproduce the issue
 # Start solr with the techproducts example collection: {{solr start -e 
techproducts}}
 # Add a stopword to 
SOLR_DIR/example/techproducts/solr/techproducts/conf/stopwords.txt, for example 
"at"
 # Reload the core
 # Execute a function query:
{code:java}
http://localhost:8983/solr/techproducts/select?fieldquery={!field%20f=features%20v=%27%22at%22%27}&q={!func}%20if($fieldquery,1,0){code}

The following stacktrace is produced:
{code:java}
2020-12-03 13:35:38.868 INFO  (qtp2095677157-21) [   x:techproducts] 
o.a.s.c.S.Request [techproducts]  webapp=/solr path=/select 
params={q={!func}+if($fieldquery,1,0)&fieldquery={!field+f%3Dfeatures+v%3D'"at"'}}
 status=500 QTime=34
2020-12-03 13:35:38.872 ERROR (qtp2095677157-21) [   x:techproducts] 
o.a.s.s.HttpSolrCall null:java.lang.NullPointerException
at 
org.apache.lucene.queries.function.valuesource.QueryValueSource.hashCode(QueryValueSource.java:63)
at 
org.apache.lucene.queries.function.valuesource.IfFunction.hashCode(IfFunction.java:129)
at 
org.apache.lucene.queries.function.FunctionQuery.hashCode(FunctionQuery.java:176)
at org.apache.solr.search.QueryResultKey.(QueryResultKey.java:53)
at 
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1341)
at 
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:580)
{code}

  was:
When parsing a sub query in a function query, 
{{FunctionQParser#parseValueSource}} does not check if the produced query 
object is null. When it is, it just wraps a null in a {{QueryValueSource}} 
object. This is a cause for NPE's in code consuming that object. Parsed queries 
can be null, for example when the query string only contains stopwords, so we 
need handle that condition.

h3. Steps to reproduce the issue

# Start solr with the techproducts example collection: {{solr start -e 
techproducts}}
# Add a stopword to 
SOLR_DIR/example/techproducts/solr/techproducts/conf/stopwords.txt, for example 
"at"
# Execute a function query: 
{code}http://localhost:8983/solr/techproducts/select?fieldquery={!field%20f=features%20v=%27%22at%22%27}&q={!func}%20if($fieldquery,1,0){code}

The following stacktrace is produced:
{code}
2020-12-03 13:35:38.868 INFO  (qtp2095677157-21) [   x:techproducts] 
o.a.s.c.S.Request [techproducts]  webapp=/solr path=/select 
params={q={!func}+if($fieldquery,1,0)&fieldquery={!field+f%3Dfeatures+v%3D'"at"'}}
 status=500 QTime=34
2020-12-03 13:35:38.872 ERROR (qtp2095677157-21) [   x:techproducts] 
o.a.s.s.HttpSolrCall null:java.lang.NullPointerException
at 
org.apache.lucene.queries.function.valuesource.QueryValueSource.hashCode(QueryValueSource.java:63)
at 
org.apache.lucene.queries.function.valuesource.IfFunction.hashCode(IfFunction.java:129)
at 
org.apache.lucene.queries.function.FunctionQuery.hashCode(FunctionQuery.java:176)
at org.apache.solr.search.QueryResultKey.(QueryResultKey.java:53)
at 
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1341)
at 
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:580)
{code}


> NPE caused by FunctionQParser returning a null ValueSource
> --
>
> Key: SOLR-15031
> URL: https://issues.apache.org/jira/browse/SOLR-15031
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pieter
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When parsing a sub query in a function query, 
> {{FunctionQParser#parseValueSource}} does not check if the produced query 
> object is null. When it is, it just wraps a null in a {{QueryValueSource}} 
> object. This is a cause for NPE's in code consuming that object. Parsed 
> queries can be null, for example when the query string only contains 
> stopwords, so we need handle that condition.
> h3. Steps to reproduce the issue
>  # Start solr with the techproducts example collection: {{solr start -e 
> techproducts}}
>  # Add a stopword to 
> SOLR_DIR/example/techproducts/solr/techproducts/conf/stopwords.txt, for 
> example "at"
>  # Reload the core
>  # Execute a function query:
> {code:java}
> http://localhost:8983/solr/techproducts/select?field

[GitHub] [lucene-solr] mayya-sharipova merged pull request #2117: LUCENE-9599 Disable sort optim on index sort

2020-12-04 Thread GitBox


mayya-sharipova merged pull request #2117:
URL: https://github.com/apache/lucene-solr/pull/2117


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (LUCENE-9599) Disable sort optimization in comparators on index sort

2020-12-04 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/LUCENE-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244006#comment-17244006
 ] 

ASF subversion and git services commented on LUCENE-9599:
-

Commit 7b3dbfedd95135aecaa4dca053cc63aabb422f24 in lucene-solr's branch 
refs/heads/branch_8x from Mayya Sharipova
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=7b3dbfe ]

LUCENE-9599 Disable sort optim on index sort (#2117)

Disable sort optimization in comparators on index sort.

Currently, if search sort is equal or a part of the index sort, we have
an early termination in TopFieldCollector.
But comparators are not aware of the index sort, and may run
sort optimization even if the search sort is congruent with
the index sort.

This patch:
- adds `disableSkipping` method to `FieldComparator`,
This method is called by `TopFieldCollector`, when the search sort
is congruent with the index sort.
It is also called when we can't use points for sort optimization.
- disables sort optimization in comparators in these cases.

Relates to #1351
Backport for #2075

> Disable sort optimization in comparators on index sort
> --
>
> Key: LUCENE-9599
> URL: https://issues.apache.org/jira/browse/LUCENE-9599
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Mayya Sharipova
>Priority: Minor
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> LUCENE-9280 introduced an ability for comparators to skip non-competitive 
> documents. But currently comparators are not aware of index sorting, and can 
> run sort optimization even when search sort is congruent with the index sort.
> As the early termination in this case is already handled in 
> TopFieldCollector, we need to disable sort optimization in comparators. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] heqianw opened a new pull request #2119: Added support to download index folders from gcs url

2020-12-04 Thread GitBox


heqianw opened a new pull request #2119:
URL: https://github.com/apache/lucene-solr/pull/2119


   Added support in luke to read from GCS URI. Downloads the file/folder 
locally and points the resulting browse path that location. 
   
   To use: verify that BUCKET_NAME and project_id are properly set.
   
   Run `./gradlew lucene:luke:assemble`
   Run the last `java -jar [..]` command that was written to console



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-15029) Allow Shard Leader to give up leadership gracefully via shard terms

2020-12-04 Thread Mike Drob (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-15029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244119#comment-17244119
 ] 

Mike Drob commented on SOLR-15029:
--

The existing code for increasing shard terms works when you ask it to exclude 
the leader. That was a pleasant surprise, and I've added a unit test and some 
documentation for this locally. I don't think this breaks the original term 
increment rules too badly, and doesn't even require a new method signature, 
since it is still only the leader doing the increment.

After that, I think we can issue a FORCELEADER command, we might need to add a 
flag to it to be more strict on who it will accept as a leader. Currently it 
looks like it would allow an out of sync replica to become leader (useful when 
you are just trying to get _something_ online), but in this case we want to 
explicitly limit to the highest shard term.

> Allow Shard Leader to give up leadership gracefully via shard terms
> ---
>
> Key: SOLR-15029
> URL: https://issues.apache.org/jira/browse/SOLR-15029
> Project: Solr
>  Issue Type: Improvement
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
>
> Currently we have (via SOLR-12412) that when a leader sees an index writing 
> error during an update it will give up leadership by deleting the replica and 
> adding a new replica. One stated benefit of this was that because we are 
> using the overseer and a known code path, that this is done asynchronous and 
> very efficiently.
> I would argue that this approach is too heavy handed.
> In the case of a corrupt index exception, it makes some sense to completely 
> delete the index dir and attempt to sync from a good peer. Even in this case, 
> however, it might be better to allow fingerprinting and other index delta 
> mechanisms take over and allow for a more efficient data transfer.
> In an alternate case where the index error arises due to a disconnected file 
> system (possible with shared file systems, i.e. S3, HDFS, some k8s systems) 
> and the required solution is some kind of reconnect, then this approach has 
> several shortcomings - the core delete and creations are going to fail 
> leaving dangling replicas. Further, the data is still present so there is no 
> need to do so many extra copies.
> I propose that we bring in a mechanism to give up leadership via the existing 
> shard terms language. I believe we would be able to set all replicas 
> currently equal to leader term T to T+1, and then trigger a new leader 
> election. The current leader would know it is ineligible, while the other 
> replicas that were current before the failed update would be eligible. This 
> improvement would entail adding an additional possible operation to terms 
> state machine.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] heqianw closed pull request #2119: Added support to download index folders from gcs url

2020-12-04 Thread GitBox


heqianw closed pull request #2119:
URL: https://github.com/apache/lucene-solr/pull/2119


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (LUCENE-9629) Use computed mask values in ForUtil

2020-12-04 Thread Feng Guo (Jira)


[ 
https://issues.apache.org/jira/browse/LUCENE-9629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244139#comment-17244139
 ] 

Feng Guo commented on LUCENE-9629:
--

[~jpountz] Thank you very much!

> Use computed mask values in ForUtil
> ---
>
> Key: LUCENE-9629
> URL: https://issues.apache.org/jira/browse/LUCENE-9629
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/codecs
>Reporter: Feng Guo
>Priority: Minor
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> In the class ForkUtil, mask values have been computed and stored in static 
> final vailables, but they are recomputed for every encoding, which may be 
> unnecessary. 
> anther small fix is that change
> {code:java}
> remainingBitsPerValue > remainingBitsPerLong{code}
>  to
> {code:java}
> remainingBitsPerValue >= remainingBitsPerLong{code}
> otherwise
> {code:java}
> if (remainingBitsPerValue == 0) {
>  idx++;
>  remainingBitsPerValue = bitsPerValue; 
> }
> {code}
> these code will never be used.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] gf2121 commented on pull request #2113: LUCENE-9629: Use computed masks

2020-12-04 Thread GitBox


gf2121 commented on pull request #2113:
URL: https://github.com/apache/lucene-solr/pull/2113#issuecomment-738901135


   @dweiss Thank you very much for the guiding of how to do a benchmark on 
Lucene, the lueneutil tool is really nice! 
   I repeatedly execute the wikimedium1m 20 times, the following is the result 
of the last iter. I guess it indicates that the reading performance is stable 
overall :)
   
   ```
TaskQPSbaseline   StdDevQPS   my_mod_ver StdDev 
 Pct diff p-value
  HighTermMonthSort  405.40 (11.6%)  390.20 (11.1%)   
-3.7% ( -23% -   21%) 0.295
   PKLookup  131.35  (4.5%)  128.26  (6.5%)   
-2.4% ( -12% -9%) 0.183
Respell   88.96  (8.0%)   86.91  (8.7%)   
-2.3% ( -17% -   15%) 0.383
 AndHighLow  611.88  (5.6%)  603.60  (9.2%)   
-1.4% ( -15% -   14%) 0.575
   BrowseDayOfYearTaxoFacets   21.70  (7.7%)   21.43  (9.1%)   
-1.3% ( -16% -   16%) 0.630
   HighSloppyPhrase   74.97  (4.8%)   74.10  (8.6%)   
-1.2% ( -13% -   12%) 0.601
  BrowseMonthSSDVFacets   91.65  (4.8%)   90.66  (5.8%)   
-1.1% ( -11% -9%) 0.519
  MedPhrase  110.37  (8.8%)  109.19  (7.3%)   
-1.1% ( -15% -   16%) 0.674
  OrHighMed  167.04  (8.9%)  165.47  (7.6%)   
-0.9% ( -16% -   17%) 0.718
 OrHighHigh   86.33  (9.1%)   85.61  (7.9%)   
-0.8% ( -16% -   17%) 0.755
LowSpanNear  156.83  (7.1%)  155.74  (6.5%)   
-0.7% ( -13% -   13%) 0.746
   Wildcard  139.21  (7.0%)  138.29  (9.8%)   
-0.7% ( -16% -   17%) 0.805
   BrowseDayOfYearSSDVFacets   78.95  (5.0%)   78.52  (5.8%)   
-0.5% ( -10% -   10%) 0.753
 Fuzzy1   76.09  (9.7%)   75.84  (7.7%)   
-0.3% ( -16% -   18%) 0.905
MedSloppyPhrase   48.37  (5.9%)   48.22  (5.1%)   
-0.3% ( -10% -   11%) 0.859
 IntNRQ  310.73 (10.5%)  310.11 (12.2%)   
-0.2% ( -20% -   25%) 0.955
   HighTerm  759.07  (7.3%)  757.84 (12.1%)   
-0.2% ( -18% -   20%) 0.959
  BrowseMonthTaxoFacets   24.17  (9.5%)   24.19 (10.0%)
0.1% ( -17% -   21%) 0.984
   HighIntervalsOrdered  121.98  (5.8%)  122.10  (7.9%)
0.1% ( -12% -   14%) 0.964
LowSloppyPhrase  188.90  (7.8%)  189.42  (5.7%)
0.3% ( -12% -   14%) 0.898
 AndHighMed  418.22  (9.0%)  420.49  (9.7%)
0.5% ( -16% -   21%) 0.855
MedTerm  874.56  (7.6%)  880.02 (10.9%)
0.6% ( -16% -   20%) 0.833
MedSpanNear  378.96  (7.2%)  381.70  (9.2%)
0.7% ( -14% -   18%) 0.781
 Fuzzy2   25.74  (9.8%)   25.97 (10.9%)
0.9% ( -17% -   23%) 0.777
AndHighHigh  126.51  (6.1%)  127.75  (8.5%)
1.0% ( -12% -   16%) 0.676
 HighPhrase  165.71  (8.2%)  167.64 (11.3%)
1.2% ( -16% -   22%) 0.708
   BrowseDateTaxoFacets   21.40 (10.4%)   21.69 (10.0%)
1.3% ( -17% -   24%) 0.682
LowTerm 1032.43  (8.0%) 1049.57 (10.1%)
1.7% ( -15% -   21%) 0.566
  OrHighLow  228.54  (5.0%)  232.36  (8.5%)
1.7% ( -11% -   15%) 0.446
   HighSpanNear   93.72  (7.4%)   95.46  (7.3%)
1.9% ( -12% -   17%) 0.427
  HighTermDayOfYearSort  380.08 (12.2%)  387.47  (9.4%)
1.9% ( -17% -   26%) 0.573
  LowPhrase  137.13  (8.2%)  140.30  (6.2%)
2.3% ( -11% -   18%) 0.314
Prefix3  290.55 (11.0%)  299.98 (12.9%)
3.2% ( -18% -   30%) 0.390
   ```



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] murblanc commented on a change in pull request #2101: SOLR-15016 Replica placement plugins should use container plugins API / configs

2020-12-04 Thread GitBox


murblanc commented on a change in pull request #2101:
URL: https://github.com/apache/lucene-solr/pull/2101#discussion_r536256783



##
File path: 
solr/core/src/java/org/apache/solr/cluster/placement/PlacementPluginFactory.java
##
@@ -17,15 +17,39 @@
 
 package org.apache.solr.cluster.placement;
 
+import org.apache.solr.api.ConfigurablePlugin;
+
 /**
- * Factory implemented by client code and configured in {@code solr.xml} 
allowing the creation of instances of
+ * Factory implemented by client code and configured in container plugins 
allowing the creation of instances of

Review comment:
   Can we add a reference to how "container plugins" are configured? Not 
obvious for me and I guess I'm likely not the only one...





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] murblanc commented on a change in pull request #2101: SOLR-15016 Replica placement plugins should use container plugins API / configs

2020-12-04 Thread GitBox


murblanc commented on a change in pull request #2101:
URL: https://github.com/apache/lucene-solr/pull/2101#discussion_r536257122



##
File path: 
solr/core/src/java/org/apache/solr/cluster/placement/PlacementPluginFactory.java
##
@@ -17,15 +17,39 @@
 
 package org.apache.solr.cluster.placement;
 
+import org.apache.solr.api.ConfigurablePlugin;
+
 /**
- * Factory implemented by client code and configured in {@code solr.xml} 
allowing the creation of instances of
+ * Factory implemented by client code and configured in container plugins 
allowing the creation of instances of

Review comment:
   (referencing the Affinity class and its configuration would likely be 
good enough here)





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] murblanc commented on a change in pull request #2101: SOLR-15016 Replica placement plugins should use container plugins API / configs

2020-12-04 Thread GitBox


murblanc commented on a change in pull request #2101:
URL: https://github.com/apache/lucene-solr/pull/2101#discussion_r536261667



##
File path: 
solr/core/src/java/org/apache/solr/cluster/placement/impl/PlacementPluginFactoryLoader.java
##
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.cluster.placement.impl;
+
+import org.apache.solr.api.ContainerPluginsRegistry;
+import org.apache.solr.client.solrj.request.beans.PluginMeta;
+import org.apache.solr.cluster.placement.PlacementPluginConfig;
+import org.apache.solr.cluster.placement.PlacementPluginFactory;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.lang.invoke.MethodHandles;
+
+/**
+ * Utility class to load the configured {@link PlacementPluginFactory} plugin 
and
+ * then keep it up to date as the plugin configuration changes.
+ */
+public class PlacementPluginFactoryLoader {
+  private static final Logger log = 
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+  public static void load(DelegatingPlacementPluginFactory pluginFactory, 
ContainerPluginsRegistry plugins) {
+ContainerPluginsRegistry.ApiInfo pluginFactoryInfo = 
plugins.getPlugin(PlacementPluginFactory.PLUGIN_NAME);
+if (pluginFactoryInfo != null && (pluginFactoryInfo.getInstance() 
instanceof PlacementPluginFactory)) {
+  pluginFactory.setDelegate((PlacementPluginFactory) pluginFactoryInfo.getInstance());
+}
+ContainerPluginsRegistry.PluginRegistryListener pluginListener = new 
ContainerPluginsRegistry.PluginRegistryListener() {
+  @Override
+  public void added(ContainerPluginsRegistry.ApiInfo plugin) {
+if (plugin == null || plugin.getInstance() == null) {
+  return;
+}
+Object instance = plugin.getInstance();
+if (instance instanceof PlacementPluginFactory) {
+  setDelegate(plugin.getInfo(), instance);

Review comment:
   1/3: maybe we should cast the second parameter `instance` to 
`PlacementPluginFactory` here.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] murblanc commented on a change in pull request #2101: SOLR-15016 Replica placement plugins should use container plugins API / configs

2020-12-04 Thread GitBox


murblanc commented on a change in pull request #2101:
URL: https://github.com/apache/lucene-solr/pull/2101#discussion_r536262072



##
File path: 
solr/core/src/java/org/apache/solr/cluster/placement/impl/PlacementPluginFactoryLoader.java
##
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.cluster.placement.impl;
+
+import org.apache.solr.api.ContainerPluginsRegistry;
+import org.apache.solr.client.solrj.request.beans.PluginMeta;
+import org.apache.solr.cluster.placement.PlacementPluginConfig;
+import org.apache.solr.cluster.placement.PlacementPluginFactory;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.lang.invoke.MethodHandles;
+
+/**
+ * Utility class to load the configured {@link PlacementPluginFactory} plugin 
and
+ * then keep it up to date as the plugin configuration changes.
+ */
+public class PlacementPluginFactoryLoader {
+  private static final Logger log = 
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+  public static void load(DelegatingPlacementPluginFactory pluginFactory, 
ContainerPluginsRegistry plugins) {
+ContainerPluginsRegistry.ApiInfo pluginFactoryInfo = 
plugins.getPlugin(PlacementPluginFactory.PLUGIN_NAME);
+if (pluginFactoryInfo != null && (pluginFactoryInfo.getInstance() 
instanceof PlacementPluginFactory)) {
+  pluginFactory.setDelegate((PlacementPluginFactory) pluginFactoryInfo.getInstance());
+}
+ContainerPluginsRegistry.PluginRegistryListener pluginListener = new 
ContainerPluginsRegistry.PluginRegistryListener() {
+  @Override
+  public void added(ContainerPluginsRegistry.ApiInfo plugin) {
+if (plugin == null || plugin.getInstance() == null) {
+  return;
+}
+Object instance = plugin.getInstance();
+if (instance instanceof PlacementPluginFactory) {
+  setDelegate(plugin.getInfo(), instance);
+}
+  }
+
+  @Override
+  public void deleted(ContainerPluginsRegistry.ApiInfo plugin) {
+if (plugin == null || plugin.getInstance() == null) {
+  return;
+}
+Object instance = plugin.getInstance();
+if (instance instanceof PlacementPluginFactory) {
+  setDelegate(plugin.getInfo(), null);
+}
+  }
+
+  @Override
+  public void modified(ContainerPluginsRegistry.ApiInfo old, 
ContainerPluginsRegistry.ApiInfo replacement) {
+added(replacement);
+  }
+
+  private void setDelegate(PluginMeta pluginMeta, Object instance) {

Review comment:
   2/3: change the signature of this method so that `instance` is a 
`PlacementPluginFactory`.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] murblanc commented on a change in pull request #2101: SOLR-15016 Replica placement plugins should use container plugins API / configs

2020-12-04 Thread GitBox


murblanc commented on a change in pull request #2101:
URL: https://github.com/apache/lucene-solr/pull/2101#discussion_r536262348



##
File path: 
solr/core/src/java/org/apache/solr/cluster/placement/impl/PlacementPluginFactoryLoader.java
##
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.cluster.placement.impl;
+
+import org.apache.solr.api.ContainerPluginsRegistry;
+import org.apache.solr.client.solrj.request.beans.PluginMeta;
+import org.apache.solr.cluster.placement.PlacementPluginConfig;
+import org.apache.solr.cluster.placement.PlacementPluginFactory;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.lang.invoke.MethodHandles;
+
+/**
+ * Utility class to load the configured {@link PlacementPluginFactory} plugin 
and
+ * then keep it up to date as the plugin configuration changes.
+ */
+public class PlacementPluginFactoryLoader {
+  private static final Logger log = 
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+  public static void load(DelegatingPlacementPluginFactory pluginFactory, 
ContainerPluginsRegistry plugins) {
+ContainerPluginsRegistry.ApiInfo pluginFactoryInfo = 
plugins.getPlugin(PlacementPluginFactory.PLUGIN_NAME);
+if (pluginFactoryInfo != null && (pluginFactoryInfo.getInstance() 
instanceof PlacementPluginFactory)) {
+  pluginFactory.setDelegate((PlacementPluginFactory) pluginFactoryInfo.getInstance());
+}
+ContainerPluginsRegistry.PluginRegistryListener pluginListener = new 
ContainerPluginsRegistry.PluginRegistryListener() {
+  @Override
+  public void added(ContainerPluginsRegistry.ApiInfo plugin) {
+if (plugin == null || plugin.getInstance() == null) {
+  return;
+}
+Object instance = plugin.getInstance();
+if (instance instanceof PlacementPluginFactory) {
+  setDelegate(plugin.getInfo(), instance);
+}
+  }
+
+  @Override
+  public void deleted(ContainerPluginsRegistry.ApiInfo plugin) {
+if (plugin == null || plugin.getInstance() == null) {
+  return;
+}
+Object instance = plugin.getInstance();
+if (instance instanceof PlacementPluginFactory) {
+  setDelegate(plugin.getInfo(), null);
+}
+  }
+
+  @Override
+  public void modified(ContainerPluginsRegistry.ApiInfo old, 
ContainerPluginsRegistry.ApiInfo replacement) {
+added(replacement);
+  }
+
+  private void setDelegate(PluginMeta pluginMeta, Object instance) {
+if (PlacementPluginFactory.PLUGIN_NAME.equals(pluginMeta.name)) {
+  pluginFactory.setDelegate((PlacementPluginFactory) instance);

Review comment:
   3/3: remove the cast here, `instance` would be of the expected type.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] murblanc commented on a change in pull request #2101: SOLR-15016 Replica placement plugins should use container plugins API / configs

2020-12-04 Thread GitBox


murblanc commented on a change in pull request #2101:
URL: https://github.com/apache/lucene-solr/pull/2101#discussion_r536270908



##
File path: 
solr/core/src/java/org/apache/solr/cluster/placement/plugins/MinimizeCoresPlacementFactory.java
##
@@ -40,13 +40,23 @@
  *
  * See {@link AffinityPlacementFactory} for a more realistic example and 
documentation.
  */
-public class MinimizeCoresPlacementFactory implements PlacementPluginFactory {
+public class MinimizeCoresPlacementFactory implements 
PlacementPluginFactory {
 
   @Override
-  public PlacementPlugin createPluginInstance(PlacementPluginConfig config) {
+  public PlacementPlugin createPluginInstance() {
 return new MinimizeCoresPlacementPlugin();
   }
 
+  @Override
+  public void configure(NoConfig cfg) {

Review comment:
   Maybe `configure()` in `ConfigurablePlugin` should 
have a default empty implementation so we don't have to repeat it (here, in 
`DelegatingPlacementPluginFactory` and `RandomPlacementFactory`).





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] murblanc commented on a change in pull request #2101: SOLR-15016 Replica placement plugins should use container plugins API / configs

2020-12-04 Thread GitBox


murblanc commented on a change in pull request #2101:
URL: https://github.com/apache/lucene-solr/pull/2101#discussion_r536274851



##
File path: 
solr/core/src/java/org/apache/solr/cluster/placement/plugins/MinimizeCoresPlacementFactory.java
##
@@ -40,13 +40,23 @@
  *
  * See {@link AffinityPlacementFactory} for a more realistic example and 
documentation.
  */
-public class MinimizeCoresPlacementFactory implements PlacementPluginFactory {
+public class MinimizeCoresPlacementFactory implements 
PlacementPluginFactory {
 
   @Override
-  public PlacementPlugin createPluginInstance(PlacementPluginConfig config) {
+  public PlacementPlugin createPluginInstance() {
 return new MinimizeCoresPlacementPlugin();
   }
 
+  @Override
+  public void configure(NoConfig cfg) {
+// no-op
+  }
+
+  @Override
+  public NoConfig getConfig() {

Review comment:
   Maybe similarly here, `PlacementPluginFactory` should provide a default 
implementation returning `NoConfig.INSTANCE` given that out of 4 
implementations of the interface, only one returns something else 
(`AffinityPlacementFactory`).





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] murblanc commented on a change in pull request #2101: SOLR-15016 Replica placement plugins should use container plugins API / configs

2020-12-04 Thread GitBox


murblanc commented on a change in pull request #2101:
URL: https://github.com/apache/lucene-solr/pull/2101#discussion_r536287423



##
File path: 
solr/core/src/java/org/apache/solr/cluster/placement/plugins/RandomPlacementFactory.java
##
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.cluster.placement.plugins;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Random;
+import java.util.Set;
+
+import com.google.common.annotations.VisibleForTesting;
+import org.apache.solr.cluster.Cluster;
+import org.apache.solr.cluster.Node;
+import org.apache.solr.cluster.Replica;
+import org.apache.solr.cluster.SolrCollection;
+import org.apache.solr.cluster.placement.*;
+
+/**
+ * Factory for creating {@link RandomPlacementPlugin}, a placement plugin 
implementing random placement for new
+ * collection creation while preventing two replicas of same shard from being 
placed on same node..
+ *
+ * See {@link AffinityPlacementFactory} for a more realistic example and 
documentation.
+ */
+public class RandomPlacementFactory implements PlacementPluginFactory {
+
+  @Override
+  public PlacementPlugin createPluginInstance() {
+return new RandomPlacementPlugin();
+  }
+
+  public static class RandomPlacementPlugin implements PlacementPlugin {
+private Random random = new Random();

Review comment:
   @TomMD is there a way to annotate the code when we know/decide this is 
not an issue to silence muse-dev? (@madrob suggested to ask you).





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] saatchibhalla commented on pull request #2040: SOLR-14965 add overseer queue size metrics

2020-12-04 Thread GitBox


saatchibhalla commented on pull request #2040:
URL: https://github.com/apache/lucene-solr/pull/2040#issuecomment-738939855


   Thanks @dsmiley! Looks good to me.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] murblanc commented on pull request #2101: SOLR-15016 Replica placement plugins should use container plugins API / configs

2020-12-04 Thread GitBox


murblanc commented on pull request #2101:
URL: https://github.com/apache/lucene-solr/pull/2101#issuecomment-738942564


   Looks good to me!



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Created] (LUCENE-9632) Replace our TimSort with JDK TimSort

2020-12-04 Thread Mike Drob (Jira)
Mike Drob created LUCENE-9632:
-

 Summary: Replace our TimSort with JDK TimSort
 Key: LUCENE-9632
 URL: https://issues.apache.org/jira/browse/LUCENE-9632
 Project: Lucene - Core
  Issue Type: Improvement
Affects Versions: master (9.0)
Reporter: Mike Drob


JDK 11 has a TimSort with very similar lineage to our TimSort (written by Josh 
Bloch, based on Tim Peter's python list sort), we should consider replacing our 
fork with a version that we don't have to maintain.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (LUCENE-9632) Replace our TimSort with JDK TimSort

2020-12-04 Thread Mike Drob (Jira)


[ 
https://issues.apache.org/jira/browse/LUCENE-9632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244199#comment-17244199
 ] 

Mike Drob commented on LUCENE-9632:
---

[~uschindler] - is there some historical subtlety that I am not aware of that 
should prevent us from doing this?

> Replace our TimSort with JDK TimSort
> 
>
> Key: LUCENE-9632
> URL: https://issues.apache.org/jira/browse/LUCENE-9632
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: master (9.0)
>Reporter: Mike Drob
>Priority: Major
>
> JDK 11 has a TimSort with very similar lineage to our TimSort (written by 
> Josh Bloch, based on Tim Peter's python list sort), we should consider 
> replacing our fork with a version that we don't have to maintain.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Resolved] (LUCENE-9599) Disable sort optimization in comparators on index sort

2020-12-04 Thread Mayya Sharipova (Jira)


 [ 
https://issues.apache.org/jira/browse/LUCENE-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mayya Sharipova resolved LUCENE-9599.
-
Fix Version/s: 8.8
   Resolution: Fixed

> Disable sort optimization in comparators on index sort
> --
>
> Key: LUCENE-9599
> URL: https://issues.apache.org/jira/browse/LUCENE-9599
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Mayya Sharipova
>Priority: Minor
> Fix For: 8.8
>
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> LUCENE-9280 introduced an ability for comparators to skip non-competitive 
> documents. But currently comparators are not aware of index sorting, and can 
> run sort optimization even when search sort is congruent with the index sort.
> As the early termination in this case is already handled in 
> TopFieldCollector, we need to disable sort optimization in comparators. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] sigram commented on a change in pull request #2101: SOLR-15016 Replica placement plugins should use container plugins API / configs

2020-12-04 Thread GitBox


sigram commented on a change in pull request #2101:
URL: https://github.com/apache/lucene-solr/pull/2101#discussion_r536353469



##
File path: 
solr/core/src/java/org/apache/solr/cluster/placement/PlacementPluginFactory.java
##
@@ -17,15 +17,39 @@
 
 package org.apache.solr.cluster.placement;
 
+import org.apache.solr.api.ConfigurablePlugin;
+
 /**
- * Factory implemented by client code and configured in {@code solr.xml} 
allowing the creation of instances of
+ * Factory implemented by client code and configured in container plugins 
allowing the creation of instances of

Review comment:
   I'll add a reference to `ContainerPluginsApi#Edit` - JSON formats and 
the location of the config in `/clusterprops.json` are an implementation 
detail. Also, container plugin configuration is/will be described in the 
RefGuide (SOLR-15022).





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14182) Move metric reporters config from solr.xml to ZK cluster properties

2020-12-04 Thread David Smiley (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244271#comment-17244271
 ] 

David Smiley commented on SOLR-14182:
-

+1 to everything Tomas said.

> Move metric reporters config from solr.xml to ZK cluster properties
> ---
>
> Key: SOLR-14182
> URL: https://issues.apache.org/jira/browse/SOLR-14182
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 8.4
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
>
> Metric reporters are currently configured statically in solr.xml, which makes 
> it difficult to change dynamically or in a containerized environment.
> We should move this section to ZK /cluster.properties and add a back-compat 
> migration shim.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] sigram commented on a change in pull request #2101: SOLR-15016 Replica placement plugins should use container plugins API / configs

2020-12-04 Thread GitBox


sigram commented on a change in pull request #2101:
URL: https://github.com/apache/lucene-solr/pull/2101#discussion_r536361329



##
File path: 
solr/core/src/java/org/apache/solr/cluster/placement/plugins/MinimizeCoresPlacementFactory.java
##
@@ -40,13 +40,23 @@
  *
  * See {@link AffinityPlacementFactory} for a more realistic example and 
documentation.
  */
-public class MinimizeCoresPlacementFactory implements PlacementPluginFactory {
+public class MinimizeCoresPlacementFactory implements 
PlacementPluginFactory {
 
   @Override
-  public PlacementPlugin createPluginInstance(PlacementPluginConfig config) {
+  public PlacementPlugin createPluginInstance() {
 return new MinimizeCoresPlacementPlugin();
   }
 
+  @Override
+  public void configure(NoConfig cfg) {
+// no-op
+  }
+
+  @Override
+  public NoConfig getConfig() {

Review comment:
   I think it needs to return null here, I don't see any other way to 
return an instance that is assignable without ClassCastException to any ``.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14965) Adding metrics to track size of Overseer queues

2020-12-04 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244299#comment-17244299
 ] 

ASF subversion and git services commented on SOLR-14965:


Commit 19ed90337767412de66ebb1757e63607372ae9b3 in lucene-solr's branch 
refs/heads/master from saatchibhalla
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=19ed903 ]

SOLR-14965: add overseer queue size metrics (#2040)

Adds two metrics to the SolrCloud Overseer: 
solr_metrics_overseer_stateUpdateQueueSize and 
solr_metrics_overseer_collectionWorkQueueSize with corresponding entries in the 
the Prometheus exporter's default/stock configuration.

Co-authored-by: Saatchi Bhalla 

> Adding metrics to track size of Overseer queues
> ---
>
> Key: SOLR-14965
> URL: https://issues.apache.org/jira/browse/SOLR-14965
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling, SolrCloud
>Reporter: Saatchi Bhalla
>Priority: Minor
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> The Overseer work queues stored in ZK and abstracted by the Overseer can give 
> us a good indication of the health of the cluster - if messages are taking 
> too long to dequeue or the queue is growing too large, we know that the 
> Overseer is overloaded and we are going to overwhelm the cluster.
>  
> This work will add metrics to track the size of the Overseer queues (number 
> of messages enqueued). 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] dsmiley merged pull request #2040: SOLR-14965 add overseer queue size metrics

2020-12-04 Thread GitBox


dsmiley merged pull request #2040:
URL: https://github.com/apache/lucene-solr/pull/2040


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (LUCENE-9599) Disable sort optimization in comparators on index sort

2020-12-04 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/LUCENE-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244321#comment-17244321
 ] 

ASF subversion and git services commented on LUCENE-9599:
-

Commit b73e87aa9e930879abe989a24867b2012db56d21 in lucene-solr's branch 
refs/heads/branch_8x from David Smiley
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=b73e87a ]

LUCENE-9599: fix compilation issue in Java 8


> Disable sort optimization in comparators on index sort
> --
>
> Key: LUCENE-9599
> URL: https://issues.apache.org/jira/browse/LUCENE-9599
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Mayya Sharipova
>Priority: Minor
> Fix For: 8.8
>
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> LUCENE-9280 introduced an ability for comparators to skip non-competitive 
> documents. But currently comparators are not aware of index sorting, and can 
> run sort optimization even when search sort is congruent with the index sort.
> As the early termination in this case is already handled in 
> TopFieldCollector, we need to disable sort optimization in comparators. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14965) Adding metrics to track size of Overseer queues

2020-12-04 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244335#comment-17244335
 ] 

ASF subversion and git services commented on SOLR-14965:


Commit 7731cd6c1213964f49da9bd0b7dbeb5567042e80 in lucene-solr's branch 
refs/heads/branch_8x from saatchibhalla
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=7731cd6 ]

SOLR-14965: add overseer queue size metrics (#2040)

Adds two metrics to the SolrCloud Overseer: 
solr_metrics_overseer_stateUpdateQueueSize and 
solr_metrics_overseer_collectionWorkQueueSize with corresponding entries in the 
the Prometheus exporter's default/stock configuration.

Co-authored-by: Saatchi Bhalla 

(cherry picked from commit 19ed90337767412de66ebb1757e63607372ae9b3)


> Adding metrics to track size of Overseer queues
> ---
>
> Key: SOLR-14965
> URL: https://issues.apache.org/jira/browse/SOLR-14965
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling, SolrCloud
>Reporter: Saatchi Bhalla
>Priority: Minor
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> The Overseer work queues stored in ZK and abstracted by the Overseer can give 
> us a good indication of the health of the cluster - if messages are taking 
> too long to dequeue or the queue is growing too large, we know that the 
> Overseer is overloaded and we are going to overwhelm the cluster.
>  
> This work will add metrics to track the size of the Overseer queues (number 
> of messages enqueued). 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-14965) Adding metrics to track size of Overseer queues

2020-12-04 Thread David Smiley (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-14965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated SOLR-14965:

Fix Version/s: 8.8
 Assignee: David Smiley
   Resolution: Fixed
   Status: Resolved  (was: Patch Available)

Thanks for contributing Saatchi!

> Adding metrics to track size of Overseer queues
> ---
>
> Key: SOLR-14965
> URL: https://issues.apache.org/jira/browse/SOLR-14965
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling, SolrCloud
>Reporter: Saatchi Bhalla
>Assignee: David Smiley
>Priority: Minor
> Fix For: 8.8
>
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> The Overseer work queues stored in ZK and abstracted by the Overseer can give 
> us a good indication of the health of the cluster - if messages are taking 
> too long to dequeue or the queue is growing too large, we know that the 
> Overseer is overloaded and we are going to overwhelm the cluster.
>  
> This work will add metrics to track the size of the Overseer queues (number 
> of messages enqueued). 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] madrob opened a new pull request #2120: SOLR-15029 More gracefully give up shard leadership

2020-12-04 Thread GitBox


madrob opened a new pull request #2120:
URL: https://github.com/apache/lucene-solr/pull/2120


   https://issues.apache.org/jira/browse/SOLR-15029



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-15029) Allow Shard Leader to give up leadership gracefully via shard terms

2020-12-04 Thread Mike Drob (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-15029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244347#comment-17244347
 ] 

Mike Drob commented on SOLR-15029:
--

Put up a draft PR for how I think this could work, it needs more testing but 
it's taken a lot of digging to get the sequence right in my head and I wanted 
to post something before the weekend and I lose all my context again.

> Allow Shard Leader to give up leadership gracefully via shard terms
> ---
>
> Key: SOLR-15029
> URL: https://issues.apache.org/jira/browse/SOLR-15029
> Project: Solr
>  Issue Type: Improvement
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currently we have (via SOLR-12412) that when a leader sees an index writing 
> error during an update it will give up leadership by deleting the replica and 
> adding a new replica. One stated benefit of this was that because we are 
> using the overseer and a known code path, that this is done asynchronous and 
> very efficiently.
> I would argue that this approach is too heavy handed.
> In the case of a corrupt index exception, it makes some sense to completely 
> delete the index dir and attempt to sync from a good peer. Even in this case, 
> however, it might be better to allow fingerprinting and other index delta 
> mechanisms take over and allow for a more efficient data transfer.
> In an alternate case where the index error arises due to a disconnected file 
> system (possible with shared file systems, i.e. S3, HDFS, some k8s systems) 
> and the required solution is some kind of reconnect, then this approach has 
> several shortcomings - the core delete and creations are going to fail 
> leaving dangling replicas. Further, the data is still present so there is no 
> need to do so many extra copies.
> I propose that we bring in a mechanism to give up leadership via the existing 
> shard terms language. I believe we would be able to set all replicas 
> currently equal to leader term T to T+1, and then trigger a new leader 
> election. The current leader would know it is ineligible, while the other 
> replicas that were current before the failed update would be eligible. This 
> improvement would entail adding an additional possible operation to terms 
> state machine.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (LUCENE-9632) Replace our TimSort with JDK TimSort

2020-12-04 Thread Uwe Schindler (Jira)


[ 
https://issues.apache.org/jira/browse/LUCENE-9632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244359#comment-17244359
 ] 

Uwe Schindler commented on LUCENE-9632:
---

Not sure why you ask me, but the Tomorrow implementation is not only used for 
standard sorting of lists and arrays. Lucene has its own sorting infrastructure 
to allow sorting multiple arrays and lists in parallel (the first array is 
sorted and the parallel ones are getting same element swaps like the first 
one). With plain Java apis that's not possible.

In addition: Arrays/Collections.sort() makes clones of the objects instead of 
sorting in place.

> Replace our TimSort with JDK TimSort
> 
>
> Key: LUCENE-9632
> URL: https://issues.apache.org/jira/browse/LUCENE-9632
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: master (9.0)
>Reporter: Mike Drob
>Priority: Major
>
> JDK 11 has a TimSort with very similar lineage to our TimSort (written by 
> Josh Bloch, based on Tim Peter's python list sort), we should consider 
> replacing our fork with a version that we don't have to maintain.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Comment Edited] (LUCENE-9632) Replace our TimSort with JDK TimSort

2020-12-04 Thread Uwe Schindler (Jira)


[ 
https://issues.apache.org/jira/browse/LUCENE-9632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244359#comment-17244359
 ] 

Uwe Schindler edited comment on LUCENE-9632 at 12/4/20, 11:52 PM:
--

Not sure why you ask me, but the Timsort implementation is not only used for 
standard sorting of lists and arrays. Lucene has its own sorting infrastructure 
to allow sorting multiple arrays and lists in parallel (the first array is 
sorted and the parallel ones are getting same element swaps like the first 
one). With plain Java apis that's not possible.

In addition: Arrays/Collections.sort() makes clones of the objects instead of 
sorting in place.


was (Author: thetaphi):
Not sure why you ask me, but the Tomorrow implementation is not only used for 
standard sorting of lists and arrays. Lucene has its own sorting infrastructure 
to allow sorting multiple arrays and lists in parallel (the first array is 
sorted and the parallel ones are getting same element swaps like the first 
one). With plain Java apis that's not possible.

In addition: Arrays/Collections.sort() makes clones of the objects instead of 
sorting in place.

> Replace our TimSort with JDK TimSort
> 
>
> Key: LUCENE-9632
> URL: https://issues.apache.org/jira/browse/LUCENE-9632
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: master (9.0)
>Reporter: Mike Drob
>Priority: Major
>
> JDK 11 has a TimSort with very similar lineage to our TimSort (written by 
> Josh Bloch, based on Tim Peter's python list sort), we should consider 
> replacing our fork with a version that we don't have to maintain.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] chatman commented on pull request #1963: SOLR-14827: Refactor schema loading to not use XPath

2020-12-04 Thread GitBox


chatman commented on pull request #1963:
URL: https://github.com/apache/lucene-solr/pull/1963#issuecomment-739124443


   Updated to latest master.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] madrob commented on a change in pull request #2120: SOLR-15029 More gracefully give up shard leadership

2020-12-04 Thread GitBox


madrob commented on a change in pull request #2120:
URL: https://github.com/apache/lucene-solr/pull/2120#discussion_r53651



##
File path: solr/core/src/java/org/apache/solr/cloud/ZkShardTerms.java
##
@@ -315,7 +312,8 @@ private boolean saveTerms(ShardTerms newTerms) throws 
KeeperException.NoNodeExce
   refreshTerms();
 } catch (KeeperException.NoNodeException e) {
   throw e;
-} catch (Exception e) {
+} catch (KeeperException | InterruptedException e) {

Review comment:
   this no longer catches/wraps runtime exception





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Created] (SOLR-15033) 8x Smoke test fails missing clustering jar

2020-12-04 Thread Mike Drob (Jira)
Mike Drob created SOLR-15033:


 Summary: 8x Smoke test fails missing clustering jar
 Key: SOLR-15033
 URL: https://issues.apache.org/jira/browse/SOLR-15033
 Project: Solr
  Issue Type: Task
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Mike Drob


[smoker] RuntimeError: Missing artifact 
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-SmokeRelease-8.x/lucene/build/smokeTestRelease/tmp/maven/org/apache/solr/solr-clustering/8.8.0/solr-clustering-8.8.0.jar

I think this was removed? Build needs to be caught up. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] munendrasn opened a new pull request #2121: SOLR-10860: Return proper error code for bad input incase of inplace updates

2020-12-04 Thread GitBox


munendrasn opened a new pull request #2121:
URL: https://github.com/apache/lucene-solr/pull/2121


   Additionally,
   * Return proper error when inc operation is specified for non-numeric fields
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-10860) in-place updates currently throw NumberFormatException instead of a Bad Request SolrException for bad input

2020-12-04 Thread Munendra S N (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-10860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244410#comment-17244410
 ] 

Munendra S N commented on SOLR-10860:
-

I have rebased the patch against latest upstream and created Github PR

> in-place updates currently throw NumberFormatException instead of a Bad 
> Request SolrException for bad input
> ---
>
> Key: SOLR-10860
> URL: https://issues.apache.org/jira/browse/SOLR-10860
> Project: Solr
>  Issue Type: Bug
>Reporter: Tomas Eduardo Fernandez Lobbe
>Assignee: Ishan Chattopadhyaya
>Priority: Major
> Attachments: SOLR-10860.patch, SOLR-10860.patch, SOLR-10860.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Assigned] (SOLR-10860) in-place updates currently throw NumberFormatException instead of a Bad Request SolrException for bad input

2020-12-04 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-10860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N reassigned SOLR-10860:
---

Assignee: Munendra S N  (was: Ishan Chattopadhyaya)

> in-place updates currently throw NumberFormatException instead of a Bad 
> Request SolrException for bad input
> ---
>
> Key: SOLR-10860
> URL: https://issues.apache.org/jira/browse/SOLR-10860
> Project: Solr
>  Issue Type: Bug
>Reporter: Tomas Eduardo Fernandez Lobbe
>Assignee: Munendra S N
>Priority: Major
> Attachments: SOLR-10860.patch, SOLR-10860.patch, SOLR-10860.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Assigned] (SOLR-14950) schema api sometimes fails to replace-field-type

2020-12-04 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-14950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N reassigned SOLR-14950:
---

Assignee: Munendra S N

> schema api sometimes fails to replace-field-type
> 
>
> Key: SOLR-14950
> URL: https://issues.apache.org/jira/browse/SOLR-14950
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>Affects Versions: 8.4.1
>Reporter: Andrew Shumway
>Assignee: Munendra S N
>Priority: Minor
>  Labels: schema
>
> When updating a schema via POST to  {{/solr/COLLECTION/schema}} using  
> {{replace-field-type}} , the call fails if the schema has existing 
> copy-fields whose destination references a non-dynamic field of the  
> {{field-type}} which is being updated.
>  
> Example steps: # on a blank schema, add a field type and copy-field by POSTing
> {
>   "add-field-type": [
> {
>   "name": "spellcheck", "class": "solr.TextField","positionIncrementGap": 
> "100",
>   "analyzer": {
> "tokenizer": {
>   "class": "solr.StandardTokenizerFactory"
> }
>   }
> }
>   ],
>   "add-field": [
> {
>   "name": "spellcheck",  "type": "spellcheck",
>   "multiValued": true, "indexed": true, "stored": true
> }
>   ],
>   "add-copy-field": [
> {
>   "source": "important_data_s",
>   "dest": "spellcheck"
> }
>   ]
> }
> 2. Later, send an update for the field type
> {
>   "replace-field-type": [
> {
>   "name": "spellcheck", "class": "solr.TextField","positionIncrementGap": 
> "100",
>   "analyzer": {
> "tokenizer": {
>   "class": "solr.StandardTokenizerFactory"
> }
>   }
> }
>   ]
> }
> This results in {{"errorMessages":["Source or Destination SchemaField can't 
> be NULL.\n"]}],}}The error does not trigger if, as mentioned above, the 
> destination of the existing copy-field is dynamic OR if the source field 
> mapping contains a wildcard e.g. {{*foo_s}} (edited) 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] munendrasn opened a new pull request #2122: SOLR-14950: Fix copyfield regeneration with explicit src/dest matching dyn rule

2020-12-04 Thread GitBox


munendrasn opened a new pull request #2122:
URL: https://github.com/apache/lucene-solr/pull/2122


   CopyFields are regenerated in case of replace-field or replace-field-type.
   While regenerating, source and destination are checked against fields but 
source/dest
   could match dynamic rule too.
   For example,
   
   
   here, something_s is not present in schema but matches the dynamic rule.
   
   To handle the above case, need to check dynamicFieldCache too while 
regenerating the
   copyFields
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14950) schema api sometimes fails to replace-field-type

2020-12-04 Thread Munendra S N (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244412#comment-17244412
 ] 

Munendra S N commented on SOLR-14950:
-

During regeneration of copyFields, only fields map is used to fetch SchemaField 
but in the give copyField, source might match dynamic rule.
[~aeshumway] could you please the above PR and let me know if that fixes your 
issue?

> schema api sometimes fails to replace-field-type
> 
>
> Key: SOLR-14950
> URL: https://issues.apache.org/jira/browse/SOLR-14950
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>Affects Versions: 8.4.1
>Reporter: Andrew Shumway
>Assignee: Munendra S N
>Priority: Minor
>  Labels: schema
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When updating a schema via POST to  {{/solr/COLLECTION/schema}} using  
> {{replace-field-type}} , the call fails if the schema has existing 
> copy-fields whose destination references a non-dynamic field of the  
> {{field-type}} which is being updated.
>  
> Example steps: # on a blank schema, add a field type and copy-field by POSTing
> {
>   "add-field-type": [
> {
>   "name": "spellcheck", "class": "solr.TextField","positionIncrementGap": 
> "100",
>   "analyzer": {
> "tokenizer": {
>   "class": "solr.StandardTokenizerFactory"
> }
>   }
> }
>   ],
>   "add-field": [
> {
>   "name": "spellcheck",  "type": "spellcheck",
>   "multiValued": true, "indexed": true, "stored": true
> }
>   ],
>   "add-copy-field": [
> {
>   "source": "important_data_s",
>   "dest": "spellcheck"
> }
>   ]
> }
> 2. Later, send an update for the field type
> {
>   "replace-field-type": [
> {
>   "name": "spellcheck", "class": "solr.TextField","positionIncrementGap": 
> "100",
>   "analyzer": {
> "tokenizer": {
>   "class": "solr.StandardTokenizerFactory"
> }
>   }
> }
>   ]
> }
> This results in {{"errorMessages":["Source or Destination SchemaField can't 
> be NULL.\n"]}],}}The error does not trigger if, as mentioned above, the 
> destination of the existing copy-field is dynamic OR if the source field 
> mapping contains a wildcard e.g. {{*foo_s}} (edited) 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Assigned] (SOLR-14514) json.facets: method:stream is incompatible with allBuckets:true

2020-12-04 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-14514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N reassigned SOLR-14514:
---

Assignee: Munendra S N

> json.facets: method:stream is incompatible with allBuckets:true
> ---
>
> Key: SOLR-14514
> URL: https://issues.apache.org/jira/browse/SOLR-14514
> Project: Solr
>  Issue Type: Bug
>  Components: Facet Module
>Reporter: Chris M. Hostetter
>Assignee: Munendra S N
>Priority: Minor
>
> {{FacetFieldProcessorByEnumTermsStream}} has never supported 
> {{allBuckets:true}} but it also doesn't fail outright if {{allBuckets:true}} 
> is specified -- instead the bucket is silently missing from the response.
> Given how the {{method}} option is only used as a suggestion, and the actuall 
> processor can change based on hueristics about the request, this means that 
> the behavior of combining {{allBuckets:true}} with {{method:stream}} can vary 
> depending on the other options specified -- notably the {{sort}}
> {noformat}
> % curl -sS -X POST http://localhost:8983/solr/techproducts/query -d 
> 'omitHeader=true&rows=0&q=*:*&json.facet={
>   x : {
> type : terms,
> method: stream,
> field : manu_id_s,
> allBuckets : true,
> limit : 2,
> } }'
> 
> {
>   "response":{"numFound":32,"start":0,"docs":[]
>   },
>   "facets":{
> "count":32,
> "x":{
>   "allBuckets":{
> "count":18},
>   "buckets":[{
>   "val":"corsair",
>   "count":3},
> {
>   "val":"belkin",
>   "count":2}]}}}
> % curl -sS -X POST http://localhost:8983/solr/techproducts/query -d 
> 'omitHeader=true&rows=0&q=*:*&json.facet={
>   x : {
> type : terms,
> method: stream,
> field : manu_id_s,
> allBuckets : true,
> limit : 2,
> sort: "index asc"
> } }'
> {
>   "response":{"numFound":32,"start":0,"docs":[]
>   },
>   "facets":{
> "count":32,
> "x":{
>   "buckets":[{
>   "val":"apple",
>   "count":1},
> {
>   "val":"asus",
>   "count":1}]}}}
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-14514) json.facets: method:stream is incompatible with allBuckets:true

2020-12-04 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-14514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N updated SOLR-14514:

Attachment: SOLR-14514.patch

> json.facets: method:stream is incompatible with allBuckets:true
> ---
>
> Key: SOLR-14514
> URL: https://issues.apache.org/jira/browse/SOLR-14514
> Project: Solr
>  Issue Type: Bug
>  Components: Facet Module
>Reporter: Chris M. Hostetter
>Assignee: Munendra S N
>Priority: Minor
> Attachments: SOLR-14514.patch
>
>
> {{FacetFieldProcessorByEnumTermsStream}} has never supported 
> {{allBuckets:true}} but it also doesn't fail outright if {{allBuckets:true}} 
> is specified -- instead the bucket is silently missing from the response.
> Given how the {{method}} option is only used as a suggestion, and the actuall 
> processor can change based on hueristics about the request, this means that 
> the behavior of combining {{allBuckets:true}} with {{method:stream}} can vary 
> depending on the other options specified -- notably the {{sort}}
> {noformat}
> % curl -sS -X POST http://localhost:8983/solr/techproducts/query -d 
> 'omitHeader=true&rows=0&q=*:*&json.facet={
>   x : {
> type : terms,
> method: stream,
> field : manu_id_s,
> allBuckets : true,
> limit : 2,
> } }'
> 
> {
>   "response":{"numFound":32,"start":0,"docs":[]
>   },
>   "facets":{
> "count":32,
> "x":{
>   "allBuckets":{
> "count":18},
>   "buckets":[{
>   "val":"corsair",
>   "count":3},
> {
>   "val":"belkin",
>   "count":2}]}}}
> % curl -sS -X POST http://localhost:8983/solr/techproducts/query -d 
> 'omitHeader=true&rows=0&q=*:*&json.facet={
>   x : {
> type : terms,
> method: stream,
> field : manu_id_s,
> allBuckets : true,
> limit : 2,
> sort: "index asc"
> } }'
> {
>   "response":{"numFound":32,"start":0,"docs":[]
>   },
>   "facets":{
> "count":32,
> "x":{
>   "buckets":[{
>   "val":"apple",
>   "count":1},
> {
>   "val":"asus",
>   "count":1}]}}}
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14514) json.facets: method:stream is incompatible with allBuckets:true

2020-12-04 Thread Munendra S N (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244413#comment-17244413
 ] 

Munendra S N commented on SOLR-14514:
-

 [^SOLR-14514.patch] 
In the current patch, FacetParser avoids picking stream/enum processor when any 
one of allBuckets, numBuckets, and missing is true. I have updated the 
documentation accordingly

> json.facets: method:stream is incompatible with allBuckets:true
> ---
>
> Key: SOLR-14514
> URL: https://issues.apache.org/jira/browse/SOLR-14514
> Project: Solr
>  Issue Type: Bug
>  Components: Facet Module
>Reporter: Chris M. Hostetter
>Assignee: Munendra S N
>Priority: Minor
> Attachments: SOLR-14514.patch
>
>
> {{FacetFieldProcessorByEnumTermsStream}} has never supported 
> {{allBuckets:true}} but it also doesn't fail outright if {{allBuckets:true}} 
> is specified -- instead the bucket is silently missing from the response.
> Given how the {{method}} option is only used as a suggestion, and the actuall 
> processor can change based on hueristics about the request, this means that 
> the behavior of combining {{allBuckets:true}} with {{method:stream}} can vary 
> depending on the other options specified -- notably the {{sort}}
> {noformat}
> % curl -sS -X POST http://localhost:8983/solr/techproducts/query -d 
> 'omitHeader=true&rows=0&q=*:*&json.facet={
>   x : {
> type : terms,
> method: stream,
> field : manu_id_s,
> allBuckets : true,
> limit : 2,
> } }'
> 
> {
>   "response":{"numFound":32,"start":0,"docs":[]
>   },
>   "facets":{
> "count":32,
> "x":{
>   "allBuckets":{
> "count":18},
>   "buckets":[{
>   "val":"corsair",
>   "count":3},
> {
>   "val":"belkin",
>   "count":2}]}}}
> % curl -sS -X POST http://localhost:8983/solr/techproducts/query -d 
> 'omitHeader=true&rows=0&q=*:*&json.facet={
>   x : {
> type : terms,
> method: stream,
> field : manu_id_s,
> allBuckets : true,
> limit : 2,
> sort: "index asc"
> } }'
> {
>   "response":{"numFound":32,"start":0,"docs":[]
>   },
>   "facets":{
> "count":32,
> "x":{
>   "buckets":[{
>   "val":"apple",
>   "count":1},
> {
>   "val":"asus",
>   "count":1}]}}}
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-12539) JSON Facet shorthand list syntax (Comma seperated list) doesn't trim leading/trailing spaces -- affects range "other" and "include" options (probably more)

2020-12-04 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-12539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N updated SOLR-12539:

Attachment: SOLR-12539.patch

> JSON Facet shorthand list syntax (Comma seperated list) doesn't trim 
> leading/trailing spaces -- affects range "other" and "include" options 
> (probably more)
> ---
>
> Key: SOLR-12539
> URL: https://issues.apache.org/jira/browse/SOLR-12539
> Project: Solr
>  Issue Type: Bug
>  Components: Facet Module
>Reporter: Chris M. Hostetter
>Assignee: Munendra S N
>Priority: Major
> Attachments: SOLR-12539.patch
>
>
> when doing a {{type: range}} JSON Facet request, if you use the supported 
> "comma seperated string" syntax instead of a true JSON List for specifing 
> options like {{other}} or {{include}} then any whitepsace in your string is 
> left in and breaks the parsin of those options.
> This probably affects other features of JSON Faceting that also support this 
> comma seperated string shorthand (tag exclusions?)
> 
> This works...
> {code}
> bin/solr -e techproducts
> ...
> $ curl http://localhost:8983/solr/techproducts/query -d 
> 'q=*:*&rows=0&omitHeader=true&json.facet=
> {x:{type:range,
> field:price,
> start:0,
> end:100,
> gap:20,
> other:"before,after"}}'
> {
>   "response":{"numFound":32,"start":0,"docs":[]
>   },
>   "facets":{
> "count":32,
> "x":{
>   "buckets":[{
>   "val":0.0,
>   "count":5},
> ...
>   "before":{
> "count":0},
>   "after":{
> "count":9
> {code}
> This doesn't (note the subtle amount of whitespace in the error message)...
> {code}
> $ curl http://localhost:8983/solr/techproducts/query -d 
> 'q=*:*&rows=0&omitHeader=true&json.facet=
> {x:{type:range,
> field:price,
> start:0,
> end:100,
> gap:20,
> other:"before, after"}}'
> {
>   "error":{
> "metadata":[
>   "error-class","org.apache.solr.common.SolrException",
>   "root-error-class","java.lang.IllegalArgumentException"],
> "msg":" after is not a valid type of 'other' range facet information",
> "code":400}}
> {code}
> ...exagerated...
> {code}
> $ curl http://localhost:8983/solr/techproducts/query -d 
> 'q=*:*&rows=0&omitHeader=true&json.facet=
> {x:{type:range,
> field:price,
> start:0,
> end:100,
> gap:20,
> other:"before,   after"}}'
> {
>   "error":{
> "metadata":[
>   "error-class","org.apache.solr.common.SolrException",
>   "root-error-class","java.lang.IllegalArgumentException"],
> "msg":"   after is not a valid type of 'other' 
> range facet information",
> "code":400}}
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Assigned] (SOLR-12539) JSON Facet shorthand list syntax (Comma seperated list) doesn't trim leading/trailing spaces -- affects range "other" and "include" options (probably more)

2020-12-04 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-12539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N reassigned SOLR-12539:
---

Assignee: Munendra S N

> JSON Facet shorthand list syntax (Comma seperated list) doesn't trim 
> leading/trailing spaces -- affects range "other" and "include" options 
> (probably more)
> ---
>
> Key: SOLR-12539
> URL: https://issues.apache.org/jira/browse/SOLR-12539
> Project: Solr
>  Issue Type: Bug
>  Components: Facet Module
>Reporter: Chris M. Hostetter
>Assignee: Munendra S N
>Priority: Major
> Attachments: SOLR-12539.patch
>
>
> when doing a {{type: range}} JSON Facet request, if you use the supported 
> "comma seperated string" syntax instead of a true JSON List for specifing 
> options like {{other}} or {{include}} then any whitepsace in your string is 
> left in and breaks the parsin of those options.
> This probably affects other features of JSON Faceting that also support this 
> comma seperated string shorthand (tag exclusions?)
> 
> This works...
> {code}
> bin/solr -e techproducts
> ...
> $ curl http://localhost:8983/solr/techproducts/query -d 
> 'q=*:*&rows=0&omitHeader=true&json.facet=
> {x:{type:range,
> field:price,
> start:0,
> end:100,
> gap:20,
> other:"before,after"}}'
> {
>   "response":{"numFound":32,"start":0,"docs":[]
>   },
>   "facets":{
> "count":32,
> "x":{
>   "buckets":[{
>   "val":0.0,
>   "count":5},
> ...
>   "before":{
> "count":0},
>   "after":{
> "count":9
> {code}
> This doesn't (note the subtle amount of whitespace in the error message)...
> {code}
> $ curl http://localhost:8983/solr/techproducts/query -d 
> 'q=*:*&rows=0&omitHeader=true&json.facet=
> {x:{type:range,
> field:price,
> start:0,
> end:100,
> gap:20,
> other:"before, after"}}'
> {
>   "error":{
> "metadata":[
>   "error-class","org.apache.solr.common.SolrException",
>   "root-error-class","java.lang.IllegalArgumentException"],
> "msg":" after is not a valid type of 'other' range facet information",
> "code":400}}
> {code}
> ...exagerated...
> {code}
> $ curl http://localhost:8983/solr/techproducts/query -d 
> 'q=*:*&rows=0&omitHeader=true&json.facet=
> {x:{type:range,
> field:price,
> start:0,
> end:100,
> gap:20,
> other:"before,   after"}}'
> {
>   "error":{
> "metadata":[
>   "error-class","org.apache.solr.common.SolrException",
>   "root-error-class","java.lang.IllegalArgumentException"],
> "msg":"   after is not a valid type of 'other' 
> range facet information",
> "code":400}}
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-12539) JSON Facet shorthand list syntax (Comma seperated list) doesn't trim leading/trailing spaces -- affects range "other" and "include" options (probably more)

2020-12-04 Thread Munendra S N (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-12539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244414#comment-17244414
 ] 

Munendra S N commented on SOLR-12539:
-

 [^SOLR-12539.patch] 
include, other and excludeTags were affected due to this. With the latest 
patch, all these cases are handled

> JSON Facet shorthand list syntax (Comma seperated list) doesn't trim 
> leading/trailing spaces -- affects range "other" and "include" options 
> (probably more)
> ---
>
> Key: SOLR-12539
> URL: https://issues.apache.org/jira/browse/SOLR-12539
> Project: Solr
>  Issue Type: Bug
>  Components: Facet Module
>Reporter: Chris M. Hostetter
>Assignee: Munendra S N
>Priority: Major
> Attachments: SOLR-12539.patch
>
>
> when doing a {{type: range}} JSON Facet request, if you use the supported 
> "comma seperated string" syntax instead of a true JSON List for specifing 
> options like {{other}} or {{include}} then any whitepsace in your string is 
> left in and breaks the parsin of those options.
> This probably affects other features of JSON Faceting that also support this 
> comma seperated string shorthand (tag exclusions?)
> 
> This works...
> {code}
> bin/solr -e techproducts
> ...
> $ curl http://localhost:8983/solr/techproducts/query -d 
> 'q=*:*&rows=0&omitHeader=true&json.facet=
> {x:{type:range,
> field:price,
> start:0,
> end:100,
> gap:20,
> other:"before,after"}}'
> {
>   "response":{"numFound":32,"start":0,"docs":[]
>   },
>   "facets":{
> "count":32,
> "x":{
>   "buckets":[{
>   "val":0.0,
>   "count":5},
> ...
>   "before":{
> "count":0},
>   "after":{
> "count":9
> {code}
> This doesn't (note the subtle amount of whitespace in the error message)...
> {code}
> $ curl http://localhost:8983/solr/techproducts/query -d 
> 'q=*:*&rows=0&omitHeader=true&json.facet=
> {x:{type:range,
> field:price,
> start:0,
> end:100,
> gap:20,
> other:"before, after"}}'
> {
>   "error":{
> "metadata":[
>   "error-class","org.apache.solr.common.SolrException",
>   "root-error-class","java.lang.IllegalArgumentException"],
> "msg":" after is not a valid type of 'other' range facet information",
> "code":400}}
> {code}
> ...exagerated...
> {code}
> $ curl http://localhost:8983/solr/techproducts/query -d 
> 'q=*:*&rows=0&omitHeader=true&json.facet=
> {x:{type:range,
> field:price,
> start:0,
> end:100,
> gap:20,
> other:"before,   after"}}'
> {
>   "error":{
> "metadata":[
>   "error-class","org.apache.solr.common.SolrException",
>   "root-error-class","java.lang.IllegalArgumentException"],
> "msg":"   after is not a valid type of 'other' 
> range facet information",
> "code":400}}
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-12559) FunctionQParser.FLAG_USE_FIELDNAME_SOURCE doesn't work when subparsers are involved

2020-12-04 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N updated SOLR-12559:

Attachment: SOLR-12559.patch

> FunctionQParser.FLAG_USE_FIELDNAME_SOURCE doesn't work when subparsers are 
> involved
> ---
>
> Key: SOLR-12559
> URL: https://issues.apache.org/jira/browse/SOLR-12559
> Project: Solr
>  Issue Type: Bug
>  Components: Facet Module
>Reporter: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-12559.patch
>
>
> While working on a patch dealing with json facet syntax parsing, i was trying 
> to write a test verifying the equivalence of 2 json facet requests that 
> should be identical and discovered that when SOLR-10613 was implemented to 
> defer the parsing of field names via {{FieldNameValueSource}} the 
> implementation did not account for the invocation of sub parsers via param 
> references.
> specifically -- this json facet request will produce two AggValueSources 
> which are not {{equals()}}...
> {noformat}
> curl http://localhost:8983/solr/query -d 'q=*:*&my_field=foo_i&
> json.facet={
>   x : "min(foo_i)",
>   y : "min($my_field)"
> }'
> {noformat}
> "x" uses {{FieldNameValueSource}} while "y" directly uses an 
> {{IntValueSource}}
> (It's not immediately obvious to me if this currently causes any user visible 
> bugs or performance hicups, but it will definitely be problematic for users 
> once we add support for {{min(multivalued_field_i)}} )



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-12559) FunctionQParser.FLAG_USE_FIELDNAME_SOURCE doesn't work when subparsers are involved

2020-12-04 Thread Munendra S N (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244417#comment-17244417
 ] 

Munendra S N commented on SOLR-12559:
-

 [^SOLR-12559.patch] 
The patch handles the 1 level dereferenced case like {{min($my_field)}} with 
my_field=some_s but doesn't handle cases like {{min($my_field)}} with 
my_field=some_s and some_s=actual_field.

> FunctionQParser.FLAG_USE_FIELDNAME_SOURCE doesn't work when subparsers are 
> involved
> ---
>
> Key: SOLR-12559
> URL: https://issues.apache.org/jira/browse/SOLR-12559
> Project: Solr
>  Issue Type: Bug
>  Components: Facet Module
>Reporter: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-12559.patch
>
>
> While working on a patch dealing with json facet syntax parsing, i was trying 
> to write a test verifying the equivalence of 2 json facet requests that 
> should be identical and discovered that when SOLR-10613 was implemented to 
> defer the parsing of field names via {{FieldNameValueSource}} the 
> implementation did not account for the invocation of sub parsers via param 
> references.
> specifically -- this json facet request will produce two AggValueSources 
> which are not {{equals()}}...
> {noformat}
> curl http://localhost:8983/solr/query -d 'q=*:*&my_field=foo_i&
> json.facet={
>   x : "min(foo_i)",
>   y : "min($my_field)"
> }'
> {noformat}
> "x" uses {{FieldNameValueSource}} while "y" directly uses an 
> {{IntValueSource}}
> (It's not immediately obvious to me if this currently causes any user visible 
> bugs or performance hicups, but it will definitely be problematic for users 
> once we add support for {{min(multivalued_field_i)}} )



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Assigned] (SOLR-12559) FunctionQParser.FLAG_USE_FIELDNAME_SOURCE doesn't work when subparsers are involved

2020-12-04 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N reassigned SOLR-12559:
---

Assignee: Munendra S N

> FunctionQParser.FLAG_USE_FIELDNAME_SOURCE doesn't work when subparsers are 
> involved
> ---
>
> Key: SOLR-12559
> URL: https://issues.apache.org/jira/browse/SOLR-12559
> Project: Solr
>  Issue Type: Bug
>  Components: Facet Module
>Reporter: Chris M. Hostetter
>Assignee: Munendra S N
>Priority: Major
> Attachments: SOLR-12559.patch
>
>
> While working on a patch dealing with json facet syntax parsing, i was trying 
> to write a test verifying the equivalence of 2 json facet requests that 
> should be identical and discovered that when SOLR-10613 was implemented to 
> defer the parsing of field names via {{FieldNameValueSource}} the 
> implementation did not account for the invocation of sub parsers via param 
> references.
> specifically -- this json facet request will produce two AggValueSources 
> which are not {{equals()}}...
> {noformat}
> curl http://localhost:8983/solr/query -d 'q=*:*&my_field=foo_i&
> json.facet={
>   x : "min(foo_i)",
>   y : "min($my_field)"
> }'
> {noformat}
> "x" uses {{FieldNameValueSource}} while "y" directly uses an 
> {{IntValueSource}}
> (It's not immediately obvious to me if this currently causes any user visible 
> bugs or performance hicups, but it will definitely be problematic for users 
> once we add support for {{min(multivalued_field_i)}} )



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-14950) schema api sometimes fails to replace-field-type

2020-12-04 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-14950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N updated SOLR-14950:

Status: Patch Available  (was: Open)

> schema api sometimes fails to replace-field-type
> 
>
> Key: SOLR-14950
> URL: https://issues.apache.org/jira/browse/SOLR-14950
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>Affects Versions: 8.4.1
>Reporter: Andrew Shumway
>Assignee: Munendra S N
>Priority: Minor
>  Labels: schema
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When updating a schema via POST to  {{/solr/COLLECTION/schema}} using  
> {{replace-field-type}} , the call fails if the schema has existing 
> copy-fields whose destination references a non-dynamic field of the  
> {{field-type}} which is being updated.
>  
> Example steps: # on a blank schema, add a field type and copy-field by POSTing
> {
>   "add-field-type": [
> {
>   "name": "spellcheck", "class": "solr.TextField","positionIncrementGap": 
> "100",
>   "analyzer": {
> "tokenizer": {
>   "class": "solr.StandardTokenizerFactory"
> }
>   }
> }
>   ],
>   "add-field": [
> {
>   "name": "spellcheck",  "type": "spellcheck",
>   "multiValued": true, "indexed": true, "stored": true
> }
>   ],
>   "add-copy-field": [
> {
>   "source": "important_data_s",
>   "dest": "spellcheck"
> }
>   ]
> }
> 2. Later, send an update for the field type
> {
>   "replace-field-type": [
> {
>   "name": "spellcheck", "class": "solr.TextField","positionIncrementGap": 
> "100",
>   "analyzer": {
> "tokenizer": {
>   "class": "solr.StandardTokenizerFactory"
> }
>   }
> }
>   ]
> }
> This results in {{"errorMessages":["Source or Destination SchemaField can't 
> be NULL.\n"]}],}}The error does not trigger if, as mentioned above, the 
> destination of the existing copy-field is dynamic OR if the source field 
> mapping contains a wildcard e.g. {{*foo_s}} (edited) 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-12559) FunctionQParser.FLAG_USE_FIELDNAME_SOURCE doesn't work when subparsers are involved

2020-12-04 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N updated SOLR-12559:

Status: Patch Available  (was: Open)

> FunctionQParser.FLAG_USE_FIELDNAME_SOURCE doesn't work when subparsers are 
> involved
> ---
>
> Key: SOLR-12559
> URL: https://issues.apache.org/jira/browse/SOLR-12559
> Project: Solr
>  Issue Type: Bug
>  Components: Facet Module
>Reporter: Chris M. Hostetter
>Assignee: Munendra S N
>Priority: Major
> Attachments: SOLR-12559.patch
>
>
> While working on a patch dealing with json facet syntax parsing, i was trying 
> to write a test verifying the equivalence of 2 json facet requests that 
> should be identical and discovered that when SOLR-10613 was implemented to 
> defer the parsing of field names via {{FieldNameValueSource}} the 
> implementation did not account for the invocation of sub parsers via param 
> references.
> specifically -- this json facet request will produce two AggValueSources 
> which are not {{equals()}}...
> {noformat}
> curl http://localhost:8983/solr/query -d 'q=*:*&my_field=foo_i&
> json.facet={
>   x : "min(foo_i)",
>   y : "min($my_field)"
> }'
> {noformat}
> "x" uses {{FieldNameValueSource}} while "y" directly uses an 
> {{IntValueSource}}
> (It's not immediately obvious to me if this currently causes any user visible 
> bugs or performance hicups, but it will definitely be problematic for users 
> once we add support for {{min(multivalued_field_i)}} )



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-12539) JSON Facet shorthand list syntax (Comma seperated list) doesn't trim leading/trailing spaces -- affects range "other" and "include" options (probably more)

2020-12-04 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-12539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N updated SOLR-12539:

Status: Patch Available  (was: Open)

> JSON Facet shorthand list syntax (Comma seperated list) doesn't trim 
> leading/trailing spaces -- affects range "other" and "include" options 
> (probably more)
> ---
>
> Key: SOLR-12539
> URL: https://issues.apache.org/jira/browse/SOLR-12539
> Project: Solr
>  Issue Type: Bug
>  Components: Facet Module
>Reporter: Chris M. Hostetter
>Assignee: Munendra S N
>Priority: Major
> Attachments: SOLR-12539.patch
>
>
> when doing a {{type: range}} JSON Facet request, if you use the supported 
> "comma seperated string" syntax instead of a true JSON List for specifing 
> options like {{other}} or {{include}} then any whitepsace in your string is 
> left in and breaks the parsin of those options.
> This probably affects other features of JSON Faceting that also support this 
> comma seperated string shorthand (tag exclusions?)
> 
> This works...
> {code}
> bin/solr -e techproducts
> ...
> $ curl http://localhost:8983/solr/techproducts/query -d 
> 'q=*:*&rows=0&omitHeader=true&json.facet=
> {x:{type:range,
> field:price,
> start:0,
> end:100,
> gap:20,
> other:"before,after"}}'
> {
>   "response":{"numFound":32,"start":0,"docs":[]
>   },
>   "facets":{
> "count":32,
> "x":{
>   "buckets":[{
>   "val":0.0,
>   "count":5},
> ...
>   "before":{
> "count":0},
>   "after":{
> "count":9
> {code}
> This doesn't (note the subtle amount of whitespace in the error message)...
> {code}
> $ curl http://localhost:8983/solr/techproducts/query -d 
> 'q=*:*&rows=0&omitHeader=true&json.facet=
> {x:{type:range,
> field:price,
> start:0,
> end:100,
> gap:20,
> other:"before, after"}}'
> {
>   "error":{
> "metadata":[
>   "error-class","org.apache.solr.common.SolrException",
>   "root-error-class","java.lang.IllegalArgumentException"],
> "msg":" after is not a valid type of 'other' range facet information",
> "code":400}}
> {code}
> ...exagerated...
> {code}
> $ curl http://localhost:8983/solr/techproducts/query -d 
> 'q=*:*&rows=0&omitHeader=true&json.facet=
> {x:{type:range,
> field:price,
> start:0,
> end:100,
> gap:20,
> other:"before,   after"}}'
> {
>   "error":{
> "metadata":[
>   "error-class","org.apache.solr.common.SolrException",
>   "root-error-class","java.lang.IllegalArgumentException"],
> "msg":"   after is not a valid type of 'other' 
> range facet information",
> "code":400}}
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-14514) json.facets: method:stream is incompatible with allBuckets:true

2020-12-04 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-14514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N updated SOLR-14514:

Status: Patch Available  (was: Open)

> json.facets: method:stream is incompatible with allBuckets:true
> ---
>
> Key: SOLR-14514
> URL: https://issues.apache.org/jira/browse/SOLR-14514
> Project: Solr
>  Issue Type: Bug
>  Components: Facet Module
>Reporter: Chris M. Hostetter
>Assignee: Munendra S N
>Priority: Minor
> Attachments: SOLR-14514.patch
>
>
> {{FacetFieldProcessorByEnumTermsStream}} has never supported 
> {{allBuckets:true}} but it also doesn't fail outright if {{allBuckets:true}} 
> is specified -- instead the bucket is silently missing from the response.
> Given how the {{method}} option is only used as a suggestion, and the actuall 
> processor can change based on hueristics about the request, this means that 
> the behavior of combining {{allBuckets:true}} with {{method:stream}} can vary 
> depending on the other options specified -- notably the {{sort}}
> {noformat}
> % curl -sS -X POST http://localhost:8983/solr/techproducts/query -d 
> 'omitHeader=true&rows=0&q=*:*&json.facet={
>   x : {
> type : terms,
> method: stream,
> field : manu_id_s,
> allBuckets : true,
> limit : 2,
> } }'
> 
> {
>   "response":{"numFound":32,"start":0,"docs":[]
>   },
>   "facets":{
> "count":32,
> "x":{
>   "allBuckets":{
> "count":18},
>   "buckets":[{
>   "val":"corsair",
>   "count":3},
> {
>   "val":"belkin",
>   "count":2}]}}}
> % curl -sS -X POST http://localhost:8983/solr/techproducts/query -d 
> 'omitHeader=true&rows=0&q=*:*&json.facet={
>   x : {
> type : terms,
> method: stream,
> field : manu_id_s,
> allBuckets : true,
> limit : 2,
> sort: "index asc"
> } }'
> {
>   "response":{"numFound":32,"start":0,"docs":[]
>   },
>   "facets":{
> "count":32,
> "x":{
>   "buckets":[{
>   "val":"apple",
>   "count":1},
> {
>   "val":"asus",
>   "count":1}]}}}
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-10732) potential optimizations in callers of SolrIndexSearcher.numDocs when docset is empty

2020-12-04 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-10732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N updated SOLR-10732:

Attachment: SOLR-10732.patch

> potential optimizations in callers of SolrIndexSearcher.numDocs when docset 
> is empty
> 
>
> Key: SOLR-10732
> URL: https://issues.apache.org/jira/browse/SOLR-10732
> Project: Solr
>  Issue Type: Improvement
>Reporter: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-10732.patch
>
>
> spin off of SOLR-10727...
> {quote}
> ...why not (also) optimize it slightly higher up and completely avoid the 
> construction of the Query objects? (and in some cases: additional overhead)
> for example: the first usage of {{SolrIndexSearcher.numDocs(Query,DocSet)}} i 
> found was {{RangeFacetProcessor.rangeCount(DocSet subset,...)}} ... if the 
> first line of that method was {{if (0 == subset.size()) return 0}} then we'd 
> not only optimize away the SolrIndexSearcher hit, but also fetching the 
> SchemaField & building the range query (not to mention the much more 
> expensive {{getGroupedFacetQueryCount}} in the grouping case)
> At a glance, most other callers of 
> {{SolrIndexSearcher.numDocs(Query,DocSet)}} could be trivially optimize this 
> way as well -- at a minimum to eliminate Query parsing/construction.
> {quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] zacharymorn commented on a change in pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-12-04 Thread GitBox


zacharymorn commented on a change in pull request #2052:
URL: https://github.com/apache/lucene-solr/pull/2052#discussion_r536531755



##
File path: lucene/misc/native/build.gradle
##
@@ -51,11 +47,7 @@ tasks.withType(CppCompile).configureEach {
   // is present.
   systemIncludes.from file("${javaHome}/include")
 
-  for (def path : [

Review comment:
   Hahahahahaha! YES DEFINITELY! Good one Dawid!





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-10732) potential optimizations in callers of SolrIndexSearcher.numDocs when docset is empty

2020-12-04 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-10732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N updated SOLR-10732:

Status: Patch Available  (was: Open)

> potential optimizations in callers of SolrIndexSearcher.numDocs when docset 
> is empty
> 
>
> Key: SOLR-10732
> URL: https://issues.apache.org/jira/browse/SOLR-10732
> Project: Solr
>  Issue Type: Improvement
>Reporter: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-10732.patch
>
>
> spin off of SOLR-10727...
> {quote}
> ...why not (also) optimize it slightly higher up and completely avoid the 
> construction of the Query objects? (and in some cases: additional overhead)
> for example: the first usage of {{SolrIndexSearcher.numDocs(Query,DocSet)}} i 
> found was {{RangeFacetProcessor.rangeCount(DocSet subset,...)}} ... if the 
> first line of that method was {{if (0 == subset.size()) return 0}} then we'd 
> not only optimize away the SolrIndexSearcher hit, but also fetching the 
> SchemaField & building the range query (not to mention the much more 
> expensive {{getGroupedFacetQueryCount}} in the grouping case)
> At a glance, most other callers of 
> {{SolrIndexSearcher.numDocs(Query,DocSet)}} could be trivially optimize this 
> way as well -- at a minimum to eliminate Query parsing/construction.
> {quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-10732) potential optimizations in callers of SolrIndexSearcher.numDocs when docset is empty

2020-12-04 Thread Munendra S N (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-10732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244419#comment-17244419
 ] 

Munendra S N commented on SOLR-10732:
-

 [^SOLR-10732.patch] 
All tests are passing. I will wait for feedback, if any before committing 

> potential optimizations in callers of SolrIndexSearcher.numDocs when docset 
> is empty
> 
>
> Key: SOLR-10732
> URL: https://issues.apache.org/jira/browse/SOLR-10732
> Project: Solr
>  Issue Type: Improvement
>Reporter: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-10732.patch
>
>
> spin off of SOLR-10727...
> {quote}
> ...why not (also) optimize it slightly higher up and completely avoid the 
> construction of the Query objects? (and in some cases: additional overhead)
> for example: the first usage of {{SolrIndexSearcher.numDocs(Query,DocSet)}} i 
> found was {{RangeFacetProcessor.rangeCount(DocSet subset,...)}} ... if the 
> first line of that method was {{if (0 == subset.size()) return 0}} then we'd 
> not only optimize away the SolrIndexSearcher hit, but also fetching the 
> SchemaField & building the range query (not to mention the much more 
> expensive {{getGroupedFacetQueryCount}} in the grouping case)
> At a glance, most other callers of 
> {{SolrIndexSearcher.numDocs(Query,DocSet)}} could be trivially optimize this 
> way as well -- at a minimum to eliminate Query parsing/construction.
> {quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-14451) debug=query does not reliably ensure json.facet debug info returned in cloud clusters

2020-12-04 Thread Munendra S N (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-14451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Munendra S N updated SOLR-14451:

Attachment: SOLR-14451.patch

> debug=query does not reliably ensure json.facet debug info returned in cloud 
> clusters
> -
>
> Key: SOLR-14451
> URL: https://issues.apache.org/jira/browse/SOLR-14451
> Project: Solr
>  Issue Type: Bug
>  Components: Facet Module, SolrCloud
>Reporter: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-14451.patch
>
>
> While working on a randomized cloud test in SOLR-13132 that needs to inspect 
> the {{"facet-trace"}} debug output to inspect the processor used (among other 
> details), i discovered that {{FacetModule}} would not reliably initialize the 
> {{FactDebugInfo}} container when using {{debug=query}} in the request.
> AFAICT, it *does* appear to always track & return facet debugging info when 
> using {{debug=true}} instead - but i haven't rigerously proved this.
> Under the covers, {{FacetModule}} is just checking 
> {{ResponseBuilder.isDebug()}} - an old "back compat" API for components to 
> check before we started breaking down multiple types of "debug" info - that 
> should return true anytime {{debug=query}} is set, so there is really no 
> logical reason why this shouldn't work... it's got to be a bug somewhere (it 
> doesn't seem tied to the request, but does break on diff randomized indexes, 
> so perhaps it's related to the document/bucket distribution? or maybe 
> something more low level and unrelated to FacetModule at all?)
> 
> It only seems to be a problem in cloud mode - i haven't observed 
> FacetModule's debug output behave any differently between {{debug=query}} vs 
> {{debug=true}} in single node mode.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14451) debug=query does not reliably ensure json.facet debug info returned in cloud clusters

2020-12-04 Thread Munendra S N (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244423#comment-17244423
 ] 

Munendra S N commented on SOLR-14451:
-

 [^SOLR-14451.patch] 
Initial patch. Propagate debug=query to all shard request. Still need to add 
test cases.
While working on this, I noticed when debug=timing, response contains query 
debug information instead of just containing timings. Is this expected?

> debug=query does not reliably ensure json.facet debug info returned in cloud 
> clusters
> -
>
> Key: SOLR-14451
> URL: https://issues.apache.org/jira/browse/SOLR-14451
> Project: Solr
>  Issue Type: Bug
>  Components: Facet Module, SolrCloud
>Reporter: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-14451.patch
>
>
> While working on a randomized cloud test in SOLR-13132 that needs to inspect 
> the {{"facet-trace"}} debug output to inspect the processor used (among other 
> details), i discovered that {{FacetModule}} would not reliably initialize the 
> {{FactDebugInfo}} container when using {{debug=query}} in the request.
> AFAICT, it *does* appear to always track & return facet debugging info when 
> using {{debug=true}} instead - but i haven't rigerously proved this.
> Under the covers, {{FacetModule}} is just checking 
> {{ResponseBuilder.isDebug()}} - an old "back compat" API for components to 
> check before we started breaking down multiple types of "debug" info - that 
> should return true anytime {{debug=query}} is set, so there is really no 
> logical reason why this shouldn't work... it's got to be a bug somewhere (it 
> doesn't seem tied to the request, but does break on diff randomized indexes, 
> so perhaps it's related to the document/bucket distribution? or maybe 
> something more low level and unrelated to FacetModule at all?)
> 
> It only seems to be a problem in cloud mode - i haven't observed 
> FacetModule's debug output behave any differently between {{debug=query}} vs 
> {{debug=true}} in single node mode.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (LUCENE-9623) Add module descriptor (module-info.java) to lucene jars

2020-12-04 Thread Tomoko Uchida (Jira)


[ 
https://issues.apache.org/jira/browse/LUCENE-9623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244424#comment-17244424
 ] 

Tomoko Uchida commented on LUCENE-9623:
---

As for tests, I think it might be a good start to run luke module's tests with 
the module paths instead of classpaths. Luke depends on a number of lucene 
jars, though it does not use all sub-modules, and its tests for internal APIs - 
thin wrapper of lucene APIs - can be run without Swing UI.

> Add module descriptor (module-info.java) to lucene jars
> ---
>
> Key: LUCENE-9623
> URL: https://issues.apache.org/jira/browse/LUCENE-9623
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/build
>Affects Versions: master (9.0)
>Reporter: Tomoko Uchida
>Priority: Major
> Attachments: generate-all-module-info.sh
>
>
> For a starter, module descriptors can be automatically generated by jdeps 
> utility.
> There are two choices.
> 1. generate "open" modules which allows reflective accesses with 
> --generate-open-module option
> 2. generate non-open modules with --generate-module-info option
> Which is the better - not fully sure, but maybe 2 (non-open modules)?
> Also, we need to choose proper module names - just using gradle project path 
> for it is OK?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] dweiss commented on pull request #2113: LUCENE-9629: Use computed masks

2020-12-04 Thread GitBox


dweiss commented on pull request #2113:
URL: https://github.com/apache/lucene-solr/pull/2113#issuecomment-739139541


   Thanks for satisfying my curiosity!



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (LUCENE-9623) Add module descriptor (module-info.java) to lucene jars

2020-12-04 Thread Tomoko Uchida (Jira)


[ 
https://issues.apache.org/jira/browse/LUCENE-9623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244429#comment-17244429
 ] 

Tomoko Uchida commented on LUCENE-9623:
---

{quote}exports are done manual
{quote}
Ideally, we should manually maintain that which packages are exported and which 
ones are kept private, but I'm afraid it is a heavy burden for devs. Maybe, we 
could take a hybrid approach (at least initial releases with modules) - use 
manually maintained module-info.java if there exists it for a module, otherwise 
automatically generate it by jdeps tool ?

> Add module descriptor (module-info.java) to lucene jars
> ---
>
> Key: LUCENE-9623
> URL: https://issues.apache.org/jira/browse/LUCENE-9623
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/build
>Affects Versions: master (9.0)
>Reporter: Tomoko Uchida
>Priority: Major
> Attachments: generate-all-module-info.sh
>
>
> For a starter, module descriptors can be automatically generated by jdeps 
> utility.
> There are two choices.
> 1. generate "open" modules which allows reflective accesses with 
> --generate-open-module option
> 2. generate non-open modules with --generate-module-info option
> Which is the better - not fully sure, but maybe 2 (non-open modules)?
> Also, we need to choose proper module names - just using gradle project path 
> for it is OK?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org