[jira] [Created] (SOLR-14208) Reproducible test failure on TestBulkSchemaConcurrent
Shalin Shekhar Mangar created SOLR-14208:
Summary: Reproducible test failure on TestBulkSchemaConcurrent
Key: SOLR-14208
URL: https://issues.apache.org/jira/browse/SOLR-14208
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: Tests
Reporter: Shalin Shekhar Mangar
I found the following test failure on master branch while running tests on
SOLR-14207. The test failure is reproducible without the SOLR-14207 patch.
{code}
ant test -Dtestcase=TestBulkSchemaConcurrent -Dtests.method=test
-Dtests.seed=AE6DC9DB591DAB9E -Dtests.slow=true -Dtests.badapples=true
-Dtests.locale=hi-IN -Dtests.timezone=Atlantic/Madeira -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
{code}
The logs are full of the following warning repeated over and over:
{code}
[junit4] 2> 32396 WARN (qtp1791658098-110) [n:127.0.0.1:46453_rx_%2Fr
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n5 ]
o.a.s.s.SchemaManager Unable to retrieve fresh managed schema managed-schema
[junit4] 2> => java.lang.IllegalArgumentException: Path must
start with / character
[junit4] 2>at
org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:51)
[junit4] 2> java.lang.IllegalArgumentException: Path must start with /
character
[junit4] 2>at
org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:51)
~[zookeeper-3.5.5.jar:3.5.5]
[junit4] 2>at
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:2000)
~[zookeeper-3.5.5.jar:3.5.5]
[junit4] 2>at
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:314)
~[java/:?]
[junit4] 2>at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
~[java/:?]
[junit4] 2>at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:314)
~[java/:?]
[junit4] 2>at
org.apache.solr.schema.SchemaManager.getFreshManagedSchema(SchemaManager.java:427)
~[java/:?]
[junit4] 2>at
org.apache.solr.schema.SchemaManager.doOperations(SchemaManager.java:107)
~[java/:?]
[junit4] 2>at
org.apache.solr.schema.SchemaManager.performOperations(SchemaManager.java:92)
~[java/:?]
[junit4] 2>at
org.apache.solr.handler.SchemaHandler.handleRequestBody(SchemaHandler.java:90)
~[java/:?]
[junit4] 2>at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:208)
~[java/:?]
[junit4] 2>at
org.apache.solr.core.SolrCore.execute(SolrCore.java:2582) ~[java/:?]
[junit4] 2>at
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:799) ~[java/:?]
[junit4] 2>at
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:578) ~[java/:?]
[junit4] 2>at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:419)
~[java/:?]
[junit4] 2>at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
~[java/:?]
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9164) Should not consider ACE a tragedy if IW is closed
[
https://issues.apache.org/jira/browse/LUCENE-9164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021854#comment-17021854
]
Lucene/Solr QA commented on LUCENE-9164:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 0m 22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
22s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 6m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | LUCENE-9164 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12991592/LUCENE-9164.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene1-us-west 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-LUCENE-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / c53cbb12f4d |
| ant | version: Apache Ant(TM) version 1.10.5 compiled on March 28 2019 |
| Default Java | LTS |
| Test Results |
https://builds.apache.org/job/PreCommit-LUCENE-Build/248/testReport/ |
| modules | C: lucene/core U: lucene/core |
| Console output |
https://builds.apache.org/job/PreCommit-LUCENE-Build/248/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Should not consider ACE a tragedy if IW is closed
> -
>
> Key: LUCENE-9164
> URL: https://issues.apache.org/jira/browse/LUCENE-9164
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: master (9.0), 8.5, 8.4.2
>Reporter: Nhat Nguyen
>Assignee: Nhat Nguyen
>Priority: Major
> Attachments: LUCENE-9164.patch
>
>
> If IndexWriter is closed or being closed, AlreadyClosedException is expected.
> We should not consider it a tragic event in this case.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14207) Fix logging statements with less or more arguments than placeholders
[ https://issues.apache.org/jira/browse/SOLR-14207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021855#comment-17021855 ] ASF subversion and git services commented on SOLR-14207: Commit 04193d5252bcbdb30d0e29447a72d4e9006e7f3f in lucene-solr's branch refs/heads/master from Shalin Shekhar Mangar [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=04193d5 ] SOLR-14207: Fix logging statements with less or more arguments than placeholders > Fix logging statements with less or more arguments than placeholders > > > Key: SOLR-14207 > URL: https://issues.apache.org/jira/browse/SOLR-14207 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: logging >Reporter: Shalin Shekhar Mangar >Assignee: Shalin Shekhar Mangar >Priority: Minor > Fix For: master (9.0), 8.5 > > Attachments: SOLR-14207.patch > > > I found bad logging statements in the solr-exporter which had different > number of arguments than placeholders. Ran an inspection check in Idea and > found many more places with similar problems. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14207) Fix logging statements with less or more arguments than placeholders
[ https://issues.apache.org/jira/browse/SOLR-14207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021857#comment-17021857 ] ASF subversion and git services commented on SOLR-14207: Commit 0dea3c7060b3c3c219a550c5ecdceea86e34c403 in lucene-solr's branch refs/heads/branch_8x from Shalin Shekhar Mangar [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=0dea3c7 ] SOLR-14207: Fix logging statements with less or more arguments than placeholders (cherry picked from commit 04193d5252bcbdb30d0e29447a72d4e9006e7f3f) > Fix logging statements with less or more arguments than placeholders > > > Key: SOLR-14207 > URL: https://issues.apache.org/jira/browse/SOLR-14207 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: logging >Reporter: Shalin Shekhar Mangar >Assignee: Shalin Shekhar Mangar >Priority: Minor > Fix For: master (9.0), 8.5 > > Attachments: SOLR-14207.patch > > > I found bad logging statements in the solr-exporter which had different > number of arguments than placeholders. Ran an inspection check in Idea and > found many more places with similar problems. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14207) Fix logging statements with less or more arguments than placeholders
[ https://issues.apache.org/jira/browse/SOLR-14207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar resolved SOLR-14207. -- Resolution: Fixed > Fix logging statements with less or more arguments than placeholders > > > Key: SOLR-14207 > URL: https://issues.apache.org/jira/browse/SOLR-14207 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: logging >Reporter: Shalin Shekhar Mangar >Assignee: Shalin Shekhar Mangar >Priority: Minor > Fix For: master (9.0), 8.5 > > Attachments: SOLR-14207.patch > > > I found bad logging statements in the solr-exporter which had different > number of arguments than placeholders. Ran an inspection check in Idea and > found many more places with similar problems. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] shalinmangar commented on issue #1152: SOLR-14172: Collection metadata remains in zookeeper if too many shards requested
shalinmangar commented on issue #1152: SOLR-14172: Collection metadata remains in zookeeper if too many shards requested URL: https://github.com/apache/lucene-solr/pull/1152#issuecomment-577585953 Thanks @asalamon74 for the PR. This bug is my fault. I think it is best to change the SolrException being thrown inside `buildReplicaPositions` to an `AssignmentException`. After all it is the intended use of that exception. Also, the SolrException thrown in the catch clause inside CreateCollectionCmd.call should be a bad request instead of a server error as it is today. I'll put up a patch on Jira with these changes and commit. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14172) Collection metadata remains in zookeeper if too many shards requested
[
https://issues.apache.org/jira/browse/SOLR-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shalin Shekhar Mangar updated SOLR-14172:
-
Attachment: SOLR-14172.patch
Fix Version/s: 8.5
master (9.0)
Assignee: Shalin Shekhar Mangar
Status: Open (was: Open)
This patch incorporates the test added by Andras Salamon in PR #1152 but the
actual fix is slightly different.
This patch changes the buildReplicaPositions method to throw an
AssignmentException instead of SolrException in case the maxShardsPerNode is
insufficient. It also changes the Create Collection API to return a BAD_REQUEST
code instead of SERVER_ERROR in case of assignment exception. I'll note this
behavior change in the upgrade notes.
> Collection metadata remains in zookeeper if too many shards requested
> -
>
> Key: SOLR-14172
> URL: https://issues.apache.org/jira/browse/SOLR-14172
> Project: Solr
> Issue Type: Bug
>Affects Versions: 8.3.1
>Reporter: Andras Salamon
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: master (9.0), 8.5
>
> Attachments: SOLR-14172.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When I try to create a collection and request too many shards, collection
> creation fails with the expected error message:
> {noformat}
> $ curl -i --retry 5 -s -L -k --negotiate -u :
> 'http://asalamon-cdpd-rebase831-a-1.vpc.cloudera.com:8983/solr/admin/collections?action=CREATE&name=TooManyShardstest1&numShards=4&collection.configName=zk_init_too&maxShardsPerNode=1'
> HTTP/1.1 400 Bad Request
> Content-Type: application/json;charset=utf-8
> Content-Length: 1562
> {
> "responseHeader":{
> "status":400,
> "QTime":122},
> "Operation create caused
> exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> Cannot create collection TooManyShardstest1. Value of maxShardsPerNode is 1,
> and the number of nodes currently live or live and part of your createNodeSet
> is 3. This allows a maximum of 3 to be created. Value of numShards is 4,
> value of nrtReplicas is 1, value of tlogReplicas is 0 and value of
> pullReplicas is 0. This requires 4 shards to be created (higher than the
> allowed number)",
> "exception":{
> "msg":"Cannot create collection TooManyShardstest1. Value of
> maxShardsPerNode is 1, and the number of nodes currently live or live and
> part of your createNodeSet is 3. This allows a maximum of 3 to be created.
> Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is
> 0 and value of pullReplicas is 0. This requires 4 shards to be created
> (higher than the allowed number)",
> "rspCode":400},
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"Cannot create collection TooManyShardstest1. Value of
> maxShardsPerNode is 1, and the number of nodes currently live or live and
> part of your createNodeSet is 3. This allows a maximum of 3 to be created.
> Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is
> 0 and value of pullReplicas is 0. This requires 4 shards to be created
> (higher than the allowed number)",
> "code":400}}
> {noformat}
> Although the collection creation was not successful if I list the collections
> it shows the new collection:
> {noformat}
> $ solr collection --list
> TooManyShardstest1 (1)
> {noformat}
> Looks like metadata remains in Zookeeper:
> {noformat}
> $ zkcli.sh -zkhost asalamon-cdpd-rebase831-a-1.vpc.cloudera.com:2181/solr
> -cmd ls /collections
> INFO - 2020-01-06 04:54:01.851;
> org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect
> to ZooKeeper
> INFO - 2020-01-06 04:54:01.880;
> org.apache.solr.common.cloud.ConnectionManager; zkClient has connected
> INFO - 2020-01-06 04:54:01.881;
> org.apache.solr.common.cloud.ConnectionManager; Client is connected to
> ZooKeeper
> /collections (1)
> /collections/TooManyShardstest1 (1)
> DATA:
> {"configName":"zk_init_too"}
> /collections/TooManyShardstest1/state.json (0)
> DATA:
> {"TooManyShardstest1":{
> "pullReplicas":"0",
> "replicationFactor":"1",
> "router":{"name":"compositeId"},
> "maxShardsPerNode":"1",
> "autoAddReplicas":"false",
> "nrtReplicas":"1",
> "tlogReplicas":"0",
> "shards":{
> "shard1":{
> "range":"8000-bfff",
> "state":"active",
> "replicas":{}},
> "shard2":{
> "range":"c000-",
> "
[jira] [Commented] (SOLR-12238) Synonym Query Style Boost By Payload
[
https://issues.apache.org/jira/browse/SOLR-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021882#comment-17021882
]
Lucene/Solr QA commented on SOLR-12238:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} SOLR-12238 does not apply to master. Rebase required? Wrong
Branch? See
https://wiki.apache.org/solr/HowToContribute#Creating_the_patch_file for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-12238 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12921949/SOLR-12238.patch |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/660/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Synonym Query Style Boost By Payload
>
>
> Key: SOLR-12238
> URL: https://issues.apache.org/jira/browse/SOLR-12238
> Project: Solr
> Issue Type: Improvement
> Components: query parsers
>Affects Versions: 7.2
>Reporter: Alessandro Benedetti
>Priority: Major
> Attachments: SOLR-12238.patch, SOLR-12238.patch, SOLR-12238.patch,
> SOLR-12238.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This improvement is built on top of the Synonym Query Style feature and
> brings the possibility of boosting synonym queries using the payload
> associated.
> It introduces two new modalities for the Synonym Query Style :
> PICK_BEST_BOOST_BY_PAYLOAD -> build a Disjunction query with the clauses
> boosted by payload
> AS_DISTINCT_TERMS_BOOST_BY_PAYLOAD -> build a Boolean query with the clauses
> boosted by payload
> This new synonym query styles will assume payloads are available so they must
> be used in conjunction with a token filter able to produce payloads.
> An synonym.txt example could be :
> # Synonyms used by Payload Boost
> tiger => tiger|1.0, Big_Cat|0.8, Shere_Khan|0.9
> leopard => leopard, Big_Cat|0.8, Bagheera|0.9
> lion => lion|1.0, panthera leo|0.99, Simba|0.8
> snow_leopard => panthera uncia|0.99, snow leopard|1.0
> A simple token filter to populate the payloads from such synonym.txt is :
> delimiter="|"/>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14172) Collection metadata remains in zookeeper if too many shards requested
[
https://issues.apache.org/jira/browse/SOLR-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shalin Shekhar Mangar updated SOLR-14172:
-
Attachment: SOLR-14172.patch
> Collection metadata remains in zookeeper if too many shards requested
> -
>
> Key: SOLR-14172
> URL: https://issues.apache.org/jira/browse/SOLR-14172
> Project: Solr
> Issue Type: Bug
>Affects Versions: 8.3.1
>Reporter: Andras Salamon
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: master (9.0), 8.5
>
> Attachments: SOLR-14172.patch, SOLR-14172.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When I try to create a collection and request too many shards, collection
> creation fails with the expected error message:
> {noformat}
> $ curl -i --retry 5 -s -L -k --negotiate -u :
> 'http://asalamon-cdpd-rebase831-a-1.vpc.cloudera.com:8983/solr/admin/collections?action=CREATE&name=TooManyShardstest1&numShards=4&collection.configName=zk_init_too&maxShardsPerNode=1'
> HTTP/1.1 400 Bad Request
> Content-Type: application/json;charset=utf-8
> Content-Length: 1562
> {
> "responseHeader":{
> "status":400,
> "QTime":122},
> "Operation create caused
> exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> Cannot create collection TooManyShardstest1. Value of maxShardsPerNode is 1,
> and the number of nodes currently live or live and part of your createNodeSet
> is 3. This allows a maximum of 3 to be created. Value of numShards is 4,
> value of nrtReplicas is 1, value of tlogReplicas is 0 and value of
> pullReplicas is 0. This requires 4 shards to be created (higher than the
> allowed number)",
> "exception":{
> "msg":"Cannot create collection TooManyShardstest1. Value of
> maxShardsPerNode is 1, and the number of nodes currently live or live and
> part of your createNodeSet is 3. This allows a maximum of 3 to be created.
> Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is
> 0 and value of pullReplicas is 0. This requires 4 shards to be created
> (higher than the allowed number)",
> "rspCode":400},
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"Cannot create collection TooManyShardstest1. Value of
> maxShardsPerNode is 1, and the number of nodes currently live or live and
> part of your createNodeSet is 3. This allows a maximum of 3 to be created.
> Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is
> 0 and value of pullReplicas is 0. This requires 4 shards to be created
> (higher than the allowed number)",
> "code":400}}
> {noformat}
> Although the collection creation was not successful if I list the collections
> it shows the new collection:
> {noformat}
> $ solr collection --list
> TooManyShardstest1 (1)
> {noformat}
> Looks like metadata remains in Zookeeper:
> {noformat}
> $ zkcli.sh -zkhost asalamon-cdpd-rebase831-a-1.vpc.cloudera.com:2181/solr
> -cmd ls /collections
> INFO - 2020-01-06 04:54:01.851;
> org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect
> to ZooKeeper
> INFO - 2020-01-06 04:54:01.880;
> org.apache.solr.common.cloud.ConnectionManager; zkClient has connected
> INFO - 2020-01-06 04:54:01.881;
> org.apache.solr.common.cloud.ConnectionManager; Client is connected to
> ZooKeeper
> /collections (1)
> /collections/TooManyShardstest1 (1)
> DATA:
> {"configName":"zk_init_too"}
> /collections/TooManyShardstest1/state.json (0)
> DATA:
> {"TooManyShardstest1":{
> "pullReplicas":"0",
> "replicationFactor":"1",
> "router":{"name":"compositeId"},
> "maxShardsPerNode":"1",
> "autoAddReplicas":"false",
> "nrtReplicas":"1",
> "tlogReplicas":"0",
> "shards":{
> "shard1":{
> "range":"8000-bfff",
> "state":"active",
> "replicas":{}},
> "shard2":{
> "range":"c000-",
> "state":"active",
> "replicas":{}},
> "shard3":{
> "range":"0-3fff",
> "state":"active",
> "replicas":{}},
> "shard4":{
> "range":"4000-7fff",
> "state":"active",
> "replicas":{}
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@
[jira] [Commented] (SOLR-14172) Collection metadata remains in zookeeper if too many shards requested
[
https://issues.apache.org/jira/browse/SOLR-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021890#comment-17021890
]
Shalin Shekhar Mangar commented on SOLR-14172:
--
I attached a new patch which adds a failure message in case the collection
creation request is successful.
> Collection metadata remains in zookeeper if too many shards requested
> -
>
> Key: SOLR-14172
> URL: https://issues.apache.org/jira/browse/SOLR-14172
> Project: Solr
> Issue Type: Bug
>Affects Versions: 8.3.1
>Reporter: Andras Salamon
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: master (9.0), 8.5
>
> Attachments: SOLR-14172.patch, SOLR-14172.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When I try to create a collection and request too many shards, collection
> creation fails with the expected error message:
> {noformat}
> $ curl -i --retry 5 -s -L -k --negotiate -u :
> 'http://asalamon-cdpd-rebase831-a-1.vpc.cloudera.com:8983/solr/admin/collections?action=CREATE&name=TooManyShardstest1&numShards=4&collection.configName=zk_init_too&maxShardsPerNode=1'
> HTTP/1.1 400 Bad Request
> Content-Type: application/json;charset=utf-8
> Content-Length: 1562
> {
> "responseHeader":{
> "status":400,
> "QTime":122},
> "Operation create caused
> exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> Cannot create collection TooManyShardstest1. Value of maxShardsPerNode is 1,
> and the number of nodes currently live or live and part of your createNodeSet
> is 3. This allows a maximum of 3 to be created. Value of numShards is 4,
> value of nrtReplicas is 1, value of tlogReplicas is 0 and value of
> pullReplicas is 0. This requires 4 shards to be created (higher than the
> allowed number)",
> "exception":{
> "msg":"Cannot create collection TooManyShardstest1. Value of
> maxShardsPerNode is 1, and the number of nodes currently live or live and
> part of your createNodeSet is 3. This allows a maximum of 3 to be created.
> Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is
> 0 and value of pullReplicas is 0. This requires 4 shards to be created
> (higher than the allowed number)",
> "rspCode":400},
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"Cannot create collection TooManyShardstest1. Value of
> maxShardsPerNode is 1, and the number of nodes currently live or live and
> part of your createNodeSet is 3. This allows a maximum of 3 to be created.
> Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is
> 0 and value of pullReplicas is 0. This requires 4 shards to be created
> (higher than the allowed number)",
> "code":400}}
> {noformat}
> Although the collection creation was not successful if I list the collections
> it shows the new collection:
> {noformat}
> $ solr collection --list
> TooManyShardstest1 (1)
> {noformat}
> Looks like metadata remains in Zookeeper:
> {noformat}
> $ zkcli.sh -zkhost asalamon-cdpd-rebase831-a-1.vpc.cloudera.com:2181/solr
> -cmd ls /collections
> INFO - 2020-01-06 04:54:01.851;
> org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect
> to ZooKeeper
> INFO - 2020-01-06 04:54:01.880;
> org.apache.solr.common.cloud.ConnectionManager; zkClient has connected
> INFO - 2020-01-06 04:54:01.881;
> org.apache.solr.common.cloud.ConnectionManager; Client is connected to
> ZooKeeper
> /collections (1)
> /collections/TooManyShardstest1 (1)
> DATA:
> {"configName":"zk_init_too"}
> /collections/TooManyShardstest1/state.json (0)
> DATA:
> {"TooManyShardstest1":{
> "pullReplicas":"0",
> "replicationFactor":"1",
> "router":{"name":"compositeId"},
> "maxShardsPerNode":"1",
> "autoAddReplicas":"false",
> "nrtReplicas":"1",
> "tlogReplicas":"0",
> "shards":{
> "shard1":{
> "range":"8000-bfff",
> "state":"active",
> "replicas":{}},
> "shard2":{
> "range":"c000-",
> "state":"active",
> "replicas":{}},
> "shard3":{
> "range":"0-3fff",
> "state":"active",
> "replicas":{}},
> "shard4":{
> "range":"4000-7fff",
> "state":"active",
> "replicas":{}
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---
[GitHub] [lucene-solr] asfgit closed pull request #1152: SOLR-14172: Collection metadata remains in zookeeper if too many shards requested
asfgit closed pull request #1152: SOLR-14172: Collection metadata remains in zookeeper if too many shards requested URL: https://github.com/apache/lucene-solr/pull/1152 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14172) Collection metadata remains in zookeeper if too many shards requested
[
https://issues.apache.org/jira/browse/SOLR-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021903#comment-17021903
]
ASF subversion and git services commented on SOLR-14172:
Commit 84270dc6cfae79efccea188175b07e43c6b5d5b7 in lucene-solr's branch
refs/heads/master from Shalin Shekhar Mangar
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=84270dc ]
SOLR-14172: Collection metadata remains in zookeeper if too many shards are
requested.
This also fixes a bug where an inability to assign a node based on existing
autoscaling policy resulted in a server error instead of a bad request.
This closes #1152.
> Collection metadata remains in zookeeper if too many shards requested
> -
>
> Key: SOLR-14172
> URL: https://issues.apache.org/jira/browse/SOLR-14172
> Project: Solr
> Issue Type: Bug
>Affects Versions: 8.3.1
>Reporter: Andras Salamon
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: master (9.0), 8.5
>
> Attachments: SOLR-14172.patch, SOLR-14172.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When I try to create a collection and request too many shards, collection
> creation fails with the expected error message:
> {noformat}
> $ curl -i --retry 5 -s -L -k --negotiate -u :
> 'http://asalamon-cdpd-rebase831-a-1.vpc.cloudera.com:8983/solr/admin/collections?action=CREATE&name=TooManyShardstest1&numShards=4&collection.configName=zk_init_too&maxShardsPerNode=1'
> HTTP/1.1 400 Bad Request
> Content-Type: application/json;charset=utf-8
> Content-Length: 1562
> {
> "responseHeader":{
> "status":400,
> "QTime":122},
> "Operation create caused
> exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> Cannot create collection TooManyShardstest1. Value of maxShardsPerNode is 1,
> and the number of nodes currently live or live and part of your createNodeSet
> is 3. This allows a maximum of 3 to be created. Value of numShards is 4,
> value of nrtReplicas is 1, value of tlogReplicas is 0 and value of
> pullReplicas is 0. This requires 4 shards to be created (higher than the
> allowed number)",
> "exception":{
> "msg":"Cannot create collection TooManyShardstest1. Value of
> maxShardsPerNode is 1, and the number of nodes currently live or live and
> part of your createNodeSet is 3. This allows a maximum of 3 to be created.
> Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is
> 0 and value of pullReplicas is 0. This requires 4 shards to be created
> (higher than the allowed number)",
> "rspCode":400},
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"Cannot create collection TooManyShardstest1. Value of
> maxShardsPerNode is 1, and the number of nodes currently live or live and
> part of your createNodeSet is 3. This allows a maximum of 3 to be created.
> Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is
> 0 and value of pullReplicas is 0. This requires 4 shards to be created
> (higher than the allowed number)",
> "code":400}}
> {noformat}
> Although the collection creation was not successful if I list the collections
> it shows the new collection:
> {noformat}
> $ solr collection --list
> TooManyShardstest1 (1)
> {noformat}
> Looks like metadata remains in Zookeeper:
> {noformat}
> $ zkcli.sh -zkhost asalamon-cdpd-rebase831-a-1.vpc.cloudera.com:2181/solr
> -cmd ls /collections
> INFO - 2020-01-06 04:54:01.851;
> org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect
> to ZooKeeper
> INFO - 2020-01-06 04:54:01.880;
> org.apache.solr.common.cloud.ConnectionManager; zkClient has connected
> INFO - 2020-01-06 04:54:01.881;
> org.apache.solr.common.cloud.ConnectionManager; Client is connected to
> ZooKeeper
> /collections (1)
> /collections/TooManyShardstest1 (1)
> DATA:
> {"configName":"zk_init_too"}
> /collections/TooManyShardstest1/state.json (0)
> DATA:
> {"TooManyShardstest1":{
> "pullReplicas":"0",
> "replicationFactor":"1",
> "router":{"name":"compositeId"},
> "maxShardsPerNode":"1",
> "autoAddReplicas":"false",
> "nrtReplicas":"1",
> "tlogReplicas":"0",
> "shards":{
> "shard1":{
> "range":"8000-bfff",
> "state":"active",
> "replicas":{}},
> "shard2":{
> "range":"c000-",
> "state":"active",
> "replicas":{}},
> "shard3":{
>
[jira] [Commented] (SOLR-14172) Collection metadata remains in zookeeper if too many shards requested
[
https://issues.apache.org/jira/browse/SOLR-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021904#comment-17021904
]
ASF subversion and git services commented on SOLR-14172:
Commit 6458d4f63e38dfb37356b29c5bb8a72f6ce73398 in lucene-solr's branch
refs/heads/branch_8x from Shalin Shekhar Mangar
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6458d4f ]
SOLR-14172: Collection metadata remains in zookeeper if too many shards are
requested.
This also fixes a bug where an inability to assign a node based on existing
autoscaling policy resulted in a server error instead of a bad request.
This closes #1152.
(cherry picked from commit 84270dc6cfae79efccea188175b07e43c6b5d5b7)
> Collection metadata remains in zookeeper if too many shards requested
> -
>
> Key: SOLR-14172
> URL: https://issues.apache.org/jira/browse/SOLR-14172
> Project: Solr
> Issue Type: Bug
>Affects Versions: 8.3.1
>Reporter: Andras Salamon
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: master (9.0), 8.5
>
> Attachments: SOLR-14172.patch, SOLR-14172.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When I try to create a collection and request too many shards, collection
> creation fails with the expected error message:
> {noformat}
> $ curl -i --retry 5 -s -L -k --negotiate -u :
> 'http://asalamon-cdpd-rebase831-a-1.vpc.cloudera.com:8983/solr/admin/collections?action=CREATE&name=TooManyShardstest1&numShards=4&collection.configName=zk_init_too&maxShardsPerNode=1'
> HTTP/1.1 400 Bad Request
> Content-Type: application/json;charset=utf-8
> Content-Length: 1562
> {
> "responseHeader":{
> "status":400,
> "QTime":122},
> "Operation create caused
> exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> Cannot create collection TooManyShardstest1. Value of maxShardsPerNode is 1,
> and the number of nodes currently live or live and part of your createNodeSet
> is 3. This allows a maximum of 3 to be created. Value of numShards is 4,
> value of nrtReplicas is 1, value of tlogReplicas is 0 and value of
> pullReplicas is 0. This requires 4 shards to be created (higher than the
> allowed number)",
> "exception":{
> "msg":"Cannot create collection TooManyShardstest1. Value of
> maxShardsPerNode is 1, and the number of nodes currently live or live and
> part of your createNodeSet is 3. This allows a maximum of 3 to be created.
> Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is
> 0 and value of pullReplicas is 0. This requires 4 shards to be created
> (higher than the allowed number)",
> "rspCode":400},
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"Cannot create collection TooManyShardstest1. Value of
> maxShardsPerNode is 1, and the number of nodes currently live or live and
> part of your createNodeSet is 3. This allows a maximum of 3 to be created.
> Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is
> 0 and value of pullReplicas is 0. This requires 4 shards to be created
> (higher than the allowed number)",
> "code":400}}
> {noformat}
> Although the collection creation was not successful if I list the collections
> it shows the new collection:
> {noformat}
> $ solr collection --list
> TooManyShardstest1 (1)
> {noformat}
> Looks like metadata remains in Zookeeper:
> {noformat}
> $ zkcli.sh -zkhost asalamon-cdpd-rebase831-a-1.vpc.cloudera.com:2181/solr
> -cmd ls /collections
> INFO - 2020-01-06 04:54:01.851;
> org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect
> to ZooKeeper
> INFO - 2020-01-06 04:54:01.880;
> org.apache.solr.common.cloud.ConnectionManager; zkClient has connected
> INFO - 2020-01-06 04:54:01.881;
> org.apache.solr.common.cloud.ConnectionManager; Client is connected to
> ZooKeeper
> /collections (1)
> /collections/TooManyShardstest1 (1)
> DATA:
> {"configName":"zk_init_too"}
> /collections/TooManyShardstest1/state.json (0)
> DATA:
> {"TooManyShardstest1":{
> "pullReplicas":"0",
> "replicationFactor":"1",
> "router":{"name":"compositeId"},
> "maxShardsPerNode":"1",
> "autoAddReplicas":"false",
> "nrtReplicas":"1",
> "tlogReplicas":"0",
> "shards":{
> "shard1":{
> "range":"8000-bfff",
> "state":"active",
> "replicas":{}},
> "shard2":{
> "range":"c000-",
> "state":"
[jira] [Resolved] (SOLR-14172) Collection metadata remains in zookeeper if too many shards requested
[
https://issues.apache.org/jira/browse/SOLR-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shalin Shekhar Mangar resolved SOLR-14172.
--
Resolution: Fixed
Thanks Andras for the PR and Kevin for his review!
> Collection metadata remains in zookeeper if too many shards requested
> -
>
> Key: SOLR-14172
> URL: https://issues.apache.org/jira/browse/SOLR-14172
> Project: Solr
> Issue Type: Bug
>Affects Versions: 8.3.1
>Reporter: Andras Salamon
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: master (9.0), 8.5
>
> Attachments: SOLR-14172.patch, SOLR-14172.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When I try to create a collection and request too many shards, collection
> creation fails with the expected error message:
> {noformat}
> $ curl -i --retry 5 -s -L -k --negotiate -u :
> 'http://asalamon-cdpd-rebase831-a-1.vpc.cloudera.com:8983/solr/admin/collections?action=CREATE&name=TooManyShardstest1&numShards=4&collection.configName=zk_init_too&maxShardsPerNode=1'
> HTTP/1.1 400 Bad Request
> Content-Type: application/json;charset=utf-8
> Content-Length: 1562
> {
> "responseHeader":{
> "status":400,
> "QTime":122},
> "Operation create caused
> exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> Cannot create collection TooManyShardstest1. Value of maxShardsPerNode is 1,
> and the number of nodes currently live or live and part of your createNodeSet
> is 3. This allows a maximum of 3 to be created. Value of numShards is 4,
> value of nrtReplicas is 1, value of tlogReplicas is 0 and value of
> pullReplicas is 0. This requires 4 shards to be created (higher than the
> allowed number)",
> "exception":{
> "msg":"Cannot create collection TooManyShardstest1. Value of
> maxShardsPerNode is 1, and the number of nodes currently live or live and
> part of your createNodeSet is 3. This allows a maximum of 3 to be created.
> Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is
> 0 and value of pullReplicas is 0. This requires 4 shards to be created
> (higher than the allowed number)",
> "rspCode":400},
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"Cannot create collection TooManyShardstest1. Value of
> maxShardsPerNode is 1, and the number of nodes currently live or live and
> part of your createNodeSet is 3. This allows a maximum of 3 to be created.
> Value of numShards is 4, value of nrtReplicas is 1, value of tlogReplicas is
> 0 and value of pullReplicas is 0. This requires 4 shards to be created
> (higher than the allowed number)",
> "code":400}}
> {noformat}
> Although the collection creation was not successful if I list the collections
> it shows the new collection:
> {noformat}
> $ solr collection --list
> TooManyShardstest1 (1)
> {noformat}
> Looks like metadata remains in Zookeeper:
> {noformat}
> $ zkcli.sh -zkhost asalamon-cdpd-rebase831-a-1.vpc.cloudera.com:2181/solr
> -cmd ls /collections
> INFO - 2020-01-06 04:54:01.851;
> org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect
> to ZooKeeper
> INFO - 2020-01-06 04:54:01.880;
> org.apache.solr.common.cloud.ConnectionManager; zkClient has connected
> INFO - 2020-01-06 04:54:01.881;
> org.apache.solr.common.cloud.ConnectionManager; Client is connected to
> ZooKeeper
> /collections (1)
> /collections/TooManyShardstest1 (1)
> DATA:
> {"configName":"zk_init_too"}
> /collections/TooManyShardstest1/state.json (0)
> DATA:
> {"TooManyShardstest1":{
> "pullReplicas":"0",
> "replicationFactor":"1",
> "router":{"name":"compositeId"},
> "maxShardsPerNode":"1",
> "autoAddReplicas":"false",
> "nrtReplicas":"1",
> "tlogReplicas":"0",
> "shards":{
> "shard1":{
> "range":"8000-bfff",
> "state":"active",
> "replicas":{}},
> "shard2":{
> "range":"c000-",
> "state":"active",
> "replicas":{}},
> "shard3":{
> "range":"0-3fff",
> "state":"active",
> "replicas":{}},
> "shard4":{
> "range":"4000-7fff",
> "state":"active",
> "replicas":{}
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
Fo
[jira] [Commented] (SOLR-13897) Unsafe publication of Terms object in ZkShardTerms
[ https://issues.apache.org/jira/browse/SOLR-13897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021920#comment-17021920 ] Shalin Shekhar Mangar commented on SOLR-13897: -- Updated patch so that it applies on master. > Unsafe publication of Terms object in ZkShardTerms > -- > > Key: SOLR-13897 > URL: https://issues.apache.org/jira/browse/SOLR-13897 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Affects Versions: 8.2, 8.3 >Reporter: Shalin Shekhar Mangar >Assignee: Shalin Shekhar Mangar >Priority: Major > Fix For: master (9.0) > > Attachments: SOLR-13897.patch, SOLR-13897.patch, SOLR-13897.patch, > SOLR-13897.patch > > > The Terms object in ZkShardTerms is written using a write lock but reading is > allowed freely. This is not safe and can cause visibility issues and > associated race conditions under contention. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12238) Synonym Query Style Boost By Payload
[ https://issues.apache.org/jira/browse/SOLR-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021919#comment-17021919 ] Alessandro Benedetti commented on SOLR-12238: - Hi [~dsmiley], you are right. The github pull request was linked in the Jira, I link it here: [https://github.com/apache/lucene-solr/pull/357/files|https://github.com/apache/lucene-solr/pull/357/files] Do you want me to split the functionality in two Jiras (and split the code changes in two) or just migrate this to Lucene? It has been a while, so I'll need to catch up with it, I am not sure the Lucene modifications are useful on their own. What's the most common approach when the Lucene modifications are pre-requisite specifically for a Solr feature? > Synonym Query Style Boost By Payload > > > Key: SOLR-12238 > URL: https://issues.apache.org/jira/browse/SOLR-12238 > Project: Solr > Issue Type: Improvement > Components: query parsers >Affects Versions: 7.2 >Reporter: Alessandro Benedetti >Priority: Major > Attachments: SOLR-12238.patch, SOLR-12238.patch, SOLR-12238.patch, > SOLR-12238.patch > > Time Spent: 10m > Remaining Estimate: 0h > > This improvement is built on top of the Synonym Query Style feature and > brings the possibility of boosting synonym queries using the payload > associated. > It introduces two new modalities for the Synonym Query Style : > PICK_BEST_BOOST_BY_PAYLOAD -> build a Disjunction query with the clauses > boosted by payload > AS_DISTINCT_TERMS_BOOST_BY_PAYLOAD -> build a Boolean query with the clauses > boosted by payload > This new synonym query styles will assume payloads are available so they must > be used in conjunction with a token filter able to produce payloads. > An synonym.txt example could be : > # Synonyms used by Payload Boost > tiger => tiger|1.0, Big_Cat|0.8, Shere_Khan|0.9 > leopard => leopard, Big_Cat|0.8, Bagheera|0.9 > lion => lion|1.0, panthera leo|0.99, Simba|0.8 > snow_leopard => panthera uncia|0.99, snow leopard|1.0 > A simple token filter to populate the payloads from such synonym.txt is : > delimiter="|"/> -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13897) Unsafe publication of Terms object in ZkShardTerms
[ https://issues.apache.org/jira/browse/SOLR-13897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-13897: - Attachment: SOLR-13897.patch > Unsafe publication of Terms object in ZkShardTerms > -- > > Key: SOLR-13897 > URL: https://issues.apache.org/jira/browse/SOLR-13897 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Affects Versions: 8.2, 8.3 >Reporter: Shalin Shekhar Mangar >Assignee: Shalin Shekhar Mangar >Priority: Major > Fix For: master (9.0) > > Attachments: SOLR-13897.patch, SOLR-13897.patch, SOLR-13897.patch, > SOLR-13897.patch > > > The Terms object in ZkShardTerms is written using a write lock but reading is > allowed freely. This is not safe and can cause visibility issues and > associated race conditions under contention. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-10305) uniqueKey field store=false will throw null NullPointerException
[ https://issues.apache.org/jira/browse/SOLR-10305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Khludnev updated SOLR-10305: Summary: uniqueKey field store=false will throw null NullPointerException (was: uniqueKey filed store=false will throw null NullPointerException) > uniqueKey field store=false will throw null NullPointerException > > > Key: SOLR-10305 > URL: https://issues.apache.org/jira/browse/SOLR-10305 > Project: Solr > Issue Type: Bug >Affects Versions: 5.3.1 >Reporter: jin jing >Priority: Major > > i found if set uniqueKey store=false ,when insert some index,and query as *:* > will throw nullPointer: > java.lang.NullPointerException > at > org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:1095) > at > org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:754) > at > org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:733) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13579) Create resource management API
[ https://issues.apache.org/jira/browse/SOLR-13579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021943#comment-17021943 ] Noble Paul commented on SOLR-13579: --- Public touch points are * configuration, such as solrconfig.xml or files/nodes in ZK * remote APIs * files stored in the file system etc * CLI commands * system properties during startup > Create resource management API > -- > > Key: SOLR-13579 > URL: https://issues.apache.org/jira/browse/SOLR-13579 > Project: Solr > Issue Type: New Feature >Reporter: Andrzej Bialecki >Assignee: Andrzej Bialecki >Priority: Major > Attachments: SOLR-13579.patch, SOLR-13579.patch, SOLR-13579.patch, > SOLR-13579.patch, SOLR-13579.patch, SOLR-13579.patch, SOLR-13579.patch, > SOLR-13579.patch, SOLR-13579.patch, SOLR-13579.patch > > Time Spent: 3h > Remaining Estimate: 0h > > Resource management framework API supporting the goals outlined in SOLR-13578. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13579) Create resource management API
[ https://issues.apache.org/jira/browse/SOLR-13579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021943#comment-17021943 ] Noble Paul edited comment on SOLR-13579 at 1/23/20 10:39 AM: - Public touch points are * configuration, such as solrconfig.xml or files/nodes in ZK * remote APIs (read/write) * files stored in the file system etc * CLI commands * system properties during startup was (Author: noble.paul): Public touch points are * configuration, such as solrconfig.xml or files/nodes in ZK * remote APIs * files stored in the file system etc * CLI commands * system properties during startup > Create resource management API > -- > > Key: SOLR-13579 > URL: https://issues.apache.org/jira/browse/SOLR-13579 > Project: Solr > Issue Type: New Feature >Reporter: Andrzej Bialecki >Assignee: Andrzej Bialecki >Priority: Major > Attachments: SOLR-13579.patch, SOLR-13579.patch, SOLR-13579.patch, > SOLR-13579.patch, SOLR-13579.patch, SOLR-13579.patch, SOLR-13579.patch, > SOLR-13579.patch, SOLR-13579.patch, SOLR-13579.patch > > Time Spent: 3h > Remaining Estimate: 0h > > Resource management framework API supporting the goals outlined in SOLR-13578. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-11207) Add OWASP dependency checker to detect security vulnerabilities in third party libraries
[ https://issues.apache.org/jira/browse/SOLR-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021946#comment-17021946 ] Jan Høydahl commented on SOLR-11207: See updated PR, I think this is ready for merge. Still an opt-in gradle task that is not executed as part of default checks, which gives a soft introduction of the tool. We can enable it for all once we have removed false positives and upgraded the real problems. > Add OWASP dependency checker to detect security vulnerabilities in third > party libraries > > > Key: SOLR-11207 > URL: https://issues.apache.org/jira/browse/SOLR-11207 > Project: Solr > Issue Type: Improvement > Components: Build >Affects Versions: 6.0 >Reporter: Hrishikesh Gadre >Assignee: Jan Høydahl >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > > Lucene/Solr project depends on number of third party libraries. Some of those > libraries contain security vulnerabilities. Upgrading to versions of those > libraries that have fixes for those vulnerabilities is a simple, critical > step we can take to improve the security of the system. But for that we need > a tool which can scan the Lucene/Solr dependencies and look up the security > database for known vulnerabilities. > I found that [OWASP > dependency-checker|https://jeremylong.github.io/DependencyCheck/dependency-check-ant/] > can be used for this purpose. It provides a ant task which we can include in > the Lucene/Solr build. We also need to figure out how (and when) to invoke > this dependency-checker. But this can be figured out once we complete the > first step of integrating this tool with the Lucene/Solr build system. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9153) Lucene Query parser append space if query length is greater than 255
[ https://issues.apache.org/jira/browse/LUCENE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021984#comment-17021984 ] Akanksha Jain commented on LUCENE-9153: --- Hello [~romseygeek] & [romseygeek|https://github.com/apache/lucene-solr/commits?author=romseygeek] Can these changes be backported to Lucene 4.7.1? It will make our code robust and maintainable. > Lucene Query parser append space if query length is greater than 255 > > > Key: LUCENE-9153 > URL: https://issues.apache.org/jira/browse/LUCENE-9153 > Project: Lucene - Core > Issue Type: Bug >Reporter: Akanksha Jain >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Hello Everyone > > I am working with Lucene 4.7.1 > When parsing query using Lucene query parser. If query length is greater than > 255 bytes, it returns query with space appended after every 255 bytes, which > is causing further issues in my project. > > Can you please let me know why the term (parsed query contain > Arraylist) max length is 255 bytes. Why space is appended in between > the query? > > I will really appreciate it if someone can help me with this. > Do let me know if you have not understood my query and require some reference > > For analysis, Please check QueryBuilder.java class which has method > createFieldQuery() -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9153) Lucene Query parser append space if query length is greater than 255
[ https://issues.apache.org/jira/browse/LUCENE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021992#comment-17021992 ] Alan Woodward commented on LUCENE-9153: --- 4.7 isn't being maintained anymore, I'm afraid - I'll backport to 8.5, which will be the next release. You should be able to build your own version of 4.7.1 with this change if you need it though. > Lucene Query parser append space if query length is greater than 255 > > > Key: LUCENE-9153 > URL: https://issues.apache.org/jira/browse/LUCENE-9153 > Project: Lucene - Core > Issue Type: Bug >Reporter: Akanksha Jain >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Hello Everyone > > I am working with Lucene 4.7.1 > When parsing query using Lucene query parser. If query length is greater than > 255 bytes, it returns query with space appended after every 255 bytes, which > is causing further issues in my project. > > Can you please let me know why the term (parsed query contain > Arraylist) max length is 255 bytes. Why space is appended in between > the query? > > I will really appreciate it if someone can help me with this. > Do let me know if you have not understood my query and require some reference > > For analysis, Please check QueryBuilder.java class which has method > createFieldQuery() -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9160) override heap / jvm params for tests in gradle build
[
https://issues.apache.org/jira/browse/LUCENE-9160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022083#comment-17022083
]
Robert Muir commented on LUCENE-9160:
-
My guess is there is only really 8. Impossible to tell inside a VM :) I will
open an issue, the gradle build divides number of cpus by 2, then artifically
caps this at 4, I think we should change that. Divide by 2 is fine, but
machines have more cores these days. 8 would have been a better default here.
> override heap / jvm params for tests in gradle build
>
>
> Key: LUCENE-9160
> URL: https://issues.apache.org/jira/browse/LUCENE-9160
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Robert Muir
>Priority: Major
> Fix For: master (9.0)
>
> Attachments: LUCENE-9160.patch, LUCENE-9160.patch
>
>
> Currently the gradle.properties that is generated lets you control the heap
> and flags for the gradle build jvms.
> But there is no way to control these flags for the actual forked VMs running
> the unit tests. For example, minHeap is hardcoded at 256m and maxHeap at
> 512m.
> I would like to change minHeap to 512m as well, for a fixed heap, and set
> some other jvm flags, such as {{-XX:+UseParallelGC}} so that my tests are not
> slow for silly reasons :)
> I think it is stuff jenkins CI would need as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9165) change generate-defaults.gradle not to cap testsJvms at 4
Robert Muir created LUCENE-9165:
---
Summary: change generate-defaults.gradle not to cap testsJvms at 4
Key: LUCENE-9165
URL: https://issues.apache.org/jira/browse/LUCENE-9165
Project: Lucene - Core
Issue Type: Improvement
Reporter: Robert Muir
{code}
def cpus = Runtime.runtime.availableProcessors()
def testsJvms = (int) Math.max(1d, Math.min(cpus * 0.5d, 4))
{code}
Dividing by 2 is still good (hyperthreading is still a thing), but the
artificial cap of 4 is a bottleneck if the hardware has more available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9165) change generate-defaults.gradle not to cap testsJvms at 4
[
https://issues.apache.org/jira/browse/LUCENE-9165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022112#comment-17022112
]
David Smiley commented on LUCENE-9165:
--
+1 !
> change generate-defaults.gradle not to cap testsJvms at 4
> -
>
> Key: LUCENE-9165
> URL: https://issues.apache.org/jira/browse/LUCENE-9165
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Robert Muir
>Priority: Major
>
> {code}
> def cpus = Runtime.runtime.availableProcessors()
> def testsJvms = (int) Math.max(1d, Math.min(cpus * 0.5d, 4))
> {code}
> Dividing by 2 is still good (hyperthreading is still a thing), but the
> artificial cap of 4 is a bottleneck if the hardware has more available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14159) Fix errors in TestCloudConsistency
[ https://issues.apache.org/jira/browse/SOLR-14159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson updated SOLR-14159: -- Attachment: stdout stdout Status: Open (was: Open) [~hossman] I had 2 failures out of 1,000 last nigh. One is "ADDREPLICA failed to create replica" and the other is "Expected mime type application/octet-stream but got text/html. 404 Not Found404 Not Found" They may well be "something else", attached for your perusal. > Fix errors in TestCloudConsistency > -- > > Key: SOLR-14159 > URL: https://issues.apache.org/jira/browse/SOLR-14159 > Project: Solr > Issue Type: Bug >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: FailWaitingForCollection_WithHossFix, FailWithHossFix, > SOLR-14159_debug.patch, SOLR-14159_proxy_logging.patch, > SOLR-14159_waitFor_testfix.patch, SOLR-14159_waitFor_testfix.patch, > WithHossFix.patch, stdout, stdout, stdout > > > Moving over here from SOLR-13486 as per Hoss. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14196) AdminUI login not working for JWTAuth when blockUnknown=false
[
https://issues.apache.org/jira/browse/SOLR-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022119#comment-17022119
]
ASF subversion and git services commented on SOLR-14196:
Commit e744f7977e87462d47d7bead3c767824bd655236 in lucene-solr's branch
refs/heads/master from Jan Høydahl
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e744f79 ]
SOLR-14196: AdminUI login not working for JWTAuth when blockUnknown=false
(#1190)
> AdminUI login not working for JWTAuth when blockUnknown=false
> -
>
> Key: SOLR-14196
> URL: https://issues.apache.org/jira/browse/SOLR-14196
> Project: Solr
> Issue Type: Bug
> Components: Admin UI
>Affects Versions: 8.1, 8.2, 8.3, 8.4, 8.4.1
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When {{blockUnknown=false}} it is not the AuthenticationPlugin that sends the
> HTTP response header {{WWW-Authenticate}}, but it is done by {{HttpSolrCall}}
> based on a 401 response from AuthorizationPlugin.
> Admin UI uses info from {{WWW-Authenticate}} header to choose what login
> screen to display, and for JWT it also requires an {{X-Solr-Authinfo}} header
> generated by the JWT plugin.
> h3. Solution
> The BasicAuth plugin solves this by storing the map of headers to set as an
> attribute on ServletReuest with key {{AuthenticationPlugin}}, if we fall
> through to Authorization plugin. We can do that as well with JWT plugin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] janhoy merged pull request #1190: SOLR-14196: AdminUI login not working for JWTAuth when blockUnknown false
janhoy merged pull request #1190: SOLR-14196: AdminUI login not working for JWTAuth when blockUnknown false URL: https://github.com/apache/lucene-solr/pull/1190 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14196) AdminUI login not working for JWTAuth when blockUnknown=false
[
https://issues.apache.org/jira/browse/SOLR-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Høydahl updated SOLR-14196:
---
Fix Version/s: 8.5
> AdminUI login not working for JWTAuth when blockUnknown=false
> -
>
> Key: SOLR-14196
> URL: https://issues.apache.org/jira/browse/SOLR-14196
> Project: Solr
> Issue Type: Bug
> Components: Admin UI
>Affects Versions: 8.1, 8.2, 8.3, 8.4, 8.4.1
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Fix For: 8.5
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When {{blockUnknown=false}} it is not the AuthenticationPlugin that sends the
> HTTP response header {{WWW-Authenticate}}, but it is done by {{HttpSolrCall}}
> based on a 401 response from AuthorizationPlugin.
> Admin UI uses info from {{WWW-Authenticate}} header to choose what login
> screen to display, and for JWT it also requires an {{X-Solr-Authinfo}} header
> generated by the JWT plugin.
> h3. Solution
> The BasicAuth plugin solves this by storing the map of headers to set as an
> attribute on ServletReuest with key {{AuthenticationPlugin}}, if we fall
> through to Authorization plugin. We can do that as well with JWT plugin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14196) AdminUI login not working for JWTAuth when blockUnknown=false
[
https://issues.apache.org/jira/browse/SOLR-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Høydahl resolved SOLR-14196.
Resolution: Fixed
> AdminUI login not working for JWTAuth when blockUnknown=false
> -
>
> Key: SOLR-14196
> URL: https://issues.apache.org/jira/browse/SOLR-14196
> Project: Solr
> Issue Type: Bug
> Components: Admin UI
>Affects Versions: 8.1, 8.2, 8.3, 8.4, 8.4.1
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Fix For: 8.5
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When {{blockUnknown=false}} it is not the AuthenticationPlugin that sends the
> HTTP response header {{WWW-Authenticate}}, but it is done by {{HttpSolrCall}}
> based on a 401 response from AuthorizationPlugin.
> Admin UI uses info from {{WWW-Authenticate}} header to choose what login
> screen to display, and for JWT it also requires an {{X-Solr-Authinfo}} header
> generated by the JWT plugin.
> h3. Solution
> The BasicAuth plugin solves this by storing the map of headers to set as an
> attribute on ServletReuest with key {{AuthenticationPlugin}}, if we fall
> through to Authorization plugin. We can do that as well with JWT plugin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14196) AdminUI login not working for JWTAuth when blockUnknown=false
[
https://issues.apache.org/jira/browse/SOLR-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022123#comment-17022123
]
ASF subversion and git services commented on SOLR-14196:
Commit afdbd9e8d56fa491d990f55baea9188b3f3dba95 in lucene-solr's branch
refs/heads/branch_8x from Jan Høydahl
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=afdbd9e ]
SOLR-14196: AdminUI login not working for JWTAuth when blockUnknown=false
(#1190)
(cherry picked from commit e744f7977e87462d47d7bead3c767824bd655236)
> AdminUI login not working for JWTAuth when blockUnknown=false
> -
>
> Key: SOLR-14196
> URL: https://issues.apache.org/jira/browse/SOLR-14196
> Project: Solr
> Issue Type: Bug
> Components: Admin UI
>Affects Versions: 8.1, 8.2, 8.3, 8.4, 8.4.1
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Fix For: 8.5
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When {{blockUnknown=false}} it is not the AuthenticationPlugin that sends the
> HTTP response header {{WWW-Authenticate}}, but it is done by {{HttpSolrCall}}
> based on a 401 response from AuthorizationPlugin.
> Admin UI uses info from {{WWW-Authenticate}} header to choose what login
> screen to display, and for JWT it also requires an {{X-Solr-Authinfo}} header
> generated by the JWT plugin.
> h3. Solution
> The BasicAuth plugin solves this by storing the map of headers to set as an
> attribute on ServletReuest with key {{AuthenticationPlugin}}, if we fall
> through to Authorization plugin. We can do that as well with JWT plugin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Assigned] (SOLR-14205) ConnectionImpl.isValid() does not behave as described in Connection javadocs
[ https://issues.apache.org/jira/browse/SOLR-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden reassigned SOLR-14205: --- Assignee: Kevin Risden > ConnectionImpl.isValid() does not behave as described in Connection javadocs > > > Key: SOLR-14205 > URL: https://issues.apache.org/jira/browse/SOLR-14205 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Parallel SQL >Affects Versions: 8.4.1 >Reporter: Nick Vercammen >Assignee: Kevin Risden >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > The ConnectionImpl.isValid() returns false when you pass 0 as the timeout. > According to the javadocs > ([https://docs.oracle.com/en/java/javase/11/docs/api/java.sql/java/sql/Connection.html#isValid(int)]) > a timeout = 0 means no timeout. In the current implementation a timeout = 0 > means that the connection is always invalid. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14205) ConnectionImpl.isValid() does not behave as described in Connection javadocs
[ https://issues.apache.org/jira/browse/SOLR-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated SOLR-14205: Fix Version/s: 8.5 > ConnectionImpl.isValid() does not behave as described in Connection javadocs > > > Key: SOLR-14205 > URL: https://issues.apache.org/jira/browse/SOLR-14205 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Parallel SQL >Affects Versions: 8.4.1 >Reporter: Nick Vercammen >Assignee: Kevin Risden >Priority: Major > Fix For: 8.5 > > Time Spent: 10m > Remaining Estimate: 0h > > The ConnectionImpl.isValid() returns false when you pass 0 as the timeout. > According to the javadocs > ([https://docs.oracle.com/en/java/javase/11/docs/api/java.sql/java/sql/Connection.html#isValid(int)]) > a timeout = 0 means no timeout. In the current implementation a timeout = 0 > means that the connection is always invalid. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14132) Upgrade Angular JS 1.3.8 to 1.7.9
[ https://issues.apache.org/jira/browse/SOLR-14132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated SOLR-14132: Fix Version/s: 8.5 > Upgrade Angular JS 1.3.8 to 1.7.9 > - > > Key: SOLR-14132 > URL: https://issues.apache.org/jira/browse/SOLR-14132 > Project: Solr > Issue Type: Improvement > Components: Admin UI >Reporter: Robert Muir >Assignee: Kevin Risden >Priority: Major > Fix For: 8.5 > > Attachments: SOLR-14132.patch, SOLR-14132.patch, Screen Shot > 2019-12-14 at 10.35.19 AM.png > > Time Spent: 50m > Remaining Estimate: 0h > > I tried it just in case it is an easy win: it cannot be done without some > javascript code changes, so some javascript wrestling is needed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] risdenk closed pull request #1196: SOLR-14132: Upgrade Angular JS 1.3.8 to 1.7.9
risdenk closed pull request #1196: SOLR-14132: Upgrade Angular JS 1.3.8 to 1.7.9 URL: https://github.com/apache/lucene-solr/pull/1196 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14132) Upgrade Angular JS 1.3.8 to 1.7.9
[ https://issues.apache.org/jira/browse/SOLR-14132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022146#comment-17022146 ] ASF subversion and git services commented on SOLR-14132: Commit 9b6fc1b9fc4edfe230aa1897c66cc7e9b6bcc438 in lucene-solr's branch refs/heads/master from Kevin Risden [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=9b6fc1b ] SOLR-14132: Upgrade Angular JS 1.3.8 to 1.7.9 * Upgrade Angular JS 1.3.8 to 1.7.9 * Upgrade Angular Chosen v1.3.0 and Chosen to v1.8.7 * Remove older jquery 1.7.2 version * Remove non minified Angular JS files Closes #1196 Signed-off-by: Kevin Risden > Upgrade Angular JS 1.3.8 to 1.7.9 > - > > Key: SOLR-14132 > URL: https://issues.apache.org/jira/browse/SOLR-14132 > Project: Solr > Issue Type: Improvement > Components: Admin UI >Reporter: Robert Muir >Assignee: Kevin Risden >Priority: Major > Fix For: 8.5 > > Attachments: SOLR-14132.patch, SOLR-14132.patch, Screen Shot > 2019-12-14 at 10.35.19 AM.png > > Time Spent: 1h > Remaining Estimate: 0h > > I tried it just in case it is an easy win: it cannot be done without some > javascript code changes, so some javascript wrestling is needed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9165) change generate-defaults.gradle not to cap testsJvms at 4
[
https://issues.apache.org/jira/browse/LUCENE-9165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022147#comment-17022147
]
Erick Erickson commented on LUCENE-9165:
+1 for changing it.
Meanwhile I think you can override this (and other settings) in
~/.gradle/gradle.properties without polluting your checkout.
> change generate-defaults.gradle not to cap testsJvms at 4
> -
>
> Key: LUCENE-9165
> URL: https://issues.apache.org/jira/browse/LUCENE-9165
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Robert Muir
>Priority: Major
>
> {code}
> def cpus = Runtime.runtime.availableProcessors()
> def testsJvms = (int) Math.max(1d, Math.min(cpus * 0.5d, 4))
> {code}
> Dividing by 2 is still good (hyperthreading is still a thing), but the
> artificial cap of 4 is a bottleneck if the hardware has more available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14209) Upgrade JQuery
Kevin Risden created SOLR-14209: --- Summary: Upgrade JQuery Key: SOLR-14209 URL: https://issues.apache.org/jira/browse/SOLR-14209 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Components: Admin UI, contrib - Velocity Reporter: Kevin Risden Currently JQuery is on 2.1.3. It would be good to upgrade to the latest version if possible. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14132) Upgrade Angular JS 1.3.8 to 1.7.9
[ https://issues.apache.org/jira/browse/SOLR-14132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022149#comment-17022149 ] Kevin Risden commented on SOLR-14132: - Created SOLR-14209 for JQuery upgrade. > Upgrade Angular JS 1.3.8 to 1.7.9 > - > > Key: SOLR-14132 > URL: https://issues.apache.org/jira/browse/SOLR-14132 > Project: Solr > Issue Type: Improvement > Components: Admin UI >Reporter: Robert Muir >Assignee: Kevin Risden >Priority: Major > Fix For: 8.5 > > Attachments: SOLR-14132.patch, SOLR-14132.patch, Screen Shot > 2019-12-14 at 10.35.19 AM.png > > Time Spent: 1h > Remaining Estimate: 0h > > I tried it just in case it is an easy win: it cannot be done without some > javascript code changes, so some javascript wrestling is needed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14132) Upgrade Angular JS 1.3.8 to 1.7.9
[ https://issues.apache.org/jira/browse/SOLR-14132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated SOLR-14132: Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Angular JS 1.3.8 to 1.7.9 > - > > Key: SOLR-14132 > URL: https://issues.apache.org/jira/browse/SOLR-14132 > Project: Solr > Issue Type: Improvement > Components: Admin UI >Reporter: Robert Muir >Assignee: Kevin Risden >Priority: Major > Fix For: 8.5 > > Attachments: SOLR-14132.patch, SOLR-14132.patch, Screen Shot > 2019-12-14 at 10.35.19 AM.png > > Time Spent: 1h > Remaining Estimate: 0h > > I tried it just in case it is an easy win: it cannot be done without some > javascript code changes, so some javascript wrestling is needed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14132) Upgrade Angular JS 1.3.8 to 1.7.9
[ https://issues.apache.org/jira/browse/SOLR-14132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022178#comment-17022178 ] ASF subversion and git services commented on SOLR-14132: Commit 4dc793e648bef147fe217d3c044cdc4f52b671c6 in lucene-solr's branch refs/heads/branch_8x from Kevin Risden [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4dc793e ] SOLR-14132: Upgrade Angular JS 1.3.8 to 1.7.9 * Upgrade Angular JS 1.3.8 to 1.7.9 * Upgrade Angular Chosen v1.3.0 and Chosen to v1.8.7 * Remove older jquery 1.7.2 version * Remove non minified Angular JS files Closes #1196 Signed-off-by: Kevin Risden > Upgrade Angular JS 1.3.8 to 1.7.9 > - > > Key: SOLR-14132 > URL: https://issues.apache.org/jira/browse/SOLR-14132 > Project: Solr > Issue Type: Improvement > Components: Admin UI >Reporter: Robert Muir >Assignee: Kevin Risden >Priority: Major > Fix For: 8.5 > > Attachments: SOLR-14132.patch, SOLR-14132.patch, Screen Shot > 2019-12-14 at 10.35.19 AM.png > > Time Spent: 1h > Remaining Estimate: 0h > > I tried it just in case it is an easy win: it cannot be done without some > javascript code changes, so some javascript wrestling is needed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14205) ConnectionImpl.isValid() does not behave as described in Connection javadocs
[ https://issues.apache.org/jira/browse/SOLR-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated SOLR-14205: Issue Type: Bug (was: Improvement) > ConnectionImpl.isValid() does not behave as described in Connection javadocs > > > Key: SOLR-14205 > URL: https://issues.apache.org/jira/browse/SOLR-14205 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Parallel SQL >Affects Versions: 8.4.1 >Reporter: Nick Vercammen >Assignee: Kevin Risden >Priority: Major > Fix For: 8.5 > > Time Spent: 10m > Remaining Estimate: 0h > > The ConnectionImpl.isValid() returns false when you pass 0 as the timeout. > According to the javadocs > ([https://docs.oracle.com/en/java/javase/11/docs/api/java.sql/java/sql/Connection.html#isValid(int)]) > a timeout = 0 means no timeout. In the current implementation a timeout = 0 > means that the connection is always invalid. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14205) Do not fail when given timeout to connectionImpl.isValid() = 0
[ https://issues.apache.org/jira/browse/SOLR-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated SOLR-14205: Summary: Do not fail when given timeout to connectionImpl.isValid() = 0 (was: ConnectionImpl.isValid() does not behave as described in Connection javadocs) > Do not fail when given timeout to connectionImpl.isValid() = 0 > -- > > Key: SOLR-14205 > URL: https://issues.apache.org/jira/browse/SOLR-14205 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Parallel SQL >Affects Versions: 8.4.1 >Reporter: Nick Vercammen >Assignee: Kevin Risden >Priority: Major > Fix For: 8.5 > > Time Spent: 10m > Remaining Estimate: 0h > > The ConnectionImpl.isValid() returns false when you pass 0 as the timeout. > According to the javadocs > ([https://docs.oracle.com/en/java/javase/11/docs/api/java.sql/java/sql/Connection.html#isValid(int)]) > a timeout = 0 means no timeout. In the current implementation a timeout = 0 > means that the connection is always invalid. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] risdenk commented on issue #1204: SOLR-14205 Do not fail when given timeout to connectionImpl.isValid()…
risdenk commented on issue #1204: SOLR-14205 Do not fail when given timeout to connectionImpl.isValid()… URL: https://github.com/apache/lucene-solr/pull/1204#issuecomment-577724461 Will merge this shortly. Thanks @nvercamm This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] risdenk closed pull request #1204: SOLR-14205 Do not fail when given timeout to connectionImpl.isValid()…
risdenk closed pull request #1204: SOLR-14205 Do not fail when given timeout to connectionImpl.isValid()… URL: https://github.com/apache/lucene-solr/pull/1204 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14205) Do not fail when given timeout to connectionImpl.isValid() = 0
[ https://issues.apache.org/jira/browse/SOLR-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022188#comment-17022188 ] ASF subversion and git services commented on SOLR-14205: Commit 60a2926546d163123ee672a8b5580449bad1991d in lucene-solr's branch refs/heads/master from Nick Vercammen [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=60a2926 ] SOLR-14205 Do not fail when given timeout to connectionImpl.isValid() = 0 Closes #1204 Signed-off-by: Kevin Risden > Do not fail when given timeout to connectionImpl.isValid() = 0 > -- > > Key: SOLR-14205 > URL: https://issues.apache.org/jira/browse/SOLR-14205 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Parallel SQL >Affects Versions: 8.4.1 >Reporter: Nick Vercammen >Assignee: Kevin Risden >Priority: Major > Fix For: 8.5 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The ConnectionImpl.isValid() returns false when you pass 0 as the timeout. > According to the javadocs > ([https://docs.oracle.com/en/java/javase/11/docs/api/java.sql/java/sql/Connection.html#isValid(int)]) > a timeout = 0 means no timeout. In the current implementation a timeout = 0 > means that the connection is always invalid. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14205) Do not fail when given timeout to connectionImpl.isValid() = 0
[ https://issues.apache.org/jira/browse/SOLR-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022190#comment-17022190 ] ASF subversion and git services commented on SOLR-14205: Commit f34e1de7d85acd04ad11a4502b43c403525d5ad5 in lucene-solr's branch refs/heads/branch_8x from Nick Vercammen [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=f34e1de ] SOLR-14205 Do not fail when given timeout to connectionImpl.isValid() = 0 Closes #1204 Signed-off-by: Kevin Risden > Do not fail when given timeout to connectionImpl.isValid() = 0 > -- > > Key: SOLR-14205 > URL: https://issues.apache.org/jira/browse/SOLR-14205 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Parallel SQL >Affects Versions: 8.4.1 >Reporter: Nick Vercammen >Assignee: Kevin Risden >Priority: Major > Fix For: 8.5 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The ConnectionImpl.isValid() returns false when you pass 0 as the timeout. > According to the javadocs > ([https://docs.oracle.com/en/java/javase/11/docs/api/java.sql/java/sql/Connection.html#isValid(int)]) > a timeout = 0 means no timeout. In the current implementation a timeout = 0 > means that the connection is always invalid. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14205) Do not fail when given timeout to connectionImpl.isValid() = 0
[ https://issues.apache.org/jira/browse/SOLR-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden resolved SOLR-14205. - Resolution: Fixed > Do not fail when given timeout to connectionImpl.isValid() = 0 > -- > > Key: SOLR-14205 > URL: https://issues.apache.org/jira/browse/SOLR-14205 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Parallel SQL >Affects Versions: 8.4.1 >Reporter: Nick Vercammen >Assignee: Kevin Risden >Priority: Major > Fix For: 8.5 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The ConnectionImpl.isValid() returns false when you pass 0 as the timeout. > According to the javadocs > ([https://docs.oracle.com/en/java/javase/11/docs/api/java.sql/java/sql/Connection.html#isValid(int)]) > a timeout = 0 means no timeout. In the current implementation a timeout = 0 > means that the connection is always invalid. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14205) Do not fail when given timeout to connectionImpl.isValid() = 0
[ https://issues.apache.org/jira/browse/SOLR-14205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022191#comment-17022191 ] Kevin Risden commented on SOLR-14205: - Thanks for reporting and fixing [~vercani] > Do not fail when given timeout to connectionImpl.isValid() = 0 > -- > > Key: SOLR-14205 > URL: https://issues.apache.org/jira/browse/SOLR-14205 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Parallel SQL >Affects Versions: 8.4.1 >Reporter: Nick Vercammen >Assignee: Kevin Risden >Priority: Major > Fix For: 8.5 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The ConnectionImpl.isValid() returns false when you pass 0 as the timeout. > According to the javadocs > ([https://docs.oracle.com/en/java/javase/11/docs/api/java.sql/java/sql/Connection.html#isValid(int)]) > a timeout = 0 means no timeout. In the current implementation a timeout = 0 > means that the connection is always invalid. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Assigned] (SOLR-11554) Support handling OPTIONS request for Hadoop authentication filter
[ https://issues.apache.org/jira/browse/SOLR-11554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden reassigned SOLR-11554: --- Assignee: Kevin Risden > Support handling OPTIONS request for Hadoop authentication filter > - > > Key: SOLR-11554 > URL: https://issues.apache.org/jira/browse/SOLR-11554 > Project: Solr > Issue Type: Bug >Affects Versions: 6.4 >Reporter: Hrishikesh Gadre >Assignee: Kevin Risden >Priority: Minor > Attachments: SOLR-11554.patch > > > As part of SOLR-9513 we added a Solr authentication plugin which uses Hadoop > security framework. The HTTP client interface provided by Hadoop framework > does not send the authentication information preemptively. Instead it sends > an OPTIONS request first. If the server responds with 401 error, it resends > the request with the proper authentication information. This jira is to > handle the OPTIONS request as part of the Solr authentication plugin for > Hadoop. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-11554) Support handling OPTIONS request for Hadoop authentication filter
[ https://issues.apache.org/jira/browse/SOLR-11554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated SOLR-11554: Fix Version/s: 8.5 > Support handling OPTIONS request for Hadoop authentication filter > - > > Key: SOLR-11554 > URL: https://issues.apache.org/jira/browse/SOLR-11554 > Project: Solr > Issue Type: Bug >Affects Versions: 6.4 >Reporter: Hrishikesh Gadre >Assignee: Kevin Risden >Priority: Minor > Fix For: 8.5 > > Attachments: SOLR-11554.patch > > > As part of SOLR-9513 we added a Solr authentication plugin which uses Hadoop > security framework. The HTTP client interface provided by Hadoop framework > does not send the authentication information preemptively. Instead it sends > an OPTIONS request first. If the server responds with 401 error, it resends > the request with the proper authentication information. This jira is to > handle the OPTIONS request as part of the Solr authentication plugin for > Hadoop. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8347) BlendedInfixSuggester to handle multi term matches better
[
https://issues.apache.org/jira/browse/LUCENE-8347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1700#comment-1700
]
Alessandro Benedetti commented on LUCENE-8347:
--
[~mikemccand], [~dsmiley] I will take this back under my radar, fix the
conflicts and take a look.
LUCENE-8343 has been settled so we should be able to work on this now!
> BlendedInfixSuggester to handle multi term matches better
> -
>
> Key: LUCENE-8347
> URL: https://issues.apache.org/jira/browse/LUCENE-8347
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Affects Versions: 7.3.1
>Reporter: Alessandro Benedetti
>Priority: Major
> Attachments: LUCENE-8347.patch, LUCENE-8347.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently the blendedInfix suggester considers just the first match position
> when scoring a suggestion.
> From the lucene-dev mailing list :
> "
> If I write more than one term in the query, let's say
>
> "Mini Bar Fridge"
>
> I would expect in the results something like (note that allTermsRequired=true
> and the schema weight field always returns 1000)
>
> - *Mini Bar Fridge* something
> - *Mini Bar Fridge* something else
> - *Mini Bar* something *Fridge*
> - *Mini Bar* something else *Fridge*
> - *Mini* something *Bar Fridge*
> ...
>
> Instead I see this:
>
> - *Mini Bar* something *Fridge*
> - *Mini Bar* something else *Fridge*
> - *Mini Bar Fridge* something
> - *Mini Bar Fridge* something else
> - *Mini* something *Bar Fridge*
> ...
>
> After having a look at the suggester code
> (BlendedInfixSuggester.createCoefficient), I see that the component takes in
> account only one position, which is the lowest position (among the three
> matching terms) within the term vector ("mini" in the example above) so all
> the suggestions above have the same weight
> "
> Scope of this Jira issue is to improve the BlendedInfix to better manage
> those scenarios.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8329) Size Estimator wrongly calculate Disk space in MB
[ https://issues.apache.org/jira/browse/LUCENE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1706#comment-1706 ] Alessandro Benedetti commented on LUCENE-8329: -- Any interest in fixing this? If there is not any intention in maintaining it, should we just remove it entirely? It could be misleading for people using it... > Size Estimator wrongly calculate Disk space in MB > - > > Key: LUCENE-8329 > URL: https://issues.apache.org/jira/browse/LUCENE-8329 > Project: Lucene - Core > Issue Type: Bug > Components: general/build >Affects Versions: 7.3.1 >Reporter: Alessandro Benedetti >Priority: Minor > Attachments: LUCENE-8329.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The size estimator dev tool ( dev-tools/size-estimator-lucene-solr.xls > )currently : > * Wrongly calculates disk size in MB ( showing GB) > * Doesn't specify clearly that the space needed by the optimize is FREE space > * Avg. Document Size (KB) when they are more correctly Avg. Document Field > Size (KB) > Scope of this issue is just to fix these small mistakes. > Out of scope is any improvement to the tool ( potentially separate Jira > issues will follow) > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13892) Add postfilter support to {!join} queries
[
https://issues.apache.org/jira/browse/SOLR-13892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022248#comment-17022248
]
Jason Gerlowski commented on SOLR-13892:
The PR has all the changes I wanted to put in. The tests have been updated and
I tweaked some of the join docs in the ref-guide (mainly adding a Parameters
section).
If I don't get any feedback on the PR I'll merge it in a few days.
> Add postfilter support to {!join} queries
> -
>
> Key: SOLR-13892
> URL: https://issues.apache.org/jira/browse/SOLR-13892
> Project: Solr
> Issue Type: Improvement
> Components: query parsers
>Affects Versions: master (9.0)
>Reporter: Jason Gerlowski
>Assignee: Jason Gerlowski
>Priority: Major
> Attachments: SOLR-13892.patch, SOLR-13892.patch,
> join-increasing-from-matches-tpi.png, join-increasing-from-matches1.png
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> The JoinQParserPlugin would be a lot performant in many use-cases if it could
> operate as a post-filter, especially when doc-values for the involved fields
> are available.
> With this issue, I'd like to propose a post-filter implementation for the
> {{join}} qparser.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14210) Introduce Node-level status handler for replicas
Houston Putman created SOLR-14210:
-
Summary: Introduce Node-level status handler for replicas
Key: SOLR-14210
URL: https://issues.apache.org/jira/browse/SOLR-14210
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: master (9.0), 8.5
Reporter: Houston Putman
h2. Background
As was brought up in SOLR-13055, in order to run Solr in a more cloud-native
way, we need some additional features around node-level healthchecks.
{quote}Like in Kubernetes we need 'liveliness' and 'readiness' probe explained
in
[https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/n]
determine if a node is live and ready to serve live traffic.
{quote}
However there are issues around kubernetes managing it's own rolling restarts.
With the current healthcheck setup, it's easy to envision a scenario in which
Solr reports itself as "healthy" when all of its replicas are actually
recovering. Therefore kubernetes, seeing a healthy pod would then go and
restart the next Solr node. This can happen until all replicas are "recovering"
and none are healthy. (maybe the last one restarted will be "down", but still
there are no "active" replicas)
h2. Proposal
I propose we make an additional healthcheck handler that returns whether all
replicas hosted by that Solr node are healthy and "active". That way we will be
able to use the [default kubernetes rolling restart
logic|https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies]
with Solr.
To add on to [Jan's point
here|https://issues.apache.org/jira/browse/SOLR-13055?focusedCommentId=16716559&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16716559],
this handler should be more friendly for other Content-Types and should use
bettter HTTP response statuses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] anshumg merged pull request #1203: SOLR-14206: Annotate HttpSolrCall as thread-safe
anshumg merged pull request #1203: SOLR-14206: Annotate HttpSolrCall as thread-safe URL: https://github.com/apache/lucene-solr/pull/1203 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14206) Annotate classes for thread safety
[ https://issues.apache.org/jira/browse/SOLR-14206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022257#comment-17022257 ] ASF subversion and git services commented on SOLR-14206: Commit 3c0146196aefd7fc05429650ea205a04bbb00524 in lucene-solr's branch refs/heads/master from Anshum Gupta [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=3c01461 ] SOLR-14206: Annotate HttpSolrCall as thread-safe (#1203) * SOLR-14206: Annotate HttpSolrCall and V2HttpCall as thread-safe > Annotate classes for thread safety > -- > > Key: SOLR-14206 > URL: https://issues.apache.org/jira/browse/SOLR-14206 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Anshum Gupta >Assignee: Anshum Gupta >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > We added options to annotate classes. I'll start to annotate classes that we > know are already thread safe (or not). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14206) Annotate classes for thread safety
[ https://issues.apache.org/jira/browse/SOLR-14206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022256#comment-17022256 ] ASF subversion and git services commented on SOLR-14206: Commit 3c0146196aefd7fc05429650ea205a04bbb00524 in lucene-solr's branch refs/heads/master from Anshum Gupta [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=3c01461 ] SOLR-14206: Annotate HttpSolrCall as thread-safe (#1203) * SOLR-14206: Annotate HttpSolrCall and V2HttpCall as thread-safe > Annotate classes for thread safety > -- > > Key: SOLR-14206 > URL: https://issues.apache.org/jira/browse/SOLR-14206 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Anshum Gupta >Assignee: Anshum Gupta >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > We added options to annotate classes. I'll start to annotate classes that we > know are already thread safe (or not). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request #1200: SOLR-12045: emphasize deployment caveat.
mkhludnev commented on a change in pull request #1200: SOLR-12045: emphasize
deployment caveat.
URL: https://github.com/apache/lucene-solr/pull/1200#discussion_r370229545
##
File path: solr/solr-ref-guide/src/analytics.adoc
##
@@ -33,7 +33,7 @@ Since the Analytics framework is a _search component_, it
must be declared as su
For distributed analytics requests over cloud collections, the component uses
the `AnalyticsHandler` strictly for inter-shard communication.
The Analytics Handler should not be used by users to submit analytics requests.
-To use the Analytics Component, the first step is to install this contrib
module's plugins into Solr -- see the
<> section on how to do this.
+To use the Analytics Component, the first step is to install this contrib
module's plugins into Solr -- see the
<> section on how to do
this. Note: method with `` directories doesn't work, also consider copying
`solr-analytic-x.x.jar` to
`${solr.install.dir}/server/solr-webapp/webapp/WEB-INF/lib/`.
Review comment:
@HoustonPutman , I've applied your perfect wording, slightly fixing
reference syntax in according my understanding. Sadly I can't check it quickly
locally. I'll check Jenkins refguide build.
The next question: Do we need to put the same clarification to `/master` ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] anshumg opened a new pull request #1205: SOLR-14206: Annotate HttpSolrCall as thread-safe
anshumg opened a new pull request #1205: SOLR-14206: Annotate HttpSolrCall as thread-safe URL: https://github.com/apache/lucene-solr/pull/1205 Annotate HttpSolrCall and V2HttpCall as thread-safe This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13897) Unsafe publication of Terms object in ZkShardTerms
[
https://issues.apache.org/jira/browse/SOLR-13897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022268#comment-17022268
]
Lucene/Solr QA commented on SOLR-13897:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 0m 59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 88m 5s{color}
| {color:red} core in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 92m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.cloud.SyncSliceTest |
| | solr.schema.TestBulkSchemaConcurrent |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-13897 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12991634/SOLR-13897.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene2-us-west.apache.org 4.4.0-170-generic #199-Ubuntu SMP
Thu Nov 14 01:45:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 9b6fc1b |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 |
| Default Java | LTS |
| unit |
https://builds.apache.org/job/PreCommit-SOLR-Build/661/artifact/out/patch-unit-solr_core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/661/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/661/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Unsafe publication of Terms object in ZkShardTerms
> --
>
> Key: SOLR-13897
> URL: https://issues.apache.org/jira/browse/SOLR-13897
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
>Affects Versions: 8.2, 8.3
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: master (9.0)
>
> Attachments: SOLR-13897.patch, SOLR-13897.patch, SOLR-13897.patch,
> SOLR-13897.patch
>
>
> The Terms object in ZkShardTerms is written using a write lock but reading is
> allowed freely. This is not safe and can cause visibility issues and
> associated race conditions under contention.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14210) Introduce Node-level status handler for replicas
[
https://issues.apache.org/jira/browse/SOLR-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022279#comment-17022279
]
Jan Høydahl commented on SOLR-14210:
Can this be treated as “shallow” vs “deep” health check? I.e. the deep check
assures that requests will likely succeed and not only that the node is alive.
Could be a sub handler “node/health/deep” or similar.
Also I think that health check endpoints should be explicitly excluded from
authentication like the PKI keys endpoint.
> Introduce Node-level status handler for replicas
>
>
> Key: SOLR-14210
> URL: https://issues.apache.org/jira/browse/SOLR-14210
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: master (9.0), 8.5
>Reporter: Houston Putman
>Priority: Major
>
> h2. Background
> As was brought up in SOLR-13055, in order to run Solr in a more cloud-native
> way, we need some additional features around node-level healthchecks.
> {quote}Like in Kubernetes we need 'liveliness' and 'readiness' probe
> explained in
> [https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/n]
> determine if a node is live and ready to serve live traffic.
> {quote}
>
> However there are issues around kubernetes managing it's own rolling
> restarts. With the current healthcheck setup, it's easy to envision a
> scenario in which Solr reports itself as "healthy" when all of its replicas
> are actually recovering. Therefore kubernetes, seeing a healthy pod would
> then go and restart the next Solr node. This can happen until all replicas
> are "recovering" and none are healthy. (maybe the last one restarted will be
> "down", but still there are no "active" replicas)
> h2. Proposal
> I propose we make an additional healthcheck handler that returns whether all
> replicas hosted by that Solr node are healthy and "active". That way we will
> be able to use the [default kubernetes rolling restart
> logic|https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies]
> with Solr.
> To add on to [Jan's point
> here|https://issues.apache.org/jira/browse/SOLR-13055?focusedCommentId=16716559&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16716559],
> this handler should be more friendly for other Content-Types and should use
> bettter HTTP response statuses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14211) TestBulkSchemaConcurrent failures
Andrzej Bialecki created SOLR-14211: --- Summary: TestBulkSchemaConcurrent failures Key: SOLR-14211 URL: https://issues.apache.org/jira/browse/SOLR-14211 Project: Solr Issue Type: Bug Security Level: Public (Default Security Level. Issues are Public) Affects Versions: 8.5 Reporter: Andrzej Bialecki Assignee: Andrzej Bialecki Fix For: 8.5 This bug is caused by a recent change in SOLR-14192 - SchemaManager tries to locate in ZK a schema resource name without prepending the full configset path. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #1200: SOLR-12045: emphasize deployment caveat.
HoustonPutman commented on a change in pull request #1200: SOLR-12045:
emphasize deployment caveat.
URL: https://github.com/apache/lucene-solr/pull/1200#discussion_r370250057
##
File path: solr/solr-ref-guide/src/analytics.adoc
##
@@ -33,7 +33,7 @@ Since the Analytics framework is a _search component_, it
must be declared as su
For distributed analytics requests over cloud collections, the component uses
the `AnalyticsHandler` strictly for inter-shard communication.
The Analytics Handler should not be used by users to submit analytics requests.
-To use the Analytics Component, the first step is to install this contrib
module's plugins into Solr -- see the
<> section on how to do this.
+To use the Analytics Component, the first step is to install this contrib
module's plugins into Solr -- see the
<> section on how to do
this. Note: method with `` directories doesn't work, also consider copying
`solr-analytic-x.x.jar` to
`${solr.install.dir}/server/solr-webapp/webapp/WEB-INF/lib/`.
Review comment:
I made a small change. Also tested it locally, and it looks good. I would
port this to master as well.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] janhoy commented on issue #1121: SOLR-11207: Add OWASP dependency checker to gradle build
janhoy commented on issue #1121: SOLR-11207: Add OWASP dependency checker to gradle build URL: https://github.com/apache/lucene-solr/pull/1121#issuecomment-577786859 We could have a validation.owasp.threshold as alternative to the “fail” prop, to decide on what CVSS score to fail the build. Now it is hard coded to 7. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14212) " plugins with packages" seems incomplete
Christine Poerschke created SOLR-14212: -- Summary: " plugins with packages" seems incomplete Key: SOLR-14212 URL: https://issues.apache.org/jira/browse/SOLR-14212 Project: Solr Issue Type: Sub-task Affects Versions: 8.5 Reporter: Christine Poerschke SOLR-14125 follow-on, details to follow. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14212) " plugins with packages" seems incomplete
[
https://issues.apache.org/jira/browse/SOLR-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022301#comment-17022301
]
Christine Poerschke commented on SOLR-14212:
[~mdrob] noticed in [~epugh]'s https://github.com/apache/lucene-solr/pull/1033
for SOLR-13965 a suspected bug w.r.t. {{SolrConfig.classVsSolrPluginInfo}} use
in GraphHandler and StreamHander.
{code}
- SolrConfig.classVsSolrPluginInfo.get(Expressible.class)
+ SolrConfig.classVsSolrPluginInfo.get(Expressible.class.getName())
{code}
seems an apparent fix. Looking into further today though I noticed that there
is test coverage for {{expressible}} in
https://github.com/apache/lucene-solr/blob/master/solr/core/src/test/org/apache/solr/pkg/TestPackages.java
but the tests seem to pass without this fix; although there also is some
commented out expressible-related code.
Attaching illustrative patch (the commenting-in commented-out test part of
which does not compile).
> " plugins with packages" seems incomplete
> --
>
> Key: SOLR-14212
> URL: https://issues.apache.org/jira/browse/SOLR-14212
> Project: Solr
> Issue Type: Sub-task
>Affects Versions: 8.5
>Reporter: Christine Poerschke
>Priority: Major
>
> SOLR-14125 follow-on, details to follow.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14212) " plugins with packages" seems incomplete
[ https://issues.apache.org/jira/browse/SOLR-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christine Poerschke updated SOLR-14212: --- Attachment: SOLR-14212.patch > " plugins with packages" seems incomplete > -- > > Key: SOLR-14212 > URL: https://issues.apache.org/jira/browse/SOLR-14212 > Project: Solr > Issue Type: Sub-task >Affects Versions: 8.5 >Reporter: Christine Poerschke >Priority: Major > Attachments: SOLR-14212.patch > > > SOLR-14125 follow-on, details to follow. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14125) SOLR-14125: Make plugins work with packages
[ https://issues.apache.org/jira/browse/SOLR-14125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022305#comment-17022305 ] Christine Poerschke commented on SOLR-14125: Created SOLR-14212 follow-on ticket. > SOLR-14125: Make plugins work with packages > -- > > Key: SOLR-14125 > URL: https://issues.apache.org/jira/browse/SOLR-14125 > Project: Solr > Issue Type: Sub-task >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Labels: packagemanager > Fix For: 8.5 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Today, the class is loaded at the core load time an a stong reference is > held. Package loading need it to be lazily loaded -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] cpoerschke commented on a change in pull request #1033: SOLR-13965: Use Plugin to add new expressions to GraphHandler
cpoerschke commented on a change in pull request #1033: SOLR-13965: Use Plugin
to add new expressions to GraphHandler
URL: https://github.com/apache/lucene-solr/pull/1033#discussion_r370262941
##
File path: solr/core/src/java/org/apache/solr/handler/GraphHandler.java
##
@@ -92,24 +104,29 @@ public void inform(SolrCore core) {
}
// This pulls all the overrides and additions from the config
+List pluginInfos =
core.getSolrConfig().getPluginInfos(Expressible.class.getName());
+
+// Check deprecated approach.
Object functionMappingsObj = initArgs.get("streamFunctions");
if(null != functionMappingsObj){
+ log.warn("solrconfig.xml: is deprecated for adding
additional streaming functions to GraphHandler.");
NamedList functionMappings = (NamedList)functionMappingsObj;
for(Entry functionMapping : functionMappings) {
String key = functionMapping.getKey();
PluginInfo pluginInfo = new PluginInfo(key,
Collections.singletonMap("class", functionMapping.getValue()));
-
-if (pluginInfo.pkgName == null) {
- Class clazz =
core.getResourceLoader().findClass((String) functionMapping.getValue(),
- Expressible.class);
- streamFactory.withFunctionName(key, clazz);
-} else {
- StreamHandler.ExpressibleHolder holder = new
StreamHandler.ExpressibleHolder(pluginInfo, core,
SolrConfig.classVsSolrPluginInfo.get(Expressible.class));
- streamFactory.withFunctionName(key, () -> holder.getClazz());
-}
-
+pluginInfos.add(pluginInfo);
}
+}
+for (PluginInfo pluginInfo : pluginInfos) {
+ if (pluginInfo.pkgName != null) {
+ExpressibleHolder holder = new ExpressibleHolder(pluginInfo, core,
SolrConfig.classVsSolrPluginInfo.get(Expressible.class));
Review comment:
> ... feel free to file a new JIRA ...
Opened https://issues.apache.org/jira/browse/SOLR-14212 for this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14210) Introduce Node-level status handler for replicas
[
https://issues.apache.org/jira/browse/SOLR-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022315#comment-17022315
]
Houston Putman commented on SOLR-14210:
---
I would love for this to be configurable like that, as there are definitely
different levels of "health" that users are interested in. But I'm not sure if
I understand how "the deep check assures that requests will likely succeed"
would work. Would it check the entire cluster status to make sure that at least
one indexable replica (TLOG/NRT) is "active" for each shard? Or would it just
check the replicas on that node?
Also I definitely agree that every health check endpoint should be excluded
from authentication. (Not ping though, as each ping request issues a query)
> Introduce Node-level status handler for replicas
>
>
> Key: SOLR-14210
> URL: https://issues.apache.org/jira/browse/SOLR-14210
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: master (9.0), 8.5
>Reporter: Houston Putman
>Priority: Major
>
> h2. Background
> As was brought up in SOLR-13055, in order to run Solr in a more cloud-native
> way, we need some additional features around node-level healthchecks.
> {quote}Like in Kubernetes we need 'liveliness' and 'readiness' probe
> explained in
> [https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/n]
> determine if a node is live and ready to serve live traffic.
> {quote}
>
> However there are issues around kubernetes managing it's own rolling
> restarts. With the current healthcheck setup, it's easy to envision a
> scenario in which Solr reports itself as "healthy" when all of its replicas
> are actually recovering. Therefore kubernetes, seeing a healthy pod would
> then go and restart the next Solr node. This can happen until all replicas
> are "recovering" and none are healthy. (maybe the last one restarted will be
> "down", but still there are no "active" replicas)
> h2. Proposal
> I propose we make an additional healthcheck handler that returns whether all
> replicas hosted by that Solr node are healthy and "active". That way we will
> be able to use the [default kubernetes rolling restart
> logic|https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies]
> with Solr.
> To add on to [Jan's point
> here|https://issues.apache.org/jira/browse/SOLR-13055?focusedCommentId=16716559&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16716559],
> this handler should be more friendly for other Content-Types and should use
> bettter HTTP response statuses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on issue #1121: SOLR-11207: Add OWASP dependency checker to gradle build
dweiss commented on issue #1121: SOLR-11207: Add OWASP dependency checker to gradle build URL: https://github.com/apache/lucene-solr/pull/1121#issuecomment-577800416 I think it should just fail if enabled. Then it'd be a single property: "validation.owasp=[true|false]? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1121: SOLR-11207: Add OWASP dependency checker to gradle build
dweiss commented on a change in pull request #1121: SOLR-11207: Add OWASP
dependency checker to gradle build
URL: https://github.com/apache/lucene-solr/pull/1121#discussion_r370273915
##
File path: gradle/validation/dependency-check.gradle
##
@@ -0,0 +1,14 @@
+// This adds OWASP vulnerability validation of project dependencies
+// Not part of 'check' task by default, must be called explicitly, e.g.
gradlew dependencyCheckAnalyze
+// Start build with -Pvalidation.owasp.fail=true to fail build on owasp errors
(CVSS >= 7)
+// Start build with -Pvalidation.owasp.skip=true to skip OWASP checks during
'check' phase
+
+dependencyCheck {
Review comment:
Does it automatically hook to all projects or is it configured for root
project only?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1121: SOLR-11207: Add OWASP dependency checker to gradle build
dweiss commented on a change in pull request #1121: SOLR-11207: Add OWASP
dependency checker to gradle build
URL: https://github.com/apache/lucene-solr/pull/1121#discussion_r370273536
##
File path: build.gradle
##
@@ -58,6 +59,7 @@ apply from:
file('gradle/validation/versions-props-sorted.gradle')
apply from: file('gradle/validation/validate-source-patterns.gradle')
apply from: file('gradle/validation/config-file-sanity.gradle')
apply from: file('gradle/validation/rat-sources.gradle')
+apply from: file('gradle/validation/dependency-check.gradle')
Review comment:
Perhaps change the name to "owasp-dependency-check" so that it's distinctly
different from other dependency checks (there are some -- checksums, licenses,
etc.)?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14192) Race condition between SchemaManager and ZkIndexSchemaReader
[
https://issues.apache.org/jira/browse/SOLR-14192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022357#comment-17022357
]
ASF subversion and git services commented on SOLR-14192:
Commit 434f90265b676211e06a4727327e775cf35c42bd in lucene-solr's branch
refs/heads/master from Andrzej Bialecki
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=434f902 ]
SOLR-14211: Fix a bug introduced in SOLR-14192.
> Race condition between SchemaManager and ZkIndexSchemaReader
>
>
> Key: SOLR-14192
> URL: https://issues.apache.org/jira/browse/SOLR-14192
> Project: Solr
> Issue Type: Bug
>Affects Versions: 8.4
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
> Fix For: 8.5
>
> Attachments: SOLR-14192.patch
>
>
> Spin-off from SOLR-14128 and SOLR-13368.
> In SolrCloud when a SolrCore is created and it uses managed schema then its
> {{ManagedIndexSchemaFactory}} performs an automatic upgrade of the initial
> {{schema.xml}} to {{managed-schema}}. This includes removing the original
> {{schema.xml}} file.
> SOLR-13368 added some locking to make sure the changed resource name (i.e.
> {{managed-schema}}) becomes visible only when this process is complete, and
> that in-flight requests to /admin/schema block until this process is
> complete, to avoid returning inconsistent data. This locking mechanism uses
> simple Object monitors.
> However, if there's more than 1 node in the cluster the subsequent request to
> retrieve schema may execute on a core that still hasn't reloaded its schema
> ({{ZkIndexSchemaReader}} uses a ZK watcher, which may take some time to
> trigger), and the resource name in that stale schema still points to
> {{schema.xml}}, which by this time no longer exists because it was removed by
> {{ManagedIndexSchemaFactory}} in the first core.
> As I see it there are two bugs here:
> # there's no distributed locking when this upgrade is performed, so it's
> natural that there are multiple cores racing against each other to perform
> this upgrade.
> # the upgrade process removes {{schema.xml}} too early - it triggers all
> other cores by creating the {{managed-schema}} file, and then other cores
> reload from the new managed schema - but it should wait until this reload is
> complete on all cores because only then it's safe to delete the non-managed
> resource as it's no longer in use by any core.
> Issue 1. can be solved by adding an ephemeral znode lock so that only one
> core can perform the upgrade. Issue 2. can be solved by using
> {{ManagedIndexSchema.waitForSchemaZkVersionAgreement}} after upgrade, and
> deleting {{schema.xml}} only after it's done.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-11554) Support handling OPTIONS request for Hadoop authentication filter
[ https://issues.apache.org/jira/browse/SOLR-11554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022367#comment-17022367 ] ASF subversion and git services commented on SOLR-11554: Commit b150202c852fc2c8a5312ede46684e7fff2f2a0b in lucene-solr's branch refs/heads/master from Kevin Risden [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=b150202c ] SOLR-11554: Support handling OPTIONS request for Hadoop authentication filter Signed-off-by: Kevin Risden > Support handling OPTIONS request for Hadoop authentication filter > - > > Key: SOLR-11554 > URL: https://issues.apache.org/jira/browse/SOLR-11554 > Project: Solr > Issue Type: Bug >Affects Versions: 6.4 >Reporter: Hrishikesh Gadre >Assignee: Kevin Risden >Priority: Minor > Fix For: 8.5 > > Attachments: SOLR-11554.patch > > > As part of SOLR-9513 we added a Solr authentication plugin which uses Hadoop > security framework. The HTTP client interface provided by Hadoop framework > does not send the authentication information preemptively. Instead it sends > an OPTIONS request first. If the server responds with 401 error, it resends > the request with the proper authentication information. This jira is to > handle the OPTIONS request as part of the Solr authentication plugin for > Hadoop. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] janhoy commented on a change in pull request #1121: SOLR-11207: Add OWASP dependency checker to gradle build
janhoy commented on a change in pull request #1121: SOLR-11207: Add OWASP
dependency checker to gradle build
URL: https://github.com/apache/lucene-solr/pull/1121#discussion_r370276649
##
File path: gradle/validation/dependency-check.gradle
##
@@ -0,0 +1,14 @@
+// This adds OWASP vulnerability validation of project dependencies
+// Not part of 'check' task by default, must be called explicitly, e.g.
gradlew dependencyCheckAnalyze
+// Start build with -Pvalidation.owasp.fail=true to fail build on owasp errors
(CVSS >= 7)
+// Start build with -Pvalidation.owasp.skip=true to skip OWASP checks during
'check' phase
+
+dependencyCheck {
Review comment:
I don’t know. Have not tested other than root.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1121: SOLR-11207: Add OWASP dependency checker to gradle build
dweiss commented on a change in pull request #1121: SOLR-11207: Add OWASP
dependency checker to gradle build
URL: https://github.com/apache/lucene-solr/pull/1121#discussion_r370278321
##
File path: gradle/validation/dependency-check.gradle
##
@@ -0,0 +1,14 @@
+// This adds OWASP vulnerability validation of project dependencies
+// Not part of 'check' task by default, must be called explicitly, e.g.
gradlew dependencyCheckAnalyze
+// Start build with -Pvalidation.owasp.fail=true to fail build on owasp errors
(CVSS >= 7)
+// Start build with -Pvalidation.owasp.skip=true to skip OWASP checks during
'check' phase
+
+dependencyCheck {
Review comment:
You can try to run gradlew -p lucene/core ... and see what happens. :)
If it's just applicable to the root project (which it may be) then I'd wrap
it in a configure(rootProject) { ... } closure so that it's explicit in the
code.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14193) Update tutorial.adoc(line no:664) so that command executes in windows enviroment
[
https://issues.apache.org/jira/browse/SOLR-14193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022383#comment-17022383
]
Cassandra Targett commented on SOLR-14193:
--
[~balaji-s], I'm a little confused what to do with this. I see there was also a
GitHub pull request [1], but you closed it without explanation shortly after
creating it. Is that not the way to solve the problem? Should this Jira issue
also be closed?
[1] https://github.com/apache/lucene-solr/pull/1180
> Update tutorial.adoc(line no:664) so that command executes in windows
> enviroment
>
>
> Key: SOLR-14193
> URL: https://issues.apache.org/jira/browse/SOLR-14193
> Project: Solr
> Issue Type: Bug
> Components: documentation
>Affects Versions: 8.4
>Reporter: balaji sundaram
>Priority: Minor
>
>
> {{When executing the following command in windows 10 "java -jar -Dc=films
> -Dparams=f.genre.split=true&f.directed_by.split=true&f.genre.separator=|&f.directed_by.separator=|
> -Dauto example\exampledocs\post.jar example\films\*.csv", it throws error "&
> was unexpected at this time."}}
> Fix: the command should escape "&" and "|" symbol{{}}
> {{}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on issue #1184: LUCENE-9142 Refactor IntSet operations for determinize
madrob commented on issue #1184: LUCENE-9142 Refactor IntSet operations for determinize URL: https://github.com/apache/lucene-solr/pull/1184#issuecomment-577828200 Started over on this, tried to be less ambitious but still improving the situation I think. Unclear why precommit failed according to GitHub, when I click through it looks like it succeeds? @mikemccand @dweiss @uschindler can you take another look? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-11554) Support handling OPTIONS request for Hadoop authentication filter
[ https://issues.apache.org/jira/browse/SOLR-11554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden resolved SOLR-11554. - Resolution: Fixed Thanks [~gezapeti] > Support handling OPTIONS request for Hadoop authentication filter > - > > Key: SOLR-11554 > URL: https://issues.apache.org/jira/browse/SOLR-11554 > Project: Solr > Issue Type: Bug >Affects Versions: 6.4 >Reporter: Hrishikesh Gadre >Assignee: Kevin Risden >Priority: Minor > Fix For: 8.5 > > Attachments: SOLR-11554.patch > > > As part of SOLR-9513 we added a Solr authentication plugin which uses Hadoop > security framework. The HTTP client interface provided by Hadoop framework > does not send the authentication information preemptively. Instead it sends > an OPTIONS request first. If the server responds with 401 error, it resends > the request with the proper authentication information. This jira is to > handle the OPTIONS request as part of the Solr authentication plugin for > Hadoop. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-11554) Support handling OPTIONS request for Hadoop authentication filter
[ https://issues.apache.org/jira/browse/SOLR-11554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022368#comment-17022368 ] ASF subversion and git services commented on SOLR-11554: Commit 89ded7060d2685a66ee25e218dbc8eb82f1e7380 in lucene-solr's branch refs/heads/branch_8x from Kevin Risden [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=89ded70 ] SOLR-11554: Support handling OPTIONS request for Hadoop authentication filter Signed-off-by: Kevin Risden > Support handling OPTIONS request for Hadoop authentication filter > - > > Key: SOLR-11554 > URL: https://issues.apache.org/jira/browse/SOLR-11554 > Project: Solr > Issue Type: Bug >Affects Versions: 6.4 >Reporter: Hrishikesh Gadre >Assignee: Kevin Risden >Priority: Minor > Fix For: 8.5 > > Attachments: SOLR-11554.patch > > > As part of SOLR-9513 we added a Solr authentication plugin which uses Hadoop > security framework. The HTTP client interface provided by Hadoop framework > does not send the authentication information preemptively. Instead it sends > an OPTIONS request first. If the server responds with 401 error, it resends > the request with the proper authentication information. This jira is to > handle the OPTIONS request as part of the Solr authentication plugin for > Hadoop. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob edited a comment on issue #1184: LUCENE-9142 Refactor IntSet operations for determinize
madrob edited a comment on issue #1184: LUCENE-9142 Refactor IntSet operations for determinize URL: https://github.com/apache/lucene-solr/pull/1184#issuecomment-577828200 Started over on this, tried to be less ambitious but still improving the situation I think. ~Unclear why precommit failed according to GitHub, when I click through it looks like it succeeds?~ Needed to refresh for this. @mikemccand @dweiss @uschindler can you take another look? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14211) TestBulkSchemaConcurrent failures
[ https://issues.apache.org/jira/browse/SOLR-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022356#comment-17022356 ] ASF subversion and git services commented on SOLR-14211: Commit 434f90265b676211e06a4727327e775cf35c42bd in lucene-solr's branch refs/heads/master from Andrzej Bialecki [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=434f902 ] SOLR-14211: Fix a bug introduced in SOLR-14192. > TestBulkSchemaConcurrent failures > - > > Key: SOLR-14211 > URL: https://issues.apache.org/jira/browse/SOLR-14211 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 8.5 >Reporter: Andrzej Bialecki >Assignee: Andrzej Bialecki >Priority: Major > Fix For: 8.5 > > > This bug is caused by a recent change in SOLR-14192 - SchemaManager tries to > locate in ZK a schema resource name without prepending the full configset > path. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9160) override heap / jvm params for tests in gradle build
[
https://issues.apache.org/jira/browse/LUCENE-9160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022347#comment-17022347
]
Dawid Weiss commented on LUCENE-9160:
-
I never had a chance to experiment on those super-beefy machines but I'm sure
we can alter the defaults.
{code}
// Approximate a common-sense default for running gradle with parallel
// workers: half the count of available cpus but not more than 12.
def cpus = Runtime.runtime.availableProcessors()
def maxWorkers = (int) Math.max(1d, Math.min(cpus * 0.5d, 12))
def testsJvms = (int) Math.max(1d, Math.min(cpus * 0.5d, 4))
{code}
My machines quickly saturate I/O and memory bandwidth for higher test
parallelism, especially for Solr. The above is just off-the-top-off-my-head
default. It can be certainly improved.
> override heap / jvm params for tests in gradle build
>
>
> Key: LUCENE-9160
> URL: https://issues.apache.org/jira/browse/LUCENE-9160
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Robert Muir
>Priority: Major
> Fix For: master (9.0)
>
> Attachments: LUCENE-9160.patch, LUCENE-9160.patch
>
>
> Currently the gradle.properties that is generated lets you control the heap
> and flags for the gradle build jvms.
> But there is no way to control these flags for the actual forked VMs running
> the unit tests. For example, minHeap is hardcoded at 256m and maxHeap at
> 512m.
> I would like to change minHeap to 512m as well, for a fixed heap, and set
> some other jvm flags, such as {{-XX:+UseParallelGC}} so that my tests are not
> slow for silly reasons :)
> I think it is stuff jenkins CI would need as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9160) override heap / jvm params for tests in gradle build
[
https://issues.apache.org/jira/browse/LUCENE-9160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022352#comment-17022352
]
Robert Muir commented on LUCENE-9160:
-
Dawid I opened LUCENE-9165 to discuss further.
Also keep in mindthis jira ticket alters the defaults in ways that impact this.
For example, when running lucene tests with 3 VMs I see load average around 4.0
instead of 15.0-16.0 before this very patch was committed!
That's because I don't have 3 CICompiler threads *per* JVM doing a lot of
useless C2 recompilation. So it makes things more efficient and I think we
should raise the hard cap of 4 jvms to 8 or 12.
> override heap / jvm params for tests in gradle build
>
>
> Key: LUCENE-9160
> URL: https://issues.apache.org/jira/browse/LUCENE-9160
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Robert Muir
>Priority: Major
> Fix For: master (9.0)
>
> Attachments: LUCENE-9160.patch, LUCENE-9160.patch
>
>
> Currently the gradle.properties that is generated lets you control the heap
> and flags for the gradle build jvms.
> But there is no way to control these flags for the actual forked VMs running
> the unit tests. For example, minHeap is hardcoded at 256m and maxHeap at
> 512m.
> I would like to change minHeap to 512m as well, for a fixed heap, and set
> some other jvm flags, such as {{-XX:+UseParallelGC}} so that my tests are not
> slow for silly reasons :)
> I think it is stuff jenkins CI would need as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14213) Configuring Solr Cloud to use Shared Storage
Andy Vuong created SOLR-14213: - Summary: Configuring Solr Cloud to use Shared Storage Key: SOLR-14213 URL: https://issues.apache.org/jira/browse/SOLR-14213 Project: Solr Issue Type: Sub-task Components: SolrCloud Reporter: Andy Vuong Clients can currently create shared collections by sending a collection admin command such as *_solr/admin/collections?action=CREATE&name=gettingstarted&sharedIndex=true&numShards=1_* There are a set of shared storage specific classes such as SharedStorageManager that get initialized on startup when the CoreContainer loads. There are also components that are lazily loaded when shared storage functionality is needed. This was initially written this way because a Solr Cloud cluster could spin up and not used shared collections in which case shared store components wouldn’t need to be loaded. There is also no support for configuring Solr Cloud to use shared storage via config files. Lazy loading leads to some poor code and initialization flow that should be revisited. This JIRA is for designing the configuration of Solr Cloud to use shared storage and initializing shared storage components based on this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1171: SOLR-13892: Add 'top-level' docValues Join implementation
dsmiley commented on a change in pull request #1171: SOLR-13892: Add
'top-level' docValues Join implementation
URL: https://github.com/apache/lucene-solr/pull/1171#discussion_r370282592
##
File path: solr/core/src/java/org/apache/solr/search/JoinQParserPlugin.java
##
@@ -59,67 +60,124 @@
import org.apache.solr.search.join.ScoreJoinQParserPlugin;
import org.apache.solr.util.RTimer;
import org.apache.solr.util.RefCounted;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
public class JoinQParserPlugin extends QParserPlugin {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
public static final String NAME = "join";
+ /** Choose the internal algorithm */
+ private static final String METHOD = "method";
+
+ private static class JoinParams {
+final String fromField;
+final String fromCore;
+final Query fromQuery;
+final long fromCoreOpenTime;
+final String toField;
+
+public JoinParams(String fromField, String fromCore, Query fromQuery, long
fromCoreOpenTime, String toField) {
+ this.fromField = fromField;
+ this.fromCore = fromCore;
+ this.fromQuery = fromQuery;
+ this.fromCoreOpenTime = fromCoreOpenTime;
+ this.toField = toField;
+}
+ }
+
+ private enum Method {
+index {
+ @Override
+ Query makeFilter(QParser qparser) throws SyntaxError {
+final JoinParams jParams = parseJoin(qparser);
+final JoinQuery q = new JoinQuery(jParams.fromField, jParams.toField,
jParams.fromCore, jParams.fromQuery);
+q.fromCoreOpenTime = jParams.fromCoreOpenTime;
+return q;
+ }
+},
+dvWithScore {
+ @Override
+ Query makeFilter(QParser qparser) throws SyntaxError {
+return new ScoreJoinQParserPlugin().createParser(qparser.qstr,
qparser.localParams, qparser.params, qparser.req).parse();
+ }
+},
+topLevelDV {
+ @Override
+ Query makeFilter(QParser qparser) throws SyntaxError {
+final JoinParams jParams = parseJoin(qparser);
+final JoinQuery q = new TopLevelJoinQuery(jParams.fromField,
jParams.toField, jParams.fromCore, jParams.fromQuery);
+q.fromCoreOpenTime = jParams.fromCoreOpenTime;
+return q;
+ }
+};
+
+abstract Query makeFilter(QParser qparser) throws SyntaxError;
+
+JoinParams parseJoin(QParser qparser) throws SyntaxError {
+ final String fromField = qparser.getParam("from");
+ final String fromIndex = qparser.getParam("fromIndex");
+ final String toField = qparser.getParam("to");
+ final String v = qparser.localParams.get("v");
+ final String coreName;
+
+ Query fromQuery;
+ long fromCoreOpenTime = 0;
+
+ if (fromIndex != null &&
!fromIndex.equals(qparser.req.getCore().getCoreDescriptor().getName()) ) {
+CoreContainer container = qparser.req.getCore().getCoreContainer();
+
+// if in SolrCloud mode, fromIndex should be the name of a
single-sharded collection
+coreName = ScoreJoinQParserPlugin.getCoreName(fromIndex, container);
+
+final SolrCore fromCore = container.getCore(coreName);
+if (fromCore == null) {
+ throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
+ "Cross-core join: no such core " + coreName);
+}
+
+RefCounted fromHolder = null;
+LocalSolrQueryRequest otherReq = new LocalSolrQueryRequest(fromCore,
qparser.params);
+try {
Review comment:
Can you use try-with-resources style for SolrCore and otherReq? You might
need to create a static method that gets the core and throws if non-null so
that you are able to have a one-liner fetch of the core in the "try". I
generally like to use try-with-resources when possible as it's easier to reason
to guarantee things get closed, and it's sometimes less code.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1171: SOLR-13892: Add 'top-level' docValues Join implementation
dsmiley commented on a change in pull request #1171: SOLR-13892: Add
'top-level' docValues Join implementation
URL: https://github.com/apache/lucene-solr/pull/1171#discussion_r370319920
##
File path: solr/solr-ref-guide/src/other-parsers.adoc
##
@@ -591,36 +591,95 @@ The hash range query parser uses a special cache to
improve the speedup of the q
== Join Query Parser
-`JoinQParser` extends the `QParserPlugin`. It allows normalizing relationships
between documents with a join operation. This is different from the concept of
a join in a relational database because no information is being truly joined.
An appropriate SQL analogy would be an "inner query".
+The Join query parser allows users to run queries that normalize relationships
between documents.
+Solr runs a subquery of the user's choosing (the `v` param), identifies all
the values that matching documents have in a field of interest (the `from`
param), and then returns documents where those values are contained in a second
field of interest (the `to` param).
-Examples:
-
-Find all products containing the word "ipod", join them against manufacturer
docs and return the list of manufacturers:
+In practice, these semantics are much like "inner queries" in a SQL engine.
+As an example, consider the Solr query below:
[source,text]
-{!join from=manu_id_s to=id}ipod
+/solr/techproducts/select?q={!join from=manu_id_s to=id}title:ipod
-Find all manufacturer docs named "belkin", join them against product docs, and
filter the list to only products with a price less than $12:
+This query, which returns a document for each manufacturer that makes a
product with "ipod" in the title, is semantically identical to the SQL query
below:
[source,text]
-q = {!join from=id to=manu_id_s}compName_s:Belkin
-fq = price:[* TO 12]
+SELECT *
+FROM techproducts
+WHERE id IN (
+SELECT manu_id_s
+FROM techproducts
+WHERE title='ipod'
+ )
-The join operation is done on a term basis, so the "from" and "to" fields must
use compatible field types. For example: joining between a `StrField` and a
`IntPointField` will not work, likewise joining between a `StrField` and a
`TextField` that uses `LowerCaseFilterFactory` will only work for values that
are already lower cased in the string field.
+The join operation is done on a term basis, so the `from` and `to` fields must
use compatible field types.
+For example: joining between a `StrField` and a `IntPointField` will not work.
+Likewise joining between a `StrField` and a `TextField` that uses
`LowerCaseFilterFactory` will only work for values that are already lower cased
in the string field.
+
+=== Parameters
+
+This query parser takes the following parameters:
+
+`from`::
+The name of a field which contains values to look for in the "to" field.
+Can be single or multi-valued, but must have a field type compatible with the
field represented in the "to" field.
+This parameter is required.
+
+`to`::
+The name of a field whose value(s) will be checked against those found in the
"from" field.
+Can be single or multi-valued, but must have a field type compatible with the
"from" field.
+This parameter is required.
+
+`fromIndex`::
+The name of the index to run the "from" query (`v` parameter) on and where
"from" values are gathered.
+Must be located on the same node as the core processing the request.
+This parameter is optional; it defaults to the value of the processing core if
not specified.
+See <> below for more
information.
+
+`score`::
+An optional parameter that instructs Solr to return information about the
"from" query scores.
+The value of this parameter controls what type of aggregation information is
returned.
+Options include `avg` (average), `max` (maximum), `min` (minimum), `total`
(total), or `none`.
++
+If `method` is not specified but `score` is, then the `dvWithScore` method is
used.
+If `method` is specified and is not `dvWithScore`, then the `score` value is
ignored.
+See the `method` parameter documentation below for more details.
-=== Join Parser Scoring
-You can optionally use the `score` parameter to return scores of the
subordinate query. The values to use for this parameter define the type of
aggregation, which are `avg` (average), `max` (maximum), `min` (minimum)
`total`, or `none`.
+`method`::
+An optional parameter used to determine which of several query implementations
should be used by Solr.
+Options are restricted to: `index`, `dvWithScore`, and `topLevelDV`.
+If unspecified the default value is `index`, unless the `score` parameter is
present which overrides it to `dvWithScore`.
+Each implementation has its own performance characteristics, and users are
encouraged to experiment to determine which implementation is most performant
for their use-case.
+Details and performance heuristics are given below.
++
+`index` the default `method` unless the `score` parameter is specified.
+Uses inverted in
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1171: SOLR-13892: Add 'top-level' docValues Join implementation
dsmiley commented on a change in pull request #1171: SOLR-13892: Add
'top-level' docValues Join implementation
URL: https://github.com/apache/lucene-solr/pull/1171#discussion_r370309875
##
File path:
solr/core/src/java/org/apache/solr/search/join/MultiValueTermOrdinalCollector.java
##
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search.join;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.index.SortedSetDocValues;
+import org.apache.lucene.search.ScoreMode;
+import org.apache.lucene.util.LongBitSet;
+import org.apache.solr.search.DelegatingCollector;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Populates a bitset of (top-level) ordinals based on field values in a
multi-valued field.
+ */
+public class MultiValueTermOrdinalCollector extends DelegatingCollector {
Review comment:
Why extend DelegatingCollector?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1171: SOLR-13892: Add 'top-level' docValues Join implementation
dsmiley commented on a change in pull request #1171: SOLR-13892: Add
'top-level' docValues Join implementation
URL: https://github.com/apache/lucene-solr/pull/1171#discussion_r370307365
##
File path: solr/core/src/java/org/apache/solr/search/TopLevelJoinQuery.java
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+
+import org.apache.lucene.index.DocValues;
+import org.apache.lucene.index.LeafReader;
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.index.SortedSetDocValues;
+import org.apache.lucene.search.Collector;
+import org.apache.lucene.search.ConstantScoreScorer;
+import org.apache.lucene.search.ConstantScoreWeight;
+import org.apache.lucene.search.DocIdSetIterator;
+import org.apache.lucene.search.IndexSearcher;
+import org.apache.lucene.search.Query;
+import org.apache.lucene.search.ScoreMode;
+import org.apache.lucene.search.Scorer;
+import org.apache.lucene.search.TwoPhaseIterator;
+import org.apache.lucene.search.Weight;
+import org.apache.lucene.util.BytesRef;
+import org.apache.lucene.util.LongBitSet;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.schema.IndexSchema;
+import org.apache.solr.schema.SchemaField;
+import org.apache.solr.search.join.MultiValueTermOrdinalCollector;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class TopLevelJoinQuery extends JoinQuery {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+ public TopLevelJoinQuery(String fromField, String toField, String coreName,
Query subQuery) {
+super(fromField, toField, coreName, subQuery);
+ }
+
+ @Override
+ public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode,
float boost) throws IOException {
+if (! (searcher instanceof SolrIndexSearcher)) {
+ log.debug("Falling back to JoinQueryWeight because searcher [{}] is not
the required SolrIndexSearcher", searcher);
+ return super.createWeight(searcher, scoreMode, boost);
+}
+
+final SolrIndexSearcher solrSearcher = (SolrIndexSearcher) searcher;
+final JoinQueryWeight weight = new JoinQueryWeight(solrSearcher,
ScoreMode.COMPLETE_NO_SCORES, 1.0f);
+final SolrIndexSearcher fromSearcher = weight.fromSearcher;
+final SolrIndexSearcher toSearcher = weight.toSearcher;
+
+try {
+ final SortedSetDocValues topLevelFromDocValues =
validateAndFetchDocValues(fromSearcher, fromField, "from");
+ final SortedSetDocValues topLevelToDocValues =
validateAndFetchDocValues(toSearcher, toField, "to");
+ if (topLevelFromDocValues.getValueCount() == 0 ||
topLevelToDocValues.getValueCount() == 0) {
+return createNoMatchesWeight(boost);
+ }
+
+ final LongBitSet fromOrdBitSet =
findOrdinalsMatchingFromQuery(fromSearcher, topLevelFromDocValues);
+ final LongBitSet toOrdBitSet = new
LongBitSet(topLevelToDocValues.getValueCount());
+ final BitsetBounds toBitsetBounds =
convertFromOrdinalsIntoToField(fromOrdBitSet, topLevelFromDocValues,
toOrdBitSet, topLevelToDocValues);
+
+ final boolean toMultivalued =
toSearcher.getSchema().getFieldOrNull(toField).multiValued();
+ return new ConstantScoreWeight(this, boost) {
+public Scorer scorer(LeafReaderContext context) throws IOException {
+ if (toBitsetBounds.lower == BitsetBounds.NO_MATCHES) {
+return null;
+ }
+
+ final DocIdSetIterator toApproximation = (toMultivalued) ?
context.reader().getSortedSetDocValues(toField) :
+ context.reader().getSortedDocValues(toField);
+ if (toApproximation == null) {
+return null;
+ }
+
+ final int docBase = context.docBase;
+ return new ConstantScoreScorer(this, this.score(), scoreMode, new
TwoPhaseIterator(toApproximation) {
+public boolean matches() throws IOException {
+ final boolean hasDoc = topLevelToDocValues.advanceExact(docBase
+ approximation.docID());
+ if (hasDoc) {
+for (long ord = topLevelToDocValues.ne
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1171: SOLR-13892: Add 'top-level' docValues Join implementation
dsmiley commented on a change in pull request #1171: SOLR-13892: Add
'top-level' docValues Join implementation
URL: https://github.com/apache/lucene-solr/pull/1171#discussion_r370313149
##
File path:
solr/core/src/java/org/apache/solr/search/join/MultiValueTermOrdinalCollector.java
##
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search.join;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.index.SortedSetDocValues;
+import org.apache.lucene.search.ScoreMode;
+import org.apache.lucene.util.LongBitSet;
+import org.apache.solr.search.DelegatingCollector;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Populates a bitset of (top-level) ordinals based on field values in a
multi-valued field.
+ */
+public class MultiValueTermOrdinalCollector extends DelegatingCollector {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+ private int docBase;
+ private SortedSetDocValues topLevelDocValues;
+ private final String fieldName;
+ private final LongBitSet topLevelDocValuesBitSet;
Review comment:
document this is the outgoing data structure expressing the results. Could
be a trivial comment
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1171: SOLR-13892: Add 'top-level' docValues Join implementation
dsmiley commented on a change in pull request #1171: SOLR-13892: Add
'top-level' docValues Join implementation
URL: https://github.com/apache/lucene-solr/pull/1171#discussion_r370304007
##
File path: solr/core/src/java/org/apache/solr/search/JoinQParserPlugin.java
##
@@ -59,67 +60,124 @@
import org.apache.solr.search.join.ScoreJoinQParserPlugin;
import org.apache.solr.util.RTimer;
import org.apache.solr.util.RefCounted;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
public class JoinQParserPlugin extends QParserPlugin {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
public static final String NAME = "join";
+ /** Choose the internal algorithm */
+ private static final String METHOD = "method";
+
+ private static class JoinParams {
+final String fromField;
+final String fromCore;
+final Query fromQuery;
+final long fromCoreOpenTime;
+final String toField;
+
+public JoinParams(String fromField, String fromCore, Query fromQuery, long
fromCoreOpenTime, String toField) {
+ this.fromField = fromField;
+ this.fromCore = fromCore;
+ this.fromQuery = fromQuery;
+ this.fromCoreOpenTime = fromCoreOpenTime;
+ this.toField = toField;
+}
+ }
+
+ private enum Method {
+index {
+ @Override
+ Query makeFilter(QParser qparser) throws SyntaxError {
+final JoinParams jParams = parseJoin(qparser);
+final JoinQuery q = new JoinQuery(jParams.fromField, jParams.toField,
jParams.fromCore, jParams.fromQuery);
+q.fromCoreOpenTime = jParams.fromCoreOpenTime;
+return q;
+ }
+},
+dvWithScore {
+ @Override
+ Query makeFilter(QParser qparser) throws SyntaxError {
+return new ScoreJoinQParserPlugin().createParser(qparser.qstr,
qparser.localParams, qparser.params, qparser.req).parse();
+ }
+},
+topLevelDV {
+ @Override
+ Query makeFilter(QParser qparser) throws SyntaxError {
+final JoinParams jParams = parseJoin(qparser);
+final JoinQuery q = new TopLevelJoinQuery(jParams.fromField,
jParams.toField, jParams.fromCore, jParams.fromQuery);
+q.fromCoreOpenTime = jParams.fromCoreOpenTime;
+return q;
+ }
+};
+
+abstract Query makeFilter(QParser qparser) throws SyntaxError;
+
+JoinParams parseJoin(QParser qparser) throws SyntaxError {
+ final String fromField = qparser.getParam("from");
+ final String fromIndex = qparser.getParam("fromIndex");
+ final String toField = qparser.getParam("to");
+ final String v = qparser.localParams.get("v");
+ final String coreName;
+
+ Query fromQuery;
+ long fromCoreOpenTime = 0;
+
+ if (fromIndex != null &&
!fromIndex.equals(qparser.req.getCore().getCoreDescriptor().getName()) ) {
+CoreContainer container = qparser.req.getCore().getCoreContainer();
+
+// if in SolrCloud mode, fromIndex should be the name of a
single-sharded collection
+coreName = ScoreJoinQParserPlugin.getCoreName(fromIndex, container);
+
+final SolrCore fromCore = container.getCore(coreName);
+if (fromCore == null) {
+ throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
+ "Cross-core join: no such core " + coreName);
+}
+
+RefCounted fromHolder = null;
+LocalSolrQueryRequest otherReq = new LocalSolrQueryRequest(fromCore,
qparser.params);
+try {
+ QParser parser = QParser.getParser(v, otherReq);
+ fromQuery = parser.getQuery();
+ fromHolder = fromCore.getRegisteredSearcher();
+ if (fromHolder != null) fromCoreOpenTime =
fromHolder.get().getOpenNanoTime();
+} finally {
+ otherReq.close();
+ fromCore.close();
+ if (fromHolder != null) fromHolder.decref();
+}
+ } else {
+coreName = null;
+QParser fromQueryParser = qparser.subQuery(v, null);
+fromQueryParser.setIsFilter(true);
Review comment:
Perhaps these two lines, setting is filter on the QParser and calling
getQuery should be pulled out of the block as it is (or should be) in common
with the positive side of the if-else?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1171: SOLR-13892: Add 'top-level' docValues Join implementation
dsmiley commented on a change in pull request #1171: SOLR-13892: Add
'top-level' docValues Join implementation
URL: https://github.com/apache/lucene-solr/pull/1171#discussion_r370319600
##
File path: solr/solr-ref-guide/src/other-parsers.adoc
##
@@ -591,36 +591,95 @@ The hash range query parser uses a special cache to
improve the speedup of the q
== Join Query Parser
-`JoinQParser` extends the `QParserPlugin`. It allows normalizing relationships
between documents with a join operation. This is different from the concept of
a join in a relational database because no information is being truly joined.
An appropriate SQL analogy would be an "inner query".
+The Join query parser allows users to run queries that normalize relationships
between documents.
+Solr runs a subquery of the user's choosing (the `v` param), identifies all
the values that matching documents have in a field of interest (the `from`
param), and then returns documents where those values are contained in a second
field of interest (the `to` param).
-Examples:
-
-Find all products containing the word "ipod", join them against manufacturer
docs and return the list of manufacturers:
+In practice, these semantics are much like "inner queries" in a SQL engine.
+As an example, consider the Solr query below:
[source,text]
-{!join from=manu_id_s to=id}ipod
+/solr/techproducts/select?q={!join from=manu_id_s to=id}title:ipod
-Find all manufacturer docs named "belkin", join them against product docs, and
filter the list to only products with a price less than $12:
+This query, which returns a document for each manufacturer that makes a
product with "ipod" in the title, is semantically identical to the SQL query
below:
[source,text]
-q = {!join from=id to=manu_id_s}compName_s:Belkin
-fq = price:[* TO 12]
+SELECT *
+FROM techproducts
+WHERE id IN (
+SELECT manu_id_s
+FROM techproducts
+WHERE title='ipod'
+ )
-The join operation is done on a term basis, so the "from" and "to" fields must
use compatible field types. For example: joining between a `StrField` and a
`IntPointField` will not work, likewise joining between a `StrField` and a
`TextField` that uses `LowerCaseFilterFactory` will only work for values that
are already lower cased in the string field.
+The join operation is done on a term basis, so the `from` and `to` fields must
use compatible field types.
+For example: joining between a `StrField` and a `IntPointField` will not work.
+Likewise joining between a `StrField` and a `TextField` that uses
`LowerCaseFilterFactory` will only work for values that are already lower cased
in the string field.
+
+=== Parameters
+
+This query parser takes the following parameters:
+
+`from`::
+The name of a field which contains values to look for in the "to" field.
+Can be single or multi-valued, but must have a field type compatible with the
field represented in the "to" field.
+This parameter is required.
+
+`to`::
+The name of a field whose value(s) will be checked against those found in the
"from" field.
+Can be single or multi-valued, but must have a field type compatible with the
"from" field.
+This parameter is required.
+
+`fromIndex`::
+The name of the index to run the "from" query (`v` parameter) on and where
"from" values are gathered.
+Must be located on the same node as the core processing the request.
+This parameter is optional; it defaults to the value of the processing core if
not specified.
+See <> below for more
information.
+
+`score`::
+An optional parameter that instructs Solr to return information about the
"from" query scores.
+The value of this parameter controls what type of aggregation information is
returned.
+Options include `avg` (average), `max` (maximum), `min` (minimum), `total`
(total), or `none`.
++
+If `method` is not specified but `score` is, then the `dvWithScore` method is
used.
+If `method` is specified and is not `dvWithScore`, then the `score` value is
ignored.
+See the `method` parameter documentation below for more details.
-=== Join Parser Scoring
-You can optionally use the `score` parameter to return scores of the
subordinate query. The values to use for this parameter define the type of
aggregation, which are `avg` (average), `max` (maximum), `min` (minimum)
`total`, or `none`.
+`method`::
+An optional parameter used to determine which of several query implementations
should be used by Solr.
+Options are restricted to: `index`, `dvWithScore`, and `topLevelDV`.
+If unspecified the default value is `index`, unless the `score` parameter is
present which overrides it to `dvWithScore`.
+Each implementation has its own performance characteristics, and users are
encouraged to experiment to determine which implementation is most performant
for their use-case.
+Details and performance heuristics are given below.
++
+`index` the default `method` unless the `score` parameter is specified.
+Uses inverted in
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1171: SOLR-13892: Add 'top-level' docValues Join implementation
dsmiley commented on a change in pull request #1171: SOLR-13892: Add
'top-level' docValues Join implementation
URL: https://github.com/apache/lucene-solr/pull/1171#discussion_r370311943
##
File path:
solr/core/src/java/org/apache/solr/search/join/MultiValueTermOrdinalCollector.java
##
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search.join;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.index.SortedSetDocValues;
+import org.apache.lucene.search.ScoreMode;
+import org.apache.lucene.util.LongBitSet;
+import org.apache.solr.search.DelegatingCollector;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Populates a bitset of (top-level) ordinals based on field values in a
multi-valued field.
+ */
+public class MultiValueTermOrdinalCollector extends DelegatingCollector {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
Review comment:
Nothing to log
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1171: SOLR-13892: Add 'top-level' docValues Join implementation
dsmiley commented on a change in pull request #1171: SOLR-13892: Add
'top-level' docValues Join implementation
URL: https://github.com/apache/lucene-solr/pull/1171#discussion_r370318149
##
File path:
solr/core/src/java/org/apache/solr/search/join/TopLevelDVTermsCollector.java
##
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search.join;
+
+import java.io.IOException;
+
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.index.SortedSetDocValues;
+import org.apache.lucene.search.LeafCollector;
+import org.apache.lucene.search.Scorable;
+import org.apache.lucene.util.LongBitSet;
+import org.apache.solr.search.DelegatingCollector;
+
+/**
+ * Collects all documents with a field value matching a set value in an
ordinal bitset.
+ *
+ * Implementation is similar to {@link
org.apache.lucene.search.join.TermsCollector}, but uses top-level ordinals
+ * explicitly and has wider visibility.
+ */
+public class TopLevelDVTermsCollector extends DelegatingCollector {
Review comment:
This class seems unused; no?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1171: SOLR-13892: Add 'top-level' docValues Join implementation
dsmiley commented on a change in pull request #1171: SOLR-13892: Add
'top-level' docValues Join implementation
URL: https://github.com/apache/lucene-solr/pull/1171#discussion_r370304976
##
File path: solr/core/src/java/org/apache/solr/search/JoinQParserPlugin.java
##
@@ -139,11 +197,16 @@ public static Query createJoinQuery(Query subQuery,
String fromField, String toF
class JoinQuery extends Query {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
String fromField;
String toField;
String fromIndex; // TODO: name is missleading here compared to
JoinQParserPlugin usage - here it must be a core name
Query q;
long fromCoreOpenTime;
+ private boolean cache;
Review comment:
Not used?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1171: SOLR-13892: Add 'top-level' docValues Join implementation
dsmiley commented on a change in pull request #1171: SOLR-13892: Add
'top-level' docValues Join implementation
URL: https://github.com/apache/lucene-solr/pull/1171#discussion_r370281149
##
File path: solr/core/src/java/org/apache/solr/search/JoinQParserPlugin.java
##
@@ -59,67 +60,124 @@
import org.apache.solr.search.join.ScoreJoinQParserPlugin;
import org.apache.solr.util.RTimer;
import org.apache.solr.util.RefCounted;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
public class JoinQParserPlugin extends QParserPlugin {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
public static final String NAME = "join";
+ /** Choose the internal algorithm */
+ private static final String METHOD = "method";
+
+ private static class JoinParams {
+final String fromField;
+final String fromCore;
+final Query fromQuery;
+final long fromCoreOpenTime;
+final String toField;
+
+public JoinParams(String fromField, String fromCore, Query fromQuery, long
fromCoreOpenTime, String toField) {
+ this.fromField = fromField;
+ this.fromCore = fromCore;
+ this.fromQuery = fromQuery;
+ this.fromCoreOpenTime = fromCoreOpenTime;
+ this.toField = toField;
+}
+ }
+
+ private enum Method {
+index {
+ @Override
+ Query makeFilter(QParser qparser) throws SyntaxError {
+final JoinParams jParams = parseJoin(qparser);
+final JoinQuery q = new JoinQuery(jParams.fromField, jParams.toField,
jParams.fromCore, jParams.fromQuery);
+q.fromCoreOpenTime = jParams.fromCoreOpenTime;
+return q;
+ }
+},
+dvWithScore {
+ @Override
+ Query makeFilter(QParser qparser) throws SyntaxError {
+return new ScoreJoinQParserPlugin().createParser(qparser.qstr,
qparser.localParams, qparser.params, qparser.req).parse();
+ }
+},
+topLevelDV {
+ @Override
+ Query makeFilter(QParser qparser) throws SyntaxError {
+final JoinParams jParams = parseJoin(qparser);
+final JoinQuery q = new TopLevelJoinQuery(jParams.fromField,
jParams.toField, jParams.fromCore, jParams.fromQuery);
+q.fromCoreOpenTime = jParams.fromCoreOpenTime;
+return q;
+ }
+};
+
+abstract Query makeFilter(QParser qparser) throws SyntaxError;
+
+JoinParams parseJoin(QParser qparser) throws SyntaxError {
+ final String fromField = qparser.getParam("from");
+ final String fromIndex = qparser.getParam("fromIndex");
+ final String toField = qparser.getParam("to");
+ final String v = qparser.localParams.get("v");
Review comment:
Use `QueryParsing.V` thus also signals this isn't special
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1171: SOLR-13892: Add 'top-level' docValues Join implementation
dsmiley commented on a change in pull request #1171: SOLR-13892: Add
'top-level' docValues Join implementation
URL: https://github.com/apache/lucene-solr/pull/1171#discussion_r370320580
##
File path: solr/solr-ref-guide/src/other-parsers.adoc
##
@@ -591,36 +591,95 @@ The hash range query parser uses a special cache to
improve the speedup of the q
== Join Query Parser
-`JoinQParser` extends the `QParserPlugin`. It allows normalizing relationships
between documents with a join operation. This is different from the concept of
a join in a relational database because no information is being truly joined.
An appropriate SQL analogy would be an "inner query".
+The Join query parser allows users to run queries that normalize relationships
between documents.
+Solr runs a subquery of the user's choosing (the `v` param), identifies all
the values that matching documents have in a field of interest (the `from`
param), and then returns documents where those values are contained in a second
field of interest (the `to` param).
-Examples:
-
-Find all products containing the word "ipod", join them against manufacturer
docs and return the list of manufacturers:
+In practice, these semantics are much like "inner queries" in a SQL engine.
+As an example, consider the Solr query below:
[source,text]
-{!join from=manu_id_s to=id}ipod
+/solr/techproducts/select?q={!join from=manu_id_s to=id}title:ipod
-Find all manufacturer docs named "belkin", join them against product docs, and
filter the list to only products with a price less than $12:
+This query, which returns a document for each manufacturer that makes a
product with "ipod" in the title, is semantically identical to the SQL query
below:
[source,text]
-q = {!join from=id to=manu_id_s}compName_s:Belkin
-fq = price:[* TO 12]
+SELECT *
+FROM techproducts
+WHERE id IN (
+SELECT manu_id_s
+FROM techproducts
+WHERE title='ipod'
+ )
-The join operation is done on a term basis, so the "from" and "to" fields must
use compatible field types. For example: joining between a `StrField` and a
`IntPointField` will not work, likewise joining between a `StrField` and a
`TextField` that uses `LowerCaseFilterFactory` will only work for values that
are already lower cased in the string field.
+The join operation is done on a term basis, so the `from` and `to` fields must
use compatible field types.
+For example: joining between a `StrField` and a `IntPointField` will not work.
+Likewise joining between a `StrField` and a `TextField` that uses
`LowerCaseFilterFactory` will only work for values that are already lower cased
in the string field.
+
+=== Parameters
+
+This query parser takes the following parameters:
+
+`from`::
+The name of a field which contains values to look for in the "to" field.
+Can be single or multi-valued, but must have a field type compatible with the
field represented in the "to" field.
+This parameter is required.
+
+`to`::
+The name of a field whose value(s) will be checked against those found in the
"from" field.
+Can be single or multi-valued, but must have a field type compatible with the
"from" field.
+This parameter is required.
+
+`fromIndex`::
+The name of the index to run the "from" query (`v` parameter) on and where
"from" values are gathered.
+Must be located on the same node as the core processing the request.
+This parameter is optional; it defaults to the value of the processing core if
not specified.
+See <> below for more
information.
+
+`score`::
+An optional parameter that instructs Solr to return information about the
"from" query scores.
+The value of this parameter controls what type of aggregation information is
returned.
+Options include `avg` (average), `max` (maximum), `min` (minimum), `total`
(total), or `none`.
++
+If `method` is not specified but `score` is, then the `dvWithScore` method is
used.
+If `method` is specified and is not `dvWithScore`, then the `score` value is
ignored.
+See the `method` parameter documentation below for more details.
-=== Join Parser Scoring
-You can optionally use the `score` parameter to return scores of the
subordinate query. The values to use for this parameter define the type of
aggregation, which are `avg` (average), `max` (maximum), `min` (minimum)
`total`, or `none`.
+`method`::
+An optional parameter used to determine which of several query implementations
should be used by Solr.
+Options are restricted to: `index`, `dvWithScore`, and `topLevelDV`.
+If unspecified the default value is `index`, unless the `score` parameter is
present which overrides it to `dvWithScore`.
+Each implementation has its own performance characteristics, and users are
encouraged to experiment to determine which implementation is most performant
for their use-case.
+Details and performance heuristics are given below.
++
+`index` the default `method` unless the `score` parameter is specified.
+Uses inverted in
